From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A96E8A00C4; Mon, 25 Jul 2022 18:10:53 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 511AE410FA; Mon, 25 Jul 2022 18:10:53 +0200 (CEST) Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) by mails.dpdk.org (Postfix) with ESMTP id 1DFD140684 for ; Mon, 25 Jul 2022 18:10:52 +0200 (CEST) Received: by mail-pf1-f182.google.com with SMTP id d10so10843132pfd.9 for ; Mon, 25 Jul 2022 09:10:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20210112.gappssmtp.com; s=20210112; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2WV5htsgZ073ApKOXWSwQUaLMUUO/GB44nu8Pvx8kYg=; b=MP8pcICYYfjh2merDTl2f6hGG0psD0TBmOXTKpbrnoeS0eipf0WLGIt4OtzOEfNzNK Yq9OYJb6IyT5N1PCZEFUxCErl2Z1ZNP4i4rjWaPGcnHz8J2HiNjUroxvk2yCEiXLWIAp rl1yTl/B7QwvCYGQgcQH+aqrjKPBDRnmHbPXuqnsW6xY/CaEJyw3WFyozwYDLa0ldrAb 0t790Fz6ueytlToQ1dTUI1P30XE6jiUd1ZTReDJWm4FM3k3HrOFX6LCcWq2Vxu7yqmhu QIwFJd0e2WIZk779D/QrxIgmUTZ2LFvLgU8A4i2JZmEiGcOYxv8lb/S4AK8GCjnCU5V+ Dxcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2WV5htsgZ073ApKOXWSwQUaLMUUO/GB44nu8Pvx8kYg=; b=Qd8fhCrLZmM7C2eBRkbf+L/P+KXQBFUsEsk6ldX/NuZmY1ASw2U4Fw3p5xD+7aoRtc wYulfs8xSBCz/IfbXLHz44UzkzrxPI/27XTA/Bk6ScAq8+XL2cTQFOXp9nm11xZGc7IT XL75+aO1ZbJnm4jhVGj4+mkMa3S+vjr07ZO1Io2rRFj/Em/Z6/Ru4IzkKsDd9TaeuvyH nR48213WiJXFhT4ACAfkYFcjagxpWAgATAC9H1irVHfZewYbYGHRRb1pF70M0783AewD ZbP+JFpZmlf5FO08fbkqLlTcYKv+cqN9ORe064hAeP98FMMUhh45g0ukmRm3nDjPg5XO YaJQ== X-Gm-Message-State: AJIora82M6q9Ib4AHoyeLiWF71EO+H3GSA6DsDV4+mBVXgO8lJ5l1uvB ODHOcxAdlJjpEmjhWNb9J/Wl5w== X-Google-Smtp-Source: AGRyM1uCz9PecW5ng7oy5SI86w2bgudoJ9TLOSiLHkIFljbmgx/2L8bAzeYLv/VpBt9KLRwDPJ/M3w== X-Received: by 2002:a05:6a00:892:b0:52b:c986:c781 with SMTP id q18-20020a056a00089200b0052bc986c781mr13688483pfj.64.1658765451081; Mon, 25 Jul 2022 09:10:51 -0700 (PDT) Received: from hermes.local (204-195-120-218.wavecable.com. [204.195.120.218]) by smtp.gmail.com with ESMTPSA id 35-20020a631163000000b00419ab8f8d2csm8362112pgr.20.2022.07.25.09.10.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jul 2022 09:10:50 -0700 (PDT) Date: Mon, 25 Jul 2022 09:10:49 -0700 From: Stephen Hemminger To: =?UTF-8?B?TcOhcmlv?= Kuka Cc: Reshma Pattan , Ray Kinsella , dev@dpdk.org Subject: Re: [PATCH] pcapng: fix write more packets than IOV_MAX limit Message-ID: <20220725091049.5d9d013c@hermes.local> In-Reply-To: <20220725152811.409447-1-kuka@cesnet.cz> References: <20220725152811.409447-1-kuka@cesnet.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Mon, 25 Jul 2022 17:28:11 +0200 M=C3=A1rio Kuka wrote: > The rte_pcapng_write_packets() function fails when we try to write more > packets than the IOV_MAX limit. The error is caused by the writev() > system call, which is limited by the IOV_MAX limit. The iovcnt argument > is valid if it is greater than 0 and less than or equal to IOV_MAX as > defined in . >=20 > To avoid this problem, we can split the iovec buffer into smaller > chunks with a maximum size of IOV_MAX and write them sequentially by > calling the writev() repeatedly. >=20 > Fixes: 8d23ce8f5ee9 ("pcapng: add new library for writing pcapng files") > Cc: stephen@networkplumber.org >=20 > Signed-off-by: M=C3=A1rio Kuka > --- Something like this (compile tested only). diff --git a/lib/pcapng/rte_pcapng.c b/lib/pcapng/rte_pcapng.c index 06ad712bd1eb..e41cf909e120 100644 --- a/lib/pcapng/rte_pcapng.c +++ b/lib/pcapng/rte_pcapng.c @@ -551,33 +551,16 @@ rte_pcapng_copy(uint16_t port_id, uint32_t queue, return NULL; } =20 -/* Count how many segments are in this array of mbufs */ -static unsigned int -mbuf_burst_segs(struct rte_mbuf *pkts[], unsigned int n) -{ - unsigned int i, iovcnt; - - for (iovcnt =3D 0, i =3D 0; i < n; i++) { - const struct rte_mbuf *m =3D pkts[i]; - - __rte_mbuf_sanity_check(m, 1); - - iovcnt +=3D m->nb_segs; - } - return iovcnt; -} - /* Write pre-formatted packets to file. */ ssize_t rte_pcapng_write_packets(rte_pcapng_t *self, struct rte_mbuf *pkts[], uint16_t nb_pkts) { - int iovcnt =3D mbuf_burst_segs(pkts, nb_pkts); - struct iovec iov[iovcnt]; - unsigned int i, cnt; - ssize_t ret; + struct iovec iov[IOV_MAX]; + unsigned int i, cnt =3D 0; + ssize_t ret, total =3D 0; =20 - for (i =3D cnt =3D 0; i < nb_pkts; i++) { + for (i =3D 0; i < nb_pkts; i++) { struct rte_mbuf *m =3D pkts[i]; struct pcapng_enhance_packet_block *epb; =20 @@ -589,6 +572,20 @@ rte_pcapng_write_packets(rte_pcapng_t *self, return -1; } =20 + /* + * Handle case of highly fragmented and large burst size + * Note: this assumes that max segments per mbuf < IOV_MAX + */ + if (unlikely(cnt + m->nb_segs >=3D IOV_MAX)) { + ret =3D writev(self->outfd, iov, cnt); + if (unlikely(ret < 0)) { + rte_errno =3D errno; + return -1; + } + total +=3D ret; + cnt =3D 0; + } + /* * The DPDK port is recorded during pcapng_copy. * Map that to PCAPNG interface in file. @@ -601,10 +598,12 @@ rte_pcapng_write_packets(rte_pcapng_t *self, } while ((m =3D m->next)); } =20 - ret =3D writev(self->outfd, iov, iovcnt); - if (unlikely(ret < 0)) + ret =3D writev(self->outfd, iov, cnt); + if (unlikely(ret < 0)) { rte_errno =3D errno; - return ret; + return -1; + } + return total + ret; } =20 /* Create new pcapng writer handle */