From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A5676A04EF; Tue, 2 Jun 2020 08:00:02 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 583AD1BFC7; Tue, 2 Jun 2020 08:00:02 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2046.outbound.protection.outlook.com [40.107.20.46]) by dpdk.org (Postfix) with ESMTP id 0CC431BFAE; Tue, 2 Jun 2020 08:00:01 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jCqj0l/+RngNntZaQM7O4XaEnXbwL1HKNNXjefOE1+Ru9/kstNcNebKPcOaiyh6tMbYKqMjU3tjGot7/Ps1q6jzVor1ysIOGu5xqhawqigq0qQ0kIv8xghhgzs5yfwLStrdtwr8Y0haB9z2lKbhf7I14EA8UBeofwdJv2S7BYrfPXpV4juZD5iLcfnSnFGfPtquPNLElj9mo2bQvZg7bt3EZieJGZtk/QjaK5V0Ahtys+sc2E1dRpvwVgiFivjG2pf4gCuSyyAsENRI3Ye/erPr06jecsdNCdrbWPBYe8BmIkRx94rFXkY1UJoql5KkvJBSmAMf7epl18qc3nIj2Wg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EFoqyJzRtHLTRQvOcUql2J3kkfjmx6Ya5DzDg0Dc1Bk=; b=PqSaQqVmd1YlQs3dBKWwulXVXdFxAfb06F1oxKSGBt7xR/+won8kiv4KN2MM2y25lCI9wjPSoblRMawG70wl3M3AKWOomym+9kNzSQdliwv23oA6ot3i0HHw/R2lAF7ZHSlbVyz/inyUp3IB/JDzTvAbGlGc/0CD1yAriWgMm5Y67r6CEiG/rrC1b47ZGnXJZdKu8vPpoSbbynaVk0T037oceyeylfavJVapM4QkFmOJuHd6ypku/8uEw83QZupvq+M1m6t0Qll/QZnfjEsFJhyfcJ44/r3qobh4zXPQr4FcEUfN5kkP2fxuVUaG7CtMPpvlb1jM0Z/JKs7LUXidzg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EFoqyJzRtHLTRQvOcUql2J3kkfjmx6Ya5DzDg0Dc1Bk=; b=Ssk1Ebu/FhrHPzFWDit3QrCY7ML5EAWZcYCdAMfp7ShNwVa5Fl5Y2Ftuf1cU6yqV7mWNvY++xazy0F6XNwxCSWEvss5MC88byt4IeXOTo+nn7utw79euKT4hqcMg0jq762tGatYPPkeW33xJvK3fZaxk+O8lE/f2jGTzk6RTxWU= Received: from AM0PR0502MB4019.eurprd05.prod.outlook.com (2603:10a6:208:f::11) by AM0PR0502MB3891.eurprd05.prod.outlook.com (2603:10a6:208:1b::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.19; Tue, 2 Jun 2020 05:59:59 +0000 Received: from AM0PR0502MB4019.eurprd05.prod.outlook.com ([fe80::b189:d782:4c74:7998]) by AM0PR0502MB4019.eurprd05.prod.outlook.com ([fe80::b189:d782:4c74:7998%7]) with mapi id 15.20.3045.024; Tue, 2 Jun 2020 05:59:59 +0000 From: Matan Azrad To: Alexander Kozyrev , "dev@dpdk.org" CC: "stable@dpdk.org" , Raslan Darawsheh , Slava Ovsiienko Thread-Topic: [dpdk-dev] [PATCH] net/mlx5: fix vectorized Rx burst termination Thread-Index: AQHWOJEEu1zUk2D1OUe5UPgikt/5XajE1Awg Date: Tue, 2 Jun 2020 05:59:59 +0000 Message-ID: References: <1591069841-7846-1-git-send-email-akozyrev@mellanox.com> In-Reply-To: <1591069841-7846-1-git-send-email-akozyrev@mellanox.com> Accept-Language: en-US, he-IL Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: mellanox.com; dkim=none (message not signed) header.d=none;mellanox.com; dmarc=none action=none header.from=mellanox.com; x-originating-ip: [87.68.154.208] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: c11bfbe4-3a30-4ded-8b77-08d806ba2efe x-ms-traffictypediagnostic: AM0PR0502MB3891: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtFwd x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-forefront-prvs: 0422860ED4 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: pWU4VPNSmAoVPSnGK8/UFz0tAn7G+TyKlj8nw0CxW0sJkNS1DlYENOn7Qa2rVCrovgLpYJ5ijGeKsW/5mW5zRHt47MJggayZ+7HYn9C9FMzF2vU5x81fJjQAeCe/BdMkWRFwi1kaPJpfwOxUTczILuE7PYCcfjp2XD3l0hb97+6SqL1eg32cRDNeusIqCVws/JVqQ4fuHCXkJQryIyuiGH+twPnrNYcw60z7STRnnGrHZ17MqEaQWCvC1QGZIEkw/5FSYFau4kVY0F9kJi4zDcppS3sfdIDyxCl3DYPVBNxQOUduc+FRUVGdxEzmPGahCWSlsjqEMckJTDYaOIU1BQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR0502MB4019.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(396003)(136003)(346002)(366004)(376002)(186003)(26005)(66946007)(107886003)(76116006)(66476007)(4326008)(9686003)(66556008)(66446008)(5660300002)(64756008)(86362001)(55016002)(71200400001)(83380400001)(2906002)(52536014)(8936002)(316002)(33656002)(7696005)(110136005)(54906003)(478600001)(8676002)(450100002)(6506007); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: b5KsDDTfLg+T+JKhQ4y9O53kdp5xRO2BuIZF1A+GGsPYbRRaPPI6gP3mGstRWJo1mhyPdd44jxNdfHOmoaw2SDWjU9V+0r4fW9z3a8uc5i0S2Vjw+lFfiQ9Y5PbemI8bRKNCSw1faB8re607k4Jx2M/A78cFWxhyQg67A6OJ6v8S2SpDhvvkzH6hkjuS/j7xnOv+uTOhJ7i0FmYbG5UKnBltCuCLuyTeSawFxdEZ745S4tRYII0wKd5e7SjdbjbYT5OWz1aB21TimDoYFLc4CsyIGL9NsiCPr3VF+WrhilKARy8B0QyEKa0XuWlUy9kqmO+PQaslwbM5PzSTK8GCXM6k50Sg4b9NQtSJkzxZkJLvf8zEUcJ1enGXGgHlsqge56CsWsYvRxhIZX1Cc9jqh5dd1MwfuVRvKPBgmlfTkR/4kIG+kAWpde2bnMYJKC3Uqwrb0Rg2UmRgPMXR21+mB6M9OQZ/gJBxC3UyMvvyEcTly7Frc/tDiLG1oFV0WZnf Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: c11bfbe4-3a30-4ded-8b77-08d806ba2efe X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Jun 2020 05:59:59.4353 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 9Xt7jcZC421YtNGqGSiYkG9Y8lnvbegQEcBAY3l54IZYLn40B2owntUpPIs0YgYT04cPEN4Seuh5pMzdeh513Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB3891 Subject: Re: [dpdk-dev] [PATCH] net/mlx5: fix vectorized Rx burst termination X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Alexander Kozyrev > Maximum burst size of Vectorized Rx burst routine is set to > MLX5_VPMD_RX_MAX_BURST(64). This limits the performance of any > application that would like to gather more than 64 packets from the singl= e Rx > burst for batch processing (i.e. VPP). >=20 > The situation gets worse with a mix of zipped and unzipped CQEs. > They are processed separately and the Rx burst function returns small > number of packets every call. >=20 > Repeat the cycle of gathering packets from the vectorized Rx routine unti= l a > requested number of packets are collected or there are no more CQEs left = to > process. >=20 > Fixes: 6cb559d67b83 ("net/mlx5: add vectorized Rx/Tx burst for x86") > Cc: stable@dpdk.org >=20 > Signed-off-by: Alexander Kozyrev > Acked-by: Slava Ovsiienko Acked-by: Matan Azrad > --- > drivers/net/mlx5/mlx5_rxtx_vec.c | 19 +++++++++++++------ > drivers/net/mlx5/mlx5_rxtx_vec_altivec.h | 13 ++++++++++--- > drivers/net/mlx5/mlx5_rxtx_vec_neon.h | 13 ++++++++++--- > drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 13 ++++++++++--- > 4 files changed, 43 insertions(+), 15 deletions(-) >=20 > diff --git a/drivers/net/mlx5/mlx5_rxtx_vec.c > b/drivers/net/mlx5/mlx5_rxtx_vec.c > index 1518bdd..b38bd20 100644 > --- a/drivers/net/mlx5/mlx5_rxtx_vec.c > +++ b/drivers/net/mlx5/mlx5_rxtx_vec.c > @@ -103,13 +103,20 @@ > mlx5_rx_burst_vec(void *dpdk_rxq, struct rte_mbuf **pkts, uint16_t > pkts_n) { > struct mlx5_rxq_data *rxq =3D dpdk_rxq; > - uint16_t nb_rx; > + uint16_t nb_rx =3D 0; > + uint16_t tn =3D 0; > uint64_t err =3D 0; > - > - nb_rx =3D rxq_burst_v(rxq, pkts, pkts_n, &err); > - if (unlikely(err | rxq->err_state)) > - nb_rx =3D rxq_handle_pending_error(rxq, pkts, nb_rx); > - return nb_rx; > + bool no_cq =3D false; > + > + do { > + nb_rx =3D rxq_burst_v(rxq, pkts + tn, pkts_n - tn, &err, > &no_cq); > + if (unlikely(err | rxq->err_state)) > + nb_rx =3D rxq_handle_pending_error(rxq, pkts + tn, > nb_rx); > + tn +=3D nb_rx; > + if (unlikely(no_cq)) > + break; > + } while (tn !=3D pkts_n); > + return tn; > } >=20 > /** > diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h > b/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h > index 26715ef..b55138a 100644 > --- a/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h > +++ b/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h > @@ -564,13 +564,15 @@ > * @param[out] err > * Pointer to a flag. Set non-zero value if pkts array has at least on= e error > * packet to handle. > + * @param[out] no_cq > + * Pointer to a boolean. Set true if no new CQE seen. > * > * @return > * Number of packets received including errors (<=3D pkts_n). > */ > static inline uint16_t > rxq_burst_v(struct mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t > pkts_n, > - uint64_t *err) > + uint64_t *err, bool *no_cq) > { > const uint16_t q_n =3D 1 << rxq->cqe_n; > const uint16_t q_mask =3D q_n - 1; > @@ -663,8 +665,10 @@ > /* Not to cross queue end. */ > pkts_n =3D RTE_MIN(pkts_n, q_n - elts_idx); > pkts_n =3D RTE_MIN(pkts_n, q_n - cq_idx); > - if (!pkts_n) > + if (!pkts_n) { > + *no_cq =3D !rcvd_pkt; > return rcvd_pkt; > + } > /* At this point, there shouldn't be any remaining packets. */ > MLX5_ASSERT(rxq->decompressed =3D=3D 0); >=20 > @@ -1079,8 +1083,10 @@ > break; > } > /* If no new CQE seen, return without updating cq_db. */ > - if (unlikely(!nocmp_n && comp_idx =3D=3D > MLX5_VPMD_DESCS_PER_LOOP)) > + if (unlikely(!nocmp_n && comp_idx =3D=3D > MLX5_VPMD_DESCS_PER_LOOP)) { > + *no_cq =3D true; > return rcvd_pkt; > + } > /* Update the consumer indexes for non-compressed CQEs. */ > MLX5_ASSERT(nocmp_n <=3D pkts_n); > rxq->cq_ci +=3D nocmp_n; > @@ -1108,6 +1114,7 @@ > } > rte_compiler_barrier(); > *rxq->cq_db =3D rte_cpu_to_be_32(rxq->cq_ci); > + *no_cq =3D !rcvd_pkt; > return rcvd_pkt; > } >=20 > diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h > b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h > index ecafbf8..3007c03 100644 > --- a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h > +++ b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h > @@ -378,13 +378,15 @@ > * @param[out] err > * Pointer to a flag. Set non-zero value if pkts array has at least on= e error > * packet to handle. > + * @param[out] no_cq > + * Pointer to a boolean. Set true if no new CQE seen. > * > * @return > * Number of packets received including errors (<=3D pkts_n). > */ > static inline uint16_t > rxq_burst_v(struct mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t > pkts_n, > - uint64_t *err) > + uint64_t *err, bool *no_cq) > { > const uint16_t q_n =3D 1 << rxq->cqe_n; > const uint16_t q_mask =3D q_n - 1; > @@ -485,8 +487,10 @@ > /* Not to cross queue end. */ > pkts_n =3D RTE_MIN(pkts_n, q_n - elts_idx); > pkts_n =3D RTE_MIN(pkts_n, q_n - cq_idx); > - if (!pkts_n) > + if (!pkts_n) { > + *no_cq =3D !rcvd_pkt; > return rcvd_pkt; > + } > /* At this point, there shouldn't be any remained packets. */ > MLX5_ASSERT(rxq->decompressed =3D=3D 0); > /* > @@ -745,8 +749,10 @@ > break; > } > /* If no new CQE seen, return without updating cq_db. */ > - if (unlikely(!nocmp_n && comp_idx =3D=3D > MLX5_VPMD_DESCS_PER_LOOP)) > + if (unlikely(!nocmp_n && comp_idx =3D=3D > MLX5_VPMD_DESCS_PER_LOOP)) { > + *no_cq =3D true; > return rcvd_pkt; > + } > /* Update the consumer indexes for non-compressed CQEs. */ > MLX5_ASSERT(nocmp_n <=3D pkts_n); > rxq->cq_ci +=3D nocmp_n; > @@ -774,6 +780,7 @@ > } > rte_cio_wmb(); > *rxq->cq_db =3D rte_cpu_to_be_32(rxq->cq_ci); > + *no_cq =3D !rcvd_pkt; > return rcvd_pkt; > } >=20 > diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h > b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h > index 6847ae7..da5960a 100644 > --- a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h > +++ b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h > @@ -385,13 +385,15 @@ > * @param[out] err > * Pointer to a flag. Set non-zero value if pkts array has at least on= e error > * packet to handle. > + * @param[out] no_cq > + * Pointer to a boolean. Set true if no new CQE seen. > * > * @return > * Number of packets received including errors (<=3D pkts_n). > */ > static inline uint16_t > rxq_burst_v(struct mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t > pkts_n, > - uint64_t *err) > + uint64_t *err, bool *no_cq) > { > const uint16_t q_n =3D 1 << rxq->cqe_n; > const uint16_t q_mask =3D q_n - 1; > @@ -473,8 +475,10 @@ > /* Not to cross queue end. */ > pkts_n =3D RTE_MIN(pkts_n, q_n - elts_idx); > pkts_n =3D RTE_MIN(pkts_n, q_n - cq_idx); > - if (!pkts_n) > + if (!pkts_n) { > + *no_cq =3D !rcvd_pkt; > return rcvd_pkt; > + } > /* At this point, there shouldn't be any remained packets. */ > MLX5_ASSERT(rxq->decompressed =3D=3D 0); > /* > @@ -696,8 +700,10 @@ > break; > } > /* If no new CQE seen, return without updating cq_db. */ > - if (unlikely(!nocmp_n && comp_idx =3D=3D > MLX5_VPMD_DESCS_PER_LOOP)) > + if (unlikely(!nocmp_n && comp_idx =3D=3D > MLX5_VPMD_DESCS_PER_LOOP)) { > + *no_cq =3D true; > return rcvd_pkt; > + } > /* Update the consumer indexes for non-compressed CQEs. */ > MLX5_ASSERT(nocmp_n <=3D pkts_n); > rxq->cq_ci +=3D nocmp_n; > @@ -725,6 +731,7 @@ > } > rte_compiler_barrier(); > *rxq->cq_db =3D rte_cpu_to_be_32(rxq->cq_ci); > + *no_cq =3D !rcvd_pkt; > return rcvd_pkt; > } >=20 > -- > 1.8.3.1