From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C6BB4A057B; Thu, 2 Apr 2020 12:01:12 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9DBA51BE8D; Thu, 2 Apr 2020 12:01:12 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2083.outbound.protection.outlook.com [40.107.21.83]) by dpdk.org (Postfix) with ESMTP id C64A42C15 for ; Thu, 2 Apr 2020 12:01:10 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mTmt95l0X82Ic0Kf4R5X4RynnIiTXl1uZEOS3wItiXCli1hlFNCOL5izcj3il9iGrUgWoXlcHwMsgukt1WFzlie+fxyHjeE19DgXHv7Km+qqOnXf/lTdeKM330nMAomVbTixv6JmxTM+hZHCs4o6Wxg+287K+Jn4XG150Bt1dk3s7X4tI//+QkkX2eYDhYJPJuw9AQmqoreMKCltOxRwS+ZXAbIpWUlCs1/UFmciQfQ+3jXlT5qQeN8xcIkE0w8xCSgFgyoWFlvaz1pvc8EZGwEofUoUHMlE2mJYajJ98qzdShZy1z0fWwbjuxmU/4+Rv8ldocqcctQhWvOcsLjOWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=L77szdaP2YRaZ9P7rAlcdkYBZ88hSxKbIC5iw9/Tl9g=; b=ZzLDh10UkFy0WnW/p6nyMv9NfaLvuNnXrNW5M2ZFpL7uUimHlY28CTDyzvXrX8U5OwRRShNCw70yMeABzxWH9W1lsmRK/VI5Rd76WABaBXCO3zTPBO3GvIr8LgvI6HKaPPM7D1POey8xTcdpr/R3OpiVkFo/wurUdN7dSrZUuWOwZPkvG1HXm32tp8BUP5IUS3jJctny9dymH323QXSxPJEovmNHrNFtvrey5afHJJ92YXWqUFSHRjvhdmf349r8ICba98ljMHxBeBYg1v6AVcGcLaU1MEGdgmK54IkN3I2G8b/SkpAnGM31A+2cNihhFAxldLXyU0GMfvZ42LXLxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=L77szdaP2YRaZ9P7rAlcdkYBZ88hSxKbIC5iw9/Tl9g=; b=fjHXQNih4FeVdejS2nP0ewwGmK1Ulc7/AwxViZHV5sZLivm1pgcfD0E/d2R9LS9MGBPcnlgxG3IeahosAaywsm0qn86XshiZVIBYeT9SYnVttlX4PPoSboF1+fpV4RJioTAR+3TXspIb9g/DrzRA3I1gjw1XVMtVL3oGGwTDX2w= Received: from AM4PR05MB3265.eurprd05.prod.outlook.com (10.171.188.154) by AM4PR05MB3266.eurprd05.prod.outlook.com (10.171.190.152) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2856.20; Thu, 2 Apr 2020 10:01:07 +0000 Received: from AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::da5:7919:35c1:894]) by AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::da5:7919:35c1:894%6]) with mapi id 15.20.2856.019; Thu, 2 Apr 2020 10:01:07 +0000 From: Slava Ovsiienko To: Alexander Kozyrev , "dev@dpdk.org" CC: Raslan Darawsheh , Matan Azrad , "ferruh.yigit@intel.com" , Thomas Monjalon Thread-Topic: [PATCH 2/4] net/mlx5: enable MPRQ multi-stride operations Thread-Index: AQHWB6bW8O7GxIU+L0is31SAQxq2lahlm7ug Date: Thu, 2 Apr 2020 10:01:07 +0000 Message-ID: References: <1585691559-17409-1-git-send-email-akozyrev@mellanox.com> <1585691559-17409-3-git-send-email-akozyrev@mellanox.com> In-Reply-To: <1585691559-17409-3-git-send-email-akozyrev@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=viacheslavo@mellanox.com; x-originating-ip: [95.164.10.10] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: bd2014df-e400-40a2-afac-08d7d6ecc345 x-ms-traffictypediagnostic: AM4PR05MB3266:|AM4PR05MB3266: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr,ExtFwd x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-forefront-prvs: 0361212EA8 x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM4PR05MB3265.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(396003)(136003)(376002)(39860400002)(366004)(346002)(64756008)(66946007)(52536014)(66476007)(5660300002)(2906002)(33656002)(8936002)(66446008)(9686003)(76116006)(66556008)(8676002)(81166006)(110136005)(81156014)(6506007)(54906003)(55016002)(26005)(53546011)(86362001)(7696005)(71200400001)(316002)(186003)(4326008)(478600001); DIR:OUT; SFP:1101; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: n6BAMNeAWDbB06PvHpenpVrZwzPATtIBdC3AgxNIiscqP3LY6kBUgRgtlSKlX1x9grGBBN4EupgzQomqfZrQHlvnYVSurYbIs8quOEjpgisXOys+eWIkvAb3xLCxmhUrLtcetJoNJF8oFixRlgZzLJuhAlAvGdczZJIRS84KplRZPVN5OTnpfbgg7WALKTU7A3/2uqrgW2hG6B3f2/vmOlCRdZ6p8RVRtByNOkpOUsSCL8hx8DVrJysDNhOXaDBriG++y7wRFAjOOsoU4NW6Q0rkFG0euLZ9ADGc+oKt/pxro8ma1rVxPGjsr+5ZPCt6Yasf0mMzD8wSnjjRfch8sv1v+FND0aX9cgZ/AMxDyp/j/ICISpTeEYbgAEEDw5Ojnnlm+wTES6v7loFJlFGJjW6gIO02sZL7RvY5Q3l5SCjoyPL9wiFEvSczmnVSuYBX x-ms-exchange-antispam-messagedata: z6PxXfLeQljikQhm20ZAsctwZhfRbXuEeZD2Mo8EzQbTKPeHNNuhi46NoRzTC72wp/fy9tqPukHEzAWAqAEgaT2CDXNv4suGNn2DMEasi+TGVnzDpAZlj6tjOHzZe366KsipNgIQwe8kKTEFNSCLZw== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: bd2014df-e400-40a2-afac-08d7d6ecc345 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Apr 2020 10:01:07.2025 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 3QNgX+xMefW+EVXe1hgzg90yjJPIM2RdrML+6Nf9YbUyHm5SABO0mhz9MS7d5qbOCnKa1M5gH5zT0FdoqYuHLRblq0jSsKDJeZhAcEc97m0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3266 Subject: Re: [dpdk-dev] [PATCH 2/4] net/mlx5: enable MPRQ multi-stride operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Alexander Kozyrev > Sent: Wednesday, April 1, 2020 0:53 > To: dev@dpdk.org > Cc: Raslan Darawsheh ; Matan Azrad > ; Slava Ovsiienko ; > ferruh.yigit@intel.com; Thomas Monjalon > Subject: [PATCH 2/4] net/mlx5: enable MPRQ multi-stride operations >=20 > MPRQ feature should be updated to allow a packet to be received into > multiple strides in order to support the MTU exceeding 8KB. > Special care is needed to prevent the headroom corruption in the multi-st= ride > mode since the headroom space is borrowed by the PMD from the tail of the > preceding stride. Copy the whole packet into a separate mbuf in this case= or > just the overlapping data if the Rx scattering is supported by an applica= tion. >=20 > Signed-off-by: Alexander Kozyrev Acked-by: Viacheslav Ovsiienko > --- > drivers/net/mlx5/mlx5_rxq.c | 25 ++++------------ > drivers/net/mlx5/mlx5_rxtx.c | 68 +++++++++++++++++++--------------------= ----- > drivers/net/mlx5/mlx5_rxtx.h | 2 +- > 3 files changed, 35 insertions(+), 60 deletions(-) >=20 > diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c in= dex > 85fcfe6..a64f536 100644 > --- a/drivers/net/mlx5/mlx5_rxq.c > +++ b/drivers/net/mlx5/mlx5_rxq.c > @@ -1793,9 +1793,7 @@ struct mlx5_rxq_ctrl * > struct mlx5_priv *priv =3D dev->data->dev_private; > struct mlx5_rxq_ctrl *tmpl; > unsigned int mb_len =3D rte_pktmbuf_data_room_size(mp); > - unsigned int mprq_stride_size; > struct mlx5_dev_config *config =3D &priv->config; > - unsigned int strd_headroom_en; > /* > * Always allocate extra slots, even if eventually > * the vector Rx will not be used. > @@ -1841,27 +1839,13 @@ struct mlx5_rxq_ctrl * > tmpl->socket =3D socket; > if (dev->data->dev_conf.intr_conf.rxq) > tmpl->irq =3D 1; > - /* > - * LRO packet may consume all the stride memory, hence we cannot > - * guaranty head-room near the packet memory in the stride. > - * In this case scatter is, for sure, enabled and an empty mbuf may be > - * added in the start for the head-room. > - */ > - if (lro_on_queue && RTE_PKTMBUF_HEADROOM > 0 && > - non_scatter_min_mbuf_size > mb_len) { > - strd_headroom_en =3D 0; > - mprq_stride_size =3D RTE_MIN(max_rx_pkt_len, > - 1u << config- > >mprq.max_stride_size_n); > - } else { > - strd_headroom_en =3D 1; > - mprq_stride_size =3D non_scatter_min_mbuf_size; > - } > if (!config->mprq.stride_num_n) > config->mprq.stride_num_n =3D MLX5_MPRQ_STRIDE_NUM_N; > if (!config->mprq.stride_size_n) > - config->mprq.stride_size_n =3D (mprq_stride_size <=3D > + config->mprq.stride_size_n =3D (non_scatter_min_mbuf_size <=3D > (1U << config->mprq.max_stride_size_n)) ? > - log2above(mprq_stride_size) : > MLX5_MPRQ_STRIDE_SIZE_N; > + log2above(non_scatter_min_mbuf_size) : > + MLX5_MPRQ_STRIDE_SIZE_N; > /* > * This Rx queue can be configured as a Multi-Packet RQ if all of the > * following conditions are met: > @@ -1877,7 +1861,8 @@ struct mlx5_rxq_ctrl * > tmpl->rxq.strd_num_n =3D config->mprq.stride_num_n; > tmpl->rxq.strd_sz_n =3D config->mprq.stride_size_n; > tmpl->rxq.strd_shift_en =3D MLX5_MPRQ_TWO_BYTE_SHIFT; > - tmpl->rxq.strd_headroom_en =3D strd_headroom_en; > + tmpl->rxq.strd_scatter_en =3D > + !!(offloads & DEV_RX_OFFLOAD_SCATTER); > tmpl->rxq.mprq_max_memcpy_len =3D > RTE_MIN(first_mb_free_size, > config->mprq.max_memcpy_len); > max_lro_size =3D RTE_MIN(max_rx_pkt_len, diff --git > a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index > f3bf763..4c27952 100644 > --- a/drivers/net/mlx5/mlx5_rxtx.c > +++ b/drivers/net/mlx5/mlx5_rxtx.c > @@ -1658,21 +1658,20 @@ enum mlx5_txcmp_code { > unsigned int i =3D 0; > uint32_t rq_ci =3D rxq->rq_ci; > uint16_t consumed_strd =3D rxq->consumed_strd; > - uint16_t headroom_sz =3D rxq->strd_headroom_en * > RTE_PKTMBUF_HEADROOM; > struct mlx5_mprq_buf *buf =3D (*rxq->mprq_bufs)[rq_ci & wq_mask]; >=20 > while (i < pkts_n) { > struct rte_mbuf *pkt; > void *addr; > int ret; > - unsigned int len; > + uint32_t len; > uint16_t strd_cnt; > uint16_t strd_idx; > uint32_t offset; > uint32_t byte_cnt; > + int32_t hdrm_overlap; > volatile struct mlx5_mini_cqe8 *mcqe =3D NULL; > uint32_t rss_hash_res =3D 0; > - uint8_t lro_num_seg; >=20 > if (consumed_strd =3D=3D strd_n) { > /* Replace WQE only if the buffer is still in use. */ > @@ -1719,18 +1718,6 @@ enum mlx5_txcmp_code { > MLX5_ASSERT(strd_idx < strd_n); > MLX5_ASSERT(!((rte_be_to_cpu_16(cqe->wqe_id) ^ rq_ci) & > wq_mask)); > - lro_num_seg =3D cqe->lro_num_seg; > - /* > - * Currently configured to receive a packet per a stride. But if > - * MTU is adjusted through kernel interface, device could > - * consume multiple strides without raising an error. In this > - * case, the packet should be dropped because it is bigger > than > - * the max_rx_pkt_len. > - */ > - if (unlikely(!lro_num_seg && strd_cnt > 1)) { > - ++rxq->stats.idropped; > - continue; > - } > pkt =3D rte_pktmbuf_alloc(rxq->mp); > if (unlikely(pkt =3D=3D NULL)) { > ++rxq->stats.rx_nombuf; > @@ -1742,12 +1729,16 @@ enum mlx5_txcmp_code { > len -=3D RTE_ETHER_CRC_LEN; > offset =3D strd_idx * strd_sz + strd_shift; > addr =3D RTE_PTR_ADD(mlx5_mprq_buf_addr(buf, strd_n), > offset); > + hdrm_overlap =3D len + RTE_PKTMBUF_HEADROOM - strd_cnt * > strd_sz; > /* > * Memcpy packets to the target mbuf if: > * - The size of packet is smaller than > mprq_max_memcpy_len. > * - Out of buffer in the Mempool for Multi-Packet RQ. > + * - There is no space for a headroom and scatter is disabled. > */ > - if (len <=3D rxq->mprq_max_memcpy_len || rxq->mprq_repl =3D=3D > NULL) { > + if (len <=3D rxq->mprq_max_memcpy_len || > + rxq->mprq_repl =3D=3D NULL || > + (hdrm_overlap > 0 && !rxq->strd_scatter_en)) { > /* > * When memcpy'ing packet due to out-of-buffer, the > * packet must be smaller than the target mbuf. > @@ -1769,7 +1760,7 @@ enum mlx5_txcmp_code { > rte_atomic16_add_return(&buf->refcnt, 1); > MLX5_ASSERT((uint16_t)rte_atomic16_read(&buf- > >refcnt) <=3D > strd_n + 1); > - buf_addr =3D RTE_PTR_SUB(addr, headroom_sz); > + buf_addr =3D RTE_PTR_SUB(addr, > RTE_PKTMBUF_HEADROOM); > /* > * MLX5 device doesn't use iova but it is necessary in a > * case where the Rx packet is transmitted via a @@ - > 1788,43 +1779,42 @@ enum mlx5_txcmp_code { > rte_pktmbuf_attach_extbuf(pkt, buf_addr, buf_iova, > buf_len, shinfo); > /* Set mbuf head-room. */ > - pkt->data_off =3D headroom_sz; > + SET_DATA_OFF(pkt, RTE_PKTMBUF_HEADROOM); > MLX5_ASSERT(pkt->ol_flags =3D=3D > EXT_ATTACHED_MBUF); > - /* > - * Prevent potential overflow due to MTU change > through > - * kernel interface. > - */ > - if (unlikely(rte_pktmbuf_tailroom(pkt) < len)) { > - rte_pktmbuf_free_seg(pkt); > - ++rxq->stats.idropped; > - continue; > - } > + MLX5_ASSERT(rte_pktmbuf_tailroom(pkt) < > + len - (hdrm_overlap > 0 ? hdrm_overlap : 0)); > DATA_LEN(pkt) =3D len; > /* > - * LRO packet may consume all the stride memory, in > this > - * case packet head-room space is not guaranteed so > must > - * to add an empty mbuf for the head-room. > + * Copy the last fragment of a packet (up to > headroom > + * size bytes) in case there is a stride overlap with > + * a next packet's headroom. Allocate a separate > mbuf > + * to store this fragment and link it. Scatter is on. > */ > - if (!rxq->strd_headroom_en) { > - struct rte_mbuf *headroom_mbuf =3D > - rte_pktmbuf_alloc(rxq->mp); > + if (hdrm_overlap > 0) { > + MLX5_ASSERT(rxq->strd_scatter_en); > + struct rte_mbuf *seg =3D > + rte_pktmbuf_alloc(rxq->mp); >=20 > - if (unlikely(headroom_mbuf =3D=3D NULL)) { > + if (unlikely(seg =3D=3D NULL)) { > rte_pktmbuf_free_seg(pkt); > ++rxq->stats.rx_nombuf; > break; > } > - PORT(pkt) =3D rxq->port_id; > - NEXT(headroom_mbuf) =3D pkt; > - pkt =3D headroom_mbuf; > + SET_DATA_OFF(seg, 0); > + rte_memcpy(rte_pktmbuf_mtod(seg, void *), > + RTE_PTR_ADD(addr, len - > hdrm_overlap), > + hdrm_overlap); > + DATA_LEN(seg) =3D hdrm_overlap; > + DATA_LEN(pkt) =3D len - hdrm_overlap; > + NEXT(pkt) =3D seg; > NB_SEGS(pkt) =3D 2; > } > } > rxq_cq_to_mbuf(rxq, pkt, cqe, rss_hash_res); > - if (lro_num_seg > 1) { > + if (cqe->lro_num_seg > 1) { > mlx5_lro_update_hdr(addr, cqe, len); > pkt->ol_flags |=3D PKT_RX_LRO; > - pkt->tso_segsz =3D strd_sz; > + pkt->tso_segsz =3D len / cqe->lro_num_seg; > } > PKT_LEN(pkt) =3D len; > PORT(pkt) =3D rxq->port_id; > diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h > index 939778a..d155c24 100644 > --- a/drivers/net/mlx5/mlx5_rxtx.h > +++ b/drivers/net/mlx5/mlx5_rxtx.h > @@ -119,7 +119,7 @@ struct mlx5_rxq_data { > unsigned int strd_sz_n:4; /* Log 2 of stride size. */ > unsigned int strd_shift_en:1; /* Enable 2bytes shift on a stride. */ > unsigned int err_state:2; /* enum mlx5_rxq_err_state. */ > - unsigned int strd_headroom_en:1; /* Enable mbuf headroom in > MPRQ. */ > + unsigned int strd_scatter_en:1; /* Scattered packets from a stride. */ > unsigned int lro:1; /* Enable LRO. */ > unsigned int :1; /* Remaining bits. */ > volatile uint32_t *rq_db; > -- > 1.8.3.1