From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0049.outbound.protection.outlook.com [104.47.1.49]) by dpdk.org (Postfix) with ESMTP id 6F5F74CBB for ; Sun, 25 Mar 2018 18:17:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=PKdajNZzSbFVpkvNYugbOSfa+EBiUb3m8LCMEcOcii0=; b=aPmKmOBxgnobWUCThGcNW/DywqarLG/fGZxSgB92Pi03oE/iF+740Kt37s+dRwqHrs3lsk6ZEgnMo6G6vox3Bft+B4k2h+dI4Dtg70a91xCgjNlwyP2MkjvOW412HVQIoFGkg/MgBu7kAaa7qhMI3IZHqKjU3jerFODlD9+0Luw= Received: from AM0PR0502MB3875.eurprd05.prod.outlook.com (52.133.48.17) by AM0PR0502MB3892.eurprd05.prod.outlook.com (52.133.48.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.609.10; Sun, 25 Mar 2018 16:17:51 +0000 Received: from AM0PR0502MB3875.eurprd05.prod.outlook.com ([fe80::45e8:2d27:5aec:2459]) by AM0PR0502MB3875.eurprd05.prod.outlook.com ([fe80::45e8:2d27:5aec:2459%13]) with mapi id 15.20.0609.012; Sun, 25 Mar 2018 16:17:51 +0000 From: Ophir Munk To: Adrien Mazarguil CC: "dev@dpdk.org" , Thomas Monjalon , "Olga Shern" , Shahaf Shuler Thread-Topic: [PATCH v2] net/mlx4: support CRC strip toggling Thread-Index: AQHTv6CCcHWS0bEl8UySzkaSKho2qqPd7g2AgAM2g+A= Date: Sun, 25 Mar 2018 16:17:51 +0000 Message-ID: References: <1521450055-11523-1-git-send-email-ophirmu@mellanox.com> <1521477410-8936-1-git-send-email-ophirmu@mellanox.com> <20180323145307.GA3994@6wind.com> In-Reply-To: <20180323145307.GA3994@6wind.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=ophirmu@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM0PR0502MB3892; 7:4748s43uI8i5pAWGj1GrUTy2xHz5atgwHn2iYMvu/9qPO2cYfa7XI/ClLcVUdQnbRAtkJ4GOj1OjWsgSuVpmxVQV7X3gTX+4fq28faNfXjYOHJCG7c9lPt18eRyuGu7H8q96TQZnSkN/8Vbh+MnWllgWoJJnZU8Hb9J6qul+sHmkogsT/ZHRvJb/O6+cDNSU6o2iIRNgqagz8bPJnAeVpq321MfCz6k+dBS4fBkEPMOWtPBM6PKDvhlqhXG2CHnT x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 316b440d-b245-4c5f-bd81-08d5926bf554 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4604075)(3008032)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:AM0PR0502MB3892; x-ms-traffictypediagnostic: AM0PR0502MB3892: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3002001)(10201501046)(3231221)(944501327)(52105095)(93006095)(93001095)(6055026)(6041310)(20161123562045)(20161123558120)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(6072148)(201708071742011); SRVR:AM0PR0502MB3892; BCL:0; PCL:0; RULEID:; SRVR:AM0PR0502MB3892; x-forefront-prvs: 0622A98CD5 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(39380400002)(346002)(39860400002)(366004)(376002)(199004)(189003)(13464003)(5250100002)(5890100001)(305945005)(478600001)(14454004)(74316002)(8936002)(8676002)(81156014)(81166006)(2900100001)(229853002)(7736002)(26005)(53546011)(5660300001)(316002)(102836004)(6916009)(186003)(105586002)(107886003)(68736007)(33656002)(6506007)(25786009)(99286004)(3660700001)(2906002)(53936002)(4326008)(59450400001)(6246003)(7696005)(76176011)(3280700002)(11346002)(9686003)(3846002)(54906003)(97736004)(6436002)(55016002)(106356001)(6116002)(66066001)(86362001)(446003); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR0502MB3892; H:AM0PR0502MB3875.eurprd05.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: B7LK0bdr6zk7/ZGBhD3IEwPWZLUDzvee1zUnCjFE4jOazwPJW+RzsuJnEiK9n9N0x65QPMjWBWkbebjXUJqkl5beyRIBcmu7okAv5SmKm0yZqrfhkgCQARxF8vZNF6tykbkf8/nmbc828i70/Ihdgv+DmlL9tyaoaddGyRDB5GU1NkGHyO04Ib+toxJDR9ql3gW222J33+ftI/7X/xam5fKrrZrUVBhosldTwpX5Iuuue0AdaLbIcRu7rjP2Umo+NEfyiAwUKGGo7q+Bv+F1NJoeYt/q3SSA5MT4bj3m4c7/mjDIugL2fynvyesIDCF1HhHC5eQHdiTD0OGM5SGdqA== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 316b440d-b245-4c5f-bd81-08d5926bf554 X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Mar 2018 16:17:51.7139 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB3892 Subject: Re: [dpdk-dev] [PATCH v2] net/mlx4: support CRC strip toggling X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Mar 2018 16:17:55 -0000 Hi Adrien, v3 is ready and will be sent soon. Please see inline more comments. > -----Original Message----- > From: Adrien Mazarguil [mailto:adrien.mazarguil@6wind.com] > Sent: Friday, March 23, 2018 5:53 PM > To: Ophir Munk > Cc: dev@dpdk.org; Thomas Monjalon ; Olga Shern > ; Shahaf Shuler > Subject: Re: [PATCH v2] net/mlx4: support CRC strip toggling >=20 > Hi Ophir, >=20 > On Mon, Mar 19, 2018 at 04:36:50PM +0000, Ophir Munk wrote: > > Previous to this commit mlx4 CRC stripping was executed by default and > > there was no verbs API to disable it. > > > > Current support in MLNX_OFED_4.3-1.5.0.0 and above >=20 > I suggest to drop this line as it's not exclusive to MLNX_OFED; the > documented minimum requirements are already correct and rdma-core v15 > also supports it. >=20 Actually rdma-core v15 does not expose CRC capability in mlx4. It is probab= ly expected in rdma-core v18. The OFED support for CRC stripping only starts with in MLNX_OFED_4.3-1.5.0.= 0. Regardless - for v3 I have dropped the versioning information (as requested= ) since I think versioning information=20 can be present in documentation rather than in commit logs. > > Signed-off-by: Ophir Munk >=20 > A few more comments below. >=20 > > --- > > v1: initial version > > v2: following internal reviews > > > > drivers/net/mlx4/mlx4.c | 4 ++++ > > drivers/net/mlx4/mlx4.h | 1 + > > drivers/net/mlx4/mlx4_rxq.c | 33 +++++++++++++++++++++++++++++++-- > > drivers/net/mlx4/mlx4_rxtx.c | 5 ++++- drivers/net/mlx4/mlx4_rxtx.h > > | 1 + > > 5 files changed, 41 insertions(+), 3 deletions(-) > > > > diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c index > > ee93daf..d4f4323 100644 > > --- a/drivers/net/mlx4/mlx4.c > > +++ b/drivers/net/mlx4/mlx4.c > > @@ -578,6 +578,10 @@ mlx4_pci_probe(struct rte_pci_driver *pci_drv, > struct rte_pci_device *pci_dev) > > } > > DEBUG("supported RSS hash fields mask: %016" PRIx64, > > priv->hw_rss_sup); > > + priv->hw_fcs_strip =3D !!(device_attr_ex.raw_packet_caps & > > + > IBV_RAW_PACKET_CAP_SCATTER_FCS); >=20 > Minor nit, indentation contains one extra space. The full block is already with an extra space. So removing the extra space = in just this line will appear miss-aligned with the remaining block. I suggest leaving this line as is for v3. >=20 > > + DEBUG("FCS stripping toggling is %ssupported", > > + (priv->hw_fcs_strip ? "" : "not ")); >=20 > Another minor nit, mlx5 prints "configuration" instead of "toggling", wou= ldn't > it make sense for both PMDs to print the same information? >=20 I have followed your review and changed "toggling" to "configuration" in v3= . However please note that the commit title is "support CRC strip toggling". So using the same wording= "toggling" is clearer in my view. (it was also an input I got from internal review prior to ML submission). > Also the extra set of parentheses around the conditional expression could= be > removed. >=20 Done for v3. Please note that another line had an extra set of parentheses = (not included in this patch) which was updated as well in v3. > > /* Configure the first MAC address by default. */ > > if (mlx4_get_mac(priv, &mac.addr_bytes)) { > > ERROR("cannot get MAC address, is mlx4_en > loaded?" > > diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index > > 19c8a22..3ae3ce6 100644 > > --- a/drivers/net/mlx4/mlx4.h > > +++ b/drivers/net/mlx4/mlx4.h > > @@ -105,6 +105,7 @@ struct priv { > > uint32_t isolated:1; /**< Toggle isolated mode. */ > > uint32_t hw_csum:1; /**< Checksum offload is supported. */ > > uint32_t hw_csum_l2tun:1; /**< Checksum support for L2 tunnels. */ > > + uint32_t hw_fcs_strip:1; /**< FCS stripping toggling is supported. > > +*/ > > uint64_t hw_rss_sup; /**< Supported RSS hash fields (Verbs format). > */ > > struct rte_intr_handle intr_handle; /**< Port interrupt handle. */ > > struct mlx4_drop *drop; /**< Shared resources for drop flow rules. > > */ diff --git a/drivers/net/mlx4/mlx4_rxq.c > > b/drivers/net/mlx4/mlx4_rxq.c index 7a036ed..6748355 100644 > > --- a/drivers/net/mlx4/mlx4_rxq.c > > +++ b/drivers/net/mlx4/mlx4_rxq.c > > @@ -491,6 +491,8 @@ mlx4_rxq_attach(struct rxq *rxq) > > const char *msg; > > struct ibv_cq *cq =3D NULL; > > struct ibv_wq *wq =3D NULL; > > + unsigned int create_flags =3D 0; > > + unsigned int comp_mask =3D 0; >=20 > I suggest using uint32_t to align these with Verb's definitions for struc= t > ibv_wq_init_attr. >=20 Done in v3 > > volatile struct mlx4_wqe_data_seg (*wqes)[]; > > unsigned int i; > > int ret; > > @@ -503,6 +505,11 @@ mlx4_rxq_attach(struct rxq *rxq) > > msg =3D "CQ creation failure"; > > goto error; > > } > > + /* By default, FCS (CRC) is stripped by hardware. */ > > + if (rxq->crc_present) { > > + create_flags |=3D IBV_WQ_FLAGS_SCATTER_FCS; > > + comp_mask |=3D IBV_WQ_INIT_ATTR_FLAGS; > > + } > > wq =3D mlx4_glue->create_wq > > (priv->ctx, > > &(struct ibv_wq_init_attr){ > > @@ -511,6 +518,8 @@ mlx4_rxq_attach(struct rxq *rxq) > > .max_sge =3D sges_n, > > .pd =3D priv->pd, > > .cq =3D cq, > > + .comp_mask =3D comp_mask, > > + .create_flags =3D create_flags, > > }); > > if (!wq) { > > ret =3D errno ? errno : EINVAL; > > @@ -649,9 +658,10 @@ mlx4_rxq_detach(struct rxq *rxq) uint64_t > > mlx4_get_rx_queue_offloads(struct priv *priv) { > > - uint64_t offloads =3D DEV_RX_OFFLOAD_SCATTER | > > - DEV_RX_OFFLOAD_CRC_STRIP; > > + uint64_t offloads =3D DEV_RX_OFFLOAD_SCATTER; > > > > + if (priv->hw_fcs_strip) > > + offloads |=3D DEV_RX_OFFLOAD_CRC_STRIP; > > if (priv->hw_csum) > > offloads |=3D DEV_RX_OFFLOAD_CHECKSUM; > > return offloads; > > @@ -781,6 +791,24 @@ mlx4_rx_queue_setup(struct rte_eth_dev *dev, > uint16_t idx, uint16_t desc, > > (void *)dev, idx); > > return -rte_errno; > > } > > + /* By default, FCS (CRC) is stripped by hardware. */ > > + unsigned char crc_present; >=20 > This variable must be grouped with others at the beginning of the block a= nd > use the same type as its counterpart in struct rxq for consistency, uint3= 2_t. >=20 Done in v3 > > + if (conf->offloads & DEV_RX_OFFLOAD_CRC_STRIP) { > > + crc_present =3D 0; > > + } else if (priv->hw_fcs_strip) { > > + crc_present =3D 1; > > + } else { > > + WARN("%p: CRC stripping has been disabled but will still" > > + " be performed by hardware, make sure MLNX_OFED and" > > + " firmware are up to date", > > + (void *)dev); > > + crc_present =3D 0; > > + } > > + DEBUG("%p: CRC stripping is %s, %u bytes will be subtracted from" > > + " incoming frames to hide it", > > + (void *)dev, > > + crc_present ? "disabled" : "enabled", > > + crc_present << 2); >=20 > The above block must appear prior to the mlx4_zmallocv_socket() call wher= e > other configuration checks are performed. Done in v3 >=20 > > *rxq =3D (struct rxq){ > > .priv =3D priv, > > .mp =3D mp, > > @@ -794,6 +822,7 @@ mlx4_rx_queue_setup(struct rte_eth_dev *dev, > uint16_t idx, uint16_t desc, > > .csum_l2tun =3D priv->hw_csum_l2tun && > > (conf->offloads & > DEV_RX_OFFLOAD_CHECKSUM), > > .l2tun_offload =3D priv->hw_csum_l2tun, > > + .crc_present =3D crc_present, >=20 > One more nit: this line should appear before the l2tun_offload assignment= to > match the order of definitions in struct rxq. >=20 Done in v3 > > .stats =3D { > > .idx =3D idx, > > }, > > diff --git a/drivers/net/mlx4/mlx4_rxtx.c > > b/drivers/net/mlx4/mlx4_rxtx.c index 8ca8b77..84a7bf1 100644 > > --- a/drivers/net/mlx4/mlx4_rxtx.c > > +++ b/drivers/net/mlx4/mlx4_rxtx.c > > @@ -934,12 +934,12 @@ mlx4_rx_burst(void *dpdk_rxq, struct rte_mbuf > **pkts, uint16_t pkts_n) > > goto skip; > > } > > pkt =3D seg; > > + assert(len >=3D (rxq->crc_present << 2)); > > /* Update packet information. */ > > pkt->packet_type =3D > > rxq_cq_to_pkt_type(cqe, rxq- > >l2tun_offload); > > pkt->ol_flags =3D PKT_RX_RSS_HASH; > > pkt->hash.rss =3D cqe->immed_rss_invalid; > > - pkt->pkt_len =3D len; > > if (rxq->csum | rxq->csum_l2tun) { > > uint32_t flags =3D > > mlx4_cqe_flags(cqe, > > @@ -951,6 +951,9 @@ mlx4_rx_burst(void *dpdk_rxq, struct rte_mbuf > **pkts, uint16_t pkts_n) > > rxq->csum, > > rxq->csum_l2tun); > > } > > + if (rxq->crc_present) > > + len -=3D ETHER_CRC_LEN; > > + pkt->pkt_len =3D len; >=20 > I suggest to move this hunk above, where the pkt->pkt_len assignment was > previously made for a shorter diff. Done in v3 >=20 > > } > > rep->nb_segs =3D 1; > > rep->port =3D rxq->port_id; > > diff --git a/drivers/net/mlx4/mlx4_rxtx.h > > b/drivers/net/mlx4/mlx4_rxtx.h index c12bd39..a0633bf 100644 > > --- a/drivers/net/mlx4/mlx4_rxtx.h > > +++ b/drivers/net/mlx4/mlx4_rxtx.h > > @@ -52,6 +52,7 @@ struct rxq { > > volatile uint32_t *rq_db; /**< RQ doorbell record. */ > > uint32_t csum:1; /**< Enable checksum offloading. */ > > uint32_t csum_l2tun:1; /**< Same for L2 tunnels. */ > > + uint32_t crc_present:1; /**< CRC must be subtracted. */ > > uint32_t l2tun_offload:1; /**< L2 tunnel offload is enabled. */ > > struct mlx4_cq mcq; /**< Info for directly manipulating the CQ. */ > > struct mlx4_rxq_stats stats; /**< Rx queue counters. */ > > -- > > 2.7.4 > > >=20 > -- > Adrien Mazarguil > 6WIND