From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 96729A04A2; Wed, 3 Jun 2020 14:52:38 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 613B01D167; Wed, 3 Jun 2020 14:52:38 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id E3B491D15A; Wed, 3 Jun 2020 14:52:36 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 053CkWps024527; Wed, 3 Jun 2020 05:52:35 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=date : from : to : cc : subject : message-id : references : content-type : in-reply-to : mime-version; s=pfpt0818; bh=GcEUxa+TmIUAK1qmKmXZTt/dKLngEYDJSBwVBiqoPrc=; b=YYlqLG48ucQbfYqb7fdvudq3/Ttdubkclsv3GHtSd3zdOOWvU1y4V0AL1BN40YfmUbrb IV7s4OQW3g8ArAQWV3+1r2uhQaxWM+ekHgfo9Iq4yPmkf9MnnnrJ708ecWT3iCsRtDL0 uvs9hTJnla0a9pDd35XKjl/hukBuMFgOy57PQeBhZwOIt5Ih/mvHx5R2XzaPsXdBiIzZ nIc3YYzNKPdFh9s/Pf6vy9cOtSgcatWv5t+EEFU8YJdzKi+J9KIotLMSH1UVz2u+Z7Jz TTdglkmMfSvXKvo0Fq9hCM2Sn0S/nlRoQQ2xEHIj93RGDF79yYYwAj+nyHdP5nQWSUUp gQ== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0a-0016f401.pphosted.com with ESMTP id 31bmupyhwc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 03 Jun 2020 05:52:35 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 3 Jun 2020 05:52:33 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.177) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 3 Jun 2020 05:52:33 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HoSb32RFecRtBZLwFzJ8LM5+XeYvt2Xpk2v36s/Dox6Lrp7ZCgE/UZFBkAV0Aa6JlqnG27wjQaPadSZGI7XVcSnyaINC6ZXWYb5j85SQ0VIhuhOGc/gL1RDN045dQ8dDzvOTMnhs25qUk9kLJit25Q4wjCxbAvVhBluTWNUDxwObVEguUiffIYe+DricqWnlpCchgM4Drf/UxIfmrSeExeD/rM8LPezfZEE8Tlqv8uhNLTRWmt08J6jjwzPqxGIzCCR7YWYZ4GUCDx9RDsfU3qIkpkMitdNcZ6csLmZR2YMfPs9Zfo5gIEzA6Tf2628mhREdnEb5S0xsekZMImK+kQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GcEUxa+TmIUAK1qmKmXZTt/dKLngEYDJSBwVBiqoPrc=; b=boTgAr9bkfc7+hLXMbt6UzDz+Ok8ZLEQhd2TUQsC8DFVgGRRvLUK1qrtccDGriFb9P/pdlKhyQRV8Qf448PucD33LpwlBQ8+TiMq8OOAr8qG5LTebo2RXNH7KrG0rb/2TqYhMlu8dzMf7A1YjCdD0AS635XIYUBaxHbrmaTG+pKwx72BKCEnc+shs7fpmPRF/wSZ38FAm4PKILjz7WT2bSZZ2vdNg6O5bc/GfzTrABavl5sghrWsgmaE2xyYep2u2DXzE9KpOlXDOcSU9dst69owG/PKvP90rs2kxowWbqZ4eWYviiSKZxn/FD7uRjguP57/lpDzOr8DfgD1q0ov+g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GcEUxa+TmIUAK1qmKmXZTt/dKLngEYDJSBwVBiqoPrc=; b=nYNNuOFEZrx0yYBJS+C2r4We6AX/IrKGJYN1Ien5aRu9Ql2XNB4+1P6KNVWcvXm52eZ9NNI5K3Ox6ui5Zj+8o/JAHDKsSKxdUkXvx7MyDyVVjYE/2YxposmOfj6OjvVpuLa59yCctdSql4dQj8KbLaoMkUSddrqSYBE4eGuMyAI= Authentication-Results: 6wind.com; dkim=none (message not signed) header.d=none;6wind.com; dmarc=none action=none header.from=marvell.com; Received: from BN8PR18MB2913.namprd18.prod.outlook.com (2603:10b6:408:71::27) by BN8PR18MB2914.namprd18.prod.outlook.com (2603:10b6:408:a6::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.17; Wed, 3 Jun 2020 12:52:31 +0000 Received: from BN8PR18MB2913.namprd18.prod.outlook.com ([fe80::5d7e:2de3:8f95:4c26]) by BN8PR18MB2913.namprd18.prod.outlook.com ([fe80::5d7e:2de3:8f95:4c26%7]) with mapi id 15.20.3066.018; Wed, 3 Jun 2020 12:52:31 +0000 Date: Wed, 3 Jun 2020 18:22:14 +0530 From: Nithin Dabilpuram To: Olivier Matz CC: "Ananyev, Konstantin" , Jerin Jacob , "Dumitrescu, Cristian" , Nithin Dabilpuram , Thomas Monjalon , "Yigit, Ferruh" , Andrew Rybchenko , Ori Kam , "Burakov, Anatoly" , "Mcnamara, John" , "Kovacevic, Marko" , dpdk-dev , Jerin Jacob , Krzysztof Kanas , "techboard@dpdk.org" Message-ID: <20200603125214.GA8237@outlook.office365.com> References: <20200515100845.GA19989@outlook.office365.com> <20200515135746.GB9696@outlook.office365.com> <20200528154328.GA3029@outlook.office365.com> <20200602142537.GA6274@outlook.office365.com> <20200603082844.GG12564@platinum> <20200603104414.GA28936@outlook.office365.com> <20200603113822.GI12564@platinum> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200603113822.GI12564@platinum> User-Agent: Mutt/1.12.2 (34cd43c) (2019-09-21) X-ClientProxiedBy: BM1PR0101CA0040.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1a::26) To BN8PR18MB2913.namprd18.prod.outlook.com (2603:10b6:408:71::27) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from outlook.office365.com (115.113.156.2) by BM1PR0101CA0040.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:1a::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3066.18 via Frontend Transport; Wed, 3 Jun 2020 12:52:27 +0000 X-Originating-IP: [115.113.156.2] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9a9e1109-ee0b-4b06-d50e-08d807bcfa69 X-MS-TrafficTypeDiagnostic: BN8PR18MB2914: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3631; X-Forefront-PRVS: 04238CD941 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: H5D43CFRjjE2fiuQTL9mYqVmXFx8Nu3m75p7rNijAQluO98JKDtlGNnx02L/JlxOibnwYDUXHtzrCpn0hAyf6GbNWkHwoWfiukezulRkPzZ0TyyJbJCjmzu8WAw5ZFaLfezqFBbJiVfkX0fyHToDk4GEAkh2Rg1uF2HJsc3LZXh4NksufYm5/420uhvLtX72430WN98b/GS5YwyN6ft+2BHbAEGQUkOtDtNBaolqA007ZW+RQi/C381wBlV9MrlJ8EIgHLfeaFqeN8Vc7QIDxmoPhmO4RNB4NIeqvgYTvKmL7/b1evkLzs0ar5vPcbTWq5BlMxspB6P4R7yiuiNSuNhiZEuY7ATCdIA+WGhfIEmSCq+Wg+84MZfe2uMryllR97Scaog8GP7eLmEi16Oe9g== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN8PR18MB2913.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(396003)(366004)(136003)(376002)(346002)(39860400002)(8676002)(52116002)(6506007)(55236004)(16526019)(6916009)(7696005)(7416002)(956004)(26005)(66946007)(186003)(66556008)(4326008)(66476007)(2906002)(6666004)(5660300002)(9686003)(478600001)(83380400001)(55016002)(86362001)(33656002)(966005)(54906003)(316002)(1076003)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: XZwRKip3+aTmjzg/cEnQEqWf7DWb/4nIwy/O6QMO8stmSpM7n3NFF1///OsNwnhbmU5eg6nNScYQUKUPbmPofrt3/ER6wDxhTSnovkrXnnX+/sID3Z2fSEm8/h10DCPO80nQbiejlzE/aCJolFi/UHNtuGBuJWS9NTIEQnASmt3/rqkMutoGuIp25vFBYvvGvYO9xA1sJ0iO3beRxQRW+S+Xt02CDBZTEGPYtdc6T2zAMdMV7Pg0lkERdVVCELyhr2Cf/RljwqI4gwnVZYwBzxDCp59s2m/tEd5+2Tk2qza4j50r6/30pEbEyiFtu0uS8hDXJ3dvnfGhuWs939h4vKmlDsZxOiJWrQDrAA0PG5kqxa66j+86M4mMHbIJuaFpTQzEXW3Td3eXeLh2Ubm8yZJPzyfev3jN3kirOIU47eEyA8JOhQIR8LBJlYKibEEXXGlmtACX0h5Sew7I9/pnB8rptByQasGHC7Sihdmg+qQ= X-MS-Exchange-CrossTenant-Network-Message-Id: 9a9e1109-ee0b-4b06-d50e-08d807bcfa69 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2020 12:52:31.6986 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: mTjxPhqO4AcAkh5XGFpvZjQ2Xiwkr931weVRQUiBPmKVE/QeIUMADlWJgHuPN8GxHc39W+hgXS1tszYvFDO2Qw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR18MB2914 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216, 18.0.687 definitions=2020-06-03_12:2020-06-02, 2020-06-03 signatures=0 Subject: Re: [dpdk-dev] [EXT] Re: [PATCH 1/3] mbuf: add Tx offloads for packet marking X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Jun 03, 2020 at 01:38:22PM +0200, Olivier Matz wrote: > Hi Nithin, > > On Wed, Jun 03, 2020 at 04:14:14PM +0530, Nithin Dabilpuram wrote: > > On Wed, Jun 03, 2020 at 10:28:44AM +0200, Olivier Matz wrote: > > > Hi, > > > > > > On Tue, Jun 02, 2020 at 07:55:37PM +0530, Nithin Dabilpuram wrote: > > > > On Tue, Jun 02, 2020 at 10:53:08AM +0000, Ananyev, Konstantin wrote: > > > > > Hi Jerin, > > > > > > > > > > > > > > I also share Olivier's concern about consuming 3 bits in ol_flags for that feature. > > > > > > > > > Can it probably be squeezed somehow? > > > > > > > > > Let say we reserve one flag that this information is present or not, and > > > > > > > > > re-use one of rx-only fields for store additional information (packet_type, or so). > > > > > > > > > Or might be some other approach. > > > > > > > > > > > > > > > > We are fine with this approach where we define one bit in Tx offloads for pkt > > > > > > > > marking and and 3 bits reused from Rx offload flags area. > > > > > > > > > > > > > > > > For example: > > > > > > > > > > > > > > > > @@ -186,10 +186,16 @@ extern "C" { > > > > > > > > > > > > > > > > /* add new RX flags here, don't forget to update PKT_FIRST_FREE */ > > > > > > > > > > > > > > > > +/* Reused Rx offload bits for Tx offloads */ > > > > > > > > +#define PKT_X_TX_MARK_VLAN_DEI (1ULL << 0) > > > > > > > > +#define PKT_X_TX_MARK_IP_DSCP (1ULL << 1) > > > > > > > > +#define PKT_X_TX_MARK_IP_ECN (1ULL << 2) > > > > > > > > + > > > > > > > > #define PKT_FIRST_FREE (1ULL << 23) > > > > > > > > -#define PKT_LAST_FREE (1ULL << 40) > > > > > > > > +#define PKT_LAST_FREE (1ULL << 39) > > > > > > > > > > > > > > > > /* add new TX flags here, don't forget to update PKT_LAST_FREE */ > > > > > > > > +#define PKT_TX_MARK_EN (1ULL << 40) > > > > > > > > > > > > > > > > Is this fine ? > > > > > > > > > > > > > > Any thoughts on this approach which uses only 1 bit in Tx flags out of 18 > > > > > > > and reuse unused Rx flag bits ? > > > > > > > > > > My thought was not about re-defining the flags (I think it is better to keep them intact), > > > > > but adding a union for one of rx-only fields (packet_type/rss/timestamp). > > > > > > > > Ok. Adding a union field at packet_type field is also fine like below. > > > > > > > > @@ -187,9 +187,10 @@ extern "C" { > > > > /* add new RX flags here, don't forget to update PKT_FIRST_FREE */ > > > > > > > > #define PKT_FIRST_FREE (1ULL << 23) > > > > -#define PKT_LAST_FREE (1ULL << 40) > > > > +#define PKT_LAST_FREE (1ULL << 39) > > > > > > > > /* add new TX flags here, don't forget to update PKT_LAST_FREE */ > > > > +#define PKT_TX_MARK_EN (1ULL << 40) > > > > > > > > /** > > > > * Outer UDP checksum offload flag. This flag is used for enabling > > > > @@ -461,6 +462,14 @@ enum { > > > > #endif > > > > }; > > > > > > > > +/* Tx packet marking flags in rte_mbuf::tx_mark. > > > > + * Valid only when PKT_TX_MARK_EN is set in > > > > + * rte_mbuf::ol_flags. > > > > + */ > > > > +#define TX_MARK_VLAN_DEI (1ULL << 0) > > > > +#define TX_MARK_IP_DSCP (1ULL << 1) > > > > +#define TX_MARK_IP_ECN (1ULL << 2) > > > > + > > > > /** > > > > * The generic rte_mbuf, containing a packet mbuf. > > > > */ > > > > @@ -543,6 +552,10 @@ struct rte_mbuf { > > > > }; > > > > uint32_t inner_l4_type:4; /**< Inner L4 type. */ > > > > }; > > > > + struct { > > > > + uint32_t reserved:29; > > > > + uint32_t tx_mark:3; > > > > + }; > > > > }; > > > > > > > > > > > > > > > > Please correct me if this is not what you mean. > > > > > > I'm not a big fan of reusing Rx fields or flags for Tx. > > > It's not obvious for an application than adding a tx_mark will overwrite > > > the packet_type. I understand that the risk is limited because packet_type > > > is Rx and the marks are Tx, but there is still one. > > > > I'm also not a big fan but just wanted to take this approach so that, > > it can both conserve space and also help fast path. > > Reusing Rx area is however not a new thing as is already followed for > > mbuf->txadapter field. > > Yes, and in my opinion this is something we should avoid when possible, > because it makes some features exclusive (ex: the big union with > sched/rss/adapter/usr/...). > > > Apart from documentation issue, Is there any other issue or future > > ramification with using Rx field's for Tx ? > > No, I don't see any other issue except the ones we already mentioned (doc, code clarity, ). > > > If it is only about documentation, then we can add more documentation to make things clear. > > > > > > To summarize the different proposed approaches (please correct me if I'm wrong): > > > > > > a- add 3 Tx mbuf flags > > > (-) consumes limited resource > > > > > > b- add 3 dynamic flags > > > (-) slower > > > > - Tx burst Vector implementation can't be done for this tx offload as > > offset keeps changing. > > A vector implementation can be done. But yes, it would be slower than > with a static flag. Very slow atleast in our HW as, we try to translate ol_flags to HW descriptor flags in addition to extra operations to be done like offset calculations etc. So if we have fixed offsets, then it is easy to have static constant 128/256 bit words with offsets and use things like shuffle/table lookup to reorganize multiple mbuf flags to descriptor fields in a single instruction. > > > > > > > c- add 1 Tx flag and union with Rx field > > > (-) exclusive with Rx field > > > (-) still consumes one flag > > > > > > My preference is still b-, for these reasons: > > > > > > - There are many different DPDK use cases, and resources in mbuf is tight. > > > Recent contributions (rte_flow and ice driver) already made use of dynamic > > > fields/flags. > > - Since RTE_FLOW metadata is 32-bit field, it is a clear candidate for > > dynamic flags. > > I'm not sure to get why it is a better candidate than packet marking. > You mean because it requires more room in mbuf? Yes, I feel space consumption is one way to decide whether it should be a dynfield or static field. IMO, other parameter to judge could be whether the field definition/usage itself is well know standard and is a part of RTE spec or its definition is vendor specific. > > > - ICE PMD's dynamic field is however a vendor specific field and only for > > ICE PMD users. > > Yes, but ICE PMD users may be as important as packet marking users. Agree, I only meant that the flag ICE PMD registered cannot be used for other PMD's so by using dynamic field, we are avoiding wastage of a static field that is needed only by one specific PMD irrespective of whether that PMD is probed or not. > > > In this case, it is just 1 bit out of 18 free bits available in ol_flags. > > > > > > > > - When I implemented the dynamic fields/flags feature, I did a test which > > > showed that the cost of having a dynamic offset was few cycles (on my test > > > platform, it was~3 cycles for reading a field and ~2 cycles for writing a > > > field). > > > > I think this cost is of the case where the address where the dyn_offset is > > stored is already in cache as it needs to be read first. > > This fetch of the value (in case it is not in cache) can be done once per bulk, > so I'm not sure the impact would be high. Agreed, for bulk case offset loading should have less impact. > > > Regards, > Olivier > > > > > > > > > > > > Regards, > > > Olivier > > > > > > > > > > > > > > > > > > > > > > > > + Techboard > > > > > > > > > > > > There is a related thread going on > > > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__mails.dpdk.org_archives_dev_2020-2DMay_168810.html&d=DwIGaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=FZ_tPCbgFOh18zwRPO9H0yDx8VW38vuapifdDfc8SFQ&m=nyV4Rud03HW6DbWMpyvOCulQNkagmfo0wKtrwQ7zmmg&s=VuktoUb_xoLsHKdB9mV87x67cP9tXk3DqVXptt9nF_s&e= > > > > > > > > > > > > If there is no consensus on email, then I would like to add this item > > > > > > to the next TB meeting. > > > > > > > > > > Ok, I'll add that to tomorrow meeting agenda. > > > > > Konstantin > > > > > > > > > > >