From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40085.outbound.protection.outlook.com [40.107.4.85]) by dpdk.org (Postfix) with ESMTP id 4195B1041 for ; Fri, 21 Sep 2018 11:00:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jkPdEZALTSzXsGz2NhLKzRiOdk9YvjpIjmvDnwkCE/c=; b=obPM5VauZFyDBEhGIXW58/C/g2o5bJ7zIzoXX66llpHTC/Yv41zw/hIhKr6pvMpAEk4jtRln9gODdN5CC7gsJ9tek8NK1QAe5Tv8Hgc5mU8uVSyv8ey/JwNpiquVrnyZ8U2W0vZXODrYU6n5uFuIzmXZd0FyL8Qca9Bl2wOIlVw= Received: from DB7PR08MB3385.eurprd08.prod.outlook.com (20.176.238.90) by DB7PR08MB3082.eurprd08.prod.outlook.com (52.134.110.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1143.18; Fri, 21 Sep 2018 09:00:32 +0000 Received: from DB7PR08MB3385.eurprd08.prod.outlook.com ([fe80::f0e6:1c3d:9285:472f]) by DB7PR08MB3385.eurprd08.prod.outlook.com ([fe80::f0e6:1c3d:9285:472f%3]) with mapi id 15.20.1143.017; Fri, 21 Sep 2018 09:00:31 +0000 From: "Phil Yang (Arm Technology China)" To: Honnappa Nagarahalli , Jerin Jacob CC: "dev@dpdk.org" , nd , "kkokkilagadda@caviumnetworks.com" , "Gavin Hu (Arm Technology China)" , "ferruh.yigit@intel.com" , Ola Liljedahl Thread-Topic: [PATCH v2 2/3] kni: fix kni fifo synchronization Thread-Index: AQHUULwDTZT/ca7MCUaWgBQi/M2/B6T5SZKAgAAEgwCAAO32AIAAAeWAgAALuICAACTRoA== Date: Fri, 21 Sep 2018 09:00:31 +0000 Message-ID: References: <1537363820-3827-1-git-send-email-phil.yang@arm.com> <1537364560-4124-1-git-send-email-phil.yang@arm.com> <1537364560-4124-2-git-send-email-phil.yang@arm.com> <20180920082846.GB19425@jerin> <20180920153700.GA9459@jerin> <20180921055529.GA15861@jerin> In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Phil.Yang@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB7PR08MB3082; 6:85FzcTVg1PyNQYwX/RRtjrB+ndjL4bTsgSEEIPuPkR/SRU8CX0ZD0E6Vwsuol+s6XL63Ocy8331AsleFKdKMekdqZAaTWxgPLa/7dFb4Wgz5DLj6r8iuVIquII0cwpyiQF2FrqngMhFzrvE2Gdb+EPL/rtMf2S+FKEfzYRNnJmqBMXsOxk4GExXzI80mvF6+ncTvCw3+hAlrK/oQz/kPVPgtfOSXdTh3x8W0/MURJaM9qTG9k3/lG3pGQR4RMao/bxrr0RoyWPXox8oaPNcNL2/vw1hEnEFB8IBGsXYJTmThrSJH9+FjP/tiA5hQbLcaAkiBnz1FHzFZvUwjhZ83cMkLtyfstkpABwone0wQK4PEr5apzahLXJ5o89RbG3y5SOyFwG26JV2fK3//HiRwXwmZ/xRJVpsCGiWRPCthNVwdW2QlJQSs5Stb2l4qFmntLdN7DBbgzWbvWl5I+11D2Q==; 5:iKT8OEsPg3zy4Pk9CuArU31m+TbR225ziHZ/2jRcy54hDDLY3/ZAnNhLuAJe+JK9c6CBaT1Hgyk8tU04le+HSn0hvV6E2piavEH0vPuaQk/JEZ+WIrYch3nQ67i8zVyFwy3Ipwg/P7FQXY3td5tsvV0Sbu7lCjzlI244SOehw94=; 7:CoOPrI0Xe7Wl67YsNS0sguMEEJK9La/o2ANEFS9L4c3UaI1xTrCny9uXxfBCBfea9Jh/ebaW1x2QqBVdMG1EKuyb2VU59ElVTs/KVWHk7DIkJDeoUAyrOmI5yJFIrVX7DQfxy2Zb6mMi3PpAMcfqSeU7fzeaEWw1m/gwyUtOl8l4uxOpAiDV0+2wMee8ZuwkFzyksPfm5Nja86235HlD+p3b7ZhnfUSrVVUvJbzdNEYEwR3NadKap2txBBsSC8fr x-ms-exchange-antispam-srfa-diagnostics: SOS;SOR; x-ms-office365-filtering-correlation-id: 462004bc-b3db-4672-e02e-08d61fa0af8a x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:DB7PR08MB3082; x-ms-traffictypediagnostic: DB7PR08MB3082: nodisclaimer: True x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(180628864354917)(163750095850)(228905959029699); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3231355)(944501410)(52105095)(3002001)(10201501046)(93006095)(93001095)(6055026)(149027)(150027)(6041310)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123558120)(201708071742011)(7699051); SRVR:DB7PR08MB3082; BCL:0; PCL:0; RULEID:; SRVR:DB7PR08MB3082; x-forefront-prvs: 0802ADD973 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(979002)(366004)(346002)(136003)(39860400002)(396003)(376002)(57704003)(13464003)(189003)(199004)(86362001)(3846002)(6506007)(5660300001)(25786009)(93886005)(8936002)(6116002)(4326008)(74316002)(14444005)(106356001)(6246003)(71190400001)(53546011)(71200400001)(256004)(81156014)(105586002)(2906002)(33656002)(14454004)(72206003)(66066001)(478600001)(81166006)(7736002)(26005)(97736004)(486006)(305945005)(2900100001)(68736007)(55016002)(53936002)(9686003)(6436002)(8676002)(110136005)(5250100002)(99286004)(54906003)(76176011)(446003)(7696005)(102836004)(229853002)(11346002)(476003)(316002)(969003)(989001)(999001)(1009001)(1019001); DIR:OUT; SFP:1101; SCL:1; SRVR:DB7PR08MB3082; H:DB7PR08MB3385.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: GHRLwYN0nS/iKQ032k+o1mYns6YI+75k8wIchagICoJQHs2O+agJ1TGruH1B5EaWdobpees4zYC4WPgpAljMqOnsFhfmoHGSFbrq3qZuq6Txv/Rm+YFIen+SrpzfAEhWdrooj95YJxKnBSIb8MGKhVjyLvH/b6m4hiqwfJt9PZ+gdEv3ex0RDXqAHChJRWgkwfdMBOFpOvLDwAJBb8Vyg7JpvU5fmnMndE+J4GkJAob3+ue56i/ISqiIBzGqyg8D4grSbg6/wVmNGYjMwr6FkJPwIyAc72lkxWpfl6C071Ovxy8H0VOUesq5ljkwEKatzIz6cGcrns1dBmUg2Xt3PIkHglLp1DxD4gWrAzCp9ns= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: 462004bc-b3db-4672-e02e-08d61fa0af8a X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Sep 2018 09:00:31.8915 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3082 Subject: Re: [dpdk-dev] [PATCH v2 2/3] kni: fix kni fifo synchronization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2018 09:00:33 -0000 + Ola Liljedahl Thanks, Phil Yang > -----Original Message----- > From: Honnappa Nagarahalli > Sent: Friday, September 21, 2018 2:37 PM > To: Jerin Jacob > Cc: Phil Yang (Arm Technology China) ; dev@dpdk.org; n= d > ; kkokkilagadda@caviumnetworks.com; Gavin Hu (Arm > Technology China) ; ferruh.yigit@intel.com > Subject: RE: [PATCH v2 2/3] kni: fix kni fifo synchronization >=20 > > > > > > > > > > > > > > @@ -69,5 +89,13 @@ kni_fifo_get(struct rte_kni_fifo *fifo, > > > > > > > void **data, unsigned num) static inline uint32_t > > > > > > > kni_fifo_count(struct rte_kni_fifo *fifo) { > > > > > > > +#ifdef RTE_USE_C11_MEM_MODEL > > > > > > > + unsigned fifo_write =3D __atomic_load_n(&fifo->write, > > > > > > > + __ATOMIC_AC= QUIRE); > > > > > > > + unsigned fifo_read =3D __atomic_load_n(&fifo->read, > > > > > > > + > > > > > > > +__ATOMIC_ACQUIRE); > > > > > > > > > > > > Isn't too heavy to have two __ATOMIC_ACQUIREs? a simple > > > > > > rte_smp_rmb() would be enough here. Right? > > > > > > or > > > > > > Do we need __ATOMIC_ACQUIRE for fifo_write case? > > > > > > > > > > > We also had some amount of debate internally on this: > > > > > 1) We do not want to use rte_smp_rmb() as we want to keep the > > > > > memory > > > > models separated (for ex: while using C11, use C11 everywhere). It > > > > is also not sufficient, please see 3) below. > > > > > > > > But Nothing technically wrong in using rte_smp_rmb() here in terms > > > > functionally and code generated by the compiler. > > > > > > rte_smp_rmb() generates 'DMB ISHLD'. This works fine, but it is not o= ptimal. > > 'LDAR' is a better option which is generated when C11 atomics are used. > > > > Yes. But which one is optimal 1 x DMB ISHLD vs 2 x LDAR ? >=20 > Good point. I am not sure which one is optimal, it needs to be measured. = 'DMB > ISHLD' orders 'all' earlier loads against 'all' later loads and stores. '= LDAR' orders > the 'specific' load with 'all' later loads and stores. >=20 > > > > > > > > > > > > > > 2) This API can get called from writer or reader, so both the > > > > > loads have to be __ATOMIC_ACQUIRE > > > > > 3) Other option is to use __ATOMIC_RELAXED. That would allow any > > > > loads/stores around of this API to get reordered, especially since > > > > this is an inline function. This would put burden on the > > > > application to manage the ordering depending on its usage. It will > > > > also require the application to understand the implementation of th= is API. > > > > > > > > __ATOMIC_RELAXED may be fine too for _count() case as it may not > > > > very important to get the exact count for the exact very moment, > > > > Application can retry. > > > > > > > > I am in favor of performance effective implementation. > > > > > > The requirement on the correctness of the count depends on the usage > > > of > > this function. I see the following usage: > > > > > > In the file kni_net.c, function: kni_net_tx: > > > > > > if (kni_fifo_free_count(kni->tx_q) =3D=3D 0 || > > > kni_fifo_count(kni->alloc_q) =3D=3D 0) { > > > /** > > > * If no free entry in tx_q or no entry in alloc_q, > > > * drops skb and goes out. > > > */ > > > goto drop; > > > } > > > > > > There is no retry here, the packet is dropped. > > > > OK. Then pick an implementation which is an optimal this case. > > I think, then rte_smp_rmb() makes sense here as > > a) no #ifdef clutter > > b) it is optimal compared to 2 x LDAR > > > As I understand, one of the principals of using C11 model is to match the= store > releases and load acquires. IMO, combining C11 memory model with barrier > based functions makes the code unreadable. > I realized rte_smp_rmb() is required for x86 as well to prevent compiler > reordering. We can add that in the non-C11 case. This way, we will have c= lean > code for both the options (similar to rte_ring). > So, if 'RTE_USE_C11_MEM_MODEL' is set to 'n', then the 'rte_smp_rmb' woul= d > be used. >=20 > We can look at handling the #ifdef clutter based on Ferruh's feedback. >=20 > > > > > > > > > > > > > > > > > > > > > > > > > > Other than that, I prefer to avoid ifdef clutter by > > > > > > introducing two separate file just like ring C11 implementation= . > > > > > > > > > > > > I don't have strong opinion on this this part, I let KNI > > > > > > MAINTAINER to decide on how to accommodate this change. > > > > > > > > > > I prefer to change this as well, I am open for suggestions. > > > > > Introducing two separate files would be too much for this library= . > > > > > A better > > > > way would be to have something similar to 'smp_store_release' > > > > provided by the kernel. i.e. create #defines for loads/stores. > > > > Hide the clutter behind the #defines. > > > > > > > > No Strong opinion on this, leaving to KNI Maintainer. > > > Will wait on this before re-spinning the patch > > > > > > > > > > > This patch needs to split by two, > > > > a) Fixes for non C11 implementation(i.e new addition to > > > > rte_smp_wmb()) > > > > b) add support for C11 implementation. > > > Agree > > > > > > > > > > > > > > > > > > > > > > > > > + return (fifo->len + fifo_write - fifo_read) & > > > > > > > +(fifo->len - 1); #else > > > > > > > return (fifo->len + fifo->write - fifo->read) & > > > > > > > (fifo->len > > > > > > > - 1); > Requires rte_smp_rmb() for x86 to prevent compiler reordering. >=20 > > > > > > > +#endif > > > > > > > } > > > > > > > -- > > > > > > > 2.7.4 > > > > > > >