From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from rcdn-iport-6.cisco.com (rcdn-iport-6.cisco.com [173.37.86.77]) by dpdk.org (Postfix) with ESMTP id 7CDD69192 for ; Mon, 4 Jan 2016 15:43:45 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=4243; q=dns/txt; s=iport; t=1451918625; x=1453128225; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=tmr1cWkzU2bGcppyxBHrJFy+fZ5nKkERqcHnwhj2m8A=; b=ktrTSNKKe52qfqhsEEQFAWZjJOWMLA+wb0dQB/xhTi+aIUUGYQZ+MIje 1xibM+aKvZkRF1jr6WVEpfrf8XbIggASAt3/nK3bX8a8GuFsQSBhhs4zw Y+fzXuQ6GtYPCln4Tc60lvYGnPMBW2R7mgZYx82bmKLL2ac15eLYWBF80 E=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0D/AQBUhIpW/4cNJK1egzpSbQaIU7N1A?= =?us-ascii?q?Q2BZCKFbQKBGzgUAQEBAQEBAYEKhDQBAQEDAScTPwwEAgEIEQMBAQEBHgkHMhQ?= =?us-ascii?q?JCAIEAQ0FCIgfCA7AZQEBAQEBAQEBAQEBAQEBAQEBAQEBARQEhlaEf4QlIoR1B?= =?us-ascii?q?YdejygBhT+ICoFjhEaIWUSKA4NyASABAUKECnKDR0EBgQcBAQE?= X-IronPort-AV: E=Sophos;i="5.20,520,1444694400"; d="scan'208";a="60548843" Received: from alln-core-2.cisco.com ([173.36.13.135]) by rcdn-iport-6.cisco.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 04 Jan 2016 14:43:44 +0000 Received: from XCH-RTP-001.cisco.com (xch-rtp-001.cisco.com [64.101.220.141]) by alln-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id u04EhhKG028224 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Mon, 4 Jan 2016 14:43:44 GMT Received: from xch-rtp-017.cisco.com (64.101.220.157) by XCH-RTP-001.cisco.com (64.101.220.141) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Mon, 4 Jan 2016 09:43:42 -0500 Received: from xch-rtp-017.cisco.com ([64.101.220.157]) by XCH-RTP-017.cisco.com ([64.101.220.157]) with mapi id 15.00.1104.009; Mon, 4 Jan 2016 09:43:42 -0500 From: "Hanoch Haim (hhaim)" To: Olivier MATZ , "bruce.richardson@intel.com" Thread-Topic: [dpdk-dev] [PATCH v2] mbuf: optimize rte_mbuf_refcnt_update Thread-Index: AQHRRvdSBdlb5jcirkKWcHjQnHrDiJ7ragvg Date: Mon, 4 Jan 2016 14:43:42 +0000 Message-ID: <7f5255b98dcb4f2396ada16d5eb43e5a@XCH-RTP-017.cisco.com> References: <568A7959.7030506@6wind.com> In-Reply-To: <568A7959.7030506@6wind.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [64.103.60.135] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dev@dpdk.org" , "Ido Barnea \(ibarnea\)" , "Itay Marom \(imarom\)" Subject: Re: [dpdk-dev] [PATCH v2] mbuf: optimize rte_mbuf_refcnt_update X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2016 14:43:45 -0000 Hi Oliver,=20 Let's take your drawing as a reference and add my question The use case is sending a duplicate multicast packet by many threads. I can split it to x threads to do the job and with atomic-ref (my multicast= not mbuf) count it until it reaches zero. In my following example the two cores (0 and 1) sending the indirect m1/m2 = do alloc/attach/send=20 core0 | core1 --------------------------------- |----------------= ----------------------- m_const=3Drte_pktmbuf_alloc(mp) | | while true: | while True: m1 =3Drte_pktmbuf_alloc(mp_64) | m2 =3Drte_pktmbuf_alloc(m= p_64) rte_pktmbuf_attach(m1, m_const) | rte_pktmbuf_attach(m1, m_con= st) tx_burst(m1) | tx_burst(m2) Is this example is not valid?=20 BTW this is our workaround=20 =20 core0 | core1 --------------------------------- |-----------------------= ---------------- m_const=3Drte_pktmbuf_alloc(mp) | rte_mbuf_refcnt_update(m_const,1)| <<-- workaround=20 | while true: | while True: m1 =3Drte_pktmbuf_alloc(mp_64) | m2 =3Drte_pktmbuf_alloc(mp_64) rte_pktmbuf_attach(m1, m_const) | rte_pktmbuf_attach(m1, m_const) tx_burst(m1) | tx_burst(m2) thanks, Hanoh -----Original Message----- From: Olivier MATZ [mailto:olivier.matz@6wind.com]=20 Sent: Monday, January 04, 2016 3:53 PM To: Hanoch Haim (hhaim); bruce.richardson@intel.com Cc: dev@dpdk.org; Ido Barnea (ibarnea); Itay Marom (imarom) Subject: Re: [dpdk-dev] [PATCH v2] mbuf: optimize rte_mbuf_refcnt_update Hi Hanoch, Please find some comments below. On 12/27/2015 10:39 AM, Hanoch Haim (hhaim) wrote: > Hi Bruce, >=20 > I'm Hanoch from Cisco Systems works for the https://github.com/cisco-sys= tem-traffic-generator/trex-core traffic generator project. >=20 > While upgrading from DPDK 1.8 to 2.2 Ido found that the following=20 > commit creates a mbuf corruption and result in Tx hang >=20 >=20 >=20 > commit f20b50b946da9070d21e392e4dbc7d9f68bc983e >=20 > Author: Olivier Matz >=20 > Date: Mon Jun 8 16:57:22 2015 +0200 >=20 >=20 >=20 > Looking at the change it is clear why there is an issue, wanted to get yo= ur input. >=20 >=20 >=20 > Init >=20 > ----- >=20 > alloc const mbuf =3D=3D> mbuf-a (ref=3D1) >=20 >=20 >=20 > Simple case that works >=20 > --------------------- >=20 >=20 >=20 > thread 1 , tx: alloc-mbuf->attach(mbuf-a) (ref=3D2) inc- non atomic >=20 > thread 1 , tx: alloc-mbuf->attach(mbuf-a) (ref32) inc- atomic do you mean "(ref=3D3)" ? [hh] yes ref=3D3.=20 >=20 > thread 1 , drv : free() (ref=3D2) dec- atomic >=20 > thread 1 , drv : free() (ref=3D3) dec - non atomic do you mean "(ref=3D1)" ? >=20 > Simple case that does not work >=20 > --------------------- >=20 >=20 >=20 > Both do that in parallel >=20 >=20 >=20 > thread 2 tx : alloc-mbuf->attach(mbuf-a) (ref=3D2) inc- non atomic >=20 > thread 1 tx : alloc-mbuf->attach(mbuf-a) (ref=3D2) inc- non atomic It is not allowed to call a function from the mbuf API in parallel. Example: core0 | core1 --------------------------------|--------------------------------------- m =3D rte_pktmbuf_alloc(m); | enqueue(m); | |m =3D dequeue(); do_something(m); |do_something(m); do_something() is not allowed because it accesses the same mbuf structure. do_something() can be any function of mbuf API: rte_pktmbuf_prepend(), rte_= pktmbuf_attach(), ... This is allowed: core0 | core1 --------------------------------|--------------------------------------- m =3D rte_pktmbuf_alloc(m); | m2 =3D rte_pktmbuf_attach(m); | enqueue(m2); | |m2 =3D dequeue(); do_something(m); |do_something(m2); Regards, Olivier