From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <olivier.matz@6wind.com>
Received: from mail.droids-corp.org (zoll.droids-corp.org [94.23.50.67])
 by dpdk.org (Postfix) with ESMTP id C576F8E5E
 for <dev@dpdk.org>; Mon,  4 Jan 2016 14:53:37 +0100 (CET)
Received: from was59-1-82-226-113-214.fbx.proxad.net ([82.226.113.214]
 helo=[192.168.0.10])
 by mail.droids-corp.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.84) (envelope-from <olivier.matz@6wind.com>)
 id 1aG5an-0000zY-Cv; Mon, 04 Jan 2016 14:54:37 +0100
To: "Hanoch Haim (hhaim)" <hhaim@cisco.com>,
 "bruce.richardson@intel.com" <bruce.richardson@intel.com>
References: <d18f2062724a4453a1f709dcf4f30792@XCH-RTP-017.cisco.com>
From: Olivier MATZ <olivier.matz@6wind.com>
X-Enigmail-Draft-Status: N1110
Message-ID: <568A7959.7030506@6wind.com>
Date: Mon, 4 Jan 2016 14:53:29 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
 Icedove/38.3.0
MIME-Version: 1.0
In-Reply-To: <d18f2062724a4453a1f709dcf4f30792@XCH-RTP-017.cisco.com>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
Cc: "dev@dpdk.org" <dev@dpdk.org>, "Ido Barnea \(ibarnea\)" <ibarnea@cisco.com>,
 "Itay Marom \(imarom\)" <imarom@cisco.com>
Subject: Re: [dpdk-dev] [PATCH v2] mbuf: optimize rte_mbuf_refcnt_update
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Mon, 04 Jan 2016 13:53:37 -0000

Hi Hanoch,

Please find some comments below.

On 12/27/2015 10:39 AM, Hanoch Haim (hhaim) wrote:
> Hi Bruce,
> 
> I'm Hanoch from Cisco Systems works for  the https://github.com/cisco-system-traffic-generator/trex-core traffic generator project.
> 
> While upgrading from DPDK 1.8 to 2.2 Ido found that the following commit creates a mbuf corruption and result in Tx hang
> 
> 
> 
> commit f20b50b946da9070d21e392e4dbc7d9f68bc983e
> 
> Author: Olivier Matz <olivier.matz@6wind.com>
> 
> Date:   Mon Jun 8 16:57:22 2015 +0200
> 
> 
> 
> Looking at the change it is clear why there is an issue, wanted to get your input.
> 
> 
> 
> Init
> 
> -----
> 
> alloc const mbuf  ==> mbuf-a (ref=1)
> 
> 
> 
> Simple case that works
> 
> ---------------------
> 
> 
> 
> thread 1 , tx: alloc-mbuf->attach(mbuf-a) (ref=2)  inc- non atomic
> 
> thread 1 , tx: alloc-mbuf->attach(mbuf-a) (ref32)  inc- atomic

do you mean "(ref=3)" ?

> 
> thread 1 , drv : free()                    (ref=2) dec- atomic
> 
> thread 1 , drv : free()                    (ref=3) dec - non atomic

do you mean "(ref=1)" ?


> 
> Simple case that does not work
> 
> ---------------------
> 
> 
> 
> Both do that in parallel
> 
> 
> 
> thread 2 tx : alloc-mbuf->attach(mbuf-a)  (ref=2)  inc- non atomic
> 
> thread 1 tx : alloc-mbuf->attach(mbuf-a)  (ref=2)  inc- non atomic


It is not allowed to call a function from the mbuf API in parallel.

Example:

	core0			|	core1
--------------------------------|---------------------------------------
m = rte_pktmbuf_alloc(m);	|
enqueue(m);			|
				|m = dequeue();
do_something(m);		|do_something(m);


do_something() is not allowed because it accesses the same mbuf
structure.
do_something() can be any function of mbuf API: rte_pktmbuf_prepend(),
rte_pktmbuf_attach(), ...


This is allowed:

	core0			|	core1
--------------------------------|---------------------------------------
m = rte_pktmbuf_alloc(m);	|
m2 = rte_pktmbuf_attach(m);	|
enqueue(m2);			|
				|m2 = dequeue();
do_something(m);		|do_something(m2);



Regards,
Olivier