From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <stephen@networkplumber.org>
Received: from mail-pa0-f41.google.com (mail-pa0-f41.google.com
 [209.85.220.41]) by dpdk.org (Postfix) with ESMTP id D89B658DD
 for <dev@dpdk.org>; Thu, 27 Mar 2014 20:04:50 +0100 (CET)
Received: by mail-pa0-f41.google.com with SMTP id fa1so3920279pad.0
 for <dev@dpdk.org>; Thu, 27 Mar 2014 12:06:24 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20130820;
 h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to
 :references:mime-version:content-type:content-transfer-encoding;
 bh=p1Gdt9+3YD7kf8Jwejmoh9suUa8kSEtFdVS3LBQ1qQU=;
 b=ZBNQvHbfRqL6dXBl7E8dOQ+8E31D8EaRvodPFPzw6fuXfrT4JakHdRRDkfrycE3/6P
 2PipzLHhL67XZZUoqA5dolnIGJCpLro8Q8r6tnWOGtQuVAoVapdQd29ABNgsgn9fMUdB
 woYpy1sPmSYd569I9w9afO+xYIBegrVjibEl1dP9dT7LoHnJOgTIuMJQ+Nb3LMIatNsa
 8i0/g48bMDcTwQfU/OvgQhokFc0f62BWoMg+AZW/a0aaSE+zfgOtopvKxwP4wpZy1FsR
 sHh0otMg2K8FsSyDkgVldsR4sIUl8Zw78w8m3xiZ6/SRfOklRaG9BkWSkLSenwPWd9WR
 jbrQ==
X-Gm-Message-State: ALoCoQlKCIlo80LWkOyFBG4hDcLhg9n/epOgVGn/BGJ/4Tc04NbK8tI1GkuOrfc8ubZRtWfQxL9i
X-Received: by 10.66.27.48 with SMTP id q16mr3678977pag.9.1395947183663;
 Thu, 27 Mar 2014 12:06:23 -0700 (PDT)
Received: from samsung-9 ([12.232.193.53])
 by mx.google.com with ESMTPSA id db3sm12471666pbb.10.2014.03.27.12.06.22
 for <multiple recipients>
 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Thu, 27 Mar 2014 12:06:23 -0700 (PDT)
Date: Thu, 27 Mar 2014 12:06:20 -0700
From: Stephen Hemminger <stephen@networkplumber.org>
To: Olivier MATZ <olivier.matz@6wind.com>
Message-ID: <20140327120620.07f1496b@samsung-9>
In-Reply-To: <53345655.9030907@6wind.com>
References: <53345655.9030907@6wind.com>
X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; x86_64-redhat-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] memory barriers in rte_ring
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Thu, 27 Mar 2014 19:04:52 -0000

On Thu, 27 Mar 2014 17:48:21 +0100
Olivier MATZ <olivier.matz@6wind.com> wrote:

> Hi,
> 
> The commit 286bd05bf7 [1] removed the memory barriers in the ring
> functions. This patch is present in DPDK since version 1.4.0r0, so I
> guess it does not cause any issue.
> 
> But after checking the excellent Linux kernel documentation about memory
> barriers [2], I'm wondering why memory barriers would not be required in
> that case.
> 
> To illustrate the previous behavior (before dpdk 1.4):
> 
>    ring_enqueue()
>      - move producer_head to reserve space in ring (atomically if
>        multi producers)
>      - write objects between producer_head and producer_tail
>      - wmb() to ensure that STORE operations are issued
>      - write producer_tail
> 
>    ring_dequeue()
>      - move consumer_head (atomically if multi consumers)
>      - rmb() to ensure that LOAD operations are issued: the read of
>        consumer_head must occur before the reading of objects ptrs.
>        In fact, rmb() is probably not needed here because knowing the
>        value of consumer_head is required before reading the objects
>        table.
>      - read objects between consumer_head and consumer_tail
>      - write consumer_tail
> 
> The memory barriers have been removed, but in my understanding at least
> the wmb() would be needed according to the generic memory barrier
> documentation. Maybe this is not needed on newest Intel processors?
> Could anyone from Intel enlight me on this?
> 
> Thanks & regards,
> Olivier
> 
> 
> [1] 
> http://dpdk.org/browse/dpdk/commit/lib/librte_ring/rte_ring.h?id=286bd05bf70d1da1b6017007276c267a1e012c1d
> 
> [2] http://lxr.free-electrons.com/source/Documentation/memory-barriers.txt

Short answer, only a compiler barrier is necessary.

Long answer: for the multple CPU access ring, it is equivalent to smp_wmb and smp_rmb
 in Linux kernel. For x86 where DPDK is used, this can normally be replaced by simpler
 compiler barrier. In kernel there is a special flage X86_OOSTORE which is only enabled
 for a few special cases, for most cases it is not. When cpu doesnt do out of order
 stores, there are no cases where other cpu will see wrong state.