From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) by dpdk.org (Postfix) with ESMTP id 6F85F9A for ; Sat, 18 Jan 2014 14:09:01 +0100 (CET) Received: by mail-wi0-f170.google.com with SMTP id ex4so3007241wid.1 for ; Sat, 18 Jan 2014 05:10:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:to:subject:date:mime-version:content-type :importance; bh=XCjStvGjWTb92Zclzl+fJ9s01a/sJoCgyQDJp7iwM94=; b=kf1ClVh+B4rGcofQsJPUmccnYwNRUxQnCKk7evroe3qN8m7rLVWm+2nzzF74oOEUNj LgFDC7kYA3bvTax9rnLnW2SmAnYY1iKq+s/fSI4cTv9c9i5smO9+TN6kIEz4iiHtgz0D pFvwNBJrLnbG+YAkuIKq3+tIPwxOkX1MPt1jZzPC1HEPS0E89u5Qz1KZl+UHy2kalySF u0+NnYS53wfPtk9VxvGwVJ+7BZDbyXQDRHCg+pr/+LBlQhof87Dnjh99GWDbAbcAQ4Zo mgjPFmgORyA+I0l8CPajIj4HbyMUTamTo6KH3U+ycIsNxTPJ6CR4nFMfGvGvlHnP6xpg pjbQ== X-Received: by 10.180.208.79 with SMTP id mc15mr11813wic.14.1390050616234; Sat, 18 Jan 2014 05:10:16 -0800 (PST) Received: from CARLOSPC ([195.57.230.177]) by mx.google.com with ESMTPSA id n3sm2018698wix.10.2014.01.18.05.10.15 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 18 Jan 2014 05:10:15 -0800 (PST) Message-ID: <2AF1C8858EE34CAF98E426C4B8132229@aervox.com> From: "Carlos Franco" To: Date: Sat, 18 Jan 2014 14:10:14 +0100 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal Importance: Normal X-Mailer: Microsoft Windows Live Mail 16.4.3505.912 X-MimeOLE: Produced By Microsoft MimeOLE V16.4.3505.912 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] Preemption in rings X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Jan 2014 13:09:01 -0000 Hello I have read the documentation and the answer of Olivier about a question = to clarify the preemption restriction in the rings. He wrote: - a pthread doing multi-producers enqueues on a given ring must not be = preempted by another pthread doing a multi-producer enqueue on the same = ring. - a pthread doing multi-consumers dequeues on a given ring must not be = preempted by another pthread doing a multi-consumer dequeue on the same = ring. I suppose this is a something to be taken into account with threads not = created by the DPDK rte, because the threads of DPDK are attached to a = core and will never be scheduled into the cores of other DPDK threads, = no? I mean, lets only take into account the enqueuing. if there is a ring R = where ONLY the DPDK threads enqueue packets, even if they are = interrupted (IO access, sleep, mutex... ), and other non DPDK threads = are scheduled into their cores, as long as only DPDK threads enqueue in = the ring it is safe, no? If not, would prohibit the access to the DPDK used cores (isocpus) work? Thanks Carlos >From olivier.matz@6wind.com Mon Jan 20 14:08:44 2014 Return-Path: Received: from mail.droids-corp.org (zoll.droids-corp.org [94.23.50.67]) by dpdk.org (Postfix) with ESMTP id 0D76B58D2 for ; Mon, 20 Jan 2014 14:08:44 +0100 (CET) Received: from was59-1-82-226-113-214.fbx.proxad.net ([82.226.113.214] helo=[192.168.0.10]) by mail.droids-corp.org with esmtpsa (TLS1.0:DHE_RSA_CAMELLIA_256_CBC_SHA1:256) (Exim 4.80) (envelope-from ) id 1W5Ecj-0005ha-KV; Mon, 20 Jan 2014 14:10:44 +0100 Message-ID: <52DD20F3.8070801@6wind.com> Date: Mon, 20 Jan 2014 14:13:23 +0100 From: Olivier MATZ User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20131005 Icedove/17.0.9 MIME-Version: 1.0 To: Carlos Franco References: <2AF1C8858EE34CAF98E426C4B8132229@aervox.com> In-Reply-To: <2AF1C8858EE34CAF98E426C4B8132229@aervox.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: dev@dpdk.org Subject: Re: [dpdk-dev] Preemption in rings X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 13:08:44 -0000 Hi Carlos, On 01/18/2014 02:10 PM, Carlos Franco wrote: > - a pthread doing multi-producers enqueues on a given ring must > not be preempted by another pthread doing a multi-producer > enqueue on the same ring. > > - a pthread doing multi-consumers dequeues on a given ring must > not be preempted by another pthread doing a multi-consumer > dequeue on the same ring. > > I suppose this is a something to be taken into account with > threads not created by the DPDK rte, because the threads of DPDK > are attached to a core and will never be scheduled into the cores > of other DPDK threads, no? Yes. > I mean, lets only take into account the enqueuing. if there is a > ring R where ONLY the DPDK threads enqueue packets, even if they > are interrupted (IO access, sleep, mutex... ), and other non DPDK > threads are scheduled into their cores, as long as only DPDK > threads enqueue in the ring it is safe, no? It's safe, but it may cause performance issue. For instance, if a DPDK pthread is interrupted by the kernel during a critical period of an enqueue, all other DPDK pthreads enqueueing on this ring will block. Of course, this could also happen with a rte_spinlock(). > If not, would prohibit the access to the DPDK used > cores (isocpus) work? Yes, using "isolcpu" is a good idea to avoid the problem described above. From my experience, using a simple "taskset" to force the other applications to run on different cores than the DPDK is enough to avoid significant performance issues. Indeed, the only problem would be related to kernel work (interruptions and kernel threads), and they shouldn't last long enough to fill the queues. Regards, Olivier