From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f169.google.com (mail-wi0-f169.google.com [209.85.212.169]) by dpdk.org (Postfix) with ESMTP id 60E123F9 for ; Tue, 7 Jan 2014 15:26:35 +0100 (CET) Received: by mail-wi0-f169.google.com with SMTP id j9so4078434wiv.2 for ; Tue, 07 Jan 2014 06:27:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=oUyHtXa+xJQyD84ow1i0E6KWZlOkSRL+XSPYBKyblgU=; b=Gm6Tct3jab2WWDXMTyyak4jc2N7VeWMl4TGuatSpzGeEb1kRNM09VWpiyebr8dzJ3o ho7TvcZyZgjPDNkq0Vp/oqIRZpMccNzHlH3z3nBViRbhPb7AAxjJBa1wnYORL64dvubD KcYv1ZYxAr+pcC+oADvj+qq+JOiQvTc478LSjBVInVRbsBTsJC9k9pT1nAMVlHqzKMkB dj4XPA0VQS6LPv4vGLzH9PKvVEGuFv5IphusIGBTP6AdhyqNzSsveqc6O2UB0EWKLlxz /+wLDmgP5jB33sEuVlpgWhyb/+u++HZ+C0l+FL7CTgHAYqD4Kl95zP2wVmFXCvS42C2G OE/Q== X-Gm-Message-State: ALoCoQk8TTtBzgPswdVqVD0mYMhJ0bZC2SxKk4sPqU/EQQlPwku9+/1bz6IW2ErN+33/XvmDOUV5 X-Received: by 10.180.188.141 with SMTP id ga13mr16919758wic.55.1389104866918; Tue, 07 Jan 2014 06:27:46 -0800 (PST) Received: from [10.16.0.195] (6wind.net2.nerim.net. [213.41.180.237]) by mx.google.com with ESMTPSA id hv3sm3819770wib.5.2014.01.07.06.27.45 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 07 Jan 2014 06:27:46 -0800 (PST) Message-ID: <52CC0EE0.60404@6wind.com> Date: Tue, 07 Jan 2014 15:27:44 +0100 From: Olivier MATZ User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20131005 Icedove/17.0.9 MIME-Version: 1.0 To: Jyotiswarup Raiturkar References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] rte_mempools / rte_rings thread safe? X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 14:26:35 -0000 Hi Jyotiswarup, On 12/28/2013 03:55 PM, Jyotiswarup Raiturkar wrote: > The rte_mempool and rte_ring libs have > multi-producer/multi-consumer versions. But it's also mentioned > in the header files that the implementation is not > pre-emtable : " Note: the mempool implementation is not > preemptable. A lcore must not be interrupted by another task > that uses the same mempool (because it uses a ring which is not > preemptable)" Correct. > - Does having mutually exclusive core masks for a set of > threads which use the ring/mempool suffice for thread safety > (threads will have different core ids but they will not be > pinned to cores) ? As you noticed, and as explained in [1], one problem with the mempool is related to the cache structure, which is table indexed by the lcore_id. This cache structure is accessed without lock as it is supposed that each pthread has its own lcore_id. So having different lcore_ids would solve this problem even if the pthreads are not pinned to different cores. But this is not the only problem. One design paradigm of the dpdk is that the execution units are never interrupted. Let's imagine the following case, similar to what you describe: - 2 pthreads are running on the same cpu (taskset on with only one bit set in the mask). Their respective lcore_id (internal variable) are 0 and 1. - pthread 0 takes a spinlock - the kernel interrupts pthread 0, and schedules pthread 1 - pthread 1 wants to take the same spinlock, but it cannot be released by pthread 0 as it is not running - pthread 1 will loop using 100% of the cpu for nothing The same can occur in rte_ring. If a pthread is interrupted in an enqueue process, between lines 392 and 414 (see [2]), it would prevent any other pthread to enqueue another object. So, as you can see, doing this could lead to significant performance issues. [1] http://www.dpdk.org/ml/archives/dev/2013-December/001002.html [2] http://dpdk.org/browse/dpdk/tree/lib/librte_ring/rte_ring.h?id=142dfe1eedb215bd2a0762afcc65ef5a7fba10aa > - If i want to use this data structures in a pthread ( created > outside of DPDK environment), is it ok to use this if i > do "RTE_DEFINE_PER_LCORE(unsigned, _lcore_id);" with a core_id > exclusive of all other core masks for DPDK processes. The short answer is no. However, it is possible to use a rte_ring between an external process and a dpdk application at the following conditions: (on the external app, the pthreads must be pinned to a cpu OR on the external app the ring must be single consumer/producer) AND (external apps must do enqueues only and dpdk app dequeues only OR external apps must do dequeues only and dpdk app enqueues only) Regards, Olivier