From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 29FD1A0524; Mon, 24 Feb 2020 20:35:41 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 57C371BE85; Mon, 24 Feb 2020 20:35:40 +0100 (CET) Received: from mail-pl1-f195.google.com (mail-pl1-f195.google.com [209.85.214.195]) by dpdk.org (Postfix) with ESMTP id B0E621F1C for ; Mon, 24 Feb 2020 20:35:38 +0100 (CET) Received: by mail-pl1-f195.google.com with SMTP id j7so4462354plt.1 for ; Mon, 24 Feb 2020 11:35:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FDNZdmyy0FZrRedv4FmUnZ4Z3lRgsuEWOPBdCPZEI4s=; b=nF8iqq0ur1JpyBp8mMadZmqpzcIv1/snn2IdjpUgd0GFZzry56cIweQExsNtCD2kSj GSGa+e6iKTbWRahcRZ/TXPYhsJ3JAPIHBH+ocfUm6hTbdhv+YXE4MDN8dHPzhDU7XePz sAEqQ52/184cA9xkyRl+S+VrYhMqWw+eRoNPedcpiH7i170mZDxQWeHfUI/TwdMPIpb8 QXZC5AG5w+r6QS1oaZ09Cag4aMrbmyGJOWhXQiwSw35/Eb+akfstDor5YqoUxnGux7wm V1Pw0X7KWSe6MqDoM1IkBtQZsrow1NnukMQorM3Zq9RrwMzridTWxCwO8635gj4qgl6D VoEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FDNZdmyy0FZrRedv4FmUnZ4Z3lRgsuEWOPBdCPZEI4s=; b=BNa0Yn9uyAlkAJBMCBenRWfW3ga5th6lLJ45Hp7suZ5fRRtgNf3VW4gelcaCrnJFKb bepRU4x+2jADXk6wOH05rKIH9ZzFJDEaxlTLmNMYq9KGLVXtzpTThXKV8sy//Ag+1VpU lLq5zh3Xk/UZ+8SjnB84sOceIcFqrZXA9An6o/hi4FcQvdrUS88j35MYfr6PhlrfbZ4g mAGs9rEriNKg0ZQ+IGTHvju54mlrp3wCREfMsXCaM0+hfRoXzg4sG54LbI+CZpGc7orw fZ0Lvn8T2Yml/jyhWE9wnfW+Q7k3wrPRTYNpI6j9RNAVfi/vYQJ54plcA8oTxHO8HGZy AUgw== X-Gm-Message-State: APjAAAV8mzSY14h+ezyZ0GfulaqwDsZrfpNfr4x/gLI8KaFa/rF6hsEB qVbNiuGdfkgtCWC4lTh1TpOBaw== X-Google-Smtp-Source: APXvYqzB91AWjEA/LqWL/kBhAs/yMBZnPr/gof4qDo1r070b9l+aq1ssc7QHYV3Bbb91TZXQLSJy8w== X-Received: by 2002:a17:902:9b8a:: with SMTP id y10mr48777356plp.114.1582572937671; Mon, 24 Feb 2020 11:35:37 -0800 (PST) Received: from hermes.lan (204-195-22-127.wavecable.com. [204.195.22.127]) by smtp.gmail.com with ESMTPSA id b18sm13975415pfd.63.2020.02.24.11.35.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2020 11:35:37 -0800 (PST) Date: Mon, 24 Feb 2020 11:35:29 -0800 From: Stephen Hemminger To: Jerin Jacob Cc: Konstantin Ananyev , dpdk-dev , Olivier Matz Message-ID: <20200224113529.4c1c94ab@hermes.lan> In-Reply-To: References: <20200224113515.1744-1-konstantin.ananyev@intel.com> <20200224085919.3e73fda7@hermes.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [RFC 0/6] New sync modes for ring X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Mon, 24 Feb 2020 23:29:57 +0530 Jerin Jacob wrote: > On Mon, Feb 24, 2020 at 10:29 PM Stephen Hemminger > wrote: > > > > On Mon, 24 Feb 2020 11:35:09 +0000 > > Konstantin Ananyev wrote: > > > > > Upfront note - that RFC is not a complete patch. > > > It introduces an ABI breakage, plus it doesn't update ring_elem > > > code properly, etc. > > > I plan to deal with all these things in later versions. > > > Right now I seek an initial feedback about proposed ideas. > > > Would also ask people to repeat performance tests (see below) > > > on their platforms to confirm the impact. > > > > > > More and more customers use(/try to use) DPDK based apps within > > > overcommitted systems (multiple acttive threads over same pysical cores): > > > VM, container deployments, etc. > > > One quite common problem they hit: Lock-Holder-Preemption with rte_ring. > > > LHP is quite a common problem for spin-based sync primitives > > > (spin-locks, etc.) on overcommitted systems. > > > The situation gets much worse when some sort of > > > fair-locking technique is used (ticket-lock, etc.). > > > As now not only lock-owner but also lock-waiters scheduling > > > order matters a lot. > > > This is a well-known problem for kernel within VMs: > > > http://www-archive.xenproject.org/files/xensummitboston08/LHP.pdf > > > https://www.cs.hs-rm.de/~kaiser/events/wamos2017/Slides/selcuk.pdf > > > The problem with rte_ring is that while head accusion is sort of > > > un-fair locking, waiting on tail is very similar to ticket lock schema - > > > tail has to be updated in particular order. > > > That makes current rte_ring implementation to perform > > > really pure on some overcommited scenarios. > > > > Rather than reform rte_ring to fit this scenario, it would make > > more sense to me to introduce another primitive. The current lockless > > ring performs very well for the isolated thread model that DPDK > > was built around. This looks like a case of customers violating > > the usage model of the DPDK and then being surprised at the fallout. > > I agree with Stephen here. > > I think, adding more runtime check in the enqueue() and dequeue() will > have a bad effect on the low-end cores too. > But I agree with the problem statement that in the virtualization use > case, It may be possible to have N virtual cores runs on a physical > core. > > IMO, The best solution would be keeping the ring API same and have a > different flavor in "compile-time". Something like > liburcu did for accommodating different flavors. > > i.e urcu-qsbr.h and urcu-bp.h will identical definition of API. The > application can simply include ONE header file in a C file based on > the flavor. > If need both at runtime. Need to have function pointer or so in the > application and define the function in different c file by including > the approaite flavor in C file. This would also be a good time to consider the tradeoffs of the heavy use of inlining that is done in rte_ring vs the impact that has on API/ABI stability.