From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 94DB2A0524; Mon, 24 Feb 2020 19:00:17 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 63A911BE85; Mon, 24 Feb 2020 19:00:17 +0100 (CET) Received: from mail-io1-f65.google.com (mail-io1-f65.google.com [209.85.166.65]) by dpdk.org (Postfix) with ESMTP id 1499F2C39 for ; Mon, 24 Feb 2020 19:00:16 +0100 (CET) Received: by mail-io1-f65.google.com with SMTP id x1so11186509iop.7 for ; Mon, 24 Feb 2020 10:00:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=0G26MNdTsJqFIzFVV81ZJzRrDGVvk+Fnl0S3BD+/xU8=; b=J67xijqxEMRD5/7FXMHYOTxZB2BNS6hu929zRaM7I76xqwk0/+/acjcn/9gcLPtlSv 2qe0hcg4eaCfI0S0GxnCfX9dh0Yf9J3kdE45QYms6+9U12TLEMugzVT9NhAn3jWKdHX2 pvDaxILBW7Fyb25IYmLiF3rvGJStsUZF8dvl8UpJjcPgKNYq/lhWxZ1U4neWC3uD9nC+ T7T77wxB//7vGXdUe5mQH+QdNHS9mIYNMSMUJKEbH6/FnWCRWYK4Rg+PEW1l6gA13Ut9 scMD7tSR/b58skr1gLsAjjIvd5RTZoO1uZpYAL9uVDW98iljlh2WKynz8RaTBqw0CgzZ lGdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=0G26MNdTsJqFIzFVV81ZJzRrDGVvk+Fnl0S3BD+/xU8=; b=hak5JrA12w0dIj4A5cnNpyW9sozAqzQ3tnF5Ug4tRIyMsUzjZtOGE9UdWju5k0gL6h MahV77B62Cz/14BQZQgUwF9gEGYfSj9rJ8U0jsGYr5BHqQLU4b23Y0pxPbwjtgq31tMY Yc1qWFzH1pKjkGzG2UZjfVDdItraVxGxT+owkaddr37EXOpHpieIo4zoQI2DoaTYXAPz GjQXRAsKcth8E8Qcz8yTgq36PiTjBNkPaA3IZbxW5TkVk/6w9DFqFflra0kHWlABMSjz FHampAKoNnOmCIu1CrYxHDahdgHeYSpEDbjS52PtbLq9jM3PT1WFBLlJ0laYzMCEs+SH K/Yg== X-Gm-Message-State: APjAAAW7e5oGh0GaMVQNDlAq0MSHntBql476KNL2jsMJ1+ZJFJKBBjn/ ODoopKKB8XVXydkjxYseEX16TASS4cyLJc0HflQ= X-Google-Smtp-Source: APXvYqxa+MMzR07nlaJJrHGxdfIVjSeg2UYjmokFN3Cb8Dx5SYddk9bmFn5X3sQl1oSPif0TSoj4ZlWNN21XKYpd5PU= X-Received: by 2002:a6b:39c4:: with SMTP id g187mr52012913ioa.271.1582567214909; Mon, 24 Feb 2020 10:00:14 -0800 (PST) MIME-Version: 1.0 References: <20200224113515.1744-1-konstantin.ananyev@intel.com> <20200224085919.3e73fda7@hermes.lan> In-Reply-To: <20200224085919.3e73fda7@hermes.lan> From: Jerin Jacob Date: Mon, 24 Feb 2020 23:29:57 +0530 Message-ID: To: Stephen Hemminger Cc: Konstantin Ananyev , dpdk-dev , Olivier Matz Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [RFC 0/6] New sync modes for ring X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Mon, Feb 24, 2020 at 10:29 PM Stephen Hemminger wrote: > > On Mon, 24 Feb 2020 11:35:09 +0000 > Konstantin Ananyev wrote: > > > Upfront note - that RFC is not a complete patch. > > It introduces an ABI breakage, plus it doesn't update ring_elem > > code properly, etc. > > I plan to deal with all these things in later versions. > > Right now I seek an initial feedback about proposed ideas. > > Would also ask people to repeat performance tests (see below) > > on their platforms to confirm the impact. > > > > More and more customers use(/try to use) DPDK based apps within > > overcommitted systems (multiple acttive threads over same pysical cores): > > VM, container deployments, etc. > > One quite common problem they hit: Lock-Holder-Preemption with rte_ring. > > LHP is quite a common problem for spin-based sync primitives > > (spin-locks, etc.) on overcommitted systems. > > The situation gets much worse when some sort of > > fair-locking technique is used (ticket-lock, etc.). > > As now not only lock-owner but also lock-waiters scheduling > > order matters a lot. > > This is a well-known problem for kernel within VMs: > > http://www-archive.xenproject.org/files/xensummitboston08/LHP.pdf > > https://www.cs.hs-rm.de/~kaiser/events/wamos2017/Slides/selcuk.pdf > > The problem with rte_ring is that while head accusion is sort of > > un-fair locking, waiting on tail is very similar to ticket lock schema - > > tail has to be updated in particular order. > > That makes current rte_ring implementation to perform > > really pure on some overcommited scenarios. > > Rather than reform rte_ring to fit this scenario, it would make > more sense to me to introduce another primitive. The current lockless > ring performs very well for the isolated thread model that DPDK > was built around. This looks like a case of customers violating > the usage model of the DPDK and then being surprised at the fallout. I agree with Stephen here. I think, adding more runtime check in the enqueue() and dequeue() will have a bad effect on the low-end cores too. But I agree with the problem statement that in the virtualization use case, It may be possible to have N virtual cores runs on a physical core. IMO, The best solution would be keeping the ring API same and have a different flavor in "compile-time". Something like liburcu did for accommodating different flavors. i.e urcu-qsbr.h and urcu-bp.h will identical definition of API. The application can simply include ONE header file in a C file based on the flavor. If need both at runtime. Need to have function pointer or so in the application and define the function in different c file by including the approaite flavor in C file. #include /* QSBR RCU flavor */ #include /* Bulletproof RCU flavor */ >