From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-f176.google.com (mail-wr0-f176.google.com [209.85.128.176]) by dpdk.org (Postfix) with ESMTP id DC7B239EA for ; Thu, 16 Feb 2017 14:48:10 +0100 (CET) Received: by mail-wr0-f176.google.com with SMTP id z61so11985650wrc.1 for ; Thu, 16 Feb 2017 05:48:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pvK8uOcva9IEYzhzEjuPpr8QLpbJ+cMlKhb03Gkn6Wk=; b=C4Gsw6z5WFZKDJg7kIsHzNIK6PvhJsF2L1Q+9TecuPjyetbxY60QmRYpqhb/q4wNe5 js1QcXCZIPJBAuGF49cpU0Z4GQ5CPunwjfUExwW2ByWNc3y9ghgHVarWBBVWZI0iV0GV GYihaK/NM+TzjHrZrBZGIvW8yiN9iCeUzGZ2etrlM3htyftbrYe0vJ20plZGY4gL99Lu bODATE1c+ijBTF1/mR5q1zPc1fpEvvncJJkrT2o+ohelxKfuhZYVzNP5pFgqVs3FJ+3s rJmlQKTCutqkv0y9RPCkVWrbh6fH59G6je/2mEH2TUZ/ELeTyWHEGMJ0jrjvcY2W9bHd mgQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pvK8uOcva9IEYzhzEjuPpr8QLpbJ+cMlKhb03Gkn6Wk=; b=Wi+tKWNbG2mlGPx/FCjF5bU81FT5xUDakAwI268krytIdzv6Ty9Qml2WLYpjYPmH4K n5U7lW8eInfPqBYBRUvUlhevd4Rr4KDOdgpnVx3w1halVul30gnMUkyAZXilv7l2dMUR tS30j67RAVWxG06KH+JaFJKAM2cX0RLzAEgFEHpoz8OYzfhavwSgX0dGLIP78GfbWwWu AlezInGmUa4vt6hygdpzX4B1JDKQmTpnKRHpm5BeQHLfNVRrvSx5tNSFiBBIar3oSitV NFp6djFpWSuBeBOXANXRoUq5OgjjjVNz5bP8kFNsdUkjcsZYTI6nVLZEmTiImQgoZ8Zb OIRA== X-Gm-Message-State: AMke39nZHvzC1O+hNJZb7xcTtWjssHjjSKqCTI5HidpJBKmzjDbx5NJQhxL24ZZoOA7cXDg9 X-Received: by 10.223.174.183 with SMTP id y52mr2564447wrc.112.1487252890390; Thu, 16 Feb 2017 05:48:10 -0800 (PST) Received: from platinum (2a01cb0c03c651000226b0fffeed02fc.ipv6.abo.wanadoo.fr. [2a01:cb0c:3c6:5100:226:b0ff:feed:2fc]) by smtp.gmail.com with ESMTPSA id 39sm9052228wrv.55.2017.02.16.05.48.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 16 Feb 2017 05:48:10 -0800 (PST) Date: Thu, 16 Feb 2017 14:48:07 +0100 From: Olivier Matz To: "Ananyev, Konstantin" Cc: "dev@dpdk.org" Message-ID: <20170216144807.7add2c71@platinum> In-Reply-To: <2601191342CEEE43887BDE71AB9772583F111A29@irsmsx105.ger.corp.intel.com> References: <1485271173-13408-1-git-send-email-olivier.matz@6wind.com> <2601191342CEEE43887BDE71AB9772583F111A29@irsmsx105.ger.corp.intel.com> X-Mailer: Claws Mail 3.14.1 (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [RFC 0/8] mbuf: structure reorganization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Feb 2017 13:48:11 -0000 Hi Konstantin, Thanks for the feedback. Comments inline. On Mon, 6 Feb 2017 18:41:27 +0000, "Ananyev, Konstantin" wrote: > Hi Olivier, > Looks good in general, some comments from me below. > Thanks > Konstantin > > > > > The main changes are: > > - reorder structure to increase vector performance on some non-ia > > platforms. > > - add a 64bits timestamp field in the 1st cache line > > Wonder why it deserves to be in first cache line? > How it differs from seqn below (pure SW stuff right now). In case the timestamp is set from a NIC value, it is set in the Rx path. So that's why I think it deserve to be located in the 1st cache line. As you said, the seqn is a pure sw stuff right: it is set in a lib, not in a PMD rx path. > > - m->next, m->nb_segs, and m->refcnt are always initialized for > > mbufs in the pool, avoiding the need of setting m->next (located in > > the 2nd cache line) in the Rx path for mono-segment packets. > > - change port and nb_segs to 16 bits > > Not that I am completely against it, > but changing nb_segs to 16 bits seems like an overkill to me. > I think we can keep and extra 8bits for something more useful in > future. In my case, I use the m->next field to chain more than 256 segments for L4 socket buffers. It also updates nb_seg that can overflow. It's not a big issue since at the end, nb_seg is decremented for each segment. On the other hand, if I enable some sanity checks on mbufs, it complains because the number of segments is not equal to nb_seg. There is also another use case with fragmentation as discussed recently: http://dpdk.org/dev/patchwork/patch/19819/ Of course, dealing with a long mbuf list is not that efficient, but the application can maintain another structure to accelerate the access to the middle/end of the list. Finally, we have other ideas to get additional 8 bits if required in the future, so I don't think it's really a problem. > > > - move seqn in the 2nd cache line > > > > Things discussed but not done in the patchset: > > - move refcnt and nb_segs to the 2nd cache line: many drivers sets > > them in the Rx path, so it could introduce a performance > > regression, or > > I wonder can refcnt only be moved into the 2-nd cacheline? > As I understand thanks to other change (from above) m->refcnt > will already be initialized, so RX code don't need to touch it. > Though yes, it still would require changes in all PMDs. Yes, I agree, some fields could be moved in the 2nd cache line once all PMDs stop to write them in RX path. I propose to issue some guidelines to PMD maintainers at the same time the patchset is pushed. Then we can consider changing it in a future version, in case we need more room in the 1st mbuf cache line. > > > it would require to change all the drivers, which is not an easy > > task. > > - remove the m->port field: too much impact on many examples and > > libraries, and some people highlighted they are using it. > > Ok, but can it be moved into the second cache-line? I think no: it is set by the PMDs in RX path, it would impact performance. > > > - moving m->next in the 1st cache line: there is not enough room, > > and having it set to NULL for unused mbuf should remove the need > > for it. > > - merge seqn and timestamp together in a union: we could imagine > > use cases were both are activated. There is no flag indicating the > > presence of seqn, so it looks preferable to keep them separated for > > now. > > > > I made some basic performance tests (ixgbe) and see no regression, > > but the patchset requires more testing. > > > > [1] http://dpdk.org/ml/archives/dev/2016-October/049338.html > > By the way, additional performance tests on this patchset from PMD vendors would be helpful. Olivier