From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <olivier.matz@6wind.com> Received: from mail-wm0-f50.google.com (mail-wm0-f50.google.com [74.125.82.50]) by dpdk.org (Postfix) with ESMTP id 0D67D2A5B for <dev@dpdk.org>; Tue, 24 Jan 2017 17:16:47 +0100 (CET) Received: by mail-wm0-f50.google.com with SMTP id c85so189372900wmi.1 for <dev@dpdk.org>; Tue, 24 Jan 2017 08:16:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:date:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1Mr+zyj0+iYAxOsywyr4PXrRM0+BLEnzr0dqBSq/7Yw=; b=PZEuZ1+oF3Ma3l8cZpLBFcYS/dvOE4LJrCHKEzF7CD1mJrfKvQlbzASJqaKsG0oYB1 1NtJiomwW9u0ol/KiroszBZsZ7UHF+OoFKEoK4m//Sc693BTj7N9MhHpFJ/8bEbxLYmA TTam8kTIIUV5pPS3Fyx+eBhM4CGa6somNSeGPdKjNBjjmZwrCbRdHGMcu4vPF4ZzoRZ8 hz3lQmhJCxDP+0zzsoKNcwahHjzXWHMkolMT2eusY6a+Vp9yhbXZqsOXbmSfNsKI2zz/ 3ZjfEOLqMsC9p4/f9EnIIp29MZsLgL3Z+Cx14u2WXH9w4ox2cf0phrTDzctOc2+sy7nI K8Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:date:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1Mr+zyj0+iYAxOsywyr4PXrRM0+BLEnzr0dqBSq/7Yw=; b=bAb1EHBUyk640oPM54jwVdPCiai8nGz3FLha93fwIPZVo3Ml7Tfyr4pBKwqMaM1P8t J79llrBSwIYwu0W8MTr9rb+szaL56i4vM4+vfqFB5F/HZNch7HG6YrDnG9YK27Di4nsv IPlhVY78cCh1ye28ovRjwS4erEq/YuKAHmSBsXiW5raC046IhoxT81rscldwclseWWz3 ihEjs/xfIJtnEJSFPhPIjNBOgOsbXm7GN4lvRq8Oaz2ji08eqzxdsRfT94CDr50ncEat uOdB0leWlkeFdVqTbFpMeTa4Ebzr8tZZApz89i1arD64vJd2JKPObr8KDP6/WRgOcqtW ps1Q== X-Gm-Message-State: AIkVDXLkhRVKKoiFROoBK8IAyYc7mcoCU3keI0fS7+LD710LKy3jij2qDGym3cP/uWH90TFX X-Received: by 10.28.173.74 with SMTP id w71mr17612836wme.14.1485274606703; Tue, 24 Jan 2017 08:16:46 -0800 (PST) Received: from glumotte.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id c81sm26954843wmf.22.2017.01.24.08.16.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 24 Jan 2017 08:16:46 -0800 (PST) From: Olivier MATZ <olivier.matz@6wind.com> X-Google-Original-From: Olivier MATZ <olivier.matz@dev.6wind.com> Date: Tue, 24 Jan 2017 17:16:40 +0100 To: Bruce Richardson <bruce.richardson@intel.com> Cc: Olivier Matz <olivier.matz@6wind.com>, dev@dpdk.org Message-ID: <20170124171640.0f4a8d4e@glumotte.dev.6wind.com> In-Reply-To: <20170124155908.GB172024@bricha3-MOBL3.ger.corp.intel.com> References: <1485271173-13408-1-git-send-email-olivier.matz@6wind.com> <20170124155908.GB172024@bricha3-MOBL3.ger.corp.intel.com> X-Mailer: Claws Mail 3.14.1 (GTK+ 2.24.30; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [RFC 0/8] mbuf: structure reorganization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> X-List-Received-Date: Tue, 24 Jan 2017 16:16:47 -0000 On Tue, 24 Jan 2017 15:59:08 +0000, Bruce Richardson <bruce.richardson@intel.com> wrote: > On Tue, Jan 24, 2017 at 04:19:25PM +0100, Olivier Matz wrote: > > Based on discussion done in [1], this patchset reorganizes the mbuf. > > > > Hi Olivier, > > thanks for all the work on this. From a quick scan of the patches, and > the description below, it looks like a good set of changes. Comments > below to see about kick-starting some further discussion about some of > the other changes you propose. > > > The main changes are: > > - reorder structure to increase vector performance on some non-ia > > platforms. > > - add a 64bits timestamp field in the 1st cache line > > - m->next, m->nb_segs, and m->refcnt are always initialized for > > mbufs in the pool, avoiding the need of setting m->next (located in > > the 2nd cache line) in the Rx path for mono-segment packets. > > - change port and nb_segs to 16 bits > > - move seqn in the 2nd cache line > > > > Things discussed but not done in the patchset: > > - move refcnt and nb_segs to the 2nd cache line: many drivers sets > > them in the Rx path, so it could introduce a performance > > regression, or it would require to change all the drivers, which is > > not an easy task. > > But if we do make this change and update the drivers, some of them > should perform a little better, since they do fewer writes. I don't > think the fastest vector drivers will be affected, since they already > coalesce the writes to these fields with other writes, but others > drivers may well be improved by the change. Yes, that's something I forgot to say in the cover letter: after this patchset, the Rx path of drivers could be optimized a bit by removing writes to m->next, m->nb_segs and m->refcnt. The patch 4/8 gives an idea of what could be done. Once most drivers are updated, we could reconsider moving nb_segs and refcnt in the second cache line. > > > - remove the m->port field: too much impact on many examples and > > libraries, and some people highlighted they are using it. > > - moving m->next in the 1st cache line: there is not enough room, > > and having it set to NULL for unused mbuf should remove the need > > for it. > > I agree. > > > - merge seqn and timestamp together in a union: we could imagine > > use cases were both are activated. There is no flag indicating the > > presence of seqn, so it looks preferable to keep them separated for > > now. > > What were the use-cases? If we have a timestamp, surely sequence can > be determined from that? Even if you use the TSC as a timestamp per > burst, you can still sequence the packets cheaply by just adding 1 to > each subsequent value. Assuming the timestamp is in nanosecond, it is not a sequence number, so I'm not sure it should be hijacked for this purpose. A timestamp can be used to reorder packets, but having a sequence number is better because you can be sure that when you get packets 1, 3, 2, 0 that no packet is missing between 0 and 3. For that reason, I guess both features could be used at the same time. Regards, Olivier