From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f50.google.com (mail-pa0-f50.google.com [209.85.220.50]) by dpdk.org (Postfix) with ESMTP id 9200CA2F for ; Mon, 2 Nov 2015 17:24:11 +0100 (CET) Received: by pacfv9 with SMTP id fv9so159391165pac.3 for ; Mon, 02 Nov 2015 08:24:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber_org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=H4MNiMK1wb4FGMBGqJZOTDbk2PLBL6rItbSlXSfxsSg=; b=CpYtdR5J0Inkl9Vjr/A4rjqYyc7CJXN1fE37JC5Us9OM4TatxIMMfeyYvoaT7YgJmy wgQ6p5j0NbS2eibSZTsN+uZvvFm0Qu7igqjOJUdMBrrgQp14fO9ZH5RUqEQNpxkjrcn9 4O8+K8TLfiu3n8+0iUfNihP857fRzG5bf4S1mUeHppBrjJM+ynql3AbcLrfmSTLrktNR iGvyoMFcuLogo9VHNEmlmqg5sLGxdu2Yz+Bc/egJSNAZuxeP2Qpu3vIMcqRy3o3L5NOF 01nCfaC2LkJTfxvdZu9bcdhTEZT8zJ4jB5G+pcQH5wKQnEYv/KGqjPMJj/nUO6wFEeWq xMlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-type:content-transfer-encoding; bh=H4MNiMK1wb4FGMBGqJZOTDbk2PLBL6rItbSlXSfxsSg=; b=PlpC3+F1dI7QUYEbqq1zDQJYwi3FNSFKJNDxP5R5xfPOskLBlfFHJIf73uRtR+TpnB xt67uZ0bXtrSS2D5Oa9T5fdYLvGVFD5FFT1omlDiGaCti1xUoo64uN/qXDquJUReAOEm uH0pGwdyIBtyQCsc+3qogKIQHGCRTbBa0Dj48RRXs1Ml7ngnhR2vyji6fL5d0QShvOnh G09QlMrAGyBpviK6/ygA490Q0daPR65pMOMCDn7gKxWzeamPmeAwjauTAhRdxjGuDkm8 cQbjO1DfzkGDjwZ002QZB/Zj1Y9rf5dBHm+Fhnr6JgDhTBTF9cbYShC+e30EQeH79Gw+ Sykw== X-Gm-Message-State: ALoCoQlVHbJ5CNZ0W7ya8gd9ynJcYGJ/lOa5SmKOQM1IwdwnKMEkop5DeQnDnrU5ohRh4nOWHtSq X-Received: by 10.66.186.141 with SMTP id fk13mr27778452pac.7.1446481450876; Mon, 02 Nov 2015 08:24:10 -0800 (PST) Received: from xeon-e3 (static-50-53-82-155.bvtn.or.frontiernet.net. [50.53.82.155]) by smtp.gmail.com with ESMTPSA id tj2sm24929266pab.4.2015.11.02.08.24.10 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Nov 2015 08:24:10 -0800 (PST) Date: Mon, 2 Nov 2015 08:24:20 -0800 From: Stephen Hemminger To: Arnon Warshavsky Message-ID: <20151102082420.768aea4a@xeon-e3> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] Reshuffling of rte_mbuf structure. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2015 16:24:12 -0000 On Sun, 1 Nov 2015 06:45:31 +0200 Arnon Warshavsky wrote: > My 2 cents, > > This was brought up in the recent user space summit, and it seems that > indeed there is no one cache lines arrangement that fits all. > OTOH multiple compile time options to suffice all flavors, would make it > unpleasant to read maintain test and debug. > (I think there was quiet a consensus in favor of reducing compile options > in general) > > Currently I manage similar deviations via our own source control which I > admit to be quite a pain. > I would prefer an option of code manipulation/generation by some script > during dpdk install, > which takes the default version of rte_mbuf.h, > along with an optional user file (json,xml,elvish,whatever) defining the > structure replacements, > creating your custom version, and placing it instead of the installed copy > of rte_mbuf.h. > Maybe the only facility required from dpdk is just the ability to register > calls to such user scripts at some install stage(s), providing the mean > along with responsibility to the user. > > /Arnon > > > > On Sat, Oct 31, 2015 at 6:44 AM, shesha Sreenivasamurthy (shesha) < > shesha@cisco.com> wrote: > > > In Cisco, we are using DPDK for a very high speed packet processor > > application. We don't use NIC TCP offload / RSS hashing. Putting those > > fields in the first cache-line - and the obligatory mb->next datum in the > > second cache line - causes significant LSU pressure and performance > > degradation. If it does not affect other applications, I would like to > > propose reshuffling of fields so that the obligator "next" field falls in > > first cache line and RSS hashing goes to next. If this re-shuffling indeed > > hurts other applications, another idea is to make it compile time > > configurable. Please provide feedback. > > > > -- > > - Thanks > > char * (*shesha) (uint64_t cache, uint8_t F00D) > > { return 0x0000C0DE; } > > Having different layouts will be a disaster for distro's they have to choose one. And I hate to introduce more configuration! But we see the same issue. It would make sense if there were configuration options for some common optimizations NO_TX_OFFLOAD, NO_MULTISEG, NO_REFCOUNT and then the mbuf got optimized for those combinations. Seems better than config options like LAYOUT1, LAYOUT2, ... In this specific case, I think lots of driver could be check nb_segs == 1 and avoiding the next field for simple packets. Long term, I think this will be losing battle. As DPDK grows more features, the current mbuf structure will grow there is really nothing stopping the bloat of meta data.