From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <konstantin.ananyev@intel.com>
Received: from mga04.intel.com (mga04.intel.com [192.55.52.120])
 by dpdk.org (Postfix) with ESMTP id 8564B2B93
 for <dev@dpdk.org>; Fri, 31 Mar 2017 11:18:26 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple;
 d=intel.com; i=@intel.com; q=dns/txt; s=intel;
 t=1490951906; x=1522487906;
 h=from:to:cc:subject:date:message-id:references:
 in-reply-to:content-transfer-encoding:mime-version;
 bh=NKKkOMRtHlyzm9ZC2kJEuQ3KY74DQ8iDUTF512PW0z4=;
 b=Yw5SiIqJaWcdgsnqPD4QoJdek6cHrxt2VW9vFfMOtp54WNIlsQcWN0Jf
 YD9stZVg9EIrqkKFe/s1gs1kCX0akA==;
Received: from fmsmga003.fm.intel.com ([10.253.24.29])
 by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
 31 Mar 2017 02:18:25 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.36,251,1486454400"; d="scan'208";a="840345373"
Received: from irsmsx104.ger.corp.intel.com ([163.33.3.159])
 by FMSMGA003.fm.intel.com with ESMTP; 31 Mar 2017 02:18:23 -0700
Received: from irsmsx109.ger.corp.intel.com ([169.254.13.12]) by
 IRSMSX104.ger.corp.intel.com ([163.33.3.159]) with mapi id 14.03.0319.002;
 Fri, 31 Mar 2017 10:18:23 +0100
From: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>
To: Olivier Matz <olivier.matz@6wind.com>, "Richardson, Bruce"
 <bruce.richardson@intel.com>
CC: "dev@dpdk.org" <dev@dpdk.org>, "mb@smartsharesystems.com"
 <mb@smartsharesystems.com>, "Chilikin, Andrey" <andrey.chilikin@intel.com>,
 "jblunck@infradead.org" <jblunck@infradead.org>, "nelio.laranjeiro@6wind.com"
 <nelio.laranjeiro@6wind.com>, "arybchenko@solarflare.com"
 <arybchenko@solarflare.com>
Thread-Topic: [dpdk-dev] [PATCH 0/9] mbuf: structure reorganization
Thread-Index: AQHSl/BM96NM/aKG30C3i/t54/q6k6GsCWqAgABGqICAAOACAIAAKlIAgAAFugCAAFkDYIAAAV1QgACFrWCAAHAVAIAABFSAgAAE9oCAABUBQA==
Date: Fri, 31 Mar 2017 09:18:22 +0000
Message-ID: <2601191342CEEE43887BDE71AB9772583FAE30C3@IRSMSX109.ger.corp.intel.com>
References: <1488966121-22853-1-git-send-email-olivier.matz@6wind.com>
 <20170329175629.68810924@platinum>
 <20170329200923.GA11516@bricha3-MOBL3.ger.corp.intel.com>
 <20170330093108.GA10652@bricha3-MOBL3.ger.corp.intel.com>
 <20170330140236.0d2ebac8@platinum>
 <20170330122305.GA14272@bricha3-MOBL3.ger.corp.intel.com>
 <2601191342CEEE43887BDE71AB9772583FAE2A51@IRSMSX109.ger.corp.intel.com>
 <2601191342CEEE43887BDE71AB9772583FAE2A6E@IRSMSX109.ger.corp.intel.com>
 <2601191342CEEE43887BDE71AB9772583FAE2DD8@IRSMSX109.ger.corp.intel.com>
 <20170331102610.3f82e21e@platinum>
 <20170331084139.GB7668@bricha3-MOBL3.ger.corp.intel.com>
 <20170331105925.135c7377@platinum>
In-Reply-To: <20170331105925.135c7377@platinum>
Accept-Language: en-IE, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [163.33.239.182]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Subject: Re: [dpdk-dev] [PATCH 0/9] mbuf: structure reorganization
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Fri, 31 Mar 2017 09:18:27 -0000


Hi guys,

>=20
> On Fri, 31 Mar 2017 09:41:39 +0100, Bruce Richardson <bruce.richardson@in=
tel.com> wrote:
> > On Fri, Mar 31, 2017 at 10:26:10AM +0200, Olivier Matz wrote:
> > > Hi,
> > >
> > > On Fri, 31 Mar 2017 01:00:49 +0000, "Ananyev, Konstantin" <konstantin=
.ananyev@intel.com> wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Richardson, Bruce
> > > > > > > Sent: Thursday, March 30, 2017 1:23 PM
> > > > > > > To: Olivier Matz <olivier.matz@6wind.com>
> > > > > > > Cc: dev@dpdk.org; Ananyev, Konstantin <konstantin.ananyev@int=
el.com>; mb@smartsharesystems.com; Chilikin, Andrey
> > > > > > > <andrey.chilikin@intel.com>; jblunck@infradead.org; nelio.lar=
anjeiro@6wind.com; arybchenko@solarflare.com
> > > > > > > Subject: Re: [dpdk-dev] [PATCH 0/9] mbuf: structure reorganiz=
ation
> > > > > > >
> > > > > > > On Thu, Mar 30, 2017 at 02:02:36PM +0200, Olivier Matz wrote:
> > > > > > > > On Thu, 30 Mar 2017 10:31:08 +0100, Bruce Richardson <bruce=
.richardson@intel.com> wrote:
> > > > > > > > > On Wed, Mar 29, 2017 at 09:09:23PM +0100, Bruce Richardso=
n wrote:
> > > > > > > > > > On Wed, Mar 29, 2017 at 05:56:29PM +0200, Olivier Matz =
wrote:
> > > > > > > > > > > Hi,
> > > > > > > > > > >
> > > > > > > > > > > Does anyone have any other comment on this series?
> > > > > > > > > > > Can it be applied?
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > Olivier
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I assume all driver maintainers have done performance a=
nalysis to check
> > > > > > > > > > for regressions. Perhaps they can confirm this is the c=
ase.
> > > > > > > > > >
> > > > > > > > > > 	/Bruce
> > > > > > > > > > >
> > > > > > > > > In the absence, of anyone else reporting performance numb=
ers with this
> > > > > > > > > patchset, I ran a single-thread testpmd test using 2 x 40=
G ports (i40e)
> > > > > > > > > driver. With RX & TX descriptor ring sizes of 512 or abov=
e, I'm seeing a
> > > > > > > > > fairly noticable performance drop. I still need to dig in=
 more, e.g. do
> > > > > > > > > an RFC2544 zero-loss test, and also bisect the patchset t=
o see what
> > > > > > > > > parts may be causing the problem.
> > > > > > > > >
> > > > > > > > > Has anyone else tried any other drivers or systems to see=
 what the perf
> > > > > > > > > impact of this set may be?
> > > > > > > >
> > > > > > > > I did, of course. I didn't see any noticeable performance d=
rop on
> > > > > > > > ixgbe (4 NICs, one port per NIC, 1 core). I can replay the =
test with
> > > > > > > > current version.
> > > > > > > >
> > > > > > > I had no doubt you did some perf testing! :-)
> > > > > > >
> > > > > > > Perhaps the regression I see is limited to i40e driver. I've =
confirmed I
> > > > > > > still see it with that driver in zero-loss tests, so next ste=
p is to try
> > > > > > > and localise what change in the patchset is causing it.
> > > > > > >
> > > > > > > Ideally, though, I think we should see acks or other comments=
 from
> > > > > > > driver maintainers at least confirming that they have tested.=
 You cannot
> > > > > > > be held responsible for testing every DPDK driver before you =
submit work
> > > > > > > like this.
> > > > > > >
> > > > > >
> > > > > > Unfortunately  I also see a regression.
> > > > > > Did a quick flood test on 2.8 GHZ IVB with 4x10Gb.
> > > > >
> > > > > Sorry, forgot to mention - it is on ixgbe.
> > > > > So it doesn't look like i40e specific.
> > > > >
> > > > > > Observed a drop even with default testpmd RXD/TXD numbers (128/=
512):
> > > > > > from 50.8 Mpps down to 47.8 Mpps.
> > > > > > From what I am seeing the particular patch that causing it:
> > > > > > [dpdk-dev,3/9] mbuf: set mbuf fields while in pool
> > > > > >
> > > > > > cc version 5.3.1 20160406 (Red Hat 5.3.1-6) (GCC)
> > > > > > cmdline:
> > > > > > ./dpdk.org-1705-mbuf1/x86_64-native-linuxapp-gcc/app/testpmd  -=
-lcores=3D'7,8'  -n 4 --socket-mem=3D'1024,0'  -w 04:00.1 -w
> 07:00.1 -w
> > > > > > 0b:00.1 -w 0e:00.1 -- -i
> > > > > >
> > > >
> > > > After applying the patch below got nearly original numbers (though =
not quite) on my box.
> > > > dpdk.org mainline:           50.8
> > > > with Olivier patch:           47.8
> > > > with patch below:            50.4
> > > > What I tried to do in it - avoid unnecessary updates of mbuf inside=
 rte_pktmbuf_prefree_seg().
> > > > For one segment per packet it seems to help.
> > > > Though so far I didn't try it on i40e and didn't do any testing for=
 multi-seg scenario.
> > > > Konstantin
> > >
> > > I replayed my tests, and I can also see a performance loss with 1c/1t
> > > (ixgbe), not in the same magnitude however. Here is what I have in MP=
PS:
> > >
> > > 1c/1t   1c/2t
> > > 53.3    58.7     current
> > > 52.1    58.8     original patchset
> > > 53.3    58.8     removed patches 3 and 9
> > > 53.1    58.7     with konstantin's patch
> > >
> > > So we have 2 options here:
> > >
> > > 1/ integrate Konstantin's patch in the patchset (thank you, by the wa=
y)
> > > 2/ remove patch 3, and keep it for later until we have something that
> > >    really no impact
> > >
> > > I'd prefer 1/, knowing that the difference is really small in terms
> > > of cycles per packet.
> > >
> > >
> > 1 is certainly the more attractive option. However, I think we can
> > afford to spend a little more time looking at this before we decide.
> > I'll try and check out the perf numbers I get with i40e with
> > Konstantin's patch today. We also need to double check the other
> > possible issues he reported in his other emails. While I don't want thi=
s
> > patchset held up for a long time, I think an extra 24/48 hours is
> > probably needed on it.
> >
>=20
> Yes, now that we have the "test momentum", try not to loose it ;)
>=20
> I'm guilty to have missed the performance loss, but honnestly,
> I'm a bit sad that nobody tried to this patchset before (it
> is available for more than 2 months), knowing this is probably one of
> the most critical part of dpdk. I think we need to be better next
> time.
>=20
> Anyway, thank you for your test and feedback now.

I am also leaning towards option 1, but agree that some extra testing first
need to be done before making the final decision.
BTW, path #9 need to be removed anyway, even if will go for path #1.
Konstantin