From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <jigsaw@gmail.com>
Received: from mail-wg0-f43.google.com (mail-wg0-f43.google.com [74.125.82.43])
by dpdk.org (Postfix) with ESMTP id 6EC422A9
for <dev@dpdk.org>; Fri, 7 Nov 2014 15:21:46 +0100 (CET)
Received: by mail-wg0-f43.google.com with SMTP id y10so3907796wgg.16
for <dev@dpdk.org>; Fri, 07 Nov 2014 06:31:18 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
h=mime-version:in-reply-to:references:date:message-id:subject:from:to
:cc:content-type;
bh=aTHS6x5TR95gg4/lDXltvmX0RpRmOI2gOOjmiLH9Y/M=;
b=o1++sbPXYGfiEH3fJzQm30Nq4Y6ecg5qSpbgkPgwYGdhToXHPbNxPkx9BANAztKG/e
VkMQGoxrjdHTNj/fNfCybpsVf3KJjV0/XIiCShDl+hBT2HsTD6DSJGLK93dN2CyKE3Cn
flxXJWZCy+WgCFhw1Dtp1BliK2vCvJu1f8CeQYeTJ9/9v3RQfNUBmaLe6iH4Oa1DCc7p
qeWinmbHbPfec6NnQ8BoFu7CxNTE8Tg5n5CrzwHHvOAEarJ/nxNj9GM05S1TE5FLzXE1
qGWcPqZmDxYrNVv06vxeWH+DBj81XCoM7P7AHIvRd2l9FOd+bqPu8e5p5NxR4CgrfW16
U1lw==
MIME-Version: 1.0
X-Received: by 10.180.95.201 with SMTP id dm9mr5486712wib.27.1415370678481;
Fri, 07 Nov 2014 06:31:18 -0800 (PST)
Received: by 10.27.86.144 with HTTP; Fri, 7 Nov 2014 06:31:18 -0800 (PST)
In-Reply-To: <20141107135303.GB12092@bricha3-MOBL3>
References: <1415194237-1219-1-git-send-email-jigsaw@gmail.com>
<CAHVfvh4X_sUPUzSJTqBdEnkS94t2Jwj_98Vg0xbUS3MPSeo2ZA@mail.gmail.com>
<20141106092228.GA3056@bricha3-MOBL3> <9190772.1rnKUO3oNV@xps13>
<545b6b74.a96db40a.26af.ffffe7fb@mx.google.com>
<20141106135951.GB7252@bricha3-MOBL3>
<CAHVfvh4U4PZKZSue_kKDQKATC2snb_=10OD08LGmUtieBc_LzA@mail.gmail.com>
<CAHVfvh5SzJ-kpQQ9h=1wmMihiitcJXeR9mcNa1in8x6Gb6tSqQ@mail.gmail.com>
<20141107094521.GB4628@bricha3-MOBL3>
<CAHVfvh6y4f7+bMhzmwOu5c0Y4wzwNaxj4sQPtq8cabGbdHrzXg@mail.gmail.com>
<20141107135303.GB12092@bricha3-MOBL3>
Date: Fri, 7 Nov 2014 16:31:18 +0200
Message-ID: <CAHVfvh7ggGB_q1Rs1c3-9PRwDr_GKA+etaMXRSeKCfUKoUx8hQ@mail.gmail.com>
From: jigsaw <jigsaw@gmail.com>
To: Bruce Richardson <bruce.richardson@intel.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Content-Filtered-By: Mailman/MimeDel 2.1.15
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev]
=?utf-8?b?562U5aSNOiAgW1BBVENIXSBBZGQgdXNlciBkZWZpbmVk?=
=?utf-8?q?_tag_calculation_callback_tolibrte=5Fdistributor=2E?=
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Fri, 07 Nov 2014 14:21:46 -0000
Hi Bruce,
Pls have a quick look at the diff to see if this is exactly what you mean
about the bitmask.
I just wrote it without even compiling, just to express the idea. So it may
leave some places unpatched.
If this is agreed, I will make a decent test to verify it before sending
the patch for RFC.
diff --git a/lib/librte_distributor/rte_distributor.c
b/lib/librte_distributor/rte_di
index 585ff88..d606bcf 100644
--- a/lib/librte_distributor/rte_distributor.c
+++ b/lib/librte_distributor/rte_distributor.c
@@ -92,6 +92,8 @@ struct rte_distributor {
unsigned num_workers; /**< Number of workers
polling */
uint32_t in_flight_tags[RTE_MAX_LCORE];
+ uint32_t in_flight_bitmask;
+
struct rte_distributor_backlog backlog[RTE_MAX_LCORE];
union rte_distributor_buffer bufs[RTE_MAX_LCORE];
@@ -188,6 +190,7 @@ static inline void
handle_worker_shutdown(struct rte_distributor *d, unsigned wkr)
{
d->in_flight_tags[wkr] =3D 0;
+ d->in_flight_mask &=3D ~(1 << wkr);
d->bufs[wkr].bufptr64 =3D 0;
if (unlikely(d->backlog[wkr].count !=3D 0)) {
/* On return of a packet, we need to move the
@@ -241,6 +244,7 @@ process_returns(struct rte_distributor *d)
else {
d->bufs[wkr].bufptr64 =3D RTE_DISTRIB_GET_B=
UF;
d->in_flight_tags[wkr] =3D 0;
+ d->in_flight_mask &=3D ~(1 << wkr);
}
oldbuf =3D data >> RTE_DISTRIB_FLAG_BITS;
} else if (data & RTE_DISTRIB_RETURN_BUF) {
@@ -282,12 +286,13 @@ rte_distributor_process(struct rte_distributor *d,
next_mb =3D mbufs[next_idx++];
next_value =3D (((int64_t)(uintptr_t)next_mb)
<< RTE_DISTRIB_FLAG_BITS);
- new_tag =3D (next_mb->hash.rss | 1);
+ new_tag =3D next_mb->hash.rss;
uint32_t match =3D 0;
unsigned i;
for (i =3D 0; i < d->num_workers; i++)
- match |=3D (!(d->in_flight_tags[i] ^ new_ta=
g)
+ match |=3D (((!(d->in_flight_tags[i] ^
new_tag)) &
+ (d->in_flight_bitmask >> i)=
)
<< i);
if (match) {
@@ -309,6 +314,7 @@ rte_distributor_process(struct rte_distributor *d,
else {
d->bufs[wkr].bufptr64 =3D next_value;
d->in_flight_tags[wkr] =3D new_tag;
+ d->in_flight_bitmask |=3D 1 << wkr;
next_mb =3D NULL;
}
oldbuf =3D data >> RTE_DISTRIB_FLAG_BITS;
On Fri, Nov 7, 2014 at 3:53 PM, Bruce Richardson <bruce.richardson@intel.co=
m
> wrote:
> On Fri, Nov 07, 2014 at 02:38:13PM +0200, jigsaw wrote:
> > Hi Bruce,
> >
> > >> If a tag value of zero is ever passed in, then it will start matchi=
ng
> > against cores which are not doing any processing.
> >
> > Yes, this is true according to current bookkeeping of inflight tags.
> >
> > But if the slot in in_flight_tags is not a uint32_t but a struct which
> has
> > a filed as indication of "on/off", and also with corresponding changes =
in
> > looking for a matched tag, then the need for 1 bit mask can be
> eliminated.
> > Of course this change requires a little bit more, O(n), memory space an=
d
> > costs O(n) more branch misses. But the benefit is a more free interface
> to
> > user app.
> >
> > This is just another trade-off. Since I am in need of such freedom, I a=
m
> > more interested in the free use of 32bits.
>
> If you do implement such a change, I would suggest you simply add a bitma=
sk
> to the distributor indicating valid workers. Then when we do the check
> for tag matches, we just need an extra "and" instruction to eliminate
> invalid
> workers from the match.
>
> /Bruce
>
> >
> > thx &
> > rgds,
> > -qinglai
> >
> >
> > On Fri, Nov 7, 2014 at 11:45 AM, Bruce Richardson <
> > bruce.richardson@intel.com> wrote:
> >
> > > On Thu, Nov 06, 2014 at 09:52:25PM +0200, jigsaw wrote:
> > > > Hi Bruce,
> > > >
> > > > Actually IMHO it is good to leave the freedom to user to decide how
> to
> > > > interpret the tag value, i.e. remove the OR 1 bit.
> > > > If the tag value is zero, then we assume the programmer know what h=
e
> is
> > > > doing. Of course this shall be clearly documented in the
> comment/doxgen.
> > > >
> > > >
> > > > thx &
> > > > rgds,
> > > > -qinglai
> > >
> > > I don't believe that will work. If a tag value of zero is ever passed
> > > in, then it will start matching against cores which are not doing any
> > > processing. Then it will get queued up to get sent to those cores, an=
d
> so
> > > never get processed.
> > > We need a bit somewhere inside the tag to permanently set - though it
> can
> > > be configurable.
> > >
> > > /Bruce
> > >
> > > >
> > > > On Thu, Nov 6, 2014 at 8:01 PM, jigsaw <jigsaw@gmail.com> wrote:
> > > >
> > > > > Hi Bruce,
> > > > >
> > > > > In my use case, unfortunately the tag is not hash. And the tag ca=
n
> be
> > > on
> > > > > either low or high bits, depending on configuration.
> > > > > I wonder if it is possible to let the user to decide which bit to
> mask,
> > > > > i.e. to add another param to rte_distributor_create to define the
> mask.
> > > > >
> > > > > thx &
> > > > > rgds,
> > > > > -qinglai
> > > > >
> > > > > On Thu, Nov 6, 2014 at 3:59 PM, Bruce Richardson <
> > > > > bruce.richardson@intel.com> wrote:
> > > > >
> > > > >> On Thu, Nov 06, 2014 at 02:36:09PM +0200, Qinglai Xiao wrote:
> > > > >> > Hi Bruce,
> > > > >> >
> > > > >> > There is a subtle case in which tag values are 2 and 3,
> > > respectively.
> > > > >> Then these two tags cannot be distinguished. There should be a
> better
> > > way
> > > > >> so as to handle this situation.
> > > > >>
> > > > >> It's not just in that, case, it's in any case where a pair of ta=
gs
> > > differ
> > > > >> by
> > > > >> only a single bit. I've been assuming that the tag is likely to
> be a
> > > hash
> > > > >> value in most cases - given that it's only 32-bit - in which cas=
e
> it
> > > just
> > > > >> doesn't
> > > > >> matter which bit we chose to permanently set to 1, but if there
> are
> > > > >> scenarios
> > > > >> where it's likely that the low bits are used but the high ones n=
ot
> > > so, we
> > > > >> can
> > > > >> look to change which bit is set to 1. Either way, the distributo=
r
> just
> > > > >> uses a
> > > > >> 31-bit tag rather than a 32-bit one.
> > > > >>
> > > > >> /Bruce
> > > > >>
> > > > >> >
> > > > >> > thx &
> > > > >> > rgds
> > > > >> > -qinglai
> > > > >> >
> > > > >> > -----=E5=8E=9F=E5=A7=8B=E9=82=AE=E4=BB=B6-----
> > > > >> > =E5=8F=91=E4=BB=B6=E4=BA=BA: "Thomas Monjalon" <thomas.monjalo=
n@6wind.com>
> > > > >> > =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: =E2=80=8E2014/=E2=80=8E1=
1/=E2=80=8E6 12:36
> > > > >> > =E6=94=B6=E4=BB=B6=E4=BA=BA: "Bruce Richardson" <bruce.richard=
son@intel.com>
> > > > >> > =E6=8A=84=E9=80=81: "dev@dpdk.org" <dev@dpdk.org>; "jigsaw" <j=
igsaw@gmail.com>
> > > > >> > =E4=B8=BB=E9=A2=98: Re: [dpdk-dev] [PATCH] Add user defined ta=
g calculation
> callback
> > > > >> tolibrte_distributor.
> > > > >> >
> > > > >> > 2014-11-06 09:22, Bruce Richardson:
> > > > >> > > On Wed, Nov 05, 2014 at 07:24:13PM +0200, jigsaw wrote:
> > > > >> > > >
> > > > >>
> > >
> http://dpdk.org/browse/dpdk/tree/lib/librte_distributor/rte_distributor.c=
#n285
> > > > >> > > >
> > > > >> > > > new_tag =3D (next_mb->hash.rss | 1);
> > > > >> > > >
> > > > >> > > > Why the logical OR is needed?
> > > > >> > >
> > > > >> > > That's needed to ensure that we never track a tag with an
> actual
> > > > >> value of zero.
> > > > >> > > We instead always force the low bit to be 1, so that we can
> use
> > > zero
> > > > >> as an
> > > > >> > > "empty" value.
> > > > >> >
> > > > >> > Bruce, could you check how this code may be better commented
> please?
> > > > >> > This discussion shows that the distributor library probably
> needs
> > > more
> > > > >> > explanations in the code or doxygen.
> > > > >> >
> > > > >> > Thanks
> > > > >> > --
> > > > >> > Thomas
> > > > >>
> > > > >
> > > > >
> > >
>