Re: [dpdk-dev] [PATCH RFC] librte_reorder: new reorder library

[Neil Horman <nhorman@tuxdriver.com>]

On Thu, Oct 09, 2014 at 10:21:35AM -0700, Matthew Hall wrote:
> On Thu, Oct 09, 2014 at 12:09:42PM -0400, Neil Horman wrote:
> > From what you've said above, sequence assignment needs to occur prior to any
> > order breaking event.  That means you either need to do it in individual PMD's
> > on RX, or in the rte_eth library if you want to make it common.  On the TX side
> > you need to ensure that the application applies sequence numbers uniquely and in
> > a predictable fashion in order for the library to work.  that seems like a large
> > external requirement for this functionality to work, and I was wondering how you
> > were going to address that.
> 
> To me it seems like the actualy mechanism of applying the sequence number to 
> the packet belongs in librte_reorder or librte_mbuf.
> 
Perhaps, but this is why I was asking about wanting to merge this functionality
into another library, because segmentation and reordering isn't really an
isolated function, its an operation that needs hooks in a multiple points in the
stack, and as such may be better suited to integration into another library
(distributor as I mentioned before, or mbuf as you note here).  Otherwise you're
left with a library that works only if some external mechanism is implemented
properly for each pmd/application that wants to support it.

> Then, a config define or runtime setting should be used to determine if the 
> operation actually does something or becomes a no-op. Likely a define is 
> needed so the compiler can optimize the code out when it's not used, like for 
> RSS-based cases like mine.
> 
I'm sure that would be the case.  Thats not the hard part though.  The hard part
is figuring out the right policy and way to assign sequence numbers when their
used, in such a way that minimal external code change is needed to support it,
and in such a way that it "always works"

> If the type of sequencing needed could be applied during allocation from the 
> pool, that would be nice because it would be transparent, as long as the PMD's 
> retrieved their new mbuf's the right way.
> 
Sure, that would be a fine way to do it, but the mbuf pool currently I don't
think differentiates between flows, does it?  And as such we either have to:

a) teach it about flows and let PMD's specify a flow from which to draw a
sequence number

b) Allow all nics to share a sequence pool (i.e. serialize sequence number
access and take the performance hit).

c) Segment the sequence name space in such a way that each rx/tx queue can
distribute sequence numbers independently

Not saying any of this is bad, just a question I'm curious as to the answer to.

> If the sequencing to be applied based on properties from L4-L7, then it seems 
> like the distributor would have to be sure to do call the right functions 
> itself, as would other customers, because there'd be no way to magically get 
> them right before you've looked at L4-L7 data first.
> 
True, though from a tx standpoint the order the data is marshalled gives you the
opportunity to sequence the data without explicit L4-L7 knoweldge

Neil