[dpdk-dev] Fwd: Disable receiving new flows on a core

[dpdk-dev] Fwd: Disable receiving new flows on a core

[Chintan Inbay]

Begin forwarded message:

From: chintaninbay@outlook.com<mailto:chintaninbay@outlook.com>
Date: November 1, 2018 at 6:19:49 PM PDT
To: users@dpdk.org<mailto:users@dpdk.org>
Subject: Disable receiving new flows on a core

Hi,

We use RSS for distributing flows across cores. Sometimes, in special cases we want to disable receiving new flows to a core. Only new flows. So say if RETA is programmed to receive on cores (0,1,2,3), then the new set would be (0,1,2).  Existing flows going to (3) should be retained and forwarded. Only new flows should not be sent to (3).

Eventually when core (3) is ready again, we need to re program RETA so that all four cores can take flows. So new set of cores would be (0,1,2,3)

Is this possible?

Also if the process on core 3 goes down or crashes, is there a way to notify RSS hardware or you’d have to reprogram RETA in a signal handler?

Thanks,
Chintan

Re: [dpdk-dev] Fwd: Disable receiving new flows on a core

[Bruce Richardson]

On Mon, Nov 05, 2018 at 12:08:48AM +0000, Chintan Inbay wrote:
> 
> 
> 
> Begin forwarded message:
> 
> From: chintaninbay@outlook.com<mailto:chintaninbay@outlook.com>
> Date: November 1, 2018 at 6:19:49 PM PDT
> To: users@dpdk.org<mailto:users@dpdk.org>
> Subject: Disable receiving new flows on a core
> 
> Hi,
> 
> We use RSS for distributing flows across cores. Sometimes, in special cases we want to disable receiving new flows to a core. Only new flows. So say if RETA is programmed to receive on cores (0,1,2,3), then the new set would be (0,1,2).  Existing flows going to (3) should be retained and forwarded. Only new flows should not be sent to (3).
> 
> Eventually when core (3) is ready again, we need to re program RETA so that all four cores can take flows. So new set of cores would be (0,1,2,3)
> 
> Is this possible?

Not really, no. RSS does not track flows individually, only the hashes of
the flow's key fields. Therefore there is no way to separate old and new
flows, all you can do is move all flows that hash to value X to a new core.
Some of those may be old flows and some of them new. Note too that the size
of the RETA table limits the number of bits of that hash that are checked.
It's generally less than 10 bits.

/Bruce

> 
> Also if the process on core 3 goes down or crashes, is there a way to notify RSS hardware or you’d have to reprogram RETA in a signal handler?
> 
> Thanks,
> Chintan

Re: [dpdk-dev] Fwd: Disable receiving new flows on a core

[Chintan Inbay]

Thank you for your response. My guess was that to implement this feature in hardware isn't straightforward and would require support for:

 - allocating additional memory and keeping track of flow hashes per core
 - parsing TCP packets for SYN flag
 - If flow hash matches for a core, send it to a core only if it is non-SYN
 - Send the SYN packet with a hash that matches the disabled core to a new core.

Which probably needs a lot of work. It would be a great feature for roadmap if Intel decides to provide it. However I don't know how difficult it would be and if dpdk framework can eventually support it.

I foresee it as very useful feature for micro-upgrades or per-core software upgrades. When upgrading software on a core-by-core basis without losing connection state. Enabling this feature would let a core drain the flows, then perform a restart with new software.

-Chintan
________________________________
From: Bruce Richardson <bruce.richardson@intel.com>
Sent: Monday, November 5, 2018 9:37 AM
To: Chintan Inbay
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] Fwd: Disable receiving new flows on a core

On Mon, Nov 05, 2018 at 12:08:48AM +0000, Chintan Inbay wrote:
>
>
>
> Begin forwarded message:
>
> From: chintaninbay@outlook.com<mailto:chintaninbay@outlook.com>
> Date: November 1, 2018 at 6:19:49 PM PDT
> To: users@dpdk.org<mailto:users@dpdk.org>
> Subject: Disable receiving new flows on a core
>
> Hi,
>
> We use RSS for distributing flows across cores. Sometimes, in special cases we want to disable receiving new flows to a core. Only new flows. So say if RETA is programmed to receive on cores (0,1,2,3), then the new set would be (0,1,2).  Existing flows going to (3) should be retained and forwarded. Only new flows should not be sent to (3).
>
> Eventually when core (3) is ready again, we need to re program RETA so that all four cores can take flows. So new set of cores would be (0,1,2,3)
>
> Is this possible?

Not really, no. RSS does not track flows individually, only the hashes of
the flow's key fields. Therefore there is no way to separate old and new
flows, all you can do is move all flows that hash to value X to a new core.
Some of those may be old flows and some of them new. Note too that the size
of the RETA table limits the number of bits of that hash that are checked.
It's generally less than 10 bits.

/Bruce

>
> Also if the process on core 3 goes down or crashes, is there a way to notify RSS hardware or you’d have to reprogram RETA in a signal handler?
>
> Thanks,
> Chintan