Re: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd

["Ruifeng Wang (Arm Technology China)" <Ruifeng.Wang@arm.com>]

> -----Original Message-----
> From: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> Sent: Wednesday, September 11, 2019 16:35
> To: Ruifeng Wang (Arm Technology China) <Ruifeng.Wang@arm.com>;
> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; Kantecki, Tomasz
> <tomasz.kantecki@intel.com>
> Cc: dev@dpdk.org; Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>;
> nd <nd@arm.com>; nd <nd@arm.com>; nd <nd@arm.com>
> Subject: RE: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd
> 
> 
> 
> > -----Original Message-----
> > From: Ruifeng Wang (Arm Technology China)
> > [mailto:Ruifeng.Wang@arm.com]
> > Sent: Wednesday, September 11, 2019 7:58 AM
> > To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; Ananyev,
> > Konstantin <konstantin.ananyev@intel.com>; Kantecki, Tomasz
> > <tomasz.kantecki@intel.com>
> > Cc: dev@dpdk.org; Gavin Hu (Arm Technology China)
> <Gavin.Hu@arm.com>;
> > nd <nd@arm.com>; nd <nd@arm.com>; nd <nd@arm.com>
> > Subject: RE: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd
> >
> > > -----Original Message-----
> > > From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> > > Sent: Wednesday, September 11, 2019 13:38
> > > To: Ruifeng Wang (Arm Technology China) <Ruifeng.Wang@arm.com>;
> > > Ananyev, Konstantin <konstantin.ananyev@intel.com>; Kantecki,
> Tomasz
> > > <tomasz.kantecki@intel.com>
> > > Cc: dev@dpdk.org; Gavin Hu (Arm Technology China)
> > > <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> > > <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>; nd
> <nd@arm.com>
> > > Subject: RE: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd
> > >
> > > <snip>
> > > > >
> > > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Lock-free mode is supported by hash library and LPM library.
> > > > > > > > Now we add an option for l3fwd example to enable the
> > > > > > > > lock-free
> > > > mode.
> > > > > > > > Necessary preparation steps are added to use lock-free LPM
> mode.
> > > > > > >
> > > > > > > Can I ask about the purpose of these changes?
> > > > > > > Right now in  l3fwd both lpm and hash tables are static and
> > > > > > > hard-
> > > coded.
> > > > > > > we initialize them at startup and then just do read from them.
> > > > > > > Do you plan to enhance l3fwd with ability to dynamically
> > > > > > > update tables contents?
> > > > > > > Though fir that we first have to get rid of hard-coded
> > > > > > > values (config file or
> > > > > so).
> > > > > > > Konstantin
> > > > > > >
> > > > > > Thanks for your questions.
> > > > > > Currently, we have no plan to enhance l3fwd with ability to
> > > > > > dynamically
> > > > > update table contents.
> > > > > > Lock-free method is being integrated into Hash library and LPM
> > > > > > library.  Lock-free algorithms are not only about control
> > > > > > plane (adding or deleting routes), they affect the data path
> > > > > > performance as
> > > well.
> > > > > > Since l3fwd application is showcasing data path performance,
> > > > > > we need to show the impact of including the quiescent state
> > > > > > reporting on data
> > > > path.
> > > > > > This change also serves as an example of using the RCU APIs.
> > > > >
> > > > >
> > > > > But what you suggest doesn't provide the complete picture.
> > > > > With dynamic updates in place (via control path) the data-path
> > > > > impact might be completely different then without.
> > > > > Again without dynamic updates how can you test that your
> > > > > data-path
> > > > > lock- free approach does work as expected?
> > > > > Also it can't even be used as a reference implementation for
> > > > > users, as half of the functionality they need to implement is simply
> missing.
> > > > > My opinion - we either need to leave l3fwd as it is (static
> > > > > routes), or implement a proper control path with ability to
> > > > > dynamically update routes before starting to introduce some
> > > > > synchronization schemes (RCU or whatever).
> > > > >
> > > > > Konstantin
> > > > >
> > > >
> > > > Agree that dynamic control path updates should be included for a
> > > > whole picture.
> > > > I will add dynamic update to l3fwd and reroll the patch series.
> > > > Thanks.
> > > I think we should have an agreement on what exactly we mean by
> > > 'dynamically update routes'.
> > > IMO, we should not disturb the existing static routes as there might
> > > be automated tests running in the labs. I suggest that we should
> > > add/delete new routes/hash entries which are different from the
> > > existing routes/hash entries. This should be sufficient to showcase
> > > the functionality as well as measure the impact.
> > >
> > Yes, existing static routes should be kept intact.
> > To perform regular route/hash entries add/delete, a dedicated lcore will be
> needed.
> > An interactive prompt is not an option since we need automatic add/delete.
> > We can skip master core for data path main loop. And perform unrelated
> route/hash entries add/delete regularly on master core.
> > The impact is that command lines used in tests will need update since
> master core will no longer do data path work.
> 
> Not sure why it has to be  master core?
> Why interrupt thread wouldn't do?
> I think what we need to:
> 1. introduce reading routes from config file instead of having them hard-
> coded within the app.
> 2. add ability to update routes dynamically.
>     Probably the easiest (and commonly used way) re-read conf file and
> update routes on the signal (SIGUSR1 or so).
> Konstantin
> 
> 
Sorry for delayed response. Just back from vacation. 
Thanks for your suggestion.
I will try the config file based updating approach and get back with new version.

> > > >
> > > > > >
> > > > > > > >
> > > > > > > > Patch 2/2 has dependency on RCU QSBR integration with LPM
> > > library:
> > > > > > > > http://patches.dpdk.org/project/dpdk/list/?series=6288
> > > > > > > >
> > > > > > > >
> > > > > > > > Ruifeng Wang (2):
> > > > > > > >   examples/l3fwd: add lock-free option for l3fwd
> > > > > > > >   examples/l3fwd: integrate RCU QSBR for LPM mode
> > > > > > > >
> > > > > > > >  doc/guides/sample_app_ug/l3_forward.rst |  3 ++
> > > > > > > >  examples/l3fwd/Makefile                 |  1 +
> > > > > > > >  examples/l3fwd/l3fwd.h                  |  4 +-
> > > > > > > >  examples/l3fwd/l3fwd_em.c               | 10 +++-
> > > > > > > >  examples/l3fwd/l3fwd_lpm.c              | 72
> > > > > +++++++++++++++++++++++--
> > > > > > > >  examples/l3fwd/main.c                   | 27 ++++++++--
> > > > > > > >  examples/l3fwd/meson.build              |  1 +
> > > > > > > >  7 files changed, 108 insertions(+), 10 deletions(-)
> > > > > > > >
> > > > > > > > --
> > > > > > > > 2.17.1
> > > >
> > >