* Re: [RFC 0/4] ethdev: rework config restore
2024-10-09 16:18 3% ` Dariusz Sosnowski
@ 2024-10-09 23:16 0% ` Ferruh Yigit
0 siblings, 0 replies; 200+ results
From: Ferruh Yigit @ 2024-10-09 23:16 UTC (permalink / raw)
To: Dariusz Sosnowski, Konstantin Ananyev,
NBU-Contact-Thomas Monjalon (EXTERNAL),
Andrew Rybchenko
Cc: dev, Bruce Richardson
On 10/9/2024 5:18 PM, Dariusz Sosnowski wrote:
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@amd.com>
>> Sent: Wednesday, October 9, 2024 03:08
>> To: Konstantin Ananyev <konstantin.ananyev@huawei.com>; Dariusz Sosnowski
>> <dsosnowski@nvidia.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
>> <thomas@monjalon.net>; Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>> Cc: dev@dpdk.org; Bruce Richardson <bruce.richardson@intel.com>
>> Subject: Re: [RFC 0/4] ethdev: rework config restore
>>
>> External email: Use caution opening links or attachments
>>
>>
>> On 10/8/2024 6:21 PM, Konstantin Ananyev wrote:
>>>
>>>
>>>>>>>> We have been working on optimizing the latency of calls to
>>>>>>>> rte_eth_dev_start(), on ports spawned by mlx5 PMD. Most of the
>>>>>>>> work requires changes in the implementation of
>>>>>>>> .dev_start() PMD callback, but I also wanted to start a
>>>>>>>> discussion regarding configuration restore.
>>>>>>>>
>>>>>>>> rte_eth_dev_start() does a few things on top of calling .dev_start()
>> callback:
>>>>>>>>
>>>>>>>> - Before calling it:
>>>>>>>> - eth_dev_mac_restore() - if device supports
>>>>>>>> RTE_ETH_DEV_NOLIVE_MAC_ADDR;
>>>>>>>> - After calling it:
>>>>>>>> - eth_dev_mac_restore() - if device does not support
>>>>>>> RTE_ETH_DEV_NOLIVE_MAC_ADDR;
>>>>>>>> - restore promiscuous config
>>>>>>>> - restore all multicast config
>>>>>>>>
>>>>>>>> eth_dev_mac_restore() iterates over all known MAC addresses -
>>>>>>>> stored in rte_eth_dev_data.mac_addrs array - and calls
>>>>>>>> .mac_addr_set() and .mac_addr_add() callbacks to apply these MAC
>> addresses.
>>>>>>>>
>>>>>>>> Promiscuous config restore checks if promiscuous mode is enabled
>>>>>>>> or not, and calls .promiscuous_enable() or .promiscuous_disable()
>> callback.
>>>>>>>>
>>>>>>>> All multicast config restore checks if all multicast mode is
>>>>>>>> enabled or not, and calls .allmulticast_enable() or .allmulticast_disable()
>> callback.
>>>>>>>>
>>>>>>>> Callbacks are called directly in all of these cases, to bypass
>>>>>>>> the checks for applying the same configuration, which exist in relevant
>> APIs.
>>>>>>>> Checks are bypassed to force drivers to reapply the configuration.
>>>>>>>>
>>>>>>>> Let's consider what happens in the following sequence of API calls.
>>>>>>>>
>>>>>>>> 1. rte_eth_dev_configure()
>>>>>>>> 2. rte_eth_tx_queue_setup()
>>>>>>>> 3. rte_eth_rx_queue_setup()
>>>>>>>> 4. rte_eth_promiscuous_enable()
>>>>>>>> - Call dev->dev_ops->promiscuous_enable()
>>>>>>>> - Stores promiscuous state in dev->data->promiscuous 5.
>>>>>>>> rte_eth_allmulticast_enable()
>>>>>>>> - Call dev->dev_ops->allmulticast_enable()
>>>>>>>> - Stores allmulticast state in dev->data->allmulticast 6.
>>>>>>>> rte_eth_dev_start()
>>>>>>>> - Call dev->dev_ops->dev_start()
>>>>>>>> - Call dev->dev_ops->mac_addr_set() - apply default MAC address
>>>>>>>> - Call dev->dev_ops->promiscuous_enable()
>>>>>>>> - Call dev->dev_ops->allmulticast_enable()
>>>>>>>>
>>>>>>>> Even though all configuration is available in dev->data after
>>>>>>>> step 5, library forces reapplying this configuration in step 6.
>>>>>>>>
>>>>>>>> In mlx5 PMD case all relevant callbacks require communication
>>>>>>>> with the kernel driver, to configure the device (mlx5 PMD must
>>>>>>>> create/destroy new kernel flow rules and/or change netdev config).
>>>>>>>>
>>>>>>>> mlx5 PMD handles applying all configuration in .dev_start(), so
>>>>>>>> the following forced callbacks force additional communication
>>>>>>>> with the kernel. The
>>>>>>> same configuration is applied multiple times.
>>>>>>>>
>>>>>>>> As an optimization, mlx5 PMD could check if a given configuration
>>>>>>>> was applied, but this would duplicate the functionality of the
>>>>>>>> library (for example rte_eth_promiscuous_enable() does not call
>>>>>>>> the driver if
>>>>>>>> dev->data->promiscuous is set).
>>>>>>>>
>>>>>>>> Question: Since all of the configuration is available before
>>>>>>>> .dev_start() callback is called, why ethdev library does not
>>>>>>>> expect .dev_start() to
>>>>>>> take this configuration into account?
>>>>>>>> In other words, why library has to reapply the configuration?
>>>>>>>>
>>>>>>>> I could not find any particular reason why configuration restore
>>>>>>>> exists as part of the process (it was in the initial DPDK commit).
>>>>>>>>
>>>>>>>
>>>>>>> My assumption is .dev_stop() cause these values reset in some
>>>>>>> devices, so
>>>>>>> .dev_start() restores them back.
>>>>>>> @Bruce or @Konstantin may remember the history.
>>>>>
>>>>> Yep, as I remember, at least some Intel PMDs calling hw_reset() ad
>>>>> dec_stop() and even dev_start() to make sure that HW is in a clean (known)
>> state.
>>>>>
>>>>>>>
>>>>>>> But I agree this is device specific behavior, and can be managed
>>>>>>> by what device requires.
>>>>>
>>>>> Probably yes.
>>>>>
>>>>>>>
>>>>>>>> The patches included in this RFC, propose a mechanism which would
>>>>>>>> help with managing which drivers rely on forceful configuration restore.
>>>>>>>> Drivers could advertise if forceful configuration restore is
>>>>>>>> needed through `RTE_ETH_DEV_*_FORCE_RESTORE` device flag. If this
>>>>>>>> flag is set, then the driver in question requires ethdev to
>>>>>>>> forcefully restore
>>>>>>> configuration.
>>>>>>>>
>>>>>>>
>>>>>>> OK to use flag for it, but not sure about using 'dev_info->dev_flags'
>>>>>>> (RTE_ETH_DEV_*) for this, as this flag is shared with user and
>>>>>>> this is all dpdk internal.
>>>>>>>
>>>>>>> What about to have a dedicated flag for it? We can have a
>>>>>>> dedicated set of flag values for restore.
>>>>>>
>>>>>> Agreed. What do you think about the following?
>>>>>
>>>>> Instead of exposing that, can we probably make it transparent to the
>>>>> user and probably ethdev layer too?
>>>>>
>>>>
>>>> +1 to make it transparent to user, but not sure if we can make it
>>>> transparent to ethdev layer.
>>>
>>> Just to be clear:
>>> Let say, using example from above:
>>>
>>> rte_eth_dev_start()
>>> - Call dev->dev_ops->dev_start()
>>> - Call dev->dev_ops->mac_addr_set() - apply default MAC address
>>> - Call dev->dev_ops->promiscuous_enable()
>>> - Call dev->dev_ops->allmulticast_enable()
>>>
>>> We probably can introduce ethdev internal function (still visible to
>>> PMDs) that would do last 3 steps:
>>> ethdev_replay_user_conf(...)
>>> - Call dev->dev_ops->mac_addr_set() - apply default MAC address
>>> - Call dev->dev_ops->promiscuous_enable()
>>> - Call dev->dev_ops->allmulticast_enable()
>>>
>>> And let PMD itself to decide does it needs to call it at dev_start() or not.
>>> So it will become:
>>> rte_eth_dev_start()
>>> - Call dev->dev_ops->dev_start()
>>> -Call ethdev_replay_user_conf(.)
>>> - Call dev->dev_ops->mac_addr_set() - apply default MAC address
>>> - Call dev->dev_ops->promiscuous_enable()
>>> -Call dev->dev_ops->allmulticast_enable()
>>>
>>> For PMDs that do need to restore user provided config And
>>> rte_eth_dev_start()
>>> - Call dev->dev_ops->dev_start()
>>>
>>> For those who do not.
>>>
>>
>> OK, got it what you mean.
>> Pushing restore functionality to PMDs works, but this may be doing redundant
>> work on each PMD.
>>
>> Instead Dariusz suggests PMD to provide a flag to ehtdev to what to restore and
>> common code in ethdev does the work.
>> My below dedicated data struct comment is to have this flag in a new struct,
>> overall like following:
>>
>> rte_eth_dev_start()
>> - Call dev->dev_ops->dev_start()
>> - Call dev->dev_ops->get_restore_flags(ethdev, RTE_ETH_START, &flags)
>> - if (flags & MAC) dev->dev_ops->mac_addr_set()
>> - if (flags & PROMISC) dev->dev_ops->promiscuous_enable()
>> - ...
>
> Could you please explain what is the benefit of exposing flags through dev_ops callback vs a dedicated flags field in rte_eth_dev_data?
> In both solutions:
> - config restore is transparent to the user,
> - drivers can omit config restore (either by not implementing the callback or not providing the flags),
> - an ABI change is introduced (not a huge concern, at least for 24.11).
>
> I understand that my initial proposal with "internal_flags" was too vague,
> but renaming and splitting this field into:
>
> - dev_start_restore_flags
> - dev_reset_restore_flags
> - and so on...
>
> seems sufficient, at least in my opinion.
>
Hi Dariusz,
Putting flags to rte_eth_dev_data works, and it is easier since there is
direct access from rte_eth_dev to rte_eth_dev_data, so you don't need
new dev_ops. So this is a valid option.
But benefit of new dev_ops is to keep "struct rte_eth_dev_data" clean.
"struct rte_eth_dev_data" is integral data structure for ethdev and it
is used in multiple locations, mostly related to the datapath and all
drivers needs to deal with fields of this struct.
Like [rx]_queues, dev_private, dev_conf all important and used a lot.
I want to protect "struct rte_eth_dev_data" from noise as much as
possible, though what is noise is not always that clear.
This restore flag is not critical, and I expect most of the drivers
won't care and populate this restore flag at all. That is why to me it
is better have dedicated struct for it and only drivers care about
restore feature know it.
>>
>> So PMDs only will provide what to restore with an internal API and common
>> ethdev layer will restore it.
>> If no restore required PMD may not implement .get_restore_flags() at all.
>>
>> Additionally, RTE_ETH_START, RTE_ETH_RESET etc flag can be provided to internal
>> API to get what to restore in different states...
>>
>>>> Suggested 'internal_flag' in "struct rte_eth_dev_data" can be
>>>> confusing and open to interpretation what to use it for and by time
>>>> become source of defect.
>>>
>>> Yes, same thoughts.
>>>
>>>> Instead what do you think to have a separate, dedicated data struct for it?
>>>
>>> Hmm... not sure I understood you here...
>>>
>>>>
>>>>> Might be we can move this restoration code into the new ethdev
>>>>> helper function,(ethdevd_user_config_restore() or so) that PMD can invoke
>> during its dev_start() if needed?
>>>>>
>>>>>>
>>>>>> #define RTE_ETH_DEV_INTERNAL_PROMISC_FORCE_RESTORE RTE_BIT32(0)
>>>>>> #define RTE_ETH_DEV_INTERNAL_ALLMULTI_FORCE_RESTORE RTE_BIT32(1)
>>>>>> #define RTE_ETH_DEV_INTERNAL_MAC_ADDR_FORCE_RESTORE
>> RTE_BIT32(2)
>>>>>>
>>>>>> struct rte_eth_dev_data {
>>>>>> /* snip */
>>>>>>
>>>>>> uint32_t dev_flags;
>>>>>>
>>>>>> /**
>>>>>> * Internal device capabilities, used only by ethdev library.
>>>>>> * Certain functionalities provided by the library might enabled/disabled,
>>>>>> * based on driver exposing certain capabilities.
>>>>>> */
>>>>>> uint32_t internal_flags;
>>>>>>
>>>>>> /* snip */
>>>>>> };
>>>>>>
>>>>>>> Also perhaps we have go into details what needs to be restored
>>>>>>> after 'stop' and what needs to be restored after 'reset' and use similar
>> mechanism etc...
>>>>>>
>>>>>> I think we should look into that.
>>>>>> Any 'codification' of semantics between drivers and ethdev library is good in
>> my opinion.
>>>>>>
>>>>>> At least right now, ethdev does not change any configuration in 'stop' and
>> 'reset' from what I see.
>>>>>> But that's on library side only.
>>>>>>
>>>>>>>> This way, if we would conclude that it makes sense for
>>>>>>>> .dev_start() to handle all starting configuration aspects, we
>>>>>>>> could track which drivers still rely
>>>>>>> on configuration restore.
>>>>>>>>
>>>>>>>> Dariusz Sosnowski (4):
>>>>>>>> ethdev: rework config restore
>>>>>>>> ethdev: omit promiscuous config restore if not required
>>>>>>>> ethdev: omit all multicast config restore if not required
>>>>>>>> ethdev: omit MAC address restore if not required
>>>>>>>>
>>>>>>>> lib/ethdev/rte_ethdev.c | 39
>>>>>>>> ++++++++++++++++++++++++++++++++++-----
>>>>>>>> lib/ethdev/rte_ethdev.h | 18 ++++++++++++++++++
>>>>>>>> 2 files changed, 52 insertions(+), 5 deletions(-)
>>>>>>>>
>>>>>>>> --
>>>>>>>> 2.39.5
>>>>>>>>
>>>>>
>>>
>
^ permalink raw reply [relevance 0%]
* RE: [PATCH] cryptodev: remove unnecessary list end
2024-10-09 12:52 0% ` Morten Brørup
@ 2024-10-09 20:38 0% ` Akhil Goyal
0 siblings, 0 replies; 200+ results
From: Akhil Goyal @ 2024-10-09 20:38 UTC (permalink / raw)
To: Morten Brørup, dev
Cc: thomas, david.marchand, hemant.agrawal, Anoob Joseph,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde,
Nagadheeraj Rottela, ferruh.yigit
> > From: Akhil Goyal [mailto:gakhil@marvell.com]
> > Sent: Wednesday, 9 October 2024 13.25
> >
> > RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END and
> > RTE_CRYPTO_RSA_PADDING_TYPE_LIST_END are not used anywhere and
> > mostly it do not fulfil any objective.
> > Rather it does not allow addition of new algorithms and
> > new padding type as it result in ABI breakage.
> > Hence, these are removed.
> >
> > Signed-off-by: Akhil Goyal <gakhil@marvell.com>
> > ---
> > lib/cryptodev/rte_crypto_asym.h | 3 ---
> > 1 file changed, 3 deletions(-)
> >
> > diff --git a/lib/cryptodev/rte_crypto_asym.h
> > b/lib/cryptodev/rte_crypto_asym.h
> > index 39d3da3952..524753e2b2 100644
> > --- a/lib/cryptodev/rte_crypto_asym.h
> > +++ b/lib/cryptodev/rte_crypto_asym.h
> > @@ -119,8 +119,6 @@ enum rte_crypto_asym_xform_type {
> > * Performs Encrypt, Decrypt, Sign and Verify.
> > * Refer to rte_crypto_asym_op_type.
> > */
> > - RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
> > - /**< End of list */
> > };
> >
> > /**
> > @@ -168,7 +166,6 @@ enum rte_crypto_rsa_padding_type {
> > /**< RSA PKCS#1 OAEP padding scheme */
> > RTE_CRYPTO_RSA_PADDING_PSS,
> > /**< RSA PKCS#1 PSS padding scheme */
> > - RTE_CRYPTO_RSA_PADDING_TYPE_LIST_END
> > };
> >
> > /**
> > --
> > 2.25.1
>
> Acked-by: Morten Brørup <mb@smartsharesystems.com>
Applied to dpdk-next-crypto
Updated release notes
^ permalink raw reply [relevance 0%]
* Re: [PATCH v3 0/5] add feature arc in rte_graph
2024-10-09 13:29 3% ` [PATCH v3 " Nitin Saxena
2024-10-09 13:29 4% ` [PATCH v3 2/5] graph: add feature arc option in graph create Nitin Saxena
2024-10-09 14:21 0% ` [PATCH v3 0/5] add feature arc in rte_graph Christophe Fontaine
@ 2024-10-09 17:37 0% ` Stephen Hemminger
2 siblings, 0 replies; 200+ results
From: Stephen Hemminger @ 2024-10-09 17:37 UTC (permalink / raw)
To: Nitin Saxena
Cc: Jerin Jacob, Kiran Kumar K, Nithin Dabilpuram, Zhirun Yan,
Robin Jarry, Christophe Fontaine, dev, Nitin Saxena
On Wed, 9 Oct 2024 18:59:57 +0530
Nitin Saxena <nsaxena@marvell.com> wrote:
> Feature arc represents an ordered list of features/protocols at a given
> networking layer. It is a high level abstraction to connect various
> rte_graph nodes, as feature nodes, and allow packets steering across
> these nodes in a generic manner.
>
> Features (or feature nodes) are nodes which handles partial or complete
> handling of a protocol in fast path. Like ipv4-rewrite node, which adds
> rewrite data to an outgoing IPv4 packet.
>
> However in above example, outgoing interface(say "eth0") may have
> outbound IPsec policy enabled, hence packets must be steered from
> ipv4-rewrite node to ipsec-outbound-policy node for outbound IPsec
> policy lookup. On the other hand, packets routed to another interface
> (eth1) will not be sent to ipsec-outbound-policy node as IPsec feature
> is disabled on eth1. Feature-arc allows rte_graph applications to manage
> such constraints easily
>
> Feature arc abstraction allows rte_graph based application to
>
> 1. Seamlessly steer packets across feature nodes based on whether
> feature is enabled or disabled on an interface. Features enabled on one
> interface may not be enabled on another interface with in a same feature
> arc.
>
> 2. Allow enabling/disabling of features on an interface at runtime,
> so that if a feature is disabled, packets associated with that interface
> won't be steered to corresponding feature node.
>
> 3. Provides mechanism to hook custom/user-defined nodes to a feature
> node and allow packet steering from feature node to custom node without
> changing former's fast path function
>
> 4. Allow expressing features in a particular sequential order so that
> packets are steered in an ordered way across nodes in fast path. For
> eg: if IPsec and IPv4 features are enabled on an ingress interface,
> packets must be sent to IPsec inbound policy node first and then to ipv4
> lookup node.
>
> This patch series adds feature arc library in rte_graph and also adds
> "ipv4-output" feature arc handling in "ipv4-rewrite" node.
>
> Changes in v3:
> - rte_graph_feature_arc_t typedef from uint64_t to uintptr_t to fix
> compilation on 32-bit machine
> - Updated images in .png format
> - Added ABI change section in release notes
> - Fixed DPDK CI failures
>
> Changes in v2:
> - Added unit tests for feature arc
> - Fixed issues found in testing
> - Added new public APIs rte_graph_feature_arc_feature_to_node(),
> rte_graph_feature_arc_feature_to_name(),
> rte_graph_feature_arc_num_features()
> - Added programming guide for feature arc
> - Added release notes for feature arc
>
> Nitin Saxena (5):
> graph: add feature arc support
> graph: add feature arc option in graph create
> graph: add IPv4 output feature arc
> test/graph_feature_arc: add functional tests
> docs: add programming guide for feature arc
>
> app/test/meson.build | 1 +
> app/test/test_graph_feature_arc.c | 1410 +++++++++++++++++++
> doc/guides/prog_guide/graph_lib.rst | 288 ++++
> doc/guides/prog_guide/img/feature_arc-1.png | Bin 0 -> 61532 bytes
> doc/guides/prog_guide/img/feature_arc-2.png | Bin 0 -> 155806 bytes
> doc/guides/prog_guide/img/feature_arc-3.png | Bin 0 -> 143697 bytes
> doc/guides/rel_notes/release_24_11.rst | 13 +
> lib/graph/graph.c | 1 +
> lib/graph/graph_feature_arc.c | 1223 ++++++++++++++++
> lib/graph/graph_populate.c | 7 +-
> lib/graph/graph_private.h | 3 +
> lib/graph/meson.build | 2 +
> lib/graph/node.c | 2 +
> lib/graph/rte_graph.h | 3 +
> lib/graph/rte_graph_feature_arc.h | 431 ++++++
> lib/graph/rte_graph_feature_arc_worker.h | 674 +++++++++
> lib/graph/version.map | 20 +
> lib/node/ip4_rewrite.c | 476 +++++--
> lib/node/ip4_rewrite_priv.h | 15 +-
> lib/node/node_private.h | 20 +-
> lib/node/rte_node_ip4_api.h | 3 +
> 21 files changed, 4494 insertions(+), 98 deletions(-)
> create mode 100644 app/test/test_graph_feature_arc.c
> create mode 100644 doc/guides/prog_guide/img/feature_arc-1.png
> create mode 100644 doc/guides/prog_guide/img/feature_arc-2.png
> create mode 100644 doc/guides/prog_guide/img/feature_arc-3.png
> create mode 100644 lib/graph/graph_feature_arc.c
> create mode 100644 lib/graph/rte_graph_feature_arc.h
> create mode 100644 lib/graph/rte_graph_feature_arc_worker.h
>
Looks good, but likely missing an RTE_ATOMIC() around the feature enable bitmask.
Build fails:
####################################################################################
#### [Begin job log] "ubuntu-22.04-clang-stdatomic" at step Build and test
####################################################################################
rte_atomic_store_explicit(&arc->feature_enable_bitmask[passive_list], bitmask,
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../lib/eal/include/rte_stdatomic.h:76:2: note: expanded from macro 'rte_atomic_store_explicit'
atomic_store_explicit(ptr, val, memorder)
^ ~~~
/usr/lib/llvm-14/lib/clang/14.0.0/include/stdatomic.h:127:31: note: expanded from macro 'atomic_store_explicit'
#define atomic_store_explicit __c11_atomic_store
^
../lib/graph/graph_feature_arc.c:1084:2: error: address argument to atomic operation must be a pointer to _Atomic type ('rte_graph_feature_rt_list_t *' (aka 'unsigned short *') invalid)
rte_atomic_store_explicit(&arc->active_feature_list, passive_list,
^ ~~~~~~~~~~~~~~~~~~~~~~~~~
../lib/eal/include/rte_stdatomic.h:76:2: note: expanded from macro 'rte_atomic_store_explicit'
atomic_store_explicit(ptr, val, memorder)
^ ~~~
/usr/lib/llvm-14/lib/clang/14.0.0/include/stdatomic.h:127:31: note: expanded from macro 'atomic_store_explicit'
#define atomic_store_explicit __c11_atomic_store
^
10 errors generated.
^ permalink raw reply [relevance 0%]
* RE: [RFC 0/4] ethdev: rework config restore
@ 2024-10-09 16:18 3% ` Dariusz Sosnowski
2024-10-09 23:16 0% ` Ferruh Yigit
0 siblings, 1 reply; 200+ results
From: Dariusz Sosnowski @ 2024-10-09 16:18 UTC (permalink / raw)
To: Ferruh Yigit, Konstantin Ananyev,
NBU-Contact-Thomas Monjalon (EXTERNAL),
Andrew Rybchenko
Cc: dev, Bruce Richardson
> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@amd.com>
> Sent: Wednesday, October 9, 2024 03:08
> To: Konstantin Ananyev <konstantin.ananyev@huawei.com>; Dariusz Sosnowski
> <dsosnowski@nvidia.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> <thomas@monjalon.net>; Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Cc: dev@dpdk.org; Bruce Richardson <bruce.richardson@intel.com>
> Subject: Re: [RFC 0/4] ethdev: rework config restore
>
> External email: Use caution opening links or attachments
>
>
> On 10/8/2024 6:21 PM, Konstantin Ananyev wrote:
> >
> >
> >>>>>> We have been working on optimizing the latency of calls to
> >>>>>> rte_eth_dev_start(), on ports spawned by mlx5 PMD. Most of the
> >>>>>> work requires changes in the implementation of
> >>>>>> .dev_start() PMD callback, but I also wanted to start a
> >>>>>> discussion regarding configuration restore.
> >>>>>>
> >>>>>> rte_eth_dev_start() does a few things on top of calling .dev_start()
> callback:
> >>>>>>
> >>>>>> - Before calling it:
> >>>>>> - eth_dev_mac_restore() - if device supports
> >>>>>> RTE_ETH_DEV_NOLIVE_MAC_ADDR;
> >>>>>> - After calling it:
> >>>>>> - eth_dev_mac_restore() - if device does not support
> >>>>> RTE_ETH_DEV_NOLIVE_MAC_ADDR;
> >>>>>> - restore promiscuous config
> >>>>>> - restore all multicast config
> >>>>>>
> >>>>>> eth_dev_mac_restore() iterates over all known MAC addresses -
> >>>>>> stored in rte_eth_dev_data.mac_addrs array - and calls
> >>>>>> .mac_addr_set() and .mac_addr_add() callbacks to apply these MAC
> addresses.
> >>>>>>
> >>>>>> Promiscuous config restore checks if promiscuous mode is enabled
> >>>>>> or not, and calls .promiscuous_enable() or .promiscuous_disable()
> callback.
> >>>>>>
> >>>>>> All multicast config restore checks if all multicast mode is
> >>>>>> enabled or not, and calls .allmulticast_enable() or .allmulticast_disable()
> callback.
> >>>>>>
> >>>>>> Callbacks are called directly in all of these cases, to bypass
> >>>>>> the checks for applying the same configuration, which exist in relevant
> APIs.
> >>>>>> Checks are bypassed to force drivers to reapply the configuration.
> >>>>>>
> >>>>>> Let's consider what happens in the following sequence of API calls.
> >>>>>>
> >>>>>> 1. rte_eth_dev_configure()
> >>>>>> 2. rte_eth_tx_queue_setup()
> >>>>>> 3. rte_eth_rx_queue_setup()
> >>>>>> 4. rte_eth_promiscuous_enable()
> >>>>>> - Call dev->dev_ops->promiscuous_enable()
> >>>>>> - Stores promiscuous state in dev->data->promiscuous 5.
> >>>>>> rte_eth_allmulticast_enable()
> >>>>>> - Call dev->dev_ops->allmulticast_enable()
> >>>>>> - Stores allmulticast state in dev->data->allmulticast 6.
> >>>>>> rte_eth_dev_start()
> >>>>>> - Call dev->dev_ops->dev_start()
> >>>>>> - Call dev->dev_ops->mac_addr_set() - apply default MAC address
> >>>>>> - Call dev->dev_ops->promiscuous_enable()
> >>>>>> - Call dev->dev_ops->allmulticast_enable()
> >>>>>>
> >>>>>> Even though all configuration is available in dev->data after
> >>>>>> step 5, library forces reapplying this configuration in step 6.
> >>>>>>
> >>>>>> In mlx5 PMD case all relevant callbacks require communication
> >>>>>> with the kernel driver, to configure the device (mlx5 PMD must
> >>>>>> create/destroy new kernel flow rules and/or change netdev config).
> >>>>>>
> >>>>>> mlx5 PMD handles applying all configuration in .dev_start(), so
> >>>>>> the following forced callbacks force additional communication
> >>>>>> with the kernel. The
> >>>>> same configuration is applied multiple times.
> >>>>>>
> >>>>>> As an optimization, mlx5 PMD could check if a given configuration
> >>>>>> was applied, but this would duplicate the functionality of the
> >>>>>> library (for example rte_eth_promiscuous_enable() does not call
> >>>>>> the driver if
> >>>>>> dev->data->promiscuous is set).
> >>>>>>
> >>>>>> Question: Since all of the configuration is available before
> >>>>>> .dev_start() callback is called, why ethdev library does not
> >>>>>> expect .dev_start() to
> >>>>> take this configuration into account?
> >>>>>> In other words, why library has to reapply the configuration?
> >>>>>>
> >>>>>> I could not find any particular reason why configuration restore
> >>>>>> exists as part of the process (it was in the initial DPDK commit).
> >>>>>>
> >>>>>
> >>>>> My assumption is .dev_stop() cause these values reset in some
> >>>>> devices, so
> >>>>> .dev_start() restores them back.
> >>>>> @Bruce or @Konstantin may remember the history.
> >>>
> >>> Yep, as I remember, at least some Intel PMDs calling hw_reset() ad
> >>> dec_stop() and even dev_start() to make sure that HW is in a clean (known)
> state.
> >>>
> >>>>>
> >>>>> But I agree this is device specific behavior, and can be managed
> >>>>> by what device requires.
> >>>
> >>> Probably yes.
> >>>
> >>>>>
> >>>>>> The patches included in this RFC, propose a mechanism which would
> >>>>>> help with managing which drivers rely on forceful configuration restore.
> >>>>>> Drivers could advertise if forceful configuration restore is
> >>>>>> needed through `RTE_ETH_DEV_*_FORCE_RESTORE` device flag. If this
> >>>>>> flag is set, then the driver in question requires ethdev to
> >>>>>> forcefully restore
> >>>>> configuration.
> >>>>>>
> >>>>>
> >>>>> OK to use flag for it, but not sure about using 'dev_info->dev_flags'
> >>>>> (RTE_ETH_DEV_*) for this, as this flag is shared with user and
> >>>>> this is all dpdk internal.
> >>>>>
> >>>>> What about to have a dedicated flag for it? We can have a
> >>>>> dedicated set of flag values for restore.
> >>>>
> >>>> Agreed. What do you think about the following?
> >>>
> >>> Instead of exposing that, can we probably make it transparent to the
> >>> user and probably ethdev layer too?
> >>>
> >>
> >> +1 to make it transparent to user, but not sure if we can make it
> >> transparent to ethdev layer.
> >
> > Just to be clear:
> > Let say, using example from above:
> >
> > rte_eth_dev_start()
> > - Call dev->dev_ops->dev_start()
> > - Call dev->dev_ops->mac_addr_set() - apply default MAC address
> > - Call dev->dev_ops->promiscuous_enable()
> > - Call dev->dev_ops->allmulticast_enable()
> >
> > We probably can introduce ethdev internal function (still visible to
> > PMDs) that would do last 3 steps:
> > ethdev_replay_user_conf(...)
> > - Call dev->dev_ops->mac_addr_set() - apply default MAC address
> > - Call dev->dev_ops->promiscuous_enable()
> > - Call dev->dev_ops->allmulticast_enable()
> >
> > And let PMD itself to decide does it needs to call it at dev_start() or not.
> > So it will become:
> > rte_eth_dev_start()
> > - Call dev->dev_ops->dev_start()
> > -Call ethdev_replay_user_conf(.)
> > - Call dev->dev_ops->mac_addr_set() - apply default MAC address
> > - Call dev->dev_ops->promiscuous_enable()
> > -Call dev->dev_ops->allmulticast_enable()
> >
> > For PMDs that do need to restore user provided config And
> > rte_eth_dev_start()
> > - Call dev->dev_ops->dev_start()
> >
> > For those who do not.
> >
>
> OK, got it what you mean.
> Pushing restore functionality to PMDs works, but this may be doing redundant
> work on each PMD.
>
> Instead Dariusz suggests PMD to provide a flag to ehtdev to what to restore and
> common code in ethdev does the work.
> My below dedicated data struct comment is to have this flag in a new struct,
> overall like following:
>
> rte_eth_dev_start()
> - Call dev->dev_ops->dev_start()
> - Call dev->dev_ops->get_restore_flags(ethdev, RTE_ETH_START, &flags)
> - if (flags & MAC) dev->dev_ops->mac_addr_set()
> - if (flags & PROMISC) dev->dev_ops->promiscuous_enable()
> - ...
Could you please explain what is the benefit of exposing flags through dev_ops callback vs a dedicated flags field in rte_eth_dev_data?
In both solutions:
- config restore is transparent to the user,
- drivers can omit config restore (either by not implementing the callback or not providing the flags),
- an ABI change is introduced (not a huge concern, at least for 24.11).
I understand that my initial proposal with "internal_flags" was too vague,
but renaming and splitting this field into:
- dev_start_restore_flags
- dev_reset_restore_flags
- and so on...
seems sufficient, at least in my opinion.
>
> So PMDs only will provide what to restore with an internal API and common
> ethdev layer will restore it.
> If no restore required PMD may not implement .get_restore_flags() at all.
>
> Additionally, RTE_ETH_START, RTE_ETH_RESET etc flag can be provided to internal
> API to get what to restore in different states...
>
> >> Suggested 'internal_flag' in "struct rte_eth_dev_data" can be
> >> confusing and open to interpretation what to use it for and by time
> >> become source of defect.
> >
> > Yes, same thoughts.
> >
> >> Instead what do you think to have a separate, dedicated data struct for it?
> >
> > Hmm... not sure I understood you here...
> >
> >>
> >>> Might be we can move this restoration code into the new ethdev
> >>> helper function,(ethdevd_user_config_restore() or so) that PMD can invoke
> during its dev_start() if needed?
> >>>
> >>>>
> >>>> #define RTE_ETH_DEV_INTERNAL_PROMISC_FORCE_RESTORE RTE_BIT32(0)
> >>>> #define RTE_ETH_DEV_INTERNAL_ALLMULTI_FORCE_RESTORE RTE_BIT32(1)
> >>>> #define RTE_ETH_DEV_INTERNAL_MAC_ADDR_FORCE_RESTORE
> RTE_BIT32(2)
> >>>>
> >>>> struct rte_eth_dev_data {
> >>>> /* snip */
> >>>>
> >>>> uint32_t dev_flags;
> >>>>
> >>>> /**
> >>>> * Internal device capabilities, used only by ethdev library.
> >>>> * Certain functionalities provided by the library might enabled/disabled,
> >>>> * based on driver exposing certain capabilities.
> >>>> */
> >>>> uint32_t internal_flags;
> >>>>
> >>>> /* snip */
> >>>> };
> >>>>
> >>>>> Also perhaps we have go into details what needs to be restored
> >>>>> after 'stop' and what needs to be restored after 'reset' and use similar
> mechanism etc...
> >>>>
> >>>> I think we should look into that.
> >>>> Any 'codification' of semantics between drivers and ethdev library is good in
> my opinion.
> >>>>
> >>>> At least right now, ethdev does not change any configuration in 'stop' and
> 'reset' from what I see.
> >>>> But that's on library side only.
> >>>>
> >>>>>> This way, if we would conclude that it makes sense for
> >>>>>> .dev_start() to handle all starting configuration aspects, we
> >>>>>> could track which drivers still rely
> >>>>> on configuration restore.
> >>>>>>
> >>>>>> Dariusz Sosnowski (4):
> >>>>>> ethdev: rework config restore
> >>>>>> ethdev: omit promiscuous config restore if not required
> >>>>>> ethdev: omit all multicast config restore if not required
> >>>>>> ethdev: omit MAC address restore if not required
> >>>>>>
> >>>>>> lib/ethdev/rte_ethdev.c | 39
> >>>>>> ++++++++++++++++++++++++++++++++++-----
> >>>>>> lib/ethdev/rte_ethdev.h | 18 ++++++++++++++++++
> >>>>>> 2 files changed, 52 insertions(+), 5 deletions(-)
> >>>>>>
> >>>>>> --
> >>>>>> 2.39.5
> >>>>>>
> >>>
> >
^ permalink raw reply [relevance 3%]
* Re: [PATCH v3 0/5] add feature arc in rte_graph
2024-10-09 13:29 3% ` [PATCH v3 " Nitin Saxena
2024-10-09 13:29 4% ` [PATCH v3 2/5] graph: add feature arc option in graph create Nitin Saxena
@ 2024-10-09 14:21 0% ` Christophe Fontaine
2024-10-09 17:37 0% ` Stephen Hemminger
2 siblings, 0 replies; 200+ results
From: Christophe Fontaine @ 2024-10-09 14:21 UTC (permalink / raw)
To: Nitin Saxena
Cc: Jerin Jacob, Kiran Kumar K, Nithin Dabilpuram, Zhirun Yan,
Robin Jarry, dev, Nitin Saxena
> On 9 Oct 2024, at 15:29, Nitin Saxena <nsaxena@marvell.com> wrote:
>
> Feature arc represents an ordered list of features/protocols at a given
> networking layer. It is a high level abstraction to connect various
> rte_graph nodes, as feature nodes, and allow packets steering across
> these nodes in a generic manner.
>
> Features (or feature nodes) are nodes which handles partial or complete
> handling of a protocol in fast path. Like ipv4-rewrite node, which adds
> rewrite data to an outgoing IPv4 packet.
>
> However in above example, outgoing interface(say "eth0") may have
> outbound IPsec policy enabled, hence packets must be steered from
> ipv4-rewrite node to ipsec-outbound-policy node for outbound IPsec
> policy lookup. On the other hand, packets routed to another interface
> (eth1) will not be sent to ipsec-outbound-policy node as IPsec feature
> is disabled on eth1. Feature-arc allows rte_graph applications to manage
> such constraints easily
>
> Feature arc abstraction allows rte_graph based application to
>
> 1. Seamlessly steer packets across feature nodes based on whether
> feature is enabled or disabled on an interface. Features enabled on one
> interface may not be enabled on another interface with in a same feature
> arc.
>
> 2. Allow enabling/disabling of features on an interface at runtime,
> so that if a feature is disabled, packets associated with that interface
> won't be steered to corresponding feature node.
>
> 3. Provides mechanism to hook custom/user-defined nodes to a feature
> node and allow packet steering from feature node to custom node without
> changing former's fast path function
Hi,
As a general comment, I would have expected a modification on rte_graph_walk so the nodes would *not* need to be aware of the enabled feature arcs.
Yet, in the series, the nodes need to be aware if a feature arc is enabled or not.
Isn’t this a contradiction or did I miss something ?
Christophe
>
> 4. Allow expressing features in a particular sequential order so that
> packets are steered in an ordered way across nodes in fast path. For
> eg: if IPsec and IPv4 features are enabled on an ingress interface,
> packets must be sent to IPsec inbound policy node first and then to ipv4
> lookup node.
>
> This patch series adds feature arc library in rte_graph and also adds
> "ipv4-output" feature arc handling in "ipv4-rewrite" node.
>
> Changes in v3:
> - rte_graph_feature_arc_t typedef from uint64_t to uintptr_t to fix
> compilation on 32-bit machine
> - Updated images in .png format
> - Added ABI change section in release notes
> - Fixed DPDK CI failures
>
> Changes in v2:
> - Added unit tests for feature arc
> - Fixed issues found in testing
> - Added new public APIs rte_graph_feature_arc_feature_to_node(),
> rte_graph_feature_arc_feature_to_name(),
> rte_graph_feature_arc_num_features()
> - Added programming guide for feature arc
> - Added release notes for feature arc
>
> Nitin Saxena (5):
> graph: add feature arc support
> graph: add feature arc option in graph create
> graph: add IPv4 output feature arc
> test/graph_feature_arc: add functional tests
> docs: add programming guide for feature arc
>
> app/test/meson.build | 1 +
> app/test/test_graph_feature_arc.c | 1410 +++++++++++++++++++
> doc/guides/prog_guide/graph_lib.rst | 288 ++++
> doc/guides/prog_guide/img/feature_arc-1.png | Bin 0 -> 61532 bytes
> doc/guides/prog_guide/img/feature_arc-2.png | Bin 0 -> 155806 bytes
> doc/guides/prog_guide/img/feature_arc-3.png | Bin 0 -> 143697 bytes
> doc/guides/rel_notes/release_24_11.rst | 13 +
> lib/graph/graph.c | 1 +
> lib/graph/graph_feature_arc.c | 1223 ++++++++++++++++
> lib/graph/graph_populate.c | 7 +-
> lib/graph/graph_private.h | 3 +
> lib/graph/meson.build | 2 +
> lib/graph/node.c | 2 +
> lib/graph/rte_graph.h | 3 +
> lib/graph/rte_graph_feature_arc.h | 431 ++++++
> lib/graph/rte_graph_feature_arc_worker.h | 674 +++++++++
> lib/graph/version.map | 20 +
> lib/node/ip4_rewrite.c | 476 +++++--
> lib/node/ip4_rewrite_priv.h | 15 +-
> lib/node/node_private.h | 20 +-
> lib/node/rte_node_ip4_api.h | 3 +
> 21 files changed, 4494 insertions(+), 98 deletions(-)
> create mode 100644 app/test/test_graph_feature_arc.c
> create mode 100644 doc/guides/prog_guide/img/feature_arc-1.png
> create mode 100644 doc/guides/prog_guide/img/feature_arc-2.png
> create mode 100644 doc/guides/prog_guide/img/feature_arc-3.png
> create mode 100644 lib/graph/graph_feature_arc.c
> create mode 100644 lib/graph/rte_graph_feature_arc.h
> create mode 100644 lib/graph/rte_graph_feature_arc_worker.h
>
> --
> 2.43.0
>
^ permalink raw reply [relevance 0%]
* Re: [PATCH] cryptodev: remove unnecessary list end
2024-10-09 11:24 3% ` [PATCH] cryptodev: remove unnecessary list end Akhil Goyal
2024-10-09 12:52 0% ` Morten Brørup
@ 2024-10-09 14:06 0% ` Hemant Agrawal
1 sibling, 0 replies; 200+ results
From: Hemant Agrawal @ 2024-10-09 14:06 UTC (permalink / raw)
To: dev
On 09-10-2024 16:54, Akhil Goyal wrote:
> RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END and
> RTE_CRYPTO_RSA_PADDING_TYPE_LIST_END are not used anywhere and
> mostly it do not fulfil any objective.
> Rather it does not allow addition of new algorithms and
> new padding type as it result in ABI breakage.
> Hence, these are removed.
>
> Signed-off-by: Akhil Goyal <gakhil@marvell.com>
> ---
> lib/cryptodev/rte_crypto_asym.h | 3 ---
> 1 file changed, 3 deletions(-)
>
> diff --git a/lib/cryptodev/rte_crypto_asym.h b/lib/cryptodev/rte_crypto_asym.h
> index 39d3da3952..524753e2b2 100644
> --- a/lib/cryptodev/rte_crypto_asym.h
> +++ b/lib/cryptodev/rte_crypto_asym.h
> @@ -119,8 +119,6 @@ enum rte_crypto_asym_xform_type {
> * Performs Encrypt, Decrypt, Sign and Verify.
> * Refer to rte_crypto_asym_op_type.
> */
> - RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
> - /**< End of list */
> };
>
> /**
> @@ -168,7 +166,6 @@ enum rte_crypto_rsa_padding_type {
> /**< RSA PKCS#1 OAEP padding scheme */
> RTE_CRYPTO_RSA_PADDING_PSS,
> /**< RSA PKCS#1 PSS padding scheme */
> - RTE_CRYPTO_RSA_PADDING_TYPE_LIST_END
> };
>
> /**
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
^ permalink raw reply [relevance 0%]
* [PATCH v3 2/5] graph: add feature arc option in graph create
2024-10-09 13:29 3% ` [PATCH v3 " Nitin Saxena
@ 2024-10-09 13:29 4% ` Nitin Saxena
2024-10-09 14:21 0% ` [PATCH v3 0/5] add feature arc in rte_graph Christophe Fontaine
2024-10-09 17:37 0% ` Stephen Hemminger
2 siblings, 0 replies; 200+ results
From: Nitin Saxena @ 2024-10-09 13:29 UTC (permalink / raw)
To: Jerin Jacob, Kiran Kumar K, Nithin Dabilpuram, Zhirun Yan,
Robin Jarry, Christophe Fontaine
Cc: dev, Nitin Saxena, Pavan Nikhilesh
Added option in graph create to call feature-specific process node
functions. This removes extra overhead for checking feature arc status
in nodes where application is not using feature arc processing
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Signed-off-by: Nitin Saxena <nsaxena@marvell.com>
---
doc/guides/rel_notes/release_24_11.rst | 3 +++
lib/graph/graph.c | 1 +
lib/graph/graph_populate.c | 7 ++++++-
lib/graph/graph_private.h | 3 +++
lib/graph/node.c | 2 ++
lib/graph/rte_graph.h | 3 +++
6 files changed, 18 insertions(+), 1 deletion(-)
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index d6d64518e0..5195badf54 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -122,6 +122,9 @@ ABI Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* graph: Added new `feature_arc_enable` parameter in `struct rte_graph_param` to
+ allow `rte_graph_walk()` call `feat_arc_proc` node callback function, if it
+ is provided in node registration
Known Issues
------------
diff --git a/lib/graph/graph.c b/lib/graph/graph.c
index d5b8c9f918..b0ad3a83ae 100644
--- a/lib/graph/graph.c
+++ b/lib/graph/graph.c
@@ -455,6 +455,7 @@ rte_graph_create(const char *name, struct rte_graph_param *prm)
graph->parent_id = RTE_GRAPH_ID_INVALID;
graph->lcore_id = RTE_MAX_LCORE;
graph->num_pkt_to_capture = prm->num_pkt_to_capture;
+ graph->feature_arc_enabled = prm->feature_arc_enable;
if (prm->pcap_filename)
rte_strscpy(graph->pcap_filename, prm->pcap_filename, RTE_GRAPH_PCAP_FILE_SZ);
diff --git a/lib/graph/graph_populate.c b/lib/graph/graph_populate.c
index ed596a7711..5d8aa7b903 100644
--- a/lib/graph/graph_populate.c
+++ b/lib/graph/graph_populate.c
@@ -79,8 +79,13 @@ graph_nodes_populate(struct graph *_graph)
if (graph_pcap_is_enable()) {
node->process = graph_pcap_dispatch;
node->original_process = graph_node->node->process;
- } else
+ if (_graph->feature_arc_enabled && graph_node->node->feat_arc_proc)
+ node->original_process = graph_node->node->feat_arc_proc;
+ } else {
node->process = graph_node->node->process;
+ if (_graph->feature_arc_enabled && graph_node->node->feat_arc_proc)
+ node->process = graph_node->node->feat_arc_proc;
+ }
memcpy(node->name, graph_node->node->name, RTE_GRAPH_NAMESIZE);
pid = graph_node->node->parent_id;
if (pid != RTE_NODE_ID_INVALID) { /* Cloned node */
diff --git a/lib/graph/graph_private.h b/lib/graph/graph_private.h
index d557d55f2d..58ba0abeff 100644
--- a/lib/graph/graph_private.h
+++ b/lib/graph/graph_private.h
@@ -56,6 +56,7 @@ struct node {
unsigned int lcore_id;
/**< Node runs on the Lcore ID used for mcore dispatch model. */
rte_node_process_t process; /**< Node process function. */
+ rte_node_process_t feat_arc_proc; /**< Node feature-arch process function. */
rte_node_init_t init; /**< Node init function. */
rte_node_fini_t fini; /**< Node fini function. */
rte_node_t id; /**< Allocated identifier for the node. */
@@ -126,6 +127,8 @@ struct graph {
/**< Number of packets to be captured per core. */
char pcap_filename[RTE_GRAPH_PCAP_FILE_SZ];
/**< pcap file name/path. */
+ uint8_t feature_arc_enabled;
+ /**< Graph feature arc. */
STAILQ_HEAD(gnode_list, graph_node) node_list;
/**< Nodes in a graph. */
};
diff --git a/lib/graph/node.c b/lib/graph/node.c
index 99a9622779..d8fd273543 100644
--- a/lib/graph/node.c
+++ b/lib/graph/node.c
@@ -90,6 +90,7 @@ __rte_node_register(const struct rte_node_register *reg)
goto free;
node->flags = reg->flags;
node->process = reg->process;
+ node->feat_arc_proc = reg->feat_arc_proc;
node->init = reg->init;
node->fini = reg->fini;
node->nb_edges = reg->nb_edges;
@@ -137,6 +138,7 @@ node_clone(struct node *node, const char *name)
/* Clone the source node */
reg->flags = node->flags;
reg->process = node->process;
+ reg->feat_arc_proc = node->feat_arc_proc;
reg->init = node->init;
reg->fini = node->fini;
reg->nb_edges = node->nb_edges;
diff --git a/lib/graph/rte_graph.h b/lib/graph/rte_graph.h
index ecfec2068a..1a3bd7e1ba 100644
--- a/lib/graph/rte_graph.h
+++ b/lib/graph/rte_graph.h
@@ -172,6 +172,8 @@ struct rte_graph_param {
uint32_t mp_capacity; /**< Capacity of memory pool for dispatch model. */
} dispatch;
};
+
+ bool feature_arc_enable; /**< Enable Graph feature arc. */
};
/**
@@ -470,6 +472,7 @@ struct rte_node_register {
uint64_t flags; /**< Node configuration flag. */
#define RTE_NODE_SOURCE_F (1ULL << 0) /**< Node type is source. */
rte_node_process_t process; /**< Node process function. */
+ rte_node_process_t feat_arc_proc; /**< Node feature-arch process function. */
rte_node_init_t init; /**< Node init function. */
rte_node_fini_t fini; /**< Node fini function. */
rte_node_t id; /**< Node Identifier. */
--
2.43.0
^ permalink raw reply [relevance 4%]
* [PATCH v3 0/5] add feature arc in rte_graph
@ 2024-10-09 13:29 3% ` Nitin Saxena
2024-10-09 13:29 4% ` [PATCH v3 2/5] graph: add feature arc option in graph create Nitin Saxena
` (2 more replies)
0 siblings, 3 replies; 200+ results
From: Nitin Saxena @ 2024-10-09 13:29 UTC (permalink / raw)
To: Jerin Jacob, Kiran Kumar K, Nithin Dabilpuram, Zhirun Yan,
Robin Jarry, Christophe Fontaine
Cc: dev, Nitin Saxena
Feature arc represents an ordered list of features/protocols at a given
networking layer. It is a high level abstraction to connect various
rte_graph nodes, as feature nodes, and allow packets steering across
these nodes in a generic manner.
Features (or feature nodes) are nodes which handles partial or complete
handling of a protocol in fast path. Like ipv4-rewrite node, which adds
rewrite data to an outgoing IPv4 packet.
However in above example, outgoing interface(say "eth0") may have
outbound IPsec policy enabled, hence packets must be steered from
ipv4-rewrite node to ipsec-outbound-policy node for outbound IPsec
policy lookup. On the other hand, packets routed to another interface
(eth1) will not be sent to ipsec-outbound-policy node as IPsec feature
is disabled on eth1. Feature-arc allows rte_graph applications to manage
such constraints easily
Feature arc abstraction allows rte_graph based application to
1. Seamlessly steer packets across feature nodes based on whether
feature is enabled or disabled on an interface. Features enabled on one
interface may not be enabled on another interface with in a same feature
arc.
2. Allow enabling/disabling of features on an interface at runtime,
so that if a feature is disabled, packets associated with that interface
won't be steered to corresponding feature node.
3. Provides mechanism to hook custom/user-defined nodes to a feature
node and allow packet steering from feature node to custom node without
changing former's fast path function
4. Allow expressing features in a particular sequential order so that
packets are steered in an ordered way across nodes in fast path. For
eg: if IPsec and IPv4 features are enabled on an ingress interface,
packets must be sent to IPsec inbound policy node first and then to ipv4
lookup node.
This patch series adds feature arc library in rte_graph and also adds
"ipv4-output" feature arc handling in "ipv4-rewrite" node.
Changes in v3:
- rte_graph_feature_arc_t typedef from uint64_t to uintptr_t to fix
compilation on 32-bit machine
- Updated images in .png format
- Added ABI change section in release notes
- Fixed DPDK CI failures
Changes in v2:
- Added unit tests for feature arc
- Fixed issues found in testing
- Added new public APIs rte_graph_feature_arc_feature_to_node(),
rte_graph_feature_arc_feature_to_name(),
rte_graph_feature_arc_num_features()
- Added programming guide for feature arc
- Added release notes for feature arc
Nitin Saxena (5):
graph: add feature arc support
graph: add feature arc option in graph create
graph: add IPv4 output feature arc
test/graph_feature_arc: add functional tests
docs: add programming guide for feature arc
app/test/meson.build | 1 +
app/test/test_graph_feature_arc.c | 1410 +++++++++++++++++++
doc/guides/prog_guide/graph_lib.rst | 288 ++++
doc/guides/prog_guide/img/feature_arc-1.png | Bin 0 -> 61532 bytes
doc/guides/prog_guide/img/feature_arc-2.png | Bin 0 -> 155806 bytes
doc/guides/prog_guide/img/feature_arc-3.png | Bin 0 -> 143697 bytes
doc/guides/rel_notes/release_24_11.rst | 13 +
lib/graph/graph.c | 1 +
lib/graph/graph_feature_arc.c | 1223 ++++++++++++++++
lib/graph/graph_populate.c | 7 +-
lib/graph/graph_private.h | 3 +
lib/graph/meson.build | 2 +
lib/graph/node.c | 2 +
lib/graph/rte_graph.h | 3 +
lib/graph/rte_graph_feature_arc.h | 431 ++++++
lib/graph/rte_graph_feature_arc_worker.h | 674 +++++++++
lib/graph/version.map | 20 +
lib/node/ip4_rewrite.c | 476 +++++--
lib/node/ip4_rewrite_priv.h | 15 +-
lib/node/node_private.h | 20 +-
lib/node/rte_node_ip4_api.h | 3 +
21 files changed, 4494 insertions(+), 98 deletions(-)
create mode 100644 app/test/test_graph_feature_arc.c
create mode 100644 doc/guides/prog_guide/img/feature_arc-1.png
create mode 100644 doc/guides/prog_guide/img/feature_arc-2.png
create mode 100644 doc/guides/prog_guide/img/feature_arc-3.png
create mode 100644 lib/graph/graph_feature_arc.c
create mode 100644 lib/graph/rte_graph_feature_arc.h
create mode 100644 lib/graph/rte_graph_feature_arc_worker.h
--
2.43.0
^ permalink raw reply [relevance 3%]
* RE: [PATCH] cryptodev: remove unnecessary list end
2024-10-09 11:24 3% ` [PATCH] cryptodev: remove unnecessary list end Akhil Goyal
@ 2024-10-09 12:52 0% ` Morten Brørup
2024-10-09 20:38 0% ` Akhil Goyal
2024-10-09 14:06 0% ` Hemant Agrawal
1 sibling, 1 reply; 200+ results
From: Morten Brørup @ 2024-10-09 12:52 UTC (permalink / raw)
To: Akhil Goyal, dev
Cc: thomas, david.marchand, hemant.agrawal, anoobj,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde, rnagadheeraj,
ferruh.yigit
> From: Akhil Goyal [mailto:gakhil@marvell.com]
> Sent: Wednesday, 9 October 2024 13.25
>
> RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END and
> RTE_CRYPTO_RSA_PADDING_TYPE_LIST_END are not used anywhere and
> mostly it do not fulfil any objective.
> Rather it does not allow addition of new algorithms and
> new padding type as it result in ABI breakage.
> Hence, these are removed.
>
> Signed-off-by: Akhil Goyal <gakhil@marvell.com>
> ---
> lib/cryptodev/rte_crypto_asym.h | 3 ---
> 1 file changed, 3 deletions(-)
>
> diff --git a/lib/cryptodev/rte_crypto_asym.h
> b/lib/cryptodev/rte_crypto_asym.h
> index 39d3da3952..524753e2b2 100644
> --- a/lib/cryptodev/rte_crypto_asym.h
> +++ b/lib/cryptodev/rte_crypto_asym.h
> @@ -119,8 +119,6 @@ enum rte_crypto_asym_xform_type {
> * Performs Encrypt, Decrypt, Sign and Verify.
> * Refer to rte_crypto_asym_op_type.
> */
> - RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
> - /**< End of list */
> };
>
> /**
> @@ -168,7 +166,6 @@ enum rte_crypto_rsa_padding_type {
> /**< RSA PKCS#1 OAEP padding scheme */
> RTE_CRYPTO_RSA_PADDING_PSS,
> /**< RSA PKCS#1 PSS padding scheme */
> - RTE_CRYPTO_RSA_PADDING_TYPE_LIST_END
> };
>
> /**
> --
> 2.25.1
Acked-by: Morten Brørup <mb@smartsharesystems.com>
^ permalink raw reply [relevance 0%]
* [PATCH] cryptodev: remove unnecessary list end
2024-09-05 10:14 4% [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs Akhil Goyal
2024-09-05 15:09 4% ` Morten Brørup
2024-10-04 3:54 5% ` Ferruh Yigit
@ 2024-10-09 11:24 3% ` Akhil Goyal
2024-10-09 12:52 0% ` Morten Brørup
2024-10-09 14:06 0% ` Hemant Agrawal
2 siblings, 2 replies; 200+ results
From: Akhil Goyal @ 2024-10-09 11:24 UTC (permalink / raw)
To: dev
Cc: thomas, david.marchand, hemant.agrawal, anoobj,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde, rnagadheeraj,
mb, ferruh.yigit, Akhil Goyal
RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END and
RTE_CRYPTO_RSA_PADDING_TYPE_LIST_END are not used anywhere and
mostly it do not fulfil any objective.
Rather it does not allow addition of new algorithms and
new padding type as it result in ABI breakage.
Hence, these are removed.
Signed-off-by: Akhil Goyal <gakhil@marvell.com>
---
lib/cryptodev/rte_crypto_asym.h | 3 ---
1 file changed, 3 deletions(-)
diff --git a/lib/cryptodev/rte_crypto_asym.h b/lib/cryptodev/rte_crypto_asym.h
index 39d3da3952..524753e2b2 100644
--- a/lib/cryptodev/rte_crypto_asym.h
+++ b/lib/cryptodev/rte_crypto_asym.h
@@ -119,8 +119,6 @@ enum rte_crypto_asym_xform_type {
* Performs Encrypt, Decrypt, Sign and Verify.
* Refer to rte_crypto_asym_op_type.
*/
- RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
- /**< End of list */
};
/**
@@ -168,7 +166,6 @@ enum rte_crypto_rsa_padding_type {
/**< RSA PKCS#1 OAEP padding scheme */
RTE_CRYPTO_RSA_PADDING_PSS,
/**< RSA PKCS#1 PSS padding scheme */
- RTE_CRYPTO_RSA_PADDING_TYPE_LIST_END
};
/**
--
2.25.1
^ permalink raw reply [relevance 3%]
* RE: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-10-06 11:10 0% ` Morten Brørup
@ 2024-10-09 11:21 0% ` Akhil Goyal
0 siblings, 0 replies; 200+ results
From: Akhil Goyal @ 2024-10-09 11:21 UTC (permalink / raw)
To: Morten Brørup, fengchengwen, dev
Cc: thomas, david.marchand, hemant.agrawal, Anoob Joseph,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde,
Nagadheeraj Rottela, mdr
> > From: Akhil Goyal [mailto:gakhil@marvell.com]
> > Sent: Thursday, 3 October 2024 09.01
> >
> > Hi Morten,
> > >
> > > Apologies for delayed response.
> > > > Maybe a combination, returning the lowest end of the two versions
> > of the list,
> > > > would work...
> > > >
> > > > ----------------------------------
> > > > Common header file (rte_common.h):
> > > > ----------------------------------
> > > >
> > > > /* Add at end of enum list in the header file. */
> > > > #define RTE_ENUM_LIST_END(name) \
> > > > _ # name # _ENUM_LIST_END /**< @internal */
> > > >
> > > > /* Add somewhere in header file, preferably after the enum list. */
> > > > #define rte_declare_enum_list_end(name) \
> > > > /** @internal */ \
> > > > int _# name # _enum_list_end(void); \
> > > > \
> > > > static int name # _enum_list_end(void) \
> > > > { \
> > > > static int cached = 0; \
> > > > \
> > > > if (likely(cached != 0)) \
> > > > return cached; \
> > > > \
> > > > return cached = RTE_MIN( \
> > > > RTE_ENUM_LIST_END(name), \
> > > > _ # name # _enum_list_end()); \
> > > > } \
> > > > \
> > > > int _# name # _enum_list_end(void)
> > > >
> > > > /* Add in the library/driver implementation. */
> > > > #define rte_define_enum_list_end(name) \
> > > > int _# name # _enum_list_end(void) \
> > > > { \
> > > > return RTE_ENUM_LIST_END(name); \
> > > > } \
> > > > \
> > > > int _# name # _enum_list_end(void)
> > > >
> > > > --------------------
> > > > Library header file:
> > > > --------------------
> > > >
> > > > enum rte_crypto_asym_op_type {
> > > > RTE_CRYPTO_ASYM_OP_VERIFY,
> > > > /**< Signature Verification operation */
> > > > RTE_ENUM_LIST_END(rte_crypto_asym_op)
> > >
> > > Will the ABI check be ok for adding anything in between
> > > RTE_CRYPTO_ASYM_OP_VERIFY and
> > > RTE_ENUM_LIST_END(rte_crypto_asym_op)?
> > > Don’t we need to add exception for that if we somehow make it
> > internal by
> > > adding a comment only?
> > > Library is actually not restricting the application to not use
> > > RTE_ENUM_LIST_END(rte_crypto_asym_op) directly.
> > >
> > > Also we may need to expose the .c file internal function as
> > experimental in
> > > version.map
> > >
> > > > }
> > > >
> > > > rte_declare_enum_list_end(rte_crypto_asym_op);
> > > >
> > > > ---------------
> > > > Library C file:
> > > > ---------------
> > > >
> > > > rte_define_enum_list_end(rte_crypto_asym_op);
> > >
> > > If we want to make it a generic thing in rte_common.h
> > > Will the below change be ok?
> > > ----------------------------------
> > > Common header file (rte_common.h):
> > > ----------------------------------
> > > #define rte_define_enum_list_end(name, last_value) \
> > > static inline int name ## _enum_list_end(void) \
> > > { \
> > > return last_value + 1; \
> > > }
> > >
> > > ----------------
> > > Lib header file
> > > ----------------
> > > //After the enum definition define the list end as below
> > > rte_define_enum_list_end(rte_crypto_asym_op,
> > > RTE_CRYPTO_ASYM_OP_VERIFY);
> > >
> > >
> > > And wherever list end is needed use
> > rte_crypto_asym_op_enum_list_end()?
> > >
> > > With this change, abi check will not complain for any new addition at
> > the end of
> > > enum.
> > > And we do not need to expose any internal API in version.map.
> > >
> > Can we move forward with above suggestion?
>
> Sorry about the late reply, Akhil.
>
> It seems Ferruh and David have picked up this discussion with good arguments.
>
> I have no preferences for a generic solution, especially if this is an isolated case. A
> generic solution can be added at any time later.
Ok, since we are not talking about a generic solution.
We can remove RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
And RTE_CRYPTO_RSA_PADDING_TYPE_LIST_END.
These enums are not used anywhere and are not required at all.
And for RTE_CRYPTO_ASYM_OP_LIST_END, we can keep it as is.
The rte_crypto_asym_op_type is not expected to grow in future and
We need the list end for loops as well as arrays.
Hence sending a patch to remove RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
And RTE_CRYPTO_RSA_PADDING_TYPE_LIST_END.
This is inline with symmetric crypto xform type list.
^ permalink raw reply [relevance 0%]
* Re: [PATCH v2 1/3] net: add thread-safe crc api
2024-10-09 7:48 0% ` Kusztal, ArkadiuszX
@ 2024-10-09 9:11 0% ` Ferruh Yigit
0 siblings, 0 replies; 200+ results
From: Ferruh Yigit @ 2024-10-09 9:11 UTC (permalink / raw)
To: Kusztal, ArkadiuszX, Marchand, David; +Cc: dev, Ji, Kai, Dooley, Brian
On 10/9/2024 8:48 AM, Kusztal, ArkadiuszX wrote:
>
>
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@amd.com>
>> Sent: Wednesday, October 9, 2024 3:03 AM
>> To: Kusztal, ArkadiuszX <arkadiuszx.kusztal@intel.com>; Marchand, David
>> <david.marchand@redhat.com>
>> Cc: dev@dpdk.org; Ji, Kai <kai.ji@intel.com>; Dooley, Brian
>> <brian.dooley@intel.com>
>> Subject: Re: [PATCH v2 1/3] net: add thread-safe crc api
>>
>> On 10/8/2024 9:51 PM, Kusztal, ArkadiuszX wrote:
>>> Hi Ferruh,
>>> Thanks for the review, comments inline,
>>>
>>>> -----Original Message-----
>>>> From: Ferruh Yigit <ferruh.yigit@amd.com>
>>>> Sent: Tuesday, October 8, 2024 5:43 AM
>>>> To: Kusztal, ArkadiuszX <arkadiuszx.kusztal@intel.com>; Marchand,
>>>> David <david.marchand@redhat.com>
>>>> Cc: dev@dpdk.org; Ji, Kai <kai.ji@intel.com>; Dooley, Brian
>>>> <brian.dooley@intel.com>
>>>> Subject: Re: [PATCH v2 1/3] net: add thread-safe crc api
>>>>
>>>> On 10/1/2024 7:11 PM, Arkadiusz Kusztal wrote:
>>>>> The current net CRC API is not thread-safe, this patch solves this
>>>>> by adding another, thread-safe API functions.
>>>>> This API is also safe to use across multiple processes, yet with
>>>>> limitations on max-simd-bitwidth, which will be checked only by the
>>>>> process that created the CRC context; all other processes will use
>>>>> the same CRC function when used with the same CRC context.
>>>>> It is an undefined behavior when process binaries are compiled with
>>>>> different SIMD capabilities when the same CRC context is used.
>>>>>
>>>>> Signed-off-by: Arkadiusz Kusztal <arkadiuszx.kusztal@intel.com>
>>>>
>>>> <...>
>>>>
>>>>> +static struct
>>>>> +{
>>>>> + uint32_t (*f[RTE_NET_CRC_REQS])
>>>>> + (const uint8_t *data, uint32_t data_len);
>>>>>
>>>>
>>>> It increases readability to typedef function pointers.
>>>
>>> Agree, though this typedef would be used here only, that’s why I left it out.
>>> But I can add it then.
>>>
>>>>
>>>>
>>>>> +} handlers[RTE_NET_CRC_AVX512 + 1];
>>>>>
>>>>> -/**
>>>>> - * Reflect the bits about the middle
>>>>> - *
>>>>> - * @param val
>>>>> - * value to be reflected
>>>>> - *
>>>>> - * @return
>>>>> - * reflected value
>>>>> - */
>>>>> -static uint32_t
>>>>> +static inline uint32_t
>>>>>
>>>>
>>>> Does changing to 'inline' required, as function is static compiler
>>>> can do the same.
>>>
>>> True, though it may be more readable sometimes.
>>> Of course there is no way that in O3 these functions would not be inlined by
>> the compiler, regardless if the inline hint is present or not.
>>>
>>>>
>>>>> reflect_32bits(uint32_t val)
>>>>> {
>>>>> uint32_t i, res = 0;
>>>>> @@ -99,26 +43,7 @@ reflect_32bits(uint32_t val)
>>>>> return res;
>>>>> }
>>>>>
>>>>> -static void
>>>>> -crc32_eth_init_lut(uint32_t poly,
>>>>> - uint32_t *lut)
>>>>> -{
>>>>> - uint32_t i, j;
>>>>> -
>>>>> - for (i = 0; i < CRC_LUT_SIZE; i++) {
>>>>> - uint32_t crc = reflect_32bits(i);
>>>>> -
>>>>> - for (j = 0; j < 8; j++) {
>>>>> - if (crc & 0x80000000L)
>>>>> - crc = (crc << 1) ^ poly;
>>>>> - else
>>>>> - crc <<= 1;
>>>>> - }
>>>>> - lut[i] = reflect_32bits(crc);
>>>>> - }
>>>>> -}
>>>>> -
>>>>> -static __rte_always_inline uint32_t
>>>>> +static inline uint32_t
>>>>>
>>>>
>>>> Why not forcing inline anymore?
>>>> Are these inline changes related to the thread-safety?
>>>
>>> O3 will inline it anyway, and with always_inline it will be inline even in debug
>> mode. I just see no reason forcing it upon the compiler.
>>>
>>>>
>>>>> crc32_eth_calc_lut(const uint8_t *data,
>>>>> uint32_t data_len,
>>>>> uint32_t crc,
>>>>> @@ -130,20 +55,9 @@ crc32_eth_calc_lut(const uint8_t *data,
>>>>> return crc;
>>>>> }
>>>>>
>>>>> -static void
>>>>> -rte_net_crc_scalar_init(void)
>>>>> -{
>>>>> - /* 32-bit crc init */
>>>>> - crc32_eth_init_lut(CRC32_ETH_POLYNOMIAL, crc32_eth_lut);
>>>>> -
>>>>> - /* 16-bit CRC init */
>>>>> - crc32_eth_init_lut(CRC16_CCITT_POLYNOMIAL << 16, crc16_ccitt_lut);
>>>>> -}
>>>>> -
>>>>> static inline uint32_t
>>>>> -rte_crc16_ccitt_handler(const uint8_t *data, uint32_t data_len)
>>>>> +crc16_ccitt(const uint8_t *data, uint32_t data_len)
>>>>> {
>>>>> - /* return 16-bit CRC value */
>>>>>
>>>>
>>>> Why not keep comments? Are they wrong?
>>>
>>> Functions names are very self-explanatory, that’s why I dropped comments. I
>> can add comments if needed.
>>>
>>
>> I am for restricting changes to the target of the patch which is making CRC
>> calculation thread safe, unless code changes invalidates the comments, lets
>> keep them. Same goes with inline related modifications.
>>
>>>>
>>>> <...>
>>>>
>>>>> +static void
>>>>> +crc_scalar_init(void)
>>>>> +{
>>>>> + crc32_eth_init_lut(CRC32_ETH_POLYNOMIAL, crc32_eth_lut);
>>>>> + crc32_eth_init_lut(CRC16_CCITT_POLYNOMIAL << 16, crc16_ccitt_lut);
>>>>> +
>>>>> + handlers[RTE_NET_CRC_SCALAR].f[RTE_NET_CRC16_CCITT] =
>>>> crc16_ccitt;
>>>>> + handlers[RTE_NET_CRC_SCALAR].f[RTE_NET_CRC32_ETH] = crc32_eth;
>>>>>
>>>>
>>>> +1 to remove global handlers pointer and add context,
>>>>
>>>> But current handlers array content is static, it can be set when
>>>> defined, instead of functions.
>>>
>>> Can do it for scalars, but for SIMD there is this runtime check like this:
>>> if (AVX512_VPCLMULQDQ_CPU_SUPPORTED) { So compiled binary on
>> AVX512
>>> machine could filter it out on the machine which does not support it.
>>> There is no NULL check in crc function so it would not change much when
>> called -> Invalid address vs Invalid instruction.
>>> There are not many checks there, as this is CRC after all, it should be a small as
>> possible, yet probably NULL check could be advisable in crc function then.
>>>
>>
>> There is already AVX512_VPCLMULQDQ_CPU_SUPPORTED etc checks in
>> 'rte_net_crc_set()'.
>> So does it work to update 'handlers' statically, without condition, but have
>> conditions when use them.
>>
>>>
>>>>
>>>> <...>
>>>>
>>>>> -static uint32_t
>>>>> -rte_crc32_eth_default_handler(const uint8_t *data, uint32_t
>>>>> data_len)
>>>>> +struct rte_net_crc rte_net_crc_set(enum rte_net_crc_alg alg,
>>>>> + enum rte_net_crc_type type)
>>>>> {
>>>>> - handlers = NULL;
>>>>> - if (max_simd_bitwidth == 0)
>>>>> - max_simd_bitwidth = rte_vect_get_max_simd_bitwidth();
>>>>> -
>>>>> - handlers = avx512_vpclmulqdq_get_handlers();
>>>>> - if (handlers != NULL)
>>>>> - return handlers[RTE_NET_CRC32_ETH](data, data_len);
>>>>> - handlers = sse42_pclmulqdq_get_handlers();
>>>>> - if (handlers != NULL)
>>>>> - return handlers[RTE_NET_CRC32_ETH](data, data_len);
>>>>> - handlers = neon_pmull_get_handlers();
>>>>> - if (handlers != NULL)
>>>>> - return handlers[RTE_NET_CRC32_ETH](data, data_len);
>>>>> - handlers = handlers_scalar;
>>>>> - return handlers[RTE_NET_CRC32_ETH](data, data_len);
>>>>> -}
>>>>> + uint16_t max_simd_bitwidth;
>>>>>
>>>>> -/* Public API */
>>>>> -
>>>>> -void
>>>>> -rte_net_crc_set_alg(enum rte_net_crc_alg alg) -{
>>>>> - handlers = NULL;
>>>>> - if (max_simd_bitwidth == 0)
>>>>> - max_simd_bitwidth = rte_vect_get_max_simd_bitwidth();
>>>>> + max_simd_bitwidth = rte_vect_get_max_simd_bitwidth();
>>>>>
>>>>> switch (alg) {
>>>>> case RTE_NET_CRC_AVX512:
>>>>> - handlers = avx512_vpclmulqdq_get_handlers();
>>>>> - if (handlers != NULL)
>>>>> - break;
>>>>> +#ifdef CC_X86_64_AVX512_VPCLMULQDQ_SUPPORT
>>>>> + if (AVX512_VPCLMULQDQ_CPU_SUPPORTED &&
>>>>> + max_simd_bitwidth >= RTE_VECT_SIMD_512) {
>>>>> + return (struct rte_net_crc){ RTE_NET_CRC_AVX512,
>>>> type };
>>>>> + }
>>>>> +#endif
>>>>> /* fall-through */
>>>>> case RTE_NET_CRC_SSE42:
>>>>> - handlers = sse42_pclmulqdq_get_handlers();
>>>>> - break; /* for x86, always break here */
>>>>> +#ifdef CC_X86_64_SSE42_PCLMULQDQ_SUPPORT
>>>>> + if (SSE42_PCLMULQDQ_CPU_SUPPORTED &&
>>>>> + max_simd_bitwidth >= RTE_VECT_SIMD_128) {
>>>>> + return (struct rte_net_crc){ RTE_NET_CRC_SSE42, type
>>>> };
>>>>> + }
>>>>> +#endif
>>>>> + break;
>>>>> case RTE_NET_CRC_NEON:
>>>>> - handlers = neon_pmull_get_handlers();
>>>>> - /* fall-through */
>>>>> - case RTE_NET_CRC_SCALAR:
>>>>> - /* fall-through */
>>>>> +#ifdef CC_ARM64_NEON_PMULL_SUPPORT
>>>>> + if (NEON_PMULL_CPU_SUPPORTED &&
>>>>> + max_simd_bitwidth >= RTE_VECT_SIMD_128) {
>>>>> + return (struct rte_net_crc){ RTE_NET_CRC_NEON, type
>>>> };
>>>>> + }
>>>>> +#endif
>>>>>
>>>>
>>>> Is it more readable as following, up to you:
>>>
>>> Agree, I will change it.
>>>
>>>>
>>>> ```
>>>> rte_net_crc_set(alg, type) {
>>>> enum rte_net_crc_alg new_alg = RTE_NET_CRC_SCALAR;
>>>> switch (alg) {
>>>> case AVX512:
>>>> new_alg = ..
>>>> case NEON:
>>>> new_alg = ..
>>>> }
>>>> return struct rte_net_crc){ new_alg, type }; ```
>>>>
>>>>
>>>>
>>>>
>>>>> + break;
>>>>> default:
>>>>> break;
>>>>> }
>>>>> -
>>>>> - if (handlers == NULL)
>>>>> - handlers = handlers_scalar;
>>>>> + return (struct rte_net_crc){ RTE_NET_CRC_SCALAR, type };
>>>>> }
>>>>>
>>>>> -uint32_t
>>>>> -rte_net_crc_calc(const void *data,
>>>>> - uint32_t data_len,
>>>>> - enum rte_net_crc_type type)
>>>>> +uint32_t rte_net_crc(const struct rte_net_crc *ctx,
>>>>> + const void *data, const uint32_t data_len)
>>>>> {
>>>>> - uint32_t ret;
>>>>> - rte_net_crc_handler f_handle;
>>>>> -
>>>>> - f_handle = handlers[type];
>>>>> - ret = f_handle(data, data_len);
>>>>> -
>>>>> - return ret;
>>>>> + return handlers[ctx->alg].f[ctx->type](data, data_len);
>>>>>
>>>>
>>>> 'rte_net_crc()' gets input from user and "struct rte_net_crc" is not
>>>> opaque, so user can provide invalid input, ctx->alg & ctx->type.
>>>> To protect against it input values should be checked before using.
>>>>
>>>> Or I think user not need to know the details of the "struct
>>>> rte_net_crc", so it can be an opaque variable for user.
>>>
>>> I would love it to be opaque, but then I would have to return a pointer, which
>> then would involve some allocations/deallocations and I wanted to keep is as
>> simple as possible.
>>> So probably the checks would be a way to go.
>>>
>>
>> True, +1 for simplicity.
>>
>>>>
>>>> <...>
>>>>
>>>>> -/**
>>>>> - * CRC compute API
>>>>> - *
>>>>> - * @param data
>>>>> - * Pointer to the packet data for CRC computation
>>>>> - * @param data_len
>>>>> - * Data length for CRC computation
>>>>> - * @param type
>>>>> - * CRC type (enum rte_net_crc_type)
>>>>> - *
>>>>> - * @return
>>>>> - * CRC value
>>>>> - */
>>>>> -uint32_t
>>>>> -rte_net_crc_calc(const void *data,
>>>>> - uint32_t data_len,
>>>>> +struct rte_net_crc rte_net_crc_set(enum rte_net_crc_alg alg,
>>>>> enum rte_net_crc_type type);
>>>>>
>>>>> +uint32_t rte_net_crc(const struct rte_net_crc *ctx,
>>>>> + const void *data, const uint32_t data_len);
>>>>> +
>>>>>
>>>>
>>>> As these are APIs, can you please add doxygen comments to them?
>>> +1
>>> I think this change could be deferred to 25.03.
>>> Adding this API without removing the old one should be possible without any
>> unwanted consequences?
>>>
>>
>> This is not a new functionality but replacement of existing one, so it will be
>> confusing to have two set of APIs for same functionality with similar names:
>> rte_net_crc_calc() and rte_net_crc()
>> rte_net_crc_set_alg() and rte_net_crc_set()
>>
>> Also there are some internal functions used by these APIs and supporting both
>> new and old may force to have two version of these internal functions and it will
>> create complexity/noise.
>>
>> As this release is ABI break release, it is easier to update APIs (although
>> deprecation notice is missing for this change).
>>
>>
>> As an alternative option, do you think applying ABI versioning in 25.03 works for
>> these APIs?
>> If so, old version can be cleaned in v25.11.
>
> I would go with versioning. I can preserve old function names as it's basically the same functionality.
>
ack
^ permalink raw reply [relevance 0%]
* RE: [PATCH v2 1/3] net: add thread-safe crc api
2024-10-09 1:03 4% ` Ferruh Yigit
@ 2024-10-09 7:48 0% ` Kusztal, ArkadiuszX
2024-10-09 9:11 0% ` Ferruh Yigit
0 siblings, 1 reply; 200+ results
From: Kusztal, ArkadiuszX @ 2024-10-09 7:48 UTC (permalink / raw)
To: Ferruh Yigit, Marchand, David; +Cc: dev, Ji, Kai, Dooley, Brian
> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@amd.com>
> Sent: Wednesday, October 9, 2024 3:03 AM
> To: Kusztal, ArkadiuszX <arkadiuszx.kusztal@intel.com>; Marchand, David
> <david.marchand@redhat.com>
> Cc: dev@dpdk.org; Ji, Kai <kai.ji@intel.com>; Dooley, Brian
> <brian.dooley@intel.com>
> Subject: Re: [PATCH v2 1/3] net: add thread-safe crc api
>
> On 10/8/2024 9:51 PM, Kusztal, ArkadiuszX wrote:
> > Hi Ferruh,
> > Thanks for the review, comments inline,
> >
> >> -----Original Message-----
> >> From: Ferruh Yigit <ferruh.yigit@amd.com>
> >> Sent: Tuesday, October 8, 2024 5:43 AM
> >> To: Kusztal, ArkadiuszX <arkadiuszx.kusztal@intel.com>; Marchand,
> >> David <david.marchand@redhat.com>
> >> Cc: dev@dpdk.org; Ji, Kai <kai.ji@intel.com>; Dooley, Brian
> >> <brian.dooley@intel.com>
> >> Subject: Re: [PATCH v2 1/3] net: add thread-safe crc api
> >>
> >> On 10/1/2024 7:11 PM, Arkadiusz Kusztal wrote:
> >>> The current net CRC API is not thread-safe, this patch solves this
> >>> by adding another, thread-safe API functions.
> >>> This API is also safe to use across multiple processes, yet with
> >>> limitations on max-simd-bitwidth, which will be checked only by the
> >>> process that created the CRC context; all other processes will use
> >>> the same CRC function when used with the same CRC context.
> >>> It is an undefined behavior when process binaries are compiled with
> >>> different SIMD capabilities when the same CRC context is used.
> >>>
> >>> Signed-off-by: Arkadiusz Kusztal <arkadiuszx.kusztal@intel.com>
> >>
> >> <...>
> >>
> >>> +static struct
> >>> +{
> >>> + uint32_t (*f[RTE_NET_CRC_REQS])
> >>> + (const uint8_t *data, uint32_t data_len);
> >>>
> >>
> >> It increases readability to typedef function pointers.
> >
> > Agree, though this typedef would be used here only, that’s why I left it out.
> > But I can add it then.
> >
> >>
> >>
> >>> +} handlers[RTE_NET_CRC_AVX512 + 1];
> >>>
> >>> -/**
> >>> - * Reflect the bits about the middle
> >>> - *
> >>> - * @param val
> >>> - * value to be reflected
> >>> - *
> >>> - * @return
> >>> - * reflected value
> >>> - */
> >>> -static uint32_t
> >>> +static inline uint32_t
> >>>
> >>
> >> Does changing to 'inline' required, as function is static compiler
> >> can do the same.
> >
> > True, though it may be more readable sometimes.
> > Of course there is no way that in O3 these functions would not be inlined by
> the compiler, regardless if the inline hint is present or not.
> >
> >>
> >>> reflect_32bits(uint32_t val)
> >>> {
> >>> uint32_t i, res = 0;
> >>> @@ -99,26 +43,7 @@ reflect_32bits(uint32_t val)
> >>> return res;
> >>> }
> >>>
> >>> -static void
> >>> -crc32_eth_init_lut(uint32_t poly,
> >>> - uint32_t *lut)
> >>> -{
> >>> - uint32_t i, j;
> >>> -
> >>> - for (i = 0; i < CRC_LUT_SIZE; i++) {
> >>> - uint32_t crc = reflect_32bits(i);
> >>> -
> >>> - for (j = 0; j < 8; j++) {
> >>> - if (crc & 0x80000000L)
> >>> - crc = (crc << 1) ^ poly;
> >>> - else
> >>> - crc <<= 1;
> >>> - }
> >>> - lut[i] = reflect_32bits(crc);
> >>> - }
> >>> -}
> >>> -
> >>> -static __rte_always_inline uint32_t
> >>> +static inline uint32_t
> >>>
> >>
> >> Why not forcing inline anymore?
> >> Are these inline changes related to the thread-safety?
> >
> > O3 will inline it anyway, and with always_inline it will be inline even in debug
> mode. I just see no reason forcing it upon the compiler.
> >
> >>
> >>> crc32_eth_calc_lut(const uint8_t *data,
> >>> uint32_t data_len,
> >>> uint32_t crc,
> >>> @@ -130,20 +55,9 @@ crc32_eth_calc_lut(const uint8_t *data,
> >>> return crc;
> >>> }
> >>>
> >>> -static void
> >>> -rte_net_crc_scalar_init(void)
> >>> -{
> >>> - /* 32-bit crc init */
> >>> - crc32_eth_init_lut(CRC32_ETH_POLYNOMIAL, crc32_eth_lut);
> >>> -
> >>> - /* 16-bit CRC init */
> >>> - crc32_eth_init_lut(CRC16_CCITT_POLYNOMIAL << 16, crc16_ccitt_lut);
> >>> -}
> >>> -
> >>> static inline uint32_t
> >>> -rte_crc16_ccitt_handler(const uint8_t *data, uint32_t data_len)
> >>> +crc16_ccitt(const uint8_t *data, uint32_t data_len)
> >>> {
> >>> - /* return 16-bit CRC value */
> >>>
> >>
> >> Why not keep comments? Are they wrong?
> >
> > Functions names are very self-explanatory, that’s why I dropped comments. I
> can add comments if needed.
> >
>
> I am for restricting changes to the target of the patch which is making CRC
> calculation thread safe, unless code changes invalidates the comments, lets
> keep them. Same goes with inline related modifications.
>
> >>
> >> <...>
> >>
> >>> +static void
> >>> +crc_scalar_init(void)
> >>> +{
> >>> + crc32_eth_init_lut(CRC32_ETH_POLYNOMIAL, crc32_eth_lut);
> >>> + crc32_eth_init_lut(CRC16_CCITT_POLYNOMIAL << 16, crc16_ccitt_lut);
> >>> +
> >>> + handlers[RTE_NET_CRC_SCALAR].f[RTE_NET_CRC16_CCITT] =
> >> crc16_ccitt;
> >>> + handlers[RTE_NET_CRC_SCALAR].f[RTE_NET_CRC32_ETH] = crc32_eth;
> >>>
> >>
> >> +1 to remove global handlers pointer and add context,
> >>
> >> But current handlers array content is static, it can be set when
> >> defined, instead of functions.
> >
> > Can do it for scalars, but for SIMD there is this runtime check like this:
> > if (AVX512_VPCLMULQDQ_CPU_SUPPORTED) { So compiled binary on
> AVX512
> > machine could filter it out on the machine which does not support it.
> > There is no NULL check in crc function so it would not change much when
> called -> Invalid address vs Invalid instruction.
> > There are not many checks there, as this is CRC after all, it should be a small as
> possible, yet probably NULL check could be advisable in crc function then.
> >
>
> There is already AVX512_VPCLMULQDQ_CPU_SUPPORTED etc checks in
> 'rte_net_crc_set()'.
> So does it work to update 'handlers' statically, without condition, but have
> conditions when use them.
>
> >
> >>
> >> <...>
> >>
> >>> -static uint32_t
> >>> -rte_crc32_eth_default_handler(const uint8_t *data, uint32_t
> >>> data_len)
> >>> +struct rte_net_crc rte_net_crc_set(enum rte_net_crc_alg alg,
> >>> + enum rte_net_crc_type type)
> >>> {
> >>> - handlers = NULL;
> >>> - if (max_simd_bitwidth == 0)
> >>> - max_simd_bitwidth = rte_vect_get_max_simd_bitwidth();
> >>> -
> >>> - handlers = avx512_vpclmulqdq_get_handlers();
> >>> - if (handlers != NULL)
> >>> - return handlers[RTE_NET_CRC32_ETH](data, data_len);
> >>> - handlers = sse42_pclmulqdq_get_handlers();
> >>> - if (handlers != NULL)
> >>> - return handlers[RTE_NET_CRC32_ETH](data, data_len);
> >>> - handlers = neon_pmull_get_handlers();
> >>> - if (handlers != NULL)
> >>> - return handlers[RTE_NET_CRC32_ETH](data, data_len);
> >>> - handlers = handlers_scalar;
> >>> - return handlers[RTE_NET_CRC32_ETH](data, data_len);
> >>> -}
> >>> + uint16_t max_simd_bitwidth;
> >>>
> >>> -/* Public API */
> >>> -
> >>> -void
> >>> -rte_net_crc_set_alg(enum rte_net_crc_alg alg) -{
> >>> - handlers = NULL;
> >>> - if (max_simd_bitwidth == 0)
> >>> - max_simd_bitwidth = rte_vect_get_max_simd_bitwidth();
> >>> + max_simd_bitwidth = rte_vect_get_max_simd_bitwidth();
> >>>
> >>> switch (alg) {
> >>> case RTE_NET_CRC_AVX512:
> >>> - handlers = avx512_vpclmulqdq_get_handlers();
> >>> - if (handlers != NULL)
> >>> - break;
> >>> +#ifdef CC_X86_64_AVX512_VPCLMULQDQ_SUPPORT
> >>> + if (AVX512_VPCLMULQDQ_CPU_SUPPORTED &&
> >>> + max_simd_bitwidth >= RTE_VECT_SIMD_512) {
> >>> + return (struct rte_net_crc){ RTE_NET_CRC_AVX512,
> >> type };
> >>> + }
> >>> +#endif
> >>> /* fall-through */
> >>> case RTE_NET_CRC_SSE42:
> >>> - handlers = sse42_pclmulqdq_get_handlers();
> >>> - break; /* for x86, always break here */
> >>> +#ifdef CC_X86_64_SSE42_PCLMULQDQ_SUPPORT
> >>> + if (SSE42_PCLMULQDQ_CPU_SUPPORTED &&
> >>> + max_simd_bitwidth >= RTE_VECT_SIMD_128) {
> >>> + return (struct rte_net_crc){ RTE_NET_CRC_SSE42, type
> >> };
> >>> + }
> >>> +#endif
> >>> + break;
> >>> case RTE_NET_CRC_NEON:
> >>> - handlers = neon_pmull_get_handlers();
> >>> - /* fall-through */
> >>> - case RTE_NET_CRC_SCALAR:
> >>> - /* fall-through */
> >>> +#ifdef CC_ARM64_NEON_PMULL_SUPPORT
> >>> + if (NEON_PMULL_CPU_SUPPORTED &&
> >>> + max_simd_bitwidth >= RTE_VECT_SIMD_128) {
> >>> + return (struct rte_net_crc){ RTE_NET_CRC_NEON, type
> >> };
> >>> + }
> >>> +#endif
> >>>
> >>
> >> Is it more readable as following, up to you:
> >
> > Agree, I will change it.
> >
> >>
> >> ```
> >> rte_net_crc_set(alg, type) {
> >> enum rte_net_crc_alg new_alg = RTE_NET_CRC_SCALAR;
> >> switch (alg) {
> >> case AVX512:
> >> new_alg = ..
> >> case NEON:
> >> new_alg = ..
> >> }
> >> return struct rte_net_crc){ new_alg, type }; ```
> >>
> >>
> >>
> >>
> >>> + break;
> >>> default:
> >>> break;
> >>> }
> >>> -
> >>> - if (handlers == NULL)
> >>> - handlers = handlers_scalar;
> >>> + return (struct rte_net_crc){ RTE_NET_CRC_SCALAR, type };
> >>> }
> >>>
> >>> -uint32_t
> >>> -rte_net_crc_calc(const void *data,
> >>> - uint32_t data_len,
> >>> - enum rte_net_crc_type type)
> >>> +uint32_t rte_net_crc(const struct rte_net_crc *ctx,
> >>> + const void *data, const uint32_t data_len)
> >>> {
> >>> - uint32_t ret;
> >>> - rte_net_crc_handler f_handle;
> >>> -
> >>> - f_handle = handlers[type];
> >>> - ret = f_handle(data, data_len);
> >>> -
> >>> - return ret;
> >>> + return handlers[ctx->alg].f[ctx->type](data, data_len);
> >>>
> >>
> >> 'rte_net_crc()' gets input from user and "struct rte_net_crc" is not
> >> opaque, so user can provide invalid input, ctx->alg & ctx->type.
> >> To protect against it input values should be checked before using.
> >>
> >> Or I think user not need to know the details of the "struct
> >> rte_net_crc", so it can be an opaque variable for user.
> >
> > I would love it to be opaque, but then I would have to return a pointer, which
> then would involve some allocations/deallocations and I wanted to keep is as
> simple as possible.
> > So probably the checks would be a way to go.
> >
>
> True, +1 for simplicity.
>
> >>
> >> <...>
> >>
> >>> -/**
> >>> - * CRC compute API
> >>> - *
> >>> - * @param data
> >>> - * Pointer to the packet data for CRC computation
> >>> - * @param data_len
> >>> - * Data length for CRC computation
> >>> - * @param type
> >>> - * CRC type (enum rte_net_crc_type)
> >>> - *
> >>> - * @return
> >>> - * CRC value
> >>> - */
> >>> -uint32_t
> >>> -rte_net_crc_calc(const void *data,
> >>> - uint32_t data_len,
> >>> +struct rte_net_crc rte_net_crc_set(enum rte_net_crc_alg alg,
> >>> enum rte_net_crc_type type);
> >>>
> >>> +uint32_t rte_net_crc(const struct rte_net_crc *ctx,
> >>> + const void *data, const uint32_t data_len);
> >>> +
> >>>
> >>
> >> As these are APIs, can you please add doxygen comments to them?
> > +1
> > I think this change could be deferred to 25.03.
> > Adding this API without removing the old one should be possible without any
> unwanted consequences?
> >
>
> This is not a new functionality but replacement of existing one, so it will be
> confusing to have two set of APIs for same functionality with similar names:
> rte_net_crc_calc() and rte_net_crc()
> rte_net_crc_set_alg() and rte_net_crc_set()
>
> Also there are some internal functions used by these APIs and supporting both
> new and old may force to have two version of these internal functions and it will
> create complexity/noise.
>
> As this release is ABI break release, it is easier to update APIs (although
> deprecation notice is missing for this change).
>
>
> As an alternative option, do you think applying ABI versioning in 25.03 works for
> these APIs?
> If so, old version can be cleaned in v25.11.
I would go with versioning. I can preserve old function names as it's basically the same functionality.
>
>
> > I still have some second thoughts about this max-simd-width. DPDK does
> > not impose any restrictions on this parameter in the multi-process usage, there
> may be some room to alter some things there.
> >
> >>
> >>> #ifdef __cplusplus
> >>> }
> >>> #endif
> >>> diff --git a/lib/net/version.map b/lib/net/version.map index
> >>> bec4ce23ea..47daf1464a 100644
> >>> --- a/lib/net/version.map
> >>> +++ b/lib/net/version.map
> >>> @@ -4,11 +4,25 @@ DPDK_25 {
> >>> rte_eth_random_addr;
> >>> rte_ether_format_addr;
> >>> rte_ether_unformat_addr;
> >>> - rte_net_crc_calc;
> >>> - rte_net_crc_set_alg;
> >>> rte_net_get_ptype;
> >>> rte_net_make_rarp_packet;
> >>> rte_net_skip_ip6_ext;
> >>> + rte_net_crc;
> >>> + rte_net_crc_set;
> >>>
> >>> local: *;
> >>> };
> >>> +
> >>> +INTERNAL {
> >>> + global:
> >>> +
> >>> + rte_net_crc_sse42_init;
> >>> + rte_crc16_ccitt_sse42_handler;
> >>> + rte_crc32_eth_sse42_handler;
> >>> + rte_net_crc_avx512_init;
> >>> + rte_crc16_ccitt_avx512_handler;
> >>> + rte_crc32_eth_avx512_handler;
> >>> + rte_net_crc_neon_init;
> >>> + rte_crc16_ccitt_neon_handler;
> >>> + rte_crc32_eth_neon_handler;
> >>> +};
> >>>
> >>
> >> +1 to David's comment, these are used only within component, no need
> >> +to
> >> export.
> >>
> >
^ permalink raw reply [relevance 0%]
* [PATCH v6 1/4] kvargs: add one new process API
@ 2024-10-09 4:50 3% ` Chengwen Feng
0 siblings, 0 replies; 200+ results
From: Chengwen Feng @ 2024-10-09 4:50 UTC (permalink / raw)
To: thomas, ferruh.yigit; +Cc: dev, stephen
The rte_kvargs_process() was used to handle key=value (e.g.
socket_id=0), it also supports to handle only-key (e.g. socket_id).
But many drivers's callback can only handle key=value, it will segment
fault if handles only-key. so the patchset [1] was introduced.
Because the patchset [1] modified too much drivers, therefore:
1) A new API rte_kvargs_process_opt() was introduced, it inherits the
function of rte_kvargs_process() which could handle both key=value and
only-key cases.
2) Constraint the rte_kvargs_process() can only handle key=value cases,
it will return -1 when handle only-key case (that is the matched key's
value is NULL).
This patch also make sure the rte_kvargs_process_opt() and
rte_kvargs_process() API both return -1 when the kvlist parameter is
NULL.
[1] https://patches.dpdk.org/project/dpdk/patch/20230320092110.37295-1-fengchengwen@huawei.com/
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
doc/guides/rel_notes/release_24_11.rst | 13 ++++++++
lib/kvargs/rte_kvargs.c | 43 ++++++++++++++++++++------
lib/kvargs/rte_kvargs.h | 39 +++++++++++++++++++++--
lib/kvargs/version.map | 7 +++++
4 files changed, 90 insertions(+), 12 deletions(-)
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index e0a9aa55a1..873f0639dc 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -96,6 +96,19 @@ API Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* **Updated kvargs process API.**
+
+ * Introduced rte_kvargs_process_opt() API, which inherits the function
+ of rte_kvargs_process() and could handle both key=value and only-key
+ cases.
+
+ * Constraint rte_kvargs_process() API can only handle key=value cases,
+ it will return -1 when handle only-key case (that is the matched key's
+ value is NULL).
+
+ * Make sure rte_kvargs_process_opt() and rte_kvargs_process() API both
+ return -1 when the kvlist parameter is NULL.
+
ABI Changes
-----------
diff --git a/lib/kvargs/rte_kvargs.c b/lib/kvargs/rte_kvargs.c
index c77bb82feb..b02f22f5a2 100644
--- a/lib/kvargs/rte_kvargs.c
+++ b/lib/kvargs/rte_kvargs.c
@@ -167,31 +167,56 @@ rte_kvargs_count(const struct rte_kvargs *kvlist, const char *key_match)
return ret;
}
-/*
- * For each matching key, call the given handler function.
- */
-int
-rte_kvargs_process(const struct rte_kvargs *kvlist,
- const char *key_match,
- arg_handler_t handler,
- void *opaque_arg)
+static int
+kvargs_process_common(const struct rte_kvargs *kvlist,
+ const char *key_match,
+ arg_handler_t handler,
+ void *opaque_arg,
+ bool support_only_key)
{
const struct rte_kvargs_pair *pair;
unsigned i;
if (kvlist == NULL)
- return 0;
+ return -1;
for (i = 0; i < kvlist->count; i++) {
pair = &kvlist->pairs[i];
if (key_match == NULL || strcmp(pair->key, key_match) == 0) {
+ if (!support_only_key && pair->value == NULL)
+ return -1;
if ((*handler)(pair->key, pair->value, opaque_arg) < 0)
return -1;
}
}
+
return 0;
}
+/*
+ * For each matching key in key=value, call the given handler function.
+ */
+int
+rte_kvargs_process(const struct rte_kvargs *kvlist,
+ const char *key_match,
+ arg_handler_t handler,
+ void *opaque_arg)
+{
+ return kvargs_process_common(kvlist, key_match, handler, opaque_arg, false);
+}
+
+/*
+ * For each matching key in key=value or only-key, call the given handler function.
+ */
+int
+rte_kvargs_process_opt(const struct rte_kvargs *kvlist,
+ const char *key_match,
+ arg_handler_t handler,
+ void *opaque_arg)
+{
+ return kvargs_process_common(kvlist, key_match, handler, opaque_arg, true);
+}
+
/* free the rte_kvargs structure */
void
rte_kvargs_free(struct rte_kvargs *kvlist)
diff --git a/lib/kvargs/rte_kvargs.h b/lib/kvargs/rte_kvargs.h
index b0d1301c61..b37cd4902f 100644
--- a/lib/kvargs/rte_kvargs.h
+++ b/lib/kvargs/rte_kvargs.h
@@ -6,6 +6,8 @@
#ifndef _RTE_KVARGS_H_
#define _RTE_KVARGS_H_
+#include <rte_compat.h>
+
/**
* @file
* RTE Argument parsing
@@ -166,14 +168,17 @@ const char *rte_kvargs_get_with_value(const struct rte_kvargs *kvlist,
const char *key, const char *value);
/**
- * Call a handler function for each key/value matching the key
+ * Call a handler function for each key=value matching the key
*
- * For each key/value association that matches the given key, calls the
+ * For each key=value association that matches the given key, calls the
* handler function with the for a given arg_name passing the value on the
* dictionary for that key and a given extra argument.
*
+ * @note Compared to @see rte_kvargs_process_opt, this API will return -1
+ * when handle only-key case (that is the matched key's value is NULL).
+ *
* @param kvlist
- * The rte_kvargs structure. No error if NULL.
+ * The rte_kvargs structure.
* @param key_match
* The key on which the handler should be called, or NULL to process handler
* on all associations
@@ -189,6 +194,34 @@ const char *rte_kvargs_get_with_value(const struct rte_kvargs *kvlist,
int rte_kvargs_process(const struct rte_kvargs *kvlist,
const char *key_match, arg_handler_t handler, void *opaque_arg);
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Call a handler function for each key=value or only-key matching the key
+ *
+ * For each key=value or only-key association that matches the given key, calls
+ * the handler function with the for a given arg_name passing the value on the
+ * dictionary for that key and a given extra argument.
+ *
+ * @param kvlist
+ * The rte_kvargs structure.
+ * @param key_match
+ * The key on which the handler should be called, or NULL to process handler
+ * on all associations
+ * @param handler
+ * The function to call for each matching key
+ * @param opaque_arg
+ * A pointer passed unchanged to the handler
+ *
+ * @return
+ * - 0 on success
+ * - Negative on error
+ */
+__rte_experimental
+int rte_kvargs_process_opt(const struct rte_kvargs *kvlist,
+ const char *key_match, arg_handler_t handler, void *opaque_arg);
+
/**
* Count the number of associations matching the given key
*
diff --git a/lib/kvargs/version.map b/lib/kvargs/version.map
index b50f1a97a1..178f476236 100644
--- a/lib/kvargs/version.map
+++ b/lib/kvargs/version.map
@@ -11,3 +11,10 @@ DPDK_25 {
local: *;
};
+
+EXPERIMENTAL {
+ global:
+
+ # added in 24.11
+ rte_kvargs_process_opt;
+};
--
2.17.1
^ permalink raw reply [relevance 3%]
* Re: [PATCH v2 1/3] net: add thread-safe crc api
@ 2024-10-09 1:03 4% ` Ferruh Yigit
2024-10-09 7:48 0% ` Kusztal, ArkadiuszX
0 siblings, 1 reply; 200+ results
From: Ferruh Yigit @ 2024-10-09 1:03 UTC (permalink / raw)
To: Kusztal, ArkadiuszX, Marchand, David; +Cc: dev, Ji, Kai, Dooley, Brian
On 10/8/2024 9:51 PM, Kusztal, ArkadiuszX wrote:
> Hi Ferruh,
> Thanks for the review, comments inline,
>
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@amd.com>
>> Sent: Tuesday, October 8, 2024 5:43 AM
>> To: Kusztal, ArkadiuszX <arkadiuszx.kusztal@intel.com>; Marchand, David
>> <david.marchand@redhat.com>
>> Cc: dev@dpdk.org; Ji, Kai <kai.ji@intel.com>; Dooley, Brian
>> <brian.dooley@intel.com>
>> Subject: Re: [PATCH v2 1/3] net: add thread-safe crc api
>>
>> On 10/1/2024 7:11 PM, Arkadiusz Kusztal wrote:
>>> The current net CRC API is not thread-safe, this patch solves this by
>>> adding another, thread-safe API functions.
>>> This API is also safe to use across multiple processes, yet with
>>> limitations on max-simd-bitwidth, which will be checked only by the
>>> process that created the CRC context; all other processes will use the
>>> same CRC function when used with the same CRC context.
>>> It is an undefined behavior when process binaries are compiled with
>>> different SIMD capabilities when the same CRC context is used.
>>>
>>> Signed-off-by: Arkadiusz Kusztal <arkadiuszx.kusztal@intel.com>
>>
>> <...>
>>
>>> +static struct
>>> +{
>>> + uint32_t (*f[RTE_NET_CRC_REQS])
>>> + (const uint8_t *data, uint32_t data_len);
>>>
>>
>> It increases readability to typedef function pointers.
>
> Agree, though this typedef would be used here only, that’s why I left it out.
> But I can add it then.
>
>>
>>
>>> +} handlers[RTE_NET_CRC_AVX512 + 1];
>>>
>>> -/**
>>> - * Reflect the bits about the middle
>>> - *
>>> - * @param val
>>> - * value to be reflected
>>> - *
>>> - * @return
>>> - * reflected value
>>> - */
>>> -static uint32_t
>>> +static inline uint32_t
>>>
>>
>> Does changing to 'inline' required, as function is static compiler can do the
>> same.
>
> True, though it may be more readable sometimes.
> Of course there is no way that in O3 these functions would not be inlined by the compiler, regardless if the inline hint is present or not.
>
>>
>>> reflect_32bits(uint32_t val)
>>> {
>>> uint32_t i, res = 0;
>>> @@ -99,26 +43,7 @@ reflect_32bits(uint32_t val)
>>> return res;
>>> }
>>>
>>> -static void
>>> -crc32_eth_init_lut(uint32_t poly,
>>> - uint32_t *lut)
>>> -{
>>> - uint32_t i, j;
>>> -
>>> - for (i = 0; i < CRC_LUT_SIZE; i++) {
>>> - uint32_t crc = reflect_32bits(i);
>>> -
>>> - for (j = 0; j < 8; j++) {
>>> - if (crc & 0x80000000L)
>>> - crc = (crc << 1) ^ poly;
>>> - else
>>> - crc <<= 1;
>>> - }
>>> - lut[i] = reflect_32bits(crc);
>>> - }
>>> -}
>>> -
>>> -static __rte_always_inline uint32_t
>>> +static inline uint32_t
>>>
>>
>> Why not forcing inline anymore?
>> Are these inline changes related to the thread-safety?
>
> O3 will inline it anyway, and with always_inline it will be inline even in debug mode. I just see no reason forcing it upon the compiler.
>
>>
>>> crc32_eth_calc_lut(const uint8_t *data,
>>> uint32_t data_len,
>>> uint32_t crc,
>>> @@ -130,20 +55,9 @@ crc32_eth_calc_lut(const uint8_t *data,
>>> return crc;
>>> }
>>>
>>> -static void
>>> -rte_net_crc_scalar_init(void)
>>> -{
>>> - /* 32-bit crc init */
>>> - crc32_eth_init_lut(CRC32_ETH_POLYNOMIAL, crc32_eth_lut);
>>> -
>>> - /* 16-bit CRC init */
>>> - crc32_eth_init_lut(CRC16_CCITT_POLYNOMIAL << 16, crc16_ccitt_lut);
>>> -}
>>> -
>>> static inline uint32_t
>>> -rte_crc16_ccitt_handler(const uint8_t *data, uint32_t data_len)
>>> +crc16_ccitt(const uint8_t *data, uint32_t data_len)
>>> {
>>> - /* return 16-bit CRC value */
>>>
>>
>> Why not keep comments? Are they wrong?
>
> Functions names are very self-explanatory, that’s why I dropped comments. I can add comments if needed.
>
I am for restricting changes to the target of the patch which is making
CRC calculation thread safe, unless code changes invalidates the
comments, lets keep them. Same goes with inline related modifications.
>>
>> <...>
>>
>>> +static void
>>> +crc_scalar_init(void)
>>> +{
>>> + crc32_eth_init_lut(CRC32_ETH_POLYNOMIAL, crc32_eth_lut);
>>> + crc32_eth_init_lut(CRC16_CCITT_POLYNOMIAL << 16, crc16_ccitt_lut);
>>> +
>>> + handlers[RTE_NET_CRC_SCALAR].f[RTE_NET_CRC16_CCITT] =
>> crc16_ccitt;
>>> + handlers[RTE_NET_CRC_SCALAR].f[RTE_NET_CRC32_ETH] = crc32_eth;
>>>
>>
>> +1 to remove global handlers pointer and add context,
>>
>> But current handlers array content is static, it can be set when defined, instead
>> of functions.
>
> Can do it for scalars, but for SIMD there is this runtime check like this:
> if (AVX512_VPCLMULQDQ_CPU_SUPPORTED) {
> So compiled binary on AVX512 machine could filter it out on the machine which does not support it.
> There is no NULL check in crc function so it would not change much when called -> Invalid address vs Invalid instruction.
> There are not many checks there, as this is CRC after all, it should be a small as possible, yet probably NULL check could be advisable in crc function then.
>
There is already AVX512_VPCLMULQDQ_CPU_SUPPORTED etc checks in
'rte_net_crc_set()'.
So does it work to update 'handlers' statically, without condition, but
have conditions when use them.
>
>>
>> <...>
>>
>>> -static uint32_t
>>> -rte_crc32_eth_default_handler(const uint8_t *data, uint32_t data_len)
>>> +struct rte_net_crc rte_net_crc_set(enum rte_net_crc_alg alg,
>>> + enum rte_net_crc_type type)
>>> {
>>> - handlers = NULL;
>>> - if (max_simd_bitwidth == 0)
>>> - max_simd_bitwidth = rte_vect_get_max_simd_bitwidth();
>>> -
>>> - handlers = avx512_vpclmulqdq_get_handlers();
>>> - if (handlers != NULL)
>>> - return handlers[RTE_NET_CRC32_ETH](data, data_len);
>>> - handlers = sse42_pclmulqdq_get_handlers();
>>> - if (handlers != NULL)
>>> - return handlers[RTE_NET_CRC32_ETH](data, data_len);
>>> - handlers = neon_pmull_get_handlers();
>>> - if (handlers != NULL)
>>> - return handlers[RTE_NET_CRC32_ETH](data, data_len);
>>> - handlers = handlers_scalar;
>>> - return handlers[RTE_NET_CRC32_ETH](data, data_len);
>>> -}
>>> + uint16_t max_simd_bitwidth;
>>>
>>> -/* Public API */
>>> -
>>> -void
>>> -rte_net_crc_set_alg(enum rte_net_crc_alg alg) -{
>>> - handlers = NULL;
>>> - if (max_simd_bitwidth == 0)
>>> - max_simd_bitwidth = rte_vect_get_max_simd_bitwidth();
>>> + max_simd_bitwidth = rte_vect_get_max_simd_bitwidth();
>>>
>>> switch (alg) {
>>> case RTE_NET_CRC_AVX512:
>>> - handlers = avx512_vpclmulqdq_get_handlers();
>>> - if (handlers != NULL)
>>> - break;
>>> +#ifdef CC_X86_64_AVX512_VPCLMULQDQ_SUPPORT
>>> + if (AVX512_VPCLMULQDQ_CPU_SUPPORTED &&
>>> + max_simd_bitwidth >= RTE_VECT_SIMD_512) {
>>> + return (struct rte_net_crc){ RTE_NET_CRC_AVX512,
>> type };
>>> + }
>>> +#endif
>>> /* fall-through */
>>> case RTE_NET_CRC_SSE42:
>>> - handlers = sse42_pclmulqdq_get_handlers();
>>> - break; /* for x86, always break here */
>>> +#ifdef CC_X86_64_SSE42_PCLMULQDQ_SUPPORT
>>> + if (SSE42_PCLMULQDQ_CPU_SUPPORTED &&
>>> + max_simd_bitwidth >= RTE_VECT_SIMD_128) {
>>> + return (struct rte_net_crc){ RTE_NET_CRC_SSE42, type
>> };
>>> + }
>>> +#endif
>>> + break;
>>> case RTE_NET_CRC_NEON:
>>> - handlers = neon_pmull_get_handlers();
>>> - /* fall-through */
>>> - case RTE_NET_CRC_SCALAR:
>>> - /* fall-through */
>>> +#ifdef CC_ARM64_NEON_PMULL_SUPPORT
>>> + if (NEON_PMULL_CPU_SUPPORTED &&
>>> + max_simd_bitwidth >= RTE_VECT_SIMD_128) {
>>> + return (struct rte_net_crc){ RTE_NET_CRC_NEON, type
>> };
>>> + }
>>> +#endif
>>>
>>
>> Is it more readable as following, up to you:
>
> Agree, I will change it.
>
>>
>> ```
>> rte_net_crc_set(alg, type) {
>> enum rte_net_crc_alg new_alg = RTE_NET_CRC_SCALAR;
>> switch (alg) {
>> case AVX512:
>> new_alg = ..
>> case NEON:
>> new_alg = ..
>> }
>> return struct rte_net_crc){ new_alg, type }; ```
>>
>>
>>
>>
>>> + break;
>>> default:
>>> break;
>>> }
>>> -
>>> - if (handlers == NULL)
>>> - handlers = handlers_scalar;
>>> + return (struct rte_net_crc){ RTE_NET_CRC_SCALAR, type };
>>> }
>>>
>>> -uint32_t
>>> -rte_net_crc_calc(const void *data,
>>> - uint32_t data_len,
>>> - enum rte_net_crc_type type)
>>> +uint32_t rte_net_crc(const struct rte_net_crc *ctx,
>>> + const void *data, const uint32_t data_len)
>>> {
>>> - uint32_t ret;
>>> - rte_net_crc_handler f_handle;
>>> -
>>> - f_handle = handlers[type];
>>> - ret = f_handle(data, data_len);
>>> -
>>> - return ret;
>>> + return handlers[ctx->alg].f[ctx->type](data, data_len);
>>>
>>
>> 'rte_net_crc()' gets input from user and "struct rte_net_crc" is not opaque, so
>> user can provide invalid input, ctx->alg & ctx->type.
>> To protect against it input values should be checked before using.
>>
>> Or I think user not need to know the details of the "struct rte_net_crc", so it can
>> be an opaque variable for user.
>
> I would love it to be opaque, but then I would have to return a pointer, which then would involve some allocations/deallocations and I wanted to keep is as simple as possible.
> So probably the checks would be a way to go.
>
True, +1 for simplicity.
>>
>> <...>
>>
>>> -/**
>>> - * CRC compute API
>>> - *
>>> - * @param data
>>> - * Pointer to the packet data for CRC computation
>>> - * @param data_len
>>> - * Data length for CRC computation
>>> - * @param type
>>> - * CRC type (enum rte_net_crc_type)
>>> - *
>>> - * @return
>>> - * CRC value
>>> - */
>>> -uint32_t
>>> -rte_net_crc_calc(const void *data,
>>> - uint32_t data_len,
>>> +struct rte_net_crc rte_net_crc_set(enum rte_net_crc_alg alg,
>>> enum rte_net_crc_type type);
>>>
>>> +uint32_t rte_net_crc(const struct rte_net_crc *ctx,
>>> + const void *data, const uint32_t data_len);
>>> +
>>>
>>
>> As these are APIs, can you please add doxygen comments to them?
> +1
> I think this change could be deferred to 25.03.
> Adding this API without removing the old one should be possible without any unwanted consequences?
>
This is not a new functionality but replacement of existing one, so it
will be confusing to have two set of APIs for same functionality with
similar names:
rte_net_crc_calc() and rte_net_crc()
rte_net_crc_set_alg() and rte_net_crc_set()
Also there are some internal functions used by these APIs and supporting
both new and old may force to have two version of these internal
functions and it will create complexity/noise.
As this release is ABI break release, it is easier to update APIs
(although deprecation notice is missing for this change).
As an alternative option, do you think applying ABI versioning in 25.03
works for these APIs?
If so, old version can be cleaned in v25.11.
> I still have some second thoughts about this max-simd-width. DPDK does not impose any restrictions on this parameter in the multi-process usage,
> there may be some room to alter some things there.
>
>>
>>> #ifdef __cplusplus
>>> }
>>> #endif
>>> diff --git a/lib/net/version.map b/lib/net/version.map index
>>> bec4ce23ea..47daf1464a 100644
>>> --- a/lib/net/version.map
>>> +++ b/lib/net/version.map
>>> @@ -4,11 +4,25 @@ DPDK_25 {
>>> rte_eth_random_addr;
>>> rte_ether_format_addr;
>>> rte_ether_unformat_addr;
>>> - rte_net_crc_calc;
>>> - rte_net_crc_set_alg;
>>> rte_net_get_ptype;
>>> rte_net_make_rarp_packet;
>>> rte_net_skip_ip6_ext;
>>> + rte_net_crc;
>>> + rte_net_crc_set;
>>>
>>> local: *;
>>> };
>>> +
>>> +INTERNAL {
>>> + global:
>>> +
>>> + rte_net_crc_sse42_init;
>>> + rte_crc16_ccitt_sse42_handler;
>>> + rte_crc32_eth_sse42_handler;
>>> + rte_net_crc_avx512_init;
>>> + rte_crc16_ccitt_avx512_handler;
>>> + rte_crc32_eth_avx512_handler;
>>> + rte_net_crc_neon_init;
>>> + rte_crc16_ccitt_neon_handler;
>>> + rte_crc32_eth_neon_handler;
>>> +};
>>>
>>
>> +1 to David's comment, these are used only within component, no need to
>> export.
>>
>
^ permalink raw reply [relevance 4%]
* Re: [PATCH v9] eal: add build-time option to omit trace
2024-10-07 11:46 3% ` [PATCH v9] " Morten Brørup
2024-10-08 7:16 0% ` Morten Brørup
@ 2024-10-08 10:15 0% ` David Marchand
1 sibling, 0 replies; 200+ results
From: David Marchand @ 2024-10-08 10:15 UTC (permalink / raw)
To: Morten Brørup; +Cc: Jerin Jacob, Sunil Kumar Kori, dev, Stephen Hemminger
On Mon, Oct 7, 2024 at 1:46 PM Morten Brørup <mb@smartsharesystems.com> wrote:
>
> Some applications want to omit the trace feature.
> Either to reduce the memory footprint, to reduce the exposed attack
> surface, or for other reasons.
>
> This patch adds an option in rte_config.h to include or omit trace in the
> build. Trace is included by default.
>
> Omitting trace works by omitting all trace points.
> For API and ABI compatibility, the trace feature itself remains.
>
> Furthermore, a public function to determine if trace is build time enabled
> is added; mainly for the benefit of the dpdk-test application.
>
> Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
> Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> Acked-by: Jerin Jacob <jerinj@marvell.com>
Series applied.
Thanks Morten.
--
David Marchand
^ permalink raw reply [relevance 0%]
* RE: [PATCH v9] eal: add build-time option to omit trace
2024-10-07 11:46 3% ` [PATCH v9] " Morten Brørup
@ 2024-10-08 7:16 0% ` Morten Brørup
2024-10-08 10:15 0% ` David Marchand
1 sibling, 0 replies; 200+ results
From: Morten Brørup @ 2024-10-08 7:16 UTC (permalink / raw)
To: thomas, Jerin Jacob, Sunil Kumar Kori; +Cc: dev, Stephen Hemminger
> From: Morten Brørup [mailto:mb@smartsharesystems.com]
> Sent: Monday, 7 October 2024 13.46
>
> Some applications want to omit the trace feature.
> Either to reduce the memory footprint, to reduce the exposed attack
> surface, or for other reasons.
>
> This patch adds an option in rte_config.h to include or omit trace in
> the
> build. Trace is included by default.
>
> Omitting trace works by omitting all trace points.
> For API and ABI compatibility, the trace feature itself remains.
>
> Furthermore, a public function to determine if trace is build time
> enabled
> is added; mainly for the benefit of the dpdk-test application.
>
> Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
> Acked-by: Stephen Hemminger <stephen@networkplumber.org>
> Acked-by: Jerin Jacob <jerinj@marvell.com>
> ---
Both trace_autotest and trace_perf_autotest have been tested (in a Linux environment) to behave as expected, both with and without RTE_TRACE defined.
Tested-by: Morten Brørup <mb@smartsharesystems.com>
^ permalink raw reply [relevance 0%]
* RE: [PATCH v2 2/4] power: refactor uncore power management library
2024-08-27 13:02 3% ` lihuisong (C)
@ 2024-10-08 6:19 0% ` Tummala, Sivaprasad
0 siblings, 0 replies; 200+ results
From: Tummala, Sivaprasad @ 2024-10-08 6:19 UTC (permalink / raw)
To: lihuisong (C)
Cc: dev, david.hunt, anatoly.burakov, radu.nicolau, jerinj,
cristian.dumitrescu, konstantin.ananyev, Yigit, Ferruh, gakhil
[AMD Official Use Only - AMD Internal Distribution Only]
Hi Lihuisong,
> -----Original Message-----
> From: lihuisong (C) <lihuisong@huawei.com>
> Sent: Tuesday, August 27, 2024 6:33 PM
> To: Tummala, Sivaprasad <Sivaprasad.Tummala@amd.com>
> Cc: dev@dpdk.org; david.hunt@intel.com; anatoly.burakov@intel.com;
> radu.nicolau@intel.com; jerinj@marvell.com; cristian.dumitrescu@intel.com;
> konstantin.ananyev@huawei.com; Yigit, Ferruh <Ferruh.Yigit@amd.com>;
> gakhil@marvell.com
> Subject: Re: [PATCH v2 2/4] power: refactor uncore power management library
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> Hi Sivaprasad,
>
> Suggest to split this patch into two patches for easiler to review:
> patch-1: abstract a file for uncore dvfs core level, namely, the
> rte_power_uncore_ops.c you did.
> patch-2: move and rename, lib/power/power_intel_uncore.c =>
> drivers/power/intel_uncore/intel_uncore.c
>
> patch[1/4] is also too big and not good to review.
>
> In addition, I have some question and am not sure if we can adjust uncore init
> process.
>
> /Huisong
>
>
> 在 2024/8/26 21:06, Sivaprasad Tummala 写道:
> > This patch refactors the power management library, addressing uncore
> > power management. The primary changes involve the creation of
> > dedicated directories for each driver within 'drivers/power/uncore/*'.
> > The adjustment of meson.build files enables the selective activation
> > of individual drivers.
> >
> > This refactor significantly improves code organization, enhances
> > clarity and boosts maintainability. It lays the foundation for more
> > focused development on individual drivers and facilitates seamless
> > integration of future enhancements, particularly the AMD uncore driver.
> >
> > Signed-off-by: Sivaprasad Tummala <sivaprasad.tummala@amd.com>
> > ---
> > .../power/intel_uncore/intel_uncore.c | 18 +-
> > .../power/intel_uncore/intel_uncore.h | 8 +-
> > drivers/power/intel_uncore/meson.build | 6 +
> > drivers/power/meson.build | 3 +-
> > lib/power/meson.build | 2 +-
> > lib/power/rte_power_uncore.c | 205 ++++++---------
> > lib/power/rte_power_uncore.h | 87 ++++---
> > lib/power/rte_power_uncore_ops.h | 239 ++++++++++++++++++
> > lib/power/version.map | 1 +
> > 9 files changed, 405 insertions(+), 164 deletions(-)
> > rename lib/power/power_intel_uncore.c =>
> drivers/power/intel_uncore/intel_uncore.c (95%)
> > rename lib/power/power_intel_uncore.h =>
> drivers/power/intel_uncore/intel_uncore.h (97%)
> > create mode 100644 drivers/power/intel_uncore/meson.build
> > create mode 100644 lib/power/rte_power_uncore_ops.h
> >
> > diff --git a/lib/power/power_intel_uncore.c
> > b/drivers/power/intel_uncore/intel_uncore.c
> > similarity index 95%
> > rename from lib/power/power_intel_uncore.c rename to
> > drivers/power/intel_uncore/intel_uncore.c
> > index 4eb9c5900a..804ad5d755 100644
> > --- a/lib/power/power_intel_uncore.c
> > +++ b/drivers/power/intel_uncore/intel_uncore.c
> > @@ -8,7 +8,7 @@
> >
> > #include <rte_memcpy.h>
> >
> > -#include "power_intel_uncore.h"
> > +#include "intel_uncore.h"
> > #include "power_common.h"
> >
> > #define MAX_NUMA_DIE 8
> > @@ -475,3 +475,19 @@ power_intel_uncore_get_num_dies(unsigned int pkg)
> >
> > return count;
> > }
> <...>
> >
> > -#endif /* POWER_INTEL_UNCORE_H */
> > +#endif /* INTEL_UNCORE_H */
> > diff --git a/drivers/power/intel_uncore/meson.build
> > b/drivers/power/intel_uncore/meson.build
> > new file mode 100644
> > index 0000000000..876df8ad14
> > --- /dev/null
> > +++ b/drivers/power/intel_uncore/meson.build
> > @@ -0,0 +1,6 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel
> > +Corporation # Copyright(c) 2024 Advanced Micro Devices, Inc.
> > +
> > +sources = files('intel_uncore.c')
> > +deps += ['power']
> > diff --git a/drivers/power/meson.build b/drivers/power/meson.build
> > index 8c7215c639..c83047af94 100644
> > --- a/drivers/power/meson.build
> > +++ b/drivers/power/meson.build
> > @@ -6,7 +6,8 @@ drivers = [
> > 'amd_pstate',
> > 'cppc',
> > 'kvm_vm',
> > - 'pstate'
> > + 'pstate',
> > + 'intel_uncore'
> The cppc, amd_pstate and so on belong to cpufreq scope.
> And intel_uncore belongs to uncore dvfs scope.
> They are not the same level. So I proposes that we need to create one directory
> called like cpufreq or core.
> This 'intel_uncore' name don't seems appropriate. what do you think the following
> directory structure:
> drivers/power/uncore/intel_uncore.c
> drivers/power/uncore/amd_uncore.c (according to the patch[4/4]).
At present, Meson does not support detecting an additional level of subdirectories within drivers/*.
All the drivers maintain a consistent subdirectory structure.
> > ]
> > std_deps = ['power']
> > diff --git a/lib/power/meson.build b/lib/power/meson.build index
> > f3e3451cdc..9b13d98810 100644
> > --- a/lib/power/meson.build
> > +++ b/lib/power/meson.build
> > @@ -13,7 +13,6 @@ if not is_linux
> > endif
> > sources = files(
> > 'power_common.c',
> > - 'power_intel_uncore.c',
> > 'rte_power.c',
> > 'rte_power_uncore.c',
> > 'rte_power_pmd_mgmt.c',
> > @@ -24,6 +23,7 @@ headers = files(
> > 'rte_power_guest_channel.h',
> > 'rte_power_pmd_mgmt.h',
> > 'rte_power_uncore.h',
> > + 'rte_power_uncore_ops.h',
> > )
> > if cc.has_argument('-Wno-cast-qual')
> > cflags += '-Wno-cast-qual'
> > diff --git a/lib/power/rte_power_uncore.c
> > b/lib/power/rte_power_uncore.c index 48c75a5da0..9f8771224f 100644
> > --- a/lib/power/rte_power_uncore.c
> > +++ b/lib/power/rte_power_uncore.c
> > @@ -1,6 +1,7 @@
> > /* SPDX-License-Identifier: BSD-3-Clause
> > * Copyright(c) 2010-2014 Intel Corporation
> > * Copyright(c) 2023 AMD Corporation
> > + * Copyright(c) 2024 Advanced Micro Devices, Inc.
> > */
> >
> > #include <errno.h>
> > @@ -12,98 +13,50 @@
> > #include "rte_power_uncore.h"
> > #include "power_intel_uncore.h"
> >
> > -enum rte_uncore_power_mgmt_env default_uncore_env =
> > RTE_UNCORE_PM_ENV_NOT_SET;
> > +static enum rte_uncore_power_mgmt_env global_uncore_env =
> > +RTE_UNCORE_PM_ENV_NOT_SET; static struct rte_power_uncore_ops
> > +*global_uncore_ops;
> >
> > static rte_spinlock_t global_env_cfg_lock =
> > RTE_SPINLOCK_INITIALIZER;
> > +static RTE_TAILQ_HEAD(, rte_power_uncore_ops) uncore_ops_list =
> > + TAILQ_HEAD_INITIALIZER(uncore_ops_list);
> >
> > -static uint32_t
> > -power_get_dummy_uncore_freq(unsigned int pkg __rte_unused,
> > - unsigned int die __rte_unused)
> > -{
> > - return 0;
> > -}
> > -
> > -static int
> > -power_set_dummy_uncore_freq(unsigned int pkg __rte_unused,
> > - unsigned int die __rte_unused, uint32_t index __rte_unused)
> > -{
> > - return 0;
> > -}
> > +const char *uncore_env_str[] = {
> > + "not set",
> > + "auto-detect",
> > + "intel-uncore",
> > + "amd-hsmp"
> > +};
> Why open the "auto-detect" mode to user?
> Why not set this automatically at framework initialization?
> After all, the uncore driver is fixed for one platform.
The auto-detection feature has been implemented to enable seamless migration across platforms
without requiring any changes to the application
> >
> > -static int
> > -power_dummy_uncore_freq_max(unsigned int pkg __rte_unused,
> > - unsigned int die __rte_unused)
> > -{
> > - return 0;
> > -}
> > -
> <...>
> > -static int
> > -power_dummy_uncore_get_num_freqs(unsigned int pkg __rte_unused,
> > - unsigned int die __rte_unused)
> > +/* register the ops struct in rte_power_uncore_ops, return 0 on
> > +success. */ int rte_power_register_uncore_ops(struct
> > +rte_power_uncore_ops *driver_ops)
> > {
> > - return 0;
> > -}
> > + if (!driver_ops->init || !driver_ops->exit || !driver_ops->get_num_pkgs ||
> > + !driver_ops->get_num_dies || !driver_ops->get_num_freqs ||
> > + !driver_ops->get_avail_freqs || !driver_ops->get_freq ||
> > + !driver_ops->set_freq || !driver_ops->freq_max ||
> > + !driver_ops->freq_min) {
> > + POWER_LOG(ERR, "Missing callbacks while registering power ops");
> > + return -1;
> > + }
> > + if (driver_ops->cb)
> > + driver_ops->cb();
> >
> > -static unsigned int
> > -power_dummy_uncore_get_num_pkgs(void)
> > -{
> > - return 0;
> > -}
> > + TAILQ_INSERT_TAIL(&uncore_ops_list, driver_ops, next);
> >
> > -static unsigned int
> > -power_dummy_uncore_get_num_dies(unsigned int pkg __rte_unused) -{
> > return 0;
> > }
> > -
> > -/* function pointers */
> > -rte_power_get_uncore_freq_t rte_power_get_uncore_freq =
> > power_get_dummy_uncore_freq; -rte_power_set_uncore_freq_t
> > rte_power_set_uncore_freq = power_set_dummy_uncore_freq;
> > -rte_power_uncore_freq_change_t rte_power_uncore_freq_max =
> > power_dummy_uncore_freq_max; -rte_power_uncore_freq_change_t
> > rte_power_uncore_freq_min = power_dummy_uncore_freq_min;
> > -rte_power_uncore_freqs_t rte_power_uncore_freqs =
> > power_dummy_uncore_freqs; -rte_power_uncore_get_num_freqs_t
> > rte_power_uncore_get_num_freqs = power_dummy_uncore_get_num_freqs;
> > -rte_power_uncore_get_num_pkgs_t rte_power_uncore_get_num_pkgs =
> > power_dummy_uncore_get_num_pkgs; -rte_power_uncore_get_num_dies_t
> > rte_power_uncore_get_num_dies = power_dummy_uncore_get_num_dies;
> > -
> > -static void
> > -reset_power_uncore_function_ptrs(void)
> > -{
> > - rte_power_get_uncore_freq = power_get_dummy_uncore_freq;
> > - rte_power_set_uncore_freq = power_set_dummy_uncore_freq;
> > - rte_power_uncore_freq_max = power_dummy_uncore_freq_max;
> > - rte_power_uncore_freq_min = power_dummy_uncore_freq_min;
> > - rte_power_uncore_freqs = power_dummy_uncore_freqs;
> > - rte_power_uncore_get_num_freqs = power_dummy_uncore_get_num_freqs;
> > - rte_power_uncore_get_num_pkgs = power_dummy_uncore_get_num_pkgs;
> > - rte_power_uncore_get_num_dies = power_dummy_uncore_get_num_dies;
> > -}
> > -
> > int
> > rte_power_set_uncore_env(enum rte_uncore_power_mgmt_env env)
> > {
> > - int ret;
> > + int ret = -1;
> > + struct rte_power_uncore_ops *ops;
> >
> > rte_spinlock_lock(&global_env_cfg_lock);
> >
> > - if (default_uncore_env != RTE_UNCORE_PM_ENV_NOT_SET) {
> > + if (global_uncore_env != RTE_UNCORE_PM_ENV_NOT_SET) {
> > POWER_LOG(ERR, "Uncore Power Management Env already set.");
> > - rte_spinlock_unlock(&global_env_cfg_lock);
> > - return -1;
> > + goto out;
> > }
> >
> <...>
> > + if (env <= RTE_DIM(uncore_env_str)) {
> > + RTE_TAILQ_FOREACH(ops, &uncore_ops_list, next)
> > + if (strncmp(ops->name, uncore_env_str[env],
> > + RTE_POWER_UNCORE_DRIVER_NAMESZ) == 0) {
> > + global_uncore_env = env;
> > + global_uncore_ops = ops;
> > + ret = 0;
> > + goto out;
> > + }
> > + POWER_LOG(ERR, "Power Management (%s) not supported",
> > + uncore_env_str[env]);
> > + } else
> > + POWER_LOG(ERR, "Invalid Power Management Environment");
> >
> > - default_uncore_env = env;
> > out:
> > rte_spinlock_unlock(&global_env_cfg_lock);
> > return ret;
> > @@ -139,15 +89,22 @@ void
> > rte_power_unset_uncore_env(void)
> > {
> > rte_spinlock_lock(&global_env_cfg_lock);
> > - default_uncore_env = RTE_UNCORE_PM_ENV_NOT_SET;
> > - reset_power_uncore_function_ptrs();
> > + global_uncore_env = RTE_UNCORE_PM_ENV_NOT_SET;
> > rte_spinlock_unlock(&global_env_cfg_lock);
> > }
> >
>
> How about abstract an ABI interface to intialize or set the uncore driver on platform
> by automatical.
>
> And later do power_intel_uncore_init_on_die() for each die on different package.
>
> > enum rte_uncore_power_mgmt_env
> > rte_power_get_uncore_env(void)
> > {
> > - return default_uncore_env;
> > + return global_uncore_env;
> > +}
> > +
> > +struct rte_power_uncore_ops *
> > +rte_power_get_uncore_ops(void)
> > +{
> > + RTE_ASSERT(global_uncore_ops != NULL);
> > +
> > + return global_uncore_ops;
> > }
> >
> > int
> > @@ -155,27 +112,29 @@ rte_power_uncore_init(unsigned int pkg, unsigned
> > int die)
> This pkg means the socket id on the platform, right?
> If so, I am not sure that the
> uncore_info[RTE_MAX_NUMA_NODES][MAX_NUMA_DIE] used in uncore lib is
> universal for all uncore driver.
> For example, uncore driver just support do uncore dvfs based on the socket unit.
> What shoud we do for this? we may need to think twice.
Yes, pkg represents a socket id. In platforms with a single uncore controller per socket,
the die ID should be set to '0' for the corresponding socket ID (pkg).
.
> > {
> > int ret = -1;
> >
> <...>
^ permalink raw reply [relevance 0%]
* Re: [PATCH v9 0/6] Introduce event pre-scheduling
2024-10-07 13:09 3% ` [PATCH v9 0/6] Introduce " pbhagavatula
2024-10-07 13:09 3% ` [PATCH v9 1/6] eventdev: introduce " pbhagavatula
@ 2024-10-08 5:17 0% ` Jerin Jacob
1 sibling, 0 replies; 200+ results
From: Jerin Jacob @ 2024-10-08 5:17 UTC (permalink / raw)
To: pbhagavatula
Cc: jerinj, sthotton, abdullah.sevincer, hemant.agrawal,
sachin.saxena, harry.van.haaren, mattias.ronnblom, liangma,
peter.mccarthy, stephen, dev
On Tue, Oct 8, 2024 at 3:14 AM <pbhagavatula@marvell.com> wrote:
>
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
>
> Event pre-scheduling improves scheduling performance by assigning events
> to event ports in advance when dequeues are issued.
> This series introduces various types and levels of pre-scheduling to the
> eventdev library.
>
> pre-scheduling Types:
> * RTE_EVENT_PRESCHEDULE_NONE: No pre-scheduling.
> * RTE_EVENT_PRESCHEDULE: Always issue a pre-schedule when dequeue is issued.
> * RTE_EVENT_PRESCHEDULE_ADAPTIVE: Delay issuing pre-schedule until there
> are no forward progress constraints with the held flow contexts.
>
> pre-scheduling Levels:
> * Event Device Level Pre-scheduling: Pre-scheduling can be enabled or disabled at the
> event device during configuration. Event devices can indicate pre-scheduling
> capabilities using `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
> `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event device info
> function `info.event_dev_cap`.
> * Event Port Level Prefetch: Pre-scheduling can be selectively enabled or disabled
> at the event port during runtime. Event devices can indicate this capability
> using `RTE_EVENT_PORT_CAP_PER_PORT_PRESCHEDULE` via the event device info
> function `info.event_port_cap`.
> * Application Controlled Prefetch Hint: Applications can provide hints to the
> event device to start pre-scheduling events using the new API
> `rte_event_port_pre-schedule`. Event devices can indicate this capabilities using
> `RTE_EVENT_DEV_CAP_PRESCHEDULE_EXPLICIT` via the event device info function
> `info.event_dev_cap`.
>
> The rte_event_dequeue_burst operation initiates the pre-schedule operation, which
> completes in parallel without affecting the flow context of the dequeued events and
> dequeue latency. On the next dequeue operation, the pre-scheduleed events are
> dequeued, and pre-schedule operation is initiated again.
>
> In the case of application-controlled pre-schedule hints, the currently held flow
> contexts, if any, are not affected by the pre-schedule operation. On the next
> dequeue operation, the pre-scheduleed events are returned, but pre-schedule is not
> initiated again until the application provides the hint again. If pre-scheduling
> is already enabled at the event device level or event port level, the hint is ignored.
>
> v2 Changes:
> - s/prefetch/pre-schedule (Mattias).
> v3 Changes:
> - Add CNXK preschedule implementation.
> - Update test-eventdev to use prescheduling.
> - Update examples to use prescheduling.
> v4 Changes:
> - Fix compilation.
> v5 Changes:
> - Update ABI changes
> - s/RTE_EVENT_DEV_PRESCHEDULE/RTE_EVENT_PRESCHEDULE/
> - s/RTE_EVENT_DEV_CAP_SW_PRESCHEDULE/RTE_EVENT_DEV_CAP_PRESCHEDULE_EXPLICIT/
> - s/RTE_EVENT_PORT_CAP_EVENT_PER_PORT_PRESCHEDULE/RTE_EVENT_PORT_CAP_PER_PORT_PRESCHEDULE
> - Various documentation fixes and updates.
> v6 Changes:
> - Mark new APIs as experimental (Stephen).
> v7 Changes:
> - Use correct patchset for rebase.
> v8 Changes:
> - fix checkpatch issues.
> v9 Changes:
> - Rebase update capability bits, fix release notes format.
Series-Acked-by: Jerin Jacob <jerinj@marvell.com>
Series applied to dpdk-next-net-eventdev/for-main. Thanks
^ permalink raw reply [relevance 0%]
* Re: [PATCH v4 1/1] dmadev: support priority configuration
2024-10-03 11:53 4% ` [PATCH v4 " Vamsi Krishna
@ 2024-10-08 2:36 0% ` fengchengwen
0 siblings, 0 replies; 200+ results
From: fengchengwen @ 2024-10-08 2:36 UTC (permalink / raw)
To: Vamsi Krishna, thomas, bruce.richardson, mb
Cc: dev, kevin.laatz, jerinj, conor.walsh, gmuthukrishn, vvelumuri,
g.singh, sachin.saxena, hemant.agrawal, Amit Prakash Shukla
Hi,
Sorry to reply now, because I just got back from our National Day vacation.
From the commit it seem confused about the dmadev and HW channel, Current one dmadev is corresponding a HW channel if its a hardware device.
So I suggest try not to mention HW channel, but the dmadev or DMA device.
Suggest the tile of the commit: dmadev: support strict priority configuration
On 2024/10/3 19:53, Vamsi Krishna wrote:
> From: Vamsi Attunuru <vattunuru@marvell.com>
>
> Some DMA controllers offer the ability to configure priority level
> for the hardware command queues, allowing for the prioritization of
Let align the queues to channels
for the hardware DMA channels,
> DMA command execution based on queue importance.
based on channel importance.
>
> This patch introduces the necessary fields in the dmadev structures to
> retrieve information about the hardware-supported priority levels and to
> enable priority configuration from the application.
This patch supports such strict priority configuration. If the dmadev supports, it should
declared the capability flag RTE_DMA_CAPA_PRI_POLICY_SP, then application could enable strict priority configuration.
>
> Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
> Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
> ---
> V4 changes:
> * Rebased onto the latest
>
> V3 changes:
> * Corrected patch title
>
> V2 changes:
> * Reverted removed text from release_24_11.rst
>
> V1 changes:
> * Added trace support
> * Added new capability flag
>
> Deprecation notice:
> https://patches.dpdk.org/project/dpdk/patch/20240730144612.2132848-1-amitprakashs@marvell.com/
>
> * Assuming we do not anticipate any advanced scheduling schemes for dmadev queues,
> this patch is intended to support a strict priority scheme.
>
> doc/guides/rel_notes/release_24_11.rst | 8 ++++++++
> lib/dmadev/rte_dmadev.c | 15 +++++++++++++++
> lib/dmadev/rte_dmadev.h | 21 +++++++++++++++++++++
> lib/dmadev/rte_dmadev_trace.h | 2 ++
> 4 files changed, 46 insertions(+)
>
> diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
> index e0a9aa55a1..9672d8c679 100644
> --- a/doc/guides/rel_notes/release_24_11.rst
> +++ b/doc/guides/rel_notes/release_24_11.rst
> @@ -67,6 +67,11 @@ New Features
>
> The new statistics are useful for debugging and profiling.
>
> +* **Added strict priority capability flag in dmadev.**
> +
> + Added new capability flag ``RTE_DMA_CAPA_PRI_POLICY_SP`` to check if the
> + DMA device supports assigning fixed priority to its channels, allowing
> + for better control over resource allocation and scheduling.
Please refine:
Add strict priority capability for dmadev.
Added new capability flag ``RTE_DMA_CAPA_PRI_POLICY_SP`` to check if the DMA device supports assigned fixed priority, allowing
for better control over resource allocation and scheduling.
>
> Removed Items
> -------------
> @@ -112,6 +117,9 @@ ABI Changes
> Also, make sure to start the actual text at the margin.
> =======================================================
>
> +* dmadev: Added ``nb_priorities`` field to ``rte_dma_info`` structure and
> + ``priority`` field to ``rte_dma_conf`` structure to get device supported
> + priority levels and configure required priority from the application.
>
> Known Issues
> ------------
> diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
> index 845727210f..3d9063dee3 100644
> --- a/lib/dmadev/rte_dmadev.c
> +++ b/lib/dmadev/rte_dmadev.c
> @@ -497,6 +497,21 @@ rte_dma_configure(int16_t dev_id, const struct rte_dma_conf *dev_conf)
> return -EINVAL;
> }
>
> + if (dev_conf->priority && !(dev_info.dev_capa & RTE_DMA_CAPA_PRI_POLICY_SP)) {
> + RTE_DMA_LOG(ERR, "Device %d don't support prioritization", dev_id);
> + return -EINVAL;
> + }
> +
> + if (dev_info.nb_priorities == 1) {
> + RTE_DMA_LOG(ERR, "Device %d must support more than 1 priority, or else 0", dev_id);
> + return -EINVAL;
> + }
Please consider other driver which don't set nb_priorities, then it will failed in the above branch.
Suggest add this verify in rte_dma_info_get(), make sure that this field should be > 1 if it supported PRI_POLICY_SP.
> +
> + if (dev_info.nb_priorities && (dev_conf->priority >= dev_info.nb_priorities)) {
> + RTE_DMA_LOG(ERR, "Device %d configure invalid priority", dev_id);
> + return -EINVAL;
> + }
> +
> if (*dev->dev_ops->dev_configure == NULL)
> return -ENOTSUP;
> ret = (*dev->dev_ops->dev_configure)(dev, dev_conf,
> diff --git a/lib/dmadev/rte_dmadev.h b/lib/dmadev/rte_dmadev.h
> index 5474a5281d..e5f730c327 100644
> --- a/lib/dmadev/rte_dmadev.h
> +++ b/lib/dmadev/rte_dmadev.h
> @@ -268,6 +268,16 @@ int16_t rte_dma_next_dev(int16_t start_dev_id);
> #define RTE_DMA_CAPA_OPS_COPY_SG RTE_BIT64(33)
> /** Support fill operation. */
> #define RTE_DMA_CAPA_OPS_FILL RTE_BIT64(34)
> +/** Support strict prioritization at DMA HW channel level
> + *
> + * If device supports HW channel prioritization then application could
> + * assign fixed priority to the DMA HW channel using 'priority' field in
> + * struct rte_dma_conf. Number of supported priority levels will be known
> + * from 'nb_priorities' field in struct rte_dma_info.
> + *
> + * DMA devices which support prioritization can advertise this capability.
How about:
Support strict priority scheduling.
Application could assign fixed priority to the DMA device using 'priority' field in
struct rte_dma_conf. Number of supported priority levels will be known from 'nb_priorities'
field in struct rte_dma_info.
> + */
> +#define RTE_DMA_CAPA_PRI_POLICY_SP RTE_BIT64(35)
This capa is a control plane flag, so please add after RTE_DMA_CAPA_M2D_AUTO_FREE.
> /**@}*/
>
> /**
> @@ -297,6 +307,10 @@ struct rte_dma_info {
> int16_t numa_node;
> /** Number of virtual DMA channel configured. */
> uint16_t nb_vchans;
> + /** Number of priority levels (must be > 1), if supported by DMA HW channel.
> + * 0 otherwise.
How about "Number of priority levels (must be > 1) if supported priority scheduling."
A DMA HW channel was just one dmadev, suggest don't introduce it.
> + */
> + uint16_t nb_priorities;
> };
>
> /**
> @@ -332,6 +346,13 @@ struct rte_dma_conf {
> * @see RTE_DMA_CAPA_SILENT
> */
> bool enable_silent;
> + /* The priority of the DMA HW channel.
> + * This value cannot be greater than or equal to the field 'nb_priorities'
> + * of struct rte_dma_info which get from rte_dma_info_get().
> + * Among the values between '0' and 'nb_priorities - 1', lowest value
> + * indicates higher priority and vice-versa.
How about (you could retouch it. I'm not a native English speaker.):
The priority of the DMA device.
If the the DMA device don't support priority scheduling, this value should be zero.
Otherwise, the value should lower than the field 'nb_priorities' of struct rte_dma_info
which get from rte_dma_info_get(). And also lowest value indicates higher priority and vice-versa.
> + */
> + uint16_t priority;
> };
>
> /**
> diff --git a/lib/dmadev/rte_dmadev_trace.h b/lib/dmadev/rte_dmadev_trace.h
> index e55c4c6091..be089c065c 100644
> --- a/lib/dmadev/rte_dmadev_trace.h
> +++ b/lib/dmadev/rte_dmadev_trace.h
> @@ -35,6 +35,7 @@ RTE_TRACE_POINT(
> rte_trace_point_emit_u16(dev_info->max_sges);
> rte_trace_point_emit_i16(dev_info->numa_node);
> rte_trace_point_emit_u16(dev_info->nb_vchans);
> + rte_trace_point_emit_u16(dev_info->nb_priorities);
> )
>
> RTE_TRACE_POINT(
> @@ -48,6 +49,7 @@ RTE_TRACE_POINT(
> int enable_silent = (int)dev_conf->enable_silent;
> rte_trace_point_emit_i16(dev_id);
> rte_trace_point_emit_u16(dev_conf->nb_vchans);
> + rte_trace_point_emit_u16(dev_conf->priority);
> rte_trace_point_emit_int(enable_silent);
> rte_trace_point_emit_int(ret);
> )
Please also modify dma_capability_name and dmadev_handle_dev_info,
^ permalink raw reply [relevance 0%]
* Re: [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port
2024-09-29 5:52 3% ` [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port Huisong Li
2024-09-29 5:52 2% ` [PATCH RESEND v7 2/5] ethdev: fix skip valid port in probing callback Huisong Li
@ 2024-10-08 2:32 0% ` lihuisong (C)
1 sibling, 0 replies; 200+ results
From: lihuisong (C) @ 2024-10-08 2:32 UTC (permalink / raw)
To: thomas, ferruh.yigit, andrew.rybchenko; +Cc: dev, fengchengwen, liuyonglong
Hi Thomas and Ferruh,
We've discussed it on and off a few times, and we've reached some consensus.
They've been going through more than 2 years😅
Can you have a look at this series again?
If we really don't need it, I will drop it from my upstreaming list.
/Huisong
在 2024/9/29 13:52, Huisong Li 写道:
> This patchset fix some bugs and support attaching and detaching port
> in primary and secondary.
>
> ---
> -v7: fix conflicts
> -v6: adjust rte_eth_dev_is_used position based on alphabetical order
> in version.map
> -v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi break.
> -v4: fix a misspelling.
> -v3:
> #1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification
> for other bus type.
> #2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve
> the probelm in patch 2/5.
> -v2: resend due to CI unexplained failure.
>
> Huisong Li (5):
> drivers/bus: restore driver assignment at front of probing
> ethdev: fix skip valid port in probing callback
> app/testpmd: check the validity of the port
> app/testpmd: add attach and detach port for multiple process
> app/testpmd: stop forwarding in new or destroy event
>
> app/test-pmd/testpmd.c | 47 +++++++++++++++---------
> app/test-pmd/testpmd.h | 1 -
> drivers/bus/auxiliary/auxiliary_common.c | 9 ++++-
> drivers/bus/dpaa/dpaa_bus.c | 9 ++++-
> drivers/bus/fslmc/fslmc_bus.c | 8 +++-
> drivers/bus/ifpga/ifpga_bus.c | 12 ++++--
> drivers/bus/pci/pci_common.c | 9 ++++-
> drivers/bus/vdev/vdev.c | 10 ++++-
> drivers/bus/vmbus/vmbus_common.c | 9 ++++-
> drivers/net/bnxt/bnxt_ethdev.c | 3 +-
> drivers/net/bonding/bonding_testpmd.c | 1 -
> drivers/net/mlx5/mlx5.c | 2 +-
> lib/ethdev/ethdev_driver.c | 13 +++++--
> lib/ethdev/ethdev_driver.h | 12 ++++++
> lib/ethdev/ethdev_pci.h | 2 +-
> lib/ethdev/rte_class_eth.c | 2 +-
> lib/ethdev/rte_ethdev.c | 4 +-
> lib/ethdev/rte_ethdev.h | 4 +-
> lib/ethdev/version.map | 1 +
> 19 files changed, 114 insertions(+), 44 deletions(-)
>
^ permalink raw reply [relevance 0%]
* [PATCH v9 1/6] eventdev: introduce event pre-scheduling
2024-10-07 13:09 3% ` [PATCH v9 0/6] Introduce " pbhagavatula
@ 2024-10-07 13:09 3% ` pbhagavatula
2024-10-08 5:17 0% ` [PATCH v9 0/6] Introduce " Jerin Jacob
1 sibling, 0 replies; 200+ results
From: pbhagavatula @ 2024-10-07 13:09 UTC (permalink / raw)
To: jerinj, sthotton, abdullah.sevincer, hemant.agrawal,
sachin.saxena, harry.van.haaren, mattias.ronnblom, liangma,
peter.mccarthy, stephen
Cc: dev, Pavan Nikhilesh
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
Event pre-scheduling improves scheduling performance by assigning events
to event ports in advance when dequeues are issued.
The dequeue operation initiates the pre-schedule operation, which completes
in parallel without affecting the dequeued event flow contexts and
dequeue latency.
Event devices can indicate pre-scheduling capabilities using
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event device info
function `info.event_dev_cap`.
Applications can select the pre-schedule type and configure it through
`rte_event_dev_config.preschedule_type` during `rte_event_dev_configure`.
The supported pre-schedule types are:
* `RTE_EVENT_PRESCHEDULE_NONE` - No pre-scheduling.
* `RTE_EVENT_PRESCHEDULE` - Always issue a pre-schedule on dequeue.
* `RTE_EVENT_PRESCHEDULE_ADAPTIVE` - Delay issuing pre-schedule until
there are no forward progress constraints with the held flow contexts.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
app/test/test_eventdev.c | 108 ++++++++++++++++++++
doc/guides/eventdevs/features/default.ini | 1 +
doc/guides/prog_guide/eventdev/eventdev.rst | 23 +++++
doc/guides/rel_notes/release_24_11.rst | 10 ++
lib/eventdev/rte_eventdev.h | 49 +++++++++
5 files changed, 191 insertions(+)
diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c
index 9a6c8f470c..a45d1396d7 100644
--- a/app/test/test_eventdev.c
+++ b/app/test/test_eventdev.c
@@ -1251,6 +1251,112 @@ test_eventdev_profile_switch(void)
return TEST_SUCCESS;
}
+static int
+preschedule_test(enum rte_event_dev_preschedule_type preschedule_type, const char *preschedule_name)
+{
+#define NB_EVENTS 1024
+ uint64_t start, total;
+ struct rte_event ev;
+ int rc, cnt;
+
+ ev.event_type = RTE_EVENT_TYPE_CPU;
+ ev.queue_id = 0;
+ ev.op = RTE_EVENT_OP_NEW;
+ ev.u64 = 0xBADF00D0;
+
+ for (cnt = 0; cnt < NB_EVENTS; cnt++) {
+ ev.flow_id = cnt;
+ rc = rte_event_enqueue_burst(TEST_DEV_ID, 0, &ev, 1);
+ TEST_ASSERT(rc == 1, "Failed to enqueue event");
+ }
+
+ RTE_SET_USED(preschedule_type);
+ total = 0;
+ while (cnt) {
+ start = rte_rdtsc_precise();
+ rc = rte_event_dequeue_burst(TEST_DEV_ID, 0, &ev, 1, 0);
+ if (rc) {
+ total += rte_rdtsc_precise() - start;
+ cnt--;
+ }
+ }
+ printf("Preschedule type : %s, avg cycles %" PRIu64 "\n", preschedule_name,
+ total / NB_EVENTS);
+
+ return TEST_SUCCESS;
+}
+
+static int
+preschedule_configure(enum rte_event_dev_preschedule_type type, struct rte_event_dev_info *info)
+{
+ struct rte_event_dev_config dev_conf;
+ struct rte_event_queue_conf qcfg;
+ struct rte_event_port_conf pcfg;
+ int rc;
+
+ devconf_set_default_sane_values(&dev_conf, info);
+ dev_conf.nb_event_ports = 1;
+ dev_conf.nb_event_queues = 1;
+ dev_conf.preschedule_type = type;
+
+ rc = rte_event_dev_configure(TEST_DEV_ID, &dev_conf);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+
+ rc = rte_event_port_default_conf_get(TEST_DEV_ID, 0, &pcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to get port0 default config");
+ rc = rte_event_port_setup(TEST_DEV_ID, 0, &pcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to setup port0");
+
+ rc = rte_event_queue_default_conf_get(TEST_DEV_ID, 0, &qcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to get queue0 default config");
+ rc = rte_event_queue_setup(TEST_DEV_ID, 0, &qcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to setup queue0");
+
+ rc = rte_event_port_link(TEST_DEV_ID, 0, NULL, NULL, 0);
+ TEST_ASSERT(rc == (int)dev_conf.nb_event_queues, "Failed to link port, device %d",
+ TEST_DEV_ID);
+
+ rc = rte_event_dev_start(TEST_DEV_ID);
+ TEST_ASSERT_SUCCESS(rc, "Failed to start event device");
+
+ return 0;
+}
+
+static int
+test_eventdev_preschedule_configure(void)
+{
+ struct rte_event_dev_info info;
+ int rc;
+
+ rte_event_dev_info_get(TEST_DEV_ID, &info);
+
+ if ((info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE) == 0)
+ return TEST_SKIPPED;
+
+ rc = preschedule_configure(RTE_EVENT_PRESCHEDULE_NONE, &info);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+ rc = preschedule_test(RTE_EVENT_PRESCHEDULE_NONE, "RTE_EVENT_PRESCHEDULE_NONE");
+ TEST_ASSERT_SUCCESS(rc, "Failed to test preschedule RTE_EVENT_PRESCHEDULE_NONE");
+
+ rte_event_dev_stop(TEST_DEV_ID);
+ rc = preschedule_configure(RTE_EVENT_PRESCHEDULE, &info);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+ rc = preschedule_test(RTE_EVENT_PRESCHEDULE, "RTE_EVENT_PRESCHEDULE");
+ TEST_ASSERT_SUCCESS(rc, "Failed to test preschedule RTE_EVENT_PRESCHEDULE");
+
+ if (info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE) {
+ rte_event_dev_stop(TEST_DEV_ID);
+ rc = preschedule_configure(RTE_EVENT_PRESCHEDULE_ADAPTIVE, &info);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+ rc = preschedule_test(RTE_EVENT_PRESCHEDULE_ADAPTIVE,
+ "RTE_EVENT_PRESCHEDULE_ADAPTIVE");
+ TEST_ASSERT_SUCCESS(rc,
+ "Failed to test preschedule RTE_EVENT_PRESCHEDULE_ADAPTIVE");
+ }
+
+ return TEST_SUCCESS;
+}
+
static int
test_eventdev_close(void)
{
@@ -1311,6 +1417,8 @@ static struct unit_test_suite eventdev_common_testsuite = {
test_eventdev_start_stop),
TEST_CASE_ST(eventdev_configure_setup, eventdev_stop_device,
test_eventdev_profile_switch),
+ TEST_CASE_ST(eventdev_configure_setup, NULL,
+ test_eventdev_preschedule_configure),
TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
test_eventdev_link),
TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
diff --git a/doc/guides/eventdevs/features/default.ini b/doc/guides/eventdevs/features/default.ini
index 7c4ee99238..fa24ba38b4 100644
--- a/doc/guides/eventdevs/features/default.ini
+++ b/doc/guides/eventdevs/features/default.ini
@@ -23,6 +23,7 @@ maintenance_free =
runtime_queue_attr =
profile_links =
independent_enq =
+preschedule =
;
; Features of a default Ethernet Rx adapter.
diff --git a/doc/guides/prog_guide/eventdev/eventdev.rst b/doc/guides/prog_guide/eventdev/eventdev.rst
index 515744d8ec..a630ac7ebe 100644
--- a/doc/guides/prog_guide/eventdev/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev/eventdev.rst
@@ -357,6 +357,29 @@ Worker path:
// Process the event received.
}
+Event Pre-scheduling
+~~~~~~~~~~~~~~~~~~~~
+
+Event pre-scheduling improves scheduling performance by assigning events to event ports in advance
+when dequeues are issued.
+The `rte_event_dequeue_burst` operation initiates the pre-schedule operation, which completes
+in parallel without affecting the dequeued event flow contexts and dequeue latency.
+On the next dequeue operation, the pre-scheduled events are dequeued and pre-schedule is initiated
+again.
+
+An application can use event pre-scheduling if the event device supports it at either device
+level or at a individual port level.
+The application must check pre-schedule capability by checking if ``rte_event_dev_info.event_dev_cap``
+has the bit ``RTE_EVENT_DEV_CAP_PRESCHEDULE`` or ``RTE_EVENT_DEV_CAP_PRESCHEDULE_ADAPTIVE`` set, if
+present pre-scheduling can be enabled at device
+configuration time by setting appropriate pre-schedule type in ``rte_event_dev_config.preschedule``.
+
+The following pre-schedule types are supported:
+ * ``RTE_EVENT_PRESCHEDULE_NONE`` - No pre-scheduling.
+ * ``RTE_EVENT_PRESCHEDULE`` - Always issue a pre-schedule when dequeue is issued.
+ * ``RTE_EVENT_PRESCHEDULE_ADAPTIVE`` - Issue pre-schedule when dequeue is issued and there are
+ no forward progress constraints.
+
Starting the EventDev
~~~~~~~~~~~~~~~~~~~~~
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index f806c08244..94e1d30777 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -79,6 +79,14 @@ New Features
* Updated DSW driver for independent enqueue feature.
+* **Added event device pre-scheduling support.**
+
+ Added support for pre-scheduling of events to event ports to improve
+ scheduling performance and latency.
+
+ * Added ``rte_event_dev_config::preschedule_type`` to configure the device
+ level pre-scheduling type.
+
Removed Items
-------------
@@ -124,6 +132,8 @@ ABI Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* eventdev: Added ``preschedule_type`` field to ``rte_event_dev_config``
+ structure.
Known Issues
------------
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 73a44b2ac5..00bbd3b28f 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -465,6 +465,30 @@ struct rte_event;
* only applies to ports that have enabled independent enqueue feature.
*/
+#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE (1ULL << 17)
+/**< Event device supports event pre-scheduling.
+ *
+ * When this capability is available, the application can enable event pre-scheduling on the event
+ * device to pre-schedule events to a event port when `rte_event_dequeue_burst()`
+ * is issued.
+ * The pre-schedule process starts with the `rte_event_dequeue_burst()` call and the
+ * pre-scheduled events are returned on the next `rte_event_dequeue_burst()` call.
+ *
+ * @see rte_event_dev_configure()
+ */
+
+#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE (1ULL << 18)
+/**< Event device supports adaptive event pre-scheduling.
+ *
+ * When this capability is available, the application can enable adaptive pre-scheduling
+ * on the event device where the events are pre-scheduled when there are no forward
+ * progress constraints with the currently held flow contexts.
+ * The pre-schedule process starts with the `rte_event_dequeue_burst()` call and the
+ * pre-scheduled events are returned on the next `rte_event_dequeue_burst()` call.
+ *
+ * @see rte_event_dev_configure()
+ */
+
/* Event device priority levels */
#define RTE_EVENT_DEV_PRIORITY_HIGHEST 0
/**< Highest priority level for events and queues.
@@ -699,6 +723,26 @@ rte_event_dev_attr_get(uint8_t dev_id, uint32_t attr_id,
* @see rte_event_dequeue_timeout_ticks(), rte_event_dequeue_burst()
*/
+/** Event device pre-schedule type enumeration. */
+enum rte_event_dev_preschedule_type {
+ RTE_EVENT_PRESCHEDULE_NONE,
+ /**< Disable pre-schedule across the event device or on a given event port.
+ * @ref rte_event_dev_config.preschedule_type
+ */
+ RTE_EVENT_PRESCHEDULE,
+ /**< Enable pre-schedule always across the event device or a given event port.
+ * @ref rte_event_dev_config.preschedule_type
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
+ */
+ RTE_EVENT_PRESCHEDULE_ADAPTIVE,
+ /**< Enable adaptive pre-schedule across the event device or a given event port.
+ * Delay issuing pre-schedule until there are no forward progress constraints with
+ * the held flow contexts.
+ * @ref rte_event_dev_config.preschedule_type
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
+ */
+};
+
/** Event device configuration structure */
struct rte_event_dev_config {
uint32_t dequeue_timeout_ns;
@@ -771,6 +815,11 @@ struct rte_event_dev_config {
* optimized for single-link usage, this field is a hint for how many
* to allocate; otherwise, regular event ports and queues will be used.
*/
+ enum rte_event_dev_preschedule_type preschedule_type;
+ /**< Event pre-schedule type to use across the event device, if supported.
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
+ */
};
/**
--
2.25.1
^ permalink raw reply [relevance 3%]
* [PATCH v9 0/6] Introduce event pre-scheduling
2024-10-06 17:06 3% ` [PATCH v8 0/6] Introduce " pbhagavatula
2024-10-06 17:06 3% ` [PATCH v8 1/6] eventdev: introduce " pbhagavatula
@ 2024-10-07 13:09 3% ` pbhagavatula
2024-10-07 13:09 3% ` [PATCH v9 1/6] eventdev: introduce " pbhagavatula
2024-10-08 5:17 0% ` [PATCH v9 0/6] Introduce " Jerin Jacob
1 sibling, 2 replies; 200+ results
From: pbhagavatula @ 2024-10-07 13:09 UTC (permalink / raw)
To: jerinj, sthotton, abdullah.sevincer, hemant.agrawal,
sachin.saxena, harry.van.haaren, mattias.ronnblom, liangma,
peter.mccarthy, stephen
Cc: dev, Pavan Nikhilesh
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
Event pre-scheduling improves scheduling performance by assigning events
to event ports in advance when dequeues are issued.
This series introduces various types and levels of pre-scheduling to the
eventdev library.
pre-scheduling Types:
* RTE_EVENT_PRESCHEDULE_NONE: No pre-scheduling.
* RTE_EVENT_PRESCHEDULE: Always issue a pre-schedule when dequeue is issued.
* RTE_EVENT_PRESCHEDULE_ADAPTIVE: Delay issuing pre-schedule until there
are no forward progress constraints with the held flow contexts.
pre-scheduling Levels:
* Event Device Level Pre-scheduling: Pre-scheduling can be enabled or disabled at the
event device during configuration. Event devices can indicate pre-scheduling
capabilities using `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event device info
function `info.event_dev_cap`.
* Event Port Level Prefetch: Pre-scheduling can be selectively enabled or disabled
at the event port during runtime. Event devices can indicate this capability
using `RTE_EVENT_PORT_CAP_PER_PORT_PRESCHEDULE` via the event device info
function `info.event_port_cap`.
* Application Controlled Prefetch Hint: Applications can provide hints to the
event device to start pre-scheduling events using the new API
`rte_event_port_pre-schedule`. Event devices can indicate this capabilities using
`RTE_EVENT_DEV_CAP_PRESCHEDULE_EXPLICIT` via the event device info function
`info.event_dev_cap`.
The rte_event_dequeue_burst operation initiates the pre-schedule operation, which
completes in parallel without affecting the flow context of the dequeued events and
dequeue latency. On the next dequeue operation, the pre-scheduleed events are
dequeued, and pre-schedule operation is initiated again.
In the case of application-controlled pre-schedule hints, the currently held flow
contexts, if any, are not affected by the pre-schedule operation. On the next
dequeue operation, the pre-scheduleed events are returned, but pre-schedule is not
initiated again until the application provides the hint again. If pre-scheduling
is already enabled at the event device level or event port level, the hint is ignored.
v2 Changes:
- s/prefetch/pre-schedule (Mattias).
v3 Changes:
- Add CNXK preschedule implementation.
- Update test-eventdev to use prescheduling.
- Update examples to use prescheduling.
v4 Changes:
- Fix compilation.
v5 Changes:
- Update ABI changes
- s/RTE_EVENT_DEV_PRESCHEDULE/RTE_EVENT_PRESCHEDULE/
- s/RTE_EVENT_DEV_CAP_SW_PRESCHEDULE/RTE_EVENT_DEV_CAP_PRESCHEDULE_EXPLICIT/
- s/RTE_EVENT_PORT_CAP_EVENT_PER_PORT_PRESCHEDULE/RTE_EVENT_PORT_CAP_PER_PORT_PRESCHEDULE
- Various documentation fixes and updates.
v6 Changes:
- Mark new APIs as experimental (Stephen).
v7 Changes:
- Use correct patchset for rebase.
v8 Changes:
- fix checkpatch issues.
v9 Changes:
- Rebase update capability bits, fix release notes format.
Pavan Nikhilesh (6):
eventdev: introduce event pre-scheduling
eventdev: add event port pre-schedule modify
eventdev: add event preschedule hint
event/cnkx: add pre-schedule support
app/test-eventdev: add pre-scheduling support
examples: use eventdev pre-scheduling
app/test-eventdev/evt_common.h | 45 +++--
app/test-eventdev/evt_options.c | 17 ++
app/test-eventdev/evt_options.h | 1 +
app/test/test_eventdev.c | 143 ++++++++++++++++
doc/guides/eventdevs/cnxk.rst | 10 --
doc/guides/eventdevs/features/cnxk.ini | 1 +
doc/guides/eventdevs/features/default.ini | 1 +
doc/guides/prog_guide/eventdev/eventdev.rst | 48 ++++++
doc/guides/rel_notes/release_24_11.rst | 16 ++
doc/guides/tools/testeventdev.rst | 6 +
drivers/event/cnxk/cn10k_eventdev.c | 19 ++-
drivers/event/cnxk/cn10k_worker.c | 21 +++
drivers/event/cnxk/cn10k_worker.h | 2 +
drivers/event/cnxk/cnxk_eventdev.c | 2 -
drivers/event/cnxk/cnxk_eventdev.h | 1 -
.../pipeline_worker_generic.c | 6 +
.../eventdev_pipeline/pipeline_worker_tx.c | 6 +
examples/ipsec-secgw/event_helper.c | 6 +
examples/l2fwd-event/l2fwd_event_generic.c | 6 +
.../l2fwd-event/l2fwd_event_internal_port.c | 6 +
examples/l3fwd/l3fwd_event_generic.c | 6 +
examples/l3fwd/l3fwd_event_internal_port.c | 6 +
lib/eventdev/eventdev_pmd.h | 4 +
lib/eventdev/eventdev_private.c | 41 ++++-
lib/eventdev/eventdev_trace_points.c | 6 +
lib/eventdev/rte_eventdev.h | 161 ++++++++++++++++++
lib/eventdev/rte_eventdev_core.h | 14 +-
lib/eventdev/rte_eventdev_trace_fp.h | 19 ++-
lib/eventdev/version.map | 6 +
29 files changed, 597 insertions(+), 29 deletions(-)
--
2.25.1
^ permalink raw reply [relevance 3%]
* [PATCH v9] eal: add build-time option to omit trace
2024-09-18 8:55 3% [PATCH] eal: add build-time option to omit trace Morten Brørup
` (6 preceding siblings ...)
2024-10-06 14:03 2% ` [PATCH v7] " Morten Brørup
@ 2024-10-07 11:46 3% ` Morten Brørup
2024-10-08 7:16 0% ` Morten Brørup
2024-10-08 10:15 0% ` David Marchand
7 siblings, 2 replies; 200+ results
From: Morten Brørup @ 2024-10-07 11:46 UTC (permalink / raw)
To: Jerin Jacob, Sunil Kumar Kori; +Cc: dev, Morten Brørup, Stephen Hemminger
Some applications want to omit the trace feature.
Either to reduce the memory footprint, to reduce the exposed attack
surface, or for other reasons.
This patch adds an option in rte_config.h to include or omit trace in the
build. Trace is included by default.
Omitting trace works by omitting all trace points.
For API and ABI compatibility, the trace feature itself remains.
Furthermore, a public function to determine if trace is build time enabled
is added; mainly for the benefit of the dpdk-test application.
Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Jerin Jacob <jerinj@marvell.com>
---
v9:
* Assume library and application are built with same rte_config.h.
* Renamed internal function __rte_trace_point_generic_is_enabled() to
rte_trace_feature_is_enabled(), which is public. (Jerin Jacob)
* Removed changes that became superfluous with the above change.
v8:
* Added Stephen's Ack to v4, forgot to carry over.
v7:
* Updated version.map to not export __rte_trace_feature_is_enabled for
Windows target.
v6:
* Removed test_trace_perf.c changes; they don't compile for Windows
target, and are superfluous.
v5:
* Added public function rte_trace_feature_is_enabled(), to test if trace
is build time enabled in both the DPDK and the application. Use in test
application instead of private function. (Jerin Jacob)
v4:
* Added check for generic trace enabled when registering trace points, in
RTE_INIT. (Jerin Jacob)
* Test application uses function instead of macro to check if generic
trace is enabled. (Jerin Jacob)
* Performance test application uses function to check if generic trace is
enabled.
v3:
* Simpler version with much fewer ifdefs. (Jerin Jacob)
v2:
* Added/modified macros required for building applications with trace
omitted.
---
app/test/test_trace.c | 4 ++++
config/rte_config.h | 1 +
lib/eal/include/rte_trace.h | 19 +++++++++++++++++++
lib/eal/include/rte_trace_point.h | 3 +++
lib/eal/include/rte_trace_point_register.h | 2 ++
5 files changed, 29 insertions(+)
diff --git a/app/test/test_trace.c b/app/test/test_trace.c
index 00809f433b..8ea1443044 100644
--- a/app/test/test_trace.c
+++ b/app/test/test_trace.c
@@ -245,6 +245,10 @@ static struct unit_test_suite trace_tests = {
static int
test_trace(void)
{
+ if (!rte_trace_feature_is_enabled()) {
+ printf("Trace omitted at build-time, skipping test\n");
+ return TEST_SKIPPED;
+ }
return unit_test_suite_runner(&trace_tests);
}
diff --git a/config/rte_config.h b/config/rte_config.h
index dd7bb0d35b..fd6f8a2f1a 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -49,6 +49,7 @@
#define RTE_MAX_TAILQ 32
#define RTE_LOG_DP_LEVEL RTE_LOG_INFO
#define RTE_MAX_VFIO_CONTAINERS 64
+#define RTE_TRACE 1
/* bsd module defines */
#define RTE_CONTIGMEM_MAX_NUM_BUFS 64
diff --git a/lib/eal/include/rte_trace.h b/lib/eal/include/rte_trace.h
index a6e991fad3..6eac95188b 100644
--- a/lib/eal/include/rte_trace.h
+++ b/lib/eal/include/rte_trace.h
@@ -35,6 +35,25 @@ extern "C" {
__rte_experimental
bool rte_trace_is_enabled(void);
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * Test if trace feature is enabled at compile time.
+ *
+ * @return
+ * true if trace feature is enabled, false otherwise.
+ */
+static __rte_always_inline
+bool rte_trace_feature_is_enabled(void)
+{
+#ifdef RTE_TRACE
+ return true;
+#else
+ return false;
+#endif
+}
+
/**
* Enumerate trace mode operation.
*/
diff --git a/lib/eal/include/rte_trace_point.h b/lib/eal/include/rte_trace_point.h
index 41e2a7f99e..dfe89302ed 100644
--- a/lib/eal/include/rte_trace_point.h
+++ b/lib/eal/include/rte_trace_point.h
@@ -30,6 +30,7 @@ extern "C" {
#include <rte_per_lcore.h>
#include <rte_stdatomic.h>
#include <rte_string_fns.h>
+#include <rte_trace.h>
#include <rte_uuid.h>
/** The tracepoint object. */
@@ -359,6 +360,8 @@ __rte_trace_point_emit_ev_header(void *mem, uint64_t in)
#define __rte_trace_point_emit_header_generic(t) \
void *mem; \
do { \
+ if (!rte_trace_feature_is_enabled()) \
+ return; \
const uint64_t val = rte_atomic_load_explicit(t, rte_memory_order_acquire); \
if (likely(!(val & __RTE_TRACE_FIELD_ENABLE_MASK))) \
return; \
diff --git a/lib/eal/include/rte_trace_point_register.h b/lib/eal/include/rte_trace_point_register.h
index 41260e5964..283dcef75d 100644
--- a/lib/eal/include/rte_trace_point_register.h
+++ b/lib/eal/include/rte_trace_point_register.h
@@ -23,6 +23,8 @@ rte_trace_point_t __rte_section("__rte_trace_point") __##trace; \
static const char __##trace##_name[] = RTE_STR(name); \
RTE_INIT(trace##_init) \
{ \
+ if (!rte_trace_feature_is_enabled()) \
+ return; \
__rte_trace_point_register(&__##trace, __##trace##_name, \
(void (*)(void)) trace); \
}
--
2.43.0
^ permalink raw reply [relevance 3%]
* RE: [PATCH v6] eal: add build-time option to omit trace
2024-10-07 5:45 0% ` Jerin Jacob
@ 2024-10-07 6:07 0% ` Morten Brørup
0 siblings, 0 replies; 200+ results
From: Morten Brørup @ 2024-10-07 6:07 UTC (permalink / raw)
To: Jerin Jacob; +Cc: Jerin Jacob, Sunil Kumar Kori, dev
> From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> Sent: Monday, 7 October 2024 07.45
>
> On Sun, Oct 6, 2024 at 7:44 PM Morten Brørup <mb@smartsharesystems.com>
> wrote:
> >
> > Some applications want to omit the trace feature.
> > Either to reduce the memory footprint, to reduce the exposed attack
> > surface, or for other reasons.
> >
> > This patch adds an option in rte_config.h to include or omit trace in
> the
> > build. Trace is included by default.
> >
> > Omitting trace works by omitting all trace points.
> > For API and ABI compatibility, the trace feature itself remains.
> >
> > Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
> > ---
> > v6:
> > Removed test_trace_perf.c changes; they don't compile for Windows
> > target, and are superfluous.
> > v5:
> > Added public function rte_trace_feature_is_enabled(), to test if
> trace
> > is build time enabled in both the DPDK and the application. Use in
> test
> > application instead of private function. (Jerin Jacob)
> > v4:
> > * Added check for generic trace enabled when registering trace
> points, in
> > RTE_INIT. (Jerin Jacob)
> > * Test application uses function instead of macro to check if generic
> > trace is enabled. (Jerin Jacob)
> > * Performance test application uses function to check if generic
> trace is
> > enabled.
> > v3:
> > * Simpler version with much fewer ifdefs. (Jerin Jacob)
> > v2:
> > * Added/modified macros required for building applications with trace
> > omitted.
> > ---
> > app/test/test_trace.c | 4 +++
> > config/rte_config.h | 1 +
> > lib/eal/common/eal_common_trace.c | 10 +++++++
> > lib/eal/include/rte_trace.h | 33
> ++++++++++++++++++++++
> > lib/eal/include/rte_trace_point.h | 21 ++++++++++++++
> > lib/eal/include/rte_trace_point_register.h | 2 ++
> > lib/eal/version.map | 3 ++
> > 7 files changed, 74 insertions(+)
> >
> > diff --git a/app/test/test_trace.c b/app/test/test_trace.c
> > index 00809f433b..8ea1443044 100644
> > --- a/app/test/test_trace.c
> > +++ b/app/test/test_trace.c
> > @@ -245,6 +245,10 @@ static struct unit_test_suite trace_tests = {
> > static int
> > test_trace(void)
> > {
> > + if (!rte_trace_feature_is_enabled()) {
> > + printf("Trace omitted at build-time, skipping
> test\n");
> > + return TEST_SKIPPED;
> > + }
> > return unit_test_suite_runner(&trace_tests);
> > }
> >
> > diff --git a/config/rte_config.h b/config/rte_config.h
> > index dd7bb0d35b..fd6f8a2f1a 100644
> > --- a/config/rte_config.h
> > +++ b/config/rte_config.h
> > @@ -49,6 +49,7 @@
> > #define RTE_MAX_TAILQ 32
> > #define RTE_LOG_DP_LEVEL RTE_LOG_INFO
> > #define RTE_MAX_VFIO_CONTAINERS 64
> > +#define RTE_TRACE 1
> >
> > /* bsd module defines */
> > #define RTE_CONTIGMEM_MAX_NUM_BUFS 64
> > diff --git a/lib/eal/common/eal_common_trace.c
> b/lib/eal/common/eal_common_trace.c
> > index 918f49bf4f..06130c756d 100644
> > --- a/lib/eal/common/eal_common_trace.c
> > +++ b/lib/eal/common/eal_common_trace.c
> > @@ -100,6 +100,16 @@ rte_trace_is_enabled(void)
> > return rte_atomic_load_explicit(&trace.status,
> rte_memory_order_acquire) != 0;
> > }
> >
> > +bool
> > +__rte_trace_feature_is_enabled(void)
> > +{
> > +#ifdef RTE_TRACE
> > + return true;
> > +#else
> > + return false;
> > +#endif
> > +}
> > +static __rte_always_inline
> > +bool rte_trace_feature_is_enabled(void)
> > +{
> > +#ifdef RTE_TRACE
> > + return __rte_trace_feature_is_enabled();
> > +#else
> > + return false;
> > +#endif
> > +}
> > +__rte_experimental
> > +static __rte_always_inline bool
> > +__rte_trace_point_generic_is_enabled(void)
> > +{
> > +#ifdef RTE_TRACE
> > + return true;
> > +#else
> > + return false;
> > +#endif
>
> __rte_trace_feature_is_enabled(), rte_trace_feature_is_enabled(),
> __rte_trace_point_generic_is_enabled() are duplicates.
> There is no harm in using a public API inside the implementation.
> Please keep only rte_trace_feature_is_enabled()
> and use it across implementation and app/test.
They are there to support DPDK being built as a library, and the application being built separately with a different rte_config.h.
But I suppose we generally don't support that. I will spin another version, which assumes that both DPDK (library) and application are built with the same rte_config.h.
I will rename __rte_trace_point_generic_is_enabled() to rte_trace_feature_is_enabled(), make it non-internal, and use that everywhere. It still needs to be inline, so the compiler can recognize it and omit the implementation when building with RTE_TRACE disabled.
>
> > +}
> > +
> > /**
> > * @internal
> > *
>
> > rte_vfio_get_device_info; # WINDOWS_NO_EXPORT
> > +
> > + # added in 24.11
> > + __rte_trace_feature_is_enabled;
>
> rte_trace_feature_is_enabled;
It will be inline, and thus removed from here.
>
> With the above changes,
>
> Acked-by: Jerin Jacob <jerinj@marvell.com>
Thank you for the quick response, Jerin.
^ permalink raw reply [relevance 0%]
* Re: [PATCH v6] eal: add build-time option to omit trace
2024-10-06 13:58 2% ` [PATCH v6] " Morten Brørup
@ 2024-10-07 5:45 0% ` Jerin Jacob
2024-10-07 6:07 0% ` Morten Brørup
0 siblings, 1 reply; 200+ results
From: Jerin Jacob @ 2024-10-07 5:45 UTC (permalink / raw)
To: Morten Brørup; +Cc: Jerin Jacob, Sunil Kumar Kori, dev
On Sun, Oct 6, 2024 at 7:44 PM Morten Brørup <mb@smartsharesystems.com> wrote:
>
> Some applications want to omit the trace feature.
> Either to reduce the memory footprint, to reduce the exposed attack
> surface, or for other reasons.
>
> This patch adds an option in rte_config.h to include or omit trace in the
> build. Trace is included by default.
>
> Omitting trace works by omitting all trace points.
> For API and ABI compatibility, the trace feature itself remains.
>
> Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
> ---
> v6:
> Removed test_trace_perf.c changes; they don't compile for Windows
> target, and are superfluous.
> v5:
> Added public function rte_trace_feature_is_enabled(), to test if trace
> is build time enabled in both the DPDK and the application. Use in test
> application instead of private function. (Jerin Jacob)
> v4:
> * Added check for generic trace enabled when registering trace points, in
> RTE_INIT. (Jerin Jacob)
> * Test application uses function instead of macro to check if generic
> trace is enabled. (Jerin Jacob)
> * Performance test application uses function to check if generic trace is
> enabled.
> v3:
> * Simpler version with much fewer ifdefs. (Jerin Jacob)
> v2:
> * Added/modified macros required for building applications with trace
> omitted.
> ---
> app/test/test_trace.c | 4 +++
> config/rte_config.h | 1 +
> lib/eal/common/eal_common_trace.c | 10 +++++++
> lib/eal/include/rte_trace.h | 33 ++++++++++++++++++++++
> lib/eal/include/rte_trace_point.h | 21 ++++++++++++++
> lib/eal/include/rte_trace_point_register.h | 2 ++
> lib/eal/version.map | 3 ++
> 7 files changed, 74 insertions(+)
>
> diff --git a/app/test/test_trace.c b/app/test/test_trace.c
> index 00809f433b..8ea1443044 100644
> --- a/app/test/test_trace.c
> +++ b/app/test/test_trace.c
> @@ -245,6 +245,10 @@ static struct unit_test_suite trace_tests = {
> static int
> test_trace(void)
> {
> + if (!rte_trace_feature_is_enabled()) {
> + printf("Trace omitted at build-time, skipping test\n");
> + return TEST_SKIPPED;
> + }
> return unit_test_suite_runner(&trace_tests);
> }
>
> diff --git a/config/rte_config.h b/config/rte_config.h
> index dd7bb0d35b..fd6f8a2f1a 100644
> --- a/config/rte_config.h
> +++ b/config/rte_config.h
> @@ -49,6 +49,7 @@
> #define RTE_MAX_TAILQ 32
> #define RTE_LOG_DP_LEVEL RTE_LOG_INFO
> #define RTE_MAX_VFIO_CONTAINERS 64
> +#define RTE_TRACE 1
>
> /* bsd module defines */
> #define RTE_CONTIGMEM_MAX_NUM_BUFS 64
> diff --git a/lib/eal/common/eal_common_trace.c b/lib/eal/common/eal_common_trace.c
> index 918f49bf4f..06130c756d 100644
> --- a/lib/eal/common/eal_common_trace.c
> +++ b/lib/eal/common/eal_common_trace.c
> @@ -100,6 +100,16 @@ rte_trace_is_enabled(void)
> return rte_atomic_load_explicit(&trace.status, rte_memory_order_acquire) != 0;
> }
>
> +bool
> +__rte_trace_feature_is_enabled(void)
> +{
> +#ifdef RTE_TRACE
> + return true;
> +#else
> + return false;
> +#endif
> +}
> +static __rte_always_inline
> +bool rte_trace_feature_is_enabled(void)
> +{
> +#ifdef RTE_TRACE
> + return __rte_trace_feature_is_enabled();
> +#else
> + return false;
> +#endif
> +}
> +__rte_experimental
> +static __rte_always_inline bool
> +__rte_trace_point_generic_is_enabled(void)
> +{
> +#ifdef RTE_TRACE
> + return true;
> +#else
> + return false;
> +#endif
__rte_trace_feature_is_enabled(), rte_trace_feature_is_enabled(),
__rte_trace_point_generic_is_enabled() are duplicates.
There is no harm in using a public API inside the implementation.
Please keep only rte_trace_feature_is_enabled()
and use it across implementation and app/test.
> +}
> +
> /**
> * @internal
> *
> rte_vfio_get_device_info; # WINDOWS_NO_EXPORT
> +
> + # added in 24.11
> + __rte_trace_feature_is_enabled;
rte_trace_feature_is_enabled;
With the above changes,
Acked-by: Jerin Jacob <jerinj@marvell.com>
> };
>
> INTERNAL {
> --
> 2.43.0
>
^ permalink raw reply [relevance 0%]
* [PATCH v8 1/6] eventdev: introduce event pre-scheduling
2024-10-06 17:06 3% ` [PATCH v8 0/6] Introduce " pbhagavatula
@ 2024-10-06 17:06 3% ` pbhagavatula
2024-10-07 13:09 3% ` [PATCH v9 0/6] Introduce " pbhagavatula
1 sibling, 0 replies; 200+ results
From: pbhagavatula @ 2024-10-06 17:06 UTC (permalink / raw)
To: jerinj, sthotton, abdullah.sevincer, hemant.agrawal,
sachin.saxena, harry.van.haaren, mattias.ronnblom, liangma,
peter.mccarthy, stephen
Cc: dev, Pavan Nikhilesh
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
Event pre-scheduling improves scheduling performance by assigning events
to event ports in advance when dequeues are issued.
The dequeue operation initiates the pre-schedule operation, which completes
in parallel without affecting the dequeued event flow contexts and
dequeue latency.
Event devices can indicate pre-scheduling capabilities using
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event device info
function `info.event_dev_cap`.
Applications can select the pre-schedule type and configure it through
`rte_event_dev_config.preschedule_type` during `rte_event_dev_configure`.
The supported pre-schedule types are:
* `RTE_EVENT_PRESCHEDULE_NONE` - No pre-scheduling.
* `RTE_EVENT_PRESCHEDULE` - Always issue a pre-schedule on dequeue.
* `RTE_EVENT_PRESCHEDULE_ADAPTIVE` - Delay issuing pre-schedule until
there are no forward progress constraints with the held flow contexts.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
app/test/test_eventdev.c | 108 ++++++++++++++++++++
doc/guides/eventdevs/features/default.ini | 1 +
doc/guides/prog_guide/eventdev/eventdev.rst | 23 +++++
doc/guides/rel_notes/release_24_11.rst | 10 ++
lib/eventdev/rte_eventdev.h | 49 +++++++++
5 files changed, 191 insertions(+)
diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c
index 9a6c8f470c..a45d1396d7 100644
--- a/app/test/test_eventdev.c
+++ b/app/test/test_eventdev.c
@@ -1251,6 +1251,112 @@ test_eventdev_profile_switch(void)
return TEST_SUCCESS;
}
+static int
+preschedule_test(enum rte_event_dev_preschedule_type preschedule_type, const char *preschedule_name)
+{
+#define NB_EVENTS 1024
+ uint64_t start, total;
+ struct rte_event ev;
+ int rc, cnt;
+
+ ev.event_type = RTE_EVENT_TYPE_CPU;
+ ev.queue_id = 0;
+ ev.op = RTE_EVENT_OP_NEW;
+ ev.u64 = 0xBADF00D0;
+
+ for (cnt = 0; cnt < NB_EVENTS; cnt++) {
+ ev.flow_id = cnt;
+ rc = rte_event_enqueue_burst(TEST_DEV_ID, 0, &ev, 1);
+ TEST_ASSERT(rc == 1, "Failed to enqueue event");
+ }
+
+ RTE_SET_USED(preschedule_type);
+ total = 0;
+ while (cnt) {
+ start = rte_rdtsc_precise();
+ rc = rte_event_dequeue_burst(TEST_DEV_ID, 0, &ev, 1, 0);
+ if (rc) {
+ total += rte_rdtsc_precise() - start;
+ cnt--;
+ }
+ }
+ printf("Preschedule type : %s, avg cycles %" PRIu64 "\n", preschedule_name,
+ total / NB_EVENTS);
+
+ return TEST_SUCCESS;
+}
+
+static int
+preschedule_configure(enum rte_event_dev_preschedule_type type, struct rte_event_dev_info *info)
+{
+ struct rte_event_dev_config dev_conf;
+ struct rte_event_queue_conf qcfg;
+ struct rte_event_port_conf pcfg;
+ int rc;
+
+ devconf_set_default_sane_values(&dev_conf, info);
+ dev_conf.nb_event_ports = 1;
+ dev_conf.nb_event_queues = 1;
+ dev_conf.preschedule_type = type;
+
+ rc = rte_event_dev_configure(TEST_DEV_ID, &dev_conf);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+
+ rc = rte_event_port_default_conf_get(TEST_DEV_ID, 0, &pcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to get port0 default config");
+ rc = rte_event_port_setup(TEST_DEV_ID, 0, &pcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to setup port0");
+
+ rc = rte_event_queue_default_conf_get(TEST_DEV_ID, 0, &qcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to get queue0 default config");
+ rc = rte_event_queue_setup(TEST_DEV_ID, 0, &qcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to setup queue0");
+
+ rc = rte_event_port_link(TEST_DEV_ID, 0, NULL, NULL, 0);
+ TEST_ASSERT(rc == (int)dev_conf.nb_event_queues, "Failed to link port, device %d",
+ TEST_DEV_ID);
+
+ rc = rte_event_dev_start(TEST_DEV_ID);
+ TEST_ASSERT_SUCCESS(rc, "Failed to start event device");
+
+ return 0;
+}
+
+static int
+test_eventdev_preschedule_configure(void)
+{
+ struct rte_event_dev_info info;
+ int rc;
+
+ rte_event_dev_info_get(TEST_DEV_ID, &info);
+
+ if ((info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE) == 0)
+ return TEST_SKIPPED;
+
+ rc = preschedule_configure(RTE_EVENT_PRESCHEDULE_NONE, &info);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+ rc = preschedule_test(RTE_EVENT_PRESCHEDULE_NONE, "RTE_EVENT_PRESCHEDULE_NONE");
+ TEST_ASSERT_SUCCESS(rc, "Failed to test preschedule RTE_EVENT_PRESCHEDULE_NONE");
+
+ rte_event_dev_stop(TEST_DEV_ID);
+ rc = preschedule_configure(RTE_EVENT_PRESCHEDULE, &info);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+ rc = preschedule_test(RTE_EVENT_PRESCHEDULE, "RTE_EVENT_PRESCHEDULE");
+ TEST_ASSERT_SUCCESS(rc, "Failed to test preschedule RTE_EVENT_PRESCHEDULE");
+
+ if (info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE) {
+ rte_event_dev_stop(TEST_DEV_ID);
+ rc = preschedule_configure(RTE_EVENT_PRESCHEDULE_ADAPTIVE, &info);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+ rc = preschedule_test(RTE_EVENT_PRESCHEDULE_ADAPTIVE,
+ "RTE_EVENT_PRESCHEDULE_ADAPTIVE");
+ TEST_ASSERT_SUCCESS(rc,
+ "Failed to test preschedule RTE_EVENT_PRESCHEDULE_ADAPTIVE");
+ }
+
+ return TEST_SUCCESS;
+}
+
static int
test_eventdev_close(void)
{
@@ -1311,6 +1417,8 @@ static struct unit_test_suite eventdev_common_testsuite = {
test_eventdev_start_stop),
TEST_CASE_ST(eventdev_configure_setup, eventdev_stop_device,
test_eventdev_profile_switch),
+ TEST_CASE_ST(eventdev_configure_setup, NULL,
+ test_eventdev_preschedule_configure),
TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
test_eventdev_link),
TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
diff --git a/doc/guides/eventdevs/features/default.ini b/doc/guides/eventdevs/features/default.ini
index 1cc4303fe5..c8d5ed2d74 100644
--- a/doc/guides/eventdevs/features/default.ini
+++ b/doc/guides/eventdevs/features/default.ini
@@ -22,6 +22,7 @@ carry_flow_id =
maintenance_free =
runtime_queue_attr =
profile_links =
+preschedule =
;
; Features of a default Ethernet Rx adapter.
diff --git a/doc/guides/prog_guide/eventdev/eventdev.rst b/doc/guides/prog_guide/eventdev/eventdev.rst
index fb6dfce102..9da5531859 100644
--- a/doc/guides/prog_guide/eventdev/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev/eventdev.rst
@@ -357,6 +357,29 @@ Worker path:
// Process the event received.
}
+Event Pre-scheduling
+~~~~~~~~~~~~~~~~~~~~
+
+Event pre-scheduling improves scheduling performance by assigning events to event ports in advance
+when dequeues are issued.
+The `rte_event_dequeue_burst` operation initiates the pre-schedule operation, which completes
+in parallel without affecting the dequeued event flow contexts and dequeue latency.
+On the next dequeue operation, the pre-scheduled events are dequeued and pre-schedule is initiated
+again.
+
+An application can use event pre-scheduling if the event device supports it at either device
+level or at a individual port level.
+The application must check pre-schedule capability by checking if ``rte_event_dev_info.event_dev_cap``
+has the bit ``RTE_EVENT_DEV_CAP_PRESCHEDULE`` or ``RTE_EVENT_DEV_CAP_PRESCHEDULE_ADAPTIVE`` set, if
+present pre-scheduling can be enabled at device
+configuration time by setting appropriate pre-schedule type in ``rte_event_dev_config.preschedule``.
+
+The following pre-schedule types are supported:
+ * ``RTE_EVENT_PRESCHEDULE_NONE`` - No pre-scheduling.
+ * ``RTE_EVENT_PRESCHEDULE`` - Always issue a pre-schedule when dequeue is issued.
+ * ``RTE_EVENT_PRESCHEDULE_ADAPTIVE`` - Issue pre-schedule when dequeue is issued and there are
+ no forward progress constraints.
+
Starting the EventDev
~~~~~~~~~~~~~~~~~~~~~
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index e0a9aa55a1..a294a753e7 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -67,6 +67,14 @@ New Features
The new statistics are useful for debugging and profiling.
+* **Added event device pre-scheduling support.**
+
+ Added support for pre-scheduling of events to event ports to improve
+ scheduling performance and latency.
+
+ * Added ``rte_event_dev_config::preschedule_type`` to configure the device
+ level pre-scheduling type.
+
Removed Items
-------------
@@ -112,6 +120,8 @@ ABI Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* eventdev: Added ``preschedule_type`` field to ``rte_event_dev_config``
+ structure.
Known Issues
------------
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 08e5f9320b..4b69e74577 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -446,6 +446,30 @@ struct rte_event;
* @see RTE_SCHED_TYPE_PARALLEL
*/
+#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE (1ULL << 16)
+/**< Event device supports event pre-scheduling.
+ *
+ * When this capability is available, the application can enable event pre-scheduling on the event
+ * device to pre-schedule events to a event port when `rte_event_dequeue_burst()`
+ * is issued.
+ * The pre-schedule process starts with the `rte_event_dequeue_burst()` call and the
+ * pre-scheduled events are returned on the next `rte_event_dequeue_burst()` call.
+ *
+ * @see rte_event_dev_configure()
+ */
+
+#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE (1ULL << 17)
+/**< Event device supports adaptive event pre-scheduling.
+ *
+ * When this capability is available, the application can enable adaptive pre-scheduling
+ * on the event device where the events are pre-scheduled when there are no forward
+ * progress constraints with the currently held flow contexts.
+ * The pre-schedule process starts with the `rte_event_dequeue_burst()` call and the
+ * pre-scheduled events are returned on the next `rte_event_dequeue_burst()` call.
+ *
+ * @see rte_event_dev_configure()
+ */
+
/* Event device priority levels */
#define RTE_EVENT_DEV_PRIORITY_HIGHEST 0
/**< Highest priority level for events and queues.
@@ -680,6 +704,26 @@ rte_event_dev_attr_get(uint8_t dev_id, uint32_t attr_id,
* @see rte_event_dequeue_timeout_ticks(), rte_event_dequeue_burst()
*/
+/** Event device pre-schedule type enumeration. */
+enum rte_event_dev_preschedule_type {
+ RTE_EVENT_PRESCHEDULE_NONE,
+ /**< Disable pre-schedule across the event device or on a given event port.
+ * @ref rte_event_dev_config.preschedule_type
+ */
+ RTE_EVENT_PRESCHEDULE,
+ /**< Enable pre-schedule always across the event device or a given event port.
+ * @ref rte_event_dev_config.preschedule_type
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
+ */
+ RTE_EVENT_PRESCHEDULE_ADAPTIVE,
+ /**< Enable adaptive pre-schedule across the event device or a given event port.
+ * Delay issuing pre-schedule until there are no forward progress constraints with
+ * the held flow contexts.
+ * @ref rte_event_dev_config.preschedule_type
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
+ */
+};
+
/** Event device configuration structure */
struct rte_event_dev_config {
uint32_t dequeue_timeout_ns;
@@ -752,6 +796,11 @@ struct rte_event_dev_config {
* optimized for single-link usage, this field is a hint for how many
* to allocate; otherwise, regular event ports and queues will be used.
*/
+ enum rte_event_dev_preschedule_type preschedule_type;
+ /**< Event pre-schedule type to use across the event device, if supported.
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
+ */
};
/**
--
2.25.1
^ permalink raw reply [relevance 3%]
* [PATCH v8 0/6] Introduce event pre-scheduling
2024-10-05 7:59 3% ` [PATCH v7 0/6] Introduce " pbhagavatula
2024-10-05 7:59 3% ` [PATCH v7 1/6] eventdev: introduce " pbhagavatula
@ 2024-10-06 17:06 3% ` pbhagavatula
2024-10-06 17:06 3% ` [PATCH v8 1/6] eventdev: introduce " pbhagavatula
2024-10-07 13:09 3% ` [PATCH v9 0/6] Introduce " pbhagavatula
1 sibling, 2 replies; 200+ results
From: pbhagavatula @ 2024-10-06 17:06 UTC (permalink / raw)
To: jerinj, sthotton, abdullah.sevincer, hemant.agrawal,
sachin.saxena, harry.van.haaren, mattias.ronnblom, liangma,
peter.mccarthy, stephen
Cc: dev, Pavan Nikhilesh
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
Event pre-scheduling improves scheduling performance by assigning events
to event ports in advance when dequeues are issued.
This series introduces various types and levels of pre-scheduling to the
eventdev library.
pre-scheduling Types:
* RTE_EVENT_PRESCHEDULE_NONE: No pre-scheduling.
* RTE_EVENT_PRESCHEDULE: Always issue a pre-schedule when dequeue is issued.
* RTE_EVENT_PRESCHEDULE_ADAPTIVE: Delay issuing pre-schedule until there
are no forward progress constraints with the held flow contexts.
pre-scheduling Levels:
* Event Device Level Pre-scheduling: Pre-scheduling can be enabled or disabled at the
event device during configuration. Event devices can indicate pre-scheduling
capabilities using `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event device info
function `info.event_dev_cap`.
* Event Port Level Prefetch: Pre-scheduling can be selectively enabled or disabled
at the event port during runtime. Event devices can indicate this capability
using `RTE_EVENT_PORT_CAP_PER_PORT_PRESCHEDULE` via the event device info
function `info.event_port_cap`.
* Application Controlled Prefetch Hint: Applications can provide hints to the
event device to start pre-scheduling events using the new API
`rte_event_port_pre-schedule`. Event devices can indicate this capabilities using
`RTE_EVENT_DEV_CAP_PRESCHEDULE_EXPLICIT` via the event device info function
`info.event_dev_cap`.
The rte_event_dequeue_burst operation initiates the pre-schedule operation, which
completes in parallel without affecting the flow context of the dequeued events and
dequeue latency. On the next dequeue operation, the pre-scheduleed events are
dequeued, and pre-schedule operation is initiated again.
In the case of application-controlled pre-schedule hints, the currently held flow
contexts, if any, are not affected by the pre-schedule operation. On the next
dequeue operation, the pre-scheduleed events are returned, but pre-schedule is not
initiated again until the application provides the hint again. If pre-scheduling
is already enabled at the event device level or event port level, the hint is ignored.
v2 Changes:
- s/prefetch/pre-schedule (Mattias).
v3 Changes:
- Add CNXK preschedule implementation.
- Update test-eventdev to use prescheduling.
- Update examples to use prescheduling.
v4 Changes:
- Fix compilation.
v5 Changes:
- Update ABI changes
- s/RTE_EVENT_DEV_PRESCHEDULE/RTE_EVENT_PRESCHEDULE/
- s/RTE_EVENT_DEV_CAP_SW_PRESCHEDULE/RTE_EVENT_DEV_CAP_PRESCHEDULE_EXPLICIT/
- s/RTE_EVENT_PORT_CAP_EVENT_PER_PORT_PRESCHEDULE/RTE_EVENT_PORT_CAP_PER_PORT_PRESCHEDULE
- Various documentation fixes and updates.
v6 Changes:
- Mark new APIs as experimental (Stephen).
v7 Changes:
- Use correct patchset for rebase.
v8 Changes:
- fix checkpatch issues.
Pavan Nikhilesh (6):
eventdev: introduce event pre-scheduling
eventdev: add event port pre-schedule modify
eventdev: add event preschedule hint
event/cnkx: add pre-schedule support
app/test-eventdev: add pre-scheduling support
examples: use eventdev pre-scheduling
app/test-eventdev/evt_common.h | 45 +++--
app/test-eventdev/evt_options.c | 17 ++
app/test-eventdev/evt_options.h | 1 +
app/test/test_eventdev.c | 143 ++++++++++++++++
doc/guides/eventdevs/cnxk.rst | 10 --
doc/guides/eventdevs/features/cnxk.ini | 1 +
doc/guides/eventdevs/features/default.ini | 1 +
doc/guides/prog_guide/eventdev/eventdev.rst | 48 ++++++
doc/guides/rel_notes/release_24_11.rst | 13 ++
doc/guides/tools/testeventdev.rst | 6 +
drivers/event/cnxk/cn10k_eventdev.c | 19 ++-
drivers/event/cnxk/cn10k_worker.c | 21 +++
drivers/event/cnxk/cn10k_worker.h | 2 +
drivers/event/cnxk/cnxk_eventdev.c | 2 -
drivers/event/cnxk/cnxk_eventdev.h | 1 -
.../pipeline_worker_generic.c | 6 +
.../eventdev_pipeline/pipeline_worker_tx.c | 6 +
examples/ipsec-secgw/event_helper.c | 6 +
examples/l2fwd-event/l2fwd_event_generic.c | 6 +
.../l2fwd-event/l2fwd_event_internal_port.c | 6 +
examples/l3fwd/l3fwd_event_generic.c | 6 +
examples/l3fwd/l3fwd_event_internal_port.c | 6 +
lib/eventdev/eventdev_pmd.h | 4 +
lib/eventdev/eventdev_private.c | 41 ++++-
lib/eventdev/eventdev_trace_points.c | 6 +
lib/eventdev/rte_eventdev.h | 161 ++++++++++++++++++
lib/eventdev/rte_eventdev_core.h | 14 +-
lib/eventdev/rte_eventdev_trace_fp.h | 19 ++-
lib/eventdev/version.map | 6 +
29 files changed, 594 insertions(+), 29 deletions(-)
--
2.25.1
^ permalink raw reply [relevance 3%]
* Re: [RFC v2 0/2] Add high-performance timer facility
2024-10-06 13:43 0% ` Morten Brørup
@ 2024-10-06 14:43 0% ` Mattias Rönnblom
0 siblings, 0 replies; 200+ results
From: Mattias Rönnblom @ 2024-10-06 14:43 UTC (permalink / raw)
To: Morten Brørup, Stephen Hemminger, Mattias Rönnblom
Cc: dev, Erik Gabriel Carrillo, David Marchand, maria.lingemark,
Stefan Sundkvist, Tyler Retzlaff
On 2024-10-06 15:43, Morten Brørup wrote:
>> From: Mattias Rönnblom [mailto:hofors@lysator.liu.se]
>> Sent: Sunday, 6 October 2024 15.03
>>
>> On 2024-10-03 23:32, Morten Brørup wrote:
>>>> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
>>>> Sent: Thursday, 3 October 2024 20.37
>>>>
>>>> On Wed, 15 Mar 2023 18:03:40 +0100
>>>> Mattias Rönnblom <mattias.ronnblom@ericsson.com> wrote:
>>>>
>>>>> This patchset is an attempt to introduce a high-performance, highly
>>>>> scalable timer facility into DPDK.
>>>>>
>>>>> More specifically, the goals for the htimer library are:
>>>>>
>>>>> * Efficient handling of a handful up to hundreds of thousands of
>>>>> concurrent timers.
>>>>> * Make adding and canceling timers low-overhead, constant-time
>>>>> operations.
>>>>> * Provide a service functionally equivalent to that of
>>>>> <rte_timer.h>. API/ABI backward compatibility is secondary.
>>>>
>>>> Worthwhile goals, and the problem needs to be addressed.
>>>> But this patch never got accepted.
>>>
>>> I think work on it was put on hold due to the requested changes
>> requiring a significant development effort.
>>> I too look forward to work on this being resumed. ;-)
>>>
>>>>
>>>> Please fix/improve/extend existing rte_timer instead.
>>>
>>> The rte_timer API is too "fat" for use in the fast path with millions
>> of timers, e.g. TCP flow timers.
>>>
>>> Shoehorning a fast path feature into a slow path API is not going to
>> cut it. I support having a separate htimer library with its own API for
>> high volume, high-performance fast path timers.
>>>
>>> When striving for low latency across the internet, timing is
>> everything. Packet pacing is the "new" hot thing in congestion control
>> algorithms, and a simple software implementation would require a timer
>> firing once per packet.
>>>
>>
>> I think DPDK should have two public APIs in the timer area.
>
> Agree.
>
>> One is a
>> just a bare-bones hierarchical timer wheel API, without callbacks,
>> auto-created per-lcore instances, MT safety or any other of the
>> <rte_timer.h> bells and whistles. It also doesn't make any assumptions
>> about the time source (other it being monotonic) or resolution.
>
> The <rte_timer.h> library does not - and is never going to - provide sufficient performance for timer intensive applications, such as packet pacing and fast path TCP/QUIC/whatever congestion control. It is too "fat" for this.
>
> We need a new library with a new API for that.
> I agree with Mattias' description of the requirements for such a library.
>
>>
>> The other is a new variant of <rte_timer.h>, using the core HTW library
>> for its implementation (and being public, it may also expose this
>> library in its header files, which may be required for efficient
>> operation). The new <rte_timer.h> would provide the same kind of
>> functionality as the old API, but with some quirks and bugs fixed, plus
>> potentially some new functionality added. For example, it would be
>> useful to allow non-preemption safe threads to add and remove timers
>> (something rte_timer and its spinlocks doesn't allow).
>
> Agree.
>
> Until that becomes part of DPDK, we will have to stick with what <rte_timer.h> currently offers.
>
>>
>> I would consider both "fast path APIs".
>>
>> In addition, there should probably also be a time source API.
>
> A third library, orthogonal to the two other timer libraries.
> But I see why you mention it: It could be somewhat related to the design and implementation of the <rte_timer.h> library.
> But, let's please forget about a time source API for now.
>
>>
>> Considering the lead time of relatively small contributions like the
>> bitops extensions and the new bitset API (which still aren't in), I
>> can't imagine how long time it would take to get in a semi-backward
>> compatible rte_timer with a new implementation, plus a new timer wheel
>> library, into DPDK.
>
> Well said!
>
> Instead of aiming for an unreachable target, let's instead take this approach:
> - Provide the new high-performance HTW library as a stand-alone library.
> - Postpone improving the <rte_timer.h> library; it can be done any time in the future, if someone cares to do it. And it can use the HTW library or not, whichever is appropriate.
>
> Doing both simultaneously would require a substantial effort, and would cause much backpressure from the community (due to the modified <rte_timer.h> API and implementation).
>
> Although it might be beneficial for the design of the HTW library to consider how an improved <rte_timer.h> would use it, it is not the primary use case of the HTW library, so co-design is not a requirement here.
>
Postponing rte_timer improvements would also mean postponing most of the
benefits of the new timer wheel, in my opinion.
In most scenarios, I think you want to have all application modules
sharing timer wheel instances, preferably without having to agree on a
proprietary timer API. Here rte_timer shines.
Also, you want to get the HTW library *exactly* right for the rte_timer
use case. Making it a public API would make changes to its API painful,
to address any shortcomings you accidentally designed in. To be on the
safe side, you would need to have a new rte_timer implementation ready
upon submitting a HTW library.
That in turn would require a techboard ACK on the necessity of rte_timer
API tweaks, otherwise all your work may be wasted.
^ permalink raw reply [relevance 0%]
* RE: [PATCH v7] eal: add build-time option to omit trace
2024-10-06 14:03 2% ` [PATCH v7] " Morten Brørup
@ 2024-10-06 14:09 0% ` Morten Brørup
0 siblings, 0 replies; 200+ results
From: Morten Brørup @ 2024-10-06 14:09 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger
> From: Morten Brørup [mailto:mb@smartsharesystems.com]
> Sent: Sunday, 6 October 2024 16.04
>
> Some applications want to omit the trace feature.
> Either to reduce the memory footprint, to reduce the exposed attack
> surface, or for other reasons.
>
> This patch adds an option in rte_config.h to include or omit trace in
> the
> build. Trace is included by default.
>
> Omitting trace works by omitting all trace points.
> For API and ABI compatibility, the trace feature itself remains.
>
> Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
Forgot to carry over Stephen's Ack to v4, so here it comes for the benefit of Patchwork...
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
^ permalink raw reply [relevance 0%]
* [PATCH v7] eal: add build-time option to omit trace
2024-09-18 8:55 3% [PATCH] eal: add build-time option to omit trace Morten Brørup
` (5 preceding siblings ...)
2024-10-06 13:58 2% ` [PATCH v6] " Morten Brørup
@ 2024-10-06 14:03 2% ` Morten Brørup
2024-10-06 14:09 0% ` Morten Brørup
2024-10-07 11:46 3% ` [PATCH v9] " Morten Brørup
7 siblings, 1 reply; 200+ results
From: Morten Brørup @ 2024-10-06 14:03 UTC (permalink / raw)
To: Jerin Jacob, Sunil Kumar Kori; +Cc: dev, Morten Brørup
Some applications want to omit the trace feature.
Either to reduce the memory footprint, to reduce the exposed attack
surface, or for other reasons.
This patch adds an option in rte_config.h to include or omit trace in the
build. Trace is included by default.
Omitting trace works by omitting all trace points.
For API and ABI compatibility, the trace feature itself remains.
Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
---
v7:
Updated version.map to not export __rte_trace_feature_is_enabled for
Windows target.
v6:
Removed test_trace_perf.c changes; they don't compile for Windows
target, and are superfluous.
v5:
Added public function rte_trace_feature_is_enabled(), to test if trace
is build time enabled in both the DPDK and the application. Use in test
application instead of private function. (Jerin Jacob)
v4:
* Added check for generic trace enabled when registering trace points, in
RTE_INIT. (Jerin Jacob)
* Test application uses function instead of macro to check if generic
trace is enabled. (Jerin Jacob)
* Performance test application uses function to check if generic trace is
enabled.
v3:
* Simpler version with much fewer ifdefs. (Jerin Jacob)
v2:
* Added/modified macros required for building applications with trace
omitted.
---
app/test/test_trace.c | 4 +++
config/rte_config.h | 1 +
lib/eal/common/eal_common_trace.c | 10 +++++++
lib/eal/include/rte_trace.h | 33 ++++++++++++++++++++++
lib/eal/include/rte_trace_point.h | 21 ++++++++++++++
lib/eal/include/rte_trace_point_register.h | 2 ++
lib/eal/version.map | 3 ++
7 files changed, 74 insertions(+)
diff --git a/app/test/test_trace.c b/app/test/test_trace.c
index 00809f433b..8ea1443044 100644
--- a/app/test/test_trace.c
+++ b/app/test/test_trace.c
@@ -245,6 +245,10 @@ static struct unit_test_suite trace_tests = {
static int
test_trace(void)
{
+ if (!rte_trace_feature_is_enabled()) {
+ printf("Trace omitted at build-time, skipping test\n");
+ return TEST_SKIPPED;
+ }
return unit_test_suite_runner(&trace_tests);
}
diff --git a/config/rte_config.h b/config/rte_config.h
index dd7bb0d35b..fd6f8a2f1a 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -49,6 +49,7 @@
#define RTE_MAX_TAILQ 32
#define RTE_LOG_DP_LEVEL RTE_LOG_INFO
#define RTE_MAX_VFIO_CONTAINERS 64
+#define RTE_TRACE 1
/* bsd module defines */
#define RTE_CONTIGMEM_MAX_NUM_BUFS 64
diff --git a/lib/eal/common/eal_common_trace.c b/lib/eal/common/eal_common_trace.c
index 918f49bf4f..06130c756d 100644
--- a/lib/eal/common/eal_common_trace.c
+++ b/lib/eal/common/eal_common_trace.c
@@ -100,6 +100,16 @@ rte_trace_is_enabled(void)
return rte_atomic_load_explicit(&trace.status, rte_memory_order_acquire) != 0;
}
+bool
+__rte_trace_feature_is_enabled(void)
+{
+#ifdef RTE_TRACE
+ return true;
+#else
+ return false;
+#endif
+}
+
static void
trace_mode_set(rte_trace_point_t *t, enum rte_trace_mode mode)
{
diff --git a/lib/eal/include/rte_trace.h b/lib/eal/include/rte_trace.h
index a6e991fad3..340df4f8a0 100644
--- a/lib/eal/include/rte_trace.h
+++ b/lib/eal/include/rte_trace.h
@@ -35,6 +35,39 @@ extern "C" {
__rte_experimental
bool rte_trace_is_enabled(void);
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * @internal
+ *
+ * Test if trace feature is enabled at compile time.
+ *
+ * @return
+ * true if trace feature is enabled, false otherwise.
+ */
+__rte_experimental
+bool __rte_trace_feature_is_enabled(void);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * Test if trace feature is enabled at compile time.
+ *
+ * @return
+ * true if trace feature is enabled, false otherwise.
+ */
+static __rte_always_inline
+bool rte_trace_feature_is_enabled(void)
+{
+#ifdef RTE_TRACE
+ return __rte_trace_feature_is_enabled();
+#else
+ return false;
+#endif
+}
+
/**
* Enumerate trace mode operation.
*/
diff --git a/lib/eal/include/rte_trace_point.h b/lib/eal/include/rte_trace_point.h
index 41e2a7f99e..b80688ce89 100644
--- a/lib/eal/include/rte_trace_point.h
+++ b/lib/eal/include/rte_trace_point.h
@@ -212,6 +212,25 @@ bool rte_trace_point_is_enabled(rte_trace_point_t *tp);
__rte_experimental
rte_trace_point_t *rte_trace_point_lookup(const char *name);
+/**
+ * @internal
+ *
+ * Test if the tracepoint compile-time option is enabled.
+ *
+ * @return
+ * true if tracepoint enabled, false otherwise.
+ */
+__rte_experimental
+static __rte_always_inline bool
+__rte_trace_point_generic_is_enabled(void)
+{
+#ifdef RTE_TRACE
+ return true;
+#else
+ return false;
+#endif
+}
+
/**
* @internal
*
@@ -359,6 +378,8 @@ __rte_trace_point_emit_ev_header(void *mem, uint64_t in)
#define __rte_trace_point_emit_header_generic(t) \
void *mem; \
do { \
+ if (!__rte_trace_point_generic_is_enabled()) \
+ return; \
const uint64_t val = rte_atomic_load_explicit(t, rte_memory_order_acquire); \
if (likely(!(val & __RTE_TRACE_FIELD_ENABLE_MASK))) \
return; \
diff --git a/lib/eal/include/rte_trace_point_register.h b/lib/eal/include/rte_trace_point_register.h
index 41260e5964..429b993fc2 100644
--- a/lib/eal/include/rte_trace_point_register.h
+++ b/lib/eal/include/rte_trace_point_register.h
@@ -23,6 +23,8 @@ rte_trace_point_t __rte_section("__rte_trace_point") __##trace; \
static const char __##trace##_name[] = RTE_STR(name); \
RTE_INIT(trace##_init) \
{ \
+ if (!__rte_trace_point_generic_is_enabled()) \
+ return; \
__rte_trace_point_register(&__##trace, __##trace##_name, \
(void (*)(void)) trace); \
}
diff --git a/lib/eal/version.map b/lib/eal/version.map
index e3ff412683..6fd3dba85e 100644
--- a/lib/eal/version.map
+++ b/lib/eal/version.map
@@ -396,6 +396,9 @@ EXPERIMENTAL {
# added in 24.03
rte_vfio_get_device_info; # WINDOWS_NO_EXPORT
+
+ # added in 24.11
+ __rte_trace_feature_is_enabled; # WINDOWS_NO_EXPORT
};
INTERNAL {
--
2.43.0
^ permalink raw reply [relevance 2%]
* [PATCH v6] eal: add build-time option to omit trace
2024-09-18 8:55 3% [PATCH] eal: add build-time option to omit trace Morten Brørup
` (4 preceding siblings ...)
2024-10-06 12:38 2% ` [PATCH v5] " Morten Brørup
@ 2024-10-06 13:58 2% ` Morten Brørup
2024-10-07 5:45 0% ` Jerin Jacob
2024-10-06 14:03 2% ` [PATCH v7] " Morten Brørup
2024-10-07 11:46 3% ` [PATCH v9] " Morten Brørup
7 siblings, 1 reply; 200+ results
From: Morten Brørup @ 2024-10-06 13:58 UTC (permalink / raw)
To: Jerin Jacob, Sunil Kumar Kori; +Cc: dev, Morten Brørup
Some applications want to omit the trace feature.
Either to reduce the memory footprint, to reduce the exposed attack
surface, or for other reasons.
This patch adds an option in rte_config.h to include or omit trace in the
build. Trace is included by default.
Omitting trace works by omitting all trace points.
For API and ABI compatibility, the trace feature itself remains.
Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
---
v6:
Removed test_trace_perf.c changes; they don't compile for Windows
target, and are superfluous.
v5:
Added public function rte_trace_feature_is_enabled(), to test if trace
is build time enabled in both the DPDK and the application. Use in test
application instead of private function. (Jerin Jacob)
v4:
* Added check for generic trace enabled when registering trace points, in
RTE_INIT. (Jerin Jacob)
* Test application uses function instead of macro to check if generic
trace is enabled. (Jerin Jacob)
* Performance test application uses function to check if generic trace is
enabled.
v3:
* Simpler version with much fewer ifdefs. (Jerin Jacob)
v2:
* Added/modified macros required for building applications with trace
omitted.
---
app/test/test_trace.c | 4 +++
config/rte_config.h | 1 +
lib/eal/common/eal_common_trace.c | 10 +++++++
lib/eal/include/rte_trace.h | 33 ++++++++++++++++++++++
lib/eal/include/rte_trace_point.h | 21 ++++++++++++++
lib/eal/include/rte_trace_point_register.h | 2 ++
lib/eal/version.map | 3 ++
7 files changed, 74 insertions(+)
diff --git a/app/test/test_trace.c b/app/test/test_trace.c
index 00809f433b..8ea1443044 100644
--- a/app/test/test_trace.c
+++ b/app/test/test_trace.c
@@ -245,6 +245,10 @@ static struct unit_test_suite trace_tests = {
static int
test_trace(void)
{
+ if (!rte_trace_feature_is_enabled()) {
+ printf("Trace omitted at build-time, skipping test\n");
+ return TEST_SKIPPED;
+ }
return unit_test_suite_runner(&trace_tests);
}
diff --git a/config/rte_config.h b/config/rte_config.h
index dd7bb0d35b..fd6f8a2f1a 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -49,6 +49,7 @@
#define RTE_MAX_TAILQ 32
#define RTE_LOG_DP_LEVEL RTE_LOG_INFO
#define RTE_MAX_VFIO_CONTAINERS 64
+#define RTE_TRACE 1
/* bsd module defines */
#define RTE_CONTIGMEM_MAX_NUM_BUFS 64
diff --git a/lib/eal/common/eal_common_trace.c b/lib/eal/common/eal_common_trace.c
index 918f49bf4f..06130c756d 100644
--- a/lib/eal/common/eal_common_trace.c
+++ b/lib/eal/common/eal_common_trace.c
@@ -100,6 +100,16 @@ rte_trace_is_enabled(void)
return rte_atomic_load_explicit(&trace.status, rte_memory_order_acquire) != 0;
}
+bool
+__rte_trace_feature_is_enabled(void)
+{
+#ifdef RTE_TRACE
+ return true;
+#else
+ return false;
+#endif
+}
+
static void
trace_mode_set(rte_trace_point_t *t, enum rte_trace_mode mode)
{
diff --git a/lib/eal/include/rte_trace.h b/lib/eal/include/rte_trace.h
index a6e991fad3..340df4f8a0 100644
--- a/lib/eal/include/rte_trace.h
+++ b/lib/eal/include/rte_trace.h
@@ -35,6 +35,39 @@ extern "C" {
__rte_experimental
bool rte_trace_is_enabled(void);
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * @internal
+ *
+ * Test if trace feature is enabled at compile time.
+ *
+ * @return
+ * true if trace feature is enabled, false otherwise.
+ */
+__rte_experimental
+bool __rte_trace_feature_is_enabled(void);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * Test if trace feature is enabled at compile time.
+ *
+ * @return
+ * true if trace feature is enabled, false otherwise.
+ */
+static __rte_always_inline
+bool rte_trace_feature_is_enabled(void)
+{
+#ifdef RTE_TRACE
+ return __rte_trace_feature_is_enabled();
+#else
+ return false;
+#endif
+}
+
/**
* Enumerate trace mode operation.
*/
diff --git a/lib/eal/include/rte_trace_point.h b/lib/eal/include/rte_trace_point.h
index 41e2a7f99e..b80688ce89 100644
--- a/lib/eal/include/rte_trace_point.h
+++ b/lib/eal/include/rte_trace_point.h
@@ -212,6 +212,25 @@ bool rte_trace_point_is_enabled(rte_trace_point_t *tp);
__rte_experimental
rte_trace_point_t *rte_trace_point_lookup(const char *name);
+/**
+ * @internal
+ *
+ * Test if the tracepoint compile-time option is enabled.
+ *
+ * @return
+ * true if tracepoint enabled, false otherwise.
+ */
+__rte_experimental
+static __rte_always_inline bool
+__rte_trace_point_generic_is_enabled(void)
+{
+#ifdef RTE_TRACE
+ return true;
+#else
+ return false;
+#endif
+}
+
/**
* @internal
*
@@ -359,6 +378,8 @@ __rte_trace_point_emit_ev_header(void *mem, uint64_t in)
#define __rte_trace_point_emit_header_generic(t) \
void *mem; \
do { \
+ if (!__rte_trace_point_generic_is_enabled()) \
+ return; \
const uint64_t val = rte_atomic_load_explicit(t, rte_memory_order_acquire); \
if (likely(!(val & __RTE_TRACE_FIELD_ENABLE_MASK))) \
return; \
diff --git a/lib/eal/include/rte_trace_point_register.h b/lib/eal/include/rte_trace_point_register.h
index 41260e5964..429b993fc2 100644
--- a/lib/eal/include/rte_trace_point_register.h
+++ b/lib/eal/include/rte_trace_point_register.h
@@ -23,6 +23,8 @@ rte_trace_point_t __rte_section("__rte_trace_point") __##trace; \
static const char __##trace##_name[] = RTE_STR(name); \
RTE_INIT(trace##_init) \
{ \
+ if (!__rte_trace_point_generic_is_enabled()) \
+ return; \
__rte_trace_point_register(&__##trace, __##trace##_name, \
(void (*)(void)) trace); \
}
diff --git a/lib/eal/version.map b/lib/eal/version.map
index e3ff412683..15c694c2da 100644
--- a/lib/eal/version.map
+++ b/lib/eal/version.map
@@ -396,6 +396,9 @@ EXPERIMENTAL {
# added in 24.03
rte_vfio_get_device_info; # WINDOWS_NO_EXPORT
+
+ # added in 24.11
+ __rte_trace_feature_is_enabled;
};
INTERNAL {
--
2.43.0
^ permalink raw reply [relevance 2%]
* RE: [RFC v2 0/2] Add high-performance timer facility
2024-10-06 13:02 0% ` Mattias Rönnblom
@ 2024-10-06 13:43 0% ` Morten Brørup
2024-10-06 14:43 0% ` Mattias Rönnblom
0 siblings, 1 reply; 200+ results
From: Morten Brørup @ 2024-10-06 13:43 UTC (permalink / raw)
To: Mattias Rönnblom, Stephen Hemminger, Mattias Rönnblom
Cc: dev, Erik Gabriel Carrillo, David Marchand, maria.lingemark,
Stefan Sundkvist, Tyler Retzlaff
> From: Mattias Rönnblom [mailto:hofors@lysator.liu.se]
> Sent: Sunday, 6 October 2024 15.03
>
> On 2024-10-03 23:32, Morten Brørup wrote:
> >> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> >> Sent: Thursday, 3 October 2024 20.37
> >>
> >> On Wed, 15 Mar 2023 18:03:40 +0100
> >> Mattias Rönnblom <mattias.ronnblom@ericsson.com> wrote:
> >>
> >>> This patchset is an attempt to introduce a high-performance, highly
> >>> scalable timer facility into DPDK.
> >>>
> >>> More specifically, the goals for the htimer library are:
> >>>
> >>> * Efficient handling of a handful up to hundreds of thousands of
> >>> concurrent timers.
> >>> * Make adding and canceling timers low-overhead, constant-time
> >>> operations.
> >>> * Provide a service functionally equivalent to that of
> >>> <rte_timer.h>. API/ABI backward compatibility is secondary.
> >>
> >> Worthwhile goals, and the problem needs to be addressed.
> >> But this patch never got accepted.
> >
> > I think work on it was put on hold due to the requested changes
> requiring a significant development effort.
> > I too look forward to work on this being resumed. ;-)
> >
> >>
> >> Please fix/improve/extend existing rte_timer instead.
> >
> > The rte_timer API is too "fat" for use in the fast path with millions
> of timers, e.g. TCP flow timers.
> >
> > Shoehorning a fast path feature into a slow path API is not going to
> cut it. I support having a separate htimer library with its own API for
> high volume, high-performance fast path timers.
> >
> > When striving for low latency across the internet, timing is
> everything. Packet pacing is the "new" hot thing in congestion control
> algorithms, and a simple software implementation would require a timer
> firing once per packet.
> >
>
> I think DPDK should have two public APIs in the timer area.
Agree.
> One is a
> just a bare-bones hierarchical timer wheel API, without callbacks,
> auto-created per-lcore instances, MT safety or any other of the
> <rte_timer.h> bells and whistles. It also doesn't make any assumptions
> about the time source (other it being monotonic) or resolution.
The <rte_timer.h> library does not - and is never going to - provide sufficient performance for timer intensive applications, such as packet pacing and fast path TCP/QUIC/whatever congestion control. It is too "fat" for this.
We need a new library with a new API for that.
I agree with Mattias' description of the requirements for such a library.
>
> The other is a new variant of <rte_timer.h>, using the core HTW library
> for its implementation (and being public, it may also expose this
> library in its header files, which may be required for efficient
> operation). The new <rte_timer.h> would provide the same kind of
> functionality as the old API, but with some quirks and bugs fixed, plus
> potentially some new functionality added. For example, it would be
> useful to allow non-preemption safe threads to add and remove timers
> (something rte_timer and its spinlocks doesn't allow).
Agree.
Until that becomes part of DPDK, we will have to stick with what <rte_timer.h> currently offers.
>
> I would consider both "fast path APIs".
>
> In addition, there should probably also be a time source API.
A third library, orthogonal to the two other timer libraries.
But I see why you mention it: It could be somewhat related to the design and implementation of the <rte_timer.h> library.
But, let's please forget about a time source API for now.
>
> Considering the lead time of relatively small contributions like the
> bitops extensions and the new bitset API (which still aren't in), I
> can't imagine how long time it would take to get in a semi-backward
> compatible rte_timer with a new implementation, plus a new timer wheel
> library, into DPDK.
Well said!
Instead of aiming for an unreachable target, let's instead take this approach:
- Provide the new high-performance HTW library as a stand-alone library.
- Postpone improving the <rte_timer.h> library; it can be done any time in the future, if someone cares to do it. And it can use the HTW library or not, whichever is appropriate.
Doing both simultaneously would require a substantial effort, and would cause much backpressure from the community (due to the modified <rte_timer.h> API and implementation).
Although it might be beneficial for the design of the HTW library to consider how an improved <rte_timer.h> would use it, it is not the primary use case of the HTW library, so co-design is not a requirement here.
^ permalink raw reply [relevance 0%]
* Re: [RFC v2 0/2] Add high-performance timer facility
2024-10-03 21:32 0% ` Morten Brørup
@ 2024-10-06 13:02 0% ` Mattias Rönnblom
2024-10-06 13:43 0% ` Morten Brørup
0 siblings, 1 reply; 200+ results
From: Mattias Rönnblom @ 2024-10-06 13:02 UTC (permalink / raw)
To: Morten Brørup, Stephen Hemminger, Mattias Rönnblom
Cc: dev, Erik Gabriel Carrillo, David Marchand, maria.lingemark,
Stefan Sundkvist, Tyler Retzlaff
On 2024-10-03 23:32, Morten Brørup wrote:
>> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
>> Sent: Thursday, 3 October 2024 20.37
>>
>> On Wed, 15 Mar 2023 18:03:40 +0100
>> Mattias Rönnblom <mattias.ronnblom@ericsson.com> wrote:
>>
>>> This patchset is an attempt to introduce a high-performance, highly
>>> scalable timer facility into DPDK.
>>>
>>> More specifically, the goals for the htimer library are:
>>>
>>> * Efficient handling of a handful up to hundreds of thousands of
>>> concurrent timers.
>>> * Make adding and canceling timers low-overhead, constant-time
>>> operations.
>>> * Provide a service functionally equivalent to that of
>>> <rte_timer.h>. API/ABI backward compatibility is secondary.
>>
>> Worthwhile goals, and the problem needs to be addressed.
>> But this patch never got accepted.
>
> I think work on it was put on hold due to the requested changes requiring a significant development effort.
> I too look forward to work on this being resumed. ;-)
>
>>
>> Please fix/improve/extend existing rte_timer instead.
>
> The rte_timer API is too "fat" for use in the fast path with millions of timers, e.g. TCP flow timers.
>
> Shoehorning a fast path feature into a slow path API is not going to cut it. I support having a separate htimer library with its own API for high volume, high-performance fast path timers.
>
> When striving for low latency across the internet, timing is everything. Packet pacing is the "new" hot thing in congestion control algorithms, and a simple software implementation would require a timer firing once per packet.
>
I think DPDK should have two public APIs in the timer area. One is a
just a bare-bones hierarchical timer wheel API, without callbacks,
auto-created per-lcore instances, MT safety or any other of the
<rte_timer.h> bells and whistles. It also doesn't make any assumptions
about the time source (other it being monotonic) or resolution.
The other is a new variant of <rte_timer.h>, using the core HTW library
for its implementation (and being public, it may also expose this
library in its header files, which may be required for efficient
operation). The new <rte_timer.h> would provide the same kind of
functionality as the old API, but with some quirks and bugs fixed, plus
potentially some new functionality added. For example, it would be
useful to allow non-preemption safe threads to add and remove timers
(something rte_timer and its spinlocks doesn't allow).
I would consider both "fast path APIs".
In addition, there should probably also be a time source API.
Considering the lead time of relatively small contributions like the
bitops extensions and the new bitset API (which still aren't in), I
can't imagine how long time it would take to get in a semi-backward
compatible rte_timer with a new implementation, plus a new timer wheel
library, into DPDK.
^ permalink raw reply [relevance 0%]
* [PATCH v5] eal: add build-time option to omit trace
2024-09-18 8:55 3% [PATCH] eal: add build-time option to omit trace Morten Brørup
` (3 preceding siblings ...)
2024-09-24 13:39 3% ` [PATCH v4] " Morten Brørup
@ 2024-10-06 12:38 2% ` Morten Brørup
2024-10-06 13:58 2% ` [PATCH v6] " Morten Brørup
` (2 subsequent siblings)
7 siblings, 0 replies; 200+ results
From: Morten Brørup @ 2024-10-06 12:38 UTC (permalink / raw)
To: Jerin Jacob, Sunil Kumar Kori; +Cc: dev, Morten Brørup
Some applications want to omit the trace feature.
Either to reduce the memory footprint, to reduce the exposed attack
surface, or for other reasons.
This patch adds an option in rte_config.h to include or omit trace in the
build. Trace is included by default.
Omitting trace works by omitting all trace points.
For API and ABI compatibility, the trace feature itself remains.
Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
---
v5:
Added public function rte_trace_feature_is_enabled(), to test if trace
is build time enabled in both the DPDK and the application. Use in test
application instead of private function. (Jerin Jacob)
v4:
* Added check for generic trace enabled when registering trace points, in
RTE_INIT. (Jerin Jacob)
* Test application uses function instead of macro to check if generic
trace is enabled. (Jerin Jacob)
* Performance test application uses function to check if generic trace is
enabled.
v3:
* Simpler version with much fewer ifdefs. (Jerin Jacob)
v2:
* Added/modified macros required for building applications with trace
omitted.
---
app/test/test_trace.c | 4 +++
app/test/test_trace_perf.c | 6 ++++
config/rte_config.h | 1 +
lib/eal/common/eal_common_trace.c | 10 +++++++
lib/eal/include/rte_trace.h | 33 ++++++++++++++++++++++
lib/eal/include/rte_trace_point.h | 21 ++++++++++++++
lib/eal/include/rte_trace_point_register.h | 2 ++
lib/eal/version.map | 3 ++
8 files changed, 80 insertions(+)
diff --git a/app/test/test_trace.c b/app/test/test_trace.c
index 00809f433b..8ea1443044 100644
--- a/app/test/test_trace.c
+++ b/app/test/test_trace.c
@@ -245,6 +245,10 @@ static struct unit_test_suite trace_tests = {
static int
test_trace(void)
{
+ if (!rte_trace_feature_is_enabled()) {
+ printf("Trace omitted at build-time, skipping test\n");
+ return TEST_SKIPPED;
+ }
return unit_test_suite_runner(&trace_tests);
}
diff --git a/app/test/test_trace_perf.c b/app/test/test_trace_perf.c
index 8257cc02be..a805fa171b 100644
--- a/app/test/test_trace_perf.c
+++ b/app/test/test_trace_perf.c
@@ -8,6 +8,7 @@
#include <rte_eal_trace.h>
#include <rte_malloc.h>
#include <rte_lcore.h>
+#include <rte_trace.h>
#include "test.h"
#include "test_trace.h"
@@ -150,6 +151,11 @@ test_trace_perf(void)
struct test_data *data;
size_t sz;
+ if (!rte_trace_feature_is_enabled()) {
+ printf("Trace omitted at build-time, skipping test\n");
+ return TEST_SKIPPED;
+ }
+
nb_cores = rte_lcore_count();
nb_workers = nb_cores - 1;
if (nb_cores < 2) {
diff --git a/config/rte_config.h b/config/rte_config.h
index dd7bb0d35b..fd6f8a2f1a 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -49,6 +49,7 @@
#define RTE_MAX_TAILQ 32
#define RTE_LOG_DP_LEVEL RTE_LOG_INFO
#define RTE_MAX_VFIO_CONTAINERS 64
+#define RTE_TRACE 1
/* bsd module defines */
#define RTE_CONTIGMEM_MAX_NUM_BUFS 64
diff --git a/lib/eal/common/eal_common_trace.c b/lib/eal/common/eal_common_trace.c
index 918f49bf4f..06130c756d 100644
--- a/lib/eal/common/eal_common_trace.c
+++ b/lib/eal/common/eal_common_trace.c
@@ -100,6 +100,16 @@ rte_trace_is_enabled(void)
return rte_atomic_load_explicit(&trace.status, rte_memory_order_acquire) != 0;
}
+bool
+__rte_trace_feature_is_enabled(void)
+{
+#ifdef RTE_TRACE
+ return true;
+#else
+ return false;
+#endif
+}
+
static void
trace_mode_set(rte_trace_point_t *t, enum rte_trace_mode mode)
{
diff --git a/lib/eal/include/rte_trace.h b/lib/eal/include/rte_trace.h
index a6e991fad3..340df4f8a0 100644
--- a/lib/eal/include/rte_trace.h
+++ b/lib/eal/include/rte_trace.h
@@ -35,6 +35,39 @@ extern "C" {
__rte_experimental
bool rte_trace_is_enabled(void);
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * @internal
+ *
+ * Test if trace feature is enabled at compile time.
+ *
+ * @return
+ * true if trace feature is enabled, false otherwise.
+ */
+__rte_experimental
+bool __rte_trace_feature_is_enabled(void);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
+ *
+ * Test if trace feature is enabled at compile time.
+ *
+ * @return
+ * true if trace feature is enabled, false otherwise.
+ */
+static __rte_always_inline
+bool rte_trace_feature_is_enabled(void)
+{
+#ifdef RTE_TRACE
+ return __rte_trace_feature_is_enabled();
+#else
+ return false;
+#endif
+}
+
/**
* Enumerate trace mode operation.
*/
diff --git a/lib/eal/include/rte_trace_point.h b/lib/eal/include/rte_trace_point.h
index 41e2a7f99e..b80688ce89 100644
--- a/lib/eal/include/rte_trace_point.h
+++ b/lib/eal/include/rte_trace_point.h
@@ -212,6 +212,25 @@ bool rte_trace_point_is_enabled(rte_trace_point_t *tp);
__rte_experimental
rte_trace_point_t *rte_trace_point_lookup(const char *name);
+/**
+ * @internal
+ *
+ * Test if the tracepoint compile-time option is enabled.
+ *
+ * @return
+ * true if tracepoint enabled, false otherwise.
+ */
+__rte_experimental
+static __rte_always_inline bool
+__rte_trace_point_generic_is_enabled(void)
+{
+#ifdef RTE_TRACE
+ return true;
+#else
+ return false;
+#endif
+}
+
/**
* @internal
*
@@ -359,6 +378,8 @@ __rte_trace_point_emit_ev_header(void *mem, uint64_t in)
#define __rte_trace_point_emit_header_generic(t) \
void *mem; \
do { \
+ if (!__rte_trace_point_generic_is_enabled()) \
+ return; \
const uint64_t val = rte_atomic_load_explicit(t, rte_memory_order_acquire); \
if (likely(!(val & __RTE_TRACE_FIELD_ENABLE_MASK))) \
return; \
diff --git a/lib/eal/include/rte_trace_point_register.h b/lib/eal/include/rte_trace_point_register.h
index 41260e5964..429b993fc2 100644
--- a/lib/eal/include/rte_trace_point_register.h
+++ b/lib/eal/include/rte_trace_point_register.h
@@ -23,6 +23,8 @@ rte_trace_point_t __rte_section("__rte_trace_point") __##trace; \
static const char __##trace##_name[] = RTE_STR(name); \
RTE_INIT(trace##_init) \
{ \
+ if (!__rte_trace_point_generic_is_enabled()) \
+ return; \
__rte_trace_point_register(&__##trace, __##trace##_name, \
(void (*)(void)) trace); \
}
diff --git a/lib/eal/version.map b/lib/eal/version.map
index e3ff412683..15c694c2da 100644
--- a/lib/eal/version.map
+++ b/lib/eal/version.map
@@ -396,6 +396,9 @@ EXPERIMENTAL {
# added in 24.03
rte_vfio_get_device_info; # WINDOWS_NO_EXPORT
+
+ # added in 24.11
+ __rte_trace_feature_is_enabled;
};
INTERNAL {
--
2.43.0
^ permalink raw reply [relevance 2%]
* [PATCH] net/ena: restructure the llq policy user setting
@ 2024-10-06 12:32 3% shaibran
0 siblings, 0 replies; 200+ results
From: shaibran @ 2024-10-06 12:32 UTC (permalink / raw)
To: ferruh.yigit; +Cc: dev, Shai Brandes
From: Shai Brandes <shaibran@amazon.com>
Replaced `enable_llq`, `normal_llq_hdr` and `large_llq_hdr`
devargs with a new shared devarg named `llq_policy` that
implements the same logic and accepts the following values:
0 - Disable LLQ.
Use with extreme caution as it leads to a huge performance
degradation on AWS instances built with Nitro v4 onwards.
1 - Accept device recommended LLQ policy (Default).
Device can recommend normal or large LLQ policy.
2 - Enforce normal LLQ policy.
3 - Enforce large LLQ policy.
Required for packets with header that exceed 96 bytes on
AWS instances built with Nitro v2 and earlier.
Signed-off-by: Shai Brandes <shaibran@amazon.com>
Reviewed-by: Amit Bernstein <amitbern@amazon.com>
---
doc/guides/nics/ena.rst | 20 ++---
doc/guides/rel_notes/release_24_11.rst | 8 ++
drivers/net/ena/ena_ethdev.c | 103 ++++++++-----------------
drivers/net/ena/ena_ethdev.h | 3 -
4 files changed, 48 insertions(+), 86 deletions(-)
diff --git a/doc/guides/nics/ena.rst b/doc/guides/nics/ena.rst
index 2b105834a0..1467c0c190 100644
--- a/doc/guides/nics/ena.rst
+++ b/doc/guides/nics/ena.rst
@@ -107,11 +107,14 @@ Configuration
Runtime Configuration
^^^^^^^^^^^^^^^^^^^^^
- * **large_llq_hdr** (default 0)
+ * **llq_policy** (default 1)
- Enables or disables usage of large LLQ headers. This option will have
- effect only if the device also supports large LLQ headers. Otherwise, the
- default value will be used.
+ Controls whether use device recommended header policy or override it.
+ 0 - Disable LLQ (Use with extreme caution as it leads to a huge performance
+ degradation on AWS instances built with Nitro v4 onwards).
+ 1 - Accept device recommended LLQ policy (Default).
+ 2 - Enforce normal LLQ policy.
+ 3 - Enforce large LLQ policy.
* **normal_llq_hdr** (default 0)
@@ -126,15 +129,6 @@ Runtime Configuration
timer service. Setting this parameter to 0 disables this feature. Maximum
allowed value is 60 seconds.
- * **enable_llq** (default 1)
-
- Determines whenever the driver should use the LLQ (if it's available) or
- not.
-
- **NOTE: On the 6th generation AWS instances disabling LLQ may lead to a
- huge performance degradation. In general disabling LLQ is highly not
- recommended!**
-
* **control_poll_interval** (default 0)
Enable polling-based functionality of the admin queues,
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index dfa4795f85..b88e05f349 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -75,6 +75,12 @@ New Features
registers by module names and get the information (names, values and other
attributes) of the filtered registers.
+* **Updated Amazon ENA (Elastic Network Adapter) net driver.**
+
+ * Modified the PMD API that controls the LLQ header policy.
+ * Replaced ``enable_llq``, ``normal_llq_hdr`` and ``large_llq_hdr`` devargs
+ with a new shared devarg ``llq_policy`` that keeps the same logic.
+
* **Updated Cisco enic driver.**
* Added SR-IOV VF support.
@@ -112,6 +118,8 @@ API Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* drivers/net/ena: Removed ``enable_llq``, ``normal_llq_hdr`` and ``large_llq_hdr`` devargs
+ and replaced it with a new shared devarg ``llq_policy`` that keeps the same logic.
ABI Changes
-----------
diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index e0c239e88f..18e0be6d5c 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -79,18 +79,25 @@ struct ena_stats {
ENA_STAT_ENTRY(stat, srd)
/* Device arguments */
-#define ENA_DEVARG_LARGE_LLQ_HDR "large_llq_hdr"
-#define ENA_DEVARG_NORMAL_LLQ_HDR "normal_llq_hdr"
+
+/* llq_policy Controls whether to disable LLQ, use device recommended
+ * header policy or overriding the device recommendation.
+ * 0 - Disable LLQ. Use with extreme caution as it leads to a huge
+ * performance degradation on AWS instances built with Nitro v4 onwards.
+ * 1 - Accept device recommended LLQ policy (Default).
+ * Device can recommend normal or large LLQ policy.
+ * 2 - Enforce normal LLQ policy.
+ * 3 - Enforce large LLQ policy.
+ * Required for packets with header that exceed 96 bytes on
+ * AWS instances built with Nitro v2 and Nitro v1.
+ */
+#define ENA_DEVARG_LLQ_POLICY "llq_policy"
+
/* Timeout in seconds after which a single uncompleted Tx packet should be
* considered as a missing.
*/
#define ENA_DEVARG_MISS_TXC_TO "miss_txc_to"
-/*
- * Controls whether LLQ should be used (if available). Enabled by default.
- * NOTE: It's highly not recommended to disable the LLQ, as it may lead to a
- * huge performance degradation on 6th generation AWS instances.
- */
-#define ENA_DEVARG_ENABLE_LLQ "enable_llq"
+
/*
* Controls the period of time (in milliseconds) between two consecutive inspections of
* the control queues when the driver is in poll mode and not using interrupts.
@@ -294,9 +301,9 @@ static int ena_xstats_get_by_id(struct rte_eth_dev *dev,
const uint64_t *ids,
uint64_t *values,
unsigned int n);
-static int ena_process_bool_devarg(const char *key,
- const char *value,
- void *opaque);
+static int ena_process_llq_policy_devarg(const char *key,
+ const char *value,
+ void *opaque);
static int ena_parse_devargs(struct ena_adapter *adapter,
struct rte_devargs *devargs);
static void ena_copy_customer_metrics(struct ena_adapter *adapter,
@@ -312,7 +319,6 @@ static int ena_rx_queue_intr_disable(struct rte_eth_dev *dev,
static int ena_configure_aenq(struct ena_adapter *adapter);
static int ena_mp_primary_handle(const struct rte_mp_msg *mp_msg,
const void *peer);
-static ena_llq_policy ena_define_llq_hdr_policy(struct ena_adapter *adapter);
static bool ena_use_large_llq_hdr(struct ena_adapter *adapter, uint8_t recommended_entry_size);
static const struct eth_dev_ops ena_dev_ops = {
@@ -2320,9 +2326,6 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev)
/* Assign default devargs values */
adapter->missing_tx_completion_to = ENA_TX_TIMEOUT;
- adapter->enable_llq = true;
- adapter->use_large_llq_hdr = false;
- adapter->use_normal_llq_hdr = false;
/* Get user bypass */
rc = ena_parse_devargs(adapter, pci_dev->device.devargs);
@@ -2330,8 +2333,6 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev)
PMD_INIT_LOG(CRIT, "Failed to parse devargs\n");
goto err;
}
- adapter->llq_header_policy = ena_define_llq_hdr_policy(adapter);
-
rc = ena_com_allocate_customer_metrics_buffer(ena_dev);
if (rc != 0) {
PMD_INIT_LOG(CRIT, "Failed to allocate customer metrics buffer\n");
@@ -3734,44 +3735,29 @@ static int ena_process_uint_devarg(const char *key,
return 0;
}
-static int ena_process_bool_devarg(const char *key,
- const char *value,
- void *opaque)
+static int ena_process_llq_policy_devarg(const char *key, const char *value, void *opaque)
{
struct ena_adapter *adapter = opaque;
- bool bool_value;
+ uint32_t policy;
- /* Parse the value. */
- if (strcmp(value, "1") == 0) {
- bool_value = true;
- } else if (strcmp(value, "0") == 0) {
- bool_value = false;
+ policy = strtoul(value, NULL, DECIMAL_BASE);
+ if (policy < ENA_LLQ_POLICY_LAST) {
+ adapter->llq_header_policy = policy;
} else {
- PMD_INIT_LOG(ERR,
- "Invalid value: '%s' for key '%s'. Accepted: '0' or '1'\n",
- value, key);
+ PMD_INIT_LOG(ERR, "Invalid value: '%s' for key '%s'. valid [0-3]\n", value, key);
return -EINVAL;
}
-
- /* Now, assign it to the proper adapter field. */
- if (strcmp(key, ENA_DEVARG_LARGE_LLQ_HDR) == 0)
- adapter->use_large_llq_hdr = bool_value;
- else if (strcmp(key, ENA_DEVARG_NORMAL_LLQ_HDR) == 0)
- adapter->use_normal_llq_hdr = bool_value;
- else if (strcmp(key, ENA_DEVARG_ENABLE_LLQ) == 0)
- adapter->enable_llq = bool_value;
-
+ PMD_DRV_LOG(INFO,
+ "LLQ policy is %u [0 - disabled, 1 - device recommended, 2 - normal, 3 - large]\n",
+ adapter->llq_header_policy);
return 0;
}
-static int ena_parse_devargs(struct ena_adapter *adapter,
- struct rte_devargs *devargs)
+static int ena_parse_devargs(struct ena_adapter *adapter, struct rte_devargs *devargs)
{
static const char * const allowed_args[] = {
- ENA_DEVARG_LARGE_LLQ_HDR,
- ENA_DEVARG_NORMAL_LLQ_HDR,
+ ENA_DEVARG_LLQ_POLICY,
ENA_DEVARG_MISS_TXC_TO,
- ENA_DEVARG_ENABLE_LLQ,
ENA_DEVARG_CONTROL_PATH_POLL_INTERVAL,
NULL,
};
@@ -3783,27 +3769,17 @@ static int ena_parse_devargs(struct ena_adapter *adapter,
kvlist = rte_kvargs_parse(devargs->args, allowed_args);
if (kvlist == NULL) {
- PMD_INIT_LOG(ERR, "Invalid device arguments: %s\n",
- devargs->args);
+ PMD_INIT_LOG(ERR, "Invalid device arguments: %s\n", devargs->args);
return -EINVAL;
}
-
- rc = rte_kvargs_process(kvlist, ENA_DEVARG_LARGE_LLQ_HDR,
- ena_process_bool_devarg, adapter);
- if (rc != 0)
- goto exit;
- rc = rte_kvargs_process(kvlist, ENA_DEVARG_NORMAL_LLQ_HDR,
- ena_process_bool_devarg, adapter);
+ rc = rte_kvargs_process(kvlist, ENA_DEVARG_LLQ_POLICY,
+ ena_process_llq_policy_devarg, adapter);
if (rc != 0)
goto exit;
rc = rte_kvargs_process(kvlist, ENA_DEVARG_MISS_TXC_TO,
ena_process_uint_devarg, adapter);
if (rc != 0)
goto exit;
- rc = rte_kvargs_process(kvlist, ENA_DEVARG_ENABLE_LLQ,
- ena_process_bool_devarg, adapter);
- if (rc != 0)
- goto exit;
rc = rte_kvargs_process(kvlist, ENA_DEVARG_CONTROL_PATH_POLL_INTERVAL,
ena_process_uint_devarg, adapter);
if (rc != 0)
@@ -4027,9 +4003,7 @@ RTE_PMD_REGISTER_PCI(net_ena, rte_ena_pmd);
RTE_PMD_REGISTER_PCI_TABLE(net_ena, pci_id_ena_map);
RTE_PMD_REGISTER_KMOD_DEP(net_ena, "* igb_uio | uio_pci_generic | vfio-pci");
RTE_PMD_REGISTER_PARAM_STRING(net_ena,
- ENA_DEVARG_LARGE_LLQ_HDR "=<0|1> "
- ENA_DEVARG_NORMAL_LLQ_HDR "=<0|1> "
- ENA_DEVARG_ENABLE_LLQ "=<0|1> "
+ ENA_DEVARG_LLQ_POLICY "=<0|1|2|3> "
ENA_DEVARG_MISS_TXC_TO "=<uint>"
ENA_DEVARG_CONTROL_PATH_POLL_INTERVAL "=<0-1000>");
RTE_LOG_REGISTER_SUFFIX(ena_logtype_init, init, NOTICE);
@@ -4217,17 +4191,6 @@ ena_mp_primary_handle(const struct rte_mp_msg *mp_msg, const void *peer)
return rte_mp_reply(&mp_rsp, peer);
}
-static ena_llq_policy ena_define_llq_hdr_policy(struct ena_adapter *adapter)
-{
- if (!adapter->enable_llq)
- return ENA_LLQ_POLICY_DISABLED;
- if (adapter->use_large_llq_hdr)
- return ENA_LLQ_POLICY_LARGE;
- if (adapter->use_normal_llq_hdr)
- return ENA_LLQ_POLICY_NORMAL;
- return ENA_LLQ_POLICY_RECOMMENDED;
-}
-
static bool ena_use_large_llq_hdr(struct ena_adapter *adapter, uint8_t recommended_entry_size)
{
if (adapter->llq_header_policy == ENA_LLQ_POLICY_LARGE) {
diff --git a/drivers/net/ena/ena_ethdev.h b/drivers/net/ena/ena_ethdev.h
index 7d82d222ce..fe7d4a2d65 100644
--- a/drivers/net/ena/ena_ethdev.h
+++ b/drivers/net/ena/ena_ethdev.h
@@ -337,9 +337,6 @@ struct ena_adapter {
uint32_t active_aenq_groups;
bool trigger_reset;
- bool enable_llq;
- bool use_large_llq_hdr;
- bool use_normal_llq_hdr;
ena_llq_policy llq_header_policy;
uint32_t last_tx_comp_qid;
--
2.17.1
^ permalink raw reply [relevance 3%]
* RE: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-10-03 7:00 0% ` Akhil Goyal
@ 2024-10-06 11:10 0% ` Morten Brørup
2024-10-09 11:21 0% ` Akhil Goyal
0 siblings, 1 reply; 200+ results
From: Morten Brørup @ 2024-10-06 11:10 UTC (permalink / raw)
To: Akhil Goyal, fengchengwen, dev
Cc: thomas, david.marchand, hemant.agrawal, Anoob Joseph,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde,
Nagadheeraj Rottela, mdr
> From: Akhil Goyal [mailto:gakhil@marvell.com]
> Sent: Thursday, 3 October 2024 09.01
>
> Hi Morten,
> >
> > Apologies for delayed response.
> > > Maybe a combination, returning the lowest end of the two versions
> of the list,
> > > would work...
> > >
> > > ----------------------------------
> > > Common header file (rte_common.h):
> > > ----------------------------------
> > >
> > > /* Add at end of enum list in the header file. */
> > > #define RTE_ENUM_LIST_END(name) \
> > > _ # name # _ENUM_LIST_END /**< @internal */
> > >
> > > /* Add somewhere in header file, preferably after the enum list. */
> > > #define rte_declare_enum_list_end(name) \
> > > /** @internal */ \
> > > int _# name # _enum_list_end(void); \
> > > \
> > > static int name # _enum_list_end(void) \
> > > { \
> > > static int cached = 0; \
> > > \
> > > if (likely(cached != 0)) \
> > > return cached; \
> > > \
> > > return cached = RTE_MIN( \
> > > RTE_ENUM_LIST_END(name), \
> > > _ # name # _enum_list_end()); \
> > > } \
> > > \
> > > int _# name # _enum_list_end(void)
> > >
> > > /* Add in the library/driver implementation. */
> > > #define rte_define_enum_list_end(name) \
> > > int _# name # _enum_list_end(void) \
> > > { \
> > > return RTE_ENUM_LIST_END(name); \
> > > } \
> > > \
> > > int _# name # _enum_list_end(void)
> > >
> > > --------------------
> > > Library header file:
> > > --------------------
> > >
> > > enum rte_crypto_asym_op_type {
> > > RTE_CRYPTO_ASYM_OP_VERIFY,
> > > /**< Signature Verification operation */
> > > RTE_ENUM_LIST_END(rte_crypto_asym_op)
> >
> > Will the ABI check be ok for adding anything in between
> > RTE_CRYPTO_ASYM_OP_VERIFY and
> > RTE_ENUM_LIST_END(rte_crypto_asym_op)?
> > Don’t we need to add exception for that if we somehow make it
> internal by
> > adding a comment only?
> > Library is actually not restricting the application to not use
> > RTE_ENUM_LIST_END(rte_crypto_asym_op) directly.
> >
> > Also we may need to expose the .c file internal function as
> experimental in
> > version.map
> >
> > > }
> > >
> > > rte_declare_enum_list_end(rte_crypto_asym_op);
> > >
> > > ---------------
> > > Library C file:
> > > ---------------
> > >
> > > rte_define_enum_list_end(rte_crypto_asym_op);
> >
> > If we want to make it a generic thing in rte_common.h
> > Will the below change be ok?
> > ----------------------------------
> > Common header file (rte_common.h):
> > ----------------------------------
> > #define rte_define_enum_list_end(name, last_value) \
> > static inline int name ## _enum_list_end(void) \
> > { \
> > return last_value + 1; \
> > }
> >
> > ----------------
> > Lib header file
> > ----------------
> > //After the enum definition define the list end as below
> > rte_define_enum_list_end(rte_crypto_asym_op,
> > RTE_CRYPTO_ASYM_OP_VERIFY);
> >
> >
> > And wherever list end is needed use
> rte_crypto_asym_op_enum_list_end()?
> >
> > With this change, abi check will not complain for any new addition at
> the end of
> > enum.
> > And we do not need to expose any internal API in version.map.
> >
> Can we move forward with above suggestion?
Sorry about the late reply, Akhil.
It seems Ferruh and David have picked up this discussion with good arguments.
I have no preferences for a generic solution, especially if this is an isolated case. A generic solution can be added at any time later.
^ permalink raw reply [relevance 0%]
* RE: [PATCH dpdk v2 00/16] IPv6 APIs overhaul
` (9 preceding siblings ...)
2024-10-01 8:17 3% ` [PATCH dpdk v2 13/16] rte_flow: " Robin Jarry
@ 2024-10-06 9:04 3% ` Morten Brørup
10 siblings, 0 replies; 200+ results
From: Morten Brørup @ 2024-10-06 9:04 UTC (permalink / raw)
To: Robin Jarry, dev
Cc: Stephen Hemminger, Vladimir Medvedkin, Konstantin Ananyev,
Bruce Richardson
> From: Robin Jarry [mailto:rjarry@redhat.com]
> Sent: Tuesday, 1 October 2024 10.17
>
> Hi everyone,
>
> As discussed recently [1], here is a first draft of the IPv6 APIs
> rework. The
> API change was announced before the 24.07 release [2]. This series is
> intended
> for 24.11.
>
> [1] http://inbox.dpdk.org/dev/D2SR8T1H39CJ.JRQFI6JEH0OX@redhat.com/
> [2]
> https://git.dpdk.org/dpdk/commit/?id=835d4c41e0ab58a115c2170c886ba6d3cc
> 1b5764
>
> I tried to keep the patches as small as possible; unfortunately some of
> them
> are quite big and cannot be broken down if we want to preserve a
> bisectable
> tree.
>
> Let me know what you think.
>
> Thanks!
Sorry about the late review. And thank you for this series!
In addition to my patch specific comments, here's some general feedback for the series.
Mixing ip6 and ipv6, and IP6 and IPV6 makes it hard to guess names.
Please use ipv6 and IPV6 everywhere, also in file names.
On the other hand... If file names use "6" postfix to indicate IP Version 6, e.g. rte_fib6.h, rte_ip6.h, then stick with that; just make sure it is consistent.
My point is: Where possible, please be consistent in naming.
NB: For API/ABI compatibility, don't clean up existing names for consistency.
I recall seeing some other patch replacing hardcoded IPv6 addresses given as zero-terminated strings (which are, in fact, 17 byte in size) with arrays containing 16 hex integers, to fix some modern compiler warning about mixed sizes. If adding new hardcoded IPv6 addresses, please use { 0x12, 0x34, 0x56, ...} instead of "\x12\x34\x56...".
-Morten
^ permalink raw reply [relevance 3%]
* RE: [PATCH dpdk v2 03/16] net: add structure for ipv6 addresses
@ 2024-10-06 8:18 3% ` Morten Brørup
0 siblings, 0 replies; 200+ results
From: Morten Brørup @ 2024-10-06 8:18 UTC (permalink / raw)
To: Robin Jarry, Stephen Hemminger, Wathsala Vithanage, Min Zhou,
David Christensen, Stanislaw Kardach
Cc: dev, bruce.richardson, konstantin.v.ananyev, Vipin Varghese
+to: Various CPU architecture maintainers
+to: Stephen (already participating in this discussion)
+cc: x86 CPU architecture maintainers
> From: Robin Jarry [mailto:rjarry@redhat.com]
> Sent: Tuesday, 1 October 2024 10.17
>
> There is currently no structure defined for IPv6 addresses. Introduce
> one that is simply a uint8_t array of 16 elements without any union.
> The
> idea is to ensure this structure alignment is 1 so that it can be
> mapped
> directly on unaligned packet memory.
>
> Signed-off-by: Robin Jarry <rjarry@redhat.com>
> ---
> lib/net/rte_ip6.h | 10 ++++++++++
> 1 file changed, 10 insertions(+)
>
> diff --git a/lib/net/rte_ip6.h b/lib/net/rte_ip6.h
> index 5ad1dd25db08..52c41088681e 100644
> --- a/lib/net/rte_ip6.h
> +++ b/lib/net/rte_ip6.h
> @@ -35,6 +35,16 @@
> extern "C" {
> #endif
>
> +#define RTE_IPV6_ADDR_SIZE 16
> +#define RTE_IPV6_MAX_DEPTH 128
> +
> +/**
> + * IPv6 Address
> + */
> +struct rte_ipv6_addr {
> + unsigned char a[RTE_IPV6_ADDR_SIZE];
> +};
This has been discussed before, but I want to double check...
If - sometime in the future - we want to add a union to offer a 2-byte access variant and make the structure to 2-byte aligned (which is the common case in Ethernet packets), it will break both API and ABI. This seems unlikely to get accepted at a later time, so I think we are - right now - in a situation where it's now or never:
struct rte_ipv6_addr {
__extension__
union {
unsigned char a[RTE_IPV6_ADDR_SIZE];
uint16_t w[RTE_IPV6_ADDR_SIZE / 2];
};
};
Unless some of the CPU folks want the 2-byte aligned variant, stick with what you offered.
^ permalink raw reply [relevance 3%]
* [PATCH v7 1/6] eventdev: introduce event pre-scheduling
2024-10-05 7:59 3% ` [PATCH v7 0/6] Introduce " pbhagavatula
@ 2024-10-05 7:59 3% ` pbhagavatula
2024-10-06 17:06 3% ` [PATCH v8 0/6] Introduce " pbhagavatula
1 sibling, 0 replies; 200+ results
From: pbhagavatula @ 2024-10-05 7:59 UTC (permalink / raw)
To: jerinj, sthotton, abdullah.sevincer, hemant.agrawal,
sachin.saxena, harry.van.haaren, mattias.ronnblom, liangma,
peter.mccarthy, stephen
Cc: dev, Pavan Nikhilesh
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
Event pre-scheduling improves scheduling performance by assigning events
to event ports in advance when dequeues are issued.
The dequeue operation initiates the pre-schedule operation, which completes
in parallel without affecting the dequeued event flow contexts and
dequeue latency.
Event devices can indicate pre-scheduling capabilities using
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event device info
function `info.event_dev_cap`.
Applications can select the pre-schedule type and configure it through
`rte_event_dev_config.preschedule_type` during `rte_event_dev_configure`.
The supported pre-schedule types are:
* `RTE_EVENT_PRESCHEDULE_NONE` - No pre-scheduling.
* `RTE_EVENT_PRESCHEDULE` - Always issue a pre-schedule on dequeue.
* `RTE_EVENT_PRESCHEDULE_ADAPTIVE` - Delay issuing pre-schedule until
there are no forward progress constraints with the held flow contexts.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
app/test/test_eventdev.c | 108 ++++++++++++++++++++
doc/guides/eventdevs/features/default.ini | 1 +
doc/guides/prog_guide/eventdev/eventdev.rst | 23 +++++
doc/guides/rel_notes/release_24_11.rst | 10 ++
lib/eventdev/rte_eventdev.h | 49 +++++++++
5 files changed, 191 insertions(+)
diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c
index 9a6c8f470c..a45d1396d7 100644
--- a/app/test/test_eventdev.c
+++ b/app/test/test_eventdev.c
@@ -1251,6 +1251,112 @@ test_eventdev_profile_switch(void)
return TEST_SUCCESS;
}
+static int
+preschedule_test(enum rte_event_dev_preschedule_type preschedule_type, const char *preschedule_name)
+{
+#define NB_EVENTS 1024
+ uint64_t start, total;
+ struct rte_event ev;
+ int rc, cnt;
+
+ ev.event_type = RTE_EVENT_TYPE_CPU;
+ ev.queue_id = 0;
+ ev.op = RTE_EVENT_OP_NEW;
+ ev.u64 = 0xBADF00D0;
+
+ for (cnt = 0; cnt < NB_EVENTS; cnt++) {
+ ev.flow_id = cnt;
+ rc = rte_event_enqueue_burst(TEST_DEV_ID, 0, &ev, 1);
+ TEST_ASSERT(rc == 1, "Failed to enqueue event");
+ }
+
+ RTE_SET_USED(preschedule_type);
+ total = 0;
+ while (cnt) {
+ start = rte_rdtsc_precise();
+ rc = rte_event_dequeue_burst(TEST_DEV_ID, 0, &ev, 1, 0);
+ if (rc) {
+ total += rte_rdtsc_precise() - start;
+ cnt--;
+ }
+ }
+ printf("Preschedule type : %s, avg cycles %" PRIu64 "\n", preschedule_name,
+ total / NB_EVENTS);
+
+ return TEST_SUCCESS;
+}
+
+static int
+preschedule_configure(enum rte_event_dev_preschedule_type type, struct rte_event_dev_info *info)
+{
+ struct rte_event_dev_config dev_conf;
+ struct rte_event_queue_conf qcfg;
+ struct rte_event_port_conf pcfg;
+ int rc;
+
+ devconf_set_default_sane_values(&dev_conf, info);
+ dev_conf.nb_event_ports = 1;
+ dev_conf.nb_event_queues = 1;
+ dev_conf.preschedule_type = type;
+
+ rc = rte_event_dev_configure(TEST_DEV_ID, &dev_conf);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+
+ rc = rte_event_port_default_conf_get(TEST_DEV_ID, 0, &pcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to get port0 default config");
+ rc = rte_event_port_setup(TEST_DEV_ID, 0, &pcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to setup port0");
+
+ rc = rte_event_queue_default_conf_get(TEST_DEV_ID, 0, &qcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to get queue0 default config");
+ rc = rte_event_queue_setup(TEST_DEV_ID, 0, &qcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to setup queue0");
+
+ rc = rte_event_port_link(TEST_DEV_ID, 0, NULL, NULL, 0);
+ TEST_ASSERT(rc == (int)dev_conf.nb_event_queues, "Failed to link port, device %d",
+ TEST_DEV_ID);
+
+ rc = rte_event_dev_start(TEST_DEV_ID);
+ TEST_ASSERT_SUCCESS(rc, "Failed to start event device");
+
+ return 0;
+}
+
+static int
+test_eventdev_preschedule_configure(void)
+{
+ struct rte_event_dev_info info;
+ int rc;
+
+ rte_event_dev_info_get(TEST_DEV_ID, &info);
+
+ if ((info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE) == 0)
+ return TEST_SKIPPED;
+
+ rc = preschedule_configure(RTE_EVENT_PRESCHEDULE_NONE, &info);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+ rc = preschedule_test(RTE_EVENT_PRESCHEDULE_NONE, "RTE_EVENT_PRESCHEDULE_NONE");
+ TEST_ASSERT_SUCCESS(rc, "Failed to test preschedule RTE_EVENT_PRESCHEDULE_NONE");
+
+ rte_event_dev_stop(TEST_DEV_ID);
+ rc = preschedule_configure(RTE_EVENT_PRESCHEDULE, &info);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+ rc = preschedule_test(RTE_EVENT_PRESCHEDULE, "RTE_EVENT_PRESCHEDULE");
+ TEST_ASSERT_SUCCESS(rc, "Failed to test preschedule RTE_EVENT_PRESCHEDULE");
+
+ if (info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE) {
+ rte_event_dev_stop(TEST_DEV_ID);
+ rc = preschedule_configure(RTE_EVENT_PRESCHEDULE_ADAPTIVE, &info);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+ rc = preschedule_test(RTE_EVENT_PRESCHEDULE_ADAPTIVE,
+ "RTE_EVENT_PRESCHEDULE_ADAPTIVE");
+ TEST_ASSERT_SUCCESS(rc,
+ "Failed to test preschedule RTE_EVENT_PRESCHEDULE_ADAPTIVE");
+ }
+
+ return TEST_SUCCESS;
+}
+
static int
test_eventdev_close(void)
{
@@ -1311,6 +1417,8 @@ static struct unit_test_suite eventdev_common_testsuite = {
test_eventdev_start_stop),
TEST_CASE_ST(eventdev_configure_setup, eventdev_stop_device,
test_eventdev_profile_switch),
+ TEST_CASE_ST(eventdev_configure_setup, NULL,
+ test_eventdev_preschedule_configure),
TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
test_eventdev_link),
TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
diff --git a/doc/guides/eventdevs/features/default.ini b/doc/guides/eventdevs/features/default.ini
index 1cc4303fe5..c8d5ed2d74 100644
--- a/doc/guides/eventdevs/features/default.ini
+++ b/doc/guides/eventdevs/features/default.ini
@@ -22,6 +22,7 @@ carry_flow_id =
maintenance_free =
runtime_queue_attr =
profile_links =
+preschedule =
;
; Features of a default Ethernet Rx adapter.
diff --git a/doc/guides/prog_guide/eventdev/eventdev.rst b/doc/guides/prog_guide/eventdev/eventdev.rst
index fb6dfce102..9da5531859 100644
--- a/doc/guides/prog_guide/eventdev/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev/eventdev.rst
@@ -357,6 +357,29 @@ Worker path:
// Process the event received.
}
+Event Pre-scheduling
+~~~~~~~~~~~~~~~~~~~~
+
+Event pre-scheduling improves scheduling performance by assigning events to event ports in advance
+when dequeues are issued.
+The `rte_event_dequeue_burst` operation initiates the pre-schedule operation, which completes
+in parallel without affecting the dequeued event flow contexts and dequeue latency.
+On the next dequeue operation, the pre-scheduled events are dequeued and pre-schedule is initiated
+again.
+
+An application can use event pre-scheduling if the event device supports it at either device
+level or at a individual port level.
+The application must check pre-schedule capability by checking if ``rte_event_dev_info.event_dev_cap``
+has the bit ``RTE_EVENT_DEV_CAP_PRESCHEDULE`` or ``RTE_EVENT_DEV_CAP_PRESCHEDULE_ADAPTIVE`` set, if
+present pre-scheduling can be enabled at device
+configuration time by setting appropriate pre-schedule type in ``rte_event_dev_config.preschedule``.
+
+The following pre-schedule types are supported:
+ * ``RTE_EVENT_PRESCHEDULE_NONE`` - No pre-scheduling.
+ * ``RTE_EVENT_PRESCHEDULE`` - Always issue a pre-schedule when dequeue is issued.
+ * ``RTE_EVENT_PRESCHEDULE_ADAPTIVE`` - Issue pre-schedule when dequeue is issued and there are
+ no forward progress constraints.
+
Starting the EventDev
~~~~~~~~~~~~~~~~~~~~~
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index e0a9aa55a1..a294a753e7 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -67,6 +67,14 @@ New Features
The new statistics are useful for debugging and profiling.
+* **Added event device pre-scheduling support.**
+
+ Added support for pre-scheduling of events to event ports to improve
+ scheduling performance and latency.
+
+ * Added ``rte_event_dev_config::preschedule_type`` to configure the device
+ level pre-scheduling type.
+
Removed Items
-------------
@@ -112,6 +120,8 @@ ABI Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* eventdev: Added ``preschedule_type`` field to ``rte_event_dev_config``
+ structure.
Known Issues
------------
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 08e5f9320b..4b69e74577 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -446,6 +446,30 @@ struct rte_event;
* @see RTE_SCHED_TYPE_PARALLEL
*/
+#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE (1ULL << 16)
+/**< Event device supports event pre-scheduling.
+ *
+ * When this capability is available, the application can enable event pre-scheduling on the event
+ * device to pre-schedule events to a event port when `rte_event_dequeue_burst()`
+ * is issued.
+ * The pre-schedule process starts with the `rte_event_dequeue_burst()` call and the
+ * pre-scheduled events are returned on the next `rte_event_dequeue_burst()` call.
+ *
+ * @see rte_event_dev_configure()
+ */
+
+#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE (1ULL << 17)
+/**< Event device supports adaptive event pre-scheduling.
+ *
+ * When this capability is available, the application can enable adaptive pre-scheduling
+ * on the event device where the events are pre-scheduled when there are no forward
+ * progress constraints with the currently held flow contexts.
+ * The pre-schedule process starts with the `rte_event_dequeue_burst()` call and the
+ * pre-scheduled events are returned on the next `rte_event_dequeue_burst()` call.
+ *
+ * @see rte_event_dev_configure()
+ */
+
/* Event device priority levels */
#define RTE_EVENT_DEV_PRIORITY_HIGHEST 0
/**< Highest priority level for events and queues.
@@ -680,6 +704,26 @@ rte_event_dev_attr_get(uint8_t dev_id, uint32_t attr_id,
* @see rte_event_dequeue_timeout_ticks(), rte_event_dequeue_burst()
*/
+/** Event device pre-schedule type enumeration. */
+enum rte_event_dev_preschedule_type {
+ RTE_EVENT_PRESCHEDULE_NONE,
+ /**< Disable pre-schedule across the event device or on a given event port.
+ * @ref rte_event_dev_config.preschedule_type
+ */
+ RTE_EVENT_PRESCHEDULE,
+ /**< Enable pre-schedule always across the event device or a given event port.
+ * @ref rte_event_dev_config.preschedule_type
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
+ */
+ RTE_EVENT_PRESCHEDULE_ADAPTIVE,
+ /**< Enable adaptive pre-schedule across the event device or a given event port.
+ * Delay issuing pre-schedule until there are no forward progress constraints with
+ * the held flow contexts.
+ * @ref rte_event_dev_config.preschedule_type
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
+ */
+};
+
/** Event device configuration structure */
struct rte_event_dev_config {
uint32_t dequeue_timeout_ns;
@@ -752,6 +796,11 @@ struct rte_event_dev_config {
* optimized for single-link usage, this field is a hint for how many
* to allocate; otherwise, regular event ports and queues will be used.
*/
+ enum rte_event_dev_preschedule_type preschedule_type;
+ /**< Event pre-schedule type to use across the event device, if supported.
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
+ */
};
/**
--
2.25.1
^ permalink raw reply [relevance 3%]
* [PATCH v7 0/6] Introduce event pre-scheduling
2024-10-05 7:25 3% ` [PATCH v6 " pbhagavatula
2024-10-05 7:25 3% ` [PATCH v6 1/6] eventdev: introduce " pbhagavatula
@ 2024-10-05 7:59 3% ` pbhagavatula
2024-10-05 7:59 3% ` [PATCH v7 1/6] eventdev: introduce " pbhagavatula
2024-10-06 17:06 3% ` [PATCH v8 0/6] Introduce " pbhagavatula
1 sibling, 2 replies; 200+ results
From: pbhagavatula @ 2024-10-05 7:59 UTC (permalink / raw)
To: jerinj, sthotton, abdullah.sevincer, hemant.agrawal,
sachin.saxena, harry.van.haaren, mattias.ronnblom, liangma,
peter.mccarthy, stephen
Cc: dev, Pavan Nikhilesh
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
Event pre-scheduling improves scheduling performance by assigning events
to event ports in advance when dequeues are issued.
This series introduces various types and levels of pre-scheduling to the
eventdev library.
pre-scheduling Types:
* RTE_EVENT_PRESCHEDULE_NONE: No pre-scheduling.
* RTE_EVENT_PRESCHEDULE: Always issue a pre-schedule when dequeue is issued.
* RTE_EVENT_PRESCHEDULE_ADAPTIVE: Delay issuing pre-schedule until there
are no forward progress constraints with the held flow contexts.
pre-scheduling Levels:
* Event Device Level Pre-scheduling: Pre-scheduling can be enabled or disabled at the
event device during configuration. Event devices can indicate pre-scheduling
capabilities using `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event device info
function `info.event_dev_cap`.
* Event Port Level Prefetch: Pre-scheduling can be selectively enabled or disabled
at the event port during runtime. Event devices can indicate this capability
using `RTE_EVENT_PORT_CAP_PER_PORT_PRESCHEDULE` via the event device info
function `info.event_port_cap`.
* Application Controlled Prefetch Hint: Applications can provide hints to the
event device to start pre-scheduling events using the new API
`rte_event_port_pre-schedule`. Event devices can indicate this capabilities using
`RTE_EVENT_DEV_CAP_PRESCHEDULE_EXPLICIT` via the event device info function
`info.event_dev_cap`.
The rte_event_dequeue_burst operation initiates the pre-schedule operation, which
completes in parallel without affecting the flow context of the dequeued events and
dequeue latency. On the next dequeue operation, the pre-scheduleed events are
dequeued, and pre-schedule operation is initiated again.
In the case of application-controlled pre-schedule hints, the currently held flow
contexts, if any, are not affected by the pre-schedule operation. On the next
dequeue operation, the pre-scheduleed events are returned, but pre-schedule is not
initiated again until the application provides the hint again. If pre-scheduling
is already enabled at the event device level or event port level, the hint is ignored.
v2 Changes:
- s/prefetch/pre-schedule (Mattias).
v3 Changes:
- Add CNXK preschedule implementation.
- Update test-eventdev to use prescheduling.
- Update examples to use prescheduling.
v4 Changes:
- Fix compilation.
v5 Changes:
- Update ABI changes
- s/RTE_EVENT_DEV_PRESCHEDULE/RTE_EVENT_PRESCHEDULE/
- s/RTE_EVENT_DEV_CAP_SW_PRESCHEDULE/RTE_EVENT_DEV_CAP_PRESCHEDULE_EXPLICIT/
- s/RTE_EVENT_PORT_CAP_EVENT_PER_PORT_PRESCHEDULE/RTE_EVENT_PORT_CAP_PER_PORT_PRESCHEDULE
- Various documentation fixes and updates.
v6 Changes:
- Mark new APIs as experimental (Stephen).
v7 Changes:
- Use correct patchset for rebase.
Pavan Nikhilesh (6):
eventdev: introduce event pre-scheduling
eventdev: add event port pre-schedule modify
eventdev: add event preschedule hint
event/cnkx: add pre-schedule support
app/test-eventdev: add pre-scheduling support
examples: use eventdev pre-scheduling
app/test-eventdev/evt_common.h | 45 +++--
app/test-eventdev/evt_options.c | 17 ++
app/test-eventdev/evt_options.h | 1 +
app/test/test_eventdev.c | 143 ++++++++++++++++
doc/guides/eventdevs/cnxk.rst | 10 --
doc/guides/eventdevs/features/cnxk.ini | 1 +
doc/guides/eventdevs/features/default.ini | 1 +
doc/guides/prog_guide/eventdev/eventdev.rst | 48 ++++++
doc/guides/rel_notes/release_24_11.rst | 13 ++
doc/guides/tools/testeventdev.rst | 6 +
drivers/event/cnxk/cn10k_eventdev.c | 19 ++-
drivers/event/cnxk/cn10k_worker.c | 21 +++
drivers/event/cnxk/cn10k_worker.h | 2 +
drivers/event/cnxk/cnxk_eventdev.c | 2 -
drivers/event/cnxk/cnxk_eventdev.h | 1 -
.../pipeline_worker_generic.c | 6 +
.../eventdev_pipeline/pipeline_worker_tx.c | 6 +
examples/ipsec-secgw/event_helper.c | 6 +
examples/l2fwd-event/l2fwd_event_generic.c | 6 +
.../l2fwd-event/l2fwd_event_internal_port.c | 6 +
examples/l3fwd/l3fwd_event_generic.c | 6 +
examples/l3fwd/l3fwd_event_internal_port.c | 6 +
lib/eventdev/eventdev_pmd.h | 4 +
lib/eventdev/eventdev_private.c | 41 ++++-
lib/eventdev/eventdev_trace_points.c | 6 +
lib/eventdev/rte_eventdev.h | 159 ++++++++++++++++++
lib/eventdev/rte_eventdev_core.h | 14 +-
lib/eventdev/rte_eventdev_trace_fp.h | 19 ++-
lib/eventdev/version.map | 6 +
29 files changed, 592 insertions(+), 29 deletions(-)
--
2.25.1
^ permalink raw reply [relevance 3%]
* [PATCH v6 1/6] eventdev: introduce event pre-scheduling
2024-10-05 7:25 3% ` [PATCH v6 " pbhagavatula
@ 2024-10-05 7:25 3% ` pbhagavatula
2024-10-05 7:59 3% ` [PATCH v7 0/6] Introduce " pbhagavatula
1 sibling, 0 replies; 200+ results
From: pbhagavatula @ 2024-10-05 7:25 UTC (permalink / raw)
To: jerinj, sthotton, abdullah.sevincer, hemant.agrawal,
sachin.saxena, harry.van.haaren, mattias.ronnblom, liangma,
peter.mccarthy, stephen
Cc: dev, Pavan Nikhilesh
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
Event pre-scheduling improves scheduling performance by assigning events
to event ports in advance when dequeues are issued.
The dequeue operation initiates the pre-schedule operation, which completes
in parallel without affecting the dequeued event flow contexts and
dequeue latency.
Event devices can indicate pre-scheduling capabilities using
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event device info
function `info.event_dev_cap`.
Applications can select the pre-schedule type and configure it through
`rte_event_dev_config.preschedule_type` during `rte_event_dev_configure`.
The supported pre-schedule types are:
* `RTE_EVENT_PRESCHEDULE_NONE` - No pre-scheduling.
* `RTE_EVENT_PRESCHEDULE` - Always issue a pre-schedule on dequeue.
* `RTE_EVENT_PRESCHEDULE_ADAPTIVE` - Delay issuing pre-schedule until
there are no forward progress constraints with the held flow contexts.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
app/test/test_eventdev.c | 108 ++++++++++++++++++++
doc/guides/eventdevs/features/default.ini | 1 +
doc/guides/prog_guide/eventdev/eventdev.rst | 23 +++++
doc/guides/rel_notes/release_24_11.rst | 10 ++
lib/eventdev/rte_eventdev.h | 49 +++++++++
5 files changed, 191 insertions(+)
diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c
index 9a6c8f470c..a45d1396d7 100644
--- a/app/test/test_eventdev.c
+++ b/app/test/test_eventdev.c
@@ -1251,6 +1251,112 @@ test_eventdev_profile_switch(void)
return TEST_SUCCESS;
}
+static int
+preschedule_test(enum rte_event_dev_preschedule_type preschedule_type, const char *preschedule_name)
+{
+#define NB_EVENTS 1024
+ uint64_t start, total;
+ struct rte_event ev;
+ int rc, cnt;
+
+ ev.event_type = RTE_EVENT_TYPE_CPU;
+ ev.queue_id = 0;
+ ev.op = RTE_EVENT_OP_NEW;
+ ev.u64 = 0xBADF00D0;
+
+ for (cnt = 0; cnt < NB_EVENTS; cnt++) {
+ ev.flow_id = cnt;
+ rc = rte_event_enqueue_burst(TEST_DEV_ID, 0, &ev, 1);
+ TEST_ASSERT(rc == 1, "Failed to enqueue event");
+ }
+
+ RTE_SET_USED(preschedule_type);
+ total = 0;
+ while (cnt) {
+ start = rte_rdtsc_precise();
+ rc = rte_event_dequeue_burst(TEST_DEV_ID, 0, &ev, 1, 0);
+ if (rc) {
+ total += rte_rdtsc_precise() - start;
+ cnt--;
+ }
+ }
+ printf("Preschedule type : %s, avg cycles %" PRIu64 "\n", preschedule_name,
+ total / NB_EVENTS);
+
+ return TEST_SUCCESS;
+}
+
+static int
+preschedule_configure(enum rte_event_dev_preschedule_type type, struct rte_event_dev_info *info)
+{
+ struct rte_event_dev_config dev_conf;
+ struct rte_event_queue_conf qcfg;
+ struct rte_event_port_conf pcfg;
+ int rc;
+
+ devconf_set_default_sane_values(&dev_conf, info);
+ dev_conf.nb_event_ports = 1;
+ dev_conf.nb_event_queues = 1;
+ dev_conf.preschedule_type = type;
+
+ rc = rte_event_dev_configure(TEST_DEV_ID, &dev_conf);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+
+ rc = rte_event_port_default_conf_get(TEST_DEV_ID, 0, &pcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to get port0 default config");
+ rc = rte_event_port_setup(TEST_DEV_ID, 0, &pcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to setup port0");
+
+ rc = rte_event_queue_default_conf_get(TEST_DEV_ID, 0, &qcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to get queue0 default config");
+ rc = rte_event_queue_setup(TEST_DEV_ID, 0, &qcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to setup queue0");
+
+ rc = rte_event_port_link(TEST_DEV_ID, 0, NULL, NULL, 0);
+ TEST_ASSERT(rc == (int)dev_conf.nb_event_queues, "Failed to link port, device %d",
+ TEST_DEV_ID);
+
+ rc = rte_event_dev_start(TEST_DEV_ID);
+ TEST_ASSERT_SUCCESS(rc, "Failed to start event device");
+
+ return 0;
+}
+
+static int
+test_eventdev_preschedule_configure(void)
+{
+ struct rte_event_dev_info info;
+ int rc;
+
+ rte_event_dev_info_get(TEST_DEV_ID, &info);
+
+ if ((info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE) == 0)
+ return TEST_SKIPPED;
+
+ rc = preschedule_configure(RTE_EVENT_PRESCHEDULE_NONE, &info);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+ rc = preschedule_test(RTE_EVENT_PRESCHEDULE_NONE, "RTE_EVENT_PRESCHEDULE_NONE");
+ TEST_ASSERT_SUCCESS(rc, "Failed to test preschedule RTE_EVENT_PRESCHEDULE_NONE");
+
+ rte_event_dev_stop(TEST_DEV_ID);
+ rc = preschedule_configure(RTE_EVENT_PRESCHEDULE, &info);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+ rc = preschedule_test(RTE_EVENT_PRESCHEDULE, "RTE_EVENT_PRESCHEDULE");
+ TEST_ASSERT_SUCCESS(rc, "Failed to test preschedule RTE_EVENT_PRESCHEDULE");
+
+ if (info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE) {
+ rte_event_dev_stop(TEST_DEV_ID);
+ rc = preschedule_configure(RTE_EVENT_PRESCHEDULE_ADAPTIVE, &info);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+ rc = preschedule_test(RTE_EVENT_PRESCHEDULE_ADAPTIVE,
+ "RTE_EVENT_PRESCHEDULE_ADAPTIVE");
+ TEST_ASSERT_SUCCESS(rc,
+ "Failed to test preschedule RTE_EVENT_PRESCHEDULE_ADAPTIVE");
+ }
+
+ return TEST_SUCCESS;
+}
+
static int
test_eventdev_close(void)
{
@@ -1311,6 +1417,8 @@ static struct unit_test_suite eventdev_common_testsuite = {
test_eventdev_start_stop),
TEST_CASE_ST(eventdev_configure_setup, eventdev_stop_device,
test_eventdev_profile_switch),
+ TEST_CASE_ST(eventdev_configure_setup, NULL,
+ test_eventdev_preschedule_configure),
TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
test_eventdev_link),
TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
diff --git a/doc/guides/eventdevs/features/default.ini b/doc/guides/eventdevs/features/default.ini
index 1cc4303fe5..c8d5ed2d74 100644
--- a/doc/guides/eventdevs/features/default.ini
+++ b/doc/guides/eventdevs/features/default.ini
@@ -22,6 +22,7 @@ carry_flow_id =
maintenance_free =
runtime_queue_attr =
profile_links =
+preschedule =
;
; Features of a default Ethernet Rx adapter.
diff --git a/doc/guides/prog_guide/eventdev/eventdev.rst b/doc/guides/prog_guide/eventdev/eventdev.rst
index fb6dfce102..9da5531859 100644
--- a/doc/guides/prog_guide/eventdev/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev/eventdev.rst
@@ -357,6 +357,29 @@ Worker path:
// Process the event received.
}
+Event Pre-scheduling
+~~~~~~~~~~~~~~~~~~~~
+
+Event pre-scheduling improves scheduling performance by assigning events to event ports in advance
+when dequeues are issued.
+The `rte_event_dequeue_burst` operation initiates the pre-schedule operation, which completes
+in parallel without affecting the dequeued event flow contexts and dequeue latency.
+On the next dequeue operation, the pre-scheduled events are dequeued and pre-schedule is initiated
+again.
+
+An application can use event pre-scheduling if the event device supports it at either device
+level or at a individual port level.
+The application must check pre-schedule capability by checking if ``rte_event_dev_info.event_dev_cap``
+has the bit ``RTE_EVENT_DEV_CAP_PRESCHEDULE`` or ``RTE_EVENT_DEV_CAP_PRESCHEDULE_ADAPTIVE`` set, if
+present pre-scheduling can be enabled at device
+configuration time by setting appropriate pre-schedule type in ``rte_event_dev_config.preschedule``.
+
+The following pre-schedule types are supported:
+ * ``RTE_EVENT_PRESCHEDULE_NONE`` - No pre-scheduling.
+ * ``RTE_EVENT_PRESCHEDULE`` - Always issue a pre-schedule when dequeue is issued.
+ * ``RTE_EVENT_PRESCHEDULE_ADAPTIVE`` - Issue pre-schedule when dequeue is issued and there are
+ no forward progress constraints.
+
Starting the EventDev
~~~~~~~~~~~~~~~~~~~~~
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index e0a9aa55a1..a294a753e7 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -67,6 +67,14 @@ New Features
The new statistics are useful for debugging and profiling.
+* **Added event device pre-scheduling support.**
+
+ Added support for pre-scheduling of events to event ports to improve
+ scheduling performance and latency.
+
+ * Added ``rte_event_dev_config::preschedule_type`` to configure the device
+ level pre-scheduling type.
+
Removed Items
-------------
@@ -112,6 +120,8 @@ ABI Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* eventdev: Added ``preschedule_type`` field to ``rte_event_dev_config``
+ structure.
Known Issues
------------
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 08e5f9320b..4b69e74577 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -446,6 +446,30 @@ struct rte_event;
* @see RTE_SCHED_TYPE_PARALLEL
*/
+#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE (1ULL << 16)
+/**< Event device supports event pre-scheduling.
+ *
+ * When this capability is available, the application can enable event pre-scheduling on the event
+ * device to pre-schedule events to a event port when `rte_event_dequeue_burst()`
+ * is issued.
+ * The pre-schedule process starts with the `rte_event_dequeue_burst()` call and the
+ * pre-scheduled events are returned on the next `rte_event_dequeue_burst()` call.
+ *
+ * @see rte_event_dev_configure()
+ */
+
+#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE (1ULL << 17)
+/**< Event device supports adaptive event pre-scheduling.
+ *
+ * When this capability is available, the application can enable adaptive pre-scheduling
+ * on the event device where the events are pre-scheduled when there are no forward
+ * progress constraints with the currently held flow contexts.
+ * The pre-schedule process starts with the `rte_event_dequeue_burst()` call and the
+ * pre-scheduled events are returned on the next `rte_event_dequeue_burst()` call.
+ *
+ * @see rte_event_dev_configure()
+ */
+
/* Event device priority levels */
#define RTE_EVENT_DEV_PRIORITY_HIGHEST 0
/**< Highest priority level for events and queues.
@@ -680,6 +704,26 @@ rte_event_dev_attr_get(uint8_t dev_id, uint32_t attr_id,
* @see rte_event_dequeue_timeout_ticks(), rte_event_dequeue_burst()
*/
+/** Event device pre-schedule type enumeration. */
+enum rte_event_dev_preschedule_type {
+ RTE_EVENT_PRESCHEDULE_NONE,
+ /**< Disable pre-schedule across the event device or on a given event port.
+ * @ref rte_event_dev_config.preschedule_type
+ */
+ RTE_EVENT_PRESCHEDULE,
+ /**< Enable pre-schedule always across the event device or a given event port.
+ * @ref rte_event_dev_config.preschedule_type
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
+ */
+ RTE_EVENT_PRESCHEDULE_ADAPTIVE,
+ /**< Enable adaptive pre-schedule across the event device or a given event port.
+ * Delay issuing pre-schedule until there are no forward progress constraints with
+ * the held flow contexts.
+ * @ref rte_event_dev_config.preschedule_type
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
+ */
+};
+
/** Event device configuration structure */
struct rte_event_dev_config {
uint32_t dequeue_timeout_ns;
@@ -752,6 +796,11 @@ struct rte_event_dev_config {
* optimized for single-link usage, this field is a hint for how many
* to allocate; otherwise, regular event ports and queues will be used.
*/
+ enum rte_event_dev_preschedule_type preschedule_type;
+ /**< Event pre-schedule type to use across the event device, if supported.
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
+ */
};
/**
--
2.25.1
^ permalink raw reply [relevance 3%]
* [PATCH v6 0/6] Introduce event pre-scheduling
2024-10-04 16:24 3% ` [PATCH v5 0/6] Introduce event pre-scheduling pbhagavatula
2024-10-04 16:24 3% ` [PATCH v5 1/6] eventdev: introduce " pbhagavatula
2024-10-04 16:35 0% ` [PATCH v5 0/6] Introduce " Stephen Hemminger
@ 2024-10-05 7:25 3% ` pbhagavatula
2024-10-05 7:25 3% ` [PATCH v6 1/6] eventdev: introduce " pbhagavatula
2024-10-05 7:59 3% ` [PATCH v7 0/6] Introduce " pbhagavatula
2 siblings, 2 replies; 200+ results
From: pbhagavatula @ 2024-10-05 7:25 UTC (permalink / raw)
To: jerinj, sthotton, abdullah.sevincer, hemant.agrawal,
sachin.saxena, harry.van.haaren, mattias.ronnblom, liangma,
peter.mccarthy, stephen
Cc: dev, Pavan Nikhilesh
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
Event pre-scheduling improves scheduling performance by assigning events
to event ports in advance when dequeues are issued.
This series introduces various types and levels of pre-scheduling to the
eventdev library.
pre-scheduling Types:
* RTE_EVENT_PRESCHEDULE_NONE: No pre-scheduling.
* RTE_EVENT_PRESCHEDULE: Always issue a pre-schedule when dequeue is issued.
* RTE_EVENT_PRESCHEDULE_ADAPTIVE: Delay issuing pre-schedule until there
are no forward progress constraints with the held flow contexts.
pre-scheduling Levels:
* Event Device Level Pre-scheduling: Pre-scheduling can be enabled or disabled at the
event device during configuration. Event devices can indicate pre-scheduling
capabilities using `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event device info
function `info.event_dev_cap`.
* Event Port Level Prefetch: Pre-scheduling can be selectively enabled or disabled
at the event port during runtime. Event devices can indicate this capability
using `RTE_EVENT_PORT_CAP_PER_PORT_PRESCHEDULE` via the event device info
function `info.event_port_cap`.
* Application Controlled Prefetch Hint: Applications can provide hints to the
event device to start pre-scheduling events using the new API
`rte_event_port_pre-schedule`. Event devices can indicate this capabilities using
`RTE_EVENT_DEV_CAP_PRESCHEDULE_EXPLICIT` via the event device info function
`info.event_dev_cap`.
The rte_event_dequeue_burst operation initiates the pre-schedule operation, which
completes in parallel without affecting the flow context of the dequeued events and
dequeue latency. On the next dequeue operation, the pre-scheduleed events are
dequeued, and pre-schedule operation is initiated again.
In the case of application-controlled pre-schedule hints, the currently held flow
contexts, if any, are not affected by the pre-schedule operation. On the next
dequeue operation, the pre-scheduleed events are returned, but pre-schedule is not
initiated again until the application provides the hint again. If pre-scheduling
is already enabled at the event device level or event port level, the hint is ignored.
v2 Changes:
- s/prefetch/pre-schedule (Mattias).
v3 Changes:
- Add CNXK preschedule implementation.
- Update test-eventdev to use prescheduling.
- Update examples to use prescheduling.
v4 Changes:
- Fix compilation.
v5 Changes:
- Update ABI changes
- s/RTE_EVENT_DEV_PRESCHEDULE/RTE_EVENT_PRESCHEDULE/
- s/RTE_EVENT_DEV_CAP_SW_PRESCHEDULE/RTE_EVENT_DEV_CAP_PRESCHEDULE_EXPLICIT/
- s/RTE_EVENT_PORT_CAP_EVENT_PER_PORT_PRESCHEDULE/RTE_EVENT_PORT_CAP_PER_PORT_PRESCHEDULE
- Various documentation fixes and updates.
v6 Changes:
- Mark new APIs as experimental (Stephen).
Pavan Nikhilesh (6):
eventdev: introduce event pre-scheduling
eventdev: add event port pre-schedule modify
eventdev: add event preschedule hint
event/cnkx: add pre-schedule support
app/test-eventdev: add pre-scheduling support
examples: use eventdev pre-scheduling
app/test-eventdev/evt_common.h | 45 +++--
app/test-eventdev/evt_options.c | 17 ++
app/test-eventdev/evt_options.h | 1 +
app/test/test_eventdev.c | 143 ++++++++++++++++
doc/guides/eventdevs/cnxk.rst | 10 --
doc/guides/eventdevs/features/cnxk.ini | 1 +
doc/guides/eventdevs/features/default.ini | 1 +
doc/guides/prog_guide/eventdev/eventdev.rst | 48 ++++++
doc/guides/rel_notes/release_24_11.rst | 13 ++
doc/guides/tools/testeventdev.rst | 6 +
drivers/event/cnxk/cn10k_eventdev.c | 19 ++-
drivers/event/cnxk/cn10k_worker.c | 21 +++
drivers/event/cnxk/cn10k_worker.h | 2 +
drivers/event/cnxk/cnxk_eventdev.c | 2 -
drivers/event/cnxk/cnxk_eventdev.h | 1 -
.../pipeline_worker_generic.c | 6 +
.../eventdev_pipeline/pipeline_worker_tx.c | 6 +
examples/ipsec-secgw/event_helper.c | 6 +
examples/l2fwd-event/l2fwd_event_generic.c | 6 +
.../l2fwd-event/l2fwd_event_internal_port.c | 6 +
examples/l3fwd/l3fwd_event_generic.c | 6 +
examples/l3fwd/l3fwd_event_internal_port.c | 6 +
lib/eventdev/eventdev_pmd.h | 4 +
lib/eventdev/eventdev_private.c | 41 ++++-
lib/eventdev/eventdev_trace_points.c | 6 +
lib/eventdev/rte_eventdev.h | 159 ++++++++++++++++++
lib/eventdev/rte_eventdev_core.h | 14 +-
lib/eventdev/rte_eventdev_trace_fp.h | 19 ++-
lib/eventdev/version.map | 6 +
29 files changed, 592 insertions(+), 29 deletions(-)
--
2.25.1
^ permalink raw reply [relevance 3%]
* RE: [RFC v2] ethdev: an API for cache stashing hints
2024-10-04 17:52 4% ` Stephen Hemminger
@ 2024-10-04 18:46 0% ` Wathsala Wathawana Vithanage
0 siblings, 0 replies; 200+ results
From: Wathsala Wathawana Vithanage @ 2024-10-04 18:46 UTC (permalink / raw)
To: Stephen Hemminger, Ferruh Yigit
Cc: dev, thomas, Andrew Rybchenko, nd, Dhruv Tripathi,
Honnappa Nagarahalli, Varghese, Vipin, nd
> > >>
> > >> And I can see set() has two different APIs,
> > >> 'rte_eth_dev_stashing_hints_rx' & 'rte_eth_dev_stashing_hints_tx',
> > >> is there a reason to have two separate APIs instead of having one
> > >> which gets RX & TX as argument, as done in internal device ops?
> > >
> > > Some types/hints may only apply to a single queue direction, so I
> > > thought it would be better to separate them out into separate Rx and
> > > Tx APIs for ease of comprehension/use for the developer.
> > > In fact, underneath, it uses one API for both Rx and Tx.
> > >
> >
> > Hi Wathsala,
> >
> > Do you still pursue this RFC, should we expect a new version for this
> > release?
> >
> > Did you have any change to measure the impact of the changes in this patch?
> >
> >
> > Btw, do you think the LLC aware lcore selection patch [1] can be
> > relevant or can it help for the cases this patch addresses?
>
>
> Don't think this is ready for 24.11 release. The patch fails multiple tests, has
> some doc issues and would need more exmaples and support.
>
> Since it requires an ABI change, if you want to pursue it further send a new
> version and maybe it will be ready for next ABI breaking release 25.11.
I agree, I don't think it will be ready for 24.11 release. I will be sending out another version of this
in the coming days.
As we discussed in the summit, this also requires some work in the kernel to make it functional.
^ permalink raw reply [relevance 0%]
* Re: [RFC v2] ethdev: an API for cache stashing hints
@ 2024-10-04 17:52 4% ` Stephen Hemminger
2024-10-04 18:46 0% ` Wathsala Wathawana Vithanage
0 siblings, 1 reply; 200+ results
From: Stephen Hemminger @ 2024-10-04 17:52 UTC (permalink / raw)
To: Ferruh Yigit
Cc: Wathsala Wathawana Vithanage, dev, thomas, Andrew Rybchenko, nd,
Dhruv Tripathi, Honnappa Nagarahalli, Varghese, Vipin
On Sun, 22 Sep 2024 22:43:55 +0100
Ferruh Yigit <ferruh.yigit@amd.com> wrote:
> >>
> >> And I can see set() has two different APIs, 'rte_eth_dev_stashing_hints_rx' &
> >> 'rte_eth_dev_stashing_hints_tx', is there a reason to have two separate APIs
> >> instead of having one which gets RX & TX as argument, as done in internal
> >> device ops?
> >
> > Some types/hints may only apply to a single queue direction, so I thought it
> > would be better to separate them out into separate Rx and Tx APIs for ease
> > of comprehension/use for the developer.
> > In fact, underneath, it uses one API for both Rx and Tx.
> >
>
> Hi Wathsala,
>
> Do you still pursue this RFC, should we expect a new version for this
> release?
>
> Did you have any change to measure the impact of the changes in this patch?
>
>
> Btw, do you think the LLC aware lcore selection patch [1] can be
> relevant or can it help for the cases this patch addresses?
Don't think this is ready for 24.11 release. The patch fails multiple
tests, has some doc issues and would need more exmaples and support.
Since it requires an ABI change, if you want to pursue it further
send a new version and maybe it will be ready for next ABI breaking release 25.11.
^ permalink raw reply [relevance 4%]
* RE: [EXTERNAL] Re: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-10-04 9:38 5% ` Dodji Seketeli
@ 2024-10-04 17:45 4% ` Akhil Goyal
0 siblings, 0 replies; 200+ results
From: Akhil Goyal @ 2024-10-04 17:45 UTC (permalink / raw)
To: Dodji Seketeli, Ferruh Yigit
Cc: dev, thomas, david.marchand, hemant.agrawal, Anoob Joseph,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde,
Nagadheeraj Rottela, mdr
> Hello,
>
> Ferruh Yigit <ferruh.yigit@amd.com> writes:
>
> > On 9/5/2024 11:14 AM, Akhil Goyal wrote:
> >> Replace *_LIST_END enumerators from asymmetric crypto
> >> lib to avoid ABI breakage for every new addition in
> >> enums with inline APIs.
> >>
> >> Signed-off-by: Akhil Goyal <gakhil@marvell.com>
> >> ---
> >> This patch was discussed in ML long time back.
> >> https://patches.dpdk.org/project/dpdk/patch/20211008204516.3497060-1-gakhil@marvell.com/
> >> Now added inline APIs for getting the list end which need to be updated
> >> for each new entry to the enum. This shall help in avoiding ABI break
> >> for adding new algo.
> >>
> >
> > Hi Akhil,
> >
> > *I think* this hides the problem instead of fixing it, and this may be
> > partially because of the tooling (libabigail) limitation.
>
> So, I am not sure to understand how this would be a libabigail
> limitation in general. Let me explain.
>
> The abidiff tool that compares two binaries and emits reports about
> their ABI changes can be given a suppression specification file which
> instructs the tool about what type of change to avoid emitting reports
> about.
>
> It turns out there is a construct specifically designed to handle the
> case where an enumerator is added right before the last enumerator of a
> enum. Such a change causes the value of the last enumerator to
> increase.
>
> For instance, consider this enum:
>
> enum foo_enum
> {
> FOO_FIRST,
> FOO_SECOND,
> FOO_END
> };
>
> If I add a new enumerator right before the last enumerator, I would get
> this:
>
> enum foo_enum
> {
> FOO_FIRST,
> FOO_SECOND,
> FOO_THIRD,
> FOO_END
> };
>
> This change cause the value of the the FOOD_END enumerator to increase.
> And that increase might be problematic. At the moment, for it being
> problematic or not has to be the result of a careful review.
>
> So, by default, abidiff will complain by saying that the value of
> FOO_END was changed.
>
> But you, as a community of practice, can decide that this kind of change
> to the value of the last enumerator is not a problematic change, after
> careful review of your code and practice. You thus can specify that
> the tool using a suppression specification which has the following
> syntax:
>
> [suppress_type]
> type_kind = enum
> changed_enumerators = FOO_END, ANOTHER_ENUM_END,
> AND_ANOTHER_ENUM_END
>
> or, alternatively, you can specify the enumerators you want to suppress
> the changes for as a list of regular expressions:
>
> [suppress_type]
> type_kind = enum
> changed_enumerators_regexp = .*_END$, .*_LAST$, .*_LIST_END$
>
> Wouldn't that be enough to address your use case here (honest question)?
>
> > This patch prevents the libabigail warning, true, but it doesn't improve
> > anything from the application's perspective.
> > Before or after this patch, application knows a fixed value as END value.
> >
> > Not all changes in the END (MAX) enum values cause ABI break, but tool
> > warns on all, that is why I think this may be tooling limitation [1].
> > (Just to stress, I am NOT talking about regular enum values change, I am
> > talking about only END (MAX) value changes caused by appending new enum
> > items.)
> >
> > As far as I can see (please Dodji, David correct me if I am wrong) ABI
> > break only happens if application and library exchange enum values in
> > the API (directly or within a struct).
>
> Sorry, I am not sure to understand what you mean by "exchange enum values".
>
> I would say is that if a change to an enumerator value causes a change
> to the layout of a type which is part of the ABI, then that enumerator
> value change might change the ABI. That ABI change might be problematic
> (i.e an ABI break) or not.
>
> >
> > Exchanging enum values via API cause ABI issues because:
> > 1. enum size is not fixed, it uses min storage size that can hold all
> > values, adding new enums may cause its size change and of course this
> > breaks ABI.
> > 2. Library can send enum values that application doesn't know, this may
> > cause application behave undefined.
> > 3. Application sending MAX value (or more) to API expects error, but
> > that may be a valid value in the new library and applications gets
> > unexpected result.
> >
> >
> > Let's assume above 1. happens, with this patch warning is prevented, but
> > ABI break still can occur.
> >
> > One option can be not exchanging enums in APIs at all, this way changes
> > in the END (MAX) enum values are harmless. But I can see cryptodev does
> > this a lot.
> >
> > When enum is exchanged in APIs, and if we want to be strict about the
> > ABI, safest option can be not appending to enums at all, and keeping END
> > (MAX) items can be a way to enable warnings for this.
> >
> >
> > More relaxed option is protect against only 1. since that is the real
> > ABI issue, 2 & 3 are more defensive programming, so as long as enum size
> > is not changed we may ignore the END (MAX) value change warnings.
> >
> >
> >
> > [1] It would be better if tool gives END (MAX) enum value warnings only
> > if it is exchanged in an API, but not sure if this can be possible to
> > detect.
>
> I believe that if you want to know if an enumerator value is *USED* by a
> type (which I believe is at the root of what you are alluding to), then
> you would need a static analysis tool that works at the source level.
> Or, you need a human review of the code once the binary analysis tool
> told you that that value of the enumerator changed.
>
> Why ? please let me give you an example:
>
> enum foo_enum
> {
> FOO_FIRST,
> FOO_SECOND,
> FOO_END
> };
>
> int array[FOO_END];
>
> Once this is compiled into binary, what libabigail is going to see by
> analyzing the binary is that 'array' is an array of 2 integers. The
> information about the size of the array being initially an enumerator
> value is lost. To detect that, you need source level analysis.
>
> But then, by reviewing the code, this is a construct that you can spot
> and allow or forbid, depending on your goals as a project.
>
In the above example if in newer library a FOO_THIRD is added.
FOO_END value will change and will cause ABI break for change in existing value.
But if we replace it with inline function to get the list_end and use it in array like below.
So if FOO_THIRD is added, we will also update foo_enum_list_end() function to return (FOO_THIRD+1)
enum foo_enum
{
FOO_FIRST,
FOO_SECOND,
};
static inline int foo_enum_list_end()
{
return FOO_SECOND + 1;
}
int array[foo_enum_list_end()];
Will this cause an ABI break if we add this array in application or in library?
^ permalink raw reply [relevance 4%]
* RE: [EXTERNAL] Re: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-10-04 7:04 0% ` David Marchand
@ 2024-10-04 17:27 0% ` Akhil Goyal
0 siblings, 0 replies; 200+ results
From: Akhil Goyal @ 2024-10-04 17:27 UTC (permalink / raw)
To: David Marchand, Ferruh Yigit, Morten Brørup
Cc: dev, Dodji Seketeli, thomas, hemant.agrawal, Anoob Joseph,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde,
Nagadheeraj Rottela, mdr
> On Fri, Oct 4, 2024 at 5:55 AM Ferruh Yigit <ferruh.yigit@amd.com> wrote:
> >
> > On 9/5/2024 11:14 AM, Akhil Goyal wrote:
> > > Replace *_LIST_END enumerators from asymmetric crypto
> > > lib to avoid ABI breakage for every new addition in
> > > enums with inline APIs.
> > >
> > > Signed-off-by: Akhil Goyal <gakhil@marvell.com>
> > > ---
> > > This patch was discussed in ML long time back.
> > > https://patches.dpdk.org/project/dpdk/patch/20211008204516.3497060-1-gakhil@marvell.com/
> > > Now added inline APIs for getting the list end which need to be updated
> > > for each new entry to the enum. This shall help in avoiding ABI break
> > > for adding new algo.
> > >
> >
> > Hi Akhil,
> >
> > *I think* this hides the problem instead of fixing it, and this may be
> > partially because of the tooling (libabigail) limitation.
> >
> > This patch prevents the libabigail warning, true, but it doesn't improve
> > anything from the application's perspective.
> > Before or after this patch, application knows a fixed value as END value.
> >
> > Not all changes in the END (MAX) enum values cause ABI break, but tool
> > warns on all, that is why I think this may be tooling limitation [1].
> > (Just to stress, I am NOT talking about regular enum values change, I am
> > talking about only END (MAX) value changes caused by appending new enum
> > items.)
> >
> > As far as I can see (please Dodji, David correct me if I am wrong) ABI
> > break only happens if application and library exchange enum values in
> > the API (directly or within a struct).
>
> - There is also the following issue:
> A library publicly exports an array sized against a END (MAX) enum in the API.
> https://developers.redhat.com/blog/2019/05/06/how-c-array-sizes-become-part-of-the-binary-interface-of-a-library
>
> I had made comments for an issue in the cryptodev library in the past:
> https://inbox.dpdk.org/dev/CAJFAV8xs5CVdE2xwRtaxk5vE_PiQMV5LY5tKStk3R1gOuRTsUw@mail.gmail.com/
>
Yes this issue is being discussed multiple times.
One discussion is mentioned in patch description also.
There haven’t been a consensus yet.
>
> - Sorry to deviate from the _END enum discussion that tries to define
> a solution for all cases, but all I see in the cryptodev patch is a
> need for an enumerator... for an internal unit test.
> From the RFC patch, I would simply change the
> rte_crypto_asym_xform_type_list_end helper into a non inline symbol
> and mark it __rte_internal, or move this helper to a non public header
> used by the unit test.
That means we are not allowing applications to use it. Right?
So then why not we remove these LIST_END completely.
Test app can manage without it as well.
This was the original RFC to remove all list end.
https://patches.dpdk.org/project/dpdk/patch/20211008204516.3497060-1-gakhil@marvell.com/
As per our last discussion we need to find the LIST_END using a function and not via MACRO or enum.
This RFC tries to do that with an inline function which will give the list end based on the last enum value
Which is defined in that header file.
The only thing is we have an overhead of updating that inline function with a different value on every new addition which are normally not very frequent.
>
> Or add a (internal) rte_crypto_asym_xform_type_next_op() used through
> a a RTE_CRYPTO_FOREACH_ASYM_OP() macro.
>
>
> --
> David Marchand
^ permalink raw reply [relevance 0%]
* RE: [EXTERNAL] Re: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-10-04 4:00 0% ` Ferruh Yigit
@ 2024-10-04 17:26 4% ` Akhil Goyal
0 siblings, 0 replies; 200+ results
From: Akhil Goyal @ 2024-10-04 17:26 UTC (permalink / raw)
To: Ferruh Yigit, Morten Brørup, fengchengwen, dev
Cc: thomas, david.marchand, hemant.agrawal, Anoob Joseph,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde,
Nagadheeraj Rottela, mdr
> > If we want to make it a generic thing in rte_common.h
> > Will the below change be ok?
> > ----------------------------------
> > Common header file (rte_common.h):
> > ----------------------------------
> > #define rte_define_enum_list_end(name, last_value) \
> > static inline int name ## _enum_list_end(void) \
> > { \
> > return last_value + 1; \
> > }
> >
> > ----------------
> > Lib header file
> > ----------------
> > //After the enum definition define the list end as below
> > rte_define_enum_list_end(rte_crypto_asym_op,
> RTE_CRYPTO_ASYM_OP_VERIFY);
> >
>
> I assume Morten suggests his macros to escape from maintenance cost of
> updating inline function each time a new enum is added.
Except for the maintenance cost, is there any other issue with this?
Will the ABI break for any new additions?
I believe this is a simple change to define and maintain LIST_END wherever required.
We can add a comment in the enum for asking people to update it for new additions.
The macros suggested by Morten seem a bit complex and
would add #if in the enum to compile it as internal.
Also we would need to expose "_ ## name ## _enum_list_end" function also in version.map
While the "name ## _enum_list_end" would be used by the application.
I would suggest to remove the list_end from crypto library instead of adding #if in enum.
Like it was suggested long back. Application can live without it.
https://patches.dpdk.org/project/dpdk/patch/20211008204516.3497060-1-gakhil@marvell.com/
The list_end were removed for all symmetric crypto enums when we worked on ABI stability.
But asymmetric was left as it was experimental at that time and we missed them as it was not complaining.
But now since it is marked as stable. We need to deal with it as soon as possible.
>
> But with above suggestion, that cost is still there, so I don't think
> this one has a benefit against the original suggestion.
Yes the idea is same for both.
It is just that this approach is generalized to be used anywhere.
^ permalink raw reply [relevance 4%]
* RE: [EXTERNAL] Re: [PATCH v5 0/6] Introduce event pre-scheduling
2024-10-04 16:35 0% ` [PATCH v5 0/6] Introduce " Stephen Hemminger
@ 2024-10-04 16:50 0% ` Pavan Nikhilesh Bhagavatula
0 siblings, 0 replies; 200+ results
From: Pavan Nikhilesh Bhagavatula @ 2024-10-04 16:50 UTC (permalink / raw)
To: Stephen Hemminger
Cc: Jerin Jacob, Shijith Thotton, abdullah.sevincer, hemant.agrawal,
sachin.saxena, harry.van.haaren, mattias.ronnblom, liangma,
peter.mccarthy, dev
> > From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> >
> > Event pre-scheduling improves scheduling performance by assigning events
> > to event ports in advance when dequeues are issued.
> > This series introduces various types and levels of pre-scheduling to the
> > eventdev library.
> >
> > pre-scheduling Types:
> > * RTE_EVENT_PRESCHEDULE_NONE: No pre-scheduling.
> > * RTE_EVENT_PRESCHEDULE: Always issue a pre-schedule when dequeue is
> issued.
> > * RTE_EVENT_PRESCHEDULE_ADAPTIVE: Delay issuing pre-schedule until
> there
> > are no forward progress constraints with the held flow contexts.
> >
> > pre-scheduling Levels:
> > * Event Device Level Pre-scheduling: Pre-scheduling can be enabled or
> disabled at the
> > event device during configuration. Event devices can indicate pre-
> scheduling
> > capabilities using `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
> > `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event
> device info
> > function `info.event_dev_cap`.
> > * Event Port Level Prefetch: Pre-scheduling can be selectively enabled or
> disabled
> > at the event port during runtime. Event devices can indicate this capability
> > using `RTE_EVENT_PORT_CAP_PER_PORT_PRESCHEDULE` via the event
> device info
> > function `info.event_port_cap`.
> > * Application Controlled Prefetch Hint: Applications can provide hints to the
> > event device to start pre-scheduling events using the new API
> > `rte_event_port_pre-schedule`. Event devices can indicate this capabilities
> using
> > `RTE_EVENT_DEV_CAP_PRESCHEDULE_EXPLICIT` via the event device info
> function
> > `info.event_dev_cap`.
> >
> > The rte_event_dequeue_burst operation initiates the pre-schedule
> operation, which
> > completes in parallel without affecting the flow context of the dequeued
> events and
> > dequeue latency. On the next dequeue operation, the pre-scheduleed events
> are
> > dequeued, and pre-schedule operation is initiated again.
> >
> > In the case of application-controlled pre-schedule hints, the currently held
> flow
> > contexts, if any, are not affected by the pre-schedule operation. On the next
> > dequeue operation, the pre-scheduleed events are returned, but pre-
> schedule is not
> > initiated again until the application provides the hint again. If pre-scheduling
> > is already enabled at the event device level or event port level, the hint is
> ignored.
> >
> > v2 Changes:
> > - s/prefetch/pre-schedule (Mattias).
> > v3 Changes:
> > - Add CNXK preschedule implementation.
> > - Update test-eventdev to use prescheduling.
> > - Update examples to use prescheduling.
> > v4 Changes:
> > - Fix compilation.
> > v5 Changes:
> > - Update ABI changes
> > - s/RTE_EVENT_DEV_PRESCHEDULE/RTE_EVENT_PRESCHEDULE/
> > -
> s/RTE_EVENT_DEV_CAP_SW_PRESCHEDULE/RTE_EVENT_DEV_CAP_PRESCHE
> DULE_EXPLICIT/
> > -
> s/RTE_EVENT_PORT_CAP_EVENT_PER_PORT_PRESCHEDULE/RTE_EVENT_PO
> RT_CAP_PER_PORT_PRESCHEDULE
> > - Various documentation fixes and updates.
> >
> >
> > Pavan Nikhilesh (6):
> > eventdev: introduce event pre-scheduling
> > eventdev: add event port pre-schedule modify
> > eventdev: add event preschedule hint
> > event/cnkx: add pre-schedule support
> > app/test-eventdev: add pre-scheduling support
> > examples: use eventdev pre-scheduling
> >
> > app/test-eventdev/evt_common.h | 45 +++--
> > app/test-eventdev/evt_options.c | 17 ++
> > app/test-eventdev/evt_options.h | 1 +
> > app/test/test_eventdev.c | 143 ++++++++++++++++
> > doc/guides/eventdevs/cnxk.rst | 10 --
> > doc/guides/eventdevs/features/cnxk.ini | 1 +
> > doc/guides/eventdevs/features/default.ini | 1 +
> > doc/guides/prog_guide/eventdev/eventdev.rst | 48 ++++++
> > doc/guides/rel_notes/release_24_11.rst | 13 ++
> > doc/guides/tools/testeventdev.rst | 6 +
> > drivers/event/cnxk/cn10k_eventdev.c | 19 ++-
> > drivers/event/cnxk/cn10k_worker.c | 21 +++
> > drivers/event/cnxk/cn10k_worker.h | 2 +
> > drivers/event/cnxk/cnxk_eventdev.c | 2 -
> > drivers/event/cnxk/cnxk_eventdev.h | 1 -
> > .../pipeline_worker_generic.c | 6 +
> > .../eventdev_pipeline/pipeline_worker_tx.c | 6 +
> > examples/ipsec-secgw/event_helper.c | 6 +
> > examples/l2fwd-event/l2fwd_event_generic.c | 6 +
> > .../l2fwd-event/l2fwd_event_internal_port.c | 6 +
> > examples/l3fwd/l3fwd_event_generic.c | 6 +
> > examples/l3fwd/l3fwd_event_internal_port.c | 6 +
> > lib/eventdev/eventdev_pmd.h | 4 +
> > lib/eventdev/eventdev_private.c | 41 ++++-
> > lib/eventdev/eventdev_trace_points.c | 6 +
> > lib/eventdev/rte_eventdev.h | 159 ++++++++++++++++++
> > lib/eventdev/rte_eventdev_core.h | 14 +-
> > lib/eventdev/rte_eventdev_trace_fp.h | 19 ++-
> > lib/eventdev/version.map | 6 +
> > 29 files changed, 592 insertions(+), 29 deletions(-)
>
> Looks good, but the new API's should be experimental right?
Ah yes, my bad, Thanks Stephen, I will mark them as experimental in next version.
^ permalink raw reply [relevance 0%]
* Re: [PATCH v5 0/6] Introduce event pre-scheduling
2024-10-04 16:24 3% ` [PATCH v5 0/6] Introduce event pre-scheduling pbhagavatula
2024-10-04 16:24 3% ` [PATCH v5 1/6] eventdev: introduce " pbhagavatula
@ 2024-10-04 16:35 0% ` Stephen Hemminger
2024-10-04 16:50 0% ` [EXTERNAL] " Pavan Nikhilesh Bhagavatula
2024-10-05 7:25 3% ` [PATCH v6 " pbhagavatula
2 siblings, 1 reply; 200+ results
From: Stephen Hemminger @ 2024-10-04 16:35 UTC (permalink / raw)
To: pbhagavatula
Cc: jerinj, sthotton, abdullah.sevincer, hemant.agrawal,
sachin.saxena, harry.van.haaren, mattias.ronnblom, liangma,
peter.mccarthy, dev
On Fri, 4 Oct 2024 21:54:45 +0530
<pbhagavatula@marvell.com> wrote:
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
>
> Event pre-scheduling improves scheduling performance by assigning events
> to event ports in advance when dequeues are issued.
> This series introduces various types and levels of pre-scheduling to the
> eventdev library.
>
> pre-scheduling Types:
> * RTE_EVENT_PRESCHEDULE_NONE: No pre-scheduling.
> * RTE_EVENT_PRESCHEDULE: Always issue a pre-schedule when dequeue is issued.
> * RTE_EVENT_PRESCHEDULE_ADAPTIVE: Delay issuing pre-schedule until there
> are no forward progress constraints with the held flow contexts.
>
> pre-scheduling Levels:
> * Event Device Level Pre-scheduling: Pre-scheduling can be enabled or disabled at the
> event device during configuration. Event devices can indicate pre-scheduling
> capabilities using `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
> `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event device info
> function `info.event_dev_cap`.
> * Event Port Level Prefetch: Pre-scheduling can be selectively enabled or disabled
> at the event port during runtime. Event devices can indicate this capability
> using `RTE_EVENT_PORT_CAP_PER_PORT_PRESCHEDULE` via the event device info
> function `info.event_port_cap`.
> * Application Controlled Prefetch Hint: Applications can provide hints to the
> event device to start pre-scheduling events using the new API
> `rte_event_port_pre-schedule`. Event devices can indicate this capabilities using
> `RTE_EVENT_DEV_CAP_PRESCHEDULE_EXPLICIT` via the event device info function
> `info.event_dev_cap`.
>
> The rte_event_dequeue_burst operation initiates the pre-schedule operation, which
> completes in parallel without affecting the flow context of the dequeued events and
> dequeue latency. On the next dequeue operation, the pre-scheduleed events are
> dequeued, and pre-schedule operation is initiated again.
>
> In the case of application-controlled pre-schedule hints, the currently held flow
> contexts, if any, are not affected by the pre-schedule operation. On the next
> dequeue operation, the pre-scheduleed events are returned, but pre-schedule is not
> initiated again until the application provides the hint again. If pre-scheduling
> is already enabled at the event device level or event port level, the hint is ignored.
>
> v2 Changes:
> - s/prefetch/pre-schedule (Mattias).
> v3 Changes:
> - Add CNXK preschedule implementation.
> - Update test-eventdev to use prescheduling.
> - Update examples to use prescheduling.
> v4 Changes:
> - Fix compilation.
> v5 Changes:
> - Update ABI changes
> - s/RTE_EVENT_DEV_PRESCHEDULE/RTE_EVENT_PRESCHEDULE/
> - s/RTE_EVENT_DEV_CAP_SW_PRESCHEDULE/RTE_EVENT_DEV_CAP_PRESCHEDULE_EXPLICIT/
> - s/RTE_EVENT_PORT_CAP_EVENT_PER_PORT_PRESCHEDULE/RTE_EVENT_PORT_CAP_PER_PORT_PRESCHEDULE
> - Various documentation fixes and updates.
>
>
> Pavan Nikhilesh (6):
> eventdev: introduce event pre-scheduling
> eventdev: add event port pre-schedule modify
> eventdev: add event preschedule hint
> event/cnkx: add pre-schedule support
> app/test-eventdev: add pre-scheduling support
> examples: use eventdev pre-scheduling
>
> app/test-eventdev/evt_common.h | 45 +++--
> app/test-eventdev/evt_options.c | 17 ++
> app/test-eventdev/evt_options.h | 1 +
> app/test/test_eventdev.c | 143 ++++++++++++++++
> doc/guides/eventdevs/cnxk.rst | 10 --
> doc/guides/eventdevs/features/cnxk.ini | 1 +
> doc/guides/eventdevs/features/default.ini | 1 +
> doc/guides/prog_guide/eventdev/eventdev.rst | 48 ++++++
> doc/guides/rel_notes/release_24_11.rst | 13 ++
> doc/guides/tools/testeventdev.rst | 6 +
> drivers/event/cnxk/cn10k_eventdev.c | 19 ++-
> drivers/event/cnxk/cn10k_worker.c | 21 +++
> drivers/event/cnxk/cn10k_worker.h | 2 +
> drivers/event/cnxk/cnxk_eventdev.c | 2 -
> drivers/event/cnxk/cnxk_eventdev.h | 1 -
> .../pipeline_worker_generic.c | 6 +
> .../eventdev_pipeline/pipeline_worker_tx.c | 6 +
> examples/ipsec-secgw/event_helper.c | 6 +
> examples/l2fwd-event/l2fwd_event_generic.c | 6 +
> .../l2fwd-event/l2fwd_event_internal_port.c | 6 +
> examples/l3fwd/l3fwd_event_generic.c | 6 +
> examples/l3fwd/l3fwd_event_internal_port.c | 6 +
> lib/eventdev/eventdev_pmd.h | 4 +
> lib/eventdev/eventdev_private.c | 41 ++++-
> lib/eventdev/eventdev_trace_points.c | 6 +
> lib/eventdev/rte_eventdev.h | 159 ++++++++++++++++++
> lib/eventdev/rte_eventdev_core.h | 14 +-
> lib/eventdev/rte_eventdev_trace_fp.h | 19 ++-
> lib/eventdev/version.map | 6 +
> 29 files changed, 592 insertions(+), 29 deletions(-)
Looks good, but the new API's should be experimental right?
^ permalink raw reply [relevance 0%]
* [PATCH v5 1/6] eventdev: introduce event pre-scheduling
2024-10-04 16:24 3% ` [PATCH v5 0/6] Introduce event pre-scheduling pbhagavatula
@ 2024-10-04 16:24 3% ` pbhagavatula
2024-10-04 16:35 0% ` [PATCH v5 0/6] Introduce " Stephen Hemminger
2024-10-05 7:25 3% ` [PATCH v6 " pbhagavatula
2 siblings, 0 replies; 200+ results
From: pbhagavatula @ 2024-10-04 16:24 UTC (permalink / raw)
To: jerinj, sthotton, abdullah.sevincer, hemant.agrawal,
sachin.saxena, harry.van.haaren, mattias.ronnblom, liangma,
peter.mccarthy
Cc: dev, Pavan Nikhilesh
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
Event pre-scheduling improves scheduling performance by assigning events
to event ports in advance when dequeues are issued.
The dequeue operation initiates the pre-schedule operation, which completes
in parallel without affecting the dequeued event flow contexts and
dequeue latency.
Event devices can indicate pre-scheduling capabilities using
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event device info
function `info.event_dev_cap`.
Applications can select the pre-schedule type and configure it through
`rte_event_dev_config.preschedule_type` during `rte_event_dev_configure`.
The supported pre-schedule types are:
* `RTE_EVENT_PRESCHEDULE_NONE` - No pre-scheduling.
* `RTE_EVENT_PRESCHEDULE` - Always issue a pre-schedule on dequeue.
* `RTE_EVENT_PRESCHEDULE_ADAPTIVE` - Delay issuing pre-schedule until
there are no forward progress constraints with the held flow contexts.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
app/test/test_eventdev.c | 108 ++++++++++++++++++++
doc/guides/eventdevs/features/default.ini | 1 +
doc/guides/prog_guide/eventdev/eventdev.rst | 23 +++++
doc/guides/rel_notes/release_24_11.rst | 10 ++
lib/eventdev/rte_eventdev.h | 49 +++++++++
5 files changed, 191 insertions(+)
diff --git a/app/test/test_eventdev.c b/app/test/test_eventdev.c
index 9a6c8f470c..a45d1396d7 100644
--- a/app/test/test_eventdev.c
+++ b/app/test/test_eventdev.c
@@ -1251,6 +1251,112 @@ test_eventdev_profile_switch(void)
return TEST_SUCCESS;
}
+static int
+preschedule_test(enum rte_event_dev_preschedule_type preschedule_type, const char *preschedule_name)
+{
+#define NB_EVENTS 1024
+ uint64_t start, total;
+ struct rte_event ev;
+ int rc, cnt;
+
+ ev.event_type = RTE_EVENT_TYPE_CPU;
+ ev.queue_id = 0;
+ ev.op = RTE_EVENT_OP_NEW;
+ ev.u64 = 0xBADF00D0;
+
+ for (cnt = 0; cnt < NB_EVENTS; cnt++) {
+ ev.flow_id = cnt;
+ rc = rte_event_enqueue_burst(TEST_DEV_ID, 0, &ev, 1);
+ TEST_ASSERT(rc == 1, "Failed to enqueue event");
+ }
+
+ RTE_SET_USED(preschedule_type);
+ total = 0;
+ while (cnt) {
+ start = rte_rdtsc_precise();
+ rc = rte_event_dequeue_burst(TEST_DEV_ID, 0, &ev, 1, 0);
+ if (rc) {
+ total += rte_rdtsc_precise() - start;
+ cnt--;
+ }
+ }
+ printf("Preschedule type : %s, avg cycles %" PRIu64 "\n", preschedule_name,
+ total / NB_EVENTS);
+
+ return TEST_SUCCESS;
+}
+
+static int
+preschedule_configure(enum rte_event_dev_preschedule_type type, struct rte_event_dev_info *info)
+{
+ struct rte_event_dev_config dev_conf;
+ struct rte_event_queue_conf qcfg;
+ struct rte_event_port_conf pcfg;
+ int rc;
+
+ devconf_set_default_sane_values(&dev_conf, info);
+ dev_conf.nb_event_ports = 1;
+ dev_conf.nb_event_queues = 1;
+ dev_conf.preschedule_type = type;
+
+ rc = rte_event_dev_configure(TEST_DEV_ID, &dev_conf);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+
+ rc = rte_event_port_default_conf_get(TEST_DEV_ID, 0, &pcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to get port0 default config");
+ rc = rte_event_port_setup(TEST_DEV_ID, 0, &pcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to setup port0");
+
+ rc = rte_event_queue_default_conf_get(TEST_DEV_ID, 0, &qcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to get queue0 default config");
+ rc = rte_event_queue_setup(TEST_DEV_ID, 0, &qcfg);
+ TEST_ASSERT_SUCCESS(rc, "Failed to setup queue0");
+
+ rc = rte_event_port_link(TEST_DEV_ID, 0, NULL, NULL, 0);
+ TEST_ASSERT(rc == (int)dev_conf.nb_event_queues, "Failed to link port, device %d",
+ TEST_DEV_ID);
+
+ rc = rte_event_dev_start(TEST_DEV_ID);
+ TEST_ASSERT_SUCCESS(rc, "Failed to start event device");
+
+ return 0;
+}
+
+static int
+test_eventdev_preschedule_configure(void)
+{
+ struct rte_event_dev_info info;
+ int rc;
+
+ rte_event_dev_info_get(TEST_DEV_ID, &info);
+
+ if ((info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE) == 0)
+ return TEST_SKIPPED;
+
+ rc = preschedule_configure(RTE_EVENT_PRESCHEDULE_NONE, &info);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+ rc = preschedule_test(RTE_EVENT_PRESCHEDULE_NONE, "RTE_EVENT_PRESCHEDULE_NONE");
+ TEST_ASSERT_SUCCESS(rc, "Failed to test preschedule RTE_EVENT_PRESCHEDULE_NONE");
+
+ rte_event_dev_stop(TEST_DEV_ID);
+ rc = preschedule_configure(RTE_EVENT_PRESCHEDULE, &info);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+ rc = preschedule_test(RTE_EVENT_PRESCHEDULE, "RTE_EVENT_PRESCHEDULE");
+ TEST_ASSERT_SUCCESS(rc, "Failed to test preschedule RTE_EVENT_PRESCHEDULE");
+
+ if (info.event_dev_cap & RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE) {
+ rte_event_dev_stop(TEST_DEV_ID);
+ rc = preschedule_configure(RTE_EVENT_PRESCHEDULE_ADAPTIVE, &info);
+ TEST_ASSERT_SUCCESS(rc, "Failed to configure eventdev");
+ rc = preschedule_test(RTE_EVENT_PRESCHEDULE_ADAPTIVE,
+ "RTE_EVENT_PRESCHEDULE_ADAPTIVE");
+ TEST_ASSERT_SUCCESS(rc,
+ "Failed to test preschedule RTE_EVENT_PRESCHEDULE_ADAPTIVE");
+ }
+
+ return TEST_SUCCESS;
+}
+
static int
test_eventdev_close(void)
{
@@ -1311,6 +1417,8 @@ static struct unit_test_suite eventdev_common_testsuite = {
test_eventdev_start_stop),
TEST_CASE_ST(eventdev_configure_setup, eventdev_stop_device,
test_eventdev_profile_switch),
+ TEST_CASE_ST(eventdev_configure_setup, NULL,
+ test_eventdev_preschedule_configure),
TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
test_eventdev_link),
TEST_CASE_ST(eventdev_setup_device, eventdev_stop_device,
diff --git a/doc/guides/eventdevs/features/default.ini b/doc/guides/eventdevs/features/default.ini
index 1cc4303fe5..c8d5ed2d74 100644
--- a/doc/guides/eventdevs/features/default.ini
+++ b/doc/guides/eventdevs/features/default.ini
@@ -22,6 +22,7 @@ carry_flow_id =
maintenance_free =
runtime_queue_attr =
profile_links =
+preschedule =
;
; Features of a default Ethernet Rx adapter.
diff --git a/doc/guides/prog_guide/eventdev/eventdev.rst b/doc/guides/prog_guide/eventdev/eventdev.rst
index fb6dfce102..9da5531859 100644
--- a/doc/guides/prog_guide/eventdev/eventdev.rst
+++ b/doc/guides/prog_guide/eventdev/eventdev.rst
@@ -357,6 +357,29 @@ Worker path:
// Process the event received.
}
+Event Pre-scheduling
+~~~~~~~~~~~~~~~~~~~~
+
+Event pre-scheduling improves scheduling performance by assigning events to event ports in advance
+when dequeues are issued.
+The `rte_event_dequeue_burst` operation initiates the pre-schedule operation, which completes
+in parallel without affecting the dequeued event flow contexts and dequeue latency.
+On the next dequeue operation, the pre-scheduled events are dequeued and pre-schedule is initiated
+again.
+
+An application can use event pre-scheduling if the event device supports it at either device
+level or at a individual port level.
+The application must check pre-schedule capability by checking if ``rte_event_dev_info.event_dev_cap``
+has the bit ``RTE_EVENT_DEV_CAP_PRESCHEDULE`` or ``RTE_EVENT_DEV_CAP_PRESCHEDULE_ADAPTIVE`` set, if
+present pre-scheduling can be enabled at device
+configuration time by setting appropriate pre-schedule type in ``rte_event_dev_config.preschedule``.
+
+The following pre-schedule types are supported:
+ * ``RTE_EVENT_PRESCHEDULE_NONE`` - No pre-scheduling.
+ * ``RTE_EVENT_PRESCHEDULE`` - Always issue a pre-schedule when dequeue is issued.
+ * ``RTE_EVENT_PRESCHEDULE_ADAPTIVE`` - Issue pre-schedule when dequeue is issued and there are
+ no forward progress constraints.
+
Starting the EventDev
~~~~~~~~~~~~~~~~~~~~~
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 0ff70d9057..39b1314e1c 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -55,6 +55,14 @@ New Features
Also, make sure to start the actual text at the margin.
=======================================================
+* **Added event device pre-scheduling support.**
+
+ Added support for pre-scheduling of events to event ports to improve
+ scheduling performance and latency.
+
+ * Added ``rte_event_dev_config::preschedule_type`` to configure the device
+ level pre-scheduling type.
+
Removed Items
-------------
@@ -100,6 +108,8 @@ ABI Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* eventdev: Added ``preschedule_type`` field to ``rte_event_dev_config``
+ structure.
Known Issues
------------
diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
index 08e5f9320b..4b69e74577 100644
--- a/lib/eventdev/rte_eventdev.h
+++ b/lib/eventdev/rte_eventdev.h
@@ -446,6 +446,30 @@ struct rte_event;
* @see RTE_SCHED_TYPE_PARALLEL
*/
+#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE (1ULL << 16)
+/**< Event device supports event pre-scheduling.
+ *
+ * When this capability is available, the application can enable event pre-scheduling on the event
+ * device to pre-schedule events to a event port when `rte_event_dequeue_burst()`
+ * is issued.
+ * The pre-schedule process starts with the `rte_event_dequeue_burst()` call and the
+ * pre-scheduled events are returned on the next `rte_event_dequeue_burst()` call.
+ *
+ * @see rte_event_dev_configure()
+ */
+
+#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE (1ULL << 17)
+/**< Event device supports adaptive event pre-scheduling.
+ *
+ * When this capability is available, the application can enable adaptive pre-scheduling
+ * on the event device where the events are pre-scheduled when there are no forward
+ * progress constraints with the currently held flow contexts.
+ * The pre-schedule process starts with the `rte_event_dequeue_burst()` call and the
+ * pre-scheduled events are returned on the next `rte_event_dequeue_burst()` call.
+ *
+ * @see rte_event_dev_configure()
+ */
+
/* Event device priority levels */
#define RTE_EVENT_DEV_PRIORITY_HIGHEST 0
/**< Highest priority level for events and queues.
@@ -680,6 +704,26 @@ rte_event_dev_attr_get(uint8_t dev_id, uint32_t attr_id,
* @see rte_event_dequeue_timeout_ticks(), rte_event_dequeue_burst()
*/
+/** Event device pre-schedule type enumeration. */
+enum rte_event_dev_preschedule_type {
+ RTE_EVENT_PRESCHEDULE_NONE,
+ /**< Disable pre-schedule across the event device or on a given event port.
+ * @ref rte_event_dev_config.preschedule_type
+ */
+ RTE_EVENT_PRESCHEDULE,
+ /**< Enable pre-schedule always across the event device or a given event port.
+ * @ref rte_event_dev_config.preschedule_type
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
+ */
+ RTE_EVENT_PRESCHEDULE_ADAPTIVE,
+ /**< Enable adaptive pre-schedule across the event device or a given event port.
+ * Delay issuing pre-schedule until there are no forward progress constraints with
+ * the held flow contexts.
+ * @ref rte_event_dev_config.preschedule_type
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
+ */
+};
+
/** Event device configuration structure */
struct rte_event_dev_config {
uint32_t dequeue_timeout_ns;
@@ -752,6 +796,11 @@ struct rte_event_dev_config {
* optimized for single-link usage, this field is a hint for how many
* to allocate; otherwise, regular event ports and queues will be used.
*/
+ enum rte_event_dev_preschedule_type preschedule_type;
+ /**< Event pre-schedule type to use across the event device, if supported.
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
+ * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
+ */
};
/**
--
2.25.1
^ permalink raw reply [relevance 3%]
* [PATCH v5 0/6] Introduce event pre-scheduling
` (2 preceding siblings ...)
@ 2024-10-04 16:24 3% ` pbhagavatula
2024-10-04 16:24 3% ` [PATCH v5 1/6] eventdev: introduce " pbhagavatula
` (2 more replies)
3 siblings, 3 replies; 200+ results
From: pbhagavatula @ 2024-10-04 16:24 UTC (permalink / raw)
To: jerinj, sthotton, abdullah.sevincer, hemant.agrawal,
sachin.saxena, harry.van.haaren, mattias.ronnblom, liangma,
peter.mccarthy
Cc: dev, Pavan Nikhilesh
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
Event pre-scheduling improves scheduling performance by assigning events
to event ports in advance when dequeues are issued.
This series introduces various types and levels of pre-scheduling to the
eventdev library.
pre-scheduling Types:
* RTE_EVENT_PRESCHEDULE_NONE: No pre-scheduling.
* RTE_EVENT_PRESCHEDULE: Always issue a pre-schedule when dequeue is issued.
* RTE_EVENT_PRESCHEDULE_ADAPTIVE: Delay issuing pre-schedule until there
are no forward progress constraints with the held flow contexts.
pre-scheduling Levels:
* Event Device Level Pre-scheduling: Pre-scheduling can be enabled or disabled at the
event device during configuration. Event devices can indicate pre-scheduling
capabilities using `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
`RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event device info
function `info.event_dev_cap`.
* Event Port Level Prefetch: Pre-scheduling can be selectively enabled or disabled
at the event port during runtime. Event devices can indicate this capability
using `RTE_EVENT_PORT_CAP_PER_PORT_PRESCHEDULE` via the event device info
function `info.event_port_cap`.
* Application Controlled Prefetch Hint: Applications can provide hints to the
event device to start pre-scheduling events using the new API
`rte_event_port_pre-schedule`. Event devices can indicate this capabilities using
`RTE_EVENT_DEV_CAP_PRESCHEDULE_EXPLICIT` via the event device info function
`info.event_dev_cap`.
The rte_event_dequeue_burst operation initiates the pre-schedule operation, which
completes in parallel without affecting the flow context of the dequeued events and
dequeue latency. On the next dequeue operation, the pre-scheduleed events are
dequeued, and pre-schedule operation is initiated again.
In the case of application-controlled pre-schedule hints, the currently held flow
contexts, if any, are not affected by the pre-schedule operation. On the next
dequeue operation, the pre-scheduleed events are returned, but pre-schedule is not
initiated again until the application provides the hint again. If pre-scheduling
is already enabled at the event device level or event port level, the hint is ignored.
v2 Changes:
- s/prefetch/pre-schedule (Mattias).
v3 Changes:
- Add CNXK preschedule implementation.
- Update test-eventdev to use prescheduling.
- Update examples to use prescheduling.
v4 Changes:
- Fix compilation.
v5 Changes:
- Update ABI changes
- s/RTE_EVENT_DEV_PRESCHEDULE/RTE_EVENT_PRESCHEDULE/
- s/RTE_EVENT_DEV_CAP_SW_PRESCHEDULE/RTE_EVENT_DEV_CAP_PRESCHEDULE_EXPLICIT/
- s/RTE_EVENT_PORT_CAP_EVENT_PER_PORT_PRESCHEDULE/RTE_EVENT_PORT_CAP_PER_PORT_PRESCHEDULE
- Various documentation fixes and updates.
Pavan Nikhilesh (6):
eventdev: introduce event pre-scheduling
eventdev: add event port pre-schedule modify
eventdev: add event preschedule hint
event/cnkx: add pre-schedule support
app/test-eventdev: add pre-scheduling support
examples: use eventdev pre-scheduling
app/test-eventdev/evt_common.h | 45 +++--
app/test-eventdev/evt_options.c | 17 ++
app/test-eventdev/evt_options.h | 1 +
app/test/test_eventdev.c | 143 ++++++++++++++++
doc/guides/eventdevs/cnxk.rst | 10 --
doc/guides/eventdevs/features/cnxk.ini | 1 +
doc/guides/eventdevs/features/default.ini | 1 +
doc/guides/prog_guide/eventdev/eventdev.rst | 48 ++++++
doc/guides/rel_notes/release_24_11.rst | 13 ++
doc/guides/tools/testeventdev.rst | 6 +
drivers/event/cnxk/cn10k_eventdev.c | 19 ++-
drivers/event/cnxk/cn10k_worker.c | 21 +++
drivers/event/cnxk/cn10k_worker.h | 2 +
drivers/event/cnxk/cnxk_eventdev.c | 2 -
drivers/event/cnxk/cnxk_eventdev.h | 1 -
.../pipeline_worker_generic.c | 6 +
.../eventdev_pipeline/pipeline_worker_tx.c | 6 +
examples/ipsec-secgw/event_helper.c | 6 +
examples/l2fwd-event/l2fwd_event_generic.c | 6 +
.../l2fwd-event/l2fwd_event_internal_port.c | 6 +
examples/l3fwd/l3fwd_event_generic.c | 6 +
examples/l3fwd/l3fwd_event_internal_port.c | 6 +
lib/eventdev/eventdev_pmd.h | 4 +
lib/eventdev/eventdev_private.c | 41 ++++-
lib/eventdev/eventdev_trace_points.c | 6 +
lib/eventdev/rte_eventdev.h | 159 ++++++++++++++++++
lib/eventdev/rte_eventdev_core.h | 14 +-
lib/eventdev/rte_eventdev_trace_fp.h | 19 ++-
lib/eventdev/version.map | 6 +
29 files changed, 592 insertions(+), 29 deletions(-)
--
2.25.1
^ permalink raw reply [relevance 3%]
* Re: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-10-04 3:54 5% ` Ferruh Yigit
2024-10-04 7:04 0% ` David Marchand
@ 2024-10-04 9:38 5% ` Dodji Seketeli
2024-10-04 17:45 4% ` [EXTERNAL] " Akhil Goyal
1 sibling, 1 reply; 200+ results
From: Dodji Seketeli @ 2024-10-04 9:38 UTC (permalink / raw)
To: Ferruh Yigit
Cc: Akhil Goyal, dev, Dodji Seketeli, thomas, david.marchand,
hemant.agrawal, anoobj, pablo.de.lara.guarch, fiona.trahe,
declan.doherty, matan, g.singh, fanzhang.oss, jianjay.zhou,
asomalap, ruifeng.wang, konstantin.v.ananyev, radu.nicolau,
ajit.khaparde, rnagadheeraj, mdr
Hello,
Ferruh Yigit <ferruh.yigit@amd.com> writes:
> On 9/5/2024 11:14 AM, Akhil Goyal wrote:
>> Replace *_LIST_END enumerators from asymmetric crypto
>> lib to avoid ABI breakage for every new addition in
>> enums with inline APIs.
>>
>> Signed-off-by: Akhil Goyal <gakhil@marvell.com>
>> ---
>> This patch was discussed in ML long time back.
>> https://patches.dpdk.org/project/dpdk/patch/20211008204516.3497060-1-gakhil@marvell.com/
>> Now added inline APIs for getting the list end which need to be updated
>> for each new entry to the enum. This shall help in avoiding ABI break
>> for adding new algo.
>>
>
> Hi Akhil,
>
> *I think* this hides the problem instead of fixing it, and this may be
> partially because of the tooling (libabigail) limitation.
So, I am not sure to understand how this would be a libabigail
limitation in general. Let me explain.
The abidiff tool that compares two binaries and emits reports about
their ABI changes can be given a suppression specification file which
instructs the tool about what type of change to avoid emitting reports
about.
It turns out there is a construct specifically designed to handle the
case where an enumerator is added right before the last enumerator of a
enum. Such a change causes the value of the last enumerator to
increase.
For instance, consider this enum:
enum foo_enum
{
FOO_FIRST,
FOO_SECOND,
FOO_END
};
If I add a new enumerator right before the last enumerator, I would get
this:
enum foo_enum
{
FOO_FIRST,
FOO_SECOND,
FOO_THIRD,
FOO_END
};
This change cause the value of the the FOOD_END enumerator to increase.
And that increase might be problematic. At the moment, for it being
problematic or not has to be the result of a careful review.
So, by default, abidiff will complain by saying that the value of
FOO_END was changed.
But you, as a community of practice, can decide that this kind of change
to the value of the last enumerator is not a problematic change, after
careful review of your code and practice. You thus can specify that
the tool using a suppression specification which has the following
syntax:
[suppress_type]
type_kind = enum
changed_enumerators = FOO_END, ANOTHER_ENUM_END, AND_ANOTHER_ENUM_END
or, alternatively, you can specify the enumerators you want to suppress
the changes for as a list of regular expressions:
[suppress_type]
type_kind = enum
changed_enumerators_regexp = .*_END$, .*_LAST$, .*_LIST_END$
Wouldn't that be enough to address your use case here (honest question)?
> This patch prevents the libabigail warning, true, but it doesn't improve
> anything from the application's perspective.
> Before or after this patch, application knows a fixed value as END value.
>
> Not all changes in the END (MAX) enum values cause ABI break, but tool
> warns on all, that is why I think this may be tooling limitation [1].
> (Just to stress, I am NOT talking about regular enum values change, I am
> talking about only END (MAX) value changes caused by appending new enum
> items.)
>
> As far as I can see (please Dodji, David correct me if I am wrong) ABI
> break only happens if application and library exchange enum values in
> the API (directly or within a struct).
Sorry, I am not sure to understand what you mean by "exchange enum values".
I would say is that if a change to an enumerator value causes a change
to the layout of a type which is part of the ABI, then that enumerator
value change might change the ABI. That ABI change might be problematic
(i.e an ABI break) or not.
>
> Exchanging enum values via API cause ABI issues because:
> 1. enum size is not fixed, it uses min storage size that can hold all
> values, adding new enums may cause its size change and of course this
> breaks ABI.
> 2. Library can send enum values that application doesn't know, this may
> cause application behave undefined.
> 3. Application sending MAX value (or more) to API expects error, but
> that may be a valid value in the new library and applications gets
> unexpected result.
>
>
> Let's assume above 1. happens, with this patch warning is prevented, but
> ABI break still can occur.
>
> One option can be not exchanging enums in APIs at all, this way changes
> in the END (MAX) enum values are harmless. But I can see cryptodev does
> this a lot.
>
> When enum is exchanged in APIs, and if we want to be strict about the
> ABI, safest option can be not appending to enums at all, and keeping END
> (MAX) items can be a way to enable warnings for this.
>
>
> More relaxed option is protect against only 1. since that is the real
> ABI issue, 2 & 3 are more defensive programming, so as long as enum size
> is not changed we may ignore the END (MAX) value change warnings.
>
>
>
> [1] It would be better if tool gives END (MAX) enum value warnings only
> if it is exchanged in an API, but not sure if this can be possible to
> detect.
I believe that if you want to know if an enumerator value is *USED* by a
type (which I believe is at the root of what you are alluding to), then
you would need a static analysis tool that works at the source level.
Or, you need a human review of the code once the binary analysis tool
told you that that value of the enumerator changed.
Why ? please let me give you an example:
enum foo_enum
{
FOO_FIRST,
FOO_SECOND,
FOO_END
};
int array[FOO_END];
Once this is compiled into binary, what libabigail is going to see by
analyzing the binary is that 'array' is an array of 2 integers. The
information about the size of the array being initially an enumerator
value is lost. To detect that, you need source level analysis.
But then, by reviewing the code, this is a construct that you can spot
and allow or forbid, depending on your goals as a project.
[...]
Cheers,
--
Dodji
^ permalink raw reply [relevance 5%]
* Re: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-10-04 3:54 5% ` Ferruh Yigit
@ 2024-10-04 7:04 0% ` David Marchand
2024-10-04 17:27 0% ` [EXTERNAL] " Akhil Goyal
2024-10-04 9:38 5% ` Dodji Seketeli
1 sibling, 1 reply; 200+ results
From: David Marchand @ 2024-10-04 7:04 UTC (permalink / raw)
To: Ferruh Yigit
Cc: Akhil Goyal, dev, Dodji Seketeli, thomas, hemant.agrawal, anoobj,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde, rnagadheeraj,
mdr
On Fri, Oct 4, 2024 at 5:55 AM Ferruh Yigit <ferruh.yigit@amd.com> wrote:
>
> On 9/5/2024 11:14 AM, Akhil Goyal wrote:
> > Replace *_LIST_END enumerators from asymmetric crypto
> > lib to avoid ABI breakage for every new addition in
> > enums with inline APIs.
> >
> > Signed-off-by: Akhil Goyal <gakhil@marvell.com>
> > ---
> > This patch was discussed in ML long time back.
> > https://patches.dpdk.org/project/dpdk/patch/20211008204516.3497060-1-gakhil@marvell.com/
> > Now added inline APIs for getting the list end which need to be updated
> > for each new entry to the enum. This shall help in avoiding ABI break
> > for adding new algo.
> >
>
> Hi Akhil,
>
> *I think* this hides the problem instead of fixing it, and this may be
> partially because of the tooling (libabigail) limitation.
>
> This patch prevents the libabigail warning, true, but it doesn't improve
> anything from the application's perspective.
> Before or after this patch, application knows a fixed value as END value.
>
> Not all changes in the END (MAX) enum values cause ABI break, but tool
> warns on all, that is why I think this may be tooling limitation [1].
> (Just to stress, I am NOT talking about regular enum values change, I am
> talking about only END (MAX) value changes caused by appending new enum
> items.)
>
> As far as I can see (please Dodji, David correct me if I am wrong) ABI
> break only happens if application and library exchange enum values in
> the API (directly or within a struct).
- There is also the following issue:
A library publicly exports an array sized against a END (MAX) enum in the API.
https://developers.redhat.com/blog/2019/05/06/how-c-array-sizes-become-part-of-the-binary-interface-of-a-library
I had made comments for an issue in the cryptodev library in the past:
https://inbox.dpdk.org/dev/CAJFAV8xs5CVdE2xwRtaxk5vE_PiQMV5LY5tKStk3R1gOuRTsUw@mail.gmail.com/
- Sorry to deviate from the _END enum discussion that tries to define
a solution for all cases, but all I see in the cryptodev patch is a
need for an enumerator... for an internal unit test.
From the RFC patch, I would simply change the
rte_crypto_asym_xform_type_list_end helper into a non inline symbol
and mark it __rte_internal, or move this helper to a non public header
used by the unit test.
Or add a (internal) rte_crypto_asym_xform_type_next_op() used through
a a RTE_CRYPTO_FOREACH_ASYM_OP() macro.
--
David Marchand
^ permalink raw reply [relevance 0%]
* [PATCH v4 1/2] ipsec: allow stateless IPsec processing
2024-10-03 13:45 3% ` [PATCH v3 1/2] " Aakash Sasidharan
@ 2024-10-04 6:34 3% ` Aakash Sasidharan
0 siblings, 0 replies; 200+ results
From: Aakash Sasidharan @ 2024-10-04 6:34 UTC (permalink / raw)
To: Konstantin Ananyev, Vladimir Medvedkin
Cc: gakhil, jerinj, anoobj, vvelumuri, asasidharan, dev
Introduce stateless packet preparation API for IPsec
processing. The new API would allow preparation of IPsec
packets without altering the internal state of an IPsec
session.
For outbound IPsec processing, the change enables user to
provide sequence number to be used for the IPsec operation.
Signed-off-by: Aakash Sasidharan <asasidharan@marvell.com>
---
doc/guides/prog_guide/ipsec_lib.rst | 6 ++
doc/guides/rel_notes/release_24_11.rst | 6 ++
lib/ipsec/esp_outb.c | 101 +++++++++++++++++--------
lib/ipsec/rte_ipsec.h | 84 ++++++++++++++++++++
lib/ipsec/sa.c | 4 +-
lib/ipsec/sa.h | 8 ++
6 files changed, 176 insertions(+), 33 deletions(-)
diff --git a/doc/guides/prog_guide/ipsec_lib.rst b/doc/guides/prog_guide/ipsec_lib.rst
index 0bdbdad1e4..395512f4f9 100644
--- a/doc/guides/prog_guide/ipsec_lib.rst
+++ b/doc/guides/prog_guide/ipsec_lib.rst
@@ -328,6 +328,12 @@ statistics. Per SA telemetry statistics can be enabled using
``rte_ipsec_telemetry_sa_add`` and disabled using
``rte_ipsec_telemetry_sa_del``. Note that these calls are not thread safe.
+Stateless IPsec packet processing support
+-----------------------------------------
+Support for stateless IPsec packet processing allowing use of custom
+sequence number to be used for IPsec outbound processing.
+rte_ipsec_pkt_stateless_prepare() takes as input the state parameter
+from the application and prepares the packet for IPsec processing.
Limitations
-----------
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 0ff70d9057..0964f13a12 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -84,6 +84,12 @@ API Changes
Also, make sure to start the actual text at the margin.
=======================================================
+ * ipsec: Add support for stateless IPsec processing.
+
+ Added new API to support stateless IPsec processing.
+ The new API are ``rte_ipsec_pkr_crypto_prepare_stateless`` and
+ ``rte_ipsec_pkt_cpu_prepare_stateless`` .
+
ABI Changes
-----------
diff --git a/lib/ipsec/esp_outb.c b/lib/ipsec/esp_outb.c
index ec87b1dce2..617fc52b21 100644
--- a/lib/ipsec/esp_outb.c
+++ b/lib/ipsec/esp_outb.c
@@ -288,28 +288,22 @@ outb_pkt_xprepare(const struct rte_ipsec_sa *sa, rte_be64_t sqc,
/*
* setup/update packets and crypto ops for ESP outbound tunnel case.
*/
-uint16_t
-esp_outb_tun_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
- struct rte_crypto_op *cop[], uint16_t num)
+static inline uint16_t
+esp_outb_tun_prepare_helper(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
+ struct rte_crypto_op *cop[], uint16_t n, uint64_t sqn)
{
int32_t rc;
- uint32_t i, k, n;
- uint64_t sqn;
+ uint32_t i, k;
rte_be64_t sqc;
struct rte_ipsec_sa *sa;
struct rte_cryptodev_sym_session *cs;
union sym_op_data icv;
uint64_t iv[IPSEC_MAX_IV_QWORD];
- uint32_t dr[num];
+ uint32_t dr[n];
sa = ss->sa;
cs = ss->crypto.ses;
- n = num;
- sqn = esn_outb_update_sqn(sa, &n);
- if (n != num)
- rte_errno = EOVERFLOW;
-
k = 0;
for (i = 0; i != n; i++) {
@@ -339,6 +333,30 @@ esp_outb_tun_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
return k;
}
+uint16_t
+esp_outb_tun_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
+ struct rte_crypto_op *cop[], uint16_t num)
+{
+ uint64_t sqn;
+ uint32_t n;
+
+ n = num;
+ sqn = esn_outb_update_sqn(ss->sa, &n);
+ if (n != num)
+ rte_errno = EOVERFLOW;
+
+ return esp_outb_tun_prepare_helper(ss, mb, cop, n, sqn);
+}
+
+uint16_t
+esp_outb_tun_prepare_stateless(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
+ struct rte_crypto_op *cop[], uint16_t num, struct rte_ipsec_state *state)
+{
+ uint64_t sqn = state->sqn;
+
+ return esp_outb_tun_prepare_helper(ss, mb, cop, num, sqn);
+}
+
/*
* setup/update packet data and metadata for ESP outbound transport case.
*/
@@ -529,33 +547,27 @@ outb_cpu_crypto_prepare(const struct rte_ipsec_sa *sa, uint32_t *pofs,
return clen;
}
-static uint16_t
-cpu_outb_pkt_prepare(const struct rte_ipsec_session *ss,
- struct rte_mbuf *mb[], uint16_t num,
- esp_outb_prepare_t prepare, uint32_t cofs_mask)
+static inline uint16_t
+cpu_outb_pkt_prepare_helper(const struct rte_ipsec_session *ss,
+ struct rte_mbuf *mb[], uint16_t n, esp_outb_prepare_t prepare,
+ uint32_t cofs_mask, uint64_t sqn)
{
int32_t rc;
- uint64_t sqn;
rte_be64_t sqc;
struct rte_ipsec_sa *sa;
- uint32_t i, k, n;
+ uint32_t i, k;
uint32_t l2, l3;
union sym_op_data icv;
- struct rte_crypto_va_iova_ptr iv[num];
- struct rte_crypto_va_iova_ptr aad[num];
- struct rte_crypto_va_iova_ptr dgst[num];
- uint32_t dr[num];
- uint32_t l4ofs[num];
- uint32_t clen[num];
- uint64_t ivbuf[num][IPSEC_MAX_IV_QWORD];
+ struct rte_crypto_va_iova_ptr iv[n];
+ struct rte_crypto_va_iova_ptr aad[n];
+ struct rte_crypto_va_iova_ptr dgst[n];
+ uint32_t dr[n];
+ uint32_t l4ofs[n];
+ uint32_t clen[n];
+ uint64_t ivbuf[n][IPSEC_MAX_IV_QWORD];
sa = ss->sa;
- n = num;
- sqn = esn_outb_update_sqn(sa, &n);
- if (n != num)
- rte_errno = EOVERFLOW;
-
for (i = 0, k = 0; i != n; i++) {
l2 = mb[i]->l2_len;
@@ -604,15 +616,40 @@ uint16_t
cpu_outb_tun_pkt_prepare(const struct rte_ipsec_session *ss,
struct rte_mbuf *mb[], uint16_t num)
{
- return cpu_outb_pkt_prepare(ss, mb, num, outb_tun_pkt_prepare, 0);
+ uint64_t sqn;
+ uint32_t n;
+
+ n = num;
+ sqn = esn_outb_update_sqn(ss->sa, &n);
+ if (n != num)
+ rte_errno = EOVERFLOW;
+
+ return cpu_outb_pkt_prepare_helper(ss, mb, n, outb_tun_pkt_prepare, 0, sqn);
+}
+
+uint16_t
+cpu_outb_tun_pkt_prepare_stateless(const struct rte_ipsec_session *ss,
+ struct rte_mbuf *mb[], uint16_t num, struct rte_ipsec_state *state)
+{
+ uint64_t sqn = state->sqn;
+
+ return cpu_outb_pkt_prepare_helper(ss, mb, num, outb_tun_pkt_prepare, 0, sqn);
}
uint16_t
cpu_outb_trs_pkt_prepare(const struct rte_ipsec_session *ss,
struct rte_mbuf *mb[], uint16_t num)
{
- return cpu_outb_pkt_prepare(ss, mb, num, outb_trs_pkt_prepare,
- UINT32_MAX);
+ uint64_t sqn;
+ uint32_t n;
+
+ n = num;
+ sqn = esn_outb_update_sqn(ss->sa, &n);
+ if (n != num)
+ rte_errno = EOVERFLOW;
+
+ return cpu_outb_pkt_prepare_helper(ss, mb, n, outb_trs_pkt_prepare,
+ UINT32_MAX, sqn);
}
/*
diff --git a/lib/ipsec/rte_ipsec.h b/lib/ipsec/rte_ipsec.h
index f15f6f2966..316ace1f19 100644
--- a/lib/ipsec/rte_ipsec.h
+++ b/lib/ipsec/rte_ipsec.h
@@ -23,11 +23,24 @@ extern "C" {
struct rte_ipsec_session;
+/**
+ * IPsec state for stateless processing of a batch of IPsec packets.
+ */
+struct rte_ipsec_state {
+ /**
+ * 64 bit sequence number to be used for the first packet of the
+ * batch of packets.
+ */
+ uint64_t sqn;
+};
+
/**
* IPsec session specific functions that will be used to:
* - prepare - for input mbufs and given IPsec session prepare crypto ops
* that can be enqueued into the cryptodev associated with given session
* (see *rte_ipsec_pkt_crypto_prepare* below for more details).
+ * - prepare_stateless - similar to prepare, but further processing is done
+ * based on IPsec state provided by the 'state' parameter.
* - process - finalize processing of packets after crypto-dev finished
* with them or process packets that are subjects to inline IPsec offload
* (see rte_ipsec_pkt_process for more details).
@@ -42,6 +55,17 @@ struct rte_ipsec_sa_pkt_func {
struct rte_mbuf *mb[],
uint16_t num);
} prepare;
+ union {
+ uint16_t (*async)(const struct rte_ipsec_session *ss,
+ struct rte_mbuf *mb[],
+ struct rte_crypto_op *cop[],
+ uint16_t num,
+ struct rte_ipsec_state *state);
+ uint16_t (*sync)(const struct rte_ipsec_session *ss,
+ struct rte_mbuf *mb[],
+ uint16_t num,
+ struct rte_ipsec_state *state);
+ } prepare_stateless;
uint16_t (*process)(const struct rte_ipsec_session *ss,
struct rte_mbuf *mb[],
uint16_t num);
@@ -128,6 +152,66 @@ rte_ipsec_pkt_cpu_prepare(const struct rte_ipsec_session *ss,
return ss->pkt_func.prepare.sync(ss, mb, num);
}
+/**
+ * Same as *rte_ipsec_pkt_crypto_prepare*, but processing is done based on
+ * IPsec state provided by the 'state' parameter. Internal IPsec state won't
+ * be updated when this API is called.
+ *
+ * For input mbufs and given IPsec session prepare crypto ops that can be
+ * enqueued into the cryptodev associated with given session.
+ * expects that for each input packet:
+ * - l2_len, l3_len are setup correctly
+ * Note that erroneous mbufs are not freed by the function,
+ * but are placed beyond last valid mbuf in the *mb* array.
+ * It is a user responsibility to handle them further.
+ * @param ss
+ * Pointer to the *rte_ipsec_session* object the packets belong to.
+ * @param mb
+ * The address of an array of *num* pointers to *rte_mbuf* structures
+ * which contain the input packets.
+ * @param cop
+ * The address of an array of *num* pointers to the output *rte_crypto_op*
+ * structures.
+ * @param num
+ * The maximum number of packets to process.
+ * @param state
+ * The IPsec state to be used for processing current batch of packets.
+ * @return
+ * Number of successfully processed packets, with error code set in rte_errno.
+ */
+__rte_experimental
+static inline uint16_t
+rte_ipsec_pkt_crypto_prepare_stateless(const struct rte_ipsec_session *ss,
+ struct rte_mbuf *mb[], struct rte_crypto_op *cop[], uint16_t num,
+ struct rte_ipsec_state *state)
+{
+ return ss->pkt_func.prepare_stateless.async(ss, mb, cop, num, state);
+}
+
+/**
+ * Same as *rte_ipsec_pkt_crypto_prepare_stateless*, but processing is done
+ * in synchronous mode.
+ *
+ * @param ss
+ * Pointer to the *rte_ipsec_session* object the packets belong to.
+ * @param mb
+ * The address of an array of *num* pointers to *rte_mbuf* structures
+ * which contain the input packets.
+ * @param num
+ * The maximum number of packets to process.
+ * @param state
+ * The IPsec state to be used for processing current batch of packets.
+ * @return
+ * Number of successfully processed packets, with error code set in rte_errno.
+ */
+__rte_experimental
+static inline uint16_t
+rte_ipsec_pkt_cpu_prepare_stateless(const struct rte_ipsec_session *ss,
+ struct rte_mbuf *mb[], uint16_t num, struct rte_ipsec_state *state)
+{
+ return ss->pkt_func.prepare_stateless.sync(ss, mb, num, state);
+}
+
/**
* Finalise processing of packets after crypto-dev finished with them or
* process packets that are subjects to inline IPsec offload.
diff --git a/lib/ipsec/sa.c b/lib/ipsec/sa.c
index 2297bd6d72..741e079831 100644
--- a/lib/ipsec/sa.c
+++ b/lib/ipsec/sa.c
@@ -710,6 +710,7 @@ lksd_none_pkt_func_select(const struct rte_ipsec_sa *sa,
case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV4):
case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV6):
pf->prepare.async = esp_outb_tun_prepare;
+ pf->prepare_stateless.async = esp_outb_tun_prepare_stateless;
pf->process = (sa->sqh_len != 0) ?
esp_outb_sqh_process : pkt_flag_process;
break;
@@ -748,6 +749,7 @@ cpu_crypto_pkt_func_select(const struct rte_ipsec_sa *sa,
case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV4):
case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV6):
pf->prepare.sync = cpu_outb_tun_pkt_prepare;
+ pf->prepare_stateless.sync = cpu_outb_tun_pkt_prepare_stateless;
pf->process = (sa->sqh_len != 0) ?
esp_outb_sqh_process : pkt_flag_process;
break;
@@ -810,7 +812,7 @@ ipsec_sa_pkt_func_select(const struct rte_ipsec_session *ss,
int32_t rc;
rc = 0;
- pf[0] = (struct rte_ipsec_sa_pkt_func) { {NULL}, NULL };
+ pf[0] = (struct rte_ipsec_sa_pkt_func) { {NULL}, {NULL}, NULL };
switch (ss->type) {
case RTE_SECURITY_ACTION_TYPE_NONE:
diff --git a/lib/ipsec/sa.h b/lib/ipsec/sa.h
index 719b5c735c..9b53586b2d 100644
--- a/lib/ipsec/sa.h
+++ b/lib/ipsec/sa.h
@@ -179,6 +179,10 @@ uint16_t
esp_outb_tun_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
struct rte_crypto_op *cop[], uint16_t num);
+uint16_t
+esp_outb_tun_prepare_stateless(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
+ struct rte_crypto_op *cop[], uint16_t num, struct rte_ipsec_state *state);
+
uint16_t
esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
struct rte_crypto_op *cop[], uint16_t num);
@@ -207,6 +211,10 @@ uint16_t
cpu_outb_tun_pkt_prepare(const struct rte_ipsec_session *ss,
struct rte_mbuf *mb[], uint16_t num);
uint16_t
+cpu_outb_tun_pkt_prepare_stateless(const struct rte_ipsec_session *ss,
+ struct rte_mbuf *mb[], uint16_t num, struct rte_ipsec_state *state);
+
+uint16_t
cpu_outb_trs_pkt_prepare(const struct rte_ipsec_session *ss,
struct rte_mbuf *mb[], uint16_t num);
--
2.25.1
^ permalink raw reply [relevance 3%]
* Re: [PATCH v4 3/6] eventdev: add SW event preschedule hint
@ 2024-10-04 5:14 3% ` Jerin Jacob
0 siblings, 0 replies; 200+ results
From: Jerin Jacob @ 2024-10-04 5:14 UTC (permalink / raw)
To: pbhagavatula
Cc: jerinj, sthotton, abdullah.sevincer, hemant.agrawal,
sachin.saxena, harry.van.haaren, mattias.ronnblom, liangma,
peter.mccarthy, dev
On Wed, Oct 2, 2024 at 2:19 AM <pbhagavatula@marvell.com> wrote:
>
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
>
> Add a new eventdev API to provide a hint to the eventdev PMD to
> pre-schedule the next event into the event port, without releasing
> the current flow context.
> Event device that support this feature advertises the capability
> using the RTE_EVENT_DEV_CAP_SW_PRESCHEDULE capability flag.
Change to RTE_EVENT_DEV_CAP_PRESCHEDULE_* prefix like other capas.
>
> Application can invoke `rte_event_port_preschedule` to hint the PMD.
>
> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
> ---
> doc/guides/prog_guide/eventdev/eventdev.rst | 8 ++++
> doc/guides/rel_notes/release_24_11.rst | 3 +-
> lib/eventdev/eventdev_pmd.h | 2 +
> lib/eventdev/eventdev_private.c | 21 ++++++++-
> lib/eventdev/eventdev_trace_points.c | 3 ++
> lib/eventdev/rte_eventdev.h | 49 +++++++++++++++++++++
> lib/eventdev/rte_eventdev_core.h | 5 +++
> lib/eventdev/rte_eventdev_trace_fp.h | 8 ++++
> lib/eventdev/version.map | 2 +
> 9 files changed, 98 insertions(+), 3 deletions(-)
>
> diff --git a/doc/guides/prog_guide/eventdev/eventdev.rst b/doc/guides/prog_guide/eventdev/eventdev.rst
> index 2deab0333e..1d8b86ab66 100644
> --- a/doc/guides/prog_guide/eventdev/eventdev.rst
> +++ b/doc/guides/prog_guide/eventdev/eventdev.rst
> @@ -390,6 +390,14 @@ To enable or disable event pre-scheduling at a given event port, the application
> // Disable pre-scheduling if thread is about to be scheduled out and issue dequeue() to drain
> // pending events.
>
> +Event Pre-schedule Hint can be used to provide a hint to the eventdev PMD to pre-schedule the next
pre-schedule
Hint -> hint
Application may provide pre-schedule hint to eventdev PMD to pre-schedule ....
> +event without releasing the current flow context. Event device that support this feature advertises
> +the capability using the ``RTE_EVENT_DEV_CAP_SW_PRESCHEDULE`` capability flag.
SW is a very overloaded term, How about RTE_EVENT_DEV_CAP_PRESCHEDULE_EXPLICT?
Tell it is HINT. No need to check the capability in fast path.
> +If pre-scheduling is already enabled at a event device or event port level then the hint is ignored.
> +
> +.. code-block:: c
> +
> + rte_event_port_preschedule(dev_id, port_id, RTE_EVENT_DEV_PRESCHEDULE);
Since it is used at port level. Remove _DEV_.
>
> Starting the EventDev
> ~~~~~~~~~~~~~~~~~~~~~
> diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
> index 6e36ac7b7e..3ada21c084 100644
> --- a/doc/guides/rel_notes/release_24_11.rst
> +++ b/doc/guides/rel_notes/release_24_11.rst
> @@ -64,7 +64,8 @@ New Features
> level pre-scheduling type.
> * Added ``rte_event_port_preschedule_modify`` to modify pre-scheduling type
> on a given event port.
> -
> + * Added ``rte_event_port_preschedule`` to allow applications to decide when
> + to pre-schedule events on an event port.
As an explicit request or so.
>
> Removed Items
> -------------
> diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h
> index 9ea23aa6cd..0bee2347ef 100644
> --- a/lib/eventdev/eventdev_pmd.h
> +++ b/lib/eventdev/eventdev_pmd.h
> @@ -186,6 +186,8 @@ struct __rte_cache_aligned rte_eventdev {
> /**< Pointer to PMD Event switch profile function. */
> event_preschedule_modify_t preschedule_modify;
> /**< Pointer to PMD Event port pre-schedule type modify function. */
> + event_preschedule_t preschedule;
> + /**< Pointer to PMD Event port pre-schedule function. */
>
> uint64_t reserved_64s[3]; /**< Reserved for future fields */
> void *reserved_ptrs[3]; /**< Reserved for future fields */
> diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c
> index dc37f736f8..6aed1cba9a 100644
> --- a/lib/eventdev/eventdev_private.c
> +++ b/lib/eventdev/eventdev_private.c
> @@ -111,6 +111,19 @@ dummy_event_port_preschedule_modify_hint(__rte_unused void *port,
> return -ENOTSUP;
> }
>
> +static void
> +dummy_event_port_preschedule(__rte_unused void *port,
> + __rte_unused rte_event_dev_preschedule_type_t preschedule)
> +{
> + RTE_EDEV_LOG_ERR("pre-schedule requested for unconfigured event device");
> +}
> +
> +static void
> +dummy_event_port_preschedule_hint(__rte_unused void *port,
> + __rte_unused rte_event_dev_preschedule_type_t preschedule)
> +{
> +}
> +
> void
> event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
> {
> @@ -124,12 +137,12 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op)
> .dequeue_burst = dummy_event_dequeue_burst,
> .maintain = dummy_event_maintain,
> .txa_enqueue = dummy_event_tx_adapter_enqueue,
> - .txa_enqueue_same_dest =
> - dummy_event_tx_adapter_enqueue_same_dest,
> + .txa_enqueue_same_dest = dummy_event_tx_adapter_enqueue_same_dest,
> .ca_enqueue = dummy_event_crypto_adapter_enqueue,
> .dma_enqueue = dummy_event_dma_adapter_enqueue,
> .profile_switch = dummy_event_port_profile_switch,
> .preschedule_modify = dummy_event_port_preschedule_modify,
> + .preschedule = dummy_event_port_preschedule,
> .data = dummy_data,
> };
>
> @@ -153,8 +166,12 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op,
> fp_op->dma_enqueue = dev->dma_enqueue;
> fp_op->profile_switch = dev->profile_switch;
> fp_op->preschedule_modify = dev->preschedule_modify;
> + fp_op->preschedule = dev->preschedule;
> fp_op->data = dev->data->ports;
>
> if (fp_op->preschedule_modify == NULL)
> fp_op->preschedule_modify = dummy_event_port_preschedule_modify_hint;
> +
> + if (fp_op->preschedule == NULL)
> + fp_op->preschedule = dummy_event_port_preschedule_hint;
> }
> diff --git a/lib/eventdev/eventdev_trace_points.c b/lib/eventdev/eventdev_trace_points.c
> index e41674123c..e7af1591f7 100644
> --- a/lib/eventdev/eventdev_trace_points.c
> +++ b/lib/eventdev/eventdev_trace_points.c
> @@ -52,6 +52,9 @@ RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_port_profile_switch,
> RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_port_preschedule_modify,
> lib.eventdev.port.preschedule.modify)
>
> +RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_port_preschedule,
> + lib.eventdev.port.preschedule)
> +
> /* Eventdev Rx adapter trace points */
> RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_eth_rx_adapter_create,
> lib.eventdev.rx.adapter.create)
> diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
> index 0add0093ac..8df6a8bee1 100644
> --- a/lib/eventdev/rte_eventdev.h
> +++ b/lib/eventdev/rte_eventdev.h
> @@ -480,6 +480,15 @@ struct rte_event;
> * @see rte_event_port_preschedule_modify()
> */
>
> +#define RTE_EVENT_DEV_CAP_SW_PRESCHEDULE (1ULL << 19)
> +/**< Event device supports software prescheduling.
> + *
> + * When this flag is set, the application can issue preschedule request on
> + * a event port.
> + *
> + * @see rte_event_port_preschedule()
> + */
> +
> /* Event device priority levels */
> #define RTE_EVENT_DEV_PRIORITY_HIGHEST 0
> /**< Highest priority level for events and queues.
> @@ -2977,6 +2986,46 @@ rte_event_port_preschedule_modify(uint8_t dev_id, uint8_t port_id,
> return fp_ops->preschedule_modify(port, type);
> }
>
> +/**
> + * Provide a hint to the event device to pre-schedule events to event port .
> + *
> + * Hint the event device to pre-schedule events to the event port.
> + * The call doesn't not guarantee that the events will be pre-scheduleed.
> + * The call doesn't release the flow context currently held by the event port.
> + * The event device should support RTE_EVENT_DEV_CAP_SW_PRESCHEDULE capability.
Reword the comments.
> + *
> + * When pre-scheduling is enabled at an event device or event port level, the
> + * hint is ignored.
> + *
> + * Subsequent calls to rte_event_dequeue_burst() will dequeue the pre-schedule
> + * events but pre-schedule operation is not issued again.
> + *
> + * @param dev_id
> + * The identifier of the device.
> + * @param port_id
> + * The identifier of the event port.
> + * @param type
> + * The pre-schedule type to use on the event port.
> + */
> +static inline void
> +rte_event_port_preschedule(uint8_t dev_id, uint8_t port_id, rte_event_dev_preschedule_type_t type)
> +{
> + const struct rte_event_fp_ops *fp_ops;
> + void *port;
> +
> + fp_ops = &rte_event_fp_ops[dev_id];
> + port = fp_ops->data[port_id];
> +
> +#ifdef RTE_LIBRTE_EVENTDEV_DEBUG
> + if (dev_id >= RTE_EVENT_MAX_DEVS || port_id >= RTE_EVENT_MAX_PORTS_PER_DEV)
> + return;
> + if (port == NULL)
> + return;
> +#endif
> + rte_eventdev_trace_port_preschedule(dev_id, port_id, type);
> +
> + fp_ops->preschedule(port, type);
> +}
> #ifdef __cplusplus
> }
> #endif
> diff --git a/lib/eventdev/rte_eventdev_core.h b/lib/eventdev/rte_eventdev_core.h
> index 2275888a6b..21988abb4f 100644
> --- a/lib/eventdev/rte_eventdev_core.h
> +++ b/lib/eventdev/rte_eventdev_core.h
> @@ -53,6 +53,9 @@ typedef int (*event_preschedule_modify_t)(void *port,
> rte_event_dev_preschedule_type_t preschedule_type);
> /**< @internal Modify pre-schedule type on the event port. */
>
> +typedef void (*event_preschedule_t)(void *port, rte_event_dev_preschedule_type_t preschedule_type);
> +/**< @internal Issue pre-schedule on an event port. */
> +
> struct __rte_cache_aligned rte_event_fp_ops {
> void **data;
> /**< points to array of internal port data pointers */
> @@ -82,6 +85,8 @@ struct __rte_cache_aligned rte_event_fp_ops {
> /**< PMD Event switch profile function. */
> event_preschedule_modify_t preschedule_modify;
> /**< PMD Event port pre-schedule switch. */
> + event_preschedule_t preschedule;
> + /**< PMD Event port pre-schedule. */
Use reserved not to break ABI
> uintptr_t reserved[4];
> };
^ permalink raw reply [relevance 3%]
* Re: [PATCH v4 2/6] eventdev: add event port pre-schedule modify
@ 2024-10-04 5:02 3% ` Jerin Jacob
0 siblings, 0 replies; 200+ results
From: Jerin Jacob @ 2024-10-04 5:02 UTC (permalink / raw)
To: pbhagavatula
Cc: jerinj, sthotton, abdullah.sevincer, hemant.agrawal,
sachin.saxena, harry.van.haaren, mattias.ronnblom, liangma,
peter.mccarthy, dev
On Tue, Oct 1, 2024 at 7:14 PM <pbhagavatula@marvell.com> wrote:
>
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
>
> Some event devices allow pre-schedule types to be modified at
> runtime on an event port.
> Add `RTE_EVENT_DEV_CAP_EVENT_PER_PORT_PRESCHEDULE` capability
> to indicate that the event device supports this feature.
>
> Add `rte_event_port_preschedule_modify()` API to modify the
> pre-schedule type at runtime.
>
> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
>
> +To enable or disable event pre-scheduling at a given event port, the application can use
> +``rte_event_port_preschedule_modify()`` API.
# Talk about RTE_EVENT_DEV_CAP_EVENT_PER_PORT_PRESCHEDULE capability.
# Also tell this is HINT. i.e application does not need to check this
cap in fast path.
> +
> +.. code-block:: c
> +
> + rte_event_port_preschedule_modify(dev_id, port_id, RTE_EVENT_DEV_PRESCHEDULE);
> + // Dequeue events from the event port with normal dequeue() function.
> + rte_event_port_preschedule_modify(dev_id, port_id, RTE_EVENT_DEV_PRESCHEDULE_NONE);
> + // Disable pre-scheduling if thread is about to be scheduled out and issue dequeue() to drain
> + // pending events.
> +
> +
> Starting the EventDev
> ~~~~~~~~~~~~~~~~~~~~~
>
> diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
> index eae5cc326b..6e36ac7b7e 100644
> --- a/doc/guides/rel_notes/release_24_11.rst
> +++ b/doc/guides/rel_notes/release_24_11.rst
> @@ -62,6 +62,8 @@ New Features
>
> * Added ``rte_event_dev_config::preschedule_type`` to configure the device
> level pre-scheduling type.
> + * Added ``rte_event_port_preschedule_modify`` to modify pre-scheduling type
> + on a given event port.
>
>
> Removed Items
> diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h
> index 7a5699f14b..9ea23aa6cd 100644
> --- a/lib/eventdev/eventdev_pmd.h
> +++ b/lib/eventdev/eventdev_pmd.h
> @@ -184,6 +184,8 @@ struct __rte_cache_aligned rte_eventdev {
> /**< Pointer to PMD DMA adapter enqueue function. */
> event_profile_switch_t profile_switch;
> /**< Pointer to PMD Event switch profile function. */
> + event_preschedule_modify_t preschedule_modify;
> + /**< Pointer to PMD Event port pre-schedule type modify function. */
>
> uint64_t reserved_64s[3]; /**< Reserved for future fields */
> void *reserved_ptrs[3]; /**< Reserved for future fields */
> diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c
> index 017f97ccab..dc37f736f8 100644
> --- a/lib/eventdev/eventdev_private.c
> +++ b/lib/eventdev/eventdev_private.c
> @@ -96,6 +96,21 @@ dummy_event_port_profile_switch(__rte_unused void *port, __rte_unused uint8_t pr
> return -EINVAL;
> }
>
> diff --git a/lib/eventdev/eventdev_trace_points.c b/lib/eventdev/eventdev_trace_points.c
> index 8024e07531..e41674123c 100644
> --- a/lib/eventdev/eventdev_trace_points.c
> +++ b/lib/eventdev/eventdev_trace_points.c
> @@ -49,6 +49,9 @@ RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_maintain,
> RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_port_profile_switch,
> lib.eventdev.port.profile.switch)
>
> +RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_port_preschedule_modify,
> + lib.eventdev.port.preschedule.modify)
> +
> /* Eventdev Rx adapter trace points */
> RTE_TRACE_POINT_REGISTER(rte_eventdev_trace_eth_rx_adapter_create,
> lib.eventdev.rx.adapter.create)
> diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h
> index 5ea7f5a07b..0add0093ac 100644
> --- a/lib/eventdev/rte_eventdev.h
> +++ b/lib/eventdev/rte_eventdev.h
> @@ -470,6 +470,16 @@ struct rte_event;
> * @see rte_event_dev_configure()
> */
>
> +#define RTE_EVENT_DEV_CAP_EVENT_PER_PORT_PRESCHEDULE (1ULL << 18)
> +/**< Event device supports event pre-scheduling per event port.
> + *
> + * When this flag is set, the event device allows controlling the event
> + * pre-scheduling at a event port granularity.
Tell it is HINT.
> + *
> + * @see rte_event_dev_configure()
> + * @see rte_event_port_preschedule_modify()
> + */
> +
> /* Event device priority levels */
> #define RTE_EVENT_DEV_PRIORITY_HIGHEST 0
> /**< Highest priority level for events and queues.
> @@ -708,18 +718,23 @@ typedef enum {
> RTE_EVENT_DEV_PRESCHEDULE_NONE = 0,
> /* Disable pre-schedule across the event device or on a given event port.
> * @ref rte_event_dev_config.preschedule_type
> + * @ref rte_event_port_preschedule_modify()
> */
> RTE_EVENT_DEV_PRESCHEDULE,
> /* Enable pre-schedule always across the event device or a given event port.
> * @ref rte_event_dev_config.preschedule_type
> + * @ref rte_event_port_preschedule_modify()
> * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
> + * @see RTE_EVENT_DEV_CAP_EVENT_PER_PORT_PRESCHEDULE
> */
> RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE,
> /* Enable adaptive pre-schedule across the event device or a given event port.
> * Delay issuing pre-schedule until there are no forward progress constraints with
> * the held flow contexts.
> * @ref rte_event_dev_config.preschedule_type
> + * @ref rte_event_port_preschedule_modify()
> * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
> + * @see RTE_EVENT_DEV_CAP_EVENT_PER_PORT_PRESCHEDULE
> */
> } rte_event_dev_preschedule_type_t;
>
> @@ -2922,6 +2937,46 @@ rte_event_port_profile_switch(uint8_t dev_id, uint8_t port_id, uint8_t profile_i
> return fp_ops->profile_switch(port, profile_id);
> }
>
> +/**
> + * Change the pre-schedule type to use on an event port.
Modify the pre-schedule of the given event port ?
> + *
> + * This function is used to change the current pre-schedule type configured
> + * on an event port, the pre-schedule type can be set to none to disable pre-scheduling.
> + * This effects the subsequent ``rte_event_dequeue_burst`` call.
> + * The event device should support RTE_EVENT_DEV_CAP_PER_PORT_PRESCHEDULE capability.
Tell it is HINT and the application does not need to check the capa
> + *
> + * @param dev_id
> + * The identifier of the device.
> + * @param port_id
> + * The identifier of the event port.
> + * @param type
> + * The preschedule type to use on the event port.
> + * @return
> + * - 0 on success.
> + * - -EINVAL if *dev_id*, *port_id*, or *type* is invalid.
> + */
> +static inline int
> +rte_event_port_preschedule_modify(uint8_t dev_id, uint8_t port_id,
> + rte_event_dev_preschedule_type_t type)
Remove rte_event_dev_preschedule_type_t and use enum
rte_event_dev_preschedule_type
> +{
> + const struct rte_event_fp_ops *fp_ops;
> + void *port;
> +
> + fp_ops = &rte_event_fp_ops[dev_id];
> + port = fp_ops->data[port_id];
> +
> +#ifdef RTE_LIBRTE_EVENTDEV_DEBUG
> + if (dev_id >= RTE_EVENT_MAX_DEVS || port_id >= RTE_EVENT_MAX_PORTS_PER_DEV)
> + return -EINVAL;
> +
> + if (port == NULL)
> + return -EINVAL;
> +#endif
> + rte_eventdev_trace_port_preschedule_modify(dev_id, port_id, type);
> +
> + return fp_ops->preschedule_modify(port, type);
> +}
> +
> #ifdef __cplusplus
> }
> #endif
> diff --git a/lib/eventdev/rte_eventdev_core.h b/lib/eventdev/rte_eventdev_core.h
> index fc8e1556ab..2275888a6b 100644
> --- a/lib/eventdev/rte_eventdev_core.h
> +++ b/lib/eventdev/rte_eventdev_core.h
> @@ -49,6 +49,10 @@ typedef uint16_t (*event_dma_adapter_enqueue_t)(void *port, struct rte_event ev[
> typedef int (*event_profile_switch_t)(void *port, uint8_t profile);
> /**< @internal Switch active link profile on the event port. */
>
> +typedef int (*event_preschedule_modify_t)(void *port,
> + rte_event_dev_preschedule_type_t preschedule_type);
> +/**< @internal Modify pre-schedule type on the event port. */
> +
> struct __rte_cache_aligned rte_event_fp_ops {
> void **data;
> /**< points to array of internal port data pointers */
> @@ -76,6 +80,8 @@ struct __rte_cache_aligned rte_event_fp_ops {
> /**< PMD DMA adapter enqueue function. */
> event_profile_switch_t profile_switch;
> /**< PMD Event switch profile function. */
> + event_preschedule_modify_t preschedule_modify;
> + /**< PMD Event port pre-schedule switch. */
Use reserved not to break ABI
> uintptr_t reserved[4];
> };
>
> diff --git a/lib/eventdev/rte_eventdev_trace_fp.h b/lib/eventdev/rte_eventdev_trace_fp.h
> index 04d510ad00..78baed94de 100644
> --- a/lib/eventdev/rte_eventdev_trace_fp.h
> +++ b/lib/eventdev/rte_eventdev_trace_fp.h
> @@ -8,7 +8,7 @@
> /**
> * @file
> *
> - * API for ethdev trace support
> + * API for eventdev trace support
> */
>
> #ifdef __cplusplus
> @@ -54,6 +54,15 @@ RTE_TRACE_POINT_FP(
> rte_trace_point_emit_u8(profile);
> )
>
> +RTE_TRACE_POINT_FP(
> + rte_eventdev_trace_port_preschedule_modify,
> + RTE_TRACE_POINT_ARGS(uint8_t dev_id, uint8_t port_id,
> + int type),
> + rte_trace_point_emit_u8(dev_id);
> + rte_trace_point_emit_u8(port_id);
> + rte_trace_point_emit_int(type);
> +)
> +
> RTE_TRACE_POINT_FP(
> rte_eventdev_trace_eth_tx_adapter_enqueue,
> RTE_TRACE_POINT_ARGS(uint8_t dev_id, uint8_t port_id, void *ev_table,
> diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
> index 4947bb4ec6..b6d63ba576 100644
> --- a/lib/eventdev/version.map
> +++ b/lib/eventdev/version.map
> @@ -147,6 +147,10 @@ EXPERIMENTAL {
> rte_event_port_profile_unlink;
> rte_event_port_profile_links_get;
> __rte_eventdev_trace_port_profile_switch;
> +
> + # added in 24.11
> + rte_event_port_preschedule_modify;
> + __rte_eventdev_trace_port_preschedule_modify;
> };
>
> INTERNAL {
> --
> 2.25.1
>
^ permalink raw reply [relevance 3%]
* Re: [PATCH v4 1/6] eventdev: introduce event pre-scheduling
@ 2024-10-04 4:47 3% ` Jerin Jacob
0 siblings, 0 replies; 200+ results
From: Jerin Jacob @ 2024-10-04 4:47 UTC (permalink / raw)
To: pbhagavatula
Cc: jerinj, sthotton, abdullah.sevincer, hemant.agrawal,
sachin.saxena, harry.van.haaren, mattias.ronnblom, liangma,
peter.mccarthy, dev
On Tue, Oct 1, 2024 at 7:44 PM <pbhagavatula@marvell.com> wrote:
>
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
>
> Event pre-scheduling improves scheduling performance by assigning events
> to event ports in advance when dequeues are issued.
> The dequeue operation initiates the pre-schedule operation, which completes
> in parallel without affecting the dequeued event flow contexts and
> dequeue latency.
>
> Event devices can indicate pre-scheduling capabilities using
> `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE` and
> `RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE` via the event device info
> function `info.event_dev_cap`.
>
> Applications can select the pre-schedule type and configure it through
> `rte_event_dev_config.preschedule_type` during `rte_event_dev_configure`.
>
> The supported pre-schedule types are:
> * `RTE_EVENT_DEV_PRESCHEDULE_NONE` - No pre-scheduling.
> * `RTE_EVENT_DEV_PRESCHEDULE` - Always issue a pre-schedule on dequeue.
> * `RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE` - Delay issuing pre-schedule until
> there are no forward progress constraints with the held flow contexts.
>
> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
eventdev PMD maintainers,
Is anyone planning to review this series more? The general outlook of
the patch looks good to me. I will have a few minor comments. If there
are no other comments, I will merge early next week as RC1 PR.
> ---
> app/test/test_eventdev.c | 108 ++++++++++++++++++++
> doc/guides/eventdevs/features/default.ini | 1 +
> doc/guides/prog_guide/eventdev/eventdev.rst | 22 ++++
> doc/guides/rel_notes/release_24_11.rst | 8 ++
> lib/eventdev/rte_eventdev.h | 48 +++++++++
> 5 files changed, 187 insertions(+)
>
>
> +Event Pre-scheduling
> +~~~~~~~~~~~~~~~~~~~~
> +
> +Event pre-scheduling improves scheduling performance by assigning events to event ports in advance
> +when dequeues are issued.
> +The `rte_event_dequeue_burst` operation initiates the pre-schedule operation, which completes
> +in parallel without affecting the dequeued event flow contexts and dequeue latency.
> +On the next dequeue operation, the pre-scheduled events are dequeued and pre-schedule is initiated
> +again.
> +
> +An application can use event pre-scheduling if the event device supports it at either device
> +level or at a individual port level.
> +The application can check pre-schedule capability by checking if ``rte_event_dev_info.event_dev_cap``
can -> must
> +has the bit ``RTE_EVENT_DEV_CAP_PRESCHEDULE`` set, if present pre-scheduling can be enabled at device
> +configuration time by setting appropriate pre-schedule type in ``rte_event_dev_config.preschedule``.
Missing RTE_EVENT_DEV_CAP_PRESCHEDULE_ADAPTIVE cap doc.
> +
> +Currently, the following pre-schedule types are supported:
I think, we can remove “Currently”
> + * ``RTE_EVENT_DEV_PRESCHEDULE_NONE`` - No pre-scheduling.
> + * ``RTE_EVENT_DEV_PRESCHEDULE`` - Always issue a pre-schedule when dequeue is issued.
> + * ``RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE`` - Issue pre-schedule when dequeue is issued and there are
> + no forward progress constraints.
> +
>
> +#define RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE (1ULL << 16)
> +/**< Event device supports event pre-scheduling.
> + *
> + * When this capability is available, the application can enable event pre-scheduling on the event
> + * device to pre-schedule events to a event port when `rte_event_dequeue_burst()`
> + * is issued.
> + * The pre-schedule process starts with the `rte_event_dequeue_burst()` call and the
> + * pre-scheduled events are returned on the next `rte_event_dequeue_burst()` call.
> + *
> + * @see rte_event_dev_configure()
> + */
Doxygen for the new enum is missing.
>
> +typedef enum {
> + RTE_EVENT_DEV_PRESCHEDULE_NONE = 0,
Explicit 0 is not needed.
> + /* Disable pre-schedule across the event device or on a given event port.
Use Doxygen format across the series, i.e /**<
> + * @ref rte_event_dev_config.preschedule_type
> + */
> + RTE_EVENT_DEV_PRESCHEDULE,
> + /* Enable pre-schedule always across the event device or a given event port.
> + * @ref rte_event_dev_config.preschedule_type
> + * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
> + */
> + RTE_EVENT_DEV_PRESCHEDULE_ADAPTIVE,
> + /* Enable adaptive pre-schedule across the event device or a given event port.
> + * Delay issuing pre-schedule until there are no forward progress constraints with
> + * the held flow contexts.
> + * @ref rte_event_dev_config.preschedule_type
> + * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
> + */
> +} rte_event_dev_preschedule_type_t;
> +
> /** Event device configuration structure */
> struct rte_event_dev_config {
> uint32_t dequeue_timeout_ns;
> @@ -752,6 +795,11 @@ struct rte_event_dev_config {
> * optimized for single-link usage, this field is a hint for how many
> * to allocate; otherwise, regular event ports and queues will be used.
> */
> + rte_event_dev_preschedule_type_t preschedule_type;
Please add ABI changes in doc/guides/rel_notes/release_24_11.rst
> + /**< Event pre-schedule type to use across the event device, if supported.
> + * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE
> + * @see RTE_EVENT_DEV_CAP_EVENT_PRESCHEDULE_ADAPTIVE
> + */
> };
>
> /**
> --
> 2.25.1
>
^ permalink raw reply [relevance 3%]
* Re: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-09-23 20:41 4% ` Akhil Goyal
2024-10-03 7:00 0% ` Akhil Goyal
@ 2024-10-04 4:00 0% ` Ferruh Yigit
2024-10-04 17:26 4% ` [EXTERNAL] " Akhil Goyal
1 sibling, 1 reply; 200+ results
From: Ferruh Yigit @ 2024-10-04 4:00 UTC (permalink / raw)
To: Akhil Goyal, Morten Brørup, fengchengwen, dev
Cc: thomas, david.marchand, hemant.agrawal, Anoob Joseph,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde,
Nagadheeraj Rottela, mdr
On 9/23/2024 9:41 PM, Akhil Goyal wrote:
> Hi Morten,
>
> Apologies for delayed response.
>> Maybe a combination, returning the lowest end of the two versions of the list,
>> would work...
>>
>> ----------------------------------
>> Common header file (rte_common.h):
>> ----------------------------------
>>
>> /* Add at end of enum list in the header file. */
>> #define RTE_ENUM_LIST_END(name) \
>> _ # name # _ENUM_LIST_END /**< @internal */
>>
>> /* Add somewhere in header file, preferably after the enum list. */
>> #define rte_declare_enum_list_end(name) \
>> /** @internal */ \
>> int _# name # _enum_list_end(void); \
>> \
>> static int name # _enum_list_end(void) \
>> { \
>> static int cached = 0; \
>> \
>> if (likely(cached != 0)) \
>> return cached; \
>> \
>> return cached = RTE_MIN( \
>> RTE_ENUM_LIST_END(name), \
>> _ # name # _enum_list_end()); \
>> } \
>> \
>> int _# name # _enum_list_end(void)
>>
>> /* Add in the library/driver implementation. */
>> #define rte_define_enum_list_end(name) \
>> int _# name # _enum_list_end(void) \
>> { \
>> return RTE_ENUM_LIST_END(name); \
>> } \
>> \
>> int _# name # _enum_list_end(void)
>>
>> --------------------
>> Library header file:
>> --------------------
>>
>> enum rte_crypto_asym_op_type {
>> RTE_CRYPTO_ASYM_OP_VERIFY,
>> /**< Signature Verification operation */
>> RTE_ENUM_LIST_END(rte_crypto_asym_op)
>
> Will the ABI check be ok for adding anything in between
> RTE_CRYPTO_ASYM_OP_VERIFY and RTE_ENUM_LIST_END(rte_crypto_asym_op)?
> Don’t we need to add exception for that if we somehow make it internal by adding a comment only?
> Library is actually not restricting the application to not use RTE_ENUM_LIST_END(rte_crypto_asym_op) directly.
>
> Also we may need to expose the .c file internal function as experimental in version.map
>
>> }
>>
>> rte_declare_enum_list_end(rte_crypto_asym_op);
>>
>> ---------------
>> Library C file:
>> ---------------
>>
>> rte_define_enum_list_end(rte_crypto_asym_op);
>
> If we want to make it a generic thing in rte_common.h
> Will the below change be ok?
> ----------------------------------
> Common header file (rte_common.h):
> ----------------------------------
> #define rte_define_enum_list_end(name, last_value) \
> static inline int name ## _enum_list_end(void) \
> { \
> return last_value + 1; \
> }
>
> ----------------
> Lib header file
> ----------------
> //After the enum definition define the list end as below
> rte_define_enum_list_end(rte_crypto_asym_op, RTE_CRYPTO_ASYM_OP_VERIFY);
>
I assume Morten suggests his macros to escape from maintenance cost of
updating inline function each time a new enum is added.
But with above suggestion, that cost is still there, so I don't think
this one has a benefit against the original suggestion.
^ permalink raw reply [relevance 0%]
* Re: [EXTERNAL] Re: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-09-06 7:45 0% ` [EXTERNAL] " Akhil Goyal
@ 2024-10-04 3:56 0% ` Ferruh Yigit
0 siblings, 0 replies; 200+ results
From: Ferruh Yigit @ 2024-10-04 3:56 UTC (permalink / raw)
To: Akhil Goyal, fengchengwen, Morten Brørup, dev
Cc: thomas, david.marchand, hemant.agrawal, Anoob Joseph,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde,
Nagadheeraj Rottela, mdr
On 9/6/2024 8:45 AM, Akhil Goyal wrote:
>>>
>>> Here's an idea...
>>>
>>> We can introduce a generic design pattern where we keep the _LIST_END enum
>> value at the end, somehow marking it private (and not part of the API/ABI), and
>> move the _list_end() function inside the C file, so it uses the _LIST_END enum
>> value that the library was built with. E.g. like this:
>>>
>>>
>>> In the header file:
>>>
>>> enum rte_crypto_asym_op_type {
>>> RTE_CRYPTO_ASYM_OP_VERIFY,
>>> /**< Signature Verification operation */
>>> #if RTE_BUILDING_INTERNAL
>>> __RTE_CRYPTO_ASYM_OP_LIST_END /* internal */
>>> #endif
>>> }
>>>
>>> int rte_crypto_asym_op_list_end(void);
>>>
>>>
>>> And in the associated library code file, when including rte_crypto_asym.h:
>>>
>>> #define RTE_BUILDING_INTERNAL
>>> #include <cryptodev/rte_crypto_asym.h>
>>>
>>> int
>>> rte_crypto_asym_op_list_end(void)
>>> {
>>> return __RTE_CRYPTO_ASYM_OP_LIST_END;
>>> }
>>
>> It's more generic, and also keep LIST_END in the define, we just add new enum
>> before it.
>> But based on my understanding of ABI compatibility, from the point view of
>> application,
>> this API should return old-value even with the new library, but it will return new-
>> value
>> with new library. It could also break ABI.
>>
>> So this API should force inline, just as this patch did. But it seem can't work if
>> move
>> this API to header file and add static inline.
>>
> Yes, moving to c file does not seem to solve the purpose.
> So should we move with the way the patch is submitted or we have some other suggestion?
>
+1, when it is not inline function, this approach won't work.
^ permalink raw reply [relevance 0%]
* Re: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-09-05 10:14 4% [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs Akhil Goyal
2024-09-05 15:09 4% ` Morten Brørup
@ 2024-10-04 3:54 5% ` Ferruh Yigit
2024-10-04 7:04 0% ` David Marchand
2024-10-04 9:38 5% ` Dodji Seketeli
2024-10-09 11:24 3% ` [PATCH] cryptodev: remove unnecessary list end Akhil Goyal
2 siblings, 2 replies; 200+ results
From: Ferruh Yigit @ 2024-10-04 3:54 UTC (permalink / raw)
To: Akhil Goyal, dev, Dodji Seketeli
Cc: thomas, david.marchand, hemant.agrawal, anoobj,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde, rnagadheeraj,
mdr
On 9/5/2024 11:14 AM, Akhil Goyal wrote:
> Replace *_LIST_END enumerators from asymmetric crypto
> lib to avoid ABI breakage for every new addition in
> enums with inline APIs.
>
> Signed-off-by: Akhil Goyal <gakhil@marvell.com>
> ---
> This patch was discussed in ML long time back.
> https://patches.dpdk.org/project/dpdk/patch/20211008204516.3497060-1-gakhil@marvell.com/
> Now added inline APIs for getting the list end which need to be updated
> for each new entry to the enum. This shall help in avoiding ABI break
> for adding new algo.
>
Hi Akhil,
*I think* this hides the problem instead of fixing it, and this may be
partially because of the tooling (libabigail) limitation.
This patch prevents the libabigail warning, true, but it doesn't improve
anything from the application's perspective.
Before or after this patch, application knows a fixed value as END value.
Not all changes in the END (MAX) enum values cause ABI break, but tool
warns on all, that is why I think this may be tooling limitation [1].
(Just to stress, I am NOT talking about regular enum values change, I am
talking about only END (MAX) value changes caused by appending new enum
items.)
As far as I can see (please Dodji, David correct me if I am wrong) ABI
break only happens if application and library exchange enum values in
the API (directly or within a struct).
Exchanging enum values via API cause ABI issues because:
1. enum size is not fixed, it uses min storage size that can hold all
values, adding new enums may cause its size change and of course this
breaks ABI.
2. Library can send enum values that application doesn't know, this may
cause application behave undefined.
3. Application sending MAX value (or more) to API expects error, but
that may be a valid value in the new library and applications gets
unexpected result.
Let's assume above 1. happens, with this patch warning is prevented, but
ABI break still can occur.
One option can be not exchanging enums in APIs at all, this way changes
in the END (MAX) enum values are harmless. But I can see cryptodev does
this a lot.
When enum is exchanged in APIs, and if we want to be strict about the
ABI, safest option can be not appending to enums at all, and keeping END
(MAX) items can be a way to enable warnings for this.
More relaxed option is protect against only 1. since that is the real
ABI issue, 2 & 3 are more defensive programming, so as long as enum size
is not changed we may ignore the END (MAX) value change warnings.
[1] It would be better if tool gives END (MAX) enum value warnings only
if it is exchanged in an API, but not sure if this can be possible to
detect.
> app/test/test_cryptodev_asym.c | 2 +-
> lib/cryptodev/rte_crypto_asym.h | 18 ++++++++++++++----
> 2 files changed, 15 insertions(+), 5 deletions(-)
>
> diff --git a/app/test/test_cryptodev_asym.c b/app/test/test_cryptodev_asym.c
> index f0b5d38543..f1ece475b8 100644
> --- a/app/test/test_cryptodev_asym.c
> +++ b/app/test/test_cryptodev_asym.c
> @@ -581,7 +581,7 @@ static inline void print_asym_capa(
> rte_cryptodev_asym_get_xform_string(capa->xform_type));
> printf("operation supported -");
>
> - for (i = 0; i < RTE_CRYPTO_ASYM_OP_LIST_END; i++) {
> + for (i = 0; i < rte_crypto_asym_op_list_end(); i++) {
> /* check supported operations */
> if (rte_cryptodev_asym_xform_capability_check_optype(capa, i)) {
> if (capa->xform_type == RTE_CRYPTO_ASYM_XFORM_DH)
> diff --git a/lib/cryptodev/rte_crypto_asym.h b/lib/cryptodev/rte_crypto_asym.h
> index 39d3da3952..290b300f84 100644
> --- a/lib/cryptodev/rte_crypto_asym.h
> +++ b/lib/cryptodev/rte_crypto_asym.h
> @@ -72,6 +72,7 @@ enum rte_crypto_curve_id {
> * Asymmetric crypto transformation types.
> * Each xform type maps to one asymmetric algorithm
> * performing specific operation
> + * Note: Update rte_crypto_asym_xform_type_list_end() for every new type added.
> */
> enum rte_crypto_asym_xform_type {
> RTE_CRYPTO_ASYM_XFORM_UNSPECIFIED = 0,
> @@ -119,12 +120,17 @@ enum rte_crypto_asym_xform_type {
> * Performs Encrypt, Decrypt, Sign and Verify.
> * Refer to rte_crypto_asym_op_type.
> */
> - RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
> - /**< End of list */
> };
>
> +static inline int
> +rte_crypto_asym_xform_type_list_end(void)
> +{
> + return RTE_CRYPTO_ASYM_XFORM_SM2 + 1;
> +}
> +
> /**
> * Asymmetric crypto operation type variants
> + * Note: Update rte_crypto_asym_op_list_end for every new type added.
> */
> enum rte_crypto_asym_op_type {
> RTE_CRYPTO_ASYM_OP_ENCRYPT,
> @@ -135,9 +141,14 @@ enum rte_crypto_asym_op_type {
> /**< Signature Generation operation */
> RTE_CRYPTO_ASYM_OP_VERIFY,
> /**< Signature Verification operation */
> - RTE_CRYPTO_ASYM_OP_LIST_END
> };
>
> +static inline int
> +rte_crypto_asym_op_list_end(void)
> +{
> + return RTE_CRYPTO_ASYM_OP_VERIFY + 1;
> +}
> +
> /**
> * Asymmetric crypto key exchange operation type
> */
> @@ -168,7 +179,6 @@ enum rte_crypto_rsa_padding_type {
> /**< RSA PKCS#1 OAEP padding scheme */
> RTE_CRYPTO_RSA_PADDING_PSS,
> /**< RSA PKCS#1 PSS padding scheme */
> - RTE_CRYPTO_RSA_PADDING_TYPE_LIST_END
> };
>
> /**
^ permalink raw reply [relevance 5%]
* Re: [PATCH v2] lib/hash: new feature adding existing key
@ 2024-10-03 22:37 0% ` Stephen Hemminger
0 siblings, 0 replies; 200+ results
From: Stephen Hemminger @ 2024-10-03 22:37 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, Abdullah Ömer Yamaç,
Yipeng Wang, Sameh Gobriel, Bruce Richardson, Vladimir Medvedkin,
David Marchand, Abdullah Ömer Yamaç
On Fri, 16 Feb 2024 13:43:52 +0100
Thomas Monjalon <thomas@monjalon.net> wrote:
> Any review please?
> If maintainers agree with the idea, we should announce the ABI change.
>
>
> 23/10/2023 10:29, Abdullah Ömer Yamaç:
> > From: Abdullah Ömer Yamaç <omer.yamac@ceng.metu.edu.tr>
> >
> > In some use cases inserting data with the same key shouldn't be
> > overwritten. We use a new flag in this patch to disable overwriting
> > data for the same key.
> >
> > Signed-off-by: Abdullah Ömer Yamaç <omer.yamac@ceng.metu.edu.tr>
> >
> > ---
> > Cc: Yipeng Wang <yipeng1.wang@intel.com>
> > Cc: Sameh Gobriel <sameh.gobriel@intel.com>
> > Cc: Bruce Richardson <bruce.richardson@intel.com>
> > Cc: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
> > Cc: David Marchand <david.marchand@redhat.com>
> > ---
> > lib/hash/rte_cuckoo_hash.c | 10 +++++++++-
> > lib/hash/rte_cuckoo_hash.h | 2 ++
> > lib/hash/rte_hash.h | 4 ++++
> > 3 files changed, 15 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/hash/rte_cuckoo_hash.c b/lib/hash/rte_cuckoo_hash.c
> > index 19b23f2a97..fe8f21bee4 100644
> > --- a/lib/hash/rte_cuckoo_hash.c
> > +++ b/lib/hash/rte_cuckoo_hash.c
> > @@ -32,7 +32,8 @@
> > RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY | \
> > RTE_HASH_EXTRA_FLAGS_EXT_TABLE | \
> > RTE_HASH_EXTRA_FLAGS_NO_FREE_ON_DEL | \
> > - RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF)
> > + RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF | \
> > + RTE_HASH_EXTRA_FLAGS_DISABLE_UPDATE_EXISTING_KEY)
> >
> > #define FOR_EACH_BUCKET(CURRENT_BKT, START_BUCKET) \
> > for (CURRENT_BKT = START_BUCKET; \
> > @@ -148,6 +149,7 @@ rte_hash_create(const struct rte_hash_parameters *params)
> > unsigned int readwrite_concur_support = 0;
> > unsigned int writer_takes_lock = 0;
> > unsigned int no_free_on_del = 0;
> > + unsigned int no_update_data = 0;
> > uint32_t *ext_bkt_to_free = NULL;
> > uint32_t *tbl_chng_cnt = NULL;
> > struct lcore_cache *local_free_slots = NULL;
> > @@ -216,6 +218,9 @@ rte_hash_create(const struct rte_hash_parameters *params)
> > no_free_on_del = 1;
> > }
> >
> > + if (params->extra_flag & RTE_HASH_EXTRA_FLAGS_DISABLE_UPDATE_EXISTING_KEY)
> > + no_update_data = 1;
> > +
> > /* Store all keys and leave the first entry as a dummy entry for lookup_bulk */
> > if (use_local_cache)
> > /*
> > @@ -428,6 +433,7 @@ rte_hash_create(const struct rte_hash_parameters *params)
> > h->ext_table_support = ext_table_support;
> > h->writer_takes_lock = writer_takes_lock;
> > h->no_free_on_del = no_free_on_del;
> > + h->no_update_data = no_update_data;
> > h->readwrite_concur_lf_support = readwrite_concur_lf_support;
> >
> > #if defined(RTE_ARCH_X86)
> > @@ -707,6 +713,8 @@ search_and_update(const struct rte_hash *h, void *data, const void *key,
> > k = (struct rte_hash_key *) ((char *)keys +
> > bkt->key_idx[i] * h->key_entry_size);
> > if (rte_hash_cmp_eq(key, k->key, h) == 0) {
> > + if (h->no_update_data == 1)
> > + return -EINVAL;
This is buggy, the caller assumes -1 on error in several places.
See:
ret = search_and_update(h, data, key, prim_bkt, sig);
if (ret != -1) {
__hash_rw_writer_unlock(h);
*ret_val = ret;
return 1;
}
These paths would exercised when table had to expand.
Also any new functionality like this would need tests in functional test.
^ permalink raw reply [relevance 0%]
* RE: [RFC v2 0/2] Add high-performance timer facility
2024-10-03 18:36 0% ` Stephen Hemminger
@ 2024-10-03 21:32 0% ` Morten Brørup
2024-10-06 13:02 0% ` Mattias Rönnblom
0 siblings, 1 reply; 200+ results
From: Morten Brørup @ 2024-10-03 21:32 UTC (permalink / raw)
To: Stephen Hemminger, Mattias Rönnblom
Cc: dev, Erik Gabriel Carrillo, David Marchand, maria.lingemark,
Stefan Sundkvist, Tyler Retzlaff
> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> Sent: Thursday, 3 October 2024 20.37
>
> On Wed, 15 Mar 2023 18:03:40 +0100
> Mattias Rönnblom <mattias.ronnblom@ericsson.com> wrote:
>
> > This patchset is an attempt to introduce a high-performance, highly
> > scalable timer facility into DPDK.
> >
> > More specifically, the goals for the htimer library are:
> >
> > * Efficient handling of a handful up to hundreds of thousands of
> > concurrent timers.
> > * Make adding and canceling timers low-overhead, constant-time
> > operations.
> > * Provide a service functionally equivalent to that of
> > <rte_timer.h>. API/ABI backward compatibility is secondary.
>
> Worthwhile goals, and the problem needs to be addressed.
> But this patch never got accepted.
I think work on it was put on hold due to the requested changes requiring a significant development effort.
I too look forward to work on this being resumed. ;-)
>
> Please fix/improve/extend existing rte_timer instead.
The rte_timer API is too "fat" for use in the fast path with millions of timers, e.g. TCP flow timers.
Shoehorning a fast path feature into a slow path API is not going to cut it. I support having a separate htimer library with its own API for high volume, high-performance fast path timers.
When striving for low latency across the internet, timing is everything. Packet pacing is the "new" hot thing in congestion control algorithms, and a simple software implementation would require a timer firing once per packet.
^ permalink raw reply [relevance 0%]
* Re: [RFC v2 0/2] Add high-performance timer facility
@ 2024-10-03 18:36 0% ` Stephen Hemminger
2024-10-03 21:32 0% ` Morten Brørup
0 siblings, 1 reply; 200+ results
From: Stephen Hemminger @ 2024-10-03 18:36 UTC (permalink / raw)
To: Mattias Rönnblom
Cc: dev, Erik Gabriel Carrillo, David Marchand, maria.lingemark,
Stefan Sundkvist, Morten Brørup, Tyler Retzlaff
On Wed, 15 Mar 2023 18:03:40 +0100
Mattias Rönnblom <mattias.ronnblom@ericsson.com> wrote:
> This patchset is an attempt to introduce a high-performance, highly
> scalable timer facility into DPDK.
>
> More specifically, the goals for the htimer library are:
>
> * Efficient handling of a handful up to hundreds of thousands of
> concurrent timers.
> * Make adding and canceling timers low-overhead, constant-time
> operations.
> * Provide a service functionally equivalent to that of
> <rte_timer.h>. API/ABI backward compatibility is secondary.
Worthwhile goals, and the problem needs to be addressed.
But this patch never got accepted.
Please fix/improve/extend existing rte_timer instead.
^ permalink raw reply [relevance 0%]
* [PATCH v3 1/2] ipsec: allow stateless IPsec processing
@ 2024-10-03 13:45 3% ` Aakash Sasidharan
2024-10-04 6:34 3% ` [PATCH v4 " Aakash Sasidharan
0 siblings, 1 reply; 200+ results
From: Aakash Sasidharan @ 2024-10-03 13:45 UTC (permalink / raw)
To: Konstantin Ananyev, Vladimir Medvedkin
Cc: gakhil, jerinj, anoobj, vvelumuri, asasidharan, dev
Introduce stateless packet preparation API for IPsec
processing. The new API would allow preparation of IPsec
packets without altering the internal state of an IPsec
session.
For outbound IPsec processing, the change enables user to
provide sequence number to be used for the IPsec operation.
Signed-off-by: Aakash Sasidharan <asasidharan@marvell.com>
---
doc/guides/prog_guide/ipsec_lib.rst | 6 ++
doc/guides/rel_notes/release_24_11.rst | 6 ++
lib/ipsec/esp_outb.c | 101 +++++++++++++++++--------
lib/ipsec/rte_ipsec.h | 84 ++++++++++++++++++++
lib/ipsec/sa.c | 4 +-
lib/ipsec/sa.h | 8 ++
6 files changed, 176 insertions(+), 33 deletions(-)
diff --git a/doc/guides/prog_guide/ipsec_lib.rst b/doc/guides/prog_guide/ipsec_lib.rst
index 0bdbdad1e4..395512f4f9 100644
--- a/doc/guides/prog_guide/ipsec_lib.rst
+++ b/doc/guides/prog_guide/ipsec_lib.rst
@@ -328,6 +328,12 @@ statistics. Per SA telemetry statistics can be enabled using
``rte_ipsec_telemetry_sa_add`` and disabled using
``rte_ipsec_telemetry_sa_del``. Note that these calls are not thread safe.
+Stateless IPsec packet processing support
+-----------------------------------------
+Support for stateless IPsec packet processing allowing use of custom
+sequence number to be used for IPsec outbound processing.
+rte_ipsec_pkt_stateless_prepare() takes as input the state parameter
+from the application and prepares the packet for IPsec processing.
Limitations
-----------
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 0ff70d9057..0964f13a12 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -84,6 +84,12 @@ API Changes
Also, make sure to start the actual text at the margin.
=======================================================
+ * ipsec: Add support for stateless IPsec processing.
+
+ Added new API to support stateless IPsec processing.
+ The new API are ``rte_ipsec_pkr_crypto_prepare_stateless`` and
+ ``rte_ipsec_pkt_cpu_prepare_stateless`` .
+
ABI Changes
-----------
diff --git a/lib/ipsec/esp_outb.c b/lib/ipsec/esp_outb.c
index ec87b1dce2..617fc52b21 100644
--- a/lib/ipsec/esp_outb.c
+++ b/lib/ipsec/esp_outb.c
@@ -288,28 +288,22 @@ outb_pkt_xprepare(const struct rte_ipsec_sa *sa, rte_be64_t sqc,
/*
* setup/update packets and crypto ops for ESP outbound tunnel case.
*/
-uint16_t
-esp_outb_tun_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
- struct rte_crypto_op *cop[], uint16_t num)
+static inline uint16_t
+esp_outb_tun_prepare_helper(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
+ struct rte_crypto_op *cop[], uint16_t n, uint64_t sqn)
{
int32_t rc;
- uint32_t i, k, n;
- uint64_t sqn;
+ uint32_t i, k;
rte_be64_t sqc;
struct rte_ipsec_sa *sa;
struct rte_cryptodev_sym_session *cs;
union sym_op_data icv;
uint64_t iv[IPSEC_MAX_IV_QWORD];
- uint32_t dr[num];
+ uint32_t dr[n];
sa = ss->sa;
cs = ss->crypto.ses;
- n = num;
- sqn = esn_outb_update_sqn(sa, &n);
- if (n != num)
- rte_errno = EOVERFLOW;
-
k = 0;
for (i = 0; i != n; i++) {
@@ -339,6 +333,30 @@ esp_outb_tun_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
return k;
}
+uint16_t
+esp_outb_tun_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
+ struct rte_crypto_op *cop[], uint16_t num)
+{
+ uint64_t sqn;
+ uint32_t n;
+
+ n = num;
+ sqn = esn_outb_update_sqn(ss->sa, &n);
+ if (n != num)
+ rte_errno = EOVERFLOW;
+
+ return esp_outb_tun_prepare_helper(ss, mb, cop, n, sqn);
+}
+
+uint16_t
+esp_outb_tun_prepare_stateless(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
+ struct rte_crypto_op *cop[], uint16_t num, struct rte_ipsec_state *state)
+{
+ uint64_t sqn = state->sqn;
+
+ return esp_outb_tun_prepare_helper(ss, mb, cop, num, sqn);
+}
+
/*
* setup/update packet data and metadata for ESP outbound transport case.
*/
@@ -529,33 +547,27 @@ outb_cpu_crypto_prepare(const struct rte_ipsec_sa *sa, uint32_t *pofs,
return clen;
}
-static uint16_t
-cpu_outb_pkt_prepare(const struct rte_ipsec_session *ss,
- struct rte_mbuf *mb[], uint16_t num,
- esp_outb_prepare_t prepare, uint32_t cofs_mask)
+static inline uint16_t
+cpu_outb_pkt_prepare_helper(const struct rte_ipsec_session *ss,
+ struct rte_mbuf *mb[], uint16_t n, esp_outb_prepare_t prepare,
+ uint32_t cofs_mask, uint64_t sqn)
{
int32_t rc;
- uint64_t sqn;
rte_be64_t sqc;
struct rte_ipsec_sa *sa;
- uint32_t i, k, n;
+ uint32_t i, k;
uint32_t l2, l3;
union sym_op_data icv;
- struct rte_crypto_va_iova_ptr iv[num];
- struct rte_crypto_va_iova_ptr aad[num];
- struct rte_crypto_va_iova_ptr dgst[num];
- uint32_t dr[num];
- uint32_t l4ofs[num];
- uint32_t clen[num];
- uint64_t ivbuf[num][IPSEC_MAX_IV_QWORD];
+ struct rte_crypto_va_iova_ptr iv[n];
+ struct rte_crypto_va_iova_ptr aad[n];
+ struct rte_crypto_va_iova_ptr dgst[n];
+ uint32_t dr[n];
+ uint32_t l4ofs[n];
+ uint32_t clen[n];
+ uint64_t ivbuf[n][IPSEC_MAX_IV_QWORD];
sa = ss->sa;
- n = num;
- sqn = esn_outb_update_sqn(sa, &n);
- if (n != num)
- rte_errno = EOVERFLOW;
-
for (i = 0, k = 0; i != n; i++) {
l2 = mb[i]->l2_len;
@@ -604,15 +616,40 @@ uint16_t
cpu_outb_tun_pkt_prepare(const struct rte_ipsec_session *ss,
struct rte_mbuf *mb[], uint16_t num)
{
- return cpu_outb_pkt_prepare(ss, mb, num, outb_tun_pkt_prepare, 0);
+ uint64_t sqn;
+ uint32_t n;
+
+ n = num;
+ sqn = esn_outb_update_sqn(ss->sa, &n);
+ if (n != num)
+ rte_errno = EOVERFLOW;
+
+ return cpu_outb_pkt_prepare_helper(ss, mb, n, outb_tun_pkt_prepare, 0, sqn);
+}
+
+uint16_t
+cpu_outb_tun_pkt_prepare_stateless(const struct rte_ipsec_session *ss,
+ struct rte_mbuf *mb[], uint16_t num, struct rte_ipsec_state *state)
+{
+ uint64_t sqn = state->sqn;
+
+ return cpu_outb_pkt_prepare_helper(ss, mb, num, outb_tun_pkt_prepare, 0, sqn);
}
uint16_t
cpu_outb_trs_pkt_prepare(const struct rte_ipsec_session *ss,
struct rte_mbuf *mb[], uint16_t num)
{
- return cpu_outb_pkt_prepare(ss, mb, num, outb_trs_pkt_prepare,
- UINT32_MAX);
+ uint64_t sqn;
+ uint32_t n;
+
+ n = num;
+ sqn = esn_outb_update_sqn(ss->sa, &n);
+ if (n != num)
+ rte_errno = EOVERFLOW;
+
+ return cpu_outb_pkt_prepare_helper(ss, mb, n, outb_trs_pkt_prepare,
+ UINT32_MAX, sqn);
}
/*
diff --git a/lib/ipsec/rte_ipsec.h b/lib/ipsec/rte_ipsec.h
index f15f6f2966..316ace1f19 100644
--- a/lib/ipsec/rte_ipsec.h
+++ b/lib/ipsec/rte_ipsec.h
@@ -23,11 +23,24 @@ extern "C" {
struct rte_ipsec_session;
+/**
+ * IPsec state for stateless processing of a batch of IPsec packets.
+ */
+struct rte_ipsec_state {
+ /**
+ * 64 bit sequence number to be used for the first packet of the
+ * batch of packets.
+ */
+ uint64_t sqn;
+};
+
/**
* IPsec session specific functions that will be used to:
* - prepare - for input mbufs and given IPsec session prepare crypto ops
* that can be enqueued into the cryptodev associated with given session
* (see *rte_ipsec_pkt_crypto_prepare* below for more details).
+ * - prepare_stateless - similar to prepare, but further processing is done
+ * based on IPsec state provided by the 'state' parameter.
* - process - finalize processing of packets after crypto-dev finished
* with them or process packets that are subjects to inline IPsec offload
* (see rte_ipsec_pkt_process for more details).
@@ -42,6 +55,17 @@ struct rte_ipsec_sa_pkt_func {
struct rte_mbuf *mb[],
uint16_t num);
} prepare;
+ union {
+ uint16_t (*async)(const struct rte_ipsec_session *ss,
+ struct rte_mbuf *mb[],
+ struct rte_crypto_op *cop[],
+ uint16_t num,
+ struct rte_ipsec_state *state);
+ uint16_t (*sync)(const struct rte_ipsec_session *ss,
+ struct rte_mbuf *mb[],
+ uint16_t num,
+ struct rte_ipsec_state *state);
+ } prepare_stateless;
uint16_t (*process)(const struct rte_ipsec_session *ss,
struct rte_mbuf *mb[],
uint16_t num);
@@ -128,6 +152,66 @@ rte_ipsec_pkt_cpu_prepare(const struct rte_ipsec_session *ss,
return ss->pkt_func.prepare.sync(ss, mb, num);
}
+/**
+ * Same as *rte_ipsec_pkt_crypto_prepare*, but processing is done based on
+ * IPsec state provided by the 'state' parameter. Internal IPsec state won't
+ * be updated when this API is called.
+ *
+ * For input mbufs and given IPsec session prepare crypto ops that can be
+ * enqueued into the cryptodev associated with given session.
+ * expects that for each input packet:
+ * - l2_len, l3_len are setup correctly
+ * Note that erroneous mbufs are not freed by the function,
+ * but are placed beyond last valid mbuf in the *mb* array.
+ * It is a user responsibility to handle them further.
+ * @param ss
+ * Pointer to the *rte_ipsec_session* object the packets belong to.
+ * @param mb
+ * The address of an array of *num* pointers to *rte_mbuf* structures
+ * which contain the input packets.
+ * @param cop
+ * The address of an array of *num* pointers to the output *rte_crypto_op*
+ * structures.
+ * @param num
+ * The maximum number of packets to process.
+ * @param state
+ * The IPsec state to be used for processing current batch of packets.
+ * @return
+ * Number of successfully processed packets, with error code set in rte_errno.
+ */
+__rte_experimental
+static inline uint16_t
+rte_ipsec_pkt_crypto_prepare_stateless(const struct rte_ipsec_session *ss,
+ struct rte_mbuf *mb[], struct rte_crypto_op *cop[], uint16_t num,
+ struct rte_ipsec_state *state)
+{
+ return ss->pkt_func.prepare_stateless.async(ss, mb, cop, num, state);
+}
+
+/**
+ * Same as *rte_ipsec_pkt_crypto_prepare_stateless*, but processing is done
+ * in synchronous mode.
+ *
+ * @param ss
+ * Pointer to the *rte_ipsec_session* object the packets belong to.
+ * @param mb
+ * The address of an array of *num* pointers to *rte_mbuf* structures
+ * which contain the input packets.
+ * @param num
+ * The maximum number of packets to process.
+ * @param state
+ * The IPsec state to be used for processing current batch of packets.
+ * @return
+ * Number of successfully processed packets, with error code set in rte_errno.
+ */
+__rte_experimental
+static inline uint16_t
+rte_ipsec_pkt_cpu_prepare_stateless(const struct rte_ipsec_session *ss,
+ struct rte_mbuf *mb[], uint16_t num, struct rte_ipsec_state *state)
+{
+ return ss->pkt_func.prepare_stateless.sync(ss, mb, num, state);
+}
+
/**
* Finalise processing of packets after crypto-dev finished with them or
* process packets that are subjects to inline IPsec offload.
diff --git a/lib/ipsec/sa.c b/lib/ipsec/sa.c
index 2297bd6d72..741e079831 100644
--- a/lib/ipsec/sa.c
+++ b/lib/ipsec/sa.c
@@ -710,6 +710,7 @@ lksd_none_pkt_func_select(const struct rte_ipsec_sa *sa,
case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV4):
case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV6):
pf->prepare.async = esp_outb_tun_prepare;
+ pf->prepare_stateless.async = esp_outb_tun_prepare_stateless;
pf->process = (sa->sqh_len != 0) ?
esp_outb_sqh_process : pkt_flag_process;
break;
@@ -748,6 +749,7 @@ cpu_crypto_pkt_func_select(const struct rte_ipsec_sa *sa,
case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV4):
case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV6):
pf->prepare.sync = cpu_outb_tun_pkt_prepare;
+ pf->prepare_stateless.sync = cpu_outb_tun_pkt_prepare_stateless;
pf->process = (sa->sqh_len != 0) ?
esp_outb_sqh_process : pkt_flag_process;
break;
@@ -810,7 +812,7 @@ ipsec_sa_pkt_func_select(const struct rte_ipsec_session *ss,
int32_t rc;
rc = 0;
- pf[0] = (struct rte_ipsec_sa_pkt_func) { {NULL}, NULL };
+ pf[0] = (struct rte_ipsec_sa_pkt_func) { {NULL}, {NULL}, NULL };
switch (ss->type) {
case RTE_SECURITY_ACTION_TYPE_NONE:
diff --git a/lib/ipsec/sa.h b/lib/ipsec/sa.h
index 719b5c735c..9b53586b2d 100644
--- a/lib/ipsec/sa.h
+++ b/lib/ipsec/sa.h
@@ -179,6 +179,10 @@ uint16_t
esp_outb_tun_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
struct rte_crypto_op *cop[], uint16_t num);
+uint16_t
+esp_outb_tun_prepare_stateless(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
+ struct rte_crypto_op *cop[], uint16_t num, struct rte_ipsec_state *state);
+
uint16_t
esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
struct rte_crypto_op *cop[], uint16_t num);
@@ -207,6 +211,10 @@ uint16_t
cpu_outb_tun_pkt_prepare(const struct rte_ipsec_session *ss,
struct rte_mbuf *mb[], uint16_t num);
uint16_t
+cpu_outb_tun_pkt_prepare_stateless(const struct rte_ipsec_session *ss,
+ struct rte_mbuf *mb[], uint16_t num, struct rte_ipsec_state *state);
+
+uint16_t
cpu_outb_trs_pkt_prepare(const struct rte_ipsec_session *ss,
struct rte_mbuf *mb[], uint16_t num);
--
2.25.1
^ permalink raw reply [relevance 3%]
* [PATCH v4 1/1] dmadev: support priority configuration
2024-10-03 11:41 4% ` [PATCH v3 " Vamsi Krishna
@ 2024-10-03 11:53 4% ` Vamsi Krishna
2024-10-08 2:36 0% ` fengchengwen
0 siblings, 1 reply; 200+ results
From: Vamsi Krishna @ 2024-10-03 11:53 UTC (permalink / raw)
To: thomas, fengchengwen, bruce.richardson, mb
Cc: dev, kevin.laatz, jerinj, conor.walsh, gmuthukrishn, vvelumuri,
g.singh, sachin.saxena, hemant.agrawal, Vamsi Attunuru,
Amit Prakash Shukla
From: Vamsi Attunuru <vattunuru@marvell.com>
Some DMA controllers offer the ability to configure priority level
for the hardware command queues, allowing for the prioritization of
DMA command execution based on queue importance.
This patch introduces the necessary fields in the dmadev structures to
retrieve information about the hardware-supported priority levels and to
enable priority configuration from the application.
Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
V4 changes:
* Rebased onto the latest
V3 changes:
* Corrected patch title
V2 changes:
* Reverted removed text from release_24_11.rst
V1 changes:
* Added trace support
* Added new capability flag
Deprecation notice:
https://patches.dpdk.org/project/dpdk/patch/20240730144612.2132848-1-amitprakashs@marvell.com/
* Assuming we do not anticipate any advanced scheduling schemes for dmadev queues,
this patch is intended to support a strict priority scheme.
doc/guides/rel_notes/release_24_11.rst | 8 ++++++++
lib/dmadev/rte_dmadev.c | 15 +++++++++++++++
lib/dmadev/rte_dmadev.h | 21 +++++++++++++++++++++
lib/dmadev/rte_dmadev_trace.h | 2 ++
4 files changed, 46 insertions(+)
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index e0a9aa55a1..9672d8c679 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -67,6 +67,11 @@ New Features
The new statistics are useful for debugging and profiling.
+* **Added strict priority capability flag in dmadev.**
+
+ Added new capability flag ``RTE_DMA_CAPA_PRI_POLICY_SP`` to check if the
+ DMA device supports assigning fixed priority to its channels, allowing
+ for better control over resource allocation and scheduling.
Removed Items
-------------
@@ -112,6 +117,9 @@ ABI Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* dmadev: Added ``nb_priorities`` field to ``rte_dma_info`` structure and
+ ``priority`` field to ``rte_dma_conf`` structure to get device supported
+ priority levels and configure required priority from the application.
Known Issues
------------
diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
index 845727210f..3d9063dee3 100644
--- a/lib/dmadev/rte_dmadev.c
+++ b/lib/dmadev/rte_dmadev.c
@@ -497,6 +497,21 @@ rte_dma_configure(int16_t dev_id, const struct rte_dma_conf *dev_conf)
return -EINVAL;
}
+ if (dev_conf->priority && !(dev_info.dev_capa & RTE_DMA_CAPA_PRI_POLICY_SP)) {
+ RTE_DMA_LOG(ERR, "Device %d don't support prioritization", dev_id);
+ return -EINVAL;
+ }
+
+ if (dev_info.nb_priorities == 1) {
+ RTE_DMA_LOG(ERR, "Device %d must support more than 1 priority, or else 0", dev_id);
+ return -EINVAL;
+ }
+
+ if (dev_info.nb_priorities && (dev_conf->priority >= dev_info.nb_priorities)) {
+ RTE_DMA_LOG(ERR, "Device %d configure invalid priority", dev_id);
+ return -EINVAL;
+ }
+
if (*dev->dev_ops->dev_configure == NULL)
return -ENOTSUP;
ret = (*dev->dev_ops->dev_configure)(dev, dev_conf,
diff --git a/lib/dmadev/rte_dmadev.h b/lib/dmadev/rte_dmadev.h
index 5474a5281d..e5f730c327 100644
--- a/lib/dmadev/rte_dmadev.h
+++ b/lib/dmadev/rte_dmadev.h
@@ -268,6 +268,16 @@ int16_t rte_dma_next_dev(int16_t start_dev_id);
#define RTE_DMA_CAPA_OPS_COPY_SG RTE_BIT64(33)
/** Support fill operation. */
#define RTE_DMA_CAPA_OPS_FILL RTE_BIT64(34)
+/** Support strict prioritization at DMA HW channel level
+ *
+ * If device supports HW channel prioritization then application could
+ * assign fixed priority to the DMA HW channel using 'priority' field in
+ * struct rte_dma_conf. Number of supported priority levels will be known
+ * from 'nb_priorities' field in struct rte_dma_info.
+ *
+ * DMA devices which support prioritization can advertise this capability.
+ */
+#define RTE_DMA_CAPA_PRI_POLICY_SP RTE_BIT64(35)
/**@}*/
/**
@@ -297,6 +307,10 @@ struct rte_dma_info {
int16_t numa_node;
/** Number of virtual DMA channel configured. */
uint16_t nb_vchans;
+ /** Number of priority levels (must be > 1), if supported by DMA HW channel.
+ * 0 otherwise.
+ */
+ uint16_t nb_priorities;
};
/**
@@ -332,6 +346,13 @@ struct rte_dma_conf {
* @see RTE_DMA_CAPA_SILENT
*/
bool enable_silent;
+ /* The priority of the DMA HW channel.
+ * This value cannot be greater than or equal to the field 'nb_priorities'
+ * of struct rte_dma_info which get from rte_dma_info_get().
+ * Among the values between '0' and 'nb_priorities - 1', lowest value
+ * indicates higher priority and vice-versa.
+ */
+ uint16_t priority;
};
/**
diff --git a/lib/dmadev/rte_dmadev_trace.h b/lib/dmadev/rte_dmadev_trace.h
index e55c4c6091..be089c065c 100644
--- a/lib/dmadev/rte_dmadev_trace.h
+++ b/lib/dmadev/rte_dmadev_trace.h
@@ -35,6 +35,7 @@ RTE_TRACE_POINT(
rte_trace_point_emit_u16(dev_info->max_sges);
rte_trace_point_emit_i16(dev_info->numa_node);
rte_trace_point_emit_u16(dev_info->nb_vchans);
+ rte_trace_point_emit_u16(dev_info->nb_priorities);
)
RTE_TRACE_POINT(
@@ -48,6 +49,7 @@ RTE_TRACE_POINT(
int enable_silent = (int)dev_conf->enable_silent;
rte_trace_point_emit_i16(dev_id);
rte_trace_point_emit_u16(dev_conf->nb_vchans);
+ rte_trace_point_emit_u16(dev_conf->priority);
rte_trace_point_emit_int(enable_silent);
rte_trace_point_emit_int(ret);
)
--
2.34.1
^ permalink raw reply [relevance 4%]
* [PATCH v3 1/1] dmadev: support priority configuration
2024-09-18 10:13 4% ` [RFC v2 " Vamsi Krishna
@ 2024-10-03 11:41 4% ` Vamsi Krishna
2024-10-03 11:53 4% ` [PATCH v4 " Vamsi Krishna
0 siblings, 1 reply; 200+ results
From: Vamsi Krishna @ 2024-10-03 11:41 UTC (permalink / raw)
To: thomas, fengchengwen, bruce.richardson, mb
Cc: dev, kevin.laatz, jerinj, conor.walsh, gmuthukrishn, vvelumuri,
g.singh, sachin.saxena, hemant.agrawal, Vamsi Attunuru,
Amit Prakash Shukla
From: Vamsi Attunuru <vattunuru@marvell.com>
Some DMA controllers offer the ability to configure priority level
for the hardware command queues, allowing for the prioritization of
DMA command execution based on queue importance.
This patch introduces the necessary fields in the dmadev structures to
retrieve information about the hardware-supported priority levels and to
enable priority configuration from the application.
Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
V3 changes:
* Corrected patch title
V2 changes:
* Reverted removed text from release_24_11.rst
V1 changes:
* Added trace support
* Added new capability flag
Deprecation notice:
https://patches.dpdk.org/project/dpdk/patch/20240730144612.2132848-1-amitprakashs@marvell.com/
* Assuming we do not anticipate any advanced scheduling schemes for dmadev queues,
this patch is intended to support a strict priority scheme.
doc/guides/rel_notes/release_24_11.rst | 8 ++++++++
lib/dmadev/rte_dmadev.c | 15 +++++++++++++++
lib/dmadev/rte_dmadev.h | 21 +++++++++++++++++++++
lib/dmadev/rte_dmadev_trace.h | 2 ++
4 files changed, 46 insertions(+)
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 0ff70d9057..fc3610deff 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -55,6 +55,11 @@ New Features
Also, make sure to start the actual text at the margin.
=======================================================
+* **Added strict priority capability flag in dmadev.**
+
+ Added new capability flag ``RTE_DMA_CAPA_PRI_POLICY_SP`` to check if the
+ DMA device supports assigning fixed priority to its channels, allowing
+ for better control over resource allocation and scheduling.
Removed Items
-------------
@@ -100,6 +105,9 @@ ABI Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* dmadev: Added ``nb_priorities`` field to ``rte_dma_info`` structure and
+ ``priority`` field to ``rte_dma_conf`` structure to get device supported
+ priority levels and configure required priority from the application.
Known Issues
------------
diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
index 845727210f..3d9063dee3 100644
--- a/lib/dmadev/rte_dmadev.c
+++ b/lib/dmadev/rte_dmadev.c
@@ -497,6 +497,21 @@ rte_dma_configure(int16_t dev_id, const struct rte_dma_conf *dev_conf)
return -EINVAL;
}
+ if (dev_conf->priority && !(dev_info.dev_capa & RTE_DMA_CAPA_PRI_POLICY_SP)) {
+ RTE_DMA_LOG(ERR, "Device %d don't support prioritization", dev_id);
+ return -EINVAL;
+ }
+
+ if (dev_info.nb_priorities == 1) {
+ RTE_DMA_LOG(ERR, "Device %d must support more than 1 priority, or else 0", dev_id);
+ return -EINVAL;
+ }
+
+ if (dev_info.nb_priorities && (dev_conf->priority >= dev_info.nb_priorities)) {
+ RTE_DMA_LOG(ERR, "Device %d configure invalid priority", dev_id);
+ return -EINVAL;
+ }
+
if (*dev->dev_ops->dev_configure == NULL)
return -ENOTSUP;
ret = (*dev->dev_ops->dev_configure)(dev, dev_conf,
diff --git a/lib/dmadev/rte_dmadev.h b/lib/dmadev/rte_dmadev.h
index 5474a5281d..e5f730c327 100644
--- a/lib/dmadev/rte_dmadev.h
+++ b/lib/dmadev/rte_dmadev.h
@@ -268,6 +268,16 @@ int16_t rte_dma_next_dev(int16_t start_dev_id);
#define RTE_DMA_CAPA_OPS_COPY_SG RTE_BIT64(33)
/** Support fill operation. */
#define RTE_DMA_CAPA_OPS_FILL RTE_BIT64(34)
+/** Support strict prioritization at DMA HW channel level
+ *
+ * If device supports HW channel prioritization then application could
+ * assign fixed priority to the DMA HW channel using 'priority' field in
+ * struct rte_dma_conf. Number of supported priority levels will be known
+ * from 'nb_priorities' field in struct rte_dma_info.
+ *
+ * DMA devices which support prioritization can advertise this capability.
+ */
+#define RTE_DMA_CAPA_PRI_POLICY_SP RTE_BIT64(35)
/**@}*/
/**
@@ -297,6 +307,10 @@ struct rte_dma_info {
int16_t numa_node;
/** Number of virtual DMA channel configured. */
uint16_t nb_vchans;
+ /** Number of priority levels (must be > 1), if supported by DMA HW channel.
+ * 0 otherwise.
+ */
+ uint16_t nb_priorities;
};
/**
@@ -332,6 +346,13 @@ struct rte_dma_conf {
* @see RTE_DMA_CAPA_SILENT
*/
bool enable_silent;
+ /* The priority of the DMA HW channel.
+ * This value cannot be greater than or equal to the field 'nb_priorities'
+ * of struct rte_dma_info which get from rte_dma_info_get().
+ * Among the values between '0' and 'nb_priorities - 1', lowest value
+ * indicates higher priority and vice-versa.
+ */
+ uint16_t priority;
};
/**
diff --git a/lib/dmadev/rte_dmadev_trace.h b/lib/dmadev/rte_dmadev_trace.h
index e55c4c6091..be089c065c 100644
--- a/lib/dmadev/rte_dmadev_trace.h
+++ b/lib/dmadev/rte_dmadev_trace.h
@@ -35,6 +35,7 @@ RTE_TRACE_POINT(
rte_trace_point_emit_u16(dev_info->max_sges);
rte_trace_point_emit_i16(dev_info->numa_node);
rte_trace_point_emit_u16(dev_info->nb_vchans);
+ rte_trace_point_emit_u16(dev_info->nb_priorities);
)
RTE_TRACE_POINT(
@@ -48,6 +49,7 @@ RTE_TRACE_POINT(
int enable_silent = (int)dev_conf->enable_silent;
rte_trace_point_emit_i16(dev_id);
rte_trace_point_emit_u16(dev_conf->nb_vchans);
+ rte_trace_point_emit_u16(dev_conf->priority);
rte_trace_point_emit_int(enable_silent);
rte_trace_point_emit_int(ret);
)
--
2.34.1
^ permalink raw reply [relevance 4%]
* Re: [PATCH 2/2] ethdev: fix race on ports for telemetry commands
2024-10-03 9:46 3% ` Bruce Richardson
@ 2024-10-03 9:58 0% ` David Marchand
0 siblings, 0 replies; 200+ results
From: David Marchand @ 2024-10-03 9:58 UTC (permalink / raw)
To: Bruce Richardson
Cc: Robin Jarry, dev, ktraynor, stable, Thomas Monjalon,
Ferruh Yigit, Andrew Rybchenko, Keith Wiles, Ciara Power
On Thu, Oct 3, 2024 at 11:46 AM Bruce Richardson
<bruce.richardson@intel.com> wrote:
>
> On Wed, Oct 02, 2024 at 09:26:10PM +0200, Robin Jarry wrote:
> > David Marchand, Oct 02, 2024 at 21:18:
> > > On Wed, Oct 2, 2024 at 9:09 PM Robin Jarry <rjarry@redhat.com> wrote:
> > > > I was going to suggest adding a rte_spinlock_t* parameter to a new
> > > > telemetry register function that would need to be held while the
> > > > callback is invoked. Or if we want to keep doors open to other kinds of
> > > > lock, a wrapper callback.
> > >
> > > Well, as you had experimented this approach, we know this does not
> > > work: the ethdev lock is in dpdk shared memory which is not available
> > > yet at the time RTE_INIT() is called.
> > >
> > > A single callback is strange, I guess you mean pre/post callbacks then.
> >
> > It could be a single function that will wrap the callbacks. E.g.:
> >
> > static int
> > eth_dev_telemetry_with_lock(
> > telemetry_cb fn, const char *cmd, const char *params, struct rte_tel_data *d)
> > {
> > int ret;
> > rte_spinlock_lock(rte_mcfg_ethdev_get_lock());
> > ret = fn(cmd, params, d);
> > rte_spinlock_unlock(rte_mcfg_ethdev_get_lock());
> > return ret;
> > }
> >
> > RTE_INIT(ethdev_init_telemetry)
> > {
> > ....
> > rte_telemetry_register_cmd("/ethdev/stats", eth_dev_handle_port_stats,
> > "Returns the common stats for a port. Parameters: int port_id",
> > eth_dev_telemetry_with_lock);
> > ....
> > }
> >
> > I'm not sure which solution is the uglier :D
> >
>
> I don't actually mind this latter solution, except that the order of the
> parameters should be reversed (and it breaks the ABI, unless we add a
> special new function for it) For me, the wrapper function should be the
> main callback, and the real (unwrapped) function the extra parameter to be
> called. That extra parameter to callbacks should just be a generic pointer,
> so it can be data or function that is passed around.
>
> rte_telemetry_register_param_cmd(const char *cmd, telemetry_cb fn,
> void *param, const char *help)
>
> Or more specifically:
>
>
> rte_telemetry_register_param_cmd("/ethdev/stats",
> eth_dev_telemetry_with_lock, /* callback */
> eth_dev_handle_port_stats, /* parameter */
> "Returns the common stats for a port. Parameters: int port_id");
Ok, this way seems nicer.
I'll let Robin submit a v2.
--
David Marchand
^ permalink raw reply [relevance 0%]
* Re: [PATCH 2/2] ethdev: fix race on ports for telemetry commands
@ 2024-10-03 9:46 3% ` Bruce Richardson
2024-10-03 9:58 0% ` David Marchand
0 siblings, 1 reply; 200+ results
From: Bruce Richardson @ 2024-10-03 9:46 UTC (permalink / raw)
To: Robin Jarry
Cc: David Marchand, dev, ktraynor, stable, Thomas Monjalon,
Ferruh Yigit, Andrew Rybchenko, Keith Wiles, Ciara Power
On Wed, Oct 02, 2024 at 09:26:10PM +0200, Robin Jarry wrote:
> David Marchand, Oct 02, 2024 at 21:18:
> > On Wed, Oct 2, 2024 at 9:09 PM Robin Jarry <rjarry@redhat.com> wrote:
> > > I was going to suggest adding a rte_spinlock_t* parameter to a new
> > > telemetry register function that would need to be held while the
> > > callback is invoked. Or if we want to keep doors open to other kinds of
> > > lock, a wrapper callback.
> >
> > Well, as you had experimented this approach, we know this does not
> > work: the ethdev lock is in dpdk shared memory which is not available
> > yet at the time RTE_INIT() is called.
> >
> > A single callback is strange, I guess you mean pre/post callbacks then.
>
> It could be a single function that will wrap the callbacks. E.g.:
>
> static int
> eth_dev_telemetry_with_lock(
> telemetry_cb fn, const char *cmd, const char *params, struct rte_tel_data *d)
> {
> int ret;
> rte_spinlock_lock(rte_mcfg_ethdev_get_lock());
> ret = fn(cmd, params, d);
> rte_spinlock_unlock(rte_mcfg_ethdev_get_lock());
> return ret;
> }
>
> RTE_INIT(ethdev_init_telemetry)
> {
> ....
> rte_telemetry_register_cmd("/ethdev/stats", eth_dev_handle_port_stats,
> "Returns the common stats for a port. Parameters: int port_id",
> eth_dev_telemetry_with_lock);
> ....
> }
>
> I'm not sure which solution is the uglier :D
>
I don't actually mind this latter solution, except that the order of the
parameters should be reversed (and it breaks the ABI, unless we add a
special new function for it) For me, the wrapper function should be the
main callback, and the real (unwrapped) function the extra parameter to be
called. That extra parameter to callbacks should just be a generic pointer,
so it can be data or function that is passed around.
rte_telemetry_register_param_cmd(const char *cmd, telemetry_cb fn,
void *param, const char *help)
Or more specifically:
rte_telemetry_register_param_cmd("/ethdev/stats",
eth_dev_telemetry_with_lock, /* callback */
eth_dev_handle_port_stats, /* parameter */
"Returns the common stats for a port. Parameters: int port_id");
/Bruce
PS: Yes, I realise that using void * as function pointers is not always
recommended, but we already use dlsym which does so!
^ permalink raw reply [relevance 3%]
* RE: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-09-23 20:41 4% ` Akhil Goyal
@ 2024-10-03 7:00 0% ` Akhil Goyal
2024-10-06 11:10 0% ` Morten Brørup
2024-10-04 4:00 0% ` Ferruh Yigit
1 sibling, 1 reply; 200+ results
From: Akhil Goyal @ 2024-10-03 7:00 UTC (permalink / raw)
To: Akhil Goyal, Morten Brørup, fengchengwen, dev
Cc: thomas, david.marchand, hemant.agrawal, Anoob Joseph,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde,
Nagadheeraj Rottela, mdr
Hi Morten,
>
> Apologies for delayed response.
> > Maybe a combination, returning the lowest end of the two versions of the list,
> > would work...
> >
> > ----------------------------------
> > Common header file (rte_common.h):
> > ----------------------------------
> >
> > /* Add at end of enum list in the header file. */
> > #define RTE_ENUM_LIST_END(name) \
> > _ # name # _ENUM_LIST_END /**< @internal */
> >
> > /* Add somewhere in header file, preferably after the enum list. */
> > #define rte_declare_enum_list_end(name) \
> > /** @internal */ \
> > int _# name # _enum_list_end(void); \
> > \
> > static int name # _enum_list_end(void) \
> > { \
> > static int cached = 0; \
> > \
> > if (likely(cached != 0)) \
> > return cached; \
> > \
> > return cached = RTE_MIN( \
> > RTE_ENUM_LIST_END(name), \
> > _ # name # _enum_list_end()); \
> > } \
> > \
> > int _# name # _enum_list_end(void)
> >
> > /* Add in the library/driver implementation. */
> > #define rte_define_enum_list_end(name) \
> > int _# name # _enum_list_end(void) \
> > { \
> > return RTE_ENUM_LIST_END(name); \
> > } \
> > \
> > int _# name # _enum_list_end(void)
> >
> > --------------------
> > Library header file:
> > --------------------
> >
> > enum rte_crypto_asym_op_type {
> > RTE_CRYPTO_ASYM_OP_VERIFY,
> > /**< Signature Verification operation */
> > RTE_ENUM_LIST_END(rte_crypto_asym_op)
>
> Will the ABI check be ok for adding anything in between
> RTE_CRYPTO_ASYM_OP_VERIFY and
> RTE_ENUM_LIST_END(rte_crypto_asym_op)?
> Don’t we need to add exception for that if we somehow make it internal by
> adding a comment only?
> Library is actually not restricting the application to not use
> RTE_ENUM_LIST_END(rte_crypto_asym_op) directly.
>
> Also we may need to expose the .c file internal function as experimental in
> version.map
>
> > }
> >
> > rte_declare_enum_list_end(rte_crypto_asym_op);
> >
> > ---------------
> > Library C file:
> > ---------------
> >
> > rte_define_enum_list_end(rte_crypto_asym_op);
>
> If we want to make it a generic thing in rte_common.h
> Will the below change be ok?
> ----------------------------------
> Common header file (rte_common.h):
> ----------------------------------
> #define rte_define_enum_list_end(name, last_value) \
> static inline int name ## _enum_list_end(void) \
> { \
> return last_value + 1; \
> }
>
> ----------------
> Lib header file
> ----------------
> //After the enum definition define the list end as below
> rte_define_enum_list_end(rte_crypto_asym_op,
> RTE_CRYPTO_ASYM_OP_VERIFY);
>
>
> And wherever list end is needed use rte_crypto_asym_op_enum_list_end()?
>
> With this change, abi check will not complain for any new addition at the end of
> enum.
> And we do not need to expose any internal API in version.map.
>
Can we move forward with above suggestion?
^ permalink raw reply [relevance 0%]
* RE: [PATCH v2 1/3] net: add thread-safe crc api
2024-10-01 21:44 3% ` Stephen Hemminger
@ 2024-10-02 8:28 0% ` Kusztal, ArkadiuszX
0 siblings, 0 replies; 200+ results
From: Kusztal, ArkadiuszX @ 2024-10-02 8:28 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: dev, ferruh.yigit, Ji, Kai, Dooley, Brian
> -----Original Message-----
> From: Stephen Hemminger <stephen@networkplumber.org>
> Sent: Tuesday, October 1, 2024 11:44 PM
> To: Kusztal, ArkadiuszX <arkadiuszx.kusztal@intel.com>
> Cc: dev@dpdk.org; ferruh.yigit@amd.com; Ji, Kai <kai.ji@intel.com>; Dooley,
> Brian <brian.dooley@intel.com>
> Subject: Re: [PATCH v2 1/3] net: add thread-safe crc api
>
> On Tue, 1 Oct 2024 19:11:48 +0100
> Arkadiusz Kusztal <arkadiuszx.kusztal@intel.com> wrote:
>
> > diff --git a/lib/net/version.map b/lib/net/version.map index
> > bec4ce23ea..47daf1464a 100644
> > --- a/lib/net/version.map
> > +++ b/lib/net/version.map
> > @@ -4,11 +4,25 @@ DPDK_25 {
> > rte_eth_random_addr;
> > rte_ether_format_addr;
> > rte_ether_unformat_addr;
> > - rte_net_crc_calc;
> > - rte_net_crc_set_alg;
> > rte_net_get_ptype;
> > rte_net_make_rarp_packet;
> > rte_net_skip_ip6_ext;
> > + rte_net_crc;
> > + rte_net_crc_set;
> >
> > local: *;
> > };
>
> Please keep the symbols in version.map in alphabetic order.
My apologies, I will keep it in mind for the future
>
> And this would be an unannounced change to API/ABI which could cause pain
> for some users.
I know it is late for such a change, I may defer it then.
^ permalink raw reply [relevance 0%]
* Re: [PATCH v2 1/3] net: add thread-safe crc api
@ 2024-10-01 21:44 3% ` Stephen Hemminger
2024-10-02 8:28 0% ` Kusztal, ArkadiuszX
1 sibling, 1 reply; 200+ results
From: Stephen Hemminger @ 2024-10-01 21:44 UTC (permalink / raw)
To: Arkadiusz Kusztal; +Cc: dev, ferruh.yigit, kai.ji, brian.dooley
On Tue, 1 Oct 2024 19:11:48 +0100
Arkadiusz Kusztal <arkadiuszx.kusztal@intel.com> wrote:
> diff --git a/lib/net/version.map b/lib/net/version.map
> index bec4ce23ea..47daf1464a 100644
> --- a/lib/net/version.map
> +++ b/lib/net/version.map
> @@ -4,11 +4,25 @@ DPDK_25 {
> rte_eth_random_addr;
> rte_ether_format_addr;
> rte_ether_unformat_addr;
> - rte_net_crc_calc;
> - rte_net_crc_set_alg;
> rte_net_get_ptype;
> rte_net_make_rarp_packet;
> rte_net_skip_ip6_ext;
> + rte_net_crc;
> + rte_net_crc_set;
>
> local: *;
> };
Please keep the symbols in version.map in alphabetic order.
And this would be an unannounced change to API/ABI which could cause
pain for some users.
^ permalink raw reply [relevance 3%]
* [PATCH dpdk v2 13/16] rte_flow: use ipv6 addr struct
` (8 preceding siblings ...)
2024-10-01 8:17 4% ` [PATCH dpdk v2 12/16] gro: " Robin Jarry
@ 2024-10-01 8:17 3% ` Robin Jarry
2024-10-06 9:04 3% ` [PATCH dpdk v2 00/16] IPv6 APIs overhaul Morten Brørup
10 siblings, 0 replies; 200+ results
From: Robin Jarry @ 2024-10-01 8:17 UTC (permalink / raw)
To: dev, Wisam Jaddo, Potnuri Bharat Teja, Chaoyong He, Ori Kam,
Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko
Update rte_flow_tunnel, rte_flow_action_set_ipv6,
rte_flow_item_icmp6_nd_na and rte_flow_item_icmp6_nd_ns to use the
recently introduced IPv6 address structure instead of uint8_t[16]
arrays.
Update all code accordingly.
Signed-off-by: Robin Jarry <rjarry@redhat.com>
---
Notes:
v2: updated changelog for 24.11
app/test-flow-perf/actions_gen.c | 4 ++--
doc/guides/rel_notes/deprecation.rst | 10 ----------
doc/guides/rel_notes/release_24_11.rst | 2 ++
drivers/net/cxgbe/cxgbe_flow.c | 4 ++--
drivers/net/nfp/flower/nfp_flower_flow.c | 2 +-
lib/ethdev/rte_flow.h | 16 +++++++++-------
6 files changed, 16 insertions(+), 22 deletions(-)
diff --git a/app/test-flow-perf/actions_gen.c b/app/test-flow-perf/actions_gen.c
index b5336e83ff9f..54fcbacb98ce 100644
--- a/app/test-flow-perf/actions_gen.c
+++ b/app/test-flow-perf/actions_gen.c
@@ -304,7 +304,7 @@ add_set_src_ipv6(struct rte_flow_action *actions,
/* IPv6 value to set is random each time */
for (i = 0; i < 16; i++) {
- set_ipv6[para.core_idx].ipv6_addr[i] = ipv6 & 0xff;
+ set_ipv6[para.core_idx].ipv6_addr.a[i] = ipv6 & 0xff;
ipv6 = ipv6 >> 8;
}
@@ -327,7 +327,7 @@ add_set_dst_ipv6(struct rte_flow_action *actions,
/* IPv6 value to set is random each time */
for (i = 0; i < 16; i++) {
- set_ipv6[para.core_idx].ipv6_addr[i] = ipv6 & 0xff;
+ set_ipv6[para.core_idx].ipv6_addr.a[i] = ipv6 & 0xff;
ipv6 = ipv6 >> 8;
}
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index f384a4b414c9..7b77987bffa2 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -61,16 +61,6 @@ Deprecation Notices
us extending existing enum/define.
One solution can be using a fixed size array instead of ``.*MAX.*`` value.
-* net: A new IPv6 address structure will be introduced in DPDK 24.11.
- It will replace all ad-hoc ``uint8_t[16]`` arrays in all public APIs and structures.
- The following libraries and symbols are expected to be affected:
-
- ethdev
- - ``struct rte_flow_item_icmp6_nd_ns``
- - ``struct rte_flow_item_icmp6_nd_na``
- - ``struct rte_flow_action_set_ipv6``
- - ``struct rte_flow_tunnel``
-
* net, ethdev: The flow item ``RTE_FLOW_ITEM_TYPE_VXLAN_GPE``
is replaced with ``RTE_FLOW_ITEM_TYPE_VXLAN``.
The struct ``rte_flow_item_vxlan_gpe`` and its mask ``rte_flow_item_vxlan_gpe_mask``
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index d6d75033bf8f..db9da2eb7ef8 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -99,6 +99,8 @@ API Changes
``uint8_t[16]`` fields.
* gro: ``tcp6_flow_key`` was modified to use ``rte_ipv6_addr`` structures instead of ``uint8_t[16]``
fields.
+* ethdev: ``rte_flow_*`` structures were modified to use ``rte_ipv6_addr`` structures instead of
+ ``uint8_t[16]`` fields.
ABI Changes
-----------
diff --git a/drivers/net/cxgbe/cxgbe_flow.c b/drivers/net/cxgbe/cxgbe_flow.c
index 37c566b131e9..00c55af4bcc6 100644
--- a/drivers/net/cxgbe/cxgbe_flow.c
+++ b/drivers/net/cxgbe/cxgbe_flow.c
@@ -680,7 +680,7 @@ ch_rte_parse_atype_switch(const struct rte_flow_action *a,
"found.");
ipv6 = (const struct rte_flow_action_set_ipv6 *)a->conf;
- memcpy(fs->nat_fip, ipv6->ipv6_addr, sizeof(ipv6->ipv6_addr));
+ memcpy(fs->nat_fip, &ipv6->ipv6_addr, sizeof(ipv6->ipv6_addr));
*nmode |= 1 << 0;
break;
case RTE_FLOW_ACTION_TYPE_SET_IPV6_DST:
@@ -693,7 +693,7 @@ ch_rte_parse_atype_switch(const struct rte_flow_action *a,
"found.");
ipv6 = (const struct rte_flow_action_set_ipv6 *)a->conf;
- memcpy(fs->nat_lip, ipv6->ipv6_addr, sizeof(ipv6->ipv6_addr));
+ memcpy(fs->nat_lip, &ipv6->ipv6_addr, sizeof(ipv6->ipv6_addr));
*nmode |= 1 << 1;
break;
case RTE_FLOW_ACTION_TYPE_SET_TP_SRC:
diff --git a/drivers/net/nfp/flower/nfp_flower_flow.c b/drivers/net/nfp/flower/nfp_flower_flow.c
index 3f597d394b24..6db1fb141d3e 100644
--- a/drivers/net/nfp/flower/nfp_flower_flow.c
+++ b/drivers/net/nfp/flower/nfp_flower_flow.c
@@ -2978,7 +2978,7 @@ nfp_flow_action_set_ipv6(char *act_data,
set_ip->reserved = 0;
for (i = 0; i < 4; i++) {
- rte_memcpy(&tmp, &set_ipv6->ipv6_addr[i * 4], 4);
+ rte_memcpy(&tmp, &set_ipv6->ipv6_addr.a[i * 4], 4);
set_ip->ipv6[i].exact = tmp;
set_ip->ipv6[i].mask = RTE_BE32(0xffffffff);
}
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index b3ede7ccdaef..628292c1df07 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -1505,16 +1505,17 @@ struct rte_flow_item_icmp6_nd_ns {
uint8_t code; /**< ICMPv6 code, normally 0. */
rte_be16_t checksum; /**< ICMPv6 checksum. */
rte_be32_t reserved; /**< Reserved, normally 0. */
- uint8_t target_addr[16]; /**< Target address. */
+ struct rte_ipv6_addr target_addr; /**< Target address. */
};
/** Default mask for RTE_FLOW_ITEM_TYPE_ICMP6_ND_NS. */
#ifndef __cplusplus
static const
struct rte_flow_item_icmp6_nd_ns rte_flow_item_icmp6_nd_ns_mask = {
- .target_addr =
+ .target_addr = {
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
+ }
};
#endif
@@ -1532,16 +1533,17 @@ struct rte_flow_item_icmp6_nd_na {
* reserved (29b).
*/
rte_be32_t rso_reserved;
- uint8_t target_addr[16]; /**< Target address. */
+ struct rte_ipv6_addr target_addr; /**< Target address. */
};
/** Default mask for RTE_FLOW_ITEM_TYPE_ICMP6_ND_NA. */
#ifndef __cplusplus
static const
struct rte_flow_item_icmp6_nd_na rte_flow_item_icmp6_nd_na_mask = {
- .target_addr =
+ .target_addr = {
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
+ }
};
#endif
@@ -3812,7 +3814,7 @@ struct rte_flow_action_set_ipv4 {
* specified outermost IPv6 header.
*/
struct rte_flow_action_set_ipv6 {
- uint8_t ipv6_addr[16];
+ struct rte_ipv6_addr ipv6_addr;
};
/**
@@ -5196,8 +5198,8 @@ struct rte_flow_tunnel {
rte_be32_t dst_addr; /**< IPv4 destination address. */
} ipv4;
struct {
- uint8_t src_addr[16]; /**< IPv6 source address. */
- uint8_t dst_addr[16]; /**< IPv6 destination address. */
+ struct rte_ipv6_addr src_addr; /**< IPv6 source address. */
+ struct rte_ipv6_addr dst_addr; /**< IPv6 destination address. */
} ipv6;
};
rte_be16_t tp_src; /**< Tunnel port source. */
--
2.46.1
^ permalink raw reply [relevance 3%]
* [PATCH dpdk v2 12/16] gro: use ipv6 addr struct
` (7 preceding siblings ...)
2024-10-01 8:17 3% ` [PATCH dpdk v2 11/16] thash: " Robin Jarry
@ 2024-10-01 8:17 4% ` Robin Jarry
2024-10-01 8:17 3% ` [PATCH dpdk v2 13/16] rte_flow: " Robin Jarry
2024-10-06 9:04 3% ` [PATCH dpdk v2 00/16] IPv6 APIs overhaul Morten Brørup
10 siblings, 0 replies; 200+ results
From: Robin Jarry @ 2024-10-01 8:17 UTC (permalink / raw)
To: dev, Jiayu Hu
Update tcp6_flow_key to use the recently introduced IPv6 address
structure instead of uint8_t[16] arrays.
Signed-off-by: Robin Jarry <rjarry@redhat.com>
---
Notes:
v2: updated changelog for 24.11
doc/guides/rel_notes/deprecation.rst | 2 --
doc/guides/rel_notes/release_24_11.rst | 2 ++
lib/gro/gro_tcp6.c | 8 ++++----
lib/gro/gro_tcp6.h | 6 ++++--
4 files changed, 10 insertions(+), 8 deletions(-)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index dc0c6e4182ff..f384a4b414c9 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -70,8 +70,6 @@ Deprecation Notices
- ``struct rte_flow_item_icmp6_nd_na``
- ``struct rte_flow_action_set_ipv6``
- ``struct rte_flow_tunnel``
- gro
- - ``struct tcp6_flow_key``
* net, ethdev: The flow item ``RTE_FLOW_ITEM_TYPE_VXLAN_GPE``
is replaced with ``RTE_FLOW_ITEM_TYPE_VXLAN``.
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 78d222be180e..d6d75033bf8f 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -97,6 +97,8 @@ API Changes
``uint8_t[16]`` fields.
* hash: ``rte_ipv6_tuple`` was modified to use ``rte_ipv6_addr`` structures instead of
``uint8_t[16]`` fields.
+* gro: ``tcp6_flow_key`` was modified to use ``rte_ipv6_addr`` structures instead of ``uint8_t[16]``
+ fields.
ABI Changes
-----------
diff --git a/lib/gro/gro_tcp6.c b/lib/gro/gro_tcp6.c
index 6edfb6045cf6..0c6adbf2ecdb 100644
--- a/lib/gro/gro_tcp6.c
+++ b/lib/gro/gro_tcp6.c
@@ -99,8 +99,8 @@ insert_new_flow(struct gro_tcp6_tbl *tbl,
dst = &(tbl->flows[flow_idx].key);
ASSIGN_COMMON_TCP_KEY((&src->cmn_key), (&dst->cmn_key));
- memcpy(&dst->src_addr[0], &src->src_addr[0], sizeof(dst->src_addr));
- memcpy(&dst->dst_addr[0], &src->dst_addr[0], sizeof(dst->dst_addr));
+ rte_ipv6_addr_cpy(&dst->src_addr, &src->src_addr);
+ rte_ipv6_addr_cpy(&dst->dst_addr, &src->dst_addr);
dst->vtc_flow = src->vtc_flow;
tbl->flows[flow_idx].start_index = item_idx;
@@ -168,8 +168,8 @@ gro_tcp6_reassemble(struct rte_mbuf *pkt,
rte_ether_addr_copy(&(eth_hdr->src_addr), &(key.cmn_key.eth_saddr));
rte_ether_addr_copy(&(eth_hdr->dst_addr), &(key.cmn_key.eth_daddr));
- memcpy(&key.src_addr[0], &ipv6_hdr->src_addr, sizeof(key.src_addr));
- memcpy(&key.dst_addr[0], &ipv6_hdr->dst_addr, sizeof(key.dst_addr));
+ rte_ipv6_addr_cpy(&key.src_addr, &ipv6_hdr->src_addr);
+ rte_ipv6_addr_cpy(&key.dst_addr, &ipv6_hdr->dst_addr);
key.cmn_key.src_port = tcp_hdr->src_port;
key.cmn_key.dst_port = tcp_hdr->dst_port;
key.cmn_key.recv_ack = tcp_hdr->recv_ack;
diff --git a/lib/gro/gro_tcp6.h b/lib/gro/gro_tcp6.h
index 073122f0ec84..acf3971bb6e9 100644
--- a/lib/gro/gro_tcp6.h
+++ b/lib/gro/gro_tcp6.h
@@ -5,6 +5,8 @@
#ifndef _GRO_TCP6_H_
#define _GRO_TCP6_H_
+#include <rte_ip6.h>
+
#include "gro_tcp.h"
#define GRO_TCP6_TBL_MAX_ITEM_NUM (1024UL * 1024UL)
@@ -12,8 +14,8 @@
/* Header fields representing a TCP/IPv6 flow */
struct tcp6_flow_key {
struct cmn_tcp_key cmn_key;
- uint8_t src_addr[16];
- uint8_t dst_addr[16];
+ struct rte_ipv6_addr src_addr;
+ struct rte_ipv6_addr dst_addr;
rte_be32_t vtc_flow;
};
--
2.46.1
^ permalink raw reply [relevance 4%]
* [PATCH dpdk v2 11/16] thash: use ipv6 addr struct
` (6 preceding siblings ...)
2024-10-01 8:17 3% ` [PATCH dpdk v2 10/16] ipsec: " Robin Jarry
@ 2024-10-01 8:17 3% ` Robin Jarry
2024-10-01 8:17 4% ` [PATCH dpdk v2 12/16] gro: " Robin Jarry
` (2 subsequent siblings)
10 siblings, 0 replies; 200+ results
From: Robin Jarry @ 2024-10-01 8:17 UTC (permalink / raw)
To: dev, Yipeng Wang, Sameh Gobriel, Bruce Richardson, Vladimir Medvedkin
Update rte_ipv6_tuple to use the recently added IPv6 address structure
instead of uint8_t[16] arrays.
Signed-off-by: Robin Jarry <rjarry@redhat.com>
---
Notes:
v2: updated changelog for 24.11
app/test/test_thash.c | 46 ++++++++++----------------
doc/guides/rel_notes/deprecation.rst | 2 --
doc/guides/rel_notes/release_24_11.rst | 2 ++
lib/hash/rte_thash.h | 20 +++++------
4 files changed, 29 insertions(+), 41 deletions(-)
diff --git a/app/test/test_thash.c b/app/test/test_thash.c
index 952da6a52954..262f84433461 100644
--- a/app/test/test_thash.c
+++ b/app/test/test_thash.c
@@ -25,8 +25,8 @@ struct test_thash_v4 {
};
struct test_thash_v6 {
- uint8_t dst_ip[16];
- uint8_t src_ip[16];
+ struct rte_ipv6_addr dst_ip;
+ struct rte_ipv6_addr src_ip;
uint16_t dst_port;
uint16_t src_port;
uint32_t hash_l3;
@@ -49,25 +49,19 @@ struct test_thash_v4 v4_tbl[] = {
struct test_thash_v6 v6_tbl[] = {
/*3ffe:2501:200:3::1*/
-{{0x3f, 0xfe, 0x25, 0x01, 0x02, 0x00, 0x00, 0x03,
-0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01,},
+{{.a = "\x3f\xfe\x25\x01\x02\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x01"},
/*3ffe:2501:200:1fff::7*/
-{0x3f, 0xfe, 0x25, 0x01, 0x02, 0x00, 0x1f, 0xff,
-0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x07,},
+{.a = "\x3f\xfe\x25\x01\x02\x00\x1f\xff\x00\x00\x00\x00\x00\x00\x00\x07"},
1766, 2794, 0x2cc18cd5, 0x40207d3d},
/*ff02::1*/
-{{0xff, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01,},
+{{.a = "\xff\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01"},
/*3ffe:501:8::260:97ff:fe40:efab*/
-{0x3f, 0xfe, 0x05, 0x01, 0x00, 0x08, 0x00, 0x00,
-0x02, 0x60, 0x97, 0xff, 0xfe, 0x40, 0xef, 0xab,},
+{.a = "\x3f\xfe\x05\x01\x00\x08\x00\x00\x02\x60\x97\xff\xfe\x40\xef\xab"},
4739, 14230, 0x0f0c461c, 0xdde51bbf},
/*fe80::200:f8ff:fe21:67cf*/
-{{0xfe, 0x80, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
-0x02, 0x00, 0xf8, 0xff, 0xfe, 0x21, 0x67, 0xcf,},
+{{.a = "\xfe\x80\x00\x00\x00\x00\x00\x00\x02\x00\xf8\xff\xfe\x21\x67\xcf"},
/*3ffe:1900:4545:3:200:f8ff:fe21:67cf*/
-{0x3f, 0xfe, 0x19, 0x00, 0x45, 0x45, 0x00, 0x03,
-0x02, 0x00, 0xf8, 0xff, 0xfe, 0x21, 0x67, 0xcf,},
+{.a = "\x3f\xfe\x19\x00\x45\x45\x00\x03\x02\x00\xf8\xff\xfe\x21\x67\xcf"},
38024, 44251, 0x4b61e985, 0x02d1feef},
};
@@ -110,7 +104,7 @@ static const uint8_t big_rss_key[] = {
static int
test_toeplitz_hash_calc(void)
{
- uint32_t i, j;
+ uint32_t i;
union rte_thash_tuple tuple;
uint32_t rss_l3, rss_l3l4;
uint8_t rss_key_be[RTE_DIM(default_rss_key)];
@@ -145,10 +139,8 @@ test_toeplitz_hash_calc(void)
}
for (i = 0; i < RTE_DIM(v6_tbl); i++) {
/*Fill ipv6 hdr*/
- for (j = 0; j < RTE_DIM(ipv6_hdr.src_addr.a); j++)
- ipv6_hdr.src_addr.a[j] = v6_tbl[i].src_ip[j];
- for (j = 0; j < RTE_DIM(ipv6_hdr.dst_addr.a); j++)
- ipv6_hdr.dst_addr.a[j] = v6_tbl[i].dst_ip[j];
+ rte_ipv6_addr_cpy(&ipv6_hdr.src_addr, &v6_tbl[i].src_ip);
+ rte_ipv6_addr_cpy(&ipv6_hdr.dst_addr, &v6_tbl[i].dst_ip);
/*Load and convert ipv6 address into tuple*/
rte_thash_load_v6_addrs(&ipv6_hdr, &tuple);
tuple.v6.sport = v6_tbl[i].src_port;
@@ -176,7 +168,7 @@ test_toeplitz_hash_calc(void)
static int
test_toeplitz_hash_gfni(void)
{
- uint32_t i, j;
+ uint32_t i;
union rte_thash_tuple tuple;
uint32_t rss_l3, rss_l3l4;
uint64_t rss_key_matrixes[RTE_DIM(default_rss_key)];
@@ -204,10 +196,8 @@ test_toeplitz_hash_gfni(void)
}
for (i = 0; i < RTE_DIM(v6_tbl); i++) {
- for (j = 0; j < RTE_DIM(tuple.v6.src_addr); j++)
- tuple.v6.src_addr[j] = v6_tbl[i].src_ip[j];
- for (j = 0; j < RTE_DIM(tuple.v6.dst_addr); j++)
- tuple.v6.dst_addr[j] = v6_tbl[i].dst_ip[j];
+ rte_ipv6_addr_cpy(&tuple.v6.src_addr, &v6_tbl[i].src_ip);
+ rte_ipv6_addr_cpy(&tuple.v6.dst_addr, &v6_tbl[i].dst_ip);
tuple.v6.sport = rte_cpu_to_be_16(v6_tbl[i].dst_port);
tuple.v6.dport = rte_cpu_to_be_16(v6_tbl[i].src_port);
rss_l3 = rte_thash_gfni(rss_key_matrixes, (uint8_t *)&tuple,
@@ -299,7 +289,7 @@ enum {
static int
test_toeplitz_hash_gfni_bulk(void)
{
- uint32_t i, j;
+ uint32_t i;
union rte_thash_tuple tuple[2];
uint8_t *tuples[2];
uint32_t rss[2] = { 0 };
@@ -328,10 +318,8 @@ test_toeplitz_hash_gfni_bulk(void)
rte_memcpy(tuples[0], &tuple[0], RTE_THASH_V4_L4_LEN * 4);
/*Load IPv6 headers and copy it into the corresponding tuple*/
- for (j = 0; j < RTE_DIM(tuple[1].v6.src_addr); j++)
- tuple[1].v6.src_addr[j] = v6_tbl[i].src_ip[j];
- for (j = 0; j < RTE_DIM(tuple[1].v6.dst_addr); j++)
- tuple[1].v6.dst_addr[j] = v6_tbl[i].dst_ip[j];
+ rte_ipv6_addr_cpy(&tuple[1].v6.src_addr, &v6_tbl[i].src_ip);
+ rte_ipv6_addr_cpy(&tuple[1].v6.dst_addr, &v6_tbl[i].dst_ip);
tuple[1].v6.sport = rte_cpu_to_be_16(v6_tbl[i].dst_port);
tuple[1].v6.dport = rte_cpu_to_be_16(v6_tbl[i].src_port);
rte_memcpy(tuples[1], &tuple[1], RTE_THASH_V6_L4_LEN * 4);
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 838e22b44444..dc0c6e4182ff 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -72,8 +72,6 @@ Deprecation Notices
- ``struct rte_flow_tunnel``
gro
- ``struct tcp6_flow_key``
- hash
- - ``struct rte_ipv6_tuple``
* net, ethdev: The flow item ``RTE_FLOW_ITEM_TYPE_VXLAN_GPE``
is replaced with ``RTE_FLOW_ITEM_TYPE_VXLAN``.
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index debaf99e4a71..78d222be180e 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -95,6 +95,8 @@ API Changes
use ``rte_ipv6_addr`` structures instead of ``uint8_t[16]`` fields.
* ipsec: ``rte_ipsec_sadv6_key`` was modified to use ``rte_ipv6_addr`` structures instead of
``uint8_t[16]`` fields.
+* hash: ``rte_ipv6_tuple`` was modified to use ``rte_ipv6_addr`` structures instead of
+ ``uint8_t[16]`` fields.
ABI Changes
-----------
diff --git a/lib/hash/rte_thash.h b/lib/hash/rte_thash.h
index 9aaaacfd5fa4..ddc4e345097d 100644
--- a/lib/hash/rte_thash.h
+++ b/lib/hash/rte_thash.h
@@ -89,8 +89,8 @@ struct rte_ipv4_tuple {
* ports/sctp_tag have to be CPU byte order
*/
struct rte_ipv6_tuple {
- uint8_t src_addr[16];
- uint8_t dst_addr[16];
+ struct rte_ipv6_addr src_addr;
+ struct rte_ipv6_addr dst_addr;
union {
struct {
uint16_t dport;
@@ -141,22 +141,22 @@ rte_thash_load_v6_addrs(const struct rte_ipv6_hdr *orig,
{
#ifdef RTE_ARCH_X86
__m128i ipv6 = _mm_loadu_si128((const __m128i *)&orig->src_addr);
- *(__m128i *)targ->v6.src_addr =
+ *(__m128i *)&targ->v6.src_addr =
_mm_shuffle_epi8(ipv6, rte_thash_ipv6_bswap_mask);
ipv6 = _mm_loadu_si128((const __m128i *)&orig->dst_addr);
- *(__m128i *)targ->v6.dst_addr =
+ *(__m128i *)&targ->v6.dst_addr =
_mm_shuffle_epi8(ipv6, rte_thash_ipv6_bswap_mask);
#elif defined(__ARM_NEON)
- uint8x16_t ipv6 = vld1q_u8((uint8_t const *)&orig->src_addr);
- vst1q_u8((uint8_t *)targ->v6.src_addr, vrev32q_u8(ipv6));
- ipv6 = vld1q_u8((uint8_t const *)&orig->dst_addr);
- vst1q_u8((uint8_t *)targ->v6.dst_addr, vrev32q_u8(ipv6));
+ uint8x16_t ipv6 = vld1q_u8(orig->src_addr.a);
+ vst1q_u8(targ->v6.src_addr.a, vrev32q_u8(ipv6));
+ ipv6 = vld1q_u8(orig->dst_addr.a);
+ vst1q_u8(targ->v6.dst_addr.a, vrev32q_u8(ipv6));
#else
int i;
for (i = 0; i < 4; i++) {
- *((uint32_t *)targ->v6.src_addr + i) =
+ *((uint32_t *)targ->v6.src_addr.a + i) =
rte_be_to_cpu_32(*((const uint32_t *)orig->src_addr.a + i));
- *((uint32_t *)targ->v6.dst_addr + i) =
+ *((uint32_t *)targ->v6.dst_addr.a + i) =
rte_be_to_cpu_32(*((const uint32_t *)orig->dst_addr.a + i));
}
#endif
--
2.46.1
^ permalink raw reply [relevance 3%]
* [PATCH dpdk v2 10/16] ipsec: use ipv6 addr struct
` (5 preceding siblings ...)
2024-10-01 8:17 3% ` [PATCH dpdk v2 09/16] pipeline: use ipv6 addr struct Robin Jarry
@ 2024-10-01 8:17 3% ` Robin Jarry
2024-10-01 8:17 3% ` [PATCH dpdk v2 11/16] thash: " Robin Jarry
` (3 subsequent siblings)
10 siblings, 0 replies; 200+ results
From: Robin Jarry @ 2024-10-01 8:17 UTC (permalink / raw)
To: dev, Konstantin Ananyev, Vladimir Medvedkin, Radu Nicolau, Akhil Goyal
Update rte_ipsec_sadv6_key to use the recently introduced IPv6 address
structure instead of ad-hoc uint8_t[16] arrays.
Signed-off-by: Robin Jarry <rjarry@redhat.com>
---
Notes:
v2: updated changelog for 24.11
app/test-sad/main.c | 24 +++++++-------
app/test/test_ipsec_sad.c | 44 +++++++++++++-------------
doc/guides/rel_notes/deprecation.rst | 2 --
doc/guides/rel_notes/release_24_11.rst | 2 ++
examples/ipsec-secgw/sad.h | 10 +++---
lib/ipsec/rte_ipsec_sad.h | 7 ++--
6 files changed, 46 insertions(+), 43 deletions(-)
diff --git a/app/test-sad/main.c b/app/test-sad/main.c
index addfc0714521..7253f6488dce 100644
--- a/app/test-sad/main.c
+++ b/app/test-sad/main.c
@@ -248,8 +248,8 @@ get_random_rules(struct rule *tbl, uint32_t nb_rules, int rule_tbl)
(uint64_t)(edge + step));
if (config.ipv6) {
for (j = 0; j < 16; j++) {
- tbl[i].tuple.v6.dip[j] = rte_rand();
- tbl[i].tuple.v6.sip[j] = rte_rand();
+ tbl[i].tuple.v6.dip.a[j] = rte_rand();
+ tbl[i].tuple.v6.sip.a[j] = rte_rand();
}
} else {
tbl[i].tuple.v4.dip = rte_rand();
@@ -274,9 +274,9 @@ get_random_rules(struct rule *tbl, uint32_t nb_rules, int rule_tbl)
(uint64_t)(edge + step));
if (config.ipv6) {
for (j = 0; j < 16; j++) {
- tbl[i].tuple.v6.dip[j] =
+ tbl[i].tuple.v6.dip.a[j] =
rte_rand();
- tbl[i].tuple.v6.sip[j] =
+ tbl[i].tuple.v6.sip.a[j] =
rte_rand();
}
} else {
@@ -289,11 +289,11 @@ get_random_rules(struct rule *tbl, uint32_t nb_rules, int rule_tbl)
config.nb_rules].tuple.v4.spi;
if (config.ipv6) {
int r_idx = i % config.nb_rules;
- memcpy(tbl[i].tuple.v6.dip,
- rules_tbl[r_idx].tuple.v6.dip,
+ memcpy(&tbl[i].tuple.v6.dip,
+ &rules_tbl[r_idx].tuple.v6.dip,
sizeof(tbl[i].tuple.v6.dip));
- memcpy(tbl[i].tuple.v6.sip,
- rules_tbl[r_idx].tuple.v6.sip,
+ memcpy(&tbl[i].tuple.v6.sip,
+ &rules_tbl[r_idx].tuple.v6.sip,
sizeof(tbl[i].tuple.v6.sip));
} else {
tbl[i].tuple.v4.dip = rules_tbl[i %
@@ -472,8 +472,8 @@ print_result(const union rte_ipsec_sad_key *key, void *res)
v4 = &key->v4;
v6 = &key->v6;
spi = (config.ipv6 == 0) ? v4->spi : v6->spi;
- dip = (config.ipv6 == 0) ? &v4->dip : (const void *)v6->dip;
- sip = (config.ipv6 == 0) ? &v4->sip : (const void *)v6->sip;
+ dip = (config.ipv6 == 0) ? &v4->dip : (const void *)&v6->dip;
+ sip = (config.ipv6 == 0) ? &v4->sip : (const void *)&v6->sip;
if (res == NULL) {
printf("TUPLE: ");
@@ -500,8 +500,8 @@ print_result(const union rte_ipsec_sad_key *key, void *res)
v4 = &rule->tuple.v4;
v6 = &rule->tuple.v6;
spi = (config.ipv6 == 0) ? v4->spi : v6->spi;
- dip = (config.ipv6 == 0) ? &v4->dip : (const void *)v6->dip;
- sip = (config.ipv6 == 0) ? &v4->sip : (const void *)v6->sip;
+ dip = (config.ipv6 == 0) ? &v4->dip : (const void *)&v6->dip;
+ sip = (config.ipv6 == 0) ? &v4->sip : (const void *)&v6->sip;
printf("\n\tpoints to RULE ID %zu ",
RTE_PTR_DIFF(res, rules_tbl)/sizeof(struct rule));
print_tuple(af, spi, dip, sip);
diff --git a/app/test/test_ipsec_sad.c b/app/test/test_ipsec_sad.c
index 7534f16f89d0..79ea8ee915dd 100644
--- a/app/test/test_ipsec_sad.c
+++ b/app/test/test_ipsec_sad.c
@@ -212,7 +212,7 @@ test_add_invalid(void)
{
int status;
struct rte_ipsec_sadv4_key tuple_v4 = {10, 20, 30};
- struct rte_ipsec_sadv6_key tuple_v6 = {10, {20, }, {30, } };
+ struct rte_ipsec_sadv6_key tuple_v6 = {10, {.a = {20, }}, {.a = {30, }} };
status = __test_add_invalid(0, (union rte_ipsec_sad_key *)&tuple_v4);
if (status != TEST_SUCCESS)
@@ -271,8 +271,8 @@ test_delete_invalid(void)
{
int status;
struct rte_ipsec_sadv4_key tuple_v4 = {SPI, DIP, SIP};
- struct rte_ipsec_sadv6_key tuple_v6 = {SPI, {0xbe, 0xef, },
- {0xf0, 0x0d, } };
+ struct rte_ipsec_sadv6_key tuple_v6 = {
+ SPI, {.a = {0xbe, 0xef, }}, {.a = {0xf0, 0x0d, }} };
status = __test_delete_invalid(0, (union rte_ipsec_sad_key *)&tuple_v4);
if (status != TEST_SUCCESS)
@@ -329,7 +329,7 @@ test_lookup_invalid(void)
{
int status;
struct rte_ipsec_sadv4_key tuple_v4 = {10, 20, 30};
- struct rte_ipsec_sadv6_key tuple_v6 = {10, {20, }, {30, } };
+ struct rte_ipsec_sadv6_key tuple_v6 = {10, {.a = {20, }}, {.a = {30, }} };
status = __test_lookup_invalid(0,
(union rte_ipsec_sad_key *)&tuple_v4);
@@ -405,10 +405,10 @@ test_lookup_basic(void)
int status;
struct rte_ipsec_sadv4_key tuple_v4 = {SPI, DIP, SIP};
struct rte_ipsec_sadv4_key tuple_v4_1 = {SPI, BAD, BAD};
- struct rte_ipsec_sadv6_key tuple_v6 = {SPI, {0xbe, 0xef, },
- {0xf0, 0x0d, } };
- struct rte_ipsec_sadv6_key tuple_v6_1 = {SPI, {0x0b, 0xad, },
- {0x0b, 0xad, } };
+ struct rte_ipsec_sadv6_key tuple_v6 = {
+ SPI, {.a = {0xbe, 0xef, }}, {.a = {0xf0, 0x0d, }} };
+ struct rte_ipsec_sadv6_key tuple_v6_1 = {
+ SPI, {.a = {0x0b, 0xad, }}, {.a = {0x0b, 0xad, }} };
status = __test_lookup_basic(0, (union rte_ipsec_sad_key *)&tuple_v4,
(union rte_ipsec_sad_key *)&tuple_v4_1);
@@ -654,14 +654,14 @@ test_lookup_adv(void)
struct rte_ipsec_sadv4_key tuple_v4_3 = {BAD, DIP, SIP};
/* key to install*/
- struct rte_ipsec_sadv6_key tuple_v6 = {SPI, {0xbe, 0xef, },
- {0xf0, 0x0d, } };
- struct rte_ipsec_sadv6_key tuple_v6_1 = {SPI, {0xbe, 0xef, },
- {0x0b, 0xad, } };
- struct rte_ipsec_sadv6_key tuple_v6_2 = {SPI, {0x0b, 0xad, },
- {0xf0, 0x0d, } };
- struct rte_ipsec_sadv6_key tuple_v6_3 = {BAD, {0xbe, 0xef, },
- {0xf0, 0x0d, } };
+ struct rte_ipsec_sadv6_key tuple_v6 = {
+ SPI, {.a = {0xbe, 0xef, }}, {.a = {0xf0, 0x0d, }} };
+ struct rte_ipsec_sadv6_key tuple_v6_1 = {
+ SPI, {.a = {0xbe, 0xef, }}, {.a = {0x0b, 0xad, }} };
+ struct rte_ipsec_sadv6_key tuple_v6_2 = {
+ SPI, {.a = {0x0b, 0xad, }}, {.a = {0xf0, 0x0d, }} };
+ struct rte_ipsec_sadv6_key tuple_v6_3 = {
+ BAD, {.a = {0xbe, 0xef, }}, {.a = {0xf0, 0x0d, }} };
const union rte_ipsec_sad_key *key_arr[] = {
(union rte_ipsec_sad_key *)&tuple_v4,
@@ -852,12 +852,12 @@ test_lookup_order(void)
struct rte_ipsec_sadv4_key tuple_v4_1 = {SPI, DIP, BAD};
struct rte_ipsec_sadv4_key tuple_v4_2 = {SPI, BAD, SIP};
/* key to install*/
- struct rte_ipsec_sadv6_key tuple_v6 = {SPI, {0xbe, 0xef, },
- {0xf0, 0x0d, } };
- struct rte_ipsec_sadv6_key tuple_v6_1 = {SPI, {0xbe, 0xef, },
- {0x0b, 0xad, } };
- struct rte_ipsec_sadv6_key tuple_v6_2 = {SPI, {0x0b, 0xad, },
- {0xf0, 0x0d, } };
+ struct rte_ipsec_sadv6_key tuple_v6 = {
+ SPI, {.a = {0xbe, 0xef, }}, {.a = {0xf0, 0x0d, }} };
+ struct rte_ipsec_sadv6_key tuple_v6_1 = {
+ SPI, {.a = {0xbe, 0xef, }}, {.a = {0x0b, 0xad, }} };
+ struct rte_ipsec_sadv6_key tuple_v6_2 = {
+ SPI, {.a = {0x0b, 0xad, }}, {.a = {0xf0, 0x0d, }} };
status = __test_lookup_order(0, (union rte_ipsec_sad_key *)&tuple_v4,
(union rte_ipsec_sad_key *)&tuple_v4_1,
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index fcccb22593d8..838e22b44444 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -74,8 +74,6 @@ Deprecation Notices
- ``struct tcp6_flow_key``
hash
- ``struct rte_ipv6_tuple``
- ipsec
- - ``struct rte_ipsec_sadv6_key``
* net, ethdev: The flow item ``RTE_FLOW_ITEM_TYPE_VXLAN_GPE``
is replaced with ``RTE_FLOW_ITEM_TYPE_VXLAN``.
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 3eb43ba8520f..debaf99e4a71 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -93,6 +93,8 @@ API Changes
``uint8_t[16]`` parameter.
* pipeline: ``rte_table_action_ipv6_header`` and ``rte_table_action_nat_params`` were modified to
use ``rte_ipv6_addr`` structures instead of ``uint8_t[16]`` fields.
+* ipsec: ``rte_ipsec_sadv6_key`` was modified to use ``rte_ipv6_addr`` structures instead of
+ ``uint8_t[16]`` fields.
ABI Changes
-----------
diff --git a/examples/ipsec-secgw/sad.h b/examples/ipsec-secgw/sad.h
index fdb1d2ef1790..4108abb9e698 100644
--- a/examples/ipsec-secgw/sad.h
+++ b/examples/ipsec-secgw/sad.h
@@ -9,6 +9,8 @@
#include <rte_ip6.h>
#include <rte_ipsec_sad.h>
+#include "ipsec.h"
+
#define SA_CACHE_SZ 128
#define SPI2IDX(spi, mask) ((spi) & (mask))
@@ -39,8 +41,8 @@ cmp_sa_key(struct ipsec_sa *sa, int is_v4, struct rte_ipv4_hdr *ipv4,
(sa->dst.ip.ip4 == ipv4->dst_addr)) ||
/* IPv6 check */
(!is_v4 && (sa_type == IP6_TUNNEL) &&
- (!memcmp(sa->src.ip.ip6.ip6, &ipv6->src_addr, 16)) &&
- (!memcmp(sa->dst.ip.ip6.ip6, &ipv6->dst_addr, 16))))
+ (!memcmp(&sa->src.ip.ip6.ip6, &ipv6->src_addr, 16)) &&
+ (!memcmp(&sa->dst.ip.ip6.ip6, &ipv6->dst_addr, 16))))
return 1;
return 0;
@@ -130,9 +132,9 @@ sad_lookup(struct ipsec_sad *sad, struct rte_mbuf *pkts[],
}
}
v6[nb_v6].spi = esp->spi;
- memcpy(v6[nb_v6].dip, &ipv6->dst_addr,
+ memcpy(&v6[nb_v6].dip, &ipv6->dst_addr,
sizeof(ipv6->dst_addr));
- memcpy(v6[nb_v6].sip, &ipv6->src_addr,
+ memcpy(&v6[nb_v6].sip, &ipv6->src_addr,
sizeof(ipv6->src_addr));
keys_v6[nb_v6] = (const union rte_ipsec_sad_key *)
&v6[nb_v6];
diff --git a/lib/ipsec/rte_ipsec_sad.h b/lib/ipsec/rte_ipsec_sad.h
index 0adf89d9310c..035ae4f3b3eb 100644
--- a/lib/ipsec/rte_ipsec_sad.h
+++ b/lib/ipsec/rte_ipsec_sad.h
@@ -6,8 +6,9 @@
#ifndef _RTE_IPSEC_SAD_H_
#define _RTE_IPSEC_SAD_H_
-#include <stdint.h>
+#include <rte_ip6.h>
+#include <stdint.h>
/**
* @file rte_ipsec_sad.h
@@ -38,8 +39,8 @@ struct rte_ipsec_sadv4_key {
struct rte_ipsec_sadv6_key {
uint32_t spi;
- uint8_t dip[16];
- uint8_t sip[16];
+ struct rte_ipv6_addr dip;
+ struct rte_ipv6_addr sip;
};
union rte_ipsec_sad_key {
--
2.46.1
^ permalink raw reply [relevance 3%]
* [PATCH dpdk v2 08/16] graph,node: use ipv6 addr struct and utils
` (3 preceding siblings ...)
2024-10-01 8:17 1% ` [PATCH dpdk v2 05/16] fib6,rib6,lpm6: use ipv6 addr struct Robin Jarry
@ 2024-10-01 8:17 2% ` Robin Jarry
2024-10-01 8:17 3% ` [PATCH dpdk v2 09/16] pipeline: use ipv6 addr struct Robin Jarry
` (5 subsequent siblings)
10 siblings, 0 replies; 200+ results
From: Robin Jarry @ 2024-10-01 8:17 UTC (permalink / raw)
To: dev, Sunil Kumar Kori, Rakesh Kudurumalla, Jerin Jacob,
Kiran Kumar K, Nithin Dabilpuram, Zhirun Yan, Pavan Nikhilesh
Replace ad-hoc uint8_t[16] arrays with the previously introduced IPv6
address structure.
Signed-off-by: Robin Jarry <rjarry@redhat.com>
---
Notes:
v2: updated changelog for 24.11
app/graph/ethdev.c | 40 +++++++---------------
app/graph/ethdev.h | 9 +++--
app/graph/ip6_route.c | 47 ++++++--------------------
app/graph/meson.build | 2 +-
app/graph/neigh.c | 22 +++++-------
app/graph/neigh_priv.h | 4 ++-
app/graph/route.h | 8 +++--
doc/guides/rel_notes/deprecation.rst | 2 --
doc/guides/rel_notes/release_24_11.rst | 2 ++
examples/l3fwd-graph/main.c | 30 ++++++----------
lib/node/ip6_lookup.c | 17 ++++------
lib/node/rte_node_ip6_api.h | 3 +-
12 files changed, 67 insertions(+), 119 deletions(-)
diff --git a/app/graph/ethdev.c b/app/graph/ethdev.c
index cfc1b1856910..19c5ab685464 100644
--- a/app/graph/ethdev.c
+++ b/app/graph/ethdev.c
@@ -124,30 +124,19 @@ ethdev_portid_by_ip4(uint32_t ip, uint32_t mask)
}
int16_t
-ethdev_portid_by_ip6(uint8_t *ip, uint8_t *mask)
+ethdev_portid_by_ip6(struct rte_ipv6_addr *ip, struct rte_ipv6_addr *mask)
{
- int portid = -EINVAL;
struct ethdev *port;
- int j;
TAILQ_FOREACH(port, ð_node, next) {
- for (j = 0; j < ETHDEV_IPV6_ADDR_LEN; j++) {
- if (mask == NULL) {
- if ((port->ip6_addr.ip[j] & port->ip6_addr.mask[j]) !=
- (ip[j] & port->ip6_addr.mask[j]))
- break;
-
- } else {
- if ((port->ip6_addr.ip[j] & port->ip6_addr.mask[j]) !=
- (ip[j] & mask[j]))
- break;
- }
- }
- if (j == ETHDEV_IPV6_ADDR_LEN)
+ uint8_t depth = rte_ipv6_mask_depth(&port->ip6_addr.mask);
+ if (mask != NULL)
+ depth = RTE_MAX(depth, rte_ipv6_mask_depth(mask));
+ if (rte_ipv6_addr_eq_prefix(&port->ip6_addr.ip, ip, depth))
return port->config.port_id;
}
- return portid;
+ return -EINVAL;
}
void
@@ -285,7 +274,7 @@ ethdev_ip6_addr_add(const char *name, struct ipv6_addr_config *config)
{
struct ethdev *eth_hdl;
uint16_t portid = 0;
- int rc, i;
+ int rc;
rc = rte_eth_dev_get_port_by_name(name, &portid);
if (rc < 0)
@@ -294,10 +283,8 @@ ethdev_ip6_addr_add(const char *name, struct ipv6_addr_config *config)
eth_hdl = ethdev_port_by_id(portid);
if (eth_hdl) {
- for (i = 0; i < ETHDEV_IPV6_ADDR_LEN; i++) {
- eth_hdl->ip6_addr.ip[i] = config->ip[i];
- eth_hdl->ip6_addr.mask[i] = config->mask[i];
- }
+ rte_ipv6_addr_cpy(ð_hdl->ip6_addr.ip, &config->ip);
+ rte_ipv6_addr_cpy(ð_hdl->ip6_addr.mask, &config->mask);
return 0;
}
rc = -EINVAL;
@@ -624,13 +611,10 @@ cmd_ethdev_dev_ip6_addr_add_parsed(void *parsed_result, __rte_unused struct cmdl
{
struct cmd_ethdev_dev_ip6_addr_add_result *res = parsed_result;
struct ipv6_addr_config config;
- int rc = -EINVAL, i;
+ int rc = -EINVAL;
- for (i = 0; i < ETHDEV_IPV6_ADDR_LEN; i++)
- config.ip[i] = res->ip.addr.ipv6.s6_addr[i];
-
- for (i = 0; i < ETHDEV_IPV6_ADDR_LEN; i++)
- config.mask[i] = res->mask.addr.ipv6.s6_addr[i];
+ rte_memcpy(&config.ip, &res->ip.addr.ipv6, sizeof(config.ip));
+ rte_memcpy(&config.mask, &res->mask.addr.ipv6, sizeof(config.mask));
rc = ethdev_ip6_addr_add(res->dev, &config);
if (rc < 0)
diff --git a/app/graph/ethdev.h b/app/graph/ethdev.h
index d0de593fc743..046689ee5fc7 100644
--- a/app/graph/ethdev.h
+++ b/app/graph/ethdev.h
@@ -6,8 +6,7 @@
#define APP_GRAPH_ETHDEV_H
#include <cmdline_parse.h>
-
-#define ETHDEV_IPV6_ADDR_LEN 16
+#include <rte_ip6.h>
struct ipv4_addr_config {
uint32_t ip;
@@ -15,8 +14,8 @@ struct ipv4_addr_config {
};
struct ipv6_addr_config {
- uint8_t ip[ETHDEV_IPV6_ADDR_LEN];
- uint8_t mask[ETHDEV_IPV6_ADDR_LEN];
+ struct rte_ipv6_addr ip;
+ struct rte_ipv6_addr mask;
};
extern uint32_t enabled_port_mask;
@@ -25,7 +24,7 @@ void ethdev_start(void);
void ethdev_stop(void);
void *ethdev_mempool_list_by_portid(uint16_t portid);
int16_t ethdev_portid_by_ip4(uint32_t ip, uint32_t mask);
-int16_t ethdev_portid_by_ip6(uint8_t *ip, uint8_t *mask);
+int16_t ethdev_portid_by_ip6(struct rte_ipv6_addr *ip, struct rte_ipv6_addr *mask);
int16_t ethdev_txport_by_rxport_get(uint16_t portid_rx);
void ethdev_list_clean(void);
diff --git a/app/graph/ip6_route.c b/app/graph/ip6_route.c
index 834719ecaeb4..5460e5070b42 100644
--- a/app/graph/ip6_route.c
+++ b/app/graph/ip6_route.c
@@ -11,6 +11,7 @@
#include <cmdline_socket.h>
#include <rte_node_ip6_api.h>
+#include <rte_ip6.h>
#include "module_api.h"
#include "route_priv.h"
@@ -43,38 +44,20 @@ find_route6_entry(struct route_ipv6_config *route)
return NULL;
}
-static uint8_t
-convert_ip6_netmask_to_depth(uint8_t *netmask)
-{
- uint8_t setbits = 0;
- uint8_t mask;
- int i;
-
- for (i = 0; i < ETHDEV_IPV6_ADDR_LEN; i++) {
- mask = netmask[i];
- while (mask & 0x80) {
- mask = mask << 1;
- setbits++;
- }
- }
-
- return setbits;
-}
-
static int
route6_rewirte_table_update(struct route_ipv6_config *ipv6route)
{
uint8_t depth;
int portid;
- portid = ethdev_portid_by_ip6(ipv6route->gateway, ipv6route->mask);
+ portid = ethdev_portid_by_ip6(&ipv6route->gateway, &ipv6route->mask);
if (portid < 0) {
printf("Invalid portid found to install the route\n");
return portid;
}
- depth = convert_ip6_netmask_to_depth(ipv6route->mask);
+ depth = rte_ipv6_mask_depth(&ipv6route->mask);
- return rte_node_ip6_route_add(ipv6route->ip, depth, portid,
+ return rte_node_ip6_route_add(&ipv6route->ip, depth, portid,
RTE_NODE_IP6_LOOKUP_NEXT_REWRITE);
}
@@ -84,7 +67,6 @@ route_ip6_add(struct route_ipv6_config *route)
{
struct route_ipv6_config *ipv6route;
int rc = -EINVAL;
- int j;
ipv6route = find_route6_entry(route);
if (!ipv6route) {
@@ -95,11 +77,9 @@ route_ip6_add(struct route_ipv6_config *route)
return 0;
}
- for (j = 0; j < ETHDEV_IPV6_ADDR_LEN; j++) {
- ipv6route->ip[j] = route->ip[j];
- ipv6route->mask[j] = route->mask[j];
- ipv6route->gateway[j] = route->gateway[j];
- }
+ rte_ipv6_addr_cpy(&ipv6route->ip, &route->ip);
+ rte_ipv6_addr_cpy(&ipv6route->mask, &route->mask);
+ rte_ipv6_addr_cpy(&ipv6route->gateway, &route->gateway);
ipv6route->is_used = true;
if (!graph_status_get())
@@ -154,16 +134,11 @@ cmd_ipv6_lookup_route_add_ipv6_parsed(void *parsed_result, __rte_unused struct c
{
struct cmd_ipv6_lookup_route_add_ipv6_result *res = parsed_result;
struct route_ipv6_config config;
- int rc = -EINVAL, i;
+ int rc = -EINVAL;
- for (i = 0; i < ETHDEV_IPV6_ADDR_LEN; i++)
- config.ip[i] = res->ip.addr.ipv6.s6_addr[i];
-
- for (i = 0; i < ETHDEV_IPV6_ADDR_LEN; i++)
- config.mask[i] = res->mask.addr.ipv6.s6_addr[i];
-
- for (i = 0; i < ETHDEV_IPV6_ADDR_LEN; i++)
- config.gateway[i] = res->via_ip.addr.ipv6.s6_addr[i];
+ rte_memcpy(&config.ip, &res->ip.addr.ipv6, sizeof(config.ip));
+ rte_memcpy(&config.mask, &res->mask.addr.ipv6, sizeof(config.mask));
+ rte_memcpy(&config.gateway, &res->via_ip.addr.ipv6, sizeof(config.gateway));
rc = route_ip6_add(&config);
if (rc)
diff --git a/app/graph/meson.build b/app/graph/meson.build
index 6dc54d5ee63f..344e4a418fc0 100644
--- a/app/graph/meson.build
+++ b/app/graph/meson.build
@@ -9,7 +9,7 @@ if not build
subdir_done()
endif
-deps += ['graph', 'eal', 'lpm', 'ethdev', 'node', 'cmdline']
+deps += ['graph', 'eal', 'lpm', 'ethdev', 'node', 'cmdline', 'net']
sources = files(
'cli.c',
'conn.c',
diff --git a/app/graph/neigh.c b/app/graph/neigh.c
index 79fd542c8948..0b35f4d6c294 100644
--- a/app/graph/neigh.c
+++ b/app/graph/neigh.c
@@ -62,12 +62,12 @@ find_neigh4_entry(uint32_t ip, uint64_t mac)
}
static struct neigh_ipv6_config *
-find_neigh6_entry(uint8_t *ip, uint64_t mac)
+find_neigh6_entry(struct rte_ipv6_addr *ip, uint64_t mac)
{
struct neigh_ipv6_config *v6_config;
TAILQ_FOREACH(v6_config, &neigh6, next) {
- if (!(memcmp(v6_config->ip, ip, 16)) && (v6_config->mac == mac))
+ if (rte_ipv6_addr_eq(&v6_config->ip, ip) && v6_config->mac == mac)
return v6_config;
}
return NULL;
@@ -82,7 +82,7 @@ ip6_rewrite_node_add(struct neigh_ipv6_config *v6_config)
int16_t portid = 0;
int rc;
- portid = ethdev_portid_by_ip6(v6_config->ip, NULL);
+ portid = ethdev_portid_by_ip6(&v6_config->ip, NULL);
if (portid < 0) {
printf("Invalid portid found to add neigh\n");
return -EINVAL;
@@ -170,11 +170,10 @@ neigh_ip4_add(uint32_t ip, uint64_t mac)
}
static int
-neigh_ip6_add(uint8_t *ip, uint64_t mac)
+neigh_ip6_add(struct rte_ipv6_addr *ip, uint64_t mac)
{
struct neigh_ipv6_config *v6_config;
int rc = -EINVAL;
- int j;
v6_config = find_neigh6_entry(ip, mac);
@@ -184,9 +183,7 @@ neigh_ip6_add(uint8_t *ip, uint64_t mac)
return -ENOMEM;
}
- for (j = 0; j < ETHDEV_IPV6_ADDR_LEN; j++)
- v6_config->ip[j] = ip[j];
-
+ rte_ipv6_addr_cpy(&v6_config->ip, ip);
v6_config->mac = mac;
v6_config->is_used = true;
@@ -261,19 +258,18 @@ cmd_neigh_add_ipv6_parsed(void *parsed_result, __rte_unused struct cmdline *cl,
void *data __rte_unused)
{
struct cmd_neigh_add_ipv6_result *res = parsed_result;
- uint8_t ip[ETHDEV_IPV6_ADDR_LEN];
- int rc = -EINVAL, i;
+ struct rte_ipv6_addr ip;
+ int rc = -EINVAL;
uint64_t mac;
- for (i = 0; i < ETHDEV_IPV6_ADDR_LEN; i++)
- ip[i] = res->ip.addr.ipv6.s6_addr[i];
+ rte_memcpy(&ip, &res->ip.addr.ipv6, sizeof(ip));
if (parser_mac_read(&mac, res->mac)) {
printf(MSG_ARG_INVALID, "mac");
return;
}
- rc = neigh_ip6_add(ip, mac);
+ rc = neigh_ip6_add(&ip, mac);
if (rc < 0)
printf(MSG_CMD_FAIL, res->neigh);
}
diff --git a/app/graph/neigh_priv.h b/app/graph/neigh_priv.h
index 1a7106c309bc..b2a7607e01bb 100644
--- a/app/graph/neigh_priv.h
+++ b/app/graph/neigh_priv.h
@@ -5,6 +5,8 @@
#ifndef APP_GRAPH_NEIGH_PRIV_H
#define APP_GRAPH_NEIGH_PRIV_H
+#include <rte_ip6.h>
+
#define MAX_NEIGH_ENTRIES 32
struct neigh_ipv4_config {
@@ -18,7 +20,7 @@ TAILQ_HEAD(neigh4_head, neigh_ipv4_config);
struct neigh_ipv6_config {
TAILQ_ENTRY(neigh_ipv6_config) next;
- uint8_t ip[16];
+ struct rte_ipv6_addr ip;
uint64_t mac;
bool is_used;
};
diff --git a/app/graph/route.h b/app/graph/route.h
index 23a7951d2dad..455aab8634d4 100644
--- a/app/graph/route.h
+++ b/app/graph/route.h
@@ -5,6 +5,8 @@
#ifndef APP_GRAPH_ROUTE_H
#define APP_GRAPH_ROUTE_H
+#include <rte_ip6.h>
+
#define MAX_ROUTE_ENTRIES 32
struct route_ipv4_config {
@@ -19,9 +21,9 @@ TAILQ_HEAD(ip4_route, route_ipv4_config);
struct route_ipv6_config {
TAILQ_ENTRY(route_ipv6_config) next;
- uint8_t ip[16];
- uint8_t mask[16];
- uint8_t gateway[16];
+ struct rte_ipv6_addr ip;
+ struct rte_ipv6_addr mask;
+ struct rte_ipv6_addr gateway;
bool is_used;
};
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index ee958c689b3a..c268c497834c 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -76,8 +76,6 @@ Deprecation Notices
- ``struct rte_ipv6_tuple``
ipsec
- ``struct rte_ipsec_sadv6_key``
- node
- - ``rte_node_ip6_route_add()``
pipeline
- ``struct rte_table_action_ipv6_header``
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 984ef5e35fb9..1d272982810c 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -89,6 +89,8 @@ API Changes
``uint8_t[16]`` fields.
* rib6,fib6,lpm6: All public API functions were modified to use ``struct rte_ipv6_addr`` instead of
``uint8_t[16]`` parameters.
+* node: ``rte_node_ip6_route_add()`` was modified to use a ``struct rte_ipv6_addr`` instead of
+ ``uint8_t[16]`` parameter.
ABI Changes
-----------
diff --git a/examples/l3fwd-graph/main.c b/examples/l3fwd-graph/main.c
index a13dc011380d..c2067a05f3ef 100644
--- a/examples/l3fwd-graph/main.c
+++ b/examples/l3fwd-graph/main.c
@@ -151,7 +151,7 @@ struct ipv4_l3fwd_lpm_route {
};
struct ipv6_l3fwd_lpm_route {
- uint8_t ip[RTE_LPM6_IPV6_ADDR_SIZE];
+ struct rte_ipv6_addr ip;
uint8_t depth;
uint8_t if_out;
};
@@ -171,22 +171,14 @@ static struct ipv4_l3fwd_lpm_route ipv4_l3fwd_lpm_route_array[] = {
(sizeof(ipv6_l3fwd_lpm_route_array) / \
sizeof(ipv6_l3fwd_lpm_route_array[0]))
static struct ipv6_l3fwd_lpm_route ipv6_l3fwd_lpm_route_array[] = {
- {{0x20, 0x01, 0xdb, 0x08, 0x12, 0x34, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
- 0x00}, 48, 0},
- {{0x20, 0x01, 0xdb, 0x08, 0x12, 0x34, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
- 0x01}, 48, 1},
- {{0x20, 0x01, 0xdb, 0x08, 0x12, 0x34, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
- 0x02}, 48, 2},
- {{0x20, 0x01, 0xdb, 0x08, 0x12, 0x34, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
- 0x03}, 48, 3},
- {{0x20, 0x01, 0xdb, 0x08, 0x12, 0x34, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
- 0x04}, 48, 4},
- {{0x20, 0x01, 0xdb, 0x08, 0x12, 0x34, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
- 0x05}, 48, 5},
- {{0x20, 0x01, 0xdb, 0x08, 0x12, 0x34, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
- 0x06}, 48, 6},
- {{0x20, 0x01, 0xdb, 0x08, 0x12, 0x34, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
- 0x02}, 48, 7},
+ {{.a = "\x20\x01\xdb\x08\x12\x34\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"}, 48, 0},
+ {{.a = "\x20\x01\xdb\x08\x12\x34\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01"}, 48, 1},
+ {{.a = "\x20\x01\xdb\x08\x12\x34\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02"}, 48, 2},
+ {{.a = "\x20\x01\xdb\x08\x12\x34\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03"}, 48, 3},
+ {{.a = "\x20\x01\xdb\x08\x12\x34\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04"}, 48, 4},
+ {{.a = "\x20\x01\xdb\x08\x12\x34\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05"}, 48, 5},
+ {{.a = "\x20\x01\xdb\x08\x12\x34\x00\x00\x00\x00\x00\x00\x00\x00\x00\x06"}, 48, 6},
+ {{.a = "\x20\x01\xdb\x08\x12\x34\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02"}, 48, 7},
};
static int
@@ -1371,14 +1363,14 @@ main(int argc, char **argv)
dst_port = ipv6_l3fwd_lpm_route_array[i].if_out;
- memcpy(in6.s6_addr, ipv6_l3fwd_lpm_route_array[i].ip, RTE_LPM6_IPV6_ADDR_SIZE);
+ memcpy(in6.s6_addr, &ipv6_l3fwd_lpm_route_array[i].ip, RTE_IPV6_ADDR_SIZE);
snprintf(route_str, sizeof(route_str), "%s / %d (%d)",
inet_ntop(AF_INET6, &in6, abuf, sizeof(abuf)),
ipv6_l3fwd_lpm_route_array[i].depth,
ipv6_l3fwd_lpm_route_array[i].if_out);
/* Use route index 'i' as next hop id */
- ret = rte_node_ip6_route_add(ipv6_l3fwd_lpm_route_array[i].ip,
+ ret = rte_node_ip6_route_add(&ipv6_l3fwd_lpm_route_array[i].ip,
ipv6_l3fwd_lpm_route_array[i].depth, i,
RTE_NODE_IP6_LOOKUP_NEXT_REWRITE);
diff --git a/lib/node/ip6_lookup.c b/lib/node/ip6_lookup.c
index 1ad6e3fd18f2..0806500d86cf 100644
--- a/lib/node/ip6_lookup.c
+++ b/lib/node/ip6_lookup.c
@@ -112,28 +112,28 @@ ip6_lookup_node_process_scalar(struct rte_graph *graph, struct rte_node *node,
sizeof(struct rte_ether_hdr));
/* Extract hop_limits as ipv6 hdr is in cache */
node_mbuf_priv1(mbuf0, dyn)->ttl = ipv6_hdr->hop_limits;
- rte_memcpy(&ip_batch[0], &ipv6_hdr->dst_addr, 16);
+ rte_ipv6_addr_cpy(&ip_batch[0], &ipv6_hdr->dst_addr);
/* Extract DIP of mbuf1 */
ipv6_hdr = rte_pktmbuf_mtod_offset(mbuf1, struct rte_ipv6_hdr *,
sizeof(struct rte_ether_hdr));
/* Extract hop_limits as ipv6 hdr is in cache */
node_mbuf_priv1(mbuf1, dyn)->ttl = ipv6_hdr->hop_limits;
- rte_memcpy(&ip_batch[1], &ipv6_hdr->dst_addr, 16);
+ rte_ipv6_addr_cpy(&ip_batch[1], &ipv6_hdr->dst_addr);
/* Extract DIP of mbuf2 */
ipv6_hdr = rte_pktmbuf_mtod_offset(mbuf2, struct rte_ipv6_hdr *,
sizeof(struct rte_ether_hdr));
/* Extract hop_limits as ipv6 hdr is in cache */
node_mbuf_priv1(mbuf2, dyn)->ttl = ipv6_hdr->hop_limits;
- rte_memcpy(&ip_batch[2], &ipv6_hdr->dst_addr, 16);
+ rte_ipv6_addr_cpy(&ip_batch[2], &ipv6_hdr->dst_addr);
/* Extract DIP of mbuf3 */
ipv6_hdr = rte_pktmbuf_mtod_offset(mbuf3, struct rte_ipv6_hdr *,
sizeof(struct rte_ether_hdr));
/* Extract hop_limits as ipv6 hdr is in cache */
node_mbuf_priv1(mbuf3, dyn)->ttl = ipv6_hdr->hop_limits;
- rte_memcpy(&ip_batch[3], &ipv6_hdr->dst_addr, 16);
+ rte_ipv6_addr_cpy(&ip_batch[3], &ipv6_hdr->dst_addr);
rte_lpm6_lookup_bulk_func(lpm6, ip_batch, next_hop, 4);
@@ -258,17 +258,15 @@ ip6_lookup_node_process_scalar(struct rte_graph *graph, struct rte_node *node,
}
int
-rte_node_ip6_route_add(const uint8_t *ip, uint8_t depth, uint16_t next_hop,
+rte_node_ip6_route_add(const struct rte_ipv6_addr *ip, uint8_t depth, uint16_t next_hop,
enum rte_node_ip6_lookup_next next_node)
{
char abuf[INET6_ADDRSTRLEN];
- struct in6_addr in6;
uint8_t socket;
uint32_t val;
int ret;
- memcpy(in6.s6_addr, ip, RTE_LPM6_IPV6_ADDR_SIZE);
- inet_ntop(AF_INET6, &in6, abuf, sizeof(abuf));
+ inet_ntop(AF_INET6, ip, abuf, sizeof(abuf));
/* Embedded next node id into 24 bit next hop */
val = ((next_node << 16) | next_hop) & ((1ull << 24) - 1);
node_dbg("ip6_lookup", "LPM: Adding route %s / %d nh (0x%x)", abuf,
@@ -278,8 +276,7 @@ rte_node_ip6_route_add(const uint8_t *ip, uint8_t depth, uint16_t next_hop,
if (!ip6_lookup_nm.lpm_tbl[socket])
continue;
- ret = rte_lpm6_add(ip6_lookup_nm.lpm_tbl[socket],
- (const struct rte_ipv6_addr *)ip, depth, val);
+ ret = rte_lpm6_add(ip6_lookup_nm.lpm_tbl[socket], ip, depth, val);
if (ret < 0) {
node_err("ip6_lookup",
"Unable to add entry %s / %d nh (%x) to LPM "
diff --git a/lib/node/rte_node_ip6_api.h b/lib/node/rte_node_ip6_api.h
index a538dc2ea701..dc8c86184afe 100644
--- a/lib/node/rte_node_ip6_api.h
+++ b/lib/node/rte_node_ip6_api.h
@@ -21,6 +21,7 @@ extern "C" {
#include <rte_common.h>
#include <rte_compat.h>
+#include <rte_ip6.h>
/**
* IP6 lookup next nodes.
@@ -48,7 +49,7 @@ enum rte_node_ip6_lookup_next {
* 0 on success, negative otherwise.
*/
__rte_experimental
-int rte_node_ip6_route_add(const uint8_t *ip, uint8_t depth, uint16_t next_hop,
+int rte_node_ip6_route_add(const struct rte_ipv6_addr *ip, uint8_t depth, uint16_t next_hop,
enum rte_node_ip6_lookup_next next_node);
/**
--
2.46.1
^ permalink raw reply [relevance 2%]
* [PATCH dpdk v2 09/16] pipeline: use ipv6 addr struct
` (4 preceding siblings ...)
2024-10-01 8:17 2% ` [PATCH dpdk v2 08/16] graph,node: use ipv6 addr struct and utils Robin Jarry
@ 2024-10-01 8:17 3% ` Robin Jarry
2024-10-01 8:17 3% ` [PATCH dpdk v2 10/16] ipsec: " Robin Jarry
` (4 subsequent siblings)
10 siblings, 0 replies; 200+ results
From: Robin Jarry @ 2024-10-01 8:17 UTC (permalink / raw)
To: dev, Cristian Dumitrescu
Update rte_table_action_ipv6_header and rte_table_action_nat_params to
use the recently added IPv6 address structure instead of uint8_t[16]
arrays.
Signed-off-by: Robin Jarry <rjarry@redhat.com>
---
Notes:
v2: updated changelog for 24.11
doc/guides/rel_notes/deprecation.rst | 2 --
doc/guides/rel_notes/release_24_11.rst | 2 ++
examples/ip_pipeline/cli.c | 12 ++++-----
lib/pipeline/rte_table_action.c | 36 ++++++++++----------------
lib/pipeline/rte_table_action.h | 7 ++---
5 files changed, 26 insertions(+), 33 deletions(-)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index c268c497834c..fcccb22593d8 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -76,8 +76,6 @@ Deprecation Notices
- ``struct rte_ipv6_tuple``
ipsec
- ``struct rte_ipsec_sadv6_key``
- pipeline
- - ``struct rte_table_action_ipv6_header``
* net, ethdev: The flow item ``RTE_FLOW_ITEM_TYPE_VXLAN_GPE``
is replaced with ``RTE_FLOW_ITEM_TYPE_VXLAN``.
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 1d272982810c..3eb43ba8520f 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -91,6 +91,8 @@ API Changes
``uint8_t[16]`` parameters.
* node: ``rte_node_ip6_route_add()`` was modified to use a ``struct rte_ipv6_addr`` instead of
``uint8_t[16]`` parameter.
+* pipeline: ``rte_table_action_ipv6_header`` and ``rte_table_action_nat_params`` were modified to
+ use ``rte_ipv6_addr`` structures instead of ``uint8_t[16]`` fields.
ABI Changes
-----------
diff --git a/examples/ip_pipeline/cli.c b/examples/ip_pipeline/cli.c
index e8269ea90c11..66fdeac8f501 100644
--- a/examples/ip_pipeline/cli.c
+++ b/examples/ip_pipeline/cli.c
@@ -3540,8 +3540,8 @@ parse_table_action_encap(char **tokens,
parser_read_uint8(&hop_limit, tokens[5]))
return 0;
- memcpy(a->encap.vxlan.ipv6.sa, sa.s6_addr, 16);
- memcpy(a->encap.vxlan.ipv6.da, da.s6_addr, 16);
+ memcpy(&a->encap.vxlan.ipv6.sa, sa.s6_addr, 16);
+ memcpy(&a->encap.vxlan.ipv6.da, da.s6_addr, 16);
a->encap.vxlan.ipv6.flow_label = flow_label;
a->encap.vxlan.ipv6.dscp = dscp;
a->encap.vxlan.ipv6.hop_limit = hop_limit;
@@ -3615,7 +3615,7 @@ parse_table_action_nat(char **tokens,
return 0;
a->nat.ip_version = 0;
- memcpy(a->nat.addr.ipv6, addr.s6_addr, 16);
+ memcpy(&a->nat.addr.ipv6, addr.s6_addr, 16);
a->nat.port = port;
a->action_mask |= 1 << RTE_TABLE_ACTION_NAT;
return 4;
@@ -4956,9 +4956,9 @@ table_rule_show(const char *pipeline_name,
(uint32_t)a->encap.vxlan.ipv4.ttl);
} else {
fprintf(f, " ipv6 ");
- ipv6_addr_show(f, a->encap.vxlan.ipv6.sa);
+ ipv6_addr_show(f, a->encap.vxlan.ipv6.sa.a);
fprintf(f, " ");
- ipv6_addr_show(f, a->encap.vxlan.ipv6.da);
+ ipv6_addr_show(f, a->encap.vxlan.ipv6.da.a);
fprintf(f, " %u %u %u ",
a->encap.vxlan.ipv6.flow_label,
(uint32_t)a->encap.vxlan.ipv6.dscp,
@@ -4980,7 +4980,7 @@ table_rule_show(const char *pipeline_name,
if (a->nat.ip_version)
ipv4_addr_show(f, a->nat.addr.ipv4);
else
- ipv6_addr_show(f, a->nat.addr.ipv6);
+ ipv6_addr_show(f, a->nat.addr.ipv6.a);
fprintf(f, " %u ", (uint32_t)(a->nat.port));
}
diff --git a/lib/pipeline/rte_table_action.c b/lib/pipeline/rte_table_action.c
index a04d5121e1f0..8479f984e4ea 100644
--- a/lib/pipeline/rte_table_action.c
+++ b/lib/pipeline/rte_table_action.c
@@ -872,12 +872,8 @@ encap_vxlan_apply(void *data,
d->ipv6.payload_len = 0; /* not pre-computed */
d->ipv6.proto = IP_PROTO_UDP;
d->ipv6.hop_limits = p->vxlan.ipv6.hop_limit;
- memcpy(&d->ipv6.src_addr,
- p->vxlan.ipv6.sa,
- sizeof(p->vxlan.ipv6.sa));
- memcpy(&d->ipv6.dst_addr,
- p->vxlan.ipv6.da,
- sizeof(p->vxlan.ipv6.da));
+ rte_ipv6_addr_cpy(&d->ipv6.src_addr, &p->vxlan.ipv6.sa);
+ rte_ipv6_addr_cpy(&d->ipv6.dst_addr, &p->vxlan.ipv6.da);
/* UDP */
d->udp.src_port = rte_htons(p->vxlan.udp.sp);
@@ -907,12 +903,8 @@ encap_vxlan_apply(void *data,
d->ipv6.payload_len = 0; /* not pre-computed */
d->ipv6.proto = IP_PROTO_UDP;
d->ipv6.hop_limits = p->vxlan.ipv6.hop_limit;
- memcpy(&d->ipv6.src_addr,
- p->vxlan.ipv6.sa,
- sizeof(p->vxlan.ipv6.sa));
- memcpy(&d->ipv6.dst_addr,
- p->vxlan.ipv6.da,
- sizeof(p->vxlan.ipv6.da));
+ rte_ipv6_addr_cpy(&d->ipv6.src_addr, &p->vxlan.ipv6.sa);
+ rte_ipv6_addr_cpy(&d->ipv6.dst_addr, &p->vxlan.ipv6.da);
/* UDP */
d->udp.src_port = rte_htons(p->vxlan.udp.sp);
@@ -1211,7 +1203,7 @@ struct nat_ipv4_data {
} __rte_packed;
struct nat_ipv6_data {
- uint8_t addr[16];
+ struct rte_ipv6_addr addr;
uint16_t port;
} __rte_packed;
@@ -1258,7 +1250,7 @@ nat_apply(void *data,
} else {
struct nat_ipv6_data *d = data;
- memcpy(d->addr, p->addr.ipv6, sizeof(d->addr));
+ rte_ipv6_addr_cpy(&d->addr, &p->addr.ipv6);
d->port = rte_htons(p->port);
}
@@ -1438,11 +1430,11 @@ pkt_ipv6_work_nat(struct rte_ipv6_hdr *ip,
tcp_cksum = nat_ipv6_tcp_udp_checksum_update(tcp->cksum,
(uint16_t *)&ip->src_addr,
- (uint16_t *)data->addr,
+ (uint16_t *)&data->addr,
tcp->src_port,
data->port);
- rte_memcpy(&ip->src_addr, data->addr, 16);
+ rte_ipv6_addr_cpy(&ip->src_addr, &data->addr);
tcp->src_port = data->port;
tcp->cksum = tcp_cksum;
} else {
@@ -1451,11 +1443,11 @@ pkt_ipv6_work_nat(struct rte_ipv6_hdr *ip,
udp_cksum = nat_ipv6_tcp_udp_checksum_update(udp->dgram_cksum,
(uint16_t *)&ip->src_addr,
- (uint16_t *)data->addr,
+ (uint16_t *)&data->addr,
udp->src_port,
data->port);
- rte_memcpy(&ip->src_addr, data->addr, 16);
+ rte_ipv6_addr_cpy(&ip->src_addr, &data->addr);
udp->src_port = data->port;
udp->dgram_cksum = udp_cksum;
}
@@ -1466,11 +1458,11 @@ pkt_ipv6_work_nat(struct rte_ipv6_hdr *ip,
tcp_cksum = nat_ipv6_tcp_udp_checksum_update(tcp->cksum,
(uint16_t *)&ip->dst_addr,
- (uint16_t *)data->addr,
+ (uint16_t *)&data->addr,
tcp->dst_port,
data->port);
- rte_memcpy(&ip->dst_addr, data->addr, 16);
+ rte_ipv6_addr_cpy(&ip->dst_addr, &data->addr);
tcp->dst_port = data->port;
tcp->cksum = tcp_cksum;
} else {
@@ -1479,11 +1471,11 @@ pkt_ipv6_work_nat(struct rte_ipv6_hdr *ip,
udp_cksum = nat_ipv6_tcp_udp_checksum_update(udp->dgram_cksum,
(uint16_t *)&ip->dst_addr.a,
- (uint16_t *)data->addr,
+ (uint16_t *)&data->addr,
udp->dst_port,
data->port);
- rte_memcpy(&ip->dst_addr, data->addr, 16);
+ rte_ipv6_addr_cpy(&ip->dst_addr, &data->addr);
udp->dst_port = data->port;
udp->dgram_cksum = udp_cksum;
}
diff --git a/lib/pipeline/rte_table_action.h b/lib/pipeline/rte_table_action.h
index 5dffbeb70031..b119ac03dc6e 100644
--- a/lib/pipeline/rte_table_action.h
+++ b/lib/pipeline/rte_table_action.h
@@ -60,6 +60,7 @@ extern "C" {
#include <rte_compat.h>
#include <rte_ether.h>
+#include <rte_ip6.h>
#include <rte_meter.h>
#include <rte_table_hash.h>
@@ -419,8 +420,8 @@ struct rte_table_action_ipv4_header {
/** Pre-computed IPv6 header fields for encapsulation action. */
struct rte_table_action_ipv6_header {
- uint8_t sa[16]; /**< Source address. */
- uint8_t da[16]; /**< Destination address. */
+ struct rte_ipv6_addr sa; /**< Source address. */
+ struct rte_ipv6_addr da; /**< Destination address. */
uint32_t flow_label; /**< Flow label. */
uint8_t dscp; /**< DiffServ Code Point (DSCP). */
uint8_t hop_limit; /**< Hop Limit (HL). */
@@ -597,7 +598,7 @@ struct rte_table_action_nat_params {
uint32_t ipv4;
/** IPv6 address; only valid when *ip_version* is set to 0. */
- uint8_t ipv6[16];
+ struct rte_ipv6_addr ipv6;
} addr;
/** Port. */
--
2.46.1
^ permalink raw reply [relevance 3%]
* [PATCH dpdk v2 05/16] fib6,rib6,lpm6: use ipv6 addr struct
` (2 preceding siblings ...)
2024-10-01 8:17 1% ` [PATCH dpdk v2 04/16] net: use ipv6 structure for header addresses Robin Jarry
@ 2024-10-01 8:17 1% ` Robin Jarry
2024-10-01 8:17 2% ` [PATCH dpdk v2 08/16] graph,node: use ipv6 addr struct and utils Robin Jarry
` (6 subsequent siblings)
10 siblings, 0 replies; 200+ results
From: Robin Jarry @ 2024-10-01 8:17 UTC (permalink / raw)
To: dev, Vladimir Medvedkin, Cristian Dumitrescu, Bruce Richardson,
Konstantin Ananyev, Wathsala Vithanage, Radu Nicolau,
Akhil Goyal, Nithin Dabilpuram, Pavan Nikhilesh
Replace ad-hoc uint8_t[16] array types in the API of rte_fib6, rte_rib6
and rte_lpm6. Update all code accordingly.
Unfortunately, these three libraries are strongly intertwined, it is not
possible to split this change in separate commits.
Signed-off-by: Robin Jarry <rjarry@redhat.com>
---
Notes:
v2: updated changelog for 24.11
app/test-fib/main.c | 51 +-
app/test-pipeline/pipeline_lpm_ipv6.c | 10 +-
app/test/test_fib6.c | 52 +-
app/test/test_fib6_perf.c | 8 +-
app/test/test_lpm6.c | 523 +++---
app/test/test_lpm6_data.h | 2025 ++++++++++++------------
app/test/test_lpm6_perf.c | 10 +-
app/test/test_rib6.c | 65 +-
app/test/test_table_combined.c | 2 +-
app/test/test_table_tables.c | 8 +-
doc/guides/rel_notes/deprecation.rst | 20 -
doc/guides/rel_notes/release_24_11.rst | 2 +
examples/ip_fragmentation/main.c | 24 +-
examples/ip_pipeline/thread.c | 2 +-
examples/ip_reassembly/main.c | 24 +-
examples/ipsec-secgw/ipsec_lpm_neon.h | 7 +-
examples/ipsec-secgw/ipsec_worker.c | 6 +-
examples/ipsec-secgw/ipsec_worker.h | 4 +-
examples/ipsec-secgw/rt.c | 22 +-
examples/l3fwd/l3fwd_fib.c | 39 +-
examples/l3fwd/l3fwd_lpm.c | 8 +-
examples/l3fwd/l3fwd_route.h | 7 +-
examples/l3fwd/lpm_route_parse.c | 6 +-
lib/fib/meson.build | 4 +-
lib/fib/rte_fib6.c | 13 +-
lib/fib/rte_fib6.h | 11 +-
lib/fib/trie.c | 99 +-
lib/fib/trie.h | 16 +-
lib/fib/trie_avx512.c | 38 +-
lib/fib/trie_avx512.h | 10 +-
lib/lpm/meson.build | 1 +
lib/lpm/rte_lpm6.c | 106 +-
lib/lpm/rte_lpm6.h | 14 +-
lib/node/ip6_lookup.c | 16 +-
lib/rib/meson.build | 2 +-
lib/rib/rte_rib6.c | 74 +-
lib/rib/rte_rib6.h | 13 +-
lib/table/rte_table_lpm_ipv6.c | 12 +-
lib/table/rte_table_lpm_ipv6.h | 4 +-
39 files changed, 1683 insertions(+), 1675 deletions(-)
diff --git a/app/test-fib/main.c b/app/test-fib/main.c
index c49bfe8bcec3..a43a0ae4df5d 100644
--- a/app/test-fib/main.c
+++ b/app/test-fib/main.c
@@ -123,7 +123,7 @@ struct rt_rule_4 {
};
struct rt_rule_6 {
- uint8_t addr[16];
+ struct rte_ipv6_addr addr;
uint8_t depth;
uint64_t nh;
};
@@ -306,15 +306,15 @@ shuffle_rt_6(struct rt_rule_6 *rt, int n)
for (i = 0; i < n; i++) {
j = rte_rand() % n;
- memcpy(tmp.addr, rt[i].addr, 16);
+ memcpy(&tmp.addr, &rt[i].addr, 16);
tmp.depth = rt[i].depth;
tmp.nh = rt[i].nh;
- memcpy(rt[i].addr, rt[j].addr, 16);
+ memcpy(&rt[i].addr, &rt[j].addr, 16);
rt[i].depth = rt[j].depth;
rt[i].nh = rt[j].nh;
- memcpy(rt[j].addr, tmp.addr, 16);
+ memcpy(&rt[j].addr, &tmp.addr, 16);
rt[j].depth = tmp.depth;
rt[j].nh = tmp.nh;
}
@@ -364,7 +364,7 @@ gen_random_rt_6(struct rt_rule_6 *rt, int nh_sz)
uint32_t a, i, j, k = 0;
if (config.nb_routes_per_depth[0] != 0) {
- memset(rt[k].addr, 0, 16);
+ memset(&rt[k].addr, 0, 16);
rt[k].depth = 0;
rt[k++].nh = rte_rand() & get_max_nh(nh_sz);
}
@@ -380,7 +380,7 @@ gen_random_rt_6(struct rt_rule_6 *rt, int nh_sz)
uint64_t rnd_val = get_rnd_rng((uint64_t)edge,
(uint64_t)(edge + step));
rnd = rte_cpu_to_be_32(rnd_val << (32 - i));
- complete_v6_addr((uint32_t *)rt[k].addr,
+ complete_v6_addr((uint32_t *)&rt[k].addr,
rnd, a);
rt[k].depth = (a * 32) + i;
rt[k].nh = rte_rand() & get_max_nh(nh_sz);
@@ -390,19 +390,19 @@ gen_random_rt_6(struct rt_rule_6 *rt, int nh_sz)
}
static inline void
-set_rnd_ipv6(uint8_t *addr, uint8_t *route, int depth)
+set_rnd_ipv6(struct rte_ipv6_addr *addr, struct rte_ipv6_addr *route, int depth)
{
int i;
for (i = 0; i < 16; i++)
- addr[i] = rte_rand();
+ addr->a[i] = rte_rand();
for (i = 0; i < 16; i++) {
if (depth >= 8)
- addr[i] = route[i];
+ addr->a[i] = route->a[i];
else if (depth > 0) {
- addr[i] &= (uint16_t)UINT8_MAX >> depth;
- addr[i] |= route[i] & UINT8_MAX << (8 - depth);
+ addr->a[i] &= (uint16_t)UINT8_MAX >> depth;
+ addr->a[i] |= route->a[i] & UINT8_MAX << (8 - depth);
} else
return;
depth -= 8;
@@ -413,7 +413,7 @@ static void
gen_rnd_lookup_tbl(int af)
{
uint32_t *tbl4 = config.lookup_tbl;
- uint8_t *tbl6 = config.lookup_tbl;
+ struct rte_ipv6_addr *tbl6 = config.lookup_tbl;
struct rt_rule_4 *rt4 = (struct rt_rule_4 *)config.rt;
struct rt_rule_6 *rt6 = (struct rt_rule_6 *)config.rt;
uint32_t i, j;
@@ -432,11 +432,10 @@ gen_rnd_lookup_tbl(int af)
for (i = 0, j = 0; i < config.nb_lookup_ips;
i++, j = (j + 1) % config.nb_routes) {
if ((rte_rand() % 100) < config.rnd_lookup_ips_ratio) {
- set_rnd_ipv6(&tbl6[i * 16], rt6[j].addr, 0);
+ set_rnd_ipv6(&tbl6[i], &rt6[j].addr, 0);
config.nb_lookup_ips_rnd++;
} else {
- set_rnd_ipv6(&tbl6[i * 16], rt6[j].addr,
- rt6[j].depth);
+ set_rnd_ipv6(&tbl6[i], &rt6[j].addr, rt6[j].depth);
}
}
}
@@ -522,7 +521,7 @@ parse_rt_6(FILE *f)
s = NULL;
}
- ret = _inet_net_pton(AF_INET6, in[RT_PREFIX], rt[j].addr);
+ ret = _inet_net_pton(AF_INET6, in[RT_PREFIX], &rt[j].addr);
if (ret < 0)
return ret;
@@ -1023,7 +1022,7 @@ dump_rt_6(struct rt_rule_6 *rt)
}
for (i = 0; i < config.nb_routes; i++) {
- fprintf(f, NIPQUAD6_FMT"/%d %"PRIu64"\n", NIPQUAD6(rt[i].addr),
+ fprintf(f, NIPQUAD6_FMT"/%d %"PRIu64"\n", NIPQUAD6(&rt[i].addr),
rt[i].depth, rt[i].nh);
}
@@ -1043,7 +1042,7 @@ run_v6(void)
int ret = 0;
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config lpm_conf;
- uint8_t *tbl6;
+ struct rte_ipv6_addr *tbl6;
uint64_t fib_nh[BURST_SZ];
int32_t lpm_nh[BURST_SZ];
@@ -1094,7 +1093,7 @@ run_v6(void)
for (k = config.print_fract, i = 0; k > 0; k--) {
start = rte_rdtsc_precise();
for (j = 0; j < (config.nb_routes - i) / k; j++) {
- ret = rte_fib6_add(fib, rt[i + j].addr,
+ ret = rte_fib6_add(fib, &rt[i + j].addr,
rt[i + j].depth, rt[i + j].nh);
if (unlikely(ret != 0)) {
printf("Can not add a route to FIB, err %d\n",
@@ -1120,7 +1119,7 @@ run_v6(void)
for (k = config.print_fract, i = 0; k > 0; k--) {
start = rte_rdtsc_precise();
for (j = 0; j < (config.nb_routes - i) / k; j++) {
- ret = rte_lpm6_add(lpm, rt[i + j].addr,
+ ret = rte_lpm6_add(lpm, &rt[i + j].addr,
rt[i + j].depth, rt[i + j].nh);
if (ret != 0) {
if (rt[i + j].depth == 0)
@@ -1139,7 +1138,7 @@ run_v6(void)
acc = 0;
for (i = 0; i < config.nb_lookup_ips; i += BURST_SZ) {
start = rte_rdtsc_precise();
- ret = rte_fib6_lookup_bulk(fib, (uint8_t (*)[16])(tbl6 + i*16),
+ ret = rte_fib6_lookup_bulk(fib, &tbl6[i],
fib_nh, BURST_SZ);
acc += rte_rdtsc_precise() - start;
if (ret != 0) {
@@ -1154,7 +1153,7 @@ run_v6(void)
for (i = 0; i < config.nb_lookup_ips; i += BURST_SZ) {
start = rte_rdtsc_precise();
ret = rte_lpm6_lookup_bulk_func(lpm,
- (uint8_t (*)[16])(tbl6 + i*16),
+ &tbl6[i],
lpm_nh, BURST_SZ);
acc += rte_rdtsc_precise() - start;
if (ret != 0) {
@@ -1166,10 +1165,10 @@ run_v6(void)
for (i = 0; i < config.nb_lookup_ips; i += BURST_SZ) {
rte_fib6_lookup_bulk(fib,
- (uint8_t (*)[16])(tbl6 + i*16),
+ &tbl6[i],
fib_nh, BURST_SZ);
rte_lpm6_lookup_bulk_func(lpm,
- (uint8_t (*)[16])(tbl6 + i*16),
+ &tbl6[i],
lpm_nh, BURST_SZ);
for (j = 0; j < BURST_SZ; j++) {
if ((fib_nh[j] != (uint32_t)lpm_nh[j]) &&
@@ -1186,7 +1185,7 @@ run_v6(void)
for (k = config.print_fract, i = 0; k > 0; k--) {
start = rte_rdtsc_precise();
for (j = 0; j < (config.nb_routes - i) / k; j++)
- rte_fib6_delete(fib, rt[i + j].addr, rt[i + j].depth);
+ rte_fib6_delete(fib, &rt[i + j].addr, rt[i + j].depth);
printf("AVG FIB delete %"PRIu64"\n",
(rte_rdtsc_precise() - start) / j);
@@ -1197,7 +1196,7 @@ run_v6(void)
for (k = config.print_fract, i = 0; k > 0; k--) {
start = rte_rdtsc_precise();
for (j = 0; j < (config.nb_routes - i) / k; j++)
- rte_lpm6_delete(lpm, rt[i + j].addr,
+ rte_lpm6_delete(lpm, &rt[i + j].addr,
rt[i + j].depth);
printf("AVG LPM delete %"PRIu64"\n",
diff --git a/app/test-pipeline/pipeline_lpm_ipv6.c b/app/test-pipeline/pipeline_lpm_ipv6.c
index 207ffbeff00f..cfebfbbc81ca 100644
--- a/app/test-pipeline/pipeline_lpm_ipv6.c
+++ b/app/test-pipeline/pipeline_lpm_ipv6.c
@@ -127,16 +127,16 @@ app_main_loop_worker_pipeline_lpm_ipv6(void) {
ip = rte_bswap32(i << (24 -
rte_popcount32(app.n_ports - 1)));
- memcpy(key.ip, &ip, sizeof(uint32_t));
+ memcpy(&key.ip, &ip, sizeof(uint32_t));
printf("Adding rule to IPv6 LPM table (IPv6 destination = "
"%.2x%.2x:%.2x%.2x:%.2x%.2x:%.2x%.2x:"
"%.2x%.2x:%.2x%.2x:%.2x%.2x:%.2x%.2x/%u => "
"port out = %u)\n",
- key.ip[0], key.ip[1], key.ip[2], key.ip[3],
- key.ip[4], key.ip[5], key.ip[6], key.ip[7],
- key.ip[8], key.ip[9], key.ip[10], key.ip[11],
- key.ip[12], key.ip[13], key.ip[14], key.ip[15],
+ key.ip.a[0], key.ip.a[1], key.ip.a[2], key.ip.a[3],
+ key.ip.a[4], key.ip.a[5], key.ip.a[6], key.ip.a[7],
+ key.ip.a[8], key.ip.a[9], key.ip.a[10], key.ip.a[11],
+ key.ip.a[12], key.ip.a[13], key.ip.a[14], key.ip.a[15],
key.depth, i);
status = rte_pipeline_table_entry_add(p, table_id, &key, &entry,
diff --git a/app/test/test_fib6.c b/app/test/test_fib6.c
index 2f836238fbf7..c3b947d789bb 100644
--- a/app/test/test_fib6.c
+++ b/app/test/test_fib6.c
@@ -147,7 +147,7 @@ test_add_del_invalid(void)
struct rte_fib6 *fib = NULL;
struct rte_fib6_conf config;
uint64_t nh = 100;
- uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE] = {0};
+ struct rte_ipv6_addr ip = {0};
int ret;
uint8_t depth = 24;
@@ -157,12 +157,12 @@ test_add_del_invalid(void)
config.type = RTE_FIB6_DUMMY;
/* rte_fib6_add: fib == NULL */
- ret = rte_fib6_add(NULL, ip, depth, nh);
+ ret = rte_fib6_add(NULL, &ip, depth, nh);
RTE_TEST_ASSERT(ret < 0,
"Call succeeded with invalid parameters\n");
/* rte_fib6_delete: fib == NULL */
- ret = rte_fib6_delete(NULL, ip, depth);
+ ret = rte_fib6_delete(NULL, &ip, depth);
RTE_TEST_ASSERT(ret < 0,
"Call succeeded with invalid parameters\n");
@@ -171,12 +171,12 @@ test_add_del_invalid(void)
RTE_TEST_ASSERT(fib != NULL, "Failed to create FIB\n");
/* rte_fib6_add: depth > RTE_FIB6_MAXDEPTH */
- ret = rte_fib6_add(fib, ip, RTE_FIB6_MAXDEPTH + 1, nh);
+ ret = rte_fib6_add(fib, &ip, RTE_FIB6_MAXDEPTH + 1, nh);
RTE_TEST_ASSERT(ret < 0,
"Call succeeded with invalid parameters\n");
/* rte_fib6_delete: depth > RTE_FIB6_MAXDEPTH */
- ret = rte_fib6_delete(fib, ip, RTE_FIB6_MAXDEPTH + 1);
+ ret = rte_fib6_delete(fib, &ip, RTE_FIB6_MAXDEPTH + 1);
RTE_TEST_ASSERT(ret < 0,
"Call succeeded with invalid parameters\n");
@@ -212,8 +212,8 @@ test_get_invalid(void)
*/
static int
lookup_and_check_asc(struct rte_fib6 *fib,
- uint8_t ip_arr[RTE_FIB6_MAXDEPTH][RTE_FIB6_IPV6_ADDR_SIZE],
- uint8_t ip_missing[][RTE_FIB6_IPV6_ADDR_SIZE], uint64_t def_nh,
+ struct rte_ipv6_addr *ip_arr,
+ struct rte_ipv6_addr *ip_missing, uint64_t def_nh,
uint32_t n)
{
uint64_t nh_arr[RTE_FIB6_MAXDEPTH];
@@ -240,8 +240,8 @@ lookup_and_check_asc(struct rte_fib6 *fib,
static int
lookup_and_check_desc(struct rte_fib6 *fib,
- uint8_t ip_arr[RTE_FIB6_MAXDEPTH][RTE_FIB6_IPV6_ADDR_SIZE],
- uint8_t ip_missing[][RTE_FIB6_IPV6_ADDR_SIZE], uint64_t def_nh,
+ struct rte_ipv6_addr *ip_arr,
+ struct rte_ipv6_addr *ip_missing, uint64_t def_nh,
uint32_t n)
{
uint64_t nh_arr[RTE_FIB6_MAXDEPTH];
@@ -270,61 +270,61 @@ static int
check_fib(struct rte_fib6 *fib)
{
uint64_t def_nh = 100;
- uint8_t ip_arr[RTE_FIB6_MAXDEPTH][RTE_FIB6_IPV6_ADDR_SIZE];
- uint8_t ip_add[RTE_FIB6_IPV6_ADDR_SIZE] = {0};
- uint8_t ip_missing[1][RTE_FIB6_IPV6_ADDR_SIZE] = { {255} };
+ struct rte_ipv6_addr ip_arr[RTE_FIB6_MAXDEPTH];
+ struct rte_ipv6_addr ip_add = {.a = {[0] = 128}};
+ struct rte_ipv6_addr ip_missing = {
+ .a = {[0] = 127, [1 ... 15] = 255},
+ };
uint32_t i, j;
int ret;
- ip_add[0] = 128;
- ip_missing[0][0] = 127;
for (i = 0; i < RTE_FIB6_MAXDEPTH; i++) {
for (j = 0; j < RTE_FIB6_IPV6_ADDR_SIZE; j++) {
- ip_arr[i][j] = ip_add[j] |
+ ip_arr[i].a[j] = ip_add.a[j] |
~get_msk_part(RTE_FIB6_MAXDEPTH - i, j);
}
}
- ret = lookup_and_check_desc(fib, ip_arr, ip_missing, def_nh, 0);
+ ret = lookup_and_check_desc(fib, ip_arr, &ip_missing, def_nh, 0);
RTE_TEST_ASSERT(ret == TEST_SUCCESS, "Lookup and check fails\n");
for (i = 1; i <= RTE_FIB6_MAXDEPTH; i++) {
- ret = rte_fib6_add(fib, ip_add, i, i);
+ ret = rte_fib6_add(fib, &ip_add, i, i);
RTE_TEST_ASSERT(ret == 0, "Failed to add a route\n");
- ret = lookup_and_check_asc(fib, ip_arr, ip_missing, def_nh, i);
+ ret = lookup_and_check_asc(fib, ip_arr, &ip_missing, def_nh, i);
RTE_TEST_ASSERT(ret == TEST_SUCCESS,
"Lookup and check fails\n");
}
for (i = RTE_FIB6_MAXDEPTH; i > 1; i--) {
- ret = rte_fib6_delete(fib, ip_add, i);
+ ret = rte_fib6_delete(fib, &ip_add, i);
RTE_TEST_ASSERT(ret == 0, "Failed to delete a route\n");
- ret = lookup_and_check_asc(fib, ip_arr, ip_missing,
+ ret = lookup_and_check_asc(fib, ip_arr, &ip_missing,
def_nh, i - 1);
RTE_TEST_ASSERT(ret == TEST_SUCCESS,
"Lookup and check fails\n");
}
- ret = rte_fib6_delete(fib, ip_add, i);
+ ret = rte_fib6_delete(fib, &ip_add, i);
RTE_TEST_ASSERT(ret == 0, "Failed to delete a route\n");
- ret = lookup_and_check_desc(fib, ip_arr, ip_missing, def_nh, 0);
+ ret = lookup_and_check_desc(fib, ip_arr, &ip_missing, def_nh, 0);
RTE_TEST_ASSERT(ret == TEST_SUCCESS,
"Lookup and check fails\n");
for (i = 0; i < RTE_FIB6_MAXDEPTH; i++) {
- ret = rte_fib6_add(fib, ip_add, RTE_FIB6_MAXDEPTH - i,
+ ret = rte_fib6_add(fib, &ip_add, RTE_FIB6_MAXDEPTH - i,
RTE_FIB6_MAXDEPTH - i);
RTE_TEST_ASSERT(ret == 0, "Failed to add a route\n");
- ret = lookup_and_check_desc(fib, ip_arr, ip_missing,
+ ret = lookup_and_check_desc(fib, ip_arr, &ip_missing,
def_nh, i + 1);
RTE_TEST_ASSERT(ret == TEST_SUCCESS,
"Lookup and check fails\n");
}
for (i = 1; i <= RTE_FIB6_MAXDEPTH; i++) {
- ret = rte_fib6_delete(fib, ip_add, i);
+ ret = rte_fib6_delete(fib, &ip_add, i);
RTE_TEST_ASSERT(ret == 0, "Failed to delete a route\n");
- ret = lookup_and_check_desc(fib, ip_arr, ip_missing, def_nh,
+ ret = lookup_and_check_desc(fib, ip_arr, &ip_missing, def_nh,
RTE_FIB6_MAXDEPTH - i);
RTE_TEST_ASSERT(ret == TEST_SUCCESS,
"Lookup and check fails\n");
diff --git a/app/test/test_fib6_perf.c b/app/test/test_fib6_perf.c
index fe713e7094e5..d51bbdbdd6fa 100644
--- a/app/test/test_fib6_perf.c
+++ b/app/test/test_fib6_perf.c
@@ -73,7 +73,7 @@ test_fib6_perf(void)
uint64_t next_hop_add;
int status = 0;
int64_t count = 0;
- uint8_t ip_batch[NUM_IPS_ENTRIES][16];
+ struct rte_ipv6_addr ip_batch[NUM_IPS_ENTRIES];
uint64_t next_hops[NUM_IPS_ENTRIES];
conf.type = RTE_FIB6_TRIE;
@@ -101,7 +101,7 @@ test_fib6_perf(void)
for (i = 0; i < NUM_ROUTE_ENTRIES; i++) {
next_hop_add = (i & ((1 << 14) - 1)) + 1;
- if (rte_fib6_add(fib, large_route_table[i].ip,
+ if (rte_fib6_add(fib, &large_route_table[i].ip,
large_route_table[i].depth, next_hop_add) == 0)
status++;
}
@@ -117,7 +117,7 @@ test_fib6_perf(void)
count = 0;
for (i = 0; i < NUM_IPS_ENTRIES; i++)
- memcpy(ip_batch[i], large_ips_table[i].ip, 16);
+ memcpy(&ip_batch[i], &large_ips_table[i].ip, 16);
for (i = 0; i < ITERATIONS; i++) {
@@ -140,7 +140,7 @@ test_fib6_perf(void)
for (i = 0; i < NUM_ROUTE_ENTRIES; i++) {
/* rte_fib_delete(fib, ip, depth) */
- status += rte_fib6_delete(fib, large_route_table[i].ip,
+ status += rte_fib6_delete(fib, &large_route_table[i].ip,
large_route_table[i].depth);
}
diff --git a/app/test/test_lpm6.c b/app/test/test_lpm6.c
index 1d8a0afa1155..b0b662593116 100644
--- a/app/test/test_lpm6.c
+++ b/app/test/test_lpm6.c
@@ -93,27 +93,28 @@ rte_lpm6_test tests6[] = {
#define PASS 0
static void
-IPv6(uint8_t *ip, uint8_t b1, uint8_t b2, uint8_t b3, uint8_t b4, uint8_t b5,
- uint8_t b6, uint8_t b7, uint8_t b8, uint8_t b9, uint8_t b10,
- uint8_t b11, uint8_t b12, uint8_t b13, uint8_t b14, uint8_t b15,
- uint8_t b16)
+IPv6(struct rte_ipv6_addr *ip,
+ uint8_t b1, uint8_t b2, uint8_t b3, uint8_t b4,
+ uint8_t b5, uint8_t b6, uint8_t b7, uint8_t b8,
+ uint8_t b9, uint8_t b10, uint8_t b11, uint8_t b12,
+ uint8_t b13, uint8_t b14, uint8_t b15, uint8_t b16)
{
- ip[0] = b1;
- ip[1] = b2;
- ip[2] = b3;
- ip[3] = b4;
- ip[4] = b5;
- ip[5] = b6;
- ip[6] = b7;
- ip[7] = b8;
- ip[8] = b9;
- ip[9] = b10;
- ip[10] = b11;
- ip[11] = b12;
- ip[12] = b13;
- ip[13] = b14;
- ip[14] = b15;
- ip[15] = b16;
+ ip->a[0] = b1;
+ ip->a[1] = b2;
+ ip->a[2] = b3;
+ ip->a[3] = b4;
+ ip->a[4] = b5;
+ ip->a[5] = b6;
+ ip->a[6] = b7;
+ ip->a[7] = b8;
+ ip->a[8] = b9;
+ ip->a[9] = b10;
+ ip->a[10] = b11;
+ ip->a[11] = b12;
+ ip->a[12] = b13;
+ ip->a[13] = b14;
+ ip->a[14] = b15;
+ ip->a[15] = b16;
}
/*
@@ -250,7 +251,7 @@ test4(void)
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
+ struct rte_ipv6_addr ip = {0};
uint8_t depth = 24, next_hop = 100;
int32_t status = 0;
@@ -259,7 +260,7 @@ test4(void)
config.flags = 0;
/* rte_lpm6_add: lpm == NULL */
- status = rte_lpm6_add(NULL, ip, depth, next_hop);
+ status = rte_lpm6_add(NULL, &ip, depth, next_hop);
TEST_LPM_ASSERT(status < 0);
/*Create valid lpm to use in rest of test. */
@@ -267,11 +268,11 @@ test4(void)
TEST_LPM_ASSERT(lpm != NULL);
/* rte_lpm6_add: depth < 1 */
- status = rte_lpm6_add(lpm, ip, 0, next_hop);
+ status = rte_lpm6_add(lpm, &ip, 0, next_hop);
TEST_LPM_ASSERT(status < 0);
/* rte_lpm6_add: depth > MAX_DEPTH */
- status = rte_lpm6_add(lpm, ip, (MAX_DEPTH + 1), next_hop);
+ status = rte_lpm6_add(lpm, &ip, (MAX_DEPTH + 1), next_hop);
TEST_LPM_ASSERT(status < 0);
rte_lpm6_free(lpm);
@@ -288,7 +289,7 @@ test5(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
+ struct rte_ipv6_addr ip = {0};
uint8_t depth = 24;
int32_t status = 0;
@@ -297,7 +298,7 @@ test5(void)
config.flags = 0;
/* rte_lpm_delete: lpm == NULL */
- status = rte_lpm6_delete(NULL, ip, depth);
+ status = rte_lpm6_delete(NULL, &ip, depth);
TEST_LPM_ASSERT(status < 0);
/*Create valid lpm to use in rest of test. */
@@ -305,11 +306,11 @@ test5(void)
TEST_LPM_ASSERT(lpm != NULL);
/* rte_lpm_delete: depth < 1 */
- status = rte_lpm6_delete(lpm, ip, 0);
+ status = rte_lpm6_delete(lpm, &ip, 0);
TEST_LPM_ASSERT(status < 0);
/* rte_lpm_delete: depth > MAX_DEPTH */
- status = rte_lpm6_delete(lpm, ip, (MAX_DEPTH + 1));
+ status = rte_lpm6_delete(lpm, &ip, (MAX_DEPTH + 1));
TEST_LPM_ASSERT(status < 0);
rte_lpm6_free(lpm);
@@ -326,7 +327,7 @@ test6(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
+ struct rte_ipv6_addr ip = {0};
uint32_t next_hop_return = 0;
int32_t status = 0;
@@ -335,7 +336,7 @@ test6(void)
config.flags = 0;
/* rte_lpm6_lookup: lpm == NULL */
- status = rte_lpm6_lookup(NULL, ip, &next_hop_return);
+ status = rte_lpm6_lookup(NULL, &ip, &next_hop_return);
TEST_LPM_ASSERT(status < 0);
/*Create valid lpm to use in rest of test. */
@@ -347,7 +348,7 @@ test6(void)
TEST_LPM_ASSERT(status < 0);
/* rte_lpm6_lookup: next_hop = NULL */
- status = rte_lpm6_lookup(lpm, ip, NULL);
+ status = rte_lpm6_lookup(lpm, &ip, NULL);
TEST_LPM_ASSERT(status < 0);
rte_lpm6_free(lpm);
@@ -364,7 +365,7 @@ test7(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[10][16];
+ struct rte_ipv6_addr ips[10];
int32_t next_hop_return[10];
int32_t status = 0;
@@ -373,7 +374,7 @@ test7(void)
config.flags = 0;
/* rte_lpm6_lookup: lpm == NULL */
- status = rte_lpm6_lookup_bulk_func(NULL, ip, next_hop_return, 10);
+ status = rte_lpm6_lookup_bulk_func(NULL, ips, next_hop_return, RTE_DIM(ips));
TEST_LPM_ASSERT(status < 0);
/*Create valid lpm to use in rest of test. */
@@ -381,11 +382,11 @@ test7(void)
TEST_LPM_ASSERT(lpm != NULL);
/* rte_lpm6_lookup: ip = NULL */
- status = rte_lpm6_lookup_bulk_func(lpm, NULL, next_hop_return, 10);
+ status = rte_lpm6_lookup_bulk_func(lpm, NULL, next_hop_return, RTE_DIM(ips));
TEST_LPM_ASSERT(status < 0);
/* rte_lpm6_lookup: next_hop = NULL */
- status = rte_lpm6_lookup_bulk_func(lpm, ip, NULL, 10);
+ status = rte_lpm6_lookup_bulk_func(lpm, ips, NULL, RTE_DIM(ips));
TEST_LPM_ASSERT(status < 0);
rte_lpm6_free(lpm);
@@ -402,7 +403,7 @@ test8(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[10][16];
+ struct rte_ipv6_addr ips[10];
uint8_t depth[10];
int32_t status = 0;
@@ -411,7 +412,7 @@ test8(void)
config.flags = 0;
/* rte_lpm6_delete: lpm == NULL */
- status = rte_lpm6_delete_bulk_func(NULL, ip, depth, 10);
+ status = rte_lpm6_delete_bulk_func(NULL, ips, depth, RTE_DIM(ips));
TEST_LPM_ASSERT(status < 0);
/*Create valid lpm to use in rest of test. */
@@ -419,11 +420,11 @@ test8(void)
TEST_LPM_ASSERT(lpm != NULL);
/* rte_lpm6_delete: ip = NULL */
- status = rte_lpm6_delete_bulk_func(lpm, NULL, depth, 10);
+ status = rte_lpm6_delete_bulk_func(lpm, NULL, depth, RTE_DIM(ips));
TEST_LPM_ASSERT(status < 0);
/* rte_lpm6_delete: next_hop = NULL */
- status = rte_lpm6_delete_bulk_func(lpm, ip, NULL, 10);
+ status = rte_lpm6_delete_bulk_func(lpm, ips, NULL, RTE_DIM(ips));
TEST_LPM_ASSERT(status < 0);
rte_lpm6_free(lpm);
@@ -441,7 +442,7 @@ test9(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
+ struct rte_ipv6_addr ip = {0};
uint8_t depth = 16;
uint32_t next_hop_add = 100, next_hop_return = 0;
int32_t status = 0;
@@ -454,21 +455,21 @@ test9(void)
lpm = rte_lpm6_create(__func__, SOCKET_ID_ANY, &config);
TEST_LPM_ASSERT(lpm != NULL);
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
for (i = 0; i < UINT8_MAX; i++) {
- ip[2] = i;
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ ip.a[2] = i;
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
}
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
for (i = 0; i < UINT8_MAX; i++) {
- ip[2] = i;
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ ip.a[2] = i;
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
}
@@ -486,7 +487,7 @@ test10(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
+ struct rte_ipv6_addr ip = {0};
uint8_t depth;
uint32_t next_hop_add = 100;
int32_t status = 0;
@@ -501,20 +502,20 @@ test10(void)
for (i = 1; i < 128; i++) {
depth = (uint8_t)i;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
}
depth = 128;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == -ENOSPC);
depth = 127;
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
depth = 128;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
rte_lpm6_free(lpm);
@@ -531,7 +532,7 @@ test11(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
+ struct rte_ipv6_addr ip = {0};
uint8_t depth;
uint32_t next_hop_add = 100;
int32_t status = 0;
@@ -544,37 +545,37 @@ test11(void)
TEST_LPM_ASSERT(lpm != NULL);
depth = 128;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- ip[0] = 1;
+ ip.a[0] = 1;
depth = 25;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
depth = 33;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
depth = 41;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
depth = 49;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == -ENOSPC);
depth = 41;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
rte_lpm6_free(lpm);
@@ -592,7 +593,7 @@ test12(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
+ struct rte_ipv6_addr ip = {0};
uint8_t depth;
uint32_t next_hop_add = 100;
int32_t status = 0;
@@ -605,16 +606,16 @@ test12(void)
TEST_LPM_ASSERT(lpm != NULL);
depth = 128;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- ip[0] = 1;
+ ip.a[0] = 1;
depth = 41;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
depth = 49;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == -ENOSPC);
rte_lpm6_free(lpm);
@@ -631,7 +632,7 @@ test13(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
+ struct rte_ipv6_addr ip = {0};
uint8_t depth;
uint32_t next_hop_add = 100;
int32_t status = 0;
@@ -644,23 +645,23 @@ test13(void)
TEST_LPM_ASSERT(lpm != NULL);
depth = 1;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
depth = 2;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
depth = 3;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == -ENOSPC);
depth = 2;
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
depth = 3;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
rte_lpm6_free(lpm);
@@ -679,7 +680,7 @@ test14(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
+ struct rte_ipv6_addr ip = {0};
uint8_t depth = 25;
uint32_t next_hop_add = 100;
int32_t status = 0;
@@ -693,24 +694,24 @@ test14(void)
TEST_LPM_ASSERT(lpm != NULL);
for (i = 0; i < 256; i++) {
- ip[0] = (uint8_t)i;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ ip.a[0] = (uint8_t)i;
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
}
- ip[0] = 255;
- ip[1] = 1;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ ip.a[0] = 255;
+ ip.a[1] = 1;
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == -ENOSPC);
- ip[0] = 255;
- ip[1] = 0;
- status = rte_lpm6_delete(lpm, ip, depth);
+ ip.a[0] = 255;
+ ip.a[1] = 0;
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
- ip[0] = 255;
- ip[1] = 1;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ ip.a[0] = 255;
+ ip.a[1] = 1;
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
rte_lpm6_free(lpm);
@@ -726,7 +727,7 @@ test15(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
+ struct rte_ipv6_addr ip = {0};
uint8_t depth = 24;
uint32_t next_hop_add = 100, next_hop_return = 0;
int32_t status = 0;
@@ -738,16 +739,16 @@ test15(void)
lpm = rte_lpm6_create(__func__, SOCKET_ID_ANY, &config);
TEST_LPM_ASSERT(lpm != NULL);
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
rte_lpm6_free(lpm);
@@ -763,7 +764,7 @@ test16(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[] = {12,12,1,0,0,0,0,0,0,0,0,0,0,0,0,0};
+ struct rte_ipv6_addr ip = {.a = {12, 12, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}};
uint8_t depth = 128;
uint32_t next_hop_add = 100, next_hop_return = 0;
int32_t status = 0;
@@ -775,16 +776,16 @@ test16(void)
lpm = rte_lpm6_create(__func__, SOCKET_ID_ANY, &config);
TEST_LPM_ASSERT(lpm != NULL);
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
rte_lpm6_free(lpm);
@@ -806,9 +807,13 @@ test17(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip1[] = {127,255,255,255,255,255,255,255,255,
- 255,255,255,255,255,255,255};
- uint8_t ip2[] = {128,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};
+ struct rte_ipv6_addr ip1 = {
+ .a = {
+ 127, 255, 255, 255, 255, 255, 255, 255,
+ 255, 255, 255, 255, 255, 255, 255, 255
+ },
+ };
+ struct rte_ipv6_addr ip2 = {.a = {128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}};
uint8_t depth;
uint32_t next_hop_add, next_hop_return;
int32_t status = 0;
@@ -825,14 +830,14 @@ test17(void)
/* Let the next_hop_add value = depth. Just for change. */
next_hop_add = depth;
- status = rte_lpm6_add(lpm, ip2, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip2, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
/* Check IP in first half of tbl24 which should be empty. */
- status = rte_lpm6_lookup(lpm, ip1, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip1, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
- status = rte_lpm6_lookup(lpm, ip2, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip2, &next_hop_return);
TEST_LPM_ASSERT((status == 0) &&
(next_hop_return == next_hop_add));
}
@@ -841,10 +846,10 @@ test17(void)
for (depth = 16; depth >= 1; depth--) {
next_hop_add = (depth - 1);
- status = rte_lpm6_delete(lpm, ip2, depth);
+ status = rte_lpm6_delete(lpm, &ip2, depth);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip2, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip2, &next_hop_return);
if (depth != 1) {
TEST_LPM_ASSERT((status == 0) &&
@@ -854,7 +859,7 @@ test17(void)
TEST_LPM_ASSERT(status == -ENOENT);
}
- status = rte_lpm6_lookup(lpm, ip1, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip1, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
}
@@ -874,7 +879,7 @@ test18(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[16], ip_1[16], ip_2[16];
+ struct rte_ipv6_addr ip, ip_1, ip_2;
uint8_t depth, depth_1, depth_2;
uint32_t next_hop_add, next_hop_add_1,
next_hop_add_2, next_hop_return;
@@ -885,58 +890,58 @@ test18(void)
config.flags = 0;
/* Add & lookup to hit invalid TBL24 entry */
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 24;
next_hop_add = 100;
lpm = rte_lpm6_create(__func__, SOCKET_ID_ANY, &config);
TEST_LPM_ASSERT(lpm != NULL);
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
rte_lpm6_delete_all(lpm);
/* Add & lookup to hit valid TBL24 entry not extended */
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 23;
next_hop_add = 100;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
depth = 24;
next_hop_add = 101;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
depth = 24;
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
depth = 23;
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
rte_lpm6_delete_all(lpm);
@@ -944,37 +949,37 @@ test18(void)
/* Add & lookup to hit valid extended TBL24 entry with invalid TBL8
* entry.
*/
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 32;
next_hop_add = 100;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
- IPv6(ip, 128, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 32;
next_hop_add = 101;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 32;
next_hop_add = 100;
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
rte_lpm6_delete_all(lpm);
@@ -982,38 +987,38 @@ test18(void)
/* Add & lookup to hit valid extended TBL24 entry with valid TBL8
* entry
*/
- IPv6(ip_1, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip_1, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth_1 = 25;
next_hop_add_1 = 101;
- IPv6(ip_2, 128, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip_2, 128, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth_2 = 32;
next_hop_add_2 = 102;
next_hop_return = 0;
- status = rte_lpm6_add(lpm, ip_1, depth_1, next_hop_add_1);
+ status = rte_lpm6_add(lpm, &ip_1, depth_1, next_hop_add_1);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip_1, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip_1, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add_1));
- status = rte_lpm6_add(lpm, ip_2, depth_2, next_hop_add_2);
+ status = rte_lpm6_add(lpm, &ip_2, depth_2, next_hop_add_2);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip_2, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip_2, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add_2));
- status = rte_lpm6_delete(lpm, ip_2, depth_2);
+ status = rte_lpm6_delete(lpm, &ip_2, depth_2);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip_2, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip_2, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add_1));
- status = rte_lpm6_delete(lpm, ip_1, depth_1);
+ status = rte_lpm6_delete(lpm, &ip_1, depth_1);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip_1, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip_1, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
rte_lpm6_free(lpm);
@@ -1037,7 +1042,7 @@ test19(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[16];
+ struct rte_ipv6_addr ip;
uint8_t depth;
uint32_t next_hop_add, next_hop_return;
int32_t status = 0;
@@ -1052,35 +1057,35 @@ test19(void)
lpm = rte_lpm6_create(__func__, SOCKET_ID_ANY, &config);
TEST_LPM_ASSERT(lpm != NULL);
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 16;
next_hop_add = 100;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
rte_lpm6_delete_all(lpm);
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 25;
next_hop_add = 100;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
rte_lpm6_delete_all(lpm);
@@ -1090,45 +1095,45 @@ test19(void)
* (& delete & lookup)
*/
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 24;
next_hop_add = 100;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- IPv6(ip, 128, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 32;
next_hop_add = 101;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
next_hop_add = 100;
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 24;
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
- IPv6(ip, 128, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 32;
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
rte_lpm6_delete_all(lpm);
@@ -1138,28 +1143,28 @@ test19(void)
* (& delete & lookup)
*/
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 24;
next_hop_add = 100;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
next_hop_add = 101;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
rte_lpm6_delete_all(lpm);
@@ -1169,56 +1174,56 @@ test19(void)
* (& delete & lookup)
*/
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 32;
next_hop_add = 100;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
next_hop_add = 101;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
rte_lpm6_delete_all(lpm);
/* Delete a rule that is not present in the TBL24 & lookup */
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 24;
next_hop_add = 100;
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status < 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
rte_lpm6_delete_all(lpm);
/* Delete a rule that is not present in the TBL8 & lookup */
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 32;
next_hop_add = 100;
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status < 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
rte_lpm6_free(lpm);
@@ -1236,7 +1241,7 @@ test20(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[16];
+ struct rte_ipv6_addr ip;
uint8_t depth;
uint32_t next_hop_add, next_hop_return;
int32_t status = 0;
@@ -1248,45 +1253,45 @@ test20(void)
lpm = rte_lpm6_create(__func__, SOCKET_ID_ANY, &config);
TEST_LPM_ASSERT(lpm != NULL);
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 24;
next_hop_add = 100;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10);
depth = 128;
next_hop_add = 101;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
next_hop_add = 100;
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 24;
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10);
depth = 128;
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
rte_lpm6_free(lpm);
@@ -1304,7 +1309,7 @@ test21(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip_batch[4][16];
+ struct rte_ipv6_addr ip_batch[4];
uint8_t depth;
uint32_t next_hop_add;
int32_t next_hop_return[4];
@@ -1317,28 +1322,28 @@ test21(void)
lpm = rte_lpm6_create(__func__, SOCKET_ID_ANY, &config);
TEST_LPM_ASSERT(lpm != NULL);
- IPv6(ip_batch[0], 128, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip_batch[0], 128, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 48;
next_hop_add = 100;
- status = rte_lpm6_add(lpm, ip_batch[0], depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip_batch[0], depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- IPv6(ip_batch[1], 128, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip_batch[1], 128, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 48;
next_hop_add = 101;
- status = rte_lpm6_add(lpm, ip_batch[1], depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip_batch[1], depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- IPv6(ip_batch[2], 128, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip_batch[2], 128, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 48;
next_hop_add = 102;
- status = rte_lpm6_add(lpm, ip_batch[2], depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip_batch[2], depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- IPv6(ip_batch[3], 128, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip_batch[3], 128, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
status = rte_lpm6_lookup_bulk_func(lpm, ip_batch,
next_hop_return, 4);
@@ -1363,7 +1368,7 @@ test22(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip_batch[5][16];
+ struct rte_ipv6_addr ip_batch[5];
uint8_t depth[5];
uint32_t next_hop_add;
int32_t next_hop_return[5];
@@ -1378,39 +1383,39 @@ test22(void)
/* Adds 5 rules and look them up */
- IPv6(ip_batch[0], 128, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip_batch[0], 128, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth[0] = 48;
next_hop_add = 101;
- status = rte_lpm6_add(lpm, ip_batch[0], depth[0], next_hop_add);
+ status = rte_lpm6_add(lpm, &ip_batch[0], depth[0], next_hop_add);
TEST_LPM_ASSERT(status == 0);
- IPv6(ip_batch[1], 128, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip_batch[1], 128, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth[1] = 48;
next_hop_add = 102;
- status = rte_lpm6_add(lpm, ip_batch[1], depth[1], next_hop_add);
+ status = rte_lpm6_add(lpm, &ip_batch[1], depth[1], next_hop_add);
TEST_LPM_ASSERT(status == 0);
- IPv6(ip_batch[2], 128, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip_batch[2], 128, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth[2] = 48;
next_hop_add = 103;
- status = rte_lpm6_add(lpm, ip_batch[2], depth[2], next_hop_add);
+ status = rte_lpm6_add(lpm, &ip_batch[2], depth[2], next_hop_add);
TEST_LPM_ASSERT(status == 0);
- IPv6(ip_batch[3], 128, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip_batch[3], 128, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth[3] = 48;
next_hop_add = 104;
- status = rte_lpm6_add(lpm, ip_batch[3], depth[3], next_hop_add);
+ status = rte_lpm6_add(lpm, &ip_batch[3], depth[3], next_hop_add);
TEST_LPM_ASSERT(status == 0);
- IPv6(ip_batch[4], 128, 0, 0, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip_batch[4], 128, 0, 0, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth[4] = 48;
next_hop_add = 105;
- status = rte_lpm6_add(lpm, ip_batch[4], depth[4], next_hop_add);
+ status = rte_lpm6_add(lpm, &ip_batch[4], depth[4], next_hop_add);
TEST_LPM_ASSERT(status == 0);
status = rte_lpm6_lookup_bulk_func(lpm, ip_batch,
@@ -1443,11 +1448,11 @@ test22(void)
/* Use the delete_bulk function to delete two, one invalid. Lookup again */
- IPv6(ip_batch[4], 128, 0, 0, 0, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip_batch[4], 128, 0, 0, 0, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
status = rte_lpm6_delete_bulk_func(lpm, &ip_batch[3], depth, 2);
TEST_LPM_ASSERT(status == 0);
- IPv6(ip_batch[4], 128, 0, 0, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip_batch[4], 128, 0, 0, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
status = rte_lpm6_lookup_bulk_func(lpm, ip_batch,
next_hop_return, 5);
TEST_LPM_ASSERT(status == 0 && next_hop_return[0] == -1
@@ -1481,7 +1486,7 @@ test23(void)
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
uint32_t i;
- uint8_t ip[16];
+ struct rte_ipv6_addr ip;
uint8_t depth;
uint32_t next_hop_add, next_hop_return;
int32_t status = 0;
@@ -1493,22 +1498,22 @@ test23(void)
lpm = rte_lpm6_create(__func__, SOCKET_ID_ANY, &config);
TEST_LPM_ASSERT(lpm != NULL);
- IPv6(ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
+ IPv6(&ip, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);
depth = 128;
next_hop_add = 100;
for (i = 0; i < 30; i++) {
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) &&
(next_hop_return == next_hop_add));
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT(status == -ENOENT);
}
@@ -1565,7 +1570,7 @@ test25(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[16];
+ struct rte_ipv6_addr ip;
uint32_t i;
uint8_t depth;
uint32_t next_hop_add, next_hop_return, next_hop_expected;
@@ -1579,10 +1584,10 @@ test25(void)
TEST_LPM_ASSERT(lpm != NULL);
for (i = 0; i < 1000; i++) {
- memcpy(ip, large_route_table[i].ip, 16);
+ memcpy(&ip, &large_route_table[i].ip, 16);
depth = large_route_table[i].depth;
next_hop_add = large_route_table[i].next_hop;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
}
@@ -1590,10 +1595,10 @@ test25(void)
generate_large_ips_table(1);
for (i = 0; i < 100000; i++) {
- memcpy(ip, large_ips_table[i].ip, 16);
+ memcpy(&ip, &large_ips_table[i].ip, 16);
next_hop_expected = large_ips_table[i].next_hop;
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) &&
(next_hop_return == next_hop_expected));
}
@@ -1615,9 +1620,15 @@ test26(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip_10_32[] = {10, 10, 10, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
- uint8_t ip_10_24[] = {10, 10, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
- uint8_t ip_20_25[] = {10, 10, 20, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
+ struct rte_ipv6_addr ip_10_32 = {
+ .a = {10, 10, 10, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
+ };
+ struct rte_ipv6_addr ip_10_24 = {
+ .a = {10, 10, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
+ };
+ struct rte_ipv6_addr ip_20_25 = {
+ .a = {10, 10, 20, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
+ };
uint8_t d_ip_10_32 = 32;
uint8_t d_ip_10_24 = 24;
uint8_t d_ip_20_25 = 25;
@@ -1634,29 +1645,26 @@ test26(void)
lpm = rte_lpm6_create(__func__, SOCKET_ID_ANY, &config);
TEST_LPM_ASSERT(lpm != NULL);
- if ((status = rte_lpm6_add(lpm, ip_10_32, d_ip_10_32,
- next_hop_ip_10_32)) < 0)
- return -1;
+ status = rte_lpm6_add(lpm, &ip_10_32, d_ip_10_32, next_hop_ip_10_32);
+ TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip_10_32, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip_10_32, &next_hop_return);
uint32_t test_hop_10_32 = next_hop_return;
TEST_LPM_ASSERT(status == 0);
TEST_LPM_ASSERT(next_hop_return == next_hop_ip_10_32);
- if ((status = rte_lpm6_add(lpm, ip_10_24, d_ip_10_24,
- next_hop_ip_10_24)) < 0)
- return -1;
+ status = rte_lpm6_add(lpm, &ip_10_24, d_ip_10_24, next_hop_ip_10_24);
+ TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip_10_24, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip_10_24, &next_hop_return);
uint32_t test_hop_10_24 = next_hop_return;
TEST_LPM_ASSERT(status == 0);
TEST_LPM_ASSERT(next_hop_return == next_hop_ip_10_24);
- if ((status = rte_lpm6_add(lpm, ip_20_25, d_ip_20_25,
- next_hop_ip_20_25)) < 0)
- return -1;
+ status = rte_lpm6_add(lpm, &ip_20_25, d_ip_20_25, next_hop_ip_20_25);
+ TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip_20_25, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip_20_25, &next_hop_return);
uint32_t test_hop_20_25 = next_hop_return;
TEST_LPM_ASSERT(status == 0);
TEST_LPM_ASSERT(next_hop_return == next_hop_ip_20_25);
@@ -1671,11 +1679,11 @@ test26(void)
return -1;
}
- status = rte_lpm6_lookup(lpm, ip_10_32, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip_10_32, &next_hop_return);
TEST_LPM_ASSERT(status == 0);
TEST_LPM_ASSERT(next_hop_return == next_hop_ip_10_32);
- status = rte_lpm6_lookup(lpm, ip_10_24, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip_10_24, &next_hop_return);
TEST_LPM_ASSERT(status == 0);
TEST_LPM_ASSERT(next_hop_return == next_hop_ip_10_24);
@@ -1695,7 +1703,12 @@ test27(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[] = {128,128,128,128,128,128,128,128,128,128,128,128,128,128,0,0};
+ struct rte_ipv6_addr ip = {
+ .a = {
+ 128, 128, 128, 128, 128, 128, 128, 128,
+ 128, 128, 128, 128, 128, 128, 0, 0
+ },
+ };
uint8_t depth = 128;
uint32_t next_hop_add = 100, next_hop_return;
int32_t status = 0;
@@ -1710,19 +1723,19 @@ test27(void)
depth = 128;
next_hop_add = 128;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
depth = 112;
next_hop_add = 112;
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
for (i = 0; i < 256; i++) {
- ip[14] = (uint8_t)i;
+ ip.a[14] = i;
for (j = 0; j < 256; j++) {
- ip[15] = (uint8_t)j;
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ ip.a[15] = j;
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
if (i == 0 && j == 0)
TEST_LPM_ASSERT(status == 0 && next_hop_return == 128);
else
@@ -1746,7 +1759,9 @@ test28(void)
{
struct rte_lpm6 *lpm = NULL;
struct rte_lpm6_config config;
- uint8_t ip[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };
+ struct rte_ipv6_addr ip = {
+ .a = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
+ };
uint8_t depth = 16;
uint32_t next_hop_add = 0x001FFFFF, next_hop_return = 0;
int32_t status = 0;
@@ -1758,13 +1773,13 @@ test28(void)
lpm = rte_lpm6_create(__func__, SOCKET_ID_ANY, &config);
TEST_LPM_ASSERT(lpm != NULL);
- status = rte_lpm6_add(lpm, ip, depth, next_hop_add);
+ status = rte_lpm6_add(lpm, &ip, depth, next_hop_add);
TEST_LPM_ASSERT(status == 0);
- status = rte_lpm6_lookup(lpm, ip, &next_hop_return);
+ status = rte_lpm6_lookup(lpm, &ip, &next_hop_return);
TEST_LPM_ASSERT((status == 0) && (next_hop_return == next_hop_add));
- status = rte_lpm6_delete(lpm, ip, depth);
+ status = rte_lpm6_delete(lpm, &ip, depth);
TEST_LPM_ASSERT(status == 0);
rte_lpm6_free(lpm);
diff --git a/app/test/test_lpm6_data.h b/app/test/test_lpm6_data.h
index 8ddb59563ee4..d9cb4cebb35d 100644
--- a/app/test/test_lpm6_data.h
+++ b/app/test/test_lpm6_data.h
@@ -7,16 +7,17 @@
#include <stdint.h>
#include <stdlib.h>
+#include <rte_ip6.h>
#include <rte_random.h>
struct rules_tbl_entry {
- uint8_t ip[16];
+ struct rte_ipv6_addr ip;
uint8_t depth;
uint8_t next_hop;
};
struct ips_tbl_entry {
- uint8_t ip[16];
+ struct rte_ipv6_addr ip;
uint8_t next_hop;
};
@@ -29,1006 +30,1006 @@ struct ips_tbl_entry {
*/
static struct rules_tbl_entry large_route_table[] = {
- {{66, 70, 154, 143, 197, 233, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 50, 146},
- {{107, 79, 18, 235, 142, 84, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 54, 141},
- {{247, 132, 113, 1, 215, 247, 183, 239, 128, 0, 0, 0, 0, 0, 0, 0}, 67, 23},
- {{48, 19, 41, 12, 76, 101, 114, 160, 45, 103, 134, 146, 128, 0, 0, 0}, 97, 252},
- {{5, 70, 208, 170, 19, 0, 116, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 54, 6},
- {{1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 9, 137},
- {{12, 188, 26, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 31, 9},
- {{1, 235, 101, 202, 26, 92, 23, 22, 179, 223, 128, 0, 0, 0, 0, 0}, 82, 9},
- {{215, 19, 224, 102, 45, 133, 102, 249, 56, 20, 214, 219, 93, 125, 52, 0}, 120, 163},
- {{178, 183, 109, 64, 136, 84, 11, 53, 217, 102, 0, 0, 0, 0, 0, 0}, 79, 197},
- {{212, 39, 158, 71, 253, 98, 248, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 54, 249},
- {{92, 58, 159, 130, 105, 56, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 47, 88},
- {{118, 140, 65, 198, 212, 93, 144, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 52, 104},
- {{86, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 10, 36},
- {{79, 135, 242, 193, 197, 11, 200, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 54, 239},
- {{163, 228, 239, 80, 41, 66, 176, 176, 0, 0, 0, 0, 0, 0, 0, 0}, 67, 201},
- {{31, 9, 231, 42, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 33, 94},
- {{108, 144, 205, 39, 215, 26, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 51, 241},
- {{247, 217, 172, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 24, 239},
- {{24, 186, 73, 182, 240, 251, 125, 165, 0, 0, 0, 0, 0, 0, 0, 0}, 66, 151},
- {{245, 112, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 12, 137},
- {{44, 94, 138, 224, 168, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 41, 231},
- {{184, 221, 109, 135, 225, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 44, 11},
- {{51, 179, 136, 184, 30, 118, 24, 16, 26, 161, 206, 101, 0, 0, 0, 0}, 96, 20},
- {{48, 46, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 15, 68},
- {{143, 235, 237, 220, 89, 119, 187, 143, 209, 94, 46, 58, 120, 0, 0, 0}, 101, 64},
- {{121, 190, 90, 177, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 33, 152},
- {{128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 6, 217},
- {{128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 2, 101},
- {{111, 214, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 15, 58},
- {{162, 23, 52, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 27, 254},
- {{76, 103, 44, 79, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 32, 148},
- {{80, 85, 219, 214, 12, 4, 65, 129, 162, 148, 208, 78, 39, 69, 94, 184}, 126, 126},
- {{80, 54, 251, 28, 152, 23, 244, 192, 151, 83, 6, 144, 223, 213, 224, 128}, 123, 76},
- {{39, 232, 237, 103, 191, 188, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 54, 240},
- {{20, 231, 89, 210, 167, 173, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 54, 33},
- {{125, 67, 198, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 25, 47},
- {{26, 239, 153, 5, 213, 121, 31, 114, 161, 46, 84, 15, 148, 160, 0, 0}, 109, 41},
- {{102, 212, 159, 118, 223, 115, 134, 172, 0, 0, 0, 0, 0, 0, 0, 0}, 62, 72},
- {{85, 181, 241, 127, 3, 44, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 46, 43},
- {{61, 199, 131, 226, 3, 230, 94, 119, 240, 0, 0, 0, 0, 0, 0, 0}, 68, 26},
- {{0, 143, 160, 184, 162, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 42, 139},
- {{170, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 13, 219},
- {{61, 122, 24, 251, 124, 122, 202, 192, 0, 0, 0, 0, 0, 0, 0, 0}, 58, 105},
- {{33, 219, 226, 3, 180, 190, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 47, 210},
- {{51, 251, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 17, 151},
- {{106, 185, 11, 122, 197, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 42, 28},
- {{192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 9, 64},
- {{239, 195, 77, 239, 131, 156, 2, 246, 191, 178, 204, 160, 21, 213, 30, 128}, 121, 9},
- {{141, 207, 181, 99, 55, 245, 151, 228, 65, 50, 85, 16, 0, 0, 0, 0}, 92, 250},
- {{110, 159, 230, 251, 224, 210, 58, 49, 0, 0, 0, 0, 0, 0, 0, 0}, 66, 200},
- {{134, 26, 104, 32, 129, 41, 201, 50, 164, 69, 178, 156, 156, 133, 8, 218}, 127, 132},
- {{253, 207, 116, 105, 210, 166, 186, 99, 182, 0, 0, 0, 0, 0, 0, 0}, 71, 182},
- {{211, 73, 38, 80, 183, 168, 52, 138, 25, 214, 112, 8, 252, 0, 0, 0}, 102, 7},
- {{200, 244, 108, 238, 164, 141, 215, 39, 233, 249, 120, 80, 112, 0, 0, 0}, 100, 146},
- {{107, 44, 250, 202, 64, 37, 107, 105, 140, 0, 0, 0, 0, 0, 0, 0}, 70, 98},
- {{93, 86, 56, 27, 159, 195, 126, 39, 240, 201, 48, 0, 0, 0, 0, 0}, 86, 179},
- {{32, 202, 214, 242, 39, 141, 61, 146, 138, 96, 0, 0, 0, 0, 0, 0}, 77, 245},
- {{167, 77, 249, 28, 210, 196, 227, 241, 0, 0, 0, 0, 0, 0, 0, 0}, 64, 2},
- {{241, 59, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 17, 5},
- {{143, 68, 146, 210, 173, 155, 251, 173, 0, 0, 0, 0, 0, 0, 0, 0}, 66, 169},
- {{167, 180, 226, 144, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 33, 52},
- {{241, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 9, 177},
- {{238, 9, 168, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 27, 74},
- {{203, 148, 16, 96, 125, 18, 86, 1, 91, 244, 251, 20, 31, 14, 75, 128}, 122, 212},
- {{111, 227, 137, 94, 65, 21, 77, 137, 119, 130, 159, 19, 159, 45, 18, 192}, 122, 238},
- {{59, 144, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 19, 18},
- {{110, 192, 255, 120, 84, 215, 3, 130, 38, 224, 0, 0, 0, 0, 0, 0}, 75, 155},
- {{152, 79, 219, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 24, 97},
- {{118, 186, 157, 248, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 32, 8},
- {{70, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 9, 123},
- {{253, 119, 114, 227, 18, 243, 81, 61, 238, 107, 190, 144, 0, 0, 0, 0}, 92, 11},
- {{166, 112, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 13, 211},
- {{43, 95, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 20, 116},
- {{94, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 11, 57},
- {{182, 251, 195, 132, 66, 7, 208, 146, 223, 231, 211, 181, 25, 176, 0, 0}, 108, 178},
- {{152, 166, 111, 233, 194, 17, 230, 41, 221, 253, 69, 123, 108, 0, 0, 0}, 102, 93},
- {{106, 141, 235, 190, 82, 241, 152, 186, 195, 81, 86, 144, 0, 0, 0, 0}, 92, 3},
- {{32, 81, 210, 153, 151, 29, 11, 62, 127, 177, 194, 254, 103, 83, 58, 128}, 121, 162},
- {{79, 112, 224, 26, 174, 39, 98, 181, 115, 57, 209, 189, 136, 48, 0, 0}, 109, 125},
- {{106, 197, 83, 151, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 34, 33},
- {{190, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 9, 254},
- {{156, 73, 249, 148, 55, 192, 20, 42, 142, 128, 0, 0, 0, 0, 0, 0}, 74, 66},
- {{64, 107, 36, 120, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 30, 4},
- {{115, 148, 71, 250, 158, 174, 168, 249, 106, 110, 196, 0, 0, 0, 0, 0}, 86, 122},
- {{18, 139, 152, 44, 38, 88, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 46, 59},
- {{55, 229, 117, 106, 146, 95, 74, 220, 122, 0, 84, 202, 183, 138, 120, 0}, 117, 99},
- {{153, 211, 3, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 27, 41},
- {{1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 8, 112},
- {{49, 192, 102, 142, 216, 3, 114, 64, 165, 128, 168, 0, 0, 0, 0, 0}, 85, 255},
- {{201, 143, 240, 240, 209, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 44, 106},
- {{158, 19, 164, 196, 87, 162, 33, 120, 0, 0, 0, 0, 0, 0, 0, 0}, 62, 170},
- {{5, 114, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 16, 86},
- {{34, 170, 246, 62, 198, 85, 193, 227, 252, 68, 0, 0, 0, 0, 0, 0}, 79, 155},
- {{21, 52, 9, 86, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 35, 65},
- {{203, 81, 49, 171, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 36, 39},
- {{211, 218, 87, 244, 93, 181, 118, 41, 156, 143, 254, 0, 0, 0, 0, 0}, 90, 162},
- {{77, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 10, 69},
- {{158, 219, 219, 39, 4, 219, 100, 63, 0, 0, 0, 0, 0, 0, 0, 0}, 64, 163},
- {{61, 50, 232, 1, 185, 252, 243, 54, 189, 240, 170, 192, 0, 0, 0, 0}, 90, 116},
- {{241, 143, 33, 19, 247, 55, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 53, 19},
- {{61, 28, 61, 252, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 36, 48},
- {{102, 112, 194, 108, 90, 253, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 49, 230},
- {{74, 88, 58, 66, 172, 41, 144, 204, 195, 240, 0, 0, 0, 0, 0, 0}, 78, 155},
- {{44, 148, 187, 58, 190, 59, 190, 187, 124, 138, 222, 131, 0, 0, 0, 0}, 96, 158},
- {{67, 7, 216, 139, 93, 224, 20, 135, 186, 86, 209, 111, 60, 80, 0, 0}, 113, 252},
- {{209, 26, 12, 174, 5, 101, 164, 181, 237, 63, 192, 57, 54, 120, 0, 0}, 110, 176},
- {{4, 66, 232, 52, 239, 56, 48, 58, 192, 0, 0, 0, 0, 0, 0, 0}, 66, 211},
- {{158, 165, 2, 144, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 28, 15},
- {{85, 204, 245, 198, 68, 44, 39, 71, 32, 0, 0, 0, 0, 0, 0, 0}, 68, 95},
- {{181, 134, 25, 87, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 34, 169},
- {{26, 230, 61, 36, 79, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 44, 249},
- {{5, 170, 198, 139, 65, 186, 188, 45, 42, 253, 165, 89, 206, 0, 0, 0}, 105, 61},
- {{211, 245, 62, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 23, 63},
- {{117, 68, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 14, 43},
- {{103, 17, 123, 102, 70, 206, 90, 92, 124, 198, 0, 0, 0, 0, 0, 0}, 81, 228},
- {{192, 237, 88, 244, 53, 30, 61, 160, 143, 64, 0, 0, 0, 0, 0, 0}, 78, 165},
- {{199, 82, 217, 183, 2, 179, 195, 6, 0, 0, 0, 0, 0, 0, 0, 0}, 64, 3},
- {{157, 230, 79, 162, 57, 125, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 57, 211},
- {{27, 67, 64, 235, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 33, 210},
- {{72, 158, 163, 106, 193, 137, 190, 7, 250, 165, 249, 73, 64, 0, 0, 0}, 99, 61},
- {{34, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 10, 120},
- {{215, 141, 95, 192, 189, 62, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 47, 94},
- {{31, 181, 56, 141, 120, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 41, 153},
- {{153, 73, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 18, 221},
- {{162, 107, 41, 189, 165, 155, 22, 139, 165, 72, 96, 0, 0, 0, 0, 0}, 87, 163},
- {{218, 17, 204, 165, 217, 251, 107, 45, 29, 15, 192, 167, 75, 0, 0, 0}, 106, 188},
- {{200, 124, 238, 213, 35, 228, 94, 141, 86, 187, 101, 60, 115, 52, 131, 16}, 124, 15},
- {{74, 237, 160, 56, 141, 217, 191, 16, 0, 0, 0, 0, 0, 0, 0, 0}, 63, 28},
- {{163, 47, 242, 103, 173, 217, 88, 154, 38, 200, 32, 0, 0, 0, 0, 0}, 84, 240},
- {{20, 227, 128, 28, 144, 147, 22, 13, 94, 129, 107, 88, 0, 0, 0, 0}, 93, 59},
- {{95, 144, 229, 107, 218, 125, 204, 233, 161, 42, 180, 64, 0, 0, 0, 0}, 90, 195},
- {{155, 220, 83, 208, 108, 16, 134, 156, 128, 0, 0, 0, 0, 0, 0, 0}, 66, 10},
- {{179, 138, 55, 80, 190, 153, 12, 237, 22, 120, 69, 0, 0, 0, 0, 0}, 88, 206},
- {{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 2, 137},
- {{3, 119, 148, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 22, 225},
- {{13, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 10, 223},
- {{117, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 15, 29},
- {{164, 19, 195, 47, 136, 190, 156, 255, 30, 74, 143, 134, 162, 0, 0, 0}, 103, 166},
- {{40, 235, 94, 135, 135, 230, 71, 33, 64, 233, 0, 0, 0, 0, 0, 0}, 80, 178},
- {{222, 151, 166, 97, 129, 250, 140, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 55, 38},
- {{174, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 9, 141},
- {{6, 189, 100, 150, 250, 13, 46, 98, 228, 139, 50, 52, 52, 196, 128, 0}, 116, 230},
- {{75, 252, 89, 205, 37, 52, 106, 79, 188, 120, 54, 119, 160, 0, 0, 0}, 99, 124},
- {{38, 18, 146, 6, 63, 64, 231, 10, 152, 199, 5, 143, 147, 4, 252, 0}, 118, 54},
- {{111, 119, 169, 51, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 32, 162},
- {{105, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 13, 32},
- {{143, 57, 57, 101, 98, 182, 74, 227, 205, 143, 253, 237, 8, 0, 0, 0}, 102, 237},
- {{30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 7, 215},
- {{14, 232, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 22, 138},
- {{14, 53, 67, 216, 229, 155, 149, 139, 31, 253, 184, 126, 133, 108, 40, 0}, 118, 73},
- {{22, 58, 40, 143, 188, 132, 239, 14, 181, 252, 81, 192, 0, 0, 0, 0}, 90, 43},
- {{11, 222, 185, 243, 248, 150, 79, 230, 214, 213, 3, 23, 193, 196, 0, 0}, 112, 88},
- {{14, 226, 198, 117, 84, 93, 22, 96, 77, 241, 173, 68, 68, 204, 72, 0}, 119, 91},
- {{15, 103, 247, 219, 150, 142, 92, 50, 144, 0, 0, 0, 0, 0, 0, 0}, 69, 140},
- {{0, 213, 77, 244, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 37, 65},
- {{178, 174, 174, 239, 72, 181, 36, 217, 40, 169, 12, 104, 149, 157, 125, 128}, 122, 201},
- {{118, 53, 55, 17, 97, 227, 243, 176, 2, 0, 0, 0, 0, 0, 0, 0}, 72, 69},
- {{21, 253, 4, 47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 35, 170},
- {{5, 249, 186, 133, 68, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 40, 192},
- {{47, 79, 35, 66, 11, 178, 161, 28, 87, 180, 45, 128, 0, 0, 0, 0}, 89, 21},
- {{242, 227, 20, 73, 150, 196, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 46, 35},
- {{121, 169, 102, 118, 157, 192, 154, 186, 126, 0, 0, 0, 0, 0, 0, 0}, 71, 235},
- {{9, 138, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 21, 240},
- {{45, 173, 14, 72, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 30, 136},
- {{127, 47, 51, 201, 236, 45, 142, 80, 0, 0, 0, 0, 0, 0, 0, 0}, 60, 186},
- {{247, 233, 34, 38, 181, 207, 127, 20, 224, 118, 59, 148, 0, 0, 0, 0}, 95, 174},
- {{126, 187, 198, 104, 245, 223, 219, 18, 31, 124, 0, 0, 0, 0, 0, 0}, 79, 153},
- {{3, 163, 107, 228, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 35, 118},
- {{167, 109, 2, 95, 11, 62, 45, 128, 0, 0, 0, 0, 0, 0, 0, 0}, 60, 113},
- {{76, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 6, 58},
- {{58, 190, 204, 151, 222, 147, 47, 78, 38, 203, 9, 17, 64, 0, 0, 0}, 101, 206},
- {{254, 220, 254, 220, 204, 79, 35, 127, 242, 63, 106, 232, 127, 180, 0, 0}, 111, 42},
- {{77, 156, 8, 209, 181, 37, 70, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 55, 230},
- {{65, 89, 137, 76, 208, 199, 166, 90, 128, 0, 0, 0, 0, 0, 0, 0}, 67, 6},
- {{47, 232, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 13, 254},
- {{172, 154, 12, 108, 77, 37, 106, 8, 234, 7, 248, 212, 112, 160, 0, 0}, 108, 214},
- {{254, 117, 239, 244, 154, 89, 166, 241, 12, 108, 127, 153, 206, 160, 0, 0}, 107, 43},
- {{113, 160, 206, 52, 143, 12, 9, 148, 224, 0, 0, 0, 0, 0, 0, 0}, 67, 178},
- {{178, 130, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 16, 179},
- {{229, 177, 28, 106, 59, 75, 182, 241, 36, 79, 224, 0, 0, 0, 0, 0}, 87, 236},
- {{156, 72, 93, 193, 50, 235, 75, 228, 88, 115, 89, 119, 128, 0, 0, 0}, 98, 184},
- {{28, 232, 28, 249, 83, 105, 211, 7, 136, 147, 231, 64, 0, 0, 0, 0}, 91, 95},
- {{217, 33, 23, 107, 74, 42, 135, 197, 144, 34, 40, 243, 13, 126, 36, 136}, 127, 152},
- {{64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 2, 113},
- {{85, 172, 121, 126, 213, 57, 225, 54, 197, 73, 85, 251, 9, 64, 0, 0}, 108, 137},
- {{104, 46, 25, 71, 86, 220, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 46, 224},
- {{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 6, 61},
- {{241, 113, 254, 106, 53, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 41, 205},
- {{29, 36, 12, 244, 197, 127, 240, 8, 167, 134, 154, 248, 199, 123, 143, 240}, 124, 170},
- {{58, 29, 129, 94, 43, 139, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 53, 117},
- {{213, 124, 147, 196, 7, 82, 67, 70, 228, 0, 0, 0, 0, 0, 0, 0}, 70, 225},
- {{164, 168, 161, 140, 87, 85, 250, 41, 34, 0, 0, 0, 0, 0, 0, 0}, 72, 34},
- {{186, 142, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 17, 5},
- {{237, 249, 9, 70, 247, 97, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 50, 92},
- {{155, 92, 145, 218, 125, 226, 226, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 55, 230},
- {{35, 169, 62, 156, 86, 4, 125, 219, 119, 113, 191, 75, 198, 113, 0, 0}, 112, 61},
- {{207, 63, 96, 186, 26, 68, 115, 161, 163, 59, 190, 166, 18, 78, 232, 0}, 117, 221},
- {{86, 40, 200, 199, 247, 86, 159, 179, 191, 184, 117, 173, 211, 158, 0, 128}, 121, 105},
- {{104, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 11, 181},
- {{205, 35, 123, 178, 36, 64, 62, 153, 195, 250, 0, 0, 0, 0, 0, 0}, 79, 110},
- {{117, 40, 57, 157, 138, 160, 223, 59, 155, 145, 64, 0, 0, 0, 0, 0}, 86, 103},
- {{74, 166, 140, 146, 74, 72, 229, 99, 167, 124, 107, 117, 217, 14, 246, 64}, 123, 218},
- {{12, 222, 244, 183, 83, 146, 42, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 56, 146},
- {{11, 98, 146, 110, 95, 96, 80, 142, 224, 0, 0, 0, 0, 0, 0, 0}, 67, 90},
- {{235, 5, 187, 199, 30, 170, 82, 187, 228, 159, 22, 25, 204, 112, 0, 0}, 108, 197},
- {{35, 96, 146, 145, 155, 116, 252, 181, 29, 205, 230, 246, 30, 0, 0, 0}, 103, 158},
- {{174, 38, 56, 244, 227, 102, 252, 237, 128, 86, 0, 0, 0, 0, 0, 0}, 81, 118},
- {{65, 134, 37, 58, 90, 125, 60, 84, 0, 0, 0, 0, 0, 0, 0, 0}, 62, 95},
- {{253, 117, 135, 98, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 33, 152},
- {{111, 115, 188, 184, 130, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 45, 239},
- {{202, 24, 89, 9, 149, 45, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 50, 48},
- {{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 5, 228},
- {{244, 98, 52, 140, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 30, 247},
- {{151, 167, 43, 178, 116, 194, 173, 126, 236, 98, 40, 0, 0, 0, 0, 0}, 85, 12},
- {{60, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 10, 129},
- {{208, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 4, 50},
- {{126, 11, 216, 242, 7, 45, 121, 208, 110, 135, 210, 75, 59, 182, 228, 42}, 128, 250},
- {{217, 26, 184, 146, 3, 18, 240, 15, 135, 8, 0, 0, 0, 0, 0, 0}, 77, 249},
- {{128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 1, 230},
- {{145, 28, 29, 184, 2, 85, 234, 135, 98, 111, 136, 32, 0, 0, 0, 0}, 92, 228},
- {{108, 104, 255, 254, 34, 95, 72, 157, 0, 0, 0, 0, 0, 0, 0, 0}, 64, 181},
- {{153, 61, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 16, 206},
- {{22, 250, 130, 201, 132, 248, 189, 108, 0, 0, 0, 0, 0, 0, 0, 0}, 63, 122},
- {{158, 165, 234, 18, 44, 61, 82, 61, 235, 0, 0, 0, 0, 0, 0, 0}, 72, 81},
- {{236, 57, 124, 110, 124, 218, 82, 70, 142, 78, 18, 128, 0, 0, 0, 0}, 95, 175},
- {{94, 209, 200, 201, 149, 162, 248, 134, 239, 226, 1, 237, 16, 134, 56, 0}, 118, 170},
- {{187, 42, 31, 144, 236, 70, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 47, 174},
- {{90, 214, 185, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 29, 104},
- {{194, 220, 211, 212, 211, 32, 196, 98, 71, 62, 153, 103, 80, 35, 128, 0}, 114, 113},
- {{24, 255, 158, 64, 180, 148, 10, 81, 243, 247, 0, 0, 0, 0, 0, 0}, 80, 89},
- {{231, 155, 100, 242, 112, 160, 160, 95, 98, 253, 219, 21, 239, 90, 0, 0}, 113, 151},
- {{225, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 11, 108},
- {{136, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 7, 224},
- {{250, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 9, 95},
- {{72, 168, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 14, 173},
- {{185, 51, 51, 167, 18, 44, 36, 59, 35, 135, 20, 104, 0, 0, 0, 0}, 93, 176},
- {{57, 146, 252, 60, 197, 68, 39, 162, 80, 198, 137, 50, 97, 92, 124, 0}, 119, 84},
- {{254, 46, 242, 105, 86, 94, 96, 14, 130, 176, 0, 0, 0, 0, 0, 0}, 78, 104},
- {{247, 202, 176, 76, 69, 40, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 49, 236},
- {{50, 233, 203, 77, 42, 21, 115, 163, 166, 138, 192, 52, 178, 37, 112, 0}, 116, 153},
- {{62, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 11, 190},
- {{53, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 13, 202},
- {{198, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 8, 54},
- {{189, 234, 106, 247, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 34, 156},
- {{110, 24, 228, 65, 216, 147, 9, 48, 60, 179, 172, 91, 115, 185, 227, 96}, 126, 245},
- {{128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 6, 218},
- {{74, 177, 89, 218, 248, 18, 176, 39, 118, 173, 201, 152, 0, 0, 0, 0}, 93, 72},
- {{31, 13, 153, 92, 27, 122, 150, 232, 88, 95, 202, 171, 208, 158, 0, 0}, 112, 183},
- {{128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 2, 183},
- {{63, 37, 46, 158, 139, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 42, 241},
- {{53, 209, 59, 13, 202, 70, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 47, 106},
- {{184, 44, 149, 221, 180, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 40, 180},
- {{222, 134, 37, 62, 223, 193, 39, 246, 15, 151, 200, 146, 0, 0, 0, 0}, 96, 142},
- {{199, 176, 189, 37, 233, 177, 252, 216, 94, 175, 253, 119, 96, 0, 0, 0}, 100, 6},
- {{44, 195, 201, 106, 209, 120, 122, 38, 43, 30, 142, 22, 196, 175, 100, 0}, 118, 33},
- {{33, 166, 10, 174, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 34, 224},
- {{54, 1, 189, 195, 133, 49, 36, 80, 138, 200, 0, 0, 0, 0, 0, 0}, 78, 14},
- {{241, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 10, 149},
- {{221, 131, 4, 247, 112, 89, 187, 119, 219, 80, 122, 156, 216, 160, 0, 0}, 108, 131},
- {{102, 20, 46, 129, 202, 247, 129, 1, 237, 71, 103, 58, 217, 44, 4, 0}, 121, 133},
- {{107, 156, 151, 44, 215, 98, 171, 126, 85, 32, 42, 128, 0, 0, 0, 0}, 89, 33},
- {{54, 25, 70, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 28, 204},
- {{149, 211, 242, 14, 112, 219, 176, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 52, 43},
- {{95, 26, 143, 193, 8, 76, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 51, 168},
- {{63, 102, 244, 176, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 28, 180},
- {{64, 85, 124, 226, 59, 239, 64, 130, 68, 122, 93, 74, 32, 37, 0, 0}, 112, 208},
- {{113, 90, 253, 149, 3, 218, 34, 215, 3, 143, 192, 64, 0, 0, 0, 0}, 90, 25},
- {{75, 231, 33, 5, 11, 94, 117, 104, 150, 60, 72, 161, 96, 38, 0, 0}, 111, 50},
- {{52, 13, 248, 1, 251, 14, 50, 29, 212, 123, 130, 177, 101, 96, 0, 0}, 109, 110},
- {{248, 221, 150, 132, 252, 82, 96, 2, 80, 232, 97, 239, 253, 64, 0, 0}, 109, 21},
- {{136, 77, 164, 161, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 36, 147},
- {{1, 33, 66, 254, 144, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 43, 56},
- {{181, 25, 186, 225, 109, 190, 76, 158, 118, 122, 20, 64, 125, 55, 8, 0}, 117, 144},
- {{191, 187, 160, 140, 17, 6, 80, 120, 236, 212, 104, 144, 128, 0, 0, 0}, 100, 198},
- {{201, 61, 150, 254, 70, 77, 214, 211, 171, 163, 245, 64, 0, 0, 0, 0}, 90, 235},
- {{143, 226, 190, 50, 252, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 38, 105},
- {{65, 168, 226, 36, 201, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 45, 138},
- {{136, 40, 65, 90, 47, 16, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 49, 122},
- {{94, 189, 224, 200, 170, 11, 79, 172, 0, 0, 0, 0, 0, 0, 0, 0}, 65, 193},
- {{236, 41, 169, 234, 14, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 43, 231},
- {{1, 40, 140, 95, 81, 173, 250, 248, 0, 0, 0, 0, 0, 0, 0, 0}, 64, 250},
- {{83, 176, 146, 112, 89, 156, 57, 220, 125, 48, 44, 0, 0, 0, 0, 0}, 86, 24},
- {{76, 125, 228, 249, 243, 160, 106, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 55, 191},
- {{10, 203, 204, 49, 212, 115, 125, 4, 239, 122, 81, 34, 1, 198, 216, 0}, 117, 111},
- {{74, 214, 23, 44, 211, 40, 161, 61, 237, 190, 155, 59, 173, 42, 0, 0}, 111, 205},
- {{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 1, 133},
- {{127, 0, 130, 61, 209, 5, 232, 35, 35, 42, 114, 52, 169, 234, 191, 0}, 122, 122},
- {{201, 107, 210, 13, 187, 62, 145, 28, 31, 189, 56, 0, 0, 0, 0, 0}, 87, 227},
- {{147, 171, 63, 145, 47, 160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 46, 53},
- {{93, 232, 10, 97, 21, 243, 213, 135, 200, 0, 0, 0, 0, 0, 0, 0}, 72, 224},
- {{144, 121, 41, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 26, 199},
- {{116, 105, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 17, 79},
- {{142, 149, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 21, 19},
- {{97, 0, 228, 158, 50, 233, 251, 249, 0, 66, 197, 226, 0, 0, 0, 0}, 96, 211},
- {{114, 228, 199, 155, 175, 104, 26, 213, 66, 249, 120, 218, 164, 252, 212, 0}, 120, 6},
- {{224, 166, 76, 200, 121, 60, 110, 65, 60, 95, 137, 190, 92, 218, 218, 0}, 121, 143},
- {{139, 219, 92, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 31, 135},
- {{203, 237, 64, 189, 28, 13, 75, 197, 219, 243, 172, 3, 142, 32, 0, 0}, 109, 21},
- {{237, 186, 88, 254, 124, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 38, 220},
- {{182, 230, 93, 162, 129, 25, 56, 196, 112, 0, 0, 0, 0, 0, 0, 0}, 68, 151},
- {{245, 45, 69, 226, 90, 212, 254, 16, 0, 0, 0, 0, 0, 0, 0, 0}, 60, 111},
- {{107, 229, 240, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 20, 63},
- {{119, 208, 177, 235, 222, 252, 219, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 57, 112},
- {{178, 151, 220, 162, 120, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 41, 48},
- {{109, 26, 95, 170, 166, 151, 137, 83, 226, 82, 5, 114, 253, 210, 18, 12}, 126, 100},
- {{126, 27, 252, 19, 219, 129, 121, 48, 0, 0, 0, 0, 0, 0, 0, 0}, 60, 156},
- {{211, 195, 152, 145, 154, 93, 228, 215, 135, 101, 28, 82, 0, 0, 0, 0}, 95, 120},
- {{252, 107, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 16, 5},
- {{192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 4, 103},
- {{64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 4, 84},
- {{225, 179, 43, 43, 222, 145, 205, 238, 164, 158, 147, 229, 56, 0, 0, 0}, 101, 24},
- {{208, 127, 151, 24, 64, 113, 47, 85, 209, 79, 144, 0, 0, 0, 0, 0}, 86, 81},
- {{178, 144, 203, 104, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 29, 96},
- {{56, 227, 139, 4, 86, 87, 180, 1, 215, 167, 237, 156, 111, 64, 47, 0}, 121, 6},
- {{80, 76, 204, 119, 172, 169, 254, 81, 104, 166, 219, 44, 173, 161, 212, 0}, 119, 40},
- {{129, 141, 139, 34, 241, 101, 223, 144, 0, 0, 0, 0, 0, 0, 0, 0}, 62, 143},
- {{85, 102, 137, 98, 65, 103, 54, 142, 144, 0, 0, 0, 0, 0, 0, 0}, 68, 69},
- {{56, 31, 159, 13, 201, 139, 161, 31, 89, 137, 4, 0, 0, 0, 0, 0}, 92, 48},
- {{229, 221, 54, 216, 223, 27, 196, 1, 0, 0, 0, 0, 0, 0, 0, 0}, 64, 115},
- {{5, 144, 176, 43, 180, 187, 20, 49, 59, 73, 108, 34, 83, 32, 192, 0}, 115, 130},
- {{24, 217, 205, 193, 74, 123, 160, 106, 103, 74, 200, 0, 0, 0, 0, 0}, 86, 57},
- {{247, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 8, 97},
- {{12, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 9, 146},
- {{160, 28, 201, 119, 148, 93, 251, 118, 28, 179, 123, 52, 71, 232, 48, 0}, 117, 194},
- {{152, 126, 17, 54, 101, 56, 130, 1, 205, 41, 207, 90, 151, 123, 128, 0}, 114, 129},
- {{77, 165, 29, 239, 95, 242, 34, 1, 11, 204, 135, 239, 128, 0, 0, 0}, 97, 159},
- {{183, 108, 146, 118, 74, 190, 7, 141, 9, 92, 2, 2, 8, 218, 120, 0}, 117, 242},
- {{37, 152, 29, 239, 242, 53, 56, 143, 219, 22, 14, 158, 49, 0, 0, 0}, 104, 162},
- {{198, 53, 241, 102, 240, 244, 97, 203, 62, 128, 213, 214, 220, 0, 0, 0}, 102, 140},
- {{144, 89, 48, 42, 249, 231, 189, 178, 232, 199, 30, 58, 63, 57, 0, 0}, 113, 77},
- {{68, 212, 177, 123, 44, 224, 19, 172, 89, 87, 192, 0, 0, 0, 0, 0}, 82, 121},
- {{252, 29, 179, 224, 4, 121, 205, 67, 152, 0, 0, 0, 0, 0, 0, 0}, 69, 102},
- {{28, 110, 164, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 23, 28},
- {{24, 88, 231, 1, 4, 71, 71, 241, 252, 14, 197, 0, 0, 0, 0, 0}, 89, 154},
- {{63, 131, 43, 76, 58, 140, 163, 74, 158, 80, 0, 0, 0, 0, 0, 0}, 76, 39},
- {{56, 28, 147, 149, 98, 93, 216, 216, 203, 156, 0, 0, 0, 0, 0, 0}, 78, 163},
- {{134, 169, 6, 103, 161, 244, 134, 117, 16, 0, 0, 0, 0, 0, 0, 0}, 68, 42},
- {{143, 247, 125, 190, 106, 50, 204, 98, 250, 151, 161, 96, 0, 0, 0, 0}, 92, 207},
- {{235, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 8, 25},
- {{46, 72, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 18, 150},
- {{171, 35, 128, 117, 74, 29, 199, 67, 109, 176, 0, 0, 0, 0, 0, 0}, 76, 103},
- {{220, 233, 236, 112, 135, 136, 215, 43, 42, 0, 0, 0, 0, 0, 0, 0}, 71, 155},
- {{228, 11, 144, 117, 206, 192, 118, 25, 141, 78, 4, 105, 0, 0, 0, 0}, 96, 142},
- {{195, 67, 194, 229, 14, 53, 129, 7, 30, 208, 38, 100, 182, 59, 0, 0}, 112, 2},
- {{25, 76, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 19, 59},
- {{160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 4, 112},
- {{26, 203, 217, 152, 16, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 48, 166},
- {{250, 213, 14, 235, 110, 171, 174, 23, 102, 128, 0, 0, 0, 0, 0, 0}, 73, 62},
- {{175, 230, 160, 13, 187, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 50, 176},
- {{92, 155, 156, 93, 191, 73, 28, 82, 187, 129, 57, 5, 16, 0, 0, 0}, 100, 6},
- {{45, 203, 3, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 29, 26},
- {{120, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 5, 6},
- {{216, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 5, 13},
- {{135, 215, 0, 71, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 37, 41},
- {{221, 149, 1, 40, 112, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 36, 135},
- {{95, 143, 255, 194, 2, 157, 191, 113, 10, 229, 204, 56, 0, 0, 0, 0}, 93, 171},
- {{202, 212, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 19, 20},
- {{147, 203, 238, 120, 194, 23, 25, 58, 208, 177, 169, 0, 0, 0, 0, 0}, 89, 119},
- {{137, 170, 113, 252, 215, 194, 224, 146, 233, 87, 86, 192, 26, 46, 0, 0}, 112, 49},
- {{224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 4, 141},
- {{250, 90, 241, 174, 163, 72, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 47, 132},
- {{66, 190, 202, 144, 122, 86, 22, 103, 107, 164, 57, 54, 228, 128, 0, 0}, 105, 176},
- {{76, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 12, 186},
- {{120, 246, 1, 52, 187, 163, 78, 105, 224, 0, 0, 0, 0, 0, 0, 0}, 67, 93},
- {{137, 242, 136, 71, 98, 10, 53, 97, 160, 85, 132, 127, 185, 222, 0, 0}, 111, 242},
- {{255, 133, 181, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 24, 163},
- {{128, 177, 92, 155, 91, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 42, 184},
- {{45, 120, 186, 192, 240, 199, 178, 95, 32, 0, 0, 0, 0, 0, 0, 0}, 68, 188},
- {{151, 98, 103, 254, 90, 6, 10, 109, 14, 158, 69, 29, 140, 237, 40, 232}, 126, 193},
- {{148, 164, 81, 85, 76, 14, 84, 64, 89, 176, 0, 0, 0, 0, 0, 0}, 78, 63},
- {{145, 187, 165, 136, 88, 30, 107, 191, 205, 120, 119, 216, 158, 123, 64, 0}, 115, 160},
- {{78, 120, 28, 243, 216, 180, 87, 19, 253, 16, 110, 33, 228, 24, 232, 0}, 117, 251},
- {{74, 6, 166, 166, 183, 157, 96, 84, 151, 0, 0, 0, 0, 0, 0, 0}, 72, 228},
- {{89, 96, 4, 221, 214, 253, 58, 49, 9, 0, 0, 0, 0, 0, 0, 0}, 72, 168},
- {{97, 9, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 18, 194},
- {{213, 215, 45, 200, 170, 120, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 47, 166},
- {{5, 14, 92, 0, 28, 245, 130, 202, 32, 40, 207, 77, 166, 170, 246, 64}, 122, 210},
- {{77, 45, 43, 71, 202, 0, 157, 146, 59, 91, 225, 0, 0, 0, 0, 0}, 89, 254},
- {{101, 174, 94, 168, 162, 171, 71, 12, 16, 224, 0, 0, 0, 0, 0, 0}, 75, 49},
- {{58, 17, 187, 194, 87, 73, 215, 103, 180, 12, 40, 66, 0, 0, 0, 0}, 96, 95},
- {{160, 91, 68, 81, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 33, 193},
- {{94, 112, 249, 13, 167, 245, 101, 64, 0, 0, 0, 0, 0, 0, 0, 0}, 58, 155},
- {{236, 194, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 15, 133},
- {{168, 243, 103, 221, 120, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 38, 10},
- {{86, 194, 218, 188, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 33, 31},
- {{232, 3, 134, 67, 63, 196, 86, 14, 170, 243, 77, 134, 187, 140, 72, 18}, 127, 98},
- {{55, 253, 19, 201, 199, 71, 229, 218, 54, 64, 12, 162, 0, 0, 0, 0}, 96, 22},
- {{142, 34, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 20, 214},
- {{213, 16, 208, 50, 100, 33, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 50, 217},
- {{117, 237, 132, 185, 184, 246, 79, 42, 103, 98, 162, 243, 128, 0, 0, 0}, 98, 102},
- {{120, 25, 214, 222, 61, 157, 203, 102, 3, 146, 192, 0, 0, 0, 0, 0}, 83, 169},
- {{222, 46, 254, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 27, 152},
- {{254, 70, 158, 171, 11, 245, 223, 97, 70, 17, 27, 192, 186, 0, 0, 0}, 103, 214},
- {{192, 128, 228, 17, 68, 20, 44, 31, 52, 34, 212, 1, 224, 0, 0, 0}, 99, 178},
- {{237, 229, 203, 8, 121, 176, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 45, 164},
- {{6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 7, 15},
- {{71, 197, 251, 122, 138, 232, 12, 241, 116, 240, 0, 0, 0, 0, 0, 0}, 76, 94},
- {{18, 241, 135, 210, 233, 54, 121, 185, 4, 0, 0, 0, 0, 0, 0, 0}, 70, 239},
- {{32, 50, 213, 63, 73, 217, 180, 21, 187, 128, 0, 0, 0, 0, 0, 0}, 73, 82},
- {{203, 166, 233, 73, 92, 182, 212, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 55, 54},
- {{56, 162, 126, 4, 18, 195, 192, 64, 164, 156, 119, 196, 64, 0, 0, 0}, 98, 47},
- {{120, 87, 81, 136, 180, 179, 68, 148, 243, 38, 80, 0, 0, 0, 0, 0}, 84, 214},
- {{64, 244, 193, 50, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 37, 215},
- {{91, 168, 253, 158, 131, 83, 159, 163, 113, 169, 112, 0, 0, 0, 0, 0}, 84, 153},
- {{159, 103, 102, 132, 111, 46, 18, 77, 36, 15, 137, 33, 177, 31, 243, 192}, 122, 245},
- {{123, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 8, 118},
- {{67, 81, 226, 190, 7, 79, 71, 250, 155, 245, 44, 81, 215, 213, 171, 224}, 123, 128},
- {{103, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 8, 7},
- {{246, 44, 168, 200, 198, 238, 52, 196, 125, 115, 0, 0, 0, 0, 0, 0}, 80, 152},
- {{205, 14, 186, 252, 239, 213, 59, 119, 105, 37, 140, 209, 4, 231, 0, 0}, 114, 248},
- {{70, 91, 254, 106, 94, 71, 170, 19, 158, 242, 192, 0, 0, 0, 0, 0}, 85, 143},
- {{250, 86, 233, 184, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 30, 159},
- {{122, 222, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 24, 11},
- {{27, 224, 235, 70, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 31, 110},
- {{239, 100, 224, 3, 46, 127, 150, 251, 204, 120, 228, 64, 0, 0, 0, 0}, 97, 181},
- {{144, 115, 182, 206, 146, 13, 21, 111, 37, 70, 179, 129, 173, 82, 93, 128}, 121, 4},
- {{73, 190, 57, 243, 49, 51, 15, 209, 0, 0, 0, 0, 0, 0, 0, 0}, 67, 101},
- {{18, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 11, 38},
- {{23, 37, 236, 177, 186, 7, 209, 135, 114, 44, 0, 0, 0, 0, 0, 0}, 78, 57},
- {{200, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 17, 142},
- {{181, 255, 153, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 24, 184},
- {{135, 168, 6, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 27, 91},
- {{200, 224, 33, 245, 120, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 41, 224},
- {{70, 111, 10, 62, 200, 224, 38, 204, 14, 164, 0, 0, 0, 0, 0, 0}, 78, 114},
- {{158, 133, 252, 18, 242, 12, 16, 60, 5, 52, 251, 179, 38, 235, 12, 0}, 118, 184},
- {{2, 23, 116, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 28, 215},
- {{33, 25, 170, 74, 215, 134, 151, 181, 175, 232, 20, 155, 189, 242, 13, 0}, 120, 167},
- {{160, 186, 218, 183, 167, 84, 59, 152, 13, 137, 80, 128, 0, 0, 0, 0}, 89, 233},
- {{32, 141, 196, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 29, 101},
- {{207, 24, 202, 226, 191, 136, 78, 124, 160, 0, 0, 0, 0, 0, 0, 0}, 67, 139},
- {{210, 173, 172, 27, 197, 57, 114, 146, 169, 32, 0, 0, 0, 0, 0, 0}, 79, 32},
- {{95, 113, 12, 123, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 32, 57},
- {{129, 108, 186, 28, 19, 229, 96, 134, 199, 254, 199, 64, 0, 0, 0, 0}, 91, 151},
- {{103, 226, 38, 123, 35, 199, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 49, 0},
- {{41, 117, 43, 35, 208, 115, 73, 64, 0, 0, 0, 0, 0, 0, 0, 0}, 63, 227},
- {{42, 220, 61, 34, 199, 183, 42, 16, 223, 135, 0, 135, 213, 150, 100, 0}, 118, 124},
- {{165, 227, 96, 243, 112, 171, 117, 106, 50, 37, 82, 60, 80, 0, 0, 0}, 104, 228},
- {{158, 60, 111, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 27, 64},
- {{124, 108, 88, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 25, 179},
- {{232, 68, 132, 159, 156, 103, 95, 190, 76, 0, 0, 0, 0, 0, 0, 0}, 70, 107},
- {{70, 77, 240, 209, 72, 63, 63, 45, 125, 79, 77, 41, 13, 0, 0, 0}, 104, 206},
- {{146, 254, 7, 5, 68, 240, 67, 237, 112, 0, 0, 0, 0, 0, 0, 0}, 68, 95},
- {{162, 223, 117, 27, 2, 156, 94, 170, 157, 114, 162, 50, 0, 0, 0, 0}, 96, 219},
- {{161, 62, 191, 68, 239, 73, 100, 37, 168, 254, 139, 202, 252, 65, 74, 0}, 119, 138},
- {{248, 122, 115, 81, 15, 158, 136, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 53, 84},
- {{8, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 11, 161},
- {{142, 96, 105, 133, 251, 57, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 52, 25},
- {{138, 196, 139, 131, 233, 93, 65, 242, 86, 169, 7, 72, 82, 128, 0, 0}, 107, 113},
- {{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 2, 46},
- {{175, 151, 75, 238, 26, 12, 100, 186, 0, 0, 0, 0, 0, 0, 0, 0}, 63, 72},
- {{82, 205, 211, 176, 170, 79, 57, 153, 161, 218, 32, 48, 0, 0, 0, 0}, 93, 230},
- {{227, 123, 232, 74, 236, 202, 211, 121, 200, 8, 59, 189, 81, 219, 144, 0}, 117, 142},
- {{205, 196, 89, 90, 103, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 41, 134},
- {{63, 145, 23, 127, 102, 216, 49, 36, 168, 164, 59, 133, 18, 146, 0, 0}, 112, 100},
- {{213, 72, 154, 16, 230, 236, 218, 203, 223, 51, 31, 251, 103, 64, 0, 0}, 109, 45},
- {{126, 148, 232, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 30, 219},
- {{160, 72, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 16, 52},
- {{137, 38, 146, 20, 99, 188, 83, 123, 159, 159, 64, 0, 0, 0, 0, 0}, 83, 240},
- {{123, 228, 36, 44, 242, 29, 51, 228, 140, 60, 237, 0, 0, 0, 0, 0}, 90, 13},
- {{163, 169, 25, 89, 190, 114, 165, 158, 140, 210, 192, 0, 0, 0, 0, 0}, 84, 191},
- {{225, 38, 70, 89, 218, 236, 60, 5, 69, 163, 248, 50, 163, 64, 0, 0}, 106, 95},
- {{91, 94, 36, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 28, 65},
- {{209, 238, 110, 0, 2, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 45, 195},
- {{57, 17, 224, 164, 69, 95, 138, 172, 111, 55, 239, 167, 160, 0, 0, 0}, 103, 21},
- {{112, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 4, 114},
- {{102, 96, 223, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 31, 92},
- {{137, 204, 150, 75, 193, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 42, 237},
- {{136, 56, 252, 240, 85, 48, 248, 231, 17, 49, 47, 238, 15, 233, 159, 184}, 125, 172},
- {{57, 31, 132, 123, 234, 255, 37, 82, 167, 204, 37, 158, 128, 0, 0, 0}, 98, 116},
- {{55, 198, 139, 219, 161, 156, 140, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 55, 54},
- {{44, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 8, 203},
- {{53, 38, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 16, 74},
- {{227, 62, 107, 236, 118, 156, 60, 34, 31, 179, 76, 221, 0, 0, 0, 0}, 96, 220},
- {{105, 40, 240, 216, 91, 61, 19, 128, 224, 0, 0, 0, 0, 0, 0, 0}, 67, 219},
- {{96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 3, 179},
- {{118, 142, 251, 249, 128, 105, 113, 16, 0, 0, 0, 0, 0, 0, 0, 0}, 61, 194},
- {{101, 70, 196, 238, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 32, 187},
- {{245, 173, 165, 177, 200, 161, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 50, 79},
- {{0, 198, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 19, 87},
- {{92, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 8, 126},
- {{125, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 11, 106},
- {{56, 59, 35, 82, 101, 72, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 50, 96},
- {{184, 72, 77, 251, 8, 166, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 47, 45},
- {{143, 74, 132, 205, 218, 247, 30, 160, 145, 199, 138, 12, 89, 220, 0, 0}, 110, 8},
- {{30, 178, 111, 225, 73, 79, 173, 52, 0, 0, 0, 0, 0, 0, 0, 0}, 62, 226},
- {{224, 48, 154, 231, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 36, 222},
- {{123, 144, 170, 143, 85, 169, 130, 245, 214, 0, 0, 0, 0, 0, 0, 0}, 71, 218},
- {{166, 224, 212, 100, 149, 55, 35, 210, 246, 108, 41, 245, 127, 174, 128, 0}, 116, 59},
- {{75, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 8, 80},
- {{197, 128, 190, 87, 47, 53, 92, 64, 0, 0, 0, 0, 0, 0, 0, 0}, 58, 177},
- {{249, 10, 76, 217, 225, 20, 124, 205, 44, 159, 190, 8, 0, 0, 0, 0}, 98, 44},
- {{180, 226, 0, 167, 137, 232, 174, 120, 113, 95, 22, 184, 0, 0, 0, 0}, 93, 206},
- {{123, 153, 102, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 27, 64},
- {{5, 144, 206, 158, 239, 189, 171, 120, 69, 46, 128, 237, 0, 0, 0, 0}, 96, 236},
- {{159, 235, 60, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 22, 101},
- {{42, 194, 150, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 26, 49},
- {{205, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 11, 179},
- {{19, 65, 141, 20, 127, 77, 70, 205, 151, 115, 157, 23, 118, 128, 0, 0}, 109, 112},
- {{96, 11, 214, 40, 245, 251, 61, 64, 128, 241, 183, 183, 0, 0, 0, 0}, 96, 31},
- {{120, 4, 235, 112, 34, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 41, 111},
- {{110, 127, 207, 76, 100, 148, 130, 206, 249, 2, 104, 0, 0, 0, 0, 0}, 86, 65},
- {{226, 190, 191, 249, 173, 96, 127, 200, 62, 20, 0, 0, 0, 0, 0, 0}, 78, 222},
- {{89, 88, 182, 14, 78, 122, 213, 192, 0, 0, 0, 0, 0, 0, 0, 0}, 58, 4},
- {{167, 94, 163, 227, 28, 111, 117, 103, 224, 0, 0, 0, 0, 0, 0, 0}, 67, 67},
- {{57, 220, 53, 116, 243, 184, 242, 134, 16, 70, 83, 61, 161, 128, 0, 0}, 109, 197},
- {{63, 235, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 22, 121},
- {{128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 1, 167},
- {{15, 159, 42, 167, 76, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 38, 140},
- {{216, 252, 113, 40, 239, 46, 172, 48, 103, 250, 82, 179, 136, 64, 0, 0}, 106, 193},
- {{158, 147, 16, 44, 124, 56, 44, 48, 138, 64, 169, 0, 0, 0, 0, 0}, 90, 47},
- {{238, 238, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 21, 187},
- {{63, 159, 177, 162, 106, 212, 172, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 56, 102},
- {{59, 40, 252, 185, 187, 216, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 46, 237},
- {{2, 218, 11, 68, 173, 196, 16, 223, 2, 18, 122, 215, 154, 0, 0, 0}, 103, 237},
- {{3, 9, 206, 73, 108, 196, 183, 119, 141, 162, 10, 180, 115, 32, 0, 0}, 107, 115},
- {{17, 227, 208, 146, 63, 201, 73, 239, 29, 79, 80, 0, 0, 0, 0, 0}, 84, 217},
- {{115, 180, 176, 241, 52, 209, 6, 64, 189, 76, 0, 0, 0, 0, 0, 0}, 79, 21},
- {{191, 88, 98, 245, 91, 46, 137, 254, 170, 80, 11, 55, 212, 28, 128, 0}, 113, 3},
- {{97, 141, 171, 175, 22, 233, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 48, 62},
- {{32, 204, 102, 191, 164, 242, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 47, 80},
- {{29, 133, 210, 252, 124, 66, 160, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 51, 184},
- {{207, 179, 54, 144, 116, 67, 29, 64, 13, 199, 0, 0, 0, 0, 0, 0}, 80, 197},
- {{129, 216, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 13, 63},
- {{50, 152, 249, 143, 174, 234, 240, 48, 158, 255, 80, 105, 0, 0, 0, 0}, 99, 62},
- {{105, 208, 95, 218, 44, 11, 87, 134, 109, 18, 138, 66, 17, 69, 128, 0}, 114, 231},
- {{151, 79, 158, 220, 122, 101, 210, 164, 64, 0, 0, 0, 0, 0, 0, 0}, 67, 158},
- {{236, 97, 87, 155, 254, 137, 122, 208, 168, 201, 194, 118, 224, 0, 0, 0}, 101, 118},
- {{14, 229, 193, 248, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 30, 237},
- {{46, 154, 50, 80, 92, 147, 158, 86, 1, 112, 0, 0, 0, 0, 0, 0}, 79, 15},
- {{88, 131, 21, 84, 62, 86, 7, 110, 142, 251, 242, 110, 194, 175, 247, 0}, 122, 84},
- {{229, 216, 111, 92, 173, 32, 63, 70, 36, 84, 6, 74, 136, 166, 38, 0}, 119, 205},
- {{121, 147, 216, 245, 37, 189, 146, 63, 145, 74, 128, 0, 0, 0, 0, 0}, 82, 220},
- {{44, 26, 254, 11, 181, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 40, 42},
- {{209, 114, 97, 249, 227, 159, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 51, 144},
- {{184, 244, 43, 117, 56, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 37, 74},
- {{60, 81, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 19, 89},
- {{18, 40, 21, 113, 226, 91, 195, 88, 161, 19, 142, 0, 0, 0, 0, 0}, 88, 77},
- {{57, 0, 212, 158, 56, 51, 108, 198, 59, 5, 137, 196, 0, 0, 0, 0}, 94, 2},
- {{168, 252, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 14, 75},
- {{64, 181, 254, 103, 1, 230, 117, 199, 128, 0, 0, 0, 0, 0, 0, 0}, 65, 18},
- {{212, 48, 214, 127, 78, 176, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 46, 246},
- {{155, 185, 236, 163, 204, 49, 129, 120, 183, 47, 10, 243, 65, 92, 192, 0}, 114, 10},
- {{94, 200, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 14, 207},
- {{19, 210, 136, 113, 73, 79, 132, 196, 224, 0, 0, 0, 0, 0, 0, 0}, 68, 41},
- {{24, 203, 246, 242, 241, 223, 150, 237, 213, 202, 11, 128, 0, 0, 0, 0}, 89, 102},
- {{115, 59, 171, 221, 172, 181, 170, 67, 115, 205, 44, 107, 162, 67, 56, 0}, 118, 118},
- {{250, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 10, 146},
- {{203, 240, 28, 158, 182, 12, 86, 182, 142, 47, 143, 57, 239, 0, 0, 0}, 104, 122},
- {{196, 218, 109, 52, 2, 0, 64, 153, 34, 250, 240, 185, 117, 0, 0, 0}, 107, 6},
- {{137, 131, 191, 40, 72, 209, 74, 64, 0, 0, 0, 0, 0, 0, 0, 0}, 58, 18},
- {{236, 126, 167, 37, 185, 20, 34, 207, 76, 0, 0, 0, 0, 0, 0, 0}, 70, 83},
- {{129, 192, 245, 137, 251, 52, 75, 68, 81, 112, 146, 133, 64, 0, 0, 0}, 99, 90},
- {{7, 31, 148, 19, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 32, 140},
- {{112, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 4, 242},
- {{167, 50, 202, 179, 74, 146, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 50, 31},
- {{44, 188, 186, 250, 229, 71, 28, 118, 35, 253, 245, 191, 199, 18, 0, 0}, 111, 9},
- {{128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 1, 230},
- {{156, 163, 215, 175, 71, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 47, 50},
- {{67, 24, 151, 198, 242, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 41, 34},
- {{134, 107, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 17, 11},
- {{35, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 8, 71},
- {{46, 196, 84, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 22, 146},
- {{82, 172, 8, 26, 154, 34, 125, 188, 5, 149, 159, 44, 78, 222, 236, 176}, 124, 249},
- {{78, 157, 79, 70, 252, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 39, 143},
- {{231, 5, 210, 247, 198, 5, 157, 191, 206, 225, 149, 142, 207, 40, 0, 0}, 110, 17},
- {{38, 254, 235, 199, 191, 60, 43, 159, 190, 243, 203, 185, 184, 218, 132, 0}, 119, 60},
- {{128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 1, 162},
- {{95, 240, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 12, 5},
- {{17, 128, 244, 178, 160, 78, 83, 92, 0, 0, 0, 0, 0, 0, 0, 0}, 62, 139},
- {{18, 102, 62, 251, 44, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 39, 8},
- {{30, 75, 108, 40, 231, 166, 233, 220, 163, 176, 252, 210, 60, 30, 128, 0}, 114, 246},
- {{18, 3, 207, 64, 25, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 42, 171},
- {{52, 83, 235, 61, 164, 236, 83, 173, 143, 105, 14, 0, 0, 0, 0, 0}, 88, 206},
- {{166, 175, 186, 208, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 28, 163},
- {{221, 154, 82, 98, 41, 126, 85, 52, 0, 0, 0, 0, 0, 0, 0, 0}, 62, 166},
- {{94, 84, 182, 120, 204, 232, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 51, 128},
- {{27, 174, 227, 228, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 31, 59},
- {{218, 12, 4, 156, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 32, 179},
- {{9, 5, 190, 195, 60, 216, 80, 150, 128, 117, 86, 128, 128, 112, 98, 208}, 124, 87},
- {{7, 226, 104, 112, 212, 9, 172, 124, 209, 121, 170, 229, 44, 178, 128, 0}, 114, 29},
- {{47, 71, 174, 76, 52, 83, 23, 18, 106, 48, 56, 32, 0, 0, 0, 0}, 91, 184},
- {{51, 168, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 14, 45},
- {{28, 182, 167, 124, 28, 22, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 55, 144},
- {{34, 61, 14, 51, 253, 17, 19, 170, 49, 206, 188, 207, 247, 167, 192, 0}, 114, 119},
- {{2, 235, 18, 14, 195, 66, 237, 30, 0, 0, 0, 0, 0, 0, 0, 0}, 64, 113},
- {{51, 182, 142, 133, 127, 96, 159, 132, 99, 161, 64, 0, 0, 0, 0, 0}, 82, 50},
- {{170, 145, 230, 123, 215, 189, 73, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 56, 207},
- {{151, 166, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 21, 3},
- {{16, 141, 196, 129, 132, 207, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 58, 13},
- {{205, 25, 184, 191, 201, 206, 109, 224, 0, 0, 0, 0, 0, 0, 0, 0}, 59, 42},
- {{48, 114, 33, 103, 247, 255, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 50, 31},
- {{179, 156, 119, 146, 125, 21, 42, 146, 237, 213, 191, 132, 0, 0, 0, 0}, 94, 30},
- {{179, 129, 186, 144, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 29, 94},
- {{17, 179, 217, 188, 128, 212, 4, 4, 152, 0, 0, 0, 0, 0, 0, 0}, 71, 190},
- {{132, 63, 74, 89, 209, 64, 63, 192, 0, 0, 0, 0, 0, 0, 0, 0}, 59, 238},
- {{16, 50, 248, 88, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 30, 20},
- {{189, 96, 58, 53, 191, 235, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 51, 84},
- {{111, 98, 6, 65, 35, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 42, 108},
- {{118, 223, 83, 220, 110, 122, 23, 112, 185, 155, 73, 0, 0, 0, 0, 0}, 89, 136},
- {{173, 191, 150, 197, 204, 35, 169, 79, 31, 214, 251, 240, 0, 0, 0, 0}, 93, 196},
- {{26, 76, 129, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 28, 67},
- {{231, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 12, 104},
- {{93, 172, 223, 252, 203, 0, 206, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 55, 15},
- {{53, 142, 203, 124, 104, 51, 241, 12, 161, 17, 101, 245, 120, 110, 192, 199}, 128, 237},
- {{9, 77, 120, 197, 193, 10, 237, 174, 233, 2, 165, 11, 229, 47, 144, 0}, 116, 224},
- {{99, 161, 189, 88, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 36, 179},
- {{18, 8, 76, 66, 2, 185, 206, 132, 224, 0, 0, 0, 0, 0, 0, 0}, 67, 84},
- {{169, 53, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 22, 65},
- {{136, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 5, 178},
- {{131, 162, 144, 124, 12, 98, 242, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 55, 154},
- {{75, 50, 129, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 27, 106},
- {{212, 183, 40, 225, 152, 136, 174, 91, 0, 0, 0, 0, 0, 0, 0, 0}, 67, 125},
- {{158, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 9, 118},
- {{7, 48, 132, 149, 169, 212, 198, 137, 202, 0, 0, 0, 0, 0, 0, 0}, 73, 52},
- {{173, 195, 129, 163, 141, 249, 40, 64, 0, 0, 0, 0, 0, 0, 0, 0}, 58, 173},
- {{109, 79, 75, 219, 205, 182, 22, 245, 223, 17, 146, 78, 109, 119, 128, 0}, 113, 8},
- {{174, 195, 24, 182, 215, 198, 214, 86, 34, 128, 0, 0, 0, 0, 0, 0}, 74, 211},
- {{22, 40, 51, 109, 70, 91, 152, 56, 0, 0, 0, 0, 0, 0, 0, 0}, 61, 253},
- {{169, 115, 246, 126, 65, 118, 219, 192, 0, 0, 0, 0, 0, 0, 0, 0}, 59, 47},
- {{154, 37, 70, 124, 107, 123, 232, 241, 164, 142, 71, 226, 182, 126, 0, 0}, 112, 73},
- {{6, 108, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 19, 192},
- {{216, 167, 158, 158, 222, 19, 96, 28, 40, 6, 70, 12, 147, 27, 85, 240}, 128, 55},
- {{72, 222, 52, 69, 69, 206, 163, 106, 235, 206, 80, 128, 0, 0, 0, 0}, 94, 147},
- {{150, 112, 106, 56, 15, 243, 154, 97, 134, 110, 160, 20, 183, 144, 234, 8}, 125, 86},
- {{58, 186, 106, 58, 124, 171, 53, 85, 33, 100, 64, 0, 0, 0, 0, 0}, 82, 16},
- {{7, 195, 22, 31, 62, 217, 209, 46, 90, 49, 189, 50, 168, 126, 0, 0}, 111, 167},
- {{92, 44, 159, 198, 185, 94, 231, 177, 64, 0, 0, 0, 0, 0, 0, 0}, 67, 148},
- {{169, 108, 190, 162, 23, 39, 140, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 54, 66},
- {{161, 5, 3, 11, 158, 157, 166, 212, 246, 22, 140, 101, 92, 0, 0, 0}, 104, 70},
- {{71, 191, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 16, 166},
- {{48, 136, 194, 145, 57, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 44, 109},
- {{144, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 5, 226},
- {{223, 209, 10, 57, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 32, 8},
- {{154, 79, 170, 9, 43, 139, 249, 176, 186, 72, 216, 0, 0, 0, 0, 0}, 85, 218},
- {{1, 8, 123, 205, 167, 134, 128, 102, 10, 72, 0, 0, 0, 0, 0, 0}, 78, 54},
- {{31, 105, 48, 77, 103, 187, 99, 67, 96, 0, 0, 0, 0, 0, 0, 0}, 67, 48},
- {{14, 73, 54, 76, 232, 35, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 51, 244},
- {{14, 109, 251, 190, 36, 253, 99, 120, 94, 64, 0, 0, 0, 0, 0, 0}, 74, 50},
- {{122, 170, 9, 134, 124, 91, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 54, 173},
- {{246, 10, 85, 88, 82, 217, 95, 56, 216, 203, 160, 0, 0, 0, 0, 0}, 84, 245},
- {{77, 100, 114, 207, 150, 177, 69, 134, 74, 131, 147, 117, 177, 64, 210, 128}, 121, 54},
- {{171, 123, 22, 138, 132, 229, 250, 81, 186, 227, 146, 27, 170, 205, 128, 0}, 113, 86},
- {{128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 2, 115},
- {{12, 35, 50, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 24, 144},
- {{255, 124, 179, 165, 169, 250, 66, 171, 223, 125, 247, 0, 0, 0, 0, 0}, 89, 171},
- {{244, 235, 211, 10, 251, 255, 206, 6, 198, 12, 50, 136, 0, 0, 0, 0}, 93, 231},
- {{221, 77, 237, 41, 50, 33, 103, 24, 25, 127, 208, 0, 0, 0, 0, 0}, 88, 34},
- {{216, 69, 47, 53, 117, 24, 56, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 53, 225},
- {{180, 87, 25, 236, 56, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 38, 174},
- {{110, 32, 24, 34, 116, 133, 245, 128, 123, 95, 125, 122, 100, 129, 128, 0}, 113, 37},
- {{27, 117, 179, 112, 133, 137, 110, 193, 246, 201, 219, 65, 56, 234, 106, 128}, 121, 39},
- {{186, 117, 252, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 23, 59},
- {{243, 119, 54, 16, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 34, 96},
- {{160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 3, 147},
- {{78, 48, 117, 200, 245, 118, 115, 240, 170, 125, 84, 103, 33, 168, 0, 0}, 110, 56},
- {{201, 253, 184, 254, 143, 81, 95, 42, 243, 147, 96, 145, 23, 26, 0, 0}, 111, 234},
- {{41, 215, 84, 136, 234, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 46, 199},
- {{91, 244, 137, 184, 231, 95, 135, 10, 184, 0, 0, 0, 0, 0, 0, 0}, 69, 191},
- {{113, 31, 181, 245, 21, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 40, 235},
- {{181, 216, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 20, 45},
- {{87, 26, 119, 229, 97, 255, 9, 43, 32, 0, 0, 0, 0, 0, 0, 0}, 67, 164},
- {{205, 112, 67, 163, 196, 148, 5, 105, 8, 138, 144, 3, 171, 213, 159, 128}, 121, 130},
- {{136, 27, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 18, 166},
- {{2, 175, 111, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 24, 140},
- {{222, 131, 85, 218, 16, 229, 44, 230, 243, 76, 250, 139, 1, 203, 108, 0}, 118, 47},
- {{101, 180, 77, 142, 194, 73, 196, 246, 107, 100, 194, 72, 204, 124, 0, 0}, 111, 148},
- {{96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 3, 103},
- {{46, 62, 191, 130, 110, 128, 235, 62, 68, 39, 58, 152, 207, 204, 96, 0}, 116, 94},
- {{111, 11, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 19, 85},
- {{58, 43, 14, 93, 102, 210, 117, 208, 222, 171, 130, 41, 16, 16, 0, 0}, 109, 250},
- {{141, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 16, 153},
- {{170, 153, 160, 170, 144, 235, 122, 8, 106, 34, 24, 32, 102, 57, 12, 168}, 125, 182},
- {{34, 113, 163, 107, 61, 177, 39, 172, 242, 2, 130, 0, 0, 0, 0, 0}, 94, 23},
- {{222, 191, 239, 110, 162, 191, 195, 181, 80, 50, 85, 240, 88, 32, 0, 0}, 108, 38},
- {{179, 82, 253, 151, 212, 0, 72, 253, 175, 22, 34, 78, 53, 32, 0, 0}, 110, 121},
- {{10, 162, 20, 46, 164, 64, 88, 1, 202, 204, 124, 0, 0, 0, 0, 0}, 87, 146},
- {{210, 99, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 16, 138},
- {{183, 200, 1, 2, 51, 6, 66, 142, 20, 77, 48, 244, 0, 0, 0, 0}, 94, 149},
- {{29, 20, 224, 57, 204, 161, 131, 254, 53, 133, 163, 0, 0, 0, 0, 0}, 88, 232},
- {{75, 58, 170, 52, 146, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 45, 255},
- {{92, 21, 1, 113, 185, 88, 160, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 51, 148},
- {{103, 180, 222, 187, 129, 117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 48, 117},
- {{32, 76, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 14, 237},
- {{7, 60, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 19, 113},
- {{167, 122, 205, 185, 21, 199, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 57, 162},
- {{21, 78, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 21, 225},
- {{92, 159, 167, 169, 136, 176, 95, 255, 87, 137, 112, 16, 0, 0, 0, 0}, 92, 210},
- {{84, 120, 160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 22, 34},
- {{126, 5, 126, 176, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 29, 224},
- {{4, 58, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 16, 143},
- {{239, 154, 181, 182, 189, 211, 244, 53, 144, 0, 0, 0, 0, 0, 0, 0}, 68, 216},
- {{254, 188, 139, 167, 135, 47, 147, 239, 187, 106, 228, 156, 234, 234, 102, 0}, 120, 239},
- {{225, 168, 138, 92, 193, 255, 47, 233, 11, 154, 205, 86, 209, 88, 0, 0}, 111, 54},
- {{223, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 9, 35},
- {{235, 252, 115, 10, 151, 104, 193, 207, 38, 228, 229, 245, 42, 13, 108, 0}, 119, 230},
- {{1, 137, 53, 36, 210, 184, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 45, 234},
- {{149, 182, 72, 197, 92, 229, 9, 10, 220, 128, 72, 19, 4, 58, 192, 0}, 115, 70},
- {{105, 73, 57, 108, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 32, 246},
- {{189, 61, 230, 24, 235, 82, 58, 102, 97, 111, 121, 252, 156, 94, 191, 166}, 127, 217},
- {{193, 108, 231, 86, 140, 14, 192, 4, 135, 80, 129, 166, 158, 61, 230, 20}, 128, 201},
- {{110, 77, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 17, 49},
- {{3, 102, 36, 231, 15, 242, 143, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 57, 2},
- {{81, 189, 220, 168, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 38, 64},
- {{168, 75, 133, 180, 91, 165, 77, 232, 0, 0, 0, 0, 0, 0, 0, 0}, 62, 239},
- {{106, 179, 186, 109, 81, 234, 233, 167, 101, 160, 90, 102, 174, 234, 208, 0}, 116, 47},
- {{46, 105, 234, 21, 23, 247, 169, 33, 47, 5, 0, 0, 0, 0, 0, 0}, 80, 43},
- {{152, 144, 100, 142, 129, 23, 227, 50, 67, 81, 249, 116, 0, 0, 0, 0}, 94, 17},
- {{109, 74, 145, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 26, 5},
- {{100, 243, 22, 230, 38, 44, 128, 86, 132, 57, 0, 0, 0, 0, 0, 0}, 81, 240},
- {{153, 251, 115, 65, 104, 179, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 50, 197},
- {{43, 113, 60, 224, 36, 20, 42, 161, 24, 223, 192, 0, 0, 0, 0, 0}, 84, 192},
- {{61, 77, 121, 176, 138, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 47, 160},
- {{119, 194, 146, 49, 59, 242, 25, 220, 122, 104, 80, 0, 0, 0, 0, 0}, 84, 199},
- {{254, 162, 155, 47, 187, 3, 1, 114, 142, 191, 152, 44, 144, 26, 202, 0}, 127, 217},
- {{176, 1, 114, 42, 191, 145, 43, 1, 141, 18, 64, 0, 0, 0, 0, 0}, 83, 75},
- {{170, 244, 67, 132, 145, 163, 76, 213, 85, 237, 248, 22, 207, 64, 0, 0}, 106, 222},
- {{102, 190, 58, 32, 75, 15, 89, 163, 64, 7, 168, 0, 0, 0, 0, 0}, 85, 39},
- {{124, 170, 35, 47, 152, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 45, 9},
- {{192, 221, 20, 228, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 33, 217},
- {{208, 178, 136, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 25, 142},
- {{188, 68, 77, 30, 68, 153, 102, 180, 0, 0, 0, 0, 0, 0, 0, 0}, 62, 18},
- {{114, 178, 121, 188, 205, 233, 35, 77, 34, 197, 158, 174, 101, 0, 0, 0}, 104, 180},
- {{195, 98, 67, 12, 13, 43, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 56, 205},
- {{146, 190, 42, 222, 14, 54, 28, 128, 0, 0, 0, 0, 0, 0, 0, 0}, 57, 251},
- {{185, 202, 148, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 22, 178},
- {{138, 30, 129, 95, 224, 161, 120, 128, 0, 0, 0, 0, 0, 0, 0, 0}, 59, 198},
- {{69, 181, 5, 227, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 34, 84},
- {{90, 180, 0, 164, 227, 75, 174, 119, 128, 0, 0, 0, 0, 0, 0, 0}, 66, 128},
- {{20, 60, 58, 119, 245, 177, 162, 186, 13, 112, 211, 239, 128, 0, 0, 0}, 97, 75},
- {{158, 124, 157, 25, 230, 139, 51, 212, 76, 109, 236, 210, 48, 0, 0, 0}, 101, 192},
- {{125, 108, 242, 36, 94, 13, 36, 106, 90, 51, 83, 217, 131, 151, 0, 0}, 114, 60},
- {{222, 218, 162, 158, 15, 53, 191, 178, 0, 0, 0, 0, 0, 0, 0, 0}, 63, 169},
- {{104, 202, 127, 109, 73, 16, 17, 12, 0, 0, 0, 0, 0, 0, 0, 0}, 62, 10},
- {{172, 171, 246, 26, 176, 34, 22, 152, 246, 56, 173, 120, 105, 60, 92, 0}, 118, 64},
- {{190, 22, 171, 206, 109, 186, 179, 128, 253, 182, 108, 212, 220, 167, 171, 180}, 127, 182},
- {{119, 208, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 13, 29},
- {{160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 3, 39},
- {{170, 144, 64, 2, 107, 166, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 51, 93},
- {{234, 9, 96, 20, 156, 157, 1, 34, 88, 0, 0, 0, 0, 0, 0, 0}, 75, 228},
- {{147, 237, 16, 120, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 34, 236},
- {{182, 189, 162, 158, 223, 90, 173, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 56, 190},
- {{116, 148, 142, 240, 10, 253, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 49, 217},
- {{211, 73, 140, 69, 252, 27, 75, 46, 37, 6, 147, 32, 0, 0, 0, 0}, 93, 74},
- {{148, 61, 120, 49, 220, 65, 150, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 61, 180},
- {{172, 35, 202, 180, 129, 75, 240, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 55, 91},
- {{215, 109, 147, 157, 32, 28, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 54, 230},
- {{151, 26, 182, 112, 205, 220, 160, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 52, 175},
- {{73, 91, 93, 61, 196, 3, 66, 26, 149, 96, 0, 0, 0, 0, 0, 0}, 75, 171},
- {{203, 163, 52, 247, 28, 119, 56, 223, 138, 70, 174, 97, 77, 59, 46, 0}, 120, 202},
- {{251, 50, 228, 178, 202, 195, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 49, 113},
- {{217, 159, 164, 199, 14, 237, 170, 184, 100, 231, 92, 222, 0, 0, 0, 0}, 96, 187},
- {{16, 161, 85, 193, 202, 21, 3, 155, 63, 116, 124, 203, 34, 13, 215, 0}, 120, 38},
- {{111, 52, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 26, 35},
- {{69, 12, 116, 151, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 38, 115},
- {{187, 60, 97, 40, 112, 101, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 51, 18},
- {{230, 194, 136, 255, 206, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 39, 34},
- {{179, 239, 170, 107, 3, 13, 212, 67, 177, 69, 8, 0, 0, 0, 0, 0}, 87, 75},
- {{11, 58, 130, 89, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 33, 232},
- {{217, 178, 43, 203, 234, 20, 234, 186, 157, 88, 146, 192, 0, 0, 0, 0}, 91, 154},
- {{6, 180, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 19, 195},
- {{157, 154, 218, 158, 39, 224, 103, 230, 164, 0, 0, 0, 0, 0, 0, 0}, 70, 122},
- {{225, 10, 131, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 24, 97},
- {{16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 5, 220},
- {{166, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 7, 80},
- {{29, 190, 131, 215, 232, 246, 41, 226, 52, 192, 0, 0, 0, 0, 0, 0}, 77, 133},
- {{138, 74, 163, 93, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 38, 93},
- {{229, 64, 97, 41, 28, 243, 249, 185, 97, 35, 49, 27, 175, 24, 0, 0}, 110, 176},
- {{6, 73, 94, 160, 186, 216, 84, 117, 233, 169, 146, 234, 0, 0, 0, 0}, 95, 68},
- {{163, 40, 242, 81, 224, 35, 72, 194, 176, 78, 224, 174, 12, 0, 0, 0}, 103, 247},
- {{2, 205, 40, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 29, 240},
- {{174, 225, 240, 160, 212, 8, 246, 67, 36, 0, 0, 0, 0, 0, 0, 0}, 74, 83},
- {{5, 117, 182, 141, 166, 249, 196, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 55, 132},
- {{46, 152, 169, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 24, 217},
- {{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 3, 214},
- {{233, 202, 159, 219, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 34, 193},
- {{172, 54, 159, 5, 14, 245, 106, 182, 2, 0, 0, 0, 0, 0, 0, 0}, 71, 61},
- {{241, 222, 251, 114, 200, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 39, 65},
- {{31, 243, 190, 4, 207, 198, 249, 59, 167, 127, 93, 64, 0, 0, 0, 0}, 91, 108},
- {{201, 35, 222, 208, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 28, 244},
- {{187, 105, 13, 114, 238, 197, 145, 23, 169, 116, 91, 28, 0, 0, 0, 0}, 95, 194},
- {{251, 251, 121, 168, 152, 178, 147, 188, 229, 123, 154, 242, 190, 165, 173, 48}, 124, 82},
- {{66, 187, 191, 164, 31, 196, 40, 186, 148, 115, 134, 57, 222, 254, 48, 0}, 116, 45},
- {{209, 17, 111, 41, 154, 94, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 48, 224},
- {{40, 245, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 22, 17},
- {{72, 121, 151, 83, 170, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 44, 133},
- {{171, 172, 101, 238, 201, 148, 23, 81, 4, 11, 64, 0, 0, 0, 0, 0}, 85, 125},
- {{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 3, 42},
- {{20, 46, 27, 93, 195, 184, 6, 162, 109, 225, 22, 152, 0, 0, 0, 0}, 96, 140},
- {{243, 122, 30, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 28, 91},
- {{89, 250, 80, 72, 148, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 48, 92},
- {{187, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 13, 125},
- {{172, 160, 143, 114, 128, 239, 174, 133, 176, 154, 159, 134, 10, 0, 0, 0}, 106, 249},
- {{254, 202, 113, 112, 173, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 40, 202},
- {{80, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 13, 107},
- {{222, 216, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 13, 124},
- {{219, 138, 253, 12, 188, 197, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 53, 57},
- {{124, 41, 173, 8, 202, 192, 61, 254, 174, 48, 239, 112, 0, 0, 0, 0}, 92, 181},
- {{195, 236, 245, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 24, 107},
- {{83, 82, 42, 244, 136, 191, 197, 81, 91, 154, 216, 85, 29, 150, 198, 22}, 128, 101},
- {{112, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 4, 102},
- {{44, 30, 219, 248, 214, 88, 225, 132, 0, 0, 0, 0, 0, 0, 0, 0}, 62, 136},
- {{41, 171, 206, 178, 195, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 45, 114},
- {{159, 15, 180, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 22, 215},
- {{42, 188, 37, 174, 86, 40, 4, 84, 174, 216, 0, 0, 0, 0, 0, 0}, 79, 249},
- {{185, 227, 85, 177, 219, 95, 250, 227, 69, 154, 118, 0, 0, 0, 0, 0}, 88, 29},
- {{22, 185, 238, 100, 25, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 44, 71},
- {{122, 149, 117, 77, 88, 250, 187, 203, 136, 22, 85, 42, 105, 234, 79, 8}, 127, 112},
- {{93, 152, 229, 140, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 31, 72},
- {{129, 37, 165, 167, 241, 24, 37, 40, 2, 128, 0, 0, 0, 0, 0, 0}, 73, 155},
- {{30, 202, 177, 3, 253, 202, 164, 248, 0, 0, 0, 0, 0, 0, 0, 0}, 61, 66},
- {{176, 25, 220, 120, 194, 228, 10, 45, 225, 142, 192, 96, 0, 0, 0, 0}, 91, 77},
- {{96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 3, 109},
- {{82, 56, 12, 204, 61, 45, 147, 240, 221, 0, 0, 0, 0, 0, 0, 0}, 72, 37},
- {{242, 38, 240, 41, 140, 75, 250, 37, 175, 115, 97, 224, 0, 0, 0, 0}, 91, 56},
- {{251, 192, 23, 90, 135, 56, 252, 56, 79, 219, 80, 167, 22, 0, 0, 0}, 103, 5},
- {{62, 128, 139, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 25, 15},
- {{214, 1, 84, 232, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 29, 183},
- {{207, 90, 237, 137, 171, 140, 227, 88, 250, 26, 197, 162, 163, 0, 0, 0}, 105, 171},
- {{196, 151, 235, 232, 114, 248, 1, 207, 193, 184, 186, 71, 157, 0, 0, 0}, 112, 202},
- {{152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 5, 136},
- {{9, 174, 211, 200, 120, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 37, 107},
- {{89, 150, 95, 28, 209, 13, 125, 159, 254, 244, 110, 0, 0, 0, 0, 0}, 87, 193},
- {{23, 28, 202, 10, 90, 158, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 52, 4},
- {{48, 25, 180, 9, 84, 236, 6, 144, 30, 198, 41, 56, 0, 0, 0, 0}, 96, 68},
- {{252, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 7, 40},
- {{20, 165, 57, 130, 164, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 39, 255},
- {{167, 56, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 18, 108},
- {{91, 204, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 14, 219},
- {{24, 46, 9, 4, 170, 150, 56, 130, 127, 120, 118, 104, 168, 48, 0, 0}, 108, 12},
- {{156, 60, 245, 247, 189, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 40, 84},
- {{148, 104, 187, 174, 129, 28, 127, 162, 92, 222, 52, 18, 0, 0, 0, 0}, 96, 33},
- {{38, 253, 182, 153, 233, 194, 159, 41, 94, 193, 254, 160, 0, 0, 0, 0}, 91, 199},
- {{156, 77, 105, 235, 145, 216, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 46, 52},
- {{100, 211, 238, 147, 65, 222, 99, 73, 252, 113, 46, 113, 52, 136, 0, 0}, 113, 184},
- {{13, 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 18, 124},
- {{29, 240, 141, 230, 78, 237, 25, 135, 131, 6, 65, 77, 77, 248, 0, 0}, 109, 128},
- {{15, 192, 109, 31, 149, 221, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 49, 255},
- {{80, 185, 170, 71, 41, 58, 158, 106, 253, 7, 2, 184, 173, 0, 0, 0}, 105, 146},
- {{16, 229, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 24, 172},
- {{169, 2, 153, 9, 169, 203, 245, 154, 184, 0, 0, 0, 0, 0, 0, 0}, 70, 116},
- {{144, 135, 239, 164, 142, 187, 64, 109, 0, 0, 0, 0, 0, 0, 0, 0}, 66, 189},
- {{170, 78, 252, 227, 242, 199, 130, 251, 200, 0, 0, 0, 0, 0, 0, 0}, 70, 10},
- {{232, 18, 15, 126, 166, 126, 58, 25, 209, 62, 76, 79, 0, 0, 0, 0}, 98, 184},
- {{170, 82, 72, 53, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 33, 98},
- {{152, 100, 37, 122, 242, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 42, 37},
- {{174, 231, 230, 33, 71, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 46, 174},
- {{74, 225, 252, 153, 202, 8, 162, 39, 64, 0, 0, 0, 0, 0, 0, 0}, 67, 251},
- {{167, 186, 101, 187, 122, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 42, 115},
- {{83, 7, 21, 122, 243, 67, 171, 146, 145, 160, 168, 103, 223, 64, 0, 0}, 107, 252},
- {{83, 132, 219, 86, 86, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 40, 176},
- {{22, 113, 72, 102, 73, 16, 236, 57, 197, 122, 31, 0, 0, 0, 0, 0}, 91, 155},
- {{250, 59, 64, 35, 72, 112, 159, 85, 200, 5, 193, 39, 152, 185, 148, 16}, 124, 36},
- {{220, 21, 48, 164, 224, 121, 17, 69, 10, 118, 106, 0, 0, 0, 0, 0}, 88, 202},
- {{160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 5, 208},
- {{247, 64, 83, 125, 195, 225, 50, 76, 18, 104, 0, 0, 0, 0, 0, 0}, 77, 158},
- {{78, 91, 31, 202, 189, 25, 13, 133, 220, 0, 0, 0, 0, 0, 0, 0}, 72, 136},
- {{105, 197, 26, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 28, 191},
- {{14, 31, 154, 242, 241, 231, 55, 151, 223, 56, 134, 255, 113, 206, 69, 0}, 120, 126},
- {{247, 193, 58, 176, 16, 71, 31, 120, 213, 104, 231, 83, 26, 118, 91, 135}, 128, 139},
- {{136, 32, 240, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 25, 216},
- {{100, 238, 112, 40, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 29, 93},
- {{80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 4, 196},
- {{233, 224, 254, 57, 33, 205, 140, 217, 181, 72, 0, 0, 0, 0, 0, 0}, 81, 119},
- {{107, 75, 65, 158, 128, 142, 191, 188, 188, 240, 148, 243, 116, 0, 0, 0}, 104, 93},
- {{39, 70, 120, 114, 69, 237, 95, 48, 233, 176, 91, 154, 0, 0, 0, 0}, 96, 183},
- {{10, 61, 43, 101, 64, 102, 208, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 52, 207},
- {{151, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 9, 102},
- {{210, 241, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 19, 36},
- {{52, 222, 249, 31, 108, 137, 199, 1, 242, 173, 184, 144, 0, 0, 0, 0}, 93, 41},
- {{123, 111, 88, 192, 69, 70, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 47, 70},
- {{180, 82, 188, 125, 140, 8, 196, 74, 0, 0, 0, 0, 0, 0, 0, 0}, 63, 218},
- {{77, 158, 34, 101, 196, 102, 56, 220, 42, 143, 181, 187, 240, 64, 161, 0}, 120, 226},
- {{88, 220, 222, 38, 23, 108, 5, 148, 185, 110, 20, 14, 67, 61, 0, 0}, 114, 25},
- {{90, 65, 220, 165, 197, 133, 110, 92, 228, 19, 2, 17, 0, 0, 0, 0}, 98, 6},
- {{35, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 11, 26},
- {{103, 123, 49, 209, 228, 229, 144, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 52, 149},
- {{50, 244, 58, 191, 95, 156, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 46, 127},
- {{140, 169, 75, 77, 78, 86, 40, 16, 0, 0, 0, 0, 0, 0, 0, 0}, 62, 144},
- {{99, 176, 175, 83, 114, 50, 214, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 56, 213},
- {{19, 208, 211, 76, 85, 176, 247, 64, 0, 0, 0, 0, 0, 0, 0, 0}, 58, 115},
- {{153, 28, 188, 113, 211, 116, 7, 178, 136, 205, 96, 0, 0, 0, 0, 0}, 83, 146},
- {{160, 180, 220, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 26, 58},
- {{234, 6, 112, 19, 61, 74, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 47, 222},
- {{97, 110, 34, 117, 149, 148, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 50, 16},
- {{99, 173, 119, 73, 250, 30, 144, 30, 128, 0, 0, 0, 0, 0, 0, 0}, 65, 169},
- {{169, 134, 111, 89, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 40, 175},
- {{134, 80, 227, 43, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 32, 3},
- {{231, 243, 35, 80, 75, 207, 128, 137, 54, 170, 71, 238, 0, 0, 0, 0}, 96, 2},
- {{189, 190, 121, 135, 160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 36, 193},
- {{143, 155, 216, 193, 239, 205, 204, 153, 143, 236, 69, 23, 200, 211, 0, 0}, 118, 151},
- {{32, 1, 115, 244, 33, 219, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 51, 182},
- {{220, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 10, 148},
- {{206, 87, 135, 235, 116, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 42, 53},
- {{152, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 11, 87},
- {{58, 146, 188, 233, 230, 236, 192, 214, 168, 128, 0, 0, 0, 0, 0, 0}, 73, 235},
- {{84, 220, 82, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 23, 51},
- {{106, 145, 142, 42, 186, 186, 58, 1, 48, 98, 165, 131, 48, 156, 192, 0}, 116, 11},
- {{53, 219, 120, 242, 166, 214, 81, 130, 64, 0, 0, 0, 0, 0, 0, 0}, 68, 28},
- {{240, 120, 76, 163, 32, 197, 181, 251, 98, 220, 29, 226, 0, 0, 0, 0}, 96, 73},
- {{234, 197, 12, 160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 28, 216},
- {{191, 94, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 16, 99},
- {{200, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 18, 35},
- {{29, 129, 47, 83, 19, 75, 158, 1, 28, 24, 26, 147, 82, 119, 140, 100}, 127, 195},
- {{241, 174, 26, 53, 152, 112, 200, 134, 84, 187, 177, 176, 42, 64, 0, 0}, 108, 176},
- {{77, 171, 145, 48, 195, 84, 190, 36, 122, 199, 18, 0, 0, 0, 0, 0}, 87, 217},
- {{105, 104, 135, 53, 226, 118, 238, 169, 9, 253, 132, 162, 217, 123, 191, 96}, 126, 244},
- {{160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 3, 125},
- {{41, 85, 143, 128, 91, 137, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 50, 219},
- {{116, 110, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 18, 165},
- {{75, 213, 44, 16, 43, 157, 34, 171, 98, 117, 109, 151, 5, 60, 224, 0}, 117, 6},
- {{229, 23, 116, 61, 80, 139, 200, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 53, 47},
- {{83, 123, 74, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 23, 73},
- {{151, 243, 45, 217, 216, 158, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 47, 98},
- {{171, 184, 110, 211, 237, 114, 144, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 52, 21},
- {{7, 246, 199, 119, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 32, 142},
- {{103, 47, 70, 17, 31, 232, 44, 75, 145, 155, 100, 216, 0, 0, 0, 0}, 93, 34},
- {{65, 170, 169, 100, 167, 147, 142, 251, 20, 64, 0, 0, 0, 0, 0, 0}, 74, 41},
- {{235, 6, 229, 248, 151, 137, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 55, 80},
- {{156, 39, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 22, 11},
- {{92, 188, 82, 192, 142, 249, 190, 128, 0, 0, 0, 0, 0, 0, 0, 0}, 58, 254},
- {{253, 218, 181, 46, 134, 144, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 45, 95},
- {{189, 19, 31, 244, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 40, 8},
- {{30, 116, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 14, 212},
- {{81, 226, 13, 173, 79, 123, 223, 124, 108, 80, 83, 238, 0, 0, 0, 0}, 95, 217},
- {{126, 211, 206, 82, 147, 215, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 57, 15},
- {{42, 229, 135, 197, 196, 243, 94, 181, 133, 34, 16, 0, 0, 0, 0, 0}, 84, 66},
- {{68, 210, 158, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 28, 122},
- {{183, 63, 223, 94, 81, 41, 203, 20, 236, 212, 220, 199, 0, 0, 0, 0}, 97, 12},
- {{131, 146, 2, 125, 174, 43, 231, 20, 194, 0, 0, 0, 0, 0, 0, 0}, 71, 171},
- {{31, 180, 246, 158, 28, 192, 236, 39, 237, 55, 74, 195, 171, 192, 0, 0}, 106, 42},
- {{179, 10, 70, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 28, 194},
- {{147, 51, 85, 185, 234, 209, 236, 87, 147, 17, 7, 68, 148, 32, 0, 0}, 107, 237},
- {{177, 178, 6, 40, 46, 166, 87, 198, 214, 234, 23, 224, 0, 0, 0, 0}, 93, 151},
- {{201, 53, 40, 20, 49, 4, 38, 139, 133, 217, 214, 134, 89, 200, 0, 0}, 109, 238},
- {{4, 26, 181, 37, 206, 129, 233, 32, 0, 0, 0, 0, 0, 0, 0, 0}, 59, 128},
- {{81, 58, 248, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 26, 227},
- {{18, 238, 250, 161, 57, 246, 208, 118, 14, 76, 73, 25, 65, 22, 152, 120}, 127, 138},
- {{31, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 10, 60},
- {{115, 195, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 18, 148},
- {{116, 22, 75, 33, 16, 129, 35, 124, 10, 112, 31, 213, 181, 108, 177, 46}, 128, 129},
- {{117, 214, 20, 80, 83, 51, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 49, 202},
- {{120, 75, 124, 149, 120, 123, 242, 151, 181, 164, 128, 0, 0, 0, 0, 0}, 81, 88},
- {{87, 238, 168, 62, 88, 166, 52, 104, 219, 169, 93, 128, 0, 0, 0, 0}, 90, 3},
- {{237, 44, 224, 146, 52, 85, 245, 192, 65, 137, 37, 95, 156, 176, 0, 0}, 108, 243},
- {{214, 241, 51, 63, 73, 61, 193, 165, 23, 108, 0, 0, 0, 0, 0, 0}, 80, 95},
- {{87, 242, 21, 157, 45, 188, 36, 62, 66, 243, 64, 0, 0, 0, 0, 0}, 87, 255},
- {{0, 97, 220, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 26, 48},
- {{227, 206, 189, 31, 222, 8, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 50, 38},
- {{174, 27, 0, 16, 13, 150, 33, 122, 154, 59, 236, 35, 248, 178, 64, 0}, 115, 20},
- {{39, 20, 125, 69, 252, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 43, 41},
- {{141, 232, 1, 12, 125, 229, 168, 14, 125, 116, 180, 0, 0, 0, 0, 0}, 92, 133},
- {{93, 238, 40, 228, 254, 203, 251, 6, 60, 82, 243, 242, 0, 0, 0, 0}, 95, 189},
- {{44, 115, 200, 17, 146, 223, 115, 253, 126, 206, 152, 90, 0, 0, 0, 0}, 95, 151},
- {{213, 58, 235, 255, 6, 163, 61, 10, 224, 0, 0, 0, 0, 0, 0, 0}, 68, 100},
- {{25, 86, 139, 116, 190, 88, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 49, 118},
- {{113, 40, 65, 141, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 34, 164},
- {{149, 205, 200, 186, 19, 126, 215, 199, 94, 37, 100, 32, 128, 0, 0, 0}, 98, 71},
- {{39, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 9, 251},
- {{81, 87, 80, 173, 163, 166, 104, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 57, 51},
- {{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 3, 185},
- {{140, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 10, 144},
- {{6, 42, 1, 178, 250, 53, 186, 178, 114, 121, 192, 0, 0, 0, 0, 0}, 84, 51},
- {{2, 17, 234, 51, 169, 5, 219, 149, 245, 237, 4, 0, 0, 0, 0, 0}, 87, 32},
- {{112, 187, 173, 17, 229, 171, 225, 170, 8, 0, 0, 0, 0, 0, 0, 0}, 70, 137},
- {{203, 71, 140, 237, 113, 96, 123, 16, 0, 0, 0, 0, 0, 0, 0, 0}, 60, 2},
- {{99, 138, 207, 2, 244, 25, 211, 98, 0, 0, 0, 0, 0, 0, 0, 0}, 63, 163},
- {{114, 42, 98, 246, 252, 48, 233, 118, 63, 226, 157, 226, 192, 0, 0, 0}, 100, 162},
- {{161, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 10, 192},
- {{233, 70, 240, 45, 240, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 36, 185},
- {{28, 123, 31, 176, 235, 229, 169, 192, 0, 0, 0, 0, 0, 0, 0, 0}, 59, 51},
- {{146, 197, 243, 235, 243, 56, 140, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 54, 93},
- {{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 2, 159},
- {{141, 92, 13, 27, 87, 241, 171, 143, 220, 0, 0, 0, 0, 0, 0, 0}, 72, 189},
- {{164, 151, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 21, 248},
- {{35, 188, 248, 79, 39, 151, 232, 215, 248, 245, 185, 144, 78, 102, 173, 128}, 123, 38},
- {{193, 232, 166, 60, 62, 80, 230, 225, 165, 240, 0, 0, 0, 0, 0, 0}, 76, 167},
- {{109, 229, 118, 155, 43, 154, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 51, 28},
- {{160, 62, 63, 212, 218, 138, 154, 108, 163, 127, 197, 237, 183, 44, 140, 192}, 125, 37},
- {{196, 37, 51, 146, 26, 85, 53, 31, 216, 141, 52, 218, 153, 32, 0, 0}, 107, 234},
- {{228, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 9, 70},
- {{154, 248, 20, 242, 154, 244, 63, 17, 121, 52, 70, 84, 118, 208, 0, 0}, 108, 50},
- {{41, 100, 27, 84, 106, 112, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 51, 171},
- {{81, 99, 197, 139, 30, 150, 230, 216, 81, 190, 84, 165, 29, 64, 128, 0}, 113, 236},
- {{112, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 4, 3},
- {{164, 119, 253, 126, 160, 249, 183, 191, 119, 111, 224, 0, 0, 0, 0, 0}, 86, 64},
- {{138, 58, 198, 254, 0, 197, 60, 91, 132, 199, 181, 251, 78, 160, 0, 0}, 108, 213},
- {{209, 89, 168, 236, 146, 169, 100, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 54, 15},
- {{131, 210, 208, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 20, 145},
- {{165, 190, 157, 7, 131, 5, 147, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 57, 27},
- {{179, 226, 57, 204, 187, 70, 52, 81, 119, 162, 229, 42, 47, 185, 9, 162}, 127, 75},
- {{98, 235, 155, 51, 107, 167, 127, 137, 254, 246, 162, 171, 180, 13, 233, 0}, 123, 76},
- {{107, 79, 76, 90, 94, 151, 155, 31, 33, 115, 19, 204, 98, 115, 0, 0}, 113, 247},
- {{143, 46, 30, 175, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 43, 121},
- {{155, 85, 217, 180, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 30, 214},
- {{58, 62, 156, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 26, 221},
- {{92, 155, 53, 3, 39, 108, 155, 200, 0, 0, 0, 0, 0, 0, 0, 0}, 63, 102},
- {{64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 2, 191},
- {{63, 134, 251, 59, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, 39, 197},
- {{234, 149, 220, 106, 0, 144, 214, 128, 35, 102, 0, 0, 0, 0, 0, 0}, 79, 106},
+{{.a = {66, 70, 154, 143, 197, 233, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 50, 146},
+{{.a = {107, 79, 18, 235, 142, 84, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 54, 141},
+{{.a = {247, 132, 113, 1, 215, 247, 183, 239, 128, 0, 0, 0, 0, 0, 0, 0}}, 67, 23},
+{{.a = {48, 19, 41, 12, 76, 101, 114, 160, 45, 103, 134, 146, 128, 0, 0, 0}}, 97, 252},
+{{.a = {5, 70, 208, 170, 19, 0, 116, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 54, 6},
+{{.a = {1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 9, 137},
+{{.a = {12, 188, 26, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 31, 9},
+{{.a = {1, 235, 101, 202, 26, 92, 23, 22, 179, 223, 128, 0, 0, 0, 0, 0}}, 82, 9},
+{{.a = {215, 19, 224, 102, 45, 133, 102, 249, 56, 20, 214, 219, 93, 125, 52, 0}}, 120, 163},
+{{.a = {178, 183, 109, 64, 136, 84, 11, 53, 217, 102, 0, 0, 0, 0, 0, 0}}, 79, 197},
+{{.a = {212, 39, 158, 71, 253, 98, 248, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 54, 249},
+{{.a = {92, 58, 159, 130, 105, 56, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 47, 88},
+{{.a = {118, 140, 65, 198, 212, 93, 144, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 52, 104},
+{{.a = {86, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 10, 36},
+{{.a = {79, 135, 242, 193, 197, 11, 200, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 54, 239},
+{{.a = {163, 228, 239, 80, 41, 66, 176, 176, 0, 0, 0, 0, 0, 0, 0, 0}}, 67, 201},
+{{.a = {31, 9, 231, 42, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 33, 94},
+{{.a = {108, 144, 205, 39, 215, 26, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 51, 241},
+{{.a = {247, 217, 172, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 24, 239},
+{{.a = {24, 186, 73, 182, 240, 251, 125, 165, 0, 0, 0, 0, 0, 0, 0, 0}}, 66, 151},
+{{.a = {245, 112, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 12, 137},
+{{.a = {44, 94, 138, 224, 168, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 41, 231},
+{{.a = {184, 221, 109, 135, 225, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 44, 11},
+{{.a = {51, 179, 136, 184, 30, 118, 24, 16, 26, 161, 206, 101, 0, 0, 0, 0}}, 96, 20},
+{{.a = {48, 46, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 15, 68},
+{{.a = {143, 235, 237, 220, 89, 119, 187, 143, 209, 94, 46, 58, 120, 0, 0, 0}}, 101, 64},
+{{.a = {121, 190, 90, 177, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 33, 152},
+{{.a = {128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 6, 217},
+{{.a = {128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 2, 101},
+{{.a = {111, 214, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 15, 58},
+{{.a = {162, 23, 52, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 27, 254},
+{{.a = {76, 103, 44, 79, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 32, 148},
+{{.a = {80, 85, 219, 214, 12, 4, 65, 129, 162, 148, 208, 78, 39, 69, 94, 184}}, 126, 126},
+{{.a = {80, 54, 251, 28, 152, 23, 244, 192, 151, 83, 6, 144, 223, 213, 224, 128}}, 123, 76},
+{{.a = {39, 232, 237, 103, 191, 188, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 54, 240},
+{{.a = {20, 231, 89, 210, 167, 173, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 54, 33},
+{{.a = {125, 67, 198, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 25, 47},
+{{.a = {26, 239, 153, 5, 213, 121, 31, 114, 161, 46, 84, 15, 148, 160, 0, 0}}, 109, 41},
+{{.a = {102, 212, 159, 118, 223, 115, 134, 172, 0, 0, 0, 0, 0, 0, 0, 0}}, 62, 72},
+{{.a = {85, 181, 241, 127, 3, 44, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 46, 43},
+{{.a = {61, 199, 131, 226, 3, 230, 94, 119, 240, 0, 0, 0, 0, 0, 0, 0}}, 68, 26},
+{{.a = {0, 143, 160, 184, 162, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 42, 139},
+{{.a = {170, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 13, 219},
+{{.a = {61, 122, 24, 251, 124, 122, 202, 192, 0, 0, 0, 0, 0, 0, 0, 0}}, 58, 105},
+{{.a = {33, 219, 226, 3, 180, 190, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 47, 210},
+{{.a = {51, 251, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 17, 151},
+{{.a = {106, 185, 11, 122, 197, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 42, 28},
+{{.a = {192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 9, 64},
+{{.a = {239, 195, 77, 239, 131, 156, 2, 246, 191, 178, 204, 160, 21, 213, 30, 128}}, 121, 9},
+{{.a = {141, 207, 181, 99, 55, 245, 151, 228, 65, 50, 85, 16, 0, 0, 0, 0}}, 92, 250},
+{{.a = {110, 159, 230, 251, 224, 210, 58, 49, 0, 0, 0, 0, 0, 0, 0, 0}}, 66, 200},
+{{.a = {134, 26, 104, 32, 129, 41, 201, 50, 164, 69, 178, 156, 156, 133, 8, 218}}, 127, 132},
+{{.a = {253, 207, 116, 105, 210, 166, 186, 99, 182, 0, 0, 0, 0, 0, 0, 0}}, 71, 182},
+{{.a = {211, 73, 38, 80, 183, 168, 52, 138, 25, 214, 112, 8, 252, 0, 0, 0}}, 102, 7},
+{{.a = {200, 244, 108, 238, 164, 141, 215, 39, 233, 249, 120, 80, 112, 0, 0, 0}}, 100, 146},
+{{.a = {107, 44, 250, 202, 64, 37, 107, 105, 140, 0, 0, 0, 0, 0, 0, 0}}, 70, 98},
+{{.a = {93, 86, 56, 27, 159, 195, 126, 39, 240, 201, 48, 0, 0, 0, 0, 0}}, 86, 179},
+{{.a = {32, 202, 214, 242, 39, 141, 61, 146, 138, 96, 0, 0, 0, 0, 0, 0}}, 77, 245},
+{{.a = {167, 77, 249, 28, 210, 196, 227, 241, 0, 0, 0, 0, 0, 0, 0, 0}}, 64, 2},
+{{.a = {241, 59, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 17, 5},
+{{.a = {143, 68, 146, 210, 173, 155, 251, 173, 0, 0, 0, 0, 0, 0, 0, 0}}, 66, 169},
+{{.a = {167, 180, 226, 144, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 33, 52},
+{{.a = {241, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 9, 177},
+{{.a = {238, 9, 168, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 27, 74},
+{{.a = {203, 148, 16, 96, 125, 18, 86, 1, 91, 244, 251, 20, 31, 14, 75, 128}}, 122, 212},
+{{.a = {111, 227, 137, 94, 65, 21, 77, 137, 119, 130, 159, 19, 159, 45, 18, 192}}, 122, 238},
+{{.a = {59, 144, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 19, 18},
+{{.a = {110, 192, 255, 120, 84, 215, 3, 130, 38, 224, 0, 0, 0, 0, 0, 0}}, 75, 155},
+{{.a = {152, 79, 219, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 24, 97},
+{{.a = {118, 186, 157, 248, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 32, 8},
+{{.a = {70, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 9, 123},
+{{.a = {253, 119, 114, 227, 18, 243, 81, 61, 238, 107, 190, 144, 0, 0, 0, 0}}, 92, 11},
+{{.a = {166, 112, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 13, 211},
+{{.a = {43, 95, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 20, 116},
+{{.a = {94, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 11, 57},
+{{.a = {182, 251, 195, 132, 66, 7, 208, 146, 223, 231, 211, 181, 25, 176, 0, 0}}, 108, 178},
+{{.a = {152, 166, 111, 233, 194, 17, 230, 41, 221, 253, 69, 123, 108, 0, 0, 0}}, 102, 93},
+{{.a = {106, 141, 235, 190, 82, 241, 152, 186, 195, 81, 86, 144, 0, 0, 0, 0}}, 92, 3},
+{{.a = {32, 81, 210, 153, 151, 29, 11, 62, 127, 177, 194, 254, 103, 83, 58, 128}}, 121, 162},
+{{.a = {79, 112, 224, 26, 174, 39, 98, 181, 115, 57, 209, 189, 136, 48, 0, 0}}, 109, 125},
+{{.a = {106, 197, 83, 151, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 34, 33},
+{{.a = {190, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 9, 254},
+{{.a = {156, 73, 249, 148, 55, 192, 20, 42, 142, 128, 0, 0, 0, 0, 0, 0}}, 74, 66},
+{{.a = {64, 107, 36, 120, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 30, 4},
+{{.a = {115, 148, 71, 250, 158, 174, 168, 249, 106, 110, 196, 0, 0, 0, 0, 0}}, 86, 122},
+{{.a = {18, 139, 152, 44, 38, 88, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 46, 59},
+{{.a = {55, 229, 117, 106, 146, 95, 74, 220, 122, 0, 84, 202, 183, 138, 120, 0}}, 117, 99},
+{{.a = {153, 211, 3, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 27, 41},
+{{.a = {1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 8, 112},
+{{.a = {49, 192, 102, 142, 216, 3, 114, 64, 165, 128, 168, 0, 0, 0, 0, 0}}, 85, 255},
+{{.a = {201, 143, 240, 240, 209, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 44, 106},
+{{.a = {158, 19, 164, 196, 87, 162, 33, 120, 0, 0, 0, 0, 0, 0, 0, 0}}, 62, 170},
+{{.a = {5, 114, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 16, 86},
+{{.a = {34, 170, 246, 62, 198, 85, 193, 227, 252, 68, 0, 0, 0, 0, 0, 0}}, 79, 155},
+{{.a = {21, 52, 9, 86, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 35, 65},
+{{.a = {203, 81, 49, 171, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 36, 39},
+{{.a = {211, 218, 87, 244, 93, 181, 118, 41, 156, 143, 254, 0, 0, 0, 0, 0}}, 90, 162},
+{{.a = {77, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 10, 69},
+{{.a = {158, 219, 219, 39, 4, 219, 100, 63, 0, 0, 0, 0, 0, 0, 0, 0}}, 64, 163},
+{{.a = {61, 50, 232, 1, 185, 252, 243, 54, 189, 240, 170, 192, 0, 0, 0, 0}}, 90, 116},
+{{.a = {241, 143, 33, 19, 247, 55, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 53, 19},
+{{.a = {61, 28, 61, 252, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 36, 48},
+{{.a = {102, 112, 194, 108, 90, 253, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 49, 230},
+{{.a = {74, 88, 58, 66, 172, 41, 144, 204, 195, 240, 0, 0, 0, 0, 0, 0}}, 78, 155},
+{{.a = {44, 148, 187, 58, 190, 59, 190, 187, 124, 138, 222, 131, 0, 0, 0, 0}}, 96, 158},
+{{.a = {67, 7, 216, 139, 93, 224, 20, 135, 186, 86, 209, 111, 60, 80, 0, 0}}, 113, 252},
+{{.a = {209, 26, 12, 174, 5, 101, 164, 181, 237, 63, 192, 57, 54, 120, 0, 0}}, 110, 176},
+{{.a = {4, 66, 232, 52, 239, 56, 48, 58, 192, 0, 0, 0, 0, 0, 0, 0}}, 66, 211},
+{{.a = {158, 165, 2, 144, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 28, 15},
+{{.a = {85, 204, 245, 198, 68, 44, 39, 71, 32, 0, 0, 0, 0, 0, 0, 0}}, 68, 95},
+{{.a = {181, 134, 25, 87, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 34, 169},
+{{.a = {26, 230, 61, 36, 79, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 44, 249},
+{{.a = {5, 170, 198, 139, 65, 186, 188, 45, 42, 253, 165, 89, 206, 0, 0, 0}}, 105, 61},
+{{.a = {211, 245, 62, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 23, 63},
+{{.a = {117, 68, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 14, 43},
+{{.a = {103, 17, 123, 102, 70, 206, 90, 92, 124, 198, 0, 0, 0, 0, 0, 0}}, 81, 228},
+{{.a = {192, 237, 88, 244, 53, 30, 61, 160, 143, 64, 0, 0, 0, 0, 0, 0}}, 78, 165},
+{{.a = {199, 82, 217, 183, 2, 179, 195, 6, 0, 0, 0, 0, 0, 0, 0, 0}}, 64, 3},
+{{.a = {157, 230, 79, 162, 57, 125, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 57, 211},
+{{.a = {27, 67, 64, 235, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 33, 210},
+{{.a = {72, 158, 163, 106, 193, 137, 190, 7, 250, 165, 249, 73, 64, 0, 0, 0}}, 99, 61},
+{{.a = {34, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 10, 120},
+{{.a = {215, 141, 95, 192, 189, 62, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 47, 94},
+{{.a = {31, 181, 56, 141, 120, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 41, 153},
+{{.a = {153, 73, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 18, 221},
+{{.a = {162, 107, 41, 189, 165, 155, 22, 139, 165, 72, 96, 0, 0, 0, 0, 0}}, 87, 163},
+{{.a = {218, 17, 204, 165, 217, 251, 107, 45, 29, 15, 192, 167, 75, 0, 0, 0}}, 106, 188},
+{{.a = {200, 124, 238, 213, 35, 228, 94, 141, 86, 187, 101, 60, 115, 52, 131, 16}}, 124, 15},
+{{.a = {74, 237, 160, 56, 141, 217, 191, 16, 0, 0, 0, 0, 0, 0, 0, 0}}, 63, 28},
+{{.a = {163, 47, 242, 103, 173, 217, 88, 154, 38, 200, 32, 0, 0, 0, 0, 0}}, 84, 240},
+{{.a = {20, 227, 128, 28, 144, 147, 22, 13, 94, 129, 107, 88, 0, 0, 0, 0}}, 93, 59},
+{{.a = {95, 144, 229, 107, 218, 125, 204, 233, 161, 42, 180, 64, 0, 0, 0, 0}}, 90, 195},
+{{.a = {155, 220, 83, 208, 108, 16, 134, 156, 128, 0, 0, 0, 0, 0, 0, 0}}, 66, 10},
+{{.a = {179, 138, 55, 80, 190, 153, 12, 237, 22, 120, 69, 0, 0, 0, 0, 0}}, 88, 206},
+{{.a = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 2, 137},
+{{.a = {3, 119, 148, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 22, 225},
+{{.a = {13, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 10, 223},
+{{.a = {117, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 15, 29},
+{{.a = {164, 19, 195, 47, 136, 190, 156, 255, 30, 74, 143, 134, 162, 0, 0, 0}}, 103, 166},
+{{.a = {40, 235, 94, 135, 135, 230, 71, 33, 64, 233, 0, 0, 0, 0, 0, 0}}, 80, 178},
+{{.a = {222, 151, 166, 97, 129, 250, 140, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 55, 38},
+{{.a = {174, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 9, 141},
+{{.a = {6, 189, 100, 150, 250, 13, 46, 98, 228, 139, 50, 52, 52, 196, 128, 0}}, 116, 230},
+{{.a = {75, 252, 89, 205, 37, 52, 106, 79, 188, 120, 54, 119, 160, 0, 0, 0}}, 99, 124},
+{{.a = {38, 18, 146, 6, 63, 64, 231, 10, 152, 199, 5, 143, 147, 4, 252, 0}}, 118, 54},
+{{.a = {111, 119, 169, 51, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 32, 162},
+{{.a = {105, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 13, 32},
+{{.a = {143, 57, 57, 101, 98, 182, 74, 227, 205, 143, 253, 237, 8, 0, 0, 0}}, 102, 237},
+{{.a = {30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 7, 215},
+{{.a = {14, 232, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 22, 138},
+{{.a = {14, 53, 67, 216, 229, 155, 149, 139, 31, 253, 184, 126, 133, 108, 40, 0}}, 118, 73},
+{{.a = {22, 58, 40, 143, 188, 132, 239, 14, 181, 252, 81, 192, 0, 0, 0, 0}}, 90, 43},
+{{.a = {11, 222, 185, 243, 248, 150, 79, 230, 214, 213, 3, 23, 193, 196, 0, 0}}, 112, 88},
+{{.a = {14, 226, 198, 117, 84, 93, 22, 96, 77, 241, 173, 68, 68, 204, 72, 0}}, 119, 91},
+{{.a = {15, 103, 247, 219, 150, 142, 92, 50, 144, 0, 0, 0, 0, 0, 0, 0}}, 69, 140},
+{{.a = {0, 213, 77, 244, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 37, 65},
+{{.a = {178, 174, 174, 239, 72, 181, 36, 217, 40, 169, 12, 104, 149, 157, 125, 128}}, 122, 201},
+{{.a = {118, 53, 55, 17, 97, 227, 243, 176, 2, 0, 0, 0, 0, 0, 0, 0}}, 72, 69},
+{{.a = {21, 253, 4, 47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 35, 170},
+{{.a = {5, 249, 186, 133, 68, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 40, 192},
+{{.a = {47, 79, 35, 66, 11, 178, 161, 28, 87, 180, 45, 128, 0, 0, 0, 0}}, 89, 21},
+{{.a = {242, 227, 20, 73, 150, 196, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 46, 35},
+{{.a = {121, 169, 102, 118, 157, 192, 154, 186, 126, 0, 0, 0, 0, 0, 0, 0}}, 71, 235},
+{{.a = {9, 138, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 21, 240},
+{{.a = {45, 173, 14, 72, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 30, 136},
+{{.a = {127, 47, 51, 201, 236, 45, 142, 80, 0, 0, 0, 0, 0, 0, 0, 0}}, 60, 186},
+{{.a = {247, 233, 34, 38, 181, 207, 127, 20, 224, 118, 59, 148, 0, 0, 0, 0}}, 95, 174},
+{{.a = {126, 187, 198, 104, 245, 223, 219, 18, 31, 124, 0, 0, 0, 0, 0, 0}}, 79, 153},
+{{.a = {3, 163, 107, 228, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 35, 118},
+{{.a = {167, 109, 2, 95, 11, 62, 45, 128, 0, 0, 0, 0, 0, 0, 0, 0}}, 60, 113},
+{{.a = {76, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 6, 58},
+{{.a = {58, 190, 204, 151, 222, 147, 47, 78, 38, 203, 9, 17, 64, 0, 0, 0}}, 101, 206},
+{{.a = {254, 220, 254, 220, 204, 79, 35, 127, 242, 63, 106, 232, 127, 180, 0, 0}}, 111, 42},
+{{.a = {77, 156, 8, 209, 181, 37, 70, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 55, 230},
+{{.a = {65, 89, 137, 76, 208, 199, 166, 90, 128, 0, 0, 0, 0, 0, 0, 0}}, 67, 6},
+{{.a = {47, 232, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 13, 254},
+{{.a = {172, 154, 12, 108, 77, 37, 106, 8, 234, 7, 248, 212, 112, 160, 0, 0}}, 108, 214},
+{{.a = {254, 117, 239, 244, 154, 89, 166, 241, 12, 108, 127, 153, 206, 160, 0, 0}}, 107, 43},
+{{.a = {113, 160, 206, 52, 143, 12, 9, 148, 224, 0, 0, 0, 0, 0, 0, 0}}, 67, 178},
+{{.a = {178, 130, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 16, 179},
+{{.a = {229, 177, 28, 106, 59, 75, 182, 241, 36, 79, 224, 0, 0, 0, 0, 0}}, 87, 236},
+{{.a = {156, 72, 93, 193, 50, 235, 75, 228, 88, 115, 89, 119, 128, 0, 0, 0}}, 98, 184},
+{{.a = {28, 232, 28, 249, 83, 105, 211, 7, 136, 147, 231, 64, 0, 0, 0, 0}}, 91, 95},
+{{.a = {217, 33, 23, 107, 74, 42, 135, 197, 144, 34, 40, 243, 13, 126, 36, 136}}, 127, 152},
+{{.a = {64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 2, 113},
+{{.a = {85, 172, 121, 126, 213, 57, 225, 54, 197, 73, 85, 251, 9, 64, 0, 0}}, 108, 137},
+{{.a = {104, 46, 25, 71, 86, 220, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 46, 224},
+{{.a = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 6, 61},
+{{.a = {241, 113, 254, 106, 53, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 41, 205},
+{{.a = {29, 36, 12, 244, 197, 127, 240, 8, 167, 134, 154, 248, 199, 123, 143, 240}}, 124, 170},
+{{.a = {58, 29, 129, 94, 43, 139, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 53, 117},
+{{.a = {213, 124, 147, 196, 7, 82, 67, 70, 228, 0, 0, 0, 0, 0, 0, 0}}, 70, 225},
+{{.a = {164, 168, 161, 140, 87, 85, 250, 41, 34, 0, 0, 0, 0, 0, 0, 0}}, 72, 34},
+{{.a = {186, 142, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 17, 5},
+{{.a = {237, 249, 9, 70, 247, 97, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 50, 92},
+{{.a = {155, 92, 145, 218, 125, 226, 226, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 55, 230},
+{{.a = {35, 169, 62, 156, 86, 4, 125, 219, 119, 113, 191, 75, 198, 113, 0, 0}}, 112, 61},
+{{.a = {207, 63, 96, 186, 26, 68, 115, 161, 163, 59, 190, 166, 18, 78, 232, 0}}, 117, 221},
+{{.a = {86, 40, 200, 199, 247, 86, 159, 179, 191, 184, 117, 173, 211, 158, 0, 128}}, 121, 105},
+{{.a = {104, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 11, 181},
+{{.a = {205, 35, 123, 178, 36, 64, 62, 153, 195, 250, 0, 0, 0, 0, 0, 0}}, 79, 110},
+{{.a = {117, 40, 57, 157, 138, 160, 223, 59, 155, 145, 64, 0, 0, 0, 0, 0}}, 86, 103},
+{{.a = {74, 166, 140, 146, 74, 72, 229, 99, 167, 124, 107, 117, 217, 14, 246, 64}}, 123, 218},
+{{.a = {12, 222, 244, 183, 83, 146, 42, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 56, 146},
+{{.a = {11, 98, 146, 110, 95, 96, 80, 142, 224, 0, 0, 0, 0, 0, 0, 0}}, 67, 90},
+{{.a = {235, 5, 187, 199, 30, 170, 82, 187, 228, 159, 22, 25, 204, 112, 0, 0}}, 108, 197},
+{{.a = {35, 96, 146, 145, 155, 116, 252, 181, 29, 205, 230, 246, 30, 0, 0, 0}}, 103, 158},
+{{.a = {174, 38, 56, 244, 227, 102, 252, 237, 128, 86, 0, 0, 0, 0, 0, 0}}, 81, 118},
+{{.a = {65, 134, 37, 58, 90, 125, 60, 84, 0, 0, 0, 0, 0, 0, 0, 0}}, 62, 95},
+{{.a = {253, 117, 135, 98, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 33, 152},
+{{.a = {111, 115, 188, 184, 130, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 45, 239},
+{{.a = {202, 24, 89, 9, 149, 45, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 50, 48},
+{{.a = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 5, 228},
+{{.a = {244, 98, 52, 140, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 30, 247},
+{{.a = {151, 167, 43, 178, 116, 194, 173, 126, 236, 98, 40, 0, 0, 0, 0, 0}}, 85, 12},
+{{.a = {60, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 10, 129},
+{{.a = {208, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 4, 50},
+{{.a = {126, 11, 216, 242, 7, 45, 121, 208, 110, 135, 210, 75, 59, 182, 228, 42}}, 128, 250},
+{{.a = {217, 26, 184, 146, 3, 18, 240, 15, 135, 8, 0, 0, 0, 0, 0, 0}}, 77, 249},
+{{.a = {128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 1, 230},
+{{.a = {145, 28, 29, 184, 2, 85, 234, 135, 98, 111, 136, 32, 0, 0, 0, 0}}, 92, 228},
+{{.a = {108, 104, 255, 254, 34, 95, 72, 157, 0, 0, 0, 0, 0, 0, 0, 0}}, 64, 181},
+{{.a = {153, 61, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 16, 206},
+{{.a = {22, 250, 130, 201, 132, 248, 189, 108, 0, 0, 0, 0, 0, 0, 0, 0}}, 63, 122},
+{{.a = {158, 165, 234, 18, 44, 61, 82, 61, 235, 0, 0, 0, 0, 0, 0, 0}}, 72, 81},
+{{.a = {236, 57, 124, 110, 124, 218, 82, 70, 142, 78, 18, 128, 0, 0, 0, 0}}, 95, 175},
+{{.a = {94, 209, 200, 201, 149, 162, 248, 134, 239, 226, 1, 237, 16, 134, 56, 0}}, 118, 170},
+{{.a = {187, 42, 31, 144, 236, 70, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 47, 174},
+{{.a = {90, 214, 185, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 29, 104},
+{{.a = {194, 220, 211, 212, 211, 32, 196, 98, 71, 62, 153, 103, 80, 35, 128, 0}}, 114, 113},
+{{.a = {24, 255, 158, 64, 180, 148, 10, 81, 243, 247, 0, 0, 0, 0, 0, 0}}, 80, 89},
+{{.a = {231, 155, 100, 242, 112, 160, 160, 95, 98, 253, 219, 21, 239, 90, 0, 0}}, 113, 151},
+{{.a = {225, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 11, 108},
+{{.a = {136, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 7, 224},
+{{.a = {250, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 9, 95},
+{{.a = {72, 168, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 14, 173},
+{{.a = {185, 51, 51, 167, 18, 44, 36, 59, 35, 135, 20, 104, 0, 0, 0, 0}}, 93, 176},
+{{.a = {57, 146, 252, 60, 197, 68, 39, 162, 80, 198, 137, 50, 97, 92, 124, 0}}, 119, 84},
+{{.a = {254, 46, 242, 105, 86, 94, 96, 14, 130, 176, 0, 0, 0, 0, 0, 0}}, 78, 104},
+{{.a = {247, 202, 176, 76, 69, 40, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 49, 236},
+{{.a = {50, 233, 203, 77, 42, 21, 115, 163, 166, 138, 192, 52, 178, 37, 112, 0}}, 116, 153},
+{{.a = {62, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 11, 190},
+{{.a = {53, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 13, 202},
+{{.a = {198, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 8, 54},
+{{.a = {189, 234, 106, 247, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 34, 156},
+{{.a = {110, 24, 228, 65, 216, 147, 9, 48, 60, 179, 172, 91, 115, 185, 227, 96}}, 126, 245},
+{{.a = {128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 6, 218},
+{{.a = {74, 177, 89, 218, 248, 18, 176, 39, 118, 173, 201, 152, 0, 0, 0, 0}}, 93, 72},
+{{.a = {31, 13, 153, 92, 27, 122, 150, 232, 88, 95, 202, 171, 208, 158, 0, 0}}, 112, 183},
+{{.a = {128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 2, 183},
+{{.a = {63, 37, 46, 158, 139, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 42, 241},
+{{.a = {53, 209, 59, 13, 202, 70, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 47, 106},
+{{.a = {184, 44, 149, 221, 180, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 40, 180},
+{{.a = {222, 134, 37, 62, 223, 193, 39, 246, 15, 151, 200, 146, 0, 0, 0, 0}}, 96, 142},
+{{.a = {199, 176, 189, 37, 233, 177, 252, 216, 94, 175, 253, 119, 96, 0, 0, 0}}, 100, 6},
+{{.a = {44, 195, 201, 106, 209, 120, 122, 38, 43, 30, 142, 22, 196, 175, 100, 0}}, 118, 33},
+{{.a = {33, 166, 10, 174, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 34, 224},
+{{.a = {54, 1, 189, 195, 133, 49, 36, 80, 138, 200, 0, 0, 0, 0, 0, 0}}, 78, 14},
+{{.a = {241, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 10, 149},
+{{.a = {221, 131, 4, 247, 112, 89, 187, 119, 219, 80, 122, 156, 216, 160, 0, 0}}, 108, 131},
+{{.a = {102, 20, 46, 129, 202, 247, 129, 1, 237, 71, 103, 58, 217, 44, 4, 0}}, 121, 133},
+{{.a = {107, 156, 151, 44, 215, 98, 171, 126, 85, 32, 42, 128, 0, 0, 0, 0}}, 89, 33},
+{{.a = {54, 25, 70, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 28, 204},
+{{.a = {149, 211, 242, 14, 112, 219, 176, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 52, 43},
+{{.a = {95, 26, 143, 193, 8, 76, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 51, 168},
+{{.a = {63, 102, 244, 176, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 28, 180},
+{{.a = {64, 85, 124, 226, 59, 239, 64, 130, 68, 122, 93, 74, 32, 37, 0, 0}}, 112, 208},
+{{.a = {113, 90, 253, 149, 3, 218, 34, 215, 3, 143, 192, 64, 0, 0, 0, 0}}, 90, 25},
+{{.a = {75, 231, 33, 5, 11, 94, 117, 104, 150, 60, 72, 161, 96, 38, 0, 0}}, 111, 50},
+{{.a = {52, 13, 248, 1, 251, 14, 50, 29, 212, 123, 130, 177, 101, 96, 0, 0}}, 109, 110},
+{{.a = {248, 221, 150, 132, 252, 82, 96, 2, 80, 232, 97, 239, 253, 64, 0, 0}}, 109, 21},
+{{.a = {136, 77, 164, 161, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 36, 147},
+{{.a = {1, 33, 66, 254, 144, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 43, 56},
+{{.a = {181, 25, 186, 225, 109, 190, 76, 158, 118, 122, 20, 64, 125, 55, 8, 0}}, 117, 144},
+{{.a = {191, 187, 160, 140, 17, 6, 80, 120, 236, 212, 104, 144, 128, 0, 0, 0}}, 100, 198},
+{{.a = {201, 61, 150, 254, 70, 77, 214, 211, 171, 163, 245, 64, 0, 0, 0, 0}}, 90, 235},
+{{.a = {143, 226, 190, 50, 252, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 38, 105},
+{{.a = {65, 168, 226, 36, 201, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 45, 138},
+{{.a = {136, 40, 65, 90, 47, 16, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 49, 122},
+{{.a = {94, 189, 224, 200, 170, 11, 79, 172, 0, 0, 0, 0, 0, 0, 0, 0}}, 65, 193},
+{{.a = {236, 41, 169, 234, 14, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 43, 231},
+{{.a = {1, 40, 140, 95, 81, 173, 250, 248, 0, 0, 0, 0, 0, 0, 0, 0}}, 64, 250},
+{{.a = {83, 176, 146, 112, 89, 156, 57, 220, 125, 48, 44, 0, 0, 0, 0, 0}}, 86, 24},
+{{.a = {76, 125, 228, 249, 243, 160, 106, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 55, 191},
+{{.a = {10, 203, 204, 49, 212, 115, 125, 4, 239, 122, 81, 34, 1, 198, 216, 0}}, 117, 111},
+{{.a = {74, 214, 23, 44, 211, 40, 161, 61, 237, 190, 155, 59, 173, 42, 0, 0}}, 111, 205},
+{{.a = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 1, 133},
+{{.a = {127, 0, 130, 61, 209, 5, 232, 35, 35, 42, 114, 52, 169, 234, 191, 0}}, 122, 122},
+{{.a = {201, 107, 210, 13, 187, 62, 145, 28, 31, 189, 56, 0, 0, 0, 0, 0}}, 87, 227},
+{{.a = {147, 171, 63, 145, 47, 160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 46, 53},
+{{.a = {93, 232, 10, 97, 21, 243, 213, 135, 200, 0, 0, 0, 0, 0, 0, 0}}, 72, 224},
+{{.a = {144, 121, 41, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 26, 199},
+{{.a = {116, 105, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 17, 79},
+{{.a = {142, 149, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 21, 19},
+{{.a = {97, 0, 228, 158, 50, 233, 251, 249, 0, 66, 197, 226, 0, 0, 0, 0}}, 96, 211},
+{{.a = {114, 228, 199, 155, 175, 104, 26, 213, 66, 249, 120, 218, 164, 252, 212, 0}}, 120, 6},
+{{.a = {224, 166, 76, 200, 121, 60, 110, 65, 60, 95, 137, 190, 92, 218, 218, 0}}, 121, 143},
+{{.a = {139, 219, 92, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 31, 135},
+{{.a = {203, 237, 64, 189, 28, 13, 75, 197, 219, 243, 172, 3, 142, 32, 0, 0}}, 109, 21},
+{{.a = {237, 186, 88, 254, 124, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 38, 220},
+{{.a = {182, 230, 93, 162, 129, 25, 56, 196, 112, 0, 0, 0, 0, 0, 0, 0}}, 68, 151},
+{{.a = {245, 45, 69, 226, 90, 212, 254, 16, 0, 0, 0, 0, 0, 0, 0, 0}}, 60, 111},
+{{.a = {107, 229, 240, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 20, 63},
+{{.a = {119, 208, 177, 235, 222, 252, 219, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 57, 112},
+{{.a = {178, 151, 220, 162, 120, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 41, 48},
+{{.a = {109, 26, 95, 170, 166, 151, 137, 83, 226, 82, 5, 114, 253, 210, 18, 12}}, 126, 100},
+{{.a = {126, 27, 252, 19, 219, 129, 121, 48, 0, 0, 0, 0, 0, 0, 0, 0}}, 60, 156},
+{{.a = {211, 195, 152, 145, 154, 93, 228, 215, 135, 101, 28, 82, 0, 0, 0, 0}}, 95, 120},
+{{.a = {252, 107, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 16, 5},
+{{.a = {192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 4, 103},
+{{.a = {64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 4, 84},
+{{.a = {225, 179, 43, 43, 222, 145, 205, 238, 164, 158, 147, 229, 56, 0, 0, 0}}, 101, 24},
+{{.a = {208, 127, 151, 24, 64, 113, 47, 85, 209, 79, 144, 0, 0, 0, 0, 0}}, 86, 81},
+{{.a = {178, 144, 203, 104, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 29, 96},
+{{.a = {56, 227, 139, 4, 86, 87, 180, 1, 215, 167, 237, 156, 111, 64, 47, 0}}, 121, 6},
+{{.a = {80, 76, 204, 119, 172, 169, 254, 81, 104, 166, 219, 44, 173, 161, 212, 0}}, 119, 40},
+{{.a = {129, 141, 139, 34, 241, 101, 223, 144, 0, 0, 0, 0, 0, 0, 0, 0}}, 62, 143},
+{{.a = {85, 102, 137, 98, 65, 103, 54, 142, 144, 0, 0, 0, 0, 0, 0, 0}}, 68, 69},
+{{.a = {56, 31, 159, 13, 201, 139, 161, 31, 89, 137, 4, 0, 0, 0, 0, 0}}, 92, 48},
+{{.a = {229, 221, 54, 216, 223, 27, 196, 1, 0, 0, 0, 0, 0, 0, 0, 0}}, 64, 115},
+{{.a = {5, 144, 176, 43, 180, 187, 20, 49, 59, 73, 108, 34, 83, 32, 192, 0}}, 115, 130},
+{{.a = {24, 217, 205, 193, 74, 123, 160, 106, 103, 74, 200, 0, 0, 0, 0, 0}}, 86, 57},
+{{.a = {247, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 8, 97},
+{{.a = {12, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 9, 146},
+{{.a = {160, 28, 201, 119, 148, 93, 251, 118, 28, 179, 123, 52, 71, 232, 48, 0}}, 117, 194},
+{{.a = {152, 126, 17, 54, 101, 56, 130, 1, 205, 41, 207, 90, 151, 123, 128, 0}}, 114, 129},
+{{.a = {77, 165, 29, 239, 95, 242, 34, 1, 11, 204, 135, 239, 128, 0, 0, 0}}, 97, 159},
+{{.a = {183, 108, 146, 118, 74, 190, 7, 141, 9, 92, 2, 2, 8, 218, 120, 0}}, 117, 242},
+{{.a = {37, 152, 29, 239, 242, 53, 56, 143, 219, 22, 14, 158, 49, 0, 0, 0}}, 104, 162},
+{{.a = {198, 53, 241, 102, 240, 244, 97, 203, 62, 128, 213, 214, 220, 0, 0, 0}}, 102, 140},
+{{.a = {144, 89, 48, 42, 249, 231, 189, 178, 232, 199, 30, 58, 63, 57, 0, 0}}, 113, 77},
+{{.a = {68, 212, 177, 123, 44, 224, 19, 172, 89, 87, 192, 0, 0, 0, 0, 0}}, 82, 121},
+{{.a = {252, 29, 179, 224, 4, 121, 205, 67, 152, 0, 0, 0, 0, 0, 0, 0}}, 69, 102},
+{{.a = {28, 110, 164, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 23, 28},
+{{.a = {24, 88, 231, 1, 4, 71, 71, 241, 252, 14, 197, 0, 0, 0, 0, 0}}, 89, 154},
+{{.a = {63, 131, 43, 76, 58, 140, 163, 74, 158, 80, 0, 0, 0, 0, 0, 0}}, 76, 39},
+{{.a = {56, 28, 147, 149, 98, 93, 216, 216, 203, 156, 0, 0, 0, 0, 0, 0}}, 78, 163},
+{{.a = {134, 169, 6, 103, 161, 244, 134, 117, 16, 0, 0, 0, 0, 0, 0, 0}}, 68, 42},
+{{.a = {143, 247, 125, 190, 106, 50, 204, 98, 250, 151, 161, 96, 0, 0, 0, 0}}, 92, 207},
+{{.a = {235, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 8, 25},
+{{.a = {46, 72, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 18, 150},
+{{.a = {171, 35, 128, 117, 74, 29, 199, 67, 109, 176, 0, 0, 0, 0, 0, 0}}, 76, 103},
+{{.a = {220, 233, 236, 112, 135, 136, 215, 43, 42, 0, 0, 0, 0, 0, 0, 0}}, 71, 155},
+{{.a = {228, 11, 144, 117, 206, 192, 118, 25, 141, 78, 4, 105, 0, 0, 0, 0}}, 96, 142},
+{{.a = {195, 67, 194, 229, 14, 53, 129, 7, 30, 208, 38, 100, 182, 59, 0, 0}}, 112, 2},
+{{.a = {25, 76, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 19, 59},
+{{.a = {160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 4, 112},
+{{.a = {26, 203, 217, 152, 16, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 48, 166},
+{{.a = {250, 213, 14, 235, 110, 171, 174, 23, 102, 128, 0, 0, 0, 0, 0, 0}}, 73, 62},
+{{.a = {175, 230, 160, 13, 187, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 50, 176},
+{{.a = {92, 155, 156, 93, 191, 73, 28, 82, 187, 129, 57, 5, 16, 0, 0, 0}}, 100, 6},
+{{.a = {45, 203, 3, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 29, 26},
+{{.a = {120, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 5, 6},
+{{.a = {216, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 5, 13},
+{{.a = {135, 215, 0, 71, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 37, 41},
+{{.a = {221, 149, 1, 40, 112, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 36, 135},
+{{.a = {95, 143, 255, 194, 2, 157, 191, 113, 10, 229, 204, 56, 0, 0, 0, 0}}, 93, 171},
+{{.a = {202, 212, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 19, 20},
+{{.a = {147, 203, 238, 120, 194, 23, 25, 58, 208, 177, 169, 0, 0, 0, 0, 0}}, 89, 119},
+{{.a = {137, 170, 113, 252, 215, 194, 224, 146, 233, 87, 86, 192, 26, 46, 0, 0}}, 112, 49},
+{{.a = {224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 4, 141},
+{{.a = {250, 90, 241, 174, 163, 72, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 47, 132},
+{{.a = {66, 190, 202, 144, 122, 86, 22, 103, 107, 164, 57, 54, 228, 128, 0, 0}}, 105, 176},
+{{.a = {76, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 12, 186},
+{{.a = {120, 246, 1, 52, 187, 163, 78, 105, 224, 0, 0, 0, 0, 0, 0, 0}}, 67, 93},
+{{.a = {137, 242, 136, 71, 98, 10, 53, 97, 160, 85, 132, 127, 185, 222, 0, 0}}, 111, 242},
+{{.a = {255, 133, 181, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 24, 163},
+{{.a = {128, 177, 92, 155, 91, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 42, 184},
+{{.a = {45, 120, 186, 192, 240, 199, 178, 95, 32, 0, 0, 0, 0, 0, 0, 0}}, 68, 188},
+{{.a = {151, 98, 103, 254, 90, 6, 10, 109, 14, 158, 69, 29, 140, 237, 40, 232}}, 126, 193},
+{{.a = {148, 164, 81, 85, 76, 14, 84, 64, 89, 176, 0, 0, 0, 0, 0, 0}}, 78, 63},
+{{.a = {145, 187, 165, 136, 88, 30, 107, 191, 205, 120, 119, 216, 158, 123, 64, 0}}, 115, 160},
+{{.a = {78, 120, 28, 243, 216, 180, 87, 19, 253, 16, 110, 33, 228, 24, 232, 0}}, 117, 251},
+{{.a = {74, 6, 166, 166, 183, 157, 96, 84, 151, 0, 0, 0, 0, 0, 0, 0}}, 72, 228},
+{{.a = {89, 96, 4, 221, 214, 253, 58, 49, 9, 0, 0, 0, 0, 0, 0, 0}}, 72, 168},
+{{.a = {97, 9, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 18, 194},
+{{.a = {213, 215, 45, 200, 170, 120, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 47, 166},
+{{.a = {5, 14, 92, 0, 28, 245, 130, 202, 32, 40, 207, 77, 166, 170, 246, 64}}, 122, 210},
+{{.a = {77, 45, 43, 71, 202, 0, 157, 146, 59, 91, 225, 0, 0, 0, 0, 0}}, 89, 254},
+{{.a = {101, 174, 94, 168, 162, 171, 71, 12, 16, 224, 0, 0, 0, 0, 0, 0}}, 75, 49},
+{{.a = {58, 17, 187, 194, 87, 73, 215, 103, 180, 12, 40, 66, 0, 0, 0, 0}}, 96, 95},
+{{.a = {160, 91, 68, 81, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 33, 193},
+{{.a = {94, 112, 249, 13, 167, 245, 101, 64, 0, 0, 0, 0, 0, 0, 0, 0}}, 58, 155},
+{{.a = {236, 194, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 15, 133},
+{{.a = {168, 243, 103, 221, 120, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 38, 10},
+{{.a = {86, 194, 218, 188, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 33, 31},
+{{.a = {232, 3, 134, 67, 63, 196, 86, 14, 170, 243, 77, 134, 187, 140, 72, 18}}, 127, 98},
+{{.a = {55, 253, 19, 201, 199, 71, 229, 218, 54, 64, 12, 162, 0, 0, 0, 0}}, 96, 22},
+{{.a = {142, 34, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 20, 214},
+{{.a = {213, 16, 208, 50, 100, 33, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 50, 217},
+{{.a = {117, 237, 132, 185, 184, 246, 79, 42, 103, 98, 162, 243, 128, 0, 0, 0}}, 98, 102},
+{{.a = {120, 25, 214, 222, 61, 157, 203, 102, 3, 146, 192, 0, 0, 0, 0, 0}}, 83, 169},
+{{.a = {222, 46, 254, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 27, 152},
+{{.a = {254, 70, 158, 171, 11, 245, 223, 97, 70, 17, 27, 192, 186, 0, 0, 0}}, 103, 214},
+{{.a = {192, 128, 228, 17, 68, 20, 44, 31, 52, 34, 212, 1, 224, 0, 0, 0}}, 99, 178},
+{{.a = {237, 229, 203, 8, 121, 176, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 45, 164},
+{{.a = {6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 7, 15},
+{{.a = {71, 197, 251, 122, 138, 232, 12, 241, 116, 240, 0, 0, 0, 0, 0, 0}}, 76, 94},
+{{.a = {18, 241, 135, 210, 233, 54, 121, 185, 4, 0, 0, 0, 0, 0, 0, 0}}, 70, 239},
+{{.a = {32, 50, 213, 63, 73, 217, 180, 21, 187, 128, 0, 0, 0, 0, 0, 0}}, 73, 82},
+{{.a = {203, 166, 233, 73, 92, 182, 212, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 55, 54},
+{{.a = {56, 162, 126, 4, 18, 195, 192, 64, 164, 156, 119, 196, 64, 0, 0, 0}}, 98, 47},
+{{.a = {120, 87, 81, 136, 180, 179, 68, 148, 243, 38, 80, 0, 0, 0, 0, 0}}, 84, 214},
+{{.a = {64, 244, 193, 50, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 37, 215},
+{{.a = {91, 168, 253, 158, 131, 83, 159, 163, 113, 169, 112, 0, 0, 0, 0, 0}}, 84, 153},
+{{.a = {159, 103, 102, 132, 111, 46, 18, 77, 36, 15, 137, 33, 177, 31, 243, 192}}, 122, 245},
+{{.a = {123, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 8, 118},
+{{.a = {67, 81, 226, 190, 7, 79, 71, 250, 155, 245, 44, 81, 215, 213, 171, 224}}, 123, 128},
+{{.a = {103, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 8, 7},
+{{.a = {246, 44, 168, 200, 198, 238, 52, 196, 125, 115, 0, 0, 0, 0, 0, 0}}, 80, 152},
+{{.a = {205, 14, 186, 252, 239, 213, 59, 119, 105, 37, 140, 209, 4, 231, 0, 0}}, 114, 248},
+{{.a = {70, 91, 254, 106, 94, 71, 170, 19, 158, 242, 192, 0, 0, 0, 0, 0}}, 85, 143},
+{{.a = {250, 86, 233, 184, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 30, 159},
+{{.a = {122, 222, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 24, 11},
+{{.a = {27, 224, 235, 70, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 31, 110},
+{{.a = {239, 100, 224, 3, 46, 127, 150, 251, 204, 120, 228, 64, 0, 0, 0, 0}}, 97, 181},
+{{.a = {144, 115, 182, 206, 146, 13, 21, 111, 37, 70, 179, 129, 173, 82, 93, 128}}, 121, 4},
+{{.a = {73, 190, 57, 243, 49, 51, 15, 209, 0, 0, 0, 0, 0, 0, 0, 0}}, 67, 101},
+{{.a = {18, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 11, 38},
+{{.a = {23, 37, 236, 177, 186, 7, 209, 135, 114, 44, 0, 0, 0, 0, 0, 0}}, 78, 57},
+{{.a = {200, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 17, 142},
+{{.a = {181, 255, 153, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 24, 184},
+{{.a = {135, 168, 6, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 27, 91},
+{{.a = {200, 224, 33, 245, 120, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 41, 224},
+{{.a = {70, 111, 10, 62, 200, 224, 38, 204, 14, 164, 0, 0, 0, 0, 0, 0}}, 78, 114},
+{{.a = {158, 133, 252, 18, 242, 12, 16, 60, 5, 52, 251, 179, 38, 235, 12, 0}}, 118, 184},
+{{.a = {2, 23, 116, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 28, 215},
+{{.a = {33, 25, 170, 74, 215, 134, 151, 181, 175, 232, 20, 155, 189, 242, 13, 0}}, 120, 167},
+{{.a = {160, 186, 218, 183, 167, 84, 59, 152, 13, 137, 80, 128, 0, 0, 0, 0}}, 89, 233},
+{{.a = {32, 141, 196, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 29, 101},
+{{.a = {207, 24, 202, 226, 191, 136, 78, 124, 160, 0, 0, 0, 0, 0, 0, 0}}, 67, 139},
+{{.a = {210, 173, 172, 27, 197, 57, 114, 146, 169, 32, 0, 0, 0, 0, 0, 0}}, 79, 32},
+{{.a = {95, 113, 12, 123, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 32, 57},
+{{.a = {129, 108, 186, 28, 19, 229, 96, 134, 199, 254, 199, 64, 0, 0, 0, 0}}, 91, 151},
+{{.a = {103, 226, 38, 123, 35, 199, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 49, 0},
+{{.a = {41, 117, 43, 35, 208, 115, 73, 64, 0, 0, 0, 0, 0, 0, 0, 0}}, 63, 227},
+{{.a = {42, 220, 61, 34, 199, 183, 42, 16, 223, 135, 0, 135, 213, 150, 100, 0}}, 118, 124},
+{{.a = {165, 227, 96, 243, 112, 171, 117, 106, 50, 37, 82, 60, 80, 0, 0, 0}}, 104, 228},
+{{.a = {158, 60, 111, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 27, 64},
+{{.a = {124, 108, 88, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 25, 179},
+{{.a = {232, 68, 132, 159, 156, 103, 95, 190, 76, 0, 0, 0, 0, 0, 0, 0}}, 70, 107},
+{{.a = {70, 77, 240, 209, 72, 63, 63, 45, 125, 79, 77, 41, 13, 0, 0, 0}}, 104, 206},
+{{.a = {146, 254, 7, 5, 68, 240, 67, 237, 112, 0, 0, 0, 0, 0, 0, 0}}, 68, 95},
+{{.a = {162, 223, 117, 27, 2, 156, 94, 170, 157, 114, 162, 50, 0, 0, 0, 0}}, 96, 219},
+{{.a = {161, 62, 191, 68, 239, 73, 100, 37, 168, 254, 139, 202, 252, 65, 74, 0}}, 119, 138},
+{{.a = {248, 122, 115, 81, 15, 158, 136, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 53, 84},
+{{.a = {8, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 11, 161},
+{{.a = {142, 96, 105, 133, 251, 57, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 52, 25},
+{{.a = {138, 196, 139, 131, 233, 93, 65, 242, 86, 169, 7, 72, 82, 128, 0, 0}}, 107, 113},
+{{.a = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 2, 46},
+{{.a = {175, 151, 75, 238, 26, 12, 100, 186, 0, 0, 0, 0, 0, 0, 0, 0}}, 63, 72},
+{{.a = {82, 205, 211, 176, 170, 79, 57, 153, 161, 218, 32, 48, 0, 0, 0, 0}}, 93, 230},
+{{.a = {227, 123, 232, 74, 236, 202, 211, 121, 200, 8, 59, 189, 81, 219, 144, 0}}, 117, 142},
+{{.a = {205, 196, 89, 90, 103, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 41, 134},
+{{.a = {63, 145, 23, 127, 102, 216, 49, 36, 168, 164, 59, 133, 18, 146, 0, 0}}, 112, 100},
+{{.a = {213, 72, 154, 16, 230, 236, 218, 203, 223, 51, 31, 251, 103, 64, 0, 0}}, 109, 45},
+{{.a = {126, 148, 232, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 30, 219},
+{{.a = {160, 72, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 16, 52},
+{{.a = {137, 38, 146, 20, 99, 188, 83, 123, 159, 159, 64, 0, 0, 0, 0, 0}}, 83, 240},
+{{.a = {123, 228, 36, 44, 242, 29, 51, 228, 140, 60, 237, 0, 0, 0, 0, 0}}, 90, 13},
+{{.a = {163, 169, 25, 89, 190, 114, 165, 158, 140, 210, 192, 0, 0, 0, 0, 0}}, 84, 191},
+{{.a = {225, 38, 70, 89, 218, 236, 60, 5, 69, 163, 248, 50, 163, 64, 0, 0}}, 106, 95},
+{{.a = {91, 94, 36, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 28, 65},
+{{.a = {209, 238, 110, 0, 2, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 45, 195},
+{{.a = {57, 17, 224, 164, 69, 95, 138, 172, 111, 55, 239, 167, 160, 0, 0, 0}}, 103, 21},
+{{.a = {112, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 4, 114},
+{{.a = {102, 96, 223, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 31, 92},
+{{.a = {137, 204, 150, 75, 193, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 42, 237},
+{{.a = {136, 56, 252, 240, 85, 48, 248, 231, 17, 49, 47, 238, 15, 233, 159, 184}}, 125, 172},
+{{.a = {57, 31, 132, 123, 234, 255, 37, 82, 167, 204, 37, 158, 128, 0, 0, 0}}, 98, 116},
+{{.a = {55, 198, 139, 219, 161, 156, 140, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 55, 54},
+{{.a = {44, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 8, 203},
+{{.a = {53, 38, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 16, 74},
+{{.a = {227, 62, 107, 236, 118, 156, 60, 34, 31, 179, 76, 221, 0, 0, 0, 0}}, 96, 220},
+{{.a = {105, 40, 240, 216, 91, 61, 19, 128, 224, 0, 0, 0, 0, 0, 0, 0}}, 67, 219},
+{{.a = {96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 3, 179},
+{{.a = {118, 142, 251, 249, 128, 105, 113, 16, 0, 0, 0, 0, 0, 0, 0, 0}}, 61, 194},
+{{.a = {101, 70, 196, 238, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 32, 187},
+{{.a = {245, 173, 165, 177, 200, 161, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 50, 79},
+{{.a = {0, 198, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 19, 87},
+{{.a = {92, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 8, 126},
+{{.a = {125, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 11, 106},
+{{.a = {56, 59, 35, 82, 101, 72, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 50, 96},
+{{.a = {184, 72, 77, 251, 8, 166, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 47, 45},
+{{.a = {143, 74, 132, 205, 218, 247, 30, 160, 145, 199, 138, 12, 89, 220, 0, 0}}, 110, 8},
+{{.a = {30, 178, 111, 225, 73, 79, 173, 52, 0, 0, 0, 0, 0, 0, 0, 0}}, 62, 226},
+{{.a = {224, 48, 154, 231, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 36, 222},
+{{.a = {123, 144, 170, 143, 85, 169, 130, 245, 214, 0, 0, 0, 0, 0, 0, 0}}, 71, 218},
+{{.a = {166, 224, 212, 100, 149, 55, 35, 210, 246, 108, 41, 245, 127, 174, 128, 0}}, 116, 59},
+{{.a = {75, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 8, 80},
+{{.a = {197, 128, 190, 87, 47, 53, 92, 64, 0, 0, 0, 0, 0, 0, 0, 0}}, 58, 177},
+{{.a = {249, 10, 76, 217, 225, 20, 124, 205, 44, 159, 190, 8, 0, 0, 0, 0}}, 98, 44},
+{{.a = {180, 226, 0, 167, 137, 232, 174, 120, 113, 95, 22, 184, 0, 0, 0, 0}}, 93, 206},
+{{.a = {123, 153, 102, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 27, 64},
+{{.a = {5, 144, 206, 158, 239, 189, 171, 120, 69, 46, 128, 237, 0, 0, 0, 0}}, 96, 236},
+{{.a = {159, 235, 60, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 22, 101},
+{{.a = {42, 194, 150, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 26, 49},
+{{.a = {205, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 11, 179},
+{{.a = {19, 65, 141, 20, 127, 77, 70, 205, 151, 115, 157, 23, 118, 128, 0, 0}}, 109, 112},
+{{.a = {96, 11, 214, 40, 245, 251, 61, 64, 128, 241, 183, 183, 0, 0, 0, 0}}, 96, 31},
+{{.a = {120, 4, 235, 112, 34, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 41, 111},
+{{.a = {110, 127, 207, 76, 100, 148, 130, 206, 249, 2, 104, 0, 0, 0, 0, 0}}, 86, 65},
+{{.a = {226, 190, 191, 249, 173, 96, 127, 200, 62, 20, 0, 0, 0, 0, 0, 0}}, 78, 222},
+{{.a = {89, 88, 182, 14, 78, 122, 213, 192, 0, 0, 0, 0, 0, 0, 0, 0}}, 58, 4},
+{{.a = {167, 94, 163, 227, 28, 111, 117, 103, 224, 0, 0, 0, 0, 0, 0, 0}}, 67, 67},
+{{.a = {57, 220, 53, 116, 243, 184, 242, 134, 16, 70, 83, 61, 161, 128, 0, 0}}, 109, 197},
+{{.a = {63, 235, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 22, 121},
+{{.a = {128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 1, 167},
+{{.a = {15, 159, 42, 167, 76, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 38, 140},
+{{.a = {216, 252, 113, 40, 239, 46, 172, 48, 103, 250, 82, 179, 136, 64, 0, 0}}, 106, 193},
+{{.a = {158, 147, 16, 44, 124, 56, 44, 48, 138, 64, 169, 0, 0, 0, 0, 0}}, 90, 47},
+{{.a = {238, 238, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 21, 187},
+{{.a = {63, 159, 177, 162, 106, 212, 172, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 56, 102},
+{{.a = {59, 40, 252, 185, 187, 216, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 46, 237},
+{{.a = {2, 218, 11, 68, 173, 196, 16, 223, 2, 18, 122, 215, 154, 0, 0, 0}}, 103, 237},
+{{.a = {3, 9, 206, 73, 108, 196, 183, 119, 141, 162, 10, 180, 115, 32, 0, 0}}, 107, 115},
+{{.a = {17, 227, 208, 146, 63, 201, 73, 239, 29, 79, 80, 0, 0, 0, 0, 0}}, 84, 217},
+{{.a = {115, 180, 176, 241, 52, 209, 6, 64, 189, 76, 0, 0, 0, 0, 0, 0}}, 79, 21},
+{{.a = {191, 88, 98, 245, 91, 46, 137, 254, 170, 80, 11, 55, 212, 28, 128, 0}}, 113, 3},
+{{.a = {97, 141, 171, 175, 22, 233, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 48, 62},
+{{.a = {32, 204, 102, 191, 164, 242, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 47, 80},
+{{.a = {29, 133, 210, 252, 124, 66, 160, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 51, 184},
+{{.a = {207, 179, 54, 144, 116, 67, 29, 64, 13, 199, 0, 0, 0, 0, 0, 0}}, 80, 197},
+{{.a = {129, 216, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 13, 63},
+{{.a = {50, 152, 249, 143, 174, 234, 240, 48, 158, 255, 80, 105, 0, 0, 0, 0}}, 99, 62},
+{{.a = {105, 208, 95, 218, 44, 11, 87, 134, 109, 18, 138, 66, 17, 69, 128, 0}}, 114, 231},
+{{.a = {151, 79, 158, 220, 122, 101, 210, 164, 64, 0, 0, 0, 0, 0, 0, 0}}, 67, 158},
+{{.a = {236, 97, 87, 155, 254, 137, 122, 208, 168, 201, 194, 118, 224, 0, 0, 0}}, 101, 118},
+{{.a = {14, 229, 193, 248, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 30, 237},
+{{.a = {46, 154, 50, 80, 92, 147, 158, 86, 1, 112, 0, 0, 0, 0, 0, 0}}, 79, 15},
+{{.a = {88, 131, 21, 84, 62, 86, 7, 110, 142, 251, 242, 110, 194, 175, 247, 0}}, 122, 84},
+{{.a = {229, 216, 111, 92, 173, 32, 63, 70, 36, 84, 6, 74, 136, 166, 38, 0}}, 119, 205},
+{{.a = {121, 147, 216, 245, 37, 189, 146, 63, 145, 74, 128, 0, 0, 0, 0, 0}}, 82, 220},
+{{.a = {44, 26, 254, 11, 181, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 40, 42},
+{{.a = {209, 114, 97, 249, 227, 159, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 51, 144},
+{{.a = {184, 244, 43, 117, 56, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 37, 74},
+{{.a = {60, 81, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 19, 89},
+{{.a = {18, 40, 21, 113, 226, 91, 195, 88, 161, 19, 142, 0, 0, 0, 0, 0}}, 88, 77},
+{{.a = {57, 0, 212, 158, 56, 51, 108, 198, 59, 5, 137, 196, 0, 0, 0, 0}}, 94, 2},
+{{.a = {168, 252, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 14, 75},
+{{.a = {64, 181, 254, 103, 1, 230, 117, 199, 128, 0, 0, 0, 0, 0, 0, 0}}, 65, 18},
+{{.a = {212, 48, 214, 127, 78, 176, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 46, 246},
+{{.a = {155, 185, 236, 163, 204, 49, 129, 120, 183, 47, 10, 243, 65, 92, 192, 0}}, 114, 10},
+{{.a = {94, 200, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 14, 207},
+{{.a = {19, 210, 136, 113, 73, 79, 132, 196, 224, 0, 0, 0, 0, 0, 0, 0}}, 68, 41},
+{{.a = {24, 203, 246, 242, 241, 223, 150, 237, 213, 202, 11, 128, 0, 0, 0, 0}}, 89, 102},
+{{.a = {115, 59, 171, 221, 172, 181, 170, 67, 115, 205, 44, 107, 162, 67, 56, 0}}, 118, 118},
+{{.a = {250, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 10, 146},
+{{.a = {203, 240, 28, 158, 182, 12, 86, 182, 142, 47, 143, 57, 239, 0, 0, 0}}, 104, 122},
+{{.a = {196, 218, 109, 52, 2, 0, 64, 153, 34, 250, 240, 185, 117, 0, 0, 0}}, 107, 6},
+{{.a = {137, 131, 191, 40, 72, 209, 74, 64, 0, 0, 0, 0, 0, 0, 0, 0}}, 58, 18},
+{{.a = {236, 126, 167, 37, 185, 20, 34, 207, 76, 0, 0, 0, 0, 0, 0, 0}}, 70, 83},
+{{.a = {129, 192, 245, 137, 251, 52, 75, 68, 81, 112, 146, 133, 64, 0, 0, 0}}, 99, 90},
+{{.a = {7, 31, 148, 19, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 32, 140},
+{{.a = {112, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 4, 242},
+{{.a = {167, 50, 202, 179, 74, 146, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 50, 31},
+{{.a = {44, 188, 186, 250, 229, 71, 28, 118, 35, 253, 245, 191, 199, 18, 0, 0}}, 111, 9},
+{{.a = {128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 1, 230},
+{{.a = {156, 163, 215, 175, 71, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 47, 50},
+{{.a = {67, 24, 151, 198, 242, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 41, 34},
+{{.a = {134, 107, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 17, 11},
+{{.a = {35, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 8, 71},
+{{.a = {46, 196, 84, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 22, 146},
+{{.a = {82, 172, 8, 26, 154, 34, 125, 188, 5, 149, 159, 44, 78, 222, 236, 176}}, 124, 249},
+{{.a = {78, 157, 79, 70, 252, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 39, 143},
+{{.a = {231, 5, 210, 247, 198, 5, 157, 191, 206, 225, 149, 142, 207, 40, 0, 0}}, 110, 17},
+{{.a = {38, 254, 235, 199, 191, 60, 43, 159, 190, 243, 203, 185, 184, 218, 132, 0}}, 119, 60},
+{{.a = {128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 1, 162},
+{{.a = {95, 240, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 12, 5},
+{{.a = {17, 128, 244, 178, 160, 78, 83, 92, 0, 0, 0, 0, 0, 0, 0, 0}}, 62, 139},
+{{.a = {18, 102, 62, 251, 44, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 39, 8},
+{{.a = {30, 75, 108, 40, 231, 166, 233, 220, 163, 176, 252, 210, 60, 30, 128, 0}}, 114, 246},
+{{.a = {18, 3, 207, 64, 25, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 42, 171},
+{{.a = {52, 83, 235, 61, 164, 236, 83, 173, 143, 105, 14, 0, 0, 0, 0, 0}}, 88, 206},
+{{.a = {166, 175, 186, 208, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 28, 163},
+{{.a = {221, 154, 82, 98, 41, 126, 85, 52, 0, 0, 0, 0, 0, 0, 0, 0}}, 62, 166},
+{{.a = {94, 84, 182, 120, 204, 232, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 51, 128},
+{{.a = {27, 174, 227, 228, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 31, 59},
+{{.a = {218, 12, 4, 156, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 32, 179},
+{{.a = {9, 5, 190, 195, 60, 216, 80, 150, 128, 117, 86, 128, 128, 112, 98, 208}}, 124, 87},
+{{.a = {7, 226, 104, 112, 212, 9, 172, 124, 209, 121, 170, 229, 44, 178, 128, 0}}, 114, 29},
+{{.a = {47, 71, 174, 76, 52, 83, 23, 18, 106, 48, 56, 32, 0, 0, 0, 0}}, 91, 184},
+{{.a = {51, 168, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 14, 45},
+{{.a = {28, 182, 167, 124, 28, 22, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 55, 144},
+{{.a = {34, 61, 14, 51, 253, 17, 19, 170, 49, 206, 188, 207, 247, 167, 192, 0}}, 114, 119},
+{{.a = {2, 235, 18, 14, 195, 66, 237, 30, 0, 0, 0, 0, 0, 0, 0, 0}}, 64, 113},
+{{.a = {51, 182, 142, 133, 127, 96, 159, 132, 99, 161, 64, 0, 0, 0, 0, 0}}, 82, 50},
+{{.a = {170, 145, 230, 123, 215, 189, 73, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 56, 207},
+{{.a = {151, 166, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 21, 3},
+{{.a = {16, 141, 196, 129, 132, 207, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 58, 13},
+{{.a = {205, 25, 184, 191, 201, 206, 109, 224, 0, 0, 0, 0, 0, 0, 0, 0}}, 59, 42},
+{{.a = {48, 114, 33, 103, 247, 255, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 50, 31},
+{{.a = {179, 156, 119, 146, 125, 21, 42, 146, 237, 213, 191, 132, 0, 0, 0, 0}}, 94, 30},
+{{.a = {179, 129, 186, 144, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 29, 94},
+{{.a = {17, 179, 217, 188, 128, 212, 4, 4, 152, 0, 0, 0, 0, 0, 0, 0}}, 71, 190},
+{{.a = {132, 63, 74, 89, 209, 64, 63, 192, 0, 0, 0, 0, 0, 0, 0, 0}}, 59, 238},
+{{.a = {16, 50, 248, 88, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 30, 20},
+{{.a = {189, 96, 58, 53, 191, 235, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 51, 84},
+{{.a = {111, 98, 6, 65, 35, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 42, 108},
+{{.a = {118, 223, 83, 220, 110, 122, 23, 112, 185, 155, 73, 0, 0, 0, 0, 0}}, 89, 136},
+{{.a = {173, 191, 150, 197, 204, 35, 169, 79, 31, 214, 251, 240, 0, 0, 0, 0}}, 93, 196},
+{{.a = {26, 76, 129, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 28, 67},
+{{.a = {231, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 12, 104},
+{{.a = {93, 172, 223, 252, 203, 0, 206, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 55, 15},
+{{.a = {53, 142, 203, 124, 104, 51, 241, 12, 161, 17, 101, 245, 120, 110, 192, 199}}, 128, 237},
+{{.a = {9, 77, 120, 197, 193, 10, 237, 174, 233, 2, 165, 11, 229, 47, 144, 0}}, 116, 224},
+{{.a = {99, 161, 189, 88, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 36, 179},
+{{.a = {18, 8, 76, 66, 2, 185, 206, 132, 224, 0, 0, 0, 0, 0, 0, 0}}, 67, 84},
+{{.a = {169, 53, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 22, 65},
+{{.a = {136, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 5, 178},
+{{.a = {131, 162, 144, 124, 12, 98, 242, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 55, 154},
+{{.a = {75, 50, 129, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 27, 106},
+{{.a = {212, 183, 40, 225, 152, 136, 174, 91, 0, 0, 0, 0, 0, 0, 0, 0}}, 67, 125},
+{{.a = {158, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 9, 118},
+{{.a = {7, 48, 132, 149, 169, 212, 198, 137, 202, 0, 0, 0, 0, 0, 0, 0}}, 73, 52},
+{{.a = {173, 195, 129, 163, 141, 249, 40, 64, 0, 0, 0, 0, 0, 0, 0, 0}}, 58, 173},
+{{.a = {109, 79, 75, 219, 205, 182, 22, 245, 223, 17, 146, 78, 109, 119, 128, 0}}, 113, 8},
+{{.a = {174, 195, 24, 182, 215, 198, 214, 86, 34, 128, 0, 0, 0, 0, 0, 0}}, 74, 211},
+{{.a = {22, 40, 51, 109, 70, 91, 152, 56, 0, 0, 0, 0, 0, 0, 0, 0}}, 61, 253},
+{{.a = {169, 115, 246, 126, 65, 118, 219, 192, 0, 0, 0, 0, 0, 0, 0, 0}}, 59, 47},
+{{.a = {154, 37, 70, 124, 107, 123, 232, 241, 164, 142, 71, 226, 182, 126, 0, 0}}, 112, 73},
+{{.a = {6, 108, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 19, 192},
+{{.a = {216, 167, 158, 158, 222, 19, 96, 28, 40, 6, 70, 12, 147, 27, 85, 240}}, 128, 55},
+{{.a = {72, 222, 52, 69, 69, 206, 163, 106, 235, 206, 80, 128, 0, 0, 0, 0}}, 94, 147},
+{{.a = {150, 112, 106, 56, 15, 243, 154, 97, 134, 110, 160, 20, 183, 144, 234, 8}}, 125, 86},
+{{.a = {58, 186, 106, 58, 124, 171, 53, 85, 33, 100, 64, 0, 0, 0, 0, 0}}, 82, 16},
+{{.a = {7, 195, 22, 31, 62, 217, 209, 46, 90, 49, 189, 50, 168, 126, 0, 0}}, 111, 167},
+{{.a = {92, 44, 159, 198, 185, 94, 231, 177, 64, 0, 0, 0, 0, 0, 0, 0}}, 67, 148},
+{{.a = {169, 108, 190, 162, 23, 39, 140, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 54, 66},
+{{.a = {161, 5, 3, 11, 158, 157, 166, 212, 246, 22, 140, 101, 92, 0, 0, 0}}, 104, 70},
+{{.a = {71, 191, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 16, 166},
+{{.a = {48, 136, 194, 145, 57, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 44, 109},
+{{.a = {144, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 5, 226},
+{{.a = {223, 209, 10, 57, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 32, 8},
+{{.a = {154, 79, 170, 9, 43, 139, 249, 176, 186, 72, 216, 0, 0, 0, 0, 0}}, 85, 218},
+{{.a = {1, 8, 123, 205, 167, 134, 128, 102, 10, 72, 0, 0, 0, 0, 0, 0}}, 78, 54},
+{{.a = {31, 105, 48, 77, 103, 187, 99, 67, 96, 0, 0, 0, 0, 0, 0, 0}}, 67, 48},
+{{.a = {14, 73, 54, 76, 232, 35, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 51, 244},
+{{.a = {14, 109, 251, 190, 36, 253, 99, 120, 94, 64, 0, 0, 0, 0, 0, 0}}, 74, 50},
+{{.a = {122, 170, 9, 134, 124, 91, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 54, 173},
+{{.a = {246, 10, 85, 88, 82, 217, 95, 56, 216, 203, 160, 0, 0, 0, 0, 0}}, 84, 245},
+{{.a = {77, 100, 114, 207, 150, 177, 69, 134, 74, 131, 147, 117, 177, 64, 210, 128}}, 121, 54},
+{{.a = {171, 123, 22, 138, 132, 229, 250, 81, 186, 227, 146, 27, 170, 205, 128, 0}}, 113, 86},
+{{.a = {128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 2, 115},
+{{.a = {12, 35, 50, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 24, 144},
+{{.a = {255, 124, 179, 165, 169, 250, 66, 171, 223, 125, 247, 0, 0, 0, 0, 0}}, 89, 171},
+{{.a = {244, 235, 211, 10, 251, 255, 206, 6, 198, 12, 50, 136, 0, 0, 0, 0}}, 93, 231},
+{{.a = {221, 77, 237, 41, 50, 33, 103, 24, 25, 127, 208, 0, 0, 0, 0, 0}}, 88, 34},
+{{.a = {216, 69, 47, 53, 117, 24, 56, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 53, 225},
+{{.a = {180, 87, 25, 236, 56, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 38, 174},
+{{.a = {110, 32, 24, 34, 116, 133, 245, 128, 123, 95, 125, 122, 100, 129, 128, 0}}, 113, 37},
+{{.a = {27, 117, 179, 112, 133, 137, 110, 193, 246, 201, 219, 65, 56, 234, 106, 128}}, 121, 39},
+{{.a = {186, 117, 252, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 23, 59},
+{{.a = {243, 119, 54, 16, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 34, 96},
+{{.a = {160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 3, 147},
+{{.a = {78, 48, 117, 200, 245, 118, 115, 240, 170, 125, 84, 103, 33, 168, 0, 0}}, 110, 56},
+{{.a = {201, 253, 184, 254, 143, 81, 95, 42, 243, 147, 96, 145, 23, 26, 0, 0}}, 111, 234},
+{{.a = {41, 215, 84, 136, 234, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 46, 199},
+{{.a = {91, 244, 137, 184, 231, 95, 135, 10, 184, 0, 0, 0, 0, 0, 0, 0}}, 69, 191},
+{{.a = {113, 31, 181, 245, 21, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 40, 235},
+{{.a = {181, 216, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 20, 45},
+{{.a = {87, 26, 119, 229, 97, 255, 9, 43, 32, 0, 0, 0, 0, 0, 0, 0}}, 67, 164},
+{{.a = {205, 112, 67, 163, 196, 148, 5, 105, 8, 138, 144, 3, 171, 213, 159, 128}}, 121, 130},
+{{.a = {136, 27, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 18, 166},
+{{.a = {2, 175, 111, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 24, 140},
+{{.a = {222, 131, 85, 218, 16, 229, 44, 230, 243, 76, 250, 139, 1, 203, 108, 0}}, 118, 47},
+{{.a = {101, 180, 77, 142, 194, 73, 196, 246, 107, 100, 194, 72, 204, 124, 0, 0}}, 111, 148},
+{{.a = {96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 3, 103},
+{{.a = {46, 62, 191, 130, 110, 128, 235, 62, 68, 39, 58, 152, 207, 204, 96, 0}}, 116, 94},
+{{.a = {111, 11, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 19, 85},
+{{.a = {58, 43, 14, 93, 102, 210, 117, 208, 222, 171, 130, 41, 16, 16, 0, 0}}, 109, 250},
+{{.a = {141, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 16, 153},
+{{.a = {170, 153, 160, 170, 144, 235, 122, 8, 106, 34, 24, 32, 102, 57, 12, 168}}, 125, 182},
+{{.a = {34, 113, 163, 107, 61, 177, 39, 172, 242, 2, 130, 0, 0, 0, 0, 0}}, 94, 23},
+{{.a = {222, 191, 239, 110, 162, 191, 195, 181, 80, 50, 85, 240, 88, 32, 0, 0}}, 108, 38},
+{{.a = {179, 82, 253, 151, 212, 0, 72, 253, 175, 22, 34, 78, 53, 32, 0, 0}}, 110, 121},
+{{.a = {10, 162, 20, 46, 164, 64, 88, 1, 202, 204, 124, 0, 0, 0, 0, 0}}, 87, 146},
+{{.a = {210, 99, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 16, 138},
+{{.a = {183, 200, 1, 2, 51, 6, 66, 142, 20, 77, 48, 244, 0, 0, 0, 0}}, 94, 149},
+{{.a = {29, 20, 224, 57, 204, 161, 131, 254, 53, 133, 163, 0, 0, 0, 0, 0}}, 88, 232},
+{{.a = {75, 58, 170, 52, 146, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 45, 255},
+{{.a = {92, 21, 1, 113, 185, 88, 160, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 51, 148},
+{{.a = {103, 180, 222, 187, 129, 117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 48, 117},
+{{.a = {32, 76, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 14, 237},
+{{.a = {7, 60, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 19, 113},
+{{.a = {167, 122, 205, 185, 21, 199, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 57, 162},
+{{.a = {21, 78, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 21, 225},
+{{.a = {92, 159, 167, 169, 136, 176, 95, 255, 87, 137, 112, 16, 0, 0, 0, 0}}, 92, 210},
+{{.a = {84, 120, 160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 22, 34},
+{{.a = {126, 5, 126, 176, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 29, 224},
+{{.a = {4, 58, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 16, 143},
+{{.a = {239, 154, 181, 182, 189, 211, 244, 53, 144, 0, 0, 0, 0, 0, 0, 0}}, 68, 216},
+{{.a = {254, 188, 139, 167, 135, 47, 147, 239, 187, 106, 228, 156, 234, 234, 102, 0}}, 120, 239},
+{{.a = {225, 168, 138, 92, 193, 255, 47, 233, 11, 154, 205, 86, 209, 88, 0, 0}}, 111, 54},
+{{.a = {223, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 9, 35},
+{{.a = {235, 252, 115, 10, 151, 104, 193, 207, 38, 228, 229, 245, 42, 13, 108, 0}}, 119, 230},
+{{.a = {1, 137, 53, 36, 210, 184, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 45, 234},
+{{.a = {149, 182, 72, 197, 92, 229, 9, 10, 220, 128, 72, 19, 4, 58, 192, 0}}, 115, 70},
+{{.a = {105, 73, 57, 108, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 32, 246},
+{{.a = {189, 61, 230, 24, 235, 82, 58, 102, 97, 111, 121, 252, 156, 94, 191, 166}}, 127, 217},
+{{.a = {193, 108, 231, 86, 140, 14, 192, 4, 135, 80, 129, 166, 158, 61, 230, 20}}, 128, 201},
+{{.a = {110, 77, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 17, 49},
+{{.a = {3, 102, 36, 231, 15, 242, 143, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 57, 2},
+{{.a = {81, 189, 220, 168, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 38, 64},
+{{.a = {168, 75, 133, 180, 91, 165, 77, 232, 0, 0, 0, 0, 0, 0, 0, 0}}, 62, 239},
+{{.a = {106, 179, 186, 109, 81, 234, 233, 167, 101, 160, 90, 102, 174, 234, 208, 0}}, 116, 47},
+{{.a = {46, 105, 234, 21, 23, 247, 169, 33, 47, 5, 0, 0, 0, 0, 0, 0}}, 80, 43},
+{{.a = {152, 144, 100, 142, 129, 23, 227, 50, 67, 81, 249, 116, 0, 0, 0, 0}}, 94, 17},
+{{.a = {109, 74, 145, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 26, 5},
+{{.a = {100, 243, 22, 230, 38, 44, 128, 86, 132, 57, 0, 0, 0, 0, 0, 0}}, 81, 240},
+{{.a = {153, 251, 115, 65, 104, 179, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 50, 197},
+{{.a = {43, 113, 60, 224, 36, 20, 42, 161, 24, 223, 192, 0, 0, 0, 0, 0}}, 84, 192},
+{{.a = {61, 77, 121, 176, 138, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 47, 160},
+{{.a = {119, 194, 146, 49, 59, 242, 25, 220, 122, 104, 80, 0, 0, 0, 0, 0}}, 84, 199},
+{{.a = {254, 162, 155, 47, 187, 3, 1, 114, 142, 191, 152, 44, 144, 26, 202, 0}}, 127, 217},
+{{.a = {176, 1, 114, 42, 191, 145, 43, 1, 141, 18, 64, 0, 0, 0, 0, 0}}, 83, 75},
+{{.a = {170, 244, 67, 132, 145, 163, 76, 213, 85, 237, 248, 22, 207, 64, 0, 0}}, 106, 222},
+{{.a = {102, 190, 58, 32, 75, 15, 89, 163, 64, 7, 168, 0, 0, 0, 0, 0}}, 85, 39},
+{{.a = {124, 170, 35, 47, 152, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 45, 9},
+{{.a = {192, 221, 20, 228, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 33, 217},
+{{.a = {208, 178, 136, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 25, 142},
+{{.a = {188, 68, 77, 30, 68, 153, 102, 180, 0, 0, 0, 0, 0, 0, 0, 0}}, 62, 18},
+{{.a = {114, 178, 121, 188, 205, 233, 35, 77, 34, 197, 158, 174, 101, 0, 0, 0}}, 104, 180},
+{{.a = {195, 98, 67, 12, 13, 43, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 56, 205},
+{{.a = {146, 190, 42, 222, 14, 54, 28, 128, 0, 0, 0, 0, 0, 0, 0, 0}}, 57, 251},
+{{.a = {185, 202, 148, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 22, 178},
+{{.a = {138, 30, 129, 95, 224, 161, 120, 128, 0, 0, 0, 0, 0, 0, 0, 0}}, 59, 198},
+{{.a = {69, 181, 5, 227, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 34, 84},
+{{.a = {90, 180, 0, 164, 227, 75, 174, 119, 128, 0, 0, 0, 0, 0, 0, 0}}, 66, 128},
+{{.a = {20, 60, 58, 119, 245, 177, 162, 186, 13, 112, 211, 239, 128, 0, 0, 0}}, 97, 75},
+{{.a = {158, 124, 157, 25, 230, 139, 51, 212, 76, 109, 236, 210, 48, 0, 0, 0}}, 101, 192},
+{{.a = {125, 108, 242, 36, 94, 13, 36, 106, 90, 51, 83, 217, 131, 151, 0, 0}}, 114, 60},
+{{.a = {222, 218, 162, 158, 15, 53, 191, 178, 0, 0, 0, 0, 0, 0, 0, 0}}, 63, 169},
+{{.a = {104, 202, 127, 109, 73, 16, 17, 12, 0, 0, 0, 0, 0, 0, 0, 0}}, 62, 10},
+{{.a = {172, 171, 246, 26, 176, 34, 22, 152, 246, 56, 173, 120, 105, 60, 92, 0}}, 118, 64},
+{{.a = {190, 22, 171, 206, 109, 186, 179, 128, 253, 182, 108, 212, 220, 167, 171, 180}}, 127, 182},
+{{.a = {119, 208, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 13, 29},
+{{.a = {160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 3, 39},
+{{.a = {170, 144, 64, 2, 107, 166, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 51, 93},
+{{.a = {234, 9, 96, 20, 156, 157, 1, 34, 88, 0, 0, 0, 0, 0, 0, 0}}, 75, 228},
+{{.a = {147, 237, 16, 120, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 34, 236},
+{{.a = {182, 189, 162, 158, 223, 90, 173, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 56, 190},
+{{.a = {116, 148, 142, 240, 10, 253, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 49, 217},
+{{.a = {211, 73, 140, 69, 252, 27, 75, 46, 37, 6, 147, 32, 0, 0, 0, 0}}, 93, 74},
+{{.a = {148, 61, 120, 49, 220, 65, 150, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 61, 180},
+{{.a = {172, 35, 202, 180, 129, 75, 240, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 55, 91},
+{{.a = {215, 109, 147, 157, 32, 28, 52, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 54, 230},
+{{.a = {151, 26, 182, 112, 205, 220, 160, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 52, 175},
+{{.a = {73, 91, 93, 61, 196, 3, 66, 26, 149, 96, 0, 0, 0, 0, 0, 0}}, 75, 171},
+{{.a = {203, 163, 52, 247, 28, 119, 56, 223, 138, 70, 174, 97, 77, 59, 46, 0}}, 120, 202},
+{{.a = {251, 50, 228, 178, 202, 195, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 49, 113},
+{{.a = {217, 159, 164, 199, 14, 237, 170, 184, 100, 231, 92, 222, 0, 0, 0, 0}}, 96, 187},
+{{.a = {16, 161, 85, 193, 202, 21, 3, 155, 63, 116, 124, 203, 34, 13, 215, 0}}, 120, 38},
+{{.a = {111, 52, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 26, 35},
+{{.a = {69, 12, 116, 151, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 38, 115},
+{{.a = {187, 60, 97, 40, 112, 101, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 51, 18},
+{{.a = {230, 194, 136, 255, 206, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 39, 34},
+{{.a = {179, 239, 170, 107, 3, 13, 212, 67, 177, 69, 8, 0, 0, 0, 0, 0}}, 87, 75},
+{{.a = {11, 58, 130, 89, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 33, 232},
+{{.a = {217, 178, 43, 203, 234, 20, 234, 186, 157, 88, 146, 192, 0, 0, 0, 0}}, 91, 154},
+{{.a = {6, 180, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 19, 195},
+{{.a = {157, 154, 218, 158, 39, 224, 103, 230, 164, 0, 0, 0, 0, 0, 0, 0}}, 70, 122},
+{{.a = {225, 10, 131, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 24, 97},
+{{.a = {16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 5, 220},
+{{.a = {166, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 7, 80},
+{{.a = {29, 190, 131, 215, 232, 246, 41, 226, 52, 192, 0, 0, 0, 0, 0, 0}}, 77, 133},
+{{.a = {138, 74, 163, 93, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 38, 93},
+{{.a = {229, 64, 97, 41, 28, 243, 249, 185, 97, 35, 49, 27, 175, 24, 0, 0}}, 110, 176},
+{{.a = {6, 73, 94, 160, 186, 216, 84, 117, 233, 169, 146, 234, 0, 0, 0, 0}}, 95, 68},
+{{.a = {163, 40, 242, 81, 224, 35, 72, 194, 176, 78, 224, 174, 12, 0, 0, 0}}, 103, 247},
+{{.a = {2, 205, 40, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 29, 240},
+{{.a = {174, 225, 240, 160, 212, 8, 246, 67, 36, 0, 0, 0, 0, 0, 0, 0}}, 74, 83},
+{{.a = {5, 117, 182, 141, 166, 249, 196, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 55, 132},
+{{.a = {46, 152, 169, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 24, 217},
+{{.a = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 3, 214},
+{{.a = {233, 202, 159, 219, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 34, 193},
+{{.a = {172, 54, 159, 5, 14, 245, 106, 182, 2, 0, 0, 0, 0, 0, 0, 0}}, 71, 61},
+{{.a = {241, 222, 251, 114, 200, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 39, 65},
+{{.a = {31, 243, 190, 4, 207, 198, 249, 59, 167, 127, 93, 64, 0, 0, 0, 0}}, 91, 108},
+{{.a = {201, 35, 222, 208, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 28, 244},
+{{.a = {187, 105, 13, 114, 238, 197, 145, 23, 169, 116, 91, 28, 0, 0, 0, 0}}, 95, 194},
+{{.a = {251, 251, 121, 168, 152, 178, 147, 188, 229, 123, 154, 242, 190, 165, 173, 48}}, 124, 82},
+{{.a = {66, 187, 191, 164, 31, 196, 40, 186, 148, 115, 134, 57, 222, 254, 48, 0}}, 116, 45},
+{{.a = {209, 17, 111, 41, 154, 94, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 48, 224},
+{{.a = {40, 245, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 22, 17},
+{{.a = {72, 121, 151, 83, 170, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 44, 133},
+{{.a = {171, 172, 101, 238, 201, 148, 23, 81, 4, 11, 64, 0, 0, 0, 0, 0}}, 85, 125},
+{{.a = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 3, 42},
+{{.a = {20, 46, 27, 93, 195, 184, 6, 162, 109, 225, 22, 152, 0, 0, 0, 0}}, 96, 140},
+{{.a = {243, 122, 30, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 28, 91},
+{{.a = {89, 250, 80, 72, 148, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 48, 92},
+{{.a = {187, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 13, 125},
+{{.a = {172, 160, 143, 114, 128, 239, 174, 133, 176, 154, 159, 134, 10, 0, 0, 0}}, 106, 249},
+{{.a = {254, 202, 113, 112, 173, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 40, 202},
+{{.a = {80, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 13, 107},
+{{.a = {222, 216, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 13, 124},
+{{.a = {219, 138, 253, 12, 188, 197, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 53, 57},
+{{.a = {124, 41, 173, 8, 202, 192, 61, 254, 174, 48, 239, 112, 0, 0, 0, 0}}, 92, 181},
+{{.a = {195, 236, 245, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 24, 107},
+{{.a = {83, 82, 42, 244, 136, 191, 197, 81, 91, 154, 216, 85, 29, 150, 198, 22}}, 128, 101},
+{{.a = {112, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 4, 102},
+{{.a = {44, 30, 219, 248, 214, 88, 225, 132, 0, 0, 0, 0, 0, 0, 0, 0}}, 62, 136},
+{{.a = {41, 171, 206, 178, 195, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 45, 114},
+{{.a = {159, 15, 180, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 22, 215},
+{{.a = {42, 188, 37, 174, 86, 40, 4, 84, 174, 216, 0, 0, 0, 0, 0, 0}}, 79, 249},
+{{.a = {185, 227, 85, 177, 219, 95, 250, 227, 69, 154, 118, 0, 0, 0, 0, 0}}, 88, 29},
+{{.a = {22, 185, 238, 100, 25, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 44, 71},
+{{.a = {122, 149, 117, 77, 88, 250, 187, 203, 136, 22, 85, 42, 105, 234, 79, 8}}, 127, 112},
+{{.a = {93, 152, 229, 140, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 31, 72},
+{{.a = {129, 37, 165, 167, 241, 24, 37, 40, 2, 128, 0, 0, 0, 0, 0, 0}}, 73, 155},
+{{.a = {30, 202, 177, 3, 253, 202, 164, 248, 0, 0, 0, 0, 0, 0, 0, 0}}, 61, 66},
+{{.a = {176, 25, 220, 120, 194, 228, 10, 45, 225, 142, 192, 96, 0, 0, 0, 0}}, 91, 77},
+{{.a = {96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 3, 109},
+{{.a = {82, 56, 12, 204, 61, 45, 147, 240, 221, 0, 0, 0, 0, 0, 0, 0}}, 72, 37},
+{{.a = {242, 38, 240, 41, 140, 75, 250, 37, 175, 115, 97, 224, 0, 0, 0, 0}}, 91, 56},
+{{.a = {251, 192, 23, 90, 135, 56, 252, 56, 79, 219, 80, 167, 22, 0, 0, 0}}, 103, 5},
+{{.a = {62, 128, 139, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 25, 15},
+{{.a = {214, 1, 84, 232, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 29, 183},
+{{.a = {207, 90, 237, 137, 171, 140, 227, 88, 250, 26, 197, 162, 163, 0, 0, 0}}, 105, 171},
+{{.a = {196, 151, 235, 232, 114, 248, 1, 207, 193, 184, 186, 71, 157, 0, 0, 0}}, 112, 202},
+{{.a = {152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 5, 136},
+{{.a = {9, 174, 211, 200, 120, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 37, 107},
+{{.a = {89, 150, 95, 28, 209, 13, 125, 159, 254, 244, 110, 0, 0, 0, 0, 0}}, 87, 193},
+{{.a = {23, 28, 202, 10, 90, 158, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 52, 4},
+{{.a = {48, 25, 180, 9, 84, 236, 6, 144, 30, 198, 41, 56, 0, 0, 0, 0}}, 96, 68},
+{{.a = {252, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 7, 40},
+{{.a = {20, 165, 57, 130, 164, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 39, 255},
+{{.a = {167, 56, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 18, 108},
+{{.a = {91, 204, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 14, 219},
+{{.a = {24, 46, 9, 4, 170, 150, 56, 130, 127, 120, 118, 104, 168, 48, 0, 0}}, 108, 12},
+{{.a = {156, 60, 245, 247, 189, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 40, 84},
+{{.a = {148, 104, 187, 174, 129, 28, 127, 162, 92, 222, 52, 18, 0, 0, 0, 0}}, 96, 33},
+{{.a = {38, 253, 182, 153, 233, 194, 159, 41, 94, 193, 254, 160, 0, 0, 0, 0}}, 91, 199},
+{{.a = {156, 77, 105, 235, 145, 216, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 46, 52},
+{{.a = {100, 211, 238, 147, 65, 222, 99, 73, 252, 113, 46, 113, 52, 136, 0, 0}}, 113, 184},
+{{.a = {13, 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 18, 124},
+{{.a = {29, 240, 141, 230, 78, 237, 25, 135, 131, 6, 65, 77, 77, 248, 0, 0}}, 109, 128},
+{{.a = {15, 192, 109, 31, 149, 221, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 49, 255},
+{{.a = {80, 185, 170, 71, 41, 58, 158, 106, 253, 7, 2, 184, 173, 0, 0, 0}}, 105, 146},
+{{.a = {16, 229, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 24, 172},
+{{.a = {169, 2, 153, 9, 169, 203, 245, 154, 184, 0, 0, 0, 0, 0, 0, 0}}, 70, 116},
+{{.a = {144, 135, 239, 164, 142, 187, 64, 109, 0, 0, 0, 0, 0, 0, 0, 0}}, 66, 189},
+{{.a = {170, 78, 252, 227, 242, 199, 130, 251, 200, 0, 0, 0, 0, 0, 0, 0}}, 70, 10},
+{{.a = {232, 18, 15, 126, 166, 126, 58, 25, 209, 62, 76, 79, 0, 0, 0, 0}}, 98, 184},
+{{.a = {170, 82, 72, 53, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 33, 98},
+{{.a = {152, 100, 37, 122, 242, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 42, 37},
+{{.a = {174, 231, 230, 33, 71, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 46, 174},
+{{.a = {74, 225, 252, 153, 202, 8, 162, 39, 64, 0, 0, 0, 0, 0, 0, 0}}, 67, 251},
+{{.a = {167, 186, 101, 187, 122, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 42, 115},
+{{.a = {83, 7, 21, 122, 243, 67, 171, 146, 145, 160, 168, 103, 223, 64, 0, 0}}, 107, 252},
+{{.a = {83, 132, 219, 86, 86, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 40, 176},
+{{.a = {22, 113, 72, 102, 73, 16, 236, 57, 197, 122, 31, 0, 0, 0, 0, 0}}, 91, 155},
+{{.a = {250, 59, 64, 35, 72, 112, 159, 85, 200, 5, 193, 39, 152, 185, 148, 16}}, 124, 36},
+{{.a = {220, 21, 48, 164, 224, 121, 17, 69, 10, 118, 106, 0, 0, 0, 0, 0}}, 88, 202},
+{{.a = {160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 5, 208},
+{{.a = {247, 64, 83, 125, 195, 225, 50, 76, 18, 104, 0, 0, 0, 0, 0, 0}}, 77, 158},
+{{.a = {78, 91, 31, 202, 189, 25, 13, 133, 220, 0, 0, 0, 0, 0, 0, 0}}, 72, 136},
+{{.a = {105, 197, 26, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 28, 191},
+{{.a = {14, 31, 154, 242, 241, 231, 55, 151, 223, 56, 134, 255, 113, 206, 69, 0}}, 120, 126},
+{{.a = {247, 193, 58, 176, 16, 71, 31, 120, 213, 104, 231, 83, 26, 118, 91, 135}}, 128, 139},
+{{.a = {136, 32, 240, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 25, 216},
+{{.a = {100, 238, 112, 40, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 29, 93},
+{{.a = {80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 4, 196},
+{{.a = {233, 224, 254, 57, 33, 205, 140, 217, 181, 72, 0, 0, 0, 0, 0, 0}}, 81, 119},
+{{.a = {107, 75, 65, 158, 128, 142, 191, 188, 188, 240, 148, 243, 116, 0, 0, 0}}, 104, 93},
+{{.a = {39, 70, 120, 114, 69, 237, 95, 48, 233, 176, 91, 154, 0, 0, 0, 0}}, 96, 183},
+{{.a = {10, 61, 43, 101, 64, 102, 208, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 52, 207},
+{{.a = {151, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 9, 102},
+{{.a = {210, 241, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 19, 36},
+{{.a = {52, 222, 249, 31, 108, 137, 199, 1, 242, 173, 184, 144, 0, 0, 0, 0}}, 93, 41},
+{{.a = {123, 111, 88, 192, 69, 70, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 47, 70},
+{{.a = {180, 82, 188, 125, 140, 8, 196, 74, 0, 0, 0, 0, 0, 0, 0, 0}}, 63, 218},
+{{.a = {77, 158, 34, 101, 196, 102, 56, 220, 42, 143, 181, 187, 240, 64, 161, 0}}, 120, 226},
+{{.a = {88, 220, 222, 38, 23, 108, 5, 148, 185, 110, 20, 14, 67, 61, 0, 0}}, 114, 25},
+{{.a = {90, 65, 220, 165, 197, 133, 110, 92, 228, 19, 2, 17, 0, 0, 0, 0}}, 98, 6},
+{{.a = {35, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 11, 26},
+{{.a = {103, 123, 49, 209, 228, 229, 144, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 52, 149},
+{{.a = {50, 244, 58, 191, 95, 156, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 46, 127},
+{{.a = {140, 169, 75, 77, 78, 86, 40, 16, 0, 0, 0, 0, 0, 0, 0, 0}}, 62, 144},
+{{.a = {99, 176, 175, 83, 114, 50, 214, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 56, 213},
+{{.a = {19, 208, 211, 76, 85, 176, 247, 64, 0, 0, 0, 0, 0, 0, 0, 0}}, 58, 115},
+{{.a = {153, 28, 188, 113, 211, 116, 7, 178, 136, 205, 96, 0, 0, 0, 0, 0}}, 83, 146},
+{{.a = {160, 180, 220, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 26, 58},
+{{.a = {234, 6, 112, 19, 61, 74, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 47, 222},
+{{.a = {97, 110, 34, 117, 149, 148, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 50, 16},
+{{.a = {99, 173, 119, 73, 250, 30, 144, 30, 128, 0, 0, 0, 0, 0, 0, 0}}, 65, 169},
+{{.a = {169, 134, 111, 89, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 40, 175},
+{{.a = {134, 80, 227, 43, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 32, 3},
+{{.a = {231, 243, 35, 80, 75, 207, 128, 137, 54, 170, 71, 238, 0, 0, 0, 0}}, 96, 2},
+{{.a = {189, 190, 121, 135, 160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 36, 193},
+{{.a = {143, 155, 216, 193, 239, 205, 204, 153, 143, 236, 69, 23, 200, 211, 0, 0}}, 118, 151},
+{{.a = {32, 1, 115, 244, 33, 219, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 51, 182},
+{{.a = {220, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 10, 148},
+{{.a = {206, 87, 135, 235, 116, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 42, 53},
+{{.a = {152, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 11, 87},
+{{.a = {58, 146, 188, 233, 230, 236, 192, 214, 168, 128, 0, 0, 0, 0, 0, 0}}, 73, 235},
+{{.a = {84, 220, 82, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 23, 51},
+{{.a = {106, 145, 142, 42, 186, 186, 58, 1, 48, 98, 165, 131, 48, 156, 192, 0}}, 116, 11},
+{{.a = {53, 219, 120, 242, 166, 214, 81, 130, 64, 0, 0, 0, 0, 0, 0, 0}}, 68, 28},
+{{.a = {240, 120, 76, 163, 32, 197, 181, 251, 98, 220, 29, 226, 0, 0, 0, 0}}, 96, 73},
+{{.a = {234, 197, 12, 160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 28, 216},
+{{.a = {191, 94, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 16, 99},
+{{.a = {200, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 18, 35},
+{{.a = {29, 129, 47, 83, 19, 75, 158, 1, 28, 24, 26, 147, 82, 119, 140, 100}}, 127, 195},
+{{.a = {241, 174, 26, 53, 152, 112, 200, 134, 84, 187, 177, 176, 42, 64, 0, 0}}, 108, 176},
+{{.a = {77, 171, 145, 48, 195, 84, 190, 36, 122, 199, 18, 0, 0, 0, 0, 0}}, 87, 217},
+{{.a = {105, 104, 135, 53, 226, 118, 238, 169, 9, 253, 132, 162, 217, 123, 191, 96}}, 126, 244},
+{{.a = {160, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 3, 125},
+{{.a = {41, 85, 143, 128, 91, 137, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 50, 219},
+{{.a = {116, 110, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 18, 165},
+{{.a = {75, 213, 44, 16, 43, 157, 34, 171, 98, 117, 109, 151, 5, 60, 224, 0}}, 117, 6},
+{{.a = {229, 23, 116, 61, 80, 139, 200, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 53, 47},
+{{.a = {83, 123, 74, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 23, 73},
+{{.a = {151, 243, 45, 217, 216, 158, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 47, 98},
+{{.a = {171, 184, 110, 211, 237, 114, 144, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 52, 21},
+{{.a = {7, 246, 199, 119, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 32, 142},
+{{.a = {103, 47, 70, 17, 31, 232, 44, 75, 145, 155, 100, 216, 0, 0, 0, 0}}, 93, 34},
+{{.a = {65, 170, 169, 100, 167, 147, 142, 251, 20, 64, 0, 0, 0, 0, 0, 0}}, 74, 41},
+{{.a = {235, 6, 229, 248, 151, 137, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 55, 80},
+{{.a = {156, 39, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 22, 11},
+{{.a = {92, 188, 82, 192, 142, 249, 190, 128, 0, 0, 0, 0, 0, 0, 0, 0}}, 58, 254},
+{{.a = {253, 218, 181, 46, 134, 144, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 45, 95},
+{{.a = {189, 19, 31, 244, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 40, 8},
+{{.a = {30, 116, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 14, 212},
+{{.a = {81, 226, 13, 173, 79, 123, 223, 124, 108, 80, 83, 238, 0, 0, 0, 0}}, 95, 217},
+{{.a = {126, 211, 206, 82, 147, 215, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 57, 15},
+{{.a = {42, 229, 135, 197, 196, 243, 94, 181, 133, 34, 16, 0, 0, 0, 0, 0}}, 84, 66},
+{{.a = {68, 210, 158, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 28, 122},
+{{.a = {183, 63, 223, 94, 81, 41, 203, 20, 236, 212, 220, 199, 0, 0, 0, 0}}, 97, 12},
+{{.a = {131, 146, 2, 125, 174, 43, 231, 20, 194, 0, 0, 0, 0, 0, 0, 0}}, 71, 171},
+{{.a = {31, 180, 246, 158, 28, 192, 236, 39, 237, 55, 74, 195, 171, 192, 0, 0}}, 106, 42},
+{{.a = {179, 10, 70, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 28, 194},
+{{.a = {147, 51, 85, 185, 234, 209, 236, 87, 147, 17, 7, 68, 148, 32, 0, 0}}, 107, 237},
+{{.a = {177, 178, 6, 40, 46, 166, 87, 198, 214, 234, 23, 224, 0, 0, 0, 0}}, 93, 151},
+{{.a = {201, 53, 40, 20, 49, 4, 38, 139, 133, 217, 214, 134, 89, 200, 0, 0}}, 109, 238},
+{{.a = {4, 26, 181, 37, 206, 129, 233, 32, 0, 0, 0, 0, 0, 0, 0, 0}}, 59, 128},
+{{.a = {81, 58, 248, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 26, 227},
+{{.a = {18, 238, 250, 161, 57, 246, 208, 118, 14, 76, 73, 25, 65, 22, 152, 120}}, 127, 138},
+{{.a = {31, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 10, 60},
+{{.a = {115, 195, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 18, 148},
+{{.a = {116, 22, 75, 33, 16, 129, 35, 124, 10, 112, 31, 213, 181, 108, 177, 46}}, 128, 129},
+{{.a = {117, 214, 20, 80, 83, 51, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 49, 202},
+{{.a = {120, 75, 124, 149, 120, 123, 242, 151, 181, 164, 128, 0, 0, 0, 0, 0}}, 81, 88},
+{{.a = {87, 238, 168, 62, 88, 166, 52, 104, 219, 169, 93, 128, 0, 0, 0, 0}}, 90, 3},
+{{.a = {237, 44, 224, 146, 52, 85, 245, 192, 65, 137, 37, 95, 156, 176, 0, 0}}, 108, 243},
+{{.a = {214, 241, 51, 63, 73, 61, 193, 165, 23, 108, 0, 0, 0, 0, 0, 0}}, 80, 95},
+{{.a = {87, 242, 21, 157, 45, 188, 36, 62, 66, 243, 64, 0, 0, 0, 0, 0}}, 87, 255},
+{{.a = {0, 97, 220, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 26, 48},
+{{.a = {227, 206, 189, 31, 222, 8, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 50, 38},
+{{.a = {174, 27, 0, 16, 13, 150, 33, 122, 154, 59, 236, 35, 248, 178, 64, 0}}, 115, 20},
+{{.a = {39, 20, 125, 69, 252, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 43, 41},
+{{.a = {141, 232, 1, 12, 125, 229, 168, 14, 125, 116, 180, 0, 0, 0, 0, 0}}, 92, 133},
+{{.a = {93, 238, 40, 228, 254, 203, 251, 6, 60, 82, 243, 242, 0, 0, 0, 0}}, 95, 189},
+{{.a = {44, 115, 200, 17, 146, 223, 115, 253, 126, 206, 152, 90, 0, 0, 0, 0}}, 95, 151},
+{{.a = {213, 58, 235, 255, 6, 163, 61, 10, 224, 0, 0, 0, 0, 0, 0, 0}}, 68, 100},
+{{.a = {25, 86, 139, 116, 190, 88, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 49, 118},
+{{.a = {113, 40, 65, 141, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 34, 164},
+{{.a = {149, 205, 200, 186, 19, 126, 215, 199, 94, 37, 100, 32, 128, 0, 0, 0}}, 98, 71},
+{{.a = {39, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 9, 251},
+{{.a = {81, 87, 80, 173, 163, 166, 104, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 57, 51},
+{{.a = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 3, 185},
+{{.a = {140, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 10, 144},
+{{.a = {6, 42, 1, 178, 250, 53, 186, 178, 114, 121, 192, 0, 0, 0, 0, 0}}, 84, 51},
+{{.a = {2, 17, 234, 51, 169, 5, 219, 149, 245, 237, 4, 0, 0, 0, 0, 0}}, 87, 32},
+{{.a = {112, 187, 173, 17, 229, 171, 225, 170, 8, 0, 0, 0, 0, 0, 0, 0}}, 70, 137},
+{{.a = {203, 71, 140, 237, 113, 96, 123, 16, 0, 0, 0, 0, 0, 0, 0, 0}}, 60, 2},
+{{.a = {99, 138, 207, 2, 244, 25, 211, 98, 0, 0, 0, 0, 0, 0, 0, 0}}, 63, 163},
+{{.a = {114, 42, 98, 246, 252, 48, 233, 118, 63, 226, 157, 226, 192, 0, 0, 0}}, 100, 162},
+{{.a = {161, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 10, 192},
+{{.a = {233, 70, 240, 45, 240, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 36, 185},
+{{.a = {28, 123, 31, 176, 235, 229, 169, 192, 0, 0, 0, 0, 0, 0, 0, 0}}, 59, 51},
+{{.a = {146, 197, 243, 235, 243, 56, 140, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 54, 93},
+{{.a = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 2, 159},
+{{.a = {141, 92, 13, 27, 87, 241, 171, 143, 220, 0, 0, 0, 0, 0, 0, 0}}, 72, 189},
+{{.a = {164, 151, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 21, 248},
+{{.a = {35, 188, 248, 79, 39, 151, 232, 215, 248, 245, 185, 144, 78, 102, 173, 128}}, 123, 38},
+{{.a = {193, 232, 166, 60, 62, 80, 230, 225, 165, 240, 0, 0, 0, 0, 0, 0}}, 76, 167},
+{{.a = {109, 229, 118, 155, 43, 154, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 51, 28},
+{{.a = {160, 62, 63, 212, 218, 138, 154, 108, 163, 127, 197, 237, 183, 44, 140, 192}}, 125, 37},
+{{.a = {196, 37, 51, 146, 26, 85, 53, 31, 216, 141, 52, 218, 153, 32, 0, 0}}, 107, 234},
+{{.a = {228, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 9, 70},
+{{.a = {154, 248, 20, 242, 154, 244, 63, 17, 121, 52, 70, 84, 118, 208, 0, 0}}, 108, 50},
+{{.a = {41, 100, 27, 84, 106, 112, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 51, 171},
+{{.a = {81, 99, 197, 139, 30, 150, 230, 216, 81, 190, 84, 165, 29, 64, 128, 0}}, 113, 236},
+{{.a = {112, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 4, 3},
+{{.a = {164, 119, 253, 126, 160, 249, 183, 191, 119, 111, 224, 0, 0, 0, 0, 0}}, 86, 64},
+{{.a = {138, 58, 198, 254, 0, 197, 60, 91, 132, 199, 181, 251, 78, 160, 0, 0}}, 108, 213},
+{{.a = {209, 89, 168, 236, 146, 169, 100, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 54, 15},
+{{.a = {131, 210, 208, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 20, 145},
+{{.a = {165, 190, 157, 7, 131, 5, 147, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 57, 27},
+{{.a = {179, 226, 57, 204, 187, 70, 52, 81, 119, 162, 229, 42, 47, 185, 9, 162}}, 127, 75},
+{{.a = {98, 235, 155, 51, 107, 167, 127, 137, 254, 246, 162, 171, 180, 13, 233, 0}}, 123, 76},
+{{.a = {107, 79, 76, 90, 94, 151, 155, 31, 33, 115, 19, 204, 98, 115, 0, 0}}, 113, 247},
+{{.a = {143, 46, 30, 175, 224, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 43, 121},
+{{.a = {155, 85, 217, 180, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 30, 214},
+{{.a = {58, 62, 156, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 26, 221},
+{{.a = {92, 155, 53, 3, 39, 108, 155, 200, 0, 0, 0, 0, 0, 0, 0, 0}}, 63, 102},
+{{.a = {64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 2, 191},
+{{.a = {63, 134, 251, 59, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, 39, 197},
+{{.a = {234, 149, 220, 106, 0, 144, 214, 128, 35, 102, 0, 0, 0, 0, 0, 0}}, 79, 106},
};
#define NUM_ROUTE_ENTRIES RTE_DIM(large_route_table)
@@ -1067,19 +1068,19 @@ static inline void mask_ip6_prefix(uint8_t *ip_out,
/* check if IPv6 address ip[] match the rule with IPv6 address ip_rule[]
* and depth. if matched, return 0, else return -1.
*/
-static inline int check_lpm6_rule(uint8_t *ip,
- const uint8_t *ip_rule, uint8_t depth)
+static inline int check_lpm6_rule(const struct rte_ipv6_addr *ip,
+ const struct rte_ipv6_addr *ip_rule, uint8_t depth)
{
int k;
uint8_t mask;
for (k = 0; k < 16; k++) {
if (depth >= 8) {
- if (ip[k] != ip_rule[k])
+ if (ip->a[k] != ip_rule->a[k])
return -1;
} else if (depth > 0) {
mask = (uint8_t)((unsigned int)(-1) << (8 - depth));
- if ((ip[k] & mask) == (ip_rule[k] & mask))
+ if ((ip->a[k] & mask) == (ip_rule->a[k] & mask))
return 0;
else
return -1;
@@ -1098,7 +1099,7 @@ static inline int check_lpm6_rule(uint8_t *ip,
* if found that some item in rule[] is matched return 0,
* else return -1;
*/
-static int get_next_hop(uint8_t *ip, uint8_t *next_hop,
+static int get_next_hop(const struct rte_ipv6_addr *ip, uint8_t *next_hop,
const struct rules_tbl_entry *rule, int rule_num)
{
int i;
@@ -1107,7 +1108,7 @@ static int get_next_hop(uint8_t *ip, uint8_t *next_hop,
for (i = 0; i < rule_num; i++) {
if (rule[i].depth >= max_depth) {
- result = check_lpm6_rule(ip, rule[i].ip, rule[i].depth);
+ result = check_lpm6_rule(ip, &rule[i].ip, rule[i].depth);
if (result == 0) {
*next_hop = rule[i].next_hop;
max_depth = rule[i].depth;
@@ -1131,12 +1132,12 @@ static void generate_large_ips_table(int gen_expected_next_hop)
for (i = 0; i < NUM_IPS_ENTRIES; i++) {
for (j = 0; j < 16; j++)
- large_ips_table[i].ip[j] = rte_rand();
+ large_ips_table[i].ip.a[j] = rte_rand();
}
for (k = j = 0, i = 0; i < NUM_IPS_ENTRIES; i++) {
- mask_ip6_prefix(large_ips_table[i].ip,
- large_route_table[j].ip, large_route_table[j].depth);
+ mask_ip6_prefix(large_ips_table[i].ip.a,
+ large_route_table[j].ip.a, large_route_table[j].depth);
k++;
if (k == (NUM_IPS_ENTRIES / NUM_ROUTE_ENTRIES)) {
j++;
@@ -1150,7 +1151,7 @@ static void generate_large_ips_table(int gen_expected_next_hop)
return;
for (k = 0; k < NUM_IPS_ENTRIES; k++)
- get_next_hop(large_ips_table[k].ip,
+ get_next_hop(&large_ips_table[k].ip,
&(large_ips_table[k].next_hop),
large_route_table,
NUM_ROUTE_ENTRIES);
diff --git a/app/test/test_lpm6_perf.c b/app/test/test_lpm6_perf.c
index c847dcb18375..0f9e3698c4da 100644
--- a/app/test/test_lpm6_perf.c
+++ b/app/test/test_lpm6_perf.c
@@ -82,7 +82,7 @@ test_lpm6_perf(void)
begin = rte_rdtsc();
for (i = 0; i < NUM_ROUTE_ENTRIES; i++) {
- if (rte_lpm6_add(lpm, large_route_table[i].ip,
+ if (rte_lpm6_add(lpm, &large_route_table[i].ip,
large_route_table[i].depth, next_hop_add) == 0)
status++;
}
@@ -101,7 +101,7 @@ test_lpm6_perf(void)
begin = rte_rdtsc();
for (j = 0; j < NUM_IPS_ENTRIES; j ++) {
- if (rte_lpm6_lookup(lpm, large_ips_table[j].ip,
+ if (rte_lpm6_lookup(lpm, &large_ips_table[j].ip,
&next_hop_return) != 0)
count++;
}
@@ -117,11 +117,11 @@ test_lpm6_perf(void)
total_time = 0;
count = 0;
- uint8_t ip_batch[NUM_IPS_ENTRIES][16];
+ struct rte_ipv6_addr ip_batch[NUM_IPS_ENTRIES];
int32_t next_hops[NUM_IPS_ENTRIES];
for (i = 0; i < NUM_IPS_ENTRIES; i++)
- memcpy(ip_batch[i], large_ips_table[i].ip, 16);
+ memcpy(&ip_batch[i], &large_ips_table[i].ip, 16);
for (i = 0; i < ITERATIONS; i ++) {
@@ -144,7 +144,7 @@ test_lpm6_perf(void)
for (i = 0; i < NUM_ROUTE_ENTRIES; i++) {
/* rte_lpm_delete(lpm, ip, depth) */
- status += rte_lpm6_delete(lpm, large_route_table[i].ip,
+ status += rte_lpm6_delete(lpm, &large_route_table[i].ip,
large_route_table[i].depth);
}
diff --git a/app/test/test_rib6.c b/app/test/test_rib6.c
index 33596fddb4e5..f3b6f1474348 100644
--- a/app/test/test_rib6.c
+++ b/app/test/test_rib6.c
@@ -6,7 +6,7 @@
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
-#include <rte_ip.h>
+#include <rte_ip6.h>
#include <rte_rib6.h>
#include "test.h"
@@ -118,14 +118,14 @@ test_insert_invalid(void)
struct rte_rib6 *rib = NULL;
struct rte_rib6_node *node, *node1;
struct rte_rib6_conf config;
- uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE] = {0};
+ struct rte_ipv6_addr ip = {0};
uint8_t depth = 24;
config.max_nodes = MAX_RULES;
config.ext_sz = 0;
/* rte_rib6_insert: rib == NULL */
- node = rte_rib6_insert(NULL, ip, depth);
+ node = rte_rib6_insert(NULL, &ip, depth);
RTE_TEST_ASSERT(node == NULL,
"Call succeeded with invalid parameters\n");
@@ -134,14 +134,14 @@ test_insert_invalid(void)
RTE_TEST_ASSERT(rib != NULL, "Failed to create RIB\n");
/* rte_rib6_insert: depth > MAX_DEPTH */
- node = rte_rib6_insert(rib, ip, MAX_DEPTH + 1);
+ node = rte_rib6_insert(rib, &ip, MAX_DEPTH + 1);
RTE_TEST_ASSERT(node == NULL,
"Call succeeded with invalid parameters\n");
/* insert the same ip/depth twice*/
- node = rte_rib6_insert(rib, ip, depth);
+ node = rte_rib6_insert(rib, &ip, depth);
RTE_TEST_ASSERT(node != NULL, "Failed to insert rule\n");
- node1 = rte_rib6_insert(rib, ip, depth);
+ node1 = rte_rib6_insert(rib, &ip, depth);
RTE_TEST_ASSERT(node1 == NULL,
"Call succeeded with invalid parameters\n");
@@ -162,9 +162,10 @@ test_get_fn(void)
struct rte_rib6_node *node;
struct rte_rib6_conf config;
void *ext;
- uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE] = {192, 0, 2, 0, 0, 0, 0, 0,
- 0, 0, 0, 0, 0, 0, 0, 0};
- uint8_t ip_ret[RTE_RIB6_IPV6_ADDR_SIZE];
+ struct rte_ipv6_addr ip = {
+ .a = {192, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
+ };
+ struct rte_ipv6_addr ip_ret;
uint64_t nh_set = 10;
uint64_t nh_ret;
uint8_t depth = 24;
@@ -177,11 +178,11 @@ test_get_fn(void)
rib = rte_rib6_create(__func__, SOCKET_ID_ANY, &config);
RTE_TEST_ASSERT(rib != NULL, "Failed to create RIB\n");
- node = rte_rib6_insert(rib, ip, depth);
+ node = rte_rib6_insert(rib, &ip, depth);
RTE_TEST_ASSERT(node != NULL, "Failed to insert rule\n");
/* test rte_rib6_get_ip() with incorrect args */
- ret = rte_rib6_get_ip(NULL, ip_ret);
+ ret = rte_rib6_get_ip(NULL, &ip_ret);
RTE_TEST_ASSERT(ret < 0,
"Call succeeded with invalid parameters\n");
ret = rte_rib6_get_ip(node, NULL);
@@ -215,8 +216,8 @@ test_get_fn(void)
"Call succeeded with invalid parameters\n");
/* check the return values */
- ret = rte_rib6_get_ip(node, ip_ret);
- RTE_TEST_ASSERT((ret == 0) && (rte_rib6_is_equal(ip_ret, ip)),
+ ret = rte_rib6_get_ip(node, &ip_ret);
+ RTE_TEST_ASSERT((ret == 0) && (rte_rib6_is_equal(ip_ret.a, ip.a)),
"Failed to get proper node ip\n");
ret = rte_rib6_get_depth(node, &depth_ret);
RTE_TEST_ASSERT((ret == 0) && (depth_ret == depth),
@@ -243,8 +244,9 @@ test_basic(void)
struct rte_rib6_node *node;
struct rte_rib6_conf config;
- uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE] = {192, 0, 2, 0, 0, 0, 0, 0,
- 0, 0, 0, 0, 0, 0, 0, 0};
+ struct rte_ipv6_addr ip = {
+ .a = {192, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
+ };
uint64_t next_hop_add = 10;
uint64_t next_hop_return;
uint8_t depth = 24;
@@ -256,21 +258,21 @@ test_basic(void)
rib = rte_rib6_create(__func__, SOCKET_ID_ANY, &config);
RTE_TEST_ASSERT(rib != NULL, "Failed to create RIB\n");
- node = rte_rib6_insert(rib, ip, depth);
+ node = rte_rib6_insert(rib, &ip, depth);
RTE_TEST_ASSERT(node != NULL, "Failed to insert rule\n");
status = rte_rib6_set_nh(node, next_hop_add);
RTE_TEST_ASSERT(status == 0,
"Failed to set rte_rib_node field\n");
- node = rte_rib6_lookup(rib, ip);
+ node = rte_rib6_lookup(rib, &ip);
RTE_TEST_ASSERT(node != NULL, "Failed to lookup\n");
status = rte_rib6_get_nh(node, &next_hop_return);
RTE_TEST_ASSERT((status == 0) && (next_hop_add == next_hop_return),
"Failed to get proper nexthop\n");
- node = rte_rib6_lookup_exact(rib, ip, depth);
+ node = rte_rib6_lookup_exact(rib, &ip, depth);
RTE_TEST_ASSERT(node != NULL,
"Failed to lookup\n");
@@ -278,12 +280,12 @@ test_basic(void)
RTE_TEST_ASSERT((status == 0) && (next_hop_add == next_hop_return),
"Failed to get proper nexthop\n");
- rte_rib6_remove(rib, ip, depth);
+ rte_rib6_remove(rib, &ip, depth);
- node = rte_rib6_lookup(rib, ip);
+ node = rte_rib6_lookup(rib, &ip);
RTE_TEST_ASSERT(node == NULL,
"Lookup returns non existent rule\n");
- node = rte_rib6_lookup_exact(rib, ip, depth);
+ node = rte_rib6_lookup_exact(rib, &ip, depth);
RTE_TEST_ASSERT(node == NULL,
"Lookup returns non existent rule\n");
@@ -299,12 +301,15 @@ test_tree_traversal(void)
struct rte_rib6_node *node;
struct rte_rib6_conf config;
- uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE] = {10, 0, 2, 130, 0, 0, 0, 0,
- 0, 0, 0, 0, 0, 0, 0, 0};
- uint8_t ip1[RTE_RIB6_IPV6_ADDR_SIZE] = {10, 0, 2, 0, 0, 0, 0, 0,
- 0, 0, 0, 0, 0, 0, 0, 0};
- uint8_t ip2[RTE_RIB6_IPV6_ADDR_SIZE] = {10, 0, 2, 130, 0, 0, 0, 0,
- 0, 0, 0, 0, 0, 0, 0, 80};
+ struct rte_ipv6_addr ip = {
+ .a = {10, 0, 2, 130, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
+ };
+ struct rte_ipv6_addr ip1 = {
+ .a = {10, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
+ };
+ struct rte_ipv6_addr ip2 = {
+ .a = {10, 0, 2, 130, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 80},
+ };
uint8_t depth = 126;
config.max_nodes = MAX_RULES;
@@ -313,13 +318,13 @@ test_tree_traversal(void)
rib = rte_rib6_create(__func__, SOCKET_ID_ANY, &config);
RTE_TEST_ASSERT(rib != NULL, "Failed to create RIB\n");
- node = rte_rib6_insert(rib, ip1, depth);
+ node = rte_rib6_insert(rib, &ip1, depth);
RTE_TEST_ASSERT(node != NULL, "Failed to insert rule\n");
- node = rte_rib6_insert(rib, ip2, depth);
+ node = rte_rib6_insert(rib, &ip2, depth);
RTE_TEST_ASSERT(node != NULL, "Failed to insert rule\n");
node = NULL;
- node = rte_rib6_get_nxt(rib, ip, 32, node, RTE_RIB6_GET_NXT_ALL);
+ node = rte_rib6_get_nxt(rib, &ip, 32, node, RTE_RIB6_GET_NXT_ALL);
RTE_TEST_ASSERT(node != NULL, "Failed to get rib_node\n");
rte_rib6_free(rib);
diff --git a/app/test/test_table_combined.c b/app/test/test_table_combined.c
index 1c2efe649ded..04503baa5180 100644
--- a/app/test/test_table_combined.c
+++ b/app/test/test_table_combined.c
@@ -362,7 +362,7 @@ test_table_lpm_ipv6_combined(void)
struct rte_table_lpm_ipv6_key lpm_ipv6_key = {
.depth = 16,
};
- memset(lpm_ipv6_key.ip, 0xad, 16);
+ memset(&lpm_ipv6_key.ip, 0xad, 16);
struct table_packets table_packets;
diff --git a/app/test/test_table_tables.c b/app/test/test_table_tables.c
index 26908e6112fc..920aa555cbd2 100644
--- a/app/test/test_table_tables.c
+++ b/app/test/test_table_tables.c
@@ -525,10 +525,10 @@ test_table_lpm_ipv6(void)
/* Add */
struct rte_table_lpm_ipv6_key lpm_key;
- lpm_key.ip[0] = 0xad;
- lpm_key.ip[1] = 0xad;
- lpm_key.ip[2] = 0xad;
- lpm_key.ip[3] = 0xad;
+ lpm_key.ip.a[0] = 0xad;
+ lpm_key.ip.a[1] = 0xad;
+ lpm_key.ip.a[2] = 0xad;
+ lpm_key.ip.a[3] = 0xad;
table = rte_table_lpm_ipv6_ops.f_create(&lpm_params, 0, entry_size);
if (table == NULL)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 439867289cf3..ee958c689b3a 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -70,36 +70,16 @@ Deprecation Notices
- ``struct rte_flow_item_icmp6_nd_na``
- ``struct rte_flow_action_set_ipv6``
- ``struct rte_flow_tunnel``
- fib
- - ``rte_fib6_add()``
- - ``rte_fib6_delete()``
- - ``rte_fib6_lookup_bulk()``
gro
- ``struct tcp6_flow_key``
hash
- ``struct rte_ipv6_tuple``
ipsec
- ``struct rte_ipsec_sadv6_key``
- lpm
- - ``rte_lpm6_add()``
- - ``rte_lpm6_is_rule_present()``
- - ``rte_lpm6_delete()``
- - ``rte_lpm6_delete_bulk_func()``
- - ``rte_lpm6_lookup()``
- - ``rte_lpm6_lookup_bulk_func()``
node
- ``rte_node_ip6_route_add()``
pipeline
- ``struct rte_table_action_ipv6_header``
- rib
- - ``rte_rib6_lookup()``
- - ``rte_rib6_lookup_exact()``
- - ``rte_rib6_get_nxt()``
- - ``rte_rib6_insert()``
- - ``rte_rib6_remove()``
- - ``rte_rib6_get_ip()``
- table
- - ``struct rte_table_lpm_ipv6_key``
* net, ethdev: The flow item ``RTE_FLOW_ITEM_TYPE_VXLAN_GPE``
is replaced with ``RTE_FLOW_ITEM_TYPE_VXLAN``.
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 2d5b29b5a2d4..984ef5e35fb9 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -87,6 +87,8 @@ API Changes
* net: IPv6 related symbols were moved from ``<rte_ip.h>`` to the new ``<rte_ip6.h>`` header.
* net: The ``rte_ipv6_hdr`` structure was modified to use ``struct rte_ipv6_addr`` instead of
``uint8_t[16]`` fields.
+* rib6,fib6,lpm6: All public API functions were modified to use ``struct rte_ipv6_addr`` instead of
+ ``uint8_t[16]`` parameters.
ABI Changes
-----------
diff --git a/examples/ip_fragmentation/main.c b/examples/ip_fragmentation/main.c
index 4c0fa5054a2e..611b5818d88c 100644
--- a/examples/ip_fragmentation/main.c
+++ b/examples/ip_fragmentation/main.c
@@ -184,21 +184,21 @@ struct l3fwd_ipv4_route l3fwd_ipv4_route_array[] = {
*/
struct l3fwd_ipv6_route {
- uint8_t ip[IPV6_ADDR_LEN];
+ struct rte_ipv6_addr ip;
uint8_t depth;
uint8_t if_out;
};
/* Default l3fwd_ipv6_route_array table. 8< */
static struct l3fwd_ipv6_route l3fwd_ipv6_route_array[] = {
- {{1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, 48, 0},
- {{2,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, 48, 1},
- {{3,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, 48, 2},
- {{4,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, 48, 3},
- {{5,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, 48, 4},
- {{6,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, 48, 5},
- {{7,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, 48, 6},
- {{8,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, 48, 7},
+ {{.a = {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}}, 48, 0},
+ {{.a = {2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}}, 48, 1},
+ {{.a = {3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}}, 48, 2},
+ {{.a = {4, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}}, 48, 3},
+ {{.a = {5, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}}, 48, 4},
+ {{.a = {6, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}}, 48, 5},
+ {{.a = {7, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}}, 48, 6},
+ {{.a = {8, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}}, 48, 7},
};
/* >8 End of default l3fwd_ipv6_route_array table. */
@@ -311,7 +311,7 @@ l3fwd_simple_forward(struct rte_mbuf *m, struct lcore_queue_conf *qconf,
ip_hdr = rte_pktmbuf_mtod(m, struct rte_ipv6_hdr *);
/* Find destination port */
- if (rte_lpm6_lookup(rxq->lpm6, ip_hdr->dst_addr.a,
+ if (rte_lpm6_lookup(rxq->lpm6, &ip_hdr->dst_addr,
&next_hop) == 0 &&
(enabled_port_mask & 1 << next_hop) != 0) {
port_out = next_hop;
@@ -751,7 +751,7 @@ init_routing_table(void)
/* populate the LPM6 table */
for (i = 0; i < RTE_DIM(l3fwd_ipv6_route_array); i++) {
ret = rte_lpm6_add(lpm6,
- l3fwd_ipv6_route_array[i].ip,
+ &l3fwd_ipv6_route_array[i].ip,
l3fwd_ipv6_route_array[i].depth,
l3fwd_ipv6_route_array[i].if_out);
@@ -764,7 +764,7 @@ init_routing_table(void)
RTE_LOG(INFO, IP_FRAG, "Socket %i: adding route " IPv6_BYTES_FMT
"/%d (port %d)\n",
socket,
- IPv6_BYTES(l3fwd_ipv6_route_array[i].ip),
+ IPv6_BYTES(l3fwd_ipv6_route_array[i].ip.a),
l3fwd_ipv6_route_array[i].depth,
l3fwd_ipv6_route_array[i].if_out);
}
diff --git a/examples/ip_pipeline/thread.c b/examples/ip_pipeline/thread.c
index 8a912dc1b392..9d8082b73080 100644
--- a/examples/ip_pipeline/thread.c
+++ b/examples/ip_pipeline/thread.c
@@ -2563,7 +2563,7 @@ match_convert(struct table_rule_match *mh,
ml->lpm_ipv4.ip = mh->match.lpm.ipv4;
ml->lpm_ipv4.depth = mh->match.lpm.depth;
} else {
- memcpy(ml->lpm_ipv6.ip,
+ memcpy(&ml->lpm_ipv6.ip,
mh->match.lpm.ipv6, sizeof(ml->lpm_ipv6.ip));
ml->lpm_ipv6.depth = mh->match.lpm.depth;
}
diff --git a/examples/ip_reassembly/main.c b/examples/ip_reassembly/main.c
index 4da692eb23e6..06383cc36289 100644
--- a/examples/ip_reassembly/main.c
+++ b/examples/ip_reassembly/main.c
@@ -205,21 +205,21 @@ struct l3fwd_ipv4_route l3fwd_ipv4_route_array[] = {
*/
struct l3fwd_ipv6_route {
- uint8_t ip[IPV6_ADDR_LEN];
+ struct rte_ipv6_addr ip;
uint8_t depth;
uint8_t if_out;
};
/* Default l3fwd_ipv6_route_array table. 8< */
static struct l3fwd_ipv6_route l3fwd_ipv6_route_array[] = {
- {{1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, 48, 0},
- {{2,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, 48, 1},
- {{3,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, 48, 2},
- {{4,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, 48, 3},
- {{5,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, 48, 4},
- {{6,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, 48, 5},
- {{7,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, 48, 6},
- {{8,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, 48, 7},
+ {{.a = {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}}, 48, 0},
+ {{.a = {2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}}, 48, 1},
+ {{.a = {3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}}, 48, 2},
+ {{.a = {4, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}}, 48, 3},
+ {{.a = {5, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}}, 48, 4},
+ {{.a = {6, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}}, 48, 5},
+ {{.a = {7, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}}, 48, 6},
+ {{.a = {8, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}}, 48, 7},
};
/* >8 End of default l3fwd_ipv6_route_array table. */
@@ -400,7 +400,7 @@ reassemble(struct rte_mbuf *m, uint16_t portid, uint32_t queue,
}
/* Find destination port */
- if (rte_lpm6_lookup(rxq->lpm6, ip_hdr->dst_addr.a,
+ if (rte_lpm6_lookup(rxq->lpm6, &ip_hdr->dst_addr,
&next_hop) == 0 &&
(enabled_port_mask & 1 << next_hop) != 0) {
dst_port = next_hop;
@@ -797,7 +797,7 @@ init_routing_table(void)
/* populate the LPM6 table */
for (i = 0; i < RTE_DIM(l3fwd_ipv6_route_array); i++) {
ret = rte_lpm6_add(lpm6,
- l3fwd_ipv6_route_array[i].ip,
+ &l3fwd_ipv6_route_array[i].ip,
l3fwd_ipv6_route_array[i].depth,
l3fwd_ipv6_route_array[i].if_out);
@@ -810,7 +810,7 @@ init_routing_table(void)
RTE_LOG(INFO, IP_RSMBL, "Socket %i: adding route " IPv6_BYTES_FMT
"/%d (port %d)\n",
socket,
- IPv6_BYTES(l3fwd_ipv6_route_array[i].ip),
+ IPv6_BYTES(l3fwd_ipv6_route_array[i].ip.a),
l3fwd_ipv6_route_array[i].depth,
l3fwd_ipv6_route_array[i].if_out);
}
diff --git a/examples/ipsec-secgw/ipsec_lpm_neon.h b/examples/ipsec-secgw/ipsec_lpm_neon.h
index 9573f53ae957..865b9624a86e 100644
--- a/examples/ipsec-secgw/ipsec_lpm_neon.h
+++ b/examples/ipsec-secgw/ipsec_lpm_neon.h
@@ -5,6 +5,8 @@
#ifndef IPSEC_LPM_NEON_H
#define IPSEC_LPM_NEON_H
+#include <rte_ip6.h>
+
#include <arm_neon.h>
#include "ipsec_neon.h"
@@ -114,7 +116,7 @@ process_single_pkt(struct rt_ctx *rt_ctx, struct rte_mbuf *pkt,
static inline void
route6_pkts_neon(struct rt_ctx *rt_ctx, struct rte_mbuf **pkts, int nb_rx)
{
- uint8_t dst_ip6[MAX_PKT_BURST][16];
+ struct rte_ipv6_addr dst_ip6[MAX_PKT_BURST];
uint16_t dst_port[MAX_PKT_BURST];
struct rte_ether_hdr *eth_hdr;
struct rte_ipv6_hdr *ipv6_hdr;
@@ -142,8 +144,7 @@ route6_pkts_neon(struct rt_ctx *rt_ctx, struct rte_mbuf **pkts, int nb_rx)
* required to get the hop
*/
ipv6_hdr = (struct rte_ipv6_hdr *)(eth_hdr + 1);
- memcpy(&dst_ip6[lpm_pkts][0],
- ipv6_hdr->dst_addr, 16);
+ memcpy(&dst_ip6[lpm_pkts], &ipv6_hdr->dst_addr, 16);
lpm_pkts++;
}
}
diff --git a/examples/ipsec-secgw/ipsec_worker.c b/examples/ipsec-secgw/ipsec_worker.c
index c9c43ebd2b99..3a83b7c0bf4b 100644
--- a/examples/ipsec-secgw/ipsec_worker.c
+++ b/examples/ipsec-secgw/ipsec_worker.c
@@ -298,7 +298,7 @@ route4_pkt(struct rte_mbuf *pkt, struct rt_ctx *rt_ctx)
static inline uint16_t
route6_pkt(struct rte_mbuf *pkt, struct rt_ctx *rt_ctx)
{
- uint8_t dst_ip[16];
+ struct rte_ipv6_addr dst_ip;
uint8_t *ip6_dst;
uint16_t offset;
uint32_t hop;
@@ -306,9 +306,9 @@ route6_pkt(struct rte_mbuf *pkt, struct rt_ctx *rt_ctx)
offset = RTE_ETHER_HDR_LEN + offsetof(struct ip6_hdr, ip6_dst);
ip6_dst = rte_pktmbuf_mtod_offset(pkt, uint8_t *, offset);
- memcpy(&dst_ip[0], ip6_dst, 16);
+ memcpy(&dst_ip, ip6_dst, 16);
- ret = rte_lpm6_lookup((struct rte_lpm6 *)rt_ctx, dst_ip, &hop);
+ ret = rte_lpm6_lookup((struct rte_lpm6 *)rt_ctx, &dst_ip, &hop);
if (ret == 0) {
/* We have a hit */
diff --git a/examples/ipsec-secgw/ipsec_worker.h b/examples/ipsec-secgw/ipsec_worker.h
index 8f6a1c06aa7f..8f9616129362 100644
--- a/examples/ipsec-secgw/ipsec_worker.h
+++ b/examples/ipsec-secgw/ipsec_worker.h
@@ -560,7 +560,7 @@ static __rte_always_inline void
route6_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint32_t nb_pkts)
{
int32_t hop[MAX_PKT_BURST * 2];
- uint8_t dst_ip[MAX_PKT_BURST * 2][16];
+ struct rte_ipv6_addr dst_ip[MAX_PKT_BURST * 2];
struct rte_ether_hdr *ethhdr;
uint8_t *ip6_dst;
uint32_t pkt_hop = 0;
@@ -586,7 +586,7 @@ route6_pkts(struct rt_ctx *rt_ctx, struct rte_mbuf *pkts[], uint32_t nb_pkts)
offset = offsetof(struct ip6_hdr, ip6_dst);
ip6_dst = rte_pktmbuf_mtod_offset(pkt, uint8_t *,
offset);
- memcpy(&dst_ip[lpm_pkts][0], ip6_dst, 16);
+ memcpy(&dst_ip[lpm_pkts], ip6_dst, 16);
lpm_pkts++;
}
}
diff --git a/examples/ipsec-secgw/rt.c b/examples/ipsec-secgw/rt.c
index ce854ccb6018..059fc0c8f28c 100644
--- a/examples/ipsec-secgw/rt.c
+++ b/examples/ipsec-secgw/rt.c
@@ -25,7 +25,7 @@ struct ip4_route {
};
struct ip6_route {
- uint8_t ip[16];
+ struct rte_ipv6_addr ip;
uint8_t depth;
uint8_t if_out;
};
@@ -99,7 +99,7 @@ parse_rt_tokens(char **tokens, uint32_t n_tokens,
tokens[ti]);
if (status->status < 0)
return;
- memcpy(route_ipv6->ip, ip.s6_addr, 16);
+ memcpy(&route_ipv6->ip, ip.s6_addr, 16);
route_ipv6->depth = (uint8_t)depth;
}
}
@@ -183,7 +183,7 @@ rt_init(struct socket_ctx *ctx, int32_t socket_id)
/* populate the LPM table */
for (i = 0; i < nb_rt_ip6; i++) {
- ret = rte_lpm6_add(lpm6, rt_ip6[i].ip, rt_ip6[i].depth,
+ ret = rte_lpm6_add(lpm6, &rt_ip6[i].ip, rt_ip6[i].depth,
rt_ip6[i].if_out);
if (ret < 0)
rte_exit(EXIT_FAILURE, "Fail to add entry num %u to %s "
@@ -191,14 +191,14 @@ rt_init(struct socket_ctx *ctx, int32_t socket_id)
printf("LPM6: Adding route "
" %hx:%hx:%hx:%hx:%hx:%hx:%hx:%hx/%hhx (%hhx)\n",
- (uint16_t)((rt_ip6[i].ip[0] << 8) | rt_ip6[i].ip[1]),
- (uint16_t)((rt_ip6[i].ip[2] << 8) | rt_ip6[i].ip[3]),
- (uint16_t)((rt_ip6[i].ip[4] << 8) | rt_ip6[i].ip[5]),
- (uint16_t)((rt_ip6[i].ip[6] << 8) | rt_ip6[i].ip[7]),
- (uint16_t)((rt_ip6[i].ip[8] << 8) | rt_ip6[i].ip[9]),
- (uint16_t)((rt_ip6[i].ip[10] << 8) | rt_ip6[i].ip[11]),
- (uint16_t)((rt_ip6[i].ip[12] << 8) | rt_ip6[i].ip[13]),
- (uint16_t)((rt_ip6[i].ip[14] << 8) | rt_ip6[i].ip[15]),
+ (uint16_t)((rt_ip6[i].ip.a[0] << 8) | rt_ip6[i].ip.a[1]),
+ (uint16_t)((rt_ip6[i].ip.a[2] << 8) | rt_ip6[i].ip.a[3]),
+ (uint16_t)((rt_ip6[i].ip.a[4] << 8) | rt_ip6[i].ip.a[5]),
+ (uint16_t)((rt_ip6[i].ip.a[6] << 8) | rt_ip6[i].ip.a[7]),
+ (uint16_t)((rt_ip6[i].ip.a[8] << 8) | rt_ip6[i].ip.a[9]),
+ (uint16_t)((rt_ip6[i].ip.a[10] << 8) | rt_ip6[i].ip.a[11]),
+ (uint16_t)((rt_ip6[i].ip.a[12] << 8) | rt_ip6[i].ip.a[13]),
+ (uint16_t)((rt_ip6[i].ip.a[14] << 8) | rt_ip6[i].ip.a[15]),
rt_ip6[i].depth, rt_ip6[i].if_out);
}
diff --git a/examples/l3fwd/l3fwd_fib.c b/examples/l3fwd/l3fwd_fib.c
index 4a9534e9dbef..046aa1027b5c 100644
--- a/examples/l3fwd/l3fwd_fib.c
+++ b/examples/l3fwd/l3fwd_fib.c
@@ -46,7 +46,7 @@ static struct rte_fib6 *ipv6_l3fwd_fib_lookup_struct[NB_SOCKETS];
static inline void
fib_parse_packet(struct rte_mbuf *mbuf,
uint32_t *ipv4, uint32_t *ipv4_cnt,
- uint8_t ipv6[RTE_FIB6_IPV6_ADDR_SIZE],
+ struct rte_ipv6_addr *ipv6,
uint32_t *ipv6_cnt, uint8_t *ip_type)
{
struct rte_ether_hdr *eth_hdr;
@@ -65,7 +65,7 @@ fib_parse_packet(struct rte_mbuf *mbuf,
/* IPv6 */
else {
ipv6_hdr = (struct rte_ipv6_hdr *)(eth_hdr + 1);
- rte_mov16(ipv6, ipv6_hdr->dst_addr.a);
+ rte_mov16(ipv6->a, ipv6_hdr->dst_addr.a);
*ip_type = 0;
(*ipv6_cnt)++;
}
@@ -120,7 +120,7 @@ fib_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,
uint16_t portid, struct lcore_conf *qconf)
{
uint32_t ipv4_arr[nb_rx];
- uint8_t ipv6_arr[nb_rx][RTE_FIB6_IPV6_ADDR_SIZE];
+ struct rte_ipv6_addr ipv6_arr[nb_rx];
uint16_t hops[nb_rx];
uint64_t hopsv4[nb_rx], hopsv6[nb_rx];
uint8_t type_arr[nb_rx];
@@ -140,7 +140,7 @@ fib_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,
i + FIB_PREFETCH_OFFSET], void *));
fib_parse_packet(pkts_burst[i],
&ipv4_arr[ipv4_cnt], &ipv4_cnt,
- ipv6_arr[ipv6_cnt], &ipv6_cnt,
+ &ipv6_arr[ipv6_cnt], &ipv6_cnt,
&type_arr[i]);
}
@@ -148,7 +148,7 @@ fib_send_packets(int nb_rx, struct rte_mbuf **pkts_burst,
for (; i < nb_rx; i++)
fib_parse_packet(pkts_burst[i],
&ipv4_arr[ipv4_cnt], &ipv4_cnt,
- ipv6_arr[ipv6_cnt], &ipv6_cnt,
+ &ipv6_arr[ipv6_cnt], &ipv6_cnt,
&type_arr[i]);
/* Lookup IPv4 hops if IPv4 packets are present. */
@@ -270,7 +270,7 @@ fib_event_loop(struct l3fwd_event_resources *evt_rsrc,
unsigned int lcore_id;
uint32_t ipv4_arr[MAX_PKT_BURST];
- uint8_t ipv6_arr[MAX_PKT_BURST][RTE_FIB6_IPV6_ADDR_SIZE];
+ struct rte_ipv6_addr ipv6_arr[MAX_PKT_BURST];
uint64_t hopsv4[MAX_PKT_BURST], hopsv6[MAX_PKT_BURST];
uint16_t nh, hops[MAX_PKT_BURST];
uint8_t type_arr[MAX_PKT_BURST];
@@ -323,7 +323,7 @@ fib_event_loop(struct l3fwd_event_resources *evt_rsrc,
fib_parse_packet(events[i].mbuf,
&ipv4_arr[ipv4_cnt], &ipv4_cnt,
- ipv6_arr[ipv6_cnt], &ipv6_cnt,
+ &ipv6_arr[ipv6_cnt], &ipv6_cnt,
&type_arr[i]);
}
@@ -340,7 +340,7 @@ fib_event_loop(struct l3fwd_event_resources *evt_rsrc,
fib_parse_packet(events[i].mbuf,
&ipv4_arr[ipv4_cnt], &ipv4_cnt,
- ipv6_arr[ipv6_cnt], &ipv6_cnt,
+ &ipv6_arr[ipv6_cnt], &ipv6_cnt,
&type_arr[i]);
}
@@ -436,7 +436,7 @@ fib_event_main_loop_tx_q_burst(__rte_unused void *dummy)
static __rte_always_inline void
fib_process_event_vector(struct rte_event_vector *vec, uint8_t *type_arr,
- uint8_t **ipv6_arr, uint64_t *hopsv4, uint64_t *hopsv6,
+ struct rte_ipv6_addr *ipv6_arr, uint64_t *hopsv4, uint64_t *hopsv6,
uint32_t *ipv4_arr, uint16_t *hops)
{
uint32_t ipv4_arr_assem, ipv6_arr_assem;
@@ -463,13 +463,13 @@ fib_process_event_vector(struct rte_event_vector *vec, uint8_t *type_arr,
rte_prefetch0(rte_pktmbuf_mtod(mbufs[i + FIB_PREFETCH_OFFSET],
void *));
fib_parse_packet(mbufs[i], &ipv4_arr[ipv4_cnt], &ipv4_cnt,
- ipv6_arr[ipv6_cnt], &ipv6_cnt, &type_arr[i]);
+ &ipv6_arr[ipv6_cnt], &ipv6_cnt, &type_arr[i]);
}
/* Parse remaining packet info. */
for (; i < vec->nb_elem; i++)
fib_parse_packet(mbufs[i], &ipv4_arr[ipv4_cnt], &ipv4_cnt,
- ipv6_arr[ipv6_cnt], &ipv6_cnt, &type_arr[i]);
+ &ipv6_arr[ipv6_cnt], &ipv6_cnt, &type_arr[i]);
/* Lookup IPv4 hops if IPv4 packets are present. */
if (likely(ipv4_cnt > 0))
@@ -480,7 +480,7 @@ fib_process_event_vector(struct rte_event_vector *vec, uint8_t *type_arr,
if (ipv6_cnt > 0)
rte_fib6_lookup_bulk(
lconf->ipv6_lookup_struct,
- (uint8_t(*)[RTE_FIB6_IPV6_ADDR_SIZE])ipv6_arr, hopsv6,
+ ipv6_arr, hopsv6,
ipv6_cnt);
/* Assign ports looked up in fib depending on IPv4 or IPv6 */
@@ -522,7 +522,8 @@ fib_event_loop_vector(struct l3fwd_event_resources *evt_rsrc,
const uint8_t event_d_id = evt_rsrc->event_d_id;
const uint16_t deq_len = evt_rsrc->deq_depth;
struct rte_event events[MAX_PKT_BURST];
- uint8_t *type_arr, **ipv6_arr, *ptr;
+ uint8_t *type_arr;
+ struct rte_ipv6_addr *ipv6_arr;
int nb_enq = 0, nb_deq = 0, i;
uint64_t *hopsv4, *hopsv6;
uint32_t *ipv4_arr;
@@ -533,7 +534,7 @@ fib_event_loop_vector(struct l3fwd_event_resources *evt_rsrc,
"vector_fib",
(sizeof(uint32_t) + sizeof(uint8_t) + sizeof(uint64_t) +
sizeof(uint64_t) + sizeof(uint16_t) + sizeof(uint8_t *) +
- (sizeof(uint8_t) * RTE_FIB6_IPV6_ADDR_SIZE)) *
+ sizeof(struct rte_ipv6_addr)) *
evt_rsrc->vector_size,
RTE_CACHE_LINE_SIZE);
if (mem == 0)
@@ -543,11 +544,7 @@ fib_event_loop_vector(struct l3fwd_event_resources *evt_rsrc,
hopsv4 = (uint64_t *)&type_arr[evt_rsrc->vector_size];
hopsv6 = (uint64_t *)&hopsv4[evt_rsrc->vector_size];
hops = (uint16_t *)&hopsv6[evt_rsrc->vector_size];
- ipv6_arr = (uint8_t **)&hops[evt_rsrc->vector_size];
-
- ptr = (uint8_t *)&ipv6_arr[evt_rsrc->vector_size];
- for (i = 0; i < evt_rsrc->vector_size; i++)
- ipv6_arr[i] = &ptr[RTE_FIB6_IPV6_ADDR_SIZE + i];
+ ipv6_arr = (struct rte_ipv6_addr *)&hops[evt_rsrc->vector_size];
if (event_p_id < 0) {
rte_free((void *)mem);
@@ -732,7 +729,7 @@ setup_fib(const int socketid)
rte_eth_dev_info_get(route_base_v6[i].if_out,
&dev_info);
ret = rte_fib6_add(ipv6_l3fwd_fib_lookup_struct[socketid],
- route_base_v6[i].ip_8,
+ &route_base_v6[i].ip6,
route_base_v6[i].depth,
route_base_v6[i].if_out);
@@ -744,7 +741,7 @@ setup_fib(const int socketid)
i, socketid);
}
- if (inet_ntop(AF_INET6, route_base_v6[i].ip_8,
+ if (inet_ntop(AF_INET6, &route_base_v6[i].ip6,
abuf, sizeof(abuf)) != NULL) {
printf("FIB: Adding route %s / %d (%d) [%s]\n", abuf,
route_base_v6[i].depth,
diff --git a/examples/l3fwd/l3fwd_lpm.c b/examples/l3fwd/l3fwd_lpm.c
index 422fdb70054d..fc4f5878fcd8 100644
--- a/examples/l3fwd/l3fwd_lpm.c
+++ b/examples/l3fwd/l3fwd_lpm.c
@@ -62,7 +62,7 @@ lpm_get_ipv6_dst_port(const struct rte_ipv6_hdr *ipv6_hdr,
uint16_t portid,
struct rte_lpm6 *ipv6_l3fwd_lookup_struct)
{
- const uint8_t *dst_ip = ipv6_hdr->dst_addr.a;
+ const struct rte_ipv6_addr *dst_ip = &ipv6_hdr->dst_addr;
uint32_t next_hop;
if (rte_lpm6_lookup(ipv6_l3fwd_lookup_struct, dst_ip, &next_hop) == 0)
@@ -122,7 +122,7 @@ lpm_get_dst_port_with_ipv4(const struct lcore_conf *qconf, struct rte_mbuf *pkt,
ipv6_hdr = (struct rte_ipv6_hdr *)(eth_hdr + 1);
return (uint16_t) ((rte_lpm6_lookup(qconf->ipv6_lookup_struct,
- ipv6_hdr->dst_addr.a, &next_hop) == 0)
+ &ipv6_hdr->dst_addr, &next_hop) == 0)
? next_hop : portid);
}
@@ -635,7 +635,7 @@ setup_lpm(const int socketid)
rte_eth_dev_info_get(route_base_v6[i].if_out,
&dev_info);
ret = rte_lpm6_add(ipv6_l3fwd_lpm_lookup_struct[socketid],
- route_base_v6[i].ip_8,
+ &route_base_v6[i].ip6,
route_base_v6[i].depth,
route_base_v6[i].if_out);
@@ -647,7 +647,7 @@ setup_lpm(const int socketid)
}
printf("LPM: Adding route %s / %d (%d) [%s]\n",
- inet_ntop(AF_INET6, route_base_v6[i].ip_8, abuf,
+ inet_ntop(AF_INET6, &route_base_v6[i].ip6, abuf,
sizeof(abuf)),
route_base_v6[i].depth,
route_base_v6[i].if_out, rte_dev_name(dev_info.device));
diff --git a/examples/l3fwd/l3fwd_route.h b/examples/l3fwd/l3fwd_route.h
index 467c4d285915..ceabf4fa8e3c 100644
--- a/examples/l3fwd/l3fwd_route.h
+++ b/examples/l3fwd/l3fwd_route.h
@@ -2,6 +2,8 @@
* Copyright(c) 2021 Intel Corporation
*/
+#include <rte_ip6.h>
+
/* Log file related character defs. */
#define COMMENT_LEAD_CHAR ('#')
#define ROUTE_LEAD_CHAR ('R')
@@ -53,10 +55,7 @@ struct ipv6_5tuple {
struct lpm_route_rule {
union {
uint32_t ip;
- union {
- uint32_t ip_32[IPV6_ADDR_U32];
- uint8_t ip_8[IPV6_ADDR_LEN];
- };
+ struct rte_ipv6_addr ip6;
};
uint8_t depth;
uint8_t if_out;
diff --git a/examples/l3fwd/lpm_route_parse.c b/examples/l3fwd/lpm_route_parse.c
index f2028d79e180..d553ebdbcc5c 100644
--- a/examples/l3fwd/lpm_route_parse.c
+++ b/examples/l3fwd/lpm_route_parse.c
@@ -108,7 +108,7 @@ lpm_parse_v6_rule(char *str, struct lpm_route_rule *v)
return -EINVAL;
}
- rc = lpm_parse_v6_net(in[CB_FLD_DST_ADDR], v->ip_32, &v->depth);
+ rc = lpm_parse_v6_net(in[CB_FLD_DST_ADDR], (uint32_t *)&v->ip6, &v->depth);
GET_CB_FIELD(in[CB_FLD_IF_OUT], v->if_out, 0, UINT8_MAX, 0);
@@ -164,8 +164,8 @@ lpm_add_default_v6_rules(void)
route_base_v6 = calloc(route_num_v6, rule_size);
for (i = 0; i < (unsigned int)route_num_v6; i++) {
- memcpy(route_base_v6[i].ip_8, ipv6_l3fwd_route_array[i].ip,
- sizeof(route_base_v6[i].ip_8));
+ memcpy(&route_base_v6[i].ip, ipv6_l3fwd_route_array[i].ip,
+ sizeof(route_base_v6[i].ip));
route_base_v6[i].depth = ipv6_l3fwd_route_array[i].depth;
route_base_v6[i].if_out = ipv6_l3fwd_route_array[i].if_out;
}
diff --git a/lib/fib/meson.build b/lib/fib/meson.build
index 6795f41a0ad2..268007ef6c93 100644
--- a/lib/fib/meson.build
+++ b/lib/fib/meson.build
@@ -10,7 +10,7 @@ endif
sources = files('rte_fib.c', 'rte_fib6.c', 'dir24_8.c', 'trie.c')
headers = files('rte_fib.h', 'rte_fib6.h')
-deps += ['rib']
+deps += ['net', 'rib']
# compile AVX512 version if:
# we are building 64-bit binary AND binutils can generate proper code
@@ -54,7 +54,7 @@ if dpdk_conf.has('RTE_ARCH_X86_64') and binutils_ok
if cc.has_argument('-mavx512bw')
trie_avx512_tmp = static_library('trie_avx512_tmp',
'trie_avx512.c',
- dependencies: static_rte_eal,
+ dependencies: [static_rte_eal, static_rte_net],
c_args: cflags + ['-mavx512f', \
'-mavx512dq', '-mavx512bw'])
objs += trie_avx512_tmp.extract_objects('trie_avx512.c')
diff --git a/lib/fib/rte_fib6.c b/lib/fib/rte_fib6.c
index 9ad990724a8b..d95cac79cbb5 100644
--- a/lib/fib/rte_fib6.c
+++ b/lib/fib/rte_fib6.c
@@ -14,6 +14,7 @@
#include <rte_malloc.h>
#include <rte_string_fns.h>
+#include <rte_ip6.h>
#include <rte_rib6.h>
#include <rte_fib6.h>
@@ -49,7 +50,7 @@ struct rte_fib6 {
};
static void
-dummy_lookup(void *fib_p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE],
+dummy_lookup(void *fib_p, const struct rte_ipv6_addr *ips,
uint64_t *next_hops, const unsigned int n)
{
unsigned int i;
@@ -57,7 +58,7 @@ dummy_lookup(void *fib_p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE],
struct rte_rib6_node *node;
for (i = 0; i < n; i++) {
- node = rte_rib6_lookup(fib->rib, ips[i]);
+ node = rte_rib6_lookup(fib->rib, &ips[i]);
if (node != NULL)
rte_rib6_get_nh(node, &next_hops[i]);
else
@@ -66,7 +67,7 @@ dummy_lookup(void *fib_p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE],
}
static int
-dummy_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE],
+dummy_modify(struct rte_fib6 *fib, const struct rte_ipv6_addr *ip,
uint8_t depth, uint64_t next_hop, int op)
{
struct rte_rib6_node *node;
@@ -118,7 +119,7 @@ init_dataplane(struct rte_fib6 *fib, __rte_unused int socket_id,
}
int
-rte_fib6_add(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE],
+rte_fib6_add(struct rte_fib6 *fib, const struct rte_ipv6_addr *ip,
uint8_t depth, uint64_t next_hop)
{
if ((fib == NULL) || (ip == NULL) || (fib->modify == NULL) ||
@@ -128,7 +129,7 @@ rte_fib6_add(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE],
}
int
-rte_fib6_delete(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE],
+rte_fib6_delete(struct rte_fib6 *fib, const struct rte_ipv6_addr *ip,
uint8_t depth)
{
if ((fib == NULL) || (ip == NULL) || (fib->modify == NULL) ||
@@ -139,7 +140,7 @@ rte_fib6_delete(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE],
int
rte_fib6_lookup_bulk(struct rte_fib6 *fib,
- uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE],
+ const struct rte_ipv6_addr *ips,
uint64_t *next_hops, int n)
{
FIB6_RETURN_IF_TRUE((fib == NULL) || (ips == NULL) ||
diff --git a/lib/fib/rte_fib6.h b/lib/fib/rte_fib6.h
index 2eb8b8267647..d26a0c7c0b77 100644
--- a/lib/fib/rte_fib6.h
+++ b/lib/fib/rte_fib6.h
@@ -17,6 +17,7 @@
#include <stdint.h>
+#include <rte_ip6.h>
#ifdef __cplusplus
extern "C" {
@@ -37,11 +38,11 @@ enum rte_fib6_type {
/** Modify FIB function */
typedef int (*rte_fib6_modify_fn_t)(struct rte_fib6 *fib,
- const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], uint8_t depth,
+ const struct rte_ipv6_addr *ip, uint8_t depth,
uint64_t next_hop, int op);
/** FIB bulk lookup function */
typedef void (*rte_fib6_lookup_fn_t)(void *fib,
- uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE],
+ const struct rte_ipv6_addr *ips,
uint64_t *next_hops, const unsigned int n);
enum rte_fib6_op {
@@ -134,7 +135,7 @@ rte_fib6_free(struct rte_fib6 *fib);
* 0 on success, negative value otherwise
*/
int
-rte_fib6_add(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE],
+rte_fib6_add(struct rte_fib6 *fib, const struct rte_ipv6_addr *ip,
uint8_t depth, uint64_t next_hop);
/**
@@ -151,7 +152,7 @@ rte_fib6_add(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE],
*/
int
rte_fib6_delete(struct rte_fib6 *fib,
- const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], uint8_t depth);
+ const struct rte_ipv6_addr *ip, uint8_t depth);
/**
* Lookup multiple IP addresses in the FIB.
@@ -172,7 +173,7 @@ rte_fib6_delete(struct rte_fib6 *fib,
*/
int
rte_fib6_lookup_bulk(struct rte_fib6 *fib,
- uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE],
+ const struct rte_ipv6_addr *ips,
uint64_t *next_hops, int n);
/**
diff --git a/lib/fib/trie.c b/lib/fib/trie.c
index 09470e7287cc..bd0c7ec63b7f 100644
--- a/lib/fib/trie.c
+++ b/lib/fib/trie.c
@@ -225,7 +225,7 @@ tbl8_recycle(struct rte_trie_tbl *dp, void *par, uint64_t tbl8_idx)
#define BYTE_SIZE 8
static inline uint32_t
-get_idx(const uint8_t *ip, uint32_t prev_idx, int bytes, int first_byte)
+get_idx(const struct rte_ipv6_addr *ip, uint32_t prev_idx, int bytes, int first_byte)
{
int i;
uint32_t idx = 0;
@@ -233,7 +233,7 @@ get_idx(const uint8_t *ip, uint32_t prev_idx, int bytes, int first_byte)
for (i = first_byte; i < (first_byte + bytes); i++) {
bitshift = (int8_t)(((first_byte + bytes - 1) - i)*BYTE_SIZE);
- idx |= ip[i] << bitshift;
+ idx |= ip->a[i] << bitshift;
}
return (prev_idx * TRIE_TBL8_GRP_NUM_ENT) + idx;
}
@@ -280,7 +280,7 @@ recycle_root_path(struct rte_trie_tbl *dp, const uint8_t *ip_part,
}
static inline int
-build_common_root(struct rte_trie_tbl *dp, const uint8_t *ip,
+build_common_root(struct rte_trie_tbl *dp, const struct rte_ipv6_addr *ip,
int common_bytes, void **tbl)
{
void *tbl_ptr = NULL;
@@ -355,8 +355,8 @@ write_edge(struct rte_trie_tbl *dp, const uint8_t *ip_part, uint64_t next_hop,
#define TBL8_LEN (RTE_FIB6_IPV6_ADDR_SIZE - TBL24_BYTES)
static int
-install_to_dp(struct rte_trie_tbl *dp, const uint8_t *ledge, const uint8_t *r,
- uint64_t next_hop)
+install_to_dp(struct rte_trie_tbl *dp, const struct rte_ipv6_addr *ledge,
+ const struct rte_ipv6_addr *r, uint64_t next_hop)
{
void *common_root_tbl;
void *ent;
@@ -364,18 +364,18 @@ install_to_dp(struct rte_trie_tbl *dp, const uint8_t *ledge, const uint8_t *r,
int i;
int common_bytes;
int llen, rlen;
- uint8_t redge[16];
+ struct rte_ipv6_addr redge;
/* decrement redge by 1*/
- rte_rib6_copy_addr(redge, r);
+ rte_rib6_copy_addr(redge.a, r->a);
for (i = 15; i >= 0; i--) {
- redge[i]--;
- if (redge[i] != 0xff)
+ redge.a[i]--;
+ if (redge.a[i] != 0xff)
break;
}
for (common_bytes = 0; common_bytes < 15; common_bytes++) {
- if (ledge[common_bytes] != redge[common_bytes])
+ if (ledge->a[common_bytes] != redge.a[common_bytes])
break;
}
@@ -386,14 +386,14 @@ install_to_dp(struct rte_trie_tbl *dp, const uint8_t *ledge, const uint8_t *r,
uint8_t first_tbl8_byte = RTE_MAX(common_bytes, TBL24_BYTES);
for (i = IPV6_MAX_IDX; i > first_tbl8_byte; i--) {
- if (ledge[i] != 0)
+ if (ledge->a[i] != 0)
break;
}
llen = i - first_tbl8_byte + (common_bytes < 3);
for (i = IPV6_MAX_IDX; i > first_tbl8_byte; i--) {
- if (redge[i] != UINT8_MAX)
+ if (redge.a[i] != UINT8_MAX)
break;
}
rlen = i - first_tbl8_byte + (common_bytes < 3);
@@ -403,10 +403,10 @@ install_to_dp(struct rte_trie_tbl *dp, const uint8_t *ledge, const uint8_t *r,
uint8_t first_idx_len = (common_bytes < 3) ? 3 : 1;
uint32_t left_idx = get_idx(ledge, 0, first_idx_len, first_byte_idx);
- uint32_t right_idx = get_idx(redge, 0, first_idx_len, first_byte_idx);
+ uint32_t right_idx = get_idx(&redge, 0, first_idx_len, first_byte_idx);
ent = get_tbl_p_by_idx(common_root_tbl, left_idx, dp->nh_sz);
- ret = write_edge(dp, &ledge[first_tbl8_byte + !(common_bytes < 3)],
+ ret = write_edge(dp, &ledge->a[first_tbl8_byte + !(common_bytes < 3)],
next_hop, llen, LEDGE, ent);
if (ret < 0)
return ret;
@@ -418,7 +418,7 @@ install_to_dp(struct rte_trie_tbl *dp, const uint8_t *ledge, const uint8_t *r,
right_idx - (left_idx + 1));
}
ent = get_tbl_p_by_idx(common_root_tbl, right_idx, dp->nh_sz);
- ret = write_edge(dp, &redge[first_tbl8_byte + !((common_bytes < 3))],
+ ret = write_edge(dp, &redge.a[first_tbl8_byte + !((common_bytes < 3))],
next_hop, rlen, REDGE, ent);
if (ret < 0)
return ret;
@@ -426,12 +426,12 @@ install_to_dp(struct rte_trie_tbl *dp, const uint8_t *ledge, const uint8_t *r,
uint8_t common_tbl8 = (common_bytes < TBL24_BYTES) ?
0 : common_bytes - (TBL24_BYTES - 1);
ent = get_tbl24_p(dp, ledge, dp->nh_sz);
- recycle_root_path(dp, ledge + TBL24_BYTES, common_tbl8, ent);
+ recycle_root_path(dp, ledge->a + TBL24_BYTES, common_tbl8, ent);
return 0;
}
static void
-get_nxt_net(uint8_t *ip, uint8_t depth)
+get_nxt_net(struct rte_ipv6_addr *ip, uint8_t depth)
{
int i;
uint8_t part_depth;
@@ -440,12 +440,12 @@ get_nxt_net(uint8_t *ip, uint8_t depth)
for (i = 0, part_depth = depth; part_depth > 8; part_depth -= 8, i++)
;
- prev_byte = ip[i];
- ip[i] += 1 << (8 - part_depth);
- if (ip[i] < prev_byte) {
+ prev_byte = ip->a[i];
+ ip->a[i] += 1 << (8 - part_depth);
+ if (ip->a[i] < prev_byte) {
while (i > 0) {
- ip[--i] += 1;
- if (ip[i] != 0)
+ ip->a[--i] += 1;
+ if (ip->a[i] != 0)
break;
}
}
@@ -461,19 +461,18 @@ v6_addr_is_zero(const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE])
static int
modify_dp(struct rte_trie_tbl *dp, struct rte_rib6 *rib,
- const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE],
+ const struct rte_ipv6_addr *ip,
uint8_t depth, uint64_t next_hop)
{
struct rte_rib6_node *tmp = NULL;
- uint8_t ledge[RTE_FIB6_IPV6_ADDR_SIZE];
- uint8_t redge[RTE_FIB6_IPV6_ADDR_SIZE];
+ struct rte_ipv6_addr ledge, redge;
int ret;
uint8_t tmp_depth;
if (next_hop > get_max_nh(dp->nh_sz))
return -EINVAL;
- rte_rib6_copy_addr(ledge, ip);
+ rte_rib6_copy_addr(ledge.a, ip->a);
do {
tmp = rte_rib6_get_nxt(rib, ip, depth, tmp,
RTE_RIB6_GET_NXT_COVER);
@@ -481,32 +480,30 @@ modify_dp(struct rte_trie_tbl *dp, struct rte_rib6 *rib,
rte_rib6_get_depth(tmp, &tmp_depth);
if (tmp_depth == depth)
continue;
- rte_rib6_get_ip(tmp, redge);
- if (rte_rib6_is_equal(ledge, redge)) {
- get_nxt_net(ledge, tmp_depth);
+ rte_rib6_get_ip(tmp, &redge);
+ if (rte_rib6_is_equal(ledge.a, redge.a)) {
+ get_nxt_net(&ledge, tmp_depth);
continue;
}
- ret = install_to_dp(dp, ledge, redge,
- next_hop);
+ ret = install_to_dp(dp, &ledge, &redge, next_hop);
if (ret != 0)
return ret;
- get_nxt_net(redge, tmp_depth);
- rte_rib6_copy_addr(ledge, redge);
+ get_nxt_net(&redge, tmp_depth);
+ rte_rib6_copy_addr(ledge.a, redge.a);
/*
* we got to the end of address space
* and wrapped around
*/
- if (v6_addr_is_zero(ledge))
+ if (v6_addr_is_zero(ledge.a))
break;
} else {
- rte_rib6_copy_addr(redge, ip);
- get_nxt_net(redge, depth);
- if (rte_rib6_is_equal(ledge, redge) &&
- !v6_addr_is_zero(ledge))
+ rte_rib6_copy_addr(redge.a, ip->a);
+ get_nxt_net(&redge, depth);
+ if (rte_rib6_is_equal(ledge.a, redge.a) &&
+ !v6_addr_is_zero(ledge.a))
break;
- ret = install_to_dp(dp, ledge, redge,
- next_hop);
+ ret = install_to_dp(dp, &ledge, &redge, next_hop);
if (ret != 0)
return ret;
}
@@ -516,7 +513,7 @@ modify_dp(struct rte_trie_tbl *dp, struct rte_rib6 *rib,
}
int
-trie_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE],
+trie_modify(struct rte_fib6 *fib, const struct rte_ipv6_addr *ip,
uint8_t depth, uint64_t next_hop, int op)
{
struct rte_trie_tbl *dp;
@@ -524,7 +521,7 @@ trie_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE],
struct rte_rib6_node *tmp = NULL;
struct rte_rib6_node *node;
struct rte_rib6_node *parent;
- uint8_t ip_masked[RTE_FIB6_IPV6_ADDR_SIZE];
+ struct rte_ipv6_addr ip_masked;
int i, ret = 0;
uint64_t par_nh, node_nh;
uint8_t tmp_depth, depth_diff = 0, parent_depth = 24;
@@ -538,10 +535,10 @@ trie_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE],
RTE_ASSERT(rib);
for (i = 0; i < RTE_FIB6_IPV6_ADDR_SIZE; i++)
- ip_masked[i] = ip[i] & get_msk_part(depth, i);
+ ip_masked.a[i] = ip->a[i] & get_msk_part(depth, i);
if (depth > 24) {
- tmp = rte_rib6_get_nxt(rib, ip_masked,
+ tmp = rte_rib6_get_nxt(rib, &ip_masked,
RTE_ALIGN_FLOOR(depth, 8), NULL,
RTE_RIB6_GET_NXT_COVER);
if (tmp == NULL) {
@@ -555,14 +552,14 @@ trie_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE],
depth_diff = depth_diff >> 3;
}
}
- node = rte_rib6_lookup_exact(rib, ip_masked, depth);
+ node = rte_rib6_lookup_exact(rib, &ip_masked, depth);
switch (op) {
case RTE_FIB6_ADD:
if (node != NULL) {
rte_rib6_get_nh(node, &node_nh);
if (node_nh == next_hop)
return 0;
- ret = modify_dp(dp, rib, ip_masked, depth, next_hop);
+ ret = modify_dp(dp, rib, &ip_masked, depth, next_hop);
if (ret == 0)
rte_rib6_set_nh(node, next_hop);
return 0;
@@ -572,7 +569,7 @@ trie_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE],
dp->number_tbl8s - depth_diff))
return -ENOSPC;
- node = rte_rib6_insert(rib, ip_masked, depth);
+ node = rte_rib6_insert(rib, &ip_masked, depth);
if (node == NULL)
return -rte_errno;
rte_rib6_set_nh(node, next_hop);
@@ -582,9 +579,9 @@ trie_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE],
if (par_nh == next_hop)
return 0;
}
- ret = modify_dp(dp, rib, ip_masked, depth, next_hop);
+ ret = modify_dp(dp, rib, &ip_masked, depth, next_hop);
if (ret != 0) {
- rte_rib6_remove(rib, ip_masked, depth);
+ rte_rib6_remove(rib, &ip_masked, depth);
return ret;
}
@@ -599,10 +596,10 @@ trie_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE],
rte_rib6_get_nh(parent, &par_nh);
rte_rib6_get_nh(node, &node_nh);
if (par_nh != node_nh)
- ret = modify_dp(dp, rib, ip_masked, depth,
+ ret = modify_dp(dp, rib, &ip_masked, depth,
par_nh);
} else
- ret = modify_dp(dp, rib, ip_masked, depth, dp->def_nh);
+ ret = modify_dp(dp, rib, &ip_masked, depth, dp->def_nh);
if (ret != 0)
return ret;
diff --git a/lib/fib/trie.h b/lib/fib/trie.h
index 2c20184a26a3..6e235edaf89c 100644
--- a/lib/fib/trie.h
+++ b/lib/fib/trie.h
@@ -8,6 +8,8 @@
#include <stdalign.h>
+#include <rte_fib6.h>
+
/**
* @file
* RTE IPv6 Longest Prefix Match (LPM)
@@ -42,13 +44,13 @@ struct rte_trie_tbl {
};
static inline uint32_t
-get_tbl24_idx(const uint8_t *ip)
+get_tbl24_idx(const struct rte_ipv6_addr *ip)
{
- return ip[0] << 16|ip[1] << 8|ip[2];
+ return ip->a[0] << 16|ip->a[1] << 8|ip->a[2];
}
static inline void *
-get_tbl24_p(struct rte_trie_tbl *dp, const uint8_t *ip, uint8_t nh_sz)
+get_tbl24_p(struct rte_trie_tbl *dp, const struct rte_ipv6_addr *ip, uint8_t nh_sz)
{
uint32_t tbl24_idx;
@@ -107,7 +109,7 @@ is_entry_extended(uint64_t ent)
#define LOOKUP_FUNC(suffix, type, nh_sz) \
static inline void rte_trie_lookup_bulk_##suffix(void *p, \
- uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE], \
+ const struct rte_ipv6_addr *ips, \
uint64_t *next_hops, const unsigned int n) \
{ \
struct rte_trie_tbl *dp = (struct rte_trie_tbl *)p; \
@@ -115,10 +117,10 @@ static inline void rte_trie_lookup_bulk_##suffix(void *p, \
uint32_t i, j; \
\
for (i = 0; i < n; i++) { \
- tmp = ((type *)dp->tbl24)[get_tbl24_idx(&ips[i][0])]; \
+ tmp = ((type *)dp->tbl24)[get_tbl24_idx(&ips[i])]; \
j = 3; \
while (is_entry_extended(tmp)) { \
- tmp = ((type *)dp->tbl8)[ips[i][j++] + \
+ tmp = ((type *)dp->tbl8)[ips[i].a[j++] + \
((tmp >> 1) * TRIE_TBL8_GRP_NUM_ENT)]; \
} \
next_hops[i] = tmp >> 1; \
@@ -138,7 +140,7 @@ rte_fib6_lookup_fn_t
trie_get_lookup_fn(void *p, enum rte_fib6_lookup_type type);
int
-trie_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE],
+trie_modify(struct rte_fib6 *fib, const struct rte_ipv6_addr *ip,
uint8_t depth, uint64_t next_hop, int op);
#endif /* _TRIE_H_ */
diff --git a/lib/fib/trie_avx512.c b/lib/fib/trie_avx512.c
index d4d70d84bf60..f49482a95dfa 100644
--- a/lib/fib/trie_avx512.c
+++ b/lib/fib/trie_avx512.c
@@ -9,7 +9,7 @@
#include "trie_avx512.h"
static __rte_always_inline void
-transpose_x16(uint8_t ips[16][RTE_FIB6_IPV6_ADDR_SIZE],
+transpose_x16(const struct rte_ipv6_addr *ips,
__m512i *first, __m512i *second, __m512i *third, __m512i *fourth)
{
__m512i tmp1, tmp2, tmp3, tmp4;
@@ -21,10 +21,10 @@ transpose_x16(uint8_t ips[16][RTE_FIB6_IPV6_ADDR_SIZE],
};
/* load all ip addresses */
- tmp1 = _mm512_loadu_si512(&ips[0][0]);
- tmp2 = _mm512_loadu_si512(&ips[4][0]);
- tmp3 = _mm512_loadu_si512(&ips[8][0]);
- tmp4 = _mm512_loadu_si512(&ips[12][0]);
+ tmp1 = _mm512_loadu_si512(&ips[0]);
+ tmp2 = _mm512_loadu_si512(&ips[4]);
+ tmp3 = _mm512_loadu_si512(&ips[8]);
+ tmp4 = _mm512_loadu_si512(&ips[12]);
/* transpose 4 byte chunks of 16 ips */
tmp5 = _mm512_unpacklo_epi32(tmp1, tmp2);
@@ -48,7 +48,7 @@ transpose_x16(uint8_t ips[16][RTE_FIB6_IPV6_ADDR_SIZE],
}
static __rte_always_inline void
-transpose_x8(uint8_t ips[8][RTE_FIB6_IPV6_ADDR_SIZE],
+transpose_x8(const struct rte_ipv6_addr *ips,
__m512i *first, __m512i *second)
{
__m512i tmp1, tmp2, tmp3, tmp4;
@@ -57,8 +57,8 @@ transpose_x8(uint8_t ips[8][RTE_FIB6_IPV6_ADDR_SIZE],
},
};
- tmp1 = _mm512_loadu_si512(&ips[0][0]);
- tmp2 = _mm512_loadu_si512(&ips[4][0]);
+ tmp1 = _mm512_loadu_si512(&ips[0]);
+ tmp2 = _mm512_loadu_si512(&ips[4]);
tmp3 = _mm512_unpacklo_epi64(tmp1, tmp2);
*first = _mm512_permutexvar_epi64(perm_idxes.z, tmp3);
@@ -67,7 +67,7 @@ transpose_x8(uint8_t ips[8][RTE_FIB6_IPV6_ADDR_SIZE],
}
static __rte_always_inline void
-trie_vec_lookup_x16x2(void *p, uint8_t ips[32][RTE_FIB6_IPV6_ADDR_SIZE],
+trie_vec_lookup_x16x2(void *p, const struct rte_ipv6_addr *ips,
uint64_t *next_hops, int size)
{
struct rte_trie_tbl *dp = (struct rte_trie_tbl *)p;
@@ -213,7 +213,7 @@ trie_vec_lookup_x16x2(void *p, uint8_t ips[32][RTE_FIB6_IPV6_ADDR_SIZE],
}
static void
-trie_vec_lookup_x8x2_8b(void *p, uint8_t ips[16][RTE_FIB6_IPV6_ADDR_SIZE],
+trie_vec_lookup_x8x2_8b(void *p, const struct rte_ipv6_addr *ips,
uint64_t *next_hops)
{
struct rte_trie_tbl *dp = (struct rte_trie_tbl *)p;
@@ -306,40 +306,40 @@ trie_vec_lookup_x8x2_8b(void *p, uint8_t ips[16][RTE_FIB6_IPV6_ADDR_SIZE],
}
void
-rte_trie_vec_lookup_bulk_2b(void *p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE],
+rte_trie_vec_lookup_bulk_2b(void *p, const struct rte_ipv6_addr *ips,
uint64_t *next_hops, const unsigned int n)
{
uint32_t i;
for (i = 0; i < (n / 32); i++) {
- trie_vec_lookup_x16x2(p, (uint8_t (*)[16])&ips[i * 32][0],
+ trie_vec_lookup_x16x2(p, &ips[i * 32],
next_hops + i * 32, sizeof(uint16_t));
}
- rte_trie_lookup_bulk_2b(p, (uint8_t (*)[16])&ips[i * 32][0],
+ rte_trie_lookup_bulk_2b(p, &ips[i * 32],
next_hops + i * 32, n - i * 32);
}
void
-rte_trie_vec_lookup_bulk_4b(void *p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE],
+rte_trie_vec_lookup_bulk_4b(void *p, const struct rte_ipv6_addr *ips,
uint64_t *next_hops, const unsigned int n)
{
uint32_t i;
for (i = 0; i < (n / 32); i++) {
- trie_vec_lookup_x16x2(p, (uint8_t (*)[16])&ips[i * 32][0],
+ trie_vec_lookup_x16x2(p, &ips[i * 32],
next_hops + i * 32, sizeof(uint32_t));
}
- rte_trie_lookup_bulk_4b(p, (uint8_t (*)[16])&ips[i * 32][0],
+ rte_trie_lookup_bulk_4b(p, &ips[i * 32],
next_hops + i * 32, n - i * 32);
}
void
-rte_trie_vec_lookup_bulk_8b(void *p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE],
+rte_trie_vec_lookup_bulk_8b(void *p, const struct rte_ipv6_addr *ips,
uint64_t *next_hops, const unsigned int n)
{
uint32_t i;
for (i = 0; i < (n / 16); i++) {
- trie_vec_lookup_x8x2_8b(p, (uint8_t (*)[16])&ips[i * 16][0],
+ trie_vec_lookup_x8x2_8b(p, &ips[i * 16],
next_hops + i * 16);
}
- rte_trie_lookup_bulk_8b(p, (uint8_t (*)[16])&ips[i * 16][0],
+ rte_trie_lookup_bulk_8b(p, &ips[i * 16],
next_hops + i * 16, n - i * 16);
}
diff --git a/lib/fib/trie_avx512.h b/lib/fib/trie_avx512.h
index ef8c7f0e3474..1028a4899fc7 100644
--- a/lib/fib/trie_avx512.h
+++ b/lib/fib/trie_avx512.h
@@ -5,16 +5,20 @@
#ifndef _TRIE_AVX512_H_
#define _TRIE_AVX512_H_
+#include <stdint.h>
+
+struct rte_ipv6_addr;
+
void
-rte_trie_vec_lookup_bulk_2b(void *p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE],
+rte_trie_vec_lookup_bulk_2b(void *p, const struct rte_ipv6_addr *ips,
uint64_t *next_hops, const unsigned int n);
void
-rte_trie_vec_lookup_bulk_4b(void *p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE],
+rte_trie_vec_lookup_bulk_4b(void *p, const struct rte_ipv6_addr *ips,
uint64_t *next_hops, const unsigned int n);
void
-rte_trie_vec_lookup_bulk_8b(void *p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE],
+rte_trie_vec_lookup_bulk_8b(void *p, const struct rte_ipv6_addr *ips,
uint64_t *next_hops, const unsigned int n);
#endif /* _TRIE_AVX512_H_ */
diff --git a/lib/lpm/meson.build b/lib/lpm/meson.build
index ae30f80b69c8..fae4f79fb938 100644
--- a/lib/lpm/meson.build
+++ b/lib/lpm/meson.build
@@ -20,3 +20,4 @@ indirect_headers += files(
)
deps += ['hash']
deps += ['rcu']
+deps += ['net']
diff --git a/lib/lpm/rte_lpm6.c b/lib/lpm/rte_lpm6.c
index 5bc17601ab71..d3f1c8485f5f 100644
--- a/lib/lpm/rte_lpm6.c
+++ b/lib/lpm/rte_lpm6.c
@@ -67,14 +67,14 @@ struct rte_lpm6_tbl_entry {
/** Rules tbl entry structure. */
struct rte_lpm6_rule {
- uint8_t ip[RTE_LPM6_IPV6_ADDR_SIZE]; /**< Rule IP address. */
+ struct rte_ipv6_addr ip; /**< Rule IP address. */
uint32_t next_hop; /**< Rule next hop. */
uint8_t depth; /**< Rule depth. */
};
/** Rules tbl entry key. */
struct rte_lpm6_rule_key {
- uint8_t ip[RTE_LPM6_IPV6_ADDR_SIZE]; /**< Rule IP address. */
+ struct rte_ipv6_addr ip; /**< Rule IP address. */
uint32_t depth; /**< Rule depth. */
};
@@ -213,9 +213,9 @@ tbl8_available(struct rte_lpm6 *lpm)
* note that ip must be already masked
*/
static inline void
-rule_key_init(struct rte_lpm6_rule_key *key, uint8_t *ip, uint8_t depth)
+rule_key_init(struct rte_lpm6_rule_key *key, const struct rte_ipv6_addr *ip, uint8_t depth)
{
- ip6_copy_addr(key->ip, ip);
+ ip6_copy_addr(key->ip.a, ip->a);
key->depth = depth;
}
@@ -231,7 +231,7 @@ rebuild_lpm(struct rte_lpm6 *lpm)
while (rte_hash_iterate(lpm->rules_tbl, (void *) &rule_key,
(void **) &next_hop, &iter) >= 0)
- rte_lpm6_add(lpm, rule_key->ip, rule_key->depth,
+ rte_lpm6_add(lpm, &rule_key->ip, rule_key->depth,
(uint32_t) next_hop);
}
@@ -460,7 +460,7 @@ rule_find_with_key(struct rte_lpm6 *lpm,
/* Find a rule */
static int
-rule_find(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
+rule_find(struct rte_lpm6 *lpm, struct rte_ipv6_addr *ip, uint8_t depth,
uint32_t *next_hop)
{
struct rte_lpm6_rule_key rule_key;
@@ -481,7 +481,7 @@ rule_find(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
* <0 - error
*/
static inline int
-rule_add(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth, uint32_t next_hop)
+rule_add(struct rte_lpm6 *lpm, struct rte_ipv6_addr *ip, uint8_t depth, uint32_t next_hop)
{
int ret, rule_exist;
struct rte_lpm6_rule_key rule_key;
@@ -570,7 +570,7 @@ init_tbl8_header(struct rte_lpm6 *lpm, uint32_t tbl_ind,
* of the bytes being inspected in this step.
*/
static uint32_t
-get_bitshift(const uint8_t *ip, uint8_t first_byte, uint8_t bytes)
+get_bitshift(const struct rte_ipv6_addr *ip, uint8_t first_byte, uint8_t bytes)
{
uint32_t entry_ind, i;
int8_t bitshift;
@@ -581,7 +581,7 @@ get_bitshift(const uint8_t *ip, uint8_t first_byte, uint8_t bytes)
if (bitshift < 0)
bitshift = 0;
- entry_ind = entry_ind | ip[i-1] << bitshift;
+ entry_ind = entry_ind | ip->a[i-1] << bitshift;
}
return entry_ind;
@@ -596,7 +596,7 @@ get_bitshift(const uint8_t *ip, uint8_t first_byte, uint8_t bytes)
*/
static inline int
simulate_add_step(struct rte_lpm6 *lpm, struct rte_lpm6_tbl_entry *tbl,
- struct rte_lpm6_tbl_entry **next_tbl, const uint8_t *ip,
+ struct rte_lpm6_tbl_entry **next_tbl, const struct rte_ipv6_addr *ip,
uint8_t bytes, uint8_t first_byte, uint8_t depth,
uint32_t *need_tbl_nb)
{
@@ -649,7 +649,7 @@ simulate_add_step(struct rte_lpm6 *lpm, struct rte_lpm6_tbl_entry *tbl,
static inline int
add_step(struct rte_lpm6 *lpm, struct rte_lpm6_tbl_entry *tbl,
uint32_t tbl_ind, struct rte_lpm6_tbl_entry **next_tbl,
- uint32_t *next_tbl_ind, uint8_t *ip, uint8_t bytes,
+ uint32_t *next_tbl_ind, struct rte_ipv6_addr *ip, uint8_t bytes,
uint8_t first_byte, uint8_t depth, uint32_t next_hop,
uint8_t is_new_rule)
{
@@ -814,7 +814,7 @@ add_step(struct rte_lpm6 *lpm, struct rte_lpm6_tbl_entry *tbl,
* -ENOSPC not enough tbl8 left
*/
static int
-simulate_add(struct rte_lpm6 *lpm, const uint8_t *masked_ip, uint8_t depth)
+simulate_add(struct rte_lpm6 *lpm, const struct rte_ipv6_addr *masked_ip, uint8_t depth)
{
struct rte_lpm6_tbl_entry *tbl;
struct rte_lpm6_tbl_entry *tbl_next = NULL;
@@ -851,7 +851,7 @@ simulate_add(struct rte_lpm6 *lpm, const uint8_t *masked_ip, uint8_t depth)
* Add a route
*/
int
-rte_lpm6_add(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
+rte_lpm6_add(struct rte_lpm6 *lpm, const struct rte_ipv6_addr *ip, uint8_t depth,
uint32_t next_hop)
{
struct rte_lpm6_tbl_entry *tbl;
@@ -859,7 +859,7 @@ rte_lpm6_add(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
/* init to avoid compiler warning */
uint32_t tbl_next_num = 123456;
int status;
- uint8_t masked_ip[RTE_LPM6_IPV6_ADDR_SIZE];
+ struct rte_ipv6_addr masked_ip;
int i;
/* Check user arguments. */
@@ -867,16 +867,16 @@ rte_lpm6_add(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
return -EINVAL;
/* Copy the IP and mask it to avoid modifying user's input data. */
- ip6_copy_addr(masked_ip, ip);
- ip6_mask_addr(masked_ip, depth);
+ ip6_copy_addr(masked_ip.a, ip->a);
+ ip6_mask_addr(masked_ip.a, depth);
/* Simulate adding a new route */
- int ret = simulate_add(lpm, masked_ip, depth);
+ int ret = simulate_add(lpm, &masked_ip, depth);
if (ret < 0)
return ret;
/* Add the rule to the rule table. */
- int is_new_rule = rule_add(lpm, masked_ip, depth, next_hop);
+ int is_new_rule = rule_add(lpm, &masked_ip, depth, next_hop);
/* If there is no space available for new rule return error. */
if (is_new_rule < 0)
return is_new_rule;
@@ -884,7 +884,7 @@ rte_lpm6_add(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
/* Inspect the first three bytes through tbl24 on the first step. */
tbl = lpm->tbl24;
status = add_step(lpm, tbl, TBL24_IND, &tbl_next, &tbl_next_num,
- masked_ip, ADD_FIRST_BYTE, 1, depth, next_hop,
+ &masked_ip, ADD_FIRST_BYTE, 1, depth, next_hop,
is_new_rule);
assert(status >= 0);
@@ -895,7 +895,7 @@ rte_lpm6_add(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
for (i = ADD_FIRST_BYTE; i < RTE_LPM6_IPV6_ADDR_SIZE && status == 1; i++) {
tbl = tbl_next;
status = add_step(lpm, tbl, tbl_next_num, &tbl_next,
- &tbl_next_num, masked_ip, 1, (uint8_t)(i + 1),
+ &tbl_next_num, &masked_ip, 1, (uint8_t)(i + 1),
depth, next_hop, is_new_rule);
assert(status >= 0);
}
@@ -910,7 +910,7 @@ rte_lpm6_add(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
*/
static inline int
lookup_step(const struct rte_lpm6 *lpm, const struct rte_lpm6_tbl_entry *tbl,
- const struct rte_lpm6_tbl_entry **tbl_next, const uint8_t *ip,
+ const struct rte_lpm6_tbl_entry **tbl_next, const struct rte_ipv6_addr *ip,
uint8_t first_byte, uint32_t *next_hop)
{
uint32_t tbl8_index, tbl_entry;
@@ -922,7 +922,7 @@ lookup_step(const struct rte_lpm6 *lpm, const struct rte_lpm6_tbl_entry *tbl,
if ((tbl_entry & RTE_LPM6_VALID_EXT_ENTRY_BITMASK) ==
RTE_LPM6_VALID_EXT_ENTRY_BITMASK) {
- tbl8_index = ip[first_byte-1] +
+ tbl8_index = ip->a[first_byte-1] +
((tbl_entry & RTE_LPM6_TBL8_BITMASK) *
RTE_LPM6_TBL8_GROUP_NUM_ENTRIES);
@@ -940,7 +940,7 @@ lookup_step(const struct rte_lpm6 *lpm, const struct rte_lpm6_tbl_entry *tbl,
* Looks up an IP
*/
int
-rte_lpm6_lookup(const struct rte_lpm6 *lpm, const uint8_t *ip,
+rte_lpm6_lookup(const struct rte_lpm6 *lpm, const struct rte_ipv6_addr *ip,
uint32_t *next_hop)
{
const struct rte_lpm6_tbl_entry *tbl;
@@ -954,7 +954,7 @@ rte_lpm6_lookup(const struct rte_lpm6 *lpm, const uint8_t *ip,
return -EINVAL;
first_byte = LOOKUP_FIRST_BYTE;
- tbl24_index = (ip[0] << BYTES2_SIZE) | (ip[1] << BYTE_SIZE) | ip[2];
+ tbl24_index = (ip->a[0] << BYTES2_SIZE) | (ip->a[1] << BYTE_SIZE) | ip->a[2];
/* Calculate pointer to the first entry to be inspected */
tbl = &lpm->tbl24[tbl24_index];
@@ -973,7 +973,7 @@ rte_lpm6_lookup(const struct rte_lpm6 *lpm, const uint8_t *ip,
*/
int
rte_lpm6_lookup_bulk_func(const struct rte_lpm6 *lpm,
- uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE],
+ struct rte_ipv6_addr *ips,
int32_t *next_hops, unsigned int n)
{
unsigned int i;
@@ -989,8 +989,8 @@ rte_lpm6_lookup_bulk_func(const struct rte_lpm6 *lpm,
for (i = 0; i < n; i++) {
first_byte = LOOKUP_FIRST_BYTE;
- tbl24_index = (ips[i][0] << BYTES2_SIZE) |
- (ips[i][1] << BYTE_SIZE) | ips[i][2];
+ tbl24_index = (ips[i].a[0] << BYTES2_SIZE) |
+ (ips[i].a[1] << BYTE_SIZE) | ips[i].a[2];
/* Calculate pointer to the first entry to be inspected */
tbl = &lpm->tbl24[tbl24_index];
@@ -999,7 +999,7 @@ rte_lpm6_lookup_bulk_func(const struct rte_lpm6 *lpm,
/* Continue inspecting following levels
* until success or failure
*/
- status = lookup_step(lpm, tbl, &tbl_next, ips[i],
+ status = lookup_step(lpm, tbl, &tbl_next, &ips[i],
first_byte++, &next_hop);
tbl = tbl_next;
} while (status == 1);
@@ -1017,10 +1017,10 @@ rte_lpm6_lookup_bulk_func(const struct rte_lpm6 *lpm,
* Look for a rule in the high-level rules table
*/
int
-rte_lpm6_is_rule_present(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
+rte_lpm6_is_rule_present(struct rte_lpm6 *lpm, const struct rte_ipv6_addr *ip, uint8_t depth,
uint32_t *next_hop)
{
- uint8_t masked_ip[RTE_LPM6_IPV6_ADDR_SIZE];
+ struct rte_ipv6_addr masked_ip;
/* Check user arguments. */
if ((lpm == NULL) || next_hop == NULL || ip == NULL ||
@@ -1028,10 +1028,10 @@ rte_lpm6_is_rule_present(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
return -EINVAL;
/* Copy the IP and mask it to avoid modifying user's input data. */
- ip6_copy_addr(masked_ip, ip);
- ip6_mask_addr(masked_ip, depth);
+ ip6_copy_addr(masked_ip.a, ip->a);
+ ip6_mask_addr(masked_ip.a, depth);
- return rule_find(lpm, masked_ip, depth, next_hop);
+ return rule_find(lpm, &masked_ip, depth, next_hop);
}
/*
@@ -1042,7 +1042,7 @@ rte_lpm6_is_rule_present(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
* <0 on failure
*/
static inline int
-rule_delete(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth)
+rule_delete(struct rte_lpm6 *lpm, struct rte_ipv6_addr *ip, uint8_t depth)
{
int ret;
struct rte_lpm6_rule_key rule_key;
@@ -1067,10 +1067,10 @@ rule_delete(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth)
*/
int
rte_lpm6_delete_bulk_func(struct rte_lpm6 *lpm,
- uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE], uint8_t *depths,
+ struct rte_ipv6_addr *ips, uint8_t *depths,
unsigned n)
{
- uint8_t masked_ip[RTE_LPM6_IPV6_ADDR_SIZE];
+ struct rte_ipv6_addr masked_ip;
unsigned i;
/* Check input arguments. */
@@ -1078,9 +1078,9 @@ rte_lpm6_delete_bulk_func(struct rte_lpm6 *lpm,
return -EINVAL;
for (i = 0; i < n; i++) {
- ip6_copy_addr(masked_ip, ips[i]);
- ip6_mask_addr(masked_ip, depths[i]);
- rule_delete(lpm, masked_ip, depths[i]);
+ ip6_copy_addr(masked_ip.a, ips[i].a);
+ ip6_mask_addr(masked_ip.a, depths[i]);
+ rule_delete(lpm, &masked_ip, depths[i]);
}
/*
@@ -1141,7 +1141,7 @@ depth_to_mask_1b(uint8_t depth)
* Find a less specific rule
*/
static int
-rule_find_less_specific(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
+rule_find_less_specific(struct rte_lpm6 *lpm, struct rte_ipv6_addr *ip, uint8_t depth,
struct rte_lpm6_rule *rule)
{
int ret;
@@ -1163,12 +1163,12 @@ rule_find_less_specific(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
mask = depth_to_mask_1b(mask);
rule_key.depth = depth;
- rule_key.ip[depth >> 3] &= mask;
+ rule_key.ip.a[depth >> 3] &= mask;
ret = rule_find_with_key(lpm, &rule_key, &next_hop);
if (ret) {
rule->depth = depth;
- ip6_copy_addr(rule->ip, rule_key.ip);
+ ip6_copy_addr(rule->ip.a, rule_key.ip.a);
rule->next_hop = next_hop;
return 1;
}
@@ -1181,13 +1181,13 @@ rule_find_less_specific(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
* Find range of tbl8 cells occupied by a rule
*/
static void
-rule_find_range(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
+rule_find_range(struct rte_lpm6 *lpm, const struct rte_ipv6_addr *ip, uint8_t depth,
struct rte_lpm6_tbl_entry **from,
struct rte_lpm6_tbl_entry **to,
uint32_t *out_tbl_ind)
{
uint32_t ind;
- uint32_t first_3bytes = (uint32_t)ip[0] << 16 | ip[1] << 8 | ip[2];
+ uint32_t first_3bytes = (uint32_t)ip->a[0] << 16 | ip->a[1] << 8 | ip->a[2];
if (depth <= 24) {
/* rule is within the top level */
@@ -1213,7 +1213,7 @@ rule_find_range(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
* until we reach the last one
*/
while (depth > 8) {
- tbl += ip[byte];
+ tbl += ip->a[byte];
assert(tbl->ext_entry == 1);
/* go to the next level/tbl8 */
tbl_ind = tbl->lpm6_tbl8_gindex;
@@ -1224,7 +1224,7 @@ rule_find_range(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
}
/* last level/tbl8 */
- ind = ip[byte] & depth_to_mask_1b(depth);
+ ind = ip->a[byte] & depth_to_mask_1b(depth);
*from = &tbl[ind];
ind += (1 << (8 - depth)) - 1;
*to = &tbl[ind];
@@ -1288,9 +1288,9 @@ remove_tbl(struct rte_lpm6 *lpm, struct rte_lpm_tbl8_hdr *tbl_hdr,
* Deletes a rule
*/
int
-rte_lpm6_delete(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth)
+rte_lpm6_delete(struct rte_lpm6 *lpm, const struct rte_ipv6_addr *ip, uint8_t depth)
{
- uint8_t masked_ip[RTE_LPM6_IPV6_ADDR_SIZE];
+ struct rte_ipv6_addr masked_ip;
struct rte_lpm6_rule lsp_rule_obj;
struct rte_lpm6_rule *lsp_rule;
int ret;
@@ -1302,21 +1302,21 @@ rte_lpm6_delete(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth)
return -EINVAL;
/* Copy the IP and mask it to avoid modifying user's input data. */
- ip6_copy_addr(masked_ip, ip);
- ip6_mask_addr(masked_ip, depth);
+ ip6_copy_addr(masked_ip.a, ip->a);
+ ip6_mask_addr(masked_ip.a, depth);
/* Delete the rule from the rule table. */
- ret = rule_delete(lpm, masked_ip, depth);
+ ret = rule_delete(lpm, &masked_ip, depth);
if (ret < 0)
return -ENOENT;
/* find rule cells */
- rule_find_range(lpm, masked_ip, depth, &from, &to, &tbl_ind);
+ rule_find_range(lpm, &masked_ip, depth, &from, &to, &tbl_ind);
/* find a less specific rule (a rule with smaller depth)
* note: masked_ip will be modified, don't use it anymore
*/
- ret = rule_find_less_specific(lpm, masked_ip, depth,
+ ret = rule_find_less_specific(lpm, &masked_ip, depth,
&lsp_rule_obj);
lsp_rule = ret ? &lsp_rule_obj : NULL;
diff --git a/lib/lpm/rte_lpm6.h b/lib/lpm/rte_lpm6.h
index c93683e6240c..87392194237c 100644
--- a/lib/lpm/rte_lpm6.h
+++ b/lib/lpm/rte_lpm6.h
@@ -9,6 +9,8 @@
* RTE Longest Prefix Match for IPv6 (LPM6)
*/
+#include <rte_ip6.h>
+
#include <stdint.h>
#ifdef __cplusplus
@@ -92,7 +94,7 @@ rte_lpm6_free(struct rte_lpm6 *lpm);
* 0 on success, negative value otherwise
*/
int
-rte_lpm6_add(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
+rte_lpm6_add(struct rte_lpm6 *lpm, const struct rte_ipv6_addr *ip, uint8_t depth,
uint32_t next_hop);
/**
@@ -111,7 +113,7 @@ rte_lpm6_add(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
* 1 if the rule exists, 0 if it does not, a negative value on failure
*/
int
-rte_lpm6_is_rule_present(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
+rte_lpm6_is_rule_present(struct rte_lpm6 *lpm, const struct rte_ipv6_addr *ip, uint8_t depth,
uint32_t *next_hop);
/**
@@ -127,7 +129,7 @@ rte_lpm6_is_rule_present(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth,
* 0 on success, negative value otherwise
*/
int
-rte_lpm6_delete(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth);
+rte_lpm6_delete(struct rte_lpm6 *lpm, const struct rte_ipv6_addr *ip, uint8_t depth);
/**
* Delete a rule from the LPM table.
@@ -145,7 +147,7 @@ rte_lpm6_delete(struct rte_lpm6 *lpm, const uint8_t *ip, uint8_t depth);
*/
int
rte_lpm6_delete_bulk_func(struct rte_lpm6 *lpm,
- uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE], uint8_t *depths, unsigned n);
+ struct rte_ipv6_addr *ips, uint8_t *depths, unsigned int n);
/**
* Delete all rules from the LPM table.
@@ -169,7 +171,7 @@ rte_lpm6_delete_all(struct rte_lpm6 *lpm);
* -EINVAL for incorrect arguments, -ENOENT on lookup miss, 0 on lookup hit
*/
int
-rte_lpm6_lookup(const struct rte_lpm6 *lpm, const uint8_t *ip, uint32_t *next_hop);
+rte_lpm6_lookup(const struct rte_lpm6 *lpm, const struct rte_ipv6_addr *ip, uint32_t *next_hop);
/**
* Lookup multiple IP addresses in an LPM table.
@@ -189,7 +191,7 @@ rte_lpm6_lookup(const struct rte_lpm6 *lpm, const uint8_t *ip, uint32_t *next_ho
*/
int
rte_lpm6_lookup_bulk_func(const struct rte_lpm6 *lpm,
- uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE],
+ struct rte_ipv6_addr *ips,
int32_t *next_hops, unsigned int n);
#ifdef __cplusplus
diff --git a/lib/node/ip6_lookup.c b/lib/node/ip6_lookup.c
index 6bbcf14e2aa8..1ad6e3fd18f2 100644
--- a/lib/node/ip6_lookup.c
+++ b/lib/node/ip6_lookup.c
@@ -74,7 +74,7 @@ ip6_lookup_node_process_scalar(struct rte_graph *graph, struct rte_node *node,
/* Get stream for the speculated next node */
to_next = rte_node_next_stream_get(graph, node, next_index, nb_objs);
while (n_left_from >= 4) {
- uint8_t ip_batch[4][16];
+ struct rte_ipv6_addr ip_batch[4];
int32_t next_hop[4];
uint16_t next[4];
@@ -112,28 +112,28 @@ ip6_lookup_node_process_scalar(struct rte_graph *graph, struct rte_node *node,
sizeof(struct rte_ether_hdr));
/* Extract hop_limits as ipv6 hdr is in cache */
node_mbuf_priv1(mbuf0, dyn)->ttl = ipv6_hdr->hop_limits;
- rte_memcpy(ip_batch[0], &ipv6_hdr->dst_addr, 16);
+ rte_memcpy(&ip_batch[0], &ipv6_hdr->dst_addr, 16);
/* Extract DIP of mbuf1 */
ipv6_hdr = rte_pktmbuf_mtod_offset(mbuf1, struct rte_ipv6_hdr *,
sizeof(struct rte_ether_hdr));
/* Extract hop_limits as ipv6 hdr is in cache */
node_mbuf_priv1(mbuf1, dyn)->ttl = ipv6_hdr->hop_limits;
- rte_memcpy(ip_batch[1], &ipv6_hdr->dst_addr, 16);
+ rte_memcpy(&ip_batch[1], &ipv6_hdr->dst_addr, 16);
/* Extract DIP of mbuf2 */
ipv6_hdr = rte_pktmbuf_mtod_offset(mbuf2, struct rte_ipv6_hdr *,
sizeof(struct rte_ether_hdr));
/* Extract hop_limits as ipv6 hdr is in cache */
node_mbuf_priv1(mbuf2, dyn)->ttl = ipv6_hdr->hop_limits;
- rte_memcpy(ip_batch[2], &ipv6_hdr->dst_addr, 16);
+ rte_memcpy(&ip_batch[2], &ipv6_hdr->dst_addr, 16);
/* Extract DIP of mbuf3 */
ipv6_hdr = rte_pktmbuf_mtod_offset(mbuf3, struct rte_ipv6_hdr *,
sizeof(struct rte_ether_hdr));
/* Extract hop_limits as ipv6 hdr is in cache */
node_mbuf_priv1(mbuf3, dyn)->ttl = ipv6_hdr->hop_limits;
- rte_memcpy(ip_batch[3], &ipv6_hdr->dst_addr, 16);
+ rte_memcpy(&ip_batch[3], &ipv6_hdr->dst_addr, 16);
rte_lpm6_lookup_bulk_func(lpm6, ip_batch, next_hop, 4);
@@ -223,7 +223,7 @@ ip6_lookup_node_process_scalar(struct rte_graph *graph, struct rte_node *node,
/* Extract TTL as IPv6 hdr is in cache */
node_mbuf_priv1(mbuf0, dyn)->ttl = ipv6_hdr->hop_limits;
- rc = rte_lpm6_lookup(lpm6, ipv6_hdr->dst_addr.a, &next_hop);
+ rc = rte_lpm6_lookup(lpm6, &ipv6_hdr->dst_addr, &next_hop);
next_hop = (rc == 0) ? next_hop : drop_nh;
node_mbuf_priv1(mbuf0, dyn)->nh = (uint16_t)next_hop;
@@ -278,8 +278,8 @@ rte_node_ip6_route_add(const uint8_t *ip, uint8_t depth, uint16_t next_hop,
if (!ip6_lookup_nm.lpm_tbl[socket])
continue;
- ret = rte_lpm6_add(ip6_lookup_nm.lpm_tbl[socket], ip, depth,
- val);
+ ret = rte_lpm6_add(ip6_lookup_nm.lpm_tbl[socket],
+ (const struct rte_ipv6_addr *)ip, depth, val);
if (ret < 0) {
node_err("ip6_lookup",
"Unable to add entry %s / %d nh (%x) to LPM "
diff --git a/lib/rib/meson.build b/lib/rib/meson.build
index 7bacbb453592..e98f70848189 100644
--- a/lib/rib/meson.build
+++ b/lib/rib/meson.build
@@ -4,4 +4,4 @@
sources = files('rte_rib.c', 'rte_rib6.c')
headers = files('rte_rib.h', 'rte_rib6.h')
-deps += ['mempool']
+deps += ['net', 'mempool']
diff --git a/lib/rib/rte_rib6.c b/lib/rib/rte_rib6.c
index 89c8390c63be..a1ac418eb87a 100644
--- a/lib/rib/rte_rib6.c
+++ b/lib/rib/rte_rib6.c
@@ -35,7 +35,7 @@ struct rte_rib6_node {
struct rte_rib6_node *right;
struct rte_rib6_node *parent;
uint64_t nh;
- uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE];
+ struct rte_ipv6_addr ip;
uint8_t depth;
uint8_t flag;
uint64_t ext[];
@@ -79,7 +79,7 @@ is_covered(const uint8_t ip1[RTE_RIB6_IPV6_ADDR_SIZE],
}
static inline int
-get_dir(const uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE], uint8_t depth)
+get_dir(const struct rte_ipv6_addr *ip, uint8_t depth)
{
uint8_t index, msk;
@@ -98,12 +98,12 @@ get_dir(const uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE], uint8_t depth)
*/
msk = 1 << (7 - (depth & 7));
- return (ip[index] & msk) != 0;
+ return (ip->a[index] & msk) != 0;
}
static inline struct rte_rib6_node *
get_nxt_node(struct rte_rib6_node *node,
- const uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE])
+ const struct rte_ipv6_addr *ip)
{
if (node->depth == RIB6_MAXDEPTH)
return NULL;
@@ -133,7 +133,7 @@ node_free(struct rte_rib6 *rib, struct rte_rib6_node *ent)
struct rte_rib6_node *
rte_rib6_lookup(struct rte_rib6 *rib,
- const uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE])
+ const struct rte_ipv6_addr *ip)
{
struct rte_rib6_node *cur;
struct rte_rib6_node *prev = NULL;
@@ -144,7 +144,7 @@ rte_rib6_lookup(struct rte_rib6 *rib,
}
cur = rib->tree;
- while ((cur != NULL) && is_covered(ip, cur->ip, cur->depth)) {
+ while ((cur != NULL) && is_covered(ip->a, cur->ip.a, cur->depth)) {
if (is_valid_node(cur))
prev = cur;
cur = get_nxt_node(cur, ip);
@@ -169,10 +169,10 @@ rte_rib6_lookup_parent(struct rte_rib6_node *ent)
struct rte_rib6_node *
rte_rib6_lookup_exact(struct rte_rib6 *rib,
- const uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE], uint8_t depth)
+ const struct rte_ipv6_addr *ip, uint8_t depth)
{
struct rte_rib6_node *cur;
- uint8_t tmp_ip[RTE_RIB6_IPV6_ADDR_SIZE];
+ struct rte_ipv6_addr tmp_ip;
int i;
if (unlikely(rib == NULL || ip == NULL || depth > RIB6_MAXDEPTH)) {
@@ -182,19 +182,19 @@ rte_rib6_lookup_exact(struct rte_rib6 *rib,
cur = rib->tree;
for (i = 0; i < RTE_RIB6_IPV6_ADDR_SIZE; i++)
- tmp_ip[i] = ip[i] & get_msk_part(depth, i);
+ tmp_ip.a[i] = ip->a[i] & get_msk_part(depth, i);
while (cur != NULL) {
- if (rte_rib6_is_equal(cur->ip, tmp_ip) &&
+ if (rte_rib6_is_equal(cur->ip.a, tmp_ip.a) &&
(cur->depth == depth) &&
is_valid_node(cur))
return cur;
- if (!(is_covered(tmp_ip, cur->ip, cur->depth)) ||
+ if (!(is_covered(tmp_ip.a, cur->ip.a, cur->depth)) ||
(cur->depth >= depth))
break;
- cur = get_nxt_node(cur, tmp_ip);
+ cur = get_nxt_node(cur, &tmp_ip);
}
return NULL;
@@ -207,11 +207,11 @@ rte_rib6_lookup_exact(struct rte_rib6 *rib,
*/
struct rte_rib6_node *
rte_rib6_get_nxt(struct rte_rib6 *rib,
- const uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE],
+ const struct rte_ipv6_addr *ip,
uint8_t depth, struct rte_rib6_node *last, int flag)
{
struct rte_rib6_node *tmp, *prev = NULL;
- uint8_t tmp_ip[RTE_RIB6_IPV6_ADDR_SIZE];
+ struct rte_ipv6_addr tmp_ip;
int i;
if (unlikely(rib == NULL || ip == NULL || depth > RIB6_MAXDEPTH)) {
@@ -220,19 +220,19 @@ rte_rib6_get_nxt(struct rte_rib6 *rib,
}
for (i = 0; i < RTE_RIB6_IPV6_ADDR_SIZE; i++)
- tmp_ip[i] = ip[i] & get_msk_part(depth, i);
+ tmp_ip.a[i] = ip->a[i] & get_msk_part(depth, i);
if (last == NULL) {
tmp = rib->tree;
while ((tmp) && (tmp->depth < depth))
- tmp = get_nxt_node(tmp, tmp_ip);
+ tmp = get_nxt_node(tmp, &tmp_ip);
} else {
tmp = last;
while ((tmp->parent != NULL) && (is_right_node(tmp) ||
(tmp->parent->right == NULL))) {
tmp = tmp->parent;
if (is_valid_node(tmp) &&
- (is_covered(tmp->ip, tmp_ip, depth) &&
+ (is_covered(tmp->ip.a, tmp_ip.a, depth) &&
(tmp->depth > depth)))
return tmp;
}
@@ -240,7 +240,7 @@ rte_rib6_get_nxt(struct rte_rib6 *rib,
}
while (tmp) {
if (is_valid_node(tmp) &&
- (is_covered(tmp->ip, tmp_ip, depth) &&
+ (is_covered(tmp->ip.a, tmp_ip.a, depth) &&
(tmp->depth > depth))) {
prev = tmp;
if (flag == RTE_RIB6_GET_NXT_COVER)
@@ -253,7 +253,7 @@ rte_rib6_get_nxt(struct rte_rib6 *rib,
void
rte_rib6_remove(struct rte_rib6 *rib,
- const uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE], uint8_t depth)
+ const struct rte_ipv6_addr *ip, uint8_t depth)
{
struct rte_rib6_node *cur, *prev, *child;
@@ -286,14 +286,14 @@ rte_rib6_remove(struct rte_rib6 *rib,
struct rte_rib6_node *
rte_rib6_insert(struct rte_rib6 *rib,
- const uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE], uint8_t depth)
+ const struct rte_ipv6_addr *ip, uint8_t depth)
{
struct rte_rib6_node **tmp;
struct rte_rib6_node *prev = NULL;
struct rte_rib6_node *new_node = NULL;
struct rte_rib6_node *common_node = NULL;
- uint8_t common_prefix[RTE_RIB6_IPV6_ADDR_SIZE];
- uint8_t tmp_ip[RTE_RIB6_IPV6_ADDR_SIZE];
+ struct rte_ipv6_addr common_prefix;
+ struct rte_ipv6_addr tmp_ip;
int i, d;
uint8_t common_depth, ip_xor;
@@ -305,9 +305,9 @@ rte_rib6_insert(struct rte_rib6 *rib,
tmp = &rib->tree;
for (i = 0; i < RTE_RIB6_IPV6_ADDR_SIZE; i++)
- tmp_ip[i] = ip[i] & get_msk_part(depth, i);
+ tmp_ip.a[i] = ip->a[i] & get_msk_part(depth, i);
- new_node = rte_rib6_lookup_exact(rib, tmp_ip, depth);
+ new_node = rte_rib6_lookup_exact(rib, &tmp_ip, depth);
if (new_node != NULL) {
rte_errno = EEXIST;
return NULL;
@@ -321,7 +321,7 @@ rte_rib6_insert(struct rte_rib6 *rib,
new_node->left = NULL;
new_node->right = NULL;
new_node->parent = NULL;
- rte_rib6_copy_addr(new_node->ip, tmp_ip);
+ rte_rib6_copy_addr(new_node->ip.a, tmp_ip.a);
new_node->depth = depth;
new_node->flag = RTE_RIB_VALID_NODE;
@@ -340,7 +340,7 @@ rte_rib6_insert(struct rte_rib6 *rib,
* but node with proper search criteria is found.
* Validate intermediate node and return.
*/
- if (rte_rib6_is_equal(tmp_ip, (*tmp)->ip) &&
+ if (rte_rib6_is_equal(tmp_ip.a, (*tmp)->ip.a) &&
(depth == (*tmp)->depth)) {
node_free(rib, new_node);
(*tmp)->flag |= RTE_RIB_VALID_NODE;
@@ -348,20 +348,20 @@ rte_rib6_insert(struct rte_rib6 *rib,
return *tmp;
}
- if (!is_covered(tmp_ip, (*tmp)->ip, (*tmp)->depth) ||
+ if (!is_covered(tmp_ip.a, (*tmp)->ip.a, (*tmp)->depth) ||
((*tmp)->depth >= depth)) {
break;
}
prev = *tmp;
- tmp = (get_dir(tmp_ip, (*tmp)->depth)) ? &(*tmp)->right :
+ tmp = (get_dir(&tmp_ip, (*tmp)->depth)) ? &(*tmp)->right :
&(*tmp)->left;
}
/* closest node found, new_node should be inserted in the middle */
common_depth = RTE_MIN(depth, (*tmp)->depth);
for (i = 0, d = 0; i < RTE_RIB6_IPV6_ADDR_SIZE; i++) {
- ip_xor = tmp_ip[i] ^ (*tmp)->ip[i];
+ ip_xor = tmp_ip.a[i] ^ (*tmp)->ip.a[i];
if (ip_xor == 0)
d += 8;
else {
@@ -373,12 +373,12 @@ rte_rib6_insert(struct rte_rib6 *rib,
common_depth = RTE_MIN(d, common_depth);
for (i = 0; i < RTE_RIB6_IPV6_ADDR_SIZE; i++)
- common_prefix[i] = tmp_ip[i] & get_msk_part(common_depth, i);
+ common_prefix.a[i] = tmp_ip.a[i] & get_msk_part(common_depth, i);
- if (rte_rib6_is_equal(common_prefix, tmp_ip) &&
+ if (rte_rib6_is_equal(common_prefix.a, tmp_ip.a) &&
(common_depth == depth)) {
/* insert as a parent */
- if (get_dir((*tmp)->ip, depth))
+ if (get_dir(&(*tmp)->ip, depth))
new_node->right = *tmp;
else
new_node->left = *tmp;
@@ -393,13 +393,13 @@ rte_rib6_insert(struct rte_rib6 *rib,
rte_errno = ENOMEM;
return NULL;
}
- rte_rib6_copy_addr(common_node->ip, common_prefix);
+ rte_rib6_copy_addr(common_node->ip.a, common_prefix.a);
common_node->depth = common_depth;
common_node->flag = 0;
common_node->parent = (*tmp)->parent;
new_node->parent = common_node;
(*tmp)->parent = common_node;
- if (get_dir((*tmp)->ip, common_depth) == 1) {
+ if (get_dir(&(*tmp)->ip, common_depth) == 1) {
common_node->left = new_node;
common_node->right = *tmp;
} else {
@@ -414,13 +414,13 @@ rte_rib6_insert(struct rte_rib6 *rib,
int
rte_rib6_get_ip(const struct rte_rib6_node *node,
- uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE])
+ struct rte_ipv6_addr *ip)
{
if (unlikely(node == NULL || ip == NULL)) {
rte_errno = EINVAL;
return -1;
}
- rte_rib6_copy_addr(ip, node->ip);
+ rte_rib6_copy_addr(ip->a, node->ip.a);
return 0;
}
@@ -604,7 +604,7 @@ rte_rib6_free(struct rte_rib6 *rib)
while ((tmp = rte_rib6_get_nxt(rib, 0, 0, tmp,
RTE_RIB6_GET_NXT_ALL)) != NULL)
- rte_rib6_remove(rib, tmp->ip, tmp->depth);
+ rte_rib6_remove(rib, &tmp->ip, tmp->depth);
rte_mempool_free(rib->node_pool);
diff --git a/lib/rib/rte_rib6.h b/lib/rib/rte_rib6.h
index 775286f965f2..47dcb6d15183 100644
--- a/lib/rib/rte_rib6.h
+++ b/lib/rib/rte_rib6.h
@@ -16,6 +16,7 @@
#include <rte_memcpy.h>
#include <rte_common.h>
+#include <rte_ip6.h>
#ifdef __cplusplus
extern "C" {
@@ -124,7 +125,7 @@ get_msk_part(uint8_t depth, int byte) {
*/
struct rte_rib6_node *
rte_rib6_lookup(struct rte_rib6 *rib,
- const uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE]);
+ const struct rte_ipv6_addr *ip);
/**
* Lookup less specific route into the RIB structure
@@ -154,7 +155,7 @@ rte_rib6_lookup_parent(struct rte_rib6_node *ent);
*/
struct rte_rib6_node *
rte_rib6_lookup_exact(struct rte_rib6 *rib,
- const uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE], uint8_t depth);
+ const struct rte_ipv6_addr *ip, uint8_t depth);
/**
* Retrieve next more specific prefix from the RIB
@@ -181,7 +182,7 @@ rte_rib6_lookup_exact(struct rte_rib6 *rib,
*/
struct rte_rib6_node *
rte_rib6_get_nxt(struct rte_rib6 *rib,
- const uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE],
+ const struct rte_ipv6_addr *ip,
uint8_t depth, struct rte_rib6_node *last, int flag);
/**
@@ -196,7 +197,7 @@ rte_rib6_get_nxt(struct rte_rib6 *rib,
*/
void
rte_rib6_remove(struct rte_rib6 *rib,
- const uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE], uint8_t depth);
+ const struct rte_ipv6_addr *ip, uint8_t depth);
/**
* Insert prefix into the RIB
@@ -213,7 +214,7 @@ rte_rib6_remove(struct rte_rib6 *rib,
*/
struct rte_rib6_node *
rte_rib6_insert(struct rte_rib6 *rib,
- const uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE], uint8_t depth);
+ const struct rte_ipv6_addr *ip, uint8_t depth);
/**
* Get an ip from rte_rib6_node
@@ -228,7 +229,7 @@ rte_rib6_insert(struct rte_rib6 *rib,
*/
int
rte_rib6_get_ip(const struct rte_rib6_node *node,
- uint8_t ip[RTE_RIB6_IPV6_ADDR_SIZE]);
+ struct rte_ipv6_addr *ip);
/**
* Get a depth from rte_rib6_node
diff --git a/lib/table/rte_table_lpm_ipv6.c b/lib/table/rte_table_lpm_ipv6.c
index c1a7412f92cf..dea11130d3d5 100644
--- a/lib/table/rte_table_lpm_ipv6.c
+++ b/lib/table/rte_table_lpm_ipv6.c
@@ -207,7 +207,7 @@ rte_table_lpm_ipv6_entry_add(
}
/* Check if rule is already present in the table */
- status = rte_lpm6_is_rule_present(lpm->lpm, ip_prefix->ip,
+ status = rte_lpm6_is_rule_present(lpm->lpm, &ip_prefix->ip,
ip_prefix->depth, &nht_pos0);
nht_pos0_valid = status > 0;
@@ -225,7 +225,7 @@ rte_table_lpm_ipv6_entry_add(
}
/* Add rule to low level LPM table */
- if (rte_lpm6_add(lpm->lpm, ip_prefix->ip, ip_prefix->depth,
+ if (rte_lpm6_add(lpm->lpm, &ip_prefix->ip, ip_prefix->depth,
nht_pos) < 0) {
TABLE_LOG(ERR, "%s: LPM IPv6 rule add failed", __func__);
return -1;
@@ -270,7 +270,7 @@ rte_table_lpm_ipv6_entry_delete(
}
/* Return if rule is not present in the table */
- status = rte_lpm6_is_rule_present(lpm->lpm, ip_prefix->ip,
+ status = rte_lpm6_is_rule_present(lpm->lpm, &ip_prefix->ip,
ip_prefix->depth, &nht_pos);
if (status < 0) {
TABLE_LOG(ERR, "%s: LPM IPv6 algorithmic error",
@@ -283,7 +283,7 @@ rte_table_lpm_ipv6_entry_delete(
}
/* Delete rule from the low-level LPM table */
- status = rte_lpm6_delete(lpm->lpm, ip_prefix->ip, ip_prefix->depth);
+ status = rte_lpm6_delete(lpm->lpm, &ip_prefix->ip, ip_prefix->depth);
if (status) {
TABLE_LOG(ERR, "%s: LPM IPv6 rule delete failed",
__func__);
@@ -323,11 +323,11 @@ rte_table_lpm_ipv6_lookup(
if (pkt_mask & pkts_mask) {
struct rte_mbuf *pkt = pkts[i];
- uint8_t *ip = RTE_MBUF_METADATA_UINT8_PTR(pkt,
- lpm->offset);
+ const struct rte_ipv6_addr *ip;
int status;
uint32_t nht_pos;
+ ip = (struct rte_ipv6_addr *)RTE_MBUF_METADATA_UINT8_PTR(pkt, lpm->offset);
status = rte_lpm6_lookup(lpm->lpm, ip, &nht_pos);
if (status == 0) {
pkts_out_mask |= pkt_mask;
diff --git a/lib/table/rte_table_lpm_ipv6.h b/lib/table/rte_table_lpm_ipv6.h
index 96ddbd32c20e..d96ce1179954 100644
--- a/lib/table/rte_table_lpm_ipv6.h
+++ b/lib/table/rte_table_lpm_ipv6.h
@@ -45,7 +45,7 @@ extern "C" {
#include "rte_table.h"
-#define RTE_LPM_IPV6_ADDR_SIZE 16
+#include <rte_ip6.h>
/** LPM table parameters */
struct rte_table_lpm_ipv6_params {
@@ -73,7 +73,7 @@ each rule covering for a multitude of lookup keys (destination IP addresses)
that share the same data (next hop). */
struct rte_table_lpm_ipv6_key {
/** IP address */
- uint8_t ip[RTE_LPM_IPV6_ADDR_SIZE];
+ struct rte_ipv6_addr ip;
/** IP address depth. The most significant "depth" bits of the IP
address specify the network part of the IP address, while the rest of
--
2.46.1
^ permalink raw reply [relevance 1%]
* [PATCH dpdk v2 04/16] net: use ipv6 structure for header addresses
2024-10-01 8:17 2% ` [PATCH dpdk v2 02/16] net: split ipv6 symbols in separate header Robin Jarry
@ 2024-10-01 8:17 1% ` Robin Jarry
2024-10-01 8:17 1% ` [PATCH dpdk v2 05/16] fib6,rib6,lpm6: use ipv6 addr struct Robin Jarry
` (7 subsequent siblings)
10 siblings, 0 replies; 200+ results
From: Robin Jarry @ 2024-10-01 8:17 UTC (permalink / raw)
To: dev, Wisam Jaddo, Cristian Dumitrescu, Konstantin Ananyev,
Yipeng Wang, Sameh Gobriel, Bruce Richardson, Vladimir Medvedkin,
Ajit Khaparde, Somnath Kotur, Chas Williams, Min Hu (Connor),
Potnuri Bharat Teja, Hemant Agrawal, Sachin Saxena, Ziyang Xuan,
Xiaoyun Wang, Jie Hai, Yisen Zhuang, Jingjing Wu,
Dariusz Sosnowski, Viacheslav Ovsiienko, Bing Zhao, Ori Kam,
Suanming Mou, Matan Azrad, Chaoyong He, Devendra Singh Rawat,
Alok Prasad, Andrew Rybchenko, Stephen Hemminger, Jiawen Wu,
Jian Wang, Radu Nicolau, Akhil Goyal, Thomas Monjalon,
Ferruh Yigit, Nithin Dabilpuram, Pavan Nikhilesh
The rte_ipv6_hdr uses ad-hoc uint8_t[16] arrays to represent addresses.
Replace these arrays with the previously introduced rte_ipv6_addr
structure. Adapt all code accordingly.
Signed-off-by: Robin Jarry <rjarry@redhat.com>
---
Notes:
v2: updated changelog for 24.11
app/test-flow-perf/items_gen.c | 4 +--
app/test-pipeline/pipeline_hash.c | 4 +--
app/test/packet_burst_generator.c | 4 +--
app/test/test_ipfrag.c | 4 +--
app/test/test_reassembly_perf.c | 20 +++++------
app/test/test_thash.c | 8 ++---
doc/guides/rel_notes/deprecation.rst | 2 --
doc/guides/rel_notes/release_24_11.rst | 2 ++
drivers/net/bnxt/bnxt_flow.c | 12 +++----
drivers/net/bonding/rte_eth_bond_pmd.c | 4 +--
drivers/net/cxgbe/cxgbe_flow.c | 14 ++++----
drivers/net/dpaa2/dpaa2_flow.c | 22 ++++++------
drivers/net/hinic/hinic_pmd_flow.c | 6 ++--
drivers/net/hinic/hinic_pmd_tx.c | 2 +-
drivers/net/hns3/hns3_flow.c | 8 ++---
drivers/net/i40e/i40e_flow.c | 12 +++----
drivers/net/iavf/iavf_fdir.c | 8 ++---
drivers/net/iavf/iavf_fsub.c | 8 ++---
drivers/net/iavf/iavf_ipsec_crypto.c | 6 ++--
drivers/net/ice/ice_fdir_filter.c | 12 +++----
drivers/net/ice/ice_switch_filter.c | 16 ++++-----
drivers/net/igc/igc_flow.c | 4 +--
drivers/net/ixgbe/ixgbe_flow.c | 12 +++----
drivers/net/ixgbe/ixgbe_ipsec.c | 4 +--
drivers/net/mlx5/hws/mlx5dr_definer.c | 36 +++++++++----------
drivers/net/mlx5/mlx5_flow.c | 6 ++--
drivers/net/mlx5/mlx5_flow_dv.c | 16 +++++----
drivers/net/mlx5/mlx5_flow_hw.c | 10 +++---
drivers/net/mlx5/mlx5_flow_verbs.c | 8 ++---
drivers/net/nfp/flower/nfp_flower_flow.c | 34 ++++++++++--------
drivers/net/nfp/nfp_net_flow.c | 44 +++++++++++++-----------
drivers/net/qede/qede_filter.c | 4 +--
drivers/net/sfc/sfc_flow.c | 28 +++++++--------
drivers/net/tap/tap_flow.c | 8 ++---
drivers/net/txgbe/txgbe_flow.c | 12 +++----
drivers/net/txgbe/txgbe_ipsec.c | 4 +--
examples/ip_fragmentation/main.c | 2 +-
examples/ip_pipeline/pipeline.c | 16 ++++-----
examples/ip_reassembly/main.c | 2 +-
examples/ipsec-secgw/flow.c | 12 +++----
examples/ipsec-secgw/ipsec.c | 8 ++---
examples/ipsec-secgw/sa.c | 4 +--
examples/ipsec-secgw/sad.h | 10 +++---
examples/l3fwd/l3fwd_fib.c | 2 +-
examples/l3fwd/l3fwd_lpm.c | 4 +--
lib/ethdev/rte_flow.h | 6 ++--
lib/hash/rte_thash.h | 12 +++----
lib/ip_frag/rte_ipv6_reassembly.c | 4 +--
lib/net/rte_ip6.h | 6 ++--
lib/node/ip6_lookup.c | 10 +++---
lib/pipeline/rte_swx_ipsec.c | 6 ++--
lib/pipeline/rte_table_action.c | 24 ++++++-------
52 files changed, 278 insertions(+), 258 deletions(-)
diff --git a/app/test-flow-perf/items_gen.c b/app/test-flow-perf/items_gen.c
index 4ae72509d445..c740e1838ffb 100644
--- a/app/test-flow-perf/items_gen.c
+++ b/app/test-flow-perf/items_gen.c
@@ -78,8 +78,8 @@ add_ipv6(struct rte_flow_item *items,
for (i = 0; i < 16; i++) {
/* Currently src_ip is limited to 32 bit */
if (i < 4)
- ipv6_specs[ti].hdr.src_addr[15 - i] = para.src_ip >> (i * 8);
- ipv6_masks[ti].hdr.src_addr[15 - i] = 0xff;
+ ipv6_specs[ti].hdr.src_addr.a[15 - i] = para.src_ip >> (i * 8);
+ ipv6_masks[ti].hdr.src_addr.a[15 - i] = 0xff;
}
items[items_counter].type = RTE_FLOW_ITEM_TYPE_IPV6;
diff --git a/app/test-pipeline/pipeline_hash.c b/app/test-pipeline/pipeline_hash.c
index cab9c2098014..194e5c5dcc53 100644
--- a/app/test-pipeline/pipeline_hash.c
+++ b/app/test-pipeline/pipeline_hash.c
@@ -432,7 +432,6 @@ app_main_loop_rx_metadata(void) {
struct rte_ipv4_hdr *ip_hdr;
struct rte_ipv6_hdr *ipv6_hdr;
uint32_t ip_dst;
- uint8_t *ipv6_dst;
uint32_t *signature, *k32;
m = app.mbuf_rx.array[j];
@@ -452,9 +451,8 @@ app_main_loop_rx_metadata(void) {
} else if (RTE_ETH_IS_IPV6_HDR(m->packet_type)) {
ipv6_hdr = (struct rte_ipv6_hdr *)
&m_data[sizeof(struct rte_ether_hdr)];
- ipv6_dst = ipv6_hdr->dst_addr;
- memcpy(key, ipv6_dst, 16);
+ memcpy(key, &ipv6_hdr->dst_addr, 16);
} else
continue;
diff --git a/app/test/packet_burst_generator.c b/app/test/packet_burst_generator.c
index 2cd34abc1a65..76a409a56452 100644
--- a/app/test/packet_burst_generator.c
+++ b/app/test/packet_burst_generator.c
@@ -149,8 +149,8 @@ initialize_ipv6_header(struct rte_ipv6_hdr *ip_hdr, uint8_t *src_addr,
ip_hdr->proto = IPPROTO_UDP;
ip_hdr->hop_limits = IP_DEFTTL;
- rte_memcpy(ip_hdr->src_addr, src_addr, sizeof(ip_hdr->src_addr));
- rte_memcpy(ip_hdr->dst_addr, dst_addr, sizeof(ip_hdr->dst_addr));
+ rte_memcpy(&ip_hdr->src_addr, src_addr, sizeof(ip_hdr->src_addr));
+ rte_memcpy(&ip_hdr->dst_addr, dst_addr, sizeof(ip_hdr->dst_addr));
return (uint16_t) (pkt_data_len + sizeof(struct rte_ipv6_hdr));
}
diff --git a/app/test/test_ipfrag.c b/app/test/test_ipfrag.c
index 8e4df220a214..4053bd9f0f79 100644
--- a/app/test/test_ipfrag.c
+++ b/app/test/test_ipfrag.c
@@ -238,8 +238,8 @@ v6_allocate_packet_of(struct rte_mbuf *b, int fill, size_t s, uint8_t ttl,
hdr->proto = proto;
hdr->hop_limits = ttl;
- memset(hdr->src_addr, 0x08, sizeof(hdr->src_addr));
- memset(hdr->dst_addr, 0x04, sizeof(hdr->src_addr));
+ memset(hdr->src_addr.a, 0x08, sizeof(hdr->src_addr));
+ memset(hdr->dst_addr.a, 0x04, sizeof(hdr->src_addr));
}
static inline void
diff --git a/app/test/test_reassembly_perf.c b/app/test/test_reassembly_perf.c
index 3912179022fc..70112c1f62da 100644
--- a/app/test/test_reassembly_perf.c
+++ b/app/test/test_reassembly_perf.c
@@ -340,17 +340,17 @@ ipv6_frag_fill_data(struct rte_mbuf **mbuf, uint8_t nb_frags, uint32_t flow_id,
rte_cpu_to_be_16(pkt_len - sizeof(struct rte_ipv6_hdr));
ip_hdr->proto = IPPROTO_FRAGMENT;
ip_hdr->hop_limits = IP_DEFTTL;
- memcpy(ip_hdr->src_addr, ip6_addr, sizeof(ip_hdr->src_addr));
- memcpy(ip_hdr->dst_addr, ip6_addr, sizeof(ip_hdr->dst_addr));
- ip_hdr->src_addr[7] = (flow_id >> 16) & 0xf;
- ip_hdr->src_addr[7] |= 0x10;
- ip_hdr->src_addr[8] = (flow_id >> 8) & 0xff;
- ip_hdr->src_addr[9] = flow_id & 0xff;
+ memcpy(&ip_hdr->src_addr, ip6_addr, sizeof(ip_hdr->src_addr));
+ memcpy(&ip_hdr->dst_addr, ip6_addr, sizeof(ip_hdr->dst_addr));
+ ip_hdr->src_addr.a[7] = (flow_id >> 16) & 0xf;
+ ip_hdr->src_addr.a[7] |= 0x10;
+ ip_hdr->src_addr.a[8] = (flow_id >> 8) & 0xff;
+ ip_hdr->src_addr.a[9] = flow_id & 0xff;
- ip_hdr->dst_addr[7] = (flow_id >> 16) & 0xf;
- ip_hdr->dst_addr[7] |= 0x20;
- ip_hdr->dst_addr[8] = (flow_id >> 8) & 0xff;
- ip_hdr->dst_addr[9] = flow_id & 0xff;
+ ip_hdr->dst_addr.a[7] = (flow_id >> 16) & 0xf;
+ ip_hdr->dst_addr.a[7] |= 0x20;
+ ip_hdr->dst_addr.a[8] = (flow_id >> 8) & 0xff;
+ ip_hdr->dst_addr.a[9] = flow_id & 0xff;
frag_hdr->next_header = IPPROTO_UDP;
frag_hdr->reserved = 0;
diff --git a/app/test/test_thash.c b/app/test/test_thash.c
index 65d42fd90085..952da6a52954 100644
--- a/app/test/test_thash.c
+++ b/app/test/test_thash.c
@@ -145,10 +145,10 @@ test_toeplitz_hash_calc(void)
}
for (i = 0; i < RTE_DIM(v6_tbl); i++) {
/*Fill ipv6 hdr*/
- for (j = 0; j < RTE_DIM(ipv6_hdr.src_addr); j++)
- ipv6_hdr.src_addr[j] = v6_tbl[i].src_ip[j];
- for (j = 0; j < RTE_DIM(ipv6_hdr.dst_addr); j++)
- ipv6_hdr.dst_addr[j] = v6_tbl[i].dst_ip[j];
+ for (j = 0; j < RTE_DIM(ipv6_hdr.src_addr.a); j++)
+ ipv6_hdr.src_addr.a[j] = v6_tbl[i].src_ip[j];
+ for (j = 0; j < RTE_DIM(ipv6_hdr.dst_addr.a); j++)
+ ipv6_hdr.dst_addr.a[j] = v6_tbl[i].dst_ip[j];
/*Load and convert ipv6 address into tuple*/
rte_thash_load_v6_addrs(&ipv6_hdr, &tuple);
tuple.v6.sport = v6_tbl[i].src_port;
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 211f59fdc9a2..439867289cf3 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -87,8 +87,6 @@ Deprecation Notices
- ``rte_lpm6_delete_bulk_func()``
- ``rte_lpm6_lookup()``
- ``rte_lpm6_lookup_bulk_func()``
- net
- - ``struct rte_ipv6_hdr``
node
- ``rte_node_ip6_route_add()``
pipeline
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 95b9809f4145..2d5b29b5a2d4 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -85,6 +85,8 @@ API Changes
=======================================================
* net: IPv6 related symbols were moved from ``<rte_ip.h>`` to the new ``<rte_ip6.h>`` header.
+* net: The ``rte_ipv6_hdr`` structure was modified to use ``struct rte_ipv6_addr`` instead of
+ ``uint8_t[16]`` fields.
ABI Changes
-----------
diff --git a/drivers/net/bnxt/bnxt_flow.c b/drivers/net/bnxt/bnxt_flow.c
index f25bc6ff78c7..bdc47ee78755 100644
--- a/drivers/net/bnxt/bnxt_flow.c
+++ b/drivers/net/bnxt/bnxt_flow.c
@@ -424,22 +424,22 @@ bnxt_validate_and_parse_flow_type(const struct rte_flow_attr *attr,
EM_FLOW_ALLOC_INPUT_EN_DST_IPADDR;
rte_memcpy(filter->src_ipaddr,
- ipv6_spec->hdr.src_addr, 16);
+ &ipv6_spec->hdr.src_addr, 16);
rte_memcpy(filter->dst_ipaddr,
- ipv6_spec->hdr.dst_addr, 16);
+ &ipv6_spec->hdr.dst_addr, 16);
- if (!bnxt_check_zero_bytes(ipv6_mask->hdr.src_addr,
+ if (!bnxt_check_zero_bytes(ipv6_mask->hdr.src_addr.a,
16)) {
rte_memcpy(filter->src_ipaddr_mask,
- ipv6_mask->hdr.src_addr, 16);
+ &ipv6_mask->hdr.src_addr, 16);
en |= !use_ntuple ? 0 :
NTUPLE_FLTR_ALLOC_INPUT_EN_SRC_IPADDR_MASK;
}
- if (!bnxt_check_zero_bytes(ipv6_mask->hdr.dst_addr,
+ if (!bnxt_check_zero_bytes(ipv6_mask->hdr.dst_addr.a,
16)) {
rte_memcpy(filter->dst_ipaddr_mask,
- ipv6_mask->hdr.dst_addr, 16);
+ &ipv6_mask->hdr.dst_addr, 16);
en |= !use_ntuple ? 0 :
NTUPLE_FLTR_ALLOC_INPUT_EN_DST_IPADDR_MASK;
}
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index c40d18d128b6..b89732f4a28a 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -690,9 +690,9 @@ static inline uint32_t
ipv6_hash(struct rte_ipv6_hdr *ipv6_hdr)
{
unaligned_uint32_t *word_src_addr =
- (unaligned_uint32_t *)&(ipv6_hdr->src_addr[0]);
+ (unaligned_uint32_t *)&ipv6_hdr->src_addr;
unaligned_uint32_t *word_dst_addr =
- (unaligned_uint32_t *)&(ipv6_hdr->dst_addr[0]);
+ (unaligned_uint32_t *)&ipv6_hdr->dst_addr;
return (word_src_addr[0] ^ word_dst_addr[0]) ^
(word_src_addr[1] ^ word_dst_addr[1]) ^
diff --git a/drivers/net/cxgbe/cxgbe_flow.c b/drivers/net/cxgbe/cxgbe_flow.c
index f5787c247f1f..37c566b131e9 100644
--- a/drivers/net/cxgbe/cxgbe_flow.c
+++ b/drivers/net/cxgbe/cxgbe_flow.c
@@ -411,15 +411,15 @@ ch_rte_parsetype_ipv6(const void *dmask, const struct rte_flow_item *item,
RTE_IPV6_HDR_TC_SHIFT,
tos);
- if (memcmp(val->hdr.dst_addr, z, sizeof(val->hdr.dst_addr)) ||
+ if (memcmp(&val->hdr.dst_addr, z, sizeof(val->hdr.dst_addr)) ||
(umask &&
- memcmp(umask->hdr.dst_addr, z, sizeof(umask->hdr.dst_addr))))
+ memcmp(&umask->hdr.dst_addr, z, sizeof(umask->hdr.dst_addr))))
CXGBE_FILL_FS_MEMCPY(val->hdr.dst_addr, mask->hdr.dst_addr,
lip);
- if (memcmp(val->hdr.src_addr, z, sizeof(val->hdr.src_addr)) ||
+ if (memcmp(&val->hdr.src_addr, z, sizeof(val->hdr.src_addr)) ||
(umask &&
- memcmp(umask->hdr.src_addr, z, sizeof(umask->hdr.src_addr))))
+ memcmp(&umask->hdr.src_addr, z, sizeof(umask->hdr.src_addr))))
CXGBE_FILL_FS_MEMCPY(val->hdr.src_addr, mask->hdr.src_addr,
fip);
@@ -918,12 +918,14 @@ static struct chrte_fparse parseitem[] = {
.fptr = ch_rte_parsetype_ipv6,
.dmask = &(const struct rte_flow_item_ipv6) {
.hdr = {
- .src_addr =
+ .src_addr = { .a =
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
- .dst_addr =
+ },
+ .dst_addr = { .a =
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
+ },
.vtc_flow = RTE_BE32(0xff000000),
},
},
diff --git a/drivers/net/dpaa2/dpaa2_flow.c b/drivers/net/dpaa2/dpaa2_flow.c
index 6c7bac4d483a..5f762d319477 100644
--- a/drivers/net/dpaa2/dpaa2_flow.c
+++ b/drivers/net/dpaa2/dpaa2_flow.c
@@ -117,12 +117,14 @@ static const struct rte_flow_item_ipv4 dpaa2_flow_item_ipv4_mask = {
static const struct rte_flow_item_ipv6 dpaa2_flow_item_ipv6_mask = {
.hdr = {
- .src_addr =
+ .src_addr = { .a =
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
- .dst_addr =
+ },
+ .dst_addr = { .a =
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
+ },
.proto = 0xff
},
};
@@ -1480,16 +1482,16 @@ dpaa2_configure_flow_generic_ip(
mask_ipv4->hdr.dst_addr)) {
flow->ipaddr_rule.ipaddr_type = FLOW_IPV4_ADDR;
} else if (mask_ipv6 &&
- (memcmp((const char *)mask_ipv6->hdr.src_addr,
+ (memcmp(&mask_ipv6->hdr.src_addr,
zero_cmp, NH_FLD_IPV6_ADDR_SIZE) ||
- memcmp((const char *)mask_ipv6->hdr.dst_addr,
+ memcmp(&mask_ipv6->hdr.dst_addr,
zero_cmp, NH_FLD_IPV6_ADDR_SIZE))) {
flow->ipaddr_rule.ipaddr_type = FLOW_IPV6_ADDR;
}
if ((mask_ipv4 && mask_ipv4->hdr.src_addr) ||
(mask_ipv6 &&
- memcmp((const char *)mask_ipv6->hdr.src_addr,
+ memcmp(&mask_ipv6->hdr.src_addr,
zero_cmp, NH_FLD_IPV6_ADDR_SIZE))) {
index = dpaa2_flow_extract_search(
&priv->extract.qos_key_extract.dpkg,
@@ -1528,13 +1530,13 @@ dpaa2_configure_flow_generic_ip(
if (spec_ipv4)
key = &spec_ipv4->hdr.src_addr;
else
- key = &spec_ipv6->hdr.src_addr[0];
+ key = &spec_ipv6->hdr.src_addr;
if (mask_ipv4) {
mask = &mask_ipv4->hdr.src_addr;
size = NH_FLD_IPV4_ADDR_SIZE;
prot = NET_PROT_IPV4;
} else {
- mask = &mask_ipv6->hdr.src_addr[0];
+ mask = &mask_ipv6->hdr.src_addr;
size = NH_FLD_IPV6_ADDR_SIZE;
prot = NET_PROT_IPV6;
}
@@ -1571,7 +1573,7 @@ dpaa2_configure_flow_generic_ip(
if ((mask_ipv4 && mask_ipv4->hdr.dst_addr) ||
(mask_ipv6 &&
- memcmp((const char *)mask_ipv6->hdr.dst_addr,
+ memcmp(&mask_ipv6->hdr.dst_addr,
zero_cmp, NH_FLD_IPV6_ADDR_SIZE))) {
index = dpaa2_flow_extract_search(
&priv->extract.qos_key_extract.dpkg,
@@ -1618,13 +1620,13 @@ dpaa2_configure_flow_generic_ip(
if (spec_ipv4)
key = &spec_ipv4->hdr.dst_addr;
else
- key = spec_ipv6->hdr.dst_addr;
+ key = &spec_ipv6->hdr.dst_addr;
if (mask_ipv4) {
mask = &mask_ipv4->hdr.dst_addr;
size = NH_FLD_IPV4_ADDR_SIZE;
prot = NET_PROT_IPV4;
} else {
- mask = &mask_ipv6->hdr.dst_addr[0];
+ mask = &mask_ipv6->hdr.dst_addr;
size = NH_FLD_IPV6_ADDR_SIZE;
prot = NET_PROT_IPV6;
}
diff --git a/drivers/net/hinic/hinic_pmd_flow.c b/drivers/net/hinic/hinic_pmd_flow.c
index d1a564a16303..8fdd5a35be9f 100644
--- a/drivers/net/hinic/hinic_pmd_flow.c
+++ b/drivers/net/hinic/hinic_pmd_flow.c
@@ -962,7 +962,7 @@ static int hinic_normal_item_check_ip(const struct rte_flow_item **in_out_item,
/* check ipv6 src addr mask, ipv6 src addr is 16 bytes */
for (i = 0; i < 16; i++) {
- if (ipv6_mask->hdr.src_addr[i] == UINT8_MAX) {
+ if (ipv6_mask->hdr.src_addr.a[i] == UINT8_MAX) {
rte_flow_error_set(error, EINVAL,
RTE_FLOW_ERROR_TYPE_ITEM, item,
"Not supported by fdir filter, do not support src ipv6");
@@ -978,13 +978,13 @@ static int hinic_normal_item_check_ip(const struct rte_flow_item **in_out_item,
}
for (i = 0; i < 16; i++) {
- if (ipv6_mask->hdr.dst_addr[i] == UINT8_MAX)
+ if (ipv6_mask->hdr.dst_addr.a[i] == UINT8_MAX)
rule->mask.dst_ipv6_mask |= 1 << i;
}
ipv6_spec = (const struct rte_flow_item_ipv6 *)item->spec;
rte_memcpy(rule->hinic_fdir.dst_ipv6,
- ipv6_spec->hdr.dst_addr, 16);
+ &ipv6_spec->hdr.dst_addr, 16);
/*
* Check if the next not void item is TCP or UDP or ICMP.
diff --git a/drivers/net/hinic/hinic_pmd_tx.c b/drivers/net/hinic/hinic_pmd_tx.c
index f09b1a6e1ea6..22fb0bffafcc 100644
--- a/drivers/net/hinic/hinic_pmd_tx.c
+++ b/drivers/net/hinic/hinic_pmd_tx.c
@@ -743,7 +743,7 @@ hinic_ipv6_phdr_cksum(const struct rte_ipv6_hdr *ipv6_hdr, uint64_t ol_flags)
else
psd_hdr.len = ipv6_hdr->payload_len;
- sum = __rte_raw_cksum(ipv6_hdr->src_addr,
+ sum = __rte_raw_cksum(&ipv6_hdr->src_addr,
sizeof(ipv6_hdr->src_addr) + sizeof(ipv6_hdr->dst_addr), 0);
sum = __rte_raw_cksum(&psd_hdr, sizeof(psd_hdr), sum);
return __rte_raw_cksum_reduce(sum);
diff --git a/drivers/net/hns3/hns3_flow.c b/drivers/net/hns3/hns3_flow.c
index 37eb2b4c3807..bf1eee506dde 100644
--- a/drivers/net/hns3/hns3_flow.c
+++ b/drivers/net/hns3/hns3_flow.c
@@ -822,10 +822,10 @@ hns3_parse_ipv6(const struct rte_flow_item *item, struct hns3_fdir_rule *rule,
"Only support src & dst ip,proto in IPV6");
}
net_addr_to_host(rule->key_conf.mask.src_ip,
- (const rte_be32_t *)ipv6_mask->hdr.src_addr,
+ (const rte_be32_t *)&ipv6_mask->hdr.src_addr,
IP_ADDR_LEN);
net_addr_to_host(rule->key_conf.mask.dst_ip,
- (const rte_be32_t *)ipv6_mask->hdr.dst_addr,
+ (const rte_be32_t *)&ipv6_mask->hdr.dst_addr,
IP_ADDR_LEN);
rule->key_conf.mask.ip_proto = ipv6_mask->hdr.proto;
if (rule->key_conf.mask.src_ip[IP_ADDR_KEY_ID])
@@ -838,10 +838,10 @@ hns3_parse_ipv6(const struct rte_flow_item *item, struct hns3_fdir_rule *rule,
ipv6_spec = item->spec;
net_addr_to_host(rule->key_conf.spec.src_ip,
- (const rte_be32_t *)ipv6_spec->hdr.src_addr,
+ (const rte_be32_t *)&ipv6_spec->hdr.src_addr,
IP_ADDR_LEN);
net_addr_to_host(rule->key_conf.spec.dst_ip,
- (const rte_be32_t *)ipv6_spec->hdr.dst_addr,
+ (const rte_be32_t *)&ipv6_spec->hdr.dst_addr,
IP_ADDR_LEN);
rule->key_conf.spec.ip_proto = ipv6_spec->hdr.proto;
diff --git a/drivers/net/i40e/i40e_flow.c b/drivers/net/i40e/i40e_flow.c
index 92165c8422d5..c6857727e8be 100644
--- a/drivers/net/i40e/i40e_flow.c
+++ b/drivers/net/i40e/i40e_flow.c
@@ -1953,13 +1953,13 @@ i40e_flow_parse_fdir_pattern(struct rte_eth_dev *dev,
return -rte_errno;
}
- if (!memcmp(ipv6_mask->hdr.src_addr,
+ if (!memcmp(&ipv6_mask->hdr.src_addr,
ipv6_addr_mask,
- RTE_DIM(ipv6_mask->hdr.src_addr)))
+ sizeof(ipv6_mask->hdr.src_addr)))
input_set |= I40E_INSET_IPV6_SRC;
- if (!memcmp(ipv6_mask->hdr.dst_addr,
+ if (!memcmp(&ipv6_mask->hdr.dst_addr,
ipv6_addr_mask,
- RTE_DIM(ipv6_mask->hdr.dst_addr)))
+ sizeof(ipv6_mask->hdr.dst_addr)))
input_set |= I40E_INSET_IPV6_DST;
if ((ipv6_mask->hdr.vtc_flow &
@@ -1987,9 +1987,9 @@ i40e_flow_parse_fdir_pattern(struct rte_eth_dev *dev,
I40E_FDIR_IPTYPE_IPV6;
rte_memcpy(filter->input.flow.ipv6_flow.src_ip,
- ipv6_spec->hdr.src_addr, 16);
+ &ipv6_spec->hdr.src_addr, 16);
rte_memcpy(filter->input.flow.ipv6_flow.dst_ip,
- ipv6_spec->hdr.dst_addr, 16);
+ &ipv6_spec->hdr.dst_addr, 16);
/* Check if it is fragment. */
if (ipv6_spec->hdr.proto ==
diff --git a/drivers/net/iavf/iavf_fdir.c b/drivers/net/iavf/iavf_fdir.c
index 811a10287b70..321346425465 100644
--- a/drivers/net/iavf/iavf_fdir.c
+++ b/drivers/net/iavf/iavf_fdir.c
@@ -1048,14 +1048,14 @@ iavf_fdir_parse_pattern(__rte_unused struct iavf_adapter *ad,
HOP_LIMIT);
}
- if (!memcmp(ipv6_mask->hdr.src_addr, ipv6_addr_mask,
- RTE_DIM(ipv6_mask->hdr.src_addr))) {
+ if (!memcmp(&ipv6_mask->hdr.src_addr, ipv6_addr_mask,
+ sizeof(ipv6_mask->hdr.src_addr))) {
input_set |= IAVF_INSET_IPV6_SRC;
VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, IPV6,
SRC);
}
- if (!memcmp(ipv6_mask->hdr.dst_addr, ipv6_addr_mask,
- RTE_DIM(ipv6_mask->hdr.dst_addr))) {
+ if (!memcmp(&ipv6_mask->hdr.dst_addr, ipv6_addr_mask,
+ sizeof(ipv6_mask->hdr.dst_addr))) {
input_set |= IAVF_INSET_IPV6_DST;
VIRTCHNL_ADD_PROTO_HDR_FIELD_BIT(hdr, IPV6,
DST);
diff --git a/drivers/net/iavf/iavf_fsub.c b/drivers/net/iavf/iavf_fsub.c
index 74e1e7099b8c..eb5a3feab189 100644
--- a/drivers/net/iavf/iavf_fsub.c
+++ b/drivers/net/iavf/iavf_fsub.c
@@ -354,23 +354,23 @@ iavf_fsub_parse_pattern(const struct rte_flow_item pattern[],
}
for (j = 0; j < IAVF_IPV6_ADDR_LENGTH; j++) {
- if (ipv6_mask->hdr.src_addr[j]) {
+ if (ipv6_mask->hdr.src_addr.a[j]) {
*input |= IAVF_INSET_IPV6_SRC;
break;
}
}
for (j = 0; j < IAVF_IPV6_ADDR_LENGTH; j++) {
- if (ipv6_mask->hdr.dst_addr[j]) {
+ if (ipv6_mask->hdr.dst_addr.a[j]) {
*input |= IAVF_INSET_IPV6_DST;
break;
}
}
for (j = 0; j < IAVF_IPV6_ADDR_LENGTH; j++) {
- if (ipv6_mask->hdr.src_addr[j])
+ if (ipv6_mask->hdr.src_addr.a[j])
input_set_byte++;
- if (ipv6_mask->hdr.dst_addr[j])
+ if (ipv6_mask->hdr.dst_addr.a[j])
input_set_byte++;
}
diff --git a/drivers/net/iavf/iavf_ipsec_crypto.c b/drivers/net/iavf/iavf_ipsec_crypto.c
index 6fd45ff45f3d..17bea5880aa3 100644
--- a/drivers/net/iavf/iavf_ipsec_crypto.c
+++ b/drivers/net/iavf/iavf_ipsec_crypto.c
@@ -1738,8 +1738,8 @@ static void
parse_ipv6_item(const struct rte_flow_item_ipv6 *item,
struct rte_ipv6_hdr *ipv6)
{
- memcpy(ipv6->src_addr, item->hdr.src_addr, 16);
- memcpy(ipv6->dst_addr, item->hdr.dst_addr, 16);
+ memcpy(&ipv6->src_addr, &item->hdr.src_addr, 16);
+ memcpy(&ipv6->dst_addr, &item->hdr.dst_addr, 16);
}
static void
@@ -1904,7 +1904,7 @@ iavf_ipsec_flow_create(struct iavf_adapter *ad,
ipsec_flow->spi,
0,
0,
- ipsec_flow->ipv6_hdr.dst_addr,
+ ipsec_flow->ipv6_hdr.dst_addr.a,
0,
ipsec_flow->is_udp,
ipsec_flow->udp_hdr.dst_port);
diff --git a/drivers/net/ice/ice_fdir_filter.c b/drivers/net/ice/ice_fdir_filter.c
index edd8cc8f1ab4..8c85779b5401 100644
--- a/drivers/net/ice/ice_fdir_filter.c
+++ b/drivers/net/ice/ice_fdir_filter.c
@@ -2097,11 +2097,11 @@ ice_fdir_parse_pattern(__rte_unused struct ice_adapter *ad,
return -rte_errno;
}
- if (!memcmp(ipv6_mask->hdr.src_addr, ipv6_addr_mask,
- RTE_DIM(ipv6_mask->hdr.src_addr)))
+ if (!memcmp(&ipv6_mask->hdr.src_addr, ipv6_addr_mask,
+ sizeof(ipv6_mask->hdr.src_addr)))
*input_set |= ICE_INSET_IPV6_SRC;
- if (!memcmp(ipv6_mask->hdr.dst_addr, ipv6_addr_mask,
- RTE_DIM(ipv6_mask->hdr.dst_addr)))
+ if (!memcmp(&ipv6_mask->hdr.dst_addr, ipv6_addr_mask,
+ sizeof(ipv6_mask->hdr.dst_addr)))
*input_set |= ICE_INSET_IPV6_DST;
if ((ipv6_mask->hdr.vtc_flow &
@@ -2113,8 +2113,8 @@ ice_fdir_parse_pattern(__rte_unused struct ice_adapter *ad,
if (ipv6_mask->hdr.hop_limits == UINT8_MAX)
*input_set |= ICE_INSET_IPV6_HOP_LIMIT;
- rte_memcpy(&p_v6->dst_ip, ipv6_spec->hdr.dst_addr, 16);
- rte_memcpy(&p_v6->src_ip, ipv6_spec->hdr.src_addr, 16);
+ rte_memcpy(&p_v6->dst_ip, &ipv6_spec->hdr.dst_addr, 16);
+ rte_memcpy(&p_v6->src_ip, &ipv6_spec->hdr.src_addr, 16);
vtc_flow_cpu = rte_be_to_cpu_32(ipv6_spec->hdr.vtc_flow);
p_v6->tc = (uint8_t)(vtc_flow_cpu >> ICE_FDIR_IPV6_TC_OFFSET);
p_v6->proto = ipv6_spec->hdr.proto;
diff --git a/drivers/net/ice/ice_switch_filter.c b/drivers/net/ice/ice_switch_filter.c
index 122b87f625a7..28bc775a2c34 100644
--- a/drivers/net/ice/ice_switch_filter.c
+++ b/drivers/net/ice/ice_switch_filter.c
@@ -665,13 +665,13 @@ ice_switch_parse_pattern(const struct rte_flow_item pattern[],
}
for (j = 0; j < ICE_IPV6_ADDR_LENGTH; j++) {
- if (ipv6_mask->hdr.src_addr[j]) {
+ if (ipv6_mask->hdr.src_addr.a[j]) {
*input |= ICE_INSET_IPV6_SRC;
break;
}
}
for (j = 0; j < ICE_IPV6_ADDR_LENGTH; j++) {
- if (ipv6_mask->hdr.dst_addr[j]) {
+ if (ipv6_mask->hdr.dst_addr.a[j]) {
*input |= ICE_INSET_IPV6_DST;
break;
}
@@ -691,18 +691,18 @@ ice_switch_parse_pattern(const struct rte_flow_item pattern[],
f = &list[t].h_u.ipv6_hdr;
s = &list[t].m_u.ipv6_hdr;
for (j = 0; j < ICE_IPV6_ADDR_LENGTH; j++) {
- if (ipv6_mask->hdr.src_addr[j]) {
+ if (ipv6_mask->hdr.src_addr.a[j]) {
f->src_addr[j] =
- ipv6_spec->hdr.src_addr[j];
+ ipv6_spec->hdr.src_addr.a[j];
s->src_addr[j] =
- ipv6_mask->hdr.src_addr[j];
+ ipv6_mask->hdr.src_addr.a[j];
input_set_byte++;
}
- if (ipv6_mask->hdr.dst_addr[j]) {
+ if (ipv6_mask->hdr.dst_addr.a[j]) {
f->dst_addr[j] =
- ipv6_spec->hdr.dst_addr[j];
+ ipv6_spec->hdr.dst_addr.a[j];
s->dst_addr[j] =
- ipv6_mask->hdr.dst_addr[j];
+ ipv6_mask->hdr.dst_addr.a[j];
input_set_byte++;
}
}
diff --git a/drivers/net/igc/igc_flow.c b/drivers/net/igc/igc_flow.c
index b677a0d61340..b778ac26135a 100644
--- a/drivers/net/igc/igc_flow.c
+++ b/drivers/net/igc/igc_flow.c
@@ -435,8 +435,8 @@ igc_parse_pattern_ipv6(const struct rte_flow_item *item,
if (mask->hdr.vtc_flow ||
mask->hdr.payload_len ||
mask->hdr.hop_limits ||
- !igc_is_zero_ipv6_addr(mask->hdr.src_addr) ||
- !igc_is_zero_ipv6_addr(mask->hdr.dst_addr))
+ !igc_is_zero_ipv6_addr(&mask->hdr.src_addr) ||
+ !igc_is_zero_ipv6_addr(&mask->hdr.dst_addr))
return rte_flow_error_set(error, EINVAL,
RTE_FLOW_ERROR_TYPE_ITEM, item,
"IPv6 only support protocol");
diff --git a/drivers/net/ixgbe/ixgbe_flow.c b/drivers/net/ixgbe/ixgbe_flow.c
index 687341c6b8d3..1b35ed5faabe 100644
--- a/drivers/net/ixgbe/ixgbe_flow.c
+++ b/drivers/net/ixgbe/ixgbe_flow.c
@@ -1917,9 +1917,9 @@ ixgbe_parse_fdir_filter_normal(struct rte_eth_dev *dev,
/* check src addr mask */
for (j = 0; j < 16; j++) {
- if (ipv6_mask->hdr.src_addr[j] == 0) {
+ if (ipv6_mask->hdr.src_addr.a[j] == 0) {
rule->mask.src_ipv6_mask &= ~(1 << j);
- } else if (ipv6_mask->hdr.src_addr[j] != UINT8_MAX) {
+ } else if (ipv6_mask->hdr.src_addr.a[j] != UINT8_MAX) {
memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
rte_flow_error_set(error, EINVAL,
RTE_FLOW_ERROR_TYPE_ITEM,
@@ -1930,9 +1930,9 @@ ixgbe_parse_fdir_filter_normal(struct rte_eth_dev *dev,
/* check dst addr mask */
for (j = 0; j < 16; j++) {
- if (ipv6_mask->hdr.dst_addr[j] == 0) {
+ if (ipv6_mask->hdr.dst_addr.a[j] == 0) {
rule->mask.dst_ipv6_mask &= ~(1 << j);
- } else if (ipv6_mask->hdr.dst_addr[j] != UINT8_MAX) {
+ } else if (ipv6_mask->hdr.dst_addr.a[j] != UINT8_MAX) {
memset(rule, 0, sizeof(struct ixgbe_fdir_rule));
rte_flow_error_set(error, EINVAL,
RTE_FLOW_ERROR_TYPE_ITEM,
@@ -1945,9 +1945,9 @@ ixgbe_parse_fdir_filter_normal(struct rte_eth_dev *dev,
rule->b_spec = TRUE;
ipv6_spec = item->spec;
rte_memcpy(rule->ixgbe_fdir.formatted.src_ip,
- ipv6_spec->hdr.src_addr, 16);
+ &ipv6_spec->hdr.src_addr, 16);
rte_memcpy(rule->ixgbe_fdir.formatted.dst_ip,
- ipv6_spec->hdr.dst_addr, 16);
+ &ipv6_spec->hdr.dst_addr, 16);
}
/**
diff --git a/drivers/net/ixgbe/ixgbe_ipsec.c b/drivers/net/ixgbe/ixgbe_ipsec.c
index d3313085560d..7615a19911ba 100644
--- a/drivers/net/ixgbe/ixgbe_ipsec.c
+++ b/drivers/net/ixgbe/ixgbe_ipsec.c
@@ -681,9 +681,9 @@ ixgbe_crypto_add_ingress_sa_from_flow(const void *sess,
ic_session->src_ip.type = IPv6;
ic_session->dst_ip.type = IPv6;
rte_memcpy(ic_session->src_ip.ipv6,
- ipv6->hdr.src_addr, 16);
+ &ipv6->hdr.src_addr, 16);
rte_memcpy(ic_session->dst_ip.ipv6,
- ipv6->hdr.dst_addr, 16);
+ &ipv6->hdr.dst_addr, 16);
} else {
const struct rte_flow_item_ipv4 *ipv4 = ip_spec;
ic_session->src_ip.type = IPv4;
diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.c b/drivers/net/mlx5/hws/mlx5dr_definer.c
index 51a3f7be4b75..7326d2cf42a5 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.c
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.c
@@ -176,14 +176,14 @@ struct mlx5dr_definer_conv_data {
X(SET, ipv6_proto, v->hdr.proto, rte_flow_item_ipv6) \
X(SET, ipv6_routing_hdr, IPPROTO_ROUTING, rte_flow_item_ipv6) \
X(SET, ipv6_hop_limits, v->hdr.hop_limits, rte_flow_item_ipv6) \
- X(SET_BE32P, ipv6_src_addr_127_96, &v->hdr.src_addr[0], rte_flow_item_ipv6) \
- X(SET_BE32P, ipv6_src_addr_95_64, &v->hdr.src_addr[4], rte_flow_item_ipv6) \
- X(SET_BE32P, ipv6_src_addr_63_32, &v->hdr.src_addr[8], rte_flow_item_ipv6) \
- X(SET_BE32P, ipv6_src_addr_31_0, &v->hdr.src_addr[12], rte_flow_item_ipv6) \
- X(SET_BE32P, ipv6_dst_addr_127_96, &v->hdr.dst_addr[0], rte_flow_item_ipv6) \
- X(SET_BE32P, ipv6_dst_addr_95_64, &v->hdr.dst_addr[4], rte_flow_item_ipv6) \
- X(SET_BE32P, ipv6_dst_addr_63_32, &v->hdr.dst_addr[8], rte_flow_item_ipv6) \
- X(SET_BE32P, ipv6_dst_addr_31_0, &v->hdr.dst_addr[12], rte_flow_item_ipv6) \
+ X(SET_BE32P, ipv6_src_addr_127_96, &v->hdr.src_addr.a[0], rte_flow_item_ipv6) \
+ X(SET_BE32P, ipv6_src_addr_95_64, &v->hdr.src_addr.a[4], rte_flow_item_ipv6) \
+ X(SET_BE32P, ipv6_src_addr_63_32, &v->hdr.src_addr.a[8], rte_flow_item_ipv6) \
+ X(SET_BE32P, ipv6_src_addr_31_0, &v->hdr.src_addr.a[12], rte_flow_item_ipv6) \
+ X(SET_BE32P, ipv6_dst_addr_127_96, &v->hdr.dst_addr.a[0], rte_flow_item_ipv6) \
+ X(SET_BE32P, ipv6_dst_addr_95_64, &v->hdr.dst_addr.a[4], rte_flow_item_ipv6) \
+ X(SET_BE32P, ipv6_dst_addr_63_32, &v->hdr.dst_addr.a[8], rte_flow_item_ipv6) \
+ X(SET_BE32P, ipv6_dst_addr_31_0, &v->hdr.dst_addr.a[12], rte_flow_item_ipv6) \
X(SET, ipv6_version, STE_IPV6, rte_flow_item_ipv6) \
X(SET, ipv6_frag, v->has_frag_ext, rte_flow_item_ipv6) \
X(SET, icmp_protocol, STE_ICMP, rte_flow_item_icmp) \
@@ -1161,8 +1161,8 @@ mlx5dr_definer_conv_item_ipv6(struct mlx5dr_definer_conv_data *cd,
m->has_esp_ext || m->has_dest_ext || m->has_mobil_ext ||
m->has_hip_ext || m->has_shim6_ext ||
(l && (l->has_frag_ext || l->hdr.vtc_flow || l->hdr.proto ||
- !is_mem_zero(l->hdr.src_addr, 16) ||
- !is_mem_zero(l->hdr.dst_addr, 16)))) {
+ !is_mem_zero(l->hdr.src_addr.a, 16) ||
+ !is_mem_zero(l->hdr.dst_addr.a, 16)))) {
rte_errno = ENOTSUP;
return rte_errno;
}
@@ -1219,56 +1219,56 @@ mlx5dr_definer_conv_item_ipv6(struct mlx5dr_definer_conv_data *cd,
DR_CALC_SET(fc, eth_l3, time_to_live_hop_limit, inner);
}
- if (!is_mem_zero(m->hdr.src_addr, 4)) {
+ if (!is_mem_zero(m->hdr.src_addr.a, 4)) {
fc = &cd->fc[DR_CALC_FNAME(IPV6_SRC_127_96, inner)];
fc->item_idx = item_idx;
fc->tag_set = &mlx5dr_definer_ipv6_src_addr_127_96_set;
DR_CALC_SET(fc, ipv6_src, ipv6_address_127_96, inner);
}
- if (!is_mem_zero(m->hdr.src_addr + 4, 4)) {
+ if (!is_mem_zero(m->hdr.src_addr.a + 4, 4)) {
fc = &cd->fc[DR_CALC_FNAME(IPV6_SRC_95_64, inner)];
fc->item_idx = item_idx;
fc->tag_set = &mlx5dr_definer_ipv6_src_addr_95_64_set;
DR_CALC_SET(fc, ipv6_src, ipv6_address_95_64, inner);
}
- if (!is_mem_zero(m->hdr.src_addr + 8, 4)) {
+ if (!is_mem_zero(m->hdr.src_addr.a + 8, 4)) {
fc = &cd->fc[DR_CALC_FNAME(IPV6_SRC_63_32, inner)];
fc->item_idx = item_idx;
fc->tag_set = &mlx5dr_definer_ipv6_src_addr_63_32_set;
DR_CALC_SET(fc, ipv6_src, ipv6_address_63_32, inner);
}
- if (!is_mem_zero(m->hdr.src_addr + 12, 4)) {
+ if (!is_mem_zero(m->hdr.src_addr.a + 12, 4)) {
fc = &cd->fc[DR_CALC_FNAME(IPV6_SRC_31_0, inner)];
fc->item_idx = item_idx;
fc->tag_set = &mlx5dr_definer_ipv6_src_addr_31_0_set;
DR_CALC_SET(fc, ipv6_src, ipv6_address_31_0, inner);
}
- if (!is_mem_zero(m->hdr.dst_addr, 4)) {
+ if (!is_mem_zero(m->hdr.dst_addr.a, 4)) {
fc = &cd->fc[DR_CALC_FNAME(IPV6_DST_127_96, inner)];
fc->item_idx = item_idx;
fc->tag_set = &mlx5dr_definer_ipv6_dst_addr_127_96_set;
DR_CALC_SET(fc, ipv6_dst, ipv6_address_127_96, inner);
}
- if (!is_mem_zero(m->hdr.dst_addr + 4, 4)) {
+ if (!is_mem_zero(m->hdr.dst_addr.a + 4, 4)) {
fc = &cd->fc[DR_CALC_FNAME(IPV6_DST_95_64, inner)];
fc->item_idx = item_idx;
fc->tag_set = &mlx5dr_definer_ipv6_dst_addr_95_64_set;
DR_CALC_SET(fc, ipv6_dst, ipv6_address_95_64, inner);
}
- if (!is_mem_zero(m->hdr.dst_addr + 8, 4)) {
+ if (!is_mem_zero(m->hdr.dst_addr.a + 8, 4)) {
fc = &cd->fc[DR_CALC_FNAME(IPV6_DST_63_32, inner)];
fc->item_idx = item_idx;
fc->tag_set = &mlx5dr_definer_ipv6_dst_addr_63_32_set;
DR_CALC_SET(fc, ipv6_dst, ipv6_address_63_32, inner);
}
- if (!is_mem_zero(m->hdr.dst_addr + 12, 4)) {
+ if (!is_mem_zero(m->hdr.dst_addr.a + 12, 4)) {
fc = &cd->fc[DR_CALC_FNAME(IPV6_DST_31_0, inner)];
fc->item_idx = item_idx;
fc->tag_set = &mlx5dr_definer_ipv6_dst_addr_31_0_set;
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index 72fb3a55baf4..ba03a246fc84 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -2933,12 +2933,14 @@ mlx5_flow_validate_item_ipv6(const struct rte_eth_dev *dev,
const struct rte_flow_item_ipv6 *spec = item->spec;
const struct rte_flow_item_ipv6 nic_mask = {
.hdr = {
- .src_addr =
+ .src_addr = { .a =
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
- .dst_addr =
+ },
+ .dst_addr = { .a =
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
+ },
.vtc_flow = RTE_BE32(0xffffffff),
.proto = 0xff,
},
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 89057edbcfc2..30b9dd898ffc 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -7697,12 +7697,14 @@ const struct rte_flow_item_ipv4 nic_ipv4_mask = {
const struct rte_flow_item_ipv6 nic_ipv6_mask = {
.hdr = {
- .src_addr =
+ .src_addr = { .a =
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
- .dst_addr =
+ },
+ .dst_addr = { .a =
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
+ },
.vtc_flow = RTE_BE32(0xffffffff),
.proto = 0xff,
.hop_limits = 0xff,
@@ -9549,12 +9551,14 @@ flow_dv_translate_item_ipv6(void *key, const struct rte_flow_item *item,
const struct rte_flow_item_ipv6 *ipv6_v;
const struct rte_flow_item_ipv6 nic_mask = {
.hdr = {
- .src_addr =
+ .src_addr = { .a =
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
- .dst_addr =
+ },
+ .dst_addr = { .a =
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
+ },
.vtc_flow = RTE_BE32(0xffffffff),
.proto = 0xff,
.hop_limits = 0xff,
@@ -9577,11 +9581,11 @@ flow_dv_translate_item_ipv6(void *key, const struct rte_flow_item *item,
l24_v = MLX5_ADDR_OF(fte_match_set_lyr_2_4, headers_v,
dst_ipv4_dst_ipv6.ipv6_layout.ipv6);
for (i = 0; i < size; ++i)
- l24_v[i] = ipv6_m->hdr.dst_addr[i] & ipv6_v->hdr.dst_addr[i];
+ l24_v[i] = ipv6_m->hdr.dst_addr.a[i] & ipv6_v->hdr.dst_addr.a[i];
l24_v = MLX5_ADDR_OF(fte_match_set_lyr_2_4, headers_v,
src_ipv4_src_ipv6.ipv6_layout.ipv6);
for (i = 0; i < size; ++i)
- l24_v[i] = ipv6_m->hdr.src_addr[i] & ipv6_v->hdr.src_addr[i];
+ l24_v[i] = ipv6_m->hdr.src_addr.a[i] & ipv6_v->hdr.src_addr.a[i];
/* TOS. */
vtc_v = rte_be_to_cpu_32(ipv6_m->hdr.vtc_flow & ipv6_v->hdr.vtc_flow);
MLX5_SET(fte_match_set_lyr_2_4, headers_v, ip_ecn, vtc_v >> 20);
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index d243b59b71cd..1b6c79a3f332 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -8250,12 +8250,14 @@ const struct rte_flow_item_ipv6 hws_nic_ipv6_mask = {
.payload_len = RTE_BE16(0xffff),
.proto = 0xff,
.hop_limits = 0xff,
- .src_addr =
+ .src_addr = { .a =
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
- .dst_addr =
+ },
+ .dst_addr = { .a =
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
+ },
},
.has_frag_ext = 1,
};
@@ -14645,10 +14647,10 @@ flow_hw_calc_encap_hash(struct rte_eth_dev *dev,
break;
case RTE_FLOW_ITEM_TYPE_IPV6:
memcpy(data.dst.ipv6_addr,
- ((const struct rte_flow_item_ipv6 *)(pattern->spec))->hdr.dst_addr,
+ &((const struct rte_flow_item_ipv6 *)(pattern->spec))->hdr.dst_addr,
sizeof(data.dst.ipv6_addr));
memcpy(data.src.ipv6_addr,
- ((const struct rte_flow_item_ipv6 *)(pattern->spec))->hdr.src_addr,
+ &((const struct rte_flow_item_ipv6 *)(pattern->spec))->hdr.src_addr,
sizeof(data.src.ipv6_addr));
break;
case RTE_FLOW_ITEM_TYPE_UDP:
diff --git a/drivers/net/mlx5/mlx5_flow_verbs.c b/drivers/net/mlx5/mlx5_flow_verbs.c
index 3a4356c0f650..5b4a4eda3bbc 100644
--- a/drivers/net/mlx5/mlx5_flow_verbs.c
+++ b/drivers/net/mlx5/mlx5_flow_verbs.c
@@ -600,13 +600,13 @@ flow_verbs_translate_item_ipv6(struct mlx5_flow *dev_flow,
uint32_t vtc_flow_val;
uint32_t vtc_flow_mask;
- memcpy(&ipv6.val.src_ip, spec->hdr.src_addr,
+ memcpy(&ipv6.val.src_ip, &spec->hdr.src_addr,
RTE_DIM(ipv6.val.src_ip));
- memcpy(&ipv6.val.dst_ip, spec->hdr.dst_addr,
+ memcpy(&ipv6.val.dst_ip, &spec->hdr.dst_addr,
RTE_DIM(ipv6.val.dst_ip));
- memcpy(&ipv6.mask.src_ip, mask->hdr.src_addr,
+ memcpy(&ipv6.mask.src_ip, &mask->hdr.src_addr,
RTE_DIM(ipv6.mask.src_ip));
- memcpy(&ipv6.mask.dst_ip, mask->hdr.dst_addr,
+ memcpy(&ipv6.mask.dst_ip, &mask->hdr.dst_addr,
RTE_DIM(ipv6.mask.dst_ip));
vtc_flow_val = rte_be_to_cpu_32(spec->hdr.vtc_flow);
vtc_flow_mask = rte_be_to_cpu_32(mask->hdr.vtc_flow);
diff --git a/drivers/net/nfp/flower/nfp_flower_flow.c b/drivers/net/nfp/flower/nfp_flower_flow.c
index 0078455658ac..3f597d394b24 100644
--- a/drivers/net/nfp/flower/nfp_flower_flow.c
+++ b/drivers/net/nfp/flower/nfp_flower_flow.c
@@ -2027,18 +2027,18 @@ nfp_flow_merge_ipv6(struct nfp_flow_merge_param *param)
ipv6_gre_tun->ip_ext.tos = vtc_flow >> RTE_IPV6_HDR_TC_SHIFT;
ipv6_gre_tun->ip_ext.ttl = hdr->hop_limits;
- memcpy(ipv6_gre_tun->ipv6.ipv6_src, hdr->src_addr,
+ memcpy(ipv6_gre_tun->ipv6.ipv6_src, &hdr->src_addr,
sizeof(ipv6_gre_tun->ipv6.ipv6_src));
- memcpy(ipv6_gre_tun->ipv6.ipv6_dst, hdr->dst_addr,
+ memcpy(ipv6_gre_tun->ipv6.ipv6_dst, &hdr->dst_addr,
sizeof(ipv6_gre_tun->ipv6.ipv6_dst));
} else {
ipv6_udp_tun = (struct nfp_flower_ipv6_udp_tun *)(*param->mbuf_off);
ipv6_udp_tun->ip_ext.tos = vtc_flow >> RTE_IPV6_HDR_TC_SHIFT;
ipv6_udp_tun->ip_ext.ttl = hdr->hop_limits;
- memcpy(ipv6_udp_tun->ipv6.ipv6_src, hdr->src_addr,
+ memcpy(ipv6_udp_tun->ipv6.ipv6_src, &hdr->src_addr,
sizeof(ipv6_udp_tun->ipv6.ipv6_src));
- memcpy(ipv6_udp_tun->ipv6.ipv6_dst, hdr->dst_addr,
+ memcpy(ipv6_udp_tun->ipv6.ipv6_dst, &hdr->dst_addr,
sizeof(ipv6_udp_tun->ipv6.ipv6_dst));
}
} else {
@@ -2061,8 +2061,8 @@ nfp_flow_merge_ipv6(struct nfp_flow_merge_param *param)
ipv6->ip_ext.tos = vtc_flow >> RTE_IPV6_HDR_TC_SHIFT;
ipv6->ip_ext.proto = hdr->proto;
ipv6->ip_ext.ttl = hdr->hop_limits;
- memcpy(ipv6->ipv6_src, hdr->src_addr, sizeof(ipv6->ipv6_src));
- memcpy(ipv6->ipv6_dst, hdr->dst_addr, sizeof(ipv6->ipv6_dst));
+ memcpy(ipv6->ipv6_src, &hdr->src_addr, sizeof(ipv6->ipv6_src));
+ memcpy(ipv6->ipv6_dst, &hdr->dst_addr, sizeof(ipv6->ipv6_dst));
ipv6_end:
*param->mbuf_off += sizeof(struct nfp_flower_ipv6);
@@ -2518,10 +2518,14 @@ static const struct nfp_flow_item_proc nfp_flow_item_proc_list[] = {
.vtc_flow = RTE_BE32(0x0ff00000),
.proto = 0xff,
.hop_limits = 0xff,
- .src_addr = "\xff\xff\xff\xff\xff\xff\xff\xff"
+ .src_addr = { .a =
+ "\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
- .dst_addr = "\xff\xff\xff\xff\xff\xff\xff\xff"
+ },
+ .dst_addr = { .a =
+ "\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
+ },
},
.has_frag_ext = 1,
},
@@ -3324,8 +3328,8 @@ nfp_flower_add_tun_neigh_v6_encap(struct nfp_app_fw_flower *app_fw_flower,
struct nfp_flower_cmsg_tun_neigh_v6 payload;
tun->payload.v6_flag = 1;
- memcpy(tun->payload.dst.dst_ipv6, ipv6->hdr.dst_addr, sizeof(tun->payload.dst.dst_ipv6));
- memcpy(tun->payload.src.src_ipv6, ipv6->hdr.src_addr, sizeof(tun->payload.src.src_ipv6));
+ memcpy(tun->payload.dst.dst_ipv6, &ipv6->hdr.dst_addr, sizeof(tun->payload.dst.dst_ipv6));
+ memcpy(tun->payload.src.src_ipv6, &ipv6->hdr.src_addr, sizeof(tun->payload.src.src_ipv6));
memcpy(tun->payload.dst_addr, eth->dst_addr.addr_bytes, RTE_ETHER_ADDR_LEN);
memcpy(tun->payload.src_addr, eth->src_addr.addr_bytes, RTE_ETHER_ADDR_LEN);
@@ -3345,8 +3349,8 @@ nfp_flower_add_tun_neigh_v6_encap(struct nfp_app_fw_flower *app_fw_flower,
sizeof(struct nfp_flower_meta_tci));
memset(&payload, 0, sizeof(struct nfp_flower_cmsg_tun_neigh_v6));
- memcpy(payload.dst_ipv6, ipv6->hdr.dst_addr, sizeof(payload.dst_ipv6));
- memcpy(payload.src_ipv6, ipv6->hdr.src_addr, sizeof(payload.src_ipv6));
+ memcpy(payload.dst_ipv6, &ipv6->hdr.dst_addr, sizeof(payload.dst_ipv6));
+ memcpy(payload.src_ipv6, &ipv6->hdr.src_addr, sizeof(payload.src_ipv6));
memcpy(payload.common.dst_mac, eth->dst_addr.addr_bytes, RTE_ETHER_ADDR_LEN);
memcpy(payload.common.src_mac, eth->src_addr.addr_bytes, RTE_ETHER_ADDR_LEN);
payload.common.port_id = port->in_port;
@@ -3573,7 +3577,7 @@ nfp_flow_action_vxlan_encap_v6(struct nfp_app_fw_flower *app_fw_flower,
pre_tun = (struct nfp_fl_act_pre_tun *)actions;
memset(pre_tun, 0, act_pre_size);
- nfp_flow_pre_tun_v6_process(pre_tun, ipv6->hdr.dst_addr);
+ nfp_flow_pre_tun_v6_process(pre_tun, ipv6->hdr.dst_addr.a);
set_tun = (struct nfp_fl_act_set_tun *)(act_data + act_pre_size);
memset(set_tun, 0, act_set_size);
@@ -3944,7 +3948,7 @@ nfp_flow_action_geneve_encap_v6(struct nfp_app_fw_flower *app_fw_flower,
pre_tun = (struct nfp_fl_act_pre_tun *)actions;
memset(pre_tun, 0, act_pre_size);
- nfp_flow_pre_tun_v6_process(pre_tun, ipv6->hdr.dst_addr);
+ nfp_flow_pre_tun_v6_process(pre_tun, ipv6->hdr.dst_addr.a);
set_tun = (struct nfp_fl_act_set_tun *)(act_data + act_pre_size);
memset(set_tun, 0, act_set_size);
@@ -4021,7 +4025,7 @@ nfp_flow_action_nvgre_encap_v6(struct nfp_app_fw_flower *app_fw_flower,
pre_tun = (struct nfp_fl_act_pre_tun *)actions;
memset(pre_tun, 0, act_pre_size);
- nfp_flow_pre_tun_v6_process(pre_tun, ipv6->hdr.dst_addr);
+ nfp_flow_pre_tun_v6_process(pre_tun, ipv6->hdr.dst_addr.a);
set_tun = (struct nfp_fl_act_set_tun *)(act_data + act_pre_size);
memset(set_tun, 0, act_set_size);
diff --git a/drivers/net/nfp/nfp_net_flow.c b/drivers/net/nfp/nfp_net_flow.c
index 2c7801ed41f4..aa3fa08ddf25 100644
--- a/drivers/net/nfp/nfp_net_flow.c
+++ b/drivers/net/nfp/nfp_net_flow.c
@@ -294,28 +294,28 @@ nfp_net_flow_merge_ipv6(struct rte_flow *nfp_flow,
ipv6->l4_protocol_mask = mask->hdr.proto;
for (i = 0; i < sizeof(ipv6->src_ipv6); i += 4) {
- ipv6->src_ipv6_mask[i] = mask->hdr.src_addr[i + 3];
- ipv6->src_ipv6_mask[i + 1] = mask->hdr.src_addr[i + 2];
- ipv6->src_ipv6_mask[i + 2] = mask->hdr.src_addr[i + 1];
- ipv6->src_ipv6_mask[i + 3] = mask->hdr.src_addr[i];
+ ipv6->src_ipv6_mask[i] = mask->hdr.src_addr.a[i + 3];
+ ipv6->src_ipv6_mask[i + 1] = mask->hdr.src_addr.a[i + 2];
+ ipv6->src_ipv6_mask[i + 2] = mask->hdr.src_addr.a[i + 1];
+ ipv6->src_ipv6_mask[i + 3] = mask->hdr.src_addr.a[i];
- ipv6->dst_ipv6_mask[i] = mask->hdr.dst_addr[i + 3];
- ipv6->dst_ipv6_mask[i + 1] = mask->hdr.dst_addr[i + 2];
- ipv6->dst_ipv6_mask[i + 2] = mask->hdr.dst_addr[i + 1];
- ipv6->dst_ipv6_mask[i + 3] = mask->hdr.dst_addr[i];
+ ipv6->dst_ipv6_mask[i] = mask->hdr.dst_addr.a[i + 3];
+ ipv6->dst_ipv6_mask[i + 1] = mask->hdr.dst_addr.a[i + 2];
+ ipv6->dst_ipv6_mask[i + 2] = mask->hdr.dst_addr.a[i + 1];
+ ipv6->dst_ipv6_mask[i + 3] = mask->hdr.dst_addr.a[i];
}
ipv6->l4_protocol = spec->hdr.proto;
for (i = 0; i < sizeof(ipv6->src_ipv6); i += 4) {
- ipv6->src_ipv6[i] = spec->hdr.src_addr[i + 3];
- ipv6->src_ipv6[i + 1] = spec->hdr.src_addr[i + 2];
- ipv6->src_ipv6[i + 2] = spec->hdr.src_addr[i + 1];
- ipv6->src_ipv6[i + 3] = spec->hdr.src_addr[i];
+ ipv6->src_ipv6[i] = spec->hdr.src_addr.a[i + 3];
+ ipv6->src_ipv6[i + 1] = spec->hdr.src_addr.a[i + 2];
+ ipv6->src_ipv6[i + 2] = spec->hdr.src_addr.a[i + 1];
+ ipv6->src_ipv6[i + 3] = spec->hdr.src_addr.a[i];
- ipv6->dst_ipv6[i] = spec->hdr.dst_addr[i + 3];
- ipv6->dst_ipv6[i + 1] = spec->hdr.dst_addr[i + 2];
- ipv6->dst_ipv6[i + 2] = spec->hdr.dst_addr[i + 1];
- ipv6->dst_ipv6[i + 3] = spec->hdr.dst_addr[i];
+ ipv6->dst_ipv6[i] = spec->hdr.dst_addr.a[i + 3];
+ ipv6->dst_ipv6[i + 1] = spec->hdr.dst_addr.a[i + 2];
+ ipv6->dst_ipv6[i + 2] = spec->hdr.dst_addr.a[i + 1];
+ ipv6->dst_ipv6[i + 3] = spec->hdr.dst_addr.a[i];
}
return 0;
@@ -403,10 +403,14 @@ static const struct nfp_net_flow_item_proc nfp_net_flow_item_proc_list[] = {
.mask_support = &(const struct rte_flow_item_ipv6){
.hdr = {
.proto = 0xff,
- .src_addr = "\xff\xff\xff\xff\xff\xff\xff\xff"
- "\xff\xff\xff\xff\xff\xff\xff\xff",
- .dst_addr = "\xff\xff\xff\xff\xff\xff\xff\xff"
- "\xff\xff\xff\xff\xff\xff\xff\xff",
+ .src_addr = { .a =
+ "\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\xff\xff\xff\xff\xff\xff",
+ },
+ .dst_addr = { .a =
+ "\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\xff\xff\xff\xff\xff\xff",
+ },
},
},
.mask_default = &rte_flow_item_ipv6_mask,
diff --git a/drivers/net/qede/qede_filter.c b/drivers/net/qede/qede_filter.c
index d98266eac55c..14fb4338e9c7 100644
--- a/drivers/net/qede/qede_filter.c
+++ b/drivers/net/qede/qede_filter.c
@@ -794,9 +794,9 @@ qede_flow_parse_pattern(__rte_unused struct rte_eth_dev *dev,
spec = pattern->spec;
memcpy(flow->entry.tuple.src_ipv6,
- spec->hdr.src_addr, IPV6_ADDR_LEN);
+ &spec->hdr.src_addr, IPV6_ADDR_LEN);
memcpy(flow->entry.tuple.dst_ipv6,
- spec->hdr.dst_addr, IPV6_ADDR_LEN);
+ &spec->hdr.dst_addr, IPV6_ADDR_LEN);
flow->entry.tuple.eth_proto =
RTE_ETHER_TYPE_IPV6;
}
diff --git a/drivers/net/sfc/sfc_flow.c b/drivers/net/sfc/sfc_flow.c
index 1b50aefe5c48..0a812d9928ab 100644
--- a/drivers/net/sfc/sfc_flow.c
+++ b/drivers/net/sfc/sfc_flow.c
@@ -575,14 +575,14 @@ sfc_flow_parse_ipv6(const struct rte_flow_item *item,
const uint16_t ether_type_ipv6 = rte_cpu_to_le_16(EFX_ETHER_TYPE_IPV6);
const struct rte_flow_item_ipv6 supp_mask = {
.hdr = {
- .src_addr = { 0xff, 0xff, 0xff, 0xff,
- 0xff, 0xff, 0xff, 0xff,
- 0xff, 0xff, 0xff, 0xff,
- 0xff, 0xff, 0xff, 0xff },
- .dst_addr = { 0xff, 0xff, 0xff, 0xff,
- 0xff, 0xff, 0xff, 0xff,
- 0xff, 0xff, 0xff, 0xff,
- 0xff, 0xff, 0xff, 0xff },
+ .src_addr = { .a =
+ "\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\xff\xff\xff\xff\xff\xff",
+ },
+ .dst_addr = { .a =
+ "\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\xff\xff\xff\xff\xff\xff",
+ },
.proto = 0xff,
}
};
@@ -618,28 +618,28 @@ sfc_flow_parse_ipv6(const struct rte_flow_item *item,
* IPv6 addresses are in big-endian byte order in item and in
* efx_spec
*/
- if (memcmp(mask->hdr.src_addr, supp_mask.hdr.src_addr,
+ if (memcmp(&mask->hdr.src_addr, &supp_mask.hdr.src_addr,
sizeof(mask->hdr.src_addr)) == 0) {
efx_spec->efs_match_flags |= EFX_FILTER_MATCH_REM_HOST;
RTE_BUILD_BUG_ON(sizeof(efx_spec->efs_rem_host) !=
sizeof(spec->hdr.src_addr));
- rte_memcpy(&efx_spec->efs_rem_host, spec->hdr.src_addr,
+ rte_memcpy(&efx_spec->efs_rem_host, &spec->hdr.src_addr,
sizeof(efx_spec->efs_rem_host));
- } else if (!sfc_flow_is_zero(mask->hdr.src_addr,
+ } else if (!sfc_flow_is_zero(mask->hdr.src_addr.a,
sizeof(mask->hdr.src_addr))) {
goto fail_bad_mask;
}
- if (memcmp(mask->hdr.dst_addr, supp_mask.hdr.dst_addr,
+ if (memcmp(&mask->hdr.dst_addr, &supp_mask.hdr.dst_addr,
sizeof(mask->hdr.dst_addr)) == 0) {
efx_spec->efs_match_flags |= EFX_FILTER_MATCH_LOC_HOST;
RTE_BUILD_BUG_ON(sizeof(efx_spec->efs_loc_host) !=
sizeof(spec->hdr.dst_addr));
- rte_memcpy(&efx_spec->efs_loc_host, spec->hdr.dst_addr,
+ rte_memcpy(&efx_spec->efs_loc_host, &spec->hdr.dst_addr,
sizeof(efx_spec->efs_loc_host));
- } else if (!sfc_flow_is_zero(mask->hdr.dst_addr,
+ } else if (!sfc_flow_is_zero(mask->hdr.dst_addr.a,
sizeof(mask->hdr.dst_addr))) {
goto fail_bad_mask;
}
diff --git a/drivers/net/tap/tap_flow.c b/drivers/net/tap/tap_flow.c
index 0a90c0487b46..8f5a8d64587b 100644
--- a/drivers/net/tap/tap_flow.c
+++ b/drivers/net/tap/tap_flow.c
@@ -209,11 +209,11 @@ static const struct tap_flow_items tap_flow_items[] = {
RTE_FLOW_ITEM_TYPE_TCP),
.mask = &(const struct rte_flow_item_ipv6){
.hdr = {
- .src_addr = {
+ .src_addr = { .a =
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
},
- .dst_addr = {
+ .dst_addr = { .a =
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
},
@@ -617,13 +617,13 @@ tap_flow_create_ipv6(const struct rte_flow_item *item, void *data)
info->eth_type = htons(ETH_P_IPV6);
if (!spec)
return 0;
- if (memcmp(mask->hdr.dst_addr, empty_addr, 16)) {
+ if (memcmp(&mask->hdr.dst_addr, empty_addr, 16)) {
tap_nlattr_add(&msg->nh, TCA_FLOWER_KEY_IPV6_DST,
sizeof(spec->hdr.dst_addr), &spec->hdr.dst_addr);
tap_nlattr_add(&msg->nh, TCA_FLOWER_KEY_IPV6_DST_MASK,
sizeof(mask->hdr.dst_addr), &mask->hdr.dst_addr);
}
- if (memcmp(mask->hdr.src_addr, empty_addr, 16)) {
+ if (memcmp(&mask->hdr.src_addr, empty_addr, 16)) {
tap_nlattr_add(&msg->nh, TCA_FLOWER_KEY_IPV6_SRC,
sizeof(spec->hdr.src_addr), &spec->hdr.src_addr);
tap_nlattr_add(&msg->nh, TCA_FLOWER_KEY_IPV6_SRC_MASK,
diff --git a/drivers/net/txgbe/txgbe_flow.c b/drivers/net/txgbe/txgbe_flow.c
index 7ef52d0b0fcd..5d2dd453687c 100644
--- a/drivers/net/txgbe/txgbe_flow.c
+++ b/drivers/net/txgbe/txgbe_flow.c
@@ -1807,9 +1807,9 @@ txgbe_parse_fdir_filter_normal(struct rte_eth_dev *dev __rte_unused,
/* check src addr mask */
for (j = 0; j < 16; j++) {
- if (ipv6_mask->hdr.src_addr[j] == UINT8_MAX) {
+ if (ipv6_mask->hdr.src_addr.a[j] == UINT8_MAX) {
rule->mask.src_ipv6_mask |= 1 << j;
- } else if (ipv6_mask->hdr.src_addr[j] != 0) {
+ } else if (ipv6_mask->hdr.src_addr.a[j] != 0) {
memset(rule, 0, sizeof(struct txgbe_fdir_rule));
rte_flow_error_set(error, EINVAL,
RTE_FLOW_ERROR_TYPE_ITEM,
@@ -1820,9 +1820,9 @@ txgbe_parse_fdir_filter_normal(struct rte_eth_dev *dev __rte_unused,
/* check dst addr mask */
for (j = 0; j < 16; j++) {
- if (ipv6_mask->hdr.dst_addr[j] == UINT8_MAX) {
+ if (ipv6_mask->hdr.dst_addr.a[j] == UINT8_MAX) {
rule->mask.dst_ipv6_mask |= 1 << j;
- } else if (ipv6_mask->hdr.dst_addr[j] != 0) {
+ } else if (ipv6_mask->hdr.dst_addr.a[j] != 0) {
memset(rule, 0, sizeof(struct txgbe_fdir_rule));
rte_flow_error_set(error, EINVAL,
RTE_FLOW_ERROR_TYPE_ITEM,
@@ -1835,9 +1835,9 @@ txgbe_parse_fdir_filter_normal(struct rte_eth_dev *dev __rte_unused,
rule->b_spec = TRUE;
ipv6_spec = item->spec;
rte_memcpy(rule->input.src_ip,
- ipv6_spec->hdr.src_addr, 16);
+ &ipv6_spec->hdr.src_addr, 16);
rte_memcpy(rule->input.dst_ip,
- ipv6_spec->hdr.dst_addr, 16);
+ &ipv6_spec->hdr.dst_addr, 16);
}
/**
diff --git a/drivers/net/txgbe/txgbe_ipsec.c b/drivers/net/txgbe/txgbe_ipsec.c
index f9f8108fb894..4bad7fc76d42 100644
--- a/drivers/net/txgbe/txgbe_ipsec.c
+++ b/drivers/net/txgbe/txgbe_ipsec.c
@@ -659,9 +659,9 @@ txgbe_crypto_add_ingress_sa_from_flow(const void *sess,
ic_session->src_ip.type = IPv6;
ic_session->dst_ip.type = IPv6;
rte_memcpy(ic_session->src_ip.ipv6,
- ipv6->hdr.src_addr, 16);
+ &ipv6->hdr.src_addr, 16);
rte_memcpy(ic_session->dst_ip.ipv6,
- ipv6->hdr.dst_addr, 16);
+ &ipv6->hdr.dst_addr, 16);
} else {
const struct rte_flow_item_ipv4 *ipv4 = ip_spec;
ic_session->src_ip.type = IPv4;
diff --git a/examples/ip_fragmentation/main.c b/examples/ip_fragmentation/main.c
index 736eae6f05ee..4c0fa5054a2e 100644
--- a/examples/ip_fragmentation/main.c
+++ b/examples/ip_fragmentation/main.c
@@ -311,7 +311,7 @@ l3fwd_simple_forward(struct rte_mbuf *m, struct lcore_queue_conf *qconf,
ip_hdr = rte_pktmbuf_mtod(m, struct rte_ipv6_hdr *);
/* Find destination port */
- if (rte_lpm6_lookup(rxq->lpm6, ip_hdr->dst_addr,
+ if (rte_lpm6_lookup(rxq->lpm6, ip_hdr->dst_addr.a,
&next_hop) == 0 &&
(enabled_port_mask & 1 << next_hop) != 0) {
port_out = next_hop;
diff --git a/examples/ip_pipeline/pipeline.c b/examples/ip_pipeline/pipeline.c
index 301c52d061be..d08f8c24c80f 100644
--- a/examples/ip_pipeline/pipeline.c
+++ b/examples/ip_pipeline/pipeline.c
@@ -638,7 +638,7 @@ static const struct rte_acl_field_def table_acl_field_format_ipv6[] = {
.size = sizeof(uint32_t),
.field_index = 1,
.input_index = 1,
- .offset = offsetof(struct rte_ipv6_hdr, src_addr[0]),
+ .offset = offsetof(struct rte_ipv6_hdr, src_addr.a[0]),
},
[2] = {
@@ -646,7 +646,7 @@ static const struct rte_acl_field_def table_acl_field_format_ipv6[] = {
.size = sizeof(uint32_t),
.field_index = 2,
.input_index = 2,
- .offset = offsetof(struct rte_ipv6_hdr, src_addr[4]),
+ .offset = offsetof(struct rte_ipv6_hdr, src_addr.a[4]),
},
[3] = {
@@ -654,7 +654,7 @@ static const struct rte_acl_field_def table_acl_field_format_ipv6[] = {
.size = sizeof(uint32_t),
.field_index = 3,
.input_index = 3,
- .offset = offsetof(struct rte_ipv6_hdr, src_addr[8]),
+ .offset = offsetof(struct rte_ipv6_hdr, src_addr.a[8]),
},
[4] = {
@@ -662,7 +662,7 @@ static const struct rte_acl_field_def table_acl_field_format_ipv6[] = {
.size = sizeof(uint32_t),
.field_index = 4,
.input_index = 4,
- .offset = offsetof(struct rte_ipv6_hdr, src_addr[12]),
+ .offset = offsetof(struct rte_ipv6_hdr, src_addr.a[12]),
},
/* Destination IP address (IPv6) */
@@ -671,7 +671,7 @@ static const struct rte_acl_field_def table_acl_field_format_ipv6[] = {
.size = sizeof(uint32_t),
.field_index = 5,
.input_index = 5,
- .offset = offsetof(struct rte_ipv6_hdr, dst_addr[0]),
+ .offset = offsetof(struct rte_ipv6_hdr, dst_addr.a[0]),
},
[6] = {
@@ -679,7 +679,7 @@ static const struct rte_acl_field_def table_acl_field_format_ipv6[] = {
.size = sizeof(uint32_t),
.field_index = 6,
.input_index = 6,
- .offset = offsetof(struct rte_ipv6_hdr, dst_addr[4]),
+ .offset = offsetof(struct rte_ipv6_hdr, dst_addr.a[4]),
},
[7] = {
@@ -687,7 +687,7 @@ static const struct rte_acl_field_def table_acl_field_format_ipv6[] = {
.size = sizeof(uint32_t),
.field_index = 7,
.input_index = 7,
- .offset = offsetof(struct rte_ipv6_hdr, dst_addr[8]),
+ .offset = offsetof(struct rte_ipv6_hdr, dst_addr.a[8]),
},
[8] = {
@@ -695,7 +695,7 @@ static const struct rte_acl_field_def table_acl_field_format_ipv6[] = {
.size = sizeof(uint32_t),
.field_index = 8,
.input_index = 8,
- .offset = offsetof(struct rte_ipv6_hdr, dst_addr[12]),
+ .offset = offsetof(struct rte_ipv6_hdr, dst_addr.a[12]),
},
/* Source Port */
diff --git a/examples/ip_reassembly/main.c b/examples/ip_reassembly/main.c
index c7019078f7b4..4da692eb23e6 100644
--- a/examples/ip_reassembly/main.c
+++ b/examples/ip_reassembly/main.c
@@ -400,7 +400,7 @@ reassemble(struct rte_mbuf *m, uint16_t portid, uint32_t queue,
}
/* Find destination port */
- if (rte_lpm6_lookup(rxq->lpm6, ip_hdr->dst_addr,
+ if (rte_lpm6_lookup(rxq->lpm6, ip_hdr->dst_addr.a,
&next_hop) == 0 &&
(enabled_port_mask & 1 << next_hop) != 0) {
dst_port = next_hop;
diff --git a/examples/ipsec-secgw/flow.c b/examples/ipsec-secgw/flow.c
index 05a62c3020fa..8a0c79e1510a 100644
--- a/examples/ipsec-secgw/flow.c
+++ b/examples/ipsec-secgw/flow.c
@@ -85,7 +85,7 @@ ipv6_hdr_print(struct rte_ipv6_hdr *hdr)
{
uint8_t *addr;
- addr = hdr->src_addr;
+ addr = hdr->src_addr.a;
printf("src: %4hx:%4hx:%4hx:%4hx:%4hx:%4hx:%4hx:%4hx \t",
(uint16_t)((addr[0] << 8) | addr[1]),
(uint16_t)((addr[2] << 8) | addr[3]),
@@ -96,7 +96,7 @@ ipv6_hdr_print(struct rte_ipv6_hdr *hdr)
(uint16_t)((addr[12] << 8) | addr[13]),
(uint16_t)((addr[14] << 8) | addr[15]));
- addr = hdr->dst_addr;
+ addr = hdr->dst_addr.a;
printf("dst: %4hx:%4hx:%4hx:%4hx:%4hx:%4hx:%4hx:%4hx",
(uint16_t)((addr[0] << 8) | addr[1]),
(uint16_t)((addr[2] << 8) | addr[3]),
@@ -196,8 +196,8 @@ parse_flow_tokens(char **tokens, uint32_t n_tokens,
INCREMENT_TOKEN_INDEX(ti, n_tokens, status);
if (status->status < 0)
return;
- if (ipv6_addr_cpy(rule->ipv6.spec.hdr.src_addr,
- rule->ipv6.mask.hdr.src_addr,
+ if (ipv6_addr_cpy(rule->ipv6.spec.hdr.src_addr.a,
+ rule->ipv6.mask.hdr.src_addr.a,
tokens[ti], status))
return;
}
@@ -205,8 +205,8 @@ parse_flow_tokens(char **tokens, uint32_t n_tokens,
INCREMENT_TOKEN_INDEX(ti, n_tokens, status);
if (status->status < 0)
return;
- if (ipv6_addr_cpy(rule->ipv6.spec.hdr.dst_addr,
- rule->ipv6.mask.hdr.dst_addr,
+ if (ipv6_addr_cpy(rule->ipv6.spec.hdr.dst_addr.a,
+ rule->ipv6.mask.hdr.dst_addr.a,
tokens[ti], status))
return;
}
diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c
index b52b0ffc3d22..ebde28639c12 100644
--- a/examples/ipsec-secgw/ipsec.c
+++ b/examples/ipsec-secgw/ipsec.c
@@ -529,9 +529,9 @@ create_inline_session(struct socket_ctx *skt_ctx, struct ipsec_sa *sa,
sa->pattern[1].type = RTE_FLOW_ITEM_TYPE_IPV6;
sa->pattern[1].spec = &sa->ipv6_spec;
- memcpy(sa->ipv6_spec.hdr.dst_addr,
+ memcpy(&sa->ipv6_spec.hdr.dst_addr,
sa->dst.ip.ip6.ip6_b, 16);
- memcpy(sa->ipv6_spec.hdr.src_addr,
+ memcpy(&sa->ipv6_spec.hdr.src_addr,
sa->src.ip.ip6.ip6_b, 16);
} else if (IS_IP4(sa->flags)) {
sa->pattern[1].mask = &rte_flow_item_ipv4_mask;
@@ -735,9 +735,9 @@ create_ipsec_esp_flow(struct ipsec_sa *sa)
sa->pattern[1].mask = &rte_flow_item_ipv6_mask;
sa->pattern[1].type = RTE_FLOW_ITEM_TYPE_IPV6;
sa->pattern[1].spec = &sa->ipv6_spec;
- memcpy(sa->ipv6_spec.hdr.dst_addr,
+ memcpy(&sa->ipv6_spec.hdr.dst_addr,
sa->dst.ip.ip6.ip6_b, sizeof(sa->dst.ip.ip6.ip6_b));
- memcpy(sa->ipv6_spec.hdr.src_addr,
+ memcpy(&sa->ipv6_spec.hdr.src_addr,
sa->src.ip.ip6.ip6_b, sizeof(sa->src.ip.ip6.ip6_b));
sa->pattern[2].type = RTE_FLOW_ITEM_TYPE_ESP;
sa->pattern[2].spec = &sa->esp_spec;
diff --git a/examples/ipsec-secgw/sa.c b/examples/ipsec-secgw/sa.c
index c4bac17cd77c..1a0afd2ed2e8 100644
--- a/examples/ipsec-secgw/sa.c
+++ b/examples/ipsec-secgw/sa.c
@@ -1571,8 +1571,8 @@ ipsec_sa_init(struct ipsec_sa *lsa, struct rte_ipsec_sa *sa, uint32_t sa_size,
};
if (IS_IP6_TUNNEL(lsa->flags)) {
- memcpy(v6.src_addr, lsa->src.ip.ip6.ip6_b, sizeof(v6.src_addr));
- memcpy(v6.dst_addr, lsa->dst.ip.ip6.ip6_b, sizeof(v6.dst_addr));
+ memcpy(&v6.src_addr, lsa->src.ip.ip6.ip6_b, sizeof(v6.src_addr));
+ memcpy(&v6.dst_addr, lsa->dst.ip.ip6.ip6_b, sizeof(v6.dst_addr));
}
rc = fill_ipsec_sa_prm(&prm, lsa, &v4, &v6);
diff --git a/examples/ipsec-secgw/sad.h b/examples/ipsec-secgw/sad.h
index 3224b6252c8d..fdb1d2ef1790 100644
--- a/examples/ipsec-secgw/sad.h
+++ b/examples/ipsec-secgw/sad.h
@@ -5,6 +5,8 @@
#ifndef __SAD_H__
#define __SAD_H__
+#include <rte_ip.h>
+#include <rte_ip6.h>
#include <rte_ipsec_sad.h>
#define SA_CACHE_SZ 128
@@ -37,8 +39,8 @@ cmp_sa_key(struct ipsec_sa *sa, int is_v4, struct rte_ipv4_hdr *ipv4,
(sa->dst.ip.ip4 == ipv4->dst_addr)) ||
/* IPv6 check */
(!is_v4 && (sa_type == IP6_TUNNEL) &&
- (!memcmp(sa->src.ip.ip6.ip6, ipv6->src_addr, 16)) &&
- (!memcmp(sa->dst.ip.ip6.ip6, ipv6->dst_addr, 16))))
+ (!memcmp(sa->src.ip.ip6.ip6, &ipv6->src_addr, 16)) &&
+ (!memcmp(sa->dst.ip.ip6.ip6, &ipv6->dst_addr, 16))))
return 1;
return 0;
@@ -128,9 +130,9 @@ sad_lookup(struct ipsec_sad *sad, struct rte_mbuf *pkts[],
}
}
v6[nb_v6].spi = esp->spi;
- memcpy(v6[nb_v6].dip, ipv6->dst_addr,
+ memcpy(v6[nb_v6].dip, &ipv6->dst_addr,
sizeof(ipv6->dst_addr));
- memcpy(v6[nb_v6].sip, ipv6->src_addr,
+ memcpy(v6[nb_v6].sip, &ipv6->src_addr,
sizeof(ipv6->src_addr));
keys_v6[nb_v6] = (const union rte_ipsec_sad_key *)
&v6[nb_v6];
diff --git a/examples/l3fwd/l3fwd_fib.c b/examples/l3fwd/l3fwd_fib.c
index f38b19af3f57..4a9534e9dbef 100644
--- a/examples/l3fwd/l3fwd_fib.c
+++ b/examples/l3fwd/l3fwd_fib.c
@@ -65,7 +65,7 @@ fib_parse_packet(struct rte_mbuf *mbuf,
/* IPv6 */
else {
ipv6_hdr = (struct rte_ipv6_hdr *)(eth_hdr + 1);
- rte_mov16(ipv6, (const uint8_t *)ipv6_hdr->dst_addr);
+ rte_mov16(ipv6, ipv6_hdr->dst_addr.a);
*ip_type = 0;
(*ipv6_cnt)++;
}
diff --git a/examples/l3fwd/l3fwd_lpm.c b/examples/l3fwd/l3fwd_lpm.c
index e8fd95aae9ce..422fdb70054d 100644
--- a/examples/l3fwd/l3fwd_lpm.c
+++ b/examples/l3fwd/l3fwd_lpm.c
@@ -62,7 +62,7 @@ lpm_get_ipv6_dst_port(const struct rte_ipv6_hdr *ipv6_hdr,
uint16_t portid,
struct rte_lpm6 *ipv6_l3fwd_lookup_struct)
{
- const uint8_t *dst_ip = ipv6_hdr->dst_addr;
+ const uint8_t *dst_ip = ipv6_hdr->dst_addr.a;
uint32_t next_hop;
if (rte_lpm6_lookup(ipv6_l3fwd_lookup_struct, dst_ip, &next_hop) == 0)
@@ -122,7 +122,7 @@ lpm_get_dst_port_with_ipv4(const struct lcore_conf *qconf, struct rte_mbuf *pkt,
ipv6_hdr = (struct rte_ipv6_hdr *)(eth_hdr + 1);
return (uint16_t) ((rte_lpm6_lookup(qconf->ipv6_lookup_struct,
- ipv6_hdr->dst_addr, &next_hop) == 0)
+ ipv6_hdr->dst_addr.a, &next_hop) == 0)
? next_hop : portid);
}
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index 3a3ab781c494..b3ede7ccdaef 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -1006,12 +1006,14 @@ struct rte_flow_item_ipv6 {
#ifndef __cplusplus
static const struct rte_flow_item_ipv6 rte_flow_item_ipv6_mask = {
.hdr = {
- .src_addr =
+ .src_addr = { .a =
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
- .dst_addr =
+ },
+ .dst_addr = { .a =
"\xff\xff\xff\xff\xff\xff\xff\xff"
"\xff\xff\xff\xff\xff\xff\xff\xff",
+ },
},
};
#endif
diff --git a/lib/hash/rte_thash.h b/lib/hash/rte_thash.h
index af40afd2d475..9aaaacfd5fa4 100644
--- a/lib/hash/rte_thash.h
+++ b/lib/hash/rte_thash.h
@@ -140,24 +140,24 @@ rte_thash_load_v6_addrs(const struct rte_ipv6_hdr *orig,
union rte_thash_tuple *targ)
{
#ifdef RTE_ARCH_X86
- __m128i ipv6 = _mm_loadu_si128((const __m128i *)orig->src_addr);
+ __m128i ipv6 = _mm_loadu_si128((const __m128i *)&orig->src_addr);
*(__m128i *)targ->v6.src_addr =
_mm_shuffle_epi8(ipv6, rte_thash_ipv6_bswap_mask);
- ipv6 = _mm_loadu_si128((const __m128i *)orig->dst_addr);
+ ipv6 = _mm_loadu_si128((const __m128i *)&orig->dst_addr);
*(__m128i *)targ->v6.dst_addr =
_mm_shuffle_epi8(ipv6, rte_thash_ipv6_bswap_mask);
#elif defined(__ARM_NEON)
- uint8x16_t ipv6 = vld1q_u8((uint8_t const *)orig->src_addr);
+ uint8x16_t ipv6 = vld1q_u8((uint8_t const *)&orig->src_addr);
vst1q_u8((uint8_t *)targ->v6.src_addr, vrev32q_u8(ipv6));
- ipv6 = vld1q_u8((uint8_t const *)orig->dst_addr);
+ ipv6 = vld1q_u8((uint8_t const *)&orig->dst_addr);
vst1q_u8((uint8_t *)targ->v6.dst_addr, vrev32q_u8(ipv6));
#else
int i;
for (i = 0; i < 4; i++) {
*((uint32_t *)targ->v6.src_addr + i) =
- rte_be_to_cpu_32(*((const uint32_t *)orig->src_addr + i));
+ rte_be_to_cpu_32(*((const uint32_t *)orig->src_addr.a + i));
*((uint32_t *)targ->v6.dst_addr + i) =
- rte_be_to_cpu_32(*((const uint32_t *)orig->dst_addr + i));
+ rte_be_to_cpu_32(*((const uint32_t *)orig->dst_addr.a + i));
}
#endif
}
diff --git a/lib/ip_frag/rte_ipv6_reassembly.c b/lib/ip_frag/rte_ipv6_reassembly.c
index 88863a98d1fe..9471ce5333d7 100644
--- a/lib/ip_frag/rte_ipv6_reassembly.c
+++ b/lib/ip_frag/rte_ipv6_reassembly.c
@@ -143,8 +143,8 @@ rte_ipv6_frag_reassemble_packet(struct rte_ip_frag_tbl *tbl,
int32_t ip_len;
int32_t trim;
- rte_memcpy(&key.src_dst[0], ip_hdr->src_addr, 16);
- rte_memcpy(&key.src_dst[2], ip_hdr->dst_addr, 16);
+ rte_memcpy(&key.src_dst[0], &ip_hdr->src_addr, 16);
+ rte_memcpy(&key.src_dst[2], &ip_hdr->dst_addr, 16);
key.id = frag_hdr->id;
key.key_len = IPV6_KEYLEN;
diff --git a/lib/net/rte_ip6.h b/lib/net/rte_ip6.h
index 52c41088681e..5354a2bd42b4 100644
--- a/lib/net/rte_ip6.h
+++ b/lib/net/rte_ip6.h
@@ -53,8 +53,8 @@ struct rte_ipv6_hdr {
rte_be16_t payload_len; /**< IP payload size, including ext. headers */
uint8_t proto; /**< Protocol, next header. */
uint8_t hop_limits; /**< Hop limits. */
- uint8_t src_addr[16]; /**< IP address of source host. */
- uint8_t dst_addr[16]; /**< IP address of destination host(s). */
+ struct rte_ipv6_addr src_addr; /**< IP address of source host. */
+ struct rte_ipv6_addr dst_addr; /**< IP address of destination host(s). */
} __rte_packed;
/* IPv6 routing extension type definition. */
@@ -123,7 +123,7 @@ rte_ipv6_phdr_cksum(const struct rte_ipv6_hdr *ipv6_hdr, uint64_t ol_flags)
psd_hdr.len = ipv6_hdr->payload_len;
}
- sum = __rte_raw_cksum(ipv6_hdr->src_addr,
+ sum = __rte_raw_cksum(&ipv6_hdr->src_addr,
sizeof(ipv6_hdr->src_addr) + sizeof(ipv6_hdr->dst_addr),
0);
sum = __rte_raw_cksum(&psd_hdr, sizeof(psd_hdr), sum);
diff --git a/lib/node/ip6_lookup.c b/lib/node/ip6_lookup.c
index 309964f60fd6..6bbcf14e2aa8 100644
--- a/lib/node/ip6_lookup.c
+++ b/lib/node/ip6_lookup.c
@@ -112,28 +112,28 @@ ip6_lookup_node_process_scalar(struct rte_graph *graph, struct rte_node *node,
sizeof(struct rte_ether_hdr));
/* Extract hop_limits as ipv6 hdr is in cache */
node_mbuf_priv1(mbuf0, dyn)->ttl = ipv6_hdr->hop_limits;
- rte_memcpy(ip_batch[0], ipv6_hdr->dst_addr, 16);
+ rte_memcpy(ip_batch[0], &ipv6_hdr->dst_addr, 16);
/* Extract DIP of mbuf1 */
ipv6_hdr = rte_pktmbuf_mtod_offset(mbuf1, struct rte_ipv6_hdr *,
sizeof(struct rte_ether_hdr));
/* Extract hop_limits as ipv6 hdr is in cache */
node_mbuf_priv1(mbuf1, dyn)->ttl = ipv6_hdr->hop_limits;
- rte_memcpy(ip_batch[1], ipv6_hdr->dst_addr, 16);
+ rte_memcpy(ip_batch[1], &ipv6_hdr->dst_addr, 16);
/* Extract DIP of mbuf2 */
ipv6_hdr = rte_pktmbuf_mtod_offset(mbuf2, struct rte_ipv6_hdr *,
sizeof(struct rte_ether_hdr));
/* Extract hop_limits as ipv6 hdr is in cache */
node_mbuf_priv1(mbuf2, dyn)->ttl = ipv6_hdr->hop_limits;
- rte_memcpy(ip_batch[2], ipv6_hdr->dst_addr, 16);
+ rte_memcpy(ip_batch[2], &ipv6_hdr->dst_addr, 16);
/* Extract DIP of mbuf3 */
ipv6_hdr = rte_pktmbuf_mtod_offset(mbuf3, struct rte_ipv6_hdr *,
sizeof(struct rte_ether_hdr));
/* Extract hop_limits as ipv6 hdr is in cache */
node_mbuf_priv1(mbuf3, dyn)->ttl = ipv6_hdr->hop_limits;
- rte_memcpy(ip_batch[3], ipv6_hdr->dst_addr, 16);
+ rte_memcpy(ip_batch[3], &ipv6_hdr->dst_addr, 16);
rte_lpm6_lookup_bulk_func(lpm6, ip_batch, next_hop, 4);
@@ -223,7 +223,7 @@ ip6_lookup_node_process_scalar(struct rte_graph *graph, struct rte_node *node,
/* Extract TTL as IPv6 hdr is in cache */
node_mbuf_priv1(mbuf0, dyn)->ttl = ipv6_hdr->hop_limits;
- rc = rte_lpm6_lookup(lpm6, ipv6_hdr->dst_addr, &next_hop);
+ rc = rte_lpm6_lookup(lpm6, ipv6_hdr->dst_addr.a, &next_hop);
next_hop = (rc == 0) ? next_hop : drop_nh;
node_mbuf_priv1(mbuf0, dyn)->nh = (uint16_t)next_hop;
diff --git a/lib/pipeline/rte_swx_ipsec.c b/lib/pipeline/rte_swx_ipsec.c
index 56a401d0fa63..5920178f38e5 100644
--- a/lib/pipeline/rte_swx_ipsec.c
+++ b/lib/pipeline/rte_swx_ipsec.c
@@ -1387,13 +1387,11 @@ tunnel_ipv6_header_set(struct rte_ipv6_hdr *h, struct rte_swx_ipsec_sa_params *p
.payload_len = 0, /* Cannot be pre-computed. */
.proto = IPPROTO_ESP,
.hop_limits = 64,
- .src_addr = {0},
- .dst_addr = {0},
};
memcpy(h, &ipv6_hdr, sizeof(ipv6_hdr));
- memcpy(h->src_addr, p->encap.tunnel.ipv6.src_addr.s6_addr, 16);
- memcpy(h->dst_addr, p->encap.tunnel.ipv6.dst_addr.s6_addr, 16);
+ memcpy(&h->src_addr, p->encap.tunnel.ipv6.src_addr.s6_addr, 16);
+ memcpy(&h->dst_addr, p->encap.tunnel.ipv6.dst_addr.s6_addr, 16);
}
/* IPsec library SA parameters. */
diff --git a/lib/pipeline/rte_table_action.c b/lib/pipeline/rte_table_action.c
index ebbdd3f4cebe..a04d5121e1f0 100644
--- a/lib/pipeline/rte_table_action.c
+++ b/lib/pipeline/rte_table_action.c
@@ -872,10 +872,10 @@ encap_vxlan_apply(void *data,
d->ipv6.payload_len = 0; /* not pre-computed */
d->ipv6.proto = IP_PROTO_UDP;
d->ipv6.hop_limits = p->vxlan.ipv6.hop_limit;
- memcpy(d->ipv6.src_addr,
+ memcpy(&d->ipv6.src_addr,
p->vxlan.ipv6.sa,
sizeof(p->vxlan.ipv6.sa));
- memcpy(d->ipv6.dst_addr,
+ memcpy(&d->ipv6.dst_addr,
p->vxlan.ipv6.da,
sizeof(p->vxlan.ipv6.da));
@@ -907,10 +907,10 @@ encap_vxlan_apply(void *data,
d->ipv6.payload_len = 0; /* not pre-computed */
d->ipv6.proto = IP_PROTO_UDP;
d->ipv6.hop_limits = p->vxlan.ipv6.hop_limit;
- memcpy(d->ipv6.src_addr,
+ memcpy(&d->ipv6.src_addr,
p->vxlan.ipv6.sa,
sizeof(p->vxlan.ipv6.sa));
- memcpy(d->ipv6.dst_addr,
+ memcpy(&d->ipv6.dst_addr,
p->vxlan.ipv6.da,
sizeof(p->vxlan.ipv6.da));
@@ -1437,12 +1437,12 @@ pkt_ipv6_work_nat(struct rte_ipv6_hdr *ip,
uint16_t tcp_cksum;
tcp_cksum = nat_ipv6_tcp_udp_checksum_update(tcp->cksum,
- (uint16_t *)ip->src_addr,
+ (uint16_t *)&ip->src_addr,
(uint16_t *)data->addr,
tcp->src_port,
data->port);
- rte_memcpy(ip->src_addr, data->addr, 16);
+ rte_memcpy(&ip->src_addr, data->addr, 16);
tcp->src_port = data->port;
tcp->cksum = tcp_cksum;
} else {
@@ -1450,12 +1450,12 @@ pkt_ipv6_work_nat(struct rte_ipv6_hdr *ip,
uint16_t udp_cksum;
udp_cksum = nat_ipv6_tcp_udp_checksum_update(udp->dgram_cksum,
- (uint16_t *)ip->src_addr,
+ (uint16_t *)&ip->src_addr,
(uint16_t *)data->addr,
udp->src_port,
data->port);
- rte_memcpy(ip->src_addr, data->addr, 16);
+ rte_memcpy(&ip->src_addr, data->addr, 16);
udp->src_port = data->port;
udp->dgram_cksum = udp_cksum;
}
@@ -1465,12 +1465,12 @@ pkt_ipv6_work_nat(struct rte_ipv6_hdr *ip,
uint16_t tcp_cksum;
tcp_cksum = nat_ipv6_tcp_udp_checksum_update(tcp->cksum,
- (uint16_t *)ip->dst_addr,
+ (uint16_t *)&ip->dst_addr,
(uint16_t *)data->addr,
tcp->dst_port,
data->port);
- rte_memcpy(ip->dst_addr, data->addr, 16);
+ rte_memcpy(&ip->dst_addr, data->addr, 16);
tcp->dst_port = data->port;
tcp->cksum = tcp_cksum;
} else {
@@ -1478,12 +1478,12 @@ pkt_ipv6_work_nat(struct rte_ipv6_hdr *ip,
uint16_t udp_cksum;
udp_cksum = nat_ipv6_tcp_udp_checksum_update(udp->dgram_cksum,
- (uint16_t *)ip->dst_addr,
+ (uint16_t *)&ip->dst_addr.a,
(uint16_t *)data->addr,
udp->dst_port,
data->port);
- rte_memcpy(ip->dst_addr, data->addr, 16);
+ rte_memcpy(&ip->dst_addr, data->addr, 16);
udp->dst_port = data->port;
udp->dgram_cksum = udp_cksum;
}
--
2.46.1
^ permalink raw reply [relevance 1%]
* [PATCH dpdk v2 02/16] net: split ipv6 symbols in separate header
@ 2024-10-01 8:17 2% ` Robin Jarry
` (9 subsequent siblings)
10 siblings, 0 replies; 200+ results
From: Robin Jarry @ 2024-10-01 8:17 UTC (permalink / raw)
To: dev, Akhil Goyal, Fan Zhang, Nithin Dabilpuram, Kiran Kumar K,
Sunil Kumar Kori, Satha Rao, Harman Kalra, Ankur Dwivedi,
Anoob Joseph, Tejasree Kondoj, Gagandeep Singh, Hemant Agrawal,
Cristian Dumitrescu, Ori Kam, Thomas Monjalon, Ferruh Yigit,
Andrew Rybchenko, Yipeng Wang, Sameh Gobriel, Bruce Richardson,
Vladimir Medvedkin, Konstantin Ananyev, Maxime Coquelin,
Chenbo Xia
Move all ipv6 related symbols to a dedicated header. Update all code
accordingly.
Signed-off-by: Robin Jarry <rjarry@redhat.com>
---
app/test/packet_burst_generator.c | 1 +
app/test/test_cryptodev_security_ipsec.c | 1 +
doc/guides/rel_notes/release_24_11.rst | 1 +
drivers/common/cnxk/cnxk_security.c | 1 +
drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 1 +
drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 1 +
examples/ip_pipeline/pipeline.c | 1 +
lib/ethdev/rte_flow.h | 1 +
lib/hash/rte_thash.h | 1 +
lib/ip_frag/rte_ip_frag.h | 1 +
lib/ipsec/iph.h | 1 +
lib/net/meson.build | 1 +
lib/net/rte_ip.h | 319 ------------------
lib/net/{rte_ip.h => rte_ip6.h} | 347 +-------------------
lib/net/rte_net.c | 1 +
lib/net/rte_net.h | 1 +
lib/pipeline/rte_swx_ipsec.c | 1 +
lib/pipeline/rte_table_action.c | 1 +
lib/vhost/virtio_net.c | 1 +
19 files changed, 21 insertions(+), 662 deletions(-)
copy lib/net/{rte_ip.h => rte_ip6.h} (50%)
diff --git a/app/test/packet_burst_generator.c b/app/test/packet_burst_generator.c
index 867a88da0055..2cd34abc1a65 100644
--- a/app/test/packet_burst_generator.c
+++ b/app/test/packet_burst_generator.c
@@ -5,6 +5,7 @@
#include <rte_byteorder.h>
#include <rte_mbuf.h>
#include <rte_ip.h>
+#include <rte_ip6.h>
#include <rte_os_shim.h>
#include "packet_burst_generator.h"
diff --git a/app/test/test_cryptodev_security_ipsec.c b/app/test/test_cryptodev_security_ipsec.c
index 1aba1ad9934b..9ac4a6e599b0 100644
--- a/app/test/test_cryptodev_security_ipsec.c
+++ b/app/test/test_cryptodev_security_ipsec.c
@@ -6,6 +6,7 @@
#include <rte_cryptodev.h>
#include <rte_esp.h>
#include <rte_ip.h>
+#include <rte_ip6.h>
#include <rte_security.h>
#include <rte_tcp.h>
#include <rte_udp.h>
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 0ff70d90578e..95b9809f4145 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -84,6 +84,7 @@ API Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* net: IPv6 related symbols were moved from ``<rte_ip.h>`` to the new ``<rte_ip6.h>`` header.
ABI Changes
-----------
diff --git a/drivers/common/cnxk/cnxk_security.c b/drivers/common/cnxk/cnxk_security.c
index 15b0bedf43e2..e275d6cad3ea 100644
--- a/drivers/common/cnxk/cnxk_security.c
+++ b/drivers/common/cnxk/cnxk_security.c
@@ -2,6 +2,7 @@
* Copyright(C) 2021 Marvell.
*/
+#include <rte_ip6.h>
#include <rte_udp.h>
#include "cnxk_security.h"
diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
index f443cb9563ec..a3c737ef40f7 100644
--- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
+++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c
@@ -6,6 +6,7 @@
#include <cryptodev_pmd.h>
#include <rte_event_crypto_adapter.h>
#include <rte_ip.h>
+#include <rte_ip6.h>
#include <rte_vect.h>
#include "roc_cpt.h"
diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c
index c1f7181d5587..5bf690e3d82a 100644
--- a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c
+++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c
@@ -10,6 +10,7 @@
#include <unistd.h>
#include <rte_ip.h>
+#include <rte_ip6.h>
#include <rte_udp.h>
#include <rte_mbuf.h>
#include <rte_cryptodev.h>
diff --git a/examples/ip_pipeline/pipeline.c b/examples/ip_pipeline/pipeline.c
index 63352257c6e9..301c52d061be 100644
--- a/examples/ip_pipeline/pipeline.c
+++ b/examples/ip_pipeline/pipeline.c
@@ -7,6 +7,7 @@
#include <rte_common.h>
#include <rte_ip.h>
+#include <rte_ip6.h>
#include <rte_tcp.h>
#include <rte_string_fns.h>
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index f864578f806b..3a3ab781c494 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -23,6 +23,7 @@
#include <rte_arp.h>
#include <rte_icmp.h>
#include <rte_ip.h>
+#include <rte_ip6.h>
#include <rte_sctp.h>
#include <rte_tcp.h>
#include <rte_udp.h>
diff --git a/lib/hash/rte_thash.h b/lib/hash/rte_thash.h
index 30b657e67a7d..af40afd2d475 100644
--- a/lib/hash/rte_thash.h
+++ b/lib/hash/rte_thash.h
@@ -23,6 +23,7 @@ extern "C" {
#include <rte_byteorder.h>
#include <rte_ip.h>
+#include <rte_ip6.h>
#include <rte_common.h>
#include <rte_thash_gfni.h>
diff --git a/lib/ip_frag/rte_ip_frag.h b/lib/ip_frag/rte_ip_frag.h
index 2ad318096b7b..cb06d5f5977a 100644
--- a/lib/ip_frag/rte_ip_frag.h
+++ b/lib/ip_frag/rte_ip_frag.h
@@ -23,6 +23,7 @@ extern "C" {
#include <rte_malloc.h>
#include <rte_memory.h>
#include <rte_ip.h>
+#include <rte_ip6.h>
#include <rte_byteorder.h>
struct rte_mbuf;
diff --git a/lib/ipsec/iph.h b/lib/ipsec/iph.h
index 861f16905ad0..815a3c90d76b 100644
--- a/lib/ipsec/iph.h
+++ b/lib/ipsec/iph.h
@@ -6,6 +6,7 @@
#define _IPH_H_
#include <rte_ip.h>
+#include <rte_ip6.h>
/**
* @file iph.h
diff --git a/lib/net/meson.build b/lib/net/meson.build
index 2e65bd19b7d4..35ac334a18cf 100644
--- a/lib/net/meson.build
+++ b/lib/net/meson.build
@@ -3,6 +3,7 @@
headers = files(
'rte_ip.h',
+ 'rte_ip6.h',
'rte_cksum.h',
'rte_tcp.h',
'rte_udp.h',
diff --git a/lib/net/rte_ip.h b/lib/net/rte_ip.h
index 0ae7c0565047..e3c8d0163f64 100644
--- a/lib/net/rte_ip.h
+++ b/lib/net/rte_ip.h
@@ -374,325 +374,6 @@ rte_ipv4_udptcp_cksum_mbuf_verify(const struct rte_mbuf *m,
return 0;
}
-/**
- * IPv6 Header
- */
-struct rte_ipv6_hdr {
- rte_be32_t vtc_flow; /**< IP version, traffic class & flow label. */
- rte_be16_t payload_len; /**< IP payload size, including ext. headers */
- uint8_t proto; /**< Protocol, next header. */
- uint8_t hop_limits; /**< Hop limits. */
- uint8_t src_addr[16]; /**< IP address of source host. */
- uint8_t dst_addr[16]; /**< IP address of destination host(s). */
-} __rte_packed;
-
-/* IPv6 routing extension type definition. */
-#define RTE_IPV6_SRCRT_TYPE_4 4
-
-/**
- * IPv6 Routing Extension Header
- */
-struct rte_ipv6_routing_ext {
- uint8_t next_hdr; /**< Protocol, next header. */
- uint8_t hdr_len; /**< Header length. */
- uint8_t type; /**< Extension header type. */
- uint8_t segments_left; /**< Valid segments number. */
- __extension__
- union {
- rte_be32_t flags; /**< Packet control data per type. */
- struct {
- uint8_t last_entry; /**< The last_entry field of SRH */
- uint8_t flag; /**< Packet flag. */
- rte_be16_t tag; /**< Packet tag. */
- };
- };
- /* Next are 128-bit IPv6 address fields to describe segments. */
-} __rte_packed;
-
-/* IPv6 vtc_flow: IPv / TC / flow_label */
-#define RTE_IPV6_HDR_FL_SHIFT 0
-#define RTE_IPV6_HDR_TC_SHIFT 20
-#define RTE_IPV6_HDR_FL_MASK ((1u << RTE_IPV6_HDR_TC_SHIFT) - 1)
-#define RTE_IPV6_HDR_TC_MASK (0xff << RTE_IPV6_HDR_TC_SHIFT)
-#define RTE_IPV6_HDR_DSCP_MASK (0xfc << RTE_IPV6_HDR_TC_SHIFT)
-#define RTE_IPV6_HDR_ECN_MASK (0x03 << RTE_IPV6_HDR_TC_SHIFT)
-#define RTE_IPV6_HDR_ECN_CE RTE_IPV6_HDR_ECN_MASK
-
-#define RTE_IPV6_MIN_MTU 1280 /**< Minimum MTU for IPv6, see RFC 8200. */
-
-/**
- * Process the pseudo-header checksum of an IPv6 header.
- *
- * Depending on the ol_flags, the pseudo-header checksum expected by the
- * drivers is not the same. For instance, when TSO is enabled, the IPv6
- * payload length must not be included in the packet.
- *
- * When ol_flags is 0, it computes the standard pseudo-header checksum.
- *
- * @param ipv6_hdr
- * The pointer to the contiguous IPv6 header.
- * @param ol_flags
- * The ol_flags of the associated mbuf.
- * @return
- * The non-complemented checksum to set in the L4 header.
- */
-static inline uint16_t
-rte_ipv6_phdr_cksum(const struct rte_ipv6_hdr *ipv6_hdr, uint64_t ol_flags)
-{
- uint32_t sum;
- struct {
- rte_be32_t len; /* L4 length. */
- rte_be32_t proto; /* L4 protocol - top 3 bytes must be zero */
- } psd_hdr;
-
- psd_hdr.proto = (uint32_t)(ipv6_hdr->proto << 24);
- if (ol_flags & (RTE_MBUF_F_TX_TCP_SEG | RTE_MBUF_F_TX_UDP_SEG)) {
- psd_hdr.len = 0;
- } else {
- psd_hdr.len = ipv6_hdr->payload_len;
- }
-
- sum = __rte_raw_cksum(ipv6_hdr->src_addr,
- sizeof(ipv6_hdr->src_addr) + sizeof(ipv6_hdr->dst_addr),
- 0);
- sum = __rte_raw_cksum(&psd_hdr, sizeof(psd_hdr), sum);
- return __rte_raw_cksum_reduce(sum);
-}
-
-/**
- * @internal Calculate the non-complemented IPv6 L4 checksum
- */
-static inline uint16_t
-__rte_ipv6_udptcp_cksum(const struct rte_ipv6_hdr *ipv6_hdr, const void *l4_hdr)
-{
- uint32_t cksum;
- uint32_t l4_len;
-
- l4_len = rte_be_to_cpu_16(ipv6_hdr->payload_len);
-
- cksum = rte_raw_cksum(l4_hdr, l4_len);
- cksum += rte_ipv6_phdr_cksum(ipv6_hdr, 0);
-
- cksum = ((cksum & 0xffff0000) >> 16) + (cksum & 0xffff);
-
- return (uint16_t)cksum;
-}
-
-/**
- * Process the IPv6 UDP or TCP checksum.
- *
- * The IPv6 header must not be followed by extension headers. The layer 4
- * checksum must be set to 0 in the L4 header by the caller.
- *
- * @param ipv6_hdr
- * The pointer to the contiguous IPv6 header.
- * @param l4_hdr
- * The pointer to the beginning of the L4 header.
- * @return
- * The complemented checksum to set in the L4 header.
- */
-static inline uint16_t
-rte_ipv6_udptcp_cksum(const struct rte_ipv6_hdr *ipv6_hdr, const void *l4_hdr)
-{
- uint16_t cksum = __rte_ipv6_udptcp_cksum(ipv6_hdr, l4_hdr);
-
- cksum = ~cksum;
-
- /*
- * Per RFC 768: If the computed checksum is zero for UDP,
- * it is transmitted as all ones
- * (the equivalent in one's complement arithmetic).
- */
- if (cksum == 0 && ipv6_hdr->proto == IPPROTO_UDP)
- cksum = 0xffff;
-
- return cksum;
-}
-
-/**
- * @internal Calculate the non-complemented IPv6 L4 checksum of a packet
- */
-static inline uint16_t
-__rte_ipv6_udptcp_cksum_mbuf(const struct rte_mbuf *m,
- const struct rte_ipv6_hdr *ipv6_hdr,
- uint16_t l4_off)
-{
- uint16_t raw_cksum;
- uint32_t cksum;
-
- if (unlikely(l4_off > m->pkt_len))
- return 0; /* invalid params, return a dummy value */
-
- if (rte_raw_cksum_mbuf(m, l4_off, rte_be_to_cpu_16(ipv6_hdr->payload_len), &raw_cksum))
- return 0;
-
- cksum = raw_cksum + rte_ipv6_phdr_cksum(ipv6_hdr, 0);
-
- cksum = ((cksum & 0xffff0000) >> 16) + (cksum & 0xffff);
-
- return (uint16_t)cksum;
-}
-
-/**
- * Process the IPv6 UDP or TCP checksum of a packet.
- *
- * The IPv6 header must not be followed by extension headers. The layer 4
- * checksum must be set to 0 in the L4 header by the caller.
- *
- * @param m
- * The pointer to the mbuf.
- * @param ipv6_hdr
- * The pointer to the contiguous IPv6 header.
- * @param l4_off
- * The offset in bytes to start L4 checksum.
- * @return
- * The complemented checksum to set in the L4 header.
- */
-static inline uint16_t
-rte_ipv6_udptcp_cksum_mbuf(const struct rte_mbuf *m,
- const struct rte_ipv6_hdr *ipv6_hdr, uint16_t l4_off)
-{
- uint16_t cksum = __rte_ipv6_udptcp_cksum_mbuf(m, ipv6_hdr, l4_off);
-
- cksum = ~cksum;
-
- /*
- * Per RFC 768: If the computed checksum is zero for UDP,
- * it is transmitted as all ones
- * (the equivalent in one's complement arithmetic).
- */
- if (cksum == 0 && ipv6_hdr->proto == IPPROTO_UDP)
- cksum = 0xffff;
-
- return cksum;
-}
-
-/**
- * Validate the IPv6 UDP or TCP checksum.
- *
- * In case of UDP, the caller must first check if udp_hdr->dgram_cksum is 0:
- * this is either invalid or means no checksum in some situations. See 8.1
- * (Upper-Layer Checksums) in RFC 8200.
- *
- * @param ipv6_hdr
- * The pointer to the contiguous IPv6 header.
- * @param l4_hdr
- * The pointer to the beginning of the L4 header.
- * @return
- * Return 0 if the checksum is correct, else -1.
- */
-static inline int
-rte_ipv6_udptcp_cksum_verify(const struct rte_ipv6_hdr *ipv6_hdr,
- const void *l4_hdr)
-{
- uint16_t cksum = __rte_ipv6_udptcp_cksum(ipv6_hdr, l4_hdr);
-
- if (cksum != 0xffff)
- return -1;
-
- return 0;
-}
-
-/**
- * Validate the IPv6 UDP or TCP checksum of a packet.
- *
- * In case of UDP, the caller must first check if udp_hdr->dgram_cksum is 0:
- * this is either invalid or means no checksum in some situations. See 8.1
- * (Upper-Layer Checksums) in RFC 8200.
- *
- * @param m
- * The pointer to the mbuf.
- * @param ipv6_hdr
- * The pointer to the contiguous IPv6 header.
- * @param l4_off
- * The offset in bytes to start L4 checksum.
- * @return
- * Return 0 if the checksum is correct, else -1.
- */
-static inline int
-rte_ipv6_udptcp_cksum_mbuf_verify(const struct rte_mbuf *m,
- const struct rte_ipv6_hdr *ipv6_hdr,
- uint16_t l4_off)
-{
- uint16_t cksum = __rte_ipv6_udptcp_cksum_mbuf(m, ipv6_hdr, l4_off);
-
- if (cksum != 0xffff)
- return -1;
-
- return 0;
-}
-
-/** IPv6 fragment extension header. */
-#define RTE_IPV6_EHDR_MF_SHIFT 0
-#define RTE_IPV6_EHDR_MF_MASK 1
-#define RTE_IPV6_EHDR_FO_SHIFT 3
-#define RTE_IPV6_EHDR_FO_MASK (~((1 << RTE_IPV6_EHDR_FO_SHIFT) - 1))
-#define RTE_IPV6_EHDR_FO_ALIGN (1 << RTE_IPV6_EHDR_FO_SHIFT)
-
-#define RTE_IPV6_FRAG_USED_MASK (RTE_IPV6_EHDR_MF_MASK | RTE_IPV6_EHDR_FO_MASK)
-
-#define RTE_IPV6_GET_MF(x) ((x) & RTE_IPV6_EHDR_MF_MASK)
-#define RTE_IPV6_GET_FO(x) ((x) >> RTE_IPV6_EHDR_FO_SHIFT)
-
-#define RTE_IPV6_SET_FRAG_DATA(fo, mf) \
- (((fo) & RTE_IPV6_EHDR_FO_MASK) | ((mf) & RTE_IPV6_EHDR_MF_MASK))
-
-struct rte_ipv6_fragment_ext {
- uint8_t next_header; /**< Next header type */
- uint8_t reserved; /**< Reserved */
- rte_be16_t frag_data; /**< All fragmentation data */
- rte_be32_t id; /**< Packet ID */
-} __rte_packed;
-
-/* IPv6 fragment extension header size */
-#define RTE_IPV6_FRAG_HDR_SIZE sizeof(struct rte_ipv6_fragment_ext)
-
-/**
- * Parse next IPv6 header extension
- *
- * This function checks if proto number is an IPv6 extensions and parses its
- * data if so, providing information on next header and extension length.
- *
- * @param p
- * Pointer to an extension raw data.
- * @param proto
- * Protocol number extracted from the "next header" field from
- * the IPv6 header or the previous extension.
- * @param ext_len
- * Extension data length.
- * @return
- * next protocol number if proto is an IPv6 extension, -EINVAL otherwise
- */
-static inline int
-rte_ipv6_get_next_ext(const uint8_t *p, int proto, size_t *ext_len)
-{
- int next_proto;
-
- switch (proto) {
- case IPPROTO_AH:
- next_proto = *p++;
- *ext_len = (*p + 2) * sizeof(uint32_t);
- break;
-
- case IPPROTO_HOPOPTS:
- case IPPROTO_ROUTING:
- case IPPROTO_DSTOPTS:
- next_proto = *p++;
- *ext_len = (*p + 1) * sizeof(uint64_t);
- break;
-
- case IPPROTO_FRAGMENT:
- next_proto = *p;
- *ext_len = RTE_IPV6_FRAG_HDR_SIZE;
- break;
-
- default:
- return -EINVAL;
- }
-
- return next_proto;
-}
-
#ifdef __cplusplus
}
#endif
diff --git a/lib/net/rte_ip.h b/lib/net/rte_ip6.h
similarity index 50%
copy from lib/net/rte_ip.h
copy to lib/net/rte_ip6.h
index 0ae7c0565047..5ad1dd25db08 100644
--- a/lib/net/rte_ip.h
+++ b/lib/net/rte_ip6.h
@@ -6,13 +6,13 @@
* All rights reserved.
*/
-#ifndef _RTE_IP_H_
-#define _RTE_IP_H_
+#ifndef _RTE_IP6_H_
+#define _RTE_IP6_H_
/**
* @file
*
- * IP-related defines
+ * IPv6-related defines
*/
#include <stdint.h>
@@ -24,7 +24,6 @@
#include <sys/types.h>
#include <netinet/in.h>
#include <arpa/inet.h>
-#include <netinet/ip.h>
#include <netinet/ip6.h>
#endif
@@ -36,344 +35,6 @@
extern "C" {
#endif
-/**
- * IPv4 Header
- */
-struct rte_ipv4_hdr {
- __extension__
- union {
- uint8_t version_ihl; /**< version and header length */
- struct {
-#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
- uint8_t ihl:4; /**< header length */
- uint8_t version:4; /**< version */
-#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN
- uint8_t version:4; /**< version */
- uint8_t ihl:4; /**< header length */
-#endif
- };
- };
- uint8_t type_of_service; /**< type of service */
- rte_be16_t total_length; /**< length of packet */
- rte_be16_t packet_id; /**< packet ID */
- rte_be16_t fragment_offset; /**< fragmentation offset */
- uint8_t time_to_live; /**< time to live */
- uint8_t next_proto_id; /**< protocol ID */
- rte_be16_t hdr_checksum; /**< header checksum */
- rte_be32_t src_addr; /**< source address */
- rte_be32_t dst_addr; /**< destination address */
-} __rte_packed;
-
-/** Create IPv4 address */
-#define RTE_IPV4(a, b, c, d) ((uint32_t)(((a) & 0xff) << 24) | \
- (((b) & 0xff) << 16) | \
- (((c) & 0xff) << 8) | \
- ((d) & 0xff))
-
-/** Maximal IPv4 packet length (including a header) */
-#define RTE_IPV4_MAX_PKT_LEN 65535
-
-/** Internet header length mask for version_ihl field */
-#define RTE_IPV4_HDR_IHL_MASK (0x0f)
-/**
- * Internet header length field multiplier (IHL field specifies overall header
- * length in number of 4-byte words)
- */
-#define RTE_IPV4_IHL_MULTIPLIER (4)
-
-/* Type of Service fields */
-#define RTE_IPV4_HDR_DSCP_MASK (0xfc)
-#define RTE_IPV4_HDR_ECN_MASK (0x03)
-#define RTE_IPV4_HDR_ECN_CE RTE_IPV4_HDR_ECN_MASK
-
-/* Fragment Offset * Flags. */
-#define RTE_IPV4_HDR_DF_SHIFT 14
-#define RTE_IPV4_HDR_MF_SHIFT 13
-#define RTE_IPV4_HDR_FO_SHIFT 3
-
-#define RTE_IPV4_HDR_DF_FLAG (1 << RTE_IPV4_HDR_DF_SHIFT)
-#define RTE_IPV4_HDR_MF_FLAG (1 << RTE_IPV4_HDR_MF_SHIFT)
-
-#define RTE_IPV4_HDR_OFFSET_MASK ((1 << RTE_IPV4_HDR_MF_SHIFT) - 1)
-
-#define RTE_IPV4_HDR_OFFSET_UNITS 8
-
-/* IPv4 options */
-#define RTE_IPV4_HDR_OPT_EOL 0
-#define RTE_IPV4_HDR_OPT_NOP 1
-#define RTE_IPV4_HDR_OPT_COPIED(v) ((v) & 0x80)
-#define RTE_IPV4_HDR_OPT_MAX_LEN 40
-
-/*
- * IPv4 address types
- */
-#define RTE_IPV4_ANY ((uint32_t)0x00000000) /**< 0.0.0.0 */
-#define RTE_IPV4_LOOPBACK ((uint32_t)0x7f000001) /**< 127.0.0.1 */
-#define RTE_IPV4_BROADCAST ((uint32_t)0xe0000000) /**< 224.0.0.0 */
-#define RTE_IPV4_ALLHOSTS_GROUP ((uint32_t)0xe0000001) /**< 224.0.0.1 */
-#define RTE_IPV4_ALLRTRS_GROUP ((uint32_t)0xe0000002) /**< 224.0.0.2 */
-#define RTE_IPV4_MAX_LOCAL_GROUP ((uint32_t)0xe00000ff) /**< 224.0.0.255 */
-
-/*
- * IPv4 Multicast-related macros
- */
-#define RTE_IPV4_MIN_MCAST \
- RTE_IPV4(224, 0, 0, 0) /**< Minimal IPv4-multicast address */
-#define RTE_IPV4_MAX_MCAST \
- RTE_IPV4(239, 255, 255, 255) /**< Maximum IPv4 multicast address */
-
-#define RTE_IS_IPV4_MCAST(x) \
- ((x) >= RTE_IPV4_MIN_MCAST && (x) <= RTE_IPV4_MAX_MCAST)
- /**< check if IPv4 address is multicast */
-
-/* IPv4 default fields values */
-#define RTE_IPV4_MIN_IHL (0x5)
-#define RTE_IPV4_VHL_DEF ((IPVERSION << 4) | RTE_IPV4_MIN_IHL)
-
-/**
- * Get the length of an IPv4 header.
- *
- * @param ipv4_hdr
- * Pointer to the IPv4 header.
- * @return
- * The length of the IPv4 header (with options if present) in bytes.
- */
-static inline uint8_t
-rte_ipv4_hdr_len(const struct rte_ipv4_hdr *ipv4_hdr)
-{
- return (uint8_t)((ipv4_hdr->version_ihl & RTE_IPV4_HDR_IHL_MASK) *
- RTE_IPV4_IHL_MULTIPLIER);
-}
-
-/**
- * Process the IPv4 checksum of an IPv4 header.
- *
- * The checksum field must be set to 0 by the caller.
- *
- * @param ipv4_hdr
- * The pointer to the contiguous IPv4 header.
- * @return
- * The complemented checksum to set in the IP packet.
- */
-static inline uint16_t
-rte_ipv4_cksum(const struct rte_ipv4_hdr *ipv4_hdr)
-{
- uint16_t cksum;
- cksum = rte_raw_cksum(ipv4_hdr, rte_ipv4_hdr_len(ipv4_hdr));
- return (uint16_t)~cksum;
-}
-
-/**
- * Process the pseudo-header checksum of an IPv4 header.
- *
- * The checksum field must be set to 0 by the caller.
- *
- * Depending on the ol_flags, the pseudo-header checksum expected by the
- * drivers is not the same. For instance, when TSO is enabled, the IP
- * payload length must not be included in the packet.
- *
- * When ol_flags is 0, it computes the standard pseudo-header checksum.
- *
- * @param ipv4_hdr
- * The pointer to the contiguous IPv4 header.
- * @param ol_flags
- * The ol_flags of the associated mbuf.
- * @return
- * The non-complemented checksum to set in the L4 header.
- */
-static inline uint16_t
-rte_ipv4_phdr_cksum(const struct rte_ipv4_hdr *ipv4_hdr, uint64_t ol_flags)
-{
- struct ipv4_psd_header {
- uint32_t src_addr; /* IP address of source host. */
- uint32_t dst_addr; /* IP address of destination host. */
- uint8_t zero; /* zero. */
- uint8_t proto; /* L4 protocol type. */
- uint16_t len; /* L4 length. */
- } psd_hdr;
-
- uint32_t l3_len;
-
- psd_hdr.src_addr = ipv4_hdr->src_addr;
- psd_hdr.dst_addr = ipv4_hdr->dst_addr;
- psd_hdr.zero = 0;
- psd_hdr.proto = ipv4_hdr->next_proto_id;
- if (ol_flags & (RTE_MBUF_F_TX_TCP_SEG | RTE_MBUF_F_TX_UDP_SEG)) {
- psd_hdr.len = 0;
- } else {
- l3_len = rte_be_to_cpu_16(ipv4_hdr->total_length);
- psd_hdr.len = rte_cpu_to_be_16((uint16_t)(l3_len -
- rte_ipv4_hdr_len(ipv4_hdr)));
- }
- return rte_raw_cksum(&psd_hdr, sizeof(psd_hdr));
-}
-
-/**
- * @internal Calculate the non-complemented IPv4 L4 checksum
- */
-static inline uint16_t
-__rte_ipv4_udptcp_cksum(const struct rte_ipv4_hdr *ipv4_hdr, const void *l4_hdr)
-{
- uint32_t cksum;
- uint32_t l3_len, l4_len;
- uint8_t ip_hdr_len;
-
- ip_hdr_len = rte_ipv4_hdr_len(ipv4_hdr);
- l3_len = rte_be_to_cpu_16(ipv4_hdr->total_length);
- if (l3_len < ip_hdr_len)
- return 0;
-
- l4_len = l3_len - ip_hdr_len;
-
- cksum = rte_raw_cksum(l4_hdr, l4_len);
- cksum += rte_ipv4_phdr_cksum(ipv4_hdr, 0);
-
- cksum = ((cksum & 0xffff0000) >> 16) + (cksum & 0xffff);
-
- return (uint16_t)cksum;
-}
-
-/**
- * Process the IPv4 UDP or TCP checksum.
- *
- * The layer 4 checksum must be set to 0 in the L4 header by the caller.
- *
- * @param ipv4_hdr
- * The pointer to the contiguous IPv4 header.
- * @param l4_hdr
- * The pointer to the beginning of the L4 header.
- * @return
- * The complemented checksum to set in the L4 header.
- */
-static inline uint16_t
-rte_ipv4_udptcp_cksum(const struct rte_ipv4_hdr *ipv4_hdr, const void *l4_hdr)
-{
- uint16_t cksum = __rte_ipv4_udptcp_cksum(ipv4_hdr, l4_hdr);
-
- cksum = ~cksum;
-
- /*
- * Per RFC 768: If the computed checksum is zero for UDP,
- * it is transmitted as all ones
- * (the equivalent in one's complement arithmetic).
- */
- if (cksum == 0 && ipv4_hdr->next_proto_id == IPPROTO_UDP)
- cksum = 0xffff;
-
- return cksum;
-}
-
-/**
- * @internal Calculate the non-complemented IPv4 L4 checksum of a packet
- */
-static inline uint16_t
-__rte_ipv4_udptcp_cksum_mbuf(const struct rte_mbuf *m,
- const struct rte_ipv4_hdr *ipv4_hdr,
- uint16_t l4_off)
-{
- uint16_t raw_cksum;
- uint32_t cksum;
- uint16_t len;
-
- if (unlikely(l4_off > m->pkt_len))
- return 0; /* invalid params, return a dummy value */
-
- len = rte_be_to_cpu_16(ipv4_hdr->total_length) - (uint16_t)rte_ipv4_hdr_len(ipv4_hdr);
-
- if (rte_raw_cksum_mbuf(m, l4_off, len, &raw_cksum))
- return 0;
-
- cksum = raw_cksum + rte_ipv4_phdr_cksum(ipv4_hdr, 0);
-
- cksum = ((cksum & 0xffff0000) >> 16) + (cksum & 0xffff);
-
- return (uint16_t)cksum;
-}
-
-/**
- * Compute the IPv4 UDP/TCP checksum of a packet.
- *
- * @param m
- * The pointer to the mbuf.
- * @param ipv4_hdr
- * The pointer to the contiguous IPv4 header.
- * @param l4_off
- * The offset in bytes to start L4 checksum.
- * @return
- * The complemented checksum to set in the L4 header.
- */
-static inline uint16_t
-rte_ipv4_udptcp_cksum_mbuf(const struct rte_mbuf *m,
- const struct rte_ipv4_hdr *ipv4_hdr, uint16_t l4_off)
-{
- uint16_t cksum = __rte_ipv4_udptcp_cksum_mbuf(m, ipv4_hdr, l4_off);
-
- cksum = ~cksum;
-
- /*
- * Per RFC 768: If the computed checksum is zero for UDP,
- * it is transmitted as all ones
- * (the equivalent in one's complement arithmetic).
- */
- if (cksum == 0 && ipv4_hdr->next_proto_id == IPPROTO_UDP)
- cksum = 0xffff;
-
- return cksum;
-}
-
-/**
- * Validate the IPv4 UDP or TCP checksum.
- *
- * In case of UDP, the caller must first check if udp_hdr->dgram_cksum is 0
- * (i.e. no checksum).
- *
- * @param ipv4_hdr
- * The pointer to the contiguous IPv4 header.
- * @param l4_hdr
- * The pointer to the beginning of the L4 header.
- * @return
- * Return 0 if the checksum is correct, else -1.
- */
-static inline int
-rte_ipv4_udptcp_cksum_verify(const struct rte_ipv4_hdr *ipv4_hdr,
- const void *l4_hdr)
-{
- uint16_t cksum = __rte_ipv4_udptcp_cksum(ipv4_hdr, l4_hdr);
-
- if (cksum != 0xffff)
- return -1;
-
- return 0;
-}
-
-/**
- * Verify the IPv4 UDP/TCP checksum of a packet.
- *
- * In case of UDP, the caller must first check if udp_hdr->dgram_cksum is 0
- * (i.e. no checksum).
- *
- * @param m
- * The pointer to the mbuf.
- * @param ipv4_hdr
- * The pointer to the contiguous IPv4 header.
- * @param l4_off
- * The offset in bytes to start L4 checksum.
- * @return
- * Return 0 if the checksum is correct, else -1.
- */
-static inline int
-rte_ipv4_udptcp_cksum_mbuf_verify(const struct rte_mbuf *m,
- const struct rte_ipv4_hdr *ipv4_hdr,
- uint16_t l4_off)
-{
- uint16_t cksum = __rte_ipv4_udptcp_cksum_mbuf(m, ipv4_hdr, l4_off);
-
- if (cksum != 0xffff)
- return -1;
-
- return 0;
-}
-
/**
* IPv6 Header
*/
@@ -697,4 +358,4 @@ rte_ipv6_get_next_ext(const uint8_t *p, int proto, size_t *ext_len)
}
#endif
-#endif /* _RTE_IP_H_ */
+#endif /* _RTE_IP6_H_ */
diff --git a/lib/net/rte_net.c b/lib/net/rte_net.c
index d680accc1631..13aef6147ab0 100644
--- a/lib/net/rte_net.c
+++ b/lib/net/rte_net.c
@@ -9,6 +9,7 @@
#include <rte_byteorder.h>
#include <rte_ether.h>
#include <rte_ip.h>
+#include <rte_ip6.h>
#include <rte_tcp.h>
#include <rte_udp.h>
#include <rte_sctp.h>
diff --git a/lib/net/rte_net.h b/lib/net/rte_net.h
index cdc6cf956df2..a47beb60adf0 100644
--- a/lib/net/rte_net.h
+++ b/lib/net/rte_net.h
@@ -10,6 +10,7 @@ extern "C" {
#endif
#include <rte_ip.h>
+#include <rte_ip6.h>
#include <rte_udp.h>
#include <rte_tcp.h>
diff --git a/lib/pipeline/rte_swx_ipsec.c b/lib/pipeline/rte_swx_ipsec.c
index 73e8211b2818..56a401d0fa63 100644
--- a/lib/pipeline/rte_swx_ipsec.c
+++ b/lib/pipeline/rte_swx_ipsec.c
@@ -11,6 +11,7 @@
#include <rte_common.h>
#include <rte_random.h>
#include <rte_ip.h>
+#include <rte_ip6.h>
#include <rte_tailq.h>
#include <rte_eal_memconfig.h>
#include <rte_ring.h>
diff --git a/lib/pipeline/rte_table_action.c b/lib/pipeline/rte_table_action.c
index 87c3e0e2c935..ebbdd3f4cebe 100644
--- a/lib/pipeline/rte_table_action.c
+++ b/lib/pipeline/rte_table_action.c
@@ -11,6 +11,7 @@
#include <rte_memcpy.h>
#include <rte_ether.h>
#include <rte_ip.h>
+#include <rte_ip6.h>
#include <rte_tcp.h>
#include <rte_udp.h>
#include <rte_vxlan.h>
diff --git a/lib/vhost/virtio_net.c b/lib/vhost/virtio_net.c
index 370402d84903..d4dcc811c55f 100644
--- a/lib/vhost/virtio_net.c
+++ b/lib/vhost/virtio_net.c
@@ -11,6 +11,7 @@
#include <rte_net.h>
#include <rte_ether.h>
#include <rte_ip.h>
+#include <rte_ip6.h>
#include <rte_dmadev.h>
#include <rte_vhost.h>
#include <rte_tcp.h>
--
2.46.1
^ permalink raw reply [relevance 2%]
* [PATCH dpdk v7] graph: expose node context as pointers
@ 2024-09-30 9:27 4% Robin Jarry
0 siblings, 0 replies; 200+ results
From: Robin Jarry @ 2024-09-30 9:27 UTC (permalink / raw)
To: dev, Jerin Jacob, Kiran Kumar K, Nithin Dabilpuram, Zhirun Yan
In some cases, the node context data is used to store two pointers
because the data is larger than the reserved 16 bytes. Having to define
intermediate structures just to be able to cast is tedious. And without
intermediate structures, casting to opaque pointers is hard without
violating strict aliasing rules.
Add an unnamed union to allow storing opaque pointers in the node
context. Unfortunately, aligning an unnamed union that contains an array
produces inconsistent results between C and C++. To preserve ABI/API
compatibility in both C and C++, move all fast-path area fields into an
unnamed struct which is itself cache aligned. Use __rte_cache_aligned to
preserve existing alignment on architectures where cache lines are 128
bytes.
Add a static assert to ensure that the fast path area does not grow
beyond a 64 bytes cache line.
Signed-off-by: Robin Jarry <rjarry@redhat.com>
Acked-by: Kiran Kumar Kokkilagadda <kirankumark@marvell.com>
---
Notes:
v7:
* rebased on bf0ff8df59c7 ("maintainers: fix prog guide paths")
* no change, only to re-trigger ci
v6:
* Fix ABI breakage on arm64 (and all platforms that have RTE_CACHE_LINE_SIZE=128).
* This patch will cause CI failures without libabigail 2.5. See this commit
https://sourceware.org/git/?p=libabigail.git;a=commitdiff;h=f821c2be3fff2047ef8fc436f6f02301812d166f
for more details.
v5:
* Helper functions to hide casting proved to be harder than expected.
Naive casting may even be impossible without breaking strict aliasing
rules. The only other option would be to use explicit memcpy calls.
* Unnamed union tentative again. As suggested by Tyler (thank you!),
using an intermediate unnamed struct to carry the alignment produces
consistent ABI in C and C++.
* Also, Tyler (thank you!) suggested that the fast path area alignment
size may be incorrect for architectures where the cache line is not 64
bytes. There will be a 64 bytes hole in the structure at the end of
the unnamed struct before the zero length next nodes array. Use
__rte_cache_min_aligned to preserve existing alignment.
lib/graph/rte_graph_worker_common.h | 29 +++++++++++++++++++++--------
1 file changed, 21 insertions(+), 8 deletions(-)
diff --git a/lib/graph/rte_graph_worker_common.h b/lib/graph/rte_graph_worker_common.h
index 36d864e2c14e..8d8956fdddda 100644
--- a/lib/graph/rte_graph_worker_common.h
+++ b/lib/graph/rte_graph_worker_common.h
@@ -12,7 +12,9 @@
* process, enqueue and move streams of objects to the next nodes.
*/
+#include <assert.h>
#include <stdalign.h>
+#include <stddef.h>
#include <rte_common.h>
#include <rte_cycles.h>
@@ -111,14 +113,21 @@ struct __rte_cache_aligned rte_node {
} dispatch;
};
/* Fast path area */
+ __extension__ struct __rte_cache_aligned {
#define RTE_NODE_CTX_SZ 16
- alignas(RTE_CACHE_LINE_SIZE) uint8_t ctx[RTE_NODE_CTX_SZ]; /**< Node Context. */
- uint16_t size; /**< Total number of objects available. */
- uint16_t idx; /**< Number of objects used. */
- rte_graph_off_t off; /**< Offset of node in the graph reel. */
- uint64_t total_cycles; /**< Cycles spent in this node. */
- uint64_t total_calls; /**< Calls done to this node. */
- uint64_t total_objs; /**< Objects processed by this node. */
+ union {
+ uint8_t ctx[RTE_NODE_CTX_SZ];
+ __extension__ struct {
+ void *ctx_ptr;
+ void *ctx_ptr2;
+ };
+ }; /**< Node Context. */
+ uint16_t size; /**< Total number of objects available. */
+ uint16_t idx; /**< Number of objects used. */
+ rte_graph_off_t off; /**< Offset of node in the graph reel. */
+ uint64_t total_cycles; /**< Cycles spent in this node. */
+ uint64_t total_calls; /**< Calls done to this node. */
+ uint64_t total_objs; /**< Objects processed by this node. */
union {
void **objs; /**< Array of object pointers. */
uint64_t objs_u64;
@@ -127,9 +136,13 @@ struct __rte_cache_aligned rte_node {
rte_node_process_t process; /**< Process function. */
uint64_t process_u64;
};
- alignas(RTE_CACHE_LINE_MIN_SIZE) struct rte_node *nodes[]; /**< Next nodes. */
+ alignas(RTE_CACHE_LINE_MIN_SIZE) struct rte_node *nodes[]; /**< Next nodes. */
+ };
};
+static_assert(offsetof(struct rte_node, nodes) - offsetof(struct rte_node, ctx)
+ == RTE_CACHE_LINE_MIN_SIZE, "rte_node fast path area must fit in 64 bytes");
+
/**
* @internal
*
--
2.46.1
^ permalink raw reply [relevance 4%]
* [PATCH RESEND v7 2/5] ethdev: fix skip valid port in probing callback
2024-09-29 5:52 3% ` [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port Huisong Li
@ 2024-09-29 5:52 2% ` Huisong Li
2024-10-08 2:32 0% ` [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C)
1 sibling, 0 replies; 200+ results
From: Huisong Li @ 2024-09-29 5:52 UTC (permalink / raw)
To: thomas, ferruh.yigit, andrew.rybchenko, Ajit Khaparde,
Somnath Kotur, Dariusz Sosnowski, Viacheslav Ovsiienko, Ori Kam,
Suanming Mou, Matan Azrad, Stephen Hemminger
Cc: dev, fengchengwen, liuyonglong, lihuisong
The event callback in application may use the macro RTE_ETH_FOREACH_DEV to
iterate over all enabled ports to do something(like, verifying the port id
validity) when receive a probing event. If the ethdev state of a port is
not RTE_ETH_DEV_UNUSED, this port will be considered as a valid port.
However, this state is set to RTE_ETH_DEV_ATTACHED after pushing probing
event. It means that probing callback will skip this port. But this
assignment can not move to front of probing notification. See
commit be8cd210379a ("ethdev: fix port probing notification")
So this patch has to add a new state, RTE_ETH_DEV_ALLOCATED. Set the ethdev
state to RTE_ETH_DEV_ALLOCATED before pushing probing event and set it to
RTE_ETH_DEV_ATTACHED after definitely probed. And this port is valid if its
device state is 'ALLOCATED' or 'ATTACHED'.
In addition, the new state has to be placed behind 'REMOVED' to avoid ABI
break. Fortunately, this ethdev state is internal and applications can not
access it directly. So this patch encapsulates an API, rte_eth_dev_is_used,
for ethdev or PMD to call and eliminate concerns about using this state
enum value comparison.
Fixes: be8cd210379a ("ethdev: fix port probing notification")
Cc: stable@dpdk.org
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>
---
drivers/net/bnxt/bnxt_ethdev.c | 3 ++-
drivers/net/mlx5/mlx5.c | 2 +-
lib/ethdev/ethdev_driver.c | 13 ++++++++++---
lib/ethdev/ethdev_driver.h | 12 ++++++++++++
lib/ethdev/ethdev_pci.h | 2 +-
lib/ethdev/rte_class_eth.c | 2 +-
lib/ethdev/rte_ethdev.c | 4 ++--
lib/ethdev/rte_ethdev.h | 4 +++-
lib/ethdev/version.map | 1 +
9 files changed, 33 insertions(+), 10 deletions(-)
diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index c6ad764813..7401dcd8b5 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -6612,7 +6612,8 @@ bnxt_dev_uninit(struct rte_eth_dev *eth_dev)
PMD_DRV_LOG(DEBUG, "Calling Device uninit\n");
- if (eth_dev->state != RTE_ETH_DEV_UNUSED)
+
+ if (rte_eth_dev_is_used(eth_dev->state))
bnxt_dev_close_op(eth_dev);
return 0;
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 8d266b0e64..0df49e1f69 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -3371,7 +3371,7 @@ mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev)
while (port_id < RTE_MAX_ETHPORTS) {
struct rte_eth_dev *dev = &rte_eth_devices[port_id];
- if (dev->state != RTE_ETH_DEV_UNUSED &&
+ if (rte_eth_dev_is_used(dev->state) &&
dev->device &&
(dev->device == odev ||
(dev->device->driver &&
diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c
index c335a25a82..a87dbb00ff 100644
--- a/lib/ethdev/ethdev_driver.c
+++ b/lib/ethdev/ethdev_driver.c
@@ -55,8 +55,8 @@ eth_dev_find_free_port(void)
for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
/* Using shared name field to find a free port. */
if (eth_dev_shared_data->data[i].name[0] == '\0') {
- RTE_ASSERT(rte_eth_devices[i].state ==
- RTE_ETH_DEV_UNUSED);
+ RTE_ASSERT(!rte_eth_dev_is_used(
+ rte_eth_devices[i].state));
return i;
}
}
@@ -221,11 +221,18 @@ rte_eth_dev_probing_finish(struct rte_eth_dev *dev)
if (rte_eal_process_type() == RTE_PROC_SECONDARY)
eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev);
+ dev->state = RTE_ETH_DEV_ALLOCATED;
rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL);
dev->state = RTE_ETH_DEV_ATTACHED;
}
+bool rte_eth_dev_is_used(uint16_t dev_state)
+{
+ return dev_state == RTE_ETH_DEV_ALLOCATED ||
+ dev_state == RTE_ETH_DEV_ATTACHED;
+}
+
int
rte_eth_dev_release_port(struct rte_eth_dev *eth_dev)
{
@@ -243,7 +250,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev)
if (ret != 0)
return ret;
- if (eth_dev->state != RTE_ETH_DEV_UNUSED)
+ if (rte_eth_dev_is_used(eth_dev->state))
rte_eth_dev_callback_process(eth_dev,
RTE_ETH_EVENT_DESTROY, NULL);
diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h
index abed4784aa..aa35b65848 100644
--- a/lib/ethdev/ethdev_driver.h
+++ b/lib/ethdev/ethdev_driver.h
@@ -1704,6 +1704,18 @@ int rte_eth_dev_callback_process(struct rte_eth_dev *dev,
__rte_internal
void rte_eth_dev_probing_finish(struct rte_eth_dev *dev);
+/**
+ * Check if a Ethernet device state is used or not
+ *
+ * @param dev_state
+ * The state of the Ethernet device
+ * @return
+ * - true if the state of the Ethernet device is allocated or attached
+ * - false if this state is neither allocated nor attached
+ */
+__rte_internal
+bool rte_eth_dev_is_used(uint16_t dev_state);
+
/**
* Create memzone for HW rings.
* malloc can't be used as the physical address is needed.
diff --git a/lib/ethdev/ethdev_pci.h b/lib/ethdev/ethdev_pci.h
index ec4f731270..05dec6716b 100644
--- a/lib/ethdev/ethdev_pci.h
+++ b/lib/ethdev/ethdev_pci.h
@@ -179,7 +179,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device *pci_dev,
* eth device has been released.
*/
if (rte_eal_process_type() == RTE_PROC_SECONDARY &&
- eth_dev->state == RTE_ETH_DEV_UNUSED)
+ !rte_eth_dev_is_used(eth_dev->state))
return 0;
if (dev_uninit) {
diff --git a/lib/ethdev/rte_class_eth.c b/lib/ethdev/rte_class_eth.c
index b52f1dd9f2..81e70670d9 100644
--- a/lib/ethdev/rte_class_eth.c
+++ b/lib/ethdev/rte_class_eth.c
@@ -118,7 +118,7 @@ eth_dev_match(const struct rte_eth_dev *edev,
const struct rte_kvargs *kvlist = arg->kvlist;
unsigned int pair;
- if (edev->state == RTE_ETH_DEV_UNUSED)
+ if (!rte_eth_dev_is_used(edev->state))
return -1;
if (arg->device != NULL && arg->device != edev->device)
return -1;
diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
index a1f7efa913..4dc66abb7b 100644
--- a/lib/ethdev/rte_ethdev.c
+++ b/lib/ethdev/rte_ethdev.c
@@ -349,7 +349,7 @@ uint16_t
rte_eth_find_next(uint16_t port_id)
{
while (port_id < RTE_MAX_ETHPORTS &&
- rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED)
+ !rte_eth_dev_is_used(rte_eth_devices[port_id].state))
port_id++;
if (port_id >= RTE_MAX_ETHPORTS)
@@ -408,7 +408,7 @@ rte_eth_dev_is_valid_port(uint16_t port_id)
int is_valid;
if (port_id >= RTE_MAX_ETHPORTS ||
- (rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED))
+ !rte_eth_dev_is_used(rte_eth_devices[port_id].state))
is_valid = 0;
else
is_valid = 1;
diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
index a9f92006da..9cc37e8cde 100644
--- a/lib/ethdev/rte_ethdev.h
+++ b/lib/ethdev/rte_ethdev.h
@@ -2083,10 +2083,12 @@ typedef uint16_t (*rte_tx_callback_fn)(uint16_t port_id, uint16_t queue,
enum rte_eth_dev_state {
/** Device is unused before being probed. */
RTE_ETH_DEV_UNUSED = 0,
- /** Device is attached when allocated in probing. */
+ /** Device is attached when definitely probed. */
RTE_ETH_DEV_ATTACHED,
/** Device is in removed state when plug-out is detected. */
RTE_ETH_DEV_REMOVED,
+ /** Device is allocated and is set before reporting new event. */
+ RTE_ETH_DEV_ALLOCATED,
};
struct rte_eth_dev_sriov {
diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
index f63dc32aa2..6ecf1ab89d 100644
--- a/lib/ethdev/version.map
+++ b/lib/ethdev/version.map
@@ -349,6 +349,7 @@ INTERNAL {
rte_eth_dev_get_by_name;
rte_eth_dev_is_rx_hairpin_queue;
rte_eth_dev_is_tx_hairpin_queue;
+ rte_eth_dev_is_used;
rte_eth_dev_probing_finish;
rte_eth_dev_release_port;
rte_eth_dev_internal_reset;
--
2.33.0
^ permalink raw reply [relevance 2%]
* [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port
[not found] <20220825024425.10534-1-lihuisong@huawei.com>
@ 2024-09-29 5:52 3% ` Huisong Li
2024-09-29 5:52 2% ` [PATCH RESEND v7 2/5] ethdev: fix skip valid port in probing callback Huisong Li
2024-10-08 2:32 0% ` [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C)
0 siblings, 2 replies; 200+ results
From: Huisong Li @ 2024-09-29 5:52 UTC (permalink / raw)
To: thomas, ferruh.yigit, andrew.rybchenko
Cc: dev, fengchengwen, liuyonglong, lihuisong
This patchset fix some bugs and support attaching and detaching port
in primary and secondary.
---
-v7: fix conflicts
-v6: adjust rte_eth_dev_is_used position based on alphabetical order
in version.map
-v5: move 'ALLOCATED' state to the back of 'REMOVED' to avoid abi break.
-v4: fix a misspelling.
-v3:
#1 merge patch 1/6 and patch 2/6 into patch 1/5, and add modification
for other bus type.
#2 add a RTE_ETH_DEV_ALLOCATED state in rte_eth_dev_state to resolve
the probelm in patch 2/5.
-v2: resend due to CI unexplained failure.
Huisong Li (5):
drivers/bus: restore driver assignment at front of probing
ethdev: fix skip valid port in probing callback
app/testpmd: check the validity of the port
app/testpmd: add attach and detach port for multiple process
app/testpmd: stop forwarding in new or destroy event
app/test-pmd/testpmd.c | 47 +++++++++++++++---------
app/test-pmd/testpmd.h | 1 -
drivers/bus/auxiliary/auxiliary_common.c | 9 ++++-
drivers/bus/dpaa/dpaa_bus.c | 9 ++++-
drivers/bus/fslmc/fslmc_bus.c | 8 +++-
drivers/bus/ifpga/ifpga_bus.c | 12 ++++--
drivers/bus/pci/pci_common.c | 9 ++++-
drivers/bus/vdev/vdev.c | 10 ++++-
drivers/bus/vmbus/vmbus_common.c | 9 ++++-
drivers/net/bnxt/bnxt_ethdev.c | 3 +-
drivers/net/bonding/bonding_testpmd.c | 1 -
drivers/net/mlx5/mlx5.c | 2 +-
lib/ethdev/ethdev_driver.c | 13 +++++--
lib/ethdev/ethdev_driver.h | 12 ++++++
lib/ethdev/ethdev_pci.h | 2 +-
lib/ethdev/rte_class_eth.c | 2 +-
lib/ethdev/rte_ethdev.c | 4 +-
lib/ethdev/rte_ethdev.h | 4 +-
lib/ethdev/version.map | 1 +
19 files changed, 114 insertions(+), 44 deletions(-)
--
2.33.0
^ permalink raw reply [relevance 3%]
* [PATCH v9 1/8] ethdev: support report register names and filter
@ 2024-09-26 12:42 4% ` Jie Hai
0 siblings, 0 replies; 200+ results
From: Jie Hai @ 2024-09-26 12:42 UTC (permalink / raw)
To: dev, thomas, ferruh.yigit, andrew.rybchenko
Cc: lihuisong, fengchengwen, haijie1
This patch adds "filter" and "names" fields to "rte_dev_reg_info"
structure. Names of registers in data fields can be reported and
the registers can be filtered by their module names.
The new API rte_eth_dev_get_reg_info_ext() is added to support
reporting names and filtering by modules. And the original API
rte_eth_dev_get_reg_info() does not use the names and filter fields.
A local variable is used in rte_eth_dev_get_reg_info for
compatibility. If the drivers does not report the names, set them
to "index_XXX", which means the location in the register table.
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Huisong Li <lihuisong@huawei.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@amd.com>
---
doc/guides/rel_notes/release_24_11.rst | 8 ++++++
lib/ethdev/ethdev_trace.h | 2 ++
lib/ethdev/rte_dev_info.h | 11 ++++++++
lib/ethdev/rte_ethdev.c | 38 ++++++++++++++++++++++++++
lib/ethdev/rte_ethdev.h | 29 ++++++++++++++++++++
lib/ethdev/version.map | 1 +
6 files changed, 89 insertions(+)
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 5dd575fb39df..85960e71d0ca 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -60,6 +60,11 @@ New Features
* Added SR-IOV VF support.
* Added recent 1400/14000 and 15000 models to the supported list.
+* **Added support for dumping registers with names and filtering by modules.**
+
+ Added new API functions ``rte_eth_dev_get_reg_info_ext()`` to filter the
+ registers by module names and get the information (names, values and other
+ attributes) of the filtered registers.
Removed Items
-------------
@@ -108,6 +113,9 @@ ABI Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* ethdev: Added ``filter`` and ``names`` fields to ``rte_dev_reg_info``
+ structure for filtering by modules and reporting names of registers.
+
Known Issues
------------
diff --git a/lib/ethdev/ethdev_trace.h b/lib/ethdev/ethdev_trace.h
index 3bec87bfdb70..0c4780a09ef5 100644
--- a/lib/ethdev/ethdev_trace.h
+++ b/lib/ethdev/ethdev_trace.h
@@ -1152,6 +1152,8 @@ RTE_TRACE_POINT(
rte_trace_point_emit_u32(info->length);
rte_trace_point_emit_u32(info->width);
rte_trace_point_emit_u32(info->version);
+ rte_trace_point_emit_ptr(info->names);
+ rte_trace_point_emit_ptr(info->filter);
rte_trace_point_emit_int(ret);
)
diff --git a/lib/ethdev/rte_dev_info.h b/lib/ethdev/rte_dev_info.h
index 67cf0ae52668..26b777f9836e 100644
--- a/lib/ethdev/rte_dev_info.h
+++ b/lib/ethdev/rte_dev_info.h
@@ -11,6 +11,11 @@ extern "C" {
#include <stdint.h>
+#define RTE_ETH_REG_NAME_SIZE 64
+struct rte_eth_reg_name {
+ char name[RTE_ETH_REG_NAME_SIZE];
+};
+
/*
* Placeholder for accessing device registers
*/
@@ -20,6 +25,12 @@ struct rte_dev_reg_info {
uint32_t length; /**< Number of registers to fetch */
uint32_t width; /**< Size of device register */
uint32_t version; /**< Device version */
+ /**
+ * Name of target module, filter for target subset of registers.
+ * This field could affects register selection for data/length/names.
+ */
+ const char *filter;
+ struct rte_eth_reg_name *names; /**< Registers name saver */
};
/*
diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
index f1c658f49e80..30ca4a0043c5 100644
--- a/lib/ethdev/rte_ethdev.c
+++ b/lib/ethdev/rte_ethdev.c
@@ -6388,8 +6388,37 @@ rte_eth_read_clock(uint16_t port_id, uint64_t *clock)
int
rte_eth_dev_get_reg_info(uint16_t port_id, struct rte_dev_reg_info *info)
+{
+ struct rte_dev_reg_info reg_info = { 0 };
+ int ret;
+
+ if (info == NULL) {
+ RTE_ETHDEV_LOG_LINE(ERR,
+ "Cannot get ethdev port %u register info to NULL",
+ port_id);
+ return -EINVAL;
+ }
+
+ reg_info.length = info->length;
+ reg_info.data = info->data;
+
+ ret = rte_eth_dev_get_reg_info_ext(port_id, ®_info);
+ if (ret != 0)
+ return ret;
+
+ info->length = reg_info.length;
+ info->width = reg_info.width;
+ info->version = reg_info.version;
+ info->offset = reg_info.offset;
+
+ return 0;
+}
+
+int
+rte_eth_dev_get_reg_info_ext(uint16_t port_id, struct rte_dev_reg_info *info)
{
struct rte_eth_dev *dev;
+ uint32_t i;
int ret;
RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
@@ -6402,12 +6431,21 @@ rte_eth_dev_get_reg_info(uint16_t port_id, struct rte_dev_reg_info *info)
return -EINVAL;
}
+ if (info->names != NULL && info->length != 0)
+ memset(info->names, 0, sizeof(struct rte_eth_reg_name) * info->length);
+
if (*dev->dev_ops->get_reg == NULL)
return -ENOTSUP;
ret = eth_err(port_id, (*dev->dev_ops->get_reg)(dev, info));
rte_ethdev_trace_get_reg_info(port_id, info, ret);
+ /* Report the default names if drivers not report. */
+ if (ret == 0 && info->names != NULL && strlen(info->names[0].name) == 0) {
+ for (i = 0; i < info->length; i++)
+ snprintf(info->names[i].name, RTE_ETH_REG_NAME_SIZE,
+ "index_%u", info->offset + i);
+ }
return ret;
}
diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
index eef254c46330..96b117899ae9 100644
--- a/lib/ethdev/rte_ethdev.h
+++ b/lib/ethdev/rte_ethdev.h
@@ -5071,6 +5071,35 @@ __rte_experimental
int rte_eth_get_monitor_addr(uint16_t port_id, uint16_t queue_id,
struct rte_power_monitor_cond *pmc);
+/**
+ * Retrieve the filtered device registers (values and names) and
+ * register attributes (number of registers and register size)
+ *
+ * @param port_id
+ * The port identifier of the Ethernet device.
+ * @param info
+ * Pointer to rte_dev_reg_info structure to fill in.
+ * - If info->filter is NULL, return info for all registers (seen as filter
+ * none).
+ * - If info->filter is not NULL, return error if the driver does not support
+ * filter. Fill the length field with filtered register number.
+ * - If info->data is NULL, the function fills in the width and length fields.
+ * - If info->data is not NULL, ethdev considers there are enough spaces to
+ * store the registers, and the values of registers with the filter string
+ * as the module name are put into the buffer pointed at by info->data.
+ * - If info->names is not NULL, drivers should fill it or the ethdev fills it
+ * with default names.
+ * @return
+ * - (0) if successful.
+ * - (-ENOTSUP) if hardware doesn't support.
+ * - (-EINVAL) if bad parameter.
+ * - (-ENODEV) if *port_id* invalid.
+ * - (-EIO) if device is removed.
+ * - others depends on the specific operations implementation.
+ */
+__rte_experimental
+int rte_eth_dev_get_reg_info_ext(uint16_t port_id, struct rte_dev_reg_info *info);
+
/**
* Retrieve device registers and register attributes (number of registers and
* register size)
diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
index f6aca9506984..6bcfe073a818 100644
--- a/lib/ethdev/version.map
+++ b/lib/ethdev/version.map
@@ -330,6 +330,7 @@ EXPERIMENTAL {
__rte_ethdev_trace_rx_burst_nonempty;
__rte_eth_trace_call_rx_callbacks_empty;
__rte_eth_trace_call_rx_callbacks_nonempty;
+ rte_eth_dev_get_reg_info_ext;
};
INTERNAL {
--
2.33.0
^ permalink raw reply [relevance 4%]
* [PATCH v8 1/8] ethdev: support report register names and filter
@ 2024-09-25 6:40 4% ` Jie Hai
0 siblings, 0 replies; 200+ results
From: Jie Hai @ 2024-09-25 6:40 UTC (permalink / raw)
To: dev, thomas, ferruh.yigit, andrew.rybchenko
Cc: lihuisong, fengchengwen, haijie1
This patch adds "filter" and "names" fields to "rte_dev_reg_info"
structure. Names of registers in data fields can be reported and
the registers can be filtered by their module names.
The new API rte_eth_dev_get_reg_info_ext() is added to support
reporting names and filtering by modules. And the original API
rte_eth_dev_get_reg_info() does not use the names and filter fields.
A local variable is used in rte_eth_dev_get_reg_info for
compatibility. If the drivers does not report the names, set them
to "index_XXX", which means the location in the register table.
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Huisong Li <lihuisong@huawei.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@amd.com>
---
doc/guides/rel_notes/release_24_11.rst | 8 ++++++
lib/ethdev/ethdev_trace.h | 2 ++
lib/ethdev/rte_dev_info.h | 11 ++++++++
lib/ethdev/rte_ethdev.c | 38 ++++++++++++++++++++++++++
lib/ethdev/rte_ethdev.h | 29 ++++++++++++++++++++
lib/ethdev/version.map | 1 +
6 files changed, 89 insertions(+)
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 5dd575fb39df..85960e71d0ca 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -60,6 +60,11 @@ New Features
* Added SR-IOV VF support.
* Added recent 1400/14000 and 15000 models to the supported list.
+* **Added support for dumping registers with names and filtering by modules.**
+
+ Added new API functions ``rte_eth_dev_get_reg_info_ext()`` to filter the
+ registers by module names and get the information (names, values and other
+ attributes) of the filtered registers.
Removed Items
-------------
@@ -108,6 +113,9 @@ ABI Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* ethdev: Added ``filter`` and ``names`` fields to ``rte_dev_reg_info``
+ structure for filtering by modules and reporting names of registers.
+
Known Issues
------------
diff --git a/lib/ethdev/ethdev_trace.h b/lib/ethdev/ethdev_trace.h
index 3bec87bfdb70..0c4780a09ef5 100644
--- a/lib/ethdev/ethdev_trace.h
+++ b/lib/ethdev/ethdev_trace.h
@@ -1152,6 +1152,8 @@ RTE_TRACE_POINT(
rte_trace_point_emit_u32(info->length);
rte_trace_point_emit_u32(info->width);
rte_trace_point_emit_u32(info->version);
+ rte_trace_point_emit_ptr(info->names);
+ rte_trace_point_emit_ptr(info->filter);
rte_trace_point_emit_int(ret);
)
diff --git a/lib/ethdev/rte_dev_info.h b/lib/ethdev/rte_dev_info.h
index 67cf0ae52668..26b777f9836e 100644
--- a/lib/ethdev/rte_dev_info.h
+++ b/lib/ethdev/rte_dev_info.h
@@ -11,6 +11,11 @@ extern "C" {
#include <stdint.h>
+#define RTE_ETH_REG_NAME_SIZE 64
+struct rte_eth_reg_name {
+ char name[RTE_ETH_REG_NAME_SIZE];
+};
+
/*
* Placeholder for accessing device registers
*/
@@ -20,6 +25,12 @@ struct rte_dev_reg_info {
uint32_t length; /**< Number of registers to fetch */
uint32_t width; /**< Size of device register */
uint32_t version; /**< Device version */
+ /**
+ * Name of target module, filter for target subset of registers.
+ * This field could affects register selection for data/length/names.
+ */
+ const char *filter;
+ struct rte_eth_reg_name *names; /**< Registers name saver */
};
/*
diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
index f1c658f49e80..30ca4a0043c5 100644
--- a/lib/ethdev/rte_ethdev.c
+++ b/lib/ethdev/rte_ethdev.c
@@ -6388,8 +6388,37 @@ rte_eth_read_clock(uint16_t port_id, uint64_t *clock)
int
rte_eth_dev_get_reg_info(uint16_t port_id, struct rte_dev_reg_info *info)
+{
+ struct rte_dev_reg_info reg_info = { 0 };
+ int ret;
+
+ if (info == NULL) {
+ RTE_ETHDEV_LOG_LINE(ERR,
+ "Cannot get ethdev port %u register info to NULL",
+ port_id);
+ return -EINVAL;
+ }
+
+ reg_info.length = info->length;
+ reg_info.data = info->data;
+
+ ret = rte_eth_dev_get_reg_info_ext(port_id, ®_info);
+ if (ret != 0)
+ return ret;
+
+ info->length = reg_info.length;
+ info->width = reg_info.width;
+ info->version = reg_info.version;
+ info->offset = reg_info.offset;
+
+ return 0;
+}
+
+int
+rte_eth_dev_get_reg_info_ext(uint16_t port_id, struct rte_dev_reg_info *info)
{
struct rte_eth_dev *dev;
+ uint32_t i;
int ret;
RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
@@ -6402,12 +6431,21 @@ rte_eth_dev_get_reg_info(uint16_t port_id, struct rte_dev_reg_info *info)
return -EINVAL;
}
+ if (info->names != NULL && info->length != 0)
+ memset(info->names, 0, sizeof(struct rte_eth_reg_name) * info->length);
+
if (*dev->dev_ops->get_reg == NULL)
return -ENOTSUP;
ret = eth_err(port_id, (*dev->dev_ops->get_reg)(dev, info));
rte_ethdev_trace_get_reg_info(port_id, info, ret);
+ /* Report the default names if drivers not report. */
+ if (ret == 0 && info->names != NULL && strlen(info->names[0].name) == 0) {
+ for (i = 0; i < info->length; i++)
+ snprintf(info->names[i].name, RTE_ETH_REG_NAME_SIZE,
+ "index_%u", info->offset + i);
+ }
return ret;
}
diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
index eef254c46330..96b117899ae9 100644
--- a/lib/ethdev/rte_ethdev.h
+++ b/lib/ethdev/rte_ethdev.h
@@ -5071,6 +5071,35 @@ __rte_experimental
int rte_eth_get_monitor_addr(uint16_t port_id, uint16_t queue_id,
struct rte_power_monitor_cond *pmc);
+/**
+ * Retrieve the filtered device registers (values and names) and
+ * register attributes (number of registers and register size)
+ *
+ * @param port_id
+ * The port identifier of the Ethernet device.
+ * @param info
+ * Pointer to rte_dev_reg_info structure to fill in.
+ * - If info->filter is NULL, return info for all registers (seen as filter
+ * none).
+ * - If info->filter is not NULL, return error if the driver does not support
+ * filter. Fill the length field with filtered register number.
+ * - If info->data is NULL, the function fills in the width and length fields.
+ * - If info->data is not NULL, ethdev considers there are enough spaces to
+ * store the registers, and the values of registers with the filter string
+ * as the module name are put into the buffer pointed at by info->data.
+ * - If info->names is not NULL, drivers should fill it or the ethdev fills it
+ * with default names.
+ * @return
+ * - (0) if successful.
+ * - (-ENOTSUP) if hardware doesn't support.
+ * - (-EINVAL) if bad parameter.
+ * - (-ENODEV) if *port_id* invalid.
+ * - (-EIO) if device is removed.
+ * - others depends on the specific operations implementation.
+ */
+__rte_experimental
+int rte_eth_dev_get_reg_info_ext(uint16_t port_id, struct rte_dev_reg_info *info);
+
/**
* Retrieve device registers and register attributes (number of registers and
* register size)
diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
index f6aca9506984..6bcfe073a818 100644
--- a/lib/ethdev/version.map
+++ b/lib/ethdev/version.map
@@ -330,6 +330,7 @@ EXPERIMENTAL {
__rte_ethdev_trace_rx_burst_nonempty;
__rte_eth_trace_call_rx_callbacks_empty;
__rte_eth_trace_call_rx_callbacks_nonempty;
+ rte_eth_dev_get_reg_info_ext;
};
INTERNAL {
--
2.33.0
^ permalink raw reply [relevance 4%]
* [PATCH v2 1/2] cryptodev: add queue pair priority
@ 2024-09-24 16:38 9% ` Akhil Goyal
0 siblings, 0 replies; 200+ results
From: Akhil Goyal @ 2024-09-24 16:38 UTC (permalink / raw)
To: dev
Cc: thomas, david.marchand, hemant.agrawal, anoobj,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde, rnagadheeraj,
Akhil Goyal
Added a new field priority in `rte_cryptodev_qp_conf`,
to set the queue pair priority while setting up.
The priorities can be set between
`RTE_CRYPTODEV_QP_PRIORITY_HIGHEST` and
`RTE_CRYPTODEV_QP_PRIORITY_LOWEST`.
The underlying implementation may normalize the value
as per the supported priority levels.
If the implementation does not support setting up
priority, all queue pairs will be on same priority level
and this field will be ignored.
Signed-off-by: Akhil Goyal <gakhil@marvell.com>
---
doc/guides/rel_notes/deprecation.rst | 3 ---
doc/guides/rel_notes/release_24_11.rst | 3 +++
lib/cryptodev/rte_cryptodev.h | 22 ++++++++++++++++++++++
3 files changed, 25 insertions(+), 3 deletions(-)
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 211f59fdc9..3605e9cddc 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -174,9 +174,6 @@ Deprecation Notices
which got error interrupt to the application,
so that application can reset that particular queue pair.
-* cryptodev: The structure ``rte_cryptodev_qp_conf`` will be updated
- to have a new parameter to set priority of that particular queue pair.
-
* cryptodev: The enum ``rte_crypto_asym_xform_type`` and struct ``rte_crypto_asym_op``
will be extended to include new values to support EDDSA.
This will break ABI compatibility with existing applications that use these data types.
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 0ff70d9057..fcdc738cf6 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -100,6 +100,9 @@ ABI Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* cryptodev: The queue pair configuration structure ``rte_cryptodev_qp_conf``
+ is updated to have a new parameter to set priority of that particular queue pair.
+
Known Issues
------------
diff --git a/lib/cryptodev/rte_cryptodev.h b/lib/cryptodev/rte_cryptodev.h
index bec947f6d5..0a9cd718ea 100644
--- a/lib/cryptodev/rte_cryptodev.h
+++ b/lib/cryptodev/rte_cryptodev.h
@@ -608,12 +608,34 @@ enum rte_cryptodev_event_type {
RTE_CRYPTODEV_EVENT_MAX /**< max value of this enum */
};
+/* Crypto queue pair priority levels */
+#define RTE_CRYPTODEV_QP_PRIORITY_HIGHEST 0
+/**< Highest priority of a cryptodev queue pair
+ * @see rte_cryptodev_queue_pair_setup(), rte_cryptodev_enqueue_burst()
+ */
+#define RTE_CRYPTODEV_QP_PRIORITY_NORMAL 128
+/**< Normal priority of a cryptodev queue pair
+ * @see rte_cryptodev_queue_pair_setup(), rte_cryptodev_enqueue_burst()
+ */
+#define RTE_CRYPTODEV_QP_PRIORITY_LOWEST 255
+/**< Lowest priority of a cryptodev queue pair
+ * @see rte_cryptodev_queue_pair_setup(), rte_cryptodev_enqueue_burst()
+ */
+
/** Crypto device queue pair configuration structure. */
/* Structure rte_cryptodev_qp_conf 8<*/
struct rte_cryptodev_qp_conf {
uint32_t nb_descriptors; /**< Number of descriptors per queue pair */
struct rte_mempool *mp_session;
/**< The mempool for creating session in sessionless mode */
+ uint8_t priority;
+ /**< Priority for this queue pair relative to other queue pairs.
+ *
+ * The requested priority should in the range of
+ * [@ref RTE_CRYPTODEV_QP_PRIORITY_HIGHEST, @ref RTE_CRYPTODEV_QP_PRIORITY_LOWEST].
+ * The implementation may normalize the requested priority to
+ * device supported priority value.
+ */
};
/* >8 End of structure rte_cryptodev_qp_conf. */
--
2.25.1
^ permalink raw reply [relevance 9%]
* RE: [EXT] [PATCH 1/3] cryptodev: add ec points to sm2 op
@ 2024-09-24 16:30 0% ` Akhil Goyal
0 siblings, 0 replies; 200+ results
From: Akhil Goyal @ 2024-09-24 16:30 UTC (permalink / raw)
To: Kusztal, ArkadiuszX, dev
Cc: Kai Ji, Brian Dooley, Gowrishankar Muthukrishnan, Anoob Joseph
Hi Arek,
Can you resend this series if you still want to pursue so that CI can run?
> > > In the case when PMD cannot support full process of the SM2, but
> > > elliptic curve computation only, additional fields are needed to
> > > handle such a case.
> > >
> >
> > Asym crypto APIs are no longer experimental.
> > Hence adding new fields would lead to ABI break.
>
> It seems that
> `__rte_crypto_op_reset`
> `rte_crypto_op_pool_create`
> functions do not need versioning, and we could easily do it if needed.
> But the field `flags` changes an offset, and this is actually problematic.
> Which means that we cannot do this change before 24.11.
>
> >
> > > Points C1, kP therefore were added to the SM2 crypto operation struct.
> > >
> > > Signed-off-by: Arkadiusz Kusztal <arkadiuszx.kusztal@intel.com>
^ permalink raw reply [relevance 0%]
* Re: [PATCH v4] eal: add build-time option to omit trace
2024-09-24 15:53 0% ` Morten Brørup
@ 2024-09-24 15:57 0% ` Jerin Jacob
0 siblings, 0 replies; 200+ results
From: Jerin Jacob @ 2024-09-24 15:57 UTC (permalink / raw)
To: Morten Brørup; +Cc: Jerin Jacob, Sunil Kumar Kori, dev
On Tue, Sep 24, 2024 at 9:23 PM Morten Brørup <mb@smartsharesystems.com> wrote:
>
> > From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> > Sent: Tuesday, 24 September 2024 11.42
> >
> > On Tue, Sep 24, 2024 at 7:10 PM Morten Brørup <mb@smartsharesystems.com>
> > wrote:
> > >
> > > Some applications want to omit the trace feature.
> > > Either to reduce the memory footprint, to reduce the exposed attack
> > > surface, or for other reasons.
> > >
> > > This patch adds an option in rte_config.h to include or omit trace in
> > the
> > > build. Trace is included by default.
> > >
> > > Omitting trace works by omitting all trace points.
> > > For API and ABI compatibility, the trace feature itself remains.
> > >
> > > Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
> > > ---
> > > v4:
> > > * Added check for generic trace enabled when registering trace points,
> > in
> > > RTE_INIT. (Jerin Jacob)
> > > * Test application uses function instead of macro to check if generic
> > > trace is enabled. (Jerin Jacob)
> > > * Performance test application uses function to check if generic trace
> > is
> > > enabled.
> > > v3:
> > > * Simpler version with much fewer ifdefs. (Jerin Jacob)
> > > v2:
> > > * Added/modified macros required for building applications with trace
> > > omitted.
> >
> > >
> > > +/**
> > > + * @internal
> >
> > Since it is used in app/test. Can we remove it as internal and make
> > symbol as rte_trace_point_is_enabled
>
> I don't think we should make functions public if only used for test purposes.
>
> Do you think it is useful for normal usage too? Does rte_trace_is_enabled() not suffice?
It will be good for an app to know, Is trace feature disabled if the
application cares about.
Yes. rte_trace_is_enabled() suffice.
>
> >
> > > + *
> > > + * Test if the tracepoint compile-time option is enabled.
> > > + *
> > > + * @return
> > > + * true if tracepoint enabled, false otherwise.
> > > + */
> > > +__rte_experimental
> > > +static __rte_always_inline bool
> > > +__rte_trace_point_generic_is_enabled(void)
> >
> > Do we need to keep _generic_
>
> Other internal functions have _generic_ too, so I added it.
> If we decide to make it public, I agree _generic_ should be removed.
>
> >
> > Rest looks good to me.
^ permalink raw reply [relevance 0%]
* RE: [PATCH v4] eal: add build-time option to omit trace
2024-09-24 15:41 0% ` Jerin Jacob
@ 2024-09-24 15:53 0% ` Morten Brørup
2024-09-24 15:57 0% ` Jerin Jacob
0 siblings, 1 reply; 200+ results
From: Morten Brørup @ 2024-09-24 15:53 UTC (permalink / raw)
To: Jerin Jacob; +Cc: Jerin Jacob, Sunil Kumar Kori, dev
> From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> Sent: Tuesday, 24 September 2024 11.42
>
> On Tue, Sep 24, 2024 at 7:10 PM Morten Brørup <mb@smartsharesystems.com>
> wrote:
> >
> > Some applications want to omit the trace feature.
> > Either to reduce the memory footprint, to reduce the exposed attack
> > surface, or for other reasons.
> >
> > This patch adds an option in rte_config.h to include or omit trace in
> the
> > build. Trace is included by default.
> >
> > Omitting trace works by omitting all trace points.
> > For API and ABI compatibility, the trace feature itself remains.
> >
> > Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
> > ---
> > v4:
> > * Added check for generic trace enabled when registering trace points,
> in
> > RTE_INIT. (Jerin Jacob)
> > * Test application uses function instead of macro to check if generic
> > trace is enabled. (Jerin Jacob)
> > * Performance test application uses function to check if generic trace
> is
> > enabled.
> > v3:
> > * Simpler version with much fewer ifdefs. (Jerin Jacob)
> > v2:
> > * Added/modified macros required for building applications with trace
> > omitted.
>
> >
> > +/**
> > + * @internal
>
> Since it is used in app/test. Can we remove it as internal and make
> symbol as rte_trace_point_is_enabled
I don't think we should make functions public if only used for test purposes.
Do you think it is useful for normal usage too? Does rte_trace_is_enabled() not suffice?
>
> > + *
> > + * Test if the tracepoint compile-time option is enabled.
> > + *
> > + * @return
> > + * true if tracepoint enabled, false otherwise.
> > + */
> > +__rte_experimental
> > +static __rte_always_inline bool
> > +__rte_trace_point_generic_is_enabled(void)
>
> Do we need to keep _generic_
Other internal functions have _generic_ too, so I added it.
If we decide to make it public, I agree _generic_ should be removed.
>
> Rest looks good to me.
^ permalink raw reply [relevance 0%]
* Re: [PATCH v4] eal: add build-time option to omit trace
2024-09-24 13:39 3% ` [PATCH v4] " Morten Brørup
2024-09-24 15:30 0% ` Stephen Hemminger
@ 2024-09-24 15:41 0% ` Jerin Jacob
2024-09-24 15:53 0% ` Morten Brørup
1 sibling, 1 reply; 200+ results
From: Jerin Jacob @ 2024-09-24 15:41 UTC (permalink / raw)
To: Morten Brørup; +Cc: Jerin Jacob, Sunil Kumar Kori, dev
On Tue, Sep 24, 2024 at 7:10 PM Morten Brørup <mb@smartsharesystems.com> wrote:
>
> Some applications want to omit the trace feature.
> Either to reduce the memory footprint, to reduce the exposed attack
> surface, or for other reasons.
>
> This patch adds an option in rte_config.h to include or omit trace in the
> build. Trace is included by default.
>
> Omitting trace works by omitting all trace points.
> For API and ABI compatibility, the trace feature itself remains.
>
> Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
> ---
> v4:
> * Added check for generic trace enabled when registering trace points, in
> RTE_INIT. (Jerin Jacob)
> * Test application uses function instead of macro to check if generic
> trace is enabled. (Jerin Jacob)
> * Performance test application uses function to check if generic trace is
> enabled.
> v3:
> * Simpler version with much fewer ifdefs. (Jerin Jacob)
> v2:
> * Added/modified macros required for building applications with trace
> omitted.
>
> +/**
> + * @internal
Since it is used in app/test. Can we remove it as internal and make
symbol as rte_trace_point_is_enabled
> + *
> + * Test if the tracepoint compile-time option is enabled.
> + *
> + * @return
> + * true if tracepoint enabled, false otherwise.
> + */
> +__rte_experimental
> +static __rte_always_inline bool
> +__rte_trace_point_generic_is_enabled(void)
Do we need to keep _generic_
Rest looks good to me.
^ permalink raw reply [relevance 0%]
* Re: [PATCH v4] eal: add build-time option to omit trace
2024-09-24 13:39 3% ` [PATCH v4] " Morten Brørup
@ 2024-09-24 15:30 0% ` Stephen Hemminger
2024-09-24 15:41 0% ` Jerin Jacob
1 sibling, 0 replies; 200+ results
From: Stephen Hemminger @ 2024-09-24 15:30 UTC (permalink / raw)
To: Morten Brørup; +Cc: Jerin Jacob, Sunil Kumar Kori, dev
On Tue, 24 Sep 2024 13:39:57 +0000
Morten Brørup <mb@smartsharesystems.com> wrote:
> Some applications want to omit the trace feature.
> Either to reduce the memory footprint, to reduce the exposed attack
> surface, or for other reasons.
>
> This patch adds an option in rte_config.h to include or omit trace in the
> build. Trace is included by default.
>
> Omitting trace works by omitting all trace points.
> For API and ABI compatibility, the trace feature itself remains.
>
> Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
This is good compact solution. In future, it would be nice if DPDK could
use static keys to isolate rarely used features. But static keys require
runtime modification of instructions.
https://docs.kernel.org/6.7/staging/static-keys.html
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
^ permalink raw reply [relevance 0%]
* [PATCH v4] eal: add build-time option to omit trace
2024-09-18 8:55 3% [PATCH] eal: add build-time option to omit trace Morten Brørup
` (2 preceding siblings ...)
2024-09-20 9:08 3% ` [PATCH v3] " Morten Brørup
@ 2024-09-24 13:39 3% ` Morten Brørup
2024-09-24 15:30 0% ` Stephen Hemminger
2024-09-24 15:41 0% ` Jerin Jacob
2024-10-06 12:38 2% ` [PATCH v5] " Morten Brørup
` (3 subsequent siblings)
7 siblings, 2 replies; 200+ results
From: Morten Brørup @ 2024-09-24 13:39 UTC (permalink / raw)
To: Jerin Jacob, Sunil Kumar Kori; +Cc: dev, Morten Brørup
Some applications want to omit the trace feature.
Either to reduce the memory footprint, to reduce the exposed attack
surface, or for other reasons.
This patch adds an option in rte_config.h to include or omit trace in the
build. Trace is included by default.
Omitting trace works by omitting all trace points.
For API and ABI compatibility, the trace feature itself remains.
Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
---
v4:
* Added check for generic trace enabled when registering trace points, in
RTE_INIT. (Jerin Jacob)
* Test application uses function instead of macro to check if generic
trace is enabled. (Jerin Jacob)
* Performance test application uses function to check if generic trace is
enabled.
v3:
* Simpler version with much fewer ifdefs. (Jerin Jacob)
v2:
* Added/modified macros required for building applications with trace
omitted.
---
app/test/test_trace.c | 4 ++++
app/test/test_trace_perf.c | 5 +++++
config/rte_config.h | 1 +
lib/eal/include/rte_trace_point.h | 21 +++++++++++++++++++++
lib/eal/include/rte_trace_point_register.h | 2 ++
5 files changed, 33 insertions(+)
diff --git a/app/test/test_trace.c b/app/test/test_trace.c
index 00809f433b..26656fe024 100644
--- a/app/test/test_trace.c
+++ b/app/test/test_trace.c
@@ -245,6 +245,10 @@ static struct unit_test_suite trace_tests = {
static int
test_trace(void)
{
+ if (!__rte_trace_point_generic_is_enabled()) {
+ printf("Trace omitted at build-time, skipping test\n");
+ return TEST_SKIPPED;
+ }
return unit_test_suite_runner(&trace_tests);
}
diff --git a/app/test/test_trace_perf.c b/app/test/test_trace_perf.c
index 8257cc02be..504574ef8a 100644
--- a/app/test/test_trace_perf.c
+++ b/app/test/test_trace_perf.c
@@ -150,6 +150,11 @@ test_trace_perf(void)
struct test_data *data;
size_t sz;
+ if (!__rte_trace_point_generic_is_enabled()) {
+ printf("Trace omitted at build-time, skipping test\n");
+ return TEST_SKIPPED;
+ }
+
nb_cores = rte_lcore_count();
nb_workers = nb_cores - 1;
if (nb_cores < 2) {
diff --git a/config/rte_config.h b/config/rte_config.h
index dd7bb0d35b..fd6f8a2f1a 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -49,6 +49,7 @@
#define RTE_MAX_TAILQ 32
#define RTE_LOG_DP_LEVEL RTE_LOG_INFO
#define RTE_MAX_VFIO_CONTAINERS 64
+#define RTE_TRACE 1
/* bsd module defines */
#define RTE_CONTIGMEM_MAX_NUM_BUFS 64
diff --git a/lib/eal/include/rte_trace_point.h b/lib/eal/include/rte_trace_point.h
index 41e2a7f99e..b80688ce89 100644
--- a/lib/eal/include/rte_trace_point.h
+++ b/lib/eal/include/rte_trace_point.h
@@ -212,6 +212,25 @@ bool rte_trace_point_is_enabled(rte_trace_point_t *tp);
__rte_experimental
rte_trace_point_t *rte_trace_point_lookup(const char *name);
+/**
+ * @internal
+ *
+ * Test if the tracepoint compile-time option is enabled.
+ *
+ * @return
+ * true if tracepoint enabled, false otherwise.
+ */
+__rte_experimental
+static __rte_always_inline bool
+__rte_trace_point_generic_is_enabled(void)
+{
+#ifdef RTE_TRACE
+ return true;
+#else
+ return false;
+#endif
+}
+
/**
* @internal
*
@@ -359,6 +378,8 @@ __rte_trace_point_emit_ev_header(void *mem, uint64_t in)
#define __rte_trace_point_emit_header_generic(t) \
void *mem; \
do { \
+ if (!__rte_trace_point_generic_is_enabled()) \
+ return; \
const uint64_t val = rte_atomic_load_explicit(t, rte_memory_order_acquire); \
if (likely(!(val & __RTE_TRACE_FIELD_ENABLE_MASK))) \
return; \
diff --git a/lib/eal/include/rte_trace_point_register.h b/lib/eal/include/rte_trace_point_register.h
index 41260e5964..429b993fc2 100644
--- a/lib/eal/include/rte_trace_point_register.h
+++ b/lib/eal/include/rte_trace_point_register.h
@@ -23,6 +23,8 @@ rte_trace_point_t __rte_section("__rte_trace_point") __##trace; \
static const char __##trace##_name[] = RTE_STR(name); \
RTE_INIT(trace##_init) \
{ \
+ if (!__rte_trace_point_generic_is_enabled()) \
+ return; \
__rte_trace_point_register(&__##trace, __##trace##_name, \
(void (*)(void)) trace); \
}
--
2.43.0
^ permalink raw reply [relevance 3%]
* RE: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-09-06 7:54 0% ` Morten Brørup
@ 2024-09-23 20:41 4% ` Akhil Goyal
2024-10-03 7:00 0% ` Akhil Goyal
2024-10-04 4:00 0% ` Ferruh Yigit
0 siblings, 2 replies; 200+ results
From: Akhil Goyal @ 2024-09-23 20:41 UTC (permalink / raw)
To: Morten Brørup, fengchengwen, dev
Cc: thomas, david.marchand, hemant.agrawal, Anoob Joseph,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde,
Nagadheeraj Rottela, mdr
Hi Morten,
Apologies for delayed response.
> Maybe a combination, returning the lowest end of the two versions of the list,
> would work...
>
> ----------------------------------
> Common header file (rte_common.h):
> ----------------------------------
>
> /* Add at end of enum list in the header file. */
> #define RTE_ENUM_LIST_END(name) \
> _ # name # _ENUM_LIST_END /**< @internal */
>
> /* Add somewhere in header file, preferably after the enum list. */
> #define rte_declare_enum_list_end(name) \
> /** @internal */ \
> int _# name # _enum_list_end(void); \
> \
> static int name # _enum_list_end(void) \
> { \
> static int cached = 0; \
> \
> if (likely(cached != 0)) \
> return cached; \
> \
> return cached = RTE_MIN( \
> RTE_ENUM_LIST_END(name), \
> _ # name # _enum_list_end()); \
> } \
> \
> int _# name # _enum_list_end(void)
>
> /* Add in the library/driver implementation. */
> #define rte_define_enum_list_end(name) \
> int _# name # _enum_list_end(void) \
> { \
> return RTE_ENUM_LIST_END(name); \
> } \
> \
> int _# name # _enum_list_end(void)
>
> --------------------
> Library header file:
> --------------------
>
> enum rte_crypto_asym_op_type {
> RTE_CRYPTO_ASYM_OP_VERIFY,
> /**< Signature Verification operation */
> RTE_ENUM_LIST_END(rte_crypto_asym_op)
Will the ABI check be ok for adding anything in between
RTE_CRYPTO_ASYM_OP_VERIFY and RTE_ENUM_LIST_END(rte_crypto_asym_op)?
Don’t we need to add exception for that if we somehow make it internal by adding a comment only?
Library is actually not restricting the application to not use RTE_ENUM_LIST_END(rte_crypto_asym_op) directly.
Also we may need to expose the .c file internal function as experimental in version.map
> }
>
> rte_declare_enum_list_end(rte_crypto_asym_op);
>
> ---------------
> Library C file:
> ---------------
>
> rte_define_enum_list_end(rte_crypto_asym_op);
If we want to make it a generic thing in rte_common.h
Will the below change be ok?
----------------------------------
Common header file (rte_common.h):
----------------------------------
#define rte_define_enum_list_end(name, last_value) \
static inline int name ## _enum_list_end(void) \
{ \
return last_value + 1; \
}
----------------
Lib header file
----------------
//After the enum definition define the list end as below
rte_define_enum_list_end(rte_crypto_asym_op, RTE_CRYPTO_ASYM_OP_VERIFY);
And wherever list end is needed use rte_crypto_asym_op_enum_list_end()?
With this change, abi check will not complain for any new addition at the end of enum.
And we do not need to expose any internal API in version.map.
^ permalink raw reply [relevance 4%]
* Re: [PATCH v3] eal: add build-time option to omit trace
2024-09-20 9:08 3% ` [PATCH v3] " Morten Brørup
@ 2024-09-23 8:39 0% ` Jerin Jacob
0 siblings, 0 replies; 200+ results
From: Jerin Jacob @ 2024-09-23 8:39 UTC (permalink / raw)
To: Morten Brørup; +Cc: Jerin Jacob, Sunil Kumar Kori, dev
On Fri, Sep 20, 2024 at 2:38 PM Morten Brørup <mb@smartsharesystems.com> wrote:
>
> Some applications want to omit the trace feature.
> Either to reduce the memory footprint, to reduce the exposed attack
> surface, or for other reasons.
>
> This patch adds an option in rte_config.h to include or omit trace in the
> build. Trace is included by default.
>
> Omitting trace works by omitting all trace points.
> For API and ABI compatibility, the trace feature itself remains.
>
> Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
> ---
> v3:
> * Simpler version with much fewer ifdefs. (Jerin Jacob)
> v2:
> * Added/modified macros required for building applications with trace
> omitted.
> ---
> app/test/test_trace.c | 11 ++++++++++-
> config/rte_config.h | 1 +
> lib/eal/include/rte_trace_point.h | 21 +++++++++++++++++++++
> 3 files changed, 32 insertions(+), 1 deletion(-)
>
> diff --git a/app/test/test_trace.c b/app/test/test_trace.c
> index 00809f433b..7918cc865d 100644
> --- a/app/test/test_trace.c
> +++ b/app/test/test_trace.c
> @@ -12,7 +12,16 @@
>
> int app_dpdk_test_tp_count;
>
> -#ifdef RTE_EXEC_ENV_WINDOWS
> +#if !defined(RTE_TRACE)
Now that, we are is not disabling any symbols,
This conditional compilation can be removed. Use
__rte_trace_point_generic_is_enabled() in another leg.
> +
> +static int
> +test_trace(void)
> +{
> + printf("trace omitted at build-time, skipping test\n");
> + return TEST_SKIPPED;
> +}
> +
> +#elif defined(RTE_EXEC_ENV_WINDOWS)
>
> static int
> test_trace(void)
> diff --git a/config/rte_config.h b/config/rte_config.h
> index dd7bb0d35b..fd6f8a2f1a 100644
> --- a/config/rte_config.h
> +++ b/config/rte_config.h
> @@ -49,6 +49,7 @@
> #define RTE_MAX_TAILQ 32
> #define RTE_LOG_DP_LEVEL RTE_LOG_INFO
> #define RTE_MAX_VFIO_CONTAINERS 64
> +#define RTE_TRACE 1
>
> /* bsd module defines */
> #define RTE_CONTIGMEM_MAX_NUM_BUFS 64
> diff --git a/lib/eal/include/rte_trace_point.h b/lib/eal/include/rte_trace_point.h
> index 41e2a7f99e..b80688ce89 100644
> --- a/lib/eal/include/rte_trace_point.h
> +++ b/lib/eal/include/rte_trace_point.h
> @@ -212,6 +212,25 @@ bool rte_trace_point_is_enabled(rte_trace_point_t *tp);
> __rte_experimental
> rte_trace_point_t *rte_trace_point_lookup(const char *name);
>
> +/**
> + * @internal
> + *
> + * Test if the tracepoint compile-time option is enabled.
> + *
> + * @return
> + * true if tracepoint enabled, false otherwise.
> + */
> +__rte_experimental
> +static __rte_always_inline bool
> +__rte_trace_point_generic_is_enabled(void)
Add to version.map file
> +{
> +#ifdef RTE_TRACE
> + return true;
> +#else
> + return false;
> +#endif
> +}
> +
> /**
> * @internal
> *
> @@ -359,6 +378,8 @@ __rte_trace_point_emit_ev_header(void *mem, uint64_t in)
> #define __rte_trace_point_emit_header_generic(t) \
> void *mem; \
> do { \
> + if (!__rte_trace_point_generic_is_enabled()) \
> + return; \
To be super safe, Add similar check in
RTE_INIT(trace##_init)
> const uint64_t val = rte_atomic_load_explicit(t, rte_memory_order_acquire); \
> if (likely(!(val & __RTE_TRACE_FIELD_ENABLE_MASK))) \
> return; \
> --
> 2.43.0
>
^ permalink raw reply [relevance 0%]
* [PATCH v6 6/7] ci: test compiler memcpy
@ 2024-09-20 10:27 4% ` Mattias Rönnblom
0 siblings, 0 replies; 200+ results
From: Mattias Rönnblom @ 2024-09-20 10:27 UTC (permalink / raw)
To: dev
Cc: Mattias Rönnblom, Morten Brørup, Stephen Hemminger,
David Marchand, Pavan Nikhilesh, Bruce Richardson,
Mattias Rönnblom
Add compilation tests for the use_cc_memcpy build option.
Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
---
.ci/linux-build.sh | 5 +++++
.github/workflows/build.yml | 7 +++++++
devtools/test-meson-builds.sh | 4 +++-
3 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index 15ed51e4c1..a873f83d09 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -98,6 +98,11 @@ if [ "$STDATOMIC" = "true" ]; then
else
OPTS="$OPTS -Dcheck_includes=true"
fi
+if [ "$CCMEMCPY" = "true" ]; then
+ OPTS="$OPTS -Duse_cc_memcpy=true"
+else
+ OPTS="$OPTS -Duse_cc_memcpy=true"
+fi
if [ "$MINI" = "true" ]; then
OPTS="$OPTS -Denable_drivers=net/null"
OPTS="$OPTS -Ddisable_libs=*"
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index f7d3affbaa..8c52864294 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -31,6 +31,7 @@ jobs:
RISCV64: ${{ matrix.config.cross == 'riscv64' }}
RUN_TESTS: ${{ contains(matrix.config.checks, 'tests') }}
STDATOMIC: ${{ contains(matrix.config.checks, 'stdatomic') }}
+ CCMEMCPY: ${{ contains(matrix.config.checks, 'ccmemcpy') }}
strategy:
fail-fast: false
@@ -45,6 +46,12 @@ jobs:
- os: ubuntu-22.04
compiler: clang
checks: stdatomic
+ - os: ubuntu-22.04
+ compiler: gcc
+ checks: ccmemcpy
+ - os: ubuntu-22.04
+ compiler: clang
+ checks: ccmemcpy
- os: ubuntu-22.04
compiler: gcc
checks: debug+doc+examples+tests
diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index d71bb1ded0..e72146be3b 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -228,12 +228,14 @@ for c in gcc clang ; do
if [ $s = shared ] ; then
abicheck=ABI
stdatomic=-Denable_stdatomic=true
+ ccmemcpy=-Duse_cc_memcpy=true
else
abicheck=skipABI # save time and disk space
stdatomic=-Denable_stdatomic=false
+ ccmemcpy=-Duse_cc_memcpy=false
fi
export CC="$CCACHE $c"
- build build-$c-$s $c $abicheck $stdatomic --default-library=$s
+ build build-$c-$s $c $abicheck $stdatomic $ccmemcpy --default-library=$s
unset CC
done
done
--
2.43.0
^ permalink raw reply [relevance 4%]
* [PATCH v3] eal: add build-time option to omit trace
2024-09-18 8:55 3% [PATCH] eal: add build-time option to omit trace Morten Brørup
2024-09-18 9:49 0% ` Jerin Jacob
2024-09-19 8:06 3% ` [PATCH v2] " Morten Brørup
@ 2024-09-20 9:08 3% ` Morten Brørup
2024-09-23 8:39 0% ` Jerin Jacob
2024-09-24 13:39 3% ` [PATCH v4] " Morten Brørup
` (4 subsequent siblings)
7 siblings, 1 reply; 200+ results
From: Morten Brørup @ 2024-09-20 9:08 UTC (permalink / raw)
To: Jerin Jacob, Sunil Kumar Kori; +Cc: dev, Morten Brørup
Some applications want to omit the trace feature.
Either to reduce the memory footprint, to reduce the exposed attack
surface, or for other reasons.
This patch adds an option in rte_config.h to include or omit trace in the
build. Trace is included by default.
Omitting trace works by omitting all trace points.
For API and ABI compatibility, the trace feature itself remains.
Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
---
v3:
* Simpler version with much fewer ifdefs. (Jerin Jacob)
v2:
* Added/modified macros required for building applications with trace
omitted.
---
app/test/test_trace.c | 11 ++++++++++-
config/rte_config.h | 1 +
lib/eal/include/rte_trace_point.h | 21 +++++++++++++++++++++
3 files changed, 32 insertions(+), 1 deletion(-)
diff --git a/app/test/test_trace.c b/app/test/test_trace.c
index 00809f433b..7918cc865d 100644
--- a/app/test/test_trace.c
+++ b/app/test/test_trace.c
@@ -12,7 +12,16 @@
int app_dpdk_test_tp_count;
-#ifdef RTE_EXEC_ENV_WINDOWS
+#if !defined(RTE_TRACE)
+
+static int
+test_trace(void)
+{
+ printf("trace omitted at build-time, skipping test\n");
+ return TEST_SKIPPED;
+}
+
+#elif defined(RTE_EXEC_ENV_WINDOWS)
static int
test_trace(void)
diff --git a/config/rte_config.h b/config/rte_config.h
index dd7bb0d35b..fd6f8a2f1a 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -49,6 +49,7 @@
#define RTE_MAX_TAILQ 32
#define RTE_LOG_DP_LEVEL RTE_LOG_INFO
#define RTE_MAX_VFIO_CONTAINERS 64
+#define RTE_TRACE 1
/* bsd module defines */
#define RTE_CONTIGMEM_MAX_NUM_BUFS 64
diff --git a/lib/eal/include/rte_trace_point.h b/lib/eal/include/rte_trace_point.h
index 41e2a7f99e..b80688ce89 100644
--- a/lib/eal/include/rte_trace_point.h
+++ b/lib/eal/include/rte_trace_point.h
@@ -212,6 +212,25 @@ bool rte_trace_point_is_enabled(rte_trace_point_t *tp);
__rte_experimental
rte_trace_point_t *rte_trace_point_lookup(const char *name);
+/**
+ * @internal
+ *
+ * Test if the tracepoint compile-time option is enabled.
+ *
+ * @return
+ * true if tracepoint enabled, false otherwise.
+ */
+__rte_experimental
+static __rte_always_inline bool
+__rte_trace_point_generic_is_enabled(void)
+{
+#ifdef RTE_TRACE
+ return true;
+#else
+ return false;
+#endif
+}
+
/**
* @internal
*
@@ -359,6 +378,8 @@ __rte_trace_point_emit_ev_header(void *mem, uint64_t in)
#define __rte_trace_point_emit_header_generic(t) \
void *mem; \
do { \
+ if (!__rte_trace_point_generic_is_enabled()) \
+ return; \
const uint64_t val = rte_atomic_load_explicit(t, rte_memory_order_acquire); \
if (likely(!(val & __RTE_TRACE_FIELD_ENABLE_MASK))) \
return; \
--
2.43.0
^ permalink raw reply [relevance 3%]
* Re: [PATCH v2] eal: add build-time option to omit trace
2024-09-19 13:15 0% ` Morten Brørup
@ 2024-09-19 13:54 0% ` Jerin Jacob
0 siblings, 0 replies; 200+ results
From: Jerin Jacob @ 2024-09-19 13:54 UTC (permalink / raw)
To: Morten Brørup
Cc: Jerin Jacob, Sunil Kumar Kori, dev, techboard, Ferruh Yigit
On Thu, Sep 19, 2024 at 6:45 PM Morten Brørup <mb@smartsharesystems.com> wrote:
>
> > From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> > Sent: Thursday, 19 September 2024 10.52
> >
> > On Thu, Sep 19, 2024 at 1:37 PM Morten Brørup <mb@smartsharesystems.com>
> > wrote:
> > >
> > > Some applications want to omit the trace feature.
> > > Either to reduce the memory footprint, to reduce the exposed attack
> > > surface, or for other reasons.
> > >
> > > This patch adds an option in rte_config.h to include or omit trace in the
> > > build. Trace is included by default.
> > >
> > > Omitting trace works by omitting all trace points.
> > > For API and ABI compatibility, the trace feature itself remains.
> > >
> > > Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
> > > ---
> > > v2:
> > > * Added/modified macros required for building applications with trace
> > > omitted.
> > > ---
> > > app/test/test_trace.c | 11 ++++++++++-
> > > config/rte_config.h | 1 +
> > > lib/eal/include/rte_trace_point.h | 22 +++++++++++++++++++++-
> > > lib/eal/include/rte_trace_point_register.h | 17 +++++++++++++++++
> > > 4 files changed, 49 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/app/test/test_trace.c b/app/test/test_trace.c
> > > index 00809f433b..7918cc865d 100644
> > > --- a/app/test/test_trace.c
> > > +++ b/app/test/test_trace.c
> > > @@ -12,7 +12,16 @@
> > >
> > > int app_dpdk_test_tp_count;
> > >
> > > -#ifdef RTE_EXEC_ENV_WINDOWS
> > > +#if !defined(RTE_TRACE)
> > > +
> > > +static int
> > > +test_trace(void)
> > > +{
> > > + printf("trace omitted at build-time, skipping test\n");
> > > + return TEST_SKIPPED;
> > > +}
> > > +
> > > +#elif defined(RTE_EXEC_ENV_WINDOWS)
> > >
> > > static int
> > > test_trace(void)
> > > diff --git a/config/rte_config.h b/config/rte_config.h
> > > index dd7bb0d35b..fd6f8a2f1a 100644
> > > --- a/config/rte_config.h
> > > +++ b/config/rte_config.h
> > > @@ -49,6 +49,7 @@
> > > #define RTE_MAX_TAILQ 32
> > > #define RTE_LOG_DP_LEVEL RTE_LOG_INFO
> > > #define RTE_MAX_VFIO_CONTAINERS 64
> > > +#define RTE_TRACE 1
> >
> >
> > Too many ifdefs down using this, IMO, This approach will make
> > difficult to manage the code and make sure
> > RTE_TRACE 0 works always.
>
> I agree it's not pretty, but it's the best solution I can come up with, to meet the requirements driving this patch: reducing attack surface and memory footprint.
>
> >
> > I understand your concern about to reduce the expose attach surface.
> > BTW, it will be there with telemetry too.
>
> Yes, I have telemetry in mind too. :-)
> Got to work on them one by one, though.
>
> > Disabling full feature for the code via config/rte_config.h is new to
> > DPDK. I think, we need wider opinion on this(Adding techboard).
>
> Yes, it’s a good topic for discussion in the techboard.
>
> Currently, there are three levels of build-time configuration:
> 1. Meson, for prominent options.
> 2. config/rte_config.h for less prominent options, e.g. enabling RTE_ASSERT(), mempool statistics, etc.
> 3. Header files in various modules/drivers, for tweaking the individual module.
>
> There didn't seem to be any interest in enabling/disabling trace via Meson, so I chose the next level of configuration.
>
> > Also, DPDK never concerned with binary size.
>
> That is a big mistake. DPDK is not only for cloud.
> DPDK should also be for CPEs, SOHO firewalls, and similar network appliances. For some of these high-volume network appliances, memory footprint matters.
Could you share data with a real-world application of the elf size?
1)Without any change
2)Only disabling via __rte_trace_point_emit_header_generic() .. aka below patch.
3)Full disable.
I think, size command will spit out with different section's size.
This data can be used to take decision and to know how much % it
adding up?
>
> Also, when running many instances of a service in the cloud, power consumption matters. Bloat increases power consumption.
>
> > Other than binary size,
> > following patch and exposed the config through meson
> > will fix the other problems. IMO, That's the correct tradeoff.
>
> Yes; the optimal solution would be making EAL less bloated, i.e. purely an abstraction layer to the underlying hardware and O/S.
> This would make trace an independent module, which could be enabled/disabled through Meson.
>
> Bruce has contributed a lot in this area, and already made many modules optional.
>
> >
> >
> > #define __rte_trace_point_emit_header_generic(t) \
> > void *mem; \
> > do { \
> > + if (RTE_TRACE == 0) \
> > + return \
> > const uint64_t val = rte_atomic_load_explicit(t,
> > rte_memory_order_acquire); \
> > if (likely(!(val & __RTE_TRACE_FIELD_ENABLE_MASK))) \
> > return; \
> > mem = __rte_trace_mem_get(val); \
> > if (unlikely(mem == NULL)) \
> > return; \
> > mem = __rte_trace_point_emit_ev_header(mem, val); \
> > } while (0)
> >
> > >
> > > /* bsd module defines */
> > > #define RTE_CONTIGMEM_MAX_NUM_BUFS 64
> > > diff --git a/lib/eal/include/rte_trace_point.h
> > b/lib/eal/include/rte_trace_point.h
> > > index 41e2a7f99e..fd864125a7 100644
> > > --- a/lib/eal/include/rte_trace_point.h
> > > +++ b/lib/eal/include/rte_trace_point.h
> > > @@ -42,6 +42,8 @@ typedef RTE_ATOMIC(uint64_t) rte_trace_point_t;
> > > */
> > > #define RTE_TRACE_POINT_ARGS
> > >
> > > +#ifdef RTE_TRACE
> > > +
> > > /** @internal Helper macro to support RTE_TRACE_POINT and
> > RTE_TRACE_POINT_FP */
> > > #define __RTE_TRACE_POINT(_mode, _tp, _args, ...) \
> > > extern rte_trace_point_t __##_tp; \
> > > @@ -100,6 +102,20 @@ _tp _args \
> > > #define RTE_TRACE_POINT_FP(tp, args, ...) \
> > > __RTE_TRACE_POINT(fp, tp, args, __VA_ARGS__)
> > >
> > > +#else
> > > +
> > > +/** @internal Helper macro to support RTE_TRACE_POINT and
> > RTE_TRACE_POINT_FP */
> > > +#define __RTE_TRACE_POINT_VOID(_tp, ...) \
> > > +static __rte_always_inline void \
> > > +_tp(...) \
> > > +{ \
> > > +}
> > > +
> > > +#define RTE_TRACE_POINT(tp, args, ...) __RTE_TRACE_POINT_VOID(tp, args)
> > > +#define RTE_TRACE_POINT_FP(tp, args, ...) __RTE_TRACE_POINT_VOID(tp, args)
> > > +
> > > +#endif /* RTE_TRACE */
> > > +
> > > #ifdef __DOXYGEN__
> > >
> > > /**
> > > @@ -212,6 +228,7 @@ bool rte_trace_point_is_enabled(rte_trace_point_t *tp);
> > > __rte_experimental
> > > rte_trace_point_t *rte_trace_point_lookup(const char *name);
> > >
> > > +#ifdef RTE_TRACE
> > > /**
> > > * @internal
> > > *
> > > @@ -230,6 +247,7 @@ __rte_trace_point_fp_is_enabled(void)
> > > return false;
> > > #endif
> > > }
> > > +#endif /* RTE_TRACE */
> > >
> > > /**
> > > * @internal
> > > @@ -356,6 +374,8 @@ __rte_trace_point_emit_ev_header(void *mem, uint64_t in)
> > > return RTE_PTR_ADD(mem, __RTE_TRACE_EVENT_HEADER_SZ);
> > > }
> > >
> > > +#ifdef RTE_TRACE
> > > +
> > > #define __rte_trace_point_emit_header_generic(t) \
> > > void *mem; \
> > > do { \
> > > @@ -411,7 +431,7 @@ do { \
> > > RTE_SET_USED(len); \
> > > } while (0)
> > >
> > > -
> > > +#endif /* RTE_TRACE */
> > > #endif /* ALLOW_EXPERIMENTAL_API */
> > > #endif /* _RTE_TRACE_POINT_REGISTER_H_ */
> > >
> > > diff --git a/lib/eal/include/rte_trace_point_register.h
> > b/lib/eal/include/rte_trace_point_register.h
> > > index 41260e5964..567fce534e 100644
> > > --- a/lib/eal/include/rte_trace_point_register.h
> > > +++ b/lib/eal/include/rte_trace_point_register.h
> > > @@ -18,6 +18,8 @@ extern "C" {
> > >
> > > RTE_DECLARE_PER_LCORE(volatile int, trace_point_sz);
> > >
> > > +#ifdef RTE_TRACE
> > > +
> > > #define RTE_TRACE_POINT_REGISTER(trace, name) \
> > > rte_trace_point_t __rte_section("__rte_trace_point") __##trace; \
> > > static const char __##trace##_name[] = RTE_STR(name); \
> > > @@ -56,6 +58,21 @@ do { \
> > > RTE_STR(uint8_t)); \
> > > } while (0)
> > >
> > > +#else
> > > +
> > > +#define RTE_TRACE_POINT_REGISTER(...)
> > > +#define __rte_trace_point_emit_header_generic(t) RTE_SET_USED(t)
> > > +#define __rte_trace_point_emit_header_fp(t) RTE_SET_USED(t)
> > > +#define __rte_trace_point_emit(in, type) RTE_SET_USED(in)
> > > +#define rte_trace_point_emit_string(in) RTE_SET_USED(in)
> > > +#define rte_trace_point_emit_blob(in, len) \
> > > +do { \
> > > + RTE_SET_USED(in); \
> > > + RTE_SET_USED(len); \
> > > +} while (0)
> > > +
> > > +#endif /* RTE_TRACE */
> > > +
> > > #ifdef __cplusplus
> > > }
> > > #endif
> > > --
> > > 2.43.0
> > >
^ permalink raw reply [relevance 0%]
* RE: [PATCH v2] eal: add build-time option to omit trace
2024-09-19 8:51 0% ` Jerin Jacob
@ 2024-09-19 13:15 0% ` Morten Brørup
2024-09-19 13:54 0% ` Jerin Jacob
0 siblings, 1 reply; 200+ results
From: Morten Brørup @ 2024-09-19 13:15 UTC (permalink / raw)
To: Jerin Jacob; +Cc: Jerin Jacob, Sunil Kumar Kori, dev, techboard, Ferruh Yigit
> From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> Sent: Thursday, 19 September 2024 10.52
>
> On Thu, Sep 19, 2024 at 1:37 PM Morten Brørup <mb@smartsharesystems.com>
> wrote:
> >
> > Some applications want to omit the trace feature.
> > Either to reduce the memory footprint, to reduce the exposed attack
> > surface, or for other reasons.
> >
> > This patch adds an option in rte_config.h to include or omit trace in the
> > build. Trace is included by default.
> >
> > Omitting trace works by omitting all trace points.
> > For API and ABI compatibility, the trace feature itself remains.
> >
> > Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
> > ---
> > v2:
> > * Added/modified macros required for building applications with trace
> > omitted.
> > ---
> > app/test/test_trace.c | 11 ++++++++++-
> > config/rte_config.h | 1 +
> > lib/eal/include/rte_trace_point.h | 22 +++++++++++++++++++++-
> > lib/eal/include/rte_trace_point_register.h | 17 +++++++++++++++++
> > 4 files changed, 49 insertions(+), 2 deletions(-)
> >
> > diff --git a/app/test/test_trace.c b/app/test/test_trace.c
> > index 00809f433b..7918cc865d 100644
> > --- a/app/test/test_trace.c
> > +++ b/app/test/test_trace.c
> > @@ -12,7 +12,16 @@
> >
> > int app_dpdk_test_tp_count;
> >
> > -#ifdef RTE_EXEC_ENV_WINDOWS
> > +#if !defined(RTE_TRACE)
> > +
> > +static int
> > +test_trace(void)
> > +{
> > + printf("trace omitted at build-time, skipping test\n");
> > + return TEST_SKIPPED;
> > +}
> > +
> > +#elif defined(RTE_EXEC_ENV_WINDOWS)
> >
> > static int
> > test_trace(void)
> > diff --git a/config/rte_config.h b/config/rte_config.h
> > index dd7bb0d35b..fd6f8a2f1a 100644
> > --- a/config/rte_config.h
> > +++ b/config/rte_config.h
> > @@ -49,6 +49,7 @@
> > #define RTE_MAX_TAILQ 32
> > #define RTE_LOG_DP_LEVEL RTE_LOG_INFO
> > #define RTE_MAX_VFIO_CONTAINERS 64
> > +#define RTE_TRACE 1
>
>
> Too many ifdefs down using this, IMO, This approach will make
> difficult to manage the code and make sure
> RTE_TRACE 0 works always.
I agree it's not pretty, but it's the best solution I can come up with, to meet the requirements driving this patch: reducing attack surface and memory footprint.
>
> I understand your concern about to reduce the expose attach surface.
> BTW, it will be there with telemetry too.
Yes, I have telemetry in mind too. :-)
Got to work on them one by one, though.
> Disabling full feature for the code via config/rte_config.h is new to
> DPDK. I think, we need wider opinion on this(Adding techboard).
Yes, it’s a good topic for discussion in the techboard.
Currently, there are three levels of build-time configuration:
1. Meson, for prominent options.
2. config/rte_config.h for less prominent options, e.g. enabling RTE_ASSERT(), mempool statistics, etc.
3. Header files in various modules/drivers, for tweaking the individual module.
There didn't seem to be any interest in enabling/disabling trace via Meson, so I chose the next level of configuration.
> Also, DPDK never concerned with binary size.
That is a big mistake. DPDK is not only for cloud.
DPDK should also be for CPEs, SOHO firewalls, and similar network appliances. For some of these high-volume network appliances, memory footprint matters.
Also, when running many instances of a service in the cloud, power consumption matters. Bloat increases power consumption.
> Other than binary size,
> following patch and exposed the config through meson
> will fix the other problems. IMO, That's the correct tradeoff.
Yes; the optimal solution would be making EAL less bloated, i.e. purely an abstraction layer to the underlying hardware and O/S.
This would make trace an independent module, which could be enabled/disabled through Meson.
Bruce has contributed a lot in this area, and already made many modules optional.
>
>
> #define __rte_trace_point_emit_header_generic(t) \
> void *mem; \
> do { \
> + if (RTE_TRACE == 0) \
> + return \
> const uint64_t val = rte_atomic_load_explicit(t,
> rte_memory_order_acquire); \
> if (likely(!(val & __RTE_TRACE_FIELD_ENABLE_MASK))) \
> return; \
> mem = __rte_trace_mem_get(val); \
> if (unlikely(mem == NULL)) \
> return; \
> mem = __rte_trace_point_emit_ev_header(mem, val); \
> } while (0)
>
> >
> > /* bsd module defines */
> > #define RTE_CONTIGMEM_MAX_NUM_BUFS 64
> > diff --git a/lib/eal/include/rte_trace_point.h
> b/lib/eal/include/rte_trace_point.h
> > index 41e2a7f99e..fd864125a7 100644
> > --- a/lib/eal/include/rte_trace_point.h
> > +++ b/lib/eal/include/rte_trace_point.h
> > @@ -42,6 +42,8 @@ typedef RTE_ATOMIC(uint64_t) rte_trace_point_t;
> > */
> > #define RTE_TRACE_POINT_ARGS
> >
> > +#ifdef RTE_TRACE
> > +
> > /** @internal Helper macro to support RTE_TRACE_POINT and
> RTE_TRACE_POINT_FP */
> > #define __RTE_TRACE_POINT(_mode, _tp, _args, ...) \
> > extern rte_trace_point_t __##_tp; \
> > @@ -100,6 +102,20 @@ _tp _args \
> > #define RTE_TRACE_POINT_FP(tp, args, ...) \
> > __RTE_TRACE_POINT(fp, tp, args, __VA_ARGS__)
> >
> > +#else
> > +
> > +/** @internal Helper macro to support RTE_TRACE_POINT and
> RTE_TRACE_POINT_FP */
> > +#define __RTE_TRACE_POINT_VOID(_tp, ...) \
> > +static __rte_always_inline void \
> > +_tp(...) \
> > +{ \
> > +}
> > +
> > +#define RTE_TRACE_POINT(tp, args, ...) __RTE_TRACE_POINT_VOID(tp, args)
> > +#define RTE_TRACE_POINT_FP(tp, args, ...) __RTE_TRACE_POINT_VOID(tp, args)
> > +
> > +#endif /* RTE_TRACE */
> > +
> > #ifdef __DOXYGEN__
> >
> > /**
> > @@ -212,6 +228,7 @@ bool rte_trace_point_is_enabled(rte_trace_point_t *tp);
> > __rte_experimental
> > rte_trace_point_t *rte_trace_point_lookup(const char *name);
> >
> > +#ifdef RTE_TRACE
> > /**
> > * @internal
> > *
> > @@ -230,6 +247,7 @@ __rte_trace_point_fp_is_enabled(void)
> > return false;
> > #endif
> > }
> > +#endif /* RTE_TRACE */
> >
> > /**
> > * @internal
> > @@ -356,6 +374,8 @@ __rte_trace_point_emit_ev_header(void *mem, uint64_t in)
> > return RTE_PTR_ADD(mem, __RTE_TRACE_EVENT_HEADER_SZ);
> > }
> >
> > +#ifdef RTE_TRACE
> > +
> > #define __rte_trace_point_emit_header_generic(t) \
> > void *mem; \
> > do { \
> > @@ -411,7 +431,7 @@ do { \
> > RTE_SET_USED(len); \
> > } while (0)
> >
> > -
> > +#endif /* RTE_TRACE */
> > #endif /* ALLOW_EXPERIMENTAL_API */
> > #endif /* _RTE_TRACE_POINT_REGISTER_H_ */
> >
> > diff --git a/lib/eal/include/rte_trace_point_register.h
> b/lib/eal/include/rte_trace_point_register.h
> > index 41260e5964..567fce534e 100644
> > --- a/lib/eal/include/rte_trace_point_register.h
> > +++ b/lib/eal/include/rte_trace_point_register.h
> > @@ -18,6 +18,8 @@ extern "C" {
> >
> > RTE_DECLARE_PER_LCORE(volatile int, trace_point_sz);
> >
> > +#ifdef RTE_TRACE
> > +
> > #define RTE_TRACE_POINT_REGISTER(trace, name) \
> > rte_trace_point_t __rte_section("__rte_trace_point") __##trace; \
> > static const char __##trace##_name[] = RTE_STR(name); \
> > @@ -56,6 +58,21 @@ do { \
> > RTE_STR(uint8_t)); \
> > } while (0)
> >
> > +#else
> > +
> > +#define RTE_TRACE_POINT_REGISTER(...)
> > +#define __rte_trace_point_emit_header_generic(t) RTE_SET_USED(t)
> > +#define __rte_trace_point_emit_header_fp(t) RTE_SET_USED(t)
> > +#define __rte_trace_point_emit(in, type) RTE_SET_USED(in)
> > +#define rte_trace_point_emit_string(in) RTE_SET_USED(in)
> > +#define rte_trace_point_emit_blob(in, len) \
> > +do { \
> > + RTE_SET_USED(in); \
> > + RTE_SET_USED(len); \
> > +} while (0)
> > +
> > +#endif /* RTE_TRACE */
> > +
> > #ifdef __cplusplus
> > }
> > #endif
> > --
> > 2.43.0
> >
^ permalink raw reply [relevance 0%]
* Re: [PATCH v2] eal: add build-time option to omit trace
2024-09-19 8:06 3% ` [PATCH v2] " Morten Brørup
@ 2024-09-19 8:51 0% ` Jerin Jacob
2024-09-19 13:15 0% ` Morten Brørup
0 siblings, 1 reply; 200+ results
From: Jerin Jacob @ 2024-09-19 8:51 UTC (permalink / raw)
To: Morten Brørup
Cc: Jerin Jacob, Sunil Kumar Kori, dev, techboard, Ferruh Yigit
On Thu, Sep 19, 2024 at 1:37 PM Morten Brørup <mb@smartsharesystems.com> wrote:
>
> Some applications want to omit the trace feature.
> Either to reduce the memory footprint, to reduce the exposed attack
> surface, or for other reasons.
>
> This patch adds an option in rte_config.h to include or omit trace in the
> build. Trace is included by default.
>
> Omitting trace works by omitting all trace points.
> For API and ABI compatibility, the trace feature itself remains.
>
> Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
> ---
> v2:
> * Added/modified macros required for building applications with trace
> omitted.
> ---
> app/test/test_trace.c | 11 ++++++++++-
> config/rte_config.h | 1 +
> lib/eal/include/rte_trace_point.h | 22 +++++++++++++++++++++-
> lib/eal/include/rte_trace_point_register.h | 17 +++++++++++++++++
> 4 files changed, 49 insertions(+), 2 deletions(-)
>
> diff --git a/app/test/test_trace.c b/app/test/test_trace.c
> index 00809f433b..7918cc865d 100644
> --- a/app/test/test_trace.c
> +++ b/app/test/test_trace.c
> @@ -12,7 +12,16 @@
>
> int app_dpdk_test_tp_count;
>
> -#ifdef RTE_EXEC_ENV_WINDOWS
> +#if !defined(RTE_TRACE)
> +
> +static int
> +test_trace(void)
> +{
> + printf("trace omitted at build-time, skipping test\n");
> + return TEST_SKIPPED;
> +}
> +
> +#elif defined(RTE_EXEC_ENV_WINDOWS)
>
> static int
> test_trace(void)
> diff --git a/config/rte_config.h b/config/rte_config.h
> index dd7bb0d35b..fd6f8a2f1a 100644
> --- a/config/rte_config.h
> +++ b/config/rte_config.h
> @@ -49,6 +49,7 @@
> #define RTE_MAX_TAILQ 32
> #define RTE_LOG_DP_LEVEL RTE_LOG_INFO
> #define RTE_MAX_VFIO_CONTAINERS 64
> +#define RTE_TRACE 1
Too many ifdefs down using this, IMO, This approach will make
difficult to manage the code and make sure
RTE_TRACE 0 works always.
I understand your concern about to reduce the expose attach surface.
BTW, it will be there with telemetry too.
Disabling full feature for the code via config/rte_config.h is new to
DPDK. I think, we need wider opinion on this(Adding techboard).
Also, DPDK never concerned with binary size. Other than binary size,
following patch and exposed the config through meson
will fix the other problems. IMO, That's the correct tradeoff.
#define __rte_trace_point_emit_header_generic(t) \
void *mem; \
do { \
+ if (RTE_TRACE == 0) \
+ return \
const uint64_t val = rte_atomic_load_explicit(t,
rte_memory_order_acquire); \
if (likely(!(val & __RTE_TRACE_FIELD_ENABLE_MASK))) \
return; \
mem = __rte_trace_mem_get(val); \
if (unlikely(mem == NULL)) \
return; \
mem = __rte_trace_point_emit_ev_header(mem, val); \
} while (0)
>
> /* bsd module defines */
> #define RTE_CONTIGMEM_MAX_NUM_BUFS 64
> diff --git a/lib/eal/include/rte_trace_point.h b/lib/eal/include/rte_trace_point.h
> index 41e2a7f99e..fd864125a7 100644
> --- a/lib/eal/include/rte_trace_point.h
> +++ b/lib/eal/include/rte_trace_point.h
> @@ -42,6 +42,8 @@ typedef RTE_ATOMIC(uint64_t) rte_trace_point_t;
> */
> #define RTE_TRACE_POINT_ARGS
>
> +#ifdef RTE_TRACE
> +
> /** @internal Helper macro to support RTE_TRACE_POINT and RTE_TRACE_POINT_FP */
> #define __RTE_TRACE_POINT(_mode, _tp, _args, ...) \
> extern rte_trace_point_t __##_tp; \
> @@ -100,6 +102,20 @@ _tp _args \
> #define RTE_TRACE_POINT_FP(tp, args, ...) \
> __RTE_TRACE_POINT(fp, tp, args, __VA_ARGS__)
>
> +#else
> +
> +/** @internal Helper macro to support RTE_TRACE_POINT and RTE_TRACE_POINT_FP */
> +#define __RTE_TRACE_POINT_VOID(_tp, ...) \
> +static __rte_always_inline void \
> +_tp(...) \
> +{ \
> +}
> +
> +#define RTE_TRACE_POINT(tp, args, ...) __RTE_TRACE_POINT_VOID(tp, args)
> +#define RTE_TRACE_POINT_FP(tp, args, ...) __RTE_TRACE_POINT_VOID(tp, args)
> +
> +#endif /* RTE_TRACE */
> +
> #ifdef __DOXYGEN__
>
> /**
> @@ -212,6 +228,7 @@ bool rte_trace_point_is_enabled(rte_trace_point_t *tp);
> __rte_experimental
> rte_trace_point_t *rte_trace_point_lookup(const char *name);
>
> +#ifdef RTE_TRACE
> /**
> * @internal
> *
> @@ -230,6 +247,7 @@ __rte_trace_point_fp_is_enabled(void)
> return false;
> #endif
> }
> +#endif /* RTE_TRACE */
>
> /**
> * @internal
> @@ -356,6 +374,8 @@ __rte_trace_point_emit_ev_header(void *mem, uint64_t in)
> return RTE_PTR_ADD(mem, __RTE_TRACE_EVENT_HEADER_SZ);
> }
>
> +#ifdef RTE_TRACE
> +
> #define __rte_trace_point_emit_header_generic(t) \
> void *mem; \
> do { \
> @@ -411,7 +431,7 @@ do { \
> RTE_SET_USED(len); \
> } while (0)
>
> -
> +#endif /* RTE_TRACE */
> #endif /* ALLOW_EXPERIMENTAL_API */
> #endif /* _RTE_TRACE_POINT_REGISTER_H_ */
>
> diff --git a/lib/eal/include/rte_trace_point_register.h b/lib/eal/include/rte_trace_point_register.h
> index 41260e5964..567fce534e 100644
> --- a/lib/eal/include/rte_trace_point_register.h
> +++ b/lib/eal/include/rte_trace_point_register.h
> @@ -18,6 +18,8 @@ extern "C" {
>
> RTE_DECLARE_PER_LCORE(volatile int, trace_point_sz);
>
> +#ifdef RTE_TRACE
> +
> #define RTE_TRACE_POINT_REGISTER(trace, name) \
> rte_trace_point_t __rte_section("__rte_trace_point") __##trace; \
> static const char __##trace##_name[] = RTE_STR(name); \
> @@ -56,6 +58,21 @@ do { \
> RTE_STR(uint8_t)); \
> } while (0)
>
> +#else
> +
> +#define RTE_TRACE_POINT_REGISTER(...)
> +#define __rte_trace_point_emit_header_generic(t) RTE_SET_USED(t)
> +#define __rte_trace_point_emit_header_fp(t) RTE_SET_USED(t)
> +#define __rte_trace_point_emit(in, type) RTE_SET_USED(in)
> +#define rte_trace_point_emit_string(in) RTE_SET_USED(in)
> +#define rte_trace_point_emit_blob(in, len) \
> +do { \
> + RTE_SET_USED(in); \
> + RTE_SET_USED(len); \
> +} while (0)
> +
> +#endif /* RTE_TRACE */
> +
> #ifdef __cplusplus
> }
> #endif
> --
> 2.43.0
>
^ permalink raw reply [relevance 0%]
* Re: [PATCH v3 1/1] bbdev: removing unnecessary symbols from version map
2024-08-27 23:06 3% ` [PATCH v3 1/1] bbdev: removing unnecessary symbols from version map Nicolas Chautru
2024-09-13 13:51 0% ` Maxime Coquelin
@ 2024-09-19 8:26 0% ` Maxime Coquelin
1 sibling, 0 replies; 200+ results
From: Maxime Coquelin @ 2024-09-19 8:26 UTC (permalink / raw)
To: Nicolas Chautru, dev; +Cc: hemant.agrawal, david.marchand, hernan.vargas
On 8/28/24 01:06, Nicolas Chautru wrote:
> A number of inline functions should not be in the
> version map since ABI versioning would be irrelevant.
>
> Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
> ---
> lib/bbdev/rte_bbdev.h | 1 -
> lib/bbdev/rte_bbdev_op.h | 2 --
> lib/bbdev/version.map | 24 +-----------------------
> 3 files changed, 1 insertion(+), 26 deletions(-)
>
Applied to next-baseband/for-main.
Thanks,
Maxime
^ permalink raw reply [relevance 0%]
* [PATCH v2] eal: add build-time option to omit trace
2024-09-18 8:55 3% [PATCH] eal: add build-time option to omit trace Morten Brørup
2024-09-18 9:49 0% ` Jerin Jacob
@ 2024-09-19 8:06 3% ` Morten Brørup
2024-09-19 8:51 0% ` Jerin Jacob
2024-09-20 9:08 3% ` [PATCH v3] " Morten Brørup
` (5 subsequent siblings)
7 siblings, 1 reply; 200+ results
From: Morten Brørup @ 2024-09-19 8:06 UTC (permalink / raw)
To: Jerin Jacob, Sunil Kumar Kori; +Cc: dev, Morten Brørup
Some applications want to omit the trace feature.
Either to reduce the memory footprint, to reduce the exposed attack
surface, or for other reasons.
This patch adds an option in rte_config.h to include or omit trace in the
build. Trace is included by default.
Omitting trace works by omitting all trace points.
For API and ABI compatibility, the trace feature itself remains.
Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
---
v2:
* Added/modified macros required for building applications with trace
omitted.
---
app/test/test_trace.c | 11 ++++++++++-
config/rte_config.h | 1 +
lib/eal/include/rte_trace_point.h | 22 +++++++++++++++++++++-
lib/eal/include/rte_trace_point_register.h | 17 +++++++++++++++++
4 files changed, 49 insertions(+), 2 deletions(-)
diff --git a/app/test/test_trace.c b/app/test/test_trace.c
index 00809f433b..7918cc865d 100644
--- a/app/test/test_trace.c
+++ b/app/test/test_trace.c
@@ -12,7 +12,16 @@
int app_dpdk_test_tp_count;
-#ifdef RTE_EXEC_ENV_WINDOWS
+#if !defined(RTE_TRACE)
+
+static int
+test_trace(void)
+{
+ printf("trace omitted at build-time, skipping test\n");
+ return TEST_SKIPPED;
+}
+
+#elif defined(RTE_EXEC_ENV_WINDOWS)
static int
test_trace(void)
diff --git a/config/rte_config.h b/config/rte_config.h
index dd7bb0d35b..fd6f8a2f1a 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -49,6 +49,7 @@
#define RTE_MAX_TAILQ 32
#define RTE_LOG_DP_LEVEL RTE_LOG_INFO
#define RTE_MAX_VFIO_CONTAINERS 64
+#define RTE_TRACE 1
/* bsd module defines */
#define RTE_CONTIGMEM_MAX_NUM_BUFS 64
diff --git a/lib/eal/include/rte_trace_point.h b/lib/eal/include/rte_trace_point.h
index 41e2a7f99e..fd864125a7 100644
--- a/lib/eal/include/rte_trace_point.h
+++ b/lib/eal/include/rte_trace_point.h
@@ -42,6 +42,8 @@ typedef RTE_ATOMIC(uint64_t) rte_trace_point_t;
*/
#define RTE_TRACE_POINT_ARGS
+#ifdef RTE_TRACE
+
/** @internal Helper macro to support RTE_TRACE_POINT and RTE_TRACE_POINT_FP */
#define __RTE_TRACE_POINT(_mode, _tp, _args, ...) \
extern rte_trace_point_t __##_tp; \
@@ -100,6 +102,20 @@ _tp _args \
#define RTE_TRACE_POINT_FP(tp, args, ...) \
__RTE_TRACE_POINT(fp, tp, args, __VA_ARGS__)
+#else
+
+/** @internal Helper macro to support RTE_TRACE_POINT and RTE_TRACE_POINT_FP */
+#define __RTE_TRACE_POINT_VOID(_tp, ...) \
+static __rte_always_inline void \
+_tp(...) \
+{ \
+}
+
+#define RTE_TRACE_POINT(tp, args, ...) __RTE_TRACE_POINT_VOID(tp, args)
+#define RTE_TRACE_POINT_FP(tp, args, ...) __RTE_TRACE_POINT_VOID(tp, args)
+
+#endif /* RTE_TRACE */
+
#ifdef __DOXYGEN__
/**
@@ -212,6 +228,7 @@ bool rte_trace_point_is_enabled(rte_trace_point_t *tp);
__rte_experimental
rte_trace_point_t *rte_trace_point_lookup(const char *name);
+#ifdef RTE_TRACE
/**
* @internal
*
@@ -230,6 +247,7 @@ __rte_trace_point_fp_is_enabled(void)
return false;
#endif
}
+#endif /* RTE_TRACE */
/**
* @internal
@@ -356,6 +374,8 @@ __rte_trace_point_emit_ev_header(void *mem, uint64_t in)
return RTE_PTR_ADD(mem, __RTE_TRACE_EVENT_HEADER_SZ);
}
+#ifdef RTE_TRACE
+
#define __rte_trace_point_emit_header_generic(t) \
void *mem; \
do { \
@@ -411,7 +431,7 @@ do { \
RTE_SET_USED(len); \
} while (0)
-
+#endif /* RTE_TRACE */
#endif /* ALLOW_EXPERIMENTAL_API */
#endif /* _RTE_TRACE_POINT_REGISTER_H_ */
diff --git a/lib/eal/include/rte_trace_point_register.h b/lib/eal/include/rte_trace_point_register.h
index 41260e5964..567fce534e 100644
--- a/lib/eal/include/rte_trace_point_register.h
+++ b/lib/eal/include/rte_trace_point_register.h
@@ -18,6 +18,8 @@ extern "C" {
RTE_DECLARE_PER_LCORE(volatile int, trace_point_sz);
+#ifdef RTE_TRACE
+
#define RTE_TRACE_POINT_REGISTER(trace, name) \
rte_trace_point_t __rte_section("__rte_trace_point") __##trace; \
static const char __##trace##_name[] = RTE_STR(name); \
@@ -56,6 +58,21 @@ do { \
RTE_STR(uint8_t)); \
} while (0)
+#else
+
+#define RTE_TRACE_POINT_REGISTER(...)
+#define __rte_trace_point_emit_header_generic(t) RTE_SET_USED(t)
+#define __rte_trace_point_emit_header_fp(t) RTE_SET_USED(t)
+#define __rte_trace_point_emit(in, type) RTE_SET_USED(in)
+#define rte_trace_point_emit_string(in) RTE_SET_USED(in)
+#define rte_trace_point_emit_blob(in, len) \
+do { \
+ RTE_SET_USED(in); \
+ RTE_SET_USED(len); \
+} while (0)
+
+#endif /* RTE_TRACE */
+
#ifdef __cplusplus
}
#endif
--
2.43.0
^ permalink raw reply [relevance 3%]
* RE: [PATCH] eal: add build-time option to omit trace
2024-09-18 9:49 0% ` Jerin Jacob
@ 2024-09-18 10:23 0% ` Morten Brørup
0 siblings, 0 replies; 200+ results
From: Morten Brørup @ 2024-09-18 10:23 UTC (permalink / raw)
To: Jerin Jacob; +Cc: Jerin Jacob, Sunil Kumar Kori, dev
> From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> Sent: Wednesday, 18 September 2024 11.50
to omit trace
>
> On Wed, Sep 18, 2024 at 2:39 PM Morten Brørup <mb@smartsharesystems.com>
> wrote:
> >
> > Some applications want to omit the trace feature.
> > Either to reduce the memory footprint, to reduce the exposed attack
> > surface, or for other reasons.
> >
> > This patch adds an option in rte_config.h to include or omit trace in the
> > build. Trace is included by default.
> >
> > Omitting trace works by omitting all trace points.
> > For API and ABI compatibility, the trace feature itself remains.
> >
> > Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
> > ---
> > app/test/test_trace.c | 11 ++++++++++-
> > config/rte_config.h | 1 +
> > lib/eal/include/rte_trace_point.h | 6 +++++-
> > lib/eal/include/rte_trace_point_register.h | 8 ++++++++
> > 4 files changed, 24 insertions(+), 2 deletions(-)
> >
> > diff --git a/app/test/test_trace.c b/app/test/test_trace.c
> > index 00809f433b..7918cc865d 100644
> > --- a/app/test/test_trace.c
> > +++ b/app/test/test_trace.c
> > @@ -12,7 +12,16 @@
> >
> > int app_dpdk_test_tp_count;
> >
> > -#ifdef RTE_EXEC_ENV_WINDOWS
> > +#if !defined(RTE_TRACE)
> > +
> > +static int
> > +test_trace(void)
> > +{
> > + printf("trace omitted at build-time, skipping test\n");
> > + return TEST_SKIPPED;
> > +}
> > +
> > +#elif defined(RTE_EXEC_ENV_WINDOWS)
> >
> > static int
> > test_trace(void)
> > diff --git a/config/rte_config.h b/config/rte_config.h
> > index dd7bb0d35b..fd6f8a2f1a 100644
> > --- a/config/rte_config.h
> > +++ b/config/rte_config.h
> > @@ -49,6 +49,7 @@
> > #define RTE_MAX_TAILQ 32
> > #define RTE_LOG_DP_LEVEL RTE_LOG_INFO
> > #define RTE_MAX_VFIO_CONTAINERS 64
> > +#define RTE_TRACE 1
> >
> > /* bsd module defines */
> > #define RTE_CONTIGMEM_MAX_NUM_BUFS 64
> > diff --git a/lib/eal/include/rte_trace_point.h
> b/lib/eal/include/rte_trace_point.h
> > index 41e2a7f99e..1b60bba043 100644
> > --- a/lib/eal/include/rte_trace_point.h
> > +++ b/lib/eal/include/rte_trace_point.h
> > @@ -212,6 +212,7 @@ bool rte_trace_point_is_enabled(rte_trace_point_t *tp);
> > __rte_experimental
> > rte_trace_point_t *rte_trace_point_lookup(const char *name);
> >
> > +#ifdef RTE_TRACE
> > /**
> > * @internal
> > *
> > @@ -230,6 +231,7 @@ __rte_trace_point_fp_is_enabled(void)
> > return false;
> > #endif
> > }
> > +#endif /* RTE_TRACE */
> >
> > /**
> > * @internal
> > @@ -356,6 +358,8 @@ __rte_trace_point_emit_ev_header(void *mem, uint64_t in)
> > return RTE_PTR_ADD(mem, __RTE_TRACE_EVENT_HEADER_SZ);
> > }
> >
> > +#ifdef RTE_TRACE
>
>
> Please change to 1.4.5 style _if possible_ in
> https://doc.dpdk.org/guides/contributing/coding_style.html.
The Coding Style's chapter 1.4.5 only applies to O/S selection.
Boolean configuration definitions either have the value 1 or are not defined.
> Assuming linker will remove the memory from the image if it is not
> using by stubbing out.
Assumption could be false if building with optimizations disabled. Don't know.
>
> Untested.
>
> #define __rte_trace_point_emit_header_generic(t) \
> void *mem; \
> do { \
> + if (RTE_TRACE_ENABLED == 0) \
> + return \
> const uint64_t val = rte_atomic_load_explicit(t,
> rte_memory_order_acquire); \
> if (likely(!(val & __RTE_TRACE_FIELD_ENABLE_MASK))) \
> return; \
> mem = __rte_trace_mem_get(val); \
> if (unlikely(mem == NULL)) \
> return; \
> mem = __rte_trace_point_emit_ev_header(mem, val); \
> } while (0)
I was initially down that path, inspired by how FP is enabled/disabled;
using an added __rte_trace_point_is_enabled(void) inline function, like the __rte_trace_point_fp_is_enabled(void).
But I kept getting lots of warnings, either about nonexisting references, or stuff not being used.
So I gave up and did it this way instead.
After having tried the other way, this way also looks cleaner to me.
>
>
>
> > +
> > #define __rte_trace_point_emit_header_generic(t) \
> > void *mem; \
> > do { \
> > @@ -411,7 +415,7 @@ do { \
> > RTE_SET_USED(len); \
> > } while (0)
> >
> > -
> > +#endif /* RTE_TRACE */
> > #endif /* ALLOW_EXPERIMENTAL_API */
> > #endif /* _RTE_TRACE_POINT_REGISTER_H_ */
> >
> > diff --git a/lib/eal/include/rte_trace_point_register.h
> b/lib/eal/include/rte_trace_point_register.h
> > index 41260e5964..78c0ede5f1 100644
> > --- a/lib/eal/include/rte_trace_point_register.h
> > +++ b/lib/eal/include/rte_trace_point_register.h
> > @@ -18,6 +18,8 @@ extern "C" {
> >
> > RTE_DECLARE_PER_LCORE(volatile int, trace_point_sz);
> >
> > +#ifdef RTE_TRACE
> > +
> > #define RTE_TRACE_POINT_REGISTER(trace, name) \
> > rte_trace_point_t __rte_section("__rte_trace_point") __##trace; \
> > static const char __##trace##_name[] = RTE_STR(name); \
> > @@ -27,6 +29,12 @@ RTE_INIT(trace##_init) \
> > (void (*)(void)) trace); \
> > }
> >
> > +#else
> > +
> > +#define RTE_TRACE_POINT_REGISTER(trace, name)
> > +
> > +#endif /* RTE_TRACE */
> > +
> > #define __rte_trace_point_emit_header_generic(t) \
> > RTE_PER_LCORE(trace_point_sz) = __RTE_TRACE_EVENT_HEADER_SZ
> >
> > --
> > 2.43.0
> >
^ permalink raw reply [relevance 0%]
* [RFC v2 1/1] dmadev: support priority configuration
2024-09-18 9:40 10% ` [RFC v1 1/1] dmadev: support priority configuration Vamsi Krishna
@ 2024-09-18 10:13 4% ` Vamsi Krishna
2024-10-03 11:41 4% ` [PATCH v3 " Vamsi Krishna
0 siblings, 1 reply; 200+ results
From: Vamsi Krishna @ 2024-09-18 10:13 UTC (permalink / raw)
To: thomas, fengchengwen, bruce.richardson, mb
Cc: dev, kevin.laatz, jerinj, conor.walsh, gmuthukrishn, vvelumuri,
g.singh, sachin.saxena, hemant.agrawal, Vamsi Attunuru,
Amit Prakash Shukla
From: Vamsi Attunuru <vattunuru@marvell.com>
Some DMA controllers offer the ability to configure priority level
for the hardware command queues, allowing for the prioritization of
DMA command execution based on queue importance.
This patch introduces the necessary fields in the dmadev structures to
retrieve information about the hardware-supported priority levels and to
enable priority configuration from the application.
Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
V2 changes:
* Reverted removed text from release_24_11.rst
V1 changes:
* Added trace support
* Added new capability flag
Deprecation notice:
https://patches.dpdk.org/project/dpdk/patch/20240730144612.2132848-1-amitprakashs@marvell.com/
* Assuming we do not anticipate any advanced scheduling schemes for dmadev queues,
this RFC is intended to support a strict priority scheme.
doc/guides/rel_notes/release_24_11.rst | 8 ++++++++
lib/dmadev/rte_dmadev.c | 15 +++++++++++++++
lib/dmadev/rte_dmadev.h | 21 +++++++++++++++++++++
lib/dmadev/rte_dmadev_trace.h | 2 ++
4 files changed, 46 insertions(+)
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 0ff70d9057..fc3610deff 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -55,6 +55,11 @@ New Features
Also, make sure to start the actual text at the margin.
=======================================================
+* **Added strict priority capability flag in dmadev.**
+
+ Added new capability flag ``RTE_DMA_CAPA_PRI_POLICY_SP`` to check if the
+ DMA device supports assigning fixed priority to its channels, allowing
+ for better control over resource allocation and scheduling.
Removed Items
-------------
@@ -100,6 +105,9 @@ ABI Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* dmadev: Added ``nb_priorities`` field to ``rte_dma_info`` structure and
+ ``priority`` field to ``rte_dma_conf`` structure to get device supported
+ priority levels and configure required priority from the application.
Known Issues
------------
diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
index 845727210f..3d9063dee3 100644
--- a/lib/dmadev/rte_dmadev.c
+++ b/lib/dmadev/rte_dmadev.c
@@ -497,6 +497,21 @@ rte_dma_configure(int16_t dev_id, const struct rte_dma_conf *dev_conf)
return -EINVAL;
}
+ if (dev_conf->priority && !(dev_info.dev_capa & RTE_DMA_CAPA_PRI_POLICY_SP)) {
+ RTE_DMA_LOG(ERR, "Device %d don't support prioritization", dev_id);
+ return -EINVAL;
+ }
+
+ if (dev_info.nb_priorities == 1) {
+ RTE_DMA_LOG(ERR, "Device %d must support more than 1 priority, or else 0", dev_id);
+ return -EINVAL;
+ }
+
+ if (dev_info.nb_priorities && (dev_conf->priority >= dev_info.nb_priorities)) {
+ RTE_DMA_LOG(ERR, "Device %d configure invalid priority", dev_id);
+ return -EINVAL;
+ }
+
if (*dev->dev_ops->dev_configure == NULL)
return -ENOTSUP;
ret = (*dev->dev_ops->dev_configure)(dev, dev_conf,
diff --git a/lib/dmadev/rte_dmadev.h b/lib/dmadev/rte_dmadev.h
index 5474a5281d..e5f730c327 100644
--- a/lib/dmadev/rte_dmadev.h
+++ b/lib/dmadev/rte_dmadev.h
@@ -268,6 +268,16 @@ int16_t rte_dma_next_dev(int16_t start_dev_id);
#define RTE_DMA_CAPA_OPS_COPY_SG RTE_BIT64(33)
/** Support fill operation. */
#define RTE_DMA_CAPA_OPS_FILL RTE_BIT64(34)
+/** Support strict prioritization at DMA HW channel level
+ *
+ * If device supports HW channel prioritization then application could
+ * assign fixed priority to the DMA HW channel using 'priority' field in
+ * struct rte_dma_conf. Number of supported priority levels will be known
+ * from 'nb_priorities' field in struct rte_dma_info.
+ *
+ * DMA devices which support prioritization can advertise this capability.
+ */
+#define RTE_DMA_CAPA_PRI_POLICY_SP RTE_BIT64(35)
/**@}*/
/**
@@ -297,6 +307,10 @@ struct rte_dma_info {
int16_t numa_node;
/** Number of virtual DMA channel configured. */
uint16_t nb_vchans;
+ /** Number of priority levels (must be > 1), if supported by DMA HW channel.
+ * 0 otherwise.
+ */
+ uint16_t nb_priorities;
};
/**
@@ -332,6 +346,13 @@ struct rte_dma_conf {
* @see RTE_DMA_CAPA_SILENT
*/
bool enable_silent;
+ /* The priority of the DMA HW channel.
+ * This value cannot be greater than or equal to the field 'nb_priorities'
+ * of struct rte_dma_info which get from rte_dma_info_get().
+ * Among the values between '0' and 'nb_priorities - 1', lowest value
+ * indicates higher priority and vice-versa.
+ */
+ uint16_t priority;
};
/**
diff --git a/lib/dmadev/rte_dmadev_trace.h b/lib/dmadev/rte_dmadev_trace.h
index e55c4c6091..be089c065c 100644
--- a/lib/dmadev/rte_dmadev_trace.h
+++ b/lib/dmadev/rte_dmadev_trace.h
@@ -35,6 +35,7 @@ RTE_TRACE_POINT(
rte_trace_point_emit_u16(dev_info->max_sges);
rte_trace_point_emit_i16(dev_info->numa_node);
rte_trace_point_emit_u16(dev_info->nb_vchans);
+ rte_trace_point_emit_u16(dev_info->nb_priorities);
)
RTE_TRACE_POINT(
@@ -48,6 +49,7 @@ RTE_TRACE_POINT(
int enable_silent = (int)dev_conf->enable_silent;
rte_trace_point_emit_i16(dev_id);
rte_trace_point_emit_u16(dev_conf->nb_vchans);
+ rte_trace_point_emit_u16(dev_conf->priority);
rte_trace_point_emit_int(enable_silent);
rte_trace_point_emit_int(ret);
)
--
2.34.1
^ permalink raw reply [relevance 4%]
* Re: [PATCH] eal: add build-time option to omit trace
2024-09-18 8:55 3% [PATCH] eal: add build-time option to omit trace Morten Brørup
@ 2024-09-18 9:49 0% ` Jerin Jacob
2024-09-18 10:23 0% ` Morten Brørup
2024-09-19 8:06 3% ` [PATCH v2] " Morten Brørup
` (6 subsequent siblings)
7 siblings, 1 reply; 200+ results
From: Jerin Jacob @ 2024-09-18 9:49 UTC (permalink / raw)
To: Morten Brørup; +Cc: Jerin Jacob, Sunil Kumar Kori, dev
On Wed, Sep 18, 2024 at 2:39 PM Morten Brørup <mb@smartsharesystems.com> wrote:
>
> Some applications want to omit the trace feature.
> Either to reduce the memory footprint, to reduce the exposed attack
> surface, or for other reasons.
>
> This patch adds an option in rte_config.h to include or omit trace in the
> build. Trace is included by default.
>
> Omitting trace works by omitting all trace points.
> For API and ABI compatibility, the trace feature itself remains.
>
> Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
> ---
> app/test/test_trace.c | 11 ++++++++++-
> config/rte_config.h | 1 +
> lib/eal/include/rte_trace_point.h | 6 +++++-
> lib/eal/include/rte_trace_point_register.h | 8 ++++++++
> 4 files changed, 24 insertions(+), 2 deletions(-)
>
> diff --git a/app/test/test_trace.c b/app/test/test_trace.c
> index 00809f433b..7918cc865d 100644
> --- a/app/test/test_trace.c
> +++ b/app/test/test_trace.c
> @@ -12,7 +12,16 @@
>
> int app_dpdk_test_tp_count;
>
> -#ifdef RTE_EXEC_ENV_WINDOWS
> +#if !defined(RTE_TRACE)
> +
> +static int
> +test_trace(void)
> +{
> + printf("trace omitted at build-time, skipping test\n");
> + return TEST_SKIPPED;
> +}
> +
> +#elif defined(RTE_EXEC_ENV_WINDOWS)
>
> static int
> test_trace(void)
> diff --git a/config/rte_config.h b/config/rte_config.h
> index dd7bb0d35b..fd6f8a2f1a 100644
> --- a/config/rte_config.h
> +++ b/config/rte_config.h
> @@ -49,6 +49,7 @@
> #define RTE_MAX_TAILQ 32
> #define RTE_LOG_DP_LEVEL RTE_LOG_INFO
> #define RTE_MAX_VFIO_CONTAINERS 64
> +#define RTE_TRACE 1
>
> /* bsd module defines */
> #define RTE_CONTIGMEM_MAX_NUM_BUFS 64
> diff --git a/lib/eal/include/rte_trace_point.h b/lib/eal/include/rte_trace_point.h
> index 41e2a7f99e..1b60bba043 100644
> --- a/lib/eal/include/rte_trace_point.h
> +++ b/lib/eal/include/rte_trace_point.h
> @@ -212,6 +212,7 @@ bool rte_trace_point_is_enabled(rte_trace_point_t *tp);
> __rte_experimental
> rte_trace_point_t *rte_trace_point_lookup(const char *name);
>
> +#ifdef RTE_TRACE
> /**
> * @internal
> *
> @@ -230,6 +231,7 @@ __rte_trace_point_fp_is_enabled(void)
> return false;
> #endif
> }
> +#endif /* RTE_TRACE */
>
> /**
> * @internal
> @@ -356,6 +358,8 @@ __rte_trace_point_emit_ev_header(void *mem, uint64_t in)
> return RTE_PTR_ADD(mem, __RTE_TRACE_EVENT_HEADER_SZ);
> }
>
> +#ifdef RTE_TRACE
Please change to 1.4.5 style _if possible_ in
https://doc.dpdk.org/guides/contributing/coding_style.html.
Assuming linker will remove the memory from the image if it is not
using by stubbing out.
Untested.
#define __rte_trace_point_emit_header_generic(t) \
void *mem; \
do { \
+ if (RTE_TRACE_ENABLED == 0) \
+ return \
const uint64_t val = rte_atomic_load_explicit(t,
rte_memory_order_acquire); \
if (likely(!(val & __RTE_TRACE_FIELD_ENABLE_MASK))) \
return; \
mem = __rte_trace_mem_get(val); \
if (unlikely(mem == NULL)) \
return; \
mem = __rte_trace_point_emit_ev_header(mem, val); \
} while (0)
> +
> #define __rte_trace_point_emit_header_generic(t) \
> void *mem; \
> do { \
> @@ -411,7 +415,7 @@ do { \
> RTE_SET_USED(len); \
> } while (0)
>
> -
> +#endif /* RTE_TRACE */
> #endif /* ALLOW_EXPERIMENTAL_API */
> #endif /* _RTE_TRACE_POINT_REGISTER_H_ */
>
> diff --git a/lib/eal/include/rte_trace_point_register.h b/lib/eal/include/rte_trace_point_register.h
> index 41260e5964..78c0ede5f1 100644
> --- a/lib/eal/include/rte_trace_point_register.h
> +++ b/lib/eal/include/rte_trace_point_register.h
> @@ -18,6 +18,8 @@ extern "C" {
>
> RTE_DECLARE_PER_LCORE(volatile int, trace_point_sz);
>
> +#ifdef RTE_TRACE
> +
> #define RTE_TRACE_POINT_REGISTER(trace, name) \
> rte_trace_point_t __rte_section("__rte_trace_point") __##trace; \
> static const char __##trace##_name[] = RTE_STR(name); \
> @@ -27,6 +29,12 @@ RTE_INIT(trace##_init) \
> (void (*)(void)) trace); \
> }
>
> +#else
> +
> +#define RTE_TRACE_POINT_REGISTER(trace, name)
> +
> +#endif /* RTE_TRACE */
> +
> #define __rte_trace_point_emit_header_generic(t) \
> RTE_PER_LCORE(trace_point_sz) = __RTE_TRACE_EVENT_HEADER_SZ
>
> --
> 2.43.0
>
^ permalink raw reply [relevance 0%]
* [RFC v1 1/1] dmadev: support priority configuration
2024-09-13 12:10 11% [RFC v0 1/1] dmadev: provide priority configuration support Vamsi Krishna
2024-09-14 9:41 0% ` fengchengwen
@ 2024-09-18 9:40 10% ` Vamsi Krishna
2024-09-18 10:13 4% ` [RFC v2 " Vamsi Krishna
1 sibling, 1 reply; 200+ results
From: Vamsi Krishna @ 2024-09-18 9:40 UTC (permalink / raw)
To: thomas, fengchengwen, bruce.richardson, mb
Cc: dev, kevin.laatz, jerinj, conor.walsh, gmuthukrishn, vvelumuri,
g.singh, sachin.saxena, hemant.agrawal, Vamsi Attunuru,
Amit Prakash Shukla
From: Vamsi Attunuru <vattunuru@marvell.com>
Some DMA controllers offer the ability to configure priority level
for the hardware command queues, allowing for the prioritization of
DMA command execution based on queue importance.
This patch introduces the necessary fields in the dmadev structures to
retrieve information about the hardware-supported priority levels and to
enable priority configuration from the application.
Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
v1 changes:
* added trace support
* added capability flag
Deprecation notice:
https://patches.dpdk.org/project/dpdk/patch/20240730144612.2132848-1-amitprakashs@marvell.com/
* Assuming we do not anticipate any advanced scheduling schemes for dmadev queues,
this RFC is intended to support a strict prioirty scheme.
doc/guides/rel_notes/release_24_11.rst | 49 ++++----------------------
lib/dmadev/rte_dmadev.c | 15 ++++++++
lib/dmadev/rte_dmadev.h | 21 +++++++++++
lib/dmadev/rte_dmadev_trace.h | 2 ++
4 files changed, 45 insertions(+), 42 deletions(-)
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 0ff70d9057..2dc34919a9 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -24,37 +24,11 @@ DPDK Release 24.11
New Features
------------
-.. This section should contain new features added in this release.
- Sample format:
-
- * **Add a title in the past tense with a full stop.**
-
- Add a short 1-2 sentence description in the past tense.
- The description should be enough to allow someone scanning
- the release notes to understand the new feature.
-
- If the feature adds a lot of sub-features you can use a bullet list
- like this:
-
- * Added feature foo to do something.
- * Enhanced feature bar to do something else.
-
- Refer to the previous release notes for examples.
-
- Suggested order in release notes items:
- * Core libs (EAL, mempool, ring, mbuf, buses)
- * Device abstraction libs and PMDs (ordered alphabetically by vendor name)
- - ethdev (lib, PMDs)
- - cryptodev (lib, PMDs)
- - eventdev (lib, PMDs)
- - etc
- * Other libs
- * Apps, Examples, Tools (if significant)
-
- This section is a comment. Do not overwrite or remove it.
- Also, make sure to start the actual text at the margin.
- =======================================================
+* **Added strict priority capability flag in dmadev.**
+ Added new capability flag ``RTE_DMA_CAPA_PRI_POLICY_SP`` to check if the
+ DMA device supports assigning fixed priority to its channels, allowing
+ for better control over resource allocation and scheduling.
Removed Items
-------------
@@ -88,18 +62,9 @@ API Changes
ABI Changes
-----------
-.. This section should contain ABI changes. Sample format:
-
- * sample: Add a short 1-2 sentence description of the ABI change
- which was announced in the previous releases and made in this release.
- Start with a scope label like "ethdev:".
- Use fixed width quotes for ``function_names`` or ``struct_names``.
- Use the past tense.
-
- This section is a comment. Do not overwrite or remove it.
- Also, make sure to start the actual text at the margin.
- =======================================================
-
+* dmadev: Added ``nb_priorities`` field to ``rte_dma_info`` structure and
+ ``priority`` field to ``rte_dma_conf`` structure to get device supported
+ priority levels and configure required priority from the application.
Known Issues
------------
diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
index 845727210f..3d9063dee3 100644
--- a/lib/dmadev/rte_dmadev.c
+++ b/lib/dmadev/rte_dmadev.c
@@ -497,6 +497,21 @@ rte_dma_configure(int16_t dev_id, const struct rte_dma_conf *dev_conf)
return -EINVAL;
}
+ if (dev_conf->priority && !(dev_info.dev_capa & RTE_DMA_CAPA_PRI_POLICY_SP)) {
+ RTE_DMA_LOG(ERR, "Device %d don't support prioritization", dev_id);
+ return -EINVAL;
+ }
+
+ if (dev_info.nb_priorities == 1) {
+ RTE_DMA_LOG(ERR, "Device %d must support more than 1 priority, or else 0", dev_id);
+ return -EINVAL;
+ }
+
+ if (dev_info.nb_priorities && (dev_conf->priority >= dev_info.nb_priorities)) {
+ RTE_DMA_LOG(ERR, "Device %d configure invalid priority", dev_id);
+ return -EINVAL;
+ }
+
if (*dev->dev_ops->dev_configure == NULL)
return -ENOTSUP;
ret = (*dev->dev_ops->dev_configure)(dev, dev_conf,
diff --git a/lib/dmadev/rte_dmadev.h b/lib/dmadev/rte_dmadev.h
index 5474a5281d..fbe414a76f 100644
--- a/lib/dmadev/rte_dmadev.h
+++ b/lib/dmadev/rte_dmadev.h
@@ -268,6 +268,16 @@ int16_t rte_dma_next_dev(int16_t start_dev_id);
#define RTE_DMA_CAPA_OPS_COPY_SG RTE_BIT64(33)
/** Support fill operation. */
#define RTE_DMA_CAPA_OPS_FILL RTE_BIT64(34)
+/** Support strict prioritization at DMA HW channel level
+ *
+ * If device supports HW channel prioritization then application could
+ * assign fixed priority to the DMA HW channel using 'priority' field in
+ * struct rte_dma_conf. Number of supported prioirty levels will be known
+ * from 'nb_priorities' field in struct rte_dma_info.
+ *
+ * DMA devices which support prioritization can advertise this capability.
+ */
+#define RTE_DMA_CAPA_PRI_POLICY_SP RTE_BIT64(35)
/**@}*/
/**
@@ -297,6 +307,10 @@ struct rte_dma_info {
int16_t numa_node;
/** Number of virtual DMA channel configured. */
uint16_t nb_vchans;
+ /** Number of priority levels (must be > 1), if supported by DMA HW channel.
+ * 0 otherwise.
+ */
+ uint16_t nb_priorities;
};
/**
@@ -332,6 +346,13 @@ struct rte_dma_conf {
* @see RTE_DMA_CAPA_SILENT
*/
bool enable_silent;
+ /* The priority of the DMA HW channel.
+ * This value cannot be greater than or equal to the field 'nb_priorities'
+ * of struct rte_dma_info which get from rte_dma_info_get().
+ * Among the values between '0' and 'nb_priorities - 1', lowest value
+ * indicates higher priority and vice-versa.
+ */
+ uint16_t priority;
};
/**
diff --git a/lib/dmadev/rte_dmadev_trace.h b/lib/dmadev/rte_dmadev_trace.h
index e55c4c6091..be089c065c 100644
--- a/lib/dmadev/rte_dmadev_trace.h
+++ b/lib/dmadev/rte_dmadev_trace.h
@@ -35,6 +35,7 @@ RTE_TRACE_POINT(
rte_trace_point_emit_u16(dev_info->max_sges);
rte_trace_point_emit_i16(dev_info->numa_node);
rte_trace_point_emit_u16(dev_info->nb_vchans);
+ rte_trace_point_emit_u16(dev_info->nb_priorities);
)
RTE_TRACE_POINT(
@@ -48,6 +49,7 @@ RTE_TRACE_POINT(
int enable_silent = (int)dev_conf->enable_silent;
rte_trace_point_emit_i16(dev_id);
rte_trace_point_emit_u16(dev_conf->nb_vchans);
+ rte_trace_point_emit_u16(dev_conf->priority);
rte_trace_point_emit_int(enable_silent);
rte_trace_point_emit_int(ret);
)
--
2.34.1
^ permalink raw reply [relevance 10%]
* [PATCH] eal: add build-time option to omit trace
@ 2024-09-18 8:55 3% Morten Brørup
2024-09-18 9:49 0% ` Jerin Jacob
` (7 more replies)
0 siblings, 8 replies; 200+ results
From: Morten Brørup @ 2024-09-18 8:55 UTC (permalink / raw)
To: Jerin Jacob, Sunil Kumar Kori; +Cc: dev, Morten Brørup
Some applications want to omit the trace feature.
Either to reduce the memory footprint, to reduce the exposed attack
surface, or for other reasons.
This patch adds an option in rte_config.h to include or omit trace in the
build. Trace is included by default.
Omitting trace works by omitting all trace points.
For API and ABI compatibility, the trace feature itself remains.
Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
---
app/test/test_trace.c | 11 ++++++++++-
config/rte_config.h | 1 +
lib/eal/include/rte_trace_point.h | 6 +++++-
lib/eal/include/rte_trace_point_register.h | 8 ++++++++
4 files changed, 24 insertions(+), 2 deletions(-)
diff --git a/app/test/test_trace.c b/app/test/test_trace.c
index 00809f433b..7918cc865d 100644
--- a/app/test/test_trace.c
+++ b/app/test/test_trace.c
@@ -12,7 +12,16 @@
int app_dpdk_test_tp_count;
-#ifdef RTE_EXEC_ENV_WINDOWS
+#if !defined(RTE_TRACE)
+
+static int
+test_trace(void)
+{
+ printf("trace omitted at build-time, skipping test\n");
+ return TEST_SKIPPED;
+}
+
+#elif defined(RTE_EXEC_ENV_WINDOWS)
static int
test_trace(void)
diff --git a/config/rte_config.h b/config/rte_config.h
index dd7bb0d35b..fd6f8a2f1a 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -49,6 +49,7 @@
#define RTE_MAX_TAILQ 32
#define RTE_LOG_DP_LEVEL RTE_LOG_INFO
#define RTE_MAX_VFIO_CONTAINERS 64
+#define RTE_TRACE 1
/* bsd module defines */
#define RTE_CONTIGMEM_MAX_NUM_BUFS 64
diff --git a/lib/eal/include/rte_trace_point.h b/lib/eal/include/rte_trace_point.h
index 41e2a7f99e..1b60bba043 100644
--- a/lib/eal/include/rte_trace_point.h
+++ b/lib/eal/include/rte_trace_point.h
@@ -212,6 +212,7 @@ bool rte_trace_point_is_enabled(rte_trace_point_t *tp);
__rte_experimental
rte_trace_point_t *rte_trace_point_lookup(const char *name);
+#ifdef RTE_TRACE
/**
* @internal
*
@@ -230,6 +231,7 @@ __rte_trace_point_fp_is_enabled(void)
return false;
#endif
}
+#endif /* RTE_TRACE */
/**
* @internal
@@ -356,6 +358,8 @@ __rte_trace_point_emit_ev_header(void *mem, uint64_t in)
return RTE_PTR_ADD(mem, __RTE_TRACE_EVENT_HEADER_SZ);
}
+#ifdef RTE_TRACE
+
#define __rte_trace_point_emit_header_generic(t) \
void *mem; \
do { \
@@ -411,7 +415,7 @@ do { \
RTE_SET_USED(len); \
} while (0)
-
+#endif /* RTE_TRACE */
#endif /* ALLOW_EXPERIMENTAL_API */
#endif /* _RTE_TRACE_POINT_REGISTER_H_ */
diff --git a/lib/eal/include/rte_trace_point_register.h b/lib/eal/include/rte_trace_point_register.h
index 41260e5964..78c0ede5f1 100644
--- a/lib/eal/include/rte_trace_point_register.h
+++ b/lib/eal/include/rte_trace_point_register.h
@@ -18,6 +18,8 @@ extern "C" {
RTE_DECLARE_PER_LCORE(volatile int, trace_point_sz);
+#ifdef RTE_TRACE
+
#define RTE_TRACE_POINT_REGISTER(trace, name) \
rte_trace_point_t __rte_section("__rte_trace_point") __##trace; \
static const char __##trace##_name[] = RTE_STR(name); \
@@ -27,6 +29,12 @@ RTE_INIT(trace##_init) \
(void (*)(void)) trace); \
}
+#else
+
+#define RTE_TRACE_POINT_REGISTER(trace, name)
+
+#endif /* RTE_TRACE */
+
#define __rte_trace_point_emit_header_generic(t) \
RTE_PER_LCORE(trace_point_sz) = __RTE_TRACE_EVENT_HEADER_SZ
--
2.43.0
^ permalink raw reply [relevance 3%]
* RE: [EXTERNAL] Re: [RFC v0 1/1] dmadev: provide priority configuration support
2024-09-14 9:41 0% ` fengchengwen
@ 2024-09-16 16:34 0% ` Vamsi Krishna Attunuru
0 siblings, 0 replies; 200+ results
From: Vamsi Krishna Attunuru @ 2024-09-16 16:34 UTC (permalink / raw)
To: fengchengwen, thomas, bruce.richardson, mb
Cc: dev, kevin.laatz, Jerin Jacob, conor.walsh,
Gowrishankar Muthukrishnan, Vidya Sagar Velumuri, g.singh,
sachin.saxena, hemant.agrawal, Amit Prakash Shukla
>-----Original Message-----
>From: fengchengwen <fengchengwen@huawei.com>
>Sent: Saturday, September 14, 2024 3:12 PM
>To: Vamsi Krishna Attunuru <vattunuru@marvell.com>;
>thomas@monjalon.net; bruce.richardson@intel.com;
>mb@smartsharesystems.com
>Cc: dev@dpdk.org; kevin.laatz@intel.com; Jerin Jacob <jerinj@marvell.com>;
>conor.walsh@intel.com; Gowrishankar Muthukrishnan
><gmuthukrishn@marvell.com>; Vidya Sagar Velumuri
><vvelumuri@marvell.com>; g.singh@nxp.com; sachin.saxena@oss.nxp.com;
>hemant.agrawal@nxp.com; Amit Prakash Shukla
><amitprakashs@marvell.com>
>Subject: [EXTERNAL] Re: [RFC v0 1/1] dmadev: provide priority configuration
>support
>
>On 2024/9/13 20: 10, Vamsi Krishna wrote: > From: Vamsi Attunuru
><vattunuru@ marvell. com> > > Some DMA controllers offer the ability to
>configure priority level > for the hardware command queues, allowing for the
>prioritization
>On 2024/9/13 20:10, Vamsi Krishna wrote:
>> From: Vamsi Attunuru <vattunuru@marvell.com>
>>
>> Some DMA controllers offer the ability to configure priority level for
>> the hardware command queues, allowing for the prioritization of DMA
>> command execution based on queue importance.
>>
>> This patch introduces the necessary fields in the dmadev structures to
>> retrieve information about the hardware-supported priority levels and
>> to enable priority configuration from the application.
>>
>> Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
>> Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
>> ---
>>
>> Deprecation notice:
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__patches.dpdk.org_
>> project_dpdk_patch_20240730144612.2132848-2D1-2Damitprakashs-
>40marvell
>>
>.com_&d=DwICaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=WllrYaumVkxaWjgKto6E
>_rtDQshh
>>
>Ihik2jkvzFyRhW8&m=c5golEA5oYRC2qs5aUbwKAGfIFWAsjeOW4OaQaRl9pE11
>PNXlmhg
>> 2MEnn4OYFN8g&s=3XTQJ_S2k_9fzXvPXLOYdj4VFExnzZ04_uiB1qBaFv0&e=
>>
>> * Assuming we do not anticipate any advanced scheduling schemes for
>dmadev queues,
>> this RFC is intended to support a strict prioirty scheme.
>>
>> doc/guides/rel_notes/release_24_11.rst | 15 +++------------
>> lib/dmadev/rte_dmadev.c | 10 ++++++++++
>> lib/dmadev/rte_dmadev.h | 11 +++++++++++
>> 3 files changed, 24 insertions(+), 12 deletions(-)
>>
>> diff --git a/doc/guides/rel_notes/release_24_11.rst
>> b/doc/guides/rel_notes/release_24_11.rst
>> index 0ff70d9057..dc711bbf98 100644
>> --- a/doc/guides/rel_notes/release_24_11.rst
>> +++ b/doc/guides/rel_notes/release_24_11.rst
>> @@ -88,18 +88,9 @@ API Changes
>> ABI Changes
>> -----------
>>
>> -.. This section should contain ABI changes. Sample format:
>> -
>> - * sample: Add a short 1-2 sentence description of the ABI change
>> - which was announced in the previous releases and made in this release.
>> - Start with a scope label like "ethdev:".
>> - Use fixed width quotes for ``function_names`` or ``struct_names``.
>> - Use the past tense.
>> -
>> - This section is a comment. Do not overwrite or remove it.
>> - Also, make sure to start the actual text at the margin.
>> - =======================================================
>> -
>> +* dmadev: Added ``nb_priorities`` field to ``rte_dma_info`` structure
>> +and
>> + ``priority`` field to ``rte_dma_conf`` structure to get device
>> +supported
>> + priority levels and configure required priority from the application.
>>
>> Known Issues
>> ------------
>> diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c index
>> 845727210f..053170f55a 100644
>> --- a/lib/dmadev/rte_dmadev.c
>> +++ b/lib/dmadev/rte_dmadev.c
>> @@ -497,6 +497,16 @@ rte_dma_configure(int16_t dev_id, const struct
>rte_dma_conf *dev_conf)
>> return -EINVAL;
>> }
>>
>> + if (dev_info.nb_priorities == 1) {
>> + RTE_DMA_LOG(ERR, "Device %d must support more than 1
>priority, or else 0", dev_id);
>> + return -EINVAL;
>> + }
>> +
>> + if (dev_info.nb_priorities && (dev_conf->priority >=
>dev_info.nb_priorities)) {
>> + RTE_DMA_LOG(ERR, "Device %d configure invalid priority",
>dev_id);
>> + return -EINVAL;
>> + }
>
>Please add trace support
Sure, will add it.
>
>> +
>> if (*dev->dev_ops->dev_configure == NULL)
>> return -ENOTSUP;
>> ret = (*dev->dev_ops->dev_configure)(dev, dev_conf, diff --git
>> a/lib/dmadev/rte_dmadev.h b/lib/dmadev/rte_dmadev.h index
>> 5474a5281d..d59fcf1b61 100644
>> --- a/lib/dmadev/rte_dmadev.h
>> +++ b/lib/dmadev/rte_dmadev.h
>> @@ -297,6 +297,10 @@ struct rte_dma_info {
>> int16_t numa_node;
>> /** Number of virtual DMA channel configured. */
>> uint16_t nb_vchans;
>> + /** Number of priority levels (must be > 1), if supported by DMA HW
>channel.
>> + * 0 otherwise.
>> + */
>> + uint16_t nb_priorities;
>
>Suggest add priority schedule policy capability, for example:
>RTE_DMA_CAPA_PRIORITY_POLICY_SP
Sure, since strict priority is only supported and not explicitly defined as a capability. Will add it as capability.
>
>> };
>>
>> /**
>> @@ -332,6 +336,13 @@ struct rte_dma_conf {
>> * @see RTE_DMA_CAPA_SILENT
>> */
>> bool enable_silent;
>> + /* The prioirty of the DMA HW channel.
>> + * This value cannot be greater than or equal to the field 'nb_priorities'
>> + * of struct rte_dma_info which get from rte_dma_info_get().
>> + * Among the values between '0' and 'nb_priorities - 1', lowest value
>> + * indicates higher priority and vice-versa.
>> + */
>> + uint16_t priority;
>> };
>>
>> /**
^ permalink raw reply [relevance 0%]
* Re: [PATCH] ethdev: optimize activation of fast-path tracepoints
2024-09-16 8:31 3% ` David Marchand
@ 2024-09-16 8:45 0% ` Jerin Jacob
0 siblings, 0 replies; 200+ results
From: Jerin Jacob @ 2024-09-16 8:45 UTC (permalink / raw)
To: David Marchand
Cc: Adel Belkhiri, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko,
dev, Adel Belkhiri
On Mon, Sep 16, 2024 at 2:01 PM David Marchand
<david.marchand@redhat.com> wrote:
>
> On Mon, Sep 16, 2024 at 10:26 AM Jerin Jacob <jerinjacobk@gmail.com> wrote:
> > > rte_flow_dev_dump;
> > >
> > > # added in 20.05
> > > - __rte_ethdev_trace_rx_burst;
> >
> > Removal of a public symbol breaks the ABI. The good news is that 24.11
> > can break the ABI.
> > IMO, It is OK to break this ABI.
>
> This symbol is in the experimental ABI.
> And in general, I think we should keep the trace symbols as experimental.
Ack.
So I think, we need to update only "Removed items" in
doc/guides/rel_notes/release_24_11.rst
>
>
> --
> David Marchand
>
^ permalink raw reply [relevance 0%]
* Re: [PATCH] ethdev: optimize activation of fast-path tracepoints
2024-09-16 8:25 4% ` Jerin Jacob
@ 2024-09-16 8:31 3% ` David Marchand
2024-09-16 8:45 0% ` Jerin Jacob
0 siblings, 1 reply; 200+ results
From: David Marchand @ 2024-09-16 8:31 UTC (permalink / raw)
To: Jerin Jacob
Cc: Adel Belkhiri, Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko,
dev, Adel Belkhiri
On Mon, Sep 16, 2024 at 10:26 AM Jerin Jacob <jerinjacobk@gmail.com> wrote:
> > rte_flow_dev_dump;
> >
> > # added in 20.05
> > - __rte_ethdev_trace_rx_burst;
>
> Removal of a public symbol breaks the ABI. The good news is that 24.11
> can break the ABI.
> IMO, It is OK to break this ABI.
This symbol is in the experimental ABI.
And in general, I think we should keep the trace symbols as experimental.
--
David Marchand
^ permalink raw reply [relevance 3%]
* Re: [PATCH] ethdev: optimize activation of fast-path tracepoints
@ 2024-09-16 8:25 4% ` Jerin Jacob
2024-09-16 8:31 3% ` David Marchand
0 siblings, 1 reply; 200+ results
From: Jerin Jacob @ 2024-09-16 8:25 UTC (permalink / raw)
To: Adel Belkhiri
Cc: Thomas Monjalon, Ferruh Yigit, Andrew Rybchenko, dev, Adel Belkhiri
On Sat, Sep 14, 2024 at 1:59 AM Adel Belkhiri <adel.belkhiri@gmail.com> wrote:
>
> From: Adel Belkhiri <adel.belkhiri@polymtl.ca>
>
> Split the tracepoints rte_ethdev_trace_rx_burst and
> rte_eth_trace_call_rx_callbacks into two separate ones
> for empty and non-empty calls to avoid saturating
> quickly the trace buffer.
>
> Signed-off-by: Adel Belkhiri <adel.belkhiri@polymtl.ca>
> ---
> - nb_rx, nb_pkts);
> + if (unlikely(nb_rx > 0))
You may consider unlikely(nb_rx)
> rte_flow_dev_dump;
>
> # added in 20.05
> - __rte_ethdev_trace_rx_burst;
Removal of a public symbol breaks the ABI. The good news is that 24.11
can break the ABI.
IMO, It is OK to break this ABI.
Also need to update "Removed items" in doc/guides/rel_notes/release_24_11.rst
> + __rte_ethdev_trace_rx_burst_empty;
> + __rte_ethdev_trace_rx_burst_nonempty;
> __rte_ethdev_trace_tx_burst;
> rte_flow_get_aged_flows;
>
> --
> 2.34.1
>
^ permalink raw reply [relevance 4%]
* Re: [RFC v0 1/1] dmadev: provide priority configuration support
2024-09-13 12:10 11% [RFC v0 1/1] dmadev: provide priority configuration support Vamsi Krishna
@ 2024-09-14 9:41 0% ` fengchengwen
2024-09-16 16:34 0% ` [EXTERNAL] " Vamsi Krishna Attunuru
2024-09-18 9:40 10% ` [RFC v1 1/1] dmadev: support priority configuration Vamsi Krishna
1 sibling, 1 reply; 200+ results
From: fengchengwen @ 2024-09-14 9:41 UTC (permalink / raw)
To: Vamsi Krishna, thomas, bruce.richardson, mb
Cc: dev, kevin.laatz, jerinj, conor.walsh, gmuthukrishn, vvelumuri,
g.singh, sachin.saxena, hemant.agrawal, Amit Prakash Shukla
On 2024/9/13 20:10, Vamsi Krishna wrote:
> From: Vamsi Attunuru <vattunuru@marvell.com>
>
> Some DMA controllers offer the ability to configure priority level
> for the hardware command queues, allowing for the prioritization of
> DMA command execution based on queue importance.
>
> This patch introduces the necessary fields in the dmadev structures to
> retrieve information about the hardware-supported priority levels and to
> enable priority configuration from the application.
>
> Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
> Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
> ---
>
> Deprecation notice:
> https://patches.dpdk.org/project/dpdk/patch/20240730144612.2132848-1-amitprakashs@marvell.com/
>
> * Assuming we do not anticipate any advanced scheduling schemes for dmadev queues,
> this RFC is intended to support a strict prioirty scheme.
>
> doc/guides/rel_notes/release_24_11.rst | 15 +++------------
> lib/dmadev/rte_dmadev.c | 10 ++++++++++
> lib/dmadev/rte_dmadev.h | 11 +++++++++++
> 3 files changed, 24 insertions(+), 12 deletions(-)
>
> diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
> index 0ff70d9057..dc711bbf98 100644
> --- a/doc/guides/rel_notes/release_24_11.rst
> +++ b/doc/guides/rel_notes/release_24_11.rst
> @@ -88,18 +88,9 @@ API Changes
> ABI Changes
> -----------
>
> -.. This section should contain ABI changes. Sample format:
> -
> - * sample: Add a short 1-2 sentence description of the ABI change
> - which was announced in the previous releases and made in this release.
> - Start with a scope label like "ethdev:".
> - Use fixed width quotes for ``function_names`` or ``struct_names``.
> - Use the past tense.
> -
> - This section is a comment. Do not overwrite or remove it.
> - Also, make sure to start the actual text at the margin.
> - =======================================================
> -
> +* dmadev: Added ``nb_priorities`` field to ``rte_dma_info`` structure and
> + ``priority`` field to ``rte_dma_conf`` structure to get device supported
> + priority levels and configure required priority from the application.
>
> Known Issues
> ------------
> diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
> index 845727210f..053170f55a 100644
> --- a/lib/dmadev/rte_dmadev.c
> +++ b/lib/dmadev/rte_dmadev.c
> @@ -497,6 +497,16 @@ rte_dma_configure(int16_t dev_id, const struct rte_dma_conf *dev_conf)
> return -EINVAL;
> }
>
> + if (dev_info.nb_priorities == 1) {
> + RTE_DMA_LOG(ERR, "Device %d must support more than 1 priority, or else 0", dev_id);
> + return -EINVAL;
> + }
> +
> + if (dev_info.nb_priorities && (dev_conf->priority >= dev_info.nb_priorities)) {
> + RTE_DMA_LOG(ERR, "Device %d configure invalid priority", dev_id);
> + return -EINVAL;
> + }
Please add trace support
> +
> if (*dev->dev_ops->dev_configure == NULL)
> return -ENOTSUP;
> ret = (*dev->dev_ops->dev_configure)(dev, dev_conf,
> diff --git a/lib/dmadev/rte_dmadev.h b/lib/dmadev/rte_dmadev.h
> index 5474a5281d..d59fcf1b61 100644
> --- a/lib/dmadev/rte_dmadev.h
> +++ b/lib/dmadev/rte_dmadev.h
> @@ -297,6 +297,10 @@ struct rte_dma_info {
> int16_t numa_node;
> /** Number of virtual DMA channel configured. */
> uint16_t nb_vchans;
> + /** Number of priority levels (must be > 1), if supported by DMA HW channel.
> + * 0 otherwise.
> + */
> + uint16_t nb_priorities;
Suggest add priority schedule policy capability, for example:
RTE_DMA_CAPA_PRIORITY_POLICY_SP
> };
>
> /**
> @@ -332,6 +336,13 @@ struct rte_dma_conf {
> * @see RTE_DMA_CAPA_SILENT
> */
> bool enable_silent;
> + /* The prioirty of the DMA HW channel.
> + * This value cannot be greater than or equal to the field 'nb_priorities'
> + * of struct rte_dma_info which get from rte_dma_info_get().
> + * Among the values between '0' and 'nb_priorities - 1', lowest value
> + * indicates higher priority and vice-versa.
> + */
> + uint16_t priority;
> };
>
> /**
^ permalink raw reply [relevance 0%]
* [PATCH v7 1/8] ethdev: support report register names and filter
@ 2024-09-14 7:13 4% ` Jie Hai
0 siblings, 0 replies; 200+ results
From: Jie Hai @ 2024-09-14 7:13 UTC (permalink / raw)
To: dev, thomas, ferruh.yigit, andrew.rybchenko
Cc: lihuisong, fengchengwen, haijie1
This patch adds "filter" and "names" fields to "rte_dev_reg_info"
structure. Names of registers in data fields can be reported and
the registers can be filtered by their module names.
The new API rte_eth_dev_get_reg_info_ext() is added to support
reporting names and filtering by modules. And the original API
rte_eth_dev_get_reg_info() does not use the names and filter fields.
A local variable is used in rte_eth_dev_get_reg_info for
compatibility. If the drivers does not report the names, set them
to "index_XXX", which means the location in the register table.
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Huisong Li <lihuisong@huawei.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>
---
doc/guides/rel_notes/release_24_11.rst | 9 ++++++
lib/ethdev/ethdev_trace.h | 2 ++
lib/ethdev/rte_dev_info.h | 11 ++++++++
lib/ethdev/rte_ethdev.c | 38 ++++++++++++++++++++++++++
lib/ethdev/rte_ethdev.h | 29 ++++++++++++++++++++
lib/ethdev/version.map | 3 ++
6 files changed, 92 insertions(+)
mode change 100644 => 100755 doc/guides/rel_notes/release_24_11.rst
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
old mode 100644
new mode 100755
index 0ff70d90578e..d53ab8fa5bc7
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -55,6 +55,12 @@ New Features
Also, make sure to start the actual text at the margin.
=======================================================
+* **Added support for dumping registers with names and filtering by modules.**
+
+ Added new API functions ``rte_eth_dev_get_reg_info_ext()`` to filter the
+ registers by module names and get the information (names, values and other
+ attributes) of the filtered registers.
+
Removed Items
-------------
@@ -100,6 +106,9 @@ ABI Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* ethdev: Added ``filter`` and ``names`` fields to ``rte_dev_reg_info``
+ structure for filtering by modules and reporting names of registers.
+
Known Issues
------------
diff --git a/lib/ethdev/ethdev_trace.h b/lib/ethdev/ethdev_trace.h
index 3bec87bfdb70..0c4780a09ef5 100644
--- a/lib/ethdev/ethdev_trace.h
+++ b/lib/ethdev/ethdev_trace.h
@@ -1152,6 +1152,8 @@ RTE_TRACE_POINT(
rte_trace_point_emit_u32(info->length);
rte_trace_point_emit_u32(info->width);
rte_trace_point_emit_u32(info->version);
+ rte_trace_point_emit_ptr(info->names);
+ rte_trace_point_emit_ptr(info->filter);
rte_trace_point_emit_int(ret);
)
diff --git a/lib/ethdev/rte_dev_info.h b/lib/ethdev/rte_dev_info.h
index 67cf0ae52668..26b777f9836e 100644
--- a/lib/ethdev/rte_dev_info.h
+++ b/lib/ethdev/rte_dev_info.h
@@ -11,6 +11,11 @@ extern "C" {
#include <stdint.h>
+#define RTE_ETH_REG_NAME_SIZE 64
+struct rte_eth_reg_name {
+ char name[RTE_ETH_REG_NAME_SIZE];
+};
+
/*
* Placeholder for accessing device registers
*/
@@ -20,6 +25,12 @@ struct rte_dev_reg_info {
uint32_t length; /**< Number of registers to fetch */
uint32_t width; /**< Size of device register */
uint32_t version; /**< Device version */
+ /**
+ * Name of target module, filter for target subset of registers.
+ * This field could affects register selection for data/length/names.
+ */
+ const char *filter;
+ struct rte_eth_reg_name *names; /**< Registers name saver */
};
/*
diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
index f1c658f49e80..30ca4a0043c5 100644
--- a/lib/ethdev/rte_ethdev.c
+++ b/lib/ethdev/rte_ethdev.c
@@ -6388,8 +6388,37 @@ rte_eth_read_clock(uint16_t port_id, uint64_t *clock)
int
rte_eth_dev_get_reg_info(uint16_t port_id, struct rte_dev_reg_info *info)
+{
+ struct rte_dev_reg_info reg_info = { 0 };
+ int ret;
+
+ if (info == NULL) {
+ RTE_ETHDEV_LOG_LINE(ERR,
+ "Cannot get ethdev port %u register info to NULL",
+ port_id);
+ return -EINVAL;
+ }
+
+ reg_info.length = info->length;
+ reg_info.data = info->data;
+
+ ret = rte_eth_dev_get_reg_info_ext(port_id, ®_info);
+ if (ret != 0)
+ return ret;
+
+ info->length = reg_info.length;
+ info->width = reg_info.width;
+ info->version = reg_info.version;
+ info->offset = reg_info.offset;
+
+ return 0;
+}
+
+int
+rte_eth_dev_get_reg_info_ext(uint16_t port_id, struct rte_dev_reg_info *info)
{
struct rte_eth_dev *dev;
+ uint32_t i;
int ret;
RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
@@ -6402,12 +6431,21 @@ rte_eth_dev_get_reg_info(uint16_t port_id, struct rte_dev_reg_info *info)
return -EINVAL;
}
+ if (info->names != NULL && info->length != 0)
+ memset(info->names, 0, sizeof(struct rte_eth_reg_name) * info->length);
+
if (*dev->dev_ops->get_reg == NULL)
return -ENOTSUP;
ret = eth_err(port_id, (*dev->dev_ops->get_reg)(dev, info));
rte_ethdev_trace_get_reg_info(port_id, info, ret);
+ /* Report the default names if drivers not report. */
+ if (ret == 0 && info->names != NULL && strlen(info->names[0].name) == 0) {
+ for (i = 0; i < info->length; i++)
+ snprintf(info->names[i].name, RTE_ETH_REG_NAME_SIZE,
+ "index_%u", info->offset + i);
+ }
return ret;
}
diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
index 548fada1c7ad..02cb3c07f742 100644
--- a/lib/ethdev/rte_ethdev.h
+++ b/lib/ethdev/rte_ethdev.h
@@ -5071,6 +5071,35 @@ __rte_experimental
int rte_eth_get_monitor_addr(uint16_t port_id, uint16_t queue_id,
struct rte_power_monitor_cond *pmc);
+/**
+ * Retrieve the filtered device registers (values and names) and
+ * register attributes (number of registers and register size)
+ *
+ * @param port_id
+ * The port identifier of the Ethernet device.
+ * @param info
+ * Pointer to rte_dev_reg_info structure to fill in.
+ * - If info->filter is NULL, return info for all registers (seen as filter
+ * none).
+ * - If info->filter is not NULL, return error if the driver does not support
+ * filter. Fill the length field with filtered register number.
+ * - If info->data is NULL, the function fills in the width and length fields.
+ * - If info->data is not NULL, ethdev considers there are enough spaces to
+ * store the registers, and the values of registers with the filter string
+ * as the module name are put into the buffer pointed at by info->data.
+ * - If info->names is not NULL, drivers should fill it or the ethdev fills it
+ * with default names.
+ * @return
+ * - (0) if successful.
+ * - (-ENOTSUP) if hardware doesn't support.
+ * - (-EINVAL) if bad parameter.
+ * - (-ENODEV) if *port_id* invalid.
+ * - (-EIO) if device is removed.
+ * - others depends on the specific operations implementation.
+ */
+__rte_experimental
+int rte_eth_dev_get_reg_info_ext(uint16_t port_id, struct rte_dev_reg_info *info);
+
/**
* Retrieve device registers and register attributes (number of registers and
* register size)
diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
index 1669055ca566..abc2ff5bb4d5 100644
--- a/lib/ethdev/version.map
+++ b/lib/ethdev/version.map
@@ -325,6 +325,9 @@ EXPERIMENTAL {
rte_flow_template_table_resizable;
rte_flow_template_table_resize;
rte_flow_template_table_resize_complete;
+
+ # added in 24.07
+ rte_eth_dev_get_reg_info_ext;
};
INTERNAL {
--
2.22.0
^ permalink raw reply [relevance 4%]
* Re: [PATCH v3 1/1] bbdev: removing unnecessary symbols from version map
2024-08-27 23:06 3% ` [PATCH v3 1/1] bbdev: removing unnecessary symbols from version map Nicolas Chautru
@ 2024-09-13 13:51 0% ` Maxime Coquelin
2024-09-19 8:26 0% ` Maxime Coquelin
1 sibling, 0 replies; 200+ results
From: Maxime Coquelin @ 2024-09-13 13:51 UTC (permalink / raw)
To: Nicolas Chautru, dev; +Cc: hemant.agrawal, david.marchand, hernan.vargas
On 8/28/24 01:06, Nicolas Chautru wrote:
> A number of inline functions should not be in the
> version map since ABI versioning would be irrelevant.
>
> Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
> ---
> lib/bbdev/rte_bbdev.h | 1 -
> lib/bbdev/rte_bbdev_op.h | 2 --
> lib/bbdev/version.map | 24 +-----------------------
> 3 files changed, 1 insertion(+), 26 deletions(-)
>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Thanks,
Maxime
^ permalink raw reply [relevance 0%]
* Re: [PATCH v2 0/3] bbdev: sdditional queue stats
2024-08-12 23:41 3% [PATCH v2 0/3] bbdev: sdditional queue stats Nicolas Chautru
2024-08-12 23:41 6% ` [PATCH v2 1/3] bbdev: new queue stat for available enqueue depth Nicolas Chautru
2024-08-13 7:22 0% ` [PATCH v2 0/3] bbdev: sdditional queue stats Hemant Agrawal
@ 2024-09-13 13:50 0% ` Maxime Coquelin
2 siblings, 0 replies; 200+ results
From: Maxime Coquelin @ 2024-09-13 13:50 UTC (permalink / raw)
To: Nicolas Chautru, dev; +Cc: hemant.agrawal, david.marchand, hernan.vargas
On 8/13/24 01:41, Nicolas Chautru wrote:
> v2: update to ABI doc suggested by Maxime.
>
> These series include introducing a new paramter in the queue stat
> which can be used to monitor the number of available enqueue
> still possible.
> The acc PMD is then refactored to use a set of common function
> to update several queue status parameters including the new one.
> The application is also updated.
> Thanks
> Nic
>
> Nicolas Chautru (3):
> bbdev: new queue stat for available enqueue depth
> baseband/acc: refactor queue status update
> test/bbdev: update for queue stats
>
> app/test-bbdev/test_bbdev_perf.c | 1 +
> doc/guides/rel_notes/release_24_11.rst | 3 ++
> drivers/baseband/acc/acc_common.h | 18 ++++++++
> drivers/baseband/acc/rte_acc100_pmd.c | 45 ++++++-------------
> drivers/baseband/acc/rte_vrb_pmd.c | 61 ++++++++------------------
> lib/bbdev/rte_bbdev.h | 2 +
> 6 files changed, 56 insertions(+), 74 deletions(-)
>
Applied to dpdk-next-baseband/for-main
Thanks,
Maxime
^ permalink raw reply [relevance 0%]
* [RFC v0 1/1] dmadev: provide priority configuration support
@ 2024-09-13 12:10 11% Vamsi Krishna
2024-09-14 9:41 0% ` fengchengwen
2024-09-18 9:40 10% ` [RFC v1 1/1] dmadev: support priority configuration Vamsi Krishna
0 siblings, 2 replies; 200+ results
From: Vamsi Krishna @ 2024-09-13 12:10 UTC (permalink / raw)
To: thomas, fengchengwen, bruce.richardson, mb
Cc: dev, kevin.laatz, jerinj, conor.walsh, gmuthukrishn, vvelumuri,
g.singh, sachin.saxena, hemant.agrawal, Vamsi Attunuru,
Amit Prakash Shukla
From: Vamsi Attunuru <vattunuru@marvell.com>
Some DMA controllers offer the ability to configure priority level
for the hardware command queues, allowing for the prioritization of
DMA command execution based on queue importance.
This patch introduces the necessary fields in the dmadev structures to
retrieve information about the hardware-supported priority levels and to
enable priority configuration from the application.
Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
---
Deprecation notice:
https://patches.dpdk.org/project/dpdk/patch/20240730144612.2132848-1-amitprakashs@marvell.com/
* Assuming we do not anticipate any advanced scheduling schemes for dmadev queues,
this RFC is intended to support a strict prioirty scheme.
doc/guides/rel_notes/release_24_11.rst | 15 +++------------
lib/dmadev/rte_dmadev.c | 10 ++++++++++
lib/dmadev/rte_dmadev.h | 11 +++++++++++
3 files changed, 24 insertions(+), 12 deletions(-)
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 0ff70d9057..dc711bbf98 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -88,18 +88,9 @@ API Changes
ABI Changes
-----------
-.. This section should contain ABI changes. Sample format:
-
- * sample: Add a short 1-2 sentence description of the ABI change
- which was announced in the previous releases and made in this release.
- Start with a scope label like "ethdev:".
- Use fixed width quotes for ``function_names`` or ``struct_names``.
- Use the past tense.
-
- This section is a comment. Do not overwrite or remove it.
- Also, make sure to start the actual text at the margin.
- =======================================================
-
+* dmadev: Added ``nb_priorities`` field to ``rte_dma_info`` structure and
+ ``priority`` field to ``rte_dma_conf`` structure to get device supported
+ priority levels and configure required priority from the application.
Known Issues
------------
diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
index 845727210f..053170f55a 100644
--- a/lib/dmadev/rte_dmadev.c
+++ b/lib/dmadev/rte_dmadev.c
@@ -497,6 +497,16 @@ rte_dma_configure(int16_t dev_id, const struct rte_dma_conf *dev_conf)
return -EINVAL;
}
+ if (dev_info.nb_priorities == 1) {
+ RTE_DMA_LOG(ERR, "Device %d must support more than 1 priority, or else 0", dev_id);
+ return -EINVAL;
+ }
+
+ if (dev_info.nb_priorities && (dev_conf->priority >= dev_info.nb_priorities)) {
+ RTE_DMA_LOG(ERR, "Device %d configure invalid priority", dev_id);
+ return -EINVAL;
+ }
+
if (*dev->dev_ops->dev_configure == NULL)
return -ENOTSUP;
ret = (*dev->dev_ops->dev_configure)(dev, dev_conf,
diff --git a/lib/dmadev/rte_dmadev.h b/lib/dmadev/rte_dmadev.h
index 5474a5281d..d59fcf1b61 100644
--- a/lib/dmadev/rte_dmadev.h
+++ b/lib/dmadev/rte_dmadev.h
@@ -297,6 +297,10 @@ struct rte_dma_info {
int16_t numa_node;
/** Number of virtual DMA channel configured. */
uint16_t nb_vchans;
+ /** Number of priority levels (must be > 1), if supported by DMA HW channel.
+ * 0 otherwise.
+ */
+ uint16_t nb_priorities;
};
/**
@@ -332,6 +336,13 @@ struct rte_dma_conf {
* @see RTE_DMA_CAPA_SILENT
*/
bool enable_silent;
+ /* The prioirty of the DMA HW channel.
+ * This value cannot be greater than or equal to the field 'nb_priorities'
+ * of struct rte_dma_info which get from rte_dma_info_get().
+ * Among the values between '0' and 'nb_priorities - 1', lowest value
+ * indicates higher priority and vice-versa.
+ */
+ uint16_t priority;
};
/**
--
2.34.1
^ permalink raw reply [relevance 11%]
* RE: [PATCH v2 1/6] eal: add static per-lcore memory allocation facility
@ 2024-09-12 9:10 3% ` Morten Brørup
0 siblings, 0 replies; 200+ results
From: Morten Brørup @ 2024-09-12 9:10 UTC (permalink / raw)
To: Mattias Rönnblom, dev, Jerin Jacob, Chengwen Feng
Cc: Mattias Rönnblom, Stephen Hemminger, Konstantin Ananyev,
David Marchand, Anatoly Burakov
> +#define LCORE_BUFFER_SIZE (RTE_MAX_LCORE_VAR * RTE_MAX_LCORE)
Considering hugepages...
Lcore variables may be allocated before DPDK's memory allocator (rte_malloc()) is ready, so rte_malloc() cannot be used for lcore variables.
And lcore variables are not usable (shared) for DPDK multi-process, so the lcore_buffer could be allocated through the O/S APIs as anonymous hugepages, instead of using rte_malloc().
The alternative, using rte_malloc(), would disallow allocating lcore variables before DPDK's memory allocator has been initialized, which I think is too late.
Anyway, hugepages is not a "must have" here, it is a "nice to have". It can be added to the lcore variables subsystem at a later time.
Here are some thoughts about optimizing for TLB entry usage...
If lcore variables use hugepages, and LCORE_BUFFER_SIZE matches the hugepage size (2 MB), all the lcore variables will only consume 1 hugepage TLB entry.
However, this may limit the max size of an lcore variable (RTE_MAX_LCORE_VAR) too much, if the system supports many lcores (RTE_MAX_LCORE).
E.g. with 1024 lcores, the max size of an lcore variable would be 2048 bytes.
And with 128 lcores, the max size of an lcore variable would be 16 KB.
So if we want to optimize for hugepage TLB entry usage, the question becomes: What is a reasonable max size of an lcore variable?
And although hugepages is only a "nice to have", the max size of an lcore variable (RTE_MAX_LCORE_VAR) is part of the API/ABI, so we should consider it now, if we want to optimize for hugepage TLB entry usage in the future.
A few more comments below, not related to hugepages.
> +
> +static void *lcore_buffer;
> +static size_t offset = RTE_MAX_LCORE_VAR;
> +
> +static void *
> +lcore_var_alloc(size_t size, size_t align)
> +{
> + void *handle;
> + void *value;
> +
> + offset = RTE_ALIGN_CEIL(offset, align);
> +
> + if (offset + size > RTE_MAX_LCORE_VAR) {
> +#ifdef RTE_EXEC_ENV_WINDOWS
> + lcore_buffer = _aligned_malloc(LCORE_BUFFER_SIZE,
> + RTE_CACHE_LINE_SIZE);
> +#else
> + lcore_buffer = aligned_alloc(RTE_CACHE_LINE_SIZE,
> + LCORE_BUFFER_SIZE);
> +#endif
> + RTE_VERIFY(lcore_buffer != NULL);
> +
> + offset = 0;
> + }
> +
> + handle = RTE_PTR_ADD(lcore_buffer, offset);
> +
> + offset += size;
> +
> + RTE_LCORE_VAR_FOREACH_VALUE(value, handle)
> + memset(value, 0, size);
> +
> + EAL_LOG(DEBUG, "Allocated %"PRIuPTR" bytes of per-lcore data with
> a "
> + "%"PRIuPTR"-byte alignment", size, align);
> +
> + return handle;
> +}
> +
> +void *
> +rte_lcore_var_alloc(size_t size, size_t align)
> +{
> + /* Having the per-lcore buffer size aligned on cache lines
> + * assures as well as having the base pointer aligned on cache
> + * size assures that aligned offsets also translate to alipgned
> + * pointers across all values.
> + */
> + RTE_BUILD_BUG_ON(RTE_MAX_LCORE_VAR % RTE_CACHE_LINE_SIZE != 0);
> + RTE_ASSERT(align <= RTE_CACHE_LINE_SIZE);
> + RTE_ASSERT(size <= RTE_MAX_LCORE_VAR);
This specific RTE_ASSERT() should be upgraded to RTE_VERIFY(), so it is checked in non-debug builds too.
The code is slow path and not inline, and if this check doesn't pass, accessing the lcore variable will cause a buffer overrun. Prefer failing early.
> +
> + /* '0' means asking for worst-case alignment requirements */
> + if (align == 0)
> + align = alignof(max_align_t);
> +
> + RTE_ASSERT(rte_is_power_of_2(align));
> +
> + return lcore_var_alloc(size, align);
> +}
> +/**
> + * Allocate space in the per-lcore id buffers for an lcore variable.
> + *
> + * The pointer returned is only an opaque identifer of the variable. To
> + * get an actual pointer to a particular instance of the variable use
> + * @ref RTE_LCORE_VAR_VALUE or @ref RTE_LCORE_VAR_LCORE_VALUE.
> + *
> + * The lcore variable values' memory is set to zero.
> + *
> + * The allocation is always successful, barring a fatal exhaustion of
> + * the per-lcore id buffer space.
> + *
> + * rte_lcore_var_alloc() is not multi-thread safe.
> + *
> + * @param size
> + * The size (in bytes) of the variable's per-lcore id value. Must be
> > 0.
> + * @param align
> + * If 0, the values will be suitably aligned for any kind of type
> + * (i.e., alignof(max_align_t)). Otherwise, the values will be
> aligned
> + * on a multiple of *align*, which must be a power of 2 and equal or
> + * less than @c RTE_CACHE_LINE_SIZE.
> + * @return
> + * The variable's handle, stored in a void pointer value. The value
> + * is always non-NULL.
> + */
> +__rte_experimental
I don't know how useful these are, but consider adding:
#ifndef RTE_TOOLCHAIN_MSVC
__attribute__((malloc))
__attribute__((alloc_size(1)))
__attribute__((alloc_align(2)))
__attribute__((returns_nonnull))
#endif
> +void *
> +rte_lcore_var_alloc(size_t size, size_t align);
^ permalink raw reply [relevance 3%]
* Re: [PATCH v10 0/2] power: introduce PM QoS interface
2024-09-12 2:38 4% ` [PATCH v10 0/2] power: introduce PM QoS interface Huisong Li
2024-09-12 2:38 5% ` [PATCH v10 1/2] power: introduce PM QoS API on CPU wide Huisong Li
@ 2024-09-12 3:07 0% ` fengchengwen
1 sibling, 0 replies; 200+ results
From: fengchengwen @ 2024-09-12 3:07 UTC (permalink / raw)
To: Huisong Li, dev
Cc: mb, thomas, ferruh.yigit, anatoly.burakov, david.hunt,
sivaprasad.tummala, stephen, david.marchand, liuyonglong
Series-reviewed-by: Chengwen Feng <fengchengwen@huawei.com>
On 2024/9/12 10:38, Huisong Li wrote:
> The deeper the idle state, the lower the power consumption, but the longer
> the resume time. Some service are delay sensitive and very except the low
> resume time, like interrupt packet receiving mode.
>
> And the "/sys/devices/system/cpu/cpuX/power/pm_qos_resume_latency_us" sysfs
> interface is used to set and get the resume latency limit on the cpuX for
> userspace. Please see the description in kernel document[1].
> Each cpuidle governor in Linux select which idle state to enter based on
> this CPU resume latency in their idle task.
>
> The per-CPU PM QoS API can be used to control this CPU's idle state
> selection and limit just enter the shallowest idle state to low the delay
> after sleep by setting strict resume latency (zero value).
>
> [1] https://www.kernel.org/doc/html/latest/admin-guide/abi-testing.html?highlight=pm_qos_resume_latency_us#abi-sys-devices-power-pm-qos-resume-latency-us
>
> ---
> v10:
> - replace LINE_MAX with a custom macro and fix two typos.
> v9:
> - move new feature description from release_24_07.rst to release_24_11.rst.
> v8:
> - update the latest code to resolve CI warning
> v7:
> - remove a dead code rte_lcore_is_enabled in patch[2/2]
> v6:
> - update release_24_07.rst based on dpdk repo to resolve CI warning.
> v5:
> - use LINE_MAX to replace BUFSIZ, and use snprintf to replace sprintf.
> v4:
> - fix some comments basd on Stephen
> - add stdint.h include
> - add Acked-by Morten Brørup <mb@smartsharesystems.com>
> v3:
> - add RTE_POWER_xxx prefix for some macro in header
> - add the check for lcore_id with rte_lcore_is_enabled
> v2:
> - use PM QoS on CPU wide to replace the one on system wide
>
> Huisong Li (2):
> power: introduce PM QoS API on CPU wide
> examples/l3fwd-power: add PM QoS configuration
>
> doc/guides/prog_guide/power_man.rst | 24 ++++++
> doc/guides/rel_notes/release_24_11.rst | 5 ++
> examples/l3fwd-power/main.c | 24 ++++++
> lib/power/meson.build | 2 +
> lib/power/rte_power_qos.c | 111 +++++++++++++++++++++++++
> lib/power/rte_power_qos.h | 73 ++++++++++++++++
> lib/power/version.map | 4 +
> 7 files changed, 243 insertions(+)
> create mode 100644 lib/power/rte_power_qos.c
> create mode 100644 lib/power/rte_power_qos.h
>
^ permalink raw reply [relevance 0%]
* [PATCH v10 1/2] power: introduce PM QoS API on CPU wide
2024-09-12 2:38 4% ` [PATCH v10 0/2] power: introduce PM QoS interface Huisong Li
@ 2024-09-12 2:38 5% ` Huisong Li
2024-09-12 3:07 0% ` [PATCH v10 0/2] power: introduce PM QoS interface fengchengwen
1 sibling, 0 replies; 200+ results
From: Huisong Li @ 2024-09-12 2:38 UTC (permalink / raw)
To: dev
Cc: mb, thomas, ferruh.yigit, anatoly.burakov, david.hunt,
sivaprasad.tummala, stephen, david.marchand, fengchengwen,
liuyonglong, lihuisong
The deeper the idle state, the lower the power consumption, but the longer
the resume time. Some service are delay sensitive and very except the low
resume time, like interrupt packet receiving mode.
And the "/sys/devices/system/cpu/cpuX/power/pm_qos_resume_latency_us" sysfs
interface is used to set and get the resume latency limit on the cpuX for
userspace. Each cpuidle governor in Linux select which idle state to enter
based on this CPU resume latency in their idle task.
The per-CPU PM QoS API can be used to control this CPU's idle state
selection and limit just enter the shallowest idle state to low the delay
after sleep by setting strict resume latency (zero value).
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
doc/guides/prog_guide/power_man.rst | 24 ++++++
doc/guides/rel_notes/release_24_11.rst | 5 ++
lib/power/meson.build | 2 +
lib/power/rte_power_qos.c | 111 +++++++++++++++++++++++++
lib/power/rte_power_qos.h | 73 ++++++++++++++++
lib/power/version.map | 4 +
6 files changed, 219 insertions(+)
create mode 100644 lib/power/rte_power_qos.c
create mode 100644 lib/power/rte_power_qos.h
diff --git a/doc/guides/prog_guide/power_man.rst b/doc/guides/prog_guide/power_man.rst
index f6674efe2d..faa32b4320 100644
--- a/doc/guides/prog_guide/power_man.rst
+++ b/doc/guides/prog_guide/power_man.rst
@@ -249,6 +249,30 @@ Get Num Pkgs
Get Num Dies
Get the number of die's on a given package.
+
+PM QoS
+------
+
+The deeper the idle state, the lower the power consumption, but the longer
+the resume time. Some service are delay sensitive and very except the low
+resume time, like interrupt packet receiving mode.
+
+And the "/sys/devices/system/cpu/cpuX/power/pm_qos_resume_latency_us" sysfs
+interface is used to set and get the resume latency limit on the cpuX for
+userspace. Each cpuidle governor in Linux select which idle state to enter
+based on this CPU resume latency in their idle task.
+
+The per-CPU PM QoS API can be used to set and get the CPU resume latency based
+on this sysfs.
+
+The ``rte_power_qos_set_cpu_resume_latency()`` function can control the CPU's
+idle state selection in Linux and limit just to enter the shallowest idle state
+to low the delay of resuming service after sleeping by setting strict resume
+latency (zero value).
+
+The ``rte_power_qos_get_cpu_resume_latency()`` function can get the resume
+latency on specified CPU.
+
References
----------
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 0ff70d9057..bd72d0a595 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -55,6 +55,11 @@ New Features
Also, make sure to start the actual text at the margin.
=======================================================
+* **Introduce per-CPU PM QoS interface.**
+
+ * Add per-CPU PM QoS interface to low the delay after sleep by controlling
+ CPU idle state selection.
+
Removed Items
-------------
diff --git a/lib/power/meson.build b/lib/power/meson.build
index b8426589b2..8222e178b0 100644
--- a/lib/power/meson.build
+++ b/lib/power/meson.build
@@ -23,12 +23,14 @@ sources = files(
'rte_power.c',
'rte_power_uncore.c',
'rte_power_pmd_mgmt.c',
+ 'rte_power_qos.c',
)
headers = files(
'rte_power.h',
'rte_power_guest_channel.h',
'rte_power_pmd_mgmt.h',
'rte_power_uncore.h',
+ 'rte_power_qos.h',
)
if cc.has_argument('-Wno-cast-qual')
cflags += '-Wno-cast-qual'
diff --git a/lib/power/rte_power_qos.c b/lib/power/rte_power_qos.c
new file mode 100644
index 0000000000..8eb26cd41a
--- /dev/null
+++ b/lib/power/rte_power_qos.c
@@ -0,0 +1,111 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2024 HiSilicon Limited
+ */
+
+#include <errno.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <rte_lcore.h>
+#include <rte_log.h>
+
+#include "power_common.h"
+#include "rte_power_qos.h"
+
+#define PM_QOS_SYSFILE_RESUME_LATENCY_US \
+ "/sys/devices/system/cpu/cpu%u/power/pm_qos_resume_latency_us"
+
+#define PM_QOS_CPU_RESUME_LATENCY_BUF_LEN 32
+
+int
+rte_power_qos_set_cpu_resume_latency(uint16_t lcore_id, int latency)
+{
+ char buf[PM_QOS_CPU_RESUME_LATENCY_BUF_LEN];
+ FILE *f;
+ int ret;
+
+ if (!rte_lcore_is_enabled(lcore_id)) {
+ POWER_LOG(ERR, "lcore id %u is not enabled", lcore_id);
+ return -EINVAL;
+ }
+
+ if (latency < 0) {
+ POWER_LOG(ERR, "latency should be greater than and equal to 0");
+ return -EINVAL;
+ }
+
+ ret = open_core_sysfs_file(&f, "w", PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
+ if (ret != 0) {
+ POWER_LOG(ERR, "Failed to open "PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
+ return ret;
+ }
+
+ /*
+ * Based on the sysfs interface pm_qos_resume_latency_us under
+ * @PM_QOS_SYSFILE_RESUME_LATENCY_US directory in kernel, their meaning
+ * is as follows for different input string.
+ * 1> the resume latency is 0 if the input is "n/a".
+ * 2> the resume latency is no constraint if the input is "0".
+ * 3> the resume latency is the actual value to be set.
+ */
+ if (latency == 0)
+ snprintf(buf, sizeof(buf), "%s", "n/a");
+ else if (latency == RTE_POWER_QOS_RESUME_LATENCY_NO_CONSTRAINT)
+ snprintf(buf, sizeof(buf), "%u", 0);
+ else
+ snprintf(buf, sizeof(buf), "%u", latency);
+
+ ret = write_core_sysfs_s(f, buf);
+ if (ret != 0)
+ POWER_LOG(ERR, "Failed to write "PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
+
+ fclose(f);
+
+ return ret;
+}
+
+int
+rte_power_qos_get_cpu_resume_latency(uint16_t lcore_id)
+{
+ char buf[PM_QOS_CPU_RESUME_LATENCY_BUF_LEN];
+ int latency = -1;
+ FILE *f;
+ int ret;
+
+ if (!rte_lcore_is_enabled(lcore_id)) {
+ POWER_LOG(ERR, "lcore id %u is not enabled", lcore_id);
+ return -EINVAL;
+ }
+
+ ret = open_core_sysfs_file(&f, "r", PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
+ if (ret != 0) {
+ POWER_LOG(ERR, "Failed to open "PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
+ return ret;
+ }
+
+ ret = read_core_sysfs_s(f, buf, sizeof(buf));
+ if (ret != 0) {
+ POWER_LOG(ERR, "Failed to read "PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
+ goto out;
+ }
+
+ /*
+ * Based on the sysfs interface pm_qos_resume_latency_us under
+ * @PM_QOS_SYSFILE_RESUME_LATENCY_US directory in kernel, their meaning
+ * is as follows for different output string.
+ * 1> the resume latency is 0 if the output is "n/a".
+ * 2> the resume latency is no constraint if the output is "0".
+ * 3> the resume latency is the actual value in used for other string.
+ */
+ if (strcmp(buf, "n/a") == 0)
+ latency = 0;
+ else {
+ latency = strtoul(buf, NULL, 10);
+ latency = latency == 0 ? RTE_POWER_QOS_RESUME_LATENCY_NO_CONSTRAINT : latency;
+ }
+
+out:
+ fclose(f);
+
+ return latency != -1 ? latency : ret;
+}
diff --git a/lib/power/rte_power_qos.h b/lib/power/rte_power_qos.h
new file mode 100644
index 0000000000..990c488373
--- /dev/null
+++ b/lib/power/rte_power_qos.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2024 HiSilicon Limited
+ */
+
+#ifndef RTE_POWER_QOS_H
+#define RTE_POWER_QOS_H
+
+#include <stdint.h>
+
+#include <rte_compat.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * @file rte_power_qos.h
+ *
+ * PM QoS API.
+ *
+ * The CPU-wide resume latency limit has a positive impact on this CPU's idle
+ * state selection in each cpuidle governor.
+ * Please see the PM QoS on CPU wide in the following link:
+ * https://www.kernel.org/doc/html/latest/admin-guide/abi-testing.html?highlight=pm_qos_resume_latency_us#abi-sys-devices-power-pm-qos-resume-latency-us
+ *
+ * The deeper the idle state, the lower the power consumption, but the
+ * longer the resume time. Some service are delay sensitive and very except the
+ * low resume time, like interrupt packet receiving mode.
+ *
+ * In these case, per-CPU PM QoS API can be used to control this CPU's idle
+ * state selection and limit just enter the shallowest idle state to low the
+ * delay after sleep by setting strict resume latency (zero value).
+ */
+
+#define RTE_POWER_QOS_STRICT_LATENCY_VALUE 0
+#define RTE_POWER_QOS_RESUME_LATENCY_NO_CONSTRAINT ((int)(UINT32_MAX >> 1))
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * @param lcore_id
+ * target logical core id
+ *
+ * @param latency
+ * The latency should be greater than and equal to zero in microseconds unit.
+ *
+ * @return
+ * 0 on success. Otherwise negative value is returned.
+ */
+__rte_experimental
+int rte_power_qos_set_cpu_resume_latency(uint16_t lcore_id, int latency);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get the current resume latency of this logical core.
+ * The default value in kernel is @see RTE_POWER_QOS_RESUME_LATENCY_NO_CONSTRAINT
+ * if don't set it.
+ *
+ * @return
+ * Negative value on failure.
+ * >= 0 means the actual resume latency limit on this core.
+ */
+__rte_experimental
+int rte_power_qos_get_cpu_resume_latency(uint16_t lcore_id);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* RTE_POWER_QOS_H */
diff --git a/lib/power/version.map b/lib/power/version.map
index c9a226614e..08f178a39d 100644
--- a/lib/power/version.map
+++ b/lib/power/version.map
@@ -51,4 +51,8 @@ EXPERIMENTAL {
rte_power_set_uncore_env;
rte_power_uncore_freqs;
rte_power_unset_uncore_env;
+
+ # added in 24.11
+ rte_power_qos_get_cpu_resume_latency;
+ rte_power_qos_set_cpu_resume_latency;
};
--
2.22.0
^ permalink raw reply [relevance 5%]
* [PATCH v10 0/2] power: introduce PM QoS interface
2024-08-09 9:50 4% ` [PATCH v9 0/2] power: " Huisong Li
@ 2024-09-12 2:38 4% ` Huisong Li
2024-09-12 2:38 5% ` [PATCH v10 1/2] power: introduce PM QoS API on CPU wide Huisong Li
2024-09-12 3:07 0% ` [PATCH v10 0/2] power: introduce PM QoS interface fengchengwen
1 sibling, 2 replies; 200+ results
From: Huisong Li @ 2024-09-12 2:38 UTC (permalink / raw)
To: dev
Cc: mb, thomas, ferruh.yigit, anatoly.burakov, david.hunt,
sivaprasad.tummala, stephen, david.marchand, fengchengwen,
liuyonglong, lihuisong
The deeper the idle state, the lower the power consumption, but the longer
the resume time. Some service are delay sensitive and very except the low
resume time, like interrupt packet receiving mode.
And the "/sys/devices/system/cpu/cpuX/power/pm_qos_resume_latency_us" sysfs
interface is used to set and get the resume latency limit on the cpuX for
userspace. Please see the description in kernel document[1].
Each cpuidle governor in Linux select which idle state to enter based on
this CPU resume latency in their idle task.
The per-CPU PM QoS API can be used to control this CPU's idle state
selection and limit just enter the shallowest idle state to low the delay
after sleep by setting strict resume latency (zero value).
[1] https://www.kernel.org/doc/html/latest/admin-guide/abi-testing.html?highlight=pm_qos_resume_latency_us#abi-sys-devices-power-pm-qos-resume-latency-us
---
v10:
- replace LINE_MAX with a custom macro and fix two typos.
v9:
- move new feature description from release_24_07.rst to release_24_11.rst.
v8:
- update the latest code to resolve CI warning
v7:
- remove a dead code rte_lcore_is_enabled in patch[2/2]
v6:
- update release_24_07.rst based on dpdk repo to resolve CI warning.
v5:
- use LINE_MAX to replace BUFSIZ, and use snprintf to replace sprintf.
v4:
- fix some comments basd on Stephen
- add stdint.h include
- add Acked-by Morten Brørup <mb@smartsharesystems.com>
v3:
- add RTE_POWER_xxx prefix for some macro in header
- add the check for lcore_id with rte_lcore_is_enabled
v2:
- use PM QoS on CPU wide to replace the one on system wide
Huisong Li (2):
power: introduce PM QoS API on CPU wide
examples/l3fwd-power: add PM QoS configuration
doc/guides/prog_guide/power_man.rst | 24 ++++++
doc/guides/rel_notes/release_24_11.rst | 5 ++
examples/l3fwd-power/main.c | 24 ++++++
lib/power/meson.build | 2 +
lib/power/rte_power_qos.c | 111 +++++++++++++++++++++++++
lib/power/rte_power_qos.h | 73 ++++++++++++++++
lib/power/version.map | 4 +
7 files changed, 243 insertions(+)
create mode 100644 lib/power/rte_power_qos.c
create mode 100644 lib/power/rte_power_qos.h
--
2.22.0
^ permalink raw reply [relevance 4%]
* Re: [PATCH v2 1/1] bbdev: removing unnecessaray symbols from version map
2024-08-27 23:03 3% ` [PATCH v2 1/1] bbdev: removing unnecessaray symbols from version map Nicolas Chautru
@ 2024-09-10 16:20 0% ` Maxime Coquelin
0 siblings, 0 replies; 200+ results
From: Maxime Coquelin @ 2024-09-10 16:20 UTC (permalink / raw)
To: Nicolas Chautru, dev; +Cc: hemant.agrawal, david.marchand, hernan.vargas
On 8/28/24 01:03, Nicolas Chautru wrote:
> A number of inline functions should not be in the
> version map since ABI versionning would be irrelevant.
>
> Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
> ---
> lib/bbdev/rte_bbdev.h | 1 -
> lib/bbdev/rte_bbdev_op.h | 2 --
> lib/bbdev/version.map | 24 +-----------------------
> 3 files changed, 1 insertion(+), 26 deletions(-)
>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Thanks,
Maxime
^ permalink raw reply [relevance 0%]
* Re: [PATCH v2 1/3] bbdev: new queue stat for available enqueue depth
2024-08-12 23:41 6% ` [PATCH v2 1/3] bbdev: new queue stat for available enqueue depth Nicolas Chautru
@ 2024-09-10 15:16 0% ` Maxime Coquelin
0 siblings, 0 replies; 200+ results
From: Maxime Coquelin @ 2024-09-10 15:16 UTC (permalink / raw)
To: Nicolas Chautru, dev; +Cc: hemant.agrawal, david.marchand, hernan.vargas
On 8/13/24 01:41, Nicolas Chautru wrote:
> Capturing additional queue stats counter for the
> depth of enqueue batch still available on the given
> queue. This can help application to monitor that depth
> at run time.
>
> Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
> ---
> doc/guides/rel_notes/release_24_11.rst | 3 +++
> lib/bbdev/rte_bbdev.h | 2 ++
> 2 files changed, 5 insertions(+)
>
> diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
> index 0ff70d9057..a45b9b2dc6 100644
> --- a/doc/guides/rel_notes/release_24_11.rst
> +++ b/doc/guides/rel_notes/release_24_11.rst
> @@ -88,6 +88,9 @@ API Changes
> ABI Changes
> -----------
>
> + * bbdev: Structure ``rte_bbdev_stats`` was updated to add new parameter
> + to optionally report number of enqueue batch available ``enqueue_depth_avail``.
> +
> .. This section should contain ABI changes. Sample format:
>
> * sample: Add a short 1-2 sentence description of the ABI change
> diff --git a/lib/bbdev/rte_bbdev.h b/lib/bbdev/rte_bbdev.h
> index 0cbfdd1c95..25514c58ac 100644
> --- a/lib/bbdev/rte_bbdev.h
> +++ b/lib/bbdev/rte_bbdev.h
> @@ -283,6 +283,8 @@ struct rte_bbdev_stats {
> * bbdev operation
> */
> uint64_t acc_offload_cycles;
> + /** Available number of enqueue batch on that queue. */
> + uint16_t enqueue_depth_avail;
> };
>
> /**
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Thanks,
Maxime
^ permalink raw reply [relevance 0%]
* Re: [PATCH v9 1/2] power: introduce PM QoS API on CPU wide
2024-09-10 2:00 0% ` fengchengwen
@ 2024-09-10 9:32 0% ` lihuisong (C)
0 siblings, 0 replies; 200+ results
From: lihuisong (C) @ 2024-09-10 9:32 UTC (permalink / raw)
To: fengchengwen, dev
Cc: mb, thomas, ferruh.yigit, anatoly.burakov, david.hunt,
sivaprasad.tummala, stephen, david.marchand, liuyonglong
Hi Chengwen,
Thanks for your review.
在 2024/9/10 10:00, fengchengwen 写道:
> Hi Huisong
>
> Please see comments inline.
>
> Thanks
>
> On 2024/8/9 17:50, Huisong Li wrote:
>> The deeper the idle state, the lower the power consumption, but the longer
>> the resume time. Some service are delay sensitive and very except the low
>> resume time, like interrupt packet receiving mode.
>>
>> And the "/sys/devices/system/cpu/cpuX/power/pm_qos_resume_latency_us" sysfs
>> interface is used to set and get the resume latency limit on the cpuX for
>> userspace. Each cpuidle governor in Linux select which idle state to enter
>> based on this CPU resume latency in their idle task.
>>
>> The per-CPU PM QoS API can be used to control this CPU's idle state
>> selection and limit just enter the shallowest idle state to low the delay
>> after sleep by setting strict resume latency (zero value).
>>
>> Signed-off-by: Huisong Li <lihuisong@huawei.com>
>> Acked-by: Morten Brørup <mb@smartsharesystems.com>
>> ---
> ...
>
>> diff --git a/lib/power/rte_power_qos.c b/lib/power/rte_power_qos.c
>> new file mode 100644
>> index 0000000000..375746f832
>> --- /dev/null
>> +++ b/lib/power/rte_power_qos.c
>> @@ -0,0 +1,114 @@
>> +/* SPDX-License-Identifier: BSD-3-Clause
>> + * Copyright(c) 2024 HiSilicon Limited
>> + */
>> +
>> +#include <errno.h>
>> +#include <stdlib.h>
>> +#include <string.h>
>> +
>> +#include <rte_lcore.h>
>> +#include <rte_log.h>
>> +
>> +#include "power_common.h"
>> +#include "rte_power_qos.h"
>> +
>> +#define PM_QOS_SYSFILE_RESUME_LATENCY_US \
>> + "/sys/devices/system/cpu/cpu%u/power/pm_qos_resume_latency_us"
>> +
>> +int
>> +rte_power_qos_set_cpu_resume_latency(uint16_t lcore_id, int latency)
>> +{
>> + char buf[LINE_MAX];
> no need LINE_MAX, [32] would enough.
Ack
>
>> + FILE *f;
>> + int ret;
>> +
>> + if (!rte_lcore_is_enabled(lcore_id)) {
>> + POWER_LOG(ERR, "lcore id %u is not enabled", lcore_id);
>> + return -EINVAL;
>> + }
>> +
>> + if (latency < 0) {
>> + POWER_LOG(ERR, "latency should be greater than and equal to 0");
>> + return -EINVAL;
>> + }
>> +
>> + ret = open_core_sysfs_file(&f, "w", PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
>> + if (ret != 0) {
>> + POWER_LOG(ERR, "Failed to open "PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
>> + return ret;
>> + }
>> +
>> + /*
>> + * Based on the sysfs interface pm_qos_resume_latency_us under
>> + * @PM_QOS_SYSFILE_RESUME_LATENCY_US directory in kernel, their meanning
> meanning -> meaning
Ack
>
>> + * is as follows for different input string.
>> + * 1> the resume latency is 0 if the input is "n/a".
>> + * 2> the resume latency is no constraint if the input is "0".
>> + * 3> the resume latency is the actual value to be set.
>> + */
>> + if (latency == 0)
>> + snprintf(buf, sizeof(buf), "%s", "n/a");
>> + else if (latency == RTE_POWER_QOS_RESUME_LATENCY_NO_CONSTRAINT)
>> + snprintf(buf, sizeof(buf), "%u", 0);
>> + else
>> + snprintf(buf, sizeof(buf), "%u", latency);
>> +
>> + ret = write_core_sysfs_s(f, buf);
>> + if (ret != 0) {
>> + POWER_LOG(ERR, "Failed to write "PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
>> + goto out;
> no need of goto
Ack
>
>> + }
>> +
>> +out:
>> + if (f != NULL)
>> + fclose(f);
> just fclose(f) because f is valid here.
Ack
>> +
>> + return ret;
>> +}
>> +
>> +int
>> +rte_power_qos_get_cpu_resume_latency(uint16_t lcore_id)
>> +{
>> + char buf[LINE_MAX];
>> + int latency = -1;
>> + FILE *f;
>> + int ret;
>> +
>> + if (!rte_lcore_is_enabled(lcore_id)) {
>> + POWER_LOG(ERR, "lcore id %u is not enabled", lcore_id);
>> + return -EINVAL;
>> + }
>> +
>> + ret = open_core_sysfs_file(&f, "r", PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
>> + if (ret != 0) {
>> + POWER_LOG(ERR, "Failed to open "PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
>> + return ret;
>> + }
>> +
>> + ret = read_core_sysfs_s(f, buf, sizeof(buf));
>> + if (ret != 0) {
>> + POWER_LOG(ERR, "Failed to read "PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
>> + goto out;
>> + }
>> +
>> + /*
>> + * Based on the sysfs interface pm_qos_resume_latency_us under
>> + * @PM_QOS_SYSFILE_RESUME_LATENCY_US directory in kernel, their meanning
> meanning -> meaning
Ack
>
>> + * is as follows for different output string.
>> + * 1> the resume latency is 0 if the output is "n/a".
>> + * 2> the resume latency is no constraint if the output is "0".
>> + * 3> the resume latency is the actual value in used for other string.
>> + */
>> + if (strcmp(buf, "n/a") == 0)
>> + latency = 0;
>> + else {
>> + latency = strtoul(buf, NULL, 10);
>> + latency = latency == 0 ? RTE_POWER_QOS_RESUME_LATENCY_NO_CONSTRAINT : latency;
>> + }
>> +
>> +out:
>> + if (f != NULL)
>> + fclose(f);
> just fclose(f) because f is valid here.
Ack
>
>> +
>> + return latency != -1 ? latency : ret;
>> +}
>> diff --git a/lib/power/rte_power_qos.h b/lib/power/rte_power_qos.h
>> new file mode 100644
>> index 0000000000..990c488373
>> --- /dev/null
>> +++ b/lib/power/rte_power_qos.h
>> @@ -0,0 +1,73 @@
>> +/* SPDX-License-Identifier: BSD-3-Clause
>> + * Copyright(c) 2024 HiSilicon Limited
>> + */
>> +
>> +#ifndef RTE_POWER_QOS_H
>> +#define RTE_POWER_QOS_H
>> +
>> +#include <stdint.h>
>> +
>> +#include <rte_compat.h>
>> +
>> +#ifdef __cplusplus
>> +extern "C" {
>> +#endif
>> +
>> +/**
>> + * @file rte_power_qos.h
>> + *
>> + * PM QoS API.
>> + *
>> + * The CPU-wide resume latency limit has a positive impact on this CPU's idle
>> + * state selection in each cpuidle governor.
>> + * Please see the PM QoS on CPU wide in the following link:
>> + * https://www.kernel.org/doc/html/latest/admin-guide/abi-testing.html?highlight=pm_qos_resume_latency_us#abi-sys-devices-power-pm-qos-resume-latency-us
>> + *
>> + * The deeper the idle state, the lower the power consumption, but the
>> + * longer the resume time. Some service are delay sensitive and very except the
>> + * low resume time, like interrupt packet receiving mode.
>> + *
>> + * In these case, per-CPU PM QoS API can be used to control this CPU's idle
>> + * state selection and limit just enter the shallowest idle state to low the
>> + * delay after sleep by setting strict resume latency (zero value).
>> + */
>> +
>> +#define RTE_POWER_QOS_STRICT_LATENCY_VALUE 0
>> +#define RTE_POWER_QOS_RESUME_LATENCY_NO_CONSTRAINT ((int)(UINT32_MAX >> 1))
>> +
>> +/**
>> + * @warning
>> + * @b EXPERIMENTAL: this API may change without prior notice.
>> + *
>> + * @param lcore_id
>> + * target logical core id
>> + *
>> + * @param latency
>> + * The latency should be greater than and equal to zero in microseconds unit.
>> + *
>> + * @return
>> + * 0 on success. Otherwise negative value is returned.
>> + */
>> +__rte_experimental
>> +int rte_power_qos_set_cpu_resume_latency(uint16_t lcore_id, int latency);
>> +
>> +/**
>> + * @warning
>> + * @b EXPERIMENTAL: this API may change without prior notice.
>> + *
>> + * Get the current resume latency of this logical core.
>> + * The default value in kernel is @see RTE_POWER_QOS_RESUME_LATENCY_NO_CONSTRAINT
>> + * if don't set it.
>> + *
>> + * @return
>> + * Negative value on failure.
>> + * >= 0 means the actual resume latency limit on this core.
>> + */
>> +__rte_experimental
>> +int rte_power_qos_get_cpu_resume_latency(uint16_t lcore_id);
>> +
>> +#ifdef __cplusplus
>> +}
>> +#endif
>> +
>> +#endif /* RTE_POWER_QOS_H */
>> diff --git a/lib/power/version.map b/lib/power/version.map
>> index c9a226614e..4e4955a4cf 100644
>> --- a/lib/power/version.map
>> +++ b/lib/power/version.map
>> @@ -51,4 +51,8 @@ EXPERIMENTAL {
>> rte_power_set_uncore_env;
>> rte_power_uncore_freqs;
>> rte_power_unset_uncore_env;
>> +
>> + # added in 24.11
>> + rte_power_qos_set_cpu_resume_latency;
>> + rte_power_qos_get_cpu_resume_latency;
> order by alphabetic.
Ack
>
> another question, I think rename cpu with core maybe more accurate, despite sysfs export with cpu, but in DPDK it means core.
> and there are some rte_power_core_xxx name in rte_power library, I think better to keep the same.
Firstly, the rte_power_qos_set/get_cpu_resume_latency is just consistent
with linux sysfs interface. Having the same name is more releative for user.
In addition, Sivaprasad Tummala is reworking power library and the name
of rte_power_core_xxx also might be changed.
>
>> };
>>
> .
^ permalink raw reply [relevance 0%]
* Re: [PATCH v9 1/2] power: introduce PM QoS API on CPU wide
2024-08-09 9:50 5% ` [PATCH v9 1/2] power: introduce PM QoS API on CPU wide Huisong Li
@ 2024-09-10 2:00 0% ` fengchengwen
2024-09-10 9:32 0% ` lihuisong (C)
0 siblings, 1 reply; 200+ results
From: fengchengwen @ 2024-09-10 2:00 UTC (permalink / raw)
To: Huisong Li, dev
Cc: mb, thomas, ferruh.yigit, anatoly.burakov, david.hunt,
sivaprasad.tummala, stephen, david.marchand, liuyonglong
Hi Huisong
Please see comments inline.
Thanks
On 2024/8/9 17:50, Huisong Li wrote:
> The deeper the idle state, the lower the power consumption, but the longer
> the resume time. Some service are delay sensitive and very except the low
> resume time, like interrupt packet receiving mode.
>
> And the "/sys/devices/system/cpu/cpuX/power/pm_qos_resume_latency_us" sysfs
> interface is used to set and get the resume latency limit on the cpuX for
> userspace. Each cpuidle governor in Linux select which idle state to enter
> based on this CPU resume latency in their idle task.
>
> The per-CPU PM QoS API can be used to control this CPU's idle state
> selection and limit just enter the shallowest idle state to low the delay
> after sleep by setting strict resume latency (zero value).
>
> Signed-off-by: Huisong Li <lihuisong@huawei.com>
> Acked-by: Morten Brørup <mb@smartsharesystems.com>
> ---
...
> diff --git a/lib/power/rte_power_qos.c b/lib/power/rte_power_qos.c
> new file mode 100644
> index 0000000000..375746f832
> --- /dev/null
> +++ b/lib/power/rte_power_qos.c
> @@ -0,0 +1,114 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2024 HiSilicon Limited
> + */
> +
> +#include <errno.h>
> +#include <stdlib.h>
> +#include <string.h>
> +
> +#include <rte_lcore.h>
> +#include <rte_log.h>
> +
> +#include "power_common.h"
> +#include "rte_power_qos.h"
> +
> +#define PM_QOS_SYSFILE_RESUME_LATENCY_US \
> + "/sys/devices/system/cpu/cpu%u/power/pm_qos_resume_latency_us"
> +
> +int
> +rte_power_qos_set_cpu_resume_latency(uint16_t lcore_id, int latency)
> +{
> + char buf[LINE_MAX];
no need LINE_MAX, [32] would enough.
> + FILE *f;
> + int ret;
> +
> + if (!rte_lcore_is_enabled(lcore_id)) {
> + POWER_LOG(ERR, "lcore id %u is not enabled", lcore_id);
> + return -EINVAL;
> + }
> +
> + if (latency < 0) {
> + POWER_LOG(ERR, "latency should be greater than and equal to 0");
> + return -EINVAL;
> + }
> +
> + ret = open_core_sysfs_file(&f, "w", PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
> + if (ret != 0) {
> + POWER_LOG(ERR, "Failed to open "PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
> + return ret;
> + }
> +
> + /*
> + * Based on the sysfs interface pm_qos_resume_latency_us under
> + * @PM_QOS_SYSFILE_RESUME_LATENCY_US directory in kernel, their meanning
meanning -> meaning
> + * is as follows for different input string.
> + * 1> the resume latency is 0 if the input is "n/a".
> + * 2> the resume latency is no constraint if the input is "0".
> + * 3> the resume latency is the actual value to be set.
> + */
> + if (latency == 0)
> + snprintf(buf, sizeof(buf), "%s", "n/a");
> + else if (latency == RTE_POWER_QOS_RESUME_LATENCY_NO_CONSTRAINT)
> + snprintf(buf, sizeof(buf), "%u", 0);
> + else
> + snprintf(buf, sizeof(buf), "%u", latency);
> +
> + ret = write_core_sysfs_s(f, buf);
> + if (ret != 0) {
> + POWER_LOG(ERR, "Failed to write "PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
> + goto out;
no need of goto
> + }
> +
> +out:
> + if (f != NULL)
> + fclose(f);
just fclose(f) because f is valid here.
> +
> + return ret;
> +}
> +
> +int
> +rte_power_qos_get_cpu_resume_latency(uint16_t lcore_id)
> +{
> + char buf[LINE_MAX];
> + int latency = -1;
> + FILE *f;
> + int ret;
> +
> + if (!rte_lcore_is_enabled(lcore_id)) {
> + POWER_LOG(ERR, "lcore id %u is not enabled", lcore_id);
> + return -EINVAL;
> + }
> +
> + ret = open_core_sysfs_file(&f, "r", PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
> + if (ret != 0) {
> + POWER_LOG(ERR, "Failed to open "PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
> + return ret;
> + }
> +
> + ret = read_core_sysfs_s(f, buf, sizeof(buf));
> + if (ret != 0) {
> + POWER_LOG(ERR, "Failed to read "PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
> + goto out;
> + }
> +
> + /*
> + * Based on the sysfs interface pm_qos_resume_latency_us under
> + * @PM_QOS_SYSFILE_RESUME_LATENCY_US directory in kernel, their meanning
meanning -> meaning
> + * is as follows for different output string.
> + * 1> the resume latency is 0 if the output is "n/a".
> + * 2> the resume latency is no constraint if the output is "0".
> + * 3> the resume latency is the actual value in used for other string.
> + */
> + if (strcmp(buf, "n/a") == 0)
> + latency = 0;
> + else {
> + latency = strtoul(buf, NULL, 10);
> + latency = latency == 0 ? RTE_POWER_QOS_RESUME_LATENCY_NO_CONSTRAINT : latency;
> + }
> +
> +out:
> + if (f != NULL)
> + fclose(f);
just fclose(f) because f is valid here.
> +
> + return latency != -1 ? latency : ret;
> +}
> diff --git a/lib/power/rte_power_qos.h b/lib/power/rte_power_qos.h
> new file mode 100644
> index 0000000000..990c488373
> --- /dev/null
> +++ b/lib/power/rte_power_qos.h
> @@ -0,0 +1,73 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2024 HiSilicon Limited
> + */
> +
> +#ifndef RTE_POWER_QOS_H
> +#define RTE_POWER_QOS_H
> +
> +#include <stdint.h>
> +
> +#include <rte_compat.h>
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +/**
> + * @file rte_power_qos.h
> + *
> + * PM QoS API.
> + *
> + * The CPU-wide resume latency limit has a positive impact on this CPU's idle
> + * state selection in each cpuidle governor.
> + * Please see the PM QoS on CPU wide in the following link:
> + * https://www.kernel.org/doc/html/latest/admin-guide/abi-testing.html?highlight=pm_qos_resume_latency_us#abi-sys-devices-power-pm-qos-resume-latency-us
> + *
> + * The deeper the idle state, the lower the power consumption, but the
> + * longer the resume time. Some service are delay sensitive and very except the
> + * low resume time, like interrupt packet receiving mode.
> + *
> + * In these case, per-CPU PM QoS API can be used to control this CPU's idle
> + * state selection and limit just enter the shallowest idle state to low the
> + * delay after sleep by setting strict resume latency (zero value).
> + */
> +
> +#define RTE_POWER_QOS_STRICT_LATENCY_VALUE 0
> +#define RTE_POWER_QOS_RESUME_LATENCY_NO_CONSTRAINT ((int)(UINT32_MAX >> 1))
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * @param lcore_id
> + * target logical core id
> + *
> + * @param latency
> + * The latency should be greater than and equal to zero in microseconds unit.
> + *
> + * @return
> + * 0 on success. Otherwise negative value is returned.
> + */
> +__rte_experimental
> +int rte_power_qos_set_cpu_resume_latency(uint16_t lcore_id, int latency);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get the current resume latency of this logical core.
> + * The default value in kernel is @see RTE_POWER_QOS_RESUME_LATENCY_NO_CONSTRAINT
> + * if don't set it.
> + *
> + * @return
> + * Negative value on failure.
> + * >= 0 means the actual resume latency limit on this core.
> + */
> +__rte_experimental
> +int rte_power_qos_get_cpu_resume_latency(uint16_t lcore_id);
> +
> +#ifdef __cplusplus
> +}
> +#endif
> +
> +#endif /* RTE_POWER_QOS_H */
> diff --git a/lib/power/version.map b/lib/power/version.map
> index c9a226614e..4e4955a4cf 100644
> --- a/lib/power/version.map
> +++ b/lib/power/version.map
> @@ -51,4 +51,8 @@ EXPERIMENTAL {
> rte_power_set_uncore_env;
> rte_power_uncore_freqs;
> rte_power_unset_uncore_env;
> +
> + # added in 24.11
> + rte_power_qos_set_cpu_resume_latency;
> + rte_power_qos_get_cpu_resume_latency;
order by alphabetic.
another question, I think rename cpu with core maybe more accurate, despite sysfs export with cpu, but in DPDK it means core.
and there are some rte_power_core_xxx name in rte_power library, I think better to keep the same.
> };
>
^ permalink raw reply [relevance 0%]
* Re: [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK
2024-09-06 11:47 3% [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK Anatoly Burakov
2024-09-06 12:37 0% ` Morten Brørup
@ 2024-09-09 7:51 0% ` fengchengwen
1 sibling, 0 replies; 200+ results
From: fengchengwen @ 2024-09-09 7:51 UTC (permalink / raw)
To: Anatoly Burakov, dev
for commit 1~4,
Acked-by: Chengwen Feng <fengchengwen@huawei.com>
On 2024/9/6 19:47, Anatoly Burakov wrote:
> While initially, DPDK has used the term "socket ID" to refer to physical package
> ID, the last time DPDK read "physical_package_id" for socket ID was ~9 years
> ago, so it's been a while since we've actually switched over to using the term
> "socket" to mean "NUMA node".
>
> This wasn't a problem before, as most systems had one NUMA node per physical
> socket. However, in the last few years, more and more systems have multiple NUMA
> nodes per physical CPU socket. Since DPDK used NUMA nodes already, the
> transition was pretty seamless, however now we're faced with a situation when
> most of our documentation still uses outdated terms, and our API is ripe with
> references to "sockets" when in actuality we mean "NUMA nodes". This could be a
> source of confusion.
>
> While completely renaming all of our API's would be a huge effort, will take a
> long time and arguably wouldn't even be worth the API breakages (given that this
> mismatch between terminology and reality is implicitly understood by most people
> working on DPDK, and so this isn't so much of a problem in practice), we can do
> some tweaks around the edges and at least document this unfortunate reality.
>
> This patchset suggests the following changes:
>
> - Update rte_socket/rte_lcore documentation to refer to NUMA nodes rather than
> sockets - Rename internal structures' fields to better reflect this intention -
> Rename --socket-mem/--socket-limit flags to refer to NUMA rather than sockets -
> Add internal API to get physical package ID [1]
>
> The documentation is updated to refer to new EAL flags, but is otherwise left
> untouched, and instead the entry in "glossary" is amended to indicate that when
> DPDK documentation refers to "sockets", it actually means "NUMA ID's". As next
> steps, we could rename all API parameters to refer to NUMA ID rather than socket
> ID - this would not break neither API nor ABI, and instead would be a
> documentation change in practice.
>
> [1] This could be used to group lcores by physical package, see e.g. discussion
> under this patch: https://patches.dpdk.org/project/dpdk/cover/20240827151014.201-1-vipin.varghese@amd.com/
>
> Anatoly Burakov (5):
> eal: update socket ID API documentation
> lcore: rename socket ID to NUMA ID
> eal: rename socket ID to NUMA ID in internal config
> eal: rename --socket-mem/--socket-limit
> lcore: store physical package ID internally
>
> doc/guides/faq/faq.rst | 4 +--
> doc/guides/howto/lm_bond_virtio_sriov.rst | 2 +-
> doc/guides/howto/lm_virtio_vhost_user.rst | 2 +-
> doc/guides/howto/pvp_reference_benchmark.rst | 4 +--
> .../virtio_user_for_container_networking.rst | 2 +-
> doc/guides/linux_gsg/build_sample_apps.rst | 20 +++++------
> doc/guides/linux_gsg/linux_eal_parameters.rst | 16 ++++-----
> doc/guides/nics/mlx4.rst | 2 +-
> doc/guides/nics/mlx5.rst | 2 +-
> .../prog_guide/env_abstraction_layer.rst | 12 +++----
> doc/guides/prog_guide/glossary.rst | 5 ++-
> doc/guides/prog_guide/multi_proc_support.rst | 2 +-
> doc/guides/sample_app_ug/bbdev_app.rst | 6 ++--
> doc/guides/sample_app_ug/ipsec_secgw.rst | 6 ++--
> doc/guides/sample_app_ug/vdpa.rst | 2 +-
> doc/guides/sample_app_ug/vhost.rst | 4 +--
> lib/eal/common/eal_common_dynmem.c | 14 ++++----
> lib/eal/common/eal_common_lcore.c | 28 +++++++++++++---
> lib/eal/common/eal_common_options.c | 33 ++++++++++---------
> lib/eal/common/eal_common_thread.c | 12 +++----
> lib/eal/common/eal_internal_cfg.h | 10 +++---
> lib/eal/common/eal_options.h | 8 +++--
> lib/eal/common/eal_private.h | 5 ++-
> lib/eal/common/eal_thread.h | 11 +++++++
> lib/eal/common/malloc_heap.c | 2 +-
> lib/eal/freebsd/eal.c | 2 +-
> lib/eal/freebsd/eal_lcore.c | 6 ++++
> lib/eal/include/rte_lcore.h | 25 +++++++-------
> lib/eal/linux/eal.c | 22 ++++++-------
> lib/eal/linux/eal_lcore.c | 28 ++++++++++++++++
> lib/eal/linux/eal_memory.c | 22 ++++++-------
> lib/eal/windows/eal.c | 2 +-
> lib/eal/windows/eal_lcore.c | 7 ++++
> 33 files changed, 204 insertions(+), 124 deletions(-)
>
^ permalink raw reply [relevance 0%]
* Re: [PATCH 0/3] eal: mark API's as stable
2024-09-06 14:42 0% ` Morten Brørup
@ 2024-09-09 4:48 0% ` Jerin Jacob
0 siblings, 0 replies; 200+ results
From: Jerin Jacob @ 2024-09-09 4:48 UTC (permalink / raw)
To: Morten Brørup
Cc: Ferruh Yigit, David Marchand, Jerin Jacob Kollanukkaran,
Stephen Hemminger, bruce.richardson, Sunil Kumar Kori, dev,
Thomas Monjalon
On Fri, Sep 6, 2024 at 8:12 PM Morten Brørup <mb@smartsharesystems.com> wrote:
>
> > From: Ferruh Yigit [mailto:ferruh.yigit@amd.com]
> > Sent: Friday, 6 September 2024 16.12
> >
> > On 9/6/2024 11:04 AM, Morten Brørup wrote:
> > >> From: Ferruh Yigit [mailto:ferruh.yigit@amd.com]
> > >> Sent: Friday, 6 September 2024 10.54
> > >>
> > >> On 9/5/2024 3:01 PM, Jerin Jacob wrote:
> > >>> On Thu, Sep 5, 2024 at 3:14 PM Morten Brørup <mb@smartsharesystems.com>
> > >> wrote:
> > >>>>
> > >>>>> From: David Marchand [mailto:david.marchand@redhat.com]
> > >>>>> Sent: Thursday, 5 September 2024 11.03
> > >>>>>
> > >>>>> On Thu, Sep 5, 2024 at 10:55 AM Morten Brørup <mb@smartsharesystems.com>
> > >>>>> wrote:
> > >>>>>>
> > >>>>>>> From: David Marchand [mailto:david.marchand@redhat.com]
> > >>>>>>> Sent: Thursday, 5 September 2024 09.59
> > >>>>>>>
> > >>>>>>> On Wed, Sep 4, 2024 at 8:10 PM Stephen Hemminger
> > >>>>>>> <stephen@networkplumber.org> wrote:
> > >>>>>>>>
> > >>>>>>>> The API's in ethtool from before 23.11 should be marked stable.
> > >>>>>>>
> > >>>>>>> EAL* ?
> > >>>>>>>
> > >>>>>>>> Should probably include the trace api's but that is more complex
> > >> change.
> > >>>>>>>
> > >>>>>>> On the trace API itself it should be ok.
> > >>>>>>
> > >>>>>> No!
> > >>>>>
> > >>>>> *sigh*
> > >>>>>
> > >>>>>>
> > >>>>>> Trace must remain experimental until controlled by a meson option, e.g.
> > >>>>> "enable_trace", whereby trace can be completely disabled and omitted
> > from
> > >> the
> > >>>>> compiled application/libraries/drivers at build time.
> > >>>>>
> > >>>>> This seems unrelated to marking the API stable as regardless of the
> > >>>>> API state at the moment, this code is always present.
> > >>>>
> > >>>> I cannot foresee if disabling trace at build time will require changes to
> > >> the trace API. So I'm being cautious here.
> > >>>>
> > >>>> However, if Jerin (as author of the trace subsystem) foresees that it
> > will
> > >> be possible to disable trace at build time without affecting the trace API,
> > I
> > >> don't object to marking the trace API (or some of it) stable.
> > >>>
> > >>> I don't for foresee any ABI changes when adding disabling trace
> > >>> compile time support. However, I don't understand why we need to do
> > >>> that. In the sense, fast path functions are already having an option
> > >>> to compile out.
> > >>> Slow path functions can be disabled at runtime at the cost of 1 cycle
> > >>> as instrumentation cost. Having said that, I don't have any concern
> > >>> about disabling trace as an option.
> > >>>
> > >>
> > >> I agree with Jerin, I don't see motivation to disable slow path traces
> > >> when they can be disabled in runtime.
> > >> And fast path traces already have compile flag to disable them.
> > >>
> > >> Build time configurations in long term has problems too, so I am for not
> > >> using them unless we don't have to.
> > >
> > > For some use cases, trace is dead code, and should be omitted.
> > > You don't want dead code in production systems.
> > >
> > > Please remember that DPDK is also being used in highly optimized embedded
> > systems, hardware appliances and other systems where memory is not abundant.
> > >
> > > DPDK is not only for cloud and distros. ;-)
> > >
> > > The CI only tests DPDK with a build time configuration expected to be usable
> > for distros.
> > > I'm not asking to change that.
> > > I'm only asking for more build time configurability to support other use
> > cases.
> > >
> >
> > I see, but that build time configuration argument exists in multiple
> > aspects. And with meson switch we lean to dynamic configuration approach.
>
> It can be rte_config.h instead of Meson option. Either is perfectly fine with me.
>
> >
> > When a build time config introduced, again and again we are having cases
> > that specific code enabled with compile time macro broken and nobody
> > noticed.
>
> I acknowledge this risk.
if we use
if (0)
{
}
scheme instead of #ifdef scheme.
The compiler will check this leg even it is not active.
>
> Trace doesn't interact with anything, so I consider the risk extremely low in this case.
>
> > Having code enabled always and configured in runtime produces more
> > robust deliverable.
>
> The same can be said about the Linux kernel, but yet it is configurable.
>
> >
> > We are aware that DPDK is used in embedded device, but they are not
> > mostly very resource restricted devices, is it really matter to have a
> > few megabytes (I didn't check but I expect this the max binary size can
> > increase with tracing code) larger DPDK binary, does it really makes any
> > difference?
>
> Code size also affects system boot time, because those superfluous megabytes take time to decompress when booting from FLASH memory.
> For reference, the size of our system image (including Linux kernel, OpenSSL, the DPDK application, webserver, GUI, CLI, SNMP and everything) is 12 MB compressed.
>
> From a security aspect, trace also increases the attack surface and can potentially assist a hacker trying to break into a system.
>
^ permalink raw reply [relevance 0%]
* Re: [PATCH 0/3] eal: mark API's as stable
2024-09-05 14:18 0% ` Morten Brørup
@ 2024-09-08 23:58 3% ` Stephen Hemminger
0 siblings, 0 replies; 200+ results
From: Stephen Hemminger @ 2024-09-08 23:58 UTC (permalink / raw)
To: Morten Brørup
Cc: Jerin Jacob, David Marchand, Jerin Jacob Kollanukkaran,
bruce.richardson, Sunil Kumar Kori, dev, Thomas Monjalon
On Thu, 5 Sep 2024 16:18:13 +0200
Morten Brørup <mb@smartsharesystems.com> wrote:
> > From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> > Sent: Thursday, 5 September 2024 16.02
> >
> > On Thu, Sep 5, 2024 at 3:14 PM Morten Brørup <mb@smartsharesystems.com> wrote:
> > >
> > > > From: David Marchand [mailto:david.marchand@redhat.com]
> > > > Sent: Thursday, 5 September 2024 11.03
> > > >
> > > > On Thu, Sep 5, 2024 at 10:55 AM Morten Brørup <mb@smartsharesystems.com>
> > > > wrote:
> > > > >
> > > > > > From: David Marchand [mailto:david.marchand@redhat.com]
> > > > > > Sent: Thursday, 5 September 2024 09.59
> > > > > >
> > > > > > On Wed, Sep 4, 2024 at 8:10 PM Stephen Hemminger
> > > > > > <stephen@networkplumber.org> wrote:
> > > > > > >
> > > > > > > The API's in ethtool from before 23.11 should be marked stable.
> > > > > >
> > > > > > EAL* ?
> > > > > >
> > > > > > > Should probably include the trace api's but that is more complex
> > change.
> > > > > >
> > > > > > On the trace API itself it should be ok.
> > > > >
> > > > > No!
> > > >
> > > > *sigh*
> > > >
> > > > >
> > > > > Trace must remain experimental until controlled by a meson option, e.g.
> > > > "enable_trace", whereby trace can be completely disabled and omitted from
> > the
> > > > compiled application/libraries/drivers at build time.
> > > >
> > > > This seems unrelated to marking the API stable as regardless of the
> > > > API state at the moment, this code is always present.
> > >
> > > I cannot foresee if disabling trace at build time will require changes to
> > the trace API. So I'm being cautious here.
> > >
> > > However, if Jerin (as author of the trace subsystem) foresees that it will
> > be possible to disable trace at build time without affecting the trace API, I
> > don't object to marking the trace API (or some of it) stable.
> >
> > I don't for foresee any ABI changes when adding disabling trace
> > compile time support.
>
> Based on Jerin's feedback, I'm retracting my objection.
>
> > However, I don't understand why we need to do
> > that.
>
> To reduce code size.
> Relevant for embedded/memory-constrained systems.
>
> > In the sense, fast path functions are already having an option
> > to compile out.
> > Slow path functions can be disabled at runtime at the cost of 1 cycle
> > as instrumentation cost. Having said that, I don't have any concern
> > about disabling trace as an option.
>
> Great.
>
> >
> >
> > >
> > > Before doing that, rte_trace_mode_get/set() and the accompanying enum
> > rte_trace_mode should be changed to rte_trace_config_get/set() using a new
> > struct rte_trace_config (containing the enum rte_trace_mode, and expandable
> > with new fields as the need arises). This will prepare for e.g. tracing to
> > other destinations than system memory, such as a remote trace collector on the
> > network, like SYSLOG.
>
> I'm also retracting this precondition...
>
> If the need for further trace configuration should ever arise, rte_trace_config_get/set() can be added later.
> And rte_trace_mode_get/set(), if not marked as experimental anymore, will be kept for backwards compatibility.
>
> > >
> > > > Patches welcome if you want it stripped.
> > >
> > > Don't have time myself, so I suggested it as a code challenge instead. :-)
> > >
My feeling is that the the experimental flag is not intended as permanent "get out of ABI compatiablity"
^ permalink raw reply [relevance 3%]
* [PATCH v2 4/4] devtools/test-meson-builds: use cross files for 32bit build
@ 2024-09-06 16:12 13% ` Bruce Richardson
0 siblings, 0 replies; 200+ results
From: Bruce Richardson @ 2024-09-06 16:12 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
When testing the 32-bit x86 build, use a cross-file rather than using
args and pkgconfig environment variable. The advantage of using the
cross-file is that the paths are saved across runs. Without this patch,
while the '-m32' args settings are preserved between meson runs, the
PKG_CONFIG_LIBDIR value from environment is not, which can cause
rebuilds of the build-32b directory to fail if meson needs to do
a reconfiguration first.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
devtools/test-meson-builds.sh | 13 +++++--------
1 file changed, 5 insertions(+), 8 deletions(-)
diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index d71bb1ded0..4fff1f7177 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -253,21 +253,18 @@ build build-x86-generic cc skipABI -Dcheck_includes=true \
# 32-bit with default compiler
if check_cc_flags '-m32' ; then
+ target_override='i386-pc-linux-gnu'
if [ -d '/usr/lib/i386-linux-gnu' ] ; then
- # 32-bit pkgconfig on Debian/Ubuntu
- export PKG_CONFIG_LIBDIR='/usr/lib/i386-linux-gnu/pkgconfig'
+ # 32-bit pkgconfig on Debian/Ubuntu, use cross file
+ build build-32b $srcdir/config/x86/cross-32bit-debian.ini ABI
elif [ -d '/usr/lib32' ] ; then
# 32-bit pkgconfig on Arch
- export PKG_CONFIG_LIBDIR='/usr/lib32/pkgconfig'
+ build build-32b $srcdir/config/x86/cross-32bit-arch.ini ABI
else
# 32-bit pkgconfig on RHEL/Fedora (lib vs lib64)
- export PKG_CONFIG_LIBDIR='/usr/lib/pkgconfig'
+ build build-32b $srcdir/config/x86/cross-32bit-fedora.ini ABI
fi
- target_override='i386-pc-linux-gnu'
- build build-32b cc ABI -Dc_args='-m32' -Dc_link_args='-m32' \
- -Dcpp_args='-m32' -Dcpp_link_args='-m32'
target_override=
- unset PKG_CONFIG_LIBDIR
fi
# x86 MinGW
--
2.43.0
^ permalink raw reply [relevance 13%]
* RE: [PATCH 0/3] eal: mark API's as stable
2024-09-06 14:12 0% ` Ferruh Yigit
@ 2024-09-06 14:42 0% ` Morten Brørup
2024-09-09 4:48 0% ` Jerin Jacob
0 siblings, 1 reply; 200+ results
From: Morten Brørup @ 2024-09-06 14:42 UTC (permalink / raw)
To: Ferruh Yigit, Jerin Jacob, David Marchand
Cc: Jerin Jacob Kollanukkaran, Stephen Hemminger, bruce.richardson,
Sunil Kumar Kori, dev, Thomas Monjalon
> From: Ferruh Yigit [mailto:ferruh.yigit@amd.com]
> Sent: Friday, 6 September 2024 16.12
>
> On 9/6/2024 11:04 AM, Morten Brørup wrote:
> >> From: Ferruh Yigit [mailto:ferruh.yigit@amd.com]
> >> Sent: Friday, 6 September 2024 10.54
> >>
> >> On 9/5/2024 3:01 PM, Jerin Jacob wrote:
> >>> On Thu, Sep 5, 2024 at 3:14 PM Morten Brørup <mb@smartsharesystems.com>
> >> wrote:
> >>>>
> >>>>> From: David Marchand [mailto:david.marchand@redhat.com]
> >>>>> Sent: Thursday, 5 September 2024 11.03
> >>>>>
> >>>>> On Thu, Sep 5, 2024 at 10:55 AM Morten Brørup <mb@smartsharesystems.com>
> >>>>> wrote:
> >>>>>>
> >>>>>>> From: David Marchand [mailto:david.marchand@redhat.com]
> >>>>>>> Sent: Thursday, 5 September 2024 09.59
> >>>>>>>
> >>>>>>> On Wed, Sep 4, 2024 at 8:10 PM Stephen Hemminger
> >>>>>>> <stephen@networkplumber.org> wrote:
> >>>>>>>>
> >>>>>>>> The API's in ethtool from before 23.11 should be marked stable.
> >>>>>>>
> >>>>>>> EAL* ?
> >>>>>>>
> >>>>>>>> Should probably include the trace api's but that is more complex
> >> change.
> >>>>>>>
> >>>>>>> On the trace API itself it should be ok.
> >>>>>>
> >>>>>> No!
> >>>>>
> >>>>> *sigh*
> >>>>>
> >>>>>>
> >>>>>> Trace must remain experimental until controlled by a meson option, e.g.
> >>>>> "enable_trace", whereby trace can be completely disabled and omitted
> from
> >> the
> >>>>> compiled application/libraries/drivers at build time.
> >>>>>
> >>>>> This seems unrelated to marking the API stable as regardless of the
> >>>>> API state at the moment, this code is always present.
> >>>>
> >>>> I cannot foresee if disabling trace at build time will require changes to
> >> the trace API. So I'm being cautious here.
> >>>>
> >>>> However, if Jerin (as author of the trace subsystem) foresees that it
> will
> >> be possible to disable trace at build time without affecting the trace API,
> I
> >> don't object to marking the trace API (or some of it) stable.
> >>>
> >>> I don't for foresee any ABI changes when adding disabling trace
> >>> compile time support. However, I don't understand why we need to do
> >>> that. In the sense, fast path functions are already having an option
> >>> to compile out.
> >>> Slow path functions can be disabled at runtime at the cost of 1 cycle
> >>> as instrumentation cost. Having said that, I don't have any concern
> >>> about disabling trace as an option.
> >>>
> >>
> >> I agree with Jerin, I don't see motivation to disable slow path traces
> >> when they can be disabled in runtime.
> >> And fast path traces already have compile flag to disable them.
> >>
> >> Build time configurations in long term has problems too, so I am for not
> >> using them unless we don't have to.
> >
> > For some use cases, trace is dead code, and should be omitted.
> > You don't want dead code in production systems.
> >
> > Please remember that DPDK is also being used in highly optimized embedded
> systems, hardware appliances and other systems where memory is not abundant.
> >
> > DPDK is not only for cloud and distros. ;-)
> >
> > The CI only tests DPDK with a build time configuration expected to be usable
> for distros.
> > I'm not asking to change that.
> > I'm only asking for more build time configurability to support other use
> cases.
> >
>
> I see, but that build time configuration argument exists in multiple
> aspects. And with meson switch we lean to dynamic configuration approach.
It can be rte_config.h instead of Meson option. Either is perfectly fine with me.
>
> When a build time config introduced, again and again we are having cases
> that specific code enabled with compile time macro broken and nobody
> noticed.
I acknowledge this risk.
Trace doesn't interact with anything, so I consider the risk extremely low in this case.
> Having code enabled always and configured in runtime produces more
> robust deliverable.
The same can be said about the Linux kernel, but yet it is configurable.
>
> We are aware that DPDK is used in embedded device, but they are not
> mostly very resource restricted devices, is it really matter to have a
> few megabytes (I didn't check but I expect this the max binary size can
> increase with tracing code) larger DPDK binary, does it really makes any
> difference?
Code size also affects system boot time, because those superfluous megabytes take time to decompress when booting from FLASH memory.
For reference, the size of our system image (including Linux kernel, OpenSSL, the DPDK application, webserver, GUI, CLI, SNMP and everything) is 12 MB compressed.
From a security aspect, trace also increases the attack surface and can potentially assist a hacker trying to break into a system.
^ permalink raw reply [relevance 0%]
* Re: [PATCH 0/3] eal: mark API's as stable
2024-09-06 10:04 0% ` Morten Brørup
@ 2024-09-06 14:12 0% ` Ferruh Yigit
2024-09-06 14:42 0% ` Morten Brørup
0 siblings, 1 reply; 200+ results
From: Ferruh Yigit @ 2024-09-06 14:12 UTC (permalink / raw)
To: Morten Brørup, Jerin Jacob
Cc: David Marchand, Jerin Jacob Kollanukkaran, Stephen Hemminger,
bruce.richardson, Sunil Kumar Kori, dev, Thomas Monjalon
On 9/6/2024 11:04 AM, Morten Brørup wrote:
>> From: Ferruh Yigit [mailto:ferruh.yigit@amd.com]
>> Sent: Friday, 6 September 2024 10.54
>>
>> On 9/5/2024 3:01 PM, Jerin Jacob wrote:
>>> On Thu, Sep 5, 2024 at 3:14 PM Morten Brørup <mb@smartsharesystems.com>
>> wrote:
>>>>
>>>>> From: David Marchand [mailto:david.marchand@redhat.com]
>>>>> Sent: Thursday, 5 September 2024 11.03
>>>>>
>>>>> On Thu, Sep 5, 2024 at 10:55 AM Morten Brørup <mb@smartsharesystems.com>
>>>>> wrote:
>>>>>>
>>>>>>> From: David Marchand [mailto:david.marchand@redhat.com]
>>>>>>> Sent: Thursday, 5 September 2024 09.59
>>>>>>>
>>>>>>> On Wed, Sep 4, 2024 at 8:10 PM Stephen Hemminger
>>>>>>> <stephen@networkplumber.org> wrote:
>>>>>>>>
>>>>>>>> The API's in ethtool from before 23.11 should be marked stable.
>>>>>>>
>>>>>>> EAL* ?
>>>>>>>
>>>>>>>> Should probably include the trace api's but that is more complex
>> change.
>>>>>>>
>>>>>>> On the trace API itself it should be ok.
>>>>>>
>>>>>> No!
>>>>>
>>>>> *sigh*
>>>>>
>>>>>>
>>>>>> Trace must remain experimental until controlled by a meson option, e.g.
>>>>> "enable_trace", whereby trace can be completely disabled and omitted from
>> the
>>>>> compiled application/libraries/drivers at build time.
>>>>>
>>>>> This seems unrelated to marking the API stable as regardless of the
>>>>> API state at the moment, this code is always present.
>>>>
>>>> I cannot foresee if disabling trace at build time will require changes to
>> the trace API. So I'm being cautious here.
>>>>
>>>> However, if Jerin (as author of the trace subsystem) foresees that it will
>> be possible to disable trace at build time without affecting the trace API, I
>> don't object to marking the trace API (or some of it) stable.
>>>
>>> I don't for foresee any ABI changes when adding disabling trace
>>> compile time support. However, I don't understand why we need to do
>>> that. In the sense, fast path functions are already having an option
>>> to compile out.
>>> Slow path functions can be disabled at runtime at the cost of 1 cycle
>>> as instrumentation cost. Having said that, I don't have any concern
>>> about disabling trace as an option.
>>>
>>
>> I agree with Jerin, I don't see motivation to disable slow path traces
>> when they can be disabled in runtime.
>> And fast path traces already have compile flag to disable them.
>>
>> Build time configurations in long term has problems too, so I am for not
>> using them unless we don't have to.
>
> For some use cases, trace is dead code, and should be omitted.
> You don't want dead code in production systems.
>
> Please remember that DPDK is also being used in highly optimized embedded systems, hardware appliances and other systems where memory is not abundant.
>
> DPDK is not only for cloud and distros. ;-)
>
> The CI only tests DPDK with a build time configuration expected to be usable for distros.
> I'm not asking to change that.
> I'm only asking for more build time configurability to support other use cases.
>
I see, but that build time configuration argument exists in multiple
aspects. And with meson switch we lean to dynamic configuration approach.
When a build time config introduced, again and again we are having cases
that specific code enabled with compile time macro broken and nobody
noticed.
Having code enabled always and configured in runtime produces more
robust deliverable.
We are aware that DPDK is used in embedded device, but they are not
mostly very resource restricted devices, is it really matter to have a
few megabytes (I didn't check but I expect this the max binary size can
increase with tracing code) larger DPDK binary, does it really makes any
difference?
^ permalink raw reply [relevance 0%]
* RE: [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK
2024-09-06 13:17 0% ` Burakov, Anatoly
@ 2024-09-06 13:58 3% ` Morten Brørup
0 siblings, 0 replies; 200+ results
From: Morten Brørup @ 2024-09-06 13:58 UTC (permalink / raw)
To: Burakov, Anatoly, Bruce Richardson; +Cc: dev
> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
> Sent: Friday, 6 September 2024 15.18
> Subject: Re: [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK
>
> On 9/6/2024 3:07 PM, Bruce Richardson wrote:
> > On Fri, Sep 06, 2024 at 03:02:53PM +0200, Morten Brørup wrote:
> >>> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
> >>> Sent: Friday, 6 September 2024 14.46
> >>>
> >>> On 9/6/2024 2:37 PM, Morten Brørup wrote:
> >>>>> From: Anatoly Burakov [mailto:anatoly.burakov@intel.com]
> >>>>> Sent: Friday, 6 September 2024 13.47
> >>>>> To: dev@dpdk.org
> >>>>> Subject: [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in
> DPDK
> >>>>>
> >>>>> While initially, DPDK has used the term "socket ID" to refer to physical
> >>>>> package
> >>>>> ID, the last time DPDK read "physical_package_id" for socket ID was ~9
> >>> years
> >>>>> ago, so it's been a while since we've actually switched over to using
> the
> >>> term
> >>>>> "socket" to mean "NUMA node".
> >>>>>
> >>>>> This wasn't a problem before, as most systems had one NUMA node per
> >>> physical
> >>>>> socket. However, in the last few years, more and more systems have
> multiple
> >>>>> NUMA
> >>>>> nodes per physical CPU socket. Since DPDK used NUMA nodes already, the
> >>>>> transition was pretty seamless, however now we're faced with a situation
> >>> when
> >>>>> most of our documentation still uses outdated terms, and our API is ripe
> >>> with
> >>>>> references to "sockets" when in actuality we mean "NUMA nodes". This
> could
> >>> be
> >>>>> a
> >>>>> source of confusion.
> >>>>>
> >>>>> While completely renaming all of our API's would be a huge effort, will
> >>> take a
> >>>>> long time and arguably wouldn't even be worth the API breakages (given
> that
> >>>>> this
> >>>>> mismatch between terminology and reality is implicitly understood by
> most
> >>>>> people
> >>>>> working on DPDK, and so this isn't so much of a problem in practice), we
> >>> can
> >>>>> do
> >>>>> some tweaks around the edges and at least document this unfortunate
> >>> reality.
> >>>>>
> >>>>> This patchset suggests the following changes:
> >>>>>
> >>>>> - Update rte_socket/rte_lcore documentation to refer to NUMA nodes
> rather
> >>> than
> >>>>> sockets - Rename internal structures' fields to better reflect this
> >>> intention
> >>>>> -
> >>>>> Rename --socket-mem/--socket-limit flags to refer to NUMA rather than
> >>> sockets
> >>>>> -
> >>>>> Add internal API to get physical package ID [1]
> >>>>>
> >>>>> The documentation is updated to refer to new EAL flags, but is otherwise
> >>> left
> >>>>> untouched, and instead the entry in "glossary" is amended to indicate
> that
> >>>>> when
> >>>>> DPDK documentation refers to "sockets", it actually means "NUMA ID's".
> As
> >>> next
> >>>>> steps, we could rename all API parameters to refer to NUMA ID rather
> than
> >>>>> socket
> >>>>> ID - this would not break neither API nor ABI, and instead would be a
> >>>>> documentation change in practice.
> >>>>>
> >>>>> [1] This could be used to group lcores by physical package, see e.g.
> >>>>> discussion
> >>>>> under this patch:
> >>>>> https://patches.dpdk.org/project/dpdk/cover/20240827151014.201-1-
> >>>>> vipin.varghese@amd.com/
> >>>>
> >>>> Thank you for cleaning this up, Anatoly.
> >>>>
> >>>> I would prefer to take one more step and also rename functions and
> >>> parameters, e.g. rte_socket_id() -> rte_numa_id().
> >>>>
> >>>> For backwards compatibility, macros/functions with the old names can be
> >>> added.
> >>>>
> >>>
> >>> I don't think we can do such changes without deprecation notices, but
> >>> it's a good candidate for next release.
> >>
> >> Perhaps we can keep ABI compatibility by adding wrapper functions with the
> old names/parameters, which simply call the same functions with the new
> names/parameters.
> >>
> >> The Devil is in the details, and I haven't looked deeply into this. So take
> with a grain of salt.
> >>
> >>>
> >>> I have thought about including parameter renames in this patchset, but
> >>> for now I decided against doing so. I can certainly include this in the
> >>> next revision if that's something community is willing to accept.
> >>
> >> I agree with your decision on this. Renaming the parameters without
> renaming the functions could be confusing.
> >>
> >
> > I actually wonder if that is true. If we are simply renaming the parameters
> > without:
> > a) changing their types
> > b) changing the function behaviour
> > then it is neither an API nor an ABI break. If we were to do so, it would
> > be like changing a comment, since the actual parameter name is purely a
> > convenience to hint to the user what the value being passed actually does.
> >
> > That only applies for function parameters though. For any defines or macros
> > that need renaming, then we are into API break territory and we would want
> > backward compatible versions of same.
> >
>
> To be clear, I was referring to the former rather than the latter;
> renaming public API function parameters/structure fields can be done
> relatively easily and won't break anything.
Agree.
By "confusing" I was referring to source code readability; e.g. an application calls some rte_socket_id() and passes the returned value on as a numa_id parameter to some other rte_socket_xyz() function.
I have no strong opinion on this.
But I would prefer going all the way, and also fix the API, i.e. rename the functions etc.
And I'm hoping ABI/API backwards compatibility can be preserved by adding wrappers with the previous names.
> If there is consensus on
> going further than I have with this patchset, I can certainly do so.
^ permalink raw reply [relevance 3%]
* Re: [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK
2024-09-06 13:07 3% ` Bruce Richardson
@ 2024-09-06 13:17 0% ` Burakov, Anatoly
2024-09-06 13:58 3% ` Morten Brørup
0 siblings, 1 reply; 200+ results
From: Burakov, Anatoly @ 2024-09-06 13:17 UTC (permalink / raw)
To: Bruce Richardson, Morten Brørup; +Cc: dev
On 9/6/2024 3:07 PM, Bruce Richardson wrote:
> On Fri, Sep 06, 2024 at 03:02:53PM +0200, Morten Brørup wrote:
>>> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
>>> Sent: Friday, 6 September 2024 14.46
>>>
>>> On 9/6/2024 2:37 PM, Morten Brørup wrote:
>>>>> From: Anatoly Burakov [mailto:anatoly.burakov@intel.com]
>>>>> Sent: Friday, 6 September 2024 13.47
>>>>> To: dev@dpdk.org
>>>>> Subject: [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK
>>>>>
>>>>> While initially, DPDK has used the term "socket ID" to refer to physical
>>>>> package
>>>>> ID, the last time DPDK read "physical_package_id" for socket ID was ~9
>>> years
>>>>> ago, so it's been a while since we've actually switched over to using the
>>> term
>>>>> "socket" to mean "NUMA node".
>>>>>
>>>>> This wasn't a problem before, as most systems had one NUMA node per
>>> physical
>>>>> socket. However, in the last few years, more and more systems have multiple
>>>>> NUMA
>>>>> nodes per physical CPU socket. Since DPDK used NUMA nodes already, the
>>>>> transition was pretty seamless, however now we're faced with a situation
>>> when
>>>>> most of our documentation still uses outdated terms, and our API is ripe
>>> with
>>>>> references to "sockets" when in actuality we mean "NUMA nodes". This could
>>> be
>>>>> a
>>>>> source of confusion.
>>>>>
>>>>> While completely renaming all of our API's would be a huge effort, will
>>> take a
>>>>> long time and arguably wouldn't even be worth the API breakages (given that
>>>>> this
>>>>> mismatch between terminology and reality is implicitly understood by most
>>>>> people
>>>>> working on DPDK, and so this isn't so much of a problem in practice), we
>>> can
>>>>> do
>>>>> some tweaks around the edges and at least document this unfortunate
>>> reality.
>>>>>
>>>>> This patchset suggests the following changes:
>>>>>
>>>>> - Update rte_socket/rte_lcore documentation to refer to NUMA nodes rather
>>> than
>>>>> sockets - Rename internal structures' fields to better reflect this
>>> intention
>>>>> -
>>>>> Rename --socket-mem/--socket-limit flags to refer to NUMA rather than
>>> sockets
>>>>> -
>>>>> Add internal API to get physical package ID [1]
>>>>>
>>>>> The documentation is updated to refer to new EAL flags, but is otherwise
>>> left
>>>>> untouched, and instead the entry in "glossary" is amended to indicate that
>>>>> when
>>>>> DPDK documentation refers to "sockets", it actually means "NUMA ID's". As
>>> next
>>>>> steps, we could rename all API parameters to refer to NUMA ID rather than
>>>>> socket
>>>>> ID - this would not break neither API nor ABI, and instead would be a
>>>>> documentation change in practice.
>>>>>
>>>>> [1] This could be used to group lcores by physical package, see e.g.
>>>>> discussion
>>>>> under this patch:
>>>>> https://patches.dpdk.org/project/dpdk/cover/20240827151014.201-1-
>>>>> vipin.varghese@amd.com/
>>>>
>>>> Thank you for cleaning this up, Anatoly.
>>>>
>>>> I would prefer to take one more step and also rename functions and
>>> parameters, e.g. rte_socket_id() -> rte_numa_id().
>>>>
>>>> For backwards compatibility, macros/functions with the old names can be
>>> added.
>>>>
>>>
>>> I don't think we can do such changes without deprecation notices, but
>>> it's a good candidate for next release.
>>
>> Perhaps we can keep ABI compatibility by adding wrapper functions with the old names/parameters, which simply call the same functions with the new names/parameters.
>>
>> The Devil is in the details, and I haven't looked deeply into this. So take with a grain of salt.
>>
>>>
>>> I have thought about including parameter renames in this patchset, but
>>> for now I decided against doing so. I can certainly include this in the
>>> next revision if that's something community is willing to accept.
>>
>> I agree with your decision on this. Renaming the parameters without renaming the functions could be confusing.
>>
>
> I actually wonder if that is true. If we are simply renaming the parameters
> without:
> a) changing their types
> b) changing the function behaviour
> then it is neither an API nor an ABI break. If we were to do so, it would
> be like changing a comment, since the actual parameter name is purely a
> convenience to hint to the user what the value being passed actually does.
>
> That only applies for function parameters though. For any defines or macros
> that need renaming, then we are into API break territory and we would want
> backward compatible versions of same.
>
To be clear, I was referring to the former rather than the latter;
renaming public API function parameters/structure fields can be done
relatively easily and won't break anything. If there is consensus on
going further than I have with this patchset, I can certainly do so.
--
Thanks,
Anatoly
^ permalink raw reply [relevance 0%]
* Re: [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK
2024-09-06 13:02 3% ` Morten Brørup
@ 2024-09-06 13:07 3% ` Bruce Richardson
2024-09-06 13:17 0% ` Burakov, Anatoly
0 siblings, 1 reply; 200+ results
From: Bruce Richardson @ 2024-09-06 13:07 UTC (permalink / raw)
To: Morten Brørup; +Cc: Burakov, Anatoly, dev
On Fri, Sep 06, 2024 at 03:02:53PM +0200, Morten Brørup wrote:
> > From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
> > Sent: Friday, 6 September 2024 14.46
> >
> > On 9/6/2024 2:37 PM, Morten Brørup wrote:
> > >> From: Anatoly Burakov [mailto:anatoly.burakov@intel.com]
> > >> Sent: Friday, 6 September 2024 13.47
> > >> To: dev@dpdk.org
> > >> Subject: [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK
> > >>
> > >> While initially, DPDK has used the term "socket ID" to refer to physical
> > >> package
> > >> ID, the last time DPDK read "physical_package_id" for socket ID was ~9
> > years
> > >> ago, so it's been a while since we've actually switched over to using the
> > term
> > >> "socket" to mean "NUMA node".
> > >>
> > >> This wasn't a problem before, as most systems had one NUMA node per
> > physical
> > >> socket. However, in the last few years, more and more systems have multiple
> > >> NUMA
> > >> nodes per physical CPU socket. Since DPDK used NUMA nodes already, the
> > >> transition was pretty seamless, however now we're faced with a situation
> > when
> > >> most of our documentation still uses outdated terms, and our API is ripe
> > with
> > >> references to "sockets" when in actuality we mean "NUMA nodes". This could
> > be
> > >> a
> > >> source of confusion.
> > >>
> > >> While completely renaming all of our API's would be a huge effort, will
> > take a
> > >> long time and arguably wouldn't even be worth the API breakages (given that
> > >> this
> > >> mismatch between terminology and reality is implicitly understood by most
> > >> people
> > >> working on DPDK, and so this isn't so much of a problem in practice), we
> > can
> > >> do
> > >> some tweaks around the edges and at least document this unfortunate
> > reality.
> > >>
> > >> This patchset suggests the following changes:
> > >>
> > >> - Update rte_socket/rte_lcore documentation to refer to NUMA nodes rather
> > than
> > >> sockets - Rename internal structures' fields to better reflect this
> > intention
> > >> -
> > >> Rename --socket-mem/--socket-limit flags to refer to NUMA rather than
> > sockets
> > >> -
> > >> Add internal API to get physical package ID [1]
> > >>
> > >> The documentation is updated to refer to new EAL flags, but is otherwise
> > left
> > >> untouched, and instead the entry in "glossary" is amended to indicate that
> > >> when
> > >> DPDK documentation refers to "sockets", it actually means "NUMA ID's". As
> > next
> > >> steps, we could rename all API parameters to refer to NUMA ID rather than
> > >> socket
> > >> ID - this would not break neither API nor ABI, and instead would be a
> > >> documentation change in practice.
> > >>
> > >> [1] This could be used to group lcores by physical package, see e.g.
> > >> discussion
> > >> under this patch:
> > >> https://patches.dpdk.org/project/dpdk/cover/20240827151014.201-1-
> > >> vipin.varghese@amd.com/
> > >
> > > Thank you for cleaning this up, Anatoly.
> > >
> > > I would prefer to take one more step and also rename functions and
> > parameters, e.g. rte_socket_id() -> rte_numa_id().
> > >
> > > For backwards compatibility, macros/functions with the old names can be
> > added.
> > >
> >
> > I don't think we can do such changes without deprecation notices, but
> > it's a good candidate for next release.
>
> Perhaps we can keep ABI compatibility by adding wrapper functions with the old names/parameters, which simply call the same functions with the new names/parameters.
>
> The Devil is in the details, and I haven't looked deeply into this. So take with a grain of salt.
>
> >
> > I have thought about including parameter renames in this patchset, but
> > for now I decided against doing so. I can certainly include this in the
> > next revision if that's something community is willing to accept.
>
> I agree with your decision on this. Renaming the parameters without renaming the functions could be confusing.
>
I actually wonder if that is true. If we are simply renaming the parameters
without:
a) changing their types
b) changing the function behaviour
then it is neither an API nor an ABI break. If we were to do so, it would
be like changing a comment, since the actual parameter name is purely a
convenience to hint to the user what the value being passed actually does.
That only applies for function parameters though. For any defines or macros
that need renaming, then we are into API break territory and we would want
backward compatible versions of same.
/Bruce
^ permalink raw reply [relevance 3%]
* RE: [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK
2024-09-06 12:46 0% ` Burakov, Anatoly
@ 2024-09-06 13:02 3% ` Morten Brørup
2024-09-06 13:07 3% ` Bruce Richardson
0 siblings, 1 reply; 200+ results
From: Morten Brørup @ 2024-09-06 13:02 UTC (permalink / raw)
To: Burakov, Anatoly, dev
> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
> Sent: Friday, 6 September 2024 14.46
>
> On 9/6/2024 2:37 PM, Morten Brørup wrote:
> >> From: Anatoly Burakov [mailto:anatoly.burakov@intel.com]
> >> Sent: Friday, 6 September 2024 13.47
> >> To: dev@dpdk.org
> >> Subject: [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK
> >>
> >> While initially, DPDK has used the term "socket ID" to refer to physical
> >> package
> >> ID, the last time DPDK read "physical_package_id" for socket ID was ~9
> years
> >> ago, so it's been a while since we've actually switched over to using the
> term
> >> "socket" to mean "NUMA node".
> >>
> >> This wasn't a problem before, as most systems had one NUMA node per
> physical
> >> socket. However, in the last few years, more and more systems have multiple
> >> NUMA
> >> nodes per physical CPU socket. Since DPDK used NUMA nodes already, the
> >> transition was pretty seamless, however now we're faced with a situation
> when
> >> most of our documentation still uses outdated terms, and our API is ripe
> with
> >> references to "sockets" when in actuality we mean "NUMA nodes". This could
> be
> >> a
> >> source of confusion.
> >>
> >> While completely renaming all of our API's would be a huge effort, will
> take a
> >> long time and arguably wouldn't even be worth the API breakages (given that
> >> this
> >> mismatch between terminology and reality is implicitly understood by most
> >> people
> >> working on DPDK, and so this isn't so much of a problem in practice), we
> can
> >> do
> >> some tweaks around the edges and at least document this unfortunate
> reality.
> >>
> >> This patchset suggests the following changes:
> >>
> >> - Update rte_socket/rte_lcore documentation to refer to NUMA nodes rather
> than
> >> sockets - Rename internal structures' fields to better reflect this
> intention
> >> -
> >> Rename --socket-mem/--socket-limit flags to refer to NUMA rather than
> sockets
> >> -
> >> Add internal API to get physical package ID [1]
> >>
> >> The documentation is updated to refer to new EAL flags, but is otherwise
> left
> >> untouched, and instead the entry in "glossary" is amended to indicate that
> >> when
> >> DPDK documentation refers to "sockets", it actually means "NUMA ID's". As
> next
> >> steps, we could rename all API parameters to refer to NUMA ID rather than
> >> socket
> >> ID - this would not break neither API nor ABI, and instead would be a
> >> documentation change in practice.
> >>
> >> [1] This could be used to group lcores by physical package, see e.g.
> >> discussion
> >> under this patch:
> >> https://patches.dpdk.org/project/dpdk/cover/20240827151014.201-1-
> >> vipin.varghese@amd.com/
> >
> > Thank you for cleaning this up, Anatoly.
> >
> > I would prefer to take one more step and also rename functions and
> parameters, e.g. rte_socket_id() -> rte_numa_id().
> >
> > For backwards compatibility, macros/functions with the old names can be
> added.
> >
>
> I don't think we can do such changes without deprecation notices, but
> it's a good candidate for next release.
Perhaps we can keep ABI compatibility by adding wrapper functions with the old names/parameters, which simply call the same functions with the new names/parameters.
The Devil is in the details, and I haven't looked deeply into this. So take with a grain of salt.
>
> I have thought about including parameter renames in this patchset, but
> for now I decided against doing so. I can certainly include this in the
> next revision if that's something community is willing to accept.
I agree with your decision on this. Renaming the parameters without renaming the functions could be confusing.
If we cannot take the additional step to rename the functions, let's also not rename their parameters.
>
> --
> Thanks,
> Anatoly
^ permalink raw reply [relevance 3%]
* Re: [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK
2024-09-06 12:37 0% ` Morten Brørup
@ 2024-09-06 12:46 0% ` Burakov, Anatoly
2024-09-06 13:02 3% ` Morten Brørup
0 siblings, 1 reply; 200+ results
From: Burakov, Anatoly @ 2024-09-06 12:46 UTC (permalink / raw)
To: Morten Brørup, dev
On 9/6/2024 2:37 PM, Morten Brørup wrote:
>> From: Anatoly Burakov [mailto:anatoly.burakov@intel.com]
>> Sent: Friday, 6 September 2024 13.47
>> To: dev@dpdk.org
>> Subject: [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK
>>
>> While initially, DPDK has used the term "socket ID" to refer to physical
>> package
>> ID, the last time DPDK read "physical_package_id" for socket ID was ~9 years
>> ago, so it's been a while since we've actually switched over to using the term
>> "socket" to mean "NUMA node".
>>
>> This wasn't a problem before, as most systems had one NUMA node per physical
>> socket. However, in the last few years, more and more systems have multiple
>> NUMA
>> nodes per physical CPU socket. Since DPDK used NUMA nodes already, the
>> transition was pretty seamless, however now we're faced with a situation when
>> most of our documentation still uses outdated terms, and our API is ripe with
>> references to "sockets" when in actuality we mean "NUMA nodes". This could be
>> a
>> source of confusion.
>>
>> While completely renaming all of our API's would be a huge effort, will take a
>> long time and arguably wouldn't even be worth the API breakages (given that
>> this
>> mismatch between terminology and reality is implicitly understood by most
>> people
>> working on DPDK, and so this isn't so much of a problem in practice), we can
>> do
>> some tweaks around the edges and at least document this unfortunate reality.
>>
>> This patchset suggests the following changes:
>>
>> - Update rte_socket/rte_lcore documentation to refer to NUMA nodes rather than
>> sockets - Rename internal structures' fields to better reflect this intention
>> -
>> Rename --socket-mem/--socket-limit flags to refer to NUMA rather than sockets
>> -
>> Add internal API to get physical package ID [1]
>>
>> The documentation is updated to refer to new EAL flags, but is otherwise left
>> untouched, and instead the entry in "glossary" is amended to indicate that
>> when
>> DPDK documentation refers to "sockets", it actually means "NUMA ID's". As next
>> steps, we could rename all API parameters to refer to NUMA ID rather than
>> socket
>> ID - this would not break neither API nor ABI, and instead would be a
>> documentation change in practice.
>>
>> [1] This could be used to group lcores by physical package, see e.g.
>> discussion
>> under this patch:
>> https://patches.dpdk.org/project/dpdk/cover/20240827151014.201-1-
>> vipin.varghese@amd.com/
>
> Thank you for cleaning this up, Anatoly.
>
> I would prefer to take one more step and also rename functions and parameters, e.g. rte_socket_id() -> rte_numa_id().
>
> For backwards compatibility, macros/functions with the old names can be added.
>
I don't think we can do such changes without deprecation notices, but
it's a good candidate for next release.
I have thought about including parameter renames in this patchset, but
for now I decided against doing so. I can certainly include this in the
next revision if that's something community is willing to accept.
--
Thanks,
Anatoly
^ permalink raw reply [relevance 0%]
* RE: [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK
2024-09-06 11:47 3% [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK Anatoly Burakov
@ 2024-09-06 12:37 0% ` Morten Brørup
2024-09-06 12:46 0% ` Burakov, Anatoly
2024-09-09 7:51 0% ` fengchengwen
1 sibling, 1 reply; 200+ results
From: Morten Brørup @ 2024-09-06 12:37 UTC (permalink / raw)
To: Anatoly Burakov, dev
> From: Anatoly Burakov [mailto:anatoly.burakov@intel.com]
> Sent: Friday, 6 September 2024 13.47
> To: dev@dpdk.org
> Subject: [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK
>
> While initially, DPDK has used the term "socket ID" to refer to physical
> package
> ID, the last time DPDK read "physical_package_id" for socket ID was ~9 years
> ago, so it's been a while since we've actually switched over to using the term
> "socket" to mean "NUMA node".
>
> This wasn't a problem before, as most systems had one NUMA node per physical
> socket. However, in the last few years, more and more systems have multiple
> NUMA
> nodes per physical CPU socket. Since DPDK used NUMA nodes already, the
> transition was pretty seamless, however now we're faced with a situation when
> most of our documentation still uses outdated terms, and our API is ripe with
> references to "sockets" when in actuality we mean "NUMA nodes". This could be
> a
> source of confusion.
>
> While completely renaming all of our API's would be a huge effort, will take a
> long time and arguably wouldn't even be worth the API breakages (given that
> this
> mismatch between terminology and reality is implicitly understood by most
> people
> working on DPDK, and so this isn't so much of a problem in practice), we can
> do
> some tweaks around the edges and at least document this unfortunate reality.
>
> This patchset suggests the following changes:
>
> - Update rte_socket/rte_lcore documentation to refer to NUMA nodes rather than
> sockets - Rename internal structures' fields to better reflect this intention
> -
> Rename --socket-mem/--socket-limit flags to refer to NUMA rather than sockets
> -
> Add internal API to get physical package ID [1]
>
> The documentation is updated to refer to new EAL flags, but is otherwise left
> untouched, and instead the entry in "glossary" is amended to indicate that
> when
> DPDK documentation refers to "sockets", it actually means "NUMA ID's". As next
> steps, we could rename all API parameters to refer to NUMA ID rather than
> socket
> ID - this would not break neither API nor ABI, and instead would be a
> documentation change in practice.
>
> [1] This could be used to group lcores by physical package, see e.g.
> discussion
> under this patch:
> https://patches.dpdk.org/project/dpdk/cover/20240827151014.201-1-
> vipin.varghese@amd.com/
Thank you for cleaning this up, Anatoly.
I would prefer to take one more step and also rename functions and parameters, e.g. rte_socket_id() -> rte_numa_id().
For backwards compatibility, macros/functions with the old names can be added.
^ permalink raw reply [relevance 0%]
* [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK
@ 2024-09-06 11:47 3% Anatoly Burakov
2024-09-06 12:37 0% ` Morten Brørup
2024-09-09 7:51 0% ` fengchengwen
0 siblings, 2 replies; 200+ results
From: Anatoly Burakov @ 2024-09-06 11:47 UTC (permalink / raw)
To: dev
While initially, DPDK has used the term "socket ID" to refer to physical package
ID, the last time DPDK read "physical_package_id" for socket ID was ~9 years
ago, so it's been a while since we've actually switched over to using the term
"socket" to mean "NUMA node".
This wasn't a problem before, as most systems had one NUMA node per physical
socket. However, in the last few years, more and more systems have multiple NUMA
nodes per physical CPU socket. Since DPDK used NUMA nodes already, the
transition was pretty seamless, however now we're faced with a situation when
most of our documentation still uses outdated terms, and our API is ripe with
references to "sockets" when in actuality we mean "NUMA nodes". This could be a
source of confusion.
While completely renaming all of our API's would be a huge effort, will take a
long time and arguably wouldn't even be worth the API breakages (given that this
mismatch between terminology and reality is implicitly understood by most people
working on DPDK, and so this isn't so much of a problem in practice), we can do
some tweaks around the edges and at least document this unfortunate reality.
This patchset suggests the following changes:
- Update rte_socket/rte_lcore documentation to refer to NUMA nodes rather than
sockets - Rename internal structures' fields to better reflect this intention -
Rename --socket-mem/--socket-limit flags to refer to NUMA rather than sockets -
Add internal API to get physical package ID [1]
The documentation is updated to refer to new EAL flags, but is otherwise left
untouched, and instead the entry in "glossary" is amended to indicate that when
DPDK documentation refers to "sockets", it actually means "NUMA ID's". As next
steps, we could rename all API parameters to refer to NUMA ID rather than socket
ID - this would not break neither API nor ABI, and instead would be a
documentation change in practice.
[1] This could be used to group lcores by physical package, see e.g. discussion
under this patch: https://patches.dpdk.org/project/dpdk/cover/20240827151014.201-1-vipin.varghese@amd.com/
Anatoly Burakov (5):
eal: update socket ID API documentation
lcore: rename socket ID to NUMA ID
eal: rename socket ID to NUMA ID in internal config
eal: rename --socket-mem/--socket-limit
lcore: store physical package ID internally
doc/guides/faq/faq.rst | 4 +--
doc/guides/howto/lm_bond_virtio_sriov.rst | 2 +-
doc/guides/howto/lm_virtio_vhost_user.rst | 2 +-
doc/guides/howto/pvp_reference_benchmark.rst | 4 +--
.../virtio_user_for_container_networking.rst | 2 +-
doc/guides/linux_gsg/build_sample_apps.rst | 20 +++++------
doc/guides/linux_gsg/linux_eal_parameters.rst | 16 ++++-----
doc/guides/nics/mlx4.rst | 2 +-
doc/guides/nics/mlx5.rst | 2 +-
.../prog_guide/env_abstraction_layer.rst | 12 +++----
doc/guides/prog_guide/glossary.rst | 5 ++-
doc/guides/prog_guide/multi_proc_support.rst | 2 +-
doc/guides/sample_app_ug/bbdev_app.rst | 6 ++--
doc/guides/sample_app_ug/ipsec_secgw.rst | 6 ++--
doc/guides/sample_app_ug/vdpa.rst | 2 +-
doc/guides/sample_app_ug/vhost.rst | 4 +--
lib/eal/common/eal_common_dynmem.c | 14 ++++----
lib/eal/common/eal_common_lcore.c | 28 +++++++++++++---
lib/eal/common/eal_common_options.c | 33 ++++++++++---------
lib/eal/common/eal_common_thread.c | 12 +++----
lib/eal/common/eal_internal_cfg.h | 10 +++---
lib/eal/common/eal_options.h | 8 +++--
lib/eal/common/eal_private.h | 5 ++-
lib/eal/common/eal_thread.h | 11 +++++++
lib/eal/common/malloc_heap.c | 2 +-
lib/eal/freebsd/eal.c | 2 +-
lib/eal/freebsd/eal_lcore.c | 6 ++++
lib/eal/include/rte_lcore.h | 25 +++++++-------
lib/eal/linux/eal.c | 22 ++++++-------
lib/eal/linux/eal_lcore.c | 28 ++++++++++++++++
lib/eal/linux/eal_memory.c | 22 ++++++-------
lib/eal/windows/eal.c | 2 +-
lib/eal/windows/eal_lcore.c | 7 ++++
33 files changed, 204 insertions(+), 124 deletions(-)
--
2.43.5
^ permalink raw reply [relevance 3%]
* RE: [PATCH 0/3] eal: mark API's as stable
2024-09-06 8:54 0% ` Ferruh Yigit
@ 2024-09-06 10:04 0% ` Morten Brørup
2024-09-06 14:12 0% ` Ferruh Yigit
0 siblings, 1 reply; 200+ results
From: Morten Brørup @ 2024-09-06 10:04 UTC (permalink / raw)
To: Ferruh Yigit, Jerin Jacob
Cc: David Marchand, Jerin Jacob Kollanukkaran, Stephen Hemminger,
bruce.richardson, Sunil Kumar Kori, dev, Thomas Monjalon
> From: Ferruh Yigit [mailto:ferruh.yigit@amd.com]
> Sent: Friday, 6 September 2024 10.54
>
> On 9/5/2024 3:01 PM, Jerin Jacob wrote:
> > On Thu, Sep 5, 2024 at 3:14 PM Morten Brørup <mb@smartsharesystems.com>
> wrote:
> >>
> >>> From: David Marchand [mailto:david.marchand@redhat.com]
> >>> Sent: Thursday, 5 September 2024 11.03
> >>>
> >>> On Thu, Sep 5, 2024 at 10:55 AM Morten Brørup <mb@smartsharesystems.com>
> >>> wrote:
> >>>>
> >>>>> From: David Marchand [mailto:david.marchand@redhat.com]
> >>>>> Sent: Thursday, 5 September 2024 09.59
> >>>>>
> >>>>> On Wed, Sep 4, 2024 at 8:10 PM Stephen Hemminger
> >>>>> <stephen@networkplumber.org> wrote:
> >>>>>>
> >>>>>> The API's in ethtool from before 23.11 should be marked stable.
> >>>>>
> >>>>> EAL* ?
> >>>>>
> >>>>>> Should probably include the trace api's but that is more complex
> change.
> >>>>>
> >>>>> On the trace API itself it should be ok.
> >>>>
> >>>> No!
> >>>
> >>> *sigh*
> >>>
> >>>>
> >>>> Trace must remain experimental until controlled by a meson option, e.g.
> >>> "enable_trace", whereby trace can be completely disabled and omitted from
> the
> >>> compiled application/libraries/drivers at build time.
> >>>
> >>> This seems unrelated to marking the API stable as regardless of the
> >>> API state at the moment, this code is always present.
> >>
> >> I cannot foresee if disabling trace at build time will require changes to
> the trace API. So I'm being cautious here.
> >>
> >> However, if Jerin (as author of the trace subsystem) foresees that it will
> be possible to disable trace at build time without affecting the trace API, I
> don't object to marking the trace API (or some of it) stable.
> >
> > I don't for foresee any ABI changes when adding disabling trace
> > compile time support. However, I don't understand why we need to do
> > that. In the sense, fast path functions are already having an option
> > to compile out.
> > Slow path functions can be disabled at runtime at the cost of 1 cycle
> > as instrumentation cost. Having said that, I don't have any concern
> > about disabling trace as an option.
> >
>
> I agree with Jerin, I don't see motivation to disable slow path traces
> when they can be disabled in runtime.
> And fast path traces already have compile flag to disable them.
>
> Build time configurations in long term has problems too, so I am for not
> using them unless we don't have to.
For some use cases, trace is dead code, and should be omitted.
You don't want dead code in production systems.
Please remember that DPDK is also being used in highly optimized embedded systems, hardware appliances and other systems where memory is not abundant.
DPDK is not only for cloud and distros. ;-)
The CI only tests DPDK with a build time configuration expected to be usable for distros.
I'm not asking to change that.
I'm only asking for more build time configurability to support other use cases.
^ permalink raw reply [relevance 0%]
* Re: [PATCH 0/3] eal: mark API's as stable
2024-09-05 14:01 3% ` Jerin Jacob
2024-09-05 14:18 0% ` Morten Brørup
@ 2024-09-06 8:54 0% ` Ferruh Yigit
2024-09-06 10:04 0% ` Morten Brørup
1 sibling, 1 reply; 200+ results
From: Ferruh Yigit @ 2024-09-06 8:54 UTC (permalink / raw)
To: Jerin Jacob, Morten Brørup
Cc: David Marchand, Jerin Jacob Kollanukkaran, Stephen Hemminger,
bruce.richardson, Sunil Kumar Kori, dev, Thomas Monjalon
On 9/5/2024 3:01 PM, Jerin Jacob wrote:
> On Thu, Sep 5, 2024 at 3:14 PM Morten Brørup <mb@smartsharesystems.com> wrote:
>>
>>> From: David Marchand [mailto:david.marchand@redhat.com]
>>> Sent: Thursday, 5 September 2024 11.03
>>>
>>> On Thu, Sep 5, 2024 at 10:55 AM Morten Brørup <mb@smartsharesystems.com>
>>> wrote:
>>>>
>>>>> From: David Marchand [mailto:david.marchand@redhat.com]
>>>>> Sent: Thursday, 5 September 2024 09.59
>>>>>
>>>>> On Wed, Sep 4, 2024 at 8:10 PM Stephen Hemminger
>>>>> <stephen@networkplumber.org> wrote:
>>>>>>
>>>>>> The API's in ethtool from before 23.11 should be marked stable.
>>>>>
>>>>> EAL* ?
>>>>>
>>>>>> Should probably include the trace api's but that is more complex change.
>>>>>
>>>>> On the trace API itself it should be ok.
>>>>
>>>> No!
>>>
>>> *sigh*
>>>
>>>>
>>>> Trace must remain experimental until controlled by a meson option, e.g.
>>> "enable_trace", whereby trace can be completely disabled and omitted from the
>>> compiled application/libraries/drivers at build time.
>>>
>>> This seems unrelated to marking the API stable as regardless of the
>>> API state at the moment, this code is always present.
>>
>> I cannot foresee if disabling trace at build time will require changes to the trace API. So I'm being cautious here.
>>
>> However, if Jerin (as author of the trace subsystem) foresees that it will be possible to disable trace at build time without affecting the trace API, I don't object to marking the trace API (or some of it) stable.
>
> I don't for foresee any ABI changes when adding disabling trace
> compile time support. However, I don't understand why we need to do
> that. In the sense, fast path functions are already having an option
> to compile out.
> Slow path functions can be disabled at runtime at the cost of 1 cycle
> as instrumentation cost. Having said that, I don't have any concern
> about disabling trace as an option.
>
I agree with Jerin, I don't see motivation to disable slow path traces
when they can be disabled in runtime.
And fast path traces already have compile flag to disable them.
Build time configurations in long term has problems too, so I am for not
using them unless we don't have to.
>
>>
>> Before doing that, rte_trace_mode_get/set() and the accompanying enum rte_trace_mode should be changed to rte_trace_config_get/set() using a new struct rte_trace_config (containing the enum rte_trace_mode, and expandable with new fields as the need arises). This will prepare for e.g. tracing to other destinations than system memory, such as a remote trace collector on the network, like SYSLOG.
>>
>>> Patches welcome if you want it stripped.
>>
>> Don't have time myself, so I suggested it as a code challenge instead. :-)
>>
^ permalink raw reply [relevance 0%]
* Re: [PATCH] eal: increase max file descriptor for secondary process device
2024-09-05 16:20 8% [PATCH] eal: increase max file descriptor for secondary process device Stephen Hemminger
2024-09-06 1:19 0% ` fengchengwen
2024-09-06 6:23 0% ` Morten Brørup
@ 2024-09-06 7:55 3% ` Ferruh Yigit
2 siblings, 0 replies; 200+ results
From: Ferruh Yigit @ 2024-09-06 7:55 UTC (permalink / raw)
To: Stephen Hemminger, dev; +Cc: Tyler Retzlaff
On 9/5/2024 5:20 PM, Stephen Hemminger wrote:
> The TAP and XDP driver both are limited to only 8 queues when
> because of the small limit imposed by EAL. Increase the limit
> now since this release allows changing ABI.
>
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> ---
> doc/guides/rel_notes/release_24_11.rst | 5 +++++
> lib/eal/include/rte_eal.h | 2 +-
> 2 files changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
> index 0ff70d9057..5af70e04c5 100644
> --- a/doc/guides/rel_notes/release_24_11.rst
> +++ b/doc/guides/rel_notes/release_24_11.rst
> @@ -100,6 +100,11 @@ ABI Changes
> Also, make sure to start the actual text at the margin.
> =======================================================
>
> +* The maximum number of file descriptors that can be passed to a secondary process
> + has been increased from 8 to 253 (which is the maximum possible with Unix domain
> + socket). This allows for more queues when using software devices such as TAP
> + and XDP.
> +
>
> Known Issues
> ------------
> diff --git a/lib/eal/include/rte_eal.h b/lib/eal/include/rte_eal.h
> index c2256f832e..c826e143f1 100644
> --- a/lib/eal/include/rte_eal.h
> +++ b/lib/eal/include/rte_eal.h
> @@ -155,7 +155,7 @@ int rte_eal_primary_proc_alive(const char *config_file_path);
> */
> bool rte_mp_disable(void);
>
> -#define RTE_MP_MAX_FD_NUM 8 /* The max amount of fds */
> +#define RTE_MP_MAX_FD_NUM 253 /* The max amount of fds (see SCM_MAX_FD) */
> #define RTE_MP_MAX_NAME_LEN 64 /* The max length of action name */
> #define RTE_MP_MAX_PARAM_LEN 256 /* The max length of param */
> struct rte_mp_msg {
>
It would be nice to add this to deprecation notice first, but it seems
we missed, I think it is safe to update in ABI break release and good to
remove this restriction.
Acked-by: Ferruh Yigit <ferruh.yigit@amd.com>
^ permalink raw reply [relevance 3%]
* RE: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-09-06 6:32 4% ` fengchengwen
2024-09-06 7:45 0% ` [EXTERNAL] " Akhil Goyal
@ 2024-09-06 7:54 0% ` Morten Brørup
2024-09-23 20:41 4% ` Akhil Goyal
1 sibling, 1 reply; 200+ results
From: Morten Brørup @ 2024-09-06 7:54 UTC (permalink / raw)
To: fengchengwen, Akhil Goyal, dev
Cc: thomas, david.marchand, hemant.agrawal, anoobj,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde, rnagadheeraj,
mdr
> From: fengchengwen [mailto:fengchengwen@huawei.com]
> Sent: Friday, 6 September 2024 08.33
>
> On 2024/9/5 23:09, Morten Brørup wrote:
> >> +++ b/app/test/test_cryptodev_asym.c
> >> @@ -581,7 +581,7 @@ static inline void print_asym_capa(
> >> rte_cryptodev_asym_get_xform_string(capa->xform_type));
> >> printf("operation supported -");
> >>
> >> - for (i = 0; i < RTE_CRYPTO_ASYM_OP_LIST_END; i++) {
> >> + for (i = 0; i < rte_crypto_asym_op_list_end(); i++) {
> >
> >> +++ b/lib/cryptodev/rte_crypto_asym.h
> >> +static inline int
> >> +rte_crypto_asym_xform_type_list_end(void)
> >> +{
> >> + return RTE_CRYPTO_ASYM_XFORM_SM2 + 1;
> >> +}
> >> +
> >> /**
> >> * Asymmetric crypto operation type variants
> >> + * Note: Update rte_crypto_asym_op_list_end for every new type added.
> >> */
> >> enum rte_crypto_asym_op_type {
> >> RTE_CRYPTO_ASYM_OP_ENCRYPT,
> >> @@ -135,9 +141,14 @@ enum rte_crypto_asym_op_type {
> >> /**< Signature Generation operation */
> >> RTE_CRYPTO_ASYM_OP_VERIFY,
> >> /**< Signature Verification operation */
> >> - RTE_CRYPTO_ASYM_OP_LIST_END
> >> };
> >>
> >> +static inline int
> >> +rte_crypto_asym_op_list_end(void)
> >> +{
> >> + return RTE_CRYPTO_ASYM_OP_VERIFY + 1;
> >> +}
> >
> > I like the concept of replacing an "last enum value" with a "last enum
> function" for API/ABI compatibility purposes.
>
> +1
> There are many such define in DPDK, e.g. RTE_ETH_EVENT_MAX
>
> >
> > Here's an idea...
> >
> > We can introduce a generic design pattern where we keep the _LIST_END enum
> value at the end, somehow marking it private (and not part of the API/ABI),
> and move the _list_end() function inside the C file, so it uses the _LIST_END
> enum value that the library was built with. E.g. like this:
> >
> >
> > In the header file:
> >
> > enum rte_crypto_asym_op_type {
> > RTE_CRYPTO_ASYM_OP_VERIFY,
> > /**< Signature Verification operation */
> > #if RTE_BUILDING_INTERNAL
> > __RTE_CRYPTO_ASYM_OP_LIST_END /* internal */
> > #endif
> > }
> >
> > int rte_crypto_asym_op_list_end(void);
> >
> >
> > And in the associated library code file, when including rte_crypto_asym.h:
> >
> > #define RTE_BUILDING_INTERNAL
> > #include <cryptodev/rte_crypto_asym.h>
> >
> > int
> > rte_crypto_asym_op_list_end(void)
> > {
> > return __RTE_CRYPTO_ASYM_OP_LIST_END;
> > }
>
> It's more generic, and also keep LIST_END in the define, we just add new enum
> before it.
> But based on my understanding of ABI compatibility, from the point view of
> application,
> this API should return old-value even with the new library, but it will return
> new-value
> with new library. It could also break ABI.
>
> So this API should force inline, just as this patch did. But it seem can't
> work if move
> this API to header file and add static inline.
Maybe a combination, returning the lowest end of the two versions of the list, would work...
----------------------------------
Common header file (rte_common.h):
----------------------------------
/* Add at end of enum list in the header file. */
#define RTE_ENUM_LIST_END(name) \
_ # name # _ENUM_LIST_END /**< @internal */
/* Add somewhere in header file, preferably after the enum list. */
#define rte_declare_enum_list_end(name) \
/** @internal */ \
int _# name # _enum_list_end(void); \
\
static int name # _enum_list_end(void) \
{ \
static int cached = 0; \
\
if (likely(cached != 0)) \
return cached; \
\
return cached = RTE_MIN( \
RTE_ENUM_LIST_END(name), \
_ # name # _enum_list_end()); \
} \
\
int _# name # _enum_list_end(void)
/* Add in the library/driver implementation. */
#define rte_define_enum_list_end(name) \
int _# name # _enum_list_end(void) \
{ \
return RTE_ENUM_LIST_END(name); \
} \
\
int _# name # _enum_list_end(void)
--------------------
Library header file:
--------------------
enum rte_crypto_asym_op_type {
RTE_CRYPTO_ASYM_OP_VERIFY,
/**< Signature Verification operation */
RTE_ENUM_LIST_END(rte_crypto_asym_op)
}
rte_declare_enum_list_end(rte_crypto_asym_op);
---------------
Library C file:
---------------
rte_define_enum_list_end(rte_crypto_asym_op);
^ permalink raw reply [relevance 0%]
* RE: [EXTERNAL] Re: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-09-06 6:32 4% ` fengchengwen
@ 2024-09-06 7:45 0% ` Akhil Goyal
2024-10-04 3:56 0% ` Ferruh Yigit
2024-09-06 7:54 0% ` Morten Brørup
1 sibling, 1 reply; 200+ results
From: Akhil Goyal @ 2024-09-06 7:45 UTC (permalink / raw)
To: fengchengwen, Morten Brørup, dev
Cc: thomas, david.marchand, hemant.agrawal, Anoob Joseph,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde,
Nagadheeraj Rottela, mdr
> >
> > Here's an idea...
> >
> > We can introduce a generic design pattern where we keep the _LIST_END enum
> value at the end, somehow marking it private (and not part of the API/ABI), and
> move the _list_end() function inside the C file, so it uses the _LIST_END enum
> value that the library was built with. E.g. like this:
> >
> >
> > In the header file:
> >
> > enum rte_crypto_asym_op_type {
> > RTE_CRYPTO_ASYM_OP_VERIFY,
> > /**< Signature Verification operation */
> > #if RTE_BUILDING_INTERNAL
> > __RTE_CRYPTO_ASYM_OP_LIST_END /* internal */
> > #endif
> > }
> >
> > int rte_crypto_asym_op_list_end(void);
> >
> >
> > And in the associated library code file, when including rte_crypto_asym.h:
> >
> > #define RTE_BUILDING_INTERNAL
> > #include <cryptodev/rte_crypto_asym.h>
> >
> > int
> > rte_crypto_asym_op_list_end(void)
> > {
> > return __RTE_CRYPTO_ASYM_OP_LIST_END;
> > }
>
> It's more generic, and also keep LIST_END in the define, we just add new enum
> before it.
> But based on my understanding of ABI compatibility, from the point view of
> application,
> this API should return old-value even with the new library, but it will return new-
> value
> with new library. It could also break ABI.
>
> So this API should force inline, just as this patch did. But it seem can't work if
> move
> this API to header file and add static inline.
>
Yes, moving to c file does not seem to solve the purpose.
So should we move with the way the patch is submitted or we have some other suggestion?
Regards,
Akhil
^ permalink raw reply [relevance 0%]
* Re: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-09-05 15:09 4% ` Morten Brørup
2024-09-05 15:26 0% ` Kusztal, ArkadiuszX
@ 2024-09-06 6:32 4% ` fengchengwen
2024-09-06 7:45 0% ` [EXTERNAL] " Akhil Goyal
2024-09-06 7:54 0% ` Morten Brørup
1 sibling, 2 replies; 200+ results
From: fengchengwen @ 2024-09-06 6:32 UTC (permalink / raw)
To: Morten Brørup, Akhil Goyal, dev
Cc: thomas, david.marchand, hemant.agrawal, anoobj,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde, rnagadheeraj,
mdr
On 2024/9/5 23:09, Morten Brørup wrote:
>> +++ b/app/test/test_cryptodev_asym.c
>> @@ -581,7 +581,7 @@ static inline void print_asym_capa(
>> rte_cryptodev_asym_get_xform_string(capa->xform_type));
>> printf("operation supported -");
>>
>> - for (i = 0; i < RTE_CRYPTO_ASYM_OP_LIST_END; i++) {
>> + for (i = 0; i < rte_crypto_asym_op_list_end(); i++) {
>
>> +++ b/lib/cryptodev/rte_crypto_asym.h
>> +static inline int
>> +rte_crypto_asym_xform_type_list_end(void)
>> +{
>> + return RTE_CRYPTO_ASYM_XFORM_SM2 + 1;
>> +}
>> +
>> /**
>> * Asymmetric crypto operation type variants
>> + * Note: Update rte_crypto_asym_op_list_end for every new type added.
>> */
>> enum rte_crypto_asym_op_type {
>> RTE_CRYPTO_ASYM_OP_ENCRYPT,
>> @@ -135,9 +141,14 @@ enum rte_crypto_asym_op_type {
>> /**< Signature Generation operation */
>> RTE_CRYPTO_ASYM_OP_VERIFY,
>> /**< Signature Verification operation */
>> - RTE_CRYPTO_ASYM_OP_LIST_END
>> };
>>
>> +static inline int
>> +rte_crypto_asym_op_list_end(void)
>> +{
>> + return RTE_CRYPTO_ASYM_OP_VERIFY + 1;
>> +}
>
> I like the concept of replacing an "last enum value" with a "last enum function" for API/ABI compatibility purposes.
+1
There are many such define in DPDK, e.g. RTE_ETH_EVENT_MAX
>
> Here's an idea...
>
> We can introduce a generic design pattern where we keep the _LIST_END enum value at the end, somehow marking it private (and not part of the API/ABI), and move the _list_end() function inside the C file, so it uses the _LIST_END enum value that the library was built with. E.g. like this:
>
>
> In the header file:
>
> enum rte_crypto_asym_op_type {
> RTE_CRYPTO_ASYM_OP_VERIFY,
> /**< Signature Verification operation */
> #if RTE_BUILDING_INTERNAL
> __RTE_CRYPTO_ASYM_OP_LIST_END /* internal */
> #endif
> }
>
> int rte_crypto_asym_op_list_end(void);
>
>
> And in the associated library code file, when including rte_crypto_asym.h:
>
> #define RTE_BUILDING_INTERNAL
> #include <cryptodev/rte_crypto_asym.h>
>
> int
> rte_crypto_asym_op_list_end(void)
> {
> return __RTE_CRYPTO_ASYM_OP_LIST_END;
> }
It's more generic, and also keep LIST_END in the define, we just add new enum before it.
But based on my understanding of ABI compatibility, from the point view of application,
this API should return old-value even with the new library, but it will return new-value
with new library. It could also break ABI.
So this API should force inline, just as this patch did. But it seem can't work if move
this API to header file and add static inline.
>
> .
>
^ permalink raw reply [relevance 4%]
* RE: [PATCH] eal: increase max file descriptor for secondary process device
2024-09-05 16:20 8% [PATCH] eal: increase max file descriptor for secondary process device Stephen Hemminger
2024-09-06 1:19 0% ` fengchengwen
@ 2024-09-06 6:23 0% ` Morten Brørup
2024-09-06 7:55 3% ` Ferruh Yigit
2 siblings, 0 replies; 200+ results
From: Morten Brørup @ 2024-09-06 6:23 UTC (permalink / raw)
To: Stephen Hemminger, dev; +Cc: Tyler Retzlaff
> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> Sent: Thursday, 5 September 2024 18.20
>
> The TAP and XDP driver both are limited to only 8 queues when
> because of the small limit imposed by EAL. Increase the limit
> now since this release allows changing ABI.
>
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> ---
Acked-by: Morten Brørup <mb@smartsharesystems.com>
^ permalink raw reply [relevance 0%]
* Re: [PATCH] eal: increase max file descriptor for secondary process device
2024-09-05 16:20 8% [PATCH] eal: increase max file descriptor for secondary process device Stephen Hemminger
@ 2024-09-06 1:19 0% ` fengchengwen
2024-09-06 6:23 0% ` Morten Brørup
2024-09-06 7:55 3% ` Ferruh Yigit
2 siblings, 0 replies; 200+ results
From: fengchengwen @ 2024-09-06 1:19 UTC (permalink / raw)
To: Stephen Hemminger, dev; +Cc: Tyler Retzlaff
Acked-by: Chengwen Feng <fengchengwen@huawei.com>
On 2024/9/6 0:20, Stephen Hemminger wrote:
> The TAP and XDP driver both are limited to only 8 queues when
> because of the small limit imposed by EAL. Increase the limit
> now since this release allows changing ABI.
>
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> ---
> doc/guides/rel_notes/release_24_11.rst | 5 +++++
> lib/eal/include/rte_eal.h | 2 +-
> 2 files changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
> index 0ff70d9057..5af70e04c5 100644
> --- a/doc/guides/rel_notes/release_24_11.rst
> +++ b/doc/guides/rel_notes/release_24_11.rst
> @@ -100,6 +100,11 @@ ABI Changes
> Also, make sure to start the actual text at the margin.
> =======================================================
>
> +* The maximum number of file descriptors that can be passed to a secondary process
> + has been increased from 8 to 253 (which is the maximum possible with Unix domain
> + socket). This allows for more queues when using software devices such as TAP
> + and XDP.
> +
>
> Known Issues
> ------------
> diff --git a/lib/eal/include/rte_eal.h b/lib/eal/include/rte_eal.h
> index c2256f832e..c826e143f1 100644
> --- a/lib/eal/include/rte_eal.h
> +++ b/lib/eal/include/rte_eal.h
> @@ -155,7 +155,7 @@ int rte_eal_primary_proc_alive(const char *config_file_path);
> */
> bool rte_mp_disable(void);
>
> -#define RTE_MP_MAX_FD_NUM 8 /* The max amount of fds */
> +#define RTE_MP_MAX_FD_NUM 253 /* The max amount of fds (see SCM_MAX_FD) */
> #define RTE_MP_MAX_NAME_LEN 64 /* The max length of action name */
> #define RTE_MP_MAX_PARAM_LEN 256 /* The max length of param */
> struct rte_mp_msg {
>
^ permalink raw reply [relevance 0%]
* [PATCH] eal: increase max file descriptor for secondary process device
@ 2024-09-05 16:20 8% Stephen Hemminger
2024-09-06 1:19 0% ` fengchengwen
` (2 more replies)
0 siblings, 3 replies; 200+ results
From: Stephen Hemminger @ 2024-09-05 16:20 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger, Tyler Retzlaff
The TAP and XDP driver both are limited to only 8 queues when
because of the small limit imposed by EAL. Increase the limit
now since this release allows changing ABI.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
doc/guides/rel_notes/release_24_11.rst | 5 +++++
lib/eal/include/rte_eal.h | 2 +-
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 0ff70d9057..5af70e04c5 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -100,6 +100,11 @@ ABI Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* The maximum number of file descriptors that can be passed to a secondary process
+ has been increased from 8 to 253 (which is the maximum possible with Unix domain
+ socket). This allows for more queues when using software devices such as TAP
+ and XDP.
+
Known Issues
------------
diff --git a/lib/eal/include/rte_eal.h b/lib/eal/include/rte_eal.h
index c2256f832e..c826e143f1 100644
--- a/lib/eal/include/rte_eal.h
+++ b/lib/eal/include/rte_eal.h
@@ -155,7 +155,7 @@ int rte_eal_primary_proc_alive(const char *config_file_path);
*/
bool rte_mp_disable(void);
-#define RTE_MP_MAX_FD_NUM 8 /* The max amount of fds */
+#define RTE_MP_MAX_FD_NUM 253 /* The max amount of fds (see SCM_MAX_FD) */
#define RTE_MP_MAX_NAME_LEN 64 /* The max length of action name */
#define RTE_MP_MAX_PARAM_LEN 256 /* The max length of param */
struct rte_mp_msg {
--
2.45.2
^ permalink raw reply [relevance 8%]
* RE: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-09-05 15:09 4% ` Morten Brørup
@ 2024-09-05 15:26 0% ` Kusztal, ArkadiuszX
2024-09-06 6:32 4% ` fengchengwen
1 sibling, 0 replies; 200+ results
From: Kusztal, ArkadiuszX @ 2024-09-05 15:26 UTC (permalink / raw)
To: Morten Brørup, Akhil Goyal, dev
Cc: thomas, Marchand, David, hemant.agrawal, anoobj, De Lara Guarch,
Pablo, Trahe, Fiona, Doherty, Declan, matan, g.singh,
fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, Nicolau, Radu, ajit.khaparde, rnagadheeraj,
mdr
> -----Original Message-----
> From: Morten Brørup <mb@smartsharesystems.com>
> Sent: Thursday, September 5, 2024 5:09 PM
> To: Akhil Goyal <gakhil@marvell.com>; dev@dpdk.org
> Cc: thomas@monjalon.net; Marchand, David <david.marchand@redhat.com>;
> hemant.agrawal@nxp.com; anoobj@marvell.com; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>; Trahe, Fiona <fiona.trahe@intel.com>;
> Doherty, Declan <declan.doherty@intel.com>; matan@nvidia.com;
> g.singh@nxp.com; fanzhang.oss@gmail.com; jianjay.zhou@huawei.com;
> asomalap@amd.com; ruifeng.wang@arm.com;
> konstantin.v.ananyev@yandex.ru; Nicolau, Radu <radu.nicolau@intel.com>;
> ajit.khaparde@broadcom.com; rnagadheeraj@marvell.com; mdr@ashroe.eu
> Subject: RE: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
>
> > +++ b/app/test/test_cryptodev_asym.c
> > @@ -581,7 +581,7 @@ static inline void print_asym_capa(
> > rte_cryptodev_asym_get_xform_string(capa-
> >xform_type));
> > printf("operation supported -");
> >
> > - for (i = 0; i < RTE_CRYPTO_ASYM_OP_LIST_END; i++) {
> > + for (i = 0; i < rte_crypto_asym_op_list_end(); i++) {
>
> > +++ b/lib/cryptodev/rte_crypto_asym.h
> > +static inline int
> > +rte_crypto_asym_xform_type_list_end(void)
> > +{
> > + return RTE_CRYPTO_ASYM_XFORM_SM2 + 1; }
> > +
> > /**
> > * Asymmetric crypto operation type variants
> > + * Note: Update rte_crypto_asym_op_list_end for every new type added.
> > */
> > enum rte_crypto_asym_op_type {
> > RTE_CRYPTO_ASYM_OP_ENCRYPT,
> > @@ -135,9 +141,14 @@ enum rte_crypto_asym_op_type {
> > /**< Signature Generation operation */
> > RTE_CRYPTO_ASYM_OP_VERIFY,
> > /**< Signature Verification operation */
> > - RTE_CRYPTO_ASYM_OP_LIST_END
> > };
> >
> > +static inline int
> > +rte_crypto_asym_op_list_end(void)
> > +{
> > + return RTE_CRYPTO_ASYM_OP_VERIFY + 1; }
>
> I like the concept of replacing an "last enum value" with a "last enum function"
> for API/ABI compatibility purposes.
>
> Here's an idea...
>
> We can introduce a generic design pattern where we keep the _LIST_END enum
> value at the end, somehow marking it private (and not part of the API/ABI), and
> move the _list_end() function inside the C file, so it uses the _LIST_END enum
> value that the library was built with. E.g. like this:
>
Why asym crypto API does need these ENDs at all? Will any PMD or the user ever or use it? Sym crypto does not have this at all, as well as rte_crypto_asym_ke_type in asym crypto.
>
> In the header file:
>
> enum rte_crypto_asym_op_type {
> RTE_CRYPTO_ASYM_OP_VERIFY,
> /**< Signature Verification operation */ #if RTE_BUILDING_INTERNAL
> __RTE_CRYPTO_ASYM_OP_LIST_END /* internal */ #endif }
>
> int rte_crypto_asym_op_list_end(void);
>
>
> And in the associated library code file, when including rte_crypto_asym.h:
>
> #define RTE_BUILDING_INTERNAL
> #include <cryptodev/rte_crypto_asym.h>
>
> int
> rte_crypto_asym_op_list_end(void)
> {
> return __RTE_CRYPTO_ASYM_OP_LIST_END;
> }
^ permalink raw reply [relevance 0%]
* RE: [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
2024-09-05 10:14 4% [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs Akhil Goyal
@ 2024-09-05 15:09 4% ` Morten Brørup
2024-09-05 15:26 0% ` Kusztal, ArkadiuszX
2024-09-06 6:32 4% ` fengchengwen
2024-10-04 3:54 5% ` Ferruh Yigit
2024-10-09 11:24 3% ` [PATCH] cryptodev: remove unnecessary list end Akhil Goyal
2 siblings, 2 replies; 200+ results
From: Morten Brørup @ 2024-09-05 15:09 UTC (permalink / raw)
To: Akhil Goyal, dev
Cc: thomas, david.marchand, hemant.agrawal, anoobj,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde, rnagadheeraj,
mdr
> +++ b/app/test/test_cryptodev_asym.c
> @@ -581,7 +581,7 @@ static inline void print_asym_capa(
> rte_cryptodev_asym_get_xform_string(capa->xform_type));
> printf("operation supported -");
>
> - for (i = 0; i < RTE_CRYPTO_ASYM_OP_LIST_END; i++) {
> + for (i = 0; i < rte_crypto_asym_op_list_end(); i++) {
> +++ b/lib/cryptodev/rte_crypto_asym.h
> +static inline int
> +rte_crypto_asym_xform_type_list_end(void)
> +{
> + return RTE_CRYPTO_ASYM_XFORM_SM2 + 1;
> +}
> +
> /**
> * Asymmetric crypto operation type variants
> + * Note: Update rte_crypto_asym_op_list_end for every new type added.
> */
> enum rte_crypto_asym_op_type {
> RTE_CRYPTO_ASYM_OP_ENCRYPT,
> @@ -135,9 +141,14 @@ enum rte_crypto_asym_op_type {
> /**< Signature Generation operation */
> RTE_CRYPTO_ASYM_OP_VERIFY,
> /**< Signature Verification operation */
> - RTE_CRYPTO_ASYM_OP_LIST_END
> };
>
> +static inline int
> +rte_crypto_asym_op_list_end(void)
> +{
> + return RTE_CRYPTO_ASYM_OP_VERIFY + 1;
> +}
I like the concept of replacing an "last enum value" with a "last enum function" for API/ABI compatibility purposes.
Here's an idea...
We can introduce a generic design pattern where we keep the _LIST_END enum value at the end, somehow marking it private (and not part of the API/ABI), and move the _list_end() function inside the C file, so it uses the _LIST_END enum value that the library was built with. E.g. like this:
In the header file:
enum rte_crypto_asym_op_type {
RTE_CRYPTO_ASYM_OP_VERIFY,
/**< Signature Verification operation */
#if RTE_BUILDING_INTERNAL
__RTE_CRYPTO_ASYM_OP_LIST_END /* internal */
#endif
}
int rte_crypto_asym_op_list_end(void);
And in the associated library code file, when including rte_crypto_asym.h:
#define RTE_BUILDING_INTERNAL
#include <cryptodev/rte_crypto_asym.h>
int
rte_crypto_asym_op_list_end(void)
{
return __RTE_CRYPTO_ASYM_OP_LIST_END;
}
^ permalink raw reply [relevance 4%]
* RE: [PATCH 0/3] eal: mark API's as stable
2024-09-05 14:01 3% ` Jerin Jacob
@ 2024-09-05 14:18 0% ` Morten Brørup
2024-09-08 23:58 3% ` Stephen Hemminger
2024-09-06 8:54 0% ` Ferruh Yigit
1 sibling, 1 reply; 200+ results
From: Morten Brørup @ 2024-09-05 14:18 UTC (permalink / raw)
To: Jerin Jacob, David Marchand, Stephen Hemminger
Cc: Jerin Jacob Kollanukkaran, bruce.richardson, Sunil Kumar Kori,
dev, Thomas Monjalon
> From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> Sent: Thursday, 5 September 2024 16.02
>
> On Thu, Sep 5, 2024 at 3:14 PM Morten Brørup <mb@smartsharesystems.com> wrote:
> >
> > > From: David Marchand [mailto:david.marchand@redhat.com]
> > > Sent: Thursday, 5 September 2024 11.03
> > >
> > > On Thu, Sep 5, 2024 at 10:55 AM Morten Brørup <mb@smartsharesystems.com>
> > > wrote:
> > > >
> > > > > From: David Marchand [mailto:david.marchand@redhat.com]
> > > > > Sent: Thursday, 5 September 2024 09.59
> > > > >
> > > > > On Wed, Sep 4, 2024 at 8:10 PM Stephen Hemminger
> > > > > <stephen@networkplumber.org> wrote:
> > > > > >
> > > > > > The API's in ethtool from before 23.11 should be marked stable.
> > > > >
> > > > > EAL* ?
> > > > >
> > > > > > Should probably include the trace api's but that is more complex
> change.
> > > > >
> > > > > On the trace API itself it should be ok.
> > > >
> > > > No!
> > >
> > > *sigh*
> > >
> > > >
> > > > Trace must remain experimental until controlled by a meson option, e.g.
> > > "enable_trace", whereby trace can be completely disabled and omitted from
> the
> > > compiled application/libraries/drivers at build time.
> > >
> > > This seems unrelated to marking the API stable as regardless of the
> > > API state at the moment, this code is always present.
> >
> > I cannot foresee if disabling trace at build time will require changes to
> the trace API. So I'm being cautious here.
> >
> > However, if Jerin (as author of the trace subsystem) foresees that it will
> be possible to disable trace at build time without affecting the trace API, I
> don't object to marking the trace API (or some of it) stable.
>
> I don't for foresee any ABI changes when adding disabling trace
> compile time support.
Based on Jerin's feedback, I'm retracting my objection.
> However, I don't understand why we need to do
> that.
To reduce code size.
Relevant for embedded/memory-constrained systems.
> In the sense, fast path functions are already having an option
> to compile out.
> Slow path functions can be disabled at runtime at the cost of 1 cycle
> as instrumentation cost. Having said that, I don't have any concern
> about disabling trace as an option.
Great.
>
>
> >
> > Before doing that, rte_trace_mode_get/set() and the accompanying enum
> rte_trace_mode should be changed to rte_trace_config_get/set() using a new
> struct rte_trace_config (containing the enum rte_trace_mode, and expandable
> with new fields as the need arises). This will prepare for e.g. tracing to
> other destinations than system memory, such as a remote trace collector on the
> network, like SYSLOG.
I'm also retracting this precondition...
If the need for further trace configuration should ever arise, rte_trace_config_get/set() can be added later.
And rte_trace_mode_get/set(), if not marked as experimental anymore, will be kept for backwards compatibility.
> >
> > > Patches welcome if you want it stripped.
> >
> > Don't have time myself, so I suggested it as a code challenge instead. :-)
> >
^ permalink raw reply [relevance 0%]
* Re: [PATCH 0/3] eal: mark API's as stable
@ 2024-09-05 14:01 3% ` Jerin Jacob
2024-09-05 14:18 0% ` Morten Brørup
2024-09-06 8:54 0% ` Ferruh Yigit
0 siblings, 2 replies; 200+ results
From: Jerin Jacob @ 2024-09-05 14:01 UTC (permalink / raw)
To: Morten Brørup
Cc: David Marchand, Jerin Jacob Kollanukkaran, Stephen Hemminger,
bruce.richardson, Sunil Kumar Kori, dev, Thomas Monjalon
On Thu, Sep 5, 2024 at 3:14 PM Morten Brørup <mb@smartsharesystems.com> wrote:
>
> > From: David Marchand [mailto:david.marchand@redhat.com]
> > Sent: Thursday, 5 September 2024 11.03
> >
> > On Thu, Sep 5, 2024 at 10:55 AM Morten Brørup <mb@smartsharesystems.com>
> > wrote:
> > >
> > > > From: David Marchand [mailto:david.marchand@redhat.com]
> > > > Sent: Thursday, 5 September 2024 09.59
> > > >
> > > > On Wed, Sep 4, 2024 at 8:10 PM Stephen Hemminger
> > > > <stephen@networkplumber.org> wrote:
> > > > >
> > > > > The API's in ethtool from before 23.11 should be marked stable.
> > > >
> > > > EAL* ?
> > > >
> > > > > Should probably include the trace api's but that is more complex change.
> > > >
> > > > On the trace API itself it should be ok.
> > >
> > > No!
> >
> > *sigh*
> >
> > >
> > > Trace must remain experimental until controlled by a meson option, e.g.
> > "enable_trace", whereby trace can be completely disabled and omitted from the
> > compiled application/libraries/drivers at build time.
> >
> > This seems unrelated to marking the API stable as regardless of the
> > API state at the moment, this code is always present.
>
> I cannot foresee if disabling trace at build time will require changes to the trace API. So I'm being cautious here.
>
> However, if Jerin (as author of the trace subsystem) foresees that it will be possible to disable trace at build time without affecting the trace API, I don't object to marking the trace API (or some of it) stable.
I don't for foresee any ABI changes when adding disabling trace
compile time support. However, I don't understand why we need to do
that. In the sense, fast path functions are already having an option
to compile out.
Slow path functions can be disabled at runtime at the cost of 1 cycle
as instrumentation cost. Having said that, I don't have any concern
about disabling trace as an option.
>
> Before doing that, rte_trace_mode_get/set() and the accompanying enum rte_trace_mode should be changed to rte_trace_config_get/set() using a new struct rte_trace_config (containing the enum rte_trace_mode, and expandable with new fields as the need arises). This will prepare for e.g. tracing to other destinations than system memory, such as a remote trace collector on the network, like SYSLOG.
>
> > Patches welcome if you want it stripped.
>
> Don't have time myself, so I suggested it as a code challenge instead. :-)
>
^ permalink raw reply [relevance 3%]
* [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs
@ 2024-09-05 10:14 4% Akhil Goyal
2024-09-05 15:09 4% ` Morten Brørup
` (2 more replies)
0 siblings, 3 replies; 200+ results
From: Akhil Goyal @ 2024-09-05 10:14 UTC (permalink / raw)
To: dev
Cc: thomas, david.marchand, hemant.agrawal, anoobj,
pablo.de.lara.guarch, fiona.trahe, declan.doherty, matan,
g.singh, fanzhang.oss, jianjay.zhou, asomalap, ruifeng.wang,
konstantin.v.ananyev, radu.nicolau, ajit.khaparde, rnagadheeraj,
mdr, Akhil Goyal
Replace *_LIST_END enumerators from asymmetric crypto
lib to avoid ABI breakage for every new addition in
enums with inline APIs.
Signed-off-by: Akhil Goyal <gakhil@marvell.com>
---
This patch was discussed in ML long time back.
https://patches.dpdk.org/project/dpdk/patch/20211008204516.3497060-1-gakhil@marvell.com/
Now added inline APIs for getting the list end which need to be updated
for each new entry to the enum. This shall help in avoiding ABI break
for adding new algo.
app/test/test_cryptodev_asym.c | 2 +-
lib/cryptodev/rte_crypto_asym.h | 18 ++++++++++++++----
2 files changed, 15 insertions(+), 5 deletions(-)
diff --git a/app/test/test_cryptodev_asym.c b/app/test/test_cryptodev_asym.c
index f0b5d38543..f1ece475b8 100644
--- a/app/test/test_cryptodev_asym.c
+++ b/app/test/test_cryptodev_asym.c
@@ -581,7 +581,7 @@ static inline void print_asym_capa(
rte_cryptodev_asym_get_xform_string(capa->xform_type));
printf("operation supported -");
- for (i = 0; i < RTE_CRYPTO_ASYM_OP_LIST_END; i++) {
+ for (i = 0; i < rte_crypto_asym_op_list_end(); i++) {
/* check supported operations */
if (rte_cryptodev_asym_xform_capability_check_optype(capa, i)) {
if (capa->xform_type == RTE_CRYPTO_ASYM_XFORM_DH)
diff --git a/lib/cryptodev/rte_crypto_asym.h b/lib/cryptodev/rte_crypto_asym.h
index 39d3da3952..290b300f84 100644
--- a/lib/cryptodev/rte_crypto_asym.h
+++ b/lib/cryptodev/rte_crypto_asym.h
@@ -72,6 +72,7 @@ enum rte_crypto_curve_id {
* Asymmetric crypto transformation types.
* Each xform type maps to one asymmetric algorithm
* performing specific operation
+ * Note: Update rte_crypto_asym_xform_type_list_end() for every new type added.
*/
enum rte_crypto_asym_xform_type {
RTE_CRYPTO_ASYM_XFORM_UNSPECIFIED = 0,
@@ -119,12 +120,17 @@ enum rte_crypto_asym_xform_type {
* Performs Encrypt, Decrypt, Sign and Verify.
* Refer to rte_crypto_asym_op_type.
*/
- RTE_CRYPTO_ASYM_XFORM_TYPE_LIST_END
- /**< End of list */
};
+static inline int
+rte_crypto_asym_xform_type_list_end(void)
+{
+ return RTE_CRYPTO_ASYM_XFORM_SM2 + 1;
+}
+
/**
* Asymmetric crypto operation type variants
+ * Note: Update rte_crypto_asym_op_list_end for every new type added.
*/
enum rte_crypto_asym_op_type {
RTE_CRYPTO_ASYM_OP_ENCRYPT,
@@ -135,9 +141,14 @@ enum rte_crypto_asym_op_type {
/**< Signature Generation operation */
RTE_CRYPTO_ASYM_OP_VERIFY,
/**< Signature Verification operation */
- RTE_CRYPTO_ASYM_OP_LIST_END
};
+static inline int
+rte_crypto_asym_op_list_end(void)
+{
+ return RTE_CRYPTO_ASYM_OP_VERIFY + 1;
+}
+
/**
* Asymmetric crypto key exchange operation type
*/
@@ -168,7 +179,6 @@ enum rte_crypto_rsa_padding_type {
/**< RSA PKCS#1 OAEP padding scheme */
RTE_CRYPTO_RSA_PADDING_PSS,
/**< RSA PKCS#1 PSS padding scheme */
- RTE_CRYPTO_RSA_PADDING_TYPE_LIST_END
};
/**
--
2.25.1
^ permalink raw reply [relevance 4%]
* [RESEND v6 1/8] ethdev: support report register names and filter
@ 2024-09-05 6:26 4% ` Jie Hai
0 siblings, 0 replies; 200+ results
From: Jie Hai @ 2024-09-05 6:26 UTC (permalink / raw)
To: dev, thomas, ferruh.yigit, andrew.rybchenko
Cc: lihuisong, fengchengwen, haijie1
This patch adds "filter" and "names" fields to "rte_dev_reg_info"
structure. Names of registers in data fields can be reported and
the registers can be filtered by their module names.
The new API rte_eth_dev_get_reg_info_ext() is added to support
reporting names and filtering by modules. And the original API
rte_eth_dev_get_reg_info() does not use the names and filter fields.
A local variable is used in rte_eth_dev_get_reg_info for
compatibility. If the drivers does not report the names, set them
to "index_XXX", which means the location in the register table.
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Huisong Li <lihuisong@huawei.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>
---
doc/guides/rel_notes/release_24_11.rst | 7 +++++
lib/ethdev/ethdev_trace.h | 2 ++
lib/ethdev/rte_dev_info.h | 11 ++++++++
lib/ethdev/rte_ethdev.c | 38 ++++++++++++++++++++++++++
lib/ethdev/rte_ethdev.h | 29 ++++++++++++++++++++
lib/ethdev/version.map | 3 ++
6 files changed, 90 insertions(+)
mode change 100644 => 100755 doc/guides/rel_notes/release_24_11.rst
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
old mode 100644
new mode 100755
index 0ff70d9057..acec1c60a6
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -54,6 +54,11 @@ New Features
This section is a comment. Do not overwrite or remove it.
Also, make sure to start the actual text at the margin.
=======================================================
+* **Added support for dumping registers with names and filtering by modules.**
+
+ * Added new API functions ``rte_eth_dev_get_reg_info_ext()`` to filter the
+ registers by module names and get the information (names, values and other
+ attributes) of the filtered registers.
Removed Items
@@ -100,6 +105,8 @@ ABI Changes
Also, make sure to start the actual text at the margin.
=======================================================
+* ethdev: Added ``filter`` and ``names`` fields to ``rte_dev_reg_info``
+ structure for filtering by modules and reporting names of registers.
Known Issues
------------
diff --git a/lib/ethdev/ethdev_trace.h b/lib/ethdev/ethdev_trace.h
index 3bec87bfdb..0c4780a09e 100644
--- a/lib/ethdev/ethdev_trace.h
+++ b/lib/ethdev/ethdev_trace.h
@@ -1152,6 +1152,8 @@ RTE_TRACE_POINT(
rte_trace_point_emit_u32(info->length);
rte_trace_point_emit_u32(info->width);
rte_trace_point_emit_u32(info->version);
+ rte_trace_point_emit_ptr(info->names);
+ rte_trace_point_emit_ptr(info->filter);
rte_trace_point_emit_int(ret);
)
diff --git a/lib/ethdev/rte_dev_info.h b/lib/ethdev/rte_dev_info.h
index 67cf0ae526..26b777f983 100644
--- a/lib/ethdev/rte_dev_info.h
+++ b/lib/ethdev/rte_dev_info.h
@@ -11,6 +11,11 @@ extern "C" {
#include <stdint.h>
+#define RTE_ETH_REG_NAME_SIZE 64
+struct rte_eth_reg_name {
+ char name[RTE_ETH_REG_NAME_SIZE];
+};
+
/*
* Placeholder for accessing device registers
*/
@@ -20,6 +25,12 @@ struct rte_dev_reg_info {
uint32_t length; /**< Number of registers to fetch */
uint32_t width; /**< Size of device register */
uint32_t version; /**< Device version */
+ /**
+ * Name of target module, filter for target subset of registers.
+ * This field could affects register selection for data/length/names.
+ */
+ const char *filter;
+ struct rte_eth_reg_name *names; /**< Registers name saver */
};
/*
diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
index f1c658f49e..30ca4a0043 100644
--- a/lib/ethdev/rte_ethdev.c
+++ b/lib/ethdev/rte_ethdev.c
@@ -6388,8 +6388,37 @@ rte_eth_read_clock(uint16_t port_id, uint64_t *clock)
int
rte_eth_dev_get_reg_info(uint16_t port_id, struct rte_dev_reg_info *info)
+{
+ struct rte_dev_reg_info reg_info = { 0 };
+ int ret;
+
+ if (info == NULL) {
+ RTE_ETHDEV_LOG_LINE(ERR,
+ "Cannot get ethdev port %u register info to NULL",
+ port_id);
+ return -EINVAL;
+ }
+
+ reg_info.length = info->length;
+ reg_info.data = info->data;
+
+ ret = rte_eth_dev_get_reg_info_ext(port_id, ®_info);
+ if (ret != 0)
+ return ret;
+
+ info->length = reg_info.length;
+ info->width = reg_info.width;
+ info->version = reg_info.version;
+ info->offset = reg_info.offset;
+
+ return 0;
+}
+
+int
+rte_eth_dev_get_reg_info_ext(uint16_t port_id, struct rte_dev_reg_info *info)
{
struct rte_eth_dev *dev;
+ uint32_t i;
int ret;
RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
@@ -6402,12 +6431,21 @@ rte_eth_dev_get_reg_info(uint16_t port_id, struct rte_dev_reg_info *info)
return -EINVAL;
}
+ if (info->names != NULL && info->length != 0)
+ memset(info->names, 0, sizeof(struct rte_eth_reg_name) * info->length);
+
if (*dev->dev_ops->get_reg == NULL)
return -ENOTSUP;
ret = eth_err(port_id, (*dev->dev_ops->get_reg)(dev, info));
rte_ethdev_trace_get_reg_info(port_id, info, ret);
+ /* Report the default names if drivers not report. */
+ if (ret == 0 && info->names != NULL && strlen(info->names[0].name) == 0) {
+ for (i = 0; i < info->length; i++)
+ snprintf(info->names[i].name, RTE_ETH_REG_NAME_SIZE,
+ "index_%u", info->offset + i);
+ }
return ret;
}
diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
index 548fada1c7..02cb3c07f7 100644
--- a/lib/ethdev/rte_ethdev.h
+++ b/lib/ethdev/rte_ethdev.h
@@ -5071,6 +5071,35 @@ __rte_experimental
int rte_eth_get_monitor_addr(uint16_t port_id, uint16_t queue_id,
struct rte_power_monitor_cond *pmc);
+/**
+ * Retrieve the filtered device registers (values and names) and
+ * register attributes (number of registers and register size)
+ *
+ * @param port_id
+ * The port identifier of the Ethernet device.
+ * @param info
+ * Pointer to rte_dev_reg_info structure to fill in.
+ * - If info->filter is NULL, return info for all registers (seen as filter
+ * none).
+ * - If info->filter is not NULL, return error if the driver does not support
+ * filter. Fill the length field with filtered register number.
+ * - If info->data is NULL, the function fills in the width and length fields.
+ * - If info->data is not NULL, ethdev considers there are enough spaces to
+ * store the registers, and the values of registers with the filter string
+ * as the module name are put into the buffer pointed at by info->data.
+ * - If info->names is not NULL, drivers should fill it or the ethdev fills it
+ * with default names.
+ * @return
+ * - (0) if successful.
+ * - (-ENOTSUP) if hardware doesn't support.
+ * - (-EINVAL) if bad parameter.
+ * - (-ENODEV) if *port_id* invalid.
+ * - (-EIO) if device is removed.
+ * - others depends on the specific operations implementation.
+ */
+__rte_experimental
+int rte_eth_dev_get_reg_info_ext(uint16_t port_id, struct rte_dev_reg_info *info);
+
/**
* Retrieve device registers and register attributes (number of registers and
* register size)
diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
index 1669055ca5..abc2ff5bb4 100644
--- a/lib/ethdev/version.map
+++ b/lib/ethdev/version.map
@@ -325,6 +325,9 @@ EXPERIMENTAL {
rte_flow_template_table_resizable;
rte_flow_template_table_resize;
rte_flow_template_table_resize_complete;
+
+ # added in 24.07
+ rte_eth_dev_get_reg_info_ext;
};
INTERNAL {
--
2.22.0
^ permalink raw reply [relevance 4%]
* [RFC PATCH] devtools/test-meson-builds: use cross file for 32bit build
@ 2024-09-04 14:03 13% Bruce Richardson
0 siblings, 0 replies; 200+ results
From: Bruce Richardson @ 2024-09-04 14:03 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
When testing the 32-bit x86 build on debian or ubuntu linux systems, use
the cross-file rather than using args and pkgconfig environment
variable. The advantage of using the cross-file is that the paths are
saved across runs. While the '-m32' args settings are preserved in the
current setup, the PKG_CONFIG_LIBDIR value from environment is not,
which can cause rebuilds of the build-32b directory to fail if meson
needs to do a reconfiguration first.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
devtools/test-meson-builds.sh | 15 +++++++++------
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh
index d71bb1ded0..1d9d04ce7c 100755
--- a/devtools/test-meson-builds.sh
+++ b/devtools/test-meson-builds.sh
@@ -253,21 +253,24 @@ build build-x86-generic cc skipABI -Dcheck_includes=true \
# 32-bit with default compiler
if check_cc_flags '-m32' ; then
+ target_override='i386-pc-linux-gnu'
if [ -d '/usr/lib/i386-linux-gnu' ] ; then
- # 32-bit pkgconfig on Debian/Ubuntu
- export PKG_CONFIG_LIBDIR='/usr/lib/i386-linux-gnu/pkgconfig'
+ # 32-bit pkgconfig on Debian/Ubuntu, use cross file
+ build build-32b $srcdir/config/x86/cross-debian-32bit ABI
elif [ -d '/usr/lib32' ] ; then
# 32-bit pkgconfig on Arch
export PKG_CONFIG_LIBDIR='/usr/lib32/pkgconfig'
+ build build-32b cc ABI -Dc_args='-m32' -Dc_link_args='-m32' \
+ -Dcpp_args='-m32' -Dcpp_link_args='-m32'
+ unset PKG_CONFIG_LIBDIR
else
# 32-bit pkgconfig on RHEL/Fedora (lib vs lib64)
export PKG_CONFIG_LIBDIR='/usr/lib/pkgconfig'
+ build build-32b cc ABI -Dc_args='-m32' -Dc_link_args='-m32' \
+ -Dcpp_args='-m32' -Dcpp_link_args='-m32'
+ unset PKG_CONFIG_LIBDIR
fi
- target_override='i386-pc-linux-gnu'
- build build-32b cc ABI -Dc_args='-m32' -Dc_link_args='-m32' \
- -Dcpp_args='-m32' -Dcpp_link_args='-m32'
target_override=
- unset PKG_CONFIG_LIBDIR
fi
# x86 MinGW
--
2.43.0
^ permalink raw reply [relevance 13%]
* [PATCH 3/4] net/nfp: support different configuration BAR size
2024-09-03 1:41 2% ` [PATCH 2/4] net/nfp: refactor the firmware version logic Chaoyong He
@ 2024-09-03 1:41 2% ` Chaoyong He
1 sibling, 0 replies; 200+ results
From: Chaoyong He @ 2024-09-03 1:41 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Chaoyong He, Long Wu
A new NIC is introduced with different configuration
BAR size other than 32K, and this is distinguished by
the application firmware's class version ABI.
Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Long Wu <long.wu@corigine.com>
---
drivers/common/nfp/nfp_common_ctrl.h | 10 +++++++++-
drivers/net/nfp/flower/nfp_flower.c | 4 ++--
drivers/net/nfp/nfp_ethdev.c | 24 ++++++++++++++----------
drivers/net/nfp/nfp_ethdev_vf.c | 3 +++
drivers/net/nfp/nfp_net_common.c | 25 +++++++++++++++++++++++++
drivers/net/nfp/nfp_net_common.h | 3 +++
drivers/net/nfp/nfp_net_ctrl.c | 4 +++-
7 files changed, 59 insertions(+), 14 deletions(-)
diff --git a/drivers/common/nfp/nfp_common_ctrl.h b/drivers/common/nfp/nfp_common_ctrl.h
index 69596dd6f5..711577dc76 100644
--- a/drivers/common/nfp/nfp_common_ctrl.h
+++ b/drivers/common/nfp/nfp_common_ctrl.h
@@ -11,7 +11,9 @@
*
* On the NFP6000, due to THB-350, the configuration BAR is 32K in size.
*/
-#define NFP_NET_CFG_BAR_SZ (32 * 1024)
+#define NFP_NET_CFG_BAR_SZ_32K (32 * 1024)
+#define NFP_NET_CFG_BAR_SZ_8K (8 * 1024)
+#define NFP_NET_CFG_BAR_SZ_MIN NFP_NET_CFG_BAR_SZ_8K
/*
* Configuration sriov VF.
@@ -121,6 +123,10 @@
struct nfp_net_fw_ver {
uint8_t minor;
uint8_t major;
+ /**
+ * BIT0: class, refer NFP_NET_CFG_VERSION_CLASS_*
+ * BIT[7:1]: reserved
+ */
uint8_t class;
/**
* This byte can be extended for more use.
@@ -147,6 +153,8 @@ struct nfp_net_fw_ver {
#define NFP_NET_CFG_VERSION 0x0030
#define NFP_NET_CFG_VERSION_DP_NFD3 0
#define NFP_NET_CFG_VERSION_DP_NFDK 1
+#define NFP_NET_CFG_VERSION_CLASS_GENERIC 0
+#define NFP_NET_CFG_VERSION_CLASS_NO_EMEM 1
#define NFP_NET_CFG_STS 0x0034
#define NFP_NET_CFG_STS_LINK (0x1 << 0) /* Link up or down */
/* Link rate */
diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c
index 4d91d548f7..c1a3532c11 100644
--- a/drivers/net/nfp/flower/nfp_flower.c
+++ b/drivers/net/nfp/flower/nfp_flower.c
@@ -692,7 +692,7 @@ nfp_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv)
/* Map the PF ctrl bar */
snprintf(bar_name, sizeof(bar_name), "_pf%u_net_bar0", id);
pf_dev->ctrl_bar = nfp_rtsym_map(pf_dev->sym_tbl, bar_name,
- NFP_NET_CFG_BAR_SZ, &pf_dev->ctrl_area);
+ pf_dev->ctrl_bar_size, &pf_dev->ctrl_area);
if (pf_dev->ctrl_bar == NULL) {
PMD_INIT_LOG(ERR, "Cloud not map the PF vNIC ctrl bar");
ret = -ENODEV;
@@ -737,7 +737,7 @@ nfp_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv)
/* Map the ctrl vNIC ctrl bar */
snprintf(ctrl_name, sizeof(ctrl_name), "_pf%u_net_ctrl_bar", id);
ctrl_hw->super.ctrl_bar = nfp_rtsym_map(pf_dev->sym_tbl, ctrl_name,
- NFP_NET_CFG_BAR_SZ, &ctrl_hw->ctrl_area);
+ pf_dev->ctrl_bar_size, &ctrl_hw->ctrl_area);
if (ctrl_hw->super.ctrl_bar == NULL) {
PMD_INIT_LOG(ERR, "Cloud not map the ctrl vNIC ctrl bar");
ret = -ENODEV;
diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c
index d85993f70c..a09bbe52ca 100644
--- a/drivers/net/nfp/nfp_ethdev.c
+++ b/drivers/net/nfp/nfp_ethdev.c
@@ -1022,7 +1022,7 @@ nfp_net_init(struct rte_eth_dev *eth_dev,
if (pf_dev->multi_pf.enabled)
hw->ctrl_bar = pf_dev->ctrl_bar;
else
- hw->ctrl_bar = pf_dev->ctrl_bar + (port * NFP_NET_CFG_BAR_SZ);
+ hw->ctrl_bar = pf_dev->ctrl_bar + (port * pf_dev->ctrl_bar_size);
net_hw->mac_stats = pf_dev->mac_stats_bar +
(net_hw->nfp_idx * NFP_MAC_STATS_SIZE);
@@ -1555,9 +1555,10 @@ nfp_enable_multi_pf(struct nfp_pf_dev *pf_dev)
memset(&net_hw, 0, sizeof(struct nfp_net_hw));
/* Map the symbol table */
+ pf_dev->ctrl_bar_size = NFP_NET_CFG_BAR_SZ_MIN;
snprintf(name, sizeof(name), "_pf%u_net_bar0",
pf_dev->multi_pf.function_id);
- ctrl_bar = nfp_rtsym_map(pf_dev->sym_tbl, name, NFP_NET_CFG_BAR_SZ,
+ ctrl_bar = nfp_rtsym_map(pf_dev->sym_tbl, name, pf_dev->ctrl_bar_size,
&area);
if (ctrl_bar == NULL) {
PMD_INIT_LOG(ERR, "Failed to find data vNIC memory symbol");
@@ -1574,6 +1575,9 @@ nfp_enable_multi_pf(struct nfp_pf_dev *pf_dev)
goto end;
}
+ /* Set the ctrl bar size */
+ nfp_net_ctrl_bar_size_set(pf_dev);
+
if (!pf_dev->multi_pf.enabled)
goto end;
@@ -1670,7 +1674,7 @@ nfp_init_app_fw_nic(struct nfp_net_hw_priv *hw_priv)
/* Map the symbol table */
snprintf(bar_name, sizeof(bar_name), "_pf%u_net_bar0", id);
pf_dev->ctrl_bar = nfp_rtsym_map(pf_dev->sym_tbl, bar_name,
- pf_dev->total_phyports * NFP_NET_CFG_BAR_SZ,
+ pf_dev->total_phyports * pf_dev->ctrl_bar_size,
&pf_dev->ctrl_area);
if (pf_dev->ctrl_bar == NULL) {
PMD_INIT_LOG(ERR, "nfp_rtsym_map fails for %s", bar_name);
@@ -2065,11 +2069,11 @@ nfp_net_vf_config_init(struct nfp_pf_dev *pf_dev)
if (pf_dev->sriov_vf == 0)
return 0;
- min_size = NFP_NET_CFG_BAR_SZ * pf_dev->sriov_vf;
+ min_size = pf_dev->ctrl_bar_size * pf_dev->sriov_vf;
snprintf(vf_bar_name, sizeof(vf_bar_name), "_pf%d_net_vf_bar",
pf_dev->multi_pf.function_id);
pf_dev->vf_bar = nfp_rtsym_map_offset(pf_dev->sym_tbl, vf_bar_name,
- NFP_NET_CFG_BAR_SZ * pf_dev->vf_base_id,
+ pf_dev->ctrl_bar_size * pf_dev->vf_base_id,
min_size, &pf_dev->vf_area);
if (pf_dev->vf_bar == NULL) {
PMD_INIT_LOG(ERR, "Failed to get vf cfg.");
@@ -2295,15 +2299,15 @@ nfp_pf_init(struct rte_pci_device *pci_dev)
goto hwqueues_cleanup;
}
+ ret = nfp_enable_multi_pf(pf_dev);
+ if (ret != 0)
+ goto mac_stats_cleanup;
+
ret = nfp_net_vf_config_init(pf_dev);
if (ret != 0) {
PMD_INIT_LOG(ERR, "Failed to init VF config.");
- goto mac_stats_cleanup;
- }
-
- ret = nfp_enable_multi_pf(pf_dev);
- if (ret != 0)
goto vf_cfg_tbl_cleanup;
+ }
hw_priv->is_pf = true;
hw_priv->pf_dev = pf_dev;
diff --git a/drivers/net/nfp/nfp_ethdev_vf.c b/drivers/net/nfp/nfp_ethdev_vf.c
index 2e581c7e45..ab413a2c5a 100644
--- a/drivers/net/nfp/nfp_ethdev_vf.c
+++ b/drivers/net/nfp/nfp_ethdev_vf.c
@@ -295,6 +295,9 @@ nfp_netvf_init(struct rte_eth_dev *eth_dev)
goto pf_dev_free;
}
+ /* Set the ctrl bar size */
+ nfp_net_ctrl_bar_size_set(pf_dev);
+
PMD_INIT_LOG(DEBUG, "ctrl bar: %p", hw->ctrl_bar);
err = nfp_net_common_init(pf_dev, net_hw);
diff --git a/drivers/net/nfp/nfp_net_common.c b/drivers/net/nfp/nfp_net_common.c
index e4e01d8c79..5f92c2c31d 100644
--- a/drivers/net/nfp/nfp_net_common.c
+++ b/drivers/net/nfp/nfp_net_common.c
@@ -2184,6 +2184,9 @@ nfp_net_version_check(struct nfp_hw *hw,
if (!nfp_net_is_valid_nfd_version(pf_dev->ver))
return false;
+ if (!nfp_net_is_valid_version_class(pf_dev->ver))
+ return false;
+
return true;
}
@@ -2325,6 +2328,28 @@ nfp_net_is_valid_nfd_version(struct nfp_net_fw_ver version)
return false;
}
+bool
+nfp_net_is_valid_version_class(struct nfp_net_fw_ver version)
+{
+ switch (version.class) {
+ case NFP_NET_CFG_VERSION_CLASS_GENERIC:
+ return true;
+ case NFP_NET_CFG_VERSION_CLASS_NO_EMEM:
+ return true;
+ default:
+ return false;
+ }
+}
+
+void
+nfp_net_ctrl_bar_size_set(struct nfp_pf_dev *pf_dev)
+{
+ if (pf_dev->ver.class == NFP_NET_CFG_VERSION_CLASS_GENERIC)
+ pf_dev->ctrl_bar_size = NFP_NET_CFG_BAR_SZ_32K;
+ else
+ pf_dev->ctrl_bar_size = NFP_NET_CFG_BAR_SZ_8K;
+}
+
/* Disable rx and tx functions to allow for reconfiguring. */
int
nfp_net_stop(struct rte_eth_dev *dev)
diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h
index 8d0922d48c..2c54815fc9 100644
--- a/drivers/net/nfp/nfp_net_common.h
+++ b/drivers/net/nfp/nfp_net_common.h
@@ -117,6 +117,7 @@ struct nfp_pf_dev {
struct nfp_eth_table *nfp_eth_table;
uint8_t *ctrl_bar;
+ uint32_t ctrl_bar_size;
struct nfp_cpp *cpp;
struct nfp_cpp_area *ctrl_area;
@@ -353,6 +354,7 @@ void nfp_net_tx_desc_limits(struct nfp_net_hw_priv *hw_priv,
int nfp_net_check_dma_mask(struct nfp_pf_dev *pf_dev, char *name);
int nfp_net_firmware_version_get(struct rte_eth_dev *dev, char *fw_version, size_t fw_size);
bool nfp_net_is_valid_nfd_version(struct nfp_net_fw_ver version);
+bool nfp_net_is_valid_version_class(struct nfp_net_fw_ver version);
struct nfp_net_hw *nfp_net_get_hw(const struct rte_eth_dev *dev);
int nfp_net_stop(struct rte_eth_dev *dev);
int nfp_net_flow_ctrl_get(struct rte_eth_dev *dev,
@@ -379,6 +381,7 @@ int nfp_net_vf_config_app_init(struct nfp_net_hw *net_hw,
struct nfp_pf_dev *pf_dev);
bool nfp_net_version_check(struct nfp_hw *hw,
struct nfp_pf_dev *pf_dev);
+void nfp_net_ctrl_bar_size_set(struct nfp_pf_dev *pf_dev);
#define NFP_PRIV_TO_APP_FW_NIC(app_fw_priv)\
((struct nfp_app_fw_nic *)app_fw_priv)
diff --git a/drivers/net/nfp/nfp_net_ctrl.c b/drivers/net/nfp/nfp_net_ctrl.c
index ea14b98924..b34d8f140f 100644
--- a/drivers/net/nfp/nfp_net_ctrl.c
+++ b/drivers/net/nfp/nfp_net_ctrl.c
@@ -30,13 +30,15 @@ nfp_net_tlv_caps_parse(struct rte_eth_dev *dev)
uint32_t tlv_type;
struct nfp_net_hw *net_hw;
struct nfp_net_tlv_caps *caps;
+ struct nfp_net_hw_priv *hw_priv;
net_hw = dev->data->dev_private;
+ hw_priv = dev->process_private;
caps = &net_hw->tlv_caps;
nfp_net_tlv_caps_reset(caps);
data = net_hw->super.ctrl_bar + NFP_NET_CFG_TLV_BASE;
- end = net_hw->super.ctrl_bar + NFP_NET_CFG_BAR_SZ;
+ end = net_hw->super.ctrl_bar + hw_priv->pf_dev->ctrl_bar_size;
hdr = rte_read32(data);
if (hdr == 0) {
--
2.39.1
^ permalink raw reply [relevance 2%]
* [PATCH 2/4] net/nfp: refactor the firmware version logic
@ 2024-09-03 1:41 2% ` Chaoyong He
2024-09-03 1:41 2% ` [PATCH 3/4] net/nfp: support different configuration BAR size Chaoyong He
1 sibling, 0 replies; 200+ results
From: Chaoyong He @ 2024-09-03 1:41 UTC (permalink / raw)
To: dev; +Cc: oss-drivers, Chaoyong He, Long Wu
Move the 'ver' data field from 'struct nfp_net_hw' into
'struct nfp_pf_dev', also modify the related logic.
Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Long Wu <long.wu@corigine.com>
---
drivers/net/nfp/flower/nfp_flower.c | 30 ++++++-------
drivers/net/nfp/flower/nfp_flower_ctrl.c | 8 ++--
drivers/net/nfp/flower/nfp_flower_ctrl.h | 2 +-
drivers/net/nfp/nfd3/nfp_nfd3_dp.c | 2 +-
drivers/net/nfp/nfdk/nfp_nfdk_dp.c | 2 +-
drivers/net/nfp/nfp_ethdev.c | 29 +++++++-----
drivers/net/nfp/nfp_ethdev_vf.c | 30 ++++++++++---
drivers/net/nfp/nfp_net_common.c | 56 ++++++++++++++----------
drivers/net/nfp/nfp_net_common.h | 16 ++++---
drivers/net/nfp/nfp_net_meta.c | 5 ++-
drivers/net/nfp/nfp_net_meta.h | 5 ++-
drivers/net/nfp/nfp_rxtx.c | 10 ++---
12 files changed, 118 insertions(+), 77 deletions(-)
diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c
index 8d781658ea..4d91d548f7 100644
--- a/drivers/net/nfp/flower/nfp_flower.c
+++ b/drivers/net/nfp/flower/nfp_flower.c
@@ -166,15 +166,15 @@ nfp_flower_pf_nfdk_xmit_pkts(void *tx_queue,
}
static void
-nfp_flower_pf_xmit_pkts_register(struct nfp_app_fw_flower *app_fw_flower)
+nfp_flower_pf_xmit_pkts_register(struct nfp_pf_dev *pf_dev)
{
- struct nfp_net_hw *hw;
struct nfp_flower_nfd_func *nfd_func;
+ struct nfp_app_fw_flower *app_fw_flower;
- hw = app_fw_flower->pf_hw;
+ app_fw_flower = pf_dev->app_fw_priv;
nfd_func = &app_fw_flower->nfd_func;
- if (hw->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3)
+ if (pf_dev->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3)
nfd_func->pf_xmit_t = nfp_flower_pf_nfd3_xmit_pkts;
else
nfd_func->pf_xmit_t = nfp_flower_pf_nfdk_xmit_pkts;
@@ -204,14 +204,12 @@ nfp_flower_init_vnic_common(struct nfp_net_hw_priv *hw_priv,
uint64_t rx_bar_off;
uint64_t tx_bar_off;
struct nfp_pf_dev *pf_dev;
- struct rte_pci_device *pci_dev;
pf_dev = hw_priv->pf_dev;
- pci_dev = pf_dev->pci_dev;
PMD_INIT_LOG(DEBUG, "%s vNIC ctrl bar: %p", vnic_type, hw->super.ctrl_bar);
- err = nfp_net_common_init(pci_dev, hw);
+ err = nfp_net_common_init(pf_dev, hw);
if (err != 0)
return err;
@@ -612,15 +610,15 @@ nfp_flower_start_ctrl_vnic(struct nfp_app_fw_flower *app_fw_flower)
}
static void
-nfp_flower_pkt_add_metadata_register(struct nfp_app_fw_flower *app_fw_flower)
+nfp_flower_pkt_add_metadata_register(struct nfp_pf_dev *pf_dev)
{
- struct nfp_net_hw *hw;
struct nfp_flower_nfd_func *nfd_func;
+ struct nfp_app_fw_flower *app_fw_flower;
- hw = app_fw_flower->pf_hw;
+ app_fw_flower = pf_dev->app_fw_priv;
nfd_func = &app_fw_flower->nfd_func;
- if (hw->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3)
+ if (pf_dev->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3)
nfd_func->pkt_add_metadata_t = nfp_flower_nfd3_pkt_add_metadata;
else
nfd_func->pkt_add_metadata_t = nfp_flower_nfdk_pkt_add_metadata;
@@ -635,11 +633,11 @@ nfp_flower_pkt_add_metadata(struct nfp_app_fw_flower *app_fw_flower,
}
static void
-nfp_flower_nfd_func_register(struct nfp_app_fw_flower *app_fw_flower)
+nfp_flower_nfd_func_register(struct nfp_pf_dev *pf_dev)
{
- nfp_flower_pkt_add_metadata_register(app_fw_flower);
- nfp_flower_ctrl_vnic_xmit_register(app_fw_flower);
- nfp_flower_pf_xmit_pkts_register(app_fw_flower);
+ nfp_flower_pkt_add_metadata_register(pf_dev);
+ nfp_flower_ctrl_vnic_xmit_register(pf_dev);
+ nfp_flower_pf_xmit_pkts_register(pf_dev);
}
int
@@ -730,7 +728,7 @@ nfp_init_app_fw_flower(struct nfp_net_hw_priv *hw_priv)
goto pf_cpp_area_cleanup;
}
- nfp_flower_nfd_func_register(app_fw_flower);
+ nfp_flower_nfd_func_register(pf_dev);
/* The ctrl vNIC struct comes directly after the PF one */
app_fw_flower->ctrl_hw = pf_hw + 1;
diff --git a/drivers/net/nfp/flower/nfp_flower_ctrl.c b/drivers/net/nfp/flower/nfp_flower_ctrl.c
index a46b849d1b..9b957e1f1e 100644
--- a/drivers/net/nfp/flower/nfp_flower_ctrl.c
+++ b/drivers/net/nfp/flower/nfp_flower_ctrl.c
@@ -343,15 +343,15 @@ nfp_flower_ctrl_vnic_nfdk_xmit(struct nfp_app_fw_flower *app_fw_flower,
}
void
-nfp_flower_ctrl_vnic_xmit_register(struct nfp_app_fw_flower *app_fw_flower)
+nfp_flower_ctrl_vnic_xmit_register(struct nfp_pf_dev *pf_dev)
{
- struct nfp_net_hw *hw;
struct nfp_flower_nfd_func *nfd_func;
+ struct nfp_app_fw_flower *app_fw_flower;
- hw = app_fw_flower->pf_hw;
+ app_fw_flower = pf_dev->app_fw_priv;
nfd_func = &app_fw_flower->nfd_func;
- if (hw->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3)
+ if (pf_dev->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3)
nfd_func->ctrl_vnic_xmit_t = nfp_flower_ctrl_vnic_nfd3_xmit;
else
nfd_func->ctrl_vnic_xmit_t = nfp_flower_ctrl_vnic_nfdk_xmit;
diff --git a/drivers/net/nfp/flower/nfp_flower_ctrl.h b/drivers/net/nfp/flower/nfp_flower_ctrl.h
index b5d0036c01..f18c8f4095 100644
--- a/drivers/net/nfp/flower/nfp_flower_ctrl.h
+++ b/drivers/net/nfp/flower/nfp_flower_ctrl.h
@@ -11,6 +11,6 @@
void nfp_flower_ctrl_vnic_process(struct nfp_net_hw_priv *hw_priv);
uint16_t nfp_flower_ctrl_vnic_xmit(struct nfp_app_fw_flower *app_fw_flower,
struct rte_mbuf *mbuf);
-void nfp_flower_ctrl_vnic_xmit_register(struct nfp_app_fw_flower *app_fw_flower);
+void nfp_flower_ctrl_vnic_xmit_register(struct nfp_pf_dev *pf_dev);
#endif /* __NFP_FLOWER_CTRL_H__ */
diff --git a/drivers/net/nfp/nfd3/nfp_nfd3_dp.c b/drivers/net/nfp/nfd3/nfp_nfd3_dp.c
index ee96cd8e46..4ff1ae63b0 100644
--- a/drivers/net/nfp/nfd3/nfp_nfd3_dp.c
+++ b/drivers/net/nfp/nfd3/nfp_nfd3_dp.c
@@ -390,7 +390,7 @@ nfp_net_nfd3_tx_queue_setup(struct rte_eth_dev *dev,
hw = nfp_net_get_hw(dev);
hw_priv = dev->process_private;
- nfp_net_tx_desc_limits(hw, hw_priv, &min_tx_desc, &max_tx_desc);
+ nfp_net_tx_desc_limits(hw_priv, &min_tx_desc, &max_tx_desc);
/* Validating number of descriptors */
tx_desc_sz = nb_desc * sizeof(struct nfp_net_nfd3_tx_desc);
diff --git a/drivers/net/nfp/nfdk/nfp_nfdk_dp.c b/drivers/net/nfp/nfdk/nfp_nfdk_dp.c
index 2cea5688b3..68fcbe93da 100644
--- a/drivers/net/nfp/nfdk/nfp_nfdk_dp.c
+++ b/drivers/net/nfp/nfdk/nfp_nfdk_dp.c
@@ -424,7 +424,7 @@ nfp_net_nfdk_tx_queue_setup(struct rte_eth_dev *dev,
hw = nfp_net_get_hw(dev);
hw_priv = dev->process_private;
- nfp_net_tx_desc_limits(hw, hw_priv, &min_tx_desc, &max_tx_desc);
+ nfp_net_tx_desc_limits(hw_priv, &min_tx_desc, &max_tx_desc);
/* Validating number of descriptors */
tx_desc_sz = nb_desc * sizeof(struct nfp_net_nfdk_tx_desc);
diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c
index 181fd74efe..d85993f70c 100644
--- a/drivers/net/nfp/nfp_ethdev.c
+++ b/drivers/net/nfp/nfp_ethdev.c
@@ -959,10 +959,10 @@ static const struct eth_dev_ops nfp_net_eth_dev_ops = {
};
static inline void
-nfp_net_ethdev_ops_mount(struct nfp_net_hw *hw,
+nfp_net_ethdev_ops_mount(struct nfp_pf_dev *pf_dev,
struct rte_eth_dev *eth_dev)
{
- if (hw->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3)
+ if (pf_dev->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3)
eth_dev->tx_pkt_burst = nfp_net_nfd3_xmit_pkts;
else
nfp_net_nfdk_xmit_pkts_set(eth_dev);
@@ -1030,7 +1030,7 @@ nfp_net_init(struct rte_eth_dev *eth_dev,
PMD_INIT_LOG(DEBUG, "ctrl bar: %p", hw->ctrl_bar);
PMD_INIT_LOG(DEBUG, "MAC stats: %p", net_hw->mac_stats);
- err = nfp_net_common_init(pci_dev, net_hw);
+ err = nfp_net_common_init(pf_dev, net_hw);
if (err != 0)
return err;
@@ -1046,7 +1046,7 @@ nfp_net_init(struct rte_eth_dev *eth_dev,
return err;
}
- nfp_net_ethdev_ops_mount(net_hw, eth_dev);
+ nfp_net_ethdev_ops_mount(pf_dev, eth_dev);
net_hw->eth_xstats_base = rte_malloc("rte_eth_xstat", sizeof(struct rte_eth_xstat) *
nfp_net_xstats_size(eth_dev), 0);
@@ -1074,7 +1074,7 @@ nfp_net_init(struct rte_eth_dev *eth_dev,
if ((hw->cap & NFP_NET_CFG_CTRL_LSO2) != 0)
hw->cap &= ~NFP_NET_CFG_CTRL_TXVLAN;
- nfp_net_log_device_information(net_hw);
+ nfp_net_log_device_information(net_hw, pf_dev);
/* Initializing spinlock for reconfigs */
rte_spinlock_init(&hw->reconfig_lock);
@@ -1552,9 +1552,6 @@ nfp_enable_multi_pf(struct nfp_pf_dev *pf_dev)
struct nfp_cpp_area *area;
char name[RTE_ETH_NAME_MAX_LEN];
- if (!pf_dev->multi_pf.enabled)
- return 0;
-
memset(&net_hw, 0, sizeof(struct nfp_net_hw));
/* Map the symbol table */
@@ -1570,6 +1567,16 @@ nfp_enable_multi_pf(struct nfp_pf_dev *pf_dev)
hw = &net_hw.super;
hw->ctrl_bar = ctrl_bar;
+ /* Check the version from firmware */
+ if (!nfp_net_version_check(hw, pf_dev)) {
+ PMD_INIT_LOG(ERR, "Not the valid version.");
+ err = -EINVAL;
+ goto end;
+ }
+
+ if (!pf_dev->multi_pf.enabled)
+ goto end;
+
cap_extend = nn_cfg_readl(hw, NFP_NET_CFG_CAP_WORD1);
if ((cap_extend & NFP_NET_CFG_CTRL_MULTI_PF) == 0) {
PMD_INIT_LOG(ERR, "Loaded firmware doesn't support multiple PF");
@@ -2358,10 +2365,10 @@ static int
nfp_secondary_net_init(struct rte_eth_dev *eth_dev,
void *para)
{
- struct nfp_net_hw *net_hw;
+ struct nfp_net_hw_priv *hw_priv;
- net_hw = eth_dev->data->dev_private;
- nfp_net_ethdev_ops_mount(net_hw, eth_dev);
+ hw_priv = para;
+ nfp_net_ethdev_ops_mount(hw_priv->pf_dev, eth_dev);
eth_dev->process_private = para;
diff --git a/drivers/net/nfp/nfp_ethdev_vf.c b/drivers/net/nfp/nfp_ethdev_vf.c
index cdf5da3af7..2e581c7e45 100644
--- a/drivers/net/nfp/nfp_ethdev_vf.c
+++ b/drivers/net/nfp/nfp_ethdev_vf.c
@@ -235,10 +235,10 @@ static const struct eth_dev_ops nfp_netvf_eth_dev_ops = {
};
static inline void
-nfp_netvf_ethdev_ops_mount(struct nfp_net_hw *hw,
+nfp_netvf_ethdev_ops_mount(struct nfp_pf_dev *pf_dev,
struct rte_eth_dev *eth_dev)
{
- if (hw->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3)
+ if (pf_dev->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3)
eth_dev->tx_pkt_burst = nfp_net_nfd3_xmit_pkts;
else
nfp_net_nfdk_xmit_pkts_set(eth_dev);
@@ -256,6 +256,7 @@ nfp_netvf_init(struct rte_eth_dev *eth_dev)
uint32_t start_q;
struct nfp_hw *hw;
struct nfp_net_hw *net_hw;
+ struct nfp_pf_dev *pf_dev;
uint64_t tx_bar_off = 0;
uint64_t rx_bar_off = 0;
struct rte_pci_device *pci_dev;
@@ -280,13 +281,27 @@ nfp_netvf_init(struct rte_eth_dev *eth_dev)
return -ENODEV;
}
+ pf_dev = rte_zmalloc(NULL, sizeof(*pf_dev), 0);
+ if (pf_dev == NULL) {
+ PMD_INIT_LOG(ERR, "Can not allocate memory for the PF device.");
+ return -ENOMEM;
+ }
+
+ pf_dev->pci_dev = pci_dev;
+
+ /* Check the version from firmware */
+ if (!nfp_net_version_check(hw, pf_dev)) {
+ err = -EINVAL;
+ goto pf_dev_free;
+ }
+
PMD_INIT_LOG(DEBUG, "ctrl bar: %p", hw->ctrl_bar);
- err = nfp_net_common_init(pci_dev, net_hw);
+ err = nfp_net_common_init(pf_dev, net_hw);
if (err != 0)
- return err;
+ goto pf_dev_free;
- nfp_netvf_ethdev_ops_mount(net_hw, eth_dev);
+ nfp_netvf_ethdev_ops_mount(pf_dev, eth_dev);
hw_priv = rte_zmalloc(NULL, sizeof(*hw_priv), 0);
if (hw_priv == NULL) {
@@ -296,6 +311,7 @@ nfp_netvf_init(struct rte_eth_dev *eth_dev)
}
hw_priv->dev_info = dev_info;
+ hw_priv->pf_dev = pf_dev;
eth_dev->process_private = hw_priv;
@@ -330,7 +346,7 @@ nfp_netvf_init(struct rte_eth_dev *eth_dev)
if ((hw->cap & NFP_NET_CFG_CTRL_LSO2) != 0)
hw->cap &= ~NFP_NET_CFG_CTRL_TXVLAN;
- nfp_net_log_device_information(net_hw);
+ nfp_net_log_device_information(net_hw, pf_dev);
/* Initializing spinlock for reconfigs */
rte_spinlock_init(&hw->reconfig_lock);
@@ -381,6 +397,8 @@ nfp_netvf_init(struct rte_eth_dev *eth_dev)
rte_free(net_hw->eth_xstats_base);
hw_priv_free:
rte_free(hw_priv);
+pf_dev_free:
+ rte_free(pf_dev);
return err;
}
diff --git a/drivers/net/nfp/nfp_net_common.c b/drivers/net/nfp/nfp_net_common.c
index b471fd032a..e4e01d8c79 100644
--- a/drivers/net/nfp/nfp_net_common.c
+++ b/drivers/net/nfp/nfp_net_common.c
@@ -349,13 +349,14 @@ nfp_net_configure(struct rte_eth_dev *dev)
}
void
-nfp_net_log_device_information(const struct nfp_net_hw *hw)
+nfp_net_log_device_information(const struct nfp_net_hw *hw,
+ struct nfp_pf_dev *pf_dev)
{
uint32_t cap = hw->super.cap;
uint32_t cap_ext = hw->super.cap_ext;
PMD_INIT_LOG(INFO, "VER: %u.%u, Maximum supported MTU: %d",
- hw->ver.major, hw->ver.minor, hw->max_mtu);
+ pf_dev->ver.major, pf_dev->ver.minor, hw->max_mtu);
PMD_INIT_LOG(INFO, "CAP: %#x", cap);
PMD_INIT_LOG(INFO, "%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s",
@@ -1235,14 +1236,13 @@ nfp_net_rx_desc_limits(struct nfp_net_hw_priv *hw_priv,
}
void
-nfp_net_tx_desc_limits(struct nfp_net_hw *hw,
- struct nfp_net_hw_priv *hw_priv,
+nfp_net_tx_desc_limits(struct nfp_net_hw_priv *hw_priv,
uint16_t *min_tx_desc,
uint16_t *max_tx_desc)
{
uint16_t tx_dpp;
- if (hw->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3)
+ if (hw_priv->pf_dev->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3)
tx_dpp = NFD3_TX_DESC_PER_PKT;
else
tx_dpp = NFDK_TX_DESC_PER_SIMPLE_PKT;
@@ -1269,7 +1269,7 @@ nfp_net_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
return -EINVAL;
nfp_net_rx_desc_limits(hw_priv, &min_rx_desc, &max_rx_desc);
- nfp_net_tx_desc_limits(hw, hw_priv, &min_tx_desc, &max_tx_desc);
+ nfp_net_tx_desc_limits(hw_priv, &min_tx_desc, &max_tx_desc);
dev_info->max_rx_queues = (uint16_t)hw->max_rx_queues;
dev_info->max_tx_queues = (uint16_t)hw->max_tx_queues;
@@ -1373,11 +1373,13 @@ nfp_net_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
}
int
-nfp_net_common_init(struct rte_pci_device *pci_dev,
+nfp_net_common_init(struct nfp_pf_dev *pf_dev,
struct nfp_net_hw *hw)
{
const int stride = 4;
+ struct rte_pci_device *pci_dev;
+ pci_dev = pf_dev->pci_dev;
hw->device_id = pci_dev->id.device_id;
hw->vendor_id = pci_dev->id.vendor_id;
hw->subsystem_device_id = pci_dev->id.subsystem_device_id;
@@ -1391,11 +1393,7 @@ nfp_net_common_init(struct rte_pci_device *pci_dev,
return -ENODEV;
}
- nfp_net_cfg_read_version(hw);
- if (!nfp_net_is_valid_nfd_version(hw->ver))
- return -EINVAL;
-
- if (nfp_net_check_dma_mask(hw, pci_dev->name) != 0)
+ if (nfp_net_check_dma_mask(pf_dev, pci_dev->name) != 0)
return -ENODEV;
/* Get some of the read-only fields from the config BAR */
@@ -1404,10 +1402,10 @@ nfp_net_common_init(struct rte_pci_device *pci_dev,
hw->max_mtu = nn_cfg_readl(&hw->super, NFP_NET_CFG_MAX_MTU);
hw->flbufsz = DEFAULT_FLBUF_SIZE;
- nfp_net_meta_init_format(hw);
+ nfp_net_meta_init_format(hw, pf_dev);
/* Read the Rx offset configured from firmware */
- if (hw->ver.major < 2)
+ if (pf_dev->ver.major < 2)
hw->rx_offset = NFP_NET_RX_OFFSET;
else
hw->rx_offset = nn_cfg_readl(&hw->super, NFP_NET_CFG_RX_OFFSET);
@@ -2118,10 +2116,10 @@ nfp_net_set_vxlan_port(struct nfp_net_hw *net_hw,
* than 40 bits.
*/
int
-nfp_net_check_dma_mask(struct nfp_net_hw *hw,
+nfp_net_check_dma_mask(struct nfp_pf_dev *pf_dev,
char *name)
{
- if (hw->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3 &&
+ if (pf_dev->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3 &&
rte_mem_check_dma_mask(40) != 0) {
PMD_DRV_LOG(ERR, "Device %s can't be used: restricted dma mask to 40 bits!",
name);
@@ -2165,16 +2163,28 @@ nfp_net_txrwb_free(struct rte_eth_dev *eth_dev)
net_hw->txrwb_mz = NULL;
}
-void
-nfp_net_cfg_read_version(struct nfp_net_hw *hw)
+static void
+nfp_net_cfg_read_version(struct nfp_hw *hw,
+ struct nfp_pf_dev *pf_dev)
{
union {
uint32_t whole;
struct nfp_net_fw_ver split;
} version;
- version.whole = nn_cfg_readl(&hw->super, NFP_NET_CFG_VERSION);
- hw->ver = version.split;
+ version.whole = nn_cfg_readl(hw, NFP_NET_CFG_VERSION);
+ pf_dev->ver = version.split;
+}
+
+bool
+nfp_net_version_check(struct nfp_hw *hw,
+ struct nfp_pf_dev *pf_dev)
+{
+ nfp_net_cfg_read_version(hw, pf_dev);
+ if (!nfp_net_is_valid_nfd_version(pf_dev->ver))
+ return false;
+
+ return true;
}
static void
@@ -2249,6 +2259,7 @@ nfp_net_firmware_version_get(struct rte_eth_dev *dev,
size_t fw_size)
{
struct nfp_net_hw *hw;
+ struct nfp_pf_dev *pf_dev;
struct nfp_net_hw_priv *hw_priv;
char app_name[FW_VER_LEN] = {0};
char mip_name[FW_VER_LEN] = {0};
@@ -2260,6 +2271,7 @@ nfp_net_firmware_version_get(struct rte_eth_dev *dev,
hw = nfp_net_get_hw(dev);
hw_priv = dev->process_private;
+ pf_dev = hw_priv->pf_dev;
if (hw->fw_version[0] != 0) {
snprintf(fw_version, FW_VER_LEN, "%s", hw->fw_version);
@@ -2268,8 +2280,8 @@ nfp_net_firmware_version_get(struct rte_eth_dev *dev,
if (!rte_eth_dev_is_repr(dev)) {
snprintf(vnic_version, FW_VER_LEN, "%d.%d.%d.%d",
- hw->ver.extend, hw->ver.class,
- hw->ver.major, hw->ver.minor);
+ pf_dev->ver.extend, pf_dev->ver.class,
+ pf_dev->ver.major, pf_dev->ver.minor);
} else {
snprintf(vnic_version, FW_VER_LEN, "*");
}
diff --git a/drivers/net/nfp/nfp_net_common.h b/drivers/net/nfp/nfp_net_common.h
index 67ec5a2d89..8d0922d48c 100644
--- a/drivers/net/nfp/nfp_net_common.h
+++ b/drivers/net/nfp/nfp_net_common.h
@@ -108,6 +108,8 @@ struct nfp_pf_dev {
enum nfp_app_fw_id app_fw_id;
+ struct nfp_net_fw_ver ver;
+
/** Pointer to the app running on the PF */
void *app_fw_priv;
@@ -219,7 +221,6 @@ struct nfp_net_hw {
const struct rte_memzone *txrwb_mz;
/** Info from the firmware */
- struct nfp_net_fw_ver ver;
uint32_t max_mtu;
uint32_t mtu;
uint32_t rx_offset;
@@ -276,8 +277,9 @@ nfp_qcp_queue_offset(const struct nfp_dev_info *dev_info,
/* Prototypes for common NFP functions */
int nfp_net_mbox_reconfig(struct nfp_net_hw *hw, uint32_t mbox_cmd);
int nfp_net_configure(struct rte_eth_dev *dev);
-int nfp_net_common_init(struct rte_pci_device *pci_dev, struct nfp_net_hw *hw);
-void nfp_net_log_device_information(const struct nfp_net_hw *hw);
+int nfp_net_common_init(struct nfp_pf_dev *pf_dev, struct nfp_net_hw *hw);
+void nfp_net_log_device_information(const struct nfp_net_hw *hw,
+ struct nfp_pf_dev *pf_dev);
void nfp_net_enable_queues(struct rte_eth_dev *dev);
void nfp_net_disable_queues(struct rte_eth_dev *dev);
void nfp_net_params_setup(struct nfp_net_hw *hw);
@@ -345,12 +347,10 @@ int nfp_net_set_vxlan_port(struct nfp_net_hw *hw, size_t idx, uint16_t port);
void nfp_net_rx_desc_limits(struct nfp_net_hw_priv *hw_priv,
uint16_t *min_rx_desc,
uint16_t *max_rx_desc);
-void nfp_net_tx_desc_limits(struct nfp_net_hw *hw,
- struct nfp_net_hw_priv *hw_priv,
+void nfp_net_tx_desc_limits(struct nfp_net_hw_priv *hw_priv,
uint16_t *min_tx_desc,
uint16_t *max_tx_desc);
-int nfp_net_check_dma_mask(struct nfp_net_hw *hw, char *name);
-void nfp_net_cfg_read_version(struct nfp_net_hw *hw);
+int nfp_net_check_dma_mask(struct nfp_pf_dev *pf_dev, char *name);
int nfp_net_firmware_version_get(struct rte_eth_dev *dev, char *fw_version, size_t fw_size);
bool nfp_net_is_valid_nfd_version(struct nfp_net_fw_ver version);
struct nfp_net_hw *nfp_net_get_hw(const struct rte_eth_dev *dev);
@@ -377,6 +377,8 @@ uint8_t nfp_function_id_get(const struct nfp_pf_dev *pf_dev,
uint8_t port_id);
int nfp_net_vf_config_app_init(struct nfp_net_hw *net_hw,
struct nfp_pf_dev *pf_dev);
+bool nfp_net_version_check(struct nfp_hw *hw,
+ struct nfp_pf_dev *pf_dev);
#define NFP_PRIV_TO_APP_FW_NIC(app_fw_priv)\
((struct nfp_app_fw_nic *)app_fw_priv)
diff --git a/drivers/net/nfp/nfp_net_meta.c b/drivers/net/nfp/nfp_net_meta.c
index 07c6758d33..5a67f87bee 100644
--- a/drivers/net/nfp/nfp_net_meta.c
+++ b/drivers/net/nfp/nfp_net_meta.c
@@ -269,14 +269,15 @@ nfp_net_meta_parse(struct nfp_net_rx_desc *rxds,
}
void
-nfp_net_meta_init_format(struct nfp_net_hw *hw)
+nfp_net_meta_init_format(struct nfp_net_hw *hw,
+ struct nfp_pf_dev *pf_dev)
{
/*
* ABI 4.x and ctrl vNIC always use chained metadata, in other cases we allow use of
* single metadata if only RSS(v1) is supported by hw capability, and RSS(v2)
* also indicate that we are using chained metadata.
*/
- if (hw->ver.major == 4) {
+ if (pf_dev->ver.major == 4) {
hw->meta_format = NFP_NET_METAFORMAT_CHAINED;
} else if ((hw->super.cap & NFP_NET_CFG_CTRL_CHAIN_META) != 0) {
hw->meta_format = NFP_NET_METAFORMAT_CHAINED;
diff --git a/drivers/net/nfp/nfp_net_meta.h b/drivers/net/nfp/nfp_net_meta.h
index 69d08cf3a7..c3d84dff60 100644
--- a/drivers/net/nfp/nfp_net_meta.h
+++ b/drivers/net/nfp/nfp_net_meta.h
@@ -89,7 +89,10 @@ struct nfp_net_meta_parsed {
} vlan[NFP_NET_META_MAX_VLANS];
};
-void nfp_net_meta_init_format(struct nfp_net_hw *hw);
+struct nfp_pf_dev;
+
+void nfp_net_meta_init_format(struct nfp_net_hw *hw,
+ struct nfp_pf_dev *pf_dev);
void nfp_net_meta_parse(struct nfp_net_rx_desc *rxds,
struct nfp_net_rxq *rxq,
struct nfp_net_hw *hw,
diff --git a/drivers/net/nfp/nfp_rxtx.c b/drivers/net/nfp/nfp_rxtx.c
index 05218537f7..d101477161 100644
--- a/drivers/net/nfp/nfp_rxtx.c
+++ b/drivers/net/nfp/nfp_rxtx.c
@@ -816,11 +816,11 @@ nfp_net_tx_queue_setup(struct rte_eth_dev *dev,
unsigned int socket_id,
const struct rte_eth_txconf *tx_conf)
{
- struct nfp_net_hw *hw;
+ struct nfp_net_hw_priv *hw_priv;
- hw = nfp_net_get_hw(dev);
+ hw_priv = dev->process_private;
- if (hw->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3)
+ if (hw_priv->pf_dev->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3)
return nfp_net_nfd3_tx_queue_setup(dev, queue_idx,
nb_desc, socket_id, tx_conf);
else
@@ -852,10 +852,10 @@ nfp_net_tx_queue_info_get(struct rte_eth_dev *dev,
struct rte_eth_txq_info *info)
{
struct rte_eth_dev_info dev_info;
- struct nfp_net_hw *hw = nfp_net_get_hw(dev);
+ struct nfp_net_hw_priv *hw_priv = dev->process_private;
struct nfp_net_txq *txq = dev->data->tx_queues[queue_id];
- if (hw->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3)
+ if (hw_priv->pf_dev->ver.extend == NFP_NET_CFG_VERSION_DP_NFD3)
info->nb_desc = txq->tx_count / NFD3_TX_DESC_PER_PKT;
else
info->nb_desc = txq->tx_count / NFDK_TX_DESC_PER_SIMPLE_PKT;
--
2.39.1
^ permalink raw reply [relevance 2%]
* Re: [RFC 1/2] eal: add llc aware functions
@ 2024-09-02 1:20 4% ` Varghese, Vipin
0 siblings, 0 replies; 200+ results
From: Varghese, Vipin @ 2024-09-02 1:20 UTC (permalink / raw)
To: Wathsala Wathawana Vithanage, ferruh.yigit, dev; +Cc: nd
[-- Attachment #1: Type: text/plain, Size: 1706 bytes --]
<Snipped>
>> -unsigned int rte_get_next_lcore(unsigned int i, int skip_main, int wrap)
>> +#define LCORE_GET_LLC \
>> + "ls -d /sys/bus/cpu/devices/cpu%u/cache/index[0-9] | sort -r
>> | grep -m1 index[0-9] | awk -F '[x]' '{print $2}' "
>>
> This won't work for some SOCs.
Thank you for your response. please find our response and queries below
> How to ensure the index you got is for an LLC?
we referred to How CPU topology info is exported via sysfs — The Linux
Kernel documentation
<https://www.kernel.org/doc/html/latest/admin-guide/cputopology.html>
and linux/Documentation/ABI/stable/sysfs-devices-system-cpu at master ·
torvalds/linux (github.com)
<https://github.com/torvalds/linux/blob/master/Documentation/ABI/stable/sysfs-devices-system-cpu>
and
Get Cache Info in Linux on ARMv8 64-bit Platform (zhiyisun.github.io)
<https://zhiyisun.github.io/2016/06/25/Get-Cache-Info-in-Linux-on-ARMv8-64-bit-Platform.html>.
Based on my current understanding on bare metal 64Bit Linux OS (which is
supported by most Distros), the cache topology are populated into sysfs.
> Some SOCs may only show upper-level caches here, therefore cannot be use blindly without knowing the SOC.
Can you please help us understand
1. if there are specific SoC which do not populate the information at
all? If yes are they in DTS?
2. If there are specific SoC which does not export to hypervisor like
Qemu or Xen?
We can work together to make it compatible.
> Also, unacceptable to execute a shell script, consider implementing in C.
As the intention of the RFC is to share possible API and Macro, we
welcome suggestions on the implementation as agreed with Stepehen.
>
> --wathsala
>
>
[-- Attachment #2: Type: text/html, Size: 3373 bytes --]
^ permalink raw reply [relevance 4%]
* [PATCH v3 0/1] bbdev: removing unnecessaray symbols
@ 2024-08-27 23:06 3% Nicolas Chautru
2024-08-27 23:06 3% ` [PATCH v3 1/1] bbdev: removing unnecessary symbols from version map Nicolas Chautru
0 siblings, 1 reply; 200+ results
From: Nicolas Chautru @ 2024-08-27 23:06 UTC (permalink / raw)
To: dev, maxime.coquelin
Cc: hemant.agrawal, david.marchand, hernan.vargas, Nicolas Chautru
v3: typo fixes.
v2: Actually several functions can be removed from bbdev version map
since they are inline and hence ABI versionning is not relevant.
I checked with other lib (cryptodev/ethdev) and the same guideline is
followed, with inline functions not part of version.map. Similarly the
script as part of CICD do no enforce versionning for inline functions
either. Discussed a bitwith Maxime off list.
Any thoughts? Good to clean it up now.
v1: A few functions were somehow missing for the last few years in the
version map file.
Nicolas Chautru (1):
bbdev: removing unnecessary symbols from version map
lib/bbdev/rte_bbdev.h | 1 -
lib/bbdev/rte_bbdev_op.h | 2 --
lib/bbdev/version.map | 24 +-----------------------
3 files changed, 1 insertion(+), 26 deletions(-)
--
2.34.1
^ permalink raw reply [relevance 3%]
* [PATCH v3 1/1] bbdev: removing unnecessary symbols from version map
2024-08-27 23:06 3% [PATCH v3 0/1] bbdev: removing unnecessaray symbols Nicolas Chautru
@ 2024-08-27 23:06 3% ` Nicolas Chautru
2024-09-13 13:51 0% ` Maxime Coquelin
2024-09-19 8:26 0% ` Maxime Coquelin
0 siblings, 2 replies; 200+ results
From: Nicolas Chautru @ 2024-08-27 23:06 UTC (permalink / raw)
To: dev, maxime.coquelin
Cc: hemant.agrawal, david.marchand, hernan.vargas, Nicolas Chautru
A number of inline functions should not be in the
version map since ABI versioning would be irrelevant.
Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
---
lib/bbdev/rte_bbdev.h | 1 -
lib/bbdev/rte_bbdev_op.h | 2 --
lib/bbdev/version.map | 24 +-----------------------
3 files changed, 1 insertion(+), 26 deletions(-)
diff --git a/lib/bbdev/rte_bbdev.h b/lib/bbdev/rte_bbdev.h
index 25514c58ac..bd49a0a304 100644
--- a/lib/bbdev/rte_bbdev.h
+++ b/lib/bbdev/rte_bbdev.h
@@ -897,7 +897,6 @@ rte_bbdev_dequeue_fft_ops(uint16_t dev_id, uint16_t queue_id,
* The number of operations actually dequeued (this is the number of entries
* copied into the @p ops array).
*/
-__rte_experimental
static inline uint16_t
rte_bbdev_dequeue_mldts_ops(uint16_t dev_id, uint16_t queue_id,
struct rte_bbdev_mldts_op **ops, uint16_t num_ops)
diff --git a/lib/bbdev/rte_bbdev_op.h b/lib/bbdev/rte_bbdev_op.h
index 459631d0d0..5b862c13a6 100644
--- a/lib/bbdev/rte_bbdev_op.h
+++ b/lib/bbdev/rte_bbdev_op.h
@@ -1159,7 +1159,6 @@ rte_bbdev_fft_op_alloc_bulk(struct rte_mempool *mempool,
* - 0 on success.
* - EINVAL if invalid mempool is provided.
*/
-__rte_experimental
static inline int
rte_bbdev_mldts_op_alloc_bulk(struct rte_mempool *mempool,
struct rte_bbdev_mldts_op **ops, uint16_t num_ops)
@@ -1236,7 +1235,6 @@ rte_bbdev_fft_op_free_bulk(struct rte_bbdev_fft_op **ops, unsigned int num_ops)
* @param num_ops
* Number of structures
*/
-__rte_experimental
static inline void
rte_bbdev_mldts_op_free_bulk(struct rte_bbdev_mldts_op **ops, unsigned int num_ops)
{
diff --git a/lib/bbdev/version.map b/lib/bbdev/version.map
index e0d82ff752..2a5baacd90 100644
--- a/lib/bbdev/version.map
+++ b/lib/bbdev/version.map
@@ -6,21 +6,9 @@ DPDK_25 {
rte_bbdev_callback_unregister;
rte_bbdev_close;
rte_bbdev_count;
- rte_bbdev_dec_op_alloc_bulk;
- rte_bbdev_dec_op_free_bulk;
- rte_bbdev_dequeue_dec_ops;
- rte_bbdev_dequeue_enc_ops;
- rte_bbdev_dequeue_fft_ops;
rte_bbdev_device_status_str;
rte_bbdev_devices;
- rte_bbdev_enc_op_alloc_bulk;
- rte_bbdev_enc_op_free_bulk;
- rte_bbdev_enqueue_dec_ops;
- rte_bbdev_enqueue_enc_ops;
- rte_bbdev_enqueue_fft_ops;
rte_bbdev_enqueue_status_str;
- rte_bbdev_fft_op_alloc_bulk;
- rte_bbdev_fft_op_free_bulk;
rte_bbdev_find_next;
rte_bbdev_get_named_dev;
rte_bbdev_info_get;
@@ -44,14 +32,4 @@ DPDK_25 {
rte_bbdev_stop;
local: *;
-};
-
-EXPERIMENTAL {
- global:
-
- # added in 23.11
- rte_bbdev_dequeue_mldts_ops;
- rte_bbdev_enqueue_mldts_ops;
- rte_bbdev_mldts_op_alloc_bulk;
- rte_bbdev_mldts_op_free_bulk;
-};
+};
\ No newline at end of file
--
2.34.1
^ permalink raw reply [relevance 3%]
* [PATCH v2 1/1] bbdev: removing unnecessaray symbols from version map
2024-08-27 23:03 3% [PATCH v2 0/1] bbdev: removing unnecessaray symbols Nicolas Chautru
@ 2024-08-27 23:03 3% ` Nicolas Chautru
2024-09-10 16:20 0% ` Maxime Coquelin
0 siblings, 1 reply; 200+ results
From: Nicolas Chautru @ 2024-08-27 23:03 UTC (permalink / raw)
To: dev, maxime.coquelin
Cc: hemant.agrawal, david.marchand, hernan.vargas, Nicolas Chautru
A number of inline functions should not be in the
version map since ABI versionning would be irrelevant.
Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
---
lib/bbdev/rte_bbdev.h | 1 -
lib/bbdev/rte_bbdev_op.h | 2 --
lib/bbdev/version.map | 24 +-----------------------
3 files changed, 1 insertion(+), 26 deletions(-)
diff --git a/lib/bbdev/rte_bbdev.h b/lib/bbdev/rte_bbdev.h
index 25514c58ac..bd49a0a304 100644
--- a/lib/bbdev/rte_bbdev.h
+++ b/lib/bbdev/rte_bbdev.h
@@ -897,7 +897,6 @@ rte_bbdev_dequeue_fft_ops(uint16_t dev_id, uint16_t queue_id,
* The number of operations actually dequeued (this is the number of entries
* copied into the @p ops array).
*/
-__rte_experimental
static inline uint16_t
rte_bbdev_dequeue_mldts_ops(uint16_t dev_id, uint16_t queue_id,
struct rte_bbdev_mldts_op **ops, uint16_t num_ops)
diff --git a/lib/bbdev/rte_bbdev_op.h b/lib/bbdev/rte_bbdev_op.h
index 459631d0d0..5b862c13a6 100644
--- a/lib/bbdev/rte_bbdev_op.h
+++ b/lib/bbdev/rte_bbdev_op.h
@@ -1159,7 +1159,6 @@ rte_bbdev_fft_op_alloc_bulk(struct rte_mempool *mempool,
* - 0 on success.
* - EINVAL if invalid mempool is provided.
*/
-__rte_experimental
static inline int
rte_bbdev_mldts_op_alloc_bulk(struct rte_mempool *mempool,
struct rte_bbdev_mldts_op **ops, uint16_t num_ops)
@@ -1236,7 +1235,6 @@ rte_bbdev_fft_op_free_bulk(struct rte_bbdev_fft_op **ops, unsigned int num_ops)
* @param num_ops
* Number of structures
*/
-__rte_experimental
static inline void
rte_bbdev_mldts_op_free_bulk(struct rte_bbdev_mldts_op **ops, unsigned int num_ops)
{
diff --git a/lib/bbdev/version.map b/lib/bbdev/version.map
index e0d82ff752..2a5baacd90 100644
--- a/lib/bbdev/version.map
+++ b/lib/bbdev/version.map
@@ -6,21 +6,9 @@ DPDK_25 {
rte_bbdev_callback_unregister;
rte_bbdev_close;
rte_bbdev_count;
- rte_bbdev_dec_op_alloc_bulk;
- rte_bbdev_dec_op_free_bulk;
- rte_bbdev_dequeue_dec_ops;
- rte_bbdev_dequeue_enc_ops;
- rte_bbdev_dequeue_fft_ops;
rte_bbdev_device_status_str;
rte_bbdev_devices;
- rte_bbdev_enc_op_alloc_bulk;
- rte_bbdev_enc_op_free_bulk;
- rte_bbdev_enqueue_dec_ops;
- rte_bbdev_enqueue_enc_ops;
- rte_bbdev_enqueue_fft_ops;
rte_bbdev_enqueue_status_str;
- rte_bbdev_fft_op_alloc_bulk;
- rte_bbdev_fft_op_free_bulk;
rte_bbdev_find_next;
rte_bbdev_get_named_dev;
rte_bbdev_info_get;
@@ -44,14 +32,4 @@ DPDK_25 {
rte_bbdev_stop;
local: *;
-};
-
-EXPERIMENTAL {
- global:
-
- # added in 23.11
- rte_bbdev_dequeue_mldts_ops;
- rte_bbdev_enqueue_mldts_ops;
- rte_bbdev_mldts_op_alloc_bulk;
- rte_bbdev_mldts_op_free_bulk;
-};
+};
\ No newline at end of file
--
2.34.1
^ permalink raw reply [relevance 3%]
* [PATCH v2 0/1] bbdev: removing unnecessaray symbols
@ 2024-08-27 23:03 3% Nicolas Chautru
2024-08-27 23:03 3% ` [PATCH v2 1/1] bbdev: removing unnecessaray symbols from version map Nicolas Chautru
0 siblings, 1 reply; 200+ results
From: Nicolas Chautru @ 2024-08-27 23:03 UTC (permalink / raw)
To: dev, maxime.coquelin
Cc: hemant.agrawal, david.marchand, hernan.vargas, Nicolas Chautru
v2: Actually several functions can be removed from bbdev version map
since they are inline and hence ABI versionning is not relevant.
I checked with other lib (cryptodev/ethdev) and the same guideline
is followed, with inline functions not part of version.map. Similarly
the script as part of CICD do no enforce versionning for inline
functions either. Discussed a bitwith Maxime off list.
Any thoughts? Good to clean it up now.
v1: A few functions were somehow missing for the last few years
in the version map file.
Nicolas Chautru (1):
bbdev: removing unnecessaray symbols from version map
lib/bbdev/rte_bbdev.h | 1 -
lib/bbdev/rte_bbdev_op.h | 2 --
lib/bbdev/version.map | 24 +-----------------------
3 files changed, 1 insertion(+), 26 deletions(-)
--
2.34.1
^ permalink raw reply [relevance 3%]
* Re: [PATCH v2 2/4] power: refactor uncore power management library
@ 2024-08-27 13:02 3% ` lihuisong (C)
2024-10-08 6:19 0% ` Tummala, Sivaprasad
0 siblings, 1 reply; 200+ results
From: lihuisong (C) @ 2024-08-27 13:02 UTC (permalink / raw)
To: Sivaprasad Tummala
Cc: dev, david.hunt, anatoly.burakov, radu.nicolau, jerinj,
cristian.dumitrescu, konstantin.ananyev, ferruh.yigit, gakhil
Hi Sivaprasad,
Suggest to split this patch into two patches for easiler to review:
patch-1: abstract a file for uncore dvfs core level, namely, the
rte_power_uncore_ops.c you did.
patch-2: move and rename, lib/power/power_intel_uncore.c =>
drivers/power/intel_uncore/intel_uncore.c
patch[1/4] is also too big and not good to review.
In addition, I have some question and am not sure if we can adjust
uncore init process.
/Huisong
在 2024/8/26 21:06, Sivaprasad Tummala 写道:
> This patch refactors the power management library, addressing uncore
> power management. The primary changes involve the creation of dedicated
> directories for each driver within 'drivers/power/uncore/*'. The
> adjustment of meson.build files enables the selective activation
> of individual drivers.
>
> This refactor significantly improves code organization, enhances
> clarity and boosts maintainability. It lays the foundation for more
> focused development on individual drivers and facilitates seamless
> integration of future enhancements, particularly the AMD uncore driver.
>
> Signed-off-by: Sivaprasad Tummala <sivaprasad.tummala@amd.com>
> ---
> .../power/intel_uncore/intel_uncore.c | 18 +-
> .../power/intel_uncore/intel_uncore.h | 8 +-
> drivers/power/intel_uncore/meson.build | 6 +
> drivers/power/meson.build | 3 +-
> lib/power/meson.build | 2 +-
> lib/power/rte_power_uncore.c | 205 ++++++---------
> lib/power/rte_power_uncore.h | 87 ++++---
> lib/power/rte_power_uncore_ops.h | 239 ++++++++++++++++++
> lib/power/version.map | 1 +
> 9 files changed, 405 insertions(+), 164 deletions(-)
> rename lib/power/power_intel_uncore.c => drivers/power/intel_uncore/intel_uncore.c (95%)
> rename lib/power/power_intel_uncore.h => drivers/power/intel_uncore/intel_uncore.h (97%)
> create mode 100644 drivers/power/intel_uncore/meson.build
> create mode 100644 lib/power/rte_power_uncore_ops.h
>
> diff --git a/lib/power/power_intel_uncore.c b/drivers/power/intel_uncore/intel_uncore.c
> similarity index 95%
> rename from lib/power/power_intel_uncore.c
> rename to drivers/power/intel_uncore/intel_uncore.c
> index 4eb9c5900a..804ad5d755 100644
> --- a/lib/power/power_intel_uncore.c
> +++ b/drivers/power/intel_uncore/intel_uncore.c
> @@ -8,7 +8,7 @@
>
> #include <rte_memcpy.h>
>
> -#include "power_intel_uncore.h"
> +#include "intel_uncore.h"
> #include "power_common.h"
>
> #define MAX_NUMA_DIE 8
> @@ -475,3 +475,19 @@ power_intel_uncore_get_num_dies(unsigned int pkg)
>
> return count;
> }
<...>
>
> -#endif /* POWER_INTEL_UNCORE_H */
> +#endif /* INTEL_UNCORE_H */
> diff --git a/drivers/power/intel_uncore/meson.build b/drivers/power/intel_uncore/meson.build
> new file mode 100644
> index 0000000000..876df8ad14
> --- /dev/null
> +++ b/drivers/power/intel_uncore/meson.build
> @@ -0,0 +1,6 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2017 Intel Corporation
> +# Copyright(c) 2024 Advanced Micro Devices, Inc.
> +
> +sources = files('intel_uncore.c')
> +deps += ['power']
> diff --git a/drivers/power/meson.build b/drivers/power/meson.build
> index 8c7215c639..c83047af94 100644
> --- a/drivers/power/meson.build
> +++ b/drivers/power/meson.build
> @@ -6,7 +6,8 @@ drivers = [
> 'amd_pstate',
> 'cppc',
> 'kvm_vm',
> - 'pstate'
> + 'pstate',
> + 'intel_uncore'
The cppc, amd_pstate and so on belong to cpufreq scope.
And intel_uncore belongs to uncore dvfs scope.
They are not the same level. So I proposes that we need to create one
directory called like cpufreq or core.
This 'intel_uncore' name don't seems appropriate. what do you think the
following directory structure:
drivers/power/uncore/intel_uncore.c
drivers/power/uncore/amd_uncore.c (according to the patch[4/4]).
> ]
> std_deps = ['power']
> diff --git a/lib/power/meson.build b/lib/power/meson.build
> index f3e3451cdc..9b13d98810 100644
> --- a/lib/power/meson.build
> +++ b/lib/power/meson.build
> @@ -13,7 +13,6 @@ if not is_linux
> endif
> sources = files(
> 'power_common.c',
> - 'power_intel_uncore.c',
> 'rte_power.c',
> 'rte_power_uncore.c',
> 'rte_power_pmd_mgmt.c',
> @@ -24,6 +23,7 @@ headers = files(
> 'rte_power_guest_channel.h',
> 'rte_power_pmd_mgmt.h',
> 'rte_power_uncore.h',
> + 'rte_power_uncore_ops.h',
> )
> if cc.has_argument('-Wno-cast-qual')
> cflags += '-Wno-cast-qual'
> diff --git a/lib/power/rte_power_uncore.c b/lib/power/rte_power_uncore.c
> index 48c75a5da0..9f8771224f 100644
> --- a/lib/power/rte_power_uncore.c
> +++ b/lib/power/rte_power_uncore.c
> @@ -1,6 +1,7 @@
> /* SPDX-License-Identifier: BSD-3-Clause
> * Copyright(c) 2010-2014 Intel Corporation
> * Copyright(c) 2023 AMD Corporation
> + * Copyright(c) 2024 Advanced Micro Devices, Inc.
> */
>
> #include <errno.h>
> @@ -12,98 +13,50 @@
> #include "rte_power_uncore.h"
> #include "power_intel_uncore.h"
>
> -enum rte_uncore_power_mgmt_env default_uncore_env = RTE_UNCORE_PM_ENV_NOT_SET;
> +static enum rte_uncore_power_mgmt_env global_uncore_env = RTE_UNCORE_PM_ENV_NOT_SET;
> +static struct rte_power_uncore_ops *global_uncore_ops;
>
> static rte_spinlock_t global_env_cfg_lock = RTE_SPINLOCK_INITIALIZER;
> +static RTE_TAILQ_HEAD(, rte_power_uncore_ops) uncore_ops_list =
> + TAILQ_HEAD_INITIALIZER(uncore_ops_list);
>
> -static uint32_t
> -power_get_dummy_uncore_freq(unsigned int pkg __rte_unused,
> - unsigned int die __rte_unused)
> -{
> - return 0;
> -}
> -
> -static int
> -power_set_dummy_uncore_freq(unsigned int pkg __rte_unused,
> - unsigned int die __rte_unused, uint32_t index __rte_unused)
> -{
> - return 0;
> -}
> +const char *uncore_env_str[] = {
> + "not set",
> + "auto-detect",
> + "intel-uncore",
> + "amd-hsmp"
> +};
Why open the "auto-detect" mode to user?
Why not set this automatically at framework initialization?
After all, the uncore driver is fixed for one platform.
>
> -static int
> -power_dummy_uncore_freq_max(unsigned int pkg __rte_unused,
> - unsigned int die __rte_unused)
> -{
> - return 0;
> -}
> -
<...>
> -static int
> -power_dummy_uncore_get_num_freqs(unsigned int pkg __rte_unused,
> - unsigned int die __rte_unused)
> +/* register the ops struct in rte_power_uncore_ops, return 0 on success. */
> +int
> +rte_power_register_uncore_ops(struct rte_power_uncore_ops *driver_ops)
> {
> - return 0;
> -}
> + if (!driver_ops->init || !driver_ops->exit || !driver_ops->get_num_pkgs ||
> + !driver_ops->get_num_dies || !driver_ops->get_num_freqs ||
> + !driver_ops->get_avail_freqs || !driver_ops->get_freq ||
> + !driver_ops->set_freq || !driver_ops->freq_max ||
> + !driver_ops->freq_min) {
> + POWER_LOG(ERR, "Missing callbacks while registering power ops");
> + return -1;
> + }
> + if (driver_ops->cb)
> + driver_ops->cb();
>
> -static unsigned int
> -power_dummy_uncore_get_num_pkgs(void)
> -{
> - return 0;
> -}
> + TAILQ_INSERT_TAIL(&uncore_ops_list, driver_ops, next);
>
> -static unsigned int
> -power_dummy_uncore_get_num_dies(unsigned int pkg __rte_unused)
> -{
> return 0;
> }
> -
> -/* function pointers */
> -rte_power_get_uncore_freq_t rte_power_get_uncore_freq = power_get_dummy_uncore_freq;
> -rte_power_set_uncore_freq_t rte_power_set_uncore_freq = power_set_dummy_uncore_freq;
> -rte_power_uncore_freq_change_t rte_power_uncore_freq_max = power_dummy_uncore_freq_max;
> -rte_power_uncore_freq_change_t rte_power_uncore_freq_min = power_dummy_uncore_freq_min;
> -rte_power_uncore_freqs_t rte_power_uncore_freqs = power_dummy_uncore_freqs;
> -rte_power_uncore_get_num_freqs_t rte_power_uncore_get_num_freqs = power_dummy_uncore_get_num_freqs;
> -rte_power_uncore_get_num_pkgs_t rte_power_uncore_get_num_pkgs = power_dummy_uncore_get_num_pkgs;
> -rte_power_uncore_get_num_dies_t rte_power_uncore_get_num_dies = power_dummy_uncore_get_num_dies;
> -
> -static void
> -reset_power_uncore_function_ptrs(void)
> -{
> - rte_power_get_uncore_freq = power_get_dummy_uncore_freq;
> - rte_power_set_uncore_freq = power_set_dummy_uncore_freq;
> - rte_power_uncore_freq_max = power_dummy_uncore_freq_max;
> - rte_power_uncore_freq_min = power_dummy_uncore_freq_min;
> - rte_power_uncore_freqs = power_dummy_uncore_freqs;
> - rte_power_uncore_get_num_freqs = power_dummy_uncore_get_num_freqs;
> - rte_power_uncore_get_num_pkgs = power_dummy_uncore_get_num_pkgs;
> - rte_power_uncore_get_num_dies = power_dummy_uncore_get_num_dies;
> -}
> -
> int
> rte_power_set_uncore_env(enum rte_uncore_power_mgmt_env env)
> {
> - int ret;
> + int ret = -1;
> + struct rte_power_uncore_ops *ops;
>
> rte_spinlock_lock(&global_env_cfg_lock);
>
> - if (default_uncore_env != RTE_UNCORE_PM_ENV_NOT_SET) {
> + if (global_uncore_env != RTE_UNCORE_PM_ENV_NOT_SET) {
> POWER_LOG(ERR, "Uncore Power Management Env already set.");
> - rte_spinlock_unlock(&global_env_cfg_lock);
> - return -1;
> + goto out;
> }
>
<...>
> + if (env <= RTE_DIM(uncore_env_str)) {
> + RTE_TAILQ_FOREACH(ops, &uncore_ops_list, next)
> + if (strncmp(ops->name, uncore_env_str[env],
> + RTE_POWER_UNCORE_DRIVER_NAMESZ) == 0) {
> + global_uncore_env = env;
> + global_uncore_ops = ops;
> + ret = 0;
> + goto out;
> + }
> + POWER_LOG(ERR, "Power Management (%s) not supported",
> + uncore_env_str[env]);
> + } else
> + POWER_LOG(ERR, "Invalid Power Management Environment");
>
> - default_uncore_env = env;
> out:
> rte_spinlock_unlock(&global_env_cfg_lock);
> return ret;
> @@ -139,15 +89,22 @@ void
> rte_power_unset_uncore_env(void)
> {
> rte_spinlock_lock(&global_env_cfg_lock);
> - default_uncore_env = RTE_UNCORE_PM_ENV_NOT_SET;
> - reset_power_uncore_function_ptrs();
> + global_uncore_env = RTE_UNCORE_PM_ENV_NOT_SET;
> rte_spinlock_unlock(&global_env_cfg_lock);
> }
>
How about abstract an ABI interface to intialize or set the uncore
driver on platform by automatical.
And later do power_intel_uncore_init_on_die() for each die on different
package.
> enum rte_uncore_power_mgmt_env
> rte_power_get_uncore_env(void)
> {
> - return default_uncore_env;
> + return global_uncore_env;
> +}
> +
> +struct rte_power_uncore_ops *
> +rte_power_get_uncore_ops(void)
> +{
> + RTE_ASSERT(global_uncore_ops != NULL);
> +
> + return global_uncore_ops;
> }
>
> int
> @@ -155,27 +112,29 @@ rte_power_uncore_init(unsigned int pkg, unsigned int die)
This pkg means the socket id on the platform, right?
If so, I am not sure that the
uncore_info[RTE_MAX_NUMA_NODES][MAX_NUMA_DIE] used in uncore lib is
universal for all uncore driver.
For example, uncore driver just support do uncore dvfs based on the
socket unit.
What shoud we do for this? we may need to think twice.
> {
> int ret = -1;
>
<...>
^ permalink raw reply [relevance 3%]
* Re: [PATCH v2] app/testpmd: show output of commands read from file
2024-08-22 21:09 3% ` Ferruh Yigit
@ 2024-08-23 9:12 0% ` Bruce Richardson
0 siblings, 0 replies; 200+ results
From: Bruce Richardson @ 2024-08-23 9:12 UTC (permalink / raw)
To: Ferruh Yigit; +Cc: dev
On Thu, Aug 22, 2024 at 10:09:09PM +0100, Ferruh Yigit wrote:
> On 8/22/2024 6:18 PM, Bruce Richardson wrote:
> > On Thu, Aug 22, 2024 at 06:14:55PM +0100, Bruce Richardson wrote:
> >> On Thu, Aug 22, 2024 at 05:53:27PM +0100, Ferruh Yigit wrote:
> >>> On 8/22/2024 11:41 AM, Bruce Richardson wrote:
> >>>> Testpmd supports the "--cmdline-file" parameter to read a set of initial
> >>>> commands from a file. However, the only indication that this has been
> >>>> done successfully on startup is a single-line message, no output from
> >>>> the commands is seen.
> >>>>
> >>>
> >>> For user I think it makes sense to see the command [1], only concern is
> >>> if someone parsing testpmd output may be impacted on this, although I
> >>> expect it should be trivial to update the relevant parsing.
> >>>
> >>> [1]
> >>> Btw, I can still see the command output, I assume because command does
> >>> the printf itself, for example for 'show port summary 0' command:
> >>> - before patch:
> >>> ...
> >>> Number of available ports: 2
> >>> Port MAC Address Name Driver Status Link
> >>> 0 xx:xx:xx:xx:xx:xx xxxx:xx:xx.x aaaaaaaa up xxx Gbps
> >>> ...
> >>>
> >>> - after patch
> >>> ...
> >>> testpmd> show port summary 0
> >>> Number of available ports: 2
> >>> Port MAC Address Name Driver Status Link
> >>> 0 xx:xx:xx:xx:xx:xx xxxx:xx:xx.x aaaaaaaa up xxx Gbps
> >>> ...
> >>>
> >>> Only difference above is, after patch the command itself also printed.
> >>>
> >>>
> >>
> >> That's because the function uses printf itself, which is actually wrong.
> >> Any output from a cmdline function should use the "cmdline_printf" call
> >> which outputs to the proper cmdline filehandle.
> >>
>
> Got it.
> But in existing testpmd code, only a handful cmdline functions use the
> 'cmdline_printf' and most of them are in the same help function.
> At this stage I think no need to update them. There is already some
> confusion on testpmd logging between printf & TESTPMD_LOG().
>
Agree. No point in updating the existing functions to use cmdline_printf vs
printf.
One other point related to echoing commands, there are also testpmd
commands that produce no output - the commands for configuring rte_tm,
being examples right now - and having those echoed to screen when read from
a file is the only way to know what is actually happening.
> >>>> To improve usability here, we can use cmdline_new rather than
> >>>> cmdline_file_new and have the output from the various commands sent to
> >>>> stdout, allowing the user to see better what is happening.
> >>>>
> >>>> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> >>>>
> >>>> ---
> >>>> v2: use STDOUT_FILENO in place of hard-coded "1"
> >>>> ---
> >>>> app/test-pmd/cmdline.c | 14 +++++++++++++-
> >>>> 1 file changed, 13 insertions(+), 1 deletion(-)
> >>>>
> >>>> diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
> >>>> index b7759e38a8..52e64430d9 100644
> >>>> --- a/app/test-pmd/cmdline.c
> >>>> +++ b/app/test-pmd/cmdline.c
> >>>> @@ -6,6 +6,7 @@
> >>>> #include <ctype.h>
> >>>> #include <stdarg.h>
> >>>> #include <errno.h>
> >>>> +#include <fcntl.h>
> >>>> #include <stdio.h>
> >>>> #include <stdint.h>
> >>>> #include <stdlib.h>
> >>>> @@ -13431,7 +13432,18 @@ cmdline_read_from_file(const char *filename)
> >>>> {
> >>>> struct cmdline *cl;
> >>>>
> >>>> - cl = cmdline_file_new(main_ctx, "testpmd> ", filename);
> >>>> + /* cmdline_file_new does not produce any output which is not ideal here.
> >>>> + * Much better to show output of the commands, so we open filename directly
> >>>> + * and then pass that to cmdline_new with stdout as the output path.
> >>>> + */
> >>>> + int fd = open(filename, O_RDONLY);
> >>>> + if (fd < 0) {
> >>>> + fprintf(stderr, "Failed to open file %s: %s\n",
> >>>> + filename, strerror(errno));
> >>>> + return;
> >>>> + }
> >>>> +
> >>>> + cl = cmdline_new(main_ctx, "testpmd> ", fd, STDOUT_FILENO);
> >>>>
> >>>
> >>> Above is almost save as 'cmdline_file_new()' function with only
> >>> difference that it uses '-1' for 's_out'.
> >>>
> >>> If this usecase may be required by others, do you think does it have a
> >>> value to pass 's_out' to 'cmdline_file_new()' or have a new version of
> >>> API that accepts 's_out' as parameter?
> >>>
> >>
> >> Yes, I thought about this, and actually started implementing a new API for
> >> cmdline library to that. However, I decided that, given the complexity
> >> here, that it's not really necessary - especially as there is no clear way
> >> to do things. The options are:
> >>
> >> * extend cmdline_file_new to have a flag to echo to stdout (which would be
> >> the very common case here).
> >> * extend cmdline_file_new to take a file handle - this is more flexible,
> >> but slightly less usable.
> >> * add a new cmdline_file_<something>_new function to echo to stdout.
> >> * add a new cmdline_file_<something>_new function to write to a filehandle.
> >>
> >> I don't like breaking the cmdline API (and ABI), so I didn't want to do
> >> either #1 or #2, which would be the cleanest solutions. For #3 and #4,
> >> naming is hard, and deciding between them is even harder. Given the choice,
> >> I prefer #3, as I can't see #4 being very common and we always have
> >> cmdline_new as a fallback anyway.
> >>
> >> Overall, though, I threw away that work, because it didn't seem worth it,
> >> for the sake of having the user to do an extra "open" call.
> >>
> >
>
> I vote to option #1, but agree that it may not worth breaking API and ABI.
>
> Is 'cmdline_file_new_v2()' too bad a name, perhaps better to go with
> testpmd implementation, as you did in this patch.
>
Let's see what others think. I'm fine to implement this as a cmdline lib
change or a testpmd-local change only, whatever the community prefers.
/Bruce
^ permalink raw reply [relevance 0%]
* Re: [PATCH v2] app/testpmd: show output of commands read from file
2024-08-22 17:18 0% ` Bruce Richardson
@ 2024-08-22 21:09 3% ` Ferruh Yigit
2024-08-23 9:12 0% ` Bruce Richardson
0 siblings, 1 reply; 200+ results
From: Ferruh Yigit @ 2024-08-22 21:09 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev
On 8/22/2024 6:18 PM, Bruce Richardson wrote:
> On Thu, Aug 22, 2024 at 06:14:55PM +0100, Bruce Richardson wrote:
>> On Thu, Aug 22, 2024 at 05:53:27PM +0100, Ferruh Yigit wrote:
>>> On 8/22/2024 11:41 AM, Bruce Richardson wrote:
>>>> Testpmd supports the "--cmdline-file" parameter to read a set of initial
>>>> commands from a file. However, the only indication that this has been
>>>> done successfully on startup is a single-line message, no output from
>>>> the commands is seen.
>>>>
>>>
>>> For user I think it makes sense to see the command [1], only concern is
>>> if someone parsing testpmd output may be impacted on this, although I
>>> expect it should be trivial to update the relevant parsing.
>>>
>>> [1]
>>> Btw, I can still see the command output, I assume because command does
>>> the printf itself, for example for 'show port summary 0' command:
>>> - before patch:
>>> ...
>>> Number of available ports: 2
>>> Port MAC Address Name Driver Status Link
>>> 0 xx:xx:xx:xx:xx:xx xxxx:xx:xx.x aaaaaaaa up xxx Gbps
>>> ...
>>>
>>> - after patch
>>> ...
>>> testpmd> show port summary 0
>>> Number of available ports: 2
>>> Port MAC Address Name Driver Status Link
>>> 0 xx:xx:xx:xx:xx:xx xxxx:xx:xx.x aaaaaaaa up xxx Gbps
>>> ...
>>>
>>> Only difference above is, after patch the command itself also printed.
>>>
>>>
>>
>> That's because the function uses printf itself, which is actually wrong.
>> Any output from a cmdline function should use the "cmdline_printf" call
>> which outputs to the proper cmdline filehandle.
>>
Got it.
But in existing testpmd code, only a handful cmdline functions use the
'cmdline_printf' and most of them are in the same help function.
At this stage I think no need to update them. There is already some
confusion on testpmd logging between printf & TESTPMD_LOG().
>>>> To improve usability here, we can use cmdline_new rather than
>>>> cmdline_file_new and have the output from the various commands sent to
>>>> stdout, allowing the user to see better what is happening.
>>>>
>>>> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
>>>>
>>>> ---
>>>> v2: use STDOUT_FILENO in place of hard-coded "1"
>>>> ---
>>>> app/test-pmd/cmdline.c | 14 +++++++++++++-
>>>> 1 file changed, 13 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
>>>> index b7759e38a8..52e64430d9 100644
>>>> --- a/app/test-pmd/cmdline.c
>>>> +++ b/app/test-pmd/cmdline.c
>>>> @@ -6,6 +6,7 @@
>>>> #include <ctype.h>
>>>> #include <stdarg.h>
>>>> #include <errno.h>
>>>> +#include <fcntl.h>
>>>> #include <stdio.h>
>>>> #include <stdint.h>
>>>> #include <stdlib.h>
>>>> @@ -13431,7 +13432,18 @@ cmdline_read_from_file(const char *filename)
>>>> {
>>>> struct cmdline *cl;
>>>>
>>>> - cl = cmdline_file_new(main_ctx, "testpmd> ", filename);
>>>> + /* cmdline_file_new does not produce any output which is not ideal here.
>>>> + * Much better to show output of the commands, so we open filename directly
>>>> + * and then pass that to cmdline_new with stdout as the output path.
>>>> + */
>>>> + int fd = open(filename, O_RDONLY);
>>>> + if (fd < 0) {
>>>> + fprintf(stderr, "Failed to open file %s: %s\n",
>>>> + filename, strerror(errno));
>>>> + return;
>>>> + }
>>>> +
>>>> + cl = cmdline_new(main_ctx, "testpmd> ", fd, STDOUT_FILENO);
>>>>
>>>
>>> Above is almost save as 'cmdline_file_new()' function with only
>>> difference that it uses '-1' for 's_out'.
>>>
>>> If this usecase may be required by others, do you think does it have a
>>> value to pass 's_out' to 'cmdline_file_new()' or have a new version of
>>> API that accepts 's_out' as parameter?
>>>
>>
>> Yes, I thought about this, and actually started implementing a new API for
>> cmdline library to that. However, I decided that, given the complexity
>> here, that it's not really necessary - especially as there is no clear way
>> to do things. The options are:
>>
>> * extend cmdline_file_new to have a flag to echo to stdout (which would be
>> the very common case here).
>> * extend cmdline_file_new to take a file handle - this is more flexible,
>> but slightly less usable.
>> * add a new cmdline_file_<something>_new function to echo to stdout.
>> * add a new cmdline_file_<something>_new function to write to a filehandle.
>>
>> I don't like breaking the cmdline API (and ABI), so I didn't want to do
>> either #1 or #2, which would be the cleanest solutions. For #3 and #4,
>> naming is hard, and deciding between them is even harder. Given the choice,
>> I prefer #3, as I can't see #4 being very common and we always have
>> cmdline_new as a fallback anyway.
>>
>> Overall, though, I threw away that work, because it didn't seem worth it,
>> for the sake of having the user to do an extra "open" call.
>>
>
I vote to option #1, but agree that it may not worth breaking API and ABI.
Is 'cmdline_file_new_v2()' too bad a name, perhaps better to go with
testpmd implementation, as you did in this patch.
> And also to add:
> If there is clear consensus on what the correct option for this case is,
> I'm happy enough to go back and extend the cmdline library as agreed.
> :-)
^ permalink raw reply [relevance 3%]
* Re: [PATCH v2] app/testpmd: show output of commands read from file
2024-08-22 17:14 3% ` Bruce Richardson
@ 2024-08-22 17:18 0% ` Bruce Richardson
2024-08-22 21:09 3% ` Ferruh Yigit
0 siblings, 1 reply; 200+ results
From: Bruce Richardson @ 2024-08-22 17:18 UTC (permalink / raw)
To: Ferruh Yigit; +Cc: dev
On Thu, Aug 22, 2024 at 06:14:55PM +0100, Bruce Richardson wrote:
> On Thu, Aug 22, 2024 at 05:53:27PM +0100, Ferruh Yigit wrote:
> > On 8/22/2024 11:41 AM, Bruce Richardson wrote:
> > > Testpmd supports the "--cmdline-file" parameter to read a set of initial
> > > commands from a file. However, the only indication that this has been
> > > done successfully on startup is a single-line message, no output from
> > > the commands is seen.
> > >
> >
> > For user I think it makes sense to see the command [1], only concern is
> > if someone parsing testpmd output may be impacted on this, although I
> > expect it should be trivial to update the relevant parsing.
> >
> > [1]
> > Btw, I can still see the command output, I assume because command does
> > the printf itself, for example for 'show port summary 0' command:
> > - before patch:
> > ...
> > Number of available ports: 2
> > Port MAC Address Name Driver Status Link
> > 0 xx:xx:xx:xx:xx:xx xxxx:xx:xx.x aaaaaaaa up xxx Gbps
> > ...
> >
> > - after patch
> > ...
> > testpmd> show port summary 0
> > Number of available ports: 2
> > Port MAC Address Name Driver Status Link
> > 0 xx:xx:xx:xx:xx:xx xxxx:xx:xx.x aaaaaaaa up xxx Gbps
> > ...
> >
> > Only difference above is, after patch the command itself also printed.
> >
> >
>
> That's because the function uses printf itself, which is actually wrong.
> Any output from a cmdline function should use the "cmdline_printf" call
> which outputs to the proper cmdline filehandle.
>
> > > To improve usability here, we can use cmdline_new rather than
> > > cmdline_file_new and have the output from the various commands sent to
> > > stdout, allowing the user to see better what is happening.
> > >
> > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > >
> > > ---
> > > v2: use STDOUT_FILENO in place of hard-coded "1"
> > > ---
> > > app/test-pmd/cmdline.c | 14 +++++++++++++-
> > > 1 file changed, 13 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
> > > index b7759e38a8..52e64430d9 100644
> > > --- a/app/test-pmd/cmdline.c
> > > +++ b/app/test-pmd/cmdline.c
> > > @@ -6,6 +6,7 @@
> > > #include <ctype.h>
> > > #include <stdarg.h>
> > > #include <errno.h>
> > > +#include <fcntl.h>
> > > #include <stdio.h>
> > > #include <stdint.h>
> > > #include <stdlib.h>
> > > @@ -13431,7 +13432,18 @@ cmdline_read_from_file(const char *filename)
> > > {
> > > struct cmdline *cl;
> > >
> > > - cl = cmdline_file_new(main_ctx, "testpmd> ", filename);
> > > + /* cmdline_file_new does not produce any output which is not ideal here.
> > > + * Much better to show output of the commands, so we open filename directly
> > > + * and then pass that to cmdline_new with stdout as the output path.
> > > + */
> > > + int fd = open(filename, O_RDONLY);
> > > + if (fd < 0) {
> > > + fprintf(stderr, "Failed to open file %s: %s\n",
> > > + filename, strerror(errno));
> > > + return;
> > > + }
> > > +
> > > + cl = cmdline_new(main_ctx, "testpmd> ", fd, STDOUT_FILENO);
> > >
> >
> > Above is almost save as 'cmdline_file_new()' function with only
> > difference that it uses '-1' for 's_out'.
> >
> > If this usecase may be required by others, do you think does it have a
> > value to pass 's_out' to 'cmdline_file_new()' or have a new version of
> > API that accepts 's_out' as parameter?
> >
>
> Yes, I thought about this, and actually started implementing a new API for
> cmdline library to that. However, I decided that, given the complexity
> here, that it's not really necessary - especially as there is no clear way
> to do things. The options are:
>
> * extend cmdline_file_new to have a flag to echo to stdout (which would be
> the very common case here).
> * extend cmdline_file_new to take a file handle - this is more flexible,
> but slightly less usable.
> * add a new cmdline_file_<something>_new function to echo to stdout.
> * add a new cmdline_file_<something>_new function to write to a filehandle.
>
> I don't like breaking the cmdline API (and ABI), so I didn't want to do
> either #1 or #2, which would be the cleanest solutions. For #3 and #4,
> naming is hard, and deciding between them is even harder. Given the choice,
> I prefer #3, as I can't see #4 being very common and we always have
> cmdline_new as a fallback anyway.
>
> Overall, though, I threw away that work, because it didn't seem worth it,
> for the sake of having the user to do an extra "open" call.
>
And also to add:
If there is clear consensus on what the correct option for this case is,
I'm happy enough to go back and extend the cmdline library as agreed.
:-)
^ permalink raw reply [relevance 0%]
* Re: [PATCH v2] app/testpmd: show output of commands read from file
@ 2024-08-22 17:14 3% ` Bruce Richardson
2024-08-22 17:18 0% ` Bruce Richardson
0 siblings, 1 reply; 200+ results
From: Bruce Richardson @ 2024-08-22 17:14 UTC (permalink / raw)
To: Ferruh Yigit; +Cc: dev
On Thu, Aug 22, 2024 at 05:53:27PM +0100, Ferruh Yigit wrote:
> On 8/22/2024 11:41 AM, Bruce Richardson wrote:
> > Testpmd supports the "--cmdline-file" parameter to read a set of initial
> > commands from a file. However, the only indication that this has been
> > done successfully on startup is a single-line message, no output from
> > the commands is seen.
> >
>
> For user I think it makes sense to see the command [1], only concern is
> if someone parsing testpmd output may be impacted on this, although I
> expect it should be trivial to update the relevant parsing.
>
> [1]
> Btw, I can still see the command output, I assume because command does
> the printf itself, for example for 'show port summary 0' command:
> - before patch:
> ...
> Number of available ports: 2
> Port MAC Address Name Driver Status Link
> 0 xx:xx:xx:xx:xx:xx xxxx:xx:xx.x aaaaaaaa up xxx Gbps
> ...
>
> - after patch
> ...
> testpmd> show port summary 0
> Number of available ports: 2
> Port MAC Address Name Driver Status Link
> 0 xx:xx:xx:xx:xx:xx xxxx:xx:xx.x aaaaaaaa up xxx Gbps
> ...
>
> Only difference above is, after patch the command itself also printed.
>
>
That's because the function uses printf itself, which is actually wrong.
Any output from a cmdline function should use the "cmdline_printf" call
which outputs to the proper cmdline filehandle.
> > To improve usability here, we can use cmdline_new rather than
> > cmdline_file_new and have the output from the various commands sent to
> > stdout, allowing the user to see better what is happening.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> >
> > ---
> > v2: use STDOUT_FILENO in place of hard-coded "1"
> > ---
> > app/test-pmd/cmdline.c | 14 +++++++++++++-
> > 1 file changed, 13 insertions(+), 1 deletion(-)
> >
> > diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
> > index b7759e38a8..52e64430d9 100644
> > --- a/app/test-pmd/cmdline.c
> > +++ b/app/test-pmd/cmdline.c
> > @@ -6,6 +6,7 @@
> > #include <ctype.h>
> > #include <stdarg.h>
> > #include <errno.h>
> > +#include <fcntl.h>
> > #include <stdio.h>
> > #include <stdint.h>
> > #include <stdlib.h>
> > @@ -13431,7 +13432,18 @@ cmdline_read_from_file(const char *filename)
> > {
> > struct cmdline *cl;
> >
> > - cl = cmdline_file_new(main_ctx, "testpmd> ", filename);
> > + /* cmdline_file_new does not produce any output which is not ideal here.
> > + * Much better to show output of the commands, so we open filename directly
> > + * and then pass that to cmdline_new with stdout as the output path.
> > + */
> > + int fd = open(filename, O_RDONLY);
> > + if (fd < 0) {
> > + fprintf(stderr, "Failed to open file %s: %s\n",
> > + filename, strerror(errno));
> > + return;
> > + }
> > +
> > + cl = cmdline_new(main_ctx, "testpmd> ", fd, STDOUT_FILENO);
> >
>
> Above is almost save as 'cmdline_file_new()' function with only
> difference that it uses '-1' for 's_out'.
>
> If this usecase may be required by others, do you think does it have a
> value to pass 's_out' to 'cmdline_file_new()' or have a new version of
> API that accepts 's_out' as parameter?
>
Yes, I thought about this, and actually started implementing a new API for
cmdline library to that. However, I decided that, given the complexity
here, that it's not really necessary - especially as there is no clear way
to do things. The options are:
* extend cmdline_file_new to have a flag to echo to stdout (which would be
the very common case here).
* extend cmdline_file_new to take a file handle - this is more flexible,
but slightly less usable.
* add a new cmdline_file_<something>_new function to echo to stdout.
* add a new cmdline_file_<something>_new function to write to a filehandle.
I don't like breaking the cmdline API (and ABI), so I didn't want to do
either #1 or #2, which would be the cleanest solutions. For #3 and #4,
naming is hard, and deciding between them is even harder. Given the choice,
I prefer #3, as I can't see #4 being very common and we always have
cmdline_new as a fallback anyway.
Overall, though, I threw away that work, because it didn't seem worth it,
for the sake of having the user to do an extra "open" call.
> btw, I recognized that 'cmdline' library doesn't have doxygen
> documentation, which is a gap to address. Next time when someone asks
> for entry level task, we can point this one.
>
Yep, good idea.
/Bruce
^ permalink raw reply [relevance 3%]
* [PATCH v19 5/5] dts: add API doc generation
@ 2024-08-21 15:02 2% ` Juraj Linkeš
0 siblings, 0 replies; 200+ results
From: Juraj Linkeš @ 2024-08-21 15:02 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, jspewock, probb, paul.szczepanek,
Luca.Vizzarro, npratte, dmarx, alex.chapman
Cc: dev, Juraj Linkeš, Bruce Richardson
The tool used to generate DTS API docs is Sphinx, which is already in
use in DPDK. The same configuration is used to preserve style with one
DTS-specific configuration (so that the DPDK docs are unchanged) that
modifies how the sidebar displays the content. There's other Sphinx
configuration related to Python docstrings which doesn't affect DPDK doc
build. All new configuration is in a conditional block, applied only
when DTS API docs are built to not interfere with DPDK doc build.
Sphinx generates the documentation from Python docstrings. The docstring
format is the Google format [0] which requires the sphinx.ext.napoleon
extension. The other extension, sphinx.ext.intersphinx, enables linking
to objects in external documentations, such as the Python documentation.
There is one requirement for building DTS docs - the same Python version
as DTS or higher, because Sphinx's autodoc extension imports the code.
The dependencies needed to import the code don't have to be satisfied,
as the autodoc extension allows us to mock the imports. The missing
packages are taken from the DTS pyproject.toml file.
And finally, the DTS API docs can be accessed from the DPDK API doxygen
page.
[0] https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings
Cc: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Dean Marx <dmarx@iol.unh.edu>
---
buildtools/call-sphinx-build.py | 2 +
buildtools/get-dts-runtime-deps.py | 95 +++++++++++++++++++++++
buildtools/meson.build | 1 +
doc/api/doxy-api-index.md | 3 +
doc/api/doxy-api.conf.in | 2 +
doc/api/dts/custom.css | 1 +
doc/api/dts/meson.build | 31 ++++++++
doc/api/meson.build | 6 +-
doc/guides/conf.py | 44 ++++++++++-
doc/guides/contributing/documentation.rst | 2 +
doc/guides/contributing/patches.rst | 4 +
doc/guides/tools/dts.rst | 39 +++++++++-
doc/meson.build | 1 +
13 files changed, 228 insertions(+), 3 deletions(-)
create mode 100755 buildtools/get-dts-runtime-deps.py
create mode 120000 doc/api/dts/custom.css
create mode 100644 doc/api/dts/meson.build
diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
index 623e7363ee..154e9f907b 100755
--- a/buildtools/call-sphinx-build.py
+++ b/buildtools/call-sphinx-build.py
@@ -15,6 +15,8 @@
# set the version in environment for sphinx to pick up
os.environ['DPDK_VERSION'] = version
+if 'dts' in src:
+ os.environ['DTS_BUILD'] = "y"
sphinx_cmd = [sphinx] + extra_args
diff --git a/buildtools/get-dts-runtime-deps.py b/buildtools/get-dts-runtime-deps.py
new file mode 100755
index 0000000000..1636a6dbf5
--- /dev/null
+++ b/buildtools/get-dts-runtime-deps.py
@@ -0,0 +1,95 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 PANTHEON.tech s.r.o.
+#
+
+"""Utilities for DTS dependencies.
+
+The module can be used as an executable script,
+which verifies that the running Python version meets the version requirement of DTS.
+The script exits with the standard exit codes in this mode (0 is success, 1 is failure).
+
+The module also contains a function, get_missing_imports,
+which looks for runtime dependencies in the DTS pyproject.toml file
+and returns a list of module names used in an import statement (import packages) that are missing.
+This function is not used when the module is run as a script and is available to be imported.
+"""
+
+import configparser
+import importlib.metadata
+import importlib.util
+import os.path
+import platform
+
+from packaging.version import Version
+
+_VERSION_COMPARISON_CHARS = '^<>='
+_EXTRA_DEPS = {
+ 'invoke': {'version': '>=1.3'},
+ 'paramiko': {'version': '>=2.4'},
+ 'PyYAML': {'version': '^6.0', 'import_package': 'yaml'}
+}
+_DPDK_ROOT = os.path.dirname(os.path.dirname(__file__))
+_DTS_DEP_FILE_PATH = os.path.join(_DPDK_ROOT, 'dts', 'pyproject.toml')
+
+
+def _get_dependencies(cfg_file_path):
+ cfg = configparser.ConfigParser()
+ with open(cfg_file_path) as f:
+ dts_deps_file_str = f.read()
+ dts_deps_file_str = dts_deps_file_str.replace("\n]", "]")
+ cfg.read_string(dts_deps_file_str)
+
+ deps_section = cfg['tool.poetry.dependencies']
+ return {dep: {'version': deps_section[dep].strip('"\'')} for dep in deps_section}
+
+
+def get_missing_imports():
+ """Get missing DTS import packages from third party libraries.
+
+ Scan the DTS pyproject.toml file for dependencies and find those that are not installed.
+ The dependencies in pyproject.toml are listed by their distribution package names,
+ but the function finds the associated import packages - those used in import statements.
+
+ The function is not used when the module is run as a script. It should be imported.
+
+ Returns:
+ A list of missing import packages.
+ """
+ missing_imports = []
+ req_deps = _get_dependencies(_DTS_DEP_FILE_PATH)
+ req_deps.pop('python')
+
+ for req_dep, dep_data in (req_deps | _EXTRA_DEPS).items():
+ req_ver = dep_data['version']
+ try:
+ import_package = dep_data['import_package']
+ except KeyError:
+ import_package = req_dep
+ import_package = import_package.lower().replace('-', '_')
+
+ try:
+ req_ver = Version(req_ver.strip(_VERSION_COMPARISON_CHARS))
+ found_dep_ver = Version(importlib.metadata.version(req_dep))
+ if found_dep_ver < req_ver:
+ print(
+ f'The version "{found_dep_ver}" of package "{req_dep}" '
+ f'is lower than required "{req_ver}".'
+ )
+ except importlib.metadata.PackageNotFoundError:
+ print(f'Package "{req_dep}" not found.')
+ missing_imports.append(import_package)
+
+ return missing_imports
+
+
+if __name__ == '__main__':
+ python_version = _get_dependencies(_DTS_DEP_FILE_PATH).pop('python')
+ if python_version:
+ sys_ver = Version(platform.python_version())
+ req_ver = Version(python_version['version'].strip(_VERSION_COMPARISON_CHARS))
+ if sys_ver < req_ver:
+ print(
+ f'The available Python version "{sys_ver}" is lower than required "{req_ver}".'
+ )
+ exit(1)
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 3adf34e1a8..6b938d767c 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -24,6 +24,7 @@ get_numa_count_cmd = py3 + files('get-numa-count.py')
get_test_suites_cmd = py3 + files('get-test-suites.py')
has_hugepages_cmd = py3 + files('has-hugepages.py')
cmdline_gen_cmd = py3 + files('dpdk-cmdline-gen.py')
+get_dts_runtime_deps = py3 + files('get-dts-runtime-deps.py')
# install any build tools that end-users might want also
install_data([
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index f9f0300126..ab223bcdf7 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -245,3 +245,6 @@ The public API headers are grouped by topics:
[experimental APIs](@ref rte_compat.h),
[ABI versioning](@ref rte_function_versioning.h),
[version](@ref rte_version.h)
+
+- **tests**:
+ [**DTS**](@dts_api_main_page)
diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index a8823c046f..c94f02d411 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -124,6 +124,8 @@ SEARCHENGINE = YES
SORT_MEMBER_DOCS = NO
SOURCE_BROWSER = YES
+ALIASES = "dts_api_main_page=@DTS_API_MAIN_PAGE@"
+
EXAMPLE_PATH = @TOPDIR@/examples
EXAMPLE_PATTERNS = *.c
EXAMPLE_RECURSIVE = YES
diff --git a/doc/api/dts/custom.css b/doc/api/dts/custom.css
new file mode 120000
index 0000000000..3c9480c4a0
--- /dev/null
+++ b/doc/api/dts/custom.css
@@ -0,0 +1 @@
+../../guides/custom.css
\ No newline at end of file
diff --git a/doc/api/dts/meson.build b/doc/api/dts/meson.build
new file mode 100644
index 0000000000..f338eb69bf
--- /dev/null
+++ b/doc/api/dts/meson.build
@@ -0,0 +1,31 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
+if not sphinx.found()
+ subdir_done()
+endif
+
+python_ver_satisfied = run_command(get_dts_runtime_deps, check: false).returncode()
+if python_ver_satisfied != 0
+ subdir_done()
+endif
+
+cdata.set('DTS_API_MAIN_PAGE', join_paths('..', 'dts', 'html', 'index.html'))
+
+extra_sphinx_args = ['-E', '-c', join_paths(doc_source_dir, 'guides')]
+if get_option('werror')
+ extra_sphinx_args += '-W'
+endif
+
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk', 'dts')
+dts_api_html = custom_target('dts_api_html',
+ output: 'html',
+ command: [sphinx_wrapper, sphinx, meson.project_version(),
+ meson.current_source_dir(), meson.current_build_dir(), extra_sphinx_args],
+ build_by_default: get_option('enable_docs'),
+ install: get_option('enable_docs'),
+ install_dir: htmldir)
+
+doc_targets += dts_api_html
+doc_target_names += 'DTS_API_HTML'
diff --git a/doc/api/meson.build b/doc/api/meson.build
index 5b50692df9..71b861e42b 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -1,6 +1,11 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Luca Boccassi <bluca@debian.org>
+# initialize common Doxygen configuration
+cdata = configuration_data()
+
+subdir('dts')
+
doxygen = find_program('doxygen', required: get_option('enable_docs'))
if not doxygen.found()
@@ -30,7 +35,6 @@ example = custom_target('examples.dox',
build_by_default: get_option('enable_docs'))
# set up common Doxygen configuration
-cdata = configuration_data()
cdata.set('VERSION', meson.project_version())
cdata.set('API_EXAMPLES', join_paths(dpdk_build_root, 'doc', 'api', 'examples.dox'))
cdata.set('OUTPUT', join_paths(dpdk_build_root, 'doc', 'api'))
diff --git a/doc/guides/conf.py b/doc/guides/conf.py
index 0f7ff5282d..d7f3030838 100644
--- a/doc/guides/conf.py
+++ b/doc/guides/conf.py
@@ -10,7 +10,7 @@
from os.path import basename
from os.path import dirname
from os.path import join as path_join
-from sys import argv, stderr
+from sys import argv, stderr, path
import configparser
@@ -58,6 +58,48 @@
("tools/devbind", "dpdk-devbind",
"check device status and bind/unbind them from drivers", "", 8)]
+# DTS API docs additional configuration
+if environ.get('DTS_BUILD'):
+ extensions = ['sphinx.ext.napoleon', 'sphinx.ext.autodoc', 'sphinx.ext.intersphinx']
+ # Napoleon enables the Google format of Python doscstrings.
+ napoleon_numpy_docstring = False
+ napoleon_attr_annotations = True
+ napoleon_preprocess_types = True
+
+ # Autodoc pulls documentation from code.
+ autodoc_default_options = {
+ 'members': True,
+ 'member-order': 'bysource',
+ 'show-inheritance': True,
+ }
+ autodoc_class_signature = 'separated'
+ autodoc_typehints = 'both'
+ autodoc_typehints_format = 'short'
+ autodoc_typehints_description_target = 'documented'
+
+ # Intersphinx allows linking to external projects, such as Python docs.
+ intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
+
+ # DTS docstring options.
+ add_module_names = False
+ toc_object_entries = True
+ toc_object_entries_show_parents = 'hide'
+ # DTS Sidebar config.
+ html_theme_options = {
+ 'collapse_navigation': False,
+ 'navigation_depth': -1, # unlimited depth
+ }
+
+ # Add path to DTS sources so that Sphinx can find them.
+ dpdk_root = dirname(dirname(dirname(__file__)))
+ path.append(path_join(dpdk_root, 'dts'))
+
+ # Get missing DTS dependencies. Add path to buildtools to find the get_missing_imports function.
+ path.append(path_join(dpdk_root, 'buildtools'))
+ import importlib
+ # Ignore missing imports from DTS dependencies.
+ autodoc_mock_imports = importlib.import_module('get-dts-runtime-deps').get_missing_imports()
+
# ####### :numref: fallback ########
# The following hook functions add some simple handling for the :numref:
diff --git a/doc/guides/contributing/documentation.rst b/doc/guides/contributing/documentation.rst
index 68454ae0d5..7b287ce631 100644
--- a/doc/guides/contributing/documentation.rst
+++ b/doc/guides/contributing/documentation.rst
@@ -133,6 +133,8 @@ added to by the developer.
Building the Documentation
--------------------------
+.. _doc_dependencies:
+
Dependencies
~~~~~~~~~~~~
diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst
index 04c66bebc4..6629928bee 100644
--- a/doc/guides/contributing/patches.rst
+++ b/doc/guides/contributing/patches.rst
@@ -499,6 +499,10 @@ The script usage is::
For both of the above scripts, the -n option is used to specify a number of commits from HEAD,
and the -r option allows the user specify a ``git log`` range.
+Additionally, when contributing to the DTS tool, patches should also be checked using
+the ``dts-check-format.sh`` script in the ``devtools`` directory of the DPDK repo.
+To run the script, extra :ref:`Python dependencies <dts_deps>` are needed.
+
.. _contrib_check_compilation:
Checking Compilation
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 515b15e4d8..9e8929f567 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -54,6 +54,7 @@ DTS uses Poetry as its Python dependency management.
Python build/development and runtime environments are the same and DTS development environment,
DTS runtime environment or just plain DTS environment are used interchangeably.
+.. _dts_deps:
Setting up DTS environment
~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -291,8 +292,15 @@ When adding code to the DTS framework, pay attention to the rest of the code
and try not to divert much from it.
The :ref:`DTS developer tools <dts_dev_tools>` will issue warnings
when some of the basics are not met.
+You should also build the :ref:`API documentation <building_api_docs>`
+to address any issues found during the build.
-The code must be properly documented with docstrings.
+The API documentation, which is a helpful reference when developing, may be accessed
+in the code directly or generated with the :ref:`API docs build steps <building_api_docs>`.
+When adding new files or modifying the directory structure,
+the corresponding changes must be made to DTS api doc sources in ``doc/api/dts``.
+
+Speaking of which, the code must be properly documented with docstrings.
The style must conform to the `Google style
<https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings>`_.
See an example of the style `here
@@ -427,6 +435,35 @@ the DTS code check and format script.
Refer to the script for usage: ``devtools/dts-check-format.sh -h``.
+.. _building_api_docs:
+
+Building DTS API docs
+---------------------
+
+The documentation is built using the standard DPDK build system.
+See :doc:`../linux_gsg/build_dpdk` for more details on compiling DPDK with meson.
+
+The :ref:`doc build dependencies <doc_dependencies>` may be installed with Poetry:
+
+.. code-block:: console
+
+ poetry install --no-root --only docs
+ poetry install --no-root --with docs # an alternative that will also install DTS dependencies
+ poetry shell
+
+After executing the meson command, build the documentation with:
+
+.. code-block:: console
+
+ ninja -C build doc
+
+The output is generated in ``build/doc/api/dts/html``.
+
+.. note::
+
+ Make sure to fix any Sphinx warnings when adding or updating docstrings.
+
+
Configuration Schema
--------------------
diff --git a/doc/meson.build b/doc/meson.build
index 6f74706aa2..1e0cfa4127 100644
--- a/doc/meson.build
+++ b/doc/meson.build
@@ -1,6 +1,7 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Luca Boccassi <bluca@debian.org>
+doc_source_dir = meson.current_source_dir()
doc_targets = []
doc_target_names = []
subdir('api')
--
2.34.1
^ permalink raw reply [relevance 2%]
* [PATCH v18 5/5] dts: add API doc generation
@ 2024-08-20 13:18 2% ` Juraj Linkeš
0 siblings, 0 replies; 200+ results
From: Juraj Linkeš @ 2024-08-20 13:18 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, bruce.richardson, jspewock, probb,
paul.szczepanek, Luca.Vizzarro, npratte
Cc: dev, Juraj Linkeš, Dean Marx
The tool used to generate DTS API docs is Sphinx, which is already in
use in DPDK. The same configuration is used to preserve style with one
DTS-specific configuration (so that the DPDK docs are unchanged) that
modifies how the sidebar displays the content. There's other Sphinx
configuration related to Python docstrings which doesn't affect DPDK doc
build. All new configuration is in a conditional block, applied only
when DTS API docs are built to not interfere with DPDK doc build.
Sphinx generates the documentation from Python docstrings. The docstring
format is the Google format [0] which requires the sphinx.ext.napoleon
extension. The other extension, sphinx.ext.intersphinx, enables linking
to objects in external documentations, such as the Python documentation.
There is one requirement for building DTS docs - the same Python version
as DTS or higher, because Sphinx's autodoc extension imports the code.
The dependencies needed to import the code don't have to be satisfied,
as the autodoc extension allows us to mock the imports. The missing
packages are taken from the DTS pyproject.toml file.
And finally, the DTS API docs can be accessed from the DPDK API doxygen
page.
[0] https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Reviewed-by: Dean Marx <dmarx@iol.unh.edu>
---
buildtools/call-sphinx-build.py | 2 +
buildtools/get-dts-runtime-deps.py | 95 +++++++++++++++++++++++
buildtools/meson.build | 1 +
doc/api/doxy-api-index.md | 3 +
doc/api/doxy-api.conf.in | 2 +
doc/api/dts/custom.css | 1 +
doc/api/dts/meson.build | 31 ++++++++
doc/api/meson.build | 6 +-
doc/guides/conf.py | 44 ++++++++++-
doc/guides/contributing/documentation.rst | 2 +
doc/guides/contributing/patches.rst | 4 +
doc/guides/tools/dts.rst | 39 +++++++++-
doc/meson.build | 1 +
13 files changed, 228 insertions(+), 3 deletions(-)
create mode 100755 buildtools/get-dts-runtime-deps.py
create mode 120000 doc/api/dts/custom.css
create mode 100644 doc/api/dts/meson.build
diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
index 623e7363ee..154e9f907b 100755
--- a/buildtools/call-sphinx-build.py
+++ b/buildtools/call-sphinx-build.py
@@ -15,6 +15,8 @@
# set the version in environment for sphinx to pick up
os.environ['DPDK_VERSION'] = version
+if 'dts' in src:
+ os.environ['DTS_BUILD'] = "y"
sphinx_cmd = [sphinx] + extra_args
diff --git a/buildtools/get-dts-runtime-deps.py b/buildtools/get-dts-runtime-deps.py
new file mode 100755
index 0000000000..6f4d3def29
--- /dev/null
+++ b/buildtools/get-dts-runtime-deps.py
@@ -0,0 +1,95 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 PANTHEON.tech s.r.o.
+#
+
+"""Utilities for DTS dependencies.
+
+The module can be used as an executable script,
+which verifies that the running Python version meets the version requirement of DTS.
+The script exits with the standard exit codes in this mode (0 is success, 1 is failure).
+
+The module also contains a function, get_missing_imports,
+which looks for runtime dependencies in the DTS pyproject.toml file
+and returns a list of module names used in an import statement (import packages) that are missing.
+This function is not used when the module is run as a script and is available to be imported.
+"""
+
+import configparser
+import importlib.metadata
+import importlib.util
+import os.path
+import platform
+
+from packaging.version import Version
+
+_VERSION_COMPARISON_CHARS = '^<>='
+_EXTRA_DEPS = {
+ 'invoke': {'version': '>=1.3'},
+ 'paramiko': {'version': '>=2.4'},
+ 'PyYAML': {'version': '^6.0', 'import_package': 'yaml'}
+}
+_DPDK_ROOT = os.path.dirname(os.path.dirname(__file__))
+_DTS_DEP_FILE_PATH = os.path.join(_DPDK_ROOT, 'dts', 'pyproject.toml')
+
+
+def _get_dependencies(cfg_file_path):
+ cfg = configparser.ConfigParser()
+ with open(cfg_file_path) as f:
+ dts_deps_file_str = f.read()
+ dts_deps_file_str = dts_deps_file_str.replace("\n]", "]")
+ cfg.read_string(dts_deps_file_str)
+
+ deps_section = cfg['tool.poetry.dependencies']
+ return {dep: {'version': deps_section[dep].strip('"\'')} for dep in deps_section}
+
+
+def get_missing_imports():
+ """Get missing DTS import packages from third party libraries.
+
+ Scan the DTS pyproject.toml file for dependencies and find those that are not installed.
+ The dependencies in pyproject.toml are listed by their distribution package names,
+ but the function finds the associated import packages - those used in import statements.
+
+ The function is not used when the module is run as a script. It should be imported.
+
+ Returns:
+ A list of missing import packages.
+ """
+ missing_imports = []
+ req_deps = _get_dependencies(_DTS_DEP_FILE_PATH)
+ req_deps.pop('python')
+
+ for req_dep, dep_data in (req_deps | _EXTRA_DEPS).items():
+ req_ver = dep_data['version']
+ try:
+ import_package = dep_data['import_package']
+ except KeyError:
+ import_package = req_dep
+ import_package = import_package.lower().replace('-', '_')
+
+ try:
+ req_ver = Version(req_ver.strip(_VERSION_COMPARISON_CHARS))
+ found_dep_ver = Version(importlib.metadata.version(req_dep))
+ if found_dep_ver < req_ver:
+ print(
+ f'The version "{found_dep_ver}" of package "{req_dep}" '
+ f'is lower than required "{req_ver}".'
+ )
+ except importlib.metadata.PackageNotFoundError:
+ print(f'Package "{req_dep}" not found.')
+ missing_imports.append(import_package)
+
+ return missing_imports
+
+
+if __name__ == '__main__':
+ python_version = _get_dependencies(_DTS_DEP_FILE_PATH).pop('python')
+ if python_version:
+ sys_ver = Version(platform.python_version())
+ req_ver = Version(python_version.strip(_VERSION_COMPARISON_CHARS))
+ if sys_ver < req_ver:
+ print(
+ f'The available Python version "{sys_ver}" is lower than required "{req_ver}".'
+ )
+ exit(1)
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 3adf34e1a8..6b938d767c 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -24,6 +24,7 @@ get_numa_count_cmd = py3 + files('get-numa-count.py')
get_test_suites_cmd = py3 + files('get-test-suites.py')
has_hugepages_cmd = py3 + files('has-hugepages.py')
cmdline_gen_cmd = py3 + files('dpdk-cmdline-gen.py')
+get_dts_runtime_deps = py3 + files('get-dts-runtime-deps.py')
# install any build tools that end-users might want also
install_data([
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index f9f0300126..ab223bcdf7 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -245,3 +245,6 @@ The public API headers are grouped by topics:
[experimental APIs](@ref rte_compat.h),
[ABI versioning](@ref rte_function_versioning.h),
[version](@ref rte_version.h)
+
+- **tests**:
+ [**DTS**](@dts_api_main_page)
diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index a8823c046f..c94f02d411 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -124,6 +124,8 @@ SEARCHENGINE = YES
SORT_MEMBER_DOCS = NO
SOURCE_BROWSER = YES
+ALIASES = "dts_api_main_page=@DTS_API_MAIN_PAGE@"
+
EXAMPLE_PATH = @TOPDIR@/examples
EXAMPLE_PATTERNS = *.c
EXAMPLE_RECURSIVE = YES
diff --git a/doc/api/dts/custom.css b/doc/api/dts/custom.css
new file mode 120000
index 0000000000..3c9480c4a0
--- /dev/null
+++ b/doc/api/dts/custom.css
@@ -0,0 +1 @@
+../../guides/custom.css
\ No newline at end of file
diff --git a/doc/api/dts/meson.build b/doc/api/dts/meson.build
new file mode 100644
index 0000000000..f338eb69bf
--- /dev/null
+++ b/doc/api/dts/meson.build
@@ -0,0 +1,31 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
+if not sphinx.found()
+ subdir_done()
+endif
+
+python_ver_satisfied = run_command(get_dts_runtime_deps, check: false).returncode()
+if python_ver_satisfied != 0
+ subdir_done()
+endif
+
+cdata.set('DTS_API_MAIN_PAGE', join_paths('..', 'dts', 'html', 'index.html'))
+
+extra_sphinx_args = ['-E', '-c', join_paths(doc_source_dir, 'guides')]
+if get_option('werror')
+ extra_sphinx_args += '-W'
+endif
+
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk', 'dts')
+dts_api_html = custom_target('dts_api_html',
+ output: 'html',
+ command: [sphinx_wrapper, sphinx, meson.project_version(),
+ meson.current_source_dir(), meson.current_build_dir(), extra_sphinx_args],
+ build_by_default: get_option('enable_docs'),
+ install: get_option('enable_docs'),
+ install_dir: htmldir)
+
+doc_targets += dts_api_html
+doc_target_names += 'DTS_API_HTML'
diff --git a/doc/api/meson.build b/doc/api/meson.build
index 5b50692df9..71b861e42b 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -1,6 +1,11 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Luca Boccassi <bluca@debian.org>
+# initialize common Doxygen configuration
+cdata = configuration_data()
+
+subdir('dts')
+
doxygen = find_program('doxygen', required: get_option('enable_docs'))
if not doxygen.found()
@@ -30,7 +35,6 @@ example = custom_target('examples.dox',
build_by_default: get_option('enable_docs'))
# set up common Doxygen configuration
-cdata = configuration_data()
cdata.set('VERSION', meson.project_version())
cdata.set('API_EXAMPLES', join_paths(dpdk_build_root, 'doc', 'api', 'examples.dox'))
cdata.set('OUTPUT', join_paths(dpdk_build_root, 'doc', 'api'))
diff --git a/doc/guides/conf.py b/doc/guides/conf.py
index 0f7ff5282d..d7f3030838 100644
--- a/doc/guides/conf.py
+++ b/doc/guides/conf.py
@@ -10,7 +10,7 @@
from os.path import basename
from os.path import dirname
from os.path import join as path_join
-from sys import argv, stderr
+from sys import argv, stderr, path
import configparser
@@ -58,6 +58,48 @@
("tools/devbind", "dpdk-devbind",
"check device status and bind/unbind them from drivers", "", 8)]
+# DTS API docs additional configuration
+if environ.get('DTS_BUILD'):
+ extensions = ['sphinx.ext.napoleon', 'sphinx.ext.autodoc', 'sphinx.ext.intersphinx']
+ # Napoleon enables the Google format of Python doscstrings.
+ napoleon_numpy_docstring = False
+ napoleon_attr_annotations = True
+ napoleon_preprocess_types = True
+
+ # Autodoc pulls documentation from code.
+ autodoc_default_options = {
+ 'members': True,
+ 'member-order': 'bysource',
+ 'show-inheritance': True,
+ }
+ autodoc_class_signature = 'separated'
+ autodoc_typehints = 'both'
+ autodoc_typehints_format = 'short'
+ autodoc_typehints_description_target = 'documented'
+
+ # Intersphinx allows linking to external projects, such as Python docs.
+ intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
+
+ # DTS docstring options.
+ add_module_names = False
+ toc_object_entries = True
+ toc_object_entries_show_parents = 'hide'
+ # DTS Sidebar config.
+ html_theme_options = {
+ 'collapse_navigation': False,
+ 'navigation_depth': -1, # unlimited depth
+ }
+
+ # Add path to DTS sources so that Sphinx can find them.
+ dpdk_root = dirname(dirname(dirname(__file__)))
+ path.append(path_join(dpdk_root, 'dts'))
+
+ # Get missing DTS dependencies. Add path to buildtools to find the get_missing_imports function.
+ path.append(path_join(dpdk_root, 'buildtools'))
+ import importlib
+ # Ignore missing imports from DTS dependencies.
+ autodoc_mock_imports = importlib.import_module('get-dts-runtime-deps').get_missing_imports()
+
# ####### :numref: fallback ########
# The following hook functions add some simple handling for the :numref:
diff --git a/doc/guides/contributing/documentation.rst b/doc/guides/contributing/documentation.rst
index 68454ae0d5..7b287ce631 100644
--- a/doc/guides/contributing/documentation.rst
+++ b/doc/guides/contributing/documentation.rst
@@ -133,6 +133,8 @@ added to by the developer.
Building the Documentation
--------------------------
+.. _doc_dependencies:
+
Dependencies
~~~~~~~~~~~~
diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst
index 04c66bebc4..6629928bee 100644
--- a/doc/guides/contributing/patches.rst
+++ b/doc/guides/contributing/patches.rst
@@ -499,6 +499,10 @@ The script usage is::
For both of the above scripts, the -n option is used to specify a number of commits from HEAD,
and the -r option allows the user specify a ``git log`` range.
+Additionally, when contributing to the DTS tool, patches should also be checked using
+the ``dts-check-format.sh`` script in the ``devtools`` directory of the DPDK repo.
+To run the script, extra :ref:`Python dependencies <dts_deps>` are needed.
+
.. _contrib_check_compilation:
Checking Compilation
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 515b15e4d8..9e8929f567 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -54,6 +54,7 @@ DTS uses Poetry as its Python dependency management.
Python build/development and runtime environments are the same and DTS development environment,
DTS runtime environment or just plain DTS environment are used interchangeably.
+.. _dts_deps:
Setting up DTS environment
~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -291,8 +292,15 @@ When adding code to the DTS framework, pay attention to the rest of the code
and try not to divert much from it.
The :ref:`DTS developer tools <dts_dev_tools>` will issue warnings
when some of the basics are not met.
+You should also build the :ref:`API documentation <building_api_docs>`
+to address any issues found during the build.
-The code must be properly documented with docstrings.
+The API documentation, which is a helpful reference when developing, may be accessed
+in the code directly or generated with the :ref:`API docs build steps <building_api_docs>`.
+When adding new files or modifying the directory structure,
+the corresponding changes must be made to DTS api doc sources in ``doc/api/dts``.
+
+Speaking of which, the code must be properly documented with docstrings.
The style must conform to the `Google style
<https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings>`_.
See an example of the style `here
@@ -427,6 +435,35 @@ the DTS code check and format script.
Refer to the script for usage: ``devtools/dts-check-format.sh -h``.
+.. _building_api_docs:
+
+Building DTS API docs
+---------------------
+
+The documentation is built using the standard DPDK build system.
+See :doc:`../linux_gsg/build_dpdk` for more details on compiling DPDK with meson.
+
+The :ref:`doc build dependencies <doc_dependencies>` may be installed with Poetry:
+
+.. code-block:: console
+
+ poetry install --no-root --only docs
+ poetry install --no-root --with docs # an alternative that will also install DTS dependencies
+ poetry shell
+
+After executing the meson command, build the documentation with:
+
+.. code-block:: console
+
+ ninja -C build doc
+
+The output is generated in ``build/doc/api/dts/html``.
+
+.. note::
+
+ Make sure to fix any Sphinx warnings when adding or updating docstrings.
+
+
Configuration Schema
--------------------
diff --git a/doc/meson.build b/doc/meson.build
index 6f74706aa2..1e0cfa4127 100644
--- a/doc/meson.build
+++ b/doc/meson.build
@@ -1,6 +1,7 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Luca Boccassi <bluca@debian.org>
+doc_source_dir = meson.current_source_dir()
doc_targets = []
doc_target_names = []
subdir('api')
--
2.34.1
^ permalink raw reply [relevance 2%]
* Re: [dpdk-dev] [PATCH v3 5/5] devtools: test different build types
@ 2024-08-15 16:26 0% ` Stephen Hemminger
0 siblings, 0 replies; 200+ results
From: Stephen Hemminger @ 2024-08-15 16:26 UTC (permalink / raw)
To: Thomas Monjalon; +Cc: dev, bruce.richardson, david.marchand, Andrew Rybchenko
On Sun, 8 Aug 2021 14:51:38 +0200
Thomas Monjalon <thomas@monjalon.net> wrote:
> All builds were of type debugoptimized.
> It is kept only for builds having an ABI check.
> Others will have the default build type (release),
> except if specified differently as in the x86 generic build
> which will be a test of the non-optimized debug build type.
> Some static builds will test the minsize build type.
>
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
>
> ---
>
> This patch cannot be merged now because it makes clang 11.1.0 crashing.
> ---
Dropping this patch from patchwork because of the clang crash.
^ permalink raw reply [relevance 0%]
* [RFC 6/6] ring: minimize reads of the counterpart cache-line
@ 2024-08-15 8:53 2% ` Konstantin Ananyev
0 siblings, 0 replies; 200+ results
From: Konstantin Ananyev @ 2024-08-15 8:53 UTC (permalink / raw)
To: dev
Cc: honnappa.nagarahalli, jerinj, hemant.agrawal, bruce.richardson,
drc, ruifeng.wang, mb, Konstantin Ananyev
From: Konstantin Ananyev <konstantin.ananyev@huawei.com>
Note upfront: this change shouldn't affect rte_ring public API.
Though as layout of public structures have changed - it is an ABI
breakage.
This is an attempt to implement rte_ring optimization
that was suggested by Morten and discussed on this mailing
list a while ago.
The idea is to optimize MP/SP & MC/SC ring enqueue/dequeue ops
by storing along with the head its Cached Foreign Tail (CFT) value.
I.E.: for producer we cache consumer tail value and visa-versa.
To avoid races head and CFT values are read/written using atomic
64-bit ops.
In theory that might help by reducing number of times producer
needs to access consumer's cache-line and visa-versa.
In practice, I didn't see any impressive boost so far:
- ring_per_autotest micro-bench - results are a mixed bag,
Some are a bit better, some are worse.
- [so]ring_stress_autotest micro-benchs: ~10-15% improvement
- l3fwd in wqorder/wqundorder (see previous patch for details):
no real difference.
Though so far my testing scope was quite limited, I tried it only
on x86 machines. So can I ask all interested parties:
different platform vendors (ARM, PPC, etc.)
and people who do use rte_ring extensively to give it a try and come up
with the feedback.
If there would be no real performance improvements on
any platform we support, or some problems will be encountered -
I am ok to drop that patch.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@huawei.com>
---
drivers/net/mlx5/mlx5_hws_cnt.h | 5 ++--
drivers/net/ring/rte_eth_ring.c | 2 +-
lib/ring/rte_ring.c | 6 ++--
lib/ring/rte_ring_core.h | 12 +++++++-
lib/ring/rte_ring_generic_pvt.h | 46 +++++++++++++++++++++----------
lib/ring/rte_ring_peek_elem_pvt.h | 4 +--
lib/ring/soring.c | 31 +++++++++++++++------
lib/ring/soring.h | 4 +--
8 files changed, 77 insertions(+), 33 deletions(-)
diff --git a/drivers/net/mlx5/mlx5_hws_cnt.h b/drivers/net/mlx5/mlx5_hws_cnt.h
index 996ac8dd9a..663146563c 100644
--- a/drivers/net/mlx5/mlx5_hws_cnt.h
+++ b/drivers/net/mlx5/mlx5_hws_cnt.h
@@ -388,11 +388,12 @@ __mlx5_hws_cnt_pool_enqueue_revert(struct rte_ring *r, unsigned int n,
MLX5_ASSERT(r->prod.sync_type == RTE_RING_SYNC_ST);
MLX5_ASSERT(r->cons.sync_type == RTE_RING_SYNC_ST);
- current_head = rte_atomic_load_explicit(&r->prod.head, rte_memory_order_relaxed);
+ current_head = rte_atomic_load_explicit(&r->prod.head.val.pos,
+ rte_memory_order_relaxed);
MLX5_ASSERT(n <= r->capacity);
MLX5_ASSERT(n <= rte_ring_count(r));
revert2head = current_head - n;
- r->prod.head = revert2head; /* This ring should be SP. */
+ r->prod.head.val.pos = revert2head; /* This ring should be SP. */
__rte_ring_get_elem_addr(r, revert2head, sizeof(cnt_id_t), n,
&zcd->ptr1, &zcd->n1, &zcd->ptr2);
/* Update tail */
diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index 1346a0dba3..31009e90d2 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -325,7 +325,7 @@ eth_get_monitor_addr(void *rx_queue, struct rte_power_monitor_cond *pmc)
*/
pmc->addr = &rng->prod.head;
pmc->size = sizeof(rng->prod.head);
- pmc->opaque[0] = rng->prod.head;
+ pmc->opaque[0] = rng->prod.head.val.pos;
pmc->fn = ring_monitor_callback;
return 0;
}
diff --git a/lib/ring/rte_ring.c b/lib/ring/rte_ring.c
index aebb6d6728..cb2c39c7ad 100644
--- a/lib/ring/rte_ring.c
+++ b/lib/ring/rte_ring.c
@@ -102,7 +102,7 @@ reset_headtail(void *p)
switch (ht->sync_type) {
case RTE_RING_SYNC_MT:
case RTE_RING_SYNC_ST:
- ht->head = 0;
+ ht->head.raw = 0;
ht->tail = 0;
break;
case RTE_RING_SYNC_MT_RTS:
@@ -373,9 +373,9 @@ rte_ring_dump(FILE *f, const struct rte_ring *r)
fprintf(f, " size=%"PRIu32"\n", r->size);
fprintf(f, " capacity=%"PRIu32"\n", r->capacity);
fprintf(f, " ct=%"PRIu32"\n", r->cons.tail);
- fprintf(f, " ch=%"PRIu32"\n", r->cons.head);
+ fprintf(f, " ch=%"PRIu32"\n", r->cons.head.val.pos);
fprintf(f, " pt=%"PRIu32"\n", r->prod.tail);
- fprintf(f, " ph=%"PRIu32"\n", r->prod.head);
+ fprintf(f, " ph=%"PRIu32"\n", r->prod.head.val.pos);
fprintf(f, " used=%u\n", rte_ring_count(r));
fprintf(f, " avail=%u\n", rte_ring_free_count(r));
}
diff --git a/lib/ring/rte_ring_core.h b/lib/ring/rte_ring_core.h
index 270869d214..b88a1bc352 100644
--- a/lib/ring/rte_ring_core.h
+++ b/lib/ring/rte_ring_core.h
@@ -66,8 +66,17 @@ enum rte_ring_sync_type {
* Depending on sync_type format of that structure might be different,
* but offset for *sync_type* and *tail* values should remain the same.
*/
+union __rte_ring_head_cft {
+ /** raw 8B value to read/write *cnt* and *pos* as one atomic op */
+ alignas(sizeof(uint64_t)) RTE_ATOMIC(uint64_t) raw;
+ struct {
+ uint32_t pos; /**< head position */
+ uint32_t cft; /**< cached foreign tail value*/
+ } val;
+};
+
struct rte_ring_headtail {
- volatile RTE_ATOMIC(uint32_t) head; /**< prod/consumer head. */
+ uint32_t __unused;
volatile RTE_ATOMIC(uint32_t) tail; /**< prod/consumer tail. */
union {
/** sync type of prod/cons */
@@ -75,6 +84,7 @@ struct rte_ring_headtail {
/** deprecated - True if single prod/cons */
uint32_t single;
};
+ union __rte_ring_head_cft head;
};
union __rte_ring_rts_poscnt {
diff --git a/lib/ring/rte_ring_generic_pvt.h b/lib/ring/rte_ring_generic_pvt.h
index 12f3595926..e70f4ff32c 100644
--- a/lib/ring/rte_ring_generic_pvt.h
+++ b/lib/ring/rte_ring_generic_pvt.h
@@ -38,17 +38,18 @@ __rte_ring_headtail_move_head(struct rte_ring_headtail *d,
{
unsigned int max = n;
int success;
+ uint32_t tail;
+ union __rte_ring_head_cft nh, oh;
+
+ oh.raw = rte_atomic_load_explicit(&d->head.raw,
+ rte_memory_order_acquire);
do {
/* Reset n to the initial burst count */
n = max;
- *old_head = d->head;
-
- /* add rmb barrier to avoid load/load reorder in weak
- * memory model. It is noop on x86
- */
- rte_smp_rmb();
+ *old_head = oh.val.pos;
+ tail = oh.val.cft;
/*
* The subtraction is done between two unsigned 32bits value
@@ -56,24 +57,41 @@ __rte_ring_headtail_move_head(struct rte_ring_headtail *d,
* *old_head > s->tail). So 'free_entries' is always between 0
* and capacity (which is < size).
*/
- *entries = (capacity + s->tail - *old_head);
+ *entries = (capacity + tail - *old_head);
- /* check that we have enough room in ring */
- if (unlikely(n > *entries))
- n = (behavior == RTE_RING_QUEUE_FIXED) ?
+ /* attempt #1: check that we have enough room with
+ * cached-foreign-tail value.
+ * Note that actual tail value can go forward till we cached
+ * it, in that case we might have to update our cached value.
+ */
+ if (unlikely(n > *entries)) {
+
+ tail = rte_atomic_load_explicit(&s->tail,
+ rte_memory_order_relaxed);
+ *entries = (capacity + tail - *old_head);
+
+ /* attempt #2: check that we have enough room in ring */
+ if (unlikely(n > *entries))
+ n = (behavior == RTE_RING_QUEUE_FIXED) ?
0 : *entries;
+ }
if (n == 0)
return 0;
*new_head = *old_head + n;
+ nh.val.pos = *new_head;
+ nh.val.cft = tail;
+
if (is_st) {
- d->head = *new_head;
+ d->head.raw = nh.raw;
success = 1;
} else
- success = rte_atomic32_cmpset(
- (uint32_t *)(uintptr_t)&d->head,
- *old_head, *new_head);
+ success = rte_atomic_compare_exchange_strong_explicit(
+ &d->head.raw, (uint64_t *)(uintptr_t)&oh.raw,
+ nh.raw, rte_memory_order_acquire,
+ rte_memory_order_acquire);
+
} while (unlikely(success == 0));
return n;
}
diff --git a/lib/ring/rte_ring_peek_elem_pvt.h b/lib/ring/rte_ring_peek_elem_pvt.h
index b5f0822b7e..e4dd0ae094 100644
--- a/lib/ring/rte_ring_peek_elem_pvt.h
+++ b/lib/ring/rte_ring_peek_elem_pvt.h
@@ -33,7 +33,7 @@ __rte_ring_st_get_tail(struct rte_ring_headtail *ht, uint32_t *tail,
{
uint32_t h, n, t;
- h = ht->head;
+ h = ht->head.val.pos;
t = ht->tail;
n = h - t;
@@ -58,7 +58,7 @@ __rte_ring_st_set_head_tail(struct rte_ring_headtail *ht, uint32_t tail,
RTE_SET_USED(enqueue);
pos = tail + num;
- ht->head = pos;
+ ht->head.val.pos = pos;
rte_atomic_store_explicit(&ht->tail, pos, rte_memory_order_release);
}
diff --git a/lib/ring/soring.c b/lib/ring/soring.c
index 929bde9697..baa449c872 100644
--- a/lib/ring/soring.c
+++ b/lib/ring/soring.c
@@ -90,7 +90,8 @@ __rte_soring_stage_finalize(struct soring_stage_headtail *sht,
* already finished.
*/
- head = rte_atomic_load_explicit(&sht->head, rte_memory_order_relaxed);
+ head = rte_atomic_load_explicit(&sht->head.val.pos,
+ rte_memory_order_relaxed);
n = RTE_MIN(head - ot.pos, maxn);
for (i = 0, tail = ot.pos; i < n; i += k, tail += k) {
@@ -213,22 +214,36 @@ __rte_soring_stage_move_head(struct soring_stage_headtail *d,
uint32_t *old_head, uint32_t *new_head, uint32_t *avail)
{
uint32_t n, tail;
+ union __rte_ring_head_cft nh, oh;
- *old_head = rte_atomic_load_explicit(&d->head,
+ oh.raw = rte_atomic_load_explicit(&d->head.raw,
rte_memory_order_acquire);
do {
n = num;
- tail = rte_atomic_load_explicit(&s->tail,
- rte_memory_order_relaxed);
+ *old_head = oh.val.pos;
+ tail = oh.val.cft;
*avail = capacity + tail - *old_head;
- if (n > *avail)
- n = (behavior == RTE_RING_QUEUE_FIXED) ? 0 : *avail;
+
+ if (n > *avail) {
+ tail = rte_atomic_load_explicit(&s->tail,
+ rte_memory_order_relaxed);
+ *avail = capacity + tail - *old_head;
+
+ if (n > *avail)
+ n = (behavior == RTE_RING_QUEUE_FIXED) ?
+ 0 : *avail;
+ }
+
if (n == 0)
return 0;
+
*new_head = *old_head + n;
- } while (rte_atomic_compare_exchange_strong_explicit(&d->head,
- old_head, *new_head, rte_memory_order_acquire,
+ nh.val.pos = *new_head;
+ nh.val.cft = tail;
+
+ } while (rte_atomic_compare_exchange_strong_explicit(&d->head.raw,
+ &oh.raw, nh.raw, rte_memory_order_acquire,
rte_memory_order_acquire) == 0);
return n;
diff --git a/lib/ring/soring.h b/lib/ring/soring.h
index 3a3f6efa76..0fb333aa71 100644
--- a/lib/ring/soring.h
+++ b/lib/ring/soring.h
@@ -60,8 +60,8 @@ union soring_stage_tail {
struct soring_stage_headtail {
volatile union soring_stage_tail tail;
- enum rte_ring_sync_type unused; /**< unused */
- volatile RTE_ATOMIC(uint32_t) head;
+ enum rte_ring_sync_type __unused; /**< unused */
+ union __rte_ring_head_cft head;
};
/**
--
2.35.3
^ permalink raw reply [relevance 2%]
* [PATCH v17 5/5] dts: add API doc generation
@ 2024-08-14 15:05 2% ` Juraj Linkeš
0 siblings, 0 replies; 200+ results
From: Juraj Linkeš @ 2024-08-14 15:05 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, bruce.richardson, jspewock, probb,
paul.szczepanek, Luca.Vizzarro, npratte
Cc: dev, Juraj Linkeš
The tool used to generate DTS API docs is Sphinx, which is already in
use in DPDK. The same configuration is used to preserve style with one
DTS-specific configuration (so that the DPDK docs are unchanged) that
modifies how the sidebar displays the content. There's other Sphinx
configuration related to Python docstrings which doesn't affect DPDK doc
build. All new configuration is in a conditional block, applied only
when DTS API docs are built to not interfere with DPDK doc build.
Sphinx generates the documentation from Python docstrings. The docstring
format is the Google format [0] which requires the sphinx.ext.napoleon
extension. The other extension, sphinx.ext.intersphinx, enables linking
to objects in external documentations, such as the Python documentation.
There is one requirement for building DTS docs - the same Python version
as DTS or higher, because Sphinx's autodoc extension imports the code.
The dependencies needed to import the code don't have to be satisfied,
as the autodoc extension allows us to mock the imports. The missing
packages are taken from the DTS pyproject.toml file.
And finally, the DTS API docs can be accessed from the DPDK API doxygen
page.
[0] https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
buildtools/call-sphinx-build.py | 2 +
buildtools/get-dts-runtime-deps.py | 84 +++++++++++++++++++++++
buildtools/meson.build | 1 +
doc/api/doxy-api-index.md | 3 +
doc/api/doxy-api.conf.in | 2 +
doc/api/dts/custom.css | 1 +
doc/api/dts/meson.build | 31 +++++++++
doc/api/meson.build | 6 +-
doc/guides/conf.py | 44 +++++++++++-
doc/guides/contributing/documentation.rst | 2 +
doc/guides/contributing/patches.rst | 4 ++
doc/guides/tools/dts.rst | 39 ++++++++++-
doc/meson.build | 1 +
13 files changed, 217 insertions(+), 3 deletions(-)
create mode 100755 buildtools/get-dts-runtime-deps.py
create mode 120000 doc/api/dts/custom.css
create mode 100644 doc/api/dts/meson.build
diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
index 623e7363ee..154e9f907b 100755
--- a/buildtools/call-sphinx-build.py
+++ b/buildtools/call-sphinx-build.py
@@ -15,6 +15,8 @@
# set the version in environment for sphinx to pick up
os.environ['DPDK_VERSION'] = version
+if 'dts' in src:
+ os.environ['DTS_BUILD'] = "y"
sphinx_cmd = [sphinx] + extra_args
diff --git a/buildtools/get-dts-runtime-deps.py b/buildtools/get-dts-runtime-deps.py
new file mode 100755
index 0000000000..68244480a3
--- /dev/null
+++ b/buildtools/get-dts-runtime-deps.py
@@ -0,0 +1,84 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 PANTHEON.tech s.r.o.
+#
+
+"""Utilities for DTS dependencies.
+
+The module can be used as an executable script,
+which verifies that the running Python version meets the version requirement of DTS.
+The script exits with the standard exit codes in this mode (0 is success, 1 is failure).
+
+The module also contains a function, get_missing_imports,
+which looks for runtime dependencies in the DTS pyproject.toml file
+and returns a list of module names used in an import statement (import packages) that are missing.
+This function is not used when the module is run as a script and is available to be imported.
+"""
+
+import configparser
+import importlib.metadata
+import importlib.util
+import os.path
+import platform
+
+from packaging.version import Version
+
+_VERSION_COMPARISON_CHARS = '^<>='
+_EXTRA_DEPS = {'invoke': '>=1.3', 'paramiko': '>=2.4'}
+_DPDK_ROOT = os.path.dirname(os.path.dirname(__file__))
+_DTS_DEP_FILE_PATH = os.path.join(_DPDK_ROOT, 'dts', 'pyproject.toml')
+
+
+def _get_dependencies(cfg_file_path):
+ cfg = configparser.ConfigParser()
+ with open(cfg_file_path) as f:
+ dts_deps_file_str = f.read()
+ dts_deps_file_str = dts_deps_file_str.replace("\n]", "]")
+ cfg.read_string(dts_deps_file_str)
+
+ deps_section = cfg['tool.poetry.dependencies']
+ return {dep: deps_section[dep].strip('"\'') for dep in deps_section}
+
+
+def get_missing_imports():
+ """Get missing DTS import packages from third party libraries.
+
+ Scan the DTS pyproject.toml file for dependencies and find those that are not installed.
+ The dependencies in pyproject.toml are listed by their distribution package names,
+ but the function finds the associated import packages - those used in import statements.
+
+ The function is not used when the module is run as a script. It should be imported.
+
+ Returns:
+ A list of missing import packages.
+ """
+ missing_imports = []
+ req_deps = _get_dependencies(_DTS_DEP_FILE_PATH)
+ req_deps.pop('python')
+
+ for req_dep, req_ver in (req_deps | _EXTRA_DEPS).items():
+ try:
+ req_ver = Version(req_ver.strip(_VERSION_COMPARISON_CHARS))
+ found_dep_ver = Version(importlib.metadata.version(req_dep))
+ if found_dep_ver < req_ver:
+ print(
+ f'The version "{found_dep_ver}" of package "{req_dep}" '
+ f'is lower than required "{req_ver}".'
+ )
+ except importlib.metadata.PackageNotFoundError:
+ print(f'Package "{req_dep}" not found.')
+ missing_imports.append(req_dep.lower().replace('-', '_'))
+
+ return missing_imports
+
+
+if __name__ == '__main__':
+ python_version = _get_dependencies(_DTS_DEP_FILE_PATH).pop('python')
+ if python_version:
+ sys_ver = Version(platform.python_version())
+ req_ver = Version(python_version.strip(_VERSION_COMPARISON_CHARS))
+ if sys_ver < req_ver:
+ print(
+ f'The available Python version "{sys_ver}" is lower than required "{req_ver}".'
+ )
+ exit(1)
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 3adf34e1a8..6b938d767c 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -24,6 +24,7 @@ get_numa_count_cmd = py3 + files('get-numa-count.py')
get_test_suites_cmd = py3 + files('get-test-suites.py')
has_hugepages_cmd = py3 + files('has-hugepages.py')
cmdline_gen_cmd = py3 + files('dpdk-cmdline-gen.py')
+get_dts_runtime_deps = py3 + files('get-dts-runtime-deps.py')
# install any build tools that end-users might want also
install_data([
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index f9f0300126..ab223bcdf7 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -245,3 +245,6 @@ The public API headers are grouped by topics:
[experimental APIs](@ref rte_compat.h),
[ABI versioning](@ref rte_function_versioning.h),
[version](@ref rte_version.h)
+
+- **tests**:
+ [**DTS**](@dts_api_main_page)
diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index a8823c046f..c94f02d411 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -124,6 +124,8 @@ SEARCHENGINE = YES
SORT_MEMBER_DOCS = NO
SOURCE_BROWSER = YES
+ALIASES = "dts_api_main_page=@DTS_API_MAIN_PAGE@"
+
EXAMPLE_PATH = @TOPDIR@/examples
EXAMPLE_PATTERNS = *.c
EXAMPLE_RECURSIVE = YES
diff --git a/doc/api/dts/custom.css b/doc/api/dts/custom.css
new file mode 120000
index 0000000000..3c9480c4a0
--- /dev/null
+++ b/doc/api/dts/custom.css
@@ -0,0 +1 @@
+../../guides/custom.css
\ No newline at end of file
diff --git a/doc/api/dts/meson.build b/doc/api/dts/meson.build
new file mode 100644
index 0000000000..f338eb69bf
--- /dev/null
+++ b/doc/api/dts/meson.build
@@ -0,0 +1,31 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
+if not sphinx.found()
+ subdir_done()
+endif
+
+python_ver_satisfied = run_command(get_dts_runtime_deps, check: false).returncode()
+if python_ver_satisfied != 0
+ subdir_done()
+endif
+
+cdata.set('DTS_API_MAIN_PAGE', join_paths('..', 'dts', 'html', 'index.html'))
+
+extra_sphinx_args = ['-E', '-c', join_paths(doc_source_dir, 'guides')]
+if get_option('werror')
+ extra_sphinx_args += '-W'
+endif
+
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk', 'dts')
+dts_api_html = custom_target('dts_api_html',
+ output: 'html',
+ command: [sphinx_wrapper, sphinx, meson.project_version(),
+ meson.current_source_dir(), meson.current_build_dir(), extra_sphinx_args],
+ build_by_default: get_option('enable_docs'),
+ install: get_option('enable_docs'),
+ install_dir: htmldir)
+
+doc_targets += dts_api_html
+doc_target_names += 'DTS_API_HTML'
diff --git a/doc/api/meson.build b/doc/api/meson.build
index 5b50692df9..71b861e42b 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -1,6 +1,11 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Luca Boccassi <bluca@debian.org>
+# initialize common Doxygen configuration
+cdata = configuration_data()
+
+subdir('dts')
+
doxygen = find_program('doxygen', required: get_option('enable_docs'))
if not doxygen.found()
@@ -30,7 +35,6 @@ example = custom_target('examples.dox',
build_by_default: get_option('enable_docs'))
# set up common Doxygen configuration
-cdata = configuration_data()
cdata.set('VERSION', meson.project_version())
cdata.set('API_EXAMPLES', join_paths(dpdk_build_root, 'doc', 'api', 'examples.dox'))
cdata.set('OUTPUT', join_paths(dpdk_build_root, 'doc', 'api'))
diff --git a/doc/guides/conf.py b/doc/guides/conf.py
index 0f7ff5282d..d7f3030838 100644
--- a/doc/guides/conf.py
+++ b/doc/guides/conf.py
@@ -10,7 +10,7 @@
from os.path import basename
from os.path import dirname
from os.path import join as path_join
-from sys import argv, stderr
+from sys import argv, stderr, path
import configparser
@@ -58,6 +58,48 @@
("tools/devbind", "dpdk-devbind",
"check device status and bind/unbind them from drivers", "", 8)]
+# DTS API docs additional configuration
+if environ.get('DTS_BUILD'):
+ extensions = ['sphinx.ext.napoleon', 'sphinx.ext.autodoc', 'sphinx.ext.intersphinx']
+ # Napoleon enables the Google format of Python doscstrings.
+ napoleon_numpy_docstring = False
+ napoleon_attr_annotations = True
+ napoleon_preprocess_types = True
+
+ # Autodoc pulls documentation from code.
+ autodoc_default_options = {
+ 'members': True,
+ 'member-order': 'bysource',
+ 'show-inheritance': True,
+ }
+ autodoc_class_signature = 'separated'
+ autodoc_typehints = 'both'
+ autodoc_typehints_format = 'short'
+ autodoc_typehints_description_target = 'documented'
+
+ # Intersphinx allows linking to external projects, such as Python docs.
+ intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
+
+ # DTS docstring options.
+ add_module_names = False
+ toc_object_entries = True
+ toc_object_entries_show_parents = 'hide'
+ # DTS Sidebar config.
+ html_theme_options = {
+ 'collapse_navigation': False,
+ 'navigation_depth': -1, # unlimited depth
+ }
+
+ # Add path to DTS sources so that Sphinx can find them.
+ dpdk_root = dirname(dirname(dirname(__file__)))
+ path.append(path_join(dpdk_root, 'dts'))
+
+ # Get missing DTS dependencies. Add path to buildtools to find the get_missing_imports function.
+ path.append(path_join(dpdk_root, 'buildtools'))
+ import importlib
+ # Ignore missing imports from DTS dependencies.
+ autodoc_mock_imports = importlib.import_module('get-dts-runtime-deps').get_missing_imports()
+
# ####### :numref: fallback ########
# The following hook functions add some simple handling for the :numref:
diff --git a/doc/guides/contributing/documentation.rst b/doc/guides/contributing/documentation.rst
index 68454ae0d5..7b287ce631 100644
--- a/doc/guides/contributing/documentation.rst
+++ b/doc/guides/contributing/documentation.rst
@@ -133,6 +133,8 @@ added to by the developer.
Building the Documentation
--------------------------
+.. _doc_dependencies:
+
Dependencies
~~~~~~~~~~~~
diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst
index 04c66bebc4..6629928bee 100644
--- a/doc/guides/contributing/patches.rst
+++ b/doc/guides/contributing/patches.rst
@@ -499,6 +499,10 @@ The script usage is::
For both of the above scripts, the -n option is used to specify a number of commits from HEAD,
and the -r option allows the user specify a ``git log`` range.
+Additionally, when contributing to the DTS tool, patches should also be checked using
+the ``dts-check-format.sh`` script in the ``devtools`` directory of the DPDK repo.
+To run the script, extra :ref:`Python dependencies <dts_deps>` are needed.
+
.. _contrib_check_compilation:
Checking Compilation
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 515b15e4d8..9e8929f567 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -54,6 +54,7 @@ DTS uses Poetry as its Python dependency management.
Python build/development and runtime environments are the same and DTS development environment,
DTS runtime environment or just plain DTS environment are used interchangeably.
+.. _dts_deps:
Setting up DTS environment
~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -291,8 +292,15 @@ When adding code to the DTS framework, pay attention to the rest of the code
and try not to divert much from it.
The :ref:`DTS developer tools <dts_dev_tools>` will issue warnings
when some of the basics are not met.
+You should also build the :ref:`API documentation <building_api_docs>`
+to address any issues found during the build.
-The code must be properly documented with docstrings.
+The API documentation, which is a helpful reference when developing, may be accessed
+in the code directly or generated with the :ref:`API docs build steps <building_api_docs>`.
+When adding new files or modifying the directory structure,
+the corresponding changes must be made to DTS api doc sources in ``doc/api/dts``.
+
+Speaking of which, the code must be properly documented with docstrings.
The style must conform to the `Google style
<https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings>`_.
See an example of the style `here
@@ -427,6 +435,35 @@ the DTS code check and format script.
Refer to the script for usage: ``devtools/dts-check-format.sh -h``.
+.. _building_api_docs:
+
+Building DTS API docs
+---------------------
+
+The documentation is built using the standard DPDK build system.
+See :doc:`../linux_gsg/build_dpdk` for more details on compiling DPDK with meson.
+
+The :ref:`doc build dependencies <doc_dependencies>` may be installed with Poetry:
+
+.. code-block:: console
+
+ poetry install --no-root --only docs
+ poetry install --no-root --with docs # an alternative that will also install DTS dependencies
+ poetry shell
+
+After executing the meson command, build the documentation with:
+
+.. code-block:: console
+
+ ninja -C build doc
+
+The output is generated in ``build/doc/api/dts/html``.
+
+.. note::
+
+ Make sure to fix any Sphinx warnings when adding or updating docstrings.
+
+
Configuration Schema
--------------------
diff --git a/doc/meson.build b/doc/meson.build
index 6f74706aa2..1e0cfa4127 100644
--- a/doc/meson.build
+++ b/doc/meson.build
@@ -1,6 +1,7 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Luca Boccassi <bluca@debian.org>
+doc_source_dir = meson.current_source_dir()
doc_targets = []
doc_target_names = []
subdir('api')
--
2.34.1
^ permalink raw reply [relevance 2%]
* [PATCH v3 26/26] config: add computed max queues define for compatibility
@ 2024-08-14 10:49 5% ` Bruce Richardson
0 siblings, 0 replies; 200+ results
From: Bruce Richardson @ 2024-08-14 10:49 UTC (permalink / raw)
To: dev; +Cc: ferruh.yigit, thomas, mb, Bruce Richardson
End applications may use the RTE_MAX_QUEUES_PER_PORT define in their
structure definitions, so keep a define present in DPDK for backward
compatibility. Rather than having a hard-coded value, we can use the
maximum of the Rx and Tx values as the overall max value. Rather than
using a macro which does the MAX() calculation inside it, we can compute
the actual value at configuration time and write it using meson.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
config/meson.build | 8 ++++++++
doc/guides/rel_notes/deprecation.rst | 11 +++++++++++
doc/guides/rel_notes/release_24_11.rst | 8 +++++++-
3 files changed, 26 insertions(+), 1 deletion(-)
diff --git a/config/meson.build b/config/meson.build
index fc41354c53..9677636754 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -372,6 +372,14 @@ if get_option('mbuf_refcnt_atomic')
endif
dpdk_conf.set10('RTE_IOVA_IN_MBUF', get_option('enable_iova_as_pa'))
+# set old MAX_QUEUES_PER_PORT option for compatibility. Compute
+# value as max of Rx and Tx counts
+if get_option('max_ethport_rx_queues') > get_option('max_ethport_tx_queues')
+ dpdk_conf.set('RTE_MAX_QUEUES_PER_PORT', get_option('max_ethport_rx_queues'))
+else
+ dpdk_conf.set('RTE_MAX_QUEUES_PER_PORT', get_option('max_ethport_tx_queues'))
+endif
+
compile_time_cpuflags = []
subdir(arch_subdir)
dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS', ','.join(compile_time_cpuflags))
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 211f59fdc9..e4ba00040f 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -17,6 +17,17 @@ Other API and ABI deprecation notices are to be posted below.
Deprecation Notices
-------------------
+* config: The define ``RTE_MAX_QUEUES_PER_PORT`` should be considered deprecated
+ and may be removed in a future release.
+ Its use in apps should be replaced by ``RTE_MAX_ETHPORT_RX_QUEUES`` or ``RTE_MAX_ETHPORT_TX_QUEUES``,
+ as appropriate.
+
+* config: The ``RTE_MAX_QUEUES_PER_PORT`` value is no longer hard-coded to 1024.
+ Its value is now computed at configuration time to be the maximum of the configured max Rx and Tx queue values,
+ given by the meson options ``max_ethport_rx_queues`` and ``max_ethport_tx_queues``.
+ If these are unmodified from the defaults,
+ the value of ``RTE_MAX_QUEUES_PER_PORT`` will be 256.
+
* build: The ``enable_kmods`` option is deprecated and will be removed in a future release.
Setting/clearing the option has no impact on the build.
Instead, kernel modules will be always built for OS's where out-of-tree kernel modules
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 825cc0fad9..130564d38e 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -67,9 +67,15 @@ New Features
The default max values for Rx and Tx queue limits are reduced from 1024, in previous releases,
to 256 in this release.
- For application that require large numbers of queues,
+ For applications that require large numbers of queues,
these defaults can be changed via the meson configuration options described above.
+.. note::
+
+ The define ``RTE_MAX_QUEUES_PER_PORT`` is kept for backward compatibility.
+ Its value is no longer hard-coded,
+ but is set, at configuration time, to the maximum of the configured max Rx and Tx queue values.
+
Removed Items
-------------
--
2.43.0
^ permalink raw reply [relevance 5%]
* [RFC PATCH v2 26/26] config: add computed max queues define for compatibility
@ 2024-08-13 16:00 5% ` Bruce Richardson
0 siblings, 0 replies; 200+ results
From: Bruce Richardson @ 2024-08-13 16:00 UTC (permalink / raw)
To: dev; +Cc: ferruh.yigit, thomas, mb, Bruce Richardson, Bruce Richardson
End applications may use the RTE_MAX_QUEUES_PER_PORT define in their
structure definitions, so keep a define present in DPDK for backward
compatibility. Rather than having a hard-coded value, we can use the
maximum of the Rx and Tx values as the overall max value. Rather than
using a macro which does the MAX() calculation inside it, we can compute
the actual value at configuration time and write it using meson.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
config/meson.build | 8 ++++++++
doc/guides/rel_notes/deprecation.rst | 11 +++++++++++
doc/guides/rel_notes/release_24_11.rst | 8 +++++++-
3 files changed, 26 insertions(+), 1 deletion(-)
diff --git a/config/meson.build b/config/meson.build
index fc41354c53..9677636754 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -372,6 +372,14 @@ if get_option('mbuf_refcnt_atomic')
endif
dpdk_conf.set10('RTE_IOVA_IN_MBUF', get_option('enable_iova_as_pa'))
+# set old MAX_QUEUES_PER_PORT option for compatibility. Compute
+# value as max of Rx and Tx counts
+if get_option('max_ethport_rx_queues') > get_option('max_ethport_tx_queues')
+ dpdk_conf.set('RTE_MAX_QUEUES_PER_PORT', get_option('max_ethport_rx_queues'))
+else
+ dpdk_conf.set('RTE_MAX_QUEUES_PER_PORT', get_option('max_ethport_tx_queues'))
+endif
+
compile_time_cpuflags = []
subdir(arch_subdir)
dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS', ','.join(compile_time_cpuflags))
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 211f59fdc9..e4ba00040f 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -17,6 +17,17 @@ Other API and ABI deprecation notices are to be posted below.
Deprecation Notices
-------------------
+* config: The define ``RTE_MAX_QUEUES_PER_PORT`` should be considered deprecated
+ and may be removed in a future release.
+ Its use in apps should be replaced by ``RTE_MAX_ETHPORT_RX_QUEUES`` or ``RTE_MAX_ETHPORT_TX_QUEUES``,
+ as appropriate.
+
+* config: The ``RTE_MAX_QUEUES_PER_PORT`` value is no longer hard-coded to 1024.
+ Its value is now computed at configuration time to be the maximum of the configured max Rx and Tx queue values,
+ given by the meson options ``max_ethport_rx_queues`` and ``max_ethport_tx_queues``.
+ If these are unmodified from the defaults,
+ the value of ``RTE_MAX_QUEUES_PER_PORT`` will be 256.
+
* build: The ``enable_kmods`` option is deprecated and will be removed in a future release.
Setting/clearing the option has no impact on the build.
Instead, kernel modules will be always built for OS's where out-of-tree kernel modules
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 825cc0fad9..130564d38e 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -67,9 +67,15 @@ New Features
The default max values for Rx and Tx queue limits are reduced from 1024, in previous releases,
to 256 in this release.
- For application that require large numbers of queues,
+ For applications that require large numbers of queues,
these defaults can be changed via the meson configuration options described above.
+.. note::
+
+ The define ``RTE_MAX_QUEUES_PER_PORT`` is kept for backward compatibility.
+ Its value is no longer hard-coded,
+ but is set, at configuration time, to the maximum of the configured max Rx and Tx queue values.
+
Removed Items
-------------
--
2.43.0
^ permalink raw reply [relevance 5%]
* Re: [PATCH v2 0/3] bbdev: sdditional queue stats
2024-08-12 23:41 3% [PATCH v2 0/3] bbdev: sdditional queue stats Nicolas Chautru
2024-08-12 23:41 6% ` [PATCH v2 1/3] bbdev: new queue stat for available enqueue depth Nicolas Chautru
@ 2024-08-13 7:22 0% ` Hemant Agrawal
2024-09-13 13:50 0% ` Maxime Coquelin
2 siblings, 0 replies; 200+ results
From: Hemant Agrawal @ 2024-08-13 7:22 UTC (permalink / raw)
To: dev
Series-
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
On 13-08-2024 05:11, Nicolas Chautru wrote:
> v2: update to ABI doc suggested by Maxime.
>
> These series include introducing a new paramter in the queue stat
> which can be used to monitor the number of available enqueue
> still possible.
> The acc PMD is then refactored to use a set of common function
> to update several queue status parameters including the new one.
> The application is also updated.
> Thanks
> Nic
>
> Nicolas Chautru (3):
> bbdev: new queue stat for available enqueue depth
> baseband/acc: refactor queue status update
> test/bbdev: update for queue stats
>
> app/test-bbdev/test_bbdev_perf.c | 1 +
> doc/guides/rel_notes/release_24_11.rst | 3 ++
> drivers/baseband/acc/acc_common.h | 18 ++++++++
> drivers/baseband/acc/rte_acc100_pmd.c | 45 ++++++-------------
> drivers/baseband/acc/rte_vrb_pmd.c | 61 ++++++++------------------
> lib/bbdev/rte_bbdev.h | 2 +
> 6 files changed, 56 insertions(+), 74 deletions(-)
>
^ permalink raw reply [relevance 0%]
* [PATCH v2 1/3] bbdev: new queue stat for available enqueue depth
2024-08-12 23:41 3% [PATCH v2 0/3] bbdev: sdditional queue stats Nicolas Chautru
@ 2024-08-12 23:41 6% ` Nicolas Chautru
2024-09-10 15:16 0% ` Maxime Coquelin
2024-08-13 7:22 0% ` [PATCH v2 0/3] bbdev: sdditional queue stats Hemant Agrawal
2024-09-13 13:50 0% ` Maxime Coquelin
2 siblings, 1 reply; 200+ results
From: Nicolas Chautru @ 2024-08-12 23:41 UTC (permalink / raw)
To: dev, maxime.coquelin
Cc: hemant.agrawal, david.marchand, hernan.vargas, Nicolas Chautru
Capturing additional queue stats counter for the
depth of enqueue batch still available on the given
queue. This can help application to monitor that depth
at run time.
Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
---
doc/guides/rel_notes/release_24_11.rst | 3 +++
lib/bbdev/rte_bbdev.h | 2 ++
2 files changed, 5 insertions(+)
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 0ff70d9057..a45b9b2dc6 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -88,6 +88,9 @@ API Changes
ABI Changes
-----------
+ * bbdev: Structure ``rte_bbdev_stats`` was updated to add new parameter
+ to optionally report number of enqueue batch available ``enqueue_depth_avail``.
+
.. This section should contain ABI changes. Sample format:
* sample: Add a short 1-2 sentence description of the ABI change
diff --git a/lib/bbdev/rte_bbdev.h b/lib/bbdev/rte_bbdev.h
index 0cbfdd1c95..25514c58ac 100644
--- a/lib/bbdev/rte_bbdev.h
+++ b/lib/bbdev/rte_bbdev.h
@@ -283,6 +283,8 @@ struct rte_bbdev_stats {
* bbdev operation
*/
uint64_t acc_offload_cycles;
+ /** Available number of enqueue batch on that queue. */
+ uint16_t enqueue_depth_avail;
};
/**
--
2.34.1
^ permalink raw reply [relevance 6%]
* [PATCH v2 0/3] bbdev: sdditional queue stats
@ 2024-08-12 23:41 3% Nicolas Chautru
2024-08-12 23:41 6% ` [PATCH v2 1/3] bbdev: new queue stat for available enqueue depth Nicolas Chautru
` (2 more replies)
0 siblings, 3 replies; 200+ results
From: Nicolas Chautru @ 2024-08-12 23:41 UTC (permalink / raw)
To: dev, maxime.coquelin
Cc: hemant.agrawal, david.marchand, hernan.vargas, Nicolas Chautru
v2: update to ABI doc suggested by Maxime.
These series include introducing a new paramter in the queue stat
which can be used to monitor the number of available enqueue
still possible.
The acc PMD is then refactored to use a set of common function
to update several queue status parameters including the new one.
The application is also updated.
Thanks
Nic
Nicolas Chautru (3):
bbdev: new queue stat for available enqueue depth
baseband/acc: refactor queue status update
test/bbdev: update for queue stats
app/test-bbdev/test_bbdev_perf.c | 1 +
doc/guides/rel_notes/release_24_11.rst | 3 ++
drivers/baseband/acc/acc_common.h | 18 ++++++++
drivers/baseband/acc/rte_acc100_pmd.c | 45 ++++++-------------
drivers/baseband/acc/rte_vrb_pmd.c | 61 ++++++++------------------
lib/bbdev/rte_bbdev.h | 2 +
6 files changed, 56 insertions(+), 74 deletions(-)
--
2.34.1
^ permalink raw reply [relevance 3%]
* Re: [PATCH v1 1/3] bbdev: new queue stat for available enqueue depth
2024-08-12 17:27 0% ` Chautru, Nicolas
@ 2024-08-12 19:44 0% ` Maxime Coquelin
0 siblings, 0 replies; 200+ results
From: Maxime Coquelin @ 2024-08-12 19:44 UTC (permalink / raw)
To: Chautru, Nicolas, dev; +Cc: hemant.agrawal, Marchand, David, Vargas, Hernan
Hi Nicolas,
On 8/12/24 19:27, Chautru, Nicolas wrote:
> Hi Maxime,
>
> The branch origin/next-baseband-for-main doesn’t have yet the updates from main, such as the ./doc/rel_notes for 24.11.
> Can you refresh this? Or let me know how best to proceed.
I rebased the branches to latest main, you can proceed.
Thanks,
Maxime
> Thanks,
> Nic
>
>> -----Original Message-----
>> From: Maxime Coquelin <maxime.coquelin@redhat.com>
>> Sent: Monday, August 12, 2024 2:29 AM
>> To: Chautru, Nicolas <nicolas.chautru@intel.com>; dev@dpdk.org
>> Cc: hemant.agrawal@nxp.com; Marchand, David
>> <david.marchand@redhat.com>; Vargas, Hernan
>> <hernan.vargas@intel.com>
>> Subject: Re: [PATCH v1 1/3] bbdev: new queue stat for available enqueue
>> depth
>>
>> Hi Nicolas,
>>
>> On 4/4/24 23:04, Nicolas Chautru wrote:
>>> Capturing additional queue stats counter for the depth of enqueue
>>> batch still available on the given queue. This can help application to
>>> monitor that depth at run time.
>>>
>>> Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
>>> ---
>>> lib/bbdev/rte_bbdev.h | 2 ++
>>> 1 file changed, 2 insertions(+)
>>>
>>> diff --git a/lib/bbdev/rte_bbdev.h b/lib/bbdev/rte_bbdev.h index
>>> 0cbfdd1c95..25514c58ac 100644
>>> --- a/lib/bbdev/rte_bbdev.h
>>> +++ b/lib/bbdev/rte_bbdev.h
>>> @@ -283,6 +283,8 @@ struct rte_bbdev_stats {
>>> * bbdev operation
>>> */
>>> uint64_t acc_offload_cycles;
>>> + /** Available number of enqueue batch on that queue. */
>>> + uint16_t enqueue_depth_avail;
>>> };
>>>
>>> /**
>>
>> I think it needs to be documented in the ABI change section.
>>
>> With that done, feel free to add:
>>
>> Maxime Coquelin <maxime.coquelin@redhat.com>
>>
>> Thanks,
>> Maxime
>
^ permalink raw reply [relevance 0%]
* RE: [PATCH v1 1/3] bbdev: new queue stat for available enqueue depth
2024-08-12 9:28 3% ` Maxime Coquelin
2024-08-12 9:56 0% ` Maxime Coquelin
@ 2024-08-12 17:27 0% ` Chautru, Nicolas
2024-08-12 19:44 0% ` Maxime Coquelin
1 sibling, 1 reply; 200+ results
From: Chautru, Nicolas @ 2024-08-12 17:27 UTC (permalink / raw)
To: Maxime Coquelin, dev; +Cc: hemant.agrawal, Marchand, David, Vargas, Hernan
Hi Maxime,
The branch origin/next-baseband-for-main doesn’t have yet the updates from main, such as the ./doc/rel_notes for 24.11.
Can you refresh this? Or let me know how best to proceed.
Thanks,
Nic
> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Monday, August 12, 2024 2:29 AM
> To: Chautru, Nicolas <nicolas.chautru@intel.com>; dev@dpdk.org
> Cc: hemant.agrawal@nxp.com; Marchand, David
> <david.marchand@redhat.com>; Vargas, Hernan
> <hernan.vargas@intel.com>
> Subject: Re: [PATCH v1 1/3] bbdev: new queue stat for available enqueue
> depth
>
> Hi Nicolas,
>
> On 4/4/24 23:04, Nicolas Chautru wrote:
> > Capturing additional queue stats counter for the depth of enqueue
> > batch still available on the given queue. This can help application to
> > monitor that depth at run time.
> >
> > Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
> > ---
> > lib/bbdev/rte_bbdev.h | 2 ++
> > 1 file changed, 2 insertions(+)
> >
> > diff --git a/lib/bbdev/rte_bbdev.h b/lib/bbdev/rte_bbdev.h index
> > 0cbfdd1c95..25514c58ac 100644
> > --- a/lib/bbdev/rte_bbdev.h
> > +++ b/lib/bbdev/rte_bbdev.h
> > @@ -283,6 +283,8 @@ struct rte_bbdev_stats {
> > * bbdev operation
> > */
> > uint64_t acc_offload_cycles;
> > + /** Available number of enqueue batch on that queue. */
> > + uint16_t enqueue_depth_avail;
> > };
> >
> > /**
>
> I think it needs to be documented in the ABI change section.
>
> With that done, feel free to add:
>
> Maxime Coquelin <maxime.coquelin@redhat.com>
>
> Thanks,
> Maxime
^ permalink raw reply [relevance 0%]
* Re: [PATCH v1 1/3] bbdev: new queue stat for available enqueue depth
2024-08-12 9:28 3% ` Maxime Coquelin
@ 2024-08-12 9:56 0% ` Maxime Coquelin
2024-08-12 17:27 0% ` Chautru, Nicolas
1 sibling, 0 replies; 200+ results
From: Maxime Coquelin @ 2024-08-12 9:56 UTC (permalink / raw)
To: Nicolas Chautru, dev; +Cc: hemant.agrawal, david.marchand, hernan.vargas
On 8/12/24 11:28, Maxime Coquelin wrote:
> Hi Nicolas,
>
> On 4/4/24 23:04, Nicolas Chautru wrote:
>> Capturing additional queue stats counter for the
>> depth of enqueue batch still available on the given
>> queue. This can help application to monitor that depth
>> at run time.
>>
>> Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
>> ---
>> lib/bbdev/rte_bbdev.h | 2 ++
>> 1 file changed, 2 insertions(+)
>>
>> diff --git a/lib/bbdev/rte_bbdev.h b/lib/bbdev/rte_bbdev.h
>> index 0cbfdd1c95..25514c58ac 100644
>> --- a/lib/bbdev/rte_bbdev.h
>> +++ b/lib/bbdev/rte_bbdev.h
>> @@ -283,6 +283,8 @@ struct rte_bbdev_stats {
>> * bbdev operation
>> */
>> uint64_t acc_offload_cycles;
>> + /** Available number of enqueue batch on that queue. */
>> + uint16_t enqueue_depth_avail;
>> };
>> /**
>
> I think it needs to be documented in the ABI change section.
>
> With that done, feel free to add:
>
> Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>
> Thanks,
> Maxime
^ permalink raw reply [relevance 0%]
* Re: [PATCH v1 1/3] bbdev: new queue stat for available enqueue depth
@ 2024-08-12 9:28 3% ` Maxime Coquelin
2024-08-12 9:56 0% ` Maxime Coquelin
2024-08-12 17:27 0% ` Chautru, Nicolas
0 siblings, 2 replies; 200+ results
From: Maxime Coquelin @ 2024-08-12 9:28 UTC (permalink / raw)
To: Nicolas Chautru, dev; +Cc: hemant.agrawal, david.marchand, hernan.vargas
Hi Nicolas,
On 4/4/24 23:04, Nicolas Chautru wrote:
> Capturing additional queue stats counter for the
> depth of enqueue batch still available on the given
> queue. This can help application to monitor that depth
> at run time.
>
> Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
> ---
> lib/bbdev/rte_bbdev.h | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/lib/bbdev/rte_bbdev.h b/lib/bbdev/rte_bbdev.h
> index 0cbfdd1c95..25514c58ac 100644
> --- a/lib/bbdev/rte_bbdev.h
> +++ b/lib/bbdev/rte_bbdev.h
> @@ -283,6 +283,8 @@ struct rte_bbdev_stats {
> * bbdev operation
> */
> uint64_t acc_offload_cycles;
> + /** Available number of enqueue batch on that queue. */
> + uint16_t enqueue_depth_avail;
> };
>
> /**
I think it needs to be documented in the ABI change section.
With that done, feel free to add:
Maxime Coquelin <maxime.coquelin@redhat.com>
Thanks,
Maxime
^ permalink raw reply [relevance 3%]
* [PATCH v9 1/2] power: introduce PM QoS API on CPU wide
2024-08-09 9:50 4% ` [PATCH v9 0/2] power: " Huisong Li
@ 2024-08-09 9:50 5% ` Huisong Li
2024-09-10 2:00 0% ` fengchengwen
0 siblings, 1 reply; 200+ results
From: Huisong Li @ 2024-08-09 9:50 UTC (permalink / raw)
To: dev
Cc: mb, thomas, ferruh.yigit, anatoly.burakov, david.hunt,
sivaprasad.tummala, stephen, david.marchand, liuyonglong,
lihuisong
The deeper the idle state, the lower the power consumption, but the longer
the resume time. Some service are delay sensitive and very except the low
resume time, like interrupt packet receiving mode.
And the "/sys/devices/system/cpu/cpuX/power/pm_qos_resume_latency_us" sysfs
interface is used to set and get the resume latency limit on the cpuX for
userspace. Each cpuidle governor in Linux select which idle state to enter
based on this CPU resume latency in their idle task.
The per-CPU PM QoS API can be used to control this CPU's idle state
selection and limit just enter the shallowest idle state to low the delay
after sleep by setting strict resume latency (zero value).
Signed-off-by: Huisong Li <lihuisong@huawei.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
doc/guides/prog_guide/power_man.rst | 24 ++++++
doc/guides/rel_notes/release_24_11.rst | 5 ++
lib/power/meson.build | 2 +
lib/power/rte_power_qos.c | 114 +++++++++++++++++++++++++
lib/power/rte_power_qos.h | 73 ++++++++++++++++
lib/power/version.map | 4 +
6 files changed, 222 insertions(+)
create mode 100644 lib/power/rte_power_qos.c
create mode 100644 lib/power/rte_power_qos.h
diff --git a/doc/guides/prog_guide/power_man.rst b/doc/guides/prog_guide/power_man.rst
index f6674efe2d..faa32b4320 100644
--- a/doc/guides/prog_guide/power_man.rst
+++ b/doc/guides/prog_guide/power_man.rst
@@ -249,6 +249,30 @@ Get Num Pkgs
Get Num Dies
Get the number of die's on a given package.
+
+PM QoS
+------
+
+The deeper the idle state, the lower the power consumption, but the longer
+the resume time. Some service are delay sensitive and very except the low
+resume time, like interrupt packet receiving mode.
+
+And the "/sys/devices/system/cpu/cpuX/power/pm_qos_resume_latency_us" sysfs
+interface is used to set and get the resume latency limit on the cpuX for
+userspace. Each cpuidle governor in Linux select which idle state to enter
+based on this CPU resume latency in their idle task.
+
+The per-CPU PM QoS API can be used to set and get the CPU resume latency based
+on this sysfs.
+
+The ``rte_power_qos_set_cpu_resume_latency()`` function can control the CPU's
+idle state selection in Linux and limit just to enter the shallowest idle state
+to low the delay of resuming service after sleeping by setting strict resume
+latency (zero value).
+
+The ``rte_power_qos_get_cpu_resume_latency()`` function can get the resume
+latency on specified CPU.
+
References
----------
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 0ff70d9057..bd72d0a595 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -55,6 +55,11 @@ New Features
Also, make sure to start the actual text at the margin.
=======================================================
+* **Introduce per-CPU PM QoS interface.**
+
+ * Add per-CPU PM QoS interface to low the delay after sleep by controlling
+ CPU idle state selection.
+
Removed Items
-------------
diff --git a/lib/power/meson.build b/lib/power/meson.build
index b8426589b2..8222e178b0 100644
--- a/lib/power/meson.build
+++ b/lib/power/meson.build
@@ -23,12 +23,14 @@ sources = files(
'rte_power.c',
'rte_power_uncore.c',
'rte_power_pmd_mgmt.c',
+ 'rte_power_qos.c',
)
headers = files(
'rte_power.h',
'rte_power_guest_channel.h',
'rte_power_pmd_mgmt.h',
'rte_power_uncore.h',
+ 'rte_power_qos.h',
)
if cc.has_argument('-Wno-cast-qual')
cflags += '-Wno-cast-qual'
diff --git a/lib/power/rte_power_qos.c b/lib/power/rte_power_qos.c
new file mode 100644
index 0000000000..375746f832
--- /dev/null
+++ b/lib/power/rte_power_qos.c
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2024 HiSilicon Limited
+ */
+
+#include <errno.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <rte_lcore.h>
+#include <rte_log.h>
+
+#include "power_common.h"
+#include "rte_power_qos.h"
+
+#define PM_QOS_SYSFILE_RESUME_LATENCY_US \
+ "/sys/devices/system/cpu/cpu%u/power/pm_qos_resume_latency_us"
+
+int
+rte_power_qos_set_cpu_resume_latency(uint16_t lcore_id, int latency)
+{
+ char buf[LINE_MAX];
+ FILE *f;
+ int ret;
+
+ if (!rte_lcore_is_enabled(lcore_id)) {
+ POWER_LOG(ERR, "lcore id %u is not enabled", lcore_id);
+ return -EINVAL;
+ }
+
+ if (latency < 0) {
+ POWER_LOG(ERR, "latency should be greater than and equal to 0");
+ return -EINVAL;
+ }
+
+ ret = open_core_sysfs_file(&f, "w", PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
+ if (ret != 0) {
+ POWER_LOG(ERR, "Failed to open "PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
+ return ret;
+ }
+
+ /*
+ * Based on the sysfs interface pm_qos_resume_latency_us under
+ * @PM_QOS_SYSFILE_RESUME_LATENCY_US directory in kernel, their meanning
+ * is as follows for different input string.
+ * 1> the resume latency is 0 if the input is "n/a".
+ * 2> the resume latency is no constraint if the input is "0".
+ * 3> the resume latency is the actual value to be set.
+ */
+ if (latency == 0)
+ snprintf(buf, sizeof(buf), "%s", "n/a");
+ else if (latency == RTE_POWER_QOS_RESUME_LATENCY_NO_CONSTRAINT)
+ snprintf(buf, sizeof(buf), "%u", 0);
+ else
+ snprintf(buf, sizeof(buf), "%u", latency);
+
+ ret = write_core_sysfs_s(f, buf);
+ if (ret != 0) {
+ POWER_LOG(ERR, "Failed to write "PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
+ goto out;
+ }
+
+out:
+ if (f != NULL)
+ fclose(f);
+
+ return ret;
+}
+
+int
+rte_power_qos_get_cpu_resume_latency(uint16_t lcore_id)
+{
+ char buf[LINE_MAX];
+ int latency = -1;
+ FILE *f;
+ int ret;
+
+ if (!rte_lcore_is_enabled(lcore_id)) {
+ POWER_LOG(ERR, "lcore id %u is not enabled", lcore_id);
+ return -EINVAL;
+ }
+
+ ret = open_core_sysfs_file(&f, "r", PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
+ if (ret != 0) {
+ POWER_LOG(ERR, "Failed to open "PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
+ return ret;
+ }
+
+ ret = read_core_sysfs_s(f, buf, sizeof(buf));
+ if (ret != 0) {
+ POWER_LOG(ERR, "Failed to read "PM_QOS_SYSFILE_RESUME_LATENCY_US, lcore_id);
+ goto out;
+ }
+
+ /*
+ * Based on the sysfs interface pm_qos_resume_latency_us under
+ * @PM_QOS_SYSFILE_RESUME_LATENCY_US directory in kernel, their meanning
+ * is as follows for different output string.
+ * 1> the resume latency is 0 if the output is "n/a".
+ * 2> the resume latency is no constraint if the output is "0".
+ * 3> the resume latency is the actual value in used for other string.
+ */
+ if (strcmp(buf, "n/a") == 0)
+ latency = 0;
+ else {
+ latency = strtoul(buf, NULL, 10);
+ latency = latency == 0 ? RTE_POWER_QOS_RESUME_LATENCY_NO_CONSTRAINT : latency;
+ }
+
+out:
+ if (f != NULL)
+ fclose(f);
+
+ return latency != -1 ? latency : ret;
+}
diff --git a/lib/power/rte_power_qos.h b/lib/power/rte_power_qos.h
new file mode 100644
index 0000000000..990c488373
--- /dev/null
+++ b/lib/power/rte_power_qos.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2024 HiSilicon Limited
+ */
+
+#ifndef RTE_POWER_QOS_H
+#define RTE_POWER_QOS_H
+
+#include <stdint.h>
+
+#include <rte_compat.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * @file rte_power_qos.h
+ *
+ * PM QoS API.
+ *
+ * The CPU-wide resume latency limit has a positive impact on this CPU's idle
+ * state selection in each cpuidle governor.
+ * Please see the PM QoS on CPU wide in the following link:
+ * https://www.kernel.org/doc/html/latest/admin-guide/abi-testing.html?highlight=pm_qos_resume_latency_us#abi-sys-devices-power-pm-qos-resume-latency-us
+ *
+ * The deeper the idle state, the lower the power consumption, but the
+ * longer the resume time. Some service are delay sensitive and very except the
+ * low resume time, like interrupt packet receiving mode.
+ *
+ * In these case, per-CPU PM QoS API can be used to control this CPU's idle
+ * state selection and limit just enter the shallowest idle state to low the
+ * delay after sleep by setting strict resume latency (zero value).
+ */
+
+#define RTE_POWER_QOS_STRICT_LATENCY_VALUE 0
+#define RTE_POWER_QOS_RESUME_LATENCY_NO_CONSTRAINT ((int)(UINT32_MAX >> 1))
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * @param lcore_id
+ * target logical core id
+ *
+ * @param latency
+ * The latency should be greater than and equal to zero in microseconds unit.
+ *
+ * @return
+ * 0 on success. Otherwise negative value is returned.
+ */
+__rte_experimental
+int rte_power_qos_set_cpu_resume_latency(uint16_t lcore_id, int latency);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get the current resume latency of this logical core.
+ * The default value in kernel is @see RTE_POWER_QOS_RESUME_LATENCY_NO_CONSTRAINT
+ * if don't set it.
+ *
+ * @return
+ * Negative value on failure.
+ * >= 0 means the actual resume latency limit on this core.
+ */
+__rte_experimental
+int rte_power_qos_get_cpu_resume_latency(uint16_t lcore_id);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* RTE_POWER_QOS_H */
diff --git a/lib/power/version.map b/lib/power/version.map
index c9a226614e..4e4955a4cf 100644
--- a/lib/power/version.map
+++ b/lib/power/version.map
@@ -51,4 +51,8 @@ EXPERIMENTAL {
rte_power_set_uncore_env;
rte_power_uncore_freqs;
rte_power_unset_uncore_env;
+
+ # added in 24.11
+ rte_power_qos_set_cpu_resume_latency;
+ rte_power_qos_get_cpu_resume_latency;
};
--
2.22.0
^ permalink raw reply [relevance 5%]
* [PATCH v9 0/2] power: introduce PM QoS interface
@ 2024-08-09 9:50 4% ` Huisong Li
2024-08-09 9:50 5% ` [PATCH v9 1/2] power: introduce PM QoS API on CPU wide Huisong Li
2024-09-12 2:38 4% ` [PATCH v10 0/2] power: introduce PM QoS interface Huisong Li
1 sibling, 1 reply; 200+ results
From: Huisong Li @ 2024-08-09 9:50 UTC (permalink / raw)
To: dev
Cc: mb, thomas, ferruh.yigit, anatoly.burakov, david.hunt,
sivaprasad.tummala, stephen, david.marchand, liuyonglong,
lihuisong
The deeper the idle state, the lower the power consumption, but the longer
the resume time. Some service are delay sensitive and very except the low
resume time, like interrupt packet receiving mode.
And the "/sys/devices/system/cpu/cpuX/power/pm_qos_resume_latency_us" sysfs
interface is used to set and get the resume latency limit on the cpuX for
userspace. Please see the description in kernel document[1].
Each cpuidle governor in Linux select which idle state to enter based on
this CPU resume latency in their idle task.
The per-CPU PM QoS API can be used to control this CPU's idle state
selection and limit just enter the shallowest idle state to low the delay
after sleep by setting strict resume latency (zero value).
[1] https://www.kernel.org/doc/html/latest/admin-guide/abi-testing.html?highlight=pm_qos_resume_latency_us#abi-sys-devices-power-pm-qos-resume-latency-us
---
v9:
- move new feature description from release_24_07.rst to release_24_11.rst.
v8:
- update the latest code to resolve CI warning
v7:
- remove a dead code rte_lcore_is_enabled in patch[2/2]
v6:
- update release_24_07.rst based on dpdk repo to resolve CI warning.
v5:
- use LINE_MAX to replace BUFSIZ, and use snprintf to replace sprintf.
v4:
- fix some comments basd on Stephen
- add stdint.h include
- add Acked-by Morten Brørup <mb@smartsharesystems.com>
v3:
- add RTE_POWER_xxx prefix for some macro in header
- add the check for lcore_id with rte_lcore_is_enabled
v2:
- use PM QoS on CPU wide to replace the one on system wide
Huisong Li (2):
power: introduce PM QoS API on CPU wide
examples/l3fwd-power: add PM QoS configuration
doc/guides/prog_guide/power_man.rst | 24 ++++++
doc/guides/rel_notes/release_24_11.rst | 5 ++
examples/l3fwd-power/main.c | 24 ++++++
lib/power/meson.build | 2 +
lib/power/rte_power_qos.c | 114 +++++++++++++++++++++++++
lib/power/rte_power_qos.h | 73 ++++++++++++++++
lib/power/version.map | 4 +
7 files changed, 246 insertions(+)
create mode 100644 lib/power/rte_power_qos.c
create mode 100644 lib/power/rte_power_qos.h
--
2.22.0
^ permalink raw reply [relevance 4%]
* Re: [OS-Team] [dpdklab] Re: [PATCH] version: 24.11-rc0
2024-08-08 12:55 3% ` David Marchand
@ 2024-08-08 21:45 3% ` Patrick Robb
0 siblings, 0 replies; 200+ results
From: Patrick Robb @ 2024-08-08 21:45 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Thomas Monjalon, ci, dpdklab
Thanks I will merge the change disabling ABI checks at UNH at start of
day tomorrow.
On Thu, Aug 8, 2024 at 8:56 AM David Marchand <david.marchand@redhat.com> wrote:
>
> On Thu, Aug 8, 2024 at 2:00 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> >
> > 08/08/2024 10:03, David Marchand:
> > > Start a new release cycle with empty release notes.
> > >
> > > The ABI version becomes 25.0.
> > > The map files are updated to the new ABI major number (25).
> > > The ABI exceptions are dropped and CI ABI checks are disabled because
> > > compatibility is not preserved.
> > >
> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > Acked-by: Thomas Monjalon <thomas@monjalon.net>
>
> Applied, thanks.
>
> Heads up to CI: ABI checks must be disabled during v24.11 release.
>
>
> --
> David Marchand
>
^ permalink raw reply [relevance 3%]
* Re: [PATCH] version: 24.11-rc0
2024-08-08 12:00 0% ` Thomas Monjalon
@ 2024-08-08 12:55 3% ` David Marchand
2024-08-08 21:45 3% ` [OS-Team] [dpdklab] " Patrick Robb
0 siblings, 1 reply; 200+ results
From: David Marchand @ 2024-08-08 12:55 UTC (permalink / raw)
To: David Marchand; +Cc: dev, Thomas Monjalon, ci, dpdklab
On Thu, Aug 8, 2024 at 2:00 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 08/08/2024 10:03, David Marchand:
> > Start a new release cycle with empty release notes.
> >
> > The ABI version becomes 25.0.
> > The map files are updated to the new ABI major number (25).
> > The ABI exceptions are dropped and CI ABI checks are disabled because
> > compatibility is not preserved.
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
Applied, thanks.
Heads up to CI: ABI checks must be disabled during v24.11 release.
--
David Marchand
^ permalink raw reply [relevance 3%]
* Re: [PATCH] version: 24.11-rc0
2024-08-08 8:03 12% [PATCH] version: 24.11-rc0 David Marchand
@ 2024-08-08 12:00 0% ` Thomas Monjalon
2024-08-08 12:55 3% ` David Marchand
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2024-08-08 12:00 UTC (permalink / raw)
To: David Marchand; +Cc: dev
08/08/2024 10:03, David Marchand:
> Start a new release cycle with empty release notes.
>
> The ABI version becomes 25.0.
> The map files are updated to the new ABI major number (25).
> The ABI exceptions are dropped and CI ABI checks are disabled because
> compatibility is not preserved.
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
^ permalink raw reply [relevance 0%]
* [PATCH v16 5/5] dts: add API doc generation
@ 2024-08-08 8:54 2% ` Juraj Linkeš
0 siblings, 0 replies; 200+ results
From: Juraj Linkeš @ 2024-08-08 8:54 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, bruce.richardson, jspewock, probb,
paul.szczepanek, Luca.Vizzarro, npratte
Cc: dev, Juraj Linkeš
The tool used to generate DTS API docs is Sphinx, which is already in
use in DPDK. The same configuration is used to preserve style with one
DTS-specific configuration (so that the DPDK docs are unchanged) that
modifies how the sidebar displays the content. There's other Sphinx
configuration related to Python docstrings which doesn't affect DPDK doc
build. All new configuration is in a conditional block, applied only
when DTS API docs are built to not interfere with DPDK doc build.
Sphinx generates the documentation from Python docstrings. The docstring
format is the Google format [0] which requires the sphinx.ext.napoleon
extension. The other extension, sphinx.ext.intersphinx, enables linking
to objects in external documentations, such as the Python documentation.
There is one requirement for building DTS docs - the same Python version
as DTS or higher, because Sphinx's autodoc extension imports the code.
The dependencies needed to import the code don't have to be satisfied,
as the autodoc extension allows us to mock the imports. The missing
packages are taken from the DTS pyproject.toml file.
And finally, the DTS API docs can be accessed from the DPDK API doxygen
page.
[0] https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
buildtools/call-sphinx-build.py | 2 +
buildtools/get-dts-runtime-deps.py | 72 +++++++++++++++++++++++
buildtools/meson.build | 1 +
doc/api/doxy-api-index.md | 3 +
doc/api/doxy-api.conf.in | 2 +
doc/api/dts/custom.css | 1 +
doc/api/dts/meson.build | 29 +++++++++
doc/api/meson.build | 13 ++++
doc/guides/conf.py | 44 +++++++++++++-
doc/guides/contributing/documentation.rst | 2 +
doc/guides/contributing/patches.rst | 4 ++
doc/guides/tools/dts.rst | 39 +++++++++++-
doc/meson.build | 1 +
13 files changed, 211 insertions(+), 2 deletions(-)
create mode 100755 buildtools/get-dts-runtime-deps.py
create mode 120000 doc/api/dts/custom.css
create mode 100644 doc/api/dts/meson.build
diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
index 623e7363ee..45724ffcd4 100755
--- a/buildtools/call-sphinx-build.py
+++ b/buildtools/call-sphinx-build.py
@@ -15,6 +15,8 @@
# set the version in environment for sphinx to pick up
os.environ['DPDK_VERSION'] = version
+if src.find('dts') != -1:
+ os.environ['DTS_BUILD'] = "y"
sphinx_cmd = [sphinx] + extra_args
diff --git a/buildtools/get-dts-runtime-deps.py b/buildtools/get-dts-runtime-deps.py
new file mode 100755
index 0000000000..5d629dd09d
--- /dev/null
+++ b/buildtools/get-dts-runtime-deps.py
@@ -0,0 +1,72 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 PANTHEON.tech s.r.o.
+#
+
+"""Utilities for DTS dependencies.
+
+The module can be used as an executable script,
+which verifies that the running Python version meets the version requirement of DTS.
+The script exits with the standard exit codes in this mode (0 is success, 1 is failure).
+
+The module also contains a function, get_missing_imports,
+which looks for runtime and doc generation dependencies in the DTS pyproject.toml file
+and returns a list of module names used in an import statement that are missing.
+"""
+
+import configparser
+import importlib.metadata
+import importlib.util
+import os.path
+import platform
+
+from packaging.version import Version
+
+_VERSION_COMPARISON_CHARS = '^<>='
+_EXTRA_DEPS = {'invoke': '>=1.3', 'paramiko': '>=2.4'}
+_DPDK_ROOT = os.path.dirname(os.path.dirname(__file__))
+_DTS_DEP_FILE_PATH = os.path.join(_DPDK_ROOT, 'dts', 'pyproject.toml')
+
+
+def _get_dependencies(cfg_file_path):
+ cfg = configparser.ConfigParser()
+ with open(cfg_file_path) as f:
+ dts_deps_file_str = f.read()
+ dts_deps_file_str = dts_deps_file_str.replace("\n]", "]")
+ cfg.read_string(dts_deps_file_str)
+
+ deps_section = cfg['tool.poetry.dependencies']
+ return {dep: deps_section[dep].strip('"\'') for dep in deps_section}
+
+
+def get_missing_imports():
+ missing_imports = []
+ req_deps = _get_dependencies(_DTS_DEP_FILE_PATH)
+ req_deps.pop('python')
+
+ for req_dep, req_ver in (req_deps | _EXTRA_DEPS).items():
+ try:
+ req_ver = Version(req_ver.strip(_VERSION_COMPARISON_CHARS))
+ found_dep_ver = Version(importlib.metadata.version(req_dep))
+ if found_dep_ver < req_ver:
+ print(
+ f'The version "{found_dep_ver}" of package "{req_dep}" '
+ f'is lower than required "{req_ver}".'
+ )
+ except importlib.metadata.PackageNotFoundError:
+ print(f'Package "{req_dep}" not found.')
+ missing_imports.append(req_dep.lower().replace('-', '_'))
+
+ return missing_imports
+
+
+if __name__ == '__main__':
+ python_version = _get_dependencies(_DTS_DEP_FILE_PATH).pop('python')
+ if python_version:
+ sys_ver = Version(platform.python_version())
+ req_ver = Version(python_version.strip(_VERSION_COMPARISON_CHARS))
+ if sys_ver < req_ver:
+ print(
+ f'The available Python version "{sys_ver}" is lower than required "{req_ver}".'
+ )
+ exit(1)
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 3adf34e1a8..6b938d767c 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -24,6 +24,7 @@ get_numa_count_cmd = py3 + files('get-numa-count.py')
get_test_suites_cmd = py3 + files('get-test-suites.py')
has_hugepages_cmd = py3 + files('has-hugepages.py')
cmdline_gen_cmd = py3 + files('dpdk-cmdline-gen.py')
+get_dts_runtime_deps = py3 + files('get-dts-runtime-deps.py')
# install any build tools that end-users might want also
install_data([
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index f9f0300126..ab223bcdf7 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -245,3 +245,6 @@ The public API headers are grouped by topics:
[experimental APIs](@ref rte_compat.h),
[ABI versioning](@ref rte_function_versioning.h),
[version](@ref rte_version.h)
+
+- **tests**:
+ [**DTS**](@dts_api_main_page)
diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index a8823c046f..c94f02d411 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -124,6 +124,8 @@ SEARCHENGINE = YES
SORT_MEMBER_DOCS = NO
SOURCE_BROWSER = YES
+ALIASES = "dts_api_main_page=@DTS_API_MAIN_PAGE@"
+
EXAMPLE_PATH = @TOPDIR@/examples
EXAMPLE_PATTERNS = *.c
EXAMPLE_RECURSIVE = YES
diff --git a/doc/api/dts/custom.css b/doc/api/dts/custom.css
new file mode 120000
index 0000000000..3c9480c4a0
--- /dev/null
+++ b/doc/api/dts/custom.css
@@ -0,0 +1 @@
+../../guides/custom.css
\ No newline at end of file
diff --git a/doc/api/dts/meson.build b/doc/api/dts/meson.build
new file mode 100644
index 0000000000..b4b6f9d269
--- /dev/null
+++ b/doc/api/dts/meson.build
@@ -0,0 +1,29 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
+if not sphinx.found()
+ subdir_done()
+endif
+
+python_ver_satisfied = run_command(get_dts_runtime_deps, check: false).returncode()
+if python_ver_satisfied != 0
+ subdir_done()
+endif
+
+extra_sphinx_args = ['-E', '-c', join_paths(doc_source_dir, 'guides')]
+if get_option('werror')
+ extra_sphinx_args += '-W'
+endif
+
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk', 'dts')
+dts_api_html = custom_target('dts_api_html',
+ output: 'html',
+ command: [sphinx_wrapper, sphinx, meson.project_version(),
+ meson.current_source_dir(), meson.current_build_dir(), extra_sphinx_args],
+ build_by_default: get_option('enable_docs'),
+ install: get_option('enable_docs'),
+ install_dir: htmldir)
+
+dts_doc_targets += dts_api_html
+dts_doc_target_names += 'DTS_API_HTML'
diff --git a/doc/api/meson.build b/doc/api/meson.build
index 5b50692df9..788129336b 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -1,6 +1,18 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Luca Boccassi <bluca@debian.org>
+dts_doc_targets = []
+dts_doc_target_names = []
+subdir('dts')
+
+if dts_doc_targets.length() == 0
+ dts_message = 'No DTS docs targets found'
+else
+ dts_message = 'Building DTS docs:'
+endif
+run_target('dts-doc', command: [echo, dts_message, dts_doc_target_names],
+ depends: dts_doc_targets)
+
doxygen = find_program('doxygen', required: get_option('enable_docs'))
if not doxygen.found()
@@ -40,6 +52,7 @@ cdata.set('WARN_AS_ERROR', 'NO')
if get_option('werror')
cdata.set('WARN_AS_ERROR', 'YES')
endif
+cdata.set('DTS_API_MAIN_PAGE', join_paths('..', 'dts', 'html', 'index.html'))
# configure HTML Doxygen run
html_cdata = configuration_data()
diff --git a/doc/guides/conf.py b/doc/guides/conf.py
index 0f7ff5282d..d7f3030838 100644
--- a/doc/guides/conf.py
+++ b/doc/guides/conf.py
@@ -10,7 +10,7 @@
from os.path import basename
from os.path import dirname
from os.path import join as path_join
-from sys import argv, stderr
+from sys import argv, stderr, path
import configparser
@@ -58,6 +58,48 @@
("tools/devbind", "dpdk-devbind",
"check device status and bind/unbind them from drivers", "", 8)]
+# DTS API docs additional configuration
+if environ.get('DTS_BUILD'):
+ extensions = ['sphinx.ext.napoleon', 'sphinx.ext.autodoc', 'sphinx.ext.intersphinx']
+ # Napoleon enables the Google format of Python doscstrings.
+ napoleon_numpy_docstring = False
+ napoleon_attr_annotations = True
+ napoleon_preprocess_types = True
+
+ # Autodoc pulls documentation from code.
+ autodoc_default_options = {
+ 'members': True,
+ 'member-order': 'bysource',
+ 'show-inheritance': True,
+ }
+ autodoc_class_signature = 'separated'
+ autodoc_typehints = 'both'
+ autodoc_typehints_format = 'short'
+ autodoc_typehints_description_target = 'documented'
+
+ # Intersphinx allows linking to external projects, such as Python docs.
+ intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
+
+ # DTS docstring options.
+ add_module_names = False
+ toc_object_entries = True
+ toc_object_entries_show_parents = 'hide'
+ # DTS Sidebar config.
+ html_theme_options = {
+ 'collapse_navigation': False,
+ 'navigation_depth': -1, # unlimited depth
+ }
+
+ # Add path to DTS sources so that Sphinx can find them.
+ dpdk_root = dirname(dirname(dirname(__file__)))
+ path.append(path_join(dpdk_root, 'dts'))
+
+ # Get missing DTS dependencies. Add path to buildtools to find the get_missing_imports function.
+ path.append(path_join(dpdk_root, 'buildtools'))
+ import importlib
+ # Ignore missing imports from DTS dependencies.
+ autodoc_mock_imports = importlib.import_module('get-dts-runtime-deps').get_missing_imports()
+
# ####### :numref: fallback ########
# The following hook functions add some simple handling for the :numref:
diff --git a/doc/guides/contributing/documentation.rst b/doc/guides/contributing/documentation.rst
index 68454ae0d5..7b287ce631 100644
--- a/doc/guides/contributing/documentation.rst
+++ b/doc/guides/contributing/documentation.rst
@@ -133,6 +133,8 @@ added to by the developer.
Building the Documentation
--------------------------
+.. _doc_dependencies:
+
Dependencies
~~~~~~~~~~~~
diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst
index 04c66bebc4..6629928bee 100644
--- a/doc/guides/contributing/patches.rst
+++ b/doc/guides/contributing/patches.rst
@@ -499,6 +499,10 @@ The script usage is::
For both of the above scripts, the -n option is used to specify a number of commits from HEAD,
and the -r option allows the user specify a ``git log`` range.
+Additionally, when contributing to the DTS tool, patches should also be checked using
+the ``dts-check-format.sh`` script in the ``devtools`` directory of the DPDK repo.
+To run the script, extra :ref:`Python dependencies <dts_deps>` are needed.
+
.. _contrib_check_compilation:
Checking Compilation
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 515b15e4d8..18cc7908cf 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -54,6 +54,7 @@ DTS uses Poetry as its Python dependency management.
Python build/development and runtime environments are the same and DTS development environment,
DTS runtime environment or just plain DTS environment are used interchangeably.
+.. _dts_deps:
Setting up DTS environment
~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -291,8 +292,15 @@ When adding code to the DTS framework, pay attention to the rest of the code
and try not to divert much from it.
The :ref:`DTS developer tools <dts_dev_tools>` will issue warnings
when some of the basics are not met.
+You should also build the :ref:`API documentation <building_api_docs>`
+to address any issues found during the build.
-The code must be properly documented with docstrings.
+The API documentation, which is a helpful reference when developing, may be accessed
+in the code directly or generated with the :ref:`API docs build steps <building_api_docs>`.
+When adding new files or modifying the directory structure,
+the corresponding changes must be made to DTS api doc sources in ``doc/api/dts``.
+
+Speaking of which, the code must be properly documented with docstrings.
The style must conform to the `Google style
<https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings>`_.
See an example of the style `here
@@ -427,6 +435,35 @@ the DTS code check and format script.
Refer to the script for usage: ``devtools/dts-check-format.sh -h``.
+.. _building_api_docs:
+
+Building DTS API docs
+---------------------
+
+The documentation is built using the standard DPDK build system.
+See :doc:`../linux_gsg/build_dpdk` for more details on compiling DPDK with meson.
+
+The :ref:`doc build dependencies <doc_dependencies>` may be installed with Poetry:
+
+.. code-block:: console
+
+ poetry install --no-root --only docs
+ poetry install --no-root --with docs # an alternative that will also install DTS dependencies
+ poetry shell
+
+After executing the meson command, build the documentation with:
+
+.. code-block:: console
+
+ ninja -C build dts-doc
+
+The output is generated in ``build/doc/api/dts/html``.
+
+.. note::
+
+ Make sure to fix any Sphinx warnings when adding or updating docstrings.
+
+
Configuration Schema
--------------------
diff --git a/doc/meson.build b/doc/meson.build
index 6f74706aa2..1e0cfa4127 100644
--- a/doc/meson.build
+++ b/doc/meson.build
@@ -1,6 +1,7 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Luca Boccassi <bluca@debian.org>
+doc_source_dir = meson.current_source_dir()
doc_targets = []
doc_target_names = []
subdir('api')
--
2.34.1
^ permalink raw reply [relevance 2%]
* [PATCH] version: 24.11-rc0
@ 2024-08-08 8:03 12% David Marchand
2024-08-08 12:00 0% ` Thomas Monjalon
0 siblings, 1 reply; 200+ results
From: David Marchand @ 2024-08-08 8:03 UTC (permalink / raw)
To: dev; +Cc: thomas
Start a new release cycle with empty release notes.
The ABI version becomes 25.0.
The map files are updated to the new ABI major number (25).
The ABI exceptions are dropped and CI ABI checks are disabled because
compatibility is not preserved.
Signed-off-by: David Marchand <david.marchand@redhat.com>
---
.github/workflows/build.yml | 4 +-
ABI_VERSION | 2 +-
VERSION | 2 +-
devtools/libabigail.abignore | 19 ---
doc/guides/rel_notes/index.rst | 1 +
doc/guides/rel_notes/release_24_11.rst | 136 +++++++++++++++++++++
drivers/baseband/acc/version.map | 2 +-
drivers/baseband/fpga_5gnr_fec/version.map | 2 +-
drivers/baseband/fpga_lte_fec/version.map | 2 +-
drivers/bus/fslmc/version.map | 2 +-
drivers/bus/pci/version.map | 2 +-
drivers/bus/platform/version.map | 2 +-
drivers/bus/vdev/version.map | 2 +-
drivers/bus/vmbus/version.map | 2 +-
drivers/crypto/octeontx/version.map | 2 +-
drivers/crypto/scheduler/version.map | 2 +-
drivers/dma/dpaa2/version.map | 2 +-
drivers/event/cnxk/version.map | 2 +-
drivers/event/dlb2/version.map | 2 +-
drivers/mempool/cnxk/version.map | 2 +-
drivers/mempool/dpaa2/version.map | 2 +-
drivers/net/atlantic/version.map | 2 +-
drivers/net/bnxt/version.map | 2 +-
drivers/net/bonding/version.map | 2 +-
drivers/net/cnxk/version.map | 2 +-
drivers/net/dpaa/version.map | 2 +-
drivers/net/dpaa2/version.map | 2 +-
drivers/net/i40e/version.map | 2 +-
drivers/net/iavf/version.map | 2 +-
drivers/net/ice/version.map | 2 +-
drivers/net/ipn3ke/version.map | 2 +-
drivers/net/ixgbe/version.map | 2 +-
drivers/net/mlx5/version.map | 2 +-
drivers/net/octeontx/version.map | 2 +-
drivers/net/ring/version.map | 2 +-
drivers/net/softnic/version.map | 2 +-
drivers/net/vhost/version.map | 2 +-
drivers/raw/ifpga/version.map | 2 +-
drivers/version.map | 2 +-
lib/acl/version.map | 2 +-
lib/bbdev/version.map | 2 +-
lib/bitratestats/version.map | 2 +-
lib/bpf/version.map | 2 +-
lib/cfgfile/version.map | 2 +-
lib/cmdline/version.map | 2 +-
lib/compressdev/version.map | 2 +-
lib/cryptodev/version.map | 2 +-
lib/distributor/version.map | 2 +-
lib/dmadev/version.map | 2 +-
lib/eal/version.map | 2 +-
lib/efd/version.map | 2 +-
lib/ethdev/version.map | 2 +-
lib/eventdev/version.map | 2 +-
lib/fib/version.map | 2 +-
lib/graph/version.map | 2 +-
lib/gro/version.map | 2 +-
lib/gso/version.map | 2 +-
lib/hash/version.map | 14 +--
lib/ip_frag/version.map | 2 +-
lib/ipsec/version.map | 2 +-
lib/jobstats/version.map | 2 +-
lib/kvargs/version.map | 2 +-
lib/latencystats/version.map | 2 +-
lib/log/version.map | 4 +-
lib/lpm/version.map | 2 +-
lib/mbuf/version.map | 2 +-
lib/member/version.map | 2 +-
lib/mempool/version.map | 2 +-
lib/meter/version.map | 2 +-
lib/metrics/version.map | 2 +-
lib/net/version.map | 2 +-
lib/node/version.map | 2 +-
lib/pcapng/version.map | 2 +-
lib/pci/version.map | 2 +-
lib/pdump/version.map | 2 +-
lib/pipeline/version.map | 2 +-
lib/port/version.map | 2 +-
lib/power/version.map | 2 +-
lib/rawdev/version.map | 2 +-
lib/rcu/version.map | 2 +-
lib/reorder/version.map | 2 +-
lib/rib/version.map | 2 +-
lib/ring/version.map | 2 +-
lib/sched/version.map | 2 +-
lib/security/version.map | 2 +-
lib/stack/version.map | 2 +-
lib/table/version.map | 2 +-
lib/telemetry/version.map | 2 +-
lib/timer/version.map | 2 +-
lib/vhost/version.map | 2 +-
90 files changed, 232 insertions(+), 114 deletions(-)
create mode 100644 doc/guides/rel_notes/release_24_11.rst
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index dbf25626d4..f7d3affbaa 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -27,7 +27,7 @@ jobs:
MINGW: ${{ matrix.config.cross == 'mingw' }}
MINI: ${{ matrix.config.mini != '' }}
PPC64LE: ${{ matrix.config.cross == 'ppc64le' }}
- REF_GIT_TAG: v24.03
+ REF_GIT_TAG: none
RISCV64: ${{ matrix.config.cross == 'riscv64' }}
RUN_TESTS: ${{ contains(matrix.config.checks, 'tests') }}
STDATOMIC: ${{ contains(matrix.config.checks, 'stdatomic') }}
@@ -47,7 +47,7 @@ jobs:
checks: stdatomic
- os: ubuntu-22.04
compiler: gcc
- checks: abi+debug+doc+examples+tests
+ checks: debug+doc+examples+tests
- os: ubuntu-22.04
compiler: clang
checks: asan+doc+tests
diff --git a/ABI_VERSION b/ABI_VERSION
index 9dc0ade502..be8e64f5a3 100644
--- a/ABI_VERSION
+++ b/ABI_VERSION
@@ -1 +1 @@
-24.2
+25.0
diff --git a/VERSION b/VERSION
index 7af777b08b..7491a6c168 100644
--- a/VERSION
+++ b/VERSION
@@ -1 +1 @@
-24.07.0
+24.11.0-rc0
diff --git a/devtools/libabigail.abignore b/devtools/libabigail.abignore
index 96b16059a8..21b8cd6113 100644
--- a/devtools/libabigail.abignore
+++ b/devtools/libabigail.abignore
@@ -33,22 +33,3 @@
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; Temporary exceptions till next major ABI version ;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-[suppress_type]
- name = rte_mbuf
- type_kind = struct
- has_size_change = no
- has_data_member = {cacheline0, rearm_data, rx_descriptor_fields1, cacheline1}
-
-[suppress_type]
- name = rte_pipeline_table_entry
-
-[suppress_type]
- name = rte_rcu_qsbr
-
-[suppress_type]
- name = rte_eth_fp_ops
- has_data_member_inserted_between = {offset_of(reserved2), end}
-
-[suppress_type]
- name = rte_crypto_fp_ops
- has_data_member_inserted_between = {offset_of(reserved), end}
diff --git a/doc/guides/rel_notes/index.rst b/doc/guides/rel_notes/index.rst
index 77a92b308f..74ddae3e81 100644
--- a/doc/guides/rel_notes/index.rst
+++ b/doc/guides/rel_notes/index.rst
@@ -8,6 +8,7 @@ Release Notes
:maxdepth: 1
:numbered:
+ release_24_11
release_24_07
release_24_03
release_23_11
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
new file mode 100644
index 0000000000..0ff70d9057
--- /dev/null
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -0,0 +1,136 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright 2024 The DPDK contributors
+
+.. include:: <isonum.txt>
+
+DPDK Release 24.11
+==================
+
+.. **Read this first.**
+
+ The text in the sections below explains how to update the release notes.
+
+ Use proper spelling, capitalization and punctuation in all sections.
+
+ Variable and config names should be quoted as fixed width text:
+ ``LIKE_THIS``.
+
+ Build the docs and view the output file to ensure the changes are correct::
+
+ ninja -C build doc
+ xdg-open build/doc/guides/html/rel_notes/release_24_11.html
+
+
+New Features
+------------
+
+.. This section should contain new features added in this release.
+ Sample format:
+
+ * **Add a title in the past tense with a full stop.**
+
+ Add a short 1-2 sentence description in the past tense.
+ The description should be enough to allow someone scanning
+ the release notes to understand the new feature.
+
+ If the feature adds a lot of sub-features you can use a bullet list
+ like this:
+
+ * Added feature foo to do something.
+ * Enhanced feature bar to do something else.
+
+ Refer to the previous release notes for examples.
+
+ Suggested order in release notes items:
+ * Core libs (EAL, mempool, ring, mbuf, buses)
+ * Device abstraction libs and PMDs (ordered alphabetically by vendor name)
+ - ethdev (lib, PMDs)
+ - cryptodev (lib, PMDs)
+ - eventdev (lib, PMDs)
+ - etc
+ * Other libs
+ * Apps, Examples, Tools (if significant)
+
+ This section is a comment. Do not overwrite or remove it.
+ Also, make sure to start the actual text at the margin.
+ =======================================================
+
+
+Removed Items
+-------------
+
+.. This section should contain removed items in this release. Sample format:
+
+ * Add a short 1-2 sentence description of the removed item
+ in the past tense.
+
+ This section is a comment. Do not overwrite or remove it.
+ Also, make sure to start the actual text at the margin.
+ =======================================================
+
+
+API Changes
+-----------
+
+.. This section should contain API changes. Sample format:
+
+ * sample: Add a short 1-2 sentence description of the API change
+ which was announced in the previous releases and made in this release.
+ Start with a scope label like "ethdev:".
+ Use fixed width quotes for ``function_names`` or ``struct_names``.
+ Use the past tense.
+
+ This section is a comment. Do not overwrite or remove it.
+ Also, make sure to start the actual text at the margin.
+ =======================================================
+
+
+ABI Changes
+-----------
+
+.. This section should contain ABI changes. Sample format:
+
+ * sample: Add a short 1-2 sentence description of the ABI change
+ which was announced in the previous releases and made in this release.
+ Start with a scope label like "ethdev:".
+ Use fixed width quotes for ``function_names`` or ``struct_names``.
+ Use the past tense.
+
+ This section is a comment. Do not overwrite or remove it.
+ Also, make sure to start the actual text at the margin.
+ =======================================================
+
+
+Known Issues
+------------
+
+.. This section should contain new known issues in this release. Sample format:
+
+ * **Add title in present tense with full stop.**
+
+ Add a short 1-2 sentence description of the known issue
+ in the present tense. Add information on any known workarounds.
+
+ This section is a comment. Do not overwrite or remove it.
+ Also, make sure to start the actual text at the margin.
+ =======================================================
+
+
+Tested Platforms
+----------------
+
+.. This section should contain a list of platforms that were tested
+ with this release.
+
+ The format is:
+
+ * <vendor> platform with <vendor> <type of devices> combinations
+
+ * List of CPU
+ * List of OS
+ * List of devices
+ * Other relevant details...
+
+ This section is a comment. Do not overwrite or remove it.
+ Also, make sure to start the actual text at the margin.
+ =======================================================
diff --git a/drivers/baseband/acc/version.map b/drivers/baseband/acc/version.map
index fa39a63f0f..3f427caf67 100644
--- a/drivers/baseband/acc/version.map
+++ b/drivers/baseband/acc/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
local: *;
};
diff --git a/drivers/baseband/fpga_5gnr_fec/version.map b/drivers/baseband/fpga_5gnr_fec/version.map
index 855ce55703..fb32805028 100644
--- a/drivers/baseband/fpga_5gnr_fec/version.map
+++ b/drivers/baseband/fpga_5gnr_fec/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
local: *;
};
diff --git a/drivers/baseband/fpga_lte_fec/version.map b/drivers/baseband/fpga_lte_fec/version.map
index 2c8e60375d..f6b2961ba2 100644
--- a/drivers/baseband/fpga_lte_fec/version.map
+++ b/drivers/baseband/fpga_lte_fec/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
local: *;
};
diff --git a/drivers/bus/fslmc/version.map b/drivers/bus/fslmc/version.map
index f6bdf877bf..e19b8d1f6b 100644
--- a/drivers/bus/fslmc/version.map
+++ b/drivers/bus/fslmc/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_fslmc_vfio_mem_dmamap;
diff --git a/drivers/bus/pci/version.map b/drivers/bus/pci/version.map
index 5d9dced5b2..cd653de5ac 100644
--- a/drivers/bus/pci/version.map
+++ b/drivers/bus/pci/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_pci_dump;
diff --git a/drivers/bus/platform/version.map b/drivers/bus/platform/version.map
index 9e7111dd38..37c4a74f82 100644
--- a/drivers/bus/platform/version.map
+++ b/drivers/bus/platform/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
local: *;
};
diff --git a/drivers/bus/vdev/version.map b/drivers/bus/vdev/version.map
index 16f187734b..51a0f1d5e1 100644
--- a/drivers/bus/vdev/version.map
+++ b/drivers/bus/vdev/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_vdev_add_custom_scan;
diff --git a/drivers/bus/vmbus/version.map b/drivers/bus/vmbus/version.map
index 08b008b311..365f71529f 100644
--- a/drivers/bus/vmbus/version.map
+++ b/drivers/bus/vmbus/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_vmbus_chan_close;
diff --git a/drivers/crypto/octeontx/version.map b/drivers/crypto/octeontx/version.map
index 54a0912e76..8803f974b2 100644
--- a/drivers/crypto/octeontx/version.map
+++ b/drivers/crypto/octeontx/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
local: *;
};
diff --git a/drivers/crypto/scheduler/version.map b/drivers/crypto/scheduler/version.map
index 23380fb3c5..d7ba3874f2 100644
--- a/drivers/crypto/scheduler/version.map
+++ b/drivers/crypto/scheduler/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_cryptodev_scheduler_load_user_scheduler;
diff --git a/drivers/dma/dpaa2/version.map b/drivers/dma/dpaa2/version.map
index 713ed41f0c..fc16517f7a 100644
--- a/drivers/dma/dpaa2/version.map
+++ b/drivers/dma/dpaa2/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
local: *;
};
diff --git a/drivers/event/cnxk/version.map b/drivers/event/cnxk/version.map
index 3dd9a8fdd1..a275ec2977 100644
--- a/drivers/event/cnxk/version.map
+++ b/drivers/event/cnxk/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
local: *;
};
diff --git a/drivers/event/dlb2/version.map b/drivers/event/dlb2/version.map
index 1d0a0a75d7..c37d2302cd 100644
--- a/drivers/event/dlb2/version.map
+++ b/drivers/event/dlb2/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
local: *;
};
diff --git a/drivers/mempool/cnxk/version.map b/drivers/mempool/cnxk/version.map
index 8249417527..c2905a610e 100644
--- a/drivers/mempool/cnxk/version.map
+++ b/drivers/mempool/cnxk/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
local: *;
};
diff --git a/drivers/mempool/dpaa2/version.map b/drivers/mempool/dpaa2/version.map
index b2bf63eb79..c1acfc0c64 100644
--- a/drivers/mempool/dpaa2/version.map
+++ b/drivers/mempool/dpaa2/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_dpaa2_mbuf_from_buf_addr;
diff --git a/drivers/net/atlantic/version.map b/drivers/net/atlantic/version.map
index cbe9ee9263..5644e150ac 100644
--- a/drivers/net/atlantic/version.map
+++ b/drivers/net/atlantic/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
local: *;
};
diff --git a/drivers/net/bnxt/version.map b/drivers/net/bnxt/version.map
index ff82396ca1..d29521f990 100644
--- a/drivers/net/bnxt/version.map
+++ b/drivers/net/bnxt/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_pmd_bnxt_get_vf_rx_status;
diff --git a/drivers/net/bonding/version.map b/drivers/net/bonding/version.map
index 09ee21c55f..a309469b1f 100644
--- a/drivers/net/bonding/version.map
+++ b/drivers/net/bonding/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_eth_bond_8023ad_agg_selection_get;
diff --git a/drivers/net/cnxk/version.map b/drivers/net/cnxk/version.map
index 77f574bb16..1ad0616bdf 100644
--- a/drivers/net/cnxk/version.map
+++ b/drivers/net/cnxk/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
local: *;
};
diff --git a/drivers/net/dpaa/version.map b/drivers/net/dpaa/version.map
index c06f4a56de..3fdb63caf3 100644
--- a/drivers/net/dpaa/version.map
+++ b/drivers/net/dpaa/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_pmd_dpaa_set_tx_loopback;
diff --git a/drivers/net/dpaa2/version.map b/drivers/net/dpaa2/version.map
index 283bcb42c1..ba756d26bd 100644
--- a/drivers/net/dpaa2/version.map
+++ b/drivers/net/dpaa2/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_pmd_dpaa2_mux_flow_create;
diff --git a/drivers/net/i40e/version.map b/drivers/net/i40e/version.map
index 52b7a3269a..e5d20fee71 100644
--- a/drivers/net/i40e/version.map
+++ b/drivers/net/i40e/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_pmd_i40e_add_vf_mac_addr;
diff --git a/drivers/net/iavf/version.map b/drivers/net/iavf/version.map
index 135a4ccd3d..98de64cca2 100644
--- a/drivers/net/iavf/version.map
+++ b/drivers/net/iavf/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
local: *;
};
diff --git a/drivers/net/ice/version.map b/drivers/net/ice/version.map
index 8449e98aba..0052043264 100644
--- a/drivers/net/ice/version.map
+++ b/drivers/net/ice/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
local: *;
};
diff --git a/drivers/net/ipn3ke/version.map b/drivers/net/ipn3ke/version.map
index 4a8f5e499a..e10d44858f 100644
--- a/drivers/net/ipn3ke/version.map
+++ b/drivers/net/ipn3ke/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
local: *;
};
diff --git a/drivers/net/ixgbe/version.map b/drivers/net/ixgbe/version.map
index 9a6ef29b1d..8c4c0ca542 100644
--- a/drivers/net/ixgbe/version.map
+++ b/drivers/net/ixgbe/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_pmd_ixgbe_bypass_event_show;
diff --git a/drivers/net/mlx5/version.map b/drivers/net/mlx5/version.map
index 104fa53df6..560f7ef79b 100644
--- a/drivers/net/mlx5/version.map
+++ b/drivers/net/mlx5/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
local: *;
};
diff --git a/drivers/net/octeontx/version.map b/drivers/net/octeontx/version.map
index 219933550d..861dd3450e 100644
--- a/drivers/net/octeontx/version.map
+++ b/drivers/net/octeontx/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_octeontx_pchan_map;
diff --git a/drivers/net/ring/version.map b/drivers/net/ring/version.map
index 62d9a77f9c..3b408c8ba5 100644
--- a/drivers/net/ring/version.map
+++ b/drivers/net/ring/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_eth_from_ring;
diff --git a/drivers/net/softnic/version.map b/drivers/net/softnic/version.map
index f67475684c..15daeceb73 100644
--- a/drivers/net/softnic/version.map
+++ b/drivers/net/softnic/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_pmd_softnic_manage;
diff --git a/drivers/net/vhost/version.map b/drivers/net/vhost/version.map
index 4825afd411..63890911d8 100644
--- a/drivers/net/vhost/version.map
+++ b/drivers/net/vhost/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_eth_vhost_get_queue_event;
diff --git a/drivers/raw/ifpga/version.map b/drivers/raw/ifpga/version.map
index 7fc1b5e8ae..ebe50925a8 100644
--- a/drivers/raw/ifpga/version.map
+++ b/drivers/raw/ifpga/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_pmd_ifpga_cleanup;
diff --git a/drivers/version.map b/drivers/version.map
index 5535c79061..17cc97bda6 100644
--- a/drivers/version.map
+++ b/drivers/version.map
@@ -1,3 +1,3 @@
-DPDK_24 {
+DPDK_25 {
local: *;
};
diff --git a/lib/acl/version.map b/lib/acl/version.map
index fe3127a3a9..782b1fe464 100644
--- a/lib/acl/version.map
+++ b/lib/acl/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_acl_add_rules;
diff --git a/lib/bbdev/version.map b/lib/bbdev/version.map
index 1840d2b2a4..e0d82ff752 100644
--- a/lib/bbdev/version.map
+++ b/lib/bbdev/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_bbdev_allocate;
diff --git a/lib/bitratestats/version.map b/lib/bitratestats/version.map
index 08831a62f4..edda26d552 100644
--- a/lib/bitratestats/version.map
+++ b/lib/bitratestats/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_stats_bitrate_calc;
diff --git a/lib/bpf/version.map b/lib/bpf/version.map
index 2e957494e9..239c62a96c 100644
--- a/lib/bpf/version.map
+++ b/lib/bpf/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_bpf_convert;
diff --git a/lib/cfgfile/version.map b/lib/cfgfile/version.map
index a3fe9b62f3..927b4822fe 100644
--- a/lib/cfgfile/version.map
+++ b/lib/cfgfile/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_cfgfile_add_entry;
diff --git a/lib/cmdline/version.map b/lib/cmdline/version.map
index c2a4f95e5b..6bcfebfcec 100644
--- a/lib/cmdline/version.map
+++ b/lib/cmdline/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
cirbuf_add_buf_head;
diff --git a/lib/compressdev/version.map b/lib/compressdev/version.map
index 2461b087b5..3849ae2740 100644
--- a/lib/compressdev/version.map
+++ b/lib/compressdev/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_comp_get_feature_name;
diff --git a/lib/cryptodev/version.map b/lib/cryptodev/version.map
index fdac0d876e..594c501855 100644
--- a/lib/cryptodev/version.map
+++ b/lib/cryptodev/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
__rte_cryptodev_trace_dequeue_burst;
diff --git a/lib/distributor/version.map b/lib/distributor/version.map
index 2670c4201c..b5ec7dfaca 100644
--- a/lib/distributor/version.map
+++ b/lib/distributor/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_distributor_clear_returns;
diff --git a/lib/dmadev/version.map b/lib/dmadev/version.map
index f2df3025fc..822aaa2d3b 100644
--- a/lib/dmadev/version.map
+++ b/lib/dmadev/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_dma_close;
diff --git a/lib/eal/version.map b/lib/eal/version.map
index 3df50c3fbb..e3ff412683 100644
--- a/lib/eal/version.map
+++ b/lib/eal/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
__rte_panic;
diff --git a/lib/efd/version.map b/lib/efd/version.map
index baac60f7bc..354c7f88bd 100644
--- a/lib/efd/version.map
+++ b/lib/efd/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_efd_create;
diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
index 79f6f5293b..1669055ca5 100644
--- a/lib/ethdev/version.map
+++ b/lib/ethdev/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_eth_add_first_rx_callback;
diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
index 520b190bb8..4947bb4ec6 100644
--- a/lib/eventdev/version.map
+++ b/lib/eventdev/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
__rte_eventdev_trace_crypto_adapter_enqueue;
diff --git a/lib/fib/version.map b/lib/fib/version.map
index 62dbada6bc..c6d2769611 100644
--- a/lib/fib/version.map
+++ b/lib/fib/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_fib6_add;
diff --git a/lib/graph/version.map b/lib/graph/version.map
index c84446cdba..2c83425ddc 100644
--- a/lib/graph/version.map
+++ b/lib/graph/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
__rte_graph_mcore_dispatch_sched_node_enqueue;
diff --git a/lib/gro/version.map b/lib/gro/version.map
index 13803ec814..c21c137fcd 100644
--- a/lib/gro/version.map
+++ b/lib/gro/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_gro_ctx_create;
diff --git a/lib/gso/version.map b/lib/gso/version.map
index f159b3f199..815baeb3e5 100644
--- a/lib/gso/version.map
+++ b/lib/gso/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_gso_segment;
diff --git a/lib/hash/version.map b/lib/hash/version.map
index d348dd9196..11a5394a45 100644
--- a/lib/hash/version.map
+++ b/lib/hash/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_fbk_hash_create;
@@ -47,16 +47,16 @@ DPDK_24 {
local: *;
};
-INTERNAL {
+EXPERIMENTAL {
global:
- rte_thash_gfni_stub;
- rte_thash_gfni_bulk_stub;
+ # added in 24.07
+ rte_hash_rcu_qsbr_dq_reclaim;
};
-EXPERIMENTAL {
+INTERNAL {
global:
- # added in 24.07
- rte_hash_rcu_qsbr_dq_reclaim;
+ rte_thash_gfni_stub;
+ rte_thash_gfni_bulk_stub;
};
diff --git a/lib/ip_frag/version.map b/lib/ip_frag/version.map
index 3e7e573dc4..0c001c7bd5 100644
--- a/lib/ip_frag/version.map
+++ b/lib/ip_frag/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_ip_frag_free_death_row;
diff --git a/lib/ipsec/version.map b/lib/ipsec/version.map
index 9d01ebeadc..308f9d2e0d 100644
--- a/lib/ipsec/version.map
+++ b/lib/ipsec/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_ipsec_pkt_crypto_group;
diff --git a/lib/jobstats/version.map b/lib/jobstats/version.map
index 3b8f9d6ac4..55100e0699 100644
--- a/lib/jobstats/version.map
+++ b/lib/jobstats/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_jobstats_abort;
diff --git a/lib/kvargs/version.map b/lib/kvargs/version.map
index cda85d171f..b50f1a97a1 100644
--- a/lib/kvargs/version.map
+++ b/lib/kvargs/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_kvargs_count;
diff --git a/lib/latencystats/version.map b/lib/latencystats/version.map
index 86ded322cb..e8806c0046 100644
--- a/lib/latencystats/version.map
+++ b/lib/latencystats/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_latencystats_get;
diff --git a/lib/log/version.map b/lib/log/version.map
index 0648f8831a..19d7f9cdb6 100644
--- a/lib/log/version.map
+++ b/lib/log/version.map
@@ -1,10 +1,10 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_log;
+ rte_log_can_log;
rte_log_cur_msg_loglevel;
rte_log_cur_msg_logtype;
- rte_log_can_log;
rte_log_dump;
rte_log_get_global_level;
rte_log_get_level;
diff --git a/lib/lpm/version.map b/lib/lpm/version.map
index b6bee8c18b..29d577c24b 100644
--- a/lib/lpm/version.map
+++ b/lib/lpm/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_lpm6_add;
diff --git a/lib/mbuf/version.map b/lib/mbuf/version.map
index daa65e2bbd..76f1832924 100644
--- a/lib/mbuf/version.map
+++ b/lib/mbuf/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
__rte_pktmbuf_linearize;
diff --git a/lib/member/version.map b/lib/member/version.map
index 3aeba8826b..fdc7adacf9 100644
--- a/lib/member/version.map
+++ b/lib/member/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_member_add;
diff --git a/lib/mempool/version.map b/lib/mempool/version.map
index 5e303e5d5f..6f16d417ae 100644
--- a/lib/mempool/version.map
+++ b/lib/mempool/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_mempool_audit;
diff --git a/lib/meter/version.map b/lib/meter/version.map
index 9628bd8cd9..ae434f34b5 100644
--- a/lib/meter/version.map
+++ b/lib/meter/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_meter_srtcm_config;
diff --git a/lib/metrics/version.map b/lib/metrics/version.map
index 9766a1af5b..f9c1996a7d 100644
--- a/lib/metrics/version.map
+++ b/lib/metrics/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_metrics_deinit;
diff --git a/lib/net/version.map b/lib/net/version.map
index 3e293c4715..bec4ce23ea 100644
--- a/lib/net/version.map
+++ b/lib/net/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_eth_random_addr;
diff --git a/lib/node/version.map b/lib/node/version.map
index 6bdb944c4c..a402182fbe 100644
--- a/lib/node/version.map
+++ b/lib/node/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_node_eth_config;
diff --git a/lib/pcapng/version.map b/lib/pcapng/version.map
index 81c9652ad6..9f634b653e 100644
--- a/lib/pcapng/version.map
+++ b/lib/pcapng/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_pcapng_add_interface;
diff --git a/lib/pci/version.map b/lib/pci/version.map
index aeca8a1c9e..f0f6ffef9f 100644
--- a/lib/pci/version.map
+++ b/lib/pci/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_pci_addr_cmp;
diff --git a/lib/pdump/version.map b/lib/pdump/version.map
index ea5bd157cd..6eea4c1530 100644
--- a/lib/pdump/version.map
+++ b/lib/pdump/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_pdump_disable;
diff --git a/lib/pipeline/version.map b/lib/pipeline/version.map
index 6997b69340..b56d022664 100644
--- a/lib/pipeline/version.map
+++ b/lib/pipeline/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_pipeline_ah_packet_drop;
diff --git a/lib/port/version.map b/lib/port/version.map
index fefcf29063..98fe0b08ab 100644
--- a/lib/port/version.map
+++ b/lib/port/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_port_ethdev_reader_ops;
diff --git a/lib/power/version.map b/lib/power/version.map
index ad92a65f91..c9a226614e 100644
--- a/lib/power/version.map
+++ b/lib/power/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_power_check_env_supported;
diff --git a/lib/rawdev/version.map b/lib/rawdev/version.map
index 21064a889b..f95d5dabae 100644
--- a/lib/rawdev/version.map
+++ b/lib/rawdev/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_rawdev_close;
diff --git a/lib/rcu/version.map b/lib/rcu/version.map
index 982ffd59d9..d96c4c4109 100644
--- a/lib/rcu/version.map
+++ b/lib/rcu/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_rcu_log_type;
diff --git a/lib/reorder/version.map b/lib/reorder/version.map
index 5baeab56f8..18e97942e1 100644
--- a/lib/reorder/version.map
+++ b/lib/reorder/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_reorder_create;
diff --git a/lib/rib/version.map b/lib/rib/version.map
index 39da637f75..145d9c2602 100644
--- a/lib/rib/version.map
+++ b/lib/rib/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_rib6_create;
diff --git a/lib/ring/version.map b/lib/ring/version.map
index 9eb6e254c8..8da094a69a 100644
--- a/lib/ring/version.map
+++ b/lib/ring/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_ring_create;
diff --git a/lib/sched/version.map b/lib/sched/version.map
index be1decaeee..a6ca9ee1ad 100644
--- a/lib/sched/version.map
+++ b/lib/sched/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_approx;
diff --git a/lib/security/version.map b/lib/security/version.map
index 7709ef41a3..2a4795f31d 100644
--- a/lib/security/version.map
+++ b/lib/security/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
__rte_security_set_pkt_metadata;
diff --git a/lib/stack/version.map b/lib/stack/version.map
index d191ef7791..53c7d3d1c5 100644
--- a/lib/stack/version.map
+++ b/lib/stack/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_stack_create;
diff --git a/lib/table/version.map b/lib/table/version.map
index 6c89910732..718138554e 100644
--- a/lib/table/version.map
+++ b/lib/table/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_table_acl_ops;
diff --git a/lib/telemetry/version.map b/lib/telemetry/version.map
index 7d12c92905..2907d28aa0 100644
--- a/lib/telemetry/version.map
+++ b/lib/telemetry/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_tel_data_add_array_container;
diff --git a/lib/timer/version.map b/lib/timer/version.map
index b180708e24..3f19be22d3 100644
--- a/lib/timer/version.map
+++ b/lib/timer/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_timer_alt_dump_stats;
diff --git a/lib/vhost/version.map b/lib/vhost/version.map
index 25b52e47d2..30bc312262 100644
--- a/lib/vhost/version.map
+++ b/lib/vhost/version.map
@@ -1,4 +1,4 @@
-DPDK_24 {
+DPDK_25 {
global:
rte_vdpa_find_device_by_name;
--
2.45.2
^ permalink raw reply [relevance 12%]
* RE: [EXTERNAL] Re: [PATCH] doc: announce cryptodev change to support EDDSA
@ 2024-08-07 17:21 0% ` Gowrishankar Muthukrishnan
0 siblings, 0 replies; 200+ results
From: Gowrishankar Muthukrishnan @ 2024-08-07 17:21 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Anoob Joseph, zhangfei.gao, Kusztal, ArkadiuszX, dev, Richardson,
Bruce, ciara.power, Jerin Jacob, fanzhang.oss, Ji, Kai,
jack.bond-preston, Marchand, David, hemant.agrawal,
De Lara Guarch, Pablo, Trahe, Fiona, Doherty, Declan, matan,
ruifeng.wang, Gujjar, Abhinandan S, maxime.coquelin, chenbox,
sunilprakashrao.uttarwar, andrew.boyer, ajit.khaparde,
raveendra.padasalagi, vikas.gupta, g.singh, jianjay.zhou, Daly,
Lee
> It means we are not able to add an algo without breaking ABI.
> Is it something we can improve?
>
Sure Thomas, we will address it in our patch, ensuring the long term solution as well.
Regards,
Gowrishankar
^ permalink raw reply [relevance 0%]
* RE: [PATCH] doc: announce cryptodev changes to offload RSA in VirtIO
2024-07-31 14:26 0% ` Thomas Monjalon
@ 2024-08-07 13:31 0% ` Kusztal, ArkadiuszX
0 siblings, 0 replies; 200+ results
From: Kusztal, ArkadiuszX @ 2024-08-07 13:31 UTC (permalink / raw)
To: Thomas Monjalon, Gowrishankar Muthukrishnan
Cc: dev, dev, Anoob Joseph, Richardson, Bruce, ciara.power,
Jerin Jacob, fanzhang.oss, Ji, Kai, jack.bond-preston, Marchand,
David, hemant.agrawal, De Lara Guarch, Pablo, Trahe, Fiona,
Doherty, Declan, matan, ruifeng.wang, Gujjar, Abhinandan S,
maxime.coquelin, chenbox, sunilprakashrao.uttarwar, andrew.boyer,
ajit.khaparde, raveendra.padasalagi, vikas.gupta, zhangfei.gao,
g.singh, jianjay.zhou, Daly, Lee
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Wednesday, July 31, 2024 4:27 PM
> To: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
> Cc: dev@dpdk.org; Kusztal, ArkadiuszX <arkadiuszx.kusztal@intel.com>;
> dev@dpdk.org; Anoob Joseph <anoobj@marvell.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; ciara.power@intel.com; Jerin Jacob
> <jerinj@marvell.com>; fanzhang.oss@gmail.com; Ji, Kai <kai.ji@intel.com>;
> jack.bond-preston@foss.arm.com; Marchand, David
> <david.marchand@redhat.com>; hemant.agrawal@nxp.com; De Lara Guarch,
> Pablo <pablo.de.lara.guarch@intel.com>; Trahe, Fiona
> <fiona.trahe@intel.com>; Doherty, Declan <declan.doherty@intel.com>;
> matan@nvidia.com; ruifeng.wang@arm.com; Gujjar, Abhinandan S
> <abhinandan.gujjar@intel.com>; maxime.coquelin@redhat.com;
> chenbox@nvidia.com; sunilprakashrao.uttarwar@amd.com;
> andrew.boyer@amd.com; ajit.khaparde@broadcom.com;
> raveendra.padasalagi@broadcom.com; vikas.gupta@broadcom.com;
> zhangfei.gao@linaro.org; g.singh@nxp.com; jianjay.zhou@huawei.com; Daly,
> Lee <lee.daly@intel.com>
> Subject: Re: [PATCH] doc: announce cryptodev changes to offload RSA in VirtIO
>
> I'm not sure why we don't have a consensus on an idea proposed as RFC in
> September 2023.
>
> Because there is not enough involvement outside of the Marvell team, I will
> keep a vague announce for the first item:
>
> cryptodev: Some changes may happen to manage RSA padding for virtio-crypto.
>
> The second item is applied verbatim, thanks.
>
>
> 31/07/2024 14:51, Thomas Monjalon:
> > 30/07/2024 16:39, Gowrishankar Muthukrishnan:
> > > Hi,
> > > We need to fix padding info in DPDK as per VirtIO specification in order to
> support RSA in virtio devices. VirtIO-crypto specification and DPDK specification
> differs in the way padding is handled.
> > > With current DPDK & virtio specification, it is impossible to support RSA in
> virtio-crypto. If you think DPDK spec should not be modified, we will try to
> amend the virtIO spec to match DPDK, but since we do not know if the virtIO
> community would accept, can we merge the deprecation notice?
> >
> > There is a long list of Cc but I see no support outside of Marvell.
> >
> >
> >
> > > >>> +* cryptodev: The struct rte_crypto_rsa_padding will be moved
> > > >>> +from
> > > >>> + rte_crypto_rsa_op_param struct to rte_crypto_rsa_xform
> > > >>> +struct,
> > > >>> + breaking ABI. The new location is recommended to comply with
> > > >>> + virtio-crypto specification. Applications and drivers using
> > > >>> + this struct will be updated.
> > > >>> +
> > >
> > >
> > > >> The problem here, I see is that there is one private key but multiple
> combinations of padding.
> > > >> Therefore, for every padding variation, we need to copy the same private
> key anew, duplicating it in memory.
> > > >> The only reason for me to keep a session-like struct in asymmetric crypto
> was exactly this.
> > > >
> > > > Each padding scheme in RSA has its own pros and cons (in terms of
> implementations as well).
> > > > When we share the same private key for Sign (and its public key in
> > > > case of Encryption) between multiple crypto ops (varying by
> > > > padding schemes among cops), a vulnerable attack against one
> > > > scheme could potentially open door to used private key in the session and
> hence take advantage on other crypto operations.
> > > >
> > > > I think, this could be one reason for why VirtIO spec mandates padding info
> as session parameter.
> > > > Hence, more than duplicating in memory, private and public keys
> > > > are secured and in catastrophe, only that session could be destroyed.
Hi Gowrishankar,
Sorry for the delayed response.
I do not have any particular security issues in mind here, and if PMD need to copy keys internally, for alignment or padding purposes, redundancy problems can be overcome. My concern was, that it is the more natural way of handling the API; we have one key, multiple padding schemes, so we reflect this logic in the API.
Both options are widely used; libcrypto, for example is setting padding within session, other languages like Go, Rust are setting it as an argument to the method of the key struct.
If this is that problematic with VirtIO compatibility, I say this change is okay.
> > >
> > >
> > > >>> +* cryptodev: The rte_crypto_rsa_xform struct member to hold
> > > >>> +private key
> > > >>> + in either exponent or quintuple format is changed from union
> > > >>> +to struct
> > > >>> + data type. This change is to support ASN.1 syntax (RFC 3447 Appendix
> A.1.2).
> > > >>> + This change will not break existing applications.
> > > > >
> > > > > This one I agree. RFC 8017 obsoletes RFC 3447.
>
>
^ permalink raw reply [relevance 0%]
* [PATCH v15 5/5] dts: add API doc generation
@ 2024-08-06 15:19 2% ` Juraj Linkeš
0 siblings, 0 replies; 200+ results
From: Juraj Linkeš @ 2024-08-06 15:19 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, bruce.richardson, jspewock, probb,
paul.szczepanek, Luca.Vizzarro, npratte
Cc: dev, Juraj Linkeš
The tool used to generate DTS API docs is Sphinx, which is already in
use in DPDK. The same configuration is used to preserve style with one
DTS-specific configuration (so that the DPDK docs are unchanged) that
modifies how the sidebar displays the content.
Sphinx generates the documentation from Python docstrings. The docstring
format is the Google format [0] which requires the sphinx.ext.napoleon
extension. The other extension, sphinx.ext.intersphinx, enables linking
to objects in external documentations, such as the Python documentation.
There is one requirement for building DTS docs - the same Python version
as DTS or higher, because Sphinx's autodoc extension imports the code.
The dependencies needed to import the code don't have to be satisfied,
as the autodoc extension allows us to mock the imports. The missing
packages are taken from the DTS pyproject.toml file.
[0] https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
buildtools/call-sphinx-build.py | 2 +
buildtools/get-dts-deps.py | 78 +++++++++++++++++++++++
buildtools/meson.build | 1 +
doc/api/doxy-api-index.md | 3 +
doc/api/doxy-api.conf.in | 2 +
doc/api/dts/custom.css | 1 +
doc/api/dts/meson.build | 29 +++++++++
doc/api/meson.build | 13 ++++
doc/guides/conf.py | 41 +++++++++++-
doc/guides/contributing/documentation.rst | 2 +
doc/guides/contributing/patches.rst | 4 ++
doc/guides/tools/dts.rst | 39 +++++++++++-
doc/meson.build | 1 +
13 files changed, 214 insertions(+), 2 deletions(-)
create mode 100755 buildtools/get-dts-deps.py
create mode 120000 doc/api/dts/custom.css
create mode 100644 doc/api/dts/meson.build
diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
index 623e7363ee..45724ffcd4 100755
--- a/buildtools/call-sphinx-build.py
+++ b/buildtools/call-sphinx-build.py
@@ -15,6 +15,8 @@
# set the version in environment for sphinx to pick up
os.environ['DPDK_VERSION'] = version
+if src.find('dts') != -1:
+ os.environ['DTS_BUILD'] = "y"
sphinx_cmd = [sphinx] + extra_args
diff --git a/buildtools/get-dts-deps.py b/buildtools/get-dts-deps.py
new file mode 100755
index 0000000000..309b83cb5c
--- /dev/null
+++ b/buildtools/get-dts-deps.py
@@ -0,0 +1,78 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 PANTHEON.tech s.r.o.
+#
+
+"""Utilities for DTS dependencies.
+
+The module can be used as an executable script,
+which verifies that the running Python version meets the version requirement of DTS.
+The script returns the standard exit codes in this mode (0 is success, 1 is failure).
+
+The module also contains a function, get_missing_imports,
+which looks for runtime and doc generation dependencies in the DTS pyproject.toml file
+a returns a list of module names used in an import statement that are missing.
+"""
+
+import configparser
+import importlib.metadata
+import importlib.util
+import os.path
+import platform
+
+_VERSION_COMPARISON_CHARS = '^<>='
+_EXTRA_DEPS = {'invoke': '>=1.3', 'paramiko': '>=2.4'}
+_DPDK_ROOT = os.path.dirname(os.path.dirname(__file__))
+_DTS_DEP_FILE_PATH = os.path.join(_DPDK_ROOT, 'dts', 'pyproject.toml')
+
+
+def _get_version_tuple(version_str):
+ return tuple(map(int, version_str.split(".")))
+
+
+def _get_dependencies(cfg_file_path):
+ cfg = configparser.ConfigParser()
+ with open(cfg_file_path) as f:
+ dts_deps_file_str = f.read()
+ dts_deps_file_str = dts_deps_file_str.replace("\n]", "]")
+ cfg.read_string(dts_deps_file_str)
+
+ deps_section = cfg['tool.poetry.dependencies']
+ deps = {dep: deps_section[dep].strip('"\'') for dep in deps_section}
+ doc_deps_section = cfg['tool.poetry.group.docs.dependencies']
+ doc_deps = {dep: doc_deps_section[dep].strip("\"'") for dep in doc_deps_section}
+
+ return deps | doc_deps
+
+
+def get_missing_imports():
+ missing_imports = []
+ req_deps = _get_dependencies(_DTS_DEP_FILE_PATH)
+ req_deps.pop('python')
+
+ for req_dep, req_ver in (req_deps | _EXTRA_DEPS).items():
+ try:
+ req_ver = _get_version_tuple(req_ver.strip(_VERSION_COMPARISON_CHARS))
+ found_dep_ver = _get_version_tuple(importlib.metadata.version(req_dep))
+ if found_dep_ver < req_ver:
+ print(
+ f'The version "{found_dep_ver}" of package "{req_dep}" '
+ f'is lower than required "{req_ver}".'
+ )
+ except importlib.metadata.PackageNotFoundError:
+ print(f'Package "{req_dep}" not found.')
+ missing_imports.append(req_dep.lower().replace('-', '_'))
+
+ return missing_imports
+
+
+if __name__ == '__main__':
+ python_version = _get_dependencies(_DTS_DEP_FILE_PATH).pop('python')
+ if python_version:
+ sys_ver = _get_version_tuple(platform.python_version())
+ req_ver = _get_version_tuple(python_version.strip(_VERSION_COMPARISON_CHARS))
+ if sys_ver < req_ver:
+ print(
+ f'The available Python version "{sys_ver}" is lower than required "{req_ver}".'
+ )
+ exit(1)
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 3adf34e1a8..599653bea4 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -24,6 +24,7 @@ get_numa_count_cmd = py3 + files('get-numa-count.py')
get_test_suites_cmd = py3 + files('get-test-suites.py')
has_hugepages_cmd = py3 + files('has-hugepages.py')
cmdline_gen_cmd = py3 + files('dpdk-cmdline-gen.py')
+get_dts_deps = py3 + files('get-dts-deps.py')
# install any build tools that end-users might want also
install_data([
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index f9f0300126..ab223bcdf7 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -245,3 +245,6 @@ The public API headers are grouped by topics:
[experimental APIs](@ref rte_compat.h),
[ABI versioning](@ref rte_function_versioning.h),
[version](@ref rte_version.h)
+
+- **tests**:
+ [**DTS**](@dts_api_main_page)
diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index a8823c046f..c94f02d411 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -124,6 +124,8 @@ SEARCHENGINE = YES
SORT_MEMBER_DOCS = NO
SOURCE_BROWSER = YES
+ALIASES = "dts_api_main_page=@DTS_API_MAIN_PAGE@"
+
EXAMPLE_PATH = @TOPDIR@/examples
EXAMPLE_PATTERNS = *.c
EXAMPLE_RECURSIVE = YES
diff --git a/doc/api/dts/custom.css b/doc/api/dts/custom.css
new file mode 120000
index 0000000000..3c9480c4a0
--- /dev/null
+++ b/doc/api/dts/custom.css
@@ -0,0 +1 @@
+../../guides/custom.css
\ No newline at end of file
diff --git a/doc/api/dts/meson.build b/doc/api/dts/meson.build
new file mode 100644
index 0000000000..329b60cb1f
--- /dev/null
+++ b/doc/api/dts/meson.build
@@ -0,0 +1,29 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
+if not sphinx.found()
+ subdir_done()
+endif
+
+python_ver_satisfied = run_command(get_dts_deps).returncode()
+if python_ver_satisfied != 0
+ subdir_done()
+endif
+
+extra_sphinx_args = ['-E', '-c', join_paths(doc_source_dir, 'guides')]
+if get_option('werror')
+ extra_sphinx_args += '-W'
+endif
+
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk', 'dts')
+dts_api_html = custom_target('dts_api_html',
+ output: 'html',
+ command: [sphinx_wrapper, sphinx, meson.project_version(),
+ meson.current_source_dir(), meson.current_build_dir(), extra_sphinx_args],
+ build_by_default: get_option('enable_docs'),
+ install: get_option('enable_docs'),
+ install_dir: htmldir)
+
+dts_doc_targets += dts_api_html
+dts_doc_target_names += 'DTS_API_HTML'
diff --git a/doc/api/meson.build b/doc/api/meson.build
index 5b50692df9..788129336b 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -1,6 +1,18 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Luca Boccassi <bluca@debian.org>
+dts_doc_targets = []
+dts_doc_target_names = []
+subdir('dts')
+
+if dts_doc_targets.length() == 0
+ dts_message = 'No DTS docs targets found'
+else
+ dts_message = 'Building DTS docs:'
+endif
+run_target('dts-doc', command: [echo, dts_message, dts_doc_target_names],
+ depends: dts_doc_targets)
+
doxygen = find_program('doxygen', required: get_option('enable_docs'))
if not doxygen.found()
@@ -40,6 +52,7 @@ cdata.set('WARN_AS_ERROR', 'NO')
if get_option('werror')
cdata.set('WARN_AS_ERROR', 'YES')
endif
+cdata.set('DTS_API_MAIN_PAGE', join_paths('..', 'dts', 'html', 'index.html'))
# configure HTML Doxygen run
html_cdata = configuration_data()
diff --git a/doc/guides/conf.py b/doc/guides/conf.py
index 0f7ff5282d..eab3387874 100644
--- a/doc/guides/conf.py
+++ b/doc/guides/conf.py
@@ -10,7 +10,7 @@
from os.path import basename
from os.path import dirname
from os.path import join as path_join
-from sys import argv, stderr
+from sys import argv, stderr, path
import configparser
@@ -24,6 +24,45 @@
file=stderr)
pass
+# Napoleon enables the Google format of Python doscstrings, used in DTS.
+# Intersphinx allows linking to external projects, such as Python docs, also used in DTS.
+extensions = ['sphinx.ext.napoleon', 'sphinx.ext.intersphinx']
+
+# DTS Python docstring options.
+autodoc_default_options = {
+ 'members': True,
+ 'member-order': 'bysource',
+ 'show-inheritance': True,
+}
+autodoc_class_signature = 'separated'
+autodoc_typehints = 'both'
+autodoc_typehints_format = 'short'
+autodoc_typehints_description_target = 'documented'
+napoleon_numpy_docstring = False
+napoleon_attr_annotations = True
+napoleon_preprocess_types = True
+add_module_names = False
+toc_object_entries = True
+toc_object_entries_show_parents = 'hide'
+intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
+
+if environ.get('DTS_BUILD'):
+ # Add path to DTS sources so that Sphinx can find them.
+ dpdk_root = dirname(dirname(dirname(__file__)))
+ path.append(path_join(dpdk_root, 'dts'))
+
+ # Get missing DTS dependencies. Add path to buildtools to find the get_missing_imports function.
+ path.append(path_join(dpdk_root, 'buildtools'))
+ import importlib
+ # Ignore missing imports from DTS dependencies.
+ autodoc_mock_imports = importlib.import_module('get-dts-deps').get_missing_imports()
+
+ # DTS Sidebar config.
+ html_theme_options = {
+ 'collapse_navigation': False,
+ 'navigation_depth': -1, # unlimited depth
+ }
+
stop_on_error = ('-W' in argv)
project = 'Data Plane Development Kit'
diff --git a/doc/guides/contributing/documentation.rst b/doc/guides/contributing/documentation.rst
index 68454ae0d5..7b287ce631 100644
--- a/doc/guides/contributing/documentation.rst
+++ b/doc/guides/contributing/documentation.rst
@@ -133,6 +133,8 @@ added to by the developer.
Building the Documentation
--------------------------
+.. _doc_dependencies:
+
Dependencies
~~~~~~~~~~~~
diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst
index 04c66bebc4..6629928bee 100644
--- a/doc/guides/contributing/patches.rst
+++ b/doc/guides/contributing/patches.rst
@@ -499,6 +499,10 @@ The script usage is::
For both of the above scripts, the -n option is used to specify a number of commits from HEAD,
and the -r option allows the user specify a ``git log`` range.
+Additionally, when contributing to the DTS tool, patches should also be checked using
+the ``dts-check-format.sh`` script in the ``devtools`` directory of the DPDK repo.
+To run the script, extra :ref:`Python dependencies <dts_deps>` are needed.
+
.. _contrib_check_compilation:
Checking Compilation
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 515b15e4d8..18cc7908cf 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -54,6 +54,7 @@ DTS uses Poetry as its Python dependency management.
Python build/development and runtime environments are the same and DTS development environment,
DTS runtime environment or just plain DTS environment are used interchangeably.
+.. _dts_deps:
Setting up DTS environment
~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -291,8 +292,15 @@ When adding code to the DTS framework, pay attention to the rest of the code
and try not to divert much from it.
The :ref:`DTS developer tools <dts_dev_tools>` will issue warnings
when some of the basics are not met.
+You should also build the :ref:`API documentation <building_api_docs>`
+to address any issues found during the build.
-The code must be properly documented with docstrings.
+The API documentation, which is a helpful reference when developing, may be accessed
+in the code directly or generated with the :ref:`API docs build steps <building_api_docs>`.
+When adding new files or modifying the directory structure,
+the corresponding changes must be made to DTS api doc sources in ``doc/api/dts``.
+
+Speaking of which, the code must be properly documented with docstrings.
The style must conform to the `Google style
<https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings>`_.
See an example of the style `here
@@ -427,6 +435,35 @@ the DTS code check and format script.
Refer to the script for usage: ``devtools/dts-check-format.sh -h``.
+.. _building_api_docs:
+
+Building DTS API docs
+---------------------
+
+The documentation is built using the standard DPDK build system.
+See :doc:`../linux_gsg/build_dpdk` for more details on compiling DPDK with meson.
+
+The :ref:`doc build dependencies <doc_dependencies>` may be installed with Poetry:
+
+.. code-block:: console
+
+ poetry install --no-root --only docs
+ poetry install --no-root --with docs # an alternative that will also install DTS dependencies
+ poetry shell
+
+After executing the meson command, build the documentation with:
+
+.. code-block:: console
+
+ ninja -C build dts-doc
+
+The output is generated in ``build/doc/api/dts/html``.
+
+.. note::
+
+ Make sure to fix any Sphinx warnings when adding or updating docstrings.
+
+
Configuration Schema
--------------------
diff --git a/doc/meson.build b/doc/meson.build
index 6f74706aa2..1e0cfa4127 100644
--- a/doc/meson.build
+++ b/doc/meson.build
@@ -1,6 +1,7 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Luca Boccassi <bluca@debian.org>
+doc_source_dir = meson.current_source_dir()
doc_targets = []
doc_target_names = []
subdir('api')
--
2.34.1
^ permalink raw reply [relevance 2%]
* [PATCH v14 6/6] dts: add API doc generation
@ 2024-08-06 11:17 2% ` Juraj Linkeš
0 siblings, 0 replies; 200+ results
From: Juraj Linkeš @ 2024-08-06 11:17 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, bruce.richardson, jspewock, probb,
paul.szczepanek, Luca.Vizzarro, npratte
Cc: dev, Juraj Linkeš
The tool used to generate DTS API docs is Sphinx, which is already in
use in DPDK. The same configuration is used to preserve style with one
DTS-specific configuration (so that the DPDK docs are unchanged) that
modifies how the sidebar displays the content.
Sphinx generates the documentation from Python docstrings. The docstring
format is the Google format [0] which requires the sphinx.ext.napoleon
extension. The other extension, sphinx.ext.intersphinx, enables linking
to objects in external documentations, such as the Python documentation.
There is one requirement for building DTS docs - the same Python version
as DTS or higher, because Sphinx's autodoc extension imports the code.
The dependencies needed to import the code don't have to be satisfied,
as the autodoc extension allows us to mock the imports. The missing
packages are taken from the DTS pyproject.toml file.
The generated DTS API docs are linked with the DPDK API docs according
to their placement after installing them with 'meson install'. However,
the build path differs from the install path, requiring a symlink from
DPDK API doc build path to DTS API build path to produce the proper link
in the build directory.
[0] https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
buildtools/call-sphinx-build.py | 10 ++-
buildtools/get-dts-deps.py | 78 +++++++++++++++++++++++
buildtools/meson.build | 1 +
doc/api/doxy-api-index.md | 3 +
doc/api/doxy-api.conf.in | 2 +
doc/api/meson.build | 1 +
doc/guides/conf.py | 41 +++++++++++-
doc/guides/contributing/documentation.rst | 2 +
doc/guides/contributing/patches.rst | 4 ++
doc/guides/meson.build | 1 +
doc/guides/tools/dts.rst | 39 +++++++++++-
dts/doc/meson.build | 43 +++++++++++++
dts/meson.build | 15 +++++
meson.build | 1 +
14 files changed, 238 insertions(+), 3 deletions(-)
create mode 100755 buildtools/get-dts-deps.py
create mode 100644 dts/doc/meson.build
create mode 100644 dts/meson.build
diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
index 623e7363ee..5dd59907cd 100755
--- a/buildtools/call-sphinx-build.py
+++ b/buildtools/call-sphinx-build.py
@@ -15,6 +15,11 @@
# set the version in environment for sphinx to pick up
os.environ['DPDK_VERSION'] = version
+conf_src = src
+if src.find('dts') != -1:
+ if '-c' in extra_args:
+ conf_src = extra_args[extra_args.index('-c') + 1]
+ os.environ['DTS_BUILD'] = "y"
sphinx_cmd = [sphinx] + extra_args
@@ -23,6 +28,9 @@
for root, dirs, files in os.walk(src):
srcfiles.extend([join(root, f) for f in files])
+if not os.path.exists(dst):
+ os.makedirs(dst)
+
# run sphinx, putting the html output in a "html" directory
with open(join(dst, 'sphinx_html.out'), 'w') as out:
process = run(sphinx_cmd + ['-b', 'html', src, join(dst, 'html')],
@@ -34,7 +42,7 @@
# copy custom CSS file
css = 'custom.css'
-src_css = join(src, css)
+src_css = join(conf_src, css)
dst_css = join(dst, 'html', '_static', 'css', css)
if not os.path.exists(dst_css) or not filecmp.cmp(src_css, dst_css):
os.makedirs(os.path.dirname(dst_css), exist_ok=True)
diff --git a/buildtools/get-dts-deps.py b/buildtools/get-dts-deps.py
new file mode 100755
index 0000000000..309b83cb5c
--- /dev/null
+++ b/buildtools/get-dts-deps.py
@@ -0,0 +1,78 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 PANTHEON.tech s.r.o.
+#
+
+"""Utilities for DTS dependencies.
+
+The module can be used as an executable script,
+which verifies that the running Python version meets the version requirement of DTS.
+The script returns the standard exit codes in this mode (0 is success, 1 is failure).
+
+The module also contains a function, get_missing_imports,
+which looks for runtime and doc generation dependencies in the DTS pyproject.toml file
+a returns a list of module names used in an import statement that are missing.
+"""
+
+import configparser
+import importlib.metadata
+import importlib.util
+import os.path
+import platform
+
+_VERSION_COMPARISON_CHARS = '^<>='
+_EXTRA_DEPS = {'invoke': '>=1.3', 'paramiko': '>=2.4'}
+_DPDK_ROOT = os.path.dirname(os.path.dirname(__file__))
+_DTS_DEP_FILE_PATH = os.path.join(_DPDK_ROOT, 'dts', 'pyproject.toml')
+
+
+def _get_version_tuple(version_str):
+ return tuple(map(int, version_str.split(".")))
+
+
+def _get_dependencies(cfg_file_path):
+ cfg = configparser.ConfigParser()
+ with open(cfg_file_path) as f:
+ dts_deps_file_str = f.read()
+ dts_deps_file_str = dts_deps_file_str.replace("\n]", "]")
+ cfg.read_string(dts_deps_file_str)
+
+ deps_section = cfg['tool.poetry.dependencies']
+ deps = {dep: deps_section[dep].strip('"\'') for dep in deps_section}
+ doc_deps_section = cfg['tool.poetry.group.docs.dependencies']
+ doc_deps = {dep: doc_deps_section[dep].strip("\"'") for dep in doc_deps_section}
+
+ return deps | doc_deps
+
+
+def get_missing_imports():
+ missing_imports = []
+ req_deps = _get_dependencies(_DTS_DEP_FILE_PATH)
+ req_deps.pop('python')
+
+ for req_dep, req_ver in (req_deps | _EXTRA_DEPS).items():
+ try:
+ req_ver = _get_version_tuple(req_ver.strip(_VERSION_COMPARISON_CHARS))
+ found_dep_ver = _get_version_tuple(importlib.metadata.version(req_dep))
+ if found_dep_ver < req_ver:
+ print(
+ f'The version "{found_dep_ver}" of package "{req_dep}" '
+ f'is lower than required "{req_ver}".'
+ )
+ except importlib.metadata.PackageNotFoundError:
+ print(f'Package "{req_dep}" not found.')
+ missing_imports.append(req_dep.lower().replace('-', '_'))
+
+ return missing_imports
+
+
+if __name__ == '__main__':
+ python_version = _get_dependencies(_DTS_DEP_FILE_PATH).pop('python')
+ if python_version:
+ sys_ver = _get_version_tuple(platform.python_version())
+ req_ver = _get_version_tuple(python_version.strip(_VERSION_COMPARISON_CHARS))
+ if sys_ver < req_ver:
+ print(
+ f'The available Python version "{sys_ver}" is lower than required "{req_ver}".'
+ )
+ exit(1)
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 3adf34e1a8..599653bea4 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -24,6 +24,7 @@ get_numa_count_cmd = py3 + files('get-numa-count.py')
get_test_suites_cmd = py3 + files('get-test-suites.py')
has_hugepages_cmd = py3 + files('has-hugepages.py')
cmdline_gen_cmd = py3 + files('dpdk-cmdline-gen.py')
+get_dts_deps = py3 + files('get-dts-deps.py')
# install any build tools that end-users might want also
install_data([
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index f9f0300126..ab223bcdf7 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -245,3 +245,6 @@ The public API headers are grouped by topics:
[experimental APIs](@ref rte_compat.h),
[ABI versioning](@ref rte_function_versioning.h),
[version](@ref rte_version.h)
+
+- **tests**:
+ [**DTS**](@dts_api_main_page)
diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index a8823c046f..c94f02d411 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -124,6 +124,8 @@ SEARCHENGINE = YES
SORT_MEMBER_DOCS = NO
SOURCE_BROWSER = YES
+ALIASES = "dts_api_main_page=@DTS_API_MAIN_PAGE@"
+
EXAMPLE_PATH = @TOPDIR@/examples
EXAMPLE_PATTERNS = *.c
EXAMPLE_RECURSIVE = YES
diff --git a/doc/api/meson.build b/doc/api/meson.build
index b828b1ed66..b893931b92 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -41,6 +41,7 @@ cdata.set('WARN_AS_ERROR', 'NO')
if get_option('werror')
cdata.set('WARN_AS_ERROR', 'YES')
endif
+cdata.set('DTS_API_MAIN_PAGE', join_paths('..', 'dts', 'html', 'index.html'))
# configure HTML Doxygen run
html_cdata = configuration_data()
diff --git a/doc/guides/conf.py b/doc/guides/conf.py
index 0f7ff5282d..eab3387874 100644
--- a/doc/guides/conf.py
+++ b/doc/guides/conf.py
@@ -10,7 +10,7 @@
from os.path import basename
from os.path import dirname
from os.path import join as path_join
-from sys import argv, stderr
+from sys import argv, stderr, path
import configparser
@@ -24,6 +24,45 @@
file=stderr)
pass
+# Napoleon enables the Google format of Python doscstrings, used in DTS.
+# Intersphinx allows linking to external projects, such as Python docs, also used in DTS.
+extensions = ['sphinx.ext.napoleon', 'sphinx.ext.intersphinx']
+
+# DTS Python docstring options.
+autodoc_default_options = {
+ 'members': True,
+ 'member-order': 'bysource',
+ 'show-inheritance': True,
+}
+autodoc_class_signature = 'separated'
+autodoc_typehints = 'both'
+autodoc_typehints_format = 'short'
+autodoc_typehints_description_target = 'documented'
+napoleon_numpy_docstring = False
+napoleon_attr_annotations = True
+napoleon_preprocess_types = True
+add_module_names = False
+toc_object_entries = True
+toc_object_entries_show_parents = 'hide'
+intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
+
+if environ.get('DTS_BUILD'):
+ # Add path to DTS sources so that Sphinx can find them.
+ dpdk_root = dirname(dirname(dirname(__file__)))
+ path.append(path_join(dpdk_root, 'dts'))
+
+ # Get missing DTS dependencies. Add path to buildtools to find the get_missing_imports function.
+ path.append(path_join(dpdk_root, 'buildtools'))
+ import importlib
+ # Ignore missing imports from DTS dependencies.
+ autodoc_mock_imports = importlib.import_module('get-dts-deps').get_missing_imports()
+
+ # DTS Sidebar config.
+ html_theme_options = {
+ 'collapse_navigation': False,
+ 'navigation_depth': -1, # unlimited depth
+ }
+
stop_on_error = ('-W' in argv)
project = 'Data Plane Development Kit'
diff --git a/doc/guides/contributing/documentation.rst b/doc/guides/contributing/documentation.rst
index 68454ae0d5..7b287ce631 100644
--- a/doc/guides/contributing/documentation.rst
+++ b/doc/guides/contributing/documentation.rst
@@ -133,6 +133,8 @@ added to by the developer.
Building the Documentation
--------------------------
+.. _doc_dependencies:
+
Dependencies
~~~~~~~~~~~~
diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst
index 04c66bebc4..6629928bee 100644
--- a/doc/guides/contributing/patches.rst
+++ b/doc/guides/contributing/patches.rst
@@ -499,6 +499,10 @@ The script usage is::
For both of the above scripts, the -n option is used to specify a number of commits from HEAD,
and the -r option allows the user specify a ``git log`` range.
+Additionally, when contributing to the DTS tool, patches should also be checked using
+the ``dts-check-format.sh`` script in the ``devtools`` directory of the DPDK repo.
+To run the script, extra :ref:`Python dependencies <dts_deps>` are needed.
+
.. _contrib_check_compilation:
Checking Compilation
diff --git a/doc/guides/meson.build b/doc/guides/meson.build
index f8bbfba9f5..b34b7b8eb0 100644
--- a/doc/guides/meson.build
+++ b/doc/guides/meson.build
@@ -1,6 +1,7 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Intel Corporation
+doc_guides_source_dir = meson.current_source_dir()
sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
if not sphinx.found()
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 515b15e4d8..bd715f8072 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -54,6 +54,7 @@ DTS uses Poetry as its Python dependency management.
Python build/development and runtime environments are the same and DTS development environment,
DTS runtime environment or just plain DTS environment are used interchangeably.
+.. _dts_deps:
Setting up DTS environment
~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -291,8 +292,15 @@ When adding code to the DTS framework, pay attention to the rest of the code
and try not to divert much from it.
The :ref:`DTS developer tools <dts_dev_tools>` will issue warnings
when some of the basics are not met.
+You should also build the :ref:`API documentation <building_api_docs>`
+to address any issues found during the build.
-The code must be properly documented with docstrings.
+The API documentation, which is a helpful reference when developing, may be accessed
+in the code directly or generated with the :ref:`API docs build steps <building_api_docs>`.
+When adding new files or modifying the directory structure,
+the corresponding changes must be made to DTS api doc sources in ``dts/doc``.
+
+Speaking of which, the code must be properly documented with docstrings.
The style must conform to the `Google style
<https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings>`_.
See an example of the style `here
@@ -427,6 +435,35 @@ the DTS code check and format script.
Refer to the script for usage: ``devtools/dts-check-format.sh -h``.
+.. _building_api_docs:
+
+Building DTS API docs
+---------------------
+
+The documentation is built using the standard DPDK build system.
+See :doc:`../linux_gsg/build_dpdk` for more details on compiling DPDK with meson.
+
+The :ref:`doc build dependencies <doc_dependencies>` may be installed with Poetry:
+
+.. code-block:: console
+
+ poetry install --no-root --only docs
+ poetry install --no-root --with docs # an alternative that will also install DTS dependencies
+ poetry shell
+
+After executing the meson command, build the documentation with:
+
+.. code-block:: console
+
+ ninja -C build dts-doc
+
+The output is generated in ``build/doc/api/dts/html``.
+
+.. note::
+
+ Make sure to fix any Sphinx warnings when adding or updating docstrings.
+
+
Configuration Schema
--------------------
diff --git a/dts/doc/meson.build b/dts/doc/meson.build
new file mode 100644
index 0000000000..d48a7f2003
--- /dev/null
+++ b/dts/doc/meson.build
@@ -0,0 +1,43 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
+if not sphinx.found()
+ subdir_done()
+endif
+
+python_ver_satisfied = run_command(get_dts_deps).returncode()
+if python_ver_satisfied != 0
+ subdir_done()
+endif
+
+dts_doc_api_build_dir = join_paths(doc_api_build_dir, 'dts')
+
+extra_sphinx_args = ['-E', '-c', doc_guides_source_dir]
+if get_option('werror')
+ extra_sphinx_args += '-W'
+endif
+
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk', 'dts')
+dts_api_html = custom_target('dts_api_html',
+ output: 'html',
+ command: [sphinx_wrapper, sphinx, meson.project_version(),
+ meson.current_source_dir(), meson.current_build_dir(), extra_sphinx_args],
+ build_by_default: get_option('enable_docs'),
+ install: get_option('enable_docs'),
+ install_dir: htmldir)
+
+doc_targets += dts_api_html
+doc_target_names += 'DTS_API_HTML'
+
+build_html_dir = join_paths(meson.current_build_dir(), 'html')
+dts_api_symlink = custom_target('dts_api_symlink',
+ output: 'symlink',
+ depends: dts_api_html,
+ command: ['mkdir', '-p', dts_doc_api_build_dir, '&&',
+ 'ln', '-sf', build_html_dir, dts_doc_api_build_dir],
+ build_by_default: get_option('enable_docs'),
+ install: false)
+
+doc_targets += dts_api_symlink
+doc_target_names += 'DTS_API_SYMLINK'
diff --git a/dts/meson.build b/dts/meson.build
new file mode 100644
index 0000000000..6ed3c93fe1
--- /dev/null
+++ b/dts/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+doc_targets = []
+doc_target_names = []
+
+subdir('doc')
+
+if doc_targets.length() == 0
+ message = 'No docs targets found'
+else
+ message = 'Built docs:'
+endif
+run_target('dts-doc', command: [echo, message, doc_target_names],
+ depends: doc_targets)
diff --git a/meson.build b/meson.build
index 8b248d4505..835973a0ce 100644
--- a/meson.build
+++ b/meson.build
@@ -87,6 +87,7 @@ subdir('app')
# build docs
subdir('doc')
+subdir('dts')
# build any examples explicitly requested - useful for developers - and
# install any example code into the appropriate install path
--
2.34.1
^ permalink raw reply [relevance 2%]
* [PATCH v13 6/6] dts: add API doc generation
@ 2024-08-06 8:46 2% ` Juraj Linkeš
0 siblings, 0 replies; 200+ results
From: Juraj Linkeš @ 2024-08-06 8:46 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, bruce.richardson, jspewock, probb,
paul.szczepanek, Luca.Vizzarro, npratte
Cc: dev, Juraj Linkeš
The tool used to generate DTS API docs is Sphinx, which is already in
use in DPDK. The same configuration is used to preserve style with one
DTS-specific configuration (so that the DPDK docs are unchanged) that
modifies how the sidebar displays the content.
Sphinx generates the documentation from Python docstrings. The docstring
format is the Google format [0] which requires the sphinx.ext.napoleon
extension. The other extension, sphinx.ext.intersphinx, enables linking
to objects in external documentations, such as the Python documentation.
There is one requirement for building DTS docs - the same Python version
as DTS or higher, because Sphinx's autodoc extension imports the code.
The dependencies needed to import the code don't have to be satisfied,
as the autodoc extension allows us to mock the imports. The missing
packages are taken from the DTS pyproject.toml file.
[0] https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
buildtools/call-sphinx-build.py | 10 ++-
buildtools/get-dts-deps.py | 78 +++++++++++++++++++++++
buildtools/meson.build | 1 +
doc/api/doxy-api-index.md | 3 +
doc/api/doxy-api.conf.in | 2 +
doc/api/meson.build | 1 +
doc/guides/conf.py | 41 +++++++++++-
doc/guides/contributing/documentation.rst | 2 +
doc/guides/contributing/patches.rst | 4 ++
doc/guides/meson.build | 1 +
doc/guides/tools/dts.rst | 39 +++++++++++-
dts/doc/meson.build | 30 +++++++++
dts/meson.build | 15 +++++
meson.build | 1 +
14 files changed, 225 insertions(+), 3 deletions(-)
create mode 100755 buildtools/get-dts-deps.py
create mode 100644 dts/doc/meson.build
create mode 100644 dts/meson.build
diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
index 623e7363ee..5dd59907cd 100755
--- a/buildtools/call-sphinx-build.py
+++ b/buildtools/call-sphinx-build.py
@@ -15,6 +15,11 @@
# set the version in environment for sphinx to pick up
os.environ['DPDK_VERSION'] = version
+conf_src = src
+if src.find('dts') != -1:
+ if '-c' in extra_args:
+ conf_src = extra_args[extra_args.index('-c') + 1]
+ os.environ['DTS_BUILD'] = "y"
sphinx_cmd = [sphinx] + extra_args
@@ -23,6 +28,9 @@
for root, dirs, files in os.walk(src):
srcfiles.extend([join(root, f) for f in files])
+if not os.path.exists(dst):
+ os.makedirs(dst)
+
# run sphinx, putting the html output in a "html" directory
with open(join(dst, 'sphinx_html.out'), 'w') as out:
process = run(sphinx_cmd + ['-b', 'html', src, join(dst, 'html')],
@@ -34,7 +42,7 @@
# copy custom CSS file
css = 'custom.css'
-src_css = join(src, css)
+src_css = join(conf_src, css)
dst_css = join(dst, 'html', '_static', 'css', css)
if not os.path.exists(dst_css) or not filecmp.cmp(src_css, dst_css):
os.makedirs(os.path.dirname(dst_css), exist_ok=True)
diff --git a/buildtools/get-dts-deps.py b/buildtools/get-dts-deps.py
new file mode 100755
index 0000000000..309b83cb5c
--- /dev/null
+++ b/buildtools/get-dts-deps.py
@@ -0,0 +1,78 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 PANTHEON.tech s.r.o.
+#
+
+"""Utilities for DTS dependencies.
+
+The module can be used as an executable script,
+which verifies that the running Python version meets the version requirement of DTS.
+The script returns the standard exit codes in this mode (0 is success, 1 is failure).
+
+The module also contains a function, get_missing_imports,
+which looks for runtime and doc generation dependencies in the DTS pyproject.toml file
+a returns a list of module names used in an import statement that are missing.
+"""
+
+import configparser
+import importlib.metadata
+import importlib.util
+import os.path
+import platform
+
+_VERSION_COMPARISON_CHARS = '^<>='
+_EXTRA_DEPS = {'invoke': '>=1.3', 'paramiko': '>=2.4'}
+_DPDK_ROOT = os.path.dirname(os.path.dirname(__file__))
+_DTS_DEP_FILE_PATH = os.path.join(_DPDK_ROOT, 'dts', 'pyproject.toml')
+
+
+def _get_version_tuple(version_str):
+ return tuple(map(int, version_str.split(".")))
+
+
+def _get_dependencies(cfg_file_path):
+ cfg = configparser.ConfigParser()
+ with open(cfg_file_path) as f:
+ dts_deps_file_str = f.read()
+ dts_deps_file_str = dts_deps_file_str.replace("\n]", "]")
+ cfg.read_string(dts_deps_file_str)
+
+ deps_section = cfg['tool.poetry.dependencies']
+ deps = {dep: deps_section[dep].strip('"\'') for dep in deps_section}
+ doc_deps_section = cfg['tool.poetry.group.docs.dependencies']
+ doc_deps = {dep: doc_deps_section[dep].strip("\"'") for dep in doc_deps_section}
+
+ return deps | doc_deps
+
+
+def get_missing_imports():
+ missing_imports = []
+ req_deps = _get_dependencies(_DTS_DEP_FILE_PATH)
+ req_deps.pop('python')
+
+ for req_dep, req_ver in (req_deps | _EXTRA_DEPS).items():
+ try:
+ req_ver = _get_version_tuple(req_ver.strip(_VERSION_COMPARISON_CHARS))
+ found_dep_ver = _get_version_tuple(importlib.metadata.version(req_dep))
+ if found_dep_ver < req_ver:
+ print(
+ f'The version "{found_dep_ver}" of package "{req_dep}" '
+ f'is lower than required "{req_ver}".'
+ )
+ except importlib.metadata.PackageNotFoundError:
+ print(f'Package "{req_dep}" not found.')
+ missing_imports.append(req_dep.lower().replace('-', '_'))
+
+ return missing_imports
+
+
+if __name__ == '__main__':
+ python_version = _get_dependencies(_DTS_DEP_FILE_PATH).pop('python')
+ if python_version:
+ sys_ver = _get_version_tuple(platform.python_version())
+ req_ver = _get_version_tuple(python_version.strip(_VERSION_COMPARISON_CHARS))
+ if sys_ver < req_ver:
+ print(
+ f'The available Python version "{sys_ver}" is lower than required "{req_ver}".'
+ )
+ exit(1)
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 3adf34e1a8..599653bea4 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -24,6 +24,7 @@ get_numa_count_cmd = py3 + files('get-numa-count.py')
get_test_suites_cmd = py3 + files('get-test-suites.py')
has_hugepages_cmd = py3 + files('has-hugepages.py')
cmdline_gen_cmd = py3 + files('dpdk-cmdline-gen.py')
+get_dts_deps = py3 + files('get-dts-deps.py')
# install any build tools that end-users might want also
install_data([
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index f9f0300126..ab223bcdf7 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -245,3 +245,6 @@ The public API headers are grouped by topics:
[experimental APIs](@ref rte_compat.h),
[ABI versioning](@ref rte_function_versioning.h),
[version](@ref rte_version.h)
+
+- **tests**:
+ [**DTS**](@dts_api_main_page)
diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index a8823c046f..c94f02d411 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -124,6 +124,8 @@ SEARCHENGINE = YES
SORT_MEMBER_DOCS = NO
SOURCE_BROWSER = YES
+ALIASES = "dts_api_main_page=@DTS_API_MAIN_PAGE@"
+
EXAMPLE_PATH = @TOPDIR@/examples
EXAMPLE_PATTERNS = *.c
EXAMPLE_RECURSIVE = YES
diff --git a/doc/api/meson.build b/doc/api/meson.build
index b828b1ed66..b893931b92 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -41,6 +41,7 @@ cdata.set('WARN_AS_ERROR', 'NO')
if get_option('werror')
cdata.set('WARN_AS_ERROR', 'YES')
endif
+cdata.set('DTS_API_MAIN_PAGE', join_paths('..', 'dts', 'html', 'index.html'))
# configure HTML Doxygen run
html_cdata = configuration_data()
diff --git a/doc/guides/conf.py b/doc/guides/conf.py
index 0f7ff5282d..eab3387874 100644
--- a/doc/guides/conf.py
+++ b/doc/guides/conf.py
@@ -10,7 +10,7 @@
from os.path import basename
from os.path import dirname
from os.path import join as path_join
-from sys import argv, stderr
+from sys import argv, stderr, path
import configparser
@@ -24,6 +24,45 @@
file=stderr)
pass
+# Napoleon enables the Google format of Python doscstrings, used in DTS.
+# Intersphinx allows linking to external projects, such as Python docs, also used in DTS.
+extensions = ['sphinx.ext.napoleon', 'sphinx.ext.intersphinx']
+
+# DTS Python docstring options.
+autodoc_default_options = {
+ 'members': True,
+ 'member-order': 'bysource',
+ 'show-inheritance': True,
+}
+autodoc_class_signature = 'separated'
+autodoc_typehints = 'both'
+autodoc_typehints_format = 'short'
+autodoc_typehints_description_target = 'documented'
+napoleon_numpy_docstring = False
+napoleon_attr_annotations = True
+napoleon_preprocess_types = True
+add_module_names = False
+toc_object_entries = True
+toc_object_entries_show_parents = 'hide'
+intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
+
+if environ.get('DTS_BUILD'):
+ # Add path to DTS sources so that Sphinx can find them.
+ dpdk_root = dirname(dirname(dirname(__file__)))
+ path.append(path_join(dpdk_root, 'dts'))
+
+ # Get missing DTS dependencies. Add path to buildtools to find the get_missing_imports function.
+ path.append(path_join(dpdk_root, 'buildtools'))
+ import importlib
+ # Ignore missing imports from DTS dependencies.
+ autodoc_mock_imports = importlib.import_module('get-dts-deps').get_missing_imports()
+
+ # DTS Sidebar config.
+ html_theme_options = {
+ 'collapse_navigation': False,
+ 'navigation_depth': -1, # unlimited depth
+ }
+
stop_on_error = ('-W' in argv)
project = 'Data Plane Development Kit'
diff --git a/doc/guides/contributing/documentation.rst b/doc/guides/contributing/documentation.rst
index 68454ae0d5..7b287ce631 100644
--- a/doc/guides/contributing/documentation.rst
+++ b/doc/guides/contributing/documentation.rst
@@ -133,6 +133,8 @@ added to by the developer.
Building the Documentation
--------------------------
+.. _doc_dependencies:
+
Dependencies
~~~~~~~~~~~~
diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst
index 04c66bebc4..6629928bee 100644
--- a/doc/guides/contributing/patches.rst
+++ b/doc/guides/contributing/patches.rst
@@ -499,6 +499,10 @@ The script usage is::
For both of the above scripts, the -n option is used to specify a number of commits from HEAD,
and the -r option allows the user specify a ``git log`` range.
+Additionally, when contributing to the DTS tool, patches should also be checked using
+the ``dts-check-format.sh`` script in the ``devtools`` directory of the DPDK repo.
+To run the script, extra :ref:`Python dependencies <dts_deps>` are needed.
+
.. _contrib_check_compilation:
Checking Compilation
diff --git a/doc/guides/meson.build b/doc/guides/meson.build
index f8bbfba9f5..b34b7b8eb0 100644
--- a/doc/guides/meson.build
+++ b/doc/guides/meson.build
@@ -1,6 +1,7 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Intel Corporation
+doc_guides_source_dir = meson.current_source_dir()
sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
if not sphinx.found()
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 515b15e4d8..bd715f8072 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -54,6 +54,7 @@ DTS uses Poetry as its Python dependency management.
Python build/development and runtime environments are the same and DTS development environment,
DTS runtime environment or just plain DTS environment are used interchangeably.
+.. _dts_deps:
Setting up DTS environment
~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -291,8 +292,15 @@ When adding code to the DTS framework, pay attention to the rest of the code
and try not to divert much from it.
The :ref:`DTS developer tools <dts_dev_tools>` will issue warnings
when some of the basics are not met.
+You should also build the :ref:`API documentation <building_api_docs>`
+to address any issues found during the build.
-The code must be properly documented with docstrings.
+The API documentation, which is a helpful reference when developing, may be accessed
+in the code directly or generated with the :ref:`API docs build steps <building_api_docs>`.
+When adding new files or modifying the directory structure,
+the corresponding changes must be made to DTS api doc sources in ``dts/doc``.
+
+Speaking of which, the code must be properly documented with docstrings.
The style must conform to the `Google style
<https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings>`_.
See an example of the style `here
@@ -427,6 +435,35 @@ the DTS code check and format script.
Refer to the script for usage: ``devtools/dts-check-format.sh -h``.
+.. _building_api_docs:
+
+Building DTS API docs
+---------------------
+
+The documentation is built using the standard DPDK build system.
+See :doc:`../linux_gsg/build_dpdk` for more details on compiling DPDK with meson.
+
+The :ref:`doc build dependencies <doc_dependencies>` may be installed with Poetry:
+
+.. code-block:: console
+
+ poetry install --no-root --only docs
+ poetry install --no-root --with docs # an alternative that will also install DTS dependencies
+ poetry shell
+
+After executing the meson command, build the documentation with:
+
+.. code-block:: console
+
+ ninja -C build dts-doc
+
+The output is generated in ``build/doc/api/dts/html``.
+
+.. note::
+
+ Make sure to fix any Sphinx warnings when adding or updating docstrings.
+
+
Configuration Schema
--------------------
diff --git a/dts/doc/meson.build b/dts/doc/meson.build
new file mode 100644
index 0000000000..c2df99bbc6
--- /dev/null
+++ b/dts/doc/meson.build
@@ -0,0 +1,30 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
+if not sphinx.found()
+ subdir_done()
+endif
+
+python_ver_satisfied = run_command(get_dts_deps).returncode()
+if python_ver_satisfied != 0
+ subdir_done()
+endif
+
+dts_doc_api_build_dir = join_paths(doc_api_build_dir, 'dts')
+
+extra_sphinx_args = ['-E', '-c', doc_guides_source_dir]
+if get_option('werror')
+ extra_sphinx_args += '-W'
+endif
+
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk', 'dts')
+dts_api_html = custom_target('dts_api_html',
+ output: 'html',
+ command: [sphinx_wrapper, sphinx, meson.project_version(),
+ meson.current_source_dir(), dts_doc_api_build_dir, extra_sphinx_args],
+ build_by_default: get_option('enable_docs'),
+ install: get_option('enable_docs'),
+ install_dir: htmldir)
+doc_targets += dts_api_html
+doc_target_names += 'DTS_API_HTML'
diff --git a/dts/meson.build b/dts/meson.build
new file mode 100644
index 0000000000..6ed3c93fe1
--- /dev/null
+++ b/dts/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+doc_targets = []
+doc_target_names = []
+
+subdir('doc')
+
+if doc_targets.length() == 0
+ message = 'No docs targets found'
+else
+ message = 'Built docs:'
+endif
+run_target('dts-doc', command: [echo, message, doc_target_names],
+ depends: doc_targets)
diff --git a/meson.build b/meson.build
index 8b248d4505..835973a0ce 100644
--- a/meson.build
+++ b/meson.build
@@ -87,6 +87,7 @@ subdir('app')
# build docs
subdir('doc')
+subdir('dts')
# build any examples explicitly requested - useful for developers - and
# install any example code into the appropriate install path
--
2.34.1
^ permalink raw reply [relevance 2%]
* [PATCH v12 5/5] dts: add API doc generation
@ 2024-08-06 6:14 2% ` Juraj Linkeš
0 siblings, 0 replies; 200+ results
From: Juraj Linkeš @ 2024-08-06 6:14 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, bruce.richardson, jspewock, probb,
paul.szczepanek, Luca.Vizzarro, npratte
Cc: dev, Juraj Linkeš
The tool used to generate DTS API docs is Sphinx, which is already in
use in DPDK. The same configuration is used to preserve style with one
DTS-specific configuration (so that the DPDK docs are unchanged) that
modifies how the sidebar displays the content.
Sphinx generates the documentation from Python docstrings. The docstring
format is the Google format [0] which requires the sphinx.ext.napoleon
extension. The other extension, sphinx.ext.intersphinx, enables linking
to objects in external documentations, such as the Python documentation.
There is one requirement for building DTS docs - the same Python version
as DTS or higher, because Sphinx's autodoc extension imports the code.
The dependencies needed to import the code don't have to be satisfied,
as the autodoc extension allows us to mock the imports. The missing
packages are taken from the DTS pyproject.toml file.
[0] https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
buildtools/call-sphinx-build.py | 10 ++-
buildtools/get-dts-deps.py | 78 +++++++++++++++++++++++
buildtools/meson.build | 1 +
doc/api/doxy-api-index.md | 3 +
doc/api/doxy-api.conf.in | 2 +
doc/api/meson.build | 1 +
doc/guides/conf.py | 41 +++++++++++-
doc/guides/contributing/documentation.rst | 2 +
doc/guides/contributing/patches.rst | 4 ++
doc/guides/meson.build | 1 +
doc/guides/tools/dts.rst | 39 +++++++++++-
dts/doc/meson.build | 30 +++++++++
dts/meson.build | 15 +++++
meson.build | 1 +
14 files changed, 225 insertions(+), 3 deletions(-)
create mode 100755 buildtools/get-dts-deps.py
create mode 100644 dts/doc/meson.build
create mode 100644 dts/meson.build
diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
index 623e7363ee..5dd59907cd 100755
--- a/buildtools/call-sphinx-build.py
+++ b/buildtools/call-sphinx-build.py
@@ -15,6 +15,11 @@
# set the version in environment for sphinx to pick up
os.environ['DPDK_VERSION'] = version
+conf_src = src
+if src.find('dts') != -1:
+ if '-c' in extra_args:
+ conf_src = extra_args[extra_args.index('-c') + 1]
+ os.environ['DTS_BUILD'] = "y"
sphinx_cmd = [sphinx] + extra_args
@@ -23,6 +28,9 @@
for root, dirs, files in os.walk(src):
srcfiles.extend([join(root, f) for f in files])
+if not os.path.exists(dst):
+ os.makedirs(dst)
+
# run sphinx, putting the html output in a "html" directory
with open(join(dst, 'sphinx_html.out'), 'w') as out:
process = run(sphinx_cmd + ['-b', 'html', src, join(dst, 'html')],
@@ -34,7 +42,7 @@
# copy custom CSS file
css = 'custom.css'
-src_css = join(src, css)
+src_css = join(conf_src, css)
dst_css = join(dst, 'html', '_static', 'css', css)
if not os.path.exists(dst_css) or not filecmp.cmp(src_css, dst_css):
os.makedirs(os.path.dirname(dst_css), exist_ok=True)
diff --git a/buildtools/get-dts-deps.py b/buildtools/get-dts-deps.py
new file mode 100755
index 0000000000..309b83cb5c
--- /dev/null
+++ b/buildtools/get-dts-deps.py
@@ -0,0 +1,78 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 PANTHEON.tech s.r.o.
+#
+
+"""Utilities for DTS dependencies.
+
+The module can be used as an executable script,
+which verifies that the running Python version meets the version requirement of DTS.
+The script returns the standard exit codes in this mode (0 is success, 1 is failure).
+
+The module also contains a function, get_missing_imports,
+which looks for runtime and doc generation dependencies in the DTS pyproject.toml file
+a returns a list of module names used in an import statement that are missing.
+"""
+
+import configparser
+import importlib.metadata
+import importlib.util
+import os.path
+import platform
+
+_VERSION_COMPARISON_CHARS = '^<>='
+_EXTRA_DEPS = {'invoke': '>=1.3', 'paramiko': '>=2.4'}
+_DPDK_ROOT = os.path.dirname(os.path.dirname(__file__))
+_DTS_DEP_FILE_PATH = os.path.join(_DPDK_ROOT, 'dts', 'pyproject.toml')
+
+
+def _get_version_tuple(version_str):
+ return tuple(map(int, version_str.split(".")))
+
+
+def _get_dependencies(cfg_file_path):
+ cfg = configparser.ConfigParser()
+ with open(cfg_file_path) as f:
+ dts_deps_file_str = f.read()
+ dts_deps_file_str = dts_deps_file_str.replace("\n]", "]")
+ cfg.read_string(dts_deps_file_str)
+
+ deps_section = cfg['tool.poetry.dependencies']
+ deps = {dep: deps_section[dep].strip('"\'') for dep in deps_section}
+ doc_deps_section = cfg['tool.poetry.group.docs.dependencies']
+ doc_deps = {dep: doc_deps_section[dep].strip("\"'") for dep in doc_deps_section}
+
+ return deps | doc_deps
+
+
+def get_missing_imports():
+ missing_imports = []
+ req_deps = _get_dependencies(_DTS_DEP_FILE_PATH)
+ req_deps.pop('python')
+
+ for req_dep, req_ver in (req_deps | _EXTRA_DEPS).items():
+ try:
+ req_ver = _get_version_tuple(req_ver.strip(_VERSION_COMPARISON_CHARS))
+ found_dep_ver = _get_version_tuple(importlib.metadata.version(req_dep))
+ if found_dep_ver < req_ver:
+ print(
+ f'The version "{found_dep_ver}" of package "{req_dep}" '
+ f'is lower than required "{req_ver}".'
+ )
+ except importlib.metadata.PackageNotFoundError:
+ print(f'Package "{req_dep}" not found.')
+ missing_imports.append(req_dep.lower().replace('-', '_'))
+
+ return missing_imports
+
+
+if __name__ == '__main__':
+ python_version = _get_dependencies(_DTS_DEP_FILE_PATH).pop('python')
+ if python_version:
+ sys_ver = _get_version_tuple(platform.python_version())
+ req_ver = _get_version_tuple(python_version.strip(_VERSION_COMPARISON_CHARS))
+ if sys_ver < req_ver:
+ print(
+ f'The available Python version "{sys_ver}" is lower than required "{req_ver}".'
+ )
+ exit(1)
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 3adf34e1a8..599653bea4 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -24,6 +24,7 @@ get_numa_count_cmd = py3 + files('get-numa-count.py')
get_test_suites_cmd = py3 + files('get-test-suites.py')
has_hugepages_cmd = py3 + files('has-hugepages.py')
cmdline_gen_cmd = py3 + files('dpdk-cmdline-gen.py')
+get_dts_deps = py3 + files('get-dts-deps.py')
# install any build tools that end-users might want also
install_data([
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index f9f0300126..ab223bcdf7 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -245,3 +245,6 @@ The public API headers are grouped by topics:
[experimental APIs](@ref rte_compat.h),
[ABI versioning](@ref rte_function_versioning.h),
[version](@ref rte_version.h)
+
+- **tests**:
+ [**DTS**](@dts_api_main_page)
diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index a8823c046f..c94f02d411 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -124,6 +124,8 @@ SEARCHENGINE = YES
SORT_MEMBER_DOCS = NO
SOURCE_BROWSER = YES
+ALIASES = "dts_api_main_page=@DTS_API_MAIN_PAGE@"
+
EXAMPLE_PATH = @TOPDIR@/examples
EXAMPLE_PATTERNS = *.c
EXAMPLE_RECURSIVE = YES
diff --git a/doc/api/meson.build b/doc/api/meson.build
index b828b1ed66..b893931b92 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -41,6 +41,7 @@ cdata.set('WARN_AS_ERROR', 'NO')
if get_option('werror')
cdata.set('WARN_AS_ERROR', 'YES')
endif
+cdata.set('DTS_API_MAIN_PAGE', join_paths('..', 'dts', 'html', 'index.html'))
# configure HTML Doxygen run
html_cdata = configuration_data()
diff --git a/doc/guides/conf.py b/doc/guides/conf.py
index 0f7ff5282d..eab3387874 100644
--- a/doc/guides/conf.py
+++ b/doc/guides/conf.py
@@ -10,7 +10,7 @@
from os.path import basename
from os.path import dirname
from os.path import join as path_join
-from sys import argv, stderr
+from sys import argv, stderr, path
import configparser
@@ -24,6 +24,45 @@
file=stderr)
pass
+# Napoleon enables the Google format of Python doscstrings, used in DTS.
+# Intersphinx allows linking to external projects, such as Python docs, also used in DTS.
+extensions = ['sphinx.ext.napoleon', 'sphinx.ext.intersphinx']
+
+# DTS Python docstring options.
+autodoc_default_options = {
+ 'members': True,
+ 'member-order': 'bysource',
+ 'show-inheritance': True,
+}
+autodoc_class_signature = 'separated'
+autodoc_typehints = 'both'
+autodoc_typehints_format = 'short'
+autodoc_typehints_description_target = 'documented'
+napoleon_numpy_docstring = False
+napoleon_attr_annotations = True
+napoleon_preprocess_types = True
+add_module_names = False
+toc_object_entries = True
+toc_object_entries_show_parents = 'hide'
+intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
+
+if environ.get('DTS_BUILD'):
+ # Add path to DTS sources so that Sphinx can find them.
+ dpdk_root = dirname(dirname(dirname(__file__)))
+ path.append(path_join(dpdk_root, 'dts'))
+
+ # Get missing DTS dependencies. Add path to buildtools to find the get_missing_imports function.
+ path.append(path_join(dpdk_root, 'buildtools'))
+ import importlib
+ # Ignore missing imports from DTS dependencies.
+ autodoc_mock_imports = importlib.import_module('get-dts-deps').get_missing_imports()
+
+ # DTS Sidebar config.
+ html_theme_options = {
+ 'collapse_navigation': False,
+ 'navigation_depth': -1, # unlimited depth
+ }
+
stop_on_error = ('-W' in argv)
project = 'Data Plane Development Kit'
diff --git a/doc/guides/contributing/documentation.rst b/doc/guides/contributing/documentation.rst
index 68454ae0d5..7b287ce631 100644
--- a/doc/guides/contributing/documentation.rst
+++ b/doc/guides/contributing/documentation.rst
@@ -133,6 +133,8 @@ added to by the developer.
Building the Documentation
--------------------------
+.. _doc_dependencies:
+
Dependencies
~~~~~~~~~~~~
diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst
index 04c66bebc4..6629928bee 100644
--- a/doc/guides/contributing/patches.rst
+++ b/doc/guides/contributing/patches.rst
@@ -499,6 +499,10 @@ The script usage is::
For both of the above scripts, the -n option is used to specify a number of commits from HEAD,
and the -r option allows the user specify a ``git log`` range.
+Additionally, when contributing to the DTS tool, patches should also be checked using
+the ``dts-check-format.sh`` script in the ``devtools`` directory of the DPDK repo.
+To run the script, extra :ref:`Python dependencies <dts_deps>` are needed.
+
.. _contrib_check_compilation:
Checking Compilation
diff --git a/doc/guides/meson.build b/doc/guides/meson.build
index f8bbfba9f5..b34b7b8eb0 100644
--- a/doc/guides/meson.build
+++ b/doc/guides/meson.build
@@ -1,6 +1,7 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Intel Corporation
+doc_guides_source_dir = meson.current_source_dir()
sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
if not sphinx.found()
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 515b15e4d8..bd715f8072 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -54,6 +54,7 @@ DTS uses Poetry as its Python dependency management.
Python build/development and runtime environments are the same and DTS development environment,
DTS runtime environment or just plain DTS environment are used interchangeably.
+.. _dts_deps:
Setting up DTS environment
~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -291,8 +292,15 @@ When adding code to the DTS framework, pay attention to the rest of the code
and try not to divert much from it.
The :ref:`DTS developer tools <dts_dev_tools>` will issue warnings
when some of the basics are not met.
+You should also build the :ref:`API documentation <building_api_docs>`
+to address any issues found during the build.
-The code must be properly documented with docstrings.
+The API documentation, which is a helpful reference when developing, may be accessed
+in the code directly or generated with the :ref:`API docs build steps <building_api_docs>`.
+When adding new files or modifying the directory structure,
+the corresponding changes must be made to DTS api doc sources in ``dts/doc``.
+
+Speaking of which, the code must be properly documented with docstrings.
The style must conform to the `Google style
<https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings>`_.
See an example of the style `here
@@ -427,6 +435,35 @@ the DTS code check and format script.
Refer to the script for usage: ``devtools/dts-check-format.sh -h``.
+.. _building_api_docs:
+
+Building DTS API docs
+---------------------
+
+The documentation is built using the standard DPDK build system.
+See :doc:`../linux_gsg/build_dpdk` for more details on compiling DPDK with meson.
+
+The :ref:`doc build dependencies <doc_dependencies>` may be installed with Poetry:
+
+.. code-block:: console
+
+ poetry install --no-root --only docs
+ poetry install --no-root --with docs # an alternative that will also install DTS dependencies
+ poetry shell
+
+After executing the meson command, build the documentation with:
+
+.. code-block:: console
+
+ ninja -C build dts-doc
+
+The output is generated in ``build/doc/api/dts/html``.
+
+.. note::
+
+ Make sure to fix any Sphinx warnings when adding or updating docstrings.
+
+
Configuration Schema
--------------------
diff --git a/dts/doc/meson.build b/dts/doc/meson.build
new file mode 100644
index 0000000000..c2df99bbc6
--- /dev/null
+++ b/dts/doc/meson.build
@@ -0,0 +1,30 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
+if not sphinx.found()
+ subdir_done()
+endif
+
+python_ver_satisfied = run_command(get_dts_deps).returncode()
+if python_ver_satisfied != 0
+ subdir_done()
+endif
+
+dts_doc_api_build_dir = join_paths(doc_api_build_dir, 'dts')
+
+extra_sphinx_args = ['-E', '-c', doc_guides_source_dir]
+if get_option('werror')
+ extra_sphinx_args += '-W'
+endif
+
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk', 'dts')
+dts_api_html = custom_target('dts_api_html',
+ output: 'html',
+ command: [sphinx_wrapper, sphinx, meson.project_version(),
+ meson.current_source_dir(), dts_doc_api_build_dir, extra_sphinx_args],
+ build_by_default: get_option('enable_docs'),
+ install: get_option('enable_docs'),
+ install_dir: htmldir)
+doc_targets += dts_api_html
+doc_target_names += 'DTS_API_HTML'
diff --git a/dts/meson.build b/dts/meson.build
new file mode 100644
index 0000000000..6ed3c93fe1
--- /dev/null
+++ b/dts/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+doc_targets = []
+doc_target_names = []
+
+subdir('doc')
+
+if doc_targets.length() == 0
+ message = 'No docs targets found'
+else
+ message = 'Built docs:'
+endif
+run_target('dts-doc', command: [echo, message, doc_target_names],
+ depends: doc_targets)
diff --git a/meson.build b/meson.build
index 8b248d4505..835973a0ce 100644
--- a/meson.build
+++ b/meson.build
@@ -87,6 +87,7 @@ subdir('app')
# build docs
subdir('doc')
+subdir('dts')
# build any examples explicitly requested - useful for developers - and
# install any example code into the appropriate install path
--
2.34.1
^ permalink raw reply [relevance 2%]
* [PATCH v11 5/5] dts: add API doc generation
@ 2024-08-05 13:59 2% ` Juraj Linkeš
0 siblings, 0 replies; 200+ results
From: Juraj Linkeš @ 2024-08-05 13:59 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, bruce.richardson, jspewock, probb,
paul.szczepanek, Luca.Vizzarro, npratte
Cc: dev, Juraj Linkeš
The tool used to generate DTS API docs is Sphinx, which is already in
use in DPDK. The same configuration is used to preserve style with one
DTS-specific configuration (so that the DPDK docs are unchanged) that
modifies how the sidebar displays the content.
Sphinx generates the documentation from Python docstrings. The docstring
format is the Google format [0] which requires the sphinx.ext.napoleon
extension. The other extension, sphinx.ext.intersphinx, enables linking
to objects in external documentations, such as the Python documentation.
There is one requirement for building DTS docs - the same Python version
as DTS or higher, because Sphinx's autodoc extension imports the code.
The dependencies needed to import the code don't have to be satisfied,
as the autodoc extension allows us to mock the imports. The missing
packages are taken from the DTS pyproject.toml file.
[0] https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
buildtools/call-sphinx-build.py | 10 ++-
buildtools/get-dts-deps.py | 78 +++++++++++++++++++++++
buildtools/meson.build | 1 +
doc/api/doxy-api-index.md | 3 +
doc/api/doxy-api.conf.in | 2 +
doc/api/meson.build | 1 +
doc/guides/conf.py | 41 +++++++++++-
doc/guides/contributing/documentation.rst | 2 +
doc/guides/contributing/patches.rst | 4 ++
doc/guides/meson.build | 1 +
doc/guides/tools/dts.rst | 39 +++++++++++-
dts/doc/meson.build | 30 +++++++++
dts/meson.build | 15 +++++
meson.build | 1 +
14 files changed, 225 insertions(+), 3 deletions(-)
create mode 100755 buildtools/get-dts-deps.py
create mode 100644 dts/doc/meson.build
create mode 100644 dts/meson.build
diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
index 623e7363ee..5dd59907cd 100755
--- a/buildtools/call-sphinx-build.py
+++ b/buildtools/call-sphinx-build.py
@@ -15,6 +15,11 @@
# set the version in environment for sphinx to pick up
os.environ['DPDK_VERSION'] = version
+conf_src = src
+if src.find('dts') != -1:
+ if '-c' in extra_args:
+ conf_src = extra_args[extra_args.index('-c') + 1]
+ os.environ['DTS_BUILD'] = "y"
sphinx_cmd = [sphinx] + extra_args
@@ -23,6 +28,9 @@
for root, dirs, files in os.walk(src):
srcfiles.extend([join(root, f) for f in files])
+if not os.path.exists(dst):
+ os.makedirs(dst)
+
# run sphinx, putting the html output in a "html" directory
with open(join(dst, 'sphinx_html.out'), 'w') as out:
process = run(sphinx_cmd + ['-b', 'html', src, join(dst, 'html')],
@@ -34,7 +42,7 @@
# copy custom CSS file
css = 'custom.css'
-src_css = join(src, css)
+src_css = join(conf_src, css)
dst_css = join(dst, 'html', '_static', 'css', css)
if not os.path.exists(dst_css) or not filecmp.cmp(src_css, dst_css):
os.makedirs(os.path.dirname(dst_css), exist_ok=True)
diff --git a/buildtools/get-dts-deps.py b/buildtools/get-dts-deps.py
new file mode 100755
index 0000000000..7114aeb710
--- /dev/null
+++ b/buildtools/get-dts-deps.py
@@ -0,0 +1,78 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 PANTHEON.tech s.r.o.
+#
+
+"""Utilities for DTS dependencies.
+
+The module can be used as an executable script,
+which verifies that the running Python version meets the version requirement of DTS.
+The script returns the standard exit codes in this mode (0 is success, 1 is failure).
+
+The module also contains a function, get_missing_imports,
+which looks for runtime and doc generation dependencies in the DTS pyproject.toml file
+a returns a list of module names used in an import statement that are missing.
+"""
+
+import configparser
+import importlib.metadata
+import importlib.util
+import os.path
+import platform
+
+_VERSION_COMPARISON_CHARS = '^<>='
+_EXTRA_DEPS = {'invoke': '>=1.3'}
+_DPDK_ROOT = os.path.dirname(os.path.dirname(__file__))
+_DTS_DEP_FILE_PATH = os.path.join(_DPDK_ROOT, 'dts', 'pyproject.toml')
+
+
+def _get_version_tuple(version_str):
+ return tuple(map(int, version_str.split(".")))
+
+
+def _get_dependencies(cfg_file_path):
+ cfg = configparser.ConfigParser()
+ with open(cfg_file_path) as f:
+ dts_deps_file_str = f.read()
+ dts_deps_file_str = dts_deps_file_str.replace("\n]", "]")
+ cfg.read_string(dts_deps_file_str)
+
+ deps_section = cfg['tool.poetry.dependencies']
+ deps = {dep: deps_section[dep].strip('"\'') for dep in deps_section}
+ doc_deps_section = cfg['tool.poetry.group.docs.dependencies']
+ doc_deps = {dep: doc_deps_section[dep].strip("\"'") for dep in doc_deps_section}
+
+ return deps | doc_deps
+
+
+def get_missing_imports():
+ missing_imports = []
+ req_deps = _get_dependencies(_DTS_DEP_FILE_PATH)
+ req_deps.pop('python')
+
+ for req_dep, req_ver in (req_deps | _EXTRA_DEPS).items():
+ try:
+ req_ver = _get_version_tuple(req_ver.strip(_VERSION_COMPARISON_CHARS))
+ found_dep_ver = _get_version_tuple(importlib.metadata.version(req_dep))
+ if found_dep_ver < req_ver:
+ print(
+ f'The version "{found_dep_ver}" of package "{req_dep}" '
+ f'is lower than required "{req_ver}".'
+ )
+ except importlib.metadata.PackageNotFoundError:
+ print(f'Package "{req_dep}" not found.')
+ missing_imports.append(req_dep.lower().replace('-', '_'))
+
+ return missing_imports
+
+
+if __name__ == '__main__':
+ python_version = _get_dependencies(_DTS_DEP_FILE_PATH).pop('python')
+ if python_version:
+ sys_ver = _get_version_tuple(platform.python_version())
+ req_ver = _get_version_tuple(python_version.strip(_VERSION_COMPARISON_CHARS))
+ if sys_ver < req_ver:
+ print(
+ f'The available Python version "{sys_ver}" is lower than required "{req_ver}".'
+ )
+ exit(1)
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 3adf34e1a8..599653bea4 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -24,6 +24,7 @@ get_numa_count_cmd = py3 + files('get-numa-count.py')
get_test_suites_cmd = py3 + files('get-test-suites.py')
has_hugepages_cmd = py3 + files('has-hugepages.py')
cmdline_gen_cmd = py3 + files('dpdk-cmdline-gen.py')
+get_dts_deps = py3 + files('get-dts-deps.py')
# install any build tools that end-users might want also
install_data([
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index f9f0300126..ab223bcdf7 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -245,3 +245,6 @@ The public API headers are grouped by topics:
[experimental APIs](@ref rte_compat.h),
[ABI versioning](@ref rte_function_versioning.h),
[version](@ref rte_version.h)
+
+- **tests**:
+ [**DTS**](@dts_api_main_page)
diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index a8823c046f..c94f02d411 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -124,6 +124,8 @@ SEARCHENGINE = YES
SORT_MEMBER_DOCS = NO
SOURCE_BROWSER = YES
+ALIASES = "dts_api_main_page=@DTS_API_MAIN_PAGE@"
+
EXAMPLE_PATH = @TOPDIR@/examples
EXAMPLE_PATTERNS = *.c
EXAMPLE_RECURSIVE = YES
diff --git a/doc/api/meson.build b/doc/api/meson.build
index b828b1ed66..b893931b92 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -41,6 +41,7 @@ cdata.set('WARN_AS_ERROR', 'NO')
if get_option('werror')
cdata.set('WARN_AS_ERROR', 'YES')
endif
+cdata.set('DTS_API_MAIN_PAGE', join_paths('..', 'dts', 'html', 'index.html'))
# configure HTML Doxygen run
html_cdata = configuration_data()
diff --git a/doc/guides/conf.py b/doc/guides/conf.py
index 0f7ff5282d..eab3387874 100644
--- a/doc/guides/conf.py
+++ b/doc/guides/conf.py
@@ -10,7 +10,7 @@
from os.path import basename
from os.path import dirname
from os.path import join as path_join
-from sys import argv, stderr
+from sys import argv, stderr, path
import configparser
@@ -24,6 +24,45 @@
file=stderr)
pass
+# Napoleon enables the Google format of Python doscstrings, used in DTS.
+# Intersphinx allows linking to external projects, such as Python docs, also used in DTS.
+extensions = ['sphinx.ext.napoleon', 'sphinx.ext.intersphinx']
+
+# DTS Python docstring options.
+autodoc_default_options = {
+ 'members': True,
+ 'member-order': 'bysource',
+ 'show-inheritance': True,
+}
+autodoc_class_signature = 'separated'
+autodoc_typehints = 'both'
+autodoc_typehints_format = 'short'
+autodoc_typehints_description_target = 'documented'
+napoleon_numpy_docstring = False
+napoleon_attr_annotations = True
+napoleon_preprocess_types = True
+add_module_names = False
+toc_object_entries = True
+toc_object_entries_show_parents = 'hide'
+intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
+
+if environ.get('DTS_BUILD'):
+ # Add path to DTS sources so that Sphinx can find them.
+ dpdk_root = dirname(dirname(dirname(__file__)))
+ path.append(path_join(dpdk_root, 'dts'))
+
+ # Get missing DTS dependencies. Add path to buildtools to find the get_missing_imports function.
+ path.append(path_join(dpdk_root, 'buildtools'))
+ import importlib
+ # Ignore missing imports from DTS dependencies.
+ autodoc_mock_imports = importlib.import_module('get-dts-deps').get_missing_imports()
+
+ # DTS Sidebar config.
+ html_theme_options = {
+ 'collapse_navigation': False,
+ 'navigation_depth': -1, # unlimited depth
+ }
+
stop_on_error = ('-W' in argv)
project = 'Data Plane Development Kit'
diff --git a/doc/guides/contributing/documentation.rst b/doc/guides/contributing/documentation.rst
index 68454ae0d5..7b287ce631 100644
--- a/doc/guides/contributing/documentation.rst
+++ b/doc/guides/contributing/documentation.rst
@@ -133,6 +133,8 @@ added to by the developer.
Building the Documentation
--------------------------
+.. _doc_dependencies:
+
Dependencies
~~~~~~~~~~~~
diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/patches.rst
index 04c66bebc4..6629928bee 100644
--- a/doc/guides/contributing/patches.rst
+++ b/doc/guides/contributing/patches.rst
@@ -499,6 +499,10 @@ The script usage is::
For both of the above scripts, the -n option is used to specify a number of commits from HEAD,
and the -r option allows the user specify a ``git log`` range.
+Additionally, when contributing to the DTS tool, patches should also be checked using
+the ``dts-check-format.sh`` script in the ``devtools`` directory of the DPDK repo.
+To run the script, extra :ref:`Python dependencies <dts_deps>` are needed.
+
.. _contrib_check_compilation:
Checking Compilation
diff --git a/doc/guides/meson.build b/doc/guides/meson.build
index f8bbfba9f5..b34b7b8eb0 100644
--- a/doc/guides/meson.build
+++ b/doc/guides/meson.build
@@ -1,6 +1,7 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Intel Corporation
+doc_guides_source_dir = meson.current_source_dir()
sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
if not sphinx.found()
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 515b15e4d8..bd715f8072 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -54,6 +54,7 @@ DTS uses Poetry as its Python dependency management.
Python build/development and runtime environments are the same and DTS development environment,
DTS runtime environment or just plain DTS environment are used interchangeably.
+.. _dts_deps:
Setting up DTS environment
~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -291,8 +292,15 @@ When adding code to the DTS framework, pay attention to the rest of the code
and try not to divert much from it.
The :ref:`DTS developer tools <dts_dev_tools>` will issue warnings
when some of the basics are not met.
+You should also build the :ref:`API documentation <building_api_docs>`
+to address any issues found during the build.
-The code must be properly documented with docstrings.
+The API documentation, which is a helpful reference when developing, may be accessed
+in the code directly or generated with the :ref:`API docs build steps <building_api_docs>`.
+When adding new files or modifying the directory structure,
+the corresponding changes must be made to DTS api doc sources in ``dts/doc``.
+
+Speaking of which, the code must be properly documented with docstrings.
The style must conform to the `Google style
<https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings>`_.
See an example of the style `here
@@ -427,6 +435,35 @@ the DTS code check and format script.
Refer to the script for usage: ``devtools/dts-check-format.sh -h``.
+.. _building_api_docs:
+
+Building DTS API docs
+---------------------
+
+The documentation is built using the standard DPDK build system.
+See :doc:`../linux_gsg/build_dpdk` for more details on compiling DPDK with meson.
+
+The :ref:`doc build dependencies <doc_dependencies>` may be installed with Poetry:
+
+.. code-block:: console
+
+ poetry install --no-root --only docs
+ poetry install --no-root --with docs # an alternative that will also install DTS dependencies
+ poetry shell
+
+After executing the meson command, build the documentation with:
+
+.. code-block:: console
+
+ ninja -C build dts-doc
+
+The output is generated in ``build/doc/api/dts/html``.
+
+.. note::
+
+ Make sure to fix any Sphinx warnings when adding or updating docstrings.
+
+
Configuration Schema
--------------------
diff --git a/dts/doc/meson.build b/dts/doc/meson.build
new file mode 100644
index 0000000000..c2df99bbc6
--- /dev/null
+++ b/dts/doc/meson.build
@@ -0,0 +1,30 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
+if not sphinx.found()
+ subdir_done()
+endif
+
+python_ver_satisfied = run_command(get_dts_deps).returncode()
+if python_ver_satisfied != 0
+ subdir_done()
+endif
+
+dts_doc_api_build_dir = join_paths(doc_api_build_dir, 'dts')
+
+extra_sphinx_args = ['-E', '-c', doc_guides_source_dir]
+if get_option('werror')
+ extra_sphinx_args += '-W'
+endif
+
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk', 'dts')
+dts_api_html = custom_target('dts_api_html',
+ output: 'html',
+ command: [sphinx_wrapper, sphinx, meson.project_version(),
+ meson.current_source_dir(), dts_doc_api_build_dir, extra_sphinx_args],
+ build_by_default: get_option('enable_docs'),
+ install: get_option('enable_docs'),
+ install_dir: htmldir)
+doc_targets += dts_api_html
+doc_target_names += 'DTS_API_HTML'
diff --git a/dts/meson.build b/dts/meson.build
new file mode 100644
index 0000000000..6ed3c93fe1
--- /dev/null
+++ b/dts/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+doc_targets = []
+doc_target_names = []
+
+subdir('doc')
+
+if doc_targets.length() == 0
+ message = 'No docs targets found'
+else
+ message = 'Built docs:'
+endif
+run_target('dts-doc', command: [echo, message, doc_target_names],
+ depends: doc_targets)
diff --git a/meson.build b/meson.build
index 8b248d4505..835973a0ce 100644
--- a/meson.build
+++ b/meson.build
@@ -87,6 +87,7 @@ subdir('app')
# build docs
subdir('doc')
+subdir('dts')
# build any examples explicitly requested - useful for developers - and
# install any example code into the appropriate install path
--
2.34.1
^ permalink raw reply [relevance 2%]
* [PATCH v11 1/7] mbuf: replace term sanity check
@ 2024-08-01 15:46 2% ` Stephen Hemminger
0 siblings, 0 replies; 200+ results
From: Stephen Hemminger @ 2024-08-01 15:46 UTC (permalink / raw)
To: dev; +Cc: Stephen Hemminger, Andrew Rybchenko, Morten Brørup
Replace rte_mbuf_sanity_check() with rte_mbuf_verify()
to match the similar macro RTE_VERIFY() in rte_debug.h
The term sanity check is on the Tier 2 list of words
that should be replaced.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
---
app/test/test_mbuf.c | 28 +++++------
doc/guides/prog_guide/mbuf_lib.rst | 4 +-
doc/guides/rel_notes/deprecation.rst | 3 ++
drivers/net/avp/avp_ethdev.c | 18 +++----
drivers/net/sfc/sfc_ef100_rx.c | 6 +--
drivers/net/sfc/sfc_ef10_essb_rx.c | 4 +-
drivers/net/sfc/sfc_ef10_rx.c | 4 +-
drivers/net/sfc/sfc_rx.c | 2 +-
examples/ipv4_multicast/main.c | 2 +-
lib/mbuf/rte_mbuf.c | 23 +++++----
lib/mbuf/rte_mbuf.h | 71 +++++++++++++++-------------
lib/mbuf/version.map | 1 +
12 files changed, 90 insertions(+), 76 deletions(-)
diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
index 17be977f31..3fbb5dea8b 100644
--- a/app/test/test_mbuf.c
+++ b/app/test/test_mbuf.c
@@ -262,8 +262,8 @@ test_one_pktmbuf(struct rte_mempool *pktmbuf_pool)
GOTO_FAIL("Buffer should be continuous");
memset(hdr, 0x55, MBUF_TEST_HDR2_LEN);
- rte_mbuf_sanity_check(m, 1);
- rte_mbuf_sanity_check(m, 0);
+ rte_mbuf_verify(m, 1);
+ rte_mbuf_verify(m, 0);
rte_pktmbuf_dump(stdout, m, 0);
/* this prepend should fail */
@@ -1162,7 +1162,7 @@ test_refcnt_mbuf(void)
#ifdef RTE_EXEC_ENV_WINDOWS
static int
-test_failing_mbuf_sanity_check(struct rte_mempool *pktmbuf_pool)
+test_failing_mbuf_verify(struct rte_mempool *pktmbuf_pool)
{
RTE_SET_USED(pktmbuf_pool);
return TEST_SKIPPED;
@@ -1181,12 +1181,12 @@ mbuf_check_pass(struct rte_mbuf *buf)
}
static int
-test_failing_mbuf_sanity_check(struct rte_mempool *pktmbuf_pool)
+test_failing_mbuf_verify(struct rte_mempool *pktmbuf_pool)
{
struct rte_mbuf *buf;
struct rte_mbuf badbuf;
- printf("Checking rte_mbuf_sanity_check for failure conditions\n");
+ printf("Checking rte_mbuf_verify for failure conditions\n");
/* get a good mbuf to use to make copies */
buf = rte_pktmbuf_alloc(pktmbuf_pool);
@@ -1708,7 +1708,7 @@ test_mbuf_validate_tx_offload(const char *test_name,
GOTO_FAIL("%s: mbuf allocation failed!\n", __func__);
if (rte_pktmbuf_pkt_len(m) != 0)
GOTO_FAIL("%s: Bad packet length\n", __func__);
- rte_mbuf_sanity_check(m, 0);
+ rte_mbuf_verify(m, 0);
m->ol_flags = ol_flags;
m->tso_segsz = segsize;
ret = rte_validate_tx_offload(m);
@@ -1915,7 +1915,7 @@ test_pktmbuf_read(struct rte_mempool *pktmbuf_pool)
GOTO_FAIL("%s: mbuf allocation failed!\n", __func__);
if (rte_pktmbuf_pkt_len(m) != 0)
GOTO_FAIL("%s: Bad packet length\n", __func__);
- rte_mbuf_sanity_check(m, 0);
+ rte_mbuf_verify(m, 0);
data = rte_pktmbuf_append(m, MBUF_TEST_DATA_LEN2);
if (data == NULL)
@@ -1964,7 +1964,7 @@ test_pktmbuf_read_from_offset(struct rte_mempool *pktmbuf_pool)
if (rte_pktmbuf_pkt_len(m) != 0)
GOTO_FAIL("%s: Bad packet length\n", __func__);
- rte_mbuf_sanity_check(m, 0);
+ rte_mbuf_verify(m, 0);
/* prepend an ethernet header */
hdr = (struct ether_hdr *)rte_pktmbuf_prepend(m, hdr_len);
@@ -2109,7 +2109,7 @@ create_packet(struct rte_mempool *pktmbuf_pool,
GOTO_FAIL("%s: mbuf allocation failed!\n", __func__);
if (rte_pktmbuf_pkt_len(pkt_seg) != 0)
GOTO_FAIL("%s: Bad packet length\n", __func__);
- rte_mbuf_sanity_check(pkt_seg, 0);
+ rte_mbuf_verify(pkt_seg, 0);
/* Add header only for the first segment */
if (test_data->flags == MBUF_HEADER && seg == 0) {
hdr_len = sizeof(struct rte_ether_hdr);
@@ -2321,7 +2321,7 @@ test_pktmbuf_ext_shinfo_init_helper(struct rte_mempool *pktmbuf_pool)
GOTO_FAIL("%s: mbuf allocation failed!\n", __func__);
if (rte_pktmbuf_pkt_len(m) != 0)
GOTO_FAIL("%s: Bad packet length\n", __func__);
- rte_mbuf_sanity_check(m, 0);
+ rte_mbuf_verify(m, 0);
ext_buf_addr = rte_malloc("External buffer", buf_len,
RTE_CACHE_LINE_SIZE);
@@ -2482,8 +2482,8 @@ test_pktmbuf_ext_pinned_buffer(struct rte_mempool *std_pool)
GOTO_FAIL("%s: test_pktmbuf_copy(pinned) failed\n",
__func__);
- if (test_failing_mbuf_sanity_check(pinned_pool) < 0)
- GOTO_FAIL("%s: test_failing_mbuf_sanity_check(pinned)"
+ if (test_failing_mbuf_verify(pinned_pool) < 0)
+ GOTO_FAIL("%s: test_failing_mbuf_verify(pinned)"
" failed\n", __func__);
if (test_mbuf_linearize_check(pinned_pool) < 0)
@@ -2857,8 +2857,8 @@ test_mbuf(void)
goto err;
}
- if (test_failing_mbuf_sanity_check(pktmbuf_pool) < 0) {
- printf("test_failing_mbuf_sanity_check() failed\n");
+ if (test_failing_mbuf_verify(pktmbuf_pool) < 0) {
+ printf("test_failing_mbuf_verify() failed\n");
goto err;
}
diff --git a/doc/guides/prog_guide/mbuf_lib.rst b/doc/guides/prog_guide/mbuf_lib.rst
index 749f9c97a8..0a197437a0 100644
--- a/doc/guides/prog_guide/mbuf_lib.rst
+++ b/doc/guides/prog_guide/mbuf_lib.rst
@@ -266,8 +266,8 @@ can be found in several of the sample applications, for example, the IPv4 Multic
Debug
-----
-In debug mode, the functions of the mbuf library perform sanity checks before any operation (such as, buffer corruption,
-bad type, and so on).
+In debug mode, the functions of the mbuf library perform consistency checks
+before any operation (such as, buffer corruption, bad type, and so on).
Use Cases
---------
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 211f59fdc9..17f08500aa 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -230,3 +230,6 @@ Deprecation Notices
The structures ``rte_node``, ``rte_node_register``
and ``rte_graph_cluster_node_stats`` will be extended
to include node error counters and error description.
+
+* mbuf: The function ``rte_mbuf_sanity_check`` is deprecated.
+ Use the new function ``rte_mbuf_verify`` instead.
diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c
index 6733462c86..bafc08fd60 100644
--- a/drivers/net/avp/avp_ethdev.c
+++ b/drivers/net/avp/avp_ethdev.c
@@ -1231,7 +1231,7 @@ _avp_mac_filter(struct avp_dev *avp, struct rte_mbuf *m)
#ifdef RTE_LIBRTE_AVP_DEBUG_BUFFERS
static inline void
-__avp_dev_buffer_sanity_check(struct avp_dev *avp, struct rte_avp_desc *buf)
+__avp_dev_buffer_check(struct avp_dev *avp, struct rte_avp_desc *buf)
{
struct rte_avp_desc *first_buf;
struct rte_avp_desc *pkt_buf;
@@ -1272,12 +1272,12 @@ __avp_dev_buffer_sanity_check(struct avp_dev *avp, struct rte_avp_desc *buf)
first_buf->pkt_len, pkt_len);
}
-#define avp_dev_buffer_sanity_check(a, b) \
- __avp_dev_buffer_sanity_check((a), (b))
+#define avp_dev_buffer_check(a, b) \
+ __avp_dev_buffer_check((a), (b))
#else /* RTE_LIBRTE_AVP_DEBUG_BUFFERS */
-#define avp_dev_buffer_sanity_check(a, b) do {} while (0)
+#define avp_dev_buffer_check(a, b) do {} while (0)
#endif
@@ -1302,7 +1302,7 @@ avp_dev_copy_from_buffers(struct avp_dev *avp,
void *pkt_data;
unsigned int i;
- avp_dev_buffer_sanity_check(avp, buf);
+ avp_dev_buffer_check(avp, buf);
/* setup the first source buffer */
pkt_buf = avp_dev_translate_buffer(avp, buf);
@@ -1370,7 +1370,7 @@ avp_dev_copy_from_buffers(struct avp_dev *avp,
rte_pktmbuf_pkt_len(m) = total_length;
m->vlan_tci = vlan_tci;
- __rte_mbuf_sanity_check(m, 1);
+ __rte_mbuf_verify(m, 1);
return m;
}
@@ -1614,7 +1614,7 @@ avp_dev_copy_to_buffers(struct avp_dev *avp,
char *pkt_data;
unsigned int i;
- __rte_mbuf_sanity_check(mbuf, 1);
+ __rte_mbuf_verify(mbuf, 1);
m = mbuf;
src_offset = 0;
@@ -1680,7 +1680,7 @@ avp_dev_copy_to_buffers(struct avp_dev *avp,
first_buf->vlan_tci = mbuf->vlan_tci;
}
- avp_dev_buffer_sanity_check(avp, buffers[0]);
+ avp_dev_buffer_check(avp, buffers[0]);
return total_length;
}
@@ -1798,7 +1798,7 @@ avp_xmit_scattered_pkts(void *tx_queue,
#ifdef RTE_LIBRTE_AVP_DEBUG_BUFFERS
for (i = 0; i < nb_pkts; i++)
- avp_dev_buffer_sanity_check(avp, tx_bufs[i]);
+ avp_dev_buffer_check(avp, tx_bufs[i]);
#endif
/* send the packets */
diff --git a/drivers/net/sfc/sfc_ef100_rx.c b/drivers/net/sfc/sfc_ef100_rx.c
index e283879e6b..5ebfba4dcf 100644
--- a/drivers/net/sfc/sfc_ef100_rx.c
+++ b/drivers/net/sfc/sfc_ef100_rx.c
@@ -179,7 +179,7 @@ sfc_ef100_rx_qrefill(struct sfc_ef100_rxq *rxq)
struct sfc_ef100_rx_sw_desc *rxd;
rte_iova_t dma_addr;
- __rte_mbuf_raw_sanity_check(m);
+ __rte_mbuf_raw_verify(m);
dma_addr = rte_mbuf_data_iova_default(m);
if (rxq->flags & SFC_EF100_RXQ_NIC_DMA_MAP) {
@@ -551,7 +551,7 @@ sfc_ef100_rx_process_ready_pkts(struct sfc_ef100_rxq *rxq,
rxq->ready_pkts--;
pkt = sfc_ef100_rx_next_mbuf(rxq);
- __rte_mbuf_raw_sanity_check(pkt);
+ __rte_mbuf_raw_verify(pkt);
RTE_BUILD_BUG_ON(sizeof(pkt->rearm_data[0]) !=
sizeof(rxq->rearm_data));
@@ -575,7 +575,7 @@ sfc_ef100_rx_process_ready_pkts(struct sfc_ef100_rxq *rxq,
struct rte_mbuf *seg;
seg = sfc_ef100_rx_next_mbuf(rxq);
- __rte_mbuf_raw_sanity_check(seg);
+ __rte_mbuf_raw_verify(seg);
seg->data_off = RTE_PKTMBUF_HEADROOM;
diff --git a/drivers/net/sfc/sfc_ef10_essb_rx.c b/drivers/net/sfc/sfc_ef10_essb_rx.c
index 78bd430363..74647e2792 100644
--- a/drivers/net/sfc/sfc_ef10_essb_rx.c
+++ b/drivers/net/sfc/sfc_ef10_essb_rx.c
@@ -125,7 +125,7 @@ sfc_ef10_essb_next_mbuf(const struct sfc_ef10_essb_rxq *rxq,
struct rte_mbuf *m;
m = (struct rte_mbuf *)((uintptr_t)mbuf + rxq->buf_stride);
- __rte_mbuf_raw_sanity_check(m);
+ __rte_mbuf_raw_verify(m);
return m;
}
@@ -136,7 +136,7 @@ sfc_ef10_essb_mbuf_by_index(const struct sfc_ef10_essb_rxq *rxq,
struct rte_mbuf *m;
m = (struct rte_mbuf *)((uintptr_t)mbuf + idx * rxq->buf_stride);
- __rte_mbuf_raw_sanity_check(m);
+ __rte_mbuf_raw_verify(m);
return m;
}
diff --git a/drivers/net/sfc/sfc_ef10_rx.c b/drivers/net/sfc/sfc_ef10_rx.c
index 60442930b3..f4fc815570 100644
--- a/drivers/net/sfc/sfc_ef10_rx.c
+++ b/drivers/net/sfc/sfc_ef10_rx.c
@@ -148,7 +148,7 @@ sfc_ef10_rx_qrefill(struct sfc_ef10_rxq *rxq)
struct sfc_ef10_rx_sw_desc *rxd;
rte_iova_t phys_addr;
- __rte_mbuf_raw_sanity_check(m);
+ __rte_mbuf_raw_verify(m);
SFC_ASSERT((id & ~ptr_mask) == 0);
rxd = &rxq->sw_ring[id];
@@ -297,7 +297,7 @@ sfc_ef10_rx_process_event(struct sfc_ef10_rxq *rxq, efx_qword_t rx_ev,
rxd = &rxq->sw_ring[pending++ & ptr_mask];
m = rxd->mbuf;
- __rte_mbuf_raw_sanity_check(m);
+ __rte_mbuf_raw_verify(m);
m->data_off = RTE_PKTMBUF_HEADROOM;
rte_pktmbuf_data_len(m) = seg_len;
diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c
index a193229265..c885ce2b05 100644
--- a/drivers/net/sfc/sfc_rx.c
+++ b/drivers/net/sfc/sfc_rx.c
@@ -120,7 +120,7 @@ sfc_efx_rx_qrefill(struct sfc_efx_rxq *rxq)
++i, id = (id + 1) & rxq->ptr_mask) {
m = objs[i];
- __rte_mbuf_raw_sanity_check(m);
+ __rte_mbuf_raw_verify(m);
rxd = &rxq->sw_desc[id];
rxd->mbuf = m;
diff --git a/examples/ipv4_multicast/main.c b/examples/ipv4_multicast/main.c
index 1eed645d02..3bfab37012 100644
--- a/examples/ipv4_multicast/main.c
+++ b/examples/ipv4_multicast/main.c
@@ -258,7 +258,7 @@ mcast_out_pkt(struct rte_mbuf *pkt, int use_clone)
hdr->pkt_len = (uint16_t)(hdr->data_len + pkt->pkt_len);
hdr->nb_segs = pkt->nb_segs + 1;
- __rte_mbuf_sanity_check(hdr, 1);
+ __rte_mbuf_verify(hdr, 1);
return hdr;
}
/* >8 End of mcast_out_kt. */
diff --git a/lib/mbuf/rte_mbuf.c b/lib/mbuf/rte_mbuf.c
index 559d5ad8a7..fc5d4ba29d 100644
--- a/lib/mbuf/rte_mbuf.c
+++ b/lib/mbuf/rte_mbuf.c
@@ -367,9 +367,9 @@ rte_pktmbuf_pool_create_extbuf(const char *name, unsigned int n,
return mp;
}
-/* do some sanity checks on a mbuf: panic if it fails */
+/* do some checks on a mbuf: panic if it fails */
void
-rte_mbuf_sanity_check(const struct rte_mbuf *m, int is_header)
+rte_mbuf_verify(const struct rte_mbuf *m, int is_header)
{
const char *reason;
@@ -377,6 +377,13 @@ rte_mbuf_sanity_check(const struct rte_mbuf *m, int is_header)
rte_panic("%s\n", reason);
}
+/* For ABI compatibility, to be removed in next release */
+void
+rte_mbuf_sanity_check(const struct rte_mbuf *m, int is_header)
+{
+ rte_mbuf_verify(m, is_header);
+}
+
int rte_mbuf_check(const struct rte_mbuf *m, int is_header,
const char **reason)
{
@@ -496,7 +503,7 @@ void rte_pktmbuf_free_bulk(struct rte_mbuf **mbufs, unsigned int count)
if (unlikely(m == NULL))
continue;
- __rte_mbuf_sanity_check(m, 1);
+ __rte_mbuf_verify(m, 1);
do {
m_next = m->next;
@@ -546,7 +553,7 @@ rte_pktmbuf_clone(struct rte_mbuf *md, struct rte_mempool *mp)
return NULL;
}
- __rte_mbuf_sanity_check(mc, 1);
+ __rte_mbuf_verify(mc, 1);
return mc;
}
@@ -596,7 +603,7 @@ rte_pktmbuf_copy(const struct rte_mbuf *m, struct rte_mempool *mp,
struct rte_mbuf *mc, *m_last, **prev;
/* garbage in check */
- __rte_mbuf_sanity_check(m, 1);
+ __rte_mbuf_verify(m, 1);
/* check for request to copy at offset past end of mbuf */
if (unlikely(off >= m->pkt_len))
@@ -660,7 +667,7 @@ rte_pktmbuf_copy(const struct rte_mbuf *m, struct rte_mempool *mp,
}
/* garbage out check */
- __rte_mbuf_sanity_check(mc, 1);
+ __rte_mbuf_verify(mc, 1);
return mc;
}
@@ -671,7 +678,7 @@ rte_pktmbuf_dump(FILE *f, const struct rte_mbuf *m, unsigned dump_len)
unsigned int len;
unsigned int nb_segs;
- __rte_mbuf_sanity_check(m, 1);
+ __rte_mbuf_verify(m, 1);
fprintf(f, "dump mbuf at %p, iova=%#" PRIx64 ", buf_len=%u\n", m, rte_mbuf_iova_get(m),
m->buf_len);
@@ -689,7 +696,7 @@ rte_pktmbuf_dump(FILE *f, const struct rte_mbuf *m, unsigned dump_len)
nb_segs = m->nb_segs;
while (m && nb_segs != 0) {
- __rte_mbuf_sanity_check(m, 0);
+ __rte_mbuf_verify(m, 0);
fprintf(f, " segment at %p, data=%p, len=%u, off=%u, refcnt=%u\n",
m, rte_pktmbuf_mtod(m, void *),
diff --git a/lib/mbuf/rte_mbuf.h b/lib/mbuf/rte_mbuf.h
index babe16c72c..35e89e60e2 100644
--- a/lib/mbuf/rte_mbuf.h
+++ b/lib/mbuf/rte_mbuf.h
@@ -339,13 +339,13 @@ rte_pktmbuf_priv_flags(struct rte_mempool *mp)
#ifdef RTE_LIBRTE_MBUF_DEBUG
-/** check mbuf type in debug mode */
-#define __rte_mbuf_sanity_check(m, is_h) rte_mbuf_sanity_check(m, is_h)
+/** do mbuf type in debug mode */
+#define __rte_mbuf_verify(m, is_h) rte_mbuf_verify(m, is_h)
#else /* RTE_LIBRTE_MBUF_DEBUG */
-/** check mbuf type in debug mode */
-#define __rte_mbuf_sanity_check(m, is_h) do { } while (0)
+/** ignore mbuf checks if not in debug mode */
+#define __rte_mbuf_verify(m, is_h) do { } while (0)
#endif /* RTE_LIBRTE_MBUF_DEBUG */
@@ -514,10 +514,9 @@ rte_mbuf_ext_refcnt_update(struct rte_mbuf_ext_shared_info *shinfo,
/**
- * Sanity checks on an mbuf.
+ * Check that the mbuf is valid and panic if corrupted.
*
- * Check the consistency of the given mbuf. The function will cause a
- * panic if corruption is detected.
+ * Acts assertion that mbuf is consistent. If not it calls rte_panic().
*
* @param m
* The mbuf to be checked.
@@ -526,13 +525,17 @@ rte_mbuf_ext_refcnt_update(struct rte_mbuf_ext_shared_info *shinfo,
* of a packet (in this case, some fields like nb_segs are not checked)
*/
void
+rte_mbuf_verify(const struct rte_mbuf *m, int is_header);
+
+/* Older deprecated name for rte_mbuf_verify() */
+void __rte_deprecated
rte_mbuf_sanity_check(const struct rte_mbuf *m, int is_header);
/**
- * Sanity checks on a mbuf.
+ * Do consistency checks on a mbuf.
*
- * Almost like rte_mbuf_sanity_check(), but this function gives the reason
- * if corruption is detected rather than panic.
+ * Check the consistency of the given mbuf and if not valid
+ * return the reason.
*
* @param m
* The mbuf to be checked.
@@ -551,7 +554,7 @@ int rte_mbuf_check(const struct rte_mbuf *m, int is_header,
const char **reason);
/**
- * Sanity checks on a reinitialized mbuf in debug mode.
+ * Do checks on a reinitialized mbuf in debug mode.
*
* Check the consistency of the given reinitialized mbuf.
* The function will cause a panic if corruption is detected.
@@ -563,16 +566,16 @@ int rte_mbuf_check(const struct rte_mbuf *m, int is_header,
* The mbuf to be checked.
*/
static __rte_always_inline void
-__rte_mbuf_raw_sanity_check(__rte_unused const struct rte_mbuf *m)
+__rte_mbuf_raw_verify(__rte_unused const struct rte_mbuf *m)
{
RTE_ASSERT(rte_mbuf_refcnt_read(m) == 1);
RTE_ASSERT(m->next == NULL);
RTE_ASSERT(m->nb_segs == 1);
- __rte_mbuf_sanity_check(m, 0);
+ __rte_mbuf_verify(m, 0);
}
/** For backwards compatibility. */
-#define MBUF_RAW_ALLOC_CHECK(m) __rte_mbuf_raw_sanity_check(m)
+#define MBUF_RAW_ALLOC_CHECK(m) __rte_mbuf_raw_verify(m)
/**
* Allocate an uninitialized mbuf from mempool *mp*.
@@ -599,7 +602,7 @@ static inline struct rte_mbuf *rte_mbuf_raw_alloc(struct rte_mempool *mp)
if (rte_mempool_get(mp, (void **)&m) < 0)
return NULL;
- __rte_mbuf_raw_sanity_check(m);
+ __rte_mbuf_raw_verify(m);
return m;
}
@@ -622,7 +625,7 @@ rte_mbuf_raw_free(struct rte_mbuf *m)
{
RTE_ASSERT(!RTE_MBUF_CLONED(m) &&
(!RTE_MBUF_HAS_EXTBUF(m) || RTE_MBUF_HAS_PINNED_EXTBUF(m)));
- __rte_mbuf_raw_sanity_check(m);
+ __rte_mbuf_raw_verify(m);
rte_mempool_put(m->pool, m);
}
@@ -885,7 +888,7 @@ static inline void rte_pktmbuf_reset(struct rte_mbuf *m)
rte_pktmbuf_reset_headroom(m);
m->data_len = 0;
- __rte_mbuf_sanity_check(m, 1);
+ __rte_mbuf_verify(m, 1);
}
/**
@@ -941,22 +944,22 @@ static inline int rte_pktmbuf_alloc_bulk(struct rte_mempool *pool,
switch (count % 4) {
case 0:
while (idx != count) {
- __rte_mbuf_raw_sanity_check(mbufs[idx]);
+ __rte_mbuf_raw_verify(mbufs[idx]);
rte_pktmbuf_reset(mbufs[idx]);
idx++;
/* fall-through */
case 3:
- __rte_mbuf_raw_sanity_check(mbufs[idx]);
+ __rte_mbuf_raw_verify(mbufs[idx]);
rte_pktmbuf_reset(mbufs[idx]);
idx++;
/* fall-through */
case 2:
- __rte_mbuf_raw_sanity_check(mbufs[idx]);
+ __rte_mbuf_raw_verify(mbufs[idx]);
rte_pktmbuf_reset(mbufs[idx]);
idx++;
/* fall-through */
case 1:
- __rte_mbuf_raw_sanity_check(mbufs[idx]);
+ __rte_mbuf_raw_verify(mbufs[idx]);
rte_pktmbuf_reset(mbufs[idx]);
idx++;
/* fall-through */
@@ -1187,8 +1190,8 @@ static inline void rte_pktmbuf_attach(struct rte_mbuf *mi, struct rte_mbuf *m)
mi->pkt_len = mi->data_len;
mi->nb_segs = 1;
- __rte_mbuf_sanity_check(mi, 1);
- __rte_mbuf_sanity_check(m, 0);
+ __rte_mbuf_verify(mi, 1);
+ __rte_mbuf_verify(m, 0);
}
/**
@@ -1343,7 +1346,7 @@ static inline int __rte_pktmbuf_pinned_extbuf_decref(struct rte_mbuf *m)
static __rte_always_inline struct rte_mbuf *
rte_pktmbuf_prefree_seg(struct rte_mbuf *m)
{
- __rte_mbuf_sanity_check(m, 0);
+ __rte_mbuf_verify(m, 0);
if (likely(rte_mbuf_refcnt_read(m) == 1)) {
@@ -1414,7 +1417,7 @@ static inline void rte_pktmbuf_free(struct rte_mbuf *m)
struct rte_mbuf *m_next;
if (m != NULL)
- __rte_mbuf_sanity_check(m, 1);
+ __rte_mbuf_verify(m, 1);
while (m != NULL) {
m_next = m->next;
@@ -1495,7 +1498,7 @@ rte_pktmbuf_copy(const struct rte_mbuf *m, struct rte_mempool *mp,
*/
static inline void rte_pktmbuf_refcnt_update(struct rte_mbuf *m, int16_t v)
{
- __rte_mbuf_sanity_check(m, 1);
+ __rte_mbuf_verify(m, 1);
do {
rte_mbuf_refcnt_update(m, v);
@@ -1512,7 +1515,7 @@ static inline void rte_pktmbuf_refcnt_update(struct rte_mbuf *m, int16_t v)
*/
static inline uint16_t rte_pktmbuf_headroom(const struct rte_mbuf *m)
{
- __rte_mbuf_sanity_check(m, 0);
+ __rte_mbuf_verify(m, 0);
return m->data_off;
}
@@ -1526,7 +1529,7 @@ static inline uint16_t rte_pktmbuf_headroom(const struct rte_mbuf *m)
*/
static inline uint16_t rte_pktmbuf_tailroom(const struct rte_mbuf *m)
{
- __rte_mbuf_sanity_check(m, 0);
+ __rte_mbuf_verify(m, 0);
return (uint16_t)(m->buf_len - rte_pktmbuf_headroom(m) -
m->data_len);
}
@@ -1541,7 +1544,7 @@ static inline uint16_t rte_pktmbuf_tailroom(const struct rte_mbuf *m)
*/
static inline struct rte_mbuf *rte_pktmbuf_lastseg(struct rte_mbuf *m)
{
- __rte_mbuf_sanity_check(m, 1);
+ __rte_mbuf_verify(m, 1);
while (m->next != NULL)
m = m->next;
return m;
@@ -1585,7 +1588,7 @@ static inline struct rte_mbuf *rte_pktmbuf_lastseg(struct rte_mbuf *m)
static inline char *rte_pktmbuf_prepend(struct rte_mbuf *m,
uint16_t len)
{
- __rte_mbuf_sanity_check(m, 1);
+ __rte_mbuf_verify(m, 1);
if (unlikely(len > rte_pktmbuf_headroom(m)))
return NULL;
@@ -1620,7 +1623,7 @@ static inline char *rte_pktmbuf_append(struct rte_mbuf *m, uint16_t len)
void *tail;
struct rte_mbuf *m_last;
- __rte_mbuf_sanity_check(m, 1);
+ __rte_mbuf_verify(m, 1);
m_last = rte_pktmbuf_lastseg(m);
if (unlikely(len > rte_pktmbuf_tailroom(m_last)))
@@ -1648,7 +1651,7 @@ static inline char *rte_pktmbuf_append(struct rte_mbuf *m, uint16_t len)
*/
static inline char *rte_pktmbuf_adj(struct rte_mbuf *m, uint16_t len)
{
- __rte_mbuf_sanity_check(m, 1);
+ __rte_mbuf_verify(m, 1);
if (unlikely(len > m->data_len))
return NULL;
@@ -1680,7 +1683,7 @@ static inline int rte_pktmbuf_trim(struct rte_mbuf *m, uint16_t len)
{
struct rte_mbuf *m_last;
- __rte_mbuf_sanity_check(m, 1);
+ __rte_mbuf_verify(m, 1);
m_last = rte_pktmbuf_lastseg(m);
if (unlikely(len > m_last->data_len))
@@ -1702,7 +1705,7 @@ static inline int rte_pktmbuf_trim(struct rte_mbuf *m, uint16_t len)
*/
static inline int rte_pktmbuf_is_contiguous(const struct rte_mbuf *m)
{
- __rte_mbuf_sanity_check(m, 1);
+ __rte_mbuf_verify(m, 1);
return m->nb_segs == 1;
}
diff --git a/lib/mbuf/version.map b/lib/mbuf/version.map
index daa65e2bbd..c85370e430 100644
--- a/lib/mbuf/version.map
+++ b/lib/mbuf/version.map
@@ -31,6 +31,7 @@ DPDK_24 {
rte_mbuf_set_platform_mempool_ops;
rte_mbuf_set_user_mempool_ops;
rte_mbuf_user_mempool_ops;
+ rte_mbuf_verify;
rte_pktmbuf_clone;
rte_pktmbuf_copy;
rte_pktmbuf_dump;
--
2.43.0
^ permalink raw reply [relevance 2%]
* Re: [PATCH v8 5/5] dts: add API doc generation
@ 2024-08-01 13:03 0% ` Juraj Linkeš
0 siblings, 0 replies; 200+ results
From: Juraj Linkeš @ 2024-08-01 13:03 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Honnappa.Nagarahalli, bruce.richardson, jspewock, probb,
paul.szczepanek, Luca.Vizzarro, npratte, dev
On 30. 7. 2024 15:51, Thomas Monjalon wrote:
> 12/07/2024 10:57, Juraj Linkeš:
>> The tool used to generate DTS API docs is Sphinx, which is already in
>> use in DPDK. The same configuration is used to preserve style with one
>> DTS-specific configuration (so that the DPDK docs are unchanged) that
>> modifies how the sidebar displays the content.
>
> What is changed in the sidebar?
>
These are the two changes:
html_theme_options = {
'collapse_navigation': False,
'navigation_depth': -1,
}
The first allows you to explore the structure without needing to enter
any specific section - it puts the + at each section so everything is
expandable.
The second just means that each section can be fully expanded (there's
no limit).
>
>> --- a/doc/api/doxy-api-index.md
>> +++ b/doc/api/doxy-api-index.md
>> @@ -244,3 +244,6 @@ The public API headers are grouped by topics:
>> [experimental APIs](@ref rte_compat.h),
>> [ABI versioning](@ref rte_function_versioning.h),
>> [version](@ref rte_version.h)
>> +
>> +- **tests**:
>> + [**DTS**](@dts_api_main_page)
>
> OK looks good
>
>
>> --- a/doc/api/doxy-api.conf.in
>> +++ b/doc/api/doxy-api.conf.in
>> @@ -124,6 +124,8 @@ SEARCHENGINE = YES
>> SORT_MEMBER_DOCS = NO
>> SOURCE_BROWSER = YES
>>
>> +ALIASES = "dts_api_main_page=@DTS_API_MAIN_PAGE@"
>
> Why is it needed?
> That's the only way to reference it in doxy-api-index.md?
> Would be nice to explain in the commit log.
>
I can add something to the commit log. The questions are answered below,
in your other related comment.
>> --- a/doc/api/meson.build
>> +++ b/doc/api/meson.build
>> +# A local reference must be relative to the main index.html page
>> +# The path below can't be taken from the DTS meson file as that would
>> +# require recursive subdir traversal (doc, dts, then doc again)
>
> This comment is really obscure.
>
I guess it is. I just wanted to explain that there's not way to do this
without spelling out the path this way. At least I didn't find a way.
Should I remove the comment or reword it?
>> +cdata.set('DTS_API_MAIN_PAGE', join_paths('..', 'dts', 'html', 'index.html'))
>
> Oh I think I get it:
> - DTS_API_MAIN_PAGE is the Meson variable
> - dts_api_main_page is the Doxygen variable
>
Yes, this is a way to make it work. Maybe there's something else (I'm
not that familiar with Doxygen), but from what I can tell, there wasn't
a command line option that would set a variable (passing the path form
Meson to Doxygen) and nothing else I found worked.
Is this solution ok? If we want to explore something else, is there
someone with more experience with Doxygen who could help?
>
>> +# Napoleon enables the Google format of Python doscstrings, used in DTS
>> +# Intersphinx allows linking to external projects, such as Python docs, also used in DTS
>
> Close sentences with a dot, it is easier to read.
>
Ack.
>> +extensions = ['sphinx.ext.napoleon', 'sphinx.ext.intersphinx']
>> +
>> +# DTS Python docstring options
>> +autodoc_default_options = {
>> + 'members': True,
>> + 'member-order': 'bysource',
>> + 'show-inheritance': True,
>> +}
>> +autodoc_class_signature = 'separated'
>> +autodoc_typehints = 'both'
>> +autodoc_typehints_format = 'short'
>> +autodoc_typehints_description_target = 'documented'
>> +napoleon_numpy_docstring = False
>> +napoleon_attr_annotations = True
>> +napoleon_preprocess_types = True
>> +add_module_names = False
>> +toc_object_entries = True
>> +toc_object_entries_show_parents = 'hide'
>> +intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
>> +
>> +dts_root = environ.get('DTS_ROOT')
>
> Why does it need to be passed as an environment variable?
> Isn't it a fixed absolute path?
>
The path to DTS needs to be passed in some way (and added to sys.path)
so that Sphinx knows where the sources are in order to import them.
Do you want us to not pass the path, but just hardcode it here? I didn't
really think about that, maybe that could work.
>> +if dts_root:
>> + path.append(dts_root)
>> + # DTS Sidebar config
>> + html_theme_options = {
>> + 'collapse_navigation': False,
>> + 'navigation_depth': -1,
>> + }
>
> [...]
>
>> +To build DTS API docs, install the dependencies with Poetry, then enter its shell:
>
> I don't plan to use Poetry on my machine.
> Can we simply describe the dependencies even if the versions are not specified?
>
The reason we don't list the dependencies anywhere is that doing it with
Poetry is much easier (and a bit safer, as Poetry is going to install
tested versions).
But I can add references to the two relevant sections of
dts/pyproject.toml which contain the dependencies with a note that they
can be installed with pip (and I guess that would be another
dependency), but at that point it's that not much different than using
Poetry.
>> +
>> +.. code-block:: console
>> +
>> + poetry install --no-root --with docs
>> + poetry shell
>> +
>> +The documentation is built using the standard DPDK build system.
>> +After executing the meson command and entering Poetry's shell, build the documentation with:
>> +
>> +.. code-block:: console
>> +
>> + ninja -C build dts-doc
>
> Don't we rely on the Meson option "enable_docs"?
I had a discussion about this with Bruce, but I can't find it anywhere,
so here's what I remember:
1. We didn't want to tie the dts api doc build to dpdk doc build because
of the dependencies.
2. There's a way to build docs without the enable_docs option (running
ninja with the target), which is what we added for dts. This doesn't tie
the dts api doc build to the dpdk doc build.
3. We had an "enable_dts_docs" Meson option in the past (to keep it
separate from dpdk doc build), but decided to drop it. My memory is hazy
on this, but I think it was, again, because of the additional steps
needed to bring up the dependency (poetry shell) - at that point,
supporting just the ninja build way is sufficient. Bruce may shed more
light on this.
>> +
>> +The output is generated in ``build/doc/api/dts/html``.
>> +
>> +.. Note::
>
> In general the RST expressions are lowercase.
>
Ack.
>> +
>> + Make sure to fix any Sphinx warnings when adding or updating docstrings,
>> + and also run the ``devtools/dts-check-format.sh`` script and address any issues it finds.
>
> It looks like something to write in the contributing guide.
>
I could add it there, where is the right place? In patches.rst, section
"Checking the Patches"?
>
>> +++ b/dts/doc/meson.build
>> @@ -0,0 +1,27 @@
>> +# SPDX-License-Identifier: BSD-3-Clause
>> +# Copyright(c) 2023 PANTHEON.tech s.r.o.
>> +
>> +sphinx = find_program('sphinx-build', required: false)
>> +sphinx_apidoc = find_program('sphinx-apidoc', required: false)
>> +
>> +if not sphinx.found() or not sphinx_apidoc.found()
>
> You should include the option "enable_docs" here.
>
>> + subdir_done()
>> +endif
>
>
>
^ permalink raw reply [relevance 0%]
* [PATCH v10 5/5] dts: add API doc generation
@ 2024-08-01 9:37 3% ` Juraj Linkeš
0 siblings, 0 replies; 200+ results
From: Juraj Linkeš @ 2024-08-01 9:37 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, bruce.richardson, jspewock, probb,
paul.szczepanek, Luca.Vizzarro, npratte
Cc: dev, Juraj Linkeš, Luca Vizzarro
The tool used to generate DTS API docs is Sphinx, which is already in
use in DPDK. The same configuration is used to preserve style with one
DTS-specific configuration (so that the DPDK docs are unchanged) that
modifies how the sidebar displays the content.
Sphinx generates the documentation from Python docstrings. The docstring
format is the Google format [0] which requires the sphinx.ext.napoleon
extension. The other extension, sphinx.ext.intersphinx, enables linking
to objects in external documentations, such as the Python documentation.
There are two requirements for building DTS docs:
* The same Python version as DTS or higher, because Sphinx imports the
code.
* Also the same Python packages as DTS, for the same reason.
[0] https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Luca Vizzarro <luca.vizzarro@arm.com>
Tested-by: Nicholas Pratte <npratte@iol.unh.edu>
---
buildtools/call-sphinx-build.py | 5 ++++-
doc/api/doxy-api-index.md | 3 +++
doc/api/doxy-api.conf.in | 2 ++
doc/api/meson.build | 4 ++++
doc/guides/conf.py | 33 +++++++++++++++++++++++++++++++-
doc/guides/meson.build | 1 +
doc/guides/tools/dts.rst | 34 ++++++++++++++++++++++++++++++++-
dts/doc/meson.build | 27 ++++++++++++++++++++++++++
dts/meson.build | 16 ++++++++++++++++
meson.build | 1 +
10 files changed, 123 insertions(+), 3 deletions(-)
create mode 100644 dts/doc/meson.build
create mode 100644 dts/meson.build
diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
index 2034160049..c55d4f6bc9 100755
--- a/buildtools/call-sphinx-build.py
+++ b/buildtools/call-sphinx-build.py
@@ -16,10 +16,13 @@
parser.add_argument('version')
parser.add_argument('src')
parser.add_argument('dst')
+parser.add_argument('--dts-root', default=None)
args, extra_args = parser.parse_known_args()
# set the version in environment for sphinx to pick up
os.environ['DPDK_VERSION'] = args.version
+if args.dts_root:
+ os.environ['DTS_ROOT'] = args.dts_root
sphinx_cmd = [args.sphinx] + extra_args
@@ -46,7 +49,7 @@
css = 'custom.css'
src_css = join(args.src, css)
dst_css = join(args.dst, 'html', '_static', 'css', css)
-if not os.path.exists(dst_css) or not filecmp.cmp(src_css, dst_css):
+if os.path.exists(src_css) and (not os.path.exists(dst_css) or not filecmp.cmp(src_css, dst_css)):
os.makedirs(os.path.dirname(dst_css), exist_ok=True)
shutil.copyfile(src_css, dst_css)
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index f9f0300126..ab223bcdf7 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -245,3 +245,6 @@ The public API headers are grouped by topics:
[experimental APIs](@ref rte_compat.h),
[ABI versioning](@ref rte_function_versioning.h),
[version](@ref rte_version.h)
+
+- **tests**:
+ [**DTS**](@dts_api_main_page)
diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index a8823c046f..c94f02d411 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -124,6 +124,8 @@ SEARCHENGINE = YES
SORT_MEMBER_DOCS = NO
SOURCE_BROWSER = YES
+ALIASES = "dts_api_main_page=@DTS_API_MAIN_PAGE@"
+
EXAMPLE_PATH = @TOPDIR@/examples
EXAMPLE_PATTERNS = *.c
EXAMPLE_RECURSIVE = YES
diff --git a/doc/api/meson.build b/doc/api/meson.build
index b828b1ed66..ffc75d7b5a 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -41,6 +41,10 @@ cdata.set('WARN_AS_ERROR', 'NO')
if get_option('werror')
cdata.set('WARN_AS_ERROR', 'YES')
endif
+# A local reference must be relative to the main index.html page
+# The path below can't be taken from the DTS meson file as that would
+# require recursive subdir traversal (doc, dts, then doc again)
+cdata.set('DTS_API_MAIN_PAGE', join_paths('..', 'dts', 'html', 'index.html'))
# configure HTML Doxygen run
html_cdata = configuration_data()
diff --git a/doc/guides/conf.py b/doc/guides/conf.py
index 8b440fb2a9..b442a1f76c 100644
--- a/doc/guides/conf.py
+++ b/doc/guides/conf.py
@@ -9,7 +9,7 @@
from os import environ
from os.path import basename, dirname
from os.path import join as path_join
-from sys import argv, stderr
+from sys import argv, stderr, path
import configparser
@@ -23,6 +23,37 @@
file=stderr)
pass
+# Napoleon enables the Google format of Python doscstrings, used in DTS
+# Intersphinx allows linking to external projects, such as Python docs, also used in DTS
+extensions = ['sphinx.ext.napoleon', 'sphinx.ext.intersphinx']
+
+# DTS Python docstring options
+autodoc_default_options = {
+ 'members': True,
+ 'member-order': 'bysource',
+ 'show-inheritance': True,
+}
+autodoc_class_signature = 'separated'
+autodoc_typehints = 'both'
+autodoc_typehints_format = 'short'
+autodoc_typehints_description_target = 'documented'
+napoleon_numpy_docstring = False
+napoleon_attr_annotations = True
+napoleon_preprocess_types = True
+add_module_names = False
+toc_object_entries = True
+toc_object_entries_show_parents = 'hide'
+intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
+
+dts_root = environ.get('DTS_ROOT')
+if dts_root:
+ path.append(dts_root)
+ # DTS Sidebar config
+ html_theme_options = {
+ 'collapse_navigation': False,
+ 'navigation_depth': -1,
+ }
+
stop_on_error = ('-W' in argv)
project = 'Data Plane Development Kit'
diff --git a/doc/guides/meson.build b/doc/guides/meson.build
index f8bbfba9f5..b34b7b8eb0 100644
--- a/doc/guides/meson.build
+++ b/doc/guides/meson.build
@@ -1,6 +1,7 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Intel Corporation
+doc_guides_source_dir = meson.current_source_dir()
sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
if not sphinx.found()
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 515b15e4d8..77df7a0378 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -292,7 +292,12 @@ and try not to divert much from it.
The :ref:`DTS developer tools <dts_dev_tools>` will issue warnings
when some of the basics are not met.
-The code must be properly documented with docstrings.
+The API documentation, which is a helpful reference when developing, may be accessed
+in the code directly or generated with the :ref:`API docs build steps <building_api_docs>`.
+When adding new files or modifying the directory structure,
+the corresponding changes must be made to DTS api doc sources in ``dts/doc``.
+
+Speaking of which, the code must be properly documented with docstrings.
The style must conform to the `Google style
<https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings>`_.
See an example of the style `here
@@ -427,6 +432,33 @@ the DTS code check and format script.
Refer to the script for usage: ``devtools/dts-check-format.sh -h``.
+.. _building_api_docs:
+
+Building DTS API docs
+---------------------
+
+To build DTS API docs, install the dependencies with Poetry, then enter its shell:
+
+.. code-block:: console
+
+ poetry install --no-root --with docs
+ poetry shell
+
+The documentation is built using the standard DPDK build system.
+After executing the meson command and entering Poetry's shell, build the documentation with:
+
+.. code-block:: console
+
+ ninja -C build dts-doc
+
+The output is generated in ``build/doc/api/dts/html``.
+
+.. Note::
+
+ Make sure to fix any Sphinx warnings when adding or updating docstrings,
+ and also run the ``devtools/dts-check-format.sh`` script and address any issues it finds.
+
+
Configuration Schema
--------------------
diff --git a/dts/doc/meson.build b/dts/doc/meson.build
new file mode 100644
index 0000000000..01b7b51034
--- /dev/null
+++ b/dts/doc/meson.build
@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+sphinx = find_program('sphinx-build', required: false)
+sphinx_apidoc = find_program('sphinx-apidoc', required: false)
+
+if not sphinx.found() or not sphinx_apidoc.found()
+ subdir_done()
+endif
+
+dts_doc_api_build_dir = join_paths(doc_api_build_dir, 'dts')
+
+extra_sphinx_args = ['-E', '-c', doc_guides_source_dir, '--dts-root', dts_dir]
+if get_option('werror')
+ extra_sphinx_args += '-W'
+endif
+
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk', 'dts')
+dts_api_html = custom_target('dts_api_html',
+ output: 'html',
+ command: [sphinx_wrapper, sphinx, meson.project_version(),
+ meson.current_source_dir(), dts_doc_api_build_dir, extra_sphinx_args],
+ build_by_default: false,
+ install: get_option('enable_docs'),
+ install_dir: htmldir)
+doc_targets += dts_api_html
+doc_target_names += 'DTS_API_HTML'
diff --git a/dts/meson.build b/dts/meson.build
new file mode 100644
index 0000000000..e8ce0f06ac
--- /dev/null
+++ b/dts/meson.build
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+doc_targets = []
+doc_target_names = []
+dts_dir = meson.current_source_dir()
+
+subdir('doc')
+
+if doc_targets.length() == 0
+ message = 'No docs targets found'
+else
+ message = 'Built docs:'
+endif
+run_target('dts-doc', command: [echo, message, doc_target_names],
+ depends: doc_targets)
diff --git a/meson.build b/meson.build
index 8b248d4505..835973a0ce 100644
--- a/meson.build
+++ b/meson.build
@@ -87,6 +87,7 @@ subdir('app')
# build docs
subdir('doc')
+subdir('dts')
# build any examples explicitly requested - useful for developers - and
# install any example code into the appropriate install path
--
2.34.1
^ permalink raw reply [relevance 3%]
* [PATCH v9 5/5] dts: add API doc generation
@ 2024-08-01 9:18 3% ` Juraj Linkeš
0 siblings, 0 replies; 200+ results
From: Juraj Linkeš @ 2024-08-01 9:18 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, bruce.richardson, jspewock, probb,
paul.szczepanek, Luca.Vizzarro, npratte
Cc: dev, Juraj Linkeš, Luca Vizzarro
The tool used to generate DTS API docs is Sphinx, which is already in
use in DPDK. The same configuration is used to preserve style with one
DTS-specific configuration (so that the DPDK docs are unchanged) that
modifies how the sidebar displays the content.
Sphinx generates the documentation from Python docstrings. The docstring
format is the Google format [0] which requires the sphinx.ext.napoleon
extension. The other extension, sphinx.ext.intersphinx, enables linking
to objects in external documentations, such as the Python documentation.
There are two requirements for building DTS docs:
* The same Python version as DTS or higher, because Sphinx imports the
code.
* Also the same Python packages as DTS, for the same reason.
[0] https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
Reviewed-by: Luca Vizzarro <luca.vizzarro@arm.com>
Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Luca Vizzarro <luca.vizzarro@arm.com>
Tested-by: Nicholas Pratte <npratte@iol.unh.edu>
---
buildtools/call-sphinx-build.py | 3 +++
doc/api/doxy-api-index.md | 3 +++
doc/api/doxy-api.conf.in | 2 ++
doc/api/meson.build | 4 ++++
doc/guides/conf.py | 33 +++++++++++++++++++++++++++++++-
doc/guides/meson.build | 1 +
doc/guides/tools/dts.rst | 34 ++++++++++++++++++++++++++++++++-
dts/doc/meson.build | 27 ++++++++++++++++++++++++++
dts/meson.build | 16 ++++++++++++++++
meson.build | 1 +
10 files changed, 122 insertions(+), 2 deletions(-)
create mode 100644 dts/doc/meson.build
create mode 100644 dts/meson.build
diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
index 2034160049..102f496599 100755
--- a/buildtools/call-sphinx-build.py
+++ b/buildtools/call-sphinx-build.py
@@ -16,10 +16,13 @@
parser.add_argument('version')
parser.add_argument('src')
parser.add_argument('dst')
+parser.add_argument('--dts-root', default=None)
args, extra_args = parser.parse_known_args()
# set the version in environment for sphinx to pick up
os.environ['DPDK_VERSION'] = args.version
+if args.dts_root:
+ os.environ['DTS_ROOT'] = args.dts_root
sphinx_cmd = [args.sphinx] + extra_args
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index f9f0300126..ab223bcdf7 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -245,3 +245,6 @@ The public API headers are grouped by topics:
[experimental APIs](@ref rte_compat.h),
[ABI versioning](@ref rte_function_versioning.h),
[version](@ref rte_version.h)
+
+- **tests**:
+ [**DTS**](@dts_api_main_page)
diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
index a8823c046f..c94f02d411 100644
--- a/doc/api/doxy-api.conf.in
+++ b/doc/api/doxy-api.conf.in
@@ -124,6 +124,8 @@ SEARCHENGINE = YES
SORT_MEMBER_DOCS = NO
SOURCE_BROWSER = YES
+ALIASES = "dts_api_main_page=@DTS_API_MAIN_PAGE@"
+
EXAMPLE_PATH = @TOPDIR@/examples
EXAMPLE_PATTERNS = *.c
EXAMPLE_RECURSIVE = YES
diff --git a/doc/api/meson.build b/doc/api/meson.build
index b828b1ed66..ffc75d7b5a 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -41,6 +41,10 @@ cdata.set('WARN_AS_ERROR', 'NO')
if get_option('werror')
cdata.set('WARN_AS_ERROR', 'YES')
endif
+# A local reference must be relative to the main index.html page
+# The path below can't be taken from the DTS meson file as that would
+# require recursive subdir traversal (doc, dts, then doc again)
+cdata.set('DTS_API_MAIN_PAGE', join_paths('..', 'dts', 'html', 'index.html'))
# configure HTML Doxygen run
html_cdata = configuration_data()
diff --git a/doc/guides/conf.py b/doc/guides/conf.py
index 8b440fb2a9..b442a1f76c 100644
--- a/doc/guides/conf.py
+++ b/doc/guides/conf.py
@@ -9,7 +9,7 @@
from os import environ
from os.path import basename, dirname
from os.path import join as path_join
-from sys import argv, stderr
+from sys import argv, stderr, path
import configparser
@@ -23,6 +23,37 @@
file=stderr)
pass
+# Napoleon enables the Google format of Python doscstrings, used in DTS
+# Intersphinx allows linking to external projects, such as Python docs, also used in DTS
+extensions = ['sphinx.ext.napoleon', 'sphinx.ext.intersphinx']
+
+# DTS Python docstring options
+autodoc_default_options = {
+ 'members': True,
+ 'member-order': 'bysource',
+ 'show-inheritance': True,
+}
+autodoc_class_signature = 'separated'
+autodoc_typehints = 'both'
+autodoc_typehints_format = 'short'
+autodoc_typehints_description_target = 'documented'
+napoleon_numpy_docstring = False
+napoleon_attr_annotations = True
+napoleon_preprocess_types = True
+add_module_names = False
+toc_object_entries = True
+toc_object_entries_show_parents = 'hide'
+intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
+
+dts_root = environ.get('DTS_ROOT')
+if dts_root:
+ path.append(dts_root)
+ # DTS Sidebar config
+ html_theme_options = {
+ 'collapse_navigation': False,
+ 'navigation_depth': -1,
+ }
+
stop_on_error = ('-W' in argv)
project = 'Data Plane Development Kit'
diff --git a/doc/guides/meson.build b/doc/guides/meson.build
index f8bbfba9f5..b34b7b8eb0 100644
--- a/doc/guides/meson.build
+++ b/doc/guides/meson.build
@@ -1,6 +1,7 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2018 Intel Corporation
+doc_guides_source_dir = meson.current_source_dir()
sphinx = find_program('sphinx-build', required: get_option('enable_docs'))
if not sphinx.found()
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 515b15e4d8..77df7a0378 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -292,7 +292,12 @@ and try not to divert much from it.
The :ref:`DTS developer tools <dts_dev_tools>` will issue warnings
when some of the basics are not met.
-The code must be properly documented with docstrings.
+The API documentation, which is a helpful reference when developing, may be accessed
+in the code directly or generated with the :ref:`API docs build steps <building_api_docs>`.
+When adding new files or modifying the directory structure,
+the corresponding changes must be made to DTS api doc sources in ``dts/doc``.
+
+Speaking of which, the code must be properly documented with docstrings.
The style must conform to the `Google style
<https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings>`_.
See an example of the style `here
@@ -427,6 +432,33 @@ the DTS code check and format script.
Refer to the script for usage: ``devtools/dts-check-format.sh -h``.
+.. _building_api_docs:
+
+Building DTS API docs
+---------------------
+
+To build DTS API docs, install the dependencies with Poetry, then enter its shell:
+
+.. code-block:: console
+
+ poetry install --no-root --with docs
+ poetry shell
+
+The documentation is built using the standard DPDK build system.
+After executing the meson command and entering Poetry's shell, build the documentation with:
+
+.. code-block:: console
+
+ ninja -C build dts-doc
+
+The output is generated in ``build/doc/api/dts/html``.
+
+.. Note::
+
+ Make sure to fix any Sphinx warnings when adding or updating docstrings,
+ and also run the ``devtools/dts-check-format.sh`` script and address any issues it finds.
+
+
Configuration Schema
--------------------
diff --git a/dts/doc/meson.build b/dts/doc/meson.build
new file mode 100644
index 0000000000..01b7b51034
--- /dev/null
+++ b/dts/doc/meson.build
@@ -0,0 +1,27 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+sphinx = find_program('sphinx-build', required: false)
+sphinx_apidoc = find_program('sphinx-apidoc', required: false)
+
+if not sphinx.found() or not sphinx_apidoc.found()
+ subdir_done()
+endif
+
+dts_doc_api_build_dir = join_paths(doc_api_build_dir, 'dts')
+
+extra_sphinx_args = ['-E', '-c', doc_guides_source_dir, '--dts-root', dts_dir]
+if get_option('werror')
+ extra_sphinx_args += '-W'
+endif
+
+htmldir = join_paths(get_option('datadir'), 'doc', 'dpdk', 'dts')
+dts_api_html = custom_target('dts_api_html',
+ output: 'html',
+ command: [sphinx_wrapper, sphinx, meson.project_version(),
+ meson.current_source_dir(), dts_doc_api_build_dir, extra_sphinx_args],
+ build_by_default: false,
+ install: get_option('enable_docs'),
+ install_dir: htmldir)
+doc_targets += dts_api_html
+doc_target_names += 'DTS_API_HTML'
diff --git a/dts/meson.build b/dts/meson.build
new file mode 100644
index 0000000000..e8ce0f06ac
--- /dev/null
+++ b/dts/meson.build
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 PANTHEON.tech s.r.o.
+
+doc_targets = []
+doc_target_names = []
+dts_dir = meson.current_source_dir()
+
+subdir('doc')
+
+if doc_targets.length() == 0
+ message = 'No docs targets found'
+else
+ message = 'Built docs:'
+endif
+run_target('dts-doc', command: [echo, message, doc_target_names],
+ depends: doc_targets)
diff --git a/meson.build b/meson.build
index 8b248d4505..835973a0ce 100644
--- a/meson.build
+++ b/meson.build
@@ -87,6 +87,7 @@ subdir('app')
# build docs
subdir('doc')
+subdir('dts')
# build any examples explicitly requested - useful for developers - and
# install any example code into the appropriate install path
--
2.34.1
^ permalink raw reply [relevance 3%]
* DPDK 24.07 released
@ 2024-07-31 19:22 3% Thomas Monjalon
0 siblings, 0 replies; 200+ results
From: Thomas Monjalon @ 2024-07-31 19:22 UTC (permalink / raw)
To: announce
A new major release is available:
https://fast.dpdk.org/rel/dpdk-24.07.tar.xz
This is the work achieved during the last months:
954 commits from 191 authors
1631 files changed, 80005 insertions(+), 25069 deletions(-)
It is not planned to start a maintenance branch for 24.07.
This version is ABI-compatible with 23.11 and 24.03.
Below are some new features:
- pointer compression library
- SVE support in the hash library
- FEC support in Intel i40e and ice
- Intel E830 support in ice driver
- Napatech ntnic driver initialization
- AMD Pensando ionic crypto driver
- UADK compress driver
- Marvell Odyssey ODM DMA driver
- log cleanups in drivers
- more cleanups to prepare MSVC build
More details in the release notes:
https://doc.dpdk.org/guides/rel_notes/release_24_07.html
There are 49 new contributors (including authors, reviewers and testers).
Welcome to Adrian Pielech, Alessio Igor Bogani, Alex Chapman,
Alexander Skorichenko, Anthony Harivel, Aviraj CJ, Barbara Skobiej,
Chenming Chang, Daniel Gregory, Daniil Ushkov, Dean Marx, Eryk Rybak,
Francis Racicot, Haoqian He, Harjot Singh, Hongbo Li, Igor Gutorov,
Jan Sokolowski, Jedrzej Jagielski, Jiang Yu, Joel Kavanagh, Jun Wang,
Kiran Vedere, Mahmoud Maatuq, Marcin Jurczak, Marek Mical,
Marek Zalfresso-jundzillo, Michael Theodore Stolarchuk, Pabitra Dalai,
Pawel Sobczyk, Piotr Raczynski, Potnuri Bharat Teja,
Prathisna Padmasanan, Przemek Kitszel, Radoslaw Tyl, Remigiusz Konca,
Serhii Iliushyk, Shaiq Wani, Shreesh Adiga, Shuo Li, Soumyadeep Hore,
Sriram Yagnaraman, Tathagat Priyadarshi, Tomasz Wakula, Varun Sethi,
Vipin Padmam Ramesh, Waldemar Dworakowski, Yoan Picchi, Yochai Hagvi,
and Yuan Zhiyuan.
Below is the number of commits per employer (with authors count):
247 Intel (77)
135 NVIDIA (23)
111 Corigine (5)
91 Marvell (18)
54 Red Hat (5)
46 AMD (5)
40 Arm (5)
39 networkplumber.org (1)
36 Microsoft (3)
23 NXP (7)
22 Napatech (2)
21 Trustnet (1)
18 Huawei (5)
15 Amazon (1)
...
A big thank to all courageous people who took on the non rewarding task
of reviewing other's job.
Based on Reviewed-by and Acked-by tags, the top non-PMD reviewers are:
35 Morten Brørup <mb@smartsharesystems.com>
27 Juraj Linkeš <juraj.linkes@pantheon.tech>
25 Ferruh Yigit <ferruh.yigit@amd.com>
24 Jeremy Spewock <jspewock@iol.unh.edu>
24 Akhil Goyal <gakhil@marvell.com>
22 Paul Szczepanek <paul.szczepanek@arm.com>
19 Bruce Richardson <bruce.richardson@intel.com>
18 Stephen Hemminger <stephen@networkplumber.org>
The next version will be 24.11 in November.
The new features for 24.11 can be submitted during the next 5 weeks:
http://core.dpdk.org/roadmap#dates
Please share your roadmap.
Don't forget to register for the DPDK Summit in September:
https://events.linuxfoundation.org/dpdk-summit/
Thanks everyone, see you in Montreal
^ permalink raw reply [relevance 3%]
* Re: [PATCH v4] doc: announce changes to dma device structures
@ 2024-07-31 16:06 3% ` Thomas Monjalon
0 siblings, 0 replies; 200+ results
From: Thomas Monjalon @ 2024-07-31 16:06 UTC (permalink / raw)
To: Amit Prakash Shukla
Cc: dev, fengchengwen, kevin.laatz, bruce.richardson, conor.walsh,
gmuthukrishn, vvelumuri, g.singh, sachin.saxena, hemant.agrawal,
dev, jerinj, vattunuru, anoobj, mb, Jerin Jacob
31/07/2024 13:01, Thomas Monjalon:
> 30/07/2024 19:27, Jerin Jacob:
> > On Tue, Jul 30, 2024 at 8:25 PM Amit Prakash Shukla
> > <amitprakashs@marvell.com> wrote:
> > >
> > > A new flag RTE_DMA_CAPA_QOS will be introduced to advertise dma
> > > device's QoS capability. In order to support the parameters for this
> > > flag, new fields will be added in rte_dma_info and rte_dma_conf
> > > structures to get device supported priority levels and to configure the
> > > required priority level.
> > >
> > > Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
> > > Signed-off-by: Amit Prakash Shukla <amitprakashs@marvell.com>
> >
> >
> > Acked-by: Jerin Jacob <jerinj@marvell.com>
>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
The RFC and the deprecation notices are sent a bit late.
We cannot conclude there is consensus.
I propose to raise it to the techboard if an ABI breakage is still required for 24.11.
As dmadev is quite new, I don't think it is big issue.
^ permalink raw reply [relevance 3%]
* Re: [PATCH] doc: announce cryptodev changes to offload RSA in VirtIO
@ 2024-07-31 14:26 0% ` Thomas Monjalon
2024-08-07 13:31 0% ` Kusztal, ArkadiuszX
0 siblings, 1 reply; 200+ results
From: Thomas Monjalon @ 2024-07-31 14:26 UTC (permalink / raw)
To: Gowrishankar Muthukrishnan
Cc: dev, Kusztal, ArkadiuszX, dev, Anoob Joseph, Richardson, Bruce,
ciara.power, Jerin Jacob, fanzhang.oss, Ji, Kai,
jack.bond-preston, Marchand, David, hemant.agrawal,
De Lara Guarch, Pablo, Trahe, Fiona, Doherty, Declan, matan,
ruifeng.wang, Gujjar, Abhinandan S, maxime.coquelin, chenbox,
sunilprakashrao.uttarwar, andrew.boyer, ajit.khaparde,
raveendra.padasalagi, vikas.gupta, zhangfei.gao, g.singh,
jianjay.zhou, Daly, Lee
I'm not sure why we don't have a consensus on an idea proposed as RFC in September 2023.
Because there is not enough involvement outside of the Marvell team,
I will keep a vague announce for the first item:
cryptodev: Some changes may happen to manage RSA padding for virtio-crypto.
The second item is applied verbatim, thanks.
31/07/2024 14:51, Thomas Monjalon:
> 30/07/2024 16:39, Gowrishankar Muthukrishnan:
> > Hi,
> > We need to fix padding info in DPDK as per VirtIO specification in order to support RSA in virtio devices. VirtIO-crypto specification and DPDK specification differs in the way padding is handled.
> > With current DPDK & virtio specification, it is impossible to support RSA in virtio-crypto. If you think DPDK spec should not be modified, we will try to amend the virtIO spec to match DPDK, but since we do not know if the virtIO community would accept, can we merge the deprecation notice?
>
> There is a long list of Cc but I see no support outside of Marvell.
>
>
>
> > >>> +* cryptodev: The struct rte_crypto_rsa_padding will be moved from
> > >>> + rte_crypto_rsa_op_param struct to rte_crypto_rsa_xform struct,
> > >>> + breaking ABI. The new location is recommended to comply with
> > >>> + virtio-crypto specification. Applications and drivers using
> > >>> + this struct will be updated.
> > >>> +
> >
> >
> > >> The problem here, I see is that there is one private key but multiple combinations of padding.
> > >> Therefore, for every padding variation, we need to copy the same private key anew, duplicating it in memory.
> > >> The only reason for me to keep a session-like struct in asymmetric crypto was exactly this.
> > >
> > > Each padding scheme in RSA has its own pros and cons (in terms of implementations as well).
> > > When we share the same private key for Sign (and its public key in case of Encryption) between
> > > multiple crypto ops (varying by padding schemes among cops), a vulnerable attack against one scheme
> > > could potentially open door to used private key in the session and hence take advantage
> > > on other crypto operations.
> > >
> > > I think, this could be one reason for why VirtIO spec mandates padding info as session parameter.
> > > Hence, more than duplicating in memory, private and public keys are secured and in catastrophe,
> > > only that session could be destroyed.
> >
> >
> > >>> +* cryptodev: The rte_crypto_rsa_xform struct member to hold private key
> > >>> + in either exponent or quintuple format is changed from union to
> > >>> +struct
> > >>> + data type. This change is to support ASN.1 syntax (RFC 3447 Appendix A.1.2).
> > >>> + This change will not break existing applications.
> > > >
> > > > This one I agree. RFC 8017 obsoletes RFC 3447.
^ permalink raw reply [relevance 0%]
Results 201-400 of ~18000 next (older) | prev (newer) | reverse | sort options + mbox downloads above
-- links below jump to the message on this page --
2021-04-12 21:53 [dpdk-dev] [PATCH] devtools: test different build types Thomas Monjalon
2021-08-08 12:51 ` [dpdk-dev] [PATCH v3 0/5] more build tests Thomas Monjalon
2021-08-08 12:51 ` [dpdk-dev] [PATCH v3 5/5] devtools: test different build types Thomas Monjalon
2024-08-15 16:26 0% ` Stephen Hemminger
2023-02-28 9:39 [RFC 0/2] Add high-performance timer facility Mattias Rönnblom
2023-03-15 17:03 ` [RFC v2 " Mattias Rönnblom
2024-10-03 18:36 0% ` Stephen Hemminger
2024-10-03 21:32 0% ` Morten Brørup
2024-10-06 13:02 0% ` Mattias Rönnblom
2024-10-06 13:43 0% ` Morten Brørup
2024-10-06 14:43 0% ` Mattias Rönnblom
2023-03-14 12:48 [PATCH 0/5] fix segment fault when parse args Chengwen Feng
2024-10-09 4:50 ` [PATCH v6 0/4] " Chengwen Feng
2024-10-09 4:50 3% ` [PATCH v6 1/4] kvargs: add one new process API Chengwen Feng
2023-09-29 15:42 [PATCH] lib/hash: new feature adding existing key David Marchand
2023-10-23 8:29 ` [PATCH v2] " Abdullah Ömer Yamaç
2024-02-16 12:43 ` Thomas Monjalon
2024-10-03 22:37 0% ` Stephen Hemminger
2023-11-15 13:36 [PATCH v1 0/2] dts: api docs generation Juraj Linkeš
2024-07-12 8:57 ` [PATCH v8 0/5] dts: API " Juraj Linkeš
2024-07-12 8:57 ` [PATCH v8 5/5] dts: add API doc generation Juraj Linkeš
2024-07-30 13:51 ` Thomas Monjalon
2024-08-01 13:03 0% ` Juraj Linkeš
2024-08-01 9:18 ` [PATCH v9 0/5] dts: API docs generation Juraj Linkeš
2024-08-01 9:18 3% ` [PATCH v9 5/5] dts: add API doc generation Juraj Linkeš
2024-08-01 9:37 ` [PATCH v10 0/5] dts: API docs generation Juraj Linkeš
2024-08-01 9:37 3% ` [PATCH v10 5/5] dts: add API doc generation Juraj Linkeš
2024-08-05 13:59 ` [PATCH v11 0/5] dts: API docs generation Juraj Linkeš
2024-08-05 13:59 2% ` [PATCH v11 5/5] dts: add API doc generation Juraj Linkeš
2024-08-06 6:13 ` [PATCH v12 0/5] dts: API docs generation Juraj Linkeš
2024-08-06 6:14 2% ` [PATCH v12 5/5] dts: add API doc generation Juraj Linkeš
2024-08-06 8:46 ` [PATCH v13 0/6] API docs generation Juraj Linkeš
2024-08-06 8:46 2% ` [PATCH v13 6/6] dts: add API doc generation Juraj Linkeš
2024-08-06 11:17 ` [PATCH v14 0/6] API docs generation Juraj Linkeš
2024-08-06 11:17 2% ` [PATCH v14 6/6] dts: add API doc generation Juraj Linkeš
2024-08-06 15:19 ` [PATCH v15 0/5] API docs generation Juraj Linkeš
2024-08-06 15:19 2% ` [PATCH v15 5/5] dts: add API doc generation Juraj Linkeš
2024-08-08 8:54 ` [PATCH v16 0/5] API docs generation Juraj Linkeš
2024-08-08 8:54 2% ` [PATCH v16 5/5] dts: add API doc generation Juraj Linkeš
2024-08-14 15:05 ` [PATCH v17 0/5] API docs generation Juraj Linkeš
2024-08-14 15:05 2% ` [PATCH v17 5/5] dts: add API doc generation Juraj Linkeš
2024-08-20 13:18 ` [PATCH v18 0/5] API docs generation Juraj Linkeš
2024-08-20 13:18 2% ` [PATCH v18 5/5] dts: add API doc generation Juraj Linkeš
2024-08-21 15:02 ` [PATCH v19 0/5] DTS API docs generation Juraj Linkeš
2024-08-21 15:02 2% ` [PATCH v19 5/5] dts: add API doc generation Juraj Linkeš
2023-12-14 1:56 [PATCH] ethdev: add dump regs for telemetry Jie Hai
2024-09-05 6:26 ` [RESEND v6 0/8] support dump reigser names and filter Jie Hai
2024-09-05 6:26 4% ` [RESEND v6 1/8] ethdev: support report register " Jie Hai
2024-09-14 7:13 ` [PATCH v7 0/8] support dump reigser " Jie Hai
2024-09-14 7:13 4% ` [PATCH v7 1/8] ethdev: support report register " Jie Hai
2024-09-25 6:40 ` [PATCH v8 0/8] support dump " Jie Hai
2024-09-25 6:40 4% ` [PATCH v8 1/8] ethdev: support report " Jie Hai
2024-09-26 12:42 ` [PATCH v9 0/8] support dump reigster " Jie Hai
2024-09-26 12:42 4% ` [PATCH v9 1/8] ethdev: support report register " Jie Hai
2024-01-29 18:59 [PATCH 1/3] cryptodev: add ec points to sm2 op Arkadiusz Kusztal
2024-02-01 8:07 ` [EXT] " Akhil Goyal
2024-02-01 13:25 ` Kusztal, ArkadiuszX
2024-09-24 16:30 0% ` Akhil Goyal
2024-03-20 10:55 [PATCH 0/2] introduce PM QoS interface Huisong Li
2024-08-09 9:50 4% ` [PATCH v9 0/2] power: " Huisong Li
2024-08-09 9:50 5% ` [PATCH v9 1/2] power: introduce PM QoS API on CPU wide Huisong Li
2024-09-10 2:00 0% ` fengchengwen
2024-09-10 9:32 0% ` lihuisong (C)
2024-09-12 2:38 4% ` [PATCH v10 0/2] power: introduce PM QoS interface Huisong Li
2024-09-12 2:38 5% ` [PATCH v10 1/2] power: introduce PM QoS API on CPU wide Huisong Li
2024-09-12 3:07 0% ` [PATCH v10 0/2] power: introduce PM QoS interface fengchengwen
2024-04-04 21:04 [PATCH v1 0/3] Additional queue stats Nicolas Chautru
2024-04-04 21:04 ` [PATCH v1 1/3] bbdev: new queue stat for available enqueue depth Nicolas Chautru
2024-08-12 9:28 3% ` Maxime Coquelin
2024-08-12 9:56 0% ` Maxime Coquelin
2024-08-12 17:27 0% ` Chautru, Nicolas
2024-08-12 19:44 0% ` Maxime Coquelin
2024-05-29 23:33 [PATCH v10 00/20] Remove use of noninclusive term sanity Stephen Hemminger
2024-08-01 15:46 ` [PATCH v11 0/7] Remove usage of wording sanity check Stephen Hemminger
2024-08-01 15:46 2% ` [PATCH v11 1/7] mbuf: replace term " Stephen Hemminger
2024-07-15 22:11 [RFC v2] ethdev: an API for cache stashing hints Wathsala Vithanage
2024-07-22 11:18 ` Ferruh Yigit
2024-07-26 20:01 ` Wathsala Wathawana Vithanage
2024-09-22 21:43 ` Ferruh Yigit
2024-10-04 17:52 4% ` Stephen Hemminger
2024-10-04 18:46 0% ` Wathsala Wathawana Vithanage
2024-07-20 16:50 [PATCH v1 0/4] power: refactor power management library Sivaprasad Tummala
2024-08-26 13:06 ` [PATCH v2 " Sivaprasad Tummala
2024-08-26 13:06 ` [PATCH v2 2/4] power: refactor uncore " Sivaprasad Tummala
2024-08-27 13:02 3% ` lihuisong (C)
2024-10-08 6:19 0% ` Tummala, Sivaprasad
2024-07-22 14:53 [PATCH] doc: announce cryptodev change to support EDDSA Gowrishankar Muthukrishnan
2024-07-25 15:01 ` Kusztal, ArkadiuszX
2024-07-31 12:57 ` Thomas Monjalon
2024-08-07 17:21 0% ` [EXTERNAL] " Gowrishankar Muthukrishnan
2024-07-22 14:55 [PATCH] doc: announce cryptodev changes to offload RSA in VirtIO Gowrishankar Muthukrishnan
2024-07-30 14:39 ` Gowrishankar Muthukrishnan
2024-07-31 12:51 ` Thomas Monjalon
2024-07-31 14:26 0% ` Thomas Monjalon
2024-08-07 13:31 0% ` Kusztal, ArkadiuszX
2024-07-24 7:53 [PATCH v5 1/6] net/octeon_ep: add missing vector API header include Mattias Rönnblom
2024-09-20 10:27 ` [PATCH v6 0/7] Optionally have rte_memcpy delegate to compiler memcpy Mattias Rönnblom
2024-09-20 10:27 4% ` [PATCH v6 6/7] ci: test " Mattias Rönnblom
2024-07-30 12:53 [PATCH v3] doc: announce changes to dma device structures Amit Prakash Shukla
2024-07-30 17:27 ` [PATCH v4] " Jerin Jacob
2024-07-31 11:01 ` Thomas Monjalon
2024-07-31 16:06 3% ` Thomas Monjalon
2024-07-31 11:54 [PATCH 1/2] cryptodev: add queue pair priority Akhil Goyal
2024-09-24 16:37 ` [PATCH v2 0/2] " Akhil Goyal
2024-09-24 16:38 9% ` [PATCH v2 1/2] " Akhil Goyal
2024-07-31 19:22 3% DPDK 24.07 released Thomas Monjalon
2024-08-08 8:03 12% [PATCH] version: 24.11-rc0 David Marchand
2024-08-08 12:00 0% ` Thomas Monjalon
2024-08-08 12:55 3% ` David Marchand
2024-08-08 21:45 3% ` [OS-Team] [dpdklab] " Patrick Robb
2024-08-12 13:29 [RFC PATCH] config: make queues per port a meson config option Bruce Richardson
2024-08-13 15:59 ` [RFC PATCH v2 00/26] add meson config options for queues per port Bruce Richardson
2024-08-13 16:00 5% ` [RFC PATCH v2 26/26] config: add computed max queues define for compatibility Bruce Richardson
2024-08-14 10:49 ` [PATCH v3 00/26] add meson config options for queues per port Bruce Richardson
2024-08-14 10:49 5% ` [PATCH v3 26/26] config: add computed max queues define for compatibility Bruce Richardson
2024-08-12 23:41 3% [PATCH v2 0/3] bbdev: sdditional queue stats Nicolas Chautru
2024-08-12 23:41 6% ` [PATCH v2 1/3] bbdev: new queue stat for available enqueue depth Nicolas Chautru
2024-09-10 15:16 0% ` Maxime Coquelin
2024-08-13 7:22 0% ` [PATCH v2 0/3] bbdev: sdditional queue stats Hemant Agrawal
2024-09-13 13:50 0% ` Maxime Coquelin
2024-08-15 8:53 [RFC 0/6] Stage-Ordered API and other extensions for ring library Konstantin Ananyev
2024-08-15 8:53 2% ` [RFC 6/6] ring: minimize reads of the counterpart cache-line Konstantin Ananyev
2024-08-21 16:25 [PATCH dpdk v1 00/15] IPv6 APIs overhaul Robin Jarry
2024-10-01 8:17 ` [PATCH dpdk v2 00/16] " Robin Jarry
2024-10-01 8:17 2% ` [PATCH dpdk v2 02/16] net: split ipv6 symbols in separate header Robin Jarry
2024-10-01 8:17 ` [PATCH dpdk v2 03/16] net: add structure for ipv6 addresses Robin Jarry
2024-10-06 8:18 3% ` Morten Brørup
2024-10-01 8:17 1% ` [PATCH dpdk v2 04/16] net: use ipv6 structure for header addresses Robin Jarry
2024-10-01 8:17 1% ` [PATCH dpdk v2 05/16] fib6,rib6,lpm6: use ipv6 addr struct Robin Jarry
2024-10-01 8:17 2% ` [PATCH dpdk v2 08/16] graph,node: use ipv6 addr struct and utils Robin Jarry
2024-10-01 8:17 3% ` [PATCH dpdk v2 09/16] pipeline: use ipv6 addr struct Robin Jarry
2024-10-01 8:17 3% ` [PATCH dpdk v2 10/16] ipsec: " Robin Jarry
2024-10-01 8:17 3% ` [PATCH dpdk v2 11/16] thash: " Robin Jarry
2024-10-01 8:17 4% ` [PATCH dpdk v2 12/16] gro: " Robin Jarry
2024-10-01 8:17 3% ` [PATCH dpdk v2 13/16] rte_flow: " Robin Jarry
2024-10-06 9:04 3% ` [PATCH dpdk v2 00/16] IPv6 APIs overhaul Morten Brørup
2024-08-22 10:36 [PATCH] app/testpmd: show output of commands read from file Bruce Richardson
2024-08-22 10:41 ` [PATCH v2] " Bruce Richardson
2024-08-22 16:53 ` Ferruh Yigit
2024-08-22 17:14 3% ` Bruce Richardson
2024-08-22 17:18 0% ` Bruce Richardson
2024-08-22 21:09 3% ` Ferruh Yigit
2024-08-23 9:12 0% ` Bruce Richardson
2024-08-27 15:10 [RFC 0/2] introduce LLC aware functions Vipin Varghese
2024-08-27 15:10 ` [RFC 1/2] eal: add llc " Vipin Varghese
2024-08-27 20:56 ` Wathsala Wathawana Vithanage
2024-09-02 1:20 4% ` Varghese, Vipin
2024-08-27 23:03 3% [PATCH v2 0/1] bbdev: removing unnecessaray symbols Nicolas Chautru
2024-08-27 23:03 3% ` [PATCH v2 1/1] bbdev: removing unnecessaray symbols from version map Nicolas Chautru
2024-09-10 16:20 0% ` Maxime Coquelin
2024-08-27 23:06 3% [PATCH v3 0/1] bbdev: removing unnecessaray symbols Nicolas Chautru
2024-08-27 23:06 3% ` [PATCH v3 1/1] bbdev: removing unnecessary symbols from version map Nicolas Chautru
2024-09-13 13:51 0% ` Maxime Coquelin
2024-09-19 8:26 0% ` Maxime Coquelin
2024-09-03 1:41 [PATCH 0/4] Support new card using NFP 3800 chip Chaoyong He
2024-09-03 1:41 2% ` [PATCH 2/4] net/nfp: refactor the firmware version logic Chaoyong He
2024-09-03 1:41 2% ` [PATCH 3/4] net/nfp: support different configuration BAR size Chaoyong He
2024-09-04 13:57 [PATCH] config: add 32-bit cross-compilation x86 target Bruce Richardson
2024-09-06 16:12 ` [PATCH v2 0/4] simplify doing 32-bit DPDK builds Bruce Richardson
2024-09-06 16:12 13% ` [PATCH v2 4/4] devtools/test-meson-builds: use cross files for 32bit build Bruce Richardson
2024-09-04 14:03 13% [RFC PATCH] devtools/test-meson-builds: use cross file " Bruce Richardson
2024-09-04 18:08 [PATCH 0/3] eal: mark API's as stable Stephen Hemminger
2024-09-05 7:58 ` David Marchand
2024-09-05 8:55 ` Morten Brørup
2024-09-05 9:03 ` David Marchand
2024-09-05 9:44 ` Morten Brørup
2024-09-05 14:01 3% ` Jerin Jacob
2024-09-05 14:18 0% ` Morten Brørup
2024-09-08 23:58 3% ` Stephen Hemminger
2024-09-06 8:54 0% ` Ferruh Yigit
2024-09-06 10:04 0% ` Morten Brørup
2024-09-06 14:12 0% ` Ferruh Yigit
2024-09-06 14:42 0% ` Morten Brørup
2024-09-09 4:48 0% ` Jerin Jacob
2024-09-05 10:14 4% [PATCH] [RFC] cryptodev: replace LIST_END enumerators with APIs Akhil Goyal
2024-09-05 15:09 4% ` Morten Brørup
2024-09-05 15:26 0% ` Kusztal, ArkadiuszX
2024-09-06 6:32 4% ` fengchengwen
2024-09-06 7:45 0% ` [EXTERNAL] " Akhil Goyal
2024-10-04 3:56 0% ` Ferruh Yigit
2024-09-06 7:54 0% ` Morten Brørup
2024-09-23 20:41 4% ` Akhil Goyal
2024-10-03 7:00 0% ` Akhil Goyal
2024-10-06 11:10 0% ` Morten Brørup
2024-10-09 11:21 0% ` Akhil Goyal
2024-10-04 4:00 0% ` Ferruh Yigit
2024-10-04 17:26 4% ` [EXTERNAL] " Akhil Goyal
2024-10-04 3:54 5% ` Ferruh Yigit
2024-10-04 7:04 0% ` David Marchand
2024-10-04 17:27 0% ` [EXTERNAL] " Akhil Goyal
2024-10-04 9:38 5% ` Dodji Seketeli
2024-10-04 17:45 4% ` [EXTERNAL] " Akhil Goyal
2024-10-09 11:24 3% ` [PATCH] cryptodev: remove unnecessary list end Akhil Goyal
2024-10-09 12:52 0% ` Morten Brørup
2024-10-09 20:38 0% ` Akhil Goyal
2024-10-09 14:06 0% ` Hemant Agrawal
2024-09-05 16:20 8% [PATCH] eal: increase max file descriptor for secondary process device Stephen Hemminger
2024-09-06 1:19 0% ` fengchengwen
2024-09-06 6:23 0% ` Morten Brørup
2024-09-06 7:55 3% ` Ferruh Yigit
2024-09-06 11:47 3% [RFC PATCH v1 0/5] Adjust wording for NUMA vs. socket ID in DPDK Anatoly Burakov
2024-09-06 12:37 0% ` Morten Brørup
2024-09-06 12:46 0% ` Burakov, Anatoly
2024-09-06 13:02 3% ` Morten Brørup
2024-09-06 13:07 3% ` Bruce Richardson
2024-09-06 13:17 0% ` Burakov, Anatoly
2024-09-06 13:58 3% ` Morten Brørup
2024-09-09 7:51 0% ` fengchengwen
2024-09-08 11:57 [PATCH v2] ipsec: allow stateless IPsec processing Aakash Sasidharan
2024-10-03 13:45 3% ` [PATCH v3 1/2] " Aakash Sasidharan
2024-10-04 6:34 3% ` [PATCH v4 " Aakash Sasidharan
2024-09-10 7:03 [PATCH 1/6] eal: add static per-lcore memory allocation facility Mattias Rönnblom
2024-09-11 17:04 ` [PATCH v2 0/6] Lcore variables Mattias Rönnblom
2024-09-11 17:04 ` [PATCH v2 1/6] eal: add static per-lcore memory allocation facility Mattias Rönnblom
2024-09-12 9:10 3% ` Morten Brørup
2024-09-13 12:10 11% [RFC v0 1/1] dmadev: provide priority configuration support Vamsi Krishna
2024-09-14 9:41 0% ` fengchengwen
2024-09-16 16:34 0% ` [EXTERNAL] " Vamsi Krishna Attunuru
2024-09-18 9:40 10% ` [RFC v1 1/1] dmadev: support priority configuration Vamsi Krishna
2024-09-18 10:13 4% ` [RFC v2 " Vamsi Krishna
2024-10-03 11:41 4% ` [PATCH v3 " Vamsi Krishna
2024-10-03 11:53 4% ` [PATCH v4 " Vamsi Krishna
2024-10-08 2:36 0% ` fengchengwen
2024-09-13 17:58 [PATCH] ethdev: optimize activation of fast-path tracepoints Adel Belkhiri
2024-09-16 8:25 4% ` Jerin Jacob
2024-09-16 8:31 3% ` David Marchand
2024-09-16 8:45 0% ` Jerin Jacob
2024-09-18 8:55 3% [PATCH] eal: add build-time option to omit trace Morten Brørup
2024-09-18 9:49 0% ` Jerin Jacob
2024-09-18 10:23 0% ` Morten Brørup
2024-09-19 8:06 3% ` [PATCH v2] " Morten Brørup
2024-09-19 8:51 0% ` Jerin Jacob
2024-09-19 13:15 0% ` Morten Brørup
2024-09-19 13:54 0% ` Jerin Jacob
2024-09-20 9:08 3% ` [PATCH v3] " Morten Brørup
2024-09-23 8:39 0% ` Jerin Jacob
2024-09-24 13:39 3% ` [PATCH v4] " Morten Brørup
2024-09-24 15:30 0% ` Stephen Hemminger
2024-09-24 15:41 0% ` Jerin Jacob
2024-09-24 15:53 0% ` Morten Brørup
2024-09-24 15:57 0% ` Jerin Jacob
2024-10-06 12:38 2% ` [PATCH v5] " Morten Brørup
2024-10-06 13:58 2% ` [PATCH v6] " Morten Brørup
2024-10-07 5:45 0% ` Jerin Jacob
2024-10-07 6:07 0% ` Morten Brørup
2024-10-06 14:03 2% ` [PATCH v7] " Morten Brørup
2024-10-06 14:09 0% ` Morten Brørup
2024-10-07 11:46 3% ` [PATCH v9] " Morten Brørup
2024-10-08 7:16 0% ` Morten Brørup
2024-10-08 10:15 0% ` David Marchand
2024-09-18 9:21 [RFC 0/4] ethdev: rework config restore Dariusz Sosnowski
2024-09-29 23:31 ` Ferruh Yigit
2024-10-04 19:13 ` Dariusz Sosnowski
2024-10-07 9:27 ` Konstantin Ananyev
2024-10-07 22:56 ` Ferruh Yigit
2024-10-08 17:21 ` Konstantin Ananyev
2024-10-09 1:07 ` Ferruh Yigit
2024-10-09 16:18 3% ` Dariusz Sosnowski
2024-10-09 23:16 0% ` Ferruh Yigit
[not found] <20220825024425.10534-1-lihuisong@huawei.com>
2024-09-29 5:52 3% ` [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port Huisong Li
2024-09-29 5:52 2% ` [PATCH RESEND v7 2/5] ethdev: fix skip valid port in probing callback Huisong Li
2024-10-08 2:32 0% ` [PATCH RESEND v7 0/5] app/testpmd: support multiple process attach and detach port lihuisong (C)
2024-09-30 9:27 4% [PATCH dpdk v7] graph: expose node context as pointers Robin Jarry
2024-10-01 6:14 [PATCH v3 0/6] Introduce event pre-scheduling pbhagavatula
2024-10-01 13:18 ` [PATCH v4 " pbhagavatula
2024-10-01 13:18 ` [PATCH v4 1/6] eventdev: introduce " pbhagavatula
2024-10-04 4:47 3% ` Jerin Jacob
2024-10-01 13:18 ` [PATCH v4 2/6] eventdev: add event port pre-schedule modify pbhagavatula
2024-10-04 5:02 3% ` Jerin Jacob
2024-10-01 13:18 ` [PATCH v4 3/6] eventdev: add SW event preschedule hint pbhagavatula
2024-10-04 5:14 3% ` Jerin Jacob
2024-10-04 16:24 3% ` [PATCH v5 0/6] Introduce event pre-scheduling pbhagavatula
2024-10-04 16:24 3% ` [PATCH v5 1/6] eventdev: introduce " pbhagavatula
2024-10-04 16:35 0% ` [PATCH v5 0/6] Introduce " Stephen Hemminger
2024-10-04 16:50 0% ` [EXTERNAL] " Pavan Nikhilesh Bhagavatula
2024-10-05 7:25 3% ` [PATCH v6 " pbhagavatula
2024-10-05 7:25 3% ` [PATCH v6 1/6] eventdev: introduce " pbhagavatula
2024-10-05 7:59 3% ` [PATCH v7 0/6] Introduce " pbhagavatula
2024-10-05 7:59 3% ` [PATCH v7 1/6] eventdev: introduce " pbhagavatula
2024-10-06 17:06 3% ` [PATCH v8 0/6] Introduce " pbhagavatula
2024-10-06 17:06 3% ` [PATCH v8 1/6] eventdev: introduce " pbhagavatula
2024-10-07 13:09 3% ` [PATCH v9 0/6] Introduce " pbhagavatula
2024-10-07 13:09 3% ` [PATCH v9 1/6] eventdev: introduce " pbhagavatula
2024-10-08 5:17 0% ` [PATCH v9 0/6] Introduce " Jerin Jacob
2024-10-01 18:11 [PATCH v2 0/3] net: add thread-safe crc api Arkadiusz Kusztal
2024-10-01 18:11 ` [PATCH v2 1/3] " Arkadiusz Kusztal
2024-10-01 21:44 3% ` Stephen Hemminger
2024-10-02 8:28 0% ` Kusztal, ArkadiuszX
2024-10-08 3:42 ` Ferruh Yigit
2024-10-08 20:51 ` Kusztal, ArkadiuszX
2024-10-09 1:03 4% ` Ferruh Yigit
2024-10-09 7:48 0% ` Kusztal, ArkadiuszX
2024-10-09 9:11 0% ` Ferruh Yigit
2024-10-02 15:57 [PATCH 0/2] Fix race in ethdev telemetry David Marchand
2024-10-02 15:57 ` [PATCH 2/2] ethdev: fix race on ports for telemetry commands David Marchand
2024-10-02 16:27 ` Bruce Richardson
2024-10-02 19:06 ` David Marchand
2024-10-02 19:09 ` Robin Jarry
2024-10-02 19:18 ` David Marchand
2024-10-02 19:26 ` Robin Jarry
2024-10-03 9:46 3% ` Bruce Richardson
2024-10-03 9:58 0% ` David Marchand
2024-10-06 12:32 3% [PATCH] net/ena: restructure the llq policy user setting shaibran
2024-10-08 13:30 [RFC PATCH v2 0/5] add feature arc in rte_graph Nitin Saxena
2024-10-09 13:29 3% ` [PATCH v3 " Nitin Saxena
2024-10-09 13:29 4% ` [PATCH v3 2/5] graph: add feature arc option in graph create Nitin Saxena
2024-10-09 14:21 0% ` [PATCH v3 0/5] add feature arc in rte_graph Christophe Fontaine
2024-10-09 17:37 0% ` Stephen Hemminger
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).