* Re: [dpdk-dev] [PATCH v3 3/7] ethdev: copy ethdev 'fast' API into separate structure
@ 2021-10-02 0:14 Pavan Nikhilesh Bhagavatula
2021-10-03 20:58 ` Ananyev, Konstantin
0 siblings, 1 reply; 4+ messages in thread
From: Pavan Nikhilesh Bhagavatula @ 2021-10-02 0:14 UTC (permalink / raw)
To: Konstantin Ananyev, dev
Cc: xiaoyun.li, Anoob Joseph, Jerin Jacob Kollanukkaran,
Nithin Kumar Dabilpuram, Ankur Dwivedi, shepard.siegel, ed.czeck,
john.miller, Igor Russkikh, ajit.khaparde, somnath.kotur,
rahul.lakkireddy, hemant.agrawal, sachin.saxena, haiyue.wang,
johndale, hyonkim, qi.z.zhang, xiao.w.wang, humin29,
yisen.zhuang, oulijun, beilei.xing, jingjing.wu, qiming.yang,
matan, viacheslavo, sthemmin, longli, heinrich.kuhn,
Kiran Kumar Kokkilagadda, andrew.rybchenko, Maciej Czekaj [C],
jiawenwu, jianwang, maxime.coquelin, chenbo.xia, thomas,
ferruh.yigit, mdr, jay.jayatheerthan
>Copy public function pointers (rx_pkt_burst(), etc.) and related
>pointers to internal data from rte_eth_dev structure into a
>separate flat array. That array will remain in a public header.
>The intention here is to make rte_eth_dev and related structures
>internal.
>That should allow future possible changes to core eth_dev structures
>to be transparent to the user and help to avoid ABI/API breakages.
>The plan is to keep minimal part of data from rte_eth_dev public,
>so we still can use inline functions for 'fast' calls
>(like rte_eth_rx_burst(), etc.) to avoid/minimize slowdown.
>
>Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
>---
> lib/ethdev/ethdev_private.c | 52
>++++++++++++++++++++++++++++++++++++
> lib/ethdev/ethdev_private.h | 7 +++++
> lib/ethdev/rte_ethdev.c | 17 ++++++++++++
> lib/ethdev/rte_ethdev_core.h | 45
>+++++++++++++++++++++++++++++++
> 4 files changed, 121 insertions(+)
>
<snip>
>+void
>+eth_dev_fp_ops_reset(struct rte_eth_fp_ops *fpo)
>+{
>+ static void *dummy_data[RTE_MAX_QUEUES_PER_PORT];
>+ static const struct rte_eth_fp_ops dummy_ops = {
>+ .rx_pkt_burst = dummy_eth_rx_burst,
>+ .tx_pkt_burst = dummy_eth_tx_burst,
>+ .rxq = {.data = dummy_data, .clbk = dummy_data,},
>+ .txq = {.data = dummy_data, .clbk = dummy_data,},
>+ };
>+
>+ *fpo = dummy_ops;
>+}
>+
>+void
>+eth_dev_fp_ops_setup(struct rte_eth_fp_ops *fpo,
>+ const struct rte_eth_dev *dev)
>+{
>+ fpo->rx_pkt_burst = dev->rx_pkt_burst;
>+ fpo->tx_pkt_burst = dev->tx_pkt_burst;
>+ fpo->tx_pkt_prepare = dev->tx_pkt_prepare;
>+ fpo->rx_queue_count = dev->rx_queue_count;
>+ fpo->rx_descriptor_status = dev->rx_descriptor_status;
>+ fpo->tx_descriptor_status = dev->tx_descriptor_status;
>+
>+ fpo->rxq.data = dev->data->rx_queues;
>+ fpo->rxq.clbk = (void **)(uintptr_t)dev->post_rx_burst_cbs;
>+
>+ fpo->txq.data = dev->data->tx_queues;
>+ fpo->txq.clbk = (void **)(uintptr_t)dev->pre_tx_burst_cbs;
>+}
>diff --git a/lib/ethdev/ethdev_private.h b/lib/ethdev/ethdev_private.h
>index 3724429577..40333e7651 100644
>--- a/lib/ethdev/ethdev_private.h
>+++ b/lib/ethdev/ethdev_private.h
>@@ -26,4 +26,11 @@ eth_find_device(const struct rte_eth_dev
>*_start, rte_eth_cmp_t cmp,
> /* Parse devargs value for representor parameter. */
> int rte_eth_devargs_parse_representor_ports(char *str, void *data);
>
>+/* reset eth 'fast' API to dummy values */
>+void eth_dev_fp_ops_reset(struct rte_eth_fp_ops *fpo);
>+
>+/* setup eth 'fast' API to ethdev values */
>+void eth_dev_fp_ops_setup(struct rte_eth_fp_ops *fpo,
>+ const struct rte_eth_dev *dev);
>+
> #endif /* _ETH_PRIVATE_H_ */
>diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
>index 424bc260fa..9fbb1bc3db 100644
>--- a/lib/ethdev/rte_ethdev.c
>+++ b/lib/ethdev/rte_ethdev.c
>@@ -44,6 +44,9 @@
> static const char *MZ_RTE_ETH_DEV_DATA = "rte_eth_dev_data";
> struct rte_eth_dev rte_eth_devices[RTE_MAX_ETHPORTS];
>
>+/* public 'fast' API */
>+struct rte_eth_fp_ops rte_eth_fp_ops[RTE_MAX_ETHPORTS];
>+
> /* spinlock for eth device callbacks */
> static rte_spinlock_t eth_dev_cb_lock = RTE_SPINLOCK_INITIALIZER;
>
>@@ -1788,6 +1791,9 @@ rte_eth_dev_start(uint16_t port_id)
> (*dev->dev_ops->link_update)(dev, 0);
> }
>
>+ /* expose selection of PMD rx/tx function */
>+ eth_dev_fp_ops_setup(rte_eth_fp_ops + port_id, dev);
>+
Secondary process will not set these properly I believe as it might not
call start() if it does primary process ops will not be set.
One simple solution is to call ops_setup() around rte_eth_dev_attach_secondary()
but if application doesn't invoke start() on Primary the ops will not be set for it.
> rte_ethdev_trace_start(port_id);
> return 0;
> }
>@@ -1810,6 +1816,9 @@ rte_eth_dev_stop(uint16_t port_id)
> return 0;
> }
>
>+ /* point rx/tx functions to dummy ones */
>+ eth_dev_fp_ops_reset(rte_eth_fp_ops + port_id);
>+
> dev->data->dev_started = 0;
> ret = (*dev->dev_ops->dev_stop)(dev);
> rte_ethdev_trace_stop(port_id, ret);
>2.26.3
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [dpdk-dev] [PATCH v3 3/7] ethdev: copy ethdev 'fast' API into separate structure
2021-10-02 0:14 [dpdk-dev] [PATCH v3 3/7] ethdev: copy ethdev 'fast' API into separate structure Pavan Nikhilesh Bhagavatula
@ 2021-10-03 20:58 ` Ananyev, Konstantin
2021-10-03 21:10 ` Pavan Nikhilesh Bhagavatula
0 siblings, 1 reply; 4+ messages in thread
From: Ananyev, Konstantin @ 2021-10-03 20:58 UTC (permalink / raw)
To: Pavan Nikhilesh Bhagavatula, dev
Cc: Li, Xiaoyun, Anoob Joseph, Jerin Jacob Kollanukkaran,
Nithin Kumar Dabilpuram, Ankur Dwivedi, shepard.siegel, ed.czeck,
john.miller, Igor Russkikh, ajit.khaparde, somnath.kotur,
rahul.lakkireddy, hemant.agrawal, sachin.saxena, Wang, Haiyue,
Daley, John, hyonkim, Zhang, Qi Z, Wang, Xiao W, humin29,
yisen.zhuang, oulijun, Xing, Beilei, Wu, Jingjing, Yang, Qiming,
matan, viacheslavo, sthemmin, longli, heinrich.kuhn,
Kiran Kumar Kokkilagadda, andrew.rybchenko, Maciej Czekaj [C],
jiawenwu, jianwang, maxime.coquelin, Xia, Chenbo, thomas, Yigit,
Ferruh, mdr, Jayatheerthan, Jay
> >Copy public function pointers (rx_pkt_burst(), etc.) and related
> >pointers to internal data from rte_eth_dev structure into a
> >separate flat array. That array will remain in a public header.
> >The intention here is to make rte_eth_dev and related structures
> >internal.
> >That should allow future possible changes to core eth_dev structures
> >to be transparent to the user and help to avoid ABI/API breakages.
> >The plan is to keep minimal part of data from rte_eth_dev public,
> >so we still can use inline functions for 'fast' calls
> >(like rte_eth_rx_burst(), etc.) to avoid/minimize slowdown.
> >
> >Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
> >---
> > lib/ethdev/ethdev_private.c | 52
> >++++++++++++++++++++++++++++++++++++
> > lib/ethdev/ethdev_private.h | 7 +++++
> > lib/ethdev/rte_ethdev.c | 17 ++++++++++++
> > lib/ethdev/rte_ethdev_core.h | 45
> >+++++++++++++++++++++++++++++++
> > 4 files changed, 121 insertions(+)
> >
>
> <snip>
>
> >+void
> >+eth_dev_fp_ops_reset(struct rte_eth_fp_ops *fpo)
> >+{
> >+ static void *dummy_data[RTE_MAX_QUEUES_PER_PORT];
> >+ static const struct rte_eth_fp_ops dummy_ops = {
> >+ .rx_pkt_burst = dummy_eth_rx_burst,
> >+ .tx_pkt_burst = dummy_eth_tx_burst,
> >+ .rxq = {.data = dummy_data, .clbk = dummy_data,},
> >+ .txq = {.data = dummy_data, .clbk = dummy_data,},
> >+ };
> >+
> >+ *fpo = dummy_ops;
> >+}
> >+
> >+void
> >+eth_dev_fp_ops_setup(struct rte_eth_fp_ops *fpo,
> >+ const struct rte_eth_dev *dev)
> >+{
> >+ fpo->rx_pkt_burst = dev->rx_pkt_burst;
> >+ fpo->tx_pkt_burst = dev->tx_pkt_burst;
> >+ fpo->tx_pkt_prepare = dev->tx_pkt_prepare;
> >+ fpo->rx_queue_count = dev->rx_queue_count;
> >+ fpo->rx_descriptor_status = dev->rx_descriptor_status;
> >+ fpo->tx_descriptor_status = dev->tx_descriptor_status;
> >+
> >+ fpo->rxq.data = dev->data->rx_queues;
> >+ fpo->rxq.clbk = (void **)(uintptr_t)dev->post_rx_burst_cbs;
> >+
> >+ fpo->txq.data = dev->data->tx_queues;
> >+ fpo->txq.clbk = (void **)(uintptr_t)dev->pre_tx_burst_cbs;
> >+}
> >diff --git a/lib/ethdev/ethdev_private.h b/lib/ethdev/ethdev_private.h
> >index 3724429577..40333e7651 100644
> >--- a/lib/ethdev/ethdev_private.h
> >+++ b/lib/ethdev/ethdev_private.h
> >@@ -26,4 +26,11 @@ eth_find_device(const struct rte_eth_dev
> >*_start, rte_eth_cmp_t cmp,
> > /* Parse devargs value for representor parameter. */
> > int rte_eth_devargs_parse_representor_ports(char *str, void *data);
> >
> >+/* reset eth 'fast' API to dummy values */
> >+void eth_dev_fp_ops_reset(struct rte_eth_fp_ops *fpo);
> >+
> >+/* setup eth 'fast' API to ethdev values */
> >+void eth_dev_fp_ops_setup(struct rte_eth_fp_ops *fpo,
> >+ const struct rte_eth_dev *dev);
> >+
> > #endif /* _ETH_PRIVATE_H_ */
> >diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
> >index 424bc260fa..9fbb1bc3db 100644
> >--- a/lib/ethdev/rte_ethdev.c
> >+++ b/lib/ethdev/rte_ethdev.c
> >@@ -44,6 +44,9 @@
> > static const char *MZ_RTE_ETH_DEV_DATA = "rte_eth_dev_data";
> > struct rte_eth_dev rte_eth_devices[RTE_MAX_ETHPORTS];
> >
> >+/* public 'fast' API */
> >+struct rte_eth_fp_ops rte_eth_fp_ops[RTE_MAX_ETHPORTS];
> >+
> > /* spinlock for eth device callbacks */
> > static rte_spinlock_t eth_dev_cb_lock = RTE_SPINLOCK_INITIALIZER;
> >
> >@@ -1788,6 +1791,9 @@ rte_eth_dev_start(uint16_t port_id)
> > (*dev->dev_ops->link_update)(dev, 0);
> > }
> >
> >+ /* expose selection of PMD rx/tx function */
> >+ eth_dev_fp_ops_setup(rte_eth_fp_ops + port_id, dev);
> >+
>
> Secondary process will not set these properly I believe as it might not
> call start() if it does primary process ops will not be set.
That's a very good point, have to admit - I missed that part.
>
> One simple solution is to call ops_setup() around rte_eth_dev_attach_secondary()
> but if application doesn't invoke start() on Primary the ops will not be set for it.
I think rte_eth_dev_attach_secondary() wouldn't work, as majority of the PMDs setup
fast ops function pointers after it.
From reading the code rte_eth_dev_probing_finish() seems like a good choice -
as it is always the final point in device initialization for secondary process.
BTW, we also need something similar at de-init phase.
rte_eth_dev_release_port() seems like a good candidate for it.
>
> > rte_ethdev_trace_start(port_id);
> > return 0;
> > }
> >@@ -1810,6 +1816,9 @@ rte_eth_dev_stop(uint16_t port_id)
> > return 0;
> > }
> >
> >+ /* point rx/tx functions to dummy ones */
> >+ eth_dev_fp_ops_reset(rte_eth_fp_ops + port_id);
> >+
> > dev->data->dev_started = 0;
> > ret = (*dev->dev_ops->dev_stop)(dev);
> > rte_ethdev_trace_stop(port_id, ret);
> >2.26.3
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [dpdk-dev] [PATCH v3 3/7] ethdev: copy ethdev 'fast' API into separate structure
2021-10-03 20:58 ` Ananyev, Konstantin
@ 2021-10-03 21:10 ` Pavan Nikhilesh Bhagavatula
0 siblings, 0 replies; 4+ messages in thread
From: Pavan Nikhilesh Bhagavatula @ 2021-10-03 21:10 UTC (permalink / raw)
To: Ananyev, Konstantin, dev
Cc: Li, Xiaoyun, Anoob Joseph, Jerin Jacob Kollanukkaran,
Nithin Kumar Dabilpuram, Ankur Dwivedi, shepard.siegel, ed.czeck,
john.miller, Igor Russkikh, ajit.khaparde, somnath.kotur,
rahul.lakkireddy, hemant.agrawal, sachin.saxena, Wang, Haiyue,
Daley, John, hyonkim, Zhang, Qi Z, Wang, Xiao W, humin29,
yisen.zhuang, oulijun, Xing, Beilei, Wu, Jingjing, Yang, Qiming,
matan, viacheslavo, sthemmin, longli, heinrich.kuhn,
Kiran Kumar Kokkilagadda, andrew.rybchenko, Maciej Czekaj [C],
jiawenwu, jianwang, maxime.coquelin, Xia, Chenbo, thomas, Yigit,
Ferruh, mdr, Jayatheerthan, Jay
>
>> >Copy public function pointers (rx_pkt_burst(), etc.) and related
>> >pointers to internal data from rte_eth_dev structure into a
>> >separate flat array. That array will remain in a public header.
>> >The intention here is to make rte_eth_dev and related structures
>> >internal.
>> >That should allow future possible changes to core eth_dev
>structures
>> >to be transparent to the user and help to avoid ABI/API breakages.
>> >The plan is to keep minimal part of data from rte_eth_dev public,
>> >so we still can use inline functions for 'fast' calls
>> >(like rte_eth_rx_burst(), etc.) to avoid/minimize slowdown.
>> >
>> >Signed-off-by: Konstantin Ananyev
><konstantin.ananyev@intel.com>
>> >---
>> > lib/ethdev/ethdev_private.c | 52
>> >++++++++++++++++++++++++++++++++++++
>> > lib/ethdev/ethdev_private.h | 7 +++++
>> > lib/ethdev/rte_ethdev.c | 17 ++++++++++++
>> > lib/ethdev/rte_ethdev_core.h | 45
>> >+++++++++++++++++++++++++++++++
>> > 4 files changed, 121 insertions(+)
>> >
>>
>> <snip>
>>
>> >+void
>> >+eth_dev_fp_ops_reset(struct rte_eth_fp_ops *fpo)
>> >+{
>> >+ static void *dummy_data[RTE_MAX_QUEUES_PER_PORT];
>> >+ static const struct rte_eth_fp_ops dummy_ops = {
>> >+ .rx_pkt_burst = dummy_eth_rx_burst,
>> >+ .tx_pkt_burst = dummy_eth_tx_burst,
>> >+ .rxq = {.data = dummy_data, .clbk = dummy_data,},
>> >+ .txq = {.data = dummy_data, .clbk = dummy_data,},
>> >+ };
>> >+
>> >+ *fpo = dummy_ops;
>> >+}
>> >+
>> >+void
>> >+eth_dev_fp_ops_setup(struct rte_eth_fp_ops *fpo,
>> >+ const struct rte_eth_dev *dev)
>> >+{
>> >+ fpo->rx_pkt_burst = dev->rx_pkt_burst;
>> >+ fpo->tx_pkt_burst = dev->tx_pkt_burst;
>> >+ fpo->tx_pkt_prepare = dev->tx_pkt_prepare;
>> >+ fpo->rx_queue_count = dev->rx_queue_count;
>> >+ fpo->rx_descriptor_status = dev->rx_descriptor_status;
>> >+ fpo->tx_descriptor_status = dev->tx_descriptor_status;
>> >+
>> >+ fpo->rxq.data = dev->data->rx_queues;
>> >+ fpo->rxq.clbk = (void **)(uintptr_t)dev->post_rx_burst_cbs;
>> >+
>> >+ fpo->txq.data = dev->data->tx_queues;
>> >+ fpo->txq.clbk = (void **)(uintptr_t)dev->pre_tx_burst_cbs;
>> >+}
>> >diff --git a/lib/ethdev/ethdev_private.h
>b/lib/ethdev/ethdev_private.h
>> >index 3724429577..40333e7651 100644
>> >--- a/lib/ethdev/ethdev_private.h
>> >+++ b/lib/ethdev/ethdev_private.h
>> >@@ -26,4 +26,11 @@ eth_find_device(const struct rte_eth_dev
>> >*_start, rte_eth_cmp_t cmp,
>> > /* Parse devargs value for representor parameter. */
>> > int rte_eth_devargs_parse_representor_ports(char *str, void
>*data);
>> >
>> >+/* reset eth 'fast' API to dummy values */
>> >+void eth_dev_fp_ops_reset(struct rte_eth_fp_ops *fpo);
>> >+
>> >+/* setup eth 'fast' API to ethdev values */
>> >+void eth_dev_fp_ops_setup(struct rte_eth_fp_ops *fpo,
>> >+ const struct rte_eth_dev *dev);
>> >+
>> > #endif /* _ETH_PRIVATE_H_ */
>> >diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
>> >index 424bc260fa..9fbb1bc3db 100644
>> >--- a/lib/ethdev/rte_ethdev.c
>> >+++ b/lib/ethdev/rte_ethdev.c
>> >@@ -44,6 +44,9 @@
>> > static const char *MZ_RTE_ETH_DEV_DATA = "rte_eth_dev_data";
>> > struct rte_eth_dev rte_eth_devices[RTE_MAX_ETHPORTS];
>> >
>> >+/* public 'fast' API */
>> >+struct rte_eth_fp_ops rte_eth_fp_ops[RTE_MAX_ETHPORTS];
>> >+
>> > /* spinlock for eth device callbacks */
>> > static rte_spinlock_t eth_dev_cb_lock =
>RTE_SPINLOCK_INITIALIZER;
>> >
>> >@@ -1788,6 +1791,9 @@ rte_eth_dev_start(uint16_t port_id)
>> > (*dev->dev_ops->link_update)(dev, 0);
>> > }
>> >
>> >+ /* expose selection of PMD rx/tx function */
>> >+ eth_dev_fp_ops_setup(rte_eth_fp_ops + port_id, dev);
>> >+
>>
>> Secondary process will not set these properly I believe as it might not
>> call start() if it does primary process ops will not be set.
>
>That's a very good point, have to admit - I missed that part.
>
>>
>> One simple solution is to call ops_setup() around
>rte_eth_dev_attach_secondary()
>> but if application doesn't invoke start() on Primary the ops will not be
>set for it.
>
>I think rte_eth_dev_attach_secondary() wouldn't work, as majority of
>the PMDs setup
>fast ops function pointers after it.
>From reading the code rte_eth_dev_probing_finish() seems like a good
>choice -
>as it is always the final point in device initialization for secondary
>process.
Ack, make sense to me, I did a similar thing for event device in
http://patches.dpdk.org/project/dpdk/patch/20211003082710.8398-4-pbhagavatula@marvell.com/
>
>BTW, we also need something similar at de-init phase.
>rte_eth_dev_release_port() seems like a good candidate for it.
>
Hindsight I should have added reset to rte_event_pmd_pci_remove(), I will add it in next version.
>
>>
>> > rte_ethdev_trace_start(port_id);
>> > return 0;
>> > }
>> >@@ -1810,6 +1816,9 @@ rte_eth_dev_stop(uint16_t port_id)
>> > return 0;
>> > }
>> >
>> >+ /* point rx/tx functions to dummy ones */
>> >+ eth_dev_fp_ops_reset(rte_eth_fp_ops + port_id);
>> >+
>> > dev->data->dev_started = 0;
>> > ret = (*dev->dev_ops->dev_stop)(dev);
>> > rte_ethdev_trace_stop(port_id, ret);
>> >2.26.3
^ permalink raw reply [flat|nested] 4+ messages in thread
* [dpdk-dev] [RFC v2 0/5] hide eth dev related structures
@ 2021-09-22 14:09 Konstantin Ananyev
2021-10-01 14:02 ` [dpdk-dev] [PATCH v3 0/7] " Konstantin Ananyev
0 siblings, 1 reply; 4+ messages in thread
From: Konstantin Ananyev @ 2021-09-22 14:09 UTC (permalink / raw)
To: dev
Cc: xiaoyun.li, anoobj, jerinj, ndabilpuram, adwivedi,
shepard.siegel, ed.czeck, john.miller, irusskikh, ajit.khaparde,
somnath.kotur, rahul.lakkireddy, hemant.agrawal, sachin.saxena,
haiyue.wang, johndale, hyonkim, qi.z.zhang, xiao.w.wang, humin29,
yisen.zhuang, oulijun, beilei.xing, jingjing.wu, qiming.yang,
matan, viacheslavo, sthemmin, longli, heinrich.kuhn, kirankumark,
andrew.rybchenko, mczekaj, jiawenwu, jianwang, maxime.coquelin,
chenbo.xia, thomas, ferruh.yigit, mdr, jay.jayatheerthan,
Konstantin Ananyev
The aim of these patch series is to make rte_ethdev core data structures
(rte_eth_dev, rte_eth_dev_data, rte_eth_rxtx_callback, etc.) internal to
DPDK and not visible to the user.
That should allow future possible changes to core ethdev related structures
to be transparent to the user and help to improve ABI/API stability.
Note that current ethdev API is preserved, but it is a formal ABI break.
The work is based on previous discussions at:
https://www.mail-archive.com/dev@dpdk.org/msg211405.html
https://www.mail-archive.com/dev@dpdk.org/msg216685.html
and consists of the following main points:
1. Copy public 'fast' function pointers (rx_pkt_burst(), etc.) and
related data pointer from rte_eth_dev into a separate flat array.
We keep it public to still be able to use inline functions for these
'fast' calls (like rte_eth_rx_burst(), etc.) to avoid/minimize slowdown.
Note that apart from function pointers itself, each element of this
flat array also contains two opaque pointers for each ethdev:
1) a pointer to an array of internal queue data pointers
2) points to array of queue callback data pointers.
Note that exposing this extra information allows us to avoid extra
changes inside PMD level, plus should help to avoid possible
performance degradation.
2. Change implementation of 'fast' inline ethdev functions
(rte_eth_rx_burst(), etc.) to use new public flat array.
While it is an ABI breakage, this change is intended to be transparent
for both users (no changes in user app is required) and PMD developers
(no changes in PMD is required).
One extra note - with new implementation RX/TX callback invocation
will cost one extra function call with this changes. That might cause
some slowdown for code-path with RX/TX callbacks heavily involved.
Hope such tradeoff is acceptable for the community.
3. Move rte_eth_dev, rte_eth_dev_data, rte_eth_rxtx_callback and related
things into internal header: <ethdev_driver.h>.
That approach was selected to:
- Avoid(/minimize) possible performance losses.
- Minimize required changes inside PMDs.
Performance testing results (ICX 2.0GHz):
- testpmd macswap fwd mode, plus
a) no RX/TX callbacks:
performance numbers remains the same before and after the patch
b) bpf-load rx 0 0 JM ./dpdk.org/examples/bpf/t3.o:
~2% slowdown
Would like to thank Ferruh and Jerrin for reviewing and testing previous
version of this RFC.
All interested parties please provide your feedback for v2.
If there would be no major objections, I plan to submit a proper v3
patch in next few days.
Konstantin Ananyev (5):
ethdev: allocate max space for internal queue array
ethdev: change input parameters for rx_queue_count
ethdev: copy ethdev 'burst' API into separate structure
ethdev: make burst functions to use new flat array
ethdev: hide eth dev related structures
app/test-pmd/config.c | 23 +-
drivers/common/octeontx2/otx2_sec_idev.c | 2 +-
drivers/crypto/octeontx2/otx2_cryptodev_ops.c | 2 +-
drivers/net/ark/ark_ethdev_rx.c | 4 +-
drivers/net/ark/ark_ethdev_rx.h | 3 +-
drivers/net/atlantic/atl_ethdev.h | 2 +-
drivers/net/atlantic/atl_rxtx.c | 9 +-
drivers/net/bnxt/bnxt_ethdev.c | 8 +-
drivers/net/cxgbe/base/adapter.h | 2 +-
drivers/net/dpaa/dpaa_ethdev.c | 9 +-
drivers/net/dpaa2/dpaa2_ethdev.c | 9 +-
drivers/net/dpaa2/dpaa2_ptp.c | 2 +-
drivers/net/e1000/e1000_ethdev.h | 6 +-
drivers/net/e1000/em_rxtx.c | 4 +-
drivers/net/e1000/igb_rxtx.c | 4 +-
drivers/net/enic/enic_ethdev.c | 12 +-
drivers/net/fm10k/fm10k.h | 2 +-
drivers/net/fm10k/fm10k_rxtx.c | 4 +-
drivers/net/hns3/hns3_rxtx.c | 7 +-
drivers/net/hns3/hns3_rxtx.h | 2 +-
drivers/net/i40e/i40e_rxtx.c | 4 +-
drivers/net/i40e/i40e_rxtx.h | 3 +-
drivers/net/iavf/iavf_rxtx.c | 4 +-
drivers/net/iavf/iavf_rxtx.h | 2 +-
drivers/net/ice/ice_rxtx.c | 4 +-
drivers/net/ice/ice_rxtx.h | 2 +-
drivers/net/igc/igc_txrx.c | 5 +-
drivers/net/igc/igc_txrx.h | 3 +-
drivers/net/ixgbe/ixgbe_ethdev.h | 3 +-
drivers/net/ixgbe/ixgbe_rxtx.c | 4 +-
drivers/net/mlx5/mlx5_rx.c | 26 +-
drivers/net/mlx5/mlx5_rx.h | 2 +-
drivers/net/netvsc/hn_rxtx.c | 4 +-
drivers/net/netvsc/hn_var.h | 3 +-
drivers/net/nfp/nfp_rxtx.c | 4 +-
drivers/net/nfp/nfp_rxtx.h | 3 +-
drivers/net/octeontx2/otx2_ethdev.h | 2 +-
drivers/net/octeontx2/otx2_ethdev_ops.c | 8 +-
drivers/net/sfc/sfc_ethdev.c | 12 +-
drivers/net/thunderx/nicvf_ethdev.c | 3 +-
drivers/net/thunderx/nicvf_rxtx.c | 4 +-
drivers/net/thunderx/nicvf_rxtx.h | 2 +-
drivers/net/txgbe/txgbe_ethdev.h | 3 +-
drivers/net/txgbe/txgbe_rxtx.c | 4 +-
drivers/net/vhost/rte_eth_vhost.c | 4 +-
lib/ethdev/ethdev_driver.h | 152 +++++++++
lib/ethdev/ethdev_private.c | 84 +++++
lib/ethdev/ethdev_private.h | 7 +
lib/ethdev/rte_ethdev.c | 78 +++--
lib/ethdev/rte_ethdev.h | 288 ++++++++++++------
lib/ethdev/rte_ethdev_core.h | 168 +++-------
lib/ethdev/version.map | 6 +
lib/eventdev/rte_event_eth_rx_adapter.c | 2 +-
lib/eventdev/rte_event_eth_tx_adapter.c | 2 +-
lib/eventdev/rte_eventdev.c | 2 +-
55 files changed, 643 insertions(+), 380 deletions(-)
--
2.26.3
^ permalink raw reply [flat|nested] 4+ messages in thread* [dpdk-dev] [PATCH v3 0/7] hide eth dev related structures
2021-09-22 14:09 [dpdk-dev] [RFC v2 0/5] hide eth dev related structures Konstantin Ananyev
@ 2021-10-01 14:02 ` Konstantin Ananyev
2021-10-01 14:02 ` [dpdk-dev] [PATCH v3 3/7] ethdev: copy ethdev 'fast' API into separate structure Konstantin Ananyev
0 siblings, 1 reply; 4+ messages in thread
From: Konstantin Ananyev @ 2021-10-01 14:02 UTC (permalink / raw)
To: dev
Cc: xiaoyun.li, anoobj, jerinj, ndabilpuram, adwivedi,
shepard.siegel, ed.czeck, john.miller, irusskikh, ajit.khaparde,
somnath.kotur, rahul.lakkireddy, hemant.agrawal, sachin.saxena,
haiyue.wang, johndale, hyonkim, qi.z.zhang, xiao.w.wang, humin29,
yisen.zhuang, oulijun, beilei.xing, jingjing.wu, qiming.yang,
matan, viacheslavo, sthemmin, longli, heinrich.kuhn, kirankumark,
andrew.rybchenko, mczekaj, jiawenwu, jianwang, maxime.coquelin,
chenbo.xia, thomas, ferruh.yigit, mdr, jay.jayatheerthan,
Konstantin Ananyev
v3 changes:
- Changes in public struct naming (Jerin/Haiyue)
- Split patches
- Update docs
- Shamelessly included Andrew's patch:
https://patches.dpdk.org/project/dpdk/patch/20210928154856.1015020-1-andrew.rybchenko@oktetlabs.ru/
into these series.
I have to do similar thing here, so decided to avoid duplicated effort.
The aim of these patch series is to make rte_ethdev core data structures
(rte_eth_dev, rte_eth_dev_data, rte_eth_rxtx_callback, etc.) internal to
DPDK and not visible to the user.
That should allow future possible changes to core ethdev related structures
to be transparent to the user and help to improve ABI/API stability.
Note that current ethdev API is preserved, but it is a formal ABI break.
The work is based on previous discussions at:
https://www.mail-archive.com/dev@dpdk.org/msg211405.html
https://www.mail-archive.com/dev@dpdk.org/msg216685.html
and consists of the following main points:
1. Copy public 'fast' function pointers (rx_pkt_burst(), etc.) and
related data pointer from rte_eth_dev into a separate flat array.
We keep it public to still be able to use inline functions for these
'fast' calls (like rte_eth_rx_burst(), etc.) to avoid/minimize slowdown.
Note that apart from function pointers itself, each element of this
flat array also contains two opaque pointers for each ethdev:
1) a pointer to an array of internal queue data pointers
2) points to array of queue callback data pointers.
Note that exposing this extra information allows us to avoid extra
changes inside PMD level, plus should help to avoid possible
performance degradation.
2. Change implementation of 'fast' inline ethdev functions
(rte_eth_rx_burst(), etc.) to use new public flat array.
While it is an ABI breakage, this change is intended to be transparent
for both users (no changes in user app is required) and PMD developers
(no changes in PMD is required).
One extra note - with new implementation RX/TX callback invocation
will cost one extra function call with this changes. That might cause
some slowdown for code-path with RX/TX callbacks heavily involved.
Hope such trade-off is acceptable for the community.
3. Move rte_eth_dev, rte_eth_dev_data, rte_eth_rxtx_callback and related
things into internal header: <ethdev_driver.h>.
That approach was selected to:
- Avoid(/minimize) possible performance losses.
- Minimize required changes inside PMDs.
Performance testing results (ICX 2.0GHz, E810 (ice)):
- testpmd macswap fwd mode, plus
a) no RX/TX callbacks:
no actual slowdown observed
b) bpf-load rx 0 0 JM ./dpdk.org/examples/bpf/t3.o:
~2% slowdown
- l3fwd: no actual slowdown observed
Would like to thank Ferruh and Jerrin for reviewing and testing previous
versions of these series. All other interested parties please don't be shy
and provide your feedback.
Konstantin Ananyev (7):
ethdev: allocate max space for internal queue array
ethdev: change input parameters for rx_queue_count
ethdev: copy ethdev 'fast' API into separate structure
ethdev: make burst functions to use new flat array
ethdev: add API to retrieve multiple ethernet addresses
ethdev: remove legacy Rx descriptor done API
ethdev: hide eth dev related structures
app/test-pmd/config.c | 23 +-
doc/guides/nics/features.rst | 6 +-
doc/guides/rel_notes/deprecation.rst | 5 -
doc/guides/rel_notes/release_21_11.rst | 21 ++
drivers/common/octeontx2/otx2_sec_idev.c | 2 +-
drivers/crypto/octeontx2/otx2_cryptodev_ops.c | 2 +-
drivers/net/ark/ark_ethdev_rx.c | 4 +-
drivers/net/ark/ark_ethdev_rx.h | 3 +-
drivers/net/atlantic/atl_ethdev.h | 2 +-
drivers/net/atlantic/atl_rxtx.c | 9 +-
drivers/net/bnxt/bnxt_ethdev.c | 8 +-
drivers/net/cxgbe/base/adapter.h | 2 +-
drivers/net/dpaa/dpaa_ethdev.c | 9 +-
drivers/net/dpaa2/dpaa2_ethdev.c | 9 +-
drivers/net/dpaa2/dpaa2_ptp.c | 2 +-
drivers/net/e1000/e1000_ethdev.h | 10 +-
drivers/net/e1000/em_ethdev.c | 1 -
drivers/net/e1000/em_rxtx.c | 21 +-
drivers/net/e1000/igb_ethdev.c | 2 -
drivers/net/e1000/igb_rxtx.c | 21 +-
drivers/net/enic/enic_ethdev.c | 12 +-
drivers/net/fm10k/fm10k.h | 5 +-
drivers/net/fm10k/fm10k_ethdev.c | 1 -
drivers/net/fm10k/fm10k_rxtx.c | 29 +-
drivers/net/hns3/hns3_rxtx.c | 7 +-
drivers/net/hns3/hns3_rxtx.h | 2 +-
drivers/net/i40e/i40e_ethdev.c | 1 -
drivers/net/i40e/i40e_ethdev_vf.c | 1 -
drivers/net/i40e/i40e_rxtx.c | 30 +-
drivers/net/i40e/i40e_rxtx.h | 4 +-
drivers/net/iavf/iavf_rxtx.c | 4 +-
drivers/net/iavf/iavf_rxtx.h | 2 +-
drivers/net/ice/ice_rxtx.c | 4 +-
drivers/net/ice/ice_rxtx.h | 2 +-
drivers/net/igc/igc_ethdev.c | 1 -
drivers/net/igc/igc_txrx.c | 23 +-
drivers/net/igc/igc_txrx.h | 5 +-
drivers/net/ixgbe/ixgbe_ethdev.c | 2 -
drivers/net/ixgbe/ixgbe_ethdev.h | 5 +-
drivers/net/ixgbe/ixgbe_rxtx.c | 22 +-
drivers/net/mlx5/mlx5_rx.c | 26 +-
drivers/net/mlx5/mlx5_rx.h | 2 +-
drivers/net/netvsc/hn_rxtx.c | 4 +-
drivers/net/netvsc/hn_var.h | 3 +-
drivers/net/nfp/nfp_rxtx.c | 4 +-
drivers/net/nfp/nfp_rxtx.h | 3 +-
drivers/net/octeontx2/otx2_ethdev.c | 1 -
drivers/net/octeontx2/otx2_ethdev.h | 3 +-
drivers/net/octeontx2/otx2_ethdev_ops.c | 20 +-
drivers/net/sfc/sfc_ethdev.c | 29 +-
drivers/net/thunderx/nicvf_ethdev.c | 3 +-
drivers/net/thunderx/nicvf_rxtx.c | 4 +-
drivers/net/thunderx/nicvf_rxtx.h | 2 +-
drivers/net/txgbe/txgbe_ethdev.h | 3 +-
drivers/net/txgbe/txgbe_rxtx.c | 4 +-
drivers/net/vhost/rte_eth_vhost.c | 4 +-
drivers/net/virtio/virtio_ethdev.c | 1 -
lib/ethdev/ethdev_driver.h | 149 +++++++++
lib/ethdev/ethdev_private.c | 83 +++++
lib/ethdev/ethdev_private.h | 7 +
lib/ethdev/rte_ethdev.c | 79 +++--
lib/ethdev/rte_ethdev.h | 288 ++++++++++++------
lib/ethdev/rte_ethdev_core.h | 171 +++--------
lib/ethdev/version.map | 6 +
lib/eventdev/rte_event_eth_rx_adapter.c | 2 +-
lib/eventdev/rte_event_eth_tx_adapter.c | 2 +-
lib/eventdev/rte_eventdev.c | 2 +-
67 files changed, 661 insertions(+), 568 deletions(-)
--
2.26.3
^ permalink raw reply [flat|nested] 4+ messages in thread* [dpdk-dev] [PATCH v3 3/7] ethdev: copy ethdev 'fast' API into separate structure
2021-10-01 14:02 ` [dpdk-dev] [PATCH v3 0/7] " Konstantin Ananyev
@ 2021-10-01 14:02 ` Konstantin Ananyev
0 siblings, 0 replies; 4+ messages in thread
From: Konstantin Ananyev @ 2021-10-01 14:02 UTC (permalink / raw)
To: dev
Cc: xiaoyun.li, anoobj, jerinj, ndabilpuram, adwivedi,
shepard.siegel, ed.czeck, john.miller, irusskikh, ajit.khaparde,
somnath.kotur, rahul.lakkireddy, hemant.agrawal, sachin.saxena,
haiyue.wang, johndale, hyonkim, qi.z.zhang, xiao.w.wang, humin29,
yisen.zhuang, oulijun, beilei.xing, jingjing.wu, qiming.yang,
matan, viacheslavo, sthemmin, longli, heinrich.kuhn, kirankumark,
andrew.rybchenko, mczekaj, jiawenwu, jianwang, maxime.coquelin,
chenbo.xia, thomas, ferruh.yigit, mdr, jay.jayatheerthan,
Konstantin Ananyev
Copy public function pointers (rx_pkt_burst(), etc.) and related
pointers to internal data from rte_eth_dev structure into a
separate flat array. That array will remain in a public header.
The intention here is to make rte_eth_dev and related structures internal.
That should allow future possible changes to core eth_dev structures
to be transparent to the user and help to avoid ABI/API breakages.
The plan is to keep minimal part of data from rte_eth_dev public,
so we still can use inline functions for 'fast' calls
(like rte_eth_rx_burst(), etc.) to avoid/minimize slowdown.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
---
lib/ethdev/ethdev_private.c | 52 ++++++++++++++++++++++++++++++++++++
lib/ethdev/ethdev_private.h | 7 +++++
lib/ethdev/rte_ethdev.c | 17 ++++++++++++
lib/ethdev/rte_ethdev_core.h | 45 +++++++++++++++++++++++++++++++
4 files changed, 121 insertions(+)
diff --git a/lib/ethdev/ethdev_private.c b/lib/ethdev/ethdev_private.c
index 012cf73ca2..3eeda6e9f9 100644
--- a/lib/ethdev/ethdev_private.c
+++ b/lib/ethdev/ethdev_private.c
@@ -174,3 +174,55 @@ rte_eth_devargs_parse_representor_ports(char *str, void *data)
RTE_LOG(ERR, EAL, "wrong representor format: %s\n", str);
return str == NULL ? -1 : 0;
}
+
+static uint16_t
+dummy_eth_rx_burst(__rte_unused void *rxq,
+ __rte_unused struct rte_mbuf **rx_pkts,
+ __rte_unused uint16_t nb_pkts)
+{
+ RTE_ETHDEV_LOG(ERR, "rx_pkt_burst for unconfigured port\n");
+ rte_errno = ENOTSUP;
+ return 0;
+}
+
+static uint16_t
+dummy_eth_tx_burst(__rte_unused void *txq,
+ __rte_unused struct rte_mbuf **tx_pkts,
+ __rte_unused uint16_t nb_pkts)
+{
+ RTE_ETHDEV_LOG(ERR, "tx_pkt_burst for unconfigured port\n");
+ rte_errno = ENOTSUP;
+ return 0;
+}
+
+void
+eth_dev_fp_ops_reset(struct rte_eth_fp_ops *fpo)
+{
+ static void *dummy_data[RTE_MAX_QUEUES_PER_PORT];
+ static const struct rte_eth_fp_ops dummy_ops = {
+ .rx_pkt_burst = dummy_eth_rx_burst,
+ .tx_pkt_burst = dummy_eth_tx_burst,
+ .rxq = {.data = dummy_data, .clbk = dummy_data,},
+ .txq = {.data = dummy_data, .clbk = dummy_data,},
+ };
+
+ *fpo = dummy_ops;
+}
+
+void
+eth_dev_fp_ops_setup(struct rte_eth_fp_ops *fpo,
+ const struct rte_eth_dev *dev)
+{
+ fpo->rx_pkt_burst = dev->rx_pkt_burst;
+ fpo->tx_pkt_burst = dev->tx_pkt_burst;
+ fpo->tx_pkt_prepare = dev->tx_pkt_prepare;
+ fpo->rx_queue_count = dev->rx_queue_count;
+ fpo->rx_descriptor_status = dev->rx_descriptor_status;
+ fpo->tx_descriptor_status = dev->tx_descriptor_status;
+
+ fpo->rxq.data = dev->data->rx_queues;
+ fpo->rxq.clbk = (void **)(uintptr_t)dev->post_rx_burst_cbs;
+
+ fpo->txq.data = dev->data->tx_queues;
+ fpo->txq.clbk = (void **)(uintptr_t)dev->pre_tx_burst_cbs;
+}
diff --git a/lib/ethdev/ethdev_private.h b/lib/ethdev/ethdev_private.h
index 3724429577..40333e7651 100644
--- a/lib/ethdev/ethdev_private.h
+++ b/lib/ethdev/ethdev_private.h
@@ -26,4 +26,11 @@ eth_find_device(const struct rte_eth_dev *_start, rte_eth_cmp_t cmp,
/* Parse devargs value for representor parameter. */
int rte_eth_devargs_parse_representor_ports(char *str, void *data);
+/* reset eth 'fast' API to dummy values */
+void eth_dev_fp_ops_reset(struct rte_eth_fp_ops *fpo);
+
+/* setup eth 'fast' API to ethdev values */
+void eth_dev_fp_ops_setup(struct rte_eth_fp_ops *fpo,
+ const struct rte_eth_dev *dev);
+
#endif /* _ETH_PRIVATE_H_ */
diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
index 424bc260fa..9fbb1bc3db 100644
--- a/lib/ethdev/rte_ethdev.c
+++ b/lib/ethdev/rte_ethdev.c
@@ -44,6 +44,9 @@
static const char *MZ_RTE_ETH_DEV_DATA = "rte_eth_dev_data";
struct rte_eth_dev rte_eth_devices[RTE_MAX_ETHPORTS];
+/* public 'fast' API */
+struct rte_eth_fp_ops rte_eth_fp_ops[RTE_MAX_ETHPORTS];
+
/* spinlock for eth device callbacks */
static rte_spinlock_t eth_dev_cb_lock = RTE_SPINLOCK_INITIALIZER;
@@ -1788,6 +1791,9 @@ rte_eth_dev_start(uint16_t port_id)
(*dev->dev_ops->link_update)(dev, 0);
}
+ /* expose selection of PMD rx/tx function */
+ eth_dev_fp_ops_setup(rte_eth_fp_ops + port_id, dev);
+
rte_ethdev_trace_start(port_id);
return 0;
}
@@ -1810,6 +1816,9 @@ rte_eth_dev_stop(uint16_t port_id)
return 0;
}
+ /* point rx/tx functions to dummy ones */
+ eth_dev_fp_ops_reset(rte_eth_fp_ops + port_id);
+
dev->data->dev_started = 0;
ret = (*dev->dev_ops->dev_stop)(dev);
rte_ethdev_trace_stop(port_id, ret);
@@ -4568,6 +4577,14 @@ rte_eth_mirror_rule_reset(uint16_t port_id, uint8_t rule_id)
return eth_err(port_id, (*dev->dev_ops->mirror_rule_reset)(dev, rule_id));
}
+RTE_INIT(eth_dev_init_fp_ops)
+{
+ uint32_t i;
+
+ for (i = 0; i != RTE_DIM(rte_eth_fp_ops); i++)
+ eth_dev_fp_ops_reset(rte_eth_fp_ops + i);
+}
+
RTE_INIT(eth_dev_init_cb_lists)
{
uint16_t i;
diff --git a/lib/ethdev/rte_ethdev_core.h b/lib/ethdev/rte_ethdev_core.h
index 00f27c643a..1335ef8bb2 100644
--- a/lib/ethdev/rte_ethdev_core.h
+++ b/lib/ethdev/rte_ethdev_core.h
@@ -53,6 +53,51 @@ typedef int (*eth_rx_descriptor_status_t)(void *rxq, uint16_t offset);
typedef int (*eth_tx_descriptor_status_t)(void *txq, uint16_t offset);
/**< @internal Check the status of a Tx descriptor */
+/**
+ * @internal
+ * Structure used to hold opaque pointernals to internal ethdev RX/TXi
+ * queues data.
+ * The main purpose to expose these pointers at all - allow compiler
+ * to fetch this data for 'fast' ethdev inline functions in advance.
+ */
+struct rte_ethdev_qdata {
+ void **data;
+ /**< points to array of internal queue data pointers */
+ void **clbk;
+ /**< points to array of queue callback data pointers */
+};
+
+/**
+ * @internal
+ * 'fast' ethdev funcions and related data are hold in a flat array.
+ * one entry per ethdev.
+ */
+struct rte_eth_fp_ops {
+
+ /** first 64B line */
+ eth_rx_burst_t rx_pkt_burst;
+ /**< PMD receive function. */
+ eth_tx_burst_t tx_pkt_burst;
+ /**< PMD transmit function. */
+ eth_tx_prep_t tx_pkt_prepare;
+ /**< PMD transmit prepare function. */
+ eth_rx_queue_count_t rx_queue_count;
+ /**< Get the number of used RX descriptors. */
+ eth_rx_descriptor_status_t rx_descriptor_status;
+ /**< Check the status of a Rx descriptor. */
+ eth_tx_descriptor_status_t tx_descriptor_status;
+ /**< Check the status of a Tx descriptor. */
+ uintptr_t reserved[2];
+
+ /** second 64B line */
+ struct rte_ethdev_qdata rxq;
+ struct rte_ethdev_qdata txq;
+ uintptr_t reserved2[4];
+
+} __rte_cache_aligned;
+
+extern struct rte_eth_fp_ops rte_eth_fp_ops[RTE_MAX_ETHPORTS];
+
/**
* @internal
--
2.26.3
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-10-03 21:11 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-02 0:14 [dpdk-dev] [PATCH v3 3/7] ethdev: copy ethdev 'fast' API into separate structure Pavan Nikhilesh Bhagavatula
2021-10-03 20:58 ` Ananyev, Konstantin
2021-10-03 21:10 ` Pavan Nikhilesh Bhagavatula
-- strict thread matches above, loose matches on Subject: below --
2021-09-22 14:09 [dpdk-dev] [RFC v2 0/5] hide eth dev related structures Konstantin Ananyev
2021-10-01 14:02 ` [dpdk-dev] [PATCH v3 0/7] " Konstantin Ananyev
2021-10-01 14:02 ` [dpdk-dev] [PATCH v3 3/7] ethdev: copy ethdev 'fast' API into separate structure Konstantin Ananyev
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).