From: Thomas Monjalon <thomas@monjalon.net>
To: David Marchand <david.marchand@redhat.com>,
Olivier Matz <olivier.matz@6wind.com>
Cc: dev@dpdk.org, Ray Kinsella <mdr@ashroe.eu>,
Parav Pandit <parav@nvidia.com>, Xueming Li <xuemingl@nvidia.com>,
Hemant Agrawal <hemant.agrawal@nxp.com>,
Sachin Saxena <sachin.saxena@oss.nxp.com>,
Stephen Hemminger <sthemmin@microsoft.com>,
Long Li <longli@microsoft.com>,
Ferruh Yigit <ferruh.yigit@amd.com>,
Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Subject: Re: [PATCH v2] drivers/bus: set device NUMA node to unknown by default
Date: Wed, 05 Oct 2022 11:32:17 +0200 [thread overview]
Message-ID: <32019833.aRNtrjHk3s@thomas> (raw)
In-Reply-To: <Yz1Iqfwde4kTkHp6@platinum>
05/10/2022 11:04, Olivier Matz:
> Hi David,
>
> On Wed, Oct 05, 2022 at 10:52:49AM +0200, David Marchand wrote:
> > On Tue, Oct 4, 2022 at 4:59 PM Olivier Matz <olivier.matz@6wind.com> wrote:
> > >
> > > The dev->device.numa_node field is set by each bus driver for
> > > every device it manages to indicate on which NUMA node this device lies.
> > >
> > > When this information is unknown, the assigned value is not consistent
> > > across the bus drivers.
> > >
> > > Set the default value to SOCKET_ID_ANY (-1) by all bus drivers
> > > when the NUMA information is unavailable. This change impacts
> > > rte_eth_dev_socket_id() in the same manner.
> > >
> > > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> > > ---
> > >
> > > v2
> > > * use SOCKET_ID_ANY instead of -1 in drivers/dma/idxd (David)
> > > * document the behavior change of rte_eth_dev_socket_id()
> > > * fix few examples where rte_eth_dev_socket_id() was expected to
> > > return 0 on unknown socket
> >
> > Cc: ethdev maintainers.
> >
> > [snip]
> >
> > > diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
> > > index 53fe21453c..d52f823694 100644
> > > --- a/doc/guides/rel_notes/release_22_11.rst
> > > +++ b/doc/guides/rel_notes/release_22_11.rst
> > > @@ -317,6 +317,12 @@ ABI Changes
> > > * eventdev: Added ``weight`` and ``affinity`` fields
> > > to ``rte_event_queue_conf`` structure.
> > >
> > > +* bus: Changed the device numa node to -1 when NUMA information is unavailable.
> > > + The ``dev->device.numa_node`` field is set by each bus driver for
> > > + every device it manages to indicate on which NUMA node this device lies.
> > > + When this information is unknown, the assigned value was not consistent
> > > + across the bus drivers. This similarly impacts ``rte_eth_dev_socket_id()``.
> > > +
> > >
> > > Known Issues
> > > ------------
> >
> > [snip]
> >
> > > diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
> > > index a21f58b9cd..dd8d25d6d4 100644
> > > --- a/lib/ethdev/rte_ethdev.h
> > > +++ b/lib/ethdev/rte_ethdev.h
> > > @@ -2445,8 +2445,8 @@ int rte_eth_hairpin_unbind(uint16_t tx_port, uint16_t rx_port);
> > > * The port identifier of the Ethernet device
> > > * @return
> > > * The NUMA socket ID to which the Ethernet device is connected or
> > > - * a default of zero if the socket could not be determined.
> > > - * -1 is returned is the port_id value is out of range.
> > > + * a default of -1 (SOCKET_ID_ANY) if the socket could not be determined.
> > > + * -1 is also returned if the port_id is invalid.
> > > */
> > > int rte_eth_dev_socket_id(uint16_t port_id);
> >
> > It would be better to distinguish the two cases, using rte_errno.
> > Something like:
> >
> > diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
> > index 2821770e2d..1baf302804 100644
> > --- a/lib/ethdev/rte_ethdev.c
> > +++ b/lib/ethdev/rte_ethdev.c
> > @@ -562,8 +562,16 @@ rte_eth_dev_owner_get(const uint16_t port_id,
> > struct rte_eth_dev_owner *owner)
> > int
> > rte_eth_dev_socket_id(uint16_t port_id)
> > {
> > - RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -1);
> > - return rte_eth_devices[port_id].data->numa_node;
> > + int socket_id = SOCKET_ID_ANY;
> > +
> > + if (!rte_eth_dev_is_valid_port(port_id))
> > + rte_errno = EINVAL;
> > + } else {
> > + socket_id = rte_eth_devices[port_id].data->numa_node;
> > + if (socket_id == SOCKET_ID_ANY)
> > + rte_errno = 0;
> > + }
> > + return socket_id;
> > }
> >
> > void *
> > diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
> > index dd8d25d6d4..03456b2dbb 100644
> > --- a/lib/ethdev/rte_ethdev.h
> > +++ b/lib/ethdev/rte_ethdev.h
> > @@ -2444,9 +2444,11 @@ int rte_eth_hairpin_unbind(uint16_t tx_port,
> > uint16_t rx_port);
> > * @param port_id
> > * The port identifier of the Ethernet device
> > * @return
> > - * The NUMA socket ID to which the Ethernet device is connected or
> > - * a default of -1 (SOCKET_ID_ANY) if the socket could not be determined.
> > - * -1 is also returned if the port_id is invalid.
> > + * - The NUMA socket ID which the Ethernet device is connected to.
> > + * - -1 (which translates to SOCKET_ID_ANY) if the socket could not be
> > + * determined. rte_errno is then set to:
> > + * - EINVAL is the port_id is invalid,
> > + * - 0 is the socket could not be determined,
> > */
> > int rte_eth_dev_socket_id(uint16_t port_id);
> >
> > WDYT?
>
> As discussed off-list, it is indeed better.
+1
Note that the decision to take in case of EINVAL
requires some work for each call to rte_eth_dev_socket_id().
I suppose we can leave it as a future exercise.
next prev parent reply other threads:[~2022-10-05 9:32 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-29 12:05 [PATCH] " Olivier Matz
2022-09-30 7:10 ` David Marchand
2022-09-30 8:11 ` Olivier Matz
2022-10-04 14:58 ` [PATCH v2] " Olivier Matz
2022-10-05 8:52 ` David Marchand
2022-10-05 9:04 ` Olivier Matz
2022-10-05 9:32 ` Thomas Monjalon [this message]
2022-10-06 19:31 ` David Marchand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=32019833.aRNtrjHk3s@thomas \
--to=thomas@monjalon.net \
--cc=andrew.rybchenko@oktetlabs.ru \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@amd.com \
--cc=hemant.agrawal@nxp.com \
--cc=longli@microsoft.com \
--cc=mdr@ashroe.eu \
--cc=olivier.matz@6wind.com \
--cc=parav@nvidia.com \
--cc=sachin.saxena@oss.nxp.com \
--cc=sthemmin@microsoft.com \
--cc=xuemingl@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).