From: "Morten Brørup" <mb@smartsharesystems.com>
To: "Didier Pallard" <didier.pallard@6wind.com>,
"Akhil Goyal" <gakhil@marvell.com>,
"Fan Zhang" <fanzhang.oss@gmail.com>,
"Olivier Matz" <olivier.matz@6wind.com>, <thomas@monjalon.net>
Cc: <dev@dpdk.org>
Subject: RE: [RFC] Fix cryptodev socket id for devices on unknown NUMA node
Date: Tue, 17 Jan 2023 12:32:14 +0100 [thread overview]
Message-ID: <98CBD80474FA8B44BF855DF32C47DC35D8767D@smartserver.smartshare.dk> (raw)
In-Reply-To: <20230117101646.2521875-1-didier.pallard@6wind.com>
> From: Didier Pallard [mailto:didier.pallard@6wind.com]
> Sent: Tuesday, 17 January 2023 11.17
>
> Since DPDK 22.11 and below commit:
> https://git.dpdk.org/dpdk/commit/?id=7dcd73e37965ba0bfa430efeac362fe183
> ed0ae2
> rte_cryptodev_socket_id() could return an incorrect value of 255.
> Problem has been seen during configuration of the qat device
> on an Atom C3000 architecture. On this arch, PCI is not depending on
> any numa socket, causing device numa_node to be equal to SOCKET_ID_ANY.
Disclaimer: I'm not up to speed with this topic or patch, so feel free to ignore my comments here! I'm only speaking up because I fear we are increasing the risk of bugs here. But again, please bear with me, if I have totally misunderstood this!
I was under the impression that single-socket systems used socket_id 0 as default. How can the PCI bus (or QAT device) not depend on any socket? It must be connected somewhere.
Doesn't assigning socket_id = -1 for devices (QAT or anything else) introduce a big risk of bugs, e.g. in comparisons? The special socket_id value -1 should have only two meanings: 1) return value "error", or 2) input value "any". Now it also can mean 3) "unknown"? How do comparison functions work for that... is "any" == "unknown"? And does searching for "0" match "unknown"? It might, or might not, but searching for "any" does match "0". And how about searching for "unknown", if such a value is propagate around in the system.
And if we started considering socket_id == -1 valid with that patch, should the return type of rte_socket_id(void) be signed instead of unsigned?
> Due to incorrect cast to uint8_t, this value is stored as 255
> in cryptodev structure and returned as such by
> rte_cryptodev_socket_id()
> function.
>
> Below patch proposes one way to fix the issue: casting to a signed
> int8_t
> instead of the uint8_t. (it could also be casted to an int, that is the
> usual type for numa_node, but this may break the ABI). This makes the
> SOCKET_ID_ANY being propagated up to the user.
> Another solution could be to always store a valid numa_node in this
> field
> instead of just copying the numa_node field of the device, but this
> requires to fix most crypto PMDs, that are currently just copying the
> device value.
>
> What is the preferred solution?
I prefer that garbage data is not propagated, even if it requires fixing many places.
But as I indicated above, I wonder if part of the root cause stems from considering socket_id == -1 valid data in some structures. (It could be allowed temporarily, e.g. in a template to indicate that the field is uninitialized. But it should not propagate outside the templates when instantiating objects based on the templates.)
>
> ---
> cryptodev: fix numa_node type
>
> Since below commit, numa_node can be set to SOCKET_ID_ANY.
> Do not cast numa_node to an unsigned uint8, else SOCKET_ID_ANY
> is converted to 255, causing rte_cryptodev_socket_id to return
> an incorrect value.
>
> Fixes: 7dcd73e37965 ("drivers/bus: set device NUMA node to unknown by
> default")
> Signed-off-by: Didier Pallard <didier.pallard@6wind.com>
> ---
> lib/cryptodev/cryptodev_pmd.h | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/lib/cryptodev/cryptodev_pmd.h
> b/lib/cryptodev/cryptodev_pmd.h
> index 0020102eb7db..db4745d620f4 100644
> --- a/lib/cryptodev/cryptodev_pmd.h
> +++ b/lib/cryptodev/cryptodev_pmd.h
> @@ -64,8 +64,8 @@ struct rte_cryptodev_pmd_init_params {
> struct rte_cryptodev_data {
> /** Device ID for this instance */
> uint8_t dev_id;
> - /** Socket ID where memory is allocated */
> - uint8_t socket_id;
> + /** Socket ID of the device */
> + int8_t socket_id;
> /** Unique identifier name */
> char name[RTE_CRYPTODEV_NAME_MAX_LEN];
>
> --
> 2.30.2
>
next prev parent reply other threads:[~2023-01-17 11:32 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-17 10:16 Didier Pallard
2023-01-17 11:32 ` Morten Brørup [this message]
2023-01-17 13:03 ` Bruce Richardson
2023-01-17 13:36 ` Morten Brørup
2023-01-17 13:59 ` Bruce Richardson
2023-01-18 8:52 ` Morten Brørup
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=98CBD80474FA8B44BF855DF32C47DC35D8767D@smartserver.smartshare.dk \
--to=mb@smartsharesystems.com \
--cc=dev@dpdk.org \
--cc=didier.pallard@6wind.com \
--cc=fanzhang.oss@gmail.com \
--cc=gakhil@marvell.com \
--cc=olivier.matz@6wind.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).