From: "Tu, Lijuan" <lijuan.tu@intel.com>
To: Alejandro Lucero <alejandro.lucero@netronome.com>,
"dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v3 1/6] mem: add function for checking memsegs IOVAs addresses
Date: Wed, 10 Oct 2018 08:56:05 +0000 [thread overview]
Message-ID: <8CE3E05A3F976642AAB0F4675D0AD20E0B974AA8@SHSMSX101.ccr.corp.intel.com> (raw)
In-Reply-To: <1538743527-8285-2-git-send-email-alejandro.lucero@netronome.com>
Hi
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Alejandro Lucero
> Sent: Friday, October 5, 2018 8:45 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] [PATCH v3 1/6] mem: add function for checking
> memsegs IOVAs addresses
>
> A device can suffer addressing limitations. This function checks memsegs
> have iovas within the supported range based on dma mask.
>
> PMDs should use this function during initialization if device suffers
> addressing limitations, returning an error if this function returns memsegs
> out of range.
>
> Another usage is for emulated IOMMU hardware with addressing limitations.
>
> It is necessary to save the most restricted dma mask for checking out
> memory allocated dynamically after initialization.
>
> Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com>
> Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
> doc/guides/rel_notes/release_18_11.rst | 10 ++++
> lib/librte_eal/common/eal_common_memory.c | 60
> +++++++++++++++++++++++
> lib/librte_eal/common/include/rte_eal_memconfig.h | 3 ++
> lib/librte_eal/common/include/rte_memory.h | 3 ++
> lib/librte_eal/common/malloc_heap.c | 12 +++++
> lib/librte_eal/linuxapp/eal/eal.c | 2 +
> lib/librte_eal/rte_eal_version.map | 1 +
> 7 files changed, 91 insertions(+)
>
> diff --git a/doc/guides/rel_notes/release_18_11.rst
> b/doc/guides/rel_notes/release_18_11.rst
> index 2133a5b..c806dc6 100644
> --- a/doc/guides/rel_notes/release_18_11.rst
> +++ b/doc/guides/rel_notes/release_18_11.rst
> @@ -104,6 +104,14 @@ New Features
> the specified port. The port must be stopped before the command call in
> order
> to reconfigure queues.
>
> +* **Added check for ensuring allocated memory addressable by devices.**
> +
> + Some devices can have addressing limitations so a new function,
> + ``rte_eal_check_dma_mask``, has been added for checking allocated
> + memory is not out of the device range. Because now memory can be
> + dynamically allocated after initialization, a dma mask is kept and
> + any new allocated memory will be checked out against that dma mask
> + and rejected if out of range. If more than one device has addressing
> limitations, the dma mask is the more restricted one.
>
> API Changes
> -----------
> @@ -156,6 +164,8 @@ ABI Changes
> ``rte_config`` structure on account of improving DPDK usability
> when
> using either ``--legacy-mem`` or ``--single-file-segments`` flags.
>
> +* eal: added ``dma_maskbits`` to ``rte_mem_config`` for keeping more
> restricted
> + dma mask based on devices addressing limitations.
>
> Removed Items
> -------------
> diff --git a/lib/librte_eal/common/eal_common_memory.c
> b/lib/librte_eal/common/eal_common_memory.c
> index 0b69804..c482f0d 100644
> --- a/lib/librte_eal/common/eal_common_memory.c
> +++ b/lib/librte_eal/common/eal_common_memory.c
> @@ -385,6 +385,66 @@ struct virtiova {
> rte_memseg_walk(dump_memseg, f);
> }
>
> +static int
> +check_iova(const struct rte_memseg_list *msl __rte_unused,
> + const struct rte_memseg *ms, void *arg) {
> + uint64_t *mask = arg;
> + rte_iova_t iova;
> +
> + /* higher address within segment */
> + iova = (ms->iova + ms->len) - 1;
> + if (!(iova & *mask))
> + return 0;
> +
> + RTE_LOG(DEBUG, EAL, "memseg iova %"PRIx64", len %zx, out of
> range\n",
> + ms->iova, ms->len);
> +
> + RTE_LOG(DEBUG, EAL, "\tusing dma mask %"PRIx64"\n", *mask);
> + return 1;
> +}
> +
> +#if defined(RTE_ARCH_64)
> +#define MAX_DMA_MASK_BITS 63
> +#else
> +#define MAX_DMA_MASK_BITS 31
> +#endif
> +
> +/* check memseg iovas are within the required range based on dma mask
> +*/ int __rte_experimental rte_eal_check_dma_mask(uint8_t maskbits) {
> + struct rte_mem_config *mcfg =
> rte_eal_get_configuration()->mem_config;
> + uint64_t mask;
> +
> + /* sanity check */
> + if (maskbits > MAX_DMA_MASK_BITS) {
> + RTE_LOG(ERR, EAL, "wrong dma mask size %u (Max: %u)\n",
> + maskbits, MAX_DMA_MASK_BITS);
> + return -1;
> + }
> +
> + /* create dma mask */
> + mask = ~((1ULL << maskbits) - 1);
> +
> + if (rte_memseg_walk(check_iova, &mask))
[Lijuan]In my environment, testpmd halts at rte_memseg_walk() when maskbits is 0.
> + /*
> + * Dma mask precludes hugepage usage.
> + * This device can not be used and we do not need to keep
> + * the dma mask.
> + */
> + return 1;
> +
> + /*
> + * we need to keep the more restricted maskbit for checking
> + * potential dynamic memory allocation in the future.
> + */
> + mcfg->dma_maskbits = mcfg->dma_maskbits == 0 ? maskbits :
> + RTE_MIN(mcfg->dma_maskbits, maskbits);
> +
> + return 0;
> +}
> +
> /* return the number of memory channels */ unsigned
> rte_memory_get_nchannel(void) { diff --git
> a/lib/librte_eal/common/include/rte_eal_memconfig.h
> b/lib/librte_eal/common/include/rte_eal_memconfig.h
> index 62a21c2..b5dff70 100644
> --- a/lib/librte_eal/common/include/rte_eal_memconfig.h
> +++ b/lib/librte_eal/common/include/rte_eal_memconfig.h
> @@ -81,6 +81,9 @@ struct rte_mem_config {
> /* legacy mem and single file segments options are shared */
> uint32_t legacy_mem;
> uint32_t single_file_segments;
> +
> + /* keeps the more restricted dma mask */
> + uint8_t dma_maskbits;
> } __attribute__((__packed__));
>
>
> diff --git a/lib/librte_eal/common/include/rte_memory.h
> b/lib/librte_eal/common/include/rte_memory.h
> index 14bd277..c349d6c 100644
> --- a/lib/librte_eal/common/include/rte_memory.h
> +++ b/lib/librte_eal/common/include/rte_memory.h
> @@ -454,6 +454,9 @@ typedef int (*rte_memseg_list_walk_t)(const struct
> rte_memseg_list *msl,
> */
> unsigned rte_memory_get_nrank(void);
>
> +/* check memsegs iovas are within a range based on dma mask */ int
> +rte_eal_check_dma_mask(uint8_t maskbits);
> +
> /**
> * Drivers based on uio will not load unless physical
> * addresses are obtainable. It is only possible to get diff --git
> a/lib/librte_eal/common/malloc_heap.c
> b/lib/librte_eal/common/malloc_heap.c
> index ac7bbb3..3b5b2b6 100644
> --- a/lib/librte_eal/common/malloc_heap.c
> +++ b/lib/librte_eal/common/malloc_heap.c
> @@ -259,11 +259,13 @@ struct malloc_elem *
> int socket, unsigned int flags, size_t align, size_t bound,
> bool contig, struct rte_memseg **ms, int n_segs) {
> + struct rte_mem_config *mcfg =
> rte_eal_get_configuration()->mem_config;
> struct rte_memseg_list *msl;
> struct malloc_elem *elem = NULL;
> size_t alloc_sz;
> int allocd_pages;
> void *ret, *map_addr;
> + uint64_t mask;
>
> alloc_sz = (size_t)pg_sz * n_segs;
>
> @@ -291,6 +293,16 @@ struct malloc_elem *
> goto fail;
> }
>
> + if (mcfg->dma_maskbits) {
> + mask = ~((1ULL << mcfg->dma_maskbits) - 1);
> + if (rte_eal_check_dma_mask(mask)) {
> + RTE_LOG(ERR, EAL,
> + "%s(): couldn't allocate memory due to DMA mask\n",
> + __func__);
> + goto fail;
> + }
> + }
> +
> /* add newly minted memsegs to malloc heap */
> elem = malloc_heap_add_memory(heap, msl, map_addr, alloc_sz);
>
> diff --git a/lib/librte_eal/linuxapp/eal/eal.c
> b/lib/librte_eal/linuxapp/eal/eal.c
> index 4a55d3b..dfe1b8c 100644
> --- a/lib/librte_eal/linuxapp/eal/eal.c
> +++ b/lib/librte_eal/linuxapp/eal/eal.c
> @@ -263,6 +263,8 @@ enum rte_iova_mode
> * processes could later map the config into this exact location */
> rte_config.mem_config->mem_cfg_addr = (uintptr_t)
> rte_mem_cfg_addr;
>
> + rte_config.mem_config->dma_maskbits = 0;
> +
> }
>
> /* attach to an existing shared memory config */ diff --git
> a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map
> index 73282bb..2baefce 100644
> --- a/lib/librte_eal/rte_eal_version.map
> +++ b/lib/librte_eal/rte_eal_version.map
> @@ -291,6 +291,7 @@ EXPERIMENTAL {
> rte_devargs_parsef;
> rte_devargs_remove;
> rte_devargs_type_count;
> + rte_eal_check_dma_mask;
> rte_eal_cleanup;
> rte_eal_hotplug_add;
> rte_eal_hotplug_remove;
> --
> 1.9.1
next prev parent reply other threads:[~2018-10-10 9:00 UTC|newest]
Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-10-05 12:45 [dpdk-dev] [PATCH v3 0/6] use IOVAs check based on DMA mask Alejandro Lucero
2018-10-05 12:45 ` [dpdk-dev] [PATCH v3 1/6] mem: add function for checking memsegs IOVAs addresses Alejandro Lucero
2018-10-10 8:56 ` Tu, Lijuan [this message]
2018-10-11 9:26 ` Alejandro Lucero
2018-10-28 21:03 ` Thomas Monjalon
2018-10-05 12:45 ` [dpdk-dev] [PATCH v3 2/6] mem: use address hint for mapping hugepages Alejandro Lucero
2018-10-29 16:08 ` Dariusz Stojaczyk
2018-10-29 16:40 ` Alejandro Lucero
2018-10-05 12:45 ` [dpdk-dev] [PATCH v3 3/6] bus/pci: check iommu addressing limitation just once Alejandro Lucero
2018-10-05 12:45 ` [dpdk-dev] [PATCH v3 4/6] bus/pci: use IOVAs dmak mask check when setting IOVA mode Alejandro Lucero
2018-10-05 12:45 ` [dpdk-dev] [PATCH v3 5/6] net/nfp: check hugepages IOVAs based on DMA mask Alejandro Lucero
2018-10-05 12:45 ` [dpdk-dev] [PATCH v3 6/6] net/nfp: support IOVA VA mode Alejandro Lucero
2018-10-28 21:04 ` [dpdk-dev] [PATCH v3 0/6] use IOVAs check based on DMA mask Thomas Monjalon
2018-10-29 8:23 ` Yao, Lei A
2018-10-29 8:42 ` Thomas Monjalon
2018-10-29 9:07 ` Thomas Monjalon
2018-10-29 9:25 ` Alejandro Lucero
2018-10-29 9:44 ` Yao, Lei A
2018-10-29 9:36 ` Yao, Lei A
2018-10-29 9:48 ` Thomas Monjalon
2018-10-29 10:11 ` Alejandro Lucero
2018-10-29 10:15 ` Alejandro Lucero
2018-10-29 11:39 ` Alejandro Lucero
2018-10-29 11:46 ` Thomas Monjalon
2018-10-29 12:55 ` Alejandro Lucero
2018-10-29 13:18 ` Yao, Lei A
2018-10-29 13:40 ` Alejandro Lucero
2018-10-29 14:18 ` Thomas Monjalon
2018-10-29 14:35 ` Alejandro Lucero
2018-10-29 18:54 ` Yongseok Koh
2018-10-29 19:37 ` Alejandro Lucero
2018-10-30 10:10 ` Burakov, Anatoly
2018-10-30 10:11 ` Burakov, Anatoly
2018-10-30 10:19 ` Alejandro Lucero
2018-10-30 3:20 ` Lin, Xueqin
2018-10-30 9:41 ` Alejandro Lucero
2018-10-30 10:33 ` Lin, Xueqin
2018-10-30 10:38 ` Alejandro Lucero
2018-10-30 12:21 ` Lin, Xueqin
2018-10-30 12:37 ` Alejandro Lucero
2018-10-30 14:04 ` Alejandro Lucero
2018-10-30 14:14 ` Burakov, Anatoly
2018-10-30 14:45 ` Alejandro Lucero
2018-10-30 14:45 ` Lin, Xueqin
2018-10-30 14:57 ` Alejandro Lucero
2018-10-30 15:09 ` Lin, Xueqin
2018-10-30 10:18 ` Burakov, Anatoly
2018-10-30 10:23 ` Alejandro Lucero
-- strict thread matches above, loose matches on Subject: below --
2018-07-04 12:53 Alejandro Lucero
2018-07-04 12:53 ` [dpdk-dev] [PATCH v3 1/6] mem: add function for checking memsegs IOVAs addresses Alejandro Lucero
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8CE3E05A3F976642AAB0F4675D0AD20E0B974AA8@SHSMSX101.ccr.corp.intel.com \
--to=lijuan.tu@intel.com \
--cc=alejandro.lucero@netronome.com \
--cc=dev@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).