From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id CB3A51B470 for ; Wed, 10 Oct 2018 11:00:08 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2018 02:00:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,363,1534834800"; d="scan'208";a="80008688" Received: from fmsmsx106.amr.corp.intel.com ([10.18.124.204]) by orsmga007.jf.intel.com with ESMTP; 10 Oct 2018 01:56:24 -0700 Received: from fmsmsx102.amr.corp.intel.com (10.18.124.200) by FMSMSX106.amr.corp.intel.com (10.18.124.204) with Microsoft SMTP Server (TLS) id 14.3.319.2; Wed, 10 Oct 2018 01:56:23 -0700 Received: from shsmsx104.ccr.corp.intel.com (10.239.4.70) by FMSMSX102.amr.corp.intel.com (10.18.124.200) with Microsoft SMTP Server (TLS) id 14.3.319.2; Wed, 10 Oct 2018 01:56:23 -0700 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.60]) by SHSMSX104.ccr.corp.intel.com ([169.254.5.183]) with mapi id 14.03.0319.002; Wed, 10 Oct 2018 16:56:05 +0800 From: "Tu, Lijuan" To: Alejandro Lucero , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v3 1/6] mem: add function for checking memsegs IOVAs addresses Thread-Index: AQHUXKnhxX2o2cEJWE2jwhF6RJzloKUYM8ZA Date: Wed, 10 Oct 2018 08:56:05 +0000 Message-ID: <8CE3E05A3F976642AAB0F4675D0AD20E0B974AA8@SHSMSX101.ccr.corp.intel.com> References: <1538743527-8285-1-git-send-email-alejandro.lucero@netronome.com> <1538743527-8285-2-git-send-email-alejandro.lucero@netronome.com> In-Reply-To: <1538743527-8285-2-git-send-email-alejandro.lucero@netronome.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.0.400.15 dlp-reaction: no-action x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZWFmNTFlNzItMDZiOS00ODI1LTkyMjctMGQ1NjY3Y2E1Y2RmIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiSDg4T0ZHblluOEV0ZTZxNlNueVRPdE5xVkorUnVCNzMxRVZLaG1yeXAreWI5TjRVV3B3dHZlVUxKTVdsTEhxciJ9 x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v3 1/6] mem: add function for checking memsegs IOVAs addresses X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Oct 2018 09:00:09 -0000 Hi=20 > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Alejandro Lucero > Sent: Friday, October 5, 2018 8:45 PM > To: dev@dpdk.org > Subject: [dpdk-dev] [PATCH v3 1/6] mem: add function for checking > memsegs IOVAs addresses >=20 > A device can suffer addressing limitations. This function checks memsegs > have iovas within the supported range based on dma mask. >=20 > PMDs should use this function during initialization if device suffers > addressing limitations, returning an error if this function returns memse= gs > out of range. >=20 > Another usage is for emulated IOMMU hardware with addressing limitations. >=20 > It is necessary to save the most restricted dma mask for checking out > memory allocated dynamically after initialization. >=20 > Signed-off-by: Alejandro Lucero > Reviewed-by: Anatoly Burakov > --- > doc/guides/rel_notes/release_18_11.rst | 10 ++++ > lib/librte_eal/common/eal_common_memory.c | 60 > +++++++++++++++++++++++ > lib/librte_eal/common/include/rte_eal_memconfig.h | 3 ++ > lib/librte_eal/common/include/rte_memory.h | 3 ++ > lib/librte_eal/common/malloc_heap.c | 12 +++++ > lib/librte_eal/linuxapp/eal/eal.c | 2 + > lib/librte_eal/rte_eal_version.map | 1 + > 7 files changed, 91 insertions(+) >=20 > diff --git a/doc/guides/rel_notes/release_18_11.rst > b/doc/guides/rel_notes/release_18_11.rst > index 2133a5b..c806dc6 100644 > --- a/doc/guides/rel_notes/release_18_11.rst > +++ b/doc/guides/rel_notes/release_18_11.rst > @@ -104,6 +104,14 @@ New Features > the specified port. The port must be stopped before the command call i= n > order > to reconfigure queues. >=20 > +* **Added check for ensuring allocated memory addressable by devices.** > + > + Some devices can have addressing limitations so a new function, > + ``rte_eal_check_dma_mask``, has been added for checking allocated > + memory is not out of the device range. Because now memory can be > + dynamically allocated after initialization, a dma mask is kept and > + any new allocated memory will be checked out against that dma mask > + and rejected if out of range. If more than one device has addressing > limitations, the dma mask is the more restricted one. >=20 > API Changes > ----------- > @@ -156,6 +164,8 @@ ABI Changes > ``rte_config`` structure on account of improving DPDK usability > when > using either ``--legacy-mem`` or ``--single-file-segments`` flags= . >=20 > +* eal: added ``dma_maskbits`` to ``rte_mem_config`` for keeping more > restricted > + dma mask based on devices addressing limitations. >=20 > Removed Items > ------------- > diff --git a/lib/librte_eal/common/eal_common_memory.c > b/lib/librte_eal/common/eal_common_memory.c > index 0b69804..c482f0d 100644 > --- a/lib/librte_eal/common/eal_common_memory.c > +++ b/lib/librte_eal/common/eal_common_memory.c > @@ -385,6 +385,66 @@ struct virtiova { > rte_memseg_walk(dump_memseg, f); > } >=20 > +static int > +check_iova(const struct rte_memseg_list *msl __rte_unused, > + const struct rte_memseg *ms, void *arg) { > + uint64_t *mask =3D arg; > + rte_iova_t iova; > + > + /* higher address within segment */ > + iova =3D (ms->iova + ms->len) - 1; > + if (!(iova & *mask)) > + return 0; > + > + RTE_LOG(DEBUG, EAL, "memseg iova %"PRIx64", len %zx, out of > range\n", > + ms->iova, ms->len); > + > + RTE_LOG(DEBUG, EAL, "\tusing dma mask %"PRIx64"\n", *mask); > + return 1; > +} > + > +#if defined(RTE_ARCH_64) > +#define MAX_DMA_MASK_BITS 63 > +#else > +#define MAX_DMA_MASK_BITS 31 > +#endif > + > +/* check memseg iovas are within the required range based on dma mask > +*/ int __rte_experimental rte_eal_check_dma_mask(uint8_t maskbits) { > + struct rte_mem_config *mcfg =3D > rte_eal_get_configuration()->mem_config; > + uint64_t mask; > + > + /* sanity check */ > + if (maskbits > MAX_DMA_MASK_BITS) { > + RTE_LOG(ERR, EAL, "wrong dma mask size %u (Max: %u)\n", > + maskbits, MAX_DMA_MASK_BITS); > + return -1; > + } > + > + /* create dma mask */ > + mask =3D ~((1ULL << maskbits) - 1); > + > + if (rte_memseg_walk(check_iova, &mask)) [Lijuan]In my environment, testpmd halts at rte_memseg_walk() when maskbits= is 0. > + /* > + * Dma mask precludes hugepage usage. > + * This device can not be used and we do not need to keep > + * the dma mask. > + */ > + return 1; > + > + /* > + * we need to keep the more restricted maskbit for checking > + * potential dynamic memory allocation in the future. > + */ > + mcfg->dma_maskbits =3D mcfg->dma_maskbits =3D=3D 0 ? maskbits : > + RTE_MIN(mcfg->dma_maskbits, maskbits); > + > + return 0; > +} > + > /* return the number of memory channels */ unsigned > rte_memory_get_nchannel(void) { diff --git > a/lib/librte_eal/common/include/rte_eal_memconfig.h > b/lib/librte_eal/common/include/rte_eal_memconfig.h > index 62a21c2..b5dff70 100644 > --- a/lib/librte_eal/common/include/rte_eal_memconfig.h > +++ b/lib/librte_eal/common/include/rte_eal_memconfig.h > @@ -81,6 +81,9 @@ struct rte_mem_config { > /* legacy mem and single file segments options are shared */ > uint32_t legacy_mem; > uint32_t single_file_segments; > + > + /* keeps the more restricted dma mask */ > + uint8_t dma_maskbits; > } __attribute__((__packed__)); >=20 >=20 > diff --git a/lib/librte_eal/common/include/rte_memory.h > b/lib/librte_eal/common/include/rte_memory.h > index 14bd277..c349d6c 100644 > --- a/lib/librte_eal/common/include/rte_memory.h > +++ b/lib/librte_eal/common/include/rte_memory.h > @@ -454,6 +454,9 @@ typedef int (*rte_memseg_list_walk_t)(const struct > rte_memseg_list *msl, > */ > unsigned rte_memory_get_nrank(void); >=20 > +/* check memsegs iovas are within a range based on dma mask */ int > +rte_eal_check_dma_mask(uint8_t maskbits); > + > /** > * Drivers based on uio will not load unless physical > * addresses are obtainable. It is only possible to get diff --git > a/lib/librte_eal/common/malloc_heap.c > b/lib/librte_eal/common/malloc_heap.c > index ac7bbb3..3b5b2b6 100644 > --- a/lib/librte_eal/common/malloc_heap.c > +++ b/lib/librte_eal/common/malloc_heap.c > @@ -259,11 +259,13 @@ struct malloc_elem * > int socket, unsigned int flags, size_t align, size_t bound, > bool contig, struct rte_memseg **ms, int n_segs) { > + struct rte_mem_config *mcfg =3D > rte_eal_get_configuration()->mem_config; > struct rte_memseg_list *msl; > struct malloc_elem *elem =3D NULL; > size_t alloc_sz; > int allocd_pages; > void *ret, *map_addr; > + uint64_t mask; >=20 > alloc_sz =3D (size_t)pg_sz * n_segs; >=20 > @@ -291,6 +293,16 @@ struct malloc_elem * > goto fail; > } >=20 > + if (mcfg->dma_maskbits) { > + mask =3D ~((1ULL << mcfg->dma_maskbits) - 1); > + if (rte_eal_check_dma_mask(mask)) { > + RTE_LOG(ERR, EAL, > + "%s(): couldn't allocate memory due to DMA mask\n", > + __func__); > + goto fail; > + } > + } > + > /* add newly minted memsegs to malloc heap */ > elem =3D malloc_heap_add_memory(heap, msl, map_addr, alloc_sz); >=20 > diff --git a/lib/librte_eal/linuxapp/eal/eal.c > b/lib/librte_eal/linuxapp/eal/eal.c > index 4a55d3b..dfe1b8c 100644 > --- a/lib/librte_eal/linuxapp/eal/eal.c > +++ b/lib/librte_eal/linuxapp/eal/eal.c > @@ -263,6 +263,8 @@ enum rte_iova_mode > * processes could later map the config into this exact location */ > rte_config.mem_config->mem_cfg_addr =3D (uintptr_t) > rte_mem_cfg_addr; >=20 > + rte_config.mem_config->dma_maskbits =3D 0; > + > } >=20 > /* attach to an existing shared memory config */ diff --git > a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map > index 73282bb..2baefce 100644 > --- a/lib/librte_eal/rte_eal_version.map > +++ b/lib/librte_eal/rte_eal_version.map > @@ -291,6 +291,7 @@ EXPERIMENTAL { > rte_devargs_parsef; > rte_devargs_remove; > rte_devargs_type_count; > + rte_eal_check_dma_mask; > rte_eal_cleanup; > rte_eal_hotplug_add; > rte_eal_hotplug_remove; > -- > 1.9.1