From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id B54495F2E for ; Thu, 1 Nov 2018 11:57:36 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Nov 2018 03:57:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,451,1534834800"; d="scan'208";a="246161579" Received: from aburakov-mobl1.ger.corp.intel.com (HELO [10.237.220.72]) ([10.237.220.72]) by orsmga004.jf.intel.com with ESMTP; 01 Nov 2018 03:57:34 -0700 To: Alejandro Lucero Cc: dev References: <20181031172931.11894-1-alejandro.lucero@netronome.com> <20181031172931.11894-4-alejandro.lucero@netronome.com> <2be201f7-308f-53e1-715a-6c37b92de10b@intel.com> From: "Burakov, Anatoly" Message-ID: <1496b1e0-f3de-2a02-a5d9-3e6c65eb93d4@intel.com> Date: Thu, 1 Nov 2018 10:57:34 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] [PATCH 3/7] mem: add function for setting DMA mask X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Nov 2018 10:57:37 -0000 On 01-Nov-18 10:48 AM, Alejandro Lucero wrote: > > > On Thu, Nov 1, 2018 at 10:11 AM Burakov, Anatoly > > wrote: > > On 31-Oct-18 5:29 PM, Alejandro Lucero wrote: > > This patch adds the possibility of setting a dma mask to be used > > once the memory initialization is done. > > > > This is currently needed when IOVA mode is set by PCI related > > code and an x86 IOMMU hardware unit is present. Current code calls > > rte_mem_check_dma_mask but it is wrong to do so at that point > > because the memory has not been initialized yet. > > > > Signed-off-by: Alejandro Lucero > > > --- > >   lib/librte_eal/common/eal_common_memory.c  | 10 ++++++++++ > >   lib/librte_eal/common/include/rte_memory.h | 10 ++++++++++ > >   lib/librte_eal/rte_eal_version.map         |  1 + > >   3 files changed, 21 insertions(+) > > > > diff --git a/lib/librte_eal/common/eal_common_memory.c > b/lib/librte_eal/common/eal_common_memory.c > > index e0f08f39a..24b72fcb0 100644 > > --- a/lib/librte_eal/common/eal_common_memory.c > > +++ b/lib/librte_eal/common/eal_common_memory.c > > @@ -480,6 +480,16 @@ rte_mem_check_dma_mask(uint8_t maskbits) > >       return 0; > >   } > > > > +/* set dma mask to use when memory initialization is done */ > > +void __rte_experimental > > +rte_mem_set_dma_mask(uint8_t maskbits) > > +{ > > +     struct rte_mem_config *mcfg = > rte_eal_get_configuration()->mem_config; > > + > > +     mcfg->dma_maskbits = mcfg->dma_maskbits == 0 ? maskbits : > > +                          RTE_MIN(mcfg->dma_maskbits, maskbits); > > +} > > + > >   /* return the number of memory channels */ > >   unsigned rte_memory_get_nchannel(void) > >   { > > diff --git a/lib/librte_eal/common/include/rte_memory.h > b/lib/librte_eal/common/include/rte_memory.h > > index ad3f3cfb0..eff028db1 100644 > > --- a/lib/librte_eal/common/include/rte_memory.h > > +++ b/lib/librte_eal/common/include/rte_memory.h > > @@ -466,6 +466,16 @@ unsigned rte_memory_get_nrank(void); > >   /* check memsegs iovas are within a range based on dma mask */ > >   int __rte_experimental rte_mem_check_dma_mask(uint8_t maskbits); > > > > +/** > > + *  * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice > > I think there's a copy-paste error here (extra star and indent before > warning). > > > I will fix this. > > Thanks. > > > > + * > > + *  Set dma mask to use once memory initialization is done. > > + *  Previous function rte_mem_check_dma_mask can not be used > > + *  safely until memory has been initialized. > > IMO this really requires a big bold warning saying that this function > must only be called early during initialization, before memory is > initialized, and not to be called in user code. > > > what about adding a new EAL variable for keeping memory initialization > status? > It would be something like "uninit" until is changed to "done" after the > memory initialization is completed. > That variable would help to avoid effective calls to set the dma mask > after initialization. I'm not opposed to it in principle, however, this is a slippery slope towards having each and every subsystem storing their init status :) That said, while it's not a complete solution because it won't prevent spurious calls to this function _after_ memory has initialized, there is a variable in internal_config that you can use to deny calls to this function after EAL init is complete (see internal_config->init_complete). You could also store a static variable in eal_common_memory.c, and change it at the end of rte_eal_memory_init() call, but that does not really complete the memory initialization, because we then go to rte_eal_malloc_heap_init(), which actually completes the memory init. I would greatly prefer using internal_config->init_complete - it is enough to forbid user code from calling it, while driver/DPDK developers presumably know what they're doing enough to read the warning and not try to call this after memory init :) > > > + */ > > +void __rte_experimental rte_mem_set_dma_mask(uint8_t maskbits); > > + > >   /** > >    * Drivers based on uio will not load unless physical > >    * addresses are obtainable. It is only possible to get > > diff --git a/lib/librte_eal/rte_eal_version.map > b/lib/librte_eal/rte_eal_version.map > > index ef8126a97..ae24b5c73 100644 > > --- a/lib/librte_eal/rte_eal_version.map > > +++ b/lib/librte_eal/rte_eal_version.map > > @@ -296,6 +296,7 @@ EXPERIMENTAL { > >       rte_devargs_remove; > >       rte_devargs_type_count; > >       rte_mem_check_dma_mask; > > +     rte_mem_set_dma_mask; > > Same, this needs to be alphabetic. > > >       rte_eal_cleanup; > >       rte_fbarray_attach; > >       rte_fbarray_destroy; > > > > > -- > Thanks, > Anatoly > -- Thanks, Anatoly