From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-f68.google.com (mail-ed1-f68.google.com [209.85.208.68]) by dpdk.org (Postfix) with ESMTP id 141045F1C for ; Thu, 1 Nov 2018 12:33:00 +0100 (CET) Received: by mail-ed1-f68.google.com with SMTP id c1-v6so16277665ede.5 for ; Thu, 01 Nov 2018 04:33:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=1q2oEGqdfN5P2xII0icnjRzf+wyC3XNN6cfGVkegmYc=; b=CIzJ6Lh0Vp+q6knUBkCT+D0lU9Bf2HJtjAGKNwk4d32m9Rm34p1wTnI2Mk10gfeK8J Pkr0UX1SuF1tSS3C3b3jcKNcE+rJJPPf/4cjO2lPd4Js2gfpm+jgTvI9xzEEdKcqcpN5 YYVvpWBtQnipZVz6FqKQ6+62nvGZSdrWe/RqVUPsM9plN0ilpT9KlosgzWtJ2OJunHO1 6g5vsCPNE4re/S2pCs/SqWkCWpWQJP+n6RfPoZS/PTYgsYqeWdU5q3HGWWCoCbAOS4cO COpPYcUvTeNTTovKDXxTewyUFgccSFjbSQ+djqM2CO94S5Ytj3Xe781DCPm/dF+PIDBM 0ScA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=1q2oEGqdfN5P2xII0icnjRzf+wyC3XNN6cfGVkegmYc=; b=GZgUkUPLLZDZOMeQaRnHGrB206i/x4vphadjEi9WX/tyQkFnm4iOc4BdIZGAUNAsCb XJcLL+RWeWtEuPh/++qnnTNVXh6P5OSxZ/9PX0WYC08flViKrRzRqSvdPsqNtTgaer/K dNlazGkZHy3/i6MgVk6NTEVE5d93iP7QLN5OSOW2xHV8mqpUC09FYqA009Ic7YKpAUND Un6KPtNGu0NKdU27xtQyXQXycP8C22fdd7Wd2LuZtpy15oWnNzzwusudxlw8zdZNPEOi SrBR7ddS8ykVPUA8FewazN1XhUf/3z2cynDOcjTSBMMiXSeqfC+6GtFNBkX0t6BSB04L COVw== X-Gm-Message-State: AGRZ1gJb2N/AWWCIt3pUw0Ke4QKGXQ/rY6hAjRuqGsJu3dEbwMuF8xVj 2ipK3Tvmb/iq9a/IdEK+ROD4RaiP7v5NBMnXR4uylLA2 X-Google-Smtp-Source: AJdET5fyNkE1aCYRCLL+3iXwzS9d9UG6K7K7EmtUBXPC4xB5QhTCW46iTZfQQSVgH/iMR9dw92r+vUQBXgiB3TAVQhg= X-Received: by 2002:aa7:cf1a:: with SMTP id a26-v6mr4746779edy.91.1541071979662; Thu, 01 Nov 2018 04:32:59 -0700 (PDT) MIME-Version: 1.0 References: <20181031172931.11894-1-alejandro.lucero@netronome.com> <20181031172931.11894-6-alejandro.lucero@netronome.com> <3d4ebdde-326a-811d-4bf7-52498495bade@intel.com> In-Reply-To: <3d4ebdde-326a-811d-4bf7-52498495bade@intel.com> From: Alejandro Lucero Date: Thu, 1 Nov 2018 11:32:48 +0000 Message-ID: To: "Burakov, Anatoly" Cc: dev Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH 5/7] mem: modify error message for DMA mask check X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Nov 2018 11:33:00 -0000 On Thu, Nov 1, 2018 at 11:12 AM Burakov, Anatoly wrote: > On 01-Nov-18 11:03 AM, Alejandro Lucero wrote: > > > > > > On Thu, Nov 1, 2018 at 10:29 AM Burakov, Anatoly > > > wrote: > > > > On 31-Oct-18 5:29 PM, Alejandro Lucero wrote: > > > If DMA mask checks shows mapped memory out of the supported range > > > specified by the DMA mask, nothing can be done but return an error > > > an report the error. This can imply the app not being executed at > > > all or precluding dynamic memory allocation once the app is > running. > > > In any case, we can advice the user to force IOVA as PA if > currently > > > IOVA being VA and user being root. > > > > > > Signed-off-by: Alejandro Lucero > > > > > --- > > > > General comment - legacy memory will also need this check, correct? > > > > > > Yes, there is another patch adding this for both, legacy-mem and no-huge > > options. > > > > > lib/librte_eal/common/malloc_heap.c | 35 > > +++++++++++++++++++++++++---- > > > 1 file changed, 31 insertions(+), 4 deletions(-) > > > > > > diff --git a/lib/librte_eal/common/malloc_heap.c > > b/lib/librte_eal/common/malloc_heap.c > > > index 7d423089d..711622f19 100644 > > > --- a/lib/librte_eal/common/malloc_heap.c > > > +++ b/lib/librte_eal/common/malloc_heap.c > > > @@ -5,8 +5,10 @@ > > > #include > > > #include > > > #include > > > +#include > > > #include > > > #include > > > +#include > > > #include > > > > > > #include > > > @@ -294,7 +296,6 @@ alloc_pages_on_heap(struct malloc_heap *heap, > > uint64_t pg_sz, size_t elt_size, > > > size_t alloc_sz; > > > int allocd_pages; > > > void *ret, *map_addr; > > > - uint64_t mask; > > > > > > alloc_sz = (size_t)pg_sz * n_segs; > > > > > > @@ -322,11 +323,37 @@ alloc_pages_on_heap(struct malloc_heap > > *heap, uint64_t pg_sz, size_t elt_size, > > > goto fail; > > > } > > > > > > + /* Once we have all the memseg lists configured, if there > > is a dma mask > > > + * set, check iova addresses are not out of range. > > Otherwise the device > > > + * setting the dma mask could have problems with the mapped > > memory. > > > + * > > > + * There are two situations when this can happen: > > > + * 1) memory initialization > > > + * 2) dynamic memory allocation > > > + * > > > + * For 1), an error when checking dma mask implies app can > > not be > > > + * executed. For 2) implies the new memory can not be added. > > > + */ > > > if (mcfg->dma_maskbits) { > > > if (rte_mem_check_dma_mask(mcfg->dma_maskbits)) { > > > - RTE_LOG(ERR, EAL, > > > - "%s(): couldn't allocate memory due > > to DMA mask\n", > > > - __func__); > > > + /* Currently this can only happen if IOMMU > > is enabled > > > + * with RTE_ARCH_X86. It is not safe to use > > this memory > > > + * so returning an error here. > > > > I don't think it's RTE_ARCH_X86-only. It can be any other arch with > an > > IOMMU that's reporting addressing limitations. > > > > > > Right, but it is just IOMMU hardware from this architecture having the > > current limitation. > > I was trying to just explain why this can happen but I can remove the > > reference to specific > > architecture problems. > > > > > > > + * > > > + * If IOVA is VA, advice to try with > > '--iova-mode pa' > > > + * which could solve some situations when > > IOVA VA is not > > > + * really needed. > > > + */ > > > + uid_t user = getuid(); > > > + if ((rte_eal_iova_mode() == RTE_IOVA_VA) && > > user == 0) > > > > rte_eal_using_phys_addrs()? > > > > (the above function name is a bit of a misnomer, it really checks if > > they are available, but not necessarily used - it will return true in > > RTE_IOVA_VA mode if you're running as root) > > > > > > rte_eal_iova_mode returns rte_eal_get_configuration()->iova_mode what > > is set during initialization. It can be PA not just because IOMMU (not > > after the patch) > > but because some PMD does not reports IOVA VA support or because UIO is > > in use. > > Checking for root is because IOVA PA can not be used if non root. > > You've misinterpreted my comment. > > rte_eal_using_phys_addrs() will effectively return true if you're > running as root. There's no need for an uid check. > > Ok. I got it now, and it is definitely better than adding the uid check. Thanks > The "misnomer" comment was about the rte_eal_using_phys_addrs() - it > reads like it would return false in IOVA_VA mode, but in reality, it > will return true even if IOVA_VA mode - it really should be named > "rte_eal_phys_addrs_available()" rather than > "rte_eal_using_phys_addrs()". This would make it clearer. > > > > > > > > + RTE_LOG(ERR, EAL, > > > + "%s(): couldn't allocate > > memory due to DMA mask.\n" > > > + "Try with 'iova-mode=pa'\n", > > > + __func__); > > > + else > > > + RTE_LOG(ERR, EAL, > > > + "%s(): couldn't allocate > > memory due to DMA mask\n", > > > + __func__); > > > > I don't think the error message is terribly descriptive. Looking at > it > > through the eyes of someone who sees it for the first time and who > has > > no idea what "iova-mode=pa" is, i think it would be more useful to > word > > it the following way: > > > > couldn't allocate memory due to IOVA exceeding limits of current DMA > > mask. > > [for non-using phys addrs case] Please try initializing EAL with > > --iova-mode=pa parameter. > > > > > > I'm happy with using your terrific description instead ;-) > > Thanks! > > > > Also, generally newlines in RTE_LOG look ugly unless you indent the > > line :) > > > > > goto fail; > > > } > > > } > > > > > > > > > -- > > Thanks, > > Anatoly > > > > > -- > Thanks, > Anatoly >