DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
To: Alejandro Lucero <alejandro.lucero@netronome.com>
Cc: dev <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH 5/7] mem: modify error message for DMA mask check
Date: Thu, 1 Nov 2018 11:12:17 +0000	[thread overview]
Message-ID: <3d4ebdde-326a-811d-4bf7-52498495bade@intel.com> (raw)
In-Reply-To: <CAD+H993BMQb=SiR3pXwOpjCq_9_eRVPT-Uh=Dx=SsfxAHAFWjg@mail.gmail.com>

On 01-Nov-18 11:03 AM, Alejandro Lucero wrote:
> 
> 
> On Thu, Nov 1, 2018 at 10:29 AM Burakov, Anatoly 
> <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>> wrote:
> 
>     On 31-Oct-18 5:29 PM, Alejandro Lucero wrote:
>      > If DMA mask checks shows mapped memory out of the supported range
>      > specified by the DMA mask, nothing can be done but return an error
>      > an report the error. This can imply the app not being executed at
>      > all or precluding dynamic memory allocation once the app is running.
>      > In any case, we can advice the user to force IOVA as PA if currently
>      > IOVA being VA and user being root.
>      >
>      > Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com
>     <mailto:alejandro.lucero@netronome.com>>
>      > ---
> 
>     General comment - legacy memory will also need this check, correct?
> 
> 
> Yes, there is another patch adding this for both, legacy-mem and no-huge 
> options.
> 
>      >   lib/librte_eal/common/malloc_heap.c | 35
>     +++++++++++++++++++++++++----
>      >   1 file changed, 31 insertions(+), 4 deletions(-)
>      >
>      > diff --git a/lib/librte_eal/common/malloc_heap.c
>     b/lib/librte_eal/common/malloc_heap.c
>      > index 7d423089d..711622f19 100644
>      > --- a/lib/librte_eal/common/malloc_heap.c
>      > +++ b/lib/librte_eal/common/malloc_heap.c
>      > @@ -5,8 +5,10 @@
>      >   #include <stddef.h>
>      >   #include <stdlib.h>
>      >   #include <stdio.h>
>      > +#include <unistd.h>
>      >   #include <stdarg.h>
>      >   #include <errno.h>
>      > +#include <sys/types.h>
>      >   #include <sys/queue.h>
>      >
>      >   #include <rte_memory.h>
>      > @@ -294,7 +296,6 @@ alloc_pages_on_heap(struct malloc_heap *heap,
>     uint64_t pg_sz, size_t elt_size,
>      >       size_t alloc_sz;
>      >       int allocd_pages;
>      >       void *ret, *map_addr;
>      > -     uint64_t mask;
>      >
>      >       alloc_sz = (size_t)pg_sz * n_segs;
>      >
>      > @@ -322,11 +323,37 @@ alloc_pages_on_heap(struct malloc_heap
>     *heap, uint64_t pg_sz, size_t elt_size,
>      >               goto fail;
>      >       }
>      >
>      > +     /* Once we have all the memseg lists configured, if there
>     is a dma mask
>      > +      * set, check iova addresses are not out of range.
>     Otherwise the device
>      > +      * setting the dma mask could have problems with the mapped
>     memory.
>      > +      *
>      > +      * There are two situations when this can happen:
>      > +      *      1) memory initialization
>      > +      *      2) dynamic memory allocation
>      > +      *
>      > +      * For 1), an error when checking dma mask implies app can
>     not be
>      > +      * executed. For 2) implies the new memory can not be added.
>      > +      */
>      >       if (mcfg->dma_maskbits) {
>      >               if (rte_mem_check_dma_mask(mcfg->dma_maskbits)) {
>      > -                     RTE_LOG(ERR, EAL,
>      > -                             "%s(): couldn't allocate memory due
>     to DMA mask\n",
>      > -                             __func__);
>      > +                     /* Currently this can only happen if IOMMU
>     is enabled
>      > +                      * with RTE_ARCH_X86. It is not safe to use
>     this memory
>      > +                      * so returning an error here.
> 
>     I don't think it's RTE_ARCH_X86-only. It can be any other arch with an
>     IOMMU that's reporting addressing limitations.
> 
> 
> Right, but it is just IOMMU hardware from this architecture having the 
> current limitation.
> I was trying to just explain why this can happen but I can remove the 
> reference to specific
>   architecture problems.
> 
> 
>      > +                      *
>      > +                      * If IOVA is VA, advice to try with
>     '--iova-mode pa'
>      > +                      * which could solve some situations when
>     IOVA VA is not
>      > +                      * really needed.
>      > +                      */
>      > +                     uid_t user = getuid();
>      > +                     if ((rte_eal_iova_mode() == RTE_IOVA_VA) &&
>     user == 0)
> 
>     rte_eal_using_phys_addrs()?
> 
>     (the above function name is a bit of a misnomer, it really checks if
>     they are available, but not necessarily used - it will return true in
>     RTE_IOVA_VA mode if you're running as root)
> 
> 
> rte_eal_iova_mode returns rte_eal_get_configuration()->iova_mode what
>   is set during initialization. It can be PA not just because IOMMU (not 
> after the patch)
> but because some PMD does not reports IOVA VA support or because UIO is 
> in use.
> Checking for root is because IOVA PA can not be used if non root.

You've misinterpreted my comment.

rte_eal_using_phys_addrs() will effectively return true if you're 
running as root. There's no need for an uid check.

The "misnomer" comment was about the rte_eal_using_phys_addrs() - it 
reads like it would return false in IOVA_VA mode, but in reality, it 
will return true even if IOVA_VA mode - it really should be named 
"rte_eal_phys_addrs_available()" rather than 
"rte_eal_using_phys_addrs()". This would make it clearer.

> 
> 
>      > +                             RTE_LOG(ERR, EAL,
>      > +                                     "%s(): couldn't allocate
>     memory due to DMA mask.\n"
>      > +                                     "Try with 'iova-mode=pa'\n",
>      > +                                     __func__);
>      > +                     else
>      > +                             RTE_LOG(ERR, EAL,
>      > +                                     "%s(): couldn't allocate
>     memory due to DMA mask\n",
>      > +                                     __func__);
> 
>     I don't think the error message is terribly descriptive. Looking at it
>     through the eyes of someone who sees it for the first time and who has
>     no idea what "iova-mode=pa" is, i think it would be more useful to word
>     it the following way:
> 
>     couldn't allocate memory due to IOVA exceeding limits of current DMA
>     mask.
>     [for non-using phys addrs case] Please try initializing EAL with
>     --iova-mode=pa parameter.
> 
> 
> I'm happy with using your terrific description instead ;-)
> Thanks!
> 
>     Also, generally newlines in RTE_LOG look ugly unless you indent the
>     line :)
> 
>      >                       goto fail;
>      >               }
>      >       }
>      >
> 
> 
>     -- 
>     Thanks,
>     Anatoly
> 


-- 
Thanks,
Anatoly

  reply	other threads:[~2018-11-01 11:12 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-10-31 17:29 [dpdk-dev] [PATCH 0/7] fix " Alejandro Lucero
2018-10-31 17:29 ` [dpdk-dev] [PATCH 1/7] mem: fix call to " Alejandro Lucero
2018-11-01 10:11   ` Burakov, Anatoly
2018-10-31 17:29 ` [dpdk-dev] [PATCH 2/7] mem: use proper prefix Alejandro Lucero
2018-11-01 10:08   ` Burakov, Anatoly
2018-11-01 10:40     ` Alejandro Lucero
2018-11-01 14:50     ` Thomas Monjalon
2018-11-01 15:03       ` Burakov, Anatoly
2018-11-01 16:18         ` Alejandro Lucero
2018-10-31 17:29 ` [dpdk-dev] [PATCH 3/7] mem: add function for setting DMA mask Alejandro Lucero
2018-11-01 10:11   ` Burakov, Anatoly
2018-11-01 10:48     ` Alejandro Lucero
2018-11-01 10:57       ` Burakov, Anatoly
2018-11-01 11:30         ` Alejandro Lucero
2018-11-01 14:32           ` Alejandro Lucero
2018-10-31 17:29 ` [dpdk-dev] [PATCH 4/7] bus/pci: avoid call to DMA mask check Alejandro Lucero
2018-11-01 10:12   ` Burakov, Anatoly
2018-10-31 17:29 ` [dpdk-dev] [PATCH 5/7] mem: modify error message for " Alejandro Lucero
2018-11-01 10:29   ` Burakov, Anatoly
2018-11-01 11:03     ` Alejandro Lucero
2018-11-01 11:12       ` Burakov, Anatoly [this message]
2018-11-01 11:32         ` Alejandro Lucero
2018-10-31 17:29 ` [dpdk-dev] [PATCH 6/7] mem: add safe and unsafe versions for checking DMA mask Alejandro Lucero
2018-11-01 10:38   ` Burakov, Anatoly
2018-11-01 13:34     ` Alejandro Lucero
2018-10-31 17:29 ` [dpdk-dev] [PATCH 7/7] eal/mem: use DMA mask check for legacy memory Alejandro Lucero
2018-11-01 10:40   ` Burakov, Anatoly
2018-11-01 13:39     ` Alejandro Lucero
2018-11-01 14:28       ` Burakov, Anatoly
2018-11-01 14:32         ` Alejandro Lucero
2018-10-31 17:47 ` [dpdk-dev] [PATCH 0/7] fix DMA mask check Alejandro Lucero
2018-11-02  5:52   ` Hyong Youb Kim
2018-11-01 10:13 ` Mattias Rönnblom
2018-11-01 17:28 ` Ferruh Yigit

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3d4ebdde-326a-811d-4bf7-52498495bade@intel.com \
    --to=anatoly.burakov@intel.com \
    --cc=alejandro.lucero@netronome.com \
    --cc=dev@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).