From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-f65.google.com (mail-ed1-f65.google.com [209.85.208.65]) by dpdk.org (Postfix) with ESMTP id 9A72C1C01 for ; Mon, 29 Oct 2018 17:41:05 +0100 (CET) Received: by mail-ed1-f65.google.com with SMTP id x2-v6so7903786eds.3 for ; Mon, 29 Oct 2018 09:41:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=/IIgIsDZDDCUKf4114ojRNN9zftLjLxHQOve/O0Txl0=; b=GmMU7mXnM4gAO+QcImrk1xasvHbETVynaMYYzG1isFmZMWLhYKeL90yyzxDSXx5gGE BBcy0kEaT9lepNCrOFzaXXFqev64N9kBM0BQomSKY4akeLS+SF5L1N6Jh+HEz1KH+Rq4 i92qOO2V2hxSr1B1KwhRFs3oSSUAS/cVkHv2NreuMYdc9WEfybdK7DposIWxEb/1zZaF OV3SkyKUMt+vmo8Z6+ixhSHd0PPhudxqv0zIupE0N2YcXrcMjJdewagWXn1Thh8BvkOo ulk8MRArW3bzQsE1Fkt9VzyXDgd59n4kYtIaxR1GyuLSIlMoWiDHdYNk35A8Va5tKWop TMwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=/IIgIsDZDDCUKf4114ojRNN9zftLjLxHQOve/O0Txl0=; b=Qs6aa29rKjRWikwZjvHuLn6ahQyVH1RAO4pFEOto/YhhDA7/aD7y1RtQAV3F32O083 Jkdz8vMzYsDu1R2i1TCHqdhPNcBXqqm+sfPJQbMfqJn3oSClzCrqldl0rdcHjJvWx/KE YGPFPjQXlZb08T0Lb7GMKSu0WQG9XVF/l69IG8MD7RJjRClQDB8bXZqTdTZWx6806CHp 8smV8TpP9ouKGHG+qijyrjg6f81aCxhpzk+n2k1xsq6BJ8/0HdcNeKV/si4ZPc3kCcPU YcXgGa5VelhuweLGXQwbN+W4pn1QHMzKKwIHuIYp840WIHn++SU2/x0Ojwqx8WZ2RrhB o+LA== X-Gm-Message-State: AGRZ1gLtY8iDyuU3GQe763tzSCOZksGpTh2i2FQ7OzGseStAvpBEtSZs I0hIIFaDL2jNIWhjar8YU0jY0mQkIG47XMbdLc9I3w== X-Google-Smtp-Source: AJdET5cggDoz4xUgpyZob/ax21O+NcxBrBFKynKxqJ+SZZU75+Rpxuft+ooPg7J6hgWVFd92F5McZ5Ox3sV+Mgu0x+E= X-Received: by 2002:a50:9386:: with SMTP id o6-v6mr14618393eda.248.1540831265209; Mon, 29 Oct 2018 09:41:05 -0700 (PDT) MIME-Version: 1.0 References: <1538743527-8285-1-git-send-email-alejandro.lucero@netronome.com> <1538743527-8285-3-git-send-email-alejandro.lucero@netronome.com> In-Reply-To: From: Alejandro Lucero Date: Mon, 29 Oct 2018 16:40:54 +0000 Message-ID: To: darek.stojaczyk@gmail.com Cc: Thomas Monjalon , "Burakov, Anatoly" , dev Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH v3 2/6] mem: use address hint for mapping hugepages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Oct 2018 16:41:05 -0000 Hi Dariousz, On Mon, Oct 29, 2018 at 4:08 PM Dariusz Stojaczyk wrote: > On Fri, Oct 5, 2018 at 2:47 PM Alejandro Lucero > wrote: > > > > Linux kernel uses a really high address as starting address for > > serving mmaps calls. If there exist addressing limitations and > > IOVA mode is VA, this starting address is likely too high for > > those devices. However, it is possible to use a lower address in > > the process virtual address space as with 64 bits there is a lot > > of available space. > > > > This patch adds an address hint as starting address for 64 bits > > systems and increments the hint for next invocations. If the mmap > > call does not use the hint address, repeat the mmap call using > > the hint address incremented by page size. > > > > Signed-off-by: Alejandro Lucero > > Reviewed-by: Anatoly Burakov > > --- > > lib/librte_eal/common/eal_common_memory.c | 34 > ++++++++++++++++++++++++++++++- > > 1 file changed, 33 insertions(+), 1 deletion(-) > > > > diff --git a/lib/librte_eal/common/eal_common_memory.c > b/lib/librte_eal/common/eal_common_memory.c > > index c482f0d..853c44c 100644 > > --- a/lib/librte_eal/common/eal_common_memory.c > > +++ b/lib/librte_eal/common/eal_common_memory.c > > @@ -37,6 +37,23 @@ > > static void *next_baseaddr; > > static uint64_t system_page_sz; > > > > +#ifdef RTE_ARCH_64 > > +/* > > + * Linux kernel uses a really high address as starting address for > serving > > + * mmaps calls. If there exists addressing limitations and IOVA mode is > VA, > > + * this starting address is likely too high for those devices. However, > it > > + * is possible to use a lower address in the process virtual address > space > > + * as with 64 bits there is a lot of available space. > > + * > > + * Current known limitations are 39 or 40 bits. Setting the starting > address > > + * at 4GB implies there are 508GB or 1020GB for mapping the available > > + * hugepages. This is likely enough for most systems, although a device > with > > + * addressing limitations should call rte_eal_check_dma_mask for > ensuring all > > + * memory is within supported range. > > + */ > > +static uint64_t baseaddr = 0x100000000; > > +#endif > > This breaks running with ASAN unless a custom --base-virtaddr option > is specified. The default base-virtaddr introduced by this patch falls > into an area that's already reserved by ASAN. > > See here: > https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm > The only available address space starts at 0x10007fff8000, which > unfortunately doesn't fit in 39 bits. > > Right now the very first eal_get_virtual_area() in EAL initialization > is used with 4KB pagesize, meaning that DPDK will try to mmap at each > 4KB-aligned offset all the way from 0x100000000 to 0x10007fff8000, > which takes quite a long, long time. > > I'm not sure about the solution to this problem, but I verify that > starting DPDK 18.11-rc1 with `--base-virtaddr 0x200000000000` works > just fine under ASAN. > > Do we have documentation about using Address Sanitizer? I understand the goal but, which is the cost? Do you have numbers about the impact on performance? Solving this is not trivial. I would say someone interested in this but using a hardware with addressing limitations needs to choose. Could it be possible to modify the virtual addresses used by default? I guess the shadow regions can be higher that the default ones. > D. > > > > > >