From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 848AFA00C5; Tue, 26 Jul 2022 21:59:15 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AD44640E25; Tue, 26 Jul 2022 21:59:14 +0200 (CEST) Received: from mail-lj1-f179.google.com (mail-lj1-f179.google.com [209.85.208.179]) by mails.dpdk.org (Postfix) with ESMTP id 240F640E0F for ; Tue, 26 Jul 2022 21:59:13 +0200 (CEST) Received: by mail-lj1-f179.google.com with SMTP id a13so17468658ljr.11 for ; Tue, 26 Jul 2022 12:59:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Bn1XCVZbpr64iExG6uem8B+N+z5h9Rio1Uglc5V9lVo=; b=kTxD2XXSB6AN8qdcDYc5nkep7UUlNxxK8pLXdLxuagD169f44K2wDgvLoKyXInUFxW 8a/Lu3r7y0n5TlFKMEmvSmtfrR+MMLWt1ICxwH9calOjHwry3nbSDObGUUOEnTZF8RBI Bwdlyl0+1Jsg9/4sKiQy/kmeuuKe4tbt8wQ3PWDDvzLNU+o7wW1n0AX67ZrT/81Txpu8 v9dfQ4UhW+8dORwLWgXJ4L8Gh5zSmOUt4GhjEBm4nzICYebe7EPEZ3ZlKuU97fsSKBTT 4QUPYEWSKfYUmlZWuNo/7WrEikPwJbftgunJglF/bFEVY8138V77c3K8WQ/jo4VTn1x7 eNBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Bn1XCVZbpr64iExG6uem8B+N+z5h9Rio1Uglc5V9lVo=; b=dpr7043a2NFZxy1/YVaBqAhTaZ1jWXJu5hSDd/hzzGweTELg3cewWIenSJqLm9oL2/ q1chqc6+jPQT1EpXqCTR+vFpAvMxuIiqHYPUtiX57PXSLizUvCLBzXj19u7tllpPbaJD gCTSBJ2vK8tdBXkfaTYAbt+etWvdaWkGh3lisPTSF2Wxa7wHkqmHzLEXvbziUzNjwjBD LPnN8v3UmMZGKeeVQQ8Fe2fuOrPBhqZfds7PQiECWZL+tDexgguK6z9IWPRQ4J2X1SLJ eDe4nf06wuicJ3NicOfGhMQh9aQCe5UWmnoqw4lYHCGPWTBj+kd6/YwAXk/VPTgVRDqD 7LRQ== X-Gm-Message-State: AJIora80jdXakyMaRRMMqcKMplDTSChgG5UR24ZPbEweYhc4puczTUR6 kPXe1zPaPZA6F/QbISOskTsfuDSZ+xs= X-Google-Smtp-Source: AGRyM1vf26El1oUdu5cWklHC4JVaKVen83YoEFbMuYsS4w01ag9AE8Zw5D7yKYtnH2CD2WP0wmDypQ== X-Received: by 2002:a2e:8689:0:b0:25e:42:42e9 with SMTP id l9-20020a2e8689000000b0025e004242e9mr4470583lji.300.1658865552256; Tue, 26 Jul 2022 12:59:12 -0700 (PDT) Received: from sovereign (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id k15-20020a0565123d8f00b0048aab0259c0sm343835lfv.107.2022.07.26.12.59.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jul 2022 12:59:11 -0700 (PDT) Date: Tue, 26 Jul 2022 22:59:10 +0300 From: Dmitry Kozlyuk To: Don Wallwork Cc: "dev@dpdk.org" Subject: Re: [RFC] EAL: legacy memory fixed address translations Message-ID: <20220726225910.26159820@sovereign> In-Reply-To: <6aaa04d8-2ac5-ced6-ec25-d42bc52a3e2f@xsightlabs.com> References: <256b5409-ddaf-d7cc-00c1-273ca76dbf71@xsightlabs.com> <6aaa04d8-2ac5-ced6-ec25-d42bc52a3e2f@xsightlabs.com> X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Don, 2022-07-26 14:33 (UTC-0400), Don Wallwork: > This proposal describes a method for translating any huge page > address from virtual to physical or vice versa using simple > addition or subtraction of a single fixed value. This allows > devices to efficiently access arbitrary huge page memory, even > stack data when worker stacks are in huge pages. What is the use case and how much is the benefit? When drivers need to process a large number of memory blocks, these are typically packets in the form of mbufs, which already have IOVA attached, so there is no translation. Does translation of mbuf VA to PA with the proposed method show significant improvement over reading mbuf->iova? When drivers need to process a few IOVA-contiguous memory blocks, they can calculate VA-to-PA offsets in advance, amortizing translation cost. Hugepage stack falls within this category. > When legacy memory mode is used, it is possible to map a single > virtual memory region large enough to cover all huge pages. During > legacy hugepage init, each hugepage is mapped into that region. Legacy mode is called "legacy" with an intent to be deprecated :) There is initial allocation (-m) and --socket-limit in dynamic mode. When initial allocation is equal to the socket limit, it should be the same behavior as in legacy mode: the number of hugepages mapped is constant and cannot grow, so the feature seems applicable as well. > Once all pages have been mapped, any unused holes in that memory > region are unmapped. Who tracks these holes and prevents translation from their VA? Why the holes appear? > This feature is applicable when rte_eal_iova_mode() == RTE_IOVA_PA One can say it always works for RTE_IOVA_VA with VA-to-PA offset of 0. > and could be enabled either by default when the legacy memory EAL > option is given, or a new EAL option could be added to specifically > enable this feature. > > It may be desirable to set a capability bit when this feature is > enabled to allow drivers to behave differently depending on the > state of that flag. The feature requires, in IOVA-as-PA mode: 1) that hugepage mapping is static (legacy mode or "-m" == "--socket-limit"); 2) that EAL has succeeded to map all hugepages in one PA-continuous block. As userspace code, DPDK cannot guarantee 2). Because this mode breaks nothing and just makes translation more efficient, DPDK can always try to implement it and then report whether it has succeeded. Applications and drivers can decide what to do by querying this API.