From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 89CE6A050B for ; Fri, 8 Apr 2022 15:26:32 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 101534067E; Fri, 8 Apr 2022 15:26:32 +0200 (CEST) Received: from mail-lf1-f47.google.com (mail-lf1-f47.google.com [209.85.167.47]) by mails.dpdk.org (Postfix) with ESMTP id 78B414003F for ; Fri, 8 Apr 2022 15:26:31 +0200 (CEST) Received: by mail-lf1-f47.google.com with SMTP id k21so15088343lfe.4 for ; Fri, 08 Apr 2022 06:26:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2lSFm4HRS0E9Yx7oJE++EQXTgZ116mocUI3OWIVxdm0=; b=Z+pcPtLOZb49Ei+welvG40DX4ZmV1gnDbuIp91XPPyJ3W7HyQead5TPw5Qov81RZXW DFEuySrDVfykDpzA5F8SK5cgYX+fNoXeDWYkhRBgt8mo65zUA9HCaIhuOA4LL38S6whC Ye2KDuywjsDQ1Ow5Zik4/ZVpS8GL9dbRWWFILOE8hrLrwaoNsXttgCG8bkrzGATPExGd EBCmKz9E/tE8VeAhFK8VBq2au0Z01fWqk/mBAG0dEW3UwA8Yl96f0tDL8WHTgjCkLf8+ AD1REevRjtZjwgNeS7jEKnO0+FA/tEQNEnvo/USYgwUOmViOU+OfQQyDcjpJJK6ROIKB tzhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2lSFm4HRS0E9Yx7oJE++EQXTgZ116mocUI3OWIVxdm0=; b=3qUkJQoQZGKp+6rAp/yVtQrSp56CouLDOL4Aupc9w5mGJz6sUyrWcLRqpMdqkThwRg 2JSjd3q/h1Wr8XzOzXPZVFWb/cGuYof7eGmSzSVvPHbGsJZ41CoU+W91dLOnoLZjYfvU /87xrFbHDB5qbj8ELoFKX8GRD+RgWzlsYd/pz+V6BKJAL/tEvxeZ0Hv4q4RBivlgLNL6 cbvxRBsB1ZsP7H9MgZnDI34aTYcITY+Swe6SG7k5dTioG+1aU/JqsYOOKxZ5jclwYJN1 JlwDStoqiKZJ5C4M8+bCcjFJKsUDR9oVGdO5rJju0L5sjXYbiCk+NDCrOX/uNAxf1FsG pUQA== X-Gm-Message-State: AOAM5312/anuO46Qo6pGibORnF7NTbICzudLOl8crqU/8hnzwMY65rF4 l4YXFxlEAknjrJM7LHnJAxn/MBSjimg= X-Google-Smtp-Source: ABdhPJxzHvLXAa7UqU6qXqyZCPmrnaBHr63p0Ki1VuRBKWkvDdWQkpgWnjynbRdxJYwJLPBeVTB+KQ== X-Received: by 2002:a05:6512:33d0:b0:44a:72b2:cd36 with SMTP id d16-20020a05651233d000b0044a72b2cd36mr12992196lfg.113.1649424390849; Fri, 08 Apr 2022 06:26:30 -0700 (PDT) Received: from sovereign (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id w27-20020ac2599b000000b004481e254f08sm2456197lfn.240.2022.04.08.06.26.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Apr 2022 06:26:30 -0700 (PDT) Date: Fri, 8 Apr 2022 16:26:29 +0300 From: Dmitry Kozlyuk To: Antonio Di Bacco Cc: users@dpdk.org Subject: Re: Shared memory between two primary DPDK processes Message-ID: <20220408162629.372dfd0d@sovereign> In-Reply-To: References: X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org 2022-04-08 14:31 (UTC+0200), Antonio Di Bacco: > I know that it is possible to share memory between a primary and secondary > process using rte_memzone_reserve_aligned to allocate memory in primary > that is "seen" also by the secondary. If we have two primary processes > (started with different file-prefix) the same approach is not feasible. I > wonder how to share a chunk of memory hosted on a hugepage between two > primaries. > > Regards. Hi Antonio, Correction: all hugepages allocated by DPDK are shared between primary and secondary processes, not only memzones. I assume we're talking about processes within one host, because your previous similar question was about sharing memory between hosts (as we have discussed offline), which is out of scope for DPDK. As for the question directly, you need to map the same part of the same file in the second primary as the hugepage is mapped from in the first primary. I don't recommend to work with file paths, because their management is not straightforward (--single-file-segments, for one) and is undocumented. There is a way to share DPDK memory segment file descriptors. Although public, this DPDK API is dangerous in the sense that you must clearly understand what you're doing and how DPDK works. Hence the question: what is the task you need this sharing for? Maybe there is a simpler way. 1. In the first primary: mz = rte_memzone_reserve() ms = rte_mem_virt2memseg(mz->addr) fd = rte_memseg_get_fd(ms) offset = rte_memseg_get_fd_offset(ms) 2. Use Unix domain sockets with SCM_RIGHTS to send "fd" and "offset" to the second primary. 3. In the second primary, after receiving "fd" and "offset": flags = MAP_SHARED | MAP_HUGE | (30 << MAP_HUGE_SHIFT) addr = mmap(fd, offset, flags) Note that "mz" may consist of multiple "ms" depending on the sizes of the zone and hugepages, and on the zone alignment. Also "addr" may (and probably will) differ from "mz->addr". It is possible to pass "mz->addr" and try to force it, like DPDK does for primary/secondary.