From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BB118A0C48 for ; Tue, 20 Jul 2021 17:44:42 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 869054068B; Tue, 20 Jul 2021 17:44:42 +0200 (CEST) Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by mails.dpdk.org (Postfix) with ESMTP id 1CD4D40689 for ; Tue, 20 Jul 2021 17:44:42 +0200 (CEST) Received: by mail-pg1-f175.google.com with SMTP id i16so5326348pgi.9 for ; Tue, 20 Jul 2021 08:44:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=i5+GU9nXgfRSNPVUUeH4scVbbr6A8nmbmrn8INhikh0=; b=OFKuNHHwfYc+93BTjNqLjt4j0JxBbrEJBqKkkMZcWl3r4+M8Ah9Kxy/G/Qd3GBbCA8 c8EJP8Tt6dLHj/jt5aJChWqWkwScA1QHXo3JTlkm76LPa0IcyXI6VWgRFfuk6ACIoDlP QeQaTo8G2SpN7SjkhRYgS3Xg2YYEjz/nIhCEtdaQs1ZGZDlNpwy3uHSSxheijxyYG9o2 HOaH3ES6dsCfS7Zi6FtjHM+kCQD1geEwrtlL9LCjBau9FKP8uGyTDnKMUBZ82DcyYLQu mBANZd758aGPl3FzWfgPiXAHTyEovp4aG1EzB/LX4kcXgXG4GkK1ifefjJuomskR3a2U YGmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=i5+GU9nXgfRSNPVUUeH4scVbbr6A8nmbmrn8INhikh0=; b=bkWX43dHimRAZbvYn/n3C4e9GQ0sn8P+xf2ohmwH9c5IRvT/PyGeuPuJkhEQ29xD2O KaUDdLgsfVHz0poY1e6LYq0VQ/gjHnWxBuPcfeiE3nhfSqF3HMPL98NIbajCUWfN+nMP aFXtqRoebVsNy4ugRYhmYg6Z6bPuekqO6qDUFe5U3xkULc4q8szl060j+gc23rurx6dh VRmf7qnGFifhrpacZ0ghXwq9ONhOF0FA2NrtVEd5D7KXtlg1bVZoafR2yD2VaJbCfJWN K9KGSBPtERjYkZnpsAyNIOFkVKik7wd/IzBCCadZwdTR5x/QpJP+01SLTnO1QYxdbHmx k/IQ== X-Gm-Message-State: AOAM531rV1PsRsc82Nzq/0W/piUURjkyjFTW1qwqX3rsEkdhCV8PNV3x BUn7BuSdTgnTx4WkzyreY/M9xA== X-Google-Smtp-Source: ABdhPJwn4xjKQvAn4RpH2NlQvGV0rsG7bIhi5GIN8tLpFzdua8mCT/gfRNVkuE6CGoulVwHKAtVl4g== X-Received: by 2002:a63:4e04:: with SMTP id c4mr30821189pgb.294.1626795881263; Tue, 20 Jul 2021 08:44:41 -0700 (PDT) Received: from hermes.local (204-195-33-123.wavecable.com. [204.195.33.123]) by smtp.gmail.com with ESMTPSA id y17sm17231370pfp.147.2021.07.20.08.44.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 08:44:40 -0700 (PDT) Date: Tue, 20 Jul 2021 08:44:30 -0700 From: Stephen Hemminger To: Fengkai Sun Cc: users@dpdk.org Message-ID: <20210720084430.10dc648c@hermes.local> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-users] Secondary process cannot call functions on hugepages X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" On Tue, 20 Jul 2021 17:28:59 +0800 Fengkai Sun wrote: > Hi list, > > Recently I read about the multi-process support of dpdk and found the > feature of keeping the hugepage mappings between primary and secondary > processes very interesting. > Therefore, I made the following experiments: > > 1. I modify the glibc to make dlopen() map shared libraries on hugepages. > This is done by changing the underlying mapping strategy from mmap to pread. > 2. I allocate a continuous address by rte_malloc(), and instruct dlopen() > to map a shared library there, let's call it lib1.so. > 3. After dlopen() succeeds, I use dlsym() in the primary process to locate > the global variables and functions in lib1.so, and it turns out they are > correctly mapped and functions can be called without a problem. > 4. Because hugepage seems to get away with the effect of ASLR, I record the > address of those global variables and functions in lib1.so, and verify them > in the secondary process. > The secondary can access the global variables at the same address, but > when it tries to call the function, a segfault occurs. > 5. I try to use dlopen() with the same arguments as the primary process in > the secondary process, but it just gives a segfault. > > Unfortunately, gdb also gives a segfault when the program starts up, so I > cannot give some useful debug info. > > My question is, does dpdk permit functions to be loaded on hugepages and be > called by multiple functions? Though the two processes see the exact same > content on the hugepage, the secondary just cannot call the function on it. Sounds like no-exec bit. You might want to look into other libraries that already handle huge pages and libraries such as hugetlbfs