From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 28CD6A0577; Mon, 13 Apr 2020 11:45:58 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8A6561BE9D; Mon, 13 Apr 2020 11:45:57 +0200 (CEST) Received: from mail-il1-f195.google.com (mail-il1-f195.google.com [209.85.166.195]) by dpdk.org (Postfix) with ESMTP id F405B2A66 for ; Mon, 13 Apr 2020 11:45:55 +0200 (CEST) Received: by mail-il1-f195.google.com with SMTP id t10so785469ilg.9 for ; Mon, 13 Apr 2020 02:45:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=simmM1tYLeeU4MWUa74kYjYA0FIo42E3PYzAVcVQINk=; b=AMdfh6XWNdL1lW0/UeYQhu0lST/BGln9IzyDAtZWQ9ZJTyWKZzVUsq1KIK0D5H3okg c7dDsVn+MTIQ3V6F+HLcoleZr7WpGhv/XDhm8I/vFtd/2l7YUn9Qgvh/2OEZDdwEoORC 6V/aJOrGJHIEmaxS9qogPwKLZAKLFneeXDxnAeilBJB97G+A5zl09TWpDUhVqiKERf2v xfwQv8aPcwuYA0Ol7TObhR0UIc4ppmHwoGuePHUN42B2OCnhn13UosXgSgme5fkLrbRt Vq0oR+kcuCbnDeY0VWEj2uZJ7Eb/1/pAtufWy8sKZjiEb9N14rgAUqpiUuxv8NUlRqg4 PvMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=simmM1tYLeeU4MWUa74kYjYA0FIo42E3PYzAVcVQINk=; b=MoTcPR0PSe3jIEOgIURnQDeZgNdLKmZBgg1mFFh4VQ52taugtrAQuZhnPy/W8qNkEO Roa9WaCs768AQyunuqHhTfqi7zZvKASj/qI2a5ebP71PsWLH9DYDj9KtWsfTE2iPt9DH xKqydTvxdy6XYPGX24U+bItkVwyRsL3mKG723SIDzMW/cdyzKFwBntXc/RkywHphygQC H59blquhzrq+2GytsGvBTXgVsGeM0IqdNnlA5P3piHZrOMhd112wiVdS7aQoy0to49VW ihQXtMCTyg4A4l89HG2j8XW2B/TqoQw8vVpjPWeN3qTtycxat4czsEzRIO3t1r0emY3K HPwg== X-Gm-Message-State: AGi0Pubf9Cta75wjtrWzt3lZTiaOBdDQz+zzncBrQbqSJ3AVFiTthZg0 typoS2MFFku1cexIypL7+ujXpDup9yQWNqT57Yw= X-Google-Smtp-Source: APiQypL88gBVc9Yo8n4dJmUxUQPDgiRq5Nm+O5Y6JZ+KM7geybrVTlDna39f/+PAIaNscm6P1pFIPAi+3fRQPyAZH/w= X-Received: by 2002:a92:48cb:: with SMTP id j72mr16272983ilg.162.1586771155103; Mon, 13 Apr 2020 02:45:55 -0700 (PDT) MIME-Version: 1.0 References: <1586768952-10554-1-git-send-email-ricudis@niometrics.com> In-Reply-To: <1586768952-10554-1-git-send-email-ricudis@niometrics.com> From: Jerin Jacob Date: Mon, 13 Apr 2020 15:15:39 +0530 Message-ID: To: Christos Ricudis Cc: Anatoly Burakov , dpdk-dev Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH] eal: Place EAL thread stack in a reserved per-lcore memzone X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Mon, Apr 13, 2020 at 2:39 PM Christos Ricudis wrote: > > Reserve a per-lcore 4MB memzone and allocate thread stack of EAL threads there for better NUMA locality of stack-allocated variables It looks like a good idea to me. Some questions/feedback. 1) It is better to get the stack size from OS through pthread_attr_getstack() rather than DPDK defining it. 2) There is an element of security issue here as one can get the lcore stack using rte_memzone_lookup(). Why we need to use memzone, just rte_malloc_socket() is enough here. Right? This will avoid the security issue. 3) Need to handle EAL --no-huge case as well. > > Signed-off-by: Christos Ricudis > --- > lib/librte_eal/linux/eal.c | 20 ++++++++++++++++++++ > 1 file changed, 20 insertions(+) > > diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c > index 9530ee5..e047107 100644 > --- a/lib/librte_eal/linux/eal.c > +++ b/lib/librte_eal/linux/eal.c > @@ -68,6 +68,8 @@ > > #define KERNEL_IOMMU_GROUPS_PATH "/sys/kernel/iommu_groups" > > +#define THREAD_STACK_SIZE_DEFAULT (4ULL * 1024ULL * 1024ULL) > +#include > /* Allow the application to print its usage message too if set */ > static rte_usage_hook_t rte_application_usage_hook = NULL; > > @@ -1224,6 +1226,24 @@ static void rte_eal_init_alert(const char *msg) > > lcore_config[i].state = WAIT; > > + pthread_attr_t attr; > + pthread_attr_init(&attr); > + size_t thread_stack_size = THREAD_STACK_SIZE_DEFAULT; > + char thread_stack_name[64]; > + snprintf(thread_stack_name, sizeof thread_stack_name, "rte:lcore:%s:%d:threadstack", rte_eal_process_type() == RTE_PROC_PRIMARY ? "p" : "s", i); > + const struct rte_memzone *mz = rte_memzone_lookup(thread_stack_name); > + if (mz == NULL) { > + if ((mz = rte_memzone_reserve(thread_stack_name, thread_stack_size, lcore_config[i].socket_id, 0)) == NULL) { > + rte_panic("Cannot allocate memzone for thread stack"); > + } > + } > + void *thread_stack = mz->addr; > + > + if (pthread_attr_setstack(&attr, thread_stack, thread_stack_size) < 0) { > + rte_panic("Cannot set thread stack\n"); > + } > + RTE_LOG(DEBUG, EAL, "Thread stack for lcore %d on socket %d set to %p\n", i, lcore_config[i].socket_id, thread_stack); > + > /* create a thread for each lcore */ > ret = pthread_create(&lcore_config[i].thread_id, NULL, > eal_thread_loop, NULL); > -- > 1.8.3.1 >