From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 696E4A0350; Tue, 23 Jun 2020 09:49:35 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3F1651D576; Tue, 23 Jun 2020 09:49:35 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by dpdk.org (Postfix) with ESMTP id 1A5B81D555 for ; Tue, 23 Jun 2020 09:49:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592898573; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=gMISvoqz4t47o+FcCe0OsuOrud71J95cK3aTmHk1vIE=; b=LyJ+B33zc1Z+VyNujGM3AM8/ejGRw8BjciT7+/2ycb3kZa6RujpA1/Bs62Sne48+BwVX6x 5mjOnV6RcA+b6/03W2wj2ZZncmAzz7MkQFCDcPpT2LJ5Ps6aZIP1aN8WjwyaKNr7Lib6dG 6At8Qbx4USXvNIfTURhbDcCBcwrJG24= Received: from mail-vk1-f200.google.com (mail-vk1-f200.google.com [209.85.221.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-440-T8jUDcooMhaUjT5-JO9tzw-1; Tue, 23 Jun 2020 03:49:29 -0400 X-MC-Unique: T8jUDcooMhaUjT5-JO9tzw-1 Received: by mail-vk1-f200.google.com with SMTP id x187so5769066vke.8 for ; Tue, 23 Jun 2020 00:49:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=gMISvoqz4t47o+FcCe0OsuOrud71J95cK3aTmHk1vIE=; b=cnozifmxMgpNPuDx/JX0Efx82YiFcOdScJebZYcuL6A29SjfMH5kRWa2y8mjryMZTC uyGevEv61gYcwdq2Poq5P1LhP8h5oRlob6LgZH1GAF6+lwhfjfAkfQhAM7lyLfk72jjo EiSpXUIzvUjWKXnUJksdUCX7gcXWAZcOlZwYvRrz0Ejfe/dRobBy6tpRWU7213kvj+ky /SU99RCUlsJLKiq2KYxAJqxeZDF0gS23sXtcx2lew9y1SIg+G+N+24A0slB5qnmbP1Z1 9OccEAWo2claJT+HkOcnfye7jW3E0t484oOVOL0V9SNdeRso81w9sIeemCR0sc49WzFq a3Jw== X-Gm-Message-State: AOAM531BoY7HgZky93UhAKzct1/p0q1g+I8/99eVIiJfgV5IQ3Hj7u7B +9T6ajNMmg/igFZQ/JPKsYmSuAzyHhzgJv/nQteBjZ8AdQmS5kQqm5S07pubAYxiu7IPko8H/u5 ybw/k73AmW7HG1uM6GXE= X-Received: by 2002:a1f:d986:: with SMTP id q128mr8917792vkg.56.1592898569322; Tue, 23 Jun 2020 00:49:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJytwqvmW3iiY8W4L48vnb1VNgEFIZk/OKj/LSgznvhkoSw6PBMPe2BL+BjVaHXS+q1uIKqUtfkuS4Zyoi9gJR0= X-Received: by 2002:a1f:d986:: with SMTP id q128mr8917780vkg.56.1592898569014; Tue, 23 Jun 2020 00:49:29 -0700 (PDT) MIME-Version: 1.0 References: <20200610144506.30505-1-david.marchand@redhat.com> <20200622132531.21857-1-david.marchand@redhat.com> <20200622132531.21857-7-david.marchand@redhat.com> In-Reply-To: From: David Marchand Date: Tue, 23 Jun 2020 09:49:18 +0200 Message-ID: To: "Ananyev, Konstantin" Cc: "dev@dpdk.org" , "jerinjacobk@gmail.com" , "Richardson, Bruce" , "mdr@ashroe.eu" , "ktraynor@redhat.com" , "Stokes, Ian" , "i.maximets@ovn.org" , Thomas Monjalon , "Mcnamara, John" , "Kovacevic, Marko" , "Burakov, Anatoly" , Olivier Matz , Andrew Rybchenko , Neil Horman X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH v3 6/9] eal: register non-EAL threads as lcores X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hello Konstantin, On Mon, Jun 22, 2020 at 5:49 PM Ananyev, Konstantin wrote: > > diff --git a/lib/librte_eal/common/eal_common_lcore.c b/lib/librte_eal/common/eal_common_lcore.c > > index 86d32a3dd7..7db05428e7 100644 > > --- a/lib/librte_eal/common/eal_common_lcore.c > > +++ b/lib/librte_eal/common/eal_common_lcore.c > > @@ -220,3 +221,38 @@ rte_socket_id_by_idx(unsigned int idx) > > } > > return config->numa_nodes[idx]; > > } > > + > > +static rte_spinlock_t lcore_lock = RTE_SPINLOCK_INITIALIZER; > > + > > +unsigned int > > +eal_lcore_non_eal_allocate(void) > > +{ > > + struct rte_config *cfg = rte_eal_get_configuration(); > > + unsigned int lcore_id; > > + > > + rte_spinlock_lock(&lcore_lock); > > I think it will break current DPDK MP modes. > The problem here - rte_config (and lcore_role[]) is in shared memory, > while the lock is local. > Simplest way probably to move lcore_lock to rte_config. Even before this series, MP has no protection on lcore placing between primary and secondary processes. Personally, I have no use for DPDK MP and marking MP as not supporting this new feature is tempting for a first phase. If this is a strong requirement, I can look at it in a second phase. What do you think? > > > + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { > > + if (cfg->lcore_role[lcore_id] != ROLE_OFF) > > + continue; > > + cfg->lcore_role[lcore_id] = ROLE_NON_EAL; > > + cfg->lcore_count++; > > + break; > > + } > > + if (lcore_id == RTE_MAX_LCORE) > > + RTE_LOG(DEBUG, EAL, "No lcore available.\n"); > > + rte_spinlock_unlock(&lcore_lock); > > + return lcore_id; > > +} > > + > > +void > > +eal_lcore_non_eal_release(unsigned int lcore_id) > > +{ > > + struct rte_config *cfg = rte_eal_get_configuration(); > > + > > + rte_spinlock_lock(&lcore_lock); > > + if (cfg->lcore_role[lcore_id] == ROLE_NON_EAL) { > > + cfg->lcore_role[lcore_id] = ROLE_OFF; > > + cfg->lcore_count--; > > + } > > + rte_spinlock_unlock(&lcore_lock); > > +} > > diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c > > index a7ae0691bf..1cbddc4b5b 100644 > > --- a/lib/librte_eal/common/eal_common_thread.c > > +++ b/lib/librte_eal/common/eal_common_thread.c > > @@ -236,3 +236,36 @@ rte_ctrl_thread_create(pthread_t *thread, const char *name, > > pthread_join(*thread, NULL); > > return -ret; > > } > > + > > +void > > +rte_thread_register(void) > > +{ > > + unsigned int lcore_id; > > + rte_cpuset_t cpuset; > > + > > + /* EAL init flushes all lcores, we can't register before. */ > > + assert(internal_config.init_complete == 1); > > + if (pthread_getaffinity_np(pthread_self(), sizeof(cpuset), > > + &cpuset) != 0) > > + CPU_ZERO(&cpuset); > > + lcore_id = eal_lcore_non_eal_allocate(); > > + if (lcore_id >= RTE_MAX_LCORE) > > + lcore_id = LCORE_ID_ANY; > > + rte_thread_init(lcore_id, &cpuset); > > So we just setting affinity to the same value, right? > Not a big deal, but might be easier to allow rte_thread_init() > to accept cpuset==NULL (and just don't change thread affinity in that case) rte_thread_init does not change the thread cpu affinity, it handles per thread (TLS included) variables initialization. So do you mean accepting cpuset == NULL and do the getaffinity in this case? rte_thread_init is EAL private for now. That saves us some code in this function, but we will call with a != NULL cpuset in all other EAL code. -- David Marchand