From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id D8257374C for ; Tue, 7 May 2019 13:03:43 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 May 2019 04:03:42 -0700 X-ExtLoop1: 1 Received: from aburakov-mobl1.ger.corp.intel.com (HELO [10.251.90.106]) ([10.251.90.106]) by fmsmga004.fm.intel.com with ESMTP; 07 May 2019 04:03:41 -0700 To: Erik Gabriel Carrillo , rsanford@akamai.com, thomas@monjalon.net Cc: dev@dpdk.org References: <1556737217-24338-1-git-send-email-erik.g.carrillo@intel.com> <1556924082-22535-1-git-send-email-erik.g.carrillo@intel.com> From: "Burakov, Anatoly" Message-ID: <7baed0b9-432f-be86-5e39-68035bc309a4@intel.com> Date: Tue, 7 May 2019 12:03:40 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <1556924082-22535-1-git-send-email-erik.g.carrillo@intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v2] timer: fix resource leak in finalize X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 May 2019 11:03:44 -0000 On 03-May-19 11:54 PM, Erik Gabriel Carrillo wrote: > The finalize function should free the memzone created in the init > function, rather than freeing the allocation the memzone references, > otherwise a memzone descriptor can be leaked. > > Fixes: c0749f7096c7 ("timer: allow management in shared memory") > > Signed-off-by: Erik Gabriel Carrillo > --- > changes in v2: > - Handle scenario where primary process exits before secondaries such > that memzone is not freed early (Anatoly) > > lib/librte_timer/rte_timer.c | 20 +++++++++++++++++--- > 1 file changed, 17 insertions(+), 3 deletions(-) > > diff --git a/lib/librte_timer/rte_timer.c b/lib/librte_timer/rte_timer.c > index eb46009..4771287 100644 > --- a/lib/librte_timer/rte_timer.c > +++ b/lib/librte_timer/rte_timer.c > @@ -60,6 +60,8 @@ struct rte_timer_data { > }; > > #define RTE_MAX_DATA_ELS 64 > +static const struct rte_memzone *rte_timer_data_mz; > +static rte_atomic16_t *rte_timer_mz_refcnt; > static struct rte_timer_data *rte_timer_data_arr; > static const uint32_t default_data_id; > static uint32_t rte_timer_subsystem_initialized; > @@ -155,6 +157,7 @@ rte_timer_subsystem_init_v1905(void) > struct rte_timer_data *data; > int i, lcore_id; > static const char *mz_name = "rte_timer_mz"; > + size_t data_arr_size = RTE_MAX_DATA_ELS * sizeof(*rte_timer_data_arr); nitpicking, but... const? > > if (rte_timer_subsystem_initialized) > return -EALREADY; > @@ -164,10 +167,14 @@ rte_timer_subsystem_init_v1905(void) > if (mz == NULL) > return -EEXIST; > > + rte_timer_data_mz = mz; > rte_timer_data_arr = mz->addr; > + rte_timer_mz_refcnt = > + (void *)((char *)mz->addr + data_arr_size); > > rte_timer_data_arr[default_data_id].internal_flags |= > FL_ALLOCATED; > + rte_atomic16_inc(rte_timer_mz_refcnt); > > rte_timer_subsystem_initialized = 1; > > @@ -175,12 +182,15 @@ rte_timer_subsystem_init_v1905(void) > } > > mz = rte_memzone_reserve_aligned(mz_name, > - RTE_MAX_DATA_ELS * sizeof(*rte_timer_data_arr), > + data_arr_size + sizeof(*rte_timer_mz_refcnt), > SOCKET_ID_ANY, 0, RTE_CACHE_LINE_SIZE); > if (mz == NULL) > return -ENOMEM; > > + rte_timer_data_mz = mz; > rte_timer_data_arr = mz->addr; > + rte_timer_mz_refcnt = (void *)((char *)mz->addr + data_arr_size); > + rte_atomic16_init(rte_timer_mz_refcnt); > > for (i = 0; i < RTE_MAX_DATA_ELS; i++) { > data = &rte_timer_data_arr[i]; > @@ -193,6 +203,7 @@ rte_timer_subsystem_init_v1905(void) > } > > rte_timer_data_arr[default_data_id].internal_flags |= FL_ALLOCATED; > + rte_atomic16_inc(rte_timer_mz_refcnt); > > rte_timer_subsystem_initialized = 1; > > @@ -205,8 +216,11 @@ BIND_DEFAULT_SYMBOL(rte_timer_subsystem_init, _v1905, 19.05); > void __rte_experimental > rte_timer_subsystem_finalize(void) > { > - if (rte_timer_data_arr) > - rte_free(rte_timer_data_arr); > + if (!rte_timer_subsystem_initialized) > + return; > + > + if (rte_atomic16_dec_and_test(rte_timer_mz_refcnt)) > + rte_memzone_free(rte_timer_data_mz); I think there's a race here. You may get preempted after test but before free, where another secondary could initialize. As far as i know, we also support a case when secondary initializes after primary stops running. Let's even suppose that we allow secondary processes to initialize the timer subsystem by reserving memzone and checking rte_errno. You would still have a chance of two init/deinit conflicting, because there's a hole between memzone allocation and atomic increment. I don't think this race can be resolved in a safe way, so we might just have to settle for a memory leak. > > rte_timer_subsystem_initialized = 0; > } > -- Thanks, Anatoly From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 5E0ECA0096 for ; Tue, 7 May 2019 13:03:53 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C766D49DF; Tue, 7 May 2019 13:03:45 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id D8257374C for ; Tue, 7 May 2019 13:03:43 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 May 2019 04:03:42 -0700 X-ExtLoop1: 1 Received: from aburakov-mobl1.ger.corp.intel.com (HELO [10.251.90.106]) ([10.251.90.106]) by fmsmga004.fm.intel.com with ESMTP; 07 May 2019 04:03:41 -0700 To: Erik Gabriel Carrillo , rsanford@akamai.com, thomas@monjalon.net Cc: dev@dpdk.org References: <1556737217-24338-1-git-send-email-erik.g.carrillo@intel.com> <1556924082-22535-1-git-send-email-erik.g.carrillo@intel.com> From: "Burakov, Anatoly" Message-ID: <7baed0b9-432f-be86-5e39-68035bc309a4@intel.com> Date: Tue, 7 May 2019 12:03:40 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <1556924082-22535-1-git-send-email-erik.g.carrillo@intel.com> Content-Type: text/plain; charset="UTF-8"; format="flowed" Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v2] timer: fix resource leak in finalize X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190507110340.pL97gWyfy-3EAEuXBE2Ivk3Om0u20KVnC5qM3qwhMCk@z> On 03-May-19 11:54 PM, Erik Gabriel Carrillo wrote: > The finalize function should free the memzone created in the init > function, rather than freeing the allocation the memzone references, > otherwise a memzone descriptor can be leaked. > > Fixes: c0749f7096c7 ("timer: allow management in shared memory") > > Signed-off-by: Erik Gabriel Carrillo > --- > changes in v2: > - Handle scenario where primary process exits before secondaries such > that memzone is not freed early (Anatoly) > > lib/librte_timer/rte_timer.c | 20 +++++++++++++++++--- > 1 file changed, 17 insertions(+), 3 deletions(-) > > diff --git a/lib/librte_timer/rte_timer.c b/lib/librte_timer/rte_timer.c > index eb46009..4771287 100644 > --- a/lib/librte_timer/rte_timer.c > +++ b/lib/librte_timer/rte_timer.c > @@ -60,6 +60,8 @@ struct rte_timer_data { > }; > > #define RTE_MAX_DATA_ELS 64 > +static const struct rte_memzone *rte_timer_data_mz; > +static rte_atomic16_t *rte_timer_mz_refcnt; > static struct rte_timer_data *rte_timer_data_arr; > static const uint32_t default_data_id; > static uint32_t rte_timer_subsystem_initialized; > @@ -155,6 +157,7 @@ rte_timer_subsystem_init_v1905(void) > struct rte_timer_data *data; > int i, lcore_id; > static const char *mz_name = "rte_timer_mz"; > + size_t data_arr_size = RTE_MAX_DATA_ELS * sizeof(*rte_timer_data_arr); nitpicking, but... const? > > if (rte_timer_subsystem_initialized) > return -EALREADY; > @@ -164,10 +167,14 @@ rte_timer_subsystem_init_v1905(void) > if (mz == NULL) > return -EEXIST; > > + rte_timer_data_mz = mz; > rte_timer_data_arr = mz->addr; > + rte_timer_mz_refcnt = > + (void *)((char *)mz->addr + data_arr_size); > > rte_timer_data_arr[default_data_id].internal_flags |= > FL_ALLOCATED; > + rte_atomic16_inc(rte_timer_mz_refcnt); > > rte_timer_subsystem_initialized = 1; > > @@ -175,12 +182,15 @@ rte_timer_subsystem_init_v1905(void) > } > > mz = rte_memzone_reserve_aligned(mz_name, > - RTE_MAX_DATA_ELS * sizeof(*rte_timer_data_arr), > + data_arr_size + sizeof(*rte_timer_mz_refcnt), > SOCKET_ID_ANY, 0, RTE_CACHE_LINE_SIZE); > if (mz == NULL) > return -ENOMEM; > > + rte_timer_data_mz = mz; > rte_timer_data_arr = mz->addr; > + rte_timer_mz_refcnt = (void *)((char *)mz->addr + data_arr_size); > + rte_atomic16_init(rte_timer_mz_refcnt); > > for (i = 0; i < RTE_MAX_DATA_ELS; i++) { > data = &rte_timer_data_arr[i]; > @@ -193,6 +203,7 @@ rte_timer_subsystem_init_v1905(void) > } > > rte_timer_data_arr[default_data_id].internal_flags |= FL_ALLOCATED; > + rte_atomic16_inc(rte_timer_mz_refcnt); > > rte_timer_subsystem_initialized = 1; > > @@ -205,8 +216,11 @@ BIND_DEFAULT_SYMBOL(rte_timer_subsystem_init, _v1905, 19.05); > void __rte_experimental > rte_timer_subsystem_finalize(void) > { > - if (rte_timer_data_arr) > - rte_free(rte_timer_data_arr); > + if (!rte_timer_subsystem_initialized) > + return; > + > + if (rte_atomic16_dec_and_test(rte_timer_mz_refcnt)) > + rte_memzone_free(rte_timer_data_mz); I think there's a race here. You may get preempted after test but before free, where another secondary could initialize. As far as i know, we also support a case when secondary initializes after primary stops running. Let's even suppose that we allow secondary processes to initialize the timer subsystem by reserving memzone and checking rte_errno. You would still have a chance of two init/deinit conflicting, because there's a hole between memzone allocation and atomic increment. I don't think this race can be resolved in a safe way, so we might just have to settle for a memory leak. > > rte_timer_subsystem_initialized = 0; > } > -- Thanks, Anatoly