[dpdk-dev] [PATCH] cryptodev: free memzone when releasing cryptodev

[dpdk-dev] [PATCH] cryptodev: free memzone when releasing cryptodev

[Junxiao Shi]

When a cryptodev is created in a primary process,
rte_cryptodev_data_alloc reserves a memzone.
However, this memzone was not released when the cryptodev
is uninitialized. After that, new cryptodev cannot be
created due to memzone name conflict.

This commit frees the memzone when a cryptodev is
uninitialized, fixing this bug. This approach is chosen
instead of keeping and reusing the old memzone, because
the new cryptodev could belong to a different NUMA socket.

Also, rte_cryptodev_data pointer is now properly recorded
in cryptodev_globals.data array.

Bugzilla ID: 105

Signed-off-by: Junxiao Shi <git@mail1.yoursunny.com>
---
 lib/librte_cryptodev/rte_cryptodev.c | 44 +++++++++++++++++++++++++++++++-----
 1 file changed, 38 insertions(+), 6 deletions(-)

diff --git a/lib/librte_cryptodev/rte_cryptodev.c b/lib/librte_cryptodev/rte_cryptodev.c
index 00c2cf4..666dfea 100644
--- a/lib/librte_cryptodev/rte_cryptodev.c
+++ b/lib/librte_cryptodev/rte_cryptodev.c
@@ -653,6 +653,31 @@ rte_cryptodev_data_alloc(uint8_t dev_id, struct rte_cryptodev_data **data,
 	return 0;
 }
 
+static inline int
+rte_cryptodev_data_free(uint8_t dev_id, struct rte_cryptodev_data **data)
+{
+	char mz_name[RTE_CRYPTODEV_NAME_MAX_LEN];
+	const struct rte_memzone *mz;
+	int n;
+
+	/* generate memzone name */
+	n = snprintf(mz_name, sizeof(mz_name), "rte_cryptodev_data_%u", dev_id);
+	if (n >= (int)sizeof(mz_name))
+		return -EINVAL;
+
+	mz = rte_memzone_lookup(mz_name);
+	if (mz == NULL)
+		return -ENOMEM;
+
+	RTE_ASSERT(*data == mz->addr);
+	*data = NULL;
+
+	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+		return rte_memzone_free(mz);
+
+	return 0;
+}
+
 static uint8_t
 rte_cryptodev_find_free_device_index(void)
 {
@@ -687,16 +712,16 @@ rte_cryptodev_pmd_allocate(const char *name, int socket_id)
 	cryptodev = rte_cryptodev_pmd_get_dev(dev_id);
 
 	if (cryptodev->data == NULL) {
-		struct rte_cryptodev_data *cryptodev_data =
-				cryptodev_globals.data[dev_id];
+		struct rte_cryptodev_data **cryptodev_data =
+				&cryptodev_globals.data[dev_id];
 
-		int retval = rte_cryptodev_data_alloc(dev_id, &cryptodev_data,
+		int retval = rte_cryptodev_data_alloc(dev_id, cryptodev_data,
 				socket_id);
 
-		if (retval < 0 || cryptodev_data == NULL)
+		if (retval < 0 || *cryptodev_data == NULL)
 			return NULL;
 
-		cryptodev->data = cryptodev_data;
+		cryptodev->data = *cryptodev_data;
 
 		strlcpy(cryptodev->data->name, name,
 			RTE_CRYPTODEV_NAME_MAX_LEN);
@@ -724,13 +749,20 @@ rte_cryptodev_pmd_release_device(struct rte_cryptodev *cryptodev)
 	if (cryptodev == NULL)
 		return -EINVAL;
 
+	uint8_t dev_id = cryptodev->data->dev_id;
+
 	/* Close device only if device operations have been set */
 	if (cryptodev->dev_ops) {
-		ret = rte_cryptodev_close(cryptodev->data->dev_id);
+		ret = rte_cryptodev_close(dev_id);
 		if (ret < 0)
 			return ret;
 	}
 
+	struct rte_cryptodev_data **cryptodev_data = &cryptodev_globals.data[dev_id];
+	ret = rte_cryptodev_data_free(dev_id, cryptodev_data);
+	if (ret < 0)
+		return ret;
+
 	cryptodev->attached = RTE_CRYPTODEV_DETACHED;
 	cryptodev_globals.nb_devs--;
 	return 0;
-- 
2.7.4

Re: [dpdk-dev] [PATCH] cryptodev: free memzone when releasing cryptodev

[Akhil Goyal]

> 
> When a cryptodev is created in a primary process,
> rte_cryptodev_data_alloc reserves a memzone.
> However, this memzone was not released when the cryptodev
> is uninitialized. After that, new cryptodev cannot be
> created due to memzone name conflict.
> 
> This commit frees the memzone when a cryptodev is
> uninitialized, fixing this bug. This approach is chosen
> instead of keeping and reusing the old memzone, because
> the new cryptodev could belong to a different NUMA socket.
> 
> Also, rte_cryptodev_data pointer is now properly recorded
> in cryptodev_globals.data array.
> 
> Bugzilla ID: 105
> 
> Signed-off-by: Junxiao Shi <git@mail1.yoursunny.com>
> ---
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

Re: [dpdk-dev] [EXT] [PATCH] cryptodev: free memzone when releasing cryptodev

[Anoob Joseph]

Hi,

Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Junxiao Shi
> Sent: Thursday, May 30, 2019 10:38 PM
> To: dev@dpdk.org
> Subject: [EXT] [dpdk-dev] [PATCH] cryptodev: free memzone when releasing
> cryptodev
> 
> External Email
> 
> ----------------------------------------------------------------------
> When a cryptodev is created in a primary process, rte_cryptodev_data_alloc
> reserves a memzone.
> However, this memzone was not released when the cryptodev is uninitialized.
> After that, new cryptodev cannot be created due to memzone name conflict.
> 
> This commit frees the memzone when a cryptodev is uninitialized, fixing this
> bug. This approach is chosen instead of keeping and reusing the old memzone,
> because the new cryptodev could belong to a different NUMA socket.
> 
> Also, rte_cryptodev_data pointer is now properly recorded in
> cryptodev_globals.data array.
> 
> Bugzilla ID: 105
> 
> Signed-off-by: Junxiao Shi <git@mail1.yoursunny.com>
> ---
>  lib/librte_cryptodev/rte_cryptodev.c | 44
> +++++++++++++++++++++++++++++++-----
>  1 file changed, 38 insertions(+), 6 deletions(-)
> 
> diff --git a/lib/librte_cryptodev/rte_cryptodev.c
> b/lib/librte_cryptodev/rte_cryptodev.c
> index 00c2cf4..666dfea 100644
> --- a/lib/librte_cryptodev/rte_cryptodev.c
> +++ b/lib/librte_cryptodev/rte_cryptodev.c
> @@ -653,6 +653,31 @@ rte_cryptodev_data_alloc(uint8_t dev_id, struct
> rte_cryptodev_data **data,
>  	return 0;
>  }
> 
> +static inline int
> +rte_cryptodev_data_free(uint8_t dev_id, struct rte_cryptodev_data
> +**data) {
> +	char mz_name[RTE_CRYPTODEV_NAME_MAX_LEN];
> +	const struct rte_memzone *mz;
> +	int n;
> +
> +	/* generate memzone name */
> +	n = snprintf(mz_name, sizeof(mz_name), "rte_cryptodev_data_%u",
> dev_id);
> +	if (n >= (int)sizeof(mz_name))
> +		return -EINVAL;

[Anoob] Is the above check needed?

> +
> +	mz = rte_memzone_lookup(mz_name);
> +	if (mz == NULL)
> +		return -ENOMEM;

[Anoob] Is the return value correct? Shouldn't it be -EINVAL?

@Akhil, thoughts?

> +
> +	RTE_ASSERT(*data == mz->addr);
> +	*data = NULL;
> +
> +	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
> +		return rte_memzone_free(mz);
> +
> +	return 0;
> +}
> +
>  static uint8_t
>  rte_cryptodev_find_free_device_index(void)
>  {
> @@ -687,16 +712,16 @@ rte_cryptodev_pmd_allocate(const char *name, int
> socket_id)
>  	cryptodev = rte_cryptodev_pmd_get_dev(dev_id);
> 
>  	if (cryptodev->data == NULL) {
> -		struct rte_cryptodev_data *cryptodev_data =
> -				cryptodev_globals.data[dev_id];
> +		struct rte_cryptodev_data **cryptodev_data =
> +				&cryptodev_globals.data[dev_id];
> 
> -		int retval = rte_cryptodev_data_alloc(dev_id, &cryptodev_data,
> +		int retval = rte_cryptodev_data_alloc(dev_id, cryptodev_data,
>  				socket_id);
> 
> -		if (retval < 0 || cryptodev_data == NULL)
> +		if (retval < 0 || *cryptodev_data == NULL)
>  			return NULL;
> 
> -		cryptodev->data = cryptodev_data;
> +		cryptodev->data = *cryptodev_data;
> 
>  		strlcpy(cryptodev->data->name, name,
>  			RTE_CRYPTODEV_NAME_MAX_LEN);
> @@ -724,13 +749,20 @@ rte_cryptodev_pmd_release_device(struct
> rte_cryptodev *cryptodev)
>  	if (cryptodev == NULL)
>  		return -EINVAL;
> 
> +	uint8_t dev_id = cryptodev->data->dev_id;
> +

[Anoob] Variables need to be declared at the start of the function.
https://doc.dpdk.org/guides/contributing/coding_style.html

>  	/* Close device only if device operations have been set */
>  	if (cryptodev->dev_ops) {
> -		ret = rte_cryptodev_close(cryptodev->data->dev_id);
> +		ret = rte_cryptodev_close(dev_id);
>  		if (ret < 0)
>  			return ret;
>  	}
> 
> +	struct rte_cryptodev_data **cryptodev_data =
> &cryptodev_globals.data[dev_id];

[Anoob] Same comment as above

> +	ret = rte_cryptodev_data_free(dev_id, cryptodev_data);
> +	if (ret < 0)
> +		return ret;
> +
>  	cryptodev->attached = RTE_CRYPTODEV_DETACHED;
>  	cryptodev_globals.nb_devs--;
>  	return 0;
> --
> 2.7.4

Re: [dpdk-dev] [EXT] [PATCH] cryptodev: free memzone when releasing cryptodev

[Akhil Goyal]

Hi Anoob,
> >
> > ----------------------------------------------------------------------
> > When a cryptodev is created in a primary process, rte_cryptodev_data_alloc
> > reserves a memzone.
> > However, this memzone was not released when the cryptodev is uninitialized.
> > After that, new cryptodev cannot be created due to memzone name conflict.
> >
> > This commit frees the memzone when a cryptodev is uninitialized, fixing this
> > bug. This approach is chosen instead of keeping and reusing the old memzone,
> > because the new cryptodev could belong to a different NUMA socket.
> >
> > Also, rte_cryptodev_data pointer is now properly recorded in
> > cryptodev_globals.data array.
> >
> > Bugzilla ID: 105
> >
> > Signed-off-by: Junxiao Shi <git@mail1.yoursunny.com>
> > ---
> >  lib/librte_cryptodev/rte_cryptodev.c | 44
> > +++++++++++++++++++++++++++++++-----
> >  1 file changed, 38 insertions(+), 6 deletions(-)
> >
> > diff --git a/lib/librte_cryptodev/rte_cryptodev.c
> > b/lib/librte_cryptodev/rte_cryptodev.c
> > index 00c2cf4..666dfea 100644
> > --- a/lib/librte_cryptodev/rte_cryptodev.c
> > +++ b/lib/librte_cryptodev/rte_cryptodev.c
> > @@ -653,6 +653,31 @@ rte_cryptodev_data_alloc(uint8_t dev_id, struct
> > rte_cryptodev_data **data,
> >  	return 0;
> >  }
> >
> > +static inline int
> > +rte_cryptodev_data_free(uint8_t dev_id, struct rte_cryptodev_data
> > +**data) {
> > +	char mz_name[RTE_CRYPTODEV_NAME_MAX_LEN];
> > +	const struct rte_memzone *mz;
> > +	int n;
> > +
> > +	/* generate memzone name */
> > +	n = snprintf(mz_name, sizeof(mz_name), "rte_cryptodev_data_%u",
> > dev_id);
> > +	if (n >= (int)sizeof(mz_name))
> > +		return -EINVAL;
> 
> [Anoob] Is the above check needed?
I believe this being used while creating the memzone, so same logic is used while freeing it.
Just to be safe.

> 
> > +
> > +	mz = rte_memzone_lookup(mz_name);
> > +	if (mz == NULL)
> > +		return -ENOMEM;
> 
> [Anoob] Is the return value correct? Shouldn't it be -EINVAL?
> 
> @Akhil, thoughts?


I believe ENOMEM is correct, as there is no memory associated with the cryptodev_data.

> 
> > +
> > +	RTE_ASSERT(*data == mz->addr);
> > +	*data = NULL;
> > +
> > +	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
> > +		return rte_memzone_free(mz);
> > +
> > +	return 0;
> > +}
> > +
> >  static uint8_t
> >  rte_cryptodev_find_free_device_index(void)
> >  {
> > @@ -687,16 +712,16 @@ rte_cryptodev_pmd_allocate(const char *name, int
> > socket_id)
> >  	cryptodev = rte_cryptodev_pmd_get_dev(dev_id);
> >
> >  	if (cryptodev->data == NULL) {
> > -		struct rte_cryptodev_data *cryptodev_data =
> > -				cryptodev_globals.data[dev_id];
> > +		struct rte_cryptodev_data **cryptodev_data =
> > +				&cryptodev_globals.data[dev_id];
> >
> > -		int retval = rte_cryptodev_data_alloc(dev_id, &cryptodev_data,
> > +		int retval = rte_cryptodev_data_alloc(dev_id, cryptodev_data,
> >  				socket_id);
> >
> > -		if (retval < 0 || cryptodev_data == NULL)
> > +		if (retval < 0 || *cryptodev_data == NULL)
> >  			return NULL;
> >
> > -		cryptodev->data = cryptodev_data;
> > +		cryptodev->data = *cryptodev_data;
> >
> >  		strlcpy(cryptodev->data->name, name,
> >  			RTE_CRYPTODEV_NAME_MAX_LEN);
> > @@ -724,13 +749,20 @@ rte_cryptodev_pmd_release_device(struct
> > rte_cryptodev *cryptodev)
> >  	if (cryptodev == NULL)
> >  		return -EINVAL;
> >
> > +	uint8_t dev_id = cryptodev->data->dev_id;
> > +
> 
> [Anoob] Variables need to be declared at the start of the function.
> https://doc.dpdk.org/guides/contributing/coding_style.html
> 
> >  	/* Close device only if device operations have been set */
> >  	if (cryptodev->dev_ops) {
> > -		ret = rte_cryptodev_close(cryptodev->data->dev_id);
> > +		ret = rte_cryptodev_close(dev_id);
> >  		if (ret < 0)
> >  			return ret;
> >  	}
> >
> > +	struct rte_cryptodev_data **cryptodev_data =
> > &cryptodev_globals.data[dev_id];
> 
> [Anoob] Same comment as above
> 
> > +	ret = rte_cryptodev_data_free(dev_id, cryptodev_data);
> > +	if (ret < 0)
> > +		return ret;
> > +
> >  	cryptodev->attached = RTE_CRYPTODEV_DETACHED;
> >  	cryptodev_globals.nb_devs--;
> >  	return 0;
> > --
> > 2.7.4

Re: [dpdk-dev] [EXT] [PATCH] cryptodev: free memzone when releasing cryptodev

[Anoob Joseph]

Hi Akhil,

Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: Akhil Goyal <akhil.goyal@nxp.com>
> Sent: Friday, June 28, 2019 11:45 AM
> To: Anoob Joseph <anoobj@marvell.com>; Junxiao Shi
> <git@mail1.yoursunny.com>; dev@dpdk.org
> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> Subject: RE: [EXT] [dpdk-dev] [PATCH] cryptodev: free memzone when
> releasing cryptodev
> 
> Hi Anoob,
> > >
> > > --------------------------------------------------------------------
> > > -- When a cryptodev is created in a primary process,
> > > rte_cryptodev_data_alloc reserves a memzone.
> > > However, this memzone was not released when the cryptodev is
> uninitialized.
> > > After that, new cryptodev cannot be created due to memzone name
> conflict.
> > >
> > > This commit frees the memzone when a cryptodev is uninitialized,
> > > fixing this bug. This approach is chosen instead of keeping and
> > > reusing the old memzone, because the new cryptodev could belong to a
> different NUMA socket.
> > >
> > > Also, rte_cryptodev_data pointer is now properly recorded in
> > > cryptodev_globals.data array.
> > >
> > > Bugzilla ID: 105
> > >
> > > Signed-off-by: Junxiao Shi <git@mail1.yoursunny.com>
> > > ---
> > >  lib/librte_cryptodev/rte_cryptodev.c | 44
> > > +++++++++++++++++++++++++++++++-----
> > >  1 file changed, 38 insertions(+), 6 deletions(-)
> > >
> > > diff --git a/lib/librte_cryptodev/rte_cryptodev.c
> > > b/lib/librte_cryptodev/rte_cryptodev.c
> > > index 00c2cf4..666dfea 100644
> > > --- a/lib/librte_cryptodev/rte_cryptodev.c
> > > +++ b/lib/librte_cryptodev/rte_cryptodev.c
> > > @@ -653,6 +653,31 @@ rte_cryptodev_data_alloc(uint8_t dev_id, struct
> > > rte_cryptodev_data **data,
> > >  	return 0;
> > >  }
> > >
> > > +static inline int
> > > +rte_cryptodev_data_free(uint8_t dev_id, struct rte_cryptodev_data
> > > +**data) {
> > > +	char mz_name[RTE_CRYPTODEV_NAME_MAX_LEN];

[Anoob] Shouldn't we use RTE_MEMZONE_NAMESIZE instead? I guess this is also coming from the existing code in rte_cryptodev_data_alloc(). May be we should fix that as well?
 
> > > +	const struct rte_memzone *mz;
> > > +	int n;
> > > +
> > > +	/* generate memzone name */
> > > +	n = snprintf(mz_name, sizeof(mz_name),
> "rte_cryptodev_data_%u",
> > > dev_id);
> > > +	if (n >= (int)sizeof(mz_name))
> > > +		return -EINVAL;
> >
> > [Anoob] Is the above check needed?
> I believe this being used while creating the memzone, so same logic is used
> while freeing it.
> Just to be safe.
> 

[Anoob] Thinking bit more, it seems like we are trying to capture a situation when the name is getting truncated because of insufficient buffer space. So it is safe to have I guess. But even in that case, 'n' will not be greater than the "size" field passed (which happens to be sizeof(mz_name) in our case).

My opinion is '==' might make more sense. But I leave that to your judgement. 
 
> >
> > > +
> > > +	mz = rte_memzone_lookup(mz_name);
> > > +	if (mz == NULL)
> > > +		return -ENOMEM;
> >
> > [Anoob] Is the return value correct? Shouldn't it be -EINVAL?
> >
> > @Akhil, thoughts?
> 
> 
> I believe ENOMEM is correct, as there is no memory associated with the
> cryptodev_data.

[Anoob] Agreed.

> 
> >
> > > +
> > > +	RTE_ASSERT(*data == mz->addr);
> > > +	*data = NULL;
> > > +
> > > +	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
> > > +		return rte_memzone_free(mz);
> > > +
> > > +	return 0;
> > > +}
> > > +
> > >  static uint8_t
> > >  rte_cryptodev_find_free_device_index(void)
> > >  {
> > > @@ -687,16 +712,16 @@ rte_cryptodev_pmd_allocate(const char
> *name,
> > > int
> > > socket_id)
> > >  	cryptodev = rte_cryptodev_pmd_get_dev(dev_id);
> > >
> > >  	if (cryptodev->data == NULL) {
> > > -		struct rte_cryptodev_data *cryptodev_data =
> > > -				cryptodev_globals.data[dev_id];
> > > +		struct rte_cryptodev_data **cryptodev_data =
> > > +				&cryptodev_globals.data[dev_id];
> > >
> > > -		int retval = rte_cryptodev_data_alloc(dev_id,
> &cryptodev_data,
> > > +		int retval = rte_cryptodev_data_alloc(dev_id,
> cryptodev_data,
> > >  				socket_id);
> > >
> > > -		if (retval < 0 || cryptodev_data == NULL)
> > > +		if (retval < 0 || *cryptodev_data == NULL)
> > >  			return NULL;
> > >
> > > -		cryptodev->data = cryptodev_data;
> > > +		cryptodev->data = *cryptodev_data;
> > >
> > >  		strlcpy(cryptodev->data->name, name,
> > >  			RTE_CRYPTODEV_NAME_MAX_LEN);
> > > @@ -724,13 +749,20 @@ rte_cryptodev_pmd_release_device(struct
> > > rte_cryptodev *cryptodev)
> > >  	if (cryptodev == NULL)
> > >  		return -EINVAL;
> > >
> > > +	uint8_t dev_id = cryptodev->data->dev_id;
> > > +
> >
> > [Anoob] Variables need to be declared at the start of the function.
> > https://doc.dpdk.org/guides/contributing/coding_style.html
> >
> > >  	/* Close device only if device operations have been set */
> > >  	if (cryptodev->dev_ops) {
> > > -		ret = rte_cryptodev_close(cryptodev->data->dev_id);
> > > +		ret = rte_cryptodev_close(dev_id);
> > >  		if (ret < 0)
> > >  			return ret;
> > >  	}
> > >
> > > +	struct rte_cryptodev_data **cryptodev_data =
> > > &cryptodev_globals.data[dev_id];
> >
> > [Anoob] Same comment as above
> >
> > > +	ret = rte_cryptodev_data_free(dev_id, cryptodev_data);
> > > +	if (ret < 0)
> > > +		return ret;
> > > +
> > >  	cryptodev->attached = RTE_CRYPTODEV_DETACHED;
> > >  	cryptodev_globals.nb_devs--;
> > >  	return 0;
> > > --
> > > 2.7.4

Re: [dpdk-dev] [EXT] [PATCH] cryptodev: free memzone when releasing cryptodev

[Anoob Joseph]

Hi Akhil,

One correction. Please see inline.

Thanks,
Anoob

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Anoob Joseph
> Sent: Friday, June 28, 2019 12:34 PM
> To: Akhil Goyal <akhil.goyal@nxp.com>; Junxiao Shi
> <git@mail1.yoursunny.com>; dev@dpdk.org
> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> Subject: Re: [dpdk-dev] [EXT] [PATCH] cryptodev: free memzone when
> releasing cryptodev
> 
> Hi Akhil,
> 
> Please see inline.
> 
> Thanks,
> Anoob
> 
> > -----Original Message-----
> > From: Akhil Goyal <akhil.goyal@nxp.com>
> > Sent: Friday, June 28, 2019 11:45 AM
> > To: Anoob Joseph <anoobj@marvell.com>; Junxiao Shi
> > <git@mail1.yoursunny.com>; dev@dpdk.org
> > Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> > Subject: RE: [EXT] [dpdk-dev] [PATCH] cryptodev: free memzone when
> > releasing cryptodev
> >
> > Hi Anoob,
> > > >
> > > > ------------------------------------------------------------------
> > > > --
> > > > -- When a cryptodev is created in a primary process,
> > > > rte_cryptodev_data_alloc reserves a memzone.
> > > > However, this memzone was not released when the cryptodev is
> > uninitialized.
> > > > After that, new cryptodev cannot be created due to memzone name
> > conflict.
> > > >
> > > > This commit frees the memzone when a cryptodev is uninitialized,
> > > > fixing this bug. This approach is chosen instead of keeping and
> > > > reusing the old memzone, because the new cryptodev could belong to
> > > > a
> > different NUMA socket.
> > > >
> > > > Also, rte_cryptodev_data pointer is now properly recorded in
> > > > cryptodev_globals.data array.
> > > >
> > > > Bugzilla ID: 105
> > > >
> > > > Signed-off-by: Junxiao Shi <git@mail1.yoursunny.com>
> > > > ---
> > > >  lib/librte_cryptodev/rte_cryptodev.c | 44
> > > > +++++++++++++++++++++++++++++++-----
> > > >  1 file changed, 38 insertions(+), 6 deletions(-)
> > > >
> > > > diff --git a/lib/librte_cryptodev/rte_cryptodev.c
> > > > b/lib/librte_cryptodev/rte_cryptodev.c
> > > > index 00c2cf4..666dfea 100644
> > > > --- a/lib/librte_cryptodev/rte_cryptodev.c
> > > > +++ b/lib/librte_cryptodev/rte_cryptodev.c
> > > > @@ -653,6 +653,31 @@ rte_cryptodev_data_alloc(uint8_t dev_id,
> > > > struct rte_cryptodev_data **data,
> > > >  	return 0;
> > > >  }
> > > >
> > > > +static inline int
> > > > +rte_cryptodev_data_free(uint8_t dev_id, struct rte_cryptodev_data
> > > > +**data) {
> > > > +	char mz_name[RTE_CRYPTODEV_NAME_MAX_LEN];
> 
> [Anoob] Shouldn't we use RTE_MEMZONE_NAMESIZE instead? I guess this is
> also coming from the existing code in rte_cryptodev_data_alloc(). May be
> we should fix that as well?
> 
> > > > +	const struct rte_memzone *mz;
> > > > +	int n;
> > > > +
> > > > +	/* generate memzone name */
> > > > +	n = snprintf(mz_name, sizeof(mz_name),
> > "rte_cryptodev_data_%u",
> > > > dev_id);
> > > > +	if (n >= (int)sizeof(mz_name))
> > > > +		return -EINVAL;
> > >
> > > [Anoob] Is the above check needed?
> > I believe this being used while creating the memzone, so same logic is
> > used while freeing it.
> > Just to be safe.
> >
> 
> [Anoob] Thinking bit more, it seems like we are trying to capture a situation
> when the name is getting truncated because of insufficient buffer space. So
> it is safe to have I guess. But even in that case, 'n' will not be greater than the
> "size" field passed (which happens to be sizeof(mz_name) in our case).
> 
> My opinion is '==' might make more sense. But I leave that to your
> judgement.

[Anoob] The check has to be retained. 

"The number of characters that would have been written if n had been sufficiently large, not counting the terminating null character." 

Please ignore my earlier comments.

> 
> > >
> > > > +
> > > > +	mz = rte_memzone_lookup(mz_name);
> > > > +	if (mz == NULL)
> > > > +		return -ENOMEM;
> > >
> > > [Anoob] Is the return value correct? Shouldn't it be -EINVAL?
> > >
> > > @Akhil, thoughts?
> >
> >
> > I believe ENOMEM is correct, as there is no memory associated with the
> > cryptodev_data.
> 
> [Anoob] Agreed.
> 
> >
> > >
> > > > +
> > > > +	RTE_ASSERT(*data == mz->addr);
> > > > +	*data = NULL;
> > > > +
> > > > +	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
> > > > +		return rte_memzone_free(mz);
> > > > +
> > > > +	return 0;
> > > > +}
> > > > +
> > > >  static uint8_t
> > > >  rte_cryptodev_find_free_device_index(void)
> > > >  {
> > > > @@ -687,16 +712,16 @@ rte_cryptodev_pmd_allocate(const char
> > *name,
> > > > int
> > > > socket_id)
> > > >  	cryptodev = rte_cryptodev_pmd_get_dev(dev_id);
> > > >
> > > >  	if (cryptodev->data == NULL) {
> > > > -		struct rte_cryptodev_data *cryptodev_data =
> > > > -				cryptodev_globals.data[dev_id];
> > > > +		struct rte_cryptodev_data **cryptodev_data =
> > > > +				&cryptodev_globals.data[dev_id];
> > > >
> > > > -		int retval = rte_cryptodev_data_alloc(dev_id,
> > &cryptodev_data,
> > > > +		int retval = rte_cryptodev_data_alloc(dev_id,
> > cryptodev_data,
> > > >  				socket_id);
> > > >
> > > > -		if (retval < 0 || cryptodev_data == NULL)
> > > > +		if (retval < 0 || *cryptodev_data == NULL)
> > > >  			return NULL;
> > > >
> > > > -		cryptodev->data = cryptodev_data;
> > > > +		cryptodev->data = *cryptodev_data;
> > > >
> > > >  		strlcpy(cryptodev->data->name, name,
> > > >  			RTE_CRYPTODEV_NAME_MAX_LEN);
> > > > @@ -724,13 +749,20 @@ rte_cryptodev_pmd_release_device(struct
> > > > rte_cryptodev *cryptodev)
> > > >  	if (cryptodev == NULL)
> > > >  		return -EINVAL;
> > > >
> > > > +	uint8_t dev_id = cryptodev->data->dev_id;
> > > > +
> > >
> > > [Anoob] Variables need to be declared at the start of the function.
> > > https://doc.dpdk.org/guides/contributing/coding_style.html
> > >
> > > >  	/* Close device only if device operations have been set */
> > > >  	if (cryptodev->dev_ops) {
> > > > -		ret = rte_cryptodev_close(cryptodev->data->dev_id);
> > > > +		ret = rte_cryptodev_close(dev_id);
> > > >  		if (ret < 0)
> > > >  			return ret;
> > > >  	}
> > > >
> > > > +	struct rte_cryptodev_data **cryptodev_data =
> > > > &cryptodev_globals.data[dev_id];
> > >
> > > [Anoob] Same comment as above
> > >
> > > > +	ret = rte_cryptodev_data_free(dev_id, cryptodev_data);
> > > > +	if (ret < 0)
> > > > +		return ret;
> > > > +
> > > >  	cryptodev->attached = RTE_CRYPTODEV_DETACHED;
> > > >  	cryptodev_globals.nb_devs--;
> > > >  	return 0;
> > > > --
> > > > 2.7.4

[dpdk-dev] [PATCH v2] cryptodev: free memzone when releasing cryptodev

[Junxiao Shi]

When a cryptodev is created in a primary process,
rte_cryptodev_data_alloc reserves a memzone.
However, this memzone was not released when the cryptodev
is uninitialized. After that, new cryptodev cannot be
created due to memzone name conflict.

This commit frees the memzone when a cryptodev is
uninitialized, fixing this bug. This approach is chosen
instead of keeping and reusing the old memzone, because
the new cryptodev could belong to a different NUMA socket.

Also, rte_cryptodev_data pointer is now properly recorded
in cryptodev_globals.data array.

Bugzilla ID: 105

Signed-off-by: Junxiao Shi <git@mail1.yoursunny.com>
---
 lib/librte_cryptodev/rte_cryptodev.c | 46 ++++++++++++++++++++++++++++++------
 1 file changed, 39 insertions(+), 7 deletions(-)

diff --git a/lib/librte_cryptodev/rte_cryptodev.c b/lib/librte_cryptodev/rte_cryptodev.c
index 00c2cf4..a7a0d4b 100644
--- a/lib/librte_cryptodev/rte_cryptodev.c
+++ b/lib/librte_cryptodev/rte_cryptodev.c
@@ -627,7 +627,7 @@ static inline int
 rte_cryptodev_data_alloc(uint8_t dev_id, struct rte_cryptodev_data **data,
 		int socket_id)
 {
-	char mz_name[RTE_CRYPTODEV_NAME_MAX_LEN];
+	char mz_name[RTE_MEMZONE_NAMESIZE];
 	const struct rte_memzone *mz;
 	int n;
 
@@ -653,6 +653,31 @@ rte_cryptodev_data_alloc(uint8_t dev_id, struct rte_cryptodev_data **data,
 	return 0;
 }
 
+static inline int
+rte_cryptodev_data_free(uint8_t dev_id, struct rte_cryptodev_data **data)
+{
+	char mz_name[RTE_MEMZONE_NAMESIZE];
+	const struct rte_memzone *mz;
+	int n;
+
+	/* generate memzone name */
+	n = snprintf(mz_name, sizeof(mz_name), "rte_cryptodev_data_%u", dev_id);
+	if (n >= (int)sizeof(mz_name))
+		return -EINVAL;
+
+	mz = rte_memzone_lookup(mz_name);
+	if (mz == NULL)
+		return -ENOMEM;
+
+	RTE_ASSERT(*data == mz->addr);
+	*data = NULL;
+
+	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+		return rte_memzone_free(mz);
+
+	return 0;
+}
+
 static uint8_t
 rte_cryptodev_find_free_device_index(void)
 {
@@ -687,16 +712,16 @@ rte_cryptodev_pmd_allocate(const char *name, int socket_id)
 	cryptodev = rte_cryptodev_pmd_get_dev(dev_id);
 
 	if (cryptodev->data == NULL) {
-		struct rte_cryptodev_data *cryptodev_data =
-				cryptodev_globals.data[dev_id];
+		struct rte_cryptodev_data **cryptodev_data =
+				&cryptodev_globals.data[dev_id];
 
-		int retval = rte_cryptodev_data_alloc(dev_id, &cryptodev_data,
+		int retval = rte_cryptodev_data_alloc(dev_id, cryptodev_data,
 				socket_id);
 
-		if (retval < 0 || cryptodev_data == NULL)
+		if (retval < 0 || *cryptodev_data == NULL)
 			return NULL;
 
-		cryptodev->data = cryptodev_data;
+		cryptodev->data = *cryptodev_data;
 
 		strlcpy(cryptodev->data->name, name,
 			RTE_CRYPTODEV_NAME_MAX_LEN);
@@ -720,17 +745,24 @@ int
 rte_cryptodev_pmd_release_device(struct rte_cryptodev *cryptodev)
 {
 	int ret;
+	uint8_t dev_id;
 
 	if (cryptodev == NULL)
 		return -EINVAL;
 
+	dev_id = cryptodev->data->dev_id;
+
 	/* Close device only if device operations have been set */
 	if (cryptodev->dev_ops) {
-		ret = rte_cryptodev_close(cryptodev->data->dev_id);
+		ret = rte_cryptodev_close(dev_id);
 		if (ret < 0)
 			return ret;
 	}
 
+	ret = rte_cryptodev_data_free(dev_id, &cryptodev_globals.data[dev_id]);
+	if (ret < 0)
+		return ret;
+
 	cryptodev->attached = RTE_CRYPTODEV_DETACHED;
 	cryptodev_globals.nb_devs--;
 	return 0;
-- 
2.7.4

Re: [dpdk-dev] [PATCH v2] cryptodev: free memzone when releasing cryptodev

[Akhil Goyal]

> 
> When a cryptodev is created in a primary process,
> rte_cryptodev_data_alloc reserves a memzone.
> However, this memzone was not released when the cryptodev
> is uninitialized. After that, new cryptodev cannot be
> created due to memzone name conflict.
> 
> This commit frees the memzone when a cryptodev is
> uninitialized, fixing this bug. This approach is chosen
> instead of keeping and reusing the old memzone, because
> the new cryptodev could belong to a different NUMA socket.
> 
> Also, rte_cryptodev_data pointer is now properly recorded
> in cryptodev_globals.data array.
> 
> Bugzilla ID: 105
> 
> Signed-off-by: Junxiao Shi <git@mail1.yoursunny.com>
> ---
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>

Applied to dpdk-next-crypto

Thanks.