From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03on0053.outbound.protection.outlook.com [104.47.40.53]) by dpdk.org (Postfix) with ESMTP id E270523C for ; Fri, 4 May 2018 04:48:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=3JWa5PMwHwFQha110dhzNL0K+tBF5j24GNghFhUnDy0=; b=oWP5bKPgHKDIj0hjnL3NFBG/QN9hZ2Y7Vzn3YsOebXvVgSNx5U8KY93i5DB3zAOIfPKrL+KcPG8ctSauMgRAgmEJo/QDB819B7s4fQfvideYBUTDaiSca9qm53JtWwqCaNDcXR1HnKIS76HPTeF2WBn0fz93KZcgoVxS9qoRJR0= Received: from CY4PR0701MB3634.namprd07.prod.outlook.com (52.132.101.164) by CY4PR0701MB3633.namprd07.prod.outlook.com (52.132.101.163) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.735.16; Fri, 4 May 2018 02:48:38 +0000 Received: from CY4PR0701MB3634.namprd07.prod.outlook.com ([fe80::b0a9:2be9:bfdf:fc0a]) by CY4PR0701MB3634.namprd07.prod.outlook.com ([fe80::b0a9:2be9:bfdf:fc0a%13]) with mapi id 15.20.0735.016; Fri, 4 May 2018 02:48:36 +0000 From: "Verma, Shally" To: Pablo de Lara , "dev@dpdk.org" CC: "fiona.trahe@intel.com" , "ahmed.mansour@nxp.com" , "Gupta, Ashish" , "Sahu, Sunila" Thread-Topic: [PATCH v6 02/14] compressdev: add queue pair management Thread-Index: AQHT3isC5cf9BOid/UmSWBgovfMrsqQe5M6A Date: Fri, 4 May 2018 02:48:36 +0000 Message-ID: References: <1517595924-25963-1-git-send-email-fiona.trahe@intel.com> <20180427132407.13385-1-pablo.de.lara.guarch@intel.com> <20180427132407.13385-3-pablo.de.lara.guarch@intel.com> In-Reply-To: <20180427132407.13385-3-pablo.de.lara.guarch@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Shally.Verma@cavium.com; x-originating-ip: [223.230.55.132] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; CY4PR0701MB3633; 7:RuFM+/4kEJDkHPcd2Diqr0ejhXvs8J3u6X6HTPpyx+aZQoj+CLy6p+Jr6AGa4ADuMn2I/0X8BdL+qzx9U17pkLodoUpujbT2vaJwUcuXVcQjOusoYE7tyKAKpFZN0Unwrioc3TVhWN5exYG3379X5oR2padWM3qf8mEsVNxfXO6c97r0CpJEM22OTHVmM9fjk9EySm1iAJs2mpJHda88SFVOmlowWgNv6kdA1lmOqkY2Wrf4+h3UFa3+y+/KoZhI x-ms-exchange-antispam-srfa-diagnostics: SOS;SOR; x-forefront-antispam-report: SFV:SKI; SCL:-1; SFV:NSPM; SFS:(10009020)(39860400002)(396003)(366004)(346002)(39380400002)(376002)(199004)(13464003)(189003)(4326008)(25786009)(8656006)(486006)(8676002)(66066001)(6436002)(81156014)(99286004)(229853002)(107886003)(6116002)(305945005)(3846002)(14454004)(105586002)(81166006)(106356001)(7736002)(110136005)(8936002)(54906003)(478600001)(68736007)(59450400001)(6246003)(86362001)(53936002)(72206003)(102836004)(9686003)(2906002)(6506007)(186003)(76176011)(26005)(55016002)(3660700001)(7696005)(5250100002)(3280700002)(97736004)(2900100001)(11346002)(2501003)(5660300001)(33656002)(316002)(74316002)(446003)(476003)(461764006); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR0701MB3633; H:CY4PR0701MB3634.namprd07.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:CY4PR0701MB3633; x-ms-traffictypediagnostic: CY4PR0701MB3633: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(185117386973197)(228905959029699); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(3002001)(93006095)(93001095)(3231254)(944501410)(52105095)(6041310)(20161123562045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123558120)(6072148)(201708071742011); SRVR:CY4PR0701MB3633; BCL:0; PCL:0; RULEID:; SRVR:CY4PR0701MB3633; x-forefront-prvs: 06628F7CA4 received-spf: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: NOMHaAkReC7m2IMJ5x09f+f8xG02KwrYybH6jHgS/l9zk68jFE8bqUWFiDOorn53UgYBNQgUi+Iwfo/j2ez+6TARiYpCqf6nc6D1Qlk4j6N7cdE4pB7ql5O/xEm1mp9qso0Qwf7dLbL/Edwr8Mf6Rojf5OidhFB7qM2aEjg5ZmD2uUgyLZp8QFf+ZuDAFLsL spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: ac19094b-4497-4b96-3d38-08d5b16988d8 X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-Network-Message-Id: ac19094b-4497-4b96-3d38-08d5b16988d8 X-MS-Exchange-CrossTenant-originalarrivaltime: 04 May 2018 02:48:36.6629 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR0701MB3633 Subject: Re: [dpdk-dev] [PATCH v6 02/14] compressdev: add queue pair management X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 May 2018 02:48:41 -0000 One quick question >-----Original Message----- >From: Pablo de Lara [mailto:pablo.de.lara.guarch@intel.com] >Sent: 27 April 2018 18:54 >To: dev@dpdk.org >Cc: fiona.trahe@intel.com; Verma, Shally ; ahmed.= mansour@nxp.com; Gupta, Ashish >; Pablo de Lara ;= Verma, Shally ; Gupta, >Ashish >Subject: [PATCH v6 02/14] compressdev: add queue pair management > >From: Fiona Trahe > >Add functions to manage device queue pairs. > >Signed-off-by: Fiona Trahe >Signed-off-by: Pablo de Lara >Signed-off-by: Shally Verma >Signed-off-by: Ashish Gupta >--- > lib/librte_compressdev/rte_compressdev.c | 178 ++++++++++++++++= ++++- > lib/librte_compressdev/rte_compressdev.h | 45 ++++++ > lib/librte_compressdev/rte_compressdev_internal.h | 5 + > lib/librte_compressdev/rte_compressdev_pmd.h | 47 ++++++ > lib/librte_compressdev/rte_compressdev_version.map | 2 + > 5 files changed, 276 insertions(+), 1 deletion(-) > >diff --git a/lib/librte_compressdev/rte_compressdev.c b/lib/librte_compres= sdev/rte_compressdev.c >index 751517c3d..6667528b5 100644 >--- a/lib/librte_compressdev/rte_compressdev.c >+++ b/lib/librte_compressdev/rte_compressdev.c >@@ -227,10 +227,136 @@ rte_compressdev_pmd_release_device(struct rte_compr= essdev *compressdev) > return 0; > } > >+uint16_t __rte_experimental >+rte_compressdev_queue_pair_count(uint8_t dev_id) >+{ >+ struct rte_compressdev *dev; >+ >+ dev =3D &rte_comp_devices[dev_id]; >+ return dev->data->nb_queue_pairs; >+} >+ >+static int >+rte_compressdev_queue_pairs_config(struct rte_compressdev *dev, >+ uint16_t nb_qpairs, int socket_id) >+{ >+ struct rte_compressdev_info dev_info; >+ void **qp; >+ unsigned int i; >+ >+ if ((dev =3D=3D NULL) || (nb_qpairs < 1)) { >+ COMPRESSDEV_LOG(ERR, "invalid param: dev %p, nb_queues %u", >+ dev, nb_qpairs); >+ return -EINVAL; >+ } >+ >+ COMPRESSDEV_LOG(DEBUG, "Setup %d queues pairs on device %u", >+ nb_qpairs, dev->data->dev_id); >+ >+ memset(&dev_info, 0, sizeof(struct rte_compressdev_info)); >+ >+ RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP); >+ (*dev->dev_ops->dev_infos_get)(dev, &dev_info); >+ >+ if ((dev_info.max_nb_queue_pairs !=3D 0) && >+ (nb_qpairs > dev_info.max_nb_queue_pairs)) { >+ COMPRESSDEV_LOG(ERR, "Invalid num queue_pairs (%u) for dev %u", >+ nb_qpairs, dev->data->dev_id); >+ return -EINVAL; >+ } >+ >+ if (dev->data->queue_pairs =3D=3D NULL) { /* first time configuration */ >+ dev->data->queue_pairs =3D rte_zmalloc_socket( >+ "compressdev->queue_pairs", >+ sizeof(dev->data->queue_pairs[0]) * nb_qpairs, >+ RTE_CACHE_LINE_SIZE, socket_id); >+ >+ if (dev->data->queue_pairs =3D=3D NULL) { >+ dev->data->nb_queue_pairs =3D 0; >+ COMPRESSDEV_LOG(ERR, >+ "failed to get memory for qp meta data, nb_queues %u", >+ nb_qpairs); >+ return -(ENOMEM); >+ } >+ } else { /* re-configure */ >+ int ret; >+ uint16_t old_nb_queues =3D dev->data->nb_queue_pairs; >+ [Shally] dev->data is shared entity among processes. So, does it mean if mu= ltiple processes call for device_configure, then every new process override= s qp allocations (and other resources) already done by any other process, l= eaving it unusable for them? Thanks Shally >+ qp =3D dev->data->queue_pairs; >+ >+ RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_pair_release, >+ -ENOTSUP); >+ >+ for (i =3D nb_qpairs; i < old_nb_queues; i++) { >+ ret =3D (*dev->dev_ops->queue_pair_release)(dev, i); >+ if (ret < 0) >+ return ret; >+ } >+ >+ qp =3D rte_realloc(qp, sizeof(qp[0]) * nb_qpairs, >+ RTE_CACHE_LINE_SIZE); >+ if (qp =3D=3D NULL) { >+ COMPRESSDEV_LOG(ERR, >+ "failed to realloc qp meta data, nb_queues %u", >+ nb_qpairs); >+ return -(ENOMEM); >+ } >+ >+ if (nb_qpairs > old_nb_queues) { >+ uint16_t new_qs =3D nb_qpairs - old_nb_queues; >+ >+ memset(qp + old_nb_queues, 0, >+ sizeof(qp[0]) * new_qs); >+ } >+ >+ dev->data->queue_pairs =3D qp; >+ >+ } >+ dev->data->nb_queue_pairs =3D nb_qpairs; >+ return 0; >+} >+ >+static int >+rte_compressdev_queue_pairs_release(struct rte_compressdev *dev) >+{ >+ uint16_t num_qps, i; >+ int ret; >+ >+ if (dev =3D=3D NULL) { >+ COMPRESSDEV_LOG(ERR, "invalid param: dev %p", dev); >+ return -EINVAL; >+ } >+ >+ num_qps =3D dev->data->nb_queue_pairs; >+ >+ if (num_qps =3D=3D 0) >+ return 0; >+ >+ COMPRESSDEV_LOG(DEBUG, "Free %d queues pairs on device %u", >+ dev->data->nb_queue_pairs, dev->data->dev_id); >+ >+ RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_pair_release, >+ -ENOTSUP); >+ >+ for (i =3D 0; i < num_qps; i++) { >+ ret =3D (*dev->dev_ops->queue_pair_release)(dev, i); >+ if (ret < 0) >+ return ret; >+ } >+ >+ if (dev->data->queue_pairs !=3D NULL) >+ rte_free(dev->data->queue_pairs); >+ dev->data->queue_pairs =3D NULL; >+ dev->data->nb_queue_pairs =3D 0; >+ >+ return 0; >+} >+ > int __rte_experimental > rte_compressdev_configure(uint8_t dev_id, struct rte_compressdev_config *= config) > { > struct rte_compressdev *dev; >+ int diag; > > if (!rte_compressdev_is_valid_dev(dev_id)) { > COMPRESSDEV_LOG(ERR, "Invalid dev_id=3D%" PRIu8, dev_id); >@@ -247,10 +373,19 @@ rte_compressdev_configure(uint8_t dev_id, struct rte= _compressdev_config *config) > > RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_configure, -ENOTSUP); > >+ /* Setup new number of queue pairs and reconfigure device. */ >+ diag =3D rte_compressdev_queue_pairs_config(dev, config->nb_queue_pairs, >+ config->socket_id); >+ if (diag !=3D 0) { >+ COMPRESSDEV_LOG(ERR, >+ "dev%d rte_comp_dev_queue_pairs_config =3D %d", >+ dev_id, diag); >+ return diag; >+ } >+ > return (*dev->dev_ops->dev_configure)(dev, config); > } > >- > int __rte_experimental > rte_compressdev_start(uint8_t dev_id) > { >@@ -327,6 +462,12 @@ rte_compressdev_close(uint8_t dev_id) > return -EBUSY; > } > >+ /* Free queue pairs memory */ >+ retval =3D rte_compressdev_queue_pairs_release(dev); >+ >+ if (retval < 0) >+ return retval; >+ > RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_close, -ENOTSUP); > retval =3D (*dev->dev_ops->dev_close)(dev); > >@@ -336,6 +477,41 @@ rte_compressdev_close(uint8_t dev_id) > return 0; > } > >+int __rte_experimental >+rte_compressdev_queue_pair_setup(uint8_t dev_id, uint16_t queue_pair_id, >+ uint32_t max_inflight_ops, int socket_id) >+{ >+ struct rte_compressdev *dev; >+ >+ if (!rte_compressdev_is_valid_dev(dev_id)) { >+ COMPRESSDEV_LOG(ERR, "Invalid dev_id=3D%" PRIu8, dev_id); >+ return -EINVAL; >+ } >+ >+ dev =3D &rte_comp_devices[dev_id]; >+ if (queue_pair_id >=3D dev->data->nb_queue_pairs) { >+ COMPRESSDEV_LOG(ERR, "Invalid queue_pair_id=3D%d", queue_pair_id); >+ return -EINVAL; >+ } >+ >+ if (dev->data->dev_started) { >+ COMPRESSDEV_LOG(ERR, >+ "device %d must be stopped to allow configuration", dev_id); >+ return -EBUSY; >+ } >+ >+ if (max_inflight_ops =3D=3D 0) { >+ COMPRESSDEV_LOG(ERR, >+ "Invalid maximum number of inflight operations"); >+ return -EINVAL; >+ } >+ >+ RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_pair_setup, -ENOTSUP); >+ >+ return (*dev->dev_ops->queue_pair_setup)(dev, queue_pair_id, >+ max_inflight_ops, socket_id); >+} >+ > void __rte_experimental > rte_compressdev_info_get(uint8_t dev_id, struct rte_compressdev_info *dev= _info) > { >diff --git a/lib/librte_compressdev/rte_compressdev.h b/lib/librte_compres= sdev/rte_compressdev.h >index 5be5973bf..81710bbea 100644 >--- a/lib/librte_compressdev/rte_compressdev.h >+++ b/lib/librte_compressdev/rte_compressdev.h >@@ -22,6 +22,10 @@ extern "C" { > /** comp device information */ > struct rte_compressdev_info { > const char *driver_name; /**< Driver name. */ >+ uint16_t max_nb_queue_pairs; >+ /**< Maximum number of queues pairs supported by device. >+ * (If 0, there is no limit in maximum number of queue pairs) >+ */ > }; > > /** >@@ -80,6 +84,9 @@ rte_compressdev_socket_id(uint8_t dev_id); > /** Compress device configuration structure */ > struct rte_compressdev_config { > int socket_id; >+ /**< Socket on which to allocate resources */ >+ uint16_t nb_queue_pairs; >+ /**< Total number of queue pairs to configure on a device */ > }; > > /** >@@ -145,6 +152,44 @@ rte_compressdev_stop(uint8_t dev_id); > int __rte_experimental > rte_compressdev_close(uint8_t dev_id); > >+/** >+ * Allocate and set up a receive queue pair for a device. >+ * This should only be called when the device is stopped. >+ * >+ * >+ * @param dev_id >+ * Compress device identifier >+ * @param queue_pair_id >+ * The index of the queue pairs to set up. The >+ * value must be in the range [0, nb_queue_pair - 1] >+ * previously supplied to rte_compressdev_configure() >+ * @param max_inflight_ops >+ * Max number of ops which the qp will have to >+ * accommodate simultaneously >+ * @param socket_id >+ * The *socket_id* argument is the socket identifier >+ * in case of NUMA. The value can be *SOCKET_ID_ANY* >+ * if there is no NUMA constraint for the DMA memory >+ * allocated for the receive queue pair >+ * @return >+ * - 0: Success, queue pair correctly set up. >+ * - <0: Queue pair configuration failed >+ */ >+int __rte_experimental >+rte_compressdev_queue_pair_setup(uint8_t dev_id, uint16_t queue_pair_id, >+ uint32_t max_inflight_ops, int socket_id); >+ >+/** >+ * Get the number of queue pairs on a specific comp device >+ * >+ * @param dev_id >+ * Compress device identifier >+ * @return >+ * - The number of configured queue pairs. >+ */ >+uint16_t __rte_experimental >+rte_compressdev_queue_pair_count(uint8_t dev_id); >+ > /** > * Retrieve the contextual information of a device. > * >diff --git a/lib/librte_compressdev/rte_compressdev_internal.h b/lib/librt= e_compressdev/rte_compressdev_internal.h >index 0a2ddcb2b..57af163c1 100644 >--- a/lib/librte_compressdev/rte_compressdev_internal.h >+++ b/lib/librte_compressdev/rte_compressdev_internal.h >@@ -58,6 +58,11 @@ struct rte_compressdev_data { > uint8_t dev_started : 1; > /**< Device state: STARTED(1)/STOPPED(0) */ > >+ void **queue_pairs; >+ /**< Array of pointers to queue pairs. */ >+ uint16_t nb_queue_pairs; >+ /**< Number of device queue pairs */ >+ > void *dev_private; > /**< PMD-specific private data */ > } __rte_cache_aligned; >diff --git a/lib/librte_compressdev/rte_compressdev_pmd.h b/lib/librte_com= pressdev/rte_compressdev_pmd.h >index 43307ee8e..14ce76b44 100644 >--- a/lib/librte_compressdev/rte_compressdev_pmd.h >+++ b/lib/librte_compressdev/rte_compressdev_pmd.h >@@ -125,6 +125,48 @@ typedef int (*compressdev_close_t)(struct rte_compres= sdev *dev); > typedef void (*compressdev_info_get_t)(struct rte_compressdev *dev, > struct rte_compressdev_info *dev_info); > >+/** >+ * Setup a queue pair for a device. >+ * >+ * @param dev >+ * Compress device >+ * @param qp_id >+ * Queue pair identifier >+ * @param max_inflight_ops >+ * Max inflight ops which qp must accommodate >+ * @param socket_id >+ * Socket identifier >+ * @return >+ * Returns 0 on success. >+ */ >+typedef int (*compressdev_queue_pair_setup_t)(struct rte_compressdev *dev= , >+ uint16_t qp_id, uint32_t max_inflight_ops, int socket_id); >+ >+/** >+ * Release memory resources allocated by given queue pair. >+ * >+ * @param dev >+ * Compress device >+ * @param qp_id >+ * Queue pair identifier >+ * @return >+ * - 0 on success. >+ * - EAGAIN if can't close as device is busy >+ */ >+typedef int (*compressdev_queue_pair_release_t)(struct rte_compressdev *d= ev, >+ uint16_t qp_id); >+ >+/** >+ * Get number of available queue pairs of a device. >+ * >+ * @param dev >+ * Compress device >+ * @return >+ * Returns number of queue pairs on success. >+ */ >+typedef uint32_t (*compressdev_queue_pair_count_t)(struct rte_compressdev= *dev); >+ >+ > /** comp device operations function pointer table */ > struct rte_compressdev_ops { > compressdev_configure_t dev_configure; /**< Configure device. */ >@@ -133,6 +175,11 @@ struct rte_compressdev_ops { > compressdev_close_t dev_close; /**< Close device. */ > > compressdev_info_get_t dev_infos_get; /**< Get device info. */ >+ >+ compressdev_queue_pair_setup_t queue_pair_setup; >+ /**< Set up a device queue pair. */ >+ compressdev_queue_pair_release_t queue_pair_release; >+ /**< Release a queue pair. */ > }; > > /** >diff --git a/lib/librte_compressdev/rte_compressdev_version.map b/lib/libr= te_compressdev/rte_compressdev_version.map >index a996abcd2..182a371d8 100644 >--- a/lib/librte_compressdev/rte_compressdev_version.map >+++ b/lib/librte_compressdev/rte_compressdev_version.map >@@ -16,6 +16,8 @@ EXPERIMENTAL { > rte_compressdev_pmd_get_named_dev; > rte_compressdev_pmd_parse_input_args; > rte_compressdev_pmd_release_device; >+ rte_compressdev_queue_pair_count; >+ rte_compressdev_queue_pair_setup; > rte_compressdev_socket_id; > rte_compressdev_start; > rte_compressdev_stop; >-- >2.14.3