From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0083.outbound.protection.outlook.com [104.47.41.83]) by dpdk.org (Postfix) with ESMTP id 83EC7271 for ; Mon, 27 Nov 2017 11:44:31 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=3zanITABqtUaNIMZvzuQ6UClo1R48EYMnaMwIHsnm20=; b=YpiO4YwUwb5hRsygEkW0j37UyCM8yPh7mpGGZoWejvu2ltyJ6KHW0M0sdgG9hJJr144RBEBG2h+SdR4sAXix630oQDQT4RZ4+ZScCsPg35XOsG80xzKgyBr2yhwE2Fq3OtZ7haXSwxpx5TuSgGXsvD3q5BgBpgzStLzw9GDvWyo= Received: from BY1PR0701MB1111.namprd07.prod.outlook.com (10.160.104.21) by SN1PR0701MB1999.namprd07.prod.outlook.com (10.163.131.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.260.4; Mon, 27 Nov 2017 10:44:29 +0000 Received: from BY1PR0701MB1111.namprd07.prod.outlook.com ([fe80::c184:ee9b:3c4c:c97c]) by BY1PR0701MB1111.namprd07.prod.outlook.com ([fe80::c184:ee9b:3c4c:c97c%14]) with mapi id 15.20.0260.006; Mon, 27 Nov 2017 10:44:29 +0000 From: "Verma, Shally" To: "dev@dpdk.org" , "Trahe, Fiona" CC: "Athreya, Narayana Prasad" , "Challa, Mahipal" Thread-Topic: [RFC v2] lib: add compressdev API Thread-Index: AdNnbLD7NQnH1HwH4UCvzElEGQOLDg== Date: Mon, 27 Nov 2017 10:44:29 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Shally.Verma@cavium.com; x-originating-ip: [14.140.2.178] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; SN1PR0701MB1999; 6:8DjrJk/1xdgf0vxku9LaiON/Dt/jG4fJOUR1Gxb3cq8YgAjQbyOny6RjeiFvW4pZa0W6Gtu/kdUy5Nn3oPVSOvUgtQMRsS97mtAOTzcbOZiFytoifNk7nEt3dn12LuCwQurTLSCl+RDsachizYwfrbUsyRO46Qjc6XvKWfz4almrbNgZ9qveda4dwAVfzYGenS2YyixH+36BYKvbvwGeQMy6952Xd+/SS1zc35PMZ423/XcypUUX4oyTBgWYACbvIso8ugob4iT9FXLT2+MKAq8ZNLx7PGVgZLc+yDj1mnOt/GUeMD5vNkpBUwwm15xc4U9qLmXELzG4B0dsoMF++uuBmfp2Vb8pjXz3A+n/X/c=; 5:EXOWJolgWrO9/djhaytnYtCM/sNihP9zYMuKhi09quM37DhHuyALyCgyWfvyoBD/g0GV30E0g3S7mW2d+ZblY6XUOqNeK/pzD4hTIzEm7oVT/uI/a91rSdCFMpx2uxgcuDPihtN0futq1v1hvs2hPlkmny+V8l8fokrNGNX8MU0=; 24:g1nX7GlDWB3/IL0qtSAQ7v47p4qoz2nM/BEH8QxmR3rEM85A3OuilI6izosI5nbv62urR62FkA2clVbiC0/Eo/Xg4cuMHeKk2Af+OHjJqFM=; 7:kgkL+Sea8OsGOdGuIF+tgMpgz9KWOGTI2xNruRDZkAIZU4ur+J6chLJegTHqy/0bpEXmnxTqFr3tYyDGWxVmLkC4bBmVH2OlEjCVJA0xvs68ZqHqu+X1o2p/+4TqYIFFdCuqA268mCz+czdPC7Wm8FyMFG6Xgd8PBsrHqVy3HrIkiGqimyPFVJSfNQBnsj1ylsyy3shrqnyRejtsevCV6UkoE1t10l3DVOyrDe3zon7U38g28mtJbWl+PNKaSLJa x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-forefront-antispam-report: SFV:SKI; SCL:-1; SFV:NSPM; SFS:(10009020)(6009001)(346002)(366004)(376002)(13464003)(189002)(199003)(6116002)(66066001)(6506006)(14454004)(102836003)(2906002)(6246003)(53936002)(99286004)(68736007)(101416001)(6436002)(8936002)(189998001)(33656002)(2900100001)(106356001)(8676002)(107886003)(3846002)(81166006)(81156014)(55236003)(5250100002)(7696005)(9686003)(72206003)(3280700002)(478600001)(25786009)(4326008)(3660700001)(55016002)(2501003)(50986999)(54356999)(305945005)(7736002)(229853002)(97736004)(54906003)(110136005)(53546010)(5660300001)(74316002)(316002)(105586002)(86362001); DIR:OUT; SFP:1101; SCL:1; SRVR:SN1PR0701MB1999; H:BY1PR0701MB1111.namprd07.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; x-ms-office365-filtering-correlation-id: 414a30a0-9d45-4652-e921-08d53583d624 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(5600026)(4604075)(4534020)(4602075)(7168020)(4627115)(201703031133081)(201702281549075)(2017052603199); SRVR:SN1PR0701MB1999; x-ms-traffictypediagnostic: SN1PR0701MB1999: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(131327999870524)(228905959029699); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040450)(2401047)(8121501046)(5005006)(10201501046)(93006095)(93001095)(3231022)(3002001)(6041248)(20161123555025)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123558100)(20161123564025)(6072148)(201708071742011); SRVR:SN1PR0701MB1999; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:SN1PR0701MB1999; x-forefront-prvs: 0504F29D72 received-spf: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-Network-Message-Id: 414a30a0-9d45-4652-e921-08d53583d624 X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Nov 2017 10:44:29.1274 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR0701MB1999 Subject: Re: [dpdk-dev] [RFC v2] lib: add compressdev API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Nov 2017 10:44:32 -0000 Hi Fiona Thanks for spec. It looks good to move ahead for API proof-concepting howe= ver still need to address few key open points from rfc doc and others minor= comments (please see inline):=20 A. new enum for operation types: 1. Add rte_comp_op_type=20 ........................................ enum rte_comp_op_type { RTE_COMP_OP_STATELESS, RTE_COMP_OP_STATEFUL } B. Change in rte_comp_op_pool_create () to input op_type so that required r= esources could be reserved for that op_type(please see inline more on this)= . C. Call enque_burst() for stateless operations. And add a new dedicated API= to handle stateful compression static inline uint16_t rte_compdev_enqueue_stream(). We will add hash and other support as incremental patch on top of this once= we align on these few open points. For rest, please see inline. > -----Original Message----- > From: Trahe, Fiona [mailto:fiona.trahe@intel.com] > Sent: 24 November 2017 22:26 > To: dev@dpdk.org; Verma, Shally > Cc: Challa, Mahipal ; Athreya, Narayana > Prasad ; > pablo.de.lara.guarch@intel.com; fiona.trahe@intel.com > Subject: [RFC v2] lib: add compressdev API >=20 // snip // > +/** > + * Compression Operation. > + * > + * This structure contains data relating to performing a compression > + * operation on the referenced mbuf data buffers. > + * > + * All compression operations are Out-of-place (OOP) operations, > + * as the size of the output data is different to the size of the input = data. > + * > + * Comp operations are enqueued and dequeued in comp PMDs using the > + * rte_compressdev_enqueue_burst() / > rte_compressdev_dequeue_burst() APIs > + */ > +struct rte_comp_op { > + [Shally] need one void *user to enable re-ordering at app if PMD services e= nqueued requests out of order (requirement from RFC doc) void *user; /**< user pointer. Set by app, opaque to PMD. Passed back in deque_burst. * app can set it to its data to ensure operations could r= eorderd, if requests are processed out of order. * Particularly useful for stateless operations */ //snip// > +/** > + * Creates an operation pool > + * > + * @param name pool name > + * @param nb_elts number of elements in pool > + * @param cache_size Number of elements to cache on lcore, see > + * *rte_mempool_create* for further details > about > + * cache size > + * @param priv_size Size of private data to allocate with each > + * operation > + * @param socket_id Socket to allocate memory on > + * > + * @return > + * - On success pointer to mempool > + * - On failure NULL > + */ > +extern struct rte_mempool * > +rte_comp_op_pool_create(const char *name, > + unsigned nb_elts, unsigned cache_size, uint16_t priv_size, > + int socket_id); > + [Shally] This can be modified to input op_type : Stateful/stateless. In su= ch case, pool will be allocated large enough to provide additional memory (= or other resources) needed by PMD for that operation such as state / histor= y buffers et el. So, the proposed API change is : rte_comp_op_pool_create(const char *name, unsigned nb_elts, unsigned cache_size, uint= 16_t priv_size, rte_comp_op_type op_type, int socket_id); Since size of such resources can be different for each implementation, thus= a PMD can expose new API in dev_ops that can returns resources size requi= rement for op_type. //snip// > + > +/** > + * Allocate an operation from a mempool with default parameters set > + * > + * @param mempool operation mempool > + * > + * @returns > + * - On success returns a valid rte_comp_op structure > + * - On failure returns NULL > + */ > +static inline struct rte_comp_op * > +rte_comp_op_alloc(struct rte_mempool *mempool) > +{ [Shally] rte_comp_op_type should be one of the input arg so that required r= esources can be reserved according to op_type=20 > + struct rte_comp_op *op =3D NULL; > + int retval; > + > + retval =3D __rte_comp_op_raw_bulk_alloc(mempool, &op, 1); > + if (unlikely(retval !=3D 1)) > + return NULL; > + > + __rte_comp_op_reset(op); > + > + return op; > +} > + > + > +/** > + * Bulk allocate operations from a mempool with default parameters set > + * > + * @param mempool comp operation mempool > + * @param ops Array to place allocated operations > + * @param nb_ops Number of operations to allocate > + * > + * @returns > + * - nb_ops if the number of operations requested were allocated. > + * - 0 if the requested number of ops are not available. > + * None are allocated in this case. > + */ > + > +static inline unsigned > +rte_comp_op_bulk_alloc(struct rte_mempool *mempool, > + struct rte_comp_op **ops, uint16_t nb_ops) > +{ [Shally] Similarly, should be extended to input rte_comp_op_type as one of= the arg > + int i; > + > + if (unlikely(__rte_comp_op_raw_bulk_alloc(mempool, ops, nb_ops) > + !=3D nb_ops)) > + return 0; > + > + for (i =3D 0; i < nb_ops; i++) > + __rte_comp_op_reset(ops[i]); > + > + return nb_ops; > +} //snip// > + > +/** > + * Enqueue a burst of operations for processing on a compression device. > + * > + * The rte_compressdev_enqueue_burst() function is invoked to place > + * comp operations on the queue *qp_id* of the device designated by > + * its *dev_id*. > + * > + * The *nb_ops* parameter is the number of operations to process which > are > + * supplied in the *ops* array of *rte_comp_op* structures. > + * > + * The rte_compressdev_enqueue_burst() function returns the number of > + * operations it actually enqueued for processing. A return value equal = to > + * *nb_ops* means that all packets have been enqueued. > + * > + * @param dev_id The identifier of the device. > + * @param qp_id The index of the queue pair on which > operations > + * are to be enqueued for processing. The value > + * must be in the range [0, nb_queue_pairs - 1] > + * previously supplied to > + * *rte_compressdev_configure*. > + * @param ops The address of an array of *nb_ops* pointers > + * to *rte_comp_op* structures which contain > + * the operations to be processed. > + * @param nb_ops The number of operations to process. > + * > + * @return > + * The number of operations actually enqueued on the device. The return > + * value can be less than the value of the *nb_ops* parameter when the > + * comp devices queue is full or if invalid parameters are specified in > + * a *rte_comp_op*. [Shally] To ensure invalid params, app should check for status value of=20 returned_num+1 ops entry which should be set to INVALID_ARGS by PMD? > + */ > +static inline uint16_t > +rte_compressdev_enqueue_burst(uint8_t dev_id, uint16_t qp_id, > + struct rte_comp_op **ops, uint16_t nb_ops) > +{ > + struct rte_compressdev *dev =3D &rte_compressdevs[dev_id]; > + > + return (*dev->enqueue_burst)( > + dev->data->queue_pairs[qp_id], ops, nb_ops); > +} > + > + > +/** compressdev session */ > +struct rte_comp_session { > + __extension__ void *sess_private_data[0]; > + /**< Private session material */ > +}; > + > + > +/** > + * Create symmetric comp session header (generic with no private data) > + * [Shally] Believe Symmetric is type from crypto //snip// Thanks Shally