From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2D649A3201 for ; Sun, 20 Oct 2019 16:10:26 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E33333772; Sun, 20 Oct 2019 16:10:24 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id DFF202C2B for ; Sun, 20 Oct 2019 16:10:22 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x9KE9o03017892; Sun, 20 Oct 2019 07:09:50 -0700 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-0016f401.pphosted.com with ESMTP id 2vr20mjum9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 20 Oct 2019 07:09:49 -0700 Received: from m0045851.ppops.net (m0045851.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id x9KE9nvx017840; Sun, 20 Oct 2019 07:09:49 -0700 Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0b-0016f401.pphosted.com with ESMTP id 2vr20mjum7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 20 Oct 2019 07:09:49 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 20 Oct 2019 07:09:47 -0700 Received: from NAM04-CO1-obe.outbound.protection.outlook.com (104.47.45.56) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1367.3 via Frontend Transport; Sun, 20 Oct 2019 07:09:47 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F0OU+UuM7iKxEJxb3Ig6E/dyFBmGp9itOAdem8h628yldA0iRvLSMcxfEGm2xwPRAuBoIc5+VOIEPy7MyHHsiTvsAde3vbcjuQEe6tDplukfHIqC8VU3GRv9WSXtT4hK0R5PqvEVFE2NesxjFr6uAWNg98nTqrIzKTBaZyPLt90t6AQdwQqwOTT0ZQoUAKVA+gOTXTChMoupJknDicdesGGpwdAnhKputY8dDQPOfuKRSO+vKbkPtb7ruDkXEYMDztTKkysm+3TZ7c46O31yU7Oi/C3+lKSCzw5WqRKtvw4opE39QAMl6gpp4AMF245QRIiWWojkdcBdOQerCBva8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vVMeT0UVb/s0KMfYwevLQeeprocBf7MwxmWlWQbSR6c=; b=Q49eiqgEy3CT14tTTUQxoY1QxJLMJxy9Qh9A5dHZ262Hntdn1ztmN/lSD9o/vlOQCJFvH9bfLFMYiMVa0ZZEppqK8Xtdu2rEnPR9fPSXbHun5r5GBe88IQlW7W+DTYu/LvWCC7myxVuFLGlaIKXbBuNYaV+BCMEhfIXvKwevBm1PSSyeMAzUJ/k9+FLrdxD/Aw5GyQNRjWjLpbTgMGlGwORxS/BuRxlFOVqqISB2+8vNVlln4PjmgIacHZNqnRJyjLktdJQgwwzMGNMPmGlmFkgTcB5hFqXrW8VfD9h6sHhhQDnbyTxO/WyuOEygsrvgi1V6NrsKhx8qQpyHtytl1Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector2-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vVMeT0UVb/s0KMfYwevLQeeprocBf7MwxmWlWQbSR6c=; b=fR6cEGR9X7tVk7HCRLFIn4lWaKy5zmYE/la53mydlpmLXe2zGWjesMEOn5G4InaT34dH/R8pDMEfc42JkAUhkOuMAb1Z9aZMjfS4bLv9j6ZKjpk+sFJnG5cQKsDJA2hU2IYXw65GleQAknriLbJNxgfZiQ+y4ShGrzHsocx6UC4= Received: from BYAPR18MB2424.namprd18.prod.outlook.com (20.179.91.149) by BYAPR18MB2439.namprd18.prod.outlook.com (20.179.93.161) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2347.22; Sun, 20 Oct 2019 14:09:45 +0000 Received: from BYAPR18MB2424.namprd18.prod.outlook.com ([fe80::c9e7:5cd0:547a:6de2]) by BYAPR18MB2424.namprd18.prod.outlook.com ([fe80::c9e7:5cd0:547a:6de2%3]) with mapi id 15.20.2367.019; Sun, 20 Oct 2019 14:09:45 +0000 From: Jerin Jacob Kollanukkaran To: Wang Xiang CC: Thomas Monjalon , "dev@dpdk.org" , Pavan Nikhilesh Bhagavatula , Shahaf Shuler , Hemant Agrawal , Opher Reviv , Alex Rosenbaum , Dovrat Zifroni , Prasun Kapoor , Nipun Gupta , "Richardson, Bruce" , "Hong, Yang A" , "Chang, Harry" , "gu.jian1@zte.com.cn" , "shanjiangh@chinatelecom.cn" , "zhangy.yun@chinatelecom.cn" , "lixingfu@huachentel.com" , "wushuai@inspur.com" , "yuyingxia@yxlink.com" , "fanchenggang@sunyainfo.com" , "davidfgao@tencent.com" , "liuzhong1@chinaunicom.cn" , "zhaoyong11@huawei.com" , "oc@yunify.com" , "jim@netgate.com" , "Ni, Hongjun" , "j.bromhead@titan-ic.com" , "deri@ntop.org" , "fc@napatech.com" , "arthur.su@lionic.com" , Guy Kaneti , Smadar Fuks , Liron Himi , "edwin.verplanke@intel.com" , "keith.wiles@intel.com" Thread-Topic: [dpdk-dev] [RFC PATCH v1] regexdev: introduce regexdev subsystem Thread-Index: AdWHUAOIhGHwEyFNS6imCH6SI6vdEQ== Date: Sun, 20 Oct 2019 14:09:45 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [106.200.247.24] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 4a7d58ba-2b31-4bd4-0b4d-08d755672913 x-ms-traffictypediagnostic: BYAPR18MB2439: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-forefront-prvs: 0196A226D1 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(396003)(346002)(39850400004)(376002)(366004)(136003)(13464003)(199004)(189003)(53754006)(52084003)(316002)(7736002)(14454004)(81156014)(7406005)(7416002)(6436002)(30864003)(5660300002)(14444005)(229853002)(256004)(81166006)(66066001)(8936002)(66446008)(305945005)(76116006)(66556008)(66476007)(66946007)(64756008)(6916009)(8676002)(74316002)(26005)(4326008)(25786009)(561944003)(9686003)(2906002)(476003)(486006)(86362001)(99286004)(52536014)(478600001)(6506007)(6246003)(55016002)(54906003)(71190400001)(71200400001)(186003)(7696005)(6116002)(3846002)(53546011)(102836004)(33656002); DIR:OUT; SFP:1101; SCL:1; SRVR:BYAPR18MB2439; H:BYAPR18MB2424.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: sDE7wLtNvSAnXlSI1bWaFQMpvvOksMrCXEt5maA+HHRPX+cwFDXMHyay8sdWAgFDqBFUSizjz9E5MiELvVsD1JTsTiBWW5fQ1dD3wlvKWEc33s0VIzasXT7bebloS620y0Oxu0E7PjqLRvAkEgae3w36ElFAKWOpP89pyBp1G28jUc4hV4DH2OwnzGeJEGCswTZcR52cQ3Mlyc7hwHokj6aMR5zTMmusN266lAhJW6aZuE3DmOmjSRoZJ4OqiSsngohXVemDbwW5F456tdGnAk+zQQn3LnUuR1drS/tR3rjjqs26VwDk2QkeEEjSdDwATaEesbLPjwxbSYayikwT7Rwqmh/v9tvIg+hqYRqdhXZo9AYk24KnFb1lUDORBydGLOB6jL02zN9fE/PKaRp+RiC0iWeySI8TZtZbEFdB60I= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 4a7d58ba-2b31-4bd4-0b4d-08d755672913 X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Oct 2019 14:09:45.3022 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ubGQ+VdKK8woKXyvMSqygBUUJ2ZG7+MnFDHIsCFh4SfwUvcQfgtTQ34yxIgO/t0NBa5yZcC+Wl/Icrmds1t80w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR18MB2439 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,1.0.8 definitions=2019-10-20_05:2019-10-18,2019-10-20 signatures=0 Subject: Re: [dpdk-dev] [RFC PATCH v1] regexdev: introduce regexdev subsystem X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Wang Xiang > Sent: Monday, October 14, 2019 7:29 PM > To: Jerin Jacob Kollanukkaran > Cc: Thomas Monjalon ; dev@dpdk.org; Pavan > Nikhilesh Bhagavatula ; Shahaf Shuler > ; Hemant Agrawal ; > Opher Reviv ; Alex Rosenbaum > ; Dovrat Zifroni ; Prasun Kapoor > ; Nipun Gupta ; Richardson, > Bruce ; Hong, Yang A ; > Chang, Harry ; gu.jian1@zte.com.cn; > shanjiangh@chinatelecom.cn; zhangy.yun@chinatelecom.cn; > lixingfu@huachentel.com; wushuai@inspur.com; yuyingxia@yxlink.com; > fanchenggang@sunyainfo.com; davidfgao@tencent.com; > liuzhong1@chinaunicom.cn; zhaoyong11@huawei.com; oc@yunify.com; > jim@netgate.com; Ni, Hongjun ; j.bromhead@titan- > ic.com; deri@ntop.org; fc@napatech.com; arthur.su@lionic.com; Guy Kaneti > ; Smadar Fuks ; Liron Himi > ; edwin.verplanke@intel.com; keith.wiles@intel.com > Subject: [EXT] Re: [dpdk-dev] [RFC PATCH v1] regexdev: introduce regexdev > subsystem >=20 > External Email >=20 > ---------------------------------------------------------------------- > On Fri, Sep 27, 2019 at 02:35:00PM +0000, Jerin Jacob Kollanukkaran wrote= : > > > -----Original Message----- > > > From: Wang Xiang > > > > > > Hi Jerin, > > > > > > Thanks for your response. More comments below and inline. > > > > > > 1) I think the size of some varaibles (e.g. nb_matches, scan_size, > > > matching offset, etc) should be increased based on what Hyperscan > supports. > > > > > > a) struct rte_regex_ops: > > > > > > uint16_t scan_size =3D> uint32_t scan_size > > > > I think, packet buffers will not be > 64K and getting more than > > contiguous 64K DMAable memory will be difficult in DPDK. > > Other than that, rte_regex_match is 64bit now, increasing width of Len > > could increase the size of "rte_regex_match". i.e Need more Bandwidth > > for response. > > Could other HW implementations share the views on max length is > > supported on their implementation? Based on that we can decide. > > > OK, let's gather ideas from HW implementation. Any inputs from Mellanox or other vendors on the "width" of the type and size of "rte_regex_match" considering the performance implications. > > > > > uint8_t nb_actual_matches =3D> uint64 nb_actual_matches > > > uint8_t nb_matches =3D> uint64 nb__matches > > > > 2^64 matches will be never possible in practical system. How about 2^16= . > > > I think the number of matches depends on the number of total rules and sc= an > size. Based on the definitions (16-bit nb_rules_per_group, 16-bit nb_grou= ps and > 16-bit scan size), the maximum possible matches could exceed 2^16. Users = may > get partial matches in this case while Hyperscan doesn't make compromises= . > It'll also be good to check other HW implementation. See above. > > > > > > > > b) struct rte_regex_match: > > > uint16_t offset =3D> uint32_t offset > > > uint16_t len =3D> uint32_t len > > > > See above. > > > > > > > > c) uint16_t > > > rte_regex_rule_db_update(uint8_t dev_id, const struct > > > rte_regex_rule *rules, > > > uint16_t nb_rules); > > > =3D> > > > uint32_t > > > rte_regex_rule_db_update(uint8_t dev_id, const struct > > > rte_regex_rule *rules, > > > uint32_t nb_rules); > > > > OK. I will change it next version. > > > > > > > > d) int > > > rte_regex_queue_pair_setup(uint8_t dev_id, uint8_t queue_pair_id, > > > const struct rte_regex_qp_conf *qp_conf); > > > =3D> > > > int > > > rte_regex_queue_pair_setup(uint8_t dev_id, uint16_t queue_pair_id= , > > > const struct rte_regex_qp_conf *qp_conf); > > > > OK. I will change it next version. > > > > > > > > e) struct rte_regex_dev_config: > > > uint8_t nb_max_matches =3D> uint64_t nb_max_matches > > > > 2^64 matches will be never possible in practical system. How about 2^16= . > > > See above. > > > > > > > > f) struct rte_regex_dev_info: > > > uint8_t max_matches =3D> uint64_t max_matches > > > > 2^64 matches will be never possible in practical system. How about 2^16= . > > > See above. > > > > > > > > 2) There are rte_regex_dev_attr_get() and rte_regex_dev_attr_set() > defined. > > > Are all the attributes below could be set by users? Is any of them re= ad-only? > > > > See below, > > > > > /** Enumerates RegEx device attribute identifier */ enum > > > rte_regex_dev_attr_id { > > > RTE_REGEX_DEV_ATTR_SOCKET_ID, > > > /**< The NUMA socket id to which the device is connected or > > > * a default of zero if the socket could not be determined. > > > * datatype: *int* > > > * operation: *get* > > > > *get* means read only. *get* and *set* means it support both > > operation > > > > > */ > > > RTE_REGEX_DEV_ATTR_MAX_MATCHES, > > > /**< Maximum number of matches per scan. > > > * datatype: *uint8_t* > > > * operation: *get* and *set* > > > * > > > * @see RTE_REGEX_OPS_RSP_MAX_MATCH_F > > > */ > > > RTE_REGEX_DEV_ATTR_MAX_SCAN_TIMEOUT, > > > /**< Upper bound scan time in ns. > > > * datatype: *uint16_t* > > > * operation: *get* and *set* > > > * > > > * @see RTE_REGEX_OPS_RSP_MAX_SCAN_TIMEOUT_F > > > */ > > > RTE_REGEX_DEV_ATTR_MAX_PREFIX, > > > /**< Maximum number of prefix detected per scan. > > > * This would be useful for denial of service detection. > > > * datatype: *uint16_t* > > > * operation: *get* and *set* > > > * > > > * @see RTE_REGEX_OPS_RSP_MAX_PREFIX_F > > > */ > > > }; > > > > > > 3) Both RTE_REGEX_PCRE_RULE_* and > > > RTE_REGEX_DEV_PCRE_UNSUP_* can be viewed as device capabilities. Can > > > we merge them with RTE_REGEX_DEV_CAPA_RUNTIME_COMPILATION_F > and have > > > a unified regex_dev_capa in struct rte_regex_dev_info. > > > > Sure. I will fix it next version. > > > > > > > > > > > 4) It'll be good if we can also define synchronous matching API for > > > users who want to have a one-off scan and wait for the results. > > > > Makes sense. I will add synchronous matching API in next version(I > > understand, it will be useful for SW Implementations). Probably expose = as > INFO flag to expose the it as preference. > > > > > > > > On Tue, Sep 10, 2019 at 08:05:39AM +0000, Jerin Jacob Kollanukkaran > wrote: > > > > Hi Xiang, > > > > > > > > Sorry for delay in response(Was busy with 19.11 proposal > > > > deadline). Please > > > see inline. > > > > > > > > > > > > > > Reply to Xiang's queries in main thread: > > > > > > > > > > Hi all, > > > > > > > > > > Some questions regarding APIs. Could you please give more insight= s? > > > > > > > > > > 1) rte_regex_ops > > > > > a) rsp_flags > > > > > These two flags RTE_REGEX_OPS_RSP_PMI_SOJ_F and > > > > > RTE_REGEX_OPS_RSP_PMI_EOJ_F are used for cross buffer scan. > > > > > RTE_REGEX_OPS_RSP_PMI_EOJ_F tells whether we have a > > > > > partial match at the end of current buffer after scan. > > > > > What's the purpose of having RTE_REGEX_OPS_RSP_PMI_SOJ_F? > > > > > > > > > > [Jerin] Since we need three states to represent partial match > > > > > buffer, RTE_REGEX_OPS_RSP_PMI_SOJ_F to represent start of the > > > > > buffer, intermediate buffers with no flag, and end of the buffer > > > > > with RTE_REGEX_OPS_RSP_PMI_EOJ > > > > > > > > > [Xiang] How could a user leverage these flags for matching? > > > > > Suppose a large buffer is divided into multiple chunks. Will > > > > > RTE_REGEX_OPS_RSP_PMI_SOJ_F cause an early quit once it isn't > > > > > set after scan the first chunk. Similarly, > > > > > RTE_REGEX_OPS_RSP_PMI_EOJ tells a user whether to stop matching > > > > > future buffers after finish the last > > > chunk? > > > > > > > > Let me describe with an example, > > > > > > > > Assume, > > > > 1) struct rte_regex_dev_info:: max_payload_size set to 1024 > > > > 2) rte_regex_dev_config:: dev_cfg_flags configured with > > > > RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_F > > > > 3) Device programmed with matching "hello\s+world" pattern > > > > 4) user enqueue struct rte_regex_ops:: buf_addr point following "da= ta" > > > > and struct rte_regex_op:: scan_size =3D 1024 > > > > > > > > data[0..1021] =3D data don???t have hello world pattern data[1022] = =3D 'h' > > > > data[1023] =3D 'e' > > > > > > > > 5) user enqueue struct rte_regex_ops:: buf_addr point following "da= ta" > > > > and struct rte_regex_op:: scan_size =3D 9 > > > > > > > > data[0] =3D 'l' > > > > data[1] =3D 'l' > > > > data[2] =3D 'o' > > > > data[3] =3D ' ' > > > > data[4] =3D 'w' > > > > data[5] =3D 'o' > > > > data[6] =3D 'r' > > > > data[7] =3D 'l' > > > > data[8] =3D 'd' > > > > > > > > If so, > > > > > > > > Response to 4) will be RTE_REGEX_OPS_RSP_PMI_SOJ_F in > rte_regex_ops:: > > > > rsp_flags on dequeue Where rte_regex_match:: offset is 1022 and > > > > len 2 > > > > > > > > Response to 5) will be RTE_REGEX_OPS_RSP_PMI_EOJ_F in > rte_regex_ops:: > > > > rsp_flags on dequeue Where rte_regex_match:: offset is 0 and len 9 > > > > > > > If the defined pattern is "hello.*world" instead of "hello\s+world", > > > and we enqueue following struct rte_regex_ops: > > > > > > 1) rte_regex_op:: scan_size =3D 1024 > > > > > > data[0..1021] =3D data don???t have hello world pattern > > > data[1022] =3D 'h' > > > data[1023] =3D 'e' > > > > > > 2) rte_regex_op:: scan_size =3D 9 > > > data[0] =3D 'l' > > > data[1] =3D 'l' > > > data[2] =3D 'o' > > > data[3] =3D ' ' > > > data[4] =3D 'w' > > > data[5] =3D 'o' > > > data[6] =3D 'r' > > > data[7] =3D 'l' > > > data[8] =3D 'd' > > > > > > 3) rte_regex_op:: scan_size =3D 5 > > > data[0] =3D 'w' > > > data[1] =3D 'o' > > > data[2] =3D 'r' > > > data[3] =3D 'l' > > > data[4] =3D 'd' > > > > > > Will response to 3) have RTE_REGEX_OPS_RSP_PMI_EOJ_F in > rte_regex_ops:: > > > rsp_flags on dequeue > > > Where rte_regex_match:: offset is 0 and len 4? > > > > Yes. > > > > > > > > I am wondering what's your expected behavior for .* or similar > > > syntax and if there are syntax compatability issues. We report all ma= tches in > Hyperscan, e.g. > > > report end match offsets 11 and 16 for pattern "hello.*world" and > > > corpus "hello worldworld". > > > > > > BTW, not sure how other hardware devices handle cross buffer scan. > > > Hyperscan doesn't reports matches for start and intermediate buffers > > > but only reports end offset if a full match is found. > > > > > > > > > > > > > > > > > RTE_REGEX_OPS_RSP_MAX_PREFIX_F: This looks like a > > > > > definition for a specific hardware implementation. I am > > > > > wondering what this PREFIX refers to:)? > > > > > > > > > > [Jerin] Yes. Looks like it is for hardware specific implementatio= n. > > > > > Introduced rte_regex_dev_attr_set/get functions to make it > > > > > portable and To add new implementation specific fields. > > > > > For example, if a rule is > > > > > /ABCDEF.*XYZ/, ABCD is considered the prefix, and EF.*XYZ is > > > > > considered the factor. The prefix is a literal string, while the > > > > > factor can contain complex regular expression constructs. As a > > > > > result, rule matching occurs in two stages: prefix matching and > > > > > factor matching. > > > > > > > > > > b) user_id or user_ptr > > > > > Under what kind of circumstances should an application > > > > > pass value into these variables for enqueue and dequeuer operatio= ns? > > > > > > > > > > [Jerin] Just like rte_crypto_ops, struct rte_regex_ops also > > > > > allocated using mempool normally, on enqueue, user can specify > > > > > user_id If needed to in order identify the op on dequeue if > > > > > required. The use case could be to store the sequence number > > > > > from application POV or storing the mbuf ptr in which pattern is > requested etc. > > > > > > > > > > > > > > > 2) rte_regex_match > > > > > a) offset; /**< Starting Byte Position for matched rule. > > > > > */ and uint16_t len; /**< Length of match in bytes */ > > > > > Looks like the matching offset is defined as *starting > > > > > matching offset* instead of *end matching offset*, e.g. report > > > > > the offset of > > > "a" instead of "c" > > > > > for pattern "abc". > > > > > If so, this makes it hard to integrate software regex > > > > > libraries such as Hyperscan and RE2 as they only report *end > > > > > matching offset* without length of match. > > > > > Although Hyperscan has API for *starting matching offset*, > > > > > it only delivers partial syntax support. So I think we have to > > > > > define *end of matching offset* for software solutions. > > > > > > > > > > [Jerin] I understand the hyperscan's HS_FLAG_SOM_LEFTMOST > tradeoffs. > > > > > I thought application would need always the length of the match. > > > > > Probably we will see how other HW implementation (from Mellanox) > > > > > etc. We will try to abstract it, probably we can make it as > > > > > function of "user requested". > > > > > [Xiang] Yes, it will be good to make it per user request. At > > > > > least from Hyperscan user's point of view, start of match and > > > > > match length are not mandatory. > > > > > > > > OK. I think, we can introduce RTE_REGEX_DEV_CFG_MATCH_AS_START In > > > > device configure. > > > > > > > > Since offset+len =3D=3D end, we can introduce following generic inl= ine > function. > > > > > > > > static inline > > > > rte_regex_match_end(truct rte_regex_match *match) { > > > > match->offset + match->len; > > > > } > > > > > > > > Example: pattern to match is "hello\s+world" and data is > > > > following data[4] =3D 'h' > > > > data[5] =3D 'e' > > > > data[6] =3D 'l' > > > > data[7] =3D 'l' > > > > data[8] =3D 'o' > > > > data[9] =3D ' ' > > > > data[10] =3D 'w' > > > > data[11] =3D 'o' > > > > data[12] =3D 'r' > > > > data[13] =3D 'l' > > > > data[14] =3D 'd' > > > > > > > > if device is configured with RTE_REGEX_DEV_CFG_MATCH_AS_START > > > > match->offset returns 4 > > > > match->len returns 11 > > > > > > > > if device is NOT configured with RTE_REGEX_DEV_CFG_MATCH_AS_START > > > > driver MAY return the following(in hyperscan case) > > > > match->offset returns 0 > > > > match->len returns 11 + 4 > > > > > > > > In both case(irrespective of flags, to make application life easy) > > > rte_regex_match_end() would return 15. > > > > If application demands for MATCH_AS_START then driver can return > > > > match->offset returns 4 and match->len returns 11 Aka set > > > > HS_FLAG_SOM_LEFTMOST in hyperscan driver, But application should > > > > use > > > rte_regex_match_end() for finding the end of the match. To make, > > > work in all cases. > > > > > > > > Is it OK? > > > > > > > Can we replace len with end offset? So we can change "offset" to > "start_offset" > > > and len to "end_ offset" in struct rte_regex_match. Users interested > > > in len could take "end_offset - start_offset". > > > We may also change RTE_REGEX_DEV_CFG_MATCH_AS_START to > > > RTE_REGEX_DEV_CFG_MATCH_START > > > > > > In your example, > > > if device is configured with RTE_REGEX_DEV_CFG_MATCH_START > > > match->start_offset returns 4 > > > match->end_offset returns 15 > > > > > > if device is NOT configured with RTE_REGEX_DEV_CFG_MATCH_START > > > match->start_offset returns 0 > > > match->end_offset returns 15 > > > > > > This part is little tricky as HW descriptions need to be rewritten on r= esponse. > > This is a one issue, I foresee earlier, to come up with > > rte_regex_match That's works for all implementation without performanc= e > issue. > > > > We have two HW implementations, both returns start_off and len. > > Lets get input from other HW implementation on the semantics of > > rte_regex_match. Based on that, we can decide how to go about it? > > Thoughts from Mellanox or other vendors? > > > Sure. Let's get more inputs on this. > > > > > > > > > > > > > > > > > 3) rte_regex_rule_db_update() > > > > > Does this mean we can dynamically add or delete rules for an > > > > > already generated database without recompile from scratch for > > > > > hardware Regex implementation? > > > > > If so, this isn't possible for software solutions as they > > > > > don't support dynamic database update and require recompile. > > > > > > > > > > [Jerin] rte_regex_rule_db_update() internally it would call > > > > > recompile function for both HW and SW. > > > > > See rte_regex_dev_config::rule_db in rte_regex_dev_configure() > > > > > for precompiled rule database case. > > > > > [Xiang] OK, sounds like we have to save the original rule-set > > > > > for the device in order to do recompile. I see both ADD and > > > > > REMOVE operators from rte_regex_rule. > > > > > For rules with REMOVE operator, what's the expected behavior to > > > > > handle them for the old rule-set? Do we need to go through the > > > > > old rule-set and remove corresponding rules before doing recompil= e? > > > > > > > > Yes. > > > > > > > I think it'll be better to change rte_regex_rule_db_update() to > > > rte_regex_rule_compile() and have users to provide a full rule-set. > > > So we don't have to maintain old rule-set and decide which one to > > > keep and remove. We can simply recompile new rule-set and get rid of > > > rte_regex_rule_op in this case. > > > > > > On virtualized, HW implementations, The RULE database is maintained by > > single body. So the above scheme, works with SW and HW implementations. > > And It make user life easy as they don't need to maintain the rules. > > > > I don't have preference on the rte_regex_rule_db_update() name, I can > > change to > > rte_regex_rule_compile() if required keeping above functionality. Let m= e > know. > > > > > OK, I'm good if your are willing to maintain it for users. Then both > rte_regex_rule_db_update() and rte_regex_rule_compile() work for me. > > > > > > > > > > > >