From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 32E9DA0559; Mon, 16 Mar 2020 14:49:55 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 592D21BE51; Mon, 16 Mar 2020 14:49:54 +0100 (CET) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2058.outbound.protection.outlook.com [40.107.22.58]) by dpdk.org (Postfix) with ESMTP id E61D12BF9 for ; Mon, 16 Mar 2020 14:49:52 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RZ1FiL/2wfjUz6P+dV6U6QVwZaU8Qol4MSjLcLDhrcDtuWCYbqZ8RVlR4gjrBOmVzDWsAMD91mtIiTbEh6NENz/i9iYjc7xRWmHa5q++1ysJYgfKV0LaWsFDbndxhLZCWRC86jnTpy1weeYhC92x3jCWIuO5nRl057nCustcpOYh9slqhHjAjP3zbJm1mdnJ2Rgk3YXxANUEw+gKaN3c3c2ZiTPlDKAOtupC7pIVLIJaAr3Q/EcuTQqPUYRU1TjrgWe0geiXkkjmmJVPOhs7WS2Q2sEQXwAeYSJAv+Ba2kKF4FuLw+w3g3FrHf8Okya6rN0TeALjQE4SICSd1jBeIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+8std+nZZy/y3VS1vRznCgY0d3b87oQDUBDS5FwsGlY=; b=bZPSnGWPgrfSKNApy1m4crwbsDHudZggUXID6g7G7vQdfvX2osVuHyJS2FhSQwCrmtmyF+6/KFEOI8p2mRnZh1q7fRq71aOwP3cKXVHm+WirXaPxYxPMYtTzzZK8Bihp/fvyT27Y/+zZWG7EkHi5h2aGB0bgEbGNlo39/zp/6ZvSpUWs2/mwOxlwvC59JSbuDNS89ccte7ZqRevWcVw0bQ+i2g+6/UzbLJTuWfNqsTMV+R418/Tt9dYymxMCI3cy7noO9CD+lE7lFLou5RAVUT0oUYsis3J4eAlpMUnq7/FQM7RAirn9+VU046MiON/x4G/POh9cjZEDJ7sSurVmVA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+8std+nZZy/y3VS1vRznCgY0d3b87oQDUBDS5FwsGlY=; b=pssm5GkS1VzVUXBsKA4nxEcSmRtajM8eeOp4zhDg7ltwehKq13VhTmVl7gZRyUTBhme1wNkwFhdal0oFywgevCStu1V0MhVQvxRGlU+lLpIby1QuZuyBulAibaGb0F1Mdmg08QImcfDzu3a+BrZtIgRPBdPUnM9Q6BHpZMwhUDk= Received: from AM6PR05MB5176.eurprd05.prod.outlook.com (20.177.196.158) by AM6PR05MB6327.eurprd05.prod.outlook.com (20.179.6.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.19; Mon, 16 Mar 2020 13:49:51 +0000 Received: from AM6PR05MB5176.eurprd05.prod.outlook.com ([fe80::80e9:7eb9:e770:941d]) by AM6PR05MB5176.eurprd05.prod.outlook.com ([fe80::80e9:7eb9:e770:941d%7]) with mapi id 15.20.2814.021; Mon, 16 Mar 2020 13:49:51 +0000 From: Ori Kam To: Wang Xiang CC: "jerinj@marvell.com" , "dev@dpdk.org" , "pbhagavatula@marvell.com" , Shahaf Shuler , "hemant.agrawal@nxp.com" , Opher Reviv , Alex Rosenbaum , "dovrat@marvell.com" , "pkapoor@marvell.com" , "nipun.gupta@nxp.com" , "bruce.richardson@intel.com" , "yang.a.hong@intel.com" , "harry.chang@intel.com" , "gu.jian1@zte.com.cn" , "shanjiangh@chinatelecom.cn" , "zhangy.yun@chinatelecom.cn" , "lixingfu@huachentel.com" , "wushuai@inspur.com" , "yuyingxia@yxlink.com" , "fanchenggang@sunyainfo.com" , "davidfgao@tencent.com" , "liuzhong1@chinaunicom.cn" , "zhaoyong11@huawei.com" , "oc@yunify.com" , "jim@netgate.com" , "hongjun.ni@intel.com" , "j.bromhead@titan-ic.com" , "deri@ntop.org" , "fc@napatech.com" , "arthur.su@lionic.com" , Thomas Monjalon Thread-Topic: [RFC v6] regexdev: introduce regexdev subsystem Thread-Index: AQHV+NWNYfm2dVQnR0mFBaqxQW35zqhJXPxQgAEU9gCAAHaisIAAzjiA//+H5WA= Date: Mon, 16 Mar 2020 13:49:51 +0000 Message-ID: References: <20190627155036.56940-1-jerinj@marvell.com> <1583836353-42867-1-git-send-email-orika@mellanox.com> <20200313012027.GA25215@hyperscan> <20200316012542.GB25215@hyperscan> <20200316204823.GA16186@hs1> In-Reply-To: <20200316204823.GA16186@hs1> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=orika@mellanox.com; x-originating-ip: [185.175.35.255] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: e57923c0-dc82-4f9c-4f23-08d7c9b0e68c x-ms-traffictypediagnostic: AM6PR05MB6327:|AM6PR05MB6327: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 03449D5DD1 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(376002)(396003)(136003)(346002)(366004)(39860400002)(199004)(9686003)(55016002)(478600001)(81166006)(81156014)(5660300002)(8936002)(54906003)(71200400001)(316002)(7696005)(186003)(26005)(2906002)(86362001)(6506007)(4326008)(53546011)(52536014)(6916009)(33656002)(7416002)(76116006)(66946007)(64756008)(66556008)(66476007)(66446008)(8676002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR05MB6327; H:AM6PR05MB5176.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: NXkH6IO3bKkWmjDB0aGBt+ld7DfWQwMbhhQJfBoP+vfNiUxXX7Sn+jKY+HQxkagGG8fFEkSCavEf0tjtLCS0nHaqtjsXG5hco8d5OWwjqZ1zNbNZkE4wlK1Jj/V5o89TZfT9UG+RujUv2z68sy7ArpVvPq5rYflQHgsPwPEfUhOW6eVDgHorQBTnlzbmU4yFucjvRcTeWDXsVjlp7g4VjWMxZnj3L2PK69FITmOvv2YwV3LM8RkaZn9z5fECkwyeME0xpyyK7lc7l0u8+DhsonRgLgZKsSBC6bP4KxNstkGHzQ4iO+x6n3+I0tDO9XMZ5IjvCJsoGF7B77VIaQhv6gkh43uRf24riuSDU5jvfVJlK8tntBSrgZQsMOzoAeEzInxhzmPpNrPKnTrlt5vn8t3ys2HiIWkXxcgzwl690TdRSa5Y/N+MSdXH9Ebv+JcZ x-ms-exchange-antispam-messagedata: 6xEuWFUaq0Ocds+nwMFawHxQ2zGLYAF9RtYJZ38bh4p7N9B+oaJGPzwq4Df4eIGe+0W+1bSCmRbckTaUoxp3LiDiIzC+zFsU/GsIxSlMF6jLiRunk/O0UraYxyWahwQz+9tkvAMUcDOStrti0tkBxA== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: e57923c0-dc82-4f9c-4f23-08d7c9b0e68c X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Mar 2020 13:49:51.3730 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 0h0eOjBRWzO2gV8T85hcbZSyLsnONWX1dwKZLvzbarFa3H1+PclpFkb/BfFRRTroIJ0j+SJXvmPLPZqdZaDjEw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR05MB6327 Subject: Re: [dpdk-dev] [RFC v6] regexdev: introduce regexdev subsystem X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Wang, PSB, if you don't have any objections and other comments,=20 I will start working on the class and will address all of this thread comme= nts=20 in the v1 patch, Thanks, Ori=20 > -----Original Message----- > From: Wang Xiang > Sent: Monday, March 16, 2020 10:48 PM > To: Ori Kam > Cc: jerinj@marvell.com; dev@dpdk.org; pbhagavatula@marvell.com; Shahaf > Shuler ; hemant.agrawal@nxp.com; Opher Reviv > ; Alex Rosenbaum ; > dovrat@marvell.com; pkapoor@marvell.com; nipun.gupta@nxp.com; > bruce.richardson@intel.com; yang.a.hong@intel.com; harry.chang@intel.com; > gu.jian1@zte.com.cn; shanjiangh@chinatelecom.cn; > zhangy.yun@chinatelecom.cn; lixingfu@huachentel.com; wushuai@inspur.com; > yuyingxia@yxlink.com; fanchenggang@sunyainfo.com; > davidfgao@tencent.com; liuzhong1@chinaunicom.cn; > zhaoyong11@huawei.com; oc@yunify.com; jim@netgate.com; > hongjun.ni@intel.com; j.bromhead@titan-ic.com; deri@ntop.org; > fc@napatech.com; arthur.su@lionic.com; Thomas Monjalon > > Subject: Re: [RFC v6] regexdev: introduce regexdev subsystem >=20 > Hi Ori, >=20 > On Mon, Mar 16, 2020 at 09:09:06AM +0000, Ori Kam wrote: > > Hi Xiang, > > > > > -----Original Message----- > > > From: Wang Xiang > > > Sent: Monday, March 16, 2020 3:26 AM > > > To: Ori Kam > > > Cc: jerinj@marvell.com; dev@dpdk.org; pbhagavatula@marvell.com; > Shahaf > > > Shuler ; hemant.agrawal@nxp.com; Opher Reviv > > > ; Alex Rosenbaum ; > > > dovrat@marvell.com; pkapoor@marvell.com; nipun.gupta@nxp.com; > > > bruce.richardson@intel.com; yang.a.hong@intel.com; > harry.chang@intel.com; > > > gu.jian1@zte.com.cn; shanjiangh@chinatelecom.cn; > > > zhangy.yun@chinatelecom.cn; lixingfu@huachentel.com; > wushuai@inspur.com; > > > yuyingxia@yxlink.com; fanchenggang@sunyainfo.com; > > > davidfgao@tencent.com; liuzhong1@chinaunicom.cn; > > > zhaoyong11@huawei.com; oc@yunify.com; jim@netgate.com; > > > hongjun.ni@intel.com; j.bromhead@titan-ic.com; deri@ntop.org; > > > fc@napatech.com; arthur.su@lionic.com; Thomas Monjalon > > > > > > Subject: Re: [RFC v6] regexdev: introduce regexdev subsystem > > > > > > On Sun, Mar 15, 2020 at 10:05:53AM +0000, Ori Kam wrote: > > > Hi Ori, > > > > > > > Hi Xiang, > > > > > > > > > > > > > -----Original Message----- > > > > > From: Wang Xiang > > > > > Sent: Friday, March 13, 2020 3:20 AM > > > > > To: Ori Kam > > > > > Cc: jerinj@marvell.com; dev@dpdk.org; pbhagavatula@marvell.com; > > > Shahaf > > > > > Shuler ; hemant.agrawal@nxp.com; Opher > Reviv > > > > > ; Alex Rosenbaum ; > > > > > dovrat@marvell.com; pkapoor@marvell.com; nipun.gupta@nxp.com; > > > > > bruce.richardson@intel.com; yang.a.hong@intel.com; > > > harry.chang@intel.com; > > > > > gu.jian1@zte.com.cn; shanjiangh@chinatelecom.cn; > > > > > zhangy.yun@chinatelecom.cn; lixingfu@huachentel.com; > > > wushuai@inspur.com; > > > > > yuyingxia@yxlink.com; fanchenggang@sunyainfo.com; > > > > > davidfgao@tencent.com; liuzhong1@chinaunicom.cn; > > > > > zhaoyong11@huawei.com; oc@yunify.com; jim@netgate.com; > > > > > hongjun.ni@intel.com; j.bromhead@titan-ic.com; deri@ntop.org; > > > > > fc@napatech.com; arthur.su@lionic.com; Thomas Monjalon > > > > > > > > > > Subject: Re: [RFC v6] regexdev: introduce regexdev subsystem > > > > > > > > > > Hi Ori, > > > > > > > > > > Sorry for the late response as I am occupied by other works. > > > > > Two comments below to make the definitions compatible to Hypersca= n. > > > > > > > > > > Thanks, > > > > > Xiang > > > > > > > > > > On Tue, Mar 10, 2020 at 10:32:33AM +0000, Ori Kam wrote: > > > > > > +#define RTE_REGEX_PCRE_RULE_MATCH_ALL_F (1ULL << 13) > > > > > > +/**< This flag marks that the results for the pattern that is = being > > > compiled > > > > > > + * should include all possible matches. > > > > > > + * @see struct rte_regex_dev_info::rule_flags, struct > > > > > rte_regex_rule::rule_flags > > > > > > + */ > > > > > > + > > > > > Can we change this flag to RTE_REGEX_DEV_CFG_MATCH_ALL since > > > Hyperscan > > > > > only supports > > > > > match all mode and users don't have to specify this flag per rule= ? > > > > > > > > > > > > > Sure, we can replace the RTE_REGEX_PCRE_RULE_MATCH_ALL_F with > > > > RTE_REGEX_DEV_CFG_MATCH_ALL, and add > > > RTE_REGEX_DEV_CAPA_SUPP_MATCH_ALL > > > > > > > Ack, thanks. > > > > > > > > > > + */ > > > > > > +__rte_experimental > > > > > > +int > > > > > > +rte_regex_dev_info_get(uint8_t dev_id, struct rte_regex_dev_in= fo > > > > > *dev_info); > > > > > > + > > > > > > +/* Enumerates RegEx device configuration flags */ > > > > > > +#define RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_F (1ULL << 0) > > > > > > +/**< Cross buffer scan refers to the ability to be able to det= ect > > > > > > + * matches that occur across buffer boundaries, where the buff= ers > are > > > > > related > > > > > > + * to each other in some way. Enable this flag when to scan pa= yload > size > > > > > > + * greater than struct rte_regex_dev_info::max_payload_size an= d/or > > > > > > + * matches can present across scan buffer boundaries. > > > > > > + * > > > > > > + * @see struct rte_regex_dev_info::max_payload_size > > > > > > + * @see struct rte_regex_dev_config::dev_cfg_flags, > > > > > rte_regex_dev_configure() > > > > > > + * @see RTE_REGEX_OPS_RSP_PMI_SOJ_F > > > > > > + * @see RTE_REGEX_OPS_RSP_PMI_EOJ_F > > > > > > + * @see RTE_REGEX_OPS_RSP_PMI_TOJ_F > > > > > > + */ > > > > > > + > > > > > Can we add another flag > > > > > RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_FULL_F? In this case, > > > > > we only return full match for cross buffer scan without any parti= al result > > > and > > > > > without returning response flags such as RTE_REGEX_OPS_RSP_PMI_*. > > > > > > > > I think that it is good in any case to return a flag if the detecti= on was > based on > > > > more than one buffer. > > > > So I don't really see the advantage of adding such a flag. > > > > As far as I understand in your case if the match started in previou= s buffer > and > > > ended > > > > in the current buffer then you will return also the flag of > > > RTE_REGEX_OPS_RSP_PMI_TOJ_F > > > > For my general knowledge, in your system if we have the following r= egex: > > > ABC > > > > In the first buffer we have xxxA size 4 and the second buffer is BC= xx > > > > If I understand correctly for first buffer you will return no match= found. > > > > For the second buffer you will return found and end offset will be = equal to > 2 > > > > Am I correct? > > > > Or you are going to return end offset 6 because it started from the > previous > > > buffer? > > > > > > > Hyperscan guarantees the same matching result regardless of the data = is in > a > > > single > > > block or scattered to multiple blocks. So we'll return end offset 6 i= n this > case > > > without giving any flag indicating whether the match is started in pr= evious > > > buffer > > > or current buffer. > > > > What will happen if the match was only in the second buffer? For exampl= e > > Like before the regex is ABC but now the first buffer is xxxx and the s= econd > buffer > > is ABCx will the result be end offset 3 or 7? > > If the answer is 3 than I think the flag is important, in order to let = the user > know > > that he should count from previous buffer. > > If the answer is 7, since only Hyperscan works with end offset if could= be > defined > > that when working with end offset and cross buffer scan is supported th= en the > > result is always true result. > > > > So I think that RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_FULL_F is not > relevant in any > > case but the flag should be used if the offset returned is 3. > > > Hyperscan returns 7 in this case, so these flags aren't necessary. >=20 > Hyperscan works in two modes: > 1) return start and end offset > 2) return end offset >=20 > Since only Hyperscan supports RTE_REGEX_DEV_CFG_MATCH_ALL, we can > define > the result always true if match all and cross buffer scan are > configured. Having the scan full flag will make users better aware of > the difference from HW solutions. If you really don't want keep this flag= , > please make this definition clear to users. The issue with the new flag is that it should always be set, so it is redun= dant if I understand correctly. I will try to make it clearer in the comment. > > > > In other related question, how do Hyperscan marks that 2 buffers should= be > treated as one? > > I think you are missing the cross_buf_id that was introduced in V3 but = was > removed due to > > lack of usage. This variable was designed to be used in order to let th= e RegEx > engine a place > > to save the engine state. > > > I agree, we need to have the cross_buf_id back to support cross buffer > scan. I will re-add it. > > > > > > > > Best, > > > > Ori > > > > > > > > > > Best, > > > Xiang >=20 > Thanks, > Xiang