From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8D6AFA0559; Mon, 16 Mar 2020 10:09:11 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9C6A61C02A; Mon, 16 Mar 2020 10:09:10 +0100 (CET) Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40046.outbound.protection.outlook.com [40.107.4.46]) by dpdk.org (Postfix) with ESMTP id AB4F5FEB for ; Mon, 16 Mar 2020 10:09:09 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F8F0lnk9tG65+iknXoRTcBihMAw2pPPMn7Fs9UgeWUpeiTDmq82H90agXTHMeyBdQ3173qu9QnVi//n8A2eK9EzL6Cv2VXa06PVmHHeDdiGNMkMyDTz18K4zx9aFEjAnC5Jn/ueAJrCL4DIYzRWCtO4P+D9YuXt/62wfF+8UEJMriXllK1VqydDcCyjp+ZJ9hz7uXS0Jn2vJu1kbqYdXvRabudVivT4Y1ClmXKOTUub/JMB/LUFnm9kmXyWn246/JpJ2fBKCtwD30Ruq7bXF9COPS4xG9GlTc/3hqdzVnlyR73Jzu+0VxFVJlkkEZ8QLSOWORZbTT4pP7D+/64SEug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AuGtrGsZk+fcAEyjI63KtqwOOCWrNZznsdZ4zlibSYw=; b=SMk+EIPS6JIN3IaBwemzEbyC6SC4VPfiSn76t9O6wD6RJANNzsmbnljRRKcHu1b+HbuFZQ0xEl5XWPomkktqRBHOGjn8YioEOG8w6YkMI+Ts48EsYxYn+4c+Vk2vnuFBivskS46Ls15oQerjO4Lqt0WPAlDZX0XdtuuOpQ/almdg86+tunAlH1D6g+1YK1BjQmK+KMtrgGhleeyzRehcOGUcCv3dw/OEI2DfBHE8w0lNNijFmiNQ7M4KrFBgLHrjfjacWO3WE8IZH8T9VCDMwvLktrRxXLyyyOUZBBMij2mo5y5VSEjx2IVGGa96IP/bvoeBNe4SFcJMkgD0XaQxMQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AuGtrGsZk+fcAEyjI63KtqwOOCWrNZznsdZ4zlibSYw=; b=p75pt/1zGfkSAamH8LsbE4p5bLCo/F0F3MJQqjPPNVEwGb1NAYE4FkQswKc8CF5AEUYQW8fgD4xX9emqX39WWn7mEzwdhuuLWO+SPygmb53UOE0plcIbQvwIyd5QYyTwaXu2S/mJiiv3JYDcYQNl4j+6jPVk29OcC5PDMav4Au0= Received: from AM6PR05MB5176.eurprd05.prod.outlook.com (20.177.196.158) by AM6PR05MB5768.eurprd05.prod.outlook.com (20.178.93.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.16; Mon, 16 Mar 2020 09:09:07 +0000 Received: from AM6PR05MB5176.eurprd05.prod.outlook.com ([fe80::80e9:7eb9:e770:941d]) by AM6PR05MB5176.eurprd05.prod.outlook.com ([fe80::80e9:7eb9:e770:941d%7]) with mapi id 15.20.2814.021; Mon, 16 Mar 2020 09:09:07 +0000 From: Ori Kam To: Wang Xiang CC: "jerinj@marvell.com" , "dev@dpdk.org" , "pbhagavatula@marvell.com" , Shahaf Shuler , "hemant.agrawal@nxp.com" , Opher Reviv , Alex Rosenbaum , "dovrat@marvell.com" , "pkapoor@marvell.com" , "nipun.gupta@nxp.com" , "bruce.richardson@intel.com" , "yang.a.hong@intel.com" , "harry.chang@intel.com" , "gu.jian1@zte.com.cn" , "shanjiangh@chinatelecom.cn" , "zhangy.yun@chinatelecom.cn" , "lixingfu@huachentel.com" , "wushuai@inspur.com" , "yuyingxia@yxlink.com" , "fanchenggang@sunyainfo.com" , "davidfgao@tencent.com" , "liuzhong1@chinaunicom.cn" , "zhaoyong11@huawei.com" , "oc@yunify.com" , "jim@netgate.com" , "hongjun.ni@intel.com" , "j.bromhead@titan-ic.com" , "deri@ntop.org" , "fc@napatech.com" , "arthur.su@lionic.com" , Thomas Monjalon Thread-Topic: [RFC v6] regexdev: introduce regexdev subsystem Thread-Index: AQHV+NWNYfm2dVQnR0mFBaqxQW35zqhJXPxQgAEU9gCAAHaisA== Date: Mon, 16 Mar 2020 09:09:06 +0000 Message-ID: References: <20190627155036.56940-1-jerinj@marvell.com> <1583836353-42867-1-git-send-email-orika@mellanox.com> <20200313012027.GA25215@hyperscan> <20200316012542.GB25215@hyperscan> In-Reply-To: <20200316012542.GB25215@hyperscan> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=orika@mellanox.com; x-originating-ip: [185.175.35.255] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 63bce4e2-12cb-41b5-a85f-08d7c989ae76 x-ms-traffictypediagnostic: AM6PR05MB5768:|AM6PR05MB5768: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 03449D5DD1 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(346002)(396003)(366004)(39860400002)(136003)(376002)(199004)(478600001)(7416002)(8676002)(7696005)(53546011)(6506007)(52536014)(5660300002)(76116006)(64756008)(66946007)(66556008)(66476007)(66446008)(54906003)(316002)(186003)(86362001)(26005)(2906002)(4326008)(6916009)(71200400001)(9686003)(55016002)(81166006)(8936002)(81156014)(33656002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR05MB5768; H:AM6PR05MB5176.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: hMKiE4v4jZtzzKy1z9yWqygtT3L+5rqF5hGkqgREUVO7DB3IVoOHWgM/KIFOUa2Yf6wXFRCElNh8fkfFxmWMmoB+AhT2Ix9zpVoETntIqwb3vss9kXbzFg6YsiykOtp3fzOBO6YHr1ehSGpMNQn24CGyG7jZ7C3QGiqGvQpBpClDlUg4MrBAtLxekAB3f+PmDbZkWI8Io/vwKwqdG4h4va2zSVfwK3qGbIlOvTN4QMYWyMLhtabfyAamsn+vSBS44mTKY8usrsNjqcdyqUlSfh2OTN63T1wrLgEhNENuRHxUCncxB6kQdhktatL5qD4SUGG5NZoFJRngBhxVPTwlckK1fs91EfxLjJ6apHB54MONuc8wsYvUhJ12OLL7BwqnIxwovgTEHpKyyfTIPrYuSB1Y2VnXwqTcifZA2PcCtZM8YLOM5d8mOH78qsSe8yIE x-ms-exchange-antispam-messagedata: 6kzdCg+CZP40VBHJvJU3VeP5qJjdYTIoHjUIJMS7atrdopLYXNOQ1R+m9HW2mEMFUZbfltzz4919dBO0Urqkx5JXXceghxBA+0FchCCCwjfjDJhQ0lWKUFxxN/qWnra6SH1wYNF/Np7hOOqs0Cv6pQ== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 63bce4e2-12cb-41b5-a85f-08d7c989ae76 X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Mar 2020 09:09:06.9491 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 03eWm+zfszXqFVfUySjkiXeQ8c7jjFL4q9Xtx/RX6DJTgR1jNE40MA4wB5339VwUP2fjvEFr0hjUkOoyLBnrZg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR05MB5768 Subject: Re: [dpdk-dev] [RFC v6] regexdev: introduce regexdev subsystem X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Xiang, > -----Original Message----- > From: Wang Xiang > Sent: Monday, March 16, 2020 3:26 AM > To: Ori Kam > Cc: jerinj@marvell.com; dev@dpdk.org; pbhagavatula@marvell.com; Shahaf > Shuler ; hemant.agrawal@nxp.com; Opher Reviv > ; Alex Rosenbaum ; > dovrat@marvell.com; pkapoor@marvell.com; nipun.gupta@nxp.com; > bruce.richardson@intel.com; yang.a.hong@intel.com; harry.chang@intel.com; > gu.jian1@zte.com.cn; shanjiangh@chinatelecom.cn; > zhangy.yun@chinatelecom.cn; lixingfu@huachentel.com; wushuai@inspur.com; > yuyingxia@yxlink.com; fanchenggang@sunyainfo.com; > davidfgao@tencent.com; liuzhong1@chinaunicom.cn; > zhaoyong11@huawei.com; oc@yunify.com; jim@netgate.com; > hongjun.ni@intel.com; j.bromhead@titan-ic.com; deri@ntop.org; > fc@napatech.com; arthur.su@lionic.com; Thomas Monjalon > > Subject: Re: [RFC v6] regexdev: introduce regexdev subsystem >=20 > On Sun, Mar 15, 2020 at 10:05:53AM +0000, Ori Kam wrote: > Hi Ori, >=20 > > Hi Xiang, > > > > > > > -----Original Message----- > > > From: Wang Xiang > > > Sent: Friday, March 13, 2020 3:20 AM > > > To: Ori Kam > > > Cc: jerinj@marvell.com; dev@dpdk.org; pbhagavatula@marvell.com; > Shahaf > > > Shuler ; hemant.agrawal@nxp.com; Opher Reviv > > > ; Alex Rosenbaum ; > > > dovrat@marvell.com; pkapoor@marvell.com; nipun.gupta@nxp.com; > > > bruce.richardson@intel.com; yang.a.hong@intel.com; > harry.chang@intel.com; > > > gu.jian1@zte.com.cn; shanjiangh@chinatelecom.cn; > > > zhangy.yun@chinatelecom.cn; lixingfu@huachentel.com; > wushuai@inspur.com; > > > yuyingxia@yxlink.com; fanchenggang@sunyainfo.com; > > > davidfgao@tencent.com; liuzhong1@chinaunicom.cn; > > > zhaoyong11@huawei.com; oc@yunify.com; jim@netgate.com; > > > hongjun.ni@intel.com; j.bromhead@titan-ic.com; deri@ntop.org; > > > fc@napatech.com; arthur.su@lionic.com; Thomas Monjalon > > > > > > Subject: Re: [RFC v6] regexdev: introduce regexdev subsystem > > > > > > Hi Ori, > > > > > > Sorry for the late response as I am occupied by other works. > > > Two comments below to make the definitions compatible to Hyperscan. > > > > > > Thanks, > > > Xiang > > > > > > On Tue, Mar 10, 2020 at 10:32:33AM +0000, Ori Kam wrote: > > > > +#define RTE_REGEX_PCRE_RULE_MATCH_ALL_F (1ULL << 13) > > > > +/**< This flag marks that the results for the pattern that is bein= g > compiled > > > > + * should include all possible matches. > > > > + * @see struct rte_regex_dev_info::rule_flags, struct > > > rte_regex_rule::rule_flags > > > > + */ > > > > + > > > Can we change this flag to RTE_REGEX_DEV_CFG_MATCH_ALL since > Hyperscan > > > only supports > > > match all mode and users don't have to specify this flag per rule? > > > > > > > Sure, we can replace the RTE_REGEX_PCRE_RULE_MATCH_ALL_F with > > RTE_REGEX_DEV_CFG_MATCH_ALL, and add > RTE_REGEX_DEV_CAPA_SUPP_MATCH_ALL > > > Ack, thanks. > > > > > > + */ > > > > +__rte_experimental > > > > +int > > > > +rte_regex_dev_info_get(uint8_t dev_id, struct rte_regex_dev_info > > > *dev_info); > > > > + > > > > +/* Enumerates RegEx device configuration flags */ > > > > +#define RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_F (1ULL << 0) > > > > +/**< Cross buffer scan refers to the ability to be able to detect > > > > + * matches that occur across buffer boundaries, where the buffers = are > > > related > > > > + * to each other in some way. Enable this flag when to scan payloa= d size > > > > + * greater than struct rte_regex_dev_info::max_payload_size and/or > > > > + * matches can present across scan buffer boundaries. > > > > + * > > > > + * @see struct rte_regex_dev_info::max_payload_size > > > > + * @see struct rte_regex_dev_config::dev_cfg_flags, > > > rte_regex_dev_configure() > > > > + * @see RTE_REGEX_OPS_RSP_PMI_SOJ_F > > > > + * @see RTE_REGEX_OPS_RSP_PMI_EOJ_F > > > > + * @see RTE_REGEX_OPS_RSP_PMI_TOJ_F > > > > + */ > > > > + > > > Can we add another flag > > > RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_FULL_F? In this case, > > > we only return full match for cross buffer scan without any partial r= esult > and > > > without returning response flags such as RTE_REGEX_OPS_RSP_PMI_*. > > > > I think that it is good in any case to return a flag if the detection w= as based on > > more than one buffer. > > So I don't really see the advantage of adding such a flag. > > As far as I understand in your case if the match started in previous bu= ffer and > ended > > in the current buffer then you will return also the flag of > RTE_REGEX_OPS_RSP_PMI_TOJ_F > > For my general knowledge, in your system if we have the following regex= : > ABC > > In the first buffer we have xxxA size 4 and the second buffer is BCxx > > If I understand correctly for first buffer you will return no match fou= nd. > > For the second buffer you will return found and end offset will be equa= l to 2 > > Am I correct? > > Or you are going to return end offset 6 because it started from the pre= vious > buffer? > > > Hyperscan guarantees the same matching result regardless of the data is i= n a > single > block or scattered to multiple blocks. So we'll return end offset 6 in th= is case > without giving any flag indicating whether the match is started in previo= us > buffer > or current buffer. What will happen if the match was only in the second buffer? For example Like before the regex is ABC but now the first buffer is xxxx and the secon= d buffer is ABCx will the result be end offset 3 or 7? If the answer is 3 than I think the flag is important, in order to let the = user know that he should count from previous buffer. If the answer is 7, since only Hyperscan works with end offset if could be = defined that when working with end offset and cross buffer scan is supported then t= he result is always true result. So I think that RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_FULL_F is not relevant = in any case but the flag should be used if the offset returned is 3. In other related question, how do Hyperscan marks that 2 buffers should be = treated as one? I think you are missing the cross_buf_id that was introduced in V3 but was = removed due to=20 lack of usage. This variable was designed to be used in order to let the Re= gEx engine a place to save the engine state. > > > > Best, > > Ori > > >=20 > Best, > Xiang