From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 58F92A0559; Mon, 16 Mar 2020 14:54:44 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2584E2BF9; Mon, 16 Mar 2020 14:54:43 +0100 (CET) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id B641CFEB for ; Mon, 16 Mar 2020 14:54:40 +0100 (CET) IronPort-SDR: CnBh0Wo7AM2O1YG9nl30UcwjcWIVDn1tVHRwjsoZs7hupSeYbusbYK1FW87HABTV19qgdrWGAq 8Uguu8S2POTw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Mar 2020 06:54:39 -0700 IronPort-SDR: WTXnzqpBNPaOVqmUGX/gTY5Qy5r3Xjec+GW2yykndGxP25/GuPIxjXndsn6yatELawkv3LExVy drd5tbvvd41g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,560,1574150400"; d="scan'208";a="445127852" Received: from hs1.sh.intel.com (HELO hs1) ([10.67.119.44]) by fmsmga006.fm.intel.com with ESMTP; 16 Mar 2020 06:54:32 -0700 Date: Mon, 16 Mar 2020 17:10:40 -0400 From: Wang Xiang To: Ori Kam Cc: "jerinj@marvell.com" , "dev@dpdk.org" , "pbhagavatula@marvell.com" , Shahaf Shuler , "hemant.agrawal@nxp.com" , Opher Reviv , Alex Rosenbaum , "dovrat@marvell.com" , "pkapoor@marvell.com" , "nipun.gupta@nxp.com" , "bruce.richardson@intel.com" , "yang.a.hong@intel.com" , "harry.chang@intel.com" , "gu.jian1@zte.com.cn" , "shanjiangh@chinatelecom.cn" , "zhangy.yun@chinatelecom.cn" , "lixingfu@huachentel.com" , "wushuai@inspur.com" , "yuyingxia@yxlink.com" , "fanchenggang@sunyainfo.com" , "davidfgao@tencent.com" , "liuzhong1@chinaunicom.cn" , "zhaoyong11@huawei.com" , "oc@yunify.com" , "jim@netgate.com" , "hongjun.ni@intel.com" , "j.bromhead@titan-ic.com" , "deri@ntop.org" , "fc@napatech.com" , "arthur.su@lionic.com" , Thomas Monjalon Message-ID: <20200316211039.GA17099@hs1> References: <20190627155036.56940-1-jerinj@marvell.com> <1583836353-42867-1-git-send-email-orika@mellanox.com> <20200313012027.GA25215@hyperscan> <20200316012542.GB25215@hyperscan> <20200316204823.GA16186@hs1> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) Subject: Re: [dpdk-dev] [RFC v6] regexdev: introduce regexdev subsystem X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Ori, Yes, please go ahead with the patch. Thanks, Xiang On Mon, Mar 16, 2020 at 01:49:51PM +0000, Ori Kam wrote: > Hi Wang, > > PSB, if you don't have any objections and other comments, > I will start working on the class and will address all of this thread comments > in the v1 patch, > > Thanks, > Ori > > > -----Original Message----- > > From: Wang Xiang > > Sent: Monday, March 16, 2020 10:48 PM > > To: Ori Kam > > Cc: jerinj@marvell.com; dev@dpdk.org; pbhagavatula@marvell.com; Shahaf > > Shuler ; hemant.agrawal@nxp.com; Opher Reviv > > ; Alex Rosenbaum ; > > dovrat@marvell.com; pkapoor@marvell.com; nipun.gupta@nxp.com; > > bruce.richardson@intel.com; yang.a.hong@intel.com; harry.chang@intel.com; > > gu.jian1@zte.com.cn; shanjiangh@chinatelecom.cn; > > zhangy.yun@chinatelecom.cn; lixingfu@huachentel.com; wushuai@inspur.com; > > yuyingxia@yxlink.com; fanchenggang@sunyainfo.com; > > davidfgao@tencent.com; liuzhong1@chinaunicom.cn; > > zhaoyong11@huawei.com; oc@yunify.com; jim@netgate.com; > > hongjun.ni@intel.com; j.bromhead@titan-ic.com; deri@ntop.org; > > fc@napatech.com; arthur.su@lionic.com; Thomas Monjalon > > > > Subject: Re: [RFC v6] regexdev: introduce regexdev subsystem > > > > Hi Ori, > > > > On Mon, Mar 16, 2020 at 09:09:06AM +0000, Ori Kam wrote: > > > Hi Xiang, > > > > > > > -----Original Message----- > > > > From: Wang Xiang > > > > Sent: Monday, March 16, 2020 3:26 AM > > > > To: Ori Kam > > > > Cc: jerinj@marvell.com; dev@dpdk.org; pbhagavatula@marvell.com; > > Shahaf > > > > Shuler ; hemant.agrawal@nxp.com; Opher Reviv > > > > ; Alex Rosenbaum ; > > > > dovrat@marvell.com; pkapoor@marvell.com; nipun.gupta@nxp.com; > > > > bruce.richardson@intel.com; yang.a.hong@intel.com; > > harry.chang@intel.com; > > > > gu.jian1@zte.com.cn; shanjiangh@chinatelecom.cn; > > > > zhangy.yun@chinatelecom.cn; lixingfu@huachentel.com; > > wushuai@inspur.com; > > > > yuyingxia@yxlink.com; fanchenggang@sunyainfo.com; > > > > davidfgao@tencent.com; liuzhong1@chinaunicom.cn; > > > > zhaoyong11@huawei.com; oc@yunify.com; jim@netgate.com; > > > > hongjun.ni@intel.com; j.bromhead@titan-ic.com; deri@ntop.org; > > > > fc@napatech.com; arthur.su@lionic.com; Thomas Monjalon > > > > > > > > Subject: Re: [RFC v6] regexdev: introduce regexdev subsystem > > > > > > > > On Sun, Mar 15, 2020 at 10:05:53AM +0000, Ori Kam wrote: > > > > Hi Ori, > > > > > > > > > Hi Xiang, > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: Wang Xiang > > > > > > Sent: Friday, March 13, 2020 3:20 AM > > > > > > To: Ori Kam > > > > > > Cc: jerinj@marvell.com; dev@dpdk.org; pbhagavatula@marvell.com; > > > > Shahaf > > > > > > Shuler ; hemant.agrawal@nxp.com; Opher > > Reviv > > > > > > ; Alex Rosenbaum ; > > > > > > dovrat@marvell.com; pkapoor@marvell.com; nipun.gupta@nxp.com; > > > > > > bruce.richardson@intel.com; yang.a.hong@intel.com; > > > > harry.chang@intel.com; > > > > > > gu.jian1@zte.com.cn; shanjiangh@chinatelecom.cn; > > > > > > zhangy.yun@chinatelecom.cn; lixingfu@huachentel.com; > > > > wushuai@inspur.com; > > > > > > yuyingxia@yxlink.com; fanchenggang@sunyainfo.com; > > > > > > davidfgao@tencent.com; liuzhong1@chinaunicom.cn; > > > > > > zhaoyong11@huawei.com; oc@yunify.com; jim@netgate.com; > > > > > > hongjun.ni@intel.com; j.bromhead@titan-ic.com; deri@ntop.org; > > > > > > fc@napatech.com; arthur.su@lionic.com; Thomas Monjalon > > > > > > > > > > > > Subject: Re: [RFC v6] regexdev: introduce regexdev subsystem > > > > > > > > > > > > Hi Ori, > > > > > > > > > > > > Sorry for the late response as I am occupied by other works. > > > > > > Two comments below to make the definitions compatible to Hyperscan. > > > > > > > > > > > > Thanks, > > > > > > Xiang > > > > > > > > > > > > On Tue, Mar 10, 2020 at 10:32:33AM +0000, Ori Kam wrote: > > > > > > > +#define RTE_REGEX_PCRE_RULE_MATCH_ALL_F (1ULL << 13) > > > > > > > +/**< This flag marks that the results for the pattern that is being > > > > compiled > > > > > > > + * should include all possible matches. > > > > > > > + * @see struct rte_regex_dev_info::rule_flags, struct > > > > > > rte_regex_rule::rule_flags > > > > > > > + */ > > > > > > > + > > > > > > Can we change this flag to RTE_REGEX_DEV_CFG_MATCH_ALL since > > > > Hyperscan > > > > > > only supports > > > > > > match all mode and users don't have to specify this flag per rule? > > > > > > > > > > > > > > > > Sure, we can replace the RTE_REGEX_PCRE_RULE_MATCH_ALL_F with > > > > > RTE_REGEX_DEV_CFG_MATCH_ALL, and add > > > > RTE_REGEX_DEV_CAPA_SUPP_MATCH_ALL > > > > > > > > > Ack, thanks. > > > > > > > > > > > > + */ > > > > > > > +__rte_experimental > > > > > > > +int > > > > > > > +rte_regex_dev_info_get(uint8_t dev_id, struct rte_regex_dev_info > > > > > > *dev_info); > > > > > > > + > > > > > > > +/* Enumerates RegEx device configuration flags */ > > > > > > > +#define RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_F (1ULL << 0) > > > > > > > +/**< Cross buffer scan refers to the ability to be able to detect > > > > > > > + * matches that occur across buffer boundaries, where the buffers > > are > > > > > > related > > > > > > > + * to each other in some way. Enable this flag when to scan payload > > size > > > > > > > + * greater than struct rte_regex_dev_info::max_payload_size and/or > > > > > > > + * matches can present across scan buffer boundaries. > > > > > > > + * > > > > > > > + * @see struct rte_regex_dev_info::max_payload_size > > > > > > > + * @see struct rte_regex_dev_config::dev_cfg_flags, > > > > > > rte_regex_dev_configure() > > > > > > > + * @see RTE_REGEX_OPS_RSP_PMI_SOJ_F > > > > > > > + * @see RTE_REGEX_OPS_RSP_PMI_EOJ_F > > > > > > > + * @see RTE_REGEX_OPS_RSP_PMI_TOJ_F > > > > > > > + */ > > > > > > > + > > > > > > Can we add another flag > > > > > > RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_FULL_F? In this case, > > > > > > we only return full match for cross buffer scan without any partial result > > > > and > > > > > > without returning response flags such as RTE_REGEX_OPS_RSP_PMI_*. > > > > > > > > > > I think that it is good in any case to return a flag if the detection was > > based on > > > > > more than one buffer. > > > > > So I don't really see the advantage of adding such a flag. > > > > > As far as I understand in your case if the match started in previous buffer > > and > > > > ended > > > > > in the current buffer then you will return also the flag of > > > > RTE_REGEX_OPS_RSP_PMI_TOJ_F > > > > > For my general knowledge, in your system if we have the following regex: > > > > ABC > > > > > In the first buffer we have xxxA size 4 and the second buffer is BCxx > > > > > If I understand correctly for first buffer you will return no match found. > > > > > For the second buffer you will return found and end offset will be equal to > > 2 > > > > > Am I correct? > > > > > Or you are going to return end offset 6 because it started from the > > previous > > > > buffer? > > > > > > > > > Hyperscan guarantees the same matching result regardless of the data is in > > a > > > > single > > > > block or scattered to multiple blocks. So we'll return end offset 6 in this > > case > > > > without giving any flag indicating whether the match is started in previous > > > > buffer > > > > or current buffer. > > > > > > What will happen if the match was only in the second buffer? For example > > > Like before the regex is ABC but now the first buffer is xxxx and the second > > buffer > > > is ABCx will the result be end offset 3 or 7? > > > If the answer is 3 than I think the flag is important, in order to let the user > > know > > > that he should count from previous buffer. > > > If the answer is 7, since only Hyperscan works with end offset if could be > > defined > > > that when working with end offset and cross buffer scan is supported then the > > > result is always true result. > > > > > > So I think that RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_FULL_F is not > > relevant in any > > > case but the flag should be used if the offset returned is 3. > > > > > Hyperscan returns 7 in this case, so these flags aren't necessary. > > > > Hyperscan works in two modes: > > 1) return start and end offset > > 2) return end offset > > > > Since only Hyperscan supports RTE_REGEX_DEV_CFG_MATCH_ALL, we can > > define > > the result always true if match all and cross buffer scan are > > configured. Having the scan full flag will make users better aware of > > the difference from HW solutions. If you really don't want keep this flag, > > please make this definition clear to users. > > The issue with the new flag is that it should always be set, so it is redundant > if I understand correctly. I will try to make it clearer in the comment. > > > > > > > In other related question, how do Hyperscan marks that 2 buffers should be > > treated as one? > > > I think you are missing the cross_buf_id that was introduced in V3 but was > > removed due to > > > lack of usage. This variable was designed to be used in order to let the RegEx > > engine a place > > > to save the engine state. > > > > > I agree, we need to have the cross_buf_id back to support cross buffer > > scan. > > I will re-add it. > > > > > > > > > > > Best, > > > > > Ori > > > > > > > > > > > > > Best, > > > > Xiang > > > > Thanks, > > Xiang