From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9558BA056D; Tue, 3 Mar 2020 08:44:21 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EECED2C16; Tue, 3 Mar 2020 08:44:20 +0100 (CET) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2087.outbound.protection.outlook.com [40.107.22.87]) by dpdk.org (Postfix) with ESMTP id 67B7B2C02 for ; Tue, 3 Mar 2020 08:44:19 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Qx3NgW1haMCTR9gqfoi0QDv7vGyklHYIN/16d2NVtr0yLSGPi3TJsx7F4X+oz03pUHBjzH2LidHw9moQnYNulXkI1UHWVVu/H6Pfje1CkwVGkhNVd/Mm0jF7jXLTIe3VqwGfPKps42B5n9WyyAP9wbY9V2P2BJ127N8obORHKKeL6yVpxD9oIYQimTVBvEUyOXmnIoT/qAtzMIvtPqCXscjrVv4lokOTgL/INPBgPIiveSNLVc9oFYRF66FqGk4gSOCoTJTkfEpejN7kg2T7dnyhzOosQbjTspaOwEaYm2WRyDbaPe4ohHBEw6vZOTIkllDPXH1INM3vtNg4oSs90w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+OP/mzLsUHHJxDcn99eTAKK7339NN8Oe8GVGM+g0/fU=; b=WBz/OdZ6SVfjRZ/ZvmH1SIJbzmkQO22IV9tsoeoRmhYu2HPsQWvtgfEXz3QS3AuADwJmtys6h+xnbuFm+ZWJHZkzXPlVpk36pewkXZFPgRI25CtzLON11eAQA4/6JwbkRKM8HMbWXM/yKFp1zgA3whA2f6cbwuRP5lhWcn5r9Kl8SQJheK7UAusLkvf5Sk2BcJGlOOIBeeJNih4VE/FghLVFor0sEOh6I3pt4V2dtCOdvSrAHCMj8ntkwaSKFnBkd8qhiRAEByH0bbqWBurchqdSL5/BWFHMHXr5oarVHXodV58Pg0CvYJR9f2Daxuun67RyxS1X/jRbFEiOsypuqQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+OP/mzLsUHHJxDcn99eTAKK7339NN8Oe8GVGM+g0/fU=; b=rTzmTJcHJdwveMBKuzKRG1Ma4ZOd0mhRwSiLzr0Afxy2g/H8VTEz7+OUEDvwcU1H9aZ+KpHsEt0Srrxj3bVrux2fqx01Pm8cpw+KbxXYpBnNRlPcTyeCeYZDXZ+rrL28mmpIN9HEhIMuUeuLfwJm9NllOS2NLZCbDV5RevNKYrU= Received: from AM6PR05MB5176.eurprd05.prod.outlook.com (20.177.196.158) by AM6PR05MB4262.eurprd05.prod.outlook.com (52.135.164.28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2772.16; Tue, 3 Mar 2020 07:44:18 +0000 Received: from AM6PR05MB5176.eurprd05.prod.outlook.com ([fe80::1888:dbf5:a84e:c53b]) by AM6PR05MB5176.eurprd05.prod.outlook.com ([fe80::1888:dbf5:a84e:c53b%6]) with mapi id 15.20.2772.019; Tue, 3 Mar 2020 07:44:18 +0000 From: Ori Kam To: Wang Xiang CC: "jerinj@marvell.com" , "dev@dpdk.org" , "pbhagavatula@marvell.com" , Shahaf Shuler , "hemant.agrawal@nxp.com" , Opher Reviv , Alex Rosenbaum , "dovrat@marvell.com" , "pkapoor@marvell.com" , "nipun.gupta@nxp.com" , "bruce.richardson@intel.com" , "yang.a.hong@intel.com" , "harry.chang@intel.com" , "gu.jian1@zte.com.cn" , "shanjiangh@chinatelecom.cn" , "zhangy.yun@chinatelecom.cn" , "lixingfu@huachentel.com" , "wushuai@inspur.com" , "yuyingxia@yxlink.com" , "fanchenggang@sunyainfo.com" , "davidfgao@tencent.com" , "liuzhong1@chinaunicom.cn" , "zhaoyong11@huawei.com" , "oc@yunify.com" , "jim@netgate.com" , "hongjun.ni@intel.com" , "j.bromhead@titan-ic.com" , "deri@ntop.org" , "fc@napatech.com" , "arthur.su@lionic.com" , Thomas Monjalon Thread-Topic: [dpdk-dev] [RFC v5] regexdev: introduce regexdev subsystem Thread-Index: AQHV7X/S8OFPtiIdU0S3b2gAwOWtcag05scAgAGTYBA= Date: Tue, 3 Mar 2020 07:44:18 +0000 Message-ID: References: <20190627155036.56940-1-jerinj@marvell.com> <1582816115-95871-1-git-send-email-orika@mellanox.com> <20200302070500.GA31870@hyperscan> In-Reply-To: <20200302070500.GA31870@hyperscan> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=orika@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: fce2486f-1419-472f-cb24-08d7bf46ae10 x-ms-traffictypediagnostic: AM6PR05MB4262:|AM6PR05MB4262: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3044; x-forefront-prvs: 03319F6FEF x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(366004)(39860400002)(346002)(376002)(396003)(136003)(189003)(199004)(4326008)(33656002)(7416002)(86362001)(6506007)(52536014)(7696005)(478600001)(2906002)(8936002)(81156014)(81166006)(71200400001)(53546011)(8676002)(55016002)(26005)(6916009)(9686003)(5660300002)(64756008)(54906003)(76116006)(66446008)(66946007)(316002)(66476007)(66556008)(186003); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR05MB4262; H:AM6PR05MB5176.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 7X1HRm2Og1NfPGZG5hSH8jdWfL8JFYZfhTZ7hZlTZ1fBhyHBrGPQPuXErWjMP8eaC+5hkYR8XrS0r/qhzT+MH+HEzgTBGozqp/Vdl4Rxx1F48LugyoVGtVwPNmFysQUCj/c/hVGq+jh35Cf33a1rDOStS+J+TK6cUmfWgpOmlTpQVsE2chBgebdutXCPXATgG+Jc4jNqAORF7T0Ime3d/0FZwyxTQgSPbIPaLGomDbe8BrYfW1a8iJDbRQw7rmeG+MWOcS/YUiD4shy0QUszPM5lW/NckL7I+CS/329ZeIyEMc0LwYRD2CSQ1tbuMCswEufbkt2mNXfOaO1e7M36NAjwuUAHx17n/n7dZ/auWBuzyRN+ZmPSqumGYrTbiMa5wxTwRu/RXzGFv7Vlbc7ESzcgr/end4OGuYtHGdxkZOwug3IMYMyL+w12LIUwX4F+ x-ms-exchange-antispam-messagedata: fowBi9TVN219jwWGa3ntnP6Pd3sCIntsGR5oEhIzSsbZ8V7zgkCwl738Lv8yB+/viJvAjudvxtfH4xSf0bEOMFa7eHKBpWZ2fiRHvdgJiN9nkIxIlFI5K11VLJE1tmzmLxI/0fJ7KKPUvVvPzgd84g== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: fce2486f-1419-472f-cb24-08d7bf46ae10 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Mar 2020 07:44:18.4407 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: j7QMMSF82SZbwGpc2sQVIrkgOiLp/SzhPGpWcykJD2qFWad4zg1ay5RaclkpKLZ1wJWy7GCZKjh77warnntWog== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR05MB4262 Subject: Re: [dpdk-dev] [RFC v5] regexdev: introduce regexdev subsystem X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Xiang, May I ask when do you plan to add the Hyperscan code to the DPDK? > -----Original Message----- > From: dev On Behalf Of Wang Xiang > Sent: Monday, March 2, 2020 9:05 AM > To: Ori Kam > Cc: jerinj@marvell.com; dev@dpdk.org; pbhagavatula@marvell.com; Shahaf > Shuler ; hemant.agrawal@nxp.com; Opher Reviv > ; Alex Rosenbaum ; > dovrat@marvell.com; pkapoor@marvell.com; nipun.gupta@nxp.com; > bruce.richardson@intel.com; yang.a.hong@intel.com; harry.chang@intel.com; > gu.jian1@zte.com.cn; shanjiangh@chinatelecom.cn; > zhangy.yun@chinatelecom.cn; lixingfu@huachentel.com; wushuai@inspur.com; > yuyingxia@yxlink.com; fanchenggang@sunyainfo.com; > davidfgao@tencent.com; liuzhong1@chinaunicom.cn; > zhaoyong11@huawei.com; oc@yunify.com; jim@netgate.com; > hongjun.ni@intel.com; j.bromhead@titan-ic.com; deri@ntop.org; > fc@napatech.com; arthur.su@lionic.com; Thomas Monjalon > > Subject: Re: [dpdk-dev] [RFC v5] regexdev: introduce regexdev subsystem >=20 > Hi Ori, >=20 > Comments below. >=20 > Thanks, > Xiang >=20 > On Thu, Feb 27, 2020 at 03:08:35PM +0000, Ori Kam wrote: > > From: Jerin Jacob > > > > Even though there are some vendors which offer Regex HW offload, due to > > lack of standard API, It is diffcult for DPDK consumer to use them > > in a portable way. > > > > This _RFC_ attempts to standardize the RegEx/DPI offload APIs for DPDK. > > > > This RFC crafted based on SW Regex API frameworks such as libpcre and > > hyperscan and a few of the RegEx HW IPs which I am aware of. > > > > RegEx pattern matching applications: > > * Next Generation Firewalls (NGFW) > > * Deep Packet and Flow Inspection (DPI) > > * Intrusion Prevention Systems (IPS) > > * DDoS Mitigation > > * Network Monitoring > > * Data Loss Prevention (DLP) > > * Smart NICs > > * Grammar based content processing > > * URL, spam and adware filtering > > * Advanced auditing and policing of user/application security policies > > * Financial data mining - parsing of streamed financial feeds > > * Application recognition. > > * Dmemory introspection. > > * Natural Language Processing (NLP) > > * Sentiment Analysis. > > * Big data databse acceleration. > > * Computational storage. > > > > Request to review from HW and SW RegEx vendors and RegEx application > > users to have portable DPDK API for RegEx. > > > > The API schematics are based cryptodev, eventdev and ethdev existing > > device API. > > > > Signed-off-by: Jerin Jacob > > Signed-off-by: Pavan Nikhilesh > > Signed-off-by: Ori Kam > > --- > > V5: > > * Remove unused iov struct. > > V4: > > * Replace iov with mbuf. > > * Small ML comments. > > V3: > > * Change subject title. > > V2: > > * Address ML comments. > > --- > > + > > +#define RTE_REGEX_DEV_SUPP_PCRE_GREEDY_F (1ULL << 6) > > +/**< RegEx device support PCRE Greedy mode. > > + * For example if the RegEx is 'AB\d*?' then '*?' represents zero or > unlimited > > + * matches. In greedy mode the pattern 'AB12345' will be matched > completely > > + * where as the ungreedy mode 'AB' will be returned as the match. > > + * @see struct rte_regex_dev_info::regex_dev_capa > > + */ > > + >=20 > Hyperscan actually supports "match all" semantic, neither greedy nor > ungreedy, > which is different from PCRE. In the case above, AB, AB1, ..., AB12345 wi= ll all > be returned as matches. Do HW solutions support this? No our HW doesn't support this. Jerin, does Marvell HW support this? > Can we add a new flag like RTE_REGEX_DEV_SUPP_PCRE_MATCHALL_F? > Similarly, we can define a flag RTE_REGEX_PCRE_RULE_MATCHALL_F so > Hyperscan > users have to set this flag during rule compile. >=20 Sure, > > +#define RTE_REGEX_DEV_SUPP_PCRE_LOOKAROUND_ASRT_F (1ULL << 7) > > +/**< RegEx device support PCRE Lookaround assertions > > + * (Zero-width assertions). Example RegEx is '[a-z]+\d+(?=3D!{3,})' if > > + * the given pattern is 'dwad1234!' the RegEx engine doesn't report an= y > matches > > + * because the assert '(?=3D!{3,})' fails. The pattern 'dwad123!!!' wo= uld return > a > > + * successful match. > > + * @see struct rte_regex_dev_info::regex_dev_capa > > + */ > > + > > + > > +/** > > + * RegEx device information > > + */ > > +struct rte_regex_dev_info { > > + const char *driver_name; /**< RegEx driver name. */ > > + struct rte_device *dev; /**< Device information. */ > > + uint16_t max_matches; > > + /**< Maximum matches per scan supported by this device. */ > > + uint16_t max_queue_pairs; > > + /**< Maximum queue pairs supported by this device. */ > > + uint16_t max_payload_size; > > + /**< Maximum payload size for a pattern match request or scan. > > + * @see RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_F > > + */ > > + uint32_t max_rules_per_group; > > + /**< Maximum rules supported per group by this device. */ > > + uint16_t max_groups; > > + /**< Maximum groups supported by this device. */ > > + uint32_t regex_dev_capa; > > + /**< RegEx device capabilities. @see RTE_REGEX_DEV_CAPA_* */ > > + uint64_t rule_flags; > > + /**< Supported compiler rule flags. > > + * @see RTE_REGEX_PCRE_RULE_*, struct rte_regex_rule::rule_flags > > + */ > > + uint8_t max_scatter_gather; > > + /**< The max supported number of buffers that can > > + * be used in a single ops. The total size of all elements > > + * must be less then max_payload_size. > > + */ >=20 > s/then/than >=20 Will fix. > > +}; > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Retrieve the contextual information of a RegEx device. > > + * > > + * @param dev_id > > + * The identifier of the device. > > + * > > + * @param[out] dev_info > > + * A pointer to a structure of type *rte_regex_dev_info* to be fille= d with > the > > + * contextual information of the device. > > + * > > + * @return > > + * - 0: Success, driver updates the contextual information of the Re= gEx > device > > + * - <0: Error code returned by the driver info get function. > > + * > > + */ > > +__rte_experimental > > +int > > +rte_regex_dev_info_get(uint8_t dev_id, struct rte_regex_dev_info > *dev_info); > > + > > +/* Enumerates RegEx device configuration flags */ > > +#define RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_F (1ULL << 0) > > +/**< Cross buffer scan refers to the ability to be able to detect > > + * matches that occur across buffer boundaries, where the buffers are > related > > + * to each other in some way. Enable this flag when to scan payload si= ze > > + * greater struct struct rte_regex_dev_info::max_payload_size and/or > > + * matches can present across scan buffer boundaries. >=20 > s/struct/than >=20 Will fix. > > + * > > + * @see struct rte_regex_dev_info::max_payload_size > > + * @see struct rte_regex_dev_config::dev_cfg_flags, > rte_regex_dev_configure() > > + * @see RTE_REGEX_OPS_RSP_PMI_SOJ_F > > + * @see RTE_REGEX_OPS_RSP_PMI_EOJ_F > > + */ > > + > > + > > +/* Enumerates RegEx response flags. */ > > +#define RTE_REGEX_OPS_RSP_PMI_SOJ_F (1 << 0) > > +/**< Indicates that the RegEx device has encountered a partial match a= t the > > + * start of scan in the given buffer. > > + * > > + * @see RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_F > > + */ > > + > Hyperscan supports cross buffer scan and only reports true matches instea= d of > partial matches. Can we have users to config this partial match capabilit= y? >=20 Do you mean something like this: RTE_REGEX_OPS_RSP_PMI_FOJ_F /**< Indicates that the RegEx device has encountered a full match in the cu= rrent buffer. * The match was started in previous buffer. */ > > +#define RTE_REGEX_OPS_RSP_PMI_EOJ_F (1 << 1) > > +/**< Indicates that the RegEx device has encountered a partial match a= t the > > + * end of scan in the given buffer. > > + * > > + * @see RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_F > > + */ > > +