From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9DE14A0521; Tue, 28 Jul 2020 06:29:55 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 41D081BFF4; Tue, 28 Jul 2020 06:29:54 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2065.outbound.protection.outlook.com [40.107.21.65]) by dpdk.org (Postfix) with ESMTP id 79043F04 for ; Tue, 28 Jul 2020 06:29:52 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=V/b8Rz5kwSPsilatZMKY+x5nHsePWvRHq57qqRkWgfv0gT5qFDIozTer0+5hS3nkeO7QQbK7V2vsPzEj1HZKi8zkXOMEiFGoXWMIw9dcrJRdFgMptQc9U4q3ltanuGIKY3bG75nIVhZ6xSiMmht32GGciG3greLd+uS1Qw5Fv4kW9lesbrWzlSXqlaNuNK4gRu3a4q+nLFpfD4IhNDwUohbH81jYnrRlrNTHaFcAeGCiLN7wCLPq5oV8qBRPGc/Sm7+WjKgeqQFb3jVbmXyzlcGVBAd7szJSr3GlELyoBTPXhnZv5xSfDhrvET7dYYiDNaC44AmZbSdCAwObq7zzKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/9gxbmrpM0Io+xob6p31dJz6Vp5pEooMQ7KGisZekSg=; b=PFOmQXSE7BKacfHBReW2zFMAYOfkQfPWtkP1obk6ToHPTk6Y1wsNYUwnkzwIi7MgBfSSCdG1DecEsVg3ph7MRtk5ukYyeEFI7NPQISsMbavbL/NL3/4jP0EOo9AOx2EUHz689wHtsRC50vAJzJOtpZYNvwjmgZ/4yAcuE9kLxroxcuYSZsNcd4DU+hsUt4RF35iLNUYzOjT/97rvz0J/OBl2Qlcnuxh2rLLvinEI5+SpT39EgcD/PMWez2PcCF2VrADaZVo8PzLFPHfKGufkE0BgpxFpYXHpmCSWqWZemKE0h/BvBJ07ysi4M5dyrAm7dgImy6ejxwGoNvbMN/T87g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/9gxbmrpM0Io+xob6p31dJz6Vp5pEooMQ7KGisZekSg=; b=Lk5BRvAxV3QOSoCW3XwRqnE1DOIN5+bQ5JHqpe+Za7lNqn0zxTMJNpUraMXbRYrhVFMqfiOXcTwv+vVOi+zstyFAsRE3N2cD75IWyfhH11q4j3G07vyDLjPcnN5/K8ulH2QJtK74pfgXfVMQ+96pIFkhkGNW2OjU4fOIY6tq0Y8= Received: from AM6PR05MB5176.eurprd05.prod.outlook.com (2603:10a6:20b:63::30) by AM7PR05MB6898.eurprd05.prod.outlook.com (2603:10a6:20b:1a7::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3216.21; Tue, 28 Jul 2020 04:29:50 +0000 Received: from AM6PR05MB5176.eurprd05.prod.outlook.com ([fe80::9024:accb:4d6b:805f]) by AM6PR05MB5176.eurprd05.prod.outlook.com ([fe80::9024:accb:4d6b:805f%3]) with mapi id 15.20.3216.034; Tue, 28 Jul 2020 04:29:50 +0000 From: Ori Kam To: Thomas Monjalon , "jerinj@marvell.com" , "xiang.w.wang@intel.com" , Matan Azrad , Slava Ovsiienko , John McNamara , Marko Kovacevic , "dev@dpdk.org" CC: "guyk@marvell.com" , "dev@dpdk.org" , "pbhagavatula@marvell.com" , Shahaf Shuler , "hemant.agrawal@nxp.com" , Opher Reviv , Alex Rosenbaum , "dovrat@marvell.com" , "pkapoor@marvell.com" , "nipun.gupta@nxp.com" , "bruce.richardson@intel.com" , "yang.a.hong@intel.com" , "harry.chang@intel.com" , "gu.jian1@zte.com.cn" , "shanjiangh@chinatelecom.cn" , "zhangy.yun@chinatelecom.cn" , "lixingfu@huachentel.com" , "wushuai@inspur.com" , "yuyingxia@yxlink.com" , "fanchenggang@sunyainfo.com" , "davidfgao@tencent.com" , "liuzhong1@chinaunicom.cn" , "zhaoyong11@huawei.com" , "oc@yunify.com" , "jim@netgate.com" , "hongjun.ni@intel.com" , "deri@ntop.org" , "fc@napatech.com" , "arthur.su@lionic.com" , Raslan Darawsheh , Yuval Avnery Thread-Topic: [dpdk-dev] [PATCH v1] app/test-regex: add RegEx test application Thread-Index: AQHWY4cxJ83xG7rs1kijuApPzY9nYqkbqlcAgAC4j2A= Date: Tue, 28 Jul 2020 04:29:50 +0000 Message-ID: References: <1595793496-73205-1-git-send-email-orika@mellanox.com> <3437146.xsi8y2SdMt@thomas> In-Reply-To: <3437146.xsi8y2SdMt@thomas> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=none action=none header.from=mellanox.com; x-originating-ip: [147.236.152.129] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: a388782e-e784-44b6-bdd7-08d832aede30 x-ms-traffictypediagnostic: AM7PR05MB6898: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtFwd,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: tJgPoA0BChS2cz0TDl8rxZveprDx03PdSHUl2hHqpHVH2vZ5Xyl/crXZk8IE4xC8sKa0RBymaEeV0XSmDNbRQFa1JMJUGaqsCREgKdBM3sCXl0M6KawAmfMsLO41Us4g3HZMzPnyPGo3am9yT2ozaLlMd3mZAY8QSERqa5wEUwnSAt/d5r2QSb6COWtZx0vSwweCpvWsT6yKoExYC9kWHBkOQsLHNgiY6xbqiTKM1mG2LgVKD2vDvExA1xhGQChAudmqrLTwXf6vzuXjGcMguifPrCTr+v806yPuQVzw7qhnMhswvUfKBW4jZlbeQhUJzaTXx6HILpzwEK/xJgz5bw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR05MB5176.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(346002)(396003)(39860400002)(136003)(376002)(4326008)(66446008)(64756008)(186003)(66476007)(8676002)(26005)(83380400001)(478600001)(71200400001)(316002)(55016002)(66556008)(7406005)(66946007)(7416002)(9686003)(76116006)(8936002)(110136005)(107886003)(86362001)(54906003)(2906002)(33656002)(6506007)(5660300002)(7696005)(52536014); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: hH0s080+wWW99ddFpcNmoDEyIMjJ7ZX3qz4uCD0YJ6pgw2Jw4t5xddZTt1LSVxCIJoDRbb23yfkS67Nmn9r0XAy8nYameBA5wHZ5/0LgB+umOQZ4HOUoP7tu85u+98S2aGkLBbMQhL3orjLF9kzOGc2u2/sP8H3Fzd9kCKWDLvgxqfHAQkGjFzM3SNQlEGSAKuD+Uz48oWe9wZ2R/7vTzeGD4aC+bqURUJv88KKMN5/PU6M50rC7uEOT+rJSjW/q6lm4/eB2dTua48IUATNtZeUAwxN/Cjya0Sne7qL6wtWG78cjxAlIZY10c/X3nGjs1r5TcVlkj4bTQ2lc58sNK8PSXhAgwa3Z00meh+uhq8I98eHrCdEpFL55RuPIh3PnuVFEslKscAvuf9cq5I4rf2RrF5+EnvdEqcM3YDawfOGpDPZxgXLgOmrmEX7hfkbJpK/Gm+8gBKHE4GdWqJr9kTbQBUlGAcBx9fMvul7ZoJ01GIKCkdZF9jNaaDulrrQn Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM6PR05MB5176.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: a388782e-e784-44b6-bdd7-08d832aede30 X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Jul 2020 04:29:50.4161 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: CMFBrvC/Lt/nflnHO1agoSFTZFxRB2ZjDh3sAEc8egYq3JKpuhrCXpoXnNohrq6Dc8IRSuciboztrxs+6rq19w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR05MB6898 Subject: Re: [dpdk-dev] [PATCH v1] app/test-regex: add RegEx test application X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Thomas, > -----Original Message----- > From: Thomas Monjalon >=20 > 26/07/2020 21:58, Ori Kam: > > --- a/app/meson.build > > +++ b/app/meson.build > > @@ -12,6 +12,7 @@ apps =3D [ > > 'test-bbdev', > > 'test-cmdline', > > 'test-compress-perf', > > + 'test-regex', > > 'test-crypto-perf', > > 'test-eventdev', > > 'test-fib', >=20 > In this list, I think the alphabetical order was chosen. >=20 Will change, I did that since in all other cases the regex is after the com= press. > > diff --git a/app/test-regex/Makefile b/app/test-regex/Makefile > > new file mode 100644 > > index 0000000..d73e776 > > --- /dev/null > > +++ b/app/test-regex/Makefile > > @@ -0,0 +1,24 @@ > > +# SPDX-License-Identifier: BSD-3-Clause > > +# Copyright(c) 2020 Mellanox Corporation >=20 > It does not comply with Mellanox copyright syntax. > Note: I already did this comment in recent past. >=20 Will fix. > > + > > +include $(RTE_SDK)/mk/rte.vars.mk > > + > > +ifeq ($(CONFIG_RTE_LIBRTE_REGEXDEV),y) > > + > > +# > > +# library name > > +# > > +APP =3D testregex > > + > > +CFLAGS +=3D -O3 > > +CFLAGS +=3D $(WERROR_FLAGS) > > +CFLAGS +=3D -Wno-deprecated-declarations >=20 > This flag is not acceptable. > Will remove. =20 > > + > > +# > > +# all source are stored in SRCS-y > > +# > > +SRCS-y :=3D main.c > > + > > +include $(RTE_SDK)/mk/rte.app.mk > > + > > +endif > [...] > > --- /dev/null > > +++ b/app/test-regex/generate_data_file.py > > @@ -0,0 +1,29 @@ > > +#!/usr/bin/env python > > +# SPDX-License-Identifier: BSD-3-Clause > > +# Copyright 2020 Mellanox Technologies, Ltd > > + > > +import random > > + > > +KEYWORD =3D 'hello world' > > +MAX_COUNT =3D 10 > > +MIN_COUNT =3D 5 > > +MAX_LEN =3D 1024 > > +REPEAT_COUNT =3D random.randrange(MIN_COUNT, MAX_COUNT) > > + > > +current_pos =3D 0; > > +match_pos =3D [] > > + > > +fd_input =3D open('input.txt','w') > > +fd_res =3D open('res.txt','w') >=20 > space missing >=20 Will remove this file. > > + > > +for i in range(REPEAT_COUNT): > > + rand =3D random.randrange(MAX_LEN) > > + fd_input.write(' ' * rand) > > + current_pos +=3D rand > > + fd_input.write(KEYWORD) > > + match_pos.append(current_pos) > > + fd_res.write('{}\n'.format(str(current_pos))) > > + current_pos +=3D len(KEYWORD) > > + > > +fd_input.close() > > +fd_res.close() >=20 > I think there is a more pythonic way of writing in a file. > At least, you can use "with". > Will remove this file. Since this file use used to generate the data for the given rule file and the rule file will be removed I will also remove this file.=20 > > --- /dev/null > > +++ b/app/test-regex/hello_world.rof2 >=20 > Already discussed in a separate thread. > This file should be generated. >=20 > > --- /dev/null > > +++ b/app/test-regex/main.c > > @@ -0,0 +1,429 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright 2020 Mellanox Technologies, Ltd > > + * > > + * This file contain the application main file > > + * This application provides a way to test the RegEx class. >=20 > In general I like comments explaining what is a file for. > But here it looks useless. >=20 Will remove. > > + */ > > + > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > + > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > + > > +#define HELP_VAL 0 > > +#define RULES_VAL 1 > > +#define DATA_VAL 2 > > +#define JOB_VAL 3 > > +#define PERF_VAL 4 > > +#define ITER_VAL 5 >=20 > Please add comments to explain what are these constants for. > I think an enum, and a common prefix would be better. > Will fix. =20 > > +#define MAX_FILE_NAME 255 > > + > > +static char rules_file[MAX_FILE_NAME]; > > +static char data_file[MAX_FILE_NAME]; > > +static uint32_t jobs; > > +static struct rte_mempool *mbuf_mp; > > +static uint8_t nb_max_matches; > > +static uint16_t nb_max_payload; > > +static int perf_test; > > +static uint32_t iter; >=20 > Please avoid global variables. >=20 Will change. > > + > > +static void > > +usage(const char *prog_name) > > +{ > > + printf("%s [EAL options] --\n" > > + " --rules NAME: precompiled rules file\n" > > + " --data NAME: data file to use\n" > > + " --nb_jobs: number of jobs to use\n" > > + " --perf N: only outputs the performance data\n" > > + " --nb_iter N: number of iteration to run\n", > > + prog_name); > > +} > > + > > +static void > > +args_parse(int argc, char **argv) > > +{ > > + char **argvopt; > > + int opt; > > + int opt_idx; > > + size_t len; > > + static struct option lgopts[] =3D { > > + { "help", 0, 0, HELP_VAL }, > > + { "rules", 1, 0, RULES_VAL }, > > + /* Rules database file to load. */ > > + { "data", 1, 0, DATA_VAL }, > > + /* Data file to load. */ > > + { "nb_jobs", 1, 0, JOB_VAL }, > > + /* Number of jobs to create. */ > > + { "perf", 0, 0, PERF_VAL}, > > + /* Perf test only */ > > + { "nb_iter", 1, 0, ITER_VAL} > > + /* Number of iterations to run with perf test */ > > + }; > > + > > + argvopt =3D argv; > > + >=20 > Useless newline. >=20 Will remove. > > + while ((opt =3D getopt_long(argc, argvopt, "", > > + lgopts, &opt_idx)) !=3D EOF) { > > + switch (opt) { >=20 > [...] > > +#define MBUF_SIZE (1 << 14) >=20 > Why this size? > Add a comment? > No reason will change it to smaller size in any case. =20 > > +static void > > +extbuf_free_cb(void *addr __rte_unused, void *fcb_opaque __rte_unused) > > +{ > > + > > +} >=20 > Empty function can be dropped. >=20 It must exist since I need to pass it to function call. > [...] > > +It is based on precomplied rule file, and an input file, both of them = can >=20 > precompiled >=20 > > +be selected using command-line options. > > + > > +In general case, each PMD has it's own rule file. >=20 > its >=20 Will fix. > > + > > +The test outputs the performance, the results matching (rule id, posit= ion, > len) >=20 > length >=20 > A list will look better. >=20 Will change. > > +for each job and also a list of matches (rule id, position , len) in a= bsulote >=20 > absolute >=20 Will fix. > > +position. > > + > > + > > +Limitations > > +~~~~~~~~~~~ > > + > > +* Only one queue is supported. > > + > > +* Supports only precompiled rules. > > + > > +EAL Options > > +~~~~~~~~~~~ > > + > > +The following are the EAL command-line options that can be used in > conjunction > > +with the ``dpdk-test-regex`` application. > > +See the DPDK Getting Started Guides for more information on these opti= ons. > > + > > + > > +* ``-w `` > > + > > + Add a PCI device in white list. >=20 > Please drop "EAL options" chapter. > It is not specific to the app. >=20 Sure. > > +Application Options > > +~~~~~~~~~~~~~~~~~~~ > > + > > + ``--rules NAME``: precompiled rule file > > + > > + ``--data NAME``: data file to use > > + > > + ``--nb_jobs N``: number of jobs to use > > + > > + ``--perf N``: only outputs the performance data > > + > > + ``--nb_iter N``: number of iteration to run > > + > > + ``--help``: prints this help >=20 > Please use definition list. > Will change. =20 > > +Compiling the Tool > > +------------------ > > + > > +The ``dpdk-test-regex`` application depends on RegEx lib ``rte_regexde= v``. >=20 > Useless >=20 Will remove, > > + > > + > > +Generating the data > > +------------------- > > + > > +In the current version, the compiled rule file is loaded with a rule t= hat > > +matches 'hello world'. To create the data file, > > +it is possible to use the included python script ``generate_data_file.= py`` > > + which generates two files, > > +``input.txt`` which holds the input buffer. An input buffer is a rando= m > number > > +of spaces chars followed by the phrase 'hello world'. > > +This sequence is repeated a random number of times. > > +The second file is ``res.txt`` which holds the position of each > > +of the 'hello world' in the input file. >=20 > A script is missing to generate a default set of input data. >=20 Why? It has the python script that generate this input. In any case I'm going to remove it. The generation of data will be done out= side DPDK just like the rule compilation. =20 >=20 > > +Running the Tool > > +---------------- > > + > > +The tool has a number of command line options. Here is the sample > command line: > > + > > +.. code-block:: console > > + > > + ./build/app/testregex -w 83:00.0 -- --rules app/test-regex/hello_wo= rld.rof2 > --data app/test-regex/input.txt --job 100 >=20 >=20 > Bottom line, I think this application is not ready for DPDK 20.08. > It's good to have it available as a patch for first users who > want to play with the new regex library. > However, I propose waiting 20.11 to integrate a better version of it. >=20 I think the change are no major remarks that can't be fixed in a day. can it target rc4?