From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0324DA0A02; Thu, 14 Jan 2021 06:21:16 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6B657140D25; Thu, 14 Jan 2021 06:21:16 +0100 (CET) Received: from nat-hk.nvidia.com (nat-hk.nvidia.com [203.18.50.4]) by mails.dpdk.org (Postfix) with ESMTP id 7424A140D24 for ; Thu, 14 Jan 2021 06:21:14 +0100 (CET) Received: from HKMAIL102.nvidia.com (Not Verified[10.18.92.77]) by nat-hk.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Thu, 14 Jan 2021 13:21:12 +0800 Received: from HKMAIL102.nvidia.com (10.18.16.11) by HKMAIL102.nvidia.com (10.18.16.11) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Thu, 14 Jan 2021 05:21:12 +0000 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.103) by HKMAIL102.nvidia.com (10.18.16.11) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Thu, 14 Jan 2021 05:21:12 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=g5b/w5rcPvp0xZcy1+jnChn87/rKUKClHCHBJT5m4GxlMP4LINXMEp+gDOUdS+LDBW+LeWRqZaCmnI7U82FRw9tf0KPB3gmhsMfHESCDFqMl6gxp4L2mEpOEoPiWN6XDer4fX0iT40YwKlxD6GCqnz/6rV6mcX7OINEsHHqpjusTtCHmiRfMEnuDnx4ZQoWLLHwtzvK2IiPRnXMvKk+XUV2a0hMTFNMG0KV5tfueAfuc1tTbzDElJAOA4d+GFTQqN1ZpMvNFrP5Keazj9O/6g8d2lehhfRbLyJ3qPlLtKEoLHoGy0oBvk3EzbUPu4W9Z8O5YKq1GQ5wPGHUFvD2P3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7dJgS5ZfZHuL/kuWFUUNC/4Gv0hcBwhDxI+ye0xdaqY=; b=YOJVOQcEt5UlGUZhIDkJUySzOnqUz4It4jMe0pdC5sNpdFbuLj0kbuU7qYW+UaWctHgOMCfU/YStPUjj5myt7v4pzKIFTmTze6kmK4VGStTnBKPK9lqd8sdB3w2f6PdSFAhKJe57k8Erp03/KUGdQ1injwHAlq1k+NVbjSjT6yaJVNoZUBfgk/ijmY0ETi3VwZ1P3UCw23xrqwwUpoK3iK7vBqtL1fWbyTfoM+N2GvaE3fRc/HrzKpvRRoe/WaEbay6FIXRu51QWB6NIQn76t56UNnmRPWFRxIThnLT/JNrqsoa+fVffLc3UWs0r/FDWDzyhgdRnda3hzK6ZRu5vbw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Received: from BN7PR12MB2707.namprd12.prod.outlook.com (2603:10b6:408:2f::29) by BN8PR12MB2930.namprd12.prod.outlook.com (2603:10b6:408:69::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3742.12; Thu, 14 Jan 2021 05:21:10 +0000 Received: from BN7PR12MB2707.namprd12.prod.outlook.com ([fe80::c44c:1e37:b3f4:e968]) by BN7PR12MB2707.namprd12.prod.outlook.com ([fe80::c44c:1e37:b3f4:e968%6]) with mapi id 15.20.3742.012; Thu, 14 Jan 2021 05:21:09 +0000 From: Alexander Kozyrev To: Bill Zhou , Wisam Monther , Suanming Mou , NBU-Contact-Thomas Monjalon CC: "dev@dpdk.org" Thread-Topic: [PATCH] app/flow-perf: support meter action Thread-Index: AQHW6arCxEx5rmGOfE2Dv6+zn2OnhqomiABA Date: Thu, 14 Jan 2021 05:21:09 +0000 Message-ID: References: <1610542256-285053-1-git-send-email-dongzhou@nvidia.com> In-Reply-To: <1610542256-285053-1-git-send-email-dongzhou@nvidia.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: nvidia.com; dkim=none (message not signed) header.d=none;nvidia.com; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [2607:fea8:e380:d8e0:1cb1:6d6f:5486:b0f1] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: cd9ead5b-f52e-454a-22b7-08d8b84c33d7 x-ms-traffictypediagnostic: BN8PR12MB2930: x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:107; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: F95j5YvKz0lHj8AzqZilbTXPs+jNMkF2GKkB5F3zysvzuuCKBnZEr46XuL+go/2cpr8N0nqwenFh1DkUXaGDNYXQGljip6XZvOqzwC4SqYJs8Pbs/YMpP2/30XUEzpaBObc5iwlHNpDajy7ra8mZfWsxjJJXV+mDWAr0xKVsY1F2+mvc9KaESvlQFGWpcmkwxEV3UiSr9V0LKwVdPHY0oER1gIsxDeH3APqEbq2cOyS+hS676AZid4uV5FO6rxb/MC/2fJVjPMtic0MtmwBkkQrjZz1wMCkbolTQnfWfR+1VTfa3dheR9JaRseBYNtNnSagQ6n51ZCufroAYrGB/zoC0hTMzlXFjyIApmfY4/nn2FIHlBbBgFEkqURGq88WcUuyQmFxrIEok0LBninZRvA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR12MB2707.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(136003)(366004)(346002)(39860400002)(9686003)(64756008)(66556008)(110136005)(66446008)(66476007)(30864003)(83380400001)(53546011)(52536014)(478600001)(8936002)(76116006)(4326008)(6506007)(86362001)(7696005)(2906002)(33656002)(66946007)(71200400001)(5660300002)(8676002)(55016002)(186003)(316002)(579004); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?sSvPvf0m/IAHaA0M3WO9e+iL3WZ1NlZT9X9GC+qwOJ0YriJrIXVzm8xv04pn?= =?us-ascii?Q?jdjb7UBU+IgOvI2oNMvglU9m+ExkFhF7diaWtss8T6vfKd9GMAojbNu4AWD0?= =?us-ascii?Q?XbLFbeEQqO7+bp7OowZv8K6hDuYeWugNNsxdkhvgIon0IGTfNMloX5Z/G1I3?= =?us-ascii?Q?Q8ydiFTgyB1TtyNxJxVqdmsoRqYvZzRqc4LowbxXfN107+c6xKtlKKMN6MD3?= =?us-ascii?Q?48BHOxEAT6DBQuz3SU8Z9JA3vQnUqJGcEAtbTHirZ1nWM+O6CkQnz7rVk2eW?= =?us-ascii?Q?iQN/wch84RGXYgw++SElTcHrm9dYcuOnjXMjQs0zN2cDDPjGWp+HWm05YA1o?= =?us-ascii?Q?VcjJVUvW4GSz/wIKF/YitfkAJGbizqEB3CgkjL6VpS2QL0GbyMClB1JEP97N?= =?us-ascii?Q?WkXSst+b0gv3nA0JiuVgNkumQp9HfjhdRDvW20rmc4JuaPhAX4U0HaNfJRKv?= =?us-ascii?Q?X06uhHgIqBDPFphsDb6NXHPBVLI6tWYA/ZqNM11QAQFRk97Pn/DldaEqMxL5?= =?us-ascii?Q?sBAfE/hNjSch5NslJA8VO9E14273FFre6ZBKt4rlFZqJilXaUWKhTyD8Tef+?= =?us-ascii?Q?455nARe+Er6KvU1Fjt3y5Yb2fNn6ngJoA4rPxBsKaNnTboRJQCVtcJZZirpe?= =?us-ascii?Q?HXLSmImzBh3jhbpdWyYkIHJOIANg+VnAdDHvbsZ6VzuoW5i3frCgDKhPbvVp?= =?us-ascii?Q?guZBERLqZMUmAfIELIcS/+7kmsgVqmkQgdY21aJ9nxCJQjldTQpqADGtcnzi?= =?us-ascii?Q?PEpCDfc8i8kOYC32ORlXQOnvgBwVRi+GIKo69hARMjk9evzNz7lCaoP085FU?= =?us-ascii?Q?GfBDf16AOzcSbHORSbriYu0WMdf/qOIq/7F3LW444SKZGlGqUcRhXPDfC4oI?= =?us-ascii?Q?sna8/12PNz3GVKtjrfS2FA0oVG3lgnJdBw9cG3RC+Rz0mL0J7PecOaf8KrOI?= =?us-ascii?Q?Jk9xqyo164hxBpNvH53yUyLQl7W/tX4TVoOEsJWtY42yrc9YLOLDiWtyyUWo?= =?us-ascii?Q?48oALcPMU+O/Xu/gLURY1jEjf8cbU3wQezItuRR1ut/JTfc=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN7PR12MB2707.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: cd9ead5b-f52e-454a-22b7-08d8b84c33d7 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Jan 2021 05:21:09.9147 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: g38eotJ8or0EeFgQe3paW4ZwgfnJrdymSv2Ox5C6HUZNTDgOFHiYRs1vsdVDfSs6Lyol1EH0scNJVEa1aclLXg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR12MB2930 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1610601672; bh=7dJgS5ZfZHuL/kuWFUUNC/4Gv0hcBwhDxI+ye0xdaqY=; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To: CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References: In-Reply-To:Accept-Language:Content-Language:X-MS-Has-Attach: X-MS-TNEF-Correlator:authentication-results:x-originating-ip: x-ms-publictraffictype:x-ms-office365-filtering-correlation-id: x-ms-traffictypediagnostic:x-ld-processed: x-ms-exchange-transport-forked:x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers:x-ms-exchange-senderadcheck: x-microsoft-antispam:x-microsoft-antispam-message-info: x-forefront-antispam-report:x-ms-exchange-antispam-messagedata: Content-Type:Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype: X-MS-Exchange-CrossTenant-userprincipalname: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=YF+cE5IlF+xzncAwIDySEJx5eLbS5i7m9556+vi9ClhLrRzXmP9Zu8DsvVskYBio7 DJc25/nkb4aLkDnv/3BEvCfWwE4io5ZGzlb0zpaazt8hc/B2Cqa7Vd08MCt6xdNAR5 +XNodNoDhiALs3rSRQaAjFGyAXKDoNgvL4gcXPReQSgHvRm34xpsZHWadQ1cx2tBGS v9KOeIREgVqRPwNc2UUXIcim/TFgSMOlgjtACFkOhabkhEiOVta7vHFO3LbBEZm+zc Ybwdq+2yTwS8PBMt8ljcMIR2xzS2vrDvn+0mVaBYSM+HbwQGDBqsOBEZMRXe+fS6G2 oxg4GIhRET8hg== Subject: Re: [dpdk-dev] [PATCH] app/flow-perf: support meter action X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > From: Dong Zhou > Sent: Wednesday, January 13, 2021 7:51 > To: Wisam Monther ; Suanming Mou > ; Alexander Kozyrev ; > NBU-Contact-Thomas Monjalon > Cc: dev@dpdk.org > Subject: [PATCH] app/flow-perf: support meter action >=20 > Currently, test-flow-perf app cannot generate flows with meter > action. This patch introduces new parameter "--meter" to generate > flows with meter action. >=20 > Signed-off-by: Dong Zhou > Reviewed-by: Wisam Jaddo > --- > app/test-flow-perf/actions_gen.c | 19 ++ > app/test-flow-perf/config.h | 2 + > app/test-flow-perf/main.c | 415 > +++++++++++++++++++++++++++++++-------- > doc/guides/tools/flow-perf.rst | 4 + > 4 files changed, 353 insertions(+), 87 deletions(-) >=20 > diff --git a/app/test-flow-perf/actions_gen.c b/app/test-flow- > perf/actions_gen.c > index c3545ba..1f5c64f 100644 > --- a/app/test-flow-perf/actions_gen.c > +++ b/app/test-flow-perf/actions_gen.c > @@ -891,6 +891,19 @@ struct action_rss_data { > actions[actions_counter].type =3D > RTE_FLOW_ACTION_TYPE_VXLAN_DECAP; > } >=20 > +static void > +add_meter(struct rte_flow_action *actions, > + uint8_t actions_counter, > + __rte_unused struct additional_para para) > +{ > + static struct rte_flow_action_meter > + meters[RTE_MAX_LCORE] __rte_cache_aligned; > + > + meters[para.core_idx].mtr_id =3D para.counter; > + actions[actions_counter].type =3D RTE_FLOW_ACTION_TYPE_METER; > + actions[actions_counter].conf =3D &meters[para.core_idx]; > +} > + > void > fill_actions(struct rte_flow_action *actions, uint64_t *flow_actions, > uint32_t counter, uint16_t next_table, uint16_t hairpinq, > @@ -1103,6 +1116,12 @@ struct action_rss_data { > ), > .funct =3D add_vxlan_decap, > }, > + { > + .mask =3D FLOW_ACTION_MASK( > + RTE_FLOW_ACTION_TYPE_METER > + ), > + .funct =3D add_meter, > + }, > }; >=20 > for (j =3D 0; j < MAX_ACTIONS_NUM; j++) { > diff --git a/app/test-flow-perf/config.h b/app/test-flow-perf/config.h > index 94e83c9..3d4696d 100644 > --- a/app/test-flow-perf/config.h > +++ b/app/test-flow-perf/config.h > @@ -16,6 +16,8 @@ > #define NR_RXD 256 > #define NR_TXD 256 > #define MAX_PORTS 64 > +#define METER_CIR 1250000 > +#define DEFAULT_METER_PROF_ID 100 >=20 > /* This is used for encap/decap & header modify actions. > * When it's 1: it means all actions have fixed values. > diff --git a/app/test-flow-perf/main.c b/app/test-flow-perf/main.c > index 3a0e4c1..4d881ec 100644 > --- a/app/test-flow-perf/main.c > +++ b/app/test-flow-perf/main.c > @@ -34,6 +34,7 @@ > #include > #include > #include > +#include >=20 > #include "config.h" > #include "flow_gen.h" > @@ -72,6 +73,8 @@ > #define LCORE_MODE_PKT 1 > #define LCORE_MODE_STATS 2 > #define MAX_STREAMS 64 > +#define METER_CREATE 1 > +#define METER_DELETE 2 >=20 > struct stream { > int tx_port; > @@ -93,11 +96,16 @@ struct lcore_info { >=20 > static struct lcore_info lcore_infos[RTE_MAX_LCORE]; >=20 > +struct used_cpu_time { > + double insertion[MAX_PORTS][RTE_MAX_LCORE]; > + double deletion[MAX_PORTS][RTE_MAX_LCORE]; > +}; > + > struct multi_cores_pool { > uint32_t cores_count; > uint32_t rules_count; > - double cpu_time_used_insertion[MAX_PORTS][RTE_MAX_LCORE]; > - double cpu_time_used_deletion[MAX_PORTS][RTE_MAX_LCORE]; > + struct used_cpu_time create_meter; > + struct used_cpu_time create_flow; > int64_t last_alloc[RTE_MAX_LCORE]; > int64_t current_alloc[RTE_MAX_LCORE]; > } __rte_cache_aligned; > @@ -195,6 +203,7 @@ struct multi_cores_pool { > printf(" --set-ipv6-dscp: add set ipv6 dscp action to flow actions\n" > "ipv6 dscp value to be set is random each flow\n"); > printf(" --flag: add flag action to flow actions\n"); > + printf(" --meter: add meter action to flow actions\n"); > printf(" --raw-encap=3D: add raw encap action to flow > actions\n" > "Data is the data needed to be encaped\n" > "Example: raw-encap=3Dether,ipv4,udp,vxlan\n"); > @@ -524,6 +533,14 @@ struct multi_cores_pool { > .map_idx =3D &actions_idx > }, > { > + .str =3D "meter", > + .mask =3D FLOW_ACTION_MASK( > + RTE_FLOW_ACTION_TYPE_METER > + ), > + .map =3D &flow_actions[0], > + .map_idx =3D &actions_idx > + }, > + { > .str =3D "vxlan-encap", > .mask =3D FLOW_ACTION_MASK( > RTE_FLOW_ACTION_TYPE_VXLAN_ENCAP > @@ -602,6 +619,7 @@ struct multi_cores_pool { > { "set-ipv4-dscp", 0, 0, 0 }, > { "set-ipv6-dscp", 0, 0, 0 }, > { "flag", 0, 0, 0 }, > + { "meter", 0, 0, 0 }, > { "raw-encap", 1, 0, 0 }, > { "raw-decap", 1, 0, 0 }, > { "vxlan-encap", 0, 0, 0 }, > @@ -874,6 +892,185 @@ struct multi_cores_pool { > } > } >=20 > + > +static inline int > +has_meter(void) > +{ > + int i; > + > + for (i =3D 0; i < MAX_ACTIONS_NUM; i++) { > + if (flow_actions[i] =3D=3D 0) > + break; > + if (flow_actions[i] > + & > FLOW_ACTION_MASK(RTE_FLOW_ACTION_TYPE_METER)) > + return 1; > + } > + return 0; > +} > + > +static void > +create_meter_rule(int port_id, uint32_t counter) > +{ > + int ret; > + struct rte_mtr_params params; > + uint32_t default_prof_id =3D 100; > + struct rte_mtr_error error; > + > + memset(¶ms, 0, sizeof(struct rte_mtr_params)); > + params.meter_enable =3D 1; > + params.stats_mask =3D 0xffff; > + params.use_prev_mtr_color =3D 0; > + params.dscp_table =3D NULL; > + > + /*create meter*/ > + params.meter_profile_id =3D default_prof_id; > + params.action[RTE_COLOR_GREEN] =3D > + MTR_POLICER_ACTION_COLOR_GREEN; > + params.action[RTE_COLOR_YELLOW] =3D > + MTR_POLICER_ACTION_COLOR_YELLOW; > + params.action[RTE_COLOR_RED] =3D > + MTR_POLICER_ACTION_DROP; > + > + ret =3D rte_mtr_create(port_id, counter, ¶ms, 1, &error); > + if (ret !=3D 0) { > + printf("Port %u create meter idx(%d) error(%d) message: > %s\n", > + port_id, counter, error.type, > + error.message ? error.message : "(no stated > reason)"); > + rte_exit(EXIT_FAILURE, "error in creating meter"); > + } > +} > + > +static void > +destroy_meter_rule(int port_id, uint32_t counter) > +{ > + struct rte_mtr_error error; > + > + if (rte_mtr_destroy(port_id, counter, &error)) { > + printf("Port %u destroy meter(%d) error(%d) message: > %s\n", > + port_id, counter, error.type, > + error.message ? error.message : "(no stated > reason)"); > + rte_exit(EXIT_FAILURE, "Error in deleting meter rule"); > + } > +} > + > +static void > +meters_handler(int port_id, uint8_t core_id, uint8_t ops) > +{ > + uint64_t start_batch; > + double cpu_time_used, insertion_rate; > + int rules_count_per_core, rules_batch_idx; > + uint32_t counter, start_counter =3D 0, end_counter; > + double cpu_time_per_batch[MAX_BATCHES_COUNT] =3D { 0 }; > + > + rules_count_per_core =3D rules_count / mc_pool.cores_count; > + > + if (core_id) > + start_counter =3D core_id * rules_count_per_core; > + end_counter =3D (core_id + 1) * rules_count_per_core; > + > + cpu_time_used =3D 0; > + start_batch =3D rte_rdtsc(); > + for (counter =3D start_counter; counter < end_counter; counter++) { > + if (ops =3D=3D METER_CREATE) > + create_meter_rule(port_id, counter); > + else > + destroy_meter_rule(port_id, counter); > + /* > + * Save the insertion rate for rules batch. > + * Check if the insertion reached the rules > + * patch counter, then save the insertion rate > + * for this batch. > + */ > + if (!((counter + 1) % rules_batch)) { > + rules_batch_idx =3D ((counter + 1) / rules_batch) - 1; > + cpu_time_per_batch[rules_batch_idx] =3D > + ((double)(rte_rdtsc() - start_batch)) > + / rte_get_tsc_hz(); > + cpu_time_used +=3D > cpu_time_per_batch[rules_batch_idx]; > + start_batch =3D rte_rdtsc(); > + } > + } > + > + /* Print insertion rates for all batches */ > + if (dump_iterations) > + print_rules_batches(cpu_time_per_batch); > + > + insertion_rate =3D > + ((double) (rules_count_per_core / cpu_time_used) / 1000); > + > + /* Insertion rate for all rules in one core */ > + printf(":: Port %d :: Core %d Meter %s :: start @[%d] - end @[%d]," > + " use:%.02fs, rate:%.02fk Rule/Sec\n", > + port_id, core_id, ops =3D=3D METER_CREATE ? "create" : > "delete", > + start_counter, end_counter - 1, > + cpu_time_used, insertion_rate); > + > + if (ops =3D=3D METER_CREATE) > + mc_pool.create_meter.insertion[port_id][core_id] > + =3D cpu_time_used; > + else > + mc_pool.create_meter.deletion[port_id][core_id] > + =3D cpu_time_used; > +} > + > +static void > +destroy_meter_profile(void) > +{ > + struct rte_mtr_error error; > + uint16_t nr_ports; > + int port_id; > + > + nr_ports =3D rte_eth_dev_count_avail(); > + for (port_id =3D 0; port_id < nr_ports; port_id++) { > + /* If port outside portmask */ > + if (!((ports_mask >> port_id) & 0x1)) > + continue; > + > + if (rte_mtr_meter_profile_delete > + (port_id, DEFAULT_METER_PROF_ID, &error)) { > + printf("Port %u del profile error(%d) message: %s\n", > + port_id, error.type, > + error.message ? error.message : "(no stated > reason)"); > + rte_exit(EXIT_FAILURE, "Error: Destroy meter profile > Failed!\n"); > + } > + } > +} > + > +static void > +create_meter_profile(void) > +{ > + uint16_t nr_ports; > + int ret, port_id; > + struct rte_mtr_meter_profile mp; > + struct rte_mtr_error error; > + > + /* > + *currently , only create one meter file for one port > + *1 meter profile -> N meter rules -> N rte flows > + */ > + memset(&mp, 0, sizeof(struct rte_mtr_meter_profile)); > + nr_ports =3D rte_eth_dev_count_avail(); > + for (port_id =3D 0; port_id < nr_ports; port_id++) { > + /* If port outside portmask */ > + if (!((ports_mask >> port_id) & 0x1)) > + continue; > + > + mp.alg =3D RTE_MTR_SRTCM_RFC2697; > + mp.srtcm_rfc2697.cir =3D METER_CIR; > + mp.srtcm_rfc2697.cbs =3D METER_CIR / 8; > + mp.srtcm_rfc2697.ebs =3D 0; > + > + ret =3D rte_mtr_meter_profile_add > + (port_id, DEFAULT_METER_PROF_ID, &mp, &error); > + if (ret !=3D 0) { > + printf("Port %u create Profile error(%d) message: > %s\n", > + port_id, error.type, > + error.message ? error.message : "(no stated > reason)"); > + rte_exit(EXIT_FAILURE, "Error: Creation meter profile > Failed!\n"); > + } > + } > +} > + > static inline void > destroy_flows(int port_id, uint8_t core_id, struct rte_flow **flows_list= ) > { > @@ -888,6 +1085,8 @@ struct multi_cores_pool { > int rules_count_per_core; >=20 > rules_count_per_core =3D rules_count / mc_pool.cores_count; > + if (flow_group > 0 && core_id =3D=3D 0) > + rules_count_per_core++; >=20 > start_batch =3D rte_rdtsc(); > for (i =3D 0; i < (uint32_t) rules_count_per_core; i++) { > @@ -927,7 +1126,7 @@ struct multi_cores_pool { > printf(":: Port %d :: Core %d :: The time for deleting %d rules is %f > seconds\n", > port_id, core_id, rules_count_per_core, cpu_time_used); >=20 > - mc_pool.cpu_time_used_deletion[port_id][core_id] =3D > cpu_time_used; > + mc_pool.create_flow.deletion[port_id][core_id] =3D cpu_time_used; > } >=20 > static struct rte_flow ** > @@ -1034,7 +1233,7 @@ struct multi_cores_pool { > printf(":: Port %d :: Core %d :: The time for creating %d in rules %f > seconds\n", > port_id, core_id, rules_count_per_core, cpu_time_used); >=20 > - mc_pool.cpu_time_used_insertion[port_id][core_id] =3D > cpu_time_used; > + mc_pool.create_flow.insertion[port_id][core_id] =3D cpu_time_used; > return flows_list; > } >=20 > @@ -1047,9 +1246,6 @@ struct multi_cores_pool { >=20 > nr_ports =3D rte_eth_dev_count_avail(); >=20 > - if (rules_batch > rules_count) > - rules_batch =3D rules_count; > - > printf(":: Rules Count per port: %d\n\n", rules_count); >=20 > for (port_id =3D 0; port_id < nr_ports; port_id++) { > @@ -1059,21 +1255,27 @@ struct multi_cores_pool { >=20 > /* Insertion part. */ > mc_pool.last_alloc[core_id] =3D > (int64_t)dump_socket_mem(stdout); > + if (has_meter()) > + meters_handler(port_id, core_id, METER_CREATE); > flows_list =3D insert_flows(port_id, core_id); > if (flows_list =3D=3D NULL) > rte_exit(EXIT_FAILURE, "Error: Insertion Failed!\n"); > mc_pool.current_alloc[core_id] =3D > (int64_t)dump_socket_mem(stdout); >=20 > /* Deletion part. */ > - if (delete_flag) > + if (delete_flag) { > destroy_flows(port_id, core_id, flows_list); > + if (has_meter()) > + meters_handler(port_id, core_id, > METER_DELETE); > + } > } > } >=20 > -static int > -run_rte_flow_handler_cores(void *data __rte_unused) > +static void > +dump_used_cpu_time(const char *item, > + uint16_t port, struct used_cpu_time > *used_time) > { > - uint16_t port; > + uint32_t i; > /* Latency: total count of rte rules divided > * over max time used by thread between all > * threads time. > @@ -1088,8 +1290,111 @@ struct multi_cores_pool { > double deletion_throughput_time; > double insertion_latency, insertion_throughput; > double deletion_latency, deletion_throughput; > + > + /* Save first insertion/deletion rates from first thread. > + * Start comparing with all threads, if any thread used > + * time more than current saved, replace it. > + * > + * Thus in the end we will have the max time used for > + * insertion/deletion by one thread. > + * > + * As for memory consumption, save the min of all threads > + * of last alloc, and save the max for all threads for > + * current alloc. > + */ > + > + insertion_latency_time =3D used_time->insertion[port][0]; > + deletion_latency_time =3D used_time->deletion[port][0]; > + insertion_throughput_time =3D used_time->insertion[port][0]; > + deletion_throughput_time =3D used_time->deletion[port][0]; > + > + i =3D mc_pool.cores_count; > + while (i-- > 1) { > + insertion_throughput_time +=3D used_time- > >insertion[port][i]; > + deletion_throughput_time +=3D used_time->deletion[port][i]; > + if (insertion_latency_time < used_time->insertion[port][i]) > + insertion_latency_time =3D used_time- > >insertion[port][i]; > + if (deletion_latency_time < used_time->deletion[port][i]) > + deletion_latency_time =3D used_time- > >deletion[port][i]; > + } > + > + insertion_latency =3D ((double) (mc_pool.rules_count > + / insertion_latency_time) / 1000); > + deletion_latency =3D ((double) (mc_pool.rules_count > + / deletion_latency_time) / 1000); > + > + insertion_throughput_time /=3D mc_pool.cores_count; > + deletion_throughput_time /=3D mc_pool.cores_count; > + insertion_throughput =3D ((double) (mc_pool.rules_count > + / insertion_throughput_time) / 1000); > + deletion_throughput =3D ((double) (mc_pool.rules_count > + / deletion_throughput_time) / 1000); > + > + /* Latency stats */ > + printf("\n%s\n:: [Latency | Insertion] All Cores :: Port %d :: ", > + item, port); > + printf("Total flows insertion rate -> %f K Rules/Sec\n", > + insertion_latency); > + printf(":: [Latency | Insertion] All Cores :: Port %d :: ", port); > + printf("The time for creating %d rules is %f seconds\n", > + mc_pool.rules_count, insertion_latency_time); > + > + /* Throughput stats */ > + printf(":: [Throughput | Insertion] All Cores :: Port %d :: ", port); > + printf("Total flows insertion rate -> %f K Rules/Sec\n", > + insertion_throughput); > + printf(":: [Throughput | Insertion] All Cores :: Port %d :: ", port); > + printf("The average time for creating %d rules is %f seconds\n", > + mc_pool.rules_count, insertion_throughput_time); > + > + if (delete_flag) { > + /* Latency stats */ > + printf(":: [Latency | Deletion] All Cores :: Port %d :: Total " > + "deletion rate -> %f K Rules/Sec\n", > + port, deletion_latency); > + printf(":: [Latency | Deletion] All Cores :: Port %d :: ", > + port); > + printf("The time for deleting %d rules is %f seconds\n", > + mc_pool.rules_count, deletion_latency_time); > + > + /* Throughput stats */ > + printf(":: [Throughput | Deletion] All Cores :: Port %d :: Total " > + "deletion rate -> %f K Rules/Sec\n", > + port, deletion_throughput); > + printf(":: [Throughput | Deletion] All Cores :: Port %d :: ", > + port); > + printf("The average time for deleting %d rules is %f > seconds\n", > + mc_pool.rules_count, deletion_throughput_time); > + } > +} > + > +static void > +dump_used_mem(uint16_t port) > +{ > + uint32_t i; > int64_t last_alloc, current_alloc; > int flow_size_in_bytes; > + > + last_alloc =3D mc_pool.last_alloc[0]; > + current_alloc =3D mc_pool.current_alloc[0]; > + > + i =3D mc_pool.cores_count; > + while (i-- > 1) { > + if (last_alloc > mc_pool.last_alloc[i]) > + last_alloc =3D mc_pool.last_alloc[i]; > + if (current_alloc < mc_pool.current_alloc[i]) > + current_alloc =3D mc_pool.current_alloc[i]; > + } > + > + flow_size_in_bytes =3D (current_alloc - last_alloc) / > mc_pool.rules_count; > + printf("\n:: Port %d :: rte_flow size in DPDK layer: %d Bytes\n", > + port, flow_size_in_bytes); > +} > + > +static int > +run_rte_flow_handler_cores(void *data __rte_unused) > +{ > + uint16_t port; I don't see how this part of the patch is related to the meter action. Probably this change deserves a separate commit with a proper message. > int lcore_counter =3D 0; > int lcore_id =3D rte_lcore_id(); > int i; > @@ -1120,83 +1425,12 @@ struct multi_cores_pool { > /* Make sure all cores finished insertion/deletion process. */ > rte_eal_mp_wait_lcore(); >=20 > - /* Save first insertion/deletion rates from first thread. > - * Start comparing with all threads, if any thread used > - * time more than current saved, replace it. > - * > - * Thus in the end we will have the max time used for > - * insertion/deletion by one thread. > - * > - * As for memory consumption, save the min of all threads > - * of last alloc, and save the max for all threads for > - * current alloc. > - */ > RTE_ETH_FOREACH_DEV(port) { > - last_alloc =3D mc_pool.last_alloc[0]; > - current_alloc =3D mc_pool.current_alloc[0]; > - > - insertion_latency_time =3D > mc_pool.cpu_time_used_insertion[port][0]; > - deletion_latency_time =3D > mc_pool.cpu_time_used_deletion[port][0]; > - insertion_throughput_time =3D > mc_pool.cpu_time_used_insertion[port][0]; > - deletion_throughput_time =3D > mc_pool.cpu_time_used_deletion[port][0]; > - i =3D mc_pool.cores_count; > - while (i-- > 1) { > - insertion_throughput_time +=3D > mc_pool.cpu_time_used_insertion[port][i]; > - deletion_throughput_time +=3D > mc_pool.cpu_time_used_deletion[port][i]; > - if (insertion_latency_time < > mc_pool.cpu_time_used_insertion[port][i]) > - insertion_latency_time =3D > mc_pool.cpu_time_used_insertion[port][i]; > - if (deletion_latency_time < > mc_pool.cpu_time_used_deletion[port][i]) > - deletion_latency_time =3D > mc_pool.cpu_time_used_deletion[port][i]; > - if (last_alloc > mc_pool.last_alloc[i]) > - last_alloc =3D mc_pool.last_alloc[i]; > - if (current_alloc < mc_pool.current_alloc[i]) > - current_alloc =3D mc_pool.current_alloc[i]; > - } > - > - flow_size_in_bytes =3D (current_alloc - last_alloc) / > mc_pool.rules_count; > - > - insertion_latency =3D ((double) (mc_pool.rules_count / > insertion_latency_time) / 1000); > - deletion_latency =3D ((double) (mc_pool.rules_count / > deletion_latency_time) / 1000); > - > - insertion_throughput_time /=3D mc_pool.cores_count; > - deletion_throughput_time /=3D mc_pool.cores_count; > - insertion_throughput =3D ((double) (mc_pool.rules_count / > insertion_throughput_time) / 1000); > - deletion_throughput =3D ((double) (mc_pool.rules_count / > deletion_throughput_time) / 1000); > - > - /* Latency stats */ > - printf("\n:: [Latency | Insertion] All Cores :: Port %d :: ", > port); > - printf("Total flows insertion rate -> %f K Rules/Sec\n", > - insertion_latency); > - printf(":: [Latency | Insertion] All Cores :: Port %d :: ", port); > - printf("The time for creating %d rules is %f seconds\n", > - mc_pool.rules_count, insertion_latency_time); > - > - /* Throughput stats */ > - printf(":: [Throughput | Insertion] All Cores :: Port %d :: ", > port); > - printf("Total flows insertion rate -> %f K Rules/Sec\n", > - insertion_throughput); > - printf(":: [Throughput | Insertion] All Cores :: Port %d :: ", > port); > - printf("The average time for creating %d rules is %f > seconds\n", > - mc_pool.rules_count, insertion_throughput_time); > - > - if (delete_flag) { > - /* Latency stats */ > - printf(":: [Latency | Deletion] All Cores :: Port %d :: > Total flows " > - "deletion rate -> %f K Rules/Sec\n", > - port, deletion_latency); > - printf(":: [Latency | Deletion] All Cores :: Port %d :: ", > port); > - printf("The time for deleting %d rules is %f > seconds\n", > - mc_pool.rules_count, deletion_latency_time); > - > - /* Throughput stats */ > - printf(":: [Throughput | Deletion] All Cores :: Port %d > :: Total flows " > - "deletion rate -> %f K Rules/Sec\n", port, > deletion_throughput); > - printf(":: [Throughput | Deletion] All Cores :: Port %d > :: ", port); > - printf("The average time for deleting %d rules is %f > seconds\n", > - mc_pool.rules_count, deletion_throughput_time); > - } > - printf("\n:: Port %d :: rte_flow size in DPDK layer: %d > Bytes\n", > - port, flow_size_in_bytes); > + dump_used_cpu_time("Meters:", > + port, &mc_pool.create_meter); > + dump_used_cpu_time("Flows:", > + port, &mc_pool.create_flow); > + dump_used_mem(port); > } >=20 > return 0; > @@ -1633,6 +1867,9 @@ struct multi_cores_pool { > if (argc > 1) > args_parse(argc, argv); >=20 > + if (rules_batch > rules_count) > + rules_batch =3D rules_count; > + > init_port(); >=20 > nb_lcores =3D rte_lcore_count(); > @@ -1642,12 +1879,16 @@ struct multi_cores_pool { >=20 > printf(":: Flows Count per port: %d\n\n", rules_count); >=20 > + if (has_meter()) > + create_meter_profile(); > rte_eal_mp_remote_launch(run_rte_flow_handler_cores, NULL, > CALL_MAIN); >=20 > if (enable_fwd) { > init_lcore_info(); > rte_eal_mp_remote_launch(start_forwarding, NULL, > CALL_MAIN); > } > + if (has_meter() && delete_flag) > + destroy_meter_profile(); >=20 > RTE_ETH_FOREACH_DEV(port) { > rte_flow_flush(port, &error); > diff --git a/doc/guides/tools/flow-perf.rst b/doc/guides/tools/flow-perf.= rst > index 40d157e..017e200 100644 > --- a/doc/guides/tools/flow-perf.rst > +++ b/doc/guides/tools/flow-perf.rst > @@ -345,3 +345,7 @@ Actions: >=20 > * ``--vxlan-decap`` > Add vxlan decap action to all flows actions. > + > +* ``--meter`` > + Add meter action to all flows actions. > + Currently, 1 meter profile -> N meter rules -> N rte flows. > -- > 1.8.3.1