From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-f193.google.com (mail-wr0-f193.google.com [209.85.128.193]) by dpdk.org (Postfix) with ESMTP id 45B591B0B for ; Fri, 8 Jun 2018 16:43:48 +0200 (CEST) Received: by mail-wr0-f193.google.com with SMTP id f16-v6so13676342wrm.3 for ; Fri, 08 Jun 2018 07:43:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=FgWnGrMBtPZeQD2ltBF5XdMfqyKamRwPpbKZzyMGo9Q=; b=cnd6new/ipbF85BTq/gFWH+F3D/39s4ImBisp09EysmrqjoqTxix/sTs9Fxv0n5Z19 58CxM5SzxPsjQwrvf8BbxPT0Dru4yr4HfFC+m2rB9epoCEJr6/at4eAfVsBT4r3fyaVN 9bmbr5lI+3sElhJxq+BO5YRa+oGKNXSyKGxMhHOVxqRC50wqj8CSH+ruXBEvzaw+ShW0 h74tY8mERpiKishBzQ9CkBNFAdHsgumDdvNJcYXvRZpFXKRlzmE8nNbNwdrBCm8fh2/u GQAfjZHVynFn8k2ODGtiQEor0Ntydx/nkoOMNWGEzW1aG2Haa/2VsCLYtmoTnyeWyPiW I35A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=FgWnGrMBtPZeQD2ltBF5XdMfqyKamRwPpbKZzyMGo9Q=; b=ZVtN/zmaACqgzcvjfiAZfRnMdSuShEyC8OGo3mK6HVC1ogVGip7wLVPcruHykzMDe4 5k+pDXb1GuO7j9hvluiBeg8QuSkS3OHNN+Fv8bmnEQ9sAqbNkKKYdsoHy8CWsebStAFp EgOfLLE48TXWwlpbYcN1p7NB13JIDyMfpASxepM7fTsXBx0h+jURCCN7Wyc5E3hAF7fq NopqMyku9q/vfLRUtPFuyTx9K0f3zkL2HcE508lKMDvQo/wu1Usd0+j209iwcqFEf0kP EsLGqSHjBOERY1QILEthwyBKZD5NDjNV1p3bhEf0ecq96mKuq2zXXUXmXqWeO2YCpmNs tAzw== X-Gm-Message-State: APt69E06jkLB/rGUIkmnh1psYlsMoN6qkg1sqjAviG43i86ZX9wc1MH6 lIapc0NaWbZB2FUIaKo2Kok/ X-Google-Smtp-Source: ADUXVKLXXDRvDlL8oj4C6JMis6Z4hc9t66ir6IWAB0KTz5YwZShCM6tZREBSn75P6Y9tTJifXXOTBA== X-Received: by 2002:a5d:4a4d:: with SMTP id v13-v6mr1754805wrs.91.1528469027893; Fri, 08 Jun 2018 07:43:47 -0700 (PDT) Received: from laranjeiro-vm.dev.6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id v12-v6sm1233439wmc.35.2018.06.08.07.43.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 08 Jun 2018 07:43:47 -0700 (PDT) Date: Fri, 8 Jun 2018 16:43:33 +0200 From: =?iso-8859-1?Q?N=E9lio?= Laranjeiro To: Xueming Li Cc: Shahaf Shuler , dev@dpdk.org Message-ID: <20180608144333.nqu6brgnipfbjrgt@laranjeiro-vm.dev.6wind.com> References: <20180608135751.23516-1-xuemingl@mellanox.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20180608135751.23516-1-xuemingl@mellanox.com> User-Agent: NeoMutt/20170113 (1.7.2) Subject: Re: [dpdk-dev] [RFC v1] net/mlx5: support multiple flow priorities X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jun 2018 14:43:48 -0000 Hi Xueming, On Fri, Jun 08, 2018 at 09:57:51PM +0800, Xueming Li wrote: > This patch supports multiple rte_flow priorities on NICs that support 16 > or more Verbs priorities. Control flow always consume 3 Verbs priorities > even in isolated mode. In isolated mode there are no control flows it is exactly why the isolated mode has been added. > For NICs that support 16 or more Verbs priorities, each rte_flow > priority consumes 6 Verbs priorities, higher 3 for tunnels, lower 3 for > non-tunnels. > > NICs that have 8 Verbs priorities only support 1 user prioirty: > User tunnel Verbs flows: 0 - 2 > User non-tunnel Verbs flows: 2 - 4 > control Verbs flows: 5 - 8 I am re-writting this engine and I've already submitted a v1 [1], and the v2 I am finalising solves this issue by inserting the rule from the more specific to the more generic in the same verbs priority. It still lacks in the collision handler but it follows the API restriction: "In order to remain as hardware-agnostic as possible, by default all rules are considered to have the same priority, which means that the order between overlapping rules (when a packet is matched by several filters) is undefined." With this all priorities are usable by the user. Question, is this patch for the stable branches? > Signed-off-by: Xueming Li > --- > drivers/net/mlx5/mlx5.c | 8 +++++- > drivers/net/mlx5/mlx5.h | 10 +++++-- > drivers/net/mlx5/mlx5_flow.c | 67 ++++++++++++++++++++++++++------------------ > 3 files changed, 54 insertions(+), 31 deletions(-) > > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c > index c933e274f..6897cc546 100644 > --- a/drivers/net/mlx5/mlx5.c > +++ b/drivers/net/mlx5/mlx5.c > @@ -1189,12 +1189,18 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, > /* Supported Verbs flow priority number detection. */ > if (verb_priorities == 0) > verb_priorities = mlx5_get_max_verbs_prio(eth_dev); > - if (verb_priorities < MLX5_VERBS_FLOW_PRIO_8) { > + if (verb_priorities < MLX5_VERBS_FLOW_MIN_PRIOS) { > DRV_LOG(ERR, "port %u wrong Verbs flow priorities: %u", > eth_dev->data->port_id, verb_priorities); > goto port_error; > } > priv->config.max_verbs_prio = verb_priorities; > + if (verb_priorities == MLX5_VERBS_FLOW_MIN_PRIOS) > + priv->config.ctrl_flow_prio = 1; > + else > + priv->config.ctrl_flow_prio = > + (verb_priorities - MLX5_VERBS_FLOW_CTRL_PRIOS) / > + MLX5_VERBS_FLOW_USER_PRIOS; > /* > * Once the device is added to the list of memory event > * callback, its global MR cache table cannot be expanded > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h > index 997b04a33..514501cc3 100644 > --- a/drivers/net/mlx5/mlx5.h > +++ b/drivers/net/mlx5/mlx5.h > @@ -115,6 +115,7 @@ struct mlx5_dev_config { > /* Rx queue count threshold to enable MPRQ. */ > } mprq; /* Configurations for Multi-Packet RQ. */ > unsigned int max_verbs_prio; /* Number of Verb flow priorities. */ > + unsigned int ctrl_flow_prio; /* Control flow priority. */ > unsigned int tso_max_payload_sz; /* Maximum TCP payload for TSO. */ > unsigned int ind_table_max_size; /* Maximum indirection table size. */ > int txq_inline; /* Maximum packet size for inlining. */ > @@ -131,8 +132,13 @@ enum mlx5_verbs_alloc_type { > MLX5_VERBS_ALLOC_TYPE_RX_QUEUE, > }; > > -/* 8 Verbs priorities. */ > -#define MLX5_VERBS_FLOW_PRIO_8 8 > +/* Minimal hardware Verbs priorities. */ > +#define MLX5_VERBS_FLOW_MIN_PRIOS 8 > +/* Verbs priorities used by control flow. */ > +#define MLX5_VERBS_FLOW_CTRL_PRIOS 3 > +/* Verbs priorities used by each user flow. */ > +#define MLX5_VERBS_FLOW_USER_PRIOS 6 > + > > /** > * Verbs allocator needs a context to know in the callback which kind of > diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c > index 994be05be..0e4b6f179 100644 > --- a/drivers/net/mlx5/mlx5_flow.c > +++ b/drivers/net/mlx5/mlx5_flow.c > @@ -31,9 +31,6 @@ > #include "mlx5_prm.h" > #include "mlx5_glue.h" > > -/* Flow priority for control plane flows. */ > -#define MLX5_CTRL_FLOW_PRIORITY 1 > - > /* Internet Protocol versions. */ > #define MLX5_IPV4 4 > #define MLX5_IPV6 6 > @@ -619,6 +616,8 @@ mlx5_flow_item_validate(const struct rte_flow_item *item, > /** > * Extract attribute to the parser. > * > + * @param dev > + * Pointer to Ethernet device. > * @param[in] attr > * Flow rule attributes. > * @param[out] error > @@ -628,9 +627,12 @@ mlx5_flow_item_validate(const struct rte_flow_item *item, > * 0 on success, a negative errno value otherwise and rte_errno is set. > */ > static int > -mlx5_flow_convert_attributes(const struct rte_flow_attr *attr, > +mlx5_flow_convert_attributes(struct rte_eth_dev *dev, > + const struct rte_flow_attr *attr, > struct rte_flow_error *error) > { > + struct priv *priv = dev->data->dev_private; > + > if (attr->group) { > rte_flow_error_set(error, ENOTSUP, > RTE_FLOW_ERROR_TYPE_ATTR_GROUP, > @@ -638,11 +640,11 @@ mlx5_flow_convert_attributes(const struct rte_flow_attr *attr, > "groups are not supported"); > return -rte_errno; > } > - if (attr->priority && attr->priority != MLX5_CTRL_FLOW_PRIORITY) { > + if (attr->priority > priv->config.ctrl_flow_prio) { > rte_flow_error_set(error, ENOTSUP, > RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY, > NULL, > - "priorities are not supported"); > + "priority not supported"); > return -rte_errno; > } > if (attr->egress) { > @@ -1039,23 +1041,33 @@ mlx5_flow_update_priority(struct rte_eth_dev *dev, > unsigned int i; > uint16_t priority; > > - /* 8 priorities >= 16 priorities > - * Control flow: 4-7 8-15 > - * User normal flow: 1-3 4-7 > - * User tunnel flow: 0-2 0-3 > - */ > - priority = attr->priority * MLX5_VERBS_FLOW_PRIO_8; > - if (priv->config.max_verbs_prio == MLX5_VERBS_FLOW_PRIO_8) > - priority /= 2; > - /* > - * Lower non-tunnel flow Verbs priority 1 if only support 8 Verbs > - * priorities, lower 4 otherwise. > - */ > - if (!parser->inner) { > - if (priv->config.max_verbs_prio == MLX5_VERBS_FLOW_PRIO_8) > - priority += 1; > + if (priv->config.max_verbs_prio == MLX5_VERBS_FLOW_MIN_PRIOS) { > + /* > + * Verbs priority mapping: > + * user tunnel flows: 0 - 2 > + * user non-tunnel flows: 2 - 4 > + * control flows: 5 - 8 > + */ > + if (attr->priority >= priv->config.ctrl_flow_prio) > + priority = MLX5_VERBS_FLOW_MIN_PRIOS - > + MLX5_VERBS_FLOW_CTRL_PRIOS; > + else if (!parser->inner) > + priority = (MLX5_VERBS_FLOW_MIN_PRIOS - > + MLX5_VERBS_FLOW_CTRL_PRIOS - > + MLX5_VERBS_FLOW_USER_PRIOS / 2); > else > - priority += MLX5_VERBS_FLOW_PRIO_8 / 2; > + priority = 0; > + } else { > + /* > + * Verbs priority mapping: > + * user flows * 6, control flows 3. > + * User flow mapping: > + * tunnel 3, non-tunnel 3. > + */ > + priority = attr->priority * MLX5_VERBS_FLOW_USER_PRIOS; > + if (attr->priority != priv->config.ctrl_flow_prio && > + !parser->inner) > + priority += MLX5_VERBS_FLOW_USER_PRIOS / 2; > } > if (parser->drop) { > parser->queue[HASH_RXQ_ETH].ibv_attr->priority = priority + > @@ -1285,7 +1297,7 @@ mlx5_flow_convert(struct rte_eth_dev *dev, > .layer = HASH_RXQ_ETH, > .mark_id = MLX5_FLOW_MARK_DEFAULT, > }; > - ret = mlx5_flow_convert_attributes(attr, error); > + ret = mlx5_flow_convert_attributes(dev, attr, error); > if (ret) > return ret; > ret = mlx5_flow_convert_actions(dev, actions, error, parser); > @@ -3019,7 +3031,7 @@ mlx5_ctrl_flow_vlan(struct rte_eth_dev *dev, > struct priv *priv = dev->data->dev_private; > const struct rte_flow_attr attr = { > .ingress = 1, > - .priority = MLX5_CTRL_FLOW_PRIORITY, > + .priority = priv->config.ctrl_flow_prio, > }; > struct rte_flow_item items[] = { > { > @@ -3739,7 +3751,7 @@ unsigned int > mlx5_get_max_verbs_prio(struct rte_eth_dev *dev) > { > struct priv *priv = dev->data->dev_private; > - unsigned int verb_priorities = MLX5_VERBS_FLOW_PRIO_8; > + unsigned int verb_priorities = MLX5_VERBS_FLOW_MIN_PRIOS; > struct { > struct ibv_flow_attr attr; > struct ibv_flow_spec_eth eth; > @@ -3773,8 +3785,7 @@ mlx5_get_max_verbs_prio(struct rte_eth_dev *dev) > break; > } > } while (1); > - DRV_LOG(DEBUG, "port %u Verbs flow priorities: %d," > - " user flow priorities: %d", > - dev->data->port_id, verb_priorities, MLX5_CTRL_FLOW_PRIORITY); > + DRV_LOG(DEBUG, "port %u Verbs flow priorities: %d", > + dev->data->port_id, verb_priorities); > return verb_priorities; > } > -- > 2.13.3 [1] http://dpdk.org/ml/archives/dev/2018-May/103043.html Regards, -- Nélio Laranjeiro 6WIND