From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f48.google.com (mail-wm0-f48.google.com [74.125.82.48]) by dpdk.org (Postfix) with ESMTP id 7D9CD58D9 for ; Thu, 17 Mar 2016 16:39:28 +0100 (CET) Received: by mail-wm0-f48.google.com with SMTP id p65so31754762wmp.1 for ; Thu, 17 Mar 2016 08:39:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=2LWj7tYwZirwtHms8WYOUYrToJhFGBMUg+Wd1N8RAQY=; b=ET0qgszTrFfrh1+aFaVu1fR8w5puLg4FBSXGMCYln2/bk1GAWbWLDZ+PukqDerdniY K2UAOnuxGu2IdE/XZl1y0oSaWuVvqTUIb18tYiY2uqlLZDUbIB8/qPz1U08hMIOxH3RV lnCmUbCdksv+PD0FtrH0tgLnB6FsgCWIiqY9ztr5gulUoawAeAiRHzmHi7y7Uos/SsiF UdIkpk2nAoacM8bxA4tBxDrgkGVE8Ocvn0tIFV1iUUcg4OWej1yAF7xV1j4MSR1kIDNG nLXSKTGodF5y8eVTuPXov8BVzIGAhhHU6C61unlcxeXDAUAxk3MuIDLqC0gwhMk9lknM LryA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=2LWj7tYwZirwtHms8WYOUYrToJhFGBMUg+Wd1N8RAQY=; b=JelQBwXqS1f+6PKqIlWmirqBEjEz9J7+aEYTbbQNRYqtKjp5jIcOwvBH2e6vHDm7U7 PgNayB+pnOT6XIcrER0wIjgRjx+XJ/ozfSpynGjP4n6fPwj1NacBK7Efc/s1wDnSvdbV t9fem66jujFah2Dy4tLR8AT3sCi6imn0+noVbnFIReoo/iLoNFY0cYU8dtjdO9p4v3ns iWD/v/9j5YlH+GzKF39VsU+fzlVT0tuon7ZpR8dU9VVK8qFmbkZLvY7d5J0tuNN1fKuJ 1QOuQAgxgi3EWBpp8GpvtNmoaIiSgkP3ww2ap5kaDWV9hRMcvqxli4PKfoUY1kbauuFU YFlA== X-Gm-Message-State: AD7BkJIRDvOT8OjnfOtPPdnufryuIuyZ+FDMKLhbNu/dfV+bwtffI1AQFeSEpePMJuM0232m X-Received: by 10.194.205.103 with SMTP id lf7mr10566126wjc.147.1458229168384; Thu, 17 Mar 2016 08:39:28 -0700 (PDT) Received: from 6wind.com (guy78-3-82-239-227-177.fbx.proxad.net. [82.239.227.177]) by smtp.gmail.com with ESMTPSA id w15sm30597506wmd.10.2016.03.17.08.39.26 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 17 Mar 2016 08:39:27 -0700 (PDT) From: Adrien Mazarguil To: dev@dpdk.org Cc: Olga Shern Date: Thu, 17 Mar 2016 16:38:57 +0100 Message-Id: <1458229138-20597-5-git-send-email-adrien.mazarguil@6wind.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1458229138-20597-1-git-send-email-adrien.mazarguil@6wind.com> References: <1457015279-3089-1-git-send-email-adrien.mazarguil@6wind.com> <1458229138-20597-1-git-send-email-adrien.mazarguil@6wind.com> Subject: [dpdk-dev] [PATCH v3 4/5] mlx5: add support for HW packet padding X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Mar 2016 15:39:28 -0000 From: Olga Shern Environment variable MLX5_PMD_ENABLE_PADDING enables HW packet padding in PCI bus transactions. When packet size is cache aligned and CRC stripping is enabled, 4 fewer bytes are written to the PCI bus. Enabling padding makes such packets aligned again. In cases where PCI bandwidth is the bottleneck, padding can improve performance by 10%. This is disabled by default since this can also decrease performance for unaligned packet sizes. Signed-off-by: Olga Shern --- doc/guides/nics/mlx5.rst | 14 ++++++++++++++ doc/guides/rel_notes/release_16_04.rst | 7 +++++++ drivers/net/mlx5/Makefile | 5 +++++ drivers/net/mlx5/mlx5.c | 28 ++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5.h | 5 +++++ drivers/net/mlx5/mlx5_rxq.c | 15 +++++++++++++++ 6 files changed, 74 insertions(+) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 8b63f3f..9df30be 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -156,6 +156,20 @@ Environment variables lower performance when there is no backpressure, it is not enabled by default. +- ``MLX5_PMD_ENABLE_PADDING`` + + Enables HW packet padding in PCI bus transactions. + + When packet size is cache aligned and CRC stripping is enabled, 4 fewer + bytes are written to the PCI bus. Enabling padding makes such packets + aligned again. + + In cases where PCI bandwidth is the bottleneck, padding can improve + performance by 10%. + + This is disabled by default since this can also decrease performance for + unaligned packet sizes. + Run-time configuration ~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/guides/rel_notes/release_16_04.rst b/doc/guides/rel_notes/release_16_04.rst index a498ef7..8eb423f 100644 --- a/doc/guides/rel_notes/release_16_04.rst +++ b/doc/guides/rel_notes/release_16_04.rst @@ -144,6 +144,13 @@ This section should contain new features added in this release. Sample format: Only available with Mellanox OFED >= 3.2. +* **Added mlx5 optional packet padding by HW.** + + Added an option to make PCI bus transactions rounded to multiple of a + cache line size for better alignment. + + Only available with Mellanox OFED >= 3.2. + * **Added af_packet dynamic removal function.** Af_packet device can now be detached using API, like other PMD devices. diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile index cc6de2d..a6a3cab 100644 --- a/drivers/net/mlx5/Makefile +++ b/drivers/net/mlx5/Makefile @@ -142,6 +142,11 @@ mlx5_autoconf.h: $(RTE_SDK)/scripts/auto-config-h.sh infiniband/verbs.h \ enum IBV_EXP_CREATE_WQ_FLAG_SCATTER_FCS \ $(AUTOCONF_OUTPUT) + $Q sh -- '$<' '$@' \ + HAVE_VERBS_RX_END_PADDING \ + infiniband/verbs.h \ + enum IBV_EXP_CREATE_WQ_FLAG_RX_END_PADDING \ + $(AUTOCONF_OUTPUT) $(SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD):.c=.o): mlx5_autoconf.h diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index acfb365..94eefb9 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -68,6 +68,25 @@ #include "mlx5_defs.h" /** + * Retrieve integer value from environment variable. + * + * @param[in] name + * Environment variable name. + * + * @return + * Integer value, 0 if the variable is not set. + */ +int +mlx5_getenv_int(const char *name) +{ + const char *val = getenv(name); + + if (val == NULL) + return 0; + return atoi(val); +} + +/** * DPDK callback to close the device. * * Destroy all queues and objects, free memory. @@ -332,6 +351,9 @@ mlx5_pci_devinit(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) #ifdef HAVE_EXP_DEVICE_ATTR_VLAN_OFFLOADS IBV_EXP_DEVICE_ATTR_VLAN_OFFLOADS | #endif /* HAVE_EXP_DEVICE_ATTR_VLAN_OFFLOADS */ +#ifdef HAVE_EXP_CREATE_WQ_FLAG_RX_END_PADDING + IBV_EXP_DEVICE_ATTR_RX_PAD_END_ALIGN | +#endif /* HAVE_EXP_CREATE_WQ_FLAG_RX_END_PADDING */ 0; #endif /* HAVE_EXP_QUERY_DEVICE */ @@ -424,6 +446,12 @@ mlx5_pci_devinit(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) DEBUG("FCS stripping configuration is %ssupported", (priv->hw_fcs_strip ? "" : "not ")); +#ifdef HAVE_VERBS_RX_END_PADDING + priv->hw_padding = !!exp_device_attr.rx_pad_end_addr_align; +#endif /* HAVE_VERBS_RX_END_PADDING */ + DEBUG("hardware RX end alignment padding is %ssupported", + (priv->hw_padding ? "" : "not ")); + #else /* HAVE_EXP_QUERY_DEVICE */ priv->ind_table_max_size = RSS_INDIRECTION_TABLE_SIZE; #endif /* HAVE_EXP_QUERY_DEVICE */ diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 9690827..1904d54 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -104,6 +104,7 @@ struct priv { unsigned int hw_csum_l2tun:1; /* Same for L2 tunnels. */ unsigned int hw_vlan_strip:1; /* VLAN stripping is supported. */ unsigned int hw_fcs_strip:1; /* FCS stripping is supported. */ + unsigned int hw_padding:1; /* End alignment padding is supported. */ unsigned int vf:1; /* This is a VF device. */ unsigned int pending_alarm:1; /* An alarm is pending. */ /* RX/TX queues. */ @@ -160,6 +161,10 @@ priv_unlock(struct priv *priv) rte_spinlock_unlock(&priv->lock); } +/* mlx5.c */ + +int mlx5_getenv_int(const char *); + /* mlx5_ethdev.c */ struct priv *mlx5_get_priv(struct rte_eth_dev *dev); diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index 19a1119..c8af77f 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1282,6 +1282,21 @@ rxq_setup(struct rte_eth_dev *dev, struct rxq *rxq, uint16_t desc, tmpl.crc_present << 2); #endif /* HAVE_VERBS_FCS */ +#ifdef HAVE_VERBS_RX_END_PADDING + if (!mlx5_getenv_int("MLX5_PMD_ENABLE_PADDING")) + ; /* Nothing else to do. */ + else if (priv->hw_padding) { + INFO("%p: enabling packet padding on queue %p", + (void *)dev, (void *)rxq); + attr.wq.flags |= IBV_EXP_CREATE_WQ_FLAG_RX_END_PADDING; + attr.wq.comp_mask |= IBV_EXP_CREATE_WQ_FLAGS; + } else + WARN("%p: packet padding has been requested but is not" + " supported, make sure MLNX_OFED and firmware are" + " up to date", + (void *)dev); +#endif /* HAVE_VERBS_RX_END_PADDING */ + tmpl.wq = ibv_exp_create_wq(priv->ctx, &attr.wq); if (tmpl.wq == NULL) { ret = (errno ? errno : EINVAL); -- 2.1.4