From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <adrien.mazarguil@6wind.com>
Received: from mail-wm0-f48.google.com (mail-wm0-f48.google.com [74.125.82.48])
 by dpdk.org (Postfix) with ESMTP id 7D9CD58D9
 for <dev@dpdk.org>; Thu, 17 Mar 2016 16:39:28 +0100 (CET)
Received: by mail-wm0-f48.google.com with SMTP id p65so31754762wmp.1
 for <dev@dpdk.org>; Thu, 17 Mar 2016 08:39:28 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=6wind-com.20150623.gappssmtp.com; s=20150623;
 h=from:to:cc:subject:date:message-id:in-reply-to:references;
 bh=2LWj7tYwZirwtHms8WYOUYrToJhFGBMUg+Wd1N8RAQY=;
 b=ET0qgszTrFfrh1+aFaVu1fR8w5puLg4FBSXGMCYln2/bk1GAWbWLDZ+PukqDerdniY
 K2UAOnuxGu2IdE/XZl1y0oSaWuVvqTUIb18tYiY2uqlLZDUbIB8/qPz1U08hMIOxH3RV
 lnCmUbCdksv+PD0FtrH0tgLnB6FsgCWIiqY9ztr5gulUoawAeAiRHzmHi7y7Uos/SsiF
 UdIkpk2nAoacM8bxA4tBxDrgkGVE8Ocvn0tIFV1iUUcg4OWej1yAF7xV1j4MSR1kIDNG
 nLXSKTGodF5y8eVTuPXov8BVzIGAhhHU6C61unlcxeXDAUAxk3MuIDLqC0gwhMk9lknM
 LryA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20130820;
 h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
 :references;
 bh=2LWj7tYwZirwtHms8WYOUYrToJhFGBMUg+Wd1N8RAQY=;
 b=JelQBwXqS1f+6PKqIlWmirqBEjEz9J7+aEYTbbQNRYqtKjp5jIcOwvBH2e6vHDm7U7
 PgNayB+pnOT6XIcrER0wIjgRjx+XJ/ozfSpynGjP4n6fPwj1NacBK7Efc/s1wDnSvdbV
 t9fem66jujFah2Dy4tLR8AT3sCi6imn0+noVbnFIReoo/iLoNFY0cYU8dtjdO9p4v3ns
 iWD/v/9j5YlH+GzKF39VsU+fzlVT0tuon7ZpR8dU9VVK8qFmbkZLvY7d5J0tuNN1fKuJ
 1QOuQAgxgi3EWBpp8GpvtNmoaIiSgkP3ww2ap5kaDWV9hRMcvqxli4PKfoUY1kbauuFU
 YFlA==
X-Gm-Message-State: AD7BkJIRDvOT8OjnfOtPPdnufryuIuyZ+FDMKLhbNu/dfV+bwtffI1AQFeSEpePMJuM0232m
X-Received: by 10.194.205.103 with SMTP id lf7mr10566126wjc.147.1458229168384; 
 Thu, 17 Mar 2016 08:39:28 -0700 (PDT)
Received: from 6wind.com (guy78-3-82-239-227-177.fbx.proxad.net.
 [82.239.227.177])
 by smtp.gmail.com with ESMTPSA id w15sm30597506wmd.10.2016.03.17.08.39.26
 (version=TLSv1/SSLv3 cipher=OTHER);
 Thu, 17 Mar 2016 08:39:27 -0700 (PDT)
From: Adrien Mazarguil <adrien.mazarguil@6wind.com>
To: dev@dpdk.org
Cc: Olga Shern <olgas@mellanox.com>
Date: Thu, 17 Mar 2016 16:38:57 +0100
Message-Id: <1458229138-20597-5-git-send-email-adrien.mazarguil@6wind.com>
X-Mailer: git-send-email 2.1.4
In-Reply-To: <1458229138-20597-1-git-send-email-adrien.mazarguil@6wind.com>
References: <1457015279-3089-1-git-send-email-adrien.mazarguil@6wind.com>
 <1458229138-20597-1-git-send-email-adrien.mazarguil@6wind.com>
Subject: [dpdk-dev] [PATCH v3 4/5] mlx5: add support for HW packet padding
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Thu, 17 Mar 2016 15:39:28 -0000

From: Olga Shern <olgas@mellanox.com>

Environment variable MLX5_PMD_ENABLE_PADDING enables HW packet padding
in PCI bus transactions.

When packet size is cache aligned and CRC stripping is enabled, 4 fewer
bytes are written to the PCI bus. Enabling padding makes such packets
aligned again.

In cases where PCI bandwidth is the bottleneck, padding can improve
performance by 10%.

This is disabled by default since this can also decrease performance for
unaligned packet sizes.

Signed-off-by: Olga Shern <olgas@mellanox.com>
---
 doc/guides/nics/mlx5.rst               | 14 ++++++++++++++
 doc/guides/rel_notes/release_16_04.rst |  7 +++++++
 drivers/net/mlx5/Makefile              |  5 +++++
 drivers/net/mlx5/mlx5.c                | 28 ++++++++++++++++++++++++++++
 drivers/net/mlx5/mlx5.h                |  5 +++++
 drivers/net/mlx5/mlx5_rxq.c            | 15 +++++++++++++++
 6 files changed, 74 insertions(+)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index 8b63f3f..9df30be 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -156,6 +156,20 @@ Environment variables
   lower performance when there is no backpressure, it is not enabled by
   default.
 
+- ``MLX5_PMD_ENABLE_PADDING``
+
+  Enables HW packet padding in PCI bus transactions.
+
+  When packet size is cache aligned and CRC stripping is enabled, 4 fewer
+  bytes are written to the PCI bus. Enabling padding makes such packets
+  aligned again.
+
+  In cases where PCI bandwidth is the bottleneck, padding can improve
+  performance by 10%.
+
+  This is disabled by default since this can also decrease performance for
+  unaligned packet sizes.
+
 Run-time configuration
 ~~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/doc/guides/rel_notes/release_16_04.rst b/doc/guides/rel_notes/release_16_04.rst
index a498ef7..8eb423f 100644
--- a/doc/guides/rel_notes/release_16_04.rst
+++ b/doc/guides/rel_notes/release_16_04.rst
@@ -144,6 +144,13 @@ This section should contain new features added in this release. Sample format:
 
   Only available with Mellanox OFED >= 3.2.
 
+* **Added mlx5 optional packet padding by HW.**
+
+  Added an option to make PCI bus transactions rounded to multiple of a
+  cache line size for better alignment.
+
+  Only available with Mellanox OFED >= 3.2.
+
 * **Added af_packet dynamic removal function.**
 
   Af_packet device can now be detached using API, like other PMD devices.
diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile
index cc6de2d..a6a3cab 100644
--- a/drivers/net/mlx5/Makefile
+++ b/drivers/net/mlx5/Makefile
@@ -142,6 +142,11 @@ mlx5_autoconf.h: $(RTE_SDK)/scripts/auto-config-h.sh
 		infiniband/verbs.h \
 		enum IBV_EXP_CREATE_WQ_FLAG_SCATTER_FCS \
 		$(AUTOCONF_OUTPUT)
+	$Q sh -- '$<' '$@' \
+		HAVE_VERBS_RX_END_PADDING \
+		infiniband/verbs.h \
+		enum IBV_EXP_CREATE_WQ_FLAG_RX_END_PADDING \
+		$(AUTOCONF_OUTPUT)
 
 $(SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD):.c=.o): mlx5_autoconf.h
 
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index acfb365..94eefb9 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -68,6 +68,25 @@
 #include "mlx5_defs.h"
 
 /**
+ * Retrieve integer value from environment variable.
+ *
+ * @param[in] name
+ *   Environment variable name.
+ *
+ * @return
+ *   Integer value, 0 if the variable is not set.
+ */
+int
+mlx5_getenv_int(const char *name)
+{
+	const char *val = getenv(name);
+
+	if (val == NULL)
+		return 0;
+	return atoi(val);
+}
+
+/**
  * DPDK callback to close the device.
  *
  * Destroy all queues and objects, free memory.
@@ -332,6 +351,9 @@ mlx5_pci_devinit(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev)
 #ifdef HAVE_EXP_DEVICE_ATTR_VLAN_OFFLOADS
 			IBV_EXP_DEVICE_ATTR_VLAN_OFFLOADS |
 #endif /* HAVE_EXP_DEVICE_ATTR_VLAN_OFFLOADS */
+#ifdef HAVE_EXP_CREATE_WQ_FLAG_RX_END_PADDING
+			IBV_EXP_DEVICE_ATTR_RX_PAD_END_ALIGN |
+#endif /* HAVE_EXP_CREATE_WQ_FLAG_RX_END_PADDING */
 			0;
 #endif /* HAVE_EXP_QUERY_DEVICE */
 
@@ -424,6 +446,12 @@ mlx5_pci_devinit(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev)
 		DEBUG("FCS stripping configuration is %ssupported",
 		      (priv->hw_fcs_strip ? "" : "not "));
 
+#ifdef HAVE_VERBS_RX_END_PADDING
+		priv->hw_padding = !!exp_device_attr.rx_pad_end_addr_align;
+#endif /* HAVE_VERBS_RX_END_PADDING */
+		DEBUG("hardware RX end alignment padding is %ssupported",
+		      (priv->hw_padding ? "" : "not "));
+
 #else /* HAVE_EXP_QUERY_DEVICE */
 		priv->ind_table_max_size = RSS_INDIRECTION_TABLE_SIZE;
 #endif /* HAVE_EXP_QUERY_DEVICE */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 9690827..1904d54 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -104,6 +104,7 @@ struct priv {
 	unsigned int hw_csum_l2tun:1; /* Same for L2 tunnels. */
 	unsigned int hw_vlan_strip:1; /* VLAN stripping is supported. */
 	unsigned int hw_fcs_strip:1; /* FCS stripping is supported. */
+	unsigned int hw_padding:1; /* End alignment padding is supported. */
 	unsigned int vf:1; /* This is a VF device. */
 	unsigned int pending_alarm:1; /* An alarm is pending. */
 	/* RX/TX queues. */
@@ -160,6 +161,10 @@ priv_unlock(struct priv *priv)
 	rte_spinlock_unlock(&priv->lock);
 }
 
+/* mlx5.c */
+
+int mlx5_getenv_int(const char *);
+
 /* mlx5_ethdev.c */
 
 struct priv *mlx5_get_priv(struct rte_eth_dev *dev);
diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
index 19a1119..c8af77f 100644
--- a/drivers/net/mlx5/mlx5_rxq.c
+++ b/drivers/net/mlx5/mlx5_rxq.c
@@ -1282,6 +1282,21 @@ rxq_setup(struct rte_eth_dev *dev, struct rxq *rxq, uint16_t desc,
 	      tmpl.crc_present << 2);
 #endif /* HAVE_VERBS_FCS */
 
+#ifdef HAVE_VERBS_RX_END_PADDING
+	if (!mlx5_getenv_int("MLX5_PMD_ENABLE_PADDING"))
+		; /* Nothing else to do. */
+	else if (priv->hw_padding) {
+		INFO("%p: enabling packet padding on queue %p",
+		     (void *)dev, (void *)rxq);
+		attr.wq.flags |= IBV_EXP_CREATE_WQ_FLAG_RX_END_PADDING;
+		attr.wq.comp_mask |= IBV_EXP_CREATE_WQ_FLAGS;
+	} else
+		WARN("%p: packet padding has been requested but is not"
+		     " supported, make sure MLNX_OFED and firmware are"
+		     " up to date",
+		     (void *)dev);
+#endif /* HAVE_VERBS_RX_END_PADDING */
+
 	tmpl.wq = ibv_exp_create_wq(priv->ctx, &attr.wq);
 	if (tmpl.wq == NULL) {
 		ret = (errno ? errno : EINVAL);
-- 
2.1.4