[dpdk-dev] [PATCH v3 0/4] bonding corrections and additions

[dpdk-dev] [PATCH v3 0/4] bonding corrections and additions

[Eric Kinzie]

This patchset makes a couple of small corrections to the bonding driver
and introduces the ability to use an external state machine for mode
4 operation.

Changes in v2:
  . eliminate external_sm field in 802.3ad configuration 
    (rte_eth_bond_8023ad_conf).
  . stop bonding device before changing the periodic callback function.
    start again if needed.
  . remove unnecessary calls to valid_bonded_port_id().
  . do not check for NULL tx_ring.
  . return error in rte_eth_bond_8023ad_ext_slowtx() if packet is not LACP.
  . remove check for external sm configuration in periodic callback
  . check for valid LACPDU in test application's rx callback
  . add "Fixes:" tags

Changes in v3:
  . update rte_eth_bond_version.map

Eric Kinzie (4):
  bond mode 4: copy entire config structure
  bond mode 4: do not ignore multicast
  bond mode 4: allow external state machine
  bond mode 4: tests for external state machine

 app/test/test_link_bonding_mode4.c                |  217 +++++++++++++++++++--
 lib/librte_pmd_bond/rte_eth_bond_8023ad.c         |  174 +++++++++++++++++
 lib/librte_pmd_bond/rte_eth_bond_8023ad.h         |   44 +++++
 lib/librte_pmd_bond/rte_eth_bond_8023ad_private.h |    2 +
 lib/librte_pmd_bond/rte_eth_bond_pmd.c            |    1 +
 lib/librte_pmd_bond/rte_eth_bond_version.map      |    3 +
 6 files changed, 430 insertions(+), 11 deletions(-)

-- 
1.7.10.4

[dpdk-dev] [PATCH v3 1/4] bond mode 4: copy entire config structure

[Eric Kinzie]

From: Eric Kinzie <ekinzie@brocade.com>

  Copy all needed fields from the mode8023ad_private structure in
  bond_mode_8023ad_conf_get().  This help ensure that a subsequent call
  to rte_eth_bond_8023ad_setup() is not passed uninitialized data that
  would result in either incorrect behavior or a failed sanity check.

Fixes: 46fb43683679 ("bond: add mode 4")

Signed-off-by: Eric Kinzie <ehkinzie@gmail.com>
---
 lib/librte_pmd_bond/rte_eth_bond_8023ad.c |    1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/librte_pmd_bond/rte_eth_bond_8023ad.c b/lib/librte_pmd_bond/rte_eth_bond_8023ad.c
index 97a828e..1009d5b 100644
--- a/lib/librte_pmd_bond/rte_eth_bond_8023ad.c
+++ b/lib/librte_pmd_bond/rte_eth_bond_8023ad.c
@@ -1013,6 +1013,7 @@ bond_mode_8023ad_conf_get(struct rte_eth_dev *dev,
 	conf->aggregate_wait_timeout_ms = mode4->aggregate_wait_timeout / ms_ticks;
 	conf->tx_period_ms = mode4->tx_period_timeout / ms_ticks;
 	conf->update_timeout_ms = mode4->update_timeout_us / 1000;
+	conf->rx_marker_period_ms = mode4->rx_marker_timeout / ms_ticks;
 }
 
 void
-- 
1.7.10.4

[dpdk-dev] [PATCH v3 2/4] bond mode 4: do not ignore multicast

[Eric Kinzie]

From: Eric Kinzie <ekinzie@brocade.com>

The bonding PMD in mode 4 puts all enslaved interfaces into promiscuous
mode in order to receive LACPDUs and must filter unwanted packets
after the traffic has been "collected".  Allow broadcast and multicast
through so that ARP and IPv6 neighbor discovery continue to work.

Fixes: 46fb43683679 ("bond: add mode 4")

Signed-off-by: Eric Kinzie <ehkinzie@gmail.com>
---
 app/test/test_link_bonding_mode4.c     |    7 +++++--
 lib/librte_pmd_bond/rte_eth_bond_pmd.c |    1 +
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/app/test/test_link_bonding_mode4.c b/app/test/test_link_bonding_mode4.c
index 02380f9..5a726af 100644
--- a/app/test/test_link_bonding_mode4.c
+++ b/app/test/test_link_bonding_mode4.c
@@ -755,8 +755,11 @@ test_mode4_rx(void)
 	rte_eth_macaddr_get(test_params.bonded_port_id, &bonded_mac);
 	ether_addr_copy(&bonded_mac, &dst_mac);
 
-	/* Assert that dst address is not bonding address */
-	dst_mac.addr_bytes[0]++;
+	/* Assert that dst address is not bonding address.  Do not set the
+	 * least significant bit of the zero byte as this would create a
+	 * multicast address.
+	 */
+	dst_mac.addr_bytes[0] += 2;
 
 	/* First try with promiscuous mode enabled.
 	 * Add 2 packets to each slave. First with bonding MAC address, second with
diff --git a/lib/librte_pmd_bond/rte_eth_bond_pmd.c b/lib/librte_pmd_bond/rte_eth_bond_pmd.c
index c937e6b..1691300 100644
--- a/lib/librte_pmd_bond/rte_eth_bond_pmd.c
+++ b/lib/librte_pmd_bond/rte_eth_bond_pmd.c
@@ -170,6 +170,7 @@ bond_ethdev_rx_burst_8023ad(void *queue, struct rte_mbuf **bufs,
 			 * mode and packet address does not match. */
 			if (unlikely(hdr->ether_type == ether_type_slow_be ||
 				!collecting || (!promisc &&
+					!is_multicast_ether_addr(&hdr->d_addr) &&
 					!is_same_ether_addr(&bond_mac, &hdr->d_addr)))) {
 
 				if (hdr->ether_type == ether_type_slow_be) {
-- 
1.7.10.4

[dpdk-dev] [PATCH v3 3/4] bond mode 4: allow external state machine

[Eric Kinzie]

From: Eric Kinzie <ekinzie@brocade.com>

  Provide functions to allow an external 802.3ad state machine to transmit
  and recieve LACPDUs and to set the collection/distribution flags on
  slave interfaces.

Signed-off-by: Eric Kinzie <ehkinzie@gmail.com>
---
 lib/librte_pmd_bond/rte_eth_bond_8023ad.c         |  173 +++++++++++++++++++++
 lib/librte_pmd_bond/rte_eth_bond_8023ad.h         |   44 ++++++
 lib/librte_pmd_bond/rte_eth_bond_8023ad_private.h |    2 +
 lib/librte_pmd_bond/rte_eth_bond_version.map      |    3 +
 4 files changed, 222 insertions(+)

diff --git a/lib/librte_pmd_bond/rte_eth_bond_8023ad.c b/lib/librte_pmd_bond/rte_eth_bond_8023ad.c
index 1009d5b..326e899 100644
--- a/lib/librte_pmd_bond/rte_eth_bond_8023ad.c
+++ b/lib/librte_pmd_bond/rte_eth_bond_8023ad.c
@@ -42,6 +42,8 @@
 
 #include "rte_eth_bond_private.h"
 
+static void bond_mode_8023ad_ext_periodic_cb(void *arg);
+
 #ifdef RTE_LIBRTE_BOND_DEBUG_8023AD
 #define MODE4_DEBUG(fmt, ...) RTE_LOG(DEBUG, PMD, "%6u [Port %u: %s] " fmt, \
 			bond_dbg_get_time_diff_ms(), slave_id, \
@@ -1014,6 +1016,7 @@ bond_mode_8023ad_conf_get(struct rte_eth_dev *dev,
 	conf->tx_period_ms = mode4->tx_period_timeout / ms_ticks;
 	conf->update_timeout_ms = mode4->update_timeout_us / 1000;
 	conf->rx_marker_period_ms = mode4->rx_marker_timeout / ms_ticks;
+	conf->slowrx_cb = mode4->slowrx_cb;
 }
 
 void
@@ -1035,8 +1038,11 @@ bond_mode_8023ad_setup(struct rte_eth_dev *dev,
 		conf->tx_period_ms = BOND_8023AD_TX_MACHINE_PERIOD_MS;
 		conf->rx_marker_period_ms = BOND_8023AD_RX_MARKER_PERIOD_MS;
 		conf->update_timeout_ms = BOND_MODE_8023AX_UPDATE_TIMEOUT_MS;
+		conf->slowrx_cb = NULL;
 	}
 
+	bond_mode_8023ad_stop(dev);
+
 	mode4->fast_periodic_timeout = conf->fast_periodic_ms * ms_ticks;
 	mode4->slow_periodic_timeout = conf->slow_periodic_ms * ms_ticks;
 	mode4->short_timeout = conf->short_timeout_ms * ms_ticks;
@@ -1045,6 +1051,10 @@ bond_mode_8023ad_setup(struct rte_eth_dev *dev,
 	mode4->tx_period_timeout = conf->tx_period_ms * ms_ticks;
 	mode4->rx_marker_timeout = conf->rx_marker_period_ms * ms_ticks;
 	mode4->update_timeout_us = conf->update_timeout_ms * 1000;
+	mode4->slowrx_cb = conf->slowrx_cb;
+
+	if (dev->data->dev_started)
+		bond_mode_8023ad_start(dev);
 }
 
 int
@@ -1062,6 +1072,13 @@ bond_mode_8023ad_enable(struct rte_eth_dev *bond_dev)
 int
 bond_mode_8023ad_start(struct rte_eth_dev *bond_dev)
 {
+	struct bond_dev_private *internals = bond_dev->data->dev_private;
+	struct mode8023ad_private *mode4 = &internals->mode4;
+
+	if (mode4->slowrx_cb)
+		return rte_eal_alarm_set(BOND_MODE_8023AX_UPDATE_TIMEOUT_MS * 1000,
+			&bond_mode_8023ad_ext_periodic_cb, bond_dev);
+
 	return rte_eal_alarm_set(BOND_MODE_8023AX_UPDATE_TIMEOUT_MS * 1000,
 			&bond_mode_8023ad_periodic_cb, bond_dev);
 }
@@ -1069,6 +1086,13 @@ bond_mode_8023ad_start(struct rte_eth_dev *bond_dev)
 void
 bond_mode_8023ad_stop(struct rte_eth_dev *bond_dev)
 {
+	struct bond_dev_private *internals = bond_dev->data->dev_private;
+	struct mode8023ad_private *mode4 = &internals->mode4;
+
+	if (mode4->slowrx_cb) {
+		rte_eal_alarm_cancel(&bond_mode_8023ad_ext_periodic_cb, bond_dev);
+		return;
+	}
 	rte_eal_alarm_cancel(&bond_mode_8023ad_periodic_cb, bond_dev);
 }
 
@@ -1215,3 +1239,152 @@ rte_eth_bond_8023ad_slave_info(uint8_t port_id, uint8_t slave_id,
 	info->agg_port_id = port->aggregator_port_id;
 	return 0;
 }
+
+int
+rte_eth_bond_8023ad_ext_collect(uint8_t port_id, uint8_t slave_id, int enabled)
+{
+	struct rte_eth_dev *bond_dev;
+	struct bond_dev_private *internals;
+	struct mode8023ad_private *mode4;
+	struct port *port;
+
+	if (rte_eth_bond_mode_get(port_id) != BONDING_MODE_8023AD)
+		return -EINVAL;
+
+	bond_dev = &rte_eth_devices[port_id];
+
+	if (!bond_dev->data->dev_started)
+		return -EINVAL;
+
+	internals = bond_dev->data->dev_private;
+	if (find_slave_by_id(internals->active_slaves,
+			internals->active_slave_count, slave_id) ==
+				internals->active_slave_count)
+		return -EINVAL;
+
+	mode4 = &internals->mode4;
+	if (mode4->slowrx_cb == NULL)
+		return -EINVAL;
+
+	port = &mode_8023ad_ports[slave_id];
+
+	if (enabled)
+		ACTOR_STATE_SET(port, COLLECTING);
+	else
+		ACTOR_STATE_CLR(port, COLLECTING);
+
+	return 0;
+}
+
+int
+rte_eth_bond_8023ad_ext_distrib(uint8_t port_id, uint8_t slave_id, int enabled)
+{
+	struct rte_eth_dev *bond_dev;
+	struct bond_dev_private *internals;
+	struct mode8023ad_private *mode4;
+	struct port *port;
+
+	if (rte_eth_bond_mode_get(port_id) != BONDING_MODE_8023AD)
+		return -EINVAL;
+
+	bond_dev = &rte_eth_devices[port_id];
+
+	if (!bond_dev->data->dev_started)
+		return -EINVAL;
+
+	internals = bond_dev->data->dev_private;
+	if (find_slave_by_id(internals->active_slaves,
+			internals->active_slave_count, slave_id) ==
+				internals->active_slave_count)
+		return -EINVAL;
+
+	mode4 = &internals->mode4;
+	if (mode4->slowrx_cb == NULL)
+		return -EINVAL;
+
+	port = &mode_8023ad_ports[slave_id];
+
+	if (enabled)
+		ACTOR_STATE_SET(port, DISTRIBUTING);
+	else
+		ACTOR_STATE_CLR(port, DISTRIBUTING);
+
+	return 0;
+}
+
+int
+rte_eth_bond_8023ad_ext_slowtx(uint8_t port_id, uint8_t slave_id,
+		struct rte_mbuf *lacp_pkt)
+{
+	struct rte_eth_dev *bond_dev;
+	struct bond_dev_private *internals;
+	struct mode8023ad_private *mode4;
+	struct port *port;
+
+	if (rte_eth_bond_mode_get(port_id) != BONDING_MODE_8023AD)
+		return -EINVAL;
+
+	bond_dev = &rte_eth_devices[port_id];
+
+	if (!bond_dev->data->dev_started)
+		return -EINVAL;
+
+	internals = bond_dev->data->dev_private;
+	if (find_slave_by_id(internals->active_slaves,
+			internals->active_slave_count, slave_id) ==
+				internals->active_slave_count)
+		return -EINVAL;
+
+	mode4 = &internals->mode4;
+	if (mode4->slowrx_cb == NULL)
+		return -EINVAL;
+
+	port = &mode_8023ad_ports[slave_id];
+
+	if (rte_pktmbuf_pkt_len(lacp_pkt) < sizeof(struct lacpdu_header))
+		return -EINVAL;
+
+	struct lacpdu_header *lacp;
+
+	/* only enqueue LACPDUs */
+	lacp = rte_pktmbuf_mtod(lacp_pkt, struct lacpdu_header *);
+	if (lacp->lacpdu.subtype != SLOW_SUBTYPE_LACP)
+		return -EINVAL;
+
+	MODE4_DEBUG("sending LACP frame\n");
+
+	return rte_ring_enqueue(port->tx_ring, lacp_pkt);
+}
+
+static void
+bond_mode_8023ad_ext_periodic_cb(void *arg)
+{
+	struct rte_eth_dev *bond_dev = arg;
+	struct bond_dev_private *internals = bond_dev->data->dev_private;
+	struct mode8023ad_private *mode4 = &internals->mode4;
+	struct port *port;
+	void *pkt = NULL;
+	uint16_t i, slave_id;
+
+	for (i = 0; i < internals->active_slave_count; i++) {
+		slave_id = internals->active_slaves[i];
+		port = &mode_8023ad_ports[slave_id];
+
+		if (rte_ring_dequeue(port->rx_ring, &pkt) == 0) {
+			struct rte_mbuf *lacp_pkt = pkt;
+			struct lacpdu_header *lacp;
+
+			lacp = rte_pktmbuf_mtod(lacp_pkt,
+						struct lacpdu_header *);
+			RTE_VERIFY(lacp->lacpdu.subtype == SLOW_SUBTYPE_LACP);
+
+			/* This is LACP frame so pass it to rx callback.
+			 * Callback is responsible for freeing mbuf.
+			 */
+			mode4->slowrx_cb(slave_id, lacp_pkt);
+		}
+	}
+
+	rte_eal_alarm_set(internals->mode4.update_timeout_us,
+			bond_mode_8023ad_ext_periodic_cb, arg);
+}
diff --git a/lib/librte_pmd_bond/rte_eth_bond_8023ad.h b/lib/librte_pmd_bond/rte_eth_bond_8023ad.h
index ebd0e93..8cfa3d3 100644
--- a/lib/librte_pmd_bond/rte_eth_bond_8023ad.h
+++ b/lib/librte_pmd_bond/rte_eth_bond_8023ad.h
@@ -64,6 +64,8 @@ extern "C" {
 #define MARKER_TLV_TYPE_INFO                0x01
 #define MARKER_TLV_TYPE_RESP                0x02
 
+typedef void (*rte_eth_bond_8023ad_ext_slowrx_fn)(uint8_t slave_id, struct rte_mbuf *lacp_pkt);
+
 enum rte_bond_8023ad_selection {
 	UNSELECTED,
 	STANDBY,
@@ -157,6 +159,7 @@ struct rte_eth_bond_8023ad_conf {
 	uint32_t tx_period_ms;
 	uint32_t rx_marker_period_ms;
 	uint32_t update_timeout_ms;
+	rte_eth_bond_8023ad_ext_slowrx_fn slowrx_cb;
 };
 
 struct rte_eth_bond_8023ad_slave_info {
@@ -219,4 +222,45 @@ rte_eth_bond_8023ad_slave_info(uint8_t port_id, uint8_t slave_id,
 }
 #endif
 
+/**
+ * Configure a slave port to start collecting.
+ *
+ * @param port_id	Bonding device id
+ * @param slave_id	Port id of valid slave.
+ * @param enabled	Non-zero when collection enabled.
+ * @return
+ *   0 - if ok
+ *   -EINVAL if slave is not valid.
+ */
+int
+rte_eth_bond_8023ad_ext_collect(uint8_t port_id, uint8_t slave_id, int enabled);
+
+/**
+ * Configure a slave port to start distributing.
+ *
+ * @param port_id	Bonding device id
+ * @param slave_id	Port id of valid slave.
+ * @param enabled	Non-zero when distribution enabled.
+ * @return
+ *   0 - if ok
+ *   -EINVAL if slave is not valid.
+ */
+int
+rte_eth_bond_8023ad_ext_distrib(uint8_t port_id, uint8_t slave_id, int enabled);
+
+/**
+ * LACPDU transmit path for external 802.3ad state machine.  Caller retains
+ * ownership of the packet on failure.
+ *
+ * @param port_id	Bonding device id
+ * @param slave_id	Port ID of valid slave device.
+ * @param lacp_pkt	mbuf containing LACPDU.
+ *
+ * @return
+ *   0 on success, negative value otherwise.
+ */
+int
+rte_eth_bond_8023ad_ext_slowtx(uint8_t port_id, uint8_t slave_id,
+		struct rte_mbuf *lacp_pkt);
+
 #endif /* RTE_ETH_BOND_8023AD_H_ */
diff --git a/lib/librte_pmd_bond/rte_eth_bond_8023ad_private.h b/lib/librte_pmd_bond/rte_eth_bond_8023ad_private.h
index 8adee70..9e15ece 100644
--- a/lib/librte_pmd_bond/rte_eth_bond_8023ad_private.h
+++ b/lib/librte_pmd_bond/rte_eth_bond_8023ad_private.h
@@ -173,6 +173,8 @@ struct mode8023ad_private {
 	uint64_t tx_period_timeout;
 	uint64_t rx_marker_timeout;
 	uint64_t update_timeout_us;
+	rte_eth_bond_8023ad_ext_slowrx_fn slowrx_cb;
+	uint8_t external_sm;
 };
 
 /**
diff --git a/lib/librte_pmd_bond/rte_eth_bond_version.map b/lib/librte_pmd_bond/rte_eth_bond_version.map
index 135999e..f32bd74 100644
--- a/lib/librte_pmd_bond/rte_eth_bond_version.map
+++ b/lib/librte_pmd_bond/rte_eth_bond_version.map
@@ -17,6 +17,9 @@ DPDK_2.0 {
 	rte_eth_bond_slaves_get;
 	rte_eth_bond_xmit_policy_get;
 	rte_eth_bond_xmit_policy_set;
+	rte_eth_bond_8023ad_ext_collect;
+	rte_eth_bond_8023ad_ext_distrib;
+	rte_eth_bond_8023ad_ext_slowtx;
 
 	local: *;
 };
-- 
1.7.10.4

[dpdk-dev] [PATCH v3 4/4] bond mode 4: tests for external state machine

[Eric Kinzie]

From: Eric Kinzie <ekinzie@brocade.com>

  This adds test cases for exercising the external state machine API to
  the mode 4 autotest.

Signed-off-by: Eric Kinzie <ehkinzie@gmail.com>
---
 app/test/test_link_bonding_mode4.c |  210 ++++++++++++++++++++++++++++++++++--
 1 file changed, 201 insertions(+), 9 deletions(-)

diff --git a/app/test/test_link_bonding_mode4.c b/app/test/test_link_bonding_mode4.c
index 5a726af..c191ac5 100644
--- a/app/test/test_link_bonding_mode4.c
+++ b/app/test/test_link_bonding_mode4.c
@@ -155,6 +155,8 @@ static struct rte_eth_conf default_pmd_conf = {
 	.lpbk_mode = 0,
 };
 
+static uint8_t lacpdu_rx_count[RTE_MAX_ETHPORTS] = {0, };
+
 #define FOR_EACH(_i, _item, _array, _size) \
 	for (_i = 0, _item = &_array[0]; _i < _size && (_item = &_array[_i]); _i++)
 
@@ -324,8 +326,26 @@ remove_slave(struct slave_conf *slave)
 	return 0;
 }
 
+static void
+lacp_recv_cb(uint8_t slave_id, struct rte_mbuf *lacp_pkt)
+{
+	struct ether_hdr *hdr;
+	struct slow_protocol_frame *slow_hdr;
+
+	RTE_VERIFY(lacp_pkt != NULL);
+
+	hdr = rte_pktmbuf_mtod(lacp_pkt, struct ether_hdr *);
+	RTE_VERIFY(hdr->ether_type == rte_cpu_to_be_16(ETHER_TYPE_SLOW));
+
+	slow_hdr = rte_pktmbuf_mtod(lacp_pkt, struct slow_protocol_frame *);
+	RTE_VERIFY(slow_hdr->slow_protocol.subtype == SLOW_SUBTYPE_LACP);
+
+	lacpdu_rx_count[slave_id]++;
+	rte_pktmbuf_free(lacp_pkt);
+}
+
 static int
-initialize_bonded_device_with_slaves(uint8_t slave_count, uint8_t start)
+initialize_bonded_device_with_slaves(uint8_t slave_count, uint8_t external_sm)
 {
 	uint8_t i;
 
@@ -341,9 +361,17 @@ initialize_bonded_device_with_slaves(uint8_t slave_count, uint8_t start)
 	rte_eth_bond_8023ad_setup(test_params.bonded_port_id, NULL);
 	rte_eth_promiscuous_disable(test_params.bonded_port_id);
 
-	if (start)
-		TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bonded_port_id),
-			"Failed to start bonded device");
+	if (external_sm) {
+		struct rte_eth_bond_8023ad_conf conf;
+
+		rte_eth_bond_8023ad_conf_get(test_params.bonded_port_id, &conf);
+		conf.slowrx_cb = lacp_recv_cb;
+		rte_eth_bond_8023ad_setup(test_params.bonded_port_id, &conf);
+
+	}
+
+	TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bonded_port_id),
+		"Failed to start bonded device");
 
 	return TEST_SUCCESS;
 }
@@ -648,7 +676,7 @@ test_mode4_lacp(void)
 {
 	int retval;
 
-	retval = initialize_bonded_device_with_slaves(TEST_LACP_SLAVE_COUT, 1);
+	retval = initialize_bonded_device_with_slaves(TEST_LACP_SLAVE_COUT, 0);
 	TEST_ASSERT_SUCCESS(retval, "Failed to initialize bonded device");
 
 	/* Test LACP handshake function */
@@ -746,7 +774,7 @@ test_mode4_rx(void)
 	struct ether_addr dst_mac;
 	struct ether_addr bonded_mac;
 
-	retval = initialize_bonded_device_with_slaves(TEST_PROMISC_SLAVE_COUNT, 1);
+	retval = initialize_bonded_device_with_slaves(TEST_PROMISC_SLAVE_COUNT, 0);
 	TEST_ASSERT_SUCCESS(retval, "Failed to initialize bonded device");
 
 	retval = bond_handshake();
@@ -923,7 +951,7 @@ test_mode4_tx_burst(void)
 	struct ether_addr dst_mac = { { 0x00, 0xFF, 0x00, 0xFF, 0x00, 0x00 } };
 	struct ether_addr bonded_mac;
 
-	retval = initialize_bonded_device_with_slaves(TEST_TX_SLAVE_COUNT, 1);
+	retval = initialize_bonded_device_with_slaves(TEST_TX_SLAVE_COUNT, 0);
 	TEST_ASSERT_SUCCESS(retval, "Failed to initialize bonded device");
 
 	retval = bond_handshake();
@@ -1107,7 +1135,7 @@ test_mode4_marker(void)
 	uint8_t i, j;
 	const uint16_t ethtype_slow_be = rte_be_to_cpu_16(ETHER_TYPE_SLOW);
 
-	retval = initialize_bonded_device_with_slaves(TEST_MARKER_SLAVE_COUT, 1);
+	retval = initialize_bonded_device_with_slaves(TEST_MARKER_SLAVE_COUT, 0);
 	TEST_ASSERT_SUCCESS(retval, "Failed to initialize bonded device");
 
 	/* Test LACP handshake function */
@@ -1192,7 +1220,7 @@ test_mode4_expired(void)
 
 	struct rte_eth_bond_8023ad_conf conf;
 
-	retval = initialize_bonded_device_with_slaves(TEST_EXPIRED_SLAVE_COUNT, 1);
+	retval = initialize_bonded_device_with_slaves(TEST_EXPIRED_SLAVE_COUNT, 0);
 	/* Set custom timeouts to make test last shorter. */
 	rte_eth_bond_8023ad_conf_get(test_params.bonded_port_id, &conf);
 	conf.fast_periodic_ms = 100;
@@ -1274,6 +1302,156 @@ test_mode4_expired(void)
 }
 
 static int
+test_mode4_ext_ctrl(void)
+{
+	/*
+	 * configure bonded interface without the external sm enabled
+	 *   . try to transmit lacpdu (should fail)
+	 *   . try to set collecting and distributing flags (should fail)
+	 * reconfigure w/external sm
+	 *   . transmit one lacpdu on each slave using new api
+	 *   . make sure each slave receives one lacpdu using the callback api
+	 *   . transmit one data pdu on each slave (should fail)
+	 *   . enable distribution and collection, send one data pdu each again
+	 */
+
+	int retval;
+	struct slave_conf *slave = NULL;
+	uint8_t i;
+
+	struct rte_mbuf *lacp_tx_buf[SLAVE_COUNT];
+	struct ether_addr src_mac, dst_mac;
+	struct lacpdu_header lacpdu = {
+		.lacpdu = {
+			.subtype = SLOW_SUBTYPE_LACP,
+		},
+	};
+
+	ether_addr_copy(&parnter_system, &src_mac);
+	ether_addr_copy(&slow_protocol_mac_addr, &dst_mac);
+
+	initialize_eth_header(&lacpdu.eth_hdr, &src_mac, &dst_mac,
+			      ETHER_TYPE_SLOW, 0, 0);
+
+	for (i = 0; i < SLAVE_COUNT; i++) {
+		lacp_tx_buf[i] = rte_pktmbuf_alloc(test_params.mbuf_pool);
+		rte_memcpy(rte_pktmbuf_mtod(lacp_tx_buf[i], char *),
+			   &lacpdu, sizeof(lacpdu));
+		rte_pktmbuf_pkt_len(lacp_tx_buf[i]) = sizeof(lacpdu);
+	}
+
+	retval = initialize_bonded_device_with_slaves(TEST_TX_SLAVE_COUNT, 0);
+	TEST_ASSERT_SUCCESS(retval, "Failed to initialize bonded device");
+
+	FOR_EACH_SLAVE(i, slave) {
+		TEST_ASSERT_FAIL(rte_eth_bond_8023ad_ext_slowtx(
+						test_params.bonded_port_id,
+						slave->port_id, lacp_tx_buf[i]),
+				 "Slave should not allow manual LACP xmit");
+		TEST_ASSERT_FAIL(rte_eth_bond_8023ad_ext_collect(
+						test_params.bonded_port_id,
+						slave->port_id, 1),
+				 "Slave should not allow external state controls");
+	}
+
+	free_pkts(lacp_tx_buf, RTE_DIM(lacp_tx_buf));
+
+	retval = remove_slaves_and_stop_bonded_device();
+	TEST_ASSERT_SUCCESS(retval, "Bonded device cleanup failed.");
+
+	return TEST_SUCCESS;
+}
+
+
+static int
+test_mode4_ext_lacp(void)
+{
+	int retval;
+	struct slave_conf *slave = NULL;
+	uint8_t all_slaves_done = 0, i;
+	uint16_t nb_pkts;
+	const unsigned delay = bond_get_update_timeout_ms();
+
+	struct rte_mbuf *lacp_tx_buf[SLAVE_COUNT];
+	struct rte_mbuf *buf[SLAVE_COUNT];
+	struct ether_addr src_mac, dst_mac;
+	struct lacpdu_header lacpdu = {
+		.lacpdu = {
+			.subtype = SLOW_SUBTYPE_LACP,
+		},
+	};
+
+	ether_addr_copy(&parnter_system, &src_mac);
+	ether_addr_copy(&slow_protocol_mac_addr, &dst_mac);
+
+	initialize_eth_header(&lacpdu.eth_hdr, &src_mac, &dst_mac,
+			      ETHER_TYPE_SLOW, 0, 0);
+
+	for (i = 0; i < SLAVE_COUNT; i++) {
+		lacp_tx_buf[i] = rte_pktmbuf_alloc(test_params.mbuf_pool);
+		rte_memcpy(rte_pktmbuf_mtod(lacp_tx_buf[i], char *),
+			   &lacpdu, sizeof(lacpdu));
+		rte_pktmbuf_pkt_len(lacp_tx_buf[i]) = sizeof(lacpdu);
+	}
+
+	retval = initialize_bonded_device_with_slaves(TEST_TX_SLAVE_COUNT, 1);
+	TEST_ASSERT_SUCCESS(retval, "Failed to initialize bonded device");
+
+	memset(lacpdu_rx_count, 0, sizeof(lacpdu_rx_count));
+
+	/* Wait for new settings to be applied. */
+	for (i = 0; i < 30; ++i)
+		rte_delay_ms(delay);
+
+	FOR_EACH_SLAVE(i, slave) {
+		retval = rte_eth_bond_8023ad_ext_slowtx(
+						test_params.bonded_port_id,
+						slave->port_id, lacp_tx_buf[i]);
+		TEST_ASSERT_SUCCESS(retval,
+				    "Slave should allow manual LACP xmit");
+	}
+
+	nb_pkts = bond_tx(NULL, 0);
+	TEST_ASSERT_EQUAL(nb_pkts, 0, "Packets transmitted unexpectedly");
+
+	FOR_EACH_SLAVE(i, slave) {
+		nb_pkts = slave_get_pkts(slave, buf, RTE_DIM(buf));
+		TEST_ASSERT_EQUAL(nb_pkts, 1, "found %u packets on slave %d\n",
+				  nb_pkts, i);
+		slave_put_pkts(slave, buf, nb_pkts);
+	}
+
+	nb_pkts = bond_rx(buf, RTE_DIM(buf));
+	free_pkts(buf, nb_pkts);
+	TEST_ASSERT_EQUAL(nb_pkts, 0, "Packets received unexpectedly");
+
+	/* wait for the periodic callback to run */
+	for (i = 0; i < 30 && all_slaves_done == 0; ++i) {
+		uint8_t s, total = 0;
+
+		rte_delay_ms(delay);
+		FOR_EACH_SLAVE(s, slave) {
+			total += lacpdu_rx_count[slave->port_id];
+		}
+
+		if (total >= SLAVE_COUNT)
+			all_slaves_done = 1;
+	}
+
+	FOR_EACH_SLAVE(i, slave) {
+		TEST_ASSERT_EQUAL(lacpdu_rx_count[slave->port_id], 1,
+				  "Slave port %u should have received 1 lacpdu (count=%u)",
+				  slave->port_id,
+				  lacpdu_rx_count[slave->port_id]);
+	}
+
+	retval = remove_slaves_and_stop_bonded_device();
+	TEST_ASSERT_SUCCESS(retval, "Test cleanup failed.");
+
+	return TEST_SUCCESS;
+}
+
+static int
 check_environment(void)
 {
 	struct slave_conf *port;
@@ -1389,6 +1567,18 @@ test_mode4_expired_wrapper(void)
 	return test_mode4_executor(&test_mode4_expired);
 }
 
+static int
+test_mode4_ext_ctrl_wrapper(void)
+{
+	return test_mode4_executor(&test_mode4_ext_ctrl);
+}
+
+static int
+test_mode4_ext_lacp_wrapper(void)
+{
+	return test_mode4_executor(&test_mode4_ext_lacp);
+}
+
 static struct unit_test_suite link_bonding_mode4_test_suite  = {
 	.suite_name = "Link Bonding mode 4 Unit Test Suite",
 	.setup = test_setup,
@@ -1399,6 +1589,8 @@ static struct unit_test_suite link_bonding_mode4_test_suite  = {
 		TEST_CASE_NAMED("test_mode4_tx_burst", test_mode4_tx_burst_wrapper),
 		TEST_CASE_NAMED("test_mode4_marker", test_mode4_marker_wrapper),
 		TEST_CASE_NAMED("test_mode4_expired", test_mode4_expired_wrapper),
+		TEST_CASE_NAMED("test_mode4_ext_ctrl", test_mode4_ext_ctrl_wrapper),
+		TEST_CASE_NAMED("test_mode4_ext_lacp", test_mode4_ext_lacp_wrapper),
 		{ NULL, NULL, NULL, NULL, NULL } /**< NULL terminate unit test array */
 	}
 };
-- 
1.7.10.4

Re: [dpdk-dev] [PATCH v3 3/4] bond mode 4: allow external state machine

[Neil Horman]

On Wed, Apr 22, 2015 at 09:27:42AM -0700, Eric Kinzie wrote:
> From: Eric Kinzie <ekinzie@brocade.com>
> 
>   Provide functions to allow an external 802.3ad state machine to transmit
>   and recieve LACPDUs and to set the collection/distribution flags on
>   slave interfaces.
> 
> Signed-off-by: Eric Kinzie <ehkinzie@gmail.com>
> ---
>  lib/librte_pmd_bond/rte_eth_bond_8023ad.c         |  173 +++++++++++++++++++++
>  lib/librte_pmd_bond/rte_eth_bond_8023ad.h         |   44 ++++++
>  lib/librte_pmd_bond/rte_eth_bond_8023ad_private.h |    2 +
>  lib/librte_pmd_bond/rte_eth_bond_version.map      |    3 +
>  4 files changed, 222 insertions(+)
> 
> diff --git a/lib/librte_pmd_bond/rte_eth_bond_8023ad.c b/lib/librte_pmd_bond/rte_eth_bond_8023ad.c
> index 1009d5b..326e899 100644
> --- a/lib/librte_pmd_bond/rte_eth_bond_8023ad.c
> +++ b/lib/librte_pmd_bond/rte_eth_bond_8023ad.c
> @@ -42,6 +42,8 @@
>  
>  #include "rte_eth_bond_private.h"
>  
> +static void bond_mode_8023ad_ext_periodic_cb(void *arg);
> +
>  #ifdef RTE_LIBRTE_BOND_DEBUG_8023AD
>  #define MODE4_DEBUG(fmt, ...) RTE_LOG(DEBUG, PMD, "%6u [Port %u: %s] " fmt, \
>  			bond_dbg_get_time_diff_ms(), slave_id, \
> @@ -1014,6 +1016,7 @@ bond_mode_8023ad_conf_get(struct rte_eth_dev *dev,
>  	conf->tx_period_ms = mode4->tx_period_timeout / ms_ticks;
>  	conf->update_timeout_ms = mode4->update_timeout_us / 1000;
>  	conf->rx_marker_period_ms = mode4->rx_marker_timeout / ms_ticks;
> +	conf->slowrx_cb = mode4->slowrx_cb;
>  }
>  
>  void
> @@ -1035,8 +1038,11 @@ bond_mode_8023ad_setup(struct rte_eth_dev *dev,
>  		conf->tx_period_ms = BOND_8023AD_TX_MACHINE_PERIOD_MS;
>  		conf->rx_marker_period_ms = BOND_8023AD_RX_MARKER_PERIOD_MS;
>  		conf->update_timeout_ms = BOND_MODE_8023AX_UPDATE_TIMEOUT_MS;
> +		conf->slowrx_cb = NULL;
>  	}
>  
> +	bond_mode_8023ad_stop(dev);
> +
>  	mode4->fast_periodic_timeout = conf->fast_periodic_ms * ms_ticks;
>  	mode4->slow_periodic_timeout = conf->slow_periodic_ms * ms_ticks;
>  	mode4->short_timeout = conf->short_timeout_ms * ms_ticks;
> @@ -1045,6 +1051,10 @@ bond_mode_8023ad_setup(struct rte_eth_dev *dev,
>  	mode4->tx_period_timeout = conf->tx_period_ms * ms_ticks;
>  	mode4->rx_marker_timeout = conf->rx_marker_period_ms * ms_ticks;
>  	mode4->update_timeout_us = conf->update_timeout_ms * 1000;
> +	mode4->slowrx_cb = conf->slowrx_cb;
> +
> +	if (dev->data->dev_started)
> +		bond_mode_8023ad_start(dev);
>  }
>  
>  int
> @@ -1062,6 +1072,13 @@ bond_mode_8023ad_enable(struct rte_eth_dev *bond_dev)
>  int
>  bond_mode_8023ad_start(struct rte_eth_dev *bond_dev)
>  {
> +	struct bond_dev_private *internals = bond_dev->data->dev_private;
> +	struct mode8023ad_private *mode4 = &internals->mode4;
> +
> +	if (mode4->slowrx_cb)
> +		return rte_eal_alarm_set(BOND_MODE_8023AX_UPDATE_TIMEOUT_MS * 1000,
> +			&bond_mode_8023ad_ext_periodic_cb, bond_dev);
> +
>  	return rte_eal_alarm_set(BOND_MODE_8023AX_UPDATE_TIMEOUT_MS * 1000,
>  			&bond_mode_8023ad_periodic_cb, bond_dev);
>  }
> @@ -1069,6 +1086,13 @@ bond_mode_8023ad_start(struct rte_eth_dev *bond_dev)
>  void
>  bond_mode_8023ad_stop(struct rte_eth_dev *bond_dev)
>  {
> +	struct bond_dev_private *internals = bond_dev->data->dev_private;
> +	struct mode8023ad_private *mode4 = &internals->mode4;
> +
> +	if (mode4->slowrx_cb) {
> +		rte_eal_alarm_cancel(&bond_mode_8023ad_ext_periodic_cb, bond_dev);
> +		return;
> +	}
>  	rte_eal_alarm_cancel(&bond_mode_8023ad_periodic_cb, bond_dev);
>  }
>  
> @@ -1215,3 +1239,152 @@ rte_eth_bond_8023ad_slave_info(uint8_t port_id, uint8_t slave_id,
>  	info->agg_port_id = port->aggregator_port_id;
>  	return 0;
>  }
> +
> +int
> +rte_eth_bond_8023ad_ext_collect(uint8_t port_id, uint8_t slave_id, int enabled)
> +{
> +	struct rte_eth_dev *bond_dev;
> +	struct bond_dev_private *internals;
> +	struct mode8023ad_private *mode4;
> +	struct port *port;
> +
> +	if (rte_eth_bond_mode_get(port_id) != BONDING_MODE_8023AD)
> +		return -EINVAL;
> +
> +	bond_dev = &rte_eth_devices[port_id];
> +
> +	if (!bond_dev->data->dev_started)
> +		return -EINVAL;
> +
> +	internals = bond_dev->data->dev_private;
> +	if (find_slave_by_id(internals->active_slaves,
> +			internals->active_slave_count, slave_id) ==
> +				internals->active_slave_count)
> +		return -EINVAL;
> +
> +	mode4 = &internals->mode4;
> +	if (mode4->slowrx_cb == NULL)
> +		return -EINVAL;
> +
> +	port = &mode_8023ad_ports[slave_id];
> +
> +	if (enabled)
> +		ACTOR_STATE_SET(port, COLLECTING);
> +	else
> +		ACTOR_STATE_CLR(port, COLLECTING);
> +
> +	return 0;
> +}
> +
> +int
> +rte_eth_bond_8023ad_ext_distrib(uint8_t port_id, uint8_t slave_id, int enabled)
> +{
> +	struct rte_eth_dev *bond_dev;
> +	struct bond_dev_private *internals;
> +	struct mode8023ad_private *mode4;
> +	struct port *port;
> +
> +	if (rte_eth_bond_mode_get(port_id) != BONDING_MODE_8023AD)
> +		return -EINVAL;
> +
> +	bond_dev = &rte_eth_devices[port_id];
> +
> +	if (!bond_dev->data->dev_started)
> +		return -EINVAL;
> +
> +	internals = bond_dev->data->dev_private;
> +	if (find_slave_by_id(internals->active_slaves,
> +			internals->active_slave_count, slave_id) ==
> +				internals->active_slave_count)
> +		return -EINVAL;
> +
> +	mode4 = &internals->mode4;
> +	if (mode4->slowrx_cb == NULL)
> +		return -EINVAL;
> +
> +	port = &mode_8023ad_ports[slave_id];
> +
> +	if (enabled)
> +		ACTOR_STATE_SET(port, DISTRIBUTING);
> +	else
> +		ACTOR_STATE_CLR(port, DISTRIBUTING);
> +
> +	return 0;
> +}
> +
> +int
> +rte_eth_bond_8023ad_ext_slowtx(uint8_t port_id, uint8_t slave_id,
> +		struct rte_mbuf *lacp_pkt)
> +{
> +	struct rte_eth_dev *bond_dev;
> +	struct bond_dev_private *internals;
> +	struct mode8023ad_private *mode4;
> +	struct port *port;
> +
> +	if (rte_eth_bond_mode_get(port_id) != BONDING_MODE_8023AD)
> +		return -EINVAL;
> +
> +	bond_dev = &rte_eth_devices[port_id];
> +
> +	if (!bond_dev->data->dev_started)
> +		return -EINVAL;
> +
> +	internals = bond_dev->data->dev_private;
> +	if (find_slave_by_id(internals->active_slaves,
> +			internals->active_slave_count, slave_id) ==
> +				internals->active_slave_count)
> +		return -EINVAL;
> +
> +	mode4 = &internals->mode4;
> +	if (mode4->slowrx_cb == NULL)
> +		return -EINVAL;
> +
> +	port = &mode_8023ad_ports[slave_id];
> +
> +	if (rte_pktmbuf_pkt_len(lacp_pkt) < sizeof(struct lacpdu_header))
> +		return -EINVAL;
> +
> +	struct lacpdu_header *lacp;
> +
> +	/* only enqueue LACPDUs */
> +	lacp = rte_pktmbuf_mtod(lacp_pkt, struct lacpdu_header *);
> +	if (lacp->lacpdu.subtype != SLOW_SUBTYPE_LACP)
> +		return -EINVAL;
> +
> +	MODE4_DEBUG("sending LACP frame\n");
> +
> +	return rte_ring_enqueue(port->tx_ring, lacp_pkt);
> +}
> +
> +static void
> +bond_mode_8023ad_ext_periodic_cb(void *arg)
> +{
> +	struct rte_eth_dev *bond_dev = arg;
> +	struct bond_dev_private *internals = bond_dev->data->dev_private;
> +	struct mode8023ad_private *mode4 = &internals->mode4;
> +	struct port *port;
> +	void *pkt = NULL;
> +	uint16_t i, slave_id;
> +
> +	for (i = 0; i < internals->active_slave_count; i++) {
> +		slave_id = internals->active_slaves[i];
> +		port = &mode_8023ad_ports[slave_id];
> +
> +		if (rte_ring_dequeue(port->rx_ring, &pkt) == 0) {
> +			struct rte_mbuf *lacp_pkt = pkt;
> +			struct lacpdu_header *lacp;
> +
> +			lacp = rte_pktmbuf_mtod(lacp_pkt,
> +						struct lacpdu_header *);
> +			RTE_VERIFY(lacp->lacpdu.subtype == SLOW_SUBTYPE_LACP);
> +
> +			/* This is LACP frame so pass it to rx callback.
> +			 * Callback is responsible for freeing mbuf.
> +			 */
> +			mode4->slowrx_cb(slave_id, lacp_pkt);
> +		}
> +	}
> +
> +	rte_eal_alarm_set(internals->mode4.update_timeout_us,
> +			bond_mode_8023ad_ext_periodic_cb, arg);
> +}
> diff --git a/lib/librte_pmd_bond/rte_eth_bond_8023ad.h b/lib/librte_pmd_bond/rte_eth_bond_8023ad.h
> index ebd0e93..8cfa3d3 100644
> --- a/lib/librte_pmd_bond/rte_eth_bond_8023ad.h
> +++ b/lib/librte_pmd_bond/rte_eth_bond_8023ad.h
> @@ -64,6 +64,8 @@ extern "C" {
>  #define MARKER_TLV_TYPE_INFO                0x01
>  #define MARKER_TLV_TYPE_RESP                0x02
>  
> +typedef void (*rte_eth_bond_8023ad_ext_slowrx_fn)(uint8_t slave_id, struct rte_mbuf *lacp_pkt);
> +
>  enum rte_bond_8023ad_selection {
>  	UNSELECTED,
>  	STANDBY,
> @@ -157,6 +159,7 @@ struct rte_eth_bond_8023ad_conf {
>  	uint32_t tx_period_ms;
>  	uint32_t rx_marker_period_ms;
>  	uint32_t update_timeout_ms;
> +	rte_eth_bond_8023ad_ext_slowrx_fn slowrx_cb;
>  };
>  

This is dangerous.  While it doesn't break ABI, it does create a situation in
which you will be prone to writing to invalid memory. By the examples that you
provide in the test application, you declare rte_eth_bond_8023ad_conf statically
within an application, so if someone is using the DPDK built as a shared
library, they may have less allocated space in the .data section then you think
with this new extended structure.  As a result in that situation you will read
from and write to unallocated memory when you access slowrx_cb.  I recommend
that you version the setup routine, so that the older version is modified to not
touch slowrx_cb.

Neil