From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id F3A6971 for ; Mon, 29 Sep 2014 14:05:55 +0200 (CEST) Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga101.fm.intel.com with ESMTP; 29 Sep 2014 05:12:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.97,862,1389772800"; d="scan'208";a="392997397" Received: from irsmsx103.ger.corp.intel.com ([163.33.3.157]) by FMSMGA003.fm.intel.com with ESMTP; 29 Sep 2014 05:06:14 -0700 Received: from irsmsx152.ger.corp.intel.com (163.33.192.66) by IRSMSX103.ger.corp.intel.com (163.33.3.157) with Microsoft SMTP Server (TLS) id 14.3.195.1; Mon, 29 Sep 2014 13:11:53 +0100 Received: from irsmsx107.ger.corp.intel.com ([169.254.10.68]) by IRSMSX152.ger.corp.intel.com ([169.254.6.118]) with mapi id 14.03.0195.001; Mon, 29 Sep 2014 13:11:53 +0100 From: "Mrzyglod, DanielX T" To: "dev@dpdk.org" Thread-Topic: [PATCH v2] ADD mode 5(tlb) to link bonding pmd Thread-Index: AQHP2aBtxd+go5QU9U6B941zZBqJapwYCQXA Date: Mon, 29 Sep 2014 12:11:52 +0000 Message-ID: <7ADD74816B4C8A45B56203CBA65FE5A64A3C85@IRSMSX107.ger.corp.intel.com> References: <1411746081-19550-1-git-send-email-danielx.t.mrzyglod@intel.com> In-Reply-To: <1411746081-19550-1-git-send-email-danielx.t.mrzyglod@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] [PATCH v2] ADD mode 5(tlb) to link bonding pmd X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Sep 2014 12:05:57 -0000 Add this Release note This patch set adds support of mode 5 to link bonding pmd This patchset depend on Declan Doherty patch set: http://dpdk.org/ml/archives/dev/2014-September/005641.html v2 change: Add Unit Tests Modification that updates obytes structure in virtualpmd driver. change internals->slaves[i].last_obytes to have proper values. Update codebase to Declan's patches. v1 change Add support for mode 5 (Transmit load balancing) into pmd driver > -----Original Message----- > From: Mrzyglod, DanielX T > Sent: Friday, September 26, 2014 5:41 PM > To: dev@dpdk.org > Cc: Mrzyglod, DanielX T > Subject: [PATCH v2] ADD mode 5(tlb) to link bonding pmd > = > = > Signed-off-by: Daniel Mrzyglod > --- > app/test/test_link_bonding.c | 501 ++++++++++++++++++++++= +++++- > app/test/virtual_pmd.c | 6 +- > app/test/virtual_pmd.h | 7 + > lib/librte_pmd_bond/rte_eth_bond.h | 23 ++ > lib/librte_pmd_bond/rte_eth_bond_args.c | 1 + > lib/librte_pmd_bond/rte_eth_bond_pmd.c | 161 ++++++++- > lib/librte_pmd_bond/rte_eth_bond_private.h | 3 +- > 7 files changed, 696 insertions(+), 6 deletions(-) > = > diff --git a/app/test/test_link_bonding.c b/app/test/test_link_bonding.c > index c4fcaf7..77f791f 100644 > --- a/app/test/test_link_bonding.c > +++ b/app/test/test_link_bonding.c > @@ -41,7 +41,7 @@ > #include > #include > #include > - > +#include > #include > #include > #include > @@ -3845,6 +3845,500 @@ testsuite_teardown(void) > return remove_slaves_and_stop_bonded_device(); > } > = > +#define NINETY_PERCENT_NUMERAL 90 > +#define ONE_HUNDRED_PERCENT_DENOMINATOR 100 > +#define ONE_HUNDRED_PERCENT_AND_TEN_NUMERAL 110 > +static int > +test_tlb_tx_burst(void) > +{ > + int i, burst_size, nb_tx; > + uint64_t nb_tx2 =3D 0; > + struct rte_mbuf *pkt_burst[MAX_PKT_BURST]; > + struct rte_eth_stats port_stats[32]; > + uint64_t sum_ports_opackets =3D 0, all_bond_opackets =3D 0, > all_bond_obytes =3D 0; > + uint16_t pktlen; > + > + TEST_ASSERT_SUCCESS(initialize_bonded_device_with_slaves > + > (BONDING_MODE_ADAPTIVE_TRANSMIT_LOAD_BALANCING, 1, 3, 1), > + "Failed to initialise bonded device"); > + > + burst_size =3D 20 * test_params->bonded_slave_count; > + > + TEST_ASSERT(burst_size < MAX_PKT_BURST, > + "Burst size specified is greater than supported.\n"); > + > + > + /* Generate 400000 test bursts in 2s of packets to transmit */ > + for (i =3D 0; i < 400000; i++) { > + /*test two types of mac src own(bonding) and others */ > + if (i % 2 =3D=3D 0) { > + initialize_eth_header(test_params->pkt_eth_hdr, > + (struct ether_addr *)src_mac, (struct > ether_addr *)dst_mac_0, 0, 0); > + } else { > + initialize_eth_header(test_params->pkt_eth_hdr, > + (struct ether_addr *)test_params- > >default_slave_mac, > + (struct ether_addr *)dst_mac_0, 0, 0); > + } > + pktlen =3D initialize_udp_header(test_params->pkt_udp_hdr, > src_port, > + dst_port_0, 16); > + pktlen =3D initialize_ipv4_header(test_params->pkt_ipv4_hdr, > src_addr, > + dst_addr_0, pktlen); > + generate_packet_burst(test_params->mbuf_pool, pkt_burst, > + test_params->pkt_eth_hdr, 0, test_params- > >pkt_ipv4_hdr, > + 1, test_params->pkt_udp_hdr, burst_size, 60, > 1); > + /* Send burst on bonded port */ > + nb_tx =3D rte_eth_tx_burst(test_params->bonded_port_id, 0, > pkt_burst, > + burst_size); > + nb_tx2 +=3D nb_tx; > + > + TEST_ASSERT_EQUAL(nb_tx, burst_size, > + "number of packet not equal burst size"); > + > + rte_delay_us(5); > + } > + > + > + /* Verify bonded port tx stats */ > + rte_eth_stats_get(test_params->bonded_port_id, &port_stats[0]); > + > + all_bond_opackets =3D port_stats[0].opackets; > + all_bond_obytes =3D port_stats[0].obytes; > + > + TEST_ASSERT_EQUAL(port_stats[0].opackets, (uint64_t)nb_tx2, > + "Bonded Port (%d) opackets value (%u) not as expected > (%d)\n", > + test_params->bonded_port_id, (unsigned > int)port_stats[0].opackets, > + burst_size); > + > + > + /* Verify slave ports tx stats */ > + for (i =3D 0; i < test_params->bonded_slave_count; i++) { > + rte_eth_stats_get(test_params->slave_port_ids[i], > &port_stats[i]); > + sum_ports_opackets +=3D port_stats[i].opackets; > + } > + > + TEST_ASSERT_EQUAL(sum_ports_opackets, > (uint64_t)all_bond_opackets, > + "Total packets sent by slaves is not equalto packets sent > by bond interface"); > + > + for (i =3D 0; i < test_params->bonded_slave_count; i++) { > + printf("port stats:%"PRIu64"\n", port_stats[i].opackets); > + /* distribution of packets on each slave within +/- 10% of the > expected value. */ > + TEST_ASSERT(port_stats[i].obytes >=3D > ((all_bond_obytes*NINETY_PERCENT_NUMERAL)/ > + (test_params- > >bonded_slave_count*ONE_HUNDRED_PERCENT_DENOMINATOR)) && > + port_stats[i].obytes <=3D > ((all_bond_obytes*ONE_HUNDRED_PERCENT_AND_TEN_NUMERAL) / > + (test_params- > >bonded_slave_count*ONE_HUNDRED_PERCENT_DENOMINATOR)), > + "Distribution is not even"); > + } > + /* Put all slaves down and try and transmit */ > + for (i =3D 0; i < test_params->bonded_slave_count; i++) { > + virtual_ethdev_simulate_link_status_interrupt( > + test_params->slave_port_ids[i], 0); > + } > + > + /* Send burst on bonded port */ > + nb_tx =3D rte_eth_tx_burst(test_params->bonded_port_id, 0, pkt_burst, > + burst_size); > + TEST_ASSERT_EQUAL(nb_tx, 0, " bad number of packet in burst"); > + > + /* Clean ugit checkout masterp and remove slaves from bonded device > */ > + return remove_slaves_and_stop_bonded_device(); > +} > + > +#define > TEST_ADAPTIVE_TRANSMIT_LOAD_BALANCING_RX_BURST_SLAVE_COUNT (4) > + > +static int > +test_tlb_rx_burst(void) > +{ > + struct rte_mbuf *gen_pkt_burst[MAX_PKT_BURST] =3D { NULL }; > + struct rte_mbuf *rx_pkt_burst[MAX_PKT_BURST] =3D { NULL }; > + > + struct rte_eth_stats port_stats; > + > + int primary_port; > + > + uint16_t i, j, nb_rx, burst_size =3D 17; > + > + /* Initialize bonded device with 4 slaves in transmit load balancing mo= de > */ > + TEST_ASSERT_SUCCESS(initialize_bonded_device_with_slaves( > + > BONDING_MODE_ADAPTIVE_TRANSMIT_LOAD_BALANCING, > + > TEST_ADAPTIVE_TRANSMIT_LOAD_BALANCING_RX_BURST_SLAVE_CO > UNT, 1, 1), > + "Failed to initialize bonded device"); > + > + > + primary_port =3D rte_eth_bond_primary_get(test_params- > >bonded_port_id); > + TEST_ASSERT(primary_port >=3D 0, > + "failed to get primary slave for bonded port (%d)", > + test_params->bonded_port_id); > + > + for (i =3D 0; i < test_params->bonded_slave_count; i++) { > + /* Generate test bursts of packets to transmit */ > + TEST_ASSERT_EQUAL(generate_test_burst( > + &gen_pkt_burst[0], burst_size, 0, 1, 0, 0, 0), > burst_size, > + "burst generation failed"); > + > + /* Add rx data to slave */ > + virtual_ethdev_add_mbufs_to_rx_queue(test_params- > >slave_port_ids[i], > + &gen_pkt_burst[0], burst_size); > + > + /* Call rx burst on bonded device */ > + nb_rx =3D rte_eth_rx_burst(test_params->bonded_port_id, 0, > + &rx_pkt_burst[0], MAX_PKT_BURST); > + > + TEST_ASSERT_EQUAL(nb_rx,burst_size,"rte_eth_rx_burst > failed\n"); > + > + if (test_params->slave_port_ids[i] =3D=3D primary_port) { > + /* Verify bonded device rx count */ > + rte_eth_stats_get(test_params->bonded_port_id, > &port_stats); > + TEST_ASSERT_EQUAL(port_stats.ipackets, > (uint64_t)burst_size, > + "Bonded Port (%d) ipackets value (%u) > not as expected (%d)\n", > + test_params->bonded_port_id, > + (unsigned int)port_stats.ipackets, > burst_size); > + > + /* Verify bonded slave devices rx count */ > + for (j =3D 0; j < test_params->bonded_slave_count; j++) { > + rte_eth_stats_get(test_params- > >slave_port_ids[j], &port_stats); > + if (i =3D=3D j) { > + > TEST_ASSERT_EQUAL(port_stats.ipackets, (uint64_t)burst_size, > + "Slave Port (%d) > ipackets value (%u) not as expected (%d)\n", > + test_params- > >slave_port_ids[i], > + (unsigned > int)port_stats.ipackets, burst_size); > + } else { > + > TEST_ASSERT_EQUAL(port_stats.ipackets, (uint64_t)0, > + "Slave Port (%d) > ipackets value (%u) not as expected (%d)\n", > + test_params- > >slave_port_ids[i], > + (unsigned > int)port_stats.ipackets, 0); > + } > + } > + } else { > + for (j =3D 0; j < test_params->bonded_slave_count; j++) { > + rte_eth_stats_get(test_params- > >slave_port_ids[j], &port_stats); > + TEST_ASSERT_EQUAL(port_stats.ipackets, > (uint64_t)0, > + "Slave Port (%d) ipackets value > (%u) not as expected (%d)\n", > + test_params- > >slave_port_ids[i], > + (unsigned > int)port_stats.ipackets, 0); > + } > + } > + > + /* free mbufs */ > + for (i =3D 0; i < burst_size; i++) > + rte_pktmbuf_free(rx_pkt_burst[i]); > + > + /* reset bonded device stats */ > + rte_eth_stats_reset(test_params->bonded_port_id); > + } > + > + /* Clean up and remove slaves from bonded device */ > + return remove_slaves_and_stop_bonded_device(); > +} > + > +static int > +test_tlb_verify_promiscuous_enable_disable(void) > +{ > + int i, primary_port, promiscuous_en; > + > + /* Initialize bonded device with 4 slaves in transmit load balancing mo= de > */ > + TEST_ASSERT_SUCCESS( initialize_bonded_device_with_slaves( > + > BONDING_MODE_ADAPTIVE_TRANSMIT_LOAD_BALANCING, 0, 4, 1), > + "Failed to initialize bonded device"); > + > + primary_port =3D rte_eth_bond_primary_get(test_params- > >bonded_port_id); > + TEST_ASSERT(primary_port >=3D 0, > + "failed to get primary slave for bonded port (%d)", > + test_params->bonded_port_id); > + > + rte_eth_promiscuous_enable(test_params->bonded_port_id); > + > + promiscuous_en =3D rte_eth_promiscuous_get(test_params- > >bonded_port_id); > + TEST_ASSERT_EQUAL(promiscuous_en, (int)1, > + "Port (%d) promiscuous mode not enabled\n", > + test_params->bonded_port_id); > + for (i =3D 0; i < test_params->bonded_slave_count; i++) { > + promiscuous_en =3D rte_eth_promiscuous_get( > + test_params->slave_port_ids[i]); > + if (primary_port =3D=3D test_params->slave_port_ids[i]) { > + TEST_ASSERT_EQUAL(promiscuous_en, (int)1, > + "Port (%d) promiscuous mode not > enabled\n", > + test_params->bonded_port_id); > + } else { > + TEST_ASSERT_EQUAL(promiscuous_en, (int)0, > + "Port (%d) promiscuous mode > enabled\n", > + test_params->bonded_port_id); > + } > + > + } > + > + rte_eth_promiscuous_disable(test_params->bonded_port_id); > + > + promiscuous_en =3D rte_eth_promiscuous_get(test_params- > >bonded_port_id); > + TEST_ASSERT_EQUAL(promiscuous_en, (int)0, > + "Port (%d) promiscuous mode not disabled\n", > + test_params->bonded_port_id); > + > + for (i =3D 0; i < test_params->bonded_slave_count; i++) { > + promiscuous_en =3D rte_eth_promiscuous_get( > + test_params->slave_port_ids[i]); > + TEST_ASSERT_EQUAL(promiscuous_en, (int)0, > + "slave port (%d) promiscuous mode not > disabled\n", > + test_params->slave_port_ids[i]); > + } > + > + /* Clean up and remove slaves from bonded device */ > + return remove_slaves_and_stop_bonded_device(); > +} > + > +static int > +test_tlb_verify_mac_assignment(void) > +{ > + struct ether_addr read_mac_addr, expected_mac_addr_0, > expected_mac_addr_1; > + > + rte_eth_macaddr_get(test_params->slave_port_ids[0], > &expected_mac_addr_0); > + rte_eth_macaddr_get(test_params->slave_port_ids[1], > &expected_mac_addr_1); > + > + /* Initialize bonded device with 2 slaves in active backup mode */ > + TEST_ASSERT_SUCCESS(initialize_bonded_device_with_slaves( > + > BONDING_MODE_ADAPTIVE_TRANSMIT_LOAD_BALANCING, 0, 2, 1), > + "Failed to initialize bonded device"); > + > + /* Verify that bonded MACs is that of first slave and that the other sl= ave > + * MAC hasn't been changed */ > + rte_eth_macaddr_get(test_params->bonded_port_id, > &read_mac_addr); > + TEST_ASSERT_SUCCESS(memcmp(&expected_mac_addr_0, > &read_mac_addr, > + sizeof(read_mac_addr)), > + "bonded port (%d) mac address not set to that of > primary port", > + test_params->bonded_port_id); > + > + rte_eth_macaddr_get(test_params->slave_port_ids[0], > &read_mac_addr); > + TEST_ASSERT_SUCCESS(memcmp(&expected_mac_addr_0, > &read_mac_addr, > + sizeof(read_mac_addr)), > + "slave port (%d) mac address not set to that of primary > port", > + test_params->slave_port_ids[0]); > + > + rte_eth_macaddr_get(test_params->slave_port_ids[1], > &read_mac_addr); > + TEST_ASSERT_SUCCESS(memcmp(&expected_mac_addr_1, > &read_mac_addr, > + sizeof(read_mac_addr)), > + "slave port (%d) mac address not as expected", > + test_params->slave_port_ids[1]); > + > + /* change primary and verify that MAC addresses haven't changed */ > + TEST_ASSERT_EQUAL(rte_eth_bond_primary_set(test_params- > >bonded_port_id, > + test_params->slave_port_ids[1]), 0, > + "Failed to set bonded port (%d) primary port to (%d)", > + test_params->bonded_port_id, test_params- > >slave_port_ids[1]); > + > + rte_eth_macaddr_get(test_params->bonded_port_id, > &read_mac_addr); > + TEST_ASSERT_SUCCESS(memcmp(&expected_mac_addr_0, > &read_mac_addr, > + sizeof(read_mac_addr)), > + "bonded port (%d) mac address not set to that of > primary port", > + test_params->bonded_port_id); > + > + rte_eth_macaddr_get(test_params->slave_port_ids[0], > &read_mac_addr); > + TEST_ASSERT_SUCCESS(memcmp(&expected_mac_addr_0, > &read_mac_addr, > + sizeof(read_mac_addr)), > + "slave port (%d) mac address not set to that of primary > port", > + test_params->slave_port_ids[0]); > + > + rte_eth_macaddr_get(test_params->slave_port_ids[1], > &read_mac_addr); > + TEST_ASSERT_SUCCESS(memcmp(&expected_mac_addr_1, > &read_mac_addr, > + sizeof(read_mac_addr)), > + "slave port (%d) mac address not as expected", > + test_params->slave_port_ids[1]); > + > + /* stop / start bonded device and verify that primary MAC address is > + * propagated to bonded device and slaves */ > + > + rte_eth_dev_stop(test_params->bonded_port_id); > + > + TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params- > >bonded_port_id), > + "Failed to start device"); > + > + rte_eth_macaddr_get(test_params->bonded_port_id, > &read_mac_addr); > + TEST_ASSERT_SUCCESS(memcmp(&expected_mac_addr_1, > &read_mac_addr, > + sizeof(read_mac_addr)), > + "bonded port (%d) mac address not set to that of > primary port", > + test_params->bonded_port_id); > + > + rte_eth_macaddr_get(test_params->slave_port_ids[0], > &read_mac_addr); > + TEST_ASSERT_SUCCESS(memcmp(&expected_mac_addr_0, > &read_mac_addr, > + sizeof(read_mac_addr)), > + "slave port (%d) mac address not as expected", > + test_params->slave_port_ids[0]); > + > + rte_eth_macaddr_get(test_params->slave_port_ids[1], > &read_mac_addr); > + TEST_ASSERT_SUCCESS(memcmp(&expected_mac_addr_1, > &read_mac_addr, > + sizeof(read_mac_addr)), > + "slave port (%d) mac address not set to that of primary > port", > + test_params->slave_port_ids[1]); > + > + > + /* Set explicit MAC address */ > + TEST_ASSERT_SUCCESS(rte_eth_bond_mac_address_set( > + test_params->bonded_port_id, (struct ether_addr > *)bonded_mac), > + "failed to set MAC addres"); > + > + rte_eth_macaddr_get(test_params->bonded_port_id, > &read_mac_addr); > + TEST_ASSERT_SUCCESS(memcmp(&bonded_mac, &read_mac_addr, > + sizeof(read_mac_addr)), > + "bonded port (%d) mac address not set to that of > bonded port", > + test_params->bonded_port_id); > + > + rte_eth_macaddr_get(test_params->slave_port_ids[0], > &read_mac_addr); > + TEST_ASSERT_SUCCESS(memcmp(&expected_mac_addr_0, > &read_mac_addr, > + sizeof(read_mac_addr)), > + "slave port (%d) mac address not as expected", > + test_params->slave_port_ids[0]); > + > + rte_eth_macaddr_get(test_params->slave_port_ids[1], > &read_mac_addr); > + TEST_ASSERT_SUCCESS(memcmp(&bonded_mac, &read_mac_addr, > + sizeof(read_mac_addr)), > + "slave port (%d) mac address not set to that of bonded > port", > + test_params->slave_port_ids[1]); > + > + /* Clean up and remove slaves from bonded device */ > + return remove_slaves_and_stop_bonded_device(); > +} > + > +static int > +test_tlb_verify_slave_link_status_change_failover(void) > +{ > + struct rte_mbuf > *pkt_burst[TEST_ADAPTIVE_TRANSMIT_LOAD_BALANCING_RX_BURST_SLAVE_ > COUNT][MAX_PKT_BURST]; > + struct rte_mbuf *rx_pkt_burst[MAX_PKT_BURST] =3D { NULL }; > + struct rte_eth_stats port_stats; > + > + uint8_t slaves[RTE_MAX_ETHPORTS]; > + > + int i, j, burst_size, slave_count, primary_port; > + > + burst_size =3D 21; > + > + memset(pkt_burst, 0, sizeof(pkt_burst)); > + > + > + > + /* Initialize bonded device with 4 slaves in round robin mode */ > + TEST_ASSERT_SUCCESS(initialize_bonded_device_with_slaves( > + > BONDING_MODE_ADAPTIVE_TRANSMIT_LOAD_BALANCING, 0, > + > TEST_ADAPTIVE_TRANSMIT_LOAD_BALANCING_RX_BURST_SLAVE_CO > UNT, 1), > + "Failed to initialize bonded device with slaves"); > + > + /* Verify Current Slaves Count /Active Slave Count is */ > + slave_count =3D rte_eth_bond_slaves_get(test_params->bonded_port_id, > slaves, > + RTE_MAX_ETHPORTS); > + TEST_ASSERT_EQUAL(slave_count, 4, > + "Number of slaves (%d) is not as expected (%d).\n", > + slave_count, 4); > + > + slave_count =3D rte_eth_bond_active_slaves_get(test_params- > >bonded_port_id, > + slaves, RTE_MAX_ETHPORTS); > + TEST_ASSERT_EQUAL(slave_count, (int)4, > + "Number of slaves (%d) is not as expected (%d).\n", > + slave_count, 4); > + > + primary_port =3D rte_eth_bond_primary_get(test_params- > >bonded_port_id); > + TEST_ASSERT_EQUAL(primary_port, test_params->slave_port_ids[0], > + "Primary port not as expected"); > + > + /* Bring 2 slaves down and verify active slave count */ > + virtual_ethdev_simulate_link_status_interrupt( > + test_params->slave_port_ids[1], 0); > + virtual_ethdev_simulate_link_status_interrupt( > + test_params->slave_port_ids[3], 0); > + > + TEST_ASSERT_EQUAL(rte_eth_bond_active_slaves_get( > + test_params->bonded_port_id, slaves, > RTE_MAX_ETHPORTS), 2, > + "Number of active slaves (%d) is not as expected (%d).", > + slave_count, 2); > + > + virtual_ethdev_simulate_link_status_interrupt( > + test_params->slave_port_ids[1], 1); > + virtual_ethdev_simulate_link_status_interrupt( > + test_params->slave_port_ids[3], 1); > + > + > + /* Bring primary port down, verify that active slave count is 3 and > primary > + * has changed */ > + virtual_ethdev_simulate_link_status_interrupt( > + test_params->slave_port_ids[0], 0); > + > + TEST_ASSERT_EQUAL(rte_eth_bond_active_slaves_get( > + test_params->bonded_port_id, slaves, > RTE_MAX_ETHPORTS), 3, > + "Number of active slaves (%d) is not as expected (%d).", > + slave_count, 3); > + > + primary_port =3D rte_eth_bond_primary_get(test_params- > >bonded_port_id); > + TEST_ASSERT_EQUAL(primary_port, test_params->slave_port_ids[2], > + "Primary port not as expected"); > + rte_delay_us(500000); > + /* Verify that pkts are sent on new primary slave */ > + for (i =3D 0; i < 4; i++) { > + TEST_ASSERT_EQUAL(generate_test_burst( > + &pkt_burst[0][0], burst_size, 0, 1, 0, 0, 0), > burst_size, > + "generate_test_burst failed\n"); > + TEST_ASSERT_EQUAL(rte_eth_tx_burst( > + test_params->bonded_port_id, 0, > &pkt_burst[0][0], burst_size), burst_size, > + "rte_eth_tx_burst failed\n"); > + rte_delay_us(11000); > + } > + > + rte_eth_stats_get(test_params->slave_port_ids[2], &port_stats); > + TEST_ASSERT_NOT_EQUAL(port_stats.opackets, (int8_t)0, > + "(%d) port_stats.opackets not as expected\n", > + test_params->slave_port_ids[2]); > + > + rte_eth_stats_get(test_params->slave_port_ids[0], &port_stats); > + TEST_ASSERT_EQUAL(port_stats.opackets, (int8_t)0, > + "(%d) port_stats.opackets not as expected\n", > + test_params->slave_port_ids[0]); > + > + rte_eth_stats_get(test_params->slave_port_ids[1], &port_stats); > + TEST_ASSERT_NOT_EQUAL(port_stats.opackets, (int8_t)0, > + "(%d) port_stats.opackets not as expected\n", > + test_params->slave_port_ids[1]); > + > + rte_eth_stats_get(test_params->slave_port_ids[3], &port_stats); > + TEST_ASSERT_NOT_EQUAL(port_stats.opackets, (int8_t)0, > + "(%d) port_stats.opackets not as expected\n", > + test_params->slave_port_ids[3]); > + > + > + /* Generate packet burst for testing */ > + > + for (i =3D 0; i < > TEST_ADAPTIVE_TRANSMIT_LOAD_BALANCING_RX_BURST_SLAVE_COUNT; > i++) { > + if (generate_test_burst(&pkt_burst[i][0], burst_size, 0, 1, 0, 0, 0) > !=3D > + burst_size) > + return -1; > + > + virtual_ethdev_add_mbufs_to_rx_queue( > + test_params->slave_port_ids[i], > &pkt_burst[i][0], burst_size); > + } > + > + if (rte_eth_rx_burst(test_params->bonded_port_id, 0, rx_pkt_burst, > + MAX_PKT_BURST) !=3D burst_size) { > + printf("rte_eth_rx_burst\n"); > + return -1; > + > + } > + > + /* Verify bonded device rx count */ > + rte_eth_stats_get(test_params->bonded_port_id, &port_stats); > + TEST_ASSERT_EQUAL(port_stats.ipackets, (uint64_t)burst_size, > + "(%d) port_stats.ipackets not as expected\n", > + test_params->bonded_port_id); > + > + /* free mbufs */ > + > + for (i =3D 0; i < > TEST_ADAPTIVE_TRANSMIT_LOAD_BALANCING_RX_BURST_SLAVE_COUNT; > i++) { > + for (j =3D 0; j < MAX_PKT_BURST; j++) { > + if (pkt_burst[i][j] !=3D NULL) { > + rte_pktmbuf_free(pkt_burst[i][j]); > + pkt_burst[i][j] =3D NULL; > + } > + } > + } > + > + > + /* Clean up and remove slaves from bonded device */ > + return remove_slaves_and_stop_bonded_device(); > +} > + > = > static struct unit_test_suite link_bonding_test_suite =3D { > .suite_name =3D "Link Bonding Unit Test Suite", > @@ -3898,6 +4392,11 @@ static struct unit_test_suite link_bonding_test_su= ite > =3D { > TEST_CASE(test_balance_verify_promiscuous_enable_disable), > TEST_CASE(test_balance_verify_mac_assignment), > = > TEST_CASE(test_balance_verify_slave_link_status_change_behaviour), > + TEST_CASE(test_tlb_tx_burst), > + TEST_CASE(test_tlb_rx_burst), > + TEST_CASE(test_tlb_verify_mac_assignment), > + TEST_CASE(test_tlb_verify_promiscuous_enable_disable), > + TEST_CASE(test_tlb_verify_slave_link_status_change_failover), > TEST_CASE(test_broadcast_tx_burst), > TEST_CASE(test_broadcast_tx_burst_slave_tx_fail), > TEST_CASE(test_broadcast_rx_burst), > diff --git a/app/test/virtual_pmd.c b/app/test/virtual_pmd.c > index fffaa35..331bec3 100644 > --- a/app/test/virtual_pmd.c > +++ b/app/test/virtual_pmd.c > @@ -359,16 +359,18 @@ virtual_ethdev_tx_burst_success(void *queue, struct > rte_mbuf **bufs, > = > struct rte_eth_dev *vrtl_eth_dev; > struct virtual_ethdev_private *dev_private; > - > + uint64_t obytes =3D 0; > int i; > = > + for (i =3D 0; i < nb_pkts; i++) > + obytes +=3D rte_pktmbuf_pkt_len(bufs[i]); > vrtl_eth_dev =3D &rte_eth_devices[tx_q->port_id]; > dev_private =3D vrtl_eth_dev->data->dev_private; > = > if (vrtl_eth_dev->data->dev_link.link_status) { > /* increment opacket count */ > dev_private->eth_stats.opackets +=3D nb_pkts; > - > + dev_private->eth_stats.obytes +=3D obytes; > /* free packets in burst */ > for (i =3D 0; i < nb_pkts; i++) { > rte_pktmbuf_free(bufs[i]); > diff --git a/app/test/virtual_pmd.h b/app/test/virtual_pmd.h > index 2462853..4118e3e 100644 > --- a/app/test/virtual_pmd.h > +++ b/app/test/virtual_pmd.h > @@ -94,6 +94,13 @@ void > virtual_ethdev_tx_burst_fn_set_tx_pkt_fail_count(uint8_t port_id, > uint8_t packet_fail_count); > = > +/* if a value greater than zero is set for packet_fail_count then virtual > + * device tx burst function will fail that many packet from burst or all > + * packets if packet_fail_count is greater than the number of packets in= the > + * burst */ > +void virtual_ethdev_tx_burst_fn_set_tx_pkt_fail_count(uint8_t port_id, > + uint8_t packet_fail_count); > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/librte_pmd_bond/rte_eth_bond.h > b/lib/librte_pmd_bond/rte_eth_bond.h > index 6811c7b..682e5c7 100644 > --- a/lib/librte_pmd_bond/rte_eth_bond.h > +++ b/lib/librte_pmd_bond/rte_eth_bond.h > @@ -75,6 +75,29 @@ extern "C" { > /**< Broadcast (Mode 3). > * In this mode all transmitted packets will be transmitted on all avail= able > * active slaves of the bonded. */ > +#define BONDING_MODE_ADAPTIVE_TRANSMIT_LOAD_BALANCING (5) > +/**< Broadcast (Mode 5) > + * Adaptive transmit load balancing: channel bonding that > + * does not require any special switch support. The > + * outgoing traffic is distributed according to the > + * current load (computed relative to the speed) on each > + * slave. Incoming traffic is received by the current > + * slave. If the receiving slave fails, another slave > + * takes over the MAC address of the failed receiving > + * slave.*/ > +#define BONDING_MODE_ADAPTIVE_LOAD_BALANCING > (6) > +/** > + * Adaptive load balancing: includes balance-tlb plus > + * receive load balancing (rlb) for IPV4 traffic, and > + * does not require any special switch support. The > + * receive load balancing is achieved by ARP negotiation. > + * The bonding driver intercepts the ARP Replies sent by > + * the local system on their way out and overwrites the > + * source hardware address with the unique hardware > + * address of one of the slaves in the bond such that > + * different peers use different hardware addresses for > + * the server. */ > + > = > /* Balance Mode Transmit Policies */ > #define BALANCE_XMIT_POLICY_LAYER2 (0) > diff --git a/lib/librte_pmd_bond/rte_eth_bond_args.c > b/lib/librte_pmd_bond/rte_eth_bond_args.c > index bbbc69b..7464af5 100644 > --- a/lib/librte_pmd_bond/rte_eth_bond_args.c > +++ b/lib/librte_pmd_bond/rte_eth_bond_args.c > @@ -171,6 +171,7 @@ bond_ethdev_parse_slave_mode_kvarg(const char *key > __rte_unused, > case BONDING_MODE_ACTIVE_BACKUP: > case BONDING_MODE_BALANCE: > case BONDING_MODE_BROADCAST: > + case BONDING_MODE_ADAPTIVE_TRANSMIT_LOAD_BALANCING: > return 0; > default: > RTE_BOND_LOG(ERR, "Invalid slave mode value (%s) specified", > value); > diff --git a/lib/librte_pmd_bond/rte_eth_bond_pmd.c > b/lib/librte_pmd_bond/rte_eth_bond_pmd.c > index 6d0fb1b..db6a934 100644 > --- a/lib/librte_pmd_bond/rte_eth_bond_pmd.c > +++ b/lib/librte_pmd_bond/rte_eth_bond_pmd.c > @@ -30,7 +30,7 @@ > * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF > THE USE > * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH > DAMAGE. > */ > - > +#include > #include > #include > #include > @@ -41,10 +41,13 @@ > #include > #include > #include > +#include > = > #include "rte_eth_bond.h" > #include "rte_eth_bond_private.h" > = > +#define REORDER_PERIOD_MS 10 > + > static uint16_t > bond_ethdev_rx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pk= ts) > { > @@ -287,6 +290,141 @@ xmit_slave_hash(const struct rte_mbuf *buf, uint8_t > slave_count, uint8_t policy) > return hash % slave_count; > } > = > +struct bwg_slave { > + uint64_t bwg_left_int; > + uint64_t bwg_left_remainder; > + uint8_t slave; > +}; > + > +static int > +bandwidth_cmp(const void *a, const void *b) > +{ > + const struct bwg_slave *bwg_a =3D a; > + const struct bwg_slave *bwg_b =3D b; > + int64_t diff =3D (int64_t)bwg_b->bwg_left_int - (int64_t)bwg_a- > >bwg_left_int; > + int64_t diff2 =3D (int64_t)bwg_b->bwg_left_remainder - > + (int64_t)bwg_a->bwg_left_remainder; > + if (diff > 0) > + return 1; > + else if (diff < 0) > + return -1; > + else if (diff2 > 0) > + return 1; > + else if (diff2 < 0) > + return -1; > + else > + return 0; > +} > + > +static void > +bandwidth_left(int port_id, uint64_t load, uint8_t update_idx, struct bw= g_slave > *bwg_slave) > +{ > + struct rte_eth_link link_status; > + > + rte_eth_link_get(port_id, &link_status); > + uint64_t link_bwg =3D link_status.link_speed * 1000000ULL / 8; > + if (link_bwg =3D=3D 0) > + return; > + link_bwg =3D (link_bwg * (update_idx+1) * REORDER_PERIOD_MS); > + bwg_slave->bwg_left_int =3D (link_bwg - 1000*load) / link_bwg; > + bwg_slave->bwg_left_remainder =3D (link_bwg - 1000*load) % link_bwg; > +} > + > +static void > +bond_ethdev_update_tlb_slave_cb(void *arg) > +{ > + struct bond_dev_private *internals =3D arg; > + struct rte_eth_stats slave_stats; > + struct bwg_slave bwg_array[RTE_MAX_ETHPORTS]; > + uint8_t slave_count; > + uint64_t tx_bytes; > + uint8_t update_stats =3D 0; > + int8_t i; > + > + internals->slave_update_idx++; > + > + > + if (internals->slave_update_idx >=3D REORDER_PERIOD_MS) > + update_stats =3D 1; > + > + for (i =3D 0; i < internals->active_slave_count; i++) { > + rte_eth_stats_get(internals->active_slaves[i], &slave_stats); > + tx_bytes =3D slave_stats.obytes - > + internals->slaves[i].last_obytes; > + bandwidth_left(internals->active_slaves[i], tx_bytes, > + internals->slave_update_idx, &bwg_array[i]); > + bwg_array[i].slave =3D internals->active_slaves[i]; > + > + if (update_stats) > + internals->slaves[i].last_obytes =3D slave_stats.obytes; > + } > + > + if (update_stats =3D=3D 1) > + internals->slave_update_idx =3D 0; > + > + slave_count =3D i; > + qsort(bwg_array, slave_count, sizeof(bwg_array[0]), bandwidth_cmp); > + for (i =3D 0; i < slave_count; i++) > + internals->active_slaves[i] =3D bwg_array[i].slave; > + > + rte_eal_alarm_set(REORDER_PERIOD_MS * 1000, > bond_ethdev_update_tlb_slave_cb, > + (struct bond_dev_private *)internals); > +} > + > +static uint16_t > +bond_ethdev_tx_burst_tlb(void *queue, struct rte_mbuf **bufs, uint16_t > nb_pkts) > +{ > + struct bond_tx_queue *bd_tx_q =3D (struct bond_tx_queue *)queue; > + struct bond_dev_private *internals =3D bd_tx_q->dev_private; > + > + struct rte_eth_dev *primary_port =3D &rte_eth_devices[internals- > >primary_port]; > + uint16_t num_tx_total =3D 0; > + uint8_t i, j; > + > + uint8_t num_of_slaves =3D internals->active_slave_count; > + uint8_t slaves[RTE_MAX_ETHPORTS]; > + > + struct ether_hdr *ether_hdr; > + struct ether_addr primary_slave_addr; > + struct ether_addr active_slave_addr; > + > + if (num_of_slaves < 1) > + return num_tx_total; > + > + memcpy(slaves, internals->active_slaves, > + sizeof(internals->active_slaves[0]) * > num_of_slaves); > + > + > + ether_addr_copy(primary_port->data->mac_addrs, > &primary_slave_addr); > + > + if (nb_pkts > 3) { > + for (i =3D 0; i < 3; i++) > + rte_prefetch0(rte_pktmbuf_mtod(bufs[i], void*)); > + } > + > + for (i =3D 0; i < num_of_slaves; i++) { > + ether_addr_copy(&internals- > >slaves[slaves[i]].persisted_mac_addr, > + &active_slave_addr); > + > + for (j =3D num_tx_total; j < nb_pkts; j++) { > + if (j + 3 < nb_pkts) > + rte_prefetch0(rte_pktmbuf_mtod(bufs[j+3], > void*)); > + > + ether_hdr =3D rte_pktmbuf_mtod(bufs[j], struct ether_hdr > *); > + if (is_same_ether_addr(ðer_hdr->s_addr, > &primary_slave_addr)) > + ether_addr_copy(&active_slave_addr, > ðer_hdr->s_addr); > + } > + > + num_tx_total +=3D rte_eth_tx_burst(slaves[i], bd_tx_q->queue_id, > + bufs + num_tx_total, nb_pkts - num_tx_total); > + > + if (num_tx_total =3D=3D nb_pkts) > + break; > + } > + > + return num_tx_total; > +} > + > static uint16_t > bond_ethdev_tx_burst_balance(void *queue, struct rte_mbuf **bufs, > uint16_t nb_pkts) > @@ -495,6 +633,7 @@ mac_address_slaves_update(struct rte_eth_dev > *bonded_eth_dev) > } > break; > case BONDING_MODE_ACTIVE_BACKUP: > + case BONDING_MODE_ADAPTIVE_TRANSMIT_LOAD_BALANCING: > default: > for (i =3D 0; i < internals->slave_count; i++) { > if (internals->slaves[i].port_id =3D=3D > @@ -544,6 +683,10 @@ bond_ethdev_mode_set(struct rte_eth_dev *eth_dev, > int mode) > eth_dev->tx_pkt_burst =3D bond_ethdev_tx_burst_broadcast; > eth_dev->rx_pkt_burst =3D bond_ethdev_rx_burst; > break; > + case BONDING_MODE_ADAPTIVE_TRANSMIT_LOAD_BALANCING: > + eth_dev->tx_pkt_burst =3D bond_ethdev_tx_burst_tlb; > + eth_dev->rx_pkt_burst =3D bond_ethdev_rx_burst_active_backup; > + break; > default: > return -1; > } > @@ -757,6 +900,9 @@ bond_ethdev_start(struct rte_eth_dev *eth_dev) > = > bond_ethdev_slave_link_status_change_monitor, > (void *)eth_dev); > = > + if (internals->mode =3D=3D > BONDING_MODE_ADAPTIVE_TRANSMIT_LOAD_BALANCING) { > + bond_ethdev_update_tlb_slave_cb(internals); > + } > return 0; > } > = > @@ -765,6 +911,9 @@ bond_ethdev_stop(struct rte_eth_dev *eth_dev) > { > struct bond_dev_private *internals =3D eth_dev->data->dev_private; > = > + if (internals->mode =3D=3D > BONDING_MODE_ADAPTIVE_TRANSMIT_LOAD_BALANCING) { > + rte_eal_alarm_cancel(bond_ethdev_update_tlb_slave_cb, > internals); > + } > internals->active_slave_count =3D 0; > internals->link_status_polling_enabled =3D 0; > = > @@ -775,6 +924,12 @@ bond_ethdev_stop(struct rte_eth_dev *eth_dev) > static void > bond_ethdev_close(struct rte_eth_dev *dev __rte_unused) > { > + struct bond_dev_private *internals =3D dev->data->dev_private; > + > + if (internals->mode =3D=3D > BONDING_MODE_ADAPTIVE_TRANSMIT_LOAD_BALANCING) { > + rte_eal_alarm_cancel(bond_ethdev_update_tlb_slave_cb, > internals); > + } > + internals->active_slave_count =3D 0; > } > = > static int > @@ -884,7 +1039,7 @@ bond_ethdev_slave_link_status_change_monitor(void > *cb_arg) > = > /* If device is currently being configured then don't check slaves link > * status, wait until next period */ > - if (rte_spinlock_trylock(&internals->lock)){ > + if (rte_spinlock_trylock(&internals->lock)) { > for (i =3D 0; i < internals->slave_count; i++) { > if (internals->slaves[i].link_status_polling_enabled) { > slave_ethdev =3D &rte_eth_devices[internals- > >slaves[i].port_id]; > @@ -1004,6 +1159,7 @@ bond_ethdev_promiscuous_enable(struct > rte_eth_dev *eth_dev) > break; > /* Promiscuous mode is propagated only to primary slave */ > case BONDING_MODE_ACTIVE_BACKUP: > + case BONDING_MODE_ADAPTIVE_TRANSMIT_LOAD_BALANCING: > default: > rte_eth_promiscuous_enable(internals->current_primary_port); > = > @@ -1028,6 +1184,7 @@ bond_ethdev_promiscuous_disable(struct > rte_eth_dev *dev) > break; > /* Promiscuous mode is propagated only to primary slave */ > case BONDING_MODE_ACTIVE_BACKUP: > + case BONDING_MODE_ADAPTIVE_TRANSMIT_LOAD_BALANCING: > default: > rte_eth_promiscuous_disable(internals->current_primary_port); > } > diff --git a/lib/librte_pmd_bond/rte_eth_bond_private.h > b/lib/librte_pmd_bond/rte_eth_bond_private.h > index 6db5144..f7186a8 100644 > --- a/lib/librte_pmd_bond/rte_eth_bond_private.h > +++ b/lib/librte_pmd_bond/rte_eth_bond_private.h > @@ -89,7 +89,6 @@ struct bond_tx_queue { > /**< Copy of TX configuration structure for queue */ > }; > = > - > /** Bonded slave devices structure */ > struct bond_ethdev_slave_ports { > uint8_t slaves[RTE_MAX_ETHPORTS]; /**< Slave port id array */ > @@ -102,6 +101,7 @@ struct bond_slave_details { > uint8_t link_status_polling_enabled; > uint8_t link_status_wait_to_complete; > uint8_t last_link_status; > + uint64_t last_obytes; > = > /**< Port Id of slave eth_dev */ > struct ether_addr persisted_mac_addr; > @@ -143,6 +143,7 @@ struct bond_dev_private { > uint8_t slave_count; /**< Number of bonded slaves > */ > struct bond_slave_details slaves[RTE_MAX_ETHPORTS]; > /**< Arary of bonded slaves details */ > + uint8_t slave_update_idx; > }; > = > extern struct eth_dev_ops default_dev_ops; > -- > 1.7.9.5 -------------------------------------------------------------- Intel Shannon Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 Business address: Dromore House, East Park, Shannon, Co. Clare This e-mail and any attachments may contain confidential material for the s= ole use of the intended recipient(s). Any review or distribution by others = is strictly prohibited. If you are not the intended recipient, please conta= ct the sender and delete all copies.