From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 0FD049A92 for ; Fri, 5 Feb 2016 06:09:23 +0100 (CET) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP; 04 Feb 2016 21:09:24 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.22,398,1449561600"; d="scan'208";a="896811968" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by fmsmga001.fm.intel.com with ESMTP; 04 Feb 2016 21:09:22 -0800 Received: from fmsmsx156.amr.corp.intel.com (10.18.116.74) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.248.2; Thu, 4 Feb 2016 21:09:22 -0800 Received: from shsmsx151.ccr.corp.intel.com (10.239.6.50) by fmsmsx156.amr.corp.intel.com (10.18.116.74) with Microsoft SMTP Server (TLS) id 14.3.248.2; Thu, 4 Feb 2016 21:09:21 -0800 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.249]) by SHSMSX151.ccr.corp.intel.com ([169.254.3.196]) with mapi id 14.03.0248.002; Fri, 5 Feb 2016 13:09:20 +0800 From: "Xu, HuilongX" To: "Lu, Wenzhuo" , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH] ixgbe: fix x550 VF tx issue Thread-Index: AQHRWmmNiLz+EyRmuUyjTMFeRzxY/p8czOQg Date: Fri, 5 Feb 2016 05:09:19 +0000 Message-ID: References: <1454053693-28673-1-git-send-email-wenzhuo.lu@intel.com> In-Reply-To: <1454053693-28673-1-git-send-email-wenzhuo.lu@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-cr-hashedpuzzle: lWc= BFaM CYCw DTXb Djlu D34l EGTk Ejk4 Fg6Y Fl1L GroD G+yO HCTO HxPj I3rW JjNM; 1; ZABlAHYAQABkAHAAZABrAC4AbwByAGcA; Sosha1_v1; 7; {33871E55-086E-42A4-862F-64921351EFF2}; aAB1AGkAbABvAG4AZwB4AC4AeAB1AEAAaQBuAHQAZQBsAC4AYwBvAG0A; Fri, 05 Feb 2016 05:09:17 GMT; UgBFADoAIABbAGQAcABkAGsALQBkAGUAdgBdACAAWwBQAEEAVABDAEgAXQAgAGkAeABnAGIAZQA6ACAAZgBpAHgAIAB4ADUANQAwACAAVgBGACAAdAB4ACAAaQBzAHMAdQBlAA== x-cr-puzzleid: {33871E55-086E-42A4-862F-64921351EFF2} x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH] ixgbe: fix x550 VF tx issue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 05:09:25 -0000 Test-by: huilong xu Summary: Test four case, and all case passed.=20 Test environment: Host:=20 os: 4.2.3-300.fc23.x86_64 NIC: 03:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Co= nnection X552/X557-AT 10GBASE-T [8086:15ad] NIC kernel driver: ixgbe 4.2.5 Guest: OS: 3.11.10-301.fc20.x86_64 Gcc: 4.8.2 20131017 (Red Hat 4.8.2-1) (GCC) Package: dpdk.org newest commit(commit b86af7b1b57ebe02ebf5298415172d5= 1f1f81e8b) +PATCH Target: x86_64-native-linuxapp-gcc Case list: Config VM =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 1. get the pci device id of DUT, for example, lspci -nn|grep -i eth 04:00.0 Ethernet controller [0200]: Intel Corporation Device [8086:15ad] (r= ev 01)=20 04:00.1 Ethernet controller [0200]: Intel Corporation Device [8086:15ad] (r= ev 01)=20 2. create 2 VFs from 2 PFs, echo 1 > /sys/bus/pci/devices/0000\:04\:00.0/sriov_numvfs echo 1 > /sys/bus/pci/devices/0000\:04\:00.1/sriov_numvfs lspci -nn|grep -i eth 04:00.0 Ethernet controller [0200]: Intel Corporation Device [8086:15ad] (r= ev 01)=20 04:00.1 Ethernet controller [0200]: Intel Corporation Device [8086:15ad] (r= ev 01) 04:10.0 Ethernet controller [0200]: Intel Corporation Device [8086:15a8] 04:10.1 Ethernet controller [0200]: Intel Corporation Device [8086:15a8] 3. detach VFs from the host, bind them to pci-stub driver, /sbin/modprobe pci-stub using `lspci -nn|grep -i ethernet` got VF device id, for example "8086 1565= ", echo "8086 15a8" > /sys/bus/pci/drivers/pci-stub/new_id echo 0000:04:10.0 > /sys/bus/pci/devices/0000:04:10.0/driver/unbind echo 0000:04:10.1 > /sys/bus/pci/drivers/pci-stub/bind echo "8086 15a8" > /sys/bus/pci/drivers/pci-stub/new_id echo 0000:04:10.0 > /sys/bus/pci/devices/0000:04:10.0/driver/unbind echo 0000:04:10.0 > /sys/bus/pci/drivers/pci-stub/bind or using the following more easy way, virsh nodedev-detach pci_0000_04_10_0;=20 virsh nodedev-detach pci_0000_04_10_1; 4. passthrough VFs 04:10.0 & 04:10.1 to vm0, and start vm0, /usr/bin/qemu-system-x86_64 -name vm0 -enable-kvm \ -cpu host -smp 4 -m 2048 -drive file=3D/home/image/sriov-fc20-1.img -vnc :1= \ -device pci-assign,host=3D04:10.0,id=3Dpt_0 \ -device pci-assign,host=3D04:10.1,id=3Dpt_1 Test Case 1: enable_mdd_dpdk_disable =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 1. config vm 2. login vm0, got VFs pci device id in vm0, assume they are 00:06.0 & 00:07= .0, bind them to igb_uio driver, and then start testpmd, set it in mac forward mode, ./tools/dpdk_nic_bind.py --bind=3Digb_uio 00:06.0 00:07.0 ./x86_64-native-linuxapp-gcc/app/testpmd -c 0x0f -n 4 -w 00:06.0 -w 00:07.0= -- -i --portmask=3D0x3 --txqflags=3D0x1000 testpmd> set fwd mac testpmd> start 3. get mac address of one VF and use it as dest mac, using scapy to send 20= 00 random packets from tester, verify the packets can't be received by one VF, and on host can find "ixgbe= 0000:04:00.0: Malicious event on VF 0 tx:100000 rx:0" Test Case 2: enable_mdd_dpdk_enable =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 1. config vm 2. login vm0, got VFs pci device id in vm0, assume they are 00:06.0 & 00:07= .0, bind them to igb_uio driver, and then start testpmd, set it in mac forward mode, ./tools/dpdk_nic_bind.py --bind=3Digb_uio 00:06.0 00:07.0 ./x86_64-native-linuxapp-gcc/app/testpmd -c 0x0f -n 4 -w 00:06.0 -w 00:07.0= -- -i --portmask=3D0x3 --txqflags=3D0x0 testpmd> set fwd mac testpmd> start 3. get mac address of one VF and use it as dest mac, using scapy to send 20= 00 random packets from tester, verify the packets can be received by one VF, and on host can't find "ixgbe= 0000:04:00.0: Malicious event on VF 0 tx:100000 rx:0" =20 Test Case 3: disable_mdd_dpdk_disable =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 1. disable mdd in kernel rmmod ixgbe insmod MDD=3D0,0 2. config vm 3. login vm0, got VFs pci device id in vm0, assume they are 00:06.0 & 00:07= .0, bind them to igb_uio driver, and then start testpmd, set it in mac forward mode, ./tools/dpdk_nic_bind.py --bind=3Digb_uio 00:06.0 00:07.0 ./x86_64-native-linuxapp-gcc/app/testpmd -c 0x0f -n 4 -w 00:06.0 -w 00:07.0= -- -i --portmask=3D0x3 --txqflags=3D0x1000 testpmd> set fwd mac testpmd> start 4. get mac address of one VF and use it as dest mac, using scapy to send 20= 00 random packets from tester, verify the packets can be received by one VF, and on host can't find "ixgbe= 0000:04:00.0: Malicious event on VF 0 tx:100000 rx:0" Test Case 4: disable_mdd_dpdk_enable =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 1. disable mdd in kernel rmmod ixgbe insmod MDD=3D0,0 2. config vm 3. login vm0, got VFs pci device id in vm0, assume they are 00:06.0 & 00:07= .0, bind them to igb_uio driver, and then start testpmd, set it in mac forward mode, ./tools/dpdk_nic_bind.py --bind=3Digb_uio 00:06.0 00:07.0 ./x86_64-native-linuxapp-gcc/app/testpmd -c 0x0f -n 4 -w 00:06.0 -w 00:07.0= -- -i --portmask=3D0x3 --txqflags=3D0x0 testpmd> set fwd mac testpmd> start 4. get mac address of one VF and use it as dest mac, using scapy to send 20= 00 random packets from tester, verify the packets can be received by one VF, and on host can't find "ixgbe= 0000:04:00.0: Malicious event on VF 0 tx:100000 rx:0" > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wenzhuo Lu > Sent: Friday, January 29, 2016 3:48 PM > To: dev@dpdk.org > Subject: [dpdk-dev] [PATCH] ixgbe: fix x550 VF tx issue >=20 > On x550, the basic tx function may not work if we use DPDK VF based on > kernel PF. The reason is kernel PF enables a new feature "malicious drive= r > detection". According to this feature, driver should create TX context > descriptor and set the "Check Context" bit in advanced data descriptor. > This patch add the support of "malicious driver detection" to let kernel > PF > based DPDK VF work. >=20 > Although CC bit should be set in IOV mode, as tested, it's no harm to set > CC bit in non-IVO mode. So, as PF and VF share the same code, will set CC > bit anyway. >=20 > Please aware there's another way, disabling MDD in PF kernel driver. > Like this, > >insmod ixgbe.ko MDD=3D0,0 >=20 > Signed-off-by: Wenzhuo Lu > --- > doc/guides/rel_notes/release_2_3.rst | 12 ++++++++++++ > drivers/net/ixgbe/ixgbe_rxtx.c | 16 ++++++++++++++-- > drivers/net/ixgbe/ixgbe_rxtx.h | 3 ++- > lib/librte_ether/rte_ethdev.h | 1 + > lib/librte_mbuf/rte_mbuf.h | 3 ++- > 5 files changed, 31 insertions(+), 4 deletions(-) >=20 > diff --git a/doc/guides/rel_notes/release_2_3.rst > b/doc/guides/rel_notes/release_2_3.rst > index 99de186..90f81e0 100644 > --- a/doc/guides/rel_notes/release_2_3.rst > +++ b/doc/guides/rel_notes/release_2_3.rst > @@ -15,6 +15,18 @@ EAL > Drivers > ~~~~~~~ >=20 > +* **ixgbe: Fixed x550 VF tx issue.** > + > + On x550, the basic tx function may not work if we use DPDK VF based on > + kernel PF. The reason is kernel PF enables a new feature "malicious > driver > + detection". According to this feature, driver should create TX context > + descriptor and set the "Check Context" bit in advanced data descriptor= . > + This patch add the support of "malicious driver detection" to let > kernel PF > + based DPDK VF work. > + > + Please aware there's another way, disabling MDD in PF kernel driver. > + Like this, > + >insmod ixgbe.ko MDD=3D0,0 >=20 > Libraries > ~~~~~~~~~ > diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c > b/drivers/net/ixgbe/ixgbe_rxtx.c > index 52a263c..c8a7740 100644 > --- a/drivers/net/ixgbe/ixgbe_rxtx.c > +++ b/drivers/net/ixgbe/ixgbe_rxtx.c > @@ -85,7 +85,8 @@ > PKT_TX_VLAN_PKT | \ > PKT_TX_IP_CKSUM | \ > PKT_TX_L4_MASK | \ > - PKT_TX_TCP_SEG) > + PKT_TX_TCP_SEG | \ > + PKT_TX_MALICIOUS_DRIVER_DETECT) >=20 > static inline struct rte_mbuf * > rte_rxmbuf_alloc(struct rte_mempool *mp) > @@ -564,6 +565,8 @@ ixgbe_xmit_cleanup(struct ixgbe_tx_queue *txq) > return (0); > } >=20 > +#define DEFAULT_CTX_L2_LEN 14 > + > uint16_t > ixgbe_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, > uint16_t nb_pkts) > @@ -614,11 +617,19 @@ ixgbe_xmit_pkts(void *tx_queue, struct rte_mbuf > **tx_pkts, > * are needed for offload functionality. > */ > ol_flags =3D tx_pkt->ol_flags; > + if (!(txq->txq_flags & ETH_TXQ_FLAGS_NOMDD)) > + ol_flags |=3D PKT_TX_MALICIOUS_DRIVER_DETECT; >=20 > /* If hardware offload required */ > tx_ol_req =3D ol_flags & IXGBE_TX_OFFLOAD_MASK; > if (tx_ol_req) { > - tx_offload.l2_len =3D tx_pkt->l2_len; > + /* if l2 len isn't provided by the caller, give it a > + * default value. > + */ > + if (tx_pkt->l2_len =3D=3D 0) > + tx_offload.l2_len =3D DEFAULT_CTX_L2_LEN; > + else > + tx_offload.l2_len =3D tx_pkt->l2_len; > tx_offload.l3_len =3D tx_pkt->l3_len; > tx_offload.l4_len =3D tx_pkt->l4_len; > tx_offload.vlan_tci =3D tx_pkt->vlan_tci; > @@ -801,6 +812,7 @@ ixgbe_xmit_pkts(void *tx_queue, struct rte_mbuf > **tx_pkts, > } >=20 > olinfo_status |=3D (pkt_len << IXGBE_ADVTXD_PAYLEN_SHIFT); > + olinfo_status |=3D IXGBE_ADVTXD_CC; >=20 > m_seg =3D tx_pkt; > do { > diff --git a/drivers/net/ixgbe/ixgbe_rxtx.h > b/drivers/net/ixgbe/ixgbe_rxtx.h > index 475a800..bcde785 100644 > --- a/drivers/net/ixgbe/ixgbe_rxtx.h > +++ b/drivers/net/ixgbe/ixgbe_rxtx.h > @@ -255,7 +255,8 @@ struct ixgbe_txq_ops { > * RTE_PMD_IXGBE_TX_MAX_BURST. > */ > #define IXGBE_SIMPLE_FLAGS ((uint32_t)ETH_TXQ_FLAGS_NOMULTSEGS | \ > - ETH_TXQ_FLAGS_NOOFFLOADS) > + ETH_TXQ_FLAGS_NOOFFLOADS | \ > + ETH_TXQ_FLAGS_NOMDD) >=20 > /* > * Populate descriptors with the following info: > diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.= h > index bada8ad..a2e32d0 100644 > --- a/lib/librte_ether/rte_ethdev.h > +++ b/lib/librte_ether/rte_ethdev.h > @@ -637,6 +637,7 @@ struct rte_eth_rxconf { > #define ETH_TXQ_FLAGS_NOXSUMSCTP 0x0200 /**< disable SCTP checksum > offload */ > #define ETH_TXQ_FLAGS_NOXSUMUDP 0x0400 /**< disable UDP checksum offloa= d > */ > #define ETH_TXQ_FLAGS_NOXSUMTCP 0x0800 /**< disable TCP checksum offloa= d > */ > +#define ETH_TXQ_FLAGS_NOMDD 0x1000 /**< disable malicious driver > detection */ > #define ETH_TXQ_FLAGS_NOOFFLOADS \ > (ETH_TXQ_FLAGS_NOVLANOFFL | ETH_TXQ_FLAGS_NOXSUMSCTP | \ > ETH_TXQ_FLAGS_NOXSUMUDP | ETH_TXQ_FLAGS_NOXSUMTCP) > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h > index f234ac9..b5ed25c 100644 > --- a/lib/librte_mbuf/rte_mbuf.h > +++ b/lib/librte_mbuf/rte_mbuf.h > @@ -102,6 +102,7 @@ extern "C" { >=20 > /* add new TX flags here */ >=20 > +#define PKT_TX_MALICIOUS_DRIVER_DETECT (1ULL << 48) > /** > * Second VLAN insertion (QinQ) flag. > */ > @@ -824,7 +825,7 @@ struct rte_mbuf { > struct rte_mempool *pool; /**< Pool from which mbuf was allocated. > */ > struct rte_mbuf *next; /**< Next segment of scattered packet. */ >=20 > - /* fields to support TX offloads */ > + /* fields to support malicious driver detection or TX offloads */ > union { > uint64_t tx_offload; /**< combined for easy fetch */ > struct { > -- > 1.9.3