From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8133843D99 for ; Wed, 24 Apr 2024 12:59:16 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F0B27402DD; Wed, 24 Apr 2024 12:59:15 +0200 (CEST) Received: from mail-qv1-f54.google.com (mail-qv1-f54.google.com [209.85.219.54]) by mails.dpdk.org (Postfix) with ESMTP id 53F6C400D6 for ; Wed, 24 Apr 2024 12:59:15 +0200 (CEST) Received: by mail-qv1-f54.google.com with SMTP id 6a1803df08f44-69b6d36b71cso33540416d6.3 for ; Wed, 24 Apr 2024 03:59:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713956354; x=1714561154; darn=dpdk.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=CZaEfm3d1MfuPnJmbA5n9DrFN95d7PDAyQBONMxraBw=; b=kdAwu8kEyHOnO5MwyuM1vu+1rADUYw+sj7Ih24k0HJbsljplOFEfIEg27vqPfpEIEQ bVkGJDNgGim8zpPUaqA4ZEbKmTYe7WCwpd36Z76I4+UUByXCAA7JBoJ3Mu+VG/6c4A3L e0HrXEiMpOdTXpK6zljeebhbCN7xEajuiPDdoGkplv/ajp7FLPZUvTO7IuoUVYeIwP1v 6AFxssoQSYNiP5cyM39Vmq6bmZQC+wWfweb+A/rsr9N5nEOxf00T0u8p36LlE3VrLcbb TjT3tRuhiRo/yJiTIGtKzGZg9xBXNgQ43OhZoLK080NXhJRvxu+U2uYt1YCXVLQKTSy8 AXaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713956354; x=1714561154; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=CZaEfm3d1MfuPnJmbA5n9DrFN95d7PDAyQBONMxraBw=; b=F8zobFtn7M6hvMaL95RNnIO1keIlWKZflqP3iZCUfg7YbwpeeokOuV/aKDTDWjkfb3 0Cx0kc7YaB8S91DhCfwF+s72UsZYuCAa26dLRanLu7kBu5fAR+NJHBG00OiMmiu4ST8/ CPps+D9QkoDRnrpaIfOWKOk9mIV70aL7fbXD5xzEK0/uIwKCvlZETVA6mtfmoFqy+ztJ XyzulawvKGej9IEgcXbGQwFb9834ImbmyFu5PKwZAPKAUK8QOJeH7A4p62+oil+qNZDx k4FpbmBf4Y2ROX/xjEquqdB08QVdnCeky4H0NEJ/RkSRiLt//lcsHM4JxldK0ywXTMYz cEgA== X-Gm-Message-State: AOJu0YwjDZLdxLGCEi2hR03ZiiOjn7ReMsWHG1UojO/LR71nTVUUhutu WdCoBiqJPsQol7E3f/np3aMjvuhxLlwP3Yaz+gawbTARnq+yqjp4BPgUZ667EQK9D+hZpIW6yYu B7kNijzUY6Nllyk8ZTlFAOVYOAsixMGnP X-Google-Smtp-Source: AGHT+IHLSon8+VICg/0y34WQqgkUeWDx8TYKM9j7slWHSf4U10B+4fxD0kXOZuBRYfiVrJ4IaqM5VK/gIFwvoDNgmks= X-Received: by 2002:a05:6214:76f:b0:69b:7b36:30fe with SMTP id f15-20020a056214076f00b0069b7b3630femr2704592qvz.38.1713956354451; Wed, 24 Apr 2024 03:59:14 -0700 (PDT) MIME-Version: 1.0 From: Pavel Vazharov Date: Wed, 24 Apr 2024 13:59:03 +0300 Message-ID: Subject: How to setup SR-IOV based flow bifurication? To: users Content-Type: multipart/alternative; boundary="000000000000aaeeac0616d58f11" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --000000000000aaeeac0616d58f11 Content-Type: text/plain; charset="UTF-8" Hi there, I'm trying to achieve traffic splitting between the OS and a DPDK application. I tried an XDP based solution but hit some problems with the zero copy support by the ixgbe driver. Now I'm trying to setup this scenario on bare metal without any VM involved - https://doc.dpdk.org/guides/howto/flow_bifurcation.html The VFs are setup to use the vfio-pci driver dpdk_devbind.py --status Network devices using DPDK-compatible driver ============================================ 0000:1b:10.0 '82599 Ethernet Controller Virtual Function 10ed' drv=vfio-pci unused= 0000:3b:10.0 '82599 Ethernet Controller Virtual Function 10ed' drv=vfio-pci unused= 0000:5e:10.0 '82599 Ethernet Controller Virtual Function 10ed' drv=vfio-pci unused= Network devices using kernel driver =================================== 0000:1a:00.0 'Ethernet Connection X722 for 1GbE 37d1' if=eth6 drv=i40e unused=vfio-pci *Active* 0000:1a:00.1 'Ethernet Connection X722 for 1GbE 37d1' if=eth7 drv=i40e unused=vfio-pci 0000:1b:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=eth0 drv=ixgbe unused=vfio-pci 0000:1b:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=eth1 drv=ixgbe unused=vfio-pci 0000:3b:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=eth2 drv=ixgbe unused=vfio-pci 0000:3b:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=eth3 drv=ixgbe unused=vfio-pci 0000:5e:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=eth4 drv=ixgbe unused=vfio-pci 0000:5e:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=eth5 drv=ixgbe unused=vfio-pci But then when I start the DPDK application I hit these errors EAL: Detected CPU lcores: 48 EAL: Detected NUMA nodes: 2 EAL: Detected static linkage of DPDK EAL: Selected IOVA mode 'PA' EAL: No free 2048 kB hugepages reported on node 1 EAL: No free 1048576 kB hugepages reported on node 0 EAL: No free 1048576 kB hugepages reported on node 1 EAL: VFIO support initialized EAL: Using IOMMU type 8 (No-IOMMU) EAL: Probe PCI driver: net_ixgbe_vf (8086:10ed) device: 0000:1b:10.0 (socket 0) eth_ixgbevf_dev_init(): VF Initialization Failure: -15 EAL: Releasing PCI mapped resource for 0000:1b:10.0 EAL: Calling pci_unmap_resource for 0000:1b:10.0 at 0x4201000000 EAL: Calling pci_unmap_resource for 0000:1b:10.0 at 0x4201004000 EAL: Requested device 0000:1b:10.0 cannot be used EAL: Using IOMMU type 8 (No-IOMMU) EAL: Probe PCI driver: net_ixgbe_vf (8086:10ed) device: 0000:3b:10.0 (socket 0) eth_ixgbevf_dev_init(): VF Initialization Failure: -15 EAL: Releasing PCI mapped resource for 0000:3b:10.0 EAL: Calling pci_unmap_resource for 0000:3b:10.0 at 0x4201008000 EAL: Calling pci_unmap_resource for 0000:3b:10.0 at 0x420100c000 EAL: Requested device 0000:3b:10.0 cannot be used EAL: Using IOMMU type 8 (No-IOMMU) EAL: Probe PCI driver: net_ixgbe_vf (8086:10ed) device: 0000:5e:10.0 (socket 0) eth_ixgbevf_dev_init(): VF Initialization Failure: -15 I searched through the source code and found that this error is: IXGBE_ERR_RESET_FAILED. So, by some reason the reset of the VFs fails. My questions are: 1. Can this whole scheme work or I'm trying something which is not supposed to work? 2. If the above is supposed to work, can you point me to some examples of how to set up the above or where I'm possibly doing something incorrectly? 3. The same application works correctly if it's given the physical devices on this server. Probably I need to modify something in the application? Thanks, Pavel. --000000000000aaeeac0616d58f11 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi there,

I'm trying to ac= hieve traffic splitting between the OS and a DPDK application.
I tried an XDP based solution but hit some problems with the zero copy su= pport by the ixgbe driver.
Now I'm trying to setup this scenar= io on bare metal without any VM involved - https://doc.dpdk.org/guides/howto/flow_= bifurcation.html

The VFs are setup to use the vfio-pc= i driver
dpdk_devbind.py --status

Network devices using DPDK-comp= atible driver
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
0000:1b:10.0 '82599 Ethernet Controller Virtual Function 10ed' = drv=3Dvfio-pci unused=3D
0000:3b:10.0 '82599 Ethernet Controller Vir= tual Function 10ed' drv=3Dvfio-pci unused=3D
0000:5e:10.0 '82599= Ethernet Controller Virtual Function 10ed' drv=3Dvfio-pci unused=3D
Network devices using kernel driver
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
0000:1a:00.0 'Ethernet Connection X722 for 1GbE 37d1' if=3Deth6= drv=3Di40e unused=3Dvfio-pci *Active*
0000:1a:00.1 'Ethernet Connec= tion X722 for 1GbE 37d1' if=3Deth7 drv=3Di40e unused=3Dvfio-pci
000= 0:1b:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if= =3Deth0 drv=3Dixgbe unused=3Dvfio-pci
0000:1b:00.1 '82599ES 10-Giga= bit SFI/SFP+ Network Connection 10fb' if=3Deth1 drv=3Dixgbe unused=3Dvf= io-pci
0000:3b:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection= 10fb' if=3Deth2 drv=3Dixgbe unused=3Dvfio-pci
0000:3b:00.1 '82= 599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=3Deth3 drv=3Dixgb= e unused=3Dvfio-pci
0000:5e:00.0 '82599ES 10-Gigabit SFI/SFP+ Netwo= rk Connection 10fb' if=3Deth4 drv=3Dixgbe unused=3Dvfio-pci
0000:5e= :00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=3Det= h5 drv=3Dixgbe unused=3Dvfio-pci

But then when I start th= e DPDK application I hit these errors

EAL: Detected CPU lcores: 48EAL: Detected NUMA nodes: 2
EAL: Detected static linkage of DPDK
EA= L: Selected IOVA mode 'PA'
EAL: No free 2048 kB hugepages report= ed on node 1
EAL: No free 1048576 kB hugepages reported on node 0
EAL= : No free 1048576 kB hugepages reported on node 1
EAL: VFIO support init= ialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net= _ixgbe_vf (8086:10ed) device: 0000:1b:10.0 (socket 0)
eth_ixgbevf_dev_in= it(): VF Initialization Failure: -15
EAL: Releasing PCI mapped resource = for 0000:1b:10.0
EAL: Calling pci_unmap_resource for 0000:1b:10.0 at 0x4= 201000000
EAL: Calling pci_unmap_resource for 0000:1b:10.0 at 0x42010040= 00
EAL: Requested device 0000:1b:10.0 cannot be used
EAL: Using IOMMU= type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ixgbe_vf (8086:10ed) devic= e: 0000:3b:10.0 (socket 0)
eth_ixgbevf_dev_init(): VF Initialization Fai= lure: -15
EAL: Releasing PCI mapped resource for 0000:3b:10.0
EAL: Ca= lling pci_unmap_resource for 0000:3b:10.0 at 0x4201008000
EAL: Calling p= ci_unmap_resource for 0000:3b:10.0 at 0x420100c000
EAL: Requested device= 0000:3b:10.0 cannot be used
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: = Probe PCI driver: net_ixgbe_vf (8086:10ed) device: 0000:5e:10.0 (socket 0)<= br>eth_ixgbevf_dev_init(): VF Initialization Failure: -15

I searched through the source code and found that this error is: IXGBE_ERR= _RESET_FAILED.
=C2=A0So, by some reason the reset of the VFs = fails.

My questions are:
1. Can this whole = scheme work or I'm trying something which is not supposed to work?
<= /div>
2. If the above is supposed to work, can you point me to some exa= mples of how to set up the above or where I'm possibly doing something = incorrectly?
3. The same application works correctly if it= 9;s given the physical devices on this server. Probably I need to modify so= mething in the application?

Thanks,
Pavel.<= br>


--000000000000aaeeac0616d58f11--