DPDK usage discussions
 help / color / mirror / Atom feed
From: Hao Li <aquatoney@gmail.com>
To: users@dpdk.org
Subject: [dpdk-users] How to configue XL710 to communicate between virtual functions in KVM?
Date: Tue, 27 Jul 2021 10:17:16 +0800	[thread overview]
Message-ID: <CANPHSbdS0DujMCLXtHk6AKNCYH4dPmS-in4KVvMTJ91KthwVbA@mail.gmail.com> (raw)

Hi all,

I have an XL710 (2 ports 40Gbe) and configure it to have 8 virtual
functions (VFs).
I then install two KVM guest OSes (both Ubuntu 20.04) and assign one VF for
each of them.

In host OS, ifconfig shows the following:

enp6s0f0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
> enp6s0f1: flags=4098<BROADCAST,MULTICAST>  mtu 1500
> enp6s0f0v0: flags=4098<BROADCAST,MULTICAST>  mtu 1500
> enp6s0f0v3: flags=4098<BROADCAST,MULTICAST>  mtu 1500
> enp6s0f0v4: flags=4098<BROADCAST,MULTICAST>  mtu 1500
> enp6s0f0v5: flags=4098<BROADCAST,MULTICAST>  mtu 1500
> enp6s0f0v6: flags=4098<BROADCAST,MULTICAST>  mtu 1500
> enp6s0f0v7: flags=4098<BROADCAST,MULTICAST>  mtu 1500


where enp6s0f0 and enp6s0f1 are two PF ports, and others are VFs from
enp6s0f0.
enp6s0f0v1 and enp6s0f0v2 disappear since they are assigned to the two KVM
VMs.

In guest OS, ifconfig shows the following:

enp1s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
> enp6s0: flags=4098<BROADCAST,MULTICAST>  mtu 1500
> lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536


where enp6s0 is the VF assigned to it.

I then bind the uio_pci_generic for this VF

Network devices using DPDK-compatible driver
> ============================================
> 0000:06:00.0 'Ethernet Virtual Function 700 Series 154c'
> drv=uio_pci_generic unused=iavf,vfio-pci
> Network devices using kernel driver
> ===================================
> 0000:01:00.0 'Virtio network device 1041' if=enp1s0 drv=virtio-pci
> unused=vfio-pci,uio_pci_generic *Active*

I can then use "-a 06:00.0,driver=i40avf" to properly start this VF inside
the VM. (at least I can send packets with it)

Now my question is, how to forward packets between two VFs that have
already been assigned to VMs?
Or, putting it in a simpler way, how to determine which VFs should be sent
to when a packet comes into the physical port?

I used to think that I just need to configure the dst mac address of the
incoming packet, but it seems not to work.
Then I learn that rte_flow can be used to direct the flows between VFs, but
since VFs are already assigned to VMs,
I don't think the host DPDK application can configure them using rte_flow.

I'm not sure I'm doing everything right, please indicate if I misunderstood
anything.

Thanks in advance!

Best,
Hao

                 reply	other threads:[~2021-07-29  7:49 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CANPHSbdS0DujMCLXtHk6AKNCYH4dPmS-in4KVvMTJ91KthwVbA@mail.gmail.com \
    --to=aquatoney@gmail.com \
    --cc=users@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).