From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 526D1A0C40 for ; Thu, 29 Jul 2021 09:49:07 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D661440687; Thu, 29 Jul 2021 09:49:06 +0200 (CEST) Received: from mail-lj1-f175.google.com (mail-lj1-f175.google.com [209.85.208.175]) by mails.dpdk.org (Postfix) with ESMTP id 620E0410EC for ; Tue, 27 Jul 2021 04:17:27 +0200 (CEST) Received: by mail-lj1-f175.google.com with SMTP id u20so13555856ljo.0 for ; Mon, 26 Jul 2021 19:17:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=po/YQvcTUwY07VY5LtVGdRSZqtpk8XTnhOBZwIWvWDQ=; b=LR1jeOKYAMjoh4QWGiqx8KwHK/l/5vIRqBmRyJO9FzkxcQlERgTCFk6aicD2YQ3jEA wHThoAVZOPUDbfI0QjITgL4SUWkE7XR/mlosTXfG7dtTApKmSxlHmSVrSV2iaiIJSPE9 ZxNiDrINTiuJMzWlgCg3/us3Z5auhs44bErJrvTdIAwxqu49fPU8P7SrTC5VrY6iHS9c AOqk44//BBeFDuAIITeOG+wAg7ofWceoTVsvzmVzeDoE0aYAnTjBREvZ4fzum4+eg1AC iFCiEbi9VHnObzSkKW/Txgo7k5+3r5jdhNgtFJtPiJD4nyLXngeGglh7+BT6y4cYamoV uHAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=po/YQvcTUwY07VY5LtVGdRSZqtpk8XTnhOBZwIWvWDQ=; b=iGBy4BhRC6Mu2ji/eWOj/wP0SSoh9y+B2YGUM4ktE2Wh9+HncR+sig7oiflBbJVAMl uTGKZtRNj3kf+4cjakSRV6QdxseunK+fFaTnsIRqRkDDHe/wvfaLWPtC+B7w9cvA6AWC H6MvB3wQz1Mr1B+QpbrLipErQs6ZWBe9n/xZP6pLGdjdqg9VO6hEDivX+cGh6bMr0EIK WvdizD6rJrDBv8/MZd5trGb6hAWXIw4Pk2sz45RGOA1GPTn3iGTiNT72oezk9PCNuSAX WoQj65p8ChcFcnPkHeG1i9tzUc/7JHZCnlZ0NXDSgi4zE1etsnckVv5yuCQDiRZJHG+W tMRA== X-Gm-Message-State: AOAM531sg1yHbkRiYgwhLzKNBnelFnrNZ4+brdL/4kL9NLm4IC91JcIc 50OsRfi9QdvhzApngErgSmskBNfqfdSvBviRCYrzY2jELjWvarnp X-Google-Smtp-Source: ABdhPJw+v4w8klAo2dlIyGDglv3qhkF9k+a4VqFYzoZjp03+qv4XDK/JvEyhUEC2a6GpXLIzF643Cxg4e1sXMvdJvOY= X-Received: by 2002:a2e:a68f:: with SMTP id q15mr13581576lje.314.1627352246686; Mon, 26 Jul 2021 19:17:26 -0700 (PDT) MIME-Version: 1.0 From: Hao Li Date: Tue, 27 Jul 2021 10:17:16 +0800 Message-ID: To: users@dpdk.org X-Mailman-Approved-At: Thu, 29 Jul 2021 09:49:05 +0200 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: [dpdk-users] How to configue XL710 to communicate between virtual functions in KVM? X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Hi all, I have an XL710 (2 ports 40Gbe) and configure it to have 8 virtual functions (VFs). I then install two KVM guest OSes (both Ubuntu 20.04) and assign one VF for each of them. In host OS, ifconfig shows the following: enp6s0f0: flags=4099 mtu 1500 > enp6s0f1: flags=4098 mtu 1500 > enp6s0f0v0: flags=4098 mtu 1500 > enp6s0f0v3: flags=4098 mtu 1500 > enp6s0f0v4: flags=4098 mtu 1500 > enp6s0f0v5: flags=4098 mtu 1500 > enp6s0f0v6: flags=4098 mtu 1500 > enp6s0f0v7: flags=4098 mtu 1500 where enp6s0f0 and enp6s0f1 are two PF ports, and others are VFs from enp6s0f0. enp6s0f0v1 and enp6s0f0v2 disappear since they are assigned to the two KVM VMs. In guest OS, ifconfig shows the following: enp1s0: flags=4163 mtu 1500 > enp6s0: flags=4098 mtu 1500 > lo: flags=73 mtu 65536 where enp6s0 is the VF assigned to it. I then bind the uio_pci_generic for this VF Network devices using DPDK-compatible driver > ============================================ > 0000:06:00.0 'Ethernet Virtual Function 700 Series 154c' > drv=uio_pci_generic unused=iavf,vfio-pci > Network devices using kernel driver > =================================== > 0000:01:00.0 'Virtio network device 1041' if=enp1s0 drv=virtio-pci > unused=vfio-pci,uio_pci_generic *Active* I can then use "-a 06:00.0,driver=i40avf" to properly start this VF inside the VM. (at least I can send packets with it) Now my question is, how to forward packets between two VFs that have already been assigned to VMs? Or, putting it in a simpler way, how to determine which VFs should be sent to when a packet comes into the physical port? I used to think that I just need to configure the dst mac address of the incoming packet, but it seems not to work. Then I learn that rte_flow can be used to direct the flows between VFs, but since VFs are already assigned to VMs, I don't think the host DPDK application can configure them using rte_flow. I'm not sure I'm doing everything right, please indicate if I misunderstood anything. Thanks in advance! Best, Hao