Re: Commit broke 32-bit testpmd app

["Roger Melton (rmelton)" <rmelton@cisco.com>]

[-- Attachment #1: Type: text/plain, Size: 7919 bytes --]

+Chris Brezovec

Hi Maxime,

Chris from our team is attending the DPDK Summit in Dublin this week.  If you have some time available, we'd appreciate it if he could meet with you to discuss the 32bit virtio issue we are seeing.

Regards,
Roger Melton

On 9/6/23 2:57 PM, Dave Johnson (davejo) wrote:
Hi Maxime,
This email is regarding the following commit:
https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657

A query had been sent previously on this topic (see below) indicating this commit appears to have broken the 32-bit testpmd app and impacted one of our products that runs as a 32-bit DPDK application.  We consequently backed the commit out of our product but would prefer to get a fix for it.  In the earlier exchange, you had asked if we were using virtio-pci or virtio-user (we are using virtio-pci) and asked for logs which Sampath provided.  It’s been a while, so let me now if you need me to send resend those logs or need any other information.

FWIW, I reproduced this using testpmd and noticed that this part of the change seems to be the interesting part (in drivers/net/virtio/virtqueue.h):

/**
* Return the IOVA (or virtual address in case of virtio-user) of mbuf
* data buffer.
*
* The address is firstly casted to the word size (sizeof(uintptr_t))
* before casting it to uint64_t. This is to make it work with different
* combination of word size (64 bit and 32 bit) and virtio device
* (virtio-pci and virtio-user).
*/
#define VIRTIO_MBUF_ADDR(mb, vq) \
      ((uint64_t)(*(uintptr_t *)((uintptr_t)(mb) + (vq)->mbuf_addr_offset))

If I revert just this part of the changeset (by re-using the VIRTIO_MBUF_ADDR to return buf_iova which matches what it had used previously), then 32-bit testpmd is able to receive traffic again:

#define VIRTIO_MBUF_ADDR(mb, vq) (mb->buf_iova)

Looking at the address produced by each of these, I see the address is the same except that the casting results in the upper bits getting cleared:

Address from patch (nonworking case) = 0x58e7c900
Address using buf_iova (working case) = 0x158e7c900
::
Address from patch (nonworking case) = 0x58e7bfc0
Address using buf_iova (working case) = 0x158e7bfc0
::
Address from patch (nonworking case) = 0x58e7b680
Address using buf_iova (working case) = 0x158e7b680
::

Regards, Dave

From: Sampath Peechu (speechu) <speechu@cisco.com><mailto:speechu@cisco.com>
Date: Monday, January 30, 2023 at 3:29 PM
To: Maxime Coquelin <maxime.coquelin@redhat.com><mailto:maxime.coquelin@redhat.com>, chenbo.xia@intel.com<mailto:chenbo.xia@intel.com> <chenbo.xia@intel.com><mailto:chenbo.xia@intel.com>, dev@dpdk.org<mailto:dev@dpdk.org> <dev@dpdk.org><mailto:dev@dpdk.org>
Cc: Roger Melton (rmelton) <rmelton@cisco.com><mailto:rmelton@cisco.com>, Malcolm Bumgardner (mbumgard) <mbumgard@cisco.com><mailto:mbumgard@cisco.com>
Subject: Re: Commit broke 32-bit testpmd app
Hi Maxime,

Could you please let us know if you got a chance to look at the debugs logs I provided?

Thanks,
Sampath


From: Sampath Peechu (speechu) <speechu@cisco.com><mailto:speechu@cisco.com>
Date: Tuesday, December 6, 2022 at 1:08 PM
To: Maxime Coquelin <maxime.coquelin@redhat.com><mailto:maxime.coquelin@redhat.com>, chenbo.xia@intel.com<mailto:chenbo.xia@intel.com> <chenbo.xia@intel.com><mailto:chenbo.xia@intel.com>, dev@dpdk.org<mailto:dev@dpdk.org> <dev@dpdk.org><mailto:dev@dpdk.org>
Cc: Roger Melton (rmelton) <rmelton@cisco.com><mailto:rmelton@cisco.com>
Subject: Re: Commit broke 32-bit testpmd app
Hi Maxime,

Did you get a chance to look into this?

Please let me know if you need anything else.

Thanks,
Sampath

From: Sampath Peechu (speechu) <speechu@cisco.com><mailto:speechu@cisco.com>
Date: Wednesday, November 23, 2022 at 5:15 PM
To: Maxime Coquelin <maxime.coquelin@redhat.com><mailto:maxime.coquelin@redhat.com>, chenbo.xia@intel.com<mailto:chenbo.xia@intel.com> <chenbo.xia@intel.com><mailto:chenbo.xia@intel.com>, dev@dpdk.org<mailto:dev@dpdk.org> <dev@dpdk.org><mailto:dev@dpdk.org>
Cc: Roger Melton (rmelton) <rmelton@cisco.com><mailto:rmelton@cisco.com>
Subject: Re: Commit broke 32-bit testpmd app
Hi Maxime,

I’m attaching the following for reference.


  *   Instructions for Centos8 test setup
  *   Diffs between the working and non-working versions (working version has the problem commit backed out)
  *   Working logs (stats show that ping packets from neighbor VM can be seen with both 64-bit and 32-bit apps)
  *   Non-working logs (stats show that ping packets from neighbor VM are seen with 64-bit app but NOT seen with 32-bit app)


============================

$ sudo ./usertools/dpdk-devbind.py --status

Network devices using DPDK-compatible driver
============================================
0000:07:00.0 'Virtio network device 1041' drv=igb_uio unused=
0000:08:00.0 'Virtio network device 1041' drv=igb_uio unused=

Network devices using kernel driver
===================================
0000:01:00.0 'Virtio network device 1041' if=enp1s0 drv=virtio-pci unused=igb_uio *Active*

…
===========================


Thanks,
Sampath



From: Maxime Coquelin <maxime.coquelin@redhat.com><mailto:maxime.coquelin@redhat.com>
Date: Tuesday, November 22, 2022 at 4:24 AM
To: Sampath Peechu (speechu) <speechu@cisco.com><mailto:speechu@cisco.com>, chenbo.xia@intel.com<mailto:chenbo.xia@intel.com> <chenbo.xia@intel.com><mailto:chenbo.xia@intel.com>, dev@dpdk.org<mailto:dev@dpdk.org> <dev@dpdk.org><mailto:dev@dpdk.org>
Cc: Roger Melton (rmelton) <rmelton@cisco.com><mailto:rmelton@cisco.com>
Subject: Re: Commit broke 32-bit testpmd app
Hi,

In my initial reply (see below), I also asked if you had logs to share.
And wondered whether it happens with Virtio PCI or Virtio-user?

Regards,
Maxime

On 11/16/22 00:30, Sampath Peechu (speechu) wrote:
> ++ dev@dpdk.org<mailto:dev@dpdk.org> <mailto:dev@dpdk.org>
>
> *From: *Maxime Coquelin <maxime.coquelin@redhat.com><mailto:maxime.coquelin@redhat.com>
> *Date: *Tuesday, November 15, 2022 at 3:19 AM
> *To: *Sampath Peechu (speechu) <speechu@cisco.com><mailto:speechu@cisco.com>, chenbo.xia@intel.com<mailto:chenbo.xia@intel.com>
> <chenbo.xia@intel.com><mailto:chenbo.xia@intel.com>
> *Cc: *Roger Melton (rmelton) <rmelton@cisco.com><mailto:rmelton@cisco.com>
> *Subject: *Re: Commit broke 32-bit testpmd app
>
> Hi Sampath,
>
>
> Please add dev@dpdk.org<mailto:dev@dpdk.org>, the upstream mailing list, if this is related
> to the upstream DPDK project.If it is using RHEL DPDK package, please
> use the appropriate support channels.
>
> On 11/14/22 23:55, Sampath Peechu (speechu) wrote:
>  > Hi Virtio Maintainers team,
>  >
>  > This email is regarding the following commit.
>  >
>  >
> https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657 <https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657> <https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657 <https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657>>
>  >
>  > The above commit appears to have broken the 32-bit testpmd app (and
>  > consequently impacted one of our products that runs as a 32-bit DPDK
>  > app). The 64-bit testpmd app does not appear to be impacted though.
>
> We'll need some logs to understand what is going on.
> Does it happen with virtio-pci or virtio-user?
>
> Regards,
> Maxime
>
>  > With the commit in place, we didn’t see any packets going through at
>  > all. After backing out the commit and rebuilding the 32-bit testpmd app
>  > in our test setup, we were able to pass traffic as expected.
>  >
>  > Could you please let us know if this is a known issue? And if there is a
>  > fix available for it?
>  >
>  > Thank you,
>  >
>  > Sampath Peechu
>  >
>  > Cisco Systems
>  >
>


[-- Attachment #2: Type: text/html, Size: 20187 bytes --]