32-bit virtio failing on DPDK v23.11.1 (and tags)

["Chris Brezovec (cbrezove)" <cbrezove@cisco.com>]

[-- Attachment #1.1: Type: text/plain, Size: 3006 bytes --]

HI Maxime,

My name is Chris Brezovec, we met and talked about some 32 bit virtio issues we were seeing at Cisco during the DPDK summit last year.  There was also a back and forth between you and Dave Johnson at Cisco last September regarding the same issue.  I have attached some of the email chain from that conversation that resulted in this commit being made to dpdk v23.11 (https://github.com/DPDK/dpdk/commit/8c41645be010ec7fa0df4f6c3790b167945154b4).

We recently picked up the v23.11.1 DPDK release and saw that 32 bit virtio is not working again, but 64-bit virtio is working.  We are noticing CVQ timeouts - PMD receives no response from host and this leads to failure of the port to start.  We were able to recreate this issue using testpmd.  We have done some tracing through the virtio changes made during the development of the v23.xx DPDK release, and believe we have identified the following rework commit to have caused a failure (https://github.com/DPDK/dpdk/commit/a632f0f64ffba3553a18bdb51a670c1b603c0ce6).

We have also tested v23.07, v23.11, v23.11.2-rc2, v24.07 and they all seem to see the same issue when running in 32-bit mode using testpmd.

We were hoping you might be able to take a quick look at the two commits and see if there might be something obvious missing in the refactor work that might have caused this issue.  I am thinking there might a location or two in the code that should be using the VIRTIO_MBUF_ADDR() or similar macro that might have been missed.

Regards,
ChrisB

This is some of the testpmd output seen on v23.11.2-rc2:

LD_LIBRARY_PATH=/home/rmelton/scratch/dpdk-v23.11.2-rc2.git/build/lib /home/rmelton/scratch/dpdk-v23.11.2-rc2.git/build/app/dpdk-testpmd -l 2-3 -a 0000:07:00.0 --log-level pmd.net.iavf.*,8 --log-level lib.eal.*,8 --log-level=lib.eal:info --log-level=lib.eal:debug --log-level=lib.ethdev:info --log-level=lib.ethdev:debug --log-level=lib.virtio:warning --log-level=lib.virtio:info --log-level=lib.virtio:debug --log-level=pmd.*:debug --iova-mode=pa -- -i

— snip —

virtio_send_command(): vq->vq_desc_head_idx = 0, status = 255, vq->hw->cvq = 0x76d9acc0 vq = 0x76d9ac80
virtio_send_command_split(): vq->vq_queue_index = 2
virtio_send_command_split(): vq->vq_free_cnt=64
vq->vq_desc_head_idx=0
virtio_dev_promiscuous_disable(): Failed to disable promisc
Failed to disable promiscuous mode for device (port 0): Resource temporarily unavailable
Error during restoring configuration for device (port 0): Resource temporarily unavailable
virtio_dev_stop(): stop
Fail to start port 0: Resource temporarily unavailable
Done
virtio_send_command(): vq->vq_desc_head_idx = 0, status = 255, vq->hw->cvq = 0x76d9acc0 vq = 0x76d9ac80
virtio_send_command_split(): vq->vq_queue_index = 2
virtio_send_command_split(): vq->vq_free_cnt=64
vq->vq_desc_head_idx=0
virtio_dev_promiscuous_enable(): Failed to enable promisc
Error during enabling promiscuous mode for port 0: Resource temporarily unavailable - ignore



[-- Attachment #1.2: Type: text/html, Size: 7948 bytes --]

[-- Attachment #2: Re- Commit broke 32-bit testpmd app.eml --]
[-- Type: application/octet-stream, Size: 32884 bytes --]

Received: from LV3PR11MB8577.namprd11.prod.outlook.com (2603:10b6:408:1b8::21)
 by MN2PR11MB4190.namprd11.prod.outlook.com with HTTPS; Wed, 20 Sep 2023
 13:06:04 +0000
Received: from MN2PR11CA0014.namprd11.prod.outlook.com (2603:10b6:208:23b::19)
 by LV3PR11MB8577.namprd11.prod.outlook.com (2603:10b6:408:1b8::21) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.28; Wed, 20 Sep
 2023 13:05:57 +0000
Received: from BL02EPF0001A106.namprd05.prod.outlook.com
 (2603:10b6:208:23b:cafe::3c) by MN2PR11CA0014.outlook.office365.com
 (2603:10b6:208:23b::19) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.29 via Frontend
 Transport; Wed, 20 Sep 2023 13:05:57 +0000
Authentication-Results: spf=fail (sender IP is 173.37.142.88)
 smtp.mailfrom=redhat.com; dkim=pass (signature was verified)
 header.d=redhat.com;dmarc=pass action=none header.from=redhat.com;
Received-SPF: Fail (protection.outlook.com: domain of redhat.com does not
 designate 173.37.142.88 as permitted sender) receiver=protection.outlook.com;
 client-ip=173.37.142.88; helo=alln-iport-1.cisco.com;
Received: from alln-iport-1.cisco.com (173.37.142.88) by
 BL02EPF0001A106.mail.protection.outlook.com (10.167.241.139) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.6792.20 via Frontend Transport; Wed, 20 Sep 2023 13:05:57 +0000
X-CSE-ConnectionGUID: 364BvZ77Qce2yYSKyW/L1w==
X-CSE-MsgGUID: T2M1IHhJQNC5y98vzzMbxw==
Received: from alln-core-10.cisco.com ([173.36.13.132])
  by alln-iport-1.cisco.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Sep 2023 13:05:49 +0000
Received: from alln-inbound-j.cisco.com (alln-inbound-j.cisco.com [173.37.147.240])
	by alln-core-10.cisco.com (8.15.2/8.15.2) with ESMTPS id 38KD5j6t014317
	(version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK)
	for <cbrezove@cisco.com>; Wed, 20 Sep 2023 13:05:49 GMT
X-CSE-ConnectionGUID: 7b2kavB8QymHXqbZhPtiKg==
X-CSE-MsgGUID: kwPRJO+MSEKP2+2gI4VkPg==
Received-SPF: Pass (alln-inbound-j.cisco.com: domain of
  maxime.coquelin@redhat.com designates 170.10.129.124 as
  permitted sender) identity=mailfrom;
  client-ip=170.10.129.124; receiver=alln-inbound-j.cisco.com;
  envelope-from="maxime.coquelin@redhat.com";
  x-sender="maxime.coquelin@redhat.com";
  x-conformance=spf_only; x-record-type="v=spf1";
  x-record-text="v=spf1 ip4:162.247.160.0/21 ip4:167.89.0.0/17
  ip4:168.245.0.0/17 ip4:170.10.128.0/24 ip4:170.10.129.0/24
  ip4:170.10.133.0/24 ip4:173.194.0.0/16 ip4:185.58.84.93/32
  ip4:192.250.208.112 ip4:192.250.208.113 ip4:192.254.112.0/20
  ip4:198.2.128.0/18 ip4:198.21.0.0/21 ip4:198.245.80.0/20
  ip4:198.37.144.0/20 ip4:199.122.120.0/21
  include:spf3.redhat.com -all"
Received-SPF: None (alln-inbound-j.cisco.com: no sender
  authenticity information available from domain of
  postmaster@us-smtp-delivery-124.mimecast.com) identity=helo;
  client-ip=170.10.129.124; receiver=alln-inbound-j.cisco.com;
  envelope-from="maxime.coquelin@redhat.com";
  x-sender="postmaster@us-smtp-delivery-124.mimecast.com";
  x-conformance=spf_only
Authentication-Results-Original: alln-inbound-j.cisco.com; spf=Pass
 smtp.mailfrom=maxime.coquelin@redhat.com; spf=None
 smtp.helo=postmaster@us-smtp-delivery-124.mimecast.com; dkim=pass (signature
 verified) header.i=@redhat.com; dmarc=pass (p=none dis=none) d=redhat.com
IronPort-SDR: 650aee2b_QD9L5HCf0QFwtsHcA+w02gdMFgcWHFlQIekm/VWausdufWn
 QGqwhj6JUJA+OZOtEZOhTVezPgTzSwRp+EuvWKQ==
X-from-outside-Cisco: 170.10.129.124
X-ThreatScanner-Verdict: Negative
X-IPAS-Result: =?us-ascii?q?A0EDAAAG7Qplh3yBCqpaGgEBAQEBAQEBAQEDAQEBARIBA?=
 =?us-ascii?q?QEBAgIBAQEBQIE7BQEBAQELAYI2eYEGBAtHhFODT4ROX4ZAgiMDhDyXaYFWF?=
 =?us-ascii?q?IERAxg+DwEBAgEBAQEIAQE7CQQBAQMBA4UAAocBAh4HATQJDgECAQIBAQEBA?=
 =?us-ascii?q?QMCAwEBAQEBAQECAQEBBAEBAQIBAQIEAgIBAQIQAQEBASIXBw4QJ4U7CCUNg?=
 =?us-ascii?q?lkZgQyBHQEBAQEBAQEBAQEBAQEBAQEBAQEXAg14AQEBAQIBEhEECwEFCAEBL?=
 =?us-ascii?q?AsBBAsJAhEDAQIBAgIjAwICRgkIBgEMBgIBAR6CXAGCOyMDEZQwjkIBjHQBA?=
 =?us-ascii?q?QF4fzOBAYIJAQEGBAMCAX2BX64wgV8JgRouAYgIAYU5GIM7eicPgVVEgRUng?=
 =?us-ascii?q?RWBbz6BU4EPAoErARIBIRUXgy+CaIcmgiCFPwMCAjKCJoMvAyeKJSqBCAhcg?=
 =?us-ascii?q?Wo9Ag1UCwtdgRFROIE7AgIRJxIUBUJwGwMHA4EEECsHBDIbBwYJFhgVJQZRB?=
 =?us-ascii?q?C0kCRMSPgSBZ4FRCoEGPxEOEYJFIgIHNjYZS4JdCRUMNQRKdhArBBQYgRMEa?=
 =?us-ascii?q?h8VHjcREhkNAwh2HQIRIzwDBQMENgoVDQshBVcDRwZLCwMCHAUDAwSBNgUPH?=
 =?us-ascii?q?gIQGgYOKQMDGVACEBQDPgMDBgMLMAQOAxkrHUACAQttPTUGAwsbBkACJ6FEA?=
 =?us-ascii?q?26Bby0+BoE7IGAbBSUjGjALCy+ScQkEsQM0B4QOgVkGDIoVlQoGDwQvhAGMb?=
 =?us-ascii?q?gOGJRSKaTOHGJgtII1BlUqEcQIEAgQFAhaBYzpscE0jUIJnCUYDGQ+BNoxqC?=
 =?us-ascii?q?QMNCYNWhWWKFj81AgkwAgcBCgEBAwmJIYIoAQE?=
IronPort-PHdr: A9a23:8wwduxQU8XxBQszZLZZTss6mANpsovmRAWYlg6HPa5pwe6iut67vI
 FbYra00ygOTA8OCu6kP1ruempujcFJDyK7CikxKSIZLWR4BhJdetC0bK+nBJGvFadXHVGgEJ
 vlET0Jv5HqhMEJYS47UblzWpWCuv3ZJQk2sfQV6Kf7oFYHMks+5y/69+4HJYwVPmTGxfa5+I
 A+5oAnPt8Qam5ZuJ6U/xxfGonZFf/ldyH91K16Ugxvz6cC88YJ5/ShXp/wv6dNLX7/gf6Q/Q
 7xYDTAmPH4w6cb2qxTNThaB62UFXGkOnRRGGwfK4AjkU5n+ryX2ruVy1jWUMs3wVrA0RC+t7
 7x3Rx/yiScILCA2/WfKgcFtlq1boRahpxtiw47IZYyeKfRzcr/Bcd4cWGFMWNtaWS5cDYOmd
 4YBDOQPMulWoIfgp1UAswWzBQeuC+zzxTFFnWP20K4g3ug9DQ3L0g4tEtQTu3rUttX1M6ISX
 PiywqbSwjTDbvZW1ing44XWdRAhuOyMUqx0ccrQz0kkCgTIjlCKpo3qPjOV0/oCv3KH4OpnT
 OKvlnAoqwVwojip3coskJfGiZ8Vyl/e6SV12po6Jdq9SENiZ9OvDZRfuT2AOYRsXsMiX39nu
 Dw8yrAet5C2fDUGxIgnyhDQZfKKcoyF7xb/WOuSPzp2hHxodr29ihuv/0WtxO7xWteo3VpWo
 SRIjMfBu3EC2RHc98SKS/1w9Vqv1zaI0gDc8OBEIUYsmKrUKp4h3r4wlocIvkjZAiD2n0D2g
 amLfUsn4uil8/nrb7f6qpOGKoN4lhvyPrkwlsG7G+g0LxYCUmeD9em91bDv51D1TbZIg/Esj
 KXUvp7XKd4Zq6O3BQJez5gu6xKiDze9zNQXg2MHIk9EeBOGkYfkI03CLfblBvmlmVusii1kx
 /XeM73hHJrNKn/Dna/8fbZm8kJc0w8zzcxH555NF74OPvbzWk7vtNPGFB84MxW4z/v5BNhyz
 I8eXGOPAqqHP6zOq1CI/f4vL/OQa48SvTbxM/kl5/jwgn8lgVIRYKuk0YcNZHylAvhqOViVb
 WToj9sbDGsGoAUzQPTviFKYUD5TY3iyX7g75jE+EI+mCJ3MR4+sgLyEwii3BIFZZmdfClCRD
 3joc4SEW/EXZSKIPMBujzwEWqK9S4M7yR6uswr6x6JhLuXP4iIYr47s1MBp5+3PkhE/7SF4A
 9yH026RV2F0gn8IRzgu0a9jukN90EmM0Kl/g/xGC9Ne/O9GUgYhNZHAyOx2Ecz9WgXEftuRV
 VmmQdSmATQpQ9wr39IAfltzF825jhDb0SqnG6UbmqCWCpIp6q7cxGDxJ8hlxHbGyqYhi14mT
 9NXO2O/nqBx+FubO4mc2X+emKuwPYER2DLK8i/D12+DuF1fVk86GfHfWH0bekzQhd/4/UjFC
 bSpDOJjekFtwNCPYolNatvyilMOBPvqJtncZCP70zO8DAiFxLSkZ4vxcGJb1yLYXgxM2Qwe4
 16KLQQ9QCC7rCiWWDBjC3rje0/vt+5kpyX/BgUPxgSSZlFhn4K49xUVmLTcUOse07kEpWZr4
 2FvBFuw1NLLI9GBvAFmOq5bZIV5qFxH2GTTsEptP5WIKL1+glkZcEJ8uEa9+Q9wD9BlmNgrq
 X5i5w1rNK+c0VUJIzaRzZH1N/vcLXDv8RSubYbM113e2crQ8aAKvqdr427/tR2kQxJxu05s1
 MNYhiP0DujiEl9LAtr4B1ws/l1hp7iAP3dur4iBz3BoOLm5vnjY1tYgFOYpmV6sKs1SPKONC
 En5FMhJYqrPM7kvml2kZRsfILoKrf9sYJr3JLOajff1bbs6xW//0TsUheIc0heC7S15GODS2
 JNX2fyA2hrCSzCul0qrsM38hdJfeDsQE2GjmyTtGIMDaqRucNQOBHy1KsK6g9Ry1ZvgQGVR+
 1PmBl9Uva3IMQaVPUD6jwZbh08c83mmnCb90TFuiDAuqPia3SnDhvrvbgEGPWgZT25kkVrqP
 c2onsgXDkOvaRQkkgfg+Vfiyvo+xsU3JG7VXUpOY23qNWhkSaW9rP+PZcVI5JIuvndcV+Kgb
 FaGD6Lnqh0B3S75WmZYzzc3djel6PCb/lR7jieHIW53vT/FfpNxw1LF6cTYVLtN2WgcSSd1k
 zTRDVWnbcOk89iOmpHIv/vtMgDAXJ0GYybb9qqZ9wmk2Gg2MRewz6yDw9HqNVAI6iP674VFf
 gGX8ASlRdOs/aagALc0GysJDgr+wdJoQ6FYi5Nu3LgLhmEkxZeJ02JbkmvZI/NJhazXbHxRF
 GUInN753zru4FZlaSyj9oHednzGhc0iR4Hnbk0mmQwg8u1EbcX17LV4hHdYmn2j9lrsS8N4x
 Ss3lbwwzWMg07wZlhIGzTzeKLVPHnZzMzLJzx2y1OqT66hFYFyMY7mSiVdhvYubJbO5kwVhQ
 RObG5h9QhAh5MNjEVvO20To5o27Kefid5UokCekgTOYgcEAe9oh0/sQgix/Pnjh+Gco0PM/k
 Ul21Iqh7+BueQ9FpMpRcTYAchHwbtkYwDzmirxSzI6OioaoGJglUi0AQM6xFarsWCkK4PLgM
 QvKWDQwqXaeTKCbVQaE8EkztTaHCY2iYkLXa3UcxM9pAQOALWRFjQwUVSl8lZk8WUTinZS9I
 BomuGhQ0xr+rR1BogoJHxD7VmGapQGibSoyRN2dahFX9QpFoUzSNJ/W9fp9ShlR5YbptwmRM
 iqebgVMA3sOXxmBDkDiO7To7t7Z7uWcB+yWNfbCYbyS7+dZUqTA3oqhh61h+TvELcCTJj9iA
 vk8j1JERmx8Et/Fli8nVjdSjCfRd4uXqRO95CBtv4XmqqitXQPq6ICGEKcUOtJqqFi6gqaGY
 vaZnz0xaS1Z2ZUF2WLSxfAB0UQThSBje3jlEbkJuSPXCqOFsrVaAx4Sd2V4M84bp6451xNGb
 NbSkci9lqYtiPkzBh9JVEe0nMavaI1CLzSnOV/HDVrNP7ODTdHS6/n+erj0CbRUiOMRthu6v
 iudHwrheD+CjDLuERuoNLMEgCaeNR1Y8IazF3QlQW3qXNPjZ1uwN8ZqgDk3x5Uui3/KPHJaO
 j95O09Atbyf6ypEj+43QjMZqCM9a7DeymDAtbWQI41z07MjGilukuNG/Hk2g6BY6i1JXr09m
 SffqMJvv0Dzl+COzjR9Vx8dz1QDzImPvEhkJeDY7swZBi6CpUpLtD/BTU1Q+4gtENDktqFOx
 8Kanab3LnJD9djT4M0QQsqSI8OcO31nOh3sS1u2REMISyCmMWbHiglTivaXozeQp4g7r5Wqk
 50XV7JfVVodDP4WCk15WtcFJd0kO1Fs2a7el8MO6Xek+VPUTdVbu52BXfWLHPjrKTCxl7RIZ
 xIUh7j/KM5AU++zk1wnYV58koPQHkPWVt0YuTVvWQgypFQeuGg7VGA43FjpLx+8+HJGX+Dhh
 QY400EtBIZlvCep+Vo8IUDG4Tc9gFVk08uwmiifKXbwNPviBN8MTXCo6A5obMu9G0EvNEWzh
 RA2bWmbAeIAyeMmLScy12q+8dNOAaIOFPMUJkVMmLfPIa1viAsUqz37lxYYva2cUd07zltsK
 dn28zpBw14xNYFtY/WIYvMTnx4IwfjS203gnuEpnF1HeB5LqTvUIXVT/hRPbOVuJjL0rLUzs
 krbymcFIC5UEKN06vNyqhFka7nGknKmg+IFexv2bLb6TevRunCcx5fZGBVqixtOzREVu+Evi
 48iaxbGDhF1iuHMT1JTbZqEcFgOJ8tKqCqKLHzI67iRh8ssZcPlUbqwKI3G/KcM3hD9TVZvR
 cJVtJRHRt71gAnZNZu1de9Vj090olytfBLfVbxIYE7ZyWZX5Zvjkdknhc8FYWtFZAc1eSSvu
 uSO919s26XFBYtqJC9dBNNMN2pqCpTrymgA5S4GVWPvlLpeklHnjXe0pyLbCCTwYoh4fPnPI
 wlhBNG/5XM09K3+zFfT9t+2y3jSE9Nkt5eP4OceodOGDflTUL9x9kWakIhDTHvsWGnKQ5azI
 NDrZo8gYMaRaD7yW0Gjiz8zU8b6Pcq8ZqmOjwbyQI9IsY6dlDk9PM65HzsaFl9+veYGrK57Y
 AQCZdI8b3uK/0wmMLejJQ6Dzti0a3u2bCBTU+Eawei+ZqJa1TtpN7Xmjn4hS5wxxvKrt0UKQ
 dBCjx3Tw+qie5gLUSX3HS84GU2Hri44mm59c+cql75gmFWR6R9FaWHNLbY1OyRestoxBE2fO
 yB/Dm4+Al6ah43e5ALp3fYZ/jBWmJBf1ugW1Rq29pLZfj+oX7Sm7JvPtC91J90vs6x0PMroJ
 dKbuZffmBTFQZXQuxHDWym/Xak/+JAYMGdDTf9ElHtwc9QBopZE4FEtW90WOqcKE6wwvvWmZ
 D5lEyMI3GpADd/G2T0Hj+y1wKecnRCVOsdHUlRMoNBJhd0TVDRzayUVqfq4T4vhkGmAWwDjw
 S8O9kFX4R4Y04N9euH77ZDZCcYWl3hTovd5WSLQCt9j8F6pEgl+bnDiRfmhmvDv1gVXnquE7
 w==
IronPort-Data: A9a23:vzodvqrhYRujVWWIBMC26FaBqK5eBmK9YRIvgKrLsJaIsI4StFCzt
 garIBmHaf2NNGf2e4t1b4rj8kwC7cWGx99kGQNuqCk8QywW8JacVYWSI27OZB+ff5bJJK5FA
 2TySTViwOQcEieEzvt4GuS59RGQ7YnRG/ykTraCY3gtLeNdYH9JoQp5nOIkiZJfj9G8Agec0
 fv/uMS31GWNglaYCUpKrfrYwP9TlK6q4mhA7wZmPasjUGL2zhH5MrpPdcldEFOkEuG4LsbiL
 87fwbew+H/u/htFIrtJRZ6iLyXm6paLVeS/oiI+t5qK23CulQRoukoPD8fwXG8M49m/c3Kd/
 /0W3XC4YV9B0qQhBI3xWTEBe811FfQuFLPveibn6aR/ZqAJGpfh66wGMa04AWEX0rl9W2JH6
 9ZIESEITiCajdC2+K66aeY506zPLOGzVG8ekmx7iCrcEe5jTZ3HQrvH/84dhW1swMVPGPvVb
 tEFLzFoaXwsYTUWZQ9RUcp4xb35wCClL1W0q3rNzUYzy23a3A103f7mN8PId9iLQ+1Pk0qYr
 36A9GP8av0fHI3Pk2vUriL87gPJtSDkYLgwM/6dzdNJuAfQnGMyVCMVbVTu9JFVjWbjBosGd
 RN8FjAVhaM47kG5Scfwdxa5u3WD+BUbXrJt//YS9RrI0a/I+0OVC2wFUDNbeZl/7pNwQD0v2
 1SAhM+vDjtq2FGIdZ6D3uie9zC7JwJFFH4HRR8dRzkF28Xig45m23ojUe1fOKKyi9T0HxT5z
 DaLsDUyit0vYSgjiPrTEbfv3GzEm3TZcuImzl6IAT/9v2uVcKbgNtP4swGKhRpVBN/BFgHpg
 ZQSpySJAAkz4XyliTfIW+ARBPSl6vKELjDGkBs2RcVn8jWo9nWqZpwW6zZ7TKuIDirmUWC2C
 KMwkVkKjHO2AJdMRfIqC25WI5lxpZUM7fy/CpjpgiNmO/CdjjOv8iB0flK31GvwikUqmqxXE
 c7FIJv1USZAUv82kGLeqwIhPVkDmXtWKYT7GsmT8vhb+eP2iIO9FO5faDNikMhnvPzsTPrpH
 yZ3apbTkksAOAEPSizW9IMYJEsWZX48H4z248VSe/SCKQMOJY3SI6C5/F/VQKQ8x/49vr6Qo
 BmVAxYEoHKh3ievAVvRNRhehEbHBssXQYQTZnB3Yj5FGhELPe6S0UvoX8dvIuB+pbAynaYco
 jtsU5zoP8mjgw/volw1RYPwoIxraFKgggfmAsZvSGVXk0dIFlOVqOz3NBDi7jcPBSeRvM4z6
 e/onADCTJZJA0woAM/KYbj9hxm8rFoMqtJUBkHoG9h0fFmz0Y5ILyeqsOQ7DftRIjr+xxyb9
 T2sPzEmmcf3rbUIreb53ZK/k9/xEs9VPFZrIG3A3LPnaQjY5jWCxKFDYsapfBfcdn/FyKW6b
 9po0sCmYe0Lo2hRurUhOr1u86E3yPW3g7p80A5fPW7qam6zAehKOUi23shot4xMyIRGuACwZ
 Fm9x9lCNZiNO+LnCFQ0Jjd5Xt+c1PoRpCbe3c41LGr++iVz2riNCmdWADWhlw1fK+FTHL4+4
 OJ8pvMT1ROzuiAqPvmCkCpQ0WaGdV4Ecqc/s6AlEJ3ZsRUqxn5CcK7jJHfPur/XUOp1M24uP
 jOwr4jBje4FxkP9LlwCJUKU1u9Z3ZkzqBRGyWEZHGuwm/3HuK4T/AZQ+jEJXAhq3k158+ZsC
 FNKaWxxB4uzpglNuuYSflqCOQ97AD+hxnfQ0HoMzW3QcFmpXDfCLUo7Iue8w3oa+GN9IBlep
 bGV52L5Yw23fsuq8HIAZl59of/JXO5OzArzqPm6Lp7UA7g7fjvXrau8bkUYqxbcIJ0QhxTHq
 MZu4NRLQ4HxbgBJkZIrE4Ob9+U0YzKVKFNSRcpO+PsyIljdXzWpyB6yK0yVUeFcFczgqEOXJ
 ZRnGZNSak6Yyi2LkAE+OYcNBL1FxNgS+9sIf+LQF14s6rewgGJgj8PNy3LYmmQuftRJlPQ9I
 KP3cxapMDSZpVlQql/3gPh0AEiKSvhaW1Skx8GwyvsDKLwbuuI1cU0S7Kq9j0/IDCRZpSCrr
 CHxTI6I6dw60ol9vZreIoMaDSWOFN7DfuCp8geyjtdwUe3yIfr+7wM7lnS3PiB9H6cgZNBsp
 LHc7P/1xBzkuZg1YUD4mr6ANfBC9JygVtptLebIFn1Qsi/Yav/Ovj8/1nGeAsFVmewE4vj9a
 Rm0Q5ayf4RNWvN25n5cWw5BGTkzVoX1aabBo3umjvKuUxIy7y3OHOmFx1TIM15JV3YvEIKkL
 A3NoNO8z4p8gKUQIQ4bFtd0B5NcC33ya5sMLtHem2GRMTi1vwmkpLDnqysF1RjKLXuhS+PR/
 pPPQ0nFRiSY4a3n4olQjN1vg0cxEn14vOgXe3Ad8f5QjxSRLjYPDcYZAKU8JqBkqA7A/7CmW
 2iVd0onMzv3Yhpcexal4NjDYBaWNtZTBvjHfA4W72GmQAboIrPZDLVYo3Iqpz88fzb41+ioJ
 O0P4nC6bFD73phtQv1V/fChx/tuwvTB3H8T5EThiIrIDg0DBakRnmlUdOaXufcrz+mW/KkKG
 YQ0eYyAaFygDFX0Dd4ld3NRGQ8UpiKqlm90KyKOxdvUsp6HiuZHzZUT/g01PqIrNKw3yHwmH
 RsbhFdhJ0iM13ATsLdvsNUs6UOxIezeBdC0dccPWiVL95yNBq8b0w/uUMbBoAzOOOKSLr8Fq
 gSR3g==
IronPort-HdrOrdr: A9a23:KYQam6Pdha2mecBcTvOjsMiBIKoaSvp037BN7TEIdfU1SL3gqy
 nKpp8mPHDP6Qr5NEtQ++xoQZPtfZqjz+8S3WBhB9eftWDd0QPGEGgF1/qE/9TOIUPDH4VmpM
 RdmsZFebjN5RATt6zHCEPRKbsdKJftytHNudvj
X-Talos-CUID: 9a23:Wx1cCWON3ecPSO5DByM990cKMJAcY3j9/G7aEU+AFWdKR+jA
X-Talos-MUID: =?us-ascii?q?9a23=3ACyAWzw3GipD9JXNtK/YJ15R82DUj4LuWMmIq0rA?=
 =?us-ascii?q?9luq8b28qNBbBsjePTdpy?=
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-AV: E=Sophos;i="6.03,162,1694736000"; 
   d="scan'208";a="98299488"
X-IronPort-Outbreak-Status: No, level 0, Unknown - Unknown
Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124])
  by alln-inbound-j.cisco.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Sep 2023 13:05:47 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
	s=mimecast20190719; t=1695215147;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=zgXSAq+JwUm4Xbg84X1deL0ZSdiP4GCL74z1O2/BjrY=;
	b=jBpz1HNqp60As7qV+jtFp4kdBzNz4fAhn493k7hDWZcPsn/6BYkOHa3/n+H05dFX+FuuEf
	haTq4EPOs30UBSm3tB4/vMuR85xFpCkq+Bm5kWCfvM0aRSRE/ezlRKziJ5uwH3RAjSYlUq
	g77W9bntIajcQsdd79VZr/SNRXzL0fA=
Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73])
 by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 us-mta-61-xigv2NFtPsi60ZNTfEe17Q-1; Wed, 20 Sep 2023 09:05:42 -0400
X-MC-Unique: xigv2NFtPsi60ZNTfEe17Q-1
Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(No client certificate requested)
	by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 58A38280D207;
	Wed, 20 Sep 2023 13:05:40 +0000 (UTC)
Received: from [10.39.208.35] (unknown [10.39.208.35])
	by smtp.corp.redhat.com (Postfix) with ESMTPS id 87CCC1686D;
	Wed, 20 Sep 2023 13:05:38 +0000 (UTC)
Message-ID: <dd2dd416-f1f9-f779-4f6a-105fd6a7ab6c@redhat.com>
Date: Wed, 20 Sep 2023 15:05:37 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.13.0
Subject: Re: Commit broke 32-bit testpmd app
From: Maxime Coquelin <maxime.coquelin@redhat.com>
To: "Roger Melton (rmelton)" <rmelton@cisco.com>,
        "Dave Johnson (davejo)" <davejo@cisco.com>,
        "dev@dpdk.org" <dev@dpdk.org>,
        "Sampath Peechu (speechu)" <speechu@cisco.com>, chenbo.xia@outlook.com
Cc: "Malcolm Bumgardner (mbumgard)" <mbumgard@cisco.com>,
        "Chris Brezovec (cbrezove)" <cbrezove@cisco.com>,
        David Marchand <david.marchand@redhat.com>
References: <MN2PR11MB45814151ACC8381ABFB1DCB9BD059@MN2PR11MB4581.namprd11.prod.outlook.com>
 <63486764-3b44-3299-6830-05435dfd78f3@redhat.com>
 <MN2PR11MB4581331F5C263CE604B44508BD049@MN2PR11MB4581.namprd11.prod.outlook.com>
 <827f912f-fc2d-6d41-ba8c-e7f3f9f2e24b@redhat.com>
 <MN2PR11MB4581CA6E374DFB974E6163CFBD0D9@MN2PR11MB4581.namprd11.prod.outlook.com>
 <MN2PR11MB4581BCFFA0FD8FF9B128F35EBD1B9@MN2PR11MB4581.namprd11.prod.outlook.com>
 <MN2PR11MB45816D7BD72CEA1C9EDC6A99BDCE9@MN2PR11MB4581.namprd11.prod.outlook.com>
 <MN2PR11MB458198A11C9CDA27C69FBA37BD1FA@MN2PR11MB4581.namprd11.prod.outlook.com>
 <MW4PR11MB65707E45F173E52B799E5DA4D3EFA@MW4PR11MB6570.namprd11.prod.outlook.com>
 <ae4c8aaf-6c12-8ddf-90b9-76e76564728d@cisco.com>
 <b01d2744-982a-f4b7-8fb3-48ee9ecacd05@redhat.com>
In-Reply-To: <b01d2744-982a-f4b7-8fb3-48ee9ecacd05@redhat.com>
X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Language: en-US
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Outbound-SMTP-Client: 173.37.147.240, alln-inbound-j.cisco.com
X-Outbound-Node: alln-core-10.cisco.com
Return-Path: maxime.coquelin@redhat.com
X-MS-Exchange-Organization-ExpirationStartTime: 20 Sep 2023 13:05:57.4519
 (UTC)
X-MS-Exchange-Organization-ExpirationStartTimeReason: OriginalSubmit
X-MS-Exchange-Organization-ExpirationInterval: 1:00:00:00.0000000
X-MS-Exchange-Organization-ExpirationIntervalReason: OriginalSubmit
X-MS-Exchange-Organization-Network-Message-Id:
 5f7c47b1-fc6f-4213-9f43-08dbb9da547c
X-EOPAttributedMessage: 0
X-MS-Exchange-Organization-MessageDirectionality: Originating
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic:
 BL02EPF0001A106:EE_|LV3PR11MB8577:EE_|MN2PR11MB4190:EE_
X-MS-Exchange-Organization-AuthSource:
 BL02EPF0001A106.namprd05.prod.outlook.com
X-MS-Exchange-Organization-AuthAs: Anonymous
X-OriginatorOrg: cisco.onmicrosoft.com
X-MS-Office365-Filtering-Correlation-Id: 5f7c47b1-fc6f-4213-9f43-08dbb9da547c
X-MS-Exchange-Organization-SCL: 1
X-Microsoft-Antispam: BCL:5;
X-Forefront-Antispam-Report:
 CIP:173.37.142.88;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:alln-iport-1.cisco.com;PTR:alln-iport-1.cisco.com;CAT:NONE;SFS:(13230031)(4636009)(286005)(451199024)(82310400011)(53546011)(31696002)(83380400001)(7636003)(7596003)(356005)(84300400001)(86362001)(36756003)(2616005)(107886003)(6266002)(426003)(26005)(110136005)(54906003)(30864003)(44832011)(8936002)(8676002)(1096003)(4326008)(336012)(31686004)(58800400005)(966005)(42696004)(43740500002);DIR:INB;
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2023 13:05:57.1394
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5f7c47b1-fc6f-4213-9f43-08dbb9da547c
X-MS-Exchange-CrossTenant-Id: 5ae1af62-9505-4097-a69a-c1553ef7840e
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5ae1af62-9505-4097-a69a-c1553ef7840e;Ip=[173.37.142.88];Helo=[alln-iport-1.cisco.com]
X-MS-Exchange-CrossTenant-AuthSource:
 BL02EPF0001A106.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR11MB8577
X-MS-Exchange-Transport-EndToEndLatency: 00:00:07.8298834
X-MS-Exchange-Processed-By-BccFoldering: 15.20.6792.025
X-Microsoft-Antispam-Mailbox-Delivery:
	ucf:1;jmr:0;auth:0;dest:C;OFR:CustomRules;ENG:(910001)(944506478)(944626604)(920097)(930097)(140003);
X-Microsoft-Antispam-Message-Info:
	=?utf-8?B?TWJFeVFNczNKRzdZNWhubHc3dEVWc2tjeE1UcS9uMTVDY2JPcUtLMlROMGJw?=
 =?utf-8?B?Uk94cDh6dGx6aTQwN3dab2FuL21saXc0eE9sVjY1TnJvbnlhV0VGOXYyQ3Zs?=
 =?utf-8?B?VjkxYldtS2tidmtodGk5ZVM3UVF5OExkYWxoMzhxZkJVRTZXTXhWUTUzV1Nm?=
 =?utf-8?B?aGpoTGtFN2J6eTEyQ0lNamgwZlVQb3VuSHEwY29Wbjg0RVFOTmxFREtIT0pE?=
 =?utf-8?B?SDJRTEJXaHZTOFVsd2pURjRjT0l2dWpuc0ZhVmJ1UHdBeTRqMmFGQW0xT2VG?=
 =?utf-8?B?bENTU1ZUTlRDUzF4ZHNkSmdHRnQxT29odHJJckh1RWk2Y3Bpenk5NEV2Rzg4?=
 =?utf-8?B?MTl5VjVlUmVYbW15NTNQZk42N3pwc2taM3J4cW53OWowOVRBTU8yTzY2RTQr?=
 =?utf-8?B?RlJJU3JJeEtsbndFQnVyVUdMMXo1WXFSL2V1ekVDNlVLTlR1YnF6OWRyZ1h4?=
 =?utf-8?B?U1VoTmRLSXdpaHZFZDlOeUxLeTRTeVFHclJXRlJYVUc4eE9PSlRSbklGR0JV?=
 =?utf-8?B?MEtsN0JLcVpUQkM4RzVRcUdnWVlBODVKKzNsTEgxTzFGY05aYlJUTmt3Rit0?=
 =?utf-8?B?MFFBZmk2MGhONytlcHpHU3Q1eFNJUldUK2ZndCt3b2ROMCtrQWgyNDhRRHN2?=
 =?utf-8?B?ZHhzMUs3VWtrUlI1aWhBUzRrNTdkWTB1blRUNGlZbjEvMTB6ejFoZEM0d21r?=
 =?utf-8?B?ZTRPZ05Sc3FoekVva1lOcHFEblZsNDM0TndRNThTeXVGSlY0SitiSUszSzcw?=
 =?utf-8?B?SnNFVm4yRldiMzhEakxUK3hnQVpnWVVSWnZqRnZRaFpQVTROSXBoL1hsNmxa?=
 =?utf-8?B?cHpyalFUc01XYlp5RWtycGxnQjZVOWxFOG5UWmJ5emx4d3BIRmRRRHhPTHJR?=
 =?utf-8?B?N0hLYzVOaDM2U091QzhmTjA5V09kNzQ0TzNkTXJPVXJaL3FQazJWM3FHQjN0?=
 =?utf-8?B?bkcyQnUrTHM3Ri9rM0wzcWcyZ2JlNFlJSHlRdUhrOXVNZytlWDREV25LSE10?=
 =?utf-8?B?azcwVEl3U2RRTnc0QlVBVnUyTHBEdDR2RERkYmxGMzVUbFp2czZoWGpCdFBP?=
 =?utf-8?B?V2RYc21GWmpVa0hXbjN0TEk1cklvTGxFbjg2eTZFVG1ScnRWWFBrL3VmUmJI?=
 =?utf-8?B?N0dWbStnNmx5NXk2cmRvMXVXU2NBdDZpRzdOMmJ4KzhZZHlhRTJJN1Z4d01q?=
 =?utf-8?B?L01KWENrS3BseWEvTXlhSHp1Z2FRT1VVaVVSR3FtM2tGcFJMemlGZHNwUi9x?=
 =?utf-8?B?akxhWlJNWDFabFdRZVNkUE56a2tvSGd1R0lnbWhyTjBVclRoYTJIMDlFWXAx?=
 =?utf-8?B?TjhNT2FqNDJJYkxvMHZRdnpCY3RXaUVzUHRPd055NFN4T2lHL0k1SkJMcmxw?=
 =?utf-8?B?TEU1WTFhM2RNTHdmc0ZrWFNOWDd5R2ZwNCs4YzFSM05FUXZJbkRCS0MvczlR?=
 =?utf-8?B?N05qSHRTTjNkNWRvSGhGMmUvcXJlc2hzVHBuSmZIMXB5ejY0VE5abkhob0FM?=
 =?utf-8?B?dXJhOGZiU01HUkN5dFpkbzEyNmhydU9yTzcxMkxzdEFQd0s5R1Vrem5QN3JQ?=
 =?utf-8?B?eWdsUWdMMWZUL3FadDd4UzkzM0ZvRWgyMjNHZmNpek50UnBIUGcyb2VYV1lE?=
 =?utf-8?B?RlpMZEZFOWZ3QTlNSFlFRXE2Sm9rZkxGVDdvK1hkay9lQzltbjByaStJNjF3?=
 =?utf-8?B?TEtXbkQrTmlPZk0rQ0g3a2dCdTNmL2hZbFJKZHdPR2czNEF3cUxIdVI4Uno4?=
 =?utf-8?B?cHUwR1d1ZWNDZjBzNXpueVBtUGVReG1GNjBYMm4wNjR2OFBoUWZRZ2kxQ3Zl?=
 =?utf-8?B?U09qLzIrVHI5elVTTVF0TlVub3BrT05HbExuamgyemZnMnUzY0lJa005STB5?=
 =?utf-8?B?Wlh6Znc2NnJmQ3BOUlQxOFRWbEVGYW8xa2JjVHk2YU5JL2dDSFpJY3ROQng2?=
 =?utf-8?B?cEhiYWFmTWdDK29YdVFHUmVxa2RsYkVqbG5wcDRBYlBTSHZPeThiOVp1VDZu?=
 =?utf-8?B?bEFDSFVSVHhiTmZTcWRXQnN1Ly9KcFNpRVVSK0Q5bGtRNEE5a0NRZkt2R3Av?=
 =?utf-8?B?WGEvNitMSWF4RnNPaVVnRjVFWVprVnlnYXdQREVLMFpXdFNrVE1Ed1I1cEVv?=
 =?utf-8?B?MnpaZFQ3Y2Q0Tng5R2R4WElqKzlVQVNpWEg4RmxSRDA1eEZjV21ldk9HcVV3?=
 =?utf-8?B?eWUwQlB0TldVQXIrVUZyRThkendhRmkvcE1RTlNSSHBwMFlJRFIxWlFEOHpr?=
 =?utf-8?B?R2x5S3c1NHR3OVd6NTlQZURsTXFhQjZ4WmV1ZTVVSk5SUEtrb0NEUW5tQ1I5?=
 =?utf-8?B?UkVZN0pjY2FGbDE2TUUvWDdSUVZwVEFhRDVyWVZ6cWZmODFKMktweXVmUnB0?=
 =?utf-8?B?S29pU3VpYi9jVDdSZm5QbzhnckVpb1lRVmY3ZDhjcS9WZUFpZTZ6enB0SXcy?=
 =?utf-8?B?akNnMlo4VFhybVpNanJ1QU9kdk5lcS9ONVo2eGtFUXlWakFuUmxyOFkwSlcr?=
 =?utf-8?B?dnJJUmR3U2l5YUY1VGI1OFkrTWI1SVA0SmdidTZWcTdnPT0=?=
MIME-Version: 1.0



On 9/20/23 09:35, Maxime Coquelin wrote:
> Hi,
> 
> I tried to reproduce without success(see attached log).
> 
> I fail to reproduce because buf_iova fits into 32 bits in my case:
> (gdb) p /x *tx_pkts[0]
> 
> $4 = {
>    cacheline0 = 0x77b19ec0,
>    buf_addr = 0x77b19f40,
>    buf_iova = 0x49519f40,
>    rearm_data = 0x77b19ed0,
> 
> However, looking at your report, something like this would work for you:
> 
> diff --git a/drivers/net/virtio/virtqueue.h 
> b/drivers/net/virtio/virtqueue.h
> index 9d4aba11a3..38efbc517a 100644
> --- a/drivers/net/virtio/virtqueue.h
> +++ b/drivers/net/virtio/virtqueue.h
> @@ -124,7 +124,7 @@ virtqueue_store_flags_packed(struct 
> vring_packed_desc *dp,
>    * (virtio-pci and virtio-user).
>    */
>   #define VIRTIO_MBUF_ADDR(mb, vq) \
> -       ((uint64_t)(*(uintptr_t *)((uintptr_t)(mb) + 
> (vq)->mbuf_addr_offset)))
> +       (*(uint64_t *)((uintptr_t)(mb) + (vq)->mbuf_addr_offset))
> 
> 
> The problem is that it would likely break Virtio-user en 32bits mode, as
> this is how it was initially implemented, and got fixed few years ago,
> as David hinted to me:
> 
> commit 260aae9ad9621e3e758f1443abb8fcbc25ece07c
> Author: Jianfeng Tan <jianfeng.tan@intel.com>
> Date:   Wed Apr 19 02:30:33 2017 +0000
> 
>      net/virtio-user: fix address on 32-bit system
> 
>      virtio-user cannot work on 32-bit system as higher 32-bit of the
>      addr field (64-bit) in the desc is filled with non-zero value
>      which should not happen for a 32-bit system.
> 
>      In case of virtio-user, we use buf_addr of mbuf to fill the
>      virtqueue desc addr. This is a regression bug. For 32-bit system,
>      the first 4 bytes of mbuf is buf_addr, with following 8 bytes for
>      buf_phyaddr. With below wrong definition, both buf_addr and lower
>      4 bytes buf_phyaddr are obtained to fill the virtqueue desc.
>        #define VIRTIO_MBUF_ADDR(mb, vq) \
>              (*(uint64_t *)((uintptr_t)(mb) + (vq)->offset))
> 
>      Fixes: 25f80d108780 ("net/virtio: fix packet corruption")
>      Cc: stable@dpdk.org
> 
>      Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
>      Acked-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
> 
> If my understanding is correct, on 32 bits, when mbuf->buf_addr is used
> (Virtio-user), we need to mask out the higher 4 bytes, while when using
> Virtio-pci we need the full 64 bits (as the physical addresses used as
> IOVA on the guest are 64 bits).

I posted a fix aiming at making it work for both Virtio-user and Virtio-
PCI 32 bits builds while not impacting 64 bits performance. Could you
please have a try and report feedback by replying to the patch?

Regards,
Maxime

> Regards,
> Maxime
> 
> On 9/13/23 15:24, Roger Melton (rmelton) wrote:
>> +Chris Brezovec
>>
>> Hi Maxime,
>>
>> Chris from our team is attending the DPDK Summit in Dublin this week. 
>> If you have some time available, we'd appreciate it if he could meet 
>> with you to discuss the 32bit virtio issue we are seeing.
>>
>> Regards,
>> Roger Melton
>>
>> On 9/6/23 2:57 PM, Dave Johnson (davejo) wrote:
>>>
>>> Hi Maxime,
>>>
>>> This email is regarding the following commit:
>>>
>>> https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657
>>>
>>> A query had been sent previously on this topic (see below) indicating 
>>> this commit appears to have broken the 32-bit testpmd app and 
>>> impacted one of our products that runs as a 32-bit DPDK application.  
>>> We consequently backed the commit out of our product but would prefer 
>>> to get a fix for it.  In the earlier exchange, you had asked if we 
>>> were using virtio-pci or virtio-user (we are using virtio-pci) and 
>>> asked for logs which Sampath provided.  It’s been a while, so let me 
>>> now if you need me to send resend those logs or need any other 
>>> information.
>>>
>>> FWIW, I reproduced this using testpmd and noticed that this part of 
>>> the change seems to be the interesting part (in 
>>> drivers/net/virtio/virtqueue.h):
>>>
>>> /**
>>>
>>> * Return the IOVA (or virtual address in case of virtio-user) of mbuf
>>>
>>> * data buffer.
>>>
>>> *
>>>
>>> * The address is firstly casted to the word size (sizeof(uintptr_t))
>>>
>>> * before casting it to uint64_t. This is to make it work with different
>>>
>>> * combination of word size (64 bit and 32 bit) and virtio device
>>>
>>> * (virtio-pci and virtio-user).
>>>
>>> */
>>>
>>> #define VIRTIO_MBUF_ADDR(mb, vq) \
>>>
>>>       ((uint64_t)(*(uintptr_t *)((uintptr_t)(mb) + 
>>> (vq)->mbuf_addr_offset))
>>>
>>> If I revert just this part of the changeset (by re-using the 
>>> VIRTIO_MBUF_ADDR to return buf_iova which matches what it had used 
>>> previously), then 32-bit testpmd is able to receive traffic again:
>>>
>>> #define VIRTIO_MBUF_ADDR(mb, vq) (mb->buf_iova)
>>>
>>> Looking at the address produced by each of these, I see the address 
>>> is the same except that the casting results in the upper bits getting 
>>> cleared:
>>>
>>> Address from patch (nonworking case) = 0x58e7c900
>>>
>>> Address using buf_iova (working case) = 0x158e7c900
>>>
>>> ::
>>>
>>> Address from patch (nonworking case) = 0x58e7bfc0
>>>
>>> Address using buf_iova (working case) = 0x158e7bfc0
>>>
>>> ::
>>>
>>> Address from patch (nonworking case) = 0x58e7b680
>>>
>>> Address using buf_iova (working case) = 0x158e7b680
>>>
>>> ::
>>>
>>> Regards, Dave
>>>
>>> *From: *Sampath Peechu (speechu) <speechu@cisco.com>
>>> *Date: *Monday, January 30, 2023 at 3:29 PM
>>> *To: *Maxime Coquelin <maxime.coquelin@redhat.com>, 
>>> chenbo.xia@intel.com <chenbo.xia@intel.com>, dev@dpdk.org <dev@dpdk.org>
>>> *Cc: *Roger Melton (rmelton) <rmelton@cisco.com>, Malcolm Bumgardner 
>>> (mbumgard) <mbumgard@cisco.com>
>>> *Subject: *Re: Commit broke 32-bit testpmd app
>>>
>>> Hi Maxime,
>>>
>>> Could you please let us know if you got a chance to look at the 
>>> debugs logs I provided?
>>>
>>> Thanks,
>>>
>>> Sampath
>>>
>>> *From: *Sampath Peechu (speechu) <speechu@cisco.com>
>>> *Date: *Tuesday, December 6, 2022 at 1:08 PM
>>> *To: *Maxime Coquelin <maxime.coquelin@redhat.com>, 
>>> chenbo.xia@intel.com <chenbo.xia@intel.com>, dev@dpdk.org <dev@dpdk.org>
>>> *Cc: *Roger Melton (rmelton) <rmelton@cisco.com>
>>> *Subject: *Re: Commit broke 32-bit testpmd app
>>>
>>> Hi Maxime,
>>>
>>> Did you get a chance to look into this?
>>>
>>> Please let me know if you need anything else.
>>>
>>> Thanks,
>>>
>>> Sampath
>>>
>>> *From: *Sampath Peechu (speechu) <speechu@cisco.com>
>>> *Date: *Wednesday, November 23, 2022 at 5:15 PM
>>> *To: *Maxime Coquelin <maxime.coquelin@redhat.com>, 
>>> chenbo.xia@intel.com <chenbo.xia@intel.com>, dev@dpdk.org <dev@dpdk.org>
>>> *Cc: *Roger Melton (rmelton) <rmelton@cisco.com>
>>> *Subject: *Re: Commit broke 32-bit testpmd app
>>>
>>> Hi Maxime,
>>>
>>> I’m attaching the following for reference.
>>>
>>>   * Instructions for Centos8 test setup
>>>   * Diffs between the working and non-working versions (working
>>>     version has the problem commit backed out)
>>>   * Working logs (stats show that ping packets from neighbor VM can be
>>>     seen with both 64-bit and 32-bit apps)
>>>   * Non-working logs (stats show that ping packets from neighbor VM
>>>     are seen with 64-bit app but NOT seen with 32-bit app)
>>>
>>> ============================
>>>
>>> $ sudo ./usertools/dpdk-devbind.py --status
>>>
>>> Network devices using DPDK-compatible driver
>>>
>>> ============================================
>>>
>>> 0000:07:00.0 'Virtio network device 1041' drv=igb_uio unused=
>>>
>>> 0000:08:00.0 'Virtio network device 1041' drv=igb_uio unused=
>>>
>>> Network devices using kernel driver
>>>
>>> ===================================
>>>
>>> 0000:01:00.0 'Virtio network device 1041' if=enp1s0 drv=virtio-pci 
>>> unused=igb_uio *Active*
>>>
>>> …
>>>
>>> ===========================
>>>
>>> Thanks,
>>>
>>> Sampath
>>>
>>> *From: *Maxime Coquelin <maxime.coquelin@redhat.com>
>>> *Date: *Tuesday, November 22, 2022 at 4:24 AM
>>> *To: *Sampath Peechu (speechu) <speechu@cisco.com>, 
>>> chenbo.xia@intel.com <chenbo.xia@intel.com>, dev@dpdk.org <dev@dpdk.org>
>>> *Cc: *Roger Melton (rmelton) <rmelton@cisco.com>
>>> *Subject: *Re: Commit broke 32-bit testpmd app
>>>
>>> Hi,
>>>
>>> In my initial reply (see below), I also asked if you had logs to share.
>>> And wondered whether it happens with Virtio PCI or Virtio-user?
>>>
>>> Regards,
>>> Maxime
>>>
>>> On 11/16/22 00:30, Sampath Peechu (speechu) wrote:
>>> > ++ dev@dpdk.org <mailto:dev@dpdk.org <mailto:dev@dpdk.org>>
>>> >
>>> > *From: *Maxime Coquelin <maxime.coquelin@redhat.com>
>>> > *Date: *Tuesday, November 15, 2022 at 3:19 AM
>>> > *To: *Sampath Peechu (speechu) <speechu@cisco.com>, 
>>> chenbo.xia@intel.com
>>> > <chenbo.xia@intel.com>
>>> > *Cc: *Roger Melton (rmelton) <rmelton@cisco.com>
>>> > *Subject: *Re: Commit broke 32-bit testpmd app
>>> >
>>> > Hi Sampath,
>>> >
>>> >
>>> > Please add dev@dpdk.org, the upstream mailing list, if this is related
>>> > to the upstream DPDK project.If it is using RHEL DPDK package, please
>>> > use the appropriate support channels.
>>> >
>>> > On 11/14/22 23:55, Sampath Peechu (speechu) wrote:
>>> >  > Hi Virtio Maintainers team,
>>> >  >
>>> >  > This email is regarding the following commit.
>>> >  >
>>> >  >
>>> > 
>>> https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657 <https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657> <https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657 <https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657>>
>>> >  >
>>> >  > The above commit appears to have broken the 32-bit testpmd app (and
>>> >  > consequently impacted one of our products that runs as a 32-bit 
>>> DPDK
>>> >  > app). The 64-bit testpmd app does not appear to be impacted though.
>>> >
>>> > We'll need some logs to understand what is going on.
>>> > Does it happen with virtio-pci or virtio-user?
>>> >
>>> > Regards,
>>> > Maxime
>>> >
>>> >  > With the commit in place, we didn’t see any packets going 
>>> through at
>>> >  > all. After backing out the commit and rebuilding the 32-bit 
>>> testpmd app
>>> >  > in our test setup, we were able to pass traffic as expected.
>>> >  >
>>> >  > Could you please let us know if this is a known issue? And if 
>>> there is a
>>> >  > fix available for it?
>>> >  >
>>> >  > Thank you,
>>> >  >
>>> >  > Sampath Peechu
>>> >  >
>>> >  > Cisco Systems
>>> >  >
>>> >
>>>
>>