From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 904DF45C4E for ; Fri, 1 Nov 2024 20:21:42 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7CB594026B; Fri, 1 Nov 2024 20:21:42 +0100 (CET) Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) by mails.dpdk.org (Postfix) with ESMTP id F16544003C for ; Fri, 1 Nov 2024 20:21:41 +0100 (CET) Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-a9aa8895facso360931166b.2 for ; Fri, 01 Nov 2024 12:21:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1730488901; x=1731093701; darn=dpdk.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=UgFal9Jxq8wCb5zXOEHPFP9q+M1GXr3QUM9lhoiJbbI=; b=lC+7EayzNlT6s5RAkL1RRdUwZKF/IdOrKV1u3IjMtXJVe7CAMM/Iho09TniKouLINN XAyZ0mzXmTlWTkHJQX/Xd+B0+92M3LzDOCplBNa8ALA2Azu8j1kG91Ntw2g5hcBuBVBL ZrbEoEUQmnABZwZSW5CWp8nveKCrZ6mo3nAIlvtUTqjRRZr5C2ex5okop9DG8/VBJvJn CliWNTurkM4vVR7P0oXBLFFPWypqumRMXegseY7b0n45ThSbYIPpA6ifVIzfILbzYLDc rEcL8t7w1xRqDxW4HJJ/nyj52kOcshjADiC4z6Lhjae9YyJFwQEvmEFgwhNxs+ara4mi D+xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730488901; x=1731093701; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=UgFal9Jxq8wCb5zXOEHPFP9q+M1GXr3QUM9lhoiJbbI=; b=F5i1Nf4//qL3fe6Fq1K2k+SXdf4s4JQG0DgJrIHT3Q/c1ONb2vCqKzx4zThMfd/4QO we73g8UaQ0Fd2nhDTd+vcbZOel4sW8eSfx5dhPperlYEbhm9Aaf+Qq3y8pg1cT4Cbyv0 7KiosTwTMRZk3sdHgS7H7uS+3j2Ux6S/fAlqSfOd7ciTYKkeWhtxCakIcqkM8ujE3DMc RNSlRH3CfG9IMFJ6PVEJJK+LV7+0XRB+d+owC40SQ+yFLl3Bx/lBpIBuT1QlzrPB2QEC 9SEjRpNnSvq9Drb0aghAwmQgqX5saXkfGMLQsZ74PL2uhBA66NZh0zXWiO1hVddMWaHG FacQ== X-Gm-Message-State: AOJu0YywcFxU2DY6LcN73vNLCt7F/D8kO5+B5+K8tbkqg99BNfhA1fmQ VQm425N1m+a7H/Au9/mIODxmrQ8HJlNy9OfYmNclCzcL93Zk25a27eHpGXVD0PRnKPApx+i7AL0 4X2qajEsIh21roIWOBOVks04wBk3InfxMZ0YV X-Google-Smtp-Source: AGHT+IHonqLpGr1Z9eL0BNsyZcSd8se+ZApvfVyFrHmSwd1ih/Jlgw20NlbZFENFRPKQrOCt4EisdM7d7xhY8KGDdes= X-Received: by 2002:a17:907:7b93:b0:a9a:b70:2a7c with SMTP id a640c23a62f3a-a9de5f3f87cmr2124421866b.25.1730488901315; Fri, 01 Nov 2024 12:21:41 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Rushil Gupta Date: Fri, 1 Nov 2024 12:21:30 -0700 Message-ID: Subject: Re: gve queue format on dpdk 22.11 To: Peter Morrow Cc: "users@dpdk.org" , "joshwash@google.com" , "jeroendb@google.com" , "jungeng.guo@intel.com" Content-Type: multipart/alternative; boundary="00000000000040b5a10625ded807" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --00000000000040b5a10625ded807 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Peter That's great! The queue-format is determined internally based on the VM instance. All gen2 (like n2, e2, g2 etc.) use GQI_QPL format. There is no way for the customer to configure it. On Wed, Oct 30, 2024 at 6:50=E2=80=AFAM Peter Morrow = wrote: > Thanks Rushil, > > I tested with 23.11 and everything is good now. Out of interest, what > determines the queue format on the device? Is there a way to change the > queue format such that 22.11 where gve support first appear could be used= ? > > Thanks, > Peter. > ------------------------------ > *From:* Rushil Gupta > *Sent:* 29 October 2024 21:14 > *To:* Peter Morrow > *Cc:* users@dpdk.org ; joshwash@google.com < > joshwash@google.com>; jeroendb@google.com ; > jungeng.guo@intel.com > *Subject:* Re: gve queue format on dpdk 22.11 > > > This email originated outside of your organization, do not visit any link= , > open any attachments or provide any sensitive information unless you > recognize the sender and are certain the content can be trusted > > Hi Peter > DQO_RDA is supported 23.07 onwards. > > On Tue, Oct 29, 2024 at 10:09=E2=80=AFAM Peter Morrow wrote: > > Hi Folks, > > Reading the docs for the the 22.11 release it clearly states that the > DQO_RDA queue format is not yet supported: > > https://doc.dpdk.org/guides-22.11/nics/gve.html > > I'm attempting to bring up our software router on GCP (VM instance type > c4-standard-8) where we are currently using dpdk 22.11 (via debian, with = a > 6.1.0-26-amd64 kernel), given the lack of support for DQO_RDA I see the > following expected messages when I start vpp (our dpdk application): > > Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_adminq_describe_device(): > Driver is running with DQO RDA queue format. > Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_adminq_describe_device(): MAC > addr: 42:01:0A:07:00:0F > Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_init_priv(): Max TX queues 2, > Max RX queues 2 > Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_dev_init(): DQO_RDA is not > implemented and will be added in the future > > As a result vpp sefaults shortly after: > > (gdb) bt > #0 gve_adminq_create_tx_queue (queue_index=3D0, priv=3D0xbc03a59c0) at > ../drivers/net/gve/base/gve_adminq.c:502 > #1 gve_adminq_create_tx_queues (priv=3D0xbc03a59c0, num_queues=3D2) at > ../drivers/net/gve/base/gve_adminq.c:516 > #2 0x00007ffda6c9c5d6 in gve_dev_start (dev=3D0x7ffe1f4f7240 > ) at ../drivers/net/gve/gve_ethdev.c:181 > #3 0x00007ffe1f489b6a in rte_eth_dev_start () from > /lib/x86_64-linux-gnu/librte_ethdev.so.23 > #4 0x00007ffff584aa14 in dpdk_device_start (xd=3Dxd@entry=3D0x7ffe204fba= c0) > at ./src/plugins/dpdk/device/common.c:393 > #5 0x00007ffff584c598 in dpdk_interface_admin_up_down (vnm=3D out>, hw_if_index=3D, flags=3D) > at ./src/plugins/dpdk/device/device.c:485 > #6 0x00007ffff6ab3ccd in vnet_sw_interface_set_flags_helper (vnm=3Dvnm@e= ntry=3D0x7ffff7f6b660 > , sw_if_index=3D, > flags=3DVNET_SW_INTERFACE_FLAG_ADMIN_UP, helper_flags=3D0, > helper_flags@entry=3DVNET_INTERFACE_SET_FLAGS_HELPER_WANT_REDISTRIBUTE) > at ./src/vnet/interface.c:545 > #7 0x00007ffff6ab489f in vnet_sw_interface_set_flags (vnm=3Dvnm@entry=3D= 0x7ffff7f6b660 > , sw_if_index=3D, > flags=3D) at ./src/vnet/interface.c:601 > #8 0x00007ffff6accee9 in vl_api_sw_interface_set_flags_t_handler > (mp=3D0x7ffe3f241ff8) at ./src/vnet/interface_api.c:100 > #9 0x00007ffff7fa6e0d in msg_handler_internal (free_it=3D0, do_it=3D1, > trace_it=3D, msg_len=3D, > the_msg=3D0x7ffe3f241ff8, am=3D0x7ffff7fb8f40 ) at > ./src/vlibapi/api_shared.c:580 > #10 vl_msg_api_handler_no_free (the_msg=3D0x7ffe3f241ff8, msg_len=3D out>) at ./src/vlibapi/api_shared.c:652 > #11 0x00007ffff7f8ed7f in vl_socket_process_api_msg (rp=3D= , > input_v=3D) at ./src/vlibmemory/socket_api.c:208 > #12 0x00007ffff7f978d3 in vl_api_clnt_process (vm=3D, > node=3D, f=3D) > at ./src/vlibmemory/memclnt_api.c:429 > #13 0x00007ffff6853966 in vlib_process_bootstrap (_a=3D) a= t > ./src/vlib/main.c:1223 > #14 0x00007ffff67ba03c in clib_calljmp () at > /usr/src/packages/BUILD/src/vppinfra/longjmp.S:123 > #15 0x00007ffe1edfcd90 in ?? () > #16 0x00007ffff6855054 in vlib_process_startup (f=3D0x0, p=3D0x7ffe2078f7= 80, > vm=3D0x7ffe1fa007c0) at ./src/vlib/main.c:1248 > #17 dispatch_process (vm=3D0x7ffe1fa007c0, p=3D, > last_time_stamp=3D, f=3D0x0) at ./src/vlib/main.c:1304 > #18 0x0000000000000000 in ?? () > (gdb) > > The segfault occurs due to the complq pointer here being NULL: > > cmd.create_tx_queue.tx_comp_ring_addr =3D > cpu_to_be64(txq->complq->tx_ring_phys_addr); > > It may be possible to upgrade to a more recent version of dpdk which > should help me progress, though before I do that I wondered if there is a= ny > other way to make progress here? Specifically, is there a way to force a > different queue format in the device? This is a c4-standard-8 VM running = in > GCE. > > Thanks! > Peter. > > --00000000000040b5a10625ded807 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Peter
That's great!
The queue-format is= determined internally based on the VM instance. All gen2 (like n2, e2, g2 = etc.) use GQI_QPL format. There is no way for the customer to configure it.=

On Wed, Oct 30, 2024 at 6:50=E2=80=AFAM Peter Morrow <peter@graphiant.com&= gt; wrote:
Thanks Rushil,

I tested with 23.11 and everything is good now. Out of interest, what deter= mines the queue format on the device? Is there a way to change the queue fo= rmat such that 22.11 where gve support first appear could be used?

Thanks,
Peter.
From: Rushil Gupta <rushilg@google.com>
Sent: 29 October 2024 21:14
To: Peter Morrow <peter@graphiant.com>
Cc: users@dpdk.o= rg <users@dpdk.o= rg>; joshwa= sh@google.com <joshwash@google.com>; jeroendb@google.com <jeroendb@google.com>; jungeng.guo@intel.com <jungeng.guo@intel.com>
Subject: Re: gve queue format on dpdk 22.11
=C2=A0

This email originated outsid= e of your organization, do not visit any link, open any attachments or prov= ide any sensitive information unless you recognize the sender and are certa= in the content can be trusted

Hi Peter
DQO_RDA is supported 23.07 onwards.

Hi Folks,

Reading the docs for the the 22.11 release it clearly states that the DQO_R= DA queue format is not yet supported:


I'm attempting to bring up our software router on GCP (VM instance type= c4-standard-8)=C2=A0where we are currently using dpdk 22.11 (via debian, w= ith a 6.1.0-26-amd64 kernel), given the lack of support for DQO_RDA I see t= he following expected messages when I start vpp (our dpdk application):

Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_adminq_describe_device(): Drive= r is running with DQO RDA queue format.
Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_adminq_describe_device(): MAC a= ddr: 42:01:0A:07:00:0F
Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_init_priv(): Max TX queues 2, M= ax RX queues 2
Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_dev_init(): DQO_RDA is not impl= emented and will be added in the future

As a result vpp sefaults shortly after:

(gdb) bt
#0 =C2=A0gve_adminq_create_tx_queue (queue_index=3D0, priv=3D0xbc03a59c0) a= t ../drivers/net/gve/base/gve_adminq.c:502
#1 =C2=A0gve_adminq_create_tx_queues (priv=3D0xbc03a59c0, num_queues=3D2) a= t ../drivers/net/gve/base/gve_adminq.c:516
#2 =C2=A00x00007ffda6c9c5d6 in gve_dev_start (dev=3D0x7ffe1f4f7240 <rte_= eth_devices>) at ../drivers/net/gve/gve_ethdev.c:181
#3 =C2=A00x00007ffe1f489b6a in rte_eth_dev_start () from /lib/x86_64-linux-= gnu/librte_ethdev.so.23
#4 =C2=A00x00007ffff584aa14 in dpdk_device_start (xd=3Dxd@entry=3D0x7ffe204= fbac0) at ./src/plugins/dpdk/device/common.c:393
#5 =C2=A00x00007ffff584c598 in dpdk_interface_admin_up_down (vnm=3D<opti= mized out>, hw_if_index=3D<optimized out>, flags=3D<optimized o= ut>)
=C2=A0 =C2=A0 at ./src/plugins/dpdk/device/device.c:485
#6 =C2=A00x00007ffff6ab3ccd in vnet_sw_interface_set_flags_helper (vnm=3Dvn= m@entry=3D0x7ffff7f6b660 <vnet_main>, sw_if_index=3D<optimized out= >,=C2=A0
=C2=A0 =C2=A0 flags=3DVNET_SW_INTERFACE_FLAG_ADMIN_UP, helper_flags=3D0, he= lper_flags@entry=3DVNET_INTERFACE_SET_FLAGS_HELPER_WANT_REDISTRIBUTE)
=C2=A0 =C2=A0 at ./src/vnet/interface.c:545
#7 =C2=A00x00007ffff6ab489f in vnet_sw_interface_set_flags (vnm=3Dvnm@entry= =3D0x7ffff7f6b660 <vnet_main>, sw_if_index=3D<optimized out>,= =C2=A0
=C2=A0 =C2=A0 flags=3D<optimized out>) at ./src/vnet/interface.c:601<= /div>
#8 =C2=A00x00007ffff6accee9 in vl_api_sw_interface_set_flags_t_handler (mp= =3D0x7ffe3f241ff8) at ./src/vnet/interface_api.c:100
#9 =C2=A00x00007ffff7fa6e0d in msg_handler_internal (free_it=3D0, do_it=3D1= , trace_it=3D<optimized out>, msg_len=3D<optimized out>,=C2=A0<= /div>
=C2=A0 =C2=A0 the_msg=3D0x7ffe3f241ff8, am=3D0x7ffff7fb8f40 <api_global_= main>) at ./src/vlibapi/api_shared.c:580
#10 vl_msg_api_handler_no_free (the_msg=3D0x7ffe3f241ff8, msg_len=3D<opt= imized out>) at ./src/vlibapi/api_shared.c:652
#11 0x00007ffff7f8ed7f in vl_socket_process_api_msg (rp=3D<optimized out= >, input_v=3D<optimized out>) at ./src/vlibmemory/socket_api.c:208=
#12 0x00007ffff7f978d3 in vl_api_clnt_process (vm=3D<optimized out>, = node=3D<optimized out>, f=3D<optimized out>)
=C2=A0 =C2=A0 at ./src/vlibmemory/memclnt_api.c:429
#13 0x00007ffff6853966 in vlib_process_bootstrap (_a=3D<optimized out>= ;) at ./src/vlib/main.c:1223
#14 0x00007ffff67ba03c in clib_calljmp () at /usr/src/packages/BUILD/src/vp= pinfra/longjmp.S:123
#15 0x00007ffe1edfcd90 in ?? ()
#16 0x00007ffff6855054 in vlib_process_startup (f=3D0x0, p=3D0x7ffe2078f780= , vm=3D0x7ffe1fa007c0) at ./src/vlib/main.c:1248
#17 dispatch_process (vm=3D0x7ffe1fa007c0, p=3D<optimized out>, last_= time_stamp=3D<optimized out>, f=3D0x0) at ./src/vlib/main.c:1304
#18 0x0000000000000000 in ?? ()
(gdb)

The segfault occurs due to the complq pointer here being NULL:

=C2=A0 =C2=A0 cmd.create_tx_queue.tx_comp_ring_addr =3D
=C2=A0 =C2=A0 =C2=A0 cpu_to_be64(txq->complq->tx_ring_phys_addr);

It may be possible to upgrade to a more recent version of dpdk which should= help me progress, though before I do that I wondered if there is any other= way to make progress here? Specifically, is there a way to force a differe= nt queue format in the device? This is a c4-standard-8 VM running in GCE.

Thanks!
Peter.

--00000000000040b5a10625ded807--