From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 168D945C0F for ; Tue, 29 Oct 2024 22:14:30 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0216642EE8; Tue, 29 Oct 2024 22:14:30 +0100 (CET) Received: from mail-ej1-f43.google.com (mail-ej1-f43.google.com [209.85.218.43]) by mails.dpdk.org (Postfix) with ESMTP id 07B0B4060F for ; Tue, 29 Oct 2024 22:14:28 +0100 (CET) Received: by mail-ej1-f43.google.com with SMTP id a640c23a62f3a-a9a3dc089d8so874359766b.3 for ; Tue, 29 Oct 2024 14:14:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1730236467; x=1730841267; darn=dpdk.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=9+Bg3WOdN3qSblTrq6AGcPFDBHwAcHimSrtujHZg8OI=; b=OHVtWqm4HiRjWsGMJrVXFu6dhN+hubd9M1mv9+RU4HOK4AQ7/2pDqdd7sni4Bf5kvu 4vNpbkpBn89O8qxvNcYnspeq/c+k0NKZB4UY6d94CPw3+Erg4fZZ4NwmDPWyycdctzmZ hZ/1M72PL5uH7nXpypN+89n6blRh44385bERjmyPNlG8W3SZm4S0LdV2l7FWhlTBposY QqbklffJ3CgIKocxb3iKaGeQWUr4P6ejxnEXBcivv23k4ynjUTBidJYOlDq+PQPbwx9W Fk+8AaP7hfs6ZB7Bx3denrP6ryQ0rJjhCpBbreXGqYPMbbF+H1Y2FKuKPbmLPDciucI2 Pj+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730236467; x=1730841267; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9+Bg3WOdN3qSblTrq6AGcPFDBHwAcHimSrtujHZg8OI=; b=oF5JbjHrSN+C9uAA4GWEGR2EUYIvjFIOxiJ8Ef3wW43CtGo9ORVCwh54NG3U7gOl8t 5epDKZqBxHtjUP9m/D2OoOo//L+0khzLjwqrZHIdubgEcHTelMqn+yDoHQliQdaTIOyl Tnn0l+Gs7nlNs3useLMafMyaKOwaiTqU0nF9KfOlaKJf++3lqQGV0ezouJB8bsKSHb0e 84Q9TL9aO4oaBlJsrKiTg7tcrFSuk6YSVGWu/BBDdQTwkaZxoGhPLVE0172siFht8TX+ HlQ6utFk3oKOSl2k6cImoTAuPtdMPovbPRAmF6UBy2Xnz8MEUF28H1jsPNSUbwdygEXn 0Rjw== X-Gm-Message-State: AOJu0YwdqacCg25RB2zjhbue8FqD0sN1GwvSYmzSOIelkystGOlxULdP OPNX4DEkFXnflHwLmf34mBgbZFhHb1uaSyF7g1FTZi1NTprbm3jnVZylCz4RY3hKSnhjJTw89zO oROWxPVNX+1uDOg/NQ417T/28d6LKbdOreLC+ X-Google-Smtp-Source: AGHT+IG2jBzcTAzYfKkEIOXBSP1E1yb3edbjHqh1Se6rVdx0OZX4RbRABrfWNd84z+VxIbG1Wticgk11AY7XMrjgeR4= X-Received: by 2002:a17:907:86a5:b0:a9a:6847:e82c with SMTP id a640c23a62f3a-a9de5d95e17mr1218220266b.15.1730236467242; Tue, 29 Oct 2024 14:14:27 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Rushil Gupta Date: Tue, 29 Oct 2024 14:14:16 -0700 Message-ID: Subject: Re: gve queue format on dpdk 22.11 To: Peter Morrow Cc: "users@dpdk.org" , "joshwash@google.com" , "jeroendb@google.com" , "jungeng.guo@intel.com" Content-Type: multipart/alternative; boundary="0000000000000289e80625a4121f" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --0000000000000289e80625a4121f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Peter DQO_RDA is supported 23.07 onwards. On Tue, Oct 29, 2024 at 10:09=E2=80=AFAM Peter Morrow = wrote: > Hi Folks, > > Reading the docs for the the 22.11 release it clearly states that the > DQO_RDA queue format is not yet supported: > > https://doc.dpdk.org/guides-22.11/nics/gve.html > > I'm attempting to bring up our software router on GCP (VM instance type > c4-standard-8) where we are currently using dpdk 22.11 (via debian, with = a > 6.1.0-26-amd64 kernel), given the lack of support for DQO_RDA I see the > following expected messages when I start vpp (our dpdk application): > > Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_adminq_describe_device(): > Driver is running with DQO RDA queue format. > Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_adminq_describe_device(): MAC > addr: 42:01:0A:07:00:0F > Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_init_priv(): Max TX queues 2, > Max RX queues 2 > Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_dev_init(): DQO_RDA is not > implemented and will be added in the future > > As a result vpp sefaults shortly after: > > (gdb) bt > #0 gve_adminq_create_tx_queue (queue_index=3D0, priv=3D0xbc03a59c0) at > ../drivers/net/gve/base/gve_adminq.c:502 > #1 gve_adminq_create_tx_queues (priv=3D0xbc03a59c0, num_queues=3D2) at > ../drivers/net/gve/base/gve_adminq.c:516 > #2 0x00007ffda6c9c5d6 in gve_dev_start (dev=3D0x7ffe1f4f7240 > ) at ../drivers/net/gve/gve_ethdev.c:181 > #3 0x00007ffe1f489b6a in rte_eth_dev_start () from > /lib/x86_64-linux-gnu/librte_ethdev.so.23 > #4 0x00007ffff584aa14 in dpdk_device_start (xd=3Dxd@entry=3D0x7ffe204fba= c0) > at ./src/plugins/dpdk/device/common.c:393 > #5 0x00007ffff584c598 in dpdk_interface_admin_up_down (vnm=3D out>, hw_if_index=3D, flags=3D) > at ./src/plugins/dpdk/device/device.c:485 > #6 0x00007ffff6ab3ccd in vnet_sw_interface_set_flags_helper (vnm=3Dvnm@e= ntry=3D0x7ffff7f6b660 > , sw_if_index=3D, > flags=3DVNET_SW_INTERFACE_FLAG_ADMIN_UP, helper_flags=3D0, > helper_flags@entry=3DVNET_INTERFACE_SET_FLAGS_HELPER_WANT_REDISTRIBUTE) > at ./src/vnet/interface.c:545 > #7 0x00007ffff6ab489f in vnet_sw_interface_set_flags (vnm=3Dvnm@entry=3D= 0x7ffff7f6b660 > , sw_if_index=3D, > flags=3D) at ./src/vnet/interface.c:601 > #8 0x00007ffff6accee9 in vl_api_sw_interface_set_flags_t_handler > (mp=3D0x7ffe3f241ff8) at ./src/vnet/interface_api.c:100 > #9 0x00007ffff7fa6e0d in msg_handler_internal (free_it=3D0, do_it=3D1, > trace_it=3D, msg_len=3D, > the_msg=3D0x7ffe3f241ff8, am=3D0x7ffff7fb8f40 ) at > ./src/vlibapi/api_shared.c:580 > #10 vl_msg_api_handler_no_free (the_msg=3D0x7ffe3f241ff8, msg_len=3D out>) at ./src/vlibapi/api_shared.c:652 > #11 0x00007ffff7f8ed7f in vl_socket_process_api_msg (rp=3D= , > input_v=3D) at ./src/vlibmemory/socket_api.c:208 > #12 0x00007ffff7f978d3 in vl_api_clnt_process (vm=3D, > node=3D, f=3D) > at ./src/vlibmemory/memclnt_api.c:429 > #13 0x00007ffff6853966 in vlib_process_bootstrap (_a=3D) a= t > ./src/vlib/main.c:1223 > #14 0x00007ffff67ba03c in clib_calljmp () at > /usr/src/packages/BUILD/src/vppinfra/longjmp.S:123 > #15 0x00007ffe1edfcd90 in ?? () > #16 0x00007ffff6855054 in vlib_process_startup (f=3D0x0, p=3D0x7ffe2078f7= 80, > vm=3D0x7ffe1fa007c0) at ./src/vlib/main.c:1248 > #17 dispatch_process (vm=3D0x7ffe1fa007c0, p=3D, > last_time_stamp=3D, f=3D0x0) at ./src/vlib/main.c:1304 > #18 0x0000000000000000 in ?? () > (gdb) > > The segfault occurs due to the complq pointer here being NULL: > > cmd.create_tx_queue.tx_comp_ring_addr =3D > cpu_to_be64(txq->complq->tx_ring_phys_addr); > > It may be possible to upgrade to a more recent version of dpdk which > should help me progress, though before I do that I wondered if there is a= ny > other way to make progress here? Specifically, is there a way to force a > different queue format in the device? This is a c4-standard-8 VM running = in > GCE. > > Thanks! > Peter. > > --0000000000000289e80625a4121f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Peter
DQO_RDA is supported 23.07 onwards.

On T= ue, Oct 29, 2024 at 10:09=E2=80=AFAM Peter Morrow <peter@graphiant.com> wrote:
<= /div>
Hi Folks,

Reading the docs for the the 22.11 release it clearly states that the DQO_R= DA queue format is not yet supported:


I'm attempting to bring up our software router on GCP (VM instance type= c4-standard-8)=C2=A0where we are currently using dpdk 22.11 (via debian, w= ith a 6.1.0-26-amd64 kernel), given the lack of support for DQO_RDA I see t= he following expected messages when I start vpp (our dpdk application):

Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_adminq_describe_device(): Drive= r is running with DQO RDA queue format.
Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_adminq_describe_device(): MAC a= ddr: 42:01:0A:07:00:0F
Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_init_priv(): Max TX queues 2, M= ax RX queues 2
Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_dev_init(): DQO_RDA is not impl= emented and will be added in the future

As a result vpp sefaults shortly after:

(gdb) bt
#0 =C2=A0gve_adminq_create_tx_queue (queue_index=3D0, priv=3D0xbc03a59c0) a= t ../drivers/net/gve/base/gve_adminq.c:502
#1 =C2=A0gve_adminq_create_tx_queues (priv=3D0xbc03a59c0, num_queues=3D2) a= t ../drivers/net/gve/base/gve_adminq.c:516
#2 =C2=A00x00007ffda6c9c5d6 in gve_dev_start (dev=3D0x7ffe1f4f7240 <rte_= eth_devices>) at ../drivers/net/gve/gve_ethdev.c:181
#3 =C2=A00x00007ffe1f489b6a in rte_eth_dev_start () from /lib/x86_64-linux-= gnu/librte_ethdev.so.23
#4 =C2=A00x00007ffff584aa14 in dpdk_device_start (xd=3Dxd@entry=3D0x7ffe204= fbac0) at ./src/plugins/dpdk/device/common.c:393
#5 =C2=A00x00007ffff584c598 in dpdk_interface_admin_up_down (vnm=3D<opti= mized out>, hw_if_index=3D<optimized out>, flags=3D<optimized o= ut>)
=C2=A0 =C2=A0 at ./src/plugins/dpdk/device/device.c:485
#6 =C2=A00x00007ffff6ab3ccd in vnet_sw_interface_set_flags_helper (vnm=3Dvn= m@entry=3D0x7ffff7f6b660 <vnet_main>, sw_if_index=3D<optimized out= >,=C2=A0
=C2=A0 =C2=A0 flags=3DVNET_SW_INTERFACE_FLAG_ADMIN_UP, helper_flags=3D0, he= lper_flags@entry=3DVNET_INTERFACE_SET_FLAGS_HELPER_WANT_REDISTRIBUTE)
=C2=A0 =C2=A0 at ./src/vnet/interface.c:545
#7 =C2=A00x00007ffff6ab489f in vnet_sw_interface_set_flags (vnm=3Dvnm@entry= =3D0x7ffff7f6b660 <vnet_main>, sw_if_index=3D<optimized out>,= =C2=A0
=C2=A0 =C2=A0 flags=3D<optimized out>) at ./src/vnet/interface.c:601<= /div>
#8 =C2=A00x00007ffff6accee9 in vl_api_sw_interface_set_flags_t_handler (mp= =3D0x7ffe3f241ff8) at ./src/vnet/interface_api.c:100
#9 =C2=A00x00007ffff7fa6e0d in msg_handler_internal (free_it=3D0, do_it=3D1= , trace_it=3D<optimized out>, msg_len=3D<optimized out>,=C2=A0<= /div>
=C2=A0 =C2=A0 the_msg=3D0x7ffe3f241ff8, am=3D0x7ffff7fb8f40 <api_global_= main>) at ./src/vlibapi/api_shared.c:580
#10 vl_msg_api_handler_no_free (the_msg=3D0x7ffe3f241ff8, msg_len=3D<opt= imized out>) at ./src/vlibapi/api_shared.c:652
#11 0x00007ffff7f8ed7f in vl_socket_process_api_msg (rp=3D<optimized out= >, input_v=3D<optimized out>) at ./src/vlibmemory/socket_api.c:208=
#12 0x00007ffff7f978d3 in vl_api_clnt_process (vm=3D<optimized out>, = node=3D<optimized out>, f=3D<optimized out>)
=C2=A0 =C2=A0 at ./src/vlibmemory/memclnt_api.c:429
#13 0x00007ffff6853966 in vlib_process_bootstrap (_a=3D<optimized out>= ;) at ./src/vlib/main.c:1223
#14 0x00007ffff67ba03c in clib_calljmp () at /usr/src/packages/BUILD/src/vp= pinfra/longjmp.S:123
#15 0x00007ffe1edfcd90 in ?? ()
#16 0x00007ffff6855054 in vlib_process_startup (f=3D0x0, p=3D0x7ffe2078f780= , vm=3D0x7ffe1fa007c0) at ./src/vlib/main.c:1248
#17 dispatch_process (vm=3D0x7ffe1fa007c0, p=3D<optimized out>, last_= time_stamp=3D<optimized out>, f=3D0x0) at ./src/vlib/main.c:1304
#18 0x0000000000000000 in ?? ()
(gdb)

The segfault occurs due to the complq pointer here being NULL:

=C2=A0 =C2=A0 cmd.create_tx_queue.tx_comp_ring_addr =3D
=C2=A0 =C2=A0 =C2=A0 cpu_to_be64(txq->complq->tx_ring_phys_addr);

It may be possible to upgrade to a more recent version of dpdk which should= help me progress, though before I do that I wondered if there is any other= way to make progress here? Specifically, is there a way to force a differe= nt queue format in the device? This is a c4-standard-8 VM running in GCE.

Thanks!
Peter.

--0000000000000289e80625a4121f--