From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EA09C42B72; Mon, 22 May 2023 17:45:23 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 78A9E40EE7; Mon, 22 May 2023 17:45:23 +0200 (CEST) Received: from mail-ua1-f47.google.com (mail-ua1-f47.google.com [209.85.222.47]) by mails.dpdk.org (Postfix) with ESMTP id B646640EE5 for ; Mon, 22 May 2023 17:45:21 +0200 (CEST) Received: by mail-ua1-f47.google.com with SMTP id a1e0cc1a2514c-783eb14ae3cso4042954241.2 for ; Mon, 22 May 2023 08:45:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684770321; x=1687362321; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=3YWvmHj5lskSJjum5ERd2jyIVP8V6KTPoe7FiJOz6yM=; b=Tj6iRjqMnj7diiOTnkz7w4Q6E3pY1Qb6NVV8i3AViipyIA44im+c4hNHQQTMC+bZsv nWzHyj3CGpOg4w7M9J4/WXJZDKO/7MjhLAoLRxLeCLauniBsILHTcjoE1CIYMGq3Anb0 DdLYjKaR3ts7j4/6225UGPu2E/o2STqktDB7AYB5h4fIiC4ZFyYoWjFozrrVN1CssuJO dwol7Z6S13c4C7bKS9E//nm/HcW0YLXn9CXR4mf66AhPUxrchMX/SlsIq5cBkMJcZPO9 +dmoErqLFDt7psbnsuK2ag6iZbEOw7WNG9TRBDtEEEXN/NmxHv5P3ElEeClEDdy9ErHC EaEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684770321; x=1687362321; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3YWvmHj5lskSJjum5ERd2jyIVP8V6KTPoe7FiJOz6yM=; b=cjbU/bjjlSfKkeAErcaPt2wBmxFCO3vfgI9gAkLapk9jaP+etX43YbKuwFHBt9ksi0 NS96CHg0CBlHHwxA/zYYynxh9gZDxsTym9eqqGROm057WxnTF+2vrLzs7lWHk9oclK49 g2zkPHD0kMDYwEouchgS0vdAv0+7rACqs7YuJMyt9CC5iSQtbr9fJtv4Ln7oDsMG8Fx9 MmkgNgzpAEAqKDOM3/PVW9oUkNFlfyw9AoZeiTFGJpBgcJ1HZ/5/SbVLyMguDI20SImk GoHp1w7pdvE8HcAkJ8ttCDgY4MOOOtOseUJoVACMOycF+uy7+PtcDPSIwFojFqzK1tJd KuNA== X-Gm-Message-State: AC+VfDxWw6uyIY1BUwxB6xY52Pxn9XUHFnobiv2aQevf8963flrZXxBH cknRWoHELM4yH4zuYKBJG/u34Qo5qsgrKTFqkG4xww== X-Google-Smtp-Source: ACHHUZ4nxXLuQv2dpPIYU4WvxSmdlNKYBp1Z6jfqZyJoj47nclnx8HFgA9wUqOS60gZaMtoKd1cTBWW+sahmPIY3X9o= X-Received: by 2002:a67:f1d7:0:b0:439:4112:814 with SMTP id v23-20020a67f1d7000000b0043941120814mr1551624vsm.2.1684770320963; Mon, 22 May 2023 08:45:20 -0700 (PDT) MIME-Version: 1.0 References: <20230519072600.1444309-1-rushilg@google.com> <20230519204618.1507956-1-rushilg@google.com> <820618d8-460f-b415-2c34-6be67f6d7c72@amd.com> In-Reply-To: <820618d8-460f-b415-2c34-6be67f6d7c72@amd.com> From: Rushil Gupta Date: Mon, 22 May 2023 08:45:10 -0700 Message-ID: Subject: Re: [v4] net/gve: check driver compatibility To: Ferruh Yigit Cc: qi.z.zhang@intel.com, jingjing.wu@intel.com, junfeng.guo@intel.com, joshwash@google.com, dev@dpdk.org, Jeroen de Borst Content-Type: multipart/alternative; boundary="0000000000008299cf05fc4a28e8" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --0000000000008299cf05fc4a28e8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable 1. This is the excerpt from the google's virtual nic spec: "In addition to the device-owned register file, vector table, and doorbells, the gVNIC device uses *DMA* (which in most cases amounts to ordinary memory access by host software since we're dealing with a virtual device, but guests must assume the device could be backed by actual hardware) to access physical memory. The following are all located in physical memory: Admin queue - 4096-byte command queue used for configuring gVNIC. Some commands require an additional dma memory region to be passed to the device. These memory regions are allocated to execute the command and freed when the command completes." The calloc by default doesn't allow memory to be shared between the dpdk process and hypervisor (where virtual device lives); so that's the reason it doesn't work. 2. I also have a query: RHEL8 compilation in ci/Intel-compilation context fails due to; is this because of if `not is_linux` meson.build:67:0: ERROR: Include dir lib/eal/linux/include does not exist. Passes: http://patchwork.dpdk.org/project/dpdk/patch/20230508191552.104540-= 1-rushilg@google.com/ Fails: http://patchwork.dpdk.org/project/dpdk/patch/20230519204618.1507956-= 1-rushilg@google.com/ On Mon, May 22, 2023 at 1:52=E2=80=AFAM Ferruh Yigit = wrote: > On 5/19/2023 9:46 PM, Rushil Gupta wrote: > > +static int > > +gve_verify_driver_compatibility(struct gve_priv *priv) > > +{ > > + const struct rte_memzone *driver_info_mem; > > + struct gve_driver_info *driver_info; > > + int err; > > + > > + driver_info_mem =3D > rte_memzone_reserve_aligned("verify_driver_compatibility", > > + sizeof(struct gve_driver_info), > > + rte_socket_id(), > > + RTE_MEMZONE_IOVA_CONTIG, PAGE_SIZE); > > + > > + if (driver_info_mem =3D=3D NULL) { > > + PMD_DRV_LOG(ERR, > > + "Could not alloc memzone for driver > compatibility"); > > + return -ENOMEM; > > + } > > + driver_info =3D (struct gve_driver_info *)driver_info_mem->addr; > > + > > + *driver_info =3D (struct gve_driver_info) { > > + .os_type =3D 5, /* DPDK */ > > + .driver_major =3D GVE_VERSION_MAJOR, > > + .driver_minor =3D GVE_VERSION_MINOR, > > + .driver_sub =3D GVE_VERSION_SUB, > > + .os_version_major =3D cpu_to_be32(DPDK_VERSION_MAJOR), > > + .os_version_minor =3D cpu_to_be32(DPDK_VERSION_MINOR), > > + .os_version_sub =3D cpu_to_be32(DPDK_VERSION_SUB), > > + .driver_capability_flags =3D { > > + cpu_to_be64(GVE_DRIVER_CAPABILITY_FLAGS1), > > + cpu_to_be64(GVE_DRIVER_CAPABILITY_FLAGS2), > > + cpu_to_be64(GVE_DRIVER_CAPABILITY_FLAGS3), > > + cpu_to_be64(GVE_DRIVER_CAPABILITY_FLAGS4), > > + }, > > + }; > > + > > + populate_driver_version_strings((char > *)driver_info->os_version_str1, > > + (char *)driver_info->os_version_str2); > > + > > + err =3D gve_adminq_verify_driver_compatibility(priv, > > + sizeof(struct gve_driver_info), (dma_addr_t)driver_info); > > Back to previous discussion, other commands pass physical address to the > admin command, but this pass virtual address. > To follow the same semantic, shouldn't above be 'driver_info_mem.iova'? > > I asked before but not able to get an answer, what is the memory type > requirement for device? > Why virtual address obtained via 'calloc()' is not working, but virtual > address from hugepages are working? > --0000000000008299cf05fc4a28e8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
1. This is the excerpt from the google's virtual = nic spec:=C2=A0
"In addition to the device-owned register fi= le, vector table, and doorbells, the gVNIC device uses DMA (which in= most cases amounts to ordinary memory access by host software since we'= ;re dealing with a virtual device, but guests must assume the device could = be backed by actual hardware) to access physical memory. The following are = all located in physical memory: Admin queue - 4096-byte command queue used = for configuring gVNIC.=C2=A0
Some commands require an additional = dma memory region to be passed to the device. These memory regions are allo= cated to execute the command and freed when the command completes."
The calloc by default doesn't allow memory to be shared be= tween the dpdk process and hypervisor=C2=A0(where virtual device lives); so= that's the reason it doesn't work.

2. I a= lso have a query: RHEL8 compilation in=C2=A0ci/Intel-compilation context fa= ils due to; is this because of=C2=A0if `not is_linux`

On Mon, May 22,= 2023 at 1:52=E2=80=AFAM Ferruh Yigit <ferruh.yigit@amd.com> wrote:
On 5/19/2023 9:46 PM, Rushil G= upta wrote:
> +static int
> +gve_verify_driver_compatibility(struct gve_priv *priv)
> +{
> +=C2=A0 =C2=A0 =C2=A0const struct rte_memzone *driver_info_mem;
> +=C2=A0 =C2=A0 =C2=A0struct gve_driver_info *driver_info;
> +=C2=A0 =C2=A0 =C2=A0int err;
> +
> +=C2=A0 =C2=A0 =C2=A0driver_info_mem =3D rte_memzone_reserve_aligned(&= quot;verify_driver_compatibility",
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0sizeof(struct gve_driver_info),
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0rte_socket_id(),
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0RTE_MEMZONE_IOVA_CONTIG, PAGE_SIZE);
> +
> +=C2=A0 =C2=A0 =C2=A0if (driver_info_mem =3D=3D NULL) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0PMD_DRV_LOG(ERR,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0"Could not alloc memzone for driver compatibility= ");
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -ENOMEM;
> +=C2=A0 =C2=A0 =C2=A0}
> +=C2=A0 =C2=A0 =C2=A0driver_info =3D (struct gve_driver_info *)driver_= info_mem->addr;
> +
> +=C2=A0 =C2=A0 =C2=A0*driver_info =3D (struct gve_driver_info) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.os_type =3D 5, /* DP= DK */
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.driver_major =3D GVE= _VERSION_MAJOR,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.driver_minor =3D GVE= _VERSION_MINOR,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.driver_sub =3D GVE_V= ERSION_SUB,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.os_version_major =3D= cpu_to_be32(DPDK_VERSION_MAJOR),
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.os_version_minor =3D= cpu_to_be32(DPDK_VERSION_MINOR),
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.os_version_sub =3D c= pu_to_be32(DPDK_VERSION_SUB),
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0.driver_capability_fl= ags =3D {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0cpu_to_be64(GVE_DRIVER_CAPABILITY_FLAGS1),
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0cpu_to_be64(GVE_DRIVER_CAPABILITY_FLAGS2),
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0cpu_to_be64(GVE_DRIVER_CAPABILITY_FLAGS3),
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0cpu_to_be64(GVE_DRIVER_CAPABILITY_FLAGS4),
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0},
> +=C2=A0 =C2=A0 =C2=A0};
> +
> +=C2=A0 =C2=A0 =C2=A0populate_driver_version_strings((char *)driver_in= fo->os_version_str1,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0(char *)driver_info->os_version_str2);
> +
> +=C2=A0 =C2=A0 =C2=A0err =3D gve_adminq_verify_driver_compatibility(pr= iv,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sizeof(struct gve_dri= ver_info), (dma_addr_t)driver_info);

Back to previous discussion, other commands pass physical address to the admin command, but this pass virtual address.
To follow the same semantic, shouldn't above be 'driver_info_mem.io= va'?

I asked before but not able to get an answer, what is the memory type
requirement for device?
Why virtual address obtained via 'calloc()' is not working, but vir= tual
address from hugepages are working?
--0000000000008299cf05fc4a28e8--