From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B813DA04B6; Mon, 7 Sep 2020 15:31:40 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1C76E1BF8A; Mon, 7 Sep 2020 15:31:40 +0200 (CEST) Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) by dpdk.org (Postfix) with ESMTP id A0D101BE0C for ; Mon, 7 Sep 2020 15:31:38 +0200 (CEST) Received: by mail-ed1-f45.google.com with SMTP id q21so12863497edv.1 for ; Mon, 07 Sep 2020 06:31:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:references:in-reply-to:mime-version:thread-index:date :message-id:subject:to:cc; bh=qa7lO7uR958orqBeCfLPfVth+7NsttUQCclBP/A+m3s=; b=Vv8tj9za0CYrCVo+nRKFCMjDT17oX32C8UNIdbvp1dNZrnLO0CmvmA/QH7l+Ws3wig 3HKKzkkKERonF+wJzK9muGFwmbT9rucV/QJCZcfwRpx38JYx8WoDL29mqBFoLN2eNh1+ iIZMh+MvhWCclZ/jHbhahp9+q03pJz0P9cCOA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:cc; bh=qa7lO7uR958orqBeCfLPfVth+7NsttUQCclBP/A+m3s=; b=iUXttc1bBMBav+u4/TuRlWcWG3OOXrhTJ4mzetFJG/jIfpfHxZMH7xBu5RoRvvXftk vl+STkr65w8v4sjkSj7W8k1oIyjkqOIKcA4z000havC1ysCTipXu0ro0DeM4sQzAjeiZ at8cytmihcNYOxV7pfC8bgyr5mCvyJmnQw5lxzHvzybtZT6Vk4Gn11f7z8zCn0znTMvm cfa6MqS8hO4Wq/gK8QruADjBd+CQ3ZBhsJyl8ja4c5i9EFw+bnMyN62i20PiLbYzWPKP qxuwGyhS28AoACx10rewdtmoLf+4CnfS/frwYuyG9yiDNSptpEQv4xTbgrD2f/lHiy9Z Hqeg== X-Gm-Message-State: AOAM532ITV30KIDPLvGiixdypH9cun0owwS1Qhq3yrqLwX6pf+pifjEQ IXRju7wM4uxTPCYWhCu6MZDyQWKtLSneXxyKzKqq4HdOEiI= X-Google-Smtp-Source: ABdhPJxJ/XSeB776ykFCRqd3nXCktzktM3Dzg3Hru4nIZjWgSABXXPID+43c9wo40bkv/7MMeh1KCqWuwKQywdWu9XY= X-Received: by 2002:a50:fd87:: with SMTP id o7mr21249790edt.180.1599485498151; Mon, 07 Sep 2020 06:31:38 -0700 (PDT) From: Vikas Gupta References: <828830f7-8f39-72e4-2f64-c20f6b56d8a9@intel.com> In-Reply-To: <828830f7-8f39-72e4-2f64-c20f6b56d8a9@intel.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQLtUBUt8DEHkLoK1GOEiSL8D8qYLADMEgqMpyk0cGA= Date: Mon, 7 Sep 2020 19:01:10 +0530 Message-ID: <4774f1c16093c58e7f93f339f65f42cd@mail.gmail.com> To: "Burakov, Anatoly" , dev@dpdk.org Cc: Ajit Kumar Khaparde , Vikram Prakash Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] Issue with VFIO/IOMMU X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" RE: [dpdk-dev] Issue with VFIO/IOMMU Hi Burakov, -----Original Message----- From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com ] Sent: Friday, September 04, 2020 7:20 PM To: Vikas Gupta ; dev@dpdk.org Cc: Ajit Kumar Khaparde ; Vikram Prakash < vikram.prakash@broadcom.com> Subject: Re: [dpdk-dev] Issue with VFIO/IOMMU On 03-Sep-20 12:09 PM, Vikas Gupta wrote: > Hi, > > I observe an issue with IOVA address returned by api > rte_memzone_reserve_aligned (flags=3D RTE_MEMZONE_IOVA_CONTIG) used for > queue memory allocation. With high level debugging, I notice that IOVA > address returned in mz->iova is not mapped by VFIO_IOMMU_MAP_DMA so in > turn SMMU exception is seen. I'm not sure i follow. How did you determine that to be the case, given that, by your own admission below, `vfio_type1_dma_mem_map` function is executed several times? [Vikas]: I`ll mention map and unmap as below in explaining through one of the exampl= e map =3D function vfio_type1_dma_mem_map called with argument do_map =3D 1 unamp =3D function type1_dma_mem_map called with argument do_map =3D 0 What I notice that for some particular address received in mz->iova, after rte_memzone_reserve_aligned is successfully returned, the map function (vfio_type1_dma_mem_map do_map =3D1) was not called prior to return of function rte_memzone_reserve_aligned. Below is one of the sequence to understand. Let=E2=80=99s say there is an address =E2=80=98*iova_fail=E2=80=99*, for wh= ich exception is raised by SMMU while dpdk-test runs with Crypto PMD. When dpdk-test is run with Crypto test suit I see that for an address* iova_fail* several times vfio_type1_dma_mem_map is called with (do_map =3D 0/1 with length =3D 2MB). I believe this happens due to call for memory allocation/free for buffers/queues. The test runs fine as long as the map is called before rte_memzone_reserve_aligned returns and similarly for unmap when same memory is freed. But after several times with map/unmap for= * iova_fail*, map is NOT called before rte_memzone_reserve_aligned is retuned though iova_fail was previously unmapped. Since it=E2=80=99s not mapped, SM= MU raises an exception. Please note issue is not frequently visible and might reproduce after pmd_crypto_autotest is run multiple times over dpdk-test. If you are not able to follow I`ll try to send the debug printfs for test. Thanks, Vikas > > > > *Details for the setup* > > Platform: Armv8 (Broadcom Stingray) > > DPDK release: DPDK 20.08 > > PMD patch: > > https://patches.dpdk.org/project/dpdk/list/?series=3D&submitter=3D1907&st= a > te=3D&q=3D&archive=3D&delegate=3D > > dpdk-test is launched using below command > > *dpdk-test --vdev -w 0000:00:00.0 --iova-mode pa * > > > > The test suite is launched over dpdk-test application command prompt > using command =E2=80=98cryptodev__autotest=E2=80=99 > > The issue is seen when several iterations of above test_suite is > executed which in turn do multiple calls to > rte_memzone_reserve_aligned, rte_mempool_create and rte_memzone_free, rte_mempool_free. > > Function *vfio_type1_dma_mem_map* with map/unmap event is executed > several times during test_suite run. > > > > Any inputs would be helpful. > > > > Thanks, > > Vikas > -- Thanks, Anatoly