From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 07986A0A02; Mon, 17 May 2021 15:10:02 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7B4714014E; Mon, 17 May 2021 15:10:01 +0200 (CEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by mails.dpdk.org (Postfix) with ESMTP id 5001640041 for ; Mon, 17 May 2021 15:09:58 +0200 (CEST) IronPort-SDR: gHPvuhWXldhyug2Nfq7l6K700u0ieNfepR7f44/eSoBu+AGXFGHV8rHGhUjLPgtK3KgLqSpXDT K6EVAkWAX4Ww== X-IronPort-AV: E=McAfee;i="6200,9189,9986"; a="187865417" X-IronPort-AV: E=Sophos;i="5.82,307,1613462400"; d="scan'208";a="187865417" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2021 06:07:02 -0700 IronPort-SDR: x6VSdNnWE03vQHY059f1lrwDOSkuonRnNLOm0e/rpdGK+CHMgDBtIKwwEJdhaVpDPeTaINa7Xr FSc8F582tt8Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,307,1613462400"; d="scan'208";a="432646021" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by orsmga007.jf.intel.com with ESMTP; 17 May 2021 06:07:02 -0700 Received: from orsmsx602.amr.corp.intel.com (10.22.229.15) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.4; Mon, 17 May 2021 06:07:02 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.4 via Frontend Transport; Mon, 17 May 2021 06:07:02 -0700 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.173) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2106.2; Mon, 17 May 2021 06:07:02 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=G+5OZMboI04JZICJYel9lgdYOqyEugV1sSwmVgAGJWNlU54wF0xuJ6cunxtgIsNWuyjM7+M749/gVh3NYSETRYusMj4fsM3WtgZw4cLpxOazzx6h2z+wJvu69JuouxMnjqjU6Y5DWbJNu7hIV4cVRWUk1RDU9XxHOC0LW/OcJIPkuzzVtzzsSTw5qjmWjwU9xeWEB3QI1G9RjNXjAjKHzxPrx5eCt3DSmYgT9MY2pV/q4DDDWPaaSjZMGFSQnbv4MFjtOX5CCaD/SOdDJgIKOtzu1P2ev9DfAn/ciXQi2gZelrT0nrZ2Mrq/kJUvSEWGQc7NbrYxqcwx4cg8sWkGag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XmekAWMdMhopcSa57CJ02eD18bRKvfUdRvNy4TknAPs=; b=Ta8rtB9q/quiycpxlWHZGA0Eg5FODH7PAcHXjaDmieftEiZP8fLkh+LCjF3rDfs0QI3TvsSUpW9nYaS4eE+wJPq7vwBgyzz0WRENOvldp20a77YjC5pVWe7go0C+TR0VUiJmAxRcDPDSHzif7w3/BPTNcV/GkLdlpZPB40aR5BJGVJIg/aQ0bg51QD9YdNBMXxjfPdIVGG1Ab7E2gnfYHJzXOf/uVHUsNz9SgA6zkLmuojGR90TTYKOuNpwzDD6BYNi+Bd/bWsPJC/Lz7ttScAefazizoIGO1swdMc+hDSxUwzgu3STQYUrNucxTWWjph8wklG1H15uNHZqU5PuNMg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XmekAWMdMhopcSa57CJ02eD18bRKvfUdRvNy4TknAPs=; b=GTzCAifkSiF88OTSo2LG3p+ijPYTJcPKGt3jvK1cB1qlULFMll4dtKSIJt2qNi53k5RiFOlZXNnztgo/dvMsPoMnPnJsBEV5omPkI+p3oR9YpBalp2g8/K8YfrJPXuj8r2gFnKiyiFBLanv2frY2xGhue6hgSpLTHJLIV1jIVLk= Received: from MN2PR11MB4063.namprd11.prod.outlook.com (2603:10b6:208:13f::22) by MN2PR11MB3597.namprd11.prod.outlook.com (2603:10b6:208:ee::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4129.26; Mon, 17 May 2021 13:06:59 +0000 Received: from MN2PR11MB4063.namprd11.prod.outlook.com ([fe80::6d37:7bdb:fdf1:e9f3]) by MN2PR11MB4063.namprd11.prod.outlook.com ([fe80::6d37:7bdb:fdf1:e9f3%4]) with mapi id 15.20.4129.031; Mon, 17 May 2021 13:06:59 +0000 From: "Xia, Chenbo" To: David Marchand , "dev@dpdk.org" CC: "thomas@monjalon.net" , "Yigit, Ferruh" , Maxime Coquelin , Zhihong Wang , Junjie Wan Thread-Topic: [PATCH] vhost: restore IOTLB mempool allocation Thread-Index: AQHXSvsYA9TPkrpXhkivgjzlL87lfKrnpBEQ Date: Mon, 17 May 2021 13:06:59 +0000 Message-ID: References: <20210517085951.28970-1-david.marchand@redhat.com> In-Reply-To: <20210517085951.28970-1-david.marchand@redhat.com> Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.5.1.3 authentication-results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.147.215] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 166b04ed-2446-436d-bf0c-08d91934a7c3 x-ms-traffictypediagnostic: MN2PR11MB3597: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: qhQ0i7wPUiKJl+c2h4Sof3Cv/6+XauAI+9TmLheE84DWz1cqps23aK6wH2iiK75s0D4Laem8ZnyOfVv3t0t7VrXv1ImnmUIXvBTIna1oKv48kOrhuiGRt+Q2wbWIlH3Zlhlc8lDhi3hQ7KhAr5LJxJ6qNB2RszwVYQH3LPVeHr7xYX9dB/68jjWK32Rr9inzyrezCbza3CTkXNHxIri06wqAqT3h9+vcC8FKkaDri1lc5AeYCKmeinBAAXsMwTC9Rt2/49lUk/4inrG9vRg4+lxEUV7MLaWoe9Hh3Vyity6AKfd+vE4Yvv1xb0dATtRWA23ep7TX3qqOvsjeSplMtRMr1Rn0JPP6GAsWladgqm432u+DKsd7chzE7WBDtUg91y+aJy1aTTc2ioBzn/DeNuCQtcVoFQUcEPxGADcuGz5qCi9Pdx/uVOPcWv2ne1sHU6fh5+4GtVQ2GhjKUcNmdIwzsWbdYamDpCLvBeIBxTsJuR/ZDjqVl3Xdre+7GPwLWHKIjyd+p/dvayZegumxtZgLg6HpvchJjpzHcf2hTY5eCGcNvzf2Hc8j+QHA4Cgk3I+VkRF3mH9cHfV7+gmHvRWxl0mRTKdeTzqwxWojurY= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR11MB4063.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(136003)(396003)(366004)(39860400002)(376002)(346002)(33656002)(110136005)(4326008)(7696005)(66446008)(66556008)(9686003)(71200400001)(55016002)(54906003)(316002)(186003)(478600001)(26005)(8936002)(53546011)(86362001)(8676002)(6506007)(52536014)(122000001)(38100700002)(2906002)(5660300002)(76116006)(83380400001)(66476007)(64756008)(66946007); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?EPG0CADY98NykvWNm5GWrp5uZsMYUO1n1S9xYejpD/loEfwgYONEBFti54l1?= =?us-ascii?Q?HI6ttozj4WsZ7l7TZv3xWPWrYXMY+zIhYSm1qqqsV3nqZVcr9IQ7+XKAyARO?= =?us-ascii?Q?h0c8k1pULgKpHMrf3Haja02eq127olIM0GhOHMvgVgerFR8FYl7M5PD1VKKI?= =?us-ascii?Q?vmn3rAUS4Z5m32esJFSO6qz89oLUpZFXOm03OhW8XbwayzUAmlRJP/mPzItb?= =?us-ascii?Q?OohM03dDS/z6JjU7BM28nO7LnzCvR96UfkhD6LxN2/CQdrUN/7fsCsoyk0mM?= =?us-ascii?Q?A26nwWlSCX3Lo8I4SPGRrcPSfP2zr2ElgEF14JJY2xRF3WZbBVR56fclFucw?= =?us-ascii?Q?0KDUErB0dVKcRHZsw6inrI0+3PZ4C8ybR7ImVCiyCCRzVkbvhkXg7bU5kvK8?= =?us-ascii?Q?WqfBXUXY9WII3RxExa5kqqXKlqyGvuucqYV5ucLxwoFEHyLbjsu/LxXSRTGF?= =?us-ascii?Q?jsJf5Fe6EOpzJ9JY8QaJXgTVjjSD6jVTYZ2dMAFDbWq2pRmlz2kIOLMRQzWE?= =?us-ascii?Q?rKB9ykJ1JvoPhpC0qUP6imn88zuZXbF1iQKDuDA6sraFqO2XbM5yQX5tEngh?= =?us-ascii?Q?r3YjhCue19uZedM5jkJIxYv1/Sem8NiwFZgp0vyw2JC0V0q99lwAMilXed7A?= =?us-ascii?Q?3BKXIilMAF2TJhrYfUo+wtp2pcs1iZoT3CPyTl/VBSBSLQq4n1Wv6pF6IycO?= =?us-ascii?Q?vjM3dJeNwmY5FjqGbmtAm1Jc0dSmEMShfh6UoPC7ihX9DZ0ugn66z4wYdUh1?= =?us-ascii?Q?hgBZCRWgFcGW1YnAnD9v2brGae2cnD/+W+v84S4p22+tYBin+M+UDcW7GGAS?= =?us-ascii?Q?7dvh9KVJL4EJFHd933JMZmxAU+aC/Orc7LTo42PWGk1UMwABuTHUr2++/Bsz?= =?us-ascii?Q?+AfcjrHx49nyRW/6bijuPvT8L4KVLlxEdXfMnnsgKYrb/zWGP3Dp4sX649M/?= =?us-ascii?Q?oyH373WsUXUnyHGrbdyjSm5nmJ9n2g/zCCqATE33SufUoZqBaiFtCnK0OO9S?= =?us-ascii?Q?M4VjxBiAFQmFOM+W+kjgbv7/WbNk40PDy6Rt0u0UM2/7TVemrhrV8buH/c9Y?= =?us-ascii?Q?bunODOMLVQ1xImvu2D/s5/mbuI4iDeg77ODPUn/2tLajMVxCNz8WTFWk6dN3?= =?us-ascii?Q?HnPtYKJJ94sOxJOgpCIWwyl2Udxo31xFnKGfBmwgc/icg4WgMX7XQ+6Lbqz4?= =?us-ascii?Q?pIRDbuM3eGGiinuZo84inFtWulJPeU4JEzlKDB1g0F2t9uHbxFrvEhBPiKEm?= =?us-ascii?Q?vIMbCqG6onh13+HuCeKaxLJvzMTuBae1OubXmVgixexCXB3XDj9jGTV5YtZK?= =?us-ascii?Q?hazuC9tNEgLb1AGA0GzH8Yfx?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR11MB4063.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 166b04ed-2446-436d-bf0c-08d91934a7c3 X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2021 13:06:59.2093 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: hih77Bf7ImEoXsbMALhN6xdCybM5KP/Fi7NKmM11XjONAU2xwpmrLw9dUb7wIfnANXBK00D2KCHbWv3/QwaeQg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB3597 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH] vhost: restore IOTLB mempool allocation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: David Marchand > Sent: Monday, May 17, 2021 5:00 PM > To: dev@dpdk.org > Cc: thomas@monjalon.net; Yigit, Ferruh ; Maxime > Coquelin ; Xia, Chenbo = ; > Zhihong Wang ; Junjie Wan > > Subject: [PATCH] vhost: restore IOTLB mempool allocation >=20 > As explained by Chenbo, IOTLB messages will be sent when some queues > are not enabled. If we initialize IOTLB in vhost_user_set_vring_num, > it could happen that IOTLB update comes when IOTLB pool of disabled > queues are not initialized. >=20 > Fixes: 968bbc7e2e50 ("vhost: avoid IOTLB mempool allocation while IOMMU > disabled") >=20 > Signed-off-by: David Marchand > --- > Summary of a discussion with Maxime: >=20 > To keep the mempool allocation optimization, we could try to initialise > the per-vring mempools at reception of the first IOTLB message. > Since those pools are used as caches, it is not an issue if some vrings > received more IOTLB updates than others. >=20 > But looking/testing this now is too late for 21.05, hence reverting is > the safer. >=20 > --- > lib/vhost/vhost.c | 5 +++-- > lib/vhost/vhost_user.c | 6 +----- > 2 files changed, 4 insertions(+), 7 deletions(-) >=20 > diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c > index 9cbcf650b6..c96f6335c8 100644 > --- a/lib/vhost/vhost.c > +++ b/lib/vhost/vhost.c > @@ -365,8 +365,7 @@ free_vq(struct virtio_net *dev, struct vhost_virtqueu= e > *vq) >=20 > vhost_free_async_mem(vq); > rte_free(vq->batch_copy_elems); > - if (vq->iotlb_pool) > - rte_mempool_free(vq->iotlb_pool); > + rte_mempool_free(vq->iotlb_pool); > rte_free(vq->log_cache); > rte_free(vq); > } > @@ -570,6 +569,8 @@ init_vring_queue(struct virtio_net *dev, uint32_t > vring_idx) > vq->kickfd =3D VIRTIO_UNINITIALIZED_EVENTFD; > vq->callfd =3D VIRTIO_UNINITIALIZED_EVENTFD; > vq->notif_enable =3D VIRTIO_UNINITIALIZED_NOTIF; > + > + vhost_user_iotlb_init(dev, vring_idx); > } >=20 > static void > diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c > index 611ff209e3..8f0eba6412 100644 > --- a/lib/vhost/vhost_user.c > +++ b/lib/vhost/vhost_user.c > @@ -469,10 +469,6 @@ vhost_user_set_vring_num(struct virtio_net **pdev, > return RTE_VHOST_MSG_RESULT_ERR; > } >=20 > - if (dev->features & (1ULL << VIRTIO_F_IOMMU_PLATFORM)) { > - if (vhost_user_iotlb_init(dev, msg->payload.state.index)) > - return RTE_VHOST_MSG_RESULT_ERR; > - } > return RTE_VHOST_MSG_RESULT_OK; > } >=20 > @@ -578,7 +574,7 @@ numa_realloc(struct virtio_net *dev, int index) > dev->virtqueue[index] =3D vq; > vhost_devices[dev->vid] =3D dev; >=20 > - if (old_vq !=3D vq && (dev->features & (1ULL << > VIRTIO_F_IOMMU_PLATFORM))) > + if (old_vq !=3D vq) > vhost_user_iotlb_init(dev, index); >=20 > return dev; > -- > 2.23.0 Reviewed-by: Chenbo Xia