From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B264F42BCC; Mon, 29 May 2023 08:33:29 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A350942D35; Mon, 29 May 2023 08:33:29 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 3A9E842D35 for ; Mon, 29 May 2023 08:33:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1685342008; x=1716878008; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=o+zCTxn8yXk/eLtv3vZy2/dy+hPtJlRTGd/bliIGzdg=; b=RlS62qgbpHHRMZHt03ftXEe3tJwo9XhLhD5YqDFUzYvTLbrNQ0WXq+3G cUTlJjtBE3Clu/YaGZWTnDeT5V3CHPQiLywdpk5D7hIhFFoZODHzWtmIE oS2rjQ9c+hgN3pYg4ezWkW6LwHiCNJyCDttviSnimS8PtxqfOzZGwdVRz ShHzlnHmeGekkhdKAt001FHWIq5Jo91SJrwpDfUIQmFNL9LK5RTospf9H d+NWKanVOzzP98DjGXMSM2TmbK3+vW7V4tZ9mE9e248gpyQC5K3xKhvBZ axqN9NkmhKP6e45kBEX4ttikCQf5NH9Lpt8tcniuvw+NzEAGFS3fshuBE A==; X-IronPort-AV: E=McAfee;i="6600,9927,10724"; a="382887679" X-IronPort-AV: E=Sophos;i="6.00,200,1681196400"; d="scan'208";a="382887679" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 May 2023 23:33:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10724"; a="880255666" X-IronPort-AV: E=Sophos;i="6.00,200,1681196400"; d="scan'208";a="880255666" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by orsmga005.jf.intel.com with ESMTP; 28 May 2023 23:33:24 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Sun, 28 May 2023 23:33:23 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Sun, 28 May 2023 23:33:23 -0700 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.176) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Sun, 28 May 2023 23:33:23 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JqUgdXv+PtsjCwUZHIr573TAMd3bp6U/L+j6+I1srofgNjLztPAJ0ots+YH2oogndv97aT8D8Ug5SLMZHtIendNDFVDfRZMXGgFr+nqwOuUWeGPhiR+Dp9+YSv2m7m415NnErzZmCmAbEm/MDMV05ti9WDHhj3zLarkjxfZBoGjqBqaDHUi9PeBzCfr8uvMrEkdYkuxaaD43qX8/aenVyudreVhL8YR1KhJ80atLMzWDpq8XPw32awqecYCq95PoTWlpjPodVPdMOMs4mYTt6gKNw+C670lKXtTc2/d+jCnA8Rb3n8zETDmj4fObeAsdDLlA+NrqiQdAq9g639vDCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=m43pfD25SbIF3puimIIko9oTqReit1QmhbX8FwmO+AU=; b=OnlFf5Hgdo/FcDxuHJMfHm9l2SM/7JGVwTawTDBsgpxge10Fdt2B7HPSUwdeGszgJAEq9ToBa1/Tkt/64RDyuC2bFGus18OmtKBHsBc/h5T+qizGtlHzqfs5JGenha7ew4XqzJhLnMpU9cHKgZEjbPzVOycNydTztaGdKSYUn8Gr+EC7JNxqo0d83IJoF/WR9IOwlVSKAnjNC4PyFSfH82ytIVyP9SSgNQk8Ww2iwTJkHucWzE2MbY2sevS0EHOh0OjgfKook6SgMmgjJr4isDgINuJlquw4iEadOacCHseHggJ7r5cwkNa3gl2uy41opwezQ0h1tgLUba/gQO36/w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from SN6PR11MB3504.namprd11.prod.outlook.com (2603:10b6:805:d0::17) by SJ0PR11MB4816.namprd11.prod.outlook.com (2603:10b6:a03:2ad::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6433.22; Mon, 29 May 2023 06:33:20 +0000 Received: from SN6PR11MB3504.namprd11.prod.outlook.com ([fe80::e427:f0a6:8472:d0f6]) by SN6PR11MB3504.namprd11.prod.outlook.com ([fe80::e427:f0a6:8472:d0f6%7]) with mapi id 15.20.6433.020; Mon, 29 May 2023 06:33:20 +0000 From: "Xia, Chenbo" To: Maxime Coquelin , "dev@dpdk.org" , "david.marchand@redhat.com" , "mkp@redhat.com" , "fbl@redhat.com" , "jasowang@redhat.com" , "Liang, Cunming" , "Xie, Yongji" , "echaudro@redhat.com" , "eperezma@redhat.com" , "amorenoz@redhat.com" , "lulu@redhat.com" Subject: RE: [PATCH v3 13/28] vhost: add helper for IOTLB misses Thread-Topic: [PATCH v3 13/28] vhost: add helper for IOTLB misses Thread-Index: AQHZjyXJLgc5S1/R9UKHBMM8lhyms69w0I0w Date: Mon, 29 May 2023 06:33:20 +0000 Message-ID: References: <20230525162551.70359-1-maxime.coquelin@redhat.com> <20230525162551.70359-14-maxime.coquelin@redhat.com> In-Reply-To: <20230525162551.70359-14-maxime.coquelin@redhat.com> Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SN6PR11MB3504:EE_|SJ0PR11MB4816:EE_ x-ms-office365-filtering-correlation-id: 80de33b9-f0eb-4970-001c-08db600e9860 x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: OnaXH72l3RquiTjq3n0/yFb6O/6ASnqMboFn5pUWXadN/tpsFqZcxofOGWGfjrwIFxIkB2xrl+jyZTEZQp38VGKYjalpKvzAU0Ps84nQpKF6jZCFmOUn3zgHeHlmjVA0wqWgAEb9k24EALKXZRQ4IeM9oHOC1KOng1J68JOfWHfeHwufT8DtXKZKDxzY6vdNR6NR2E+fZatEpcc1ZzzMqYDw7kXVYs8ecTRVylJHo973TawtyCqGqZcfQBEkaBMpQCG4kIvM6Q6HLvEQqui8O/elp/wZnnwzC5kfbTPoakivO5IMsd6dccBrM14NeOfWPnxQh9syyE/rcdcZpZB1H6q4bF73LOI4aOSJZCT6YyLGjoRqb0Drh4/iMRQWSCsTR7c29T80hKvLFM3NR4dCqFyPRo7DhteMjehzc1efUQPTIBzlXY8HRHTGFhBk9hoTwLr2UKl+ClmtVMmFMvNTDDxOUe58DYGiLEfiTIJJKtbUT+8EMKJ52F6QKVjkQ9qajGNYlHvjy8P1HVh8b29LgChUbqk3lhKCHCYRwQwJleALozmXLEJjDjzCt4RmIwkQ/zWq8f4dYGa7V4qOlUYowhJebAfDVLN2BgkX/zICbUNL5xjHpXTqBqiHTruY7FZAu06uI0jogRVA3KXneiJN+Q== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR11MB3504.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(366004)(396003)(136003)(39860400002)(346002)(376002)(451199021)(26005)(7696005)(53546011)(9686003)(71200400001)(6506007)(2906002)(316002)(33656002)(186003)(55016003)(52536014)(5660300002)(41300700001)(7416002)(8676002)(8936002)(478600001)(921005)(82960400001)(122000001)(38100700002)(86362001)(110136005)(76116006)(66946007)(38070700005)(66476007)(66556008)(83380400001)(64756008)(66446008); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?iPERfN6iyA0ZTtO5E2Y4eEoflzZHFd2eDEVs8PnkLXTHy2umPvOmwJiIhdcz?= =?us-ascii?Q?qfS3MPBf+i0gfvsNncaQIVVHqpt1g6JkhGoUGL9ooJlgU/S0RGMR9DpOt+qV?= =?us-ascii?Q?7SoeEhhMGNQvpM6DE7vJ5IrwFzGAux3X2hrjikf0PHxX2YIZCeprrM87HC4O?= =?us-ascii?Q?x+bUg7jAEcOmzP/xonsZUvq8Ppk5jJUhDGZj36C0d9LDC3WLhBThl7NRiYrq?= =?us-ascii?Q?ycnBgVUQ5zw1dMVIc1z4GowYQxMu61O4bDoLpsypkjnOHmbrK916mYSkCXoI?= =?us-ascii?Q?9R9DQTHY+9kiQ6p3rm6plZxPMmPwlkmdwvw5DS/YNVwOsz5lw6Y7zX8RQUJa?= =?us-ascii?Q?VHpZNUZu69VvGHQPEZ/ftR+fmIxKP+YzQVkhBbwQMYOFaHDRlniUmEnlnLiQ?= =?us-ascii?Q?zr5cth70CAMQUMzJFGXjJNxMD3cUTDUWBZWByvDBudwzpbzlOp7mSP3Az6si?= =?us-ascii?Q?g8zLXpPEuh+W/enB20JzSiA2mV4N/L4tdn4/6PBbjY12AJ2tJcCTtzJpNI7C?= =?us-ascii?Q?j0wv0eP51Ml2JcINskQojU+Y5afJ/N0QNSEhGaT+iFX0UQLe6l1MIHootTni?= =?us-ascii?Q?8VwGnr4Xs5NvWbvd4Y0OkGA1/fghq58s1qe1N2o6+ICJuQMJ26E2avtBAaS7?= =?us-ascii?Q?PKy7c7oYQNvomLDB9ZyiSlQ0JRxu0ErFs+ZJeRbltyvq3BBAYeSBWN/ht99X?= =?us-ascii?Q?I5pciaYYSujxzdErIMz20PTGbMKoFK8oS+v/WXKD0Qzx21yIT8JHU++D/djD?= =?us-ascii?Q?Wmw3+oW19LFVdDz8zmPBZkxYJ4HzqhY3U4PX++6uJ2ZntdQh6EftEZ3/O7V9?= =?us-ascii?Q?PmQLsJiGCeXwTJz6gYYQPSx3tWGoCP9TOxTvV7HZGYxcYnpgHJcPzWTd7Do8?= =?us-ascii?Q?h+jo4BU92HwDGps3VTNP4NJ6v+i19FnXISVGOGv/wWQs1R3Ak85OET74iTLm?= =?us-ascii?Q?ebFxb2C9Euk7qF+qJhHYnCUsSLsUtPCqqjgGMqDdD3HKwB5v/noG6pTqDonu?= =?us-ascii?Q?cqfYdQAlCZkfFb7vzq37lm2Iy5YiLLi1xc3dx61mlGHz7bCENOVYYNKSeXk9?= =?us-ascii?Q?gpgYtaZzhi4zngbHAvirEUyM7fQImHLRe0vHYBB5XaYGnxVq9dPby8xupf+G?= =?us-ascii?Q?D4Ep2i1XPGmHBdyvEXJ4T/5PPZq2YmBL8xWp6iXa317l2DyCLdz3G5VXkKam?= =?us-ascii?Q?2LMFd2Yi/6e7C1bvyy54YH9KeI+QvLL9QRzhNY0l+Qc0/e3Mz4ti2HAkLtpx?= =?us-ascii?Q?ZmSbJh7XIIRu5ojrhpYbv/FO0Gy5MOMMeqHe7OKacyzfUPwduJcC4ncYYKLM?= =?us-ascii?Q?6SyKXn/VAxeaJK+FA6k+UtDGG1cvjCKEB+tb4IxgPZ+oIywqxfL1tJwskUSe?= =?us-ascii?Q?oPMm6mQqSklLrUQhbLsIGxnhEj3G4nO/RF8O3kcIJIpIjf2oNH/UZF6Y4Hm/?= =?us-ascii?Q?fFOQEX32DwjzEasVyhrprKSpPPaDFwJxz5N8w1cHaL80wX9McMoIra9C1b9a?= =?us-ascii?Q?aoScxHhgMfvXq4LPdh5zMxloE4QSHWgMIOEjiX1ADLjyD6R/nlGpMT29D1Aw?= =?us-ascii?Q?0SQSJhYP088KfEcqxYj1Sw0vVKVxBjuS0yQs5cDM?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR11MB3504.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 80de33b9-f0eb-4970-001c-08db600e9860 X-MS-Exchange-CrossTenant-originalarrivaltime: 29 May 2023 06:33:20.5080 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: VtRob07aDf8w2J5qyJowOlgcrpiUwNDNwkFq00nDPy1bqJ5re+iJGzuFdyRstdQ+ZXitJWqBXVF1SzuxDn7fTg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB4816 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Maxime Coquelin > Sent: Friday, May 26, 2023 12:26 AM > To: dev@dpdk.org; Xia, Chenbo ; > david.marchand@redhat.com; mkp@redhat.com; fbl@redhat.com; > jasowang@redhat.com; Liang, Cunming ; Xie, Yongj= i > ; echaudro@redhat.com; eperezma@redhat.com; > amorenoz@redhat.com; lulu@redhat.com > Cc: Maxime Coquelin > Subject: [PATCH v3 13/28] vhost: add helper for IOTLB misses >=20 > This patch adds a helper for sending IOTLB misses as VDUSE > will use an ioctl while Vhost-user use a dedicated > Vhost-user backend request. >=20 > Signed-off-by: Maxime Coquelin > --- > lib/vhost/vhost.c | 13 ++++++++++++- > lib/vhost/vhost.h | 4 ++++ > lib/vhost/vhost_user.c | 6 ++++-- > lib/vhost/vhost_user.h | 1 - > 4 files changed, 20 insertions(+), 4 deletions(-) >=20 > diff --git a/lib/vhost/vhost.c b/lib/vhost/vhost.c > index 41f212315e..790eb06b28 100644 > --- a/lib/vhost/vhost.c > +++ b/lib/vhost/vhost.c > @@ -52,6 +52,12 @@ static const struct vhost_vq_stats_name_off > vhost_vq_stat_strings[] =3D { >=20 > #define VHOST_NB_VQ_STATS RTE_DIM(vhost_vq_stat_strings) >=20 > +static int > +vhost_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm) > +{ > + return dev->backend_ops->iotlb_miss(dev, iova, perm); > +} > + > uint64_t > __vhost_iova_to_vva(struct virtio_net *dev, struct vhost_virtqueue *vq, > uint64_t iova, uint64_t *size, uint8_t perm) > @@ -86,7 +92,7 @@ __vhost_iova_to_vva(struct virtio_net *dev, struct > vhost_virtqueue *vq, > vhost_user_iotlb_rd_unlock(vq); >=20 > vhost_user_iotlb_pending_insert(dev, iova, perm); > - if (vhost_user_iotlb_miss(dev, iova, perm)) { > + if (vhost_iotlb_miss(dev, iova, perm)) { > VHOST_LOG_DATA(dev->ifname, ERR, > "IOTLB miss req failed for IOVA 0x%" PRIx64 "\n", > iova); > @@ -686,6 +692,11 @@ vhost_new_device(struct vhost_backend_ops *ops) > return -1; > } >=20 > + if (ops->iotlb_miss =3D=3D NULL) { > + VHOST_LOG_CONFIG("device", ERR, "missing IOTLB miss backend > op.\n"); > + return -1; > + } > + > pthread_mutex_lock(&vhost_dev_lock); > for (i =3D 0; i < RTE_MAX_VHOST_DEVICE; i++) { > if (vhost_devices[i] =3D=3D NULL) > diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h > index f37e0b83b8..ee7640e901 100644 > --- a/lib/vhost/vhost.h > +++ b/lib/vhost/vhost.h > @@ -89,13 +89,17 @@ > for (iter =3D val; iter < num; iter++) > #endif >=20 > +struct virtio_net; > typedef void (*vhost_iotlb_remove_notify)(uint64_t addr, uint64_t off, > uint64_t size); >=20 > +typedef int (*vhost_iotlb_miss_cb)(struct virtio_net *dev, uint64_t iova= , > uint8_t perm); > + > /** > * Structure that contains backend-specific ops. > */ > struct vhost_backend_ops { > vhost_iotlb_remove_notify iotlb_remove_notify; > + vhost_iotlb_miss_cb iotlb_miss; > }; >=20 > /** > diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c > index 7655082c4b..972559a2b5 100644 > --- a/lib/vhost/vhost_user.c > +++ b/lib/vhost/vhost_user.c > @@ -3305,7 +3305,7 @@ vhost_user_msg_handler(int vid, int fd) > return ret; > } >=20 > -int > +static int > vhost_user_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t per= m) > { > int ret; > @@ -3465,7 +3465,9 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t > qid, bool enable) > return ret; > } >=20 > -static struct vhost_backend_ops vhost_user_backend_ops; > +static struct vhost_backend_ops vhost_user_backend_ops =3D { > + .iotlb_miss =3D vhost_user_iotlb_miss, > +}; >=20 > int > vhost_user_new_device(void) > diff --git a/lib/vhost/vhost_user.h b/lib/vhost/vhost_user.h > index 61456049c8..1ffeca92f3 100644 > --- a/lib/vhost/vhost_user.h > +++ b/lib/vhost/vhost_user.h > @@ -179,7 +179,6 @@ struct __rte_packed vhu_msg_context { >=20 > /* vhost_user.c */ > int vhost_user_msg_handler(int vid, int fd); > -int vhost_user_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t > perm); >=20 > /* socket.c */ > int read_fd_message(char *ifname, int sockfd, char *buf, int buflen, int > *fds, int max_fds, > -- > 2.40.1 Reviewed-by: Chenbo Xia =20