From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 14D35429E7; Tue, 25 Apr 2023 08:20:07 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A244542D13; Tue, 25 Apr 2023 08:20:06 +0200 (CEST) Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by mails.dpdk.org (Postfix) with ESMTP id ACAC842D13 for ; Tue, 25 Apr 2023 08:20:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682403604; x=1713939604; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=PudK+4zoxPDVfalzKW9wicaeFAnpLZpVAsEIQbHJQcU=; b=ZlIS+xCFrEUTT31HG5jkapHJCxziEutE0J2ZEgMKnNlET5bgKB0oeE6F 2rZRQHJ5U7MjBxXYsqbSGDr/8ntLnQjbGXNfdzd/aqG0OU8ESGU+x+c/G /lsYCu0XHy5O9STENtJaOZL9nml7W9ZzcAMpF+OH80InCJEHDqo7E+eIU xFv4pDsV58NxAluN9+OeHK02pXlhVxNxnprIh+LjCweOhvPa9ZtcxusPu CVc7ZNpnqVRiGXDmfqljZVYQxjmTaB+6/78s2H3qUhTAGYUg+dJRRXMhf 9Jp4zwFuAIvUGrFmfQuZu/L1LLzFXr8PYIBXOXvIk24lbZzG7TPsB3O+d w==; X-IronPort-AV: E=McAfee;i="6600,9927,10690"; a="330883805" X-IronPort-AV: E=Sophos;i="5.99,224,1677571200"; d="scan'208";a="330883805" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Apr 2023 23:20:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10690"; a="696049562" X-IronPort-AV: E=Sophos;i="5.99,224,1677571200"; d="scan'208";a="696049562" Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16]) by fmsmga007.fm.intel.com with ESMTP; 24 Apr 2023 23:20:03 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Mon, 24 Apr 2023 23:20:02 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Mon, 24 Apr 2023 23:20:02 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.168) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Mon, 24 Apr 2023 23:20:02 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iBzT7vnI5ME9uXfdtKc7wtnJPgk0S1e22IJDsUbVFqIk2jVmAG8G6bw59fcLOv0jd/+3zGIB4EzKYsnCe5E3qoBosFoWhWQ+nEO/dOn1uHRtjWGA5RXNXJVpTST8BNJoob5563CMcoL3CLcRGrkUVYqV89lx7/8eI9Ldj3+JLmws4aW/kNzXwCZal1uOJ0b/hZyexYwJsgwsFVPa04+pnR0hE626HIOVAobzkMPrigVjddKuEZa8MhyDtovwWdOkKTRQtZps4saQaoocOj5LbzsYw7eIEN2vVkWM1//3yovkvRYNla2wc1fdCnz/vPcYuj64QuwPIRY1NyZ06/aAIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/mEj/mg53Tp2UoT4M0La7ohkjbkszHm1hvMTn0mgnas=; b=mSQOV7eqdlyF7suyt7FD2gVJbFBJOHBm5UKHTgcYXgpjczsqQGI8dv9ffUY9PmnUVBJ4NBZ8lajxfFvxf39DcgscPPdevc3olkGof+bIoC6ahuoW3NvxE1wIbWB1wJ1R5OjgfI/zMW3vcIvdf1mULnkOiZy1lSOokfbt3HIHN0Q6DDXQAb6UG6SLgiGp1xcnZSVvoPJAnJ6G8VXWyecGMFOtNLWjsj1pBnycZXKM4S4Nkj2R7PgSy4TebofMMhwR3TWPE8wASxRUgBPZ3xesc2k3wEM0nnOrC47vUCCnbJI0MhzxKwWVTV3qdGWRP+2F/ZUC+LlLzsq9IXQkcPeheg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from SN6PR11MB3504.namprd11.prod.outlook.com (2603:10b6:805:d0::17) by SN7PR11MB7439.namprd11.prod.outlook.com (2603:10b6:806:343::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.22; Tue, 25 Apr 2023 06:20:00 +0000 Received: from SN6PR11MB3504.namprd11.prod.outlook.com ([fe80::2d7a:a878:7bed:bb11]) by SN6PR11MB3504.namprd11.prod.outlook.com ([fe80::2d7a:a878:7bed:bb11%4]) with mapi id 15.20.6319.033; Tue, 25 Apr 2023 06:20:00 +0000 From: "Xia, Chenbo" To: Maxime Coquelin , "dev@dpdk.org" , "david.marchand@redhat.com" , "mkp@redhat.com" , "fbl@redhat.com" , "jasowang@redhat.com" , "Liang, Cunming" , "Xie, Yongji" , "echaudro@redhat.com" , "eperezma@redhat.com" , "amorenoz@redhat.com" Subject: RE: [RFC 08/27] vhost: add offset field to IOTLB entries Thread-Topic: [RFC 08/27] vhost: add offset field to IOTLB entries Thread-Index: AQHZY+eU8b6Qog/xhkSecrhj9k7FGK87s/3w Date: Tue, 25 Apr 2023 06:20:00 +0000 Message-ID: References: <20230331154259.1447831-1-maxime.coquelin@redhat.com> <20230331154259.1447831-9-maxime.coquelin@redhat.com> In-Reply-To: <20230331154259.1447831-9-maxime.coquelin@redhat.com> Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SN6PR11MB3504:EE_|SN7PR11MB7439:EE_ x-ms-office365-filtering-correlation-id: 3402101e-a58e-4505-59f0-08db45551988 x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: LdjHCL4orsv3EGybZYmMfvOlRkpXoeGkMMVQt3IMUByJKmOJTQj8Dr1sw5DlGjRzOS8UfyD1WeR/sWtAiaX1d7l2QQ8L3VDZtJAPwHTpHPiDaWJ4yBjDJuipAx6Rryw2W2T8UjaZhfK40iB9z/s0mZ5M9EGFZeGG90FmbjVPUxWlPRzk5rGXWHAtB34ND2ZP4gn5/PkoMWetdF6HlYhWKJ8+HmQBsI1FHgtHuw0/gdZAePgGCR0JakUxwcskaWpIXGwcicuBDS6d7qUMfxZuCWC0crsf4qzL/EaaVWpzmF45z/NK+BeOvIIusDgZgYwRwfRibUxGhyl06ZN33VdqINBynw2uvHSF+D4+0j6WG1XS0KwVGUSZh9azRKfoNku2M9QYHsdG1CLfyp52b+REU/cTDxGOTvbGksssardXpB3xiExBHSxoIkbn83k5Q3oNTOX5atbLqsL4KNjcTEEh0CTchhoh/OIaS1JMaq2UlXueTr5GGwxL66LyhlV96D4CHPPKppMHJEZH5HnYawfSbjc3o9nPPfW3BxkCewixVTqTisnj8rIdzFEnrmNqrRTBJq4tfGmQ0QcxJY+ByxO3v6y9zC5nD8knNmjfjWRp5cSPmQxgb/l/bfBYafnb+t+LO5CLgNGi8rGm17Zbbkd5YA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR11MB3504.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(136003)(366004)(376002)(396003)(39860400002)(346002)(451199021)(82960400001)(316002)(64756008)(66446008)(38100700002)(41300700001)(122000001)(5660300002)(7416002)(52536014)(921005)(8936002)(8676002)(33656002)(86362001)(38070700005)(2906002)(66476007)(55016003)(66556008)(7696005)(71200400001)(53546011)(6506007)(26005)(9686003)(478600001)(83380400001)(186003)(76116006)(110136005)(66946007); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?XXA4oI6lB9Vl74QPGK1VMidDv6ALjQBV5YKvoYJ8GTOqbTct6U1/JkIu91qn?= =?us-ascii?Q?D2Mnixgaor3JadrNwDf4KgRMT5iRWfHh69zZsPCnqo5qgIHbeyLfMECSvqL/?= =?us-ascii?Q?b9E53rUVnSRwQ8a/1H4DPViK+yzZOK/FUQ0l0kDkAUeNkl0oK416JI5E4aeW?= =?us-ascii?Q?h3Q/zOxB5J+oAjn8icxZbuxWVFNglBd+EqdXrfa1Gq5NgseSS1C4IxpjAdCC?= =?us-ascii?Q?nO7jn23ZFem51tBWueDcFG602Wqola+mhMOnYyhoyUPRldXAsxKP/N1f8eDX?= =?us-ascii?Q?KnkVaKMFmX9lQgwrgncJklic6VjlspfO8u9aYeQqOwbK/P7Totx81ykXiLn7?= =?us-ascii?Q?gfPxDc5rbQzODDtVQvKh6x0RQoyYiI6ubybT7+68git9bzHVlqS5E8IU80Vo?= =?us-ascii?Q?Mt79CYU/ZiwlAuUXnY5N97KnC+PuBNoXbkydq0MG6rlbI6LwVixj/E+QSp7E?= =?us-ascii?Q?JXlCGCxIFk7D8i3nGhbg3oTxUySJu80N4ryasOrS/46+E6iazhwr2kwgs+YQ?= =?us-ascii?Q?9mYzW1gvuuxjFQJAPYtwD668fD4GwPoT4xs14Z1LU0eg5gTLo2iA4i1UCxLX?= =?us-ascii?Q?B+YteXSpAg1EEHJWG57cVSqHSL+ksQE+7gCU4k0ONN8DE+V5KKJirIyt1Pp7?= =?us-ascii?Q?IwibtxDTiu9KCIvwvP9lRSUbXWTdA+tsumYnPQJcu2UhZ2CqjCBw8W3EUxLY?= =?us-ascii?Q?1ggCfFG3KcvJ8QUvR6oMVNCWFm9xJEHQQO6yqanwl3/Gg75Q4vKi6C+HKdKJ?= =?us-ascii?Q?zHmGbV3CIJuXE5QPui33vvjaYhMRLy71YfGKBlyS5AVChJAVFnkPGKHeyUjm?= =?us-ascii?Q?fQKZPTmtpVRJEPnolQ0pXgO4Ltmt9LqlTSvYomf9OpSLlF/WDKZYT54Jh+ir?= =?us-ascii?Q?KXW+cs8SA0o59FsOoTFCGle9WUl2lUheaHSuTZZxq6iM3qkAZG0Ps7DqxSQ1?= =?us-ascii?Q?o3rju+GocBV2ov4dwLoF7NxrlEvqZRUl1RKPxHYv3sTPkej79cH4Yv8gUMP+?= =?us-ascii?Q?SvgpLSez2elNT6yPSUhQPTgp13LtOElIEVsspYt2iDzyAXiqnKLhi/Zv8N5r?= =?us-ascii?Q?FjRoskpRqNNmFcrDlZoz7unnKhQ4Yd6dbBLvsYbdOomNZVvJOQKwBCx8L5SG?= =?us-ascii?Q?h/A/KZsPPyV/cxVBIP6Tj2TvXlp9f6K8mE5UOpXgGG31fHQr3KpiJgXniBIL?= =?us-ascii?Q?OSUVaz8al9Iej5xrgqyKZK89hr5OgjJoPqqlceYDJLcxzea/uv+1ionKDcag?= =?us-ascii?Q?5ufm4sOkXuwBrmLKY8Co87bBVDnhoenY6kt5GLCJ+kKtgRUJoAxXcx5QTbTz?= =?us-ascii?Q?tzXp0JAdyr9XK3uKJ4ocD88oPxPWs6DpnBRjCklcTkQ5B/SeYe/z9dBW8UWf?= =?us-ascii?Q?Mb5dLED6aMIScSDVuOW8C+we/r/u3IHMpxBMk/1kcV9EBYj0FEiH2TYPzSCu?= =?us-ascii?Q?DIzR1HSJ7qVemcgJZ8+2BAGsh4P9VBnNVH4ckx4/ycwQ0RtpeYPgVJXrHJ5A?= =?us-ascii?Q?IVKZcRuYNRwDPHZWq6ntAmYBm+Tww6r2HS7puTZ1p1dXh5D8izDO1q+SVheL?= =?us-ascii?Q?oNrg/S0vrDGNboDB7MYgU0AHpyfWiKT0HVgXgmGw?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR11MB3504.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3402101e-a58e-4505-59f0-08db45551988 X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Apr 2023 06:20:00.5927 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: roELsrii9R5e8CLyo4rcLU6JDmZw6Dtrtz5Ao1/VigqaE19f2ZcQHMbmG2GSxaqxdG2E0maUG9hc6ag0swQi7w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR11MB7439 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Maxime Coquelin > Sent: Friday, March 31, 2023 11:43 PM > To: dev@dpdk.org; david.marchand@redhat.com; Xia, Chenbo > ; mkp@redhat.com; fbl@redhat.com; > jasowang@redhat.com; Liang, Cunming ; Xie, Yongj= i > ; echaudro@redhat.com; eperezma@redhat.com; > amorenoz@redhat.com > Cc: Maxime Coquelin > Subject: [RFC 08/27] vhost: add offset field to IOTLB entries >=20 > This patch is a preliminary work to prepare for VDUSE > support, for which we need to keep track of the mmaped base > address and offset in order to be able to unmap it later > when IOTLB entry is invalidated. >=20 > Signed-off-by: Maxime Coquelin > --- > lib/vhost/iotlb.c | 30 ++++++++++++++++++------------ > lib/vhost/iotlb.h | 2 +- > lib/vhost/vhost_user.c | 2 +- > 3 files changed, 20 insertions(+), 14 deletions(-) >=20 > diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c > index a91115cf1c..51f118bc48 100644 > --- a/lib/vhost/iotlb.c > +++ b/lib/vhost/iotlb.c > @@ -17,6 +17,7 @@ struct vhost_iotlb_entry { >=20 > uint64_t iova; > uint64_t uaddr; > + uint64_t uoffset; > uint64_t size; > uint8_t perm; > }; > @@ -27,15 +28,18 @@ static bool > vhost_user_iotlb_share_page(struct vhost_iotlb_entry *a, struct > vhost_iotlb_entry *b, > uint64_t align) > { > - uint64_t a_end, b_start; > + uint64_t a_start, a_end, b_start; >=20 > if (a =3D=3D NULL || b =3D=3D NULL) > return false; >=20 > + a_start =3D a->uaddr + a->uoffset; > + b_start =3D b->uaddr + b->uoffset; > + > /* Assumes entry a lower than entry b */ > - RTE_ASSERT(a->uaddr < b->uaddr); > - a_end =3D RTE_ALIGN_CEIL(a->uaddr + a->size, align); > - b_start =3D RTE_ALIGN_FLOOR(b->uaddr, align); > + RTE_ASSERT(a_start < b_start); > + a_end =3D RTE_ALIGN_CEIL(a_start + a->size, align); > + b_start =3D RTE_ALIGN_FLOOR(b_start, align); >=20 > return a_end > b_start; > } > @@ -43,11 +47,12 @@ vhost_user_iotlb_share_page(struct vhost_iotlb_entry > *a, struct vhost_iotlb_entr > static void > vhost_user_iotlb_set_dump(struct virtio_net *dev, struct > vhost_iotlb_entry *node) > { > - uint64_t align; > + uint64_t align, start; >=20 > - align =3D hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); > + start =3D node->uaddr + node->uoffset; > + align =3D hua_to_alignment(dev->mem, (void *)(uintptr_t)start); >=20 > - mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, false, > align); > + mem_set_dump((void *)(uintptr_t)start, node->size, false, align); > } >=20 > static void > @@ -56,10 +61,10 @@ vhost_user_iotlb_clear_dump(struct virtio_net *dev, > struct vhost_iotlb_entry *no > { > uint64_t align, start, end; >=20 > - start =3D node->uaddr; > - end =3D node->uaddr + node->size; > + start =3D node->uaddr + node->uoffset; > + end =3D start + node->size; >=20 > - align =3D hua_to_alignment(dev->mem, (void *)(uintptr_t)node->uaddr); > + align =3D hua_to_alignment(dev->mem, (void *)(uintptr_t)start); >=20 > /* Skip first page if shared with previous entry. */ > if (vhost_user_iotlb_share_page(prev, node, align)) > @@ -234,7 +239,7 @@ vhost_user_iotlb_cache_random_evict(struct virtio_net > *dev) >=20 > void > vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova, > uint64_t uaddr, > - uint64_t size, uint8_t perm) > + uint64_t uoffset, uint64_t size, uint8_t perm) > { > struct vhost_iotlb_entry *node, *new_node; >=20 > @@ -256,6 +261,7 @@ vhost_user_iotlb_cache_insert(struct virtio_net *dev, > uint64_t iova, uint64_t ua >=20 > new_node->iova =3D iova; > new_node->uaddr =3D uaddr; > + new_node->uoffset =3D uoffset; > new_node->size =3D size; > new_node->perm =3D perm; >=20 > @@ -344,7 +350,7 @@ vhost_user_iotlb_cache_find(struct virtio_net *dev, > uint64_t iova, uint64_t *siz >=20 > offset =3D iova - node->iova; > if (!vva) > - vva =3D node->uaddr + offset; > + vva =3D node->uaddr + node->uoffset + offset; >=20 > mapped +=3D node->size - offset; > iova =3D node->iova + node->size; > diff --git a/lib/vhost/iotlb.h b/lib/vhost/iotlb.h > index 3490b9e6be..bee36c5903 100644 > --- a/lib/vhost/iotlb.h > +++ b/lib/vhost/iotlb.h > @@ -58,7 +58,7 @@ vhost_user_iotlb_wr_unlock_all(struct virtio_net *dev) > } >=20 > void vhost_user_iotlb_cache_insert(struct virtio_net *dev, uint64_t iova= , > uint64_t uaddr, > - uint64_t size, uint8_t perm); > + uint64_t uoffset, uint64_t size, uint8_t > perm); > void vhost_user_iotlb_cache_remove(struct virtio_net *dev, uint64_t iova= , > uint64_t size); > uint64_t vhost_user_iotlb_cache_find(struct virtio_net *dev, uint64_t > iova, > uint64_t *size, uint8_t perm); > diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c > index 81ebef0137..93673d3902 100644 > --- a/lib/vhost/vhost_user.c > +++ b/lib/vhost/vhost_user.c > @@ -2641,7 +2641,7 @@ vhost_user_iotlb_msg(struct virtio_net **pdev, > if (!vva) > return RTE_VHOST_MSG_RESULT_ERR; >=20 > - vhost_user_iotlb_cache_insert(dev, imsg->iova, vva, len, imsg- > >perm); > + vhost_user_iotlb_cache_insert(dev, imsg->iova, vva, 0, len, > imsg->perm); >=20 > for (i =3D 0; i < dev->nr_vring; i++) { > struct vhost_virtqueue *vq =3D dev->virtqueue[i]; > -- > 2.39.2 Reviewed-by: Chenbo Xia =20