From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E860443075; Tue, 15 Aug 2023 21:20:51 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 94BAD42D13; Tue, 15 Aug 2023 21:20:51 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 1EE584114B for ; Tue, 15 Aug 2023 21:20:50 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 37FHB3vN021010; Tue, 15 Aug 2023 12:20:49 -0700 Received: from nam10-bn7-obe.outbound.protection.outlook.com (mail-bn7nam10lp2104.outbound.protection.outlook.com [104.47.70.104]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3se9kjbf3g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 15 Aug 2023 12:20:49 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ssekg1IN8QqA6/Cu3mGWTH+HO2b1CXvRNGnFzzx6kCz4Z7AdXj3pnvKsZTlbTXjQSnjh4hFVtNN1j90acK/ueVSj7JkvvU6H7mOkQbhxstf6D92WyRBUbUl1MChA06hwHG5lTLXlmfAW5sZu20l3j7zBOFZ3D+2IjomeTzFTgHAt7TA4yRLjF98ps81ZfANKGVN7Wg5HyNFenacCq37aZqYVyr2B2bAEFpYYO9VD5kv++NUqrgjuLWDmZIN8WbzEaunC05DrIh/97mKI/okzuXvwbZDULM8IjgeI4YcIiMuWZsvB/vmt3aBRvZekaReCefLM1JTZqhaXthZT1QLuWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nNfekMufxbqGE3mUhNhaAKQcFatmfInSSjWdyYEjoDc=; b=FWxsZAjyg40FDr5GoEgDRw8K9n9V31eN5cbD7yOrR+BW9TPDcocOLo6e0xlMvrTCtAwywXMRxnrqpHnPr+fPdXGc/xnEsE+YHchbEjXw5oHECCpHe7wI8dhrh+QeuhxBi9hFqWmkk96xxTUCW/ae1GUs82F8BI3qUKWwD13kZ8bChRpI7rq/D7lxDfT6qpUjEz5P4neJN0V9exQIGLhZ6DUooN4oimStuOUxQfRGwLnVOn3uCPi4OyfMSfSdYRs7IFPez96JRAle8RtHpJE6P6BRorM7RWnDRbStigiK5cLljfYlmFt3kPFk8s5LyM668ROmDrUAQMmzMYV45HBCVA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nNfekMufxbqGE3mUhNhaAKQcFatmfInSSjWdyYEjoDc=; b=P7vfHXgA+YvHFW3QvBBS+BC1znRMx5aZHmbZvgB6jW/NfljmtvUcPAJ7xhwhQuxLLmMBc1VkaFOI1cCRmFXJpmAFw5axNgg0CYTEvFjyOs5s2KDK1mzLqUtuVtQmsVCSHsT0f13MGu/5DZ/byFBmLoqNBESvzSvNgqbWfiDjmn0= Received: from SJ0PR18MB4477.namprd18.prod.outlook.com (2603:10b6:a03:304::23) by PH0PR18MB5190.namprd18.prod.outlook.com (2603:10b6:510:161::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6678.24; Tue, 15 Aug 2023 19:20:42 +0000 Received: from SJ0PR18MB4477.namprd18.prod.outlook.com ([fe80::3ec5:2ee2:9a93:8eea]) by SJ0PR18MB4477.namprd18.prod.outlook.com ([fe80::3ec5:2ee2:9a93:8eea%6]) with mapi id 15.20.6678.025; Tue, 15 Aug 2023 19:20:41 +0000 From: Satananda Burla To: Anatoly Burakov , "dev@dpdk.org" CC: "bruce.richardson@intel.com" Subject: RE: [EXT] [PATCH v1 0/3] Add support for inter-domain DMA operations Thread-Topic: [EXT] [PATCH v1 0/3] Add support for inter-domain DMA operations Thread-Index: AQHZzG76Rty4r40pqkSldbnVeG6/qK/qSLGg Date: Tue, 15 Aug 2023 19:20:41 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SJ0PR18MB4477:EE_|PH0PR18MB5190:EE_ x-ms-office365-filtering-correlation-id: 0826f91b-3ba6-47db-721c-08db9dc4b738 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: eaR97+R3sYOhUqNbHrbVbr/oAhOIR+bTrwkE1Sc/kTTJfWWgWD7K0qaoafo6VBEqumkVw26f81blq1Zbcu4h99RuRuPTdxXDI0xqSmmKoWTe0j25fY/GZhWsCpkKFhSnNYDoHW8Qs99cqdRLWifSVPuwn5TWKPKXD5avx6+iy2JQC0RKNa8DXcr4cef8PUvdx0IELz2waFpfE6Vcvtx+EBGTH9BXljstfRWLRlEyt1kPPqy5gs93T54ixj0Su87egz0YZEVxAv0tIe3uuCG114+CAX650E+JJKYiC1gYWqZcUeztksMid4Qu46Nc0POa5uqfSdSh0GUNKD7UOsZCYOvGKrLxenagXzURSyiZ+tZm+JO7OwVKHwPu/NPosMkPSsJnYpZdDd410g7DLa8c9DpLBxiHmCU4XLxiV2kYWLdfq2JR5pmMdxbFRPTyXgF8PSj7RvL+IGdnRnAVerfqmQHPj0naN8rlcLr5RnQtr2PYo36s8+8dpekW1kFO/EXiisCcTUKTvHSaQc6f3qMsiXof380fS4tyfFHXR48U0juWIGOp7pZauRzKvsJ1AETkL4Q3d93eH7psdWIyDOZkycrrZ/JQ5vre+hdTN0o8BUa2CqwC8GzHVQViTGlhZ4A9 x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR18MB4477.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(376002)(396003)(366004)(39860400002)(346002)(136003)(451199024)(1800799009)(186009)(52536014)(7696005)(71200400001)(110136005)(86362001)(53546011)(2906002)(26005)(41300700001)(478600001)(64756008)(55016003)(8676002)(8936002)(316002)(6506007)(33656002)(83380400001)(38070700005)(66946007)(76116006)(66556008)(4326008)(5660300002)(66446008)(66476007)(122000001)(38100700002)(9686003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?LpZ+OabnYsLSl6QICrs02yQAxumA8GZOSQuahflkIaVuNA5YqZGOB6ua9ej2?= =?us-ascii?Q?ePELYpZbcx6YGKdUs5Wg3qUyZtFSk9RL5S8sptDCBwPIGBIe0rQfsPzJT6Cj?= =?us-ascii?Q?0KWc8g1ILIEaI7csYAYcNfA/ts+wseWWrrwxHFcTLGattU0UpJat6YdFQftj?= =?us-ascii?Q?ogjmmxz/YZV8yr/Nl+HqxKbJxaZnxdD9XmAzUoCY1cDSOGDop++fMb+Znt+2?= =?us-ascii?Q?imz+B3IYh8JlxombfesMnTW/jXMOroaZs4kgh0gnFboKdvM80C6dmZ4QBL1u?= =?us-ascii?Q?Rnl2SSkzXCwktbv7qCTrls3W+p4pdSHJ1TN7OPjHjhZYbgpQUsoZxeFJXbfI?= =?us-ascii?Q?A+OOkY524KB1XlYswmQqRp+KBubCEhxMWyspJiN1ECtp7pwYV676yeldketl?= =?us-ascii?Q?B3iygXtBeqUmbcq12qpTHKVcZP7xyIBByPWtYUZQOx96o97eX95HiB/kAZ7f?= =?us-ascii?Q?kY1Ci4A44nctxhDLHUp7HcGfnerH5xZdcKg3L//JnXQ7ep7TyBs4NIXBvoqE?= =?us-ascii?Q?Od3wpRMsU4KS4gWM7dcVxiUrwa9oLIpUrzfYJ4c2JpA/kYBXt6GusjG9xWPT?= =?us-ascii?Q?sufiSmvWNauw0ASKglUjk9e9hh6wxGlsJEsqMC29sz8dPDK+cbgIruqJFZ/y?= =?us-ascii?Q?w8bHgJVZaDF97/tHxrgMy6gyJ02obATXHq6fp+LRi1a6h/zAy+jf8jInvzMX?= =?us-ascii?Q?Zkq0jUJo8aAqyOhfQYuwNvj6aOMybqh0r0EdXbSLTPdl3+ZW+rwbFfnB+Jxv?= =?us-ascii?Q?NJZekJfwk7sJiZW9XRT0Yd7fcDtNgTRVHPV+tD6DX/WXYGkKpKfQJwvnrs9m?= =?us-ascii?Q?/jxEHdqe56HqOqYsMHl7m/y3SBfD7yC7Ijgt6SLJ82N/HAJx/Pk4ZwXXRPH3?= =?us-ascii?Q?FC/0TlOAhHZXYq9RKtG67LEl4Lec67ZXLwnYTbD9vDvnK9wHl+OBI04ENMix?= =?us-ascii?Q?qnq9Z+s6rPpMA1bHcLrCerJgvsQT3qNoTo3d+70HIlSswRVBmaIKTvl93Q9m?= =?us-ascii?Q?Meolj12ri11c36a6+87uC8c+DJAhdEKR05qurEcYWFfYilFcYC2SI+shS9GY?= =?us-ascii?Q?JP4rztvlMg6Dc1eUMQQoTuWZ4xgWPfbMR9NC8x9ZWj1E+2rEvEPwzmkqZm2V?= =?us-ascii?Q?GJISNBhkm1d7KRqjmegbPk3onNrjWLALiMqhjyizrFUVS/mNvG/2XVIm2ScQ?= =?us-ascii?Q?nGvW1RQXFIKyI3uvFyFTdQRZ+DYKGd7syXSnEOngJAfz4t2bpALaZ8j/s1xI?= =?us-ascii?Q?5UC50rcwJs9IO11cZks+jiacSU0mjYOG7J9DteK1pHVBLCXclnFNWFZ2EPGn?= =?us-ascii?Q?ACUNELUWV9j/+OfrU92oG3iNXStb4y8cZ/MTJKEVd4nDXEY5TholfkNoxicX?= =?us-ascii?Q?/BTWrXIdj5mc0Xka/7keXv8SIdZzPZEZwceTxr0Wee7uCdtxZySICkaOIk1f?= =?us-ascii?Q?5nB7MNteA3O1MMEy9Ct2XF9c1/32boc2K+9MhxE81j4plnbiueOCeRD9UO4O?= =?us-ascii?Q?JOw3N7/zl+6Mo05Df7yxB4ZanTSW8+yO14OlwOTpW1/jdYLXH54OAfxWvveg?= =?us-ascii?Q?GlXuypqNjv2OPxD9VHJHpVckrGGDwyxmiIkjlUe8?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: marvell.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SJ0PR18MB4477.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0826f91b-3ba6-47db-721c-08db9dc4b738 X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Aug 2023 19:20:41.6098 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: J6JVEnH7JObfYyHyj9votgTgAxlf2n06jVep+twhvul4L1h+UzhoRuaCMEdumNZhF+4tvqcdqdykuLHrGzHZFQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR18MB5190 X-Proofpoint-GUID: TRvfZz5nsrcU9u7aLYkC3qVM1SX_jE7Y X-Proofpoint-ORIG-GUID: TRvfZz5nsrcU9u7aLYkC3qVM1SX_jE7Y X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.957,Hydra:6.0.601,FMLib:17.11.176.26 definitions=2023-08-15_17,2023-08-15_02,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Anatoly > -----Original Message----- > From: Anatoly Burakov > Sent: Friday, August 11, 2023 9:15 AM > To: dev@dpdk.org > Cc: bruce.richardson@intel.com > Subject: [EXT] [PATCH v1 0/3] Add support for inter-domain DMA > operations >=20 > External Email >=20 > ---------------------------------------------------------------------- > This patchset adds inter-domain DMA operations, and implements driver > support > for them in Intel(R) IDXD driver. >=20 > Inter-domain DMA operations are similar to regular DMA operations, > except that > source and/or destination addresses will be in virtual address space of > another > process. In this patchset, DMA device is extended to support two new > data plane > operations: inter-domain copy, and inter-domain fill. No control plane > API is > provided for dmadev to set up inter-domain communication (see below for > more > info). Thanks for posting this. Do you have usecases where a process from 3rd domain sets up transfer=20 between memories from 2 domains? i.e process 1 is src, process 2 is dest and process 3 executes transfer. The SDXI spec also defines this kind of a transfer. Have you considered extending rte_dma_port_param and rte_dma_vchan_conf to represent interdomain memory transfer setup as a separate port type like RTE_DMA_PORT_INTER_DOMAIN ? And then we could have a separate vchan dedicated for this transfer. The rte_dma_vchan can be setup with separate struct rte_dma_port_param each for source and destination. The union could be extended to provide the necessary information to pmd, this could be set of fields that would be needed by different architectures like controller id, pasid, smmu streamid and substreamid etc, if an opaque handle is needed, it could also be accommodated in the union. These transfers could also be initiated between 2 processes each having 2 dmadev VFs from the same PF as well. Marvell hardware supports this mode. Since control plane for this can differ between PMDs, it is better to setup the memory sharing outside dmadev and only pass the fields of interes= t to the PMD for completing the transfer. For instance, for PCIe EP to Host DMA transactions (MEM_TO_DEV and DEV_TO_MEM), the process of setting up shared memory from PCIe host is not part of dmadev. If we wish to make the memory sharing interface as a part of dmadev, then preferably the control plane has to be abstracted to work for all the modes and architectures. Regards Satananda