From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 19DC0A052E; Mon, 3 Feb 2020 17:11:53 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 097A71BFB2; Mon, 3 Feb 2020 17:11:43 +0100 (CET) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2125.outbound.protection.outlook.com [40.107.236.125]) by dpdk.org (Postfix) with ESMTP id E1FF41BFF8 for ; Mon, 3 Feb 2020 10:16:01 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FD+qg6nZRX6mMXo8IZ1zpuRHYR8HsAnNTzvOLFZ4RB4+4X+ZaZzDV4Be/jns2lsRbJMlHfTsMkPh1c6blOQjr5DrqQYcANGuF1x8BXO3AjnlhcnuFwYeP3JKsKFB19VfivlfcLrVuXQwPDXjCho2ng0yK4YaNc8yl96m3aqJm0slTSJLjePTRQC8DyGwm+2mlp5OiNy0Ns4Vb+gvW5Qyb1rJmPqflZN8GoqtOnBsWrrZ702qm+EFwGmxG+y2woQuljjslJ8hBFbsG8cIM/QsAruwChUr96l2vROuT56p8vhvDiSAhEdgWuS1sDgtOp8zhoRmVmhyvt+IYpZ0AOI1bQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZHeUY+8ht64nlFg37HFP3XB4kQnEnj/W8R91jeVVON4=; b=gj+2uNyEuEdIhKzsx1TrQEHAyFdXoJu851eQyQn3D/9gk+WnAMTfZCbq5FqnoQ1jyVEqTsMMNCx/MD1cGRvU4tCGWVJd40jfrwvyBz840t1fVIfp43jMgm/vnu9MFhyhKsXNVNrriCTrnVTKN1OBDl7ThAnUAfP3GbVRzEuBrgqMZiUQRVcNnO3lkBGZDI9cj+erXN6jbuY4sILrbjCWTTnFy0kLezNwkK3fZ72xf0tYA0rY5Gj4wON8vIMnbT7aH4b3HxOTUookuw5ZyYz+A6C3Tyse+xjNJDZbdVW7BIToWI8jaeNt4W065J+G8vYESm6vNu408+UYg0Q/eEOSaQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZHeUY+8ht64nlFg37HFP3XB4kQnEnj/W8R91jeVVON4=; b=PiQ0272my1wXb2AUgY+BOdKFZpkuGLwqKCHQKjuLruJY5wsStZS0lwAmVzyDkrk+4aAcdmm3BKqBzYUpIHaNcD0I0quPfpqKacuhfTItg3rZso/8TSkTTcgHAh2T9ceRLOkEFy2Bbb+1UeZ+A5NNsaTAHT0s8SjzwXA2aBRaG18= Received: from BN6PR21MB0836.namprd21.prod.outlook.com (10.173.205.14) by BN6PR21MB0177.namprd21.prod.outlook.com (10.173.200.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2707.7; Mon, 3 Feb 2020 09:16:00 +0000 Received: from BN6PR21MB0836.namprd21.prod.outlook.com ([fe80::a8e2:9c14:d8e3:753f]) by BN6PR21MB0836.namprd21.prod.outlook.com ([fe80::a8e2:9c14:d8e3:753f%11]) with mapi id 15.20.2707.018; Mon, 3 Feb 2020 09:15:59 +0000 From: Stephen Hemminger To: Dmitry Kozliuk , "dev@dpdk.org" CC: Thomas Monjalon , Pallavi Kadam , Anatoly Burakov , Ranjit Menon , Harini Ramakrishnan Thread-Topic: [EXTERNAL] Windows Support Plan Thread-Index: AQHV2gjP7+d/SP1tCE6k4UOww4b8g6gJMChq Date: Mon, 3 Feb 2020 09:15:59 +0000 Message-ID: References: <20200202233736.74bdf47f@Sovereign> In-Reply-To: <20200202233736.74bdf47f@Sovereign> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=True; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2020-02-03T09:12:52.5385472Z; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Privileged authentication-results: spf=none (sender IP is ) smtp.mailfrom=sthemmin@microsoft.com; x-originating-ip: [172.58.139.126] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 808f44e8-a156-48da-2179-08d7a889af2e x-ms-traffictypediagnostic: BN6PR21MB0177:|BN6PR21MB0177: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-forefront-prvs: 0302D4F392 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(4636009)(136003)(376002)(396003)(346002)(366004)(39860400002)(199004)(189003)(71200400001)(4326008)(316002)(52536014)(66446008)(91956017)(186003)(66946007)(64756008)(76116006)(66476007)(66556008)(9686003)(86362001)(55016002)(53546011)(26005)(6506007)(478600001)(81156014)(5660300002)(81166006)(33656002)(8676002)(8936002)(966005)(107886003)(110136005)(54906003)(10290500003)(8990500004)(2906002)(7696005)(21314003); DIR:OUT; SFP:1102; SCL:1; SRVR:BN6PR21MB0177; H:BN6PR21MB0836.namprd21.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 4YhGth1RuMM3bxAAUl2goKDywYTTV/f3YlPz/3neH712seQM8B1d5IQ8it8MsSWgAUPib3lvHJqrgR9CsxXy42W0TzlKsHy7B0f4q6XMze7ireD3PnoVHJlstP+FEXMacH91x2gUY28rrN6QGbw3oQsU6H8ZZgg5LjbahpQSpjuYUSxJzPGe60k6X1ZGPK7uSM9nkzOSKzvVNLm7rEjXgcjUJrrljYdyFRCyL9m2X7OArcYXvRF3x0QnLBxYAhXxP2cWZcX9s2CxWMHCU6aU//wgzDMNgumdufmDxnfTZmyaMgVY1aqVgEaH5fHsGauxsSFN2gj4RJlEgp3BPejFI5Fy0wYk+BCQCqkFp6euFug0K/+uMAVsxiX9P+NWY8955l22EKg8lDKf0Clj7yJ4qIXjw17SNLYQWYAt0Kx+m/SrJhHXC6nG6uI642Rq5Yp2t3B0/iG6V6zIi2wkeidg+l4IG3o2C5qddQxGiQtJZVrQJoMtlYOTs903y2eF0D3J2lDqzaQnCSDkfC+KPTSOwOIDQM2Dm+H8osLETn1CyaUx++Jcb6KgdijjKtc62wfk x-ms-exchange-antispam-messagedata: plN20kVrj96xF8XRllTAZvaoRvfKjdFor66HoKc4HQknDwbjOwQYzUXUn1n2+dU7O+Vb/2SUcAKhtuxAb4r+SIOX27OgJYWE7/skFIdO5HV1tvqaEdzsC+2/KSH4jd8gJ1u/J8/FoIgbcrKuKjz2Kg== MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: 808f44e8-a156-48da-2179-08d7a889af2e X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Feb 2020 09:15:59.8136 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 8sxWGtQz5cw+2zjRUCA4Loh8AxxT9CVYPPP6HvhiNK7pvmdbz+N+MNAUM1dzs8JxEwVfu9hoZu8X+5xWYyoOeg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR21MB0177 X-Mailman-Approved-At: Mon, 03 Feb 2020 17:11:40 +0100 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [EXTERNAL] Windows Support Plan X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" You should talk to the Windows DPDK developers. They have been presenting regularly at dpdk summits. Look up videos for mor= e info. The initial port is focused on running DPDK on bare metal with Intel NIC. Y= our version looks more aligned with Windows as guest in KVM. Get Outlook for Android ________________________________ From: Dmitry Kozliuk Sent: Sunday, February 2, 2020 9:37:36 PM To: dev@dpdk.org Cc: Thomas Monjalon ; Pallavi Kadam ; Anatoly Burakov ; Ranjit Menon ; Harini Ramakrishnan ; = Stephen Hemminger Subject: [EXTERNAL] Windows Support Plan Hi everyone! Where do I find a high-level plan of comprehensive Windows support: design decisions, implementation order, etc? Information on the subject is very scarce, one may think it is abandoned. Googling for "site:dpdk.org/ml/archives/dev/ windows" yields only two pages of disjoint messages. I learned about "netuio" days ago from a tiny remark = in a "Minutes of Technical Board Meetings" email, and even then it took enumerating "dpdk-next-windows" branches to find the source. The matter is, as a New Year's holiday project of mine I implemented Window= s support from scratch to the point it runs in QEMU with virtio-pci [0]. It i= s not of production quality, cuts some corners and lacks major features (see bottom). My primary goal was fun^W making it work. Comparing it to "windpdk-v18.08" branch of "dpdk-next-windows", I can see that 1) our implementations take rather different approaches in some cases, and 2) both have severe issues and would benefit from amalgamation. I'd like to contribute to Windows support with this code, but to do so, coordination is required, because changes are significant. Primary topics to discuss: 1. Memory management (@Anatoly) 1.1. MM changed radically since v18.08 and dpdk-next-windows does not implement it properly anyway, it allocates segment lists in a PCI b= us driver. My implementation closely follows the Linux one using VirtualAlloc2() with XXX_PLACEHOLDER flags to reserve and commit memory, but does not map hugepages to files. Is there a consensus on MM approach in Windows? Anyway, I think EAL private MM API would have to be changed, because memory reservation, allocation, and mapping are completely different operations. Hiding this with an mmap() shim doesn't look right, because mmap()'s behavior differs even among Unix platforms. 1.2. In Windows, there is no /dev/mem to implement rte_virt2iova(), so a simple kernel driver is required for mapping. Moreover, Windows kernel abstracts IOMMU, so those physical addresses may be unsuitable for DMA at all (see below). 2. Kernel drivers (@Harini, @Stephen) 2.1. The most serious issue is that Windows formally prohibits using arbitrary physical addresses with DMA in favor of allocating special buffers (presumably because IOMMU may be engaged, and there is no way to check). We can either live with it (technically, everything works with PA mode), or we could revive DMA allocation API from ethdev to ask the driver for a proper DMA buffer. 2.2. Neither netuio, nor my driver (userpci) support interrupts. I see not inherent difficulty here, but interface should be designed carefully. 2.3. Windows allows mapping I/O ports into user-space, but there is no API to change IOPL, which makes mapping useless and requires a syscall for every I/O port access. This demolishes virtio-legacy performance. Perhaps Microsoft could give some advice here. OTOH, PIO is all legacy, so might be much effort is not justified. 2.4. I believe GUIDs approach for identifying compatible devices should be strictly preferred, and not DosDevices symlinks. Think of Mellanox OFED on Linux, which uses a different driver, but could provide a compatible interface. Another reason is that a single driver can implement multiple kernel interfaces with appropriate GUIDs. 2.5. DPDK Windows driver guidelines, driver review, and certification. The quality of both netuio and userpci is below standards now (e. g. netuio does not mind its context when mapping memory, and userpci lacks synchronization), code style is a mix of Windows and DPDK, logging may be insufficient. 3. POSIX shim vs EAL wrappers (@Thomas, @Pallavi, @Ranjit) What is the policy: to implement a POSIX shim in EAL (as the latest patches from Pallavi Kadam do), or to add dependencies (as [1] suggests)= ? IMO creating a shim is wrong. First, some POSIX concepts do not easily map to Windows, like poll() interface and I/O model in general. Second, there are numerous getopt, pthread, etc. implementations for Windows, no point wasting resources and repeat them, adding bugs. I can think of two exceptions: * , which is header-only. * Berkeley sockets. Adding to public headers creates more trouble that its worth: definitions for a few structures and constants. May be there should be some to abstract platform differences. Some highlights on my implementation: * Major features NOT supported: * multi-process (due to limited time) * interrupts (limited time + explained above) * eventdev (requires access to physical memory) * hot-plug (due to limited time and Windows knowledge) * bbdev (see comments in config/common_windows) * FreeBSD (trivial, I just don't use it) * DPDK is built using MinGW-w64 with GNUmake or Meson. Drivers are built using DDK (msbuild or Visual Studio). Actually, I cross-compile DPDK and build drivers natively. * Only tested on Windows 10 in QEMU with virtio-legacy. * No docs, but there's nothing unusual for those familiar with Windows. Bind virt2phys driver to Root\virt2phys, bind userpci driver to device(s)= . * Commit history is squashed, because it was a mess from experiments. There also may be some leftover changes, but those commits are not proper patches anyway. References: [0]: https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgi= thub.com%2FPlushBeaver%2Fdpdk%2Fcommits%2Fwindows&data=3D02%7C01%7Csthe= mmin%40microsoft.com%7C485559de220c43a1fe2408d7a81fd5e9%7C72f988bf86f141af9= 1ab2d7cd011db47%7C1%7C0%7C637162727454625299&sdata=3DW%2BrqF4EWaBmwEOb7= t3fRrKfmu7GkHpIyNJ2us6Dx6QU%3D&reserved=3D0 [1]: https://nam06.safelinks.protection.outlook.com/?url=3Dhttp%3A%2F%2Fmai= ls.dpdk.org%2Farchives%2Fdev%2F2015-February%2F014245.html&data=3D02%7C= 01%7Csthemmin%40microsoft.com%7C485559de220c43a1fe2408d7a81fd5e9%7C72f988bf= 86f141af91ab2d7cd011db47%7C1%7C0%7C637162727454625299&sdata=3DHb%2FCD99= bjzhDlfrcbKdBN%2FlFkqQyN3F%2BvYlPl1VIz8w%3D&reserved=3D0 -- Dmitry Kozlyuk