From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3901DA04B1; Fri, 14 Aug 2020 01:24:57 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B02221C1B4; Fri, 14 Aug 2020 01:23:03 +0200 (CEST) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by dpdk.org (Postfix) with ESMTP id DBCD21C0CF for ; Fri, 14 Aug 2020 01:22:44 +0200 (CEST) Received: by linux.microsoft.com (Postfix, from userid 1059) id B3F4120B4916; Thu, 13 Aug 2020 16:22:43 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com B3F4120B4916 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1597360963; bh=SLZXc7axZJl04uKHnM5I4NTHS4bojddh/qwBYadFnRw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=M8GOYlw8FR+VxV5YFp04rlD5FT3x9XLvGEsqeuwP759FHbekPexcp+45zPqlpF+Rc Ugf+oKMOrv57fQAq8/hZFsVYMbBecQGaPVhxON5xW7LLwVfbJXktpT6j4QRmo+vdCK 7Q6Bg9mb9Wcc91QY6oMjxi2sX20NlQTfliSwEvjg= From: Narcisa Ana Maria Vasile To: dev@dpdk.org, thomas@monjalon.net, haramakr@linux.microsoft.com, ocardona@microsoft.com, pallavi.kadam@intel.com, dmitry.kozliuk@gmail.com Cc: ranjit.menon@intel.com, dmitrym@microsoft.com, Harini Ramakrishnan Date: Thu, 13 Aug 2020 16:21:29 -0700 Message-Id: <1597360905-74106-7-git-send-email-navasile@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1597360905-74106-1-git-send-email-navasile@linux.microsoft.com> References: <1597360905-74106-1-git-send-email-navasile@linux.microsoft.com> Subject: [dpdk-dev] [PATCH 06/22] Windows DPDK libraries and applications have now been updated to the latest public release v18.08, of the main DPDK source. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Harini Ramakrishnan --- kernel/windows/netuio/netuio.inf | 118 +++++++++ kernel/windows/netuio/netuio.rc | Bin 0 -> 4772 bytes kernel/windows/netuio/netuio_dev.c | 306 +++++++++++++++++++++++ kernel/windows/netuio/netuio_dev.h | 65 +++++ kernel/windows/netuio/netuio_drv.c | 142 +++++++++++ kernel/windows/netuio/netuio_drv.h | 33 +++ kernel/windows/netuio/netuio_interface.h | 69 +++++ kernel/windows/netuio/netuio_queue.c | 271 ++++++++++++++++++++ kernel/windows/netuio/netuio_queue.h | 30 +++ kernel/windows/netuio/resource.h | 14 ++ 10 files changed, 1048 insertions(+) create mode 100644 kernel/windows/netuio/netuio.inf create mode 100644 kernel/windows/netuio/netuio.rc create mode 100644 kernel/windows/netuio/netuio_dev.c create mode 100644 kernel/windows/netuio/netuio_dev.h create mode 100644 kernel/windows/netuio/netuio_drv.c create mode 100644 kernel/windows/netuio/netuio_drv.h create mode 100644 kernel/windows/netuio/netuio_interface.h create mode 100644 kernel/windows/netuio/netuio_queue.c create mode 100644 kernel/windows/netuio/netuio_queue.h create mode 100644 kernel/windows/netuio/resource.h diff --git a/kernel/windows/netuio/netuio.inf b/kernel/windows/netuio/netuio.inf new file mode 100644 index 000000000..0453b371a --- /dev/null +++ b/kernel/windows/netuio/netuio.inf @@ -0,0 +1,118 @@ +; +; BSD LICENSE +; +; Copyright(c) 2010-2018 Intel Corporation. All rights reserved. +; All rights reserved. +; +; Redistribution and use in source and binary forms, with or without +; modification, are permitted provided that the following conditions +; are met: +; +; * Redistributions of source code must retain the above copyright +; notice, this list of conditions and the following disclaimer. +; * Redistributions in binary form must reproduce the above copyright +; notice, this list of conditions and the following disclaimer in +; the documentation and/or other materials provided with the +; distribution. +; * Neither the name of Intel Corporation nor the names of its +; contributors may be used to endorse or promote products derived +; from this software without specific prior written permission. +; +; THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +; "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +; LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +; A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +; OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +; SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +; LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +; DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +; THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +; (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +; OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +; +; +; netuio.inf +; + +[Version] +Signature="$WINDOWS NT$" +Class=Net +ClassGuid={4d36e972-e325-11ce-bfc1-08002be10318} +Provider=%ManufacturerName% +CatalogFile=netuio.cat +DriverVer= + +;***************************************** +; Install Section +;***************************************** + +[Manufacturer] +%ManufacturerName%=Standard,NT$ARCH$ + +[Standard.NT$ARCH$] +%F1583.netuio.Description%=netuio_Device, PCI\VEN_8086&DEV_1583 +%F158A.netuio.Description%=netuio_Device, PCI\VEN_8086&DEV_158A ; I40E_DEV_ID_25G_B +%F158B.netuio.Description%=netuio_Device, PCI\VEN_8086&DEV_158B ; I40E_DEV_ID_25G_SFP28 +%F37D0.netuio.Description%=netuio_Device, PCI\VEN_8086&DEV_37D0 +%F153B.netuio.Description%=netuio_Device, PCI\VEN_8086&DEV_153B + +[netuio_Device.NT] +CopyFiles=Drivers_Dir + +[Drivers_Dir] +netuio.sys + +;-------------- Service installation +[netuio_Device.NT.Services] +AddService = netuio,%SPSVCINST_ASSOCSERVICE%, netuio_Service_Inst + +; -------------- netuio driver install sections +[netuio_Service_Inst] +DisplayName = %netuio.SVCDESC% +ServiceType = 1 ; SERVICE_KERNEL_DRIVER +StartType = 3 ; SERVICE_DEMAND_START +ErrorControl = 1 ; SERVICE_ERROR_NORMAL +ServiceBinary = %12%\netuio.sys + +; +;--- netuio_Device Coinstaller installation ------ +; + +[DestinationDirs] +DefaultDestDir = 12 +netuio_Device_CoInstaller_CopyFiles = 11 + +[netuio_Device.NT.CoInstallers] +AddReg=netuio_Device_CoInstaller_AddReg +CopyFiles=netuio_Device_CoInstaller_CopyFiles + +[netuio_Device_CoInstaller_AddReg] +HKR,,CoInstallers32,0x00010000, "WdfCoInstaller$KMDFCOINSTALLERVERSION$.dll,WdfCoInstaller" + +[netuio_Device_CoInstaller_CopyFiles] +WdfCoInstaller$KMDFCOINSTALLERVERSION$.dll + +[SourceDisksNames] +1 = %DiskName%,,,"" + +[SourceDisksFiles] +netuio.sys = 1,, +WdfCoInstaller$KMDFCOINSTALLERVERSION$.dll=1 ; make sure the number matches with SourceDisksNames + +[netuio_Device.NT.Wdf] +KmdfService = netuio, netuio_wdfsect +[netuio_wdfsect] +KmdfLibraryVersion = $KMDFVERSION$ + +[Strings] +SPSVCINST_ASSOCSERVICE= 0x00000002 +ManufacturerName="Intel" +ClassName = "Intel(R) DPDK netUIO Driver" +DiskName = "DPDK netUIO Installation Disk" +F1583.netuio.Description = "DPDK netUIO for Intel(R) Ethernet Converged Network Adapter XL710-Q2" +F158A.netuio.Description = "DPDK netUIO for Intel(R) Ethernet Network Adapter XXV710 for 25GbE backplane" +F158B.netuio.Description = "DPDK netUIO for Intel(R) Ethernet Network Adapter XXV710-DA1" +F37D0.netuio.Description = "DPDK netUIO for Intel(R) Ethernet Connection X722" +F153B.netuio.Description = "DPDK netUIO for Intel(R) Ethernet Connection I217-V" +netuio.DeviceDesc = "netuio Device" +netuio.SVCDESC = "netuio Service" diff --git a/kernel/windows/netuio/netuio.rc b/kernel/windows/netuio/netuio.rc new file mode 100644 index 0000000000000000000000000000000000000000..4b0b176e6284fcfa65662f7ea6b3b1b36fb85a6d GIT binary patch literal 4772 zcmdUy+iuf95QgU(iFYvR4G7dGEg-mTlQdMM2Q_H`sZsEes0;8MRsUc_R%I5Tf%)|Q`UN{B{sA7Heyz{$6S$J+R(<_ z6V|7U1J)*t;+ygIk@tw1q+QuHG@Ewc?y<|38#h=T^K}IEz|mF;{G0a4Dvny58;^$gd4Sb*>=YhN*0;GAJ@RKCe0BJGzC8Zxhgi_aN5>4T6iFA@eYe;M z$OpOWH3sb>ygMfUMtCYA1|oZi&&SL{9(H*1S((_Qk|;6Z&FT@WXN)Zur(wm@wUB*4uD-G- z$nHS5$BO4!r0Db2XSI(2*|IzWlpj4TdCjhk|E?C4XL*PTjk;@KR!$kY?|azZa+N~V z(flf}yysS(S6}(hBPK4@mAmXwR`DxVtTz=qr*54S(Z2SlS}xOQ=Z+e!y3dBI#{)7= zHO6165oPtJ7;V8v>s3{<+DJpq!>F@gCwzj%G5!ea!IB=?*N3J_o_E|Ce8gOq?~`ZZ z&VC8&;*Ov=Ag`MXG!ZfP#GZ3&eCBqHIOk81*C5;LL|$HMM9h0?aa89VLHfR{9WAsw zq=t0JeeLkgcJN@^(JCs_s-f!Eq~e^BC3!tlNnS$jhJQ#UPQdAQ`f7$==ahEMa~z|e zm;DqSbv>Q)q-b^NXHL-2({0;_0)?-bdAhK7dG@Qm9)ql~si4CYkE=Jk#79@yHpEk7 zw2c??v0Pfni@a3XQni=PrVl*xnVs^KRj2SZj6FQ5E=MR-&|BTmHRs*fz7u)xnGIRj z=}~c->s0ieYm8;r(EFa%$8@i4o&rLQ#4g~s&i7feqn9G zOn7Q+LZLoxg8rgbhpk+b7h?1l{>s3+=$X&nfJv23{5iMAXDi8790*P6s=t9nw7ms0 ztc;PJ;E&bxZX^5BNfrjz@C&g^@R{CtwD?8+Pu=w++t^gI7tZ%V*X5qy1nXUga729x z`y{NMqK7hi;=DD$mQ~O{o@=cf5=tRzG|R^3-=cNGJ~2@><1cDj@I`~&)CH<~ju^FD zL|hJ?O(Tb|w2?<6qE+2v8kKLDt4hV3&#U<`uT^;j*5=Wt2wYYR(w-yaVsRy3V|rLQ zxj|e0U$tiCD@zw?!Yy(5O+iF?>9a^@h_7hM`5HZu>b0-uUf)#P^iAKF_YnW4tIwY2 z>-8?{v$sEaZ~JE}_E76(QC1sSrg<3u_U~Oi?d^Z>0n4lw>-8=Tm&f~ +#include "netuio_drv.h" + +#include + +#ifdef ALLOC_PRAGMA +#pragma alloc_text (PAGE, netuio_create_device) +#pragma alloc_text (PAGE, netuio_evt_device_context_cleanup) +#pragma alloc_text (PAGE, netuio_map_hw_resources) +#pragma alloc_text (PAGE, netuio_free_hw_resources) +#endif + +/* +Routine Description: + Worker routine called to create a device and its software resources. + +Return Value: + NTSTATUS + */ +NTSTATUS +netuio_create_device(_Inout_ PWDFDEVICE_INIT DeviceInit) +{ + WDF_OBJECT_ATTRIBUTES deviceAttributes; + WDFDEVICE device; + NTSTATUS status; + + PAGED_CODE(); + WDF_OBJECT_ATTRIBUTES_INIT_CONTEXT_TYPE(&deviceAttributes, NETUIO_CONTEXT_DATA); + + // Set the device context cleanup callback. + // This function will be called when the WDF Device Object associated to the current device is destroyed + deviceAttributes.EvtCleanupCallback = netuio_evt_device_context_cleanup; + + status = WdfDeviceCreate(&DeviceInit, &deviceAttributes, &device); + + if (NT_SUCCESS(status)) { + // Create a device interface so that applications can find and talk to us. + status = WdfDeviceCreateDeviceInterface(device, &GUID_DEVINTERFACE_netUIO, NULL); + + if (NT_SUCCESS(status)) { + // Retrieve and store PCI information + status = get_pci_device_info(device); + } + + if (NT_SUCCESS(status)) { + // Create a symbolic link name for user-space access + status = create_device_specific_symbolic_link(device); + } + + if (NT_SUCCESS(status)) { + // Initialize the I/O Package and any Queues + status = netuio_queue_initialize(device); + } + + if (NT_SUCCESS(status)) { + // Allocate physically contiguous memory for user process use. We'll map it later + status = allocate_usermemory_segment(device); + } + } + + return status; +} + +/* +Routine Description: + Free all the resources allocated in AdfEvtDeviceAdd. + +Return Value: + None + */ +VOID +netuio_evt_device_context_cleanup(_In_ WDFOBJECT Device) +{ + free_usermemory_segment(Device); + return; +} + +NTSTATUS +netuio_map_hw_resources(_In_ WDFDEVICE Device, _In_ WDFCMRESLIST Resources, _In_ WDFCMRESLIST ResourcesTranslated) +{ + UNREFERENCED_PARAMETER(Resources); + + NTSTATUS status = STATUS_SUCCESS; + + PNETUIO_CONTEXT_DATA netuio_contextdata; + netuio_contextdata = netuio_get_context_data(Device); + + if (!netuio_contextdata) + return STATUS_UNSUCCESSFUL; + + PCM_PARTIAL_RESOURCE_DESCRIPTOR descriptor; + UINT8 bar_index = 0; + + // Collect device BAR resources from the ResourcesTranslated object + for (ULONG idx = 0; idx < WdfCmResourceListGetCount(ResourcesTranslated); idx++) { + descriptor = WdfCmResourceListGetDescriptor(ResourcesTranslated, idx); + if (!descriptor) { + status = STATUS_DEVICE_CONFIGURATION_ERROR; + goto end; + } + + switch (descriptor->Type) { + case CmResourceTypeMemory: + // Retrieve and map the BARs + netuio_contextdata->bar[bar_index].base_addr.QuadPart = descriptor->u.Memory.Start.QuadPart; + netuio_contextdata->bar[bar_index].size = descriptor->u.Memory.Length; + netuio_contextdata->bar[bar_index].virt_addr = + MmMapIoSpace(descriptor->u.Memory.Start, descriptor->u.Memory.Length, MmNonCached); + + if (netuio_contextdata->bar[bar_index].virt_addr == NULL) { + status = STATUS_UNSUCCESSFUL; + goto end; + } + + bar_index++; + break; + + // Don't handle any other resource type + // This could be device-private type added by the PCI bus driver. + case CmResourceTypeInterrupt: + default: + break; + } + } + + // Allocate an MDL for the device BAR, so that we can map it to the user's process context later... + if (status == STATUS_SUCCESS) { + // Bar 0 is typically the HW BAR + if (netuio_contextdata->bar[0].virt_addr) { + netuio_contextdata->dpdk_hw.mdl = IoAllocateMdl(netuio_contextdata->bar[0].virt_addr, (ULONG)netuio_contextdata->bar[0].size, FALSE, FALSE, NULL); + if (!netuio_contextdata->dpdk_hw.mdl) { + status = STATUS_INSUFFICIENT_RESOURCES; + goto end; + } + netuio_contextdata->dpdk_hw.mem.size = netuio_contextdata->bar[0].size; + } + } + +end: + return status; +} + +VOID +netuio_free_hw_resources(_In_ WDFDEVICE Device) +{ + PNETUIO_CONTEXT_DATA netuio_contextdata; + netuio_contextdata = netuio_get_context_data(Device); + + if (netuio_contextdata) { + // Free the allocated MDL + if (netuio_contextdata->dpdk_hw.mdl) + IoFreeMdl(netuio_contextdata->dpdk_hw.mdl); + + // Unmap all the BAR regions previously mapped + for (UINT8 bar_index = 0; bar_index < PCI_MAX_BAR; bar_index++) { + if (netuio_contextdata->bar[bar_index].virt_addr) + MmUnmapIoSpace(netuio_contextdata->bar[bar_index].virt_addr, netuio_contextdata->bar[bar_index].size); + } + } +} + + +static NTSTATUS +get_pci_device_info(_In_ WDFOBJECT device) +{ + NTSTATUS status = STATUS_UNSUCCESSFUL; + + PNETUIO_CONTEXT_DATA netuio_contextdata; + netuio_contextdata = netuio_get_context_data(device); + + if (!netuio_contextdata) + return status; + + netuio_contextdata->wdf_device = device; // Store for later use + + // Obtain the BUS_INTERFACE_STANDARD interface from the Bus Driver + status = WdfFdoQueryForInterface(device, &GUID_BUS_INTERFACE_STANDARD, + (PINTERFACE)&netuio_contextdata->bus_interface, + sizeof(BUS_INTERFACE_STANDARD), 1, NULL); + if (!NT_SUCCESS(status)) + return status; + + // Retrieve the B:D:F details of our device + PDEVICE_OBJECT pdo = NULL; + pdo = WdfDeviceWdmGetPhysicalDevice(device); + if (pdo) { + ULONG prop = 0, length = 0; + status = IoGetDeviceProperty(pdo, DevicePropertyBusNumber, sizeof(ULONG), (PVOID)&netuio_contextdata->addr.bus_num, &length); + status = IoGetDeviceProperty(pdo, DevicePropertyAddress, sizeof(ULONG), (PVOID)&prop, &length); + + if (NT_SUCCESS(status)) { + netuio_contextdata->addr.func_num = prop & 0x0000FFFF; + netuio_contextdata->addr.dev_num = ((prop >> 16) & 0x0000FFFF); + } + // Also, retrieve the NUMA node of the device + USHORT numaNode; + status = IoGetDeviceNumaNode(pdo, &numaNode); + if (NT_SUCCESS(status)) { + netuio_contextdata->dev_numa_node = numaNode; + } + } + + return status; +} + +static NTSTATUS +create_device_specific_symbolic_link(_In_ WDFOBJECT device) +{ + NTSTATUS status = STATUS_UNSUCCESSFUL; + UNICODE_STRING netuio_symbolic_link; + + PNETUIO_CONTEXT_DATA netuio_contextdata; + netuio_contextdata = netuio_get_context_data(device); + + if (!netuio_contextdata) + return status; + + // Build symbolic link name as _BDF (bus/device/func) + CHAR symbolic_link[64] = { 0 }; + sprintf_s(symbolic_link, sizeof(symbolic_link), "%s_%04d%02d%02d", + NETUIO_DEVICE_SYMBOLIC_LINK_ANSI, netuio_contextdata->addr.bus_num, + netuio_contextdata->addr.dev_num, netuio_contextdata->addr.func_num); + + ANSI_STRING ansi_symbolic_link; + RtlInitAnsiString(&ansi_symbolic_link, symbolic_link); + + status = RtlAnsiStringToUnicodeString(&netuio_symbolic_link, &ansi_symbolic_link, TRUE); + if (!NT_SUCCESS(status)) + return status; + + status = WdfDeviceCreateSymbolicLink(device, &netuio_symbolic_link); + + RtlFreeUnicodeString(&netuio_symbolic_link); + + return status; +} + +static NTSTATUS +allocate_usermemory_segment(_In_ WDFOBJECT device) +{ + NTSTATUS status = STATUS_SUCCESS; + + PNETUIO_CONTEXT_DATA netuio_contextdata; + netuio_contextdata = netuio_get_context_data(device); + + if (!netuio_contextdata) + return STATUS_UNSUCCESSFUL; + + PHYSICAL_ADDRESS lowest_acceptable_address; + PHYSICAL_ADDRESS highest_acceptable_address; + PHYSICAL_ADDRESS boundary_address_multiple; + + lowest_acceptable_address.QuadPart = 0x0000000000800000; + highest_acceptable_address.QuadPart = 0xFFFFFFFFFFFFFFFF; + boundary_address_multiple.QuadPart = 0; + + // Allocate physically contiguous memory for user process use + netuio_contextdata->dpdk_seg.mem.virt_addr = + MmAllocateContiguousMemorySpecifyCache(USER_MEMORY_SEGMENT_SIZE, + lowest_acceptable_address, + highest_acceptable_address, + boundary_address_multiple, + MmCached); + + if (!netuio_contextdata->dpdk_seg.mem.virt_addr) { + status = STATUS_NO_MEMORY; + goto end; + } + + netuio_contextdata->dpdk_seg.mem.size = USER_MEMORY_SEGMENT_SIZE; + + // Allocate an MDL for this memory region - so that we can map it into the user's process context later + netuio_contextdata->dpdk_seg.mdl = IoAllocateMdl((PVOID)netuio_contextdata->dpdk_seg.mem.virt_addr, USER_MEMORY_SEGMENT_SIZE, FALSE, FALSE, NULL); + if (netuio_contextdata->dpdk_seg.mdl == NULL) { + status = STATUS_NO_MEMORY; + goto end; + } + + // Store the region's physical address + netuio_contextdata->dpdk_seg.mem.phys_addr = MmGetPhysicalAddress(netuio_contextdata->dpdk_seg.mem.virt_addr); + +end: + return status; +} + +static VOID +free_usermemory_segment(_In_ WDFOBJECT device) +{ + PNETUIO_CONTEXT_DATA netuio_contextdata; + netuio_contextdata = netuio_get_context_data(device); + + if (netuio_contextdata) { + if (netuio_contextdata->dpdk_seg.mdl) + IoFreeMdl(netuio_contextdata->dpdk_seg.mdl); + + if (netuio_contextdata->dpdk_seg.mem.virt_addr) + MmFreeContiguousMemory(netuio_contextdata->dpdk_seg.mem.virt_addr); + } +} diff --git a/kernel/windows/netuio/netuio_dev.h b/kernel/windows/netuio/netuio_dev.h new file mode 100644 index 000000000..a19a4fb42 --- /dev/null +++ b/kernel/windows/netuio/netuio_dev.h @@ -0,0 +1,65 @@ +/*- +* +* Copyright(c) 2017 Intel Corporation. All rights reserved. +* +*/ + +#ifndef NETUIO_DEV_H +#define NETUIO_DEV_H + +EXTERN_C_START + +#include "netuio_interface.h" + +// Constants +#define PCI_MAX_BAR 3 +#define USER_MEMORY_SEGMENT_SIZE (256ULL * 1024ULL * 1024ULL) // 256MB + +struct pci_bar { + PHYSICAL_ADDRESS base_addr; + PVOID virt_addr; + UINT64 size; +}; + +struct mem_map_region { + PMDL mdl; // MDL describing the memory region + struct mem_region mem; // Memory region details +}; + +// The device context performs the same job as a WDM device extension in the driver frameworks +typedef struct _NETUIO_CONTEXT_DATA +{ + WDFDEVICE wdf_device; // WDF device handle to the FDO + BUS_INTERFACE_STANDARD bus_interface; // Bus interface for config space access + struct pci_bar bar[PCI_MAX_BAR]; // device BARs + struct dev_addr addr; // B:D:F details of device + USHORT dev_numa_node; // The NUMA node of the device + struct mem_map_region dpdk_hw; // mapped region for the device's register space + struct mem_map_region dpdk_seg; // mapped region allocated for DPDK process use +} NETUIO_CONTEXT_DATA, *PNETUIO_CONTEXT_DATA; + + +// This macro will generate an inline function called DeviceGetContext +// which will be used to get a pointer to the device context memory in a type safe manner. +WDF_DECLARE_CONTEXT_TYPE_WITH_NAME(NETUIO_CONTEXT_DATA, netuio_get_context_data) + + +// Function to initialize the device and its callbacks +NTSTATUS netuio_create_device(_Inout_ PWDFDEVICE_INIT DeviceInit); +NTSTATUS netuio_map_hw_resources(_In_ WDFDEVICE Device, _In_ WDFCMRESLIST Resources, _In_ WDFCMRESLIST ResourcesTranslated); +VOID netuio_free_hw_resources(_In_ WDFDEVICE Device); + + +// Function called for cleanup when device object is being destroyed +VOID netuio_evt_device_context_cleanup(_In_ WDFOBJECT Device); + +// Local function protoyypes +static NTSTATUS get_pci_device_info(_In_ WDFOBJECT device); +static NTSTATUS create_device_specific_symbolic_link(_In_ WDFOBJECT device); +static NTSTATUS allocate_usermemory_segment(_In_ WDFOBJECT device); +static VOID free_usermemory_segment(_In_ WDFOBJECT device); + + +EXTERN_C_END + +#endif // NETUIO_DEV_H diff --git a/kernel/windows/netuio/netuio_drv.c b/kernel/windows/netuio/netuio_drv.c new file mode 100644 index 000000000..d45a9ec4f --- /dev/null +++ b/kernel/windows/netuio/netuio_drv.c @@ -0,0 +1,142 @@ +/*- +* +* Copyright(c) 2017 Intel Corporation. All rights reserved. +* +*/ + + +#include "netuio_drv.h" + +#ifdef ALLOC_PRAGMA +#pragma alloc_text (INIT, DriverEntry) +#pragma alloc_text (PAGE, netuio_evt_device_add) +#pragma alloc_text (PAGE, netuio_evt_driver_context_cleanup) +#endif + + +/* +Routine Description: + DriverEntry initializes the driver and is the first routine called by the + system after the driver is loaded. DriverEntry specifies the other entry + points in the function driver, such as EvtDevice and DriverUnload. + +Return Value: + STATUS_SUCCESS if successful, + STATUS_UNSUCCESSFUL otherwise. + */ +NTSTATUS +DriverEntry(_In_ PDRIVER_OBJECT DriverObject, _In_ PUNICODE_STRING RegistryPath) +{ + WDF_DRIVER_CONFIG config; + NTSTATUS status; + WDF_OBJECT_ATTRIBUTES attributes; + + // Register a cleanup callback so that we can call WPP_CLEANUP when + // the framework driver object is deleted during driver unload. + WDF_OBJECT_ATTRIBUTES_INIT(&attributes); + attributes.EvtCleanupCallback = netuio_evt_driver_context_cleanup; + + WDF_DRIVER_CONFIG_INIT(&config, netuio_evt_device_add); + + status = WdfDriverCreate(DriverObject, RegistryPath, + &attributes, &config, + WDF_NO_HANDLE); + + if (!NT_SUCCESS(status)) { + return status; + } + + return status; +} + + +/* +Routine Description: + netuio_evt_device_add is called by the framework in response to AddDevice + call from the PnP manager. We create and initialize a device object to + represent a new instance of the device. + +Return Value: + NTSTATUS + */ +NTSTATUS +netuio_evt_device_add(_In_ WDFDRIVER Driver, _Inout_ PWDFDEVICE_INIT DeviceInit) +{ + NTSTATUS status; + WDF_PNPPOWER_EVENT_CALLBACKS pnpPowerCallbacks; + + UNREFERENCED_PARAMETER(Driver); + + PAGED_CODE(); + + // Zero out the PnpPowerCallbacks structure + WDF_PNPPOWER_EVENT_CALLBACKS_INIT(&pnpPowerCallbacks); + + // Register Plug-aNd-Play and power management callbacks + pnpPowerCallbacks.EvtDevicePrepareHardware = netuio_evt_prepare_hw; + pnpPowerCallbacks.EvtDeviceReleaseHardware = netuio_evt_release_hw; + + WdfDeviceInitSetPnpPowerEventCallbacks(DeviceInit, &pnpPowerCallbacks); + + status = netuio_create_device(DeviceInit); + + return status; +} + +/* +Routine Description : + Maps HW resources and retrieves the PCI BAR address(es) of the device + +Return Value : + STATUS_SUCCESS is successful. + STATUS_ otherwise +-*/ +NTSTATUS +netuio_evt_prepare_hw(_In_ WDFDEVICE Device, _In_ WDFCMRESLIST Resources, _In_ WDFCMRESLIST ResourcesTranslated) +{ + NTSTATUS status; + + status = netuio_map_hw_resources(Device, Resources, ResourcesTranslated); + + if (NT_SUCCESS(status)) { + PNETUIO_CONTEXT_DATA netuio_contextdata; + netuio_contextdata = netuio_get_context_data(Device); + if (netuio_contextdata) { + DbgPrintEx(DPFLTR_IHVNETWORK_ID, DPFLTR_NETUIO_INFO_LEVEL, "netUIO Driver loaded...on device (B:D:F) %04d:%02d:%02d\n", + netuio_contextdata->addr.bus_num, netuio_contextdata->addr.dev_num, netuio_contextdata->addr.func_num); + } + } + return status; +} + +/* +Routine Description : + Releases the resource mapped by netuio_evt_prepare_hw + +Return Value : + STATUS_SUCCESS always. +-*/ +NTSTATUS +netuio_evt_release_hw(_In_ WDFDEVICE Device, _In_ WDFCMRESLIST ResourcesTranslated) +{ + UNREFERENCED_PARAMETER(ResourcesTranslated); + + netuio_free_hw_resources(Device); + + return STATUS_SUCCESS; +} + +/* +Routine Description: + Free all the resources allocated in DriverEntry. + +Return Value: + None +-*/ +VOID +netuio_evt_driver_context_cleanup(_In_ WDFOBJECT DriverObject) +{ + UNREFERENCED_PARAMETER(DriverObject); + DbgPrintEx(DPFLTR_IHVNETWORK_ID, DPFLTR_NETUIO_INFO_LEVEL, "netUIO Driver unloaded.\n"); + PAGED_CODE(); +} diff --git a/kernel/windows/netuio/netuio_drv.h b/kernel/windows/netuio/netuio_drv.h new file mode 100644 index 000000000..39d7f301e --- /dev/null +++ b/kernel/windows/netuio/netuio_drv.h @@ -0,0 +1,33 @@ +/*- +* +* Copyright(c) 2017 Intel Corporation. All rights reserved. +* +*/ + + +#ifndef NETUIO_DRV_H +#define NETUIO_DRV_H + +#define INITGUID + +#include +#include + +#include "netuio_dev.h" +#include "netuio_queue.h" + +EXTERN_C_START + +// Print output constants +#define DPFLTR_NETUIO_INFO_LEVEL 35 + +// WDFDRIVER Events +DRIVER_INITIALIZE DriverEntry; +EVT_WDF_DRIVER_DEVICE_ADD netuio_evt_device_add; +EVT_WDF_OBJECT_CONTEXT_CLEANUP netuio_evt_driver_context_cleanup; +EVT_WDF_DEVICE_PREPARE_HARDWARE netuio_evt_prepare_hw; +EVT_WDF_DEVICE_RELEASE_HARDWARE netuio_evt_release_hw; + +EXTERN_C_END + +#endif // NETUIO_DRV_H diff --git a/kernel/windows/netuio/netuio_interface.h b/kernel/windows/netuio/netuio_interface.h new file mode 100644 index 000000000..b86bf5d3f --- /dev/null +++ b/kernel/windows/netuio/netuio_interface.h @@ -0,0 +1,69 @@ +/*- +* +* Copyright(c) 2017 Intel Corporation. All rights reserved. +* +*/ + + +#ifndef NETUIO_INTERFACE_H +#define NETUIO_INTERFACE_H + +// All structures in this file are packed on an 8B boundary. +#pragma pack(push) +#pragma pack(8) + +// Define an Interface Guid so that any app can find the device and talk to it. +DEFINE_GUID (GUID_DEVINTERFACE_netUIO, 0x08336f60,0x0679,0x4c6c,0x85,0xd2,0xae,0x7c,0xed,0x65,0xff,0xf7); // {08336f60-0679-4c6c-85d2-ae7ced65fff7} + +// Device name definitions +#define NETUIO_DEVICE_SYMBOLIC_LINK_ANSI "\\DosDevices\\netuio" + +// netUIO driver symbolic name (prefix) +#define NETUIO_DRIVER_NAME _T("netuio") + +// IOCTL code definitions +#define IOCTL_NETUIO_GET_HW_DATA CTL_CODE(FILE_DEVICE_NETWORK, 51, METHOD_BUFFERED, FILE_READ_ACCESS | FILE_WRITE_ACCESS) +#define IOCTL_NETUIO_PCI_CONFIG_IO CTL_CODE(FILE_DEVICE_NETWORK, 52, METHOD_BUFFERED, FILE_READ_ACCESS | FILE_WRITE_ACCESS) + +struct mem_region { + UINT64 size; // memory region size + PHYSICAL_ADDRESS phys_addr; // physical address of the memory region + PVOID virt_addr; // virtual address of the memory region + PVOID user_mapped_virt_addr; // virtual address of the region mapped into user process context +}; + +struct dev_addr { + ULONG bus_num; + USHORT dev_num; + USHORT func_num; +}; + +enum pci_io { + PCI_IO_READ = 0, + PCI_IO_WRITE = 1 +}; + +struct dpdk_private_info +{ + struct mem_region hw; + struct mem_region ms; + struct dev_addr dev_addr; + struct mem_region bar1; +// struct mem_region bar2; + UINT16 dev_id; + UINT16 sub_dev_id; + USHORT dev_numa_node; + USHORT reserved; +}; + +struct dpdk_pci_config_io +{ + struct dev_addr dev_addr; + PVOID buf; + UINT32 offset; + enum pci_io op; +}; + +#pragma pack(pop) + +#endif // NETUIO_INTERFACE_H diff --git a/kernel/windows/netuio/netuio_queue.c b/kernel/windows/netuio/netuio_queue.c new file mode 100644 index 000000000..312d4b682 --- /dev/null +++ b/kernel/windows/netuio/netuio_queue.c @@ -0,0 +1,271 @@ +/*- +* +* Copyright(c) 2017 Intel Corporation. All rights reserved. +* +*/ + + +#include "netuio_drv.h" + +#ifdef ALLOC_PRAGMA +#pragma alloc_text (PAGE, netuio_queue_initialize) +#endif + +VOID netuio_read_PCI_config(PNETUIO_CONTEXT_DATA netuio_contextdata, ULONG offset, PVOID buffer) +{ + netuio_contextdata->bus_interface.GetBusData(netuio_contextdata->bus_interface.Context, + PCI_WHICHSPACE_CONFIG, + buffer, + offset, + sizeof(UINT32)); +} + +VOID netuio_write_PCI_config(PNETUIO_CONTEXT_DATA netuio_contextdata, ULONG offset, PVOID buffer) +{ + netuio_contextdata->bus_interface.SetBusData(netuio_contextdata->bus_interface.Context, + PCI_WHICHSPACE_CONFIG, + buffer, + offset, + sizeof(UINT32)); +} + +static NTSTATUS +netuio_handle_get_hw_data_request(_In_ WDFREQUEST Request, _In_ PNETUIO_CONTEXT_DATA netuio_contextdata, + _In_ PVOID outputBuf, _In_ size_t outputBufSize) +{ + NTSTATUS status = STATUS_SUCCESS; + + WDF_REQUEST_PARAMETERS params; + WDF_REQUEST_PARAMETERS_INIT(¶ms); + WdfRequestGetParameters(Request, ¶ms); + + if (!netuio_contextdata || (outputBufSize != sizeof(struct dpdk_private_info))) { + status = STATUS_INVALID_PARAMETER; + goto end; + } + + struct dpdk_private_info *dpdk_pvt_info = (struct dpdk_private_info *)outputBuf; + RtlZeroMemory(dpdk_pvt_info, outputBufSize); + + dpdk_pvt_info->hw.phys_addr.QuadPart = netuio_contextdata->bar[0].base_addr.QuadPart; + dpdk_pvt_info->hw.user_mapped_virt_addr = netuio_contextdata->dpdk_hw.mem.user_mapped_virt_addr; + dpdk_pvt_info->hw.size = netuio_contextdata->bar[0].size; + + dpdk_pvt_info->ms.phys_addr.QuadPart = netuio_contextdata->dpdk_seg.mem.phys_addr.QuadPart; + dpdk_pvt_info->ms.user_mapped_virt_addr = netuio_contextdata->dpdk_seg.mem.user_mapped_virt_addr; + dpdk_pvt_info->ms.size = netuio_contextdata->dpdk_seg.mem.size; +end: + return status; +} + +/* +Routine Description: + The I/O dispatch callbacks for the frameworks device object are configured here. + A single default I/O Queue is configured for parallel request processing, and a + driver context memory allocation is created to hold our structure QUEUE_CONTEXT. + +Return Value: + None + */ +NTSTATUS +netuio_queue_initialize(_In_ WDFDEVICE Device) +{ + WDFQUEUE queue; + NTSTATUS status; + WDF_IO_QUEUE_CONFIG queueConfig; + + PAGED_CODE(); + + // Configure a default queue so that requests that are not + // configure-fowarded using WdfDeviceConfigureRequestDispatching to goto + // other queues get dispatched here. + WDF_IO_QUEUE_CONFIG_INIT_DEFAULT_QUEUE(&queueConfig, WdfIoQueueDispatchParallel); + + queueConfig.EvtIoDeviceControl = netuio_evt_IO_device_control; + queueConfig.EvtIoStop = netuio_evt_IO_stop; + + status = WdfIoQueueCreate(Device, + &queueConfig, + WDF_NO_OBJECT_ATTRIBUTES, + &queue); + + if( !NT_SUCCESS(status) ) { + return status; + } + + return status; +} + +/* +Routine Description: + This event is invoked when the framework receives IRP_MJ_DEVICE_CONTROL request. + +Return Value: + None + */ +VOID +netuio_evt_IO_device_control(_In_ WDFQUEUE Queue, _In_ WDFREQUEST Request, + _In_ size_t OutputBufferLength, _In_ size_t InputBufferLength, + _In_ ULONG IoControlCode) +{ + UNREFERENCED_PARAMETER(OutputBufferLength); + UNREFERENCED_PARAMETER(InputBufferLength); + + NTSTATUS status = STATUS_SUCCESS; + PVOID input_buf = NULL, output_buf = NULL; + size_t input_buf_size, output_buf_size; + size_t bytes_returned = 0; + + WDFDEVICE device = WdfIoQueueGetDevice(Queue); + + PNETUIO_CONTEXT_DATA netuio_contextdata; + netuio_contextdata = netuio_get_context_data(device); + + switch (IoControlCode) { + case IOCTL_NETUIO_GET_HW_DATA: + // First retrieve the input buffer and see if it matches our device + status = WdfRequestRetrieveInputBuffer(Request, sizeof(struct dpdk_private_info), &input_buf, &input_buf_size); + if (!NT_SUCCESS(status)) { + status = STATUS_INVALID_BUFFER_SIZE; + break; + } + + struct dpdk_private_info *dpdk_pvt_info = (struct dpdk_private_info *)input_buf; + // Ensure that the B:D:F match - otherwise, fail the IOCTL + if ((netuio_contextdata->addr.bus_num != dpdk_pvt_info->dev_addr.bus_num) || + (netuio_contextdata->addr.dev_num != dpdk_pvt_info->dev_addr.dev_num) || + (netuio_contextdata->addr.func_num != dpdk_pvt_info->dev_addr.func_num)) { + status = STATUS_NOT_SAME_DEVICE; + break; + } + + // Map the previously allocated/defined memory regions to the user's process context + MmBuildMdlForNonPagedPool(netuio_contextdata->dpdk_hw.mdl); + netuio_contextdata->dpdk_hw.mem.user_mapped_virt_addr = + MmMapLockedPagesSpecifyCache(netuio_contextdata->dpdk_hw.mdl, UserMode, MmCached, + NULL, FALSE, (NormalPagePriority | MdlMappingNoExecute)); + + MmBuildMdlForNonPagedPool(netuio_contextdata->dpdk_seg.mdl); + netuio_contextdata->dpdk_seg.mem.user_mapped_virt_addr = + MmMapLockedPagesSpecifyCache(netuio_contextdata->dpdk_seg.mdl, UserMode, MmCached, + NULL, FALSE, (NormalPagePriority | MdlMappingNoExecute)); + + if (!netuio_contextdata->dpdk_hw.mem.user_mapped_virt_addr && !netuio_contextdata->dpdk_seg.mem.user_mapped_virt_addr) { + status = STATUS_INSUFFICIENT_RESOURCES; + break; + } + + // Zero out the physically contiguous block + RtlZeroMemory(netuio_contextdata->dpdk_seg.mem.virt_addr, netuio_contextdata->dpdk_seg.mem.size); + + // Return relevant data to the caller + status = WdfRequestRetrieveOutputBuffer(Request, sizeof(struct dpdk_private_info), &output_buf, &output_buf_size); + if (!NT_SUCCESS(status)) { + status = STATUS_INVALID_BUFFER_SIZE; + break; + } + ASSERT(output_buf_size == OutputBufferLength); + status = netuio_handle_get_hw_data_request(Request, netuio_contextdata, output_buf, output_buf_size); + if (NT_SUCCESS(status)) + bytes_returned = output_buf_size; + + break; + + case IOCTL_NETUIO_PCI_CONFIG_IO: + // First retrieve the input buffer and see if it matches our device + status = WdfRequestRetrieveInputBuffer(Request, sizeof(struct dpdk_pci_config_io), &input_buf, &input_buf_size); + if (!NT_SUCCESS(status)) { + status = STATUS_INVALID_BUFFER_SIZE; + break; + } + + struct dpdk_pci_config_io *dpdk_pci_io_input = (struct dpdk_pci_config_io *)input_buf; + // Ensure that the B:D:F match - otherwise, fail the IOCTL + if ((netuio_contextdata->addr.bus_num != dpdk_pci_io_input->dev_addr.bus_num) || + (netuio_contextdata->addr.dev_num != dpdk_pci_io_input->dev_addr.dev_num) || + (netuio_contextdata->addr.func_num != dpdk_pci_io_input->dev_addr.func_num)) { + status = STATUS_NOT_SAME_DEVICE; + break; + } + // Retrieve output buffer + status = WdfRequestRetrieveOutputBuffer(Request, sizeof(UINT32), &output_buf, &output_buf_size); + if (!NT_SUCCESS(status)) { + status = STATUS_INVALID_BUFFER_SIZE; + break; + } + ASSERT(output_buf_size == OutputBufferLength); + + if (dpdk_pci_io_input->op == PCI_IO_READ) { + netuio_read_PCI_config(netuio_contextdata, dpdk_pci_io_input->offset, output_buf); + bytes_returned = output_buf_size; + } + else { + netuio_write_PCI_config(netuio_contextdata, dpdk_pci_io_input->offset, dpdk_pci_io_input->buf); + bytes_returned = 0; + } + + break; + + default: + break; + } + + WdfRequestCompleteWithInformation(Request, status, bytes_returned); + + return; +} + +/* +Routine Description: + This event is invoked for a power-managed queue before the device leaves the working state (D0). + +Return Value: + None + */ +VOID +netuio_evt_IO_stop(_In_ WDFQUEUE Queue, _In_ WDFREQUEST Request,_In_ ULONG ActionFlags) +{ + // + // In most cases, the EvtIoStop callback function completes, cancels, or postpones + // further processing of the I/O request. + // + // Typically, the driver uses the following rules: + // + // - If the driver owns the I/O request, it calls WdfRequestUnmarkCancelable + // (if the request is cancelable) and either calls WdfRequestStopAcknowledge + // with a Requeue value of TRUE, or it calls WdfRequestComplete with a + // completion status value of STATUS_SUCCESS or STATUS_CANCELLED. + // + // Before it can call these methods safely, the driver must make sure that + // its implementation of EvtIoStop has exclusive access to the request. + // + // In order to do that, the driver must synchronize access to the request + // to prevent other threads from manipulating the request concurrently. + // The synchronization method you choose will depend on your driver's design. + // + // For example, if the request is held in a shared context, the EvtIoStop callback + // might acquire an internal driver lock, take the request from the shared context, + // and then release the lock. At this point, the EvtIoStop callback owns the request + // and can safely complete or requeue the request. + // + // - If the driver has forwarded the I/O request to an I/O target, it either calls + // WdfRequestCancelSentRequest to attempt to cancel the request, or it postpones + // further processing of the request and calls WdfRequestStopAcknowledge with + // a Requeue value of FALSE. + // + // A driver might choose to take no action in EvtIoStop for requests that are + // guaranteed to complete in a small amount of time. + // + // In this case, the framework waits until the specified request is complete + // before moving the device (or system) to a lower power state or removing the device. + // Potentially, this inaction can prevent a system from entering its hibernation state + // or another low system power state. In extreme cases, it can cause the system + // to crash with bugcheck code 9F. + // + UNREFERENCED_PARAMETER(Queue); + UNREFERENCED_PARAMETER(Request); + UNREFERENCED_PARAMETER(ActionFlags); + + return; +} + diff --git a/kernel/windows/netuio/netuio_queue.h b/kernel/windows/netuio/netuio_queue.h new file mode 100644 index 000000000..6a0306516 --- /dev/null +++ b/kernel/windows/netuio/netuio_queue.h @@ -0,0 +1,30 @@ +/*- +* +* Copyright(c) 2017 Intel Corporation. All rights reserved. +* +*/ + + +#ifndef NETUIO_QUEUE_H +#define NETUIO_QUEUE_H + +EXTERN_C_START + +// This is the context that can be placed per queue and would contain per queue information. +typedef struct _QUEUE_CONTEXT { + ULONG PrivateDeviceData; // just a placeholder +} QUEUE_CONTEXT, *PQUEUE_CONTEXT; + +WDF_DECLARE_CONTEXT_TYPE_WITH_NAME(QUEUE_CONTEXT, QueueGetContext) + + +NTSTATUS +netuio_queue_initialize(_In_ WDFDEVICE hDevice); + +// Events from the IoQueue object +EVT_WDF_IO_QUEUE_IO_DEVICE_CONTROL netuio_evt_IO_device_control; +EVT_WDF_IO_QUEUE_IO_STOP netuio_evt_IO_stop; + +EXTERN_C_END + +#endif // NETUIO_QUEUE_H diff --git a/kernel/windows/netuio/resource.h b/kernel/windows/netuio/resource.h new file mode 100644 index 000000000..9789ffdf3 --- /dev/null +++ b/kernel/windows/netuio/resource.h @@ -0,0 +1,14 @@ +//{{NO_DEPENDENCIES}} +// Microsoft Visual C++ generated include file. +// Used by netuio.rc + +// Next default values for new objects +// +#ifdef APSTUDIO_INVOKED +#ifndef APSTUDIO_READONLY_SYMBOLS +#define _APS_NEXT_RESOURCE_VALUE 101 +#define _APS_NEXT_COMMAND_VALUE 40001 +#define _APS_NEXT_CONTROL_VALUE 1001 +#define _APS_NEXT_SYMED_VALUE 101 +#endif +#endif -- 2.23.0.vfs.1.1.63.g5a5ad7f