From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 52767A04C8; Sat, 19 Sep 2020 04:52:47 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id AAF6E1D94C; Sat, 19 Sep 2020 04:52:46 +0200 (CEST) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by dpdk.org (Postfix) with ESMTP id A90091D8D7 for ; Sat, 19 Sep 2020 04:52:44 +0200 (CEST) Received: by linux.microsoft.com (Postfix, from userid 1059) id CC47720B7178; Fri, 18 Sep 2020 19:52:43 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com CC47720B7178 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1600483963; bh=MSC4gsWB8heCRGcwM7m8v8eHz44/IMGL7mx/yNe+1d4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OasjZmOM/B3s0klDgw00CWtPIHJ0Nt0+BEzXYN9rW1nSWgBZmJilXDxPUvmasEf// BQmezFPaN4L2QV/u+HWqocYhPfNodznoOjV3Nn6tWvXyZI/wQDlxpnDuFurCErSX2p lULsMV7M4vCftW1lwz5nVawZh+VePtaWsdDEuk/o= From: Narcisa Ana Maria Vasile To: dev@dpdk.org, thomas@monjalon.net, ocardona@microsoft.com, haramakr@linux.microsoft.com, pallavi.kadam@intel.com, dmitry.kozliuk@gmail.com Cc: ranjit.menon@intel.com, dmitrym@microsoft.com, Narcisa Vasile , Harini Ramakrishnan Date: Fri, 18 Sep 2020 19:52:33 -0700 Message-Id: <1600483953-28579-1-git-send-email-navasile@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1599676883-11991-1-git-send-email-navasile@linux.microsoft.com> References: <1599676883-11991-1-git-send-email-navasile@linux.microsoft.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v4] windows/netuio: add Windows NetUIO kernel driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Narcisa Vasile The Windows netuio kernel driver provides the DPDK userspace application with direct access to hardware, by mapping the HW registers in userspace and allowing read/write operations from/to the device configuration space. Two IOCTLs are defined by the netuio interface: * IOCTL_NETUIO_MAP_HW_INTO_USERMODE - used for mapping the device registers into userspace * IOCTL_NETUIO_PCI_CONFIG_IO - used to read/write from/into the device configuration space Note: Requests to map the device BARs into userspace need to be processed in the thread context of the process that initiated the mapping request. Otherwise, the BARs might end up mapped into an arbitrary process address space. EvtIoInCallerContext is used to ensure the requests are handled in the right user thread context. Other requests (PCI config) are sent back to the framework and processed by the EvtIoDeviceControl callback. Cc: Harini Ramakrishnan Cc: Omar Cardona Cc: Dmitry Malloy Signed-off-by: Narcisa Vasile --- v4: Fixed issues reported by DmitryK: * Fix whitespaces issues * Fix status not checked * Remove NULL checks for device and file context getters * Use %x for BDF * Add comment for BAR traversing * Remove complex calculations that were only needed in ASSERT * Remove netuio_evt_driver_context_cleanup as not needed * Check GetBusData/SetBusData returned bytes * General cleanup windows/.gitattributes | 4 + windows/.gitignore | 2 + windows/netuio/README.rst | 58 +++++ windows/netuio/netuio.inf | 77 ++++++ windows/netuio/netuio.sln | 24 ++ windows/netuio/netuio.vcxproj | 113 +++++++++ windows/netuio/netuio.vcxproj.filters | 54 +++++ windows/netuio/netuio_dev.c | 273 +++++++++++++++++++++ windows/netuio/netuio_dev.h | 66 +++++ windows/netuio/netuio_drv.c | 131 ++++++++++ windows/netuio/netuio_drv.h | 30 +++ windows/netuio/netuio_interface.h | 88 +++++++ windows/netuio/netuio_queue.c | 334 ++++++++++++++++++++++++++ windows/netuio/netuio_queue.h | 21 ++ 14 files changed, 1275 insertions(+) create mode 100644 windows/.gitattributes create mode 100644 windows/.gitignore create mode 100644 windows/netuio/README.rst create mode 100644 windows/netuio/netuio.inf create mode 100644 windows/netuio/netuio.sln create mode 100644 windows/netuio/netuio.vcxproj create mode 100644 windows/netuio/netuio.vcxproj.filters create mode 100644 windows/netuio/netuio_dev.c create mode 100644 windows/netuio/netuio_dev.h create mode 100644 windows/netuio/netuio_drv.c create mode 100644 windows/netuio/netuio_drv.h create mode 100644 windows/netuio/netuio_interface.h create mode 100644 windows/netuio/netuio_queue.c create mode 100644 windows/netuio/netuio_queue.h diff --git a/windows/.gitattributes b/windows/.gitattributes new file mode 100644 index 000000000..13482db3d --- /dev/null +++ b/windows/.gitattributes @@ -0,0 +1,4 @@ +* text=auto + +*.sln text eol=crlf +*.vcxproj text eol=crlf diff --git a/windows/.gitignore b/windows/.gitignore new file mode 100644 index 000000000..543281e8f --- /dev/null +++ b/windows/.gitignore @@ -0,0 +1,2 @@ +x64/ +.vs/ diff --git a/windows/netuio/README.rst b/windows/netuio/README.rst new file mode 100644 index 000000000..d9c5e3c8d --- /dev/null +++ b/windows/netuio/README.rst @@ -0,0 +1,58 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2020 Microsoft Corporation. + +Compiling the NetUIO Driver from Source +======================================= + +Operating System +~~~~~~~~~~~~~~~~ + +The NetUIO source has been validated against the following operating systems: + +* Windows Server 2016 +* Windows Server 2019 + +Hardware Requirements +~~~~~~~~~~~~~~~~~~~~~ +The NetUIO driver has been validated using the following network adapters on the Windows platform: + +* +* + +Software Requirements +~~~~~~~~~~~~~~~~~~~~~ +For a list of required software tools please see the Prerequisites section in windows/README.rst. + +Building the NetUIO Driver +-------------------------- + +Follow the steps below to build the NetUIO driver and install the driver for the network adapter. + +* Clone the dpdk-kmods repository: git clone git://dpdk.org/dpdk-kmods +* Navigate to \dpdk-kmods\windows\netuio +* Load netuio.sln in Microsoft Visual Studio 2017 or 2019 +* Choose Release as the configuration mode and Build the solution +* The resultant output files can be found in x64\Release\netuio + +Installing netuio.sys Driver for development +-------------------------------------------- +.. note:: + + In a development environment, NetUIO driver can be loaded by enabling test-signing. + Please implement adequate precautionary measures before installing a test-signed driver, replacing a signed-driver. + +To ensure test-signed kernel-mode drivers can load on Windows, enable test-signing, using the following BCDEdit command. + +C:\windows\system32>Bcdedit.exe -set TESTSIGNING ON + +Windows displays the text "Test Mode" to remind users the system has test-signing enabled. +Refer to the MSDN documentation on how to Test-Sign a Driver Package. + +To procure a WHQL signed NetUIO driver for Windows, please reach out to dpdkwin@microsoft.com + +* Go to Device Manager -> Network Adapters. +* Right Click on target network adapter -> Select Update Driver. +* Select "Browse my computer for driver software". +* In the resultant window, select "Let me pick from a list of available drivers on my computer". +* Select "DPDK netUIO for Network Adapter" from the list of drivers. +* The NetUIO.sys driver is successfully installed. diff --git a/windows/netuio/netuio.inf b/windows/netuio/netuio.inf new file mode 100644 index 000000000..7121a0fa0 --- /dev/null +++ b/windows/netuio/netuio.inf @@ -0,0 +1,77 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright (c) Microsoft Corporation. All rights reserved +; +; netuio.inf +; + +[Version] +Signature="$WINDOWS NT$" +Class=%ClassName% +ClassGuid={78912BC1-CB8E-4B28-A329-F322EBADBE0F} +Provider=%Provider% +CatalogFile=netuio.cat +DriverVer= + +[ClassInstall32] +Addreg=netuioClassReg + +[netuioClassReg] +HKR,,,0,%ClassName% +HKR,,Icon,,-5 + +;***************************************** +; Install Section +;***************************************** + +[Manufacturer] +%ManufacturerName%=Standard, NT$ARCH$ + +[Standard.NT$ARCH$] +%netuio.DeviceDesc%=netuio_Device, Root\netuio + +[netuio_Device.NT] +CopyFiles=Drivers_Dir + +[Drivers_Dir] +netuio.sys + +[netuio_Device.NT.HW] +AddReg=Device.HW.Registry + +[Device.HW.Registry] +; Ensure that only administrators can access our device object. +HKR,,Security,,"D:P(A;;GA;;;SY)(A;;GA;;;BA)" + +;-------------- Service installation +[netuio_Device.NT.Services] +AddService = netuio,%SPSVCINST_ASSOCSERVICE%, netuio_Service_Inst + +; -------------- netuio driver install sections +[netuio_Service_Inst] +DisplayName = %netuio.SVCDESC% +ServiceType = 1 ; SERVICE_KERNEL_DRIVER +StartType = 3 ; SERVICE_DEMAND_START +ErrorControl = 1 ; SERVICE_ERROR_NORMAL +ServiceBinary = %12%\netuio.sys +AddReg = DMAr.reg + +[DestinationDirs] +DefaultDestDir = 12 + +[SourceDisksNames] +1 = %DiskName%,,,"" + +[SourceDisksFiles] +netuio.sys = 1,, + +[Strings] +SPSVCINST_ASSOCSERVICE= 0x00000002 +ManufacturerName = "Microsoft" +Provider = "Vendor" +ClassName = "Windows UIO" +DiskName = "DPDK netUIO Installation Disk" +netuio.DeviceDesc = "netuio Device" +netuio.SVCDESC = "netuio Service" + +[DMAr.reg] +HKR,Parameters,DmaRemappingCompatible,0x00010001,1 diff --git a/windows/netuio/netuio.sln b/windows/netuio/netuio.sln new file mode 100644 index 000000000..15c26e6b9 --- /dev/null +++ b/windows/netuio/netuio.sln @@ -0,0 +1,24 @@ + +Microsoft Visual Studio Solution File, Format Version 12.00 +# Visual Studio 14 +VisualStudioVersion = 14.0.25420.1 +MinimumVisualStudioVersion = 10.0.40219.1 +Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "netuio", "netuio.vcxproj", "{66EC91EF-AC5B-4D1E-8314-9B3E2855CCF6}" +EndProject +Global + GlobalSection(SolutionConfigurationPlatforms) = preSolution + Debug|x64 = Debug|x64 + Release|x64 = Release|x64 + EndGlobalSection + GlobalSection(ProjectConfigurationPlatforms) = postSolution + {66EC91EF-AC5B-4D1E-8314-9B3E2855CCF6}.Debug|x64.ActiveCfg = Debug|x64 + {66EC91EF-AC5B-4D1E-8314-9B3E2855CCF6}.Debug|x64.Build.0 = Debug|x64 + {66EC91EF-AC5B-4D1E-8314-9B3E2855CCF6}.Debug|x64.Deploy.0 = Debug|x64 + {66EC91EF-AC5B-4D1E-8314-9B3E2855CCF6}.Release|x64.ActiveCfg = Release|x64 + {66EC91EF-AC5B-4D1E-8314-9B3E2855CCF6}.Release|x64.Build.0 = Release|x64 + {66EC91EF-AC5B-4D1E-8314-9B3E2855CCF6}.Release|x64.Deploy.0 = Release|x64 + EndGlobalSection + GlobalSection(SolutionProperties) = preSolution + HideSolutionNode = FALSE + EndGlobalSection +EndGlobal diff --git a/windows/netuio/netuio.vcxproj b/windows/netuio/netuio.vcxproj new file mode 100644 index 000000000..87c8b940c --- /dev/null +++ b/windows/netuio/netuio.vcxproj @@ -0,0 +1,113 @@ + + + + + Debug + x64 + + + Release + x64 + + + + {66EC91EF-AC5B-4D1E-8314-9B3E2855CCF6} + {497e31cb-056b-4f31-abb8-447fd55ee5a5} + v4.5 + 12.0 + Debug + Win32 + netuio + $(LatestTargetPlatformVersion) + + + + + + true + WindowsKernelModeDriver10.0 + Driver + KMDF + Desktop + + + + + false + WindowsKernelModeDriver10.0 + Driver + KMDF + Desktop + + + + + + + + + + + DbgengKernelDebugger + $(SolutionDir)$(Platform)\$(ConfigurationName)\$(MSBuildProjectName)\ + $(SolutionDir)$(Platform)\$(ConfigurationName)\$(MSBuildProjectName)\ + true + + + DbgengKernelDebugger + $(SolutionDir)$(Platform)\$(ConfigurationName)\$(MSBuildProjectName)\ + $(SolutionDir)$(Platform)\$(ConfigurationName)\$(MSBuildProjectName)\ + true + + + + false + true + trace.h + true + false + + + 0.6.1 + + + %(AdditionalDependencies);$(KernelBufferOverflowLib);$(DDK_LIB_PATH)ntoskrnl.lib;$(DDK_LIB_PATH)hal.lib;$(DDK_LIB_PATH)wmilib.lib;$(KMDF_LIB_PATH)$(KMDF_VER_PATH)\WdfLdr.lib;$(KMDF_LIB_PATH)$(KMDF_VER_PATH)\WdfDriverEntry.lib;$(DDK_LIB_PATH)\wdmsec.lib + + + + + false + true + trace.h + true + false + + + 0.6.1 + + + %(AdditionalDependencies);$(KernelBufferOverflowLib);$(DDK_LIB_PATH)ntoskrnl.lib;$(DDK_LIB_PATH)hal.lib;$(DDK_LIB_PATH)wmilib.lib;$(KMDF_LIB_PATH)$(KMDF_VER_PATH)\WdfLdr.lib;$(KMDF_LIB_PATH)$(KMDF_VER_PATH)\WdfDriverEntry.lib;$(DDK_LIB_PATH)\wdmsec.lib + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/windows/netuio/netuio.vcxproj.filters b/windows/netuio/netuio.vcxproj.filters new file mode 100644 index 000000000..49c1da15b --- /dev/null +++ b/windows/netuio/netuio.vcxproj.filters @@ -0,0 +1,54 @@ + + + + + {4FC737F1-C7A5-4376-A066-2A32D752A2FF} + cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx + + + {93995380-89BD-4b04-88EB-625FBE52EBFB} + h;hpp;hxx;hm;inl;inc;xsd + + + {67DA6AB6-F800-4c08-8B7A-83BB121AAD01} + rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav;mfcribbon-ms + + + {8E41214B-6785-4CFE-B992-037D68949A14} + inf;inv;inx;mof;mc; + + + + + Driver Files + + + + + Header Files + + + Header Files + + + Header Files + + + Header Files + + + Header Files + + + + + Source Files + + + Source Files + + + Source Files + + + \ No newline at end of file diff --git a/windows/netuio/netuio_dev.c b/windows/netuio/netuio_dev.c new file mode 100644 index 000000000..b2deb10c5 --- /dev/null +++ b/windows/netuio/netuio_dev.c @@ -0,0 +1,273 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Microsoft Corporation. + */ + +#include +#include "netuio_drv.h" + +#include +#include + +#ifdef ALLOC_PRAGMA +#pragma alloc_text (PAGE, netuio_create_device) +#pragma alloc_text (PAGE, netuio_map_hw_resources) +#pragma alloc_text (PAGE, netuio_free_hw_resources) +#endif + +static NTSTATUS +get_pci_device_info(_In_ WDFOBJECT device) +{ + NTSTATUS status = STATUS_UNSUCCESSFUL; + + PNETUIO_CONTEXT_DATA ctx; + ctx = netuio_get_context_data(device); + + ctx->wdf_device = device; // Store for later use + + // Obtain the BUS_INTERFACE_STANDARD interface from the Bus Driver + status = WdfFdoQueryForInterface(device, + &GUID_BUS_INTERFACE_STANDARD, + (PINTERFACE)&ctx->bus_interface, + sizeof(BUS_INTERFACE_STANDARD), + 1, + NULL); + if (!NT_SUCCESS(status)) + return status; + + // Retrieve the B:D:F details of our device + PDEVICE_OBJECT pdo = NULL; + pdo = WdfDeviceWdmGetPhysicalDevice(device); + if (pdo) { + ULONG prop = 0, length = 0; + status = IoGetDeviceProperty(pdo, DevicePropertyBusNumber, sizeof(ULONG), (PVOID)&ctx->addr.bus_num, &length); + if (!NT_SUCCESS(status)) { + return status; + } + + status = IoGetDeviceProperty(pdo, DevicePropertyAddress, sizeof(ULONG), (PVOID)&prop, &length); + if (!NT_SUCCESS(status)) { + return status; + } + + ctx->addr.func_num = prop & 0x0000FFFF; + ctx->addr.dev_num = ((prop >> 16) & 0x0000FFFF); + } + + return status; +} + +static NTSTATUS +create_device_specific_symbolic_link(_In_ WDFOBJECT device) +{ + DECLARE_UNICODE_STRING_SIZE(symbolic_link, NETUIO_MAX_SYMLINK_LEN); + + PNETUIO_CONTEXT_DATA ctx; + ctx = netuio_get_context_data(device); + + // Build symbolic link name as _BDF (bus/device/func) + RtlUnicodeStringPrintf(&symbolic_link, + L"%s_%04x%02x%02x", + NETUIO_DEVICE_SYMBOLIC_LINK_UNICODE, + ctx->addr.bus_num, + ctx->addr.dev_num, + ctx->addr.func_num); + + return WdfDeviceCreateSymbolicLink(device, &symbolic_link); +} + +/* +Routine Description: + Worker routine called to create a device and its software resources. + +Return Value: + NTSTATUS + */ +_Use_decl_annotations_ +NTSTATUS +netuio_create_device(PWDFDEVICE_INIT DeviceInit) +{ + NTSTATUS status; + WDFDEVICE device = NULL; + + WDF_OBJECT_ATTRIBUTES deviceAttributes; + WDF_PNPPOWER_EVENT_CALLBACKS pnpPowerCallbacks; + WDF_FILEOBJECT_CONFIG fileConfig; + WDF_OBJECT_ATTRIBUTES fileAttributes; + + PAGED_CODE(); + + // Register PnP power management callbacks + WDF_PNPPOWER_EVENT_CALLBACKS_INIT(&pnpPowerCallbacks); + pnpPowerCallbacks.EvtDevicePrepareHardware = netuio_evt_prepare_hw; + pnpPowerCallbacks.EvtDeviceReleaseHardware = netuio_evt_release_hw; + WdfDeviceInitSetPnpPowerEventCallbacks(DeviceInit, &pnpPowerCallbacks); + + // Register callbacks for when a HANDLE is opened or closed. + WDF_OBJECT_ATTRIBUTES_INIT_CONTEXT_TYPE(&fileAttributes, NETUIO_FILE_CONTEXT_DATA); + WDF_FILEOBJECT_CONFIG_INIT(&fileConfig, WDF_NO_EVENT_CALLBACK, WDF_NO_EVENT_CALLBACK, netuio_evt_file_cleanup); + WdfDeviceInitSetFileObjectConfig(DeviceInit, &fileConfig, &fileAttributes); + + WDF_OBJECT_ATTRIBUTES_INIT_CONTEXT_TYPE(&deviceAttributes, NETUIO_CONTEXT_DATA); + WdfDeviceInitSetIoInCallerContextCallback(DeviceInit, netuio_evt_IO_in_caller_context); + + status = WdfDeviceCreate(&DeviceInit, &deviceAttributes, &device); + + if (NT_SUCCESS(status)) { + // Create a device interface so that applications can find and talk to us. + status = WdfDeviceCreateDeviceInterface(device, &GUID_DEVINTERFACE_netUIO, NULL); + } + + if (NT_SUCCESS(status)) { + // Retrieve and store PCI information + status = get_pci_device_info(device); + } + + if (NT_SUCCESS(status)) { + // Create a symbolic link name for user-space access + status = create_device_specific_symbolic_link(device); + } + + if (NT_SUCCESS(status)) { + // Initialize the I/O Package and any Queues + status = netuio_queue_initialize(device); + } + + return status; +} + +_Use_decl_annotations_ +NTSTATUS +netuio_map_hw_resources(WDFDEVICE Device, WDFCMRESLIST Resources, WDFCMRESLIST ResourcesTranslated) +{ + UNREFERENCED_PARAMETER(Resources); + + NTSTATUS status; + + PNETUIO_CONTEXT_DATA ctx; + ctx = netuio_get_context_data(Device); + + PCI_COMMON_HEADER pci_config = {0}; + ULONG bytes_returned; + + // Read PCI configuration space + bytes_returned = ctx->bus_interface.GetBusData( + ctx->bus_interface.Context, + PCI_WHICHSPACE_CONFIG, + &pci_config, + 0, + sizeof(pci_config)); + + if (bytes_returned != sizeof(pci_config)) { + status = STATUS_NOT_SUPPORTED; + goto end; + } + + // Device type is implictly enforced by .inf + ASSERT(PCI_CONFIGURATION_TYPE(&pci_config) == PCI_DEVICE_TYPE); + + RtlZeroMemory(ctx->bar, sizeof(ctx->bar)); + RtlZeroMemory(ctx->dpdk_hw, sizeof(ctx->dpdk_hw)); + + PCM_PARTIAL_RESOURCE_DESCRIPTOR descriptor; + ULONG next_descriptor = 0; + ULONG curr_bar = 0; + ULONG prev_bar = 0; + + /* + * ResourcesTranslated report MMIO BARs in the correct order, but their + * indices differ from physical ones. + + * For example: + * The values returned by WdfCmResourceListGetDescriptor can be condensed like this: + * Bar[0] = c840000c + * Bar[1] = c800000c + * Bar[2] = c844000c + + * But the BAR array has to be filled like this : + * Bar[0] = c840000c + * Bar[1] = null + * Bar[2] = c800000c + * Bar[3] = null + * Bar[4] = c844000c + * Bar[5] = null + + * Traverse PCI configuration manually to maintain physical index, + * searching for the next MMIO resource each time. + */ + for (INT bar_index = 0; bar_index < PCI_MAX_BAR; bar_index++) { + prev_bar = curr_bar; + curr_bar = pci_config.u.type0.BaseAddresses[bar_index]; + if (curr_bar == 0 || (prev_bar & PCI_TYPE_64BIT)) { + continue; + } + + // Find next CmResourceTypeMemory + do { + descriptor = WdfCmResourceListGetDescriptor(ResourcesTranslated, next_descriptor); + next_descriptor++; + + if (descriptor == NULL) { + status = STATUS_DEVICE_CONFIGURATION_ERROR; + goto end; + } + } while ((descriptor->Type != CmResourceTypeMemory) || + !(descriptor->Flags & CM_RESOURCE_MEMORY_BAR)); + + // Retrieve and map the BARs + ctx->bar[bar_index].base_addr.QuadPart = descriptor->u.Memory.Start.QuadPart; + ctx->bar[bar_index].size = descriptor->u.Memory.Length; + ctx->bar[bar_index].virt_addr = MmMapIoSpace(descriptor->u.Memory.Start, + descriptor->u.Memory.Length, + MmNonCached); + if (ctx->bar[bar_index].virt_addr == NULL) { + status = STATUS_INSUFFICIENT_RESOURCES; + goto end; + } + + // Allocate an MDL for the device BAR, so we can map it to the user's process context later. + ctx->dpdk_hw[bar_index].mdl = IoAllocateMdl(ctx->bar[bar_index].virt_addr, + (ULONG)ctx->bar[bar_index].size, + FALSE, + FALSE, + NULL); + if (!ctx->dpdk_hw[bar_index].mdl) { + status = STATUS_INSUFFICIENT_RESOURCES; + goto end; + } + + ctx->dpdk_hw[bar_index].mem.size = ctx->bar[bar_index].size; + } // for bar_index + + status = STATUS_SUCCESS; + +end: + if (status != STATUS_SUCCESS) { + netuio_free_hw_resources(Device); + } + + return status; +} + +_Use_decl_annotations_ +VOID +netuio_free_hw_resources(WDFDEVICE Device) +{ + PNETUIO_CONTEXT_DATA ctx; + ctx = netuio_get_context_data(Device); + + for (UINT8 bar_index = 0; bar_index < PCI_MAX_BAR; bar_index++) { + + // Free the allocated MDLs + if (ctx->dpdk_hw[bar_index].mdl) { + IoFreeMdl(ctx->dpdk_hw[bar_index].mdl); + } + + // Unmap all the BAR regions previously mapped + if (ctx->bar[bar_index].virt_addr) { + MmUnmapIoSpace(ctx->bar[bar_index].virt_addr, ctx->bar[bar_index].size); + } + } + + RtlZeroMemory(ctx->dpdk_hw, sizeof(ctx->dpdk_hw)); + RtlZeroMemory(ctx->bar, sizeof(ctx->bar)); +} diff --git a/windows/netuio/netuio_dev.h b/windows/netuio/netuio_dev.h new file mode 100644 index 000000000..62851141d --- /dev/null +++ b/windows/netuio/netuio_dev.h @@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Microsoft Corporation. + */ + +#ifndef NETUIO_DEV_H +#define NETUIO_DEV_H + +#include "netuio_interface.h" + +struct pci_bar { + PHYSICAL_ADDRESS base_addr; + PVOID virt_addr; + UINT64 size; +}; + +struct mem_map_region { + PMDL mdl; /**< MDL describing the memory region */ + struct mem_region mem; /**< Memory region details */ +}; + +struct dev_addr { + ULONG bus_num; + USHORT dev_num; + USHORT func_num; +}; + +/** + * The device context performs the same job as a WDM device extension in the driver frameworks + */ +typedef struct _NETUIO_CONTEXT_DATA +{ + WDFDEVICE wdf_device; /**< WDF device handle to the FDO */ + BUS_INTERFACE_STANDARD bus_interface; /**< Bus interface for config space access */ + struct pci_bar bar[PCI_MAX_BAR]; /**< device BARs */ + struct dev_addr addr; /**< B:D:F details of device */ + struct mem_map_region dpdk_hw[PCI_MAX_BAR]; /**< mapped region for the device's register space */ +} NETUIO_CONTEXT_DATA, *PNETUIO_CONTEXT_DATA; + + +/** + * This macro will generate an inline function called DeviceGetContext + * which will be used to get a pointer to the device context memory in a + * type safe manner. + */ +WDF_DECLARE_CONTEXT_TYPE_WITH_NAME(NETUIO_CONTEXT_DATA, netuio_get_context_data) + +typedef struct +{ + BOOLEAN bMapped; /**< value is set to TRUE if the User-mode mapping was done for this file object. */ +} NETUIO_FILE_CONTEXT_DATA, * PNETUIO_FILE_CONTEXT_DATA; + + +/** + * This macro will generate an inline function which will be used to get + * a pointer to the file object's context memory in a type safe manner. + */ +WDF_DECLARE_CONTEXT_TYPE_WITH_NAME(NETUIO_FILE_CONTEXT_DATA, netuio_get_file_object_context_data) + +/** + * Functions to initialize the device and its callbacks + */ +NTSTATUS netuio_create_device(_Inout_ PWDFDEVICE_INIT DeviceInit); +NTSTATUS netuio_map_hw_resources(_In_ WDFDEVICE Device, _In_ WDFCMRESLIST Resources, _In_ WDFCMRESLIST ResourcesTranslated); +VOID netuio_free_hw_resources(_In_ WDFDEVICE Device); + +#endif // NETUIO_DEV_H diff --git a/windows/netuio/netuio_drv.c b/windows/netuio/netuio_drv.c new file mode 100644 index 000000000..2d0fb3bd2 --- /dev/null +++ b/windows/netuio/netuio_drv.c @@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Microsoft Corporation. + */ + +#include "netuio_drv.h" + +#ifdef ALLOC_PRAGMA +#pragma alloc_text (INIT, DriverEntry) +#pragma alloc_text (PAGE, netuio_evt_device_add) +#endif + +/* +Routine Description: + DriverEntry initializes the driver and is the first routine called by the + system after the driver is loaded. DriverEntry specifies the other entry + points in the function driver, such as EvtDevice and DriverUnload. + +Return Value: + STATUS_SUCCESS if successful, + STATUS_UNSUCCESSFUL otherwise. + */ +NTSTATUS +DriverEntry(_In_ PDRIVER_OBJECT DriverObject, _In_ PUNICODE_STRING RegistryPath) +{ + WDF_DRIVER_CONFIG config; + NTSTATUS status; + WDF_OBJECT_ATTRIBUTES attributes; + + WDF_OBJECT_ATTRIBUTES_INIT(&attributes); + WDF_DRIVER_CONFIG_INIT(&config, netuio_evt_device_add); + + status = WdfDriverCreate(DriverObject, RegistryPath, + &attributes, &config, + WDF_NO_HANDLE); + + if (!NT_SUCCESS(status)) { + return status; + } + + return status; +} + +/* +Routine Description: + netuio_evt_device_add is called by the framework in response to AddDevice + call from the PnP manager. We create and initialize a device object to + represent a new instance of the device. + +Return Value: + NTSTATUS + */ +_Use_decl_annotations_ +NTSTATUS +netuio_evt_device_add(WDFDRIVER Driver, PWDFDEVICE_INIT DeviceInit) +{ + UNREFERENCED_PARAMETER(Driver); + return netuio_create_device(DeviceInit); +} + +/* +Routine Description : + Maps HW resources and retrieves the PCI BAR address(es) of the device + +Return Value : + STATUS_SUCCESS is successful. + STATUS_ otherwise +-*/ +_Use_decl_annotations_ +NTSTATUS +netuio_evt_prepare_hw(WDFDEVICE Device, WDFCMRESLIST Resources, WDFCMRESLIST ResourcesTranslated) +{ + NTSTATUS status; + + status = netuio_map_hw_resources(Device, Resources, ResourcesTranslated); + if (!NT_SUCCESS(status)) { + return status; + } + + PNETUIO_CONTEXT_DATA ctx = netuio_get_context_data(Device); + DbgPrintEx(DPFLTR_IHVNETWORK_ID, DPFLTR_NETUIO_INFO_LEVEL, "netUIO Driver loaded...on device (B:D:F) %04d:%02d:%02d\n", + ctx->addr.bus_num, ctx->addr.dev_num, ctx->addr.func_num); + + return status; +} + +/* +Routine Description : + Releases the resource mapped by netuio_evt_prepare_hw + +Return Value : + STATUS_SUCCESS always. +-*/ +_Use_decl_annotations_ +NTSTATUS +netuio_evt_release_hw(WDFDEVICE Device, WDFCMRESLIST ResourcesTranslated) +{ + UNREFERENCED_PARAMETER(ResourcesTranslated); + + netuio_free_hw_resources(Device); + + return STATUS_SUCCESS; +} + +/* +Routine Description : + EVT_WDF_FILE_CLEANUP callback, called when user process handle is closed. + + Undoes IOCTL_NETUIO_MAP_HW_INTO_USERMODE. + +Return value : + None +-*/ +_Use_decl_annotations_ +VOID +netuio_evt_file_cleanup(WDFFILEOBJECT FileObject) +{ + PNETUIO_FILE_CONTEXT_DATA file_ctx; + + file_ctx = netuio_get_file_object_context_data(FileObject); + + if (file_ctx->bMapped == TRUE) + { + WDFDEVICE device; + PNETUIO_CONTEXT_DATA ctx; + + device = WdfFileObjectGetDevice(FileObject); + ctx = netuio_get_context_data(device); + netuio_unmap_address_from_user_process(ctx); + file_ctx->bMapped = FALSE; + } +} diff --git a/windows/netuio/netuio_drv.h b/windows/netuio/netuio_drv.h new file mode 100644 index 000000000..469c6ac60 --- /dev/null +++ b/windows/netuio/netuio_drv.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Microsoft Corporation. + */ + +#ifndef NETUIO_DRV_H +#define NETUIO_DRV_H + +#define INITGUID + +#include +#include + +#include "netuio_dev.h" +#include "netuio_queue.h" + +/** + * Print output constants + */ +#define DPFLTR_NETUIO_INFO_LEVEL 35 + +/** + * WDFDRIVER Events + */ +DRIVER_INITIALIZE DriverEntry; +EVT_WDF_DRIVER_DEVICE_ADD netuio_evt_device_add; +EVT_WDF_DEVICE_PREPARE_HARDWARE netuio_evt_prepare_hw; +EVT_WDF_DEVICE_RELEASE_HARDWARE netuio_evt_release_hw; +EVT_WDF_FILE_CLOSE netuio_evt_file_cleanup; + +#endif // NETUIO_DRV_H diff --git a/windows/netuio/netuio_interface.h b/windows/netuio/netuio_interface.h new file mode 100644 index 000000000..232b5bc2d --- /dev/null +++ b/windows/netuio/netuio_interface.h @@ -0,0 +1,88 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Microsoft Corporation. + */ + + /** + * @file netuio kernel driver interface + */ + +#ifndef NETUIO_INTERFACE_H +#define NETUIO_INTERFACE_H + +/** + * All structures in this file are packed on an 8B boundary. + */ +#pragma pack(push) +#pragma pack(8) + +/** + * Define an Interface Guid so that any app can find the device and talk to it. + */ +DEFINE_GUID (GUID_DEVINTERFACE_netUIO, 0x08336f60,0x0679,0x4c6c,0x85,0xd2,0xae,0x7c,0xed,0x65,0xff,0xf7); // {08336f60-0679-4c6c-85d2-ae7ced65fff7} + +/** + * Device name definitions + */ +#define NETUIO_DEVICE_SYMBOLIC_LINK_UNICODE L"\\DosDevices\\netuio" +#define NETUIO_MAX_SYMLINK_LEN 255 + +/** + * IOCTL_NETUIO_MAP_HW_INTO_USERMODE is used for mapping the device registers + * into userspace. It returns the physical address, virtual address + * and the size of the memory region where the BARs were mapped. + */ +#define IOCTL_NETUIO_MAP_HW_INTO_USERMODE CTL_CODE(FILE_DEVICE_NETWORK, 51, METHOD_BUFFERED, FILE_READ_ACCESS | FILE_WRITE_ACCESS) + +/** + * IOCTL_NETUIO_PCI_CONFIG_IO is used to read/write from/into the device + * configuration space. + * + * Input: + * - the operation type (read/write) + * - the offset into the device data where the operation begins + * - the length of data to read/write. + * - in case of a write operation, the data to be written to the device + * configuration space. + * + * Output: + * - in case of a read operation, the output buffer is filled + * with the data read from the configuration space. + */ +#define IOCTL_NETUIO_PCI_CONFIG_IO CTL_CODE(FILE_DEVICE_NETWORK, 52, METHOD_BUFFERED, FILE_READ_ACCESS | FILE_WRITE_ACCESS) + +struct mem_region { + UINT64 size; /**< memory region size */ + LARGE_INTEGER phys_addr; /**< physical address of the memory region */ + PVOID virt_addr; /**< virtual address of the memory region */ + PVOID user_mapped_virt_addr; /**< virtual address of the region mapped into user process context */ +}; + +enum pci_io { + PCI_IO_READ = 0, + PCI_IO_WRITE = 1 +}; + +#define PCI_MAX_BAR 6 + +struct device_info +{ + struct mem_region hw[PCI_MAX_BAR]; +}; + +struct dpdk_pci_config_io +{ + UINT32 offset; /**< offset into the device config space where the reading/writing starts */ + UINT8 op; /**< operation type: read or write */ + UINT32 access_size; /**< 1, 2, 4, or 8 bytes */ + + union dpdk_pci_config_io_data { + UINT8 u8; + UINT16 u16; + UINT32 u32; + UINT64 u64; + } data; /**< Data to be written, in case of write operations */ +}; + +#pragma pack(pop) + +#endif // NETUIO_INTERFACE_H diff --git a/windows/netuio/netuio_queue.c b/windows/netuio/netuio_queue.c new file mode 100644 index 000000000..f7f75d14f --- /dev/null +++ b/windows/netuio/netuio_queue.c @@ -0,0 +1,334 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Microsoft Corporation. + */ + +#include "netuio_drv.h" + +#ifdef ALLOC_PRAGMA +#pragma alloc_text (PAGE, netuio_queue_initialize) +#endif + +static +BOOLEAN +netuio_get_usermode_mapping_flag(WDFREQUEST Request) +{ + WDFFILEOBJECT file_object; + + file_object = WdfRequestGetFileObject(Request); + if (file_object == NULL) + { + goto end; + } + + PNETUIO_FILE_CONTEXT_DATA file_ctx; + + file_ctx = netuio_get_file_object_context_data(file_object); + if (file_ctx->bMapped) + { + return TRUE; + } + +end: + return FALSE; +} + +static +VOID +netuio_set_usermode_mapping_flag(WDFREQUEST Request) +{ + WDFFILEOBJECT file_object; + + file_object = WdfRequestGetFileObject(Request); + if (file_object == NULL) + { + return; + } + + PNETUIO_FILE_CONTEXT_DATA file_ctx; + + file_ctx = netuio_get_file_object_context_data(file_object); + + file_ctx->bMapped = TRUE; +} + +static void +netuio_handle_get_hw_data_request(_In_ PNETUIO_CONTEXT_DATA ctx, + _In_ PVOID outputBuf, _In_ size_t outputBufSize) +{ + ASSERT(outputBufSize == sizeof(struct device_info)); + + struct device_info *dpdk_pvt_info = (struct device_info *)outputBuf; + RtlZeroMemory(dpdk_pvt_info, outputBufSize); + + for (ULONG idx = 0; idx < PCI_MAX_BAR; idx++) { + dpdk_pvt_info->hw[idx].phys_addr.QuadPart = ctx->bar[idx].base_addr.QuadPart; + dpdk_pvt_info->hw[idx].user_mapped_virt_addr = ctx->dpdk_hw[idx].mem.user_mapped_virt_addr; + dpdk_pvt_info->hw[idx].size = ctx->bar[idx].size; + } +} + +/* +Routine Description: + Maps address ranges into the usermode process's address space. The following + ranges are mapped: + + * Any PCI BARs that our device was assigned + * The scratch buffer of contiguous pages + +Return Value: + NTSTATUS +*/ +static NTSTATUS +netuio_map_address_into_user_process(_In_ PNETUIO_CONTEXT_DATA ctx, WDFREQUEST Request) +{ + NTSTATUS status = STATUS_SUCCESS; + + if (netuio_get_usermode_mapping_flag(Request)) + { + goto end; + } + + // Map any device BAR(s) to the user's process context + for (INT idx = 0; idx < PCI_MAX_BAR; idx++) { + if (ctx->dpdk_hw[idx].mdl == NULL) { + continue; + } + + MmBuildMdlForNonPagedPool(ctx->dpdk_hw[idx].mdl); + __try { + ctx->dpdk_hw[idx].mem.user_mapped_virt_addr = + MmMapLockedPagesSpecifyCache(ctx->dpdk_hw[idx].mdl, UserMode, + MmCached, NULL, FALSE, NormalPagePriority); + + if (ctx->dpdk_hw[idx].mem.user_mapped_virt_addr == NULL) { + status = STATUS_INSUFFICIENT_RESOURCES; + goto end; + } + } + __except (EXCEPTION_EXECUTE_HANDLER) { + status = GetExceptionCode(); + goto end; + } + } + +end: + if (status != STATUS_SUCCESS) { + netuio_unmap_address_from_user_process(ctx); + } + + return status; +} + +/* +Routine Description: + Unmaps all address ranges from the usermode process address space. + MUST be called in the context of the same process which created + the mapping. + +Return Value: + None + */ +_Use_decl_annotations_ +VOID +netuio_unmap_address_from_user_process(PNETUIO_CONTEXT_DATA ctx) +{ + for (INT idx = 0; idx < PCI_MAX_BAR; idx++) { + if (ctx->dpdk_hw[idx].mem.user_mapped_virt_addr != NULL) { + MmUnmapLockedPages(ctx->dpdk_hw[idx].mem.user_mapped_virt_addr, + ctx->dpdk_hw[idx].mdl); + + ctx->dpdk_hw[idx].mem.user_mapped_virt_addr = NULL; + } + } +} + +/* +Routine Description: + The I/O dispatch callbacks for the frameworks device object are configured here. + A single default I/O Queue is configured for parallel request processing, and a + driver context memory allocation is created to hold our structure QUEUE_CONTEXT. + +Return Value: + None + */ +_Use_decl_annotations_ +NTSTATUS +netuio_queue_initialize(WDFDEVICE Device) +{ + WDFQUEUE queue; + NTSTATUS status; + WDF_IO_QUEUE_CONFIG queueConfig; + + PAGED_CODE(); + + // Configure a default queue so that requests that are not + // configure-fowarded using WdfDeviceConfigureRequestDispatching to goto + // other queues get dispatched here. + WDF_IO_QUEUE_CONFIG_INIT_DEFAULT_QUEUE(&queueConfig, WdfIoQueueDispatchParallel); + + queueConfig.EvtIoDeviceControl = netuio_evt_IO_device_control; + + status = WdfIoQueueCreate(Device, + &queueConfig, + WDF_NO_OBJECT_ATTRIBUTES, + &queue); + + if( !NT_SUCCESS(status) ) { + return status; + } + + return status; +} + +/* +Routine Description: + This routine is invoked to preprocess an I/O request before being placed into a queue. + It is guaranteed that it executes in the context of the process that generated the request. + +Return Value: + None + */ +_Use_decl_annotations_ +VOID +netuio_evt_IO_in_caller_context(WDFDEVICE Device, + WDFREQUEST Request) +{ + WDF_REQUEST_PARAMETERS params = { 0 }; + NTSTATUS status = STATUS_SUCCESS; + PVOID output_buf = NULL; + size_t output_buf_size; + size_t bytes_returned = 0; + PNETUIO_CONTEXT_DATA ctx = NULL; + + ctx = netuio_get_context_data(Device); + + WDF_REQUEST_PARAMETERS_INIT(¶ms); + WdfRequestGetParameters(Request, ¶ms); + + if (params.Type != WdfRequestTypeDeviceControl) + { + status = STATUS_INVALID_DEVICE_REQUEST; + goto end; + } + + // We only need to be in the context of the process that initiated the request + //when we need to map memory to userspace. Otherwise, send the request back to framework. + if (params.Parameters.DeviceIoControl.IoControlCode != IOCTL_NETUIO_MAP_HW_INTO_USERMODE) + { + status = WdfDeviceEnqueueRequest(Device, Request); + if (!NT_SUCCESS(status)) + { + goto end; + } + return; + } + + // Return relevant data to the caller + status = WdfRequestRetrieveOutputBuffer(Request, sizeof(struct device_info), &output_buf, &output_buf_size); + if (!NT_SUCCESS(status)) { + goto end; + } + + status = netuio_map_address_into_user_process(ctx, Request); + if (!NT_SUCCESS(status)) { + goto end; + } + + netuio_set_usermode_mapping_flag(Request); + + netuio_handle_get_hw_data_request(ctx, output_buf, output_buf_size); + bytes_returned = output_buf_size; + +end: + WdfRequestCompleteWithInformation(Request, status, bytes_returned); + + return; +} + +/* +Routine Description: + This event is invoked when the framework receives IRP_MJ_DEVICE_CONTROL request. + +Return Value: + None + */ +_Use_decl_annotations_ +VOID +netuio_evt_IO_device_control(WDFQUEUE Queue, WDFREQUEST Request, + size_t OutputBufferLength, size_t InputBufferLength, + ULONG IoControlCode) +{ + UNREFERENCED_PARAMETER(OutputBufferLength); + UNREFERENCED_PARAMETER(InputBufferLength); + + NTSTATUS status = STATUS_SUCCESS; + PVOID input_buf = NULL, output_buf = NULL; + size_t input_buf_size, output_buf_size; + size_t bytes_returned = 0; + + WDFDEVICE device = WdfIoQueueGetDevice(Queue); + + PNETUIO_CONTEXT_DATA ctx; + ctx = netuio_get_context_data(device); + + if (IoControlCode != IOCTL_NETUIO_PCI_CONFIG_IO) + { + status = STATUS_INVALID_DEVICE_REQUEST; + goto end; + } + + // First retrieve the input buffer and see if it matches our device + status = WdfRequestRetrieveInputBuffer(Request, sizeof(struct dpdk_pci_config_io), &input_buf, &input_buf_size); + if (!NT_SUCCESS(status)) { + goto end; + } + + struct dpdk_pci_config_io *dpdk_pci_io_input = (struct dpdk_pci_config_io *)input_buf; + + if (dpdk_pci_io_input->access_size != 1 && + dpdk_pci_io_input->access_size != 2 && + dpdk_pci_io_input->access_size != 4 && + dpdk_pci_io_input->access_size != 8) { + status = STATUS_INVALID_PARAMETER; + goto end; + } + + // Retrieve output buffer + status = WdfRequestRetrieveOutputBuffer(Request, sizeof(UINT64), &output_buf, &output_buf_size); + if (!NT_SUCCESS(status)) { + goto end; + } + ASSERT(output_buf_size == OutputBufferLength); + + if (dpdk_pci_io_input->op == PCI_IO_READ) { + *(UINT64 *)output_buf = 0; + bytes_returned = ctx->bus_interface.GetBusData( + ctx->bus_interface.Context, + PCI_WHICHSPACE_CONFIG, + output_buf, + dpdk_pci_io_input->offset, + dpdk_pci_io_input->access_size); + } + else if (dpdk_pci_io_input->op == PCI_IO_WRITE) { + // returns bytes written + bytes_returned = ctx->bus_interface.SetBusData( + ctx->bus_interface.Context, + PCI_WHICHSPACE_CONFIG, + (PVOID)&dpdk_pci_io_input->data, + dpdk_pci_io_input->offset, + dpdk_pci_io_input->access_size); + } + else { + status = STATUS_INVALID_PARAMETER; + goto end; + } + + if (bytes_returned != dpdk_pci_io_input->access_size) { + status = STATUS_INVALID_PARAMETER; + } + +end: + WdfRequestCompleteWithInformation(Request, status, bytes_returned); + + return; +} diff --git a/windows/netuio/netuio_queue.h b/windows/netuio/netuio_queue.h new file mode 100644 index 000000000..29888537c --- /dev/null +++ b/windows/netuio/netuio_queue.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Microsoft Corporation. + */ + +#ifndef NETUIO_QUEUE_H +#define NETUIO_QUEUE_H + +VOID +netuio_unmap_address_from_user_process(_In_ PNETUIO_CONTEXT_DATA netuio_contextdata); + +NTSTATUS +netuio_queue_initialize(_In_ WDFDEVICE hDevice); + +/** + * Events from the IoQueue object + */ +EVT_WDF_IO_QUEUE_IO_DEVICE_CONTROL netuio_evt_IO_device_control; + +EVT_WDF_IO_IN_CALLER_CONTEXT netuio_evt_IO_in_caller_context; + +#endif // NETUIO_QUEUE_H -- 2.23.0.vfs.1.1.63.g5a5ad7f