From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0B01FA0A0C for ; Tue, 29 Jun 2021 21:50:57 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8AE5B411A5; Tue, 29 Jun 2021 21:50:56 +0200 (CEST) Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com [209.85.167.45]) by mails.dpdk.org (Postfix) with ESMTP id E9B5440E01 for ; Tue, 29 Jun 2021 21:50:55 +0200 (CEST) Received: by mail-lf1-f45.google.com with SMTP id k10so365024lfv.13 for ; Tue, 29 Jun 2021 12:50:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pjAipu494gtckN9Y9fOc9xOQ9YRZ1mnW4KT1J52jrKY=; b=Ivz8up77Q8O7hAQPIEoFs9fKa6VOqiGHOA3b7kItGOSTLb9OaGt8zI1UzFVNxwIwEU 3dCTd0e5fsrr/ZPCS+wKorXhKuZtPzJzFnKc1FGwGEPAft60w+f0C2a7xf6PFXrCvSg1 f0cVAFcpa3LK4tqrfe3Bawm31JgCoNE1txpnJI5B7Rgng2MBK3dS5Va9Pj8u36pEw5qV njuJAko+XCZnJPvcwxzoCycTERp5IM2aYMYvrbDCjMIdMSDSrDhyYy9C2F4POX0kIpoh 7PgwRBdvVKvJUzjrpcuny31OEzKkWvRF4j+UNmuhzPrbGCJ7B9W9XiIAqb8Wy5eger8b C+pQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pjAipu494gtckN9Y9fOc9xOQ9YRZ1mnW4KT1J52jrKY=; b=MFgEy3s41BbHljVsjphblGComt4wIezufOTrYJQjRfMSYvl7Xyatyc9CUprOvxrNH/ gKDslCJ+kvHsVWYCTAAnwclFsKcK0/hAEnlvdler+idaq/4fyAjtSeZhRmugroG2QIjY jGfJ0+i72PIffa/GK6jGOKrlj2RD+x6CyXsIJSSXsCIyy8FUxdnlya0Ydozum/SYUw2S q+u3uHC1zt4nw9Q46TWJLdI+MWBgiRNHGrxEMFfXwpuXX4Z44RioVOUYqKhfFx37x8qg kQlrmrUbLWLddPsuPTkooFPlH8KWZqFsyRXk8p8hBIx7VHhbnoeib3wasvLqjykytBzv WAUg== X-Gm-Message-State: AOAM531QDY2oqMGTM3IXZ+hD6OBSPkgTXzUFdj7VkTgg8JKzWoNRzvf7 RTYKhtW6CgVCN+pXZQArIWs= X-Google-Smtp-Source: ABdhPJzzEeJ4qYgsFEZOJsfz/3ROU2A7nHopQwfIflgr7VtC93yDK9I5/9zX3YrLTFqYmkOjjMaFNQ== X-Received: by 2002:a19:8c55:: with SMTP id i21mr16432424lfj.409.1624996255352; Tue, 29 Jun 2021 12:50:55 -0700 (PDT) Received: from sovereign (broadband-37-110-65-23.ip.moscow.rt.ru. [37.110.65.23]) by smtp.gmail.com with ESMTPSA id g5sm166147ljm.58.2021.06.29.12.50.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Jun 2021 12:50:54 -0700 (PDT) Date: Tue, 29 Jun 2021 22:50:52 +0300 From: Dmitry Kozlyuk To: Mark Cheatham Cc: "users@dpdk.org" , "Kadam, Pallavi" , "Menon, Ranjit" , "Dmitry Malloy (MESHCHANINOV)" , Narcisa Ana Maria Vasile , Harini Ramakrishnan Message-ID: <20210629224929.668f9ba9@sovereign> In-Reply-To: References: X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-users] Windows Interrupts with XL710/i40e X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Hello, Mark! + Windows maintainers from Intel and Microsoft to answer your questions regarding XL710 and kernel. First of all, sounds like you've done a great job, congratulations! Windows draft repository is not developed anymore. DPDK has its own subsystem for handling interrupts. In upstream Windows DPDK I'm implementing it currently, no patches sent yet. The plan is to use IOCP and not events, but the flow will be similar: the app subscribes to an interrupt event and gets its callback executed. If you have any requirements for DPDK beyond that, please share them. There is a bi-weekly Windows DPDK Community Call, if you want to participate in discussions and to stay in the loop. Harini (cc'd) can send an invitation. 2021-06-28 22:17 (UTC+0000), Mark Cheatham: > We have a very demanding project that requires capturing GVSP (GigE Vision > Streaming Protocol) packets at ~2.5GB/s burst rates into Windows. For the > past year or so, we have been building and using the draft DPDK repository > located here: https://git.dpdk.org/draft/dpdk-draft-windows/ > > Overall, this has worked extremely well for our needs. We were also able to > modify the source to include the e1000 (Intel I350) and store into > externally allocated buffers. Using the poll mode drivers for both e1000 > and i40e, we see some interesting behavior that I believe is due to the > cores spinning during times of low/no activity. These packets are fed into > our application that has a lot of concurrent work happening. We were able > to put in delays of a few milliseconds at points during the receive when we > know no data should be coming in. This showed noticeable improvement; > however, it would be great to have interrupt support. > > We modified the netUIO Windows kernel driver and added WdfInterruptCreate() > calls and added MSI support via the INF file. Brought in the same register > sets into the driver into the ISR callback that was created to > enable/disable interrupts. This works fine on the e1000, but for the i40e, > there are some strange lags in getting the interrupt processed. We were not > able to get an "interrupt 0 only" mode working for the i40e and had to > enable the MSI-X vectors, but we see multiple milliseconds between when the > ISR call occurs for an Rx packet and when we actually get our event set in > the DPC. We have an FPGA that is generating the GVSP packets, and for a > given image frame, it will always take ~18ms between the leader and trailer > to be transmitted, and we are seeing cases of our Rx burst read taking less > than this amount, which means that we begin reading well after the image > frame has started to arrive. Also, as soon as an interrupt happens, we > disable them and don't enable them until the read finishes. We are looking > at dealing with 50 interrupts per second, max. > > It should be noted that we are not using any of the interrupt skeleton code > for Windows in the latest DPDK version. We are using an event handle > (CreateEvent()) that's passed in to the netUIO driver, and then it's > checked for being set with a separate DeviceIoControl() call. Once the > event is set in the DPC, we kick off the Rx burst reads. I also tried > changing some of the EITR indexes but no luck there. Again, this approach > works fine in the e1000 driver with no delays. > > Could it be the firmware/NVM on the 710? What else could be missing? From > what I have read it should be on the order of <100 microseconds to issue > the interrupt. I did not see anything obviously different from the > interrupt register values on the draft repository vs. 21.05 (latest as of > this email). > > Thank you all for the input! > > Regards, > Mark