DPDK usage discussions
 help / color / mirror / Atom feed
From: Mark Cheatham <mcheatham@boulderimaging.com>
To: "users@dpdk.org" <users@dpdk.org>
Subject: [dpdk-users] Windows Interrupts with XL710/i40e
Date: Mon, 28 Jun 2021 22:17:52 +0000
Message-ID: <CY4PR08MB2566C7D2FD8B109B7D687742C2039@CY4PR08MB2566.namprd08.prod.outlook.com> (raw)

Hello All,

We have a very demanding project that requires capturing GVSP (GigE Vision Streaming Protocol) packets at ~2.5GB/s burst rates into Windows. For the past year or so, we have been building and using the draft DPDK repository located here: https://git.dpdk.org/draft/dpdk-draft-windows/

Overall, this has worked extremely well for our needs. We were also able to modify the source to include the e1000 (Intel I350) and store into externally allocated buffers.

Using the poll mode drivers for both e1000 and i40e, we see some interesting behavior that I believe is due to the cores spinning during times of low/no activity. These packets are fed into our application that has a lot of concurrent work happening. We were able to put in delays of a few milliseconds at points during the receive when we know no data should be coming in. This showed noticeable improvement; however, it would be great to have interrupt support.

We modified the netUIO Windows kernel driver and added WdfInterruptCreate() calls and added MSI support via the INF file. Brought in the same register sets into the driver into the ISR callback that was created to enable/disable interrupts. This works fine on the e1000, but for the i40e, there are some strange lags in getting the interrupt processed. We were not able to get an "interrupt 0 only" mode working for the i40e and had to enable the MSI-X vectors, but we see multiple milliseconds between when the ISR call occurs for an Rx packet and when we actually get our event set in the DPC. We have an FPGA that is generating the GVSP packets, and for a given image frame, it will always take ~18ms between the leader and trailer to be transmitted, and we are seeing cases of our Rx burst read taking less than this amount, which means that we begin reading well after the image frame has started to arrive. Also, as soon as an interrupt happens, we disable them and don't enable them until the read finishes. We are looking at dealing with 50 interrupts per second, max.

It should be noted that we are not using any of the interrupt skeleton code for Windows in the latest DPDK version. We are using an event handle (CreateEvent()) that's passed in to the netUIO driver, and then it's checked for being set with a separate DeviceIoControl() call. Once the event is set in the DPC, we kick off the Rx burst reads. I also tried changing some of the EITR indexes but no luck there. Again, this approach works fine in the e1000 driver with no delays.

Could it be the firmware/NVM on the 710? What else could be missing? From what I have read it should be on the order of <100 microseconds to issue the interrupt. I did not see anything obviously different from the interrupt register values on the draft repository vs. 21.05 (latest as of this email).

Thank you all for the input!

Regards,
Mark

             reply	other threads:[~2021-06-29  7:11 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-28 22:17 Mark Cheatham [this message]
2021-06-29 19:50 ` Dmitry Kozlyuk
2021-07-01 15:58   ` Mark Cheatham

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CY4PR08MB2566C7D2FD8B109B7D687742C2039@CY4PR08MB2566.namprd08.prod.outlook.com \
    --to=mcheatham@boulderimaging.com \
    --cc=users@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK usage discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://inbox.dpdk.org/users/0 users/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 users users/ http://inbox.dpdk.org/users \
		users@dpdk.org
	public-inbox-index users

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.users


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git