From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f54.google.com (mail-wm0-f54.google.com [74.125.82.54]) by dpdk.org (Postfix) with ESMTP id 847B72BD1 for ; Fri, 16 Jun 2017 15:40:06 +0200 (CEST) Received: by mail-wm0-f54.google.com with SMTP id u195so12258740wmd.1 for ; Fri, 16 Jun 2017 06:40:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=P80CP3a3PyKYxTHdlqxvHhQ3ou7CAJp9SFYaGbFSCC0=; b=BNvOYveV25XSyilRFivpWQe6goHjMe0y5VjazsojFXq91k6A/D51n9ylI25SJ0ga5f mcpPhxp0dca9R2+rhSPVKu/5yAVDd2H7obhRvTZbH12cUcoUD0jSX1Eq4azIlEc5dMXM B04pvv6vnfVG6GBCJVO30VAEYZYrIFCLq7YvwR7sYRkLAdXXZ+Ps2asnmsw6SzuCg6qe XAJyZpohhUckKUVIfmX8WtnWshT7kA7r/tKlrHwjRcv4nBIzOfXJoQE3MtyzbaNRpZFS pxbTEy520Pj8LcDSs2zsSWMahUvam07xKak4SZWNQHQDwxDZ1kuC9Ptc+nm/OZgQk/qX cPpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=P80CP3a3PyKYxTHdlqxvHhQ3ou7CAJp9SFYaGbFSCC0=; b=qDP71r9z9EGGhT18OnfnOVD6NdVyzUQ7l4afdPin8DombkLXIw/bZIGIAM8UfIvB4l 8EUD335dnIebAGNrlvPgorsxOn7ynbem5JRY6apiJYxkxhJ6nNUBfiRlG4Ig9seLUAp1 LCb8iQ8TXG1fV8EoUgnZO+ztXcO76PYU/6Yz+U82b/S86/pTTVNTD99vwpqLTstfjR8Y DH/+yp/B1YFv5sO0si5PTI6ha1vI99qLOE97Dd8EWjT6BTMSxDMdV51U3iRuAHDQHWlv UKMatcIoFHAk0QmcVWBY7gFQVR5W9VCHau5mjIHvenLZHOZdMCpvG1z+llk8/EJrwDxC 71vg== X-Gm-Message-State: AKS2vOz+D9gc9FNMbu2N0+WbzPL3C1XNq02RKpLfqloKqNBOmeSKGVHc iuS/yLZEmSeKSs2n X-Received: by 10.28.19.11 with SMTP id 11mr7194336wmt.123.1497620404314; Fri, 16 Jun 2017 06:40:04 -0700 (PDT) Received: from 6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id h123sm2684147wmd.34.2017.06.16.06.40.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 16 Jun 2017 06:40:03 -0700 (PDT) Date: Fri, 16 Jun 2017 15:39:55 +0200 From: Adrien Mazarguil To: Ferruh Yigit Cc: dev@dpdk.org Message-ID: <20170616133955.GD1758@6wind.com> References: <2407596e1e05260170f8b0b80806b28b63e9255d.1497439616.git.adrien.mazarguil@6wind.com> <08cf1e01-f0b9-9540-9cc5-0b7fb79d8799@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <08cf1e01-f0b9-9540-9cc5-0b7fb79d8799@intel.com> Subject: Re: [dpdk-dev] [PATCH 2/7] net/mlx4: fix Rx interrupts with multiple ports X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2017 13:40:06 -0000 On Fri, Jun 16, 2017 at 02:07:54PM +0100, Ferruh Yigit wrote: > On 6/14/2017 12:49 PM, Adrien Mazarguil wrote: > > Several Ethernet device structures are allocated on top of a common PCI > > device for mlx4 adapters with multiple ports. These inherit a common > > interrupt handle from their parent PCI device, which prevents Rx interrupts > > from working properly on all ports as their configuration is overwritten. > > > > Use a local interrupt handle to address this issue. > > Hi Adrien, > > I am not clear why local copy required, and main concern from my point > of view is if this is a common problem for all PMDs and should be > addressed in higher level? This issue only affects PMDs that handle multiple Ethernet ports through a single PCI device. Such PMDs (like mlx4) identify themselves as PCI drivers that manually have to register multiple rte_eth_dev instances through rte_eth_dev_allocate(), which they then have to initialize. > The variable is already per eth_dev, but this patch moves it the private > data. What overwrites it within eth_dev? Calling rte_eth_copy_pci_info() makes the rte_eth_dev structure inherit the default interrupt handle of the underlying PCI device. By "inherit", I mean eth_dev->intr_handle points to it, in that sense it's not per eth_dev but per PCI device. mlx4 Rx interrupts are associated with a given Verbs context, and each port has its own Verbs context, so they cannot be shared, while other PMDs using other methods for catching interrupts may be perfectly fine with a single vector associated with the PCI device. It depends on the PMD, for instance there is no such problem with mlx5 as exactly one PCI device is associated with a given port. This patch merely allocates a specific interrupt handle associated with the eth_dev itself and makes the eth_dev handle point to that instead of the default PCI handle. This "local" handle is initialized using the PCI handle as a template before modifying the pointer. It's completely safe. > > Fixes: 9f05a4b81809 ("net/mlx4: support user space Rx interrupt event") > > > > Signed-off-by: Adrien Mazarguil > > Acked-by: Moti Haimovsky > > --- > > drivers/net/mlx4/mlx4.c | 9 +++++++++ > > drivers/net/mlx4/mlx4.h | 1 + > > 2 files changed, 10 insertions(+) > > > > diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c > > index 178562e..2b4722f 100644 > > --- a/drivers/net/mlx4/mlx4.c > > +++ b/drivers/net/mlx4/mlx4.c > > @@ -6207,6 +6207,15 @@ mlx4_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) > > > > eth_dev->device->driver = &mlx4_driver.driver; > > > > + /* > > + * Copy and override interrupt handle to prevent it from > > + * being shared between all ethdev instances of a given PCI > > + * device. This is required to properly handle Rx interrupts > > + * on all ports. > > + */ > > + priv->intr_handle_dev = *eth_dev->intr_handle; > > + eth_dev->intr_handle = &priv->intr_handle_dev; > > + > > priv->dev = eth_dev; > > eth_dev->dev_ops = &mlx4_dev_ops; > > > > diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h > > index c46fc23..b74fbf8 100644 > > --- a/drivers/net/mlx4/mlx4.h > > +++ b/drivers/net/mlx4/mlx4.h > > @@ -345,6 +345,7 @@ struct priv { > > unsigned int txqs_n; /* TX queues array size. */ > > struct rxq *(*rxqs)[]; /* RX queues. */ > > struct txq *(*txqs)[]; /* TX queues. */ > > + struct rte_intr_handle intr_handle_dev; /* Device interrupt handler. */ > > struct rte_intr_handle intr_handle; /* Interrupt handler. */ > > struct rte_flow_drop *flow_drop_queue; /* Flow drop queue. */ > > LIST_HEAD(mlx4_flows, rte_flow) flows; > > > -- Adrien Mazarguil 6WIND