From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id EE5CAF919; Wed, 28 Dec 2016 12:00:59 +0100 (CET) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga104.fm.intel.com with ESMTP; 28 Dec 2016 03:00:55 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,422,1477983600"; d="scan'208";a="23663565" Received: from yliu-dev.sh.intel.com ([10.239.67.162]) by orsmga002.jf.intel.com with ESMTP; 28 Dec 2016 03:00:53 -0800 From: Yuanhan Liu To: dev@dpdk.org Cc: Yuanhan Liu , stable@dpdk.org, Thomas Monjalon , Bruce Richardson , Ferruh Yigit Date: Wed, 28 Dec 2016 19:02:37 +0800 Message-Id: <1482922962-21036-2-git-send-email-yuanhan.liu@linux.intel.com> X-Mailer: git-send-email 1.9.0 In-Reply-To: <1482922962-21036-1-git-send-email-yuanhan.liu@linux.intel.com> References: <1482391123-8149-1-git-send-email-yuanhan.liu@linux.intel.com> <1482922962-21036-1-git-send-email-yuanhan.liu@linux.intel.com> Subject: [dpdk-dev] [PATCH v2 1/6] ethdev: fix port data mismatched in multiple process model X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Dec 2016 11:01:00 -0000 Assume we have two virtio ports, 00:03.0 and 00:04.0. The first one is managed by the kernel driver, while the later one is managed by DPDK. Now we start the primary process. 00:03.0 will be skipped by DPDK virtio PMD driver (since it's being used by the kernel). 00:04.0 would be successfully initiated by DPDK virtio PMD (if nothing abnormal happens). After that, we would get a port id 0, and all the related info needed by virtio (virtio_hw) is stored at rte_eth_dev_data[0]. Then we start the secondary process. As usual, 00:03.0 will be firstly probed. It firstly tries to get a local eth_dev structure for it (by rte_eth_dev_allocate): port_id = rte_eth_dev_find_free_port(); ... eth_dev = &rte_eth_devices[port_id]; eth_dev->data = &rte_eth_dev_data[port_id]; ... return eth_dev; Since it's a first PCI device, port_id will be 0. eth_dev->data would then point to rte_eth_dev_data[0]. And here things start going wrong, as rte_eth_dev_data[0] actually stores the virtio_hw for 00:04.0. That said, in the secondary process, DPDK will continue to drive PCI device 00.03.0 (despite the fact it's been managed by kernel), with the info from PCI device 00:04.0. Which is wrong. The fix is to attach the port already registered by the primary process: iterate the rte_eth_dev_data[], and get the port id who's PCI ID matches the current PCI device. This would let us maintain same port ID for the same PCI device, keeping the chance of referencing to wrong data minimal. Fixes: af75078fece3 ("first public release") Cc: stable@dpdk.org Cc: Thomas Monjalon Cc: Bruce Richardson Cc: Ferruh Yigit Signed-off-by: Yuanhan Liu --- lib/librte_ether/rte_ethdev.c | 58 ++++++++++++++++++++++++++++++++++++++----- 1 file changed, 52 insertions(+), 6 deletions(-) diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index fde8112..c10eb9c 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -201,9 +201,6 @@ rte_eth_dev_allocate(const char *name) return NULL; } - if (rte_eth_dev_data == NULL) - rte_eth_dev_data_alloc(); - if (rte_eth_dev_allocated(name) != NULL) { RTE_PMD_DEBUG_TRACE("Ethernet Device with name %s already allocated!\n", name); @@ -231,6 +228,38 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) return 0; } +/* + * Attach to a port already registered by the primary process, which + * makes sure that the same device would both have the same port id + * in the primary and secondary process. + */ +static struct rte_eth_dev * +eth_dev_attach(const char *name) +{ + uint8_t i; + struct rte_eth_dev *eth_dev; + + for (i = 0; i < RTE_MAX_ETHPORTS; i++) { + if (strcmp(rte_eth_dev_data[i].name, name) == 0) + break; + } + if (i == RTE_MAX_ETHPORTS) { + RTE_PMD_DEBUG_TRACE( + "device %s is not driven by the primary process\n", + name); + return NULL; + } + + RTE_ASSERT(eth_dev->data->port_id == i); + + eth_dev = &rte_eth_devices[i]; + eth_dev->data = &rte_eth_dev_data[i]; + eth_dev->attached = DEV_ATTACHED; + nb_ports++; + + return eth_dev; +} + int rte_eth_dev_pci_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) @@ -246,9 +275,26 @@ rte_eth_dev_pci_probe(struct rte_pci_driver *pci_drv, rte_eal_pci_device_name(&pci_dev->addr, ethdev_name, sizeof(ethdev_name)); - eth_dev = rte_eth_dev_allocate(ethdev_name); - if (eth_dev == NULL) - return -ENOMEM; + if (rte_eth_dev_data == NULL) + rte_eth_dev_data_alloc(); + + if (rte_eal_process_type() == RTE_PROC_PRIMARY) { + eth_dev = rte_eth_dev_allocate(ethdev_name); + if (eth_dev == NULL) + return -ENOMEM; + } else { + /* + * if we failed to attach a device, it means that + * device is skipped, due to some errors. Take + * virtio-net device as example, it could be the + * device is managed by virtio-net kernel driver. + * For such case, we return a positive value, to + * let EAL skip it as well. + */ + eth_dev = eth_dev_attach(ethdev_name); + if (eth_dev == NULL) + return 1; + } if (rte_eal_process_type() == RTE_PROC_PRIMARY) { eth_dev->data->dev_private = rte_zmalloc("ethdev private structure", -- 2.8.1