From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id CB8592C0C for ; Wed, 6 Apr 2016 08:05:54 +0200 (CEST) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga101.jf.intel.com with ESMTP; 05 Apr 2016 23:05:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.24,447,1455004800"; d="scan'208";a="952659829" Received: from shwdeisgchi083.ccr.corp.intel.com (HELO [10.239.67.193]) ([10.239.67.193]) by fmsmga002.fm.intel.com with ESMTP; 05 Apr 2016 23:05:53 -0700 To: Yuanhan Liu References: <1459872587-11655-1-git-send-email-ciara.loftus@intel.com> <57049F5C.5080604@intel.com> <20160406054406.GV3080@yliu-dev.sh.intel.com> Cc: Ciara Loftus , dev@dpdk.org, mukawa@igel.co.jp From: "Tan, Jianfeng" Message-ID: <5704A73F.1040302@intel.com> Date: Wed, 6 Apr 2016 14:05:51 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.1 MIME-Version: 1.0 In-Reply-To: <20160406054406.GV3080@yliu-dev.sh.intel.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH] vhost: Fix retrieval of numa information in PMD X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Apr 2016 06:05:55 -0000 On 4/6/2016 1:44 PM, Yuanhan Liu wrote: > On Wed, Apr 06, 2016 at 01:32:12PM +0800, Tan, Jianfeng wrote: >> Hi, >> >> Just out of interest, seems that the message handling thread which runs >> new_device() is pthread_create() from the thread which calls the >> dev_start(), usually master thread, right? But it's not necessary to be the >> master thread to poll pkts from this vhost port, right? So what's the >> significance to record the numa_node information of message handling thread >> here? Shall we make the decision of numa_realloc based on the final PMD >> thread who is responsible for polling this vhost port? > It doesn't matter on which core we made the decision: the result > would be same since we are querying the numa node info of the > virtio_net dev struct. OK, according to your hint, read numa_realloc() again, it's to keep *dev (struct virtio_net), *dev->virtqueue[], on the same numa socket of shared memory with virtio? And numa_realloc() is called in vhost_set_vring_addr(), which is after new_device()? > > --yliu >> It's not related to this patch itself. And it seems good to me. >> >> >> Thanks, >> Jianfeng >> >> >> >> On 4/6/2016 12:09 AM, Ciara Loftus wrote: >>> After some testing, it was found that retrieving numa information >>> about a vhost device via a call to get_mempolicy is more >>> accurate when performed during the new_device callback versus >>> the vring_state_changed callback, in particular upon initial boot >>> of the VM. Performing this check during new_device is also >>> potentially more efficient as this callback is only triggered once >>> during device initialisation, compared with vring_state_changed >>> which may be called multiple times depending on the number of >>> queues assigned to the device. >>> >>> Reorganise the code to perform this check and assign the correct >>> socket_id to the device during the new_device callback. >>> >>> Signed-off-by: Ciara Loftus >>> --- >>> drivers/net/vhost/rte_eth_vhost.c | 28 ++++++++++++++-------------- >>> 1 file changed, 14 insertions(+), 14 deletions(-) >>> >>> diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c >>> index 4cc6bec..b1eb082 100644 >>> --- a/drivers/net/vhost/rte_eth_vhost.c >>> +++ b/drivers/net/vhost/rte_eth_vhost.c >>> @@ -229,6 +229,9 @@ new_device(struct virtio_net *dev) >>> struct pmd_internal *internal; >>> struct vhost_queue *vq; >>> unsigned i; >>> +#ifdef RTE_LIBRTE_VHOST_NUMA >>> + int newnode, ret; >>> +#endif >>> if (dev == NULL) { >>> RTE_LOG(INFO, PMD, "Invalid argument\n"); >>> @@ -244,6 +247,17 @@ new_device(struct virtio_net *dev) >>> eth_dev = list->eth_dev; >>> internal = eth_dev->data->dev_private; >>> +#ifdef RTE_LIBRTE_VHOST_NUMA >>> + ret = get_mempolicy(&newnode, NULL, 0, dev, >>> + MPOL_F_NODE | MPOL_F_ADDR); >>> + if (ret < 0) { >>> + RTE_LOG(ERR, PMD, "Unknown numa node\n"); >>> + return -1; >>> + } >>> + >>> + eth_dev->data->numa_node = newnode; If it's correct of above analysis, dev, here, could be numa_realloc() later? Thanks, Jianfeng >>> +#endif >>> + >>> for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { >>> vq = eth_dev->data->rx_queues[i]; >>> if (vq == NULL) >>> @@ -352,9 +366,6 @@ vring_state_changed(struct virtio_net *dev, uint16_t vring, int enable) >>> struct rte_vhost_vring_state *state; >>> struct rte_eth_dev *eth_dev; >>> struct internal_list *list; >>> -#ifdef RTE_LIBRTE_VHOST_NUMA >>> - int newnode, ret; >>> -#endif >>> if (dev == NULL) { >>> RTE_LOG(ERR, PMD, "Invalid argument\n"); >>> @@ -370,17 +381,6 @@ vring_state_changed(struct virtio_net *dev, uint16_t vring, int enable) >>> eth_dev = list->eth_dev; >>> /* won't be NULL */ >>> state = vring_states[eth_dev->data->port_id]; >>> - >>> -#ifdef RTE_LIBRTE_VHOST_NUMA >>> - ret = get_mempolicy(&newnode, NULL, 0, dev, >>> - MPOL_F_NODE | MPOL_F_ADDR); >>> - if (ret < 0) { >>> - RTE_LOG(ERR, PMD, "Unknown numa node\n"); >>> - return -1; >>> - } >>> - >>> - eth_dev->data->numa_node = newnode; >>> -#endif >>> rte_spinlock_lock(&state->lock); >>> state->cur[vring] = enable; >>> state->max_vring = RTE_MAX(vring, state->max_vring);