From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f178.google.com (mail-wi0-f178.google.com [209.85.212.178]) by dpdk.org (Postfix) with ESMTP id 743C88E90 for ; Thu, 15 Oct 2015 12:23:08 +0200 (CEST) Received: by wijp11 with SMTP id p11so22140530wij.0 for ; Thu, 15 Oct 2015 03:23:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=kReuGZB78UYXKZLMiCowHjJ4OJQ9bDwDYCfnnXAn3V0=; b=QYDQ0VIJiEqVbjdTG/OxDB18eceASmypkdNY3HuMooGX/Bj7s4f/ybW9I68ZBjIVnj qUzzAUoh94b3Oj8UgFnPJuKn3uvUSHX73LuBFhc6GMShO+ijsAIpiq63CyugF2TQ+l1J uonRwLBOfrUwmhZig67IOt+M4FZdZuj7AZ5TG9XRHUJTGVHfyWQgVRa/o1w1kAQTQNDm WAnpCNrA+7NkpG47tNPZIJMXqtIWaN6xQOi1nnY47pHf3X0gB31FW/tAzUR+xVVjvxPr RNHIy8QwusV0dgoyKfHx6linAOOMxvZLPRaiTZPj2yeltZKOWtHwfYchPWLro0AvlUV7 xDGg== X-Gm-Message-State: ALoCoQnylW8kRn9CnYorxqOdZeCUuhCuB9vqZS7fKvsC9of+ZlwCiXEku8zYoALOqRbBWv5Lg+Da X-Received: by 10.194.175.74 with SMTP id by10mr9466238wjc.78.1444904588226; Thu, 15 Oct 2015 03:23:08 -0700 (PDT) Received: from [192.168.0.101] ([90.152.119.35]) by smtp.googlemail.com with ESMTPSA id ft2sm10818106wic.24.2015.10.15.03.23.07 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Oct 2015 03:23:07 -0700 (PDT) To: Younghwan Go , dev@dpdk.org References: <561F64BA.2000502@ndsl.kaist.edu> From: Zoltan Kiss Message-ID: <561F7E8C.1080508@linaro.org> Date: Thu, 15 Oct 2015 11:23:08 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <561F64BA.2000502@ndsl.kaist.edu> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] Calling rte_eth_rx_burst() multiple times X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2015 10:23:08 -0000 On 15/10/15 09:32, Younghwan Go wrote: > Hi, > > I'm pretty new to playing with DPDK. I was trying to see if I can always > receive MAX_BURST packets by calling rte_eth_rx_burst() multiple times > on same pair (code shown below). I'm using DPDK-2.1.0 on 2 > dual-port Intel 82599ES 10Gbps NICs with Ubuntu 14.04.3 (kernel > 3.13.0-63-generic). > > Since packet processing is slower (~10 Gbps) than pure RX speed (~40 > Gbps), I assumed rte_eth_rx_burst() would always receive some number of > packets, eventually filling up MAX_BURST. But for multi-core case (4 > CPUs, 4 ports), rte_eth_rx_burst() starts to always return 0 after some > time, causing all cores to be blocked forever. Analyzing the DPDK code > (drivers/net/ixgbe/ixgbe_rxtx.c), I'm seeing that inside > ixgbe_rx_scan_hw_ring() function, "rxdp->wb.upper.status.error" always > returns 0 (where is this value set by the way?). I think it is set by the hardware. > > I didn't see this problem for single-core case, in which it returned > MAX_BURST packets at every rte_eth_rx_burst() call. Also, if I break out > of while loop when I receive 0, I keep receiving packets in next queue> pairs. Does anyone know why this block might happen? Or am I not > allowed to call rte_eth_rx_burst() multiple times on same > pair if I get 0? Any help will be great! Thank you! Although not mentioned in the documentation itself, as far as I know rte_eth_rx_burst() is not thread-safe. If you look in to receive functions, there are no locking anywhere. You should call it on separate queues from different threads, and configure e.g RSS to distribute the traffic by the hardware. > > ------------------------------------------------------------------------ > int cnt = MAX_BURST; // MAX_BURST = 32 > int off = 0; > do { > ret = rte_eth_rx_burst(port_id, queue_id, &m_table[off], cnt); > if (ret == 0) { > // don't break out but continue > } else if (ret > 0) { > off += ret; > cnt -= ret; > } > } while (cnt > 0); > ------------------------------------------------------------------------ > > Best, > Younghwan