From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from redflag.CS.Princeton.EDU (redflag.CS.Princeton.EDU [128.112.136.68]) by dpdk.org (Postfix) with ESMTP id C55957EB0 for ; Sun, 12 Apr 2015 07:10:05 +0200 (CEST) Received: from mail-ig0-f175.google.com (mail-ig0-f175.google.com [209.85.213.175]) (authenticated bits=0) by redflag.CS.Princeton.EDU (8.13.8/8.13.8) with ESMTP id t3C5A2It027006 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NOT) for ; Sun, 12 Apr 2015 01:10:04 -0400 Received: by igblo3 with SMTP id lo3so23917654igb.0 for ; Sat, 11 Apr 2015 22:10:01 -0700 (PDT) X-Received: by 10.50.143.38 with SMTP id sb6mr5567265igb.44.1428815401793; Sat, 11 Apr 2015 22:10:01 -0700 (PDT) MIME-Version: 1.0 From: Raghav Sethi Date: Sun, 12 Apr 2015 05:10:01 +0000 Message-ID: To: "dev@dpdk.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] Mellanox Flow Steering X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Apr 2015 05:10:06 -0000 Hi folks, I'm trying to use the flow steering features of the Mellanox card to effectively use a multicore server for a benchmark. The system has a single-port Mellanox ConnectX-3 EN, and I want to use 4 of the 32 cores present and 4 of the 16 RX queues supported by the hardware (i.e. one RX queue per core). I assign RX queues to each of the cores, but obviously without flow steering (all the packets have the same IP and UDP headers, but different dest MACs in the ethernet headers) each of the packets hits one core. I've set up the client such that it sends packets with a different destination MAC for each RX queue (e.g. RX queue 1 should get 10:00:00:00:00:00, RX queue 2 should get 10:00:00:00:00:01 and so on). I try to accomplish this by using ethtool to set flow steering rules (e.g. ethtool -U p7p1 flow-type ether dst 10:00:00:00:00:00 action 1 loc 1, ethtool -U p7p1 flow-type ether dst 10:00:00:00:00:01 action 2 loc 2..). As soon as I set up these rules though, packets matching them just stop hitting my application. All other packets go through, and removing the rules also causes the packets to go through. I'm pretty sure my application is looking at all the queues, but I tried changing the rules to try a rule for every single destination RX queue (0-16), and that doesn't work either. If it helps, my code is based on the l2fwd sample application, and is here: https://gist.github.com/raghavsethi/416fb77d74ccf81bd93e Also, I added the following to my /etc/init.d: options mlx4_core log_num_mgm_entry_size=-1, and restarted the driver before any of these tests. Any ideas what might be causing my packets to drop? In case this is a Mellanox issue, should I be talking to their customer support? Best, Raghav Sethi