From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <stephen@networkplumber.org>
Received: from mail-pf1-f193.google.com (mail-pf1-f193.google.com
 [209.85.210.193]) by dpdk.org (Postfix) with ESMTP id 109AE2BFA
 for <users@dpdk.org>; Tue, 11 Sep 2018 20:07:53 +0200 (CEST)
Received: by mail-pf1-f193.google.com with SMTP id h69-v6so12625763pfd.4
 for <users@dpdk.org>; Tue, 11 Sep 2018 11:07:52 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=networkplumber-org.20150623.gappssmtp.com; s=20150623;
 h=date:from:to:cc:subject:message-id:in-reply-to:references
 :mime-version:content-transfer-encoding;
 bh=EmTz1yTG5WTIieSlMfcbign8UWwXQYQivwBMiZoZ6WU=;
 b=JhUalB5x4ktCb0eAYCFA4gqiTmXPxobz7MThL3s99ch8ZMtFIGOXaDgP0oLjRRYLIb
 pgk14J8BUEcjXg0IBarhhusBdHh3J+3x240qi07d+LiedWNS51mUavUgyaDqS5s0S12g
 mOI1m613YZBbSpS+eadpwDRyiKjmJV9EqHoMu/FKyqkqdvTCx7EgRlCJl0S1uzgms6Mc
 nLIe0cWRFgl2gMRwk65WewCr9ptmpyd4b+hGUV3/OXLULZGcS9XcY+YDnH+7I6iSkum6
 kqgJLajvGV6ICvXrSJGWtNWTqTzi/jwqszjRqwO7fKo+POOGnx/eVBdiILmbUNhwD7WT
 dT9Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to
 :references:mime-version:content-transfer-encoding;
 bh=EmTz1yTG5WTIieSlMfcbign8UWwXQYQivwBMiZoZ6WU=;
 b=GqyGTzoS+DdP6ebW31J0XWk5f0dpXLeRZ686qURpJsahmLYGpwaChvocKU6GCI0SWm
 J+f2rGCYakViJrqKEAfHzYkvFdShv96nWLiROVOGCiouN/djrROtxYyCeAAH4yCvrLLb
 MGOcsqMRrAxQAMBgiqkSOSe+rpWMx5VlEsthi75+0AQnjP3/yTIep7k0AIq35/F2LvIh
 R7SZIDey2uKwDWDKmYC1c6kIdD9QxKq7PSMnVZ2uRXvg9pKv0YKdVVCylyL7+lP6UxJ7
 nWc4DgPfkP5O3h6izI3CpECifDUfQl7PLNzImUe0sYacJyJyMiMMTi198ZtAvz2GtNSH
 XbwQ==
X-Gm-Message-State: APzg51CnXzauF6iDjOFokRsOHxDqGXL6EEVsM9Ey6bQmeSFvJu2tmT/U
 PAwie9iLOGU8dS5+Rj8xL8f94g==
X-Google-Smtp-Source: ANB0VdbEBC5bpXIBrTW8m/iYTZhHcgolEukGY1tHwgOy5NURX8KtEyyeiYUXqnFjLt/JLvgr9vK3Jg==
X-Received: by 2002:a62:1bc2:: with SMTP id
 b185-v6mr31169357pfb.170.1536689272009; 
 Tue, 11 Sep 2018 11:07:52 -0700 (PDT)
Received: from xeon-e3 (204-195-22-127.wavecable.com. [204.195.22.127])
 by smtp.gmail.com with ESMTPSA id t17-v6sm40164663pfm.138.2018.09.11.11.07.51
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Tue, 11 Sep 2018 11:07:51 -0700 (PDT)
Date: Tue, 11 Sep 2018 11:07:44 -0700
From: Stephen Hemminger <stephen@networkplumber.org>
To: Arvind Narayanan <webguru2688@gmail.com>
Cc: keith.wiles@intel.com, users@dpdk.org
Message-ID: <20180911110744.7ef55fc2@xeon-e3>
In-Reply-To: <CAHJJQSVNoV5yhrm0a3=sNke43aO8AqTvK030MJ2vLX8CzDF_8w@mail.gmail.com>
References: <CAHJJQSVG+ogufRTeCTBQPwkQaQ88DyM0DoD_FHckXaBi7f2dRg@mail.gmail.com>
 <E77998C2-0D7A-42CB-9F2C-49BD64EF4B0C@intel.com>
 <CAHJJQSVmzS4Sg7i3taKbYEBG=hFQ-BSNcbpsg5Sp-WqmMYBeTw@mail.gmail.com>
 <ABAC8AF0-FB54-4793-9FA7-5D735A4FCEF6@intel.com>
 <CAHJJQSVNoV5yhrm0a3=sNke43aO8AqTvK030MJ2vLX8CzDF_8w@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Subject: Re: [dpdk-users] How to use software prefetching for custom
 structures to increase throughput on the fast path
X-BeenThere: users@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK usage discussions <users.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/users>,
 <mailto:users-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/users/>
List-Post: <mailto:users@dpdk.org>
List-Help: <mailto:users-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/users>,
 <mailto:users-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Tue, 11 Sep 2018 18:07:53 -0000

On Tue, 11 Sep 2018 12:18:42 -0500
Arvind Narayanan <webguru2688@gmail.com> wrote:

> If I don't do any processing, I easily get 10G. It is only when I access
> the tag when the throughput drops.
> What confuses me is if I use the following snippet, it works at line rate.
> 
> ```
> int temp_key = 1; // declared outside of the for loop
> 
> for (i = 0; i < pkt_count; i++) {
>     if (rte_hash_lookup_data(rx_table, &(temp_key), (void **)&val[i]) < 0) {
>     }
> }
> ```
> 
> But as soon as I replace `temp_key` with `my_packet->tag1`, I experience
> fall in throughput (which in a way confirms the issue is due to cache
> misses).

Your packet data is not in cache.
Doing prefetch can help but it is very timing sensitive. If prefetch is done
before data is available it won't help. And if prefetch is done just before
data is used then there isn't enough cycles to get it from memory to the cache.