From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f177.google.com (mail-wi0-f177.google.com [209.85.212.177]) by dpdk.org (Postfix) with ESMTP id C6AD28D8D for ; Thu, 15 Oct 2015 12:32:44 +0200 (CEST) Received: by wijq8 with SMTP id q8so122715827wij.0 for ; Thu, 15 Oct 2015 03:32:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=is7lfmx54tlvC1Yys5UoO2jLZzFOJdfiJLw16U+d8MY=; b=kyW++rc49fnRN0XPa7y7L7cBialGkMfkuDkcMliPptuNWx1QdGLFKHP/nQjhTS5LlM k8ZwVFRS8o2RQI95NfTtcZMWZsyj4hnc3TayBQOUUQdGbpBhDO/prsgFnVSJW2sv0B/k k2kmeDwwoisf0ZwlKM32B4lsePNwUhqoTOb+njPy6CHXmH0rdvCuTntdtDMg4GvPU1qs N+jFFNXtB1ZnEhbovNtUpBnuecytkxktl7k9c6fFiU1dLOTzfYfe78uVjBO1frOu68BO 1QA1BRaCgnCtpOa5IxgPZqdOHlVDzG+t0/02l4opt6eq/Q+0mjBMrOWX3DyywXGgJRgK puZw== X-Gm-Message-State: ALoCoQk+smKGkzXL5Rsi03ph+FIsaEG5+v+PeXclsvx8YR6schGH8TBDYQWeSrLRWCjsQnt4mJ9X X-Received: by 10.194.204.195 with SMTP id la3mr9346863wjc.77.1444905164513; Thu, 15 Oct 2015 03:32:44 -0700 (PDT) Received: from [192.168.0.101] ([90.152.119.35]) by smtp.googlemail.com with ESMTPSA id go5sm10874109wib.3.2015.10.15.03.32.43 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Oct 2015 03:32:44 -0700 (PDT) To: "Ananyev, Konstantin" , "Richardson, Bruce" , "dev@dpdk.org" References: <1441135036-7491-1-git-send-email-zoltan.kiss@linaro.org> <55ED8252.1020900@linaro.org> <59AF69C657FD0841A61C55336867B5B0359227DF@IRSMSX103.ger.corp.intel.com> <55ED9BFD.7040009@linaro.org> <59AF69C657FD0841A61C55336867B5B035922A83@IRSMSX103.ger.corp.intel.com> <56059255.5010507@linaro.org> <2601191342CEEE43887BDE71AB97725836A9CE89@irsmsx105.ger.corp.intel.com> <561E7E83.5000401@linaro.org> <2601191342CEEE43887BDE71AB97725836AAFBAB@irsmsx105.ger.corp.intel.com> From: Zoltan Kiss Message-ID: <561F80CC.4000809@linaro.org> Date: Thu, 15 Oct 2015 11:32:44 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <2601191342CEEE43887BDE71AB97725836AAFBAB@irsmsx105.ger.corp.intel.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH] ixgbe: prefetch packet headers in vector PMD receive function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2015 10:32:45 -0000 On 15/10/15 00:23, Ananyev, Konstantin wrote: > > >> -----Original Message----- >> From: Zoltan Kiss [mailto:zoltan.kiss@linaro.org] >> Sent: Wednesday, October 14, 2015 5:11 PM >> To: Ananyev, Konstantin; Richardson, Bruce; dev@dpdk.org >> Subject: Re: [PATCH] ixgbe: prefetch packet headers in vector PMD receive function >> >> >> >> On 28/09/15 00:19, Ananyev, Konstantin wrote: >>> >>> >>>> -----Original Message----- >>>> From: Zoltan Kiss [mailto:zoltan.kiss@linaro.org] >>>> Sent: Friday, September 25, 2015 7:29 PM >>>> To: Richardson, Bruce; dev@dpdk.org >>>> Cc: Ananyev, Konstantin >>>> Subject: Re: [PATCH] ixgbe: prefetch packet headers in vector PMD receive function >>>> >>>> On 07/09/15 07:41, Richardson, Bruce wrote: >>>>> >>>>>> -----Original Message----- >>>>>> From: Zoltan Kiss [mailto:zoltan.kiss@linaro.org] >>>>>> Sent: Monday, September 7, 2015 3:15 PM >>>>>> To: Richardson, Bruce; dev@dpdk.org >>>>>> Cc: Ananyev, Konstantin >>>>>> Subject: Re: [PATCH] ixgbe: prefetch packet headers in vector PMD receive >>>>>> function >>>>>> >>>>>> >>>>>> >>>>>> On 07/09/15 13:57, Richardson, Bruce wrote: >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Zoltan Kiss [mailto:zoltan.kiss@linaro.org] >>>>>>>> Sent: Monday, September 7, 2015 1:26 PM >>>>>>>> To: dev@dpdk.org >>>>>>>> Cc: Ananyev, Konstantin; Richardson, Bruce >>>>>>>> Subject: Re: [PATCH] ixgbe: prefetch packet headers in vector PMD >>>>>>>> receive function >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I just realized I've missed the "[PATCH]" tag from the subject. Did >>>>>>>> anyone had time to review this? >>>>>>>> >>>>>>> Hi Zoltan, >>>>>>> >>>>>>> the big thing that concerns me with this is the addition of new >>>>>>> instructions for each packet in the fast path. Ideally, this >>>>>>> prefetching would be better handled in the application itself, as for >>>>>>> some apps, e.g. those using pipelining, the core doing the RX from the >>>>>>> NIC may not touch the packet data at all, and the prefetches will >>>>>> instead cause a performance slowdown. >>>>>>> Is it possible to get the same performance increase - or something >>>>>>> close to it - by making changes in OVS? >>>>>> OVS already does a prefetch when it's processing the previous packet, but >>>>>> apparently it's not early enough. At least for my test scenario, where I'm >>>>>> forwarding UDP packets with the least possible overhead. I guess in tests >>>>>> where OVS does more complex processing it should be fine. >>>>>> I'll try to move the prefetch earlier in OVS codebase, but I'm not sure if >>>>>> it'll help. >>>>> I would suggest trying to prefetch more than one packet ahead. Prefetching 4 or >>>>> 8 ahead might work better, depending on the processing being done. >>>> >>>> I've moved the prefetch earlier, and it seems to work: >>>> >>>> https://patchwork.ozlabs.org/patch/519017/ >>>> >>>> However it raises the question: should we remove header prefetch from >>>> all the other drivers, and the CONFIG_RTE_PMD_PACKET_PREFETCH option? >>> >>> My vote would be for that. >>> Konstantin >> >> After some further thinking I would rather support the >> rte_packet_prefetch() macro (prefetch the header in the driver, and >> configure it through CONFIG_RTE_PMD_PACKET_PREFETCH) >> >> - the prefetch can happen earlier, so if an application needs the header >> right away, this is the fastest >> - if the application doesn't need header prefetch, it can turn it off >> compile time. Same as if we wouldn't have this option. >> - if the application has mixed needs (sometimes it needs the header >> right away, sometimes it doesn't), it can turn it off and do what it >> needs. Same as if we wouldn't have this option. >> >> A harder decision would be whether it should be turned on or off by >> default. Currently it's on, and half of the receive functions don't use it. > > Yep, it is sort of a mess right now. > Another question if we'd like to keep it and standardise it: > at what moment to prefetch: as soon as we realize that HW is done with that buffer, > or as late inside rx_burst() as possible? > I suppose there is no clear answer for that. I think if the application needs the driver start the prefetching, it does it because it's already too late when rte_eth_rx_burst() returns. So I think it's best to do it as soon as we learn that the HW is done. > That's why my thought was to just get rid of it. > Though if it would be implemented in some standardized way and disabled by default - > that's probably ok too. > BTW, couldn't that be implemented just via rte_ethdev rx callbacks mechanism? > Then we can have the code one place and don't need compile option at all - > could be ebabled/disabled dynamically on a per nic basis. > Or would it be too high overhead for that? I think if we go that way, it's better to pass an option to rte_eth_rx_burst() and tell if you want the header prefetched by the driver. That would be the most flexible. > Konstantin > > > >> >>> >>> >>>> >>>>> >>>>> /Bruce >>>