From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6B1F0A0C43; Thu, 26 Aug 2021 13:28:46 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 33E6040689; Thu, 26 Aug 2021 13:28:46 +0200 (CEST) Received: from mail-il1-f182.google.com (mail-il1-f182.google.com [209.85.166.182]) by mails.dpdk.org (Postfix) with ESMTP id 0E74040140 for ; Thu, 26 Aug 2021 13:28:45 +0200 (CEST) Received: by mail-il1-f182.google.com with SMTP id z2so2964013iln.0 for ; Thu, 26 Aug 2021 04:28:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=8DQi67mRu9IdhAOd5Bogf6TmUOOBttLO3jsRmM/o0gQ=; b=ru3jf0lH/aJIU+gmLpPGMtQhsDZW0qsAAoJ0Av8j2C+SE2XnXfkEDNsGrm0QJycACw WyKD50FrTBGqH6LOj09dGJGppZJ5TC5WNTfYSD8FnqiQ9qCKSBhNDQLHIJBYmL/HXXwC faLgOutKQR5UiK4iOfjxIAHjPAsqWDaJ2Gt9pOEY4xGAFn0nDspEgfpbi/8qbN4c65cl PoTg9n4YPffW7GSkomorm2vwaJha5PWknlwTKYMuWtQm/yN6XESrOrApRNFabUKJ5JDm y0r9TtIWijGBTQ6JOIVnetx4VMhhWbHTeXyrQmll5EZuCytQBzRvhZpSxLUoGBjZrMvd COlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=8DQi67mRu9IdhAOd5Bogf6TmUOOBttLO3jsRmM/o0gQ=; b=PGkDAX9aLdirBfo701Ct2E3RM0uFRGhOfk2Wd/mi8F7wxOqGcBCqwiVBNk0fCYSEDB 3VtDrfLY3lSacqbOQcNgh+9c6t/nyAobzJqqba9RG4A9UTKmYkHr5NvfTkf+RDE40LTj TnNj/bWnNOT9KXzCBQNLpsES8vN6m6UKFGCPUVH26PcD0O8/N4OV6jDj7YZ4F4A1QusG 3qSkC0Gu3U1GjKC4c6W/JsenYMvdYJ1l4j29DuBR7elbxCGgSS3mF9s0nntCBm01lggJ KDhHgJZmSgHmYXCB9uKMJ3lg5znOXN2Qo6DggCF4myEt29krL82McBpfnHebnzXnZxSS N4jA== X-Gm-Message-State: AOAM530+o2h49ToPqXzpqbUCBljeRN68UcPivvLY28Fn0ptbi+gZ/x5N HutMNGaSUjp22XwYPUj0Hu+JY1gvAdmGdVsJZ4k= X-Google-Smtp-Source: ABdhPJyN8OOQCSVK/QpxMMqKWpYbx9Au2rXiMkeWfWgeDneeZABqPH9goeETjBli3wGPBJNL+tlP5gLOxUIEQWa5j78= X-Received: by 2002:a92:d70f:: with SMTP id m15mr2328944iln.162.1629977324458; Thu, 26 Aug 2021 04:28:44 -0700 (PDT) MIME-Version: 1.0 References: <20210727034204.20649-1-xuemingl@nvidia.com> <20210811140418.393264-1-xuemingl@nvidia.com> <20210811140418.393264-6-xuemingl@nvidia.com> In-Reply-To: From: Jerin Jacob Date: Thu, 26 Aug 2021 16:58:18 +0530 Message-ID: To: "Xueming(Steven) Li" Cc: Jack Min , dpdk-dev , Xiaoyun Li Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH v2 06/15] app/testpmd: add common fwd wrapper function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Aug 18, 2021 at 7:38 PM Xueming(Steven) Li wrote: > > > > > -----Original Message----- > > From: Jerin Jacob > > Sent: Wednesday, August 18, 2021 7:48 PM > > To: Xueming(Steven) Li > > Cc: Jack Min ; dpdk-dev ; Xiaoyun Li > > Subject: Re: [dpdk-dev] [PATCH v2 06/15] app/testpmd: add common fwd wrapper function > > > > On Wed, Aug 18, 2021 at 4:57 PM Xueming(Steven) Li wrote: > > > > > > > > > > > > > -----Original Message----- > > > > From: Jerin Jacob > > > > Sent: Tuesday, August 17, 2021 5:37 PM > > > > To: Xueming(Steven) Li > > > > Cc: Jack Min ; dpdk-dev ; Xiaoyun > > > > Li > > > > Subject: Re: [dpdk-dev] [PATCH v2 06/15] app/testpmd: add common fwd > > > > wrapper function > > > > > > > > On Wed, Aug 11, 2021 at 7:35 PM Xueming Li wrote: > > > > > > > > > > From: Xiaoyu Min > > > > > > > > > > Added an inline common wrapper function for all fwd engines which > > > > > do the following in common: > > > > > > > > > > 1. get_start_cycles > > > > > 2. rte_eth_rx_burst(...,nb_pkt_per_burst) > > > > > 3. if rxq_share do forward_shared_rxq(), otherwise do fwd directly 4. > > > > > get_end_cycle > > > > > > > > > > Signed-off-by: Xiaoyu Min > > > > > --- > > > > > app/test-pmd/testpmd.h | 24 ++++++++++++++++++++++++ > > > > > 1 file changed, 24 insertions(+) > > > > > > > > > > diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index > > > > > 13141dfed9..b685ac48d6 100644 > > > > > --- a/app/test-pmd/testpmd.h > > > > > +++ b/app/test-pmd/testpmd.h > > > > > @@ -1022,6 +1022,30 @@ void add_tx_dynf_callback(portid_t portid); > > > > > void remove_tx_dynf_callback(portid_t portid); int > > > > > update_jumbo_frame_offload(portid_t portid); > > > > > > > > > > +static inline void > > > > > +do_burst_fwd(struct fwd_stream *fs, packet_fwd_cb fwd) { > > > > > + struct rte_mbuf *pkts_burst[MAX_PKT_BURST]; > > > > > + uint16_t nb_rx; > > > > > + uint64_t start_tsc = 0; > > > > > + > > > > > + get_start_cycles(&start_tsc); > > > > > + > > > > > + /* > > > > > + * Receive a burst of packets and forward them. > > > > > + */ > > > > > + nb_rx = rte_eth_rx_burst(fs->rx_port, fs->rx_queue, > > > > > + pkts_burst, nb_pkt_per_burst); > > > > > + inc_rx_burst_stats(fs, nb_rx); > > > > > + if (unlikely(nb_rx == 0)) > > > > > + return; > > > > > + if (unlikely(rxq_share > 0)) > > > > > > > > See below. It reads a global memory. > > > > > > > > > + forward_shared_rxq(fs, nb_rx, pkts_burst, fwd); > > > > > + else > > > > > + (*fwd)(fs, nb_rx, pkts_burst); > > > > > > > > New function pointer in fastpath. > > > > > > > > IMO, We should not create performance regression for the existing forward engine. > > > > Can we have a new forward engine just for shared memory testing? > > > > > > Yes, fully aware of the performance concern, the global could be defined around record_core_cycles to minimize the impacts. > > > Based on test data, the impacts almost invisible in legacy mode. > > > > Are you saying there is zero % regression? If not, could you share the data? > > Almost zero, here is a quick single core result of rxonly: > 32.2Mpps, 58.9cycles/packet > Revert the patch to rxonly.c: > 32.1Mpps 59.9cycles/packet > The result doesn't make sense and I realized that I used batch mbuf free, apply it now: > 32.2Mpps, 58.9cycles/packet > There were small digit jumps between testpmd restart, I picked the best one. > The result is almost same, seems the cost of each packet is small enough. > BTW, I'm testing with default burst size and queue depth. I tested this on octeontx2 with iofwd with single core with 100Gbps Without this patch - 73.5mpps With this patch - 72.8 mpps We are taking the shared queue runtime option without a separate fwd engine. and to have zero performance impact and no compile time flag Then I think, only way to have a function template . Example change to outline function template principle. static inline __pkt_burst_io_forward(struct fwd_stream *fs, const u64 flag) { Introduce new checks under if (flags & SHARED_QUEUE) } Have two versions of io_fwd_engine.packet_fwd per engine. - first version static pkt_burst_io_forward(struct fwd_stream *fs) { return __pkt_burst_io_forward(fs, 0); } - Second version static pkt_burst_io_forward_shared_queue(struct fwd_stream *fs) { return __pkt_burst_io_forward(fs, SHARED_QUEUE); } Update io_fwd_engine.packet_fwd in slowpath to respective version based on offload. If shared offoad is not selected, pkt_burst_io_forward() will be selected and __pkt_burst_io_forward() will be a compile time version of !SHARED_QUEUE aka same as existing coe. > > > > > > > > > From test perspective, better to have all forward engine to verify > > > shared rxq, test team want to run the regression with less impacts. Hope to have a solution to utilize all forwarding engines > > seamlessly. > > > > Yes. it good goal. testpmd forward performance using as synthetic bench everyone. > > I think, we are aligned to not have any regression for the generic forward engine. > > > > > > > > > > > > > > + get_end_cycles(fs, start_tsc); } > > > > > + > > > > > /* > > > > > * Work-around of a compilation error with ICC on invocations of the > > > > > * rte_be_to_cpu_16() function. > > > > > -- > > > > > 2.25.1 > > > > >