From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8C7F0A0503; Thu, 19 May 2022 18:38:38 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 73DF540687; Thu, 19 May 2022 18:38:38 +0200 (CEST) Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1anam02on2060.outbound.protection.outlook.com [40.107.96.60]) by mails.dpdk.org (Postfix) with ESMTP id 9ACB540223 for ; Thu, 19 May 2022 18:38:36 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TUHra8jELZle1ypk2hln1AParproWIWOvBHZWZ89ftHS6nBm8+ZUaiYQ/1fWDNl/zQ5Yj43fInpSAzme8b3hmhCdIawVdakrXi7dATj2jRIGhTOvnK7sy37F5kcL2gQU2lGFTVwCr4dovbsi3pIaUtL/dF6ucwNNcHy7Mlt4NTGUIucazxJKkRoiM2WcQtvagbGxNP35CdYDlgUeYwExWJ7Qh6KRRxvzv5bvn4JmpdJbzwMGF6Zf2wY0xsVZMxLkw3F6D687o+enwPXTbDZyLW/Gw/VaLdgJlIjJEdXEzk4VCxbmGrS8J0VsyBeUaLmcKlIPQ+pwdN9SbJUfr06QaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PrnxKv1vi5UoZrDce9lLO36h/YPuDH4fdPAisGfM0So=; b=U9WXafEz2fCm+9st15TW8hR4wzQmUjKyFd4poT0jwebFontexZryj7JGt0wQPY1BXb7JcTDTsm5nXEgOQ9rneKElhFFE2hA0nXSS6owesjK6mMEnTo8ArTFFLmzxTheNtsF8EsqYLDlYT783Jui/9uX2f8BZ8lyKN1MI/g4SkMPGFfy7Zjp6y8hyHJx7Dv5XnBKBKaaqDpQWU+cQO2MhVg/+y0hEUDYNnRZrtGT3EFnom5CbTtjIQJT7cO/t5F7x79doL5h0ijhe0kqfmekMO4ev/w6V6+fhOw6vu4Qm/0xPFFgrWDtE/aXoALjn4FPYQHqy3rLxTlD9LhsujvREKQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=softfail (sender ip is 149.199.80.198) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=fail (p=quarantine sp=quarantine pct=100) action=quarantine header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PrnxKv1vi5UoZrDce9lLO36h/YPuDH4fdPAisGfM0So=; b=OG2zuC2U+rlOZuP7kEmeediymCfQmQ5rHAD9jjRqNq/xpSU0iMjfyD+5oHvEGPicfkUeXA8AAz7zYq9l2Ez4mNHy3nnjoGNISAjdgV3It3Bsrsg/OPYDxs6oV0YjR73DDRv55nlxxugmH4UqquFKB28SekRjnui4v3Wi3qw2RA4= Received: from DS7PR03CA0286.namprd03.prod.outlook.com (2603:10b6:5:3ad::21) by CH2PR02MB6230.namprd02.prod.outlook.com (2603:10b6:610:6::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5250.18; Thu, 19 May 2022 16:38:33 +0000 Received: from DM3NAM02FT014.eop-nam02.prod.protection.outlook.com (2603:10b6:5:3ad:cafe::ee) by DS7PR03CA0286.outlook.office365.com (2603:10b6:5:3ad::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5250.18 via Frontend Transport; Thu, 19 May 2022 16:38:33 +0000 X-MS-Exchange-Authentication-Results: spf=softfail (sender IP is 149.199.80.198) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=fail action=quarantine header.from=amd.com; Received-SPF: SoftFail (protection.outlook.com: domain of transitioning amd.com discourages use of 149.199.80.198 as permitted sender) Received: from xir-pvapexch02.xlnx.xilinx.com (149.199.80.198) by DM3NAM02FT014.mail.protection.outlook.com (10.13.5.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5273.14 via Frontend Transport; Thu, 19 May 2022 16:38:32 +0000 Received: from xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) by xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.14; Thu, 19 May 2022 17:38:31 +0100 Received: from smtp.xilinx.com (172.21.105.197) by xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) with Microsoft SMTP Server id 15.1.2176.14 via Frontend Transport; Thu, 19 May 2022 17:38:31 +0100 Envelope-to: Joyce.Kong@arm.com, jgrajcia@cisco.com, Ruifeng.Wang@arm.com, dev@dpdk.org, nd@arm.com Received: from [172.21.34.28] (port=18164) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1nrjAV-0001Bv-EW; Thu, 19 May 2022 17:38:31 +0100 Message-ID: Date: Thu, 19 May 2022 17:38:31 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.9.0 Subject: Re: [PATCH v1 1/2] net/memif: add a Rx fast path Content-Language: en-US To: Joyce Kong , Jakub Grajciar CC: Ruifeng Wang , "dev@dpdk.org" , nd References: <20220412093243.3670187-1-joyce.kong@arm.com> <20220517105109.1086090-1-joyce.kong@arm.com> <20220517105109.1086090-2-joyce.kong@arm.com> <03c405ad-eb23-5f8a-782c-5f80bc9bf96d@xilinx.com> From: Ferruh Yigit In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 932392ba-069f-414c-85db-08da39b60359 X-MS-TrafficTypeDiagnostic: CH2PR02MB6230:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 2 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 61RqWXcM+shRH6n9YA1ZJAjht/nJZHhXDckc8FEnTP2Kp5pKQjMfSHEcxxEo0BSlJo/iL/Phf7NizslNwokQsuaKUs+NoPPXwH6x/P+4a0t5z+NkbEI5sBxCqy/ZKkwCrvyNcCLxsEOtfmt1BCjownC1GezjPTN3KaxEfhLvyB2xAInT8stJyRyUvti9LkDLkcedPW6TAtSVSL2roLvHVua6xj3M5eEUpndJx14LG3HL8GPAArh+3QxFU20qXWDsSL555/tWDL6/6sPGNEJdmB/bShvsjIh+hfjqzLKuB6lOaumazA4F2Mhb25K80Vn62BmSs2pcIyfGVq41YaJG94R1P5T0jO4BLXSRKbfJ5HpsvmIcHT0fmXTcVo59VvgXre47IhjH42Uq9U2tXNwjZ85VaFAqrLtWBP8B66mLy+5ruWnsBIr0oCr136csoEnj/PbC38FnsaeWRSlS7PuAZWUK2MDG39rg1r4J3R/tgbbLLvv8naBwDlAUpobniAMsRVrYvVytuDRaZYX3pG37ho3qyQH7TuY0U380ZZih3zjhojGfPwXYKVgW/MpTAzVEncMt2VS5BXSL2GwYxzTL70R+UQ5mgsMnLF8mU6hw+B0qrAfUWnk8tjSzboE0mrNoKmbgciKIPcSTlJbj4kvhM92XpXM8bMZqY+tsPGWZnmeZSpa2dR/qS5giX6eBJ3lPzHfMRIs6L5AdTMELDCiLfvuZTnolUFV5NnDd66ZVAQ0= X-Forefront-Antispam-Report: CIP:149.199.80.198; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:xir-pvapexch02.xlnx.xilinx.com; PTR:unknown-80-198.xilinx.com; CAT:NONE; SFS:(13230001)(4636009)(40470700004)(46966006)(70586007)(8676002)(4326008)(70206006)(86362001)(2616005)(83380400001)(8936002)(5660300002)(40460700003)(316002)(110136005)(508600001)(53546011)(2906002)(47076005)(36756003)(31686004)(82310400005)(336012)(44832011)(7636003)(26005)(9786002)(35950700001)(31696002)(54906003)(356005)(50156003)(43740500002); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.onmicrosoft.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 May 2022 16:38:32.7188 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 932392ba-069f-414c-85db-08da39b60359 X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.80.198]; Helo=[xir-pvapexch02.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: DM3NAM02FT014.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR02MB6230 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 5/19/2022 4:09 PM, Joyce Kong wrote: > [CAUTION: External Email] > >> -----Original Message----- >> From: Ferruh Yigit >> Sent: Thursday, May 19, 2022 1:06 AM >> To: Joyce Kong ; Jakub Grajciar >> Cc: Ruifeng Wang ; dev@dpdk.org; nd >> >> Subject: Re: [PATCH v1 1/2] net/memif: add a Rx fast path >> >> On 5/17/2022 11:51 AM, Joyce Kong wrote: >>> For memif non-zero-copy mode, there is a branch to compare >>> the mbuf and memif buffer size during memory copying. Add >>> a fast memory copy path by removing this branch with mbuf >>> and memif buffer size defined at compile time. The removal >>> of the branch leads to considerable performance uplift. >>> >>> When memif <= buffer size, Rx chooses the fast memcpy path, >>> otherwise it would choose the original path. >>> >>> Test with 1p1q on Ampere Altra AArch64 server, >>> -------------------------------------------- >>> buf size | memif <= mbuf | memif > mbuf | >>> -------------------------------------------- >>> non-zc gain | 4.30% | -0.52% | >>> -------------------------------------------- >>> zc gain | 2.46% | 0.70% | >>> -------------------------------------------- >>> >>> Test with 1p1q on Cascade Lake Xeon X86server, >>> ------------------------------------------- >>> buf size | memif <= mbuf | memif > mbuf | >>> ------------------------------------------- >>> non-zc gain | 2.13% | -1.40% | >>> ------------------------------------------- >>> zc gain | 0.18% | 0.48% | >>> ------------------------------------------- >>> >>> Signed-off-by: Joyce Kong >> >> <...> >> >>> + } else { >>> + while (n_slots && n_rx_pkts < nb_pkts) { >>> + mbuf_head = rte_pktmbuf_alloc(mq->mempool); >>> + if (unlikely(mbuf_head == NULL)) >>> + goto no_free_bufs; >>> + mbuf = mbuf_head; >>> + mbuf->port = mq->in_port; >>> + >>> +next_slot2: >>> + s0 = cur_slot & mask; >>> + d0 = &ring->desc[s0]; >>> >>> - rte_memcpy(rte_pktmbuf_mtod_offset(mbuf, void *, >>> - dst_off), >>> - (uint8_t *)memif_get_buffer(proc_private, d0) >> + >>> - src_off, cp_len); >>> + src_len = d0->length; >>> + dst_off = 0; >>> + src_off = 0; >> >> Hi Joyce, Jakub, >> >> Something doesn't look right in the original code (not in this patch), >> can you please help me check if I am missing something? >> >> For the memif buffer segmented case, first buffer will be copied to >> mbuf, 'dst_off' increased and jump back to process next memif segment: >> >> + d0 >> | >> v >> +++ +-+ >> |a+->+b| >> +-+ +-+ >> >> +---+ >> |a | >> +-+-+ >> ^ >> | >> + dst_off >> >> " >> if (d0->flags & MEMIF_DESC_FLAG_NEXT) >> goto next_slot; >> " >> >> But here 'dst_off' set back to '0', wont this cause next memif buffer >> segment to write to beginning of mbuf overwriting previous data? >> >> Thanks, >> Ferruh > > Hi Ferruh, > > Agree with you here, and sorry I didn’t notice it before. Perhaps moving > 'det_off = 0' to the line above 'next_slot' would solve the overwriting? > Yes, I think this solves the issue. And I wonder why this is not caught by testing. @Jakub, is the segmented memif buffers not a common use case? I did able to reproduce the issue as following (and confirm suggested change fixes it): server ./build/app/dpdk-testpmd --proc-type=primary --file-prefix=pmd1 --vdev=net_memif0,role=server,bsize=32 -- -i --txpkts=512 > set fwd txonly > start client ./build/app/dpdk-testpmd --proc-type=primary --file-prefix=pmd2 --vdev=net_memif1,bsize=32 -- -i > set fwd rxonly > set verbose 3 > start 'client' will display packets info wrong, it will be all '0'. Also it is possible to capture packets in client and confirm.