From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 536ABA00C5; Thu, 25 Aug 2022 14:22:01 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E874E40223; Thu, 25 Aug 2022 14:22:00 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2063.outbound.protection.outlook.com [40.107.223.63]) by mails.dpdk.org (Postfix) with ESMTP id 7525440156; Thu, 25 Aug 2022 14:21:58 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cjBErO8c0rkclP4c5emF6pqOU8jvIL4+UkkgROLlL/OKMr1eeNFHpDrtdzmz6k7JXckDU1H3V5Z3lgXBl0FhR0HdSc2B8gLiTUOkPagMFHAVvyCXZeJ1JSjLvKtwJY2DfcaTmr8Rxe0fj6AfRF/6nsTq8409ePSKTd3fOXfhgkA3Rn9DRsRPelngkWiElEuLyXEWYHbA8WaU1cfjpjPCjFBClQQDahXxT0TQTUpUyhiZQX/4JvfeS6TX+XrKDZ2EElbxxHy28mngaedX8I6CuOA9cZs22wHPhH0TaLZQBsM70gAtxTaMuLSvwS3kDyoOXlzSVmVVPbzXsZJgI/PNsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=y0Q1Kn47+2jTyrkW6J5+8euNmlcIkyy4noqerdw9PEk=; b=l4VC2I0ry/rRAz+HVef7jUZOv8b/zG7mY5J8SvBag8ITAVdkGbCmysi+bVd/BmeLSiQkD2SNmWy1ehqxNilJrgzNw7w2/dAMJXo4r4f1BggSojlax87cCH85u4/B6sdU7eqgET/fMJRrDGWphf0ilqCGpXNU+6bre71KGVOv2KIm67qCmcsl89IfEK+tZxOutPjBJg+VIFpigyyD+OcYRleiKm/OzzgUK7qm8bK4VNDxfyieII5Q7JOcQbXbmY1UNdwNMtgKkZxV8XZzx6+G6p14VoTCvQOgDQ7zR7bp9rPEJ5AssdanxwO9EvsjbGV/+V2vtjOfP4CKl4N4NWEqyw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.80.198) smtp.rcpttodomain=intel.com smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=y0Q1Kn47+2jTyrkW6J5+8euNmlcIkyy4noqerdw9PEk=; b=pon9x2XiDkfEot8GGLu6WSaeF73OgJKkwYjBGVJStr6V5a6NocHpxLAI39At3/mcwvl0RGsoL2WDkeWV3ILKu006Nd7Hv1Jip/T/WIPviSYqAJ6UTUnkxbV/kxHhzZU67WjWPLzRrsz+HiIuka+G6+vUVYw6LOvDOpQC8OggwVU= Received: from BN8PR12CA0029.namprd12.prod.outlook.com (2603:10b6:408:60::42) by CY5PR02MB8871.namprd02.prod.outlook.com (2603:10b6:930:3d::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5566.15; Thu, 25 Aug 2022 12:21:56 +0000 Received: from BN1NAM02FT015.eop-nam02.prod.protection.outlook.com (2603:10b6:408:60:cafe::ac) by BN8PR12CA0029.outlook.office365.com (2603:10b6:408:60::42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5566.15 via Frontend Transport; Thu, 25 Aug 2022 12:21:56 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.80.198) smtp.mailfrom=xilinx.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.80.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.80.198; helo=xir-pvapexch01.xlnx.xilinx.com; pr=C Received: from xir-pvapexch01.xlnx.xilinx.com (149.199.80.198) by BN1NAM02FT015.mail.protection.outlook.com (10.13.2.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5566.15 via Frontend Transport; Thu, 25 Aug 2022 12:21:56 +0000 Received: from xir-pvapexch01.xlnx.xilinx.com (172.21.17.15) by xir-pvapexch01.xlnx.xilinx.com (172.21.17.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.14; Thu, 25 Aug 2022 13:21:55 +0100 Received: from smtp.xilinx.com (172.21.105.198) by xir-pvapexch01.xlnx.xilinx.com (172.21.17.15) with Microsoft SMTP Server id 15.1.2176.14 via Frontend Transport; Thu, 25 Aug 2022 13:21:55 +0100 Envelope-to: yidingx.zhou@intel.com, dev@dpdk.org, qi.z.zhang@intel.com, stable@dpdk.org Received: from [10.71.194.74] (port=57099) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1oRBru-0003Jw-UI; Thu, 25 Aug 2022 13:21:55 +0100 Message-ID: Date: Thu, 25 Aug 2022 13:21:54 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.2.0 Subject: Re: [PATCH] net/pcap: reduce time for stopping device Content-Language: en-US To: "Zhou, YidingX" , "dev@dpdk.org" CC: "Zhang, Qi Z" , "stable@dpdk.org" References: <20220825072041.10768-1-yidingx.zhou@intel.com> From: Ferruh Yigit In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c85c6099-0c45-4191-d46f-08da869466dd X-MS-TrafficTypeDiagnostic: CY5PR02MB8871:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: IOGDK644Z4r+IGPWIEbrC3ZRb9qXO2qEr72z/uT7gxvmsay7+Z05X8f6iHP+Ufu/FgIM550EuayhvW6Dq4l+t1z2nKvuhlvaywBmaRg1Pfhf/TQ6qvFPsuh1YHlgXNm7RXDqgfVe3RyWVvm7GTHo6Lcj6GThKf/vDRSVhm0EDyw5AZSnFXinSIWK1yiz4mncHF8ELW7t7pSFe6s4BfTjRrEDPkpZxE6dE+IUIwEGXXIfZkdUoNvsZSfE9Na1luBvgHScPRqdVDK1H586izHkUTC5xHCKfCa8QtTZJbY6TlsxeLhJLncWDCNpHmY6dWCDsCLhmjtoiAMEmXzi1Z2Kxkl0IJdvOqRRAaTm5iWA9klHkxlrokvKjhuIGOS+t6GSRGvts0Au85bJszWRrxvmtkOLCyksEte0jFAmamF/9gAuDEhOE9e1FXBcqXxXlDrgDnUfh8Q3NctebnPEu9NBuMFyrA8Wy2Vib1m0HYttCyFF63qSCjQzMJN55ZNXoSZRRHkjBlyBtFyrGbiI+E+rKOu0uk4gx91v3y1/eHuYcWBBOwZKNMvqBYL/C7OlM+tPEPtvWjPCpCvtPgcG6UlGV2sPbfop6zUEYOoyQyAfrnvVCG+6wyzWSh9snf6LRadVW/CJD/wwcwClCWYr2tFX/940MJemhVfIwmQBTD4bxevgxNQTcS8MGKVSMet2MmyBnQgRoT1Su2UL9bTwXQR2NEhCo/5sewoy4HeaV68otWrTXoZFaxczPfUFRH7WOoM3YLk4/TkMOb8FCgX4lELGhixkogclKs/pOZ/yzsy2p9FuLDuK96PUtnzb84k3vKkF X-Forefront-Antispam-Report: CIP:149.199.80.198; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:xir-pvapexch01.xlnx.xilinx.com; PTR:unknown-80-198.xilinx.com; CAT:NONE; SFS:(13230016)(4636009)(346002)(396003)(136003)(376002)(39860400002)(36840700001)(40470700004)(46966006)(2906002)(44832011)(5660300002)(8936002)(316002)(40480700001)(9786002)(31686004)(110136005)(478600001)(41300700001)(53546011)(54906003)(36756003)(26005)(186003)(31696002)(2616005)(82310400005)(7636003)(356005)(82740400003)(8676002)(40460700003)(83380400001)(426003)(70206006)(70586007)(47076005)(336012)(4326008)(36860700001)(50156003)(43740500002); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2022 12:21:56.3723 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c85c6099-0c45-4191-d46f-08da869466dd X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.80.198]; Helo=[xir-pvapexch01.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: BN1NAM02FT015.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR02MB8871 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 8/25/2022 12:17 PM, Zhou, YidingX wrote: > > >> -----Original Message----- >> From: Ferruh Yigit >> Sent: Thursday, August 25, 2022 6:09 PM >> To: Zhou, YidingX ; dev@dpdk.org >> Cc: Zhang, Qi Z ; stable@dpdk.org >> Subject: Re: [PATCH] net/pcap: reduce time for stopping device >> >> On 8/25/2022 8:20 AM, Yiding Zhou wrote: >>> The pcap file will be synchronized to the disk when stopping the device. >>> It takes a long time if the file is large that would cause the 'detach >>> sync request' timeout when the device is closed under multi-process >>> scenario. >>> >>> This commit fixes the issue by performing synchronization in Tx path >>> >>> Fixes: 4c173302c307 ("pcap: add new driver") >>> Cc: stable@dpdk.org >>> >>> Signed-off-by: Yiding Zhou >>> --- >>> drivers/net/pcap/pcap_ethdev.c | 18 ++++++++++++++++-- >>> 1 file changed, 16 insertions(+), 2 deletions(-) >>> >>> diff --git a/drivers/net/pcap/pcap_ethdev.c >>> b/drivers/net/pcap/pcap_ethdev.c index ec29fd6bc5..52eafa5674 100644 >>> --- a/drivers/net/pcap/pcap_ethdev.c >>> +++ b/drivers/net/pcap/pcap_ethdev.c >>> @@ -3,7 +3,7 @@ >>> * Copyright(c) 2014 6WIND S.A. >>> * All rights reserved. >>> */ >>> - >>> +#include >>> #include >>> >>> #include >>> @@ -38,6 +38,8 @@ >>> >>> #define RTE_PMD_PCAP_MAX_QUEUES 16 >>> >>> +#define ETH_PCAP_SYNC_THRESHOLD 0x20000000 >>> + I guess this is 512MB, can you please comment this. Is there any specific reason to select this value, or is it arbitrary? >>> static char errbuf[PCAP_ERRBUF_SIZE]; >>> static struct timespec start_time; >>> static uint64_t start_cycles; >>> @@ -47,6 +49,8 @@ static uint8_t iface_idx; >>> static uint64_t timestamp_rx_dynflag; >>> static int timestamp_dynfield_offset = -1; >>> >>> +RTE_DEFINE_PER_LCORE(uint64_t, _pcap_cached_bytes); >>> + >>> struct queue_stat { >>> volatile unsigned long pkts; >>> volatile unsigned long bytes; >>> @@ -144,6 +148,16 @@ static struct rte_eth_link pmd_link = { >>> >>> RTE_LOG_REGISTER_DEFAULT(eth_pcap_logtype, NOTICE); >>> >>> +static inline void >>> +pcap_dumper_data_sync(pcap_dumper_t *dumper, uint32_t bytes) { 'pcap_' is the namespace for the libpcap, can you select another prefix, like 'eth_pcap_' as many driver functions does. >>> + RTE_PER_LCORE(_pcap_cached_bytes) += bytes; >>> + if (unlikely(RTE_PER_LCORE(_pcap_cached_bytes) > >> ETH_PCAP_SYNC_THRESHOLD)) { >>> + if (!fdatasync(fileno(pcap_dump_file(dumper)))) >>> + RTE_PER_LCORE(_pcap_cached_bytes) = 0; >>> + } >>> +} >>> + pcap supports multiple queue, and each queue creates a new pcap dumper and single core/thread can be used for this multiple dumpers. In that case I think above per lcore variable logic doesn't work. And instead of having a global value, what do you think to add a variable to 'struct pcap_tx_queue' for this purpose? >>> static struct queue_missed_stat* >>> queue_missed_stat_update(struct rte_eth_dev *dev, unsigned int qid) >>> { >>> @@ -421,7 +435,7 @@ eth_pcap_tx_dumper(void *queue, struct rte_mbuf >> **bufs, uint16_t nb_pkts) >>> * process stops and to make sure the pcap file is actually written, >>> * we flush the pcap dumper within each burst. >>> */ >>> - pcap_dump_flush(dumper); >>> + pcap_dumper_data_sync(dumper, tx_bytes); >> >> 'pcap_dump_flush()' should be doing the same thing, to write buffer to file, >> isn't it working? >> >> Can you check the return value of the 'pcap_dump_flush()' API, I wonder if it >> keeps failing, for some reason? >> > > 'pcap_dump_flush()' returns 0 each time without error, it calls 'fflush()' to flush userspace buffers to kernel buffers, not disk. 'fdatasync()' to ensure data is written to disk. > 'pcap_dump_flush()' API documentation says "flushes the output buffer to the ``savefile,''", but as you said it uses 'fflush()' internally, so there is a chance that data is not written to the disk. In this case, won't need to keep both, first flush and later fsync/fdatasync? Do you observe any performance difference after change, since now writing to actual disk on datapath? >>> dumper_q->tx_stat.pkts += num_tx; >>> dumper_q->tx_stat.bytes += tx_bytes; >>> dumper_q->tx_stat.err_pkts += nb_pkts - num_tx; >