From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 66D93A0C4B; Tue, 27 Jul 2021 05:42:35 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 48D0E410EE; Tue, 27 Jul 2021 05:42:35 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2053.outbound.protection.outlook.com [40.107.237.53]) by mails.dpdk.org (Postfix) with ESMTP id 36217410EC for ; Tue, 27 Jul 2021 05:42:34 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IPSj1PT4bE++42sCmIT74uVVFig4VlyjOKrN76XG6D3M0SbgR0cJJI1yajqgdwS7/xnSc1hbCimwUsb4kJz8xzQ/6V41p4F1I1XkhjoFOVAP6IQFvKBJV0NTdRzxSAuLesJox2csY+6MIwlLf4SNAtV0idAPXimg9WiuVS6FTrwigxGx1tHY2d6TYgm9NEFWWBgdOWmNZx0sK5tPJe9VGUyNuliYUSyNEosQDY8gzkrYn2eJ8WkL03Bz6yYw58ceRZGOGFRSJuT2hudC1i/ZTqgvwvYi3eqNFwXibsobn38db+pH4h7+SLIq7Vslv/SAQ+beFEg6OkWCBI8h06inxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=f5DFAENt3olMXrJtpVDXyU5MvZ4tXG6Fazpo0wXWx7g=; b=UTVKb9YbCJ6Eks8VNHGfapvs56ooF6m3CzhaptrF45ldN/AG7uj8MTpJEYZ8kF3ozapPPHIUeIR6thqRAXmj5I6AVYBS6C2zkP7wzFaSJddMbiMAwhFoJfAGR7ttWPHnoRCYPxoFHwA2JGWd3lcExzRLnIP/spbNC1uZhisruwS5iyLyotUEwpjg7buZ2yZaBB3BPv6Nw6EIus8ekUamX3FCzkoZl6nyPO5tKYH0aBIZomVotIdbI3fBnTc/GjKurQQwtc6DPUiSqgumoxqfEgO4cc/aMpWpkQT9oux/0AjY0VO8li0eP4x26ywmtVGEWEOBmCz045959E9h/bdFlA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=f5DFAENt3olMXrJtpVDXyU5MvZ4tXG6Fazpo0wXWx7g=; b=HV9IhRSHrZf22JxkPSB3WYWGeBbYF0Sh9XLMd2IqNGyZ3tbLIIRxJdtAjHkHanZLcGdTzrcHSTCT3WMmTCfRnawD68sQ9AEIPL1m7WMRx9PFmsSgc5UncuHnSLo7CpK96IsYcNtBIVkyfIhx3uZKz5Q0V5pOgy1YvtjEZXMl1rs9ZbSDoHbgo4ZUXgdnhxx3gB3B37jge0koD4wgW12GOhHeiCtYZxoyXzMRUQPHxfz82Sv0XPSeeHsORa6IM9PBxLsGJCWPvOYsxcCmOBI8UxGPISt2N9VC23ZSkjp4OEX3GyWyEJ9zYYDWN6CzgU9RiO/ipE90LmZ+lZbOWlFlgA== Received: from CO2PR04CA0145.namprd04.prod.outlook.com (2603:10b6:104::23) by BL0PR12MB4753.namprd12.prod.outlook.com (2603:10b6:208:89::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4352.29; Tue, 27 Jul 2021 03:42:33 +0000 Received: from CO1NAM11FT020.eop-nam11.prod.protection.outlook.com (2603:10b6:104:0:cafe::6a) by CO2PR04CA0145.outlook.office365.com (2603:10b6:104::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4352.26 via Frontend Transport; Tue, 27 Jul 2021 03:42:32 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT020.mail.protection.outlook.com (10.13.174.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4352.24 via Frontend Transport; Tue, 27 Jul 2021 03:42:32 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 27 Jul 2021 03:42:30 +0000 From: Xueming Li To: CC: , Viacheslav Ovsiienko , , Thomas Monjalon , Ferruh Yigit , Andrew Rybchenko Date: Tue, 27 Jul 2021 11:42:04 +0800 Message-ID: <20210727034204.20649-1-xuemingl@nvidia.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e9a8f810-eded-47bc-7ef0-08d950b09111 X-MS-TrafficTypeDiagnostic: BL0PR12MB4753: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: c88MU1SIcuScfvF7370vxTU6wsSZe/X7FXmJpsyH8WDWTD88L4CG7rRxeziG7ln5mvAUsyvu60uDADOONAPhh7RqpRoGyNnfxiNucqORoxzvJ5ILbxtvNIwOGcrNbmcFhOHoDT07X9At+2iIBVO1iOU+6k0/lCTfVTs5AnwY5udzahryAe082RstIPQwrgw9GcDBGs80jBCBr7VNyoJZomEHwkmPnq/hm03tvWhzHgpgBRmvZmJPxz1+5cW/qyToAcE7bn+PoAdiqY5VGueJ4DxKCeqcqbL/M1PuuEAFphNbh5waGf47jj3aItlYG5jbVxBJVB4ywlWsUn89Ao46z+pvKwgi8LyAiq+TJhNj8vC5rfSYkmrxMhTqWmNrrvol6A4DzaK4oMB5xO2Sm44BcpY5wwHj8z5t8qeZFQBooQIvZWQfBW6Ut8WHYC9Ck9GPAAose2Jd7Ioi1gqpyPV3g871k4mjSMgmAW1SSEe8wo120lVfvZegg3K/Nn5hqqM7SwQCixNCSNqNI8UkwCVyY4ZrptLjlwpOonwkuID9kSW4Wsh7xiE4rz+T139qug57ZT3XBswlifagr8GhER7ZDy1ybE5EwiXuFnU0hUpxtXnMJmpOrk69/Awkldk0bgObsL3eP99eH53NByzdV+fGO+9mB55Z/z0NeYfuhzLxPV1/fxhEPVtvluU0I7qBY/lBLw4hyuy2KYXv2TSk3YW+qyozZXiy4Qb9/yzqYxc5Uec= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(376002)(39860400002)(136003)(346002)(396003)(36840700001)(46966006)(4326008)(8936002)(7696005)(478600001)(70586007)(7636003)(36860700001)(83380400001)(82310400003)(426003)(6666004)(109986005)(1076003)(2906002)(186003)(8676002)(54906003)(336012)(16526019)(70206006)(86362001)(2616005)(82740400003)(5660300002)(36906005)(55016002)(26005)(316002)(36756003)(356005)(6286002)(47076005)(266003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jul 2021 03:42:32.6925 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e9a8f810-eded-47bc-7ef0-08d950b09111 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT020.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB4753 Subject: [dpdk-dev] [RFC] ethdev: introduce shared Rx queue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In eth PMD driver model, each RX queue was pre-loaded with mbufs for saving incoming packets. When number of SF or VF scale out in a switch domain, the memory consumption became significant. Most important, polling all ports leads to high cache miss, high latency and low throughput. To save memory and speed up, this patch introduces shared RX queue. Ports with same configuration in a switch domain could share RX queue set by specifying offloading flag RTE_ETH_RX_OFFLOAD_SHARED_RXQ. Polling a member port in shared RX queue receives packets for all member ports. Source port is identified by mbuf->port. Queue number of ports in shared group should be identical. Queue index is 1:1 mapped in shared group. Shared RX queue is supposed to be polled on same thread. Multiple groups is supported by group ID. Signed-off-by: Xueming Li --- lib/ethdev/rte_ethdev.c | 1 + lib/ethdev/rte_ethdev.h | 7 +++++++ 2 files changed, 8 insertions(+) diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index a1106f5896..632a0e890b 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -127,6 +127,7 @@ static const struct { RTE_RX_OFFLOAD_BIT2STR(OUTER_UDP_CKSUM), RTE_RX_OFFLOAD_BIT2STR(RSS_HASH), RTE_ETH_RX_OFFLOAD_BIT2STR(BUFFER_SPLIT), + RTE_ETH_RX_OFFLOAD_BIT2STR(SHARED_RXQ), }; #undef RTE_RX_OFFLOAD_BIT2STR diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index d2b27c351f..5c63751be0 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -1047,6 +1047,7 @@ struct rte_eth_rxconf { uint8_t rx_drop_en; /**< Drop packets if no descriptors are available. */ uint8_t rx_deferred_start; /**< Do not start queue with rte_eth_dev_start(). */ uint16_t rx_nseg; /**< Number of descriptions in rx_seg array. */ + uint32_t shared_group; /**< Shared port group index in switch domain. */ /** * Per-queue Rx offloads to be set using DEV_RX_OFFLOAD_* flags. * Only offloads set on rx_queue_offload_capa or rx_offload_capa @@ -1373,6 +1374,12 @@ struct rte_eth_conf { #define DEV_RX_OFFLOAD_OUTER_UDP_CKSUM 0x00040000 #define DEV_RX_OFFLOAD_RSS_HASH 0x00080000 #define RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT 0x00100000 +/** + * RXQ is shared within ports in switch domain to save memory and avoid + * polling every port. Any port in group could be used to receive packets. + * Real source port number saved in mbuf->port field. + */ +#define RTE_ETH_RX_OFFLOAD_SHARED_RXQ 0x00200000 #define DEV_RX_OFFLOAD_CHECKSUM (DEV_RX_OFFLOAD_IPV4_CKSUM | \ DEV_RX_OFFLOAD_UDP_CKSUM | \ -- 2.25.1