From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8CC4AA034F; Mon, 7 Feb 2022 11:27:44 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1426B410F3; Mon, 7 Feb 2022 11:27:44 +0100 (CET) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 2F750410EA for ; Mon, 7 Feb 2022 11:27:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644229662; x=1675765662; h=message-id:date:to:cc:references:from:subject: in-reply-to:content-transfer-encoding:mime-version; bh=F9ILWcmSXsb/sAVwU5XCdNFrj7Mz6eUyzsW8or60tR4=; b=gZGvjRXdHpGU076XefcVknkOmJHujY90pQ0n6RhWU6YIS8eSZHxNMNTR d9S1FlQ1SnpRcyQRyXQo5kCuyu+CVx3X5l7xYctJyaPunsdBXy4xKHUzo LU/J122FzIZa1n66Z20mPJySqm3hvhXpvFtqOWh9LmUXru4brLNeQDRDO EXDCo7D0gelu9AwlKnPjF8yWggEpUA6Sp9ddXMu90mRRg0ZDJWouC8Cyf /77n22NcSYIscTEo/cZXUaiOq2kd+rbL/ExxXUUC7w3HJ6JHwIiKtBs/4 dzO57lqYNALLQqRC56h8PFX1WrZGRj4JYVFsVMjWRPqauvBkpZw76Lji3 g==; X-IronPort-AV: E=McAfee;i="6200,9189,10250"; a="248452203" X-IronPort-AV: E=Sophos;i="5.88,349,1635231600"; d="scan'208";a="248452203" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Feb 2022 02:27:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,349,1635231600"; d="scan'208";a="567430250" Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16]) by orsmga001.jf.intel.com with ESMTP; 07 Feb 2022 02:27:41 -0800 Received: from orsmsx608.amr.corp.intel.com (10.22.229.21) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Mon, 7 Feb 2022 02:27:40 -0800 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx608.amr.corp.intel.com (10.22.229.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20 via Frontend Transport; Mon, 7 Feb 2022 02:27:40 -0800 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (104.47.74.44) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.20; Mon, 7 Feb 2022 02:27:40 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ASYb7O/ptgM/BBi/vhkAuGyQ4sFEmjEjrgjJcTWHNSs30Kdk9GZvgPmxIlVTluSnnBFpyUdiUBXJmyw1Ojq6ImwbBCYk/BjJw1RzIwQN2DwHlh/BXdDNoi+cHv8nuP6se5nzgs+PkwRhdCR3GBJhvPSvpJy/+A6Pi5HVnh3jhsl/0TrkjnqceJOWqEHk0b5c/ojClZHYcjl2EI6DwJCcz1/RDW5kMuOhI3JP2kVEHIZPoKecGEpn80N3WxgmHYvg5ht3QMiEGqjNWWyL2pTVKifUVMgzO1mhXYHWy5R7jzvTvA2pwDwLjadOAyzmb5isSDkiwMlBfXMXUnyE1IUOxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=f06l4DCrMn4xWH+QFHi/VJB8CzvUMApQ9iTxezk7MTA=; b=Xaewhy5dLpwz0dFSCOucHADYmE4y4YgWgp5Cmf4UhLn9ZMk97/KxFlmLb+0VOU8wyAt6cV7fDdf6o3EydGzAps5LRg8FJj9IQ5teUzm8fL8ZIw4zmHsd0y96p8/AjphU0tOMD5LsL/D/wXxxSYvr0wzpkme8goT1d4wPvsyGqjvCqEoSEoPl3DGR1PKtmRAgtuS8GJ7Uu3Pf/OSg0iSqyHmiI+pYxKHtqik9xXnjdGfZY5t1eg4bcGktgDJuDShxm7YlCoW5gE+xnsvyGzcFfPgZqRcEFnj9XH+i1jiRDqx3XD9QZKyPgDMNpl9GY+VZnUeGMIU8Yb6OZ33m9eDW3Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from PH0PR11MB5000.namprd11.prod.outlook.com (2603:10b6:510:41::19) by BN6PR1101MB2227.namprd11.prod.outlook.com (2603:10b6:405:51::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.12; Mon, 7 Feb 2022 10:27:39 +0000 Received: from PH0PR11MB5000.namprd11.prod.outlook.com ([fe80::5046:8550:928d:850e]) by PH0PR11MB5000.namprd11.prod.outlook.com ([fe80::5046:8550:928d:850e%7]) with mapi id 15.20.4951.019; Mon, 7 Feb 2022 10:27:39 +0000 Message-ID: <0fa260e7-d5c5-7b92-abd1-b125eb62c6f9@intel.com> Date: Mon, 7 Feb 2022 10:27:33 +0000 Content-Language: en-US To: "Loftus, Ciara" , "dev@dpdk.org" CC: "stephen@networkplumber.org" , Anatoly Burakov References: <20220112075406.54121-1-ciara.loftus@intel.com> <20220204125436.30397-1-ciara.loftus@intel.com> <927b094f-2d84-1dbb-0ad5-37dcf1e1c98e@intel.com> From: Ferruh Yigit Subject: Re: [PATCH v2] net/af_xdp: re-enable secondary process support X-User: ferruhy In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO4P265CA0050.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2ac::16) To PH0PR11MB5000.namprd11.prod.outlook.com (2603:10b6:510:41::19) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a6b9f45c-9822-44e5-d5fb-08d9ea247708 X-MS-TrafficTypeDiagnostic: BN6PR1101MB2227:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3173; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Haa8DEJjQNINmpK3Yg+icOyHGxwKtIoHUZ/5MCHyb2nB+smIXYQrhTiyMpAkriNbKryWlJVGro1TM7oRSTtG9VNWloCsDwOuzZYNi5AEF0iKC52Hs9zWMGJWB1qnfmWV6b+DUWkjhtaUfPsVJNxXjJyTeh7UwPSyPs34joB7rCu6rUxO5OMmpZmClz54fTUSHICVt6qsnhsw3NCOkVmE40ZyNQFNBEgVrx/zejKLgmjNvmrdLZnT+ZmUbjM9Z+N7Nap9brfTu7ht9TV03/k7v2NLA0XUi4K8NMQrMVMqIwW2Y4dAPyq1jjlxU3C5bhYR/qyMHVIh8j/hUGFz2WyKd9zZbqNkkG5uRk4fvBF0d9v22Y68ny+LyYMcO4RWptldfEEn2Fw3RxVlf5MNE4MqAZPR6A1EksLzgYH469VqIDDEYQH20dcSiMylvWEsm+g9DZFBzYdFGajqsMg9Us88J3FrR9p1pKMBwCyxd7mAAPFeUIG1WiGpnZ+3wSna7TdhjFyja4MF45SuTm/UQ+J3GdAetOAk5RW2QJPK3zvziHdaveBIZraeOfxtPfBHSOZHk81v/z/bFi/jACpznsOXTuCLNBc0YsS9urWDfC8NuvpmtBjJVjvJFoZzP7ZYgXr6PF1P9nWTQQ+mVUtOTK+rVGKSFTu0uiev+zstzQ2xWdQQ+bU9LvyhOSgZ1G7n/2qYK1FLiICbpNDJAeEVN8bQ0A== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR11MB5000.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(82960400001)(86362001)(2906002)(36756003)(31686004)(5660300002)(38100700002)(44832011)(53546011)(55236004)(6666004)(8676002)(8936002)(66476007)(4326008)(83380400001)(66556008)(66946007)(110136005)(6512007)(31696002)(2616005)(54906003)(316002)(508600001)(6486002)(26005)(186003)(107886003)(6506007)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Vm5qQlNPOUhsUStoL25UYVBwNkNadnM5ZXlJV0RRK3l0WmxmcGo1cWFzUDY0?= =?utf-8?B?bzFsQkxRUTVWSFhHK1QzS0hvM29MZThGS0orMW1WbDY3bUZzZmRmbVpZc0hW?= =?utf-8?B?bXhqbEgwQ1BxRVdLWkhhMEZuakpzMEhJdm9jMm5ab1ZqV0F3UnBrQWZheFo2?= =?utf-8?B?TUNJWWlLS294QTdQeU5ScENXdUFNVndqWnU4TWp6TzNsbWNNSGNKLzdQRzZt?= =?utf-8?B?TjFMQTRxNjVLekplVEpaS2ptelFKM0FEd0ZuUWxiM0ZvVFAwdG5uaTU2S0NR?= =?utf-8?B?ZnhabWFYQWV3ek8wRkZtRms1WXRuUkVESnJLME9UdUtBalFEV2lUYW9UOVdU?= =?utf-8?B?NGRzMWFMSGVDcUZXYytxUDZzdzRFdWZWS1dYNHZaY00yb0k3aGZlUWN5OEt2?= =?utf-8?B?Mlp3cTZYVHltZmp6VkVxc0JLUERncmNsQ01JVnFia3U4Vll2MDZDREJlZUdQ?= =?utf-8?B?UktSSVlmL3RVSE9mWlg5OFE4UU1YZFc2SlNzY05NSHJuVkxwRmRvRVVDelRm?= =?utf-8?B?ZnN3VnVqTUtnbG5jRDVMM2l4M0ZRUUlaREhFdGRPVVBjcUZkWU11UG5RUnRN?= =?utf-8?B?dlhNY3g0Uk9wMlcwYm5DQi9WS3ZRN1Njd3FTNlVyVVdvd2lMbUpnd3hoSkFv?= =?utf-8?B?d3R0d2NJRnhES3ZBc2ZCb1BSR05hOThCOG5HdlVyOWhZK0R1VGsyZi9mbXA1?= =?utf-8?B?cGl4VXZ1VmsyZ1oxNnUrTzNKWUtPZGM1cThFbDVPTkl6eXlMYzRxd0hJSzUx?= =?utf-8?B?K2cwaVRCNXRsb1RhTXUxMlRvWWs5NkhyWDNPWkkrUVkxd3YyOTYrRHo2a3pn?= =?utf-8?B?RjFsMXVhV1BVUElwVldwR2xzYzJsTUNraFZOWXR6RmhNTU8waVY1ODllMEZE?= =?utf-8?B?N1lrTnZEenBuQ1RueUhzSXd4VitPNG9FZDhwZVBHSEw1NzgzQlJIVmlKendH?= =?utf-8?B?cGFRdXJaKzZVT1pUSUpPV0ZkWWpPZmM5OGJlVnI2QS9iRzlvRW9rWmlXc1Rq?= =?utf-8?B?Tlp0OEVUSi9BNDZjbXZ1VlZRN0o0SUJaMWIxNk12WTkvZDN4OG0xbm1VTTRZ?= =?utf-8?B?MGRPNjE0Uk5tKzRxUFVMYURZM2REVHBFR3I0ZG9lN3V5VFk4VE9HbXhlK0Nr?= =?utf-8?B?UWtjc0M2NmQvWWI0enNVaWUrQVhhbFpvWlRza256TzZqc0dDaW8za05jQ3NI?= =?utf-8?B?N3paZytuSUphSFlRblByMFJJNENIUmxZR1VLSkV3SXBVL0dEVlg3aFcyMTd5?= =?utf-8?B?cTJCbGZ3Q2JPN2pkVW85eGhwTVNpVXJzWHJmZWRZbjNBUVYxOWkzcmMyTW5z?= =?utf-8?B?djFQcXRWemxIOFVzZlJhREEwTEtyNm10c1d3dUdCQ09EWVdhUlhMZGcxcFg4?= =?utf-8?B?THZOdk5jWkhKOS9hNmIzd0pQRFVQUU5HRVhaZXNKbElWaDVUWGhLTVVRWW5q?= =?utf-8?B?MFlKYXcyYmRPZ09MclZtS1grOERVb2FmQnJ0SFYvdGFQWGJJSytoaW1ySjh3?= =?utf-8?B?eWFPakdpVWdJVlhKQ2hLakdqWnhLL2QyakNaMURtNk05K3BaY2IzVE9SZ0RL?= =?utf-8?B?djI5Y05oM2lwWlNZdGc5bVN2RG0zUDBIeG9ZTmg5U3hsYTgzOHUxekpBTzNm?= =?utf-8?B?d2RlcW5mOGd3OHNPTGhJa0d4bkEvVDliOEswdVZkVVh6WHNOeHZxWGtNSWVW?= =?utf-8?B?dklKbmQxYy9Kc1QrdGdoaEUyMUhFSEdYcmVCUVFtWEdvbjYvaVlaV0wzM2VR?= =?utf-8?B?YmlxSm51RThEYnhUaTBCY1N4YXI2aFZmNkFQM3orSTd2MFB2WEZUbFQrYks5?= =?utf-8?B?a0EvTmxMa1FEQVpkUlRYKzdEVER3OUFLQ2RHYkt1dEk1Y0R1d2piZ1pCU3Qr?= =?utf-8?B?dzRQTG1MSEdYU2k1b2ZERlhpTFkxcHRxNHZQT3htakdIYzQ1MXg3WWRZSVlm?= =?utf-8?B?dGE4Z3owek5lUGRnUVRSTnFsYVFUanZtRC8xc0RBOURpUHk0eUhQcjhDdlBG?= =?utf-8?B?WXA4TFZQMnI4bzEvVTE2bzNnaWhiYXlQd3RTZnI5dXIwYkpySDN1bXppYjYx?= =?utf-8?B?VDFoTUYzZzJKVEJXMUJmNVhBbThlMnYvRzhiNVZjOWh4ZHBySTRVSUJGTVVr?= =?utf-8?B?a21mMUhXVTZTOG5TSmRHb0lqdzFIOTVibW11L240TkI3TjU0US9MZVF1OUV3?= =?utf-8?Q?bG+2Gy4z6C2JdN43cMHSZ9c=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: a6b9f45c-9822-44e5-d5fb-08d9ea247708 X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB5000.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Feb 2022 10:27:38.9236 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: WkzKAOhE4lNI3MiDfGDWCYjgT10DQpywHQYXoEn4SoXN6Dm7DVM52XSwwdHqfIfY9bsMm+uJXUTvh4LMgdWkaQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1101MB2227 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2/7/2022 7:49 AM, Loftus, Ciara wrote: >> >> On 2/4/2022 12:54 PM, Ciara Loftus wrote: >>> Secondary process support had been disabled for the AF_XDP PMD >>> because there was no logic in place to share the AF_XDP socket >>> file descriptors between the processes. This commit introduces >>> this logic using the IPC APIs. >>> >>> Since AF_XDP rings are single-producer single-consumer, rx/tx >>> in the secondary process is disabled. However other operations >>> including retrieval of stats are permitted. >>> >>> Signed-off-by: Ciara Loftus >>> >>> --- >>> v1 -> v2: >>> * Rebase to next-net >>> >>> RFC -> v1: >>> * Added newline to af_xdp.rst >>> * Fixed spelling errors >>> * Fixed potential NULL dereference in init_internals >>> * Fixed potential free of address-of expression in afxdp_mp_request_fds >>> --- >>> doc/guides/nics/af_xdp.rst | 9 ++ >>> doc/guides/nics/features/af_xdp.ini | 1 + >>> doc/guides/rel_notes/release_22_03.rst | 1 + >>> drivers/net/af_xdp/rte_eth_af_xdp.c | 210 >> +++++++++++++++++++++++-- >>> 4 files changed, 207 insertions(+), 14 deletions(-) >>> >>> diff --git a/doc/guides/nics/af_xdp.rst b/doc/guides/nics/af_xdp.rst >>> index db02ea1984..eb4eab28a8 100644 >>> --- a/doc/guides/nics/af_xdp.rst >>> +++ b/doc/guides/nics/af_xdp.rst >>> @@ -141,4 +141,13 @@ Limitations >>> NAPI context from a watchdog timer instead of from softirqs. More >> information >>> on this feature can be found at [1]. >>> >>> +- **Secondary Processes** >>> + >>> + Rx and Tx are not supported for secondary processes due to the single- >> producer >>> + single-consumer nature of the AF_XDP rings. However other operations >> including >>> + statistics retrieval are permitted. >> >> Hi Ciara, >> >> Isn't this limitation same for all PMDs, like not both primary & secondary can >> Rx/Tx >> from same queue at the same time. >> But primary can initiallize the PMD and secondary can do the datapath, >> or isn't af_xdp supports multiple queue, if so some queues can be used by >> primary and some by secondary for datapath. >> >> Is there anyhing special for af_xdp that prevents it? > > Hi Ferruh, > > Thanks for the review. > Each queue of the PMD corresponds to a new AF_XDP socket. > Each socket has an RX and TX ring that is mmapped from the kernel to userspace and this mapping is only valid for the primary process. > I did not figure out a way to share that mapping with the secondary process successfully. Can you think of anything that might work? > Does the application knows the buffer address for the Rx/Tx, or is abstracted to the 'fd'? If only 'fd' is used, this patch already converts 'fd' between processes. cc'ed Anatoly, but what I understand is after MP fd conversion: Primary process: FD=x Secondary process: FD=y And both x & y points to exact same socket in the kernel side. At least this is how it works for the 'tap' interface, and that is why 'fs' are in the process_private area and converted between primary and secondary, I thought it will be same for the xdp socket. Did you test the secondary Rx/Tx in the secondary after this patch?