From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B4FA041C61; Fri, 10 Feb 2023 14:05:30 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 52F9140EE6; Fri, 10 Feb 2023 14:05:30 +0100 (CET) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2040.outbound.protection.outlook.com [40.107.236.40]) by mails.dpdk.org (Postfix) with ESMTP id 94C2640687 for ; Fri, 10 Feb 2023 14:05:28 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hPFJwANumyBAYicCz256+iJme0RTOQBBy9f1ykYsS0UUGhrr183eC4E0Wzlo5ZJwK+H/3JByuVmP+1/GXVHEJ6jpNg8b/K9AV5z1QW6CvxKrudFpdM4XsWsQwrZK0BSZASdF+eHDW+rPBof5+YagMSYXpcW3PlcLsn6lFzRXqBluzZ+/iKcvi4W+E2pUzW2rPn/FGNs6qKnn1JUpgdb5Jg9i51rR4027lPsoaiN80CnLaXaH2gcwR5GJNfbCHsmRIPDJH6nuLwx/g63jFXmcrtENcFmG6IvFKeMTEfiWOCqG+95SBeOMgbNc4zAydfb8P7ZA0e5qr/ILUChgrg/JnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Y1FVCA0rS/eOx9UDS09HiPBHr4qMzEFBvMhwGoPqo4M=; b=mzT5a5YlHIVOoJY7LmldQgJ9Q5Y1/S1GNbQeHm8zJpbSCN4JkqnB1km9d6HT3bRmdcWc5vKE+1NtOp4ImI0JNzcIlYMoRej3TfM/NtwWVoIXSwjhm+viaUoe2hfETIBPPnM9Lk4SIKQ/i/MtDfRp85Avxqw/jhk9N/WPvve3sdbfHNKxYrfKBcP66UgORUJIjf1r6KG04fsRlvXk5JBj/0MY5ar/HtXdyJA1YZN7dygDUpoyAAy8VUwczhXNl+kogaLDz6XB+DJRA7jGaZo/3fDetdHni2DcbsTOVMaV4Fi5Tq3f7J/xqiqLy3zDtr50Ix11XSl/GsB8m44+gvO6Ng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Y1FVCA0rS/eOx9UDS09HiPBHr4qMzEFBvMhwGoPqo4M=; b=Pnh6Y5z6u7X1dk+31XKeU+k5+bu4Bdw955XOqDwMz9Olwl51bDxxxYYl4gsBvhsuu+0TDopnLZWWadCjnGv8sMEEwmH2Fj2RsoLgUzu0XY0MmONpWz0gQeXv5m+S4GRv3OwSbDHZVPxv1dYB6NtZJO7SZKvjhePEaU2pHh+7lhs= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by BL1PR12MB5111.namprd12.prod.outlook.com (2603:10b6:208:31b::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6086.19; Fri, 10 Feb 2023 13:05:26 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::3614:22ed:ed5:5b48]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::3614:22ed:ed5:5b48%8]) with mapi id 15.20.6086.021; Fri, 10 Feb 2023 13:05:26 +0000 Message-ID: <651bec6d-2d33-347e-3938-06c8118c6c0b@amd.com> Date: Fri, 10 Feb 2023 13:04:15 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Content-Language: en-US To: Shibin Koikkara Reeny , dev@dpdk.org, qi.z.zhang@intel.com, anatoly.burakov@intel.com, bruce.richardson@intel.com, john.mcnamara@intel.com Cc: ciara.loftus@intel.com References: <20230202165513.31012-1-shibin.koikkara.reeny@intel.com> <20230209120549.388318-1-shibin.koikkara.reeny@intel.com> From: Ferruh Yigit Subject: Re: [PATCH v4] net/af_xdp: AF_XDP PMD CNI Integration In-Reply-To: <20230209120549.388318-1-shibin.koikkara.reeny@intel.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO4P123CA0601.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:295::21) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|BL1PR12MB5111:EE_ X-MS-Office365-Filtering-Correlation-Id: aa80777e-bb54-4efc-83c6-08db0b677a18 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 00+9aw42v73OsQluW1koYDcQbva5A5GEJOhrBm5RgfvPIpeFZpVVyGzSuAlHZDj1DF6shsyUDTfgNQ7WfoBhJsfE3M1w5E9K3kIVHXD1Yqr7mRjlTXyHevfW1xuhGd/i1JopMw3WZySwhaUNQ/uDib/YcKWHhEtuKaAyeBFjigB5QjXTPysTvDaSjvoOOd+/rEe85EpsLfS8+igL3543DD5tpty57RIaT2sCodu+08eu0bUGnUvDUMekzxpvxGpQu8T8P5ZtQCF409NLTJe0uoX49zghZ15Fr6wgTqGcyDpVPFAlCpEBawXP+bvEsc3z0Ea+xoAZpNv71UVl+EKTYf7dDJeDFjGcjsuwlTYMb5XbxbqvkNbS48T48cYI0eyb9DL9+7+iEsd/GO3ccjIGIaBT7U3JdCdWIodGCo0yfeWFl126q4PpLXzZWqW9GLTB5DdAkB5JDudYJ1Lshyx2HDSygMVw52m+ZufuxR9iuPCKMCpuPwr/SMPtfrjd4IXS3e1oZ74O4YyK8wAM7HpG2ZOj7NBDZ4NW+tMIH24UKQkdpHn8f9ngm0JfW+xqFsmO9N4WKHmjJ7CESDTxWI0jHtb2XKXww+EvR1/pSCaXTE9NcfjsmJvmeyYMQOCkGxLJGWCtWdo+GAF045IkqOsTyNC7v4J/50X/NMqqBkXYhLPY8qRPJouNK+8Hx8Xkr0WHFCN6DDNwtGn+ftDXi+OzAYMl8RIgBg+aG65LTx+qZVDrq/4Wug8wcrXP3fJjdWiheeK70G/tAX5TCcDauajlJA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230025)(4636009)(136003)(366004)(396003)(346002)(376002)(39860400002)(451199018)(86362001)(5660300002)(8936002)(41300700001)(31696002)(44832011)(4326008)(66946007)(8676002)(66556008)(66476007)(2906002)(31686004)(83380400001)(38100700002)(36756003)(2616005)(26005)(6506007)(6666004)(53546011)(966005)(186003)(6512007)(478600001)(316002)(6486002)(43740500002)(45980500001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ckJoK05aaXpJYkE1b00xMHlMVm11N05mMTFwa1huRm1LYmVJWEQvNDI4YnlO?= =?utf-8?B?QW5OYXdTNHNDOFVhaHY5d1psdlNVZVN1T0VMRGVVZ3lTeHJFK0NQQmZoa3JZ?= =?utf-8?B?b0tjZWlyY0k3NW83VTVqVEhzNzBoSHdhcGNjRW5NVnBDUHliQ1R0cVZzcWJv?= =?utf-8?B?dEZNU1ZvOEF3TnNjU1UwQmE4Wmpmb2Fac3dlL3pTb3hPUDFLRTlEamtrOURj?= =?utf-8?B?Rk41Y2NRdkxJZEVNWUh2N2g1SlRYOGJwV2U3T2lpemxtSWtHTnJZZ1ZpL1ov?= =?utf-8?B?Kzg5b0tzY3RnaEZLVC9aeVh4Q3JTOEdESmNWVUZJRk84K2NVNnV1KzA1T3Zz?= =?utf-8?B?MnJtVFF0QzhrL0lvam44SWg2NUVWTHdMMjZiUkdWVU5wNVVhMGJBQU1sUEJI?= =?utf-8?B?QUlvK3NjWXpScEdRL3hYci9HUUVBNk1VWHI3eFRaWE03cmxBQ050dlNtbXBK?= =?utf-8?B?UU9zTFN4bHA1bGw4ZjlmOVYySWFMYTdvK25Oa3ZmWU1MWFF1ZHlXOEErYkRH?= =?utf-8?B?S3ZhOWJ5Wlg1Z0lPbDFVaDRvcElJWExOSnpnRkpiN1ZNUDQ3SVE5MjJHRyt4?= =?utf-8?B?UjAxMFoxZXVQdTFSWXIrUkkyR2VrRHdMb2EzVnFVZzg1WmwrYzJ0YURyOFdR?= =?utf-8?B?bnBtUDEyS1NDaTV2dE56S0htdWIwWFlaRDNPdVg1SHJUd2VJSU0zSVI2SWxY?= =?utf-8?B?aW9qWWsrVFozVU1MVmpEM2toNGNTdFprOUVrcVhRSVBibWJ5ZmRxak91TTZ1?= =?utf-8?B?OWZCQ0tVNjJGbWxHaUNzbmRpR0hhdUFFOXZZWnZyV2EyRmxNZ2t2NzNhVWMx?= =?utf-8?B?TGpVNktVbFBJOFFwUlhNVWwycUtaaWUzajFRNTNaMDNIY3ZCRlVUMVp2N3lQ?= =?utf-8?B?OHJjTnF4WnczSWlzTGZTbUJrZzcvalp2RFlXSloySzVXaThNSVVHdEl6UUsx?= =?utf-8?B?cE9UaGNEekllY05SS21iMXVjSWM4TFc0bmpHaDJKWnZyU3JlNXdvTkZ4bXhU?= =?utf-8?B?a0NZZzNJSUc5LzBaWnlLT3RLUk0yMTl4VWYzd3lad1hSakgyMW9RaUhnQ2V1?= =?utf-8?B?RVdrbDEvWHhZYVJ2dDNGQVkxNkpvd2loY01WWnovV04xdkxqWHI5SHc1V1h2?= =?utf-8?B?dHpxRWRZanlPcHFUNlAwYTZHWkVuZlZEVEpYbTFYZ0ZQdU5zMnAzK0ZBanFU?= =?utf-8?B?RUxSZ3IxYnBQUDAvdGFUY0lOZkxnWGpaNnl5bUh3UGhMMTFoa0doVDRlUEV1?= =?utf-8?B?aGVaN0VqWTRtTERIczJveXVNSyt4VDd1SnRncHh4Q3E1MFBuVlY3Q1lhVFVx?= =?utf-8?B?aGJPaHpNajhsSWRUdERzMityVjJzb21RRmtZeXowSkpQMFFRZlIwRlZFOUsx?= =?utf-8?B?NEo2MDdETDJGZ2lITm1LaWJhMDJhR0pzS2lZaFZ3NFBLaTZJa0REVU5LKzJW?= =?utf-8?B?SW1sZEUvWlVtQ0VlZjc4eUNWV3gzcmZWbnk0bUdDR2NGUi9ORlZOK0NnUC9U?= =?utf-8?B?SE04L3g5R3VudVpHZ0loT0F5MnlzbitybXBhNERrVXZ3SWJFZHNNMXlPQUQ1?= =?utf-8?B?NVJISzdCNjVSOWQ1OG9Lb28xcXVaOHZwenlwa1k3UnYvNnh1NTFZa29Fc1dJ?= =?utf-8?B?U3dYV2hNOGhYeDhoVEYxRzJqcksybnVaMXlMaGJ6QjhWNFhvVFgwR1RIcUdO?= =?utf-8?B?YTU5Y0w3NjYyelQzNFIyOE8rTFpOdDJGVjFKZTNHbTJkTGFQdVRmMTgvbE5i?= =?utf-8?B?d2JKYWtuTW85YkJKRXR3VktlM3BqbEE2Ti85L1BOSUt3dnc5d1pzRk5EWG1R?= =?utf-8?B?YXN6TmdrVUE2NENKcGJqVEdwQlREV3VYV0xmeGlDZlNSMndqb29sVkY4dXNI?= =?utf-8?B?bE1PV3k4TGZXcDRxK2QvbzRENHcvamtUSUd2ZEhqUG5qQXlzSnUvK3Mwdktz?= =?utf-8?B?UkpaRmtENHJWdEl1Kys1OGYzYTFlNjIrenRPMHlLTjVuemNXZXZQczBJRUZ3?= =?utf-8?B?eHpoNVB0SkJxQ1l5MmcrOTVLM1IxeGs0d0pqQ2V5dkdmdUJUWFJpQzNPeitC?= =?utf-8?B?SXhiVTR1aU54K2VzTnQ0NnlHV09MRG5RNW54dXpVVVpBeENmZGVYVVlxRUFs?= =?utf-8?Q?lBGjbaMMx2rmgyWLGRLyHcE7j?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: aa80777e-bb54-4efc-83c6-08db0b677a18 X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Feb 2023 13:05:26.3364 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: M0+Yhg89L7H8zrtW0Imk62THaOCumDpCswdz4LzBuezwDh9a5tNqNe5mAaQEJK82 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5111 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2/9/2023 12:05 PM, Shibin Koikkara Reeny wrote: > Integrate support for the AF_XDP CNI and device plugin [1] so that the > DPDK AF_XDP PMD can work in an unprivileged container environment. > Part of the AF_XDP PMD initialization process involves loading > an eBPF program onto the given netdev. This operation requires > privileges, which prevents the PMD from being able to work in an > unprivileged container (without root access). The plugin CNI handles > the program loading. CNI open Unix Domain Socket (UDS) and waits > listening for a client to make requests over that UDS. The client(DPDK) > connects and a "handshake" occurs, then the File Descriptor which points > to the XSKMAP associated with the loaded eBPF program is handed over > to the client. The client can then proceed with creating an AF_XDP > socket and inserting the socket into the XSKMAP pointed to by the > FD received on the UDS. > > A new vdev arg "use_cni" is created to indicate user wishes to run > the PMD in unprivileged mode and to receive the XSKMAP FD from the CNI. > When this flag is set, the XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD libbpf flag > should be used when creating the socket, which tells libbpf not to load the > default libbpf program on the netdev. We tell libbpf not to do this because > the loading is handled by the CNI in this scenario. > > Patch include howto doc explain how to configure AF_XDP CNI to > working with DPDK. > > [1]: https://github.com/intel/afxdp-plugins-for-kubernetes > > Signed-off-by: Shibin Koikkara Reeny Is Anatoly's tested-by tag still valid with this version? <...> > @@ -1413,7 +1678,23 @@ xsk_configure(struct pmd_internals *internals, struct pkt_rx_queue *rxq, > } > } > > - if (rxq->busy_budget) { > + if (internals->use_cni) { > + int err, fd, map_fd; > + > + /* get socket fd from CNI plugin */ > + map_fd = get_cni_fd(internals->if_name); > + if (map_fd < 0) { > + AF_XDP_LOG(ERR, "Failed to receive CNI plugin fd\n"); > + goto out_xsk; > + } > + /* get socket fd */ > + fd = xsk_socket__fd(rxq->xsk); > + err = bpf_map_update_elem(map_fd, &rxq->xsk_queue_idx, &fd, 0); > + if (err) { > + AF_XDP_LOG(ERR, "Failed to insert unprivileged xsk in map.\n"); > + goto out_xsk; > + } > + } else if (rxq->busy_budget) { 'use_cni' argument is added as if-else, this result 'use_cni' parameter automatically makes 'busy_budget' argument ineffective, is this intentional? If so can you please describe why? And can you please document this in the driver documentation that 'use_cni' and 'busy_budget' paramters are mutually exclusive. May be this condition can be checked and an error message sent in runtime, not sure. Similarly, another parameter check above this (not visible in this patch), xdp_prog (custom_prog_configured) is calling same APIs (bpf_map_update_elem()), if both paramters are provided, 'use_cni' will overwrite previous one, is this intentional? Are 'use_cni' & 'xdp_prog' paramters mutually exclusive? Overall is the combination of 'use_cni' paramter with other parameters tested? > ret = configure_preferred_busy_poll(rxq); > if (ret) { > AF_XDP_LOG(ERR, "Failed configure busy polling.\n"); > @@ -1584,6 +1865,27 @@ static const struct eth_dev_ops ops = { > .get_monitor_addr = eth_get_monitor_addr, > }; > > +/* CNI option works in unprivileged container environment > + * and ethernet device functionality will be reduced. So > + * additional customiszed eth_dev_ops struct is needed > + * for cni. Promiscuous enable and disable functionality > + * is removed. Why promiscuous enable and disable functionality can't be used with 'use_cni'? Can you please document the limitation in the driver document, also if possible briefly mention reason of the limitation?