From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6A829A054F; Mon, 15 Mar 2021 20:31:29 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DE4792427A8; Mon, 15 Mar 2021 20:31:28 +0100 (CET) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2059.outbound.protection.outlook.com [40.107.243.59]) by mails.dpdk.org (Postfix) with ESMTP id 9E5944069F for ; Mon, 15 Mar 2021 20:31:26 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=l7WlaaMHUs0QvmHbfrOhsy64eO9NU7BISyWN2ogqFTqyRiaysmKIpxOTjp1oPjp/Oo2/kgx2C4WGF/zWEWVs2Y1m1JqDrkjnJ7pWvmMPWpPPKLwxL82FB1+RexbUdHWOHIUvMFB7vBg4kWQZoNGi0rve6y6H9yOUdh8LvTHE2TvwBoRhWa9EROGAfqeak9fx7U0/2VRsxupmqyviwhjkhaHDaGAZgykkJXd/cPO1VEUo3VHu2LhfGvUs0doC5N0pbWcWeoScasULYrSWrtdmupm0cVEViKFYuLMB02trE9zdyc+MxbSM8Srt3SVBUvlABpGGqGVgNiM+SPYULAWRKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8Y4Ca/eKcN/zDEqI66OQt6VrgjSt42uhBNzobA+2E2Y=; b=E6Q7/XlyGbPOsIwUAXt6WsMWg4ETOrnGrYHjwp65A506O9CX7qH98id+ZZirwWWNm5SAkVyuzQoz0B3NwwfabSF8Iy+HRwJhl0gpexo8ii8CXNtohNY+nvhlaUMA79DoZQHVNrjPFTJaj/x/K4IZQf05NfdF6UW5tzcHkAO1mJqtDTZ1apprGKqez6FEzoLqm8hT/cq6p8Hovu0GgiIrdVsIhKxCWwrXYvNtyPA1IJE/sqHJbce0r0nu0arLpTwVxpbLzlpBmu0e3ODKnWY7r8XxMvoKfdxc99Rpxm0TzMkYcrYbvxTp4xG7eag5duo99w9/b/WJZUI46ehslsmhZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8Y4Ca/eKcN/zDEqI66OQt6VrgjSt42uhBNzobA+2E2Y=; b=EXQhu+Wbp+yw3QgMS2zDpS/bxV5zwB4pQNLqXPtvK4nPR763PPe4S+vMU6dnoi5dOjSISxXWzMYgCp2+yv6ibfUadL1/i86VxWudw/OdcWdp82iysUuLDOfbS6JhXeiQ2E3nuGQT+hi15ylrSSCNca4ZBih06mOvaWOnWliODj1PNzW6lCdmsMfMiQCpZ6nCgNkHyEdeFImHAsfflH0+RRdgn07y/4AyHZrztiZmNN/k1yZBHziiLNsbUCDNJshciNWF6b028DMOM1jbjikSD4vRm53L1U+NmuXSj5f63GHyPzaovfRODYNegBjfyxCqZeKB6UfivW0BoSjgeN7XmA== Received: from DM6PR12MB3753.namprd12.prod.outlook.com (2603:10b6:5:1c7::18) by DM5PR12MB2421.namprd12.prod.outlook.com (2603:10b6:4:b4::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3933.32; Mon, 15 Mar 2021 19:31:25 +0000 Received: from DM6PR12MB3753.namprd12.prod.outlook.com ([fe80::20f7:11fc:9d46:9258]) by DM6PR12MB3753.namprd12.prod.outlook.com ([fe80::20f7:11fc:9d46:9258%6]) with mapi id 15.20.3933.032; Mon, 15 Mar 2021 19:31:25 +0000 From: Slava Ovsiienko To: Jan Viktorin CC: "dev@dpdk.org" , Asaf Penso , Shahaf Shuler , Matan Azrad Thread-Topic: [PATCH 2/2] net/mlx5: avoid unbind step to enable switchdev mode Thread-Index: AQHXGbCpbh43bNULVUOsJef9QEMNqKqFMkHggAAGBwCAADYQMA== Date: Mon, 15 Mar 2021 19:31:25 +0000 Message-ID: References: <1615822438-10082-1-git-send-email-iviktorin@fit.vutbr.cz> <1615822438-10082-2-git-send-email-iviktorin@fit.vutbr.cz> <20210315171039.16b3965e@tanguero.localdomain> In-Reply-To: <20210315171039.16b3965e@tanguero.localdomain> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: cesnet.cz; dkim=none (message not signed) header.d=none;cesnet.cz; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [95.164.10.10] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 11ba3890-bef9-43ce-0932-08d8e7e8ec10 x-ms-traffictypediagnostic: DM5PR12MB2421: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 8j2WYB3ThYkYRgrwKK5lPz7YI7vMR7Nil358A1g5AylZrIzisw45FsXwyWj1rBG13B8Uqw8T6EOXiRkRRfCSGikJY0kcxTo91FOmhhCCXVjHYojuneJTiLyQCSpmDZac6d3K65In3WV2MF3HTd/gwjY5AjWxDtrRVq6qnD7kswiUMnYiv50bb2FxZ11catk70SyclH+Mykcbe2lM5L5CuYB8dm8D6IzHb6CpdZjNDBgHaprcsljfDe1zIkwv5AQMhoNCf58hi6UnkPDR72MV0ruk6gbhKm/Mw1fPBio5HkmtlI4purH9+syXNkOACvt6CDl9LOgw4vj1Cfk/H/QfwBvOZfedMctz12sXAeXaWLZQycDl/JTcd83mBYEbqfVK4Pa4WRDuj/tqmbXU1YQ71x+ecylht/ENtldCc+CD4VSwysfo9wO4va4Cr6UJ3K3UMN2+bu7Mt4DROPjSTByzON5xmn+PqyT6Nk/ujsBLlaeuZHMm8FgBjb0P3sWVCHotVlZDBDWj0GU9edKvz+69fN8xtWjshaRsKpIZQNmuC6Tnu8mEsYXYUV/8gm9MP3/b x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR12MB3753.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(39860400002)(376002)(136003)(366004)(396003)(9686003)(53546011)(6506007)(107886003)(186003)(478600001)(86362001)(76116006)(2906002)(33656002)(83380400001)(4326008)(316002)(52536014)(8936002)(66446008)(8676002)(7696005)(26005)(5660300002)(55016002)(66556008)(66946007)(6916009)(64756008)(71200400001)(66476007)(54906003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?MGNVohDmxek5hVqQTNEVLgqTpr511Q2DHMY7L8S+XLbTIbndDjF1NB8hKw+N?= =?us-ascii?Q?TQsS4EIND1BrXWYLSatPv0XHcmaDBprtTlSELuRjgin9L5FCBf/dZWiAkrGC?= =?us-ascii?Q?Jt3LQ4egF6Zd3lo03pAlEFxM6eWcf/wRBvLvtWzvOURBvKLGs3c7NaLRoqKL?= =?us-ascii?Q?q6o6RzKZxrqTO6AYpnUWGWvhTH1yvMHdMAXMlBwAk6pnV7atV58mYg5cCwkt?= =?us-ascii?Q?8oke+6j3jytgTCk948RZ8fkcuD0H2eykfbZ49LiR1JVgVwSDQBmB3c/CG4Iv?= =?us-ascii?Q?SU4QYrLy4IfVQ9hvOChumYR0TqpbVObF14xkJF4d56HOtKPfsI5jSVFpSwZN?= =?us-ascii?Q?FTUY/J/1xYVa9a+ycCDXaqI+pQcbSToL/kdoimQhAxEljINvzxCMBez80+DA?= =?us-ascii?Q?hwclPot/i6E2vx2vHyZtpYAIYR3XcC7WCPE3KBXTF1Vn2r1CWi36khMoWIQL?= =?us-ascii?Q?mIi8ywGeMsQXQFU5TvZBaprmUOLaI9IM5QDiqXKOPkEmmrXdj1mWLVqqscHh?= =?us-ascii?Q?FXO2D/xU3Mn4H7Fp5HyMIJ6HyBPsk/8Iz4pOn3wN8UysM50kE+FfwWPlWDdc?= =?us-ascii?Q?Uh4PnAd+6Iqztyc5wACbsLjh3oKV3jkgOexqG5qC92hTlwCVC7T5GHsARKfq?= =?us-ascii?Q?DBJgzNwrV/IWhKrho6IhAwwCNDIvUDnmu4UQj+3GPCMxgOGp/8am6tpHggl2?= =?us-ascii?Q?u4SsF0z+DYjCv4V51qefsN/hSMas3ltPDHISFd6MhocXKnTXfz8VXHnxPJht?= =?us-ascii?Q?8l0dwP5j0TBdjqPefZHM5mldUpj5lXrOhi6UN12EE2bIj5Q6zWoKF8xVtysy?= =?us-ascii?Q?nJnhNuXUcKHWTIt/uU3GfAdBky+CDulS9p6ptBesIPgjzn+pGd2YH5xK+SLJ?= =?us-ascii?Q?QW/f2Zo5w6Sgume4cV/JwnpsFukDYVw15nEwXlOTLGWedS4BnwEfP/7O/zdT?= =?us-ascii?Q?z0opD76T48V0g5hP+c2muoZXpo3461eeWXJhHd/pLU21CYSTRpkeyCH72XMW?= =?us-ascii?Q?hBb3/NFDQmTYHWV7jBJC2gYMJ1nYAssY6SomFujEmDeUaaySgb6M13fDvzt6?= =?us-ascii?Q?VkE7svbmS/OMQ/055X90WbTnliVBSOaPu/UR+oxDFHZ0U1Bn5ZIgKPD7b0TZ?= =?us-ascii?Q?bheWcVeN80KGgmZJEBMFV7Vx3LSNHCbx7S2MZZrEqNXmg09SQaKgAuRMT5pn?= =?us-ascii?Q?lvTrAJAylS1MJ/664ikolGeZOlCTr7TmQVurSMjrBKXcChmsVzb72VCvmZCz?= =?us-ascii?Q?PzR3K1NrKwrnSLvPNj1s82w5wXsyfGDtH/E0Lcnd4PFghp7M9X4ZzXjV5Gsr?= =?us-ascii?Q?Iv0=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3753.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 11ba3890-bef9-43ce-0932-08d8e7e8ec10 X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Mar 2021 19:31:25.0468 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: piWE1+ec7AiwVyH8PDQTZwjUa3Q6L+FXkTHHPHoQD5GDepAIK0Bcy8F04WZrd/phFMoU7n99ovJS7PGNzXuOYQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB2421 Subject: Re: [dpdk-dev] [PATCH 2/2] net/mlx5: avoid unbind step to enable switchdev mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, Jan 1. Let's suppose we have the one port NIC, PCI BDF 08:00.0, SR-IOV enabled,= 0 VFs is configured, netdev name is ens1f0 (depends on distro and kernel version).=20 2. Then, we configure 3 VFs with SR-IOV, we get : - 08:00.0 - PF, ens1f0 - 08:00.1 - VF0, ens1f1 - 08:00.2 - VF1, ens1f2 - 08:00.3 - VF2, ens1f3 3. Then, we should unbind VFs ONLY, PF should be kept untouched. - 08:00.0 - PF, ens1f0 - 08:00.1 - VF0, - - 08:00.2 - VF1, - - 08:00.3 - VF2, - 4. Then, configure switchdev mode - 08:00.0 - PF, ens1f0 - 08:00.1 - VF0, - - 08:00.2 - VF1, - - 08:00.3 - VF2, - + we get 3 representor netdevs: ens1f0_0 (no dedicated PCI device) ens1f0_1 (no dedicated PCI device) ens1f0_2 (no dedicated PCI device) 5. Then, bind VFs back (assume we are no going to map them to VMs) - 08:00.0 - PF, ens1f0 - 08:00.1 - VF0, ens1f1 - 08:00.2 - VF1, ens1f2 - 08:00.3 - VF2, ens1f3 - ens1f0_0 (no dedicated PCI device) - ens1f0_1 (no dedicated PCI device) - ens1f0_2 (no dedicated PCI device) 6. As result we should get 7 netdevs - 1 PF (Uplink representor), 3 VFs, 3 = representors (no PCI device, pure netdev) With best regards, Slava > -----Original Message----- > From: Jan Viktorin > Sent: Monday, March 15, 2021 18:11 > To: Slava Ovsiienko > Cc: dev@dpdk.org; Asaf Penso ; Shahaf Shuler > ; Matan Azrad > Subject: Re: [PATCH 2/2] net/mlx5: avoid unbind step to enable switchdev > mode >=20 > Hello Salva, >=20 > On Mon, 15 Mar 2021 15:53:51 +0000 > Slava Ovsiienko wrote: >=20 > > Hi, Jan > > > > Yes, bullet [4] explicitly requires to unbind VFs, and detach the netde= vs > from the mlx5_core driver. > > Otherwise, kernel driver refuses to be configured with switchdev mode i= n > [5]. So, [4] can't be skipped. > > After setting swithdev mode, VFs can be bound back (if it is needed, an= d > these ones are not mapped to VMs): >=20 > OK, but I believe that it is **not possible** to follow the rule [5]. > The guide explicitly says in [4] "can be rebind **after** the switchdev > mode". > Just, if you unbind the device, there is no way how to configure the > switchdev mode, this is the contradiction I mentioned in the commit. You > cannot configure switchdev mode because the interface is gone and the pat= h > /sys/class/net//compat/devlink/mode no longer exists. >=20 > So, maybe, just the formulation is wrong. So, what is the **exact > right** way how to do it? I would change the commit accordingly. Just, le= t's > make it right. Would it work this way? >=20 > # echo -n "" > /sys/bus/pci/drivers/mlx5_core/unbind > # echo -n "" > /sys/bus/pci/drivers/mlx5_core/bind > # echo switchdev > /sys/class/net//compat/devlink/mode >=20 > It is good to mention that after the rebind, the can change. >=20 > Regards, > Jan >=20 > > > > echo -n "" > > /sys/bus/pci/drivers/mlx5_core/bind > > > > With best regards, > > Slava > > > > > -----Original Message----- > > > From: Jan Viktorin > > > Sent: Monday, March 15, 2021 17:34 > > > To: dev@dpdk.org > > > Cc: Jan Viktorin ; Asaf Penso > > > ; Shahaf Shuler ; Slava > > > Ovsiienko ; Matan Azrad > > > > Subject: [PATCH 2/2] net/mlx5: avoid unbind step to enable switchdev > > > mode > > > > > > From: Jan Viktorin > > > > > > The step 4 is a contradiction. It advices to unbind the device from > > > the mlx5_core which removes the associated system network interface > (e.g. > > > eth0). In the step 5, the same system network interface (e.g. eth0) > > > is required to exist. > > > > > > Signed-off-by: Jan Viktorin > > > --- > > > doc/guides/nics/mlx5.rst | 6 +----- > > > 1 file changed, 1 insertion(+), 5 deletions(-) > > > > > > diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst > > > index 0a2dc3dee..122d8e0fc 100644 > > > --- a/doc/guides/nics/mlx5.rst > > > +++ b/doc/guides/nics/mlx5.rst > > > @@ -1370,11 +1370,7 @@ the DPDK application. > > > > > > echo /sys/class/net/ > > device>/device/sriov_numvfs > > > > > > -4. Unbind the device (can be rebind after the switchdev mode):: > > > - > > > - echo -n "" > > > > /sys/bus/pci/drivers/mlx5_core/unbind > > > - > > > -5. Enable switchdev mode:: > > > +4. Enable switchdev mode:: > > > > > > echo switchdev > /sys/class/net/ > > device>/compat/devlink/mode > > > > > > -- > > > 2.30.1 > >