From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 04E64A0548; Sun, 28 Feb 2021 14:51:33 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6E11922A240; Sun, 28 Feb 2021 14:51:33 +0100 (CET) Received: from hqnvemgate26.nvidia.com (hqnvemgate26.nvidia.com [216.228.121.65]) by mails.dpdk.org (Postfix) with ESMTP id 7C3D84069E for ; Sun, 28 Feb 2021 14:51:31 +0100 (CET) Received: from hqpgpgate101.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate26.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Sun, 28 Feb 2021 05:51:30 -0800 Received: from HQMAIL105.nvidia.com ([172.20.161.6]) by hqpgpgate101.nvidia.com (PGP Universal service); Sun, 28 Feb 2021 05:51:30 -0800 X-PGP-Universal: processed; by hqpgpgate101.nvidia.com on Sun, 28 Feb 2021 05:51:30 -0800 Received: from HQMAIL111.nvidia.com (172.20.187.18) by HQMAIL105.nvidia.com (172.20.187.12) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 28 Feb 2021 13:51:26 +0000 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (104.47.73.48) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Sun, 28 Feb 2021 13:51:26 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=InDNUd1FBCV/Ifq+Goq0Z35ZmD6hEhPblVBUfKvvSNDMJaL5wiWMiyOp7WOwMmE8BpPLY2NtK7zkhyZ6m1CnY9alFgWtEn629+nUwR5CmaXUfPCutSG/cqYw762eUsK/qICUixG7k21kkMlMpG3VWKxNkVhUXFftmkYKhHdkdKecTnc2Yng/1D8ReyB0xAxfOIriUCanryHKPCkNGorXXwa80tbKsjkFrJxrhh6VkraJsOHfisCJsIrqDX52x66LgSFge2tRPtLwKyLntZwAYFL0AsJ4X+6j4xo9wM5ZwL6/URrWXXKSscjfRuAKEe2oYw7l6cpnDJ3/CY4ItLATTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1xbh4eKZx64r1UHfZecIazziDxqCR6B+sYLdLrjySAI=; b=Ektg4hE2y5eXCrtm85lxI77ZtmWLAVVzZUCii0kdnfcZ2bgkZjJ/JlKWc/rhPFNE2+Lo9J5YhdEsL5KKCIMDwiZjet6tIyANEzwI0QITT0zPSjmbBVwzZWOUbovY2pAaW38bCsNRAnb6ECZpP40tW3Psbx46jHPiq+dn7xocwD/1v/y5M5JVzKKVG9a8zXyyQA+sk/EMV1wbSMIesnburohtM78rx36sw0dIdkHgWshLKK1/TdbD1oUHMB2YrGaCZ/2f4ORruDQCcN3XW2jRlKqr7bLkA+i3c0DmRuPhrgR6wSpRD7BfdKG3attHa0Xvjl3QP/gCR8Mb7TWjr649Dw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Received: from BY5PR12MB4324.namprd12.prod.outlook.com (2603:10b6:a03:209::10) by BY5PR12MB4817.namprd12.prod.outlook.com (2603:10b6:a03:1fe::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.19; Sun, 28 Feb 2021 13:51:24 +0000 Received: from BY5PR12MB4324.namprd12.prod.outlook.com ([fe80::1cfe:3836:bc2:f665]) by BY5PR12MB4324.namprd12.prod.outlook.com ([fe80::1cfe:3836:bc2:f665%4]) with mapi id 15.20.3890.028; Sun, 28 Feb 2021 13:51:24 +0000 From: "Xueming(Steven) Li" To: "Wang, Haiyue" , Stephen Hemminger CC: "dev@dpdk.org" , Slava Ovsiienko , Asaf Penso Thread-Topic: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor Thread-Index: AQHXAoCvAbhl2+JarE+9Tv+NEu6dkaplCKWAgAClNACAB/tVUA== Date: Sun, 28 Feb 2021 13:51:24 +0000 Message-ID: References: <1608303356-13089-2-git-send-email-xuemingl@nvidia.com> <1613272907-22563-1-git-send-email-xuemingl@nvidia.com> <20210222175431.05de1f2b@hermes.local> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [180.162.34.192] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e00dd90b-76e2-48d3-f179-08d8dbeff030 x-ms-traffictypediagnostic: BY5PR12MB4817: x-ms-exchange-minimumurldomainage: kernel.org#8759 x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-header: ProcessedBy-CMR-outbound x-ms-oob-tlc-oobclassifiers: OLM:5516; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: usnD3wTGl1CDxiUW69f9Ww4pDs+toiMBppX1kD6vGZd70dfr6qv74wQWJHj+cO7aUZ8IHV8japB3YxKNqQOKUMNDNs4CuedGzG2232G2AdQJpNy1N9uljl/GyROrKZZznsokrYPWGiP9ilUq2Js11srNLqyWoCe1TaiwsXThxesg0lECLxZsqY+42o0/mlDldLz1ILxvHoB+Ky9TqnZ6VhzI5v70Gcm+SZt0zdWZSpNT8vMZVKDZ/HVCVO3/6dFjZhql0KZub4WLBMnP8629W45VLgVk0sWmOHdFE+y2JvHXNoF09GzkNLuMC+UakZwunjLm+Ae/YbLQiGK5zrf3luE2EXANlHmRLFRz8sLcPvAf+j1VX/EYgtkIlMITkTPuty+eAsNJhytXeX0wv7rECqhY0lcIog4v3tpnWPyAi4deIjkA0m47eliAfSddZod8hyu/2PuXUFg5qTkZkbVr/H4YbiAcWeqOOKaWaR6y0+lWd5gf0vxkWY7NLIsDCw4axkh6KYKMlR3V3yioDDXZVb5eLxmL3IeOqehuqGzUDj7OLQxth+mNXT4kHmcSRcEDGY5Skkwn2jyteLq3ajpggvtYWLGogwRJyJUKL72T6Bc= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BY5PR12MB4324.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(39860400002)(346002)(376002)(136003)(396003)(64756008)(316002)(66556008)(4326008)(66476007)(66446008)(9686003)(8936002)(33656002)(5660300002)(26005)(110136005)(52536014)(8676002)(186003)(2906002)(54906003)(7696005)(86362001)(71200400001)(53546011)(55016002)(107886003)(478600001)(76116006)(6506007)(83380400001)(66946007)(966005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?v3yboU5yCzxDDUUEf7o/+Yh3N7OUTn8SOtQeL41btwCAL4KriX7ac+iFE6ME?= =?us-ascii?Q?+dgO0ffVyrdkz+f5MIAQfKraK3xXGbvE8gPl9l3yBRp3odqa+zUHmGqY/IWW?= =?us-ascii?Q?ZZyx/F6Cej2MLBM/8DGxP5ZiZ0usGzVFwmFoQaac4iV5mg2jvwMzWNroeYSV?= =?us-ascii?Q?X1Gg876q0fOychSSJy6Ekmd0fWr59pcr8k+C8NVhsME/JNWXNKI99Kq2dazd?= =?us-ascii?Q?SekVOggMox7JAlx/gAy0kLoiDA3DZXD3gxY4sY3Geo/Hc2DRFwtQ21ULOIEg?= =?us-ascii?Q?k9HczHRJuA/mitDsw+o0O1C5lgdMbuSaL4A3e8WMTCtYMqfmF904xLNeCVK0?= =?us-ascii?Q?oYY8xbGm1pVwmMDvWCO+Nga4EDTrNykxiXHmVsIHAo9rE3lGiD5z3Ojka4ip?= =?us-ascii?Q?sHAi8Kug0HrfRD9MgZo/HdG+YJ1e616p4yJMW/jEo2vNFsgMlSNJsb3kt2u+?= =?us-ascii?Q?0BFeVXx3YbzH7+IXTQkRS6tvpM8bI/IsLiMplkLfiNv3Ig/+cG9Itfm0ldK2?= =?us-ascii?Q?JQkJUo+jfV789A3CEZY48RnRrhTqCnD+Nd86qDgIl08SK479kw7sqf/NTVLR?= =?us-ascii?Q?RWEdJ1K8uvX6JxNQevyrVJOPH0s4YPJWQ9Ny38AgAP/za34xsi9PkgiJGuaL?= =?us-ascii?Q?ngETVw9YNsTLnAb7HpW/8Dc9/d1p4UQWstlN7brJMaN53XMmrqalbMGbMzr+?= =?us-ascii?Q?O6wZP/5bOl0GCskCPYXv1kip/kMp+TaNpjkddScELM4maMCOpWaf0hEqxAvm?= =?us-ascii?Q?q2j7BcbrzRsKD9o/spI4hzUCyI88GcL2zxMajke/5jHoKKrm3ZmyYwClNuU7?= =?us-ascii?Q?bKMowivBYR1ZQCP6YxwgkP3xJrBAjXJkBXkekwa5tibVwr9Lalx0g4LuFMAL?= =?us-ascii?Q?Yq51gfxh/FO1bToyRpUGXT5QT2fZkCD/h7jS9F93HdDwDQKMUwwvWCp0gfT1?= =?us-ascii?Q?J0a6lfzSpEHi6Z0nh+4GXu5F4UlBdmQVjB9vu0F0WEWyrWXDzzoJ7DwNL/2I?= =?us-ascii?Q?loSNknsaWlFNXm0NgkUlRd0gy3rt10AWnDQpIYkUvB2/I2LyuH5oKozK9rFx?= =?us-ascii?Q?pHHcXnbvQ0xd9lGzXGvp3Tjgydb+pwaH8XI6toFj3b7sdme3OwBsWIHLlrlN?= =?us-ascii?Q?QbyWnITp5X8Ti9fxhRqOOekRwiz1g+UvEwiqyanc779goIMSYjwKWAOxT4vq?= =?us-ascii?Q?NBQcTLV4OhFAPxi1/IC3voQ1ROPN/w4G2G9suTd2m3sMS8k1Pc+77l4yBu40?= =?us-ascii?Q?pFwVEhqqJgKeeUtVs2POzfxX7yYF0PZV6LhPw8D/cCKgk4OYrT7g3eA3u76v?= =?us-ascii?Q?Cj0=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BY5PR12MB4324.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: e00dd90b-76e2-48d3-f179-08d8dbeff030 X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Feb 2021 13:51:24.5465 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: NuzbqmOW6ICBhwfg9aqe2ZGC8Iy26udb8MVs4jSa2um3QCUOYeg3qZxhF+NzimLtDj0krH3aBxEMaz70gnKXiA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4817 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1614520290; bh=1xbh4eKZx64r1UHfZecIazziDxqCR6B+sYLdLrjySAI=; h=X-PGP-Universal:ARC-Seal:ARC-Message-Signature: ARC-Authentication-Results:From:To:CC:Subject:Thread-Topic: Thread-Index:Date:Message-ID:References:In-Reply-To: Accept-Language:Content-Language:X-MS-Has-Attach: X-MS-TNEF-Correlator:authentication-results:x-originating-ip: x-ms-publictraffictype:x-ms-office365-filtering-correlation-id: x-ms-traffictypediagnostic:x-ms-exchange-minimumurldomainage: x-ms-exchange-transport-forked:x-microsoft-antispam-prvs:x-header: x-ms-oob-tlc-oobclassifiers:x-ms-exchange-senderadcheck: x-microsoft-antispam:x-microsoft-antispam-message-info: x-forefront-antispam-report:x-ms-exchange-antispam-messagedata: Content-Type:Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype: X-MS-Exchange-CrossTenant-userprincipalname: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=M0Qcgxf+aTaHlEuPywUH93OmIAjOsgpWGBHHw0UV0FyRjuf8JnYuV2l161tG7h5k/ MEj2k1p52RKKhIjHiIVeG2kRCsyGhMMbUdsz85WM+Piw9t0z+3uCI2MUoWypCI+bwH 98Yz4Yr4oGUbhimmR5p2pGg4L9d6OSKeXOx0t9doIw4lpniqJZnLqG1hpGWc4h2gIr oJg6wJOwfPj7qXJGGTV5wequh9eMFdAOuGkEU1jzYTuQrZlfseQ1WRsthFC7kPImYR UAzhoCa5i7xkR5XLS4XoNHbudY96q+MKcVnpg4e2UjvpZ2M0RPWzrFpsVy76Ws0t5k jAzVrxATosNRw== Subject: Re: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction representor X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >-----Original Message----- >From: Wang, Haiyue >Sent: Tuesday, February 23, 2021 7:46 PM >To: Stephen Hemminger ; Xueming(Steven) Li >Cc: dev@dpdk.org; Slava Ovsiienko ; Asaf Penso >Subject: RE: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction represe= ntor > >> -----Original Message----- >> From: dev On Behalf Of Stephen Hemminger >> Sent: Tuesday, February 23, 2021 09:55 >> To: Xueming Li >> Cc: dev@dpdk.org; Viacheslav Ovsiienko ; Asaf >> Penso >> Subject: Re: [dpdk-dev] [PATCH v6 0/9] ethdev: support SubFunction >> representor >> >> On Sun, 14 Feb 2021 03:21:30 +0000 >> Xueming Li wrote: >> >> > SubFunction [1] is a portion of the PCI device, a SF netdev has its >> > own dedicated queues(txq, rxq). A SF netdev supports E-Switch >> > representation offload similar to existing PF and VF representors. A >> > SF shares PCI level resources with other SFs and/or with its parent PC= I function. >> > >> > From SmartNIC perspective, when PCI device is shared for multi-host, >> > representors for host controller and host PF is required. >> > >> > This patch set introduces new representor types in addtion to >> > existing VF representor. Syntax: >> > >> > [[c#]pf#]vf#: VF port representor/s from controller/pf >> > [[c#]pf#]sf#: SF port representor/s from controller/pf >> > #: VF representor - for backwards compatibility >> > >> > "#" is number instance, list or range, valid examples: >> > 1, [1,3,5], [0-3], [0,2-4,6] >> > >> > For backward compatibility, this patch also introduces new netdev >> > capability to indicate the capability of supportting SF representor. >> > >> > Version history: >> > RFC: >> > initial version [2] >> > V2: >> > - separate patch for represnetor infrastructure, controller, pf an= d >> > sf. >> > - replace representor ID macro with functions: >> > rte_eth_representor_id_encode() >> > rte_eth_representor_id_parse() >> > - new patch to allow devargs with same PCI BDF but different >> > representors. >> > - other minor code updates according to comments, thanks Andrew! >> > - update document >> > V3: >> > - improve probing of allowed devargs with same name. >> > - parse single word of kvargs as key. >> > - update kvargs test cases. >> > V4: >> > - split first representor refactor patch into >> > 1: add representor type >> > 2: refector representor list parsing >> > - push the patch supporting multi-devargs for same device. >> > V5: >> > - add comments for parsing functions >> > - update switch_representation.rst - Thanks Ajit >> > V6: >> > - split representor types into different patches, move to >> > rte_ethdev.h >> > - improvements of rte_eth_devargs_process_list() according to >> > Andrew's suggestion >> > - fixed PF probe failure for Intel i40e >> > - replace ethdev SF capability with rte_eth_representor_info_get() >> > - add new ethdev ops api to get representor info from PMD >> > - replace representor ID encode/decode with conversion from >> > representor info >> > - change ethdev representor iterator to use new ID encoding >> > >> > >> > Xueming Li (9): >> > ethdev: introduce representor type >> > ethdev: support representor port list >> > ethdev: support new VF representor syntax >> > ethdev: support sub function representor >> > ethdev: support PF index in representor >> > ethdev: support multi-host in representor >> > ethdev: new API to get representor info >> > ethdev: representor iterator compare complete info >> > kvargs: update parser to support lists >> > >> > app/test/test_kvargs.c | 46 ++++- >> > config/rte_config.h | 1 + >> > doc/guides/prog_guide/poll_mode_drv.rst | 13 +- >> > .../prog_guide/switch_representation.rst | 35 +++- >> > drivers/net/bnxt/bnxt_ethdev.c | 7 + >> > drivers/net/enic/enic_ethdev.c | 6 + >> > drivers/net/i40e/i40e_ethdev.c | 7 + >> > drivers/net/ixgbe/ixgbe_ethdev.c | 7 + >> > drivers/net/mlx5/linux/mlx5_os.c | 11 ++ >> > lib/librte_ethdev/ethdev_driver.h | 49 ++++- >> > lib/librte_ethdev/ethdev_private.c | 173 ++++++++++++-----= - >> > lib/librte_ethdev/ethdev_private.h | 3 - >> > lib/librte_ethdev/rte_class_eth.c | 40 ++-- >> > lib/librte_ethdev/rte_ethdev.c | 102 ++++++++++- >> > lib/librte_ethdev/rte_ethdev.h | 53 ++++++ >> > lib/librte_ethdev/version.map | 4 + >> > lib/librte_kvargs/rte_kvargs.c | 101 +++++++--- >> > 17 files changed, 535 insertions(+), 123 deletions(-) >> > >> >> A couple of higher level design questions: >> 1. How does Linux and other OS handle this in their API? Hi Stephen, thanks for looking this! Representor was a type of ethdev, OS independent. SF representor is a new type besides existing VF representor type. >> 2. This solution seems quite tied into a specific implementation on your= hardware. >> Is there a PCI standard for this? SF has been carved in kernel, supported by devlink. The devlink document in= the link Haiyue posted should be helpful to understand the overall picture= : https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?= id=3D24a790da0ac4d9bcce2a9becc8799241716672f6 > >> 3. Maybe a more general solution where these were represented as buses w= ould >> allow for other connection methods in the future? > >It should be "auxiliary bus", I think. ;-) https://git.kernel.org/pub/scm/= linux/kernel/git/torvalds/linux.git/tree/drivers/base/auxiliary.c Good link, Mellanox has exactly implemented SF based auxiliary bus, I'm goi= ng to post another RFC to support auxiliary bus. But this patchset still needed to support representor of SF. > >mlx5 subfunction support >https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/= ?id=3D24a790da0ac4d9bcce2a9becc8799241716672f6