From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6FCE3A00C2; Thu, 16 Jun 2022 19:11:19 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5F90C4014F; Thu, 16 Jun 2022 19:11:19 +0200 (CEST) Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2048.outbound.protection.outlook.com [40.107.95.48]) by mails.dpdk.org (Postfix) with ESMTP id E494540141 for ; Thu, 16 Jun 2022 19:11:17 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MZoOh8CodurKFL1QO3EJsHOIRuNZTU2eSy2Ykg61vUhfHixSUD2zD3pqMkJfSR/PCnZ2wXfxn3R//pPnKFocHaLU7sCiZumh9MYx5KgNAzNzBvbVUTIov86cTOakSNsrjzWPr/H0dqvwKIacBeUJTjfIuIfg7aYk9a6mAMDLmj0PEGiakH+XIT1SpmAesL0ZqmRIUFHn/Ynwyysa6Fs+0zRXgXDzzzdbUovfLF7g0VzErNueCJP4kdNcHAfsFkoOG/WQ3BLbGoGM7PrgHsYd3NqR++wmS/856UCv6XdKh1sfPYRXXow/WBMnLAhxcU8AzcqOGEOBt5E/soBFcI/efg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mYdfTAJsxpd6+itINib3In5r3hFDNH+yZdzqVOj1SzE=; b=aBVXRJmVvzfWfPQXwLGrVu/TaLHsW89I9vCC4EVJKyYNxRs0kUDkNwacTDXxxmIRj2Q99iSprt13i6b40830ZCzSF8TdWfFufpo50sq/UYEn/4UNauMGCu5h8O7kuQ94NcnYjw6LyZxCZ4SJ99GPwHd/uis79qee/O5WEP6oadFglYFzdA2c41NlFgFFywDebRx6UxDS+aPWB8eY8K2tNma8FNxS+e/BbQo2QtkQwciYsGaHedfcMtMi+qDphtyX4tjp8K1lkIliE1Lm4JVC+jsdpcdO/cFf4IxVqkfk6kwyX7dNZQowafcs+B+cv/z8NFUtP7egw29PP59hw3+nOg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mYdfTAJsxpd6+itINib3In5r3hFDNH+yZdzqVOj1SzE=; b=PupWnio7U8u2MTsMHsWPv4+nmexCwd2n78mvrPwHKl6DfRwnGE2Qwt6lNbJn70lcHe16s4TTR6reBP2d4sqqh/OnvauUCOMgfwBwZSBtdiWslR2/bOv/600TceD+CcSzShr/CEfGmOkcchycW3ZrFE0wf8HGfX6wQXVUp0uBNNNh31S6iIfb1ETXJsfKjSyujAG6pokdLx8IDgsilSCpGWX0qcrV+79KPuVm6rD6v+Cy5xYem7zsJ7dbIEd0oiICkjlLkEx+1MCJfpUJI0rJjaimJ4DdFkTNucAIAZ21eNyZNXFpgyQE1Z3hV9Gv97He6NJZDUCw69Esh8mZTgdXMQ== Received: from DM5PR16CA0017.namprd16.prod.outlook.com (2603:10b6:3:c0::27) by DS7PR12MB6093.namprd12.prod.outlook.com (2603:10b6:8:9e::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.15; Thu, 16 Jun 2022 17:11:15 +0000 Received: from DM6NAM11FT022.eop-nam11.prod.protection.outlook.com (2603:10b6:3:c0:cafe::53) by DM5PR16CA0017.outlook.office365.com (2603:10b6:3:c0::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.16 via Frontend Transport; Thu, 16 Jun 2022 17:11:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (12.22.5.236) by DM6NAM11FT022.mail.protection.outlook.com (10.13.172.210) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5332.12 via Frontend Transport; Thu, 16 Jun 2022 17:11:15 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Thu, 16 Jun 2022 17:10:28 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Thu, 16 Jun 2022 10:10:27 -0700 Received: from nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22 via Frontend Transport; Thu, 16 Jun 2022 10:10:26 -0700 From: Michael Baum To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Subject: [PATCH v2 1/2] app/testpmd: add test for remote PD and CTX Date: Thu, 16 Jun 2022 20:10:16 +0300 Message-ID: <20220616171017.2597941-2-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220616171017.2597941-1-michaelba@nvidia.com> References: <20220301202615.4103972-1-michaelba@nvidia.com> <20220616171017.2597941-1-michaelba@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 25bbe72d-4851-42f5-7555-08da4fbb38bf X-MS-TrafficTypeDiagnostic: DS7PR12MB6093:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: neoE9UVYDVZgfqm/awu9TRno5BD5NNomZ1di7O8Xuwx6txGo81lopgiAVVAF3oUx93PGJem6Vqs7l6v9/fPEqlXzPmvgXqlnaER2XlDLGtZm8AAyLogyEYjze3BFEMvpSKidtoEWm/9KK/e2Lgu7kf2gR7neyEQbrnE//DogyKk+WUBdl0bk4y5TmtAmUaVK9AQtctnm80a31CJbmWW4q+qJV24T3l+h1MlDHy8uQYZ9+93CI4TfvLGUqzJlzUg53KcFU2jpMpsdHMiKGpT865fVvSb0NcehAFxc6XU+emjgmSr/M/PGBBTK6jur+8P1LlnwAY0fUbILbuwPcebhGmSINuWmMyJxgEbwNtFG9Mbxb8OLl1TCmyT/idRmDiwVzEJOKlYRi68gPg/h/lIGUmV6nO/JhGkSrilDbPhztPEgE07/83vbDwp+edobbZJxA+MX27CcZYPClGsC0Xw/1r8ZJ6B5tTUJoTUBXYeJQZYY7tF1rxJgC59KPniRCF1FSFDCKN7eM6O7ZJxTYGTXLwRXEr88GQp+zPepWatbT1q3ocNXA7Hl0D7w5smiNrcxc0qeWSO36AgKuzO0QXo8dZVlcgblKDu3+u9Tq1OtAcAqaJ9XxBOnkOfENxrxsxXX6WB243qFkQ21XIQa3i4mEwXdHSvZt4b4NRmVrAMnFhhF3E7cuP1AitgD0GDhLjrD9URjLBHB9WJ3IWzoJYzaJQ== X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230016)(4636009)(46966006)(40470700004)(36840700001)(107886003)(356005)(2616005)(81166007)(1076003)(36860700001)(36756003)(186003)(83380400001)(316002)(336012)(47076005)(8676002)(8936002)(5660300002)(70206006)(2906002)(82310400005)(55016003)(6666004)(7696005)(4326008)(40460700003)(6916009)(54906003)(26005)(6286002)(86362001)(70586007)(498600001)(426003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2022 17:11:15.3553 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 25bbe72d-4851-42f5-7555-08da4fbb38bf X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT022.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6093 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add mlx5 internal option in testpmd run-time function "port attach" to add another parameter named "mlx5_socket" for attaching port and add 2 devargs before. The arguments are "cmd_fd" and "pd_handle" using to import device created out of PMD. Testpmd application import it using IPC, and updates the devargs list before attaching. The syntax is: testpmd > port attach (identifier) mlx5_socket=(path) Where "path" is the IPC socket path agreed on the remote process. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- app/test-pmd/cmdline.c | 14 ++- app/test-pmd/testpmd.c | 5 ++ doc/guides/nics/mlx5.rst | 44 ++++++++++ drivers/net/mlx5/mlx5_testpmd.c | 145 ++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_testpmd.h | 16 ++++ 5 files changed, 222 insertions(+), 2 deletions(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index a59e6166d5..869ecd3d2a 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -780,6 +780,12 @@ static void cmd_help_long_parsed(void *parsed_result, "port attach (ident)\n" " Attach physical or virtual dev by pci address or virtual device name\n\n" +#ifdef RTE_NET_MLX5 + "port attach (ident) mlx5_socket=(path)\n" + " Attach physical or virtual dev by pci address or virtual device name " + "and add \"cmd_fd\" and \"pd_handle\" devargs before attaching\n\n" +#endif + "port detach (port_id)\n" " Detach physical or virtual dev by port_id\n\n" @@ -1401,8 +1407,12 @@ static cmdline_parse_token_string_t cmd_operate_attach_port_identifier = static cmdline_parse_inst_t cmd_operate_attach_port = { .f = cmd_operate_attach_port_parsed, .data = NULL, - .help_str = "port attach : " - "(identifier: pci address or virtual dev name)", + .help_str = "port attach mlx5_socket=: " + "(identifier: pci address or virtual dev name" +#ifdef RTE_NET_MLX5 + ", path (optional): socket path to get cmd FD and PD handle" +#endif + ")", .tokens = { (void *)&cmd_operate_attach_port_port, (void *)&cmd_operate_attach_port_keyword, diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index e6321bdedb..d2df6732a0 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3360,6 +3360,11 @@ attach_port(char *identifier) return; } +#if defined(RTE_NET_MLX5) && !defined(RTE_EXEC_ENV_WINDOWS) + if (mlx5_test_attach_port_extend_devargs(identifier) < 0) + return; +#endif + if (rte_dev_probe(identifier) < 0) { TESTPMD_LOG(ERR, "Failed to attach port %s\n", identifier); return; diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 1b66b2bc33..392292cc95 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -1777,3 +1777,47 @@ the command sets the current shaper to 5Gbps and disables avail_thresh_triggered .. code-block:: console testpmd> mlx5 set port 1 host_shaper avail_thresh_triggered 0 rate 50 + + +Testpmd +------- + +port attach with socket path +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Attach a port specified by pci address or virtual device args and add extra +devargs to it, which is imported from external process:: + + testpmd> port attach (identifier) mlx5_socket=(path) + +where: + +* ``identifier``: pci address or virtual device args. +* ``path``: socket path to import arguments agreed by the external process. + +The mlx5 PMD enables to import CTX and PD created outside the PMD. +It gets as devargs the device's ``cmd_fd`` and ``pd_handle``, +then using those arguments to import objects. +See :ref:`mlx5 driver options ` for more information. + +When ``cmd_fd`` and ``pd_handle`` arguments are coming from another process, +the FD must be dup'd before being passed. +In this function, testpmd initializes IPC socket to get FD using SCM_RIGHTS. +It gets the external process socket path, then import the ``cmd_fd`` and +``pd_handle`` arguments and add them to devargs list. +After updating this, it calls the regular ``port attach`` function +with extended identifier. + +For example, to attach a port whose pci address is ``0000:0a:00.0`` and its +socket path is ``/var/run/import_ipc_socket``. + +.. code-block:: console + + testpmd> port attach 0000:0a:00.0 mlx5_socket=/var/run/import_ipc_socket + Attaching a new port... + testpmd: MLX5 socket path is /var/run/import_ipc_socket + testpmd: Attach port with extra devargs 0000:0a:00.0,cmd_fd=40,pd_handle=1 + EAL: Probe PCI driver: mlx5_pci (15b3:101d) device: 0000:0a:00.0 (socket 0) + Port 0 is attached. Now total ports is 1 + Done + diff --git a/drivers/net/mlx5/mlx5_testpmd.c b/drivers/net/mlx5/mlx5_testpmd.c index 98bd395ae0..46444f06e6 100644 --- a/drivers/net/mlx5/mlx5_testpmd.c +++ b/drivers/net/mlx5/mlx5_testpmd.c @@ -6,6 +6,11 @@ #include #include #include +#include +#ifndef RTE_EXEC_ENV_WINDOWS +#include +#include +#endif #include #include @@ -14,6 +19,7 @@ #include #include #include + #include "mlx5_testpmd.h" #include "testpmd.h" @@ -111,6 +117,145 @@ mlx5_test_set_port_host_shaper(uint16_t port_id, uint16_t avail_thresh_triggered return 0; } +#ifndef RTE_EXEC_ENV_WINDOWS +static const char* +mlx5_test_get_socket_path(char *extend) +{ + if (strstr(extend, "mlx5_socket=") == extend) { + const char *socket_path = strchr(extend, '=') + 1; + + TESTPMD_LOG(DEBUG, "MLX5 socket path is %s\n", socket_path); + return socket_path; + } + + TESTPMD_LOG(ERR, "Failed to extract a valid socket path from %s\n", + extend); + return NULL; +} + +static int +mlx5_test_extend_devargs(char *identifier, char *extend) +{ + struct sockaddr_un un = { + .sun_family = AF_UNIX, + }; + int cmd_fd; + int pd_handle; + struct iovec iov = { + .iov_base = &pd_handle, + .iov_len = sizeof(int), + }; + union { + char buf[CMSG_SPACE(sizeof(int))]; + struct cmsghdr align; + } control; + struct msghdr msgh = { + .msg_iov = NULL, + .msg_iovlen = 0, + }; + struct cmsghdr *cmsg; + const char *path = mlx5_test_get_socket_path(extend + 1); + size_t len = 1; + int socket_fd; + int ret; + + if (path == NULL) { + TESTPMD_LOG(ERR, "Invalid devargs extension is specified\n"); + return -1; + } + + /* Initialize IPC channel. */ + socket_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); + if (socket_fd < 0) { + TESTPMD_LOG(ERR, "Failed to create unix socket: %s\n", + strerror(errno)); + return -1; + } + rte_strlcpy(un.sun_path, path, sizeof(un.sun_path)); + if (connect(socket_fd, (struct sockaddr *)&un, sizeof(un)) < 0) { + TESTPMD_LOG(ERR, "Failed to connect %s: %s\n", un.sun_path, + strerror(errno)); + close(socket_fd); + return -1; + } + + /* Send the request message. */ + do { + ret = sendmsg(socket_fd, &msgh, 0); + } while (ret < 0 && errno == EINTR); + if (ret < 0) { + TESTPMD_LOG(ERR, "Failed to send request to (%s): %s\n", path, + strerror(errno)); + close(socket_fd); + return -1; + } + + msgh.msg_iov = &iov; + msgh.msg_iovlen = 1; + msgh.msg_control = control.buf; + msgh.msg_controllen = sizeof(control.buf); + do { + ret = recvmsg(socket_fd, &msgh, 0); + } while (ret < 0); + if (ret != sizeof(int) || (msgh.msg_flags & (MSG_TRUNC | MSG_CTRUNC))) { + TESTPMD_LOG(ERR, "truncated msg"); + close(socket_fd); + return -1; + } + + /* Translate the FD. */ + cmsg = CMSG_FIRSTHDR(&msgh); + if (cmsg == NULL || cmsg->cmsg_len != CMSG_LEN(sizeof(int)) || + cmsg->cmsg_level != SOL_SOCKET || cmsg->cmsg_type != SCM_RIGHTS) { + TESTPMD_LOG(ERR, "Fail to get FD using SCM_RIGHTS mechanism\n"); + close(socket_fd); + unlink(un.sun_path); + return -1; + } + memcpy(&cmd_fd, CMSG_DATA(cmsg), sizeof(int)); + + TESTPMD_LOG(DEBUG, "Command FD (%d) and PD handle (%d) " + "are successfully imported from remote process\n", + cmd_fd, pd_handle); + + /* Cleanup IPC channel. */ + close(socket_fd); + + /* Calculate the new length of devargs string. */ + len += snprintf(NULL, 0, ",cmd_fd=%d,pd_handle=%d", cmd_fd, pd_handle); + /* Extend the devargs string. */ + snprintf(extend, len, ",cmd_fd=%d,pd_handle=%d", cmd_fd, pd_handle); + + TESTPMD_LOG(DEBUG, "Attach port with extra devargs %s\n", identifier); + return 0; +} + +static bool +is_delimiter_path_spaces(char *extend) +{ + while (*extend != '\0') { + if (*extend != ' ') + return true; + extend++; + } + return false; +} + +int +mlx5_test_attach_port_extend_devargs(char *identifier) +{ + char *extend = strchr(identifier, ' '); + + if (extend != NULL && is_delimiter_path_spaces(extend) && + mlx5_test_extend_devargs(identifier, extend) < 0) { + TESTPMD_LOG(ERR, "Failed to extend devargs for port %s\n", + identifier); + return -1; + } + return 0; +} +#endif + /* *** SET HOST_SHAPER FOR A PORT *** */ struct cmd_port_host_shaper_result { cmdline_fixed_string_t mlx5; diff --git a/drivers/net/mlx5/mlx5_testpmd.h b/drivers/net/mlx5/mlx5_testpmd.h index 7a54658eb5..06976341a4 100644 --- a/drivers/net/mlx5/mlx5_testpmd.h +++ b/drivers/net/mlx5/mlx5_testpmd.h @@ -23,4 +23,20 @@ void mlx5_test_avail_thresh_event_handler(uint16_t port_id, uint16_t rxq_id); +/** + * Extend devargs list with "cmd_fd" and "pd_handle" coming from external + * process. It happens only in this format: + * testpmd> port attach (identifier) mlx5_socket= + * all "(identifier) mlx5_socket=" is in the same string pointed + * by the input parameter 'identifier'. + * + * @param identifier + * Identifier of port attach command line. + * + * @return + * 0 on success, -1 on failure. + */ +int +mlx5_test_attach_port_extend_devargs(char *identifier); + #endif -- 2.25.1