From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id C101336E for ; Tue, 7 Mar 2017 09:36:24 +0100 (CET) Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 Mar 2017 00:36:23 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,257,1484035200"; d="scan'208,223";a="65033118" Received: from pgsmsx102.gar.corp.intel.com ([10.221.44.80]) by orsmga004.jf.intel.com with ESMTP; 07 Mar 2017 00:36:22 -0800 Received: from pgsmsx108.gar.corp.intel.com ([169.254.8.60]) by PGSMSX102.gar.corp.intel.com ([169.254.6.180]) with mapi id 14.03.0248.002; Tue, 7 Mar 2017 16:36:21 +0800 From: "Choi, Sy Jong" To: "spp@dpdk.org" Thread-Topic: SPP PCIe hotplug changes for SPP Thread-Index: AdKLVBzvN+mkyAnqRFmtH61ikylTFACgzY+gAlGWXtA= Date: Tue, 7 Mar 2017 08:36:20 +0000 Message-ID: <697F8B1B48670548A5BAB03E8283550F36A279B8@PGSMSX108.gar.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 10.0.102.7 dlp-reaction: no-action x-originating-ip: [172.30.20.205] MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [spp] SPP PCIe hotplug changes for SPP X-BeenThere: spp@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Soft Patch Panel List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Mar 2017 08:36:25 -0000 Hi Everyone, Second patch to add protection to primary for port start/stop and attach/de= tach. Regards, Choi, Sy Jong Platform Application Engineer -----Original Message----- From: Choi, Sy Jong=20 Sent: Thursday, February 23, 2017 9:21 PM To: 'spp@dpdk.org' Subject: RE: SPP PCIe hotplug changes for SPP Hi Everyone, First patch to enable spp & primary to support port start/stop and attach/d= etach. >>From e58c90661f702276dfeae2fc1f3e43750fe2ae2c Mon Sep 17 00:00:00 2001 From: "Choi, Sy Jong" Date: Thu, 23 Feb 2017 12:54:50 +0800 Subject: [PATCH] Signed-off-by: Choi, Sy Jong spp: adding port start/stop detach/attach primary: adding port start/stop detach/attach --- src/primary/init.c | 2 +- src/primary/init.h | 2 ++ src/primary/main.c | 91 ++++++++++++++++++++++++++++++++++++++++++++++++++= +--- src/spp.py | 46 ++++++++++++++++++++++++--- 4 files changed, 131 insertions(+), 10 deletions(-) diff --git a/src/primary/init.c b/src/primary/init.c index 5ea21f9..e5809e3= 100644 --- a/src/primary/init.c +++ b/src/primary/init.c @@ -47,7 +47,7 @@ struct client *clients; =20 /* The mbuf pool for packet rx */ -static struct rte_mempool *pktmbuf_pool; +struct rte_mempool *pktmbuf_pool; =20 /* the port details */ struct port_info *ports; diff --git a/src/primary/init.h b/src/primary/init.h index 915552c..82602a2= 100644 --- a/src/primary/init.h +++ b/src/primary/init.h @@ -56,6 +56,8 @@ struct client { }; =20 extern struct client *clients; +/* The mbuf pool for packet rx */ +extern struct rte_mempool *pktmbuf_pool; =20 /* the shared port information: port numbers, rx and tx stats etc. */ ext= ern struct port_info *ports; diff --git a/src/primary/main.c b/src/primary/= main.c index 15fa53a..94cda18 100644 --- a/src/primary/main.c +++ b/src/primary/main.c @@ -185,9 +185,11 @@ static int parse_command(char *str) { char *token_list[MAX_PARAMETER] =3D {NULL}; + char name[RTE_ETH_NAME_MAX_LEN]; int ret =3D 0; int i =3D 0; - + uint8_t port_id =3D 255; //invalid port id +=09 /* tokenize the user commands from controller */ token_list[i] =3D strtok(str, " "); while (token_list[i] !=3D NULL) { @@ -205,6 +207,88 @@ parse_command(char *str) else sprintf(str, "Server Idling\n"); =20 + } else if (!strcmp(token_list[0], "port")) { + //RTE_LOG(INFO, APP, "port\n"); + if (i <=3D 1) { + //RTE_LOG(INFO, APP, "i lesser 1\n"); + return -1; =09 + } + if (strcmp(token_list[1], "stop") =3D=3D 0) { + if (i <=3D 2) { + //RTE_LOG(INFO, APP, "i lesser 2\n"); + return -1; + } + + port_id =3D atoi(token_list[2]); + if (port_id =3D=3D 255) { + //RTE_LOG(INFO, APP, "port lesser 0\n");=09 + return -1; + } + else { + rte_eth_dev_stop(port_id); + sprintf(str, "Port %d stopped\n", port_id); + } =09 + } else if (strcmp(token_list[1], "start") =3D=3D 0) { + if (i <=3D 2) { + return -1; + } + + port_id =3D atoi(token_list[2]); + if (port_id =3D=3D 255) { + return -1; + } + else { + if (init_port(port_id, pktmbuf_pool) =3D=3D 0) + sprintf(str, "Port %d started\n", port_id); + } =09 + } else if (strcmp(token_list[1], "close") =3D=3D 0) { + if (i <=3D 2) { + return -1; + } + + port_id =3D atoi(token_list[2]); + if (port_id =3D=3D 255) { + return -1; + } + else { + rte_eth_dev_close(port_id); + sprintf(str, "Port %d closed\n", port_id); + } + } else if (strcmp(token_list[1], "detach") =3D=3D 0) { + if (i <=3D 2) { + return -1; + } + + port_id =3D atoi(token_list[2]); + if (port_id =3D=3D 255) { + return -1; + } + else { + if (rte_eth_dev_detach(port_id, name) =3D=3D 0) + sprintf(str, "Port %d %s detached\n", port_id, name); + } + } else if (strcmp(token_list[1], "attach") =3D=3D 0) { + if (i <=3D 2) { + return -1; + } + + if (token_list[2] =3D=3D NULL) { + return -1; + } + else { + if (rte_eth_dev_attach(token_list[2], &port_id) =3D=3D 0) + { + sprintf(str, "Port %d %s attached\n", port_id, name); + } + else + { + sprintf(str, "PCIe %s attached failed\n", name); + } + } =09 + } else { + //RTE_LOG(INFO, APP, "else port\n"); + ret =3D -1; =09 + } =09 } else if (!strcmp(token_list[0], "exit")) { RTE_LOG(DEBUG, APP, "exit\n"); RTE_LOG(DEBUG, APP, "stop\n"); @@ -322,15 +406,14 @@ main(int argc, char *argv[]) sleep(1); continue; } - ret =3D do_receive(&connected, &sock, str); if (ret < 0) continue; - + =09 RTE_LOG(DEBUG, APP, "Received string: %s\n", str); =20 ret =3D parse_command(str); - if (ret < 0) + if (ret < 0) break; =20 /* Send the message back to client */ diff --git a/src/spp.py b/src/spp.= py index b937b5a..3da74fa 100755 --- a/src/spp.py +++ b/src/spp.py @@ -229,13 +229,44 @@ def close_all_secondary(): command_secondary(i, 'exit') SECONDARY_COUNT =3D 0 =20 +def check_pri_cmds(cmds): + """Validate primary commands before sending""" + level1 =3D ['status', 'exit', 'clear'] + level2 =3D ['add', 'del', 'port'] + add_del_args =3D ['ring', 'vhost'] + port_args =3D ['stop', 'detach', 'start', 'attach', 'close'] + valid =3D 0=09 + cmdlist =3D cmds.split(' ') + + length =3D len(cmdlist) + if length =3D=3D 1: + if cmdlist[0] in level1: + valid =3D 1 + elif length =3D=3D 3: + if cmdlist[0] in level2: + if cmdlist[0] =3D=3D 'add' or cmdlist[0] =3D=3D 'del': + if cmdlist[1] in add_del_args: + if str.isdigit(cmdlist[2]): + valid =3D 1 + elif cmdlist[0] =3D=3D 'port': + if cmdlist[1] in port_args: + if cmdlist[1] !=3D 'attach': + if str.isdigit(cmdlist[2]): + valid =3D 1 + else: + if cmdlist[2]: + valid =3D 1 + + return valid +=09 def check_sec_cmds(cmds): """Validate secondary commands before sending""" =20 - level1 =3D ['status', 'exit', 'forward', 'stop'] - level2 =3D ['add', 'patch', 'del'] + level1 =3D ['status', 'exit', 'forward', 'stop', 'start'] + level2 =3D ['add', 'patch', 'del', 'port'] patch_args =3D ['reset'] add_del_args =3D ['ring', 'vhost'] + port_args =3D ['stop', 'detach', 'start', 'attach', 'close'] cmdlist =3D cmds.split(' ') valid =3D 0 =20 @@ -256,7 +287,10 @@ def check_sec_cmds(cmds): elif cmdlist[0] =3D=3D 'patch': if str.isdigit(cmdlist[1]) and str.isdigit(cmdlist[2]): valid =3D 1 - + elif cmdlist[0] =3D=3D 'port': + if cmdlist[1] in port_args: + if str.isdigit(cmdlist[2]): + valid =3D 1 return valid =20 class Shell(cmd.Cmd): @@ -267,7 +301,8 @@ class Shell(cmd.Cmd): recorded_file =3D None =20 COMMANDS =3D ['status', 'add', 'patch', 'ring', 'vhost', - 'reset', 'exit', 'forward', 'stop', 'clear'] + 'reset', 'exit', 'forward', 'stop', 'clear', + 'pause', 'start', 'port', 'detach'] =20 def complete_pri(self, text, line, begidx, endidx): """Completion for primary process commands""" @@ -301,7 +336,8 @@ class Shell(cmd.Cmd): def do_pri(self, command): """Send command to primary process""" =20 - if command and command in self.COMMANDS: +# if command and command in self.COMMANDS: + if check_pri_cmds(command): command_primary(command) else: print ("primary invalid command") -- 1.8.3.1 Regards, Choi, Sy Jong Platform Application Engineer -----Original Message----- From: Choi, Sy Jong=20 Sent: Monday, February 20, 2017 4:37 PM To: 'spp@dpdk.org' Subject: SPP PCIe hotplug changes for SPP Hi Everyone, New Architecture for SPP to support PCIe hotplug. Based on DPDK testpmd here are the steps you need to perform PCIe hotplug 1) Detaching : port stop, port close and then port detach 2) Attaching : port attach and then port start In order to support PCI hotplug, we need to follow DPDK. And here are a few= limitation with SPP primary secondary configuration:- Limitation 1:- Port information created by primary, hotplug changes port information. So a= ttach and detaching at primary, will not update port information on seconda= ry process. - Now primary will attaching and detaching the port, because primary owns t= he port=20 - We need to put in place some rediscovery in secondary processes. Limitation 2: To update the port information on the secondary process, it require to us t= o reinit dpdk eal, which dpdk eal init can only be executed once. - Multi thread will not solve the problem. - We will use fork, so we separated the communication part and the datapath= part. The new spp primary will attach/detach port, secondary will reinit to redis= cover port changes. +----+----------+-------------------------------------------------+ =20 | | SPP NFV | | =20 | +----------+ +--------------+ fork +--------------+ | =20 | | Parent | | Child | | =20 | | Comm - spp.py| | dpdk | |=20 | | Commands | | init | | =20 | | share mem to | | fwd | | | | control child| | | | +-------------------+--------------+-----------+--------------+---+ Code changes will be posted in the following emails. Regards, Choi, Sy Jong Platform Application Engineer