[dpdk-dev] [PATCH 0/2] kni: add API to set link status on kernel interface

[dpdk-dev] [PATCH 0/2] kni: add API to set link status on kernel interface

[Dan Gora]

Hi All,

The following patches are to add support for DPDK applications to be
able to change the carrier state of Linux network interfaces in the
KNI kernel module.

The carrier state is changed by writing to the Linux /sys file:
/sys/devices/virtual/net/<name>/carrier, where <name> is the KNI
interface name.

These patches supercede:
'[PATCH v2 10/10] kni: add API to set link status on kernel interface'
https://mails.dpdk.org/archives/dev/2018-August/110383.html

Dan Gora (2):
  kni: add API to set link status on kernel interface
  kni: set default carrier state to 'off'

 kernel/linux/kni/kni_misc.c |  2 ++
 kernel/linux/kni/kni_net.c  |  2 ++
 lib/librte_kni/rte_kni.c    | 57 +++++++++++++++++++++++++++++++++++++
 lib/librte_kni/rte_kni.h    | 18 ++++++++++++
 4 files changed, 79 insertions(+)

-- 
2.19.0

[dpdk-dev] [PATCH 1/2] kni: add API to set link status on kernel interface

[Dan Gora]

Add a new API function to KNI, rte_kni_update_link() to allow DPDK
applications to update the link status for KNI network interfaces in
the linux kernel.

Signed-off-by: Dan Gora <dg@adax.com>
---
 lib/librte_kni/rte_kni.c | 57 ++++++++++++++++++++++++++++++++++++++++
 lib/librte_kni/rte_kni.h | 18 +++++++++++++
 2 files changed, 75 insertions(+)

diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c
index 65f6a2b03..afe1d7410 100644
--- a/lib/librte_kni/rte_kni.c
+++ b/lib/librte_kni/rte_kni.c
@@ -790,6 +790,63 @@ rte_kni_unregister_handlers(struct rte_kni *kni)
 
 	return 0;
 }
+
+int
+rte_kni_update_link(struct rte_kni *kni, struct rte_eth_link *link)
+{
+	char path[64];
+	char carrier[2];
+	const char *new_carrier;
+	int fd, ret;
+
+	if (kni == NULL || link == NULL)
+		return -1;
+
+	snprintf(path, sizeof(path), "/sys/devices/virtual/net/%s/carrier",
+		kni->name);
+
+	fd = open(path, O_RDWR);
+	if (fd == -1) {
+		RTE_LOG(ERR, KNI, "Failed to open file: %s.\n", path);
+		return -1;
+	}
+
+	ret = read(fd, carrier, 2);
+	if (ret < 1) {
+		/* Cannot read carrier until interface is marked
+		 * 'up', so don't log an error.
+		 */
+		close(fd);
+		return -1;
+	}
+
+	new_carrier = (link->link_status == ETH_LINK_UP) ? "1" : "0";
+	ret = write(fd, new_carrier, 1);
+	if (ret < 1) {
+		RTE_LOG(ERR, KNI, "Failed to write file: %s.\n", path);
+		close(fd);
+		return -1;
+	}
+
+	if (strncmp(carrier, new_carrier, 1)) {
+		if (link->link_status == ETH_LINK_UP) {
+			RTE_LOG(INFO, KNI, "%s NIC Link is Up %d Mbps %s %s.\n",
+				kni->name,
+				link->link_speed,
+				link->link_autoneg ? "(Fixed)" : "(AutoNeg)",
+				link->link_duplex ?
+					"Full Duplex" : "Half Duplex");
+		} else {
+			RTE_LOG(INFO, KNI, "%s NIC Link is Down.\n",
+				kni->name);
+		}
+	}
+
+	close(fd);
+
+	return 0;
+}
+
 void
 rte_kni_close(void)
 {
diff --git a/lib/librte_kni/rte_kni.h b/lib/librte_kni/rte_kni.h
index 99055e2c2..4118ae97a 100644
--- a/lib/librte_kni/rte_kni.h
+++ b/lib/librte_kni/rte_kni.h
@@ -21,6 +21,7 @@
 #include <rte_memory.h>
 #include <rte_mempool.h>
 #include <rte_ether.h>
+#include <rte_ethdev.h>
 
 #include <exec-env/rte_kni_common.h>
 
@@ -228,6 +229,23 @@ int rte_kni_register_handlers(struct rte_kni *kni, struct rte_kni_ops *ops);
  */
 int rte_kni_unregister_handlers(struct rte_kni *kni);
 
+/**
+ * Update link status info for KNI port.
+ *
+ * Update the linkup/linkdown status of a KNI interface in the kernel.
+ *
+ * @param kni
+ *  pointer to struct rte_kni.
+ * @param link
+ *  pointer to struct rte_eth_link containing new interface status.
+ *
+ * @return
+ *  On success: 0
+ *  On failure: -1
+ */
+int __rte_experimental
+rte_kni_update_link(struct rte_kni *kni, struct rte_eth_link *link);
+
 /**
  *  Close KNI device.
  */
-- 
2.19.0

Re: [dpdk-dev] [PATCH 1/2] kni: add API to set link status on kernel interface

[Ferruh Yigit]

On 9/12/2018 12:29 AM, Dan Gora wrote:
> Add a new API function to KNI, rte_kni_update_link() to allow DPDK
> applications to update the link status for KNI network interfaces in
> the linux kernel.
> 
> Signed-off-by: Dan Gora <dg@adax.com>

+1 to sysfs implementation.

But right now this API is not used at all which makes it hard to test and catch
when API broken.
Can you please implement the API either on kni sample app or kni unit test?

Also you need to add new API to .map file for shared library build. (you would
catch this if API implemented somewhere...)

Re: [dpdk-dev] [PATCH 1/2] kni: add API to set link status on kernel interface

[Dan Gora]

On Tue, Sep 18, 2018 at 1:54 PM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> On 9/12/2018 12:29 AM, Dan Gora wrote:
>> Add a new API function to KNI, rte_kni_update_link() to allow DPDK
>> applications to update the link status for KNI network interfaces in
>> the linux kernel.
>>
>> Signed-off-by: Dan Gora <dg@adax.com>
>
> +1 to sysfs implementation.
>
> But right now this API is not used at all which makes it hard to test and catch
> when API broken.
> Can you please implement the API either on kni sample app or kni unit test?
>
> Also you need to add new API to .map file for shared library build. (you would
> catch this if API implemented somewhere...)

Understood.. I'll work on this ASAP.

-dan

[dpdk-dev] [PATCH 2/2] kni: set default carrier state to 'off'

[Dan Gora]

Set the carrier state to 'off' when the interface is instantiated
or when it is marked 'up' or 'down'.  This is necessary to set the
interface to a known operational state until the carrier state is
changed with rte_kni_update_link().

Signed-off-by: Dan Gora <dg@adax.com>
---
 kernel/linux/kni/kni_misc.c | 2 ++
 kernel/linux/kni/kni_net.c  | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/kernel/linux/kni/kni_misc.c b/kernel/linux/kni/kni_misc.c
index fa69f8e63..45649499d 100644
--- a/kernel/linux/kni/kni_misc.c
+++ b/kernel/linux/kni/kni_misc.c
@@ -466,6 +466,8 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_num,
 		return -ENODEV;
 	}
 
+	netif_carrier_off(net_dev);
+
 	ret = kni_run_thread(knet, kni, dev_info.force_bind);
 	if (ret != 0)
 		return ret;
diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c
index 7fcfa106c..1f8ba0700 100644
--- a/kernel/linux/kni/kni_net.c
+++ b/kernel/linux/kni/kni_net.c
@@ -133,6 +133,7 @@ kni_net_open(struct net_device *dev)
 	struct kni_dev *kni = netdev_priv(dev);
 
 	netif_start_queue(dev);
+	netif_carrier_off(dev);
 
 	memset(&req, 0, sizeof(req));
 	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
@@ -152,6 +153,7 @@ kni_net_release(struct net_device *dev)
 	struct kni_dev *kni = netdev_priv(dev);
 
 	netif_stop_queue(dev); /* can't transmit any more */
+	netif_carrier_off(dev);
 
 	memset(&req, 0, sizeof(req));
 	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
-- 
2.19.0

Re: [dpdk-dev] [PATCH 2/2] kni: set default carrier state to 'off'

[Ferruh Yigit]

On 9/12/2018 12:29 AM, Dan Gora wrote:
> Set the carrier state to 'off' when the interface is instantiated
> or when it is marked 'up' or 'down'.  This is necessary to set the
> interface to a known operational state until the carrier state is
> changed with rte_kni_update_link().

Why setting to no-carrier mode by default? This will change the behavior of
interfaces and may effect others. And indeed I didn't get why this is required?

> 
> Signed-off-by: Dan Gora <dg@adax.com>
> ---
>  kernel/linux/kni/kni_misc.c | 2 ++
>  kernel/linux/kni/kni_net.c  | 2 ++
>  2 files changed, 4 insertions(+)
> 
> diff --git a/kernel/linux/kni/kni_misc.c b/kernel/linux/kni/kni_misc.c
> index fa69f8e63..45649499d 100644
> --- a/kernel/linux/kni/kni_misc.c
> +++ b/kernel/linux/kni/kni_misc.c
> @@ -466,6 +466,8 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_num,
>  		return -ENODEV;
>  	}
>  
> +	netif_carrier_off(net_dev);
> +
>  	ret = kni_run_thread(knet, kni, dev_info.force_bind);
>  	if (ret != 0)
>  		return ret;
> diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c
> index 7fcfa106c..1f8ba0700 100644
> --- a/kernel/linux/kni/kni_net.c
> +++ b/kernel/linux/kni/kni_net.c
> @@ -133,6 +133,7 @@ kni_net_open(struct net_device *dev)
>  	struct kni_dev *kni = netdev_priv(dev);
>  
>  	netif_start_queue(dev);
> +	netif_carrier_off(dev);
>  
>  	memset(&req, 0, sizeof(req));
>  	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
> @@ -152,6 +153,7 @@ kni_net_release(struct net_device *dev)
>  	struct kni_dev *kni = netdev_priv(dev);
>  
>  	netif_stop_queue(dev); /* can't transmit any more */
> +	netif_carrier_off(dev);
>  
>  	memset(&req, 0, sizeof(req));
>  	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
> 

Re: [dpdk-dev] [PATCH 2/2] kni: set default carrier state to 'off'

[Ferruh Yigit]

On 9/18/2018 5:15 PM, Ferruh Yigit wrote:
> On 9/12/2018 12:29 AM, Dan Gora wrote:
>> Set the carrier state to 'off' when the interface is instantiated
>> or when it is marked 'up' or 'down'.  This is necessary to set the
>> interface to a known operational state until the carrier state is
>> changed with rte_kni_update_link().
> 
> Why setting to no-carrier mode by default? This will change the behavior of
> interfaces and may effect others. And indeed I didn't get why this is required?

I just read the other thread, including Igor's and your comment about starting
the interface down, overall I got your point but my concerns is if someone has a
solution based on assumption that interface will be up when created will be
affected.

> 
>>
>> Signed-off-by: Dan Gora <dg@adax.com>
>> ---
>>  kernel/linux/kni/kni_misc.c | 2 ++
>>  kernel/linux/kni/kni_net.c  | 2 ++
>>  2 files changed, 4 insertions(+)
>>
>> diff --git a/kernel/linux/kni/kni_misc.c b/kernel/linux/kni/kni_misc.c
>> index fa69f8e63..45649499d 100644
>> --- a/kernel/linux/kni/kni_misc.c
>> +++ b/kernel/linux/kni/kni_misc.c
>> @@ -466,6 +466,8 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_num,
>>  		return -ENODEV;
>>  	}
>>  
>> +	netif_carrier_off(net_dev);
>> +
>>  	ret = kni_run_thread(knet, kni, dev_info.force_bind);
>>  	if (ret != 0)
>>  		return ret;
>> diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c
>> index 7fcfa106c..1f8ba0700 100644
>> --- a/kernel/linux/kni/kni_net.c
>> +++ b/kernel/linux/kni/kni_net.c
>> @@ -133,6 +133,7 @@ kni_net_open(struct net_device *dev)
>>  	struct kni_dev *kni = netdev_priv(dev);
>>  
>>  	netif_start_queue(dev);
>> +	netif_carrier_off(dev);
>>  
>>  	memset(&req, 0, sizeof(req));
>>  	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
>> @@ -152,6 +153,7 @@ kni_net_release(struct net_device *dev)
>>  	struct kni_dev *kni = netdev_priv(dev);
>>  
>>  	netif_stop_queue(dev); /* can't transmit any more */
>> +	netif_carrier_off(dev);
>>  
>>  	memset(&req, 0, sizeof(req));
>>  	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
>>
> 

Re: [dpdk-dev] [PATCH 2/2] kni: set default carrier state to 'off'

[Dan Gora]

On Tue, Sep 18, 2018 at 1:48 PM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> I just read the other thread, including Igor's and your comment about starting
> the interface down, overall I got your point but my concerns is if someone has a
> solution based on assumption that interface will be up when created will be
> affected.
>

The carrier state of the interface does not come up in the "up" state
currently.  It comes up either in the "down" state or in the "unknown"
state, which is really the worst of all worlds..

RHEL/Centos 7.5 running 'test->kni_autotest'

6: test_kni_port: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc
pfifo_fast state UNKNOWN group default qlen 1000
   link/ether ca:40:e8:72:68:29 brd ff:ff:ff:ff:ff:ff
   inet6 fe80::691c:755e:5257:83af/64 scope link noprefixroute
      valid_lft forever preferred_lft forever

-dan

[dpdk-dev] [PATCH v2 0/5] kni: add API to set link status on kernel interface

[Dan Gora]

Hi All,

Attached is version 2 of a patchset to add a new API function to
set the link status on kernel interfaces created with the KNI kernel
module.

-dan

v2
====

* Fix bug where "Fixed" and "AutoNeg" were transposed in the link
  status log message.

* Add rte_kni_update_link() to rte_kni_version.map

* Add rte_kni_update_link() tests to kni_autotest

* Update examples/kni to continuously monitor link status and
  update the corresponding kernel interface with
  rte_kni_update_link().

* Minor improvements to examples/kni: Add log message showing how
  to show/zero stats.  Improve zeroing statistics.

Note that checkpatches.sh compains about patch 1/5, but this appears
to be a bug with check-symbol-change or something.  If I move the
fragment of the patch modifying rte_kni_version.map to the bottom of
the patch file, it doesn't complain any more...  I just don't really
have time to investigate this right now.

thanks
dan

> Hi All,
> 
> The following patches are to add support for DPDK applications to be
> able to change the carrier state of Linux network interfaces in the
> KNI kernel module.
> 
> The carrier state is changed by writing to the Linux /sys file:
> /sys/devices/virtual/net/<name>/carrier, where <name> is the KNI
> interface name.
> 
> These patches supercede:
> '[PATCH v2 10/10] kni: add API to set link status on kernel interface'
> https://mails.dpdk.org/archives/dev/2018-August/110383.html


Dan Gora (5):
  kni: add API to set link status on kernel interface
  kni: set default carrier state to 'off'
  examples/kni: monitor and update link status continually
  examples/kni: add log msgs to show and clear stats
  examples/kni: improve zeroing statistics

 examples/kni/Makefile              |   2 +
 examples/kni/main.c                | 104 ++++++++++++++++-------------
 kernel/linux/kni/kni_misc.c        |   2 +
 kernel/linux/kni/kni_net.c         |   2 +
 lib/librte_kni/rte_kni.c           |  57 ++++++++++++++++
 lib/librte_kni/rte_kni.h           |  18 +++++
 lib/librte_kni/rte_kni_version.map |   6 ++
 test/test/test_kni.c               | 100 ++++++++++++++++++++++++++-
 8 files changed, 244 insertions(+), 47 deletions(-)

-- 
2.19.0

[dpdk-dev] [PATCH v2 1/5] kni: add API to set link status on kernel interface

[Dan Gora]

Add a new API function to KNI, rte_kni_update_link() to allow DPDK
applications to update the link status for KNI network interfaces in
the linux kernel.

Signed-off-by: Dan Gora <dg@adax.com>
---
 lib/librte_kni/rte_kni.c           |  57 ++++++++++++++++
 lib/librte_kni/rte_kni.h           |  18 ++++++
 lib/librte_kni/rte_kni_version.map |   6 ++
 test/test/test_kni.c               | 100 ++++++++++++++++++++++++++++-
 4 files changed, 180 insertions(+), 1 deletion(-)

diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c
index 65f6a2b03..0c3e17dbb 100644
--- a/lib/librte_kni/rte_kni.c
+++ b/lib/librte_kni/rte_kni.c
@@ -790,6 +790,63 @@ rte_kni_unregister_handlers(struct rte_kni *kni)
 
 	return 0;
 }
+
+int __rte_experimental
+rte_kni_update_link(struct rte_kni *kni, struct rte_eth_link *link)
+{
+	char path[64];
+	char carrier[2];
+	const char *new_carrier;
+	int fd, ret;
+
+	if (kni == NULL || link == NULL)
+		return -1;
+
+	snprintf(path, sizeof(path), "/sys/devices/virtual/net/%s/carrier",
+		kni->name);
+
+	fd = open(path, O_RDWR);
+	if (fd == -1) {
+		RTE_LOG(ERR, KNI, "Failed to open file: %s.\n", path);
+		return -1;
+	}
+
+	ret = read(fd, carrier, 2);
+	if (ret < 1) {
+		/* Cannot read carrier until interface is marked
+		 * 'up', so don't log an error.
+		 */
+		close(fd);
+		return -1;
+	}
+
+	new_carrier = (link->link_status == ETH_LINK_UP) ? "1" : "0";
+	ret = write(fd, new_carrier, 1);
+	if (ret < 1) {
+		RTE_LOG(ERR, KNI, "Failed to write file: %s.\n", path);
+		close(fd);
+		return -1;
+	}
+
+	if (strncmp(carrier, new_carrier, 1)) {
+		if (link->link_status == ETH_LINK_UP) {
+			RTE_LOG(INFO, KNI, "%s NIC Link is Up %d Mbps %s %s.\n",
+				kni->name,
+				link->link_speed,
+				link->link_autoneg ? "(AutoNeg)" : "(Fixed)",
+				link->link_duplex ?
+					"Full Duplex" : "Half Duplex");
+		} else {
+			RTE_LOG(INFO, KNI, "%s NIC Link is Down.\n",
+				kni->name);
+		}
+	}
+
+	close(fd);
+
+	return 0;
+}
+
 void
 rte_kni_close(void)
 {
diff --git a/lib/librte_kni/rte_kni.h b/lib/librte_kni/rte_kni.h
index 99055e2c2..4118ae97a 100644
--- a/lib/librte_kni/rte_kni.h
+++ b/lib/librte_kni/rte_kni.h
@@ -21,6 +21,7 @@
 #include <rte_memory.h>
 #include <rte_mempool.h>
 #include <rte_ether.h>
+#include <rte_ethdev.h>
 
 #include <exec-env/rte_kni_common.h>
 
@@ -228,6 +229,23 @@ int rte_kni_register_handlers(struct rte_kni *kni, struct rte_kni_ops *ops);
  */
 int rte_kni_unregister_handlers(struct rte_kni *kni);
 
+/**
+ * Update link status info for KNI port.
+ *
+ * Update the linkup/linkdown status of a KNI interface in the kernel.
+ *
+ * @param kni
+ *  pointer to struct rte_kni.
+ * @param link
+ *  pointer to struct rte_eth_link containing new interface status.
+ *
+ * @return
+ *  On success: 0
+ *  On failure: -1
+ */
+int __rte_experimental
+rte_kni_update_link(struct rte_kni *kni, struct rte_eth_link *link);
+
 /**
  *  Close KNI device.
  */
diff --git a/lib/librte_kni/rte_kni_version.map b/lib/librte_kni/rte_kni_version.map
index acd515eb0..c877dc6aa 100644
--- a/lib/librte_kni/rte_kni_version.map
+++ b/lib/librte_kni/rte_kni_version.map
@@ -15,3 +15,9 @@ DPDK_2.0 {
 
 	local: *;
 };
+
+EXPERIMENTAL {
+	global:
+
+	rte_kni_update_link;
+};
diff --git a/test/test/test_kni.c b/test/test/test_kni.c
index 3dcadcebd..d450cc342 100644
--- a/test/test/test_kni.c
+++ b/test/test/test_kni.c
@@ -118,6 +118,97 @@ kni_change_mtu(uint16_t port_id, unsigned int new_mtu)
 					 port_id, kni_pkt_mtu);
 	return 0;
 }
+
+static int
+kni_change_link(void)
+{
+	struct rte_eth_link link;
+	int ret;
+
+	link.link_speed = 10;
+	link.link_status = ETH_LINK_UP;
+	link.link_autoneg = ETH_LINK_AUTONEG;
+	link.link_duplex = ETH_LINK_FULL_DUPLEX;
+	ret = rte_kni_update_link(test_kni_ctx, &link);
+	if (ret != 0) {
+		printf("Failed to change link state to "
+				"Up/10Mbps/AutoNeg/Full ret=%d.\n", ret);
+		return -1;
+	}
+	rte_delay_ms(1000);
+
+	link.link_speed = 0;
+	link.link_status = ETH_LINK_DOWN;
+	link.link_autoneg = ETH_LINK_FIXED;
+	link.link_duplex = ETH_LINK_FULL_DUPLEX;
+	ret = rte_kni_update_link(test_kni_ctx, &link);
+	if (ret != 0) {
+		printf("Failed to change link state to Down ret=%d.\n", ret);
+		return -1;
+	}
+	rte_delay_ms(1000);
+
+	link.link_speed = 1000;
+	link.link_status = ETH_LINK_UP;
+	link.link_autoneg = ETH_LINK_AUTONEG;
+	link.link_duplex = ETH_LINK_HALF_DUPLEX;
+	ret = rte_kni_update_link(test_kni_ctx, &link);
+	if (ret != 0) {
+		printf("Failed to change link state to "
+				"Up/1Gbps/AutoNeg/Half ret=%d.\n", ret);
+		return -1;
+	}
+	rte_delay_ms(1000);
+
+	link.link_speed = 0;
+	link.link_status = ETH_LINK_DOWN;
+	link.link_autoneg = ETH_LINK_FIXED;
+	link.link_duplex = ETH_LINK_FULL_DUPLEX;
+	ret = rte_kni_update_link(test_kni_ctx, &link);
+	if (ret != 0) {
+		printf("Failed to change link state to Down ret=%d.\n", ret);
+		return -1;
+	}
+	rte_delay_ms(1000);
+
+	link.link_speed = 40000;
+	link.link_status = ETH_LINK_UP;
+	link.link_autoneg = ETH_LINK_FIXED;
+	link.link_duplex = ETH_LINK_FULL_DUPLEX;
+	ret = rte_kni_update_link(test_kni_ctx, &link);
+	if (ret != 0) {
+		printf("Failed to change link state to "
+				"Up/40Gbps/Fixed/Full ret=%d.\n", ret);
+		return -1;
+	}
+	rte_delay_ms(1000);
+
+	link.link_speed = 0;
+	link.link_status = ETH_LINK_DOWN;
+	link.link_autoneg = ETH_LINK_FIXED;
+	link.link_duplex = ETH_LINK_FULL_DUPLEX;
+	ret = rte_kni_update_link(test_kni_ctx, &link);
+	if (ret != 0) {
+		printf("Failed to change link state to Down ret=%d.\n", ret);
+		return -1;
+	}
+	rte_delay_ms(1000);
+
+	link.link_speed = 100000;
+	link.link_status = ETH_LINK_UP;
+	link.link_autoneg = ETH_LINK_FIXED;
+	link.link_duplex = ETH_LINK_FULL_DUPLEX;
+	ret = rte_kni_update_link(test_kni_ctx, &link);
+	if (ret != 0) {
+		printf("Failed to change link state to "
+				"Up/100Gbps/Fixed/Full ret=%d.\n", ret);
+		return -1;
+	}
+	rte_delay_ms(1000);
+
+	return 0;
+}
+
 /**
  * This loop fully tests the basic functions of KNI. e.g. transmitting,
  * receiving to, from kernel space, and kernel requests.
@@ -148,9 +239,16 @@ test_kni_loop(__rte_unused void *arg)
 			ret = -1;
 		if (system(IFCONFIG TEST_KNI_PORT" mtu 1400") == -1)
 			ret = -1;
+
+		ret = kni_change_link();
+		if (ret != 0) {
+			test_kni_processing_flag = 1;
+			return ret;
+		}
+		rte_delay_ms(KNI_TIMEOUT_MS);
 		if (system(IFCONFIG TEST_KNI_PORT" down") == -1)
 			ret = -1;
-		rte_delay_ms(KNI_TIMEOUT_MS);
+		rte_delay_ms(1000);
 		test_kni_processing_flag = 1;
 	} else if (lcore_id == lcore_ingress) {
 		struct rte_mempool *mp = test_kni_lookup_mempool();
-- 
2.19.0

Re: [dpdk-dev] [PATCH v2 1/5] kni: add API to set link status on kernel interface

[Ferruh Yigit]

On 9/19/2018 8:55 PM, Dan Gora wrote:
> Add a new API function to KNI, rte_kni_update_link() to allow DPDK
> applications to update the link status for KNI network interfaces in
> the linux kernel.
> 
> Signed-off-by: Dan Gora <dg@adax.com>

<...>

> +int __rte_experimental
> +rte_kni_update_link(struct rte_kni *kni, struct rte_eth_link *link)
> +{
> +	char path[64];
> +	char carrier[2];
> +	const char *new_carrier;
> +	int fd, ret;
> +
> +	if (kni == NULL || link == NULL)
> +		return -1;
> +
> +	snprintf(path, sizeof(path), "/sys/devices/virtual/net/%s/carrier",
> +		kni->name);
> +
> +	fd = open(path, O_RDWR);
> +	if (fd == -1) {
> +		RTE_LOG(ERR, KNI, "Failed to open file: %s.\n", path);
> +		return -1;
> +	}
> +
> +	ret = read(fd, carrier, 2);
> +	if (ret < 1) {
> +		/* Cannot read carrier until interface is marked
> +		 * 'up', so don't log an error.
> +		 */
> +		close(fd);
> +		return -1;
> +	}
> +
> +	new_carrier = (link->link_status == ETH_LINK_UP) ? "1" : "0";
> +	ret = write(fd, new_carrier, 1);
> +	if (ret < 1) {
> +		RTE_LOG(ERR, KNI, "Failed to write file: %s.\n", path);
> +		close(fd);
> +		return -1;
> +	}
> +
> +	if (strncmp(carrier, new_carrier, 1)) {
> +		if (link->link_status == ETH_LINK_UP) {
> +			RTE_LOG(INFO, KNI, "%s NIC Link is Up %d Mbps %s %s.\n",
> +				kni->name,
> +				link->link_speed,
> +				link->link_autoneg ? "(AutoNeg)" : "(Fixed)",
> +				link->link_duplex ?
> +					"Full Duplex" : "Half Duplex");

These link values are coming from user and not reflected to the kni interface,
printing them here can cause a misunderstanding that they have been applied.
I think only link status should be printed here.

<...>

> @@ -148,9 +239,16 @@ test_kni_loop(__rte_unused void *arg)
>  			ret = -1;
>  		if (system(IFCONFIG TEST_KNI_PORT" mtu 1400") == -1)
>  			ret = -1;
> +
> +		ret = kni_change_link();
> +		if (ret != 0) {
> +			test_kni_processing_flag = 1;
> +			return ret;
> +		}

I thinks this is wrong place to call kni_change_link(), this is test_kni_loop()
created by test_kni_processing() that does packet processing tests.

I think better to call directly from test_kni(), perhaps before
test_kni_processing() call?

Re: [dpdk-dev] [PATCH v2 1/5] kni: add API to set link status on kernel interface

[Dan Gora]

On Wed, Sep 26, 2018 at 10:59 AM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> On 9/19/2018 8:55 PM, Dan Gora wrote:
>> Add a new API function to KNI, rte_kni_update_link() to allow DPDK
>> applications to update the link status for KNI network interfaces in
>> the linux kernel.
>>
>> Signed-off-by: Dan Gora <dg@adax.com>
>
> <...>
>
>> +int __rte_experimental
>> +rte_kni_update_link(struct rte_kni *kni, struct rte_eth_link *link)
>> +{
>> +     char path[64];
>> +     char carrier[2];
>> +     const char *new_carrier;
>> +     int fd, ret;
>> +
>> +     if (kni == NULL || link == NULL)
>> +             return -1;
>> +
>> +     snprintf(path, sizeof(path), "/sys/devices/virtual/net/%s/carrier",
>> +             kni->name);
>> +
>> +     fd = open(path, O_RDWR);
>> +     if (fd == -1) {
>> +             RTE_LOG(ERR, KNI, "Failed to open file: %s.\n", path);
>> +             return -1;
>> +     }
>> +
>> +     ret = read(fd, carrier, 2);
>> +     if (ret < 1) {
>> +             /* Cannot read carrier until interface is marked
>> +              * 'up', so don't log an error.
>> +              */
>> +             close(fd);
>> +             return -1;
>> +     }
>> +
>> +     new_carrier = (link->link_status == ETH_LINK_UP) ? "1" : "0";
>> +     ret = write(fd, new_carrier, 1);
>> +     if (ret < 1) {
>> +             RTE_LOG(ERR, KNI, "Failed to write file: %s.\n", path);
>> +             close(fd);
>> +             return -1;
>> +     }
>> +
>> +     if (strncmp(carrier, new_carrier, 1)) {
>> +             if (link->link_status == ETH_LINK_UP) {
>> +                     RTE_LOG(INFO, KNI, "%s NIC Link is Up %d Mbps %s %s.\n",
>> +                             kni->name,
>> +                             link->link_speed,
>> +                             link->link_autoneg ? "(AutoNeg)" : "(Fixed)",
>> +                             link->link_duplex ?
>> +                                     "Full Duplex" : "Half Duplex");
>
> These link values are coming from user and not reflected to the kni interface,
> printing them here can cause a misunderstanding that they have been applied.
> I think only link status should be printed here.
>

There is nothing to "reflect" to the kernel interface, nor to apply to
the kernel interface.  This is exactly how every other kernel driver
works on link status changes.  There is no "netif_set_speed()'
function.  When a link status change occurs the kernel driver calls
netif_carrier_on/off() and prints a message like this one.

> <...>
>
>> @@ -148,9 +239,16 @@ test_kni_loop(__rte_unused void *arg)
>>                       ret = -1;
>>               if (system(IFCONFIG TEST_KNI_PORT" mtu 1400") == -1)
>>                       ret = -1;
>> +
>> +             ret = kni_change_link();
>> +             if (ret != 0) {
>> +                     test_kni_processing_flag = 1;
>> +                     return ret;
>> +             }
>
> I thinks this is wrong place to call kni_change_link(), this is test_kni_loop()
> created by test_kni_processing() that does packet processing tests.
>

Well, no it's not "wrong".  The interface has to be up to change the
link state, so this is a convenient place to do it.

> I think better to call directly from test_kni(), perhaps before
> test_kni_processing() call?

Because then we'd have to add code to set the interface up and down
again, and there really is no point.  This is as good a place as any.
It does not affect the data transfer tests at all.

-d

Re: [dpdk-dev] [PATCH v2 1/5] kni: add API to set link status on kernel interface

[Ferruh Yigit]

On 9/26/2018 3:55 PM, Dan Gora wrote:
> On Wed, Sep 26, 2018 at 10:59 AM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>> On 9/19/2018 8:55 PM, Dan Gora wrote:
>>> Add a new API function to KNI, rte_kni_update_link() to allow DPDK
>>> applications to update the link status for KNI network interfaces in
>>> the linux kernel.
>>>
>>> Signed-off-by: Dan Gora <dg@adax.com>
>>
>> <...>
>>
>>> +int __rte_experimental
>>> +rte_kni_update_link(struct rte_kni *kni, struct rte_eth_link *link)
>>> +{
>>> +     char path[64];
>>> +     char carrier[2];
>>> +     const char *new_carrier;
>>> +     int fd, ret;
>>> +
>>> +     if (kni == NULL || link == NULL)
>>> +             return -1;
>>> +
>>> +     snprintf(path, sizeof(path), "/sys/devices/virtual/net/%s/carrier",
>>> +             kni->name);
>>> +
>>> +     fd = open(path, O_RDWR);
>>> +     if (fd == -1) {
>>> +             RTE_LOG(ERR, KNI, "Failed to open file: %s.\n", path);
>>> +             return -1;
>>> +     }
>>> +
>>> +     ret = read(fd, carrier, 2);
>>> +     if (ret < 1) {
>>> +             /* Cannot read carrier until interface is marked
>>> +              * 'up', so don't log an error.
>>> +              */
>>> +             close(fd);
>>> +             return -1;
>>> +     }
>>> +
>>> +     new_carrier = (link->link_status == ETH_LINK_UP) ? "1" : "0";
>>> +     ret = write(fd, new_carrier, 1);
>>> +     if (ret < 1) {
>>> +             RTE_LOG(ERR, KNI, "Failed to write file: %s.\n", path);
>>> +             close(fd);
>>> +             return -1;
>>> +     }
>>> +
>>> +     if (strncmp(carrier, new_carrier, 1)) {
>>> +             if (link->link_status == ETH_LINK_UP) {
>>> +                     RTE_LOG(INFO, KNI, "%s NIC Link is Up %d Mbps %s %s.\n",
>>> +                             kni->name,
>>> +                             link->link_speed,
>>> +                             link->link_autoneg ? "(AutoNeg)" : "(Fixed)",
>>> +                             link->link_duplex ?
>>> +                                     "Full Duplex" : "Half Duplex");
>>
>> These link values are coming from user and not reflected to the kni interface,
>> printing them here can cause a misunderstanding that they have been applied.
>> I think only link status should be printed here.
>>
> 
> There is nothing to "reflect" to the kernel interface, nor to apply to
> the kernel interface.  This is exactly how every other kernel driver
> works on link status changes.  There is no "netif_set_speed()'
> function.  When a link status change occurs the kernel driver calls
> netif_carrier_on/off() and prints a message like this one.

I am not suggesting reflecting these into interface, I am just saying why do you
print them?

For example, "link->link_speed" this is coming as parameter to API, this API
does nothing with this value, why print is here?

I assume you are using "rte_eth_link" as parameter, instead of basic
"link_status" to make it extendible in the feature. If so please print those
other values when function extended, right now only link_status matters.

> 
>> <...>
>>
>>> @@ -148,9 +239,16 @@ test_kni_loop(__rte_unused void *arg)
>>>                       ret = -1;
>>>               if (system(IFCONFIG TEST_KNI_PORT" mtu 1400") == -1)
>>>                       ret = -1;
>>> +
>>> +             ret = kni_change_link();
>>> +             if (ret != 0) {
>>> +                     test_kni_processing_flag = 1;
>>> +                     return ret;
>>> +             }
>>
>> I thinks this is wrong place to call kni_change_link(), this is test_kni_loop()
>> created by test_kni_processing() that does packet processing tests.
>>
> 
> Well, no it's not "wrong".  The interface has to be up to change the
> link state, so this is a convenient place to do it.

Are we agree that this function in unit test is to test packet processing part
of KNI?
And for example please check following coming patch:
https://patches.dpdk.org/patch/44730/

I think you want to benefit from "system(IFCONFIG TEST_KNI_PORT" up")" since
interface needs to be up to set the link, but you can do same call, not have to
use that one.

> 
>> I think better to call directly from test_kni(), perhaps before
>> test_kni_processing() call?
> 
> Because then we'd have to add code to set the interface up and down
> again, and there really is no point.  This is as good a place as any.
> It does not affect the data transfer tests at all.

Doesn't affect the data transfer, agreed, but why confusing data transfer test
with link up/down calls? Why not have your function and set interface up and
down again?

> 
> -d
> 

Re: [dpdk-dev] [PATCH v2 1/5] kni: add API to set link status on kernel interface

[Dan Gora]

On Wed, Sep 26, 2018 at 1:42 PM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>> There is nothing to "reflect" to the kernel interface, nor to apply to
>> the kernel interface.  This is exactly how every other kernel driver
>> works on link status changes.  There is no "netif_set_speed()'
>> function.  When a link status change occurs the kernel driver calls
>> netif_carrier_on/off() and prints a message like this one.
>
> I am not suggesting reflecting these into interface, I am just saying why do you
> print them?

Because the information is useful and because every other Ethernet
driver does the same thing when the link status changes.

See the Linux kernel:

ixgbe_watchdog_link_is_up(): drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
print_link_info(): drivers/net/ethernet/cavium/liquidio/lio_main.c
bnx2_report_link(): drivers/net/ethernet/broadcom/bnx2.c

and dozens of other examples.

Imagine you plug your 1G ethernet cable into a 10/100Mbps hub.  Yes
the link will be up, but it's the wrong speed.
Imagine you had accidentally forced your 1G connection to 10Mbps
fixed.  It will only come up at 10Mbps.  Wouldn't you like to know
that autoNeg was disabled?

> For example, "link->link_speed" this is coming as parameter to API, this API
> does nothing with this value, why print is here?

Because wouldn't you like to know if your link has connected at the
correct speed?  I would.

> I assume you are using "rte_eth_link" as parameter, instead of basic
> "link_status" to make it extendible in the feature. If so please print those
> other values when function extended, right now only link_status matters.

No, all that information matters.  If you have autoNeg disabled or are
connecting to a broken piece of equipment, the link can come up but be
at the wrong speed.

There is not much to extend this function with.  The only action we
can take, other than to print the information, is to write to the /sys
file to have the KNI kernel module call netif_carrier_on/off() for us.

>>>> @@ -148,9 +239,16 @@ test_kni_loop(__rte_unused void *arg)
>>>>                       ret = -1;
>>>>               if (system(IFCONFIG TEST_KNI_PORT" mtu 1400") == -1)
>>>>                       ret = -1;
>>>> +
>>>> +             ret = kni_change_link();
>>>> +             if (ret != 0) {
>>>> +                     test_kni_processing_flag = 1;
>>>> +                     return ret;
>>>> +             }
>>>
>>> I thinks this is wrong place to call kni_change_link(), this is test_kni_loop()
>>> created by test_kni_processing() that does packet processing tests.
>>>
>>
>> Well, no it's not "wrong".  The interface has to be up to change the
>> link state, so this is a convenient place to do it.
>
> Are we agree that this function in unit test is to test packet processing part
> of KNI?
> And for example please check following coming patch:
> https://patches.dpdk.org/patch/44730/
>
> I think you want to benefit from "system(IFCONFIG TEST_KNI_PORT" up")" since
> interface needs to be up to set the link, but you can do same call, not have to
> use that one.

ok, I'll fix this.  Should I rebase the changes on top of this patch
(https://patches.dpdk.org/patch/44730/)?  Is that patch going to get
accepted soon?

>> Because then we'd have to add code to set the interface up and down
>> again, and there really is no point.  This is as good a place as any.
>> It does not affect the data transfer tests at all.
>
> Doesn't affect the data transfer, agreed, but why confusing data transfer test
> with link up/down calls? Why not have your function and set interface up and
> down again?

ok.  I was just trying to minimize the number of changes. I'll send a new patch.

Re: [dpdk-dev] [PATCH v2 1/5] kni: add API to set link status on kernel interface

[Ferruh Yigit]

On 9/26/2018 7:56 PM, Dan Gora wrote:
> On Wed, Sep 26, 2018 at 1:42 PM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>> There is nothing to "reflect" to the kernel interface, nor to apply to
>>> the kernel interface.  This is exactly how every other kernel driver
>>> works on link status changes.  There is no "netif_set_speed()'
>>> function.  When a link status change occurs the kernel driver calls
>>> netif_carrier_on/off() and prints a message like this one.
>>
>> I am not suggesting reflecting these into interface, I am just saying why do you
>> print them?
> 
> Because the information is useful and because every other Ethernet
> driver does the same thing when the link status changes.

It would be useful if it writes the values of virtual interface, but this API
prints user input.

The virtual interface may have different value, this API doesn't change anything
related other than link status, so why print user provided value.

Won't you think it will be confusing if the virtual interface values are
different than what printed.
Or won't user will think API changed those values to printed one in interface?

> 
> See the Linux kernel:
> 
> ixgbe_watchdog_link_is_up(): drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> print_link_info(): drivers/net/ethernet/cavium/liquidio/lio_main.c
> bnx2_report_link(): drivers/net/ethernet/broadcom/bnx2.c
> 
> and dozens of other examples.
> 
> Imagine you plug your 1G ethernet cable into a 10/100Mbps hub.  Yes
> the link will be up, but it's the wrong speed.
> Imagine you had accidentally forced your 1G connection to 10Mbps
> fixed.  It will only come up at 10Mbps.  Wouldn't you like to know
> that autoNeg was disabled?
> 
>> For example, "link->link_speed" this is coming as parameter to API, this API
>> does nothing with this value, why print is here?
> 
> Because wouldn't you like to know if your link has connected at the
> correct speed?  I would.
> 
>> I assume you are using "rte_eth_link" as parameter, instead of basic
>> "link_status" to make it extendible in the feature. If so please print those
>> other values when function extended, right now only link_status matters.
> 
> No, all that information matters.  If you have autoNeg disabled or are
> connecting to a broken piece of equipment, the link can come up but be
> at the wrong speed.
> 
> There is not much to extend this function with.  The only action we
> can take, other than to print the information, is to write to the /sys
> file to have the KNI kernel module call netif_carrier_on/off() for us.
> 
>>>>> @@ -148,9 +239,16 @@ test_kni_loop(__rte_unused void *arg)
>>>>>                       ret = -1;
>>>>>               if (system(IFCONFIG TEST_KNI_PORT" mtu 1400") == -1)
>>>>>                       ret = -1;
>>>>> +
>>>>> +             ret = kni_change_link();
>>>>> +             if (ret != 0) {
>>>>> +                     test_kni_processing_flag = 1;
>>>>> +                     return ret;
>>>>> +             }
>>>>
>>>> I thinks this is wrong place to call kni_change_link(), this is test_kni_loop()
>>>> created by test_kni_processing() that does packet processing tests.
>>>>
>>>
>>> Well, no it's not "wrong".  The interface has to be up to change the
>>> link state, so this is a convenient place to do it.
>>
>> Are we agree that this function in unit test is to test packet processing part
>> of KNI?
>> And for example please check following coming patch:
>> https://patches.dpdk.org/patch/44730/
>>
>> I think you want to benefit from "system(IFCONFIG TEST_KNI_PORT" up")" since
>> interface needs to be up to set the link, but you can do same call, not have to
>> use that one.
> 
> ok, I'll fix this.  Should I rebase the changes on top of this patch
> (https://patches.dpdk.org/patch/44730/)?  Is that patch going to get
> accepted soon?
> 
>>> Because then we'd have to add code to set the interface up and down
>>> again, and there really is no point.  This is as good a place as any.
>>> It does not affect the data transfer tests at all.
>>
>> Doesn't affect the data transfer, agreed, but why confusing data transfer test
>> with link up/down calls? Why not have your function and set interface up and
>> down again?
> 
> ok.  I was just trying to minimize the number of changes. I'll send a new patch.
> 

Re: [dpdk-dev] [PATCH v2 1/5] kni: add API to set link status on kernel interface

[Dan Gora]

On Thu, Sep 27, 2018 at 8:35 AM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>
> On 9/26/2018 7:56 PM, Dan Gora wrote:
> > On Wed, Sep 26, 2018 at 1:42 PM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> >>> There is nothing to "reflect" to the kernel interface, nor to apply to
> >>> the kernel interface.  This is exactly how every other kernel driver
> >>> works on link status changes.  There is no "netif_set_speed()'
> >>> function.  When a link status change occurs the kernel driver calls
> >>> netif_carrier_on/off() and prints a message like this one.
> >>
> >> I am not suggesting reflecting these into interface, I am just saying why do you
> >> print them?
> >
> > Because the information is useful and because every other Ethernet
> > driver does the same thing when the link status changes.
>
> It would be useful if it writes the values of virtual interface, but this API
> prints user input.
>
I'm sorry, Ferruh, I really don't understand what you are referring to
here.  What virtual interface are you talking about?

> The virtual interface may have different value, this API doesn't change anything
> related other than link status, so why print user provided value.

Again, because all this information is useful if the KNI interface is
reflecting the state of a real physical ethenet interface.  If the KNI
interface is not reflecting a real peice of hardware, the user can
continue not using this new API function or do something like set the
speed to 0.  This is what we get from ENA PMD drivers on amazon for
instance.  It's perfectly reasonable.

>
> Won't you think it will be confusing if the virtual interface values are
> different than what printed.

If what values are different from what?  I'm really confused as to
what use case you are referring to here.  Can you give a concrete
example?

> Or won't user will think API changed those values to printed one in interface?

again, what values?  Changed how?

Re: [dpdk-dev] [PATCH v2 1/5] kni: add API to set link status on kernel interface

[Ferruh Yigit]

On 9/27/2018 4:40 PM, Dan Gora wrote:
> On Thu, Sep 27, 2018 at 8:35 AM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>
>> On 9/26/2018 7:56 PM, Dan Gora wrote:
>>> On Wed, Sep 26, 2018 at 1:42 PM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>>>> There is nothing to "reflect" to the kernel interface, nor to apply to
>>>>> the kernel interface.  This is exactly how every other kernel driver
>>>>> works on link status changes.  There is no "netif_set_speed()'
>>>>> function.  When a link status change occurs the kernel driver calls
>>>>> netif_carrier_on/off() and prints a message like this one.
>>>>
>>>> I am not suggesting reflecting these into interface, I am just saying why do you
>>>> print them?
>>>
>>> Because the information is useful and because every other Ethernet
>>> driver does the same thing when the link status changes.
>>
>> It would be useful if it writes the values of virtual interface, but this API
>> prints user input.
>>
> I'm sorry, Ferruh, I really don't understand what you are referring to
> here.  What virtual interface are you talking about?

Virtual interface is KNI interface, Linux virtual network interface.


Let me try again,

You are adding a new API to KNI library, which is to set link status of KNI
interface.

This API prints some link related values in its log. But these values are not
applied to KNI interface or not the values coming from KNI interface. These are
just values provided by user to the API, I am saying printing these values in
log can be confusing, the user may think API applies these values to KNI interface.


> 
>> The virtual interface may have different value, this API doesn't change anything
>> related other than link status, so why print user provided value.
> 
> Again, because all this information is useful if the KNI interface is
> reflecting the state of a real physical ethenet interface.  If the KNI
> interface is not reflecting a real peice of hardware, the user can
> continue not using this new API function or do something like set the
> speed to 0.  This is what we get from ENA PMD drivers on amazon for
> instance.  It's perfectly reasonable.
> 
>>
>> Won't you think it will be confusing if the virtual interface values are
>> different than what printed.
> 
> If what values are different from what?  I'm really confused as to
> what use case you are referring to here.  Can you give a concrete
> example?
> 
>> Or won't user will think API changed those values to printed one in interface?
> 
> again, what values?  Changed how?
> 

Re: [dpdk-dev] [PATCH v2 1/5] kni: add API to set link status on kernel interface

[Dan Gora]

On Thu, Sep 27, 2018 at 6:49 PM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>> It would be useful if it writes the values of virtual interface, but this API
>>> prints user input.
>>>
>> I'm sorry, Ferruh, I really don't understand what you are referring to
>> here.  What virtual interface are you talking about?
>
> Virtual interface is KNI interface, Linux virtual network interface.
>
>
> Let me try again,
>
> You are adding a new API to KNI library, which is to set link status of KNI
> interface.
>
> This API prints some link related values in its log. But these values are not
> applied to KNI interface or not the values coming from KNI interface. These are
> just values provided by user to the API, I am saying printing these values in
> log can be confusing, the user may think API applies these values to KNI interface.

Well, yes the link_status (link up, link down) _is_ applied to the KNI
interface.  When that occurs, most people want to know what the link
speed is that the link came up at.  Again, this is how every other
Ethernet driver in linux works.  I doubt that someone would be
confused by that.

If the KNI interface is purely virtual (that is, it does not
correspond to any physical Ethernet port), then the DPDK application
writer is under no obligation to use this API function, or they can
use it and just set the speed to 0.  If the end user understands that
the KNI interface is purely virtual, then they should not be confused
that the speed is 0 or that the duplex/autoNeg values are irrelevant.

I just don't see how this could cause any confusion.

Re: [dpdk-dev] [PATCH v2 1/5] kni: add API to set link status on kernel interface

[Ferruh Yigit]

On 9/28/2018 12:05 AM, Dan Gora wrote:
> On Thu, Sep 27, 2018 at 6:49 PM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>>> It would be useful if it writes the values of virtual interface, but this API
>>>> prints user input.
>>>>
>>> I'm sorry, Ferruh, I really don't understand what you are referring to
>>> here.  What virtual interface are you talking about?
>>
>> Virtual interface is KNI interface, Linux virtual network interface.
>>
>>
>> Let me try again,
>>
>> You are adding a new API to KNI library, which is to set link status of KNI
>> interface.
>>
>> This API prints some link related values in its log. But these values are not
>> applied to KNI interface or not the values coming from KNI interface. These are
>> just values provided by user to the API, I am saying printing these values in
>> log can be confusing, the user may think API applies these values to KNI interface.
> 
> Well, yes the link_status (link up, link down) _is_ applied to the KNI
> interface.  When that occurs, most people want to know what the link
> speed is that the link came up at.  

+1 to this, people would like to know link speed of the interface.
Are you printing link speed of interface? You are printing whatever user pass to
API.
I guess you trust to user to provide correct values there, but since only link
up & down matters, what prevents user to leave other fields, like speed, just
random values?


> Again, this is how every other
> Ethernet driver in linux works.  I doubt that someone would be
> confused by that.
> 
> If the KNI interface is purely virtual (that is, it does not
> correspond to any physical Ethernet port), then the DPDK application
> writer is under no obligation to use this API function, or they can
> use it and just set the speed to 0.  If the end user understands that
> the KNI interface is purely virtual, then they should not be confused
> that the speed is 0 or that the duplex/autoNeg values are irrelevant.
> 
> I just don't see how this could cause any confusion.
> 

Re: [dpdk-dev] [PATCH v2 1/5] kni: add API to set link status on kernel interface

[Dan Gora]

On Thu, Sep 27, 2018 at 8:44 PM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>> Well, yes the link_status (link up, link down) _is_ applied to the KNI
>> interface.  When that occurs, most people want to know what the link
>> speed is that the link came up at.
>
> +1 to this, people would like to know link speed of the interface.
> Are you printing link speed of interface? You are printing whatever user pass to
> API.

There is no such thing as "link speed of the interface".  The link
speed is the speed of the underlying Ethernet link that the interface
corresponds to.  This is true for all other ethernet interfaces in the
kernel.

> I guess you trust to user to provide correct values there, but since only link
> up & down matters, what prevents user to leave other fields, like speed, just
> random values?

Nothing.  What prevents anyone from providing random values for
anything?  The point of the API was to make it super simple, just:

rte_eth_link_get_nowait(portid, &link);
rte_kni_update_link(p[portid]->kni[i], &link);

No messing around with the link info retrieved from
rte_eth_link_get(_nowait), just dump in the struct that was returned.

Re: [dpdk-dev] [PATCH v2 1/5] kni: add API to set link status on kernel interface

[Igor Ryzhov]

Hi Dan, Ferruh,

Why do we need "struct rte_eth_link" as a parameter at all?
Only link status is used in the function – let's use it only:

rte_kni_update_link(struct rte_kni *kni, int link_status) /* 0 – down, 1 –
up */

It will also solve your differences as we won't have any "redundant"
information to print :)

Best regards,
Igor

On Fri, Sep 28, 2018 at 2:52 AM Dan Gora <dg@adax.com> wrote:

> On Thu, Sep 27, 2018 at 8:44 PM, Ferruh Yigit <ferruh.yigit@intel.com>
> wrote:
> >> Well, yes the link_status (link up, link down) _is_ applied to the KNI
> >> interface.  When that occurs, most people want to know what the link
> >> speed is that the link came up at.
> >
> > +1 to this, people would like to know link speed of the interface.
> > Are you printing link speed of interface? You are printing whatever user
> pass to
> > API.
>
> There is no such thing as "link speed of the interface".  The link
> speed is the speed of the underlying Ethernet link that the interface
> corresponds to.  This is true for all other ethernet interfaces in the
> kernel.
>
> > I guess you trust to user to provide correct values there, but since
> only link
> > up & down matters, what prevents user to leave other fields, like speed,
> just
> > random values?
>
> Nothing.  What prevents anyone from providing random values for
> anything?  The point of the API was to make it super simple, just:
>
> rte_eth_link_get_nowait(portid, &link);
> rte_kni_update_link(p[portid]->kni[i], &link);
>
> No messing around with the link info retrieved from
> rte_eth_link_get(_nowait), just dump in the struct that was returned.
>

Re: [dpdk-dev] [PATCH v2 1/5] kni: add API to set link status on kernel interface

[Ferruh Yigit]

On 9/28/2018 12:51 AM, Dan Gora wrote:
> On Thu, Sep 27, 2018 at 8:44 PM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>> Well, yes the link_status (link up, link down) _is_ applied to the KNI
>>> interface.  When that occurs, most people want to know what the link
>>> speed is that the link came up at.
>>
>> +1 to this, people would like to know link speed of the interface.
>> Are you printing link speed of interface? You are printing whatever user pass to
>> API.
> 
> There is no such thing as "link speed of the interface".  The link
> speed is the speed of the underlying Ethernet link that the interface
> corresponds to.  This is true for all other ethernet interfaces in the
> kernel.

This is an API to set link status of KNI interface, KNI interface is a virtual
interface and no need to be backed by a physical interface at all.

Only kni sample application uses it in a way to match a physical interface to a
KNI interface, but please check KNI PMD where you can have multiple KNI
interface without any physical device at all.

> 
>> I guess you trust to user to provide correct values there, but since only link
>> up & down matters, what prevents user to leave other fields, like speed, just
>> random values?
> 
> Nothing.  What prevents anyone from providing random values for
> anything?  The point of the API was to make it super simple, just:
> 
> rte_eth_link_get_nowait(portid, &link);
> rte_kni_update_link(p[portid]->kni[i], &link);

You are only thinking about this use case. Normally the input to API should be
verified right, for this case there is no way to verify it and vales are not
used at all, it is just printed in API.

OK to print link information but please do so in sample application before
calling API, not in API please.

> 
> No messing around with the link info retrieved from
> rte_eth_link_get(_nowait), just dump in the struct that was returned.
> 

Re: [dpdk-dev] [PATCH v2 1/5] kni: add API to set link status on kernel interface

[Dan Gora]

On Fri, Sep 28, 2018 at 5:03 AM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>
> On 9/28/2018 12:51 AM, Dan Gora wrote:
> > On Thu, Sep 27, 2018 at 8:44 PM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> >>> Well, yes the link_status (link up, link down) _is_ applied to the KNI
> >>> interface.  When that occurs, most people want to know what the link
> >>> speed is that the link came up at.
> >>
> >> +1 to this, people would like to know link speed of the interface.
> >> Are you printing link speed of interface? You are printing whatever user pass to
> >> API.
> >
> > There is no such thing as "link speed of the interface".  The link
> > speed is the speed of the underlying Ethernet link that the interface
> > corresponds to.  This is true for all other ethernet interfaces in the
> > kernel.
>
> This is an API to set link status of KNI interface, KNI interface is a virtual
> interface and no need to be backed by a physical interface at all.

Yes, I understand that.

> Only kni sample application uses it in a way to match a physical interface to a
> KNI interface, but please check KNI PMD where you can have multiple KNI
> interface without any physical device at all.

Yes, I understand that.. As I pointed out previously, if there is no
physical device which corresponds to this KNI interface, the
application can:

1) Not use this API at all, just as they do now.
2) Use the API and set the state to linkup/0Mbps/autoNeg/Full.  This
is perfectly reasonable and no one would be confused by this.

> >> I guess you trust to user to provide correct values there, but since only link
> >> up & down matters, what prevents user to leave other fields, like speed, just
> >> random values?
> >
> > Nothing.  What prevents anyone from providing random values for
> > anything?  The point of the API was to make it super simple, just:
> >
> > rte_eth_link_get_nowait(portid, &link);
> > rte_kni_update_link(p[portid]->kni[i], &link);
>
> You are only thinking about this use case. Normally the input to API should be
> verified right, for this case there is no way to verify it and vales are not
> used at all, it is just printed in API.

In most applications it is useful to have a message printed which
shows the *change* in link status as well as the speed that the link
came up at.  If you don't provide the link speed, etc information,
then the API is not really useful.  Yes the application writer can
provide whatever they want for the link speed/duplex/autoneg, but so
what?  Their application will have a nonsensical log message.  It's
not DPDK's job to enforce sanity on everyone's application.

And no, not every input to every API function is verified.  There are
lots of examples in the DPDK where this is not the case.  The
parameters should be verified to ensure that they do not cause the
application to "break", not necessarily so that they make sense
(whatever that would mean in this case).

It was recommended that we change the rte_kni_update_link() API to
only use the link status as an input and print the log messages
outside the API in the application code.  However think about what
that would entail:

You only want to log *changes* in the link state, not log a message
when you set the same state.  This means that either:

1) the application would have to store the previous carrier state.
2) rte_kni_update_link could return the previous value read from the
/sys/.../carrier file.
3) the application could read the /sys/.../carrier file before calling
rte_kni_update_link to read the previous carrier state.

In case 1 you're obliging all users of this API to carry around this
extra state for no real reason.  The API can easily read the
/sys/.../carrier file to see the previous state while it has it open,
just as it does in this patch.

In case 2, the application can easily detect when the state changes,
but then you end up with a return value something like -1 for errors,
0 for previous_link == down, 1 for previous_link == up.  I fail to see
how this is a better API design that what has already been proposed,
but it's at least somewhat useful, even if awkward.

In the DPDK application, you get something like:

rte_eth_link_get_nowait(portid, &link);
prev_link = rte_kni_update_link(p[portid]->kni[i], link.link_status);
if (prev_link >= 0)
{
   if (prev_link == 0 && link.link_status == 1)
     printf("LINKUP: speed %d duplex %s autoneg %s\n", ....)
  else if (prev_link == 1 && link.link_status == 0)
    printf("LINKDOWN:\n");
} else {
  printf("Error: something went wrong...\n");
}

I don't really see how this is better than:

rte_eth_link_get_nowait(portid, &link);
rte_kni_update_link(p[portid]->kni[i], &link);

but that's my opinion...

In case 3, you might as well not even have this API since you're
duplicating half of the same code.

I would be willing to implement case 2, however I still think that it
is not as good a design and will unnecessarily complicate application
code compared to the current patch here.  Cases 1 and 3 are
non-starters for me.  Our application needs log messages only when the
link *changes* state.  These log messages need to include the link
speed/duplex/autoneg info.  I think that most, if not all,
applications would want that too.  It's how every other physical
ethernet driver in the linux kernel works.  DPDK applications which
use only KNI virtual interfaces without a physical interface are under
no obligation to use this API function.

thanks
dan

Re: [dpdk-dev] [PATCH v2 1/5] kni: add API to set link status on kernel interface

[Ferruh Yigit]

On 10/3/2018 8:07 PM, Dan Gora wrote:
> On Fri, Sep 28, 2018 at 5:03 AM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>
>> On 9/28/2018 12:51 AM, Dan Gora wrote:
>>> On Thu, Sep 27, 2018 at 8:44 PM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>>>> Well, yes the link_status (link up, link down) _is_ applied to the KNI
>>>>> interface.  When that occurs, most people want to know what the link
>>>>> speed is that the link came up at.
>>>>
>>>> +1 to this, people would like to know link speed of the interface.
>>>> Are you printing link speed of interface? You are printing whatever user pass to
>>>> API.
>>>
>>> There is no such thing as "link speed of the interface".  The link
>>> speed is the speed of the underlying Ethernet link that the interface
>>> corresponds to.  This is true for all other ethernet interfaces in the
>>> kernel.
>>
>> This is an API to set link status of KNI interface, KNI interface is a virtual
>> interface and no need to be backed by a physical interface at all.
> 
> Yes, I understand that.
> 
>> Only kni sample application uses it in a way to match a physical interface to a
>> KNI interface, but please check KNI PMD where you can have multiple KNI
>> interface without any physical device at all.
> 
> Yes, I understand that.. As I pointed out previously, if there is no
> physical device which corresponds to this KNI interface, the
> application can:
> 
> 1) Not use this API at all, just as they do now.

This API has nothing with if KNI interface has corresponding physical device or
not, all KNI users can use it.

> 2) Use the API and set the state to linkup/0Mbps/autoNeg/Full.  This
> is perfectly reasonable and no one would be confused by this.

That is the think, you are not setting anything, you are just printing
"0Mbps/autoNeg/Full" and assuming/hoping the _if_ there is a corresponding
physical device app sends the values of that device to API for printing.

Overall I am not sure if there is a value to discuss this more, and I don't want
this relatively small issue to block the patch, I will comment more to latest
version.

> 
>>>> I guess you trust to user to provide correct values there, but since only link
>>>> up & down matters, what prevents user to leave other fields, like speed, just
>>>> random values?
>>>
>>> Nothing.  What prevents anyone from providing random values for
>>> anything?  The point of the API was to make it super simple, just:
>>>
>>> rte_eth_link_get_nowait(portid, &link);
>>> rte_kni_update_link(p[portid]->kni[i], &link);
>>
>> You are only thinking about this use case. Normally the input to API should be
>> verified right, for this case there is no way to verify it and vales are not
>> used at all, it is just printed in API.
> 
> In most applications it is useful to have a message printed which
> shows the *change* in link status as well as the speed that the link
> came up at.  If you don't provide the link speed, etc information,
> then the API is not really useful.  Yes the application writer can
> provide whatever they want for the link speed/duplex/autoneg, but so
> what?  Their application will have a nonsensical log message.  It's
> not DPDK's job to enforce sanity on everyone's application.
> 
> And no, not every input to every API function is verified.  There are
> lots of examples in the DPDK where this is not the case.  The
> parameters should be verified to ensure that they do not cause the
> application to "break", not necessarily so that they make sense
> (whatever that would mean in this case).
> 
> It was recommended that we change the rte_kni_update_link() API to
> only use the link status as an input and print the log messages
> outside the API in the application code.  However think about what
> that would entail:
> 
> You only want to log *changes* in the link state, not log a message
> when you set the same state.  This means that either:
> 
> 1) the application would have to store the previous carrier state.
> 2) rte_kni_update_link could return the previous value read from the
> /sys/.../carrier file.
> 3) the application could read the /sys/.../carrier file before calling
> rte_kni_update_link to read the previous carrier state.
> 
> In case 1 you're obliging all users of this API to carry around this
> extra state for no real reason.  The API can easily read the
> /sys/.../carrier file to see the previous state while it has it open,
> just as it does in this patch.
> 
> In case 2, the application can easily detect when the state changes,
> but then you end up with a return value something like -1 for errors,
> 0 for previous_link == down, 1 for previous_link == up.  I fail to see
> how this is a better API design that what has already been proposed,
> but it's at least somewhat useful, even if awkward.
> 
> In the DPDK application, you get something like:
> 
> rte_eth_link_get_nowait(portid, &link);
> prev_link = rte_kni_update_link(p[portid]->kni[i], link.link_status);
> if (prev_link >= 0)
> {
>    if (prev_link == 0 && link.link_status == 1)
>      printf("LINKUP: speed %d duplex %s autoneg %s\n", ....)
>   else if (prev_link == 1 && link.link_status == 0)
>     printf("LINKDOWN:\n");
> } else {
>   printf("Error: something went wrong...\n");
> }
> 
> I don't really see how this is better than:
> 
> rte_eth_link_get_nowait(portid, &link);
> rte_kni_update_link(p[portid]->kni[i], &link);
> 
> but that's my opinion...
> 
> In case 3, you might as well not even have this API since you're
> duplicating half of the same code.
> 
> I would be willing to implement case 2, however I still think that it
> is not as good a design and will unnecessarily complicate application
> code compared to the current patch here.  Cases 1 and 3 are
> non-starters for me.  Our application needs log messages only when the
> link *changes* state.  These log messages need to include the link
> speed/duplex/autoneg info.  I think that most, if not all,
> applications would want that too.  It's how every other physical
> ethernet driver in the linux kernel works.  DPDK applications which
> use only KNI virtual interfaces without a physical interface are under
> no obligation to use this API function.
> 
> thanks
> dan
> 

Re: [dpdk-dev] [PATCH v2 1/5] kni: add API to set link status on kernel interface

[Dan Gora]

On Wed, Oct 10, 2018 at 11:09 AM Ferruh Yigit <ferruh.yigit@intel.com> wrote:

> > Yes, I understand that.. As I pointed out previously, if there is no
> > physical device which corresponds to this KNI interface, the
> > application can:
> >
> > 1) Not use this API at all, just as they do now.
>
> This API has nothing with if KNI interface has corresponding physical device or
> not, all KNI users can use it.
>

I don't know what this means.

> > 2) Use the API and set the state to linkup/0Mbps/autoNeg/Full.  This
> > is perfectly reasonable and no one would be confused by this.
>
> That is the think, you are not setting anything, you are just printing
> "0Mbps/autoNeg/Full" and assuming/hoping the _if_ there is a corresponding
> physical device app sends the values of that device to API for printing.

ugh.. Again the API doesn't care if the values are "right".  The
application writer probably does though.


> Overall I am not sure if there is a value to discuss this more, and I don't want
> this relatively small issue to block the patch, I will comment more to latest
> version.

Agreed, I don't know how else I can express myself to get my point across.

Can you at least comment on the rest of this email below that I spent
a lot of time writing, trying to explain what I'm talking about?

> >>>> I guess you trust to user to provide correct values there, but since only link
> >>>> up & down matters, what prevents user to leave other fields, like speed, just
> >>>> random values?
> >>>
> >>> Nothing.  What prevents anyone from providing random values for
> >>> anything?  The point of the API was to make it super simple, just:
> >>>
> >>> rte_eth_link_get_nowait(portid, &link);
> >>> rte_kni_update_link(p[portid]->kni[i], &link);
> >>
> >> You are only thinking about this use case. Normally the input to API should be
> >> verified right, for this case there is no way to verify it and vales are not
> >> used at all, it is just printed in API.
> >
> > In most applications it is useful to have a message printed which
> > shows the *change* in link status as well as the speed that the link
> > came up at.  If you don't provide the link speed, etc information,
> > then the API is not really useful.  Yes the application writer can
> > provide whatever they want for the link speed/duplex/autoneg, but so
> > what?  Their application will have a nonsensical log message.  It's
> > not DPDK's job to enforce sanity on everyone's application.
> >
> > And no, not every input to every API function is verified.  There are
> > lots of examples in the DPDK where this is not the case.  The
> > parameters should be verified to ensure that they do not cause the
> > application to "break", not necessarily so that they make sense
> > (whatever that would mean in this case).
> >
> > It was recommended that we change the rte_kni_update_link() API to
> > only use the link status as an input and print the log messages
> > outside the API in the application code.  However think about what
> > that would entail:
> >
> > You only want to log *changes* in the link state, not log a message
> > when you set the same state.  This means that either:
> >
> > 1) the application would have to store the previous carrier state.
> > 2) rte_kni_update_link could return the previous value read from the
> > /sys/.../carrier file.
> > 3) the application could read the /sys/.../carrier file before calling
> > rte_kni_update_link to read the previous carrier state.
> >
> > In case 1 you're obliging all users of this API to carry around this
> > extra state for no real reason.  The API can easily read the
> > /sys/.../carrier file to see the previous state while it has it open,
> > just as it does in this patch.
> >
> > In case 2, the application can easily detect when the state changes,
> > but then you end up with a return value something like -1 for errors,
> > 0 for previous_link == down, 1 for previous_link == up.  I fail to see
> > how this is a better API design that what has already been proposed,
> > but it's at least somewhat useful, even if awkward.
> >
> > In the DPDK application, you get something like:
> >
> > rte_eth_link_get_nowait(portid, &link);
> > prev_link = rte_kni_update_link(p[portid]->kni[i], link.link_status);
> > if (prev_link >= 0)
> > {
> >    if (prev_link == 0 && link.link_status == 1)
> >      printf("LINKUP: speed %d duplex %s autoneg %s\n", ....)
> >   else if (prev_link == 1 && link.link_status == 0)
> >     printf("LINKDOWN:\n");
> > } else {
> >   printf("Error: something went wrong...\n");
> > }
> >
> > I don't really see how this is better than:
> >
> > rte_eth_link_get_nowait(portid, &link);
> > rte_kni_update_link(p[portid]->kni[i], &link);
> >
> > but that's my opinion...
> >
> > In case 3, you might as well not even have this API since you're
> > duplicating half of the same code.
> >
> > I would be willing to implement case 2, however I still think that it
> > is not as good a design and will unnecessarily complicate application
> > code compared to the current patch here.  Cases 1 and 3 are
> > non-starters for me.  Our application needs log messages only when the
> > link *changes* state.  These log messages need to include the link
> > speed/duplex/autoneg info.  I think that most, if not all,
> > applications would want that too.  It's how every other physical
> > ethernet driver in the linux kernel works.  DPDK applications which
> > use only KNI virtual interfaces without a physical interface are under
> > no obligation to use this API function.
> >
> > thanks
> > dan

[dpdk-dev] [PATCH v2 2/5] kni: set default carrier state to 'off'

[Dan Gora]

Set the carrier state to 'off' when the interface is instantiated
or when it is marked 'up' or 'down'.  This is necessary to set the
interface to a known operational state until the carrier state is
changed with rte_kni_update_link().

Signed-off-by: Dan Gora <dg@adax.com>
---
 kernel/linux/kni/kni_misc.c | 2 ++
 kernel/linux/kni/kni_net.c  | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/kernel/linux/kni/kni_misc.c b/kernel/linux/kni/kni_misc.c
index fa69f8e63..45649499d 100644
--- a/kernel/linux/kni/kni_misc.c
+++ b/kernel/linux/kni/kni_misc.c
@@ -466,6 +466,8 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_num,
 		return -ENODEV;
 	}
 
+	netif_carrier_off(net_dev);
+
 	ret = kni_run_thread(knet, kni, dev_info.force_bind);
 	if (ret != 0)
 		return ret;
diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c
index 7fcfa106c..1f8ba0700 100644
--- a/kernel/linux/kni/kni_net.c
+++ b/kernel/linux/kni/kni_net.c
@@ -133,6 +133,7 @@ kni_net_open(struct net_device *dev)
 	struct kni_dev *kni = netdev_priv(dev);
 
 	netif_start_queue(dev);
+	netif_carrier_off(dev);
 
 	memset(&req, 0, sizeof(req));
 	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
@@ -152,6 +153,7 @@ kni_net_release(struct net_device *dev)
 	struct kni_dev *kni = netdev_priv(dev);
 
 	netif_stop_queue(dev); /* can't transmit any more */
+	netif_carrier_off(dev);
 
 	memset(&req, 0, sizeof(req));
 	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
-- 
2.19.0

Re: [dpdk-dev] [PATCH v2 2/5] kni: set default carrier state to 'off'

[Ferruh Yigit]

On 9/19/2018 8:55 PM, Dan Gora wrote:
> Set the carrier state to 'off' when the interface is instantiated
> or when it is marked 'up' or 'down'.  This is necessary to set the
> interface to a known operational state until the carrier state is
> changed with rte_kni_update_link().
> 
> Signed-off-by: Dan Gora <dg@adax.com>

Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

[dpdk-dev] [PATCH v2 3/5] examples/kni: monitor and update link status continually

[Dan Gora]

Update KNI example to continuously monitor the Ethernet link status of
the physical link and update the carrier status of the corresponding
interfaces with rte_kni_update_link().

Signed-off-by: Dan Gora <dg@adax.com>
---
 examples/kni/Makefile |  2 ++
 examples/kni/main.c   | 82 +++++++++++++++++++++----------------------
 2 files changed, 43 insertions(+), 41 deletions(-)

diff --git a/examples/kni/Makefile b/examples/kni/Makefile
index 7e19d2e2a..dd90d7d73 100644
--- a/examples/kni/Makefile
+++ b/examples/kni/Makefile
@@ -20,6 +20,7 @@ static: build/$(APP)-static
 
 PC_FILE := $(shell pkg-config --path libdpdk)
 CFLAGS += -O3 $(shell pkg-config --cflags libdpdk)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk)
 LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk)
 
@@ -54,6 +55,7 @@ please change the definition of the RTE_TARGET environment variable)
 endif
 
 CFLAGS += -O3
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 CFLAGS += $(WERROR_FLAGS)
 
 include $(RTE_SDK)/mk/rte.extapp.mk
diff --git a/examples/kni/main.c b/examples/kni/main.c
index 80c401c51..8c3d3839e 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -107,6 +107,8 @@ static uint32_t ports_mask = 0;
 /* Ports set in promiscuous mode off by default. */
 static int promiscuous_on = 0;
 
+static int kni_running;
+
 /* Structure type for recording kni interface specific stats */
 struct kni_interface_stats {
 	/* number of pkts received from NIC, and sent to KNI */
@@ -620,58 +622,45 @@ init_port(uint16_t port)
 }
 
 /* Check the link status of all ports in up to 9s, and print them finally */
-static void
-check_all_ports_link_status(uint32_t port_mask)
+static void *
+check_all_ports_link_status(void *arg)
 {
 #define CHECK_INTERVAL 100 /* 100ms */
 #define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */
 	uint16_t portid;
-	uint8_t count, all_ports_up, print_flag = 0;
 	struct rte_eth_link link;
+	const char *name;
+	char cmd[64];
+	unsigned int i;
+	struct kni_port_params **p = kni_port_params_array;
+	(void) arg;
+
+	RTE_ETH_FOREACH_DEV(portid) {
+		for (i = 0; i < p[portid]->nb_kni; i++) {
+			name = rte_kni_get_name(p[portid]->kni[i]);
+			snprintf(cmd, sizeof(cmd),
+					"/sbin/ifconfig %s up", name);
+			RTE_LOG(INFO, APP,
+					"Marking interface %s 'up'\n", name);
+			if (system(cmd) == -1)
+				RTE_LOG(ERR, APP,
+				  "Error: Failed to mark interface %s 'up'\n",
+						name);
+		}
+	}
 
-	printf("\nChecking link status\n");
-	fflush(stdout);
-	for (count = 0; count <= MAX_CHECK_TIME; count++) {
-		all_ports_up = 1;
+	while (kni_running) {
+		rte_delay_ms(CHECK_INTERVAL);
 		RTE_ETH_FOREACH_DEV(portid) {
-			if ((port_mask & (1 << portid)) == 0)
+			if ((ports_mask & (1 << portid)) == 0)
 				continue;
 			memset(&link, 0, sizeof(link));
 			rte_eth_link_get_nowait(portid, &link);
-			/* print link status if flag set */
-			if (print_flag == 1) {
-				if (link.link_status)
-					printf(
-					"Port%d Link Up - speed %uMbps - %s\n",
-						portid, link.link_speed,
-				(link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
-					("full-duplex") : ("half-duplex\n"));
-				else
-					printf("Port %d Link Down\n", portid);
-				continue;
-			}
-			/* clear all_ports_up flag if any link down */
-			if (link.link_status == ETH_LINK_DOWN) {
-				all_ports_up = 0;
-				break;
-			}
-		}
-		/* after finally printing all link status, get out */
-		if (print_flag == 1)
-			break;
-
-		if (all_ports_up == 0) {
-			printf(".");
-			fflush(stdout);
-			rte_delay_ms(CHECK_INTERVAL);
-		}
-
-		/* set the print_flag if all ports up or timeout */
-		if (all_ports_up == 1 || count == (MAX_CHECK_TIME - 1)) {
-			print_flag = 1;
-			printf("done\n");
+			for (i = 0; i < p[portid]->nb_kni; i++)
+				rte_kni_update_link(p[portid]->kni[i], &link);
 		}
 	}
+	return NULL;
 }
 
 /* Callback for request of changing MTU */
@@ -893,6 +882,8 @@ main(int argc, char** argv)
 	int ret;
 	uint16_t nb_sys_ports, port;
 	unsigned i;
+	void *retval;
+	pthread_t kni_link_tid;
 
 	/* Associate signal_hanlder function with USR signals */
 	signal(SIGUSR1, signal_handler);
@@ -947,7 +938,14 @@ main(int argc, char** argv)
 
 		kni_alloc(port);
 	}
-	check_all_ports_link_status(ports_mask);
+
+	kni_running = 1;
+	ret = rte_ctrl_thread_create(&kni_link_tid,
+				     "KNI link status check", NULL,
+				     check_all_ports_link_status, NULL);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Could not create link status thread!\n");
 
 	/* Launch per-lcore function on every lcore */
 	rte_eal_mp_remote_launch(main_loop, NULL, CALL_MASTER);
@@ -955,6 +953,8 @@ main(int argc, char** argv)
 		if (rte_eal_wait_lcore(i) < 0)
 			return -1;
 	}
+	kni_running = 0;
+	pthread_join(kni_link_tid, &retval);
 
 	/* Release resources */
 	RTE_ETH_FOREACH_DEV(port) {
-- 
2.19.0

Re: [dpdk-dev] [PATCH v2 3/5] examples/kni: monitor and update link status continually

[Ferruh Yigit]

On 9/19/2018 8:55 PM, Dan Gora wrote:
> Update KNI example to continuously monitor the Ethernet link status of
> the physical link and update the carrier status of the corresponding
> interfaces with rte_kni_update_link().
> 
> Signed-off-by: Dan Gora <dg@adax.com>

Also this patch sets kni interfaces "up", please add this information to commit log.

<...>

> +	RTE_ETH_FOREACH_DEV(portid) {
> +		for (i = 0; i < p[portid]->nb_kni; i++) {
> +			name = rte_kni_get_name(p[portid]->kni[i]);
> +			snprintf(cmd, sizeof(cmd),
> +					"/sbin/ifconfig %s up", name);
> +			RTE_LOG(INFO, APP,
> +					"Marking interface %s 'up'\n", name);
> +			if (system(cmd) == -1)
> +				RTE_LOG(ERR, APP,
> +				  "Error: Failed to mark interface %s 'up'\n",
> +						name);
> +		}
> +	}

This part can be separated thread, overall already done only once, and put into
its own function. And what about making this optional with command line argument.

>  
> -	printf("\nChecking link status\n");
> -	fflush(stdout);
> -	for (count = 0; count <= MAX_CHECK_TIME; count++) {
> -		all_ports_up = 1;
> +	while (kni_running) {
> +		rte_delay_ms(CHECK_INTERVAL);
>  		RTE_ETH_FOREACH_DEV(portid) {
> -			if ((port_mask & (1 << portid)) == 0)
> +			if ((ports_mask & (1 << portid)) == 0)
>  				continue;
>  			memset(&link, 0, sizeof(link));
>  			rte_eth_link_get_nowait(portid, &link);
> -			/* print link status if flag set */
> -			if (print_flag == 1) {
> -				if (link.link_status)
> -					printf(
> -					"Port%d Link Up - speed %uMbps - %s\n",
> -						portid, link.link_speed,
> -				(link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
> -					("full-duplex") : ("half-duplex\n"));
> -				else
> -					printf("Port %d Link Down\n", portid);
> -				continue;
> -			}
> -			/* clear all_ports_up flag if any link down */
> -			if (link.link_status == ETH_LINK_DOWN) {
> -				all_ports_up = 0;
> -				break;
> -			}
> -		}
> -		/* after finally printing all link status, get out */
> -		if (print_flag == 1)
> -			break;
> -
> -		if (all_ports_up == 0) {
> -			printf(".");
> -			fflush(stdout);
> -			rte_delay_ms(CHECK_INTERVAL);
> -		}

Previous logic makes sure all physical interfaces are up before packet
processing is started. Why removing this logic? I think it is good to keep it as
it is.

<...>

> @@ -947,7 +938,14 @@ main(int argc, char** argv)
>  
>  		kni_alloc(port);
>  	}
> -	check_all_ports_link_status(ports_mask);
> +
> +	kni_running = 1;
> +	ret = rte_ctrl_thread_create(&kni_link_tid,
> +				     "KNI link status check", NULL,
> +				     check_all_ports_link_status, NULL);

This thread is to reflect physical port link status to KNI interface, this can
be useful but not sure to have it as default. Why not introduce this ability,
and creating thread, as an optional feature enabled by command line option?

So overall makes two commandline option,
- one to set kni interfaces up by application
- one to create a thread to check and reflect physical port link status to KNI
interface

Re: [dpdk-dev] [PATCH v2 3/5] examples/kni: monitor and update link status continually

[Dan Gora]

On Wed, Sep 26, 2018 at 11:00 AM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> On 9/19/2018 8:55 PM, Dan Gora wrote:
>> Update KNI example to continuously monitor the Ethernet link status of
>> the physical link and update the carrier status of the corresponding
>> interfaces with rte_kni_update_link().
>>
>> Signed-off-by: Dan Gora <dg@adax.com>
>
> Also this patch sets kni interfaces "up", please add this information to commit log.
>
> <...>
>
>> +     RTE_ETH_FOREACH_DEV(portid) {
>> +             for (i = 0; i < p[portid]->nb_kni; i++) {
>> +                     name = rte_kni_get_name(p[portid]->kni[i]);
>> +                     snprintf(cmd, sizeof(cmd),
>> +                                     "/sbin/ifconfig %s up", name);
>> +                     RTE_LOG(INFO, APP,
>> +                                     "Marking interface %s 'up'\n", name);
>> +                     if (system(cmd) == -1)
>> +                             RTE_LOG(ERR, APP,
>> +                               "Error: Failed to mark interface %s 'up'\n",
>> +                                             name);
>> +             }
>> +     }
>
> This part can be separated thread, overall already done only once, and put into
> its own function. And what about making this optional with command line argument.
>

I'll just remove it.. It's unnecessary.  I don't see why the user
should have to explicitly type 'ip link set up dev <name>' external to
the application, but that's the way it was before, so that's fine with
me.

>>
>> -     printf("\nChecking link status\n");
>> -     fflush(stdout);
>> -     for (count = 0; count <= MAX_CHECK_TIME; count++) {
>> -             all_ports_up = 1;
>> +     while (kni_running) {
>> +             rte_delay_ms(CHECK_INTERVAL);
>>               RTE_ETH_FOREACH_DEV(portid) {
>> -                     if ((port_mask & (1 << portid)) == 0)
>> +                     if ((ports_mask & (1 << portid)) == 0)
>>                               continue;
>>                       memset(&link, 0, sizeof(link));
>>                       rte_eth_link_get_nowait(portid, &link);
>> -                     /* print link status if flag set */
>> -                     if (print_flag == 1) {
>> -                             if (link.link_status)
>> -                                     printf(
>> -                                     "Port%d Link Up - speed %uMbps - %s\n",
>> -                                             portid, link.link_speed,
>> -                             (link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
>> -                                     ("full-duplex") : ("half-duplex\n"));
>> -                             else
>> -                                     printf("Port %d Link Down\n", portid);
>> -                             continue;
>> -                     }
>> -                     /* clear all_ports_up flag if any link down */
>> -                     if (link.link_status == ETH_LINK_DOWN) {
>> -                             all_ports_up = 0;
>> -                             break;
>> -                     }
>> -             }
>> -             /* after finally printing all link status, get out */
>> -             if (print_flag == 1)
>> -                     break;
>> -
>> -             if (all_ports_up == 0) {
>> -                     printf(".");
>> -                     fflush(stdout);
>> -                     rte_delay_ms(CHECK_INTERVAL);
>> -             }
>
> Previous logic makes sure all physical interfaces are up before packet
> processing is started.

Uh, no it didn't.  It would wait for up to 9 seconds for the links to
come up and print a message, but if they didn't it would happily just
chug right along.

> Why removing this logic? I think it is good to keep it as it is.

Because it was kind of unnecessary in the first place.  Why 9 seconds?
 Why not 10?  Why not 60?  None of the following logic in the program
depends on the link being up.

>> @@ -947,7 +938,14 @@ main(int argc, char** argv)
>>
>>               kni_alloc(port);
>>       }
>> -     check_all_ports_link_status(ports_mask);
>> +
>> +     kni_running = 1;
>> +     ret = rte_ctrl_thread_create(&kni_link_tid,
>> +                                  "KNI link status check", NULL,
>> +                                  check_all_ports_link_status, NULL);
>
> This thread is to reflect physical port link status to KNI interface, this can
> be useful but not sure to have it as default. Why not introduce this ability,
> and creating thread, as an optional feature enabled by command line option?
>
> So overall makes two commandline option,
> - one to set kni interfaces up by application
> - one to create a thread to check and reflect physical port link status to KNI
> interface

I'm not really inclined to add new command line options here.  I'll
remove the code to set the interface 'up' so the user will continue to
have to type 'ip link set up dev <name>' externally the way they do
now.  I don't see any reason why checking the link status would
require a command line option.  It affects nothing in the datapath.

Re: [dpdk-dev] [PATCH v2 3/5] examples/kni: monitor and update link status continually

[Ferruh Yigit]

On 9/26/2018 8:16 PM, Dan Gora wrote:
> On Wed, Sep 26, 2018 at 11:00 AM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>> On 9/19/2018 8:55 PM, Dan Gora wrote:
>>> Update KNI example to continuously monitor the Ethernet link status of
>>> the physical link and update the carrier status of the corresponding
>>> interfaces with rte_kni_update_link().
>>>
>>> Signed-off-by: Dan Gora <dg@adax.com>
>>
>> Also this patch sets kni interfaces "up", please add this information to commit log.
>>
>> <...>
>>
>>> +     RTE_ETH_FOREACH_DEV(portid) {
>>> +             for (i = 0; i < p[portid]->nb_kni; i++) {
>>> +                     name = rte_kni_get_name(p[portid]->kni[i]);
>>> +                     snprintf(cmd, sizeof(cmd),
>>> +                                     "/sbin/ifconfig %s up", name);
>>> +                     RTE_LOG(INFO, APP,
>>> +                                     "Marking interface %s 'up'\n", name);
>>> +                     if (system(cmd) == -1)
>>> +                             RTE_LOG(ERR, APP,
>>> +                               "Error: Failed to mark interface %s 'up'\n",
>>> +                                             name);
>>> +             }
>>> +     }
>>
>> This part can be separated thread, overall already done only once, and put into
>> its own function. And what about making this optional with command line argument.
>>
> 
> I'll just remove it.. It's unnecessary.  I don't see why the user
> should have to explicitly type 'ip link set up dev <name>' external to
> the application, but that's the way it was before, so that's fine with
> me.

Thanks for the work Dan,
I am trying to be conservative for possible use cases of the sample, as you
pointed before perhaps people would like to use these with kind of network
manager they want to up interface when they want.

I think it was good feature to have but just lets have it optional.

> 
>>>
>>> -     printf("\nChecking link status\n");
>>> -     fflush(stdout);
>>> -     for (count = 0; count <= MAX_CHECK_TIME; count++) {
>>> -             all_ports_up = 1;
>>> +     while (kni_running) {
>>> +             rte_delay_ms(CHECK_INTERVAL);
>>>               RTE_ETH_FOREACH_DEV(portid) {
>>> -                     if ((port_mask & (1 << portid)) == 0)
>>> +                     if ((ports_mask & (1 << portid)) == 0)
>>>                               continue;
>>>                       memset(&link, 0, sizeof(link));
>>>                       rte_eth_link_get_nowait(portid, &link);
>>> -                     /* print link status if flag set */
>>> -                     if (print_flag == 1) {
>>> -                             if (link.link_status)
>>> -                                     printf(
>>> -                                     "Port%d Link Up - speed %uMbps - %s\n",
>>> -                                             portid, link.link_speed,
>>> -                             (link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
>>> -                                     ("full-duplex") : ("half-duplex\n"));
>>> -                             else
>>> -                                     printf("Port %d Link Down\n", portid);
>>> -                             continue;
>>> -                     }
>>> -                     /* clear all_ports_up flag if any link down */
>>> -                     if (link.link_status == ETH_LINK_DOWN) {
>>> -                             all_ports_up = 0;
>>> -                             break;
>>> -                     }
>>> -             }
>>> -             /* after finally printing all link status, get out */
>>> -             if (print_flag == 1)
>>> -                     break;
>>> -
>>> -             if (all_ports_up == 0) {
>>> -                     printf(".");
>>> -                     fflush(stdout);
>>> -                     rte_delay_ms(CHECK_INTERVAL);
>>> -             }
>>
>> Previous logic makes sure all physical interfaces are up before packet
>> processing is started.
> 
> Uh, no it didn't.  It would wait for up to 9 seconds for the links to
> come up and print a message, but if they didn't it would happily just
> chug right along.

Right, it will continue after 9 seconds, but it some time to interfaces to be up
before packet processing.

> 
>> Why removing this logic? I think it is good to keep it as it is.
> 
> Because it was kind of unnecessary in the first place.  Why 9 seconds?
>  Why not 10?  Why not 60?  None of the following logic in the program
> depends on the link being up.

I don't think it is unnecessary, this gives some time to interfaces to get
ready. 9 second may be coming from rte_eth_link_get() have 9 second timeout
defined, and I expect that should be coming from possible HW delay, so it make
sense to use that value here.

Anyway if all interfaces are "up" this API should not be a problem at all, I
don't see the motivation to removed it.

> 
>>> @@ -947,7 +938,14 @@ main(int argc, char** argv)
>>>
>>>               kni_alloc(port);
>>>       }
>>> -     check_all_ports_link_status(ports_mask);
>>> +
>>> +     kni_running = 1;
>>> +     ret = rte_ctrl_thread_create(&kni_link_tid,
>>> +                                  "KNI link status check", NULL,
>>> +                                  check_all_ports_link_status, NULL);
>>
>> This thread is to reflect physical port link status to KNI interface, this can
>> be useful but not sure to have it as default. Why not introduce this ability,
>> and creating thread, as an optional feature enabled by command line option?
>>
>> So overall makes two commandline option,
>> - one to set kni interfaces up by application
>> - one to create a thread to check and reflect physical port link status to KNI
>> interface
> 
> I'm not really inclined to add new command line options here.  I'll
> remove the code to set the interface 'up' so the user will continue to
> have to type 'ip link set up dev <name>' externally the way they do
> now.  I don't see any reason why checking the link status would
> require a command line option.  It affects nothing in the datapath.

First it is not just checking link status, continuously checking link status on
its own thread, and second reflecting HW link status to virtual interface.
Someone may or may not want both of those, what is the problem of making it
optional?

[dpdk-dev] [PATCH v2 4/5] examples/kni: add log msgs to show and clear stats

[Dan Gora]

Add logging messages showing the commands necessary for the user to
have the application display and zero the statistics.

Signed-off-by: Dan Gora <dg@adax.com>
---
 examples/kni/main.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/examples/kni/main.c b/examples/kni/main.c
index 8c3d3839e..ca45347d8 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -171,14 +171,13 @@ signal_handler(int signum)
 	/* When we receive a USR2 signal, reset stats */
 	if (signum == SIGUSR2) {
 		memset(&kni_stats, 0, sizeof(kni_stats));
-		printf("\n**Statistics have been reset**\n");
+		printf("\n** Statistics have been reset **\n");
 		return;
 	}
 
 	/* When we receive a RTMIN or SIGINT signal, stop kni processing */
 	if (signum == SIGRTMIN || signum == SIGINT){
-		printf("SIGRTMIN is received, and the KNI processing is "
-							"going to stop\n");
+		printf("\nSIGRTMIN/SIGINT received. KNI processing stopping.\n");
 		rte_atomic32_inc(&kni_stop);
 		return;
         }
@@ -884,6 +883,7 @@ main(int argc, char** argv)
 	unsigned i;
 	void *retval;
 	pthread_t kni_link_tid;
+	int pid;
 
 	/* Associate signal_hanlder function with USR signals */
 	signal(SIGUSR1, signal_handler);
@@ -939,6 +939,16 @@ main(int argc, char** argv)
 		kni_alloc(port);
 	}
 
+	pid = getpid();
+	RTE_LOG(INFO, APP, "========================\n");
+	RTE_LOG(INFO, APP, "KNI Running\n");
+	RTE_LOG(INFO, APP, "kill -SIGUSR1 %d\n", pid);
+	RTE_LOG(INFO, APP, "    Show KNI Statistics.\n");
+	RTE_LOG(INFO, APP, "kill -SIGUSR2 %d\n", pid);
+	RTE_LOG(INFO, APP, "    Zero KNI Statistics.\n");
+	RTE_LOG(INFO, APP, "========================\n");
+	fflush(stdout);
+
 	kni_running = 1;
 	ret = rte_ctrl_thread_create(&kni_link_tid,
 				     "KNI link status check", NULL,
-- 
2.19.0

Re: [dpdk-dev] [PATCH v2 4/5] examples/kni: add log msgs to show and clear stats

[Ferruh Yigit]

On 9/19/2018 8:55 PM, Dan Gora wrote:
> Add logging messages showing the commands necessary for the user to
> have the application display and zero the statistics.
> 
> Signed-off-by: Dan Gora <dg@adax.com>

Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

[dpdk-dev] [PATCH v2 5/5] examples/kni: improve zeroing statistics

[Dan Gora]

The worker threads incrementing the rx/tx_packets race with the signal
handler from the main thread zeroing the entire statistics structure.
This can cause the statistics to fail to be zeroed, even when there
is no traffic on those interfaces.

Improve zeroing the statistics by only incrementing rx/tx_packets
in worker threads by a non-zero amount.  This limits the race to the
periods in which traffic is actually being received or transmitted.

Signed-off-by: Dan Gora <dg@adax.com>
---
 examples/kni/main.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/examples/kni/main.c b/examples/kni/main.c
index ca45347d8..d00569740 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -223,7 +223,8 @@ kni_ingress(struct kni_port_params *p)
 		}
 		/* Burst tx to kni */
 		num = rte_kni_tx_burst(p->kni[i], pkts_burst, nb_rx);
-		kni_stats[port_id].rx_packets += num;
+		if (num)
+			kni_stats[port_id].rx_packets += num;
 
 		rte_kni_handle_request(p->kni[i]);
 		if (unlikely(num < nb_rx)) {
@@ -260,7 +261,8 @@ kni_egress(struct kni_port_params *p)
 		}
 		/* Burst tx to eth */
 		nb_tx = rte_eth_tx_burst(port_id, 0, pkts_burst, (uint16_t)num);
-		kni_stats[port_id].tx_packets += nb_tx;
+		if (nb_tx)
+			kni_stats[port_id].tx_packets += nb_tx;
 		if (unlikely(nb_tx < num)) {
 			/* Free mbufs not tx to NIC */
 			kni_burst_free_mbufs(&pkts_burst[nb_tx], num - nb_tx);
-- 
2.19.0

Re: [dpdk-dev] [PATCH v2 5/5] examples/kni: improve zeroing statistics

[Ferruh Yigit]

On 9/19/2018 8:55 PM, Dan Gora wrote:
> The worker threads incrementing the rx/tx_packets race with the signal
> handler from the main thread zeroing the entire statistics structure.
> This can cause the statistics to fail to be zeroed, even when there
> is no traffic on those interfaces.
> 
> Improve zeroing the statistics by only incrementing rx/tx_packets
> in worker threads by a non-zero amount.  This limits the race to the
> periods in which traffic is actually being received or transmitted.

Not sure about introducing an extra check to datapath for possible error on
stats zero. I am for dropping this patch, what do you think?

> 
> Signed-off-by: Dan Gora <dg@adax.com>
> ---
>  examples/kni/main.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/examples/kni/main.c b/examples/kni/main.c
> index ca45347d8..d00569740 100644
> --- a/examples/kni/main.c
> +++ b/examples/kni/main.c
> @@ -223,7 +223,8 @@ kni_ingress(struct kni_port_params *p)
>  		}
>  		/* Burst tx to kni */
>  		num = rte_kni_tx_burst(p->kni[i], pkts_burst, nb_rx);
> -		kni_stats[port_id].rx_packets += num;
> +		if (num)
> +			kni_stats[port_id].rx_packets += num;
>  
>  		rte_kni_handle_request(p->kni[i]);
>  		if (unlikely(num < nb_rx)) {
> @@ -260,7 +261,8 @@ kni_egress(struct kni_port_params *p)
>  		}
>  		/* Burst tx to eth */
>  		nb_tx = rte_eth_tx_burst(port_id, 0, pkts_burst, (uint16_t)num);
> -		kni_stats[port_id].tx_packets += nb_tx;
> +		if (nb_tx)
> +			kni_stats[port_id].tx_packets += nb_tx;
>  		if (unlikely(nb_tx < num)) {
>  			/* Free mbufs not tx to NIC */
>  			kni_burst_free_mbufs(&pkts_burst[nb_tx], num - nb_tx);
> 

Re: [dpdk-dev] [PATCH v2 5/5] examples/kni: improve zeroing statistics

[Dan Gora]

On Wed, Sep 26, 2018 at 11:01 AM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> On 9/19/2018 8:55 PM, Dan Gora wrote:
>> The worker threads incrementing the rx/tx_packets race with the signal
>> handler from the main thread zeroing the entire statistics structure.
>> This can cause the statistics to fail to be zeroed, even when there
>> is no traffic on those interfaces.
>>
>> Improve zeroing the statistics by only incrementing rx/tx_packets
>> in worker threads by a non-zero amount.  This limits the race to the
>> periods in which traffic is actually being received or transmitted.
>
> Not sure about introducing an extra check to datapath for possible error on
> stats zero. I am for dropping this patch, what do you think?

This is literally adding one instruction to the datapath.  Not even an
atomic instruction.  There is no effect on the performance caused by
this change.

Is that not better than the user (like me who experienced this)
wondering why they cannot zero the counters even when there is no
traffic?

-d

Re: [dpdk-dev] [PATCH v2 5/5] examples/kni: improve zeroing statistics

[Ferruh Yigit]

On 9/26/2018 3:48 PM, Dan Gora wrote:
> On Wed, Sep 26, 2018 at 11:01 AM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>> On 9/19/2018 8:55 PM, Dan Gora wrote:
>>> The worker threads incrementing the rx/tx_packets race with the signal
>>> handler from the main thread zeroing the entire statistics structure.
>>> This can cause the statistics to fail to be zeroed, even when there
>>> is no traffic on those interfaces.
>>>
>>> Improve zeroing the statistics by only incrementing rx/tx_packets
>>> in worker threads by a non-zero amount.  This limits the race to the
>>> periods in which traffic is actually being received or transmitted.
>>
>> Not sure about introducing an extra check to datapath for possible error on
>> stats zero. I am for dropping this patch, what do you think?
> 
> This is literally adding one instruction to the datapath.  Not even an
> atomic instruction.  There is no effect on the performance caused by
> this change.
> 
> Is that not better than the user (like me who experienced this)
> wondering why they cannot zero the counters even when there is no
> traffic?

Can we have something like, stop the forwarding, clear the stats and start
forwarding back?

Yes effect is small but it is for each packet, it doesn't make sense to me to
add extra check for each packet for the rare case of stats reset.

Re: [dpdk-dev] [PATCH v2 5/5] examples/kni: improve zeroing statistics

[Dan Gora]

On Thu, Sep 27, 2018 at 8:40 AM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> >> Not sure about introducing an extra check to datapath for possible error on
> >> stats zero. I am for dropping this patch, what do you think?
> >
> > This is literally adding one instruction to the datapath.  Not even an
> > atomic instruction.  There is no effect on the performance caused by
> > this change.
> >
> > Is that not better than the user (like me who experienced this)
> > wondering why they cannot zero the counters even when there is no
> > traffic?
>
> Can we have something like, stop the forwarding, clear the stats and start
> forwarding back?

This is what is broken.  The stats do not zero because you have the
two worker threads who are constantly performing:

 kni_stats[port_id].rx_packets += num;

and

 kni_stats[port_id].tx_packets += nb_tx;

You know how read/inc/write races work, right?

These are not atomic increments, so the other thread zeroing these
counters is _always_ going to race with these worker threads
overwriting the counters with the old values.

With no traffic it's worse because the worker threads perform these
increments even more often!

> Yes effect is small but it is for each packet, it doesn't make sense to me to
> add extra check for each packet for the rare case of stats reset.

Its not rare at all!  You cannot zero the statistics around 80% of the
time on my machine when there is no traffic.  It's trivial to
reproduce this.  Just run a little traffic, stop the traffic, zero the
stats and check the stats.

If you cannot zero the statistics reliably under any circumstance then
the statistics themselves are worthless and should be removed.  It's
better to have no stats than completely unreliable ones.

Re: [dpdk-dev] [PATCH v2 5/5] examples/kni: improve zeroing statistics

[Ferruh Yigit]

On 9/27/2018 4:53 PM, Dan Gora wrote:
> On Thu, Sep 27, 2018 at 8:40 AM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>>> Not sure about introducing an extra check to datapath for possible error on
>>>> stats zero. I am for dropping this patch, what do you think?
>>>
>>> This is literally adding one instruction to the datapath.  Not even an
>>> atomic instruction.  There is no effect on the performance caused by
>>> this change.
>>>
>>> Is that not better than the user (like me who experienced this)
>>> wondering why they cannot zero the counters even when there is no
>>> traffic?
>>
>> Can we have something like, stop the forwarding, clear the stats and start
>> forwarding back?
> 
> This is what is broken.  The stats do not zero because you have the
> two worker threads who are constantly performing:
> 
>  kni_stats[port_id].rx_packets += num;
> 
> and
> 
>  kni_stats[port_id].tx_packets += nb_tx;
> 
> You know how read/inc/write races work, right?
> 
> These are not atomic increments, so the other thread zeroing these
> counters is _always_ going to race with these worker threads
> overwriting the counters with the old values.
> 
> With no traffic it's worse because the worker threads perform these
> increments even more often!

Dear Dan,

Your implementation doesn't prevent the race when there is traffic, it can be
useful when there is no traffic, that is why my suggestion was in signal
handler, stop worker threads, zero stats, start workers again.
But for traffic case this will cause packet lost so overall may not be desirable.

> 
>> Yes effect is small but it is for each packet, it doesn't make sense to me to
>> add extra check for each packet for the rare case of stats reset.
> 
> Its not rare at all!  You cannot zero the statistics around 80% of the
> time on my machine when there is no traffic.  It's trivial to
> reproduce this.  Just run a little traffic, stop the traffic, zero the
> stats and check the stats.

I was trying to say, packet transfer is millions per second, so the waste is in
that scale, but clear stats will be called in the scale of seconds/minutes? Or
not at all perhaps for some cases?

> 
> If you cannot zero the statistics reliably under any circumstance then
> the statistics themselves are worthless and should be removed.  It's
> better to have no stats than completely unreliable ones.

How much really you rely on sample application stats clear, and stats are still
useful of course since you can use them without reset.

Re: [dpdk-dev] [PATCH v2 5/5] examples/kni: improve zeroing statistics

[Dan Gora]

On Thu, Sep 27, 2018 at 7:04 PM, Ferruh Yigit <ferruh.yigit@intel.com> wrote:

>>
>> With no traffic it's worse because the worker threads perform these
>> increments even more often!
>
> Dear Dan,
>
> Your implementation doesn't prevent the race when there is traffic, it can be

Correct.  To do that would require per-cpu counters or atomic
increments.  That all seems a bit much for a simple example app.

> useful when there is no traffic, that is why my suggestion was in signal
> handler, stop worker threads, zero stats, start workers again.
> But for traffic case this will cause packet lost so overall may not be desirable.

Yeah, of course it would cause packet loss while traffic is running,
that's why I didn't even consider that..

>>> Yes effect is small but it is for each packet, it doesn't make sense to me to
>>> add extra check for each packet for the rare case of stats reset.
>>
>> Its not rare at all!  You cannot zero the statistics around 80% of the
>> time on my machine when there is no traffic.  It's trivial to
>> reproduce this.  Just run a little traffic, stop the traffic, zero the
>> stats and check the stats.
>
> I was trying to say, packet transfer is millions per second, so the waste is in
> that scale, but clear stats will be called in the scale of seconds/minutes? Or
> not at all perhaps for some cases?

Even at 100G 64 byte packets, this one extra instruction is not going
to make any difference.  In fact I would bet money that it will
actually improve performance by decreasing the pressure on that cache
line, which is shared between the tx and rx threads (and cores!
possibly on different sockets!).

Currently you have two cores constantly dirtying the same cache line
every few instructions for no reason other than to read and write back
the same value if no data was tx/rx'd.

>> If you cannot zero the statistics reliably under any circumstance then
>> the statistics themselves are worthless and should be removed.  It's
>> better to have no stats than completely unreliable ones.
>
> How much really you rely on sample application stats clear, and stats are still
> useful of course since you can use them without reset.

Well, you can, but... Say you're doing a data transfer test and you
want to make sure that no packets are lost.  You stop the traffic,
zero the counters, run the test, look at the counters.  Oops, they are
different!  Oh yeah, the silly test app cannot zero the counters, I
forgot... So then you have to go back, find the old stats (if you
checked them first), copy that number into dc, do the subtraction by
hand, etc..

It just seems all a bit much, especially for a toy example application.

That said, this is by no means a hill I want to die on.  If you don't
want to add it, that's fine...

d

[dpdk-dev] [PATCH v3 0/6] kni: add API to set link status on kernel interface

[Dan Gora]

Hi All,

Attached is version 3 of a patchset to add a new API function to
set the link status on kernel interfaces created with the KNI kernel
module.

v3
====
* Use separate function to test rte_kni_update_link() in 'test' app.

* Separate changes to 'test' app into separate patch to facilitate
  possible merge with https://patches.dpdk.org/patch/44730/

* Remove changes to set KNI interfaces to 'up' in example/kni

> v2
> ====
> 
> * Fix bug where "Fixed" and "AutoNeg" were transposed in the link
>   status log message.
> 
> * Add rte_kni_update_link() to rte_kni_version.map
> 
> * Add rte_kni_update_link() tests to kni_autotest
> 
> * Update examples/kni to continuously monitor link status and
>   update the corresponding kernel interface with
>   rte_kni_update_link().
> 
> * Minor improvements to examples/kni: Add log message showing how
>   to show/zero stats.  Improve zeroing statistics.
> 
> Note that checkpatches.sh compains about patch 1/5, but this appears
> to be a bug with check-symbol-change or something.  If I move the
> fragment of the patch modifying rte_kni_version.map to the bottom of
> the patch file, it doesn't complain any more...  I just don't really
> have time to investigate this right now.
  
thanks
dan

Dan Gora (6):
  kni: add API to set link status on kernel interface
  kni: add link status test
  kni: set default carrier state to 'off'
  examples/kni: monitor and update link status continually
  examples/kni: add log msgs to show and clear stats
  examples/kni: improve zeroing statistics

 examples/kni/Makefile              |   2 +
 examples/kni/main.c                |  95 ++++++++++-----------
 kernel/linux/kni/kni_misc.c        |   2 +
 kernel/linux/kni/kni_net.c         |   2 +
 lib/librte_kni/rte_kni.c           |  57 +++++++++++++
 lib/librte_kni/rte_kni.h           |  18 ++++
 lib/librte_kni/rte_kni_version.map |   6 ++
 test/test/test_kni.c               | 131 +++++++++++++++++++++++++++++
 8 files changed, 264 insertions(+), 49 deletions(-)

-- 
2.19.0

[dpdk-dev] [PATCH v3 1/6] kni: add API to set link status on kernel interface

[Dan Gora]

Add a new API function to KNI, rte_kni_update_link() to allow DPDK
applications to update the link status for KNI network interfaces in
the linux kernel.

Signed-off-by: Dan Gora <dg@adax.com>
---
 lib/librte_kni/rte_kni.c           | 57 ++++++++++++++++++++++++++++++
 lib/librte_kni/rte_kni.h           | 18 ++++++++++
 lib/librte_kni/rte_kni_version.map |  6 ++++
 3 files changed, 81 insertions(+)

diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c
index 65f6a2b03..0c3e17dbb 100644
--- a/lib/librte_kni/rte_kni.c
+++ b/lib/librte_kni/rte_kni.c
@@ -790,6 +790,63 @@ rte_kni_unregister_handlers(struct rte_kni *kni)
 
 	return 0;
 }
+
+int __rte_experimental
+rte_kni_update_link(struct rte_kni *kni, struct rte_eth_link *link)
+{
+	char path[64];
+	char carrier[2];
+	const char *new_carrier;
+	int fd, ret;
+
+	if (kni == NULL || link == NULL)
+		return -1;
+
+	snprintf(path, sizeof(path), "/sys/devices/virtual/net/%s/carrier",
+		kni->name);
+
+	fd = open(path, O_RDWR);
+	if (fd == -1) {
+		RTE_LOG(ERR, KNI, "Failed to open file: %s.\n", path);
+		return -1;
+	}
+
+	ret = read(fd, carrier, 2);
+	if (ret < 1) {
+		/* Cannot read carrier until interface is marked
+		 * 'up', so don't log an error.
+		 */
+		close(fd);
+		return -1;
+	}
+
+	new_carrier = (link->link_status == ETH_LINK_UP) ? "1" : "0";
+	ret = write(fd, new_carrier, 1);
+	if (ret < 1) {
+		RTE_LOG(ERR, KNI, "Failed to write file: %s.\n", path);
+		close(fd);
+		return -1;
+	}
+
+	if (strncmp(carrier, new_carrier, 1)) {
+		if (link->link_status == ETH_LINK_UP) {
+			RTE_LOG(INFO, KNI, "%s NIC Link is Up %d Mbps %s %s.\n",
+				kni->name,
+				link->link_speed,
+				link->link_autoneg ? "(AutoNeg)" : "(Fixed)",
+				link->link_duplex ?
+					"Full Duplex" : "Half Duplex");
+		} else {
+			RTE_LOG(INFO, KNI, "%s NIC Link is Down.\n",
+				kni->name);
+		}
+	}
+
+	close(fd);
+
+	return 0;
+}
+
 void
 rte_kni_close(void)
 {
diff --git a/lib/librte_kni/rte_kni.h b/lib/librte_kni/rte_kni.h
index 99055e2c2..4118ae97a 100644
--- a/lib/librte_kni/rte_kni.h
+++ b/lib/librte_kni/rte_kni.h
@@ -21,6 +21,7 @@
 #include <rte_memory.h>
 #include <rte_mempool.h>
 #include <rte_ether.h>
+#include <rte_ethdev.h>
 
 #include <exec-env/rte_kni_common.h>
 
@@ -228,6 +229,23 @@ int rte_kni_register_handlers(struct rte_kni *kni, struct rte_kni_ops *ops);
  */
 int rte_kni_unregister_handlers(struct rte_kni *kni);
 
+/**
+ * Update link status info for KNI port.
+ *
+ * Update the linkup/linkdown status of a KNI interface in the kernel.
+ *
+ * @param kni
+ *  pointer to struct rte_kni.
+ * @param link
+ *  pointer to struct rte_eth_link containing new interface status.
+ *
+ * @return
+ *  On success: 0
+ *  On failure: -1
+ */
+int __rte_experimental
+rte_kni_update_link(struct rte_kni *kni, struct rte_eth_link *link);
+
 /**
  *  Close KNI device.
  */
diff --git a/lib/librte_kni/rte_kni_version.map b/lib/librte_kni/rte_kni_version.map
index acd515eb0..c877dc6aa 100644
--- a/lib/librte_kni/rte_kni_version.map
+++ b/lib/librte_kni/rte_kni_version.map
@@ -15,3 +15,9 @@ DPDK_2.0 {
 
 	local: *;
 };
+
+EXPERIMENTAL {
+	global:
+
+	rte_kni_update_link;
+};
-- 
2.19.0

[dpdk-dev] [PATCH v3 2/6] kni: add link status test

[Dan Gora]

Test changing the link, speed, duplex, etc. status of the KNI interface
with rte_kni_update_link().

Signed-off-by: Dan Gora <dg@adax.com>
---
 test/test/test_kni.c | 131 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 131 insertions(+)

diff --git a/test/test/test_kni.c b/test/test/test_kni.c
index 3dcadcebd..138bc9686 100644
--- a/test/test/test_kni.c
+++ b/test/test/test_kni.c
@@ -118,6 +118,133 @@ kni_change_mtu(uint16_t port_id, unsigned int new_mtu)
 					 port_id, kni_pkt_mtu);
 	return 0;
 }
+
+static int
+test_kni_link_change(void)
+{
+	struct rte_eth_link link;
+	int ret;
+	int pid;
+
+	pid = fork();
+	if (pid < 0) {
+		printf("Error: Failed to fork a process\n");
+		return -1;
+	}
+
+	if (pid == 0) {
+		printf("Starting KNI Link status change tests.\n");
+		if (system(IFCONFIG TEST_KNI_PORT" up") == -1) {
+			ret = -1;
+			goto error;
+		}
+
+		link.link_speed = 10;
+		link.link_status = ETH_LINK_UP;
+		link.link_autoneg = ETH_LINK_AUTONEG;
+		link.link_duplex = ETH_LINK_FULL_DUPLEX;
+		ret = rte_kni_update_link(test_kni_ctx, &link);
+		if (ret != 0) {
+			printf("Failed to change link state to "
+				"Up/10Mbps/AutoNeg/Full ret=%d.\n", ret);
+			goto error;
+		}
+		rte_delay_ms(1000);
+
+		link.link_speed = 0;
+		link.link_status = ETH_LINK_DOWN;
+		link.link_autoneg = ETH_LINK_FIXED;
+		link.link_duplex = ETH_LINK_FULL_DUPLEX;
+		ret = rte_kni_update_link(test_kni_ctx, &link);
+		if (ret != 0) {
+			printf("Failed to change link state to Down ret=%d.\n",
+				ret);
+			goto error;
+		}
+		rte_delay_ms(1000);
+
+		link.link_speed = 1000;
+		link.link_status = ETH_LINK_UP;
+		link.link_autoneg = ETH_LINK_AUTONEG;
+		link.link_duplex = ETH_LINK_HALF_DUPLEX;
+		ret = rte_kni_update_link(test_kni_ctx, &link);
+		if (ret != 0) {
+			printf("Failed to change link state to "
+				"Up/1Gbps/AutoNeg/Half ret=%d.\n", ret);
+			goto error;
+		}
+		rte_delay_ms(1000);
+
+		link.link_speed = 0;
+		link.link_status = ETH_LINK_DOWN;
+		link.link_autoneg = ETH_LINK_FIXED;
+		link.link_duplex = ETH_LINK_FULL_DUPLEX;
+		ret = rte_kni_update_link(test_kni_ctx, &link);
+		if (ret != 0) {
+			printf("Failed to change link state to Down ret=%d.\n",
+				ret);
+			goto error;
+		}
+		rte_delay_ms(1000);
+
+		link.link_speed = 40000;
+		link.link_status = ETH_LINK_UP;
+		link.link_autoneg = ETH_LINK_FIXED;
+		link.link_duplex = ETH_LINK_FULL_DUPLEX;
+		ret = rte_kni_update_link(test_kni_ctx, &link);
+		if (ret != 0) {
+			printf("Failed to change link state to "
+				"Up/40Gbps/Fixed/Full ret=%d.\n", ret);
+			goto error;
+		}
+		rte_delay_ms(1000);
+
+		link.link_speed = 0;
+		link.link_status = ETH_LINK_DOWN;
+		link.link_autoneg = ETH_LINK_FIXED;
+		link.link_duplex = ETH_LINK_FULL_DUPLEX;
+		ret = rte_kni_update_link(test_kni_ctx, &link);
+		if (ret != 0) {
+			printf("Failed to change link state to Down ret=%d.\n",
+				ret);
+			goto error;
+		}
+		rte_delay_ms(1000);
+
+		link.link_speed = 100000;
+		link.link_status = ETH_LINK_UP;
+		link.link_autoneg = ETH_LINK_FIXED;
+		link.link_duplex = ETH_LINK_FULL_DUPLEX;
+		ret = rte_kni_update_link(test_kni_ctx, &link);
+		if (ret != 0) {
+			printf("Failed to change link state to "
+				"Up/100Gbps/Fixed/Full ret=%d.\n", ret);
+			goto error;
+		}
+		rte_delay_ms(1000);
+
+error:
+		if (system(IFCONFIG TEST_KNI_PORT" down") == -1)
+			ret = -1;
+
+		printf("KNI Link status change tests: %s.\n",
+			(ret == 0) ? "Passed" : "Failed");
+		exit(ret);
+	} else {
+		int p_ret, status;
+
+		while (1) {
+			p_ret = waitpid(pid, &status, WNOHANG);
+			if (p_ret != 0) {
+				if (WIFEXITED(status))
+					return WEXITSTATUS(status);
+				return -1;
+			}
+			rte_delay_ms(10);
+			rte_kni_handle_request(test_kni_ctx);
+		}
+	}
+}
 /**
  * This loop fully tests the basic functions of KNI. e.g. transmitting,
  * receiving to, from kernel space, and kernel requests.
@@ -401,6 +528,10 @@ test_kni_processing(uint16_t port_id, struct rte_mempool *mp)
 		goto fail_kni;
 	}
 
+	ret = test_kni_link_change();
+	if (ret != 0)
+		goto fail_kni;
+
 	rte_eal_mp_remote_launch(test_kni_loop, NULL, CALL_MASTER);
 	RTE_LCORE_FOREACH_SLAVE(i) {
 		if (rte_eal_wait_lcore(i) < 0) {
-- 
2.19.0

[dpdk-dev] [PATCH v3 3/6] kni: set default carrier state to 'off'

[Dan Gora]

Set the carrier state to 'off' when the interface is instantiated
or when it is marked 'up' or 'down'.  This is necessary to set the
interface to a known operational state until the carrier state is
changed with rte_kni_update_link().

Signed-off-by: Dan Gora <dg@adax.com>
---
 kernel/linux/kni/kni_misc.c | 2 ++
 kernel/linux/kni/kni_net.c  | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/kernel/linux/kni/kni_misc.c b/kernel/linux/kni/kni_misc.c
index fa69f8e63..45649499d 100644
--- a/kernel/linux/kni/kni_misc.c
+++ b/kernel/linux/kni/kni_misc.c
@@ -466,6 +466,8 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_num,
 		return -ENODEV;
 	}
 
+	netif_carrier_off(net_dev);
+
 	ret = kni_run_thread(knet, kni, dev_info.force_bind);
 	if (ret != 0)
 		return ret;
diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c
index 7fcfa106c..1f8ba0700 100644
--- a/kernel/linux/kni/kni_net.c
+++ b/kernel/linux/kni/kni_net.c
@@ -133,6 +133,7 @@ kni_net_open(struct net_device *dev)
 	struct kni_dev *kni = netdev_priv(dev);
 
 	netif_start_queue(dev);
+	netif_carrier_off(dev);
 
 	memset(&req, 0, sizeof(req));
 	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
@@ -152,6 +153,7 @@ kni_net_release(struct net_device *dev)
 	struct kni_dev *kni = netdev_priv(dev);
 
 	netif_stop_queue(dev); /* can't transmit any more */
+	netif_carrier_off(dev);
 
 	memset(&req, 0, sizeof(req));
 	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
-- 
2.19.0

[dpdk-dev] [PATCH v3 4/6] examples/kni: monitor and update link status continually

[Dan Gora]

Update KNI example to continuously monitor the Ethernet link status
of the physical link and update the link status of the corresponding
interfaces with rte_kni_update_link().

Signed-off-by: Dan Gora <dg@adax.com>
---
 examples/kni/Makefile |  2 ++
 examples/kni/main.c   | 73 +++++++++++++++++--------------------------
 2 files changed, 31 insertions(+), 44 deletions(-)

diff --git a/examples/kni/Makefile b/examples/kni/Makefile
index 7e19d2e2a..dd90d7d73 100644
--- a/examples/kni/Makefile
+++ b/examples/kni/Makefile
@@ -20,6 +20,7 @@ static: build/$(APP)-static
 
 PC_FILE := $(shell pkg-config --path libdpdk)
 CFLAGS += -O3 $(shell pkg-config --cflags libdpdk)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk)
 LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk)
 
@@ -54,6 +55,7 @@ please change the definition of the RTE_TARGET environment variable)
 endif
 
 CFLAGS += -O3
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 CFLAGS += $(WERROR_FLAGS)
 
 include $(RTE_SDK)/mk/rte.extapp.mk
diff --git a/examples/kni/main.c b/examples/kni/main.c
index 80c401c51..be721b9cc 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -107,6 +107,8 @@ static uint32_t ports_mask = 0;
 /* Ports set in promiscuous mode off by default. */
 static int promiscuous_on = 0;
 
+static int kni_running;
+
 /* Structure type for recording kni interface specific stats */
 struct kni_interface_stats {
 	/* number of pkts received from NIC, and sent to KNI */
@@ -619,59 +621,31 @@ init_port(uint16_t port)
 		rte_eth_promiscuous_enable(port);
 }
 
-/* Check the link status of all ports in up to 9s, and print them finally */
-static void
-check_all_ports_link_status(uint32_t port_mask)
+/*
+ * Monitor the link status of all ports and update the
+ * corresponding KNI interface(s)
+ */
+static void *
+check_all_ports_link_status(void *arg)
 {
-#define CHECK_INTERVAL 100 /* 100ms */
-#define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */
 	uint16_t portid;
-	uint8_t count, all_ports_up, print_flag = 0;
 	struct rte_eth_link link;
+	unsigned int i;
+	struct kni_port_params **p = kni_port_params_array;
+	(void) arg;
 
-	printf("\nChecking link status\n");
-	fflush(stdout);
-	for (count = 0; count <= MAX_CHECK_TIME; count++) {
-		all_ports_up = 1;
+	while (kni_running) {
+		rte_delay_ms(100);
 		RTE_ETH_FOREACH_DEV(portid) {
-			if ((port_mask & (1 << portid)) == 0)
+			if ((ports_mask & (1 << portid)) == 0)
 				continue;
 			memset(&link, 0, sizeof(link));
 			rte_eth_link_get_nowait(portid, &link);
-			/* print link status if flag set */
-			if (print_flag == 1) {
-				if (link.link_status)
-					printf(
-					"Port%d Link Up - speed %uMbps - %s\n",
-						portid, link.link_speed,
-				(link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
-					("full-duplex") : ("half-duplex\n"));
-				else
-					printf("Port %d Link Down\n", portid);
-				continue;
-			}
-			/* clear all_ports_up flag if any link down */
-			if (link.link_status == ETH_LINK_DOWN) {
-				all_ports_up = 0;
-				break;
-			}
-		}
-		/* after finally printing all link status, get out */
-		if (print_flag == 1)
-			break;
-
-		if (all_ports_up == 0) {
-			printf(".");
-			fflush(stdout);
-			rte_delay_ms(CHECK_INTERVAL);
-		}
-
-		/* set the print_flag if all ports up or timeout */
-		if (all_ports_up == 1 || count == (MAX_CHECK_TIME - 1)) {
-			print_flag = 1;
-			printf("done\n");
+			for (i = 0; i < p[portid]->nb_kni; i++)
+				rte_kni_update_link(p[portid]->kni[i], &link);
 		}
 	}
+	return NULL;
 }
 
 /* Callback for request of changing MTU */
@@ -893,6 +867,8 @@ main(int argc, char** argv)
 	int ret;
 	uint16_t nb_sys_ports, port;
 	unsigned i;
+	void *retval;
+	pthread_t kni_link_tid;
 
 	/* Associate signal_hanlder function with USR signals */
 	signal(SIGUSR1, signal_handler);
@@ -947,7 +923,14 @@ main(int argc, char** argv)
 
 		kni_alloc(port);
 	}
-	check_all_ports_link_status(ports_mask);
+
+	kni_running = 1;
+	ret = rte_ctrl_thread_create(&kni_link_tid,
+				     "KNI link status check", NULL,
+				     check_all_ports_link_status, NULL);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Could not create link status thread!\n");
 
 	/* Launch per-lcore function on every lcore */
 	rte_eal_mp_remote_launch(main_loop, NULL, CALL_MASTER);
@@ -955,6 +938,8 @@ main(int argc, char** argv)
 		if (rte_eal_wait_lcore(i) < 0)
 			return -1;
 	}
+	kni_running = 0;
+	pthread_join(kni_link_tid, &retval);
 
 	/* Release resources */
 	RTE_ETH_FOREACH_DEV(port) {
-- 
2.19.0

[dpdk-dev] [PATCH v3 5/6] examples/kni: add log msgs to show and clear stats

[Dan Gora]

Add logging messages showing the commands necessary for the user to
have the application display and zero the statistics.

Signed-off-by: Dan Gora <dg@adax.com>
---
 examples/kni/main.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/examples/kni/main.c b/examples/kni/main.c
index be721b9cc..357cfd6fc 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -171,14 +171,13 @@ signal_handler(int signum)
 	/* When we receive a USR2 signal, reset stats */
 	if (signum == SIGUSR2) {
 		memset(&kni_stats, 0, sizeof(kni_stats));
-		printf("\n**Statistics have been reset**\n");
+		printf("\n** Statistics have been reset **\n");
 		return;
 	}
 
 	/* When we receive a RTMIN or SIGINT signal, stop kni processing */
 	if (signum == SIGRTMIN || signum == SIGINT){
-		printf("SIGRTMIN is received, and the KNI processing is "
-							"going to stop\n");
+		printf("\nSIGRTMIN/SIGINT received. KNI processing stopping.\n");
 		rte_atomic32_inc(&kni_stop);
 		return;
         }
@@ -869,6 +868,7 @@ main(int argc, char** argv)
 	unsigned i;
 	void *retval;
 	pthread_t kni_link_tid;
+	int pid;
 
 	/* Associate signal_hanlder function with USR signals */
 	signal(SIGUSR1, signal_handler);
@@ -924,6 +924,16 @@ main(int argc, char** argv)
 		kni_alloc(port);
 	}
 
+	pid = getpid();
+	RTE_LOG(INFO, APP, "========================\n");
+	RTE_LOG(INFO, APP, "KNI Running\n");
+	RTE_LOG(INFO, APP, "kill -SIGUSR1 %d\n", pid);
+	RTE_LOG(INFO, APP, "    Show KNI Statistics.\n");
+	RTE_LOG(INFO, APP, "kill -SIGUSR2 %d\n", pid);
+	RTE_LOG(INFO, APP, "    Zero KNI Statistics.\n");
+	RTE_LOG(INFO, APP, "========================\n");
+	fflush(stdout);
+
 	kni_running = 1;
 	ret = rte_ctrl_thread_create(&kni_link_tid,
 				     "KNI link status check", NULL,
-- 
2.19.0

[dpdk-dev] [PATCH v3 6/6] examples/kni: improve zeroing statistics

[Dan Gora]

The worker threads incrementing the rx/tx_packets race with the signal
handler from the main thread zeroing the entire statistics structure.
This can cause the statistics to fail to be zeroed, even when there
is no traffic on those interfaces.

Improve zeroing the statistics by only incrementing rx/tx_packets
in worker threads by a non-zero amount.  This limits the race to the
periods in which traffic is actually being received or transmitted.

Signed-off-by: Dan Gora <dg@adax.com>
---
 examples/kni/main.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/examples/kni/main.c b/examples/kni/main.c
index 357cfd6fc..cda23b028 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -223,7 +223,8 @@ kni_ingress(struct kni_port_params *p)
 		}
 		/* Burst tx to kni */
 		num = rte_kni_tx_burst(p->kni[i], pkts_burst, nb_rx);
-		kni_stats[port_id].rx_packets += num;
+		if (num)
+			kni_stats[port_id].rx_packets += num;
 
 		rte_kni_handle_request(p->kni[i]);
 		if (unlikely(num < nb_rx)) {
@@ -260,7 +261,8 @@ kni_egress(struct kni_port_params *p)
 		}
 		/* Burst tx to eth */
 		nb_tx = rte_eth_tx_burst(port_id, 0, pkts_burst, (uint16_t)num);
-		kni_stats[port_id].tx_packets += nb_tx;
+		if (nb_tx)
+			kni_stats[port_id].tx_packets += nb_tx;
 		if (unlikely(nb_tx < num)) {
 			/* Free mbufs not tx to NIC */
 			kni_burst_free_mbufs(&pkts_burst[nb_tx], num - nb_tx);
-- 
2.19.0

Re: [dpdk-dev] [PATCH v3 0/6] kni: add API to set link status on kernel interface

[Ferruh Yigit]

On 9/27/2018 1:32 AM, Dan Gora wrote:
> Hi All,
> 
> Attached is version 3 of a patchset to add a new API function to
> set the link status on kernel interfaces created with the KNI kernel
> module.
> 
> v3
> ====
> * Use separate function to test rte_kni_update_link() in 'test' app.
> 
> * Separate changes to 'test' app into separate patch to facilitate
>   possible merge with https://patches.dpdk.org/patch/44730/
> 
> * Remove changes to set KNI interfaces to 'up' in example/kni
> 
>> v2
>> ====
>>
>> * Fix bug where "Fixed" and "AutoNeg" were transposed in the link
>>   status log message.
>>
>> * Add rte_kni_update_link() to rte_kni_version.map
>>
>> * Add rte_kni_update_link() tests to kni_autotest
>>
>> * Update examples/kni to continuously monitor link status and
>>   update the corresponding kernel interface with
>>   rte_kni_update_link().
>>
>> * Minor improvements to examples/kni: Add log message showing how
>>   to show/zero stats.  Improve zeroing statistics.
>>
>> Note that checkpatches.sh compains about patch 1/5, but this appears
>> to be a bug with check-symbol-change or something.  If I move the
>> fragment of the patch modifying rte_kni_version.map to the bottom of
>> the patch file, it doesn't complain any more...  I just don't really
>> have time to investigate this right now.
>   
> thanks
> dan
> 
> Dan Gora (6):
>   kni: add API to set link status on kernel interface
>   kni: add link status test
>   kni: set default carrier state to 'off'
>   examples/kni: monitor and update link status continually
>   examples/kni: add log msgs to show and clear stats
>   examples/kni: improve zeroing statistics

Hi Dan,

We are a little away to integration deadline, it is good to clarify the status
of the patchset.

There are a few change requests to this patchset:
1- 4/6, there is an open on adding command line option to control monitor/set
link status.
2- Dropping 6/6, I guess you already agreed on this.
3- 1/6, to have or not the log message.

I would like to see the patchset in the release, what do you think about above
actions?

Re: [dpdk-dev] [PATCH v3 0/6] kni: add API to set link status on kernel interface

[Dan Gora]

On Wed, Oct 10, 2018 at 11:16 AM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>
> On 9/27/2018 1:32 AM, Dan Gora wrote:
> > Hi All,
> >
> > Attached is version 3 of a patchset to add a new API function to
> > set the link status on kernel interfaces created with the KNI kernel
> > module.
> >
> > v3
> > ====
> > * Use separate function to test rte_kni_update_link() in 'test' app.
> >
> > * Separate changes to 'test' app into separate patch to facilitate
> >   possible merge with https://patches.dpdk.org/patch/44730/
> >
> > * Remove changes to set KNI interfaces to 'up' in example/kni
> >
> >> v2
> >> ====
> >>
> >> * Fix bug where "Fixed" and "AutoNeg" were transposed in the link
> >>   status log message.
> >>
> >> * Add rte_kni_update_link() to rte_kni_version.map
> >>
> >> * Add rte_kni_update_link() tests to kni_autotest
> >>
> >> * Update examples/kni to continuously monitor link status and
> >>   update the corresponding kernel interface with
> >>   rte_kni_update_link().
> >>
> >> * Minor improvements to examples/kni: Add log message showing how
> >>   to show/zero stats.  Improve zeroing statistics.
> >>
> >> Note that checkpatches.sh compains about patch 1/5, but this appears
> >> to be a bug with check-symbol-change or something.  If I move the
> >> fragment of the patch modifying rte_kni_version.map to the bottom of
> >> the patch file, it doesn't complain any more...  I just don't really
> >> have time to investigate this right now.
> >
> > thanks
> > dan
> >
> > Dan Gora (6):
> >   kni: add API to set link status on kernel interface
> >   kni: add link status test
> >   kni: set default carrier state to 'off'
> >   examples/kni: monitor and update link status continually
> >   examples/kni: add log msgs to show and clear stats
> >   examples/kni: improve zeroing statistics
>
> Hi Dan,
>
> We are a little away to integration deadline, it is good to clarify the status
> of the patchset.
>
> There are a few change requests to this patchset:
> 1- 4/6, there is an open on adding command line option to control monitor/set
> link status.

No, I don't have any plans to add a command line option to do this.
Again, there is no reason for a command line option.

> 2- Dropping 6/6, I guess you already agreed on this.

No, I showed you that that patch to fix zeroing the statistics would
actually increase performance.  You just never responded to that
email.

> 3- 1/6, to have or not the log message.

I included a proposal in my last email.  Please take a look at that and respond.

> I would like to see the patchset in the release, what do you think about above
> actions?

I'm incredibly frustrated with this whole process to be honest...

d

Re: [dpdk-dev] [PATCH v3 0/6] kni: add API to set link status on kernel interface

[Ferruh Yigit]

On 10/10/2018 4:01 PM, Dan Gora wrote:
> On Wed, Oct 10, 2018 at 11:16 AM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>>
>> On 9/27/2018 1:32 AM, Dan Gora wrote:
>>> Hi All,
>>>
>>> Attached is version 3 of a patchset to add a new API function to
>>> set the link status on kernel interfaces created with the KNI kernel
>>> module.
>>>
>>> v3
>>> ====
>>> * Use separate function to test rte_kni_update_link() in 'test' app.
>>>
>>> * Separate changes to 'test' app into separate patch to facilitate
>>>   possible merge with https://patches.dpdk.org/patch/44730/
>>>
>>> * Remove changes to set KNI interfaces to 'up' in example/kni
>>>
>>>> v2
>>>> ====
>>>>
>>>> * Fix bug where "Fixed" and "AutoNeg" were transposed in the link
>>>>   status log message.
>>>>
>>>> * Add rte_kni_update_link() to rte_kni_version.map
>>>>
>>>> * Add rte_kni_update_link() tests to kni_autotest
>>>>
>>>> * Update examples/kni to continuously monitor link status and
>>>>   update the corresponding kernel interface with
>>>>   rte_kni_update_link().
>>>>
>>>> * Minor improvements to examples/kni: Add log message showing how
>>>>   to show/zero stats.  Improve zeroing statistics.
>>>>
>>>> Note that checkpatches.sh compains about patch 1/5, but this appears
>>>> to be a bug with check-symbol-change or something.  If I move the
>>>> fragment of the patch modifying rte_kni_version.map to the bottom of
>>>> the patch file, it doesn't complain any more...  I just don't really
>>>> have time to investigate this right now.
>>>
>>> thanks
>>> dan
>>>
>>> Dan Gora (6):
>>>   kni: add API to set link status on kernel interface
>>>   kni: add link status test
>>>   kni: set default carrier state to 'off'
>>>   examples/kni: monitor and update link status continually
>>>   examples/kni: add log msgs to show and clear stats
>>>   examples/kni: improve zeroing statistics
>>
>> Hi Dan,
>>
>> We are a little away to integration deadline, it is good to clarify the status
>> of the patchset.
>>
>> There are a few change requests to this patchset:
>> 1- 4/6, there is an open on adding command line option to control monitor/set
>> link status.
> 
> No, I don't have any plans to add a command line option to do this.
> Again, there is no reason for a command line option.
> 
>> 2- Dropping 6/6, I guess you already agreed on this.
> 
> No, I showed you that that patch to fix zeroing the statistics would
> actually increase performance.  You just never responded to that
> email.
> 
>> 3- 1/6, to have or not the log message.
> 
> I included a proposal in my last email.  Please take a look at that and respond.
> 
>> I would like to see the patchset in the release, what do you think about above
>> actions?
> 
> I'm incredibly frustrated with this whole process to be honest...

Please don't be so, you are spending time/effort to improve an open source
project which is great, thank you again.

Why don't we take this as incremental steps, first get agreed part in a
patchset. 4/6 doesn't fit this but I can implement the command line part for you
if you agree, what do you think?

Re: [dpdk-dev] [PATCH v3 0/6] kni: add API to set link status on kernel interface

[Dan Gora]

On Wed, Oct 10, 2018 at 8:00 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> > I'm incredibly frustrated with this whole process to be honest...
>
> Please don't be so, you are spending time/effort to improve an open source
> project which is great, thank you again.
>
> Why don't we take this as incremental steps, first get agreed part in a
> patchset. 4/6 doesn't fit this but I can implement the command line part for you
> if you agree, what do you think?

I can work on it, but it might take a couple of days until I have
time.  What do you think about my proposal for
patch 1/4?  (see the end of 'Message-ID:
<CAGyogRZJmAWGXX0kXk+k69rbDbRqEJgS0H_Vw8h43-qFQ_ptEQ@mail.gmail.com>',
4 emails back in this thread..)

I can make it work without the log message in the API function, but I
need some way to get the previous status.  See my proposals in that
email above..

d

[dpdk-dev] [PATCH v4 0/6] kni: add API to set link status on kernel interface

[Dan Gora]

Hi All,

Attached is version 4 of a patchset to add a new API function to
set the link status on kernel interfaces created with the KNI kernel
module.

v4
====
* Rework rte_kni_update_link to only take linkup/linkdown as parameter,
  return previous link state, and remove log messages.

* Update patch to set default carrier state to make default carrier
  state configurable by passing the 'carrier=[on|off]' option to
  the rte_kni kernel module.  This is necessary in order to allow
  applications which use KNI as pure virtual interfaces without
  corresponding physical ethernet port to use the interfaces without
  having to set the carrier state to 'on' via rte_kni_update_link()
  or by writing to /sys/devices/virtual/net/<ifaceX>/carrier.
  Note that the default is 'off'.

* Add command line flag '-m' to examples/kni to continuously monitor
  and update the KNI interface link status according to the link
  status of the corresponding physical ethernet port.


> v3
> ====
> * Use separate function to test rte_kni_update_link() in 'test' app.
> 
> * Separate changes to 'test' app into separate patch to facilitate
>   possible merge with https://patches.dpdk.org/patch/44730/
> 
> * Remove changes to set KNI interfaces to 'up' in example/kni
> 
> v2
> ====
> 
> * Fix bug where "Fixed" and "AutoNeg" were transposed in the link
>   status log message.
> 
> * Add rte_kni_update_link() to rte_kni_version.map
> 
> * Add rte_kni_update_link() tests to kni_autotest
> 
> * Update examples/kni to continuously monitor link status and
>   update the corresponding kernel interface with
>   rte_kni_update_link().
> 
> * Minor improvements to examples/kni: Add log message showing how
>   to show/zero stats.  Improve zeroing statistics.
> 
> Note that checkpatches.sh compains about patch 1/5, but this appears
> to be a bug with check-symbol-change or something.  If I move the
> fragment of the patch modifying rte_kni_version.map to the bottom of
> the patch file, it doesn't complain any more...  I just don't really
> have time to investigate this right now.
  
thanks
dan


Dan Gora (6):
  kni: add API to set link status on kernel interface
  kni: add link status test
  kni: set default carrier state of interface
  examples/kni: monitor and update link status continually
  examples/kni: add log msgs to show and clear stats
  examples/kni: improve zeroing statistics

 examples/kni/Makefile              |  2 +
 examples/kni/main.c                | 92 +++++++++++++++++++++++++++---
 kernel/linux/kni/kni_dev.h         |  3 +
 kernel/linux/kni/kni_misc.c        | 40 +++++++++++++
 kernel/linux/kni/kni_net.c         |  5 ++
 lib/librte_kni/rte_kni.c           | 41 +++++++++++++
 lib/librte_kni/rte_kni.h           | 20 +++++++
 lib/librte_kni/rte_kni_version.map |  6 ++
 test/test/test_kni.c               | 77 +++++++++++++++++++++++++
 9 files changed, 279 insertions(+), 7 deletions(-)

-- 
2.19.0

Re: [dpdk-dev] [PATCH v4 0/6] kni: add API to set link status on kernel interface

[Ferruh Yigit]

On 10/17/2018 2:04 AM, Dan Gora wrote:
> Hi All,
> 
> Attached is version 4 of a patchset to add a new API function to
> set the link status on kernel interfaces created with the KNI kernel
> module.
> 
> v4
> ====
> * Rework rte_kni_update_link to only take linkup/linkdown as parameter,
>   return previous link state, and remove log messages.
> 
> * Update patch to set default carrier state to make default carrier
>   state configurable by passing the 'carrier=[on|off]' option to
>   the rte_kni kernel module.  This is necessary in order to allow
>   applications which use KNI as pure virtual interfaces without
>   corresponding physical ethernet port to use the interfaces without
>   having to set the carrier state to 'on' via rte_kni_update_link()
>   or by writing to /sys/devices/virtual/net/<ifaceX>/carrier.
>   Note that the default is 'off'.
> 
> * Add command line flag '-m' to examples/kni to continuously monitor
>   and update the KNI interface link status according to the link
>   status of the corresponding physical ethernet port.
> 
> 
>> v3
>> ====
>> * Use separate function to test rte_kni_update_link() in 'test' app.
>>
>> * Separate changes to 'test' app into separate patch to facilitate
>>   possible merge with https://patches.dpdk.org/patch/44730/
>>
>> * Remove changes to set KNI interfaces to 'up' in example/kni
>>
>> v2
>> ====
>>
>> * Fix bug where "Fixed" and "AutoNeg" were transposed in the link
>>   status log message.
>>
>> * Add rte_kni_update_link() to rte_kni_version.map
>>
>> * Add rte_kni_update_link() tests to kni_autotest
>>
>> * Update examples/kni to continuously monitor link status and
>>   update the corresponding kernel interface with
>>   rte_kni_update_link().
>>
>> * Minor improvements to examples/kni: Add log message showing how
>>   to show/zero stats.  Improve zeroing statistics.
>>
>> Note that checkpatches.sh compains about patch 1/5, but this appears
>> to be a bug with check-symbol-change or something.  If I move the
>> fragment of the patch modifying rte_kni_version.map to the bottom of
>> the patch file, it doesn't complain any more...  I just don't really
>> have time to investigate this right now.
>   
> thanks
> dan
> 
> 
> Dan Gora (6):
>   kni: add API to set link status on kernel interface
>   kni: add link status test
>   kni: set default carrier state of interface
>   examples/kni: monitor and update link status continually
>   examples/kni: add log msgs to show and clear stats
>   examples/kni: improve zeroing statistics

Hi Dan,

Overall looks good to me, thanks for the updates.

Only it is missing some documentation, I think it is good to document:
1- KNI interface by default carrier will be off
2- New kni module parameter "carrier"
3- New sample application "-m" parameter.

I think 2 & 3 can be documented in "Kernel NIC Interface Sample Application",
unfortunately module parameters seems only documented here.

And 1 in "Kernel NIC Interface", perhaps in "KNI Creation and Deletion" section
as a detail to creation part.

Please embed documentation updates into patches that does the implementation,
instead of separate documentation patch.

And there is an issue on 3/6 to fix.

Thanks,
ferruh

[dpdk-dev] [PATCH v4 1/6] kni: add API to set link status on kernel interface

[Dan Gora]

Add a new API function to KNI, rte_kni_update_link() to allow DPDK
applications to update the link status for KNI network interfaces in
the linux kernel.

Signed-off-by: Dan Gora <dg@adax.com>
---
 lib/librte_kni/rte_kni.c           | 41 ++++++++++++++++++++++++++++++
 lib/librte_kni/rte_kni.h           | 20 +++++++++++++++
 lib/librte_kni/rte_kni_version.map |  6 +++++
 3 files changed, 67 insertions(+)

diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c
index b8edd40f4..c9726d4f8 100644
--- a/lib/librte_kni/rte_kni.c
+++ b/lib/librte_kni/rte_kni.c
@@ -717,6 +717,47 @@ rte_kni_unregister_handlers(struct rte_kni *kni)
 
 	return 0;
 }
+
+int __rte_experimental
+rte_kni_update_link(struct rte_kni *kni, unsigned int linkup)
+{
+	char path[64];
+	char old_carrier[2];
+	const char *new_carrier;
+	int old_linkup;
+	int fd, ret;
+
+	if (kni == NULL)
+		return -1;
+
+	snprintf(path, sizeof(path), "/sys/devices/virtual/net/%s/carrier",
+		kni->name);
+
+	fd = open(path, O_RDWR);
+	if (fd == -1) {
+		RTE_LOG(ERR, KNI, "Failed to open file: %s.\n", path);
+		return -1;
+	}
+
+	ret = read(fd, old_carrier, 2);
+	if (ret < 1) {
+		close(fd);
+		return -1;
+	}
+	old_linkup = (old_carrier[0] == '1');
+
+	new_carrier = linkup ? "1" : "0";
+	ret = write(fd, new_carrier, 1);
+	if (ret < 1) {
+		RTE_LOG(ERR, KNI, "Failed to write file: %s.\n", path);
+		close(fd);
+		return -1;
+	}
+
+	close(fd);
+	return old_linkup;
+}
+
 void
 rte_kni_close(void)
 {
diff --git a/lib/librte_kni/rte_kni.h b/lib/librte_kni/rte_kni.h
index 601abdfc6..02ca43b4b 100644
--- a/lib/librte_kni/rte_kni.h
+++ b/lib/librte_kni/rte_kni.h
@@ -232,6 +232,26 @@ int rte_kni_register_handlers(struct rte_kni *kni, struct rte_kni_ops *ops);
  */
 int rte_kni_unregister_handlers(struct rte_kni *kni);
 
+/**
+ * Update link carrier state for KNI port.
+ *
+ * Update the linkup/linkdown state of a KNI interface in the kernel.
+ *
+ * @param kni
+ *  pointer to struct rte_kni.
+ * @param linkup
+ *  New link state:
+ *  0 for linkdown.
+ *  > 0 for linkup.
+ *
+ * @return
+ *  On failure: -1
+ *  Previous link state == linkdown: 0
+ *  Previous link state == linkup: 1
+ */
+int __rte_experimental
+rte_kni_update_link(struct rte_kni *kni, unsigned int linkup);
+
 /**
  *  Close KNI device.
  */
diff --git a/lib/librte_kni/rte_kni_version.map b/lib/librte_kni/rte_kni_version.map
index acd515eb0..c877dc6aa 100644
--- a/lib/librte_kni/rte_kni_version.map
+++ b/lib/librte_kni/rte_kni_version.map
@@ -15,3 +15,9 @@ DPDK_2.0 {
 
 	local: *;
 };
+
+EXPERIMENTAL {
+	global:
+
+	rte_kni_update_link;
+};
-- 
2.19.0

Re: [dpdk-dev] [PATCH v4 1/6] kni: add API to set link status on kernel interface

[Ferruh Yigit]

On 10/17/2018 2:04 AM, Dan Gora wrote:
> Add a new API function to KNI, rte_kni_update_link() to allow DPDK
> applications to update the link status for KNI network interfaces in
> the linux kernel.
> 
> Signed-off-by: Dan Gora <dg@adax.com>
> ---
>  lib/librte_kni/rte_kni.c           | 41 ++++++++++++++++++++++++++++++
>  lib/librte_kni/rte_kni.h           | 20 +++++++++++++++
>  lib/librte_kni/rte_kni_version.map |  6 +++++
>  3 files changed, 67 insertions(+)
> 
> diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c
> index b8edd40f4..c9726d4f8 100644
> --- a/lib/librte_kni/rte_kni.c
> +++ b/lib/librte_kni/rte_kni.c
> @@ -717,6 +717,47 @@ rte_kni_unregister_handlers(struct rte_kni *kni)
>  
>  	return 0;
>  }
> +
> +int __rte_experimental
> +rte_kni_update_link(struct rte_kni *kni, unsigned int linkup)
> +{
> +	char path[64];
> +	char old_carrier[2];
> +	const char *new_carrier;
> +	int old_linkup;
> +	int fd, ret;
> +
> +	if (kni == NULL)
> +		return -1;
> +
> +	snprintf(path, sizeof(path), "/sys/devices/virtual/net/%s/carrier",
> +		kni->name);
> +
> +	fd = open(path, O_RDWR);
> +	if (fd == -1) {
> +		RTE_LOG(ERR, KNI, "Failed to open file: %s.\n", path);
> +		return -1;
> +	}
> +
> +	ret = read(fd, old_carrier, 2);
> +	if (ret < 1) {
> +		close(fd);
> +		return -1;
> +	}
> +	old_linkup = (old_carrier[0] == '1');
> +
> +	new_carrier = linkup ? "1" : "0";
> +	ret = write(fd, new_carrier, 1);

KNI sample application calls this API each 100ms, so this keeps writing
"carrier" file.

What do you think writing to "carrier" file only if status changed?

And it is possible to register to RTE_ETH_EVENT_INTR_LSC event in sample
application instead of polling but not sure if all drivers supports it.

Or what do you think storing the link status in sample application and call this
function only if link status changed?

Re: [dpdk-dev] [PATCH v4 1/6] kni: add API to set link status on kernel interface

[Dan Gora]

On Thu, Oct 18, 2018 at 10:44 AM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>
> KNI sample application calls this API each 100ms, so this keeps writing
> "carrier" file.
>
> What do you think writing to "carrier" file only if status changed?
>
> And it is possible to register to RTE_ETH_EVENT_INTR_LSC event in sample
> application instead of polling but not sure if all drivers supports it.

Yeah I'm not really interested in implementing this to be honest.  I
really don't have time.

> Or what do you think storing the link status in sample application and call this
> function only if link status changed?

No, one of the major points of this API was to NOT have to carry
around this state in the application.

d

[dpdk-dev] [PATCH v4 2/6] kni: add link status test

[Dan Gora]

Test changing the link status of the KNI interface with
rte_kni_update_link().

Signed-off-by: Dan Gora <dg@adax.com>
---
 test/test/test_kni.c | 77 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 77 insertions(+)

diff --git a/test/test/test_kni.c b/test/test/test_kni.c
index cf64c0b27..e38206905 100644
--- a/test/test/test_kni.c
+++ b/test/test/test_kni.c
@@ -118,6 +118,79 @@ kni_change_mtu(uint16_t port_id, unsigned int new_mtu)
 					 port_id, kni_pkt_mtu);
 	return 0;
 }
+
+static int
+test_kni_link_change(void)
+{
+	int ret;
+	int pid;
+
+	pid = fork();
+	if (pid < 0) {
+		printf("Error: Failed to fork a process\n");
+		return -1;
+	}
+
+	if (pid == 0) {
+		printf("Starting KNI Link status change tests.\n");
+		if (system(IFCONFIG TEST_KNI_PORT" up") == -1) {
+			ret = -1;
+			goto error;
+		}
+
+		ret = rte_kni_update_link(test_kni_ctx, 1);
+		if (ret < 0) {
+			printf("Failed to change link state to Up ret=%d.\n",
+				ret);
+			goto error;
+		}
+		rte_delay_ms(1000);
+		printf("KNI: Set LINKUP, previous state=%d\n", ret);
+
+		ret = rte_kni_update_link(test_kni_ctx, 0);
+		if (ret != 1) {
+			printf(
+		"Failed! Previous link state should be 1, returned %d.\n",
+				ret);
+			goto error;
+		}
+		rte_delay_ms(1000);
+		printf("KNI: Set LINKDOWN, previous state=%d\n", ret);
+
+		ret = rte_kni_update_link(test_kni_ctx, 1);
+		if (ret != 0) {
+			printf(
+		"Failed! Previous link state should be 0, returned %d.\n",
+				ret);
+			goto error;
+		}
+		printf("KNI: Set LINKUP, previous state=%d\n", ret);
+
+		ret = 0;
+		rte_delay_ms(1000);
+
+error:
+		if (system(IFCONFIG TEST_KNI_PORT" down") == -1)
+			ret = -1;
+
+		printf("KNI: Link status change tests: %s.\n",
+			(ret == 0) ? "Passed" : "Failed");
+		exit(ret);
+	} else {
+		int p_ret, status;
+
+		while (1) {
+			p_ret = waitpid(pid, &status, WNOHANG);
+			if (p_ret != 0) {
+				if (WIFEXITED(status))
+					return WEXITSTATUS(status);
+				return -1;
+			}
+			rte_delay_ms(10);
+			rte_kni_handle_request(test_kni_ctx);
+		}
+	}
+}
 /**
  * This loop fully tests the basic functions of KNI. e.g. transmitting,
  * receiving to, from kernel space, and kernel requests.
@@ -401,6 +474,10 @@ test_kni_processing(uint16_t port_id, struct rte_mempool *mp)
 		goto fail_kni;
 	}
 
+	ret = test_kni_link_change();
+	if (ret != 0)
+		goto fail_kni;
+
 	rte_eal_mp_remote_launch(test_kni_loop, NULL, CALL_MASTER);
 	RTE_LCORE_FOREACH_SLAVE(i) {
 		if (rte_eal_wait_lcore(i) < 0) {
-- 
2.19.0

[dpdk-dev] [PATCH v4 3/6] kni: set default carrier state of interface

[Dan Gora]

Add module parameter 'carrier='on|off' to set the default carrier state
for linux network interfaces created by the KNI module.  The default
carrier state is 'off'.

For KNI interfaces which need to reflect the carrier state of
a physical Ethernet port controlled by the DPDK application, the
default carrier state should be left set to 'off'.  The application
can set the carrier state of the KNI interface to reflect the state
of the physical Ethernet port using rte_kni_update_link().

For KNI interfaces which are purely virtual, the default carrier
state can be set to 'on'.  This enables the KNI interface to be
used without having to explicity set the carrier state to 'on'
using rte_kni_update_link().

Signed-off-by: Dan Gora <dg@adax.com>
---
 kernel/linux/kni/kni_dev.h  |  3 +++
 kernel/linux/kni/kni_misc.c | 40 +++++++++++++++++++++++++++++++++++++
 kernel/linux/kni/kni_net.c  |  5 +++++
 3 files changed, 48 insertions(+)

diff --git a/kernel/linux/kni/kni_dev.h b/kernel/linux/kni/kni_dev.h
index 6275ef27f..688f574a4 100644
--- a/kernel/linux/kni/kni_dev.h
+++ b/kernel/linux/kni/kni_dev.h
@@ -29,6 +29,9 @@
 
 #define MBUF_BURST_SZ 32
 
+/* Default carrier state for created KNI network interfaces */
+extern uint32_t dflt_carrier;
+
 /**
  * A structure describing the private information for a kni device.
  */
diff --git a/kernel/linux/kni/kni_misc.c b/kernel/linux/kni/kni_misc.c
index fa69f8e63..a5e7af2a6 100644
--- a/kernel/linux/kni/kni_misc.c
+++ b/kernel/linux/kni/kni_misc.c
@@ -39,6 +39,10 @@ static char *lo_mode;
 static char *kthread_mode;
 static uint32_t multiple_kthread_on;
 
+/* Default carrier state for created KNI network interfaces */
+static char *carrier;
+uint32_t dflt_carrier;
+
 #define KNI_DEV_IN_USE_BIT_NUM 0 /* Bit number for device in use */
 
 static int kni_net_id;
@@ -466,6 +470,8 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_num,
 		return -ENODEV;
 	}
 
+	netif_carrier_off(net_dev);
+
 	ret = kni_run_thread(knet, kni, dev_info.force_bind);
 	if (ret != 0)
 		return ret;
@@ -590,6 +596,22 @@ kni_parse_kthread_mode(void)
 	return 0;
 }
 
+static int __init
+kni_parse_carrier_state(void)
+{
+	if (!carrier)
+		dflt_carrier = 0;
+
+	if (strcmp(carrier, "off") == 0)
+		dflt_carrier = 0;
+	else if (strcmp(carrier, "on") == 0)
+		dflt_carrier = 1;
+	else
+		return -1;
+
+	return 0;
+}
+
 static int __init
 kni_init(void)
 {
@@ -605,6 +627,16 @@ kni_init(void)
 	else
 		pr_debug("Multiple kernel thread mode enabled\n");
 
+	if (kni_parse_carrier_state() < 0) {
+		pr_err("Invalid parameter for carrier\n");
+		return -EINVAL;
+	}
+
+	if (dflt_carrier == 0)
+		pr_debug("Default carrier state set to off.\n");
+	else
+		pr_debug("Default carrier state set to on.\n");
+
 #ifdef HAVE_SIMPLIFIED_PERNET_OPERATIONS
 	rc = register_pernet_subsys(&kni_net_ops);
 #else
@@ -663,3 +695,11 @@ MODULE_PARM_DESC(kthread_mode,
 "    multiple  Multiple kernel thread mode enabled.\n"
 "\n"
 );
+
+module_param(carrier, charp, S_IRUGO);
+MODULE_PARM_DESC(carrier,
+"Defaul carrier state for KNI interface (default=off):\n"
+"    off   Interfaces will be created with carrier state set to off.\n"
+"    on    Interfaces will be created with carrier state set to on.\n"
+"\n"
+);
diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c
index 3a542d737..7371b6d58 100644
--- a/kernel/linux/kni/kni_net.c
+++ b/kernel/linux/kni/kni_net.c
@@ -133,6 +133,10 @@ kni_net_open(struct net_device *dev)
 	struct kni_dev *kni = netdev_priv(dev);
 
 	netif_start_queue(dev);
+	if (dflt_carrier == 1)
+		netif_carrier_on(dev);
+	else
+		netif_carrier_off(dev);
 
 	memset(&req, 0, sizeof(req));
 	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
@@ -152,6 +156,7 @@ kni_net_release(struct net_device *dev)
 	struct kni_dev *kni = netdev_priv(dev);
 
 	netif_stop_queue(dev); /* can't transmit any more */
+	netif_carrier_off(dev);
 
 	memset(&req, 0, sizeof(req));
 	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
-- 
2.19.0

Re: [dpdk-dev] [PATCH v4 3/6] kni: set default carrier state of interface

[Ferruh Yigit]

On 10/17/2018 2:04 AM, Dan Gora wrote:
> +static int __init
> +kni_parse_carrier_state(void)
> +{
> +	if (!carrier)
> +		dflt_carrier = 0;
> +
> +	if (strcmp(carrier, "off") == 0)
> +		dflt_carrier = 0;
> +	else if (strcmp(carrier, "on") == 0)
> +		dflt_carrier = 1;
> +	else
> +		return -1;
> +
> +	return 0;
> +}

I think intention here is:

 if (!carrier)
 	return 0;

[dpdk-dev] [PATCH v4 4/6] examples/kni: monitor and update link status continually

[Dan Gora]

Update KNI example to add the command line flag '-m' to enable
a function to continuously monitor the Ethernet link status of
the physical link and update the link status of the corresponding
interfaces with rte_kni_update_link().

Signed-off-by: Dan Gora <dg@adax.com>
---
 examples/kni/Makefile |  2 ++
 examples/kni/main.c   | 70 +++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 70 insertions(+), 2 deletions(-)

diff --git a/examples/kni/Makefile b/examples/kni/Makefile
index 7e19d2e2a..dd90d7d73 100644
--- a/examples/kni/Makefile
+++ b/examples/kni/Makefile
@@ -20,6 +20,7 @@ static: build/$(APP)-static
 
 PC_FILE := $(shell pkg-config --path libdpdk)
 CFLAGS += -O3 $(shell pkg-config --cflags libdpdk)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk)
 LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk)
 
@@ -54,6 +55,7 @@ please change the definition of the RTE_TARGET environment variable)
 endif
 
 CFLAGS += -O3
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 CFLAGS += $(WERROR_FLAGS)
 
 include $(RTE_SDK)/mk/rte.extapp.mk
diff --git a/examples/kni/main.c b/examples/kni/main.c
index 80c401c51..634024960 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -106,6 +106,8 @@ static struct rte_mempool * pktmbuf_pool = NULL;
 static uint32_t ports_mask = 0;
 /* Ports set in promiscuous mode off by default. */
 static int promiscuous_on = 0;
+/* Monitor link status continually. off by default. */
+static int monitor_links;
 
 /* Structure type for recording kni interface specific stats */
 struct kni_interface_stats {
@@ -325,11 +327,12 @@ main_loop(__rte_unused void *arg)
 static void
 print_usage(const char *prgname)
 {
-	RTE_LOG(INFO, APP, "\nUsage: %s [EAL options] -- -p PORTMASK -P "
+	RTE_LOG(INFO, APP, "\nUsage: %s [EAL options] -- -p PORTMASK -P -m "
 		   "[--config (port,lcore_rx,lcore_tx,lcore_kthread...)"
 		   "[,(port,lcore_rx,lcore_tx,lcore_kthread...)]]\n"
 		   "    -p PORTMASK: hex bitmask of ports to use\n"
 		   "    -P : enable promiscuous mode\n"
+		   "    -m : enable monitoring of port carrier state\n"
 		   "    --config (port,lcore_rx,lcore_tx,lcore_kthread...): "
 		   "port and lcore configurations\n",
 	           prgname);
@@ -510,7 +513,7 @@ parse_args(int argc, char **argv)
 	opterr = 0;
 
 	/* Parse command line */
-	while ((opt = getopt_long(argc, argv, "p:P", longopts,
+	while ((opt = getopt_long(argc, argv, "p:Pm", longopts,
 						&longindex)) != EOF) {
 		switch (opt) {
 		case 'p':
@@ -519,6 +522,9 @@ parse_args(int argc, char **argv)
 		case 'P':
 			promiscuous_on = 1;
 			break;
+		case 'm':
+			monitor_links = 1;
+			break;
 		case 0:
 			if (!strncmp(longopts[longindex].name,
 				     CMDLINE_OPT_CONFIG,
@@ -674,6 +680,55 @@ check_all_ports_link_status(uint32_t port_mask)
 	}
 }
 
+static void
+log_link_state(struct rte_kni *kni, int prev, struct rte_eth_link *link)
+{
+	if (kni == NULL || link == NULL)
+		return;
+
+	if (prev == ETH_LINK_DOWN && link->link_status == ETH_LINK_UP) {
+		RTE_LOG(INFO, APP, "%s NIC Link is Up %d Mbps %s %s.\n",
+			rte_kni_get_name(kni),
+			link->link_speed,
+			link->link_autoneg ?  "(AutoNeg)" : "(Fixed)",
+			link->link_duplex ?  "Full Duplex" : "Half Duplex");
+	} else if (prev == ETH_LINK_UP && link->link_status == ETH_LINK_DOWN) {
+		RTE_LOG(INFO, APP, "%s NIC Link is Down.\n",
+			rte_kni_get_name(kni));
+	}
+}
+
+/*
+ * Monitor the link status of all ports and update the
+ * corresponding KNI interface(s)
+ */
+static void *
+monitor_all_ports_link_status(void *arg)
+{
+	uint16_t portid;
+	struct rte_eth_link link;
+	unsigned int i;
+	struct kni_port_params **p = kni_port_params_array;
+	int prev;
+	(void) arg;
+
+	while (monitor_links) {
+		rte_delay_ms(100);
+		RTE_ETH_FOREACH_DEV(portid) {
+			if ((ports_mask & (1 << portid)) == 0)
+				continue;
+			memset(&link, 0, sizeof(link));
+			rte_eth_link_get_nowait(portid, &link);
+			for (i = 0; i < p[portid]->nb_kni; i++) {
+				prev = rte_kni_update_link(p[portid]->kni[i],
+						link.link_status);
+				log_link_state(p[portid]->kni[i], prev, &link);
+			}
+		}
+	}
+	return NULL;
+}
+
 /* Callback for request of changing MTU */
 static int
 kni_change_mtu(uint16_t port_id, unsigned int new_mtu)
@@ -893,6 +948,8 @@ main(int argc, char** argv)
 	int ret;
 	uint16_t nb_sys_ports, port;
 	unsigned i;
+	void *retval;
+	pthread_t kni_link_tid;
 
 	/* Associate signal_hanlder function with USR signals */
 	signal(SIGUSR1, signal_handler);
@@ -949,12 +1006,21 @@ main(int argc, char** argv)
 	}
 	check_all_ports_link_status(ports_mask);
 
+	ret = rte_ctrl_thread_create(&kni_link_tid,
+				     "KNI link status check", NULL,
+				     monitor_all_ports_link_status, NULL);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Could not create link status thread!\n");
+
 	/* Launch per-lcore function on every lcore */
 	rte_eal_mp_remote_launch(main_loop, NULL, CALL_MASTER);
 	RTE_LCORE_FOREACH_SLAVE(i) {
 		if (rte_eal_wait_lcore(i) < 0)
 			return -1;
 	}
+	monitor_links = 0;
+	pthread_join(kni_link_tid, &retval);
 
 	/* Release resources */
 	RTE_ETH_FOREACH_DEV(port) {
-- 
2.19.0

[dpdk-dev] [PATCH v4 5/6] examples/kni: add log msgs to show and clear stats

[Dan Gora]

Add logging messages showing the commands necessary for the user to
have the application display and zero the statistics.

Signed-off-by: Dan Gora <dg@adax.com>
---
 examples/kni/main.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/examples/kni/main.c b/examples/kni/main.c
index 634024960..314e19c68 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -171,14 +171,13 @@ signal_handler(int signum)
 	/* When we receive a USR2 signal, reset stats */
 	if (signum == SIGUSR2) {
 		memset(&kni_stats, 0, sizeof(kni_stats));
-		printf("\n**Statistics have been reset**\n");
+		printf("\n** Statistics have been reset **\n");
 		return;
 	}
 
 	/* When we receive a RTMIN or SIGINT signal, stop kni processing */
 	if (signum == SIGRTMIN || signum == SIGINT){
-		printf("SIGRTMIN is received, and the KNI processing is "
-							"going to stop\n");
+		printf("\nSIGRTMIN/SIGINT received. KNI processing stopping.\n");
 		rte_atomic32_inc(&kni_stop);
 		return;
         }
@@ -950,6 +949,7 @@ main(int argc, char** argv)
 	unsigned i;
 	void *retval;
 	pthread_t kni_link_tid;
+	int pid;
 
 	/* Associate signal_hanlder function with USR signals */
 	signal(SIGUSR1, signal_handler);
@@ -1006,6 +1006,16 @@ main(int argc, char** argv)
 	}
 	check_all_ports_link_status(ports_mask);
 
+	pid = getpid();
+	RTE_LOG(INFO, APP, "========================\n");
+	RTE_LOG(INFO, APP, "KNI Running\n");
+	RTE_LOG(INFO, APP, "kill -SIGUSR1 %d\n", pid);
+	RTE_LOG(INFO, APP, "    Show KNI Statistics.\n");
+	RTE_LOG(INFO, APP, "kill -SIGUSR2 %d\n", pid);
+	RTE_LOG(INFO, APP, "    Zero KNI Statistics.\n");
+	RTE_LOG(INFO, APP, "========================\n");
+	fflush(stdout);
+
 	ret = rte_ctrl_thread_create(&kni_link_tid,
 				     "KNI link status check", NULL,
 				     monitor_all_ports_link_status, NULL);
-- 
2.19.0

[dpdk-dev] [PATCH v4 6/6] examples/kni: improve zeroing statistics

[Dan Gora]

The worker threads incrementing the rx/tx_packets race with the signal
handler from the main thread zeroing the entire statistics structure.
This can cause the statistics to fail to be zeroed, even when there
is no traffic on those interfaces.

Improve zeroing the statistics by only incrementing rx/tx_packets
in worker threads by a non-zero amount.  This limits the race to the
periods in which traffic is actually being received or transmitted.

Signed-off-by: Dan Gora <dg@adax.com>
---
 examples/kni/main.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/examples/kni/main.c b/examples/kni/main.c
index 314e19c68..3ae4c13a2 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -223,7 +223,8 @@ kni_ingress(struct kni_port_params *p)
 		}
 		/* Burst tx to kni */
 		num = rte_kni_tx_burst(p->kni[i], pkts_burst, nb_rx);
-		kni_stats[port_id].rx_packets += num;
+		if (num)
+			kni_stats[port_id].rx_packets += num;
 
 		rte_kni_handle_request(p->kni[i]);
 		if (unlikely(num < nb_rx)) {
@@ -260,7 +261,8 @@ kni_egress(struct kni_port_params *p)
 		}
 		/* Burst tx to eth */
 		nb_tx = rte_eth_tx_burst(port_id, 0, pkts_burst, (uint16_t)num);
-		kni_stats[port_id].tx_packets += nb_tx;
+		if (nb_tx)
+			kni_stats[port_id].tx_packets += nb_tx;
 		if (unlikely(nb_tx < num)) {
 			/* Free mbufs not tx to NIC */
 			kni_burst_free_mbufs(&pkts_burst[nb_tx], num - nb_tx);
-- 
2.19.0

[dpdk-dev] [PATCH v5 0/5] kni: add API to set link status on kernel interface

[Dan Gora]

Hi All,

Attached is version 5 of a patchset to add a new API function to
set the link status on kernel interfaces created with the KNI kernel
module.

v5
====
* Update Kernel NIC Interface document with new rte_kni option, move
  kernel module parameter description here from KNI sample app
  document, general cleanup.

* Update Kernel NIC Interface sample application (kni) document
  with new command line parameter, remove documentation for rte_kni
  kernel module but add reference to it, general cleanup.

* Change link status change sample rate for KNI example app to 500ms
  from 100ms.

* Fix bug in parameter parsing in v4 in rte_kni kernel module.

* Fix output formatting of optional parameters for rte_kni with
  modinfo.

* Fix permissions of optional parameters for rte_kni.

* Squash patch to add test for rte_kni_update_link into patch
  introducing the API function.

> v4
> ====
> * Rework rte_kni_update_link to only take linkup/linkdown as parameter,
>   return previous link state, and remove log messages.
> 
> * Update patch to set default carrier state to make default carrier
>   state configurable by passing the 'carrier=[on|off]' option to
>   the rte_kni kernel module.  This is necessary in order to allow
>   applications which use KNI as pure virtual interfaces without
>   corresponding physical ethernet port to use the interfaces without
>   having to set the carrier state to 'on' via rte_kni_update_link()
>   or by writing to /sys/devices/virtual/net/<ifaceX>/carrier.
>   Note that the default is 'off'.
> 
> * Add command line flag '-m' to examples/kni to continuously monitor
>   and update the KNI interface link status according to the link
>   status of the corresponding physical ethernet port.
> 
> 
> v3
> ====
> * Use separate function to test rte_kni_update_link() in 'test' app.
> 
> * Separate changes to 'test' app into separate patch to facilitate
>   possible merge with https://patches.dpdk.org/patch/44730/
> 
> * Remove changes to set KNI interfaces to 'up' in example/kni
> 
> v2
> ====
> 
> * Fix bug where "Fixed" and "AutoNeg" were transposed in the link
>   status log message.
> 
> * Add rte_kni_update_link() to rte_kni_version.map
> 
> * Add rte_kni_update_link() tests to kni_autotest
> 
> * Update examples/kni to continuously monitor link status and
>   update the corresponding kernel interface with
>   rte_kni_update_link().
> 
> * Minor improvements to examples/kni: Add log message showing how
>   to show/zero stats.  Improve zeroing statistics.
> 
> Note that checkpatches.sh compains about patch 1/5, but this appears
> to be a bug with check-symbol-change or something.  If I move the
> fragment of the patch modifying rte_kni_version.map to the bottom of
> the patch file, it doesn't complain any more...  I just don't really
> have time to investigate this right now.
  
thanks
dan



Dan Gora (5):
  kni: add API to set link status on kernel interface
  kni: set default carrier state of interface
  examples/kni: monitor and update link status continually
  examples/kni: add log msgs to show and clear stats
  examples/kni: improve zeroing statistics

 .../prog_guide/kernel_nic_interface.rst       | 203 ++++++++++----
 .../sample_app_ug/kernel_nic_interface.rst    | 249 +++++++++++-------
 examples/kni/Makefile                         |   2 +
 examples/kni/main.c                           |  92 ++++++-
 kernel/linux/kni/kni_dev.h                    |   3 +
 kernel/linux/kni/kni_misc.c                   |  57 +++-
 kernel/linux/kni/kni_net.c                    |   5 +
 lib/librte_kni/rte_kni.c                      |  41 +++
 lib/librte_kni/rte_kni.h                      |  20 ++
 lib/librte_kni/rte_kni_version.map            |   6 +
 test/test/test_kni.c                          |  77 ++++++
 11 files changed, 598 insertions(+), 157 deletions(-)

-- 
2.19.0

[dpdk-dev] [PATCH v5 1/5] kni: add API to set link status on kernel interface

[Dan Gora]

Add a new API function to KNI, rte_kni_update_link() to allow DPDK
applications to update the link status for KNI network interfaces in
the linux kernel.

Signed-off-by: Dan Gora <dg@adax.com>
---
 lib/librte_kni/rte_kni.c           | 41 ++++++++++++++++
 lib/librte_kni/rte_kni.h           | 20 ++++++++
 lib/librte_kni/rte_kni_version.map |  6 +++
 test/test/test_kni.c               | 77 ++++++++++++++++++++++++++++++
 4 files changed, 144 insertions(+)

diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c
index b8edd40f4..c9726d4f8 100644
--- a/lib/librte_kni/rte_kni.c
+++ b/lib/librte_kni/rte_kni.c
@@ -717,6 +717,47 @@ rte_kni_unregister_handlers(struct rte_kni *kni)
 
 	return 0;
 }
+
+int __rte_experimental
+rte_kni_update_link(struct rte_kni *kni, unsigned int linkup)
+{
+	char path[64];
+	char old_carrier[2];
+	const char *new_carrier;
+	int old_linkup;
+	int fd, ret;
+
+	if (kni == NULL)
+		return -1;
+
+	snprintf(path, sizeof(path), "/sys/devices/virtual/net/%s/carrier",
+		kni->name);
+
+	fd = open(path, O_RDWR);
+	if (fd == -1) {
+		RTE_LOG(ERR, KNI, "Failed to open file: %s.\n", path);
+		return -1;
+	}
+
+	ret = read(fd, old_carrier, 2);
+	if (ret < 1) {
+		close(fd);
+		return -1;
+	}
+	old_linkup = (old_carrier[0] == '1');
+
+	new_carrier = linkup ? "1" : "0";
+	ret = write(fd, new_carrier, 1);
+	if (ret < 1) {
+		RTE_LOG(ERR, KNI, "Failed to write file: %s.\n", path);
+		close(fd);
+		return -1;
+	}
+
+	close(fd);
+	return old_linkup;
+}
+
 void
 rte_kni_close(void)
 {
diff --git a/lib/librte_kni/rte_kni.h b/lib/librte_kni/rte_kni.h
index 601abdfc6..02ca43b4b 100644
--- a/lib/librte_kni/rte_kni.h
+++ b/lib/librte_kni/rte_kni.h
@@ -232,6 +232,26 @@ int rte_kni_register_handlers(struct rte_kni *kni, struct rte_kni_ops *ops);
  */
 int rte_kni_unregister_handlers(struct rte_kni *kni);
 
+/**
+ * Update link carrier state for KNI port.
+ *
+ * Update the linkup/linkdown state of a KNI interface in the kernel.
+ *
+ * @param kni
+ *  pointer to struct rte_kni.
+ * @param linkup
+ *  New link state:
+ *  0 for linkdown.
+ *  > 0 for linkup.
+ *
+ * @return
+ *  On failure: -1
+ *  Previous link state == linkdown: 0
+ *  Previous link state == linkup: 1
+ */
+int __rte_experimental
+rte_kni_update_link(struct rte_kni *kni, unsigned int linkup);
+
 /**
  *  Close KNI device.
  */
diff --git a/lib/librte_kni/rte_kni_version.map b/lib/librte_kni/rte_kni_version.map
index acd515eb0..c877dc6aa 100644
--- a/lib/librte_kni/rte_kni_version.map
+++ b/lib/librte_kni/rte_kni_version.map
@@ -15,3 +15,9 @@ DPDK_2.0 {
 
 	local: *;
 };
+
+EXPERIMENTAL {
+	global:
+
+	rte_kni_update_link;
+};
diff --git a/test/test/test_kni.c b/test/test/test_kni.c
index cf64c0b27..e38206905 100644
--- a/test/test/test_kni.c
+++ b/test/test/test_kni.c
@@ -118,6 +118,79 @@ kni_change_mtu(uint16_t port_id, unsigned int new_mtu)
 					 port_id, kni_pkt_mtu);
 	return 0;
 }
+
+static int
+test_kni_link_change(void)
+{
+	int ret;
+	int pid;
+
+	pid = fork();
+	if (pid < 0) {
+		printf("Error: Failed to fork a process\n");
+		return -1;
+	}
+
+	if (pid == 0) {
+		printf("Starting KNI Link status change tests.\n");
+		if (system(IFCONFIG TEST_KNI_PORT" up") == -1) {
+			ret = -1;
+			goto error;
+		}
+
+		ret = rte_kni_update_link(test_kni_ctx, 1);
+		if (ret < 0) {
+			printf("Failed to change link state to Up ret=%d.\n",
+				ret);
+			goto error;
+		}
+		rte_delay_ms(1000);
+		printf("KNI: Set LINKUP, previous state=%d\n", ret);
+
+		ret = rte_kni_update_link(test_kni_ctx, 0);
+		if (ret != 1) {
+			printf(
+		"Failed! Previous link state should be 1, returned %d.\n",
+				ret);
+			goto error;
+		}
+		rte_delay_ms(1000);
+		printf("KNI: Set LINKDOWN, previous state=%d\n", ret);
+
+		ret = rte_kni_update_link(test_kni_ctx, 1);
+		if (ret != 0) {
+			printf(
+		"Failed! Previous link state should be 0, returned %d.\n",
+				ret);
+			goto error;
+		}
+		printf("KNI: Set LINKUP, previous state=%d\n", ret);
+
+		ret = 0;
+		rte_delay_ms(1000);
+
+error:
+		if (system(IFCONFIG TEST_KNI_PORT" down") == -1)
+			ret = -1;
+
+		printf("KNI: Link status change tests: %s.\n",
+			(ret == 0) ? "Passed" : "Failed");
+		exit(ret);
+	} else {
+		int p_ret, status;
+
+		while (1) {
+			p_ret = waitpid(pid, &status, WNOHANG);
+			if (p_ret != 0) {
+				if (WIFEXITED(status))
+					return WEXITSTATUS(status);
+				return -1;
+			}
+			rte_delay_ms(10);
+			rte_kni_handle_request(test_kni_ctx);
+		}
+	}
+}
 /**
  * This loop fully tests the basic functions of KNI. e.g. transmitting,
  * receiving to, from kernel space, and kernel requests.
@@ -401,6 +474,10 @@ test_kni_processing(uint16_t port_id, struct rte_mempool *mp)
 		goto fail_kni;
 	}
 
+	ret = test_kni_link_change();
+	if (ret != 0)
+		goto fail_kni;
+
 	rte_eal_mp_remote_launch(test_kni_loop, NULL, CALL_MASTER);
 	RTE_LCORE_FOREACH_SLAVE(i) {
 		if (rte_eal_wait_lcore(i) < 0) {
-- 
2.19.0

[dpdk-dev] [PATCH v5 2/5] kni: set default carrier state of interface

[Dan Gora]

Add module parameter 'carrier='on|off' to set the default carrier state
for linux network interfaces created by the KNI module.  The default
carrier state is 'off'.

For KNI interfaces which need to reflect the carrier state of
a physical Ethernet port controlled by the DPDK application, the
default carrier state should be left set to 'off'.  The application
can set the carrier state of the KNI interface to reflect the state
of the physical Ethernet port using rte_kni_update_link().

For KNI interfaces which are purely virtual, the default carrier
state can be set to 'on'.  This enables the KNI interface to be
used without having to explicity set the carrier state to 'on'
using rte_kni_update_link().

Signed-off-by: Dan Gora <dg@adax.com>
---
 .../prog_guide/kernel_nic_interface.rst       | 203 ++++++++++++++----
 kernel/linux/kni/kni_dev.h                    |   3 +
 kernel/linux/kni/kni_misc.c                   |  57 ++++-
 kernel/linux/kni/kni_net.c                    |   5 +
 4 files changed, 213 insertions(+), 55 deletions(-)

diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst
index 8fa13fa1a..9173f2b78 100644
--- a/doc/guides/prog_guide/kernel_nic_interface.rst
+++ b/doc/guides/prog_guide/kernel_nic_interface.rst
@@ -29,58 +29,182 @@ The components of an application using the DPDK Kernel NIC Interface are shown i
 The DPDK KNI Kernel Module
 --------------------------
 
-The KNI kernel loadable module provides support for two types of devices:
+The KNI kernel loadable module ``rte_kni`` provides the kernel interface
+for DPDK applications.
 
-*   A Miscellaneous device (/dev/kni) that:
+When the ``rte_kni`` module is loaded, it will create a device ``/dev/kni``
+that is used by the DPDK KNI API functions to control and communicate with
+the kernel module.
 
-    *   Creates net devices (via ioctl  calls).
+The ``rte_kni`` kernel module contains several optional parameters which
+can be specified when the module is loaded to control its behavior:
 
-    *   Maintains a kernel thread context shared by all KNI instances
-        (simulating the RX side of the net driver).
+.. code-block:: console
 
-    *   For single kernel thread mode, maintains a kernel thread context shared by all KNI instances
-        (simulating the RX side of the net driver).
+    # modinfo rte_kni.ko
+    <snip>
+    parm:           lo_mode: KNI loopback mode (default=lo_mode_none):
+                    lo_mode_none        Kernel loopback disabled
+                    lo_mode_fifo        Enable kernel loopback with fifo
+                    lo_mode_fifo_skb    Enable kernel loopback with fifo and skb buffer
+                     (charp)
+    parm:           kthread_mode: Kernel thread mode (default=single):
+                    single    Single kernel thread mode enabled.
+                    multiple  Multiple kernel thread mode enabled.
+                     (charp)
+    parm:           carrier: Default carrier state for KNI interface (default=off):
+                    off   Interfaces will be created with carrier state set to off.
+                    on    Interfaces will be created with carrier state set to on.
+                     (charp)
 
-    *   For multiple kernel thread mode, maintains a kernel thread context for each KNI instance
-        (simulating the RX side of the net driver).
+Loading the ``rte_kni`` kernel module without any optional parameters is
+the typical way a DPDK application gets packets into and out of the kernel
+network stack.  Without any parameters, only one kernel thread is created
+for all KNI devices for packet receiving in kernel side, loopback mode is
+disabled, and the default carrier state of KNI interfaces is set to *off*.
 
-*   Net device:
+.. code-block:: console
 
-    *   Net functionality provided by implementing several operations such as netdev_ops,
-        header_ops, ethtool_ops that are defined by struct net_device,
-        including support for DPDK mbufs and FIFOs.
+    # insmod kmod/rte_kni.ko
 
-    *   The interface name is provided from userspace.
+Loopback Mode
+~~~~~~~~~~~~~
 
-    *   The MAC address can be the real NIC MAC address or random.
+For testing, the ``rte_kni`` kernel module can be loaded in loopback mode
+by specifying the ``lo_mode`` parameter:
 
-KNI Creation and Deletion
--------------------------
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko lo_mode=lo_mode_fifo
+
+The ``lo_mode_fifo`` loopback option will loop back ring enqueue/dequeue
+operations in kernel space.
+
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko lo_mode=lo_mode_fifo_skb
+
+The ``lo_mode_fifo_skb`` loopback option will loop back ring enqueue/dequeue
+operations and sk buffer copies in kernel space.
+
+If the ``lo_mode`` parameter is not specified, loopback mode is disabled.
+
+Kernel Thread Mode
+~~~~~~~~~~~~~~~~~~
+
+To provide flexibility of performance, the ``rte_kni`` KNI kernel module
+can be loaded with the ``kthread_mode`` parameter.  The ``rte_kni`` kernel
+module supports two options: "single kernel thread" mode and "multiple
+kernel thread" mode.
+
+Single kernel thread mode is enabled as follows:
+
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko kthread_mode=single
+
+This mode will create only one kernel thread for all KNI interfaces to
+receive data on the kernel side.  By default, this kernel thread is not
+bound to any particular core, but the user can set the core affinity for
+this kernel thread by setting the ``core_id`` and ``force_bind`` parameters
+in ``struct rte_kni_conf`` when the first KNI interface is created:
+
+For optimum performance, the kernel thread should be bound to a core in
+on the same socket as the DPDK lcores used in the application.
 
-The KNI interfaces are created by a DPDK application dynamically.
-The interface name and FIFO details are provided by the application through an ioctl call
-using the rte_kni_device_info struct which contains:
+The KNI kernel module can also be configured to start a separate kernel
+thread for each KNI interface created by the DPDK application.  Multiple
+kernel thread mode is enabled as follows:
 
-*   The interface name.
+.. code-block:: console
 
-*   Physical addresses of the corresponding memzones for the relevant FIFOs.
+    # insmod kmod/rte_kni.ko kthread_mode=multiple
 
-*   Mbuf mempool details, both physical and virtual (to calculate the offset for mbuf pointers).
+This mode will create a separate kernel thread for each KNI interface to
+receive data on the kernel side.  The core affinity of each ``kni_thread``
+kernel thread can be specified by setting the ``core_id`` and ``force_bind``
+parameters in ``struct rte_kni_conf`` when each KNI interface is created.
 
-*   PCI information.
+Multiple kernel thread mode can provide scalable higher performance if
+sufficient unused cores are available on the host system.
 
-*   Core affinity.
+If the ``kthread_mode`` parameter is not specified, the "single kernel
+thread" mode is used.
 
-Refer to rte_kni_common.h in the DPDK source code for more details.
+Default Carrier State
+~~~~~~~~~~~~~~~~~~~~~
 
-The physical addresses will be re-mapped into the kernel address space and stored in separate KNI contexts.
+The default carrier state of KNI interfaces created by the ``rte_kni``
+kernel module is controlled via the ``carrier`` option when the module
+is loaded.
 
-The affinity of kernel RX thread (both single and multi-threaded modes) is controlled by force_bind and
-core_id config parameters.
+If ``carrier=off`` is specified, the kernel module will leave the carrier
+state of the interface *down* when the interface is management enabled.
+The DPDK application can set the carrier state of the KNI interface using the
+``rte_kni_update_link()`` function.  This is useful for DPDK applications
+which require that the carrier state of the KNI interface reflect the
+actual link state of the corresponding physical NIC port.
 
-The KNI interfaces can be deleted by a DPDK application dynamically after being created.
-Furthermore, all those KNI interfaces not deleted will be deleted on the release operation
-of the miscellaneous device (when the DPDK application is closed).
+If ``carrier=on`` is specified, the kernel module will automatically set
+the carrier state of the interface to *up* when the interface is management
+enabled.  This is useful for DPDK applications which use the KNI interface as
+a purely virtual interface that does not correspond to any physical hardware
+and do not wish to explicitly set the carrier state of the interface with
+``rte_kni_update_link()``.  It is also useful for testing in loopback mode
+where the NIC port may not be physically connected to anything.
+
+To set the default carrier state to *on*:
+
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko carrier=on
+
+To set the default carrier state to *off*:
+
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko carrier=off
+
+If the ``carrier`` parameter is not specified, the default carrier state
+of KNI interfaces will be set to *off*.
+
+KNI Creation and Deletion
+-------------------------
+
+Before any KNI interfaces can be created, the ``rte_kni`` kernel module must
+be loaded into the kernel and configured withe ``rte_kni_init()`` function.
+
+The KNI interfaces are created by a DPDK application dynamically via the
+``rte_kni_alloc()`` function.
+
+The ``struct rte_kni_conf`` structure contains fields which allow the
+user to specify the interface name, set the MTU size, set an explicit or
+random MAC address and control the affinity of the kernel Rx thread(s)
+(both single and multi-threaded modes).
+
+The ``struct rte_kni_ops`` structure contains pointers to functions to
+handle requests from the ``rte_kni`` kernel module.  These functions
+allow DPDK applications to perform actions when the KNI interfaces are
+manipulated by control commands or functions external to the application.
+
+For example, the DPDK application may wish to enabled/disable a physical
+NIC port when a user enabled/disables a KNI interface with ``ip link set
+[up|down] dev <ifaceX>``.  The DPDK application can register a callback for
+``config_network_if`` which will be called when the interface management
+state changes.
+
+In order to run these callbacks, the application must periodically call
+the ``rte_kni_handle_request()`` function.  Any user callback function
+registered will be called directly from ``rte_kni_handle_request()``
+so care must be taken to prevent deadlock or not block any DPDK fastpath
+tasks.  Typically DPDK applications which use these callbacks will need
+to create a separate thread or secondary process to periodically call
+``rte_kni_handle_request()``.
+
+The KNI interfaces can be deleted by a DPDK application with
+``rte_kni_release()``.  All KNI interfaces not explicitly deleted will be
+deleted when the the ``/dev/kni`` device is closed, either explicitly with
+``rte_kni_close()`` or when the DPDK application is closed.
 
 DPDK mbuf Flow
 --------------
@@ -118,7 +242,7 @@ The packet is received from the Linux net stack, by calling the kni_net_tx() cal
 The mbuf is dequeued (without waiting due the cache) and filled with data from sk_buff.
 The sk_buff is then freed and the mbuf sent in the tx_q FIFO.
 
-The DPDK TX thread dequeues the mbuf and sends it to the PMD (via rte_eth_tx_burst()).
+The DPDK TX thread dequeues the mbuf and sends it to the PMD via ``rte_eth_tx_burst()``.
 It then puts the mbuf back in the cache.
 
 Ethtool
@@ -128,16 +252,3 @@ Ethtool is a Linux-specific tool with corresponding support in the kernel
 where each net device must register its own callbacks for the supported operations.
 The current implementation uses the igb/ixgbe modified Linux drivers for ethtool support.
 Ethtool is not supported in i40e and VMs (VF or EM devices).
-
-Link state and MTU change
--------------------------
-
-Link state and MTU change are network interface specific operations usually done via ifconfig.
-The request is initiated from the kernel side (in the context of the ifconfig process)
-and handled by the user space DPDK application.
-The application polls the request, calls the application handler and returns the response back into the kernel space.
-
-The application handlers can be registered upon interface creation or explicitly registered/unregistered in runtime.
-This provides flexibility in multiprocess scenarios
-(where the KNI is created in the primary process but the callbacks are handled in the secondary one).
-The constraint is that a single process can register and handle the requests.
diff --git a/kernel/linux/kni/kni_dev.h b/kernel/linux/kni/kni_dev.h
index 6275ef27f..688f574a4 100644
--- a/kernel/linux/kni/kni_dev.h
+++ b/kernel/linux/kni/kni_dev.h
@@ -29,6 +29,9 @@
 
 #define MBUF_BURST_SZ 32
 
+/* Default carrier state for created KNI network interfaces */
+extern uint32_t dflt_carrier;
+
 /**
  * A structure describing the private information for a kni device.
  */
diff --git a/kernel/linux/kni/kni_misc.c b/kernel/linux/kni/kni_misc.c
index fa69f8e63..0d2e85c30 100644
--- a/kernel/linux/kni/kni_misc.c
+++ b/kernel/linux/kni/kni_misc.c
@@ -39,6 +39,10 @@ static char *lo_mode;
 static char *kthread_mode;
 static uint32_t multiple_kthread_on;
 
+/* Default carrier state for created KNI network interfaces */
+static char *carrier;
+uint32_t dflt_carrier;
+
 #define KNI_DEV_IN_USE_BIT_NUM 0 /* Bit number for device in use */
 
 static int kni_net_id;
@@ -466,6 +470,8 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_num,
 		return -ENODEV;
 	}
 
+	netif_carrier_off(net_dev);
+
 	ret = kni_run_thread(knet, kni, dev_info.force_bind);
 	if (ret != 0)
 		return ret;
@@ -590,6 +596,24 @@ kni_parse_kthread_mode(void)
 	return 0;
 }
 
+static int __init
+kni_parse_carrier_state(void)
+{
+	if (!carrier) {
+		dflt_carrier = 0;
+		return 0;
+	}
+
+	if (strcmp(carrier, "off") == 0)
+		dflt_carrier = 0;
+	else if (strcmp(carrier, "on") == 0)
+		dflt_carrier = 1;
+	else
+		return -1;
+
+	return 0;
+}
+
 static int __init
 kni_init(void)
 {
@@ -605,6 +629,16 @@ kni_init(void)
 	else
 		pr_debug("Multiple kernel thread mode enabled\n");
 
+	if (kni_parse_carrier_state() < 0) {
+		pr_err("Invalid parameter for carrier\n");
+		return -EINVAL;
+	}
+
+	if (dflt_carrier == 0)
+		pr_debug("Default carrier state set to off.\n");
+	else
+		pr_debug("Default carrier state set to on.\n");
+
 #ifdef HAVE_SIMPLIFIED_PERNET_OPERATIONS
 	rc = register_pernet_subsys(&kni_net_ops);
 #else
@@ -647,19 +681,24 @@ kni_exit(void)
 module_init(kni_init);
 module_exit(kni_exit);
 
-module_param(lo_mode, charp, S_IRUGO | S_IWUSR);
+module_param(lo_mode, charp, 0644);
 MODULE_PARM_DESC(lo_mode,
 "KNI loopback mode (default=lo_mode_none):\n"
-"    lo_mode_none        Kernel loopback disabled\n"
-"    lo_mode_fifo        Enable kernel loopback with fifo\n"
-"    lo_mode_fifo_skb    Enable kernel loopback with fifo and skb buffer\n"
-"\n"
+"\t\tlo_mode_none        Kernel loopback disabled\n"
+"\t\tlo_mode_fifo        Enable kernel loopback with fifo\n"
+"\t\tlo_mode_fifo_skb    Enable kernel loopback with fifo and skb buffer\n"
 );
 
-module_param(kthread_mode, charp, S_IRUGO);
+module_param(kthread_mode, charp, 0644);
 MODULE_PARM_DESC(kthread_mode,
 "Kernel thread mode (default=single):\n"
-"    single    Single kernel thread mode enabled.\n"
-"    multiple  Multiple kernel thread mode enabled.\n"
-"\n"
+"\t\tsingle    Single kernel thread mode enabled.\n"
+"\t\tmultiple  Multiple kernel thread mode enabled.\n"
+);
+
+module_param(carrier, charp, 0644);
+MODULE_PARM_DESC(carrier,
+"Default carrier state for KNI interface (default=off):\n"
+"\t\toff   Interfaces will be created with carrier state set to off.\n"
+"\t\ton    Interfaces will be created with carrier state set to on.\n"
 );
diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c
index 3a542d737..7371b6d58 100644
--- a/kernel/linux/kni/kni_net.c
+++ b/kernel/linux/kni/kni_net.c
@@ -133,6 +133,10 @@ kni_net_open(struct net_device *dev)
 	struct kni_dev *kni = netdev_priv(dev);
 
 	netif_start_queue(dev);
+	if (dflt_carrier == 1)
+		netif_carrier_on(dev);
+	else
+		netif_carrier_off(dev);
 
 	memset(&req, 0, sizeof(req));
 	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
@@ -152,6 +156,7 @@ kni_net_release(struct net_device *dev)
 	struct kni_dev *kni = netdev_priv(dev);
 
 	netif_stop_queue(dev); /* can't transmit any more */
+	netif_carrier_off(dev);
 
 	memset(&req, 0, sizeof(req));
 	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
-- 
2.19.0

[dpdk-dev] [PATCH v5 3/5] examples/kni: monitor and update link status continually

[Dan Gora]

Update KNI example to add the command line flag '-m' to enable
a function to continuously monitor the Ethernet link status of
the physical link and update the link status of the corresponding
interfaces with rte_kni_update_link().

Signed-off-by: Dan Gora <dg@adax.com>
---
 .../sample_app_ug/kernel_nic_interface.rst    | 249 +++++++++++-------
 examples/kni/Makefile                         |   2 +
 examples/kni/main.c                           |  70 ++++-
 3 files changed, 224 insertions(+), 97 deletions(-)

diff --git a/doc/guides/sample_app_ug/kernel_nic_interface.rst b/doc/guides/sample_app_ug/kernel_nic_interface.rst
index 1b3ee9a5e..8de61bed6 100644
--- a/doc/guides/sample_app_ug/kernel_nic_interface.rst
+++ b/doc/guides/sample_app_ug/kernel_nic_interface.rst
@@ -31,18 +31,27 @@ This is done by creating one or more kernel net devices for each of the DPDK por
 The application allows the use of standard Linux tools (ethtool, ifconfig, tcpdump) with the DPDK ports and
 also the exchange of packets between the DPDK application and the Linux* kernel.
 
+The Kernel NIC Interface sample application requires that the
+KNI kernel module ``rte_kni`` be loaded into the kernel.  See
+:doc:`../prog_guide/kernel_nic_interface` for more information on loading
+the ``rte_kni`` kernel module.
+
 Overview
 --------
 
-The Kernel NIC Interface sample application uses two threads in user space for each physical NIC port being used,
-and allocates one or more KNI device for each physical NIC port with kernel module's support.
-For a physical NIC port, one thread reads from the port and writes to KNI devices,
-and another thread reads from KNI devices and writes the data unmodified to the physical NIC port.
-It is recommended to configure one KNI device for each physical NIC port.
-If configured with more than one KNI devices for a physical NIC port,
-it is just for performance testing, or it can work together with VMDq support in future.
+The Kernel NIC Interface sample application ``kni`` allocates one or more
+KNI interfaces for each physical NIC port.  For each physical NIC port,
+``kni`` uses two DPDK threads in user space; one thread reads from the port and
+writes to the corresponding KNI interfaces and the other thread reads from
+the KNI interfaces and writes the data unmodified to the physical NIC port.
+
+It is recommended to configure one KNI interface for each physical NIC port.
+The application can be configured with more than one KNI interface for
+each physical NIC port for performance testing or it can work together with
+VMDq support in future.
 
-The packet flow through the Kernel NIC Interface application is as shown in the following figure.
+The packet flow through the Kernel NIC Interface application is as shown
+in the following figure.
 
 .. _figure_kernel_nic:
 
@@ -50,145 +59,208 @@ The packet flow through the Kernel NIC Interface application is as shown in the
 
    Kernel NIC Application Packet Flow
 
+If link monitoring is enabled with the ``-m`` command line flag, one
+additional pthread is launched which will check the link status of each
+physical NIC port and will update the carrier status of the corresponding
+KNI interface(s) to match the physical NIC port's state.  This means that
+the KNI interface(s) will be disabled automatically when the Ethernet link
+goes down and enabled when the Ethernet link goes up.
+
+If link monitoring is enabled, the ``rte_kni`` kernel module should be
+loaded such that the default carrier state is set to *off*.  This ensures
+that the KNI interface is only enabled *after* the Ethernet link of the
+corresponding NIC port has reached the linkup state.
+
+If link monitoring is not enabled, the ``rte_kni`` kernel module should be
+loaded with the default carrier state set to *on*.  This sets the carrier
+state of the KNI interfaces to *on* when the KNI interfaces are enabled
+without regard to the actual link state of the corresponding NIC port.
+This is useful for testing in loopback mode where the NIC port may not be
+physically connected to anything.
+
 Compiling the Application
 -------------------------
 
 To compile the sample application see :doc:`compiling`.
 
-The application is located in the ``kni`` sub-directory.
+The application is located in the ``examples/kni`` sub-directory.
 
 .. note::
 
         This application is intended as a linuxapp only.
 
-Loading the Kernel Module
--------------------------
+Running the kni Example Application
+-----------------------------------
 
-Loading the KNI kernel module without any parameter is the typical way a DPDK application
-gets packets into and out of the kernel net stack.
-This way, only one kernel thread is created for all KNI devices for packet receiving in kernel side:
+The ``kni`` example application requires a number of command line options:
 
 .. code-block:: console
 
-    #insmod rte_kni.ko
+    kni [EAL options] -- -P -p PORTMASK --config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,(port,lcore_rx,lcore_tx[,lcore_kthread,...])]"
 
-Pinning the kernel thread to a specific core can be done using a taskset command such as following:
+Where:
 
-.. code-block:: console
+*   ``-P``:
 
-    #taskset -p 100000 `pgrep --fl kni_thread | awk '{print $1}'`
+    Set all ports to promiscuous mode so that packets are accepted regardless
+    of the packet's Ethernet MAC destination address.  Without this option,
+    only packets with the Ethernet MAC destination address set to the
+    Ethernet address of the port are accepted.
 
-This command line tries to pin the specific kni_thread on the 20th lcore (lcore numbering starts at 0),
-which means it needs to check if that lcore is available on the board.
-This command must be sent after the application has been launched, as insmod does not start the kni thread.
+*   ``-p PORTMASK``:
 
-For optimum performance,
-the lcore in the mask must be selected to be on the same socket as the lcores used in the KNI application.
+    Hexadecimal bitmask of ports to configure.
 
-To provide flexibility of performance, the kernel module of the KNI,
-located in the kmod sub-directory of the DPDK target directory,
-can be loaded with parameter of kthread_mode as follows:
+*   ``-m``:
 
-*   #insmod rte_kni.ko kthread_mode=single
+    Enable monitoring and updating of the Ethernet carrier state.  With this
+    option set, a thread will be started which will periodically check
+    the Ethernet link status of the physical Ethernet ports and set the
+    carrier state of the corresponding KNI network interface to match it.
+    This means that the KNI interface will be disabled automatically when
+    the Ethernet link goes down and enabled when the Ethernet link goes up.
 
-    This mode will create only one kernel thread for all KNI devices for packet receiving in kernel side.
-    By default, it is in this single kernel thread mode.
-    It can set core affinity for this kernel thread by using Linux command taskset.
+*   ``--config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,(port,lcore_rx,lcore_tx[,lcore_kthread,...])]"``:
 
-*   #insmod rte_kni.ko kthread_mode =multiple
+    Determines which lcores the Rx and Tx DPDK tasks, and (optionally)
+    the KNI kernel thread(s) are bound to for each physical port.
 
-    This mode will create a kernel thread for each KNI device for packet receiving in kernel side.
-    The core affinity of each kernel thread is set when creating the KNI device.
-    The lcore ID for each kernel thread is provided in the command line of launching the application.
-    Multiple kernel thread mode can provide scalable higher performance.
+Refer to *DPDK Getting Started Guide* for general information on running
+applications and the Environment Abstraction Layer (EAL) options.
 
-To measure the throughput in a loopback mode, the kernel module of the KNI,
-located in the kmod sub-directory of the DPDK target directory,
-can be loaded with parameters as follows:
+The ``-c coremask`` or ``-l corelist`` parameter of the EAL options must
+include the lcores specified by ``lcore_rx`` and ``lcore_tx`` for each port,
+but does not need to include lcores specified by ``lcore_kthread`` as those
+cores are used to pin the kernel threads in the ``rte_kni`` kernel module.
 
-*   #insmod rte_kni.ko lo_mode=lo_mode_fifo
+The ``--config`` parameter must include a set of
+``(port,lcore_rx,lcore_tx,[lcore_kthread,...])`` values for each physical
+port specified in the ``-p PORTMASK`` parameter.
 
-    This loopback mode will involve ring enqueue/dequeue operations in kernel space.
+The optional ``lcore_kthread`` lcore ID parameter in ``--config`` can be
+specified zero, one or more times for each physical port.
 
-*   #insmod rte_kni.ko lo_mode=lo_mode_fifo_skb
+If no lcore ID is specified for ``lcore_kthread``, one KNI interface will
+be created for the physical port ``port`` and the KNI kernel thread(s)
+will have no specific core affinity.
 
-    This loopback mode will involve ring enqueue/dequeue operations and sk buffer copies in kernel space.
+If one or more lcore IDs are specified for ``lcore_kthread``, a KNI interface
+will be created for each lcore ID specified, bound to the physical port
+``port``.  If the ``rte_kni`` kernel module is loaded in "multiple kernel
+thread" mode, a kernel thread will be created for each KNI interface and
+bound to the specified core.  If the ``rte_kni`` kernel module is loaded in
+"single kernel thread" mode, only one kernel thread is started for all KNI
+interfaces.  The kernel thread will be bound to the first ``lcore_kthread``
+lcore ID specified.
 
-Running the Application
------------------------
+Example Configurations
+~~~~~~~~~~~~~~~~~~~~~~~
 
-The application requires a number of command line options:
+The following commands will first load the ``rte_kni`` kernel module in
+"multiple kernel thread" mode.  The ``kni`` application is then started
+using two ports;  Port 0 uses lcore 4 for the Rx task, lcore 6 for the Tx
+task, and will create a single KNI interface ``vEth0_0`` with the kernel
+thread bound to lcore 8.  Port 1 uses lcore 5 for the Rx task, lcore 7
+for the Tx task, and will create a single KNI interface ``vEth1_0`` with
+the kernel thread bound to lcore 9.
 
 .. code-block:: console
 
-    kni [EAL options] -- -P -p PORTMASK --config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,port,lcore_rx,lcore_tx[,lcore_kthread,...]]"
+    # rmmod rte_kni
+    # insmod kmod/rte_kni.ko kthread_mode=multiple
+    # ./build/kni -l 4-7 -n 4 -- -P -p 0x3 -m --config="(0,4,6,8),(1,5,7,9)"
 
-Where:
-
-*   -P: Set all ports to promiscuous mode so that packets are accepted regardless of the packet's Ethernet MAC destination address.
-    Without this option, only packets with the Ethernet MAC destination address set to the Ethernet address of the port are accepted.
+The following example is identical, except an additional ``lcore_kthread``
+core is specified per physical port.  In this case, ``kni`` will create
+four KNI interfaces: ``vEth0_0``/``vEth0_1`` bound to physical port 0 and
+``vEth1_0``/``vEth1_1`` bound to physical port 1.
 
-*   -p PORTMASK: Hexadecimal bitmask of ports to configure.
+The kernel thread for each interface will be bound as follows:
 
-*   --config="(port,lcore_rx, lcore_tx[,lcore_kthread, ...]) [, port,lcore_rx, lcore_tx[,lcore_kthread, ...]]":
-    Determines which lcores of RX, TX, kernel thread are mapped to which ports.
+    * ``vEth0_0`` - bound to lcore 8.
+    * ``vEth0_1`` - bound to lcore 10.
+    * ``vEth1_0`` - bound to lcore 9.
+    * ``vEth1_1`` - bound to lcore 11
 
-Refer to *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options.
+.. code-block:: console
 
-The -c coremask or -l corelist parameter of the EAL options should include the lcores indicated by the lcore_rx and lcore_tx,
-but does not need to include lcores indicated by lcore_kthread as they are used to pin the kernel thread on.
-The -p PORTMASK parameter should include the ports indicated by the port in --config, neither more nor less.
+    # rmmod rte_kni
+    # insmod kmod/rte_kni.ko kthread_mode=multiple
+    # ./build/kni -l 4-7 -n 4 -- -P -p 0x3 -m --config="(0,4,6,8,10),(1,5,7,9,11)"
 
-The lcore_kthread in --config can be configured none, one or more lcore IDs.
-In multiple kernel thread mode, if configured none, a KNI device will be allocated for each port,
-while no specific lcore affinity will be set for its kernel thread.
-If configured one or more lcore IDs, one or more KNI devices will be allocated for each port,
-while specific lcore affinity will be set for its kernel thread.
-In single kernel thread mode, if configured none, a KNI device will be allocated for each port.
-If configured one or more lcore IDs,
-one or more KNI devices will be allocated for each port while
-no lcore affinity will be set as there is only one kernel thread for all KNI devices.
+The following example can be used to test the interface between the ``kni``
+test application and the ``rte_kni`` kernel module.  In this example, the
+``rte_kni`` kernel module is loaded in single kernel thread mode, loopback
+mode enabled, and the default carrier state is set to *on* so that the
+corresponding physical NIC port does not have to be connected in order to
+use the KNI interface.  One KNI interface ``vEth0_0`` is created for port 0
+and one KNI interface ``vEth1_0`` is created for port 1.  Since ``rte_kni``
+is loaded in "single kernel thread" mode, the one kernel thread is bound
+to lcore 8.
 
-For example, to run the application with two ports served by six lcores, one lcore of RX, one lcore of TX,
-and one lcore of kernel thread for each port:
+Since the physical NIC ports are not being used, link monitoring can be
+disabled by **not** specifying the ``-m`` flag to ``kni``:
 
 .. code-block:: console
 
-    ./build/kni -l 4-7 -n 4 -- -P -p 0x3 --config="(0,4,6,8),(1,5,7,9)"
+    # rmmod rte_kni
+    # insmod kmod/rte_kni.ko lo_mode=lo_mode_fifo carrier=on
+    # ./build/kni -l 4-7 -n 4 -- -P -p 0x3 --config="(0,4,6,8),(1,5,7,9)"
 
 KNI Operations
 --------------
 
-Once the KNI application is started, one can use different Linux* commands to manage the net interfaces.
-If more than one KNI devices configured for a physical port,
-only the first KNI device will be paired to the physical device.
-Operations on other KNI devices will not affect the physical port handled in user space application.
+Once the ``kni`` application is started, the user can use the normal
+Linux commands to manage the KNI interfaces as if they were any other
+Linux network interface.
 
-Assigning an IP address:
+Enable KNI interface and assign an IP address:
 
 .. code-block:: console
 
-    #ifconfig vEth0_0 192.168.0.1
+    # ifconfig vEth0_0 192.168.0.1
 
-Displaying the NIC registers:
+Show KNI interface configuration and statistics:
 
 .. code-block:: console
 
-    #ethtool -d vEth0_0
+    # ifconfig vEth0_0
 
-Dumping the network traffic:
+Dump network traffic:
 
 .. code-block:: console
 
-    #tcpdump -i vEth0_0
+    # tcpdump -i vEth0_0
+
+The normal Linux commands can also be used to change the MAC address and
+MTU size used by the physical NIC which corresponds to the KNI interface.
+However, if more than one KNI interface is configured for a physical port,
+these commands will only work on the first KNI interface for that port.
 
 Change the MAC address:
 
 .. code-block:: console
 
-    #ifconfig vEth0_0 hw ether 0C:01:02:03:04:08
+    # ifconfig vEth0_0 hw ether 0C:01:02:03:04:08
+
+Change the MTU size:
+
+.. code-block:: console
+
+    # ifconfig vEth0_0 mtu 1450
+
+If DPDK is compiled with ``CONFIG_RTE_KNI_KMOD_ETHTOOL=y`` and an Intel
+NIC is used, the user can use ``ethtool`` on the KNI interface as if it
+were a normal Linux kernel interface.
+
+Displaying the NIC registers:
+
+.. code-block:: console
+
+    # ethtool -d vEth0_0
 
-When the DPDK userspace application is closed, all the KNI devices are deleted from Linux*.
+When the ``kni`` application is closed, all the KNI interfaces are deleted
+from the Linux kernel.
 
 Explanation
 -----------
@@ -227,7 +299,7 @@ to see if this lcore is reading from or writing to kernel NIC interfaces.
 For the case that reads from a NIC port and writes to the kernel NIC interfaces (``kni_ingress``),
 the packet reception is the same as in L2 Forwarding sample application
 (see :ref:`l2_fwd_app_rx_tx_packets`).
-The packet transmission is done by sending mbufs into the kernel NIC interfaces by rte_kni_tx_burst().
+The packet transmission is done by sending mbufs into the kernel NIC interfaces by ``rte_kni_tx_burst()``.
 The KNI library automatically frees the mbufs after the kernel successfully copied the mbufs.
 
 For the other case that reads from kernel NIC interfaces
@@ -235,16 +307,3 @@ and writes to a physical NIC port (``kni_egress``),
 packets are retrieved by reading mbufs from kernel NIC interfaces by ``rte_kni_rx_burst()``.
 The packet transmission is the same as in the L2 Forwarding sample application
 (see :ref:`l2_fwd_app_rx_tx_packets`).
-
-Callbacks for Kernel Requests
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-To execute specific PMD operations in user space requested by some Linux* commands,
-callbacks must be implemented and filled in the struct rte_kni_ops structure.
-Currently, setting a new MTU, change in MAC address, configuring promiscusous mode and
-configuring the network interface(up/down) re supported.
-Default implementation for following is available in rte_kni library.
-Application may choose to not implement following callbacks:
-
-- ``config_mac_address``
-- ``config_promiscusity``
diff --git a/examples/kni/Makefile b/examples/kni/Makefile
index 7e19d2e2a..dd90d7d73 100644
--- a/examples/kni/Makefile
+++ b/examples/kni/Makefile
@@ -20,6 +20,7 @@ static: build/$(APP)-static
 
 PC_FILE := $(shell pkg-config --path libdpdk)
 CFLAGS += -O3 $(shell pkg-config --cflags libdpdk)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk)
 LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk)
 
@@ -54,6 +55,7 @@ please change the definition of the RTE_TARGET environment variable)
 endif
 
 CFLAGS += -O3
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 CFLAGS += $(WERROR_FLAGS)
 
 include $(RTE_SDK)/mk/rte.extapp.mk
diff --git a/examples/kni/main.c b/examples/kni/main.c
index 80c401c51..4fda51713 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -106,6 +106,8 @@ static struct rte_mempool * pktmbuf_pool = NULL;
 static uint32_t ports_mask = 0;
 /* Ports set in promiscuous mode off by default. */
 static int promiscuous_on = 0;
+/* Monitor link status continually. off by default. */
+static int monitor_links;
 
 /* Structure type for recording kni interface specific stats */
 struct kni_interface_stats {
@@ -325,11 +327,12 @@ main_loop(__rte_unused void *arg)
 static void
 print_usage(const char *prgname)
 {
-	RTE_LOG(INFO, APP, "\nUsage: %s [EAL options] -- -p PORTMASK -P "
+	RTE_LOG(INFO, APP, "\nUsage: %s [EAL options] -- -p PORTMASK -P -m "
 		   "[--config (port,lcore_rx,lcore_tx,lcore_kthread...)"
 		   "[,(port,lcore_rx,lcore_tx,lcore_kthread...)]]\n"
 		   "    -p PORTMASK: hex bitmask of ports to use\n"
 		   "    -P : enable promiscuous mode\n"
+		   "    -m : enable monitoring of port carrier state\n"
 		   "    --config (port,lcore_rx,lcore_tx,lcore_kthread...): "
 		   "port and lcore configurations\n",
 	           prgname);
@@ -510,7 +513,7 @@ parse_args(int argc, char **argv)
 	opterr = 0;
 
 	/* Parse command line */
-	while ((opt = getopt_long(argc, argv, "p:P", longopts,
+	while ((opt = getopt_long(argc, argv, "p:Pm", longopts,
 						&longindex)) != EOF) {
 		switch (opt) {
 		case 'p':
@@ -519,6 +522,9 @@ parse_args(int argc, char **argv)
 		case 'P':
 			promiscuous_on = 1;
 			break;
+		case 'm':
+			monitor_links = 1;
+			break;
 		case 0:
 			if (!strncmp(longopts[longindex].name,
 				     CMDLINE_OPT_CONFIG,
@@ -674,6 +680,55 @@ check_all_ports_link_status(uint32_t port_mask)
 	}
 }
 
+static void
+log_link_state(struct rte_kni *kni, int prev, struct rte_eth_link *link)
+{
+	if (kni == NULL || link == NULL)
+		return;
+
+	if (prev == ETH_LINK_DOWN && link->link_status == ETH_LINK_UP) {
+		RTE_LOG(INFO, APP, "%s NIC Link is Up %d Mbps %s %s.\n",
+			rte_kni_get_name(kni),
+			link->link_speed,
+			link->link_autoneg ?  "(AutoNeg)" : "(Fixed)",
+			link->link_duplex ?  "Full Duplex" : "Half Duplex");
+	} else if (prev == ETH_LINK_UP && link->link_status == ETH_LINK_DOWN) {
+		RTE_LOG(INFO, APP, "%s NIC Link is Down.\n",
+			rte_kni_get_name(kni));
+	}
+}
+
+/*
+ * Monitor the link status of all ports and update the
+ * corresponding KNI interface(s)
+ */
+static void *
+monitor_all_ports_link_status(void *arg)
+{
+	uint16_t portid;
+	struct rte_eth_link link;
+	unsigned int i;
+	struct kni_port_params **p = kni_port_params_array;
+	int prev;
+	(void) arg;
+
+	while (monitor_links) {
+		rte_delay_ms(500);
+		RTE_ETH_FOREACH_DEV(portid) {
+			if ((ports_mask & (1 << portid)) == 0)
+				continue;
+			memset(&link, 0, sizeof(link));
+			rte_eth_link_get_nowait(portid, &link);
+			for (i = 0; i < p[portid]->nb_kni; i++) {
+				prev = rte_kni_update_link(p[portid]->kni[i],
+						link.link_status);
+				log_link_state(p[portid]->kni[i], prev, &link);
+			}
+		}
+	}
+	return NULL;
+}
+
 /* Callback for request of changing MTU */
 static int
 kni_change_mtu(uint16_t port_id, unsigned int new_mtu)
@@ -893,6 +948,8 @@ main(int argc, char** argv)
 	int ret;
 	uint16_t nb_sys_ports, port;
 	unsigned i;
+	void *retval;
+	pthread_t kni_link_tid;
 
 	/* Associate signal_hanlder function with USR signals */
 	signal(SIGUSR1, signal_handler);
@@ -949,12 +1006,21 @@ main(int argc, char** argv)
 	}
 	check_all_ports_link_status(ports_mask);
 
+	ret = rte_ctrl_thread_create(&kni_link_tid,
+				     "KNI link status check", NULL,
+				     monitor_all_ports_link_status, NULL);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Could not create link status thread!\n");
+
 	/* Launch per-lcore function on every lcore */
 	rte_eal_mp_remote_launch(main_loop, NULL, CALL_MASTER);
 	RTE_LCORE_FOREACH_SLAVE(i) {
 		if (rte_eal_wait_lcore(i) < 0)
 			return -1;
 	}
+	monitor_links = 0;
+	pthread_join(kni_link_tid, &retval);
 
 	/* Release resources */
 	RTE_ETH_FOREACH_DEV(port) {
-- 
2.19.0

Re: [dpdk-dev] [PATCH v5 3/5] examples/kni: monitor and update link status continually

[Ferruh Yigit]

On 10/19/2018 1:23 AM, Dan Gora wrote:
> Update KNI example to add the command line flag '-m' to enable
> a function to continuously monitor the Ethernet link status of
> the physical link and update the link status of the corresponding
> interfaces with rte_kni_update_link().
> 
> Signed-off-by: Dan Gora <dg@adax.com>

<...>

> +Running the kni Example Application
> +-----------------------------------
>  
> -Loading the KNI kernel module without any parameter is the typical way a DPDK application
> -gets packets into and out of the kernel net stack.
> -This way, only one kernel thread is created for all KNI devices for packet receiving in kernel side:
> +The ``kni`` example application requires a number of command line options:
>  
>  .. code-block:: console
>  
> -    #insmod rte_kni.ko
> +    kni [EAL options] -- -P -p PORTMASK --config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,(port,lcore_rx,lcore_tx[,lcore_kthread,...])]"

+ [-m]:

kni [EAL options] -- -P -p PORTMASK
--config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,(port,lcore_rx,lcore_tx[,lcore_kthread,...])]"
[-m]

Can be added while merging.

<...>

> -
> -Callbacks for Kernel Requests
> -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> -
> -To execute specific PMD operations in user space requested by some Linux* commands,
> -callbacks must be implemented and filled in the struct rte_kni_ops structure.
> -Currently, setting a new MTU, change in MAC address, configuring promiscusous mode and
> -configuring the network interface(up/down) re supported.
> -Default implementation for following is available in rte_kni library.
> -Application may choose to not implement following callbacks:
> -
> -- ``config_mac_address``
> -- ``config_promiscusity``

I am for keeping this section, is there any specific reason to remove this?

Re: [dpdk-dev] [PATCH v5 3/5] examples/kni: monitor and update link status continually

[Dan Gora]

On Mon, Oct 22, 2018 at 9:51 AM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> > +Running the kni Example Application
> > +-----------------------------------
> >
> > -Loading the KNI kernel module without any parameter is the typical way a DPDK application
> > -gets packets into and out of the kernel net stack.
> > -This way, only one kernel thread is created for all KNI devices for packet receiving in kernel side:
> > +The ``kni`` example application requires a number of command line options:
> >
> >  .. code-block:: console
> >
> > -    #insmod rte_kni.ko
> > +    kni [EAL options] -- -P -p PORTMASK --config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,(port,lcore_rx,lcore_tx[,lcore_kthread,...])]"
>
> + [-m]:
>
> kni [EAL options] -- -P -p PORTMASK
> --config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,(port,lcore_rx,lcore_tx[,lcore_kthread,...])]"
> [-m]
>
> Can be added while merging.
>

ugh.. ok... I'll fix this... I thought I added it, but I guess I missed it here.

> > -Callbacks for Kernel Requests
> > -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > -
> > -To execute specific PMD operations in user space requested by some Linux* commands,
> > -callbacks must be implemented and filled in the struct rte_kni_ops structure.
> > -Currently, setting a new MTU, change in MAC address, configuring promiscusous mode and
> > -configuring the network interface(up/down) re supported.
> > -Default implementation for following is available in rte_kni library.
> > -Application may choose to not implement following callbacks:
> > -
> > -- ``config_mac_address``
> > -- ``config_promiscusity``
>
> I am for keeping this section, is there any specific reason to remove this?

Just because it's not really relevant to the KNI sample app.
Information about the KNI callbacks was moved to the main KNI
documentation page and updated there.

I'll update the info there to add the details that there are default
implementations for these two callbacks if they don't specify one.

thanks
d

[dpdk-dev] [PATCH v5 4/5] examples/kni: add log msgs to show and clear stats

[Dan Gora]

Add logging messages showing the commands necessary for the user to
have the application display and zero the statistics.

Signed-off-by: Dan Gora <dg@adax.com>
---
 examples/kni/main.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/examples/kni/main.c b/examples/kni/main.c
index 4fda51713..0e3b2a2f7 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -171,14 +171,13 @@ signal_handler(int signum)
 	/* When we receive a USR2 signal, reset stats */
 	if (signum == SIGUSR2) {
 		memset(&kni_stats, 0, sizeof(kni_stats));
-		printf("\n**Statistics have been reset**\n");
+		printf("\n** Statistics have been reset **\n");
 		return;
 	}
 
 	/* When we receive a RTMIN or SIGINT signal, stop kni processing */
 	if (signum == SIGRTMIN || signum == SIGINT){
-		printf("SIGRTMIN is received, and the KNI processing is "
-							"going to stop\n");
+		printf("\nSIGRTMIN/SIGINT received. KNI processing stopping.\n");
 		rte_atomic32_inc(&kni_stop);
 		return;
         }
@@ -950,6 +949,7 @@ main(int argc, char** argv)
 	unsigned i;
 	void *retval;
 	pthread_t kni_link_tid;
+	int pid;
 
 	/* Associate signal_hanlder function with USR signals */
 	signal(SIGUSR1, signal_handler);
@@ -1006,6 +1006,16 @@ main(int argc, char** argv)
 	}
 	check_all_ports_link_status(ports_mask);
 
+	pid = getpid();
+	RTE_LOG(INFO, APP, "========================\n");
+	RTE_LOG(INFO, APP, "KNI Running\n");
+	RTE_LOG(INFO, APP, "kill -SIGUSR1 %d\n", pid);
+	RTE_LOG(INFO, APP, "    Show KNI Statistics.\n");
+	RTE_LOG(INFO, APP, "kill -SIGUSR2 %d\n", pid);
+	RTE_LOG(INFO, APP, "    Zero KNI Statistics.\n");
+	RTE_LOG(INFO, APP, "========================\n");
+	fflush(stdout);
+
 	ret = rte_ctrl_thread_create(&kni_link_tid,
 				     "KNI link status check", NULL,
 				     monitor_all_ports_link_status, NULL);
-- 
2.19.0

[dpdk-dev] [PATCH v5 5/5] examples/kni: improve zeroing statistics

[Dan Gora]

The worker threads incrementing the rx/tx_packets race with the signal
handler from the main thread zeroing the entire statistics structure.
This can cause the statistics to fail to be zeroed, even when there
is no traffic on those interfaces.

Improve zeroing the statistics by only incrementing rx/tx_packets
in worker threads by a non-zero amount.  This limits the race to the
periods in which traffic is actually being received or transmitted.

Signed-off-by: Dan Gora <dg@adax.com>
---
 examples/kni/main.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/examples/kni/main.c b/examples/kni/main.c
index 0e3b2a2f7..e37b1ad36 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -223,7 +223,8 @@ kni_ingress(struct kni_port_params *p)
 		}
 		/* Burst tx to kni */
 		num = rte_kni_tx_burst(p->kni[i], pkts_burst, nb_rx);
-		kni_stats[port_id].rx_packets += num;
+		if (num)
+			kni_stats[port_id].rx_packets += num;
 
 		rte_kni_handle_request(p->kni[i]);
 		if (unlikely(num < nb_rx)) {
@@ -260,7 +261,8 @@ kni_egress(struct kni_port_params *p)
 		}
 		/* Burst tx to eth */
 		nb_tx = rte_eth_tx_burst(port_id, 0, pkts_burst, (uint16_t)num);
-		kni_stats[port_id].tx_packets += nb_tx;
+		if (nb_tx)
+			kni_stats[port_id].tx_packets += nb_tx;
 		if (unlikely(nb_tx < num)) {
 			/* Free mbufs not tx to NIC */
 			kni_burst_free_mbufs(&pkts_burst[nb_tx], num - nb_tx);
-- 
2.19.0

Re: [dpdk-dev] [PATCH v5 0/5] kni: add API to set link status on kernel interface

[Ferruh Yigit]

On 10/19/2018 1:23 AM, Dan Gora wrote:
> Hi All,
> 
> Attached is version 5 of a patchset to add a new API function to
> set the link status on kernel interfaces created with the KNI kernel
> module.
> 
> v5
> ====
> * Update Kernel NIC Interface document with new rte_kni option, move
>   kernel module parameter description here from KNI sample app
>   document, general cleanup.
> 
> * Update Kernel NIC Interface sample application (kni) document
>   with new command line parameter, remove documentation for rte_kni
>   kernel module but add reference to it, general cleanup.
> 
> * Change link status change sample rate for KNI example app to 500ms
>   from 100ms.
> 
> * Fix bug in parameter parsing in v4 in rte_kni kernel module.
> 
> * Fix output formatting of optional parameters for rte_kni with
>   modinfo.
> 
> * Fix permissions of optional parameters for rte_kni.
> 
> * Squash patch to add test for rte_kni_update_link into patch
>   introducing the API function.
> 
>> v4
>> ====
>> * Rework rte_kni_update_link to only take linkup/linkdown as parameter,
>>   return previous link state, and remove log messages.
>>
>> * Update patch to set default carrier state to make default carrier
>>   state configurable by passing the 'carrier=[on|off]' option to
>>   the rte_kni kernel module.  This is necessary in order to allow
>>   applications which use KNI as pure virtual interfaces without
>>   corresponding physical ethernet port to use the interfaces without
>>   having to set the carrier state to 'on' via rte_kni_update_link()
>>   or by writing to /sys/devices/virtual/net/<ifaceX>/carrier.
>>   Note that the default is 'off'.
>>
>> * Add command line flag '-m' to examples/kni to continuously monitor
>>   and update the KNI interface link status according to the link
>>   status of the corresponding physical ethernet port.
>>
>>
>> v3
>> ====
>> * Use separate function to test rte_kni_update_link() in 'test' app.
>>
>> * Separate changes to 'test' app into separate patch to facilitate
>>   possible merge with https://patches.dpdk.org/patch/44730/
>>
>> * Remove changes to set KNI interfaces to 'up' in example/kni
>>
>> v2
>> ====
>>
>> * Fix bug where "Fixed" and "AutoNeg" were transposed in the link
>>   status log message.
>>
>> * Add rte_kni_update_link() to rte_kni_version.map
>>
>> * Add rte_kni_update_link() tests to kni_autotest
>>
>> * Update examples/kni to continuously monitor link status and
>>   update the corresponding kernel interface with
>>   rte_kni_update_link().
>>
>> * Minor improvements to examples/kni: Add log message showing how
>>   to show/zero stats.  Improve zeroing statistics.
>>
>> Note that checkpatches.sh compains about patch 1/5, but this appears
>> to be a bug with check-symbol-change or something.  If I move the
>> fragment of the patch modifying rte_kni_version.map to the bottom of
>> the patch file, it doesn't complain any more...  I just don't really
>> have time to investigate this right now.
>   
> thanks
> dan
> 
> 
> 
> Dan Gora (5):
>   kni: add API to set link status on kernel interface
>   kni: set default carrier state of interface
>   examples/kni: monitor and update link status continually
>   examples/kni: add log msgs to show and clear stats
>   examples/kni: improve zeroing statistics

Thanks for the document clean up.

For series,
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>


Thomas,

There is a possible doc update, what do you think fixing it while applying? Or
do you require a new version?

Thanks,
ferruh

Re: [dpdk-dev] [PATCH v5 0/5] kni: add API to set link status on kernel interface

[Thomas Monjalon]

22/10/2018 15:03, Ferruh Yigit:
> On 10/19/2018 1:23 AM, Dan Gora wrote:
> > Hi All,
> > 
> > Attached is version 5 of a patchset to add a new API function to
> > set the link status on kernel interfaces created with the KNI kernel
> > module.
> > 
> > v5
> > ====
> > * Update Kernel NIC Interface document with new rte_kni option, move
> >   kernel module parameter description here from KNI sample app
> >   document, general cleanup.
> > 
> > * Update Kernel NIC Interface sample application (kni) document
> >   with new command line parameter, remove documentation for rte_kni
> >   kernel module but add reference to it, general cleanup.
> > 
> > * Change link status change sample rate for KNI example app to 500ms
> >   from 100ms.
> > 
> > * Fix bug in parameter parsing in v4 in rte_kni kernel module.
> > 
> > * Fix output formatting of optional parameters for rte_kni with
> >   modinfo.
> > 
> > * Fix permissions of optional parameters for rte_kni.
> > 
> > * Squash patch to add test for rte_kni_update_link into patch
> >   introducing the API function.
> > 
> >> v4
> >> ====
> >> * Rework rte_kni_update_link to only take linkup/linkdown as parameter,
> >>   return previous link state, and remove log messages.
> >>
> >> * Update patch to set default carrier state to make default carrier
> >>   state configurable by passing the 'carrier=[on|off]' option to
> >>   the rte_kni kernel module.  This is necessary in order to allow
> >>   applications which use KNI as pure virtual interfaces without
> >>   corresponding physical ethernet port to use the interfaces without
> >>   having to set the carrier state to 'on' via rte_kni_update_link()
> >>   or by writing to /sys/devices/virtual/net/<ifaceX>/carrier.
> >>   Note that the default is 'off'.
> >>
> >> * Add command line flag '-m' to examples/kni to continuously monitor
> >>   and update the KNI interface link status according to the link
> >>   status of the corresponding physical ethernet port.
> >>
> >>
> >> v3
> >> ====
> >> * Use separate function to test rte_kni_update_link() in 'test' app.
> >>
> >> * Separate changes to 'test' app into separate patch to facilitate
> >>   possible merge with https://patches.dpdk.org/patch/44730/
> >>
> >> * Remove changes to set KNI interfaces to 'up' in example/kni
> >>
> >> v2
> >> ====
> >>
> >> * Fix bug where "Fixed" and "AutoNeg" were transposed in the link
> >>   status log message.
> >>
> >> * Add rte_kni_update_link() to rte_kni_version.map
> >>
> >> * Add rte_kni_update_link() tests to kni_autotest
> >>
> >> * Update examples/kni to continuously monitor link status and
> >>   update the corresponding kernel interface with
> >>   rte_kni_update_link().
> >>
> >> * Minor improvements to examples/kni: Add log message showing how
> >>   to show/zero stats.  Improve zeroing statistics.
> >>
> >> Note that checkpatches.sh compains about patch 1/5, but this appears
> >> to be a bug with check-symbol-change or something.  If I move the
> >> fragment of the patch modifying rte_kni_version.map to the bottom of
> >> the patch file, it doesn't complain any more...  I just don't really
> >> have time to investigate this right now.
> >   
> > thanks
> > dan
> > 
> > 
> > 
> > Dan Gora (5):
> >   kni: add API to set link status on kernel interface
> >   kni: set default carrier state of interface
> >   examples/kni: monitor and update link status continually
> >   examples/kni: add log msgs to show and clear stats
> >   examples/kni: improve zeroing statistics
> 
> Thanks for the document clean up.
> 
> For series,
> Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
> 
> 
> Thomas,
> 
> There is a possible doc update, what do you think fixing it while applying? Or
> do you require a new version?

What is the update to do?

Re: [dpdk-dev] [PATCH v5 0/5] kni: add API to set link status on kernel interface

[Ferruh Yigit]

On 10/22/2018 2:08 PM, Thomas Monjalon wrote:
> 22/10/2018 15:03, Ferruh Yigit:
>> On 10/19/2018 1:23 AM, Dan Gora wrote:
>>> Hi All,
>>>
>>> Attached is version 5 of a patchset to add a new API function to
>>> set the link status on kernel interfaces created with the KNI kernel
>>> module.
>>>
>>> v5
>>> ====
>>> * Update Kernel NIC Interface document with new rte_kni option, move
>>>   kernel module parameter description here from KNI sample app
>>>   document, general cleanup.
>>>
>>> * Update Kernel NIC Interface sample application (kni) document
>>>   with new command line parameter, remove documentation for rte_kni
>>>   kernel module but add reference to it, general cleanup.
>>>
>>> * Change link status change sample rate for KNI example app to 500ms
>>>   from 100ms.
>>>
>>> * Fix bug in parameter parsing in v4 in rte_kni kernel module.
>>>
>>> * Fix output formatting of optional parameters for rte_kni with
>>>   modinfo.
>>>
>>> * Fix permissions of optional parameters for rte_kni.
>>>
>>> * Squash patch to add test for rte_kni_update_link into patch
>>>   introducing the API function.
>>>
>>>> v4
>>>> ====
>>>> * Rework rte_kni_update_link to only take linkup/linkdown as parameter,
>>>>   return previous link state, and remove log messages.
>>>>
>>>> * Update patch to set default carrier state to make default carrier
>>>>   state configurable by passing the 'carrier=[on|off]' option to
>>>>   the rte_kni kernel module.  This is necessary in order to allow
>>>>   applications which use KNI as pure virtual interfaces without
>>>>   corresponding physical ethernet port to use the interfaces without
>>>>   having to set the carrier state to 'on' via rte_kni_update_link()
>>>>   or by writing to /sys/devices/virtual/net/<ifaceX>/carrier.
>>>>   Note that the default is 'off'.
>>>>
>>>> * Add command line flag '-m' to examples/kni to continuously monitor
>>>>   and update the KNI interface link status according to the link
>>>>   status of the corresponding physical ethernet port.
>>>>
>>>>
>>>> v3
>>>> ====
>>>> * Use separate function to test rte_kni_update_link() in 'test' app.
>>>>
>>>> * Separate changes to 'test' app into separate patch to facilitate
>>>>   possible merge with https://patches.dpdk.org/patch/44730/
>>>>
>>>> * Remove changes to set KNI interfaces to 'up' in example/kni
>>>>
>>>> v2
>>>> ====
>>>>
>>>> * Fix bug where "Fixed" and "AutoNeg" were transposed in the link
>>>>   status log message.
>>>>
>>>> * Add rte_kni_update_link() to rte_kni_version.map
>>>>
>>>> * Add rte_kni_update_link() tests to kni_autotest
>>>>
>>>> * Update examples/kni to continuously monitor link status and
>>>>   update the corresponding kernel interface with
>>>>   rte_kni_update_link().
>>>>
>>>> * Minor improvements to examples/kni: Add log message showing how
>>>>   to show/zero stats.  Improve zeroing statistics.
>>>>
>>>> Note that checkpatches.sh compains about patch 1/5, but this appears
>>>> to be a bug with check-symbol-change or something.  If I move the
>>>> fragment of the patch modifying rte_kni_version.map to the bottom of
>>>> the patch file, it doesn't complain any more...  I just don't really
>>>> have time to investigate this right now.
>>>   
>>> thanks
>>> dan
>>>
>>>
>>>
>>> Dan Gora (5):
>>>   kni: add API to set link status on kernel interface
>>>   kni: set default carrier state of interface
>>>   examples/kni: monitor and update link status continually
>>>   examples/kni: add log msgs to show and clear stats
>>>   examples/kni: improve zeroing statistics
>>
>> Thanks for the document clean up.
>>
>> For series,
>> Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
>>
>>
>> Thomas,
>>
>> There is a possible doc update, what do you think fixing it while applying? Or
>> do you require a new version?
> 
> What is the update to do?

comments on patch 3/5

Re: [dpdk-dev] [PATCH v5 0/5] kni: add API to set link status on kernel interface

[Thomas Monjalon]

22/10/2018 15:14, Ferruh Yigit:
> On 10/22/2018 2:08 PM, Thomas Monjalon wrote:
> > 22/10/2018 15:03, Ferruh Yigit:
> >> On 10/19/2018 1:23 AM, Dan Gora wrote:
> >>> Hi All,
> >>>
> >>> Attached is version 5 of a patchset to add a new API function to
> >>> set the link status on kernel interfaces created with the KNI kernel
> >>> module.
> >>>
> >>> v5
> >>> ====
> >>> * Update Kernel NIC Interface document with new rte_kni option, move
> >>>   kernel module parameter description here from KNI sample app
> >>>   document, general cleanup.
> >>>
> >>> * Update Kernel NIC Interface sample application (kni) document
> >>>   with new command line parameter, remove documentation for rte_kni
> >>>   kernel module but add reference to it, general cleanup.
> >>>
> >>> * Change link status change sample rate for KNI example app to 500ms
> >>>   from 100ms.
> >>>
> >>> * Fix bug in parameter parsing in v4 in rte_kni kernel module.
> >>>
> >>> * Fix output formatting of optional parameters for rte_kni with
> >>>   modinfo.
> >>>
> >>> * Fix permissions of optional parameters for rte_kni.
> >>>
> >>> * Squash patch to add test for rte_kni_update_link into patch
> >>>   introducing the API function.
> >>>
> >>>> v4
> >>>> ====
> >>>> * Rework rte_kni_update_link to only take linkup/linkdown as parameter,
> >>>>   return previous link state, and remove log messages.
> >>>>
> >>>> * Update patch to set default carrier state to make default carrier
> >>>>   state configurable by passing the 'carrier=[on|off]' option to
> >>>>   the rte_kni kernel module.  This is necessary in order to allow
> >>>>   applications which use KNI as pure virtual interfaces without
> >>>>   corresponding physical ethernet port to use the interfaces without
> >>>>   having to set the carrier state to 'on' via rte_kni_update_link()
> >>>>   or by writing to /sys/devices/virtual/net/<ifaceX>/carrier.
> >>>>   Note that the default is 'off'.
> >>>>
> >>>> * Add command line flag '-m' to examples/kni to continuously monitor
> >>>>   and update the KNI interface link status according to the link
> >>>>   status of the corresponding physical ethernet port.
> >>>>
> >>>>
> >>>> v3
> >>>> ====
> >>>> * Use separate function to test rte_kni_update_link() in 'test' app.
> >>>>
> >>>> * Separate changes to 'test' app into separate patch to facilitate
> >>>>   possible merge with https://patches.dpdk.org/patch/44730/
> >>>>
> >>>> * Remove changes to set KNI interfaces to 'up' in example/kni
> >>>>
> >>>> v2
> >>>> ====
> >>>>
> >>>> * Fix bug where "Fixed" and "AutoNeg" were transposed in the link
> >>>>   status log message.
> >>>>
> >>>> * Add rte_kni_update_link() to rte_kni_version.map
> >>>>
> >>>> * Add rte_kni_update_link() tests to kni_autotest
> >>>>
> >>>> * Update examples/kni to continuously monitor link status and
> >>>>   update the corresponding kernel interface with
> >>>>   rte_kni_update_link().
> >>>>
> >>>> * Minor improvements to examples/kni: Add log message showing how
> >>>>   to show/zero stats.  Improve zeroing statistics.
> >>>>
> >>>> Note that checkpatches.sh compains about patch 1/5, but this appears
> >>>> to be a bug with check-symbol-change or something.  If I move the
> >>>> fragment of the patch modifying rte_kni_version.map to the bottom of
> >>>> the patch file, it doesn't complain any more...  I just don't really
> >>>> have time to investigate this right now.
> >>>   
> >>> thanks
> >>> dan
> >>>
> >>>
> >>>
> >>> Dan Gora (5):
> >>>   kni: add API to set link status on kernel interface
> >>>   kni: set default carrier state of interface
> >>>   examples/kni: monitor and update link status continually
> >>>   examples/kni: add log msgs to show and clear stats
> >>>   examples/kni: improve zeroing statistics
> >>
> >> Thanks for the document clean up.
> >>
> >> For series,
> >> Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
> >>
> >>
> >> Thomas,
> >>
> >> There is a possible doc update, what do you think fixing it while applying? Or
> >> do you require a new version?
> > 
> > What is the update to do?
> 
> comments on patch 3/5

I prefer a new version please.

[dpdk-dev] [PATCH v6 0/5] kni: add API to set link status on kernel interface

[Dan Gora]

Hi All,

Attached is version 6 of a patchset to add a new API function to
set the link status on kernel interfaces created with the KNI kernel
module.

v6
====
* Update info about callback functions in KNI library documentation.

* Minor fixup to module parameter formatting in KNI kernel module. (align
  the "charp" text...)

* Add '-m' flag to KNI sample app command line in KNI sample app
  documentation.

* Add references to KNI library documentation from KNI sample app
  documentation (yay Sphinx!)

> v5
> ====
> * Update Kernel NIC Interface document with new rte_kni option, move
>   kernel module parameter description here from KNI sample app
>   document, general cleanup.
> 
> * Update Kernel NIC Interface sample application (kni) document
>   with new command line parameter, remove documentation for rte_kni
>   kernel module but add reference to it, general cleanup.
> 
> * Change link status change sample rate for KNI example app to 500ms
>   from 100ms.
> 
> * Fix bug in parameter parsing in v4 in rte_kni kernel module.
> 
> * Fix output formatting of optional parameters for rte_kni with
>   modinfo.
> 
> * Fix permissions of optional parameters for rte_kni.
> 
> * Squash patch to add test for rte_kni_update_link into patch
>   introducing the API function.
> 
> v4
> ====
> * Rework rte_kni_update_link to only take linkup/linkdown as parameter,
>   return previous link state, and remove log messages.
> 
> * Update patch to set default carrier state to make default carrier
>   state configurable by passing the 'carrier=[on|off]' option to
>   the rte_kni kernel module.  This is necessary in order to allow
>   applications which use KNI as pure virtual interfaces without
>   corresponding physical ethernet port to use the interfaces without
>   having to set the carrier state to 'on' via rte_kni_update_link()
>   or by writing to /sys/devices/virtual/net/<ifaceX>/carrier.
>   Note that the default is 'off'.
> 
> * Add command line flag '-m' to examples/kni to continuously monitor
>   and update the KNI interface link status according to the link
>   status of the corresponding physical ethernet port.
> 
> 
> v3
> ====
> * Use separate function to test rte_kni_update_link() in 'test' app.
> 
> * Separate changes to 'test' app into separate patch to facilitate
>   possible merge with https://patches.dpdk.org/patch/44730/
> 
> * Remove changes to set KNI interfaces to 'up' in example/kni
> 
> v2
> ====
> 
> * Fix bug where "Fixed" and "AutoNeg" were transposed in the link
>   status log message.
> 
> * Add rte_kni_update_link() to rte_kni_version.map
> 
> * Add rte_kni_update_link() tests to kni_autotest
> 
> * Update examples/kni to continuously monitor link status and
>   update the corresponding kernel interface with
>   rte_kni_update_link().
> 
> * Minor improvements to examples/kni: Add log message showing how
>   to show/zero stats.  Improve zeroing statistics.
> 
> Note that checkpatches.sh compains about patch 1/5, but this appears
> to be a bug with check-symbol-change or something.  If I move the
> fragment of the patch modifying rte_kni_version.map to the bottom of
> the patch file, it doesn't complain any more...  I just don't really
> have time to investigate this right now.
  
thanks
dan

Dan Gora (5):
  kni: add API to set link status on kernel interface
  kni: set default carrier state of interface
  examples/kni: monitor and update link status continually
  examples/kni: add log msgs to show and clear stats
  examples/kni: improve zeroing statistics

 .../prog_guide/kernel_nic_interface.rst       | 239 ++++++++++++++---
 .../sample_app_ug/kernel_nic_interface.rst    | 251 +++++++++++-------
 examples/kni/Makefile                         |   2 +
 examples/kni/main.c                           |  92 ++++++-
 kernel/linux/kni/kni_dev.h                    |   3 +
 kernel/linux/kni/kni_misc.c                   |  60 ++++-
 kernel/linux/kni/kni_net.c                    |   5 +
 lib/librte_kni/rte_kni.c                      |  41 +++
 lib/librte_kni/rte_kni.h                      |  20 ++
 lib/librte_kni/rte_kni_version.map            |   6 +
 test/test/test_kni.c                          |  77 ++++++
 11 files changed, 641 insertions(+), 155 deletions(-)

-- 
2.19.0

[dpdk-dev] [PATCH v6 1/5] kni: add API to set link status on kernel interface

[Dan Gora]

Add a new API function to KNI, rte_kni_update_link() to allow DPDK
applications to update the link status for KNI network interfaces in
the linux kernel.

Signed-off-by: Dan Gora <dg@adax.com>
---
 lib/librte_kni/rte_kni.c           | 41 ++++++++++++++++
 lib/librte_kni/rte_kni.h           | 20 ++++++++
 lib/librte_kni/rte_kni_version.map |  6 +++
 test/test/test_kni.c               | 77 ++++++++++++++++++++++++++++++
 4 files changed, 144 insertions(+)

diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c
index b8edd40f4..c9726d4f8 100644
--- a/lib/librte_kni/rte_kni.c
+++ b/lib/librte_kni/rte_kni.c
@@ -717,6 +717,47 @@ rte_kni_unregister_handlers(struct rte_kni *kni)
 
 	return 0;
 }
+
+int __rte_experimental
+rte_kni_update_link(struct rte_kni *kni, unsigned int linkup)
+{
+	char path[64];
+	char old_carrier[2];
+	const char *new_carrier;
+	int old_linkup;
+	int fd, ret;
+
+	if (kni == NULL)
+		return -1;
+
+	snprintf(path, sizeof(path), "/sys/devices/virtual/net/%s/carrier",
+		kni->name);
+
+	fd = open(path, O_RDWR);
+	if (fd == -1) {
+		RTE_LOG(ERR, KNI, "Failed to open file: %s.\n", path);
+		return -1;
+	}
+
+	ret = read(fd, old_carrier, 2);
+	if (ret < 1) {
+		close(fd);
+		return -1;
+	}
+	old_linkup = (old_carrier[0] == '1');
+
+	new_carrier = linkup ? "1" : "0";
+	ret = write(fd, new_carrier, 1);
+	if (ret < 1) {
+		RTE_LOG(ERR, KNI, "Failed to write file: %s.\n", path);
+		close(fd);
+		return -1;
+	}
+
+	close(fd);
+	return old_linkup;
+}
+
 void
 rte_kni_close(void)
 {
diff --git a/lib/librte_kni/rte_kni.h b/lib/librte_kni/rte_kni.h
index 601abdfc6..02ca43b4b 100644
--- a/lib/librte_kni/rte_kni.h
+++ b/lib/librte_kni/rte_kni.h
@@ -232,6 +232,26 @@ int rte_kni_register_handlers(struct rte_kni *kni, struct rte_kni_ops *ops);
  */
 int rte_kni_unregister_handlers(struct rte_kni *kni);
 
+/**
+ * Update link carrier state for KNI port.
+ *
+ * Update the linkup/linkdown state of a KNI interface in the kernel.
+ *
+ * @param kni
+ *  pointer to struct rte_kni.
+ * @param linkup
+ *  New link state:
+ *  0 for linkdown.
+ *  > 0 for linkup.
+ *
+ * @return
+ *  On failure: -1
+ *  Previous link state == linkdown: 0
+ *  Previous link state == linkup: 1
+ */
+int __rte_experimental
+rte_kni_update_link(struct rte_kni *kni, unsigned int linkup);
+
 /**
  *  Close KNI device.
  */
diff --git a/lib/librte_kni/rte_kni_version.map b/lib/librte_kni/rte_kni_version.map
index acd515eb0..c877dc6aa 100644
--- a/lib/librte_kni/rte_kni_version.map
+++ b/lib/librte_kni/rte_kni_version.map
@@ -15,3 +15,9 @@ DPDK_2.0 {
 
 	local: *;
 };
+
+EXPERIMENTAL {
+	global:
+
+	rte_kni_update_link;
+};
diff --git a/test/test/test_kni.c b/test/test/test_kni.c
index cf64c0b27..e38206905 100644
--- a/test/test/test_kni.c
+++ b/test/test/test_kni.c
@@ -118,6 +118,79 @@ kni_change_mtu(uint16_t port_id, unsigned int new_mtu)
 					 port_id, kni_pkt_mtu);
 	return 0;
 }
+
+static int
+test_kni_link_change(void)
+{
+	int ret;
+	int pid;
+
+	pid = fork();
+	if (pid < 0) {
+		printf("Error: Failed to fork a process\n");
+		return -1;
+	}
+
+	if (pid == 0) {
+		printf("Starting KNI Link status change tests.\n");
+		if (system(IFCONFIG TEST_KNI_PORT" up") == -1) {
+			ret = -1;
+			goto error;
+		}
+
+		ret = rte_kni_update_link(test_kni_ctx, 1);
+		if (ret < 0) {
+			printf("Failed to change link state to Up ret=%d.\n",
+				ret);
+			goto error;
+		}
+		rte_delay_ms(1000);
+		printf("KNI: Set LINKUP, previous state=%d\n", ret);
+
+		ret = rte_kni_update_link(test_kni_ctx, 0);
+		if (ret != 1) {
+			printf(
+		"Failed! Previous link state should be 1, returned %d.\n",
+				ret);
+			goto error;
+		}
+		rte_delay_ms(1000);
+		printf("KNI: Set LINKDOWN, previous state=%d\n", ret);
+
+		ret = rte_kni_update_link(test_kni_ctx, 1);
+		if (ret != 0) {
+			printf(
+		"Failed! Previous link state should be 0, returned %d.\n",
+				ret);
+			goto error;
+		}
+		printf("KNI: Set LINKUP, previous state=%d\n", ret);
+
+		ret = 0;
+		rte_delay_ms(1000);
+
+error:
+		if (system(IFCONFIG TEST_KNI_PORT" down") == -1)
+			ret = -1;
+
+		printf("KNI: Link status change tests: %s.\n",
+			(ret == 0) ? "Passed" : "Failed");
+		exit(ret);
+	} else {
+		int p_ret, status;
+
+		while (1) {
+			p_ret = waitpid(pid, &status, WNOHANG);
+			if (p_ret != 0) {
+				if (WIFEXITED(status))
+					return WEXITSTATUS(status);
+				return -1;
+			}
+			rte_delay_ms(10);
+			rte_kni_handle_request(test_kni_ctx);
+		}
+	}
+}
 /**
  * This loop fully tests the basic functions of KNI. e.g. transmitting,
  * receiving to, from kernel space, and kernel requests.
@@ -401,6 +474,10 @@ test_kni_processing(uint16_t port_id, struct rte_mempool *mp)
 		goto fail_kni;
 	}
 
+	ret = test_kni_link_change();
+	if (ret != 0)
+		goto fail_kni;
+
 	rte_eal_mp_remote_launch(test_kni_loop, NULL, CALL_MASTER);
 	RTE_LCORE_FOREACH_SLAVE(i) {
 		if (rte_eal_wait_lcore(i) < 0) {
-- 
2.19.0

[dpdk-dev] [PATCH v6 2/5] kni: set default carrier state of interface

[Dan Gora]

Add module parameter 'carrier='on|off' to set the default carrier state
for linux network interfaces created by the KNI module.  The default
carrier state is 'off'.

For KNI interfaces which need to reflect the carrier state of
a physical Ethernet port controlled by the DPDK application, the
default carrier state should be left set to 'off'.  The application
can set the carrier state of the KNI interface to reflect the state
of the physical Ethernet port using rte_kni_update_link().

For KNI interfaces which are purely virtual, the default carrier
state can be set to 'on'.  This enables the KNI interface to be
used without having to explicity set the carrier state to 'on'
using rte_kni_update_link().

Signed-off-by: Dan Gora <dg@adax.com>
---
 .../prog_guide/kernel_nic_interface.rst       | 239 ++++++++++++++----
 kernel/linux/kni/kni_dev.h                    |   3 +
 kernel/linux/kni/kni_misc.c                   |  60 ++++-
 kernel/linux/kni/kni_net.c                    |   5 +
 4 files changed, 254 insertions(+), 53 deletions(-)

diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst
index 8fa13fa1a..33ea980e5 100644
--- a/doc/guides/prog_guide/kernel_nic_interface.rst
+++ b/doc/guides/prog_guide/kernel_nic_interface.rst
@@ -29,58 +29,222 @@ The components of an application using the DPDK Kernel NIC Interface are shown i
 The DPDK KNI Kernel Module
 --------------------------
 
-The KNI kernel loadable module provides support for two types of devices:
+The KNI kernel loadable module ``rte_kni`` provides the kernel interface
+for DPDK applications.
 
-*   A Miscellaneous device (/dev/kni) that:
+When the ``rte_kni`` module is loaded, it will create a device ``/dev/kni``
+that is used by the DPDK KNI API functions to control and communicate with
+the kernel module.
 
-    *   Creates net devices (via ioctl  calls).
+The ``rte_kni`` kernel module contains several optional parameters which
+can be specified when the module is loaded to control its behavior:
 
-    *   Maintains a kernel thread context shared by all KNI instances
-        (simulating the RX side of the net driver).
+.. code-block:: console
 
-    *   For single kernel thread mode, maintains a kernel thread context shared by all KNI instances
-        (simulating the RX side of the net driver).
+    # modinfo rte_kni.ko
+    <snip>
+    parm:           lo_mode: KNI loopback mode (default=lo_mode_none):
+                    lo_mode_none        Kernel loopback disabled
+                    lo_mode_fifo        Enable kernel loopback with fifo
+                    lo_mode_fifo_skb    Enable kernel loopback with fifo and skb buffer
+                     (charp)
+    parm:           kthread_mode: Kernel thread mode (default=single):
+                    single    Single kernel thread mode enabled.
+                    multiple  Multiple kernel thread mode enabled.
+                     (charp)
+    parm:           carrier: Default carrier state for KNI interface (default=off):
+                    off   Interfaces will be created with carrier state set to off.
+                    on    Interfaces will be created with carrier state set to on.
+                     (charp)
 
-    *   For multiple kernel thread mode, maintains a kernel thread context for each KNI instance
-        (simulating the RX side of the net driver).
+Loading the ``rte_kni`` kernel module without any optional parameters is
+the typical way a DPDK application gets packets into and out of the kernel
+network stack.  Without any parameters, only one kernel thread is created
+for all KNI devices for packet receiving in kernel side, loopback mode is
+disabled, and the default carrier state of KNI interfaces is set to *off*.
 
-*   Net device:
+.. code-block:: console
 
-    *   Net functionality provided by implementing several operations such as netdev_ops,
-        header_ops, ethtool_ops that are defined by struct net_device,
-        including support for DPDK mbufs and FIFOs.
+    # insmod kmod/rte_kni.ko
 
-    *   The interface name is provided from userspace.
+.. _kni_loopback_mode:
 
-    *   The MAC address can be the real NIC MAC address or random.
+Loopback Mode
+~~~~~~~~~~~~~
+
+For testing, the ``rte_kni`` kernel module can be loaded in loopback mode
+by specifying the ``lo_mode`` parameter:
+
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko lo_mode=lo_mode_fifo
+
+The ``lo_mode_fifo`` loopback option will loop back ring enqueue/dequeue
+operations in kernel space.
+
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko lo_mode=lo_mode_fifo_skb
+
+The ``lo_mode_fifo_skb`` loopback option will loop back ring enqueue/dequeue
+operations and sk buffer copies in kernel space.
+
+If the ``lo_mode`` parameter is not specified, loopback mode is disabled.
+
+.. _kni_kernel_thread_mode:
+
+Kernel Thread Mode
+~~~~~~~~~~~~~~~~~~
+
+To provide flexibility of performance, the ``rte_kni`` KNI kernel module
+can be loaded with the ``kthread_mode`` parameter.  The ``rte_kni`` kernel
+module supports two options: "single kernel thread" mode and "multiple
+kernel thread" mode.
+
+Single kernel thread mode is enabled as follows:
+
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko kthread_mode=single
+
+This mode will create only one kernel thread for all KNI interfaces to
+receive data on the kernel side.  By default, this kernel thread is not
+bound to any particular core, but the user can set the core affinity for
+this kernel thread by setting the ``core_id`` and ``force_bind`` parameters
+in ``struct rte_kni_conf`` when the first KNI interface is created:
+
+For optimum performance, the kernel thread should be bound to a core in
+on the same socket as the DPDK lcores used in the application.
+
+The KNI kernel module can also be configured to start a separate kernel
+thread for each KNI interface created by the DPDK application.  Multiple
+kernel thread mode is enabled as follows:
+
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko kthread_mode=multiple
+
+This mode will create a separate kernel thread for each KNI interface to
+receive data on the kernel side.  The core affinity of each ``kni_thread``
+kernel thread can be specified by setting the ``core_id`` and ``force_bind``
+parameters in ``struct rte_kni_conf`` when each KNI interface is created.
+
+Multiple kernel thread mode can provide scalable higher performance if
+sufficient unused cores are available on the host system.
+
+If the ``kthread_mode`` parameter is not specified, the "single kernel
+thread" mode is used.
+
+.. _kni_default_carrier_state:
+
+Default Carrier State
+~~~~~~~~~~~~~~~~~~~~~
+
+The default carrier state of KNI interfaces created by the ``rte_kni``
+kernel module is controlled via the ``carrier`` option when the module
+is loaded.
+
+If ``carrier=off`` is specified, the kernel module will leave the carrier
+state of the interface *down* when the interface is management enabled.
+The DPDK application can set the carrier state of the KNI interface using the
+``rte_kni_update_link()`` function.  This is useful for DPDK applications
+which require that the carrier state of the KNI interface reflect the
+actual link state of the corresponding physical NIC port.
+
+If ``carrier=on`` is specified, the kernel module will automatically set
+the carrier state of the interface to *up* when the interface is management
+enabled.  This is useful for DPDK applications which use the KNI interface as
+a purely virtual interface that does not correspond to any physical hardware
+and do not wish to explicitly set the carrier state of the interface with
+``rte_kni_update_link()``.  It is also useful for testing in loopback mode
+where the NIC port may not be physically connected to anything.
+
+To set the default carrier state to *on*:
+
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko carrier=on
+
+To set the default carrier state to *off*:
+
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko carrier=off
+
+If the ``carrier`` parameter is not specified, the default carrier state
+of KNI interfaces will be set to *off*.
 
 KNI Creation and Deletion
 -------------------------
 
-The KNI interfaces are created by a DPDK application dynamically.
-The interface name and FIFO details are provided by the application through an ioctl call
-using the rte_kni_device_info struct which contains:
+Before any KNI interfaces can be created, the ``rte_kni`` kernel module must
+be loaded into the kernel and configured withe ``rte_kni_init()`` function.
+
+The KNI interfaces are created by a DPDK application dynamically via the
+``rte_kni_alloc()`` function.
+
+The ``struct rte_kni_conf`` structure contains fields which allow the
+user to specify the interface name, set the MTU size, set an explicit or
+random MAC address and control the affinity of the kernel Rx thread(s)
+(both single and multi-threaded modes).
+
+The ``struct rte_kni_ops`` structure contains pointers to functions to
+handle requests from the ``rte_kni`` kernel module.  These functions
+allow DPDK applications to perform actions when the KNI interfaces are
+manipulated by control commands or functions external to the application.
+
+For example, the DPDK application may wish to enabled/disable a physical
+NIC port when a user enabled/disables a KNI interface with ``ip link set
+[up|down] dev <ifaceX>``.  The DPDK application can register a callback for
+``config_network_if`` which will be called when the interface management
+state changes.
+
+There are currently four callbacks for which the user can register
+application functions:
 
-*   The interface name.
+``config_network_if``:
 
-*   Physical addresses of the corresponding memzones for the relevant FIFOs.
+    Called when the management state of the KNI interface changes.
+    For example, when the user runs ``ip link set [up|down] dev <ifaceX>``.
 
-*   Mbuf mempool details, both physical and virtual (to calculate the offset for mbuf pointers).
+``change_mtu``:
 
-*   PCI information.
+    Called when the user changes the MTU size of the KNI
+    interface.  For example, when the user runs ``ip link set mtu <size>
+    dev <ifaceX>``.
 
-*   Core affinity.
+``config_mac_address``:
 
-Refer to rte_kni_common.h in the DPDK source code for more details.
+    Called when the user changes the MAC address of the KNI interface.
+    For example, when the user runs ``ip link set address <MAC>
+    dev <ifaceX>``.  If the user sets this callback function to NULL,
+    but sets the ``port_id`` field to a value other than -1, a default
+    callback handler in the rte_kni library ``kni_config_mac_address()``
+    will be called which calls ``rte_eth_dev_default_mac_addr_set()``
+    on the specified ``port_id``.
 
-The physical addresses will be re-mapped into the kernel address space and stored in separate KNI contexts.
+``config_promiscusity``:
 
-The affinity of kernel RX thread (both single and multi-threaded modes) is controlled by force_bind and
-core_id config parameters.
+    Called when the user changes the promiscusity state of the KNI
+    interface.  For example, when the user runs ``ip link set promisc
+    [on|off] dev <ifaceX>``. If the user sets this callback function to
+    NULL, but sets the ``port_id`` field to a value other than -1, a default
+    callback handler in the rte_kni library ``kni_config_promiscusity()``
+    will be called which calls ``rte_eth_promiscuous_<enable|disable>()``
+    on the specified ``port_id``.
 
-The KNI interfaces can be deleted by a DPDK application dynamically after being created.
-Furthermore, all those KNI interfaces not deleted will be deleted on the release operation
-of the miscellaneous device (when the DPDK application is closed).
+In order to run these callbacks, the application must periodically call
+the ``rte_kni_handle_request()`` function.  Any user callback function
+registered will be called directly from ``rte_kni_handle_request()`` so
+care must be taken to prevent deadlock and to not block any DPDK fastpath
+tasks.  Typically DPDK applications which use these callbacks will need
+to create a separate thread or secondary process to periodically call
+``rte_kni_handle_request()``.
+
+The KNI interfaces can be deleted by a DPDK application with
+``rte_kni_release()``.  All KNI interfaces not explicitly deleted will be
+deleted when the the ``/dev/kni`` device is closed, either explicitly with
+``rte_kni_close()`` or when the DPDK application is closed.
 
 DPDK mbuf Flow
 --------------
@@ -118,7 +282,7 @@ The packet is received from the Linux net stack, by calling the kni_net_tx() cal
 The mbuf is dequeued (without waiting due the cache) and filled with data from sk_buff.
 The sk_buff is then freed and the mbuf sent in the tx_q FIFO.
 
-The DPDK TX thread dequeues the mbuf and sends it to the PMD (via rte_eth_tx_burst()).
+The DPDK TX thread dequeues the mbuf and sends it to the PMD via ``rte_eth_tx_burst()``.
 It then puts the mbuf back in the cache.
 
 Ethtool
@@ -128,16 +292,3 @@ Ethtool is a Linux-specific tool with corresponding support in the kernel
 where each net device must register its own callbacks for the supported operations.
 The current implementation uses the igb/ixgbe modified Linux drivers for ethtool support.
 Ethtool is not supported in i40e and VMs (VF or EM devices).
-
-Link state and MTU change
--------------------------
-
-Link state and MTU change are network interface specific operations usually done via ifconfig.
-The request is initiated from the kernel side (in the context of the ifconfig process)
-and handled by the user space DPDK application.
-The application polls the request, calls the application handler and returns the response back into the kernel space.
-
-The application handlers can be registered upon interface creation or explicitly registered/unregistered in runtime.
-This provides flexibility in multiprocess scenarios
-(where the KNI is created in the primary process but the callbacks are handled in the secondary one).
-The constraint is that a single process can register and handle the requests.
diff --git a/kernel/linux/kni/kni_dev.h b/kernel/linux/kni/kni_dev.h
index 6275ef27f..688f574a4 100644
--- a/kernel/linux/kni/kni_dev.h
+++ b/kernel/linux/kni/kni_dev.h
@@ -29,6 +29,9 @@
 
 #define MBUF_BURST_SZ 32
 
+/* Default carrier state for created KNI network interfaces */
+extern uint32_t dflt_carrier;
+
 /**
  * A structure describing the private information for a kni device.
  */
diff --git a/kernel/linux/kni/kni_misc.c b/kernel/linux/kni/kni_misc.c
index fa69f8e63..522ae23b9 100644
--- a/kernel/linux/kni/kni_misc.c
+++ b/kernel/linux/kni/kni_misc.c
@@ -39,6 +39,10 @@ static char *lo_mode;
 static char *kthread_mode;
 static uint32_t multiple_kthread_on;
 
+/* Default carrier state for created KNI network interfaces */
+static char *carrier;
+uint32_t dflt_carrier;
+
 #define KNI_DEV_IN_USE_BIT_NUM 0 /* Bit number for device in use */
 
 static int kni_net_id;
@@ -466,6 +470,8 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_num,
 		return -ENODEV;
 	}
 
+	netif_carrier_off(net_dev);
+
 	ret = kni_run_thread(knet, kni, dev_info.force_bind);
 	if (ret != 0)
 		return ret;
@@ -590,6 +596,24 @@ kni_parse_kthread_mode(void)
 	return 0;
 }
 
+static int __init
+kni_parse_carrier_state(void)
+{
+	if (!carrier) {
+		dflt_carrier = 0;
+		return 0;
+	}
+
+	if (strcmp(carrier, "off") == 0)
+		dflt_carrier = 0;
+	else if (strcmp(carrier, "on") == 0)
+		dflt_carrier = 1;
+	else
+		return -1;
+
+	return 0;
+}
+
 static int __init
 kni_init(void)
 {
@@ -605,6 +629,16 @@ kni_init(void)
 	else
 		pr_debug("Multiple kernel thread mode enabled\n");
 
+	if (kni_parse_carrier_state() < 0) {
+		pr_err("Invalid parameter for carrier\n");
+		return -EINVAL;
+	}
+
+	if (dflt_carrier == 0)
+		pr_debug("Default carrier state set to off.\n");
+	else
+		pr_debug("Default carrier state set to on.\n");
+
 #ifdef HAVE_SIMPLIFIED_PERNET_OPERATIONS
 	rc = register_pernet_subsys(&kni_net_ops);
 #else
@@ -647,19 +681,27 @@ kni_exit(void)
 module_init(kni_init);
 module_exit(kni_exit);
 
-module_param(lo_mode, charp, S_IRUGO | S_IWUSR);
+module_param(lo_mode, charp, 0644);
 MODULE_PARM_DESC(lo_mode,
 "KNI loopback mode (default=lo_mode_none):\n"
-"    lo_mode_none        Kernel loopback disabled\n"
-"    lo_mode_fifo        Enable kernel loopback with fifo\n"
-"    lo_mode_fifo_skb    Enable kernel loopback with fifo and skb buffer\n"
-"\n"
+"\t\tlo_mode_none        Kernel loopback disabled\n"
+"\t\tlo_mode_fifo        Enable kernel loopback with fifo\n"
+"\t\tlo_mode_fifo_skb    Enable kernel loopback with fifo and skb buffer\n"
+"\t\t"
 );
 
-module_param(kthread_mode, charp, S_IRUGO);
+module_param(kthread_mode, charp, 0644);
 MODULE_PARM_DESC(kthread_mode,
 "Kernel thread mode (default=single):\n"
-"    single    Single kernel thread mode enabled.\n"
-"    multiple  Multiple kernel thread mode enabled.\n"
-"\n"
+"\t\tsingle    Single kernel thread mode enabled.\n"
+"\t\tmultiple  Multiple kernel thread mode enabled.\n"
+"\t\t"
+);
+
+module_param(carrier, charp, 0644);
+MODULE_PARM_DESC(carrier,
+"Default carrier state for KNI interface (default=off):\n"
+"\t\toff   Interfaces will be created with carrier state set to off.\n"
+"\t\ton    Interfaces will be created with carrier state set to on.\n"
+"\t\t"
 );
diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c
index 3a542d737..7371b6d58 100644
--- a/kernel/linux/kni/kni_net.c
+++ b/kernel/linux/kni/kni_net.c
@@ -133,6 +133,10 @@ kni_net_open(struct net_device *dev)
 	struct kni_dev *kni = netdev_priv(dev);
 
 	netif_start_queue(dev);
+	if (dflt_carrier == 1)
+		netif_carrier_on(dev);
+	else
+		netif_carrier_off(dev);
 
 	memset(&req, 0, sizeof(req));
 	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
@@ -152,6 +156,7 @@ kni_net_release(struct net_device *dev)
 	struct kni_dev *kni = netdev_priv(dev);
 
 	netif_stop_queue(dev); /* can't transmit any more */
+	netif_carrier_off(dev);
 
 	memset(&req, 0, sizeof(req));
 	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
-- 
2.19.0

[dpdk-dev] [PATCH v6 3/5] examples/kni: monitor and update link status continually

[Dan Gora]

Update KNI example to add the command line flag '-m' to enable
a function to continuously monitor the Ethernet link status of
the physical link and update the link status of the corresponding
interfaces with rte_kni_update_link().

Signed-off-by: Dan Gora <dg@adax.com>
---
 .../sample_app_ug/kernel_nic_interface.rst    | 251 +++++++++++-------
 examples/kni/Makefile                         |   2 +
 examples/kni/main.c                           |  70 ++++-
 3 files changed, 226 insertions(+), 97 deletions(-)

diff --git a/doc/guides/sample_app_ug/kernel_nic_interface.rst b/doc/guides/sample_app_ug/kernel_nic_interface.rst
index 1b3ee9a5e..859b3f725 100644
--- a/doc/guides/sample_app_ug/kernel_nic_interface.rst
+++ b/doc/guides/sample_app_ug/kernel_nic_interface.rst
@@ -31,18 +31,27 @@ This is done by creating one or more kernel net devices for each of the DPDK por
 The application allows the use of standard Linux tools (ethtool, ifconfig, tcpdump) with the DPDK ports and
 also the exchange of packets between the DPDK application and the Linux* kernel.
 
+The Kernel NIC Interface sample application requires that the
+KNI kernel module ``rte_kni`` be loaded into the kernel.  See
+:doc:`../prog_guide/kernel_nic_interface` for more information on loading
+the ``rte_kni`` kernel module.
+
 Overview
 --------
 
-The Kernel NIC Interface sample application uses two threads in user space for each physical NIC port being used,
-and allocates one or more KNI device for each physical NIC port with kernel module's support.
-For a physical NIC port, one thread reads from the port and writes to KNI devices,
-and another thread reads from KNI devices and writes the data unmodified to the physical NIC port.
-It is recommended to configure one KNI device for each physical NIC port.
-If configured with more than one KNI devices for a physical NIC port,
-it is just for performance testing, or it can work together with VMDq support in future.
+The Kernel NIC Interface sample application ``kni`` allocates one or more
+KNI interfaces for each physical NIC port.  For each physical NIC port,
+``kni`` uses two DPDK threads in user space; one thread reads from the port and
+writes to the corresponding KNI interfaces and the other thread reads from
+the KNI interfaces and writes the data unmodified to the physical NIC port.
+
+It is recommended to configure one KNI interface for each physical NIC port.
+The application can be configured with more than one KNI interface for
+each physical NIC port for performance testing or it can work together with
+VMDq support in future.
 
-The packet flow through the Kernel NIC Interface application is as shown in the following figure.
+The packet flow through the Kernel NIC Interface application is as shown
+in the following figure.
 
 .. _figure_kernel_nic:
 
@@ -50,145 +59,210 @@ The packet flow through the Kernel NIC Interface application is as shown in the
 
    Kernel NIC Application Packet Flow
 
+If link monitoring is enabled with the ``-m`` command line flag, one
+additional pthread is launched which will check the link status of each
+physical NIC port and will update the carrier status of the corresponding
+KNI interface(s) to match the physical NIC port's state.  This means that
+the KNI interface(s) will be disabled automatically when the Ethernet link
+goes down and enabled when the Ethernet link goes up.
+
+If link monitoring is enabled, the ``rte_kni`` kernel module should be loaded
+such that the :ref:`default carrier state <kni_default_carrier_state>` is
+set to *off*.  This ensures that the KNI interface is only enabled *after*
+the Ethernet link of the corresponding NIC port has reached the linkup state.
+
+If link monitoring is not enabled, the ``rte_kni`` kernel module should be
+loaded with the :ref:`default carrier state <kni_default_carrier_state>`
+set to *on*.  This sets the carrier state of the KNI interfaces to *on*
+when the KNI interfaces are enabled without regard to the actual link state
+of the corresponding NIC port.  This is useful for testing in loopback
+mode where the NIC port may not be physically connected to anything.
+
 Compiling the Application
 -------------------------
 
 To compile the sample application see :doc:`compiling`.
 
-The application is located in the ``kni`` sub-directory.
+The application is located in the ``examples/kni`` sub-directory.
 
 .. note::
 
         This application is intended as a linuxapp only.
 
-Loading the Kernel Module
--------------------------
+Running the kni Example Application
+-----------------------------------
 
-Loading the KNI kernel module without any parameter is the typical way a DPDK application
-gets packets into and out of the kernel net stack.
-This way, only one kernel thread is created for all KNI devices for packet receiving in kernel side:
+The ``kni`` example application requires a number of command line options:
 
 .. code-block:: console
 
-    #insmod rte_kni.ko
+    kni [EAL options] -- -p PORTMASK --config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,(port,lcore_rx,lcore_tx[,lcore_kthread,...])]" [-P] [-m]
 
-Pinning the kernel thread to a specific core can be done using a taskset command such as following:
+Where:
 
-.. code-block:: console
+*   ``-p PORTMASK``:
 
-    #taskset -p 100000 `pgrep --fl kni_thread | awk '{print $1}'`
+    Hexadecimal bitmask of ports to configure.
 
-This command line tries to pin the specific kni_thread on the 20th lcore (lcore numbering starts at 0),
-which means it needs to check if that lcore is available on the board.
-This command must be sent after the application has been launched, as insmod does not start the kni thread.
+*   ``--config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,(port,lcore_rx,lcore_tx[,lcore_kthread,...])]"``:
 
-For optimum performance,
-the lcore in the mask must be selected to be on the same socket as the lcores used in the KNI application.
+    Determines which lcores the Rx and Tx DPDK tasks, and (optionally)
+    the KNI kernel thread(s) are bound to for each physical port.
 
-To provide flexibility of performance, the kernel module of the KNI,
-located in the kmod sub-directory of the DPDK target directory,
-can be loaded with parameter of kthread_mode as follows:
+*   ``-P``:
 
-*   #insmod rte_kni.ko kthread_mode=single
+    Optional flag to set all ports to promiscuous mode so that packets are
+    accepted regardless of the packet's Ethernet MAC destination address.
+    Without this option, only packets with the Ethernet MAC destination
+    address set to the Ethernet address of the port are accepted.
 
-    This mode will create only one kernel thread for all KNI devices for packet receiving in kernel side.
-    By default, it is in this single kernel thread mode.
-    It can set core affinity for this kernel thread by using Linux command taskset.
+*   ``-m``:
 
-*   #insmod rte_kni.ko kthread_mode =multiple
+    Optional flag to enable monitoring and updating of the Ethernet
+    carrier state.  With this option set, a thread will be started which
+    will periodically check the Ethernet link status of the physical
+    Ethernet ports and set the carrier state of the corresponding KNI
+    network interface to match it.  This means that the KNI interface will
+    be disabled automatically when the Ethernet link goes down and enabled
+    when the Ethernet link goes up.
 
-    This mode will create a kernel thread for each KNI device for packet receiving in kernel side.
-    The core affinity of each kernel thread is set when creating the KNI device.
-    The lcore ID for each kernel thread is provided in the command line of launching the application.
-    Multiple kernel thread mode can provide scalable higher performance.
+Refer to *DPDK Getting Started Guide* for general information on running
+applications and the Environment Abstraction Layer (EAL) options.
 
-To measure the throughput in a loopback mode, the kernel module of the KNI,
-located in the kmod sub-directory of the DPDK target directory,
-can be loaded with parameters as follows:
+The ``-c coremask`` or ``-l corelist`` parameter of the EAL options must
+include the lcores specified by ``lcore_rx`` and ``lcore_tx`` for each port,
+but does not need to include lcores specified by ``lcore_kthread`` as those
+cores are used to pin the kernel threads in the ``rte_kni`` kernel module.
 
-*   #insmod rte_kni.ko lo_mode=lo_mode_fifo
+The ``--config`` parameter must include a set of
+``(port,lcore_rx,lcore_tx,[lcore_kthread,...])`` values for each physical
+port specified in the ``-p PORTMASK`` parameter.
 
-    This loopback mode will involve ring enqueue/dequeue operations in kernel space.
+The optional ``lcore_kthread`` lcore ID parameter in ``--config`` can be
+specified zero, one or more times for each physical port.
 
-*   #insmod rte_kni.ko lo_mode=lo_mode_fifo_skb
+If no lcore ID is specified for ``lcore_kthread``, one KNI interface will
+be created for the physical port ``port`` and the KNI kernel thread(s)
+will have no specific core affinity.
 
-    This loopback mode will involve ring enqueue/dequeue operations and sk buffer copies in kernel space.
+If one or more lcore IDs are specified for ``lcore_kthread``, a KNI interface
+will be created for each lcore ID specified, bound to the physical port
+``port``.  If the ``rte_kni`` kernel module is loaded in :ref:`multiple
+kernel thread <kni_kernel_thread_mode>` mode, a kernel thread will be created
+for each KNI interface and bound to the specified core.  If the ``rte_kni``
+kernel module is loaded in :ref:`single kernel thread <kni_kernel_thread_mode>`
+mode, only one kernel thread is started for all KNI interfaces.  The kernel
+thread will be bound to the first ``lcore_kthread`` lcore ID specified.
 
-Running the Application
------------------------
+Example Configurations
+~~~~~~~~~~~~~~~~~~~~~~~
 
-The application requires a number of command line options:
+The following commands will first load the ``rte_kni`` kernel module in
+:ref:`multiple kernel thread <kni_kernel_thread_mode>` mode.  The ``kni``
+application is then started using two ports;  Port 0 uses lcore 4 for the
+Rx task, lcore 6 for the Tx task, and will create a single KNI interface
+``vEth0_0`` with the kernel thread bound to lcore 8.  Port 1 uses lcore
+5 for the Rx task, lcore 7 for the Tx task, and will create a single KNI
+interface ``vEth1_0`` with the kernel thread bound to lcore 9.
 
 .. code-block:: console
 
-    kni [EAL options] -- -P -p PORTMASK --config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,port,lcore_rx,lcore_tx[,lcore_kthread,...]]"
+    # rmmod rte_kni
+    # insmod kmod/rte_kni.ko kthread_mode=multiple
+    # ./build/kni -l 4-7 -n 4 -- -P -p 0x3 -m --config="(0,4,6,8),(1,5,7,9)"
 
-Where:
-
-*   -P: Set all ports to promiscuous mode so that packets are accepted regardless of the packet's Ethernet MAC destination address.
-    Without this option, only packets with the Ethernet MAC destination address set to the Ethernet address of the port are accepted.
+The following example is identical, except an additional ``lcore_kthread``
+core is specified per physical port.  In this case, ``kni`` will create
+four KNI interfaces: ``vEth0_0``/``vEth0_1`` bound to physical port 0 and
+``vEth1_0``/``vEth1_1`` bound to physical port 1.
 
-*   -p PORTMASK: Hexadecimal bitmask of ports to configure.
+The kernel thread for each interface will be bound as follows:
 
-*   --config="(port,lcore_rx, lcore_tx[,lcore_kthread, ...]) [, port,lcore_rx, lcore_tx[,lcore_kthread, ...]]":
-    Determines which lcores of RX, TX, kernel thread are mapped to which ports.
+    * ``vEth0_0`` - bound to lcore 8.
+    * ``vEth0_1`` - bound to lcore 10.
+    * ``vEth1_0`` - bound to lcore 9.
+    * ``vEth1_1`` - bound to lcore 11
 
-Refer to *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options.
+.. code-block:: console
 
-The -c coremask or -l corelist parameter of the EAL options should include the lcores indicated by the lcore_rx and lcore_tx,
-but does not need to include lcores indicated by lcore_kthread as they are used to pin the kernel thread on.
-The -p PORTMASK parameter should include the ports indicated by the port in --config, neither more nor less.
+    # rmmod rte_kni
+    # insmod kmod/rte_kni.ko kthread_mode=multiple
+    # ./build/kni -l 4-7 -n 4 -- -P -p 0x3 -m --config="(0,4,6,8,10),(1,5,7,9,11)"
 
-The lcore_kthread in --config can be configured none, one or more lcore IDs.
-In multiple kernel thread mode, if configured none, a KNI device will be allocated for each port,
-while no specific lcore affinity will be set for its kernel thread.
-If configured one or more lcore IDs, one or more KNI devices will be allocated for each port,
-while specific lcore affinity will be set for its kernel thread.
-In single kernel thread mode, if configured none, a KNI device will be allocated for each port.
-If configured one or more lcore IDs,
-one or more KNI devices will be allocated for each port while
-no lcore affinity will be set as there is only one kernel thread for all KNI devices.
+The following example can be used to test the interface between the ``kni``
+test application and the ``rte_kni`` kernel module.  In this example,
+the ``rte_kni`` kernel module is loaded in :ref:`single kernel thread
+mode <kni_kernel_thread_mode>`, :ref:`loopback mode <kni_loopback_mode>`
+enabled, and the :ref:`default carrier state <kni_default_carrier_state>`
+is set to *on* so that the corresponding physical NIC port does not have
+to be connected in order to use the KNI interface.  One KNI interface
+``vEth0_0`` is created for port 0 and one KNI interface ``vEth1_0`` is
+created for port 1.  Since ``rte_kni`` is loaded in "single kernel thread"
+mode, the one kernel thread is bound to lcore 8.
 
-For example, to run the application with two ports served by six lcores, one lcore of RX, one lcore of TX,
-and one lcore of kernel thread for each port:
+Since the physical NIC ports are not being used, link monitoring can be
+disabled by **not** specifying the ``-m`` flag to ``kni``:
 
 .. code-block:: console
 
-    ./build/kni -l 4-7 -n 4 -- -P -p 0x3 --config="(0,4,6,8),(1,5,7,9)"
+    # rmmod rte_kni
+    # insmod kmod/rte_kni.ko lo_mode=lo_mode_fifo carrier=on
+    # ./build/kni -l 4-7 -n 4 -- -P -p 0x3 --config="(0,4,6,8),(1,5,7,9)"
 
 KNI Operations
 --------------
 
-Once the KNI application is started, one can use different Linux* commands to manage the net interfaces.
-If more than one KNI devices configured for a physical port,
-only the first KNI device will be paired to the physical device.
-Operations on other KNI devices will not affect the physical port handled in user space application.
+Once the ``kni`` application is started, the user can use the normal
+Linux commands to manage the KNI interfaces as if they were any other
+Linux network interface.
 
-Assigning an IP address:
+Enable KNI interface and assign an IP address:
 
 .. code-block:: console
 
-    #ifconfig vEth0_0 192.168.0.1
+    # ifconfig vEth0_0 192.168.0.1
 
-Displaying the NIC registers:
+Show KNI interface configuration and statistics:
 
 .. code-block:: console
 
-    #ethtool -d vEth0_0
+    # ifconfig vEth0_0
 
-Dumping the network traffic:
+Dump network traffic:
 
 .. code-block:: console
 
-    #tcpdump -i vEth0_0
+    # tcpdump -i vEth0_0
+
+The normal Linux commands can also be used to change the MAC address and
+MTU size used by the physical NIC which corresponds to the KNI interface.
+However, if more than one KNI interface is configured for a physical port,
+these commands will only work on the first KNI interface for that port.
 
 Change the MAC address:
 
 .. code-block:: console
 
-    #ifconfig vEth0_0 hw ether 0C:01:02:03:04:08
+    # ifconfig vEth0_0 hw ether 0C:01:02:03:04:08
+
+Change the MTU size:
+
+.. code-block:: console
+
+    # ifconfig vEth0_0 mtu 1450
+
+If DPDK is compiled with ``CONFIG_RTE_KNI_KMOD_ETHTOOL=y`` and an Intel
+NIC is used, the user can use ``ethtool`` on the KNI interface as if it
+were a normal Linux kernel interface.
+
+Displaying the NIC registers:
+
+.. code-block:: console
+
+    # ethtool -d vEth0_0
 
-When the DPDK userspace application is closed, all the KNI devices are deleted from Linux*.
+When the ``kni`` application is closed, all the KNI interfaces are deleted
+from the Linux kernel.
 
 Explanation
 -----------
@@ -227,7 +301,7 @@ to see if this lcore is reading from or writing to kernel NIC interfaces.
 For the case that reads from a NIC port and writes to the kernel NIC interfaces (``kni_ingress``),
 the packet reception is the same as in L2 Forwarding sample application
 (see :ref:`l2_fwd_app_rx_tx_packets`).
-The packet transmission is done by sending mbufs into the kernel NIC interfaces by rte_kni_tx_burst().
+The packet transmission is done by sending mbufs into the kernel NIC interfaces by ``rte_kni_tx_burst()``.
 The KNI library automatically frees the mbufs after the kernel successfully copied the mbufs.
 
 For the other case that reads from kernel NIC interfaces
@@ -235,16 +309,3 @@ and writes to a physical NIC port (``kni_egress``),
 packets are retrieved by reading mbufs from kernel NIC interfaces by ``rte_kni_rx_burst()``.
 The packet transmission is the same as in the L2 Forwarding sample application
 (see :ref:`l2_fwd_app_rx_tx_packets`).
-
-Callbacks for Kernel Requests
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-To execute specific PMD operations in user space requested by some Linux* commands,
-callbacks must be implemented and filled in the struct rte_kni_ops structure.
-Currently, setting a new MTU, change in MAC address, configuring promiscusous mode and
-configuring the network interface(up/down) re supported.
-Default implementation for following is available in rte_kni library.
-Application may choose to not implement following callbacks:
-
-- ``config_mac_address``
-- ``config_promiscusity``
diff --git a/examples/kni/Makefile b/examples/kni/Makefile
index 7e19d2e2a..dd90d7d73 100644
--- a/examples/kni/Makefile
+++ b/examples/kni/Makefile
@@ -20,6 +20,7 @@ static: build/$(APP)-static
 
 PC_FILE := $(shell pkg-config --path libdpdk)
 CFLAGS += -O3 $(shell pkg-config --cflags libdpdk)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk)
 LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk)
 
@@ -54,6 +55,7 @@ please change the definition of the RTE_TARGET environment variable)
 endif
 
 CFLAGS += -O3
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 CFLAGS += $(WERROR_FLAGS)
 
 include $(RTE_SDK)/mk/rte.extapp.mk
diff --git a/examples/kni/main.c b/examples/kni/main.c
index 80c401c51..4fda51713 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -106,6 +106,8 @@ static struct rte_mempool * pktmbuf_pool = NULL;
 static uint32_t ports_mask = 0;
 /* Ports set in promiscuous mode off by default. */
 static int promiscuous_on = 0;
+/* Monitor link status continually. off by default. */
+static int monitor_links;
 
 /* Structure type for recording kni interface specific stats */
 struct kni_interface_stats {
@@ -325,11 +327,12 @@ main_loop(__rte_unused void *arg)
 static void
 print_usage(const char *prgname)
 {
-	RTE_LOG(INFO, APP, "\nUsage: %s [EAL options] -- -p PORTMASK -P "
+	RTE_LOG(INFO, APP, "\nUsage: %s [EAL options] -- -p PORTMASK -P -m "
 		   "[--config (port,lcore_rx,lcore_tx,lcore_kthread...)"
 		   "[,(port,lcore_rx,lcore_tx,lcore_kthread...)]]\n"
 		   "    -p PORTMASK: hex bitmask of ports to use\n"
 		   "    -P : enable promiscuous mode\n"
+		   "    -m : enable monitoring of port carrier state\n"
 		   "    --config (port,lcore_rx,lcore_tx,lcore_kthread...): "
 		   "port and lcore configurations\n",
 	           prgname);
@@ -510,7 +513,7 @@ parse_args(int argc, char **argv)
 	opterr = 0;
 
 	/* Parse command line */
-	while ((opt = getopt_long(argc, argv, "p:P", longopts,
+	while ((opt = getopt_long(argc, argv, "p:Pm", longopts,
 						&longindex)) != EOF) {
 		switch (opt) {
 		case 'p':
@@ -519,6 +522,9 @@ parse_args(int argc, char **argv)
 		case 'P':
 			promiscuous_on = 1;
 			break;
+		case 'm':
+			monitor_links = 1;
+			break;
 		case 0:
 			if (!strncmp(longopts[longindex].name,
 				     CMDLINE_OPT_CONFIG,
@@ -674,6 +680,55 @@ check_all_ports_link_status(uint32_t port_mask)
 	}
 }
 
+static void
+log_link_state(struct rte_kni *kni, int prev, struct rte_eth_link *link)
+{
+	if (kni == NULL || link == NULL)
+		return;
+
+	if (prev == ETH_LINK_DOWN && link->link_status == ETH_LINK_UP) {
+		RTE_LOG(INFO, APP, "%s NIC Link is Up %d Mbps %s %s.\n",
+			rte_kni_get_name(kni),
+			link->link_speed,
+			link->link_autoneg ?  "(AutoNeg)" : "(Fixed)",
+			link->link_duplex ?  "Full Duplex" : "Half Duplex");
+	} else if (prev == ETH_LINK_UP && link->link_status == ETH_LINK_DOWN) {
+		RTE_LOG(INFO, APP, "%s NIC Link is Down.\n",
+			rte_kni_get_name(kni));
+	}
+}
+
+/*
+ * Monitor the link status of all ports and update the
+ * corresponding KNI interface(s)
+ */
+static void *
+monitor_all_ports_link_status(void *arg)
+{
+	uint16_t portid;
+	struct rte_eth_link link;
+	unsigned int i;
+	struct kni_port_params **p = kni_port_params_array;
+	int prev;
+	(void) arg;
+
+	while (monitor_links) {
+		rte_delay_ms(500);
+		RTE_ETH_FOREACH_DEV(portid) {
+			if ((ports_mask & (1 << portid)) == 0)
+				continue;
+			memset(&link, 0, sizeof(link));
+			rte_eth_link_get_nowait(portid, &link);
+			for (i = 0; i < p[portid]->nb_kni; i++) {
+				prev = rte_kni_update_link(p[portid]->kni[i],
+						link.link_status);
+				log_link_state(p[portid]->kni[i], prev, &link);
+			}
+		}
+	}
+	return NULL;
+}
+
 /* Callback for request of changing MTU */
 static int
 kni_change_mtu(uint16_t port_id, unsigned int new_mtu)
@@ -893,6 +948,8 @@ main(int argc, char** argv)
 	int ret;
 	uint16_t nb_sys_ports, port;
 	unsigned i;
+	void *retval;
+	pthread_t kni_link_tid;
 
 	/* Associate signal_hanlder function with USR signals */
 	signal(SIGUSR1, signal_handler);
@@ -949,12 +1006,21 @@ main(int argc, char** argv)
 	}
 	check_all_ports_link_status(ports_mask);
 
+	ret = rte_ctrl_thread_create(&kni_link_tid,
+				     "KNI link status check", NULL,
+				     monitor_all_ports_link_status, NULL);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Could not create link status thread!\n");
+
 	/* Launch per-lcore function on every lcore */
 	rte_eal_mp_remote_launch(main_loop, NULL, CALL_MASTER);
 	RTE_LCORE_FOREACH_SLAVE(i) {
 		if (rte_eal_wait_lcore(i) < 0)
 			return -1;
 	}
+	monitor_links = 0;
+	pthread_join(kni_link_tid, &retval);
 
 	/* Release resources */
 	RTE_ETH_FOREACH_DEV(port) {
-- 
2.19.0

[dpdk-dev] [PATCH v6 4/5] examples/kni: add log msgs to show and clear stats

[Dan Gora]

Add logging messages showing the commands necessary for the user to
have the application display and zero the statistics.

Signed-off-by: Dan Gora <dg@adax.com>
---
 examples/kni/main.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/examples/kni/main.c b/examples/kni/main.c
index 4fda51713..0e3b2a2f7 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -171,14 +171,13 @@ signal_handler(int signum)
 	/* When we receive a USR2 signal, reset stats */
 	if (signum == SIGUSR2) {
 		memset(&kni_stats, 0, sizeof(kni_stats));
-		printf("\n**Statistics have been reset**\n");
+		printf("\n** Statistics have been reset **\n");
 		return;
 	}
 
 	/* When we receive a RTMIN or SIGINT signal, stop kni processing */
 	if (signum == SIGRTMIN || signum == SIGINT){
-		printf("SIGRTMIN is received, and the KNI processing is "
-							"going to stop\n");
+		printf("\nSIGRTMIN/SIGINT received. KNI processing stopping.\n");
 		rte_atomic32_inc(&kni_stop);
 		return;
         }
@@ -950,6 +949,7 @@ main(int argc, char** argv)
 	unsigned i;
 	void *retval;
 	pthread_t kni_link_tid;
+	int pid;
 
 	/* Associate signal_hanlder function with USR signals */
 	signal(SIGUSR1, signal_handler);
@@ -1006,6 +1006,16 @@ main(int argc, char** argv)
 	}
 	check_all_ports_link_status(ports_mask);
 
+	pid = getpid();
+	RTE_LOG(INFO, APP, "========================\n");
+	RTE_LOG(INFO, APP, "KNI Running\n");
+	RTE_LOG(INFO, APP, "kill -SIGUSR1 %d\n", pid);
+	RTE_LOG(INFO, APP, "    Show KNI Statistics.\n");
+	RTE_LOG(INFO, APP, "kill -SIGUSR2 %d\n", pid);
+	RTE_LOG(INFO, APP, "    Zero KNI Statistics.\n");
+	RTE_LOG(INFO, APP, "========================\n");
+	fflush(stdout);
+
 	ret = rte_ctrl_thread_create(&kni_link_tid,
 				     "KNI link status check", NULL,
 				     monitor_all_ports_link_status, NULL);
-- 
2.19.0

Re: [dpdk-dev] [PATCH v6 4/5] examples/kni: add log msgs to show and clear stats

[Stephen Hemminger]

On Wed, 24 Oct 2018 17:27:18 -0300
Dan Gora <dg@adax.com> wrote:

> +	pid = getpid();
> +	RTE_LOG(INFO, APP, "========================\n");
> +	RTE_LOG(INFO, APP, "KNI Running\n");
> +	RTE_LOG(INFO, APP, "kill -SIGUSR1 %d\n", pid);
> +	RTE_LOG(INFO, APP, "    Show KNI Statistics.\n");
> +	RTE_LOG(INFO, APP, "kill -SIGUSR2 %d\n", pid);
> +	RTE_LOG(INFO, APP, "    Zero KNI Statistics.\n");
> +	RTE_LOG(INFO, APP, "========================\n");
> +	fflush(stdout);
> +

Thanks for updating the example to cover the new code.

This seems like an overly wordy message which should really be in the documentation
not a billboard in the code.

In my opinion, having verbose messages is unhelpful since it just clutters
the experience.

Re: [dpdk-dev] [PATCH v6 4/5] examples/kni: add log msgs to show and clear stats

[Dan Gora]

On Wed, Oct 24, 2018 at 5:46 PM Stephen Hemminger
<stephen@networkplumber.org> wrote:
> > +     pid = getpid();
> > +     RTE_LOG(INFO, APP, "========================\n");
> > +     RTE_LOG(INFO, APP, "KNI Running\n");
> > +     RTE_LOG(INFO, APP, "kill -SIGUSR1 %d\n", pid);
> > +     RTE_LOG(INFO, APP, "    Show KNI Statistics.\n");
> > +     RTE_LOG(INFO, APP, "kill -SIGUSR2 %d\n", pid);
> > +     RTE_LOG(INFO, APP, "    Zero KNI Statistics.\n");
> > +     RTE_LOG(INFO, APP, "========================\n");
> > +     fflush(stdout);
> > +
>
> Thanks for updating the example to cover the new code.
>
> This seems like an overly wordy message which should really be in the documentation
> not a billboard in the code.
>
> In my opinion, having verbose messages is unhelpful since it just clutters
> the experience.

Sigh...

This is version 6 of this patch.  You could have said something about
it at any point in the last two and a half months that I have been
struggling to get this merged.

These "features" were never documented at all, so you would have no
idea they existed unless you read the code.

The point of this patch is that you can just copy and paste the
commands directly from the screen.  This saves you from having to type
'ps -ef|grep kni', cut the PID, type 'kill -SIGUSR1', then paste the
PID.  How is that easier that what I have done?

And it's not a billboard, it's 7 lines.  Have you actually tried it?

The amount of nitpicking on these patches has been just incredible..
People get entire subsystems merged with 1/10th the hassle that I've
been given to add one stupid function.  It's extremely frustrating.
I've totally given up on trying to get my other KNI patches merged..
It's just not worth it..

dan

Re: [dpdk-dev] [PATCH v6 4/5] examples/kni: add log msgs to show and clear stats

[Stephen Hemminger]

On Wed, 24 Oct 2018 17:56:50 -0300
Dan Gora <dg@adax.com> wrote:

> On Wed, Oct 24, 2018 at 5:46 PM Stephen Hemminger
> <stephen@networkplumber.org> wrote:
> > > +     pid = getpid();
> > > +     RTE_LOG(INFO, APP, "========================\n");
> > > +     RTE_LOG(INFO, APP, "KNI Running\n");
> > > +     RTE_LOG(INFO, APP, "kill -SIGUSR1 %d\n", pid);
> > > +     RTE_LOG(INFO, APP, "    Show KNI Statistics.\n");
> > > +     RTE_LOG(INFO, APP, "kill -SIGUSR2 %d\n", pid);
> > > +     RTE_LOG(INFO, APP, "    Zero KNI Statistics.\n");
> > > +     RTE_LOG(INFO, APP, "========================\n");
> > > +     fflush(stdout);
> > > +  
> >
> > Thanks for updating the example to cover the new code.
> >
> > This seems like an overly wordy message which should really be in the documentation
> > not a billboard in the code.
> >
> > In my opinion, having verbose messages is unhelpful since it just clutters
> > the experience.  
> 
> Sigh...
> 
> This is version 6 of this patch.  You could have said something about
> it at any point in the last two and a half months that I have been
> struggling to get this merged.
> 
> These "features" were never documented at all, so you would have no
> idea they existed unless you read the code.
> 
> The point of this patch is that you can just copy and paste the
> commands directly from the screen.  This saves you from having to type
> 'ps -ef|grep kni', cut the PID, type 'kill -SIGUSR1', then paste the
> PID.  How is that easier that what I have done?
> 
> And it's not a billboard, it's 7 lines.  Have you actually tried it?
> 
> The amount of nitpicking on these patches has been just incredible..
> People get entire subsystems merged with 1/10th the hassle that I've
> been given to add one stupid function.  It's extremely frustrating.
> I've totally given up on trying to get my other KNI patches merged..
> It's just not worth it..
> 
> dan

I look at patches as they show up and don't want to overwhelm people
with a long laundry list of items. Just a case of call them as I see them.
Often a developer is focused on "does my feature work" and misses how
the new feature is not used by most people.

Remember when working on projects that the unstated policy is that all
code should look the same. Anything you introduce should look like everything
around it. Yes, this limits taste and individual freedom, but if you want to
change things then doing it in new code is not the way to do it.

The patch can go in as is. There is no reason for a message to block that.
Just trying to see what can be improved.

Don't get disheartened, 6 versions of a patch is nothing bad.
Sometimes it takes 20 or more until agreement occurs.

Re: [dpdk-dev] [PATCH v6 4/5] examples/kni: add log msgs to show and clear stats

[Dan Gora]

On Wed, Oct 24, 2018 at 6:18 PM Stephen Hemminger
<stephen@networkplumber.org> wrote:
> > >
> > > This seems like an overly wordy message which should really be in the documentation
> > > not a billboard in the code.
> > >
> > > In my opinion, having verbose messages is unhelpful since it just clutters
> > > the experience.
> >
> > Sigh...
> >
> > This is version 6 of this patch.  You could have said something about
> > it at any point in the last two and a half months that I have been
> > struggling to get this merged.
> >
> > These "features" were never documented at all, so you would have no
> > idea they existed unless you read the code.
> >
> > The point of this patch is that you can just copy and paste the
> > commands directly from the screen.  This saves you from having to type
> > 'ps -ef|grep kni', cut the PID, type 'kill -SIGUSR1', then paste the
> > PID.  How is that easier that what I have done?
> >
> > And it's not a billboard, it's 7 lines.  Have you actually tried it?
> >
> > The amount of nitpicking on these patches has been just incredible..
> > People get entire subsystems merged with 1/10th the hassle that I've
> > been given to add one stupid function.  It's extremely frustrating.
> > I've totally given up on trying to get my other KNI patches merged..
> > It's just not worth it..
> >
> > dan
>
> I look at patches as they show up and don't want to overwhelm people
> with a long laundry list of items. Just a case of call them as I see them.

I personally, and I would imagine that most people, would prefer to
get a long list of items to fix so that they can all be fixed at one
time rather than this endless churning on the mailing list and the
overhead of source control, formatting the patches, sending the
patches, etc...

> Often a developer is focused on "does my feature work" and misses how
> the new feature is not used by most people.
>
> Remember when working on projects that the unstated policy is that all
> code should look the same. Anything you introduce should look like everything
> around it. Yes, this limits taste and individual freedom, but if you want to
> change things then doing it in new code is not the way to do it.

By definition, to change things, you need new code.

In my experience 99.9999% of users do not read the documentation.
Engineers, and unix/linux engineers in particular, are notorious for
being terrible UI developers.  The KNI sample application has a
*terrible* user interface.  You run the application and.... nothing...
no indication of what happened, no indication of what you should do,
just some nonsensical EAL debug messages.

I was just trying to give the user some sense that 1: the thing is
actually running and doing something 2: there is something that you
can do to interact with the application and 3: save myself from some
extra typing.  I figured that since I hate any extra typing that most
other people would as well.

As far as it looking like everything around it; there was nothing
around it before, so there's nothing to compare it against... I
*could* have implemented a full menu thing like testpmd, but my
objective here was not to fix the KNI sample app, it was just to add
rte_kni_update_link().  That's all I really wanted...

> The patch can go in as is. There is no reason for a message to block that.
> Just trying to see what can be improved.
>
> Don't get disheartened, 6 versions of a patch is nothing bad.
> Sometimes it takes 20 or more until agreement occurs.

In the 6 months or so that I've been actively working on DPDK, I've
never seen one, especially to add one, basically trivial, function.
Two patches that caused compilation errors got merged in the last
couple of weeks.

I'll send a version 7 and add these commands to the documentation in
the other patch [3/5].. You guys can accept or reject this patch
[4/5]...

[dpdk-dev] [PATCH v6 5/5] examples/kni: improve zeroing statistics

[Dan Gora]

The worker threads incrementing the rx/tx_packets race with the signal
handler from the main thread zeroing the entire statistics structure.
This can cause the statistics to fail to be zeroed, even when there
is no traffic on those interfaces.

Improve zeroing the statistics by only incrementing rx/tx_packets
in worker threads by a non-zero amount.  This limits the race to the
periods in which traffic is actually being received or transmitted.

Signed-off-by: Dan Gora <dg@adax.com>
---
 examples/kni/main.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/examples/kni/main.c b/examples/kni/main.c
index 0e3b2a2f7..e37b1ad36 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -223,7 +223,8 @@ kni_ingress(struct kni_port_params *p)
 		}
 		/* Burst tx to kni */
 		num = rte_kni_tx_burst(p->kni[i], pkts_burst, nb_rx);
-		kni_stats[port_id].rx_packets += num;
+		if (num)
+			kni_stats[port_id].rx_packets += num;
 
 		rte_kni_handle_request(p->kni[i]);
 		if (unlikely(num < nb_rx)) {
@@ -260,7 +261,8 @@ kni_egress(struct kni_port_params *p)
 		}
 		/* Burst tx to eth */
 		nb_tx = rte_eth_tx_burst(port_id, 0, pkts_burst, (uint16_t)num);
-		kni_stats[port_id].tx_packets += nb_tx;
+		if (nb_tx)
+			kni_stats[port_id].tx_packets += nb_tx;
 		if (unlikely(nb_tx < num)) {
 			/* Free mbufs not tx to NIC */
 			kni_burst_free_mbufs(&pkts_burst[nb_tx], num - nb_tx);
-- 
2.19.0

[dpdk-dev] [PATCH v7 0/5] kni: add API to set link status on kernel interface

[Dan Gora]

Hi All,

Attached is version 7 of a patchset to add a new API function to
set the link status on kernel interfaces created with the KNI kernel
module.

v7
====
* Add commands to check/clear statistics in kni sample app in KNI sample
  app documentation.

> v6
> ====
> * Update info about callback functions in KNI library documentation.
> 
> * Minor fixup to module parameter formatting in KNI kernel module. (align
>   the "charp" text...)
> 
> * Add '-m' flag to KNI sample app command line in KNI sample app
>   documentation.
> 
> * Add references to KNI library documentation from KNI sample app
>   documentation (yay Sphinx!)
> 
> v5
> ====
> * Update Kernel NIC Interface document with new rte_kni option, move
>   kernel module parameter description here from KNI sample app
>   document, general cleanup.
> 
> * Update Kernel NIC Interface sample application (kni) document
>   with new command line parameter, remove documentation for rte_kni
>   kernel module but add reference to it, general cleanup.
> 
> * Change link status change sample rate for KNI example app to 500ms
>   from 100ms.
> 
> * Fix bug in parameter parsing in v4 in rte_kni kernel module.
> 
> * Fix output formatting of optional parameters for rte_kni with
>   modinfo.
> 
> * Fix permissions of optional parameters for rte_kni.
> 
> * Squash patch to add test for rte_kni_update_link into patch
>   introducing the API function.
> 
> v4
> ====
> * Rework rte_kni_update_link to only take linkup/linkdown as parameter,
>   return previous link state, and remove log messages.
> 
> * Update patch to set default carrier state to make default carrier
>   state configurable by passing the 'carrier=[on|off]' option to
>   the rte_kni kernel module.  This is necessary in order to allow
>   applications which use KNI as pure virtual interfaces without
>   corresponding physical ethernet port to use the interfaces without
>   having to set the carrier state to 'on' via rte_kni_update_link()
>   or by writing to /sys/devices/virtual/net/<ifaceX>/carrier.
>   Note that the default is 'off'.
> 
> * Add command line flag '-m' to examples/kni to continuously monitor
>   and update the KNI interface link status according to the link
>   status of the corresponding physical ethernet port.
> 
> 
> v3
> ====
> * Use separate function to test rte_kni_update_link() in 'test' app.
> 
> * Separate changes to 'test' app into separate patch to facilitate
>   possible merge with https://patches.dpdk.org/patch/44730/
> 
> * Remove changes to set KNI interfaces to 'up' in example/kni
> 
> v2
> ====
> 
> * Fix bug where "Fixed" and "AutoNeg" were transposed in the link
>   status log message.
> 
> * Add rte_kni_update_link() to rte_kni_version.map
> 
> * Add rte_kni_update_link() tests to kni_autotest
> 
> * Update examples/kni to continuously monitor link status and
>   update the corresponding kernel interface with
>   rte_kni_update_link().
> 
> * Minor improvements to examples/kni: Add log message showing how
>   to show/zero stats.  Improve zeroing statistics.
> 
> Note that checkpatches.sh compains about patch 1/5, but this appears
> to be a bug with check-symbol-change or something.  If I move the
> fragment of the patch modifying rte_kni_version.map to the bottom of
> the patch file, it doesn't complain any more...  I just don't really
> have time to investigate this right now.
  
thanks
dan

Dan Gora (5):
  kni: add API to set link status on kernel interface
  kni: set default carrier state of interface
  examples/kni: monitor and update link status continually
  examples/kni: add log msgs to show and clear stats
  examples/kni: improve zeroing statistics

 .../prog_guide/kernel_nic_interface.rst       | 239 +++++++++++++---
 .../sample_app_ug/kernel_nic_interface.rst    | 262 +++++++++++-------
 examples/kni/Makefile                         |   2 +
 examples/kni/main.c                           |  92 +++++-
 kernel/linux/kni/kni_dev.h                    |   3 +
 kernel/linux/kni/kni_misc.c                   |  60 +++-
 kernel/linux/kni/kni_net.c                    |   5 +
 lib/librte_kni/rte_kni.c                      |  41 +++
 lib/librte_kni/rte_kni.h                      |  20 ++
 lib/librte_kni/rte_kni_version.map            |   6 +
 test/test/test_kni.c                          |  77 +++++
 11 files changed, 652 insertions(+), 155 deletions(-)

-- 
2.19.0

[dpdk-dev] [PATCH v7 1/5] kni: add API to set link status on kernel interface

[Dan Gora]

Add a new API function to KNI, rte_kni_update_link() to allow DPDK
applications to update the link status for KNI network interfaces in
the linux kernel.

Signed-off-by: Dan Gora <dg@adax.com>
---
 lib/librte_kni/rte_kni.c           | 41 ++++++++++++++++
 lib/librte_kni/rte_kni.h           | 20 ++++++++
 lib/librte_kni/rte_kni_version.map |  6 +++
 test/test/test_kni.c               | 77 ++++++++++++++++++++++++++++++
 4 files changed, 144 insertions(+)

diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c
index b8edd40f4..c9726d4f8 100644
--- a/lib/librte_kni/rte_kni.c
+++ b/lib/librte_kni/rte_kni.c
@@ -717,6 +717,47 @@ rte_kni_unregister_handlers(struct rte_kni *kni)
 
 	return 0;
 }
+
+int __rte_experimental
+rte_kni_update_link(struct rte_kni *kni, unsigned int linkup)
+{
+	char path[64];
+	char old_carrier[2];
+	const char *new_carrier;
+	int old_linkup;
+	int fd, ret;
+
+	if (kni == NULL)
+		return -1;
+
+	snprintf(path, sizeof(path), "/sys/devices/virtual/net/%s/carrier",
+		kni->name);
+
+	fd = open(path, O_RDWR);
+	if (fd == -1) {
+		RTE_LOG(ERR, KNI, "Failed to open file: %s.\n", path);
+		return -1;
+	}
+
+	ret = read(fd, old_carrier, 2);
+	if (ret < 1) {
+		close(fd);
+		return -1;
+	}
+	old_linkup = (old_carrier[0] == '1');
+
+	new_carrier = linkup ? "1" : "0";
+	ret = write(fd, new_carrier, 1);
+	if (ret < 1) {
+		RTE_LOG(ERR, KNI, "Failed to write file: %s.\n", path);
+		close(fd);
+		return -1;
+	}
+
+	close(fd);
+	return old_linkup;
+}
+
 void
 rte_kni_close(void)
 {
diff --git a/lib/librte_kni/rte_kni.h b/lib/librte_kni/rte_kni.h
index 601abdfc6..02ca43b4b 100644
--- a/lib/librte_kni/rte_kni.h
+++ b/lib/librte_kni/rte_kni.h
@@ -232,6 +232,26 @@ int rte_kni_register_handlers(struct rte_kni *kni, struct rte_kni_ops *ops);
  */
 int rte_kni_unregister_handlers(struct rte_kni *kni);
 
+/**
+ * Update link carrier state for KNI port.
+ *
+ * Update the linkup/linkdown state of a KNI interface in the kernel.
+ *
+ * @param kni
+ *  pointer to struct rte_kni.
+ * @param linkup
+ *  New link state:
+ *  0 for linkdown.
+ *  > 0 for linkup.
+ *
+ * @return
+ *  On failure: -1
+ *  Previous link state == linkdown: 0
+ *  Previous link state == linkup: 1
+ */
+int __rte_experimental
+rte_kni_update_link(struct rte_kni *kni, unsigned int linkup);
+
 /**
  *  Close KNI device.
  */
diff --git a/lib/librte_kni/rte_kni_version.map b/lib/librte_kni/rte_kni_version.map
index acd515eb0..c877dc6aa 100644
--- a/lib/librte_kni/rte_kni_version.map
+++ b/lib/librte_kni/rte_kni_version.map
@@ -15,3 +15,9 @@ DPDK_2.0 {
 
 	local: *;
 };
+
+EXPERIMENTAL {
+	global:
+
+	rte_kni_update_link;
+};
diff --git a/test/test/test_kni.c b/test/test/test_kni.c
index cf64c0b27..e38206905 100644
--- a/test/test/test_kni.c
+++ b/test/test/test_kni.c
@@ -118,6 +118,79 @@ kni_change_mtu(uint16_t port_id, unsigned int new_mtu)
 					 port_id, kni_pkt_mtu);
 	return 0;
 }
+
+static int
+test_kni_link_change(void)
+{
+	int ret;
+	int pid;
+
+	pid = fork();
+	if (pid < 0) {
+		printf("Error: Failed to fork a process\n");
+		return -1;
+	}
+
+	if (pid == 0) {
+		printf("Starting KNI Link status change tests.\n");
+		if (system(IFCONFIG TEST_KNI_PORT" up") == -1) {
+			ret = -1;
+			goto error;
+		}
+
+		ret = rte_kni_update_link(test_kni_ctx, 1);
+		if (ret < 0) {
+			printf("Failed to change link state to Up ret=%d.\n",
+				ret);
+			goto error;
+		}
+		rte_delay_ms(1000);
+		printf("KNI: Set LINKUP, previous state=%d\n", ret);
+
+		ret = rte_kni_update_link(test_kni_ctx, 0);
+		if (ret != 1) {
+			printf(
+		"Failed! Previous link state should be 1, returned %d.\n",
+				ret);
+			goto error;
+		}
+		rte_delay_ms(1000);
+		printf("KNI: Set LINKDOWN, previous state=%d\n", ret);
+
+		ret = rte_kni_update_link(test_kni_ctx, 1);
+		if (ret != 0) {
+			printf(
+		"Failed! Previous link state should be 0, returned %d.\n",
+				ret);
+			goto error;
+		}
+		printf("KNI: Set LINKUP, previous state=%d\n", ret);
+
+		ret = 0;
+		rte_delay_ms(1000);
+
+error:
+		if (system(IFCONFIG TEST_KNI_PORT" down") == -1)
+			ret = -1;
+
+		printf("KNI: Link status change tests: %s.\n",
+			(ret == 0) ? "Passed" : "Failed");
+		exit(ret);
+	} else {
+		int p_ret, status;
+
+		while (1) {
+			p_ret = waitpid(pid, &status, WNOHANG);
+			if (p_ret != 0) {
+				if (WIFEXITED(status))
+					return WEXITSTATUS(status);
+				return -1;
+			}
+			rte_delay_ms(10);
+			rte_kni_handle_request(test_kni_ctx);
+		}
+	}
+}
 /**
  * This loop fully tests the basic functions of KNI. e.g. transmitting,
  * receiving to, from kernel space, and kernel requests.
@@ -401,6 +474,10 @@ test_kni_processing(uint16_t port_id, struct rte_mempool *mp)
 		goto fail_kni;
 	}
 
+	ret = test_kni_link_change();
+	if (ret != 0)
+		goto fail_kni;
+
 	rte_eal_mp_remote_launch(test_kni_loop, NULL, CALL_MASTER);
 	RTE_LCORE_FOREACH_SLAVE(i) {
 		if (rte_eal_wait_lcore(i) < 0) {
-- 
2.19.0

[dpdk-dev] [PATCH v7 2/5] kni: set default carrier state of interface

[Dan Gora]

Add module parameter 'carrier='on|off' to set the default carrier state
for linux network interfaces created by the KNI module.  The default
carrier state is 'off'.

For KNI interfaces which need to reflect the carrier state of
a physical Ethernet port controlled by the DPDK application, the
default carrier state should be left set to 'off'.  The application
can set the carrier state of the KNI interface to reflect the state
of the physical Ethernet port using rte_kni_update_link().

For KNI interfaces which are purely virtual, the default carrier
state can be set to 'on'.  This enables the KNI interface to be
used without having to explicity set the carrier state to 'on'
using rte_kni_update_link().

Signed-off-by: Dan Gora <dg@adax.com>
---
 .../prog_guide/kernel_nic_interface.rst       | 239 ++++++++++++++----
 kernel/linux/kni/kni_dev.h                    |   3 +
 kernel/linux/kni/kni_misc.c                   |  60 ++++-
 kernel/linux/kni/kni_net.c                    |   5 +
 4 files changed, 254 insertions(+), 53 deletions(-)

diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst
index 8fa13fa1a..33ea980e5 100644
--- a/doc/guides/prog_guide/kernel_nic_interface.rst
+++ b/doc/guides/prog_guide/kernel_nic_interface.rst
@@ -29,58 +29,222 @@ The components of an application using the DPDK Kernel NIC Interface are shown i
 The DPDK KNI Kernel Module
 --------------------------
 
-The KNI kernel loadable module provides support for two types of devices:
+The KNI kernel loadable module ``rte_kni`` provides the kernel interface
+for DPDK applications.
 
-*   A Miscellaneous device (/dev/kni) that:
+When the ``rte_kni`` module is loaded, it will create a device ``/dev/kni``
+that is used by the DPDK KNI API functions to control and communicate with
+the kernel module.
 
-    *   Creates net devices (via ioctl  calls).
+The ``rte_kni`` kernel module contains several optional parameters which
+can be specified when the module is loaded to control its behavior:
 
-    *   Maintains a kernel thread context shared by all KNI instances
-        (simulating the RX side of the net driver).
+.. code-block:: console
 
-    *   For single kernel thread mode, maintains a kernel thread context shared by all KNI instances
-        (simulating the RX side of the net driver).
+    # modinfo rte_kni.ko
+    <snip>
+    parm:           lo_mode: KNI loopback mode (default=lo_mode_none):
+                    lo_mode_none        Kernel loopback disabled
+                    lo_mode_fifo        Enable kernel loopback with fifo
+                    lo_mode_fifo_skb    Enable kernel loopback with fifo and skb buffer
+                     (charp)
+    parm:           kthread_mode: Kernel thread mode (default=single):
+                    single    Single kernel thread mode enabled.
+                    multiple  Multiple kernel thread mode enabled.
+                     (charp)
+    parm:           carrier: Default carrier state for KNI interface (default=off):
+                    off   Interfaces will be created with carrier state set to off.
+                    on    Interfaces will be created with carrier state set to on.
+                     (charp)
 
-    *   For multiple kernel thread mode, maintains a kernel thread context for each KNI instance
-        (simulating the RX side of the net driver).
+Loading the ``rte_kni`` kernel module without any optional parameters is
+the typical way a DPDK application gets packets into and out of the kernel
+network stack.  Without any parameters, only one kernel thread is created
+for all KNI devices for packet receiving in kernel side, loopback mode is
+disabled, and the default carrier state of KNI interfaces is set to *off*.
 
-*   Net device:
+.. code-block:: console
 
-    *   Net functionality provided by implementing several operations such as netdev_ops,
-        header_ops, ethtool_ops that are defined by struct net_device,
-        including support for DPDK mbufs and FIFOs.
+    # insmod kmod/rte_kni.ko
 
-    *   The interface name is provided from userspace.
+.. _kni_loopback_mode:
 
-    *   The MAC address can be the real NIC MAC address or random.
+Loopback Mode
+~~~~~~~~~~~~~
+
+For testing, the ``rte_kni`` kernel module can be loaded in loopback mode
+by specifying the ``lo_mode`` parameter:
+
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko lo_mode=lo_mode_fifo
+
+The ``lo_mode_fifo`` loopback option will loop back ring enqueue/dequeue
+operations in kernel space.
+
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko lo_mode=lo_mode_fifo_skb
+
+The ``lo_mode_fifo_skb`` loopback option will loop back ring enqueue/dequeue
+operations and sk buffer copies in kernel space.
+
+If the ``lo_mode`` parameter is not specified, loopback mode is disabled.
+
+.. _kni_kernel_thread_mode:
+
+Kernel Thread Mode
+~~~~~~~~~~~~~~~~~~
+
+To provide flexibility of performance, the ``rte_kni`` KNI kernel module
+can be loaded with the ``kthread_mode`` parameter.  The ``rte_kni`` kernel
+module supports two options: "single kernel thread" mode and "multiple
+kernel thread" mode.
+
+Single kernel thread mode is enabled as follows:
+
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko kthread_mode=single
+
+This mode will create only one kernel thread for all KNI interfaces to
+receive data on the kernel side.  By default, this kernel thread is not
+bound to any particular core, but the user can set the core affinity for
+this kernel thread by setting the ``core_id`` and ``force_bind`` parameters
+in ``struct rte_kni_conf`` when the first KNI interface is created:
+
+For optimum performance, the kernel thread should be bound to a core in
+on the same socket as the DPDK lcores used in the application.
+
+The KNI kernel module can also be configured to start a separate kernel
+thread for each KNI interface created by the DPDK application.  Multiple
+kernel thread mode is enabled as follows:
+
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko kthread_mode=multiple
+
+This mode will create a separate kernel thread for each KNI interface to
+receive data on the kernel side.  The core affinity of each ``kni_thread``
+kernel thread can be specified by setting the ``core_id`` and ``force_bind``
+parameters in ``struct rte_kni_conf`` when each KNI interface is created.
+
+Multiple kernel thread mode can provide scalable higher performance if
+sufficient unused cores are available on the host system.
+
+If the ``kthread_mode`` parameter is not specified, the "single kernel
+thread" mode is used.
+
+.. _kni_default_carrier_state:
+
+Default Carrier State
+~~~~~~~~~~~~~~~~~~~~~
+
+The default carrier state of KNI interfaces created by the ``rte_kni``
+kernel module is controlled via the ``carrier`` option when the module
+is loaded.
+
+If ``carrier=off`` is specified, the kernel module will leave the carrier
+state of the interface *down* when the interface is management enabled.
+The DPDK application can set the carrier state of the KNI interface using the
+``rte_kni_update_link()`` function.  This is useful for DPDK applications
+which require that the carrier state of the KNI interface reflect the
+actual link state of the corresponding physical NIC port.
+
+If ``carrier=on`` is specified, the kernel module will automatically set
+the carrier state of the interface to *up* when the interface is management
+enabled.  This is useful for DPDK applications which use the KNI interface as
+a purely virtual interface that does not correspond to any physical hardware
+and do not wish to explicitly set the carrier state of the interface with
+``rte_kni_update_link()``.  It is also useful for testing in loopback mode
+where the NIC port may not be physically connected to anything.
+
+To set the default carrier state to *on*:
+
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko carrier=on
+
+To set the default carrier state to *off*:
+
+.. code-block:: console
+
+    # insmod kmod/rte_kni.ko carrier=off
+
+If the ``carrier`` parameter is not specified, the default carrier state
+of KNI interfaces will be set to *off*.
 
 KNI Creation and Deletion
 -------------------------
 
-The KNI interfaces are created by a DPDK application dynamically.
-The interface name and FIFO details are provided by the application through an ioctl call
-using the rte_kni_device_info struct which contains:
+Before any KNI interfaces can be created, the ``rte_kni`` kernel module must
+be loaded into the kernel and configured withe ``rte_kni_init()`` function.
+
+The KNI interfaces are created by a DPDK application dynamically via the
+``rte_kni_alloc()`` function.
+
+The ``struct rte_kni_conf`` structure contains fields which allow the
+user to specify the interface name, set the MTU size, set an explicit or
+random MAC address and control the affinity of the kernel Rx thread(s)
+(both single and multi-threaded modes).
+
+The ``struct rte_kni_ops`` structure contains pointers to functions to
+handle requests from the ``rte_kni`` kernel module.  These functions
+allow DPDK applications to perform actions when the KNI interfaces are
+manipulated by control commands or functions external to the application.
+
+For example, the DPDK application may wish to enabled/disable a physical
+NIC port when a user enabled/disables a KNI interface with ``ip link set
+[up|down] dev <ifaceX>``.  The DPDK application can register a callback for
+``config_network_if`` which will be called when the interface management
+state changes.
+
+There are currently four callbacks for which the user can register
+application functions:
 
-*   The interface name.
+``config_network_if``:
 
-*   Physical addresses of the corresponding memzones for the relevant FIFOs.
+    Called when the management state of the KNI interface changes.
+    For example, when the user runs ``ip link set [up|down] dev <ifaceX>``.
 
-*   Mbuf mempool details, both physical and virtual (to calculate the offset for mbuf pointers).
+``change_mtu``:
 
-*   PCI information.
+    Called when the user changes the MTU size of the KNI
+    interface.  For example, when the user runs ``ip link set mtu <size>
+    dev <ifaceX>``.
 
-*   Core affinity.
+``config_mac_address``:
 
-Refer to rte_kni_common.h in the DPDK source code for more details.
+    Called when the user changes the MAC address of the KNI interface.
+    For example, when the user runs ``ip link set address <MAC>
+    dev <ifaceX>``.  If the user sets this callback function to NULL,
+    but sets the ``port_id`` field to a value other than -1, a default
+    callback handler in the rte_kni library ``kni_config_mac_address()``
+    will be called which calls ``rte_eth_dev_default_mac_addr_set()``
+    on the specified ``port_id``.
 
-The physical addresses will be re-mapped into the kernel address space and stored in separate KNI contexts.
+``config_promiscusity``:
 
-The affinity of kernel RX thread (both single and multi-threaded modes) is controlled by force_bind and
-core_id config parameters.
+    Called when the user changes the promiscusity state of the KNI
+    interface.  For example, when the user runs ``ip link set promisc
+    [on|off] dev <ifaceX>``. If the user sets this callback function to
+    NULL, but sets the ``port_id`` field to a value other than -1, a default
+    callback handler in the rte_kni library ``kni_config_promiscusity()``
+    will be called which calls ``rte_eth_promiscuous_<enable|disable>()``
+    on the specified ``port_id``.
 
-The KNI interfaces can be deleted by a DPDK application dynamically after being created.
-Furthermore, all those KNI interfaces not deleted will be deleted on the release operation
-of the miscellaneous device (when the DPDK application is closed).
+In order to run these callbacks, the application must periodically call
+the ``rte_kni_handle_request()`` function.  Any user callback function
+registered will be called directly from ``rte_kni_handle_request()`` so
+care must be taken to prevent deadlock and to not block any DPDK fastpath
+tasks.  Typically DPDK applications which use these callbacks will need
+to create a separate thread or secondary process to periodically call
+``rte_kni_handle_request()``.
+
+The KNI interfaces can be deleted by a DPDK application with
+``rte_kni_release()``.  All KNI interfaces not explicitly deleted will be
+deleted when the the ``/dev/kni`` device is closed, either explicitly with
+``rte_kni_close()`` or when the DPDK application is closed.
 
 DPDK mbuf Flow
 --------------
@@ -118,7 +282,7 @@ The packet is received from the Linux net stack, by calling the kni_net_tx() cal
 The mbuf is dequeued (without waiting due the cache) and filled with data from sk_buff.
 The sk_buff is then freed and the mbuf sent in the tx_q FIFO.
 
-The DPDK TX thread dequeues the mbuf and sends it to the PMD (via rte_eth_tx_burst()).
+The DPDK TX thread dequeues the mbuf and sends it to the PMD via ``rte_eth_tx_burst()``.
 It then puts the mbuf back in the cache.
 
 Ethtool
@@ -128,16 +292,3 @@ Ethtool is a Linux-specific tool with corresponding support in the kernel
 where each net device must register its own callbacks for the supported operations.
 The current implementation uses the igb/ixgbe modified Linux drivers for ethtool support.
 Ethtool is not supported in i40e and VMs (VF or EM devices).
-
-Link state and MTU change
--------------------------
-
-Link state and MTU change are network interface specific operations usually done via ifconfig.
-The request is initiated from the kernel side (in the context of the ifconfig process)
-and handled by the user space DPDK application.
-The application polls the request, calls the application handler and returns the response back into the kernel space.
-
-The application handlers can be registered upon interface creation or explicitly registered/unregistered in runtime.
-This provides flexibility in multiprocess scenarios
-(where the KNI is created in the primary process but the callbacks are handled in the secondary one).
-The constraint is that a single process can register and handle the requests.
diff --git a/kernel/linux/kni/kni_dev.h b/kernel/linux/kni/kni_dev.h
index 6275ef27f..688f574a4 100644
--- a/kernel/linux/kni/kni_dev.h
+++ b/kernel/linux/kni/kni_dev.h
@@ -29,6 +29,9 @@
 
 #define MBUF_BURST_SZ 32
 
+/* Default carrier state for created KNI network interfaces */
+extern uint32_t dflt_carrier;
+
 /**
  * A structure describing the private information for a kni device.
  */
diff --git a/kernel/linux/kni/kni_misc.c b/kernel/linux/kni/kni_misc.c
index fa69f8e63..522ae23b9 100644
--- a/kernel/linux/kni/kni_misc.c
+++ b/kernel/linux/kni/kni_misc.c
@@ -39,6 +39,10 @@ static char *lo_mode;
 static char *kthread_mode;
 static uint32_t multiple_kthread_on;
 
+/* Default carrier state for created KNI network interfaces */
+static char *carrier;
+uint32_t dflt_carrier;
+
 #define KNI_DEV_IN_USE_BIT_NUM 0 /* Bit number for device in use */
 
 static int kni_net_id;
@@ -466,6 +470,8 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_num,
 		return -ENODEV;
 	}
 
+	netif_carrier_off(net_dev);
+
 	ret = kni_run_thread(knet, kni, dev_info.force_bind);
 	if (ret != 0)
 		return ret;
@@ -590,6 +596,24 @@ kni_parse_kthread_mode(void)
 	return 0;
 }
 
+static int __init
+kni_parse_carrier_state(void)
+{
+	if (!carrier) {
+		dflt_carrier = 0;
+		return 0;
+	}
+
+	if (strcmp(carrier, "off") == 0)
+		dflt_carrier = 0;
+	else if (strcmp(carrier, "on") == 0)
+		dflt_carrier = 1;
+	else
+		return -1;
+
+	return 0;
+}
+
 static int __init
 kni_init(void)
 {
@@ -605,6 +629,16 @@ kni_init(void)
 	else
 		pr_debug("Multiple kernel thread mode enabled\n");
 
+	if (kni_parse_carrier_state() < 0) {
+		pr_err("Invalid parameter for carrier\n");
+		return -EINVAL;
+	}
+
+	if (dflt_carrier == 0)
+		pr_debug("Default carrier state set to off.\n");
+	else
+		pr_debug("Default carrier state set to on.\n");
+
 #ifdef HAVE_SIMPLIFIED_PERNET_OPERATIONS
 	rc = register_pernet_subsys(&kni_net_ops);
 #else
@@ -647,19 +681,27 @@ kni_exit(void)
 module_init(kni_init);
 module_exit(kni_exit);
 
-module_param(lo_mode, charp, S_IRUGO | S_IWUSR);
+module_param(lo_mode, charp, 0644);
 MODULE_PARM_DESC(lo_mode,
 "KNI loopback mode (default=lo_mode_none):\n"
-"    lo_mode_none        Kernel loopback disabled\n"
-"    lo_mode_fifo        Enable kernel loopback with fifo\n"
-"    lo_mode_fifo_skb    Enable kernel loopback with fifo and skb buffer\n"
-"\n"
+"\t\tlo_mode_none        Kernel loopback disabled\n"
+"\t\tlo_mode_fifo        Enable kernel loopback with fifo\n"
+"\t\tlo_mode_fifo_skb    Enable kernel loopback with fifo and skb buffer\n"
+"\t\t"
 );
 
-module_param(kthread_mode, charp, S_IRUGO);
+module_param(kthread_mode, charp, 0644);
 MODULE_PARM_DESC(kthread_mode,
 "Kernel thread mode (default=single):\n"
-"    single    Single kernel thread mode enabled.\n"
-"    multiple  Multiple kernel thread mode enabled.\n"
-"\n"
+"\t\tsingle    Single kernel thread mode enabled.\n"
+"\t\tmultiple  Multiple kernel thread mode enabled.\n"
+"\t\t"
+);
+
+module_param(carrier, charp, 0644);
+MODULE_PARM_DESC(carrier,
+"Default carrier state for KNI interface (default=off):\n"
+"\t\toff   Interfaces will be created with carrier state set to off.\n"
+"\t\ton    Interfaces will be created with carrier state set to on.\n"
+"\t\t"
 );
diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c
index 3a542d737..7371b6d58 100644
--- a/kernel/linux/kni/kni_net.c
+++ b/kernel/linux/kni/kni_net.c
@@ -133,6 +133,10 @@ kni_net_open(struct net_device *dev)
 	struct kni_dev *kni = netdev_priv(dev);
 
 	netif_start_queue(dev);
+	if (dflt_carrier == 1)
+		netif_carrier_on(dev);
+	else
+		netif_carrier_off(dev);
 
 	memset(&req, 0, sizeof(req));
 	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
@@ -152,6 +156,7 @@ kni_net_release(struct net_device *dev)
 	struct kni_dev *kni = netdev_priv(dev);
 
 	netif_stop_queue(dev); /* can't transmit any more */
+	netif_carrier_off(dev);
 
 	memset(&req, 0, sizeof(req));
 	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
-- 
2.19.0

[dpdk-dev] [PATCH v7 3/5] examples/kni: monitor and update link status continually

[Dan Gora]

Update KNI example to add the command line flag '-m' to enable
a function to continuously monitor the Ethernet link status of
the physical link and update the link status of the corresponding
interfaces with rte_kni_update_link().

Signed-off-by: Dan Gora <dg@adax.com>
---
 .../sample_app_ug/kernel_nic_interface.rst    | 262 +++++++++++-------
 examples/kni/Makefile                         |   2 +
 examples/kni/main.c                           |  70 ++++-
 3 files changed, 237 insertions(+), 97 deletions(-)

diff --git a/doc/guides/sample_app_ug/kernel_nic_interface.rst b/doc/guides/sample_app_ug/kernel_nic_interface.rst
index 1b3ee9a5e..6acdf0fff 100644
--- a/doc/guides/sample_app_ug/kernel_nic_interface.rst
+++ b/doc/guides/sample_app_ug/kernel_nic_interface.rst
@@ -31,18 +31,27 @@ This is done by creating one or more kernel net devices for each of the DPDK por
 The application allows the use of standard Linux tools (ethtool, ifconfig, tcpdump) with the DPDK ports and
 also the exchange of packets between the DPDK application and the Linux* kernel.
 
+The Kernel NIC Interface sample application requires that the
+KNI kernel module ``rte_kni`` be loaded into the kernel.  See
+:doc:`../prog_guide/kernel_nic_interface` for more information on loading
+the ``rte_kni`` kernel module.
+
 Overview
 --------
 
-The Kernel NIC Interface sample application uses two threads in user space for each physical NIC port being used,
-and allocates one or more KNI device for each physical NIC port with kernel module's support.
-For a physical NIC port, one thread reads from the port and writes to KNI devices,
-and another thread reads from KNI devices and writes the data unmodified to the physical NIC port.
-It is recommended to configure one KNI device for each physical NIC port.
-If configured with more than one KNI devices for a physical NIC port,
-it is just for performance testing, or it can work together with VMDq support in future.
+The Kernel NIC Interface sample application ``kni`` allocates one or more
+KNI interfaces for each physical NIC port.  For each physical NIC port,
+``kni`` uses two DPDK threads in user space; one thread reads from the port and
+writes to the corresponding KNI interfaces and the other thread reads from
+the KNI interfaces and writes the data unmodified to the physical NIC port.
+
+It is recommended to configure one KNI interface for each physical NIC port.
+The application can be configured with more than one KNI interface for
+each physical NIC port for performance testing or it can work together with
+VMDq support in future.
 
-The packet flow through the Kernel NIC Interface application is as shown in the following figure.
+The packet flow through the Kernel NIC Interface application is as shown
+in the following figure.
 
 .. _figure_kernel_nic:
 
@@ -50,145 +59,221 @@ The packet flow through the Kernel NIC Interface application is as shown in the
 
    Kernel NIC Application Packet Flow
 
+If link monitoring is enabled with the ``-m`` command line flag, one
+additional pthread is launched which will check the link status of each
+physical NIC port and will update the carrier status of the corresponding
+KNI interface(s) to match the physical NIC port's state.  This means that
+the KNI interface(s) will be disabled automatically when the Ethernet link
+goes down and enabled when the Ethernet link goes up.
+
+If link monitoring is enabled, the ``rte_kni`` kernel module should be loaded
+such that the :ref:`default carrier state <kni_default_carrier_state>` is
+set to *off*.  This ensures that the KNI interface is only enabled *after*
+the Ethernet link of the corresponding NIC port has reached the linkup state.
+
+If link monitoring is not enabled, the ``rte_kni`` kernel module should be
+loaded with the :ref:`default carrier state <kni_default_carrier_state>`
+set to *on*.  This sets the carrier state of the KNI interfaces to *on*
+when the KNI interfaces are enabled without regard to the actual link state
+of the corresponding NIC port.  This is useful for testing in loopback
+mode where the NIC port may not be physically connected to anything.
+
 Compiling the Application
 -------------------------
 
 To compile the sample application see :doc:`compiling`.
 
-The application is located in the ``kni`` sub-directory.
+The application is located in the ``examples/kni`` sub-directory.
 
 .. note::
 
         This application is intended as a linuxapp only.
 
-Loading the Kernel Module
--------------------------
+Running the kni Example Application
+-----------------------------------
 
-Loading the KNI kernel module without any parameter is the typical way a DPDK application
-gets packets into and out of the kernel net stack.
-This way, only one kernel thread is created for all KNI devices for packet receiving in kernel side:
+The ``kni`` example application requires a number of command line options:
 
 .. code-block:: console
 
-    #insmod rte_kni.ko
+    kni [EAL options] -- -p PORTMASK --config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,(port,lcore_rx,lcore_tx[,lcore_kthread,...])]" [-P] [-m]
 
-Pinning the kernel thread to a specific core can be done using a taskset command such as following:
+Where:
 
-.. code-block:: console
+*   ``-p PORTMASK``:
 
-    #taskset -p 100000 `pgrep --fl kni_thread | awk '{print $1}'`
+    Hexadecimal bitmask of ports to configure.
 
-This command line tries to pin the specific kni_thread on the 20th lcore (lcore numbering starts at 0),
-which means it needs to check if that lcore is available on the board.
-This command must be sent after the application has been launched, as insmod does not start the kni thread.
+*   ``--config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,(port,lcore_rx,lcore_tx[,lcore_kthread,...])]"``:
 
-For optimum performance,
-the lcore in the mask must be selected to be on the same socket as the lcores used in the KNI application.
+    Determines which lcores the Rx and Tx DPDK tasks, and (optionally)
+    the KNI kernel thread(s) are bound to for each physical port.
 
-To provide flexibility of performance, the kernel module of the KNI,
-located in the kmod sub-directory of the DPDK target directory,
-can be loaded with parameter of kthread_mode as follows:
+*   ``-P``:
 
-*   #insmod rte_kni.ko kthread_mode=single
+    Optional flag to set all ports to promiscuous mode so that packets are
+    accepted regardless of the packet's Ethernet MAC destination address.
+    Without this option, only packets with the Ethernet MAC destination
+    address set to the Ethernet address of the port are accepted.
 
-    This mode will create only one kernel thread for all KNI devices for packet receiving in kernel side.
-    By default, it is in this single kernel thread mode.
-    It can set core affinity for this kernel thread by using Linux command taskset.
+*   ``-m``:
 
-*   #insmod rte_kni.ko kthread_mode =multiple
+    Optional flag to enable monitoring and updating of the Ethernet
+    carrier state.  With this option set, a thread will be started which
+    will periodically check the Ethernet link status of the physical
+    Ethernet ports and set the carrier state of the corresponding KNI
+    network interface to match it.  This means that the KNI interface will
+    be disabled automatically when the Ethernet link goes down and enabled
+    when the Ethernet link goes up.
 
-    This mode will create a kernel thread for each KNI device for packet receiving in kernel side.
-    The core affinity of each kernel thread is set when creating the KNI device.
-    The lcore ID for each kernel thread is provided in the command line of launching the application.
-    Multiple kernel thread mode can provide scalable higher performance.
+Refer to *DPDK Getting Started Guide* for general information on running
+applications and the Environment Abstraction Layer (EAL) options.
 
-To measure the throughput in a loopback mode, the kernel module of the KNI,
-located in the kmod sub-directory of the DPDK target directory,
-can be loaded with parameters as follows:
+The ``-c coremask`` or ``-l corelist`` parameter of the EAL options must
+include the lcores specified by ``lcore_rx`` and ``lcore_tx`` for each port,
+but does not need to include lcores specified by ``lcore_kthread`` as those
+cores are used to pin the kernel threads in the ``rte_kni`` kernel module.
 
-*   #insmod rte_kni.ko lo_mode=lo_mode_fifo
+The ``--config`` parameter must include a set of
+``(port,lcore_rx,lcore_tx,[lcore_kthread,...])`` values for each physical
+port specified in the ``-p PORTMASK`` parameter.
 
-    This loopback mode will involve ring enqueue/dequeue operations in kernel space.
+The optional ``lcore_kthread`` lcore ID parameter in ``--config`` can be
+specified zero, one or more times for each physical port.
 
-*   #insmod rte_kni.ko lo_mode=lo_mode_fifo_skb
+If no lcore ID is specified for ``lcore_kthread``, one KNI interface will
+be created for the physical port ``port`` and the KNI kernel thread(s)
+will have no specific core affinity.
 
-    This loopback mode will involve ring enqueue/dequeue operations and sk buffer copies in kernel space.
+If one or more lcore IDs are specified for ``lcore_kthread``, a KNI interface
+will be created for each lcore ID specified, bound to the physical port
+``port``.  If the ``rte_kni`` kernel module is loaded in :ref:`multiple
+kernel thread <kni_kernel_thread_mode>` mode, a kernel thread will be created
+for each KNI interface and bound to the specified core.  If the ``rte_kni``
+kernel module is loaded in :ref:`single kernel thread <kni_kernel_thread_mode>`
+mode, only one kernel thread is started for all KNI interfaces.  The kernel
+thread will be bound to the first ``lcore_kthread`` lcore ID specified.
 
-Running the Application
------------------------
+Example Configurations
+~~~~~~~~~~~~~~~~~~~~~~~
 
-The application requires a number of command line options:
+The following commands will first load the ``rte_kni`` kernel module in
+:ref:`multiple kernel thread <kni_kernel_thread_mode>` mode.  The ``kni``
+application is then started using two ports;  Port 0 uses lcore 4 for the
+Rx task, lcore 6 for the Tx task, and will create a single KNI interface
+``vEth0_0`` with the kernel thread bound to lcore 8.  Port 1 uses lcore
+5 for the Rx task, lcore 7 for the Tx task, and will create a single KNI
+interface ``vEth1_0`` with the kernel thread bound to lcore 9.
 
 .. code-block:: console
 
-    kni [EAL options] -- -P -p PORTMASK --config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,port,lcore_rx,lcore_tx[,lcore_kthread,...]]"
-
-Where:
+    # rmmod rte_kni
+    # insmod kmod/rte_kni.ko kthread_mode=multiple
+    # ./build/kni -l 4-7 -n 4 -- -P -p 0x3 -m --config="(0,4,6,8),(1,5,7,9)"
 
-*   -P: Set all ports to promiscuous mode so that packets are accepted regardless of the packet's Ethernet MAC destination address.
-    Without this option, only packets with the Ethernet MAC destination address set to the Ethernet address of the port are accepted.
+The following example is identical, except an additional ``lcore_kthread``
+core is specified per physical port.  In this case, ``kni`` will create
+four KNI interfaces: ``vEth0_0``/``vEth0_1`` bound to physical port 0 and
+``vEth1_0``/``vEth1_1`` bound to physical port 1.
 
-*   -p PORTMASK: Hexadecimal bitmask of ports to configure.
+The kernel thread for each interface will be bound as follows:
 
-*   --config="(port,lcore_rx, lcore_tx[,lcore_kthread, ...]) [, port,lcore_rx, lcore_tx[,lcore_kthread, ...]]":
-    Determines which lcores of RX, TX, kernel thread are mapped to which ports.
+    * ``vEth0_0`` - bound to lcore 8.
+    * ``vEth0_1`` - bound to lcore 10.
+    * ``vEth1_0`` - bound to lcore 9.
+    * ``vEth1_1`` - bound to lcore 11
 
-Refer to *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options.
+.. code-block:: console
 
-The -c coremask or -l corelist parameter of the EAL options should include the lcores indicated by the lcore_rx and lcore_tx,
-but does not need to include lcores indicated by lcore_kthread as they are used to pin the kernel thread on.
-The -p PORTMASK parameter should include the ports indicated by the port in --config, neither more nor less.
+    # rmmod rte_kni
+    # insmod kmod/rte_kni.ko kthread_mode=multiple
+    # ./build/kni -l 4-7 -n 4 -- -P -p 0x3 -m --config="(0,4,6,8,10),(1,5,7,9,11)"
 
-The lcore_kthread in --config can be configured none, one or more lcore IDs.
-In multiple kernel thread mode, if configured none, a KNI device will be allocated for each port,
-while no specific lcore affinity will be set for its kernel thread.
-If configured one or more lcore IDs, one or more KNI devices will be allocated for each port,
-while specific lcore affinity will be set for its kernel thread.
-In single kernel thread mode, if configured none, a KNI device will be allocated for each port.
-If configured one or more lcore IDs,
-one or more KNI devices will be allocated for each port while
-no lcore affinity will be set as there is only one kernel thread for all KNI devices.
+The following example can be used to test the interface between the ``kni``
+test application and the ``rte_kni`` kernel module.  In this example,
+the ``rte_kni`` kernel module is loaded in :ref:`single kernel thread
+mode <kni_kernel_thread_mode>`, :ref:`loopback mode <kni_loopback_mode>`
+enabled, and the :ref:`default carrier state <kni_default_carrier_state>`
+is set to *on* so that the corresponding physical NIC port does not have
+to be connected in order to use the KNI interface.  One KNI interface
+``vEth0_0`` is created for port 0 and one KNI interface ``vEth1_0`` is
+created for port 1.  Since ``rte_kni`` is loaded in "single kernel thread"
+mode, the one kernel thread is bound to lcore 8.
 
-For example, to run the application with two ports served by six lcores, one lcore of RX, one lcore of TX,
-and one lcore of kernel thread for each port:
+Since the physical NIC ports are not being used, link monitoring can be
+disabled by **not** specifying the ``-m`` flag to ``kni``:
 
 .. code-block:: console
 
-    ./build/kni -l 4-7 -n 4 -- -P -p 0x3 --config="(0,4,6,8),(1,5,7,9)"
+    # rmmod rte_kni
+    # insmod kmod/rte_kni.ko lo_mode=lo_mode_fifo carrier=on
+    # ./build/kni -l 4-7 -n 4 -- -P -p 0x3 --config="(0,4,6,8),(1,5,7,9)"
 
 KNI Operations
 --------------
 
-Once the KNI application is started, one can use different Linux* commands to manage the net interfaces.
-If more than one KNI devices configured for a physical port,
-only the first KNI device will be paired to the physical device.
-Operations on other KNI devices will not affect the physical port handled in user space application.
+Once the ``kni`` application is started, the user can use the normal
+Linux commands to manage the KNI interfaces as if they were any other
+Linux network interface.
 
-Assigning an IP address:
+Enable KNI interface and assign an IP address:
 
 .. code-block:: console
 
-    #ifconfig vEth0_0 192.168.0.1
+    # ifconfig vEth0_0 192.168.0.1
 
-Displaying the NIC registers:
+Show KNI interface configuration and statistics:
 
 .. code-block:: console
 
-    #ethtool -d vEth0_0
+    # ifconfig vEth0_0
+
+The user can also check and reset the packet statistics inside the ``kni``
+application by sending the app the USR1 and USR2 signals:
+
+.. code-block:: console
+
+    # Print statistics
+    # kill -SIGUSR1 `pidof kni`
+
+    # Zero statistics
+    # kill -SIGUSR2 `pidof kni`
 
-Dumping the network traffic:
+Dump network traffic:
 
 .. code-block:: console
 
-    #tcpdump -i vEth0_0
+    # tcpdump -i vEth0_0
+
+The normal Linux commands can also be used to change the MAC address and
+MTU size used by the physical NIC which corresponds to the KNI interface.
+However, if more than one KNI interface is configured for a physical port,
+these commands will only work on the first KNI interface for that port.
 
 Change the MAC address:
 
 .. code-block:: console
 
-    #ifconfig vEth0_0 hw ether 0C:01:02:03:04:08
+    # ifconfig vEth0_0 hw ether 0C:01:02:03:04:08
+
+Change the MTU size:
+
+.. code-block:: console
+
+    # ifconfig vEth0_0 mtu 1450
+
+If DPDK is compiled with ``CONFIG_RTE_KNI_KMOD_ETHTOOL=y`` and an Intel
+NIC is used, the user can use ``ethtool`` on the KNI interface as if it
+were a normal Linux kernel interface.
+
+Displaying the NIC registers:
+
+.. code-block:: console
+
+    # ethtool -d vEth0_0
 
-When the DPDK userspace application is closed, all the KNI devices are deleted from Linux*.
+When the ``kni`` application is closed, all the KNI interfaces are deleted
+from the Linux kernel.
 
 Explanation
 -----------
@@ -227,7 +312,7 @@ to see if this lcore is reading from or writing to kernel NIC interfaces.
 For the case that reads from a NIC port and writes to the kernel NIC interfaces (``kni_ingress``),
 the packet reception is the same as in L2 Forwarding sample application
 (see :ref:`l2_fwd_app_rx_tx_packets`).
-The packet transmission is done by sending mbufs into the kernel NIC interfaces by rte_kni_tx_burst().
+The packet transmission is done by sending mbufs into the kernel NIC interfaces by ``rte_kni_tx_burst()``.
 The KNI library automatically frees the mbufs after the kernel successfully copied the mbufs.
 
 For the other case that reads from kernel NIC interfaces
@@ -235,16 +320,3 @@ and writes to a physical NIC port (``kni_egress``),
 packets are retrieved by reading mbufs from kernel NIC interfaces by ``rte_kni_rx_burst()``.
 The packet transmission is the same as in the L2 Forwarding sample application
 (see :ref:`l2_fwd_app_rx_tx_packets`).
-
-Callbacks for Kernel Requests
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-To execute specific PMD operations in user space requested by some Linux* commands,
-callbacks must be implemented and filled in the struct rte_kni_ops structure.
-Currently, setting a new MTU, change in MAC address, configuring promiscusous mode and
-configuring the network interface(up/down) re supported.
-Default implementation for following is available in rte_kni library.
-Application may choose to not implement following callbacks:
-
-- ``config_mac_address``
-- ``config_promiscusity``
diff --git a/examples/kni/Makefile b/examples/kni/Makefile
index 7e19d2e2a..dd90d7d73 100644
--- a/examples/kni/Makefile
+++ b/examples/kni/Makefile
@@ -20,6 +20,7 @@ static: build/$(APP)-static
 
 PC_FILE := $(shell pkg-config --path libdpdk)
 CFLAGS += -O3 $(shell pkg-config --cflags libdpdk)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 LDFLAGS_SHARED = $(shell pkg-config --libs libdpdk)
 LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs libdpdk)
 
@@ -54,6 +55,7 @@ please change the definition of the RTE_TARGET environment variable)
 endif
 
 CFLAGS += -O3
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 CFLAGS += $(WERROR_FLAGS)
 
 include $(RTE_SDK)/mk/rte.extapp.mk
diff --git a/examples/kni/main.c b/examples/kni/main.c
index 80c401c51..4fda51713 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -106,6 +106,8 @@ static struct rte_mempool * pktmbuf_pool = NULL;
 static uint32_t ports_mask = 0;
 /* Ports set in promiscuous mode off by default. */
 static int promiscuous_on = 0;
+/* Monitor link status continually. off by default. */
+static int monitor_links;
 
 /* Structure type for recording kni interface specific stats */
 struct kni_interface_stats {
@@ -325,11 +327,12 @@ main_loop(__rte_unused void *arg)
 static void
 print_usage(const char *prgname)
 {
-	RTE_LOG(INFO, APP, "\nUsage: %s [EAL options] -- -p PORTMASK -P "
+	RTE_LOG(INFO, APP, "\nUsage: %s [EAL options] -- -p PORTMASK -P -m "
 		   "[--config (port,lcore_rx,lcore_tx,lcore_kthread...)"
 		   "[,(port,lcore_rx,lcore_tx,lcore_kthread...)]]\n"
 		   "    -p PORTMASK: hex bitmask of ports to use\n"
 		   "    -P : enable promiscuous mode\n"
+		   "    -m : enable monitoring of port carrier state\n"
 		   "    --config (port,lcore_rx,lcore_tx,lcore_kthread...): "
 		   "port and lcore configurations\n",
 	           prgname);
@@ -510,7 +513,7 @@ parse_args(int argc, char **argv)
 	opterr = 0;
 
 	/* Parse command line */
-	while ((opt = getopt_long(argc, argv, "p:P", longopts,
+	while ((opt = getopt_long(argc, argv, "p:Pm", longopts,
 						&longindex)) != EOF) {
 		switch (opt) {
 		case 'p':
@@ -519,6 +522,9 @@ parse_args(int argc, char **argv)
 		case 'P':
 			promiscuous_on = 1;
 			break;
+		case 'm':
+			monitor_links = 1;
+			break;
 		case 0:
 			if (!strncmp(longopts[longindex].name,
 				     CMDLINE_OPT_CONFIG,
@@ -674,6 +680,55 @@ check_all_ports_link_status(uint32_t port_mask)
 	}
 }
 
+static void
+log_link_state(struct rte_kni *kni, int prev, struct rte_eth_link *link)
+{
+	if (kni == NULL || link == NULL)
+		return;
+
+	if (prev == ETH_LINK_DOWN && link->link_status == ETH_LINK_UP) {
+		RTE_LOG(INFO, APP, "%s NIC Link is Up %d Mbps %s %s.\n",
+			rte_kni_get_name(kni),
+			link->link_speed,
+			link->link_autoneg ?  "(AutoNeg)" : "(Fixed)",
+			link->link_duplex ?  "Full Duplex" : "Half Duplex");
+	} else if (prev == ETH_LINK_UP && link->link_status == ETH_LINK_DOWN) {
+		RTE_LOG(INFO, APP, "%s NIC Link is Down.\n",
+			rte_kni_get_name(kni));
+	}
+}
+
+/*
+ * Monitor the link status of all ports and update the
+ * corresponding KNI interface(s)
+ */
+static void *
+monitor_all_ports_link_status(void *arg)
+{
+	uint16_t portid;
+	struct rte_eth_link link;
+	unsigned int i;
+	struct kni_port_params **p = kni_port_params_array;
+	int prev;
+	(void) arg;
+
+	while (monitor_links) {
+		rte_delay_ms(500);
+		RTE_ETH_FOREACH_DEV(portid) {
+			if ((ports_mask & (1 << portid)) == 0)
+				continue;
+			memset(&link, 0, sizeof(link));
+			rte_eth_link_get_nowait(portid, &link);
+			for (i = 0; i < p[portid]->nb_kni; i++) {
+				prev = rte_kni_update_link(p[portid]->kni[i],
+						link.link_status);
+				log_link_state(p[portid]->kni[i], prev, &link);
+			}
+		}
+	}
+	return NULL;
+}
+
 /* Callback for request of changing MTU */
 static int
 kni_change_mtu(uint16_t port_id, unsigned int new_mtu)
@@ -893,6 +948,8 @@ main(int argc, char** argv)
 	int ret;
 	uint16_t nb_sys_ports, port;
 	unsigned i;
+	void *retval;
+	pthread_t kni_link_tid;
 
 	/* Associate signal_hanlder function with USR signals */
 	signal(SIGUSR1, signal_handler);
@@ -949,12 +1006,21 @@ main(int argc, char** argv)
 	}
 	check_all_ports_link_status(ports_mask);
 
+	ret = rte_ctrl_thread_create(&kni_link_tid,
+				     "KNI link status check", NULL,
+				     monitor_all_ports_link_status, NULL);
+	if (ret < 0)
+		rte_exit(EXIT_FAILURE,
+			"Could not create link status thread!\n");
+
 	/* Launch per-lcore function on every lcore */
 	rte_eal_mp_remote_launch(main_loop, NULL, CALL_MASTER);
 	RTE_LCORE_FOREACH_SLAVE(i) {
 		if (rte_eal_wait_lcore(i) < 0)
 			return -1;
 	}
+	monitor_links = 0;
+	pthread_join(kni_link_tid, &retval);
 
 	/* Release resources */
 	RTE_ETH_FOREACH_DEV(port) {
-- 
2.19.0

[dpdk-dev] [PATCH v7 4/5] examples/kni: add log msgs to show and clear stats

[Dan Gora]

Add logging messages showing the commands necessary for the user to
have the application display and zero the statistics.

Signed-off-by: Dan Gora <dg@adax.com>
---
 examples/kni/main.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/examples/kni/main.c b/examples/kni/main.c
index 4fda51713..0e3b2a2f7 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -171,14 +171,13 @@ signal_handler(int signum)
 	/* When we receive a USR2 signal, reset stats */
 	if (signum == SIGUSR2) {
 		memset(&kni_stats, 0, sizeof(kni_stats));
-		printf("\n**Statistics have been reset**\n");
+		printf("\n** Statistics have been reset **\n");
 		return;
 	}
 
 	/* When we receive a RTMIN or SIGINT signal, stop kni processing */
 	if (signum == SIGRTMIN || signum == SIGINT){
-		printf("SIGRTMIN is received, and the KNI processing is "
-							"going to stop\n");
+		printf("\nSIGRTMIN/SIGINT received. KNI processing stopping.\n");
 		rte_atomic32_inc(&kni_stop);
 		return;
         }
@@ -950,6 +949,7 @@ main(int argc, char** argv)
 	unsigned i;
 	void *retval;
 	pthread_t kni_link_tid;
+	int pid;
 
 	/* Associate signal_hanlder function with USR signals */
 	signal(SIGUSR1, signal_handler);
@@ -1006,6 +1006,16 @@ main(int argc, char** argv)
 	}
 	check_all_ports_link_status(ports_mask);
 
+	pid = getpid();
+	RTE_LOG(INFO, APP, "========================\n");
+	RTE_LOG(INFO, APP, "KNI Running\n");
+	RTE_LOG(INFO, APP, "kill -SIGUSR1 %d\n", pid);
+	RTE_LOG(INFO, APP, "    Show KNI Statistics.\n");
+	RTE_LOG(INFO, APP, "kill -SIGUSR2 %d\n", pid);
+	RTE_LOG(INFO, APP, "    Zero KNI Statistics.\n");
+	RTE_LOG(INFO, APP, "========================\n");
+	fflush(stdout);
+
 	ret = rte_ctrl_thread_create(&kni_link_tid,
 				     "KNI link status check", NULL,
 				     monitor_all_ports_link_status, NULL);
-- 
2.19.0

[dpdk-dev] [PATCH v7 5/5] examples/kni: improve zeroing statistics

[Dan Gora]

The worker threads incrementing the rx/tx_packets race with the signal
handler from the main thread zeroing the entire statistics structure.
This can cause the statistics to fail to be zeroed, even when there
is no traffic on those interfaces.

Improve zeroing the statistics by only incrementing rx/tx_packets
in worker threads by a non-zero amount.  This limits the race to the
periods in which traffic is actually being received or transmitted.

Signed-off-by: Dan Gora <dg@adax.com>
---
 examples/kni/main.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/examples/kni/main.c b/examples/kni/main.c
index 0e3b2a2f7..e37b1ad36 100644
--- a/examples/kni/main.c
+++ b/examples/kni/main.c
@@ -223,7 +223,8 @@ kni_ingress(struct kni_port_params *p)
 		}
 		/* Burst tx to kni */
 		num = rte_kni_tx_burst(p->kni[i], pkts_burst, nb_rx);
-		kni_stats[port_id].rx_packets += num;
+		if (num)
+			kni_stats[port_id].rx_packets += num;
 
 		rte_kni_handle_request(p->kni[i]);
 		if (unlikely(num < nb_rx)) {
@@ -260,7 +261,8 @@ kni_egress(struct kni_port_params *p)
 		}
 		/* Burst tx to eth */
 		nb_tx = rte_eth_tx_burst(port_id, 0, pkts_burst, (uint16_t)num);
-		kni_stats[port_id].tx_packets += nb_tx;
+		if (nb_tx)
+			kni_stats[port_id].tx_packets += nb_tx;
 		if (unlikely(nb_tx < num)) {
 			/* Free mbufs not tx to NIC */
 			kni_burst_free_mbufs(&pkts_burst[nb_tx], num - nb_tx);
-- 
2.19.0

Re: [dpdk-dev] [PATCH v7 0/5] kni: add API to set link status on kernel interface

[Ferruh Yigit]

On 10/24/2018 11:26 PM, Dan Gora wrote:
> Hi All,
> 
> Attached is version 7 of a patchset to add a new API function to
> set the link status on kernel interfaces created with the KNI kernel
> module.
> 
> v7
> ====
> * Add commands to check/clear statistics in kni sample app in KNI sample
>   app documentation.
> 
>> v6
>> ====
>> * Update info about callback functions in KNI library documentation.
>>
>> * Minor fixup to module parameter formatting in KNI kernel module. (align
>>   the "charp" text...)
>>
>> * Add '-m' flag to KNI sample app command line in KNI sample app
>>   documentation.
>>
>> * Add references to KNI library documentation from KNI sample app
>>   documentation (yay Sphinx!)
>>
>> v5
>> ====
>> * Update Kernel NIC Interface document with new rte_kni option, move
>>   kernel module parameter description here from KNI sample app
>>   document, general cleanup.
>>
>> * Update Kernel NIC Interface sample application (kni) document
>>   with new command line parameter, remove documentation for rte_kni
>>   kernel module but add reference to it, general cleanup.
>>
>> * Change link status change sample rate for KNI example app to 500ms
>>   from 100ms.
>>
>> * Fix bug in parameter parsing in v4 in rte_kni kernel module.
>>
>> * Fix output formatting of optional parameters for rte_kni with
>>   modinfo.
>>
>> * Fix permissions of optional parameters for rte_kni.
>>
>> * Squash patch to add test for rte_kni_update_link into patch
>>   introducing the API function.
>>
>> v4
>> ====
>> * Rework rte_kni_update_link to only take linkup/linkdown as parameter,
>>   return previous link state, and remove log messages.
>>
>> * Update patch to set default carrier state to make default carrier
>>   state configurable by passing the 'carrier=[on|off]' option to
>>   the rte_kni kernel module.  This is necessary in order to allow
>>   applications which use KNI as pure virtual interfaces without
>>   corresponding physical ethernet port to use the interfaces without
>>   having to set the carrier state to 'on' via rte_kni_update_link()
>>   or by writing to /sys/devices/virtual/net/<ifaceX>/carrier.
>>   Note that the default is 'off'.
>>
>> * Add command line flag '-m' to examples/kni to continuously monitor
>>   and update the KNI interface link status according to the link
>>   status of the corresponding physical ethernet port.
>>
>>
>> v3
>> ====
>> * Use separate function to test rte_kni_update_link() in 'test' app.
>>
>> * Separate changes to 'test' app into separate patch to facilitate
>>   possible merge with https://patches.dpdk.org/patch/44730/
>>
>> * Remove changes to set KNI interfaces to 'up' in example/kni
>>
>> v2
>> ====
>>
>> * Fix bug where "Fixed" and "AutoNeg" were transposed in the link
>>   status log message.
>>
>> * Add rte_kni_update_link() to rte_kni_version.map
>>
>> * Add rte_kni_update_link() tests to kni_autotest
>>
>> * Update examples/kni to continuously monitor link status and
>>   update the corresponding kernel interface with
>>   rte_kni_update_link().
>>
>> * Minor improvements to examples/kni: Add log message showing how
>>   to show/zero stats.  Improve zeroing statistics.
>>
>> Note that checkpatches.sh compains about patch 1/5, but this appears
>> to be a bug with check-symbol-change or something.  If I move the
>> fragment of the patch modifying rte_kni_version.map to the bottom of
>> the patch file, it doesn't complain any more...  I just don't really
>> have time to investigate this right now.
>   
> thanks
> dan
> 
> Dan Gora (5):
>   kni: add API to set link status on kernel interface
>   kni: set default carrier state of interface
>   examples/kni: monitor and update link status continually
>   examples/kni: add log msgs to show and clear stats
>   examples/kni: improve zeroing statistics

For series,
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

Re: [dpdk-dev] [PATCH v7 0/5] kni: add API to set link status on kernel interface

[Thomas Monjalon]

25/10/2018 14:30, Ferruh Yigit:
> On 10/24/2018 11:26 PM, Dan Gora wrote:
> >> Note that checkpatches.sh compains about patch 1/5, but this appears
> >> to be a bug with check-symbol-change or something.  If I move the
> >> fragment of the patch modifying rte_kni_version.map to the bottom of
> >> the patch file, it doesn't complain any more...  I just don't really
> >> have time to investigate this right now.

Yes we have some false positives with this tool.
Neil, please, would you have time to look at it?

> > Dan Gora (5):
> >   kni: add API to set link status on kernel interface
> >   kni: set default carrier state of interface
> >   examples/kni: monitor and update link status continually
> >   examples/kni: add log msgs to show and clear stats
> >   examples/kni: improve zeroing statistics
> 
> For series,
> Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>

Applied with a fix for meson build

[dpdk-dev] [PATCH v2 10/10] kni: add API to set link status on kernel interface

[Dan Gora]

Add a new API function to KNI, rte_kni_update_link() to allow DPDK
applications to update the link state for the KNI network interfaces
in the linux kernel.

Note that the default carrier state is set to off when the interface
is opened.

Signed-off-by: Dan Gora <dg@adax.com>
---
 kernel/linux/kni/kni_misc.c                   | 63 +++++++++++++++++++
 kernel/linux/kni/kni_net.c                    |  2 +
 .../eal/include/exec-env/rte_kni_common.h     | 19 ++++++
 lib/librte_kni/rte_kni.c                      | 37 ++++++++++-
 lib/librte_kni/rte_kni.h                      | 16 +++++
 5 files changed, 136 insertions(+), 1 deletion(-)

diff --git a/kernel/linux/kni/kni_misc.c b/kernel/linux/kni/kni_misc.c
index 1c38cfa1a..b5784ad1b 100644
--- a/kernel/linux/kni/kni_misc.c
+++ b/kernel/linux/kni/kni_misc.c
@@ -585,6 +585,66 @@ kni_ioctl_free(struct net *net, uint32_t ioctl_num,
 	return ret;
 }
 
+static int
+kni_ioctl_linkstat(struct net *net, uint32_t ioctl_num,
+		unsigned long ioctl_param)
+{
+	struct kni_net *knet = net_generic(net, kni_net_id);
+	int ret = -EINVAL;
+	struct kni_dev *dev, *n;
+	struct rte_kni_link_info link_info;
+	struct net_device *netdev;
+	uint16_t link;
+
+	if (_IOC_SIZE(ioctl_num) > sizeof(link_info))
+		return -EINVAL;
+
+	ret = copy_from_user(&link_info, (void *)ioctl_param,
+			     sizeof(link_info));
+	if (ret) {
+		pr_err("copy_from_user in kni_ioctl_release");
+		return -EIO;
+	}
+
+	/* Release the network device according to its name */
+	if (strlen(link_info.name) == 0)
+		return ret;
+
+	down_read(&knet->kni_list_lock);
+	list_for_each_entry_safe(dev, n, &knet->kni_list_head, list) {
+		if (strncmp(dev->name, link_info.name, RTE_KNI_NAMESIZE) != 0)
+			continue;
+
+		netdev = dev->net_dev;
+		if (netdev == NULL) {
+			up_read(&knet->kni_list_lock);
+			return ret;
+		}
+
+		link = link_info.link_status;
+
+		if (!netif_carrier_ok(netdev) && link) {
+			pr_info("%s NIC Link is Up %d Mbps %s.\n",
+				netdev->name,
+				link_info.link_speed,
+				link_info.link_duplex ==
+					RTE_KNI_LINK_FULL_DUPLEX ?
+					"Full Duplex" : "Half Duplex");
+			netif_carrier_on(netdev);
+		} else if (netif_carrier_ok(netdev) && !link) {
+			pr_info("%s NIC Link is Down.\n",
+				netdev->name);
+			netif_carrier_off(netdev);
+		}
+
+		ret = 0;
+		break;
+	}
+	up_read(&knet->kni_list_lock);
+
+	return ret;
+}
+
 static int
 kni_ioctl(struct inode *inode, uint32_t ioctl_num, unsigned long ioctl_param)
 {
@@ -609,6 +669,9 @@ kni_ioctl(struct inode *inode, uint32_t ioctl_num, unsigned long ioctl_param)
 	case _IOC_NR(RTE_KNI_IOCTL_FREE):
 		ret = kni_ioctl_free(net, ioctl_num, ioctl_param);
 		break;
+	case _IOC_NR(RTE_KNI_IOCTL_LINKSTAT):
+		ret = kni_ioctl_linkstat(net, ioctl_num, ioctl_param);
+		break;
 	default:
 		pr_debug("IOCTL default\n");
 		break;
diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c
index 0850be434..fea3ec7e7 100644
--- a/kernel/linux/kni/kni_net.c
+++ b/kernel/linux/kni/kni_net.c
@@ -134,6 +134,7 @@ kni_net_open(struct net_device *dev)
 	struct kni_dev *kni = netdev_priv(dev);
 
 	netif_start_queue(dev);
+	netif_carrier_off(dev);
 
 	memset(&req, 0, sizeof(req));
 	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
@@ -153,6 +154,7 @@ kni_net_release(struct net_device *dev)
 	struct kni_dev *kni = netdev_priv(dev);
 
 	netif_stop_queue(dev); /* can't transmit any more */
+	netif_carrier_off(dev);
 
 	memset(&req, 0, sizeof(req));
 	req.req_id = RTE_KNI_REQ_CFG_NETWORK_IF;
diff --git a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h
index 318a3f939..f617d8026 100644
--- a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h
+++ b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h
@@ -124,11 +124,30 @@ struct rte_kni_device_info {
 	char mac_addr[6];
 };
 
+
+struct rte_kni_link_info {
+	char name[RTE_KNI_NAMESIZE];  /**< Network device name for KNI */
+	uint32_t link_speed;        /**< ETH_SPEED_NUM_ */
+
+#define RTE_KNI_LINK_HALF_DUPLEX 0 /**< Half-duplex connection. */
+#define RTE_KNI_LINK_FULL_DUPLEX 1 /**< Full-duplex connection. */
+	uint16_t link_duplex  : 1;  /**< RTE_KNI_LINK_[HALF/FULL]_DUPLEX */
+
+#define RTE_KNI_LINK_FIXED       0 /**< No autonegotiation. */
+#define RTE_KNI_LINK_AUTONEG     1 /**< Autonegotiated. */
+	uint16_t link_autoneg : 1;  /**< RTE_KNI_LINK_[AUTONEG/FIXED] */
+
+#define RTE_KNI_LINK_DOWN        0 /**< Link is down. */
+#define RTE_KNI_LINK_UP          1 /**< Link is up. */
+	uint16_t link_status  : 1;  /**< RTE_KNI_LINK_[DOWN/UP] */
+};
+
 #define KNI_DEVICE "kni"
 
 #define RTE_KNI_IOCTL_TEST    _IOWR(0, 1, int)
 #define RTE_KNI_IOCTL_CREATE  _IOWR(0, 2, struct rte_kni_device_info)
 #define RTE_KNI_IOCTL_RELEASE _IOWR(0, 3, struct rte_kni_device_info)
 #define RTE_KNI_IOCTL_FREE    _IOWR(0, 4, struct rte_kni_device_info)
+#define RTE_KNI_IOCTL_LINKSTAT _IOWR(0, 5, struct rte_kni_link_info)
 
 #endif /* _RTE_KNI_COMMON_H_ */
diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c
index 6ef0859bf..aa3559306 100644
--- a/lib/librte_kni/rte_kni.c
+++ b/lib/librte_kni/rte_kni.c
@@ -13,7 +13,6 @@
 
 #include <rte_spinlock.h>
 #include <rte_string_fns.h>
-#include <rte_ethdev.h>
 #include <rte_malloc.h>
 #include <rte_log.h>
 #include <rte_kni.h>
@@ -817,6 +816,42 @@ rte_kni_unregister_handlers(struct rte_kni *kni)
 
 	return 0;
 }
+
+int
+rte_kni_update_link(struct rte_kni *kni, struct rte_eth_link *link)
+{
+	struct rte_kni_link_info link_info;
+
+	if (kni == NULL || !kni->in_use || link == NULL)
+		return -1;
+
+	snprintf(link_info.name, sizeof(link_info.name), "%s", kni->name);
+
+	link_info.link_speed = link->link_speed;
+	if (link->link_duplex == ETH_LINK_FULL_DUPLEX)
+		link_info.link_duplex = RTE_KNI_LINK_FULL_DUPLEX;
+	else
+		link_info.link_duplex = RTE_KNI_LINK_FULL_DUPLEX;
+
+	if (link->link_autoneg == ETH_LINK_FIXED)
+		link_info.link_autoneg = RTE_KNI_LINK_FIXED;
+	else
+		link_info.link_autoneg = RTE_KNI_LINK_AUTONEG;
+
+	if (link->link_status == ETH_LINK_UP)
+		link_info.link_status = RTE_KNI_LINK_UP;
+	else
+		link_info.link_status = RTE_KNI_LINK_DOWN;
+
+	if (ioctl(kni_fd, RTE_KNI_IOCTL_LINKSTAT, &link_info) < 0) {
+		RTE_LOG(ERR, KNI,
+			"Failed to update kni link info for dev '%s'.\n",
+			kni->name);
+		return -1;
+	}
+	return 0;
+}
+
 void
 rte_kni_close(void)
 {
diff --git a/lib/librte_kni/rte_kni.h b/lib/librte_kni/rte_kni.h
index 94516c38f..02d781a32 100644
--- a/lib/librte_kni/rte_kni.h
+++ b/lib/librte_kni/rte_kni.h
@@ -21,6 +21,7 @@
 #include <rte_memory.h>
 #include <rte_mempool.h>
 #include <rte_ether.h>
+#include <rte_ethdev.h>
 
 #include <exec-env/rte_kni_common.h>
 
@@ -255,6 +256,21 @@ int rte_kni_register_handlers(struct rte_kni *kni, struct rte_kni_ops *ops);
  */
 int rte_kni_unregister_handlers(struct rte_kni *kni);
 
+/**
+ *  Update link status info for KNI port.
+ *
+ *  Update the linkup/linkdown status of a KNI interface in the kernel.
+ *
+ *  @param kni
+ *   pointer to struct rte_kni.
+ *
+ *  @return
+ *   On success: 0
+ *   On failure: -1
+ */
+int __rte_experimental
+rte_kni_update_link(struct rte_kni *kni, struct rte_eth_link *link);
+
 /**
  *  Close KNI device.
  */
-- 
2.18.0.rc1.1.g6f333ff2f

[dpdk-dev] [PATCH 1/2] kni: add API to set link status on kernel interface

[Dan Gora]

Add a new API function to KNI, rte_kni_update_link() to allow DPDK
applications to update the link status for KNI network interfaces in
the linux kernel.

Signed-off-by: Dan Gora <dg@adax.com>
---
 lib/librte_kni/rte_kni.c | 57 ++++++++++++++++++++++++++++++++++++++++
 lib/librte_kni/rte_kni.h | 18 +++++++++++++
 2 files changed, 75 insertions(+)

diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c
index 65f6a2b03..afe1d7410 100644
--- a/lib/librte_kni/rte_kni.c
+++ b/lib/librte_kni/rte_kni.c
@@ -790,6 +790,63 @@ rte_kni_unregister_handlers(struct rte_kni *kni)
 
 	return 0;
 }
+
+int
+rte_kni_update_link(struct rte_kni *kni, struct rte_eth_link *link)
+{
+	char path[64];
+	char carrier[2];
+	const char *new_carrier;
+	int fd, ret;
+
+	if (kni == NULL || link == NULL)
+		return -1;
+
+	snprintf(path, sizeof(path), "/sys/devices/virtual/net/%s/carrier",
+		kni->name);
+
+	fd = open(path, O_RDWR);
+	if (fd == -1) {
+		RTE_LOG(ERR, KNI, "Failed to open file: %s.\n", path);
+		return -1;
+	}
+
+	ret = read(fd, carrier, 2);
+	if (ret < 1) {
+		/* Cannot read carrier until interface is marked
+		 * 'up', so don't log an error.
+		 */
+		close(fd);
+		return -1;
+	}
+
+	new_carrier = (link->link_status == ETH_LINK_UP) ? "1" : "0";
+	ret = write(fd, new_carrier, 1);
+	if (ret < 1) {
+		RTE_LOG(ERR, KNI, "Failed to write file: %s.\n", path);
+		close(fd);
+		return -1;
+	}
+
+	if (strncmp(carrier, new_carrier, 1)) {
+		if (link->link_status == ETH_LINK_UP) {
+			RTE_LOG(INFO, KNI, "%s NIC Link is Up %d Mbps %s %s.\n",
+				kni->name,
+				link->link_speed,
+				link->link_autoneg ? "(Fixed)" : "(AutoNeg)",
+				link->link_duplex ?
+					"Full Duplex" : "Half Duplex");
+		} else {
+			RTE_LOG(INFO, KNI, "%s NIC Link is Down.\n",
+				kni->name);
+		}
+	}
+
+	close(fd);
+
+	return 0;
+}
+
 void
 rte_kni_close(void)
 {
diff --git a/lib/librte_kni/rte_kni.h b/lib/librte_kni/rte_kni.h
index 99055e2c2..4118ae97a 100644
--- a/lib/librte_kni/rte_kni.h
+++ b/lib/librte_kni/rte_kni.h
@@ -21,6 +21,7 @@
 #include <rte_memory.h>
 #include <rte_mempool.h>
 #include <rte_ether.h>
+#include <rte_ethdev.h>
 
 #include <exec-env/rte_kni_common.h>
 
@@ -228,6 +229,23 @@ int rte_kni_register_handlers(struct rte_kni *kni, struct rte_kni_ops *ops);
  */
 int rte_kni_unregister_handlers(struct rte_kni *kni);
 
+/**
+ * Update link status info for KNI port.
+ *
+ * Update the linkup/linkdown status of a KNI interface in the kernel.
+ *
+ * @param kni
+ *  pointer to struct rte_kni.
+ * @param link
+ *  pointer to struct rte_eth_link containing new interface status.
+ *
+ * @return
+ *  On success: 0
+ *  On failure: -1
+ */
+int __rte_experimental
+rte_kni_update_link(struct rte_kni *kni, struct rte_eth_link *link);
+
 /**
  *  Close KNI device.
  */
-- 
2.19.0

Re: [dpdk-dev] [PATCH 1/2] kni: add API to set link status on kernel interface

[Dan Gora]

Sorry all.. I totally goofed up with git-send-email..

Please ignore this message.

On Tue, Sep 11, 2018 at 8:14 PM, Dan Gora <dg@adax.com> wrote:
> Add a new API function to KNI, rte_kni_update_link() to allow DPDK
> applications to update the link status for KNI network interfaces in
> the linux kernel.
>