DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH v1 0/7] add json power policy interface for containers
@ 2018-08-30 10:54 David Hunt
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 1/7] examples/power: add checks around hypervisor David Hunt
                   ` (7 more replies)
  0 siblings, 8 replies; 110+ messages in thread
From: David Hunt @ 2018-08-30 10:54 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, david.hunt

The current vm_power_manager example app has the capability to accept power
policies from virtual machines via virtio-serial channels. These power
policies  allow a virtual machine to give information to the power manager
to allow the power manager take care of the power management of the virtual
machine based on the information in the policy.

This power policy functionality is limited to virtual machines sending
the policies to the power manager (which runs in the Host OS), and a solution
was needed for additional methods of sending power policies to the power
manager app.

The main use-case for this modification is for containers and host
applications that wish to send polices to the power manager.

This patchset adds the capability to send power polices and power commands
to the vm_power_manager app via JSON strings through a fifo on the file
system.
For example, given the following file, policy.json:

{"policy": {
  "name": "ubuntu2",
  "command": "create",
  "policy_type": "TIME",
  "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
  "quiet_hours":[ 2, 3, 4, 5, 6 ],
  "core_list":[ 11, 12, 13 ]
}}

Then running the command:

cat policy.json >/tmp/powermonitor/fifo.0

The policy is sent to the vm_power_manager. The power manager app then parses
the JSON data, and inserts the policy into the array of policies.

Part of the patch series contains documentation updates to give all the
details of the valid name-value pairs, the data types, etc.

[1/7] examples/power: add checks around hypervisor
[2/7] lib/power: add changes for host commands/policies
[3/7] examples/power: add necessary changes to guest app
[4/7] examples/power: add host channel to power manager
[5/7] examples/power: add json string handling
[6/7] doc/vm_power_manager: add JSON interface API info
[7/7] examples/power: add json example files

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v1 1/7] examples/power: add checks around hypervisor
  2018-08-30 10:54 [dpdk-dev] [PATCH v1 0/7] add json power policy interface for containers David Hunt
@ 2018-08-30 10:54 ` David Hunt
  2018-08-30 16:59   ` Stephen Hemminger
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 2/7] lib/power: add changes for host commands/policies David Hunt
                   ` (6 subsequent siblings)
  7 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-08-30 10:54 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, david.hunt

Allow vm_power_manager to run without requiring qemy to be present
on the machine. This will be required for instances where the JSON
interface is used for commands and polices, without any VMs present.
A use case for this is a container enviromnent.

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/channel_manager.c | 71 +++++++++++++--------
 examples/vm_power_manager/channel_monitor.c |  2 +-
 2 files changed, 44 insertions(+), 29 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.c b/examples/vm_power_manager/channel_manager.c
index 927fc35ab..2bb8641d3 100644
--- a/examples/vm_power_manager/channel_manager.c
+++ b/examples/vm_power_manager/channel_manager.c
@@ -43,7 +43,8 @@ static unsigned char *global_cpumaps;
 static virVcpuInfo *global_vircpuinfo;
 static size_t global_maplen;
 
-static unsigned global_n_host_cpus;
+static unsigned int global_n_host_cpus;
+static unsigned int global_hypervisor_available;
 
 /*
  * Represents a single Virtual Machine
@@ -198,7 +199,11 @@ get_pcpus_mask(struct channel_info *chan_info, unsigned vcpu)
 {
 	struct virtual_machine_info *vm_info =
 			(struct virtual_machine_info *)chan_info->priv_info;
-	return rte_atomic64_read(&vm_info->pcpu_mask[vcpu]);
+
+	if ((global_hypervisor_available) && (vm_info != NULL))
+		return rte_atomic64_read(&vm_info->pcpu_mask[vcpu]);
+	else
+		return 0;
 }
 
 static inline int
@@ -559,6 +564,8 @@ get_all_vm(int *num_vm, int *num_vcpu)
 				VIR_CONNECT_LIST_DOMAINS_PERSISTENT;
 	unsigned int domain_flag = VIR_DOMAIN_VCPU_CONFIG;
 
+	if (!global_hypervisor_available)
+		return;
 
 	memset(global_cpumaps, 0, CHANNEL_CMDS_MAX_CPUS*global_maplen);
 	if (virNodeGetInfo(global_vir_conn_ptr, &node_info)) {
@@ -768,38 +775,42 @@ connect_hypervisor(const char *path)
 	}
 	return 0;
 }
-
 int
-channel_manager_init(const char *path)
+channel_manager_init(const char *path __rte_unused)
 {
 	virNodeInfo info;
 
 	LIST_INIT(&vm_list_head);
 	if (connect_hypervisor(path) < 0) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to initialize channel manager\n");
-		return -1;
-	}
-
-	global_maplen = VIR_CPU_MAPLEN(CHANNEL_CMDS_MAX_CPUS);
+		global_n_host_cpus = 64;
+		global_hypervisor_available = 0;
+		RTE_LOG(INFO, CHANNEL_MANAGER, "Unable to initialize channel manager\n");
+	} else {
+		global_hypervisor_available = 1;
+
+		global_maplen = VIR_CPU_MAPLEN(CHANNEL_CMDS_MAX_CPUS);
+
+		global_vircpuinfo = rte_zmalloc(NULL,
+				sizeof(*global_vircpuinfo) *
+				CHANNEL_CMDS_MAX_CPUS, RTE_CACHE_LINE_SIZE);
+		if (global_vircpuinfo == NULL) {
+			RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for CPU Info\n");
+			goto error;
+		}
+		global_cpumaps = rte_zmalloc(NULL,
+				CHANNEL_CMDS_MAX_CPUS * global_maplen,
+				RTE_CACHE_LINE_SIZE);
+		if (global_cpumaps == NULL)
+			goto error;
 
-	global_vircpuinfo = rte_zmalloc(NULL, sizeof(*global_vircpuinfo) *
-			CHANNEL_CMDS_MAX_CPUS, RTE_CACHE_LINE_SIZE);
-	if (global_vircpuinfo == NULL) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for CPU Info\n");
-		goto error;
-	}
-	global_cpumaps = rte_zmalloc(NULL, CHANNEL_CMDS_MAX_CPUS * global_maplen,
-			RTE_CACHE_LINE_SIZE);
-	if (global_cpumaps == NULL) {
-		goto error;
+		if (virNodeGetInfo(global_vir_conn_ptr, &info)) {
+			RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to retrieve node Info\n");
+			goto error;
+		}
+		global_n_host_cpus = (unsigned int)info.cpus;
 	}
 
-	if (virNodeGetInfo(global_vir_conn_ptr, &info)) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to retrieve node Info\n");
-		goto error;
-	}
 
-	global_n_host_cpus = (unsigned)info.cpus;
 
 	if (global_n_host_cpus > CHANNEL_CMDS_MAX_CPUS) {
 		RTE_LOG(WARNING, CHANNEL_MANAGER, "The number of host CPUs(%u) exceeds the "
@@ -811,7 +822,8 @@ channel_manager_init(const char *path)
 
 	return 0;
 error:
-	disconnect_hypervisor();
+	if (global_hypervisor_available)
+		disconnect_hypervisor();
 	return -1;
 }
 
@@ -838,7 +850,10 @@ channel_manager_exit(void)
 		rte_free(vm_info);
 	}
 
-	rte_free(global_cpumaps);
-	rte_free(global_vircpuinfo);
-	disconnect_hypervisor();
+	if (global_hypervisor_available) {
+		/* Only needed if hypervisor available */
+		rte_free(global_cpumaps);
+		rte_free(global_vircpuinfo);
+		disconnect_hypervisor();
+	}
 }
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index 7fa47ba97..f180d74e6 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -66,7 +66,7 @@ static void
 core_share_status(int pNo)
 {
 
-	int noVms, noVcpus, z, x, t;
+	int noVms = 0, noVcpus = 0, z, x, t;
 
 	get_all_vm(&noVms, &noVcpus);
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v1 2/7] lib/power: add changes for host commands/policies
  2018-08-30 10:54 [dpdk-dev] [PATCH v1 0/7] add json power policy interface for containers David Hunt
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 1/7] examples/power: add checks around hypervisor David Hunt
@ 2018-08-30 10:54 ` David Hunt
  2018-08-30 16:59   ` Stephen Hemminger
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 3/7] examples/power: add necessary changes to guest app David Hunt
                   ` (5 subsequent siblings)
  7 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-08-30 10:54 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, david.hunt

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 lib/librte_power/channel_commands.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/lib/librte_power/channel_commands.h b/lib/librte_power/channel_commands.h
index ee638eefa..a82724911 100644
--- a/lib/librte_power/channel_commands.h
+++ b/lib/librte_power/channel_commands.h
@@ -19,6 +19,7 @@ extern "C" {
 #define CPU_POWER               1
 #define CPU_POWER_CONNECT       2
 #define PKT_POLICY              3
+#define PKT_POLICY_REMOVE       4
 
 /* CPU Power Command Scaling */
 #define CPU_POWER_SCALE_UP      1
@@ -58,6 +59,8 @@ struct traffic {
 	uint32_t max_max_packet_thresh;
 };
 
+enum core_type { VIRTUAL = 0, PHYSICAL };
+
 struct channel_packet {
 	uint64_t resource_id; /**< core_num, device */
 	uint32_t unit;        /**< scale down/up/min/max */
@@ -70,6 +73,7 @@ struct channel_packet {
 	uint8_t vcpu_to_control[MAX_VCPU_PER_VM];
 	uint8_t num_vcpu;
 	struct timer_profile timer_policy;
+	enum core_type core_type;
 	enum workload workload;
 	enum policy_to_use policy_to_use;
 	struct t_boost_status t_boost_status;
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v1 3/7] examples/power: add necessary changes to guest app
  2018-08-30 10:54 [dpdk-dev] [PATCH v1 0/7] add json power policy interface for containers David Hunt
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 1/7] examples/power: add checks around hypervisor David Hunt
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 2/7] lib/power: add changes for host commands/policies David Hunt
@ 2018-08-30 10:54 ` David Hunt
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 4/7] examples/power: add host channel to power manager David Hunt
                   ` (4 subsequent siblings)
  7 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-08-30 10:54 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, david.hunt

The changes here are minimal, as the guest app functionality is not
changing at all, but there is a new element in the channel_packet
struct that needs to have a default set (channel_packet->core_type).

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/guest_cli/vm_power_cli_guest.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
index 0db1b804f..88b825c3b 100644
--- a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
+++ b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
@@ -92,6 +92,7 @@ set_policy_defaults(struct channel_packet *pkt)
 	pkt->timer_policy.hours_to_use_traffic_profile[0] = 8;
 	pkt->timer_policy.hours_to_use_traffic_profile[1] = 10;
 
+	pkt->core_type = VIRTUAL;
 	pkt->workload = LOW;
 	pkt->policy_to_use = TIME;
 	pkt->command = PKT_POLICY;
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v1 4/7] examples/power: add host channel to power manager
  2018-08-30 10:54 [dpdk-dev] [PATCH v1 0/7] add json power policy interface for containers David Hunt
                   ` (2 preceding siblings ...)
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 3/7] examples/power: add necessary changes to guest app David Hunt
@ 2018-08-30 10:54 ` David Hunt
  2018-09-04  7:31   ` Yao, Lei A
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 5/7] examples/power: add json string handling David Hunt
                   ` (3 subsequent siblings)
  7 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-08-30 10:54 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, david.hunt

This patch adds a fifo channel to the vm_power_manager app through which
we can send commands and polices. Intended for sending JSON strings.
The fifo is at /tmp/powermonitor/fifo.0

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/channel_manager.c | 108 +++++++++++++++
 examples/vm_power_manager/channel_manager.h |  17 ++-
 examples/vm_power_manager/channel_monitor.c | 146 +++++++++++++++-----
 examples/vm_power_manager/main.c            |   2 +
 4 files changed, 238 insertions(+), 35 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.c b/examples/vm_power_manager/channel_manager.c
index 2bb8641d3..bcd106be1 100644
--- a/examples/vm_power_manager/channel_manager.c
+++ b/examples/vm_power_manager/channel_manager.c
@@ -13,6 +13,7 @@
 
 #include <sys/queue.h>
 #include <sys/types.h>
+#include <sys/stat.h>
 #include <sys/socket.h>
 #include <sys/select.h>
 
@@ -284,6 +285,38 @@ open_non_blocking_channel(struct channel_info *info)
 	return 0;
 }
 
+static int
+open_host_channel(struct channel_info *info)
+{
+	int flags;
+
+	info->fd = open(info->channel_path, O_RDWR | O_RSYNC);
+	if (info->fd == -1) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Error(%s) opening fifo for '%s'\n",
+				strerror(errno),
+				info->channel_path);
+		return -1;
+	}
+
+	/* Get current flags */
+	flags = fcntl(info->fd, F_GETFL, 0);
+	if (flags < 0) {
+		RTE_LOG(WARNING, CHANNEL_MANAGER, "Error(%s) fcntl get flags socket for"
+				"'%s'\n", strerror(errno), info->channel_path);
+		return 1;
+	}
+	/* Set to Non Blocking */
+	flags |= O_NONBLOCK;
+	if (fcntl(info->fd, F_SETFL, flags) < 0) {
+		RTE_LOG(WARNING, CHANNEL_MANAGER,
+				"Error(%s) setting non-blocking "
+				"socket for '%s'\n",
+				strerror(errno), info->channel_path);
+		return -1;
+	}
+	return 0;
+}
+
 static int
 setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 		struct channel_info **chan_info_dptr, unsigned channel_num)
@@ -294,6 +327,7 @@ setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 	chan_info->channel_num = channel_num;
 	chan_info->priv_info = (void *)vm_info;
 	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
+	chan_info->type = CHANNEL_TYPE_BINARY;
 	if (open_non_blocking_channel(chan_info) < 0) {
 		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open channel: "
 				"'%s' for VM '%s'\n",
@@ -316,6 +350,35 @@ setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 	return 0;
 }
 
+static int
+setup_host_channel_info(struct channel_info **chan_info_dptr,
+		unsigned int channel_num)
+{
+	struct channel_info *chan_info = *chan_info_dptr;
+
+	chan_info->channel_num = channel_num;
+	chan_info->priv_info = (void *)0;
+	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
+	chan_info->type = CHANNEL_TYPE_JSON;
+	sprintf(chan_info->channel_path, "%sfifo.0", CHANNEL_MGR_SOCKET_PATH);
+
+	if (open_host_channel(chan_info) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open host channel: "
+				"'%s'\n",
+				chan_info->channel_path);
+		return -1;
+	}
+	if (add_channel_to_monitor(&chan_info) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Could add channel: "
+				"'%s' to epoll ctl\n",
+				chan_info->channel_path);
+		return -1;
+
+	}
+	chan_info->status = CHANNEL_MGR_CHANNEL_CONNECTED;
+	return 0;
+}
+
 int
 add_all_channels(const char *vm_name)
 {
@@ -470,6 +533,51 @@ add_channels(const char *vm_name, unsigned *channel_list,
 	return num_channels_enabled;
 }
 
+int
+add_host_channel(void)
+{
+	struct channel_info *chan_info;
+	char socket_path[PATH_MAX];
+	int num_channels_enabled = 0;
+	int ret;
+
+	snprintf(socket_path, sizeof(socket_path), "%sfifo.%u",
+			CHANNEL_MGR_SOCKET_PATH, 0);
+
+	errno = 0;
+	ret = mkfifo(socket_path, 0666);
+	if ((errno != EEXIST) && (ret < 0)) {
+		printf(" %d %d, %d\n", ret, EEXIST, errno);
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Cannot create fifo '%s' error: "
+				"%s\n", socket_path, strerror(errno));
+		return 0;
+	}
+
+	errno = 0;
+	if (access(socket_path, F_OK) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Channel path '%s' error: "
+				"%s\n", socket_path, strerror(errno));
+		return 0;
+	}
+	chan_info = rte_malloc(NULL, sizeof(*chan_info),
+			RTE_CACHE_LINE_SIZE);
+	if (chan_info == NULL) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for "
+				"channel '%s'\n", socket_path);
+		return 0;
+	}
+	snprintf(chan_info->channel_path,
+			sizeof(chan_info->channel_path), "%sfifo.%u",
+			CHANNEL_MGR_SOCKET_PATH, 0);
+	if (setup_host_channel_info(&chan_info, 0) < 0) {
+		rte_free(chan_info);
+		return 0;
+	}
+	num_channels_enabled++;
+
+	return num_channels_enabled;
+}
+
 int
 remove_channel(struct channel_info **chan_info_dptr)
 {
diff --git a/examples/vm_power_manager/channel_manager.h b/examples/vm_power_manager/channel_manager.h
index 872ec6140..c157cc22b 100644
--- a/examples/vm_power_manager/channel_manager.h
+++ b/examples/vm_power_manager/channel_manager.h
@@ -37,7 +37,7 @@ struct sockaddr_un _sockaddr_un;
 #define UNIX_PATH_MAX sizeof(_sockaddr_un.sun_path)
 #endif
 
-#define MAX_VMS 4
+#define MAX_VMS 64
 #define MAX_VCPUS 20
 
 
@@ -54,6 +54,11 @@ enum channel_status { CHANNEL_MGR_CHANNEL_DISCONNECTED = 0,
 	CHANNEL_MGR_CHANNEL_DISABLED,
 	CHANNEL_MGR_CHANNEL_PROCESSING};
 
+/* Communication Channel Type */
+enum channel_type { CHANNEL_TYPE_BINARY = 0,
+	CHANNEL_TYPE_INI,
+	CHANNEL_TYPE_JSON};
+
 /* VM libvirt(qemu/KVM) connection status */
 enum vm_status { CHANNEL_MGR_VM_INACTIVE = 0, CHANNEL_MGR_VM_ACTIVE};
 
@@ -66,6 +71,7 @@ struct channel_info {
 	volatile uint32_t status;    /**< Connection status(enum channel_status) */
 	int fd;                      /**< AF_UNIX socket fd */
 	unsigned channel_num;        /**< CHANNEL_MGR_SOCKET_PATH/<vm_name>.channel_num */
+	enum channel_type type;      /**< Binary, ini, json, etc. */
 	void *priv_info;             /**< Pointer to private info, do not modify */
 };
 
@@ -226,6 +232,15 @@ int add_all_channels(const char *vm_name);
 int add_channels(const char *vm_name, unsigned *channel_list,
 		unsigned num_channels);
 
+/**
+ * Set up a fifo by which host applications can send command an policies
+ * through a fifo to the vm_power_manager
+ *
+ * @return
+ *  - 0 for success
+ */
+int add_host_channel(void);
+
 /**
  * Remove a channel definition from the channel manager. This must only be
  * called from the channel monitor thread.
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index f180d74e6..0ffa1112a 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -85,6 +85,33 @@ core_share_status(int pNo)
 	}
 }
 
+
+static int
+pcpu_monitor(struct policy *pol, struct core_info *ci, int pcpu, int count)
+{
+	int ret = 0;
+
+	if (pol->pkt.policy_to_use == BRANCH_RATIO) {
+		ci->cd[pcpu].oob_enabled = 1;
+		ret = add_core_to_monitor(pcpu);
+		if (ret == 0)
+			RTE_LOG(INFO, CHANNEL_MONITOR,
+					"Monitoring pcpu %d OOB for %s\n",
+					pcpu, pol->pkt.vm_name);
+		else
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Error monitoring pcpu %d OOB for %s\n",
+					pcpu, pol->pkt.vm_name);
+
+	} else {
+		pol->core_share[count].pcpu = pcpu;
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				"Monitoring pcpu %d for %s\n",
+				pcpu, pol->pkt.vm_name);
+	}
+	return ret;
+}
+
 static void
 get_pcpu_to_control(struct policy *pol)
 {
@@ -94,34 +121,46 @@ get_pcpu_to_control(struct policy *pol)
 	int pcpu, count;
 	uint64_t mask_u64b;
 	struct core_info *ci;
-	int ret;
 
 	ci = get_core_info();
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Looking for pcpu for %s\n",
-			pol->pkt.vm_name);
-	get_info_vm(pol->pkt.vm_name, &info);
-
-	for (count = 0; count < pol->pkt.num_vcpu; count++) {
-		mask_u64b = info.pcpu_mask[pol->pkt.vcpu_to_control[count]];
-		for (pcpu = 0; mask_u64b; mask_u64b &= ~(1ULL << pcpu++)) {
-			if ((mask_u64b >> pcpu) & 1) {
-				if (pol->pkt.policy_to_use == BRANCH_RATIO) {
-					ci->cd[pcpu].oob_enabled = 1;
-					ret = add_core_to_monitor(pcpu);
-					if (ret == 0)
-						printf("Monitoring pcpu %d via Branch Ratio\n",
-								pcpu);
-					else
-						printf("Failed to start OOB Monitoring pcpu %d\n",
-								pcpu);
-
-				} else {
-					pol->core_share[count].pcpu = pcpu;
-					printf("Monitoring pcpu %d\n", pcpu);
-				}
+	RTE_LOG(INFO, CHANNEL_MONITOR,
+			"Looking for pcpu for %s\n", pol->pkt.vm_name);
+
+	/*
+	 * So now that we're handling virtual and physical cores, we need to
+	 * differenciate between them when adding them to the branch monitor.
+	 * Virtual cores need to be converted to physical cores.
+	 */
+
+
+
+
+	if (pol->pkt.core_type == VIRTUAL) {
+		/*
+		 * If the cores in the policy are virtual, we need to map them
+		 * to physical core. We look up the vm info and use that for
+		 * the mapping.
+		 */
+		get_info_vm(pol->pkt.vm_name, &info);
+		for (count = 0; count < pol->pkt.num_vcpu; count++) {
+			mask_u64b =
+				info.pcpu_mask[pol->pkt.vcpu_to_control[count]];
+			for (pcpu = 0; mask_u64b;
+					mask_u64b &= ~(1ULL << pcpu++)) {
+				if ((mask_u64b >> pcpu) & 1)
+					pcpu_monitor(pol, ci, pcpu, count);
 			}
 		}
+	} else {
+		/*
+		 * If the cores in the policy are physical, we just use
+		 * those core id's directly.
+		 */
+		for (count = 0; count < pol->pkt.num_vcpu; count++) {
+			pcpu = pol->pkt.vcpu_to_control[count];
+			pcpu_monitor(pol, ci, pcpu, count);
+		}
 	}
 }
 
@@ -160,8 +199,13 @@ update_policy(struct channel_packet *pkt)
 	unsigned int updated = 0;
 	int i;
 
+
+	RTE_LOG(INFO, CHANNEL_MONITOR,
+			"Updating policy for %s\n", pkt->vm_name);
+
 	for (i = 0; i < MAX_VMS; i++) {
 		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
+			/* Copy the contents of *pkt into the policy.pkt */
 			policies[i].pkt = *pkt;
 			get_pcpu_to_control(&policies[i]);
 			if (get_pfid(&policies[i]) == -1) {
@@ -189,6 +233,24 @@ update_policy(struct channel_packet *pkt)
 	return 0;
 }
 
+static int
+remove_policy(struct channel_packet *pkt __rte_unused)
+{
+	int i;
+
+	/*
+	 * Disabling the policy is simply a case of setting
+	 * enabled to 0
+	 */
+	for (i = 0; i < MAX_VMS; i++) {
+		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
+			policies[i].enabled = 0;
+			return 0;
+		}
+	}
+	return -1;
+}
+
 static uint64_t
 get_pkt_diff(struct policy *pol)
 {
@@ -346,7 +408,6 @@ apply_policy(struct policy *pol)
 		apply_workload_profile(pol);
 }
 
-
 static int
 process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 {
@@ -355,6 +416,8 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (chan_info == NULL)
 		return -1;
 
+	RTE_LOG(INFO, CHANNEL_MONITOR, "Processing Request %s\n", pkt->vm_name);
+
 	if (rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_CONNECTED,
 			CHANNEL_MGR_CHANNEL_PROCESSING) == 0)
 		return -1;
@@ -362,10 +425,12 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (pkt->command == CPU_POWER) {
 		core_mask = get_pcpus_mask(chan_info, pkt->resource_id);
 		if (core_mask == 0) {
-			RTE_LOG(ERR, CHANNEL_MONITOR, "Error get physical CPU mask for "
-				"channel '%s' using vCPU(%u)\n", chan_info->channel_path,
-				(unsigned)pkt->unit);
-			return -1;
+			/*
+			 * Core mask will be 0 in the case where
+			 * hypervisor is not available so we're working in
+			 * the host, so use the core as the mask.
+			 */
+			core_mask = 1 << pkt->resource_id;
 		}
 		if (__builtin_popcountll(core_mask) == 1) {
 
@@ -421,12 +486,20 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	}
 
 	if (pkt->command == PKT_POLICY) {
-		RTE_LOG(INFO, CHANNEL_MONITOR, "\nProcessing Policy request from Guest\n");
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				"\nProcessing Policy request\n");
 		update_policy(pkt);
 		policy_is_set = 1;
 	}
 
-	/* Return is not checked as channel status may have been set to DISABLED
+	if (pkt->command == PKT_POLICY_REMOVE) {
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				 "Removing policy %s\n", pkt->vm_name);
+		remove_policy(pkt);
+	}
+
+	/*
+	 * Return is not checked as channel status may have been set to DISABLED
 	 * from management thread
 	 */
 	rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_PROCESSING,
@@ -448,13 +521,16 @@ add_channel_to_monitor(struct channel_info **chan_info)
 				"to epoll\n", info->channel_path);
 		return -1;
 	}
+	RTE_LOG(ERR, CHANNEL_MONITOR, "Added channel '%s' "
+			"to monitor\n", info->channel_path);
 	return 0;
 }
 
 int
 remove_channel_from_monitor(struct channel_info *chan_info)
 {
-	if (epoll_ctl(global_event_fd, EPOLL_CTL_DEL, chan_info->fd, NULL) < 0) {
+	if (epoll_ctl(global_event_fd, EPOLL_CTL_DEL,
+			chan_info->fd, NULL) < 0) {
 		RTE_LOG(ERR, CHANNEL_MONITOR, "Unable to remove channel '%s' "
 				"from epoll\n", chan_info->channel_path);
 		return -1;
@@ -467,11 +543,13 @@ channel_monitor_init(void)
 {
 	global_event_fd = epoll_create1(0);
 	if (global_event_fd == 0) {
-		RTE_LOG(ERR, CHANNEL_MONITOR, "Error creating epoll context with "
-				"error %s\n", strerror(errno));
+		RTE_LOG(ERR, CHANNEL_MONITOR,
+				"Error creating epoll context with error %s\n",
+				strerror(errno));
 		return -1;
 	}
-	global_events_list = rte_malloc("epoll_events", sizeof(*global_events_list)
+	global_events_list = rte_malloc("epoll_events",
+			sizeof(*global_events_list)
 			* MAX_EVENTS, RTE_CACHE_LINE_SIZE);
 	if (global_events_list == NULL) {
 		RTE_LOG(ERR, CHANNEL_MONITOR, "Unable to rte_malloc for "
diff --git a/examples/vm_power_manager/main.c b/examples/vm_power_manager/main.c
index 58c5fa45c..893bf4cdd 100644
--- a/examples/vm_power_manager/main.c
+++ b/examples/vm_power_manager/main.c
@@ -421,6 +421,8 @@ main(int argc, char **argv)
 		return -1;
 	}
 
+	add_host_channel();
+
 	printf("Running core monitor on lcore id %d\n", lcore_id);
 	rte_eal_remote_launch(run_core_monitor, NULL, lcore_id);
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v1 5/7] examples/power: add json string handling
  2018-08-30 10:54 [dpdk-dev] [PATCH v1 0/7] add json power policy interface for containers David Hunt
                   ` (3 preceding siblings ...)
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 4/7] examples/power: add host channel to power manager David Hunt
@ 2018-08-30 10:54 ` David Hunt
  2018-08-30 17:00   ` Stephen Hemminger
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 6/7] doc/vm_power_manager: add JSON interface API info David Hunt
                   ` (2 subsequent siblings)
  7 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-08-30 10:54 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, david.hunt

Add JSON string handling to vm_power_manager for JSON strings received
through the fifo. The format of the JSON strings are detailed in the
next patch, the vm_power_manager user guide documentation updates.

This patch introduces a new dependency on Jansson, a C library for
encoding, decoding and manipulating JSON data. To compile the sample app
you now need to have installed libjansson4 and libjansson-dev (these may
be named slightly differently depending on your Operating System)

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/Makefile          |   2 +-
 examples/vm_power_manager/channel_monitor.c | 295 +++++++++++++++++---
 2 files changed, 252 insertions(+), 45 deletions(-)

diff --git a/examples/vm_power_manager/Makefile b/examples/vm_power_manager/Makefile
index 13a5205ba..72f9e9075 100644
--- a/examples/vm_power_manager/Makefile
+++ b/examples/vm_power_manager/Makefile
@@ -29,7 +29,7 @@ endif
 CFLAGS += -O3 -I$(RTE_SDK)/lib/librte_power/
 CFLAGS += $(WERROR_FLAGS)
 
-LDLIBS += -lvirt
+LDLIBS += -lvirt -ljansson
 
 ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
 
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index 0ffa1112a..b3f2852b4 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -9,10 +9,14 @@
 #include <signal.h>
 #include <errno.h>
 #include <string.h>
+#include <fcntl.h>
 #include <sys/types.h>
 #include <sys/epoll.h>
 #include <sys/queue.h>
 #include <sys/time.h>
+#include <sys/socket.h>
+#include <sys/select.h>
+#include <jansson.h>
 
 #include <rte_log.h>
 #include <rte_memory.h>
@@ -35,6 +39,8 @@
 
 uint64_t vsi_pkt_count_prev[384];
 uint64_t rdtsc_prev[384];
+#define MAX_JSON_STRING_LEN 1024
+char json_data[MAX_JSON_STRING_LEN];
 
 double time_period_ms = 1;
 static volatile unsigned run_loop = 1;
@@ -43,6 +49,134 @@ static unsigned int policy_is_set;
 static struct epoll_event *global_events_list;
 static struct policy policies[MAX_VMS];
 
+
+static int
+parse_json_to_pkt(json_t *element, struct channel_packet *pkt)
+{
+	const char *key;
+	json_t *value;
+	int ret;
+
+	memset(pkt, 0, sizeof(struct channel_packet));
+
+	pkt->nb_mac_to_monitor = 0;
+	pkt->t_boost_status.tbEnabled = false;
+	pkt->workload = LOW;
+	pkt->policy_to_use = TIME;
+	pkt->command = PKT_POLICY;
+	pkt->core_type = PHYSICAL;
+
+	json_object_foreach(element, key, value) {
+		if (!strcmp(key, "policy")) {
+			/* Recurse in to get the contents of profile */
+			ret = parse_json_to_pkt(value, pkt);
+			if (ret)
+				return ret;
+		} else if (!strcmp(key, "instruction")) {
+			/* Recurse in to get the contents of instruction */
+			ret = parse_json_to_pkt(value, pkt);
+			if (ret)
+				return ret;
+		} else if (!strcmp(key, "name")) {
+			strcpy(pkt->vm_name, json_string_value(value));
+		} else if (!strcmp(key, "command")) {
+			char command[32];
+			snprintf(command, 32, "%s", json_string_value(value));
+			if (!strcmp(command, "power")) {
+				pkt->command = CPU_POWER;
+			} else if (!strcmp(command, "create")) {
+				pkt->command = PKT_POLICY;
+			} else if (!strcmp(command, "destroy")) {
+				pkt->command = PKT_POLICY_REMOVE;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Invalid command received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "policy_type")) {
+			char command[32];
+			snprintf(command, 32, "%s", json_string_value(value));
+			if (!strcmp(command, "TIME")) {
+				pkt->policy_to_use = TIME;
+			} else if (!strcmp(command, "TRAFFIC")) {
+				pkt->policy_to_use = TRAFFIC;
+			} else if (!strcmp(command, "WORKLOAD")) {
+				pkt->policy_to_use = WORKLOAD;
+			} else if (!strcmp(command, "BRANCH_RATIO")) {
+				pkt->policy_to_use = BRANCH_RATIO;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Wrong policy_type received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "busy_hours")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int hour = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->timer_policy.busy_hours[i] = hour;
+			}
+		} else if (!strcmp(key, "quiet_hours")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int hour = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->timer_policy.quiet_hours[i] = hour;
+			}
+		} else if (!strcmp(key, "core_list")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int core = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->vcpu_to_control[i] = core;
+			}
+			pkt->num_vcpu = size;
+		} else if (!strcmp(key, "min_packet_thresh")) {
+			pkt->traffic_policy.min_packet_thresh =
+					(uint32_t)json_integer_value(value);
+		} else if (!strcmp(key, "avg_packet_thresh")) {
+			pkt->traffic_policy.avg_max_packet_thresh =
+					(uint32_t)json_integer_value(value);
+		} else if (!strcmp(key, "max_packet_thresh")) {
+			pkt->traffic_policy.max_max_packet_thresh =
+					(uint32_t)json_integer_value(value);
+		} else if (!strcmp(key, "unit")) {
+			char unit[32];
+			snprintf(unit, 32, "%s", json_string_value(value));
+			if (!strcmp(unit, "SCALE_UP")) {
+				pkt->unit = CPU_POWER_SCALE_UP;
+			} else if (!strcmp(unit, "SCALE_DOWN")) {
+				pkt->unit = CPU_POWER_SCALE_DOWN;
+			} else if (!strcmp(unit, "SCALE_MAX")) {
+				pkt->unit = CPU_POWER_SCALE_MAX;
+			} else if (!strcmp(unit, "SCALE_MIN")) {
+				pkt->unit = CPU_POWER_SCALE_MIN;
+			} else if (!strcmp(unit, "ENABLE_TURBO")) {
+				pkt->unit = CPU_POWER_ENABLE_TURBO;
+			} else if (!strcmp(unit, "DISABLE_TURBO")) {
+				pkt->unit = CPU_POWER_DISABLE_TURBO;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Invalid command received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "resource_id")) {
+			pkt->resource_id = (uint32_t)json_integer_value(value);
+		} else {
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+				"Unknown key received in JSON string: %s\n",
+				key);
+		}
+	}
+	return 0;
+}
+
 void channel_monitor_exit(void)
 {
 	run_loop = 0;
@@ -124,18 +258,11 @@ get_pcpu_to_control(struct policy *pol)
 
 	ci = get_core_info();
 
-	RTE_LOG(INFO, CHANNEL_MONITOR,
-			"Looking for pcpu for %s\n", pol->pkt.vm_name);
-
 	/*
 	 * So now that we're handling virtual and physical cores, we need to
 	 * differenciate between them when adding them to the branch monitor.
 	 * Virtual cores need to be converted to physical cores.
 	 */
-
-
-
-
 	if (pol->pkt.core_type == VIRTUAL) {
 		/*
 		 * If the cores in the policy are virtual, we need to map them
@@ -295,8 +422,6 @@ apply_traffic_profile(struct policy *pol)
 
 	diff = get_pkt_diff(pol);
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Applying traffic profile\n");
-
 	if (diff >= (pol->pkt.traffic_policy.max_max_packet_thresh)) {
 		for (count = 0; count < pol->pkt.num_vcpu; count++) {
 			if (pol->core_share[count].status != 1)
@@ -340,9 +465,6 @@ apply_time_profile(struct policy *pol)
 				if (pol->core_share[count].status != 1) {
 					power_manager_scale_core_max(
 						pol->core_share[count].pcpu);
-				RTE_LOG(INFO, CHANNEL_MONITOR,
-					"Scaling up core %d to max\n",
-					pol->core_share[count].pcpu);
 				}
 			}
 			break;
@@ -352,9 +474,6 @@ apply_time_profile(struct policy *pol)
 				if (pol->core_share[count].status != 1) {
 					power_manager_scale_core_min(
 						pol->core_share[count].pcpu);
-				RTE_LOG(INFO, CHANNEL_MONITOR,
-					"Scaling down core %d to min\n",
-					pol->core_share[count].pcpu);
 			}
 		}
 			break;
@@ -416,8 +535,6 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (chan_info == NULL)
 		return -1;
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Processing Request %s\n", pkt->vm_name);
-
 	if (rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_CONNECTED,
 			CHANNEL_MGR_CHANNEL_PROCESSING) == 0)
 		return -1;
@@ -486,8 +603,8 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	}
 
 	if (pkt->command == PKT_POLICY) {
-		RTE_LOG(INFO, CHANNEL_MONITOR,
-				"\nProcessing Policy request\n");
+		RTE_LOG(INFO, CHANNEL_MONITOR, "Processing policy request %s\n",
+				pkt->vm_name);
 		update_policy(pkt);
 		policy_is_set = 1;
 	}
@@ -559,6 +676,106 @@ channel_monitor_init(void)
 	return 0;
 }
 
+static void
+read_binary_packet(struct channel_info *chan_info)
+{
+	struct channel_packet pkt;
+	void *buffer = &pkt;
+	int buffer_len = sizeof(pkt);
+	int n_bytes, err = 0;
+
+	while (buffer_len > 0) {
+		n_bytes = read(chan_info->fd,
+				buffer, buffer_len);
+		if (n_bytes == buffer_len)
+			break;
+		if (n_bytes == -1) {
+			err = errno;
+			RTE_LOG(DEBUG, CHANNEL_MONITOR,
+				"Received error on "
+				"channel '%s' read: %s\n",
+				chan_info->channel_path,
+				strerror(err));
+			remove_channel(&chan_info);
+			break;
+		}
+		buffer = (char *)buffer + n_bytes;
+		buffer_len -= n_bytes;
+	}
+	if (!err)
+		process_request(&pkt, chan_info);
+}
+
+
+static void
+read_json_packet(struct channel_info *chan_info)
+{
+	struct channel_packet pkt;
+	int n_bytes, ret;
+	json_t *root;
+	json_error_t error;
+
+	/* read opening brace to closing brace */
+	do {
+		int idx = 0;
+		int indent = 0;
+		do {
+			n_bytes = read(chan_info->fd, &json_data[idx], 1);
+			if (n_bytes == 0)
+				break;
+			if (json_data[idx] == '{')
+				indent++;
+			if (json_data[idx] == '}')
+				indent--;
+			if ((indent > 0) || (idx >> 0))
+				idx++;
+			if (indent == 0)
+				json_data[idx] = 0;
+			if (idx >= MAX_JSON_STRING_LEN)
+				break;
+		} while (indent > 0);
+
+		if (indent > 0)
+			/*
+			 * We've broken out of the read loop without getting
+			 * a closing brace, so throw away the datai
+			 */
+			json_data[idx] = 0;
+
+		if (strlen(json_data) == 0)
+			continue;
+
+		printf("got [%s]\n", json_data);
+
+		root = json_loads(json_data, 0, &error);
+
+		if (root) {
+			/*
+			 * Because our data is now in the json
+			 * object, we can overwrite the pkt
+			 * with a channel_packet struct, using
+			 * parse_json_to_pkt()
+			 */
+			ret = parse_json_to_pkt(root, &pkt);
+			json_decref(root);
+			if (ret) {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Error validating JSON profile data\n");
+				break;
+			}
+			process_request(&pkt, chan_info);
+		} else {
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+					"JSON error on line %d: %s\n",
+					error.line, error.text);
+		}
+	} while (n_bytes > 0);
+}
+
+
+
+
+
 void
 run_channel_monitor(void)
 {
@@ -570,11 +787,16 @@ run_channel_monitor(void)
 		if (!run_loop)
 			break;
 		for (i = 0; i < n_events; i++) {
+			if (!global_events_list[i].data.ptr) {
+				fflush(stdout);
+				continue;
+			}
 			struct channel_info *chan_info = (struct channel_info *)
 					global_events_list[i].data.ptr;
 			if ((global_events_list[i].events & EPOLLERR) ||
 				(global_events_list[i].events & EPOLLHUP)) {
-				RTE_LOG(DEBUG, CHANNEL_MONITOR, "Remote closed connection for "
+				RTE_LOG(INFO, CHANNEL_MONITOR,
+						"Remote closed connection for "
 						"channel '%s'\n",
 						chan_info->channel_path);
 				remove_channel(&chan_info);
@@ -582,31 +804,16 @@ run_channel_monitor(void)
 			}
 			if (global_events_list[i].events & EPOLLIN) {
 
-				int n_bytes, err = 0;
-				struct channel_packet pkt;
-				void *buffer = &pkt;
-				int buffer_len = sizeof(pkt);
-
-				while (buffer_len > 0) {
-					n_bytes = read(chan_info->fd,
-							buffer, buffer_len);
-					if (n_bytes == buffer_len)
-						break;
-					if (n_bytes == -1) {
-						err = errno;
-						RTE_LOG(DEBUG, CHANNEL_MONITOR,
-							"Received error on "
-							"channel '%s' read: %s\n",
-							chan_info->channel_path,
-							strerror(err));
-						remove_channel(&chan_info);
-						break;
-					}
-					buffer = (char *)buffer + n_bytes;
-					buffer_len -= n_bytes;
+				switch (chan_info->type) {
+				case CHANNEL_TYPE_BINARY:
+					read_binary_packet(chan_info);
+					break;
+				case CHANNEL_TYPE_JSON:
+					read_json_packet(chan_info);
+					break;
+				default:
+					break;
 				}
-				if (!err)
-					process_request(&pkt, chan_info);
 			}
 		}
 		rte_delay_us(time_period_ms*1000);
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v1 6/7] doc/vm_power_manager: add JSON interface API info
  2018-08-30 10:54 [dpdk-dev] [PATCH v1 0/7] add json power policy interface for containers David Hunt
                   ` (4 preceding siblings ...)
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 5/7] examples/power: add json string handling David Hunt
@ 2018-08-30 10:54 ` David Hunt
  2018-09-04  5:17   ` Yao, Lei A
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 7/7] examples/power: add json example files David Hunt
  2018-09-12 14:49 ` [dpdk-dev] [PATCH v2 0/7] add json power policy interface for containers David Hunt
  7 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-08-30 10:54 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, david.hunt

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 .../sample_app_ug/vm_power_management.rst     | 195 ++++++++++++++++++
 1 file changed, 195 insertions(+)

diff --git a/doc/guides/sample_app_ug/vm_power_management.rst b/doc/guides/sample_app_ug/vm_power_management.rst
index 855570d6b..13a325eae 100644
--- a/doc/guides/sample_app_ug/vm_power_management.rst
+++ b/doc/guides/sample_app_ug/vm_power_management.rst
@@ -337,6 +337,201 @@ monitoring of branch ratio on cores doing busy polling via PMDs.
   and will need to be adjusted for different workloads.
 
 
+
+JSON API
+~~~~~~~~
+
+In addition to the command line interface for host command and a virtio-serial
+interface for VM power policies, there is also a JSON interface through which
+power commands and policies can be sent. Sending a command or policy to the
+power manager application is achieved by simply opening a fifo file, writing
+a JSON string to that fifo, and closing the file.
+
+The fifo is at /tmp/powermonitor/fifo.0
+
+The jason string can be a policy or instruction, and takes the following
+format:
+
+  .. code-block:: console
+
+    {"packet_type": {
+      "pair_1": value,
+      "pair_2": value
+    }}
+
+The 'packet_type' header can contain one of two values, depending on
+whether a policy or power command is being sent. The two possible values are
+"policy" and "instruction", and the expected name-value pairs is different
+depending on which type is being sent.
+
+The pairs are the format of standard JSON name-value pairs. The value type
+varies between the different name/value pairs, and may be intgers, strings,
+arrays, etc. Examples of policies follow later in this document. The allowed
+names and value types are as follows:
+
+
+:Pair Name: "name"
+:Description: Name of the VM or Host. Allows the parser to associate the
+  policy with the relevant VM or Host OS.
+:Type: string
+:Values: any valid string
+:Required: yes
+:Example:
+
+  .. code-block:: console
+
+    ""name", "ubuntu2"
+
+
+:Pair Name: "command"
+:Description: The type of packet we're sending to the power manager. We can be
+  creating or destroying a policy, or sending a direct command to adjust
+  the frequency of a core, similar to the command line interface.
+:Type: string
+:Values:
+
+  :"CREATE": used when creating a new policy,
+  :"DESTROY": used when removing a policy,
+  :"POWER": used when sending an immediate command, max, min, etc.
+:Required: yes
+:Example:
+
+    .. code-block:: console
+
+      "command", "CREATE"
+
+
+:Pair Name: "policy_type"
+:Description: Type of policy to apply. Please see vm_power_manager documentation
+  for more information on the types of policies that may be used.
+:Type: string
+:Values:
+
+  :"TIME": Time-of-day policy. Frequencies of the relevant cores are
+    scaled up/down depending on busy and quiet hours.
+  :"TRAFFIC": This policy takes statistics from the NIC and scales up
+    and down accordingly.
+  :"WORKLOAD": This policy looks at how heavily loaded the cores are,
+    and scales up and down accordingly.
+  :"BRANCH_RATIO": This out-of-band policy can look at the ratio between
+    branch hits and misses on a core, and is useful for detecting
+    how much packet processing a core is doing.
+:Required: only for CREATE/DESTROY command
+:Example:
+
+  .. code-block:: console
+
+    "policy_type", "TIME"
+
+:Pair Name: "busy_hours"
+:Description: The hours of the day in which we scale up the cores for busy
+  times.
+:Type: array of integers
+:Values: array with list of hour numbers, (0-23)
+:Required: only for TIME policy
+:Example:
+
+  .. code-block:: console
+
+    "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ]
+
+:Pair Name: "quiet_hours"
+:Description: The hours of the day in which we scale down the cores for quiet
+  times.
+:Type: array of integers
+:Values: array with list of hour numbers, (0-23)
+:Required: only for TIME policy
+:Example:
+
+  .. code-block:: console
+
+    "quiet_hours":[ 2, 3, 4, 5, 6 ]
+
+:Pair Name: "core_list"
+:Description: The cores to which to apply the policy.
+:Type: array of integers
+:Values: array with list of virtual CPUs.
+:Required: only policy CREATE/DESTROY
+:Example:
+
+  .. code-block:: console
+
+    "core_list":[ 10, 11 ]
+
+:Pair Name: "unit"
+:Description: the type of power operation to apply in the command
+:Type: string
+:Values:
+
+  :"SCALE_MAX": Scale frequency of this core to maximum
+  :"SCALE_MIN": Scale frequency of this core to minimum
+  :"SCALE_UP": Scale up frequency of this core
+  :"SCALE_DOWN": Scale down frequency of this core
+  :"ENABLE_TURBO": Enable Turbo Boost for this core
+  :"DISABLE_TURBO": Disable Turbo Boost for this core
+:Required: only for POWER instruction
+:Example:
+
+  .. code-block:: console
+
+    "unit", "SCALE_MAX"
+
+:Pair Name: "resource_id"
+:Description: The core to which to apply the power command.
+:Type: integer
+:Values: valid core id for VM or host OS.
+:Required: only POWER instruction
+:Example:
+
+  .. code-block:: console
+
+    "resource_id": 10
+
+JSON API Examples
+~~~~~~~~~~~~~~~~~
+
+Profile create example:
+
+  .. code-block:: console
+
+    {"policy": {
+      "name": "ubuntu",
+      "command": "create",
+      "policy_type": "TIME",
+      "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
+      "quiet_hours":[ 2, 3, 4, 5, 6 ],
+      "core_list":[ 11 ]
+    }}
+
+Profile destroy example:
+
+  .. code-block:: console
+
+    {"profile": {
+      "name": "ubuntu",
+      "command": "destroy",
+    }}
+
+Power command example:
+
+  .. code-block:: console
+
+    {"command": {
+      "name": "ubuntu",
+      "unit": "SCALE_MAX",
+      "resource_id": 10
+    }}
+
+To send a JSON string to the Power Manager application, simply paste the
+example JSON string into a text file and cat it into the fifo:
+
+  .. code-block:: console
+
+    cat file.json >/tmp/powermonitor/fifo.0
+
+The console of the Power Manager application should indicate the command that
+was just received via the fifo.
+
 Compiling and Running the Guest Applications
 --------------------------------------------
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v1 7/7] examples/power: add json example files
  2018-08-30 10:54 [dpdk-dev] [PATCH v1 0/7] add json power policy interface for containers David Hunt
                   ` (5 preceding siblings ...)
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 6/7] doc/vm_power_manager: add JSON interface API info David Hunt
@ 2018-08-30 10:54 ` David Hunt
  2018-09-12 14:49 ` [dpdk-dev] [PATCH v2 0/7] add json power policy interface for containers David Hunt
  7 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-08-30 10:54 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, david.hunt

This patch provides some example files in the json_examples sub-directory
for sending to the fifo.

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/json_examples/README            | 6 ++++++
 examples/vm_power_manager/json_examples/create.json       | 8 ++++++++
 examples/vm_power_manager/json_examples/destroy.json      | 4 ++++
 examples/vm_power_manager/json_examples/set_core_max.json | 6 ++++++
 examples/vm_power_manager/json_examples/set_core_min.json | 6 ++++++
 5 files changed, 30 insertions(+)
 create mode 100644 examples/vm_power_manager/json_examples/README
 create mode 100644 examples/vm_power_manager/json_examples/create.json
 create mode 100644 examples/vm_power_manager/json_examples/destroy.json
 create mode 100644 examples/vm_power_manager/json_examples/set_core_max.json
 create mode 100644 examples/vm_power_manager/json_examples/set_core_min.json

diff --git a/examples/vm_power_manager/json_examples/README b/examples/vm_power_manager/json_examples/README
new file mode 100644
index 000000000..a94c6b14b
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/README
@@ -0,0 +1,6 @@
+Sample files for sending to the vm_power_manager through the fifo.
+
+Simply cat the file to /tmp/powermonitor/fifo.0 when the vm_power_manager
+application is running.
+
+E.g. cat create.json >/tmp/powermonitor/fifo.0
diff --git a/examples/vm_power_manager/json_examples/create.json b/examples/vm_power_manager/json_examples/create.json
new file mode 100644
index 000000000..a7133d9a1
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/create.json
@@ -0,0 +1,8 @@
+{"policy": {
+  "name": "policy-1",
+  "command": "create",
+  "policy_type": "TIME",
+  "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
+  "quiet_hours":[ 2, 3, 4, 5, 6 ],
+  "core_list":[ 11, 12 ]
+}}
diff --git a/examples/vm_power_manager/json_examples/destroy.json b/examples/vm_power_manager/json_examples/destroy.json
new file mode 100644
index 000000000..587c9e7e9
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/destroy.json
@@ -0,0 +1,4 @@
+{"policy": {
+  "name": "policy-1",
+  "command": "destroy"
+}}
diff --git a/examples/vm_power_manager/json_examples/set_core_max.json b/examples/vm_power_manager/json_examples/set_core_max.json
new file mode 100644
index 000000000..497030a44
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/set_core_max.json
@@ -0,0 +1,6 @@
+{"instruction": {
+  "name": "set_power",
+  "command": "power",
+  "unit": "SCALE_MAX",
+  "resource_id": 10
+}}
diff --git a/examples/vm_power_manager/json_examples/set_core_min.json b/examples/vm_power_manager/json_examples/set_core_min.json
new file mode 100644
index 000000000..76d934fd8
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/set_core_min.json
@@ -0,0 +1,6 @@
+{"instruction": {
+  "name": "set_power",
+  "command": "power",
+  "unit": "SCALE_MIN",
+  "resource_id": 10
+}}
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v1 2/7] lib/power: add changes for host commands/policies
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 2/7] lib/power: add changes for host commands/policies David Hunt
@ 2018-08-30 16:59   ` Stephen Hemminger
  2018-09-12 10:51     ` Hunt, David
  0 siblings, 1 reply; 110+ messages in thread
From: Stephen Hemminger @ 2018-08-30 16:59 UTC (permalink / raw)
  To: David Hunt; +Cc: dev, john.mcnamara

On Thu, 30 Aug 2018 11:54:17 +0100
David Hunt <david.hunt@intel.com> wrote:

> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---
>  lib/librte_power/channel_commands.h | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/lib/librte_power/channel_commands.h b/lib/librte_power/channel_commands.h
> index ee638eefa..a82724911 100644
> --- a/lib/librte_power/channel_commands.h
> +++ b/lib/librte_power/channel_commands.h
> @@ -19,6 +19,7 @@ extern "C" {
>  #define CPU_POWER               1
>  #define CPU_POWER_CONNECT       2
>  #define PKT_POLICY              3
> +#define PKT_POLICY_REMOVE       4
>  
>  /* CPU Power Command Scaling */
>  #define CPU_POWER_SCALE_UP      1
> @@ -58,6 +59,8 @@ struct traffic {
>  	uint32_t max_max_packet_thresh;
>  };
>  
> +enum core_type { VIRTUAL = 0, PHYSICAL };
> +

Why this enum, looks like a boolean to me.

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v1 1/7] examples/power: add checks around hypervisor
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 1/7] examples/power: add checks around hypervisor David Hunt
@ 2018-08-30 16:59   ` Stephen Hemminger
  2018-09-12 10:53     ` Hunt, David
  0 siblings, 1 reply; 110+ messages in thread
From: Stephen Hemminger @ 2018-08-30 16:59 UTC (permalink / raw)
  To: David Hunt; +Cc: dev, john.mcnamara

On Thu, 30 Aug 2018 11:54:16 +0100
David Hunt <david.hunt@intel.com> wrote:

Minor nits

> +static unsigned int global_hypervisor_available;

Please use bool for boolean values.

>  /*
>   * Represents a single Virtual Machine
> @@ -198,7 +199,11 @@ get_pcpus_mask(struct channel_info *chan_info, unsigned vcpu)
>  {
>  	struct virtual_machine_info *vm_info =
>  			(struct virtual_machine_info *)chan_info->priv_info;
> -	return rte_atomic64_read(&vm_info->pcpu_mask[vcpu]);
> +
> +	if ((global_hypervisor_available) && (vm_info != NULL))


 parenthesis are unnecessary here.

I know this is pre-existing, but please don't use CamelCase:

+		if (virNodeGetInfo(global_vir_conn_ptr, &info)) {

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v1 5/7] examples/power: add json string handling
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 5/7] examples/power: add json string handling David Hunt
@ 2018-08-30 17:00   ` Stephen Hemminger
  2018-08-31 13:55     ` Hunt, David
  2018-09-12 10:54     ` Hunt, David
  0 siblings, 2 replies; 110+ messages in thread
From: Stephen Hemminger @ 2018-08-30 17:00 UTC (permalink / raw)
  To: David Hunt; +Cc: dev, john.mcnamara

On Thu, 30 Aug 2018 11:54:20 +0100
David Hunt <david.hunt@intel.com> wrote:

> Add JSON string handling to vm_power_manager for JSON strings received
> through the fifo. The format of the JSON strings are detailed in the
> next patch, the vm_power_manager user guide documentation updates.
> 
> This patch introduces a new dependency on Jansson, a C library for
> encoding, decoding and manipulating JSON data. To compile the sample app
> you now need to have installed libjansson4 and libjansson-dev (these may
> be named slightly differently depending on your Operating System)
> 
> Signed-off-by: David Hunt <david.hunt@intel.com>

If you introduce new dependency then it has to be in documentation,
and off by default in Makefile, and checked in meson build.

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v1 5/7] examples/power: add json string handling
  2018-08-30 17:00   ` Stephen Hemminger
@ 2018-08-31 13:55     ` Hunt, David
  2018-09-12 10:54     ` Hunt, David
  1 sibling, 0 replies; 110+ messages in thread
From: Hunt, David @ 2018-08-31 13:55 UTC (permalink / raw)
  To: Stephen Hemminger; +Cc: dev, john.mcnamara



On 30/8/2018 6:00 PM, Stephen Hemminger wrote:
> On Thu, 30 Aug 2018 11:54:20 +0100
> David Hunt <david.hunt@intel.com> wrote:
>
>> Add JSON string handling to vm_power_manager for JSON strings received
>> through the fifo. The format of the JSON strings are detailed in the
>> next patch, the vm_power_manager user guide documentation updates.
>>
>> This patch introduces a new dependency on Jansson, a C library for
>> encoding, decoding and manipulating JSON data. To compile the sample app
>> you now need to have installed libjansson4 and libjansson-dev (these may
>> be named slightly differently depending on your Operating System)
>>
>> Signed-off-by: David Hunt <david.hunt@intel.com>
> If you introduce new dependency then it has to be in documentation,
> and off by default in Makefile, and checked in meson build.

Hi Stephen,
     Thanks for the review. I'll address your comments, and get a new 
version up soon after Userspace.
Rgds,
Dave.

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v1 6/7] doc/vm_power_manager: add JSON interface API info
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 6/7] doc/vm_power_manager: add JSON interface API info David Hunt
@ 2018-09-04  5:17   ` Yao, Lei A
  2018-09-12 11:31     ` Hunt, David
  0 siblings, 1 reply; 110+ messages in thread
From: Yao, Lei A @ 2018-09-04  5:17 UTC (permalink / raw)
  To: Hunt, David, dev; +Cc: Mcnamara, John, Hunt, David



> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of David Hunt
> Sent: Thursday, August 30, 2018 6:54 PM
> To: dev@dpdk.org
> Cc: Mcnamara, John <john.mcnamara@intel.com>; Hunt, David
> <david.hunt@intel.com>
> Subject: [dpdk-dev] [PATCH v1 6/7] doc/vm_power_manager: add JSON
> interface API info
> 
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---
>  .../sample_app_ug/vm_power_management.rst     | 195
> ++++++++++++++++++
>  1 file changed, 195 insertions(+)
> 
> diff --git a/doc/guides/sample_app_ug/vm_power_management.rst
> b/doc/guides/sample_app_ug/vm_power_management.rst
> index 855570d6b..13a325eae 100644
> --- a/doc/guides/sample_app_ug/vm_power_management.rst
> +++ b/doc/guides/sample_app_ug/vm_power_management.rst
> @@ -337,6 +337,201 @@ monitoring of branch ratio on cores doing busy
> polling via PMDs.
>    and will need to be adjusted for different workloads.
> 
> 
> +
> +JSON API
> +~~~~~~~~
> +
> +In addition to the command line interface for host command and a virtio-
> serial
> +interface for VM power policies, there is also a JSON interface through
> which
> +power commands and policies can be sent. Sending a command or policy to
> the
> +power manager application is achieved by simply opening a fifo file, writing
> +a JSON string to that fifo, and closing the file.
> +
> +The fifo is at /tmp/powermonitor/fifo.0
> +
> +The jason string can be a policy or instruction, and takes the following
> +format:
> +
> +  .. code-block:: console
> +
> +    {"packet_type": {
> +      "pair_1": value,
> +      "pair_2": value
> +    }}
> +
> +The 'packet_type' header can contain one of two values, depending on
> +whether a policy or power command is being sent. The two possible values
> are
> +"policy" and "instruction", and the expected name-value pairs is different
> +depending on which type is being sent.
> +
> +The pairs are the format of standard JSON name-value pairs. The value type
> +varies between the different name/value pairs, and may be intgers, strings,
> +arrays, etc. Examples of policies follow later in this document. The allowed
> +names and value types are as follows:
> +
> +
> +:Pair Name: "name"
> +:Description: Name of the VM or Host. Allows the parser to associate the
> +  policy with the relevant VM or Host OS.
> +:Type: string
> +:Values: any valid string
> +:Required: yes
> +:Example:
> +
> +  .. code-block:: console
> +
> +    ""name", "ubuntu2"
> +
> +
> +:Pair Name: "command"
> +:Description: The type of packet we're sending to the power manager. We
> can be
> +  creating or destroying a policy, or sending a direct command to adjust
> +  the frequency of a core, similar to the command line interface.
> +:Type: string
> +:Values:
> +
> +  :"CREATE": used when creating a new policy,
> +  :"DESTROY": used when removing a policy,
> +  :"POWER": used when sending an immediate command, max, min, etc.
> +:Required: yes
> +:Example:
> +
> +    .. code-block:: console
> +
> +      "command", "CREATE"
> +
> +
> +:Pair Name: "policy_type"
> +:Description: Type of policy to apply. Please see vm_power_manager
> documentation
> +  for more information on the types of policies that may be used.
> +:Type: string
> +:Values:
> +
> +  :"TIME": Time-of-day policy. Frequencies of the relevant cores are
> +    scaled up/down depending on busy and quiet hours.
> +  :"TRAFFIC": This policy takes statistics from the NIC and scales up
> +    and down accordingly.
> +  :"WORKLOAD": This policy looks at how heavily loaded the cores are,
> +    and scales up and down accordingly.
> +  :"BRANCH_RATIO": This out-of-band policy can look at the ratio between
> +    branch hits and misses on a core, and is useful for detecting
> +    how much packet processing a core is doing.
> +:Required: only for CREATE/DESTROY command
> +:Example:
> +
> +  .. code-block:: console
> +
> +    "policy_type", "TIME"
> +
> +:Pair Name: "busy_hours"
> +:Description: The hours of the day in which we scale up the cores for busy
> +  times.
> +:Type: array of integers
> +:Values: array with list of hour numbers, (0-23)
> +:Required: only for TIME policy
> +:Example:
> +
> +  .. code-block:: console
> +
> +    "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ]
> +
> +:Pair Name: "quiet_hours"
> +:Description: The hours of the day in which we scale down the cores for
> quiet
> +  times.
> +:Type: array of integers
> +:Values: array with list of hour numbers, (0-23)
> +:Required: only for TIME policy
> +:Example:
> +
> +  .. code-block:: console
> +
> +    "quiet_hours":[ 2, 3, 4, 5, 6 ]
> +
Do you think we need document following three key here?
min_packet_thresh
avg_packet_thresh
max_packet_thresh
I see them in the code but not documented.
> +:Pair Name: "core_list"
> +:Description: The cores to which to apply the policy.
> +:Type: array of integers
> +:Values: array with list of virtual CPUs.
> +:Required: only policy CREATE/DESTROY
> +:Example:
> +
> +  .. code-block:: console
> +
> +    "core_list":[ 10, 11 ]
> +
> +:Pair Name: "unit"
> +:Description: the type of power operation to apply in the command
> +:Type: string
> +:Values:
> +
> +  :"SCALE_MAX": Scale frequency of this core to maximum
> +  :"SCALE_MIN": Scale frequency of this core to minimum
> +  :"SCALE_UP": Scale up frequency of this core
> +  :"SCALE_DOWN": Scale down frequency of this core
> +  :"ENABLE_TURBO": Enable Turbo Boost for this core
> +  :"DISABLE_TURBO": Disable Turbo Boost for this core
> +:Required: only for POWER instruction
> +:Example:
> +
> +  .. code-block:: console
> +
> +    "unit", "SCALE_MAX"
> +
> +:Pair Name: "resource_id"
> +:Description: The core to which to apply the power command.
> +:Type: integer
> +:Values: valid core id for VM or host OS.
> +:Required: only POWER instruction
> +:Example:
> +
> +  .. code-block:: console
> +
> +    "resource_id": 10
> +
> +JSON API Examples
> +~~~~~~~~~~~~~~~~~
> +
> +Profile create example:
> +
> +  .. code-block:: console
> +
> +    {"policy": {
> +      "name": "ubuntu",
> +      "command": "create",
> +      "policy_type": "TIME",
> +      "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
> +      "quiet_hours":[ 2, 3, 4, 5, 6 ],
> +      "core_list":[ 11 ]
> +    }}
> +
> +Profile destroy example:
> +
> +  .. code-block:: console
> +
> +    {"profile": {
> +      "name": "ubuntu",
> +      "command": "destroy",
> +    }}
> +
> +Power command example:
> +
> +  .. code-block:: console
> +
> +    {"command": {
> +      "name": "ubuntu",
> +      "unit": "SCALE_MAX",
> +      "resource_id": 10
> +    }}
> +
> +To send a JSON string to the Power Manager application, simply paste the
> +example JSON string into a text file and cat it into the fifo:
> +
> +  .. code-block:: console
> +
> +    cat file.json >/tmp/powermonitor/fifo.0
> +
> +The console of the Power Manager application should indicate the
> command that
> +was just received via the fifo.
> +
>  Compiling and Running the Guest Applications
>  --------------------------------------------
> 
> --
> 2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v1 4/7] examples/power: add host channel to power manager
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 4/7] examples/power: add host channel to power manager David Hunt
@ 2018-09-04  7:31   ` Yao, Lei A
  2018-09-12 12:07     ` Hunt, David
  0 siblings, 1 reply; 110+ messages in thread
From: Yao, Lei A @ 2018-09-04  7:31 UTC (permalink / raw)
  To: Hunt, David, dev; +Cc: Mcnamara, John, Hunt, David



> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of David Hunt
> Sent: Thursday, August 30, 2018 6:54 PM
> To: dev@dpdk.org
> Cc: Mcnamara, John <john.mcnamara@intel.com>; Hunt, David
> <david.hunt@intel.com>
> Subject: [dpdk-dev] [PATCH v1 4/7] examples/power: add host channel to
> power manager
> 
> This patch adds a fifo channel to the vm_power_manager app through which
> we can send commands and polices. Intended for sending JSON strings.
> The fifo is at /tmp/powermonitor/fifo.0
> 
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---
>  examples/vm_power_manager/channel_manager.c | 108
> +++++++++++++++
>  examples/vm_power_manager/channel_manager.h |  17 ++-
>  examples/vm_power_manager/channel_monitor.c | 146
> +++++++++++++++-----
>  examples/vm_power_manager/main.c            |   2 +
>  4 files changed, 238 insertions(+), 35 deletions(-)
> 
> diff --git a/examples/vm_power_manager/channel_manager.c
> b/examples/vm_power_manager/channel_manager.c
> index 2bb8641d3..bcd106be1 100644
> --- a/examples/vm_power_manager/channel_manager.c
> +++ b/examples/vm_power_manager/channel_manager.c
> @@ -13,6 +13,7 @@
> 
>  #include <sys/queue.h>
>  #include <sys/types.h>
> +#include <sys/stat.h>
>  #include <sys/socket.h>
>  #include <sys/select.h>
> 
> @@ -284,6 +285,38 @@ open_non_blocking_channel(struct channel_info
> *info)
>  	return 0;
>  }
> 
> +static int
> +open_host_channel(struct channel_info *info)
> +{
> +	int flags;
> +
> +	info->fd = open(info->channel_path, O_RDWR | O_RSYNC);
> +	if (info->fd == -1) {
> +		RTE_LOG(ERR, CHANNEL_MANAGER, "Error(%s) opening fifo
> for '%s'\n",
> +				strerror(errno),
> +				info->channel_path);
> +		return -1;
> +	}
> +
> +	/* Get current flags */
> +	flags = fcntl(info->fd, F_GETFL, 0);
> +	if (flags < 0) {
> +		RTE_LOG(WARNING, CHANNEL_MANAGER, "Error(%s) fcntl
> get flags socket for"
> +				"'%s'\n", strerror(errno), info-
> >channel_path);
> +		return 1;
> +	}
> +	/* Set to Non Blocking */
> +	flags |= O_NONBLOCK;
> +	if (fcntl(info->fd, F_SETFL, flags) < 0) {
> +		RTE_LOG(WARNING, CHANNEL_MANAGER,
> +				"Error(%s) setting non-blocking "
> +				"socket for '%s'\n",
> +				strerror(errno), info->channel_path);
> +		return -1;
> +	}
> +	return 0;
> +}
> +
>  static int
>  setup_channel_info(struct virtual_machine_info **vm_info_dptr,
>  		struct channel_info **chan_info_dptr, unsigned
> channel_num)
> @@ -294,6 +327,7 @@ setup_channel_info(struct virtual_machine_info
> **vm_info_dptr,
>  	chan_info->channel_num = channel_num;
>  	chan_info->priv_info = (void *)vm_info;
>  	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
> +	chan_info->type = CHANNEL_TYPE_BINARY;
>  	if (open_non_blocking_channel(chan_info) < 0) {
>  		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open
> channel: "
>  				"'%s' for VM '%s'\n",
> @@ -316,6 +350,35 @@ setup_channel_info(struct virtual_machine_info
> **vm_info_dptr,
>  	return 0;
>  }
> 
> +static int
> +setup_host_channel_info(struct channel_info **chan_info_dptr,
> +		unsigned int channel_num)
> +{
> +	struct channel_info *chan_info = *chan_info_dptr;
> +
> +	chan_info->channel_num = channel_num;
> +	chan_info->priv_info = (void *)0;
> +	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
> +	chan_info->type = CHANNEL_TYPE_JSON;
> +	sprintf(chan_info->channel_path, "%sfifo.0",
> CHANNEL_MGR_SOCKET_PATH);
> +
> +	if (open_host_channel(chan_info) < 0) {
> +		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open host
> channel: "
> +				"'%s'\n",
> +				chan_info->channel_path);
> +		return -1;
> +	}
> +	if (add_channel_to_monitor(&chan_info) < 0) {
> +		RTE_LOG(ERR, CHANNEL_MANAGER, "Could add channel: "
> +				"'%s' to epoll ctl\n",
> +				chan_info->channel_path);
> +		return -1;
> +
> +	}
> +	chan_info->status = CHANNEL_MGR_CHANNEL_CONNECTED;
> +	return 0;
> +}
> +
>  int
>  add_all_channels(const char *vm_name)
>  {
> @@ -470,6 +533,51 @@ add_channels(const char *vm_name, unsigned
> *channel_list,
>  	return num_channels_enabled;
>  }
> 
> +int
> +add_host_channel(void)
> +{
> +	struct channel_info *chan_info;
> +	char socket_path[PATH_MAX];
> +	int num_channels_enabled = 0;
> +	int ret;
> +
> +	snprintf(socket_path, sizeof(socket_path), "%sfifo.%u",
> +			CHANNEL_MGR_SOCKET_PATH, 0);
> +
> +	errno = 0;
> +	ret = mkfifo(socket_path, 0666);
> +	if ((errno != EEXIST) && (ret < 0)) {
> +		printf(" %d %d, %d\n", ret, EEXIST, errno);
> +		RTE_LOG(ERR, CHANNEL_MANAGER, "Cannot create fifo '%s'
> error: "
> +				"%s\n", socket_path, strerror(errno));
> +		return 0;
> +	}
> +
> +	errno = 0;
> +	if (access(socket_path, F_OK) < 0) {
> +		RTE_LOG(ERR, CHANNEL_MANAGER, "Channel path '%s'
> error: "
> +				"%s\n", socket_path, strerror(errno));
> +		return 0;
> +	}
> +	chan_info = rte_malloc(NULL, sizeof(*chan_info),
> +			RTE_CACHE_LINE_SIZE);
> +	if (chan_info == NULL) {
> +		RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating
> memory for "
> +				"channel '%s'\n", socket_path);
> +		return 0;
> +	}
> +	snprintf(chan_info->channel_path,
> +			sizeof(chan_info->channel_path), "%sfifo.%u",
> +			CHANNEL_MGR_SOCKET_PATH, 0);
> +	if (setup_host_channel_info(&chan_info, 0) < 0) {
> +		rte_free(chan_info);
> +		return 0;
> +	}
> +	num_channels_enabled++;
> +
> +	return num_channels_enabled;
> +}
> +
>  int
>  remove_channel(struct channel_info **chan_info_dptr)
>  {
> diff --git a/examples/vm_power_manager/channel_manager.h
> b/examples/vm_power_manager/channel_manager.h
> index 872ec6140..c157cc22b 100644
> --- a/examples/vm_power_manager/channel_manager.h
> +++ b/examples/vm_power_manager/channel_manager.h
> @@ -37,7 +37,7 @@ struct sockaddr_un _sockaddr_un;
>  #define UNIX_PATH_MAX sizeof(_sockaddr_un.sun_path)
>  #endif
> 
> -#define MAX_VMS 4
> +#define MAX_VMS 64
>  #define MAX_VCPUS 20
> 
> 
> @@ -54,6 +54,11 @@ enum channel_status
> { CHANNEL_MGR_CHANNEL_DISCONNECTED = 0,
>  	CHANNEL_MGR_CHANNEL_DISABLED,
>  	CHANNEL_MGR_CHANNEL_PROCESSING};
> 
> +/* Communication Channel Type */
> +enum channel_type { CHANNEL_TYPE_BINARY = 0,
> +	CHANNEL_TYPE_INI,
> +	CHANNEL_TYPE_JSON};
> +
>  /* VM libvirt(qemu/KVM) connection status */
>  enum vm_status { CHANNEL_MGR_VM_INACTIVE = 0,
> CHANNEL_MGR_VM_ACTIVE};
> 
> @@ -66,6 +71,7 @@ struct channel_info {
>  	volatile uint32_t status;    /**< Connection status(enum
> channel_status) */
>  	int fd;                      /**< AF_UNIX socket fd */
>  	unsigned channel_num;        /**<
> CHANNEL_MGR_SOCKET_PATH/<vm_name>.channel_num */
> +	enum channel_type type;      /**< Binary, ini, json, etc. */
>  	void *priv_info;             /**< Pointer to private info, do not modify */
>  };
> 
> @@ -226,6 +232,15 @@ int add_all_channels(const char *vm_name);
>  int add_channels(const char *vm_name, unsigned *channel_list,
>  		unsigned num_channels);
> 
> +/**
> + * Set up a fifo by which host applications can send command an policies
> + * through a fifo to the vm_power_manager
> + *
> + * @return
> + *  - 0 for success
> + */
> +int add_host_channel(void);
> +
>  /**
>   * Remove a channel definition from the channel manager. This must only be
>   * called from the channel monitor thread.
> diff --git a/examples/vm_power_manager/channel_monitor.c
> b/examples/vm_power_manager/channel_monitor.c
> index f180d74e6..0ffa1112a 100644
> --- a/examples/vm_power_manager/channel_monitor.c
> +++ b/examples/vm_power_manager/channel_monitor.c
> @@ -85,6 +85,33 @@ core_share_status(int pNo)
>  	}
>  }
> 
> +
> +static int
> +pcpu_monitor(struct policy *pol, struct core_info *ci, int pcpu, int count)
> +{
> +	int ret = 0;
> +
> +	if (pol->pkt.policy_to_use == BRANCH_RATIO) {
> +		ci->cd[pcpu].oob_enabled = 1;
> +		ret = add_core_to_monitor(pcpu);
> +		if (ret == 0)
> +			RTE_LOG(INFO, CHANNEL_MONITOR,
> +					"Monitoring pcpu %d OOB for %s\n",
> +					pcpu, pol->pkt.vm_name);
> +		else
> +			RTE_LOG(ERR, CHANNEL_MONITOR,
> +					"Error monitoring pcpu %d OOB
> for %s\n",
> +					pcpu, pol->pkt.vm_name);
> +
> +	} else {
> +		pol->core_share[count].pcpu = pcpu;
> +		RTE_LOG(INFO, CHANNEL_MONITOR,
> +				"Monitoring pcpu %d for %s\n",
> +				pcpu, pol->pkt.vm_name);
> +	}
> +	return ret;
> +}
> +
>  static void
>  get_pcpu_to_control(struct policy *pol)
>  {
> @@ -94,34 +121,46 @@ get_pcpu_to_control(struct policy *pol)
>  	int pcpu, count;
>  	uint64_t mask_u64b;
>  	struct core_info *ci;
> -	int ret;
> 
>  	ci = get_core_info();
> 
> -	RTE_LOG(INFO, CHANNEL_MONITOR, "Looking for pcpu for %s\n",
> -			pol->pkt.vm_name);
> -	get_info_vm(pol->pkt.vm_name, &info);
> -
> -	for (count = 0; count < pol->pkt.num_vcpu; count++) {
> -		mask_u64b = info.pcpu_mask[pol-
> >pkt.vcpu_to_control[count]];
> -		for (pcpu = 0; mask_u64b; mask_u64b &= ~(1ULL << pcpu++))
> {
> -			if ((mask_u64b >> pcpu) & 1) {
> -				if (pol->pkt.policy_to_use == BRANCH_RATIO)
> {
> -					ci->cd[pcpu].oob_enabled = 1;
> -					ret = add_core_to_monitor(pcpu);
> -					if (ret == 0)
> -						printf("Monitoring pcpu %d
> via Branch Ratio\n",
> -								pcpu);
> -					else
> -						printf("Failed to start OOB
> Monitoring pcpu %d\n",
> -								pcpu);
> -
> -				} else {
> -					pol->core_share[count].pcpu = pcpu;
> -					printf("Monitoring pcpu %d\n", pcpu);
> -				}
> +	RTE_LOG(INFO, CHANNEL_MONITOR,
> +			"Looking for pcpu for %s\n", pol->pkt.vm_name);
> +
> +	/*
> +	 * So now that we're handling virtual and physical cores, we need to
> +	 * differenciate between them when adding them to the branch
> monitor.
> +	 * Virtual cores need to be converted to physical cores.
> +	 */
> +
> +
> +
> +
> +	if (pol->pkt.core_type == VIRTUAL) {
> +		/*
> +		 * If the cores in the policy are virtual, we need to map them
> +		 * to physical core. We look up the vm info and use that for
> +		 * the mapping.
> +		 */
> +		get_info_vm(pol->pkt.vm_name, &info);
> +		for (count = 0; count < pol->pkt.num_vcpu; count++) {
> +			mask_u64b =
> +				info.pcpu_mask[pol-
> >pkt.vcpu_to_control[count]];
> +			for (pcpu = 0; mask_u64b;
> +					mask_u64b &= ~(1ULL << pcpu++)) {
> +				if ((mask_u64b >> pcpu) & 1)
> +					pcpu_monitor(pol, ci, pcpu, count);
>  			}
>  		}
> +	} else {
> +		/*
> +		 * If the cores in the policy are physical, we just use
> +		 * those core id's directly.
> +		 */
> +		for (count = 0; count < pol->pkt.num_vcpu; count++) {
> +			pcpu = pol->pkt.vcpu_to_control[count];
> +			pcpu_monitor(pol, ci, pcpu, count);
> +		}
>  	}
>  }
> 
> @@ -160,8 +199,13 @@ update_policy(struct channel_packet *pkt)
>  	unsigned int updated = 0;
>  	int i;
> 
> +
> +	RTE_LOG(INFO, CHANNEL_MONITOR,
> +			"Updating policy for %s\n", pkt->vm_name);
> +
>  	for (i = 0; i < MAX_VMS; i++) {
>  		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
I suggest add warning log here when no VM can match the policy name
which we send through the fifo.0. Otherwise, the user can't aware the 
policy won't be applied.  

> +			/* Copy the contents of *pkt into the policy.pkt */
>  			policies[i].pkt = *pkt;
>  			get_pcpu_to_control(&policies[i]);
>  			if (get_pfid(&policies[i]) == -1) {
> @@ -189,6 +233,24 @@ update_policy(struct channel_packet *pkt)
>  	return 0;
>  }
> 
> +static int
> +remove_policy(struct channel_packet *pkt __rte_unused)
> +{
> +	int i;
> +
> +	/*
> +	 * Disabling the policy is simply a case of setting
> +	 * enabled to 0
> +	 */
> +	for (i = 0; i < MAX_VMS; i++) {
> +		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
> +			policies[i].enabled = 0;
> +			return 0;
> +		}
> +	}
> +	return -1;
> +}
> +
>  static uint64_t
>  get_pkt_diff(struct policy *pol)
>  {
> @@ -346,7 +408,6 @@ apply_policy(struct policy *pol)
>  		apply_workload_profile(pol);
>  }
> 
> -
>  static int
>  process_request(struct channel_packet *pkt, struct channel_info
> *chan_info)
>  {
> @@ -355,6 +416,8 @@ process_request(struct channel_packet *pkt, struct
> channel_info *chan_info)
>  	if (chan_info == NULL)
>  		return -1;
> 
> +	RTE_LOG(INFO, CHANNEL_MONITOR, "Processing Request %s\n",
> pkt->vm_name);
> +
>  	if (rte_atomic32_cmpset(&(chan_info->status),
> CHANNEL_MGR_CHANNEL_CONNECTED,
>  			CHANNEL_MGR_CHANNEL_PROCESSING) == 0)
>  		return -1;
> @@ -362,10 +425,12 @@ process_request(struct channel_packet *pkt,
> struct channel_info *chan_info)
>  	if (pkt->command == CPU_POWER) {
>  		core_mask = get_pcpus_mask(chan_info, pkt->resource_id);
>  		if (core_mask == 0) {
> -			RTE_LOG(ERR, CHANNEL_MONITOR, "Error get
> physical CPU mask for "
> -				"channel '%s' using vCPU(%u)\n", chan_info-
> >channel_path,
> -				(unsigned)pkt->unit);
> -			return -1;
> +			/*
> +			 * Core mask will be 0 in the case where
> +			 * hypervisor is not available so we're working in
> +			 * the host, so use the core as the mask.
> +			 */
> +			core_mask = 1 << pkt->resource_id;
>  		}
>  		if (__builtin_popcountll(core_mask) == 1) {
> 
> @@ -421,12 +486,20 @@ process_request(struct channel_packet *pkt,
> struct channel_info *chan_info)
>  	}
> 
>  	if (pkt->command == PKT_POLICY) {
> -		RTE_LOG(INFO, CHANNEL_MONITOR, "\nProcessing Policy
> request from Guest\n");
> +		RTE_LOG(INFO, CHANNEL_MONITOR,
> +				"\nProcessing Policy request\n");
>  		update_policy(pkt);
>  		policy_is_set = 1;
>  	}
> 
> -	/* Return is not checked as channel status may have been set to
> DISABLED
> +	if (pkt->command == PKT_POLICY_REMOVE) {
> +		RTE_LOG(INFO, CHANNEL_MONITOR,
> +				 "Removing policy %s\n", pkt->vm_name);
> +		remove_policy(pkt);
> +	}
> +
> +	/*
> +	 * Return is not checked as channel status may have been set to
> DISABLED
>  	 * from management thread
>  	 */
>  	rte_atomic32_cmpset(&(chan_info->status),
> CHANNEL_MGR_CHANNEL_PROCESSING,
> @@ -448,13 +521,16 @@ add_channel_to_monitor(struct channel_info
> **chan_info)
>  				"to epoll\n", info->channel_path);
>  		return -1;
>  	}
> +	RTE_LOG(ERR, CHANNEL_MONITOR, "Added channel '%s' "
> +			"to monitor\n", info->channel_path);
>  	return 0;
>  }
> 
>  int
>  remove_channel_from_monitor(struct channel_info *chan_info)
>  {
> -	if (epoll_ctl(global_event_fd, EPOLL_CTL_DEL, chan_info->fd, NULL)
> < 0) {
> +	if (epoll_ctl(global_event_fd, EPOLL_CTL_DEL,
> +			chan_info->fd, NULL) < 0) {
>  		RTE_LOG(ERR, CHANNEL_MONITOR, "Unable to remove
> channel '%s' "
>  				"from epoll\n", chan_info->channel_path);
>  		return -1;
> @@ -467,11 +543,13 @@ channel_monitor_init(void)
>  {
>  	global_event_fd = epoll_create1(0);
>  	if (global_event_fd == 0) {
> -		RTE_LOG(ERR, CHANNEL_MONITOR, "Error creating epoll
> context with "
> -				"error %s\n", strerror(errno));
> +		RTE_LOG(ERR, CHANNEL_MONITOR,
> +				"Error creating epoll context with error %s\n",
> +				strerror(errno));
>  		return -1;
>  	}
> -	global_events_list = rte_malloc("epoll_events",
> sizeof(*global_events_list)
> +	global_events_list = rte_malloc("epoll_events",
> +			sizeof(*global_events_list)
>  			* MAX_EVENTS, RTE_CACHE_LINE_SIZE);
>  	if (global_events_list == NULL) {
>  		RTE_LOG(ERR, CHANNEL_MONITOR, "Unable to rte_malloc
> for "
> diff --git a/examples/vm_power_manager/main.c
> b/examples/vm_power_manager/main.c
> index 58c5fa45c..893bf4cdd 100644
> --- a/examples/vm_power_manager/main.c
> +++ b/examples/vm_power_manager/main.c
> @@ -421,6 +421,8 @@ main(int argc, char **argv)
>  		return -1;
>  	}
> 
> +	add_host_channel();
> +
>  	printf("Running core monitor on lcore id %d\n", lcore_id);
>  	rte_eal_remote_launch(run_core_monitor, NULL, lcore_id);
> 
> --
> 2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v1 2/7] lib/power: add changes for host commands/policies
  2018-08-30 16:59   ` Stephen Hemminger
@ 2018-09-12 10:51     ` Hunt, David
  0 siblings, 0 replies; 110+ messages in thread
From: Hunt, David @ 2018-09-12 10:51 UTC (permalink / raw)
  To: Stephen Hemminger; +Cc: dev, john.mcnamara

Hi Stephen,

On 30/8/2018 5:59 PM, Stephen Hemminger wrote:
> On Thu, 30 Aug 2018 11:54:17 +0100
> David Hunt <david.hunt@intel.com> wrote:
>
>> Signed-off-by: David Hunt <david.hunt@intel.com>
>> ---
>>   lib/librte_power/channel_commands.h | 4 ++++
>>   1 file changed, 4 insertions(+)
>>
>> diff --git a/lib/librte_power/channel_commands.h b/lib/librte_power/channel_commands.h
>> index ee638eefa..a82724911 100644
>> --- a/lib/librte_power/channel_commands.h
>> +++ b/lib/librte_power/channel_commands.h
>> @@ -19,6 +19,7 @@ extern "C" {
>>   #define CPU_POWER               1
>>   #define CPU_POWER_CONNECT       2
>>   #define PKT_POLICY              3
>> +#define PKT_POLICY_REMOVE       4
>>   
>>   /* CPU Power Command Scaling */
>>   #define CPU_POWER_SCALE_UP      1
>> @@ -58,6 +59,8 @@ struct traffic {
>>   	uint32_t max_max_packet_thresh;
>>   };
>>   
>> +enum core_type { VIRTUAL = 0, PHYSICAL };
>> +
> Why this enum, looks like a boolean to me.

I'll change this to a 'bool' in the next version.

Thanks,
Dave.

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v1 1/7] examples/power: add checks around hypervisor
  2018-08-30 16:59   ` Stephen Hemminger
@ 2018-09-12 10:53     ` Hunt, David
  0 siblings, 0 replies; 110+ messages in thread
From: Hunt, David @ 2018-09-12 10:53 UTC (permalink / raw)
  To: Stephen Hemminger; +Cc: dev, john.mcnamara

Hi Stephen,


On 30/8/2018 5:59 PM, Stephen Hemminger wrote:
> On Thu, 30 Aug 2018 11:54:16 +0100
> David Hunt <david.hunt@intel.com> wrote:
>
> Minor nits
>
>> +static unsigned int global_hypervisor_available;
> Please use bool for boolean values.

Will change in next revision.


>>   /*
>>    * Represents a single Virtual Machine
>> @@ -198,7 +199,11 @@ get_pcpus_mask(struct channel_info *chan_info, unsigned vcpu)
>>   {
>>   	struct virtual_machine_info *vm_info =
>>   			(struct virtual_machine_info *)chan_info->priv_info;
>> -	return rte_atomic64_read(&vm_info->pcpu_mask[vcpu]);
>> +
>> +	if ((global_hypervisor_available) && (vm_info != NULL))
>
>   parenthesis are unnecessary here.

Fixed in next version.

>
> I know this is pre-existing, but please don't use CamelCase:
>
> +		if (virNodeGetInfo(global_vir_conn_ptr, &info)) {

Unfortunately I've no control over this, it's part of the libvirt API.

Thanks,
Dave.

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v1 5/7] examples/power: add json string handling
  2018-08-30 17:00   ` Stephen Hemminger
  2018-08-31 13:55     ` Hunt, David
@ 2018-09-12 10:54     ` Hunt, David
  1 sibling, 0 replies; 110+ messages in thread
From: Hunt, David @ 2018-09-12 10:54 UTC (permalink / raw)
  To: Stephen Hemminger; +Cc: dev, john.mcnamara

Hi Stephen,


On 30/8/2018 6:00 PM, Stephen Hemminger wrote:
> On Thu, 30 Aug 2018 11:54:20 +0100
> David Hunt <david.hunt@intel.com> wrote:
>
>> Add JSON string handling to vm_power_manager for JSON strings received
>> through the fifo. The format of the JSON strings are detailed in the
>> next patch, the vm_power_manager user guide documentation updates.
>>
>> This patch introduces a new dependency on Jansson, a C library for
>> encoding, decoding and manipulating JSON data. To compile the sample app
>> you now need to have installed libjansson4 and libjansson-dev (these may
>> be named slightly differently depending on your Operating System)
>>
>> Signed-off-by: David Hunt <david.hunt@intel.com>
> If you introduce new dependency then it has to be in documentation,
> and off by default in Makefile, and checked in meson build.

Sure, I've added a check in the Makefile for the existence of the 
library, and it will build in the
JSON handling if present, otherwise warn the user and build without.

Thanks,
Dave.

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v1 6/7] doc/vm_power_manager: add JSON interface API info
  2018-09-04  5:17   ` Yao, Lei A
@ 2018-09-12 11:31     ` Hunt, David
  0 siblings, 0 replies; 110+ messages in thread
From: Hunt, David @ 2018-09-12 11:31 UTC (permalink / raw)
  To: Yao, Lei A, dev; +Cc: Mcnamara, John

Hi Lei,


On 4/9/2018 6:17 AM, Yao, Lei A wrote:
>
>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of David Hunt
>> Sent: Thursday, August 30, 2018 6:54 PM
>> To: dev@dpdk.org
>> Cc: Mcnamara, John <john.mcnamara@intel.com>; Hunt, David
>> <david.hunt@intel.com>
>> Subject: [dpdk-dev] [PATCH v1 6/7] doc/vm_power_manager: add JSON
>> interface API info
>>
>> Signed-off-by: David Hunt <david.hunt@intel.com>
>> ---
>>   .../sample_app_ug/vm_power_management.rst     | 195
>> ++++++++++++++++++
>>   1 file changed, 195 insertions(+)
>>
>> diff --git a/doc/guides/sample_app_ug/vm_power_management.rst
>> b/doc/guides/sample_app_ug/vm_power_management.rst
>> index 855570d6b..13a325eae 100644
>> --- a/doc/guides/sample_app_ug/vm_power_management.rst
>> +++ b/doc/guides/sample_app_ug/vm_power_management.rst
>> @@ -337,6 +337,201 @@ monitoring of branch ratio on cores doing busy
>> polling via PMDs.
>>     and will need to be adjusted for different workloads.
>>
>>
>> +
>> +JSON API
>> +~~~~~~~~
>> +
--snip--
>> +:Pair Name: "quiet_hours"
>> +:Description: The hours of the day in which we scale down the cores for
>> quiet
>> +  times.
>> +:Type: array of integers
>> +:Values: array with list of hour numbers, (0-23)
>> +:Required: only for TIME policy
>> +:Example:
>> +
>> +  .. code-block:: console
>> +
>> +    "quiet_hours":[ 2, 3, 4, 5, 6 ]
>> +
> Do you think we need document following three key here?
> min_packet_thresh
> avg_packet_thresh
> max_packet_thresh
> I see them in the code but not documented.
>

Good catch. I missed these in the docs. Also I checked the code there 
these are
used, and only two are needed. These are avg and max.
Below, avg, the freq is set to minimum
Between avg and max, the freq us set to medium
Above max, the freq is set to maximum.

I'll remove the minimum from the code, seeing as it's not used, and 
document avg and max.

Rgds,
Dave.

--snip--

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v1 4/7] examples/power: add host channel to power manager
  2018-09-04  7:31   ` Yao, Lei A
@ 2018-09-12 12:07     ` Hunt, David
  0 siblings, 0 replies; 110+ messages in thread
From: Hunt, David @ 2018-09-12 12:07 UTC (permalink / raw)
  To: Yao, Lei A, dev; +Cc: Mcnamara, John

Hi Lei,


On 4/9/2018 8:31 AM, Yao, Lei A wrote:
>
>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of David Hunt
>> Sent: Thursday, August 30, 2018 6:54 PM
>> To: dev@dpdk.org
>> Cc: Mcnamara, John <john.mcnamara@intel.com>; Hunt, David
>> <david.hunt@intel.com>
>> Subject: [dpdk-dev] [PATCH v1 4/7] examples/power: add host channel to
>> power manager
>>
>> This patch adds a fifo channel to the vm_power_manager app through which
>> we can send commands and polices. Intended for sending JSON strings.
>> The fifo is at /tmp/powermonitor/fifo.0
>>
>> Signed-off-by: David Hunt <david.hunt@intel.com>
>> ---
>>   examples/vm_power_manager/channel_manager.c | 108
>> +++++++++++++++
>>   examples/vm_power_manager/channel_manager.h |  17 ++-
>>   examples/vm_power_manager/channel_monitor.c | 146
>> +++++++++++++++-----
>>   examples/vm_power_manager/main.c            |   2 +
>>   4 files changed, 238 insertions(+), 35 deletions(-)
>>
>> diff --git a/examples/vm_power_manager/channel_manager.c
>> b/examples/vm_power_manager/channel_manager.c
>> index 2bb8641d3..bcd106be1 100644
>> --- a/examples/vm_power_manager/channel_manager.c
>> +++ b/examples/vm_power_manager/channel_manager.c
>> @@ -13,6 +13,7 @@

--snip--

>>
>> @@ -160,8 +199,13 @@ update_policy(struct channel_packet *pkt)
>>   	unsigned int updated = 0;
>>   	int i;
>>
>> +
>> +	RTE_LOG(INFO, CHANNEL_MONITOR,
>> +			"Updating policy for %s\n", pkt->vm_name);
>> +
>>   	for (i = 0; i < MAX_VMS; i++) {
>>   		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
> I suggest add warning log here when no VM can match the policy name
> which we send through the fifo.0. Otherwise, the user can't aware the
> policy won't be applied.

There's already a flag here called "updated" that if it falls through 
this loop without finding the policy name, it adds a new one, so no need 
for the message.
I will however re-word the "Updating policy" message to read "Applying 
policy". "Applying" is less confusing.

Thanks,
Dave.


---snip---

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v2 0/7] add json power policy interface for containers
  2018-08-30 10:54 [dpdk-dev] [PATCH v1 0/7] add json power policy interface for containers David Hunt
                   ` (6 preceding siblings ...)
  2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 7/7] examples/power: add json example files David Hunt
@ 2018-09-12 14:49 ` David Hunt
  2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 1/7] examples/power: add checks around hypervisor David Hunt
                     ` (7 more replies)
  7 siblings, 8 replies; 110+ messages in thread
From: David Hunt @ 2018-09-12 14:49 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao

Patch v2:
  * Fixed review comments from Stephen Hemminger and Lei A Yao.
  * Added a check in the Makefile for libjansson-dev. Will Warn user and build
    without JSON functionality if not present, will build including JSON
    functionality if it is present.

The current vm_power_manager example app has the capability to accept power
policies from virtual machines via virtio-serial channels. These power
policies  allow a virtual machine to give information to the power manager
to allow the power manager take care of the power management of the virtual
machine based on the information in the policy.

This power policy functionality is limited to virtual machines sending
the policies to the power manager (which runs in the Host OS), and a solution
was needed for additional methods of sending power policies to the power
manager app.

The main use-case for this modification is for containers and host
applications that wish to send polices to the power manager.

This patchset adds the capability to send power polices and power commands
to the vm_power_manager app via JSON strings through a fifo on the file
system.
For example, given the following file, policy.json:

{"policy": {
  "name": "ubuntu2",
  "command": "create",
  "policy_type": "TIME",
  "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
  "quiet_hours":[ 2, 3, 4, 5, 6 ],
  "core_list":[ 11, 12, 13 ]
}}

Then running the command:

cat policy.json >/tmp/powermonitor/fifo.0

The policy is sent to the vm_power_manager. The power manager app then parses
the JSON data, and inserts the policy into the array of policies.

Part of the patch series contains documentation updates to give all the
details of the valid name-value pairs, the data types, etc.

[1/7] examples/power: add checks around hypervisor
[2/7] lib/power: add changes for host commands/policies
[3/7] examples/power: add necessary changes to guest app
[4/7] examples/power: add host channel to power manager
[5/7] examples/power: add json string handling
[6/7] doc/vm_power_manager: add JSON interface API info
[7/7] examples/power: add json example files

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v2 1/7] examples/power: add checks around hypervisor
  2018-09-12 14:49 ` [dpdk-dev] [PATCH v2 0/7] add json power policy interface for containers David Hunt
@ 2018-09-12 14:49   ` David Hunt
  2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 2/7] lib/power: add changes for host commands/policies David Hunt
                     ` (6 subsequent siblings)
  7 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-12 14:49 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, David Hunt

Allow vm_power_manager to run without requiring qemu to be present
on the machine. This will be required for instances where the JSON
interface is used for commands and polices, without any VMs present.
A use case for this is a container enviromnent.

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/channel_manager.c | 71 +++++++++++++--------
 examples/vm_power_manager/channel_monitor.c |  2 +-
 2 files changed, 44 insertions(+), 29 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.c b/examples/vm_power_manager/channel_manager.c
index 927fc35ab..2e471d0c1 100644
--- a/examples/vm_power_manager/channel_manager.c
+++ b/examples/vm_power_manager/channel_manager.c
@@ -43,7 +43,8 @@ static unsigned char *global_cpumaps;
 static virVcpuInfo *global_vircpuinfo;
 static size_t global_maplen;
 
-static unsigned global_n_host_cpus;
+static unsigned int global_n_host_cpus;
+static bool global_hypervisor_available;
 
 /*
  * Represents a single Virtual Machine
@@ -198,7 +199,11 @@ get_pcpus_mask(struct channel_info *chan_info, unsigned vcpu)
 {
 	struct virtual_machine_info *vm_info =
 			(struct virtual_machine_info *)chan_info->priv_info;
-	return rte_atomic64_read(&vm_info->pcpu_mask[vcpu]);
+
+	if (global_hypervisor_available && (vm_info != NULL))
+		return rte_atomic64_read(&vm_info->pcpu_mask[vcpu]);
+	else
+		return 0;
 }
 
 static inline int
@@ -559,6 +564,8 @@ get_all_vm(int *num_vm, int *num_vcpu)
 				VIR_CONNECT_LIST_DOMAINS_PERSISTENT;
 	unsigned int domain_flag = VIR_DOMAIN_VCPU_CONFIG;
 
+	if (!global_hypervisor_available)
+		return;
 
 	memset(global_cpumaps, 0, CHANNEL_CMDS_MAX_CPUS*global_maplen);
 	if (virNodeGetInfo(global_vir_conn_ptr, &node_info)) {
@@ -768,38 +775,42 @@ connect_hypervisor(const char *path)
 	}
 	return 0;
 }
-
 int
-channel_manager_init(const char *path)
+channel_manager_init(const char *path __rte_unused)
 {
 	virNodeInfo info;
 
 	LIST_INIT(&vm_list_head);
 	if (connect_hypervisor(path) < 0) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to initialize channel manager\n");
-		return -1;
-	}
-
-	global_maplen = VIR_CPU_MAPLEN(CHANNEL_CMDS_MAX_CPUS);
+		global_n_host_cpus = 64;
+		global_hypervisor_available = 0;
+		RTE_LOG(INFO, CHANNEL_MANAGER, "Unable to initialize channel manager\n");
+	} else {
+		global_hypervisor_available = 1;
+
+		global_maplen = VIR_CPU_MAPLEN(CHANNEL_CMDS_MAX_CPUS);
+
+		global_vircpuinfo = rte_zmalloc(NULL,
+				sizeof(*global_vircpuinfo) *
+				CHANNEL_CMDS_MAX_CPUS, RTE_CACHE_LINE_SIZE);
+		if (global_vircpuinfo == NULL) {
+			RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for CPU Info\n");
+			goto error;
+		}
+		global_cpumaps = rte_zmalloc(NULL,
+				CHANNEL_CMDS_MAX_CPUS * global_maplen,
+				RTE_CACHE_LINE_SIZE);
+		if (global_cpumaps == NULL)
+			goto error;
 
-	global_vircpuinfo = rte_zmalloc(NULL, sizeof(*global_vircpuinfo) *
-			CHANNEL_CMDS_MAX_CPUS, RTE_CACHE_LINE_SIZE);
-	if (global_vircpuinfo == NULL) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for CPU Info\n");
-		goto error;
-	}
-	global_cpumaps = rte_zmalloc(NULL, CHANNEL_CMDS_MAX_CPUS * global_maplen,
-			RTE_CACHE_LINE_SIZE);
-	if (global_cpumaps == NULL) {
-		goto error;
+		if (virNodeGetInfo(global_vir_conn_ptr, &info)) {
+			RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to retrieve node Info\n");
+			goto error;
+		}
+		global_n_host_cpus = (unsigned int)info.cpus;
 	}
 
-	if (virNodeGetInfo(global_vir_conn_ptr, &info)) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to retrieve node Info\n");
-		goto error;
-	}
 
-	global_n_host_cpus = (unsigned)info.cpus;
 
 	if (global_n_host_cpus > CHANNEL_CMDS_MAX_CPUS) {
 		RTE_LOG(WARNING, CHANNEL_MANAGER, "The number of host CPUs(%u) exceeds the "
@@ -811,7 +822,8 @@ channel_manager_init(const char *path)
 
 	return 0;
 error:
-	disconnect_hypervisor();
+	if (global_hypervisor_available)
+		disconnect_hypervisor();
 	return -1;
 }
 
@@ -838,7 +850,10 @@ channel_manager_exit(void)
 		rte_free(vm_info);
 	}
 
-	rte_free(global_cpumaps);
-	rte_free(global_vircpuinfo);
-	disconnect_hypervisor();
+	if (global_hypervisor_available) {
+		/* Only needed if hypervisor available */
+		rte_free(global_cpumaps);
+		rte_free(global_vircpuinfo);
+		disconnect_hypervisor();
+	}
 }
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index 7fa47ba97..f180d74e6 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -66,7 +66,7 @@ static void
 core_share_status(int pNo)
 {
 
-	int noVms, noVcpus, z, x, t;
+	int noVms = 0, noVcpus = 0, z, x, t;
 
 	get_all_vm(&noVms, &noVcpus);
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v2 2/7] lib/power: add changes for host commands/policies
  2018-09-12 14:49 ` [dpdk-dev] [PATCH v2 0/7] add json power policy interface for containers David Hunt
  2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 1/7] examples/power: add checks around hypervisor David Hunt
@ 2018-09-12 14:49   ` David Hunt
  2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 3/7] examples/power: add necessary changes to guest app David Hunt
                     ` (5 subsequent siblings)
  7 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-12 14:49 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, David Hunt

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 lib/librte_power/channel_commands.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/lib/librte_power/channel_commands.h b/lib/librte_power/channel_commands.h
index ee638eefa..e7b93a797 100644
--- a/lib/librte_power/channel_commands.h
+++ b/lib/librte_power/channel_commands.h
@@ -19,6 +19,7 @@ extern "C" {
 #define CPU_POWER               1
 #define CPU_POWER_CONNECT       2
 #define PKT_POLICY              3
+#define PKT_POLICY_REMOVE       4
 
 /* CPU Power Command Scaling */
 #define CPU_POWER_SCALE_UP      1
@@ -58,6 +59,9 @@ struct traffic {
 	uint32_t max_max_packet_thresh;
 };
 
+#define CORE_TYPE_VIRTUAL 0
+#define CORE_TYPE_PHYSICAL 1
+
 struct channel_packet {
 	uint64_t resource_id; /**< core_num, device */
 	uint32_t unit;        /**< scale down/up/min/max */
@@ -70,6 +74,7 @@ struct channel_packet {
 	uint8_t vcpu_to_control[MAX_VCPU_PER_VM];
 	uint8_t num_vcpu;
 	struct timer_profile timer_policy;
+	bool core_type;
 	enum workload workload;
 	enum policy_to_use policy_to_use;
 	struct t_boost_status t_boost_status;
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v2 3/7] examples/power: add necessary changes to guest app
  2018-09-12 14:49 ` [dpdk-dev] [PATCH v2 0/7] add json power policy interface for containers David Hunt
  2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 1/7] examples/power: add checks around hypervisor David Hunt
  2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 2/7] lib/power: add changes for host commands/policies David Hunt
@ 2018-09-12 14:49   ` David Hunt
  2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 4/7] examples/power: add host channel to power manager David Hunt
                     ` (4 subsequent siblings)
  7 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-12 14:49 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, David Hunt

The changes here are minimal, as the guest app functionality is not
changing at all, but there is a new element in the channel_packet
struct that needs to have a default set (channel_packet->core_type).

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/guest_cli/vm_power_cli_guest.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
index 0db1b804f..88b825c3b 100644
--- a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
+++ b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
@@ -92,6 +92,7 @@ set_policy_defaults(struct channel_packet *pkt)
 	pkt->timer_policy.hours_to_use_traffic_profile[0] = 8;
 	pkt->timer_policy.hours_to_use_traffic_profile[1] = 10;
 
+	pkt->core_type = VIRTUAL;
 	pkt->workload = LOW;
 	pkt->policy_to_use = TIME;
 	pkt->command = PKT_POLICY;
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v2 4/7] examples/power: add host channel to power manager
  2018-09-12 14:49 ` [dpdk-dev] [PATCH v2 0/7] add json power policy interface for containers David Hunt
                     ` (2 preceding siblings ...)
  2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 3/7] examples/power: add necessary changes to guest app David Hunt
@ 2018-09-12 14:49   ` David Hunt
  2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 5/7] examples/power: add json string handling David Hunt
                     ` (3 subsequent siblings)
  7 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-12 14:49 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, David Hunt

This patch adds a fifo channel to the vm_power_manager app through which
we can send commands and polices. Intended for sending JSON strings.
The fifo is at /tmp/powermonitor/fifo.0

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/channel_manager.c | 108 +++++++++++++++
 examples/vm_power_manager/channel_manager.h |  17 ++-
 examples/vm_power_manager/channel_monitor.c | 146 +++++++++++++++-----
 examples/vm_power_manager/main.c            |   2 +
 4 files changed, 238 insertions(+), 35 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.c b/examples/vm_power_manager/channel_manager.c
index 2e471d0c1..88ae108a6 100644
--- a/examples/vm_power_manager/channel_manager.c
+++ b/examples/vm_power_manager/channel_manager.c
@@ -13,6 +13,7 @@
 
 #include <sys/queue.h>
 #include <sys/types.h>
+#include <sys/stat.h>
 #include <sys/socket.h>
 #include <sys/select.h>
 
@@ -284,6 +285,38 @@ open_non_blocking_channel(struct channel_info *info)
 	return 0;
 }
 
+static int
+open_host_channel(struct channel_info *info)
+{
+	int flags;
+
+	info->fd = open(info->channel_path, O_RDWR | O_RSYNC);
+	if (info->fd == -1) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Error(%s) opening fifo for '%s'\n",
+				strerror(errno),
+				info->channel_path);
+		return -1;
+	}
+
+	/* Get current flags */
+	flags = fcntl(info->fd, F_GETFL, 0);
+	if (flags < 0) {
+		RTE_LOG(WARNING, CHANNEL_MANAGER, "Error(%s) fcntl get flags socket for"
+				"'%s'\n", strerror(errno), info->channel_path);
+		return 1;
+	}
+	/* Set to Non Blocking */
+	flags |= O_NONBLOCK;
+	if (fcntl(info->fd, F_SETFL, flags) < 0) {
+		RTE_LOG(WARNING, CHANNEL_MANAGER,
+				"Error(%s) setting non-blocking "
+				"socket for '%s'\n",
+				strerror(errno), info->channel_path);
+		return -1;
+	}
+	return 0;
+}
+
 static int
 setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 		struct channel_info **chan_info_dptr, unsigned channel_num)
@@ -294,6 +327,7 @@ setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 	chan_info->channel_num = channel_num;
 	chan_info->priv_info = (void *)vm_info;
 	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
+	chan_info->type = CHANNEL_TYPE_BINARY;
 	if (open_non_blocking_channel(chan_info) < 0) {
 		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open channel: "
 				"'%s' for VM '%s'\n",
@@ -316,6 +350,35 @@ setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 	return 0;
 }
 
+static int
+setup_host_channel_info(struct channel_info **chan_info_dptr,
+		unsigned int channel_num)
+{
+	struct channel_info *chan_info = *chan_info_dptr;
+
+	chan_info->channel_num = channel_num;
+	chan_info->priv_info = (void *)0;
+	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
+	chan_info->type = CHANNEL_TYPE_JSON;
+	sprintf(chan_info->channel_path, "%sfifo.0", CHANNEL_MGR_SOCKET_PATH);
+
+	if (open_host_channel(chan_info) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open host channel: "
+				"'%s'\n",
+				chan_info->channel_path);
+		return -1;
+	}
+	if (add_channel_to_monitor(&chan_info) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Could add channel: "
+				"'%s' to epoll ctl\n",
+				chan_info->channel_path);
+		return -1;
+
+	}
+	chan_info->status = CHANNEL_MGR_CHANNEL_CONNECTED;
+	return 0;
+}
+
 int
 add_all_channels(const char *vm_name)
 {
@@ -470,6 +533,51 @@ add_channels(const char *vm_name, unsigned *channel_list,
 	return num_channels_enabled;
 }
 
+int
+add_host_channel(void)
+{
+	struct channel_info *chan_info;
+	char socket_path[PATH_MAX];
+	int num_channels_enabled = 0;
+	int ret;
+
+	snprintf(socket_path, sizeof(socket_path), "%sfifo.%u",
+			CHANNEL_MGR_SOCKET_PATH, 0);
+
+	errno = 0;
+	ret = mkfifo(socket_path, 0666);
+	if ((errno != EEXIST) && (ret < 0)) {
+		printf(" %d %d, %d\n", ret, EEXIST, errno);
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Cannot create fifo '%s' error: "
+				"%s\n", socket_path, strerror(errno));
+		return 0;
+	}
+
+	errno = 0;
+	if (access(socket_path, F_OK) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Channel path '%s' error: "
+				"%s\n", socket_path, strerror(errno));
+		return 0;
+	}
+	chan_info = rte_malloc(NULL, sizeof(*chan_info),
+			RTE_CACHE_LINE_SIZE);
+	if (chan_info == NULL) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for "
+				"channel '%s'\n", socket_path);
+		return 0;
+	}
+	snprintf(chan_info->channel_path,
+			sizeof(chan_info->channel_path), "%sfifo.%u",
+			CHANNEL_MGR_SOCKET_PATH, 0);
+	if (setup_host_channel_info(&chan_info, 0) < 0) {
+		rte_free(chan_info);
+		return 0;
+	}
+	num_channels_enabled++;
+
+	return num_channels_enabled;
+}
+
 int
 remove_channel(struct channel_info **chan_info_dptr)
 {
diff --git a/examples/vm_power_manager/channel_manager.h b/examples/vm_power_manager/channel_manager.h
index 872ec6140..c157cc22b 100644
--- a/examples/vm_power_manager/channel_manager.h
+++ b/examples/vm_power_manager/channel_manager.h
@@ -37,7 +37,7 @@ struct sockaddr_un _sockaddr_un;
 #define UNIX_PATH_MAX sizeof(_sockaddr_un.sun_path)
 #endif
 
-#define MAX_VMS 4
+#define MAX_VMS 64
 #define MAX_VCPUS 20
 
 
@@ -54,6 +54,11 @@ enum channel_status { CHANNEL_MGR_CHANNEL_DISCONNECTED = 0,
 	CHANNEL_MGR_CHANNEL_DISABLED,
 	CHANNEL_MGR_CHANNEL_PROCESSING};
 
+/* Communication Channel Type */
+enum channel_type { CHANNEL_TYPE_BINARY = 0,
+	CHANNEL_TYPE_INI,
+	CHANNEL_TYPE_JSON};
+
 /* VM libvirt(qemu/KVM) connection status */
 enum vm_status { CHANNEL_MGR_VM_INACTIVE = 0, CHANNEL_MGR_VM_ACTIVE};
 
@@ -66,6 +71,7 @@ struct channel_info {
 	volatile uint32_t status;    /**< Connection status(enum channel_status) */
 	int fd;                      /**< AF_UNIX socket fd */
 	unsigned channel_num;        /**< CHANNEL_MGR_SOCKET_PATH/<vm_name>.channel_num */
+	enum channel_type type;      /**< Binary, ini, json, etc. */
 	void *priv_info;             /**< Pointer to private info, do not modify */
 };
 
@@ -226,6 +232,15 @@ int add_all_channels(const char *vm_name);
 int add_channels(const char *vm_name, unsigned *channel_list,
 		unsigned num_channels);
 
+/**
+ * Set up a fifo by which host applications can send command an policies
+ * through a fifo to the vm_power_manager
+ *
+ * @return
+ *  - 0 for success
+ */
+int add_host_channel(void);
+
 /**
  * Remove a channel definition from the channel manager. This must only be
  * called from the channel monitor thread.
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index f180d74e6..17383e9d2 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -85,6 +85,33 @@ core_share_status(int pNo)
 	}
 }
 
+
+static int
+pcpu_monitor(struct policy *pol, struct core_info *ci, int pcpu, int count)
+{
+	int ret = 0;
+
+	if (pol->pkt.policy_to_use == BRANCH_RATIO) {
+		ci->cd[pcpu].oob_enabled = 1;
+		ret = add_core_to_monitor(pcpu);
+		if (ret == 0)
+			RTE_LOG(INFO, CHANNEL_MONITOR,
+					"Monitoring pcpu %d OOB for %s\n",
+					pcpu, pol->pkt.vm_name);
+		else
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Error monitoring pcpu %d OOB for %s\n",
+					pcpu, pol->pkt.vm_name);
+
+	} else {
+		pol->core_share[count].pcpu = pcpu;
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				"Monitoring pcpu %d for %s\n",
+				pcpu, pol->pkt.vm_name);
+	}
+	return ret;
+}
+
 static void
 get_pcpu_to_control(struct policy *pol)
 {
@@ -94,34 +121,46 @@ get_pcpu_to_control(struct policy *pol)
 	int pcpu, count;
 	uint64_t mask_u64b;
 	struct core_info *ci;
-	int ret;
 
 	ci = get_core_info();
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Looking for pcpu for %s\n",
-			pol->pkt.vm_name);
-	get_info_vm(pol->pkt.vm_name, &info);
-
-	for (count = 0; count < pol->pkt.num_vcpu; count++) {
-		mask_u64b = info.pcpu_mask[pol->pkt.vcpu_to_control[count]];
-		for (pcpu = 0; mask_u64b; mask_u64b &= ~(1ULL << pcpu++)) {
-			if ((mask_u64b >> pcpu) & 1) {
-				if (pol->pkt.policy_to_use == BRANCH_RATIO) {
-					ci->cd[pcpu].oob_enabled = 1;
-					ret = add_core_to_monitor(pcpu);
-					if (ret == 0)
-						printf("Monitoring pcpu %d via Branch Ratio\n",
-								pcpu);
-					else
-						printf("Failed to start OOB Monitoring pcpu %d\n",
-								pcpu);
-
-				} else {
-					pol->core_share[count].pcpu = pcpu;
-					printf("Monitoring pcpu %d\n", pcpu);
-				}
+	RTE_LOG(INFO, CHANNEL_MONITOR,
+			"Looking for pcpu for %s\n", pol->pkt.vm_name);
+
+	/*
+	 * So now that we're handling virtual and physical cores, we need to
+	 * differenciate between them when adding them to the branch monitor.
+	 * Virtual cores need to be converted to physical cores.
+	 */
+
+
+
+
+	if (pol->pkt.core_type == CORE_TYPE_VIRTUAL) {
+		/*
+		 * If the cores in the policy are virtual, we need to map them
+		 * to physical core. We look up the vm info and use that for
+		 * the mapping.
+		 */
+		get_info_vm(pol->pkt.vm_name, &info);
+		for (count = 0; count < pol->pkt.num_vcpu; count++) {
+			mask_u64b =
+				info.pcpu_mask[pol->pkt.vcpu_to_control[count]];
+			for (pcpu = 0; mask_u64b;
+					mask_u64b &= ~(1ULL << pcpu++)) {
+				if ((mask_u64b >> pcpu) & 1)
+					pcpu_monitor(pol, ci, pcpu, count);
 			}
 		}
+	} else {
+		/*
+		 * If the cores in the policy are physical, we just use
+		 * those core id's directly.
+		 */
+		for (count = 0; count < pol->pkt.num_vcpu; count++) {
+			pcpu = pol->pkt.vcpu_to_control[count];
+			pcpu_monitor(pol, ci, pcpu, count);
+		}
 	}
 }
 
@@ -160,8 +199,13 @@ update_policy(struct channel_packet *pkt)
 	unsigned int updated = 0;
 	int i;
 
+
+	RTE_LOG(INFO, CHANNEL_MONITOR,
+			"Applying policy for %s\n", pkt->vm_name);
+
 	for (i = 0; i < MAX_VMS; i++) {
 		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
+			/* Copy the contents of *pkt into the policy.pkt */
 			policies[i].pkt = *pkt;
 			get_pcpu_to_control(&policies[i]);
 			if (get_pfid(&policies[i]) == -1) {
@@ -189,6 +233,24 @@ update_policy(struct channel_packet *pkt)
 	return 0;
 }
 
+static int
+remove_policy(struct channel_packet *pkt __rte_unused)
+{
+	int i;
+
+	/*
+	 * Disabling the policy is simply a case of setting
+	 * enabled to 0
+	 */
+	for (i = 0; i < MAX_VMS; i++) {
+		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
+			policies[i].enabled = 0;
+			return 0;
+		}
+	}
+	return -1;
+}
+
 static uint64_t
 get_pkt_diff(struct policy *pol)
 {
@@ -346,7 +408,6 @@ apply_policy(struct policy *pol)
 		apply_workload_profile(pol);
 }
 
-
 static int
 process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 {
@@ -355,6 +416,8 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (chan_info == NULL)
 		return -1;
 
+	RTE_LOG(INFO, CHANNEL_MONITOR, "Processing Request %s\n", pkt->vm_name);
+
 	if (rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_CONNECTED,
 			CHANNEL_MGR_CHANNEL_PROCESSING) == 0)
 		return -1;
@@ -362,10 +425,12 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (pkt->command == CPU_POWER) {
 		core_mask = get_pcpus_mask(chan_info, pkt->resource_id);
 		if (core_mask == 0) {
-			RTE_LOG(ERR, CHANNEL_MONITOR, "Error get physical CPU mask for "
-				"channel '%s' using vCPU(%u)\n", chan_info->channel_path,
-				(unsigned)pkt->unit);
-			return -1;
+			/*
+			 * Core mask will be 0 in the case where
+			 * hypervisor is not available so we're working in
+			 * the host, so use the core as the mask.
+			 */
+			core_mask = 1 << pkt->resource_id;
 		}
 		if (__builtin_popcountll(core_mask) == 1) {
 
@@ -421,12 +486,20 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	}
 
 	if (pkt->command == PKT_POLICY) {
-		RTE_LOG(INFO, CHANNEL_MONITOR, "\nProcessing Policy request from Guest\n");
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				"\nProcessing Policy request\n");
 		update_policy(pkt);
 		policy_is_set = 1;
 	}
 
-	/* Return is not checked as channel status may have been set to DISABLED
+	if (pkt->command == PKT_POLICY_REMOVE) {
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				 "Removing policy %s\n", pkt->vm_name);
+		remove_policy(pkt);
+	}
+
+	/*
+	 * Return is not checked as channel status may have been set to DISABLED
 	 * from management thread
 	 */
 	rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_PROCESSING,
@@ -448,13 +521,16 @@ add_channel_to_monitor(struct channel_info **chan_info)
 				"to epoll\n", info->channel_path);
 		return -1;
 	}
+	RTE_LOG(ERR, CHANNEL_MONITOR, "Added channel '%s' "
+			"to monitor\n", info->channel_path);
 	return 0;
 }
 
 int
 remove_channel_from_monitor(struct channel_info *chan_info)
 {
-	if (epoll_ctl(global_event_fd, EPOLL_CTL_DEL, chan_info->fd, NULL) < 0) {
+	if (epoll_ctl(global_event_fd, EPOLL_CTL_DEL,
+			chan_info->fd, NULL) < 0) {
 		RTE_LOG(ERR, CHANNEL_MONITOR, "Unable to remove channel '%s' "
 				"from epoll\n", chan_info->channel_path);
 		return -1;
@@ -467,11 +543,13 @@ channel_monitor_init(void)
 {
 	global_event_fd = epoll_create1(0);
 	if (global_event_fd == 0) {
-		RTE_LOG(ERR, CHANNEL_MONITOR, "Error creating epoll context with "
-				"error %s\n", strerror(errno));
+		RTE_LOG(ERR, CHANNEL_MONITOR,
+				"Error creating epoll context with error %s\n",
+				strerror(errno));
 		return -1;
 	}
-	global_events_list = rte_malloc("epoll_events", sizeof(*global_events_list)
+	global_events_list = rte_malloc("epoll_events",
+			sizeof(*global_events_list)
 			* MAX_EVENTS, RTE_CACHE_LINE_SIZE);
 	if (global_events_list == NULL) {
 		RTE_LOG(ERR, CHANNEL_MONITOR, "Unable to rte_malloc for "
diff --git a/examples/vm_power_manager/main.c b/examples/vm_power_manager/main.c
index 58c5fa45c..893bf4cdd 100644
--- a/examples/vm_power_manager/main.c
+++ b/examples/vm_power_manager/main.c
@@ -421,6 +421,8 @@ main(int argc, char **argv)
 		return -1;
 	}
 
+	add_host_channel();
+
 	printf("Running core monitor on lcore id %d\n", lcore_id);
 	rte_eal_remote_launch(run_core_monitor, NULL, lcore_id);
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v2 5/7] examples/power: add json string handling
  2018-09-12 14:49 ` [dpdk-dev] [PATCH v2 0/7] add json power policy interface for containers David Hunt
                     ` (3 preceding siblings ...)
  2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 4/7] examples/power: add host channel to power manager David Hunt
@ 2018-09-12 14:49   ` David Hunt
  2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 6/7] doc/vm_power_manager: add JSON interface API info David Hunt
                     ` (2 subsequent siblings)
  7 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-12 14:49 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, David Hunt

Add JSON string handling to vm_power_manager for JSON strings received
through the fifo. The format of the JSON strings are detailed in the
next patch, the vm_power_manager user guide documentation updates.

This patch introduces a new dependency on Jansson, a C library for
encoding, decoding and manipulating JSON data. To compile the sample app
you now need to have installed libjansson4 and libjansson-dev (these may
be named slightly differently depending on your Operating System)

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/Makefile          |   6 +
 examples/vm_power_manager/channel_monitor.c | 297 +++++++++++++++++---
 2 files changed, 258 insertions(+), 45 deletions(-)

diff --git a/examples/vm_power_manager/Makefile b/examples/vm_power_manager/Makefile
index 13a5205ba..50147c05d 100644
--- a/examples/vm_power_manager/Makefile
+++ b/examples/vm_power_manager/Makefile
@@ -31,6 +31,12 @@ CFLAGS += $(WERROR_FLAGS)
 
 LDLIBS += -lvirt
 
+JANSSON := $(shell pkg-config --exists jansson; echo $$?)
+ifeq ($(JANSSON), 0)
+LDLIBS += $(shell pkg-config --libs jansson)
+CFLAGS += -DUSE_JANSSON
+endif
+
 ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
 
 ifeq ($(CONFIG_RTE_LIBRTE_IXGBE_PMD),y)
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index 17383e9d2..655f5e279 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -9,11 +9,18 @@
 #include <signal.h>
 #include <errno.h>
 #include <string.h>
+#include <fcntl.h>
 #include <sys/types.h>
 #include <sys/epoll.h>
 #include <sys/queue.h>
 #include <sys/time.h>
-
+#include <sys/socket.h>
+#include <sys/select.h>
+#ifdef USE_JANSSON
+#include <jansson.h>
+#else
+#pragma message "Jansson dev libs unavailable, not including JSON parsing"
+#endif
 #include <rte_log.h>
 #include <rte_memory.h>
 #include <rte_malloc.h>
@@ -35,6 +42,8 @@
 
 uint64_t vsi_pkt_count_prev[384];
 uint64_t rdtsc_prev[384];
+#define MAX_JSON_STRING_LEN 1024
+char json_data[MAX_JSON_STRING_LEN];
 
 double time_period_ms = 1;
 static volatile unsigned run_loop = 1;
@@ -43,6 +52,132 @@ static unsigned int policy_is_set;
 static struct epoll_event *global_events_list;
 static struct policy policies[MAX_VMS];
 
+#ifdef USE_JANSSON
+static int
+parse_json_to_pkt(json_t *element, struct channel_packet *pkt)
+{
+	const char *key;
+	json_t *value;
+	int ret;
+
+	memset(pkt, 0, sizeof(struct channel_packet));
+
+	pkt->nb_mac_to_monitor = 0;
+	pkt->t_boost_status.tbEnabled = false;
+	pkt->workload = LOW;
+	pkt->policy_to_use = TIME;
+	pkt->command = PKT_POLICY;
+	pkt->core_type = CORE_TYPE_PHYSICAL;
+
+	json_object_foreach(element, key, value) {
+		if (!strcmp(key, "policy")) {
+			/* Recurse in to get the contents of profile */
+			ret = parse_json_to_pkt(value, pkt);
+			if (ret)
+				return ret;
+		} else if (!strcmp(key, "instruction")) {
+			/* Recurse in to get the contents of instruction */
+			ret = parse_json_to_pkt(value, pkt);
+			if (ret)
+				return ret;
+		} else if (!strcmp(key, "name")) {
+			strcpy(pkt->vm_name, json_string_value(value));
+		} else if (!strcmp(key, "command")) {
+			char command[32];
+			snprintf(command, 32, "%s", json_string_value(value));
+			if (!strcmp(command, "power")) {
+				pkt->command = CPU_POWER;
+			} else if (!strcmp(command, "create")) {
+				pkt->command = PKT_POLICY;
+			} else if (!strcmp(command, "destroy")) {
+				pkt->command = PKT_POLICY_REMOVE;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Invalid command received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "policy_type")) {
+			char command[32];
+			snprintf(command, 32, "%s", json_string_value(value));
+			if (!strcmp(command, "TIME")) {
+				pkt->policy_to_use = TIME;
+			} else if (!strcmp(command, "TRAFFIC")) {
+				pkt->policy_to_use = TRAFFIC;
+			} else if (!strcmp(command, "WORKLOAD")) {
+				pkt->policy_to_use = WORKLOAD;
+			} else if (!strcmp(command, "BRANCH_RATIO")) {
+				pkt->policy_to_use = BRANCH_RATIO;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Wrong policy_type received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "busy_hours")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int hour = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->timer_policy.busy_hours[i] = hour;
+			}
+		} else if (!strcmp(key, "quiet_hours")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int hour = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->timer_policy.quiet_hours[i] = hour;
+			}
+		} else if (!strcmp(key, "core_list")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int core = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->vcpu_to_control[i] = core;
+			}
+			pkt->num_vcpu = size;
+		} else if (!strcmp(key, "avg_packet_thresh")) {
+			pkt->traffic_policy.avg_max_packet_thresh =
+					(uint32_t)json_integer_value(value);
+		} else if (!strcmp(key, "max_packet_thresh")) {
+			pkt->traffic_policy.max_max_packet_thresh =
+					(uint32_t)json_integer_value(value);
+		} else if (!strcmp(key, "unit")) {
+			char unit[32];
+			snprintf(unit, 32, "%s", json_string_value(value));
+			if (!strcmp(unit, "SCALE_UP")) {
+				pkt->unit = CPU_POWER_SCALE_UP;
+			} else if (!strcmp(unit, "SCALE_DOWN")) {
+				pkt->unit = CPU_POWER_SCALE_DOWN;
+			} else if (!strcmp(unit, "SCALE_MAX")) {
+				pkt->unit = CPU_POWER_SCALE_MAX;
+			} else if (!strcmp(unit, "SCALE_MIN")) {
+				pkt->unit = CPU_POWER_SCALE_MIN;
+			} else if (!strcmp(unit, "ENABLE_TURBO")) {
+				pkt->unit = CPU_POWER_ENABLE_TURBO;
+			} else if (!strcmp(unit, "DISABLE_TURBO")) {
+				pkt->unit = CPU_POWER_DISABLE_TURBO;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Invalid command received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "resource_id")) {
+			pkt->resource_id = (uint32_t)json_integer_value(value);
+		} else {
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+				"Unknown key received in JSON string: %s\n",
+				key);
+		}
+	}
+	return 0;
+}
+#endif
+
 void channel_monitor_exit(void)
 {
 	run_loop = 0;
@@ -124,18 +259,11 @@ get_pcpu_to_control(struct policy *pol)
 
 	ci = get_core_info();
 
-	RTE_LOG(INFO, CHANNEL_MONITOR,
-			"Looking for pcpu for %s\n", pol->pkt.vm_name);
-
 	/*
 	 * So now that we're handling virtual and physical cores, we need to
 	 * differenciate between them when adding them to the branch monitor.
 	 * Virtual cores need to be converted to physical cores.
 	 */
-
-
-
-
 	if (pol->pkt.core_type == CORE_TYPE_VIRTUAL) {
 		/*
 		 * If the cores in the policy are virtual, we need to map them
@@ -295,8 +423,6 @@ apply_traffic_profile(struct policy *pol)
 
 	diff = get_pkt_diff(pol);
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Applying traffic profile\n");
-
 	if (diff >= (pol->pkt.traffic_policy.max_max_packet_thresh)) {
 		for (count = 0; count < pol->pkt.num_vcpu; count++) {
 			if (pol->core_share[count].status != 1)
@@ -340,9 +466,6 @@ apply_time_profile(struct policy *pol)
 				if (pol->core_share[count].status != 1) {
 					power_manager_scale_core_max(
 						pol->core_share[count].pcpu);
-				RTE_LOG(INFO, CHANNEL_MONITOR,
-					"Scaling up core %d to max\n",
-					pol->core_share[count].pcpu);
 				}
 			}
 			break;
@@ -352,9 +475,6 @@ apply_time_profile(struct policy *pol)
 				if (pol->core_share[count].status != 1) {
 					power_manager_scale_core_min(
 						pol->core_share[count].pcpu);
-				RTE_LOG(INFO, CHANNEL_MONITOR,
-					"Scaling down core %d to min\n",
-					pol->core_share[count].pcpu);
 			}
 		}
 			break;
@@ -416,8 +536,6 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (chan_info == NULL)
 		return -1;
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Processing Request %s\n", pkt->vm_name);
-
 	if (rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_CONNECTED,
 			CHANNEL_MGR_CHANNEL_PROCESSING) == 0)
 		return -1;
@@ -486,8 +604,8 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	}
 
 	if (pkt->command == PKT_POLICY) {
-		RTE_LOG(INFO, CHANNEL_MONITOR,
-				"\nProcessing Policy request\n");
+		RTE_LOG(INFO, CHANNEL_MONITOR, "Processing policy request %s\n",
+				pkt->vm_name);
 		update_policy(pkt);
 		policy_is_set = 1;
 	}
@@ -559,6 +677,103 @@ channel_monitor_init(void)
 	return 0;
 }
 
+static void
+read_binary_packet(struct channel_info *chan_info)
+{
+	struct channel_packet pkt;
+	void *buffer = &pkt;
+	int buffer_len = sizeof(pkt);
+	int n_bytes, err = 0;
+
+	while (buffer_len > 0) {
+		n_bytes = read(chan_info->fd,
+				buffer, buffer_len);
+		if (n_bytes == buffer_len)
+			break;
+		if (n_bytes == -1) {
+			err = errno;
+			RTE_LOG(DEBUG, CHANNEL_MONITOR,
+				"Received error on "
+				"channel '%s' read: %s\n",
+				chan_info->channel_path,
+				strerror(err));
+			remove_channel(&chan_info);
+			break;
+		}
+		buffer = (char *)buffer + n_bytes;
+		buffer_len -= n_bytes;
+	}
+	if (!err)
+		process_request(&pkt, chan_info);
+}
+
+#ifdef USE_JANSSON
+static void
+read_json_packet(struct channel_info *chan_info)
+{
+	struct channel_packet pkt;
+	int n_bytes, ret;
+	json_t *root;
+	json_error_t error;
+
+	/* read opening brace to closing brace */
+	do {
+		int idx = 0;
+		int indent = 0;
+		do {
+			n_bytes = read(chan_info->fd, &json_data[idx], 1);
+			if (n_bytes == 0)
+				break;
+			if (json_data[idx] == '{')
+				indent++;
+			if (json_data[idx] == '}')
+				indent--;
+			if ((indent > 0) || (idx >> 0))
+				idx++;
+			if (indent == 0)
+				json_data[idx] = 0;
+			if (idx >= MAX_JSON_STRING_LEN)
+				break;
+		} while (indent > 0);
+
+		if (indent > 0)
+			/*
+			 * We've broken out of the read loop without getting
+			 * a closing brace, so throw away the datai
+			 */
+			json_data[idx] = 0;
+
+		if (strlen(json_data) == 0)
+			continue;
+
+		printf("got [%s]\n", json_data);
+
+		root = json_loads(json_data, 0, &error);
+
+		if (root) {
+			/*
+			 * Because our data is now in the json
+			 * object, we can overwrite the pkt
+			 * with a channel_packet struct, using
+			 * parse_json_to_pkt()
+			 */
+			ret = parse_json_to_pkt(root, &pkt);
+			json_decref(root);
+			if (ret) {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Error validating JSON profile data\n");
+				break;
+			}
+			process_request(&pkt, chan_info);
+		} else {
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+					"JSON error on line %d: %s\n",
+					error.line, error.text);
+		}
+	} while (n_bytes > 0);
+}
+#endif
+
 void
 run_channel_monitor(void)
 {
@@ -570,11 +785,16 @@ run_channel_monitor(void)
 		if (!run_loop)
 			break;
 		for (i = 0; i < n_events; i++) {
+			if (!global_events_list[i].data.ptr) {
+				fflush(stdout);
+				continue;
+			}
 			struct channel_info *chan_info = (struct channel_info *)
 					global_events_list[i].data.ptr;
 			if ((global_events_list[i].events & EPOLLERR) ||
 				(global_events_list[i].events & EPOLLHUP)) {
-				RTE_LOG(DEBUG, CHANNEL_MONITOR, "Remote closed connection for "
+				RTE_LOG(INFO, CHANNEL_MONITOR,
+						"Remote closed connection for "
 						"channel '%s'\n",
 						chan_info->channel_path);
 				remove_channel(&chan_info);
@@ -582,31 +802,18 @@ run_channel_monitor(void)
 			}
 			if (global_events_list[i].events & EPOLLIN) {
 
-				int n_bytes, err = 0;
-				struct channel_packet pkt;
-				void *buffer = &pkt;
-				int buffer_len = sizeof(pkt);
-
-				while (buffer_len > 0) {
-					n_bytes = read(chan_info->fd,
-							buffer, buffer_len);
-					if (n_bytes == buffer_len)
-						break;
-					if (n_bytes == -1) {
-						err = errno;
-						RTE_LOG(DEBUG, CHANNEL_MONITOR,
-							"Received error on "
-							"channel '%s' read: %s\n",
-							chan_info->channel_path,
-							strerror(err));
-						remove_channel(&chan_info);
-						break;
-					}
-					buffer = (char *)buffer + n_bytes;
-					buffer_len -= n_bytes;
+				switch (chan_info->type) {
+				case CHANNEL_TYPE_BINARY:
+					read_binary_packet(chan_info);
+					break;
+#ifdef USE_JANSSON
+				case CHANNEL_TYPE_JSON:
+					read_json_packet(chan_info);
+					break;
+#endif
+				default:
+					break;
 				}
-				if (!err)
-					process_request(&pkt, chan_info);
 			}
 		}
 		rte_delay_us(time_period_ms*1000);
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v2 6/7] doc/vm_power_manager: add JSON interface API info
  2018-09-12 14:49 ` [dpdk-dev] [PATCH v2 0/7] add json power policy interface for containers David Hunt
                     ` (4 preceding siblings ...)
  2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 5/7] examples/power: add json string handling David Hunt
@ 2018-09-12 14:49   ` David Hunt
  2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 7/7] examples/power: add json example files David Hunt
  2018-09-14 13:53   ` [dpdk-dev] [PATCH v3 0/8] add json power policy interface for containers David Hunt
  7 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-12 14:49 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, David Hunt

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 .../sample_app_ug/vm_power_management.rst     | 236 ++++++++++++++++++
 1 file changed, 236 insertions(+)

diff --git a/doc/guides/sample_app_ug/vm_power_management.rst b/doc/guides/sample_app_ug/vm_power_management.rst
index 855570d6b..b5b0179d2 100644
--- a/doc/guides/sample_app_ug/vm_power_management.rst
+++ b/doc/guides/sample_app_ug/vm_power_management.rst
@@ -337,6 +337,242 @@ monitoring of branch ratio on cores doing busy polling via PMDs.
   and will need to be adjusted for different workloads.
 
 
+
+JSON API
+~~~~~~~~
+
+In addition to the command line interface for host command and a virtio-serial
+interface for VM power policies, there is also a JSON interface through which
+power commands and policies can be sent. This functionality adds a dependency
+on the Jansson library, and the Jansson development package must be installed
+on the system before the JSON parsing functionality is included in the app.
+This is achieved by:
+
+  .. code-block:: console
+
+    apt-get install libjansson-dev
+
+The command and package name may be different depending on your operating
+system. It's worth noting that the app will successfully build without this
+package present, but a warning is shown during compilation, and the JSON
+parsing functionality will not be present in the app.
+
+Sending a command or policy to the power manager application is achieved by
+simply opening a fifo file, writing a JSON string to that fifo, and closing
+the file.
+
+The fifo is at /tmp/powermonitor/fifo.0
+
+The jason string can be a policy or instruction, and takes the following
+format:
+
+  .. code-block:: console
+
+    {"packet_type": {
+      "pair_1": value,
+      "pair_2": value
+    }}
+
+The 'packet_type' header can contain one of two values, depending on
+whether a policy or power command is being sent. The two possible values are
+"policy" and "instruction", and the expected name-value pairs is different
+depending on which type is being sent.
+
+The pairs are the format of standard JSON name-value pairs. The value type
+varies between the different name/value pairs, and may be intgers, strings,
+arrays, etc. Examples of policies follow later in this document. The allowed
+names and value types are as follows:
+
+
+:Pair Name: "name"
+:Description: Name of the VM or Host. Allows the parser to associate the
+  policy with the relevant VM or Host OS.
+:Type: string
+:Values: any valid string
+:Required: yes
+:Example:
+
+  .. code-block:: console
+
+    ""name", "ubuntu2"
+
+
+:Pair Name: "command"
+:Description: The type of packet we're sending to the power manager. We can be
+  creating or destroying a policy, or sending a direct command to adjust
+  the frequency of a core, similar to the command line interface.
+:Type: string
+:Values:
+
+  :"CREATE": used when creating a new policy,
+  :"DESTROY": used when removing a policy,
+  :"POWER": used when sending an immediate command, max, min, etc.
+:Required: yes
+:Example:
+
+    .. code-block:: console
+
+      "command", "CREATE"
+
+
+:Pair Name: "policy_type"
+:Description: Type of policy to apply. Please see vm_power_manager documentation
+  for more information on the types of policies that may be used.
+:Type: string
+:Values:
+
+  :"TIME": Time-of-day policy. Frequencies of the relevant cores are
+    scaled up/down depending on busy and quiet hours.
+  :"TRAFFIC": This policy takes statistics from the NIC and scales up
+    and down accordingly.
+  :"WORKLOAD": This policy looks at how heavily loaded the cores are,
+    and scales up and down accordingly.
+  :"BRANCH_RATIO": This out-of-band policy can look at the ratio between
+    branch hits and misses on a core, and is useful for detecting
+    how much packet processing a core is doing.
+:Required: only for CREATE/DESTROY command
+:Example:
+
+  .. code-block:: console
+
+    "policy_type", "TIME"
+
+:Pair Name: "busy_hours"
+:Description: The hours of the day in which we scale up the cores for busy
+  times.
+:Type: array of integers
+:Values: array with list of hour numbers, (0-23)
+:Required: only for TIME policy
+:Example:
+
+  .. code-block:: console
+
+    "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ]
+
+:Pair Name: "quiet_hours"
+:Description: The hours of the day in which we scale down the cores for quiet
+  times.
+:Type: array of integers
+:Values: array with list of hour numbers, (0-23)
+:Required: only for TIME policy
+:Example:
+
+  .. code-block:: console
+
+    "quiet_hours":[ 2, 3, 4, 5, 6 ]
+
+:Pair Name: "avg_packet_thresh"
+:Description: Threshold below which the frequency will be set to min for
+  the TRAFFIC policy. If the traffic rate is above this and below max, the
+  frequency will be set to medium.
+:Type: integer
+:Values: The number of packets below which the TRAFFIC policy applies the
+  minimum frequency, or medium frequency if between avg and max thresholds.
+:Required: only for TRAFFIC policy
+:Example:
+
+  .. code-block:: console
+
+    "avg_packet_thresh": 100000
+
+:Pair Name: "max_packet_thresh"
+:Description: Threshold above which the frequency will be set to max for
+  the TRAFFIC policy
+:Type: integer
+:Values: The number of packets per interval above which the TRAFFIC policy
+  applies the maximum frequency
+:Required: only for TRAFFIC policy
+:Example:
+
+  .. code-block:: console
+
+    "max_packet_thresh": 500000
+
+:Pair Name: "core_list"
+:Description: The cores to which to apply the policy.
+:Type: array of integers
+:Values: array with list of virtual CPUs.
+:Required: only policy CREATE/DESTROY
+:Example:
+
+  .. code-block:: console
+
+    "core_list":[ 10, 11 ]
+
+:Pair Name: "unit"
+:Description: the type of power operation to apply in the command
+:Type: string
+:Values:
+
+  :"SCALE_MAX": Scale frequency of this core to maximum
+  :"SCALE_MIN": Scale frequency of this core to minimum
+  :"SCALE_UP": Scale up frequency of this core
+  :"SCALE_DOWN": Scale down frequency of this core
+  :"ENABLE_TURBO": Enable Turbo Boost for this core
+  :"DISABLE_TURBO": Disable Turbo Boost for this core
+:Required: only for POWER instruction
+:Example:
+
+  .. code-block:: console
+
+    "unit", "SCALE_MAX"
+
+:Pair Name: "resource_id"
+:Description: The core to which to apply the power command.
+:Type: integer
+:Values: valid core id for VM or host OS.
+:Required: only POWER instruction
+:Example:
+
+  .. code-block:: console
+
+    "resource_id": 10
+
+JSON API Examples
+~~~~~~~~~~~~~~~~~
+
+Profile create example:
+
+  .. code-block:: console
+
+    {"policy": {
+      "name": "ubuntu",
+      "command": "create",
+      "policy_type": "TIME",
+      "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
+      "quiet_hours":[ 2, 3, 4, 5, 6 ],
+      "core_list":[ 11 ]
+    }}
+
+Profile destroy example:
+
+  .. code-block:: console
+
+    {"profile": {
+      "name": "ubuntu",
+      "command": "destroy",
+    }}
+
+Power command example:
+
+  .. code-block:: console
+
+    {"command": {
+      "name": "ubuntu",
+      "unit": "SCALE_MAX",
+      "resource_id": 10
+    }}
+
+To send a JSON string to the Power Manager application, simply paste the
+example JSON string into a text file and cat it into the fifo:
+
+  .. code-block:: console
+
+    cat file.json >/tmp/powermonitor/fifo.0
+
+The console of the Power Manager application should indicate the command that
+was just received via the fifo.
+
 Compiling and Running the Guest Applications
 --------------------------------------------
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v2 7/7] examples/power: add json example files
  2018-09-12 14:49 ` [dpdk-dev] [PATCH v2 0/7] add json power policy interface for containers David Hunt
                     ` (5 preceding siblings ...)
  2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 6/7] doc/vm_power_manager: add JSON interface API info David Hunt
@ 2018-09-12 14:49   ` David Hunt
  2018-09-14 13:53   ` [dpdk-dev] [PATCH v3 0/8] add json power policy interface for containers David Hunt
  7 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-12 14:49 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, David Hunt

This patch provides some example files in the json_examples sub-directory
for sending to the fifo.

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/json_examples/README            | 6 ++++++
 examples/vm_power_manager/json_examples/create.json       | 8 ++++++++
 examples/vm_power_manager/json_examples/destroy.json      | 4 ++++
 examples/vm_power_manager/json_examples/set_core_max.json | 6 ++++++
 examples/vm_power_manager/json_examples/set_core_min.json | 6 ++++++
 5 files changed, 30 insertions(+)
 create mode 100644 examples/vm_power_manager/json_examples/README
 create mode 100644 examples/vm_power_manager/json_examples/create.json
 create mode 100644 examples/vm_power_manager/json_examples/destroy.json
 create mode 100644 examples/vm_power_manager/json_examples/set_core_max.json
 create mode 100644 examples/vm_power_manager/json_examples/set_core_min.json

diff --git a/examples/vm_power_manager/json_examples/README b/examples/vm_power_manager/json_examples/README
new file mode 100644
index 000000000..a94c6b14b
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/README
@@ -0,0 +1,6 @@
+Sample files for sending to the vm_power_manager through the fifo.
+
+Simply cat the file to /tmp/powermonitor/fifo.0 when the vm_power_manager
+application is running.
+
+E.g. cat create.json >/tmp/powermonitor/fifo.0
diff --git a/examples/vm_power_manager/json_examples/create.json b/examples/vm_power_manager/json_examples/create.json
new file mode 100644
index 000000000..a7133d9a1
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/create.json
@@ -0,0 +1,8 @@
+{"policy": {
+  "name": "policy-1",
+  "command": "create",
+  "policy_type": "TIME",
+  "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
+  "quiet_hours":[ 2, 3, 4, 5, 6 ],
+  "core_list":[ 11, 12 ]
+}}
diff --git a/examples/vm_power_manager/json_examples/destroy.json b/examples/vm_power_manager/json_examples/destroy.json
new file mode 100644
index 000000000..587c9e7e9
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/destroy.json
@@ -0,0 +1,4 @@
+{"policy": {
+  "name": "policy-1",
+  "command": "destroy"
+}}
diff --git a/examples/vm_power_manager/json_examples/set_core_max.json b/examples/vm_power_manager/json_examples/set_core_max.json
new file mode 100644
index 000000000..497030a44
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/set_core_max.json
@@ -0,0 +1,6 @@
+{"instruction": {
+  "name": "set_power",
+  "command": "power",
+  "unit": "SCALE_MAX",
+  "resource_id": 10
+}}
diff --git a/examples/vm_power_manager/json_examples/set_core_min.json b/examples/vm_power_manager/json_examples/set_core_min.json
new file mode 100644
index 000000000..76d934fd8
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/set_core_min.json
@@ -0,0 +1,6 @@
+{"instruction": {
+  "name": "set_power",
+  "command": "power",
+  "unit": "SCALE_MIN",
+  "resource_id": 10
+}}
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v3 0/8] add json power policy interface for containers
  2018-09-12 14:49 ` [dpdk-dev] [PATCH v2 0/7] add json power policy interface for containers David Hunt
                     ` (6 preceding siblings ...)
  2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 7/7] examples/power: add json example files David Hunt
@ 2018-09-14 13:53   ` David Hunt
  2018-09-14 13:53     ` [dpdk-dev] [PATCH v3 1/8] examples/power: add checks around hypervisor David Hunt
                       ` (8 more replies)
  7 siblings, 9 replies; 110+ messages in thread
From: David Hunt @ 2018-09-14 13:53 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson

The current vm_power_manager example app has the capability to accept power
policies from virtual machines via virtio-serial channels. These power
policies  allow a virtual machine to give information to the power manager
to allow the power manager take care of the power management of the virtual
machine based on the information in the policy.

This power policy functionality is limited to virtual machines sending
the policies to the power manager (which runs in the Host OS), and a solution
was needed for additional methods of sending power policies to the power
manager app.

The main use-case for this modification is for containers and host
applications that wish to send polices to the power manager.

This patchset adds the capability to send power polices and power commands
to the vm_power_manager app via JSON strings through a fifo on the file
system.
For example, given the following file, policy.json:

{"policy": {
  "name": "ubuntu2",
  "command": "create",
  "policy_type": "TIME",
  "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
  "quiet_hours":[ 2, 3, 4, 5, 6 ],
  "core_list":[ 11, 12, 13 ]
}}

Then running the command:

cat policy.json >/tmp/powermonitor/fifo.0

The policy is sent to the vm_power_manager. The power manager app then parses
the JSON data, and inserts the policy into the array of policies.

Part of the patch series contains documentation updates to give all the
details of the valid name-value pairs, the data types, etc.

Patch v2:
  * Fixed review comments from Stephen Hemminger and Lei A Yao.
  * Added a check in the Makefile for libjansson-dev. Will Warn user and build
    without JSON functionality if not present, will build including JSON
    functionality if it is present.

Patch v3:
  * Added meson/ninja support for vm_power_manager and guest_cli apps
  * Fixed compilation issue with guest_cli app

[1/8] examples/power: add checks around hypervisor
[2/8] lib/power: add changes for host commands/policies
[3/8] examples/power: add necessary changes to guest app
[4/8] examples/power: add host channel to power manager
[5/8] examples/power: add json string handling
[6/8] examples/power: add meson/ninja build support
[7/8] doc/vm_power_manager: add JSON interface API info
[8/8] examples/power: add json example files

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v3 1/8] examples/power: add checks around hypervisor
  2018-09-14 13:53   ` [dpdk-dev] [PATCH v3 0/8] add json power policy interface for containers David Hunt
@ 2018-09-14 13:53     ` David Hunt
  2018-09-25  9:20       ` Burakov, Anatoly
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 2/8] lib/power: add changes for host commands/policies David Hunt
                       ` (7 subsequent siblings)
  8 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-14 13:53 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson, David Hunt

Allow vm_power_manager to run without requiring qemu to be present
on the machine. This will be required for instances where the JSON
interface is used for commands and polices, without any VMs present.
A use case for this is a container enviromnent.

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/channel_manager.c | 71 +++++++++++++--------
 examples/vm_power_manager/channel_monitor.c |  2 +-
 2 files changed, 44 insertions(+), 29 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.c b/examples/vm_power_manager/channel_manager.c
index 927fc35ab..2e471d0c1 100644
--- a/examples/vm_power_manager/channel_manager.c
+++ b/examples/vm_power_manager/channel_manager.c
@@ -43,7 +43,8 @@ static unsigned char *global_cpumaps;
 static virVcpuInfo *global_vircpuinfo;
 static size_t global_maplen;
 
-static unsigned global_n_host_cpus;
+static unsigned int global_n_host_cpus;
+static bool global_hypervisor_available;
 
 /*
  * Represents a single Virtual Machine
@@ -198,7 +199,11 @@ get_pcpus_mask(struct channel_info *chan_info, unsigned vcpu)
 {
 	struct virtual_machine_info *vm_info =
 			(struct virtual_machine_info *)chan_info->priv_info;
-	return rte_atomic64_read(&vm_info->pcpu_mask[vcpu]);
+
+	if (global_hypervisor_available && (vm_info != NULL))
+		return rte_atomic64_read(&vm_info->pcpu_mask[vcpu]);
+	else
+		return 0;
 }
 
 static inline int
@@ -559,6 +564,8 @@ get_all_vm(int *num_vm, int *num_vcpu)
 				VIR_CONNECT_LIST_DOMAINS_PERSISTENT;
 	unsigned int domain_flag = VIR_DOMAIN_VCPU_CONFIG;
 
+	if (!global_hypervisor_available)
+		return;
 
 	memset(global_cpumaps, 0, CHANNEL_CMDS_MAX_CPUS*global_maplen);
 	if (virNodeGetInfo(global_vir_conn_ptr, &node_info)) {
@@ -768,38 +775,42 @@ connect_hypervisor(const char *path)
 	}
 	return 0;
 }
-
 int
-channel_manager_init(const char *path)
+channel_manager_init(const char *path __rte_unused)
 {
 	virNodeInfo info;
 
 	LIST_INIT(&vm_list_head);
 	if (connect_hypervisor(path) < 0) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to initialize channel manager\n");
-		return -1;
-	}
-
-	global_maplen = VIR_CPU_MAPLEN(CHANNEL_CMDS_MAX_CPUS);
+		global_n_host_cpus = 64;
+		global_hypervisor_available = 0;
+		RTE_LOG(INFO, CHANNEL_MANAGER, "Unable to initialize channel manager\n");
+	} else {
+		global_hypervisor_available = 1;
+
+		global_maplen = VIR_CPU_MAPLEN(CHANNEL_CMDS_MAX_CPUS);
+
+		global_vircpuinfo = rte_zmalloc(NULL,
+				sizeof(*global_vircpuinfo) *
+				CHANNEL_CMDS_MAX_CPUS, RTE_CACHE_LINE_SIZE);
+		if (global_vircpuinfo == NULL) {
+			RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for CPU Info\n");
+			goto error;
+		}
+		global_cpumaps = rte_zmalloc(NULL,
+				CHANNEL_CMDS_MAX_CPUS * global_maplen,
+				RTE_CACHE_LINE_SIZE);
+		if (global_cpumaps == NULL)
+			goto error;
 
-	global_vircpuinfo = rte_zmalloc(NULL, sizeof(*global_vircpuinfo) *
-			CHANNEL_CMDS_MAX_CPUS, RTE_CACHE_LINE_SIZE);
-	if (global_vircpuinfo == NULL) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for CPU Info\n");
-		goto error;
-	}
-	global_cpumaps = rte_zmalloc(NULL, CHANNEL_CMDS_MAX_CPUS * global_maplen,
-			RTE_CACHE_LINE_SIZE);
-	if (global_cpumaps == NULL) {
-		goto error;
+		if (virNodeGetInfo(global_vir_conn_ptr, &info)) {
+			RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to retrieve node Info\n");
+			goto error;
+		}
+		global_n_host_cpus = (unsigned int)info.cpus;
 	}
 
-	if (virNodeGetInfo(global_vir_conn_ptr, &info)) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to retrieve node Info\n");
-		goto error;
-	}
 
-	global_n_host_cpus = (unsigned)info.cpus;
 
 	if (global_n_host_cpus > CHANNEL_CMDS_MAX_CPUS) {
 		RTE_LOG(WARNING, CHANNEL_MANAGER, "The number of host CPUs(%u) exceeds the "
@@ -811,7 +822,8 @@ channel_manager_init(const char *path)
 
 	return 0;
 error:
-	disconnect_hypervisor();
+	if (global_hypervisor_available)
+		disconnect_hypervisor();
 	return -1;
 }
 
@@ -838,7 +850,10 @@ channel_manager_exit(void)
 		rte_free(vm_info);
 	}
 
-	rte_free(global_cpumaps);
-	rte_free(global_vircpuinfo);
-	disconnect_hypervisor();
+	if (global_hypervisor_available) {
+		/* Only needed if hypervisor available */
+		rte_free(global_cpumaps);
+		rte_free(global_vircpuinfo);
+		disconnect_hypervisor();
+	}
 }
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index 7fa47ba97..f180d74e6 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -66,7 +66,7 @@ static void
 core_share_status(int pNo)
 {
 
-	int noVms, noVcpus, z, x, t;
+	int noVms = 0, noVcpus = 0, z, x, t;
 
 	get_all_vm(&noVms, &noVcpus);
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v3 2/8] lib/power: add changes for host commands/policies
  2018-09-14 13:53   ` [dpdk-dev] [PATCH v3 0/8] add json power policy interface for containers David Hunt
  2018-09-14 13:53     ` [dpdk-dev] [PATCH v3 1/8] examples/power: add checks around hypervisor David Hunt
@ 2018-09-14 13:54     ` David Hunt
  2018-09-25  9:21       ` Burakov, Anatoly
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 3/8] examples/power: add necessary changes to guest app David Hunt
                       ` (6 subsequent siblings)
  8 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-14 13:54 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson, David Hunt

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 lib/librte_power/channel_commands.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/lib/librte_power/channel_commands.h b/lib/librte_power/channel_commands.h
index ee638eefa..e7b93a797 100644
--- a/lib/librte_power/channel_commands.h
+++ b/lib/librte_power/channel_commands.h
@@ -19,6 +19,7 @@ extern "C" {
 #define CPU_POWER               1
 #define CPU_POWER_CONNECT       2
 #define PKT_POLICY              3
+#define PKT_POLICY_REMOVE       4
 
 /* CPU Power Command Scaling */
 #define CPU_POWER_SCALE_UP      1
@@ -58,6 +59,9 @@ struct traffic {
 	uint32_t max_max_packet_thresh;
 };
 
+#define CORE_TYPE_VIRTUAL 0
+#define CORE_TYPE_PHYSICAL 1
+
 struct channel_packet {
 	uint64_t resource_id; /**< core_num, device */
 	uint32_t unit;        /**< scale down/up/min/max */
@@ -70,6 +74,7 @@ struct channel_packet {
 	uint8_t vcpu_to_control[MAX_VCPU_PER_VM];
 	uint8_t num_vcpu;
 	struct timer_profile timer_policy;
+	bool core_type;
 	enum workload workload;
 	enum policy_to_use policy_to_use;
 	struct t_boost_status t_boost_status;
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v3 3/8] examples/power: add necessary changes to guest app
  2018-09-14 13:53   ` [dpdk-dev] [PATCH v3 0/8] add json power policy interface for containers David Hunt
  2018-09-14 13:53     ` [dpdk-dev] [PATCH v3 1/8] examples/power: add checks around hypervisor David Hunt
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 2/8] lib/power: add changes for host commands/policies David Hunt
@ 2018-09-14 13:54     ` David Hunt
  2018-09-25  9:23       ` Burakov, Anatoly
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 4/8] examples/power: add host channel to power manager David Hunt
                       ` (5 subsequent siblings)
  8 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-14 13:54 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson, David Hunt

The changes here are minimal, as the guest app functionality is not
changing at all, but there is a new element in the channel_packet
struct that needs to have a default set (channel_packet->core_type).

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/guest_cli/vm_power_cli_guest.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
index 0db1b804f..2d9e7689a 100644
--- a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
+++ b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
@@ -92,6 +92,7 @@ set_policy_defaults(struct channel_packet *pkt)
 	pkt->timer_policy.hours_to_use_traffic_profile[0] = 8;
 	pkt->timer_policy.hours_to_use_traffic_profile[1] = 10;
 
+	pkt->core_type = CORE_TYPE_VIRTUAL;
 	pkt->workload = LOW;
 	pkt->policy_to_use = TIME;
 	pkt->command = PKT_POLICY;
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v3 4/8] examples/power: add host channel to power manager
  2018-09-14 13:53   ` [dpdk-dev] [PATCH v3 0/8] add json power policy interface for containers David Hunt
                       ` (2 preceding siblings ...)
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 3/8] examples/power: add necessary changes to guest app David Hunt
@ 2018-09-14 13:54     ` David Hunt
  2018-09-25  9:48       ` Burakov, Anatoly
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 5/8] examples/power: add json string handling David Hunt
                       ` (4 subsequent siblings)
  8 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-14 13:54 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson, David Hunt

This patch adds a fifo channel to the vm_power_manager app through which
we can send commands and polices. Intended for sending JSON strings.
The fifo is at /tmp/powermonitor/fifo.0

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/channel_manager.c | 108 +++++++++++++++
 examples/vm_power_manager/channel_manager.h |  17 ++-
 examples/vm_power_manager/channel_monitor.c | 146 +++++++++++++++-----
 examples/vm_power_manager/main.c            |   2 +
 4 files changed, 238 insertions(+), 35 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.c b/examples/vm_power_manager/channel_manager.c
index 2e471d0c1..88ae108a6 100644
--- a/examples/vm_power_manager/channel_manager.c
+++ b/examples/vm_power_manager/channel_manager.c
@@ -13,6 +13,7 @@
 
 #include <sys/queue.h>
 #include <sys/types.h>
+#include <sys/stat.h>
 #include <sys/socket.h>
 #include <sys/select.h>
 
@@ -284,6 +285,38 @@ open_non_blocking_channel(struct channel_info *info)
 	return 0;
 }
 
+static int
+open_host_channel(struct channel_info *info)
+{
+	int flags;
+
+	info->fd = open(info->channel_path, O_RDWR | O_RSYNC);
+	if (info->fd == -1) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Error(%s) opening fifo for '%s'\n",
+				strerror(errno),
+				info->channel_path);
+		return -1;
+	}
+
+	/* Get current flags */
+	flags = fcntl(info->fd, F_GETFL, 0);
+	if (flags < 0) {
+		RTE_LOG(WARNING, CHANNEL_MANAGER, "Error(%s) fcntl get flags socket for"
+				"'%s'\n", strerror(errno), info->channel_path);
+		return 1;
+	}
+	/* Set to Non Blocking */
+	flags |= O_NONBLOCK;
+	if (fcntl(info->fd, F_SETFL, flags) < 0) {
+		RTE_LOG(WARNING, CHANNEL_MANAGER,
+				"Error(%s) setting non-blocking "
+				"socket for '%s'\n",
+				strerror(errno), info->channel_path);
+		return -1;
+	}
+	return 0;
+}
+
 static int
 setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 		struct channel_info **chan_info_dptr, unsigned channel_num)
@@ -294,6 +327,7 @@ setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 	chan_info->channel_num = channel_num;
 	chan_info->priv_info = (void *)vm_info;
 	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
+	chan_info->type = CHANNEL_TYPE_BINARY;
 	if (open_non_blocking_channel(chan_info) < 0) {
 		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open channel: "
 				"'%s' for VM '%s'\n",
@@ -316,6 +350,35 @@ setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 	return 0;
 }
 
+static int
+setup_host_channel_info(struct channel_info **chan_info_dptr,
+		unsigned int channel_num)
+{
+	struct channel_info *chan_info = *chan_info_dptr;
+
+	chan_info->channel_num = channel_num;
+	chan_info->priv_info = (void *)0;
+	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
+	chan_info->type = CHANNEL_TYPE_JSON;
+	sprintf(chan_info->channel_path, "%sfifo.0", CHANNEL_MGR_SOCKET_PATH);
+
+	if (open_host_channel(chan_info) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open host channel: "
+				"'%s'\n",
+				chan_info->channel_path);
+		return -1;
+	}
+	if (add_channel_to_monitor(&chan_info) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Could add channel: "
+				"'%s' to epoll ctl\n",
+				chan_info->channel_path);
+		return -1;
+
+	}
+	chan_info->status = CHANNEL_MGR_CHANNEL_CONNECTED;
+	return 0;
+}
+
 int
 add_all_channels(const char *vm_name)
 {
@@ -470,6 +533,51 @@ add_channels(const char *vm_name, unsigned *channel_list,
 	return num_channels_enabled;
 }
 
+int
+add_host_channel(void)
+{
+	struct channel_info *chan_info;
+	char socket_path[PATH_MAX];
+	int num_channels_enabled = 0;
+	int ret;
+
+	snprintf(socket_path, sizeof(socket_path), "%sfifo.%u",
+			CHANNEL_MGR_SOCKET_PATH, 0);
+
+	errno = 0;
+	ret = mkfifo(socket_path, 0666);
+	if ((errno != EEXIST) && (ret < 0)) {
+		printf(" %d %d, %d\n", ret, EEXIST, errno);
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Cannot create fifo '%s' error: "
+				"%s\n", socket_path, strerror(errno));
+		return 0;
+	}
+
+	errno = 0;
+	if (access(socket_path, F_OK) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Channel path '%s' error: "
+				"%s\n", socket_path, strerror(errno));
+		return 0;
+	}
+	chan_info = rte_malloc(NULL, sizeof(*chan_info),
+			RTE_CACHE_LINE_SIZE);
+	if (chan_info == NULL) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for "
+				"channel '%s'\n", socket_path);
+		return 0;
+	}
+	snprintf(chan_info->channel_path,
+			sizeof(chan_info->channel_path), "%sfifo.%u",
+			CHANNEL_MGR_SOCKET_PATH, 0);
+	if (setup_host_channel_info(&chan_info, 0) < 0) {
+		rte_free(chan_info);
+		return 0;
+	}
+	num_channels_enabled++;
+
+	return num_channels_enabled;
+}
+
 int
 remove_channel(struct channel_info **chan_info_dptr)
 {
diff --git a/examples/vm_power_manager/channel_manager.h b/examples/vm_power_manager/channel_manager.h
index 872ec6140..c157cc22b 100644
--- a/examples/vm_power_manager/channel_manager.h
+++ b/examples/vm_power_manager/channel_manager.h
@@ -37,7 +37,7 @@ struct sockaddr_un _sockaddr_un;
 #define UNIX_PATH_MAX sizeof(_sockaddr_un.sun_path)
 #endif
 
-#define MAX_VMS 4
+#define MAX_VMS 64
 #define MAX_VCPUS 20
 
 
@@ -54,6 +54,11 @@ enum channel_status { CHANNEL_MGR_CHANNEL_DISCONNECTED = 0,
 	CHANNEL_MGR_CHANNEL_DISABLED,
 	CHANNEL_MGR_CHANNEL_PROCESSING};
 
+/* Communication Channel Type */
+enum channel_type { CHANNEL_TYPE_BINARY = 0,
+	CHANNEL_TYPE_INI,
+	CHANNEL_TYPE_JSON};
+
 /* VM libvirt(qemu/KVM) connection status */
 enum vm_status { CHANNEL_MGR_VM_INACTIVE = 0, CHANNEL_MGR_VM_ACTIVE};
 
@@ -66,6 +71,7 @@ struct channel_info {
 	volatile uint32_t status;    /**< Connection status(enum channel_status) */
 	int fd;                      /**< AF_UNIX socket fd */
 	unsigned channel_num;        /**< CHANNEL_MGR_SOCKET_PATH/<vm_name>.channel_num */
+	enum channel_type type;      /**< Binary, ini, json, etc. */
 	void *priv_info;             /**< Pointer to private info, do not modify */
 };
 
@@ -226,6 +232,15 @@ int add_all_channels(const char *vm_name);
 int add_channels(const char *vm_name, unsigned *channel_list,
 		unsigned num_channels);
 
+/**
+ * Set up a fifo by which host applications can send command an policies
+ * through a fifo to the vm_power_manager
+ *
+ * @return
+ *  - 0 for success
+ */
+int add_host_channel(void);
+
 /**
  * Remove a channel definition from the channel manager. This must only be
  * called from the channel monitor thread.
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index f180d74e6..17383e9d2 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -85,6 +85,33 @@ core_share_status(int pNo)
 	}
 }
 
+
+static int
+pcpu_monitor(struct policy *pol, struct core_info *ci, int pcpu, int count)
+{
+	int ret = 0;
+
+	if (pol->pkt.policy_to_use == BRANCH_RATIO) {
+		ci->cd[pcpu].oob_enabled = 1;
+		ret = add_core_to_monitor(pcpu);
+		if (ret == 0)
+			RTE_LOG(INFO, CHANNEL_MONITOR,
+					"Monitoring pcpu %d OOB for %s\n",
+					pcpu, pol->pkt.vm_name);
+		else
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Error monitoring pcpu %d OOB for %s\n",
+					pcpu, pol->pkt.vm_name);
+
+	} else {
+		pol->core_share[count].pcpu = pcpu;
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				"Monitoring pcpu %d for %s\n",
+				pcpu, pol->pkt.vm_name);
+	}
+	return ret;
+}
+
 static void
 get_pcpu_to_control(struct policy *pol)
 {
@@ -94,34 +121,46 @@ get_pcpu_to_control(struct policy *pol)
 	int pcpu, count;
 	uint64_t mask_u64b;
 	struct core_info *ci;
-	int ret;
 
 	ci = get_core_info();
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Looking for pcpu for %s\n",
-			pol->pkt.vm_name);
-	get_info_vm(pol->pkt.vm_name, &info);
-
-	for (count = 0; count < pol->pkt.num_vcpu; count++) {
-		mask_u64b = info.pcpu_mask[pol->pkt.vcpu_to_control[count]];
-		for (pcpu = 0; mask_u64b; mask_u64b &= ~(1ULL << pcpu++)) {
-			if ((mask_u64b >> pcpu) & 1) {
-				if (pol->pkt.policy_to_use == BRANCH_RATIO) {
-					ci->cd[pcpu].oob_enabled = 1;
-					ret = add_core_to_monitor(pcpu);
-					if (ret == 0)
-						printf("Monitoring pcpu %d via Branch Ratio\n",
-								pcpu);
-					else
-						printf("Failed to start OOB Monitoring pcpu %d\n",
-								pcpu);
-
-				} else {
-					pol->core_share[count].pcpu = pcpu;
-					printf("Monitoring pcpu %d\n", pcpu);
-				}
+	RTE_LOG(INFO, CHANNEL_MONITOR,
+			"Looking for pcpu for %s\n", pol->pkt.vm_name);
+
+	/*
+	 * So now that we're handling virtual and physical cores, we need to
+	 * differenciate between them when adding them to the branch monitor.
+	 * Virtual cores need to be converted to physical cores.
+	 */
+
+
+
+
+	if (pol->pkt.core_type == CORE_TYPE_VIRTUAL) {
+		/*
+		 * If the cores in the policy are virtual, we need to map them
+		 * to physical core. We look up the vm info and use that for
+		 * the mapping.
+		 */
+		get_info_vm(pol->pkt.vm_name, &info);
+		for (count = 0; count < pol->pkt.num_vcpu; count++) {
+			mask_u64b =
+				info.pcpu_mask[pol->pkt.vcpu_to_control[count]];
+			for (pcpu = 0; mask_u64b;
+					mask_u64b &= ~(1ULL << pcpu++)) {
+				if ((mask_u64b >> pcpu) & 1)
+					pcpu_monitor(pol, ci, pcpu, count);
 			}
 		}
+	} else {
+		/*
+		 * If the cores in the policy are physical, we just use
+		 * those core id's directly.
+		 */
+		for (count = 0; count < pol->pkt.num_vcpu; count++) {
+			pcpu = pol->pkt.vcpu_to_control[count];
+			pcpu_monitor(pol, ci, pcpu, count);
+		}
 	}
 }
 
@@ -160,8 +199,13 @@ update_policy(struct channel_packet *pkt)
 	unsigned int updated = 0;
 	int i;
 
+
+	RTE_LOG(INFO, CHANNEL_MONITOR,
+			"Applying policy for %s\n", pkt->vm_name);
+
 	for (i = 0; i < MAX_VMS; i++) {
 		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
+			/* Copy the contents of *pkt into the policy.pkt */
 			policies[i].pkt = *pkt;
 			get_pcpu_to_control(&policies[i]);
 			if (get_pfid(&policies[i]) == -1) {
@@ -189,6 +233,24 @@ update_policy(struct channel_packet *pkt)
 	return 0;
 }
 
+static int
+remove_policy(struct channel_packet *pkt __rte_unused)
+{
+	int i;
+
+	/*
+	 * Disabling the policy is simply a case of setting
+	 * enabled to 0
+	 */
+	for (i = 0; i < MAX_VMS; i++) {
+		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
+			policies[i].enabled = 0;
+			return 0;
+		}
+	}
+	return -1;
+}
+
 static uint64_t
 get_pkt_diff(struct policy *pol)
 {
@@ -346,7 +408,6 @@ apply_policy(struct policy *pol)
 		apply_workload_profile(pol);
 }
 
-
 static int
 process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 {
@@ -355,6 +416,8 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (chan_info == NULL)
 		return -1;
 
+	RTE_LOG(INFO, CHANNEL_MONITOR, "Processing Request %s\n", pkt->vm_name);
+
 	if (rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_CONNECTED,
 			CHANNEL_MGR_CHANNEL_PROCESSING) == 0)
 		return -1;
@@ -362,10 +425,12 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (pkt->command == CPU_POWER) {
 		core_mask = get_pcpus_mask(chan_info, pkt->resource_id);
 		if (core_mask == 0) {
-			RTE_LOG(ERR, CHANNEL_MONITOR, "Error get physical CPU mask for "
-				"channel '%s' using vCPU(%u)\n", chan_info->channel_path,
-				(unsigned)pkt->unit);
-			return -1;
+			/*
+			 * Core mask will be 0 in the case where
+			 * hypervisor is not available so we're working in
+			 * the host, so use the core as the mask.
+			 */
+			core_mask = 1 << pkt->resource_id;
 		}
 		if (__builtin_popcountll(core_mask) == 1) {
 
@@ -421,12 +486,20 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	}
 
 	if (pkt->command == PKT_POLICY) {
-		RTE_LOG(INFO, CHANNEL_MONITOR, "\nProcessing Policy request from Guest\n");
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				"\nProcessing Policy request\n");
 		update_policy(pkt);
 		policy_is_set = 1;
 	}
 
-	/* Return is not checked as channel status may have been set to DISABLED
+	if (pkt->command == PKT_POLICY_REMOVE) {
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				 "Removing policy %s\n", pkt->vm_name);
+		remove_policy(pkt);
+	}
+
+	/*
+	 * Return is not checked as channel status may have been set to DISABLED
 	 * from management thread
 	 */
 	rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_PROCESSING,
@@ -448,13 +521,16 @@ add_channel_to_monitor(struct channel_info **chan_info)
 				"to epoll\n", info->channel_path);
 		return -1;
 	}
+	RTE_LOG(ERR, CHANNEL_MONITOR, "Added channel '%s' "
+			"to monitor\n", info->channel_path);
 	return 0;
 }
 
 int
 remove_channel_from_monitor(struct channel_info *chan_info)
 {
-	if (epoll_ctl(global_event_fd, EPOLL_CTL_DEL, chan_info->fd, NULL) < 0) {
+	if (epoll_ctl(global_event_fd, EPOLL_CTL_DEL,
+			chan_info->fd, NULL) < 0) {
 		RTE_LOG(ERR, CHANNEL_MONITOR, "Unable to remove channel '%s' "
 				"from epoll\n", chan_info->channel_path);
 		return -1;
@@ -467,11 +543,13 @@ channel_monitor_init(void)
 {
 	global_event_fd = epoll_create1(0);
 	if (global_event_fd == 0) {
-		RTE_LOG(ERR, CHANNEL_MONITOR, "Error creating epoll context with "
-				"error %s\n", strerror(errno));
+		RTE_LOG(ERR, CHANNEL_MONITOR,
+				"Error creating epoll context with error %s\n",
+				strerror(errno));
 		return -1;
 	}
-	global_events_list = rte_malloc("epoll_events", sizeof(*global_events_list)
+	global_events_list = rte_malloc("epoll_events",
+			sizeof(*global_events_list)
 			* MAX_EVENTS, RTE_CACHE_LINE_SIZE);
 	if (global_events_list == NULL) {
 		RTE_LOG(ERR, CHANNEL_MONITOR, "Unable to rte_malloc for "
diff --git a/examples/vm_power_manager/main.c b/examples/vm_power_manager/main.c
index 58c5fa45c..893bf4cdd 100644
--- a/examples/vm_power_manager/main.c
+++ b/examples/vm_power_manager/main.c
@@ -421,6 +421,8 @@ main(int argc, char **argv)
 		return -1;
 	}
 
+	add_host_channel();
+
 	printf("Running core monitor on lcore id %d\n", lcore_id);
 	rte_eal_remote_launch(run_core_monitor, NULL, lcore_id);
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v3 5/8] examples/power: add json string handling
  2018-09-14 13:53   ` [dpdk-dev] [PATCH v3 0/8] add json power policy interface for containers David Hunt
                       ` (3 preceding siblings ...)
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 4/8] examples/power: add host channel to power manager David Hunt
@ 2018-09-14 13:54     ` David Hunt
  2018-09-25 11:27       ` Burakov, Anatoly
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 6/8] examples/power: add meson/ninja build support David Hunt
                       ` (3 subsequent siblings)
  8 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-14 13:54 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson, David Hunt

Add JSON string handling to vm_power_manager for JSON strings received
through the fifo. The format of the JSON strings are detailed in the
next patch, the vm_power_manager user guide documentation updates.

This patch introduces a new dependency on Jansson, a C library for
encoding, decoding and manipulating JSON data. To compile the sample app
you now need to have installed libjansson4 and libjansson-dev (these may
be named slightly differently depending on your Operating System)

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/Makefile          |   6 +
 examples/vm_power_manager/channel_monitor.c | 297 +++++++++++++++++---
 2 files changed, 258 insertions(+), 45 deletions(-)

diff --git a/examples/vm_power_manager/Makefile b/examples/vm_power_manager/Makefile
index 13a5205ba..50147c05d 100644
--- a/examples/vm_power_manager/Makefile
+++ b/examples/vm_power_manager/Makefile
@@ -31,6 +31,12 @@ CFLAGS += $(WERROR_FLAGS)
 
 LDLIBS += -lvirt
 
+JANSSON := $(shell pkg-config --exists jansson; echo $$?)
+ifeq ($(JANSSON), 0)
+LDLIBS += $(shell pkg-config --libs jansson)
+CFLAGS += -DUSE_JANSSON
+endif
+
 ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
 
 ifeq ($(CONFIG_RTE_LIBRTE_IXGBE_PMD),y)
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index 17383e9d2..655f5e279 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -9,11 +9,18 @@
 #include <signal.h>
 #include <errno.h>
 #include <string.h>
+#include <fcntl.h>
 #include <sys/types.h>
 #include <sys/epoll.h>
 #include <sys/queue.h>
 #include <sys/time.h>
-
+#include <sys/socket.h>
+#include <sys/select.h>
+#ifdef USE_JANSSON
+#include <jansson.h>
+#else
+#pragma message "Jansson dev libs unavailable, not including JSON parsing"
+#endif
 #include <rte_log.h>
 #include <rte_memory.h>
 #include <rte_malloc.h>
@@ -35,6 +42,8 @@
 
 uint64_t vsi_pkt_count_prev[384];
 uint64_t rdtsc_prev[384];
+#define MAX_JSON_STRING_LEN 1024
+char json_data[MAX_JSON_STRING_LEN];
 
 double time_period_ms = 1;
 static volatile unsigned run_loop = 1;
@@ -43,6 +52,132 @@ static unsigned int policy_is_set;
 static struct epoll_event *global_events_list;
 static struct policy policies[MAX_VMS];
 
+#ifdef USE_JANSSON
+static int
+parse_json_to_pkt(json_t *element, struct channel_packet *pkt)
+{
+	const char *key;
+	json_t *value;
+	int ret;
+
+	memset(pkt, 0, sizeof(struct channel_packet));
+
+	pkt->nb_mac_to_monitor = 0;
+	pkt->t_boost_status.tbEnabled = false;
+	pkt->workload = LOW;
+	pkt->policy_to_use = TIME;
+	pkt->command = PKT_POLICY;
+	pkt->core_type = CORE_TYPE_PHYSICAL;
+
+	json_object_foreach(element, key, value) {
+		if (!strcmp(key, "policy")) {
+			/* Recurse in to get the contents of profile */
+			ret = parse_json_to_pkt(value, pkt);
+			if (ret)
+				return ret;
+		} else if (!strcmp(key, "instruction")) {
+			/* Recurse in to get the contents of instruction */
+			ret = parse_json_to_pkt(value, pkt);
+			if (ret)
+				return ret;
+		} else if (!strcmp(key, "name")) {
+			strcpy(pkt->vm_name, json_string_value(value));
+		} else if (!strcmp(key, "command")) {
+			char command[32];
+			snprintf(command, 32, "%s", json_string_value(value));
+			if (!strcmp(command, "power")) {
+				pkt->command = CPU_POWER;
+			} else if (!strcmp(command, "create")) {
+				pkt->command = PKT_POLICY;
+			} else if (!strcmp(command, "destroy")) {
+				pkt->command = PKT_POLICY_REMOVE;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Invalid command received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "policy_type")) {
+			char command[32];
+			snprintf(command, 32, "%s", json_string_value(value));
+			if (!strcmp(command, "TIME")) {
+				pkt->policy_to_use = TIME;
+			} else if (!strcmp(command, "TRAFFIC")) {
+				pkt->policy_to_use = TRAFFIC;
+			} else if (!strcmp(command, "WORKLOAD")) {
+				pkt->policy_to_use = WORKLOAD;
+			} else if (!strcmp(command, "BRANCH_RATIO")) {
+				pkt->policy_to_use = BRANCH_RATIO;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Wrong policy_type received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "busy_hours")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int hour = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->timer_policy.busy_hours[i] = hour;
+			}
+		} else if (!strcmp(key, "quiet_hours")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int hour = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->timer_policy.quiet_hours[i] = hour;
+			}
+		} else if (!strcmp(key, "core_list")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int core = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->vcpu_to_control[i] = core;
+			}
+			pkt->num_vcpu = size;
+		} else if (!strcmp(key, "avg_packet_thresh")) {
+			pkt->traffic_policy.avg_max_packet_thresh =
+					(uint32_t)json_integer_value(value);
+		} else if (!strcmp(key, "max_packet_thresh")) {
+			pkt->traffic_policy.max_max_packet_thresh =
+					(uint32_t)json_integer_value(value);
+		} else if (!strcmp(key, "unit")) {
+			char unit[32];
+			snprintf(unit, 32, "%s", json_string_value(value));
+			if (!strcmp(unit, "SCALE_UP")) {
+				pkt->unit = CPU_POWER_SCALE_UP;
+			} else if (!strcmp(unit, "SCALE_DOWN")) {
+				pkt->unit = CPU_POWER_SCALE_DOWN;
+			} else if (!strcmp(unit, "SCALE_MAX")) {
+				pkt->unit = CPU_POWER_SCALE_MAX;
+			} else if (!strcmp(unit, "SCALE_MIN")) {
+				pkt->unit = CPU_POWER_SCALE_MIN;
+			} else if (!strcmp(unit, "ENABLE_TURBO")) {
+				pkt->unit = CPU_POWER_ENABLE_TURBO;
+			} else if (!strcmp(unit, "DISABLE_TURBO")) {
+				pkt->unit = CPU_POWER_DISABLE_TURBO;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Invalid command received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "resource_id")) {
+			pkt->resource_id = (uint32_t)json_integer_value(value);
+		} else {
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+				"Unknown key received in JSON string: %s\n",
+				key);
+		}
+	}
+	return 0;
+}
+#endif
+
 void channel_monitor_exit(void)
 {
 	run_loop = 0;
@@ -124,18 +259,11 @@ get_pcpu_to_control(struct policy *pol)
 
 	ci = get_core_info();
 
-	RTE_LOG(INFO, CHANNEL_MONITOR,
-			"Looking for pcpu for %s\n", pol->pkt.vm_name);
-
 	/*
 	 * So now that we're handling virtual and physical cores, we need to
 	 * differenciate between them when adding them to the branch monitor.
 	 * Virtual cores need to be converted to physical cores.
 	 */
-
-
-
-
 	if (pol->pkt.core_type == CORE_TYPE_VIRTUAL) {
 		/*
 		 * If the cores in the policy are virtual, we need to map them
@@ -295,8 +423,6 @@ apply_traffic_profile(struct policy *pol)
 
 	diff = get_pkt_diff(pol);
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Applying traffic profile\n");
-
 	if (diff >= (pol->pkt.traffic_policy.max_max_packet_thresh)) {
 		for (count = 0; count < pol->pkt.num_vcpu; count++) {
 			if (pol->core_share[count].status != 1)
@@ -340,9 +466,6 @@ apply_time_profile(struct policy *pol)
 				if (pol->core_share[count].status != 1) {
 					power_manager_scale_core_max(
 						pol->core_share[count].pcpu);
-				RTE_LOG(INFO, CHANNEL_MONITOR,
-					"Scaling up core %d to max\n",
-					pol->core_share[count].pcpu);
 				}
 			}
 			break;
@@ -352,9 +475,6 @@ apply_time_profile(struct policy *pol)
 				if (pol->core_share[count].status != 1) {
 					power_manager_scale_core_min(
 						pol->core_share[count].pcpu);
-				RTE_LOG(INFO, CHANNEL_MONITOR,
-					"Scaling down core %d to min\n",
-					pol->core_share[count].pcpu);
 			}
 		}
 			break;
@@ -416,8 +536,6 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (chan_info == NULL)
 		return -1;
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Processing Request %s\n", pkt->vm_name);
-
 	if (rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_CONNECTED,
 			CHANNEL_MGR_CHANNEL_PROCESSING) == 0)
 		return -1;
@@ -486,8 +604,8 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	}
 
 	if (pkt->command == PKT_POLICY) {
-		RTE_LOG(INFO, CHANNEL_MONITOR,
-				"\nProcessing Policy request\n");
+		RTE_LOG(INFO, CHANNEL_MONITOR, "Processing policy request %s\n",
+				pkt->vm_name);
 		update_policy(pkt);
 		policy_is_set = 1;
 	}
@@ -559,6 +677,103 @@ channel_monitor_init(void)
 	return 0;
 }
 
+static void
+read_binary_packet(struct channel_info *chan_info)
+{
+	struct channel_packet pkt;
+	void *buffer = &pkt;
+	int buffer_len = sizeof(pkt);
+	int n_bytes, err = 0;
+
+	while (buffer_len > 0) {
+		n_bytes = read(chan_info->fd,
+				buffer, buffer_len);
+		if (n_bytes == buffer_len)
+			break;
+		if (n_bytes == -1) {
+			err = errno;
+			RTE_LOG(DEBUG, CHANNEL_MONITOR,
+				"Received error on "
+				"channel '%s' read: %s\n",
+				chan_info->channel_path,
+				strerror(err));
+			remove_channel(&chan_info);
+			break;
+		}
+		buffer = (char *)buffer + n_bytes;
+		buffer_len -= n_bytes;
+	}
+	if (!err)
+		process_request(&pkt, chan_info);
+}
+
+#ifdef USE_JANSSON
+static void
+read_json_packet(struct channel_info *chan_info)
+{
+	struct channel_packet pkt;
+	int n_bytes, ret;
+	json_t *root;
+	json_error_t error;
+
+	/* read opening brace to closing brace */
+	do {
+		int idx = 0;
+		int indent = 0;
+		do {
+			n_bytes = read(chan_info->fd, &json_data[idx], 1);
+			if (n_bytes == 0)
+				break;
+			if (json_data[idx] == '{')
+				indent++;
+			if (json_data[idx] == '}')
+				indent--;
+			if ((indent > 0) || (idx >> 0))
+				idx++;
+			if (indent == 0)
+				json_data[idx] = 0;
+			if (idx >= MAX_JSON_STRING_LEN)
+				break;
+		} while (indent > 0);
+
+		if (indent > 0)
+			/*
+			 * We've broken out of the read loop without getting
+			 * a closing brace, so throw away the datai
+			 */
+			json_data[idx] = 0;
+
+		if (strlen(json_data) == 0)
+			continue;
+
+		printf("got [%s]\n", json_data);
+
+		root = json_loads(json_data, 0, &error);
+
+		if (root) {
+			/*
+			 * Because our data is now in the json
+			 * object, we can overwrite the pkt
+			 * with a channel_packet struct, using
+			 * parse_json_to_pkt()
+			 */
+			ret = parse_json_to_pkt(root, &pkt);
+			json_decref(root);
+			if (ret) {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Error validating JSON profile data\n");
+				break;
+			}
+			process_request(&pkt, chan_info);
+		} else {
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+					"JSON error on line %d: %s\n",
+					error.line, error.text);
+		}
+	} while (n_bytes > 0);
+}
+#endif
+
 void
 run_channel_monitor(void)
 {
@@ -570,11 +785,16 @@ run_channel_monitor(void)
 		if (!run_loop)
 			break;
 		for (i = 0; i < n_events; i++) {
+			if (!global_events_list[i].data.ptr) {
+				fflush(stdout);
+				continue;
+			}
 			struct channel_info *chan_info = (struct channel_info *)
 					global_events_list[i].data.ptr;
 			if ((global_events_list[i].events & EPOLLERR) ||
 				(global_events_list[i].events & EPOLLHUP)) {
-				RTE_LOG(DEBUG, CHANNEL_MONITOR, "Remote closed connection for "
+				RTE_LOG(INFO, CHANNEL_MONITOR,
+						"Remote closed connection for "
 						"channel '%s'\n",
 						chan_info->channel_path);
 				remove_channel(&chan_info);
@@ -582,31 +802,18 @@ run_channel_monitor(void)
 			}
 			if (global_events_list[i].events & EPOLLIN) {
 
-				int n_bytes, err = 0;
-				struct channel_packet pkt;
-				void *buffer = &pkt;
-				int buffer_len = sizeof(pkt);
-
-				while (buffer_len > 0) {
-					n_bytes = read(chan_info->fd,
-							buffer, buffer_len);
-					if (n_bytes == buffer_len)
-						break;
-					if (n_bytes == -1) {
-						err = errno;
-						RTE_LOG(DEBUG, CHANNEL_MONITOR,
-							"Received error on "
-							"channel '%s' read: %s\n",
-							chan_info->channel_path,
-							strerror(err));
-						remove_channel(&chan_info);
-						break;
-					}
-					buffer = (char *)buffer + n_bytes;
-					buffer_len -= n_bytes;
+				switch (chan_info->type) {
+				case CHANNEL_TYPE_BINARY:
+					read_binary_packet(chan_info);
+					break;
+#ifdef USE_JANSSON
+				case CHANNEL_TYPE_JSON:
+					read_json_packet(chan_info);
+					break;
+#endif
+				default:
+					break;
 				}
-				if (!err)
-					process_request(&pkt, chan_info);
 			}
 		}
 		rte_delay_us(time_period_ms*1000);
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v3 6/8] examples/power: add meson/ninja build support
  2018-09-14 13:53   ` [dpdk-dev] [PATCH v3 0/8] add json power policy interface for containers David Hunt
                       ` (4 preceding siblings ...)
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 5/8] examples/power: add json string handling David Hunt
@ 2018-09-14 13:54     ` David Hunt
  2018-09-14 14:01       ` Bruce Richardson
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 7/8] doc/vm_power_manager: add JSON interface API info David Hunt
                       ` (2 subsequent siblings)
  8 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-14 13:54 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson, David Hunt

Add meson.build in vm_power_manager and the guest_cli subdirectory.
Building can be achieved by going to the build directory, and using

meson configure -Dexamples=vm_power_manager,vm_power_manager/guest_cli

Then, when ninja is invoked, it will build dpdk-vm_power_manger and
dpdk-guest_cli

Work still needs to be done on the meson build system to handles the case
where the target list of example apps is defined as 'all'. That will come
in a future patch.

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 .../vm_power_manager/guest_cli/meson.build    | 21 +++++++++++
 examples/vm_power_manager/meson.build         | 37 ++++++++++++++++++-
 2 files changed, 56 insertions(+), 2 deletions(-)
 create mode 100644 examples/vm_power_manager/guest_cli/meson.build

diff --git a/examples/vm_power_manager/guest_cli/meson.build b/examples/vm_power_manager/guest_cli/meson.build
new file mode 100644
index 000000000..9e821ceb8
--- /dev/null
+++ b/examples/vm_power_manager/guest_cli/meson.build
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+# Setting the name here because the default name will conflict with the
+# vm_power_manager app because of the way the directories are parsed.
+name = 'guest_cli'
+
+deps += ['power']
+
+sources = files(
+	'main.c', 'parse.c', 'vm_power_cli_guest.c'
+)
+
+opt_dep = cc.find_library('virt', required : false)
+build = opt_dep.found()
+ext_deps += opt_dep
diff --git a/examples/vm_power_manager/meson.build b/examples/vm_power_manager/meson.build
index c370d7476..f98445bc6 100644
--- a/examples/vm_power_manager/meson.build
+++ b/examples/vm_power_manager/meson.build
@@ -6,5 +6,38 @@
 # To build this example as a standalone application with an already-installed
 # DPDK instance, use 'make'
 
-# Example app currently unsupported by meson build
-build = false
+if dpdk_conf.has('RTE_LIBRTE_BNXT_PMD')
+	deps += ['pmd_bnxt']
+endif
+
+if dpdk_conf.has('RTE_LIBRTE_I40E_PMD')
+	deps += ['pmd_i40e']
+endif
+
+if dpdk_conf.has('RTE_LIBRTE_IXGBE_PMD')
+	deps += ['pmd_ixgbe']
+endif
+
+deps += ['power']
+
+
+sources = files(
+	'channel_manager.c', 'channel_monitor.c', 'main.c', 'parse.c', 'power_manager.c', 'vm_power_cli.c'
+)
+
+# If we're on X86, pull in the x86 code for the branch monitor algo.
+if dpdk_conf.has('RTE_ARCH_X86_64')
+	sources += files('oob_monitor_x86.c')
+else
+	sources += files('oob_monitor_nop.c')
+endif
+
+opt_dep = cc.find_library('virt', required : false)
+build = opt_dep.found()
+ext_deps += opt_dep
+
+opt_dep = dependency('jansson', required : false)
+if opt_dep.found()
+	ext_deps += opt_dep
+	cflags += '-DUSE_JANSSON'
+endif
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v3 7/8] doc/vm_power_manager: add JSON interface API info
  2018-09-14 13:53   ` [dpdk-dev] [PATCH v3 0/8] add json power policy interface for containers David Hunt
                       ` (5 preceding siblings ...)
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 6/8] examples/power: add meson/ninja build support David Hunt
@ 2018-09-14 13:54     ` David Hunt
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 8/8] examples/power: add json example files David Hunt
  2018-09-26 13:40     ` [dpdk-dev] [PATCH v4 0/11] add json power policy interface for containers David Hunt
  8 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-14 13:54 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson, David Hunt

Also added meson/ninja build info

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 .../sample_app_ug/vm_power_management.rst     | 272 +++++++++++++++++-
 1 file changed, 270 insertions(+), 2 deletions(-)

diff --git a/doc/guides/sample_app_ug/vm_power_management.rst b/doc/guides/sample_app_ug/vm_power_management.rst
index 855570d6b..038cf1f93 100644
--- a/doc/guides/sample_app_ug/vm_power_management.rst
+++ b/doc/guides/sample_app_ug/vm_power_management.rst
@@ -199,7 +199,7 @@ see :doc:`compiling`.
 
 The application is located in the ``vm_power_manager`` sub-directory.
 
-To build just the ``vm_power_manager`` application:
+To build just the ``vm_power_manager`` application using ``make``:
 
 .. code-block:: console
 
@@ -208,6 +208,22 @@ To build just the ``vm_power_manager`` application:
   cd ${RTE_SDK}/examples/vm_power_manager/
   make
 
+The resulting binary will be ${RTE_SDK}/build/examples/vm_power_manager
+
+To build just the ``vm_power_manager`` application using ``meson/ninja``:
+
+.. code-block:: console
+
+  export RTE_SDK=/path/to/rte_sdk
+  cd ${RTE_SDK}
+  meson build
+  cd build
+  ninja
+  meson configure -Dexamples=vm_power_manager
+  ninja
+
+The resulting binary will be ${RTE_SDK}/build/examples/dpdk-vm_power_manager
+
 Running
 ~~~~~~~
 
@@ -337,6 +353,242 @@ monitoring of branch ratio on cores doing busy polling via PMDs.
   and will need to be adjusted for different workloads.
 
 
+
+JSON API
+~~~~~~~~
+
+In addition to the command line interface for host command and a virtio-serial
+interface for VM power policies, there is also a JSON interface through which
+power commands and policies can be sent. This functionality adds a dependency
+on the Jansson library, and the Jansson development package must be installed
+on the system before the JSON parsing functionality is included in the app.
+This is achieved by:
+
+  .. code-block:: console
+
+    apt-get install libjansson-dev
+
+The command and package name may be different depending on your operating
+system. It's worth noting that the app will successfully build without this
+package present, but a warning is shown during compilation, and the JSON
+parsing functionality will not be present in the app.
+
+Sending a command or policy to the power manager application is achieved by
+simply opening a fifo file, writing a JSON string to that fifo, and closing
+the file.
+
+The fifo is at /tmp/powermonitor/fifo.0
+
+The jason string can be a policy or instruction, and takes the following
+format:
+
+  .. code-block:: console
+
+    {"packet_type": {
+      "pair_1": value,
+      "pair_2": value
+    }}
+
+The 'packet_type' header can contain one of two values, depending on
+whether a policy or power command is being sent. The two possible values are
+"policy" and "instruction", and the expected name-value pairs is different
+depending on which type is being sent.
+
+The pairs are the format of standard JSON name-value pairs. The value type
+varies between the different name/value pairs, and may be intgers, strings,
+arrays, etc. Examples of policies follow later in this document. The allowed
+names and value types are as follows:
+
+
+:Pair Name: "name"
+:Description: Name of the VM or Host. Allows the parser to associate the
+  policy with the relevant VM or Host OS.
+:Type: string
+:Values: any valid string
+:Required: yes
+:Example:
+
+  .. code-block:: console
+
+    ""name", "ubuntu2"
+
+
+:Pair Name: "command"
+:Description: The type of packet we're sending to the power manager. We can be
+  creating or destroying a policy, or sending a direct command to adjust
+  the frequency of a core, similar to the command line interface.
+:Type: string
+:Values:
+
+  :"CREATE": used when creating a new policy,
+  :"DESTROY": used when removing a policy,
+  :"POWER": used when sending an immediate command, max, min, etc.
+:Required: yes
+:Example:
+
+    .. code-block:: console
+
+      "command", "CREATE"
+
+
+:Pair Name: "policy_type"
+:Description: Type of policy to apply. Please see vm_power_manager documentation
+  for more information on the types of policies that may be used.
+:Type: string
+:Values:
+
+  :"TIME": Time-of-day policy. Frequencies of the relevant cores are
+    scaled up/down depending on busy and quiet hours.
+  :"TRAFFIC": This policy takes statistics from the NIC and scales up
+    and down accordingly.
+  :"WORKLOAD": This policy looks at how heavily loaded the cores are,
+    and scales up and down accordingly.
+  :"BRANCH_RATIO": This out-of-band policy can look at the ratio between
+    branch hits and misses on a core, and is useful for detecting
+    how much packet processing a core is doing.
+:Required: only for CREATE/DESTROY command
+:Example:
+
+  .. code-block:: console
+
+    "policy_type", "TIME"
+
+:Pair Name: "busy_hours"
+:Description: The hours of the day in which we scale up the cores for busy
+  times.
+:Type: array of integers
+:Values: array with list of hour numbers, (0-23)
+:Required: only for TIME policy
+:Example:
+
+  .. code-block:: console
+
+    "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ]
+
+:Pair Name: "quiet_hours"
+:Description: The hours of the day in which we scale down the cores for quiet
+  times.
+:Type: array of integers
+:Values: array with list of hour numbers, (0-23)
+:Required: only for TIME policy
+:Example:
+
+  .. code-block:: console
+
+    "quiet_hours":[ 2, 3, 4, 5, 6 ]
+
+:Pair Name: "avg_packet_thresh"
+:Description: Threshold below which the frequency will be set to min for
+  the TRAFFIC policy. If the traffic rate is above this and below max, the
+  frequency will be set to medium.
+:Type: integer
+:Values: The number of packets below which the TRAFFIC policy applies the
+  minimum frequency, or medium frequency if between avg and max thresholds.
+:Required: only for TRAFFIC policy
+:Example:
+
+  .. code-block:: console
+
+    "avg_packet_thresh": 100000
+
+:Pair Name: "max_packet_thresh"
+:Description: Threshold above which the frequency will be set to max for
+  the TRAFFIC policy
+:Type: integer
+:Values: The number of packets per interval above which the TRAFFIC policy
+  applies the maximum frequency
+:Required: only for TRAFFIC policy
+:Example:
+
+  .. code-block:: console
+
+    "max_packet_thresh": 500000
+
+:Pair Name: "core_list"
+:Description: The cores to which to apply the policy.
+:Type: array of integers
+:Values: array with list of virtual CPUs.
+:Required: only policy CREATE/DESTROY
+:Example:
+
+  .. code-block:: console
+
+    "core_list":[ 10, 11 ]
+
+:Pair Name: "unit"
+:Description: the type of power operation to apply in the command
+:Type: string
+:Values:
+
+  :"SCALE_MAX": Scale frequency of this core to maximum
+  :"SCALE_MIN": Scale frequency of this core to minimum
+  :"SCALE_UP": Scale up frequency of this core
+  :"SCALE_DOWN": Scale down frequency of this core
+  :"ENABLE_TURBO": Enable Turbo Boost for this core
+  :"DISABLE_TURBO": Disable Turbo Boost for this core
+:Required: only for POWER instruction
+:Example:
+
+  .. code-block:: console
+
+    "unit", "SCALE_MAX"
+
+:Pair Name: "resource_id"
+:Description: The core to which to apply the power command.
+:Type: integer
+:Values: valid core id for VM or host OS.
+:Required: only POWER instruction
+:Example:
+
+  .. code-block:: console
+
+    "resource_id": 10
+
+JSON API Examples
+~~~~~~~~~~~~~~~~~
+
+Profile create example:
+
+  .. code-block:: console
+
+    {"policy": {
+      "name": "ubuntu",
+      "command": "create",
+      "policy_type": "TIME",
+      "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
+      "quiet_hours":[ 2, 3, 4, 5, 6 ],
+      "core_list":[ 11 ]
+    }}
+
+Profile destroy example:
+
+  .. code-block:: console
+
+    {"profile": {
+      "name": "ubuntu",
+      "command": "destroy",
+    }}
+
+Power command example:
+
+  .. code-block:: console
+
+    {"command": {
+      "name": "ubuntu",
+      "unit": "SCALE_MAX",
+      "resource_id": 10
+    }}
+
+To send a JSON string to the Power Manager application, simply paste the
+example JSON string into a text file and cat it into the fifo:
+
+  .. code-block:: console
+
+    cat file.json >/tmp/powermonitor/fifo.0
+
+The console of the Power Manager application should indicate the command that
+was just received via the fifo.
+
 Compiling and Running the Guest Applications
 --------------------------------------------
 
@@ -366,7 +618,7 @@ For compiling and running l3fwd-power, see :doc:`l3_forward_power_man`.
 
 The application is located in the ``guest_cli`` sub-directory under ``vm_power_manager``.
 
-To build just the ``guest_vm_power_manager`` application:
+To build just the ``guest_vm_power_manager`` application using ``make``:
 
 .. code-block:: console
 
@@ -375,6 +627,22 @@ To build just the ``guest_vm_power_manager`` application:
   cd ${RTE_SDK}/examples/vm_power_manager/guest_cli/
   make
 
+The resulting binary will be ${RTE_SDK}/build/examples/guest_cli
+
+To build just the ``vm_power_manager`` application using ``meson/ninja``:
+
+.. code-block:: console
+
+  export RTE_SDK=/path/to/rte_sdk
+  cd ${RTE_SDK}
+  meson build
+  cd build
+  ninja
+  meson configure -Dexamples=vm_power_manager/guest_cli
+  ninja
+
+The resulting binary will be ${RTE_SDK}/build/examples/guest_cli
+
 Running
 ~~~~~~~
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v3 8/8] examples/power: add json example files
  2018-09-14 13:53   ` [dpdk-dev] [PATCH v3 0/8] add json power policy interface for containers David Hunt
                       ` (6 preceding siblings ...)
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 7/8] doc/vm_power_manager: add JSON interface API info David Hunt
@ 2018-09-14 13:54     ` David Hunt
  2018-09-26 13:40     ` [dpdk-dev] [PATCH v4 0/11] add json power policy interface for containers David Hunt
  8 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-14 13:54 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson, David Hunt

This patch provides some example files in the json_examples sub-directory
for sending to the fifo.

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/json_examples/README            | 6 ++++++
 examples/vm_power_manager/json_examples/create.json       | 8 ++++++++
 examples/vm_power_manager/json_examples/destroy.json      | 4 ++++
 examples/vm_power_manager/json_examples/set_core_max.json | 6 ++++++
 examples/vm_power_manager/json_examples/set_core_min.json | 6 ++++++
 5 files changed, 30 insertions(+)
 create mode 100644 examples/vm_power_manager/json_examples/README
 create mode 100644 examples/vm_power_manager/json_examples/create.json
 create mode 100644 examples/vm_power_manager/json_examples/destroy.json
 create mode 100644 examples/vm_power_manager/json_examples/set_core_max.json
 create mode 100644 examples/vm_power_manager/json_examples/set_core_min.json

diff --git a/examples/vm_power_manager/json_examples/README b/examples/vm_power_manager/json_examples/README
new file mode 100644
index 000000000..a94c6b14b
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/README
@@ -0,0 +1,6 @@
+Sample files for sending to the vm_power_manager through the fifo.
+
+Simply cat the file to /tmp/powermonitor/fifo.0 when the vm_power_manager
+application is running.
+
+E.g. cat create.json >/tmp/powermonitor/fifo.0
diff --git a/examples/vm_power_manager/json_examples/create.json b/examples/vm_power_manager/json_examples/create.json
new file mode 100644
index 000000000..a7133d9a1
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/create.json
@@ -0,0 +1,8 @@
+{"policy": {
+  "name": "policy-1",
+  "command": "create",
+  "policy_type": "TIME",
+  "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
+  "quiet_hours":[ 2, 3, 4, 5, 6 ],
+  "core_list":[ 11, 12 ]
+}}
diff --git a/examples/vm_power_manager/json_examples/destroy.json b/examples/vm_power_manager/json_examples/destroy.json
new file mode 100644
index 000000000..587c9e7e9
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/destroy.json
@@ -0,0 +1,4 @@
+{"policy": {
+  "name": "policy-1",
+  "command": "destroy"
+}}
diff --git a/examples/vm_power_manager/json_examples/set_core_max.json b/examples/vm_power_manager/json_examples/set_core_max.json
new file mode 100644
index 000000000..497030a44
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/set_core_max.json
@@ -0,0 +1,6 @@
+{"instruction": {
+  "name": "set_power",
+  "command": "power",
+  "unit": "SCALE_MAX",
+  "resource_id": 10
+}}
diff --git a/examples/vm_power_manager/json_examples/set_core_min.json b/examples/vm_power_manager/json_examples/set_core_min.json
new file mode 100644
index 000000000..76d934fd8
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/set_core_min.json
@@ -0,0 +1,6 @@
+{"instruction": {
+  "name": "set_power",
+  "command": "power",
+  "unit": "SCALE_MIN",
+  "resource_id": 10
+}}
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v3 6/8] examples/power: add meson/ninja build support
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 6/8] examples/power: add meson/ninja build support David Hunt
@ 2018-09-14 14:01       ` Bruce Richardson
  0 siblings, 0 replies; 110+ messages in thread
From: Bruce Richardson @ 2018-09-14 14:01 UTC (permalink / raw)
  To: David Hunt; +Cc: dev, john.mcnamara, stephen, lei.a.yao

On Fri, Sep 14, 2018 at 02:54:04PM +0100, David Hunt wrote:
> Add meson.build in vm_power_manager and the guest_cli subdirectory.
> Building can be achieved by going to the build directory, and using
> 
> meson configure -Dexamples=vm_power_manager,vm_power_manager/guest_cli
> 
> Then, when ninja is invoked, it will build dpdk-vm_power_manger and
> dpdk-guest_cli
> 
> Work still needs to be done on the meson build system to handles the case
> where the target list of example apps is defined as 'all'. That will come
> in a future patch.
> 
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---
The examples=all limitation only applies to the guest_cli app, because it's
in a subdirectory. That should be possible to manage in a later changeset,
but this is good for now, as it still has the main app buildable with the
test-meson-builds script.

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v3 1/8] examples/power: add checks around hypervisor
  2018-09-14 13:53     ` [dpdk-dev] [PATCH v3 1/8] examples/power: add checks around hypervisor David Hunt
@ 2018-09-25  9:20       ` Burakov, Anatoly
  2018-09-25 13:47         ` Hunt, David
  0 siblings, 1 reply; 110+ messages in thread
From: Burakov, Anatoly @ 2018-09-25  9:20 UTC (permalink / raw)
  To: David Hunt, dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson

On 14-Sep-18 2:53 PM, David Hunt wrote:
> Allow vm_power_manager to run without requiring qemu to be present
> on the machine. This will be required for instances where the JSON
> interface is used for commands and polices, without any VMs present.
> A use case for this is a container enviromnent.
> 
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---

<snip>

> --- a/examples/vm_power_manager/channel_monitor.c
> +++ b/examples/vm_power_manager/channel_monitor.c
> @@ -66,7 +66,7 @@ static void
>   core_share_status(int pNo)
>   {
>   
> -	int noVms, noVcpus, z, x, t;
> +	int noVms = 0, noVcpus = 0, z, x, t;
>   

This looks like a unrelated change?

>   	get_all_vm(&noVms, &noVcpus);
>   
> 


-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v3 2/8] lib/power: add changes for host commands/policies
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 2/8] lib/power: add changes for host commands/policies David Hunt
@ 2018-09-25  9:21       ` Burakov, Anatoly
  2018-09-25 13:47         ` Hunt, David
  0 siblings, 1 reply; 110+ messages in thread
From: Burakov, Anatoly @ 2018-09-25  9:21 UTC (permalink / raw)
  To: David Hunt, dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson

On 14-Sep-18 2:54 PM, David Hunt wrote:
> Signed-off-by: David Hunt <david.hunt@intel.com>

The commit message is not very descriptive as to what these chages are 
and why they are added.

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v3 3/8] examples/power: add necessary changes to guest app
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 3/8] examples/power: add necessary changes to guest app David Hunt
@ 2018-09-25  9:23       ` Burakov, Anatoly
  0 siblings, 0 replies; 110+ messages in thread
From: Burakov, Anatoly @ 2018-09-25  9:23 UTC (permalink / raw)
  To: David Hunt, dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson

On 14-Sep-18 2:54 PM, David Hunt wrote:
> The changes here are minimal, as the guest app functionality is not
> changing at all, but there is a new element in the channel_packet
> struct that needs to have a default set (channel_packet->core_type).
> 
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---

Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v3 4/8] examples/power: add host channel to power manager
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 4/8] examples/power: add host channel to power manager David Hunt
@ 2018-09-25  9:48       ` Burakov, Anatoly
  2018-09-25 13:55         ` Hunt, David
  0 siblings, 1 reply; 110+ messages in thread
From: Burakov, Anatoly @ 2018-09-25  9:48 UTC (permalink / raw)
  To: David Hunt, dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson

On 14-Sep-18 2:54 PM, David Hunt wrote:
> This patch adds a fifo channel to the vm_power_manager app through which
> we can send commands and polices. Intended for sending JSON strings.
> The fifo is at /tmp/powermonitor/fifo.0
> 
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---

A bunch of nitpick comments below :)

>   examples/vm_power_manager/channel_manager.c | 108 +++++++++++++++
>   examples/vm_power_manager/channel_manager.h |  17 ++-
>   examples/vm_power_manager/channel_monitor.c | 146 +++++++++++++++-----
>   examples/vm_power_manager/main.c            |   2 +
>   4 files changed, 238 insertions(+), 35 deletions(-)
> 

<snip>

> +				"Error(%s) setting non-blocking "
> +				"socket for '%s'\n",
> +				strerror(errno), info->channel_path);
> +		return -1;
> +	}
> +	return 0;
> +}
> +

As far as i can tell, vm power manager is a proper DPDK application, 
meaning there can technically be several of these running independently 
under different prefixes. Hardcoded paths are OK, but you probably need 
to place a write-lock on a file to prevent another VM power manager from 
(accidentally) taking over the FIFO? Init would probably fail earlier, 
but you never know :)

>   static int
>   setup_channel_info(struct virtual_machine_info **vm_info_dptr,
>   		struct channel_info **chan_info_dptr, unsigned channel_num)
> @@ -294,6 +327,7 @@ setup_channel_info(struct virtual_machine_info **vm_info_dptr,
>   	chan_info->channel_num = channel_num;
>   	chan_info->priv_info = (void *)vm_info;
>   	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
> +	chan_info->type = CHANNEL_TYPE_BINARY;
>   	if (open_non_blocking_channel(chan_info) < 0) {
>   		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open channel: "
>   				"'%s' for VM '%s'\n",
> @@ -316,6 +350,35 @@ setup_channel_info(struct virtual_machine_info **vm_info_dptr,
>   	return 0;
>   }
>   
> +static int
> +setup_host_channel_info(struct channel_info **chan_info_dptr,
> +		unsigned int channel_num)
> +{
> +	struct channel_info *chan_info = *chan_info_dptr;
> +
> +	chan_info->channel_num = channel_num;
> +	chan_info->priv_info = (void *)0;

NULL?

> +	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
> +	chan_info->type = CHANNEL_TYPE_JSON;
> +	sprintf(chan_info->channel_path, "%sfifo.0", CHANNEL_MGR_SOCKET_PATH);

Here, 0 is part of the format string...

> +
> +	if (open_host_channel(chan_info) < 0) {
> +		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open host channel: "
> +				"'%s'\n",
> +				chan_info->channel_path);
> +		return -1;
> +	}

<snip>

> +int
> +add_host_channel(void)
> +{
> +	struct channel_info *chan_info;
> +	char socket_path[PATH_MAX];
> +	int num_channels_enabled = 0;
> +	int ret;
> +
> +	snprintf(socket_path, sizeof(socket_path), "%sfifo.%u",
> +			CHANNEL_MGR_SOCKET_PATH, 0);

...while here, it's an argument. What's the significance of 0 in this 
context? Also, maybe better to put it in a function, so as to only have 
one place to fix if anything changes, instead of two?

> +
> +	errno = 0;
> +	ret = mkfifo(socket_path, 0666);

0666 seems like overly permissive to me?

> +	if ((errno != EEXIST) && (ret < 0)) {
> +		printf(" %d %d, %d\n", ret, EEXIST, errno);

This looks like a leftover debug printf?

Also, maybe if (ret < 0 && errno != EEXIST)? I don't think there's a 
need to set errno beforehand here.

> +		RTE_LOG(ERR, CHANNEL_MANAGER, "Cannot create fifo '%s' error: "
> +				"%s\n", socket_path, strerror(errno));
> +		return 0;
> +	}
> +
> +	errno = 0;

...and here too - if access() call failed, does it not always set errno 
value?

> +	if (access(socket_path, F_OK) < 0) {
> +		RTE_LOG(ERR, CHANNEL_MANAGER, "Channel path '%s' error: "
> +				"%s\n", socket_path, strerror(errno));
> +		return 0;
> +	}
> +	chan_info = rte_malloc(NULL, sizeof(*chan_info),
> +			RTE_CACHE_LINE_SIZE);

0 alignment is equivalent to RTE_CACHE_LINE_SIZE, so no need to specify 
it explicitly.

> +	if (chan_info == NULL) {
> +		RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for "
> +				"channel '%s'\n", socket_path);
> +		return 0;
> +	}
> +	snprintf(chan_info->channel_path,
> +			sizeof(chan_info->channel_path), "%sfifo.%u",
> +			CHANNEL_MGR_SOCKET_PATH, 0);

Creating FIFO path again. Definitely needs a function :)

> +	if (setup_host_channel_info(&chan_info, 0) < 0) {
> +		rte_free(chan_info);
> +		return 0;
> +	}
> +	num_channels_enabled++;
> +
> +	return num_channels_enabled;
> +}
> +
>   int
>   remove_channel(struct channel_info **chan_info_dptr)
>   {
> diff --git a/examples/vm_power_manager/channel_manager.h b/examples/vm_power_manager/channel_manager.h
> index 872ec6140..c157cc22b 100644
> --- a/examples/vm_power_manager/channel_manager.h
> +++ b/examples/vm_power_manager/channel_manager.h
> @@ -37,7 +37,7 @@ struct sockaddr_un _sockaddr_un;
>   #define UNIX_PATH_MAX sizeof(_sockaddr_un.sun_path)
>   #endif
>   
> -#define MAX_VMS 4
> +#define MAX_VMS 64

This change probably needs to be called out in commit message and 
explained. Or broken into a separate commit? Also, i think technically 
"MAX_VMS" is a bad name now that you're supporting containers as well as 
VM's. MAX_CLIENTS maybe?

>   #define MAX_VCPUS 20
>   
>   
> @@ -54,6 +54,11 @@ enum channel_status { CHANNEL_MGR_CHANNEL_DISCONNECTED = 0,
>   	CHANNEL_MGR_CHANNEL_DISABLED,
>   	CHANNEL_MGR_CHANNEL_PROCESSING};
>   
> +/* Communication Channel Type */
> +enum channel_type { CHANNEL_TYPE_BINARY = 0,

Should probably start values on a new line?

> +	CHANNEL_TYPE_INI,
> +	CHANNEL_TYPE_JSON};
> +
>   /* VM libvirt(qemu/KVM) connection status */
>   enum vm_status { CHANNEL_MGR_VM_INACTIVE = 0, CHANNEL_MGR_VM_ACTIVE};
>   
> @@ -66,6 +71,7 @@ struct channel_info {
>   	volatile uint32_t status;    /**< Connection status(enum channel_status) */

<snip>

> -					pol->core_share[count].pcpu = pcpu;
> -					printf("Monitoring pcpu %d\n", pcpu);
> -				}
> +	RTE_LOG(INFO, CHANNEL_MONITOR,
> +			"Looking for pcpu for %s\n", pol->pkt.vm_name);
> +
> +	/*
> +	 * So now that we're handling virtual and physical cores, we need to
> +	 * differenciate between them when adding them to the branch monitor.
> +	 * Virtual cores need to be converted to physical cores.
> +	 */
> +
> +
> +
> +

Needs moar newlines :)

> +	if (pol->pkt.core_type == CORE_TYPE_VIRTUAL) {
> +		/*
> +		 * If the cores in the policy are virtual, we need to map them
> +		 * to physical core. We look up the vm info and use that for
> +		 * the mapping.
> +		 */
> +		get_info_vm(pol->pkt.vm_name, &info);

<snip>

> @@ -362,10 +425,12 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
>   	if (pkt->command == CPU_POWER) {
>   		core_mask = get_pcpus_mask(chan_info, pkt->resource_id);
>   		if (core_mask == 0) {
> -			RTE_LOG(ERR, CHANNEL_MONITOR, "Error get physical CPU mask for "
> -				"channel '%s' using vCPU(%u)\n", chan_info->channel_path,
> -				(unsigned)pkt->unit);
> -			return -1;
> +			/*
> +			 * Core mask will be 0 in the case where
> +			 * hypervisor is not available so we're working in
> +			 * the host, so use the core as the mask.
> +			 */
> +			core_mask = 1 << pkt->resource_id;

1ULL?


-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v3 5/8] examples/power: add json string handling
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 5/8] examples/power: add json string handling David Hunt
@ 2018-09-25 11:27       ` Burakov, Anatoly
  2018-09-25 14:00         ` Hunt, David
  0 siblings, 1 reply; 110+ messages in thread
From: Burakov, Anatoly @ 2018-09-25 11:27 UTC (permalink / raw)
  To: David Hunt, dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson

On 14-Sep-18 2:54 PM, David Hunt wrote:
> Add JSON string handling to vm_power_manager for JSON strings received
> through the fifo. The format of the JSON strings are detailed in the
> next patch, the vm_power_manager user guide documentation updates.
> 
> This patch introduces a new dependency on Jansson, a C library for
> encoding, decoding and manipulating JSON data. To compile the sample app
> you now need to have installed libjansson4 and libjansson-dev (these may
> be named slightly differently depending on your Operating System)
> 
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---

<snip>

>   void channel_monitor_exit(void)
>   {
>   	run_loop = 0;
> @@ -124,18 +259,11 @@ get_pcpu_to_control(struct policy *pol)
>   
>   	ci = get_core_info();
>   
> -	RTE_LOG(INFO, CHANNEL_MONITOR,
> -			"Looking for pcpu for %s\n", pol->pkt.vm_name);
> -
>   	/*
>   	 * So now that we're handling virtual and physical cores, we need to
>   	 * differenciate between them when adding them to the branch monitor.
>   	 * Virtual cores need to be converted to physical cores.
>   	 */
> -
> -
> -
> -

Now you're removing those newlines you added in previous commit :)

>   	if (pol->pkt.core_type == CORE_TYPE_VIRTUAL) {
>   		/*
>   		 * If the cores in the policy are virtual, we need to map them
> @@ -295,8 +423,6 @@ apply_traffic_profile(struct policy *pol)
>   
>   	diff = get_pkt_diff(pol);
>   
> -	RTE_LOG(INFO, CHANNEL_MONITOR, "Applying traffic profile\n");
> -

Here and in a few other places: these log message removals look to be 
unrelated to this commit. Also, in my experience, more logging is better 
than less logging, especially when something goes wrong - maybe instead 
of removing them, just switch the level to debug?

>   	if (diff >= (pol->pkt.traffic_policy.max_max_packet_thresh)) {
>   		for (count = 0; count < pol->pkt.num_vcpu; count++) {
>   			if (pol->core_share[count].status != 1)
> @@ -340,9 +466,6 @@ apply_time_profile(struct policy *pol)
>   				if (pol->core_share[count].status != 1) {
>   					power_manager_scale_core_max(
>   						pol->core_share[count].pcpu);
> -				RTE_LOG(INFO, CHANNEL_MONITOR,

<snip>

> +		int idx = 0;
> +		int indent = 0;
> +		do {
> +			n_bytes = read(chan_info->fd, &json_data[idx], 1);
> +			if (n_bytes == 0)
> +				break;
> +			if (json_data[idx] == '{')
> +				indent++;
> +			if (json_data[idx] == '}')
> +				indent--;

What happens if someone sends a string with a "{" or "}" inside?

> +			if ((indent > 0) || (idx >> 0))

idx > 0?

> +				idx++;
> +			if (indent == 0)
> +				json_data[idx] = 0;
> +			if (idx >= MAX_JSON_STRING_LEN)
> +				break;

This looks like a potential overflow to me, because you increment idx, 
check if it's >= max, but still write into that location if indent == 0 
on previous line.

> +		} while (indent > 0);
> +
> +		if (indent > 0)
> +			/*
> +			 * We've broken out of the read loop without getting
> +			 * a closing brace, so throw away the datai

I think "data" is plural already, no need to invent a new word :)

> +			 */
> +			json_data[idx] = 0;

idx could potentially be overflown already due to code above?

> +
> +		if (strlen(json_data) == 0)
> +			continue;
> +
> +		printf("got [%s]\n", json_data);
> +

<snip>

>   void
>   run_channel_monitor(void)
>   {
> @@ -570,11 +785,16 @@ run_channel_monitor(void)
>   		if (!run_loop)
>   			break;
>   		for (i = 0; i < n_events; i++) {
> +			if (!global_events_list[i].data.ptr) {
> +				fflush(stdout);
> +				continue;
> +			}

This change looks unrelated to this commit.

>   			struct channel_info *chan_info = (struct channel_info *)
>   					global_events_list[i].data.ptr;
>   			if ((global_events_list[i].events & EPOLLERR) ||

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v3 1/8] examples/power: add checks around hypervisor
  2018-09-25  9:20       ` Burakov, Anatoly
@ 2018-09-25 13:47         ` Hunt, David
  0 siblings, 0 replies; 110+ messages in thread
From: Hunt, David @ 2018-09-25 13:47 UTC (permalink / raw)
  To: Burakov, Anatoly, dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson



On 25/9/2018 10:20 AM, Burakov, Anatoly wrote:
> On 14-Sep-18 2:53 PM, David Hunt wrote:
>> Allow vm_power_manager to run without requiring qemu to be present
>> on the machine. This will be required for instances where the JSON
>> interface is used for commands and polices, without any VMs present.
>> A use case for this is a container enviromnent.
>>
>> Signed-off-by: David Hunt <david.hunt@intel.com>
>> ---
>
> <snip>
>
>> --- a/examples/vm_power_manager/channel_monitor.c
>> +++ b/examples/vm_power_manager/channel_monitor.c
>> @@ -66,7 +66,7 @@ static void
>>   core_share_status(int pNo)
>>   {
>>   -    int noVms, noVcpus, z, x, t;
>> +    int noVms = 0, noVcpus = 0, z, x, t;
>
> This looks like a unrelated change?
>
>>       get_all_vm(&noVms, &noVcpus);
>>
>
>
I'll move it to a separate commit in the next version.

Thanks,
Dave.

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v3 2/8] lib/power: add changes for host commands/policies
  2018-09-25  9:21       ` Burakov, Anatoly
@ 2018-09-25 13:47         ` Hunt, David
  0 siblings, 0 replies; 110+ messages in thread
From: Hunt, David @ 2018-09-25 13:47 UTC (permalink / raw)
  To: Burakov, Anatoly, dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson



On 25/9/2018 10:21 AM, Burakov, Anatoly wrote:
> On 14-Sep-18 2:54 PM, David Hunt wrote:
>> Signed-off-by: David Hunt <david.hunt@intel.com>
>
> The commit message is not very descriptive as to what these chages are 
> and why they are added.
>

Sure, I'll add a more descriptive comment next version.

Thanks,
Dave.

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v3 4/8] examples/power: add host channel to power manager
  2018-09-25  9:48       ` Burakov, Anatoly
@ 2018-09-25 13:55         ` Hunt, David
  0 siblings, 0 replies; 110+ messages in thread
From: Hunt, David @ 2018-09-25 13:55 UTC (permalink / raw)
  To: Burakov, Anatoly, dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson

Hi Anatoly,


On 25/9/2018 10:48 AM, Burakov, Anatoly wrote:
> On 14-Sep-18 2:54 PM, David Hunt wrote:
>> This patch adds a fifo channel to the vm_power_manager app through which
>> we can send commands and polices. Intended for sending JSON strings.
>> The fifo is at /tmp/powermonitor/fifo.0
>>
>> Signed-off-by: David Hunt <david.hunt@intel.com>
>> ---
>
> A bunch of nitpick comments below :)
>
>> examples/vm_power_manager/channel_manager.c | 108 +++++++++++++++
>>   examples/vm_power_manager/channel_manager.h |  17 ++-
>>   examples/vm_power_manager/channel_monitor.c | 146 +++++++++++++++-----
>>   examples/vm_power_manager/main.c            |   2 +
>>   4 files changed, 238 insertions(+), 35 deletions(-)
>>
>
> <snip>
>
>> +                "Error(%s) setting non-blocking "
>> +                "socket for '%s'\n",
>> +                strerror(errno), info->channel_path);
>> +        return -1;
>> +    }
>> +    return 0;
>> +}
>> +
>
> As far as i can tell, vm power manager is a proper DPDK application, 
> meaning there can technically be several of these running 
> independently under different prefixes. Hardcoded paths are OK, but 
> you probably need to place a write-lock on a file to prevent another 
> VM power manager from (accidentally) taking over the FIFO? Init would 
> probably fail earlier, but you never know :)

vm_power_manager is only expected to be a single instance looking after 
the power across all cores in a system, so we do not expect multiple 
instance of this to be running.

>
>>   static int
>>   setup_channel_info(struct virtual_machine_info **vm_info_dptr,
>>           struct channel_info **chan_info_dptr, unsigned channel_num)
>> @@ -294,6 +327,7 @@ setup_channel_info(struct virtual_machine_info 
>> **vm_info_dptr,
>>       chan_info->channel_num = channel_num;
>>       chan_info->priv_info = (void *)vm_info;
>>       chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
>> +    chan_info->type = CHANNEL_TYPE_BINARY;
>>       if (open_non_blocking_channel(chan_info) < 0) {
>>           RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open channel: "
>>                   "'%s' for VM '%s'\n",
>> @@ -316,6 +350,35 @@ setup_channel_info(struct virtual_machine_info 
>> **vm_info_dptr,
>>       return 0;
>>   }
>>   +static int
>> +setup_host_channel_info(struct channel_info **chan_info_dptr,
>> +        unsigned int channel_num)
>> +{
>> +    struct channel_info *chan_info = *chan_info_dptr;
>> +
>> +    chan_info->channel_num = channel_num;
>> +    chan_info->priv_info = (void *)0;
>
> NULL?
>

Fixed in next version.


>> +    chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
>> +    chan_info->type = CHANNEL_TYPE_JSON;
>> +    sprintf(chan_info->channel_path, "%sfifo.0", 
>> CHANNEL_MGR_SOCKET_PATH);
>
> Here, 0 is part of the format string...
>
>> +
>> +    if (open_host_channel(chan_info) < 0) {
>> +        RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open host channel: "
>> +                "'%s'\n",
>> +                chan_info->channel_path);
>> +        return -1;
>> +    }
>
> <snip>
>
>> +int
>> +add_host_channel(void)
>> +{
>> +    struct channel_info *chan_info;
>> +    char socket_path[PATH_MAX];
>> +    int num_channels_enabled = 0;
>> +    int ret;
>> +
>> +    snprintf(socket_path, sizeof(socket_path), "%sfifo.%u",
>> +            CHANNEL_MGR_SOCKET_PATH, 0);
>
> ...while here, it's an argument. What's the significance of 0 in this 
> context? Also, maybe better to put it in a function, so as to only 
> have one place to fix if anything changes, instead of two?
>

I'll add a function to generate this string. And I've dropped the ".0". 
It was inherited from the VM channel code, and in this case there is no 
index needed.


>> +
>> +    errno = 0;
>> +    ret = mkfifo(socket_path, 0666);
>
> 0666 seems like overly permissive to me?
>

changed to 660


>> +    if ((errno != EEXIST) && (ret < 0)) {
>> +        printf(" %d %d, %d\n", ret, EEXIST, errno);
>
> This looks like a leftover debug printf?
>

Removed in next rev.


> Also, maybe if (ret < 0 && errno != EEXIST)? I don't think there's a 
> need to set errno beforehand here.
>
>> +        RTE_LOG(ERR, CHANNEL_MANAGER, "Cannot create fifo '%s' error: "
>> +                "%s\n", socket_path, strerror(errno));
>> +        return 0;
>> +    }
>> +
>> +    errno = 0;
>
> ...and here too - if access() call failed, does it not always set 
> errno value?


Removed in next rev.

>
>> +    if (access(socket_path, F_OK) < 0) {
>> +        RTE_LOG(ERR, CHANNEL_MANAGER, "Channel path '%s' error: "
>> +                "%s\n", socket_path, strerror(errno));
>> +        return 0;
>> +    }
>> +    chan_info = rte_malloc(NULL, sizeof(*chan_info),
>> +            RTE_CACHE_LINE_SIZE);
>
> 0 alignment is equivalent to RTE_CACHE_LINE_SIZE, so no need to 
> specify it explicitly.
>

Sure. Changed to 0.

>> +    if (chan_info == NULL) {
>> +        RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for "
>> +                "channel '%s'\n", socket_path);
>> +        return 0;
>> +    }
>> +    snprintf(chan_info->channel_path,
>> +            sizeof(chan_info->channel_path), "%sfifo.%u",
>> +            CHANNEL_MGR_SOCKET_PATH, 0);
>
> Creating FIFO path again. Definitely needs a function :)
>

Function created. :)

>> +    if (setup_host_channel_info(&chan_info, 0) < 0) {
>> +        rte_free(chan_info);
>> +        return 0;
>> +    }
>> +    num_channels_enabled++;
>> +
>> +    return num_channels_enabled;
>> +}
>> +
>>   int
>>   remove_channel(struct channel_info **chan_info_dptr)
>>   {
>> diff --git a/examples/vm_power_manager/channel_manager.h 
>> b/examples/vm_power_manager/channel_manager.h
>> index 872ec6140..c157cc22b 100644
>> --- a/examples/vm_power_manager/channel_manager.h
>> +++ b/examples/vm_power_manager/channel_manager.h
>> @@ -37,7 +37,7 @@ struct sockaddr_un _sockaddr_un;
>>   #define UNIX_PATH_MAX sizeof(_sockaddr_un.sun_path)
>>   #endif
>>   -#define MAX_VMS 4
>> +#define MAX_VMS 64
>
> This change probably needs to be called out in commit message and 
> explained. Or broken into a separate commit? Also, i think technically 
> "MAX_VMS" is a bad name now that you're supporting containers as well 
> as VM's. MAX_CLIENTS maybe?
>

Sure. I've split it out into it's own commit in next version, and 
renamed it as MAX_CLIENTS.


>>   #define MAX_VCPUS 20
>>     @@ -54,6 +54,11 @@ enum channel_status { 
>> CHANNEL_MGR_CHANNEL_DISCONNECTED = 0,
>>       CHANNEL_MGR_CHANNEL_DISABLED,
>>       CHANNEL_MGR_CHANNEL_PROCESSING};
>>   +/* Communication Channel Type */
>> +enum channel_type { CHANNEL_TYPE_BINARY = 0,
>
> Should probably start values on a new line?
>

I was sticking with the existing style, but I do prefer the new lines. 
Will change in the new version.


>> +    CHANNEL_TYPE_INI,
>> +    CHANNEL_TYPE_JSON};
>> +
>>   /* VM libvirt(qemu/KVM) connection status */
>>   enum vm_status { CHANNEL_MGR_VM_INACTIVE = 0, CHANNEL_MGR_VM_ACTIVE};
>>   @@ -66,6 +71,7 @@ struct channel_info {
>>       volatile uint32_t status;    /**< Connection status(enum 
>> channel_status) */
>
> <snip>
>
>> - pol->core_share[count].pcpu = pcpu;
>> -                    printf("Monitoring pcpu %d\n", pcpu);
>> -                }
>> +    RTE_LOG(INFO, CHANNEL_MONITOR,
>> +            "Looking for pcpu for %s\n", pol->pkt.vm_name);
>> +
>> +    /*
>> +     * So now that we're handling virtual and physical cores, we 
>> need to
>> +     * differenciate between them when adding them to the branch 
>> monitor.
>> +     * Virtual cores need to be converted to physical cores.
>> +     */
>> +
>> +
>> +
>> +
>
> Needs moar newlines :)
>

One can never have enough newlines. :)
Removed in the next version.


>> +    if (pol->pkt.core_type == CORE_TYPE_VIRTUAL) {
>> +        /*
>> +         * If the cores in the policy are virtual, we need to map them
>> +         * to physical core. We look up the vm info and use that for
>> +         * the mapping.
>> +         */
>> +        get_info_vm(pol->pkt.vm_name, &info);
>
> <snip>
>
>> @@ -362,10 +425,12 @@ process_request(struct channel_packet *pkt, 
>> struct channel_info *chan_info)
>>       if (pkt->command == CPU_POWER) {
>>           core_mask = get_pcpus_mask(chan_info, pkt->resource_id);
>>           if (core_mask == 0) {
>> -            RTE_LOG(ERR, CHANNEL_MONITOR, "Error get physical CPU 
>> mask for "
>> -                "channel '%s' using vCPU(%u)\n", 
>> chan_info->channel_path,
>> -                (unsigned)pkt->unit);
>> -            return -1;
>> +            /*
>> +             * Core mask will be 0 in the case where
>> +             * hypervisor is not available so we're working in
>> +             * the host, so use the core as the mask.
>> +             */
>> +            core_mask = 1 << pkt->resource_id;
>
> 1ULL?
>
>

Done in next version .

Thanks for the review, Anatoly.

Rgds,
Dave.

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v3 5/8] examples/power: add json string handling
  2018-09-25 11:27       ` Burakov, Anatoly
@ 2018-09-25 14:00         ` Hunt, David
  2018-09-25 14:15           ` Burakov, Anatoly
  0 siblings, 1 reply; 110+ messages in thread
From: Hunt, David @ 2018-09-25 14:00 UTC (permalink / raw)
  To: Burakov, Anatoly, dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson

Hi Anatoly,


On 25/9/2018 12:27 PM, Burakov, Anatoly wrote:
> On 14-Sep-18 2:54 PM, David Hunt wrote:
>> Add JSON string handling to vm_power_manager for JSON strings received
>> through the fifo. The format of the JSON strings are detailed in the
>> next patch, the vm_power_manager user guide documentation updates.
>>
>> This patch introduces a new dependency on Jansson, a C library for
>> encoding, decoding and manipulating JSON data. To compile the sample app
>> you now need to have installed libjansson4 and libjansson-dev (these may
>> be named slightly differently depending on your Operating System)
>>
>> Signed-off-by: David Hunt <david.hunt@intel.com>
>> ---
>
> <snip>
>
>>   void channel_monitor_exit(void)
>>   {
>>       run_loop = 0;
>> @@ -124,18 +259,11 @@ get_pcpu_to_control(struct policy *pol)
>>         ci = get_core_info();
>>   -    RTE_LOG(INFO, CHANNEL_MONITOR,
>> -            "Looking for pcpu for %s\n", pol->pkt.vm_name);
>> -
>>       /*
>>        * So now that we're handling virtual and physical cores, we 
>> need to
>>        * differenciate between them when adding them to the branch 
>> monitor.
>>        * Virtual cores need to be converted to physical cores.
>>        */
>> -
>> -
>> -
>> -
>
> Now you're removing those newlines you added in previous commit :)
>

Fixed in previous patch in the next version.


>>       if (pol->pkt.core_type == CORE_TYPE_VIRTUAL) {
>>           /*
>>            * If the cores in the policy are virtual, we need to map them
>> @@ -295,8 +423,6 @@ apply_traffic_profile(struct policy *pol)
>>         diff = get_pkt_diff(pol);
>>   -    RTE_LOG(INFO, CHANNEL_MONITOR, "Applying traffic profile\n");
>> -
>
> Here and in a few other places: these log message removals look to be 
> unrelated to this commit. Also, in my experience, more logging is 
> better than less logging, especially when something goes wrong - maybe 
> instead of removing them, just switch the level to debug?
>

Changed to Debug instead. Was causing quite a verbose output.

>>       if (diff >= (pol->pkt.traffic_policy.max_max_packet_thresh)) {
>>           for (count = 0; count < pol->pkt.num_vcpu; count++) {
>>               if (pol->core_share[count].status != 1)
>> @@ -340,9 +466,6 @@ apply_time_profile(struct policy *pol)
>>                   if (pol->core_share[count].status != 1) {
>>                       power_manager_scale_core_max(
>>                           pol->core_share[count].pcpu);
>> -                RTE_LOG(INFO, CHANNEL_MONITOR,
>
> <snip>
>
>> +        int idx = 0;
>> +        int indent = 0;
>> +        do {
>> +            n_bytes = read(chan_info->fd, &json_data[idx], 1);
>> +            if (n_bytes == 0)
>> +                break;
>> +            if (json_data[idx] == '{')
>> +                indent++;
>> +            if (json_data[idx] == '}')
>> +                indent--;
>
> What happens if someone sends a string with a "{" or "}" inside?
>

If we get to the end of the buffer without a "}", it calls the library 
to convert, will fail, and move on.  No damage done (I hope).
Also, a short un-terminated (by "}") string will also exit when no 
characters read.
So any invalid JSON string that's send to Jansson will fail to parse, 
and the application will be ready for the next (hopefully valid) JSON 
string.

>> +            if ((indent > 0) || (idx >> 0))
>
> idx > 0?
>

Yes, will fix.

>> +                idx++;
>> +            if (indent == 0)
>> +                json_data[idx] = 0;
>> +            if (idx >= MAX_JSON_STRING_LEN)
>> +                break;
>
> This looks like a potential overflow to me, because you increment idx, 
> check if it's >= max, but still write into that location if indent == 
> 0 on previous line.
>

Yes, will exit at MAX_JSON_STRING_LEN-1

>> +        } while (indent > 0);
>> +
>> +        if (indent > 0)
>> +            /*
>> +             * We've broken out of the read loop without getting
>> +             * a closing brace, so throw away the datai
>
> I think "data" is plural already, no need to invent a new word :)
>
>> +             */
>> +            json_data[idx] = 0;
>
> idx could potentially be overflown already due to code above?
>

Typo fixed in next version.

>> +
>> +        if (strlen(json_data) == 0)
>> +            continue;
>> +
>> +        printf("got [%s]\n", json_data);
>> +
>
> <snip>
>
>>   void
>>   run_channel_monitor(void)
>>   {
>> @@ -570,11 +785,16 @@ run_channel_monitor(void)
>>           if (!run_loop)
>>               break;
>>           for (i = 0; i < n_events; i++) {
>> +            if (!global_events_list[i].data.ptr) {
>> +                fflush(stdout);
>> +                continue;
>> +            }
>
> This change looks unrelated to this commit.

There was a few printfs in there, this change will be removed altogether 
in next version.

>
>>               struct channel_info *chan_info = (struct channel_info *)
>>                       global_events_list[i].data.ptr;
>>               if ((global_events_list[i].events & EPOLLERR) ||
>

Thanks,
Dave.

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v3 5/8] examples/power: add json string handling
  2018-09-25 14:00         ` Hunt, David
@ 2018-09-25 14:15           ` Burakov, Anatoly
  2018-09-25 15:15             ` Hunt, David
  0 siblings, 1 reply; 110+ messages in thread
From: Burakov, Anatoly @ 2018-09-25 14:15 UTC (permalink / raw)
  To: Hunt, David, dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson

On 25-Sep-18 3:00 PM, Hunt, David wrote:
>> Now you're removing those newlines you added in previous commit :)
>>
> 
> Fixed in previous patch in the next version.
> 
> 
>>>       if (pol->pkt.core_type == CORE_TYPE_VIRTUAL) {
>>>           /*
>>>            * If the cores in the policy are virtual, we need to map them
>>> @@ -295,8 +423,6 @@ apply_traffic_profile(struct policy *pol)
>>>         diff = get_pkt_diff(pol);
>>>   -    RTE_LOG(INFO, CHANNEL_MONITOR, "Applying traffic profile\n");
>>> -
>>
>> Here and in a few other places: these log message removals look to be 
>> unrelated to this commit. Also, in my experience, more logging is 
>> better than less logging, especially when something goes wrong - maybe 
>> instead of removing them, just switch the level to debug?
>>
> 
> Changed to Debug instead. Was causing quite a verbose output.

Hopefully in a separate commit :)

> 
>>>       if (diff >= (pol->pkt.traffic_policy.max_max_packet_thresh)) {
>>>           for (count = 0; count < pol->pkt.num_vcpu; count++) {
>>>               if (pol->core_share[count].status != 1)
>>> @@ -340,9 +466,6 @@ apply_time_profile(struct policy *pol)
>>>                   if (pol->core_share[count].status != 1) {
>>>                       power_manager_scale_core_max(
>>>                           pol->core_share[count].pcpu);
>>> -                RTE_LOG(INFO, CHANNEL_MONITOR,
>>
>> <snip>
>>
>>> +        int idx = 0;
>>> +        int indent = 0;
>>> +        do {
>>> +            n_bytes = read(chan_info->fd, &json_data[idx], 1);
>>> +            if (n_bytes == 0)
>>> +                break;
>>> +            if (json_data[idx] == '{')
>>> +                indent++;
>>> +            if (json_data[idx] == '}')
>>> +                indent--;
>>
>> What happens if someone sends a string with a "{" or "}" inside?
>>
> 
> If we get to the end of the buffer without a "}", it calls the library 
> to convert, will fail, and move on.  No damage done (I hope).
> Also, a short un-terminated (by "}") string will also exit when no 
> characters read.
> So any invalid JSON string that's send to Jansson will fail to parse, 
> and the application will be ready for the next (hopefully valid) JSON 
> string.

No, what i meant is something like this:

{ "json_value": "{"}

According to JSON validator, this is a valid JSON string, but it will 
break your code :)

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v3 5/8] examples/power: add json string handling
  2018-09-25 14:15           ` Burakov, Anatoly
@ 2018-09-25 15:15             ` Hunt, David
  2018-09-25 15:31               ` Burakov, Anatoly
  0 siblings, 1 reply; 110+ messages in thread
From: Hunt, David @ 2018-09-25 15:15 UTC (permalink / raw)
  To: Burakov, Anatoly, dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson



On 25/9/2018 3:15 PM, Burakov, Anatoly wrote:
> On 25-Sep-18 3:00 PM, Hunt, David wrote:
>>> Now you're removing those newlines you added in previous commit :)
>>>
>>
>> Fixed in previous patch in the next version.
>>
>>
>>>>       if (pol->pkt.core_type == CORE_TYPE_VIRTUAL) {
>>>>           /*
>>>>            * If the cores in the policy are virtual, we need to map 
>>>> them
>>>> @@ -295,8 +423,6 @@ apply_traffic_profile(struct policy *pol)
>>>>         diff = get_pkt_diff(pol);
>>>>   -    RTE_LOG(INFO, CHANNEL_MONITOR, "Applying traffic profile\n");
>>>> -
>>>
>>> Here and in a few other places: these log message removals look to 
>>> be unrelated to this commit. Also, in my experience, more logging is 
>>> better than less logging, especially when something goes wrong - 
>>> maybe instead of removing them, just switch the level to debug?
>>>
>>
>> Changed to Debug instead. Was causing quite a verbose output.
>
> Hopefully in a separate commit :)
>

Of Course. :)

>>
>>>>       if (diff >= (pol->pkt.traffic_policy.max_max_packet_thresh)) {
>>>>           for (count = 0; count < pol->pkt.num_vcpu; count++) {
>>>>               if (pol->core_share[count].status != 1)
>>>> @@ -340,9 +466,6 @@ apply_time_profile(struct policy *pol)
>>>>                   if (pol->core_share[count].status != 1) {
>>>>                       power_manager_scale_core_max(
>>>>                           pol->core_share[count].pcpu);
>>>> -                RTE_LOG(INFO, CHANNEL_MONITOR,
>>>
>>> <snip>
>>>
>>>> +        int idx = 0;
>>>> +        int indent = 0;
>>>> +        do {
>>>> +            n_bytes = read(chan_info->fd, &json_data[idx], 1);
>>>> +            if (n_bytes == 0)
>>>> +                break;
>>>> +            if (json_data[idx] == '{')
>>>> +                indent++;
>>>> +            if (json_data[idx] == '}')
>>>> +                indent--;
>>>
>>> What happens if someone sends a string with a "{" or "}" inside?
>>>
>>
>> If we get to the end of the buffer without a "}", it calls the 
>> library to convert, will fail, and move on.  No damage done (I hope).
>> Also, a short un-terminated (by "}") string will also exit when no 
>> characters read.
>> So any invalid JSON string that's send to Jansson will fail to parse, 
>> and the application will be ready for the next (hopefully valid) JSON 
>> string.
>
> No, what i meant is something like this:
>
> { "json_value": "{"}
>
> According to JSON validator, this is a valid JSON string, but it will 
> break your code :)
>

You are correct if this code was designed to be a general purpose JSON 
string reader. However, it's only designed to take in strings for this 
sample application, and they do not expect any brace characters embedded 
within quotes. So I think it's OK for this use case. Patches welcome, 
though! :)

Thanks,
Dave.

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v3 5/8] examples/power: add json string handling
  2018-09-25 15:15             ` Hunt, David
@ 2018-09-25 15:31               ` Burakov, Anatoly
  0 siblings, 0 replies; 110+ messages in thread
From: Burakov, Anatoly @ 2018-09-25 15:31 UTC (permalink / raw)
  To: Hunt, David, dev; +Cc: john.mcnamara, stephen, lei.a.yao, bruce.richardson

On 25-Sep-18 4:15 PM, Hunt, David wrote:

>>>> What happens if someone sends a string with a "{" or "}" inside?
>>>>
>>>
>>> If we get to the end of the buffer without a "}", it calls the 
>>> library to convert, will fail, and move on.  No damage done (I hope).
>>> Also, a short un-terminated (by "}") string will also exit when no 
>>> characters read.
>>> So any invalid JSON string that's send to Jansson will fail to parse, 
>>> and the application will be ready for the next (hopefully valid) JSON 
>>> string.
>>
>> No, what i meant is something like this:
>>
>> { "json_value": "{"}
>>
>> According to JSON validator, this is a valid JSON string, but it will 
>> break your code :)
>>
> 
> You are correct if this code was designed to be a general purpose JSON 
> string reader. However, it's only designed to take in strings for this 
> sample application, and they do not expect any brace characters embedded 
> within quotes. So I think it's OK for this use case. Patches welcome, 
> though! :)
> 

Fair enough :)

> Thanks,
> Dave.
> 
> 
> 


-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v4 0/11] add json power policy interface for containers
  2018-09-14 13:53   ` [dpdk-dev] [PATCH v3 0/8] add json power policy interface for containers David Hunt
                       ` (7 preceding siblings ...)
  2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 8/8] examples/power: add json example files David Hunt
@ 2018-09-26 13:40     ` David Hunt
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 01/11] examples/power: add checks around hypervisor David Hunt
                         ` (11 more replies)
  8 siblings, 12 replies; 110+ messages in thread
From: David Hunt @ 2018-09-26 13:40 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov

The current vm_power_manager example app has the capability to accept power
policies from virtual machines via virtio-serial channels. These power
policies  allow a virtual machine to give information to the power manager
to allow the power manager take care of the power management of the virtual
machine based on the information in the policy.

This power policy functionality is limited to virtual machines sending
the policies to the power manager (which runs in the Host OS), and a solution
was needed for additional methods of sending power policies to the power
manager app.

The main use-case for this modification is for containers and host
applications that wish to send polices to the power manager.

This patchset adds the capability to send power polices and power commands
to the vm_power_manager app via JSON strings through a fifo on the file
system.
For example, given the following file, policy.json:

{"policy": {
  "name": "ubuntu2",
  "command": "create",
  "policy_type": "TIME",
  "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
  "quiet_hours":[ 2, 3, 4, 5, 6 ],
  "core_list":[ 11, 12, 13 ]
}}

Then running the command:

cat policy.json >/tmp/powermonitor/fifo

The policy is sent to the vm_power_manager. The power manager app then parses
the JSON data, and inserts the policy into the array of policies.

Part of the patch series contains documentation updates to give all the
details of the valid name-value pairs, the data types, etc.

Patch v2:
  * Fixed review comments from Stephen Hemminger and Lei A Yao.
  * Added a check in the Makefile for libjansson-dev. Will Warn user and build
    without JSON functionality if not present, will build including JSON
    functionality if it is present.

Patch v3:
  * Added meson/ninja support for vm_power_manager and guest_cli apps
  * Fixed compilation issue with guest_cli app

Patch v4:
  * Split out some unrelated changes to separate patches in the set
  * Some changes out of review by Anatoly (Thanks!)

[01/11] examples/power: add checks around hypervisor
[02/11] examples/power: allow for number of vms to be zero
[03/11] lib/power: add changes for host commands/policies
[04/11] examples/power: add necessary changes to guest app
[05/11] examples/power: add host channel to power manager
[06/11] examples/power: increase allowed number of clients
[07/11] examples/power: add json string handling
[08/11] examples/power: clean up verbose messages
[09/11] examples/power: add meson/ninja build support
[10/11] doc/vm_power_manager: add JSON interface API info
[11/11] examples/power: add json example files

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v4 01/11] examples/power: add checks around hypervisor
  2018-09-26 13:40     ` [dpdk-dev] [PATCH v4 0/11] add json power policy interface for containers David Hunt
@ 2018-09-26 13:40       ` David Hunt
  2018-09-26 13:54         ` Burakov, Anatoly
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 02/11] examples/power: allow for number of vms to be zero David Hunt
                         ` (10 subsequent siblings)
  11 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-26 13:40 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Allow vm_power_manager to run without requiring qemu to be present
on the machine. This will be required for instances where the JSON
interface is used for commands and polices, without any VMs present.
A use case for this is a container enviromnent.

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/channel_manager.c | 71 +++++++++++++--------
 1 file changed, 43 insertions(+), 28 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.c b/examples/vm_power_manager/channel_manager.c
index 927fc35ab..2e471d0c1 100644
--- a/examples/vm_power_manager/channel_manager.c
+++ b/examples/vm_power_manager/channel_manager.c
@@ -43,7 +43,8 @@ static unsigned char *global_cpumaps;
 static virVcpuInfo *global_vircpuinfo;
 static size_t global_maplen;
 
-static unsigned global_n_host_cpus;
+static unsigned int global_n_host_cpus;
+static bool global_hypervisor_available;
 
 /*
  * Represents a single Virtual Machine
@@ -198,7 +199,11 @@ get_pcpus_mask(struct channel_info *chan_info, unsigned vcpu)
 {
 	struct virtual_machine_info *vm_info =
 			(struct virtual_machine_info *)chan_info->priv_info;
-	return rte_atomic64_read(&vm_info->pcpu_mask[vcpu]);
+
+	if (global_hypervisor_available && (vm_info != NULL))
+		return rte_atomic64_read(&vm_info->pcpu_mask[vcpu]);
+	else
+		return 0;
 }
 
 static inline int
@@ -559,6 +564,8 @@ get_all_vm(int *num_vm, int *num_vcpu)
 				VIR_CONNECT_LIST_DOMAINS_PERSISTENT;
 	unsigned int domain_flag = VIR_DOMAIN_VCPU_CONFIG;
 
+	if (!global_hypervisor_available)
+		return;
 
 	memset(global_cpumaps, 0, CHANNEL_CMDS_MAX_CPUS*global_maplen);
 	if (virNodeGetInfo(global_vir_conn_ptr, &node_info)) {
@@ -768,38 +775,42 @@ connect_hypervisor(const char *path)
 	}
 	return 0;
 }
-
 int
-channel_manager_init(const char *path)
+channel_manager_init(const char *path __rte_unused)
 {
 	virNodeInfo info;
 
 	LIST_INIT(&vm_list_head);
 	if (connect_hypervisor(path) < 0) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to initialize channel manager\n");
-		return -1;
-	}
-
-	global_maplen = VIR_CPU_MAPLEN(CHANNEL_CMDS_MAX_CPUS);
+		global_n_host_cpus = 64;
+		global_hypervisor_available = 0;
+		RTE_LOG(INFO, CHANNEL_MANAGER, "Unable to initialize channel manager\n");
+	} else {
+		global_hypervisor_available = 1;
+
+		global_maplen = VIR_CPU_MAPLEN(CHANNEL_CMDS_MAX_CPUS);
+
+		global_vircpuinfo = rte_zmalloc(NULL,
+				sizeof(*global_vircpuinfo) *
+				CHANNEL_CMDS_MAX_CPUS, RTE_CACHE_LINE_SIZE);
+		if (global_vircpuinfo == NULL) {
+			RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for CPU Info\n");
+			goto error;
+		}
+		global_cpumaps = rte_zmalloc(NULL,
+				CHANNEL_CMDS_MAX_CPUS * global_maplen,
+				RTE_CACHE_LINE_SIZE);
+		if (global_cpumaps == NULL)
+			goto error;
 
-	global_vircpuinfo = rte_zmalloc(NULL, sizeof(*global_vircpuinfo) *
-			CHANNEL_CMDS_MAX_CPUS, RTE_CACHE_LINE_SIZE);
-	if (global_vircpuinfo == NULL) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for CPU Info\n");
-		goto error;
-	}
-	global_cpumaps = rte_zmalloc(NULL, CHANNEL_CMDS_MAX_CPUS * global_maplen,
-			RTE_CACHE_LINE_SIZE);
-	if (global_cpumaps == NULL) {
-		goto error;
+		if (virNodeGetInfo(global_vir_conn_ptr, &info)) {
+			RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to retrieve node Info\n");
+			goto error;
+		}
+		global_n_host_cpus = (unsigned int)info.cpus;
 	}
 
-	if (virNodeGetInfo(global_vir_conn_ptr, &info)) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to retrieve node Info\n");
-		goto error;
-	}
 
-	global_n_host_cpus = (unsigned)info.cpus;
 
 	if (global_n_host_cpus > CHANNEL_CMDS_MAX_CPUS) {
 		RTE_LOG(WARNING, CHANNEL_MANAGER, "The number of host CPUs(%u) exceeds the "
@@ -811,7 +822,8 @@ channel_manager_init(const char *path)
 
 	return 0;
 error:
-	disconnect_hypervisor();
+	if (global_hypervisor_available)
+		disconnect_hypervisor();
 	return -1;
 }
 
@@ -838,7 +850,10 @@ channel_manager_exit(void)
 		rte_free(vm_info);
 	}
 
-	rte_free(global_cpumaps);
-	rte_free(global_vircpuinfo);
-	disconnect_hypervisor();
+	if (global_hypervisor_available) {
+		/* Only needed if hypervisor available */
+		rte_free(global_cpumaps);
+		rte_free(global_vircpuinfo);
+		disconnect_hypervisor();
+	}
 }
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v4 02/11] examples/power: allow for number of vms to be zero
  2018-09-26 13:40     ` [dpdk-dev] [PATCH v4 0/11] add json power policy interface for containers David Hunt
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 01/11] examples/power: add checks around hypervisor David Hunt
@ 2018-09-26 13:40       ` David Hunt
  2018-09-26 13:54         ` Burakov, Anatoly
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 03/11] lib/power: add changes for host commands/policies David Hunt
                         ` (9 subsequent siblings)
  11 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-26 13:40 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Previously the vm_power_manager app required to have some vms defined, so
the call to get_all_vm() always set the noVms variable. Now we're accepting
policies from the host OS (without any VMs defined), so it is now valid to
have zero VMs. This patch initialises the relevant variables to zero just
in case the call to get_all_vms() does not find any, so could return with
the variables uninitialised.

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/channel_monitor.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index 7fa47ba97..f180d74e6 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -66,7 +66,7 @@ static void
 core_share_status(int pNo)
 {
 
-	int noVms, noVcpus, z, x, t;
+	int noVms = 0, noVcpus = 0, z, x, t;
 
 	get_all_vm(&noVms, &noVcpus);
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v4 03/11] lib/power: add changes for host commands/policies
  2018-09-26 13:40     ` [dpdk-dev] [PATCH v4 0/11] add json power policy interface for containers David Hunt
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 01/11] examples/power: add checks around hypervisor David Hunt
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 02/11] examples/power: allow for number of vms to be zero David Hunt
@ 2018-09-26 13:40       ` David Hunt
  2018-09-26 13:54         ` Burakov, Anatoly
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 04/11] examples/power: add necessary changes to guest app David Hunt
                         ` (8 subsequent siblings)
  11 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-26 13:40 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

This patch does a couple of things:
  * Adds a new message type for removing policies (PKT_POLICY_REMOVE)
    Used when we want to remove a previously created policy.
  * Adds a core_type bool to the channel packet struct to specify whether
    the type of core we want to control is cirtual or physical.

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 lib/librte_power/channel_commands.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/lib/librte_power/channel_commands.h b/lib/librte_power/channel_commands.h
index ee638eefa..e7b93a797 100644
--- a/lib/librte_power/channel_commands.h
+++ b/lib/librte_power/channel_commands.h
@@ -19,6 +19,7 @@ extern "C" {
 #define CPU_POWER               1
 #define CPU_POWER_CONNECT       2
 #define PKT_POLICY              3
+#define PKT_POLICY_REMOVE       4
 
 /* CPU Power Command Scaling */
 #define CPU_POWER_SCALE_UP      1
@@ -58,6 +59,9 @@ struct traffic {
 	uint32_t max_max_packet_thresh;
 };
 
+#define CORE_TYPE_VIRTUAL 0
+#define CORE_TYPE_PHYSICAL 1
+
 struct channel_packet {
 	uint64_t resource_id; /**< core_num, device */
 	uint32_t unit;        /**< scale down/up/min/max */
@@ -70,6 +74,7 @@ struct channel_packet {
 	uint8_t vcpu_to_control[MAX_VCPU_PER_VM];
 	uint8_t num_vcpu;
 	struct timer_profile timer_policy;
+	bool core_type;
 	enum workload workload;
 	enum policy_to_use policy_to_use;
 	struct t_boost_status t_boost_status;
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v4 04/11] examples/power: add necessary changes to guest app
  2018-09-26 13:40     ` [dpdk-dev] [PATCH v4 0/11] add json power policy interface for containers David Hunt
                         ` (2 preceding siblings ...)
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 03/11] lib/power: add changes for host commands/policies David Hunt
@ 2018-09-26 13:40       ` David Hunt
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 05/11] examples/power: add host channel to power manager David Hunt
                         ` (7 subsequent siblings)
  11 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-26 13:40 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

The changes here are minimal, as the guest app functionality is not
changing at all, but there is a new element in the channel_packet
struct that needs to have a default set (channel_packet->core_type).

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/guest_cli/vm_power_cli_guest.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
index 0db1b804f..2d9e7689a 100644
--- a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
+++ b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
@@ -92,6 +92,7 @@ set_policy_defaults(struct channel_packet *pkt)
 	pkt->timer_policy.hours_to_use_traffic_profile[0] = 8;
 	pkt->timer_policy.hours_to_use_traffic_profile[1] = 10;
 
+	pkt->core_type = CORE_TYPE_VIRTUAL;
 	pkt->workload = LOW;
 	pkt->policy_to_use = TIME;
 	pkt->command = PKT_POLICY;
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v4 05/11] examples/power: add host channel to power manager
  2018-09-26 13:40     ` [dpdk-dev] [PATCH v4 0/11] add json power policy interface for containers David Hunt
                         ` (3 preceding siblings ...)
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 04/11] examples/power: add necessary changes to guest app David Hunt
@ 2018-09-26 13:40       ` David Hunt
  2018-09-26 14:22         ` Burakov, Anatoly
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 06/11] examples/power: increase allowed number of clients David Hunt
                         ` (6 subsequent siblings)
  11 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-26 13:40 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

This patch adds a fifo channel to the vm_power_manager app through which
we can send commands and polices. Intended for sending JSON strings.
The fifo is at /tmp/powermonitor/fifo

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/channel_manager.c | 109 +++++++++++++++
 examples/vm_power_manager/channel_manager.h |  17 +++
 examples/vm_power_manager/channel_monitor.c | 142 +++++++++++++++-----
 examples/vm_power_manager/main.c            |   2 +
 4 files changed, 236 insertions(+), 34 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.c b/examples/vm_power_manager/channel_manager.c
index 2e471d0c1..4fac099df 100644
--- a/examples/vm_power_manager/channel_manager.c
+++ b/examples/vm_power_manager/channel_manager.c
@@ -13,6 +13,7 @@
 
 #include <sys/queue.h>
 #include <sys/types.h>
+#include <sys/stat.h>
 #include <sys/socket.h>
 #include <sys/select.h>
 
@@ -284,6 +285,38 @@ open_non_blocking_channel(struct channel_info *info)
 	return 0;
 }
 
+static int
+open_host_channel(struct channel_info *info)
+{
+	int flags;
+
+	info->fd = open(info->channel_path, O_RDWR | O_RSYNC);
+	if (info->fd == -1) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Error(%s) opening fifo for '%s'\n",
+				strerror(errno),
+				info->channel_path);
+		return -1;
+	}
+
+	/* Get current flags */
+	flags = fcntl(info->fd, F_GETFL, 0);
+	if (flags < 0) {
+		RTE_LOG(WARNING, CHANNEL_MANAGER, "Error(%s) fcntl get flags socket for"
+				"'%s'\n", strerror(errno), info->channel_path);
+		return 1;
+	}
+	/* Set to Non Blocking */
+	flags |= O_NONBLOCK;
+	if (fcntl(info->fd, F_SETFL, flags) < 0) {
+		RTE_LOG(WARNING, CHANNEL_MANAGER,
+				"Error(%s) setting non-blocking "
+				"socket for '%s'\n",
+				strerror(errno), info->channel_path);
+		return -1;
+	}
+	return 0;
+}
+
 static int
 setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 		struct channel_info **chan_info_dptr, unsigned channel_num)
@@ -294,6 +327,7 @@ setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 	chan_info->channel_num = channel_num;
 	chan_info->priv_info = (void *)vm_info;
 	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
+	chan_info->type = CHANNEL_TYPE_BINARY;
 	if (open_non_blocking_channel(chan_info) < 0) {
 		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open channel: "
 				"'%s' for VM '%s'\n",
@@ -316,6 +350,42 @@ setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 	return 0;
 }
 
+static void
+fifo_path(char *dst, unsigned int len)
+{
+	snprintf(dst, len, "%sfifo", CHANNEL_MGR_SOCKET_PATH);
+}
+
+static int
+setup_host_channel_info(struct channel_info **chan_info_dptr,
+		unsigned int channel_num)
+{
+	struct channel_info *chan_info = *chan_info_dptr;
+
+	chan_info->channel_num = channel_num;
+	chan_info->priv_info = (void *)NULL;
+	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
+	chan_info->type = CHANNEL_TYPE_JSON;
+
+	fifo_path(chan_info->channel_path, sizeof(chan_info->channel_path));
+
+	if (open_host_channel(chan_info) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open host channel: "
+				"'%s'\n",
+				chan_info->channel_path);
+		return -1;
+	}
+	if (add_channel_to_monitor(&chan_info) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Could add channel: "
+				"'%s' to epoll ctl\n",
+				chan_info->channel_path);
+		return -1;
+
+	}
+	chan_info->status = CHANNEL_MGR_CHANNEL_CONNECTED;
+	return 0;
+}
+
 int
 add_all_channels(const char *vm_name)
 {
@@ -470,6 +540,45 @@ add_channels(const char *vm_name, unsigned *channel_list,
 	return num_channels_enabled;
 }
 
+int
+add_host_channel(void)
+{
+	struct channel_info *chan_info;
+	char socket_path[PATH_MAX];
+	int num_channels_enabled = 0;
+	int ret;
+
+	fifo_path(socket_path, sizeof(socket_path));
+
+	ret = mkfifo(socket_path, 0660);
+	if ((errno != EEXIST) && (ret < 0)) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Cannot create fifo '%s' error: "
+				"%s\n", socket_path, strerror(errno));
+		return 0;
+	}
+
+	if (access(socket_path, F_OK) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Channel path '%s' error: "
+				"%s\n", socket_path, strerror(errno));
+		return 0;
+	}
+	chan_info = rte_malloc(NULL, sizeof(*chan_info), 0);
+	if (chan_info == NULL) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for "
+				"channel '%s'\n", socket_path);
+		return 0;
+	}
+	snprintf(chan_info->channel_path,
+			sizeof(chan_info->channel_path), "%s", socket_path);
+	if (setup_host_channel_info(&chan_info, 0) < 0) {
+		rte_free(chan_info);
+		return 0;
+	}
+	num_channels_enabled++;
+
+	return num_channels_enabled;
+}
+
 int
 remove_channel(struct channel_info **chan_info_dptr)
 {
diff --git a/examples/vm_power_manager/channel_manager.h b/examples/vm_power_manager/channel_manager.h
index 872ec6140..e32235b07 100644
--- a/examples/vm_power_manager/channel_manager.h
+++ b/examples/vm_power_manager/channel_manager.h
@@ -54,6 +54,13 @@ enum channel_status { CHANNEL_MGR_CHANNEL_DISCONNECTED = 0,
 	CHANNEL_MGR_CHANNEL_DISABLED,
 	CHANNEL_MGR_CHANNEL_PROCESSING};
 
+/* Communication Channel Type */
+enum channel_type {
+	CHANNEL_TYPE_BINARY = 0,
+	CHANNEL_TYPE_INI,
+	CHANNEL_TYPE_JSON
+};
+
 /* VM libvirt(qemu/KVM) connection status */
 enum vm_status { CHANNEL_MGR_VM_INACTIVE = 0, CHANNEL_MGR_VM_ACTIVE};
 
@@ -66,6 +73,7 @@ struct channel_info {
 	volatile uint32_t status;    /**< Connection status(enum channel_status) */
 	int fd;                      /**< AF_UNIX socket fd */
 	unsigned channel_num;        /**< CHANNEL_MGR_SOCKET_PATH/<vm_name>.channel_num */
+	enum channel_type type;      /**< Binary, ini, json, etc. */
 	void *priv_info;             /**< Pointer to private info, do not modify */
 };
 
@@ -226,6 +234,15 @@ int add_all_channels(const char *vm_name);
 int add_channels(const char *vm_name, unsigned *channel_list,
 		unsigned num_channels);
 
+/**
+ * Set up a fifo by which host applications can send command an policies
+ * through a fifo to the vm_power_manager
+ *
+ * @return
+ *  - 0 for success
+ */
+int add_host_channel(void);
+
 /**
  * Remove a channel definition from the channel manager. This must only be
  * called from the channel monitor thread.
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index f180d74e6..c3c3d7bb1 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -85,6 +85,33 @@ core_share_status(int pNo)
 	}
 }
 
+
+static int
+pcpu_monitor(struct policy *pol, struct core_info *ci, int pcpu, int count)
+{
+	int ret = 0;
+
+	if (pol->pkt.policy_to_use == BRANCH_RATIO) {
+		ci->cd[pcpu].oob_enabled = 1;
+		ret = add_core_to_monitor(pcpu);
+		if (ret == 0)
+			RTE_LOG(INFO, CHANNEL_MONITOR,
+					"Monitoring pcpu %d OOB for %s\n",
+					pcpu, pol->pkt.vm_name);
+		else
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Error monitoring pcpu %d OOB for %s\n",
+					pcpu, pol->pkt.vm_name);
+
+	} else {
+		pol->core_share[count].pcpu = pcpu;
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				"Monitoring pcpu %d for %s\n",
+				pcpu, pol->pkt.vm_name);
+	}
+	return ret;
+}
+
 static void
 get_pcpu_to_control(struct policy *pol)
 {
@@ -94,34 +121,42 @@ get_pcpu_to_control(struct policy *pol)
 	int pcpu, count;
 	uint64_t mask_u64b;
 	struct core_info *ci;
-	int ret;
 
 	ci = get_core_info();
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Looking for pcpu for %s\n",
-			pol->pkt.vm_name);
-	get_info_vm(pol->pkt.vm_name, &info);
-
-	for (count = 0; count < pol->pkt.num_vcpu; count++) {
-		mask_u64b = info.pcpu_mask[pol->pkt.vcpu_to_control[count]];
-		for (pcpu = 0; mask_u64b; mask_u64b &= ~(1ULL << pcpu++)) {
-			if ((mask_u64b >> pcpu) & 1) {
-				if (pol->pkt.policy_to_use == BRANCH_RATIO) {
-					ci->cd[pcpu].oob_enabled = 1;
-					ret = add_core_to_monitor(pcpu);
-					if (ret == 0)
-						printf("Monitoring pcpu %d via Branch Ratio\n",
-								pcpu);
-					else
-						printf("Failed to start OOB Monitoring pcpu %d\n",
-								pcpu);
-
-				} else {
-					pol->core_share[count].pcpu = pcpu;
-					printf("Monitoring pcpu %d\n", pcpu);
-				}
+	RTE_LOG(INFO, CHANNEL_MONITOR,
+			"Looking for pcpu for %s\n", pol->pkt.vm_name);
+
+	/*
+	 * So now that we're handling virtual and physical cores, we need to
+	 * differenciate between them when adding them to the branch monitor.
+	 * Virtual cores need to be converted to physical cores.
+	 */
+	if (pol->pkt.core_type == CORE_TYPE_VIRTUAL) {
+		/*
+		 * If the cores in the policy are virtual, we need to map them
+		 * to physical core. We look up the vm info and use that for
+		 * the mapping.
+		 */
+		get_info_vm(pol->pkt.vm_name, &info);
+		for (count = 0; count < pol->pkt.num_vcpu; count++) {
+			mask_u64b =
+				info.pcpu_mask[pol->pkt.vcpu_to_control[count]];
+			for (pcpu = 0; mask_u64b;
+					mask_u64b &= ~(1ULL << pcpu++)) {
+				if ((mask_u64b >> pcpu) & 1)
+					pcpu_monitor(pol, ci, pcpu, count);
 			}
 		}
+	} else {
+		/*
+		 * If the cores in the policy are physical, we just use
+		 * those core id's directly.
+		 */
+		for (count = 0; count < pol->pkt.num_vcpu; count++) {
+			pcpu = pol->pkt.vcpu_to_control[count];
+			pcpu_monitor(pol, ci, pcpu, count);
+		}
 	}
 }
 
@@ -160,8 +195,13 @@ update_policy(struct channel_packet *pkt)
 	unsigned int updated = 0;
 	int i;
 
+
+	RTE_LOG(INFO, CHANNEL_MONITOR,
+			"Applying policy for %s\n", pkt->vm_name);
+
 	for (i = 0; i < MAX_VMS; i++) {
 		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
+			/* Copy the contents of *pkt into the policy.pkt */
 			policies[i].pkt = *pkt;
 			get_pcpu_to_control(&policies[i]);
 			if (get_pfid(&policies[i]) == -1) {
@@ -189,6 +229,24 @@ update_policy(struct channel_packet *pkt)
 	return 0;
 }
 
+static int
+remove_policy(struct channel_packet *pkt __rte_unused)
+{
+	int i;
+
+	/*
+	 * Disabling the policy is simply a case of setting
+	 * enabled to 0
+	 */
+	for (i = 0; i < MAX_VMS; i++) {
+		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
+			policies[i].enabled = 0;
+			return 0;
+		}
+	}
+	return -1;
+}
+
 static uint64_t
 get_pkt_diff(struct policy *pol)
 {
@@ -346,7 +404,6 @@ apply_policy(struct policy *pol)
 		apply_workload_profile(pol);
 }
 
-
 static int
 process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 {
@@ -355,6 +412,8 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (chan_info == NULL)
 		return -1;
 
+	RTE_LOG(INFO, CHANNEL_MONITOR, "Processing Request %s\n", pkt->vm_name);
+
 	if (rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_CONNECTED,
 			CHANNEL_MGR_CHANNEL_PROCESSING) == 0)
 		return -1;
@@ -362,10 +421,12 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (pkt->command == CPU_POWER) {
 		core_mask = get_pcpus_mask(chan_info, pkt->resource_id);
 		if (core_mask == 0) {
-			RTE_LOG(ERR, CHANNEL_MONITOR, "Error get physical CPU mask for "
-				"channel '%s' using vCPU(%u)\n", chan_info->channel_path,
-				(unsigned)pkt->unit);
-			return -1;
+			/*
+			 * Core mask will be 0 in the case where
+			 * hypervisor is not available so we're working in
+			 * the host, so use the core as the mask.
+			 */
+			core_mask = 1ULL << pkt->resource_id;
 		}
 		if (__builtin_popcountll(core_mask) == 1) {
 
@@ -421,12 +482,20 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	}
 
 	if (pkt->command == PKT_POLICY) {
-		RTE_LOG(INFO, CHANNEL_MONITOR, "\nProcessing Policy request from Guest\n");
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				"\nProcessing Policy request\n");
 		update_policy(pkt);
 		policy_is_set = 1;
 	}
 
-	/* Return is not checked as channel status may have been set to DISABLED
+	if (pkt->command == PKT_POLICY_REMOVE) {
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				 "Removing policy %s\n", pkt->vm_name);
+		remove_policy(pkt);
+	}
+
+	/*
+	 * Return is not checked as channel status may have been set to DISABLED
 	 * from management thread
 	 */
 	rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_PROCESSING,
@@ -448,13 +517,16 @@ add_channel_to_monitor(struct channel_info **chan_info)
 				"to epoll\n", info->channel_path);
 		return -1;
 	}
+	RTE_LOG(ERR, CHANNEL_MONITOR, "Added channel '%s' "
+			"to monitor\n", info->channel_path);
 	return 0;
 }
 
 int
 remove_channel_from_monitor(struct channel_info *chan_info)
 {
-	if (epoll_ctl(global_event_fd, EPOLL_CTL_DEL, chan_info->fd, NULL) < 0) {
+	if (epoll_ctl(global_event_fd, EPOLL_CTL_DEL,
+			chan_info->fd, NULL) < 0) {
 		RTE_LOG(ERR, CHANNEL_MONITOR, "Unable to remove channel '%s' "
 				"from epoll\n", chan_info->channel_path);
 		return -1;
@@ -467,11 +539,13 @@ channel_monitor_init(void)
 {
 	global_event_fd = epoll_create1(0);
 	if (global_event_fd == 0) {
-		RTE_LOG(ERR, CHANNEL_MONITOR, "Error creating epoll context with "
-				"error %s\n", strerror(errno));
+		RTE_LOG(ERR, CHANNEL_MONITOR,
+				"Error creating epoll context with error %s\n",
+				strerror(errno));
 		return -1;
 	}
-	global_events_list = rte_malloc("epoll_events", sizeof(*global_events_list)
+	global_events_list = rte_malloc("epoll_events",
+			sizeof(*global_events_list)
 			* MAX_EVENTS, RTE_CACHE_LINE_SIZE);
 	if (global_events_list == NULL) {
 		RTE_LOG(ERR, CHANNEL_MONITOR, "Unable to rte_malloc for "
diff --git a/examples/vm_power_manager/main.c b/examples/vm_power_manager/main.c
index 58c5fa45c..893bf4cdd 100644
--- a/examples/vm_power_manager/main.c
+++ b/examples/vm_power_manager/main.c
@@ -421,6 +421,8 @@ main(int argc, char **argv)
 		return -1;
 	}
 
+	add_host_channel();
+
 	printf("Running core monitor on lcore id %d\n", lcore_id);
 	rte_eal_remote_launch(run_core_monitor, NULL, lcore_id);
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v4 06/11] examples/power: increase allowed number of clients
  2018-09-26 13:40     ` [dpdk-dev] [PATCH v4 0/11] add json power policy interface for containers David Hunt
                         ` (4 preceding siblings ...)
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 05/11] examples/power: add host channel to power manager David Hunt
@ 2018-09-26 13:40       ` David Hunt
  2018-09-26 14:23         ` Burakov, Anatoly
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 07/11] examples/power: add json string handling David Hunt
                         ` (5 subsequent siblings)
  11 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-26 13:40 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Now that we're handling host policies, containers and virtual machines,
we'll rename MAX_VMS to MAX_CLIENTS, and increase from 4 to 64

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/channel_manager.h |  4 ++--
 examples/vm_power_manager/channel_monitor.c | 10 +++++-----
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.h b/examples/vm_power_manager/channel_manager.h
index e32235b07..d948b304c 100644
--- a/examples/vm_power_manager/channel_manager.h
+++ b/examples/vm_power_manager/channel_manager.h
@@ -37,7 +37,7 @@ struct sockaddr_un _sockaddr_un;
 #define UNIX_PATH_MAX sizeof(_sockaddr_un.sun_path)
 #endif
 
-#define MAX_VMS 4
+#define MAX_CLIENTS 64
 #define MAX_VCPUS 20
 
 
@@ -47,7 +47,7 @@ struct libvirt_vm_info {
 	uint8_t num_cpus;
 };
 
-struct libvirt_vm_info lvm_info[MAX_VMS];
+struct libvirt_vm_info lvm_info[MAX_CLIENTS];
 /* Communication Channel Status */
 enum channel_status { CHANNEL_MGR_CHANNEL_DISCONNECTED = 0,
 	CHANNEL_MGR_CHANNEL_CONNECTED,
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index c3c3d7bb1..53a4efe45 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -41,7 +41,7 @@ static volatile unsigned run_loop = 1;
 static int global_event_fd;
 static unsigned int policy_is_set;
 static struct epoll_event *global_events_list;
-static struct policy policies[MAX_VMS];
+static struct policy policies[MAX_CLIENTS];
 
 void channel_monitor_exit(void)
 {
@@ -199,7 +199,7 @@ update_policy(struct channel_packet *pkt)
 	RTE_LOG(INFO, CHANNEL_MONITOR,
 			"Applying policy for %s\n", pkt->vm_name);
 
-	for (i = 0; i < MAX_VMS; i++) {
+	for (i = 0; i < MAX_CLIENTS; i++) {
 		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
 			/* Copy the contents of *pkt into the policy.pkt */
 			policies[i].pkt = *pkt;
@@ -214,7 +214,7 @@ update_policy(struct channel_packet *pkt)
 		}
 	}
 	if (!updated) {
-		for (i = 0; i < MAX_VMS; i++) {
+		for (i = 0; i < MAX_CLIENTS; i++) {
 			if (policies[i].enabled == 0) {
 				policies[i].pkt = *pkt;
 				get_pcpu_to_control(&policies[i]);
@@ -238,7 +238,7 @@ remove_policy(struct channel_packet *pkt __rte_unused)
 	 * Disabling the policy is simply a case of setting
 	 * enabled to 0
 	 */
-	for (i = 0; i < MAX_VMS; i++) {
+	for (i = 0; i < MAX_CLIENTS; i++) {
 		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
 			policies[i].enabled = 0;
 			return 0;
@@ -609,7 +609,7 @@ run_channel_monitor(void)
 		if (policy_is_set) {
 			int j;
 
-			for (j = 0; j < MAX_VMS; j++) {
+			for (j = 0; j < MAX_CLIENTS; j++) {
 				if (policies[j].enabled == 1)
 					apply_policy(&policies[j]);
 			}
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v4 07/11] examples/power: add json string handling
  2018-09-26 13:40     ` [dpdk-dev] [PATCH v4 0/11] add json power policy interface for containers David Hunt
                         ` (5 preceding siblings ...)
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 06/11] examples/power: increase allowed number of clients David Hunt
@ 2018-09-26 13:40       ` David Hunt
  2018-09-26 14:24         ` Burakov, Anatoly
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 08/11] examples/power: clean up verbose messages David Hunt
                         ` (4 subsequent siblings)
  11 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-26 13:40 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Add JSON string handling to vm_power_manager for JSON strings received
through the fifo. The format of the JSON strings are detailed in the
next patch, the vm_power_manager user guide documentation updates.

This patch introduces a new dependency on Jansson, a C library for
encoding, decoding and manipulating JSON data. To compile the sample app
you now need to have installed libjansson4 and libjansson-dev (these may
be named slightly differently depending on your Operating System)

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/Makefile          |   6 +
 examples/vm_power_manager/channel_monitor.c | 269 ++++++++++++++++++--
 2 files changed, 250 insertions(+), 25 deletions(-)

diff --git a/examples/vm_power_manager/Makefile b/examples/vm_power_manager/Makefile
index 13a5205ba..50147c05d 100644
--- a/examples/vm_power_manager/Makefile
+++ b/examples/vm_power_manager/Makefile
@@ -31,6 +31,12 @@ CFLAGS += $(WERROR_FLAGS)
 
 LDLIBS += -lvirt
 
+JANSSON := $(shell pkg-config --exists jansson; echo $$?)
+ifeq ($(JANSSON), 0)
+LDLIBS += $(shell pkg-config --libs jansson)
+CFLAGS += -DUSE_JANSSON
+endif
+
 ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
 
 ifeq ($(CONFIG_RTE_LIBRTE_IXGBE_PMD),y)
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index 53a4efe45..73a0451f3 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -9,11 +9,18 @@
 #include <signal.h>
 #include <errno.h>
 #include <string.h>
+#include <fcntl.h>
 #include <sys/types.h>
 #include <sys/epoll.h>
 #include <sys/queue.h>
 #include <sys/time.h>
-
+#include <sys/socket.h>
+#include <sys/select.h>
+#ifdef USE_JANSSON
+#include <jansson.h>
+#else
+#pragma message "Jansson dev libs unavailable, not including JSON parsing"
+#endif
 #include <rte_log.h>
 #include <rte_memory.h>
 #include <rte_malloc.h>
@@ -35,6 +42,8 @@
 
 uint64_t vsi_pkt_count_prev[384];
 uint64_t rdtsc_prev[384];
+#define MAX_JSON_STRING_LEN 1024
+char json_data[MAX_JSON_STRING_LEN];
 
 double time_period_ms = 1;
 static volatile unsigned run_loop = 1;
@@ -43,6 +52,132 @@ static unsigned int policy_is_set;
 static struct epoll_event *global_events_list;
 static struct policy policies[MAX_CLIENTS];
 
+#ifdef USE_JANSSON
+static int
+parse_json_to_pkt(json_t *element, struct channel_packet *pkt)
+{
+	const char *key;
+	json_t *value;
+	int ret;
+
+	memset(pkt, 0, sizeof(struct channel_packet));
+
+	pkt->nb_mac_to_monitor = 0;
+	pkt->t_boost_status.tbEnabled = false;
+	pkt->workload = LOW;
+	pkt->policy_to_use = TIME;
+	pkt->command = PKT_POLICY;
+	pkt->core_type = CORE_TYPE_PHYSICAL;
+
+	json_object_foreach(element, key, value) {
+		if (!strcmp(key, "policy")) {
+			/* Recurse in to get the contents of profile */
+			ret = parse_json_to_pkt(value, pkt);
+			if (ret)
+				return ret;
+		} else if (!strcmp(key, "instruction")) {
+			/* Recurse in to get the contents of instruction */
+			ret = parse_json_to_pkt(value, pkt);
+			if (ret)
+				return ret;
+		} else if (!strcmp(key, "name")) {
+			strcpy(pkt->vm_name, json_string_value(value));
+		} else if (!strcmp(key, "command")) {
+			char command[32];
+			snprintf(command, 32, "%s", json_string_value(value));
+			if (!strcmp(command, "power")) {
+				pkt->command = CPU_POWER;
+			} else if (!strcmp(command, "create")) {
+				pkt->command = PKT_POLICY;
+			} else if (!strcmp(command, "destroy")) {
+				pkt->command = PKT_POLICY_REMOVE;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Invalid command received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "policy_type")) {
+			char command[32];
+			snprintf(command, 32, "%s", json_string_value(value));
+			if (!strcmp(command, "TIME")) {
+				pkt->policy_to_use = TIME;
+			} else if (!strcmp(command, "TRAFFIC")) {
+				pkt->policy_to_use = TRAFFIC;
+			} else if (!strcmp(command, "WORKLOAD")) {
+				pkt->policy_to_use = WORKLOAD;
+			} else if (!strcmp(command, "BRANCH_RATIO")) {
+				pkt->policy_to_use = BRANCH_RATIO;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Wrong policy_type received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "busy_hours")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int hour = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->timer_policy.busy_hours[i] = hour;
+			}
+		} else if (!strcmp(key, "quiet_hours")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int hour = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->timer_policy.quiet_hours[i] = hour;
+			}
+		} else if (!strcmp(key, "core_list")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int core = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->vcpu_to_control[i] = core;
+			}
+			pkt->num_vcpu = size;
+		} else if (!strcmp(key, "avg_packet_thresh")) {
+			pkt->traffic_policy.avg_max_packet_thresh =
+					(uint32_t)json_integer_value(value);
+		} else if (!strcmp(key, "max_packet_thresh")) {
+			pkt->traffic_policy.max_max_packet_thresh =
+					(uint32_t)json_integer_value(value);
+		} else if (!strcmp(key, "unit")) {
+			char unit[32];
+			snprintf(unit, 32, "%s", json_string_value(value));
+			if (!strcmp(unit, "SCALE_UP")) {
+				pkt->unit = CPU_POWER_SCALE_UP;
+			} else if (!strcmp(unit, "SCALE_DOWN")) {
+				pkt->unit = CPU_POWER_SCALE_DOWN;
+			} else if (!strcmp(unit, "SCALE_MAX")) {
+				pkt->unit = CPU_POWER_SCALE_MAX;
+			} else if (!strcmp(unit, "SCALE_MIN")) {
+				pkt->unit = CPU_POWER_SCALE_MIN;
+			} else if (!strcmp(unit, "ENABLE_TURBO")) {
+				pkt->unit = CPU_POWER_ENABLE_TURBO;
+			} else if (!strcmp(unit, "DISABLE_TURBO")) {
+				pkt->unit = CPU_POWER_DISABLE_TURBO;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Invalid command received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "resource_id")) {
+			pkt->resource_id = (uint32_t)json_integer_value(value);
+		} else {
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+				"Unknown key received in JSON string: %s\n",
+				key);
+		}
+	}
+	return 0;
+}
+#endif
+
 void channel_monitor_exit(void)
 {
 	run_loop = 0;
@@ -555,6 +690,103 @@ channel_monitor_init(void)
 	return 0;
 }
 
+static void
+read_binary_packet(struct channel_info *chan_info)
+{
+	struct channel_packet pkt;
+	void *buffer = &pkt;
+	int buffer_len = sizeof(pkt);
+	int n_bytes, err = 0;
+
+	while (buffer_len > 0) {
+		n_bytes = read(chan_info->fd,
+				buffer, buffer_len);
+		if (n_bytes == buffer_len)
+			break;
+		if (n_bytes == -1) {
+			err = errno;
+			RTE_LOG(DEBUG, CHANNEL_MONITOR,
+				"Received error on "
+				"channel '%s' read: %s\n",
+				chan_info->channel_path,
+				strerror(err));
+			remove_channel(&chan_info);
+			break;
+		}
+		buffer = (char *)buffer + n_bytes;
+		buffer_len -= n_bytes;
+	}
+	if (!err)
+		process_request(&pkt, chan_info);
+}
+
+#ifdef USE_JANSSON
+static void
+read_json_packet(struct channel_info *chan_info)
+{
+	struct channel_packet pkt;
+	int n_bytes, ret;
+	json_t *root;
+	json_error_t error;
+
+	/* read opening brace to closing brace */
+	do {
+		int idx = 0;
+		int indent = 0;
+		do {
+			n_bytes = read(chan_info->fd, &json_data[idx], 1);
+			if (n_bytes == 0)
+				break;
+			if (json_data[idx] == '{')
+				indent++;
+			if (json_data[idx] == '}')
+				indent--;
+			if ((indent > 0) || (idx > 0))
+				idx++;
+			if (indent == 0)
+				json_data[idx] = 0;
+			if (idx >= MAX_JSON_STRING_LEN-1)
+				break;
+		} while (indent > 0);
+
+		if (indent > 0)
+			/*
+			 * We've broken out of the read loop without getting
+			 * a closing brace, so throw away the data
+			 */
+			json_data[idx] = 0;
+
+		if (strlen(json_data) == 0)
+			continue;
+
+		printf("got [%s]\n", json_data);
+
+		root = json_loads(json_data, 0, &error);
+
+		if (root) {
+			/*
+			 * Because our data is now in the json
+			 * object, we can overwrite the pkt
+			 * with a channel_packet struct, using
+			 * parse_json_to_pkt()
+			 */
+			ret = parse_json_to_pkt(root, &pkt);
+			json_decref(root);
+			if (ret) {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Error validating JSON profile data\n");
+				break;
+			}
+			process_request(&pkt, chan_info);
+		} else {
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+					"JSON error on line %d: %s\n",
+					error.line, error.text);
+		}
+	} while (n_bytes > 0);
+}
+#endif
+
 void
 run_channel_monitor(void)
 {
@@ -578,31 +810,18 @@ run_channel_monitor(void)
 			}
 			if (global_events_list[i].events & EPOLLIN) {
 
-				int n_bytes, err = 0;
-				struct channel_packet pkt;
-				void *buffer = &pkt;
-				int buffer_len = sizeof(pkt);
-
-				while (buffer_len > 0) {
-					n_bytes = read(chan_info->fd,
-							buffer, buffer_len);
-					if (n_bytes == buffer_len)
-						break;
-					if (n_bytes == -1) {
-						err = errno;
-						RTE_LOG(DEBUG, CHANNEL_MONITOR,
-							"Received error on "
-							"channel '%s' read: %s\n",
-							chan_info->channel_path,
-							strerror(err));
-						remove_channel(&chan_info);
-						break;
-					}
-					buffer = (char *)buffer + n_bytes;
-					buffer_len -= n_bytes;
+				switch (chan_info->type) {
+				case CHANNEL_TYPE_BINARY:
+					read_binary_packet(chan_info);
+					break;
+#ifdef USE_JANSSON
+				case CHANNEL_TYPE_JSON:
+					read_json_packet(chan_info);
+					break;
+#endif
+				default:
+					break;
 				}
-				if (!err)
-					process_request(&pkt, chan_info);
 			}
 		}
 		rte_delay_us(time_period_ms*1000);
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v4 08/11] examples/power: clean up verbose messages
  2018-09-26 13:40     ` [dpdk-dev] [PATCH v4 0/11] add json power policy interface for containers David Hunt
                         ` (6 preceding siblings ...)
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 07/11] examples/power: add json string handling David Hunt
@ 2018-09-26 13:40       ` David Hunt
  2018-09-26 14:25         ` Burakov, Anatoly
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 09/11] examples/power: add meson/ninja build support David Hunt
                         ` (3 subsequent siblings)
  11 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-26 13:40 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Some messages appearing several times a second, removing as they are
unnecessary. Other less severe messages change from INFO to DEBUG

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/channel_monitor.c | 19 +++++--------------
 1 file changed, 5 insertions(+), 14 deletions(-)

diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index 73a0451f3..92389ba5d 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -259,7 +259,7 @@ get_pcpu_to_control(struct policy *pol)
 
 	ci = get_core_info();
 
-	RTE_LOG(INFO, CHANNEL_MONITOR,
+	RTE_LOG(DEBUG, CHANNEL_MONITOR,
 			"Looking for pcpu for %s\n", pol->pkt.vm_name);
 
 	/*
@@ -426,8 +426,6 @@ apply_traffic_profile(struct policy *pol)
 
 	diff = get_pkt_diff(pol);
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Applying traffic profile\n");
-
 	if (diff >= (pol->pkt.traffic_policy.max_max_packet_thresh)) {
 		for (count = 0; count < pol->pkt.num_vcpu; count++) {
 			if (pol->core_share[count].status != 1)
@@ -471,9 +469,6 @@ apply_time_profile(struct policy *pol)
 				if (pol->core_share[count].status != 1) {
 					power_manager_scale_core_max(
 						pol->core_share[count].pcpu);
-				RTE_LOG(INFO, CHANNEL_MONITOR,
-					"Scaling up core %d to max\n",
-					pol->core_share[count].pcpu);
 				}
 			}
 			break;
@@ -483,9 +478,6 @@ apply_time_profile(struct policy *pol)
 				if (pol->core_share[count].status != 1) {
 					power_manager_scale_core_min(
 						pol->core_share[count].pcpu);
-				RTE_LOG(INFO, CHANNEL_MONITOR,
-					"Scaling down core %d to min\n",
-					pol->core_share[count].pcpu);
 			}
 		}
 			break;
@@ -547,8 +539,6 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (chan_info == NULL)
 		return -1;
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Processing Request %s\n", pkt->vm_name);
-
 	if (rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_CONNECTED,
 			CHANNEL_MGR_CHANNEL_PROCESSING) == 0)
 		return -1;
@@ -617,8 +607,8 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	}
 
 	if (pkt->command == PKT_POLICY) {
-		RTE_LOG(INFO, CHANNEL_MONITOR,
-				"\nProcessing Policy request\n");
+		RTE_LOG(INFO, CHANNEL_MONITOR, "Processing policy request %s\n",
+				pkt->vm_name);
 		update_policy(pkt);
 		policy_is_set = 1;
 	}
@@ -802,7 +792,8 @@ run_channel_monitor(void)
 					global_events_list[i].data.ptr;
 			if ((global_events_list[i].events & EPOLLERR) ||
 				(global_events_list[i].events & EPOLLHUP)) {
-				RTE_LOG(DEBUG, CHANNEL_MONITOR, "Remote closed connection for "
+				RTE_LOG(INFO, CHANNEL_MONITOR,
+						"Remote closed connection for "
 						"channel '%s'\n",
 						chan_info->channel_path);
 				remove_channel(&chan_info);
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v4 09/11] examples/power: add meson/ninja build support
  2018-09-26 13:40     ` [dpdk-dev] [PATCH v4 0/11] add json power policy interface for containers David Hunt
                         ` (7 preceding siblings ...)
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 08/11] examples/power: clean up verbose messages David Hunt
@ 2018-09-26 13:40       ` David Hunt
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 10/11] doc/vm_power_manager: add JSON interface API info David Hunt
                         ` (2 subsequent siblings)
  11 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-26 13:40 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Add meson.build in vm_power_manager and the guest_cli subdirectory.
Building can be achieved by going to the build directory, and using

meson configure -Dexamples=vm_power_manager,vm_power_manager/guest_cli

Then, when ninja is invoked, it will build dpdk-vm_power_manger and
dpdk-guest_cli

Work still needs to be done on the meson build system to handles the case
where the target list of example apps is defined as 'all'. That will come
in a future patch.

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
---
 .../vm_power_manager/guest_cli/meson.build    | 21 +++++++++++
 examples/vm_power_manager/meson.build         | 37 ++++++++++++++++++-
 2 files changed, 56 insertions(+), 2 deletions(-)
 create mode 100644 examples/vm_power_manager/guest_cli/meson.build

diff --git a/examples/vm_power_manager/guest_cli/meson.build b/examples/vm_power_manager/guest_cli/meson.build
new file mode 100644
index 000000000..9e821ceb8
--- /dev/null
+++ b/examples/vm_power_manager/guest_cli/meson.build
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+# Setting the name here because the default name will conflict with the
+# vm_power_manager app because of the way the directories are parsed.
+name = 'guest_cli'
+
+deps += ['power']
+
+sources = files(
+	'main.c', 'parse.c', 'vm_power_cli_guest.c'
+)
+
+opt_dep = cc.find_library('virt', required : false)
+build = opt_dep.found()
+ext_deps += opt_dep
diff --git a/examples/vm_power_manager/meson.build b/examples/vm_power_manager/meson.build
index c370d7476..f98445bc6 100644
--- a/examples/vm_power_manager/meson.build
+++ b/examples/vm_power_manager/meson.build
@@ -6,5 +6,38 @@
 # To build this example as a standalone application with an already-installed
 # DPDK instance, use 'make'
 
-# Example app currently unsupported by meson build
-build = false
+if dpdk_conf.has('RTE_LIBRTE_BNXT_PMD')
+	deps += ['pmd_bnxt']
+endif
+
+if dpdk_conf.has('RTE_LIBRTE_I40E_PMD')
+	deps += ['pmd_i40e']
+endif
+
+if dpdk_conf.has('RTE_LIBRTE_IXGBE_PMD')
+	deps += ['pmd_ixgbe']
+endif
+
+deps += ['power']
+
+
+sources = files(
+	'channel_manager.c', 'channel_monitor.c', 'main.c', 'parse.c', 'power_manager.c', 'vm_power_cli.c'
+)
+
+# If we're on X86, pull in the x86 code for the branch monitor algo.
+if dpdk_conf.has('RTE_ARCH_X86_64')
+	sources += files('oob_monitor_x86.c')
+else
+	sources += files('oob_monitor_nop.c')
+endif
+
+opt_dep = cc.find_library('virt', required : false)
+build = opt_dep.found()
+ext_deps += opt_dep
+
+opt_dep = dependency('jansson', required : false)
+if opt_dep.found()
+	ext_deps += opt_dep
+	cflags += '-DUSE_JANSSON'
+endif
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v4 10/11] doc/vm_power_manager: add JSON interface API info
  2018-09-26 13:40     ` [dpdk-dev] [PATCH v4 0/11] add json power policy interface for containers David Hunt
                         ` (8 preceding siblings ...)
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 09/11] examples/power: add meson/ninja build support David Hunt
@ 2018-09-26 13:40       ` David Hunt
  2018-09-26 14:32         ` Kovacevic, Marko
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 11/11] examples/power: add json example files David Hunt
  2018-09-26 16:37       ` [dpdk-dev] [PATCH v5 0/10] add json power policy interface for containers David Hunt
  11 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-26 13:40 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Also added meson/ninja build info

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 .../sample_app_ug/vm_power_management.rst     | 272 +++++++++++++++++-
 1 file changed, 270 insertions(+), 2 deletions(-)

diff --git a/doc/guides/sample_app_ug/vm_power_management.rst b/doc/guides/sample_app_ug/vm_power_management.rst
index 855570d6b..931a73f55 100644
--- a/doc/guides/sample_app_ug/vm_power_management.rst
+++ b/doc/guides/sample_app_ug/vm_power_management.rst
@@ -199,7 +199,7 @@ see :doc:`compiling`.
 
 The application is located in the ``vm_power_manager`` sub-directory.
 
-To build just the ``vm_power_manager`` application:
+To build just the ``vm_power_manager`` application using ``make``:
 
 .. code-block:: console
 
@@ -208,6 +208,22 @@ To build just the ``vm_power_manager`` application:
   cd ${RTE_SDK}/examples/vm_power_manager/
   make
 
+The resulting binary will be ${RTE_SDK}/build/examples/vm_power_manager
+
+To build just the ``vm_power_manager`` application using ``meson/ninja``:
+
+.. code-block:: console
+
+  export RTE_SDK=/path/to/rte_sdk
+  cd ${RTE_SDK}
+  meson build
+  cd build
+  ninja
+  meson configure -Dexamples=vm_power_manager
+  ninja
+
+The resulting binary will be ${RTE_SDK}/build/examples/dpdk-vm_power_manager
+
 Running
 ~~~~~~~
 
@@ -337,6 +353,242 @@ monitoring of branch ratio on cores doing busy polling via PMDs.
   and will need to be adjusted for different workloads.
 
 
+
+JSON API
+~~~~~~~~
+
+In addition to the command line interface for host command and a virtio-serial
+interface for VM power policies, there is also a JSON interface through which
+power commands and policies can be sent. This functionality adds a dependency
+on the Jansson library, and the Jansson development package must be installed
+on the system before the JSON parsing functionality is included in the app.
+This is achieved by:
+
+  .. code-block:: console
+
+    apt-get install libjansson-dev
+
+The command and package name may be different depending on your operating
+system. It's worth noting that the app will successfully build without this
+package present, but a warning is shown during compilation, and the JSON
+parsing functionality will not be present in the app.
+
+Sending a command or policy to the power manager application is achieved by
+simply opening a fifo file, writing a JSON string to that fifo, and closing
+the file.
+
+The fifo is at /tmp/powermonitor/fifo
+
+The jason string can be a policy or instruction, and takes the following
+format:
+
+  .. code-block:: console
+
+    {"packet_type": {
+      "pair_1": value,
+      "pair_2": value
+    }}
+
+The 'packet_type' header can contain one of two values, depending on
+whether a policy or power command is being sent. The two possible values are
+"policy" and "instruction", and the expected name-value pairs is different
+depending on which type is being sent.
+
+The pairs are the format of standard JSON name-value pairs. The value type
+varies between the different name/value pairs, and may be intgers, strings,
+arrays, etc. Examples of policies follow later in this document. The allowed
+names and value types are as follows:
+
+
+:Pair Name: "name"
+:Description: Name of the VM or Host. Allows the parser to associate the
+  policy with the relevant VM or Host OS.
+:Type: string
+:Values: any valid string
+:Required: yes
+:Example:
+
+  .. code-block:: console
+
+    ""name", "ubuntu2"
+
+
+:Pair Name: "command"
+:Description: The type of packet we're sending to the power manager. We can be
+  creating or destroying a policy, or sending a direct command to adjust
+  the frequency of a core, similar to the command line interface.
+:Type: string
+:Values:
+
+  :"CREATE": used when creating a new policy,
+  :"DESTROY": used when removing a policy,
+  :"POWER": used when sending an immediate command, max, min, etc.
+:Required: yes
+:Example:
+
+    .. code-block:: console
+
+      "command", "CREATE"
+
+
+:Pair Name: "policy_type"
+:Description: Type of policy to apply. Please see vm_power_manager documentation
+  for more information on the types of policies that may be used.
+:Type: string
+:Values:
+
+  :"TIME": Time-of-day policy. Frequencies of the relevant cores are
+    scaled up/down depending on busy and quiet hours.
+  :"TRAFFIC": This policy takes statistics from the NIC and scales up
+    and down accordingly.
+  :"WORKLOAD": This policy looks at how heavily loaded the cores are,
+    and scales up and down accordingly.
+  :"BRANCH_RATIO": This out-of-band policy can look at the ratio between
+    branch hits and misses on a core, and is useful for detecting
+    how much packet processing a core is doing.
+:Required: only for CREATE/DESTROY command
+:Example:
+
+  .. code-block:: console
+
+    "policy_type", "TIME"
+
+:Pair Name: "busy_hours"
+:Description: The hours of the day in which we scale up the cores for busy
+  times.
+:Type: array of integers
+:Values: array with list of hour numbers, (0-23)
+:Required: only for TIME policy
+:Example:
+
+  .. code-block:: console
+
+    "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ]
+
+:Pair Name: "quiet_hours"
+:Description: The hours of the day in which we scale down the cores for quiet
+  times.
+:Type: array of integers
+:Values: array with list of hour numbers, (0-23)
+:Required: only for TIME policy
+:Example:
+
+  .. code-block:: console
+
+    "quiet_hours":[ 2, 3, 4, 5, 6 ]
+
+:Pair Name: "avg_packet_thresh"
+:Description: Threshold below which the frequency will be set to min for
+  the TRAFFIC policy. If the traffic rate is above this and below max, the
+  frequency will be set to medium.
+:Type: integer
+:Values: The number of packets below which the TRAFFIC policy applies the
+  minimum frequency, or medium frequency if between avg and max thresholds.
+:Required: only for TRAFFIC policy
+:Example:
+
+  .. code-block:: console
+
+    "avg_packet_thresh": 100000
+
+:Pair Name: "max_packet_thresh"
+:Description: Threshold above which the frequency will be set to max for
+  the TRAFFIC policy
+:Type: integer
+:Values: The number of packets per interval above which the TRAFFIC policy
+  applies the maximum frequency
+:Required: only for TRAFFIC policy
+:Example:
+
+  .. code-block:: console
+
+    "max_packet_thresh": 500000
+
+:Pair Name: "core_list"
+:Description: The cores to which to apply the policy.
+:Type: array of integers
+:Values: array with list of virtual CPUs.
+:Required: only policy CREATE/DESTROY
+:Example:
+
+  .. code-block:: console
+
+    "core_list":[ 10, 11 ]
+
+:Pair Name: "unit"
+:Description: the type of power operation to apply in the command
+:Type: string
+:Values:
+
+  :"SCALE_MAX": Scale frequency of this core to maximum
+  :"SCALE_MIN": Scale frequency of this core to minimum
+  :"SCALE_UP": Scale up frequency of this core
+  :"SCALE_DOWN": Scale down frequency of this core
+  :"ENABLE_TURBO": Enable Turbo Boost for this core
+  :"DISABLE_TURBO": Disable Turbo Boost for this core
+:Required: only for POWER instruction
+:Example:
+
+  .. code-block:: console
+
+    "unit", "SCALE_MAX"
+
+:Pair Name: "resource_id"
+:Description: The core to which to apply the power command.
+:Type: integer
+:Values: valid core id for VM or host OS.
+:Required: only POWER instruction
+:Example:
+
+  .. code-block:: console
+
+    "resource_id": 10
+
+JSON API Examples
+~~~~~~~~~~~~~~~~~
+
+Profile create example:
+
+  .. code-block:: console
+
+    {"policy": {
+      "name": "ubuntu",
+      "command": "create",
+      "policy_type": "TIME",
+      "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
+      "quiet_hours":[ 2, 3, 4, 5, 6 ],
+      "core_list":[ 11 ]
+    }}
+
+Profile destroy example:
+
+  .. code-block:: console
+
+    {"profile": {
+      "name": "ubuntu",
+      "command": "destroy",
+    }}
+
+Power command example:
+
+  .. code-block:: console
+
+    {"command": {
+      "name": "ubuntu",
+      "unit": "SCALE_MAX",
+      "resource_id": 10
+    }}
+
+To send a JSON string to the Power Manager application, simply paste the
+example JSON string into a text file and cat it into the fifo:
+
+  .. code-block:: console
+
+    cat file.json >/tmp/powermonitor/fifo
+
+The console of the Power Manager application should indicate the command that
+was just received via the fifo.
+
 Compiling and Running the Guest Applications
 --------------------------------------------
 
@@ -366,7 +618,7 @@ For compiling and running l3fwd-power, see :doc:`l3_forward_power_man`.
 
 The application is located in the ``guest_cli`` sub-directory under ``vm_power_manager``.
 
-To build just the ``guest_vm_power_manager`` application:
+To build just the ``guest_vm_power_manager`` application using ``make``:
 
 .. code-block:: console
 
@@ -375,6 +627,22 @@ To build just the ``guest_vm_power_manager`` application:
   cd ${RTE_SDK}/examples/vm_power_manager/guest_cli/
   make
 
+The resulting binary will be ${RTE_SDK}/build/examples/guest_cli
+
+To build just the ``vm_power_manager`` application using ``meson/ninja``:
+
+.. code-block:: console
+
+  export RTE_SDK=/path/to/rte_sdk
+  cd ${RTE_SDK}
+  meson build
+  cd build
+  ninja
+  meson configure -Dexamples=vm_power_manager/guest_cli
+  ninja
+
+The resulting binary will be ${RTE_SDK}/build/examples/guest_cli
+
 Running
 ~~~~~~~
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v4 11/11] examples/power: add json example files
  2018-09-26 13:40     ` [dpdk-dev] [PATCH v4 0/11] add json power policy interface for containers David Hunt
                         ` (9 preceding siblings ...)
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 10/11] doc/vm_power_manager: add JSON interface API info David Hunt
@ 2018-09-26 13:40       ` David Hunt
  2018-09-26 15:58         ` Kovacevic, Marko
  2018-09-26 16:37       ` [dpdk-dev] [PATCH v5 0/10] add json power policy interface for containers David Hunt
  11 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-26 13:40 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

This patch provides some example files in the json_examples sub-directory
for sending to the fifo.

Signed-off-by: David Hunt <david.hunt@intel.com>
---
 examples/vm_power_manager/json_examples/README            | 6 ++++++
 examples/vm_power_manager/json_examples/create.json       | 8 ++++++++
 examples/vm_power_manager/json_examples/destroy.json      | 4 ++++
 examples/vm_power_manager/json_examples/set_core_max.json | 6 ++++++
 examples/vm_power_manager/json_examples/set_core_min.json | 6 ++++++
 5 files changed, 30 insertions(+)
 create mode 100644 examples/vm_power_manager/json_examples/README
 create mode 100644 examples/vm_power_manager/json_examples/create.json
 create mode 100644 examples/vm_power_manager/json_examples/destroy.json
 create mode 100644 examples/vm_power_manager/json_examples/set_core_max.json
 create mode 100644 examples/vm_power_manager/json_examples/set_core_min.json

diff --git a/examples/vm_power_manager/json_examples/README b/examples/vm_power_manager/json_examples/README
new file mode 100644
index 000000000..0c68bf913
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/README
@@ -0,0 +1,6 @@
+Sample files for sending to the vm_power_manager through the fifo.
+
+Simply cat the file to /tmp/powermonitor/fifo when the vm_power_manager
+application is running.
+
+E.g. cat create.json >/tmp/powermonitor/fifo
diff --git a/examples/vm_power_manager/json_examples/create.json b/examples/vm_power_manager/json_examples/create.json
new file mode 100644
index 000000000..a7133d9a1
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/create.json
@@ -0,0 +1,8 @@
+{"policy": {
+  "name": "policy-1",
+  "command": "create",
+  "policy_type": "TIME",
+  "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
+  "quiet_hours":[ 2, 3, 4, 5, 6 ],
+  "core_list":[ 11, 12 ]
+}}
diff --git a/examples/vm_power_manager/json_examples/destroy.json b/examples/vm_power_manager/json_examples/destroy.json
new file mode 100644
index 000000000..587c9e7e9
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/destroy.json
@@ -0,0 +1,4 @@
+{"policy": {
+  "name": "policy-1",
+  "command": "destroy"
+}}
diff --git a/examples/vm_power_manager/json_examples/set_core_max.json b/examples/vm_power_manager/json_examples/set_core_max.json
new file mode 100644
index 000000000..497030a44
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/set_core_max.json
@@ -0,0 +1,6 @@
+{"instruction": {
+  "name": "set_power",
+  "command": "power",
+  "unit": "SCALE_MAX",
+  "resource_id": 10
+}}
diff --git a/examples/vm_power_manager/json_examples/set_core_min.json b/examples/vm_power_manager/json_examples/set_core_min.json
new file mode 100644
index 000000000..76d934fd8
--- /dev/null
+++ b/examples/vm_power_manager/json_examples/set_core_min.json
@@ -0,0 +1,6 @@
+{"instruction": {
+  "name": "set_power",
+  "command": "power",
+  "unit": "SCALE_MIN",
+  "resource_id": 10
+}}
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v4 01/11] examples/power: add checks around hypervisor
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 01/11] examples/power: add checks around hypervisor David Hunt
@ 2018-09-26 13:54         ` Burakov, Anatoly
  0 siblings, 0 replies; 110+ messages in thread
From: Burakov, Anatoly @ 2018-09-26 13:54 UTC (permalink / raw)
  To: David Hunt, dev; +Cc: john.mcnamara, stephen, lei.a.yao

On 26-Sep-18 2:40 PM, David Hunt wrote:
> Allow vm_power_manager to run without requiring qemu to be present
> on the machine. This will be required for instances where the JSON
> interface is used for commands and polices, without any VMs present.
> A use case for this is a container enviromnent.
> 
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---

Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v4 02/11] examples/power: allow for number of vms to be zero
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 02/11] examples/power: allow for number of vms to be zero David Hunt
@ 2018-09-26 13:54         ` Burakov, Anatoly
  0 siblings, 0 replies; 110+ messages in thread
From: Burakov, Anatoly @ 2018-09-26 13:54 UTC (permalink / raw)
  To: David Hunt, dev; +Cc: john.mcnamara, stephen, lei.a.yao

On 26-Sep-18 2:40 PM, David Hunt wrote:
> Previously the vm_power_manager app required to have some vms defined, so
> the call to get_all_vm() always set the noVms variable. Now we're accepting
> policies from the host OS (without any VMs defined), so it is now valid to
> have zero VMs. This patch initialises the relevant variables to zero just
> in case the call to get_all_vms() does not find any, so could return with
> the variables uninitialised.
> 
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---

Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v4 03/11] lib/power: add changes for host commands/policies
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 03/11] lib/power: add changes for host commands/policies David Hunt
@ 2018-09-26 13:54         ` Burakov, Anatoly
  0 siblings, 0 replies; 110+ messages in thread
From: Burakov, Anatoly @ 2018-09-26 13:54 UTC (permalink / raw)
  To: David Hunt, dev; +Cc: john.mcnamara, stephen, lei.a.yao

On 26-Sep-18 2:40 PM, David Hunt wrote:
> This patch does a couple of things:
>    * Adds a new message type for removing policies (PKT_POLICY_REMOVE)
>      Used when we want to remove a previously created policy.
>    * Adds a core_type bool to the channel packet struct to specify whether
>      the type of core we want to control is cirtual or physical.
> 
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---

Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v4 05/11] examples/power: add host channel to power manager
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 05/11] examples/power: add host channel to power manager David Hunt
@ 2018-09-26 14:22         ` Burakov, Anatoly
  0 siblings, 0 replies; 110+ messages in thread
From: Burakov, Anatoly @ 2018-09-26 14:22 UTC (permalink / raw)
  To: David Hunt, dev; +Cc: john.mcnamara, stephen, lei.a.yao

On 26-Sep-18 2:40 PM, David Hunt wrote:
> This patch adds a fifo channel to the vm_power_manager app through which
> we can send commands and polices. Intended for sending JSON strings.
> The fifo is at /tmp/powermonitor/fifo
> 
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---

Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v4 06/11] examples/power: increase allowed number of clients
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 06/11] examples/power: increase allowed number of clients David Hunt
@ 2018-09-26 14:23         ` Burakov, Anatoly
  0 siblings, 0 replies; 110+ messages in thread
From: Burakov, Anatoly @ 2018-09-26 14:23 UTC (permalink / raw)
  To: David Hunt, dev; +Cc: john.mcnamara, stephen, lei.a.yao

On 26-Sep-18 2:40 PM, David Hunt wrote:
> Now that we're handling host policies, containers and virtual machines,
> we'll rename MAX_VMS to MAX_CLIENTS, and increase from 4 to 64
> 
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---

Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v4 07/11] examples/power: add json string handling
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 07/11] examples/power: add json string handling David Hunt
@ 2018-09-26 14:24         ` Burakov, Anatoly
  0 siblings, 0 replies; 110+ messages in thread
From: Burakov, Anatoly @ 2018-09-26 14:24 UTC (permalink / raw)
  To: David Hunt, dev; +Cc: john.mcnamara, stephen, lei.a.yao

On 26-Sep-18 2:40 PM, David Hunt wrote:
> Add JSON string handling to vm_power_manager for JSON strings received
> through the fifo. The format of the JSON strings are detailed in the
> next patch, the vm_power_manager user guide documentation updates.
> 
> This patch introduces a new dependency on Jansson, a C library for
> encoding, decoding and manipulating JSON data. To compile the sample app
> you now need to have installed libjansson4 and libjansson-dev (these may
> be named slightly differently depending on your Operating System)
> 
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---

Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v4 08/11] examples/power: clean up verbose messages
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 08/11] examples/power: clean up verbose messages David Hunt
@ 2018-09-26 14:25         ` Burakov, Anatoly
  0 siblings, 0 replies; 110+ messages in thread
From: Burakov, Anatoly @ 2018-09-26 14:25 UTC (permalink / raw)
  To: David Hunt, dev; +Cc: john.mcnamara, stephen, lei.a.yao

On 26-Sep-18 2:40 PM, David Hunt wrote:
> Some messages appearing several times a second, removing as they are
> unnecessary. Other less severe messages change from INFO to DEBUG
> 
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---

Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v4 10/11] doc/vm_power_manager: add JSON interface API info
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 10/11] doc/vm_power_manager: add JSON interface API info David Hunt
@ 2018-09-26 14:32         ` Kovacevic, Marko
  0 siblings, 0 replies; 110+ messages in thread
From: Kovacevic, Marko @ 2018-09-26 14:32 UTC (permalink / raw)
  To: Hunt, David, dev
  Cc: Mcnamara, John, stephen, Yao, Lei A, Burakov, Anatoly, Hunt, David

> Also added meson/ninja build info
> 
> Signed-off-by: David Hunt <david.hunt@intel.com>
> ---
>  .../sample_app_ug/vm_power_management.rst     | 272
> +++++++++++++++++-

Acked-by: Marko Kovacevic <marko.kovacevic@intel.com>

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v4 11/11] examples/power: add json example files
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 11/11] examples/power: add json example files David Hunt
@ 2018-09-26 15:58         ` Kovacevic, Marko
  2018-09-26 16:14           ` Hunt, David
  0 siblings, 1 reply; 110+ messages in thread
From: Kovacevic, Marko @ 2018-09-26 15:58 UTC (permalink / raw)
  To: Hunt, David, dev
  Cc: Mcnamara, John, stephen, Yao, Lei A, Burakov, Anatoly, Hunt, David

Hi Dave,

Feel like this part is not needed in the patch as you have this already in the documentation,
People can just copy and paste the examples from the docs, so we don't need to maintain two things.

Thanks,
Marko K.

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v4 11/11] examples/power: add json example files
  2018-09-26 15:58         ` Kovacevic, Marko
@ 2018-09-26 16:14           ` Hunt, David
  0 siblings, 0 replies; 110+ messages in thread
From: Hunt, David @ 2018-09-26 16:14 UTC (permalink / raw)
  To: Kovacevic, Marko, dev
  Cc: Mcnamara, John, stephen, Yao, Lei A, Burakov, Anatoly

Good point. There's also a couple of tiny niggles I'd like to resolve in patch 10 of the set, so I'll re-spin in v5.

-----Original Message-----
From: Kovacevic, Marko 
Sent: Wednesday, 26 September, 2018 4:58 PM
To: Hunt, David <david.hunt@intel.com>; dev@dpdk.org
Cc: Mcnamara, John <john.mcnamara@intel.com>; stephen@networkplumber.org; Yao, Lei A <lei.a.yao@intel.com>; Burakov, Anatoly <anatoly.burakov@intel.com>; Hunt, David <david.hunt@intel.com>
Subject: RE: [dpdk-dev] [PATCH v4 11/11] examples/power: add json example files

Hi Dave,

Feel like this part is not needed in the patch as you have this already in the documentation, People can just copy and paste the examples from the docs, so we don't need to maintain two things.

Thanks,
Marko K.

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v5 0/10] add json power policy interface for containers
  2018-09-26 13:40     ` [dpdk-dev] [PATCH v4 0/11] add json power policy interface for containers David Hunt
                         ` (10 preceding siblings ...)
  2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 11/11] examples/power: add json example files David Hunt
@ 2018-09-26 16:37       ` David Hunt
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 01/10] examples/power: add checks around hypervisor David Hunt
                           ` (10 more replies)
  11 siblings, 11 replies; 110+ messages in thread
From: David Hunt @ 2018-09-26 16:37 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov

The current vm_power_manager example app has the capability to accept power
policies from virtual machines via virtio-serial channels. These power
policies  allow a virtual machine to give information to the power manager
to allow the power manager take care of the power management of the virtual
machine based on the information in the policy.

This power policy functionality is limited to virtual machines sending
the policies to the power manager (which runs in the Host OS), and a solution
was needed for additional methods of sending power policies to the power
manager app.

The main use-case for this modification is for containers and host
applications that wish to send polices to the power manager.

This patchset adds the capability to send power polices and power commands
to the vm_power_manager app via JSON strings through a fifo on the file
system.
For example, given the following file, policy.json:

{"policy": {
  "name": "ubuntu2",
  "command": "create",
  "policy_type": "TIME",
  "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
  "quiet_hours":[ 2, 3, 4, 5, 6 ],
  "core_list":[ 11, 12, 13 ]
}}

Then running the command:

cat policy.json >/tmp/powermonitor/fifo

The policy is sent to the vm_power_manager. The power manager app then parses
the JSON data, and inserts the policy into the array of policies.

Part of the patch series contains documentation updates to give all the
details of the valid name-value pairs, the data types, etc.

Patch v2:
  * Fixed review comments from Stephen Hemminger and Lei A Yao.
  * Added a check in the Makefile for libjansson-dev. Will Warn user and build
    without JSON functionality if not present, will build including JSON
    functionality if it is present.

Patch v3:
  * Added meson/ninja support for vm_power_manager and guest_cli apps
  * Fixed compilation issue with guest_cli app

Patch v4:
  * Split out some unrelated changes to separate patches in the set
  * Some changes out of review by Anatoly (Thanks!)

Patch v5:
  * Removed the directory with JSON examples, as they already exist in
    the documentation.
  * Fixed some typos and formatting issues in the documentation.
  * Changed the JSON examples in the documentation to 'javascript' causing
    the syntax to be highlighted nicely.
  * Inherited the Acks from previous version.

[01/10] examples/power: add checks around hypervisor
[02/10] examples/power: allow for number of vms to be zero
[03/10] lib/power: add changes for host commands/policies
[04/10] examples/power: add necessary changes to guest app
[05/10] examples/power: add host channel to power manager
[06/10] examples/power: increase allowed number of clients
[07/10] examples/power: add json string handling
[08/10] examples/power: clean up verbose messages
[09/10] examples/power: add meson/ninja build support
[10/10] doc/vm_power_manager: add JSON interface API info

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v5 01/10] examples/power: add checks around hypervisor
  2018-09-26 16:37       ` [dpdk-dev] [PATCH v5 0/10] add json power policy interface for containers David Hunt
@ 2018-09-26 16:37         ` David Hunt
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 02/10] examples/power: allow for number of vms to be zero David Hunt
                           ` (9 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-26 16:37 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Allow vm_power_manager to run without requiring qemu to be present
on the machine. This will be required for instances where the JSON
interface is used for commands and polices, without any VMs present.
A use case for this is a container enviromnent.

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/channel_manager.c | 71 +++++++++++++--------
 1 file changed, 43 insertions(+), 28 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.c b/examples/vm_power_manager/channel_manager.c
index 927fc35ab..2e471d0c1 100644
--- a/examples/vm_power_manager/channel_manager.c
+++ b/examples/vm_power_manager/channel_manager.c
@@ -43,7 +43,8 @@ static unsigned char *global_cpumaps;
 static virVcpuInfo *global_vircpuinfo;
 static size_t global_maplen;
 
-static unsigned global_n_host_cpus;
+static unsigned int global_n_host_cpus;
+static bool global_hypervisor_available;
 
 /*
  * Represents a single Virtual Machine
@@ -198,7 +199,11 @@ get_pcpus_mask(struct channel_info *chan_info, unsigned vcpu)
 {
 	struct virtual_machine_info *vm_info =
 			(struct virtual_machine_info *)chan_info->priv_info;
-	return rte_atomic64_read(&vm_info->pcpu_mask[vcpu]);
+
+	if (global_hypervisor_available && (vm_info != NULL))
+		return rte_atomic64_read(&vm_info->pcpu_mask[vcpu]);
+	else
+		return 0;
 }
 
 static inline int
@@ -559,6 +564,8 @@ get_all_vm(int *num_vm, int *num_vcpu)
 				VIR_CONNECT_LIST_DOMAINS_PERSISTENT;
 	unsigned int domain_flag = VIR_DOMAIN_VCPU_CONFIG;
 
+	if (!global_hypervisor_available)
+		return;
 
 	memset(global_cpumaps, 0, CHANNEL_CMDS_MAX_CPUS*global_maplen);
 	if (virNodeGetInfo(global_vir_conn_ptr, &node_info)) {
@@ -768,38 +775,42 @@ connect_hypervisor(const char *path)
 	}
 	return 0;
 }
-
 int
-channel_manager_init(const char *path)
+channel_manager_init(const char *path __rte_unused)
 {
 	virNodeInfo info;
 
 	LIST_INIT(&vm_list_head);
 	if (connect_hypervisor(path) < 0) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to initialize channel manager\n");
-		return -1;
-	}
-
-	global_maplen = VIR_CPU_MAPLEN(CHANNEL_CMDS_MAX_CPUS);
+		global_n_host_cpus = 64;
+		global_hypervisor_available = 0;
+		RTE_LOG(INFO, CHANNEL_MANAGER, "Unable to initialize channel manager\n");
+	} else {
+		global_hypervisor_available = 1;
+
+		global_maplen = VIR_CPU_MAPLEN(CHANNEL_CMDS_MAX_CPUS);
+
+		global_vircpuinfo = rte_zmalloc(NULL,
+				sizeof(*global_vircpuinfo) *
+				CHANNEL_CMDS_MAX_CPUS, RTE_CACHE_LINE_SIZE);
+		if (global_vircpuinfo == NULL) {
+			RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for CPU Info\n");
+			goto error;
+		}
+		global_cpumaps = rte_zmalloc(NULL,
+				CHANNEL_CMDS_MAX_CPUS * global_maplen,
+				RTE_CACHE_LINE_SIZE);
+		if (global_cpumaps == NULL)
+			goto error;
 
-	global_vircpuinfo = rte_zmalloc(NULL, sizeof(*global_vircpuinfo) *
-			CHANNEL_CMDS_MAX_CPUS, RTE_CACHE_LINE_SIZE);
-	if (global_vircpuinfo == NULL) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for CPU Info\n");
-		goto error;
-	}
-	global_cpumaps = rte_zmalloc(NULL, CHANNEL_CMDS_MAX_CPUS * global_maplen,
-			RTE_CACHE_LINE_SIZE);
-	if (global_cpumaps == NULL) {
-		goto error;
+		if (virNodeGetInfo(global_vir_conn_ptr, &info)) {
+			RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to retrieve node Info\n");
+			goto error;
+		}
+		global_n_host_cpus = (unsigned int)info.cpus;
 	}
 
-	if (virNodeGetInfo(global_vir_conn_ptr, &info)) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to retrieve node Info\n");
-		goto error;
-	}
 
-	global_n_host_cpus = (unsigned)info.cpus;
 
 	if (global_n_host_cpus > CHANNEL_CMDS_MAX_CPUS) {
 		RTE_LOG(WARNING, CHANNEL_MANAGER, "The number of host CPUs(%u) exceeds the "
@@ -811,7 +822,8 @@ channel_manager_init(const char *path)
 
 	return 0;
 error:
-	disconnect_hypervisor();
+	if (global_hypervisor_available)
+		disconnect_hypervisor();
 	return -1;
 }
 
@@ -838,7 +850,10 @@ channel_manager_exit(void)
 		rte_free(vm_info);
 	}
 
-	rte_free(global_cpumaps);
-	rte_free(global_vircpuinfo);
-	disconnect_hypervisor();
+	if (global_hypervisor_available) {
+		/* Only needed if hypervisor available */
+		rte_free(global_cpumaps);
+		rte_free(global_vircpuinfo);
+		disconnect_hypervisor();
+	}
 }
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v5 02/10] examples/power: allow for number of vms to be zero
  2018-09-26 16:37       ` [dpdk-dev] [PATCH v5 0/10] add json power policy interface for containers David Hunt
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 01/10] examples/power: add checks around hypervisor David Hunt
@ 2018-09-26 16:37         ` David Hunt
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 03/10] lib/power: add changes for host commands/policies David Hunt
                           ` (8 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-26 16:37 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Previously the vm_power_manager app required to have some vms defined, so
the call to get_all_vm() always set the noVms variable. Now we're accepting
policies from the host OS (without any VMs defined), so it is now valid to
have zero VMs. This patch initialises the relevant variables to zero just
in case the call to get_all_vms() does not find any, so could return with
the variables uninitialised.

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/channel_monitor.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index 7fa47ba97..f180d74e6 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -66,7 +66,7 @@ static void
 core_share_status(int pNo)
 {
 
-	int noVms, noVcpus, z, x, t;
+	int noVms = 0, noVcpus = 0, z, x, t;
 
 	get_all_vm(&noVms, &noVcpus);
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v5 03/10] lib/power: add changes for host commands/policies
  2018-09-26 16:37       ` [dpdk-dev] [PATCH v5 0/10] add json power policy interface for containers David Hunt
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 01/10] examples/power: add checks around hypervisor David Hunt
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 02/10] examples/power: allow for number of vms to be zero David Hunt
@ 2018-09-26 16:37         ` David Hunt
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 04/10] examples/power: add necessary changes to guest app David Hunt
                           ` (7 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-26 16:37 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

This patch does a couple of things:
  * Adds a new message type for removing policies (PKT_POLICY_REMOVE)
    Used when we want to remove a previously created policy.
  * Adds a core_type bool to the channel packet struct to specify whether
    the type of core we want to control is cirtual or physical.

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 lib/librte_power/channel_commands.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/lib/librte_power/channel_commands.h b/lib/librte_power/channel_commands.h
index ee638eefa..e7b93a797 100644
--- a/lib/librte_power/channel_commands.h
+++ b/lib/librte_power/channel_commands.h
@@ -19,6 +19,7 @@ extern "C" {
 #define CPU_POWER               1
 #define CPU_POWER_CONNECT       2
 #define PKT_POLICY              3
+#define PKT_POLICY_REMOVE       4
 
 /* CPU Power Command Scaling */
 #define CPU_POWER_SCALE_UP      1
@@ -58,6 +59,9 @@ struct traffic {
 	uint32_t max_max_packet_thresh;
 };
 
+#define CORE_TYPE_VIRTUAL 0
+#define CORE_TYPE_PHYSICAL 1
+
 struct channel_packet {
 	uint64_t resource_id; /**< core_num, device */
 	uint32_t unit;        /**< scale down/up/min/max */
@@ -70,6 +74,7 @@ struct channel_packet {
 	uint8_t vcpu_to_control[MAX_VCPU_PER_VM];
 	uint8_t num_vcpu;
 	struct timer_profile timer_policy;
+	bool core_type;
 	enum workload workload;
 	enum policy_to_use policy_to_use;
 	struct t_boost_status t_boost_status;
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v5 04/10] examples/power: add necessary changes to guest app
  2018-09-26 16:37       ` [dpdk-dev] [PATCH v5 0/10] add json power policy interface for containers David Hunt
                           ` (2 preceding siblings ...)
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 03/10] lib/power: add changes for host commands/policies David Hunt
@ 2018-09-26 16:37         ` David Hunt
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 05/10] examples/power: add host channel to power manager David Hunt
                           ` (6 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-26 16:37 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

The changes here are minimal, as the guest app functionality is not
changing at all, but there is a new element in the channel_packet
struct that needs to have a default set (channel_packet->core_type).

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/guest_cli/vm_power_cli_guest.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
index 0db1b804f..2d9e7689a 100644
--- a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
+++ b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
@@ -92,6 +92,7 @@ set_policy_defaults(struct channel_packet *pkt)
 	pkt->timer_policy.hours_to_use_traffic_profile[0] = 8;
 	pkt->timer_policy.hours_to_use_traffic_profile[1] = 10;
 
+	pkt->core_type = CORE_TYPE_VIRTUAL;
 	pkt->workload = LOW;
 	pkt->policy_to_use = TIME;
 	pkt->command = PKT_POLICY;
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v5 05/10] examples/power: add host channel to power manager
  2018-09-26 16:37       ` [dpdk-dev] [PATCH v5 0/10] add json power policy interface for containers David Hunt
                           ` (3 preceding siblings ...)
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 04/10] examples/power: add necessary changes to guest app David Hunt
@ 2018-09-26 16:37         ` David Hunt
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 06/10] examples/power: increase allowed number of clients David Hunt
                           ` (5 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-26 16:37 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

This patch adds a fifo channel to the vm_power_manager app through which
we can send commands and polices. Intended for sending JSON strings.
The fifo is at /tmp/powermonitor/fifo

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/channel_manager.c | 109 +++++++++++++++
 examples/vm_power_manager/channel_manager.h |  17 +++
 examples/vm_power_manager/channel_monitor.c | 142 +++++++++++++++-----
 examples/vm_power_manager/main.c            |   2 +
 4 files changed, 236 insertions(+), 34 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.c b/examples/vm_power_manager/channel_manager.c
index 2e471d0c1..4fac099df 100644
--- a/examples/vm_power_manager/channel_manager.c
+++ b/examples/vm_power_manager/channel_manager.c
@@ -13,6 +13,7 @@
 
 #include <sys/queue.h>
 #include <sys/types.h>
+#include <sys/stat.h>
 #include <sys/socket.h>
 #include <sys/select.h>
 
@@ -284,6 +285,38 @@ open_non_blocking_channel(struct channel_info *info)
 	return 0;
 }
 
+static int
+open_host_channel(struct channel_info *info)
+{
+	int flags;
+
+	info->fd = open(info->channel_path, O_RDWR | O_RSYNC);
+	if (info->fd == -1) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Error(%s) opening fifo for '%s'\n",
+				strerror(errno),
+				info->channel_path);
+		return -1;
+	}
+
+	/* Get current flags */
+	flags = fcntl(info->fd, F_GETFL, 0);
+	if (flags < 0) {
+		RTE_LOG(WARNING, CHANNEL_MANAGER, "Error(%s) fcntl get flags socket for"
+				"'%s'\n", strerror(errno), info->channel_path);
+		return 1;
+	}
+	/* Set to Non Blocking */
+	flags |= O_NONBLOCK;
+	if (fcntl(info->fd, F_SETFL, flags) < 0) {
+		RTE_LOG(WARNING, CHANNEL_MANAGER,
+				"Error(%s) setting non-blocking "
+				"socket for '%s'\n",
+				strerror(errno), info->channel_path);
+		return -1;
+	}
+	return 0;
+}
+
 static int
 setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 		struct channel_info **chan_info_dptr, unsigned channel_num)
@@ -294,6 +327,7 @@ setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 	chan_info->channel_num = channel_num;
 	chan_info->priv_info = (void *)vm_info;
 	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
+	chan_info->type = CHANNEL_TYPE_BINARY;
 	if (open_non_blocking_channel(chan_info) < 0) {
 		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open channel: "
 				"'%s' for VM '%s'\n",
@@ -316,6 +350,42 @@ setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 	return 0;
 }
 
+static void
+fifo_path(char *dst, unsigned int len)
+{
+	snprintf(dst, len, "%sfifo", CHANNEL_MGR_SOCKET_PATH);
+}
+
+static int
+setup_host_channel_info(struct channel_info **chan_info_dptr,
+		unsigned int channel_num)
+{
+	struct channel_info *chan_info = *chan_info_dptr;
+
+	chan_info->channel_num = channel_num;
+	chan_info->priv_info = (void *)NULL;
+	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
+	chan_info->type = CHANNEL_TYPE_JSON;
+
+	fifo_path(chan_info->channel_path, sizeof(chan_info->channel_path));
+
+	if (open_host_channel(chan_info) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open host channel: "
+				"'%s'\n",
+				chan_info->channel_path);
+		return -1;
+	}
+	if (add_channel_to_monitor(&chan_info) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Could add channel: "
+				"'%s' to epoll ctl\n",
+				chan_info->channel_path);
+		return -1;
+
+	}
+	chan_info->status = CHANNEL_MGR_CHANNEL_CONNECTED;
+	return 0;
+}
+
 int
 add_all_channels(const char *vm_name)
 {
@@ -470,6 +540,45 @@ add_channels(const char *vm_name, unsigned *channel_list,
 	return num_channels_enabled;
 }
 
+int
+add_host_channel(void)
+{
+	struct channel_info *chan_info;
+	char socket_path[PATH_MAX];
+	int num_channels_enabled = 0;
+	int ret;
+
+	fifo_path(socket_path, sizeof(socket_path));
+
+	ret = mkfifo(socket_path, 0660);
+	if ((errno != EEXIST) && (ret < 0)) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Cannot create fifo '%s' error: "
+				"%s\n", socket_path, strerror(errno));
+		return 0;
+	}
+
+	if (access(socket_path, F_OK) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Channel path '%s' error: "
+				"%s\n", socket_path, strerror(errno));
+		return 0;
+	}
+	chan_info = rte_malloc(NULL, sizeof(*chan_info), 0);
+	if (chan_info == NULL) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for "
+				"channel '%s'\n", socket_path);
+		return 0;
+	}
+	snprintf(chan_info->channel_path,
+			sizeof(chan_info->channel_path), "%s", socket_path);
+	if (setup_host_channel_info(&chan_info, 0) < 0) {
+		rte_free(chan_info);
+		return 0;
+	}
+	num_channels_enabled++;
+
+	return num_channels_enabled;
+}
+
 int
 remove_channel(struct channel_info **chan_info_dptr)
 {
diff --git a/examples/vm_power_manager/channel_manager.h b/examples/vm_power_manager/channel_manager.h
index 872ec6140..e32235b07 100644
--- a/examples/vm_power_manager/channel_manager.h
+++ b/examples/vm_power_manager/channel_manager.h
@@ -54,6 +54,13 @@ enum channel_status { CHANNEL_MGR_CHANNEL_DISCONNECTED = 0,
 	CHANNEL_MGR_CHANNEL_DISABLED,
 	CHANNEL_MGR_CHANNEL_PROCESSING};
 
+/* Communication Channel Type */
+enum channel_type {
+	CHANNEL_TYPE_BINARY = 0,
+	CHANNEL_TYPE_INI,
+	CHANNEL_TYPE_JSON
+};
+
 /* VM libvirt(qemu/KVM) connection status */
 enum vm_status { CHANNEL_MGR_VM_INACTIVE = 0, CHANNEL_MGR_VM_ACTIVE};
 
@@ -66,6 +73,7 @@ struct channel_info {
 	volatile uint32_t status;    /**< Connection status(enum channel_status) */
 	int fd;                      /**< AF_UNIX socket fd */
 	unsigned channel_num;        /**< CHANNEL_MGR_SOCKET_PATH/<vm_name>.channel_num */
+	enum channel_type type;      /**< Binary, ini, json, etc. */
 	void *priv_info;             /**< Pointer to private info, do not modify */
 };
 
@@ -226,6 +234,15 @@ int add_all_channels(const char *vm_name);
 int add_channels(const char *vm_name, unsigned *channel_list,
 		unsigned num_channels);
 
+/**
+ * Set up a fifo by which host applications can send command an policies
+ * through a fifo to the vm_power_manager
+ *
+ * @return
+ *  - 0 for success
+ */
+int add_host_channel(void);
+
 /**
  * Remove a channel definition from the channel manager. This must only be
  * called from the channel monitor thread.
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index f180d74e6..c3c3d7bb1 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -85,6 +85,33 @@ core_share_status(int pNo)
 	}
 }
 
+
+static int
+pcpu_monitor(struct policy *pol, struct core_info *ci, int pcpu, int count)
+{
+	int ret = 0;
+
+	if (pol->pkt.policy_to_use == BRANCH_RATIO) {
+		ci->cd[pcpu].oob_enabled = 1;
+		ret = add_core_to_monitor(pcpu);
+		if (ret == 0)
+			RTE_LOG(INFO, CHANNEL_MONITOR,
+					"Monitoring pcpu %d OOB for %s\n",
+					pcpu, pol->pkt.vm_name);
+		else
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Error monitoring pcpu %d OOB for %s\n",
+					pcpu, pol->pkt.vm_name);
+
+	} else {
+		pol->core_share[count].pcpu = pcpu;
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				"Monitoring pcpu %d for %s\n",
+				pcpu, pol->pkt.vm_name);
+	}
+	return ret;
+}
+
 static void
 get_pcpu_to_control(struct policy *pol)
 {
@@ -94,34 +121,42 @@ get_pcpu_to_control(struct policy *pol)
 	int pcpu, count;
 	uint64_t mask_u64b;
 	struct core_info *ci;
-	int ret;
 
 	ci = get_core_info();
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Looking for pcpu for %s\n",
-			pol->pkt.vm_name);
-	get_info_vm(pol->pkt.vm_name, &info);
-
-	for (count = 0; count < pol->pkt.num_vcpu; count++) {
-		mask_u64b = info.pcpu_mask[pol->pkt.vcpu_to_control[count]];
-		for (pcpu = 0; mask_u64b; mask_u64b &= ~(1ULL << pcpu++)) {
-			if ((mask_u64b >> pcpu) & 1) {
-				if (pol->pkt.policy_to_use == BRANCH_RATIO) {
-					ci->cd[pcpu].oob_enabled = 1;
-					ret = add_core_to_monitor(pcpu);
-					if (ret == 0)
-						printf("Monitoring pcpu %d via Branch Ratio\n",
-								pcpu);
-					else
-						printf("Failed to start OOB Monitoring pcpu %d\n",
-								pcpu);
-
-				} else {
-					pol->core_share[count].pcpu = pcpu;
-					printf("Monitoring pcpu %d\n", pcpu);
-				}
+	RTE_LOG(INFO, CHANNEL_MONITOR,
+			"Looking for pcpu for %s\n", pol->pkt.vm_name);
+
+	/*
+	 * So now that we're handling virtual and physical cores, we need to
+	 * differenciate between them when adding them to the branch monitor.
+	 * Virtual cores need to be converted to physical cores.
+	 */
+	if (pol->pkt.core_type == CORE_TYPE_VIRTUAL) {
+		/*
+		 * If the cores in the policy are virtual, we need to map them
+		 * to physical core. We look up the vm info and use that for
+		 * the mapping.
+		 */
+		get_info_vm(pol->pkt.vm_name, &info);
+		for (count = 0; count < pol->pkt.num_vcpu; count++) {
+			mask_u64b =
+				info.pcpu_mask[pol->pkt.vcpu_to_control[count]];
+			for (pcpu = 0; mask_u64b;
+					mask_u64b &= ~(1ULL << pcpu++)) {
+				if ((mask_u64b >> pcpu) & 1)
+					pcpu_monitor(pol, ci, pcpu, count);
 			}
 		}
+	} else {
+		/*
+		 * If the cores in the policy are physical, we just use
+		 * those core id's directly.
+		 */
+		for (count = 0; count < pol->pkt.num_vcpu; count++) {
+			pcpu = pol->pkt.vcpu_to_control[count];
+			pcpu_monitor(pol, ci, pcpu, count);
+		}
 	}
 }
 
@@ -160,8 +195,13 @@ update_policy(struct channel_packet *pkt)
 	unsigned int updated = 0;
 	int i;
 
+
+	RTE_LOG(INFO, CHANNEL_MONITOR,
+			"Applying policy for %s\n", pkt->vm_name);
+
 	for (i = 0; i < MAX_VMS; i++) {
 		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
+			/* Copy the contents of *pkt into the policy.pkt */
 			policies[i].pkt = *pkt;
 			get_pcpu_to_control(&policies[i]);
 			if (get_pfid(&policies[i]) == -1) {
@@ -189,6 +229,24 @@ update_policy(struct channel_packet *pkt)
 	return 0;
 }
 
+static int
+remove_policy(struct channel_packet *pkt __rte_unused)
+{
+	int i;
+
+	/*
+	 * Disabling the policy is simply a case of setting
+	 * enabled to 0
+	 */
+	for (i = 0; i < MAX_VMS; i++) {
+		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
+			policies[i].enabled = 0;
+			return 0;
+		}
+	}
+	return -1;
+}
+
 static uint64_t
 get_pkt_diff(struct policy *pol)
 {
@@ -346,7 +404,6 @@ apply_policy(struct policy *pol)
 		apply_workload_profile(pol);
 }
 
-
 static int
 process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 {
@@ -355,6 +412,8 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (chan_info == NULL)
 		return -1;
 
+	RTE_LOG(INFO, CHANNEL_MONITOR, "Processing Request %s\n", pkt->vm_name);
+
 	if (rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_CONNECTED,
 			CHANNEL_MGR_CHANNEL_PROCESSING) == 0)
 		return -1;
@@ -362,10 +421,12 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (pkt->command == CPU_POWER) {
 		core_mask = get_pcpus_mask(chan_info, pkt->resource_id);
 		if (core_mask == 0) {
-			RTE_LOG(ERR, CHANNEL_MONITOR, "Error get physical CPU mask for "
-				"channel '%s' using vCPU(%u)\n", chan_info->channel_path,
-				(unsigned)pkt->unit);
-			return -1;
+			/*
+			 * Core mask will be 0 in the case where
+			 * hypervisor is not available so we're working in
+			 * the host, so use the core as the mask.
+			 */
+			core_mask = 1ULL << pkt->resource_id;
 		}
 		if (__builtin_popcountll(core_mask) == 1) {
 
@@ -421,12 +482,20 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	}
 
 	if (pkt->command == PKT_POLICY) {
-		RTE_LOG(INFO, CHANNEL_MONITOR, "\nProcessing Policy request from Guest\n");
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				"\nProcessing Policy request\n");
 		update_policy(pkt);
 		policy_is_set = 1;
 	}
 
-	/* Return is not checked as channel status may have been set to DISABLED
+	if (pkt->command == PKT_POLICY_REMOVE) {
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				 "Removing policy %s\n", pkt->vm_name);
+		remove_policy(pkt);
+	}
+
+	/*
+	 * Return is not checked as channel status may have been set to DISABLED
 	 * from management thread
 	 */
 	rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_PROCESSING,
@@ -448,13 +517,16 @@ add_channel_to_monitor(struct channel_info **chan_info)
 				"to epoll\n", info->channel_path);
 		return -1;
 	}
+	RTE_LOG(ERR, CHANNEL_MONITOR, "Added channel '%s' "
+			"to monitor\n", info->channel_path);
 	return 0;
 }
 
 int
 remove_channel_from_monitor(struct channel_info *chan_info)
 {
-	if (epoll_ctl(global_event_fd, EPOLL_CTL_DEL, chan_info->fd, NULL) < 0) {
+	if (epoll_ctl(global_event_fd, EPOLL_CTL_DEL,
+			chan_info->fd, NULL) < 0) {
 		RTE_LOG(ERR, CHANNEL_MONITOR, "Unable to remove channel '%s' "
 				"from epoll\n", chan_info->channel_path);
 		return -1;
@@ -467,11 +539,13 @@ channel_monitor_init(void)
 {
 	global_event_fd = epoll_create1(0);
 	if (global_event_fd == 0) {
-		RTE_LOG(ERR, CHANNEL_MONITOR, "Error creating epoll context with "
-				"error %s\n", strerror(errno));
+		RTE_LOG(ERR, CHANNEL_MONITOR,
+				"Error creating epoll context with error %s\n",
+				strerror(errno));
 		return -1;
 	}
-	global_events_list = rte_malloc("epoll_events", sizeof(*global_events_list)
+	global_events_list = rte_malloc("epoll_events",
+			sizeof(*global_events_list)
 			* MAX_EVENTS, RTE_CACHE_LINE_SIZE);
 	if (global_events_list == NULL) {
 		RTE_LOG(ERR, CHANNEL_MONITOR, "Unable to rte_malloc for "
diff --git a/examples/vm_power_manager/main.c b/examples/vm_power_manager/main.c
index 58c5fa45c..893bf4cdd 100644
--- a/examples/vm_power_manager/main.c
+++ b/examples/vm_power_manager/main.c
@@ -421,6 +421,8 @@ main(int argc, char **argv)
 		return -1;
 	}
 
+	add_host_channel();
+
 	printf("Running core monitor on lcore id %d\n", lcore_id);
 	rte_eal_remote_launch(run_core_monitor, NULL, lcore_id);
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v5 06/10] examples/power: increase allowed number of clients
  2018-09-26 16:37       ` [dpdk-dev] [PATCH v5 0/10] add json power policy interface for containers David Hunt
                           ` (4 preceding siblings ...)
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 05/10] examples/power: add host channel to power manager David Hunt
@ 2018-09-26 16:37         ` David Hunt
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 07/10] examples/power: add json string handling David Hunt
                           ` (4 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-26 16:37 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Now that we're handling host policies, containers and virtual machines,
we'll rename MAX_VMS to MAX_CLIENTS, and increase from 4 to 64

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/channel_manager.h |  4 ++--
 examples/vm_power_manager/channel_monitor.c | 10 +++++-----
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.h b/examples/vm_power_manager/channel_manager.h
index e32235b07..d948b304c 100644
--- a/examples/vm_power_manager/channel_manager.h
+++ b/examples/vm_power_manager/channel_manager.h
@@ -37,7 +37,7 @@ struct sockaddr_un _sockaddr_un;
 #define UNIX_PATH_MAX sizeof(_sockaddr_un.sun_path)
 #endif
 
-#define MAX_VMS 4
+#define MAX_CLIENTS 64
 #define MAX_VCPUS 20
 
 
@@ -47,7 +47,7 @@ struct libvirt_vm_info {
 	uint8_t num_cpus;
 };
 
-struct libvirt_vm_info lvm_info[MAX_VMS];
+struct libvirt_vm_info lvm_info[MAX_CLIENTS];
 /* Communication Channel Status */
 enum channel_status { CHANNEL_MGR_CHANNEL_DISCONNECTED = 0,
 	CHANNEL_MGR_CHANNEL_CONNECTED,
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index c3c3d7bb1..53a4efe45 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -41,7 +41,7 @@ static volatile unsigned run_loop = 1;
 static int global_event_fd;
 static unsigned int policy_is_set;
 static struct epoll_event *global_events_list;
-static struct policy policies[MAX_VMS];
+static struct policy policies[MAX_CLIENTS];
 
 void channel_monitor_exit(void)
 {
@@ -199,7 +199,7 @@ update_policy(struct channel_packet *pkt)
 	RTE_LOG(INFO, CHANNEL_MONITOR,
 			"Applying policy for %s\n", pkt->vm_name);
 
-	for (i = 0; i < MAX_VMS; i++) {
+	for (i = 0; i < MAX_CLIENTS; i++) {
 		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
 			/* Copy the contents of *pkt into the policy.pkt */
 			policies[i].pkt = *pkt;
@@ -214,7 +214,7 @@ update_policy(struct channel_packet *pkt)
 		}
 	}
 	if (!updated) {
-		for (i = 0; i < MAX_VMS; i++) {
+		for (i = 0; i < MAX_CLIENTS; i++) {
 			if (policies[i].enabled == 0) {
 				policies[i].pkt = *pkt;
 				get_pcpu_to_control(&policies[i]);
@@ -238,7 +238,7 @@ remove_policy(struct channel_packet *pkt __rte_unused)
 	 * Disabling the policy is simply a case of setting
 	 * enabled to 0
 	 */
-	for (i = 0; i < MAX_VMS; i++) {
+	for (i = 0; i < MAX_CLIENTS; i++) {
 		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
 			policies[i].enabled = 0;
 			return 0;
@@ -609,7 +609,7 @@ run_channel_monitor(void)
 		if (policy_is_set) {
 			int j;
 
-			for (j = 0; j < MAX_VMS; j++) {
+			for (j = 0; j < MAX_CLIENTS; j++) {
 				if (policies[j].enabled == 1)
 					apply_policy(&policies[j]);
 			}
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v5 07/10] examples/power: add json string handling
  2018-09-26 16:37       ` [dpdk-dev] [PATCH v5 0/10] add json power policy interface for containers David Hunt
                           ` (5 preceding siblings ...)
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 06/10] examples/power: increase allowed number of clients David Hunt
@ 2018-09-26 16:37         ` David Hunt
  2018-09-30  1:54           ` Yao, Lei A
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 08/10] examples/power: clean up verbose messages David Hunt
                           ` (3 subsequent siblings)
  10 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-26 16:37 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Add JSON string handling to vm_power_manager for JSON strings received
through the fifo. The format of the JSON strings are detailed in the
next patch, the vm_power_manager user guide documentation updates.

This patch introduces a new dependency on Jansson, a C library for
encoding, decoding and manipulating JSON data. To compile the sample app
you now need to have installed libjansson4 and libjansson-dev (these may
be named slightly differently depending on your Operating System)

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/Makefile          |   6 +
 examples/vm_power_manager/channel_monitor.c | 269 ++++++++++++++++++--
 2 files changed, 250 insertions(+), 25 deletions(-)

diff --git a/examples/vm_power_manager/Makefile b/examples/vm_power_manager/Makefile
index 13a5205ba..50147c05d 100644
--- a/examples/vm_power_manager/Makefile
+++ b/examples/vm_power_manager/Makefile
@@ -31,6 +31,12 @@ CFLAGS += $(WERROR_FLAGS)
 
 LDLIBS += -lvirt
 
+JANSSON := $(shell pkg-config --exists jansson; echo $$?)
+ifeq ($(JANSSON), 0)
+LDLIBS += $(shell pkg-config --libs jansson)
+CFLAGS += -DUSE_JANSSON
+endif
+
 ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
 
 ifeq ($(CONFIG_RTE_LIBRTE_IXGBE_PMD),y)
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index 53a4efe45..73a0451f3 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -9,11 +9,18 @@
 #include <signal.h>
 #include <errno.h>
 #include <string.h>
+#include <fcntl.h>
 #include <sys/types.h>
 #include <sys/epoll.h>
 #include <sys/queue.h>
 #include <sys/time.h>
-
+#include <sys/socket.h>
+#include <sys/select.h>
+#ifdef USE_JANSSON
+#include <jansson.h>
+#else
+#pragma message "Jansson dev libs unavailable, not including JSON parsing"
+#endif
 #include <rte_log.h>
 #include <rte_memory.h>
 #include <rte_malloc.h>
@@ -35,6 +42,8 @@
 
 uint64_t vsi_pkt_count_prev[384];
 uint64_t rdtsc_prev[384];
+#define MAX_JSON_STRING_LEN 1024
+char json_data[MAX_JSON_STRING_LEN];
 
 double time_period_ms = 1;
 static volatile unsigned run_loop = 1;
@@ -43,6 +52,132 @@ static unsigned int policy_is_set;
 static struct epoll_event *global_events_list;
 static struct policy policies[MAX_CLIENTS];
 
+#ifdef USE_JANSSON
+static int
+parse_json_to_pkt(json_t *element, struct channel_packet *pkt)
+{
+	const char *key;
+	json_t *value;
+	int ret;
+
+	memset(pkt, 0, sizeof(struct channel_packet));
+
+	pkt->nb_mac_to_monitor = 0;
+	pkt->t_boost_status.tbEnabled = false;
+	pkt->workload = LOW;
+	pkt->policy_to_use = TIME;
+	pkt->command = PKT_POLICY;
+	pkt->core_type = CORE_TYPE_PHYSICAL;
+
+	json_object_foreach(element, key, value) {
+		if (!strcmp(key, "policy")) {
+			/* Recurse in to get the contents of profile */
+			ret = parse_json_to_pkt(value, pkt);
+			if (ret)
+				return ret;
+		} else if (!strcmp(key, "instruction")) {
+			/* Recurse in to get the contents of instruction */
+			ret = parse_json_to_pkt(value, pkt);
+			if (ret)
+				return ret;
+		} else if (!strcmp(key, "name")) {
+			strcpy(pkt->vm_name, json_string_value(value));
+		} else if (!strcmp(key, "command")) {
+			char command[32];
+			snprintf(command, 32, "%s", json_string_value(value));
+			if (!strcmp(command, "power")) {
+				pkt->command = CPU_POWER;
+			} else if (!strcmp(command, "create")) {
+				pkt->command = PKT_POLICY;
+			} else if (!strcmp(command, "destroy")) {
+				pkt->command = PKT_POLICY_REMOVE;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Invalid command received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "policy_type")) {
+			char command[32];
+			snprintf(command, 32, "%s", json_string_value(value));
+			if (!strcmp(command, "TIME")) {
+				pkt->policy_to_use = TIME;
+			} else if (!strcmp(command, "TRAFFIC")) {
+				pkt->policy_to_use = TRAFFIC;
+			} else if (!strcmp(command, "WORKLOAD")) {
+				pkt->policy_to_use = WORKLOAD;
+			} else if (!strcmp(command, "BRANCH_RATIO")) {
+				pkt->policy_to_use = BRANCH_RATIO;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Wrong policy_type received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "busy_hours")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int hour = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->timer_policy.busy_hours[i] = hour;
+			}
+		} else if (!strcmp(key, "quiet_hours")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int hour = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->timer_policy.quiet_hours[i] = hour;
+			}
+		} else if (!strcmp(key, "core_list")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int core = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->vcpu_to_control[i] = core;
+			}
+			pkt->num_vcpu = size;
+		} else if (!strcmp(key, "avg_packet_thresh")) {
+			pkt->traffic_policy.avg_max_packet_thresh =
+					(uint32_t)json_integer_value(value);
+		} else if (!strcmp(key, "max_packet_thresh")) {
+			pkt->traffic_policy.max_max_packet_thresh =
+					(uint32_t)json_integer_value(value);
+		} else if (!strcmp(key, "unit")) {
+			char unit[32];
+			snprintf(unit, 32, "%s", json_string_value(value));
+			if (!strcmp(unit, "SCALE_UP")) {
+				pkt->unit = CPU_POWER_SCALE_UP;
+			} else if (!strcmp(unit, "SCALE_DOWN")) {
+				pkt->unit = CPU_POWER_SCALE_DOWN;
+			} else if (!strcmp(unit, "SCALE_MAX")) {
+				pkt->unit = CPU_POWER_SCALE_MAX;
+			} else if (!strcmp(unit, "SCALE_MIN")) {
+				pkt->unit = CPU_POWER_SCALE_MIN;
+			} else if (!strcmp(unit, "ENABLE_TURBO")) {
+				pkt->unit = CPU_POWER_ENABLE_TURBO;
+			} else if (!strcmp(unit, "DISABLE_TURBO")) {
+				pkt->unit = CPU_POWER_DISABLE_TURBO;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Invalid command received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "resource_id")) {
+			pkt->resource_id = (uint32_t)json_integer_value(value);
+		} else {
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+				"Unknown key received in JSON string: %s\n",
+				key);
+		}
+	}
+	return 0;
+}
+#endif
+
 void channel_monitor_exit(void)
 {
 	run_loop = 0;
@@ -555,6 +690,103 @@ channel_monitor_init(void)
 	return 0;
 }
 
+static void
+read_binary_packet(struct channel_info *chan_info)
+{
+	struct channel_packet pkt;
+	void *buffer = &pkt;
+	int buffer_len = sizeof(pkt);
+	int n_bytes, err = 0;
+
+	while (buffer_len > 0) {
+		n_bytes = read(chan_info->fd,
+				buffer, buffer_len);
+		if (n_bytes == buffer_len)
+			break;
+		if (n_bytes == -1) {
+			err = errno;
+			RTE_LOG(DEBUG, CHANNEL_MONITOR,
+				"Received error on "
+				"channel '%s' read: %s\n",
+				chan_info->channel_path,
+				strerror(err));
+			remove_channel(&chan_info);
+			break;
+		}
+		buffer = (char *)buffer + n_bytes;
+		buffer_len -= n_bytes;
+	}
+	if (!err)
+		process_request(&pkt, chan_info);
+}
+
+#ifdef USE_JANSSON
+static void
+read_json_packet(struct channel_info *chan_info)
+{
+	struct channel_packet pkt;
+	int n_bytes, ret;
+	json_t *root;
+	json_error_t error;
+
+	/* read opening brace to closing brace */
+	do {
+		int idx = 0;
+		int indent = 0;
+		do {
+			n_bytes = read(chan_info->fd, &json_data[idx], 1);
+			if (n_bytes == 0)
+				break;
+			if (json_data[idx] == '{')
+				indent++;
+			if (json_data[idx] == '}')
+				indent--;
+			if ((indent > 0) || (idx > 0))
+				idx++;
+			if (indent == 0)
+				json_data[idx] = 0;
+			if (idx >= MAX_JSON_STRING_LEN-1)
+				break;
+		} while (indent > 0);
+
+		if (indent > 0)
+			/*
+			 * We've broken out of the read loop without getting
+			 * a closing brace, so throw away the data
+			 */
+			json_data[idx] = 0;
+
+		if (strlen(json_data) == 0)
+			continue;
+
+		printf("got [%s]\n", json_data);
+
+		root = json_loads(json_data, 0, &error);
+
+		if (root) {
+			/*
+			 * Because our data is now in the json
+			 * object, we can overwrite the pkt
+			 * with a channel_packet struct, using
+			 * parse_json_to_pkt()
+			 */
+			ret = parse_json_to_pkt(root, &pkt);
+			json_decref(root);
+			if (ret) {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Error validating JSON profile data\n");
+				break;
+			}
+			process_request(&pkt, chan_info);
+		} else {
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+					"JSON error on line %d: %s\n",
+					error.line, error.text);
+		}
+	} while (n_bytes > 0);
+}
+#endif
+
 void
 run_channel_monitor(void)
 {
@@ -578,31 +810,18 @@ run_channel_monitor(void)
 			}
 			if (global_events_list[i].events & EPOLLIN) {
 
-				int n_bytes, err = 0;
-				struct channel_packet pkt;
-				void *buffer = &pkt;
-				int buffer_len = sizeof(pkt);
-
-				while (buffer_len > 0) {
-					n_bytes = read(chan_info->fd,
-							buffer, buffer_len);
-					if (n_bytes == buffer_len)
-						break;
-					if (n_bytes == -1) {
-						err = errno;
-						RTE_LOG(DEBUG, CHANNEL_MONITOR,
-							"Received error on "
-							"channel '%s' read: %s\n",
-							chan_info->channel_path,
-							strerror(err));
-						remove_channel(&chan_info);
-						break;
-					}
-					buffer = (char *)buffer + n_bytes;
-					buffer_len -= n_bytes;
+				switch (chan_info->type) {
+				case CHANNEL_TYPE_BINARY:
+					read_binary_packet(chan_info);
+					break;
+#ifdef USE_JANSSON
+				case CHANNEL_TYPE_JSON:
+					read_json_packet(chan_info);
+					break;
+#endif
+				default:
+					break;
 				}
-				if (!err)
-					process_request(&pkt, chan_info);
 			}
 		}
 		rte_delay_us(time_period_ms*1000);
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v5 08/10] examples/power: clean up verbose messages
  2018-09-26 16:37       ` [dpdk-dev] [PATCH v5 0/10] add json power policy interface for containers David Hunt
                           ` (6 preceding siblings ...)
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 07/10] examples/power: add json string handling David Hunt
@ 2018-09-26 16:37         ` David Hunt
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 09/10] examples/power: add meson/ninja build support David Hunt
                           ` (2 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-26 16:37 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Some messages appearing several times a second, removing as they are
unnecessary. Other less severe messages change from INFO to DEBUG

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/channel_monitor.c | 19 +++++--------------
 1 file changed, 5 insertions(+), 14 deletions(-)

diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index 73a0451f3..92389ba5d 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -259,7 +259,7 @@ get_pcpu_to_control(struct policy *pol)
 
 	ci = get_core_info();
 
-	RTE_LOG(INFO, CHANNEL_MONITOR,
+	RTE_LOG(DEBUG, CHANNEL_MONITOR,
 			"Looking for pcpu for %s\n", pol->pkt.vm_name);
 
 	/*
@@ -426,8 +426,6 @@ apply_traffic_profile(struct policy *pol)
 
 	diff = get_pkt_diff(pol);
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Applying traffic profile\n");
-
 	if (diff >= (pol->pkt.traffic_policy.max_max_packet_thresh)) {
 		for (count = 0; count < pol->pkt.num_vcpu; count++) {
 			if (pol->core_share[count].status != 1)
@@ -471,9 +469,6 @@ apply_time_profile(struct policy *pol)
 				if (pol->core_share[count].status != 1) {
 					power_manager_scale_core_max(
 						pol->core_share[count].pcpu);
-				RTE_LOG(INFO, CHANNEL_MONITOR,
-					"Scaling up core %d to max\n",
-					pol->core_share[count].pcpu);
 				}
 			}
 			break;
@@ -483,9 +478,6 @@ apply_time_profile(struct policy *pol)
 				if (pol->core_share[count].status != 1) {
 					power_manager_scale_core_min(
 						pol->core_share[count].pcpu);
-				RTE_LOG(INFO, CHANNEL_MONITOR,
-					"Scaling down core %d to min\n",
-					pol->core_share[count].pcpu);
 			}
 		}
 			break;
@@ -547,8 +539,6 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (chan_info == NULL)
 		return -1;
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Processing Request %s\n", pkt->vm_name);
-
 	if (rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_CONNECTED,
 			CHANNEL_MGR_CHANNEL_PROCESSING) == 0)
 		return -1;
@@ -617,8 +607,8 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	}
 
 	if (pkt->command == PKT_POLICY) {
-		RTE_LOG(INFO, CHANNEL_MONITOR,
-				"\nProcessing Policy request\n");
+		RTE_LOG(INFO, CHANNEL_MONITOR, "Processing policy request %s\n",
+				pkt->vm_name);
 		update_policy(pkt);
 		policy_is_set = 1;
 	}
@@ -802,7 +792,8 @@ run_channel_monitor(void)
 					global_events_list[i].data.ptr;
 			if ((global_events_list[i].events & EPOLLERR) ||
 				(global_events_list[i].events & EPOLLHUP)) {
-				RTE_LOG(DEBUG, CHANNEL_MONITOR, "Remote closed connection for "
+				RTE_LOG(INFO, CHANNEL_MONITOR,
+						"Remote closed connection for "
 						"channel '%s'\n",
 						chan_info->channel_path);
 				remove_channel(&chan_info);
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v5 09/10] examples/power: add meson/ninja build support
  2018-09-26 16:37       ` [dpdk-dev] [PATCH v5 0/10] add json power policy interface for containers David Hunt
                           ` (7 preceding siblings ...)
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 08/10] examples/power: clean up verbose messages David Hunt
@ 2018-09-26 16:37         ` David Hunt
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 10/10] doc/vm_power_manager: add JSON interface API info David Hunt
  2018-10-02  8:43         ` [dpdk-dev] [PATCH v6 0/10] add json power policy interface for containers David Hunt
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-09-26 16:37 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Add meson.build in vm_power_manager and the guest_cli subdirectory.
Building can be achieved by going to the build directory, and using

meson configure -Dexamples=vm_power_manager,vm_power_manager/guest_cli

Then, when ninja is invoked, it will build dpdk-vm_power_manger and
dpdk-guest_cli

Work still needs to be done on the meson build system to handles the case
where the target list of example apps is defined as 'all'. That will come
in a future patch.

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
---
 .../vm_power_manager/guest_cli/meson.build    | 21 +++++++++++
 examples/vm_power_manager/meson.build         | 37 ++++++++++++++++++-
 2 files changed, 56 insertions(+), 2 deletions(-)
 create mode 100644 examples/vm_power_manager/guest_cli/meson.build

diff --git a/examples/vm_power_manager/guest_cli/meson.build b/examples/vm_power_manager/guest_cli/meson.build
new file mode 100644
index 000000000..9e821ceb8
--- /dev/null
+++ b/examples/vm_power_manager/guest_cli/meson.build
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+# Setting the name here because the default name will conflict with the
+# vm_power_manager app because of the way the directories are parsed.
+name = 'guest_cli'
+
+deps += ['power']
+
+sources = files(
+	'main.c', 'parse.c', 'vm_power_cli_guest.c'
+)
+
+opt_dep = cc.find_library('virt', required : false)
+build = opt_dep.found()
+ext_deps += opt_dep
diff --git a/examples/vm_power_manager/meson.build b/examples/vm_power_manager/meson.build
index c370d7476..f98445bc6 100644
--- a/examples/vm_power_manager/meson.build
+++ b/examples/vm_power_manager/meson.build
@@ -6,5 +6,38 @@
 # To build this example as a standalone application with an already-installed
 # DPDK instance, use 'make'
 
-# Example app currently unsupported by meson build
-build = false
+if dpdk_conf.has('RTE_LIBRTE_BNXT_PMD')
+	deps += ['pmd_bnxt']
+endif
+
+if dpdk_conf.has('RTE_LIBRTE_I40E_PMD')
+	deps += ['pmd_i40e']
+endif
+
+if dpdk_conf.has('RTE_LIBRTE_IXGBE_PMD')
+	deps += ['pmd_ixgbe']
+endif
+
+deps += ['power']
+
+
+sources = files(
+	'channel_manager.c', 'channel_monitor.c', 'main.c', 'parse.c', 'power_manager.c', 'vm_power_cli.c'
+)
+
+# If we're on X86, pull in the x86 code for the branch monitor algo.
+if dpdk_conf.has('RTE_ARCH_X86_64')
+	sources += files('oob_monitor_x86.c')
+else
+	sources += files('oob_monitor_nop.c')
+endif
+
+opt_dep = cc.find_library('virt', required : false)
+build = opt_dep.found()
+ext_deps += opt_dep
+
+opt_dep = dependency('jansson', required : false)
+if opt_dep.found()
+	ext_deps += opt_dep
+	cflags += '-DUSE_JANSSON'
+endif
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v5 10/10] doc/vm_power_manager: add JSON interface API info
  2018-09-26 16:37       ` [dpdk-dev] [PATCH v5 0/10] add json power policy interface for containers David Hunt
                           ` (8 preceding siblings ...)
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 09/10] examples/power: add meson/ninja build support David Hunt
@ 2018-09-26 16:37         ` David Hunt
  2018-09-29  2:42           ` Yao, Lei A
  2018-10-02  8:43         ` [dpdk-dev] [PATCH v6 0/10] add json power policy interface for containers David Hunt
  10 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-09-26 16:37 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Also added meson/ninja build info

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Marko Kovacevic <marko.kovacevic@intel.com>
---
 .../sample_app_ug/vm_power_management.rst     | 272 +++++++++++++++++-
 1 file changed, 270 insertions(+), 2 deletions(-)

diff --git a/doc/guides/sample_app_ug/vm_power_management.rst b/doc/guides/sample_app_ug/vm_power_management.rst
index 855570d6b..1afe42809 100644
--- a/doc/guides/sample_app_ug/vm_power_management.rst
+++ b/doc/guides/sample_app_ug/vm_power_management.rst
@@ -199,7 +199,7 @@ see :doc:`compiling`.
 
 The application is located in the ``vm_power_manager`` sub-directory.
 
-To build just the ``vm_power_manager`` application:
+To build just the ``vm_power_manager`` application using ``make``:
 
 .. code-block:: console
 
@@ -208,6 +208,22 @@ To build just the ``vm_power_manager`` application:
   cd ${RTE_SDK}/examples/vm_power_manager/
   make
 
+The resulting binary will be ${RTE_SDK}/build/examples/vm_power_manager
+
+To build just the ``vm_power_manager`` application using ``meson/ninja``:
+
+.. code-block:: console
+
+  export RTE_SDK=/path/to/rte_sdk
+  cd ${RTE_SDK}
+  meson build
+  cd build
+  ninja
+  meson configure -Dexamples=vm_power_manager
+  ninja
+
+The resulting binary will be ${RTE_SDK}/build/examples/dpdk-vm_power_manager
+
 Running
 ~~~~~~~
 
@@ -337,6 +353,242 @@ monitoring of branch ratio on cores doing busy polling via PMDs.
   and will need to be adjusted for different workloads.
 
 
+
+JSON API
+~~~~~~~~
+
+In addition to the command line interface for host command and a virtio-serial
+interface for VM power policies, there is also a JSON interface through which
+power commands and policies can be sent. This functionality adds a dependency
+on the Jansson library, and the Jansson development package must be installed
+on the system before the JSON parsing functionality is included in the app.
+This is achieved by:
+
+  .. code-block:: javascript
+
+    apt-get install libjansson-dev
+
+The command and package name may be different depending on your operating
+system. It's worth noting that the app will successfully build without this
+package present, but a warning is shown during compilation, and the JSON
+parsing functionality will not be present in the app.
+
+Sending a command or policy to the power manager application is achieved by
+simply opening a fifo file, writing a JSON string to that fifo, and closing
+the file.
+
+The fifo is at /tmp/powermonitor/fifo
+
+The jason string can be a policy or instruction, and takes the following
+format:
+
+  .. code-block:: javascript
+
+    {"packet_type": {
+      "pair_1": value,
+      "pair_2": value
+    }}
+
+The 'packet_type' header can contain one of two values, depending on
+whether a policy or power command is being sent. The two possible values are
+"policy" and "instruction", and the expected name-value pairs is different
+depending on which type is being sent.
+
+The pairs are the format of standard JSON name-value pairs. The value type
+varies between the different name/value pairs, and may be integers, strings,
+arrays, etc. Examples of policies follow later in this document. The allowed
+names and value types are as follows:
+
+
+:Pair Name: "name"
+:Description: Name of the VM or Host. Allows the parser to associate the
+  policy with the relevant VM or Host OS.
+:Type: string
+:Values: any valid string
+:Required: yes
+:Example:
+
+    .. code-block:: javascript
+
+      "name", "ubuntu2"
+
+
+:Pair Name: "command"
+:Description: The type of packet we're sending to the power manager. We can be
+  creating or destroying a policy, or sending a direct command to adjust
+  the frequency of a core, similar to the command line interface.
+:Type: string
+:Values:
+
+  :CREATE: used when creating a new policy,
+  :DESTROY: used when removing a policy,
+  :POWER: used when sending an immediate command, max, min, etc.
+:Required: yes
+:Example:
+
+    .. code-block:: javascript
+
+      "command", "CREATE"
+
+
+:Pair Name: "policy_type"
+:Description: Type of policy to apply. Please see vm_power_manager documentation
+  for more information on the types of policies that may be used.
+:Type: string
+:Values:
+
+  :TIME: Time-of-day policy. Frequencies of the relevant cores are
+    scaled up/down depending on busy and quiet hours.
+  :TRAFFIC: This policy takes statistics from the NIC and scales up
+    and down accordingly.
+  :WORKLOAD: This policy looks at how heavily loaded the cores are,
+    and scales up and down accordingly.
+  :BRANCH_RATIO: This out-of-band policy can look at the ratio between
+    branch hits and misses on a core, and is useful for detecting
+    how much packet processing a core is doing.
+:Required: only for CREATE/DESTROY command
+:Example:
+
+  .. code-block:: javascript
+
+    "policy_type", "TIME"
+
+:Pair Name: "busy_hours"
+:Description: The hours of the day in which we scale up the cores for busy
+  times.
+:Type: array of integers
+:Values: array with list of hour numbers, (0-23)
+:Required: only for TIME policy
+:Example:
+
+  .. code-block:: javascript
+
+    "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ]
+
+:Pair Name: "quiet_hours"
+:Description: The hours of the day in which we scale down the cores for quiet
+  times.
+:Type: array of integers
+:Values: array with list of hour numbers, (0-23)
+:Required: only for TIME policy
+:Example:
+
+  .. code-block:: javascript
+
+    "quiet_hours":[ 2, 3, 4, 5, 6 ]
+
+:Pair Name: "avg_packet_thresh"
+:Description: Threshold below which the frequency will be set to min for
+  the TRAFFIC policy. If the traffic rate is above this and below max, the
+  frequency will be set to medium.
+:Type: integer
+:Values: The number of packets below which the TRAFFIC policy applies the
+  minimum frequency, or medium frequency if between avg and max thresholds.
+:Required: only for TRAFFIC policy
+:Example:
+
+  .. code-block:: javascript
+
+    "avg_packet_thresh": 100000
+
+:Pair Name: "max_packet_thresh"
+:Description: Threshold above which the frequency will be set to max for
+  the TRAFFIC policy
+:Type: integer
+:Values: The number of packets per interval above which the TRAFFIC policy
+  applies the maximum frequency
+:Required: only for TRAFFIC policy
+:Example:
+
+  .. code-block:: javascript
+
+    "max_packet_thresh": 500000
+
+:Pair Name: "core_list"
+:Description: The cores to which to apply the policy.
+:Type: array of integers
+:Values: array with list of virtual CPUs.
+:Required: only policy CREATE/DESTROY
+:Example:
+
+  .. code-block:: javascript
+
+    "core_list":[ 10, 11 ]
+
+:Pair Name: "unit"
+:Description: the type of power operation to apply in the command
+:Type: string
+:Values:
+
+  :SCALE_MAX: Scale frequency of this core to maximum
+  :SCALE_MIN: Scale frequency of this core to minimum
+  :SCALE_UP: Scale up frequency of this core
+  :SCALE_DOWN: Scale down frequency of this core
+  :ENABLE_TURBO: Enable Turbo Boost for this core
+  :DISABLE_TURBO: Disable Turbo Boost for this core
+:Required: only for POWER instruction
+:Example:
+
+  .. code-block:: javascript
+
+    "unit", "SCALE_MAX"
+
+:Pair Name: "resource_id"
+:Description: The core to which to apply the power command.
+:Type: integer
+:Values: valid core id for VM or host OS.
+:Required: only POWER instruction
+:Example:
+
+  .. code-block:: javascript
+
+    "resource_id": 10
+
+JSON API Examples
+~~~~~~~~~~~~~~~~~
+
+Profile create example:
+
+  .. code-block:: javascript
+
+    {"policy": {
+      "name": "ubuntu",
+      "command": "create",
+      "policy_type": "TIME",
+      "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
+      "quiet_hours":[ 2, 3, 4, 5, 6 ],
+      "core_list":[ 11 ]
+    }}
+
+Profile destroy example:
+
+  .. code-block:: javascript
+
+    {"profile": {
+      "name": "ubuntu",
+      "command": "destroy",
+    }}
+
+Power command example:
+
+  .. code-block:: javascript
+
+    {"command": {
+      "name": "ubuntu",
+      "unit": "SCALE_MAX",
+      "resource_id": 10
+    }}
+
+To send a JSON string to the Power Manager application, simply paste the
+example JSON string into a text file and cat it into the fifo:
+
+  .. code-block:: console
+
+    cat file.json >/tmp/powermonitor/fifo
+
+The console of the Power Manager application should indicate the command that
+was just received via the fifo.
+
 Compiling and Running the Guest Applications
 --------------------------------------------
 
@@ -366,7 +618,7 @@ For compiling and running l3fwd-power, see :doc:`l3_forward_power_man`.
 
 The application is located in the ``guest_cli`` sub-directory under ``vm_power_manager``.
 
-To build just the ``guest_vm_power_manager`` application:
+To build just the ``guest_vm_power_manager`` application using ``make``:
 
 .. code-block:: console
 
@@ -375,6 +627,22 @@ To build just the ``guest_vm_power_manager`` application:
   cd ${RTE_SDK}/examples/vm_power_manager/guest_cli/
   make
 
+The resulting binary will be ${RTE_SDK}/build/examples/guest_cli
+
+To build just the ``vm_power_manager`` application using ``meson/ninja``:
+
+.. code-block:: console
+
+  export RTE_SDK=/path/to/rte_sdk
+  cd ${RTE_SDK}
+  meson build
+  cd build
+  ninja
+  meson configure -Dexamples=vm_power_manager/guest_cli
+  ninja
+
+The resulting binary will be ${RTE_SDK}/build/examples/guest_cli
+
 Running
 ~~~~~~~
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v5 10/10] doc/vm_power_manager: add JSON interface API info
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 10/10] doc/vm_power_manager: add JSON interface API info David Hunt
@ 2018-09-29  2:42           ` Yao, Lei A
  2018-10-01 11:02             ` Hunt, David
  0 siblings, 1 reply; 110+ messages in thread
From: Yao, Lei A @ 2018-09-29  2:42 UTC (permalink / raw)
  To: Hunt, David, dev; +Cc: Mcnamara, John, stephen, Yao, Lei A, Burakov, Anatoly

> +:Pair Name: "avg_packet_thresh"
> +:Description: Threshold below which the frequency will be set to min for
> +  the TRAFFIC policy. If the traffic rate is above this and below max, the
> +  frequency will be set to medium.
> +:Type: integer
> +:Values: The number of packets below which the TRAFFIC policy applies the
> +  minimum frequency, or medium frequency if between avg and max
> thresholds.
> +:Required: only for TRAFFIC policy
> +:Example:
> +
> +  .. code-block:: javascript
> +
> +    "avg_packet_thresh": 100000

Hi, Dave

For this traffic policy , seems in previous release, we depends on the 
application in VM to send the VF mac address info back to the host 
vm_power_manager sample in host  through virtio-serial port. 
If this JOSN interface is designed for container using VF, do we need add 
VF mac or ID related info into the TRAFFIC policy JSON file? Otherwise, 
the host don't know which port  to monitor the throughput, I guess. 

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v5 07/10] examples/power: add json string handling
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 07/10] examples/power: add json string handling David Hunt
@ 2018-09-30  1:54           ` Yao, Lei A
  2018-10-01 11:03             ` Hunt, David
  0 siblings, 1 reply; 110+ messages in thread
From: Yao, Lei A @ 2018-09-30  1:54 UTC (permalink / raw)
  To: Hunt, David, dev; +Cc: Mcnamara, John, stephen, Burakov, Anatoly

> +#ifdef USE_JANSSON
> +static int
> +parse_json_to_pkt(json_t *element, struct channel_packet *pkt)
> +{
> +	const char *key;
> +	json_t *value;
> +	int ret;
> +
> +	memset(pkt, 0, sizeof(struct channel_packet));
> +
> +	pkt->nb_mac_to_monitor = 0;
> +	pkt->t_boost_status.tbEnabled = false;
> +	pkt->workload = LOW;
> +	pkt->policy_to_use = TIME;
> +	pkt->command = PKT_POLICY;
> +	pkt->core_type = CORE_TYPE_PHYSICAL;
> +


Hi, Dave

For the workload policy ,  it's set to LOW by default,
but we can't change it again using JSON file channel. Is it 
by design?  

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v5 10/10] doc/vm_power_manager: add JSON interface API info
  2018-09-29  2:42           ` Yao, Lei A
@ 2018-10-01 11:02             ` Hunt, David
  0 siblings, 0 replies; 110+ messages in thread
From: Hunt, David @ 2018-10-01 11:02 UTC (permalink / raw)
  To: Yao, Lei A, dev; +Cc: Mcnamara, John, stephen, Burakov, Anatoly

Hi Lei,


On 29/9/2018 3:42 AM, Yao, Lei A wrote:
>> +:Pair Name: "avg_packet_thresh"
>> +:Description: Threshold below which the frequency will be set to min for
>> +  the TRAFFIC policy. If the traffic rate is above this and below max, the
>> +  frequency will be set to medium.
>> +:Type: integer
>> +:Values: The number of packets below which the TRAFFIC policy applies the
>> +  minimum frequency, or medium frequency if between avg and max
>> thresholds.
>> +:Required: only for TRAFFIC policy
>> +:Example:
>> +
>> +  .. code-block:: javascript
>> +
>> +    "avg_packet_thresh": 100000
> Hi, Dave
>
> For this traffic policy , seems in previous release, we depends on the
> application in VM to send the VF mac address info back to the host
> vm_power_manager sample in host  through virtio-serial port.
> If this JOSN interface is designed for container using VF, do we need add
> VF mac or ID related info into the TRAFFIC policy JSON file? Otherwise,
> the host don't know which port  to monitor the throughput, I guess.
>
>

Well spotted. I'll add this and upload a new version.

Thanks,
Dave.

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v5 07/10] examples/power: add json string handling
  2018-09-30  1:54           ` Yao, Lei A
@ 2018-10-01 11:03             ` Hunt, David
  0 siblings, 0 replies; 110+ messages in thread
From: Hunt, David @ 2018-10-01 11:03 UTC (permalink / raw)
  To: Yao, Lei A, dev; +Cc: Mcnamara, John, stephen, Burakov, Anatoly

Hi Lei,


On 30/9/2018 2:54 AM, Yao, Lei A wrote:
>> +#ifdef USE_JANSSON
>> +static int
>> +parse_json_to_pkt(json_t *element, struct channel_packet *pkt)
>> +{
>> +	const char *key;
>> +	json_t *value;
>> +	int ret;
>> +
>> +	memset(pkt, 0, sizeof(struct channel_packet));
>> +
>> +	pkt->nb_mac_to_monitor = 0;
>> +	pkt->t_boost_status.tbEnabled = false;
>> +	pkt->workload = LOW;
>> +	pkt->policy_to_use = TIME;
>> +	pkt->command = PKT_POLICY;
>> +	pkt->core_type = CORE_TYPE_PHYSICAL;
>> +
>
> Hi, Dave
>
> For the workload policy ,  it's set to LOW by default,
> but we can't change it again using JSON file channel. Is it
> by design?

It was because the VMs currently only send binary structures. But they
may send JSON data in the future so I should add it now.
I'll upload a new version of the patch set today.

Thanks,
Dave.

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v6 0/10] add json power policy interface for containers
  2018-09-26 16:37       ` [dpdk-dev] [PATCH v5 0/10] add json power policy interface for containers David Hunt
                           ` (9 preceding siblings ...)
  2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 10/10] doc/vm_power_manager: add JSON interface API info David Hunt
@ 2018-10-02  8:43         ` David Hunt
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 01/10] examples/power: add checks around hypervisor David Hunt
                             ` (10 more replies)
  10 siblings, 11 replies; 110+ messages in thread
From: David Hunt @ 2018-10-02  8:43 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov

The current vm_power_manager example app has the capability to accept power
policies from virtual machines via virtio-serial channels. These power
policies  allow a virtual machine to give information to the power manager
to allow the power manager take care of the power management of the virtual
machine based on the information in the policy.

This power policy functionality is limited to virtual machines sending
the policies to the power manager (which runs in the Host OS), and a solution
was needed for additional methods of sending power policies to the power
manager app.

The main use-case for this modification is for containers and host
applications that wish to send polices to the power manager.

This patchset adds the capability to send power polices and power commands
to the vm_power_manager app via JSON strings through a fifo on the file
system.
For example, given the following file, policy.json:

{"policy": {
  "name": "ubuntu2",
  "command": "create",
  "policy_type": "TIME",
  "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
  "quiet_hours":[ 2, 3, 4, 5, 6 ],
  "core_list":[ 11, 12, 13 ]
}}

Then running the command:

cat policy.json >/tmp/powermonitor/fifo

The policy is sent to the vm_power_manager. The power manager app then parses
the JSON data, and inserts the policy into the array of policies.

Part of the patch series contains documentation updates to give all the
details of the valid name-value pairs, the data types, etc.

Patch v2:
  * Fixed review comments from Stephen Hemminger and Lei A Yao.
  * Added a check in the Makefile for libjansson-dev. Will Warn user and build
    without JSON functionality if not present, will build including JSON
    functionality if it is present.

Patch v3:
  * Added meson/ninja support for vm_power_manager and guest_cli apps
  * Fixed compilation issue with guest_cli app

Patch v4:
  * Split out some unrelated changes to separate patches in the set
  * Some changes out of review by Anatoly (Thanks!)

Patch v5:
  * Removed the directory with JSON examples, as they already exist in
    the documentation.
  * Fixed some typos and formatting issues in the documentation.
  * Changed the JSON examples in the documentation to 'javascript' causing
    the syntax to be highlighted nicely.
  * Inherited the Acks from previous version.

Patch v6:
  * Added ability to set WORKLOAD policy to LOW, MEDIUM, or HIGH.
    "workload": "MEDIUM"
  * Added missing functionality to allow passing of a list of mac
    addresses for the TRAFFIC profile type.
    "mac_list":[ "de:ad:be:ef:01:01", "de:ad:be:ef:01:02" ]
  * Updated docs to include both of the above additions.

[01/10] examples/power: add checks around hypervisor
[02/10] examples/power: allow for number of vms to be zero
[03/10] lib/power: add changes for host commands/policies
[04/10] examples/power: add necessary changes to guest app
[05/10] examples/power: add host channel to power manager
[06/10] examples/power: increase allowed number of clients
[07/10] examples/power: add json string handling
[08/10] examples/power: clean up verbose messages
[09/10] examples/power: add meson/ninja build support
[10/10] doc/vm_power_manager: add JSON interface API info

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v6 01/10] examples/power: add checks around hypervisor
  2018-10-02  8:43         ` [dpdk-dev] [PATCH v6 0/10] add json power policy interface for containers David Hunt
@ 2018-10-02  8:43           ` David Hunt
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 02/10] examples/power: allow for number of vms to be zero David Hunt
                             ` (9 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-02  8:43 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Allow vm_power_manager to run without requiring qemu to be present
on the machine. This will be required for instances where the JSON
interface is used for commands and polices, without any VMs present.
A use case for this is a container enviromnent.

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/channel_manager.c | 71 +++++++++++++--------
 1 file changed, 43 insertions(+), 28 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.c b/examples/vm_power_manager/channel_manager.c
index 927fc35ab..2e471d0c1 100644
--- a/examples/vm_power_manager/channel_manager.c
+++ b/examples/vm_power_manager/channel_manager.c
@@ -43,7 +43,8 @@ static unsigned char *global_cpumaps;
 static virVcpuInfo *global_vircpuinfo;
 static size_t global_maplen;
 
-static unsigned global_n_host_cpus;
+static unsigned int global_n_host_cpus;
+static bool global_hypervisor_available;
 
 /*
  * Represents a single Virtual Machine
@@ -198,7 +199,11 @@ get_pcpus_mask(struct channel_info *chan_info, unsigned vcpu)
 {
 	struct virtual_machine_info *vm_info =
 			(struct virtual_machine_info *)chan_info->priv_info;
-	return rte_atomic64_read(&vm_info->pcpu_mask[vcpu]);
+
+	if (global_hypervisor_available && (vm_info != NULL))
+		return rte_atomic64_read(&vm_info->pcpu_mask[vcpu]);
+	else
+		return 0;
 }
 
 static inline int
@@ -559,6 +564,8 @@ get_all_vm(int *num_vm, int *num_vcpu)
 				VIR_CONNECT_LIST_DOMAINS_PERSISTENT;
 	unsigned int domain_flag = VIR_DOMAIN_VCPU_CONFIG;
 
+	if (!global_hypervisor_available)
+		return;
 
 	memset(global_cpumaps, 0, CHANNEL_CMDS_MAX_CPUS*global_maplen);
 	if (virNodeGetInfo(global_vir_conn_ptr, &node_info)) {
@@ -768,38 +775,42 @@ connect_hypervisor(const char *path)
 	}
 	return 0;
 }
-
 int
-channel_manager_init(const char *path)
+channel_manager_init(const char *path __rte_unused)
 {
 	virNodeInfo info;
 
 	LIST_INIT(&vm_list_head);
 	if (connect_hypervisor(path) < 0) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to initialize channel manager\n");
-		return -1;
-	}
-
-	global_maplen = VIR_CPU_MAPLEN(CHANNEL_CMDS_MAX_CPUS);
+		global_n_host_cpus = 64;
+		global_hypervisor_available = 0;
+		RTE_LOG(INFO, CHANNEL_MANAGER, "Unable to initialize channel manager\n");
+	} else {
+		global_hypervisor_available = 1;
+
+		global_maplen = VIR_CPU_MAPLEN(CHANNEL_CMDS_MAX_CPUS);
+
+		global_vircpuinfo = rte_zmalloc(NULL,
+				sizeof(*global_vircpuinfo) *
+				CHANNEL_CMDS_MAX_CPUS, RTE_CACHE_LINE_SIZE);
+		if (global_vircpuinfo == NULL) {
+			RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for CPU Info\n");
+			goto error;
+		}
+		global_cpumaps = rte_zmalloc(NULL,
+				CHANNEL_CMDS_MAX_CPUS * global_maplen,
+				RTE_CACHE_LINE_SIZE);
+		if (global_cpumaps == NULL)
+			goto error;
 
-	global_vircpuinfo = rte_zmalloc(NULL, sizeof(*global_vircpuinfo) *
-			CHANNEL_CMDS_MAX_CPUS, RTE_CACHE_LINE_SIZE);
-	if (global_vircpuinfo == NULL) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for CPU Info\n");
-		goto error;
-	}
-	global_cpumaps = rte_zmalloc(NULL, CHANNEL_CMDS_MAX_CPUS * global_maplen,
-			RTE_CACHE_LINE_SIZE);
-	if (global_cpumaps == NULL) {
-		goto error;
+		if (virNodeGetInfo(global_vir_conn_ptr, &info)) {
+			RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to retrieve node Info\n");
+			goto error;
+		}
+		global_n_host_cpus = (unsigned int)info.cpus;
 	}
 
-	if (virNodeGetInfo(global_vir_conn_ptr, &info)) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to retrieve node Info\n");
-		goto error;
-	}
 
-	global_n_host_cpus = (unsigned)info.cpus;
 
 	if (global_n_host_cpus > CHANNEL_CMDS_MAX_CPUS) {
 		RTE_LOG(WARNING, CHANNEL_MANAGER, "The number of host CPUs(%u) exceeds the "
@@ -811,7 +822,8 @@ channel_manager_init(const char *path)
 
 	return 0;
 error:
-	disconnect_hypervisor();
+	if (global_hypervisor_available)
+		disconnect_hypervisor();
 	return -1;
 }
 
@@ -838,7 +850,10 @@ channel_manager_exit(void)
 		rte_free(vm_info);
 	}
 
-	rte_free(global_cpumaps);
-	rte_free(global_vircpuinfo);
-	disconnect_hypervisor();
+	if (global_hypervisor_available) {
+		/* Only needed if hypervisor available */
+		rte_free(global_cpumaps);
+		rte_free(global_vircpuinfo);
+		disconnect_hypervisor();
+	}
 }
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v6 02/10] examples/power: allow for number of vms to be zero
  2018-10-02  8:43         ` [dpdk-dev] [PATCH v6 0/10] add json power policy interface for containers David Hunt
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 01/10] examples/power: add checks around hypervisor David Hunt
@ 2018-10-02  8:43           ` David Hunt
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 03/10] lib/power: add changes for host commands/policies David Hunt
                             ` (8 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-02  8:43 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Previously the vm_power_manager app required to have some vms defined, so
the call to get_all_vm() always set the noVms variable. Now we're accepting
policies from the host OS (without any VMs defined), so it is now valid to
have zero VMs. This patch initialises the relevant variables to zero just
in case the call to get_all_vms() does not find any, so could return with
the variables uninitialised.

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/channel_monitor.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index 7fa47ba97..f180d74e6 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -66,7 +66,7 @@ static void
 core_share_status(int pNo)
 {
 
-	int noVms, noVcpus, z, x, t;
+	int noVms = 0, noVcpus = 0, z, x, t;
 
 	get_all_vm(&noVms, &noVcpus);
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v6 03/10] lib/power: add changes for host commands/policies
  2018-10-02  8:43         ` [dpdk-dev] [PATCH v6 0/10] add json power policy interface for containers David Hunt
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 01/10] examples/power: add checks around hypervisor David Hunt
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 02/10] examples/power: allow for number of vms to be zero David Hunt
@ 2018-10-02  8:43           ` David Hunt
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 04/10] examples/power: add necessary changes to guest app David Hunt
                             ` (7 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-02  8:43 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

This patch does a couple of things:
  * Adds a new message type for removing policies (PKT_POLICY_REMOVE)
    Used when we want to remove a previously created policy.
  * Adds a core_type bool to the channel packet struct to specify whether
    the type of core we want to control is cirtual or physical.

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 lib/librte_power/channel_commands.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/lib/librte_power/channel_commands.h b/lib/librte_power/channel_commands.h
index ee638eefa..e7b93a797 100644
--- a/lib/librte_power/channel_commands.h
+++ b/lib/librte_power/channel_commands.h
@@ -19,6 +19,7 @@ extern "C" {
 #define CPU_POWER               1
 #define CPU_POWER_CONNECT       2
 #define PKT_POLICY              3
+#define PKT_POLICY_REMOVE       4
 
 /* CPU Power Command Scaling */
 #define CPU_POWER_SCALE_UP      1
@@ -58,6 +59,9 @@ struct traffic {
 	uint32_t max_max_packet_thresh;
 };
 
+#define CORE_TYPE_VIRTUAL 0
+#define CORE_TYPE_PHYSICAL 1
+
 struct channel_packet {
 	uint64_t resource_id; /**< core_num, device */
 	uint32_t unit;        /**< scale down/up/min/max */
@@ -70,6 +74,7 @@ struct channel_packet {
 	uint8_t vcpu_to_control[MAX_VCPU_PER_VM];
 	uint8_t num_vcpu;
 	struct timer_profile timer_policy;
+	bool core_type;
 	enum workload workload;
 	enum policy_to_use policy_to_use;
 	struct t_boost_status t_boost_status;
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v6 04/10] examples/power: add necessary changes to guest app
  2018-10-02  8:43         ` [dpdk-dev] [PATCH v6 0/10] add json power policy interface for containers David Hunt
                             ` (2 preceding siblings ...)
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 03/10] lib/power: add changes for host commands/policies David Hunt
@ 2018-10-02  8:43           ` David Hunt
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 05/10] examples/power: add host channel to power manager David Hunt
                             ` (6 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-02  8:43 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

The changes here are minimal, as the guest app functionality is not
changing at all, but there is a new element in the channel_packet
struct that needs to have a default set (channel_packet->core_type).

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/guest_cli/vm_power_cli_guest.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
index 0db1b804f..2d9e7689a 100644
--- a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
+++ b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
@@ -92,6 +92,7 @@ set_policy_defaults(struct channel_packet *pkt)
 	pkt->timer_policy.hours_to_use_traffic_profile[0] = 8;
 	pkt->timer_policy.hours_to_use_traffic_profile[1] = 10;
 
+	pkt->core_type = CORE_TYPE_VIRTUAL;
 	pkt->workload = LOW;
 	pkt->policy_to_use = TIME;
 	pkt->command = PKT_POLICY;
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v6 05/10] examples/power: add host channel to power manager
  2018-10-02  8:43         ` [dpdk-dev] [PATCH v6 0/10] add json power policy interface for containers David Hunt
                             ` (3 preceding siblings ...)
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 04/10] examples/power: add necessary changes to guest app David Hunt
@ 2018-10-02  8:43           ` David Hunt
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 06/10] examples/power: increase allowed number of clients David Hunt
                             ` (5 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-02  8:43 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

This patch adds a fifo channel to the vm_power_manager app through which
we can send commands and polices. Intended for sending JSON strings.
The fifo is at /tmp/powermonitor/fifo

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/channel_manager.c | 109 +++++++++++++++
 examples/vm_power_manager/channel_manager.h |  17 +++
 examples/vm_power_manager/channel_monitor.c | 142 +++++++++++++++-----
 examples/vm_power_manager/main.c            |   2 +
 4 files changed, 236 insertions(+), 34 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.c b/examples/vm_power_manager/channel_manager.c
index 2e471d0c1..4fac099df 100644
--- a/examples/vm_power_manager/channel_manager.c
+++ b/examples/vm_power_manager/channel_manager.c
@@ -13,6 +13,7 @@
 
 #include <sys/queue.h>
 #include <sys/types.h>
+#include <sys/stat.h>
 #include <sys/socket.h>
 #include <sys/select.h>
 
@@ -284,6 +285,38 @@ open_non_blocking_channel(struct channel_info *info)
 	return 0;
 }
 
+static int
+open_host_channel(struct channel_info *info)
+{
+	int flags;
+
+	info->fd = open(info->channel_path, O_RDWR | O_RSYNC);
+	if (info->fd == -1) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Error(%s) opening fifo for '%s'\n",
+				strerror(errno),
+				info->channel_path);
+		return -1;
+	}
+
+	/* Get current flags */
+	flags = fcntl(info->fd, F_GETFL, 0);
+	if (flags < 0) {
+		RTE_LOG(WARNING, CHANNEL_MANAGER, "Error(%s) fcntl get flags socket for"
+				"'%s'\n", strerror(errno), info->channel_path);
+		return 1;
+	}
+	/* Set to Non Blocking */
+	flags |= O_NONBLOCK;
+	if (fcntl(info->fd, F_SETFL, flags) < 0) {
+		RTE_LOG(WARNING, CHANNEL_MANAGER,
+				"Error(%s) setting non-blocking "
+				"socket for '%s'\n",
+				strerror(errno), info->channel_path);
+		return -1;
+	}
+	return 0;
+}
+
 static int
 setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 		struct channel_info **chan_info_dptr, unsigned channel_num)
@@ -294,6 +327,7 @@ setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 	chan_info->channel_num = channel_num;
 	chan_info->priv_info = (void *)vm_info;
 	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
+	chan_info->type = CHANNEL_TYPE_BINARY;
 	if (open_non_blocking_channel(chan_info) < 0) {
 		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open channel: "
 				"'%s' for VM '%s'\n",
@@ -316,6 +350,42 @@ setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 	return 0;
 }
 
+static void
+fifo_path(char *dst, unsigned int len)
+{
+	snprintf(dst, len, "%sfifo", CHANNEL_MGR_SOCKET_PATH);
+}
+
+static int
+setup_host_channel_info(struct channel_info **chan_info_dptr,
+		unsigned int channel_num)
+{
+	struct channel_info *chan_info = *chan_info_dptr;
+
+	chan_info->channel_num = channel_num;
+	chan_info->priv_info = (void *)NULL;
+	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
+	chan_info->type = CHANNEL_TYPE_JSON;
+
+	fifo_path(chan_info->channel_path, sizeof(chan_info->channel_path));
+
+	if (open_host_channel(chan_info) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open host channel: "
+				"'%s'\n",
+				chan_info->channel_path);
+		return -1;
+	}
+	if (add_channel_to_monitor(&chan_info) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Could add channel: "
+				"'%s' to epoll ctl\n",
+				chan_info->channel_path);
+		return -1;
+
+	}
+	chan_info->status = CHANNEL_MGR_CHANNEL_CONNECTED;
+	return 0;
+}
+
 int
 add_all_channels(const char *vm_name)
 {
@@ -470,6 +540,45 @@ add_channels(const char *vm_name, unsigned *channel_list,
 	return num_channels_enabled;
 }
 
+int
+add_host_channel(void)
+{
+	struct channel_info *chan_info;
+	char socket_path[PATH_MAX];
+	int num_channels_enabled = 0;
+	int ret;
+
+	fifo_path(socket_path, sizeof(socket_path));
+
+	ret = mkfifo(socket_path, 0660);
+	if ((errno != EEXIST) && (ret < 0)) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Cannot create fifo '%s' error: "
+				"%s\n", socket_path, strerror(errno));
+		return 0;
+	}
+
+	if (access(socket_path, F_OK) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Channel path '%s' error: "
+				"%s\n", socket_path, strerror(errno));
+		return 0;
+	}
+	chan_info = rte_malloc(NULL, sizeof(*chan_info), 0);
+	if (chan_info == NULL) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for "
+				"channel '%s'\n", socket_path);
+		return 0;
+	}
+	snprintf(chan_info->channel_path,
+			sizeof(chan_info->channel_path), "%s", socket_path);
+	if (setup_host_channel_info(&chan_info, 0) < 0) {
+		rte_free(chan_info);
+		return 0;
+	}
+	num_channels_enabled++;
+
+	return num_channels_enabled;
+}
+
 int
 remove_channel(struct channel_info **chan_info_dptr)
 {
diff --git a/examples/vm_power_manager/channel_manager.h b/examples/vm_power_manager/channel_manager.h
index 872ec6140..e32235b07 100644
--- a/examples/vm_power_manager/channel_manager.h
+++ b/examples/vm_power_manager/channel_manager.h
@@ -54,6 +54,13 @@ enum channel_status { CHANNEL_MGR_CHANNEL_DISCONNECTED = 0,
 	CHANNEL_MGR_CHANNEL_DISABLED,
 	CHANNEL_MGR_CHANNEL_PROCESSING};
 
+/* Communication Channel Type */
+enum channel_type {
+	CHANNEL_TYPE_BINARY = 0,
+	CHANNEL_TYPE_INI,
+	CHANNEL_TYPE_JSON
+};
+
 /* VM libvirt(qemu/KVM) connection status */
 enum vm_status { CHANNEL_MGR_VM_INACTIVE = 0, CHANNEL_MGR_VM_ACTIVE};
 
@@ -66,6 +73,7 @@ struct channel_info {
 	volatile uint32_t status;    /**< Connection status(enum channel_status) */
 	int fd;                      /**< AF_UNIX socket fd */
 	unsigned channel_num;        /**< CHANNEL_MGR_SOCKET_PATH/<vm_name>.channel_num */
+	enum channel_type type;      /**< Binary, ini, json, etc. */
 	void *priv_info;             /**< Pointer to private info, do not modify */
 };
 
@@ -226,6 +234,15 @@ int add_all_channels(const char *vm_name);
 int add_channels(const char *vm_name, unsigned *channel_list,
 		unsigned num_channels);
 
+/**
+ * Set up a fifo by which host applications can send command an policies
+ * through a fifo to the vm_power_manager
+ *
+ * @return
+ *  - 0 for success
+ */
+int add_host_channel(void);
+
 /**
  * Remove a channel definition from the channel manager. This must only be
  * called from the channel monitor thread.
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index f180d74e6..c3c3d7bb1 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -85,6 +85,33 @@ core_share_status(int pNo)
 	}
 }
 
+
+static int
+pcpu_monitor(struct policy *pol, struct core_info *ci, int pcpu, int count)
+{
+	int ret = 0;
+
+	if (pol->pkt.policy_to_use == BRANCH_RATIO) {
+		ci->cd[pcpu].oob_enabled = 1;
+		ret = add_core_to_monitor(pcpu);
+		if (ret == 0)
+			RTE_LOG(INFO, CHANNEL_MONITOR,
+					"Monitoring pcpu %d OOB for %s\n",
+					pcpu, pol->pkt.vm_name);
+		else
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Error monitoring pcpu %d OOB for %s\n",
+					pcpu, pol->pkt.vm_name);
+
+	} else {
+		pol->core_share[count].pcpu = pcpu;
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				"Monitoring pcpu %d for %s\n",
+				pcpu, pol->pkt.vm_name);
+	}
+	return ret;
+}
+
 static void
 get_pcpu_to_control(struct policy *pol)
 {
@@ -94,34 +121,42 @@ get_pcpu_to_control(struct policy *pol)
 	int pcpu, count;
 	uint64_t mask_u64b;
 	struct core_info *ci;
-	int ret;
 
 	ci = get_core_info();
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Looking for pcpu for %s\n",
-			pol->pkt.vm_name);
-	get_info_vm(pol->pkt.vm_name, &info);
-
-	for (count = 0; count < pol->pkt.num_vcpu; count++) {
-		mask_u64b = info.pcpu_mask[pol->pkt.vcpu_to_control[count]];
-		for (pcpu = 0; mask_u64b; mask_u64b &= ~(1ULL << pcpu++)) {
-			if ((mask_u64b >> pcpu) & 1) {
-				if (pol->pkt.policy_to_use == BRANCH_RATIO) {
-					ci->cd[pcpu].oob_enabled = 1;
-					ret = add_core_to_monitor(pcpu);
-					if (ret == 0)
-						printf("Monitoring pcpu %d via Branch Ratio\n",
-								pcpu);
-					else
-						printf("Failed to start OOB Monitoring pcpu %d\n",
-								pcpu);
-
-				} else {
-					pol->core_share[count].pcpu = pcpu;
-					printf("Monitoring pcpu %d\n", pcpu);
-				}
+	RTE_LOG(INFO, CHANNEL_MONITOR,
+			"Looking for pcpu for %s\n", pol->pkt.vm_name);
+
+	/*
+	 * So now that we're handling virtual and physical cores, we need to
+	 * differenciate between them when adding them to the branch monitor.
+	 * Virtual cores need to be converted to physical cores.
+	 */
+	if (pol->pkt.core_type == CORE_TYPE_VIRTUAL) {
+		/*
+		 * If the cores in the policy are virtual, we need to map them
+		 * to physical core. We look up the vm info and use that for
+		 * the mapping.
+		 */
+		get_info_vm(pol->pkt.vm_name, &info);
+		for (count = 0; count < pol->pkt.num_vcpu; count++) {
+			mask_u64b =
+				info.pcpu_mask[pol->pkt.vcpu_to_control[count]];
+			for (pcpu = 0; mask_u64b;
+					mask_u64b &= ~(1ULL << pcpu++)) {
+				if ((mask_u64b >> pcpu) & 1)
+					pcpu_monitor(pol, ci, pcpu, count);
 			}
 		}
+	} else {
+		/*
+		 * If the cores in the policy are physical, we just use
+		 * those core id's directly.
+		 */
+		for (count = 0; count < pol->pkt.num_vcpu; count++) {
+			pcpu = pol->pkt.vcpu_to_control[count];
+			pcpu_monitor(pol, ci, pcpu, count);
+		}
 	}
 }
 
@@ -160,8 +195,13 @@ update_policy(struct channel_packet *pkt)
 	unsigned int updated = 0;
 	int i;
 
+
+	RTE_LOG(INFO, CHANNEL_MONITOR,
+			"Applying policy for %s\n", pkt->vm_name);
+
 	for (i = 0; i < MAX_VMS; i++) {
 		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
+			/* Copy the contents of *pkt into the policy.pkt */
 			policies[i].pkt = *pkt;
 			get_pcpu_to_control(&policies[i]);
 			if (get_pfid(&policies[i]) == -1) {
@@ -189,6 +229,24 @@ update_policy(struct channel_packet *pkt)
 	return 0;
 }
 
+static int
+remove_policy(struct channel_packet *pkt __rte_unused)
+{
+	int i;
+
+	/*
+	 * Disabling the policy is simply a case of setting
+	 * enabled to 0
+	 */
+	for (i = 0; i < MAX_VMS; i++) {
+		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
+			policies[i].enabled = 0;
+			return 0;
+		}
+	}
+	return -1;
+}
+
 static uint64_t
 get_pkt_diff(struct policy *pol)
 {
@@ -346,7 +404,6 @@ apply_policy(struct policy *pol)
 		apply_workload_profile(pol);
 }
 
-
 static int
 process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 {
@@ -355,6 +412,8 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (chan_info == NULL)
 		return -1;
 
+	RTE_LOG(INFO, CHANNEL_MONITOR, "Processing Request %s\n", pkt->vm_name);
+
 	if (rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_CONNECTED,
 			CHANNEL_MGR_CHANNEL_PROCESSING) == 0)
 		return -1;
@@ -362,10 +421,12 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (pkt->command == CPU_POWER) {
 		core_mask = get_pcpus_mask(chan_info, pkt->resource_id);
 		if (core_mask == 0) {
-			RTE_LOG(ERR, CHANNEL_MONITOR, "Error get physical CPU mask for "
-				"channel '%s' using vCPU(%u)\n", chan_info->channel_path,
-				(unsigned)pkt->unit);
-			return -1;
+			/*
+			 * Core mask will be 0 in the case where
+			 * hypervisor is not available so we're working in
+			 * the host, so use the core as the mask.
+			 */
+			core_mask = 1ULL << pkt->resource_id;
 		}
 		if (__builtin_popcountll(core_mask) == 1) {
 
@@ -421,12 +482,20 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	}
 
 	if (pkt->command == PKT_POLICY) {
-		RTE_LOG(INFO, CHANNEL_MONITOR, "\nProcessing Policy request from Guest\n");
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				"\nProcessing Policy request\n");
 		update_policy(pkt);
 		policy_is_set = 1;
 	}
 
-	/* Return is not checked as channel status may have been set to DISABLED
+	if (pkt->command == PKT_POLICY_REMOVE) {
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				 "Removing policy %s\n", pkt->vm_name);
+		remove_policy(pkt);
+	}
+
+	/*
+	 * Return is not checked as channel status may have been set to DISABLED
 	 * from management thread
 	 */
 	rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_PROCESSING,
@@ -448,13 +517,16 @@ add_channel_to_monitor(struct channel_info **chan_info)
 				"to epoll\n", info->channel_path);
 		return -1;
 	}
+	RTE_LOG(ERR, CHANNEL_MONITOR, "Added channel '%s' "
+			"to monitor\n", info->channel_path);
 	return 0;
 }
 
 int
 remove_channel_from_monitor(struct channel_info *chan_info)
 {
-	if (epoll_ctl(global_event_fd, EPOLL_CTL_DEL, chan_info->fd, NULL) < 0) {
+	if (epoll_ctl(global_event_fd, EPOLL_CTL_DEL,
+			chan_info->fd, NULL) < 0) {
 		RTE_LOG(ERR, CHANNEL_MONITOR, "Unable to remove channel '%s' "
 				"from epoll\n", chan_info->channel_path);
 		return -1;
@@ -467,11 +539,13 @@ channel_monitor_init(void)
 {
 	global_event_fd = epoll_create1(0);
 	if (global_event_fd == 0) {
-		RTE_LOG(ERR, CHANNEL_MONITOR, "Error creating epoll context with "
-				"error %s\n", strerror(errno));
+		RTE_LOG(ERR, CHANNEL_MONITOR,
+				"Error creating epoll context with error %s\n",
+				strerror(errno));
 		return -1;
 	}
-	global_events_list = rte_malloc("epoll_events", sizeof(*global_events_list)
+	global_events_list = rte_malloc("epoll_events",
+			sizeof(*global_events_list)
 			* MAX_EVENTS, RTE_CACHE_LINE_SIZE);
 	if (global_events_list == NULL) {
 		RTE_LOG(ERR, CHANNEL_MONITOR, "Unable to rte_malloc for "
diff --git a/examples/vm_power_manager/main.c b/examples/vm_power_manager/main.c
index 58c5fa45c..893bf4cdd 100644
--- a/examples/vm_power_manager/main.c
+++ b/examples/vm_power_manager/main.c
@@ -421,6 +421,8 @@ main(int argc, char **argv)
 		return -1;
 	}
 
+	add_host_channel();
+
 	printf("Running core monitor on lcore id %d\n", lcore_id);
 	rte_eal_remote_launch(run_core_monitor, NULL, lcore_id);
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v6 06/10] examples/power: increase allowed number of clients
  2018-10-02  8:43         ` [dpdk-dev] [PATCH v6 0/10] add json power policy interface for containers David Hunt
                             ` (4 preceding siblings ...)
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 05/10] examples/power: add host channel to power manager David Hunt
@ 2018-10-02  8:43           ` David Hunt
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 07/10] examples/power: add json string handling David Hunt
                             ` (4 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-02  8:43 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Now that we're handling host policies, containers and virtual machines,
we'll rename MAX_VMS to MAX_CLIENTS, and increase from 4 to 64

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/channel_manager.h |  4 ++--
 examples/vm_power_manager/channel_monitor.c | 10 +++++-----
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.h b/examples/vm_power_manager/channel_manager.h
index e32235b07..d948b304c 100644
--- a/examples/vm_power_manager/channel_manager.h
+++ b/examples/vm_power_manager/channel_manager.h
@@ -37,7 +37,7 @@ struct sockaddr_un _sockaddr_un;
 #define UNIX_PATH_MAX sizeof(_sockaddr_un.sun_path)
 #endif
 
-#define MAX_VMS 4
+#define MAX_CLIENTS 64
 #define MAX_VCPUS 20
 
 
@@ -47,7 +47,7 @@ struct libvirt_vm_info {
 	uint8_t num_cpus;
 };
 
-struct libvirt_vm_info lvm_info[MAX_VMS];
+struct libvirt_vm_info lvm_info[MAX_CLIENTS];
 /* Communication Channel Status */
 enum channel_status { CHANNEL_MGR_CHANNEL_DISCONNECTED = 0,
 	CHANNEL_MGR_CHANNEL_CONNECTED,
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index c3c3d7bb1..53a4efe45 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -41,7 +41,7 @@ static volatile unsigned run_loop = 1;
 static int global_event_fd;
 static unsigned int policy_is_set;
 static struct epoll_event *global_events_list;
-static struct policy policies[MAX_VMS];
+static struct policy policies[MAX_CLIENTS];
 
 void channel_monitor_exit(void)
 {
@@ -199,7 +199,7 @@ update_policy(struct channel_packet *pkt)
 	RTE_LOG(INFO, CHANNEL_MONITOR,
 			"Applying policy for %s\n", pkt->vm_name);
 
-	for (i = 0; i < MAX_VMS; i++) {
+	for (i = 0; i < MAX_CLIENTS; i++) {
 		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
 			/* Copy the contents of *pkt into the policy.pkt */
 			policies[i].pkt = *pkt;
@@ -214,7 +214,7 @@ update_policy(struct channel_packet *pkt)
 		}
 	}
 	if (!updated) {
-		for (i = 0; i < MAX_VMS; i++) {
+		for (i = 0; i < MAX_CLIENTS; i++) {
 			if (policies[i].enabled == 0) {
 				policies[i].pkt = *pkt;
 				get_pcpu_to_control(&policies[i]);
@@ -238,7 +238,7 @@ remove_policy(struct channel_packet *pkt __rte_unused)
 	 * Disabling the policy is simply a case of setting
 	 * enabled to 0
 	 */
-	for (i = 0; i < MAX_VMS; i++) {
+	for (i = 0; i < MAX_CLIENTS; i++) {
 		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
 			policies[i].enabled = 0;
 			return 0;
@@ -609,7 +609,7 @@ run_channel_monitor(void)
 		if (policy_is_set) {
 			int j;
 
-			for (j = 0; j < MAX_VMS; j++) {
+			for (j = 0; j < MAX_CLIENTS; j++) {
 				if (policies[j].enabled == 1)
 					apply_policy(&policies[j]);
 			}
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v6 07/10] examples/power: add json string handling
  2018-10-02  8:43         ` [dpdk-dev] [PATCH v6 0/10] add json power policy interface for containers David Hunt
                             ` (5 preceding siblings ...)
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 06/10] examples/power: increase allowed number of clients David Hunt
@ 2018-10-02  8:43           ` David Hunt
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 08/10] examples/power: clean up verbose messages David Hunt
                             ` (3 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-02  8:43 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Add JSON string handling to vm_power_manager for JSON strings received
through the fifo. The format of the JSON strings are detailed in the
next patch, the vm_power_manager user guide documentation updates.

This patch introduces a new dependency on Jansson, a C library for
encoding, decoding and manipulating JSON data. To compile the sample app
you now need to have installed libjansson4 and libjansson-dev (these may
be named slightly differently depending on your Operating System)

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/Makefile          |   6 +
 examples/vm_power_manager/channel_monitor.c | 371 ++++++++++++++++++--
 2 files changed, 352 insertions(+), 25 deletions(-)

diff --git a/examples/vm_power_manager/Makefile b/examples/vm_power_manager/Makefile
index 13a5205ba..50147c05d 100644
--- a/examples/vm_power_manager/Makefile
+++ b/examples/vm_power_manager/Makefile
@@ -31,6 +31,12 @@ CFLAGS += $(WERROR_FLAGS)
 
 LDLIBS += -lvirt
 
+JANSSON := $(shell pkg-config --exists jansson; echo $$?)
+ifeq ($(JANSSON), 0)
+LDLIBS += $(shell pkg-config --libs jansson)
+CFLAGS += -DUSE_JANSSON
+endif
+
 ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
 
 ifeq ($(CONFIG_RTE_LIBRTE_IXGBE_PMD),y)
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index 53a4efe45..afb44a069 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -9,11 +9,18 @@
 #include <signal.h>
 #include <errno.h>
 #include <string.h>
+#include <fcntl.h>
 #include <sys/types.h>
 #include <sys/epoll.h>
 #include <sys/queue.h>
 #include <sys/time.h>
-
+#include <sys/socket.h>
+#include <sys/select.h>
+#ifdef USE_JANSSON
+#include <jansson.h>
+#else
+#pragma message "Jansson dev libs unavailable, not including JSON parsing"
+#endif
 #include <rte_log.h>
 #include <rte_memory.h>
 #include <rte_malloc.h>
@@ -35,6 +42,8 @@
 
 uint64_t vsi_pkt_count_prev[384];
 uint64_t rdtsc_prev[384];
+#define MAX_JSON_STRING_LEN 1024
+char json_data[MAX_JSON_STRING_LEN];
 
 double time_period_ms = 1;
 static volatile unsigned run_loop = 1;
@@ -43,6 +52,234 @@ static unsigned int policy_is_set;
 static struct epoll_event *global_events_list;
 static struct policy policies[MAX_CLIENTS];
 
+#ifdef USE_JANSSON
+
+union PFID {
+	struct ether_addr addr;
+	uint64_t pfid;
+};
+
+static int
+str_to_ether_addr(const char *a, struct ether_addr *ether_addr)
+{
+	int i;
+	char *end;
+	unsigned long o[ETHER_ADDR_LEN];
+
+	i = 0;
+	do {
+		errno = 0;
+		o[i] = strtoul(a, &end, 16);
+		if (errno != 0 || end == a || (end[0] != ':' && end[0] != 0))
+			return -1;
+		a = end + 1;
+	} while (++i != RTE_DIM(o) / sizeof(o[0]) && end[0] != 0);
+
+	/* Junk at the end of line */
+	if (end[0] != 0)
+		return -1;
+
+	/* Support the format XX:XX:XX:XX:XX:XX */
+	if (i == ETHER_ADDR_LEN) {
+		while (i-- != 0) {
+			if (o[i] > UINT8_MAX)
+				return -1;
+			ether_addr->addr_bytes[i] = (uint8_t)o[i];
+		}
+	/* Support the format XXXX:XXXX:XXXX */
+	} else if (i == ETHER_ADDR_LEN / 2) {
+		while (i-- != 0) {
+			if (o[i] > UINT16_MAX)
+				return -1;
+			ether_addr->addr_bytes[i * 2] =
+					(uint8_t)(o[i] >> 8);
+			ether_addr->addr_bytes[i * 2 + 1] =
+					(uint8_t)(o[i] & 0xff);
+		}
+	/* unknown format */
+	} else
+		return -1;
+
+	return 0;
+}
+
+static int
+set_policy_mac(struct channel_packet *pkt, int idx, char *mac)
+{
+	union PFID pfid;
+	int ret;
+
+	/* Use port MAC address as the vfid */
+	ret = str_to_ether_addr(mac, &pfid.addr);
+
+	if (ret != 0) {
+		RTE_LOG(ERR, CHANNEL_MONITOR,
+			"Invalid mac address received in JSON\n");
+		pkt->vfid[idx] = 0;
+		return -1;
+	}
+
+	printf("Received MAC Address: %02" PRIx8 ":%02" PRIx8 ":%02" PRIx8 ":"
+			"%02" PRIx8 ":%02" PRIx8 ":%02" PRIx8 "\n",
+			pfid.addr.addr_bytes[0], pfid.addr.addr_bytes[1],
+			pfid.addr.addr_bytes[2], pfid.addr.addr_bytes[3],
+			pfid.addr.addr_bytes[4], pfid.addr.addr_bytes[5]);
+
+	pkt->vfid[idx] = pfid.pfid;
+	return 0;
+}
+
+
+static int
+parse_json_to_pkt(json_t *element, struct channel_packet *pkt)
+{
+	const char *key;
+	json_t *value;
+	int ret;
+
+	memset(pkt, 0, sizeof(struct channel_packet));
+
+	pkt->nb_mac_to_monitor = 0;
+	pkt->t_boost_status.tbEnabled = false;
+	pkt->workload = LOW;
+	pkt->policy_to_use = TIME;
+	pkt->command = PKT_POLICY;
+	pkt->core_type = CORE_TYPE_PHYSICAL;
+
+	json_object_foreach(element, key, value) {
+		if (!strcmp(key, "policy")) {
+			/* Recurse in to get the contents of profile */
+			ret = parse_json_to_pkt(value, pkt);
+			if (ret)
+				return ret;
+		} else if (!strcmp(key, "instruction")) {
+			/* Recurse in to get the contents of instruction */
+			ret = parse_json_to_pkt(value, pkt);
+			if (ret)
+				return ret;
+		} else if (!strcmp(key, "name")) {
+			strcpy(pkt->vm_name, json_string_value(value));
+		} else if (!strcmp(key, "command")) {
+			char command[32];
+			snprintf(command, 32, "%s", json_string_value(value));
+			if (!strcmp(command, "power")) {
+				pkt->command = CPU_POWER;
+			} else if (!strcmp(command, "create")) {
+				pkt->command = PKT_POLICY;
+			} else if (!strcmp(command, "destroy")) {
+				pkt->command = PKT_POLICY_REMOVE;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Invalid command received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "policy_type")) {
+			char command[32];
+			snprintf(command, 32, "%s", json_string_value(value));
+			if (!strcmp(command, "TIME")) {
+				pkt->policy_to_use = TIME;
+			} else if (!strcmp(command, "TRAFFIC")) {
+				pkt->policy_to_use = TRAFFIC;
+			} else if (!strcmp(command, "WORKLOAD")) {
+				pkt->policy_to_use = WORKLOAD;
+			} else if (!strcmp(command, "BRANCH_RATIO")) {
+				pkt->policy_to_use = BRANCH_RATIO;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Wrong policy_type received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "workload")) {
+			char command[32];
+			snprintf(command, 32, "%s", json_string_value(value));
+			if (!strcmp(command, "HIGH")) {
+				pkt->workload = HIGH;
+			} else if (!strcmp(command, "MEDIUM")) {
+				pkt->workload = MEDIUM;
+			} else if (!strcmp(command, "LOW")) {
+				pkt->workload = LOW;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Wrong workload received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "busy_hours")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int hour = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->timer_policy.busy_hours[i] = hour;
+			}
+		} else if (!strcmp(key, "quiet_hours")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int hour = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->timer_policy.quiet_hours[i] = hour;
+			}
+		} else if (!strcmp(key, "core_list")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int core = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->vcpu_to_control[i] = core;
+			}
+			pkt->num_vcpu = size;
+		} else if (!strcmp(key, "mac_list")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				char mac[32];
+				snprintf(mac, 32, "%s", json_string_value(
+						json_array_get(value, i)));
+				set_policy_mac(pkt, i, mac);
+			}
+			pkt->nb_mac_to_monitor = size;
+		} else if (!strcmp(key, "avg_packet_thresh")) {
+			pkt->traffic_policy.avg_max_packet_thresh =
+					(uint32_t)json_integer_value(value);
+		} else if (!strcmp(key, "max_packet_thresh")) {
+			pkt->traffic_policy.max_max_packet_thresh =
+					(uint32_t)json_integer_value(value);
+		} else if (!strcmp(key, "unit")) {
+			char unit[32];
+			snprintf(unit, 32, "%s", json_string_value(value));
+			if (!strcmp(unit, "SCALE_UP")) {
+				pkt->unit = CPU_POWER_SCALE_UP;
+			} else if (!strcmp(unit, "SCALE_DOWN")) {
+				pkt->unit = CPU_POWER_SCALE_DOWN;
+			} else if (!strcmp(unit, "SCALE_MAX")) {
+				pkt->unit = CPU_POWER_SCALE_MAX;
+			} else if (!strcmp(unit, "SCALE_MIN")) {
+				pkt->unit = CPU_POWER_SCALE_MIN;
+			} else if (!strcmp(unit, "ENABLE_TURBO")) {
+				pkt->unit = CPU_POWER_ENABLE_TURBO;
+			} else if (!strcmp(unit, "DISABLE_TURBO")) {
+				pkt->unit = CPU_POWER_DISABLE_TURBO;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Invalid command received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "resource_id")) {
+			pkt->resource_id = (uint32_t)json_integer_value(value);
+		} else {
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+				"Unknown key received in JSON string: %s\n",
+				key);
+		}
+	}
+	return 0;
+}
+#endif
+
 void channel_monitor_exit(void)
 {
 	run_loop = 0;
@@ -555,6 +792,103 @@ channel_monitor_init(void)
 	return 0;
 }
 
+static void
+read_binary_packet(struct channel_info *chan_info)
+{
+	struct channel_packet pkt;
+	void *buffer = &pkt;
+	int buffer_len = sizeof(pkt);
+	int n_bytes, err = 0;
+
+	while (buffer_len > 0) {
+		n_bytes = read(chan_info->fd,
+				buffer, buffer_len);
+		if (n_bytes == buffer_len)
+			break;
+		if (n_bytes == -1) {
+			err = errno;
+			RTE_LOG(DEBUG, CHANNEL_MONITOR,
+				"Received error on "
+				"channel '%s' read: %s\n",
+				chan_info->channel_path,
+				strerror(err));
+			remove_channel(&chan_info);
+			break;
+		}
+		buffer = (char *)buffer + n_bytes;
+		buffer_len -= n_bytes;
+	}
+	if (!err)
+		process_request(&pkt, chan_info);
+}
+
+#ifdef USE_JANSSON
+static void
+read_json_packet(struct channel_info *chan_info)
+{
+	struct channel_packet pkt;
+	int n_bytes, ret;
+	json_t *root;
+	json_error_t error;
+
+	/* read opening brace to closing brace */
+	do {
+		int idx = 0;
+		int indent = 0;
+		do {
+			n_bytes = read(chan_info->fd, &json_data[idx], 1);
+			if (n_bytes == 0)
+				break;
+			if (json_data[idx] == '{')
+				indent++;
+			if (json_data[idx] == '}')
+				indent--;
+			if ((indent > 0) || (idx > 0))
+				idx++;
+			if (indent == 0)
+				json_data[idx] = 0;
+			if (idx >= MAX_JSON_STRING_LEN-1)
+				break;
+		} while (indent > 0);
+
+		if (indent > 0)
+			/*
+			 * We've broken out of the read loop without getting
+			 * a closing brace, so throw away the data
+			 */
+			json_data[idx] = 0;
+
+		if (strlen(json_data) == 0)
+			continue;
+
+		printf("got [%s]\n", json_data);
+
+		root = json_loads(json_data, 0, &error);
+
+		if (root) {
+			/*
+			 * Because our data is now in the json
+			 * object, we can overwrite the pkt
+			 * with a channel_packet struct, using
+			 * parse_json_to_pkt()
+			 */
+			ret = parse_json_to_pkt(root, &pkt);
+			json_decref(root);
+			if (ret) {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Error validating JSON profile data\n");
+				break;
+			}
+			process_request(&pkt, chan_info);
+		} else {
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+					"JSON error on line %d: %s\n",
+					error.line, error.text);
+		}
+	} while (n_bytes > 0);
+}
+#endif
+
 void
 run_channel_monitor(void)
 {
@@ -578,31 +912,18 @@ run_channel_monitor(void)
 			}
 			if (global_events_list[i].events & EPOLLIN) {
 
-				int n_bytes, err = 0;
-				struct channel_packet pkt;
-				void *buffer = &pkt;
-				int buffer_len = sizeof(pkt);
-
-				while (buffer_len > 0) {
-					n_bytes = read(chan_info->fd,
-							buffer, buffer_len);
-					if (n_bytes == buffer_len)
-						break;
-					if (n_bytes == -1) {
-						err = errno;
-						RTE_LOG(DEBUG, CHANNEL_MONITOR,
-							"Received error on "
-							"channel '%s' read: %s\n",
-							chan_info->channel_path,
-							strerror(err));
-						remove_channel(&chan_info);
-						break;
-					}
-					buffer = (char *)buffer + n_bytes;
-					buffer_len -= n_bytes;
+				switch (chan_info->type) {
+				case CHANNEL_TYPE_BINARY:
+					read_binary_packet(chan_info);
+					break;
+#ifdef USE_JANSSON
+				case CHANNEL_TYPE_JSON:
+					read_json_packet(chan_info);
+					break;
+#endif
+				default:
+					break;
 				}
-				if (!err)
-					process_request(&pkt, chan_info);
 			}
 		}
 		rte_delay_us(time_period_ms*1000);
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v6 08/10] examples/power: clean up verbose messages
  2018-10-02  8:43         ` [dpdk-dev] [PATCH v6 0/10] add json power policy interface for containers David Hunt
                             ` (6 preceding siblings ...)
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 07/10] examples/power: add json string handling David Hunt
@ 2018-10-02  8:43           ` David Hunt
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 09/10] examples/power: add meson/ninja build support David Hunt
                             ` (2 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-02  8:43 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Some messages appearing several times a second, removing as they are
unnecessary. Other less severe messages change from INFO to DEBUG

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/channel_monitor.c | 19 +++++--------------
 1 file changed, 5 insertions(+), 14 deletions(-)

diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index afb44a069..5da531542 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -361,7 +361,7 @@ get_pcpu_to_control(struct policy *pol)
 
 	ci = get_core_info();
 
-	RTE_LOG(INFO, CHANNEL_MONITOR,
+	RTE_LOG(DEBUG, CHANNEL_MONITOR,
 			"Looking for pcpu for %s\n", pol->pkt.vm_name);
 
 	/*
@@ -528,8 +528,6 @@ apply_traffic_profile(struct policy *pol)
 
 	diff = get_pkt_diff(pol);
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Applying traffic profile\n");
-
 	if (diff >= (pol->pkt.traffic_policy.max_max_packet_thresh)) {
 		for (count = 0; count < pol->pkt.num_vcpu; count++) {
 			if (pol->core_share[count].status != 1)
@@ -573,9 +571,6 @@ apply_time_profile(struct policy *pol)
 				if (pol->core_share[count].status != 1) {
 					power_manager_scale_core_max(
 						pol->core_share[count].pcpu);
-				RTE_LOG(INFO, CHANNEL_MONITOR,
-					"Scaling up core %d to max\n",
-					pol->core_share[count].pcpu);
 				}
 			}
 			break;
@@ -585,9 +580,6 @@ apply_time_profile(struct policy *pol)
 				if (pol->core_share[count].status != 1) {
 					power_manager_scale_core_min(
 						pol->core_share[count].pcpu);
-				RTE_LOG(INFO, CHANNEL_MONITOR,
-					"Scaling down core %d to min\n",
-					pol->core_share[count].pcpu);
 			}
 		}
 			break;
@@ -649,8 +641,6 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (chan_info == NULL)
 		return -1;
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Processing Request %s\n", pkt->vm_name);
-
 	if (rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_CONNECTED,
 			CHANNEL_MGR_CHANNEL_PROCESSING) == 0)
 		return -1;
@@ -719,8 +709,8 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	}
 
 	if (pkt->command == PKT_POLICY) {
-		RTE_LOG(INFO, CHANNEL_MONITOR,
-				"\nProcessing Policy request\n");
+		RTE_LOG(INFO, CHANNEL_MONITOR, "Processing policy request %s\n",
+				pkt->vm_name);
 		update_policy(pkt);
 		policy_is_set = 1;
 	}
@@ -904,7 +894,8 @@ run_channel_monitor(void)
 					global_events_list[i].data.ptr;
 			if ((global_events_list[i].events & EPOLLERR) ||
 				(global_events_list[i].events & EPOLLHUP)) {
-				RTE_LOG(DEBUG, CHANNEL_MONITOR, "Remote closed connection for "
+				RTE_LOG(INFO, CHANNEL_MONITOR,
+						"Remote closed connection for "
 						"channel '%s'\n",
 						chan_info->channel_path);
 				remove_channel(&chan_info);
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v6 09/10] examples/power: add meson/ninja build support
  2018-10-02  8:43         ` [dpdk-dev] [PATCH v6 0/10] add json power policy interface for containers David Hunt
                             ` (7 preceding siblings ...)
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 08/10] examples/power: clean up verbose messages David Hunt
@ 2018-10-02  8:43           ` David Hunt
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 10/10] doc/vm_power_manager: add JSON interface API info David Hunt
  2018-10-17 13:05           ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers David Hunt
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-02  8:43 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Add meson.build in vm_power_manager and the guest_cli subdirectory.
Building can be achieved by going to the build directory, and using

meson configure -Dexamples=vm_power_manager,vm_power_manager/guest_cli

Then, when ninja is invoked, it will build dpdk-vm_power_manger and
dpdk-guest_cli

Work still needs to be done on the meson build system to handles the case
where the target list of example apps is defined as 'all'. That will come
in a future patch.

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
---
 .../vm_power_manager/guest_cli/meson.build    | 21 +++++++++++
 examples/vm_power_manager/meson.build         | 37 ++++++++++++++++++-
 2 files changed, 56 insertions(+), 2 deletions(-)
 create mode 100644 examples/vm_power_manager/guest_cli/meson.build

diff --git a/examples/vm_power_manager/guest_cli/meson.build b/examples/vm_power_manager/guest_cli/meson.build
new file mode 100644
index 000000000..9e821ceb8
--- /dev/null
+++ b/examples/vm_power_manager/guest_cli/meson.build
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+# Setting the name here because the default name will conflict with the
+# vm_power_manager app because of the way the directories are parsed.
+name = 'guest_cli'
+
+deps += ['power']
+
+sources = files(
+	'main.c', 'parse.c', 'vm_power_cli_guest.c'
+)
+
+opt_dep = cc.find_library('virt', required : false)
+build = opt_dep.found()
+ext_deps += opt_dep
diff --git a/examples/vm_power_manager/meson.build b/examples/vm_power_manager/meson.build
index c370d7476..f98445bc6 100644
--- a/examples/vm_power_manager/meson.build
+++ b/examples/vm_power_manager/meson.build
@@ -6,5 +6,38 @@
 # To build this example as a standalone application with an already-installed
 # DPDK instance, use 'make'
 
-# Example app currently unsupported by meson build
-build = false
+if dpdk_conf.has('RTE_LIBRTE_BNXT_PMD')
+	deps += ['pmd_bnxt']
+endif
+
+if dpdk_conf.has('RTE_LIBRTE_I40E_PMD')
+	deps += ['pmd_i40e']
+endif
+
+if dpdk_conf.has('RTE_LIBRTE_IXGBE_PMD')
+	deps += ['pmd_ixgbe']
+endif
+
+deps += ['power']
+
+
+sources = files(
+	'channel_manager.c', 'channel_monitor.c', 'main.c', 'parse.c', 'power_manager.c', 'vm_power_cli.c'
+)
+
+# If we're on X86, pull in the x86 code for the branch monitor algo.
+if dpdk_conf.has('RTE_ARCH_X86_64')
+	sources += files('oob_monitor_x86.c')
+else
+	sources += files('oob_monitor_nop.c')
+endif
+
+opt_dep = cc.find_library('virt', required : false)
+build = opt_dep.found()
+ext_deps += opt_dep
+
+opt_dep = dependency('jansson', required : false)
+if opt_dep.found()
+	ext_deps += opt_dep
+	cflags += '-DUSE_JANSSON'
+endif
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v6 10/10] doc/vm_power_manager: add JSON interface API info
  2018-10-02  8:43         ` [dpdk-dev] [PATCH v6 0/10] add json power policy interface for containers David Hunt
                             ` (8 preceding siblings ...)
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 09/10] examples/power: add meson/ninja build support David Hunt
@ 2018-10-02  8:43           ` David Hunt
  2018-10-17 13:05           ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers David Hunt
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-02  8:43 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Also added meson/ninja build info

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Marko Kovacevic <marko.kovacevic@intel.com>
---
 .../sample_app_ug/vm_power_management.rst     | 300 +++++++++++++++++-
 1 file changed, 298 insertions(+), 2 deletions(-)

diff --git a/doc/guides/sample_app_ug/vm_power_management.rst b/doc/guides/sample_app_ug/vm_power_management.rst
index 855570d6b..1ad4f1490 100644
--- a/doc/guides/sample_app_ug/vm_power_management.rst
+++ b/doc/guides/sample_app_ug/vm_power_management.rst
@@ -199,7 +199,7 @@ see :doc:`compiling`.
 
 The application is located in the ``vm_power_manager`` sub-directory.
 
-To build just the ``vm_power_manager`` application:
+To build just the ``vm_power_manager`` application using ``make``:
 
 .. code-block:: console
 
@@ -208,6 +208,22 @@ To build just the ``vm_power_manager`` application:
   cd ${RTE_SDK}/examples/vm_power_manager/
   make
 
+The resulting binary will be ${RTE_SDK}/build/examples/vm_power_manager
+
+To build just the ``vm_power_manager`` application using ``meson/ninja``:
+
+.. code-block:: console
+
+  export RTE_SDK=/path/to/rte_sdk
+  cd ${RTE_SDK}
+  meson build
+  cd build
+  ninja
+  meson configure -Dexamples=vm_power_manager
+  ninja
+
+The resulting binary will be ${RTE_SDK}/build/examples/dpdk-vm_power_manager
+
 Running
 ~~~~~~~
 
@@ -337,6 +353,270 @@ monitoring of branch ratio on cores doing busy polling via PMDs.
   and will need to be adjusted for different workloads.
 
 
+
+JSON API
+~~~~~~~~
+
+In addition to the command line interface for host command and a virtio-serial
+interface for VM power policies, there is also a JSON interface through which
+power commands and policies can be sent. This functionality adds a dependency
+on the Jansson library, and the Jansson development package must be installed
+on the system before the JSON parsing functionality is included in the app.
+This is achieved by:
+
+  .. code-block:: javascript
+
+    apt-get install libjansson-dev
+
+The command and package name may be different depending on your operating
+system. It's worth noting that the app will successfully build without this
+package present, but a warning is shown during compilation, and the JSON
+parsing functionality will not be present in the app.
+
+Sending a command or policy to the power manager application is achieved by
+simply opening a fifo file, writing a JSON string to that fifo, and closing
+the file.
+
+The fifo is at /tmp/powermonitor/fifo
+
+The jason string can be a policy or instruction, and takes the following
+format:
+
+  .. code-block:: javascript
+
+    {"packet_type": {
+      "pair_1": value,
+      "pair_2": value
+    }}
+
+The 'packet_type' header can contain one of two values, depending on
+whether a policy or power command is being sent. The two possible values are
+"policy" and "instruction", and the expected name-value pairs is different
+depending on which type is being sent.
+
+The pairs are the format of standard JSON name-value pairs. The value type
+varies between the different name/value pairs, and may be integers, strings,
+arrays, etc. Examples of policies follow later in this document. The allowed
+names and value types are as follows:
+
+
+:Pair Name: "name"
+:Description: Name of the VM or Host. Allows the parser to associate the
+  policy with the relevant VM or Host OS.
+:Type: string
+:Values: any valid string
+:Required: yes
+:Example:
+
+    .. code-block:: javascript
+
+      "name", "ubuntu2"
+
+
+:Pair Name: "command"
+:Description: The type of packet we're sending to the power manager. We can be
+  creating or destroying a policy, or sending a direct command to adjust
+  the frequency of a core, similar to the command line interface.
+:Type: string
+:Values:
+
+  :CREATE: used when creating a new policy,
+  :DESTROY: used when removing a policy,
+  :POWER: used when sending an immediate command, max, min, etc.
+:Required: yes
+:Example:
+
+    .. code-block:: javascript
+
+      "command", "CREATE"
+
+
+:Pair Name: "policy_type"
+:Description: Type of policy to apply. Please see vm_power_manager documentation
+  for more information on the types of policies that may be used.
+:Type: string
+:Values:
+
+  :TIME: Time-of-day policy. Frequencies of the relevant cores are
+    scaled up/down depending on busy and quiet hours.
+  :TRAFFIC: This policy takes statistics from the NIC and scales up
+    and down accordingly.
+  :WORKLOAD: This policy looks at how heavily loaded the cores are,
+    and scales up and down accordingly.
+  :BRANCH_RATIO: This out-of-band policy can look at the ratio between
+    branch hits and misses on a core, and is useful for detecting
+    how much packet processing a core is doing.
+:Required: only for CREATE/DESTROY command
+:Example:
+
+  .. code-block:: javascript
+
+    "policy_type", "TIME"
+
+:Pair Name: "busy_hours"
+:Description: The hours of the day in which we scale up the cores for busy
+  times.
+:Type: array of integers
+:Values: array with list of hour numbers, (0-23)
+:Required: only for TIME policy
+:Example:
+
+  .. code-block:: javascript
+
+    "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ]
+
+:Pair Name: "quiet_hours"
+:Description: The hours of the day in which we scale down the cores for quiet
+  times.
+:Type: array of integers
+:Values: array with list of hour numbers, (0-23)
+:Required: only for TIME policy
+:Example:
+
+  .. code-block:: javascript
+
+    "quiet_hours":[ 2, 3, 4, 5, 6 ]
+
+:Pair Name: "avg_packet_thresh"
+:Description: Threshold below which the frequency will be set to min for
+  the TRAFFIC policy. If the traffic rate is above this and below max, the
+  frequency will be set to medium.
+:Type: integer
+:Values: The number of packets below which the TRAFFIC policy applies the
+  minimum frequency, or medium frequency if between avg and max thresholds.
+:Required: only for TRAFFIC policy
+:Example:
+
+  .. code-block:: javascript
+
+    "avg_packet_thresh": 100000
+
+:Pair Name: "max_packet_thresh"
+:Description: Threshold above which the frequency will be set to max for
+  the TRAFFIC policy
+:Type: integer
+:Values: The number of packets per interval above which the TRAFFIC policy
+  applies the maximum frequency
+:Required: only for TRAFFIC policy
+:Example:
+
+  .. code-block:: javascript
+
+    "max_packet_thresh": 500000
+
+:Pair Name: "core_list"
+:Description: The cores to which to apply the policy.
+:Type: array of integers
+:Values: array with list of virtual CPUs.
+:Required: only policy CREATE/DESTROY
+:Example:
+
+  .. code-block:: javascript
+
+    "core_list":[ 10, 11 ]
+
+:Pair Name: "workload"
+:Description: When our policy is of type WORKLOAD, we need to specify how
+  heavy our workload is.
+:Type: string
+:Values:
+
+  :HIGH: For cores running workloads that require high frequencies
+  :MEDIUM: For cores running workloads that require medium frequencies
+  :LOW: For cores running workloads that require low frequencies
+:Required: only for WORKLOAD policy types
+:Example:
+
+  .. code-block:: javascript
+
+    "workload", "MEDIUM"
+
+:Pair Name: "mac_list"
+:Description: When our policy is of type TRAFFIC, we need to specify the
+  MAC addresses that the host needs to monitor
+:Type: string
+:Values: array with a list of mac address strings.
+:Required: only for TRAFFIC policy types
+:Example:
+
+  .. code-block:: javascript
+
+    "mac_list":[ "de:ad:be:ef:01:01", "de:ad:be:ef:01:02" ]
+
+:Pair Name: "unit"
+:Description: the type of power operation to apply in the command
+:Type: string
+:Values:
+
+  :SCALE_MAX: Scale frequency of this core to maximum
+  :SCALE_MIN: Scale frequency of this core to minimum
+  :SCALE_UP: Scale up frequency of this core
+  :SCALE_DOWN: Scale down frequency of this core
+  :ENABLE_TURBO: Enable Turbo Boost for this core
+  :DISABLE_TURBO: Disable Turbo Boost for this core
+:Required: only for POWER instruction
+:Example:
+
+  .. code-block:: javascript
+
+    "unit", "SCALE_MAX"
+
+:Pair Name: "resource_id"
+:Description: The core to which to apply the power command.
+:Type: integer
+:Values: valid core id for VM or host OS.
+:Required: only POWER instruction
+:Example:
+
+  .. code-block:: javascript
+
+    "resource_id": 10
+
+JSON API Examples
+~~~~~~~~~~~~~~~~~
+
+Profile create example:
+
+  .. code-block:: javascript
+
+    {"policy": {
+      "name": "ubuntu",
+      "command": "create",
+      "policy_type": "TIME",
+      "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
+      "quiet_hours":[ 2, 3, 4, 5, 6 ],
+      "core_list":[ 11 ]
+    }}
+
+Profile destroy example:
+
+  .. code-block:: javascript
+
+    {"profile": {
+      "name": "ubuntu",
+      "command": "destroy",
+    }}
+
+Power command example:
+
+  .. code-block:: javascript
+
+    {"command": {
+      "name": "ubuntu",
+      "unit": "SCALE_MAX",
+      "resource_id": 10
+    }}
+
+To send a JSON string to the Power Manager application, simply paste the
+example JSON string into a text file and cat it into the fifo:
+
+  .. code-block:: console
+
+    cat file.json >/tmp/powermonitor/fifo
+
+The console of the Power Manager application should indicate the command that
+was just received via the fifo.
+
 Compiling and Running the Guest Applications
 --------------------------------------------
 
@@ -366,7 +646,7 @@ For compiling and running l3fwd-power, see :doc:`l3_forward_power_man`.
 
 The application is located in the ``guest_cli`` sub-directory under ``vm_power_manager``.
 
-To build just the ``guest_vm_power_manager`` application:
+To build just the ``guest_vm_power_manager`` application using ``make``:
 
 .. code-block:: console
 
@@ -375,6 +655,22 @@ To build just the ``guest_vm_power_manager`` application:
   cd ${RTE_SDK}/examples/vm_power_manager/guest_cli/
   make
 
+The resulting binary will be ${RTE_SDK}/build/examples/guest_cli
+
+To build just the ``vm_power_manager`` application using ``meson/ninja``:
+
+.. code-block:: console
+
+  export RTE_SDK=/path/to/rte_sdk
+  cd ${RTE_SDK}
+  meson build
+  cd build
+  ninja
+  meson configure -Dexamples=vm_power_manager/guest_cli
+  ninja
+
+The resulting binary will be ${RTE_SDK}/build/examples/guest_cli
+
 Running
 ~~~~~~~
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers
  2018-10-02  8:43         ` [dpdk-dev] [PATCH v6 0/10] add json power policy interface for containers David Hunt
                             ` (9 preceding siblings ...)
  2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 10/10] doc/vm_power_manager: add JSON interface API info David Hunt
@ 2018-10-17 13:05           ` David Hunt
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 01/10] examples/power: add checks around hypervisor David Hunt
                               ` (10 more replies)
  10 siblings, 11 replies; 110+ messages in thread
From: David Hunt @ 2018-10-17 13:05 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov

The current vm_power_manager example app has the capability to accept power
policies from virtual machines via virtio-serial channels. These power
policies  allow a virtual machine to give information to the power manager
to allow the power manager take care of the power management of the virtual
machine based on the information in the policy.

This power policy functionality is limited to virtual machines sending
the policies to the power manager (which runs in the Host OS), and a solution
was needed for additional methods of sending power policies to the power
manager app.

The main use-case for this modification is for containers and host
applications that wish to send polices to the power manager.

This patchset adds the capability to send power polices and power commands
to the vm_power_manager app via JSON strings through a fifo on the file
system.
For example, given the following file, policy.json:

{"policy": {
  "name": "ubuntu2",
  "command": "create",
  "policy_type": "TIME",
  "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
  "quiet_hours":[ 2, 3, 4, 5, 6 ],
  "core_list":[ 11, 12, 13 ]
}}

Then running the command:

cat policy.json >/tmp/powermonitor/fifo

The policy is sent to the vm_power_manager. The power manager app then parses
the JSON data, and inserts the policy into the array of policies.

Part of the patch series contains documentation updates to give all the
details of the valid name-value pairs, the data types, etc.

Patch v2:
  * Fixed review comments from Stephen Hemminger and Lei A Yao.
  * Added a check in the Makefile for libjansson-dev. Will Warn user and build
    without JSON functionality if not present, will build including JSON
    functionality if it is present.

Patch v3:
  * Added meson/ninja support for vm_power_manager and guest_cli apps
  * Fixed compilation issue with guest_cli app

Patch v4:
  * Split out some unrelated changes to separate patches in the set
  * Some changes out of review by Anatoly (Thanks!)

Patch v5:
  * Removed the directory with JSON examples, as they already exist in
    the documentation.
  * Fixed some typos and formatting issues in the documentation.
  * Changed the JSON examples in the documentation to 'javascript' causing
    the syntax to be highlighted nicely.
  * Inherited the Acks from previous version.

Patch v6:
  * Added ability to set WORKLOAD policy to LOW, MEDIUM, or HIGH.
    "workload": "MEDIUM"
  * Added missing functionality to allow passing of a list of mac
    addresses for the TRAFFIC profile type.
    "mac_list":[ "de:ad:be:ef:01:01", "de:ad:be:ef:01:02" ]
  * Updated docs to include both of the above additions.

Patch v7:
  * Added release note update to the doc patch.

[01/10] examples/power: add checks around hypervisor
[02/10] examples/power: allow for number of vms to be zero
[03/10] lib/power: add changes for host commands/policies
[04/10] examples/power: add necessary changes to guest app
[05/10] examples/power: add host channel to power manager
[06/10] examples/power: increase allowed number of clients
[07/10] examples/power: add json string handling
[08/10] examples/power: clean up verbose messages
[09/10] examples/power: add meson/ninja build support
[10/10] doc/vm_power_manager: add JSON interface API info

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v7 01/10] examples/power: add checks around hypervisor
  2018-10-17 13:05           ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers David Hunt
@ 2018-10-17 13:05             ` David Hunt
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 02/10] examples/power: allow for number of vms to be zero David Hunt
                               ` (9 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-17 13:05 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Allow vm_power_manager to run without requiring qemu to be present
on the machine. This will be required for instances where the JSON
interface is used for commands and polices, without any VMs present.
A use case for this is a container enviromnent.

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/channel_manager.c | 71 +++++++++++++--------
 1 file changed, 43 insertions(+), 28 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.c b/examples/vm_power_manager/channel_manager.c
index 927fc35ab..2e471d0c1 100644
--- a/examples/vm_power_manager/channel_manager.c
+++ b/examples/vm_power_manager/channel_manager.c
@@ -43,7 +43,8 @@ static unsigned char *global_cpumaps;
 static virVcpuInfo *global_vircpuinfo;
 static size_t global_maplen;
 
-static unsigned global_n_host_cpus;
+static unsigned int global_n_host_cpus;
+static bool global_hypervisor_available;
 
 /*
  * Represents a single Virtual Machine
@@ -198,7 +199,11 @@ get_pcpus_mask(struct channel_info *chan_info, unsigned vcpu)
 {
 	struct virtual_machine_info *vm_info =
 			(struct virtual_machine_info *)chan_info->priv_info;
-	return rte_atomic64_read(&vm_info->pcpu_mask[vcpu]);
+
+	if (global_hypervisor_available && (vm_info != NULL))
+		return rte_atomic64_read(&vm_info->pcpu_mask[vcpu]);
+	else
+		return 0;
 }
 
 static inline int
@@ -559,6 +564,8 @@ get_all_vm(int *num_vm, int *num_vcpu)
 				VIR_CONNECT_LIST_DOMAINS_PERSISTENT;
 	unsigned int domain_flag = VIR_DOMAIN_VCPU_CONFIG;
 
+	if (!global_hypervisor_available)
+		return;
 
 	memset(global_cpumaps, 0, CHANNEL_CMDS_MAX_CPUS*global_maplen);
 	if (virNodeGetInfo(global_vir_conn_ptr, &node_info)) {
@@ -768,38 +775,42 @@ connect_hypervisor(const char *path)
 	}
 	return 0;
 }
-
 int
-channel_manager_init(const char *path)
+channel_manager_init(const char *path __rte_unused)
 {
 	virNodeInfo info;
 
 	LIST_INIT(&vm_list_head);
 	if (connect_hypervisor(path) < 0) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to initialize channel manager\n");
-		return -1;
-	}
-
-	global_maplen = VIR_CPU_MAPLEN(CHANNEL_CMDS_MAX_CPUS);
+		global_n_host_cpus = 64;
+		global_hypervisor_available = 0;
+		RTE_LOG(INFO, CHANNEL_MANAGER, "Unable to initialize channel manager\n");
+	} else {
+		global_hypervisor_available = 1;
+
+		global_maplen = VIR_CPU_MAPLEN(CHANNEL_CMDS_MAX_CPUS);
+
+		global_vircpuinfo = rte_zmalloc(NULL,
+				sizeof(*global_vircpuinfo) *
+				CHANNEL_CMDS_MAX_CPUS, RTE_CACHE_LINE_SIZE);
+		if (global_vircpuinfo == NULL) {
+			RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for CPU Info\n");
+			goto error;
+		}
+		global_cpumaps = rte_zmalloc(NULL,
+				CHANNEL_CMDS_MAX_CPUS * global_maplen,
+				RTE_CACHE_LINE_SIZE);
+		if (global_cpumaps == NULL)
+			goto error;
 
-	global_vircpuinfo = rte_zmalloc(NULL, sizeof(*global_vircpuinfo) *
-			CHANNEL_CMDS_MAX_CPUS, RTE_CACHE_LINE_SIZE);
-	if (global_vircpuinfo == NULL) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for CPU Info\n");
-		goto error;
-	}
-	global_cpumaps = rte_zmalloc(NULL, CHANNEL_CMDS_MAX_CPUS * global_maplen,
-			RTE_CACHE_LINE_SIZE);
-	if (global_cpumaps == NULL) {
-		goto error;
+		if (virNodeGetInfo(global_vir_conn_ptr, &info)) {
+			RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to retrieve node Info\n");
+			goto error;
+		}
+		global_n_host_cpus = (unsigned int)info.cpus;
 	}
 
-	if (virNodeGetInfo(global_vir_conn_ptr, &info)) {
-		RTE_LOG(ERR, CHANNEL_MANAGER, "Unable to retrieve node Info\n");
-		goto error;
-	}
 
-	global_n_host_cpus = (unsigned)info.cpus;
 
 	if (global_n_host_cpus > CHANNEL_CMDS_MAX_CPUS) {
 		RTE_LOG(WARNING, CHANNEL_MANAGER, "The number of host CPUs(%u) exceeds the "
@@ -811,7 +822,8 @@ channel_manager_init(const char *path)
 
 	return 0;
 error:
-	disconnect_hypervisor();
+	if (global_hypervisor_available)
+		disconnect_hypervisor();
 	return -1;
 }
 
@@ -838,7 +850,10 @@ channel_manager_exit(void)
 		rte_free(vm_info);
 	}
 
-	rte_free(global_cpumaps);
-	rte_free(global_vircpuinfo);
-	disconnect_hypervisor();
+	if (global_hypervisor_available) {
+		/* Only needed if hypervisor available */
+		rte_free(global_cpumaps);
+		rte_free(global_vircpuinfo);
+		disconnect_hypervisor();
+	}
 }
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v7 02/10] examples/power: allow for number of vms to be zero
  2018-10-17 13:05           ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers David Hunt
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 01/10] examples/power: add checks around hypervisor David Hunt
@ 2018-10-17 13:05             ` David Hunt
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 03/10] lib/power: add changes for host commands/policies David Hunt
                               ` (8 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-17 13:05 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Previously the vm_power_manager app required to have some vms defined, so
the call to get_all_vm() always set the noVms variable. Now we're accepting
policies from the host OS (without any VMs defined), so it is now valid to
have zero VMs. This patch initialises the relevant variables to zero just
in case the call to get_all_vms() does not find any, so could return with
the variables uninitialised.

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/channel_monitor.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index 7fa47ba97..f180d74e6 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -66,7 +66,7 @@ static void
 core_share_status(int pNo)
 {
 
-	int noVms, noVcpus, z, x, t;
+	int noVms = 0, noVcpus = 0, z, x, t;
 
 	get_all_vm(&noVms, &noVcpus);
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v7 03/10] lib/power: add changes for host commands/policies
  2018-10-17 13:05           ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers David Hunt
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 01/10] examples/power: add checks around hypervisor David Hunt
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 02/10] examples/power: allow for number of vms to be zero David Hunt
@ 2018-10-17 13:05             ` David Hunt
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 04/10] examples/power: add necessary changes to guest app David Hunt
                               ` (7 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-17 13:05 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

This patch does a couple of things:
  * Adds a new message type for removing policies (PKT_POLICY_REMOVE)
    Used when we want to remove a previously created policy.
  * Adds a core_type bool to the channel packet struct to specify whether
    the type of core we want to control is cirtual or physical.

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 lib/librte_power/channel_commands.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/lib/librte_power/channel_commands.h b/lib/librte_power/channel_commands.h
index ee638eefa..e7b93a797 100644
--- a/lib/librte_power/channel_commands.h
+++ b/lib/librte_power/channel_commands.h
@@ -19,6 +19,7 @@ extern "C" {
 #define CPU_POWER               1
 #define CPU_POWER_CONNECT       2
 #define PKT_POLICY              3
+#define PKT_POLICY_REMOVE       4
 
 /* CPU Power Command Scaling */
 #define CPU_POWER_SCALE_UP      1
@@ -58,6 +59,9 @@ struct traffic {
 	uint32_t max_max_packet_thresh;
 };
 
+#define CORE_TYPE_VIRTUAL 0
+#define CORE_TYPE_PHYSICAL 1
+
 struct channel_packet {
 	uint64_t resource_id; /**< core_num, device */
 	uint32_t unit;        /**< scale down/up/min/max */
@@ -70,6 +74,7 @@ struct channel_packet {
 	uint8_t vcpu_to_control[MAX_VCPU_PER_VM];
 	uint8_t num_vcpu;
 	struct timer_profile timer_policy;
+	bool core_type;
 	enum workload workload;
 	enum policy_to_use policy_to_use;
 	struct t_boost_status t_boost_status;
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v7 04/10] examples/power: add necessary changes to guest app
  2018-10-17 13:05           ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers David Hunt
                               ` (2 preceding siblings ...)
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 03/10] lib/power: add changes for host commands/policies David Hunt
@ 2018-10-17 13:05             ` David Hunt
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 05/10] examples/power: add host channel to power manager David Hunt
                               ` (6 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-17 13:05 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

The changes here are minimal, as the guest app functionality is not
changing at all, but there is a new element in the channel_packet
struct that needs to have a default set (channel_packet->core_type).

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/guest_cli/vm_power_cli_guest.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
index 0db1b804f..2d9e7689a 100644
--- a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
+++ b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
@@ -92,6 +92,7 @@ set_policy_defaults(struct channel_packet *pkt)
 	pkt->timer_policy.hours_to_use_traffic_profile[0] = 8;
 	pkt->timer_policy.hours_to_use_traffic_profile[1] = 10;
 
+	pkt->core_type = CORE_TYPE_VIRTUAL;
 	pkt->workload = LOW;
 	pkt->policy_to_use = TIME;
 	pkt->command = PKT_POLICY;
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v7 05/10] examples/power: add host channel to power manager
  2018-10-17 13:05           ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers David Hunt
                               ` (3 preceding siblings ...)
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 04/10] examples/power: add necessary changes to guest app David Hunt
@ 2018-10-17 13:05             ` David Hunt
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 06/10] examples/power: increase allowed number of clients David Hunt
                               ` (5 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-17 13:05 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

This patch adds a fifo channel to the vm_power_manager app through which
we can send commands and polices. Intended for sending JSON strings.
The fifo is at /tmp/powermonitor/fifo

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/channel_manager.c | 109 +++++++++++++++
 examples/vm_power_manager/channel_manager.h |  17 +++
 examples/vm_power_manager/channel_monitor.c | 142 +++++++++++++++-----
 examples/vm_power_manager/main.c            |   2 +
 4 files changed, 236 insertions(+), 34 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.c b/examples/vm_power_manager/channel_manager.c
index 2e471d0c1..4fac099df 100644
--- a/examples/vm_power_manager/channel_manager.c
+++ b/examples/vm_power_manager/channel_manager.c
@@ -13,6 +13,7 @@
 
 #include <sys/queue.h>
 #include <sys/types.h>
+#include <sys/stat.h>
 #include <sys/socket.h>
 #include <sys/select.h>
 
@@ -284,6 +285,38 @@ open_non_blocking_channel(struct channel_info *info)
 	return 0;
 }
 
+static int
+open_host_channel(struct channel_info *info)
+{
+	int flags;
+
+	info->fd = open(info->channel_path, O_RDWR | O_RSYNC);
+	if (info->fd == -1) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Error(%s) opening fifo for '%s'\n",
+				strerror(errno),
+				info->channel_path);
+		return -1;
+	}
+
+	/* Get current flags */
+	flags = fcntl(info->fd, F_GETFL, 0);
+	if (flags < 0) {
+		RTE_LOG(WARNING, CHANNEL_MANAGER, "Error(%s) fcntl get flags socket for"
+				"'%s'\n", strerror(errno), info->channel_path);
+		return 1;
+	}
+	/* Set to Non Blocking */
+	flags |= O_NONBLOCK;
+	if (fcntl(info->fd, F_SETFL, flags) < 0) {
+		RTE_LOG(WARNING, CHANNEL_MANAGER,
+				"Error(%s) setting non-blocking "
+				"socket for '%s'\n",
+				strerror(errno), info->channel_path);
+		return -1;
+	}
+	return 0;
+}
+
 static int
 setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 		struct channel_info **chan_info_dptr, unsigned channel_num)
@@ -294,6 +327,7 @@ setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 	chan_info->channel_num = channel_num;
 	chan_info->priv_info = (void *)vm_info;
 	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
+	chan_info->type = CHANNEL_TYPE_BINARY;
 	if (open_non_blocking_channel(chan_info) < 0) {
 		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open channel: "
 				"'%s' for VM '%s'\n",
@@ -316,6 +350,42 @@ setup_channel_info(struct virtual_machine_info **vm_info_dptr,
 	return 0;
 }
 
+static void
+fifo_path(char *dst, unsigned int len)
+{
+	snprintf(dst, len, "%sfifo", CHANNEL_MGR_SOCKET_PATH);
+}
+
+static int
+setup_host_channel_info(struct channel_info **chan_info_dptr,
+		unsigned int channel_num)
+{
+	struct channel_info *chan_info = *chan_info_dptr;
+
+	chan_info->channel_num = channel_num;
+	chan_info->priv_info = (void *)NULL;
+	chan_info->status = CHANNEL_MGR_CHANNEL_DISCONNECTED;
+	chan_info->type = CHANNEL_TYPE_JSON;
+
+	fifo_path(chan_info->channel_path, sizeof(chan_info->channel_path));
+
+	if (open_host_channel(chan_info) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Could not open host channel: "
+				"'%s'\n",
+				chan_info->channel_path);
+		return -1;
+	}
+	if (add_channel_to_monitor(&chan_info) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Could add channel: "
+				"'%s' to epoll ctl\n",
+				chan_info->channel_path);
+		return -1;
+
+	}
+	chan_info->status = CHANNEL_MGR_CHANNEL_CONNECTED;
+	return 0;
+}
+
 int
 add_all_channels(const char *vm_name)
 {
@@ -470,6 +540,45 @@ add_channels(const char *vm_name, unsigned *channel_list,
 	return num_channels_enabled;
 }
 
+int
+add_host_channel(void)
+{
+	struct channel_info *chan_info;
+	char socket_path[PATH_MAX];
+	int num_channels_enabled = 0;
+	int ret;
+
+	fifo_path(socket_path, sizeof(socket_path));
+
+	ret = mkfifo(socket_path, 0660);
+	if ((errno != EEXIST) && (ret < 0)) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Cannot create fifo '%s' error: "
+				"%s\n", socket_path, strerror(errno));
+		return 0;
+	}
+
+	if (access(socket_path, F_OK) < 0) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Channel path '%s' error: "
+				"%s\n", socket_path, strerror(errno));
+		return 0;
+	}
+	chan_info = rte_malloc(NULL, sizeof(*chan_info), 0);
+	if (chan_info == NULL) {
+		RTE_LOG(ERR, CHANNEL_MANAGER, "Error allocating memory for "
+				"channel '%s'\n", socket_path);
+		return 0;
+	}
+	snprintf(chan_info->channel_path,
+			sizeof(chan_info->channel_path), "%s", socket_path);
+	if (setup_host_channel_info(&chan_info, 0) < 0) {
+		rte_free(chan_info);
+		return 0;
+	}
+	num_channels_enabled++;
+
+	return num_channels_enabled;
+}
+
 int
 remove_channel(struct channel_info **chan_info_dptr)
 {
diff --git a/examples/vm_power_manager/channel_manager.h b/examples/vm_power_manager/channel_manager.h
index 872ec6140..e32235b07 100644
--- a/examples/vm_power_manager/channel_manager.h
+++ b/examples/vm_power_manager/channel_manager.h
@@ -54,6 +54,13 @@ enum channel_status { CHANNEL_MGR_CHANNEL_DISCONNECTED = 0,
 	CHANNEL_MGR_CHANNEL_DISABLED,
 	CHANNEL_MGR_CHANNEL_PROCESSING};
 
+/* Communication Channel Type */
+enum channel_type {
+	CHANNEL_TYPE_BINARY = 0,
+	CHANNEL_TYPE_INI,
+	CHANNEL_TYPE_JSON
+};
+
 /* VM libvirt(qemu/KVM) connection status */
 enum vm_status { CHANNEL_MGR_VM_INACTIVE = 0, CHANNEL_MGR_VM_ACTIVE};
 
@@ -66,6 +73,7 @@ struct channel_info {
 	volatile uint32_t status;    /**< Connection status(enum channel_status) */
 	int fd;                      /**< AF_UNIX socket fd */
 	unsigned channel_num;        /**< CHANNEL_MGR_SOCKET_PATH/<vm_name>.channel_num */
+	enum channel_type type;      /**< Binary, ini, json, etc. */
 	void *priv_info;             /**< Pointer to private info, do not modify */
 };
 
@@ -226,6 +234,15 @@ int add_all_channels(const char *vm_name);
 int add_channels(const char *vm_name, unsigned *channel_list,
 		unsigned num_channels);
 
+/**
+ * Set up a fifo by which host applications can send command an policies
+ * through a fifo to the vm_power_manager
+ *
+ * @return
+ *  - 0 for success
+ */
+int add_host_channel(void);
+
 /**
  * Remove a channel definition from the channel manager. This must only be
  * called from the channel monitor thread.
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index f180d74e6..c3c3d7bb1 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -85,6 +85,33 @@ core_share_status(int pNo)
 	}
 }
 
+
+static int
+pcpu_monitor(struct policy *pol, struct core_info *ci, int pcpu, int count)
+{
+	int ret = 0;
+
+	if (pol->pkt.policy_to_use == BRANCH_RATIO) {
+		ci->cd[pcpu].oob_enabled = 1;
+		ret = add_core_to_monitor(pcpu);
+		if (ret == 0)
+			RTE_LOG(INFO, CHANNEL_MONITOR,
+					"Monitoring pcpu %d OOB for %s\n",
+					pcpu, pol->pkt.vm_name);
+		else
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Error monitoring pcpu %d OOB for %s\n",
+					pcpu, pol->pkt.vm_name);
+
+	} else {
+		pol->core_share[count].pcpu = pcpu;
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				"Monitoring pcpu %d for %s\n",
+				pcpu, pol->pkt.vm_name);
+	}
+	return ret;
+}
+
 static void
 get_pcpu_to_control(struct policy *pol)
 {
@@ -94,34 +121,42 @@ get_pcpu_to_control(struct policy *pol)
 	int pcpu, count;
 	uint64_t mask_u64b;
 	struct core_info *ci;
-	int ret;
 
 	ci = get_core_info();
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Looking for pcpu for %s\n",
-			pol->pkt.vm_name);
-	get_info_vm(pol->pkt.vm_name, &info);
-
-	for (count = 0; count < pol->pkt.num_vcpu; count++) {
-		mask_u64b = info.pcpu_mask[pol->pkt.vcpu_to_control[count]];
-		for (pcpu = 0; mask_u64b; mask_u64b &= ~(1ULL << pcpu++)) {
-			if ((mask_u64b >> pcpu) & 1) {
-				if (pol->pkt.policy_to_use == BRANCH_RATIO) {
-					ci->cd[pcpu].oob_enabled = 1;
-					ret = add_core_to_monitor(pcpu);
-					if (ret == 0)
-						printf("Monitoring pcpu %d via Branch Ratio\n",
-								pcpu);
-					else
-						printf("Failed to start OOB Monitoring pcpu %d\n",
-								pcpu);
-
-				} else {
-					pol->core_share[count].pcpu = pcpu;
-					printf("Monitoring pcpu %d\n", pcpu);
-				}
+	RTE_LOG(INFO, CHANNEL_MONITOR,
+			"Looking for pcpu for %s\n", pol->pkt.vm_name);
+
+	/*
+	 * So now that we're handling virtual and physical cores, we need to
+	 * differenciate between them when adding them to the branch monitor.
+	 * Virtual cores need to be converted to physical cores.
+	 */
+	if (pol->pkt.core_type == CORE_TYPE_VIRTUAL) {
+		/*
+		 * If the cores in the policy are virtual, we need to map them
+		 * to physical core. We look up the vm info and use that for
+		 * the mapping.
+		 */
+		get_info_vm(pol->pkt.vm_name, &info);
+		for (count = 0; count < pol->pkt.num_vcpu; count++) {
+			mask_u64b =
+				info.pcpu_mask[pol->pkt.vcpu_to_control[count]];
+			for (pcpu = 0; mask_u64b;
+					mask_u64b &= ~(1ULL << pcpu++)) {
+				if ((mask_u64b >> pcpu) & 1)
+					pcpu_monitor(pol, ci, pcpu, count);
 			}
 		}
+	} else {
+		/*
+		 * If the cores in the policy are physical, we just use
+		 * those core id's directly.
+		 */
+		for (count = 0; count < pol->pkt.num_vcpu; count++) {
+			pcpu = pol->pkt.vcpu_to_control[count];
+			pcpu_monitor(pol, ci, pcpu, count);
+		}
 	}
 }
 
@@ -160,8 +195,13 @@ update_policy(struct channel_packet *pkt)
 	unsigned int updated = 0;
 	int i;
 
+
+	RTE_LOG(INFO, CHANNEL_MONITOR,
+			"Applying policy for %s\n", pkt->vm_name);
+
 	for (i = 0; i < MAX_VMS; i++) {
 		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
+			/* Copy the contents of *pkt into the policy.pkt */
 			policies[i].pkt = *pkt;
 			get_pcpu_to_control(&policies[i]);
 			if (get_pfid(&policies[i]) == -1) {
@@ -189,6 +229,24 @@ update_policy(struct channel_packet *pkt)
 	return 0;
 }
 
+static int
+remove_policy(struct channel_packet *pkt __rte_unused)
+{
+	int i;
+
+	/*
+	 * Disabling the policy is simply a case of setting
+	 * enabled to 0
+	 */
+	for (i = 0; i < MAX_VMS; i++) {
+		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
+			policies[i].enabled = 0;
+			return 0;
+		}
+	}
+	return -1;
+}
+
 static uint64_t
 get_pkt_diff(struct policy *pol)
 {
@@ -346,7 +404,6 @@ apply_policy(struct policy *pol)
 		apply_workload_profile(pol);
 }
 
-
 static int
 process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 {
@@ -355,6 +412,8 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (chan_info == NULL)
 		return -1;
 
+	RTE_LOG(INFO, CHANNEL_MONITOR, "Processing Request %s\n", pkt->vm_name);
+
 	if (rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_CONNECTED,
 			CHANNEL_MGR_CHANNEL_PROCESSING) == 0)
 		return -1;
@@ -362,10 +421,12 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (pkt->command == CPU_POWER) {
 		core_mask = get_pcpus_mask(chan_info, pkt->resource_id);
 		if (core_mask == 0) {
-			RTE_LOG(ERR, CHANNEL_MONITOR, "Error get physical CPU mask for "
-				"channel '%s' using vCPU(%u)\n", chan_info->channel_path,
-				(unsigned)pkt->unit);
-			return -1;
+			/*
+			 * Core mask will be 0 in the case where
+			 * hypervisor is not available so we're working in
+			 * the host, so use the core as the mask.
+			 */
+			core_mask = 1ULL << pkt->resource_id;
 		}
 		if (__builtin_popcountll(core_mask) == 1) {
 
@@ -421,12 +482,20 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	}
 
 	if (pkt->command == PKT_POLICY) {
-		RTE_LOG(INFO, CHANNEL_MONITOR, "\nProcessing Policy request from Guest\n");
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				"\nProcessing Policy request\n");
 		update_policy(pkt);
 		policy_is_set = 1;
 	}
 
-	/* Return is not checked as channel status may have been set to DISABLED
+	if (pkt->command == PKT_POLICY_REMOVE) {
+		RTE_LOG(INFO, CHANNEL_MONITOR,
+				 "Removing policy %s\n", pkt->vm_name);
+		remove_policy(pkt);
+	}
+
+	/*
+	 * Return is not checked as channel status may have been set to DISABLED
 	 * from management thread
 	 */
 	rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_PROCESSING,
@@ -448,13 +517,16 @@ add_channel_to_monitor(struct channel_info **chan_info)
 				"to epoll\n", info->channel_path);
 		return -1;
 	}
+	RTE_LOG(ERR, CHANNEL_MONITOR, "Added channel '%s' "
+			"to monitor\n", info->channel_path);
 	return 0;
 }
 
 int
 remove_channel_from_monitor(struct channel_info *chan_info)
 {
-	if (epoll_ctl(global_event_fd, EPOLL_CTL_DEL, chan_info->fd, NULL) < 0) {
+	if (epoll_ctl(global_event_fd, EPOLL_CTL_DEL,
+			chan_info->fd, NULL) < 0) {
 		RTE_LOG(ERR, CHANNEL_MONITOR, "Unable to remove channel '%s' "
 				"from epoll\n", chan_info->channel_path);
 		return -1;
@@ -467,11 +539,13 @@ channel_monitor_init(void)
 {
 	global_event_fd = epoll_create1(0);
 	if (global_event_fd == 0) {
-		RTE_LOG(ERR, CHANNEL_MONITOR, "Error creating epoll context with "
-				"error %s\n", strerror(errno));
+		RTE_LOG(ERR, CHANNEL_MONITOR,
+				"Error creating epoll context with error %s\n",
+				strerror(errno));
 		return -1;
 	}
-	global_events_list = rte_malloc("epoll_events", sizeof(*global_events_list)
+	global_events_list = rte_malloc("epoll_events",
+			sizeof(*global_events_list)
 			* MAX_EVENTS, RTE_CACHE_LINE_SIZE);
 	if (global_events_list == NULL) {
 		RTE_LOG(ERR, CHANNEL_MONITOR, "Unable to rte_malloc for "
diff --git a/examples/vm_power_manager/main.c b/examples/vm_power_manager/main.c
index 58c5fa45c..893bf4cdd 100644
--- a/examples/vm_power_manager/main.c
+++ b/examples/vm_power_manager/main.c
@@ -421,6 +421,8 @@ main(int argc, char **argv)
 		return -1;
 	}
 
+	add_host_channel();
+
 	printf("Running core monitor on lcore id %d\n", lcore_id);
 	rte_eal_remote_launch(run_core_monitor, NULL, lcore_id);
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v7 06/10] examples/power: increase allowed number of clients
  2018-10-17 13:05           ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers David Hunt
                               ` (4 preceding siblings ...)
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 05/10] examples/power: add host channel to power manager David Hunt
@ 2018-10-17 13:05             ` David Hunt
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 07/10] examples/power: add json string handling David Hunt
                               ` (4 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-17 13:05 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Now that we're handling host policies, containers and virtual machines,
we'll rename MAX_VMS to MAX_CLIENTS, and increase from 4 to 64

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/channel_manager.h |  4 ++--
 examples/vm_power_manager/channel_monitor.c | 10 +++++-----
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.h b/examples/vm_power_manager/channel_manager.h
index e32235b07..d948b304c 100644
--- a/examples/vm_power_manager/channel_manager.h
+++ b/examples/vm_power_manager/channel_manager.h
@@ -37,7 +37,7 @@ struct sockaddr_un _sockaddr_un;
 #define UNIX_PATH_MAX sizeof(_sockaddr_un.sun_path)
 #endif
 
-#define MAX_VMS 4
+#define MAX_CLIENTS 64
 #define MAX_VCPUS 20
 
 
@@ -47,7 +47,7 @@ struct libvirt_vm_info {
 	uint8_t num_cpus;
 };
 
-struct libvirt_vm_info lvm_info[MAX_VMS];
+struct libvirt_vm_info lvm_info[MAX_CLIENTS];
 /* Communication Channel Status */
 enum channel_status { CHANNEL_MGR_CHANNEL_DISCONNECTED = 0,
 	CHANNEL_MGR_CHANNEL_CONNECTED,
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index c3c3d7bb1..53a4efe45 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -41,7 +41,7 @@ static volatile unsigned run_loop = 1;
 static int global_event_fd;
 static unsigned int policy_is_set;
 static struct epoll_event *global_events_list;
-static struct policy policies[MAX_VMS];
+static struct policy policies[MAX_CLIENTS];
 
 void channel_monitor_exit(void)
 {
@@ -199,7 +199,7 @@ update_policy(struct channel_packet *pkt)
 	RTE_LOG(INFO, CHANNEL_MONITOR,
 			"Applying policy for %s\n", pkt->vm_name);
 
-	for (i = 0; i < MAX_VMS; i++) {
+	for (i = 0; i < MAX_CLIENTS; i++) {
 		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
 			/* Copy the contents of *pkt into the policy.pkt */
 			policies[i].pkt = *pkt;
@@ -214,7 +214,7 @@ update_policy(struct channel_packet *pkt)
 		}
 	}
 	if (!updated) {
-		for (i = 0; i < MAX_VMS; i++) {
+		for (i = 0; i < MAX_CLIENTS; i++) {
 			if (policies[i].enabled == 0) {
 				policies[i].pkt = *pkt;
 				get_pcpu_to_control(&policies[i]);
@@ -238,7 +238,7 @@ remove_policy(struct channel_packet *pkt __rte_unused)
 	 * Disabling the policy is simply a case of setting
 	 * enabled to 0
 	 */
-	for (i = 0; i < MAX_VMS; i++) {
+	for (i = 0; i < MAX_CLIENTS; i++) {
 		if (strcmp(policies[i].pkt.vm_name, pkt->vm_name) == 0) {
 			policies[i].enabled = 0;
 			return 0;
@@ -609,7 +609,7 @@ run_channel_monitor(void)
 		if (policy_is_set) {
 			int j;
 
-			for (j = 0; j < MAX_VMS; j++) {
+			for (j = 0; j < MAX_CLIENTS; j++) {
 				if (policies[j].enabled == 1)
 					apply_policy(&policies[j]);
 			}
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v7 07/10] examples/power: add json string handling
  2018-10-17 13:05           ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers David Hunt
                               ` (5 preceding siblings ...)
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 06/10] examples/power: increase allowed number of clients David Hunt
@ 2018-10-17 13:05             ` David Hunt
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 08/10] examples/power: clean up verbose messages David Hunt
                               ` (3 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-17 13:05 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Add JSON string handling to vm_power_manager for JSON strings received
through the fifo. The format of the JSON strings are detailed in the
next patch, the vm_power_manager user guide documentation updates.

This patch introduces a new dependency on Jansson, a C library for
encoding, decoding and manipulating JSON data. To compile the sample app
you now need to have installed libjansson4 and libjansson-dev (these may
be named slightly differently depending on your Operating System)

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/Makefile          |   6 +
 examples/vm_power_manager/channel_monitor.c | 371 ++++++++++++++++++--
 2 files changed, 352 insertions(+), 25 deletions(-)

diff --git a/examples/vm_power_manager/Makefile b/examples/vm_power_manager/Makefile
index 13a5205ba..50147c05d 100644
--- a/examples/vm_power_manager/Makefile
+++ b/examples/vm_power_manager/Makefile
@@ -31,6 +31,12 @@ CFLAGS += $(WERROR_FLAGS)
 
 LDLIBS += -lvirt
 
+JANSSON := $(shell pkg-config --exists jansson; echo $$?)
+ifeq ($(JANSSON), 0)
+LDLIBS += $(shell pkg-config --libs jansson)
+CFLAGS += -DUSE_JANSSON
+endif
+
 ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
 
 ifeq ($(CONFIG_RTE_LIBRTE_IXGBE_PMD),y)
diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index 53a4efe45..afb44a069 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -9,11 +9,18 @@
 #include <signal.h>
 #include <errno.h>
 #include <string.h>
+#include <fcntl.h>
 #include <sys/types.h>
 #include <sys/epoll.h>
 #include <sys/queue.h>
 #include <sys/time.h>
-
+#include <sys/socket.h>
+#include <sys/select.h>
+#ifdef USE_JANSSON
+#include <jansson.h>
+#else
+#pragma message "Jansson dev libs unavailable, not including JSON parsing"
+#endif
 #include <rte_log.h>
 #include <rte_memory.h>
 #include <rte_malloc.h>
@@ -35,6 +42,8 @@
 
 uint64_t vsi_pkt_count_prev[384];
 uint64_t rdtsc_prev[384];
+#define MAX_JSON_STRING_LEN 1024
+char json_data[MAX_JSON_STRING_LEN];
 
 double time_period_ms = 1;
 static volatile unsigned run_loop = 1;
@@ -43,6 +52,234 @@ static unsigned int policy_is_set;
 static struct epoll_event *global_events_list;
 static struct policy policies[MAX_CLIENTS];
 
+#ifdef USE_JANSSON
+
+union PFID {
+	struct ether_addr addr;
+	uint64_t pfid;
+};
+
+static int
+str_to_ether_addr(const char *a, struct ether_addr *ether_addr)
+{
+	int i;
+	char *end;
+	unsigned long o[ETHER_ADDR_LEN];
+
+	i = 0;
+	do {
+		errno = 0;
+		o[i] = strtoul(a, &end, 16);
+		if (errno != 0 || end == a || (end[0] != ':' && end[0] != 0))
+			return -1;
+		a = end + 1;
+	} while (++i != RTE_DIM(o) / sizeof(o[0]) && end[0] != 0);
+
+	/* Junk at the end of line */
+	if (end[0] != 0)
+		return -1;
+
+	/* Support the format XX:XX:XX:XX:XX:XX */
+	if (i == ETHER_ADDR_LEN) {
+		while (i-- != 0) {
+			if (o[i] > UINT8_MAX)
+				return -1;
+			ether_addr->addr_bytes[i] = (uint8_t)o[i];
+		}
+	/* Support the format XXXX:XXXX:XXXX */
+	} else if (i == ETHER_ADDR_LEN / 2) {
+		while (i-- != 0) {
+			if (o[i] > UINT16_MAX)
+				return -1;
+			ether_addr->addr_bytes[i * 2] =
+					(uint8_t)(o[i] >> 8);
+			ether_addr->addr_bytes[i * 2 + 1] =
+					(uint8_t)(o[i] & 0xff);
+		}
+	/* unknown format */
+	} else
+		return -1;
+
+	return 0;
+}
+
+static int
+set_policy_mac(struct channel_packet *pkt, int idx, char *mac)
+{
+	union PFID pfid;
+	int ret;
+
+	/* Use port MAC address as the vfid */
+	ret = str_to_ether_addr(mac, &pfid.addr);
+
+	if (ret != 0) {
+		RTE_LOG(ERR, CHANNEL_MONITOR,
+			"Invalid mac address received in JSON\n");
+		pkt->vfid[idx] = 0;
+		return -1;
+	}
+
+	printf("Received MAC Address: %02" PRIx8 ":%02" PRIx8 ":%02" PRIx8 ":"
+			"%02" PRIx8 ":%02" PRIx8 ":%02" PRIx8 "\n",
+			pfid.addr.addr_bytes[0], pfid.addr.addr_bytes[1],
+			pfid.addr.addr_bytes[2], pfid.addr.addr_bytes[3],
+			pfid.addr.addr_bytes[4], pfid.addr.addr_bytes[5]);
+
+	pkt->vfid[idx] = pfid.pfid;
+	return 0;
+}
+
+
+static int
+parse_json_to_pkt(json_t *element, struct channel_packet *pkt)
+{
+	const char *key;
+	json_t *value;
+	int ret;
+
+	memset(pkt, 0, sizeof(struct channel_packet));
+
+	pkt->nb_mac_to_monitor = 0;
+	pkt->t_boost_status.tbEnabled = false;
+	pkt->workload = LOW;
+	pkt->policy_to_use = TIME;
+	pkt->command = PKT_POLICY;
+	pkt->core_type = CORE_TYPE_PHYSICAL;
+
+	json_object_foreach(element, key, value) {
+		if (!strcmp(key, "policy")) {
+			/* Recurse in to get the contents of profile */
+			ret = parse_json_to_pkt(value, pkt);
+			if (ret)
+				return ret;
+		} else if (!strcmp(key, "instruction")) {
+			/* Recurse in to get the contents of instruction */
+			ret = parse_json_to_pkt(value, pkt);
+			if (ret)
+				return ret;
+		} else if (!strcmp(key, "name")) {
+			strcpy(pkt->vm_name, json_string_value(value));
+		} else if (!strcmp(key, "command")) {
+			char command[32];
+			snprintf(command, 32, "%s", json_string_value(value));
+			if (!strcmp(command, "power")) {
+				pkt->command = CPU_POWER;
+			} else if (!strcmp(command, "create")) {
+				pkt->command = PKT_POLICY;
+			} else if (!strcmp(command, "destroy")) {
+				pkt->command = PKT_POLICY_REMOVE;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Invalid command received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "policy_type")) {
+			char command[32];
+			snprintf(command, 32, "%s", json_string_value(value));
+			if (!strcmp(command, "TIME")) {
+				pkt->policy_to_use = TIME;
+			} else if (!strcmp(command, "TRAFFIC")) {
+				pkt->policy_to_use = TRAFFIC;
+			} else if (!strcmp(command, "WORKLOAD")) {
+				pkt->policy_to_use = WORKLOAD;
+			} else if (!strcmp(command, "BRANCH_RATIO")) {
+				pkt->policy_to_use = BRANCH_RATIO;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Wrong policy_type received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "workload")) {
+			char command[32];
+			snprintf(command, 32, "%s", json_string_value(value));
+			if (!strcmp(command, "HIGH")) {
+				pkt->workload = HIGH;
+			} else if (!strcmp(command, "MEDIUM")) {
+				pkt->workload = MEDIUM;
+			} else if (!strcmp(command, "LOW")) {
+				pkt->workload = LOW;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Wrong workload received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "busy_hours")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int hour = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->timer_policy.busy_hours[i] = hour;
+			}
+		} else if (!strcmp(key, "quiet_hours")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int hour = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->timer_policy.quiet_hours[i] = hour;
+			}
+		} else if (!strcmp(key, "core_list")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				int core = (int)json_integer_value(
+						json_array_get(value, i));
+				pkt->vcpu_to_control[i] = core;
+			}
+			pkt->num_vcpu = size;
+		} else if (!strcmp(key, "mac_list")) {
+			unsigned int i;
+			size_t size = json_array_size(value);
+
+			for (i = 0; i < size; i++) {
+				char mac[32];
+				snprintf(mac, 32, "%s", json_string_value(
+						json_array_get(value, i)));
+				set_policy_mac(pkt, i, mac);
+			}
+			pkt->nb_mac_to_monitor = size;
+		} else if (!strcmp(key, "avg_packet_thresh")) {
+			pkt->traffic_policy.avg_max_packet_thresh =
+					(uint32_t)json_integer_value(value);
+		} else if (!strcmp(key, "max_packet_thresh")) {
+			pkt->traffic_policy.max_max_packet_thresh =
+					(uint32_t)json_integer_value(value);
+		} else if (!strcmp(key, "unit")) {
+			char unit[32];
+			snprintf(unit, 32, "%s", json_string_value(value));
+			if (!strcmp(unit, "SCALE_UP")) {
+				pkt->unit = CPU_POWER_SCALE_UP;
+			} else if (!strcmp(unit, "SCALE_DOWN")) {
+				pkt->unit = CPU_POWER_SCALE_DOWN;
+			} else if (!strcmp(unit, "SCALE_MAX")) {
+				pkt->unit = CPU_POWER_SCALE_MAX;
+			} else if (!strcmp(unit, "SCALE_MIN")) {
+				pkt->unit = CPU_POWER_SCALE_MIN;
+			} else if (!strcmp(unit, "ENABLE_TURBO")) {
+				pkt->unit = CPU_POWER_ENABLE_TURBO;
+			} else if (!strcmp(unit, "DISABLE_TURBO")) {
+				pkt->unit = CPU_POWER_DISABLE_TURBO;
+			} else {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Invalid command received in JSON\n");
+				return -1;
+			}
+		} else if (!strcmp(key, "resource_id")) {
+			pkt->resource_id = (uint32_t)json_integer_value(value);
+		} else {
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+				"Unknown key received in JSON string: %s\n",
+				key);
+		}
+	}
+	return 0;
+}
+#endif
+
 void channel_monitor_exit(void)
 {
 	run_loop = 0;
@@ -555,6 +792,103 @@ channel_monitor_init(void)
 	return 0;
 }
 
+static void
+read_binary_packet(struct channel_info *chan_info)
+{
+	struct channel_packet pkt;
+	void *buffer = &pkt;
+	int buffer_len = sizeof(pkt);
+	int n_bytes, err = 0;
+
+	while (buffer_len > 0) {
+		n_bytes = read(chan_info->fd,
+				buffer, buffer_len);
+		if (n_bytes == buffer_len)
+			break;
+		if (n_bytes == -1) {
+			err = errno;
+			RTE_LOG(DEBUG, CHANNEL_MONITOR,
+				"Received error on "
+				"channel '%s' read: %s\n",
+				chan_info->channel_path,
+				strerror(err));
+			remove_channel(&chan_info);
+			break;
+		}
+		buffer = (char *)buffer + n_bytes;
+		buffer_len -= n_bytes;
+	}
+	if (!err)
+		process_request(&pkt, chan_info);
+}
+
+#ifdef USE_JANSSON
+static void
+read_json_packet(struct channel_info *chan_info)
+{
+	struct channel_packet pkt;
+	int n_bytes, ret;
+	json_t *root;
+	json_error_t error;
+
+	/* read opening brace to closing brace */
+	do {
+		int idx = 0;
+		int indent = 0;
+		do {
+			n_bytes = read(chan_info->fd, &json_data[idx], 1);
+			if (n_bytes == 0)
+				break;
+			if (json_data[idx] == '{')
+				indent++;
+			if (json_data[idx] == '}')
+				indent--;
+			if ((indent > 0) || (idx > 0))
+				idx++;
+			if (indent == 0)
+				json_data[idx] = 0;
+			if (idx >= MAX_JSON_STRING_LEN-1)
+				break;
+		} while (indent > 0);
+
+		if (indent > 0)
+			/*
+			 * We've broken out of the read loop without getting
+			 * a closing brace, so throw away the data
+			 */
+			json_data[idx] = 0;
+
+		if (strlen(json_data) == 0)
+			continue;
+
+		printf("got [%s]\n", json_data);
+
+		root = json_loads(json_data, 0, &error);
+
+		if (root) {
+			/*
+			 * Because our data is now in the json
+			 * object, we can overwrite the pkt
+			 * with a channel_packet struct, using
+			 * parse_json_to_pkt()
+			 */
+			ret = parse_json_to_pkt(root, &pkt);
+			json_decref(root);
+			if (ret) {
+				RTE_LOG(ERR, CHANNEL_MONITOR,
+					"Error validating JSON profile data\n");
+				break;
+			}
+			process_request(&pkt, chan_info);
+		} else {
+			RTE_LOG(ERR, CHANNEL_MONITOR,
+					"JSON error on line %d: %s\n",
+					error.line, error.text);
+		}
+	} while (n_bytes > 0);
+}
+#endif
+
 void
 run_channel_monitor(void)
 {
@@ -578,31 +912,18 @@ run_channel_monitor(void)
 			}
 			if (global_events_list[i].events & EPOLLIN) {
 
-				int n_bytes, err = 0;
-				struct channel_packet pkt;
-				void *buffer = &pkt;
-				int buffer_len = sizeof(pkt);
-
-				while (buffer_len > 0) {
-					n_bytes = read(chan_info->fd,
-							buffer, buffer_len);
-					if (n_bytes == buffer_len)
-						break;
-					if (n_bytes == -1) {
-						err = errno;
-						RTE_LOG(DEBUG, CHANNEL_MONITOR,
-							"Received error on "
-							"channel '%s' read: %s\n",
-							chan_info->channel_path,
-							strerror(err));
-						remove_channel(&chan_info);
-						break;
-					}
-					buffer = (char *)buffer + n_bytes;
-					buffer_len -= n_bytes;
+				switch (chan_info->type) {
+				case CHANNEL_TYPE_BINARY:
+					read_binary_packet(chan_info);
+					break;
+#ifdef USE_JANSSON
+				case CHANNEL_TYPE_JSON:
+					read_json_packet(chan_info);
+					break;
+#endif
+				default:
+					break;
 				}
-				if (!err)
-					process_request(&pkt, chan_info);
 			}
 		}
 		rte_delay_us(time_period_ms*1000);
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v7 08/10] examples/power: clean up verbose messages
  2018-10-17 13:05           ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers David Hunt
                               ` (6 preceding siblings ...)
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 07/10] examples/power: add json string handling David Hunt
@ 2018-10-17 13:05             ` David Hunt
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 09/10] examples/power: add meson/ninja build support David Hunt
                               ` (2 subsequent siblings)
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-17 13:05 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Some messages appearing several times a second, removing as they are
unnecessary. Other less severe messages change from INFO to DEBUG

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/vm_power_manager/channel_monitor.c | 19 +++++--------------
 1 file changed, 5 insertions(+), 14 deletions(-)

diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c
index afb44a069..5da531542 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -361,7 +361,7 @@ get_pcpu_to_control(struct policy *pol)
 
 	ci = get_core_info();
 
-	RTE_LOG(INFO, CHANNEL_MONITOR,
+	RTE_LOG(DEBUG, CHANNEL_MONITOR,
 			"Looking for pcpu for %s\n", pol->pkt.vm_name);
 
 	/*
@@ -528,8 +528,6 @@ apply_traffic_profile(struct policy *pol)
 
 	diff = get_pkt_diff(pol);
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Applying traffic profile\n");
-
 	if (diff >= (pol->pkt.traffic_policy.max_max_packet_thresh)) {
 		for (count = 0; count < pol->pkt.num_vcpu; count++) {
 			if (pol->core_share[count].status != 1)
@@ -573,9 +571,6 @@ apply_time_profile(struct policy *pol)
 				if (pol->core_share[count].status != 1) {
 					power_manager_scale_core_max(
 						pol->core_share[count].pcpu);
-				RTE_LOG(INFO, CHANNEL_MONITOR,
-					"Scaling up core %d to max\n",
-					pol->core_share[count].pcpu);
 				}
 			}
 			break;
@@ -585,9 +580,6 @@ apply_time_profile(struct policy *pol)
 				if (pol->core_share[count].status != 1) {
 					power_manager_scale_core_min(
 						pol->core_share[count].pcpu);
-				RTE_LOG(INFO, CHANNEL_MONITOR,
-					"Scaling down core %d to min\n",
-					pol->core_share[count].pcpu);
 			}
 		}
 			break;
@@ -649,8 +641,6 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	if (chan_info == NULL)
 		return -1;
 
-	RTE_LOG(INFO, CHANNEL_MONITOR, "Processing Request %s\n", pkt->vm_name);
-
 	if (rte_atomic32_cmpset(&(chan_info->status), CHANNEL_MGR_CHANNEL_CONNECTED,
 			CHANNEL_MGR_CHANNEL_PROCESSING) == 0)
 		return -1;
@@ -719,8 +709,8 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 	}
 
 	if (pkt->command == PKT_POLICY) {
-		RTE_LOG(INFO, CHANNEL_MONITOR,
-				"\nProcessing Policy request\n");
+		RTE_LOG(INFO, CHANNEL_MONITOR, "Processing policy request %s\n",
+				pkt->vm_name);
 		update_policy(pkt);
 		policy_is_set = 1;
 	}
@@ -904,7 +894,8 @@ run_channel_monitor(void)
 					global_events_list[i].data.ptr;
 			if ((global_events_list[i].events & EPOLLERR) ||
 				(global_events_list[i].events & EPOLLHUP)) {
-				RTE_LOG(DEBUG, CHANNEL_MONITOR, "Remote closed connection for "
+				RTE_LOG(INFO, CHANNEL_MONITOR,
+						"Remote closed connection for "
 						"channel '%s'\n",
 						chan_info->channel_path);
 				remove_channel(&chan_info);
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v7 09/10] examples/power: add meson/ninja build support
  2018-10-17 13:05           ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers David Hunt
                               ` (7 preceding siblings ...)
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 08/10] examples/power: clean up verbose messages David Hunt
@ 2018-10-17 13:05             ` David Hunt
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 10/10] doc/vm_power_manager: add JSON interface API info David Hunt
  2018-10-26  8:45             ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers Thomas Monjalon
  10 siblings, 0 replies; 110+ messages in thread
From: David Hunt @ 2018-10-17 13:05 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

Add meson.build in vm_power_manager and the guest_cli subdirectory.
Building can be achieved by going to the build directory, and using

meson configure -Dexamples=vm_power_manager,vm_power_manager/guest_cli

Then, when ninja is invoked, it will build dpdk-vm_power_manger and
dpdk-guest_cli

Work still needs to be done on the meson build system to handles the case
where the target list of example apps is defined as 'all'. That will come
in a future patch.

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
---
 .../vm_power_manager/guest_cli/meson.build    | 21 +++++++++++
 examples/vm_power_manager/meson.build         | 37 ++++++++++++++++++-
 2 files changed, 56 insertions(+), 2 deletions(-)
 create mode 100644 examples/vm_power_manager/guest_cli/meson.build

diff --git a/examples/vm_power_manager/guest_cli/meson.build b/examples/vm_power_manager/guest_cli/meson.build
new file mode 100644
index 000000000..9e821ceb8
--- /dev/null
+++ b/examples/vm_power_manager/guest_cli/meson.build
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+# Setting the name here because the default name will conflict with the
+# vm_power_manager app because of the way the directories are parsed.
+name = 'guest_cli'
+
+deps += ['power']
+
+sources = files(
+	'main.c', 'parse.c', 'vm_power_cli_guest.c'
+)
+
+opt_dep = cc.find_library('virt', required : false)
+build = opt_dep.found()
+ext_deps += opt_dep
diff --git a/examples/vm_power_manager/meson.build b/examples/vm_power_manager/meson.build
index c370d7476..f98445bc6 100644
--- a/examples/vm_power_manager/meson.build
+++ b/examples/vm_power_manager/meson.build
@@ -6,5 +6,38 @@
 # To build this example as a standalone application with an already-installed
 # DPDK instance, use 'make'
 
-# Example app currently unsupported by meson build
-build = false
+if dpdk_conf.has('RTE_LIBRTE_BNXT_PMD')
+	deps += ['pmd_bnxt']
+endif
+
+if dpdk_conf.has('RTE_LIBRTE_I40E_PMD')
+	deps += ['pmd_i40e']
+endif
+
+if dpdk_conf.has('RTE_LIBRTE_IXGBE_PMD')
+	deps += ['pmd_ixgbe']
+endif
+
+deps += ['power']
+
+
+sources = files(
+	'channel_manager.c', 'channel_monitor.c', 'main.c', 'parse.c', 'power_manager.c', 'vm_power_cli.c'
+)
+
+# If we're on X86, pull in the x86 code for the branch monitor algo.
+if dpdk_conf.has('RTE_ARCH_X86_64')
+	sources += files('oob_monitor_x86.c')
+else
+	sources += files('oob_monitor_nop.c')
+endif
+
+opt_dep = cc.find_library('virt', required : false)
+build = opt_dep.found()
+ext_deps += opt_dep
+
+opt_dep = dependency('jansson', required : false)
+if opt_dep.found()
+	ext_deps += opt_dep
+	cflags += '-DUSE_JANSSON'
+endif
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* [dpdk-dev] [PATCH v7 10/10] doc/vm_power_manager: add JSON interface API info
  2018-10-17 13:05           ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers David Hunt
                               ` (8 preceding siblings ...)
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 09/10] examples/power: add meson/ninja build support David Hunt
@ 2018-10-17 13:05             ` David Hunt
  2018-10-26  0:05               ` Thomas Monjalon
  2018-10-26  8:45             ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers Thomas Monjalon
  10 siblings, 1 reply; 110+ messages in thread
From: David Hunt @ 2018-10-17 13:05 UTC (permalink / raw)
  To: dev; +Cc: john.mcnamara, stephen, lei.a.yao, anatoly.burakov, David Hunt

* added JSON interface API info
* added meson/ninja build info
* added Release Note changes

Signed-off-by: David Hunt <david.hunt@intel.com>
Acked-by: Marko Kovacevic <marko.kovacevic@intel.com>
---
 doc/guides/rel_notes/release_18_11.rst        |   8 +
 .../sample_app_ug/vm_power_management.rst     | 300 +++++++++++++++++-
 2 files changed, 306 insertions(+), 2 deletions(-)

diff --git a/doc/guides/rel_notes/release_18_11.rst b/doc/guides/rel_notes/release_18_11.rst
index 2467d04e2..6d92c5c93 100644
--- a/doc/guides/rel_notes/release_18_11.rst
+++ b/doc/guides/rel_notes/release_18_11.rst
@@ -155,6 +155,14 @@ New Features
   the SW eventdev PMD, sacrifices load balancing performance to
   gain better event scheduling throughput and scalability.
 
+* **Added JSON power policy interface for containers .**
+
+  Extended the Power Library and vm_power_manger sample app to allow power
+  policies to be submitted via a FIFO using JSON formatted strings. Previously
+  limited to Virtual Machines, this feature extends power policy functionality
+  to containers and host applications that need to have their cores frequency
+  controlled based on the rules contained in the policy.
+
 * **Added ability to switch queue deferred start flag on testpmd app.**
 
   Added a console command to testpmd app, giving ability to switch
diff --git a/doc/guides/sample_app_ug/vm_power_management.rst b/doc/guides/sample_app_ug/vm_power_management.rst
index 855570d6b..1ad4f1490 100644
--- a/doc/guides/sample_app_ug/vm_power_management.rst
+++ b/doc/guides/sample_app_ug/vm_power_management.rst
@@ -199,7 +199,7 @@ see :doc:`compiling`.
 
 The application is located in the ``vm_power_manager`` sub-directory.
 
-To build just the ``vm_power_manager`` application:
+To build just the ``vm_power_manager`` application using ``make``:
 
 .. code-block:: console
 
@@ -208,6 +208,22 @@ To build just the ``vm_power_manager`` application:
   cd ${RTE_SDK}/examples/vm_power_manager/
   make
 
+The resulting binary will be ${RTE_SDK}/build/examples/vm_power_manager
+
+To build just the ``vm_power_manager`` application using ``meson/ninja``:
+
+.. code-block:: console
+
+  export RTE_SDK=/path/to/rte_sdk
+  cd ${RTE_SDK}
+  meson build
+  cd build
+  ninja
+  meson configure -Dexamples=vm_power_manager
+  ninja
+
+The resulting binary will be ${RTE_SDK}/build/examples/dpdk-vm_power_manager
+
 Running
 ~~~~~~~
 
@@ -337,6 +353,270 @@ monitoring of branch ratio on cores doing busy polling via PMDs.
   and will need to be adjusted for different workloads.
 
 
+
+JSON API
+~~~~~~~~
+
+In addition to the command line interface for host command and a virtio-serial
+interface for VM power policies, there is also a JSON interface through which
+power commands and policies can be sent. This functionality adds a dependency
+on the Jansson library, and the Jansson development package must be installed
+on the system before the JSON parsing functionality is included in the app.
+This is achieved by:
+
+  .. code-block:: javascript
+
+    apt-get install libjansson-dev
+
+The command and package name may be different depending on your operating
+system. It's worth noting that the app will successfully build without this
+package present, but a warning is shown during compilation, and the JSON
+parsing functionality will not be present in the app.
+
+Sending a command or policy to the power manager application is achieved by
+simply opening a fifo file, writing a JSON string to that fifo, and closing
+the file.
+
+The fifo is at /tmp/powermonitor/fifo
+
+The jason string can be a policy or instruction, and takes the following
+format:
+
+  .. code-block:: javascript
+
+    {"packet_type": {
+      "pair_1": value,
+      "pair_2": value
+    }}
+
+The 'packet_type' header can contain one of two values, depending on
+whether a policy or power command is being sent. The two possible values are
+"policy" and "instruction", and the expected name-value pairs is different
+depending on which type is being sent.
+
+The pairs are the format of standard JSON name-value pairs. The value type
+varies between the different name/value pairs, and may be integers, strings,
+arrays, etc. Examples of policies follow later in this document. The allowed
+names and value types are as follows:
+
+
+:Pair Name: "name"
+:Description: Name of the VM or Host. Allows the parser to associate the
+  policy with the relevant VM or Host OS.
+:Type: string
+:Values: any valid string
+:Required: yes
+:Example:
+
+    .. code-block:: javascript
+
+      "name", "ubuntu2"
+
+
+:Pair Name: "command"
+:Description: The type of packet we're sending to the power manager. We can be
+  creating or destroying a policy, or sending a direct command to adjust
+  the frequency of a core, similar to the command line interface.
+:Type: string
+:Values:
+
+  :CREATE: used when creating a new policy,
+  :DESTROY: used when removing a policy,
+  :POWER: used when sending an immediate command, max, min, etc.
+:Required: yes
+:Example:
+
+    .. code-block:: javascript
+
+      "command", "CREATE"
+
+
+:Pair Name: "policy_type"
+:Description: Type of policy to apply. Please see vm_power_manager documentation
+  for more information on the types of policies that may be used.
+:Type: string
+:Values:
+
+  :TIME: Time-of-day policy. Frequencies of the relevant cores are
+    scaled up/down depending on busy and quiet hours.
+  :TRAFFIC: This policy takes statistics from the NIC and scales up
+    and down accordingly.
+  :WORKLOAD: This policy looks at how heavily loaded the cores are,
+    and scales up and down accordingly.
+  :BRANCH_RATIO: This out-of-band policy can look at the ratio between
+    branch hits and misses on a core, and is useful for detecting
+    how much packet processing a core is doing.
+:Required: only for CREATE/DESTROY command
+:Example:
+
+  .. code-block:: javascript
+
+    "policy_type", "TIME"
+
+:Pair Name: "busy_hours"
+:Description: The hours of the day in which we scale up the cores for busy
+  times.
+:Type: array of integers
+:Values: array with list of hour numbers, (0-23)
+:Required: only for TIME policy
+:Example:
+
+  .. code-block:: javascript
+
+    "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ]
+
+:Pair Name: "quiet_hours"
+:Description: The hours of the day in which we scale down the cores for quiet
+  times.
+:Type: array of integers
+:Values: array with list of hour numbers, (0-23)
+:Required: only for TIME policy
+:Example:
+
+  .. code-block:: javascript
+
+    "quiet_hours":[ 2, 3, 4, 5, 6 ]
+
+:Pair Name: "avg_packet_thresh"
+:Description: Threshold below which the frequency will be set to min for
+  the TRAFFIC policy. If the traffic rate is above this and below max, the
+  frequency will be set to medium.
+:Type: integer
+:Values: The number of packets below which the TRAFFIC policy applies the
+  minimum frequency, or medium frequency if between avg and max thresholds.
+:Required: only for TRAFFIC policy
+:Example:
+
+  .. code-block:: javascript
+
+    "avg_packet_thresh": 100000
+
+:Pair Name: "max_packet_thresh"
+:Description: Threshold above which the frequency will be set to max for
+  the TRAFFIC policy
+:Type: integer
+:Values: The number of packets per interval above which the TRAFFIC policy
+  applies the maximum frequency
+:Required: only for TRAFFIC policy
+:Example:
+
+  .. code-block:: javascript
+
+    "max_packet_thresh": 500000
+
+:Pair Name: "core_list"
+:Description: The cores to which to apply the policy.
+:Type: array of integers
+:Values: array with list of virtual CPUs.
+:Required: only policy CREATE/DESTROY
+:Example:
+
+  .. code-block:: javascript
+
+    "core_list":[ 10, 11 ]
+
+:Pair Name: "workload"
+:Description: When our policy is of type WORKLOAD, we need to specify how
+  heavy our workload is.
+:Type: string
+:Values:
+
+  :HIGH: For cores running workloads that require high frequencies
+  :MEDIUM: For cores running workloads that require medium frequencies
+  :LOW: For cores running workloads that require low frequencies
+:Required: only for WORKLOAD policy types
+:Example:
+
+  .. code-block:: javascript
+
+    "workload", "MEDIUM"
+
+:Pair Name: "mac_list"
+:Description: When our policy is of type TRAFFIC, we need to specify the
+  MAC addresses that the host needs to monitor
+:Type: string
+:Values: array with a list of mac address strings.
+:Required: only for TRAFFIC policy types
+:Example:
+
+  .. code-block:: javascript
+
+    "mac_list":[ "de:ad:be:ef:01:01", "de:ad:be:ef:01:02" ]
+
+:Pair Name: "unit"
+:Description: the type of power operation to apply in the command
+:Type: string
+:Values:
+
+  :SCALE_MAX: Scale frequency of this core to maximum
+  :SCALE_MIN: Scale frequency of this core to minimum
+  :SCALE_UP: Scale up frequency of this core
+  :SCALE_DOWN: Scale down frequency of this core
+  :ENABLE_TURBO: Enable Turbo Boost for this core
+  :DISABLE_TURBO: Disable Turbo Boost for this core
+:Required: only for POWER instruction
+:Example:
+
+  .. code-block:: javascript
+
+    "unit", "SCALE_MAX"
+
+:Pair Name: "resource_id"
+:Description: The core to which to apply the power command.
+:Type: integer
+:Values: valid core id for VM or host OS.
+:Required: only POWER instruction
+:Example:
+
+  .. code-block:: javascript
+
+    "resource_id": 10
+
+JSON API Examples
+~~~~~~~~~~~~~~~~~
+
+Profile create example:
+
+  .. code-block:: javascript
+
+    {"policy": {
+      "name": "ubuntu",
+      "command": "create",
+      "policy_type": "TIME",
+      "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ],
+      "quiet_hours":[ 2, 3, 4, 5, 6 ],
+      "core_list":[ 11 ]
+    }}
+
+Profile destroy example:
+
+  .. code-block:: javascript
+
+    {"profile": {
+      "name": "ubuntu",
+      "command": "destroy",
+    }}
+
+Power command example:
+
+  .. code-block:: javascript
+
+    {"command": {
+      "name": "ubuntu",
+      "unit": "SCALE_MAX",
+      "resource_id": 10
+    }}
+
+To send a JSON string to the Power Manager application, simply paste the
+example JSON string into a text file and cat it into the fifo:
+
+  .. code-block:: console
+
+    cat file.json >/tmp/powermonitor/fifo
+
+The console of the Power Manager application should indicate the command that
+was just received via the fifo.
+
 Compiling and Running the Guest Applications
 --------------------------------------------
 
@@ -366,7 +646,7 @@ For compiling and running l3fwd-power, see :doc:`l3_forward_power_man`.
 
 The application is located in the ``guest_cli`` sub-directory under ``vm_power_manager``.
 
-To build just the ``guest_vm_power_manager`` application:
+To build just the ``guest_vm_power_manager`` application using ``make``:
 
 .. code-block:: console
 
@@ -375,6 +655,22 @@ To build just the ``guest_vm_power_manager`` application:
   cd ${RTE_SDK}/examples/vm_power_manager/guest_cli/
   make
 
+The resulting binary will be ${RTE_SDK}/build/examples/guest_cli
+
+To build just the ``vm_power_manager`` application using ``meson/ninja``:
+
+.. code-block:: console
+
+  export RTE_SDK=/path/to/rte_sdk
+  cd ${RTE_SDK}
+  meson build
+  cd build
+  ninja
+  meson configure -Dexamples=vm_power_manager/guest_cli
+  ninja
+
+The resulting binary will be ${RTE_SDK}/build/examples/guest_cli
+
 Running
 ~~~~~~~
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v7 10/10] doc/vm_power_manager: add JSON interface API info
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 10/10] doc/vm_power_manager: add JSON interface API info David Hunt
@ 2018-10-26  0:05               ` Thomas Monjalon
  0 siblings, 0 replies; 110+ messages in thread
From: Thomas Monjalon @ 2018-10-26  0:05 UTC (permalink / raw)
  To: David Hunt; +Cc: dev, john.mcnamara, stephen, lei.a.yao, anatoly.burakov

17/10/2018 15:05, David Hunt:
> * added JSON interface API info

Should be in patch 7

> * added meson/ninja build info

Should be in patch 9

> * added Release Note changes

Should be in patch 7

> Signed-off-by: David Hunt <david.hunt@intel.com>
> Acked-by: Marko Kovacevic <marko.kovacevic@intel.com>

^ permalink raw reply	[flat|nested] 110+ messages in thread

* Re: [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers
  2018-10-17 13:05           ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers David Hunt
                               ` (9 preceding siblings ...)
  2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 10/10] doc/vm_power_manager: add JSON interface API info David Hunt
@ 2018-10-26  8:45             ` Thomas Monjalon
  10 siblings, 0 replies; 110+ messages in thread
From: Thomas Monjalon @ 2018-10-26  8:45 UTC (permalink / raw)
  To: David Hunt; +Cc: dev, john.mcnamara, stephen, lei.a.yao, anatoly.burakov

17/10/2018 15:05, David Hunt:
> [01/10] examples/power: add checks around hypervisor
> [02/10] examples/power: allow for number of vms to be zero
> [03/10] lib/power: add changes for host commands/policies
> [04/10] examples/power: add necessary changes to guest app
> [05/10] examples/power: add host channel to power manager
> [06/10] examples/power: increase allowed number of clients
> [07/10] examples/power: add json string handling
> [08/10] examples/power: clean up verbose messages
> [09/10] examples/power: add meson/ninja build support
> [10/10] doc/vm_power_manager: add JSON interface API info

Applied (with doc spread in other patches), thanks.

^ permalink raw reply	[flat|nested] 110+ messages in thread

end of thread, other threads:[~2018-10-26  8:45 UTC | newest]

Thread overview: 110+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-08-30 10:54 [dpdk-dev] [PATCH v1 0/7] add json power policy interface for containers David Hunt
2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 1/7] examples/power: add checks around hypervisor David Hunt
2018-08-30 16:59   ` Stephen Hemminger
2018-09-12 10:53     ` Hunt, David
2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 2/7] lib/power: add changes for host commands/policies David Hunt
2018-08-30 16:59   ` Stephen Hemminger
2018-09-12 10:51     ` Hunt, David
2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 3/7] examples/power: add necessary changes to guest app David Hunt
2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 4/7] examples/power: add host channel to power manager David Hunt
2018-09-04  7:31   ` Yao, Lei A
2018-09-12 12:07     ` Hunt, David
2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 5/7] examples/power: add json string handling David Hunt
2018-08-30 17:00   ` Stephen Hemminger
2018-08-31 13:55     ` Hunt, David
2018-09-12 10:54     ` Hunt, David
2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 6/7] doc/vm_power_manager: add JSON interface API info David Hunt
2018-09-04  5:17   ` Yao, Lei A
2018-09-12 11:31     ` Hunt, David
2018-08-30 10:54 ` [dpdk-dev] [PATCH v1 7/7] examples/power: add json example files David Hunt
2018-09-12 14:49 ` [dpdk-dev] [PATCH v2 0/7] add json power policy interface for containers David Hunt
2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 1/7] examples/power: add checks around hypervisor David Hunt
2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 2/7] lib/power: add changes for host commands/policies David Hunt
2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 3/7] examples/power: add necessary changes to guest app David Hunt
2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 4/7] examples/power: add host channel to power manager David Hunt
2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 5/7] examples/power: add json string handling David Hunt
2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 6/7] doc/vm_power_manager: add JSON interface API info David Hunt
2018-09-12 14:49   ` [dpdk-dev] [PATCH v2 7/7] examples/power: add json example files David Hunt
2018-09-14 13:53   ` [dpdk-dev] [PATCH v3 0/8] add json power policy interface for containers David Hunt
2018-09-14 13:53     ` [dpdk-dev] [PATCH v3 1/8] examples/power: add checks around hypervisor David Hunt
2018-09-25  9:20       ` Burakov, Anatoly
2018-09-25 13:47         ` Hunt, David
2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 2/8] lib/power: add changes for host commands/policies David Hunt
2018-09-25  9:21       ` Burakov, Anatoly
2018-09-25 13:47         ` Hunt, David
2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 3/8] examples/power: add necessary changes to guest app David Hunt
2018-09-25  9:23       ` Burakov, Anatoly
2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 4/8] examples/power: add host channel to power manager David Hunt
2018-09-25  9:48       ` Burakov, Anatoly
2018-09-25 13:55         ` Hunt, David
2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 5/8] examples/power: add json string handling David Hunt
2018-09-25 11:27       ` Burakov, Anatoly
2018-09-25 14:00         ` Hunt, David
2018-09-25 14:15           ` Burakov, Anatoly
2018-09-25 15:15             ` Hunt, David
2018-09-25 15:31               ` Burakov, Anatoly
2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 6/8] examples/power: add meson/ninja build support David Hunt
2018-09-14 14:01       ` Bruce Richardson
2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 7/8] doc/vm_power_manager: add JSON interface API info David Hunt
2018-09-14 13:54     ` [dpdk-dev] [PATCH v3 8/8] examples/power: add json example files David Hunt
2018-09-26 13:40     ` [dpdk-dev] [PATCH v4 0/11] add json power policy interface for containers David Hunt
2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 01/11] examples/power: add checks around hypervisor David Hunt
2018-09-26 13:54         ` Burakov, Anatoly
2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 02/11] examples/power: allow for number of vms to be zero David Hunt
2018-09-26 13:54         ` Burakov, Anatoly
2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 03/11] lib/power: add changes for host commands/policies David Hunt
2018-09-26 13:54         ` Burakov, Anatoly
2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 04/11] examples/power: add necessary changes to guest app David Hunt
2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 05/11] examples/power: add host channel to power manager David Hunt
2018-09-26 14:22         ` Burakov, Anatoly
2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 06/11] examples/power: increase allowed number of clients David Hunt
2018-09-26 14:23         ` Burakov, Anatoly
2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 07/11] examples/power: add json string handling David Hunt
2018-09-26 14:24         ` Burakov, Anatoly
2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 08/11] examples/power: clean up verbose messages David Hunt
2018-09-26 14:25         ` Burakov, Anatoly
2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 09/11] examples/power: add meson/ninja build support David Hunt
2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 10/11] doc/vm_power_manager: add JSON interface API info David Hunt
2018-09-26 14:32         ` Kovacevic, Marko
2018-09-26 13:40       ` [dpdk-dev] [PATCH v4 11/11] examples/power: add json example files David Hunt
2018-09-26 15:58         ` Kovacevic, Marko
2018-09-26 16:14           ` Hunt, David
2018-09-26 16:37       ` [dpdk-dev] [PATCH v5 0/10] add json power policy interface for containers David Hunt
2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 01/10] examples/power: add checks around hypervisor David Hunt
2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 02/10] examples/power: allow for number of vms to be zero David Hunt
2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 03/10] lib/power: add changes for host commands/policies David Hunt
2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 04/10] examples/power: add necessary changes to guest app David Hunt
2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 05/10] examples/power: add host channel to power manager David Hunt
2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 06/10] examples/power: increase allowed number of clients David Hunt
2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 07/10] examples/power: add json string handling David Hunt
2018-09-30  1:54           ` Yao, Lei A
2018-10-01 11:03             ` Hunt, David
2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 08/10] examples/power: clean up verbose messages David Hunt
2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 09/10] examples/power: add meson/ninja build support David Hunt
2018-09-26 16:37         ` [dpdk-dev] [PATCH v5 10/10] doc/vm_power_manager: add JSON interface API info David Hunt
2018-09-29  2:42           ` Yao, Lei A
2018-10-01 11:02             ` Hunt, David
2018-10-02  8:43         ` [dpdk-dev] [PATCH v6 0/10] add json power policy interface for containers David Hunt
2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 01/10] examples/power: add checks around hypervisor David Hunt
2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 02/10] examples/power: allow for number of vms to be zero David Hunt
2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 03/10] lib/power: add changes for host commands/policies David Hunt
2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 04/10] examples/power: add necessary changes to guest app David Hunt
2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 05/10] examples/power: add host channel to power manager David Hunt
2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 06/10] examples/power: increase allowed number of clients David Hunt
2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 07/10] examples/power: add json string handling David Hunt
2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 08/10] examples/power: clean up verbose messages David Hunt
2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 09/10] examples/power: add meson/ninja build support David Hunt
2018-10-02  8:43           ` [dpdk-dev] [PATCH v6 10/10] doc/vm_power_manager: add JSON interface API info David Hunt
2018-10-17 13:05           ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers David Hunt
2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 01/10] examples/power: add checks around hypervisor David Hunt
2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 02/10] examples/power: allow for number of vms to be zero David Hunt
2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 03/10] lib/power: add changes for host commands/policies David Hunt
2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 04/10] examples/power: add necessary changes to guest app David Hunt
2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 05/10] examples/power: add host channel to power manager David Hunt
2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 06/10] examples/power: increase allowed number of clients David Hunt
2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 07/10] examples/power: add json string handling David Hunt
2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 08/10] examples/power: clean up verbose messages David Hunt
2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 09/10] examples/power: add meson/ninja build support David Hunt
2018-10-17 13:05             ` [dpdk-dev] [PATCH v7 10/10] doc/vm_power_manager: add JSON interface API info David Hunt
2018-10-26  0:05               ` Thomas Monjalon
2018-10-26  8:45             ` [dpdk-dev] [PATCH v7 0/10] add json power policy interface for containers Thomas Monjalon

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).