DPDK patches and discussions
 help / color / mirror / Atom feed
From: Jerin Jacob <jerin.jacob@caviumnetworks.com>
To: Harry van Haaren <harry.van.haaren@intel.com>
Cc: dev@dpdk.org, thomas@monjalon.net, keith.wiles@intel.com,
	bruce.richardson@intel.com
Subject: Re: [dpdk-dev] [PATCH v3 1/7] service cores: header and implementation
Date: Tue, 4 Jul 2017 22:46:26 +0530	[thread overview]
Message-ID: <20170704171624.GA2732@jerin> (raw)
In-Reply-To: <1499031314-7172-2-git-send-email-harry.van.haaren@intel.com>

-----Original Message-----
> Date: Sun, 2 Jul 2017 22:35:08 +0100
> From: Harry van Haaren <harry.van.haaren@intel.com>
> To: dev@dpdk.org
> CC: jerin.jacob@caviumnetworks.com, thomas@monjalon.net,
>  keith.wiles@intel.com, bruce.richardson@intel.com, Harry van Haaren
>  <harry.van.haaren@intel.com>
> Subject: [PATCH v3 1/7] service cores: header and implementation
> X-Mailer: git-send-email 2.7.4
> 
> Add header files, update .map files with new service
> functions, and add the service header to the doxygen
> for building.
> 
> This service header API allows DPDK to use services as
> a concept of something that requires CPU cycles. An example
> is a PMD that runs in software to schedule events, where a
> hardware version exists that does not require a CPU.
> 
> The code presented here is based on an initial RFC:
> http://dpdk.org/ml/archives/dev/2017-May/065207.html
> This was then reworked, and RFC v2 with the changes posted:
> http://dpdk.org/ml/archives/dev/2017-June/067194.html
> 
> This is the fourth iteration of the service core concept,
> with 2 RFCs and this being v2 of the implementation.
> 
> Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
> diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile
> index a5bd108..2a93397 100644
> --- a/lib/librte_eal/common/Makefile
> +++ b/lib/librte_eal/common/Makefile
> @@ -41,6 +41,7 @@ INC += rte_eal_memconfig.h rte_malloc_heap.h
>  INC += rte_hexdump.h rte_devargs.h rte_bus.h rte_dev.h rte_vdev.h
>  INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h
>  INC += rte_malloc.h rte_keepalive.h rte_time.h
> +INC += rte_service.h rte_service_private.h

This will install rte_service_private.h file $RTE_TARGET. Based on
earlier email, You don't want to expose rte_service_private.h to
application. Right?
If so, I think, we remove rte_service_private.h from here and add CFLAGS
to point this header in local makefile of the _component_ which interested
in getting eal component specific functions.

>  
>  GENERIC_INC := rte_atomic.h rte_byteorder.h rte_cycles.h rte_prefetch.h
>  GENERIC_INC += rte_spinlock.h rte_memcpy.h rte_cpuflags.h rte_rwlock.h
> diff --git a/lib/librte_eal/common/eal_common_lcore.c b/lib/librte_eal/common/eal_common_lcore.c
> + * called on it.
> + * @retval 0 Service successfully switched off
> + */
> +int32_t rte_service_stop(struct rte_service_spec *service);
> +
> +/** Returns if *service* is currently running.
> + *
> + * This function retuns true if the service has been started using

s/retuns/returns/

> +int32_t rte_service_lcore_reset_all(void);
> +
> +/** Enable or disable statistics collection.
> + *
> + * This function enables per core, per-service cycle count collection.
> + * @param enabled Zero to turn off statistics collection, non-zero to enable.
> + */
> +void rte_service_set_stats_enable(int enabled);

Since it it is a "set" function, better argument is "int enable"


> +
> +/** Retrieve the list of currently enabled service cores.
> + *
> + * This function fills in an application supplied array, with each element
> + * indicating the lcore_id of a service core.
> + *
> + * Adding and removing service cores can be performed using
> + * *rte_service_lcore_add* and *rte_service_lcore_del*.
> + * @param [out] array An array of at least N items.

What is N here. It should be RTE_MAX_LCORE. Right?

> + * @param [out] The size of *array*.
> + * @retval >=0 Number of service cores that have been populated in the array
> + * @retval -ENOMEM The provided array is not large enough to fill in the
> + *          service core list. No items have been populated, call this function
> + *          with a size of at least *rte_service_core_count* items.
> + */
> +int32_t rte_service_lcore_list(uint32_t array[], uint32_t n);
> +
> +	cores_state = rte_calloc("rte_service_core_states", RTE_MAX_LCORE,
> +			sizeof(struct core_state), RTE_CACHE_LINE_SIZE);
> +	if (!cores_state) {
> +		printf("error allocating core states array\n");
> +		return -ENOMEM;
> +	}
> +
> +	int i;
> +	int count = 0;
> +	struct rte_config *cfg = rte_eal_get_configuration();
> +	for (i = 0; i < RTE_MAX_LCORE; i++) {
> +		if (lcore_config[i].core_role == ROLE_SERVICE) {
> +			if ((unsigned)i == cfg->master_lcore)
> +				continue;
> +			rte_service_lcore_add(i);
> +			count++;
> +		}
> +	}
> +
> +	rte_service_library_initialized = 1;
> +	return 0;
> +}
> +
> +void rte_service_set_stats_enable(int enabled)

Since it it is a "set" function, better argument is "int enable"

> +{
> +	uint32_t i;
> +	for (i = 0; i < RTE_MAX_LCORE; i++)
> +		cores_state[i].collect_statistics = enabled;
> +}
> +
> +/* returns 1 if service is registered and has not been unregistered
> + * Returns 0 if service never registered, or has been unregistered
> + */
> +static inline int
> +service_valid(uint32_t id) {
> +	return !!(rte_services[id].internal_flags &
> +		 (1 << SERVICE_F_REGISTERED));
> +}
> +
> +uint32_t
> +rte_service_get_count(void)
> +{
> +	return rte_service_count;
> +}
> +
> +struct rte_service_spec *
> +rte_service_get_by_id(uint32_t id)
> +{
> +	struct rte_service_spec *service = NULL;
> +	if (id < rte_service_count)
> +		service = (struct rte_service_spec *)&rte_services[id];
> +
> +	return service;
> +}
> +
> +struct rte_service_spec *rte_service_get_by_name(const char *name)
> +{
> +	struct rte_service_spec *service = NULL;
> +	int i;
> +	for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) {
> +		if (service_valid(i) &&
> +				strcmp(name, rte_services[i].spec.name) == 0)
> +			service = (struct rte_service_spec *)&rte_services[i];
> +			break;

"break" should be under "if" condition. ie { and } for the "if" condition is
missing


> +	}
> +
> +	return service;
> +}
> +
> +int32_t
> +rte_service_register(const struct rte_service_spec *spec)
> +{
> +	uint32_t i;
> +	int32_t free_slot = -1;
> +
> +	if (spec->callback == NULL || strlen(spec->name) == 0)
> +		return -EINVAL;
> +
> +	for (i = 0; i < RTE_SERVICE_NUM_MAX; i++) {
> +		if (!service_valid(i)) {
> +			free_slot = i;
> +			break;
> +		}
> +	}
> +
> +	if ((free_slot < 0) || (i == RTE_SERVICE_NUM_MAX))
> +		return -ENOSPC;
> +
> +	struct rte_service_spec_impl *s = &rte_services[free_slot];
> +	s->spec = *spec;
> +	s->internal_flags |= (1 << SERVICE_F_REGISTERED);
> +
> +	rte_smp_wmb();
> +	rte_service_count++;

My earlier comments on rte_smp_wmb() was in assumption that
rte_service_register() called be from worker cores.

Can rte_service_register() called from worker threads. No. Right?
if yes then we need an atomic variable to increment rte_service_count.


> +
> +	return 0;
> +}
> +
> +int32_t
> +rte_service_start(struct rte_service_spec *service)
> +{
> +	struct rte_service_spec_impl *s =
> +		(struct rte_service_spec_impl *)service;
> +	s->runstate = RUNSTATE_RUNNING;
> +	rte_smp_wmb();
> +	return 0;
> +}
> +
> +int32_t
> +rte_service_stop(struct rte_service_spec *service)
> +{
> +	struct rte_service_spec_impl *s =
> +		(struct rte_service_spec_impl *)service;
> +	s->runstate = RUNSTATE_STOPPED;
> +	rte_smp_wmb();
> +	return 0;
> +}
> +
> +static int32_t
> +rte_service_runner_func(void *arg)
> +{
> +	RTE_SET_USED(arg);
> +	uint32_t i;
> +	const int lcore = rte_lcore_id();
> +	struct core_state *cs = &cores_state[lcore];

Couple of comments in fastpath code. IMO, better to move "service_mask"
and access to global variable(so that it allocated from local stack) outside the loop.
global variable may share with other frequent write variables.


	const uint64_t service_mask = cs->service_mask;
	const uint32_t __rte_service_count = rte_service_count;

> +
> +	while (cores_state[lcore].runstate == RUNSTATE_RUNNING) {
> +		for (i = 0; i < rte_service_count; i++) {
> +			struct rte_service_spec_impl *s = &rte_services[i];
> +			if (s->runstate != RUNSTATE_RUNNING ||
> +					!(service_mask & (1 << i)))
> +				continue;
> +
> +			/* check if this is the only core mapped, else use
> +			 * atomic to serialize cores mapped to this service
> +			 */
> +			uint32_t *lock = (uint32_t *)&s->execute_lock;
> +			if ((s->spec.capabilities & RTE_SERVICE_CAP_MT_SAFE) ||
> +					(s->num_mapped_cores == 1 ||
> +					rte_atomic32_cmpset(lock, 0, 1))) {
> +				void *userdata = s->spec.callback_userdata;
> +
> +				if (cs->collect_statistics) {
> +					uint64_t start = rte_rdtsc();
> +					s->spec.callback(userdata);
> +					uint64_t end = rte_rdtsc();
> +					s->cycles_spent += end - start;
> +					cs->calls_per_service[i]++;
> +					s->calls++;
> +				} else {
> +					cs->calls_per_service[i]++;

Should we need this  in non stat configuration ?

> +					s->spec.callback(userdata);
> +					s->calls++;

Should we need this  in non stat configuration ?

> +				}
> +
> +				rte_atomic32_clear(&s->execute_lock);

Call this only when "rte_atomic32_cmpset" in action as it is costly for
normal case.

IMO, we need to have rte_smp_rmb() here to update the status of
cores_state[lcore].runstate or s->runstate(as it can be updated from other
lcores)

> +			}
> +		}
> +	}
> +
> +	lcore_config[lcore].state = WAIT;
> +
> +	return 0;
> +}
> +

  reply	other threads:[~2017-07-04 17:16 UTC|newest]

Thread overview: 82+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-06-23  9:06 [dpdk-dev] [PATCH 1/6] " Harry van Haaren
2017-06-23  9:06 ` [dpdk-dev] [PATCH 2/6] service cores: coremask parsing Harry van Haaren
2017-06-26 12:49   ` Jerin Jacob
2017-06-29 11:13     ` Van Haaren, Harry
2017-06-23  9:06 ` [dpdk-dev] [PATCH 3/6] service cores: EAL init changes Harry van Haaren
2017-06-26 12:55   ` Jerin Jacob
2017-06-29 11:13     ` Van Haaren, Harry
2017-06-23  9:06 ` [dpdk-dev] [PATCH 4/6] service cores: mark cores in lcore config as RTE Harry van Haaren
2017-06-23  9:06 ` [dpdk-dev] [PATCH 5/6] service core: add unit tests Harry van Haaren
2017-06-26 13:06   ` Jerin Jacob
2017-06-29 11:14     ` Van Haaren, Harry
2017-06-23  9:06 ` [dpdk-dev] [PATCH 6/6] service cores: enable event/sw with service Harry van Haaren
2017-06-26 13:46   ` Jerin Jacob
2017-06-29 11:15     ` Van Haaren, Harry
2017-06-26 11:59 ` [dpdk-dev] [PATCH 1/6] service cores: header and implementation Jerin Jacob
2017-06-29 11:13   ` Van Haaren, Harry
2017-06-29 11:23 ` [dpdk-dev] [PATCH v2 0/5] service cores: cover letter Harry van Haaren
2017-06-29 11:23   ` [dpdk-dev] [PATCH v2 1/5] service cores: header and implementation Harry van Haaren
2017-06-29 11:23   ` [dpdk-dev] [PATCH v2 2/5] service cores: EAL init changes Harry van Haaren
2017-06-29 11:23   ` [dpdk-dev] [PATCH v2 3/5] service cores: coremask parsing Harry van Haaren
2017-06-29 11:23   ` [dpdk-dev] [PATCH v2 4/5] service cores: add unit tests Harry van Haaren
2017-06-29 11:23   ` [dpdk-dev] [PATCH v2 5/5] service cores: enable event/sw with service Harry van Haaren
2017-07-02 21:35   ` [dpdk-dev] [PATCH v3 0/7] service cores: cover letter Harry van Haaren
2017-07-02 21:35     ` [dpdk-dev] [PATCH v3 1/7] service cores: header and implementation Harry van Haaren
2017-07-04 17:16       ` Jerin Jacob [this message]
2017-07-02 21:35     ` [dpdk-dev] [PATCH v3 2/7] service cores: EAL init changes Harry van Haaren
2017-07-04 11:35       ` Jerin Jacob
2017-07-07 16:28         ` Van Haaren, Harry
2017-07-02 21:35     ` [dpdk-dev] [PATCH v3 3/7] service cores: coremask parsing Harry van Haaren
2017-07-04 12:45       ` Jerin Jacob
2017-07-06 14:47         ` Van Haaren, Harry
2017-07-07 10:45           ` Jerin Jacob
2017-07-07 10:57             ` Van Haaren, Harry
2017-07-02 21:35     ` [dpdk-dev] [PATCH v3 4/7] service cores: add unit tests Harry van Haaren
2017-07-04 11:14       ` Jerin Jacob
2017-07-02 21:35     ` [dpdk-dev] [PATCH v3 5/7] service cores: enable event/sw with service Harry van Haaren
2017-07-04 10:52       ` Jerin Jacob
2017-07-07 16:28         ` Van Haaren, Harry
2017-07-02 21:35     ` [dpdk-dev] [PATCH v3 6/7] maintainers: claim service cores Harry van Haaren
2017-07-04 10:53       ` Jerin Jacob
2017-07-02 21:35     ` [dpdk-dev] [PATCH v3 7/7] doc: add service cores to doc and release notes Harry van Haaren
2017-07-02 22:16       ` Mcnamara, John
2017-07-04 10:56       ` Jerin Jacob
2017-07-07 16:41   ` [dpdk-dev] [PATCH v4 0/7] service cores: cover letter Harry van Haaren
2017-07-07 16:41     ` [dpdk-dev] [PATCH v4 1/7] service cores: header and implementation Harry van Haaren
2017-07-11  8:29       ` Jerin Jacob
2017-07-11  9:54         ` Thomas Monjalon
2017-07-11 12:32           ` Van Haaren, Harry
2017-07-11 12:44             ` Jerin Jacob
2017-07-11 12:49               ` Van Haaren, Harry
2017-07-11 14:10         ` Van Haaren, Harry
2017-07-07 16:41     ` [dpdk-dev] [PATCH v4 2/7] service cores: EAL init changes Harry van Haaren
2017-07-11  7:42       ` Jerin Jacob
2017-07-11 14:11         ` Van Haaren, Harry
2017-07-07 16:41     ` [dpdk-dev] [PATCH v4 3/7] service cores: coremask parsing Harry van Haaren
2017-07-07 16:41     ` [dpdk-dev] [PATCH v4 4/7] service cores: add unit tests Harry van Haaren
2017-07-11  8:12       ` Jerin Jacob
2017-07-07 16:41     ` [dpdk-dev] [PATCH v4 5/7] event/sw: enable SW PMD with service capability Harry van Haaren
2017-07-07 16:41     ` [dpdk-dev] [PATCH v4 6/7] doc: add service cores to doc and release notes Harry van Haaren
2017-07-07 16:41     ` [dpdk-dev] [PATCH v4 7/7] maintainers: claim service cores Harry van Haaren
2017-07-11  7:53       ` Jerin Jacob
2017-07-09 22:08     ` [dpdk-dev] [PATCH v4 0/7] service cores: cover letter Thomas Monjalon
2017-07-10  8:18       ` Van Haaren, Harry
2017-07-10 11:41         ` Jerin Jacob
2017-07-11 14:19     ` [dpdk-dev] [PATCH v5 " Harry van Haaren
2017-07-11 14:19       ` [dpdk-dev] [PATCH v5 1/7] service cores: header and implementation Harry van Haaren
2017-07-12 16:35         ` Jerin Jacob
2017-07-11 14:19       ` [dpdk-dev] [PATCH v5 2/7] service cores: EAL init changes Harry van Haaren
2017-07-11 14:19       ` [dpdk-dev] [PATCH v5 3/7] service cores: coremask parsing Harry van Haaren
2017-07-11 14:19       ` [dpdk-dev] [PATCH v5 4/7] service cores: add unit tests Harry van Haaren
2017-07-11 14:19       ` [dpdk-dev] [PATCH v5 5/7] event/sw: enable SW PMD with service capability Harry van Haaren
2017-07-11 14:19       ` [dpdk-dev] [PATCH v5 6/7] doc: add service cores to doc and release notes Harry van Haaren
2017-07-11 14:19       ` [dpdk-dev] [PATCH v5 7/7] maintainers: claim service cores Harry van Haaren
2017-07-12 16:49       ` [dpdk-dev] [PATCH v5 0/7] service cores: cover letter Jerin Jacob
2017-07-16 19:25       ` Thomas Monjalon
2017-07-17  8:07         ` Van Haaren, Harry
2017-07-17 15:21         ` [dpdk-dev] [PATCH] service: add corelist to EAL arguments Harry van Haaren
2017-07-17 15:53           ` Ananyev, Konstantin
2017-07-17 15:58             ` Van Haaren, Harry
2017-07-17 16:10               ` Ananyev, Konstantin
2017-07-17 16:16                 ` Van Haaren, Harry
2017-07-19  5:42           ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170704171624.GA2732@jerin \
    --to=jerin.jacob@caviumnetworks.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=harry.van.haaren@intel.com \
    --cc=keith.wiles@intel.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).