* [dpdk-dev] [PATCH 0/9] event/octeontx: add event timer adapter driver
@ 2018-02-16 21:36 Pavan Nikhilesh
2018-02-16 21:36 ` [dpdk-dev] [PATCH 01/10] eal: add API to align variable to previous power of 2 Pavan Nikhilesh
` (12 more replies)
0 siblings, 13 replies; 76+ messages in thread
From: Pavan Nikhilesh @ 2018-02-16 21:36 UTC (permalink / raw)
To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh
The event timer adapter[1] provides APIs to configure an event timer device
that allows an application to arm timers which on expiry push events to an
event device such as OcteonTx SSO.
The OcteonTx TIM is a co-processor that can be configured as an event timer
adapter which can be used by an application to manage event timers.
The TIM co-processor processes the event timers registered and pushes
expired event timers to SSO based on the event queue, schedule type, flow
id etc. provided as rte_event while arming the event timer. It maintains
event timers with high precision and time granularity of 1us (microsecond).
[1] http://dpdk.org/dev/patchwork/patch/33525/
Pavan Nikhilesh (9):
eal: add API to align variable to previous power of 2
mempool/octeontx: probe timvf PCIe devices
event/octeontx: add support to create and free timer adapter
event/octeontx: add support to start and stop timer device
event/octeontx: add support for arm and cancel
event/octeontx: add single producer timer arm variant
event/octeontx: optimize timer adapter resolution parameters
event/octeontx: add option to use fpavf as chunk pool
event/octeontx: add timer adapter SW traversal routine
config/common_base | 2 +
drivers/event/octeontx/Makefile | 6 +
drivers/event/octeontx/meson.build | 4 +-
drivers/event/octeontx/ssovf_evdev.c | 3 +
drivers/event/octeontx/timvf_evdev.c | 393 ++++++++++++++
drivers/event/octeontx/timvf_evdev.h | 208 +++++++
drivers/event/octeontx/timvf_worker.c | 118 ++++
drivers/event/octeontx/timvf_worker.h | 601 +++++++++++++++++++++
drivers/mempool/octeontx/Makefile | 1 +
drivers/mempool/octeontx/meson.build | 1 +
drivers/mempool/octeontx/octeontx_mbox.h | 7 +
drivers/mempool/octeontx/octeontx_timvf.c | 145 +++++
.../octeontx/rte_mempool_octeontx_version.map | 3 +
lib/librte_eal/common/include/rte_common.h | 36 ++
usertools/dpdk-devbind.py | 8 +
15 files changed, 1535 insertions(+), 1 deletion(-)
create mode 100644 drivers/event/octeontx/timvf_evdev.c
create mode 100644 drivers/event/octeontx/timvf_evdev.h
create mode 100644 drivers/event/octeontx/timvf_worker.c
create mode 100644 drivers/event/octeontx/timvf_worker.h
create mode 100644 drivers/mempool/octeontx/octeontx_timvf.c
--
2.16.1
^ permalink raw reply [flat|nested] 76+ messages in thread* [dpdk-dev] [PATCH 01/10] eal: add API to align variable to previous power of 2 2018-02-16 21:36 [dpdk-dev] [PATCH 0/9] event/octeontx: add event timer adapter driver Pavan Nikhilesh @ 2018-02-16 21:36 ` Pavan Nikhilesh 2018-02-17 4:49 ` Jerin Jacob 2018-02-16 21:36 ` [dpdk-dev] [PATCH 02/10] mempool/octeontx: probe timvf PCIe devices Pavan Nikhilesh ` (11 subsequent siblings) 12 siblings, 1 reply; 76+ messages in thread From: Pavan Nikhilesh @ 2018-02-16 21:36 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Add 32b and 64b API's to align the given variable to the previous power of 2. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- lib/librte_eal/common/include/rte_common.h | 36 ++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/lib/librte_eal/common/include/rte_common.h b/lib/librte_eal/common/include/rte_common.h index c7803e41c..126914f07 100644 --- a/lib/librte_eal/common/include/rte_common.h +++ b/lib/librte_eal/common/include/rte_common.h @@ -259,6 +259,24 @@ rte_align32pow2(uint32_t x) return x + 1; } +/** + * Aligns input parameter to the previous power of 2 + * + * @param x + * The integer value to algin + * + * @return + * Input parameter aligned to the previous power of 2 + */ +static inline uint32_t +rte_align32lowpow2(uint32_t x) +{ + x = rte_align32pow2(x); + x--; + + return x - (x >> 1); +} + /** * Aligns 64b input parameter to the next power of 2 * @@ -282,6 +300,24 @@ rte_align64pow2(uint64_t v) return v + 1; } +/** + * Aligns 64b input parameter to the previous power of 2 + * + * @param v + * The 64b value to align + * + * @return + * Input parameter aligned to the previous power of 2 + */ +static inline uint64_t +rte_align64lowpow2(uint64_t v) +{ + v = rte_align64pow2(v); + v--; + + return v - (v >> 1); +} + /*********** Macros for calculating min and max **********/ /** -- 2.16.1 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH 01/10] eal: add API to align variable to previous power of 2 2018-02-16 21:36 ` [dpdk-dev] [PATCH 01/10] eal: add API to align variable to previous power of 2 Pavan Nikhilesh @ 2018-02-17 4:49 ` Jerin Jacob 0 siblings, 0 replies; 76+ messages in thread From: Jerin Jacob @ 2018-02-17 4:49 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Sat, 17 Feb 2018 03:06:51 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH 01/10] eal: add API to align variable to > previous power of 2 > X-Mailer: git-send-email 2.16.1 > > Add 32b and 64b API's to align the given variable to the previous power > of 2. I suggest to send common code changes in separate patch series with unit test case and mention the dependency in cover letter. > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > lib/librte_eal/common/include/rte_common.h | 36 ++++++++++++++++++++++++++++++ > 1 file changed, 36 insertions(+) > > diff --git a/lib/librte_eal/common/include/rte_common.h b/lib/librte_eal/common/include/rte_common.h > index c7803e41c..126914f07 100644 > --- a/lib/librte_eal/common/include/rte_common.h > +++ b/lib/librte_eal/common/include/rte_common.h > @@ -259,6 +259,24 @@ rte_align32pow2(uint32_t x) > return x + 1; > } > > +/** > + * Aligns input parameter to the previous power of 2 > + * > + * @param x > + * The integer value to algin > + * > + * @return > + * Input parameter aligned to the previous power of 2 > + */ > +static inline uint32_t > +rte_align32lowpow2(uint32_t x) > +{ > + x = rte_align32pow2(x); > + x--; > + > + return x - (x >> 1); > +} > + > /** > * Aligns 64b input parameter to the next power of 2 > * > @@ -282,6 +300,24 @@ rte_align64pow2(uint64_t v) > return v + 1; > } > > +/** > + * Aligns 64b input parameter to the previous power of 2 > + * > + * @param v > + * The 64b value to align > + * > + * @return > + * Input parameter aligned to the previous power of 2 > + */ > +static inline uint64_t > +rte_align64lowpow2(uint64_t v) > +{ > + v = rte_align64pow2(v); > + v--; > + > + return v - (v >> 1); > +} > + > /*********** Macros for calculating min and max **********/ > > /** > -- > 2.16.1 > ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH 02/10] mempool/octeontx: probe timvf PCIe devices 2018-02-16 21:36 [dpdk-dev] [PATCH 0/9] event/octeontx: add event timer adapter driver Pavan Nikhilesh 2018-02-16 21:36 ` [dpdk-dev] [PATCH 01/10] eal: add API to align variable to previous power of 2 Pavan Nikhilesh @ 2018-02-16 21:36 ` Pavan Nikhilesh 2018-02-17 4:54 ` Jerin Jacob 2018-02-16 21:36 ` [dpdk-dev] [PATCH 03/10] event/octeontx: add support to create and free timer adapter Pavan Nikhilesh ` (10 subsequent siblings) 12 siblings, 1 reply; 76+ messages in thread From: Pavan Nikhilesh @ 2018-02-16 21:36 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh On Octeontx HW, each event timer device is enumerated as separate SRIOV VF PCIe device. In order to expose as a event timer device: On PCIe probe, the driver stores the information associated with the PCIe device and later when appliacation requests for a event timer device through `rte_event_timer_adapter_create` the driver infrastructure creates the timer adapter with earlier probed PCIe VF devices. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/mempool/octeontx/Makefile | 1 + drivers/mempool/octeontx/meson.build | 1 + drivers/mempool/octeontx/octeontx_mbox.h | 7 + drivers/mempool/octeontx/octeontx_timvf.c | 145 +++++++++++++++++++++ .../octeontx/rte_mempool_octeontx_version.map | 3 + usertools/dpdk-devbind.py | 8 ++ 6 files changed, 165 insertions(+) create mode 100644 drivers/mempool/octeontx/octeontx_timvf.c diff --git a/drivers/mempool/octeontx/Makefile b/drivers/mempool/octeontx/Makefile index dfc373e62..a0547437a 100644 --- a/drivers/mempool/octeontx/Makefile +++ b/drivers/mempool/octeontx/Makefile @@ -20,6 +20,7 @@ LIBABIVER := 1 SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += octeontx_ssovf.c SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += octeontx_mbox.c SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += octeontx_fpavf.c +SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += octeontx_timvf.c SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += rte_mempool_octeontx.c ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) diff --git a/drivers/mempool/octeontx/meson.build b/drivers/mempool/octeontx/meson.build index 1e894a56b..c938436c9 100644 --- a/drivers/mempool/octeontx/meson.build +++ b/drivers/mempool/octeontx/meson.build @@ -4,6 +4,7 @@ sources = files('octeontx_ssovf.c', 'octeontx_mbox.c', 'octeontx_fpavf.c', + 'octeontx_timvf.c', 'rte_mempool_octeontx.c' ) diff --git a/drivers/mempool/octeontx/octeontx_mbox.h b/drivers/mempool/octeontx/octeontx_mbox.h index 1b056071a..7fcc9950f 100644 --- a/drivers/mempool/octeontx/octeontx_mbox.h +++ b/drivers/mempool/octeontx/octeontx_mbox.h @@ -21,6 +21,11 @@ enum octeontx_ssovf_type { OCTEONTX_SSO_HWS, /* SSO hardware workslot vf */ }; +struct octeontx_timvf_info { + uint16_t domain; /* Domain id */ + uint8_t total_timvfs; /* Total timvf available in domain */ +}; + struct octeontx_mbox_hdr { uint16_t vfid; /* VF index or pf resource index local to the domain */ uint8_t coproc; /* Coprocessor id */ @@ -32,5 +37,7 @@ int octeontx_ssovf_info(struct octeontx_ssovf_info *info); void *octeontx_ssovf_bar(enum octeontx_ssovf_type, uint8_t id, uint8_t bar); int octeontx_ssovf_mbox_send(struct octeontx_mbox_hdr *hdr, void *txdata, uint16_t txlen, void *rxdata, uint16_t rxlen); +int octeontx_timvf_info(struct octeontx_timvf_info *info); +void *octeontx_timvf_bar(uint8_t id, uint8_t bar); #endif /* __OCTEONTX_MBOX_H__ */ diff --git a/drivers/mempool/octeontx/octeontx_timvf.c b/drivers/mempool/octeontx/octeontx_timvf.c new file mode 100644 index 000000000..b20743a33 --- /dev/null +++ b/drivers/mempool/octeontx/octeontx_timvf.c @@ -0,0 +1,145 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include <rte_eal.h> +#include <rte_io.h> +#include <rte_pci.h> +#include <rte_bus_pci.h> + +#include "octeontx_mbox.h" +#include "octeontx_pool_logs.h" + +#ifndef PCI_VENDOR_ID_CAVIUM +#define PCI_VENDOR_ID_CAVIUM (0x177D) +#endif + +#define PCI_DEVICE_ID_OCTEONTX_TIM_VF (0xA051) +#define TIM_MAX_RINGS (64) + +struct timvf_res { + uint16_t domain; + uint16_t vfid; + void *bar0; + void *bar2; + void *bar4; +}; + +struct timdev { + uint8_t total_timvfs; + struct timvf_res rings[TIM_MAX_RINGS]; +}; + +static struct timdev tdev; + +int +octeontx_timvf_info(struct octeontx_timvf_info *tinfo) +{ + int i; + struct octeontx_ssovf_info info; + + if (tinfo == NULL) + return -EINVAL; + + if (!tdev.total_timvfs) + return -ENODEV; + + if (octeontx_ssovf_info(&info) < 0) + return -EINVAL; + + for (i = 0; i < tdev.total_timvfs; i++) { + if (info.domain != tdev.rings[i].domain) { + mbox_log_err("GRP error, vfid=%d/%d domain=%d/%d %p", + i, tdev.rings[i].vfid, + info.domain, tdev.rings[i].domain, + tdev.rings[i].bar0); + return -EINVAL; + } + } + + tinfo->total_timvfs = tdev.total_timvfs; + tinfo->domain = info.domain; + return 0; +} + +void* +octeontx_timvf_bar(uint8_t id, uint8_t bar) +{ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return NULL; + + if (id > tdev.total_timvfs) + return NULL; + + switch (bar) { + case 0: + return tdev.rings[id].bar0; + case 4: + return tdev.rings[id].bar4; + default: + return NULL; + } +} + +static int +timvf_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + uint64_t val; + uint16_t vfid; + struct timvf_res *res; + + RTE_SET_USED(pci_drv); + + /* For secondary processes, the primary has done all the work */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + if (pci_dev->mem_resource[0].addr == NULL || + pci_dev->mem_resource[4].addr == NULL) { + mbox_log_err("Empty bars %p %p", + pci_dev->mem_resource[0].addr, + pci_dev->mem_resource[4].addr); + return -ENODEV; + } + + val = rte_read64((uint8_t *)pci_dev->mem_resource[0].addr + 0x100); + vfid = (val >> 23) & 0xff; + if (vfid >= TIM_MAX_RINGS) { + mbox_log_err("Invalid vfid(%d/%d)", vfid, TIM_MAX_RINGS); + return -EINVAL; + } + + res = &tdev.rings[tdev.total_timvfs]; + res->vfid = vfid; + res->bar0 = pci_dev->mem_resource[0].addr; + res->bar2 = pci_dev->mem_resource[2].addr; + res->bar4 = pci_dev->mem_resource[4].addr; + res->domain = (val >> 7) & 0xffff; + tdev.total_timvfs++; + rte_wmb(); + + mbox_log_dbg("Domain=%d VFid=%d bar0 %p total_timvfs=%d", res->domain, + res->vfid, pci_dev->mem_resource[0].addr, + tdev.total_timvfs); + return 0; +} + + +static const struct rte_pci_id pci_timvf_map[] = { + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, + PCI_DEVICE_ID_OCTEONTX_TIM_VF) + }, + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver pci_timvf = { + .id_table = pci_timvf_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_IOVA_AS_VA, + .probe = timvf_probe, + .remove = NULL, +}; + +RTE_PMD_REGISTER_PCI(octeontx_timvf, pci_timvf); diff --git a/drivers/mempool/octeontx/rte_mempool_octeontx_version.map b/drivers/mempool/octeontx/rte_mempool_octeontx_version.map index fe8cdeca0..91e0a3a93 100644 --- a/drivers/mempool/octeontx/rte_mempool_octeontx_version.map +++ b/drivers/mempool/octeontx/rte_mempool_octeontx_version.map @@ -5,5 +5,8 @@ DPDK_17.11 { octeontx_ssovf_bar; octeontx_ssovf_mbox_send; + octeontx_timvf_info; + octeontx_timvf_bar; + local: *; }; diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py index 18d938607..340643b70 100755 --- a/usertools/dpdk-devbind.py +++ b/usertools/dpdk-devbind.py @@ -22,11 +22,14 @@ 'SVendor': None, 'SDevice': None} cavium_pkx = {'Class': '08', 'Vendor': '177d', 'Device': 'a0dd,a049', 'SVendor': None, 'SDevice': None} +cavium_tim = {'Class': '08', 'Vendor': '177d', 'Device': 'a051', + 'SVendor': None, 'SDevice': None} network_devices = [network_class, cavium_pkx] crypto_devices = [encryption_class, intel_processor_class] eventdev_devices = [cavium_sso] mempool_devices = [cavium_fpa] +eventtimer_devices = [cavium_tim] # global dict ethernet devices present. Dictionary indexed by PCI address. # Each device within this is itself a dictionary of device properties @@ -565,6 +568,9 @@ def show_status(): if status_dev == "mempool" or status_dev == "all": show_device_status(mempool_devices, "Mempool") + if status_dev == "event_timer" or status_dev == "all": + show_device_status(eventtimer_devices, "Event Timer") + def parse_args(): '''Parses the command-line arguments given by the user and takes the appropriate action for each''' @@ -638,6 +644,7 @@ def do_arg_actions(): get_device_details(crypto_devices) get_device_details(eventdev_devices) get_device_details(mempool_devices) + get_device_details(eventtimer_devices) show_status() @@ -650,6 +657,7 @@ def main(): get_device_details(crypto_devices) get_device_details(eventdev_devices) get_device_details(mempool_devices) + get_device_details(eventtimer_devices) do_arg_actions() if __name__ == "__main__": -- 2.16.1 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH 02/10] mempool/octeontx: probe timvf PCIe devices 2018-02-16 21:36 ` [dpdk-dev] [PATCH 02/10] mempool/octeontx: probe timvf PCIe devices Pavan Nikhilesh @ 2018-02-17 4:54 ` Jerin Jacob 2018-02-19 9:19 ` Pavan Nikhilesh 0 siblings, 1 reply; 76+ messages in thread From: Jerin Jacob @ 2018-02-17 4:54 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Sat, 17 Feb 2018 03:06:52 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH 02/10] mempool/octeontx: probe timvf PCIe devices > X-Mailer: git-send-email 2.16.1 > > On Octeontx HW, each event timer device is enumerated as separate SRIOV VF > PCIe device. > > In order to expose as a event timer device: > On PCIe probe, the driver stores the information associated with the > PCIe device and later when appliacation requests for a event timer device > through `rte_event_timer_adapter_create` the driver infrastructure creates > the timer adapter with earlier probed PCIe VF devices. > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > drivers/mempool/octeontx/Makefile | 1 + > drivers/mempool/octeontx/meson.build | 1 + > drivers/mempool/octeontx/octeontx_mbox.h | 7 + > drivers/mempool/octeontx/octeontx_timvf.c | 145 +++++++++++++++++++++ > .../octeontx/rte_mempool_octeontx_version.map | 3 + > usertools/dpdk-devbind.py | 8 ++ I suggest to have separate patch for usertools/dpdk-devbind.py common code change. > diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py > index 18d938607..340643b70 100755 > --- a/usertools/dpdk-devbind.py > +++ b/usertools/dpdk-devbind.py > @@ -22,11 +22,14 @@ > 'SVendor': None, 'SDevice': None} > cavium_pkx = {'Class': '08', 'Vendor': '177d', 'Device': 'a0dd,a049', > 'SVendor': None, 'SDevice': None} > +cavium_tim = {'Class': '08', 'Vendor': '177d', 'Device': 'a051', > + 'SVendor': None, 'SDevice': None} > > network_devices = [network_class, cavium_pkx] > crypto_devices = [encryption_class, intel_processor_class] > eventdev_devices = [cavium_sso] > mempool_devices = [cavium_fpa] > +eventtimer_devices = [cavium_tim] In order to reduce number of different type of device, IMO, we could group this also as "eventdev_devices" as it comes as sub device of eventdev. ie. eventdev_devices = [cavium_sso, cavium_tim] ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH 02/10] mempool/octeontx: probe timvf PCIe devices 2018-02-17 4:54 ` Jerin Jacob @ 2018-02-19 9:19 ` Pavan Nikhilesh 0 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-02-19 9:19 UTC (permalink / raw) To: Jerin Jacob, santosh.shukla, erik.g.carrillo; +Cc: dev On Sat, Feb 17, 2018 at 10:24:07AM +0530, Jerin Jacob wrote: > -----Original Message----- > > Date: Sat, 17 Feb 2018 03:06:52 +0530 > > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > > erik.g.carrillo@intel.com > > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > > Subject: [dpdk-dev] [PATCH 02/10] mempool/octeontx: probe timvf PCIe devices > > X-Mailer: git-send-email 2.16.1 > > > > On Octeontx HW, each event timer device is enumerated as separate SRIOV VF > > PCIe device. > > > > In order to expose as a event timer device: > > On PCIe probe, the driver stores the information associated with the > > PCIe device and later when appliacation requests for a event timer device > > through `rte_event_timer_adapter_create` the driver infrastructure creates > > the timer adapter with earlier probed PCIe VF devices. > > > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > > --- > > drivers/mempool/octeontx/Makefile | 1 + > > drivers/mempool/octeontx/meson.build | 1 + > > drivers/mempool/octeontx/octeontx_mbox.h | 7 + > > drivers/mempool/octeontx/octeontx_timvf.c | 145 +++++++++++++++++++++ > > .../octeontx/rte_mempool_octeontx_version.map | 3 + > > usertools/dpdk-devbind.py | 8 ++ > > I suggest to have separate patch for usertools/dpdk-devbind.py > common code change. > > > diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py > > index 18d938607..340643b70 100755 > > --- a/usertools/dpdk-devbind.py > > +++ b/usertools/dpdk-devbind.py > > @@ -22,11 +22,14 @@ > > 'SVendor': None, 'SDevice': None} > > cavium_pkx = {'Class': '08', 'Vendor': '177d', 'Device': 'a0dd,a049', > > 'SVendor': None, 'SDevice': None} > > +cavium_tim = {'Class': '08', 'Vendor': '177d', 'Device': 'a051', > > + 'SVendor': None, 'SDevice': None} > > > > network_devices = [network_class, cavium_pkx] > > crypto_devices = [encryption_class, intel_processor_class] > > eventdev_devices = [cavium_sso] > > mempool_devices = [cavium_fpa] > > +eventtimer_devices = [cavium_tim] > > In order to reduce number of different type of device, IMO, we could > group this also as "eventdev_devices" as it comes as sub device of > eventdev. > > ie. > eventdev_devices = [cavium_sso, cavium_tim] Agreed, will do the specified changes and send it as a sperate patch. Thanks, Pavan. ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH 03/10] event/octeontx: add support to create and free timer adapter 2018-02-16 21:36 [dpdk-dev] [PATCH 0/9] event/octeontx: add event timer adapter driver Pavan Nikhilesh 2018-02-16 21:36 ` [dpdk-dev] [PATCH 01/10] eal: add API to align variable to previous power of 2 Pavan Nikhilesh 2018-02-16 21:36 ` [dpdk-dev] [PATCH 02/10] mempool/octeontx: probe timvf PCIe devices Pavan Nikhilesh @ 2018-02-16 21:36 ` Pavan Nikhilesh 2018-02-16 21:36 ` [dpdk-dev] [PATCH 04/10] event/octeontx: add support to start and stop timer device Pavan Nikhilesh ` (9 subsequent siblings) 12 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-02-16 21:36 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh When the application requests to create a timer device, Octeontx TIM create does the following: - Get the requested TIMvf ring based on adapter_id. - Verify the config parameters supplied. - Allocate memory required for * Buckets based on min and max timeout supplied. * Allocate the chunk pool based on the number of timers. - Clear the interrupts. On Free: - Free the allocated bucket and chunk memory. - Free private data used by TIMvf. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/Makefile | 1 + drivers/event/octeontx/meson.build | 3 +- drivers/event/octeontx/ssovf_evdev.c | 3 + drivers/event/octeontx/timvf_evdev.c | 160 +++++++++++++++++++++++++++++++++++ drivers/event/octeontx/timvf_evdev.h | 158 ++++++++++++++++++++++++++++++++++ 5 files changed, 324 insertions(+), 1 deletion(-) create mode 100644 drivers/event/octeontx/timvf_evdev.c create mode 100644 drivers/event/octeontx/timvf_evdev.h diff --git a/drivers/event/octeontx/Makefile b/drivers/event/octeontx/Makefile index 0e49efd84..f1d10a99e 100644 --- a/drivers/event/octeontx/Makefile +++ b/drivers/event/octeontx/Makefile @@ -27,6 +27,7 @@ LIBABIVER := 1 SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_worker.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_evdev.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_evdev_selftest.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += timvf_evdev.c ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_ssovf_worker.o += -fno-prefetch-loop-arrays diff --git a/drivers/event/octeontx/meson.build b/drivers/event/octeontx/meson.build index 358fc9fc9..8941f8a56 100644 --- a/drivers/event/octeontx/meson.build +++ b/drivers/event/octeontx/meson.build @@ -3,7 +3,8 @@ sources = files('ssovf_worker.c', 'ssovf_evdev.c', - 'ssovf_evdev_selftest.c' + 'ssovf_evdev_selftest.c', + 'timvf_evdev.c', ) deps += ['mempool_octeontx', 'bus_vdev', 'pmd_octeontx'] diff --git a/drivers/event/octeontx/ssovf_evdev.c b/drivers/event/octeontx/ssovf_evdev.c index a1086077d..54384d465 100644 --- a/drivers/event/octeontx/ssovf_evdev.c +++ b/drivers/event/octeontx/ssovf_evdev.c @@ -18,6 +18,7 @@ #include <rte_bus_vdev.h> #include "ssovf_evdev.h" +#include "timvf_evdev.h" int otx_logtype_ssovf; @@ -610,6 +611,8 @@ static const struct rte_eventdev_ops ssovf_ops = { .eth_rx_adapter_start = ssovf_eth_rx_adapter_start, .eth_rx_adapter_stop = ssovf_eth_rx_adapter_stop, + .timer_adapter_caps_get = timvf_timer_adapter_caps_get, + .dev_selftest = test_eventdev_octeontx, .dump = ssovf_dump, diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c new file mode 100644 index 000000000..a56ca7e71 --- /dev/null +++ b/drivers/event/octeontx/timvf_evdev.c @@ -0,0 +1,160 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include "timvf_evdev.h" + +int otx_logtype_timvf; + +RTE_INIT(otx_timvf_init_log); +static void +otx_timvf_init_log(void) +{ + otx_logtype_timvf = rte_log_register("pmd.event.octeontx.timer"); + if (otx_logtype_timvf >= 0) + rte_log_set_level(otx_logtype_timvf, RTE_LOG_NOTICE); +} + +static void +timvf_ring_info_get(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer_adapter_info *adptr_info) +{ + struct timvf_ring *timr = adptr->data->adapter_priv; + adptr_info->max_tmo_ns = timr->max_tout; + adptr_info->min_resolution_ns = timr->tck_nsec; + rte_memcpy(&adptr_info->conf, &adptr->data->conf, + sizeof(struct rte_event_timer_adapter_conf)); +} + +static int +timvf_ring_create(struct rte_event_timer_adapter *adptr) +{ + char pool_name[25]; + int ret; + uint64_t nb_timers; + struct rte_event_timer_adapter_conf *rcfg = &adptr->data->conf; + struct timvf_ring *timr; + struct octeontx_timvf_info tinfo; + + if (octeontx_timvf_info(&tinfo) < 0) + return -ENODEV; + + if (adptr->data->id >= tinfo.total_timvfs) + return -ENODEV; + + timr = rte_zmalloc("octeontx_timvf_priv", + sizeof(struct timvf_ring), 0); + if (timr == NULL) + return -ENOMEM; + + adptr->data->adapter_priv = timr; + /* Check config parameters. */ + if ((rcfg->clk_src != RTE_EVENT_TIMER_ADAPTER_EXT_CLK0) && + (!rcfg->timer_tick_ns || + rcfg->timer_tick_ns < TIM_MIN_INTERVAL)) { + timvf_log_err("Too low timer ticks"); + goto cfg_err; + } + + switch (rcfg->clk_src) { + case RTE_EVENT_TIMER_ADAPTER_CPU_CLK: + timr->clk_src = TIM_CLK_SRC_SCLK; + break; + case RTE_EVENT_TIMER_ADAPTER_EXT_CLK0: + timr->clk_src = TIM_CLK_SRC_GPIO; + break; + case RTE_EVENT_TIMER_ADAPTER_EXT_CLK1: + timr->clk_src = TIM_CLK_SRC_GTI; + break; + case RTE_EVENT_TIMER_ADAPTER_EXT_CLK2: + timr->clk_src = TIM_CLK_SRC_PTP; + break; + default: + timvf_log_err("Invalid clk source specified."); + goto cfg_err; + } + + timr->tim_ring_id = adptr->data->id; + timr->tck_nsec = rcfg->timer_tick_ns; + timr->max_tout = rcfg->max_tmo_ns; + timr->meta.nb_bkts = (timr->max_tout / timr->tck_nsec) + 1; + timr->vbar0 = octeontx_timvf_bar(timr->tim_ring_id, 0); + timr->bkt_pos = (uint8_t *)timr->vbar0 + TIM_VRING_REL; + nb_timers = rcfg->nb_timers; + timr->meta.get_target_bkt = bkt_mod; + + timr->nb_chunks = nb_timers / nb_chunk_slots; + + timr->meta.bkt = rte_zmalloc("octeontx_timvf_bucket", + (timr->meta.nb_bkts) * sizeof(struct tim_mem_bucket), + 0); + if (timr->meta.bkt == NULL) + goto mem_err; + + snprintf(pool_name, 30, "timvf_meta.chunk_pool%d", timr->tim_ring_id); + timr->meta.chunk_pool = (void *)rte_mempool_create_empty(pool_name, + timr->nb_chunks, TIM_CHUNK_SIZE, 0, 0, rte_socket_id(), + 0); + + if (!timr->meta.chunk_pool) { + rte_free(timr->meta.bkt); + timvf_log_err("Unable to create chunkpool."); + return -ENOMEM; + } + + ret = rte_mempool_set_ops_byname(timr->meta.chunk_pool, + RTE_MBUF_DEFAULT_MEMPOOL_OPS, NULL); + timvf_log_dbg("Not giving back chunks to fpa"); + + if (ret != 0) { + timvf_log_err("Unable to set chunkpool ops."); + goto mem_err; + } + + ret = rte_mempool_populate_default(timr->meta.chunk_pool); + if (ret < 0) { + timvf_log_err("Unable to set populate chunkpool."); + goto mem_err; + } + timvf_write64(0, (uint8_t *)timr->vbar0 + TIM_VRING_BASE); + timvf_write64(0, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_INT); + timvf_write64(0, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_INT_W1S); + timvf_write64(0x7, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_ENA_W1C); + timvf_write64(0x7, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_ENA_W1S); + + return 0; +mem_err: + rte_free(timr); + return -ENOMEM; +cfg_err: + rte_free(timr); + return -EINVAL; +} + +static int +timvf_ring_free(struct rte_event_timer_adapter *adptr) +{ + struct timvf_ring *timr = adptr->data->adapter_priv; + rte_mempool_free(timr->meta.chunk_pool); + rte_free(timr->meta.bkt); + rte_free(adptr->data->adapter_priv); + return 0; +} + +static struct rte_event_timer_adapter_ops timvf_ops = { + .init = timvf_ring_create, + .uninit = timvf_ring_free, + .get_info = timvf_ring_info_get, +}; + +int +timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, + uint32_t *caps, const struct rte_event_timer_adapter_ops **ops) +{ + RTE_SET_USED(dev); + RTE_SET_USED(flags); + *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; + *ops = &timvf_ops; + return -EINVAL; +} diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h new file mode 100644 index 000000000..fcc938b82 --- /dev/null +++ b/drivers/event/octeontx/timvf_evdev.h @@ -0,0 +1,158 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#ifndef __TIMVF_EVDEV_H__ +#define __TIMVF_EVDEV_H__ + +#include <rte_common.h> +#include <rte_cycles.h> +#include <rte_debug.h> +#include <rte_eal.h> +#include <rte_eventdev.h> +#include <rte_event_timer_adapter.h> +#include <rte_event_timer_adapter_pmd.h> +#include <rte_io.h> +#include <rte_lcore.h> +#include <rte_log.h> +#include <rte_malloc.h> +#include <rte_mempool.h> +#include <rte_memzone.h> +#include <rte_pci.h> +#include <rte_prefetch.h> +#include <rte_reciprocal.h> + +#include <octeontx_mbox.h> + +#define timvf_log(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, otx_logtype_timvf, \ + "[%s] %s() " fmt "\n", \ + RTE_STR(event_timer_octeontx), __func__, ## args) + +#define timvf_log_info(fmt, ...) timvf_log(INFO, fmt, ##__VA_ARGS__) +#define timvf_log_dbg(fmt, ...) timvf_log(DEBUG, fmt, ##__VA_ARGS__) +#define timvf_log_err(fmt, ...) timvf_log(ERR, fmt, ##__VA_ARGS__) +#define timvf_func_trace timvf_log_dbg + +#define TIM_COPROC (8) +#define TIM_GET_DEV_INFO (1) +#define TIM_GET_RING_INFO (2) +#define TIM_SET_RING_INFO (3) +#define TIM_RING_START_CYC_GET (4) + +#define TIM_MAX_RINGS (64) +#define TIM_DEV_PER_NODE (1) +#define TIM_VF_PER_DEV (64) +#define TIM_RING_PER_DEV (TIM_VF_PER_DEV) +#define TIM_RING_NODE_SHIFT (6) +#define TIM_RING_MASK ((TIM_RING_PER_DEV) - 1) +#define TIM_RING_INVALID (-1) + +#define TIM_MIN_INTERVAL (1E3) +#define TIM_MAX_INTERVAL ((1ull << 32) - 1) +#define TIM_MAX_BUCKETS (1ull << 20) +#define TIM_CHUNK_SIZE (4096) +#define TIM_MAX_CHUNKS_PER_BUCKET (1ull << 32) + +#define TIMVF_MAX_BURST (8) + +/* TIM VF Control/Status registers (CSRs): */ +/* VF_BAR0: */ +#define TIM_VF_NRSPERR_INT (0x0) +#define TIM_VF_NRSPERR_INT_W1S (0x8) +#define TIM_VF_NRSPERR_ENA_W1C (0x10) +#define TIM_VF_NRSPERR_ENA_W1S (0x18) +#define TIM_VRING_FR_RN_CYCLES (0x20) +#define TIM_VRING_FR_RN_GPIOS (0x28) +#define TIM_VRING_FR_RN_GTI (0x30) +#define TIM_VRING_FR_RN_PTP (0x38) +#define TIM_VRING_CTL0 (0x40) +#define TIM_VRING_CTL1 (0x50) +#define TIM_VRING_CTL2 (0x60) +#define TIM_VRING_BASE (0x100) +#define TIM_VRING_AURA (0x108) +#define TIM_VRING_REL (0x110) + +#define timvf_read64 rte_read64_relaxed +#define timvf_write64 rte_write64_relaxed + +#ifndef __hot +#define __hot __attribute__((hot)) +#endif + +extern int otx_logtype_timvf; + +static const uint16_t nb_chunk_slots = (TIM_CHUNK_SIZE / 16) - 1; + +enum timvf_clk_src { + TIM_CLK_SRC_SCLK, + TIM_CLK_SRC_GPIO, + TIM_CLK_SRC_GTI, + TIM_CLK_SRC_PTP, +}; + +/* TIM_MEM_BUCKET */ +struct tim_mem_bucket { + uint64_t first_chunk; + union { + uint64_t w1; + struct { + uint32_t nb_entry; + uint8_t sbt:1; + uint8_t hbt:1; + uint8_t bsk:1; + uint8_t rsvd:5; + uint8_t lock; + int16_t chunk_remainder; + }; + }; + uint64_t current_chunk; + uint64_t pad; +} __rte_packed; + +struct tim_mem_entry { + uint64_t w0; + uint64_t wqe; +} __rte_packed; + +struct timvf_ctrl_reg { + uint64_t rctrl0; + uint64_t rctrl1; + uint64_t rctrl2; + uint8_t use_pmu; +} __rte_packed; + +typedef uint32_t (*bkt_id)(uint32_t bkt_tcks, uint32_t nb_bkts); + +struct timvf_meta { + bkt_id get_target_bkt; + struct rte_reciprocal_u64 fast_div; + uint64_t ring_start_cyc; + uint32_t nb_bkts; + struct tim_mem_bucket *bkt; + void *chunk_pool; + uint64_t tck_int; +} __rte_cache_aligned; + +struct timvf_ring { + struct timvf_meta meta; + uint64_t tck_nsec; + void *vbar0; + void *bkt_pos; + uint64_t max_tout; + uint64_t nb_chunks; + enum timvf_clk_src clk_src; + uint16_t tim_ring_id; +} __rte_cache_aligned; + +static __rte_always_inline uint32_t __hot +bkt_mod(uint32_t rel_bkt, uint32_t nb_bkts) +{ + return rel_bkt % nb_bkts; +} + +int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, + uint32_t *caps, const struct rte_event_timer_adapter_ops **ops); + +#endif /* __TIMVF_EVDEV_H__ */ -- 2.16.1 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH 04/10] event/octeontx: add support to start and stop timer device 2018-02-16 21:36 [dpdk-dev] [PATCH 0/9] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (2 preceding siblings ...) 2018-02-16 21:36 ` [dpdk-dev] [PATCH 03/10] event/octeontx: add support to create and free timer adapter Pavan Nikhilesh @ 2018-02-16 21:36 ` Pavan Nikhilesh 2018-02-16 21:36 ` [dpdk-dev] [PATCH 05/10] event/octeontx: add support for arm and cancel Pavan Nikhilesh ` (8 subsequent siblings) 12 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-02-16 21:36 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh When application requests to start the timer adapter through `rte_event_timer_adapter_start`, Octeontx TIMvf ring does the following: - Uses mbox to communicate TIMpf driver about, * SCLK frequency used to convert ns<->cycles. * program the ring control parameters and start the ring. * get the exact cycle at which the TIMvf ring has started which can be used to estimate the bucket position. On `rte_event_timer_adapter_stop` i.e stop, Octeontx TIMvf ring does the following: - Use mbox to communicate TIMpf driver about, * reset the ring control parameters and stop the ring. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 140 +++++++++++++++++++++++++++++++++++ drivers/event/octeontx/timvf_evdev.h | 5 ++ 2 files changed, 145 insertions(+) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index a56ca7e71..dccdf1ddb 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -16,6 +16,32 @@ otx_timvf_init_log(void) rte_log_set_level(otx_logtype_timvf, RTE_LOG_NOTICE); } +struct __rte_packed timvf_mbox_dev_info { + uint64_t ring_active[4]; + uint64_t clk_freq; +}; + +/* Response messages */ +enum { + MBOX_RET_SUCCESS, + MBOX_RET_INVALID, + MBOX_RET_INTERNAL_ERR, +}; + +static int +timvf_mbox_dev_info_get(struct timvf_mbox_dev_info *info) +{ + struct octeontx_mbox_hdr hdr = {0}; + uint16_t len = sizeof(struct timvf_mbox_dev_info); + + hdr.coproc = TIM_COPROC; + hdr.msg = TIM_GET_DEV_INFO; + hdr.vfid = 0; /* TIM DEV is always 0. TIM RING ID changes. */ + + memset(info, 0, len); + return octeontx_ssovf_mbox_send(&hdr, NULL, 0, info, len); +} + static void timvf_ring_info_get(const struct rte_event_timer_adapter *adptr, struct rte_event_timer_adapter_info *adptr_info) @@ -27,6 +53,118 @@ timvf_ring_info_get(const struct rte_event_timer_adapter *adptr, sizeof(struct rte_event_timer_adapter_conf)); } +static int +timvf_ring_conf_set(struct timvf_ctrl_reg *rctl, uint8_t ring_id) +{ + struct octeontx_mbox_hdr hdr = {0}; + uint16_t len = sizeof(struct timvf_ctrl_reg); + int ret; + + hdr.coproc = TIM_COPROC; + hdr.msg = TIM_SET_RING_INFO; + hdr.vfid = ring_id; + + ret = octeontx_ssovf_mbox_send(&hdr, rctl, len, NULL, 0); + if (ret < 0 || hdr.res_code != MBOX_RET_SUCCESS) + return -EACCES; + return 0; +} + +static int +timvf_get_start_cyc(uint64_t *now, uint8_t ring_id) +{ + struct octeontx_mbox_hdr hdr = {0}; + + hdr.coproc = TIM_COPROC; + hdr.msg = TIM_RING_START_CYC_GET; + hdr.vfid = ring_id; + *now = 0; + return octeontx_ssovf_mbox_send(&hdr, NULL, 0, now, sizeof(uint64_t)); +} + +static int +timvf_ring_start(const struct rte_event_timer_adapter *adptr) +{ + int ret; + uint64_t interval = 0; + struct timvf_ctrl_reg rctrl = {0}; + struct timvf_mbox_dev_info dinfo; + struct timvf_ring *timr = adptr->data->adapter_priv; + + ret = timvf_mbox_dev_info_get(&dinfo); + if (ret < 0 || ret != sizeof(struct timvf_mbox_dev_info)) + return -EINVAL; + + /* Calculate the interval cycles according to clock source. */ + switch (timr->clk_src) { + case TIM_CLK_SRC_SCLK: + interval = NSEC2CLK(timr->tck_nsec, dinfo.clk_freq); + break; + case TIM_CLK_SRC_GPIO: + /* GPIO doesn't work on tck_nsec. */ + interval = 0; + break; + case TIM_CLK_SRC_GTI: + interval = NSEC2CLK(timr->tck_nsec, dinfo.clk_freq); + break; + case TIM_CLK_SRC_PTP: + interval = NSEC2CLK(timr->tck_nsec, dinfo.clk_freq); + break; + } + + /*CTRL0 register.*/ + rctrl.rctrl0 = interval; + + /*CTRL1 register.*/ + rctrl.rctrl1 = (uint64_t)(timr->clk_src) << 51 | + 1ull << 48 | + 1ull << 47 | + 1ull << 44 | + (timr->meta.nb_bkts - 1); + + rctrl.rctrl2 = (uint64_t)(TIM_CHUNK_SIZE / 16) << 40; + + timvf_write64((uint64_t)timr->meta.bkt, + (uint8_t *)timr->vbar0 + TIM_VRING_BASE); + if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) { + ret = -EACCES; + goto error; + } + + if (timvf_get_start_cyc(&timr->meta.ring_start_cyc, + timr->tim_ring_id) < 0) { + ret = -EACCES; + goto error; + } + timr->meta.tck_int = NSEC2CLK(timr->tck_nsec, rte_get_timer_hz()); + timr->meta.fast_div = rte_reciprocal_value_u64(timr->meta.tck_int); + timvf_log_info("nb_bkts %d min_ns %"PRIu64" min_cyc %"PRIu64"" + " maxtmo %"PRIu64"\n", + timr->meta.nb_bkts, timr->tck_nsec, interval, + timr->max_tout); + + return 0; +error: + rte_free(timr->meta.bkt); + rte_mempool_free(timr->meta.chunk_pool); + return ret; +} + +static int +timvf_ring_stop(const struct rte_event_timer_adapter *adptr) +{ + struct timvf_ring *timr = adptr->data->adapter_priv; + struct timvf_ctrl_reg rctrl = {0}; + rctrl.rctrl0 = timvf_read64((uint8_t *)timr->vbar0 + TIM_VRING_CTL0); + rctrl.rctrl1 = timvf_read64((uint8_t *)timr->vbar0 + TIM_VRING_CTL1); + rctrl.rctrl1 &= ~(1ull << 47); /* Disable */ + rctrl.rctrl2 = timvf_read64((uint8_t *)timr->vbar0 + TIM_VRING_CTL2); + + if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) + return -EACCES; + return 0; +} + static int timvf_ring_create(struct rte_event_timer_adapter *adptr) { @@ -145,6 +283,8 @@ timvf_ring_free(struct rte_event_timer_adapter *adptr) static struct rte_event_timer_adapter_ops timvf_ops = { .init = timvf_ring_create, .uninit = timvf_ring_free, + .start = timvf_ring_start, + .stop = timvf_ring_stop, .get_info = timvf_ring_info_get, }; diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index fcc938b82..a6a64587f 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -74,6 +74,11 @@ #define TIM_VRING_AURA (0x108) #define TIM_VRING_REL (0x110) + +#define NSEC_PER_SEC 1E9 +#define NSEC2CLK(__ns, __freq) (((__ns) * (__freq)) / NSEC_PER_SEC) +#define CLK2NSEC(__clk, __freq) (((__clk) * NSEC_PER_SEC) / (__freq)) + #define timvf_read64 rte_read64_relaxed #define timvf_write64 rte_write64_relaxed -- 2.16.1 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH 05/10] event/octeontx: add support for arm and cancel 2018-02-16 21:36 [dpdk-dev] [PATCH 0/9] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (3 preceding siblings ...) 2018-02-16 21:36 ` [dpdk-dev] [PATCH 04/10] event/octeontx: add support to start and stop timer device Pavan Nikhilesh @ 2018-02-16 21:36 ` Pavan Nikhilesh 2018-02-16 21:36 ` [dpdk-dev] [PATCH 06/10] event/octeontx: add single producer timer arm variant Pavan Nikhilesh ` (7 subsequent siblings) 12 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-02-16 21:36 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/Makefile | 5 + drivers/event/octeontx/meson.build | 1 + drivers/event/octeontx/timvf_evdev.c | 11 +- drivers/event/octeontx/timvf_evdev.h | 31 ++++ drivers/event/octeontx/timvf_worker.c | 53 +++++++ drivers/event/octeontx/timvf_worker.h | 278 ++++++++++++++++++++++++++++++++++ 6 files changed, 378 insertions(+), 1 deletion(-) create mode 100644 drivers/event/octeontx/timvf_worker.c create mode 100644 drivers/event/octeontx/timvf_worker.h diff --git a/drivers/event/octeontx/Makefile b/drivers/event/octeontx/Makefile index f1d10a99e..ba1ccfb04 100644 --- a/drivers/event/octeontx/Makefile +++ b/drivers/event/octeontx/Makefile @@ -27,19 +27,24 @@ LIBABIVER := 1 SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_worker.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_evdev.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_evdev_selftest.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += timvf_worker.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += timvf_evdev.c ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_ssovf_worker.o += -fno-prefetch-loop-arrays +CFLAGS_timvf_worker.o += -fno-prefetch-loop-arrays ifeq ($(shell test $(GCC_VERSION) -ge 46 && echo 1), 1) CFLAGS_ssovf_worker.o += -Ofast +CFLAGS_timvf_worker.o += -Ofast else CFLAGS_ssovf_worker.o += -O3 -ffast-math +CFLAGS_timvf_worker.o += -O3 -ffast-math endif else CFLAGS_ssovf_worker.o += -Ofast +CFLAGS_timvf_worker.o += -Ofast endif include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/event/octeontx/meson.build b/drivers/event/octeontx/meson.build index 8941f8a56..efac94cba 100644 --- a/drivers/event/octeontx/meson.build +++ b/drivers/event/octeontx/meson.build @@ -4,6 +4,7 @@ sources = files('ssovf_worker.c', 'ssovf_evdev.c', 'ssovf_evdev_selftest.c', + 'timvf_worker.c', 'timvf_evdev.c', ) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index dccdf1ddb..d0ba42263 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -261,6 +261,10 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) timvf_write64(0x7, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_ENA_W1C); timvf_write64(0x7, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_ENA_W1S); + adptr->arm_burst = timvf_timer_reg_burst_mp; + adptr->arm_tmo_tick_burst = NULL; + adptr->cancel_burst = timvf_timer_unreg_burst; + return 0; mem_err: rte_free(timr); @@ -294,7 +298,12 @@ timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, { RTE_SET_USED(dev); RTE_SET_USED(flags); + + timvf_ops.arm_burst = timvf_timer_reg_burst_mp; + timvf_ops.arm_tmo_tick_burst = NULL; + + timvf_ops.cancel_burst = timvf_timer_unreg_burst; *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; *ops = &timvf_ops; - return -EINVAL; + return 0; } diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index a6a64587f..c80e147e8 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -74,6 +74,33 @@ #define TIM_VRING_AURA (0x108) #define TIM_VRING_REL (0x110) +#define TIM_CTL1_W0_S_BUCKET 20 +#define TIM_CTL1_W0_M_BUCKET ((1ull << (40 - 20)) - 1) + +#define TIM_BUCKET_W1_S_NUM_ENTRIES (0) /*Shift*/ +#define TIM_BUCKET_W1_M_NUM_ENTRIES ((1ull << (32 - 0)) - 1) +#define TIM_BUCKET_W1_S_SBT (32) +#define TIM_BUCKET_W1_M_SBT ((1ull << (33 - 32)) - 1) +#define TIM_BUCKET_W1_S_HBT (33) +#define TIM_BUCKET_W1_M_HBT ((1ull << (34 - 33)) - 1) +#define TIM_BUCKET_W1_S_BSK (34) +#define TIM_BUCKET_W1_M_BSK ((1ull << (35 - 34)) - 1) +#define TIM_BUCKET_W1_S_LOCK (40) +#define TIM_BUCKET_W1_M_LOCK ((1ull << (48 - 40)) - 1) +#define TIM_BUCKET_W1_S_CHUNK_REMAINDER (48) +#define TIM_BUCKET_W1_M_CHUNK_REMAINDER ((1ull << (64 - 48)) - 1) + +#define TIM_BUCKET_SEMA \ + (TIM_BUCKET_CHUNK_REMAIN) + +#define TIM_BUCKET_CHUNK_REMAIN \ + (TIM_BUCKET_W1_M_CHUNK_REMAINDER << TIM_BUCKET_W1_S_CHUNK_REMAINDER) + +#define TIM_BUCKET_LOCK \ + (TIM_BUCKET_W1_M_LOCK << TIM_BUCKET_W1_S_LOCK) + +#define TIM_BUCKET_SEMA_WLOCK \ + (TIM_BUCKET_CHUNK_REMAIN | (1ull << TIM_BUCKET_W1_S_LOCK)) #define NSEC_PER_SEC 1E9 #define NSEC2CLK(__ns, __freq) (((__ns) * (__freq)) / NSEC_PER_SEC) @@ -159,5 +186,9 @@ bkt_mod(uint32_t rel_bkt, uint32_t nb_bkts) int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, uint32_t *caps, const struct rte_event_timer_adapter_ops **ops); +int timvf_timer_unreg_burst(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers); +int timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers); #endif /* __TIMVF_EVDEV_H__ */ diff --git a/drivers/event/octeontx/timvf_worker.c b/drivers/event/octeontx/timvf_worker.c new file mode 100644 index 000000000..7a924fd11 --- /dev/null +++ b/drivers/event/octeontx/timvf_worker.c @@ -0,0 +1,53 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include "timvf_worker.h" + +int +timvf_timer_unreg_burst(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers) +{ + RTE_SET_USED(adptr); + int ret; + uint16_t index; + + for (index = 0; index < nb_timers; index++) { + ret = timvf_rem_entry(tim[index]); + if (ret) { + rte_errno = -ret; + break; + } + } + return index; +} + +int +timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers) +{ + int ret; + uint16_t index; + struct tim_mem_entry entry; + struct timvf_ring *timr = adptr->data->adapter_priv; + for (index = 0; index < nb_timers; index++) { + if (unlikely(tim[index]->timeout_ticks > timr->meta.nb_bkts)) { + tim[index]->state = RTE_EVENT_TIMER_ERROR_TOOLATE; + rte_errno = -EINVAL; + break; + } + + entry.w0 = (tim[index]->ev.event & 0xFFC000000000) >> 6 | + (tim[index]->ev.event & 0xFFFFFFFFF); + entry.wqe = tim[index]->ev.u64; + ret = timvf_add_entry_mp(timr, tim[index]->timeout_ticks, + tim[index], &entry); + if (unlikely(ret)) { + rte_errno = -ret; + break; + } + } + + return index; +} diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h new file mode 100644 index 000000000..b63dd763c --- /dev/null +++ b/drivers/event/octeontx/timvf_worker.h @@ -0,0 +1,278 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include <rte_common.h> +#include <rte_branch_prediction.h> + +#include "timvf_evdev.h" + +static inline int16_t +timr_bkt_fetch_rem(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_CHUNK_REMAINDER) & + TIM_BUCKET_W1_M_CHUNK_REMAINDER; +} + +static inline int16_t +timr_bkt_get_rem(struct tim_mem_bucket *bktp) +{ + return __atomic_load_n((int16_t *)&bktp->chunk_remainder, + __ATOMIC_ACQUIRE); +} + +static inline void +timr_bkt_set_rem(struct tim_mem_bucket *bktp, uint16_t v) +{ + __atomic_store_n((int16_t *)&bktp->chunk_remainder, v, + __ATOMIC_RELEASE); +} + +static inline void +timr_bkt_sub_rem(struct tim_mem_bucket *bktp, uint16_t v) +{ + __atomic_fetch_sub((int16_t *)&bktp->chunk_remainder, v, + __ATOMIC_RELEASE); +} + +static inline uint8_t +timr_bkt_get_sbt(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_SBT) & TIM_BUCKET_W1_M_SBT; +} + +static inline uint64_t +timr_bkt_set_sbt(struct tim_mem_bucket *bktp) +{ + const uint64_t v = TIM_BUCKET_W1_M_SBT << TIM_BUCKET_W1_S_SBT; + return __atomic_fetch_or((uint64_t *)&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline uint64_t +timr_bkt_clr_sbt(struct tim_mem_bucket *bktp) +{ + const uint64_t v = ~(TIM_BUCKET_W1_M_SBT << TIM_BUCKET_W1_S_SBT); + return __atomic_fetch_and((uint64_t *)&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline uint8_t +timr_bkt_get_shbt(uint64_t w1) +{ + return ((w1 >> TIM_BUCKET_W1_S_HBT) & TIM_BUCKET_W1_M_HBT) | + ((w1 >> TIM_BUCKET_W1_S_SBT) & TIM_BUCKET_W1_M_SBT); +} + +static inline uint8_t +timr_bkt_hbt_chk(struct tim_mem_bucket *bktp) +{ + return (__atomic_load_n((uint64_t *)&bktp->w1, __ATOMIC_ACQ_REL) + >> TIM_BUCKET_W1_S_HBT) & TIM_BUCKET_W1_M_HBT; +} + +static inline uint8_t +timr_bkt_sbt_chk(struct tim_mem_bucket *bktp) +{ + return (__atomic_load_n((uint64_t *)&bktp->w1, __ATOMIC_ACQ_REL) + >> TIM_BUCKET_W1_S_SBT) & TIM_BUCKET_W1_M_SBT; +} + +static inline uint8_t +timr_bkt_get_bsk(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_BSK) & TIM_BUCKET_W1_M_BSK; +} + +static inline uint64_t +timr_bkt_clr_bsk(struct tim_mem_bucket *bktp) +{ + /*Clear everything except lock. */ + const uint64_t v = TIM_BUCKET_W1_M_LOCK << TIM_BUCKET_W1_S_LOCK; + return __atomic_fetch_and((uint64_t *)&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline uint64_t +timr_bkt_fetch_sema_lock(struct tim_mem_bucket *bktp) +{ + return __atomic_fetch_add((uint64_t *)&bktp->w1, TIM_BUCKET_SEMA_WLOCK, + __ATOMIC_ACQ_REL); +} + +static inline uint64_t +timr_bkt_fetch_sema(struct tim_mem_bucket *bktp) +{ + return __atomic_fetch_add((uint64_t *)&bktp->w1, TIM_BUCKET_SEMA, + __ATOMIC_ACQ_REL); +} + +static inline uint64_t +timr_bkt_inc_lock(struct tim_mem_bucket *bktp) +{ + const uint64_t v = 1ull << TIM_BUCKET_W1_S_LOCK; + return __atomic_fetch_add((uint64_t *)&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline void +timr_bkt_dec_lock(struct tim_mem_bucket *bktp) +{ + __atomic_add_fetch((uint8_t *)&bktp->lock, 0xff, __ATOMIC_ACQ_REL); +} + +static inline uint32_t +timr_bkt_get_nent(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_NUM_ENTRIES) & + TIM_BUCKET_W1_M_NUM_ENTRIES; +} + +static inline void +timr_bkt_inc_nent(struct tim_mem_bucket *bktp) +{ + __atomic_add_fetch((uint32_t *)&bktp->nb_entry, 1, __ATOMIC_ACQ_REL); +} + +static inline void +timr_bkt_add_nent(struct tim_mem_bucket *bktp, uint32_t v) +{ + __atomic_add_fetch((uint32_t *)&bktp->nb_entry, v, __ATOMIC_ACQ_REL); +} + +static inline uint64_t +timr_bkt_clr_nent(struct tim_mem_bucket *bktp) +{ + const uint64_t v = ~(TIM_BUCKET_W1_M_NUM_ENTRIES << + TIM_BUCKET_W1_S_NUM_ENTRIES); + return __atomic_and_fetch((uint64_t *)&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline __hot struct tim_mem_entry* +timr_clr_bkt(struct timvf_ring *timr, struct tim_mem_bucket *bkt) +{ + struct tim_mem_entry *chunk; + struct tim_mem_entry *pnext; + chunk = ((struct tim_mem_entry *)bkt->first_chunk); + chunk = (struct tim_mem_entry *)(chunk + nb_chunk_slots)->w0; + + while (chunk) { + pnext = (struct tim_mem_entry *)((chunk + nb_chunk_slots)->w0); + rte_mempool_put(timr->meta.chunk_pool, chunk); + chunk = pnext; + } + return (struct tim_mem_entry *)bkt->first_chunk; +} + +static inline int __hot +timvf_rem_entry(struct rte_event_timer *tim) +{ + uint64_t lock_sema; + struct tim_mem_entry *entry; + struct tim_mem_bucket *bkt; + if (tim->impl_opaque[1] == 0 || + tim->impl_opaque[0] == 0) + return -ENOENT; + + entry = (struct tim_mem_entry *)tim->impl_opaque[0]; + if (entry->wqe != tim->ev.u64) { + tim->impl_opaque[1] = tim->impl_opaque[0] = 0; + return -ENOENT; + } + bkt = (struct tim_mem_bucket *)tim->impl_opaque[1]; + lock_sema = timr_bkt_inc_lock(bkt); + if (timr_bkt_get_shbt(lock_sema) + || !timr_bkt_get_nent(lock_sema)) { + timr_bkt_dec_lock(bkt); + tim->impl_opaque[1] = tim->impl_opaque[0] = 0; + return -ENOENT; + } + + entry->w0 = entry->wqe = 0; + timr_bkt_dec_lock(bkt); + + tim->state = RTE_EVENT_TIMER_CANCELED; + tim->impl_opaque[1] = tim->impl_opaque[0] = 0; + return 0; +} + +/* Multi producer functions. */ +static inline int __hot +timvf_add_entry_mp(struct timvf_ring *timr, const uint32_t rel_bkt, + struct rte_event_timer *tim, const struct tim_mem_entry *pent) +{ + uint8_t lock_cnt; + int16_t rem; + uint32_t bucket; + uint32_t tbkt_id; + const uint32_t nb_bkts = timr->meta.nb_bkts; + uint64_t lock_sema; + uint64_t pos_reg; + const uint64_t start = timr->meta.ring_start_cyc; + struct tim_mem_bucket *bkt; + struct tim_mem_entry *chunk; + +__retry: + pos_reg = (rte_rdtsc() - start); + bucket = rte_reciprocal_divide_u64(pos_reg, + &timr->meta.fast_div) + rel_bkt; + tbkt_id = timr->meta.get_target_bkt(bucket, nb_bkts); + bkt = &timr->meta.bkt[tbkt_id]; + while (1) { + /* Bucket related checks. */ + + /*Get Bucket sema*/ + lock_sema = timr_bkt_fetch_sema_lock(bkt); + if (unlikely(timr_bkt_get_shbt(lock_sema))) { + timr_bkt_dec_lock(bkt); + goto __retry; + } + + RTE_SET_USED(lock_cnt); + /* Insert the work. */ + rem = timr_bkt_fetch_rem(lock_sema); + + if (rem < 0) { + /* goto diff bucket. */ + timr_bkt_dec_lock(bkt); + goto __retry; + } else if (!rem) { + /*Only one thread can be here*/ + if (bkt->nb_entry || !bkt->first_chunk) { + if (unlikely(rte_mempool_get( + timr->meta.chunk_pool, + (void **)&chunk))) { + timr_bkt_set_rem(bkt, 0); + timr_bkt_dec_lock(bkt); + tim->impl_opaque[0] = + tim->impl_opaque[1] = 0; + tim->state = RTE_EVENT_TIMER_ERROR; + return -ENOMEM; + } + if (bkt->nb_entry) { + *(uint64_t *)((struct tim_mem_entry *) + bkt->current_chunk + + nb_chunk_slots) = + (uint64_t) chunk; + } else { + bkt->first_chunk = (uint64_t) chunk; + } + *(uint64_t *)(chunk + nb_chunk_slots) = 0; + } else { + chunk = timr_clr_bkt(timr, bkt); + *(uint64_t *)(chunk + nb_chunk_slots) = 0; + bkt->first_chunk = (uint64_t) chunk; + } + bkt->current_chunk = (uint64_t) chunk; + timr_bkt_set_rem(bkt, nb_chunk_slots - 1); + } else { + chunk = (struct tim_mem_entry *)bkt->current_chunk; + chunk += nb_chunk_slots - rem; + } + /* Copy work entry. */ + *chunk = *pent; + timr_bkt_inc_nent(bkt); + timr_bkt_dec_lock(bkt); + break; + } + tim->impl_opaque[0] = (uint64_t)chunk; + tim->impl_opaque[1] = (uint64_t)bkt; + tim->state = RTE_EVENT_TIMER_ARMED; + return 0; +} -- 2.16.1 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH 06/10] event/octeontx: add single producer timer arm variant 2018-02-16 21:36 [dpdk-dev] [PATCH 0/9] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (4 preceding siblings ...) 2018-02-16 21:36 ` [dpdk-dev] [PATCH 05/10] event/octeontx: add support for arm and cancel Pavan Nikhilesh @ 2018-02-16 21:36 ` Pavan Nikhilesh 2018-03-07 19:41 ` Carrillo, Erik G 2018-02-16 21:36 ` [dpdk-dev] [PATCH 07/10] event/octeontx: optimize timer adapter resolution parameters Pavan Nikhilesh ` (6 subsequent siblings) 12 siblings, 1 reply; 76+ messages in thread From: Pavan Nikhilesh @ 2018-02-16 21:36 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh When application creates the timer adapter by passing `RTE_EVENT_TIMER_ADAPTER_F_SP_PUT` flag, we can optimize the arm sequence by removing the locking overhead. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 22 +++- drivers/event/octeontx/timvf_evdev.h | 5 + drivers/event/octeontx/timvf_worker.c | 65 ++++++++++++ drivers/event/octeontx/timvf_worker.h | 183 ++++++++++++++++++++++++++++++++++ 4 files changed, 270 insertions(+), 5 deletions(-) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index d0ba42263..6cf5d4846 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -174,6 +174,7 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) struct rte_event_timer_adapter_conf *rcfg = &adptr->data->conf; struct timvf_ring *timr; struct octeontx_timvf_info tinfo; + unsigned int mp_flags = 0; if (octeontx_timvf_info(&tinfo) < 0) return -ENODEV; @@ -224,6 +225,11 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) timr->nb_chunks = nb_timers / nb_chunk_slots; + if (rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_SP_PUT) { + mp_flags = MEMPOOL_F_SP_PUT | MEMPOOL_F_SC_GET; + timvf_log_info("Using single producer mode"); + } + timr->meta.bkt = rte_zmalloc("octeontx_timvf_bucket", (timr->meta.nb_bkts) * sizeof(struct tim_mem_bucket), 0); @@ -261,8 +267,12 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) timvf_write64(0x7, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_ENA_W1C); timvf_write64(0x7, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_ENA_W1S); - adptr->arm_burst = timvf_timer_reg_burst_mp; - adptr->arm_tmo_tick_burst = NULL; + if (mp_flags) + adptr->arm_burst = timvf_timer_reg_burst_sp; + else + adptr->arm_burst = timvf_timer_reg_burst_mp; + + adptr->arm_tmo_tick_burst = timvf_timer_reg_brst; adptr->cancel_burst = timvf_timer_unreg_burst; return 0; @@ -297,11 +307,13 @@ timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, uint32_t *caps, const struct rte_event_timer_adapter_ops **ops) { RTE_SET_USED(dev); - RTE_SET_USED(flags); - timvf_ops.arm_burst = timvf_timer_reg_burst_mp; - timvf_ops.arm_tmo_tick_burst = NULL; + if (flags & RTE_EVENT_TIMER_ADAPTER_F_SP_PUT) + timvf_ops.arm_burst = timvf_timer_reg_burst_sp; + else + timvf_ops.arm_burst = timvf_timer_reg_burst_mp; + timvf_ops.arm_tmo_tick_burst = timvf_timer_reg_brst; timvf_ops.cancel_burst = timvf_timer_unreg_burst; *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; *ops = &timvf_ops; diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index c80e147e8..b5db233bb 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -186,8 +186,13 @@ bkt_mod(uint32_t rel_bkt, uint32_t nb_bkts) int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, uint32_t *caps, const struct rte_event_timer_adapter_ops **ops); +int timvf_timer_reg_brst(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint64_t timeout_tick, + const uint16_t nb_timers); int timvf_timer_unreg_burst(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers); +int timvf_timer_reg_burst_sp(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers); int timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers); diff --git a/drivers/event/octeontx/timvf_worker.c b/drivers/event/octeontx/timvf_worker.c index 7a924fd11..3e48f3ca6 100644 --- a/drivers/event/octeontx/timvf_worker.c +++ b/drivers/event/octeontx/timvf_worker.c @@ -5,6 +5,42 @@ #include "timvf_worker.h" +int +timvf_timer_reg_brst(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint64_t timeout_tick, + const uint16_t nb_timers) +{ + int ret; + uint16_t set_timers = 0; + uint16_t idx; + uint16_t arr_idx = 0; + struct timvf_ring *timr = adptr->data->adapter_priv; + struct tim_mem_entry entry[TIMVF_MAX_BURST] __rte_cache_aligned; + + if (unlikely(timeout_tick > timr->meta.nb_bkts)) { + for (idx = 0; idx < nb_timers; idx++) + tim[idx]->state = RTE_EVENT_TIMER_ERROR_TOOLATE; + rte_errno = -EINVAL; + return 0; + } + + while (arr_idx < nb_timers) { + for (idx = 0; idx < TIMVF_MAX_BURST && (arr_idx < nb_timers); + idx++, arr_idx++) { + entry[idx].w0 = + (tim[arr_idx]->ev.event & 0xFFC000000000) >> 6 | + (tim[arr_idx]->ev.event & 0xFFFFFFFF); + entry[idx].wqe = tim[arr_idx]->ev.u64; + } + ret = timvf_add_entry_brst(timr, timeout_tick, &tim[set_timers], + entry, idx); + set_timers += ret; + if (ret != idx) + break; + } + return set_timers; +} + int timvf_timer_unreg_burst(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers) @@ -23,6 +59,35 @@ timvf_timer_unreg_burst(const struct rte_event_timer_adapter *adptr, return index; } +int +timvf_timer_reg_burst_sp(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers) +{ + int ret; + uint16_t index; + struct tim_mem_entry entry; + struct timvf_ring *timr = adptr->data->adapter_priv; + for (index = 0; index < nb_timers; index++) { + if (unlikely(tim[index]->timeout_ticks > timr->meta.nb_bkts)) { + tim[index]->state = RTE_EVENT_TIMER_ERROR_TOOLATE; + rte_errno = -EINVAL; + break; + } + + entry.w0 = (tim[index]->ev.event & 0xFFC000000000) >> 6 | + (tim[index]->ev.event & 0xFFFFFFFF); + entry.wqe = tim[index]->ev.u64; + ret = timvf_add_entry_sp(timr, tim[index]->timeout_ticks, + tim[index], &entry); + if (unlikely(ret)) { + rte_errno = -ret; + break; + } + } + + return index; +} + int timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers) diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h index b63dd763c..320eb6ac1 100644 --- a/drivers/event/octeontx/timvf_worker.h +++ b/drivers/event/octeontx/timvf_worker.h @@ -160,6 +160,118 @@ timr_clr_bkt(struct timvf_ring *timr, struct tim_mem_bucket *bkt) return (struct tim_mem_entry *)bkt->first_chunk; } +/* Burst mode functions */ +static inline int __hot +timvf_add_entry_brst(struct timvf_ring *timr, const uint16_t rel_bkt, + struct rte_event_timer **tim, const struct tim_mem_entry *ents, + const uint16_t nb_timers) +{ + int16_t rem; + int16_t crem = 0; + uint8_t lock_cnt; + uint16_t index = 0; + uint16_t chunk_remainder = 0; + uint32_t bucket; + uint32_t tbkt_id; + const uint32_t nb_bkts = timr->meta.nb_bkts; + const uint64_t start = timr->meta.ring_start_cyc; + uint64_t pos_reg; + uint64_t lock_sema; + struct tim_mem_bucket *bkt; + struct tim_mem_entry *chunk; + +__retry: + pos_reg = (rte_rdtsc() - start); + bucket = rte_reciprocal_divide_u64(pos_reg, + &timr->meta.fast_div) + rel_bkt; + tbkt_id = timr->meta.get_target_bkt(bucket, nb_bkts); + bkt = &timr->meta.bkt[tbkt_id]; + + /* Only one thread beyond this. */ + lock_sema = timr_bkt_inc_lock(bkt); + lock_cnt = (uint8_t) + ((lock_sema >> TIM_BUCKET_W1_S_LOCK) & TIM_BUCKET_W1_M_LOCK); + + if (lock_cnt) { + timr_bkt_dec_lock(bkt); + goto __retry; + } + + /* Bucket related checks. */ + if (unlikely(timr_bkt_get_shbt(lock_sema))) { + timr_bkt_dec_lock(bkt); + goto __retry; + } + + chunk_remainder = timr_bkt_fetch_rem(lock_sema); + rem = chunk_remainder - nb_timers; + if (rem < 0) { + crem = nb_chunk_slots - chunk_remainder; + if (chunk_remainder && crem) { + chunk = ((struct tim_mem_entry *)bkt->current_chunk) + + crem; + for (; index < chunk_remainder; index++) { + *chunk = *(ents + index); + tim[index]->impl_opaque[0] = (uint64_t)chunk++; + tim[index]->impl_opaque[1] = (uint64_t)bkt; + tim[index]->state = RTE_EVENT_TIMER_ARMED; + } + timr_bkt_sub_rem(bkt, chunk_remainder); + timr_bkt_add_nent(bkt, chunk_remainder); + } + rem = nb_timers - chunk_remainder; + ents = ents + chunk_remainder; + if (bkt->nb_entry || !bkt->first_chunk) { + if (unlikely(rte_mempool_get(timr->meta.chunk_pool, + (void **)&chunk))) { + /* + * No more chunks, return number of entries + * successfully copied. + */ + timr_bkt_dec_lock(bkt); + rte_errno = -ENOMEM; + tim[index]->state = RTE_EVENT_TIMER_ERROR; + return crem; + } + if (bkt->nb_entry) { + *(uint64_t *)( + (struct tim_mem_entry *)bkt->current_chunk + + nb_chunk_slots) = (uint64_t) chunk; + } else { + bkt->first_chunk = (uint64_t) chunk; + } + } else { + chunk = timr_clr_bkt(timr, bkt); + bkt->first_chunk = (uint64_t) chunk; + } + *(uint64_t *)(chunk + nb_chunk_slots) = 0; + bkt->current_chunk = (uint64_t) chunk; + + for (; index < nb_timers; index++) { + *chunk = *(ents + index); + tim[index]->impl_opaque[0] = (uint64_t)chunk++; + tim[index]->impl_opaque[1] = (uint64_t)bkt; + tim[index]->state = RTE_EVENT_TIMER_ARMED; + } + timr_bkt_set_rem(bkt, nb_chunk_slots - rem); + timr_bkt_add_nent(bkt, rem); + } else { + chunk = (struct tim_mem_entry *)bkt->current_chunk; + chunk += (nb_chunk_slots - chunk_remainder); + for (; index < nb_timers; index++) { + *chunk = *(ents + index); + tim[index]->impl_opaque[0] = (uint64_t)chunk++; + tim[index]->impl_opaque[1] = (uint64_t)bkt; + tim[index]->state = RTE_EVENT_TIMER_ARMED; + } + timr_bkt_sub_rem(bkt, nb_timers); + timr_bkt_add_nent(bkt, nb_timers); + } + + timr_bkt_dec_lock(bkt); + return nb_timers; +} + static inline int __hot timvf_rem_entry(struct rte_event_timer *tim) { @@ -192,6 +304,77 @@ timvf_rem_entry(struct rte_event_timer *tim) return 0; } +/* Single producer functions. */ +static inline int __hot +timvf_add_entry_sp(struct timvf_ring *timr, const uint32_t rel_bkt, + struct rte_event_timer *tim, const struct tim_mem_entry *pent) +{ + int16_t rem; + uint32_t bucket; + uint32_t tbkt_id; + const uint32_t nb_bkts = timr->meta.nb_bkts; + uint64_t lock_sema; + uint64_t pos_reg; + const uint64_t start = timr->meta.ring_start_cyc; + struct tim_mem_bucket *bkt; + struct tim_mem_entry *chunk; + + pos_reg = (rte_rdtsc() - start); + bucket = rte_reciprocal_divide_u64(pos_reg, + &timr->meta.fast_div) + rel_bkt; + tbkt_id = timr->meta.get_target_bkt(bucket, nb_bkts); + bkt = &timr->meta.bkt[tbkt_id]; +__retry: + /*Get Bucket sema*/ + lock_sema = timr_bkt_fetch_sema(bkt); + /* Bucket related checks. */ + if (unlikely(timr_bkt_get_shbt(lock_sema))) + goto __retry; + + /* Insert the work. */ + rem = timr_bkt_fetch_rem(lock_sema); + + if (!rem) { + /* SP mode will have only one thread. */ + if (bkt->nb_entry || !bkt->first_chunk) { + if (unlikely(rte_mempool_get(timr->meta.chunk_pool, + (void **)&chunk))) { + timr_bkt_set_rem(bkt, 0); + tim->impl_opaque[0] = + tim->impl_opaque[1] = 0; + tim->state = RTE_EVENT_TIMER_ERROR; + return -ENOMEM; + } + if (bkt->nb_entry) { + *(uint64_t *)((struct tim_mem_entry *) + bkt->current_chunk + + nb_chunk_slots) = + (uint64_t) chunk; + } else { + bkt->first_chunk = (uint64_t) chunk; + } + *(uint64_t *)(chunk + nb_chunk_slots) = 0; + } else { + chunk = timr_clr_bkt(timr, bkt); + *(uint64_t *)(chunk + nb_chunk_slots) = 0; + bkt->first_chunk = (uint64_t) chunk; + } + bkt->current_chunk = (uint64_t) chunk; + timr_bkt_set_rem(bkt, nb_chunk_slots - 1); + } else { + chunk = (struct tim_mem_entry *)bkt->current_chunk; + chunk += nb_chunk_slots - rem; + } + /* Copy work entry. */ + *chunk = *pent; + timr_bkt_inc_nent(bkt); + + tim->impl_opaque[0] = (uint64_t)chunk; + tim->impl_opaque[1] = (uint64_t)bkt; + tim->state = RTE_EVENT_TIMER_ARMED; + return 0; +} + /* Multi producer functions. */ static inline int __hot timvf_add_entry_mp(struct timvf_ring *timr, const uint32_t rel_bkt, -- 2.16.1 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH 06/10] event/octeontx: add single producer timer arm variant 2018-02-16 21:36 ` [dpdk-dev] [PATCH 06/10] event/octeontx: add single producer timer arm variant Pavan Nikhilesh @ 2018-03-07 19:41 ` Carrillo, Erik G 0 siblings, 0 replies; 76+ messages in thread From: Carrillo, Erik G @ 2018-03-07 19:41 UTC (permalink / raw) To: Pavan Nikhilesh, jerin.jacob, santosh.shukla; +Cc: dev Hi Pavan, <... snipped ...> > @@ -261,8 +267,12 @@ timvf_ring_create(struct rte_event_timer_adapter > *adptr) > timvf_write64(0x7, (uint8_t *)timr->vbar0 + > TIM_VF_NRSPERR_ENA_W1C); > timvf_write64(0x7, (uint8_t *)timr->vbar0 + > TIM_VF_NRSPERR_ENA_W1S); > > - adptr->arm_burst = timvf_timer_reg_burst_mp; > - adptr->arm_tmo_tick_burst = NULL; > + if (mp_flags) > + adptr->arm_burst = timvf_timer_reg_burst_sp; > + else > + adptr->arm_burst = timvf_timer_reg_burst_mp; > + > + adptr->arm_tmo_tick_burst = timvf_timer_reg_brst; > adptr->cancel_burst = timvf_timer_unreg_burst; > > return 0; The values assigned here to adptr->(arm_burst|arm_tmo_tick_burst|cancel_burst) would get overwritten by the caller with the values that are set immediately below. I.e., the below should be sufficient by itself. > @@ -297,11 +307,13 @@ timvf_timer_adapter_caps_get(const struct > rte_eventdev *dev, uint64_t flags, > uint32_t *caps, const struct rte_event_timer_adapter_ops > **ops) { > RTE_SET_USED(dev); > - RTE_SET_USED(flags); > > - timvf_ops.arm_burst = timvf_timer_reg_burst_mp; > - timvf_ops.arm_tmo_tick_burst = NULL; > + if (flags & RTE_EVENT_TIMER_ADAPTER_F_SP_PUT) > + timvf_ops.arm_burst = timvf_timer_reg_burst_sp; > + else > + timvf_ops.arm_burst = timvf_timer_reg_burst_mp; > > + timvf_ops.arm_tmo_tick_burst = timvf_timer_reg_brst; > timvf_ops.cancel_burst = timvf_timer_unreg_burst; > *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; > *ops = &timvf_ops; > diff --git a/drivers/event/octeontx/timvf_evdev.h > b/drivers/event/octeontx/timvf_evdev.h > index c80e147e8..b5db233bb 100644 > --- a/drivers/event/octeontx/timvf_evdev.h > +++ b/drivers/event/octeontx/timvf_evdev.h > @@ -186,8 +186,13 @@ bkt_mod(uint32_t rel_bkt, uint32_t nb_bkts) <... snipped ...> Regards, Gabriel ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH 07/10] event/octeontx: optimize timer adapter resolution parameters 2018-02-16 21:36 [dpdk-dev] [PATCH 0/9] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (5 preceding siblings ...) 2018-02-16 21:36 ` [dpdk-dev] [PATCH 06/10] event/octeontx: add single producer timer arm variant Pavan Nikhilesh @ 2018-02-16 21:36 ` Pavan Nikhilesh 2018-02-17 5:06 ` Jerin Jacob 2018-02-16 21:36 ` [dpdk-dev] [PATCH 08/10] event/octeontx: add option to use fpavf as chunk pool Pavan Nikhilesh ` (5 subsequent siblings) 12 siblings, 1 reply; 76+ messages in thread From: Pavan Nikhilesh @ 2018-02-16 21:36 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh When application sets `RTE_EVENT_TIMER_ADAPTER_F_ADJUST_RES` flag while creating adapter underlying driver is free to optimize the resolution for best possible configuration. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 51 +++++++++++++++++++++++++++++++++++- drivers/event/octeontx/timvf_evdev.h | 6 +++++ 2 files changed, 56 insertions(+), 1 deletion(-) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index 6cf5d4846..ffdfbb387 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -82,6 +82,48 @@ timvf_get_start_cyc(uint64_t *now, uint8_t ring_id) return octeontx_ssovf_mbox_send(&hdr, NULL, 0, now, sizeof(uint64_t)); } +#define ALIGN_CEIL(a, b) \ + (((a + (typeof(a)) b - 1) / ((typeof(a)) b)) * (typeof(a))b) + +static int +optimize_bucket_parameters(struct timvf_ring *timr) +{ + uint32_t hbkts; + uint32_t lbkts; + uint64_t tck_nsec; + + hbkts = rte_align32pow2(timr->meta.nb_bkts); + tck_nsec = ALIGN_CEIL(timr->max_tout / (hbkts - 1), 10); + + if ((tck_nsec < 1000 || hbkts > TIM_MAX_BUCKETS)) + hbkts = 0; + + lbkts = rte_align32lowpow2(timr->meta.nb_bkts); + tck_nsec = ALIGN_CEIL((timr->max_tout / (lbkts - 1)), 10); + + if ((tck_nsec < 1000 || hbkts > TIM_MAX_BUCKETS)) + lbkts = 0; + + if (!hbkts && !lbkts) + return 0; + + if (!hbkts) { + timr->meta.nb_bkts = lbkts; + goto end; + } else if (!lbkts) { + timr->meta.nb_bkts = hbkts; + goto end; + } + + timr->meta.nb_bkts = (hbkts - timr->meta.nb_bkts) < + (timr->meta.nb_bkts - lbkts) ? hbkts : lbkts; +end: + timr->meta.get_target_bkt = bkt_and; + timr->tck_nsec = ALIGN_CEIL((timr->max_tout / + (timr->meta.nb_bkts - 1)), 10); + return 1; +} + static int timvf_ring_start(const struct rte_event_timer_adapter *adptr) { @@ -215,7 +257,7 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) } timr->tim_ring_id = adptr->data->id; - timr->tck_nsec = rcfg->timer_tick_ns; + timr->tck_nsec = ALIGN_CEIL(rcfg->timer_tick_ns, 10); timr->max_tout = rcfg->max_tmo_ns; timr->meta.nb_bkts = (timr->max_tout / timr->tck_nsec) + 1; timr->vbar0 = octeontx_timvf_bar(timr->tim_ring_id, 0); @@ -225,6 +267,13 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) timr->nb_chunks = nb_timers / nb_chunk_slots; + /* Try to optimize the bucket parameters. */ + if ((rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_ADJUST_RES) + && !rte_is_power_of_2(timr->meta.nb_bkts)) { + optimize_bucket_parameters(timr); + timvf_log_info("Optimizing configured values"); + } + if (rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_SP_PUT) { mp_flags = MEMPOOL_F_SP_PUT | MEMPOOL_F_SC_GET; timvf_log_info("Using single producer mode"); diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index b5db233bb..5e526a36a 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -184,6 +184,12 @@ bkt_mod(uint32_t rel_bkt, uint32_t nb_bkts) return rel_bkt % nb_bkts; } +static __rte_always_inline uint32_t __hot +bkt_and(uint32_t rel_bkt, uint32_t nb_bkts) +{ + return rel_bkt & (nb_bkts - 1); +} + int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, uint32_t *caps, const struct rte_event_timer_adapter_ops **ops); int timvf_timer_reg_brst(const struct rte_event_timer_adapter *adptr, -- 2.16.1 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH 07/10] event/octeontx: optimize timer adapter resolution parameters 2018-02-16 21:36 ` [dpdk-dev] [PATCH 07/10] event/octeontx: optimize timer adapter resolution parameters Pavan Nikhilesh @ 2018-02-17 5:06 ` Jerin Jacob 2018-02-19 9:34 ` Pavan Nikhilesh 0 siblings, 1 reply; 76+ messages in thread From: Jerin Jacob @ 2018-02-17 5:06 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Sat, 17 Feb 2018 03:06:57 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH 07/10] event/octeontx: optimize timer adapter > resolution parameters > X-Mailer: git-send-email 2.16.1 > > When application sets `RTE_EVENT_TIMER_ADAPTER_F_ADJUST_RES` flag > while creating adapter underlying driver is free to optimize the > resolution for best possible configuration. > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > drivers/event/octeontx/timvf_evdev.c | 51 +++++++++++++++++++++++++++++++++++- > drivers/event/octeontx/timvf_evdev.h | 6 +++++ > 2 files changed, 56 insertions(+), 1 deletion(-) > > diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c > index 6cf5d4846..ffdfbb387 100644 > --- a/drivers/event/octeontx/timvf_evdev.c > +++ b/drivers/event/octeontx/timvf_evdev.c > @@ -82,6 +82,48 @@ timvf_get_start_cyc(uint64_t *now, uint8_t ring_id) > return octeontx_ssovf_mbox_send(&hdr, NULL, 0, now, sizeof(uint64_t)); > } > > +#define ALIGN_CEIL(a, b) \ > + (((a + (typeof(a)) b - 1) / ((typeof(a)) b)) * (typeof(a))b) > + Can this moved to common code if this not already available? ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH 07/10] event/octeontx: optimize timer adapter resolution parameters 2018-02-17 5:06 ` Jerin Jacob @ 2018-02-19 9:34 ` Pavan Nikhilesh 0 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-02-19 9:34 UTC (permalink / raw) To: Jerin Jacob, santosh.shukla, erik.g.carrillo; +Cc: dev On Sat, Feb 17, 2018 at 10:36:24AM +0530, Jerin Jacob wrote: > -----Original Message----- > > Date: Sat, 17 Feb 2018 03:06:57 +0530 > > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > > erik.g.carrillo@intel.com > > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > > Subject: [dpdk-dev] [PATCH 07/10] event/octeontx: optimize timer adapter > > resolution parameters > > X-Mailer: git-send-email 2.16.1 > > > > When application sets `RTE_EVENT_TIMER_ADAPTER_F_ADJUST_RES` flag > > while creating adapter underlying driver is free to optimize the > > resolution for best possible configuration. > > > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > > --- > > drivers/event/octeontx/timvf_evdev.c | 51 +++++++++++++++++++++++++++++++++++- > > drivers/event/octeontx/timvf_evdev.h | 6 +++++ > > 2 files changed, 56 insertions(+), 1 deletion(-) > > > > diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c > > index 6cf5d4846..ffdfbb387 100644 > > --- a/drivers/event/octeontx/timvf_evdev.c > > +++ b/drivers/event/octeontx/timvf_evdev.c > > @@ -82,6 +82,48 @@ timvf_get_start_cyc(uint64_t *now, uint8_t ring_id) > > return octeontx_ssovf_mbox_send(&hdr, NULL, 0, now, sizeof(uint64_t)); > > } > > > > +#define ALIGN_CEIL(a, b) \ > > + (((a + (typeof(a)) b - 1) / ((typeof(a)) b)) * (typeof(a))b) > > + > > Can this moved to common code if this not already available? > Currently, align macros support aligning given value to a power of 2 and not to a given multiple. Will move this to common code and send a seperate patch. ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH 08/10] event/octeontx: add option to use fpavf as chunk pool 2018-02-16 21:36 [dpdk-dev] [PATCH 0/9] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (6 preceding siblings ...) 2018-02-16 21:36 ` [dpdk-dev] [PATCH 07/10] event/octeontx: optimize timer adapter resolution parameters Pavan Nikhilesh @ 2018-02-16 21:36 ` Pavan Nikhilesh 2018-02-18 11:42 ` santosh 2018-02-23 20:17 ` Carrillo, Erik G 2018-02-16 21:36 ` [dpdk-dev] [PATCH 09/10] event/octeontx: add timer adapter SW traversal routine Pavan Nikhilesh ` (4 subsequent siblings) 12 siblings, 2 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-02-16 21:36 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Add compile-time configurable option to force TIMvf to use Octeontx FPAvf pool manager as its chunk pool. When FPAvf is used as pool manager the TIMvf automatically frees the chunks to FPAvf through gpool-id. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- config/common_base | 1 + drivers/event/octeontx/timvf_evdev.c | 23 +++++++++++++++++++++++ drivers/event/octeontx/timvf_evdev.h | 3 +++ drivers/event/octeontx/timvf_worker.h | 35 +++++++++++++++++++++++++++++++++++ 4 files changed, 62 insertions(+) diff --git a/config/common_base b/config/common_base index ad03cf433..00010de92 100644 --- a/config/common_base +++ b/config/common_base @@ -562,6 +562,7 @@ CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV=y # Compile PMD for octeontx sso event device # CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF=y +CONFIG_RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF=n # # Compile PMD for OPDL event device diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index ffdfbb387..386eaa08f 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -162,10 +162,27 @@ timvf_ring_start(const struct rte_event_timer_adapter *adptr) 1ull << 48 | 1ull << 47 | 1ull << 44 | +#ifndef RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF + 1ull << 43 | +#endif (timr->meta.nb_bkts - 1); rctrl.rctrl2 = (uint64_t)(TIM_CHUNK_SIZE / 16) << 40; +#ifdef RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF + uintptr_t pool; + pool = (uintptr_t)((struct rte_mempool *) + timr->meta.chunk_pool)->pool_id; + ret = octeontx_fpa_bufpool_gpool(pool); + if (ret < 0) { + timvf_log_dbg("Unable to get gaura id"); + ret = -ENOMEM; + goto error; + } + timvf_write64((uint64_t)ret, + (uint8_t *)timr->vbar0 + TIM_VRING_AURA); +#endif + timvf_write64((uint64_t)timr->meta.bkt, (uint8_t *)timr->vbar0 + TIM_VRING_BASE); if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) { @@ -296,9 +313,15 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) return -ENOMEM; } +#ifdef RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF + ret = rte_mempool_set_ops_byname(timr->meta.chunk_pool, + "octeontx_fpavf", NULL); + timvf_log_dbg("Giving back chunks to fpa gaura : %d", ret); +#else ret = rte_mempool_set_ops_byname(timr->meta.chunk_pool, RTE_MBUF_DEFAULT_MEMPOOL_OPS, NULL); timvf_log_dbg("Not giving back chunks to fpa"); +#endif if (ret != 0) { timvf_log_err("Unable to set chunkpool ops."); diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index 5e526a36a..02bd99a34 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -24,6 +24,9 @@ #include <rte_reciprocal.h> #include <octeontx_mbox.h> +#ifdef RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF +#include <octeontx_fpavf.h> +#endif #define timvf_log(level, fmt, args...) \ rte_log(RTE_LOG_ ## level, otx_logtype_timvf, \ diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h index 320eb6ac1..c3f37372a 100644 --- a/drivers/event/octeontx/timvf_worker.h +++ b/drivers/event/octeontx/timvf_worker.h @@ -144,6 +144,7 @@ timr_bkt_clr_nent(struct tim_mem_bucket *bktp) return __atomic_and_fetch((uint64_t *)&bktp->w1, v, __ATOMIC_ACQ_REL); } +#ifndef RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF static inline __hot struct tim_mem_entry* timr_clr_bkt(struct timvf_ring *timr, struct tim_mem_bucket *bkt) { @@ -159,6 +160,7 @@ timr_clr_bkt(struct timvf_ring *timr, struct tim_mem_bucket *bkt) } return (struct tim_mem_entry *)bkt->first_chunk; } +#endif /* Burst mode functions */ static inline int __hot @@ -241,7 +243,16 @@ timvf_add_entry_brst(struct timvf_ring *timr, const uint16_t rel_bkt, bkt->first_chunk = (uint64_t) chunk; } } else { +#ifndef RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF chunk = timr_clr_bkt(timr, bkt); +#else + if (unlikely(rte_mempool_get(timr->meta.chunk_pool, + (void **)&chunk))) { + timr_bkt_set_rem(bkt, 0); + tim[index]->state = RTE_EVENT_TIMER_ERROR; + return -ENOMEM; + } +#endif bkt->first_chunk = (uint64_t) chunk; } *(uint64_t *)(chunk + nb_chunk_slots) = 0; @@ -355,7 +366,18 @@ timvf_add_entry_sp(struct timvf_ring *timr, const uint32_t rel_bkt, } *(uint64_t *)(chunk + nb_chunk_slots) = 0; } else { +#ifndef RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF chunk = timr_clr_bkt(timr, bkt); +#else + if (unlikely(rte_mempool_get(timr->meta.chunk_pool, + (void **)&chunk))) { + timr_bkt_set_rem(bkt, 0); + tim->impl_opaque[0] = + tim->impl_opaque[1] = 0; + tim->state = RTE_EVENT_TIMER_ERROR; + return -ENOMEM; + } +#endif *(uint64_t *)(chunk + nb_chunk_slots) = 0; bkt->first_chunk = (uint64_t) chunk; } @@ -438,7 +460,20 @@ timvf_add_entry_mp(struct timvf_ring *timr, const uint32_t rel_bkt, } *(uint64_t *)(chunk + nb_chunk_slots) = 0; } else { +#ifndef RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF chunk = timr_clr_bkt(timr, bkt); +#else + if (unlikely(rte_mempool_get( + timr->meta.chunk_pool, + (void **)&chunk))) { + timr_bkt_set_rem(bkt, 0); + timr_bkt_dec_lock(bkt); + tim->impl_opaque[0] = + tim->impl_opaque[1] = 0; + tim->state = RTE_EVENT_TIMER_ERROR; + return -ENOMEM; + } +#endif *(uint64_t *)(chunk + nb_chunk_slots) = 0; bkt->first_chunk = (uint64_t) chunk; } -- 2.16.1 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH 08/10] event/octeontx: add option to use fpavf as chunk pool 2018-02-16 21:36 ` [dpdk-dev] [PATCH 08/10] event/octeontx: add option to use fpavf as chunk pool Pavan Nikhilesh @ 2018-02-18 11:42 ` santosh 2018-02-19 9:15 ` Pavan Nikhilesh 2018-02-23 20:17 ` Carrillo, Erik G 1 sibling, 1 reply; 76+ messages in thread From: santosh @ 2018-02-18 11:42 UTC (permalink / raw) To: Pavan Nikhilesh, jerin.jacob, erik.g.carrillo; +Cc: dev Hi Pavan, On Saturday 17 February 2018 03:06 AM, Pavan Nikhilesh wrote: > Add compile-time configurable option to force TIMvf to use Octeontx > FPAvf pool manager as its chunk pool. > When FPAvf is used as pool manager the TIMvf automatically frees the > chunks to FPAvf through gpool-id. > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > config/common_base | 1 + > drivers/event/octeontx/timvf_evdev.c | 23 +++++++++++++++++++++++ > drivers/event/octeontx/timvf_evdev.h | 3 +++ > drivers/event/octeontx/timvf_worker.h | 35 +++++++++++++++++++++++++++++++++++ > 4 files changed, 62 insertions(+) > > diff --git a/config/common_base b/config/common_base > index ad03cf433..00010de92 100644 > --- a/config/common_base > +++ b/config/common_base > @@ -562,6 +562,7 @@ CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV=y > # Compile PMD for octeontx sso event device > # > CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF=y > +CONFIG_RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF=n > How about using CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL? > # > # Compile PMD for OPDL event device > diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c > index ffdfbb387..386eaa08f 100644 > --- a/drivers/event/octeontx/timvf_evdev.c > +++ b/drivers/event/octeontx/timvf_evdev.c > @@ -162,10 +162,27 @@ timvf_ring_start(const struct rte_event_timer_adapter *adptr) > 1ull << 48 | > 1ull << 47 | > 1ull << 44 | > +#ifndef RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF > + 1ull << 43 | > +#endif > (timr->meta.nb_bkts - 1); > > rctrl.rctrl2 = (uint64_t)(TIM_CHUNK_SIZE / 16) << 40; > > +#ifdef RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF > + uintptr_t pool; > + pool = (uintptr_t)((struct rte_mempool *) > + timr->meta.chunk_pool)->pool_id; > + ret = octeontx_fpa_bufpool_gpool(pool); > + if (ret < 0) { > + timvf_log_dbg("Unable to get gaura id"); > + ret = -ENOMEM; > + goto error; > + } > + timvf_write64((uint64_t)ret, > + (uint8_t *)timr->vbar0 + TIM_VRING_AURA); > +#endif > + > timvf_write64((uint64_t)timr->meta.bkt, > (uint8_t *)timr->vbar0 + TIM_VRING_BASE); > if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) { > @@ -296,9 +313,15 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) > return -ENOMEM; > } > > +#ifdef RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF > + ret = rte_mempool_set_ops_byname(timr->meta.chunk_pool, > + "octeontx_fpavf", NULL); > + timvf_log_dbg("Giving back chunks to fpa gaura : %d", ret); > +#else > ret = rte_mempool_set_ops_byname(timr->meta.chunk_pool, > RTE_MBUF_DEFAULT_MEMPOOL_OPS, NULL); > timvf_log_dbg("Not giving back chunks to fpa"); > +#endif > May be use rte_mbuf_best_mempool_ops Or rte_mbuf_user_mempool_ops, that way avoid above ifdef. Also a suggestion, Try to reduce ifdef by creating a new header file called timvf_fpa_evdev.h, abstract all possible ifdefs their, create small static inline API and call them in timvf eventdev driver. [...] Thanks. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH 08/10] event/octeontx: add option to use fpavf as chunk pool 2018-02-18 11:42 ` santosh @ 2018-02-19 9:15 ` Pavan Nikhilesh 0 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-02-19 9:15 UTC (permalink / raw) To: santosh, jerin.jacob, erik.g.carrillo; +Cc: dev Hi Santosh, On Sun, Feb 18, 2018 at 05:12:33PM +0530, santosh wrote: > Hi Pavan, > > > On Saturday 17 February 2018 03:06 AM, Pavan Nikhilesh wrote: > > Add compile-time configurable option to force TIMvf to use Octeontx > > FPAvf pool manager as its chunk pool. > > When FPAvf is used as pool manager the TIMvf automatically frees the > > chunks to FPAvf through gpool-id. > > > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > > --- > > config/common_base | 1 + > > drivers/event/octeontx/timvf_evdev.c | 23 +++++++++++++++++++++++ > > drivers/event/octeontx/timvf_evdev.h | 3 +++ > > drivers/event/octeontx/timvf_worker.h | 35 +++++++++++++++++++++++++++++++++++ > > 4 files changed, 62 insertions(+) > > > > diff --git a/config/common_base b/config/common_base > > index ad03cf433..00010de92 100644 > > --- a/config/common_base > > +++ b/config/common_base > > @@ -562,6 +562,7 @@ CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV=y > > # Compile PMD for octeontx sso event device > > # > > CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF=y > > +CONFIG_RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF=n > > > > How about using CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL? I think CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL is always enabled by default,using something like CONFIG_RTE_LIBRTE_TIMVF_USE_OCTEONTX_MEMPOOL be better? i.e telling TIMvf to specifically use fpa. > > > # > > # Compile PMD for OPDL event device > > diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c > > index ffdfbb387..386eaa08f 100644 > > --- a/drivers/event/octeontx/timvf_evdev.c > > +++ b/drivers/event/octeontx/timvf_evdev.c > > @@ -162,10 +162,27 @@ timvf_ring_start(const struct rte_event_timer_adapter *adptr) > > 1ull << 48 | > > 1ull << 47 | > > 1ull << 44 | > > +#ifndef RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF > > + 1ull << 43 | > > +#endif > > (timr->meta.nb_bkts - 1); > > > > rctrl.rctrl2 = (uint64_t)(TIM_CHUNK_SIZE / 16) << 40; > > > > +#ifdef RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF > > + uintptr_t pool; > > + pool = (uintptr_t)((struct rte_mempool *) > > + timr->meta.chunk_pool)->pool_id; > > + ret = octeontx_fpa_bufpool_gpool(pool); > > + if (ret < 0) { > > + timvf_log_dbg("Unable to get gaura id"); > > + ret = -ENOMEM; > > + goto error; > > + } > > + timvf_write64((uint64_t)ret, > > + (uint8_t *)timr->vbar0 + TIM_VRING_AURA); > > +#endif > > + > > timvf_write64((uint64_t)timr->meta.bkt, > > (uint8_t *)timr->vbar0 + TIM_VRING_BASE); > > if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) { > > @@ -296,9 +313,15 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) > > return -ENOMEM; > > } > > > > +#ifdef RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF > > + ret = rte_mempool_set_ops_byname(timr->meta.chunk_pool, > > + "octeontx_fpavf", NULL); > > + timvf_log_dbg("Giving back chunks to fpa gaura : %d", ret); > > +#else > > ret = rte_mempool_set_ops_byname(timr->meta.chunk_pool, > > RTE_MBUF_DEFAULT_MEMPOOL_OPS, NULL); > > timvf_log_dbg("Not giving back chunks to fpa"); > > +#endif > > > > May be use rte_mbuf_best_mempool_ops Or rte_mbuf_user_mempool_ops, > that way avoid above ifdef. In timvf_worker.h the logic used to arm the event timer changes with the mempool used i.e. octeontx_fpavf has a different implementation vs ring. We could use rte_mbuf_best_mempool_ops() but again need check if the returned ops is fpa to enable automatic buffer recycling. > > Also a suggestion, Try to reduce ifdef by creating a new header file > called timvf_fpa_evdev.h, abstract all possible ifdefs their, > create small static inline API and call them in timvf eventdev driver. > [...] Agreed, will modify arm logic to reduce ifdef clutter. > > Thanks. > Thanks, Pavan. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH 08/10] event/octeontx: add option to use fpavf as chunk pool 2018-02-16 21:36 ` [dpdk-dev] [PATCH 08/10] event/octeontx: add option to use fpavf as chunk pool Pavan Nikhilesh 2018-02-18 11:42 ` santosh @ 2018-02-23 20:17 ` Carrillo, Erik G 2018-02-26 19:25 ` Pavan Nikhilesh 1 sibling, 1 reply; 76+ messages in thread From: Carrillo, Erik G @ 2018-02-23 20:17 UTC (permalink / raw) To: Pavan Nikhilesh, jerin.jacob, santosh.shukla; +Cc: dev Hi Pavan, > -----Original Message----- > From: Pavan Nikhilesh [mailto:pbhagavatula@caviumnetworks.com] > Sent: Friday, February 16, 2018 3:37 PM > To: jerin.jacob@caviumnetworks.com; > santosh.shukla@caviumnetworks.com; Carrillo, Erik G > <erik.g.carrillo@intel.com> > Cc: dev@dpdk.org; Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH 08/10] event/octeontx: add option to use fpavf > as chunk pool > > Add compile-time configurable option to force TIMvf to use Octeontx FPAvf > pool manager as its chunk pool. > When FPAvf is used as pool manager the TIMvf automatically frees the > chunks to FPAvf through gpool-id. > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- <...snipped...> > @@ -241,7 +243,16 @@ timvf_add_entry_brst(struct timvf_ring *timr, const > uint16_t rel_bkt, > bkt->first_chunk = (uint64_t) chunk; > } > } else { > +#ifndef RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF > chunk = timr_clr_bkt(timr, bkt); > +#else > + if (unlikely(rte_mempool_get(timr- > >meta.chunk_pool, > + (void **)&chunk))) { > + timr_bkt_set_rem(bkt, 0); > + tim[index]->state = > RTE_EVENT_TIMER_ERROR; > + return -ENOMEM; You return a negative errno value here, but in this case the caller was expecting the number that succeeded. Regards, Gabriel > + } > +#endif > bkt->first_chunk = (uint64_t) chunk; > } > *(uint64_t *)(chunk + nb_chunk_slots) = 0; @@ -355,7 > +366,18 @@ timvf_add_entry_sp(struct timvf_ring *timr, const uint32_t <...snipped...> ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH 08/10] event/octeontx: add option to use fpavf as chunk pool 2018-02-23 20:17 ` Carrillo, Erik G @ 2018-02-26 19:25 ` Pavan Nikhilesh 0 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-02-26 19:25 UTC (permalink / raw) To: Carrillo, Erik G, jerin.jacob, santosh.shukla; +Cc: dev Hi Gabriel, On Fri, Feb 23, 2018 at 08:17:07PM +0000, Carrillo, Erik G wrote: > Hi Pavan, > > > -----Original Message----- > > From: Pavan Nikhilesh [mailto:pbhagavatula@caviumnetworks.com] > > Sent: Friday, February 16, 2018 3:37 PM > > To: jerin.jacob@caviumnetworks.com; > > santosh.shukla@caviumnetworks.com; Carrillo, Erik G > > <erik.g.carrillo@intel.com> > > Cc: dev@dpdk.org; Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > > Subject: [dpdk-dev] [PATCH 08/10] event/octeontx: add option to use fpavf > > as chunk pool > > > > Add compile-time configurable option to force TIMvf to use Octeontx FPAvf > > pool manager as its chunk pool. > > When FPAvf is used as pool manager the TIMvf automatically frees the > > chunks to FPAvf through gpool-id. > > > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > > --- > > <...snipped...> > > > @@ -241,7 +243,16 @@ timvf_add_entry_brst(struct timvf_ring *timr, const > > uint16_t rel_bkt, > > bkt->first_chunk = (uint64_t) chunk; > > } > > } else { > > +#ifndef RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF > > chunk = timr_clr_bkt(timr, bkt); > > +#else > > + if (unlikely(rte_mempool_get(timr- > > >meta.chunk_pool, > > + (void **)&chunk))) { > > + timr_bkt_set_rem(bkt, 0); > > + tim[index]->state = > > RTE_EVENT_TIMER_ERROR; > > + return -ENOMEM; > > You return a negative errno value here, but in this case the caller was expecting the number that succeeded. Agreed, will add a check in `timvf_timer_reg_brst`[1] to check if call to timvf_add_entry_brst has failed. [1]http://dpdk.org/dev/patchwork/patch/35206/ > > Regards, > Gabriel Thanks, Pavan. > > > + } > > +#endif > > bkt->first_chunk = (uint64_t) chunk; > > } > > *(uint64_t *)(chunk + nb_chunk_slots) = 0; @@ -355,7 > > +366,18 @@ timvf_add_entry_sp(struct timvf_ring *timr, const uint32_t > > <...snipped...> > ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH 09/10] event/octeontx: add timer adapter SW traversal routine 2018-02-16 21:36 [dpdk-dev] [PATCH 0/9] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (7 preceding siblings ...) 2018-02-16 21:36 ` [dpdk-dev] [PATCH 08/10] event/octeontx: add option to use fpavf as chunk pool Pavan Nikhilesh @ 2018-02-16 21:36 ` Pavan Nikhilesh 2018-02-17 5:01 ` Jerin Jacob 2018-02-16 21:37 ` [dpdk-dev] [PATCH 10/10] maintainers: claim responsibility for octeontx timvf Pavan Nikhilesh ` (3 subsequent siblings) 12 siblings, 1 reply; 76+ messages in thread From: Pavan Nikhilesh @ 2018-02-16 21:36 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- config/common_base | 1 + drivers/event/octeontx/timvf_worker.h | 105 ++++++++++++++++++++++++++++++++++ 2 files changed, 106 insertions(+) diff --git a/config/common_base b/config/common_base index 00010de92..2f8c21871 100644 --- a/config/common_base +++ b/config/common_base @@ -563,6 +563,7 @@ CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV=y # CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF=y CONFIG_RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF=n +CONFIG_RTE_PMD_OCTEONTX_EVENT_TIMER_SW_TRAVERSAL=n # # Compile PMD for OPDL event device diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h index c3f37372a..1d5e27631 100644 --- a/drivers/event/octeontx/timvf_worker.h +++ b/drivers/event/octeontx/timvf_worker.h @@ -7,6 +7,23 @@ #include "timvf_evdev.h" +#ifdef RTE_PMD_OCTEONTX_EVENT_TIMER_SW_TRAVERSAL +#if defined(RTE_ARCH_ARM64) +#define timvf_store_pair(val0, val1, addr) ({ \ + asm volatile( \ + "stp %x[x0], %x[x1], [%x[p1]]" \ + ::[x0]"r"(val0), [x1]"r"(val1), [p1]"r"(addr) \ + ); }) +#else +#define timvf_store_pair(val0, val1, addr) \ +do { \ + rte_write64(val0, addr); \ + rte_write64(val1, (((uint8_t *)addr) + 8)); \ +} while (0) +#endif +#endif + + static inline int16_t timr_bkt_fetch_rem(uint64_t w1) { @@ -162,6 +179,49 @@ timr_clr_bkt(struct timvf_ring *timr, struct tim_mem_bucket *bkt) } #endif +#ifdef RTE_PMD_OCTEONTX_EVENT_TIMER_SW_TRAVERSAL +static __rte_always_inline void __hot +timvf_sw_trav(struct timvf_ring *timr, struct tim_mem_bucket *bkt) +{ + uint16_t i; + struct tim_mem_entry *chunk; + struct tim_mem_entry *wrk; + void *grp_addr; + chunk = (struct tim_mem_entry *)bkt->first_chunk; + timvf_log_info("Software traversing bucket."); + /* Only one sw thread can be here. */ + while (chunk) { + for (i = 0; i < nb_chunk_slots && bkt->nb_entry; i++) { + wrk = chunk + i; + if (!wrk->wqe) + continue; + grp_addr = octeontx_ssovf_bar(OCTEONTX_SSO_GROUP, + (uint8_t)((wrk->w0 >> 34) & 0xFF), 2); + timvf_store_pair(wrk->w0, wrk->wqe, grp_addr); + wrk->wqe = 0; + bkt->nb_entry--; + } + wrk = (struct tim_mem_entry *)((chunk + nb_chunk_slots)->w0); + rte_mempool_put(timr->meta.chunk_pool, chunk); + chunk = (struct tim_mem_entry *)wrk; + } +} +#endif + +static inline __hot void +timr_clr_bkt_full(struct timvf_ring *timr, struct tim_mem_bucket *bkt) +{ + struct tim_mem_entry *chunk; + struct tim_mem_entry *pnext; + chunk = (struct tim_mem_entry *)bkt->first_chunk; + + while (chunk) { + pnext = (struct tim_mem_entry *)((chunk + nb_chunk_slots)->w0); + rte_mempool_put(timr->meta.chunk_pool, chunk); + chunk = pnext; + } +} + /* Burst mode functions */ static inline int __hot timvf_add_entry_brst(struct timvf_ring *timr, const uint16_t rel_bkt, @@ -205,6 +265,20 @@ timvf_add_entry_brst(struct timvf_ring *timr, const uint16_t rel_bkt, goto __retry; } + /* Check for bsk & do software traversal. */ +#ifdef RTE_PMD_OCTEONTX_EVENT_TIMER_SW_TRAVERSAL + if (unlikely(timr_bkt_get_bsk(lock_sema))) { + lock_sema = timr_bkt_set_sbt(bkt); + if (unlikely(timr_bkt_get_sbt(lock_sema))) { + /* Should never hit. */ + goto __retry; + } + timvf_sw_trav(timr, bkt); + timr_bkt_clr_bsk(bkt); + goto __retry; + } +#endif + chunk_remainder = timr_bkt_fetch_rem(lock_sema); rem = chunk_remainder - nb_timers; if (rem < 0) { @@ -342,6 +416,18 @@ timvf_add_entry_sp(struct timvf_ring *timr, const uint32_t rel_bkt, if (unlikely(timr_bkt_get_shbt(lock_sema))) goto __retry; +#ifdef RTE_PMD_OCTEONTX_EVENT_TIMER_SW_TRAVERSAL + if (unlikely(timr_bkt_get_bsk(lock_sema))) { + lock_sema = timr_bkt_set_sbt(bkt); + if (unlikely(timr_bkt_get_sbt(lock_sema))) { + /* Should never hit. */ + goto __retry; + } + timvf_sw_trav(timr, bkt); + timr_bkt_clr_bsk(bkt); + goto __retry; + } +#endif /* Insert the work. */ rem = timr_bkt_fetch_rem(lock_sema); @@ -430,6 +516,25 @@ timvf_add_entry_mp(struct timvf_ring *timr, const uint32_t rel_bkt, } RTE_SET_USED(lock_cnt); +#ifdef RTE_PMD_OCTEONTX_EVENT_TIMER_SW_TRAVERSAL + lock_cnt = (uint8_t) + ((lock_sema >> TIM_BUCKET_W1_S_LOCK) & + TIM_BUCKET_W1_M_LOCK); + + if (unlikely(!lock_cnt && timr_bkt_get_bsk(lock_sema))) { + /* Only first thread processes the bucket.*/ + lock_sema = timr_bkt_set_sbt(bkt); + if (unlikely(timr_bkt_get_sbt(lock_sema))) { + timr_bkt_dec_lock(bkt); + goto __retry; + } + timvf_sw_trav(timr, bkt); + timr_bkt_clr_bsk(bkt); + timr_bkt_dec_lock(bkt); + continue; + } +#endif + /* Insert the work. */ rem = timr_bkt_fetch_rem(lock_sema); -- 2.16.1 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH 09/10] event/octeontx: add timer adapter SW traversal routine 2018-02-16 21:36 ` [dpdk-dev] [PATCH 09/10] event/octeontx: add timer adapter SW traversal routine Pavan Nikhilesh @ 2018-02-17 5:01 ` Jerin Jacob 0 siblings, 0 replies; 76+ messages in thread From: Jerin Jacob @ 2018-02-17 5:01 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Sat, 17 Feb 2018 03:06:59 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH 09/10] event/octeontx: add timer adapter SW > traversal routine > X-Mailer: git-send-email 2.16.1 > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > config/common_base | 1 + > drivers/event/octeontx/timvf_worker.h | 105 ++++++++++++++++++++++++++++++++++ > 2 files changed, 106 insertions(+) > > diff --git a/config/common_base b/config/common_base > index 00010de92..2f8c21871 100644 > --- a/config/common_base > +++ b/config/common_base > @@ -563,6 +563,7 @@ CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV=y > # > CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF=y > CONFIG_RTE_PMD_OCTEONTX_TIMVF_USE_FPAVF=n > +CONFIG_RTE_PMD_OCTEONTX_EVENT_TIMER_SW_TRAVERSAL=n Please update the driver details in doc/guides/eventdevs/octeontx.rst and the description of the config option in git commit as well the documentation. > > # > # Compile PMD for OPDL event device > diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h > index c3f37372a..1d5e27631 100644 > --- a/drivers/event/octeontx/timvf_worker.h > +++ b/drivers/event/octeontx/timvf_worker.h > @@ -7,6 +7,23 @@ > > #include "timvf_evdev.h" > > +#ifdef RTE_PMD_OCTEONTX_EVENT_TIMER_SW_TRAVERSAL > +#if defined(RTE_ARCH_ARM64) > +#define timvf_store_pair(val0, val1, addr) ({ \ > + asm volatile( \ > + "stp %x[x0], %x[x1], [%x[p1]]" \ > + ::[x0]"r"(val0), [x1]"r"(val1), [p1]"r"(addr) \ > + ); }) > +#else > +#define timvf_store_pair(val0, val1, addr) \ > +do { \ > + rte_write64(val0, addr); \ > + rte_write64(val1, (((uint8_t *)addr) + 8)); \ > +} while (0) > +#endif > +#endif How about #define timvf_store_pair ssovf_store_pair > + ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH 10/10] maintainers: claim responsibility for octeontx timvf 2018-02-16 21:36 [dpdk-dev] [PATCH 0/9] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (8 preceding siblings ...) 2018-02-16 21:36 ` [dpdk-dev] [PATCH 09/10] event/octeontx: add timer adapter SW traversal routine Pavan Nikhilesh @ 2018-02-16 21:37 ` Pavan Nikhilesh 2018-02-17 5:03 ` Jerin Jacob 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 00/11] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (2 subsequent siblings) 12 siblings, 1 reply; 76+ messages in thread From: Pavan Nikhilesh @ 2018-02-16 21:37 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- MAINTAINERS | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index a646ca3e1..35c3caeb7 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -762,6 +762,10 @@ M: Peter Mccarthy <peter.mccarthy@intel.com> F: drivers/event/opdl/ F: doc/guides/eventdevs/opdl.rst +Cavium OCTEONTX timvf +M: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> +F: drivers/event/octeontx/timvf_* + Packet processing ----------------- -- 2.16.1 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH 10/10] maintainers: claim responsibility for octeontx timvf 2018-02-16 21:37 ` [dpdk-dev] [PATCH 10/10] maintainers: claim responsibility for octeontx timvf Pavan Nikhilesh @ 2018-02-17 5:03 ` Jerin Jacob 0 siblings, 0 replies; 76+ messages in thread From: Jerin Jacob @ 2018-02-17 5:03 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Sat, 17 Feb 2018 03:07:00 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH 10/10] maintainers: claim responsibility for > octeontx timvf > X-Mailer: git-send-email 2.16.1 > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > MAINTAINERS | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/MAINTAINERS b/MAINTAINERS > index a646ca3e1..35c3caeb7 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -762,6 +762,10 @@ M: Peter Mccarthy <peter.mccarthy@intel.com> > F: drivers/event/opdl/ > F: doc/guides/eventdevs/opdl.rst > > +Cavium OCTEONTX timvf > +M: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > +F: drivers/event/octeontx/timvf_* > + Move this just below to "Cavium OCTEONTX ssovf" > > Packet processing > ----------------- > -- > 2.16.1 > ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v2 00/11] event/octeontx: add event timer adapter driver 2018-02-16 21:36 [dpdk-dev] [PATCH 0/9] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (9 preceding siblings ...) 2018-02-16 21:37 ` [dpdk-dev] [PATCH 10/10] maintainers: claim responsibility for octeontx timvf Pavan Nikhilesh @ 2018-03-14 13:52 ` Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 01/11] mempool/octeontx: probe timvf PCIe devices Pavan Nikhilesh ` (10 more replies) 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 00/11] " Pavan Nikhilesh 12 siblings, 11 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-03-14 13:52 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh The event timer adapter[1] provides APIs to configure an event timer device that allows an application to arm timers which on expiry push events to an event device such as OcteonTx SSO. The OcteonTx TIM is a co-processor that can be configured as an event timer adapter which can be used by an application to manage event timers. The TIM co-processor processes the event timers registered and pushes expired event timers to SSO based on the event queue, schedule type, flow id etc. provided as rte_event while arming the event timer. It maintains event timers with high precision and time granularity of 1us (microsecond). [1] http://dpdk.org/dev/patchwork/patch/33525/ This patch set depends on: 1. http://dpdk.org/dev/patchwork/bundle/pbhagavatula/event_timer_dev_v7 2. http://dpdk.org/dev/patchwork/patch/35216/ 3. http://dpdk.org/dev/patchwork/patch/36089/ v2 Changes: - Move common code changes to separate patches. - Reduce ifdef clutter. - fix improper return value (erik). - remove redundent ops assignment (erik). - update octeontx documentation. Pavan Nikhilesh (11): mempool/octeontx: probe timvf PCIe devices usertools: add Cavium TIM as an event device event/octeontx: add support to create and free timer adapter event/octeontx: add support to start and stop timer device event/octeontx: add multiproducer timer arm and cancel event/octeontx: add single producer timer arm variant event/octeontx: add burst mode for timer arm event/octeontx: optimize timer adapter resolution parameters event/octeontx: add option to use fpavf as chunk pool doc: update eventdev OcteonTx documentation maintainers: claim responsibility for octeontx timvf MAINTAINERS | 4 + doc/guides/eventdevs/octeontx.rst | 9 + drivers/event/octeontx/Makefile | 7 + drivers/event/octeontx/meson.build | 4 +- drivers/event/octeontx/ssovf_evdev.c | 3 + drivers/event/octeontx/timvf_evdev.c | 382 ++++++++++++++++++ drivers/event/octeontx/timvf_evdev.h | 214 ++++++++++ drivers/event/octeontx/timvf_worker.c | 127 ++++++ drivers/event/octeontx/timvf_worker.h | 445 +++++++++++++++++++++ drivers/mempool/octeontx/Makefile | 1 + drivers/mempool/octeontx/meson.build | 1 + drivers/mempool/octeontx/octeontx_mbox.h | 7 + drivers/mempool/octeontx/octeontx_timvf.c | 145 +++++++ .../octeontx/rte_mempool_octeontx_version.map | 3 + usertools/dpdk-devbind.py | 4 +- 15 files changed, 1354 insertions(+), 2 deletions(-) create mode 100644 drivers/event/octeontx/timvf_evdev.c create mode 100644 drivers/event/octeontx/timvf_evdev.h create mode 100644 drivers/event/octeontx/timvf_worker.c create mode 100644 drivers/event/octeontx/timvf_worker.h create mode 100644 drivers/mempool/octeontx/octeontx_timvf.c -- 2.16.2 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v2 01/11] mempool/octeontx: probe timvf PCIe devices 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 00/11] event/octeontx: add event timer adapter driver Pavan Nikhilesh @ 2018-03-14 13:52 ` Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 02/11] usertools: add Cavium TIM as an event device Pavan Nikhilesh ` (9 subsequent siblings) 10 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-03-14 13:52 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh On Octeontx HW, each event timer device is enumerated as separate SRIOV VF PCIe device. In order to expose as a event timer device: On PCIe probe, the driver stores the information associated with the PCIe device and later when appliacation requests for a event timer device through `rte_event_timer_adapter_create` the driver infrastructure creates the timer adapter with earlier probed PCIe VF devices. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/mempool/octeontx/Makefile | 1 + drivers/mempool/octeontx/meson.build | 1 + drivers/mempool/octeontx/octeontx_mbox.h | 7 + drivers/mempool/octeontx/octeontx_timvf.c | 145 +++++++++++++++++++++ .../octeontx/rte_mempool_octeontx_version.map | 3 + 5 files changed, 157 insertions(+) create mode 100644 drivers/mempool/octeontx/octeontx_timvf.c diff --git a/drivers/mempool/octeontx/Makefile b/drivers/mempool/octeontx/Makefile index dfc373e62..a0547437a 100644 --- a/drivers/mempool/octeontx/Makefile +++ b/drivers/mempool/octeontx/Makefile @@ -20,6 +20,7 @@ LIBABIVER := 1 SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += octeontx_ssovf.c SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += octeontx_mbox.c SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += octeontx_fpavf.c +SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += octeontx_timvf.c SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += rte_mempool_octeontx.c ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) diff --git a/drivers/mempool/octeontx/meson.build b/drivers/mempool/octeontx/meson.build index 1e894a56b..c938436c9 100644 --- a/drivers/mempool/octeontx/meson.build +++ b/drivers/mempool/octeontx/meson.build @@ -4,6 +4,7 @@ sources = files('octeontx_ssovf.c', 'octeontx_mbox.c', 'octeontx_fpavf.c', + 'octeontx_timvf.c', 'rte_mempool_octeontx.c' ) diff --git a/drivers/mempool/octeontx/octeontx_mbox.h b/drivers/mempool/octeontx/octeontx_mbox.h index 1b056071a..7fcc9950f 100644 --- a/drivers/mempool/octeontx/octeontx_mbox.h +++ b/drivers/mempool/octeontx/octeontx_mbox.h @@ -21,6 +21,11 @@ enum octeontx_ssovf_type { OCTEONTX_SSO_HWS, /* SSO hardware workslot vf */ }; +struct octeontx_timvf_info { + uint16_t domain; /* Domain id */ + uint8_t total_timvfs; /* Total timvf available in domain */ +}; + struct octeontx_mbox_hdr { uint16_t vfid; /* VF index or pf resource index local to the domain */ uint8_t coproc; /* Coprocessor id */ @@ -32,5 +37,7 @@ int octeontx_ssovf_info(struct octeontx_ssovf_info *info); void *octeontx_ssovf_bar(enum octeontx_ssovf_type, uint8_t id, uint8_t bar); int octeontx_ssovf_mbox_send(struct octeontx_mbox_hdr *hdr, void *txdata, uint16_t txlen, void *rxdata, uint16_t rxlen); +int octeontx_timvf_info(struct octeontx_timvf_info *info); +void *octeontx_timvf_bar(uint8_t id, uint8_t bar); #endif /* __OCTEONTX_MBOX_H__ */ diff --git a/drivers/mempool/octeontx/octeontx_timvf.c b/drivers/mempool/octeontx/octeontx_timvf.c new file mode 100644 index 000000000..b20743a33 --- /dev/null +++ b/drivers/mempool/octeontx/octeontx_timvf.c @@ -0,0 +1,145 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include <rte_eal.h> +#include <rte_io.h> +#include <rte_pci.h> +#include <rte_bus_pci.h> + +#include "octeontx_mbox.h" +#include "octeontx_pool_logs.h" + +#ifndef PCI_VENDOR_ID_CAVIUM +#define PCI_VENDOR_ID_CAVIUM (0x177D) +#endif + +#define PCI_DEVICE_ID_OCTEONTX_TIM_VF (0xA051) +#define TIM_MAX_RINGS (64) + +struct timvf_res { + uint16_t domain; + uint16_t vfid; + void *bar0; + void *bar2; + void *bar4; +}; + +struct timdev { + uint8_t total_timvfs; + struct timvf_res rings[TIM_MAX_RINGS]; +}; + +static struct timdev tdev; + +int +octeontx_timvf_info(struct octeontx_timvf_info *tinfo) +{ + int i; + struct octeontx_ssovf_info info; + + if (tinfo == NULL) + return -EINVAL; + + if (!tdev.total_timvfs) + return -ENODEV; + + if (octeontx_ssovf_info(&info) < 0) + return -EINVAL; + + for (i = 0; i < tdev.total_timvfs; i++) { + if (info.domain != tdev.rings[i].domain) { + mbox_log_err("GRP error, vfid=%d/%d domain=%d/%d %p", + i, tdev.rings[i].vfid, + info.domain, tdev.rings[i].domain, + tdev.rings[i].bar0); + return -EINVAL; + } + } + + tinfo->total_timvfs = tdev.total_timvfs; + tinfo->domain = info.domain; + return 0; +} + +void* +octeontx_timvf_bar(uint8_t id, uint8_t bar) +{ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return NULL; + + if (id > tdev.total_timvfs) + return NULL; + + switch (bar) { + case 0: + return tdev.rings[id].bar0; + case 4: + return tdev.rings[id].bar4; + default: + return NULL; + } +} + +static int +timvf_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + uint64_t val; + uint16_t vfid; + struct timvf_res *res; + + RTE_SET_USED(pci_drv); + + /* For secondary processes, the primary has done all the work */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + if (pci_dev->mem_resource[0].addr == NULL || + pci_dev->mem_resource[4].addr == NULL) { + mbox_log_err("Empty bars %p %p", + pci_dev->mem_resource[0].addr, + pci_dev->mem_resource[4].addr); + return -ENODEV; + } + + val = rte_read64((uint8_t *)pci_dev->mem_resource[0].addr + 0x100); + vfid = (val >> 23) & 0xff; + if (vfid >= TIM_MAX_RINGS) { + mbox_log_err("Invalid vfid(%d/%d)", vfid, TIM_MAX_RINGS); + return -EINVAL; + } + + res = &tdev.rings[tdev.total_timvfs]; + res->vfid = vfid; + res->bar0 = pci_dev->mem_resource[0].addr; + res->bar2 = pci_dev->mem_resource[2].addr; + res->bar4 = pci_dev->mem_resource[4].addr; + res->domain = (val >> 7) & 0xffff; + tdev.total_timvfs++; + rte_wmb(); + + mbox_log_dbg("Domain=%d VFid=%d bar0 %p total_timvfs=%d", res->domain, + res->vfid, pci_dev->mem_resource[0].addr, + tdev.total_timvfs); + return 0; +} + + +static const struct rte_pci_id pci_timvf_map[] = { + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, + PCI_DEVICE_ID_OCTEONTX_TIM_VF) + }, + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver pci_timvf = { + .id_table = pci_timvf_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_IOVA_AS_VA, + .probe = timvf_probe, + .remove = NULL, +}; + +RTE_PMD_REGISTER_PCI(octeontx_timvf, pci_timvf); diff --git a/drivers/mempool/octeontx/rte_mempool_octeontx_version.map b/drivers/mempool/octeontx/rte_mempool_octeontx_version.map index fe8cdeca0..91e0a3a93 100644 --- a/drivers/mempool/octeontx/rte_mempool_octeontx_version.map +++ b/drivers/mempool/octeontx/rte_mempool_octeontx_version.map @@ -5,5 +5,8 @@ DPDK_17.11 { octeontx_ssovf_bar; octeontx_ssovf_mbox_send; + octeontx_timvf_info; + octeontx_timvf_bar; + local: *; }; -- 2.16.2 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v2 02/11] usertools: add Cavium TIM as an event device 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 00/11] event/octeontx: add event timer adapter driver Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 01/11] mempool/octeontx: probe timvf PCIe devices Pavan Nikhilesh @ 2018-03-14 13:52 ` Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 03/11] event/octeontx: add support to create and free timer adapter Pavan Nikhilesh ` (8 subsequent siblings) 10 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-03-14 13:52 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- usertools/dpdk-devbind.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py index 18d938607..f6cbb2bb7 100755 --- a/usertools/dpdk-devbind.py +++ b/usertools/dpdk-devbind.py @@ -22,10 +22,12 @@ 'SVendor': None, 'SDevice': None} cavium_pkx = {'Class': '08', 'Vendor': '177d', 'Device': 'a0dd,a049', 'SVendor': None, 'SDevice': None} +cavium_tim = {'Class': '08', 'Vendor': '177d', 'Device': 'a051', + 'SVendor': None, 'SDevice': None} network_devices = [network_class, cavium_pkx] crypto_devices = [encryption_class, intel_processor_class] -eventdev_devices = [cavium_sso] +eventdev_devices = [cavium_sso, cavium_tim] mempool_devices = [cavium_fpa] # global dict ethernet devices present. Dictionary indexed by PCI address. -- 2.16.2 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v2 03/11] event/octeontx: add support to create and free timer adapter 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 00/11] event/octeontx: add event timer adapter driver Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 01/11] mempool/octeontx: probe timvf PCIe devices Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 02/11] usertools: add Cavium TIM as an event device Pavan Nikhilesh @ 2018-03-14 13:52 ` Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 04/11] event/octeontx: add support to start and stop timer device Pavan Nikhilesh ` (7 subsequent siblings) 10 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-03-14 13:52 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh When the application requests to create a timer device, Octeontx TIM create does the following: - Get the requested TIMvf ring based on adapter_id. - Verify the config parameters supplied. - Allocate memory required for * Buckets based on min and max timeout supplied. * Allocate the chunk pool based on the number of timers. - Clear the interrupts. On Free: - Free the allocated bucket and chunk memory. - Free private data used by TIMvf. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/Makefile | 2 + drivers/event/octeontx/meson.build | 3 +- drivers/event/octeontx/ssovf_evdev.c | 3 + drivers/event/octeontx/timvf_evdev.c | 161 ++++++++++++++++++++++++++++++++++ drivers/event/octeontx/timvf_evdev.h | 165 +++++++++++++++++++++++++++++++++++ 5 files changed, 333 insertions(+), 1 deletion(-) create mode 100644 drivers/event/octeontx/timvf_evdev.c create mode 100644 drivers/event/octeontx/timvf_evdev.h diff --git a/drivers/event/octeontx/Makefile b/drivers/event/octeontx/Makefile index 0e49efd84..570a79412 100644 --- a/drivers/event/octeontx/Makefile +++ b/drivers/event/octeontx/Makefile @@ -12,6 +12,7 @@ LIB = librte_pmd_octeontx_ssovf.a CFLAGS += $(WERROR_FLAGS) CFLAGS += -I$(RTE_SDK)/drivers/mempool/octeontx/ CFLAGS += -I$(RTE_SDK)/drivers/net/octeontx/ +CFLAGS += -DALLOW_EXPERIMENTAL_API LDLIBS += -lrte_eal -lrte_eventdev -lrte_mempool_octeontx -lrte_pmd_octeontx LDLIBS += -lrte_bus_pci -lrte_mempool -lrte_mbuf -lrte_kvargs @@ -27,6 +28,7 @@ LIBABIVER := 1 SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_worker.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_evdev.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_evdev_selftest.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += timvf_evdev.c ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_ssovf_worker.o += -fno-prefetch-loop-arrays diff --git a/drivers/event/octeontx/meson.build b/drivers/event/octeontx/meson.build index 358fc9fc9..8941f8a56 100644 --- a/drivers/event/octeontx/meson.build +++ b/drivers/event/octeontx/meson.build @@ -3,7 +3,8 @@ sources = files('ssovf_worker.c', 'ssovf_evdev.c', - 'ssovf_evdev_selftest.c' + 'ssovf_evdev_selftest.c', + 'timvf_evdev.c', ) deps += ['mempool_octeontx', 'bus_vdev', 'pmd_octeontx'] diff --git a/drivers/event/octeontx/ssovf_evdev.c b/drivers/event/octeontx/ssovf_evdev.c index a1086077d..54384d465 100644 --- a/drivers/event/octeontx/ssovf_evdev.c +++ b/drivers/event/octeontx/ssovf_evdev.c @@ -18,6 +18,7 @@ #include <rte_bus_vdev.h> #include "ssovf_evdev.h" +#include "timvf_evdev.h" int otx_logtype_ssovf; @@ -610,6 +611,8 @@ static const struct rte_eventdev_ops ssovf_ops = { .eth_rx_adapter_start = ssovf_eth_rx_adapter_start, .eth_rx_adapter_stop = ssovf_eth_rx_adapter_stop, + .timer_adapter_caps_get = timvf_timer_adapter_caps_get, + .dev_selftest = test_eventdev_octeontx, .dump = ssovf_dump, diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c new file mode 100644 index 000000000..473311fa4 --- /dev/null +++ b/drivers/event/octeontx/timvf_evdev.c @@ -0,0 +1,161 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include "timvf_evdev.h" + +int otx_logtype_timvf; + +RTE_INIT(otx_timvf_init_log); +static void +otx_timvf_init_log(void) +{ + otx_logtype_timvf = rte_log_register("pmd.event.octeontx.timer"); + if (otx_logtype_timvf >= 0) + rte_log_set_level(otx_logtype_timvf, RTE_LOG_NOTICE); +} + +static void +timvf_ring_info_get(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer_adapter_info *adptr_info) +{ + struct timvf_ring *timr = adptr->data->adapter_priv; + adptr_info->max_tmo_ns = timr->max_tout; + adptr_info->min_resolution_ns = timr->tck_nsec; + rte_memcpy(&adptr_info->conf, &adptr->data->conf, + sizeof(struct rte_event_timer_adapter_conf)); +} + +static int +timvf_ring_create(struct rte_event_timer_adapter *adptr) +{ + char pool_name[25]; + int ret; + uint64_t nb_timers; + struct rte_event_timer_adapter_conf *rcfg = &adptr->data->conf; + struct timvf_ring *timr; + struct octeontx_timvf_info tinfo; + const char *mempool_ops; + + if (octeontx_timvf_info(&tinfo) < 0) + return -ENODEV; + + if (adptr->data->id >= tinfo.total_timvfs) + return -ENODEV; + + timr = rte_zmalloc("octeontx_timvf_priv", + sizeof(struct timvf_ring), 0); + if (timr == NULL) + return -ENOMEM; + + adptr->data->adapter_priv = timr; + /* Check config parameters. */ + if ((rcfg->clk_src != RTE_EVENT_TIMER_ADAPTER_EXT_CLK0) && + (!rcfg->timer_tick_ns || + rcfg->timer_tick_ns < TIM_MIN_INTERVAL)) { + timvf_log_err("Too low timer ticks"); + goto cfg_err; + } + + switch (rcfg->clk_src) { + case RTE_EVENT_TIMER_ADAPTER_CPU_CLK: + timr->clk_src = TIM_CLK_SRC_SCLK; + break; + case RTE_EVENT_TIMER_ADAPTER_EXT_CLK0: + timr->clk_src = TIM_CLK_SRC_GPIO; + break; + case RTE_EVENT_TIMER_ADAPTER_EXT_CLK1: + timr->clk_src = TIM_CLK_SRC_GTI; + break; + case RTE_EVENT_TIMER_ADAPTER_EXT_CLK2: + timr->clk_src = TIM_CLK_SRC_PTP; + break; + default: + timvf_log_err("Invalid clk source specified."); + goto cfg_err; + } + + timr->tim_ring_id = adptr->data->id; + timr->tck_nsec = rcfg->timer_tick_ns; + timr->max_tout = rcfg->max_tmo_ns; + timr->meta.nb_bkts = (timr->max_tout / timr->tck_nsec) + 1; + timr->vbar0 = octeontx_timvf_bar(timr->tim_ring_id, 0); + timr->bkt_pos = (uint8_t *)timr->vbar0 + TIM_VRING_REL; + nb_timers = rcfg->nb_timers; + timr->meta.get_target_bkt = bkt_mod; + + timr->nb_chunks = nb_timers / nb_chunk_slots; + + timr->meta.bkt = rte_zmalloc("octeontx_timvf_bucket", + (timr->meta.nb_bkts) * sizeof(struct tim_mem_bucket), + 0); + if (timr->meta.bkt == NULL) + goto mem_err; + + snprintf(pool_name, 30, "timvf_meta.chunk_pool%d", timr->tim_ring_id); + timr->meta.chunk_pool = (void *)rte_mempool_create_empty(pool_name, + timr->nb_chunks, TIM_CHUNK_SIZE, 0, 0, rte_socket_id(), + 0); + + if (!timr->meta.chunk_pool) { + rte_free(timr->meta.bkt); + timvf_log_err("Unable to create chunkpool."); + return -ENOMEM; + } + + mempool_ops = rte_mbuf_best_mempool_ops(); + ret = rte_mempool_set_ops_byname(timr->meta.chunk_pool, + mempool_ops, NULL); + + if (ret != 0) { + timvf_log_err("Unable to set chunkpool ops."); + goto mem_err; + } + + ret = rte_mempool_populate_default(timr->meta.chunk_pool); + if (ret < 0) { + timvf_log_err("Unable to set populate chunkpool."); + goto mem_err; + } + timvf_write64(0, (uint8_t *)timr->vbar0 + TIM_VRING_BASE); + timvf_write64(0, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_INT); + timvf_write64(0, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_INT_W1S); + timvf_write64(0x7, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_ENA_W1C); + timvf_write64(0x7, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_ENA_W1S); + + return 0; +mem_err: + rte_free(timr); + return -ENOMEM; +cfg_err: + rte_free(timr); + return -EINVAL; +} + +static int +timvf_ring_free(struct rte_event_timer_adapter *adptr) +{ + struct timvf_ring *timr = adptr->data->adapter_priv; + rte_mempool_free(timr->meta.chunk_pool); + rte_free(timr->meta.bkt); + rte_free(adptr->data->adapter_priv); + return 0; +} + +static struct rte_event_timer_adapter_ops timvf_ops = { + .init = timvf_ring_create, + .uninit = timvf_ring_free, + .get_info = timvf_ring_info_get, +}; + +int +timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, + uint32_t *caps, const struct rte_event_timer_adapter_ops **ops) +{ + RTE_SET_USED(dev); + RTE_SET_USED(flags); + *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; + *ops = &timvf_ops; + return -EINVAL; +} diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h new file mode 100644 index 000000000..e3f558e10 --- /dev/null +++ b/drivers/event/octeontx/timvf_evdev.h @@ -0,0 +1,165 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#ifndef __TIMVF_EVDEV_H__ +#define __TIMVF_EVDEV_H__ + +#include <rte_common.h> +#include <rte_cycles.h> +#include <rte_debug.h> +#include <rte_eal.h> +#include <rte_eventdev.h> +#include <rte_event_timer_adapter.h> +#include <rte_event_timer_adapter_pmd.h> +#include <rte_io.h> +#include <rte_lcore.h> +#include <rte_log.h> +#include <rte_malloc.h> +#include <rte_mbuf_pool_ops.h> +#include <rte_mempool.h> +#include <rte_memzone.h> +#include <rte_pci.h> +#include <rte_prefetch.h> +#include <rte_reciprocal.h> + +#include <octeontx_mbox.h> + +#define timvf_log(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, otx_logtype_timvf, \ + "[%s] %s() " fmt "\n", \ + RTE_STR(event_timer_octeontx), __func__, ## args) + +#define timvf_log_info(fmt, ...) timvf_log(INFO, fmt, ##__VA_ARGS__) +#define timvf_log_dbg(fmt, ...) timvf_log(DEBUG, fmt, ##__VA_ARGS__) +#define timvf_log_err(fmt, ...) timvf_log(ERR, fmt, ##__VA_ARGS__) +#define timvf_func_trace timvf_log_dbg + +#define TIM_COPROC (8) +#define TIM_GET_DEV_INFO (1) +#define TIM_GET_RING_INFO (2) +#define TIM_SET_RING_INFO (3) +#define TIM_RING_START_CYC_GET (4) + +#define TIM_MAX_RINGS (64) +#define TIM_DEV_PER_NODE (1) +#define TIM_VF_PER_DEV (64) +#define TIM_RING_PER_DEV (TIM_VF_PER_DEV) +#define TIM_RING_NODE_SHIFT (6) +#define TIM_RING_MASK ((TIM_RING_PER_DEV) - 1) +#define TIM_RING_INVALID (-1) + +#define TIM_MIN_INTERVAL (1E3) +#define TIM_MAX_INTERVAL ((1ull << 32) - 1) +#define TIM_MAX_BUCKETS (1ull << 20) +#define TIM_CHUNK_SIZE (4096) +#define TIM_MAX_CHUNKS_PER_BUCKET (1ull << 32) + +#define TIMVF_MAX_BURST (8) + +/* TIM VF Control/Status registers (CSRs): */ +/* VF_BAR0: */ +#define TIM_VF_NRSPERR_INT (0x0) +#define TIM_VF_NRSPERR_INT_W1S (0x8) +#define TIM_VF_NRSPERR_ENA_W1C (0x10) +#define TIM_VF_NRSPERR_ENA_W1S (0x18) +#define TIM_VRING_FR_RN_CYCLES (0x20) +#define TIM_VRING_FR_RN_GPIOS (0x28) +#define TIM_VRING_FR_RN_GTI (0x30) +#define TIM_VRING_FR_RN_PTP (0x38) +#define TIM_VRING_CTL0 (0x40) +#define TIM_VRING_CTL1 (0x50) +#define TIM_VRING_CTL2 (0x60) +#define TIM_VRING_BASE (0x100) +#define TIM_VRING_AURA (0x108) +#define TIM_VRING_REL (0x110) + +#define timvf_read64 rte_read64_relaxed +#define timvf_write64 rte_write64_relaxed + +#ifndef __hot +#define __hot __attribute__((hot)) +#endif + +extern int otx_logtype_timvf; + +static const uint16_t nb_chunk_slots = (TIM_CHUNK_SIZE / 16) - 1; + +enum timvf_clk_src { + TIM_CLK_SRC_SCLK, + TIM_CLK_SRC_GPIO, + TIM_CLK_SRC_GTI, + TIM_CLK_SRC_PTP, +}; + +/* TIM_MEM_BUCKET */ +struct tim_mem_bucket { + uint64_t first_chunk; + union { + uint64_t w1; + struct { + uint32_t nb_entry; + uint8_t sbt:1; + uint8_t hbt:1; + uint8_t bsk:1; + uint8_t rsvd:5; + uint8_t lock; + int16_t chunk_remainder; + }; + }; + uint64_t current_chunk; + uint64_t pad; +} __rte_packed; + +struct tim_mem_entry { + uint64_t w0; + uint64_t wqe; +} __rte_packed; + +struct timvf_ctrl_reg { + uint64_t rctrl0; + uint64_t rctrl1; + uint64_t rctrl2; + uint8_t use_pmu; +} __rte_packed; + +struct timvf_ring; + +typedef uint32_t (*bkt_id)(const uint32_t bkt_tcks, const uint32_t nb_bkts); +typedef struct tim_mem_entry * (*refill_chunk)( + struct tim_mem_bucket * const bkt, + struct timvf_ring * const timr); + +struct timvf_meta { + bkt_id get_target_bkt; + refill_chunk refill_chunk; + struct rte_reciprocal_u64 fast_div; + uint64_t ring_start_cyc; + uint32_t nb_bkts; + struct tim_mem_bucket *bkt; + void *chunk_pool; + uint64_t tck_int; +}; + +struct timvf_ring { + struct timvf_meta meta; + uint64_t tck_nsec; + void *vbar0; + void *bkt_pos; + uint64_t max_tout; + uint64_t nb_chunks; + enum timvf_clk_src clk_src; + uint16_t tim_ring_id; +} __rte_cache_aligned; + +static __rte_always_inline uint32_t +bkt_mod(const uint32_t rel_bkt, const uint32_t nb_bkts) +{ + return rel_bkt % nb_bkts; +} + +int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, + uint32_t *caps, const struct rte_event_timer_adapter_ops **ops); + +#endif /* __TIMVF_EVDEV_H__ */ -- 2.16.2 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v2 04/11] event/octeontx: add support to start and stop timer device 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 00/11] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (2 preceding siblings ...) 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 03/11] event/octeontx: add support to create and free timer adapter Pavan Nikhilesh @ 2018-03-14 13:52 ` Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 05/11] event/octeontx: add multiproducer timer arm and cancel Pavan Nikhilesh ` (6 subsequent siblings) 10 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-03-14 13:52 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh When application requests to start the timer adapter through `rte_event_timer_adapter_start`, Octeontx TIMvf ring does the following: - Uses mbox to communicate TIMpf driver about, * SCLK frequency used to convert ns<->cycles. * program the ring control parameters and start the ring. * get the exact cycle at which the TIMvf ring has started which can be used to estimate the bucket position. On `rte_event_timer_adapter_stop` i.e stop, Octeontx TIMvf ring does the following: - Use mbox to communicate TIMpf driver about, * reset the ring control parameters and stop the ring. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 140 +++++++++++++++++++++++++++++++++++ drivers/event/octeontx/timvf_evdev.h | 5 ++ 2 files changed, 145 insertions(+) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index 473311fa4..ccf724115 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -16,6 +16,32 @@ otx_timvf_init_log(void) rte_log_set_level(otx_logtype_timvf, RTE_LOG_NOTICE); } +struct __rte_packed timvf_mbox_dev_info { + uint64_t ring_active[4]; + uint64_t clk_freq; +}; + +/* Response messages */ +enum { + MBOX_RET_SUCCESS, + MBOX_RET_INVALID, + MBOX_RET_INTERNAL_ERR, +}; + +static int +timvf_mbox_dev_info_get(struct timvf_mbox_dev_info *info) +{ + struct octeontx_mbox_hdr hdr = {0}; + uint16_t len = sizeof(struct timvf_mbox_dev_info); + + hdr.coproc = TIM_COPROC; + hdr.msg = TIM_GET_DEV_INFO; + hdr.vfid = 0; /* TIM DEV is always 0. TIM RING ID changes. */ + + memset(info, 0, len); + return octeontx_ssovf_mbox_send(&hdr, NULL, 0, info, len); +} + static void timvf_ring_info_get(const struct rte_event_timer_adapter *adptr, struct rte_event_timer_adapter_info *adptr_info) @@ -27,6 +53,118 @@ timvf_ring_info_get(const struct rte_event_timer_adapter *adptr, sizeof(struct rte_event_timer_adapter_conf)); } +static int +timvf_ring_conf_set(struct timvf_ctrl_reg *rctl, uint8_t ring_id) +{ + struct octeontx_mbox_hdr hdr = {0}; + uint16_t len = sizeof(struct timvf_ctrl_reg); + int ret; + + hdr.coproc = TIM_COPROC; + hdr.msg = TIM_SET_RING_INFO; + hdr.vfid = ring_id; + + ret = octeontx_ssovf_mbox_send(&hdr, rctl, len, NULL, 0); + if (ret < 0 || hdr.res_code != MBOX_RET_SUCCESS) + return -EACCES; + return 0; +} + +static int +timvf_get_start_cyc(uint64_t *now, uint8_t ring_id) +{ + struct octeontx_mbox_hdr hdr = {0}; + + hdr.coproc = TIM_COPROC; + hdr.msg = TIM_RING_START_CYC_GET; + hdr.vfid = ring_id; + *now = 0; + return octeontx_ssovf_mbox_send(&hdr, NULL, 0, now, sizeof(uint64_t)); +} + +static int +timvf_ring_start(const struct rte_event_timer_adapter *adptr) +{ + int ret; + uint64_t interval = 0; + struct timvf_ctrl_reg rctrl = {0}; + struct timvf_mbox_dev_info dinfo; + struct timvf_ring *timr = adptr->data->adapter_priv; + + ret = timvf_mbox_dev_info_get(&dinfo); + if (ret < 0 || ret != sizeof(struct timvf_mbox_dev_info)) + return -EINVAL; + + /* Calculate the interval cycles according to clock source. */ + switch (timr->clk_src) { + case TIM_CLK_SRC_SCLK: + interval = NSEC2CLK(timr->tck_nsec, dinfo.clk_freq); + break; + case TIM_CLK_SRC_GPIO: + /* GPIO doesn't work on tck_nsec. */ + interval = 0; + break; + case TIM_CLK_SRC_GTI: + interval = NSEC2CLK(timr->tck_nsec, dinfo.clk_freq); + break; + case TIM_CLK_SRC_PTP: + interval = NSEC2CLK(timr->tck_nsec, dinfo.clk_freq); + break; + } + + /*CTRL0 register.*/ + rctrl.rctrl0 = interval; + + /*CTRL1 register.*/ + rctrl.rctrl1 = (uint64_t)(timr->clk_src) << 51 | + 1ull << 48 | + 1ull << 47 | + 1ull << 44 | + (timr->meta.nb_bkts - 1); + + rctrl.rctrl2 = (uint64_t)(TIM_CHUNK_SIZE / 16) << 40; + + timvf_write64((uint64_t)timr->meta.bkt, + (uint8_t *)timr->vbar0 + TIM_VRING_BASE); + if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) { + ret = -EACCES; + goto error; + } + + if (timvf_get_start_cyc(&timr->meta.ring_start_cyc, + timr->tim_ring_id) < 0) { + ret = -EACCES; + goto error; + } + timr->meta.tck_int = NSEC2CLK(timr->tck_nsec, rte_get_timer_hz()); + timr->meta.fast_div = rte_reciprocal_value_u64(timr->meta.tck_int); + timvf_log_info("nb_bkts %d min_ns %"PRIu64" min_cyc %"PRIu64"" + " maxtmo %"PRIu64"\n", + timr->meta.nb_bkts, timr->tck_nsec, interval, + timr->max_tout); + + return 0; +error: + rte_free(timr->meta.bkt); + rte_mempool_free(timr->meta.chunk_pool); + return ret; +} + +static int +timvf_ring_stop(const struct rte_event_timer_adapter *adptr) +{ + struct timvf_ring *timr = adptr->data->adapter_priv; + struct timvf_ctrl_reg rctrl = {0}; + rctrl.rctrl0 = timvf_read64((uint8_t *)timr->vbar0 + TIM_VRING_CTL0); + rctrl.rctrl1 = timvf_read64((uint8_t *)timr->vbar0 + TIM_VRING_CTL1); + rctrl.rctrl1 &= ~(1ull << 47); /* Disable */ + rctrl.rctrl2 = timvf_read64((uint8_t *)timr->vbar0 + TIM_VRING_CTL2); + + if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) + return -EACCES; + return 0; +} + static int timvf_ring_create(struct rte_event_timer_adapter *adptr) { @@ -146,6 +284,8 @@ timvf_ring_free(struct rte_event_timer_adapter *adptr) static struct rte_event_timer_adapter_ops timvf_ops = { .init = timvf_ring_create, .uninit = timvf_ring_free, + .start = timvf_ring_start, + .stop = timvf_ring_stop, .get_info = timvf_ring_info_get, }; diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index e3f558e10..e155b6ce2 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -75,6 +75,11 @@ #define TIM_VRING_AURA (0x108) #define TIM_VRING_REL (0x110) + +#define NSEC_PER_SEC 1E9 +#define NSEC2CLK(__ns, __freq) (((__ns) * (__freq)) / NSEC_PER_SEC) +#define CLK2NSEC(__clk, __freq) (((__clk) * NSEC_PER_SEC) / (__freq)) + #define timvf_read64 rte_read64_relaxed #define timvf_write64 rte_write64_relaxed -- 2.16.2 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v2 05/11] event/octeontx: add multiproducer timer arm and cancel 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 00/11] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (3 preceding siblings ...) 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 04/11] event/octeontx: add support to start and stop timer device Pavan Nikhilesh @ 2018-03-14 13:52 ` Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 06/11] event/octeontx: add single producer timer arm variant Pavan Nikhilesh ` (5 subsequent siblings) 10 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-03-14 13:52 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/Makefile | 5 + drivers/event/octeontx/meson.build | 1 + drivers/event/octeontx/timvf_evdev.c | 7 +- drivers/event/octeontx/timvf_evdev.h | 32 ++++ drivers/event/octeontx/timvf_worker.c | 59 +++++++ drivers/event/octeontx/timvf_worker.h | 280 ++++++++++++++++++++++++++++++++++ 6 files changed, 383 insertions(+), 1 deletion(-) create mode 100644 drivers/event/octeontx/timvf_worker.c create mode 100644 drivers/event/octeontx/timvf_worker.h diff --git a/drivers/event/octeontx/Makefile b/drivers/event/octeontx/Makefile index 570a79412..58fa6af88 100644 --- a/drivers/event/octeontx/Makefile +++ b/drivers/event/octeontx/Makefile @@ -28,19 +28,24 @@ LIBABIVER := 1 SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_worker.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_evdev.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_evdev_selftest.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += timvf_worker.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += timvf_evdev.c ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_ssovf_worker.o += -fno-prefetch-loop-arrays +CFLAGS_timvf_worker.o += -fno-prefetch-loop-arrays ifeq ($(shell test $(GCC_VERSION) -ge 46 && echo 1), 1) CFLAGS_ssovf_worker.o += -Ofast +CFLAGS_timvf_worker.o += -Ofast else CFLAGS_ssovf_worker.o += -O3 -ffast-math +CFLAGS_timvf_worker.o += -O3 -ffast-math endif else CFLAGS_ssovf_worker.o += -Ofast +CFLAGS_timvf_worker.o += -Ofast endif include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/event/octeontx/meson.build b/drivers/event/octeontx/meson.build index 8941f8a56..efac94cba 100644 --- a/drivers/event/octeontx/meson.build +++ b/drivers/event/octeontx/meson.build @@ -4,6 +4,7 @@ sources = files('ssovf_worker.c', 'ssovf_evdev.c', 'ssovf_evdev_selftest.c', + 'timvf_worker.c', 'timvf_evdev.c', ) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index ccf724115..852e8811c 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -126,6 +126,7 @@ timvf_ring_start(const struct rte_event_timer_adapter *adptr) timvf_write64((uint64_t)timr->meta.bkt, (uint8_t *)timr->vbar0 + TIM_VRING_BASE); + timvf_set_chunk_refill(timr); if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) { ret = -EACCES; goto error; @@ -295,7 +296,11 @@ timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, { RTE_SET_USED(dev); RTE_SET_USED(flags); + + timvf_ops.arm_burst = timvf_timer_reg_burst_mp; + timvf_ops.cancel_burst = timvf_timer_unreg_burst; + *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; *ops = &timvf_ops; - return -EINVAL; + return 0; } diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index e155b6ce2..c717c19bc 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -75,6 +75,33 @@ #define TIM_VRING_AURA (0x108) #define TIM_VRING_REL (0x110) +#define TIM_CTL1_W0_S_BUCKET 20 +#define TIM_CTL1_W0_M_BUCKET ((1ull << (40 - 20)) - 1) + +#define TIM_BUCKET_W1_S_NUM_ENTRIES (0) /*Shift*/ +#define TIM_BUCKET_W1_M_NUM_ENTRIES ((1ull << (32 - 0)) - 1) +#define TIM_BUCKET_W1_S_SBT (32) +#define TIM_BUCKET_W1_M_SBT ((1ull << (33 - 32)) - 1) +#define TIM_BUCKET_W1_S_HBT (33) +#define TIM_BUCKET_W1_M_HBT ((1ull << (34 - 33)) - 1) +#define TIM_BUCKET_W1_S_BSK (34) +#define TIM_BUCKET_W1_M_BSK ((1ull << (35 - 34)) - 1) +#define TIM_BUCKET_W1_S_LOCK (40) +#define TIM_BUCKET_W1_M_LOCK ((1ull << (48 - 40)) - 1) +#define TIM_BUCKET_W1_S_CHUNK_REMAINDER (48) +#define TIM_BUCKET_W1_M_CHUNK_REMAINDER ((1ull << (64 - 48)) - 1) + +#define TIM_BUCKET_SEMA \ + (TIM_BUCKET_CHUNK_REMAIN) + +#define TIM_BUCKET_CHUNK_REMAIN \ + (TIM_BUCKET_W1_M_CHUNK_REMAINDER << TIM_BUCKET_W1_S_CHUNK_REMAINDER) + +#define TIM_BUCKET_LOCK \ + (TIM_BUCKET_W1_M_LOCK << TIM_BUCKET_W1_S_LOCK) + +#define TIM_BUCKET_SEMA_WLOCK \ + (TIM_BUCKET_CHUNK_REMAIN | (1ull << TIM_BUCKET_W1_S_LOCK)) #define NSEC_PER_SEC 1E9 #define NSEC2CLK(__ns, __freq) (((__ns) * (__freq)) / NSEC_PER_SEC) @@ -166,5 +193,10 @@ bkt_mod(const uint32_t rel_bkt, const uint32_t nb_bkts) int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, uint32_t *caps, const struct rte_event_timer_adapter_ops **ops); +int timvf_timer_unreg_burst(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers); +int timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers); +void timvf_set_chunk_refill(struct timvf_ring * const timr); #endif /* __TIMVF_EVDEV_H__ */ diff --git a/drivers/event/octeontx/timvf_worker.c b/drivers/event/octeontx/timvf_worker.c new file mode 100644 index 000000000..113603616 --- /dev/null +++ b/drivers/event/octeontx/timvf_worker.c @@ -0,0 +1,59 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include "timvf_worker.h" + +int +timvf_timer_unreg_burst(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers) +{ + RTE_SET_USED(adptr); + int ret; + uint16_t index; + + for (index = 0; index < nb_timers; index++) { + ret = timvf_rem_entry(tim[index]); + if (ret) { + rte_errno = -ret; + break; + } + } + return index; +} + +int +timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers) +{ + int ret; + uint16_t index; + struct tim_mem_entry entry; + struct timvf_ring *timr = adptr->data->adapter_priv; + for (index = 0; index < nb_timers; index++) { + if (unlikely(tim[index]->timeout_ticks > timr->meta.nb_bkts)) { + tim[index]->state = RTE_EVENT_TIMER_ERROR_TOOLATE; + rte_errno = EINVAL; + break; + } + + entry.w0 = (tim[index]->ev.event & 0xFFC000000000) >> 6 | + (tim[index]->ev.event & 0xFFFFFFFFF); + entry.wqe = tim[index]->ev.u64; + ret = timvf_add_entry_mp(timr, tim[index]->timeout_ticks, + tim[index], &entry); + if (unlikely(ret)) { + rte_errno = -ret; + break; + } + } + + return index; +} + +void +timvf_set_chunk_refill(struct timvf_ring * const timr) +{ + timr->meta.refill_chunk = timvf_refill_chunk_generic; +} diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h new file mode 100644 index 000000000..bba4985ab --- /dev/null +++ b/drivers/event/octeontx/timvf_worker.h @@ -0,0 +1,280 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include <rte_common.h> +#include <rte_branch_prediction.h> + +#include "timvf_evdev.h" + +static inline int16_t +timr_bkt_fetch_rem(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_CHUNK_REMAINDER) & + TIM_BUCKET_W1_M_CHUNK_REMAINDER; +} + +static inline int16_t +timr_bkt_get_rem(struct tim_mem_bucket *bktp) +{ + return __atomic_load_n((int16_t *)&bktp->chunk_remainder, + __ATOMIC_ACQUIRE); +} + +static inline void +timr_bkt_set_rem(struct tim_mem_bucket *bktp, uint16_t v) +{ + __atomic_store_n((int16_t *)&bktp->chunk_remainder, v, + __ATOMIC_RELEASE); +} + +static inline void +timr_bkt_sub_rem(struct tim_mem_bucket *bktp, uint16_t v) +{ + __atomic_fetch_sub((int16_t *)&bktp->chunk_remainder, v, + __ATOMIC_RELEASE); +} + +static inline uint8_t +timr_bkt_get_sbt(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_SBT) & TIM_BUCKET_W1_M_SBT; +} + +static inline uint64_t +timr_bkt_set_sbt(struct tim_mem_bucket *bktp) +{ + const uint64_t v = TIM_BUCKET_W1_M_SBT << TIM_BUCKET_W1_S_SBT; + return __atomic_fetch_or((uint64_t *)&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline uint64_t +timr_bkt_clr_sbt(struct tim_mem_bucket *bktp) +{ + const uint64_t v = ~(TIM_BUCKET_W1_M_SBT << TIM_BUCKET_W1_S_SBT); + return __atomic_fetch_and((uint64_t *)&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline uint8_t +timr_bkt_get_shbt(uint64_t w1) +{ + return ((w1 >> TIM_BUCKET_W1_S_HBT) & TIM_BUCKET_W1_M_HBT) | + ((w1 >> TIM_BUCKET_W1_S_SBT) & TIM_BUCKET_W1_M_SBT); +} + +static inline uint8_t +timr_bkt_get_hbt(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_HBT) & TIM_BUCKET_W1_M_HBT; +} + +static inline uint8_t +timr_bkt_get_bsk(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_BSK) & TIM_BUCKET_W1_M_BSK; +} + +static inline uint64_t +timr_bkt_clr_bsk(struct tim_mem_bucket *bktp) +{ + /*Clear everything except lock. */ + const uint64_t v = TIM_BUCKET_W1_M_LOCK << TIM_BUCKET_W1_S_LOCK; + return __atomic_fetch_and((uint64_t *)&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline uint64_t +timr_bkt_fetch_sema_lock(struct tim_mem_bucket *bktp) +{ + return __atomic_fetch_add((uint64_t *)&bktp->w1, TIM_BUCKET_SEMA_WLOCK, + __ATOMIC_ACQ_REL); +} + +static inline uint64_t +timr_bkt_fetch_sema(struct tim_mem_bucket *bktp) +{ + return __atomic_fetch_add((uint64_t *)&bktp->w1, TIM_BUCKET_SEMA, + __ATOMIC_RELAXED); +} + +static inline uint64_t +timr_bkt_inc_lock(struct tim_mem_bucket *bktp) +{ + const uint64_t v = 1ull << TIM_BUCKET_W1_S_LOCK; + return __atomic_fetch_add((uint64_t *)&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline void +timr_bkt_dec_lock(struct tim_mem_bucket *bktp) +{ + __atomic_add_fetch((uint8_t *)&bktp->lock, 0xff, __ATOMIC_ACQ_REL); +} + +static inline uint32_t +timr_bkt_get_nent(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_NUM_ENTRIES) & + TIM_BUCKET_W1_M_NUM_ENTRIES; +} + +static inline void +timr_bkt_inc_nent(struct tim_mem_bucket *bktp) +{ + __atomic_add_fetch((uint32_t *)&bktp->nb_entry, 1, __ATOMIC_RELAXED); +} + +static inline void +timr_bkt_add_nent(struct tim_mem_bucket *bktp, uint32_t v) +{ + __atomic_add_fetch((uint32_t *)&bktp->nb_entry, v, __ATOMIC_RELAXED); +} + +static inline uint64_t +timr_bkt_clr_nent(struct tim_mem_bucket *bktp) +{ + const uint64_t v = ~(TIM_BUCKET_W1_M_NUM_ENTRIES << + TIM_BUCKET_W1_S_NUM_ENTRIES); + return __atomic_and_fetch((uint64_t *)&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline struct tim_mem_entry * +timr_clr_bkt(struct timvf_ring * const timr, struct tim_mem_bucket * const bkt) +{ + struct tim_mem_entry *chunk; + struct tim_mem_entry *pnext; + chunk = ((struct tim_mem_entry *)bkt->first_chunk); + chunk = (struct tim_mem_entry *)(chunk + nb_chunk_slots)->w0; + + while (chunk) { + pnext = (struct tim_mem_entry *)((chunk + nb_chunk_slots)->w0); + rte_mempool_put(timr->meta.chunk_pool, chunk); + chunk = pnext; + } + return (struct tim_mem_entry *)bkt->first_chunk; +} + +static inline int +timvf_rem_entry(struct rte_event_timer *tim) +{ + uint64_t lock_sema; + struct tim_mem_entry *entry; + struct tim_mem_bucket *bkt; + if (tim->impl_opaque[1] == 0 || + tim->impl_opaque[0] == 0) + return -ENOENT; + + entry = (struct tim_mem_entry *)tim->impl_opaque[0]; + if (entry->wqe != tim->ev.u64) { + tim->impl_opaque[1] = tim->impl_opaque[0] = 0; + return -ENOENT; + } + bkt = (struct tim_mem_bucket *)tim->impl_opaque[1]; + lock_sema = timr_bkt_inc_lock(bkt); + if (timr_bkt_get_shbt(lock_sema) + || !timr_bkt_get_nent(lock_sema)) { + timr_bkt_dec_lock(bkt); + tim->impl_opaque[1] = tim->impl_opaque[0] = 0; + return -ENOENT; + } + + entry->w0 = entry->wqe = 0; + timr_bkt_dec_lock(bkt); + + tim->state = RTE_EVENT_TIMER_CANCELED; + tim->impl_opaque[1] = tim->impl_opaque[0] = 0; + return 0; +} + +static inline struct tim_mem_entry * +timvf_refill_chunk_generic(struct tim_mem_bucket * const bkt, + struct timvf_ring * const timr) +{ + struct tim_mem_entry *chunk; + + if (bkt->nb_entry || !bkt->first_chunk) { + if (unlikely(rte_mempool_get(timr->meta.chunk_pool, + (void **)&chunk))) { + return NULL; + } + if (bkt->nb_entry) { + *(uint64_t *)((struct tim_mem_entry *) + bkt->current_chunk + + nb_chunk_slots) = + (uint64_t) chunk; + } else { + bkt->first_chunk = (uint64_t) chunk; + } + } else { + chunk = timr_clr_bkt(timr, bkt); + bkt->first_chunk = (uint64_t) chunk; + } + *(uint64_t *)(chunk + nb_chunk_slots) = 0; + + return chunk; +} + +static inline struct tim_mem_bucket * +timvf_get_target_bucket(struct timvf_ring * const timr, const uint32_t rel_bkt) +{ + const uint64_t bkt_cyc = rte_rdtsc() - timr->meta.ring_start_cyc; + const uint32_t bucket = rte_reciprocal_divide_u64(bkt_cyc, + &timr->meta.fast_div) + rel_bkt; + const uint32_t tbkt_id = timr->meta.get_target_bkt(bucket, + timr->meta.nb_bkts); + return &timr->meta.bkt[tbkt_id]; +} + +/* Multi producer functions. */ +static inline int +timvf_add_entry_mp(struct timvf_ring * const timr, const uint32_t rel_bkt, + struct rte_event_timer * const tim, + const struct tim_mem_entry * const pent) +{ + uint8_t lock_cnt; + int16_t rem; + uint64_t lock_sema; + struct tim_mem_bucket *bkt; + struct tim_mem_entry *chunk; + +__retry: + bkt = timvf_get_target_bucket(timr, rel_bkt); + /* Bucket related checks. */ + /*Get Bucket sema*/ + lock_sema = timr_bkt_fetch_sema_lock(bkt); + if (unlikely(timr_bkt_get_shbt(lock_sema))) { + timr_bkt_dec_lock(bkt); + goto __retry; + } + + RTE_SET_USED(lock_cnt); + rem = timr_bkt_fetch_rem(lock_sema); + + if (rem < 0) { + /* goto diff bucket. */ + timr_bkt_dec_lock(bkt); + goto __retry; + } else if (!rem) { + /*Only one thread can be here*/ + chunk = timr->meta.refill_chunk(bkt, timr); + if (unlikely(chunk == NULL)) { + timr_bkt_set_rem(bkt, 0); + timr_bkt_dec_lock(bkt); + tim->impl_opaque[0] = tim->impl_opaque[1] = 0; + tim->state = RTE_EVENT_TIMER_ERROR; + return -ENOMEM; + } + bkt->current_chunk = (uint64_t) chunk; + timr_bkt_set_rem(bkt, nb_chunk_slots - 1); + } else { + chunk = (struct tim_mem_entry *)bkt->current_chunk; + chunk += nb_chunk_slots - rem; + } + /* Copy work entry. */ + *chunk = *pent; + timr_bkt_inc_nent(bkt); + timr_bkt_dec_lock(bkt); + + tim->impl_opaque[0] = (uint64_t)chunk; + tim->impl_opaque[1] = (uint64_t)bkt; + tim->state = RTE_EVENT_TIMER_ARMED; + return 0; +} -- 2.16.2 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v2 06/11] event/octeontx: add single producer timer arm variant 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 00/11] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (4 preceding siblings ...) 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 05/11] event/octeontx: add multiproducer timer arm and cancel Pavan Nikhilesh @ 2018-03-14 13:52 ` Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 07/11] event/octeontx: add burst mode for timer arm Pavan Nikhilesh ` (4 subsequent siblings) 10 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-03-14 13:52 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh When application creates the timer adapter by passing `RTE_EVENT_TIMER_ADAPTER_F_SP_PUT` flag, we can optimize the arm sequence by removing the locking overhead. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 15 ++++++++--- drivers/event/octeontx/timvf_evdev.h | 2 ++ drivers/event/octeontx/timvf_worker.c | 29 +++++++++++++++++++++ drivers/event/octeontx/timvf_worker.h | 47 +++++++++++++++++++++++++++++++++++ 4 files changed, 90 insertions(+), 3 deletions(-) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index 852e8811c..3d0e43229 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -176,6 +176,7 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) struct timvf_ring *timr; struct octeontx_timvf_info tinfo; const char *mempool_ops; + unsigned int mp_flags = 0; if (octeontx_timvf_info(&tinfo) < 0) return -ENODEV; @@ -226,6 +227,11 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) timr->nb_chunks = nb_timers / nb_chunk_slots; + if (rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_SP_PUT) { + mp_flags = MEMPOOL_F_SP_PUT | MEMPOOL_F_SC_GET; + timvf_log_info("Using single producer mode"); + } + timr->meta.bkt = rte_zmalloc("octeontx_timvf_bucket", (timr->meta.nb_bkts) * sizeof(struct tim_mem_bucket), 0); @@ -235,7 +241,7 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) snprintf(pool_name, 30, "timvf_meta.chunk_pool%d", timr->tim_ring_id); timr->meta.chunk_pool = (void *)rte_mempool_create_empty(pool_name, timr->nb_chunks, TIM_CHUNK_SIZE, 0, 0, rte_socket_id(), - 0); + mp_flags); if (!timr->meta.chunk_pool) { rte_free(timr->meta.bkt); @@ -295,9 +301,12 @@ timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, uint32_t *caps, const struct rte_event_timer_adapter_ops **ops) { RTE_SET_USED(dev); - RTE_SET_USED(flags); - timvf_ops.arm_burst = timvf_timer_reg_burst_mp; + if (flags & RTE_EVENT_TIMER_ADAPTER_F_SP_PUT) + timvf_ops.arm_burst = timvf_timer_reg_burst_sp; + else + timvf_ops.arm_burst = timvf_timer_reg_burst_mp; + timvf_ops.cancel_burst = timvf_timer_unreg_burst; *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index c717c19bc..232657776 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -195,6 +195,8 @@ int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, uint32_t *caps, const struct rte_event_timer_adapter_ops **ops); int timvf_timer_unreg_burst(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers); +int timvf_timer_reg_burst_sp(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers); int timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers); void timvf_set_chunk_refill(struct timvf_ring * const timr); diff --git a/drivers/event/octeontx/timvf_worker.c b/drivers/event/octeontx/timvf_worker.c index 113603616..93fd050c0 100644 --- a/drivers/event/octeontx/timvf_worker.c +++ b/drivers/event/octeontx/timvf_worker.c @@ -23,6 +23,35 @@ timvf_timer_unreg_burst(const struct rte_event_timer_adapter *adptr, return index; } +int +timvf_timer_reg_burst_sp(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers) +{ + int ret; + uint16_t index; + struct tim_mem_entry entry; + struct timvf_ring *timr = adptr->data->adapter_priv; + for (index = 0; index < nb_timers; index++) { + if (unlikely(tim[index]->timeout_ticks > timr->meta.nb_bkts)) { + tim[index]->state = RTE_EVENT_TIMER_ERROR_TOOLATE; + rte_errno = EINVAL; + break; + } + + entry.w0 = (tim[index]->ev.event & 0xFFC000000000) >> 6 | + (tim[index]->ev.event & 0xFFFFFFFF); + entry.wqe = tim[index]->ev.u64; + ret = timvf_add_entry_sp(timr, tim[index]->timeout_ticks, + tim[index], &entry); + if (unlikely(ret)) { + rte_errno = -ret; + break; + } + } + + return index; +} + int timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers) diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h index bba4985ab..7e197f5cd 100644 --- a/drivers/event/octeontx/timvf_worker.h +++ b/drivers/event/octeontx/timvf_worker.h @@ -223,6 +223,53 @@ timvf_get_target_bucket(struct timvf_ring * const timr, const uint32_t rel_bkt) return &timr->meta.bkt[tbkt_id]; } +/* Single producer functions. */ +static inline int +timvf_add_entry_sp(struct timvf_ring * const timr, const uint32_t rel_bkt, + struct rte_event_timer * const tim, + const struct tim_mem_entry * const pent) +{ + int16_t rem; + uint64_t lock_sema; + struct tim_mem_bucket *bkt; + struct tim_mem_entry *chunk; + + + bkt = timvf_get_target_bucket(timr, rel_bkt); +__retry: + /*Get Bucket sema*/ + lock_sema = timr_bkt_fetch_sema(bkt); + /* Bucket related checks. */ + if (unlikely(timr_bkt_get_hbt(lock_sema))) + goto __retry; + + /* Insert the work. */ + rem = timr_bkt_fetch_rem(lock_sema); + + if (!rem) { + chunk = timr->meta.refill_chunk(bkt, timr); + if (unlikely(chunk == NULL)) { + timr_bkt_set_rem(bkt, 0); + tim->impl_opaque[0] = tim->impl_opaque[1] = 0; + tim->state = RTE_EVENT_TIMER_ERROR; + return -ENOMEM; + } + bkt->current_chunk = (uint64_t) chunk; + timr_bkt_set_rem(bkt, nb_chunk_slots - 1); + } else { + chunk = (struct tim_mem_entry *)bkt->current_chunk; + chunk += nb_chunk_slots - rem; + } + /* Copy work entry. */ + *chunk = *pent; + timr_bkt_inc_nent(bkt); + + tim->impl_opaque[0] = (uint64_t)chunk; + tim->impl_opaque[1] = (uint64_t)bkt; + tim->state = RTE_EVENT_TIMER_ARMED; + return 0; +} + /* Multi producer functions. */ static inline int timvf_add_entry_mp(struct timvf_ring * const timr, const uint32_t rel_bkt, -- 2.16.2 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v2 07/11] event/octeontx: add burst mode for timer arm 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 00/11] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (5 preceding siblings ...) 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 06/11] event/octeontx: add single producer timer arm variant Pavan Nikhilesh @ 2018-03-14 13:52 ` Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 08/11] event/octeontx: optimize timer adapter resolution parameters Pavan Nikhilesh ` (3 subsequent siblings) 10 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-03-14 13:52 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 1 + drivers/event/octeontx/timvf_evdev.h | 3 ++ drivers/event/octeontx/timvf_worker.c | 36 +++++++++++++ drivers/event/octeontx/timvf_worker.h | 95 +++++++++++++++++++++++++++++++++++ 4 files changed, 135 insertions(+) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index 3d0e43229..fe4e8cd05 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -307,6 +307,7 @@ timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, else timvf_ops.arm_burst = timvf_timer_reg_burst_mp; + timvf_ops.arm_tmo_tick_burst = timvf_timer_reg_brst; timvf_ops.cancel_burst = timvf_timer_unreg_burst; *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index 232657776..a263ccc68 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -199,6 +199,9 @@ int timvf_timer_reg_burst_sp(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers); int timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers); +int timvf_timer_reg_brst(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint64_t timeout_tick, + const uint16_t nb_timers); void timvf_set_chunk_refill(struct timvf_ring * const timr); #endif /* __TIMVF_EVDEV_H__ */ diff --git a/drivers/event/octeontx/timvf_worker.c b/drivers/event/octeontx/timvf_worker.c index 93fd050c0..8c1da246c 100644 --- a/drivers/event/octeontx/timvf_worker.c +++ b/drivers/event/octeontx/timvf_worker.c @@ -81,6 +81,42 @@ timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, return index; } +int +timvf_timer_reg_brst(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint64_t timeout_tick, + const uint16_t nb_timers) +{ + int ret; + uint16_t set_timers = 0; + uint16_t idx; + uint16_t arr_idx = 0; + struct timvf_ring *timr = adptr->data->adapter_priv; + struct tim_mem_entry entry[TIMVF_MAX_BURST] __rte_cache_aligned; + + if (unlikely(timeout_tick > timr->meta.nb_bkts)) { + for (idx = 0; idx < nb_timers; idx++) + tim[idx]->state = RTE_EVENT_TIMER_ERROR_TOOLATE; + rte_errno = EINVAL; + return 0; + } + + while (arr_idx < nb_timers) { + for (idx = 0; idx < TIMVF_MAX_BURST && (arr_idx < nb_timers); + idx++, arr_idx++) { + entry[idx].w0 = + (tim[arr_idx]->ev.event & 0xFFC000000000) >> 6 | + (tim[arr_idx]->ev.event & 0xFFFFFFFF); + entry[idx].wqe = tim[arr_idx]->ev.u64; + } + ret = timvf_add_entry_brst(timr, timeout_tick, &tim[set_timers], + entry, idx); + set_timers += ret; + if (ret != idx) + break; + } + return set_timers; +} + void timvf_set_chunk_refill(struct timvf_ring * const timr) { diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h index 7e197f5cd..9dad5c769 100644 --- a/drivers/event/octeontx/timvf_worker.h +++ b/drivers/event/octeontx/timvf_worker.h @@ -325,3 +325,98 @@ timvf_add_entry_mp(struct timvf_ring * const timr, const uint32_t rel_bkt, tim->state = RTE_EVENT_TIMER_ARMED; return 0; } + +static inline uint16_t +timvf_cpy_wrk(uint16_t index, uint16_t cpy_lmt, + struct tim_mem_entry *chunk, + struct rte_event_timer ** const tim, + const struct tim_mem_entry * const ents, + const struct tim_mem_bucket * const bkt) +{ + for (; index < cpy_lmt; index++) { + *chunk = *(ents + index); + tim[index]->impl_opaque[0] = (uint64_t)chunk++; + tim[index]->impl_opaque[1] = (uint64_t)bkt; + tim[index]->state = RTE_EVENT_TIMER_ARMED; + } + + return index; +} + +/* Burst mode functions */ +static inline int +timvf_add_entry_brst(struct timvf_ring * const timr, const uint16_t rel_bkt, + struct rte_event_timer ** const tim, + const struct tim_mem_entry *ents, + const uint16_t nb_timers) +{ + int16_t rem; + int16_t crem = 0; + uint8_t lock_cnt; + uint16_t index = 0; + uint16_t chunk_remainder = 0; + uint64_t lock_sema; + struct tim_mem_bucket *bkt; + struct tim_mem_entry *chunk; + +__retry: + bkt = timvf_get_target_bucket(timr, rel_bkt); + + /* Only one thread beyond this. */ + lock_sema = timr_bkt_inc_lock(bkt); + lock_cnt = (uint8_t) + ((lock_sema >> TIM_BUCKET_W1_S_LOCK) & TIM_BUCKET_W1_M_LOCK); + + if (lock_cnt) { + timr_bkt_dec_lock(bkt); + goto __retry; + } + + /* Bucket related checks. */ + if (unlikely(timr_bkt_get_hbt(lock_sema))) { + timr_bkt_dec_lock(bkt); + goto __retry; + } + + chunk_remainder = timr_bkt_fetch_rem(lock_sema); + rem = chunk_remainder - nb_timers; + if (rem < 0) { + crem = nb_chunk_slots - chunk_remainder; + if (chunk_remainder && crem) { + chunk = ((struct tim_mem_entry *)bkt->current_chunk) + + crem; + + index = timvf_cpy_wrk(index, chunk_remainder, + chunk, tim, ents, bkt); + timr_bkt_sub_rem(bkt, chunk_remainder); + timr_bkt_add_nent(bkt, chunk_remainder); + } + rem = nb_timers - chunk_remainder; + ents = ents + chunk_remainder; + + chunk = timr->meta.refill_chunk(bkt, timr); + if (unlikely(chunk == NULL)) { + timr_bkt_dec_lock(bkt); + rte_errno = ENOMEM; + tim[index]->state = RTE_EVENT_TIMER_ERROR; + return crem; + } + *(uint64_t *)(chunk + nb_chunk_slots) = 0; + bkt->current_chunk = (uint64_t) chunk; + + index = timvf_cpy_wrk(index, nb_timers, chunk, tim, ents, bkt); + timr_bkt_set_rem(bkt, nb_chunk_slots - rem); + timr_bkt_add_nent(bkt, rem); + } else { + chunk = (struct tim_mem_entry *)bkt->current_chunk; + chunk += (nb_chunk_slots - chunk_remainder); + + index = timvf_cpy_wrk(index, nb_timers, + chunk, tim, ents, bkt); + timr_bkt_sub_rem(bkt, nb_timers); + timr_bkt_add_nent(bkt, nb_timers); + } + + timr_bkt_dec_lock(bkt); + return nb_timers; +} -- 2.16.2 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v2 08/11] event/octeontx: optimize timer adapter resolution parameters 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 00/11] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (6 preceding siblings ...) 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 07/11] event/octeontx: add burst mode for timer arm Pavan Nikhilesh @ 2018-03-14 13:52 ` Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 09/11] event/octeontx: add option to use fpavf as chunk pool Pavan Nikhilesh ` (2 subsequent siblings) 10 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-03-14 13:52 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh When application sets `RTE_EVENT_TIMER_ADAPTER_F_ADJUST_RES` flag while creating adapter underlying driver is free to optimize the resolution for best possible configuration. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 48 +++++++++++++++++++++++++++++++++++- drivers/event/octeontx/timvf_evdev.h | 6 +++++ 2 files changed, 53 insertions(+), 1 deletion(-) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index fe4e8cd05..43ec98ce8 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -82,6 +82,45 @@ timvf_get_start_cyc(uint64_t *now, uint8_t ring_id) return octeontx_ssovf_mbox_send(&hdr, NULL, 0, now, sizeof(uint64_t)); } +static int +optimize_bucket_parameters(struct timvf_ring *timr) +{ + uint32_t hbkts; + uint32_t lbkts; + uint64_t tck_nsec; + + hbkts = rte_align32pow2(timr->meta.nb_bkts); + tck_nsec = RTE_ALIGN_MUL_CEIL(timr->max_tout / (hbkts - 1), 10); + + if ((tck_nsec < 1000 || hbkts > TIM_MAX_BUCKETS)) + hbkts = 0; + + lbkts = rte_align32prevpow2(timr->meta.nb_bkts); + tck_nsec = RTE_ALIGN_MUL_CEIL((timr->max_tout / (lbkts - 1)), 10); + + if ((tck_nsec < 1000 || hbkts > TIM_MAX_BUCKETS)) + lbkts = 0; + + if (!hbkts && !lbkts) + return 0; + + if (!hbkts) { + timr->meta.nb_bkts = lbkts; + goto end; + } else if (!lbkts) { + timr->meta.nb_bkts = hbkts; + goto end; + } + + timr->meta.nb_bkts = (hbkts - timr->meta.nb_bkts) < + (timr->meta.nb_bkts - lbkts) ? hbkts : lbkts; +end: + timr->meta.get_target_bkt = bkt_and; + timr->tck_nsec = RTE_ALIGN_MUL_CEIL((timr->max_tout / + (timr->meta.nb_bkts - 1)), 10); + return 1; +} + static int timvf_ring_start(const struct rte_event_timer_adapter *adptr) { @@ -217,7 +256,7 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) } timr->tim_ring_id = adptr->data->id; - timr->tck_nsec = rcfg->timer_tick_ns; + timr->tck_nsec = RTE_ALIGN_MUL_CEIL(rcfg->timer_tick_ns, 10); timr->max_tout = rcfg->max_tmo_ns; timr->meta.nb_bkts = (timr->max_tout / timr->tck_nsec) + 1; timr->vbar0 = octeontx_timvf_bar(timr->tim_ring_id, 0); @@ -227,6 +266,13 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) timr->nb_chunks = nb_timers / nb_chunk_slots; + /* Try to optimize the bucket parameters. */ + if ((rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_ADJUST_RES) + && !rte_is_power_of_2(timr->meta.nb_bkts)) { + optimize_bucket_parameters(timr); + timvf_log_info("Optimizing configured values"); + } + if (rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_SP_PUT) { mp_flags = MEMPOOL_F_SP_PUT | MEMPOOL_F_SC_GET; timvf_log_info("Using single producer mode"); diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index a263ccc68..06e6a1588 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -191,6 +191,12 @@ bkt_mod(const uint32_t rel_bkt, const uint32_t nb_bkts) return rel_bkt % nb_bkts; } +static __rte_always_inline uint32_t __hot +bkt_and(uint32_t rel_bkt, uint32_t nb_bkts) +{ + return rel_bkt & (nb_bkts - 1); +} + int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, uint32_t *caps, const struct rte_event_timer_adapter_ops **ops); int timvf_timer_unreg_burst(const struct rte_event_timer_adapter *adptr, -- 2.16.2 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v2 09/11] event/octeontx: add option to use fpavf as chunk pool 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 00/11] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (7 preceding siblings ...) 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 08/11] event/octeontx: optimize timer adapter resolution parameters Pavan Nikhilesh @ 2018-03-14 13:52 ` Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 10/11] doc: update eventdev OcteonTx documentation Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 11/11] maintainers: claim responsibility for octeontx timvf Pavan Nikhilesh 10 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-03-14 13:52 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Add compile-time configurable option to force TIMvf to use Octeontx FPAvf pool manager as its chunk pool. When FPAvf is used as pool manager the TIMvf automatically frees the chunks to FPAvf through gpool-id. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 22 +++++++++++++++++++++- drivers/event/octeontx/timvf_evdev.h | 3 ++- drivers/event/octeontx/timvf_worker.c | 7 +++++-- drivers/event/octeontx/timvf_worker.h | 23 +++++++++++++++++++++++ 4 files changed, 51 insertions(+), 4 deletions(-) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index 43ec98ce8..abb03acc0 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -125,7 +125,9 @@ static int timvf_ring_start(const struct rte_event_timer_adapter *adptr) { int ret; + uint8_t use_fpa = 0; uint64_t interval = 0; + uintptr_t pool; struct timvf_ctrl_reg rctrl = {0}; struct timvf_mbox_dev_info dinfo; struct timvf_ring *timr = adptr->data->adapter_priv; @@ -151,6 +153,9 @@ timvf_ring_start(const struct rte_event_timer_adapter *adptr) break; } + if (!strcmp(rte_mbuf_best_mempool_ops(), "octeontx_fpavf")) + use_fpa = 1; + /*CTRL0 register.*/ rctrl.rctrl0 = interval; @@ -163,9 +168,24 @@ timvf_ring_start(const struct rte_event_timer_adapter *adptr) rctrl.rctrl2 = (uint64_t)(TIM_CHUNK_SIZE / 16) << 40; + if (use_fpa) { + pool = (uintptr_t)((struct rte_mempool *) + timr->meta.chunk_pool)->pool_id; + ret = octeontx_fpa_bufpool_gpool(pool); + if (ret < 0) { + timvf_log_dbg("Unable to get gaura id"); + ret = -ENOMEM; + goto error; + } + timvf_write64((uint64_t)ret, + (uint8_t *)timr->vbar0 + TIM_VRING_AURA); + } else { + rctrl.rctrl1 |= 1ull << 43; + } + timvf_write64((uint64_t)timr->meta.bkt, (uint8_t *)timr->vbar0 + TIM_VRING_BASE); - timvf_set_chunk_refill(timr); + timvf_set_chunk_refill(timr, use_fpa); if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) { ret = -EACCES; goto error; diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index 06e6a1588..813c9463d 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -25,6 +25,7 @@ #include <rte_reciprocal.h> #include <octeontx_mbox.h> +#include <octeontx_fpavf.h> #define timvf_log(level, fmt, args...) \ rte_log(RTE_LOG_ ## level, otx_logtype_timvf, \ @@ -208,6 +209,6 @@ int timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, int timvf_timer_reg_brst(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint64_t timeout_tick, const uint16_t nb_timers); -void timvf_set_chunk_refill(struct timvf_ring * const timr); +void timvf_set_chunk_refill(struct timvf_ring * const timr, uint8_t use_fpa); #endif /* __TIMVF_EVDEV_H__ */ diff --git a/drivers/event/octeontx/timvf_worker.c b/drivers/event/octeontx/timvf_worker.c index 8c1da246c..4f4469f8b 100644 --- a/drivers/event/octeontx/timvf_worker.c +++ b/drivers/event/octeontx/timvf_worker.c @@ -118,7 +118,10 @@ timvf_timer_reg_brst(const struct rte_event_timer_adapter *adptr, } void -timvf_set_chunk_refill(struct timvf_ring * const timr) +timvf_set_chunk_refill(struct timvf_ring * const timr, uint8_t use_fpa) { - timr->meta.refill_chunk = timvf_refill_chunk_generic; + if (use_fpa) + timr->meta.refill_chunk = timvf_refill_chunk_fpa; + else + timr->meta.refill_chunk = timvf_refill_chunk_generic; } diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h index 9dad5c769..39ce4535c 100644 --- a/drivers/event/octeontx/timvf_worker.h +++ b/drivers/event/octeontx/timvf_worker.h @@ -201,6 +201,7 @@ timvf_refill_chunk_generic(struct tim_mem_bucket * const bkt, nb_chunk_slots) = (uint64_t) chunk; } else { + bkt->first_chunk = (uint64_t) chunk; } } else { @@ -212,6 +213,28 @@ timvf_refill_chunk_generic(struct tim_mem_bucket * const bkt, return chunk; } +static inline struct tim_mem_entry * +timvf_refill_chunk_fpa(struct tim_mem_bucket * const bkt, + struct timvf_ring * const timr) +{ + struct tim_mem_entry *chunk; + + if (unlikely(rte_mempool_get(timr->meta.chunk_pool, (void **)&chunk))) + return NULL; + + *(uint64_t *)(chunk + nb_chunk_slots) = 0; + if (bkt->nb_entry) { + *(uint64_t *)((struct tim_mem_entry *) + bkt->current_chunk + + nb_chunk_slots) = + (uint64_t) chunk; + } else { + bkt->first_chunk = (uint64_t) chunk; + } + + return chunk; +} + static inline struct tim_mem_bucket * timvf_get_target_bucket(struct timvf_ring * const timr, const uint32_t rel_bkt) { -- 2.16.2 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v2 10/11] doc: update eventdev OcteonTx documentation 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 00/11] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (8 preceding siblings ...) 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 09/11] event/octeontx: add option to use fpavf as chunk pool Pavan Nikhilesh @ 2018-03-14 13:52 ` Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 11/11] maintainers: claim responsibility for octeontx timvf Pavan Nikhilesh 10 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-03-14 13:52 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- doc/guides/eventdevs/octeontx.rst | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/doc/guides/eventdevs/octeontx.rst b/doc/guides/eventdevs/octeontx.rst index 4fabe54f9..f4f5473fd 100644 --- a/doc/guides/eventdevs/octeontx.rst +++ b/doc/guides/eventdevs/octeontx.rst @@ -28,6 +28,9 @@ Features of the OCTEONTX SSOVF PMD are: - Open system with configurable amount of outstanding events - HW accelerated dequeue timeout support to enable power management - SR-IOV VF +- HW managed event timers support through TIMVF, with high precision and + time granularity of 1us. +- Upto 64 event timer adapters. Supported OCTEONTX SoCs ----------------------- @@ -110,3 +113,9 @@ Rx adapter support When eth_octeontx is used as Rx adapter event schedule type ``RTE_SCHED_TYPE_PARALLEL`` is not supported. + +Event timer adapter support +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When timvf is used as Event timer adapter event schedule type +``RTE_SCHED_TYPE_PARALLEL`` is not supported. -- 2.16.2 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v2 11/11] maintainers: claim responsibility for octeontx timvf 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 00/11] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (9 preceding siblings ...) 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 10/11] doc: update eventdev OcteonTx documentation Pavan Nikhilesh @ 2018-03-14 13:52 ` Pavan Nikhilesh 10 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-03-14 13:52 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- MAINTAINERS | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index a646ca3e1..4fac167d7 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -737,6 +737,10 @@ M: Santosh Shukla <santosh.shukla@caviumnetworks.com> F: drivers/event/octeontx/ F: doc/guides/eventdevs/octeontx.rst +Cavium OCTEONTX timvf +M: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> +F: drivers/event/octeontx/timvf_* + NXP DPAA2 eventdev M: Hemant Agrawal <hemant.agrawal@nxp.com> M: Nipun Gupta <nipun.gupta@nxp.com> -- 2.16.2 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver 2018-02-16 21:36 [dpdk-dev] [PATCH 0/9] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (10 preceding siblings ...) 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 00/11] event/octeontx: add event timer adapter driver Pavan Nikhilesh @ 2018-04-03 15:05 ` Pavan Nikhilesh 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 01/12] mempool/octeontx: probe timvf PCIe devices Pavan Nikhilesh ` (12 more replies) 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 00/11] " Pavan Nikhilesh 12 siblings, 13 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-03 15:05 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh The event timer adapter[1] provides APIs to configure an event timer device that allows an application to arm timers which on expiry push events to an event device such as OcteonTx SSO. The OcteonTx TIM is a co-processor that can be configured as an event timer adapter which can be used by an application to manage event timers. The TIM co-processor processes the event timers registered and pushes expired event timers to SSO based on the event queue, schedule type, flow id etc. provided as rte_event while arming the event timer. It maintains event timers with high precision and time granularity of 1us (microsecond). [1] http://dpdk.org/dev/patchwork/patch/33525/ This patch set depends on: 1. http://dpdk.org/dev/patchwork/bundle/pbhagavatula/event_timerdev_v9 2. http://dpdk.org/dev/patchwork/patch/35216/ 3. http://dpdk.org/dev/patchwork/patch/36089/ v3 Changes: - change datapath function return from int to uint16_t. - Add missing state checks while arm/cancel. - add stats. v2 Changes: - Move common code changes to separate patches. - Reduce ifdef clutter. - fix improper return value (erik). - remove redundent ops assignment (erik). - update octeontx documentation. Pavan Nikhilesh (12): mempool/octeontx: probe timvf PCIe devices usertools: add Cavium TIM as an event device event/octeontx: add support to create and free timer adapter event/octeontx: add support to start and stop timer device event/octeontx: add event timer stats get and reset event/octeontx: add multiproducer timer arm and cancel event/octeontx: add single producer timer arm variant event/octeontx: add burst mode for timer arm event/octeontx: optimize timer adapter resolution parameters event/octeontx: add option to use fpavf as chunk pool doc: update eventdev OcteonTx documentation maintainers: claim responsibility for octeontx timvf MAINTAINERS | 4 + doc/guides/eventdevs/octeontx.rst | 9 + drivers/event/octeontx/Makefile | 7 + drivers/event/octeontx/meson.build | 4 +- drivers/event/octeontx/ssovf_evdev.c | 3 + drivers/event/octeontx/timvf_evdev.c | 408 +++++++++++++++++++ drivers/event/octeontx/timvf_evdev.h | 215 ++++++++++ drivers/event/octeontx/timvf_worker.c | 161 ++++++++ drivers/event/octeontx/timvf_worker.h | 445 +++++++++++++++++++++ drivers/mempool/octeontx/Makefile | 1 + drivers/mempool/octeontx/meson.build | 1 + drivers/mempool/octeontx/octeontx_mbox.h | 7 + drivers/mempool/octeontx/octeontx_timvf.c | 145 +++++++ .../octeontx/rte_mempool_octeontx_version.map | 3 + usertools/dpdk-devbind.py | 4 +- 15 files changed, 1415 insertions(+), 2 deletions(-) create mode 100644 drivers/event/octeontx/timvf_evdev.c create mode 100644 drivers/event/octeontx/timvf_evdev.h create mode 100644 drivers/event/octeontx/timvf_worker.c create mode 100644 drivers/event/octeontx/timvf_worker.h create mode 100644 drivers/mempool/octeontx/octeontx_timvf.c -- 2.16.3 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v3 01/12] mempool/octeontx: probe timvf PCIe devices 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Pavan Nikhilesh @ 2018-04-03 15:05 ` Pavan Nikhilesh 2018-04-08 2:19 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 02/12] usertools: add Cavium TIM as an event device Pavan Nikhilesh ` (11 subsequent siblings) 12 siblings, 1 reply; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-03 15:05 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh On Octeontx HW, each event timer device is enumerated as separate SRIOV VF PCIe device. In order to expose as a event timer device: On PCIe probe, the driver stores the information associated with the PCIe device and later when appliacation requests for a event timer device through `rte_event_timer_adapter_create` the driver infrastructure creates the timer adapter with earlier probed PCIe VF devices. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/mempool/octeontx/Makefile | 1 + drivers/mempool/octeontx/meson.build | 1 + drivers/mempool/octeontx/octeontx_mbox.h | 7 + drivers/mempool/octeontx/octeontx_timvf.c | 145 +++++++++++++++++++++ .../octeontx/rte_mempool_octeontx_version.map | 3 + 5 files changed, 157 insertions(+) create mode 100644 drivers/mempool/octeontx/octeontx_timvf.c diff --git a/drivers/mempool/octeontx/Makefile b/drivers/mempool/octeontx/Makefile index dfc373e62..a0547437a 100644 --- a/drivers/mempool/octeontx/Makefile +++ b/drivers/mempool/octeontx/Makefile @@ -20,6 +20,7 @@ LIBABIVER := 1 SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += octeontx_ssovf.c SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += octeontx_mbox.c SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += octeontx_fpavf.c +SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += octeontx_timvf.c SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += rte_mempool_octeontx.c ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) diff --git a/drivers/mempool/octeontx/meson.build b/drivers/mempool/octeontx/meson.build index 1e894a56b..c938436c9 100644 --- a/drivers/mempool/octeontx/meson.build +++ b/drivers/mempool/octeontx/meson.build @@ -4,6 +4,7 @@ sources = files('octeontx_ssovf.c', 'octeontx_mbox.c', 'octeontx_fpavf.c', + 'octeontx_timvf.c', 'rte_mempool_octeontx.c' ) diff --git a/drivers/mempool/octeontx/octeontx_mbox.h b/drivers/mempool/octeontx/octeontx_mbox.h index 1b056071a..7fcc9950f 100644 --- a/drivers/mempool/octeontx/octeontx_mbox.h +++ b/drivers/mempool/octeontx/octeontx_mbox.h @@ -21,6 +21,11 @@ enum octeontx_ssovf_type { OCTEONTX_SSO_HWS, /* SSO hardware workslot vf */ }; +struct octeontx_timvf_info { + uint16_t domain; /* Domain id */ + uint8_t total_timvfs; /* Total timvf available in domain */ +}; + struct octeontx_mbox_hdr { uint16_t vfid; /* VF index or pf resource index local to the domain */ uint8_t coproc; /* Coprocessor id */ @@ -32,5 +37,7 @@ int octeontx_ssovf_info(struct octeontx_ssovf_info *info); void *octeontx_ssovf_bar(enum octeontx_ssovf_type, uint8_t id, uint8_t bar); int octeontx_ssovf_mbox_send(struct octeontx_mbox_hdr *hdr, void *txdata, uint16_t txlen, void *rxdata, uint16_t rxlen); +int octeontx_timvf_info(struct octeontx_timvf_info *info); +void *octeontx_timvf_bar(uint8_t id, uint8_t bar); #endif /* __OCTEONTX_MBOX_H__ */ diff --git a/drivers/mempool/octeontx/octeontx_timvf.c b/drivers/mempool/octeontx/octeontx_timvf.c new file mode 100644 index 000000000..b20743a33 --- /dev/null +++ b/drivers/mempool/octeontx/octeontx_timvf.c @@ -0,0 +1,145 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include <rte_eal.h> +#include <rte_io.h> +#include <rte_pci.h> +#include <rte_bus_pci.h> + +#include "octeontx_mbox.h" +#include "octeontx_pool_logs.h" + +#ifndef PCI_VENDOR_ID_CAVIUM +#define PCI_VENDOR_ID_CAVIUM (0x177D) +#endif + +#define PCI_DEVICE_ID_OCTEONTX_TIM_VF (0xA051) +#define TIM_MAX_RINGS (64) + +struct timvf_res { + uint16_t domain; + uint16_t vfid; + void *bar0; + void *bar2; + void *bar4; +}; + +struct timdev { + uint8_t total_timvfs; + struct timvf_res rings[TIM_MAX_RINGS]; +}; + +static struct timdev tdev; + +int +octeontx_timvf_info(struct octeontx_timvf_info *tinfo) +{ + int i; + struct octeontx_ssovf_info info; + + if (tinfo == NULL) + return -EINVAL; + + if (!tdev.total_timvfs) + return -ENODEV; + + if (octeontx_ssovf_info(&info) < 0) + return -EINVAL; + + for (i = 0; i < tdev.total_timvfs; i++) { + if (info.domain != tdev.rings[i].domain) { + mbox_log_err("GRP error, vfid=%d/%d domain=%d/%d %p", + i, tdev.rings[i].vfid, + info.domain, tdev.rings[i].domain, + tdev.rings[i].bar0); + return -EINVAL; + } + } + + tinfo->total_timvfs = tdev.total_timvfs; + tinfo->domain = info.domain; + return 0; +} + +void* +octeontx_timvf_bar(uint8_t id, uint8_t bar) +{ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return NULL; + + if (id > tdev.total_timvfs) + return NULL; + + switch (bar) { + case 0: + return tdev.rings[id].bar0; + case 4: + return tdev.rings[id].bar4; + default: + return NULL; + } +} + +static int +timvf_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + uint64_t val; + uint16_t vfid; + struct timvf_res *res; + + RTE_SET_USED(pci_drv); + + /* For secondary processes, the primary has done all the work */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + if (pci_dev->mem_resource[0].addr == NULL || + pci_dev->mem_resource[4].addr == NULL) { + mbox_log_err("Empty bars %p %p", + pci_dev->mem_resource[0].addr, + pci_dev->mem_resource[4].addr); + return -ENODEV; + } + + val = rte_read64((uint8_t *)pci_dev->mem_resource[0].addr + 0x100); + vfid = (val >> 23) & 0xff; + if (vfid >= TIM_MAX_RINGS) { + mbox_log_err("Invalid vfid(%d/%d)", vfid, TIM_MAX_RINGS); + return -EINVAL; + } + + res = &tdev.rings[tdev.total_timvfs]; + res->vfid = vfid; + res->bar0 = pci_dev->mem_resource[0].addr; + res->bar2 = pci_dev->mem_resource[2].addr; + res->bar4 = pci_dev->mem_resource[4].addr; + res->domain = (val >> 7) & 0xffff; + tdev.total_timvfs++; + rte_wmb(); + + mbox_log_dbg("Domain=%d VFid=%d bar0 %p total_timvfs=%d", res->domain, + res->vfid, pci_dev->mem_resource[0].addr, + tdev.total_timvfs); + return 0; +} + + +static const struct rte_pci_id pci_timvf_map[] = { + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, + PCI_DEVICE_ID_OCTEONTX_TIM_VF) + }, + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver pci_timvf = { + .id_table = pci_timvf_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_IOVA_AS_VA, + .probe = timvf_probe, + .remove = NULL, +}; + +RTE_PMD_REGISTER_PCI(octeontx_timvf, pci_timvf); diff --git a/drivers/mempool/octeontx/rte_mempool_octeontx_version.map b/drivers/mempool/octeontx/rte_mempool_octeontx_version.map index fe8cdeca0..91e0a3a93 100644 --- a/drivers/mempool/octeontx/rte_mempool_octeontx_version.map +++ b/drivers/mempool/octeontx/rte_mempool_octeontx_version.map @@ -5,5 +5,8 @@ DPDK_17.11 { octeontx_ssovf_bar; octeontx_ssovf_mbox_send; + octeontx_timvf_info; + octeontx_timvf_bar; + local: *; }; -- 2.16.3 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH v3 01/12] mempool/octeontx: probe timvf PCIe devices 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 01/12] mempool/octeontx: probe timvf PCIe devices Pavan Nikhilesh @ 2018-04-08 2:19 ` Jerin Jacob 0 siblings, 0 replies; 76+ messages in thread From: Jerin Jacob @ 2018-04-08 2:19 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Tue, 3 Apr 2018 20:35:03 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH v3 01/12] mempool/octeontx: probe timvf PCIe > devices > X-Mailer: git-send-email 2.16.3 > > On Octeontx HW, each event timer device is enumerated as separate SRIOV VF > PCIe device. > > In order to expose as a event timer device: > On PCIe probe, the driver stores the information associated with the > PCIe device and later when appliacation requests for a event timer device > through `rte_event_timer_adapter_create` the driver infrastructure creates > the timer adapter with earlier probed PCIe VF devices. > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > drivers/mempool/octeontx/Makefile | 1 + > drivers/mempool/octeontx/meson.build | 1 + > drivers/mempool/octeontx/octeontx_mbox.h | 7 + > drivers/mempool/octeontx/octeontx_timvf.c | 145 +++++++++++++++++++++ Now that, we have driver/common for mailbox. Please move the octeontx_timvf.c to driver/event/octeontx/timvf_probe.c > .../octeontx/rte_mempool_octeontx_version.map | 3 + > 5 files changed, 157 insertions(+) > create mode 100644 drivers/mempool/octeontx/octeontx_timvf.c > > diff --git a/drivers/mempool/octeontx/Makefile b/drivers/mempool/octeontx/Makefile > index dfc373e62..a0547437a 100644 > --- a/drivers/mempool/octeontx/Makefile > +++ b/drivers/mempool/octeontx/Makefile > @@ -20,6 +20,7 @@ LIBABIVER := 1 > SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += octeontx_ssovf.c > SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += octeontx_mbox.c > SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += octeontx_fpavf.c > +SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += octeontx_timvf.c > SRCS-$(CONFIG_RTE_LIBRTE_OCTEONTX_MEMPOOL) += rte_mempool_octeontx.c > > ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) > diff --git a/drivers/mempool/octeontx/meson.build b/drivers/mempool/octeontx/meson.build > index 1e894a56b..c938436c9 100644 > --- a/drivers/mempool/octeontx/meson.build > +++ b/drivers/mempool/octeontx/meson.build > @@ -4,6 +4,7 @@ > sources = files('octeontx_ssovf.c', > 'octeontx_mbox.c', > 'octeontx_fpavf.c', > + 'octeontx_timvf.c', > 'rte_mempool_octeontx.c' > ) > > diff --git a/drivers/mempool/octeontx/octeontx_mbox.h b/drivers/mempool/octeontx/octeontx_mbox.h > index 1b056071a..7fcc9950f 100644 > --- a/drivers/mempool/octeontx/octeontx_mbox.h > +++ b/drivers/mempool/octeontx/octeontx_mbox.h > @@ -21,6 +21,11 @@ enum octeontx_ssovf_type { > OCTEONTX_SSO_HWS, /* SSO hardware workslot vf */ > }; > > +struct octeontx_timvf_info { > + uint16_t domain; /* Domain id */ > + uint8_t total_timvfs; /* Total timvf available in domain */ > +}; > + > struct octeontx_mbox_hdr { > uint16_t vfid; /* VF index or pf resource index local to the domain */ > uint8_t coproc; /* Coprocessor id */ > @@ -32,5 +37,7 @@ int octeontx_ssovf_info(struct octeontx_ssovf_info *info); > void *octeontx_ssovf_bar(enum octeontx_ssovf_type, uint8_t id, uint8_t bar); > int octeontx_ssovf_mbox_send(struct octeontx_mbox_hdr *hdr, > void *txdata, uint16_t txlen, void *rxdata, uint16_t rxlen); > +int octeontx_timvf_info(struct octeontx_timvf_info *info); > +void *octeontx_timvf_bar(uint8_t id, uint8_t bar); > > #endif /* __OCTEONTX_MBOX_H__ */ > diff --git a/drivers/mempool/octeontx/octeontx_timvf.c b/drivers/mempool/octeontx/octeontx_timvf.c > new file mode 100644 > index 000000000..b20743a33 > --- /dev/null > +++ b/drivers/mempool/octeontx/octeontx_timvf.c > @@ -0,0 +1,145 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2017 Cavium, Inc > + */ > + > +#include <rte_eal.h> > +#include <rte_io.h> > +#include <rte_pci.h> > +#include <rte_bus_pci.h> > + > +#include "octeontx_mbox.h" > +#include "octeontx_pool_logs.h" > + > +#ifndef PCI_VENDOR_ID_CAVIUM > +#define PCI_VENDOR_ID_CAVIUM (0x177D) > +#endif > + > +#define PCI_DEVICE_ID_OCTEONTX_TIM_VF (0xA051) > +#define TIM_MAX_RINGS (64) > + > +struct timvf_res { > + uint16_t domain; > + uint16_t vfid; > + void *bar0; > + void *bar2; > + void *bar4; > +}; > + > +struct timdev { > + uint8_t total_timvfs; > + struct timvf_res rings[TIM_MAX_RINGS]; > +}; > + > +static struct timdev tdev; > + > +int > +octeontx_timvf_info(struct octeontx_timvf_info *tinfo) > +{ > + int i; > + struct octeontx_ssovf_info info; > + > + if (tinfo == NULL) > + return -EINVAL; > + > + if (!tdev.total_timvfs) > + return -ENODEV; > + > + if (octeontx_ssovf_info(&info) < 0) > + return -EINVAL; > + > + for (i = 0; i < tdev.total_timvfs; i++) { > + if (info.domain != tdev.rings[i].domain) { > + mbox_log_err("GRP error, vfid=%d/%d domain=%d/%d %p", > + i, tdev.rings[i].vfid, > + info.domain, tdev.rings[i].domain, > + tdev.rings[i].bar0); > + return -EINVAL; > + } > + } > + > + tinfo->total_timvfs = tdev.total_timvfs; > + tinfo->domain = info.domain; > + return 0; > +} > + > +void* > +octeontx_timvf_bar(uint8_t id, uint8_t bar) > +{ > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) > + return NULL; > + > + if (id > tdev.total_timvfs) > + return NULL; > + > + switch (bar) { > + case 0: > + return tdev.rings[id].bar0; > + case 4: > + return tdev.rings[id].bar4; > + default: > + return NULL; > + } > +} > + > +static int > +timvf_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) > +{ > + uint64_t val; > + uint16_t vfid; > + struct timvf_res *res; > + > + RTE_SET_USED(pci_drv); > + > + /* For secondary processes, the primary has done all the work */ > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) > + return 0; > + > + if (pci_dev->mem_resource[0].addr == NULL || > + pci_dev->mem_resource[4].addr == NULL) { > + mbox_log_err("Empty bars %p %p", > + pci_dev->mem_resource[0].addr, > + pci_dev->mem_resource[4].addr); > + return -ENODEV; > + } > + > + val = rte_read64((uint8_t *)pci_dev->mem_resource[0].addr + 0x100); Remove 0x100 hard code or add extra comment, something like, 0x100 /* TIM_VRINGX_BASE */ > + vfid = (val >> 23) & 0xff; > + if (vfid >= TIM_MAX_RINGS) { > + mbox_log_err("Invalid vfid(%d/%d)", vfid, TIM_MAX_RINGS); > + return -EINVAL; > + } > + > + res = &tdev.rings[tdev.total_timvfs]; > + res->vfid = vfid; > + res->bar0 = pci_dev->mem_resource[0].addr; > + res->bar2 = pci_dev->mem_resource[2].addr; > + res->bar4 = pci_dev->mem_resource[4].addr; > + res->domain = (val >> 7) & 0xffff; > + tdev.total_timvfs++; > + rte_wmb(); > + > + mbox_log_dbg("Domain=%d VFid=%d bar0 %p total_timvfs=%d", res->domain, > + res->vfid, pci_dev->mem_resource[0].addr, > + tdev.total_timvfs); > + return 0; > +} > + > + > +static const struct rte_pci_id pci_timvf_map[] = { > + { > + RTE_PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, > + PCI_DEVICE_ID_OCTEONTX_TIM_VF) > + }, > + { > + .vendor_id = 0, > + }, > +}; > + > +static struct rte_pci_driver pci_timvf = { > + .id_table = pci_timvf_map, > + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_IOVA_AS_VA, > + .probe = timvf_probe, > + .remove = NULL, > +}; > + > +RTE_PMD_REGISTER_PCI(octeontx_timvf, pci_timvf); > diff --git a/drivers/mempool/octeontx/rte_mempool_octeontx_version.map b/drivers/mempool/octeontx/rte_mempool_octeontx_version.map > index fe8cdeca0..91e0a3a93 100644 > --- a/drivers/mempool/octeontx/rte_mempool_octeontx_version.map > +++ b/drivers/mempool/octeontx/rte_mempool_octeontx_version.map > @@ -5,5 +5,8 @@ DPDK_17.11 { > octeontx_ssovf_bar; > octeontx_ssovf_mbox_send; > > + octeontx_timvf_info; > + octeontx_timvf_bar; We may not need to expose this once it is moved to driver/event/octeontx With above changes: Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com> > + > local: *; > }; > -- > 2.16.3 > ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v3 02/12] usertools: add Cavium TIM as an event device 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Pavan Nikhilesh 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 01/12] mempool/octeontx: probe timvf PCIe devices Pavan Nikhilesh @ 2018-04-03 15:05 ` Pavan Nikhilesh 2018-04-08 3:23 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 03/12] event/octeontx: add support to create and free timer adapter Pavan Nikhilesh ` (10 subsequent siblings) 12 siblings, 1 reply; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-03 15:05 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- usertools/dpdk-devbind.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py index ff4b186e8..d0c420906 100755 --- a/usertools/dpdk-devbind.py +++ b/usertools/dpdk-devbind.py @@ -22,12 +22,14 @@ 'SVendor': None, 'SDevice': None} cavium_pkx = {'Class': '08', 'Vendor': '177d', 'Device': 'a0dd,a049', 'SVendor': None, 'SDevice': None} +cavium_tim = {'Class': '08', 'Vendor': '177d', 'Device': 'a051', + 'SVendor': None, 'SDevice': None} avp_vnic = {'Class': '05', 'Vendor': '1af4', 'Device': '1110', 'SVendor': None, 'SDevice': None} network_devices = [network_class, cavium_pkx, avp_vnic] crypto_devices = [encryption_class, intel_processor_class] -eventdev_devices = [cavium_sso] +eventdev_devices = [cavium_sso, cavium_tim] mempool_devices = [cavium_fpa] # global dict ethernet devices present. Dictionary indexed by PCI address. -- 2.16.3 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH v3 02/12] usertools: add Cavium TIM as an event device 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 02/12] usertools: add Cavium TIM as an event device Pavan Nikhilesh @ 2018-04-08 3:23 ` Jerin Jacob 0 siblings, 0 replies; 76+ messages in thread From: Jerin Jacob @ 2018-04-08 3:23 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Tue, 3 Apr 2018 20:35:04 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH v3 02/12] usertools: add Cavium TIM as an event > device > X-Mailer: git-send-email 2.16.3 > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > usertools/dpdk-devbind.py | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py > index ff4b186e8..d0c420906 100755 > --- a/usertools/dpdk-devbind.py > +++ b/usertools/dpdk-devbind.py > @@ -22,12 +22,14 @@ > 'SVendor': None, 'SDevice': None} > cavium_pkx = {'Class': '08', 'Vendor': '177d', 'Device': 'a0dd,a049', > 'SVendor': None, 'SDevice': None} > +cavium_tim = {'Class': '08', 'Vendor': '177d', 'Device': 'a051', > + 'SVendor': None, 'SDevice': None} > avp_vnic = {'Class': '05', 'Vendor': '1af4', 'Device': '1110', > 'SVendor': None, 'SDevice': None} > > network_devices = [network_class, cavium_pkx, avp_vnic] > crypto_devices = [encryption_class, intel_processor_class] > -eventdev_devices = [cavium_sso] > +eventdev_devices = [cavium_sso, cavium_tim] > mempool_devices = [cavium_fpa] Cc: thomas@monjalon.net Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com> > ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v3 03/12] event/octeontx: add support to create and free timer adapter 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Pavan Nikhilesh 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 01/12] mempool/octeontx: probe timvf PCIe devices Pavan Nikhilesh 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 02/12] usertools: add Cavium TIM as an event device Pavan Nikhilesh @ 2018-04-03 15:05 ` Pavan Nikhilesh 2018-04-08 7:39 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 04/12] event/octeontx: add support to start and stop timer device Pavan Nikhilesh ` (9 subsequent siblings) 12 siblings, 1 reply; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-03 15:05 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh When the application requests to create a timer device, Octeontx TIM create does the following: - Get the requested TIMvf ring based on adapter_id. - Verify the config parameters supplied. - Allocate memory required for * Buckets based on min and max timeout supplied. * Allocate the chunk pool based on the number of timers. - Clear the interrupts. On Free: - Free the allocated bucket and chunk memory. - Free private data used by TIMvf. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/Makefile | 2 + drivers/event/octeontx/meson.build | 3 +- drivers/event/octeontx/ssovf_evdev.c | 3 + drivers/event/octeontx/timvf_evdev.c | 161 ++++++++++++++++++++++++++++++++++ drivers/event/octeontx/timvf_evdev.h | 165 +++++++++++++++++++++++++++++++++++ 5 files changed, 333 insertions(+), 1 deletion(-) create mode 100644 drivers/event/octeontx/timvf_evdev.c create mode 100644 drivers/event/octeontx/timvf_evdev.h diff --git a/drivers/event/octeontx/Makefile b/drivers/event/octeontx/Makefile index 0e49efd84..570a79412 100644 --- a/drivers/event/octeontx/Makefile +++ b/drivers/event/octeontx/Makefile @@ -12,6 +12,7 @@ LIB = librte_pmd_octeontx_ssovf.a CFLAGS += $(WERROR_FLAGS) CFLAGS += -I$(RTE_SDK)/drivers/mempool/octeontx/ CFLAGS += -I$(RTE_SDK)/drivers/net/octeontx/ +CFLAGS += -DALLOW_EXPERIMENTAL_API LDLIBS += -lrte_eal -lrte_eventdev -lrte_mempool_octeontx -lrte_pmd_octeontx LDLIBS += -lrte_bus_pci -lrte_mempool -lrte_mbuf -lrte_kvargs @@ -27,6 +28,7 @@ LIBABIVER := 1 SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_worker.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_evdev.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_evdev_selftest.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += timvf_evdev.c ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_ssovf_worker.o += -fno-prefetch-loop-arrays diff --git a/drivers/event/octeontx/meson.build b/drivers/event/octeontx/meson.build index 358fc9fc9..8941f8a56 100644 --- a/drivers/event/octeontx/meson.build +++ b/drivers/event/octeontx/meson.build @@ -3,7 +3,8 @@ sources = files('ssovf_worker.c', 'ssovf_evdev.c', - 'ssovf_evdev_selftest.c' + 'ssovf_evdev_selftest.c', + 'timvf_evdev.c', ) deps += ['mempool_octeontx', 'bus_vdev', 'pmd_octeontx'] diff --git a/drivers/event/octeontx/ssovf_evdev.c b/drivers/event/octeontx/ssovf_evdev.c index 771bcfc9e..d2feeef50 100644 --- a/drivers/event/octeontx/ssovf_evdev.c +++ b/drivers/event/octeontx/ssovf_evdev.c @@ -18,6 +18,7 @@ #include <rte_bus_vdev.h> #include "ssovf_evdev.h" +#include "timvf_evdev.h" int otx_logtype_ssovf; @@ -621,6 +622,8 @@ static struct rte_eventdev_ops ssovf_ops = { .eth_rx_adapter_start = ssovf_eth_rx_adapter_start, .eth_rx_adapter_stop = ssovf_eth_rx_adapter_stop, + .timer_adapter_caps_get = timvf_timer_adapter_caps_get, + .dev_selftest = test_eventdev_octeontx, .dump = ssovf_dump, diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c new file mode 100644 index 000000000..473311fa4 --- /dev/null +++ b/drivers/event/octeontx/timvf_evdev.c @@ -0,0 +1,161 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include "timvf_evdev.h" + +int otx_logtype_timvf; + +RTE_INIT(otx_timvf_init_log); +static void +otx_timvf_init_log(void) +{ + otx_logtype_timvf = rte_log_register("pmd.event.octeontx.timer"); + if (otx_logtype_timvf >= 0) + rte_log_set_level(otx_logtype_timvf, RTE_LOG_NOTICE); +} + +static void +timvf_ring_info_get(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer_adapter_info *adptr_info) +{ + struct timvf_ring *timr = adptr->data->adapter_priv; + adptr_info->max_tmo_ns = timr->max_tout; + adptr_info->min_resolution_ns = timr->tck_nsec; + rte_memcpy(&adptr_info->conf, &adptr->data->conf, + sizeof(struct rte_event_timer_adapter_conf)); +} + +static int +timvf_ring_create(struct rte_event_timer_adapter *adptr) +{ + char pool_name[25]; + int ret; + uint64_t nb_timers; + struct rte_event_timer_adapter_conf *rcfg = &adptr->data->conf; + struct timvf_ring *timr; + struct octeontx_timvf_info tinfo; + const char *mempool_ops; + + if (octeontx_timvf_info(&tinfo) < 0) + return -ENODEV; + + if (adptr->data->id >= tinfo.total_timvfs) + return -ENODEV; + + timr = rte_zmalloc("octeontx_timvf_priv", + sizeof(struct timvf_ring), 0); + if (timr == NULL) + return -ENOMEM; + + adptr->data->adapter_priv = timr; + /* Check config parameters. */ + if ((rcfg->clk_src != RTE_EVENT_TIMER_ADAPTER_EXT_CLK0) && + (!rcfg->timer_tick_ns || + rcfg->timer_tick_ns < TIM_MIN_INTERVAL)) { + timvf_log_err("Too low timer ticks"); + goto cfg_err; + } + + switch (rcfg->clk_src) { + case RTE_EVENT_TIMER_ADAPTER_CPU_CLK: + timr->clk_src = TIM_CLK_SRC_SCLK; + break; + case RTE_EVENT_TIMER_ADAPTER_EXT_CLK0: + timr->clk_src = TIM_CLK_SRC_GPIO; + break; + case RTE_EVENT_TIMER_ADAPTER_EXT_CLK1: + timr->clk_src = TIM_CLK_SRC_GTI; + break; + case RTE_EVENT_TIMER_ADAPTER_EXT_CLK2: + timr->clk_src = TIM_CLK_SRC_PTP; + break; + default: + timvf_log_err("Invalid clk source specified."); + goto cfg_err; + } + + timr->tim_ring_id = adptr->data->id; + timr->tck_nsec = rcfg->timer_tick_ns; + timr->max_tout = rcfg->max_tmo_ns; + timr->meta.nb_bkts = (timr->max_tout / timr->tck_nsec) + 1; + timr->vbar0 = octeontx_timvf_bar(timr->tim_ring_id, 0); + timr->bkt_pos = (uint8_t *)timr->vbar0 + TIM_VRING_REL; + nb_timers = rcfg->nb_timers; + timr->meta.get_target_bkt = bkt_mod; + + timr->nb_chunks = nb_timers / nb_chunk_slots; + + timr->meta.bkt = rte_zmalloc("octeontx_timvf_bucket", + (timr->meta.nb_bkts) * sizeof(struct tim_mem_bucket), + 0); + if (timr->meta.bkt == NULL) + goto mem_err; + + snprintf(pool_name, 30, "timvf_meta.chunk_pool%d", timr->tim_ring_id); + timr->meta.chunk_pool = (void *)rte_mempool_create_empty(pool_name, + timr->nb_chunks, TIM_CHUNK_SIZE, 0, 0, rte_socket_id(), + 0); + + if (!timr->meta.chunk_pool) { + rte_free(timr->meta.bkt); + timvf_log_err("Unable to create chunkpool."); + return -ENOMEM; + } + + mempool_ops = rte_mbuf_best_mempool_ops(); + ret = rte_mempool_set_ops_byname(timr->meta.chunk_pool, + mempool_ops, NULL); + + if (ret != 0) { + timvf_log_err("Unable to set chunkpool ops."); + goto mem_err; + } + + ret = rte_mempool_populate_default(timr->meta.chunk_pool); + if (ret < 0) { + timvf_log_err("Unable to set populate chunkpool."); + goto mem_err; + } + timvf_write64(0, (uint8_t *)timr->vbar0 + TIM_VRING_BASE); + timvf_write64(0, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_INT); + timvf_write64(0, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_INT_W1S); + timvf_write64(0x7, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_ENA_W1C); + timvf_write64(0x7, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_ENA_W1S); + + return 0; +mem_err: + rte_free(timr); + return -ENOMEM; +cfg_err: + rte_free(timr); + return -EINVAL; +} + +static int +timvf_ring_free(struct rte_event_timer_adapter *adptr) +{ + struct timvf_ring *timr = adptr->data->adapter_priv; + rte_mempool_free(timr->meta.chunk_pool); + rte_free(timr->meta.bkt); + rte_free(adptr->data->adapter_priv); + return 0; +} + +static struct rte_event_timer_adapter_ops timvf_ops = { + .init = timvf_ring_create, + .uninit = timvf_ring_free, + .get_info = timvf_ring_info_get, +}; + +int +timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, + uint32_t *caps, const struct rte_event_timer_adapter_ops **ops) +{ + RTE_SET_USED(dev); + RTE_SET_USED(flags); + *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; + *ops = &timvf_ops; + return -EINVAL; +} diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h new file mode 100644 index 000000000..e3f558e10 --- /dev/null +++ b/drivers/event/octeontx/timvf_evdev.h @@ -0,0 +1,165 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#ifndef __TIMVF_EVDEV_H__ +#define __TIMVF_EVDEV_H__ + +#include <rte_common.h> +#include <rte_cycles.h> +#include <rte_debug.h> +#include <rte_eal.h> +#include <rte_eventdev.h> +#include <rte_event_timer_adapter.h> +#include <rte_event_timer_adapter_pmd.h> +#include <rte_io.h> +#include <rte_lcore.h> +#include <rte_log.h> +#include <rte_malloc.h> +#include <rte_mbuf_pool_ops.h> +#include <rte_mempool.h> +#include <rte_memzone.h> +#include <rte_pci.h> +#include <rte_prefetch.h> +#include <rte_reciprocal.h> + +#include <octeontx_mbox.h> + +#define timvf_log(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, otx_logtype_timvf, \ + "[%s] %s() " fmt "\n", \ + RTE_STR(event_timer_octeontx), __func__, ## args) + +#define timvf_log_info(fmt, ...) timvf_log(INFO, fmt, ##__VA_ARGS__) +#define timvf_log_dbg(fmt, ...) timvf_log(DEBUG, fmt, ##__VA_ARGS__) +#define timvf_log_err(fmt, ...) timvf_log(ERR, fmt, ##__VA_ARGS__) +#define timvf_func_trace timvf_log_dbg + +#define TIM_COPROC (8) +#define TIM_GET_DEV_INFO (1) +#define TIM_GET_RING_INFO (2) +#define TIM_SET_RING_INFO (3) +#define TIM_RING_START_CYC_GET (4) + +#define TIM_MAX_RINGS (64) +#define TIM_DEV_PER_NODE (1) +#define TIM_VF_PER_DEV (64) +#define TIM_RING_PER_DEV (TIM_VF_PER_DEV) +#define TIM_RING_NODE_SHIFT (6) +#define TIM_RING_MASK ((TIM_RING_PER_DEV) - 1) +#define TIM_RING_INVALID (-1) + +#define TIM_MIN_INTERVAL (1E3) +#define TIM_MAX_INTERVAL ((1ull << 32) - 1) +#define TIM_MAX_BUCKETS (1ull << 20) +#define TIM_CHUNK_SIZE (4096) +#define TIM_MAX_CHUNKS_PER_BUCKET (1ull << 32) + +#define TIMVF_MAX_BURST (8) + +/* TIM VF Control/Status registers (CSRs): */ +/* VF_BAR0: */ +#define TIM_VF_NRSPERR_INT (0x0) +#define TIM_VF_NRSPERR_INT_W1S (0x8) +#define TIM_VF_NRSPERR_ENA_W1C (0x10) +#define TIM_VF_NRSPERR_ENA_W1S (0x18) +#define TIM_VRING_FR_RN_CYCLES (0x20) +#define TIM_VRING_FR_RN_GPIOS (0x28) +#define TIM_VRING_FR_RN_GTI (0x30) +#define TIM_VRING_FR_RN_PTP (0x38) +#define TIM_VRING_CTL0 (0x40) +#define TIM_VRING_CTL1 (0x50) +#define TIM_VRING_CTL2 (0x60) +#define TIM_VRING_BASE (0x100) +#define TIM_VRING_AURA (0x108) +#define TIM_VRING_REL (0x110) + +#define timvf_read64 rte_read64_relaxed +#define timvf_write64 rte_write64_relaxed + +#ifndef __hot +#define __hot __attribute__((hot)) +#endif + +extern int otx_logtype_timvf; + +static const uint16_t nb_chunk_slots = (TIM_CHUNK_SIZE / 16) - 1; + +enum timvf_clk_src { + TIM_CLK_SRC_SCLK, + TIM_CLK_SRC_GPIO, + TIM_CLK_SRC_GTI, + TIM_CLK_SRC_PTP, +}; + +/* TIM_MEM_BUCKET */ +struct tim_mem_bucket { + uint64_t first_chunk; + union { + uint64_t w1; + struct { + uint32_t nb_entry; + uint8_t sbt:1; + uint8_t hbt:1; + uint8_t bsk:1; + uint8_t rsvd:5; + uint8_t lock; + int16_t chunk_remainder; + }; + }; + uint64_t current_chunk; + uint64_t pad; +} __rte_packed; + +struct tim_mem_entry { + uint64_t w0; + uint64_t wqe; +} __rte_packed; + +struct timvf_ctrl_reg { + uint64_t rctrl0; + uint64_t rctrl1; + uint64_t rctrl2; + uint8_t use_pmu; +} __rte_packed; + +struct timvf_ring; + +typedef uint32_t (*bkt_id)(const uint32_t bkt_tcks, const uint32_t nb_bkts); +typedef struct tim_mem_entry * (*refill_chunk)( + struct tim_mem_bucket * const bkt, + struct timvf_ring * const timr); + +struct timvf_meta { + bkt_id get_target_bkt; + refill_chunk refill_chunk; + struct rte_reciprocal_u64 fast_div; + uint64_t ring_start_cyc; + uint32_t nb_bkts; + struct tim_mem_bucket *bkt; + void *chunk_pool; + uint64_t tck_int; +}; + +struct timvf_ring { + struct timvf_meta meta; + uint64_t tck_nsec; + void *vbar0; + void *bkt_pos; + uint64_t max_tout; + uint64_t nb_chunks; + enum timvf_clk_src clk_src; + uint16_t tim_ring_id; +} __rte_cache_aligned; + +static __rte_always_inline uint32_t +bkt_mod(const uint32_t rel_bkt, const uint32_t nb_bkts) +{ + return rel_bkt % nb_bkts; +} + +int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, + uint32_t *caps, const struct rte_event_timer_adapter_ops **ops); + +#endif /* __TIMVF_EVDEV_H__ */ -- 2.16.3 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH v3 03/12] event/octeontx: add support to create and free timer adapter 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 03/12] event/octeontx: add support to create and free timer adapter Pavan Nikhilesh @ 2018-04-08 7:39 ` Jerin Jacob 0 siblings, 0 replies; 76+ messages in thread From: Jerin Jacob @ 2018-04-08 7:39 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Tue, 3 Apr 2018 20:35:05 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH v3 03/12] event/octeontx: add support to create > and free timer adapter > X-Mailer: git-send-email 2.16.3 > > When the application requests to create a timer device, Octeontx TIM > create does the following: > - Get the requested TIMvf ring based on adapter_id. > - Verify the config parameters supplied. > - Allocate memory required for > * Buckets based on min and max timeout supplied. > * Allocate the chunk pool based on the number of timers. > - Clear the interrupts. > > On Free: > - Free the allocated bucket and chunk memory. > - Free private data used by TIMvf. > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > +static int > +timvf_ring_create(struct rte_event_timer_adapter *adptr) > +{ > + > + switch (rcfg->clk_src) { > + case RTE_EVENT_TIMER_ADAPTER_CPU_CLK: while defining the enum, equate TIM_CLK_SRC_SCLK, TIM_CLK_SRC_GPIO.. values to RTE_EVENT_TIMER_ADAPTER_CPU_CLK, RTE_EVENT_TIMER_ADAPTER_EXT_CLK0 .. etc to avoid switch case. > + timr->clk_src = TIM_CLK_SRC_SCLK; > + break; > + case RTE_EVENT_TIMER_ADAPTER_EXT_CLK0: > + timr->clk_src = TIM_CLK_SRC_GPIO; > + break; > + case RTE_EVENT_TIMER_ADAPTER_EXT_CLK1: > + timr->clk_src = TIM_CLK_SRC_GTI; > + break; > + case RTE_EVENT_TIMER_ADAPTER_EXT_CLK2: > + timr->clk_src = TIM_CLK_SRC_PTP; > + break; > + default: > + timvf_log_err("Invalid clk source specified."); > + goto cfg_err; > + } > + > + timvf_write64(0, (uint8_t *)timr->vbar0 + TIM_VRING_BASE); > + timvf_write64(0, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_INT); > + timvf_write64(0, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_INT_W1S); > + timvf_write64(0x7, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_ENA_W1C); > + timvf_write64(0x7, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_ENA_W1S); > + > + return 0; > +mem_err: > + rte_free(timr); > + return -ENOMEM; > +cfg_err: > + rte_free(timr); > + return -EINVAL; > +} > + > +static int > +timvf_ring_free(struct rte_event_timer_adapter *adptr) > +{ > + struct timvf_ring *timr = adptr->data->adapter_priv; > + rte_mempool_free(timr->meta.chunk_pool); > + rte_free(timr->meta.bkt); > + rte_free(adptr->data->adapter_priv); > + return 0; > +} > + > +static struct rte_event_timer_adapter_ops timvf_ops = { use const > + .init = timvf_ring_create, Found additional tab > + .uninit = timvf_ring_free, > + .get_info = timvf_ring_info_get, > +}; > + > +int > +timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, > + uint32_t *caps, const struct rte_event_timer_adapter_ops **ops) > +{ > + RTE_SET_USED(dev); > + RTE_SET_USED(flags); > + *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; > + *ops = &timvf_ops; > + return -EINVAL; > +} > +enum timvf_clk_src { > + TIM_CLK_SRC_SCLK, See above comment. > + TIM_CLK_SRC_GPIO, > + TIM_CLK_SRC_GTI, > + TIM_CLK_SRC_PTP, > +}; > + > +struct timvf_meta { > + bkt_id get_target_bkt; > + refill_chunk refill_chunk; > + struct rte_reciprocal_u64 fast_div; > + uint64_t ring_start_cyc; > + uint32_t nb_bkts; > + struct tim_mem_bucket *bkt; > + void *chunk_pool; > + uint64_t tck_int; > +}; > + > +struct timvf_ring { > + struct timvf_meta meta; IMO, Additional 'meta' indirection can be avoid to reduce the code clutter. > + uint64_t tck_nsec; > + void *vbar0; > + void *bkt_pos; > + uint64_t max_tout; > + uint64_t nb_chunks; > + enum timvf_clk_src clk_src; > + uint16_t tim_ring_id; > +} __rte_cache_aligned; > + ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v3 04/12] event/octeontx: add support to start and stop timer device 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (2 preceding siblings ...) 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 03/12] event/octeontx: add support to create and free timer adapter Pavan Nikhilesh @ 2018-04-03 15:05 ` Pavan Nikhilesh 2018-04-08 7:47 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 05/12] event/octeontx: add event timer stats get and reset Pavan Nikhilesh ` (8 subsequent siblings) 12 siblings, 1 reply; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-03 15:05 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh When application requests to start the timer adapter through `rte_event_timer_adapter_start`, Octeontx TIMvf ring does the following: - Uses mbox to communicate TIMpf driver about, * SCLK frequency used to convert ns<->cycles. * program the ring control parameters and start the ring. * get the exact cycle at which the TIMvf ring has started which can be used to estimate the bucket position. On `rte_event_timer_adapter_stop` i.e stop, Octeontx TIMvf ring does the following: - Use mbox to communicate TIMpf driver about, * reset the ring control parameters and stop the ring. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 140 +++++++++++++++++++++++++++++++++++ drivers/event/octeontx/timvf_evdev.h | 5 ++ 2 files changed, 145 insertions(+) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index 473311fa4..ccf724115 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -16,6 +16,32 @@ otx_timvf_init_log(void) rte_log_set_level(otx_logtype_timvf, RTE_LOG_NOTICE); } +struct __rte_packed timvf_mbox_dev_info { + uint64_t ring_active[4]; + uint64_t clk_freq; +}; + +/* Response messages */ +enum { + MBOX_RET_SUCCESS, + MBOX_RET_INVALID, + MBOX_RET_INTERNAL_ERR, +}; + +static int +timvf_mbox_dev_info_get(struct timvf_mbox_dev_info *info) +{ + struct octeontx_mbox_hdr hdr = {0}; + uint16_t len = sizeof(struct timvf_mbox_dev_info); + + hdr.coproc = TIM_COPROC; + hdr.msg = TIM_GET_DEV_INFO; + hdr.vfid = 0; /* TIM DEV is always 0. TIM RING ID changes. */ + + memset(info, 0, len); + return octeontx_ssovf_mbox_send(&hdr, NULL, 0, info, len); +} + static void timvf_ring_info_get(const struct rte_event_timer_adapter *adptr, struct rte_event_timer_adapter_info *adptr_info) @@ -27,6 +53,118 @@ timvf_ring_info_get(const struct rte_event_timer_adapter *adptr, sizeof(struct rte_event_timer_adapter_conf)); } +static int +timvf_ring_conf_set(struct timvf_ctrl_reg *rctl, uint8_t ring_id) +{ + struct octeontx_mbox_hdr hdr = {0}; + uint16_t len = sizeof(struct timvf_ctrl_reg); + int ret; + + hdr.coproc = TIM_COPROC; + hdr.msg = TIM_SET_RING_INFO; + hdr.vfid = ring_id; + + ret = octeontx_ssovf_mbox_send(&hdr, rctl, len, NULL, 0); + if (ret < 0 || hdr.res_code != MBOX_RET_SUCCESS) + return -EACCES; + return 0; +} + +static int +timvf_get_start_cyc(uint64_t *now, uint8_t ring_id) +{ + struct octeontx_mbox_hdr hdr = {0}; + + hdr.coproc = TIM_COPROC; + hdr.msg = TIM_RING_START_CYC_GET; + hdr.vfid = ring_id; + *now = 0; + return octeontx_ssovf_mbox_send(&hdr, NULL, 0, now, sizeof(uint64_t)); +} + +static int +timvf_ring_start(const struct rte_event_timer_adapter *adptr) +{ + int ret; + uint64_t interval = 0; + struct timvf_ctrl_reg rctrl = {0}; + struct timvf_mbox_dev_info dinfo; + struct timvf_ring *timr = adptr->data->adapter_priv; + + ret = timvf_mbox_dev_info_get(&dinfo); + if (ret < 0 || ret != sizeof(struct timvf_mbox_dev_info)) + return -EINVAL; + + /* Calculate the interval cycles according to clock source. */ + switch (timr->clk_src) { + case TIM_CLK_SRC_SCLK: + interval = NSEC2CLK(timr->tck_nsec, dinfo.clk_freq); + break; + case TIM_CLK_SRC_GPIO: + /* GPIO doesn't work on tck_nsec. */ + interval = 0; + break; + case TIM_CLK_SRC_GTI: + interval = NSEC2CLK(timr->tck_nsec, dinfo.clk_freq); + break; + case TIM_CLK_SRC_PTP: + interval = NSEC2CLK(timr->tck_nsec, dinfo.clk_freq); + break; + } + + /*CTRL0 register.*/ + rctrl.rctrl0 = interval; + + /*CTRL1 register.*/ + rctrl.rctrl1 = (uint64_t)(timr->clk_src) << 51 | + 1ull << 48 | + 1ull << 47 | + 1ull << 44 | + (timr->meta.nb_bkts - 1); + + rctrl.rctrl2 = (uint64_t)(TIM_CHUNK_SIZE / 16) << 40; + + timvf_write64((uint64_t)timr->meta.bkt, + (uint8_t *)timr->vbar0 + TIM_VRING_BASE); + if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) { + ret = -EACCES; + goto error; + } + + if (timvf_get_start_cyc(&timr->meta.ring_start_cyc, + timr->tim_ring_id) < 0) { + ret = -EACCES; + goto error; + } + timr->meta.tck_int = NSEC2CLK(timr->tck_nsec, rte_get_timer_hz()); + timr->meta.fast_div = rte_reciprocal_value_u64(timr->meta.tck_int); + timvf_log_info("nb_bkts %d min_ns %"PRIu64" min_cyc %"PRIu64"" + " maxtmo %"PRIu64"\n", + timr->meta.nb_bkts, timr->tck_nsec, interval, + timr->max_tout); + + return 0; +error: + rte_free(timr->meta.bkt); + rte_mempool_free(timr->meta.chunk_pool); + return ret; +} + +static int +timvf_ring_stop(const struct rte_event_timer_adapter *adptr) +{ + struct timvf_ring *timr = adptr->data->adapter_priv; + struct timvf_ctrl_reg rctrl = {0}; + rctrl.rctrl0 = timvf_read64((uint8_t *)timr->vbar0 + TIM_VRING_CTL0); + rctrl.rctrl1 = timvf_read64((uint8_t *)timr->vbar0 + TIM_VRING_CTL1); + rctrl.rctrl1 &= ~(1ull << 47); /* Disable */ + rctrl.rctrl2 = timvf_read64((uint8_t *)timr->vbar0 + TIM_VRING_CTL2); + + if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) + return -EACCES; + return 0; +} + static int timvf_ring_create(struct rte_event_timer_adapter *adptr) { @@ -146,6 +284,8 @@ timvf_ring_free(struct rte_event_timer_adapter *adptr) static struct rte_event_timer_adapter_ops timvf_ops = { .init = timvf_ring_create, .uninit = timvf_ring_free, + .start = timvf_ring_start, + .stop = timvf_ring_stop, .get_info = timvf_ring_info_get, }; diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index e3f558e10..e155b6ce2 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -75,6 +75,11 @@ #define TIM_VRING_AURA (0x108) #define TIM_VRING_REL (0x110) + +#define NSEC_PER_SEC 1E9 +#define NSEC2CLK(__ns, __freq) (((__ns) * (__freq)) / NSEC_PER_SEC) +#define CLK2NSEC(__clk, __freq) (((__clk) * NSEC_PER_SEC) / (__freq)) + #define timvf_read64 rte_read64_relaxed #define timvf_write64 rte_write64_relaxed -- 2.16.3 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH v3 04/12] event/octeontx: add support to start and stop timer device 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 04/12] event/octeontx: add support to start and stop timer device Pavan Nikhilesh @ 2018-04-08 7:47 ` Jerin Jacob 0 siblings, 0 replies; 76+ messages in thread From: Jerin Jacob @ 2018-04-08 7:47 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Tue, 3 Apr 2018 20:35:06 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH v3 04/12] event/octeontx: add support to start > and stop timer device > X-Mailer: git-send-email 2.16.3 > > When application requests to start the timer adapter through > `rte_event_timer_adapter_start`, Octeontx TIMvf ring does the > following: > - Uses mbox to communicate TIMpf driver about, > * SCLK frequency used to convert ns<->cycles. > * program the ring control parameters and start the ring. > * get the exact cycle at which the TIMvf ring has started which can be > used to estimate the bucket position. > > On `rte_event_timer_adapter_stop` i.e stop, Octeontx TIMvf ring does the > following: > - Use mbox to communicate TIMpf driver about, > * reset the ring control parameters and stop the ring. > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > drivers/event/octeontx/timvf_evdev.c | 140 +++++++++++++++++++++++++++++++++++ > drivers/event/octeontx/timvf_evdev.h | 5 ++ > 2 files changed, 145 insertions(+) > > +/* Response messages */ > +enum { > + MBOX_RET_SUCCESS, > + MBOX_RET_INVALID, > + MBOX_RET_INTERNAL_ERR, > +}; If it is duplicate definition then remove it. > + > +static int > +timvf_mbox_dev_info_get(struct timvf_mbox_dev_info *info) > +{ > + struct octeontx_mbox_hdr hdr = {0}; > + uint16_t len = sizeof(struct timvf_mbox_dev_info); > + > + hdr.coproc = TIM_COPROC; > + hdr.msg = TIM_GET_DEV_INFO; > + hdr.vfid = 0; /* TIM DEV is always 0. TIM RING ID changes. */ > + > + memset(info, 0, len); > + return octeontx_ssovf_mbox_send(&hdr, NULL, 0, info, len); rebase to latest dpdk-next-eventdev where mbox api changed to octeontx_mbox_send > +} > + > static void > timvf_ring_info_get(const struct rte_event_timer_adapter *adptr, > struct rte_event_timer_adapter_info *adptr_info) > @@ -27,6 +53,118 @@ timvf_ring_info_get(const struct rte_event_timer_adapter *adptr, > sizeof(struct rte_event_timer_adapter_conf)); > } > > +static int > +timvf_ring_start(const struct rte_event_timer_adapter *adptr) > +{ > + int ret; > + uint64_t interval = 0; This assignment can be avoided. > + struct timvf_ctrl_reg rctrl = {0}; > + struct timvf_mbox_dev_info dinfo; > + struct timvf_ring *timr = adptr->data->adapter_priv; > + > + ret = timvf_mbox_dev_info_get(&dinfo); > + if (ret < 0 || ret != sizeof(struct timvf_mbox_dev_info)) > + return -EINVAL; > + > + /* Calculate the interval cycles according to clock source. */ > + switch (timr->clk_src) { > + case TIM_CLK_SRC_SCLK: > + interval = NSEC2CLK(timr->tck_nsec, dinfo.clk_freq); > + break; > + case TIM_CLK_SRC_GPIO: > + /* GPIO doesn't work on tck_nsec. */ > + interval = 0; > + break; > + case TIM_CLK_SRC_GTI: > + interval = NSEC2CLK(timr->tck_nsec, dinfo.clk_freq); > + break; > + case TIM_CLK_SRC_PTP: > + interval = NSEC2CLK(timr->tck_nsec, dinfo.clk_freq); > + break; Shouldn't we return error if clock source is not supported? > + } > + > + /*CTRL0 register.*/ > + rctrl.rctrl0 = interval; > + > + /*CTRL1 register.*/ > + rctrl.rctrl1 = (uint64_t)(timr->clk_src) << 51 | > + 1ull << 48 | > + 1ull << 47 | > + 1ull << 44 | Add comments to this bit definitions. > + (timr->meta.nb_bkts - 1); > + > + rctrl.rctrl2 = (uint64_t)(TIM_CHUNK_SIZE / 16) << 40; > + > + timvf_write64((uint64_t)timr->meta.bkt, > + (uint8_t *)timr->vbar0 + TIM_VRING_BASE); > + if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) { > + ret = -EACCES; > + goto error; > + } > + > + if (timvf_get_start_cyc(&timr->meta.ring_start_cyc, > + timr->tim_ring_id) < 0) { > + ret = -EACCES; > + goto error; > + } > + timr->meta.tck_int = NSEC2CLK(timr->tck_nsec, rte_get_timer_hz()); > + timr->meta.fast_div = rte_reciprocal_value_u64(timr->meta.tck_int); > + timvf_log_info("nb_bkts %d min_ns %"PRIu64" min_cyc %"PRIu64"" > + " maxtmo %"PRIu64"\n", > + timr->meta.nb_bkts, timr->tck_nsec, interval, > + timr->max_tout); > + > + return 0; > +error: > + rte_free(timr->meta.bkt); > + rte_mempool_free(timr->meta.chunk_pool); > + return ret; > +} > + > +static int > +timvf_ring_stop(const struct rte_event_timer_adapter *adptr) > +{ > + struct timvf_ring *timr = adptr->data->adapter_priv; > + struct timvf_ctrl_reg rctrl = {0}; > + rctrl.rctrl0 = timvf_read64((uint8_t *)timr->vbar0 + TIM_VRING_CTL0); > + rctrl.rctrl1 = timvf_read64((uint8_t *)timr->vbar0 + TIM_VRING_CTL1); > + rctrl.rctrl1 &= ~(1ull << 47); /* Disable */ > + rctrl.rctrl2 = timvf_read64((uint8_t *)timr->vbar0 + TIM_VRING_CTL2); > + > + if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) > + return -EACCES; > + return 0; > +} > + > static int > timvf_ring_create(struct rte_event_timer_adapter *adptr) > { > @@ -146,6 +284,8 @@ timvf_ring_free(struct rte_event_timer_adapter *adptr) > static struct rte_event_timer_adapter_ops timvf_ops = { can be 'const'? > .init = timvf_ring_create, > .uninit = timvf_ring_free, > + .start = timvf_ring_start, > + .stop = timvf_ring_stop, Found two tabs > .get_info = timvf_ring_info_get, > }; > > diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h > index e3f558e10..e155b6ce2 100644 > --- a/drivers/event/octeontx/timvf_evdev.h > +++ b/drivers/event/octeontx/timvf_evdev.h > @@ -75,6 +75,11 @@ > #define TIM_VRING_AURA (0x108) > #define TIM_VRING_REL (0x110) > > + > +#define NSEC_PER_SEC 1E9 > +#define NSEC2CLK(__ns, __freq) (((__ns) * (__freq)) / NSEC_PER_SEC) > +#define CLK2NSEC(__clk, __freq) (((__clk) * NSEC_PER_SEC) / (__freq)) > + > #define timvf_read64 rte_read64_relaxed > #define timvf_write64 rte_write64_relaxed > > -- > 2.16.3 > ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v3 05/12] event/octeontx: add event timer stats get and reset 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (3 preceding siblings ...) 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 04/12] event/octeontx: add support to start and stop timer device Pavan Nikhilesh @ 2018-04-03 15:05 ` Pavan Nikhilesh 2018-04-04 1:59 ` Carrillo, Erik G 2018-04-08 7:52 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 06/12] event/octeontx: add multiproducer timer arm and cancel Pavan Nikhilesh ` (7 subsequent siblings) 12 siblings, 2 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-03 15:05 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Add functions to get and reset event timer adapter stats. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 26 ++++++++++++++++++++++++++ drivers/event/octeontx/timvf_evdev.h | 1 + 2 files changed, 27 insertions(+) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index ccf724115..4db10cdd0 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -281,12 +281,38 @@ timvf_ring_free(struct rte_event_timer_adapter *adptr) return 0; } + +static int +timvf_stats_get(const struct rte_event_timer_adapter *adapter, + struct rte_event_timer_adapter_stats *stats) +{ + struct timvf_ring *timr = adapter->data->adapter_priv; + uint64_t bkt_cyc = rte_rdtsc() - timr->meta.ring_start_cyc; + + stats->evtim_exp_count = timr->meta.tim_arm_cnt; + stats->ev_enq_count = timr->meta.tim_arm_cnt; + stats->adapter_tick_count = rte_reciprocal_divide_u64(bkt_cyc, + &timr->meta.fast_div); + return 0; +} + +static int +timvf_stats_reset(const struct rte_event_timer_adapter *adapter) +{ + struct timvf_ring *timr = adapter->data->adapter_priv; + + timr->meta.tim_arm_cnt = 0; + return 0; +} + static struct rte_event_timer_adapter_ops timvf_ops = { .init = timvf_ring_create, .uninit = timvf_ring_free, .start = timvf_ring_start, .stop = timvf_ring_stop, .get_info = timvf_ring_info_get, + .stats_get = timvf_stats_get, + .stats_reset = timvf_stats_reset, }; int diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index e155b6ce2..e98ab631a 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -145,6 +145,7 @@ struct timvf_meta { struct tim_mem_bucket *bkt; void *chunk_pool; uint64_t tck_int; + volatile uint64_t tim_arm_cnt; }; struct timvf_ring { -- 2.16.3 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH v3 05/12] event/octeontx: add event timer stats get and reset 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 05/12] event/octeontx: add event timer stats get and reset Pavan Nikhilesh @ 2018-04-04 1:59 ` Carrillo, Erik G 2018-04-08 7:52 ` Jerin Jacob 1 sibling, 0 replies; 76+ messages in thread From: Carrillo, Erik G @ 2018-04-04 1:59 UTC (permalink / raw) To: Pavan Nikhilesh, jerin.jacob, santosh.shukla; +Cc: dev > -----Original Message----- > From: Pavan Nikhilesh [mailto:pbhagavatula@caviumnetworks.com] > Sent: Tuesday, April 3, 2018 10:05 AM > To: jerin.jacob@caviumnetworks.com; > santosh.shukla@caviumnetworks.com; Carrillo, Erik G > <erik.g.carrillo@intel.com> > Cc: dev@dpdk.org; Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH v3 05/12] event/octeontx: add event timer stats > get and reset > > Add functions to get and reset event timer adapter stats. > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- <...snipped...> > --- a/drivers/event/octeontx/timvf_evdev.h > +++ b/drivers/event/octeontx/timvf_evdev.h > @@ -145,6 +145,7 @@ struct timvf_meta { > struct tim_mem_bucket *bkt; > void *chunk_pool; > uint64_t tck_int; > + volatile uint64_t tim_arm_cnt; It seems like the count could be incorrect if multiple threads arm timers concurrently. Is that the case? Thanks, Gabriel > }; > > struct timvf_ring { > -- > 2.16.3 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH v3 05/12] event/octeontx: add event timer stats get and reset 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 05/12] event/octeontx: add event timer stats get and reset Pavan Nikhilesh 2018-04-04 1:59 ` Carrillo, Erik G @ 2018-04-08 7:52 ` Jerin Jacob 1 sibling, 0 replies; 76+ messages in thread From: Jerin Jacob @ 2018-04-08 7:52 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Tue, 3 Apr 2018 20:35:07 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH v3 05/12] event/octeontx: add event timer stats > get and reset > X-Mailer: git-send-email 2.16.3 > > Add functions to get and reset event timer adapter stats. > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > drivers/event/octeontx/timvf_evdev.c | 26 ++++++++++++++++++++++++++ > drivers/event/octeontx/timvf_evdev.h | 1 + > 2 files changed, 27 insertions(+) > > diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c > index ccf724115..4db10cdd0 100644 > --- a/drivers/event/octeontx/timvf_evdev.c > +++ b/drivers/event/octeontx/timvf_evdev.c > @@ -281,12 +281,38 @@ timvf_ring_free(struct rte_event_timer_adapter *adptr) > return 0; > } > > + > +static int > +timvf_stats_get(const struct rte_event_timer_adapter *adapter, > + struct rte_event_timer_adapter_stats *stats) > +{ > + struct timvf_ring *timr = adapter->data->adapter_priv; > + uint64_t bkt_cyc = rte_rdtsc() - timr->meta.ring_start_cyc; > + > + stats->evtim_exp_count = timr->meta.tim_arm_cnt; > + stats->ev_enq_count = timr->meta.tim_arm_cnt; We are updating the these counts in fastpath. Right? As an optimization, may be we could take the arguments from vdev to choose to enable "stats" at runtime by making it two function pointers. By default it can be disabled. ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v3 06/12] event/octeontx: add multiproducer timer arm and cancel 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (4 preceding siblings ...) 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 05/12] event/octeontx: add event timer stats get and reset Pavan Nikhilesh @ 2018-04-03 15:05 ` Pavan Nikhilesh 2018-04-08 7:59 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 07/12] event/octeontx: add single producer timer arm variant Pavan Nikhilesh ` (6 subsequent siblings) 12 siblings, 1 reply; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-03 15:05 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/Makefile | 5 + drivers/event/octeontx/meson.build | 1 + drivers/event/octeontx/timvf_evdev.c | 7 +- drivers/event/octeontx/timvf_evdev.h | 32 ++++ drivers/event/octeontx/timvf_worker.c | 95 ++++++++++++ drivers/event/octeontx/timvf_worker.h | 280 ++++++++++++++++++++++++++++++++++ 6 files changed, 419 insertions(+), 1 deletion(-) create mode 100644 drivers/event/octeontx/timvf_worker.c create mode 100644 drivers/event/octeontx/timvf_worker.h diff --git a/drivers/event/octeontx/Makefile b/drivers/event/octeontx/Makefile index 570a79412..58fa6af88 100644 --- a/drivers/event/octeontx/Makefile +++ b/drivers/event/octeontx/Makefile @@ -28,19 +28,24 @@ LIBABIVER := 1 SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_worker.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_evdev.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_evdev_selftest.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += timvf_worker.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += timvf_evdev.c ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_ssovf_worker.o += -fno-prefetch-loop-arrays +CFLAGS_timvf_worker.o += -fno-prefetch-loop-arrays ifeq ($(shell test $(GCC_VERSION) -ge 46 && echo 1), 1) CFLAGS_ssovf_worker.o += -Ofast +CFLAGS_timvf_worker.o += -Ofast else CFLAGS_ssovf_worker.o += -O3 -ffast-math +CFLAGS_timvf_worker.o += -O3 -ffast-math endif else CFLAGS_ssovf_worker.o += -Ofast +CFLAGS_timvf_worker.o += -Ofast endif include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/event/octeontx/meson.build b/drivers/event/octeontx/meson.build index 8941f8a56..efac94cba 100644 --- a/drivers/event/octeontx/meson.build +++ b/drivers/event/octeontx/meson.build @@ -4,6 +4,7 @@ sources = files('ssovf_worker.c', 'ssovf_evdev.c', 'ssovf_evdev_selftest.c', + 'timvf_worker.c', 'timvf_evdev.c', ) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index 4db10cdd0..c8865e33c 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -126,6 +126,7 @@ timvf_ring_start(const struct rte_event_timer_adapter *adptr) timvf_write64((uint64_t)timr->meta.bkt, (uint8_t *)timr->vbar0 + TIM_VRING_BASE); + timvf_set_chunk_refill(timr); if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) { ret = -EACCES; goto error; @@ -321,7 +322,11 @@ timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, { RTE_SET_USED(dev); RTE_SET_USED(flags); + + timvf_ops.arm_burst = timvf_timer_reg_burst_mp; + timvf_ops.cancel_burst = timvf_timer_unreg_burst; + *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; *ops = &timvf_ops; - return -EINVAL; + return 0; } diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index e98ab631a..ddf50643f 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -75,6 +75,33 @@ #define TIM_VRING_AURA (0x108) #define TIM_VRING_REL (0x110) +#define TIM_CTL1_W0_S_BUCKET 20 +#define TIM_CTL1_W0_M_BUCKET ((1ull << (40 - 20)) - 1) + +#define TIM_BUCKET_W1_S_NUM_ENTRIES (0) /*Shift*/ +#define TIM_BUCKET_W1_M_NUM_ENTRIES ((1ull << (32 - 0)) - 1) +#define TIM_BUCKET_W1_S_SBT (32) +#define TIM_BUCKET_W1_M_SBT ((1ull << (33 - 32)) - 1) +#define TIM_BUCKET_W1_S_HBT (33) +#define TIM_BUCKET_W1_M_HBT ((1ull << (34 - 33)) - 1) +#define TIM_BUCKET_W1_S_BSK (34) +#define TIM_BUCKET_W1_M_BSK ((1ull << (35 - 34)) - 1) +#define TIM_BUCKET_W1_S_LOCK (40) +#define TIM_BUCKET_W1_M_LOCK ((1ull << (48 - 40)) - 1) +#define TIM_BUCKET_W1_S_CHUNK_REMAINDER (48) +#define TIM_BUCKET_W1_M_CHUNK_REMAINDER ((1ull << (64 - 48)) - 1) + +#define TIM_BUCKET_SEMA \ + (TIM_BUCKET_CHUNK_REMAIN) + +#define TIM_BUCKET_CHUNK_REMAIN \ + (TIM_BUCKET_W1_M_CHUNK_REMAINDER << TIM_BUCKET_W1_S_CHUNK_REMAINDER) + +#define TIM_BUCKET_LOCK \ + (TIM_BUCKET_W1_M_LOCK << TIM_BUCKET_W1_S_LOCK) + +#define TIM_BUCKET_SEMA_WLOCK \ + (TIM_BUCKET_CHUNK_REMAIN | (1ull << TIM_BUCKET_W1_S_LOCK)) #define NSEC_PER_SEC 1E9 #define NSEC2CLK(__ns, __freq) (((__ns) * (__freq)) / NSEC_PER_SEC) @@ -167,5 +194,10 @@ bkt_mod(const uint32_t rel_bkt, const uint32_t nb_bkts) int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, uint32_t *caps, const struct rte_event_timer_adapter_ops **ops); +uint16_t timvf_timer_unreg_burst(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers); +uint16_t timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers); +void timvf_set_chunk_refill(struct timvf_ring * const timr); #endif /* __TIMVF_EVDEV_H__ */ diff --git a/drivers/event/octeontx/timvf_worker.c b/drivers/event/octeontx/timvf_worker.c new file mode 100644 index 000000000..50794d37b --- /dev/null +++ b/drivers/event/octeontx/timvf_worker.c @@ -0,0 +1,95 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include "timvf_worker.h" + +static inline int +timvf_timer_reg_checks(const struct timvf_ring * const timr, + struct rte_event_timer * const tim) +{ + if (unlikely(tim->state)) { + tim->state = RTE_EVENT_TIMER_ERROR; + rte_errno = EALREADY; + goto fail; + } + + if (unlikely(!tim->timeout_ticks || + tim->timeout_ticks >= timr->meta.nb_bkts)) { + tim->state = tim->timeout_ticks ? RTE_EVENT_TIMER_ERROR_TOOLATE + : RTE_EVENT_TIMER_ERROR_TOOEARLY; + rte_errno = EINVAL; + goto fail; + } + + return 0; +fail: + return -EINVAL; +} + +static inline void +timvf_format_event(const struct rte_event_timer * const tim, + struct tim_mem_entry * const entry) +{ + entry->w0 = (tim->ev.event & 0xFFC000000000) >> 6 | + (tim->ev.event & 0xFFFFFFFFF); + entry->wqe = tim->ev.u64; +} + +uint16_t +timvf_timer_unreg_burst(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers) +{ + RTE_SET_USED(adptr); + int ret; + uint16_t index; + + for (index = 0; index < nb_timers; index++) { + if (tim[index]->state == RTE_EVENT_TIMER_CANCELED) { + rte_errno = EALREADY; + break; + } + + if (tim[index]->state != RTE_EVENT_TIMER_ARMED) { + rte_errno = EINVAL; + break; + } + ret = timvf_rem_entry(tim[index]); + if (ret) { + rte_errno = -ret; + break; + } + } + return index; +} + +uint16_t +timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers) +{ + int ret; + uint16_t index; + struct tim_mem_entry entry; + struct timvf_ring *timr = adptr->data->adapter_priv; + for (index = 0; index < nb_timers; index++) { + if (timvf_timer_reg_checks(timr, tim[index])) + break; + timvf_format_event(tim[index], &entry); + ret = timvf_add_entry_mp(timr, tim[index]->timeout_ticks, + tim[index], &entry); + if (unlikely(ret)) { + rte_errno = -ret; + break; + } + } + timr->meta.tim_arm_cnt += index; + + return index; +} + +void +timvf_set_chunk_refill(struct timvf_ring * const timr) +{ + timr->meta.refill_chunk = timvf_refill_chunk_generic; +} diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h new file mode 100644 index 000000000..bba4985ab --- /dev/null +++ b/drivers/event/octeontx/timvf_worker.h @@ -0,0 +1,280 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include <rte_common.h> +#include <rte_branch_prediction.h> + +#include "timvf_evdev.h" + +static inline int16_t +timr_bkt_fetch_rem(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_CHUNK_REMAINDER) & + TIM_BUCKET_W1_M_CHUNK_REMAINDER; +} + +static inline int16_t +timr_bkt_get_rem(struct tim_mem_bucket *bktp) +{ + return __atomic_load_n((int16_t *)&bktp->chunk_remainder, + __ATOMIC_ACQUIRE); +} + +static inline void +timr_bkt_set_rem(struct tim_mem_bucket *bktp, uint16_t v) +{ + __atomic_store_n((int16_t *)&bktp->chunk_remainder, v, + __ATOMIC_RELEASE); +} + +static inline void +timr_bkt_sub_rem(struct tim_mem_bucket *bktp, uint16_t v) +{ + __atomic_fetch_sub((int16_t *)&bktp->chunk_remainder, v, + __ATOMIC_RELEASE); +} + +static inline uint8_t +timr_bkt_get_sbt(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_SBT) & TIM_BUCKET_W1_M_SBT; +} + +static inline uint64_t +timr_bkt_set_sbt(struct tim_mem_bucket *bktp) +{ + const uint64_t v = TIM_BUCKET_W1_M_SBT << TIM_BUCKET_W1_S_SBT; + return __atomic_fetch_or((uint64_t *)&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline uint64_t +timr_bkt_clr_sbt(struct tim_mem_bucket *bktp) +{ + const uint64_t v = ~(TIM_BUCKET_W1_M_SBT << TIM_BUCKET_W1_S_SBT); + return __atomic_fetch_and((uint64_t *)&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline uint8_t +timr_bkt_get_shbt(uint64_t w1) +{ + return ((w1 >> TIM_BUCKET_W1_S_HBT) & TIM_BUCKET_W1_M_HBT) | + ((w1 >> TIM_BUCKET_W1_S_SBT) & TIM_BUCKET_W1_M_SBT); +} + +static inline uint8_t +timr_bkt_get_hbt(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_HBT) & TIM_BUCKET_W1_M_HBT; +} + +static inline uint8_t +timr_bkt_get_bsk(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_BSK) & TIM_BUCKET_W1_M_BSK; +} + +static inline uint64_t +timr_bkt_clr_bsk(struct tim_mem_bucket *bktp) +{ + /*Clear everything except lock. */ + const uint64_t v = TIM_BUCKET_W1_M_LOCK << TIM_BUCKET_W1_S_LOCK; + return __atomic_fetch_and((uint64_t *)&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline uint64_t +timr_bkt_fetch_sema_lock(struct tim_mem_bucket *bktp) +{ + return __atomic_fetch_add((uint64_t *)&bktp->w1, TIM_BUCKET_SEMA_WLOCK, + __ATOMIC_ACQ_REL); +} + +static inline uint64_t +timr_bkt_fetch_sema(struct tim_mem_bucket *bktp) +{ + return __atomic_fetch_add((uint64_t *)&bktp->w1, TIM_BUCKET_SEMA, + __ATOMIC_RELAXED); +} + +static inline uint64_t +timr_bkt_inc_lock(struct tim_mem_bucket *bktp) +{ + const uint64_t v = 1ull << TIM_BUCKET_W1_S_LOCK; + return __atomic_fetch_add((uint64_t *)&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline void +timr_bkt_dec_lock(struct tim_mem_bucket *bktp) +{ + __atomic_add_fetch((uint8_t *)&bktp->lock, 0xff, __ATOMIC_ACQ_REL); +} + +static inline uint32_t +timr_bkt_get_nent(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_NUM_ENTRIES) & + TIM_BUCKET_W1_M_NUM_ENTRIES; +} + +static inline void +timr_bkt_inc_nent(struct tim_mem_bucket *bktp) +{ + __atomic_add_fetch((uint32_t *)&bktp->nb_entry, 1, __ATOMIC_RELAXED); +} + +static inline void +timr_bkt_add_nent(struct tim_mem_bucket *bktp, uint32_t v) +{ + __atomic_add_fetch((uint32_t *)&bktp->nb_entry, v, __ATOMIC_RELAXED); +} + +static inline uint64_t +timr_bkt_clr_nent(struct tim_mem_bucket *bktp) +{ + const uint64_t v = ~(TIM_BUCKET_W1_M_NUM_ENTRIES << + TIM_BUCKET_W1_S_NUM_ENTRIES); + return __atomic_and_fetch((uint64_t *)&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline struct tim_mem_entry * +timr_clr_bkt(struct timvf_ring * const timr, struct tim_mem_bucket * const bkt) +{ + struct tim_mem_entry *chunk; + struct tim_mem_entry *pnext; + chunk = ((struct tim_mem_entry *)bkt->first_chunk); + chunk = (struct tim_mem_entry *)(chunk + nb_chunk_slots)->w0; + + while (chunk) { + pnext = (struct tim_mem_entry *)((chunk + nb_chunk_slots)->w0); + rte_mempool_put(timr->meta.chunk_pool, chunk); + chunk = pnext; + } + return (struct tim_mem_entry *)bkt->first_chunk; +} + +static inline int +timvf_rem_entry(struct rte_event_timer *tim) +{ + uint64_t lock_sema; + struct tim_mem_entry *entry; + struct tim_mem_bucket *bkt; + if (tim->impl_opaque[1] == 0 || + tim->impl_opaque[0] == 0) + return -ENOENT; + + entry = (struct tim_mem_entry *)tim->impl_opaque[0]; + if (entry->wqe != tim->ev.u64) { + tim->impl_opaque[1] = tim->impl_opaque[0] = 0; + return -ENOENT; + } + bkt = (struct tim_mem_bucket *)tim->impl_opaque[1]; + lock_sema = timr_bkt_inc_lock(bkt); + if (timr_bkt_get_shbt(lock_sema) + || !timr_bkt_get_nent(lock_sema)) { + timr_bkt_dec_lock(bkt); + tim->impl_opaque[1] = tim->impl_opaque[0] = 0; + return -ENOENT; + } + + entry->w0 = entry->wqe = 0; + timr_bkt_dec_lock(bkt); + + tim->state = RTE_EVENT_TIMER_CANCELED; + tim->impl_opaque[1] = tim->impl_opaque[0] = 0; + return 0; +} + +static inline struct tim_mem_entry * +timvf_refill_chunk_generic(struct tim_mem_bucket * const bkt, + struct timvf_ring * const timr) +{ + struct tim_mem_entry *chunk; + + if (bkt->nb_entry || !bkt->first_chunk) { + if (unlikely(rte_mempool_get(timr->meta.chunk_pool, + (void **)&chunk))) { + return NULL; + } + if (bkt->nb_entry) { + *(uint64_t *)((struct tim_mem_entry *) + bkt->current_chunk + + nb_chunk_slots) = + (uint64_t) chunk; + } else { + bkt->first_chunk = (uint64_t) chunk; + } + } else { + chunk = timr_clr_bkt(timr, bkt); + bkt->first_chunk = (uint64_t) chunk; + } + *(uint64_t *)(chunk + nb_chunk_slots) = 0; + + return chunk; +} + +static inline struct tim_mem_bucket * +timvf_get_target_bucket(struct timvf_ring * const timr, const uint32_t rel_bkt) +{ + const uint64_t bkt_cyc = rte_rdtsc() - timr->meta.ring_start_cyc; + const uint32_t bucket = rte_reciprocal_divide_u64(bkt_cyc, + &timr->meta.fast_div) + rel_bkt; + const uint32_t tbkt_id = timr->meta.get_target_bkt(bucket, + timr->meta.nb_bkts); + return &timr->meta.bkt[tbkt_id]; +} + +/* Multi producer functions. */ +static inline int +timvf_add_entry_mp(struct timvf_ring * const timr, const uint32_t rel_bkt, + struct rte_event_timer * const tim, + const struct tim_mem_entry * const pent) +{ + uint8_t lock_cnt; + int16_t rem; + uint64_t lock_sema; + struct tim_mem_bucket *bkt; + struct tim_mem_entry *chunk; + +__retry: + bkt = timvf_get_target_bucket(timr, rel_bkt); + /* Bucket related checks. */ + /*Get Bucket sema*/ + lock_sema = timr_bkt_fetch_sema_lock(bkt); + if (unlikely(timr_bkt_get_shbt(lock_sema))) { + timr_bkt_dec_lock(bkt); + goto __retry; + } + + RTE_SET_USED(lock_cnt); + rem = timr_bkt_fetch_rem(lock_sema); + + if (rem < 0) { + /* goto diff bucket. */ + timr_bkt_dec_lock(bkt); + goto __retry; + } else if (!rem) { + /*Only one thread can be here*/ + chunk = timr->meta.refill_chunk(bkt, timr); + if (unlikely(chunk == NULL)) { + timr_bkt_set_rem(bkt, 0); + timr_bkt_dec_lock(bkt); + tim->impl_opaque[0] = tim->impl_opaque[1] = 0; + tim->state = RTE_EVENT_TIMER_ERROR; + return -ENOMEM; + } + bkt->current_chunk = (uint64_t) chunk; + timr_bkt_set_rem(bkt, nb_chunk_slots - 1); + } else { + chunk = (struct tim_mem_entry *)bkt->current_chunk; + chunk += nb_chunk_slots - rem; + } + /* Copy work entry. */ + *chunk = *pent; + timr_bkt_inc_nent(bkt); + timr_bkt_dec_lock(bkt); + + tim->impl_opaque[0] = (uint64_t)chunk; + tim->impl_opaque[1] = (uint64_t)bkt; + tim->state = RTE_EVENT_TIMER_ARMED; + return 0; +} -- 2.16.3 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH v3 06/12] event/octeontx: add multiproducer timer arm and cancel 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 06/12] event/octeontx: add multiproducer timer arm and cancel Pavan Nikhilesh @ 2018-04-08 7:59 ` Jerin Jacob 0 siblings, 0 replies; 76+ messages in thread From: Jerin Jacob @ 2018-04-08 7:59 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Tue, 3 Apr 2018 20:35:08 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH v3 06/12] event/octeontx: add multiproducer > timer arm and cancel > X-Mailer: git-send-email 2.16.3 > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > +static inline int16_t > +timr_bkt_get_rem(struct tim_mem_bucket *bktp) > +{ > + return __atomic_load_n((int16_t *)&bktp->chunk_remainder, > + __ATOMIC_ACQUIRE); > +} > + > +static inline void > +timr_bkt_set_rem(struct tim_mem_bucket *bktp, uint16_t v) > +{ > + __atomic_store_n((int16_t *)&bktp->chunk_remainder, v, typecast is not required for __atomic_* gcc atomic builtins functions. > + __ATOMIC_RELEASE); > +} > + > +static inline void > +timr_bkt_sub_rem(struct tim_mem_bucket *bktp, uint16_t v) > +{ > + __atomic_fetch_sub((int16_t *)&bktp->chunk_remainder, v, > + __ATOMIC_RELEASE); > +} > + > +{ > + > +/* Multi producer functions. */ > +static inline int > +timvf_add_entry_mp(struct timvf_ring * const timr, const uint32_t rel_bkt, > + struct rte_event_timer * const tim, > + const struct tim_mem_entry * const pent) > +{ > + uint8_t lock_cnt; > + int16_t rem; > + uint64_t lock_sema; > + struct tim_mem_bucket *bkt; > + struct tim_mem_entry *chunk; > + > +__retry: > + bkt = timvf_get_target_bucket(timr, rel_bkt); > + /* Bucket related checks. */ > + /*Get Bucket sema*/ > + lock_sema = timr_bkt_fetch_sema_lock(bkt); > + if (unlikely(timr_bkt_get_shbt(lock_sema))) { > + timr_bkt_dec_lock(bkt); > + goto __retry; > + } > + > + RTE_SET_USED(lock_cnt); lock_cnt not been used. Remove it. With above changes: Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com> ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v3 07/12] event/octeontx: add single producer timer arm variant 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (5 preceding siblings ...) 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 06/12] event/octeontx: add multiproducer timer arm and cancel Pavan Nikhilesh @ 2018-04-03 15:05 ` Pavan Nikhilesh 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 08/12] event/octeontx: add burst mode for timer arm Pavan Nikhilesh ` (5 subsequent siblings) 12 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-03 15:05 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh When application creates the timer adapter by passing `RTE_EVENT_TIMER_ADAPTER_F_SP_PUT` flag, we can optimize the arm sequence by removing the locking overhead. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 15 ++++++++--- drivers/event/octeontx/timvf_evdev.h | 2 ++ drivers/event/octeontx/timvf_worker.c | 25 +++++++++++++++++++ drivers/event/octeontx/timvf_worker.h | 47 +++++++++++++++++++++++++++++++++++ 4 files changed, 86 insertions(+), 3 deletions(-) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index c8865e33c..a32892107 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -176,6 +176,7 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) struct timvf_ring *timr; struct octeontx_timvf_info tinfo; const char *mempool_ops; + unsigned int mp_flags = 0; if (octeontx_timvf_info(&tinfo) < 0) return -ENODEV; @@ -226,6 +227,11 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) timr->nb_chunks = nb_timers / nb_chunk_slots; + if (rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_SP_PUT) { + mp_flags = MEMPOOL_F_SP_PUT | MEMPOOL_F_SC_GET; + timvf_log_info("Using single producer mode"); + } + timr->meta.bkt = rte_zmalloc("octeontx_timvf_bucket", (timr->meta.nb_bkts) * sizeof(struct tim_mem_bucket), 0); @@ -235,7 +241,7 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) snprintf(pool_name, 30, "timvf_meta.chunk_pool%d", timr->tim_ring_id); timr->meta.chunk_pool = (void *)rte_mempool_create_empty(pool_name, timr->nb_chunks, TIM_CHUNK_SIZE, 0, 0, rte_socket_id(), - 0); + mp_flags); if (!timr->meta.chunk_pool) { rte_free(timr->meta.bkt); @@ -321,9 +327,12 @@ timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, uint32_t *caps, const struct rte_event_timer_adapter_ops **ops) { RTE_SET_USED(dev); - RTE_SET_USED(flags); - timvf_ops.arm_burst = timvf_timer_reg_burst_mp; + if (flags & RTE_EVENT_TIMER_ADAPTER_F_SP_PUT) + timvf_ops.arm_burst = timvf_timer_reg_burst_sp; + else + timvf_ops.arm_burst = timvf_timer_reg_burst_mp; + timvf_ops.cancel_burst = timvf_timer_unreg_burst; *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index ddf50643f..ab2de678f 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -196,6 +196,8 @@ int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, uint32_t *caps, const struct rte_event_timer_adapter_ops **ops); uint16_t timvf_timer_unreg_burst(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers); +uint16_t timvf_timer_reg_burst_sp(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers); uint16_t timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers); void timvf_set_chunk_refill(struct timvf_ring * const timr); diff --git a/drivers/event/octeontx/timvf_worker.c b/drivers/event/octeontx/timvf_worker.c index 50794d37b..139dfdc07 100644 --- a/drivers/event/octeontx/timvf_worker.c +++ b/drivers/event/octeontx/timvf_worker.c @@ -64,6 +64,31 @@ timvf_timer_unreg_burst(const struct rte_event_timer_adapter *adptr, return index; } +uint16_t +timvf_timer_reg_burst_sp(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers) +{ + int ret; + uint16_t index; + struct tim_mem_entry entry; + struct timvf_ring *timr = adptr->data->adapter_priv; + for (index = 0; index < nb_timers; index++) { + if (timvf_timer_reg_checks(timr, tim[index])) + break; + + timvf_format_event(tim[index], &entry); + ret = timvf_add_entry_sp(timr, tim[index]->timeout_ticks, + tim[index], &entry); + if (unlikely(ret)) { + rte_errno = -ret; + break; + } + } + timr->meta.tim_arm_cnt += index; + + return index; +} + uint16_t timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers) diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h index bba4985ab..7e197f5cd 100644 --- a/drivers/event/octeontx/timvf_worker.h +++ b/drivers/event/octeontx/timvf_worker.h @@ -223,6 +223,53 @@ timvf_get_target_bucket(struct timvf_ring * const timr, const uint32_t rel_bkt) return &timr->meta.bkt[tbkt_id]; } +/* Single producer functions. */ +static inline int +timvf_add_entry_sp(struct timvf_ring * const timr, const uint32_t rel_bkt, + struct rte_event_timer * const tim, + const struct tim_mem_entry * const pent) +{ + int16_t rem; + uint64_t lock_sema; + struct tim_mem_bucket *bkt; + struct tim_mem_entry *chunk; + + + bkt = timvf_get_target_bucket(timr, rel_bkt); +__retry: + /*Get Bucket sema*/ + lock_sema = timr_bkt_fetch_sema(bkt); + /* Bucket related checks. */ + if (unlikely(timr_bkt_get_hbt(lock_sema))) + goto __retry; + + /* Insert the work. */ + rem = timr_bkt_fetch_rem(lock_sema); + + if (!rem) { + chunk = timr->meta.refill_chunk(bkt, timr); + if (unlikely(chunk == NULL)) { + timr_bkt_set_rem(bkt, 0); + tim->impl_opaque[0] = tim->impl_opaque[1] = 0; + tim->state = RTE_EVENT_TIMER_ERROR; + return -ENOMEM; + } + bkt->current_chunk = (uint64_t) chunk; + timr_bkt_set_rem(bkt, nb_chunk_slots - 1); + } else { + chunk = (struct tim_mem_entry *)bkt->current_chunk; + chunk += nb_chunk_slots - rem; + } + /* Copy work entry. */ + *chunk = *pent; + timr_bkt_inc_nent(bkt); + + tim->impl_opaque[0] = (uint64_t)chunk; + tim->impl_opaque[1] = (uint64_t)bkt; + tim->state = RTE_EVENT_TIMER_ARMED; + return 0; +} + /* Multi producer functions. */ static inline int timvf_add_entry_mp(struct timvf_ring * const timr, const uint32_t rel_bkt, -- 2.16.3 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v3 08/12] event/octeontx: add burst mode for timer arm 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (6 preceding siblings ...) 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 07/12] event/octeontx: add single producer timer arm variant Pavan Nikhilesh @ 2018-04-03 15:05 ` Pavan Nikhilesh 2018-04-04 2:15 ` Carrillo, Erik G 2018-04-08 8:06 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 09/12] event/octeontx: optimize timer adapter resolution parameters Pavan Nikhilesh ` (4 subsequent siblings) 12 siblings, 2 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-03 15:05 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 1 + drivers/event/octeontx/timvf_evdev.h | 3 ++ drivers/event/octeontx/timvf_worker.c | 38 ++++++++++++++ drivers/event/octeontx/timvf_worker.h | 95 +++++++++++++++++++++++++++++++++++ 4 files changed, 137 insertions(+) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index a32892107..b23500e0d 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -333,6 +333,7 @@ timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, else timvf_ops.arm_burst = timvf_timer_reg_burst_mp; + timvf_ops.arm_tmo_tick_burst = timvf_timer_reg_brst; timvf_ops.cancel_burst = timvf_timer_unreg_burst; *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index ab2de678f..d8a6d111f 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -200,6 +200,9 @@ uint16_t timvf_timer_reg_burst_sp(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers); uint16_t timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers); +uint16_t timvf_timer_reg_brst(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint64_t timeout_tick, + const uint16_t nb_timers); void timvf_set_chunk_refill(struct timvf_ring * const timr); #endif /* __TIMVF_EVDEV_H__ */ diff --git a/drivers/event/octeontx/timvf_worker.c b/drivers/event/octeontx/timvf_worker.c index 139dfdc07..f4f40d150 100644 --- a/drivers/event/octeontx/timvf_worker.c +++ b/drivers/event/octeontx/timvf_worker.c @@ -113,6 +113,44 @@ timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, return index; } +uint16_t +timvf_timer_reg_brst(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint64_t timeout_tick, + const uint16_t nb_timers) +{ + int ret; + uint16_t set_timers = 0; + uint16_t idx; + uint16_t arr_idx = 0; + struct timvf_ring *timr = adptr->data->adapter_priv; + struct tim_mem_entry entry[TIMVF_MAX_BURST] __rte_cache_aligned; + + if (unlikely(!timeout_tick || timeout_tick >= timr->meta.nb_bkts)) { + const enum rte_event_timer_state state = timeout_tick ? + RTE_EVENT_TIMER_ERROR_TOOLATE : + RTE_EVENT_TIMER_ERROR_TOOEARLY; + for (idx = 0; idx < nb_timers; idx++) + tim[idx]->state = state; + rte_errno = EINVAL; + return 0; + } + + while (arr_idx < nb_timers) { + for (idx = 0; idx < TIMVF_MAX_BURST && (arr_idx < nb_timers); + idx++, arr_idx++) { + timvf_format_event(tim[arr_idx], &entry[idx]); + } + ret = timvf_add_entry_brst(timr, timeout_tick, &tim[set_timers], + entry, idx); + set_timers += ret; + if (ret != idx) + break; + } + timr->meta.tim_arm_cnt += set_timers; + + return set_timers; +} + void timvf_set_chunk_refill(struct timvf_ring * const timr) { diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h index 7e197f5cd..9dad5c769 100644 --- a/drivers/event/octeontx/timvf_worker.h +++ b/drivers/event/octeontx/timvf_worker.h @@ -325,3 +325,98 @@ timvf_add_entry_mp(struct timvf_ring * const timr, const uint32_t rel_bkt, tim->state = RTE_EVENT_TIMER_ARMED; return 0; } + +static inline uint16_t +timvf_cpy_wrk(uint16_t index, uint16_t cpy_lmt, + struct tim_mem_entry *chunk, + struct rte_event_timer ** const tim, + const struct tim_mem_entry * const ents, + const struct tim_mem_bucket * const bkt) +{ + for (; index < cpy_lmt; index++) { + *chunk = *(ents + index); + tim[index]->impl_opaque[0] = (uint64_t)chunk++; + tim[index]->impl_opaque[1] = (uint64_t)bkt; + tim[index]->state = RTE_EVENT_TIMER_ARMED; + } + + return index; +} + +/* Burst mode functions */ +static inline int +timvf_add_entry_brst(struct timvf_ring * const timr, const uint16_t rel_bkt, + struct rte_event_timer ** const tim, + const struct tim_mem_entry *ents, + const uint16_t nb_timers) +{ + int16_t rem; + int16_t crem = 0; + uint8_t lock_cnt; + uint16_t index = 0; + uint16_t chunk_remainder = 0; + uint64_t lock_sema; + struct tim_mem_bucket *bkt; + struct tim_mem_entry *chunk; + +__retry: + bkt = timvf_get_target_bucket(timr, rel_bkt); + + /* Only one thread beyond this. */ + lock_sema = timr_bkt_inc_lock(bkt); + lock_cnt = (uint8_t) + ((lock_sema >> TIM_BUCKET_W1_S_LOCK) & TIM_BUCKET_W1_M_LOCK); + + if (lock_cnt) { + timr_bkt_dec_lock(bkt); + goto __retry; + } + + /* Bucket related checks. */ + if (unlikely(timr_bkt_get_hbt(lock_sema))) { + timr_bkt_dec_lock(bkt); + goto __retry; + } + + chunk_remainder = timr_bkt_fetch_rem(lock_sema); + rem = chunk_remainder - nb_timers; + if (rem < 0) { + crem = nb_chunk_slots - chunk_remainder; + if (chunk_remainder && crem) { + chunk = ((struct tim_mem_entry *)bkt->current_chunk) + + crem; + + index = timvf_cpy_wrk(index, chunk_remainder, + chunk, tim, ents, bkt); + timr_bkt_sub_rem(bkt, chunk_remainder); + timr_bkt_add_nent(bkt, chunk_remainder); + } + rem = nb_timers - chunk_remainder; + ents = ents + chunk_remainder; + + chunk = timr->meta.refill_chunk(bkt, timr); + if (unlikely(chunk == NULL)) { + timr_bkt_dec_lock(bkt); + rte_errno = ENOMEM; + tim[index]->state = RTE_EVENT_TIMER_ERROR; + return crem; + } + *(uint64_t *)(chunk + nb_chunk_slots) = 0; + bkt->current_chunk = (uint64_t) chunk; + + index = timvf_cpy_wrk(index, nb_timers, chunk, tim, ents, bkt); + timr_bkt_set_rem(bkt, nb_chunk_slots - rem); + timr_bkt_add_nent(bkt, rem); + } else { + chunk = (struct tim_mem_entry *)bkt->current_chunk; + chunk += (nb_chunk_slots - chunk_remainder); + + index = timvf_cpy_wrk(index, nb_timers, + chunk, tim, ents, bkt); + timr_bkt_sub_rem(bkt, nb_timers); + timr_bkt_add_nent(bkt, nb_timers); + } + + timr_bkt_dec_lock(bkt); + return nb_timers; +} -- 2.16.3 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH v3 08/12] event/octeontx: add burst mode for timer arm 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 08/12] event/octeontx: add burst mode for timer arm Pavan Nikhilesh @ 2018-04-04 2:15 ` Carrillo, Erik G 2018-04-08 8:06 ` Jerin Jacob 1 sibling, 0 replies; 76+ messages in thread From: Carrillo, Erik G @ 2018-04-04 2:15 UTC (permalink / raw) To: Pavan Nikhilesh, jerin.jacob, santosh.shukla; +Cc: dev > -----Original Message----- > From: Pavan Nikhilesh [mailto:pbhagavatula@caviumnetworks.com] > Sent: Tuesday, April 3, 2018 10:05 AM > To: jerin.jacob@caviumnetworks.com; > santosh.shukla@caviumnetworks.com; Carrillo, Erik G > <erik.g.carrillo@intel.com> > Cc: dev@dpdk.org; Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH v3 08/12] event/octeontx: add burst mode for > timer arm > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > drivers/event/octeontx/timvf_evdev.c | 1 + > drivers/event/octeontx/timvf_evdev.h | 3 ++ > drivers/event/octeontx/timvf_worker.c | 38 ++++++++++++++ > drivers/event/octeontx/timvf_worker.h | 95 > +++++++++++++++++++++++++++++++++++ > 4 files changed, 137 insertions(+) > > diff --git a/drivers/event/octeontx/timvf_evdev.c > b/drivers/event/octeontx/timvf_evdev.c > index a32892107..b23500e0d 100644 > --- a/drivers/event/octeontx/timvf_evdev.c > +++ b/drivers/event/octeontx/timvf_evdev.c > @@ -333,6 +333,7 @@ timvf_timer_adapter_caps_get(const struct > rte_eventdev *dev, uint64_t flags, > else > timvf_ops.arm_burst = timvf_timer_reg_burst_mp; > > + timvf_ops.arm_tmo_tick_burst = timvf_timer_reg_brst; Could there be a single-producer variant of timvf_timer_reg_brst? Thanks, Gabriel > timvf_ops.cancel_burst = timvf_timer_unreg_burst; > > *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; > diff --git a/drivers/event/octeontx/timvf_evdev.h > b/drivers/event/octeontx/timvf_evdev.h > index ab2de678f..d8a6d111f 100644 > --- a/drivers/event/octeontx/timvf_evdev.h ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH v3 08/12] event/octeontx: add burst mode for timer arm 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 08/12] event/octeontx: add burst mode for timer arm Pavan Nikhilesh 2018-04-04 2:15 ` Carrillo, Erik G @ 2018-04-08 8:06 ` Jerin Jacob 1 sibling, 0 replies; 76+ messages in thread From: Jerin Jacob @ 2018-04-08 8:06 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Tue, 3 Apr 2018 20:35:10 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH v3 08/12] event/octeontx: add burst mode for > timer arm > X-Mailer: git-send-email 2.16.3 > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > drivers/event/octeontx/timvf_evdev.c | 1 + > drivers/event/octeontx/timvf_evdev.h | 3 ++ > drivers/event/octeontx/timvf_worker.c | 38 ++++++++++++++ > drivers/event/octeontx/timvf_worker.h | 95 +++++++++++++++++++++++++++++++++++ > 4 files changed, 137 insertions(+) > > diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c > index a32892107..b23500e0d 100644 > --- a/drivers/event/octeontx/timvf_evdev.c > +++ b/drivers/event/octeontx/timvf_evdev.c > @@ -333,6 +333,7 @@ timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, > else > timvf_ops.arm_burst = timvf_timer_reg_burst_mp; > > + timvf_ops.arm_tmo_tick_burst = timvf_timer_reg_brst; IMO, To inline with spec name, may we could use arm_burst/cancel_burst instead of reg_brst or unreg_burst. > timvf_ops.cancel_burst = timvf_timer_unreg_burst; > > *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; > diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h > index ab2de678f..d8a6d111f 100644 > --- a/drivers/event/octeontx/timvf_evdev.h > +++ b/drivers/event/octeontx/timvf_evdev.h > @@ -200,6 +200,9 @@ uint16_t timvf_timer_reg_burst_sp(const struct rte_event_timer_adapter *adptr, > struct rte_event_timer **tim, const uint16_t nb_timers); > uint16_t timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, > struct rte_event_timer **tim, const uint16_t nb_timers); > +uint16_t timvf_timer_reg_brst(const struct rte_event_timer_adapter *adptr, > + struct rte_event_timer **tim, const uint64_t timeout_tick, > + const uint16_t nb_timers); > void timvf_set_chunk_refill(struct timvf_ring * const timr); > > #endif /* __TIMVF_EVDEV_H__ */ > diff --git a/drivers/event/octeontx/timvf_worker.c b/drivers/event/octeontx/timvf_worker.c > index 139dfdc07..f4f40d150 100644 > --- a/drivers/event/octeontx/timvf_worker.c > +++ b/drivers/event/octeontx/timvf_worker.c > @@ -113,6 +113,44 @@ timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, > return index; > } > > +/* Burst mode functions */ > +static inline int > +timvf_add_entry_brst(struct timvf_ring * const timr, const uint16_t rel_bkt, > + struct rte_event_timer ** const tim, > + const struct tim_mem_entry *ents, > + const uint16_t nb_timers) > +{ > + int16_t rem; > + int16_t crem = 0; > + uint8_t lock_cnt; > + uint16_t index = 0; > + uint16_t chunk_remainder = 0; Looks like above all assignments to 'zero' is unnecessary. ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v3 09/12] event/octeontx: optimize timer adapter resolution parameters 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (7 preceding siblings ...) 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 08/12] event/octeontx: add burst mode for timer arm Pavan Nikhilesh @ 2018-04-03 15:05 ` Pavan Nikhilesh 2018-04-08 8:09 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 10/12] event/octeontx: add option to use fpavf as chunk pool Pavan Nikhilesh ` (3 subsequent siblings) 12 siblings, 1 reply; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-03 15:05 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh When application sets `RTE_EVENT_TIMER_ADAPTER_F_ADJUST_RES` flag while creating adapter underlying driver is free to optimize the resolution for best possible configuration. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 48 +++++++++++++++++++++++++++++++++++- drivers/event/octeontx/timvf_evdev.h | 6 +++++ 2 files changed, 53 insertions(+), 1 deletion(-) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index b23500e0d..d1f42a3bd 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -82,6 +82,45 @@ timvf_get_start_cyc(uint64_t *now, uint8_t ring_id) return octeontx_ssovf_mbox_send(&hdr, NULL, 0, now, sizeof(uint64_t)); } +static int +optimize_bucket_parameters(struct timvf_ring *timr) +{ + uint32_t hbkts; + uint32_t lbkts; + uint64_t tck_nsec; + + hbkts = rte_align32pow2(timr->meta.nb_bkts); + tck_nsec = RTE_ALIGN_MUL_CEIL(timr->max_tout / (hbkts - 1), 10); + + if ((tck_nsec < 1000 || hbkts > TIM_MAX_BUCKETS)) + hbkts = 0; + + lbkts = rte_align32prevpow2(timr->meta.nb_bkts); + tck_nsec = RTE_ALIGN_MUL_CEIL((timr->max_tout / (lbkts - 1)), 10); + + if ((tck_nsec < 1000 || hbkts > TIM_MAX_BUCKETS)) + lbkts = 0; + + if (!hbkts && !lbkts) + return 0; + + if (!hbkts) { + timr->meta.nb_bkts = lbkts; + goto end; + } else if (!lbkts) { + timr->meta.nb_bkts = hbkts; + goto end; + } + + timr->meta.nb_bkts = (hbkts - timr->meta.nb_bkts) < + (timr->meta.nb_bkts - lbkts) ? hbkts : lbkts; +end: + timr->meta.get_target_bkt = bkt_and; + timr->tck_nsec = RTE_ALIGN_MUL_CEIL((timr->max_tout / + (timr->meta.nb_bkts - 1)), 10); + return 1; +} + static int timvf_ring_start(const struct rte_event_timer_adapter *adptr) { @@ -217,7 +256,7 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) } timr->tim_ring_id = adptr->data->id; - timr->tck_nsec = rcfg->timer_tick_ns; + timr->tck_nsec = RTE_ALIGN_MUL_CEIL(rcfg->timer_tick_ns, 10); timr->max_tout = rcfg->max_tmo_ns; timr->meta.nb_bkts = (timr->max_tout / timr->tck_nsec) + 1; timr->vbar0 = octeontx_timvf_bar(timr->tim_ring_id, 0); @@ -227,6 +266,13 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) timr->nb_chunks = nb_timers / nb_chunk_slots; + /* Try to optimize the bucket parameters. */ + if ((rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_ADJUST_RES) + && !rte_is_power_of_2(timr->meta.nb_bkts)) { + optimize_bucket_parameters(timr); + timvf_log_info("Optimizing configured values"); + } + if (rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_SP_PUT) { mp_flags = MEMPOOL_F_SP_PUT | MEMPOOL_F_SC_GET; timvf_log_info("Using single producer mode"); diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index d8a6d111f..22c8c2266 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -192,6 +192,12 @@ bkt_mod(const uint32_t rel_bkt, const uint32_t nb_bkts) return rel_bkt % nb_bkts; } +static __rte_always_inline uint32_t __hot +bkt_and(uint32_t rel_bkt, uint32_t nb_bkts) +{ + return rel_bkt & (nb_bkts - 1); +} + int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, uint32_t *caps, const struct rte_event_timer_adapter_ops **ops); uint16_t timvf_timer_unreg_burst(const struct rte_event_timer_adapter *adptr, -- 2.16.3 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH v3 09/12] event/octeontx: optimize timer adapter resolution parameters 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 09/12] event/octeontx: optimize timer adapter resolution parameters Pavan Nikhilesh @ 2018-04-08 8:09 ` Jerin Jacob 0 siblings, 0 replies; 76+ messages in thread From: Jerin Jacob @ 2018-04-08 8:09 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Tue, 3 Apr 2018 20:35:11 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH v3 09/12] event/octeontx: optimize timer adapter > resolution parameters > X-Mailer: git-send-email 2.16.3 > > When application sets `RTE_EVENT_TIMER_ADAPTER_F_ADJUST_RES` flag > while creating adapter underlying driver is free to optimize the > resolution for best possible configuration. > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > static int > timvf_ring_start(const struct rte_event_timer_adapter *adptr) > { > @@ -217,7 +256,7 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) > } > > timr->tim_ring_id = adptr->data->id; > - timr->tck_nsec = rcfg->timer_tick_ns; > + timr->tck_nsec = RTE_ALIGN_MUL_CEIL(rcfg->timer_tick_ns, 10); > timr->max_tout = rcfg->max_tmo_ns; > timr->meta.nb_bkts = (timr->max_tout / timr->tck_nsec) + 1; > timr->vbar0 = octeontx_timvf_bar(timr->tim_ring_id, 0); > @@ -227,6 +266,13 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) > > timr->nb_chunks = nb_timers / nb_chunk_slots; > > + /* Try to optimize the bucket parameters. */ > + if ((rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_ADJUST_RES) > + && !rte_is_power_of_2(timr->meta.nb_bkts)) { > + optimize_bucket_parameters(timr); > + timvf_log_info("Optimizing configured values"); You could print the adjusted values here. > + } > + > if (rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_SP_PUT) { > mp_flags = MEMPOOL_F_SP_PUT | MEMPOOL_F_SC_GET; > timvf_log_info("Using single producer mode"); > diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h > index d8a6d111f..22c8c2266 100644 > --- a/drivers/event/octeontx/timvf_evdev.h > +++ b/drivers/event/octeontx/timvf_evdev.h > @@ -192,6 +192,12 @@ bkt_mod(const uint32_t rel_bkt, const uint32_t nb_bkts) > return rel_bkt % nb_bkts; > } > > +static __rte_always_inline uint32_t __hot __hot may not be required here as it in as inline function. > +bkt_and(uint32_t rel_bkt, uint32_t nb_bkts) > +{ > + return rel_bkt & (nb_bkts - 1); > +} > + > int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, > uint32_t *caps, const struct rte_event_timer_adapter_ops **ops); > uint16_t timvf_timer_unreg_burst(const struct rte_event_timer_adapter *adptr, With above change: Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com> ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v3 10/12] event/octeontx: add option to use fpavf as chunk pool 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (8 preceding siblings ...) 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 09/12] event/octeontx: optimize timer adapter resolution parameters Pavan Nikhilesh @ 2018-04-03 15:05 ` Pavan Nikhilesh 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 11/12] doc: update eventdev OcteonTx documentation Pavan Nikhilesh ` (2 subsequent siblings) 12 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-03 15:05 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Add compile-time configurable option to force TIMvf to use Octeontx FPAvf pool manager as its chunk pool. When FPAvf is used as pool manager the TIMvf automatically frees the chunks to FPAvf through gpool-id. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 22 +++++++++++++++++++++- drivers/event/octeontx/timvf_evdev.h | 3 ++- drivers/event/octeontx/timvf_worker.c | 7 +++++-- drivers/event/octeontx/timvf_worker.h | 23 +++++++++++++++++++++++ 4 files changed, 51 insertions(+), 4 deletions(-) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index d1f42a3bd..75dd5cd65 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -125,7 +125,9 @@ static int timvf_ring_start(const struct rte_event_timer_adapter *adptr) { int ret; + uint8_t use_fpa = 0; uint64_t interval = 0; + uintptr_t pool; struct timvf_ctrl_reg rctrl = {0}; struct timvf_mbox_dev_info dinfo; struct timvf_ring *timr = adptr->data->adapter_priv; @@ -151,6 +153,9 @@ timvf_ring_start(const struct rte_event_timer_adapter *adptr) break; } + if (!strcmp(rte_mbuf_best_mempool_ops(), "octeontx_fpavf")) + use_fpa = 1; + /*CTRL0 register.*/ rctrl.rctrl0 = interval; @@ -163,9 +168,24 @@ timvf_ring_start(const struct rte_event_timer_adapter *adptr) rctrl.rctrl2 = (uint64_t)(TIM_CHUNK_SIZE / 16) << 40; + if (use_fpa) { + pool = (uintptr_t)((struct rte_mempool *) + timr->meta.chunk_pool)->pool_id; + ret = octeontx_fpa_bufpool_gpool(pool); + if (ret < 0) { + timvf_log_dbg("Unable to get gaura id"); + ret = -ENOMEM; + goto error; + } + timvf_write64((uint64_t)ret, + (uint8_t *)timr->vbar0 + TIM_VRING_AURA); + } else { + rctrl.rctrl1 |= 1ull << 43; + } + timvf_write64((uint64_t)timr->meta.bkt, (uint8_t *)timr->vbar0 + TIM_VRING_BASE); - timvf_set_chunk_refill(timr); + timvf_set_chunk_refill(timr, use_fpa); if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) { ret = -EACCES; goto error; diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index 22c8c2266..96fd33ee2 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -25,6 +25,7 @@ #include <rte_reciprocal.h> #include <octeontx_mbox.h> +#include <octeontx_fpavf.h> #define timvf_log(level, fmt, args...) \ rte_log(RTE_LOG_ ## level, otx_logtype_timvf, \ @@ -209,6 +210,6 @@ uint16_t timvf_timer_reg_burst_mp(const struct rte_event_timer_adapter *adptr, uint16_t timvf_timer_reg_brst(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint64_t timeout_tick, const uint16_t nb_timers); -void timvf_set_chunk_refill(struct timvf_ring * const timr); +void timvf_set_chunk_refill(struct timvf_ring * const timr, uint8_t use_fpa); #endif /* __TIMVF_EVDEV_H__ */ diff --git a/drivers/event/octeontx/timvf_worker.c b/drivers/event/octeontx/timvf_worker.c index f4f40d150..26ec4cc74 100644 --- a/drivers/event/octeontx/timvf_worker.c +++ b/drivers/event/octeontx/timvf_worker.c @@ -152,7 +152,10 @@ timvf_timer_reg_brst(const struct rte_event_timer_adapter *adptr, } void -timvf_set_chunk_refill(struct timvf_ring * const timr) +timvf_set_chunk_refill(struct timvf_ring * const timr, uint8_t use_fpa) { - timr->meta.refill_chunk = timvf_refill_chunk_generic; + if (use_fpa) + timr->meta.refill_chunk = timvf_refill_chunk_fpa; + else + timr->meta.refill_chunk = timvf_refill_chunk_generic; } diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h index 9dad5c769..39ce4535c 100644 --- a/drivers/event/octeontx/timvf_worker.h +++ b/drivers/event/octeontx/timvf_worker.h @@ -201,6 +201,7 @@ timvf_refill_chunk_generic(struct tim_mem_bucket * const bkt, nb_chunk_slots) = (uint64_t) chunk; } else { + bkt->first_chunk = (uint64_t) chunk; } } else { @@ -212,6 +213,28 @@ timvf_refill_chunk_generic(struct tim_mem_bucket * const bkt, return chunk; } +static inline struct tim_mem_entry * +timvf_refill_chunk_fpa(struct tim_mem_bucket * const bkt, + struct timvf_ring * const timr) +{ + struct tim_mem_entry *chunk; + + if (unlikely(rte_mempool_get(timr->meta.chunk_pool, (void **)&chunk))) + return NULL; + + *(uint64_t *)(chunk + nb_chunk_slots) = 0; + if (bkt->nb_entry) { + *(uint64_t *)((struct tim_mem_entry *) + bkt->current_chunk + + nb_chunk_slots) = + (uint64_t) chunk; + } else { + bkt->first_chunk = (uint64_t) chunk; + } + + return chunk; +} + static inline struct tim_mem_bucket * timvf_get_target_bucket(struct timvf_ring * const timr, const uint32_t rel_bkt) { -- 2.16.3 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v3 11/12] doc: update eventdev OcteonTx documentation 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (9 preceding siblings ...) 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 10/12] event/octeontx: add option to use fpavf as chunk pool Pavan Nikhilesh @ 2018-04-03 15:05 ` Pavan Nikhilesh 2018-04-04 2:19 ` Carrillo, Erik G 2018-04-08 8:13 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 12/12] maintainers: claim responsibility for octeontx timvf Pavan Nikhilesh 2018-04-08 2:55 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Jerin Jacob 12 siblings, 2 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-03 15:05 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- doc/guides/eventdevs/octeontx.rst | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/doc/guides/eventdevs/octeontx.rst b/doc/guides/eventdevs/octeontx.rst index 4fabe54f9..f4f5473fd 100644 --- a/doc/guides/eventdevs/octeontx.rst +++ b/doc/guides/eventdevs/octeontx.rst @@ -28,6 +28,9 @@ Features of the OCTEONTX SSOVF PMD are: - Open system with configurable amount of outstanding events - HW accelerated dequeue timeout support to enable power management - SR-IOV VF +- HW managed event timers support through TIMVF, with high precision and + time granularity of 1us. +- Upto 64 event timer adapters. Supported OCTEONTX SoCs ----------------------- @@ -110,3 +113,9 @@ Rx adapter support When eth_octeontx is used as Rx adapter event schedule type ``RTE_SCHED_TYPE_PARALLEL`` is not supported. + +Event timer adapter support +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When timvf is used as Event timer adapter event schedule type +``RTE_SCHED_TYPE_PARALLEL`` is not supported. -- 2.16.3 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH v3 11/12] doc: update eventdev OcteonTx documentation 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 11/12] doc: update eventdev OcteonTx documentation Pavan Nikhilesh @ 2018-04-04 2:19 ` Carrillo, Erik G 2018-04-08 8:13 ` Jerin Jacob 1 sibling, 0 replies; 76+ messages in thread From: Carrillo, Erik G @ 2018-04-04 2:19 UTC (permalink / raw) To: Pavan Nikhilesh, jerin.jacob, santosh.shukla; +Cc: dev > -----Original Message----- > From: Pavan Nikhilesh [mailto:pbhagavatula@caviumnetworks.com] > Sent: Tuesday, April 3, 2018 10:05 AM > To: jerin.jacob@caviumnetworks.com; > santosh.shukla@caviumnetworks.com; Carrillo, Erik G > <erik.g.carrillo@intel.com> > Cc: dev@dpdk.org; Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH v3 11/12] doc: update eventdev OcteonTx > documentation > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > doc/guides/eventdevs/octeontx.rst | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/doc/guides/eventdevs/octeontx.rst > b/doc/guides/eventdevs/octeontx.rst > index 4fabe54f9..f4f5473fd 100644 > --- a/doc/guides/eventdevs/octeontx.rst > +++ b/doc/guides/eventdevs/octeontx.rst > @@ -28,6 +28,9 @@ Features of the OCTEONTX SSOVF PMD are: > - Open system with configurable amount of outstanding events > - HW accelerated dequeue timeout support to enable power management > - SR-IOV VF > +- HW managed event timers support through TIMVF, with high precision > +and > + time granularity of 1us. > +- Upto 64 event timer adapters. s/Upto/Up to/ Thanks, Gabriel > > Supported OCTEONTX SoCs > ----------------------- > @@ -110,3 +113,9 @@ Rx adapter support > > When eth_octeontx is used as Rx adapter event schedule type > ``RTE_SCHED_TYPE_PARALLEL`` is not supported. > + > +Event timer adapter support > +~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +When timvf is used as Event timer adapter event schedule type > +``RTE_SCHED_TYPE_PARALLEL`` is not supported. > -- > 2.16.3 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH v3 11/12] doc: update eventdev OcteonTx documentation 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 11/12] doc: update eventdev OcteonTx documentation Pavan Nikhilesh 2018-04-04 2:19 ` Carrillo, Erik G @ 2018-04-08 8:13 ` Jerin Jacob 1 sibling, 0 replies; 76+ messages in thread From: Jerin Jacob @ 2018-04-08 8:13 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Tue, 3 Apr 2018 20:35:13 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH v3 11/12] doc: update eventdev OcteonTx > documentation > X-Mailer: git-send-email 2.16.3 > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > doc/guides/eventdevs/octeontx.rst | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/doc/guides/eventdevs/octeontx.rst b/doc/guides/eventdevs/octeontx.rst > index 4fabe54f9..f4f5473fd 100644 > --- a/doc/guides/eventdevs/octeontx.rst > +++ b/doc/guides/eventdevs/octeontx.rst > @@ -28,6 +28,9 @@ Features of the OCTEONTX SSOVF PMD are: > - Open system with configurable amount of outstanding events > - HW accelerated dequeue timeout support to enable power management > - SR-IOV VF > +- HW managed event timers support through TIMVF, with high precision and > + time granularity of 1us. > +- Upto 64 event timer adapters. s/Upto/Up to/ > > Supported OCTEONTX SoCs > ----------------------- > @@ -110,3 +113,9 @@ Rx adapter support > > When eth_octeontx is used as Rx adapter event schedule type > ``RTE_SCHED_TYPE_PARALLEL`` is not supported. > + > +Event timer adapter support > +~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +When timvf is used as Event timer adapter event schedule type > +``RTE_SCHED_TYPE_PARALLEL`` is not supported. You could add the RTE_EVENT_TIMER_ADAPTER_EXT_CLK0.. RTE_EVENT_TIMER_ADAPTER_EXT_CLK3 platform mapping here. With above changes: Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com> > -- > 2.16.3 > ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v3 12/12] maintainers: claim responsibility for octeontx timvf 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (10 preceding siblings ...) 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 11/12] doc: update eventdev OcteonTx documentation Pavan Nikhilesh @ 2018-04-03 15:05 ` Pavan Nikhilesh 2018-04-08 8:15 ` Jerin Jacob 2018-04-08 2:55 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Jerin Jacob 12 siblings, 1 reply; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-03 15:05 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- MAINTAINERS | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index d4c0cc1bc..4a4a12d3d 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -736,6 +736,10 @@ M: Santosh Shukla <santosh.shukla@caviumnetworks.com> F: drivers/event/octeontx/ F: doc/guides/eventdevs/octeontx.rst +Cavium OCTEONTX timvf +M: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> +F: drivers/event/octeontx/timvf_* + NXP DPAA2 eventdev M: Hemant Agrawal <hemant.agrawal@nxp.com> M: Nipun Gupta <nipun.gupta@nxp.com> -- 2.16.3 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH v3 12/12] maintainers: claim responsibility for octeontx timvf 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 12/12] maintainers: claim responsibility for octeontx timvf Pavan Nikhilesh @ 2018-04-08 8:15 ` Jerin Jacob 0 siblings, 0 replies; 76+ messages in thread From: Jerin Jacob @ 2018-04-08 8:15 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Tue, 3 Apr 2018 20:35:14 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH v3 12/12] maintainers: claim responsibility for > octeontx timvf > X-Mailer: git-send-email 2.16.3 > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > MAINTAINERS | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/MAINTAINERS b/MAINTAINERS > index d4c0cc1bc..4a4a12d3d 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -736,6 +736,10 @@ M: Santosh Shukla <santosh.shukla@caviumnetworks.com> > F: drivers/event/octeontx/ > F: doc/guides/eventdevs/octeontx.rst > > +Cavium OCTEONTX timvf > +M: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > +F: drivers/event/octeontx/timvf_* Please squash this patch with previous doc patch. Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com> > + > NXP DPAA2 eventdev > M: Hemant Agrawal <hemant.agrawal@nxp.com> > M: Nipun Gupta <nipun.gupta@nxp.com> > -- > 2.16.3 > ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (11 preceding siblings ...) 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 12/12] maintainers: claim responsibility for octeontx timvf Pavan Nikhilesh @ 2018-04-08 2:55 ` Jerin Jacob 2018-04-08 8:58 ` Pavan Nikhilesh 12 siblings, 1 reply; 76+ messages in thread From: Jerin Jacob @ 2018-04-08 2:55 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Tue, 3 Apr 2018 20:35:02 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer > adapter driver > X-Mailer: git-send-email 2.16.3 > > The event timer adapter[1] provides APIs to configure an event timer device > that allows an application to arm timers which on expiry push events to an > event device such as OcteonTx SSO. > The OcteonTx TIM is a co-processor that can be configured as an event timer > adapter which can be used by an application to manage event timers. > > The TIM co-processor processes the event timers registered and pushes > expired event timers to SSO based on the event queue, schedule type, flow > id etc. provided as rte_event while arming the event timer. It maintains > event timers with high precision and time granularity of 1us (microsecond). > > [1] http://dpdk.org/dev/patchwork/patch/33525/ > > This patch set depends on: > 1. http://dpdk.org/dev/patchwork/bundle/pbhagavatula/event_timerdev_v9 > 2. http://dpdk.org/dev/patchwork/patch/35216/ > 3. http://dpdk.org/dev/patchwork/patch/36089/ > > v3 Changes: > - change datapath function return from int to uint16_t. > - Add missing state checks while arm/cancel. > - add stats. > > v2 Changes: > - Move common code changes to separate patches. > - Reduce ifdef clutter. > - fix improper return value (erik). > - remove redundent ops assignment (erik). > - update octeontx documentation. Please check the following : 1) It has build issue with 32bit build(gcc 7.3.1) 2) clang6 shows some crazy link errors, even though following symbol has been not used. Looks like we need -latomic undefined reference to `__atomic_fetch_add_8' 3) Fix check-git-log.sh issues $./devtools/check-git-log.sh Wrong headline lowercase: event/octeontx: add multiproducer timer arm and cancel event/octeontx: add single producer timer arm variant event/octeontx: add burst mode for timer arm ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver 2018-04-08 2:55 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Jerin Jacob @ 2018-04-08 8:58 ` Pavan Nikhilesh 0 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-08 8:58 UTC (permalink / raw) To: Jerin Jacob, santosh.shukla, erik.g.carrillo; +Cc: dev Hi Jerin, On Sun, Apr 08, 2018 at 08:25:35AM +0530, Jerin Jacob wrote: > -----Original Message----- > > Date: Tue, 3 Apr 2018 20:35:02 +0530 > > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > > erik.g.carrillo@intel.com > > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > > Subject: [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer > > adapter driver > > X-Mailer: git-send-email 2.16.3 > > > > The event timer adapter[1] provides APIs to configure an event timer device > > that allows an application to arm timers which on expiry push events to an > > event device such as OcteonTx SSO. > > The OcteonTx TIM is a co-processor that can be configured as an event timer > > adapter which can be used by an application to manage event timers. > > > > The TIM co-processor processes the event timers registered and pushes > > expired event timers to SSO based on the event queue, schedule type, flow > > id etc. provided as rte_event while arming the event timer. It maintains > > event timers with high precision and time granularity of 1us (microsecond). > > > > [1] http://dpdk.org/dev/patchwork/patch/33525/ > > > > This patch set depends on: > > 1. http://dpdk.org/dev/patchwork/bundle/pbhagavatula/event_timerdev_v9 > > 2. http://dpdk.org/dev/patchwork/patch/35216/ > > 3. http://dpdk.org/dev/patchwork/patch/36089/ > > > > v3 Changes: > > - change datapath function return from int to uint16_t. > > - Add missing state checks while arm/cancel. > > - add stats. > > > > v2 Changes: > > - Move common code changes to separate patches. > > - Reduce ifdef clutter. > > - fix improper return value (erik). > > - remove redundent ops assignment (erik). > > - update octeontx documentation. > > Please check the following : > > 1) It has build issue with 32bit build(gcc 7.3.1) > > 2) clang6 shows some crazy link errors, even though following symbol > has been not used. Looks like we need -latomic > undefined reference to `__atomic_fetch_add_8' > > 3) Fix check-git-log.sh issues > $./devtools/check-git-log.sh > Wrong headline lowercase: > event/octeontx: add multiproducer timer arm and cancel > event/octeontx: add single producer timer arm variant > event/octeontx: add burst mode for timer arm Thanks for the review, will send out v4 with the changes. Pavan. > ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v4 00/11] event/octeontx: add event timer adapter driver 2018-02-16 21:36 [dpdk-dev] [PATCH 0/9] event/octeontx: add event timer adapter driver Pavan Nikhilesh ` (11 preceding siblings ...) 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Pavan Nikhilesh @ 2018-04-09 21:00 ` Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 01/11] usertools: add Cavium TIM as an event device Pavan Nikhilesh ` (11 more replies) 12 siblings, 12 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-09 21:00 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh The event timer adapter[1] provides APIs to configure an event timer device that allows an application to arm timers which on expiry push events to an event device such as OcteonTx SSO. The OcteonTx TIM is a co-processor that can be configured as an event timer adapter which can be used by an application to manage event timers. The TIM co-processor processes the event timers registered and pushes expired event timers to SSO based on the event queue, schedule type, flow id etc. provided as rte_event while arming the event timer. It maintains event timers with high precision and time granularity of 1us (microsecond). [1] http://dpdk.org/dev/patchwork/patch/33525/ v4 Changes: - fix 32bit compilation errors. - remove dead code. - make stats configurable through vdev args. - rebased to dpdk-next-eventdev - move probe to event/octeontx from mempool/octeontx. - rename register/unregister to arm/cancel to be inline with eventdev API. - update clk src enum to be inline with eventdev clk src. Note: check-git-log.sh complains Wrong headline lowercase: event/octeontx: add multiproducer timer arm and cancel which is a false positive as it expects 'arm' to be upper case. v3 Changes: - change datapath function return from int to uint16_t. - Add missing state checks while arm/cancel. - add stats. v2 Changes: - Move common code changes to separate patches. - Reduce ifdef clutter. - fix improper return value (erik). - remove redundent ops assignment (erik). - update octeontx documentation. Pavan Nikhilesh (11): usertools: add Cavium TIM as an event device event/octeontx: add support to probe timvf PCIe devices event/octeontx: add support to create and free timer adapter event/octeontx: add support to start and stop timer device event/octeontx: add event timer stats get and reset event/octeontx: add multiproducer timer arm and cancel event/octeontx: add single producer timer arm variant event/octeontx: add burst mode for timer arm event/octeontx: optimize timer adapter resolution parameters event/octeontx: add option to use fpavf as chunk pool doc: update eventdev OcteonTx documentation MAINTAINERS | 4 + doc/guides/eventdevs/octeontx.rst | 19 ++ drivers/event/octeontx/Makefile | 8 + drivers/event/octeontx/meson.build | 6 +- drivers/event/octeontx/ssovf_evdev.c | 22 ++ drivers/event/octeontx/timvf_evdev.c | 407 +++++++++++++++++++++++ drivers/event/octeontx/timvf_evdev.h | 226 +++++++++++++ drivers/event/octeontx/timvf_probe.c | 148 +++++++++ drivers/event/octeontx/timvf_worker.c | 200 ++++++++++++ drivers/event/octeontx/timvf_worker.h | 443 ++++++++++++++++++++++++++ usertools/dpdk-devbind.py | 4 +- 11 files changed, 1485 insertions(+), 2 deletions(-) create mode 100644 drivers/event/octeontx/timvf_evdev.c create mode 100644 drivers/event/octeontx/timvf_evdev.h create mode 100644 drivers/event/octeontx/timvf_probe.c create mode 100644 drivers/event/octeontx/timvf_worker.c create mode 100644 drivers/event/octeontx/timvf_worker.h -- 2.17.0 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v4 01/11] usertools: add Cavium TIM as an event device 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 00/11] " Pavan Nikhilesh @ 2018-04-09 21:00 ` Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 02/11] event/octeontx: add support to probe timvf PCIe devices Pavan Nikhilesh ` (10 subsequent siblings) 11 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-09 21:00 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com> --- usertools/dpdk-devbind.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py index ff4b186e8..d0c420906 100755 --- a/usertools/dpdk-devbind.py +++ b/usertools/dpdk-devbind.py @@ -22,12 +22,14 @@ 'SVendor': None, 'SDevice': None} cavium_pkx = {'Class': '08', 'Vendor': '177d', 'Device': 'a0dd,a049', 'SVendor': None, 'SDevice': None} +cavium_tim = {'Class': '08', 'Vendor': '177d', 'Device': 'a051', + 'SVendor': None, 'SDevice': None} avp_vnic = {'Class': '05', 'Vendor': '1af4', 'Device': '1110', 'SVendor': None, 'SDevice': None} network_devices = [network_class, cavium_pkx, avp_vnic] crypto_devices = [encryption_class, intel_processor_class] -eventdev_devices = [cavium_sso] +eventdev_devices = [cavium_sso, cavium_tim] mempool_devices = [cavium_fpa] # global dict ethernet devices present. Dictionary indexed by PCI address. -- 2.17.0 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v4 02/11] event/octeontx: add support to probe timvf PCIe devices 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 00/11] " Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 01/11] usertools: add Cavium TIM as an event device Pavan Nikhilesh @ 2018-04-09 21:00 ` Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 03/11] event/octeontx: add support to create and free timer adapter Pavan Nikhilesh ` (9 subsequent siblings) 11 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-09 21:00 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh On Octeontx HW, each event timer device is enumerated as separate SRIOV VF PCIe device. In order to expose as a event timer device: On PCIe probe, the driver stores the information associated with the PCIe device and later when appliacation requests for a event timer device through `rte_event_timer_adapter_create` the driver infrastructure creates the timer adapter with earlier probed PCIe VF devices. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.h | 33 ++++++ drivers/event/octeontx/timvf_probe.c | 148 +++++++++++++++++++++++++++ 2 files changed, 181 insertions(+) create mode 100644 drivers/event/octeontx/timvf_evdev.h create mode 100644 drivers/event/octeontx/timvf_probe.c diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h new file mode 100644 index 000000000..7e577edf4 --- /dev/null +++ b/drivers/event/octeontx/timvf_evdev.h @@ -0,0 +1,33 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#ifndef __TIMVF_EVDEV_H__ +#define __TIMVF_EVDEV_H__ + +#include <rte_common.h> + +#include <octeontx_mbox.h> + +#define timvf_log(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, otx_logtype_timvf, \ + "[%s] %s() " fmt "\n", \ + RTE_STR(event_timer_octeontx), __func__, ## args) + +#define timvf_log_info(fmt, ...) timvf_log(INFO, fmt, ##__VA_ARGS__) +#define timvf_log_dbg(fmt, ...) timvf_log(DEBUG, fmt, ##__VA_ARGS__) +#define timvf_log_err(fmt, ...) timvf_log(ERR, fmt, ##__VA_ARGS__) +#define timvf_func_trace timvf_log_dbg + +extern int otx_logtype_timvf; + +struct timvf_info { + uint16_t domain; /* Domain id */ + uint8_t total_timvfs; /* Total timvf available in domain */ +}; + +int timvf_info(struct timvf_info *tinfo); +void *timvf_bar(uint8_t id, uint8_t bar); + +#endif /* __TIMVF_EVDEV_H__ */ diff --git a/drivers/event/octeontx/timvf_probe.c b/drivers/event/octeontx/timvf_probe.c new file mode 100644 index 000000000..08dbd2be9 --- /dev/null +++ b/drivers/event/octeontx/timvf_probe.c @@ -0,0 +1,148 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include <rte_eal.h> +#include <rte_io.h> +#include <rte_pci.h> +#include <rte_bus_pci.h> + +#include <octeontx_mbox.h> + +#include "ssovf_evdev.h" +#include "timvf_evdev.h" + +#ifndef PCI_VENDOR_ID_CAVIUM +#define PCI_VENDOR_ID_CAVIUM (0x177D) +#endif + +#define PCI_DEVICE_ID_OCTEONTX_TIM_VF (0xA051) +#define TIM_MAX_RINGS (64) + +struct timvf_res { + uint16_t domain; + uint16_t vfid; + void *bar0; + void *bar2; + void *bar4; +}; + +struct timdev { + uint8_t total_timvfs; + struct timvf_res rings[TIM_MAX_RINGS]; +}; + +static struct timdev tdev; + +int +timvf_info(struct timvf_info *tinfo) +{ + int i; + struct ssovf_info info; + + if (tinfo == NULL) + return -EINVAL; + + if (!tdev.total_timvfs) + return -ENODEV; + + if (ssovf_info(&info) < 0) + return -EINVAL; + + for (i = 0; i < tdev.total_timvfs; i++) { + if (info.domain != tdev.rings[i].domain) { + timvf_log_err("GRP error, vfid=%d/%d domain=%d/%d %p", + i, tdev.rings[i].vfid, + info.domain, tdev.rings[i].domain, + tdev.rings[i].bar0); + return -EINVAL; + } + } + + tinfo->total_timvfs = tdev.total_timvfs; + tinfo->domain = info.domain; + return 0; +} + +void* +timvf_bar(uint8_t id, uint8_t bar) +{ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return NULL; + + if (id > tdev.total_timvfs) + return NULL; + + switch (bar) { + case 0: + return tdev.rings[id].bar0; + case 4: + return tdev.rings[id].bar4; + default: + return NULL; + } +} + +static int +timvf_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + uint64_t val; + uint16_t vfid; + struct timvf_res *res; + + RTE_SET_USED(pci_drv); + + /* For secondary processes, the primary has done all the work */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + if (pci_dev->mem_resource[0].addr == NULL || + pci_dev->mem_resource[4].addr == NULL) { + timvf_log_err("Empty bars %p %p", + pci_dev->mem_resource[0].addr, + pci_dev->mem_resource[4].addr); + return -ENODEV; + } + + val = rte_read64((uint8_t *)pci_dev->mem_resource[0].addr + + 0x100 /* TIM_VRINGX_BASE */); + vfid = (val >> 23) & 0xff; + if (vfid >= TIM_MAX_RINGS) { + timvf_log_err("Invalid vfid(%d/%d)", vfid, TIM_MAX_RINGS); + return -EINVAL; + } + + res = &tdev.rings[tdev.total_timvfs]; + res->vfid = vfid; + res->bar0 = pci_dev->mem_resource[0].addr; + res->bar2 = pci_dev->mem_resource[2].addr; + res->bar4 = pci_dev->mem_resource[4].addr; + res->domain = (val >> 7) & 0xffff; + tdev.total_timvfs++; + rte_wmb(); + + timvf_log_dbg("Domain=%d VFid=%d bar0 %p total_timvfs=%d", res->domain, + res->vfid, pci_dev->mem_resource[0].addr, + tdev.total_timvfs); + return 0; +} + + +static const struct rte_pci_id pci_timvf_map[] = { + { + RTE_PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, + PCI_DEVICE_ID_OCTEONTX_TIM_VF) + }, + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver pci_timvf = { + .id_table = pci_timvf_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_IOVA_AS_VA, + .probe = timvf_probe, + .remove = NULL, +}; + +RTE_PMD_REGISTER_PCI(octeontx_timvf, pci_timvf); -- 2.17.0 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v4 03/11] event/octeontx: add support to create and free timer adapter 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 00/11] " Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 01/11] usertools: add Cavium TIM as an event device Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 02/11] event/octeontx: add support to probe timvf PCIe devices Pavan Nikhilesh @ 2018-04-09 21:00 ` Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 04/11] event/octeontx: add support to start and stop timer device Pavan Nikhilesh ` (8 subsequent siblings) 11 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-09 21:00 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh When the application requests to create a timer device, Octeontx TIM create does the following: - Get the requested TIMvf ring based on adapter_id. - Verify the config parameters supplied. - Allocate memory required for * Buckets based on min and max timeout supplied. * Allocate the chunk pool based on the number of timers. - Clear the interrupts. On Free: - Free the allocated bucket and chunk memory. - Free private data used by TIMvf. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/Makefile | 3 + drivers/event/octeontx/meson.build | 5 +- drivers/event/octeontx/ssovf_evdev.c | 10 ++ drivers/event/octeontx/timvf_evdev.c | 146 +++++++++++++++++++++++++++ drivers/event/octeontx/timvf_evdev.h | 131 ++++++++++++++++++++++++ 5 files changed, 294 insertions(+), 1 deletion(-) create mode 100644 drivers/event/octeontx/timvf_evdev.c diff --git a/drivers/event/octeontx/Makefile b/drivers/event/octeontx/Makefile index 6a6073b4d..8dbfd88c5 100644 --- a/drivers/event/octeontx/Makefile +++ b/drivers/event/octeontx/Makefile @@ -13,6 +13,7 @@ CFLAGS += $(WERROR_FLAGS) CFLAGS += -I$(RTE_SDK)/drivers/common/octeontx/ CFLAGS += -I$(RTE_SDK)/drivers/mempool/octeontx/ CFLAGS += -I$(RTE_SDK)/drivers/net/octeontx/ +CFLAGS += -DALLOW_EXPERIMENTAL_API LDLIBS += -lrte_eal -lrte_eventdev -lrte_common_octeontx -lrte_pmd_octeontx LDLIBS += -lrte_bus_pci -lrte_mempool -lrte_mbuf -lrte_kvargs @@ -29,6 +30,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_worker.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_evdev.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_evdev_selftest.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_probe.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += timvf_evdev.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += timvf_probe.c ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_ssovf_worker.o += -fno-prefetch-loop-arrays diff --git a/drivers/event/octeontx/meson.build b/drivers/event/octeontx/meson.build index 56d4207bd..90af4fd1a 100644 --- a/drivers/event/octeontx/meson.build +++ b/drivers/event/octeontx/meson.build @@ -4,7 +4,10 @@ sources = files('ssovf_worker.c', 'ssovf_evdev.c', 'ssovf_evdev_selftest.c', - 'ssovf_probe.c' + 'ssovf_probe.c', + 'timvf_evdev.c', + 'timvf_probe.c' ) +allow_experimental_apis = true deps += ['common_octeontx', 'mempool_octeontx', 'bus_vdev', 'pmd_octeontx'] diff --git a/drivers/event/octeontx/ssovf_evdev.c b/drivers/event/octeontx/ssovf_evdev.c index 2400b20d2..34496f114 100644 --- a/drivers/event/octeontx/ssovf_evdev.c +++ b/drivers/event/octeontx/ssovf_evdev.c @@ -18,6 +18,7 @@ #include <rte_bus_vdev.h> #include "ssovf_evdev.h" +#include "timvf_evdev.h" int otx_logtype_ssovf; @@ -601,6 +602,13 @@ ssovf_selftest(const char *key __rte_unused, const char *value, return 0; } +static int +ssovf_timvf_caps_get(const struct rte_eventdev *dev, uint64_t flags, + uint32_t *caps, const struct rte_event_timer_adapter_ops **ops) +{ + return timvf_timer_adapter_caps_get(dev, flags, caps, ops, 0); +} + /* Initialize and register event driver with DPDK Application */ static struct rte_eventdev_ops ssovf_ops = { .dev_infos_get = ssovf_info_get, @@ -621,6 +629,8 @@ static struct rte_eventdev_ops ssovf_ops = { .eth_rx_adapter_start = ssovf_eth_rx_adapter_start, .eth_rx_adapter_stop = ssovf_eth_rx_adapter_stop, + .timer_adapter_caps_get = ssovf_timvf_caps_get, + .dev_selftest = test_eventdev_octeontx, .dump = ssovf_dump, diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c new file mode 100644 index 000000000..5728d4944 --- /dev/null +++ b/drivers/event/octeontx/timvf_evdev.c @@ -0,0 +1,146 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include "timvf_evdev.h" + +int otx_logtype_timvf; + +RTE_INIT(otx_timvf_init_log); +static void +otx_timvf_init_log(void) +{ + otx_logtype_timvf = rte_log_register("pmd.event.octeontx.timer"); + if (otx_logtype_timvf >= 0) + rte_log_set_level(otx_logtype_timvf, RTE_LOG_NOTICE); +} + +static void +timvf_ring_info_get(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer_adapter_info *adptr_info) +{ + struct timvf_ring *timr = adptr->data->adapter_priv; + adptr_info->max_tmo_ns = timr->max_tout; + adptr_info->min_resolution_ns = timr->tck_nsec; + rte_memcpy(&adptr_info->conf, &adptr->data->conf, + sizeof(struct rte_event_timer_adapter_conf)); +} + +static int +timvf_ring_create(struct rte_event_timer_adapter *adptr) +{ + char pool_name[25]; + int ret; + uint64_t nb_timers; + struct rte_event_timer_adapter_conf *rcfg = &adptr->data->conf; + struct timvf_ring *timr; + struct timvf_info tinfo; + const char *mempool_ops; + + if (timvf_info(&tinfo) < 0) + return -ENODEV; + + if (adptr->data->id >= tinfo.total_timvfs) + return -ENODEV; + + timr = rte_zmalloc("octeontx_timvf_priv", + sizeof(struct timvf_ring), 0); + if (timr == NULL) + return -ENOMEM; + + adptr->data->adapter_priv = timr; + /* Check config parameters. */ + if ((rcfg->clk_src != RTE_EVENT_TIMER_ADAPTER_CPU_CLK) && + (!rcfg->timer_tick_ns || + rcfg->timer_tick_ns < TIM_MIN_INTERVAL)) { + timvf_log_err("Too low timer ticks"); + goto cfg_err; + } + + timr->clk_src = (int) rcfg->clk_src; + timr->tim_ring_id = adptr->data->id; + timr->tck_nsec = rcfg->timer_tick_ns; + timr->max_tout = rcfg->max_tmo_ns; + timr->nb_bkts = (timr->max_tout / timr->tck_nsec); + timr->vbar0 = timvf_bar(timr->tim_ring_id, 0); + timr->bkt_pos = (uint8_t *)timr->vbar0 + TIM_VRING_REL; + nb_timers = rcfg->nb_timers; + timr->get_target_bkt = bkt_mod; + + timr->nb_chunks = nb_timers / nb_chunk_slots; + + timr->bkt = rte_zmalloc("octeontx_timvf_bucket", + (timr->nb_bkts) * sizeof(struct tim_mem_bucket), + 0); + if (timr->bkt == NULL) + goto mem_err; + + snprintf(pool_name, 30, "timvf_chunk_pool%d", timr->tim_ring_id); + timr->chunk_pool = (void *)rte_mempool_create_empty(pool_name, + timr->nb_chunks, TIM_CHUNK_SIZE, 0, 0, rte_socket_id(), + 0); + + if (!timr->chunk_pool) { + rte_free(timr->bkt); + timvf_log_err("Unable to create chunkpool."); + return -ENOMEM; + } + + mempool_ops = rte_mbuf_best_mempool_ops(); + ret = rte_mempool_set_ops_byname(timr->chunk_pool, + mempool_ops, NULL); + + if (ret != 0) { + timvf_log_err("Unable to set chunkpool ops."); + goto mem_err; + } + + ret = rte_mempool_populate_default(timr->chunk_pool); + if (ret < 0) { + timvf_log_err("Unable to set populate chunkpool."); + goto mem_err; + } + timvf_write64(0, (uint8_t *)timr->vbar0 + TIM_VRING_BASE); + timvf_write64(0, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_INT); + timvf_write64(0, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_INT_W1S); + timvf_write64(0x7, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_ENA_W1C); + timvf_write64(0x7, (uint8_t *)timr->vbar0 + TIM_VF_NRSPERR_ENA_W1S); + + return 0; +mem_err: + rte_free(timr); + return -ENOMEM; +cfg_err: + rte_free(timr); + return -EINVAL; +} + +static int +timvf_ring_free(struct rte_event_timer_adapter *adptr) +{ + struct timvf_ring *timr = adptr->data->adapter_priv; + rte_mempool_free(timr->chunk_pool); + rte_free(timr->bkt); + rte_free(adptr->data->adapter_priv); + return 0; +} + +static struct rte_event_timer_adapter_ops timvf_ops = { + .init = timvf_ring_create, + .uninit = timvf_ring_free, + .get_info = timvf_ring_info_get, +}; + +int +timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, + uint32_t *caps, const struct rte_event_timer_adapter_ops **ops, + uint8_t enable_stats) +{ + RTE_SET_USED(dev); + RTE_SET_USED(flags); + RTE_SET_USED(enable_stats); + *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; + *ops = &timvf_ops; + return -EINVAL; +} diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index 7e577edf4..ef8ec0ced 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -7,6 +7,22 @@ #define __TIMVF_EVDEV_H__ #include <rte_common.h> +#include <rte_cycles.h> +#include <rte_debug.h> +#include <rte_eal.h> +#include <rte_eventdev.h> +#include <rte_event_timer_adapter.h> +#include <rte_event_timer_adapter_pmd.h> +#include <rte_io.h> +#include <rte_lcore.h> +#include <rte_log.h> +#include <rte_malloc.h> +#include <rte_mbuf_pool_ops.h> +#include <rte_mempool.h> +#include <rte_memzone.h> +#include <rte_pci.h> +#include <rte_prefetch.h> +#include <rte_reciprocal.h> #include <octeontx_mbox.h> @@ -20,14 +36,129 @@ #define timvf_log_err(fmt, ...) timvf_log(ERR, fmt, ##__VA_ARGS__) #define timvf_func_trace timvf_log_dbg +#define TIM_COPROC (8) +#define TIM_GET_DEV_INFO (1) +#define TIM_GET_RING_INFO (2) +#define TIM_SET_RING_INFO (3) +#define TIM_RING_START_CYC_GET (4) + +#define TIM_MAX_RINGS (64) +#define TIM_DEV_PER_NODE (1) +#define TIM_VF_PER_DEV (64) +#define TIM_RING_PER_DEV (TIM_VF_PER_DEV) +#define TIM_RING_NODE_SHIFT (6) +#define TIM_RING_MASK ((TIM_RING_PER_DEV) - 1) +#define TIM_RING_INVALID (-1) + +#define TIM_MIN_INTERVAL (1E3) +#define TIM_MAX_INTERVAL ((1ull << 32) - 1) +#define TIM_MAX_BUCKETS (1ull << 20) +#define TIM_CHUNK_SIZE (4096) +#define TIM_MAX_CHUNKS_PER_BUCKET (1ull << 32) + +#define TIMVF_MAX_BURST (8) + +/* TIM VF Control/Status registers (CSRs): */ +/* VF_BAR0: */ +#define TIM_VF_NRSPERR_INT (0x0) +#define TIM_VF_NRSPERR_INT_W1S (0x8) +#define TIM_VF_NRSPERR_ENA_W1C (0x10) +#define TIM_VF_NRSPERR_ENA_W1S (0x18) +#define TIM_VRING_FR_RN_CYCLES (0x20) +#define TIM_VRING_FR_RN_GPIOS (0x28) +#define TIM_VRING_FR_RN_GTI (0x30) +#define TIM_VRING_FR_RN_PTP (0x38) +#define TIM_VRING_CTL0 (0x40) +#define TIM_VRING_CTL1 (0x50) +#define TIM_VRING_CTL2 (0x60) +#define TIM_VRING_BASE (0x100) +#define TIM_VRING_AURA (0x108) +#define TIM_VRING_REL (0x110) + +#define timvf_read64 rte_read64_relaxed +#define timvf_write64 rte_write64_relaxed + extern int otx_logtype_timvf; +static const uint16_t nb_chunk_slots = (TIM_CHUNK_SIZE / 16) - 1; struct timvf_info { uint16_t domain; /* Domain id */ uint8_t total_timvfs; /* Total timvf available in domain */ }; +enum timvf_clk_src { + TIM_CLK_SRC_SCLK = RTE_EVENT_TIMER_ADAPTER_CPU_CLK, + TIM_CLK_SRC_GPIO = RTE_EVENT_TIMER_ADAPTER_EXT_CLK0, + TIM_CLK_SRC_GTI = RTE_EVENT_TIMER_ADAPTER_EXT_CLK1, + TIM_CLK_SRC_PTP = RTE_EVENT_TIMER_ADAPTER_EXT_CLK2, +}; + +/* TIM_MEM_BUCKET */ +struct tim_mem_bucket { + uint64_t first_chunk; + union { + uint64_t w1; + struct { + uint32_t nb_entry; + uint8_t sbt:1; + uint8_t hbt:1; + uint8_t bsk:1; + uint8_t rsvd:5; + uint8_t lock; + int16_t chunk_remainder; + }; + }; + uint64_t current_chunk; + uint64_t pad; +} __rte_packed; + +struct tim_mem_entry { + uint64_t w0; + uint64_t wqe; +} __rte_packed; + +struct timvf_ctrl_reg { + uint64_t rctrl0; + uint64_t rctrl1; + uint64_t rctrl2; + uint8_t use_pmu; +} __rte_packed; + +struct timvf_ring; + +typedef uint32_t (*bkt_id)(const uint32_t bkt_tcks, const uint32_t nb_bkts); +typedef struct tim_mem_entry * (*refill_chunk)( + struct tim_mem_bucket * const bkt, + struct timvf_ring * const timr); + +struct timvf_ring { + bkt_id get_target_bkt; + refill_chunk refill_chunk; + struct rte_reciprocal_u64 fast_div; + uint64_t ring_start_cyc; + uint32_t nb_bkts; + struct tim_mem_bucket *bkt; + void *chunk_pool; + uint64_t tck_int; + uint64_t tck_nsec; + void *vbar0; + void *bkt_pos; + uint64_t max_tout; + uint64_t nb_chunks; + enum timvf_clk_src clk_src; + uint16_t tim_ring_id; +} __rte_cache_aligned; + +static __rte_always_inline uint32_t +bkt_mod(const uint32_t rel_bkt, const uint32_t nb_bkts) +{ + return rel_bkt % nb_bkts; +} + int timvf_info(struct timvf_info *tinfo); void *timvf_bar(uint8_t id, uint8_t bar); +int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, + uint32_t *caps, const struct rte_event_timer_adapter_ops **ops, + uint8_t enable_stats); #endif /* __TIMVF_EVDEV_H__ */ -- 2.17.0 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v4 04/11] event/octeontx: add support to start and stop timer device 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 00/11] " Pavan Nikhilesh ` (2 preceding siblings ...) 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 03/11] event/octeontx: add support to create and free timer adapter Pavan Nikhilesh @ 2018-04-09 21:00 ` Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 05/11] event/octeontx: add event timer stats get and reset Pavan Nikhilesh ` (7 subsequent siblings) 11 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-09 21:00 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh When application requests to start the timer adapter through `rte_event_timer_adapter_start`, Octeontx TIMvf ring does the following: - Uses mbox to communicate TIMpf driver about, * SCLK frequency used to convert ns<->cycles. * program the ring control parameters and start the ring. * get the exact cycle at which the TIMvf ring has started which can be used to estimate the bucket position. On `rte_event_timer_adapter_stop` i.e stop, Octeontx TIMvf ring does the following: - Use mbox to communicate TIMpf driver about, * reset the ring control parameters and stop the ring. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 144 +++++++++++++++++++++++++++ drivers/event/octeontx/timvf_evdev.h | 5 + 2 files changed, 149 insertions(+) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index 5728d4944..84c6a511e 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -16,6 +16,32 @@ otx_timvf_init_log(void) rte_log_set_level(otx_logtype_timvf, RTE_LOG_NOTICE); } +struct __rte_packed timvf_mbox_dev_info { + uint64_t ring_active[4]; + uint64_t clk_freq; +}; + +/* Response messages */ +enum { + MBOX_RET_SUCCESS, + MBOX_RET_INVALID, + MBOX_RET_INTERNAL_ERR, +}; + +static int +timvf_mbox_dev_info_get(struct timvf_mbox_dev_info *info) +{ + struct octeontx_mbox_hdr hdr = {0}; + uint16_t len = sizeof(struct timvf_mbox_dev_info); + + hdr.coproc = TIM_COPROC; + hdr.msg = TIM_GET_DEV_INFO; + hdr.vfid = 0; /* TIM DEV is always 0. TIM RING ID changes. */ + + memset(info, 0, len); + return octeontx_mbox_send(&hdr, NULL, 0, info, len); +} + static void timvf_ring_info_get(const struct rte_event_timer_adapter *adptr, struct rte_event_timer_adapter_info *adptr_info) @@ -27,6 +53,122 @@ timvf_ring_info_get(const struct rte_event_timer_adapter *adptr, sizeof(struct rte_event_timer_adapter_conf)); } +static int +timvf_ring_conf_set(struct timvf_ctrl_reg *rctl, uint8_t ring_id) +{ + struct octeontx_mbox_hdr hdr = {0}; + uint16_t len = sizeof(struct timvf_ctrl_reg); + int ret; + + hdr.coproc = TIM_COPROC; + hdr.msg = TIM_SET_RING_INFO; + hdr.vfid = ring_id; + + ret = octeontx_mbox_send(&hdr, rctl, len, NULL, 0); + if (ret < 0 || hdr.res_code != MBOX_RET_SUCCESS) + return -EACCES; + return 0; +} + +static int +timvf_get_start_cyc(uint64_t *now, uint8_t ring_id) +{ + struct octeontx_mbox_hdr hdr = {0}; + + hdr.coproc = TIM_COPROC; + hdr.msg = TIM_RING_START_CYC_GET; + hdr.vfid = ring_id; + *now = 0; + return octeontx_mbox_send(&hdr, NULL, 0, now, sizeof(uint64_t)); +} + +static int +timvf_ring_start(const struct rte_event_timer_adapter *adptr) +{ + int ret; + uint64_t interval; + struct timvf_ctrl_reg rctrl; + struct timvf_mbox_dev_info dinfo; + struct timvf_ring *timr = adptr->data->adapter_priv; + + ret = timvf_mbox_dev_info_get(&dinfo); + if (ret < 0 || ret != sizeof(struct timvf_mbox_dev_info)) + return -EINVAL; + + /* Calculate the interval cycles according to clock source. */ + switch (timr->clk_src) { + case TIM_CLK_SRC_SCLK: + interval = NSEC2CLK(timr->tck_nsec, dinfo.clk_freq); + break; + case TIM_CLK_SRC_GPIO: + /* GPIO doesn't work on tck_nsec. */ + interval = 0; + break; + case TIM_CLK_SRC_GTI: + interval = NSEC2CLK(timr->tck_nsec, dinfo.clk_freq); + break; + case TIM_CLK_SRC_PTP: + interval = NSEC2CLK(timr->tck_nsec, dinfo.clk_freq); + break; + default: + timvf_log_err("Unsupported clock source configured %d", + timr->clk_src); + return -EINVAL; + } + + /*CTRL0 register.*/ + rctrl.rctrl0 = interval; + + /*CTRL1 register.*/ + rctrl.rctrl1 = (uint64_t)(timr->clk_src) << 51 | + 1ull << 48 /* LOCK_EN (Enable hw bucket lock mechanism) */ | + 1ull << 47 /* ENA */ | + 1ull << 44 /* ENA_LDWB */ | + (timr->nb_bkts - 1); + + rctrl.rctrl2 = (uint64_t)(TIM_CHUNK_SIZE / 16) << 40; + + timvf_write64((uintptr_t)timr->bkt, + (uint8_t *)timr->vbar0 + TIM_VRING_BASE); + if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) { + ret = -EACCES; + goto error; + } + + if (timvf_get_start_cyc(&timr->ring_start_cyc, + timr->tim_ring_id) < 0) { + ret = -EACCES; + goto error; + } + timr->tck_int = NSEC2CLK(timr->tck_nsec, rte_get_timer_hz()); + timr->fast_div = rte_reciprocal_value_u64(timr->tck_int); + timvf_log_info("nb_bkts %d min_ns %"PRIu64" min_cyc %"PRIu64"" + " maxtmo %"PRIu64"\n", + timr->nb_bkts, timr->tck_nsec, interval, + timr->max_tout); + + return 0; +error: + rte_free(timr->bkt); + rte_mempool_free(timr->chunk_pool); + return ret; +} + +static int +timvf_ring_stop(const struct rte_event_timer_adapter *adptr) +{ + struct timvf_ring *timr = adptr->data->adapter_priv; + struct timvf_ctrl_reg rctrl = {0}; + rctrl.rctrl0 = timvf_read64((uint8_t *)timr->vbar0 + TIM_VRING_CTL0); + rctrl.rctrl1 = timvf_read64((uint8_t *)timr->vbar0 + TIM_VRING_CTL1); + rctrl.rctrl1 &= ~(1ull << 47); /* Disable */ + rctrl.rctrl2 = timvf_read64((uint8_t *)timr->vbar0 + TIM_VRING_CTL2); + + if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) + return -EACCES; + return 0; +} + static int timvf_ring_create(struct rte_event_timer_adapter *adptr) { @@ -129,6 +271,8 @@ timvf_ring_free(struct rte_event_timer_adapter *adptr) static struct rte_event_timer_adapter_ops timvf_ops = { .init = timvf_ring_create, .uninit = timvf_ring_free, + .start = timvf_ring_start, + .stop = timvf_ring_stop, .get_info = timvf_ring_info_get, }; diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index ef8ec0ced..7663dc375 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -75,6 +75,11 @@ #define TIM_VRING_AURA (0x108) #define TIM_VRING_REL (0x110) + +#define NSEC_PER_SEC 1E9 +#define NSEC2CLK(__ns, __freq) (((__ns) * (__freq)) / NSEC_PER_SEC) +#define CLK2NSEC(__clk, __freq) (((__clk) * NSEC_PER_SEC) / (__freq)) + #define timvf_read64 rte_read64_relaxed #define timvf_write64 rte_write64_relaxed -- 2.17.0 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v4 05/11] event/octeontx: add event timer stats get and reset 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 00/11] " Pavan Nikhilesh ` (3 preceding siblings ...) 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 04/11] event/octeontx: add support to start and stop timer device Pavan Nikhilesh @ 2018-04-09 21:00 ` Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 06/11] event/octeontx: add multiproducer timer arm and cancel Pavan Nikhilesh ` (6 subsequent siblings) 11 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-09 21:00 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Add functions to get and reset event timer adapter stats. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/ssovf_evdev.c | 14 ++++++++++++- drivers/event/octeontx/timvf_evdev.c | 30 +++++++++++++++++++++++++++- drivers/event/octeontx/timvf_evdev.h | 3 +++ 3 files changed, 45 insertions(+), 2 deletions(-) diff --git a/drivers/event/octeontx/ssovf_evdev.c b/drivers/event/octeontx/ssovf_evdev.c index 34496f114..2df70b52a 100644 --- a/drivers/event/octeontx/ssovf_evdev.c +++ b/drivers/event/octeontx/ssovf_evdev.c @@ -21,6 +21,7 @@ #include "timvf_evdev.h" int otx_logtype_ssovf; +static uint8_t timvf_enable_stats; RTE_INIT(otx_ssovf_init_log); static void @@ -606,7 +607,8 @@ static int ssovf_timvf_caps_get(const struct rte_eventdev *dev, uint64_t flags, uint32_t *caps, const struct rte_event_timer_adapter_ops **ops) { - return timvf_timer_adapter_caps_get(dev, flags, caps, ops, 0); + return timvf_timer_adapter_caps_get(dev, flags, caps, ops, + timvf_enable_stats); } /* Initialize and register event driver with DPDK Application */ @@ -654,6 +656,7 @@ ssovf_vdev_probe(struct rte_vdev_device *vdev) static const char *const args[] = { SSOVF_SELFTEST_ARG, + TIMVF_ENABLE_STATS_ARG, NULL }; @@ -681,6 +684,15 @@ ssovf_vdev_probe(struct rte_vdev_device *vdev) rte_kvargs_free(kvlist); return ret; } + + ret = rte_kvargs_process(kvlist, + TIMVF_ENABLE_STATS_ARG, + ssovf_selftest, &timvf_enable_stats); + if (ret != 0) { + ssovf_log_err("%s: Error in timvf stats", name); + rte_kvargs_free(kvlist); + return ret; + } } rte_kvargs_free(kvlist); diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index 84c6a511e..c66db437e 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -268,6 +268,29 @@ timvf_ring_free(struct rte_event_timer_adapter *adptr) return 0; } +static int +timvf_stats_get(const struct rte_event_timer_adapter *adapter, + struct rte_event_timer_adapter_stats *stats) +{ + struct timvf_ring *timr = adapter->data->adapter_priv; + uint64_t bkt_cyc = rte_rdtsc() - timr->ring_start_cyc; + + stats->evtim_exp_count = timr->tim_arm_cnt; + stats->ev_enq_count = timr->tim_arm_cnt; + stats->adapter_tick_count = rte_reciprocal_divide_u64(bkt_cyc, + &timr->fast_div); + return 0; +} + +static int +timvf_stats_reset(const struct rte_event_timer_adapter *adapter) +{ + struct timvf_ring *timr = adapter->data->adapter_priv; + + timr->tim_arm_cnt = 0; + return 0; +} + static struct rte_event_timer_adapter_ops timvf_ops = { .init = timvf_ring_create, .uninit = timvf_ring_free, @@ -283,7 +306,12 @@ timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, { RTE_SET_USED(dev); RTE_SET_USED(flags); - RTE_SET_USED(enable_stats); + + if (enable_stats) { + timvf_ops.stats_get = timvf_stats_get; + timvf_ops.stats_reset = timvf_stats_reset; + } + *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; *ops = &timvf_ops; return -EINVAL; diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index 7663dc375..e89a43531 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -83,6 +83,8 @@ #define timvf_read64 rte_read64_relaxed #define timvf_write64 rte_write64_relaxed +#define TIMVF_ENABLE_STATS_ARG ("timvf_stats") + extern int otx_logtype_timvf; static const uint16_t nb_chunk_slots = (TIM_CHUNK_SIZE / 16) - 1; @@ -145,6 +147,7 @@ struct timvf_ring { struct tim_mem_bucket *bkt; void *chunk_pool; uint64_t tck_int; + volatile uint64_t tim_arm_cnt; uint64_t tck_nsec; void *vbar0; void *bkt_pos; -- 2.17.0 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v4 06/11] event/octeontx: add multiproducer timer arm and cancel 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 00/11] " Pavan Nikhilesh ` (4 preceding siblings ...) 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 05/11] event/octeontx: add event timer stats get and reset Pavan Nikhilesh @ 2018-04-09 21:00 ` Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 07/11] event/octeontx: add single producer timer arm variant Pavan Nikhilesh ` (5 subsequent siblings) 11 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-09 21:00 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com> --- drivers/event/octeontx/Makefile | 5 + drivers/event/octeontx/meson.build | 1 + drivers/event/octeontx/timvf_evdev.c | 9 +- drivers/event/octeontx/timvf_evdev.h | 35 ++++ drivers/event/octeontx/timvf_worker.c | 107 ++++++++++ drivers/event/octeontx/timvf_worker.h | 279 ++++++++++++++++++++++++++ 6 files changed, 435 insertions(+), 1 deletion(-) create mode 100644 drivers/event/octeontx/timvf_worker.c create mode 100644 drivers/event/octeontx/timvf_worker.h diff --git a/drivers/event/octeontx/Makefile b/drivers/event/octeontx/Makefile index 8dbfd88c5..90ad2217f 100644 --- a/drivers/event/octeontx/Makefile +++ b/drivers/event/octeontx/Makefile @@ -30,20 +30,25 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_worker.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_evdev.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_evdev_selftest.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += ssovf_probe.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += timvf_worker.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += timvf_evdev.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF) += timvf_probe.c ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_ssovf_worker.o += -fno-prefetch-loop-arrays +CFLAGS_timvf_worker.o += -fno-prefetch-loop-arrays ifeq ($(shell test $(GCC_VERSION) -ge 46 && echo 1), 1) CFLAGS_ssovf_worker.o += -Ofast +CFLAGS_timvf_worker.o += -Ofast else CFLAGS_ssovf_worker.o += -O3 -ffast-math +CFLAGS_timvf_worker.o += -O3 -ffast-math endif else CFLAGS_ssovf_worker.o += -Ofast +CFLAGS_timvf_worker.o += -Ofast endif include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/event/octeontx/meson.build b/drivers/event/octeontx/meson.build index 90af4fd1a..041855330 100644 --- a/drivers/event/octeontx/meson.build +++ b/drivers/event/octeontx/meson.build @@ -5,6 +5,7 @@ sources = files('ssovf_worker.c', 'ssovf_evdev.c', 'ssovf_evdev_selftest.c', 'ssovf_probe.c', + 'timvf_worker.c', 'timvf_evdev.c', 'timvf_probe.c' ) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index c66db437e..5ffb460a8 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -130,6 +130,7 @@ timvf_ring_start(const struct rte_event_timer_adapter *adptr) timvf_write64((uintptr_t)timr->bkt, (uint8_t *)timr->vbar0 + TIM_VRING_BASE); + timvf_set_chunk_refill(timr); if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) { ret = -EACCES; goto error; @@ -312,7 +313,13 @@ timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, timvf_ops.stats_reset = timvf_stats_reset; } + if (enable_stats) + timvf_ops.arm_burst = timvf_timer_arm_burst_mp_stats; + else + timvf_ops.arm_burst = timvf_timer_arm_burst_mp; + + timvf_ops.cancel_burst = timvf_timer_cancel_burst; *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; *ops = &timvf_ops; - return -EINVAL; + return 0; } diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index e89a43531..cae949c73 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -75,6 +75,33 @@ #define TIM_VRING_AURA (0x108) #define TIM_VRING_REL (0x110) +#define TIM_CTL1_W0_S_BUCKET 20 +#define TIM_CTL1_W0_M_BUCKET ((1ull << (40 - 20)) - 1) + +#define TIM_BUCKET_W1_S_NUM_ENTRIES (0) /*Shift*/ +#define TIM_BUCKET_W1_M_NUM_ENTRIES ((1ull << (32 - 0)) - 1) +#define TIM_BUCKET_W1_S_SBT (32) +#define TIM_BUCKET_W1_M_SBT ((1ull << (33 - 32)) - 1) +#define TIM_BUCKET_W1_S_HBT (33) +#define TIM_BUCKET_W1_M_HBT ((1ull << (34 - 33)) - 1) +#define TIM_BUCKET_W1_S_BSK (34) +#define TIM_BUCKET_W1_M_BSK ((1ull << (35 - 34)) - 1) +#define TIM_BUCKET_W1_S_LOCK (40) +#define TIM_BUCKET_W1_M_LOCK ((1ull << (48 - 40)) - 1) +#define TIM_BUCKET_W1_S_CHUNK_REMAINDER (48) +#define TIM_BUCKET_W1_M_CHUNK_REMAINDER ((1ull << (64 - 48)) - 1) + +#define TIM_BUCKET_SEMA \ + (TIM_BUCKET_CHUNK_REMAIN) + +#define TIM_BUCKET_CHUNK_REMAIN \ + (TIM_BUCKET_W1_M_CHUNK_REMAINDER << TIM_BUCKET_W1_S_CHUNK_REMAINDER) + +#define TIM_BUCKET_LOCK \ + (TIM_BUCKET_W1_M_LOCK << TIM_BUCKET_W1_S_LOCK) + +#define TIM_BUCKET_SEMA_WLOCK \ + (TIM_BUCKET_CHUNK_REMAIN | (1ull << TIM_BUCKET_W1_S_LOCK)) #define NSEC_PER_SEC 1E9 #define NSEC2CLK(__ns, __freq) (((__ns) * (__freq)) / NSEC_PER_SEC) @@ -168,5 +195,13 @@ void *timvf_bar(uint8_t id, uint8_t bar); int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, uint32_t *caps, const struct rte_event_timer_adapter_ops **ops, uint8_t enable_stats); +uint16_t timvf_timer_cancel_burst(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers); +uint16_t timvf_timer_arm_burst_mp(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers); +uint16_t timvf_timer_arm_burst_mp_stats( + const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers); +void timvf_set_chunk_refill(struct timvf_ring * const timr); #endif /* __TIMVF_EVDEV_H__ */ diff --git a/drivers/event/octeontx/timvf_worker.c b/drivers/event/octeontx/timvf_worker.c new file mode 100644 index 000000000..38b16d1ce --- /dev/null +++ b/drivers/event/octeontx/timvf_worker.c @@ -0,0 +1,107 @@ +/* + * SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include "timvf_worker.h" + +static inline int +timvf_timer_reg_checks(const struct timvf_ring * const timr, + struct rte_event_timer * const tim) +{ + if (unlikely(tim->state)) { + tim->state = RTE_EVENT_TIMER_ERROR; + rte_errno = EALREADY; + goto fail; + } + + if (unlikely(!tim->timeout_ticks || + tim->timeout_ticks >= timr->nb_bkts)) { + tim->state = tim->timeout_ticks ? RTE_EVENT_TIMER_ERROR_TOOLATE + : RTE_EVENT_TIMER_ERROR_TOOEARLY; + rte_errno = EINVAL; + goto fail; + } + + return 0; +fail: + return -EINVAL; +} + +static inline void +timvf_format_event(const struct rte_event_timer * const tim, + struct tim_mem_entry * const entry) +{ + entry->w0 = (tim->ev.event & 0xFFC000000000) >> 6 | + (tim->ev.event & 0xFFFFFFFFF); + entry->wqe = tim->ev.u64; +} + +uint16_t +timvf_timer_cancel_burst(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers) +{ + RTE_SET_USED(adptr); + int ret; + uint16_t index; + + for (index = 0; index < nb_timers; index++) { + if (tim[index]->state == RTE_EVENT_TIMER_CANCELED) { + rte_errno = EALREADY; + break; + } + + if (tim[index]->state != RTE_EVENT_TIMER_ARMED) { + rte_errno = EINVAL; + break; + } + ret = timvf_rem_entry(tim[index]); + if (ret) { + rte_errno = -ret; + break; + } + } + return index; +} + +uint16_t +timvf_timer_arm_burst_mp(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers) +{ + int ret; + uint16_t index; + struct tim_mem_entry entry; + struct timvf_ring *timr = adptr->data->adapter_priv; + for (index = 0; index < nb_timers; index++) { + if (timvf_timer_reg_checks(timr, tim[index])) + break; + timvf_format_event(tim[index], &entry); + ret = timvf_add_entry_mp(timr, tim[index]->timeout_ticks, + tim[index], &entry); + if (unlikely(ret)) { + rte_errno = -ret; + break; + } + } + + return index; +} + +uint16_t +timvf_timer_arm_burst_mp_stats(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers) +{ + uint16_t ret; + struct timvf_ring *timr = adptr->data->adapter_priv; + + ret = timvf_timer_arm_burst_mp(adptr, tim, nb_timers); + timr->tim_arm_cnt += ret; + + return ret; +} + +void +timvf_set_chunk_refill(struct timvf_ring * const timr) +{ + timr->refill_chunk = timvf_refill_chunk_generic; +} diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h new file mode 100644 index 000000000..c20d4d0cd --- /dev/null +++ b/drivers/event/octeontx/timvf_worker.h @@ -0,0 +1,279 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2017 Cavium, Inc + */ + +#include <rte_common.h> +#include <rte_branch_prediction.h> + +#include "timvf_evdev.h" + +static inline int16_t +timr_bkt_fetch_rem(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_CHUNK_REMAINDER) & + TIM_BUCKET_W1_M_CHUNK_REMAINDER; +} + +static inline int16_t +timr_bkt_get_rem(struct tim_mem_bucket *bktp) +{ + return __atomic_load_n(&bktp->chunk_remainder, + __ATOMIC_ACQUIRE); +} + +static inline void +timr_bkt_set_rem(struct tim_mem_bucket *bktp, uint16_t v) +{ + __atomic_store_n(&bktp->chunk_remainder, v, + __ATOMIC_RELEASE); +} + +static inline void +timr_bkt_sub_rem(struct tim_mem_bucket *bktp, uint16_t v) +{ + __atomic_fetch_sub(&bktp->chunk_remainder, v, + __ATOMIC_RELEASE); +} + +static inline uint8_t +timr_bkt_get_sbt(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_SBT) & TIM_BUCKET_W1_M_SBT; +} + +static inline uint64_t +timr_bkt_set_sbt(struct tim_mem_bucket *bktp) +{ + const uint64_t v = TIM_BUCKET_W1_M_SBT << TIM_BUCKET_W1_S_SBT; + return __atomic_fetch_or(&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline uint64_t +timr_bkt_clr_sbt(struct tim_mem_bucket *bktp) +{ + const uint64_t v = ~(TIM_BUCKET_W1_M_SBT << TIM_BUCKET_W1_S_SBT); + return __atomic_fetch_and(&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline uint8_t +timr_bkt_get_shbt(uint64_t w1) +{ + return ((w1 >> TIM_BUCKET_W1_S_HBT) & TIM_BUCKET_W1_M_HBT) | + ((w1 >> TIM_BUCKET_W1_S_SBT) & TIM_BUCKET_W1_M_SBT); +} + +static inline uint8_t +timr_bkt_get_hbt(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_HBT) & TIM_BUCKET_W1_M_HBT; +} + +static inline uint8_t +timr_bkt_get_bsk(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_BSK) & TIM_BUCKET_W1_M_BSK; +} + +static inline uint64_t +timr_bkt_clr_bsk(struct tim_mem_bucket *bktp) +{ + /*Clear everything except lock. */ + const uint64_t v = TIM_BUCKET_W1_M_LOCK << TIM_BUCKET_W1_S_LOCK; + return __atomic_fetch_and(&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline uint64_t +timr_bkt_fetch_sema_lock(struct tim_mem_bucket *bktp) +{ + return __atomic_fetch_add(&bktp->w1, TIM_BUCKET_SEMA_WLOCK, + __ATOMIC_ACQ_REL); +} + +static inline uint64_t +timr_bkt_fetch_sema(struct tim_mem_bucket *bktp) +{ + return __atomic_fetch_add(&bktp->w1, TIM_BUCKET_SEMA, + __ATOMIC_RELAXED); +} + +static inline uint64_t +timr_bkt_inc_lock(struct tim_mem_bucket *bktp) +{ + const uint64_t v = 1ull << TIM_BUCKET_W1_S_LOCK; + return __atomic_fetch_add(&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline void +timr_bkt_dec_lock(struct tim_mem_bucket *bktp) +{ + __atomic_add_fetch(&bktp->lock, 0xff, __ATOMIC_ACQ_REL); +} + +static inline uint32_t +timr_bkt_get_nent(uint64_t w1) +{ + return (w1 >> TIM_BUCKET_W1_S_NUM_ENTRIES) & + TIM_BUCKET_W1_M_NUM_ENTRIES; +} + +static inline void +timr_bkt_inc_nent(struct tim_mem_bucket *bktp) +{ + __atomic_add_fetch(&bktp->nb_entry, 1, __ATOMIC_RELAXED); +} + +static inline void +timr_bkt_add_nent(struct tim_mem_bucket *bktp, uint32_t v) +{ + __atomic_add_fetch(&bktp->nb_entry, v, __ATOMIC_RELAXED); +} + +static inline uint64_t +timr_bkt_clr_nent(struct tim_mem_bucket *bktp) +{ + const uint64_t v = ~(TIM_BUCKET_W1_M_NUM_ENTRIES << + TIM_BUCKET_W1_S_NUM_ENTRIES); + return __atomic_and_fetch(&bktp->w1, v, __ATOMIC_ACQ_REL); +} + +static inline struct tim_mem_entry * +timr_clr_bkt(struct timvf_ring * const timr, struct tim_mem_bucket * const bkt) +{ + struct tim_mem_entry *chunk; + struct tim_mem_entry *pnext; + chunk = ((struct tim_mem_entry *)(uintptr_t)bkt->first_chunk); + chunk = (struct tim_mem_entry *)(uintptr_t)(chunk + nb_chunk_slots)->w0; + + while (chunk) { + pnext = (struct tim_mem_entry *)(uintptr_t) + ((chunk + nb_chunk_slots)->w0); + rte_mempool_put(timr->chunk_pool, chunk); + chunk = pnext; + } + return (struct tim_mem_entry *)(uintptr_t)bkt->first_chunk; +} + +static inline int +timvf_rem_entry(struct rte_event_timer *tim) +{ + uint64_t lock_sema; + struct tim_mem_entry *entry; + struct tim_mem_bucket *bkt; + if (tim->impl_opaque[1] == 0 || + tim->impl_opaque[0] == 0) + return -ENOENT; + + entry = (struct tim_mem_entry *)(uintptr_t)tim->impl_opaque[0]; + if (entry->wqe != tim->ev.u64) { + tim->impl_opaque[1] = tim->impl_opaque[0] = 0; + return -ENOENT; + } + bkt = (struct tim_mem_bucket *)(uintptr_t)tim->impl_opaque[1]; + lock_sema = timr_bkt_inc_lock(bkt); + if (timr_bkt_get_shbt(lock_sema) + || !timr_bkt_get_nent(lock_sema)) { + timr_bkt_dec_lock(bkt); + tim->impl_opaque[1] = tim->impl_opaque[0] = 0; + return -ENOENT; + } + + entry->w0 = entry->wqe = 0; + timr_bkt_dec_lock(bkt); + + tim->state = RTE_EVENT_TIMER_CANCELED; + tim->impl_opaque[1] = tim->impl_opaque[0] = 0; + return 0; +} + +static inline struct tim_mem_entry * +timvf_refill_chunk_generic(struct tim_mem_bucket * const bkt, + struct timvf_ring * const timr) +{ + struct tim_mem_entry *chunk; + + if (bkt->nb_entry || !bkt->first_chunk) { + if (unlikely(rte_mempool_get(timr->chunk_pool, + (void **)&chunk))) { + return NULL; + } + if (bkt->nb_entry) { + *(uint64_t *)(((struct tim_mem_entry *)(uintptr_t) + bkt->current_chunk) + + nb_chunk_slots) = + (uintptr_t) chunk; + } else { + bkt->first_chunk = (uintptr_t) chunk; + } + } else { + chunk = timr_clr_bkt(timr, bkt); + bkt->first_chunk = (uintptr_t)chunk; + } + *(uint64_t *)(chunk + nb_chunk_slots) = 0; + + return chunk; +} + +static inline struct tim_mem_bucket * +timvf_get_target_bucket(struct timvf_ring * const timr, const uint32_t rel_bkt) +{ + const uint64_t bkt_cyc = rte_rdtsc() - timr->ring_start_cyc; + const uint32_t bucket = rte_reciprocal_divide_u64(bkt_cyc, + &timr->fast_div) + rel_bkt; + const uint32_t tbkt_id = timr->get_target_bkt(bucket, + timr->nb_bkts); + return &timr->bkt[tbkt_id]; +} + +/* Multi producer functions. */ +static inline int +timvf_add_entry_mp(struct timvf_ring * const timr, const uint32_t rel_bkt, + struct rte_event_timer * const tim, + const struct tim_mem_entry * const pent) +{ + int16_t rem; + uint64_t lock_sema; + struct tim_mem_bucket *bkt; + struct tim_mem_entry *chunk; + +__retry: + bkt = timvf_get_target_bucket(timr, rel_bkt); + /* Bucket related checks. */ + /*Get Bucket sema*/ + lock_sema = timr_bkt_fetch_sema_lock(bkt); + if (unlikely(timr_bkt_get_shbt(lock_sema))) { + timr_bkt_dec_lock(bkt); + goto __retry; + } + + rem = timr_bkt_fetch_rem(lock_sema); + + if (rem < 0) { + /* goto diff bucket. */ + timr_bkt_dec_lock(bkt); + goto __retry; + } else if (!rem) { + /*Only one thread can be here*/ + chunk = timr->refill_chunk(bkt, timr); + if (unlikely(chunk == NULL)) { + timr_bkt_set_rem(bkt, 0); + timr_bkt_dec_lock(bkt); + tim->impl_opaque[0] = tim->impl_opaque[1] = 0; + tim->state = RTE_EVENT_TIMER_ERROR; + return -ENOMEM; + } + bkt->current_chunk = (uintptr_t) chunk; + timr_bkt_set_rem(bkt, nb_chunk_slots - 1); + } else { + chunk = (struct tim_mem_entry *)(uintptr_t)bkt->current_chunk; + chunk += nb_chunk_slots - rem; + } + /* Copy work entry. */ + *chunk = *pent; + timr_bkt_inc_nent(bkt); + timr_bkt_dec_lock(bkt); + + tim->impl_opaque[0] = (uintptr_t)chunk; + tim->impl_opaque[1] = (uintptr_t)bkt; + tim->state = RTE_EVENT_TIMER_ARMED; + return 0; +} -- 2.17.0 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v4 07/11] event/octeontx: add single producer timer arm variant 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 00/11] " Pavan Nikhilesh ` (5 preceding siblings ...) 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 06/11] event/octeontx: add multiproducer timer arm and cancel Pavan Nikhilesh @ 2018-04-09 21:00 ` Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 08/11] event/octeontx: add burst mode for timer arm Pavan Nikhilesh ` (4 subsequent siblings) 11 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-09 21:00 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh When application creates the timer adapter by passing `RTE_EVENT_TIMER_ADAPTER_F_SP_PUT` flag, we can optimize the arm sequence by removing the locking overhead. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 19 ++++++++--- drivers/event/octeontx/timvf_evdev.h | 5 +++ drivers/event/octeontx/timvf_worker.c | 37 +++++++++++++++++++++ drivers/event/octeontx/timvf_worker.h | 47 +++++++++++++++++++++++++++ 4 files changed, 103 insertions(+), 5 deletions(-) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index 5ffb460a8..3b698b7c8 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -180,6 +180,7 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) struct timvf_ring *timr; struct timvf_info tinfo; const char *mempool_ops; + unsigned int mp_flags = 0; if (timvf_info(&tinfo) < 0) return -ENODEV; @@ -213,6 +214,11 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) timr->nb_chunks = nb_timers / nb_chunk_slots; + if (rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_SP_PUT) { + mp_flags = MEMPOOL_F_SP_PUT | MEMPOOL_F_SC_GET; + timvf_log_info("Using single producer mode"); + } + timr->bkt = rte_zmalloc("octeontx_timvf_bucket", (timr->nb_bkts) * sizeof(struct tim_mem_bucket), 0); @@ -222,7 +228,7 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) snprintf(pool_name, 30, "timvf_chunk_pool%d", timr->tim_ring_id); timr->chunk_pool = (void *)rte_mempool_create_empty(pool_name, timr->nb_chunks, TIM_CHUNK_SIZE, 0, 0, rte_socket_id(), - 0); + mp_flags); if (!timr->chunk_pool) { rte_free(timr->bkt); @@ -306,17 +312,20 @@ timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, uint8_t enable_stats) { RTE_SET_USED(dev); - RTE_SET_USED(flags); if (enable_stats) { timvf_ops.stats_get = timvf_stats_get; timvf_ops.stats_reset = timvf_stats_reset; } - if (enable_stats) - timvf_ops.arm_burst = timvf_timer_arm_burst_mp_stats; + if (flags & RTE_EVENT_TIMER_ADAPTER_F_SP_PUT) + timvf_ops.arm_burst = enable_stats ? + timvf_timer_arm_burst_sp_stats : + timvf_timer_arm_burst_sp; else - timvf_ops.arm_burst = timvf_timer_arm_burst_mp; + timvf_ops.arm_burst = enable_stats ? + timvf_timer_arm_burst_mp_stats : + timvf_timer_arm_burst_mp; timvf_ops.cancel_burst = timvf_timer_cancel_burst; *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index cae949c73..01ffac09a 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -197,6 +197,11 @@ int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, uint8_t enable_stats); uint16_t timvf_timer_cancel_burst(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers); +uint16_t timvf_timer_arm_burst_sp(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers); +uint16_t timvf_timer_arm_burst_sp_stats( + const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers); uint16_t timvf_timer_arm_burst_mp(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers); uint16_t timvf_timer_arm_burst_mp_stats( diff --git a/drivers/event/octeontx/timvf_worker.c b/drivers/event/octeontx/timvf_worker.c index 38b16d1ce..52b212c44 100644 --- a/drivers/event/octeontx/timvf_worker.c +++ b/drivers/event/octeontx/timvf_worker.c @@ -64,6 +64,43 @@ timvf_timer_cancel_burst(const struct rte_event_timer_adapter *adptr, return index; } +uint16_t +timvf_timer_arm_burst_sp(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers) +{ + int ret; + uint16_t index; + struct tim_mem_entry entry; + struct timvf_ring *timr = adptr->data->adapter_priv; + for (index = 0; index < nb_timers; index++) { + if (timvf_timer_reg_checks(timr, tim[index])) + break; + + timvf_format_event(tim[index], &entry); + ret = timvf_add_entry_sp(timr, tim[index]->timeout_ticks, + tim[index], &entry); + if (unlikely(ret)) { + rte_errno = -ret; + break; + } + } + + return index; +} + +uint16_t +timvf_timer_arm_burst_sp_stats(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint16_t nb_timers) +{ + uint16_t ret; + struct timvf_ring *timr = adptr->data->adapter_priv; + + ret = timvf_timer_arm_burst_sp(adptr, tim, nb_timers); + timr->tim_arm_cnt += ret; + + return ret; +} + uint16_t timvf_timer_arm_burst_mp(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers) diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h index c20d4d0cd..c3a669de9 100644 --- a/drivers/event/octeontx/timvf_worker.h +++ b/drivers/event/octeontx/timvf_worker.h @@ -224,6 +224,53 @@ timvf_get_target_bucket(struct timvf_ring * const timr, const uint32_t rel_bkt) return &timr->bkt[tbkt_id]; } +/* Single producer functions. */ +static inline int +timvf_add_entry_sp(struct timvf_ring * const timr, const uint32_t rel_bkt, + struct rte_event_timer * const tim, + const struct tim_mem_entry * const pent) +{ + int16_t rem; + uint64_t lock_sema; + struct tim_mem_bucket *bkt; + struct tim_mem_entry *chunk; + + + bkt = timvf_get_target_bucket(timr, rel_bkt); +__retry: + /*Get Bucket sema*/ + lock_sema = timr_bkt_fetch_sema(bkt); + /* Bucket related checks. */ + if (unlikely(timr_bkt_get_hbt(lock_sema))) + goto __retry; + + /* Insert the work. */ + rem = timr_bkt_fetch_rem(lock_sema); + + if (!rem) { + chunk = timr->refill_chunk(bkt, timr); + if (unlikely(chunk == NULL)) { + timr_bkt_set_rem(bkt, 0); + tim->impl_opaque[0] = tim->impl_opaque[1] = 0; + tim->state = RTE_EVENT_TIMER_ERROR; + return -ENOMEM; + } + bkt->current_chunk = (uintptr_t) chunk; + timr_bkt_set_rem(bkt, nb_chunk_slots - 1); + } else { + chunk = (struct tim_mem_entry *)(uintptr_t)bkt->current_chunk; + chunk += nb_chunk_slots - rem; + } + /* Copy work entry. */ + *chunk = *pent; + timr_bkt_inc_nent(bkt); + + tim->impl_opaque[0] = (uintptr_t)chunk; + tim->impl_opaque[1] = (uintptr_t)bkt; + tim->state = RTE_EVENT_TIMER_ARMED; + return 0; +} + /* Multi producer functions. */ static inline int timvf_add_entry_mp(struct timvf_ring * const timr, const uint32_t rel_bkt, -- 2.17.0 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v4 08/11] event/octeontx: add burst mode for timer arm 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 00/11] " Pavan Nikhilesh ` (6 preceding siblings ...) 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 07/11] event/octeontx: add single producer timer arm variant Pavan Nikhilesh @ 2018-04-09 21:00 ` Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 09/11] event/octeontx: optimize timer adapter resolution parameters Pavan Nikhilesh ` (3 subsequent siblings) 11 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-09 21:00 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 3 + drivers/event/octeontx/timvf_evdev.h | 7 ++ drivers/event/octeontx/timvf_worker.c | 53 +++++++++++++++ drivers/event/octeontx/timvf_worker.h | 95 +++++++++++++++++++++++++++ 4 files changed, 158 insertions(+) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index 3b698b7c8..b54d8b84f 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -327,6 +327,9 @@ timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, timvf_timer_arm_burst_mp_stats : timvf_timer_arm_burst_mp; + timvf_ops.arm_tmo_tick_burst = enable_stats ? + timvf_timer_arm_tmo_brst_stats : + timvf_timer_arm_tmo_brst; timvf_ops.cancel_burst = timvf_timer_cancel_burst; *caps = RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT; *ops = &timvf_ops; diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index 01ffac09a..2e905702f 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -207,6 +207,13 @@ uint16_t timvf_timer_arm_burst_mp(const struct rte_event_timer_adapter *adptr, uint16_t timvf_timer_arm_burst_mp_stats( const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers); +uint16_t timvf_timer_arm_tmo_brst(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint64_t timeout_tick, + const uint16_t nb_timers); +uint16_t timvf_timer_arm_tmo_brst_stats( + const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint64_t timeout_tick, + const uint16_t nb_timers); void timvf_set_chunk_refill(struct timvf_ring * const timr); #endif /* __TIMVF_EVDEV_H__ */ diff --git a/drivers/event/octeontx/timvf_worker.c b/drivers/event/octeontx/timvf_worker.c index 52b212c44..02e17b6f5 100644 --- a/drivers/event/octeontx/timvf_worker.c +++ b/drivers/event/octeontx/timvf_worker.c @@ -137,6 +137,59 @@ timvf_timer_arm_burst_mp_stats(const struct rte_event_timer_adapter *adptr, return ret; } +uint16_t +timvf_timer_arm_tmo_brst(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint64_t timeout_tick, + const uint16_t nb_timers) +{ + int ret; + uint16_t set_timers = 0; + uint16_t idx; + uint16_t arr_idx = 0; + struct timvf_ring *timr = adptr->data->adapter_priv; + struct tim_mem_entry entry[TIMVF_MAX_BURST] __rte_cache_aligned; + + if (unlikely(!timeout_tick || timeout_tick >= timr->nb_bkts)) { + const enum rte_event_timer_state state = timeout_tick ? + RTE_EVENT_TIMER_ERROR_TOOLATE : + RTE_EVENT_TIMER_ERROR_TOOEARLY; + for (idx = 0; idx < nb_timers; idx++) + tim[idx]->state = state; + rte_errno = EINVAL; + return 0; + } + + while (arr_idx < nb_timers) { + for (idx = 0; idx < TIMVF_MAX_BURST && (arr_idx < nb_timers); + idx++, arr_idx++) { + timvf_format_event(tim[arr_idx], &entry[idx]); + } + ret = timvf_add_entry_brst(timr, timeout_tick, &tim[set_timers], + entry, idx); + set_timers += ret; + if (ret != idx) + break; + } + + return set_timers; +} + + +uint16_t +timvf_timer_arm_tmo_brst_stats(const struct rte_event_timer_adapter *adptr, + struct rte_event_timer **tim, const uint64_t timeout_tick, + const uint16_t nb_timers) +{ + uint16_t set_timers; + struct timvf_ring *timr = adptr->data->adapter_priv; + + set_timers = timvf_timer_arm_tmo_brst(adptr, tim, timeout_tick, + nb_timers); + timr->tim_arm_cnt += set_timers; + + return set_timers; +} + void timvf_set_chunk_refill(struct timvf_ring * const timr) { diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h index c3a669de9..93254cd39 100644 --- a/drivers/event/octeontx/timvf_worker.h +++ b/drivers/event/octeontx/timvf_worker.h @@ -324,3 +324,98 @@ timvf_add_entry_mp(struct timvf_ring * const timr, const uint32_t rel_bkt, tim->state = RTE_EVENT_TIMER_ARMED; return 0; } + +static inline uint16_t +timvf_cpy_wrk(uint16_t index, uint16_t cpy_lmt, + struct tim_mem_entry *chunk, + struct rte_event_timer ** const tim, + const struct tim_mem_entry * const ents, + const struct tim_mem_bucket * const bkt) +{ + for (; index < cpy_lmt; index++) { + *chunk = *(ents + index); + tim[index]->impl_opaque[0] = (uintptr_t)chunk++; + tim[index]->impl_opaque[1] = (uintptr_t)bkt; + tim[index]->state = RTE_EVENT_TIMER_ARMED; + } + + return index; +} + +/* Burst mode functions */ +static inline int +timvf_add_entry_brst(struct timvf_ring * const timr, const uint16_t rel_bkt, + struct rte_event_timer ** const tim, + const struct tim_mem_entry *ents, + const uint16_t nb_timers) +{ + int16_t rem; + int16_t crem; + uint8_t lock_cnt; + uint16_t index = 0; + uint16_t chunk_remainder; + uint64_t lock_sema; + struct tim_mem_bucket *bkt; + struct tim_mem_entry *chunk; + +__retry: + bkt = timvf_get_target_bucket(timr, rel_bkt); + + /* Only one thread beyond this. */ + lock_sema = timr_bkt_inc_lock(bkt); + lock_cnt = (uint8_t) + ((lock_sema >> TIM_BUCKET_W1_S_LOCK) & TIM_BUCKET_W1_M_LOCK); + + if (lock_cnt) { + timr_bkt_dec_lock(bkt); + goto __retry; + } + + /* Bucket related checks. */ + if (unlikely(timr_bkt_get_hbt(lock_sema))) { + timr_bkt_dec_lock(bkt); + goto __retry; + } + + chunk_remainder = timr_bkt_fetch_rem(lock_sema); + rem = chunk_remainder - nb_timers; + if (rem < 0) { + crem = nb_chunk_slots - chunk_remainder; + if (chunk_remainder && crem) { + chunk = ((struct tim_mem_entry *) + (uintptr_t)bkt->current_chunk) + crem; + + index = timvf_cpy_wrk(index, chunk_remainder, + chunk, tim, ents, bkt); + timr_bkt_sub_rem(bkt, chunk_remainder); + timr_bkt_add_nent(bkt, chunk_remainder); + } + rem = nb_timers - chunk_remainder; + ents = ents + chunk_remainder; + + chunk = timr->refill_chunk(bkt, timr); + if (unlikely(chunk == NULL)) { + timr_bkt_dec_lock(bkt); + rte_errno = ENOMEM; + tim[index]->state = RTE_EVENT_TIMER_ERROR; + return crem; + } + *(uint64_t *)(chunk + nb_chunk_slots) = 0; + bkt->current_chunk = (uintptr_t) chunk; + + index = timvf_cpy_wrk(index, nb_timers, chunk, tim, ents, bkt); + timr_bkt_set_rem(bkt, nb_chunk_slots - rem); + timr_bkt_add_nent(bkt, rem); + } else { + chunk = (struct tim_mem_entry *)(uintptr_t)bkt->current_chunk; + chunk += (nb_chunk_slots - chunk_remainder); + + index = timvf_cpy_wrk(index, nb_timers, + chunk, tim, ents, bkt); + timr_bkt_sub_rem(bkt, nb_timers); + timr_bkt_add_nent(bkt, nb_timers); + } + + timr_bkt_dec_lock(bkt); + return nb_timers; +} -- 2.17.0 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v4 09/11] event/octeontx: optimize timer adapter resolution parameters 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 00/11] " Pavan Nikhilesh ` (7 preceding siblings ...) 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 08/11] event/octeontx: add burst mode for timer arm Pavan Nikhilesh @ 2018-04-09 21:00 ` Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 10/11] event/octeontx: add option to use fpavf as chunk pool Pavan Nikhilesh ` (2 subsequent siblings) 11 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-09 21:00 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh When application sets `RTE_EVENT_TIMER_ADAPTER_F_ADJUST_RES` flag while creating adapter underlying driver is free to optimize the resolution for best possible configuration. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 52 +++++++++++++++++++++++++++- drivers/event/octeontx/timvf_evdev.h | 6 ++++ 2 files changed, 57 insertions(+), 1 deletion(-) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index b54d8b84f..d6a8bb355 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -82,6 +82,45 @@ timvf_get_start_cyc(uint64_t *now, uint8_t ring_id) return octeontx_mbox_send(&hdr, NULL, 0, now, sizeof(uint64_t)); } +static int +optimize_bucket_parameters(struct timvf_ring *timr) +{ + uint32_t hbkts; + uint32_t lbkts; + uint64_t tck_nsec; + + hbkts = rte_align32pow2(timr->nb_bkts); + tck_nsec = RTE_ALIGN_MUL_CEIL(timr->max_tout / (hbkts - 1), 10); + + if ((tck_nsec < 1000 || hbkts > TIM_MAX_BUCKETS)) + hbkts = 0; + + lbkts = rte_align32prevpow2(timr->nb_bkts); + tck_nsec = RTE_ALIGN_MUL_CEIL((timr->max_tout / (lbkts - 1)), 10); + + if ((tck_nsec < 1000 || hbkts > TIM_MAX_BUCKETS)) + lbkts = 0; + + if (!hbkts && !lbkts) + return 0; + + if (!hbkts) { + timr->nb_bkts = lbkts; + goto end; + } else if (!lbkts) { + timr->nb_bkts = hbkts; + goto end; + } + + timr->nb_bkts = (hbkts - timr->nb_bkts) < + (timr->nb_bkts - lbkts) ? hbkts : lbkts; +end: + timr->get_target_bkt = bkt_and; + timr->tck_nsec = RTE_ALIGN_MUL_CEIL((timr->max_tout / + (timr->nb_bkts - 1)), 10); + return 1; +} + static int timvf_ring_start(const struct rte_event_timer_adapter *adptr) { @@ -204,7 +243,7 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) timr->clk_src = (int) rcfg->clk_src; timr->tim_ring_id = adptr->data->id; - timr->tck_nsec = rcfg->timer_tick_ns; + timr->tck_nsec = RTE_ALIGN_MUL_CEIL(rcfg->timer_tick_ns, 10); timr->max_tout = rcfg->max_tmo_ns; timr->nb_bkts = (timr->max_tout / timr->tck_nsec); timr->vbar0 = timvf_bar(timr->tim_ring_id, 0); @@ -214,6 +253,17 @@ timvf_ring_create(struct rte_event_timer_adapter *adptr) timr->nb_chunks = nb_timers / nb_chunk_slots; + /* Try to optimize the bucket parameters. */ + if ((rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_ADJUST_RES) + && !rte_is_power_of_2(timr->nb_bkts)) { + if (optimize_bucket_parameters(timr)) { + timvf_log_info("Optimized configured values"); + timvf_log_dbg("nb_bkts : %"PRIu32"", timr->nb_bkts); + timvf_log_dbg("tck_nsec : %"PRIu64"", timr->tck_nsec); + } else + timvf_log_info("Failed to Optimize configured values"); + } + if (rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_SP_PUT) { mp_flags = MEMPOOL_F_SP_PUT | MEMPOOL_F_SC_GET; timvf_log_info("Using single producer mode"); diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index 2e905702f..b3fc343af 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -190,6 +190,12 @@ bkt_mod(const uint32_t rel_bkt, const uint32_t nb_bkts) return rel_bkt % nb_bkts; } +static __rte_always_inline uint32_t +bkt_and(uint32_t rel_bkt, uint32_t nb_bkts) +{ + return rel_bkt & (nb_bkts - 1); +} + int timvf_info(struct timvf_info *tinfo); void *timvf_bar(uint8_t id, uint8_t bar); int timvf_timer_adapter_caps_get(const struct rte_eventdev *dev, uint64_t flags, -- 2.17.0 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v4 10/11] event/octeontx: add option to use fpavf as chunk pool 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 00/11] " Pavan Nikhilesh ` (8 preceding siblings ...) 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 09/11] event/octeontx: optimize timer adapter resolution parameters Pavan Nikhilesh @ 2018-04-09 21:00 ` Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 11/11] doc: update eventdev OcteonTx documentation Pavan Nikhilesh 2018-04-12 12:30 ` [dpdk-dev] [PATCH v4 00/11] event/octeontx: add event timer adapter driver Jerin Jacob 11 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-09 21:00 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Add compile-time configurable option to force TIMvf to use Octeontx FPAvf pool manager as its chunk pool. When FPAvf is used as pool manager the TIMvf automatically frees the chunks to FPAvf through gpool-id. Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- drivers/event/octeontx/timvf_evdev.c | 22 +++++++++++++++++++++- drivers/event/octeontx/timvf_evdev.h | 3 ++- drivers/event/octeontx/timvf_worker.c | 7 +++++-- drivers/event/octeontx/timvf_worker.h | 22 ++++++++++++++++++++++ 4 files changed, 50 insertions(+), 4 deletions(-) diff --git a/drivers/event/octeontx/timvf_evdev.c b/drivers/event/octeontx/timvf_evdev.c index d6a8bb355..b20a2f1f5 100644 --- a/drivers/event/octeontx/timvf_evdev.c +++ b/drivers/event/octeontx/timvf_evdev.c @@ -125,7 +125,9 @@ static int timvf_ring_start(const struct rte_event_timer_adapter *adptr) { int ret; + uint8_t use_fpa = 0; uint64_t interval; + uintptr_t pool; struct timvf_ctrl_reg rctrl; struct timvf_mbox_dev_info dinfo; struct timvf_ring *timr = adptr->data->adapter_priv; @@ -155,6 +157,9 @@ timvf_ring_start(const struct rte_event_timer_adapter *adptr) return -EINVAL; } + if (!strcmp(rte_mbuf_best_mempool_ops(), "octeontx_fpavf")) + use_fpa = 1; + /*CTRL0 register.*/ rctrl.rctrl0 = interval; @@ -167,9 +172,24 @@ timvf_ring_start(const struct rte_event_timer_adapter *adptr) rctrl.rctrl2 = (uint64_t)(TIM_CHUNK_SIZE / 16) << 40; + if (use_fpa) { + pool = (uintptr_t)((struct rte_mempool *) + timr->chunk_pool)->pool_id; + ret = octeontx_fpa_bufpool_gpool(pool); + if (ret < 0) { + timvf_log_dbg("Unable to get gaura id"); + ret = -ENOMEM; + goto error; + } + timvf_write64((uint64_t)ret, + (uint8_t *)timr->vbar0 + TIM_VRING_AURA); + } else { + rctrl.rctrl1 |= 1ull << 43 /* ENA_DFB (Enable don't free) */; + } + timvf_write64((uintptr_t)timr->bkt, (uint8_t *)timr->vbar0 + TIM_VRING_BASE); - timvf_set_chunk_refill(timr); + timvf_set_chunk_refill(timr, use_fpa); if (timvf_ring_conf_set(&rctrl, timr->tim_ring_id)) { ret = -EACCES; goto error; diff --git a/drivers/event/octeontx/timvf_evdev.h b/drivers/event/octeontx/timvf_evdev.h index b3fc343af..b1b2a8464 100644 --- a/drivers/event/octeontx/timvf_evdev.h +++ b/drivers/event/octeontx/timvf_evdev.h @@ -25,6 +25,7 @@ #include <rte_reciprocal.h> #include <octeontx_mbox.h> +#include <octeontx_fpavf.h> #define timvf_log(level, fmt, args...) \ rte_log(RTE_LOG_ ## level, otx_logtype_timvf, \ @@ -220,6 +221,6 @@ uint16_t timvf_timer_arm_tmo_brst_stats( const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint64_t timeout_tick, const uint16_t nb_timers); -void timvf_set_chunk_refill(struct timvf_ring * const timr); +void timvf_set_chunk_refill(struct timvf_ring * const timr, uint8_t use_fpa); #endif /* __TIMVF_EVDEV_H__ */ diff --git a/drivers/event/octeontx/timvf_worker.c b/drivers/event/octeontx/timvf_worker.c index 02e17b6f5..e681bc6b8 100644 --- a/drivers/event/octeontx/timvf_worker.c +++ b/drivers/event/octeontx/timvf_worker.c @@ -191,7 +191,10 @@ timvf_timer_arm_tmo_brst_stats(const struct rte_event_timer_adapter *adptr, } void -timvf_set_chunk_refill(struct timvf_ring * const timr) +timvf_set_chunk_refill(struct timvf_ring * const timr, uint8_t use_fpa) { - timr->refill_chunk = timvf_refill_chunk_generic; + if (use_fpa) + timr->refill_chunk = timvf_refill_chunk_fpa; + else + timr->refill_chunk = timvf_refill_chunk_generic; } diff --git a/drivers/event/octeontx/timvf_worker.h b/drivers/event/octeontx/timvf_worker.h index 93254cd39..dede1a4a4 100644 --- a/drivers/event/octeontx/timvf_worker.h +++ b/drivers/event/octeontx/timvf_worker.h @@ -213,6 +213,28 @@ timvf_refill_chunk_generic(struct tim_mem_bucket * const bkt, return chunk; } +static inline struct tim_mem_entry * +timvf_refill_chunk_fpa(struct tim_mem_bucket * const bkt, + struct timvf_ring * const timr) +{ + struct tim_mem_entry *chunk; + + if (unlikely(rte_mempool_get(timr->chunk_pool, (void **)&chunk))) + return NULL; + + *(uint64_t *)(chunk + nb_chunk_slots) = 0; + if (bkt->nb_entry) { + *(uint64_t *)(((struct tim_mem_entry *)(uintptr_t) + bkt->current_chunk) + + nb_chunk_slots) = + (uintptr_t) chunk; + } else { + bkt->first_chunk = (uintptr_t) chunk; + } + + return chunk; +} + static inline struct tim_mem_bucket * timvf_get_target_bucket(struct timvf_ring * const timr, const uint32_t rel_bkt) { -- 2.17.0 ^ permalink raw reply [flat|nested] 76+ messages in thread
* [dpdk-dev] [PATCH v4 11/11] doc: update eventdev OcteonTx documentation 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 00/11] " Pavan Nikhilesh ` (9 preceding siblings ...) 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 10/11] event/octeontx: add option to use fpavf as chunk pool Pavan Nikhilesh @ 2018-04-09 21:00 ` Pavan Nikhilesh 2018-04-12 12:30 ` [dpdk-dev] [PATCH v4 00/11] event/octeontx: add event timer adapter driver Jerin Jacob 11 siblings, 0 replies; 76+ messages in thread From: Pavan Nikhilesh @ 2018-04-09 21:00 UTC (permalink / raw) To: jerin.jacob, santosh.shukla, erik.g.carrillo; +Cc: dev, Pavan Nikhilesh Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com> --- MAINTAINERS | 4 ++++ doc/guides/eventdevs/octeontx.rst | 29 +++++++++++++++++++++++++++++ 2 files changed, 33 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 7e966580b..c201e0f7f 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -744,6 +744,10 @@ M: Santosh Shukla <santosh.shukla@caviumnetworks.com> F: drivers/event/octeontx/ F: doc/guides/eventdevs/octeontx.rst +Cavium OCTEONTX timvf +M: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> +F: drivers/event/octeontx/timvf_* + NXP DPAA2 eventdev M: Hemant Agrawal <hemant.agrawal@nxp.com> M: Nipun Gupta <nipun.gupta@nxp.com> diff --git a/doc/guides/eventdevs/octeontx.rst b/doc/guides/eventdevs/octeontx.rst index 4fabe54f9..f77bc5c54 100644 --- a/doc/guides/eventdevs/octeontx.rst +++ b/doc/guides/eventdevs/octeontx.rst @@ -28,6 +28,9 @@ Features of the OCTEONTX SSOVF PMD are: - Open system with configurable amount of outstanding events - HW accelerated dequeue timeout support to enable power management - SR-IOV VF +- HW managed event timers support through TIMVF, with high precision and + time granularity of 1us. +- Up to 64 event timer adapters. Supported OCTEONTX SoCs ----------------------- @@ -96,6 +99,16 @@ The tests are run once the vdev creation is successfully complete. --vdev="event_octeontx,self_test=1" +Enable TIMvf stats +------------------ +TIMvf stats can be enabled by using this option, by default the stats are +disabled. + +.. code-block:: console + + --vdev="event_octeontx,timvf_stats=1" + + Limitations ----------- @@ -110,3 +123,19 @@ Rx adapter support When eth_octeontx is used as Rx adapter event schedule type ``RTE_SCHED_TYPE_PARALLEL`` is not supported. + +Event timer adapter support +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When timvf is used as Event timer adapter the clock source mapping is as +follows: + +.. code-block:: console + + RTE_EVENT_TIMER_ADAPTER_CPU_CLK = TIM_CLK_SRC_SCLK + RTE_EVENT_TIMER_ADAPTER_EXT_CLK0 = TIM_CLK_SRC_GPIO + RTE_EVENT_TIMER_ADAPTER_EXT_CLK1 = TIM_CLK_SRC_GTI + RTE_EVENT_TIMER_ADAPTER_EXT_CLK2 = TIM_CLK_SRC_PTP + +When timvf is used as Event timer adapter event schedule type +``RTE_SCHED_TYPE_PARALLEL`` is not supported. -- 2.17.0 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: [dpdk-dev] [PATCH v4 00/11] event/octeontx: add event timer adapter driver 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 00/11] " Pavan Nikhilesh ` (10 preceding siblings ...) 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 11/11] doc: update eventdev OcteonTx documentation Pavan Nikhilesh @ 2018-04-12 12:30 ` Jerin Jacob 11 siblings, 0 replies; 76+ messages in thread From: Jerin Jacob @ 2018-04-12 12:30 UTC (permalink / raw) To: Pavan Nikhilesh; +Cc: santosh.shukla, erik.g.carrillo, dev -----Original Message----- > Date: Tue, 10 Apr 2018 02:30:24 +0530 > From: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, > erik.g.carrillo@intel.com > Cc: dev@dpdk.org, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > Subject: [dpdk-dev] [PATCH v4 00/11] event/octeontx: add event timer > adapter driver > X-Mailer: git-send-email 2.17.0 > > The event timer adapter[1] provides APIs to configure an event timer device > that allows an application to arm timers which on expiry push events to an > event device such as OcteonTx SSO. > The OcteonTx TIM is a co-processor that can be configured as an event timer > adapter which can be used by an application to manage event timers. > > The TIM co-processor processes the event timers registered and pushes > expired event timers to SSO based on the event queue, schedule type, flow > id etc. provided as rte_event while arming the event timer. It maintains > event timers with high precision and time granularity of 1us (microsecond). > > [1] http://dpdk.org/dev/patchwork/patch/33525/ Applied to series to dpdk-next-eventdev/master. Thanks. > ^ permalink raw reply [flat|nested] 76+ messages in thread
end of thread, other threads:[~2018-04-12 12:31 UTC | newest] Thread overview: 76+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-02-16 21:36 [dpdk-dev] [PATCH 0/9] event/octeontx: add event timer adapter driver Pavan Nikhilesh 2018-02-16 21:36 ` [dpdk-dev] [PATCH 01/10] eal: add API to align variable to previous power of 2 Pavan Nikhilesh 2018-02-17 4:49 ` Jerin Jacob 2018-02-16 21:36 ` [dpdk-dev] [PATCH 02/10] mempool/octeontx: probe timvf PCIe devices Pavan Nikhilesh 2018-02-17 4:54 ` Jerin Jacob 2018-02-19 9:19 ` Pavan Nikhilesh 2018-02-16 21:36 ` [dpdk-dev] [PATCH 03/10] event/octeontx: add support to create and free timer adapter Pavan Nikhilesh 2018-02-16 21:36 ` [dpdk-dev] [PATCH 04/10] event/octeontx: add support to start and stop timer device Pavan Nikhilesh 2018-02-16 21:36 ` [dpdk-dev] [PATCH 05/10] event/octeontx: add support for arm and cancel Pavan Nikhilesh 2018-02-16 21:36 ` [dpdk-dev] [PATCH 06/10] event/octeontx: add single producer timer arm variant Pavan Nikhilesh 2018-03-07 19:41 ` Carrillo, Erik G 2018-02-16 21:36 ` [dpdk-dev] [PATCH 07/10] event/octeontx: optimize timer adapter resolution parameters Pavan Nikhilesh 2018-02-17 5:06 ` Jerin Jacob 2018-02-19 9:34 ` Pavan Nikhilesh 2018-02-16 21:36 ` [dpdk-dev] [PATCH 08/10] event/octeontx: add option to use fpavf as chunk pool Pavan Nikhilesh 2018-02-18 11:42 ` santosh 2018-02-19 9:15 ` Pavan Nikhilesh 2018-02-23 20:17 ` Carrillo, Erik G 2018-02-26 19:25 ` Pavan Nikhilesh 2018-02-16 21:36 ` [dpdk-dev] [PATCH 09/10] event/octeontx: add timer adapter SW traversal routine Pavan Nikhilesh 2018-02-17 5:01 ` Jerin Jacob 2018-02-16 21:37 ` [dpdk-dev] [PATCH 10/10] maintainers: claim responsibility for octeontx timvf Pavan Nikhilesh 2018-02-17 5:03 ` Jerin Jacob 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 00/11] event/octeontx: add event timer adapter driver Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 01/11] mempool/octeontx: probe timvf PCIe devices Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 02/11] usertools: add Cavium TIM as an event device Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 03/11] event/octeontx: add support to create and free timer adapter Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 04/11] event/octeontx: add support to start and stop timer device Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 05/11] event/octeontx: add multiproducer timer arm and cancel Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 06/11] event/octeontx: add single producer timer arm variant Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 07/11] event/octeontx: add burst mode for timer arm Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 08/11] event/octeontx: optimize timer adapter resolution parameters Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 09/11] event/octeontx: add option to use fpavf as chunk pool Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 10/11] doc: update eventdev OcteonTx documentation Pavan Nikhilesh 2018-03-14 13:52 ` [dpdk-dev] [PATCH v2 11/11] maintainers: claim responsibility for octeontx timvf Pavan Nikhilesh 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Pavan Nikhilesh 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 01/12] mempool/octeontx: probe timvf PCIe devices Pavan Nikhilesh 2018-04-08 2:19 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 02/12] usertools: add Cavium TIM as an event device Pavan Nikhilesh 2018-04-08 3:23 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 03/12] event/octeontx: add support to create and free timer adapter Pavan Nikhilesh 2018-04-08 7:39 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 04/12] event/octeontx: add support to start and stop timer device Pavan Nikhilesh 2018-04-08 7:47 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 05/12] event/octeontx: add event timer stats get and reset Pavan Nikhilesh 2018-04-04 1:59 ` Carrillo, Erik G 2018-04-08 7:52 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 06/12] event/octeontx: add multiproducer timer arm and cancel Pavan Nikhilesh 2018-04-08 7:59 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 07/12] event/octeontx: add single producer timer arm variant Pavan Nikhilesh 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 08/12] event/octeontx: add burst mode for timer arm Pavan Nikhilesh 2018-04-04 2:15 ` Carrillo, Erik G 2018-04-08 8:06 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 09/12] event/octeontx: optimize timer adapter resolution parameters Pavan Nikhilesh 2018-04-08 8:09 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 10/12] event/octeontx: add option to use fpavf as chunk pool Pavan Nikhilesh 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 11/12] doc: update eventdev OcteonTx documentation Pavan Nikhilesh 2018-04-04 2:19 ` Carrillo, Erik G 2018-04-08 8:13 ` Jerin Jacob 2018-04-03 15:05 ` [dpdk-dev] [PATCH v3 12/12] maintainers: claim responsibility for octeontx timvf Pavan Nikhilesh 2018-04-08 8:15 ` Jerin Jacob 2018-04-08 2:55 ` [dpdk-dev] [PATCH v3 00/12] event/octeontx: add event timer adapter driver Jerin Jacob 2018-04-08 8:58 ` Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 00/11] " Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 01/11] usertools: add Cavium TIM as an event device Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 02/11] event/octeontx: add support to probe timvf PCIe devices Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 03/11] event/octeontx: add support to create and free timer adapter Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 04/11] event/octeontx: add support to start and stop timer device Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 05/11] event/octeontx: add event timer stats get and reset Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 06/11] event/octeontx: add multiproducer timer arm and cancel Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 07/11] event/octeontx: add single producer timer arm variant Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 08/11] event/octeontx: add burst mode for timer arm Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 09/11] event/octeontx: optimize timer adapter resolution parameters Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 10/11] event/octeontx: add option to use fpavf as chunk pool Pavan Nikhilesh 2018-04-09 21:00 ` [dpdk-dev] [PATCH v4 11/11] doc: update eventdev OcteonTx documentation Pavan Nikhilesh 2018-04-12 12:30 ` [dpdk-dev] [PATCH v4 00/11] event/octeontx: add event timer adapter driver Jerin Jacob
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).