From: "Carrillo, Erik G" <erik.g.carrillo@intel.com>
To: Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com>,
"Jerin Jacob Kollanukkaran" <jerinj@marvell.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v3 11/14] eventdev: move timer adapters memory to hugepage
Date: Fri, 8 Oct 2021 15:57:20 +0000 [thread overview]
Message-ID: <SA2PR11MB519671F2DAD4C0EFAABEB203B9B29@SA2PR11MB5196.namprd11.prod.outlook.com> (raw)
In-Reply-To: <PH0PR18MB4086DF37FAD76842E7D7911EDEB29@PH0PR18MB4086.namprd18.prod.outlook.com>
> -----Original Message-----
> From: Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com>
> Sent: Friday, October 8, 2021 12:38 AM
> To: Carrillo, Erik G <erik.g.carrillo@intel.com>; Jerin Jacob Kollanukkaran
> <jerinj@marvell.com>
> Cc: dev@dpdk.org
> Subject: RE: [dpdk-dev] [PATCH v3 11/14] eventdev: move timer adapters
> memory to hugepage
>
> Hi Erik,
>
> >Hi Pavan,
> >
> >Some comments below:
> >
> >> -----Original Message-----
> >> From: pbhagavatula@marvell.com <pbhagavatula@marvell.com>
> >> Sent: Wednesday, October 6, 2021 1:50 AM
> >> To: jerinj@marvell.com; Carrillo, Erik G <erik.g.carrillo@intel.com>
> >> Cc: dev@dpdk.org; Pavan Nikhilesh <pbhagavatula@marvell.com>
> >> Subject: [dpdk-dev] [PATCH v3 11/14] eventdev: move timer adapters
> >> memory to hugepage
> >>
> >> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> >>
> >> Move memory used by timer adapters to hugepage.
> >> Allocate memory on the first adapter create or lookup to address both
> >> primary and secondary process usecases.
> >> This will prevent TLB misses if any and aligns to memory structure of
> >other
> >> subsystems.
> >>
> >> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
> >> ---
> >> lib/eventdev/rte_event_timer_adapter.c | 24
> >> +++++++++++++++++++++++-
> >> 1 file changed, 23 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/lib/eventdev/rte_event_timer_adapter.c
> >> b/lib/eventdev/rte_event_timer_adapter.c
> >> index ae55407042..c4dc7a5fd4 100644
> >> --- a/lib/eventdev/rte_event_timer_adapter.c
> >> +++ b/lib/eventdev/rte_event_timer_adapter.c
> >> @@ -33,7 +33,7 @@ RTE_LOG_REGISTER_SUFFIX(evtim_logtype,
> >> adapter.timer, NOTICE);
> >> RTE_LOG_REGISTER_SUFFIX(evtim_buffer_logtype, adapter.timer,
> >NOTICE);
> >> RTE_LOG_REGISTER_SUFFIX(evtim_svc_logtype, adapter.timer.svc,
> >> NOTICE);
> >>
> >> -static struct rte_event_timer_adapter
> >> adapters[RTE_EVENT_TIMER_ADAPTER_NUM_MAX];
> >> +static struct rte_event_timer_adapter *adapters;
> >>
> >> static const struct event_timer_adapter_ops swtim_ops;
> >>
> >> @@ -138,6 +138,17 @@ rte_event_timer_adapter_create_ext(
> >> int n, ret;
> >> struct rte_eventdev *dev;
> >>
> >> + if (adapters == NULL) {
> >> + adapters = rte_zmalloc("Eventdev",
> >> + sizeof(struct
> >rte_event_timer_adapter) *
> >> +
> >> RTE_EVENT_TIMER_ADAPTER_NUM_MAX,
> >> + RTE_CACHE_LINE_SIZE);
> >> + if (adapters == NULL) {
> >> + rte_errno = ENOMEM;
> >> + return NULL;
> >> + }
> >> + }
> >> +
> >> if (conf == NULL) {
> >> rte_errno = EINVAL;
> >> return NULL;
> >> @@ -312,6 +323,17 @@ rte_event_timer_adapter_lookup(uint16_t
> >> adapter_id)
> >> int ret;
> >> struct rte_eventdev *dev;
> >>
> >> + if (adapters == NULL) {
> >> + adapters = rte_zmalloc("Eventdev",
> >> + sizeof(struct
> >rte_event_timer_adapter) *
> >> +
> >> RTE_EVENT_TIMER_ADAPTER_NUM_MAX,
> >> + RTE_CACHE_LINE_SIZE);
> >> + if (adapters == NULL) {
> >> + rte_errno = ENOMEM;
> >> + return NULL;
> >> + }
> >> + }
> >> +
> >> if (adapters[adapter_id].allocated)
> >> return &adapters[adapter_id]; /* Adapter is already
> >loaded
> >> */
> >>
> >> --
> >> 2.17.1
> >
> >The rte_event_timer_adapter struct has several fields that have per-
> >process values.
> >
> >For example, there are three fast path function pointers and each will
> >be assigned distinct addresses for each process in a multi-process
> >scenario. The "allocated" field is also per-process. With the changes
> >above, if a secondary process did a lookup() after a primary process
> >did a create(), the secondary would get a reference to an object with
> >function pointers that are invalid in the secondary process.
> >
>
> I understand, the current patch doesn't unify the memory between
> processes.
> Instead, we zmalloc the per-process 'array' that holds the adapter objects
> when ever the process calls either create or lookup and initialize the per-
> process data structure.
>
> The pointer to the adapter array is static, so when ever a process is initialized
> it will be NULL.
>
Ah, right - I missed that. This looks OK to me now.
One other thing: we never do a free of the array we zmalloc'd, but it looks like we could in rte_event_timer_adapter_free(), if we were freeing the last adapter instance.
>
> >To fully move the adapter object table into shared hugepage memory,
> >those "per-process" members would need to be collected into a per-
> >process data structure that could be independently allocated for each
> >process. However, that would add one more pointer dereference to get
> >to the fast path functions, and avoiding that was the original reason
> >to put those pointers there. This is similar to the rte_eventdev struct.
> >
> >Thanks,
> >Erik
next prev parent reply other threads:[~2021-10-08 15:57 UTC|newest]
Thread overview: 119+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-23 19:40 [dpdk-dev] [RFC 01/15] eventdev: make driver interface as internal pbhagavatula
2021-08-23 19:40 ` [dpdk-dev] [RFC 02/15] eventdev: separate internal structures pbhagavatula
2021-10-14 9:11 ` Jerin Jacob
2021-08-23 19:40 ` [dpdk-dev] [RFC 03/15] eventdev: move eventdevs globals to hugepage mem pbhagavatula
2021-08-23 19:40 ` [dpdk-dev] [RFC 04/15] eventdev: move inline APIs into separate structure pbhagavatula
2021-09-08 12:03 ` Kinsella, Ray
2021-08-23 19:40 ` [dpdk-dev] [RFC 05/15] eventdev: add helper functions for new driver API pbhagavatula
2021-09-08 12:04 ` Kinsella, Ray
2021-08-23 19:40 ` [dpdk-dev] [RFC 06/15] eventdev: use new API for inline functions pbhagavatula
2021-08-30 14:41 ` Jayatheerthan, Jay
2021-08-30 14:46 ` David Marchand
2021-10-02 20:32 ` [dpdk-dev] [EXT] " Pavan Nikhilesh Bhagavatula
2021-08-23 19:40 ` [dpdk-dev] [RFC 07/15] eventdev: make drivers to use new API pbhagavatula
2021-09-08 6:43 ` Hemant Agrawal
2021-08-23 19:40 ` [dpdk-dev] [RFC 08/15] eventdev: hide event device related structures pbhagavatula
2021-08-23 19:40 ` [dpdk-dev] [RFC 09/15] eventdev: hide timer adapter pmd file pbhagavatula
2021-08-23 19:40 ` [dpdk-dev] [RFC 10/15] eventdev: remove rte prefix for internal structs pbhagavatula
2021-08-30 14:42 ` Jayatheerthan, Jay
2021-08-23 19:40 ` [dpdk-dev] [RFC 11/15] eventdev: reserve fields in timer object pbhagavatula
2021-08-23 20:42 ` Carrillo, Erik G
2021-08-24 5:16 ` Pavan Nikhilesh Bhagavatula
2021-08-24 15:10 ` Stephen Hemminger
2021-09-01 6:48 ` [dpdk-dev] [EXT] " Pavan Nikhilesh Bhagavatula
2021-09-07 21:02 ` Carrillo, Erik G
2021-09-07 21:31 ` [dpdk-dev] " Stephen Hemminger
2021-08-23 19:40 ` [dpdk-dev] [RFC 12/15] eventdev: move timer adapters memory to hugepage pbhagavatula
2021-08-24 13:50 ` Carrillo, Erik G
2021-09-01 6:30 ` Pavan Nikhilesh Bhagavatula
2021-08-23 19:40 ` [dpdk-dev] [RFC 13/15] eventdev: promote event vector API to stable pbhagavatula
2021-08-30 14:43 ` Jayatheerthan, Jay
2021-09-08 12:05 ` Kinsella, Ray
2021-08-23 19:40 ` [dpdk-dev] [RFC 14/15] eventdev: make trace APIs internal pbhagavatula
2021-08-30 14:47 ` Jayatheerthan, Jay
2021-08-23 19:40 ` [dpdk-dev] [RFC 15/15] eventdev: promote trace variables to stable pbhagavatula
2021-09-08 12:06 ` Kinsella, Ray
2021-08-24 7:43 ` [dpdk-dev] [RFC 01/15] eventdev: make driver interface as internal Mattias Rönnblom
2021-08-24 7:47 ` Pavan Nikhilesh Bhagavatula
2021-08-24 8:05 ` Pavan Nikhilesh Bhagavatula
2021-08-30 10:25 ` Mattias Rönnblom
2021-08-30 16:00 ` [dpdk-dev] [RFC] eventdev: uninline inline API functions Mattias Rönnblom
2021-08-31 12:28 ` Jerin Jacob
2021-08-31 12:34 ` Mattias Rönnblom
2021-09-28 9:56 ` [dpdk-dev] [RFC 01/15] eventdev: make driver interface as internal Jerin Jacob
2021-10-03 8:26 ` [dpdk-dev] [PATCH v2 01/13] " pbhagavatula
2021-10-03 8:26 ` [dpdk-dev] [PATCH v2 02/13] eventdev: separate internal structures pbhagavatula
2021-10-03 8:26 ` [dpdk-dev] [PATCH v2 03/13] eventdev: allocate max space for internal arrays pbhagavatula
2021-10-03 8:27 ` [dpdk-dev] [PATCH v2 04/13] eventdev: move inline APIs into separate structure pbhagavatula
2021-10-03 8:27 ` [dpdk-dev] [PATCH v2 05/13] eventdev: use new API for inline functions pbhagavatula
2021-10-03 8:27 ` [dpdk-dev] [PATCH v2 06/13] eventdev: hide event device related structures pbhagavatula
2021-10-03 8:27 ` [dpdk-dev] [PATCH v 07/13] eventdev: hide timer adapter PMD file pbhagavatula
2021-10-03 8:27 ` [dpdk-dev] [PATCH v2 " pbhagavatula
2021-10-03 8:27 ` [dpdk-dev] [PATCH v2 08/13] eventdev: remove rte prefix for internal structs pbhagavatula
2021-10-03 8:27 ` [dpdk-dev] [PATCH v2 09/13] eventdev: rearrange fields in timer object pbhagavatula
2021-10-03 8:27 ` [dpdk-dev] [PATCH v2 10/13] eventdev: move timer adapters memory to hugepage pbhagavatula
2021-10-03 8:27 ` [dpdk-dev] [PATCH v2 11/13] eventdev: promote event vector API to stable pbhagavatula
2021-10-03 8:27 ` [dpdk-dev] [PATCH v2 12/13] eventdev: make trace APIs internal pbhagavatula
2021-10-03 8:27 ` [dpdk-dev] [PATCH v2 13/13] eventdev: mark trace variables as internal pbhagavatula
2021-10-06 6:49 ` [dpdk-dev] [PATCH v3 01/14] eventdev: make driver interface " pbhagavatula
2021-10-06 6:49 ` [dpdk-dev] [PATCH v3 02/14] eventdev: separate internal structures pbhagavatula
2021-10-06 6:50 ` [dpdk-dev] [PATCH v3 03/14] eventdev: allocate max space for internal arrays pbhagavatula
2021-10-06 6:50 ` [dpdk-dev] [PATCH v3 04/14] eventdev: move inline APIs into separate structure pbhagavatula
2021-10-14 9:20 ` Jerin Jacob
2021-10-06 6:50 ` [dpdk-dev] [PATCH v3 05/14] drivers/event: invoke probing finish function pbhagavatula
2021-10-14 9:22 ` Jerin Jacob
2021-10-06 6:50 ` [dpdk-dev] [PATCH v3 06/14] eventdev: use new API for inline functions pbhagavatula
2021-10-11 9:51 ` Gujjar, Abhinandan S
2021-10-06 6:50 ` [dpdk-dev] [PATCH v3 07/14] eventdev: hide event device related structures pbhagavatula
2021-10-06 6:50 ` [dpdk-dev] [PATCH v3 08/14] eventdev: hide timer adapter PMD file pbhagavatula
2021-10-06 6:50 ` [dpdk-dev] [PATCH v3 09/14] eventdev: remove rte prefix for internal structs pbhagavatula
2021-10-11 9:58 ` Gujjar, Abhinandan S
2021-10-06 6:50 ` [dpdk-dev] [PATCH v3 10/14] eventdev: rearrange fields in timer object pbhagavatula
2021-10-06 6:50 ` [dpdk-dev] [PATCH v3 11/14] eventdev: move timer adapters memory to hugepage pbhagavatula
2021-10-07 20:49 ` Carrillo, Erik G
2021-10-08 5:38 ` Pavan Nikhilesh Bhagavatula
2021-10-08 15:57 ` Carrillo, Erik G [this message]
2021-10-06 6:50 ` [dpdk-dev] [PATCH v3 12/14] eventdev: promote event vector API to stable pbhagavatula
2021-10-06 6:50 ` [dpdk-dev] [PATCH v3 13/14] eventdev: make trace APIs internal pbhagavatula
2021-10-11 9:59 ` Gujjar, Abhinandan S
2021-10-06 6:50 ` [dpdk-dev] [PATCH v3 14/14] eventdev: mark trace variables as internal pbhagavatula
2021-10-06 7:11 ` David Marchand
2021-10-14 9:28 ` Jerin Jacob
2021-10-14 9:05 ` [dpdk-dev] [PATCH v3 01/14] eventdev: make driver interface " Jerin Jacob
2021-10-14 9:08 ` Jerin Jacob
2021-10-15 19:02 ` [dpdk-dev] [PATCH v4 " pbhagavatula
2021-10-15 19:02 ` [dpdk-dev] [PATCH v4 02/14] eventdev: separate internal structures pbhagavatula
2021-10-15 19:02 ` [dpdk-dev] [PATCH v4 03/14] eventdev: allocate max space for internal arrays pbhagavatula
2021-10-15 19:02 ` [dpdk-dev] [PATCH v4 04/14] eventdev: move inline APIs into separate structure pbhagavatula
2021-10-15 19:02 ` [dpdk-dev] [PATCH v4 05/14] drivers/event: invoke probing finish function pbhagavatula
2021-10-17 15:34 ` Hemant Agrawal
2021-10-15 19:02 ` [dpdk-dev] [PATCH v4 06/14] eventdev: use new API for inline functions pbhagavatula
2021-10-15 19:02 ` [dpdk-dev] [PATCH v4 07/14] eventdev: hide event device related structures pbhagavatula
2021-10-18 7:07 ` Harman Kalra
2021-10-15 19:02 ` [dpdk-dev] [PATCH v4 08/14] eventdev: hide timer adapter PMD file pbhagavatula
2021-10-15 19:02 ` [dpdk-dev] [PATCH v4 09/14] eventdev: remove rte prefix for internal structs pbhagavatula
2021-10-15 19:02 ` [dpdk-dev] [PATCH v4 10/14] eventdev: rearrange fields in timer object pbhagavatula
2021-10-15 19:02 ` [dpdk-dev] [PATCH v4 11/14] eventdev: move timer adapters memory to hugepage pbhagavatula
2021-10-15 19:02 ` [dpdk-dev] [PATCH v4 12/14] eventdev: promote event vector API to stable pbhagavatula
2021-10-15 19:02 ` [dpdk-dev] [PATCH v4 13/14] eventdev: make trace APIs internal pbhagavatula
2021-10-15 19:02 ` [dpdk-dev] [PATCH v4 14/14] eventdev: mark trace variables as internal pbhagavatula
2021-10-17 5:58 ` Jerin Jacob
2021-10-18 15:06 ` [dpdk-dev] [EXT] " Pavan Nikhilesh Bhagavatula
2021-10-19 7:01 ` David Marchand
2021-10-17 15:35 ` [dpdk-dev] [PATCH v4 01/14] eventdev: make driver interface " Hemant Agrawal
2021-10-18 23:35 ` [dpdk-dev] [PATCH v5 " pbhagavatula
2021-10-18 23:35 ` [dpdk-dev] [PATCH v5 02/14] eventdev: separate internal structures pbhagavatula
2021-10-18 23:35 ` [dpdk-dev] [PATCH v5 03/14] eventdev: allocate max space for internal arrays pbhagavatula
2021-10-18 23:35 ` [dpdk-dev] [PATCH v5 04/14] eventdev: move inline APIs into separate structure pbhagavatula
2021-10-18 23:36 ` [dpdk-dev] [PATCH v5 05/14] drivers/event: invoke probing finish function pbhagavatula
2021-10-18 23:36 ` [dpdk-dev] [PATCH v5 06/14] eventdev: use new API for inline functions pbhagavatula
2021-10-18 23:36 ` [dpdk-dev] [PATCH v5 07/14] eventdev: hide event device related structures pbhagavatula
2021-10-18 23:36 ` [dpdk-dev] [PATCH v5 08/14] eventdev: hide timer adapter PMD file pbhagavatula
2021-10-18 23:36 ` [dpdk-dev] [PATCH v5 09/14] eventdev: remove rte prefix for internal structs pbhagavatula
2021-10-18 23:36 ` [dpdk-dev] [PATCH v5 10/14] eventdev: rearrange fields in timer object pbhagavatula
2021-10-18 23:36 ` [dpdk-dev] [PATCH v5 11/14] eventdev: move timer adapters memory to hugepage pbhagavatula
2021-10-20 20:24 ` Carrillo, Erik G
2021-10-18 23:36 ` [dpdk-dev] [PATCH v5 12/14] eventdev: promote event vector API to stable pbhagavatula
2021-10-18 23:36 ` [dpdk-dev] [PATCH v5 13/14] eventdev: make trace APIs internal pbhagavatula
2021-10-18 23:36 ` [dpdk-dev] [PATCH v5 14/14] eventdev: mark trace variables as internal pbhagavatula
2021-10-20 4:01 ` [dpdk-dev] [PATCH v5 01/14] eventdev: make driver interface " Jerin Jacob
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=SA2PR11MB519671F2DAD4C0EFAABEB203B9B29@SA2PR11MB5196.namprd11.prod.outlook.com \
--to=erik.g.carrillo@intel.com \
--cc=dev@dpdk.org \
--cc=jerinj@marvell.com \
--cc=pbhagavatula@marvell.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).