From: Harman Kalra <hkalra@marvell.com> To: Harman Kalra <hkalra@marvell.com>, Thomas Monjalon <thomas@monjalon.net> Cc: Raslan Darawsheh <rasland@nvidia.com>, "dev@dpdk.org" <dev@dpdk.org>, Ray Kinsella <mdr@ashroe.eu>, Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>, David Marchand <david.marchand@redhat.com>, "viacheslavo@nvidia.com" <viacheslavo@nvidia.com>, "matan@nvidia.com" <matan@nvidia.com> Subject: Re: [dpdk-dev] [EXT] Re: [PATCH v1 2/7] eal/interrupts: implement get set APIs Date: Wed, 13 Oct 2021 17:57:01 +0000 Message-ID: <BN9PR18MB4204A56AE0FB98928883EAEBC5B79@BN9PR18MB4204.namprd18.prod.outlook.com> (raw) In-Reply-To: <BN9PR18MB4204EF6156C920FBA23CEC3DC5B79@BN9PR18MB4204.namprd18.prod.outlook.com> > -----Original Message----- > From: dev <dev-bounces@dpdk.org> On Behalf Of Harman Kalra > Sent: Wednesday, October 13, 2021 11:24 PM > To: Thomas Monjalon <thomas@monjalon.net> > Cc: Raslan Darawsheh <rasland@nvidia.com>; dev@dpdk.org; Ray Kinsella > <mdr@ashroe.eu>; Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>; David > Marchand <david.marchand@redhat.com>; viacheslavo@nvidia.com; > matan@nvidia.com > Subject: Re: [dpdk-dev] [EXT] Re: [PATCH v1 2/7] eal/interrupts: implement > get set APIs > > > > > -----Original Message----- > > From: Thomas Monjalon <thomas@monjalon.net> > > Sent: Tuesday, October 12, 2021 8:53 PM > > To: Harman Kalra <hkalra@marvell.com> > > Cc: Raslan Darawsheh <rasland@nvidia.com>; dev@dpdk.org; Ray Kinsella > > <mdr@ashroe.eu>; Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>; David > > Marchand <david.marchand@redhat.com>; viacheslavo@nvidia.com; > > matan@nvidia.com > > Subject: Re: [dpdk-dev] [EXT] Re: [PATCH v1 2/7] eal/interrupts: > > implement get set APIs > > > > 04/10/2021 11:57, David Marchand: > > > On Mon, Oct 4, 2021 at 10:51 AM Harman Kalra <hkalra@marvell.com> > > wrote: > > > > > > +struct rte_intr_handle *rte_intr_handle_instance_alloc(int size, > > > > > > + bool > > > > > > +from_hugepage) { > > > > > > + struct rte_intr_handle *intr_handle; > > > > > > + int i; > > > > > > + > > > > > > + if (from_hugepage) > > > > > > + intr_handle = rte_zmalloc(NULL, > > > > > > + size * sizeof(struct rte_intr_handle), > > > > > > + 0); > > > > > > + else > > > > > > + intr_handle = calloc(1, size * sizeof(struct > > > > > > + rte_intr_handle)); > > > > > > > > > > We can call DPDK allocator in all cases. > > > > > That would avoid headaches on why multiprocess does not work in > > > > > some rarely tested cases. > > > > > Wdyt? > > > > > > > > > > Plus "from_hugepage" is misleading, you could be in --no-huge > > > > > mode, rte_zmalloc still works. > > > > > > > > <HK> In mellanox 5 driver interrupt handle instance is freed in > > > > destructor " mlx5_pmd_interrupt_handler_uninstall()" while DPDK > > > > memory allocators are already cleaned up in "rte_eal_cleanup". > > > > Hence I allocated interrupt instances for such cases from normal heap. > > > > There could be other such cases so I think its ok to keep this support. > > > > > > This is surprising. > > > Why would the mlx5 driver wait to release in a destructor? > > > It should be done once no interrupt handler is necessary (like when > > > stopping all ports), and that would be before rte_eal_cleanup(). > > > > I agree with David. > > I prefer a simpler API which always use rte_malloc, and make sure > > interrupts are always handled between rte_eal_init and rte_eal_cleanup. > > The mlx5 PMD could be reworked to match this requirement. > > In any case we should not any memory management in > > constructors/destructors. > > > Hi Thomas, David > > There are couple of more dependencies on glibc heap APIs: > 1. "rte_eal_alarm_init()" allocates an interrupt instance which is used for > timerfd, is called before "rte_eal_memory_init()" which does the memseg > init. > Not sure what all challenges we may face in moving alarm_init after > memory_init as it might break some subsystem inits. > Other option could be to allocate interrupt instance for timerfd on first > alarm_setup call. > > 2. Currently interrupt handle field inside struct rte_pci_device is static which > is changed to a pointer in this series(as struct rte_intr_handle is hidden inside > a c file and size is unknown outside). > I am allocating the memory for this interrupt instance inside > "rte_pci_probe_one_driver()" just before "pci_vfio_map_resource()" which > sets up vfio resources and calls " pci_vfio_setup_interrupts()" which setups > the interrupt support. Here challenge is "rte_bus_probe()" also gets called > before "rte_eal_memory_init()". Sorry my bad, bus probing happens after "rte_eal_memory_init()", so second point Is invalid. > > There are many other drivers which statically declares the interrupt handles > inside their respective private structures and memory for those structure > was allocated from heap. For such drivers I allocated interrupt instances also > using glibc heap APIs. > > Thanks > Harman > > > > >
next prev parent reply other threads:[~2021-10-13 17:57 UTC|newest] Thread overview: 152+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-08-26 14:57 [dpdk-dev] [RFC 0/7] make rte_intr_handle internal Harman Kalra 2021-08-26 14:57 ` [dpdk-dev] [RFC 1/7] eal: interrupt handle API prototypes Harman Kalra 2021-08-31 15:52 ` Kinsella, Ray 2021-08-26 14:57 ` [dpdk-dev] [RFC 2/7] eal/interrupts: implement get set APIs Harman Kalra 2021-08-31 15:53 ` Kinsella, Ray 2021-08-26 14:57 ` [dpdk-dev] [RFC 3/7] eal/interrupts: avoid direct access to interrupt handle Harman Kalra 2021-08-26 14:57 ` [dpdk-dev] [RFC 4/7] test/interrupt: apply get set interrupt handle APIs Harman Kalra 2021-08-26 14:57 ` [dpdk-dev] [RFC 5/7] drivers: remove direct access to interrupt handle fields Harman Kalra 2021-08-26 14:57 ` [dpdk-dev] [RFC 6/7] eal/interrupts: make interrupt handle structure opaque Harman Kalra 2021-08-26 14:57 ` [dpdk-dev] [RFC 7/7] eal/alarm: introduce alarm fini routine Harman Kalra 2021-09-03 12:40 ` [dpdk-dev] [PATCH v1 0/7] make rte_intr_handle internal Harman Kalra 2021-09-03 12:40 ` [dpdk-dev] [PATCH v1 1/7] eal: interrupt handle API prototypes Harman Kalra 2021-09-03 12:40 ` [dpdk-dev] [PATCH v1 2/7] eal/interrupts: implement get set APIs Harman Kalra 2021-09-28 15:46 ` David Marchand 2021-10-04 8:51 ` [dpdk-dev] [EXT] " Harman Kalra 2021-10-04 9:57 ` David Marchand 2021-10-12 15:22 ` Thomas Monjalon 2021-10-13 17:54 ` Harman Kalra 2021-10-13 17:57 ` Harman Kalra [this message] 2021-10-13 18:52 ` Thomas Monjalon 2021-10-14 8:22 ` Thomas Monjalon 2021-10-14 9:31 ` Harman Kalra 2021-10-14 9:37 ` David Marchand 2021-10-14 9:41 ` Thomas Monjalon 2021-10-14 10:31 ` Harman Kalra 2021-10-14 10:35 ` Thomas Monjalon 2021-10-14 10:44 ` Harman Kalra 2021-10-14 12:04 ` Thomas Monjalon 2021-10-14 10:25 ` Dmitry Kozlyuk 2021-10-03 18:05 ` [dpdk-dev] " Dmitry Kozlyuk 2021-10-04 10:37 ` [dpdk-dev] [EXT] " Harman Kalra 2021-10-04 11:18 ` Dmitry Kozlyuk 2021-10-04 14:03 ` Harman Kalra 2021-09-03 12:40 ` [dpdk-dev] [PATCH v1 3/7] eal/interrupts: avoid direct access to interrupt handle Harman Kalra 2021-09-03 12:40 ` [dpdk-dev] [PATCH v1 4/7] test/interrupt: apply get set interrupt handle APIs Harman Kalra 2021-09-03 12:41 ` [dpdk-dev] [PATCH v1 5/7] drivers: remove direct access to interrupt handle fields Harman Kalra 2021-09-03 12:41 ` [dpdk-dev] [PATCH v1 6/7] eal/interrupts: make interrupt handle structure opaque Harman Kalra 2021-10-03 18:16 ` Dmitry Kozlyuk 2021-10-04 14:09 ` [dpdk-dev] [EXT] " Harman Kalra 2021-09-03 12:41 ` [dpdk-dev] [PATCH v1 7/7] eal/alarm: introduce alarm fini routine Harman Kalra 2021-09-15 14:13 ` [dpdk-dev] [PATCH v1 0/7] make rte_intr_handle internal Harman Kalra 2021-09-23 8:20 ` David Marchand 2021-10-05 12:14 ` [dpdk-dev] [PATCH v2 0/6] " Harman Kalra 2021-10-05 12:14 ` [dpdk-dev] [PATCH v2 1/6] eal/interrupts: implement get set APIs Harman Kalra 2021-10-14 0:58 ` Dmitry Kozlyuk 2021-10-14 17:15 ` [dpdk-dev] [EXT] " Harman Kalra 2021-10-14 17:53 ` Dmitry Kozlyuk 2021-10-15 7:53 ` Thomas Monjalon 2021-10-14 7:31 ` [dpdk-dev] " David Marchand 2021-10-05 12:14 ` [dpdk-dev] [PATCH v2 2/6] eal/interrupts: avoid direct access to interrupt handle Harman Kalra 2021-10-14 0:59 ` Dmitry Kozlyuk 2021-10-14 17:31 ` [dpdk-dev] [EXT] " Harman Kalra 2021-10-14 17:53 ` Dmitry Kozlyuk 2021-10-05 12:14 ` [dpdk-dev] [PATCH v2 3/6] test/interrupt: apply get set interrupt handle APIs Harman Kalra 2021-10-05 12:15 ` [dpdk-dev] [PATCH v2 4/6] drivers: remove direct access to interrupt handle Harman Kalra 2021-10-05 12:15 ` [dpdk-dev] [PATCH v2 5/6] eal/interrupts: make interrupt handle structure opaque Harman Kalra 2021-10-05 12:15 ` [dpdk-dev] [PATCH v2 6/6] eal/alarm: introduce alarm fini routine Harman Kalra 2021-10-05 16:07 ` [dpdk-dev] [RFC 0/7] make rte_intr_handle internal Stephen Hemminger 2021-10-07 10:57 ` [dpdk-dev] [EXT] " Harman Kalra 2021-10-18 19:37 ` [dpdk-dev] [PATCH v3 " Harman Kalra 2021-10-18 19:37 ` [dpdk-dev] [PATCH v3 1/7] malloc: introduce malloc is ready API Harman Kalra 2021-10-19 15:53 ` Thomas Monjalon 2021-10-18 19:37 ` [dpdk-dev] [PATCH v3 2/7] eal/interrupts: implement get set APIs Harman Kalra 2021-10-18 22:07 ` Dmitry Kozlyuk 2021-10-19 8:50 ` [dpdk-dev] [EXT] " Harman Kalra 2021-10-19 18:44 ` Harman Kalra 2021-10-18 22:56 ` [dpdk-dev] " Stephen Hemminger 2021-10-19 8:32 ` [dpdk-dev] [EXT] " Harman Kalra 2021-10-19 15:58 ` Thomas Monjalon 2021-10-20 15:30 ` Dmitry Kozlyuk 2021-10-21 9:16 ` Harman Kalra 2021-10-21 12:33 ` Dmitry Kozlyuk 2021-10-21 13:32 ` David Marchand 2021-10-21 16:05 ` Harman Kalra 2021-10-18 19:37 ` [dpdk-dev] [PATCH v3 3/7] eal/interrupts: avoid direct access to interrupt handle Harman Kalra 2021-10-18 19:37 ` [dpdk-dev] [PATCH v3 4/7] test/interrupt: apply get set interrupt handle APIs Harman Kalra 2021-10-18 19:37 ` [dpdk-dev] [PATCH v3 5/7] drivers: remove direct access to interrupt handle Harman Kalra 2021-10-18 19:37 ` [dpdk-dev] [PATCH v3 6/7] eal/interrupts: make interrupt handle structure opaque Harman Kalra 2021-10-18 19:37 ` [dpdk-dev] [PATCH v3 7/7] eal/alarm: introduce alarm fini routine Harman Kalra 2021-10-19 18:35 ` [dpdk-dev] [PATCH v4 0/7] make rte_intr_handle internal Harman Kalra 2021-10-19 18:35 ` [dpdk-dev] [PATCH v4 1/7] malloc: introduce malloc is ready API Harman Kalra 2021-10-19 22:01 ` Dmitry Kozlyuk 2021-10-19 22:04 ` Dmitry Kozlyuk 2021-10-20 9:01 ` [dpdk-dev] [EXT] " Harman Kalra 2021-10-19 18:35 ` [dpdk-dev] [PATCH v4 2/7] eal/interrupts: implement get set APIs Harman Kalra 2021-10-20 6:14 ` David Marchand 2021-10-20 14:29 ` Dmitry Kozlyuk 2021-10-20 16:15 ` Dmitry Kozlyuk 2021-10-19 18:35 ` [dpdk-dev] [PATCH v4 3/7] eal/interrupts: avoid direct access to interrupt handle Harman Kalra 2021-10-19 21:27 ` Dmitry Kozlyuk 2021-10-20 9:25 ` [dpdk-dev] [EXT] " Harman Kalra 2021-10-20 9:52 ` Dmitry Kozlyuk 2021-10-19 18:35 ` [dpdk-dev] [PATCH v4 4/7] test/interrupt: apply get set interrupt handle APIs Harman Kalra 2021-10-19 18:35 ` [dpdk-dev] [PATCH v4 5/7] drivers: remove direct access to interrupt handle Harman Kalra 2021-10-20 1:57 ` Hyong Youb Kim (hyonkim) 2021-10-19 18:35 ` [dpdk-dev] [PATCH v4 6/7] eal/interrupts: make interrupt handle structure opaque Harman Kalra 2021-10-19 18:35 ` [dpdk-dev] [PATCH v4 7/7] eal/alarm: introduce alarm fini routine Harman Kalra 2021-10-19 21:39 ` Dmitry Kozlyuk 2021-10-22 20:49 ` [dpdk-dev] [PATCH v5 0/6] make rte_intr_handle internal Harman Kalra 2021-10-22 20:49 ` [dpdk-dev] [PATCH v5 1/6] eal/interrupts: implement get set APIs Harman Kalra 2021-10-22 23:33 ` Dmitry Kozlyuk 2021-10-22 20:49 ` [dpdk-dev] [PATCH v5 2/6] eal/interrupts: avoid direct access to interrupt handle Harman Kalra 2021-10-22 23:33 ` Dmitry Kozlyuk 2021-10-22 20:49 ` [dpdk-dev] [PATCH v5 3/6] test/interrupt: apply get set interrupt handle APIs Harman Kalra 2021-10-22 20:49 ` [dpdk-dev] [PATCH v5 4/6] drivers: remove direct access to interrupt handle Harman Kalra 2021-10-22 20:49 ` [dpdk-dev] [PATCH v5 5/6] eal/interrupts: make interrupt handle structure opaque Harman Kalra 2021-10-22 23:33 ` Dmitry Kozlyuk 2021-10-22 20:49 ` [dpdk-dev] [PATCH v5 6/6] eal/alarm: introduce alarm fini routine Harman Kalra 2021-10-22 23:33 ` Dmitry Kozlyuk 2021-10-22 23:37 ` Dmitry Kozlyuk 2021-10-24 20:04 ` [dpdk-dev] [PATCH v6 0/9] make rte_intr_handle internal David Marchand 2021-10-24 20:04 ` [dpdk-dev] [PATCH v6 1/9] interrupts: add allocator and accessors David Marchand 2021-10-24 20:04 ` [dpdk-dev] [PATCH v6 2/9] interrupts: remove direct access to interrupt handle David Marchand 2021-10-25 6:57 ` David Marchand 2021-10-24 20:04 ` [dpdk-dev] [PATCH v6 3/9] test/interrupts: " David Marchand 2021-10-24 20:04 ` [dpdk-dev] [PATCH v6 4/9] alarm: " David Marchand 2021-10-25 10:49 ` Dmitry Kozlyuk 2021-10-25 11:09 ` David Marchand 2021-10-24 20:04 ` [dpdk-dev] [PATCH v6 5/9] lib: " David Marchand 2021-10-24 20:04 ` [dpdk-dev] [PATCH v6 6/9] drivers: " David Marchand 2021-10-24 20:04 ` [dpdk-dev] [PATCH v6 7/9] interrupts: make interrupt handle structure opaque David Marchand 2021-10-24 20:04 ` [dpdk-dev] [PATCH v6 8/9] interrupts: rename device specific file descriptor David Marchand 2021-10-24 20:04 ` [dpdk-dev] [PATCH v6 9/9] interrupts: extend event list David Marchand 2021-10-25 10:49 ` Dmitry Kozlyuk 2021-10-25 11:11 ` David Marchand 2021-10-25 13:04 ` [dpdk-dev] [PATCH v5 0/6] make rte_intr_handle internal Raslan Darawsheh 2021-10-25 13:09 ` David Marchand 2021-10-25 13:34 ` [dpdk-dev] [PATCH v7 0/9] " David Marchand 2021-10-25 13:34 ` [dpdk-dev] [PATCH v7 1/9] interrupts: add allocator and accessors David Marchand 2021-10-25 13:34 ` [dpdk-dev] [PATCH v7 2/9] interrupts: remove direct access to interrupt handle David Marchand 2021-10-25 13:34 ` [dpdk-dev] [PATCH v7 3/9] test/interrupts: " David Marchand 2021-10-25 13:34 ` [dpdk-dev] [PATCH v7 4/9] alarm: " David Marchand 2021-10-25 13:34 ` [dpdk-dev] [PATCH v7 5/9] lib: " David Marchand 2021-10-28 6:14 ` Jiang, YuX 2021-10-25 13:34 ` [dpdk-dev] [PATCH v7 6/9] drivers: " David Marchand 2021-10-25 13:34 ` [dpdk-dev] [PATCH v7 7/9] interrupts: make interrupt handle structure opaque David Marchand 2021-10-25 13:34 ` [dpdk-dev] [PATCH v7 8/9] interrupts: rename device specific file descriptor David Marchand 2021-10-25 13:34 ` [dpdk-dev] [PATCH v7 9/9] interrupts: extend event list David Marchand 2021-10-25 14:27 ` [dpdk-dev] [PATCH v8 0/9] make rte_intr_handle internal David Marchand 2021-10-25 14:27 ` [dpdk-dev] [PATCH v8 1/9] interrupts: add allocator and accessors David Marchand 2021-10-25 14:27 ` [dpdk-dev] [PATCH v8 2/9] interrupts: remove direct access to interrupt handle David Marchand 2021-10-25 14:27 ` [dpdk-dev] [PATCH v8 3/9] test/interrupts: " David Marchand 2021-10-25 14:27 ` [dpdk-dev] [PATCH v8 4/9] alarm: " David Marchand 2021-10-25 14:27 ` [dpdk-dev] [PATCH v8 5/9] lib: " David Marchand 2021-10-25 14:27 ` [dpdk-dev] [PATCH v8 6/9] drivers: " David Marchand 2021-10-25 14:27 ` [dpdk-dev] [PATCH v8 7/9] interrupts: make interrupt handle structure opaque David Marchand 2021-10-25 14:27 ` [dpdk-dev] [PATCH v8 8/9] interrupts: rename device specific file descriptor David Marchand 2021-10-25 14:27 ` [dpdk-dev] [PATCH v8 9/9] interrupts: extend event list David Marchand 2021-10-28 15:58 ` Ji, Kai 2021-10-28 17:16 ` David Marchand 2021-10-25 14:32 ` [dpdk-dev] [PATCH v8 0/9] make rte_intr_handle internal Raslan Darawsheh 2021-10-25 19:24 ` David Marchand
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=BN9PR18MB4204A56AE0FB98928883EAEBC5B79@BN9PR18MB4204.namprd18.prod.outlook.com \ --to=hkalra@marvell.com \ --cc=david.marchand@redhat.com \ --cc=dev@dpdk.org \ --cc=dmitry.kozliuk@gmail.com \ --cc=matan@nvidia.com \ --cc=mdr@ashroe.eu \ --cc=rasland@nvidia.com \ --cc=thomas@monjalon.net \ --cc=viacheslavo@nvidia.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
DPDK patches and discussions This inbox may be cloned and mirrored by anyone: git clone --mirror http://inbox.dpdk.org/dev/0 dev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 dev dev/ http://inbox.dpdk.org/dev \ dev@dpdk.org public-inbox-index dev Example config snippet for mirrors. Newsgroup available over NNTP: nntp://inbox.dpdk.org/inbox.dpdk.dev AGPL code for this site: git clone https://public-inbox.org/public-inbox.git