From: Hideyuki Yamashita <yamashita.hideyuki@po.ntt-tx.co.jp>
To: "Burakov, Anatoly" <anatoly.burakov@intel.com>
Cc: dev@dpdk.org, Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>,
Nakamura Hioryuki <nakamura.hiroyuki@po.ntt-tx.co.jp>
Subject: Re: [dpdk-dev] rte_eal_hotplug_remove() generates error message
Date: Mon, 17 Dec 2018 21:10:27 +0900 [thread overview]
Message-ID: <201812171211.wBHCBgf1001687@ccmail04.silk.ntt-tx.co.jp> (raw)
In-Reply-To: <272166ee-d05f-528a-06c9-962e3237674e@intel.com>
> On 17-Dec-18 10:45 AM, Hideyuki Yamashita wrote:
> >> On 17-Dec-18 10:02 AM, Hideyuki Yamashita wrote:
> >>> Dear Thomas and all,
> >>>
> >>> I took a look on dpdk code.
> >>> I firstly write qustions and my analisys
> >>> on the current dpdk code follows after that.
> >>>
> >>> [1.Questions]
> >>> I have several questions to ask again.
> >>> Is my understanding correct about followings
> >>>
> >>> Q1: "EAL:ERROR, Invalid memory" is ignorable
> >>>
> >>> Q2: there is no big difference between calling
> >>> rte_eal_hotplug_remove(bus->name, dev->name)
> >>> and
> >>> rte_dev_remove(dev) because anyway it
> >>> reaches to rte_pmd_vhost_remove and encounter
> >>> the same error.
> >>>
> >>> [2.snip from my code]
> >>> .....
> >>> rte_eth_dev_close(port_id);
> >>> ret = rte_dev_remove(dev);
> >>> if (ret < 0)
> >>> return ret;
> >>> rte_eth_dev_release_port(&rte_eth_devices[port_id]);
> >>>
> >>> [3. My analysis on dpdk code]
> >>> static int
> >>> rte_pmd_vhost_remove(struct rte_vdev_device *dev)
> >>> {
> >>> ...........
> >>> eth_dev_close(eth_dev);
> >>>
> >>> rte_free(vring_states[eth_dev->data->port_id]);
> >>> vring_states[eth_dev->data->port_id] = NULL;
> >>>
> >>> rte_eth_dev_release_port(eth_dev);
> >>>
> >>> As you can see in rte_eth_vhost.c
> >>> It calls both eth_dev_close and rte_eth_dev_release_port.
> >>> And inside both functions, it tries to free mac_addrs.
> >>> rte_free(dev->data->mac_addrs); //in rth_dev_close
> >>> rte_free(eth_dev->data->mac_addrs); //in rte_eth_dev_release_port
> >>>
> >>> I understand that is the reason why
> >>> /* Free the memory space back to heap */
> >>> void rte_free(void *addr)
> >>> {
> >>> if (addr == NULL) return;
> >>> if (malloc_heap_free(malloc_elem_from_data(addr)) < 0)
> >>> RTE_LOG(ERR, EAL, "Error: Invalid memory\n");
> >>> }
> >>> encounter the error.
> >>> As an experiment, I commented out one of them, "ERR, Invalid memory"
> >>> disappered.
> >>>
> >>> Thanks and BR,
> >>> Hideyuki Yamashita
> >>> NTT TechnoCross
> >>>
> >>>> Adding my colleague Yasufumi and Hiroyuki as CC.
> >>>>
> >>>> We are waiting valuable advice from you.
> >>>>
> >>>> Thanks in advance,
> >>>> Hideyuki Yamashita
> >>>> NTT TechnoCross
> >>>>
> >>>>>
> >>>>> Dear Thomas and all,
> >>>>>
> >>>>> I hope you all get safely back home after DPDK summit.
> >>>>> (When I get back Japan, it is chilling. (start of winter))
> >>>>>
> >>>>> On DPDK 18.11.0, we tried to remove vhost device by using rte_eal_hotplug_remove().
> >>>>> However, following syslog message is printed.
> >>>>> “EAL: Error: Invalid memory”
> >>>>>
> >>>>> At DPDK summit San Jose, we had chance to ask Thomas how to handle the error message, and he gave us following advice:
> >>>>> Replace “rte_eal_hotplug_add()” to “rte_dev_probe(devargs)” and
> >>>>> “rte_eal_hotplug_remove()” to “rte_eth_dev_close() and rte_dev_remove(rte_dev)”
> >>>>>
> >>>>> We tested above changes, but the result is the same (i.e., same error message is printed).
> >>>>> The debug log message says:
> >>>>> ---
> >>>>> [primary]
> >>>>> VHOST_CONFIG: vhost-user server: socket created, fd: 17
> >>>>> VHOST_CONFIG: bind to /tmp/sock0
> >>>>> EAL: Error: Invalid memory
> >>>>> VHOST_CONFIG: vhost-user server: socket created, fd: 17
> >>>>> VHOST_CONFIG: bind to /tmp/sock0
> >>>>>
> >>>>> [secondary]
> >>>>> APP: devargs=eth_vhost0,iface=/tmp/sock0,queues=1
> >>>>> EAL: request: eal_dev_mp_request
> >>>>> EAL: msg: eal_dev_mp_request
> >>>>> EAL: request: bus_vdev_mp
> >>>>> EAL: msg: bus_vdev_mp
> >>>>> EAL: msg: bus_vdev_mp
> >>>>> EAL: reply: eal_dev_mp_request
> >>>>> EAL: msg: eal_dev_mp_request
> >>>>> rte_eth_promiscuous_disable: Function not supported
> >>>>> rte_eth_allmulticast_disable: Function not supported
> >>>>> APP: To Server: add
> >>>>> EAL: request: eal_dev_mp_request
> >>>>> EAL: msg: eal_dev_mp_request
> >>>>> EAL: reply: eal_dev_mp_request
> >>>>> EAL: msg: eal_dev_mp_request
> >>>>> APP: To Server: del
> >>>>> APP: devargs=eth_vhost0,iface=/tmp/sock0,queues=1
> >>>>> EAL: request: eal_dev_mp_request
> >>>>> EAL: msg: eal_dev_mp_request
> >>>>> EAL: request: bus_vdev_mp
> >>>>> EAL: msg: bus_vdev_mp
> >>>>> EAL: msg: bus_vdev_mp
> >>>>> EAL: reply: eal_dev_mp_request
> >>>>> EAL: msg: eal_dev_mp_request
> >>>>> rte_eth_promiscuous_disable: Function not supported
> >>>>> rte_eth_allmulticast_disable: Function not supported
> >>>>> APP: To Server: add
> >>>>> ---
> >>>>>
> >>>>> We would like to ask:
> >>>>> 1) Is the message “EAL: Error: Invalid memory” ignorable or not? There is no obstacle except this message to re-add the vhost device.
> >>>>> 2) Which is the better(best?) way to add/del vhost device “rte_eal_hotplug_add/remove()” or the way Thomas suggested?
> >>>>>
> >>>>> Thanks in advance and have a nice day.
> >>>>>
> >>>>> BR,
> >>>>> Hideyuki Yamashita
> >>>>> NTT TechnoCross
> >>>>
> >>>
> >>>
> >>>
> >> Hi Hideyuki,
> >>
> >> The error you're referring to (about invalid memory) means that you're trying to free a pointer that points to invalid memory. Meaning, either the pointer itself is not pointing to an allocated area, or it points to memory that has already been freed.
> >>
> >> If dev->data->mac_addrs and eth_dev->data->mac_addrs point to the same area, this is a bug, because this would lead to double free, and rte_malloc will rightly complain about invalid memory. Now, malloc won't try to do anything with the invalid memory, so the error itself is harmless *as far as malloc is concerned*, but these attempts to free the memory twice should be fixed whereever they happen.
> >>
> >> I'm not well-versed in dev infrastructure, so i wouldn't be able to say which one of the rte_free calls is an extra, unneeded one. This is something e.g. Thomas could help with, or the driver maintainer.
> >>
> >> --Thanks,
> >> Anatoly
> > Hello Anatoly,
> >
> > Thanks for your reply for my newbie question.
> > Now I understand that this error is harmless from DPDK application(SPP)
> > point of view in practice. Thanks.
> > But anyway if there is a double free logic, it is a bug and should be
> > fixed.
> > The remaining issues are
> > 1. If it is really a bug (or my mis-understanding)
> > 2. If is is a bug which function should remove rte_free(mac_addrs)
>
> From description, it looks like a bug. Correct usage of API (rte_dev_close() followed by rte_dev_remove()) should not trigger any errors. You might want to create a BugZilla entry describing the issue.
>
> >
> > Thanks,
> > Hideyuki Yamashita
> > NTT TechnoCross
> >
> >
>
> --Thanks,
> Anatoly
Hello
Thanks for your reply again.
I would like to wait for a while(at least a day) to hear
opnion from other people including Thomas or maintainer.
In addition I have to learn how to file bug in Bugzzilla
for DPDK.
Thanks,
Hideyuki Yamashita
NTT TechnoCross
next prev parent reply other threads:[~2018-12-17 12:12 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-09 4:30 [dpdk-dev] dpdk vlan strip offload bug for I350 nic? 1534898891
2018-12-09 5:23 ` =?gb18030?B?MTUzNDg5ODg5MQ==?=
2018-12-18 2:26 ` Zhao1, Wei
2018-12-10 7:53 ` [dpdk-dev] rte_eal_hotplug_remove() generates error message Hideyuki Yamashita
2018-12-11 0:54 ` Hideyuki Yamashita
2018-12-17 10:02 ` Hideyuki Yamashita
2018-12-17 10:23 ` Burakov, Anatoly
2018-12-17 10:38 ` Burakov, Anatoly
2018-12-17 11:03 ` Hideyuki Yamashita
2018-12-17 10:45 ` Hideyuki Yamashita
2018-12-17 11:21 ` Burakov, Anatoly
2018-12-17 12:10 ` Hideyuki Yamashita [this message]
2018-12-18 2:39 ` Tiwei Bie
2018-12-18 3:11 ` Hideyuki Yamashita
2018-12-18 5:12 ` Tiwei Bie
2018-12-18 5:52 ` Hideyuki Yamashita
2018-12-18 6:25 ` Tiwei Bie
2018-12-18 11:30 ` Hideyuki Yamashita
2018-12-18 13:09 ` Tiwei Bie
2018-12-27 5:52 ` Hideyuki Yamashita
2019-01-02 9:19 ` Tiwei Bie
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=201812171211.wBHCBgf1001687@ccmail04.silk.ntt-tx.co.jp \
--to=yamashita.hideyuki@po.ntt-tx.co.jp \
--cc=anatoly.burakov@intel.com \
--cc=dev@dpdk.org \
--cc=nakamura.hiroyuki@po.ntt-tx.co.jp \
--cc=ogawa.yasufumi@lab.ntt.co.jp \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).