From: Rich Lane <rich.lane@bigswitch.com>
To: Tetsuya Mukawa <mukawa@igel.co.jp>
Cc: dev@dpdk.org, ann.zhuangyanying@huawei.com
Subject: Re: [dpdk-dev] [PATCH v5 1/3] vhost: Add callback and private data for vhost PMD
Date: Wed, 23 Dec 2015 21:37:29 -0800 [thread overview]
Message-ID: <CAGSMBPMeHByvncdZWbi0mCc2B1CczH005jKQvewe9fwNMFD_uQ@mail.gmail.com> (raw)
In-Reply-To: <567B61D6.1090806@igel.co.jp>
On Wed, Dec 23, 2015 at 7:09 PM, Tetsuya Mukawa <mukawa@igel.co.jp> wrote:
> On 2015/12/22 13:47, Rich Lane wrote:
> > On Mon, Dec 21, 2015 at 7:41 PM, Yuanhan Liu <
> yuanhan.liu@linux.intel.com>
> > wrote:
> >
> >> On Fri, Dec 18, 2015 at 10:01:25AM -0800, Rich Lane wrote:
> >>> I'm using the vhost callbacks and struct virtio_net with the vhost PMD
> >> in a few
> >>> ways:
> >> Rich, thanks for the info!
> >>
> >>> 1. new_device/destroy_device: Link state change (will be covered by the
> >> link
> >>> status interrupt).
> >>> 2. new_device: Add first queue to datapath.
> >> I'm wondering why vring_state_changed() is not used, as it will also be
> >> triggered at the beginning, when the default queue (the first queue) is
> >> enabled.
> >>
> > Turns out I'd misread the code and it's already using the
> > vring_state_changed callback for the
> > first queue. Not sure if this is intentional but vring_state_changed is
> > called for the first queue
> > before new_device.
> >
> >
> >>> 3. vring_state_changed: Add/remove queue to datapath.
> >>> 4. destroy_device: Remove all queues (vring_state_changed is not called
> >> when
> >>> qemu is killed).
> >> I had a plan to invoke vring_state_changed() to disable all vrings
> >> when destroy_device() is called.
> >>
> > That would be good.
> >
> >
> >>> 5. new_device and struct virtio_net: Determine NUMA node of the VM.
> >> You can get the 'struct virtio_net' dev from all above callbacks.
> >
> >
> >> 1. Link status interrupt.
> >>
> >> To vhost pmd, new_device()/destroy_device() equals to the link status
> >> interrupt, where new_device() is a link up, and destroy_device() is link
> >> down().
> >>
> >>
> >>> 2. New queue_state_changed callback. Unlike vring_state_changed this
> >> should
> >>> cover the first queue at new_device and removal of all queues at
> >>> destroy_device.
> >> As stated above, vring_state_changed() should be able to do that, except
> >> the one on destroy_device(), which is not done yet.
> >>
> >>> 3. Per-queue or per-device NUMA node info.
> >> You can query the NUMA node info implicitly by get_mempolicy(); check
> >> numa_realloc() at lib/librte_vhost/virtio-net.c for reference.
> >>
> > Your suggestions are exactly how my application is already working. I was
> > commenting on the
> > proposed changes to the vhost PMD API. I would prefer to
> > use RTE_ETH_EVENT_INTR_LSC
> > and rte_eth_dev_socket_id for consistency with other NIC drivers, instead
> > of these vhost-specific
> > hacks. The queue state change callback is the one new API that needs to
> be
> > added because
> > normal NICs don't have this behavior.
> >
> > You could add another rte_eth_event_type for the queue state change
> > callback, and pass the
> > queue ID, RX/TX direction, and enable bit through cb_arg.
>
> Hi Rich,
>
> So far, EAL provides rte_eth_dev_callback_register() for event handling.
> DPDK app can register callback handler and "callback argument".
> And EAL will call callback handler with the argument.
> Anyway, vhost library and PMD cannot change the argument.
>
You're right, I'd mistakenly thought that the PMD controlled the void *
passed to the callback.
Here's a thought:
struct rte_eth_vhost_queue_event {
uint16_t queue_id;
bool rx;
bool enable;
};
int rte_eth_vhost_get_queue_event(uint8_t port_id, struct
rte_eth_vhost_queue_event *event);
On receiving the ethdev event the application could repeatedly call
rte_eth_vhost_get_queue_event
to find out what happened.
An issue with having the application dig into struct virtio_net is that it
can only be safely accessed from
a callback on the vhost thread. A typical application running its control
plane on lcore 0 would need to
copy all the relevant info from struct virtio_net before sending it over.
As you mentioned, queues for a single vhost port could be located on
different NUMA nodes. I think this
is an uncommon scenario but if needed you could add an API to retrieve the
NUMA node for a given port
and queue.
next prev parent reply other threads:[~2015-12-24 5:37 UTC|newest]
Thread overview: 199+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-08-31 3:55 [dpdk-dev] [RFC PATCH v2] Add VHOST PMD Tetsuya Mukawa
2015-08-31 3:55 ` [dpdk-dev] [RFC PATCH v2] vhost: " Tetsuya Mukawa
2015-09-23 17:47 ` Loftus, Ciara
2015-10-16 8:40 ` Tetsuya Mukawa
2015-10-20 14:13 ` Loftus, Ciara
2015-10-21 4:30 ` Tetsuya Mukawa
2015-10-21 10:09 ` Bruce Richardson
2015-10-16 12:52 ` Bruce Richardson
2015-10-19 1:51 ` Tetsuya Mukawa
2015-10-19 9:32 ` Loftus, Ciara
2015-10-19 9:45 ` Bruce Richardson
2015-10-19 10:50 ` Tetsuya Mukawa
2015-10-19 13:26 ` Panu Matilainen
2015-10-19 13:27 ` Richardson, Bruce
2015-10-21 4:35 ` Tetsuya Mukawa
2015-10-21 6:25 ` Panu Matilainen
2015-10-21 10:22 ` Bruce Richardson
2015-10-22 9:50 ` Tetsuya Mukawa
2015-10-27 13:44 ` Traynor, Kevin
2015-10-28 2:24 ` Tetsuya Mukawa
2015-10-22 9:45 ` [dpdk-dev] [RFC PATCH v3 0/2] " Tetsuya Mukawa
2015-10-22 9:45 ` [dpdk-dev] [RFC PATCH v3 1/2] vhost: Add callback and private data for vhost PMD Tetsuya Mukawa
2015-10-27 6:12 ` [dpdk-dev] [PATCH 0/3] Add VHOST PMD Tetsuya Mukawa
2015-10-27 6:12 ` [dpdk-dev] [PATCH 1/3] vhost: Fix wrong handling of virtqueue array index Tetsuya Mukawa
2015-10-27 6:29 ` Yuanhan Liu
2015-10-27 6:33 ` Yuanhan Liu
2015-10-27 6:47 ` Yuanhan Liu
2015-10-27 7:28 ` Tetsuya Mukawa
2015-10-27 7:34 ` Yuanhan Liu
2015-10-27 6:12 ` [dpdk-dev] [PATCH 2/3] vhost: Add callback and private data for vhost PMD Tetsuya Mukawa
2015-10-30 17:49 ` Loftus, Ciara
2015-11-02 3:15 ` Tetsuya Mukawa
2015-10-27 6:12 ` [dpdk-dev] [PATCH 3/3] vhost: Add VHOST PMD Tetsuya Mukawa
2015-11-02 3:58 ` [dpdk-dev] [PATCH v2 0/2] " Tetsuya Mukawa
2015-11-02 3:58 ` [dpdk-dev] [PATCH v2 1/2] vhost: Add callback and private data for vhost PMD Tetsuya Mukawa
2015-11-09 5:16 ` [dpdk-dev] [PATCH v3 0/2] Add VHOST PMD Tetsuya Mukawa
2015-11-09 5:17 ` [dpdk-dev] [PATCH v3 1/2] vhost: Add callback and private data for vhost PMD Tetsuya Mukawa
2015-11-09 18:16 ` Aaron Conole
2015-11-10 3:13 ` Tetsuya Mukawa
2015-11-10 7:16 ` Panu Matilainen
2015-11-10 9:48 ` Tetsuya Mukawa
2015-11-10 10:05 ` Panu Matilainen
2015-11-10 10:15 ` Tetsuya Mukawa
2015-11-09 5:17 ` [dpdk-dev] [PATCH v3 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2015-11-09 6:21 ` Yuanhan Liu
2015-11-09 6:27 ` Tetsuya Mukawa
2015-11-09 22:22 ` Stephen Hemminger
2015-11-10 3:14 ` Tetsuya Mukawa
2015-11-12 12:52 ` Wang, Zhihong
2015-11-13 3:09 ` Tetsuya Mukawa
2015-11-13 3:50 ` Wang, Zhihong
2015-11-13 4:03 ` Rich Lane
2015-11-13 4:29 ` Tetsuya Mukawa
2015-11-13 5:20 ` [dpdk-dev] [PATCH v4 0/2] " Tetsuya Mukawa
2015-11-13 5:20 ` [dpdk-dev] [PATCH v4 1/2] vhost: Add callback and private data for vhost PMD Tetsuya Mukawa
2015-11-17 13:29 ` Yuanhan Liu
2015-11-19 2:03 ` Tetsuya Mukawa
2015-11-19 2:18 ` Yuanhan Liu
2015-11-19 3:13 ` Tetsuya Mukawa
2015-11-19 3:33 ` Yuanhan Liu
2015-11-19 5:14 ` Tetsuya Mukawa
2015-11-19 5:45 ` Yuanhan Liu
2015-11-19 5:58 ` Tetsuya Mukawa
[not found] ` <20151119063137.GJ2326@yliu-dev.sh.intel.com>
2015-11-19 6:37 ` Tetsuya Mukawa
2015-11-13 5:20 ` [dpdk-dev] [PATCH v4 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2015-11-16 1:57 ` Wang, Zhihong
2015-11-20 11:43 ` Yuanhan Liu
2015-11-24 2:48 ` Tetsuya Mukawa
2015-11-24 3:40 ` Yuanhan Liu
2015-11-24 3:44 ` Tetsuya Mukawa
2015-11-21 0:15 ` Rich Lane
2015-11-24 4:41 ` Tetsuya Mukawa
2015-11-24 9:00 ` [dpdk-dev] [PATCH v5 0/3] " Tetsuya Mukawa
2015-11-24 9:00 ` [dpdk-dev] [PATCH v5 1/3] vhost: Add callback and private data for vhost PMD Tetsuya Mukawa
2015-12-17 11:42 ` Yuanhan Liu
2015-12-18 3:15 ` Tetsuya Mukawa
2015-12-18 3:36 ` Tetsuya Mukawa
2015-12-18 4:15 ` Yuanhan Liu
2015-12-18 4:28 ` Tetsuya Mukawa
2015-12-18 18:01 ` Rich Lane
2015-12-21 2:10 ` Tetsuya Mukawa
2015-12-22 4:36 ` Yuanhan Liu
2015-12-22 3:41 ` Yuanhan Liu
2015-12-22 4:47 ` Rich Lane
2015-12-22 5:47 ` Yuanhan Liu
2015-12-22 9:38 ` Rich Lane
2015-12-23 2:44 ` Yuanhan Liu
2015-12-23 22:00 ` Thomas Monjalon
2015-12-24 3:51 ` Yuanhan Liu
2015-12-24 4:07 ` Tetsuya Mukawa
2015-12-24 3:09 ` Tetsuya Mukawa
2015-12-24 3:54 ` Tetsuya Mukawa
2015-12-24 4:00 ` Yuanhan Liu
2015-12-24 4:23 ` Tetsuya Mukawa
2015-12-24 5:37 ` Rich Lane [this message]
2015-12-24 7:58 ` Tetsuya Mukawa
2015-12-28 21:59 ` Rich Lane
2016-01-06 3:56 ` Tetsuya Mukawa
2016-01-06 7:38 ` Yuanhan Liu
2015-12-18 10:03 ` Xie, Huawei
2015-12-21 2:10 ` Tetsuya Mukawa
2016-02-02 11:18 ` [dpdk-dev] [PATCH v6 0/2] Add VHOST PMD Tetsuya Mukawa
2016-02-02 19:52 ` Rich Lane
2016-02-02 11:18 ` [dpdk-dev] [PATCH v6 1/2] ethdev: Add a new event type to notify a queue state changed event Tetsuya Mukawa
2016-02-02 11:18 ` [dpdk-dev] [PATCH v6 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2016-02-02 23:43 ` Ferruh Yigit
2016-02-03 2:13 ` Tetsuya Mukawa
2016-02-03 7:48 ` Tetsuya Mukawa
2016-02-03 9:24 ` Ferruh Yigit
2016-02-03 9:35 ` Tetsuya Mukawa
2016-02-04 7:26 ` [dpdk-dev] [PATCH v7 0/2] " Tetsuya Mukawa
2016-02-04 7:26 ` [dpdk-dev] [PATCH v7 1/2] ethdev: Add a new event type to notify a queue state changed event Tetsuya Mukawa
2016-02-04 7:26 ` [dpdk-dev] [PATCH v7 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2016-02-04 11:17 ` Ferruh Yigit
2016-02-05 6:28 ` Tetsuya Mukawa
2016-02-05 6:35 ` Yuanhan Liu
2016-02-05 7:10 ` Tetsuya Mukawa
2016-02-08 9:42 ` Ferruh Yigit
2016-02-09 1:54 ` Tetsuya Mukawa
2016-02-05 11:28 ` [dpdk-dev] [PATCH v8 0/2] " Tetsuya Mukawa
2016-02-05 11:28 ` [dpdk-dev] [PATCH v8 1/2] ethdev: Add a new event type to notify a queue state changed event Tetsuya Mukawa
2016-02-06 4:57 ` Yuanhan Liu
2016-02-05 11:28 ` [dpdk-dev] [PATCH v8 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2016-02-06 5:12 ` Yuanhan Liu
2016-02-09 9:38 ` [dpdk-dev] [PATCH v9 0/2] " Tetsuya Mukawa
2016-02-24 2:45 ` Qiu, Michael
2016-02-24 5:09 ` Tetsuya Mukawa
2016-02-25 7:51 ` Qiu, Michael
2016-02-26 4:29 ` Tetsuya Mukawa
2016-02-26 8:35 ` Tetsuya Mukawa
2016-03-01 2:00 ` Qiu, Michael
2016-03-01 2:19 ` Tetsuya Mukawa
2016-03-02 2:24 ` Qiu, Michael
2016-03-04 1:12 ` Tetsuya Mukawa
2016-02-09 9:38 ` [dpdk-dev] [PATCH v9 1/2] ethdev: Add a new event type to notify a queue state changed event Tetsuya Mukawa
2016-03-04 4:17 ` [dpdk-dev] [PATCH v10 0/2] Add VHOST PMD Tetsuya Mukawa
2016-03-04 4:17 ` [dpdk-dev] [PATCH v10 1/2] ethdev: Add a new event type to notify a queue state changed event Tetsuya Mukawa
2016-03-04 4:17 ` [dpdk-dev] [PATCH v10 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2016-03-04 8:39 ` Yuanhan Liu
2016-03-04 9:58 ` Tetsuya Mukawa
2016-03-07 2:07 ` [dpdk-dev] [PATCH v11 0/2] " Tetsuya Mukawa
2016-03-07 2:07 ` [dpdk-dev] [PATCH v11 1/2] ethdev: Add a new event type to notify a queue state changed event Tetsuya Mukawa
2016-03-07 2:07 ` [dpdk-dev] [PATCH v11 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2016-03-14 12:02 ` Bruce Richardson
2016-03-15 5:35 ` Tetsuya Mukawa
2016-03-15 8:31 ` [dpdk-dev] [PATCH v12 0/2] " Tetsuya Mukawa
2016-03-15 8:31 ` [dpdk-dev] [PATCH v12 1/2] ethdev: Add a new event type to notify a queue state changed event Tetsuya Mukawa
2016-03-18 13:54 ` Thomas Monjalon
2016-03-15 8:31 ` [dpdk-dev] [PATCH v12 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2016-03-18 12:27 ` Bruce Richardson
2016-03-18 13:41 ` Tetsuya Mukawa
2016-03-18 13:52 ` Thomas Monjalon
2016-03-18 14:03 ` Tetsuya Mukawa
2016-03-18 14:13 ` Bruce Richardson
2016-03-18 14:21 ` Tetsuya Mukawa
2016-03-21 5:41 ` Tetsuya Mukawa
2016-03-21 5:45 ` [dpdk-dev] [PATCH v13 0/2] " Tetsuya Mukawa
2016-03-21 12:42 ` Bruce Richardson
2016-03-21 5:45 ` [dpdk-dev] [PATCH v13 1/2] ethdev: Add a new event type to notify a queue state changed event Tetsuya Mukawa
2016-03-21 8:37 ` Thomas Monjalon
2016-03-21 9:24 ` Tetsuya Mukawa
2016-03-21 11:05 ` Bruce Richardson
2016-03-21 13:51 ` Tetsuya Mukawa
2016-03-21 5:45 ` [dpdk-dev] [PATCH v13 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2016-03-21 15:40 ` Loftus, Ciara
2016-03-22 1:55 ` Tetsuya Mukawa
2016-03-22 2:50 ` Tetsuya Mukawa
2016-03-22 10:33 ` Loftus, Ciara
2016-02-09 9:38 ` [dpdk-dev] [PATCH v9 " Tetsuya Mukawa
2015-11-24 9:00 ` [dpdk-dev] [PATCH v5 2/3] " Tetsuya Mukawa
2015-12-18 7:45 ` Yuanhan Liu
2015-12-18 9:25 ` Tetsuya Mukawa
2015-11-24 9:00 ` [dpdk-dev] [PATCH v5 3/3] vhost: Add helper function to convert port id to virtio device pointer Tetsuya Mukawa
2015-12-17 11:47 ` Yuanhan Liu
2015-12-18 3:15 ` Tetsuya Mukawa
2015-12-18 4:19 ` Yuanhan Liu
2015-12-08 1:12 ` [dpdk-dev] [PATCH v5 0/3] Add VHOST PMD Tetsuya Mukawa
2015-12-08 2:03 ` Yuanhan Liu
2015-12-08 2:10 ` Tetsuya Mukawa
2015-11-13 5:32 ` [dpdk-dev] [PATCH v4 0/2] " Yuanhan Liu
2015-11-13 5:37 ` Tetsuya Mukawa
2015-11-13 6:50 ` Tetsuya Mukawa
2015-11-17 13:26 ` Yuanhan Liu
2015-11-19 1:20 ` Tetsuya Mukawa
2015-11-09 5:42 ` [dpdk-dev] [PATCH v3 " Yuanhan Liu
2015-11-02 3:58 ` [dpdk-dev] [PATCH v2 2/2] vhost: " Tetsuya Mukawa
2015-11-06 2:22 ` Yuanhan Liu
2015-11-06 3:54 ` Tetsuya Mukawa
2015-11-05 2:17 ` [dpdk-dev] [PATCH v2 0/2] " Tetsuya Mukawa
2015-11-09 22:25 ` [dpdk-dev] [PATCH 3/3] vhost: " Stephen Hemminger
2015-11-10 3:27 ` Tetsuya Mukawa
2015-10-27 7:54 ` [dpdk-dev] [PATCH 0/3] " Tetsuya Mukawa
2015-10-30 18:30 ` Thomas Monjalon
2015-11-02 3:15 ` Tetsuya Mukawa
2015-10-22 9:45 ` [dpdk-dev] [RFC PATCH v3 2/2] vhost: " Tetsuya Mukawa
2015-10-22 12:49 ` Bruce Richardson
2015-10-23 3:48 ` Tetsuya Mukawa
2015-10-29 14:25 ` Xie, Huawei
2015-10-30 1:18 ` Tetsuya Mukawa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAGSMBPMeHByvncdZWbi0mCc2B1CczH005jKQvewe9fwNMFD_uQ@mail.gmail.com \
--to=rich.lane@bigswitch.com \
--cc=ann.zhuangyanying@huawei.com \
--cc=dev@dpdk.org \
--cc=mukawa@igel.co.jp \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).