From: Tetsuya Mukawa <mukawa@igel.co.jp>
To: Rich Lane <rich.lane@bigswitch.com>,
Yuanhan Liu <yuanhan.liu@linux.intel.com>
Cc: dev@dpdk.org, ann.zhuangyanying@huawei.com
Subject: Re: [dpdk-dev] [PATCH v5 1/3] vhost: Add callback and private data for vhost PMD
Date: Mon, 21 Dec 2015 11:10:10 +0900 [thread overview]
Message-ID: <56775F82.3090306@igel.co.jp> (raw)
In-Reply-To: <CAGSMBPPGs8b3sGAn6A2ETaau70oeChouTOnj=Ah2sgjz_1p_nA@mail.gmail.com>
On 2015/12/19 3:01, Rich Lane wrote:
> I'm using the vhost callbacks and struct virtio_net with the vhost PMD in a
> few ways:
>
> 1. new_device/destroy_device: Link state change (will be covered by the
> link status interrupt).
> 2. new_device: Add first queue to datapath.
> 3. vring_state_changed: Add/remove queue to datapath.
> 4. destroy_device: Remove all queues (vring_state_changed is not called
> when qemu is killed).
> 5. new_device and struct virtio_net: Determine NUMA node of the VM.
>
> The vring_state_changed callback is necessary because the VM might not be
> using the maximum number of RX queues. If I boot Linux in the VM it will
> start out using one RX queue, which can be changed with ethtool. The DPDK
> app in the host needs to be notified that it can start sending traffic to
> the new queue.
>
> The vring_state_changed callback is also useful for guest TX queues to
> avoid reading from an inactive queue.
>
> API I'd like to have:
>
> 1. Link status interrupt.
> 2. New queue_state_changed callback. Unlike vring_state_changed this should
> cover the first queue at new_device and removal of all queues at
> destroy_device.
> 3. Per-queue or per-device NUMA node info.
Hi Rich and Yuanhan,
As Rich described, some users needs more information when the interrupts
comes.
And the virtio_net structure contains the information.
I guess it's very similar to interrupt handling of normal hardware.
First, a interrupt comes, then an interrupt handler checks status
register of the device to know actually what was happened.
In vhost PMD case, reading status register equals reading virtio_net
structure.
So how about below specification?
1. The link status interrupt of vhost PMD will occurs when new_device,
destroy_device and vring_state_changed events are happened.
2. Vhost PMD provides a function to let the users know virtio_net
structure of the interrupted port.
(Probably almost same as "rte_eth_vhost_portid2vdev" that I described
in "[PATCH v5 3/3] vhost: Add helper function to convert port id to
virtio device pointer")
I guess what kind of information the users need will depends on their
environments.
So just providing virtio_net structure may be good.
What do you think?
Tetsuya,
>
> On Thu, Dec 17, 2015 at 8:28 PM, Tetsuya Mukawa <mukawa@igel.co.jp> wrote:
>
>> On 2015/12/18 13:15, Yuanhan Liu wrote:
>>> On Fri, Dec 18, 2015 at 12:15:42PM +0900, Tetsuya Mukawa wrote:
>>>> On 2015/12/17 20:42, Yuanhan Liu wrote:
>>>>> On Tue, Nov 24, 2015 at 06:00:01PM +0900, Tetsuya Mukawa wrote:
>>>>>> The vhost PMD will be a wrapper of vhost library, but some of vhost
>>>>>> library APIs cannot be mapped to ethdev library APIs.
>>>>>> Becasue of this, in some cases, we still need to use vhost library
>> APIs
>>>>>> for a port created by the vhost PMD.
>>>>>>
>>>>>> Currently, when virtio device is created and destroyed, vhost library
>>>>>> will call one of callback handlers. The vhost PMD need to use this
>>>>>> pair of callback handlers to know which virtio devices are connected
>>>>>> actually.
>>>>>> Because we can register only one pair of callbacks to vhost library,
>> if
>>>>>> the PMD use it, DPDK applications cannot have a way to know the
>> events.
>>>>>> This may break legacy DPDK applications that uses vhost library. To
>> prevent
>>>>>> it, this patch adds one more pair of callbacks to vhost library
>> especially
>>>>>> for the vhost PMD.
>>>>>> With the patch, legacy applications can use the vhost PMD even if
>> they need
>>>>>> additional specific handling for virtio device creation and
>> destruction.
>>>>>> For example, legacy application can call
>>>>>> rte_vhost_enable_guest_notification() in callbacks to change setting.
>>>>> TBH, I never liked it since the beginning. Introducing two callbacks
>>>>> for one event is a bit messy, and therefore error prone.
>>>> I agree with you.
>>>>
>>>>> I have been thinking this occasionally last few weeks, and have came
>>>>> up something that we may introduce another layer callback based on
>>>>> the vhost pmd itself, by a new API:
>>>>>
>>>>> rte_eth_vhost_register_callback().
>>>>>
>>>>> And we then call those new callback inside the vhost pmd new_device()
>>>>> and vhost pmd destroy_device() implementations.
>>>>>
>>>>> And we could have same callbacks like vhost have, but I'm thinking
>>>>> that new_device() and destroy_device() doesn't sound like a good name
>>>>> to a PMD driver. Maybe a name like "link_state_changed" is better?
>>>>>
>>>>> What do you think of that?
>>>> Yes, "link_state_changed" will be good.
>>>>
>>>> BTW, I thought it was ok that an DPDK app that used vhost PMD called
>>>> vhost library APIs directly.
>>>> But probably you may feel strangeness about it. Is this correct?
>>> Unluckily, that's true :)
>>>
>>>> If so, how about implementing legacy status interrupt mechanism to vhost
>>>> PMD?
>>>> For example, an DPDK app can register callback handler like
>>>> "examples/link_status_interrupt".
>>>>
>>>> Also, if the app doesn't call vhost library APIs directly,
>>>> rte_eth_vhost_portid2vdev() will be needless, because the app doesn't
>>>> need to handle virtio device structure anymore.
>>>>
>>>>> On the other hand, I'm still thinking is that really necessary to let
>>>>> the application be able to call vhost functions like
>> rte_vhost_enable_guest_notification()
>>>>> with the vhost PMD driver?
>>>> Basic concept of my patch is that vhost PMD will provides the features
>>>> that vhost library provides.
>>> I don't think that's necessary. Let's just treat it as a normal pmd
>>> driver, having nothing to do with vhost library.
>>>
>>>> How about removing rte_vhost_enable_guest_notification() from "vhost
>>>> library"?
>>>> (I also not sure what are use cases)
>>>> If we can do this, vhost PMD also doesn't need to take care of it.
>>>> Or if rte_vhost_enable_guest_notification() will be removed in the
>>>> future, vhost PMD is able to ignore it.
>>> You could either call it in vhost-pmd (which you already have done that),
>>> or ignore it in vhost-pmd, but dont' remove it from vhost library.
>>>
>>>> Please let me correct up my thinking about your questions.
>>>> - Change concept of patch not to call vhost library APIs directly.
>>>> These should be wrapped by ethdev APIs.
>>>> - Remove rte_eth_vhost_portid2vdev(), because of above concept
>> changing.
>>>> - Implement legacy status changed interrupt to vhost PMD instead of
>>>> using own callback mechanism.
>>>> - Check if we can remove rte_vhost_enable_guest_notification() from
>>>> vhost library.
>>> So, how about making it __fare__ simple as the first step, to get merged
>>> easily, that we don't assume the applications will call any vhost library
>>> functions any more, so that we don't need the callback, and we don't need
>>> the rte_eth_vhost_portid2vdev(), either. Again, just let it be a fare
>>> normal (nothing special) pmd driver. (UNLESS, there is a real must,
>> which
>>> I don't see so far).
>>>
>>> Tetsuya, what do you think of that then?
>> I agree with you. But will wait a few days.
>> Because if someone wants to use it from vhost PMD, they probably will
>> provides use cases.
>> And if there are no use cases, let's do like above.
>>
>> Thanks,
>> Tetsuya
>>
next prev parent reply other threads:[~2015-12-21 2:10 UTC|newest]
Thread overview: 199+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-08-31 3:55 [dpdk-dev] [RFC PATCH v2] Add VHOST PMD Tetsuya Mukawa
2015-08-31 3:55 ` [dpdk-dev] [RFC PATCH v2] vhost: " Tetsuya Mukawa
2015-09-23 17:47 ` Loftus, Ciara
2015-10-16 8:40 ` Tetsuya Mukawa
2015-10-20 14:13 ` Loftus, Ciara
2015-10-21 4:30 ` Tetsuya Mukawa
2015-10-21 10:09 ` Bruce Richardson
2015-10-16 12:52 ` Bruce Richardson
2015-10-19 1:51 ` Tetsuya Mukawa
2015-10-19 9:32 ` Loftus, Ciara
2015-10-19 9:45 ` Bruce Richardson
2015-10-19 10:50 ` Tetsuya Mukawa
2015-10-19 13:26 ` Panu Matilainen
2015-10-19 13:27 ` Richardson, Bruce
2015-10-21 4:35 ` Tetsuya Mukawa
2015-10-21 6:25 ` Panu Matilainen
2015-10-21 10:22 ` Bruce Richardson
2015-10-22 9:50 ` Tetsuya Mukawa
2015-10-27 13:44 ` Traynor, Kevin
2015-10-28 2:24 ` Tetsuya Mukawa
2015-10-22 9:45 ` [dpdk-dev] [RFC PATCH v3 0/2] " Tetsuya Mukawa
2015-10-22 9:45 ` [dpdk-dev] [RFC PATCH v3 1/2] vhost: Add callback and private data for vhost PMD Tetsuya Mukawa
2015-10-27 6:12 ` [dpdk-dev] [PATCH 0/3] Add VHOST PMD Tetsuya Mukawa
2015-10-27 6:12 ` [dpdk-dev] [PATCH 1/3] vhost: Fix wrong handling of virtqueue array index Tetsuya Mukawa
2015-10-27 6:29 ` Yuanhan Liu
2015-10-27 6:33 ` Yuanhan Liu
2015-10-27 6:47 ` Yuanhan Liu
2015-10-27 7:28 ` Tetsuya Mukawa
2015-10-27 7:34 ` Yuanhan Liu
2015-10-27 6:12 ` [dpdk-dev] [PATCH 2/3] vhost: Add callback and private data for vhost PMD Tetsuya Mukawa
2015-10-30 17:49 ` Loftus, Ciara
2015-11-02 3:15 ` Tetsuya Mukawa
2015-10-27 6:12 ` [dpdk-dev] [PATCH 3/3] vhost: Add VHOST PMD Tetsuya Mukawa
2015-11-02 3:58 ` [dpdk-dev] [PATCH v2 0/2] " Tetsuya Mukawa
2015-11-02 3:58 ` [dpdk-dev] [PATCH v2 1/2] vhost: Add callback and private data for vhost PMD Tetsuya Mukawa
2015-11-09 5:16 ` [dpdk-dev] [PATCH v3 0/2] Add VHOST PMD Tetsuya Mukawa
2015-11-09 5:17 ` [dpdk-dev] [PATCH v3 1/2] vhost: Add callback and private data for vhost PMD Tetsuya Mukawa
2015-11-09 18:16 ` Aaron Conole
2015-11-10 3:13 ` Tetsuya Mukawa
2015-11-10 7:16 ` Panu Matilainen
2015-11-10 9:48 ` Tetsuya Mukawa
2015-11-10 10:05 ` Panu Matilainen
2015-11-10 10:15 ` Tetsuya Mukawa
2015-11-09 5:17 ` [dpdk-dev] [PATCH v3 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2015-11-09 6:21 ` Yuanhan Liu
2015-11-09 6:27 ` Tetsuya Mukawa
2015-11-09 22:22 ` Stephen Hemminger
2015-11-10 3:14 ` Tetsuya Mukawa
2015-11-12 12:52 ` Wang, Zhihong
2015-11-13 3:09 ` Tetsuya Mukawa
2015-11-13 3:50 ` Wang, Zhihong
2015-11-13 4:03 ` Rich Lane
2015-11-13 4:29 ` Tetsuya Mukawa
2015-11-13 5:20 ` [dpdk-dev] [PATCH v4 0/2] " Tetsuya Mukawa
2015-11-13 5:20 ` [dpdk-dev] [PATCH v4 1/2] vhost: Add callback and private data for vhost PMD Tetsuya Mukawa
2015-11-17 13:29 ` Yuanhan Liu
2015-11-19 2:03 ` Tetsuya Mukawa
2015-11-19 2:18 ` Yuanhan Liu
2015-11-19 3:13 ` Tetsuya Mukawa
2015-11-19 3:33 ` Yuanhan Liu
2015-11-19 5:14 ` Tetsuya Mukawa
2015-11-19 5:45 ` Yuanhan Liu
2015-11-19 5:58 ` Tetsuya Mukawa
[not found] ` <20151119063137.GJ2326@yliu-dev.sh.intel.com>
2015-11-19 6:37 ` Tetsuya Mukawa
2015-11-13 5:20 ` [dpdk-dev] [PATCH v4 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2015-11-16 1:57 ` Wang, Zhihong
2015-11-20 11:43 ` Yuanhan Liu
2015-11-24 2:48 ` Tetsuya Mukawa
2015-11-24 3:40 ` Yuanhan Liu
2015-11-24 3:44 ` Tetsuya Mukawa
2015-11-21 0:15 ` Rich Lane
2015-11-24 4:41 ` Tetsuya Mukawa
2015-11-24 9:00 ` [dpdk-dev] [PATCH v5 0/3] " Tetsuya Mukawa
2015-11-24 9:00 ` [dpdk-dev] [PATCH v5 1/3] vhost: Add callback and private data for vhost PMD Tetsuya Mukawa
2015-12-17 11:42 ` Yuanhan Liu
2015-12-18 3:15 ` Tetsuya Mukawa
2015-12-18 3:36 ` Tetsuya Mukawa
2015-12-18 4:15 ` Yuanhan Liu
2015-12-18 4:28 ` Tetsuya Mukawa
2015-12-18 18:01 ` Rich Lane
2015-12-21 2:10 ` Tetsuya Mukawa [this message]
2015-12-22 4:36 ` Yuanhan Liu
2015-12-22 3:41 ` Yuanhan Liu
2015-12-22 4:47 ` Rich Lane
2015-12-22 5:47 ` Yuanhan Liu
2015-12-22 9:38 ` Rich Lane
2015-12-23 2:44 ` Yuanhan Liu
2015-12-23 22:00 ` Thomas Monjalon
2015-12-24 3:51 ` Yuanhan Liu
2015-12-24 4:07 ` Tetsuya Mukawa
2015-12-24 3:09 ` Tetsuya Mukawa
2015-12-24 3:54 ` Tetsuya Mukawa
2015-12-24 4:00 ` Yuanhan Liu
2015-12-24 4:23 ` Tetsuya Mukawa
2015-12-24 5:37 ` Rich Lane
2015-12-24 7:58 ` Tetsuya Mukawa
2015-12-28 21:59 ` Rich Lane
2016-01-06 3:56 ` Tetsuya Mukawa
2016-01-06 7:38 ` Yuanhan Liu
2015-12-18 10:03 ` Xie, Huawei
2015-12-21 2:10 ` Tetsuya Mukawa
2016-02-02 11:18 ` [dpdk-dev] [PATCH v6 0/2] Add VHOST PMD Tetsuya Mukawa
2016-02-02 19:52 ` Rich Lane
2016-02-02 11:18 ` [dpdk-dev] [PATCH v6 1/2] ethdev: Add a new event type to notify a queue state changed event Tetsuya Mukawa
2016-02-02 11:18 ` [dpdk-dev] [PATCH v6 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2016-02-02 23:43 ` Ferruh Yigit
2016-02-03 2:13 ` Tetsuya Mukawa
2016-02-03 7:48 ` Tetsuya Mukawa
2016-02-03 9:24 ` Ferruh Yigit
2016-02-03 9:35 ` Tetsuya Mukawa
2016-02-04 7:26 ` [dpdk-dev] [PATCH v7 0/2] " Tetsuya Mukawa
2016-02-04 7:26 ` [dpdk-dev] [PATCH v7 1/2] ethdev: Add a new event type to notify a queue state changed event Tetsuya Mukawa
2016-02-04 7:26 ` [dpdk-dev] [PATCH v7 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2016-02-04 11:17 ` Ferruh Yigit
2016-02-05 6:28 ` Tetsuya Mukawa
2016-02-05 6:35 ` Yuanhan Liu
2016-02-05 7:10 ` Tetsuya Mukawa
2016-02-08 9:42 ` Ferruh Yigit
2016-02-09 1:54 ` Tetsuya Mukawa
2016-02-05 11:28 ` [dpdk-dev] [PATCH v8 0/2] " Tetsuya Mukawa
2016-02-05 11:28 ` [dpdk-dev] [PATCH v8 1/2] ethdev: Add a new event type to notify a queue state changed event Tetsuya Mukawa
2016-02-06 4:57 ` Yuanhan Liu
2016-02-05 11:28 ` [dpdk-dev] [PATCH v8 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2016-02-06 5:12 ` Yuanhan Liu
2016-02-09 9:38 ` [dpdk-dev] [PATCH v9 0/2] " Tetsuya Mukawa
2016-02-24 2:45 ` Qiu, Michael
2016-02-24 5:09 ` Tetsuya Mukawa
2016-02-25 7:51 ` Qiu, Michael
2016-02-26 4:29 ` Tetsuya Mukawa
2016-02-26 8:35 ` Tetsuya Mukawa
2016-03-01 2:00 ` Qiu, Michael
2016-03-01 2:19 ` Tetsuya Mukawa
2016-03-02 2:24 ` Qiu, Michael
2016-03-04 1:12 ` Tetsuya Mukawa
2016-02-09 9:38 ` [dpdk-dev] [PATCH v9 1/2] ethdev: Add a new event type to notify a queue state changed event Tetsuya Mukawa
2016-03-04 4:17 ` [dpdk-dev] [PATCH v10 0/2] Add VHOST PMD Tetsuya Mukawa
2016-03-04 4:17 ` [dpdk-dev] [PATCH v10 1/2] ethdev: Add a new event type to notify a queue state changed event Tetsuya Mukawa
2016-03-04 4:17 ` [dpdk-dev] [PATCH v10 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2016-03-04 8:39 ` Yuanhan Liu
2016-03-04 9:58 ` Tetsuya Mukawa
2016-03-07 2:07 ` [dpdk-dev] [PATCH v11 0/2] " Tetsuya Mukawa
2016-03-07 2:07 ` [dpdk-dev] [PATCH v11 1/2] ethdev: Add a new event type to notify a queue state changed event Tetsuya Mukawa
2016-03-07 2:07 ` [dpdk-dev] [PATCH v11 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2016-03-14 12:02 ` Bruce Richardson
2016-03-15 5:35 ` Tetsuya Mukawa
2016-03-15 8:31 ` [dpdk-dev] [PATCH v12 0/2] " Tetsuya Mukawa
2016-03-15 8:31 ` [dpdk-dev] [PATCH v12 1/2] ethdev: Add a new event type to notify a queue state changed event Tetsuya Mukawa
2016-03-18 13:54 ` Thomas Monjalon
2016-03-15 8:31 ` [dpdk-dev] [PATCH v12 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2016-03-18 12:27 ` Bruce Richardson
2016-03-18 13:41 ` Tetsuya Mukawa
2016-03-18 13:52 ` Thomas Monjalon
2016-03-18 14:03 ` Tetsuya Mukawa
2016-03-18 14:13 ` Bruce Richardson
2016-03-18 14:21 ` Tetsuya Mukawa
2016-03-21 5:41 ` Tetsuya Mukawa
2016-03-21 5:45 ` [dpdk-dev] [PATCH v13 0/2] " Tetsuya Mukawa
2016-03-21 12:42 ` Bruce Richardson
2016-03-21 5:45 ` [dpdk-dev] [PATCH v13 1/2] ethdev: Add a new event type to notify a queue state changed event Tetsuya Mukawa
2016-03-21 8:37 ` Thomas Monjalon
2016-03-21 9:24 ` Tetsuya Mukawa
2016-03-21 11:05 ` Bruce Richardson
2016-03-21 13:51 ` Tetsuya Mukawa
2016-03-21 5:45 ` [dpdk-dev] [PATCH v13 2/2] vhost: Add VHOST PMD Tetsuya Mukawa
2016-03-21 15:40 ` Loftus, Ciara
2016-03-22 1:55 ` Tetsuya Mukawa
2016-03-22 2:50 ` Tetsuya Mukawa
2016-03-22 10:33 ` Loftus, Ciara
2016-02-09 9:38 ` [dpdk-dev] [PATCH v9 " Tetsuya Mukawa
2015-11-24 9:00 ` [dpdk-dev] [PATCH v5 2/3] " Tetsuya Mukawa
2015-12-18 7:45 ` Yuanhan Liu
2015-12-18 9:25 ` Tetsuya Mukawa
2015-11-24 9:00 ` [dpdk-dev] [PATCH v5 3/3] vhost: Add helper function to convert port id to virtio device pointer Tetsuya Mukawa
2015-12-17 11:47 ` Yuanhan Liu
2015-12-18 3:15 ` Tetsuya Mukawa
2015-12-18 4:19 ` Yuanhan Liu
2015-12-08 1:12 ` [dpdk-dev] [PATCH v5 0/3] Add VHOST PMD Tetsuya Mukawa
2015-12-08 2:03 ` Yuanhan Liu
2015-12-08 2:10 ` Tetsuya Mukawa
2015-11-13 5:32 ` [dpdk-dev] [PATCH v4 0/2] " Yuanhan Liu
2015-11-13 5:37 ` Tetsuya Mukawa
2015-11-13 6:50 ` Tetsuya Mukawa
2015-11-17 13:26 ` Yuanhan Liu
2015-11-19 1:20 ` Tetsuya Mukawa
2015-11-09 5:42 ` [dpdk-dev] [PATCH v3 " Yuanhan Liu
2015-11-02 3:58 ` [dpdk-dev] [PATCH v2 2/2] vhost: " Tetsuya Mukawa
2015-11-06 2:22 ` Yuanhan Liu
2015-11-06 3:54 ` Tetsuya Mukawa
2015-11-05 2:17 ` [dpdk-dev] [PATCH v2 0/2] " Tetsuya Mukawa
2015-11-09 22:25 ` [dpdk-dev] [PATCH 3/3] vhost: " Stephen Hemminger
2015-11-10 3:27 ` Tetsuya Mukawa
2015-10-27 7:54 ` [dpdk-dev] [PATCH 0/3] " Tetsuya Mukawa
2015-10-30 18:30 ` Thomas Monjalon
2015-11-02 3:15 ` Tetsuya Mukawa
2015-10-22 9:45 ` [dpdk-dev] [RFC PATCH v3 2/2] vhost: " Tetsuya Mukawa
2015-10-22 12:49 ` Bruce Richardson
2015-10-23 3:48 ` Tetsuya Mukawa
2015-10-29 14:25 ` Xie, Huawei
2015-10-30 1:18 ` Tetsuya Mukawa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=56775F82.3090306@igel.co.jp \
--to=mukawa@igel.co.jp \
--cc=ann.zhuangyanying@huawei.com \
--cc=dev@dpdk.org \
--cc=rich.lane@bigswitch.com \
--cc=yuanhan.liu@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).