* [dpdk-dev] Regarding VM live migration with SRIOV
@ 2013-11-25 14:01 Prashant Upadhyaya
2013-11-27 4:39 ` Prashant Upadhyaya
0 siblings, 1 reply; 6+ messages in thread
From: Prashant Upadhyaya @ 2013-11-25 14:01 UTC (permalink / raw)
To: dev
Hi guys,
I have a VM on top of QEMU/KVM hypervisor. Guest and Host are both Fedora 18.
I am using 82599 NIC with SRIOV based VF's in the VM.
In VM I am running a DPDK based application which uses the VF.
Now I have to do a live migration of the running VM from one physical machine to the other.
Right, so has anybody tried it before in the above environment. If yes, would love to hear from you and learn from your experience of doing it instead of making the same mistakes and learning the hard way.
Regards
-Prashant
===============================================================================
Please refer to http://www.aricent.com/legal/email_disclaimer.html
for important disclosures regarding this electronic communication.
===============================================================================
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] Regarding VM live migration with SRIOV
2013-11-25 14:01 [dpdk-dev] Regarding VM live migration with SRIOV Prashant Upadhyaya
@ 2013-11-27 4:39 ` Prashant Upadhyaya
2013-11-27 4:45 ` Stephen Hemminger
0 siblings, 1 reply; 6+ messages in thread
From: Prashant Upadhyaya @ 2013-11-27 4:39 UTC (permalink / raw)
To: dev
Hi,
Let me be more specific.
Does DPDK support hot plugin/plugout of PCI devices ?
What typically needs to be done if this is to be achieved inside an application.
Typically, the NIC PF or VF appears to the DPDK application as a PCI device which is probed at startup.
Now what happens if I insert a new VF dynamically and want to use it inside the DPDK application (while it is already running), how should this typically be done ? [hotplugin]
And what happens if the DPDK application is in control of a PCI device and that PCI device is suddenly removed ? How can the application detect this and stop doing data transfer on this and sort of unload it ? [hotplugout]
If the above can be coded inside the DPDK app, then we can think of live VM migration with SRIOV -- just hotplugin and plugout the VF's.
Regards
-Prashant
-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Prashant Upadhyaya
Sent: Monday, November 25, 2013 7:32 PM
To: dev@dpdk.org
Subject: [dpdk-dev] Regarding VM live migration with SRIOV
Hi guys,
I have a VM on top of QEMU/KVM hypervisor. Guest and Host are both Fedora 18.
I am using 82599 NIC with SRIOV based VF's in the VM.
In VM I am running a DPDK based application which uses the VF.
Now I have to do a live migration of the running VM from one physical machine to the other.
Right, so has anybody tried it before in the above environment. If yes, would love to hear from you and learn from your experience of doing it instead of making the same mistakes and learning the hard way.
Regards
-Prashant
===============================================================================
Please refer to http://www.aricent.com/legal/email_disclaimer.html
for important disclosures regarding this electronic communication.
===============================================================================
===============================================================================
Please refer to http://www.aricent.com/legal/email_disclaimer.html
for important disclosures regarding this electronic communication.
===============================================================================
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] Regarding VM live migration with SRIOV
2013-11-27 4:39 ` Prashant Upadhyaya
@ 2013-11-27 4:45 ` Stephen Hemminger
2013-11-27 6:09 ` Prashant Upadhyaya
0 siblings, 1 reply; 6+ messages in thread
From: Stephen Hemminger @ 2013-11-27 4:45 UTC (permalink / raw)
To: Prashant Upadhyaya; +Cc: dev
On Wed, 27 Nov 2013 10:09:09 +0530
Prashant Upadhyaya <prashant.upadhyaya@aricent.com> wrote:
> Hi,
>
> Let me be more specific.
> Does DPDK support hot plugin/plugout of PCI devices ?
> What typically needs to be done if this is to be achieved inside an application.
>
> Typically, the NIC PF or VF appears to the DPDK application as a PCI device which is probed at startup.
> Now what happens if I insert a new VF dynamically and want to use it inside the DPDK application (while it is already running), how should this typically be done ? [hotplugin]
> And what happens if the DPDK application is in control of a PCI device and that PCI device is suddenly removed ? How can the application detect this and stop doing data transfer on this and sort of unload it ? [hotplugout]
>
> If the above can be coded inside the DPDK app, then we can think of live VM migration with SRIOV -- just hotplugin and plugout the VF's.
>
> Regards
> -Prashant
>
The current implementation does look like it supports hotplug.
All devices are discovered during rte_eal_pci_probe.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] Regarding VM live migration with SRIOV
2013-11-27 4:45 ` Stephen Hemminger
@ 2013-11-27 6:09 ` Prashant Upadhyaya
2013-11-27 6:24 ` Stephen Hemminger
0 siblings, 1 reply; 6+ messages in thread
From: Prashant Upadhyaya @ 2013-11-27 6:09 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: dev
Hi Stephen,
The rte_eal_pci_probe is typically called at the startup.
Now let's say a DPDK application is running with a PCI device (doing tx and rx) and I remove that PCI device underneath (hot plugout)
So how does the application now know that the device is gone ?
Is it that rte_eal_pci_probe should be called periodically from, let's say, the slow control path of the DPDK application ?
Regards
-Prashant
-----Original Message-----
From: Stephen Hemminger [mailto:stephen@networkplumber.org]
Sent: Wednesday, November 27, 2013 10:16 AM
To: Prashant Upadhyaya
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] Regarding VM live migration with SRIOV
On Wed, 27 Nov 2013 10:09:09 +0530
Prashant Upadhyaya <prashant.upadhyaya@aricent.com> wrote:
> Hi,
>
> Let me be more specific.
> Does DPDK support hot plugin/plugout of PCI devices ?
> What typically needs to be done if this is to be achieved inside an application.
>
> Typically, the NIC PF or VF appears to the DPDK application as a PCI device which is probed at startup.
> Now what happens if I insert a new VF dynamically and want to use it
> inside the DPDK application (while it is already running), how should
> this typically be done ? [hotplugin] And what happens if the DPDK
> application is in control of a PCI device and that PCI device is
> suddenly removed ? How can the application detect this and stop doing
> data transfer on this and sort of unload it ? [hotplugout]
>
> If the above can be coded inside the DPDK app, then we can think of live VM migration with SRIOV -- just hotplugin and plugout the VF's.
>
> Regards
> -Prashant
>
The current implementation does look like it supports hotplug.
All devices are discovered during rte_eal_pci_probe.
===============================================================================
Please refer to http://www.aricent.com/legal/email_disclaimer.html
for important disclosures regarding this electronic communication.
===============================================================================
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] Regarding VM live migration with SRIOV
2013-11-27 6:09 ` Prashant Upadhyaya
@ 2013-11-27 6:24 ` Stephen Hemminger
2013-11-29 5:55 ` Prashant Upadhyaya
0 siblings, 1 reply; 6+ messages in thread
From: Stephen Hemminger @ 2013-11-27 6:24 UTC (permalink / raw)
To: Prashant Upadhyaya; +Cc: dev
On Wed, 27 Nov 2013 11:39:28 +0530
Prashant Upadhyaya <prashant.upadhyaya@aricent.com> wrote:
> Hi Stephen,
>
> The rte_eal_pci_probe is typically called at the startup.
>
> Now let's say a DPDK application is running with a PCI device (doing tx and rx) and I remove that PCI device underneath (hot plugout)
> So how does the application now know that the device is gone ?
>
> Is it that rte_eal_pci_probe should be called periodically from, let's say, the slow control path of the DPDK application ?
>
> Regards
> -Prashant
>
Like I said current code doesn't do hotplug.
If you wanted to add it, you would have to refactor the PCI management layer.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-dev] Regarding VM live migration with SRIOV
2013-11-27 6:24 ` Stephen Hemminger
@ 2013-11-29 5:55 ` Prashant Upadhyaya
0 siblings, 0 replies; 6+ messages in thread
From: Prashant Upadhyaya @ 2013-11-29 5:55 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: dev
Hi Stephen,
Agreed that the current code does not directly support hotplug.
Now I am looking for a hint regarding how best the DPDK application can find out that the PCI device on which it was doing the I/O has been removed from underneath. In that case the application can call the ethdev stop and ethdev close functions gracefully.
Question is -- is there a way for the PMD to know that the device is gone.
Further the PCI device is a mapped memory, so when the plugout of the PCI device happen, is the DPDK application, which is doing I/O, vulnerable to a crash ?
Regards
-Prashant
-----Original Message-----
From: Stephen Hemminger [mailto:stephen@networkplumber.org]
Sent: Wednesday, November 27, 2013 11:54 AM
To: Prashant Upadhyaya
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] Regarding VM live migration with SRIOV
On Wed, 27 Nov 2013 11:39:28 +0530
Prashant Upadhyaya <prashant.upadhyaya@aricent.com> wrote:
> Hi Stephen,
>
> The rte_eal_pci_probe is typically called at the startup.
>
> Now let's say a DPDK application is running with a PCI device (doing
> tx and rx) and I remove that PCI device underneath (hot plugout) So how does the application now know that the device is gone ?
>
> Is it that rte_eal_pci_probe should be called periodically from, let's say, the slow control path of the DPDK application ?
>
> Regards
> -Prashant
>
Like I said current code doesn't do hotplug.
If you wanted to add it, you would have to refactor the PCI management layer.
===============================================================================
Please refer to http://www.aricent.com/legal/email_disclaimer.html
for important disclosures regarding this electronic communication.
===============================================================================
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-11-29 5:54 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-11-25 14:01 [dpdk-dev] Regarding VM live migration with SRIOV Prashant Upadhyaya
2013-11-27 4:39 ` Prashant Upadhyaya
2013-11-27 4:45 ` Stephen Hemminger
2013-11-27 6:09 ` Prashant Upadhyaya
2013-11-27 6:24 ` Stephen Hemminger
2013-11-29 5:55 ` Prashant Upadhyaya
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).