[dpdk-users] Pure container-ized DPDK test

[dpdk-users] Pure container-ized DPDK test

[Mathieu Devos]

Hello,

For a recent project, we're currently writing an application which is using
DKDP to hook into the NICs directly. However, as part of testing this
application, we use a CI tool which can compile the application, use a
containerized version of DPDK to test said application with the unit tests.

However, we're moving this whole process to the cloud, this is where the
issues come in.
The current setup has a VM running with several of these NICs virtualized
for this container to utilize and run the application with DPDK. This means
we just keep the VM up & running and refresh the container with the new
application in there.

Now with the cloud, we won't have this VM anymore, and we have no idea on
which host this will be running (also we won't be having access to the host
platform probably).

My (our) questions is pretty straightforward, linux kernel allows you to
instantly create new NIC aliases, however, to create new actual NICs you
have to resort to one of these 3 steps (
https://stackoverflow.com/questions/49963311/how-can-i-get-dpdk-to-recognize-a-nic-virtualized-by-linux).
However, would it be possible to run our application in a pure container,
emulate (in that container) one or multiple NICs and let DPDK hook into
those?
We're very well aware that the speed of these NICs will be atrociously low,
something DPDK was written to originally avoid, however, this is one of the
first testing setups in our QA process.

If the units tests for the application would pass within this container, we
would export it to a proper VM, and then push it down to the actual
hardware to continue testing.

The idea is that we wish to see if our application is failing as soon as
possible, and not wait to see it on hardware, just to see it all fail
because of faulty code in the application.

If any of this sounds ridiculous, or I'm understanding the whole setup just
plainly wrong, don't hesitate to correct me.
I'm very new to DPDK, and find it very interesting so far, always looking
to learn more on the topic.


Best regards,
Mathieu

-- 
*M*athieu *D*evos
M.Sc.(Tech) - Software Developer
+358 45 787 48074
Intopalo

Re: [dpdk-users] Pure container-ized DPDK test

[Shreyansh Jain]

Hello,

Comment inline ...

> -----Original Message-----
> From: users <users-bounces@dpdk.org> On Behalf Of Mathieu Devos
> Sent: Friday, September 7, 2018 5:58 PM
> To: users@dpdk.org
> Subject: [dpdk-users] Pure container-ized DPDK test
> 
> Hello,
> 
> For a recent project, we're currently writing an application which is
> using
> DKDP to hook into the NICs directly. However, as part of testing this
> application, we use a CI tool which can compile the application, use a
> containerized version of DPDK to test said application with the unit
> tests.
> 
> However, we're moving this whole process to the cloud, this is where
> the
> issues come in.
> The current setup has a VM running with several of these NICs
> virtualized
> for this container to utilize and run the application with DPDK. This
> means
> we just keep the VM up & running and refresh the container with the new
> application in there.
> 
> Now with the cloud, we won't have this VM anymore, and we have no idea
> on
> which host this will be running (also we won't be having access to the
> host
> platform probably).
> 
> My (our) questions is pretty straightforward, linux kernel allows you
> to
> instantly create new NIC aliases, however, to create new actual NICs
> you
> have to resort to one of these 3 steps (
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstac
> koverflow.com%2Fquestions%2F49963311%2Fhow-can-i-get-dpdk-to-recognize-
> a-nic-virtualized-by-
> linux&amp;data=02%7C01%7Cshreyansh.jain%40nxp.com%7Ca5065626839d417412a
> 708d614bd58f3%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C636719200809
> 153362&amp;sdata=5MgQzHNjeK%2BRDLyXgWKtgR1EZPNooTLHvYjx8AbTgA4%3D&amp;r
> eserved=0).
> However, would it be possible to run our application in a pure
> container,
> emulate (in that container) one or multiple NICs and let DPDK hook into
> those?
> We're very well aware that the speed of these NICs will be atrociously
> low,
> something DPDK was written to originally avoid, however, this is one of
> the
> first testing setups in our QA process.

It all depends on what you mean by "..emulate one or multiple NICs" and what privileges are available to the Container.

From the link you have given, only "NIC Virtual Function" and "DPDK Virtual Device" suit you. But, for either case the first step is assigning these resources (even if virtualized) to Container. Is that sorted out? Your host environment is unknown, from what I understand.

Without the true DPDK style of direct-to-NIC, following is definitely possible:
1) spawn a container with a veth pair connected to a OVS running on host and other end in containter.
2) in the container, use tun/tap layer to connect to the exposed veth.

> 
> If the units tests for the application would pass within this
> container, we
> would export it to a proper VM, and then push it down to the actual
> hardware to continue testing.
> 
> The idea is that we wish to see if our application is failing as soon
> as
> possible, and not wait to see it on hardware, just to see it all fail
> because of faulty code in the application.
> 
> If any of this sounds ridiculous, or I'm understanding the whole setup
> just
> plainly wrong, don't hesitate to correct me.
> I'm very new to DPDK, and find it very interesting so far, always
> looking
> to learn more on the topic.

Well, even after considerable time here, I am still new :D - So, keep asking - I am sure someone can answer better.

> 
> 
> Best regards,
> Mathieu
> 
> --
> *M*athieu *D*evos
> M.Sc.(Tech) - Software Developer
> +358 45 787 48074
> Intopalo