Re: [dpdk-dev] Valgrind and DPDK - RDRAND + RTE_MACHINE obstacle

Re: [dpdk-dev] Valgrind and DPDK - RDRAND + RTE_MACHINE obstacle

[Morten Jagd Christensen]

Hi all,

Thanks to Marc for sharinghis experiment and great to hear that others
might benefit form the latest developments.

I can add the following experience:

Last night I downloaded Valgrind 3.10.0, applied the patch and 
built Valgrind  with success.

Unfortunately on my hardware (Xeon E5 processors) and dpdk-1.7.1
I now get the following error:

$ sudo /usr/local/valgrind/bin/valgrind ./build/myprogram -c 0xffff -n 4
==24093== Memcheck, a memory error detector
==24093== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==24093== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info
==24093== Command: ./build/myprogram -c 0xffff -n 4
==24093== 
ERROR: This system does not support "RDRAND".
Please check that RTE_MACHINE is set correctly.

Has anyone experienced the same?

Cheers

Morten

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Marc Sune
Sent: 7. november 2014 04:36
To: Matthew Hall
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] Valgrind and DPDK - does it work ?

On 07/11/14 02:04, Matthew Hall wrote:
> On Fri, Nov 07, 2014 at 01:22:49AM +0100, Marc Sune wrote:
>> Found some time to have a close look. I also wanted to check a DPDK 
>> app against valgrind. It works!
>>
>> I downloaded and compiled valgrind from sources (3.10.0) and applied
>> (manually) this patch:
>>
>> https://bugs.kde.org/attachment.cgi?id=85950&action=edit
>>
>> (Applied around line 2216)
>>
>>  From this post:
>>
>> http://valgrind.10908.n7.nabble.com/mpich-unable-to-munmap-hugepages-
>> td49150.html
>>
>> Happy debugging
>> Marc
> Marc,
>
> This is just AMAZING!!! I have wished for it for many years for DPDK, 
> ever since I used it in beta before it went GA.
Thanks

Most kudos though (if not all), should go to T. Janjusic. I just found it
and backported to the latest stable:

https://bugs.kde.org/show_bug.cgi?id=333051

>
> Would it be possible to post your modification of Valgrind in Github, 
> Bitbucket, or some other repo? I'd like to try this out on my app, too.

Sure, it is a good idea.

I just did a quick&dirty import of the latest stable release 3.10.0. No
git-svn , or anything fancy (github import failed, silently ignoring
files...)  no gitignore too. And then I applied the patch:

https://github.com/bisdn/valgrind-hugepages

Please note the comment in README.md, I cannot provide any support or take
any responsibility. We will also delete this repo once there is a stable
release.

But you can at least try it, should work ;)

>
> Also, not sure if anybody sent this upstream to Valgrind, but if not, 
> we really should, so it just works by default from now on.
It already is, there is simply no stable release yet (see Phillipe comment):

https://bugs.kde.org/show_bug.cgi?id=333051#c8

Cheers
Marc
>
> Thanks,
> Matthew.

Re: [dpdk-dev] Valgrind and DPDK - RDRAND + RTE_MACHINE obstacle

[Burakov, Anatoly]

Hi Morten

I'm not that familiar with valgrind, but as far as I know it's using some kind of CPU emulation. Most likely the processor being emulated by valgrind doesn't have the capabilities your actual CPU has (sort of like trying to run a AVX2-compiled code on a QEMU machine emulating Westmere). You may try to edit the config file to change the RTE_MACHINE to compile DPDK for another platform.

Thanks,
Anatoly

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Morten Jagd Christensen
Sent: Wednesday, November 12, 2014 9:44 AM
To: dev@dpdk.org
Subject: Re: [dpdk-dev] Valgrind and DPDK - RDRAND + RTE_MACHINE obstacle

Hi all,

Thanks to Marc for sharinghis experiment and great to hear that others might benefit form the latest developments.

I can add the following experience:

Last night I downloaded Valgrind 3.10.0, applied the patch and built Valgrind  with success.

Unfortunately on my hardware (Xeon E5 processors) and dpdk-1.7.1 I now get the following error:

$ sudo /usr/local/valgrind/bin/valgrind ./build/myprogram -c 0xffff -n 4 ==24093== Memcheck, a memory error detector ==24093== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==24093== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info ==24093== Command: ./build/myprogram -c 0xffff -n 4 ==24093==
ERROR: This system does not support "RDRAND".
Please check that RTE_MACHINE is set correctly.

Has anyone experienced the same?

Cheers

Morten

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Marc Sune
Sent: 7. november 2014 04:36
To: Matthew Hall
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] Valgrind and DPDK - does it work ?

On 07/11/14 02:04, Matthew Hall wrote:
> On Fri, Nov 07, 2014 at 01:22:49AM +0100, Marc Sune wrote:
>> Found some time to have a close look. I also wanted to check a DPDK 
>> app against valgrind. It works!
>>
>> I downloaded and compiled valgrind from sources (3.10.0) and applied
>> (manually) this patch:
>>
>> https://bugs.kde.org/attachment.cgi?id=85950&action=edit
>>
>> (Applied around line 2216)
>>
>>  From this post:
>>
>> http://valgrind.10908.n7.nabble.com/mpich-unable-to-munmap-hugepages-
>> td49150.html
>>
>> Happy debugging
>> Marc
> Marc,
>
> This is just AMAZING!!! I have wished for it for many years for DPDK, 
> ever since I used it in beta before it went GA.
Thanks

Most kudos though (if not all), should go to T. Janjusic. I just found it and backported to the latest stable:

https://bugs.kde.org/show_bug.cgi?id=333051

>
> Would it be possible to post your modification of Valgrind in Github, 
> Bitbucket, or some other repo? I'd like to try this out on my app, too.

Sure, it is a good idea.

I just did a quick&dirty import of the latest stable release 3.10.0. No git-svn , or anything fancy (github import failed, silently ignoring
files...)  no gitignore too. And then I applied the patch:

https://github.com/bisdn/valgrind-hugepages

Please note the comment in README.md, I cannot provide any support or take any responsibility. We will also delete this repo once there is a stable release.

But you can at least try it, should work ;)

>
> Also, not sure if anybody sent this upstream to Valgrind, but if not, 
> we really should, so it just works by default from now on.
It already is, there is simply no stable release yet (see Phillipe comment):

https://bugs.kde.org/show_bug.cgi?id=333051#c8

Cheers
Marc
>
> Thanks,
> Matthew.

Re: [dpdk-dev] Valgrind and DPDK - RDRAND + RTE_MACHINE obstacle

[Marc Sune]

On 12/11/14 15:49, Burakov, Anatoly wrote:
> Hi Morten
>
> I'm not that familiar with valgrind, but as far as I know it's using some kind of CPU emulation. Most likely the processor being emulated by valgrind doesn't have the capabilities your actual CPU has (sort of like trying to run a AVX2-compiled code on a QEMU machine emulating Westmere). You may try to edit the config file to change the RTE_MACHINE to compile DPDK for another platform.

I am not aware of internal details of valgrind neither, but I believe 
Anatoly is right. I assume the application runs without valgrind.

I can tell you I am running the DPDK test applications and the DPDK 
application stuff in qemu VM for development purposes and DPDK apps run 
in valgrind without problems.

So perhaps is the valgrind copy you have? You can try the patched 
version I use for the moment, that is here:

https://github.com/bisdn/valgrind-hugepages

One recommendation: once it is working, unless you have a very powerful 
machine (which my VM isn't) I recommend to add some usleeps in the main 
I/O loops when running valgrind, otherwise due to the heavyweight 
emulation valgrind does it slows down application execution considerably.

Let's see if it helps.
Marc

>
> Thanks,
> Anatoly
>
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Morten Jagd Christensen
> Sent: Wednesday, November 12, 2014 9:44 AM
> To: dev@dpdk.org
> Subject: Re: [dpdk-dev] Valgrind and DPDK - RDRAND + RTE_MACHINE obstacle
>
> Hi all,
>
> Thanks to Marc for sharinghis experiment and great to hear that others might benefit form the latest developments.
>
> I can add the following experience:
>
> Last night I downloaded Valgrind 3.10.0, applied the patch and built Valgrind  with success.
>
> Unfortunately on my hardware (Xeon E5 processors) and dpdk-1.7.1 I now get the following error:
>
> $ sudo /usr/local/valgrind/bin/valgrind ./build/myprogram -c 0xffff -n 4 ==24093== Memcheck, a memory error detector ==24093== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
> ==24093== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info ==24093== Command: ./build/myprogram -c 0xffff -n 4 ==24093==
> ERROR: This system does not support "RDRAND".
> Please check that RTE_MACHINE is set correctly.
>
> Has anyone experienced the same?
>
> Cheers
>
> Morten
>
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Marc Sune
> Sent: 7. november 2014 04:36
> To: Matthew Hall
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] Valgrind and DPDK - does it work ?
>
> On 07/11/14 02:04, Matthew Hall wrote:
>> On Fri, Nov 07, 2014 at 01:22:49AM +0100, Marc Sune wrote:
>>> Found some time to have a close look. I also wanted to check a DPDK
>>> app against valgrind. It works!
>>>
>>> I downloaded and compiled valgrind from sources (3.10.0) and applied
>>> (manually) this patch:
>>>
>>> https://bugs.kde.org/attachment.cgi?id=85950&action=edit
>>>
>>> (Applied around line 2216)
>>>
>>>   From this post:
>>>
>>> http://valgrind.10908.n7.nabble.com/mpich-unable-to-munmap-hugepages-
>>> td49150.html
>>>
>>> Happy debugging
>>> Marc
>> Marc,
>>
>> This is just AMAZING!!! I have wished for it for many years for DPDK,
>> ever since I used it in beta before it went GA.
> Thanks
>
> Most kudos though (if not all), should go to T. Janjusic. I just found it and backported to the latest stable:
>
> https://bugs.kde.org/show_bug.cgi?id=333051
>
>> Would it be possible to post your modification of Valgrind in Github,
>> Bitbucket, or some other repo? I'd like to try this out on my app, too.
> Sure, it is a good idea.
>
> I just did a quick&dirty import of the latest stable release 3.10.0. No git-svn , or anything fancy (github import failed, silently ignoring
> files...)  no gitignore too. And then I applied the patch:
>
> https://github.com/bisdn/valgrind-hugepages
>
> Please note the comment in README.md, I cannot provide any support or take any responsibility. We will also delete this repo once there is a stable release.
>
> But you can at least try it, should work ;)
>
>> Also, not sure if anybody sent this upstream to Valgrind, but if not,
>> we really should, so it just works by default from now on.
> It already is, there is simply no stable release yet (see Phillipe comment):
>
> https://bugs.kde.org/show_bug.cgi?id=333051#c8
>
> Cheers
> Marc
>> Thanks,
>> Matthew.

Re: [dpdk-dev] Valgrind and DPDK - RDRAND + RTE_MACHINE obstacle

[Marc Sune]

On 14/11/14 10:03, Morten Jagd Christensen wrote:
> Hi Marc,
>
> It turns out that I could successively comment out the setting of cpu flags
> in mk/rte.cpuflags,
> rebuild dpdk and my application, and then run valgrid until no further error
> messages occured.
>
> But it turns out to be too slow for practical purposes :-(

(Please, do not drop the list, it might be helpful to others)

Ok, I am not sure if this can cause you any problem. Perhaps someone in 
the community can advise you here.

valgrind should only be used to do functional testing, not for 
performance. But for doing so I strongly suggest to do this:

> [snip]
> One recommendation: once it is working, unless you have a very 
> powerful machine (which my VM isn't) I recommend to add some usleeps 
> in the main I/O loops when running valgrind, otherwise due to the 
> heavyweight emulation valgrind does it slows down application 
> execution considerably. Let's see if it helps. Marc 

Best
Marc