[dpdk-users] [DISCUSSION] code snippet documentation

[dpdk-users] [DISCUSSION] code snippet documentation

[Asaf Penso]

Hello DPDK community,

I would like to bring up a discussion about a way to have code snippets as an example for proper usage.
The DPDK tree is filled with great pieces of code that are well documented and maintained in high quality.
I feel we are a bit behind when we talk about usage examples.

One way, whenever we implement a new feature, is to extend one of the test-* under the "app" folder.
This, however, provides means to test but doesn't provide a good usage example.

Another way is to check the content of the "example" folder and whenever we have a BIG new feature it seems like a good place.
This, however, doesn't provide a good option when we talk about small features.
If, for example, we extend rte_flow with an extra action then providing a full-blown example application is somewhat an entry barrier.

A third option could be to document it in one of the .rst files we have.
Obviously, this requires high maintenance and no option to assure it still compiles.

I'd like to propose another approach that will address the main two issues: remove the entry barrier and assure compilation.
In this approach, inside the "examples" folder we'll create another folder for "snippets".
Inside "snippets" we'll have several files per category, for example, rte_flow_snippets.c
Each .c file will include a main function that calls the different use cases we want to give as an example.
The purpose is not to generate traffic nor read rx/tx packets from the DPDK ports. 
The purpose is to have a good example that compiles properly.

Taking the rte_flow_snippets.c as an example its main function would look like this:

int
main(int argc, char **argv)
{
      rte_flow_snippet_match_5tuple_and_drop();
      rte_flow_snippet_match_geneve_ope_and_rss();
      ...
      Return 0;
}

Regards,
Asaf Penso

Re: [dpdk-users] [DISCUSSION] code snippet documentation

[Thomas Monjalon]

15/07/2021 09:01, Asaf Penso:
> Hello DPDK community,
> 
> I would like to bring up a discussion about a way to have code snippets as an example for proper usage.
> The DPDK tree is filled with great pieces of code that are well documented and maintained in high quality.
> I feel we are a bit behind when we talk about usage examples.
> 
> One way, whenever we implement a new feature, is to extend one of the test-* under the "app" folder.
> This, however, provides means to test but doesn't provide a good usage example.
> 
> Another way is to check the content of the "example" folder and whenever we have a BIG new feature it seems like a good place.
> This, however, doesn't provide a good option when we talk about small features.
> If, for example, we extend rte_flow with an extra action then providing a full-blown example application is somewhat an entry barrier.
> 
> A third option could be to document it in one of the .rst files we have.
> Obviously, this requires high maintenance and no option to assure it still compiles.
> 
> I'd like to propose another approach that will address the main two issues: remove the entry barrier and assure compilation.
> In this approach, inside the "examples" folder we'll create another folder for "snippets".
> Inside "snippets" we'll have several files per category, for example, rte_flow_snippets.c
> Each .c file will include a main function that calls the different use cases we want to give as an example.
> The purpose is not to generate traffic nor read rx/tx packets from the DPDK ports. 
> The purpose is to have a good example that compiles properly.
> 
> Taking the rte_flow_snippets.c as an example its main function would look like this:
> 
> int
> main(int argc, char **argv)
> {
>       rte_flow_snippet_match_5tuple_and_drop();
>       rte_flow_snippet_match_geneve_ope_and_rss();
>       ...
>       Return 0;
> }

I think we need to have a policy or justification about which snippet
is worth to have.
My thought is to avoid creating snippets which have no other value
than showing a function call.
I think there is a value if the context is not simple.

Please could you provide a more complex example?

Re: [dpdk-users] [DISCUSSION] code snippet documentation

[Ajit Khaparde]

On Thu, Jul 22, 2021 at 1:29 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 15/07/2021 09:01, Asaf Penso:
> > Hello DPDK community,
> >
> > I would like to bring up a discussion about a way to have code snippets as an example for proper usage.
> > The DPDK tree is filled with great pieces of code that are well documented and maintained in high quality.
> > I feel we are a bit behind when we talk about usage examples.
> >
> > One way, whenever we implement a new feature, is to extend one of the test-* under the "app" folder.
> > This, however, provides means to test but doesn't provide a good usage example.
> >
> > Another way is to check the content of the "example" folder and whenever we have a BIG new feature it seems like a good place.
> > This, however, doesn't provide a good option when we talk about small features.
> > If, for example, we extend rte_flow with an extra action then providing a full-blown example application is somewhat an entry barrier.
> >
> > A third option could be to document it in one of the .rst files we have.
> > Obviously, this requires high maintenance and no option to assure it still compiles.
> >
> > I'd like to propose another approach that will address the main two issues: remove the entry barrier and assure compilation.
> > In this approach, inside the "examples" folder we'll create another folder for "snippets".
> > Inside "snippets" we'll have several files per category, for example, rte_flow_snippets.c
> > Each .c file will include a main function that calls the different use cases we want to give as an example.
> > The purpose is not to generate traffic nor read rx/tx packets from the DPDK ports.
> > The purpose is to have a good example that compiles properly.
> >
> > Taking the rte_flow_snippets.c as an example its main function would look like this:
> >
> > int
> > main(int argc, char **argv)
> > {
> >       rte_flow_snippet_match_5tuple_and_drop();
> >       rte_flow_snippet_match_geneve_ope_and_rss();
> >       ...
> >       Return 0;
> > }
>
> I think we need to have a policy or justification about which snippet
> is worth to have.
> My thought is to avoid creating snippets which have no other value
> than showing a function call.
> I think there is a value if the context is not simple.

I agree. Otherwise it will be cluttered with snippets.

>
>
> Please could you provide a more complex example?
>
>

Re: [dpdk-users] [DISCUSSION] code snippet documentation

[Ferruh Yigit]

On 7/23/2021 1:02 AM, Ajit Khaparde wrote:
> On Thu, Jul 22, 2021 at 1:29 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>>
>> 15/07/2021 09:01, Asaf Penso:
>>> Hello DPDK community,
>>>
>>> I would like to bring up a discussion about a way to have code snippets as an example for proper usage.
>>> The DPDK tree is filled with great pieces of code that are well documented and maintained in high quality.
>>> I feel we are a bit behind when we talk about usage examples.
>>>
>>> One way, whenever we implement a new feature, is to extend one of the test-* under the "app" folder.
>>> This, however, provides means to test but doesn't provide a good usage example.
>>>
>>> Another way is to check the content of the "example" folder and whenever we have a BIG new feature it seems like a good place.
>>> This, however, doesn't provide a good option when we talk about small features.
>>> If, for example, we extend rte_flow with an extra action then providing a full-blown example application is somewhat an entry barrier.
>>>
>>> A third option could be to document it in one of the .rst files we have.
>>> Obviously, this requires high maintenance and no option to assure it still compiles.
>>>
>>> I'd like to propose another approach that will address the main two issues: remove the entry barrier and assure compilation.
>>> In this approach, inside the "examples" folder we'll create another folder for "snippets".
>>> Inside "snippets" we'll have several files per category, for example, rte_flow_snippets.c
>>> Each .c file will include a main function that calls the different use cases we want to give as an example.
>>> The purpose is not to generate traffic nor read rx/tx packets from the DPDK ports.
>>> The purpose is to have a good example that compiles properly.
>>>
>>> Taking the rte_flow_snippets.c as an example its main function would look like this:
>>>
>>> int
>>> main(int argc, char **argv)
>>> {
>>>       rte_flow_snippet_match_5tuple_and_drop();
>>>       rte_flow_snippet_match_geneve_ope_and_rss();
>>>       ...
>>>       Return 0;
>>> }
>>
>> I think we need to have a policy or justification about which snippet
>> is worth to have.
>> My thought is to avoid creating snippets which have no other value
>> than showing a function call.
>> I think there is a value if the context is not simple.
> 
> I agree. Otherwise it will be cluttered with snippets.
> 

I sometimes use DTS code as sample for an API, that has limited context (which I
believe sometimes needed to understand API properly) and of course compiles fine.

What about making mandatory to add a test to DTS for each API/feature, that
ensures a reasonable sample for the API call.

And if the intent is just to provide sample for API call without context, I
believe test-* can help on that, it clarifies good & bad input for an API.

>>
>>
>> Please could you provide a more complex example?
>>
>>

Re: [dpdk-users] [DISCUSSION] code snippet documentation

[Asaf Penso]

Regards,
Asaf Penso

>-----Original Message-----
>From: Ferruh Yigit <ferruh.yigit@intel.com>
>Sent: Friday, July 23, 2021 3:04 PM
>To: Ajit Khaparde <ajit.khaparde@broadcom.com>; NBU-Contact-Thomas
>Monjalon <thomas@monjalon.net>
>Cc: Asaf Penso <asafp@nvidia.com>; users@dpdk.org; dpdk-dev
><dev@dpdk.org>; David Marchand <david.marchand@redhat.com>; Bruce
>Richardson <bruce.richardson@intel.com>; Jerin Jacob Kollanukkaran
><jerinj@marvell.com>; Akhil Goyal <gakhil@marvell.com>; Maxime Coquelin
><maxime.coquelin@redhat.com>; Honnappa Nagarahalli
><Honnappa.Nagarahalli@arm.com>; Tu, Lijuan <lijuan.tu@intel.com>
>Subject: Re: [dpdk-users] [DISCUSSION] code snippet documentation
>
>On 7/23/2021 1:02 AM, Ajit Khaparde wrote:
>> On Thu, Jul 22, 2021 at 1:29 PM Thomas Monjalon <thomas@monjalon.net>
>wrote:
>>>
>>> 15/07/2021 09:01, Asaf Penso:
>>>> Hello DPDK community,
>>>>
>>>> I would like to bring up a discussion about a way to have code snippets as an
>example for proper usage.
>>>> The DPDK tree is filled with great pieces of code that are well documented
>and maintained in high quality.
>>>> I feel we are a bit behind when we talk about usage examples.
>>>>
>>>> One way, whenever we implement a new feature, is to extend one of the
>test-* under the "app" folder.
>>>> This, however, provides means to test but doesn't provide a good usage
>example.
>>>>
>>>> Another way is to check the content of the "example" folder and whenever
>we have a BIG new feature it seems like a good place.
>>>> This, however, doesn't provide a good option when we talk about small
>features.
>>>> If, for example, we extend rte_flow with an extra action then providing a full-
>blown example application is somewhat an entry barrier.
>>>>
>>>> A third option could be to document it in one of the .rst files we have.
>>>> Obviously, this requires high maintenance and no option to assure it still
>compiles.
>>>>
>>>> I'd like to propose another approach that will address the main two issues:
>remove the entry barrier and assure compilation.
>>>> In this approach, inside the "examples" folder we'll create another folder for
>"snippets".
>>>> Inside "snippets" we'll have several files per category, for
>>>> example, rte_flow_snippets.c Each .c file will include a main function that
>calls the different use cases we want to give as an example.
>>>> The purpose is not to generate traffic nor read rx/tx packets from the DPDK
>ports.
>>>> The purpose is to have a good example that compiles properly.
>>>>
>>>> Taking the rte_flow_snippets.c as an example its main function would look
>like this:
>>>>
>>>> int
>>>> main(int argc, char **argv)
>>>> {
>>>>       rte_flow_snippet_match_5tuple_and_drop();
>>>>       rte_flow_snippet_match_geneve_ope_and_rss();
>>>>       ...
>>>>       Return 0;
>>>> }
>>>
>>> I think we need to have a policy or justification about which snippet
>>> is worth to have.
>>> My thought is to avoid creating snippets which have no other value
>>> than showing a function call.
>>> I think there is a value if the context is not simple.
>>
>> I agree. Otherwise it will be cluttered with snippets.
>>
>
>I sometimes use DTS code as sample for an API, that has limited context (which I
>believe sometimes needed to understand API properly) and of course compiles
>fine.
>
>What about making mandatory to add a test to DTS for each API/feature, that
>ensures a reasonable sample for the API call.
>
>And if the intent is just to provide sample for API call without context, I believe
>test-* can help on that, it clarifies good & bad input for an API.
>

Thank you Ferruh, Ajit, and Thomas for your replies.
I think the more snippets the merrier and we can categorize them for better visibility inside the file and among different files.
Limiting would miss the point of having valid code examples for a variety of cases.
For complex code snippets I fully think that a dedicated test/example app should be written but my intention of the snippets is something simpler.
Reading only documentation is not practical enough and reviewing an example app requires too much effort. Snippet is a great tool in between.
Written DTS test can be an option but I think it is too much overhead if one just wants to show how to use a function or an object, without putting effort for the "administrative" topics around the snippet.

>>>
>>>
>>> Please could you provide a more complex example?
>>>
>>>

RE: [dpdk-users] [DISCUSSION] code snippet documentation

[Asaf Penso]

>-----Original Message-----
>From: dev <dev-bounces@dpdk.org> On Behalf Of Asaf Penso
>Sent: Sunday, July 25, 2021 7:54 AM
>To: Ferruh Yigit <ferruh.yigit@intel.com>; Ajit Khaparde
><ajit.khaparde@broadcom.com>; NBU-Contact-Thomas Monjalon
><thomas@monjalon.net>
>Cc: users@dpdk.org; dpdk-dev <dev@dpdk.org>; David Marchand
><david.marchand@redhat.com>; Bruce Richardson
><bruce.richardson@intel.com>; Jerin Jacob Kollanukkaran
><jerinj@marvell.com>; Akhil Goyal <gakhil@marvell.com>; Maxime Coquelin
><maxime.coquelin@redhat.com>; Honnappa Nagarahalli
><Honnappa.Nagarahalli@arm.com>; Tu, Lijuan <lijuan.tu@intel.com>
>Subject: Re: [dpdk-dev] [dpdk-users] [DISCUSSION] code snippet
>documentation
>
>Regards,
>Asaf Penso
>
>>-----Original Message-----
>>From: Ferruh Yigit <ferruh.yigit@intel.com>
>>Sent: Friday, July 23, 2021 3:04 PM
>>To: Ajit Khaparde <ajit.khaparde@broadcom.com>; NBU-Contact-Thomas
>>Monjalon <thomas@monjalon.net>
>>Cc: Asaf Penso <asafp@nvidia.com>; users@dpdk.org; dpdk-dev
>><dev@dpdk.org>; David Marchand <david.marchand@redhat.com>; Bruce
>>Richardson <bruce.richardson@intel.com>; Jerin Jacob Kollanukkaran
>><jerinj@marvell.com>; Akhil Goyal <gakhil@marvell.com>; Maxime Coquelin
>><maxime.coquelin@redhat.com>; Honnappa Nagarahalli
>><Honnappa.Nagarahalli@arm.com>; Tu, Lijuan <lijuan.tu@intel.com>
>>Subject: Re: [dpdk-users] [DISCUSSION] code snippet documentation
>>
>>On 7/23/2021 1:02 AM, Ajit Khaparde wrote:
>>> On Thu, Jul 22, 2021 at 1:29 PM Thomas Monjalon
><thomas@monjalon.net>
>>wrote:
>>>>
>>>> 15/07/2021 09:01, Asaf Penso:
>>>>> Hello DPDK community,
>>>>>
>>>>> I would like to bring up a discussion about a way to have code
>>>>> snippets as an
>>example for proper usage.
>>>>> The DPDK tree is filled with great pieces of code that are well
>>>>> documented
>>and maintained in high quality.
>>>>> I feel we are a bit behind when we talk about usage examples.
>>>>>
>>>>> One way, whenever we implement a new feature, is to extend one of
>>>>> the
>>test-* under the "app" folder.
>>>>> This, however, provides means to test but doesn't provide a good
>>>>> usage
>>example.
>>>>>
>>>>> Another way is to check the content of the "example" folder and
>>>>> whenever
>>we have a BIG new feature it seems like a good place.
>>>>> This, however, doesn't provide a good option when we talk about
>>>>> small
>>features.
>>>>> If, for example, we extend rte_flow with an extra action then
>>>>> providing a full-
>>blown example application is somewhat an entry barrier.
>>>>>
>>>>> A third option could be to document it in one of the .rst files we have.
>>>>> Obviously, this requires high maintenance and no option to assure
>>>>> it still
>>compiles.
>>>>>
>>>>> I'd like to propose another approach that will address the main two
>issues:
>>remove the entry barrier and assure compilation.
>>>>> In this approach, inside the "examples" folder we'll create another
>>>>> folder for
>>"snippets".
>>>>> Inside "snippets" we'll have several files per category, for
>>>>> example, rte_flow_snippets.c Each .c file will include a main
>>>>> function that
>>calls the different use cases we want to give as an example.
>>>>> The purpose is not to generate traffic nor read rx/tx packets from
>>>>> the DPDK
>>ports.
>>>>> The purpose is to have a good example that compiles properly.
>>>>>
>>>>> Taking the rte_flow_snippets.c as an example its main function
>>>>> would look
>>like this:
>>>>>
>>>>> int
>>>>> main(int argc, char **argv)
>>>>> {
>>>>>       rte_flow_snippet_match_5tuple_and_drop();
>>>>>       rte_flow_snippet_match_geneve_ope_and_rss();
>>>>>       ...
>>>>>       Return 0;
>>>>> }
>>>>
>>>> I think we need to have a policy or justification about which
>>>> snippet is worth to have.
>>>> My thought is to avoid creating snippets which have no other value
>>>> than showing a function call.
>>>> I think there is a value if the context is not simple.
>>>
>>> I agree. Otherwise it will be cluttered with snippets.
>>>
>>
>>I sometimes use DTS code as sample for an API, that has limited context
>>(which I believe sometimes needed to understand API properly) and of
>>course compiles fine.
>>
>>What about making mandatory to add a test to DTS for each API/feature,
>>that ensures a reasonable sample for the API call.
>>
>>And if the intent is just to provide sample for API call without
>>context, I believe
>>test-* can help on that, it clarifies good & bad input for an API.
>>
>
>Thank you Ferruh, Ajit, and Thomas for your replies.
>I think the more snippets the merrier and we can categorize them for better
>visibility inside the file and among different files.
>Limiting would miss the point of having valid code examples for a variety of
>cases.
>For complex code snippets I fully think that a dedicated test/example app
>should be written but my intention of the snippets is something simpler.
>Reading only documentation is not practical enough and reviewing an
>example app requires too much effort. Snippet is a great tool in between.
>Written DTS test can be an option but I think it is too much overhead if one
>just wants to show how to use a function or an object, without putting effort
>for the "administrative" topics around the snippet.
>
>>>>
>>>>
>>>> Please could you provide a more complex example?
>>>>
>>>>
A good example for a snippet would be the usage of the Flex Item.
On the one hand, it's complex enough, but on the other hand, I don't see an added value of writing all the code for packet acquisition, packet handle, verification etc.
WDYT?