[dpdk-users] DPDK application that sends rules to the NIC

[dpdk-users] DPDK application that sends rules to the NIC

[george.dit]

Hi all,

I would like to implement a DPDK application that dumps a set of rules to a
Mellanox ConnectX-5 NIC and starts a number of threads (each on a different
core) that receive the classified packets according to the rules.
The rules are of the form src IP X and dst IP Y --> Send to core Z.

Is there such an application (or similar) in the source tree? Does the
Mellanox NIC require any special treatment or is there a unified API to do
this job regardless of the NIC? Which DPDK version do you propose?

Thanks,
-- 
Georgios Katsikas
Industrial Ph.D. Student
Network Intelligence Group
Decision, Networks, and Analytics (DNA) Lab
RISE SICS
E-Mail:  georgios.katsikas@ri.se

Re: [dpdk-users] DPDK application that sends rules to the NIC

[Shahaf Shuler]

Hi Georgios,

Wednesday, September 13, 2017 11:09 AM, Georgios Katsikas:
> Hi all,
> 
> I would like to implement a DPDK application that dumps a set of rules to a
> Mellanox ConnectX-5 NIC and starts a number of threads (each on a
> different
> core) that receive the classified packets according to the rules.
> The rules are of the form src IP X and dst IP Y --> Send to core Z.
> 
> Is there such an application (or similar) in the source tree? 

You can use testpmd for that. Run in isolate mode and add rte_flow rules to direct IP packets to the specific queue that you want. 

> Does the Mellanox
> NIC require any special treatment or is there a unified API to do this job
> regardless of the NIC? 

There Is a unified API. look into rte_flow:
http://dpdk.org/doc/guides/prog_guide/rte_flow.html

> Which DPDK version do you propose?

The latest one - v17.08 

> 
> Thanks,
> --
> Georgios Katsikas
> Industrial Ph.D. Student
> Network Intelligence Group
> Decision, Networks, and Analytics (DNA) Lab RISE SICS
> E-Mail:  georgios.katsikas@ri.se

Re: [dpdk-users] DPDK application that sends rules to the NIC

[george.dit]

Hi Shahaf,

Thanks for the information, I am looking into it :)
I had some compilation issues when trying to port rte_flow from DPDK 17.02
into an external application but these issues go away if I use 17.08.

Best regards,
Georgios

On Thu, Sep 14, 2017 at 3:26 AM, Shahaf Shuler <shahafs@mellanox.com> wrote:

> Hi Georgios,
>
> Wednesday, September 13, 2017 11:09 AM, Georgios Katsikas:
> > Hi all,
> >
> > I would like to implement a DPDK application that dumps a set of rules
> to a
> > Mellanox ConnectX-5 NIC and starts a number of threads (each on a
> > different
> > core) that receive the classified packets according to the rules.
> > The rules are of the form src IP X and dst IP Y --> Send to core Z.
> >
> > Is there such an application (or similar) in the source tree?
>
> You can use testpmd for that. Run in isolate mode and add rte_flow rules
> to direct IP packets to the specific queue that you want.
>
> > Does the Mellanox
> > NIC require any special treatment or is there a unified API to do this
> job
> > regardless of the NIC?
>
> There Is a unified API. look into rte_flow:
> http://dpdk.org/doc/guides/prog_guide/rte_flow.html
>
> > Which DPDK version do you propose?
>
> The latest one - v17.08
>
> >
> > Thanks,
> > --
> > Georgios Katsikas
> > Industrial Ph.D. Student
> > Network Intelligence Group
> > Decision, Networks, and Analytics (DNA) Lab RISE SICS
> > E-Mail:  georgios.katsikas@ri.se
>



-- 
Georgios Katsikas
Industrial Ph.D. Student
Network Intelligence Group
Decision, Networks, and Analytics (DNA) Lab
RISE SICS
E-Mail:  georgios.katsikas@ri.se

Re: [dpdk-users] DPDK application that sends rules to the NIC

[george.dit]

Dear all,

I am implementing my own DPDK application to program a Mellanox NIC (mlx5
driver) but I have some issues.
Specifically I compose a simple rule that matches only the Ethernet
header's type field using value 800 (Ethernet frames that encapsulate IPv4
packets).
Then I compose an action RTE_FLOW_ACTION_TYPE_QUEUE which redirects the
matched packets to queue index 0.

Using test-pmd, this rule is: "flow create 0 ingress pattern eth type is
800 / end actions queue index 0 / end" and I get a "Flow rule #0 created"
message.
In my application I compose the same rule using:

PATTERNS

struct rte_flow_item_eth flow_item_eth_type_ipv4 = {
        .dst = {
            .addr_bytes = { 0 }
        },
        .src = {
            .addr_bytes = { 0 }
        },
        .type = RTE_BE16(ETHER_TYPE_IPv4)                // from rte_ether.h
    };

    struct rte_flow_item_eth flow_item_eth_mask_type_ipv4 = {
        .dst = {
            .addr_bytes = { 0 }
        },
        .src = {
            .addr_bytes = { 0 }
        },
        .type = 0xFFFF
 // match only the 'type' filed
    };

struct rte_flow_item patterns[] = {
        {
            .type = RTE_FLOW_ITEM_TYPE_ETH,
            .spec = &flow_item_eth_type_ipv4,
            .last = NULL,
            .mask = &flow_item_eth_mask_type_ipv4,
        },
        {
            .type = RTE_FLOW_ITEM_TYPE_END,
            .spec = NULL,
            .last = NULL,
            .mask = NULL,
        }
    };

ACTIONS

struct rte_flow_action_queue queue_conf;
queue_conf.index = 0;

struct rte_flow_action actions[] =
    {
        {
            .type = RTE_FLOW_ACTION_TYPE_QUEUE,
            .conf = &queue_conf
        },
        {
            .type = RTE_FLOW_ACTION_TYPE_END,
            .conf = NULL
        }
    };

PROBLEM

When I pass this rule to rte_flow_validate(...) it is successfully
validated, but rte_flow_create() crashes although it get the very same
arguments.
If I replace my Mellanox NIC with an Intel 82599 (using DPDK's ixgbe
driver), then the validate function returns error: "Caught error type 9
(specific pattern item): Not supported by L2 tunnel filter".
The error reported by the Intel driver is weird because there is no
tunneling.

I guess some value assignments require strict format and are sensitive
(e.g., big/little endian) so I would really appreciate your help.

Thanks in advance and best regards,
Georgios



On Thu, Sep 14, 2017 at 4:37 AM, <george.dit@gmail.com> wrote:

> Hi Shahaf,
>
> Thanks for the information, I am looking into it :)
> I had some compilation issues when trying to port rte_flow from DPDK 17.02
> into an external application but these issues go away if I use 17.08.
>
> Best regards,
> Georgios
>
> On Thu, Sep 14, 2017 at 3:26 AM, Shahaf Shuler <shahafs@mellanox.com>
> wrote:
>
>> Hi Georgios,
>>
>> Wednesday, September 13, 2017 11:09 AM, Georgios Katsikas:
>> > Hi all,
>> >
>> > I would like to implement a DPDK application that dumps a set of rules
>> to a
>> > Mellanox ConnectX-5 NIC and starts a number of threads (each on a
>> > different
>> > core) that receive the classified packets according to the rules.
>> > The rules are of the form src IP X and dst IP Y --> Send to core Z.
>> >
>> > Is there such an application (or similar) in the source tree?
>>
>> You can use testpmd for that. Run in isolate mode and add rte_flow rules
>> to direct IP packets to the specific queue that you want.
>>
>> > Does the Mellanox
>> > NIC require any special treatment or is there a unified API to do this
>> job
>> > regardless of the NIC?
>>
>> There Is a unified API. look into rte_flow:
>> http://dpdk.org/doc/guides/prog_guide/rte_flow.html
>>
>> > Which DPDK version do you propose?
>>
>> The latest one - v17.08
>>
>> >
>> > Thanks,
>> > --
>> > Georgios Katsikas
>> > Industrial Ph.D. Student
>> > Network Intelligence Group
>> > Decision, Networks, and Analytics (DNA) Lab RISE SICS
>> > E-Mail:  georgios.katsikas@ri.se
>>
>
>
>
> --
> Georgios Katsikas
> Industrial Ph.D. Student
> Network Intelligence Group
> Decision, Networks, and Analytics (DNA) Lab
> RISE SICS
> E-Mail:  georgios.katsikas@ri.se
>



-- 
Georgios Katsikas
Industrial Ph.D. Student
Network Intelligence Group
Decision, Networks, and Analytics (DNA) Lab
RISE SICS
E-Mail:  georgios.katsikas@ri.se

Re: [dpdk-users] DPDK application that sends rules to the NIC

[Nélio Laranjeiro]

Hi Georgios,

On Mon, Sep 18, 2017 at 05:26:08AM -0700, george.dit@gmail.com wrote:
> Dear all,
> 
> I am implementing my own DPDK application to program a Mellanox NIC (mlx5
> driver) but I have some issues.
> Specifically I compose a simple rule that matches only the Ethernet
> header's type field using value 800 (Ethernet frames that encapsulate IPv4
> packets).
> Then I compose an action RTE_FLOW_ACTION_TYPE_QUEUE which redirects the
> matched packets to queue index 0.
> 
> Using test-pmd, this rule is: "flow create 0 ingress pattern eth type is
> 800 / end actions queue index 0 / end" and I get a "Flow rule #0 created"
> message.
> In my application I compose the same rule using:
> 
> PATTERNS
> 
> struct rte_flow_item_eth flow_item_eth_type_ipv4 = {
>         .dst = {
>             .addr_bytes = { 0 }
>         },
>         .src = {
>             .addr_bytes = { 0 }
>         },
>         .type = RTE_BE16(ETHER_TYPE_IPv4)                // from rte_ether.h
>     };
> 
>     struct rte_flow_item_eth flow_item_eth_mask_type_ipv4 = {
>         .dst = {
>             .addr_bytes = { 0 }
>         },
>         .src = {
>             .addr_bytes = { 0 }
>         },
>         .type = 0xFFFF
>  // match only the 'type' filed
>     };
> 
> struct rte_flow_item patterns[] = {
>         {
>             .type = RTE_FLOW_ITEM_TYPE_ETH,
>             .spec = &flow_item_eth_type_ipv4,
>             .last = NULL,
>             .mask = &flow_item_eth_mask_type_ipv4,
>         },
>         {
>             .type = RTE_FLOW_ITEM_TYPE_END,
>             .spec = NULL,
>             .last = NULL,
>             .mask = NULL,
>         }
>     };
> 
> ACTIONS
> 
> struct rte_flow_action_queue queue_conf;
> queue_conf.index = 0;
> 
> struct rte_flow_action actions[] =
>     {
>         {
>             .type = RTE_FLOW_ACTION_TYPE_QUEUE,
>             .conf = &queue_conf
>         },
>         {
>             .type = RTE_FLOW_ACTION_TYPE_END,
>             .conf = NULL
>         }
>     };
> 
> PROBLEM
> 
> When I pass this rule to rte_flow_validate(...) it is successfully
> validated, but rte_flow_create() crashes although it get the very same
> arguments.

Can you explain a little more what do you mean by "crash" ?

> If I replace my Mellanox NIC with an Intel 82599 (using DPDK's ixgbe
> driver), then the validate function returns error: "Caught error type 9
> (specific pattern item): Not supported by L2 tunnel filter".
> The error reported by the Intel driver is weird because there is no
> tunneling.
>
> I guess some value assignments require strict format and are sensitive
> (e.g., big/little endian) so I would really appreciate your help.
> 
> Thanks in advance and best regards,
> Georgios

With so few informations it is not easy to help, from the code above I
don't see anything wrong.  To help can you answer the following points:

 - which version of DPDK are you using,
 - which version of MLNX_OFED,
 - Is it possible to share your snippet of code (just the pattern/action
   allocation) ?

Regards,

-- 
Nélio Laranjeiro
6WIND

Re: [dpdk-users] DPDK application that sends rules to the NIC

[george.dit]

Hi Nelio,

Thanks for the prompt reply.
I tried not to overwhelm the e-mail but you are right, there is some
missing information.

DPDK version: 17.08
MLNX_OFED: MLNX_OFED_LINUX-3.4-2.0.0.0-ubuntu16.04-x86_64.tgz

here is my code snippet:

    //////////////////////////////////////////////////////////////////////
    // Flow Attributes: Only ingress rules are currently supported
    //////////////////////////////////////////////////////////////////////
    struct rte_flow_attr attr = {
        .group    = 0,
        .priority = 0,
        .ingress  = 1,
        .egress   = 0,
        .reserved = 0,
    };
    //////////////////////////////////////////////////////////////////////

    //////////////////////////////////////////////////////////////////////
    // Flow Patterns
    //////////////////////////////////////////////////////////////////////
    // L2 - Ethernet type is always IPv4
    struct rte_flow_item_eth flow_item_eth_type_ipv4 = {
        .dst = {
            .addr_bytes = { 0 }
        },
        .src = {
            .addr_bytes = { 0 }
        },
        .type = RTE_BE16(ETHER_TYPE_IPv4)
    };

    struct rte_flow_item_eth flow_item_eth_mask_type_ipv4 = {
        .dst = {
            .addr_bytes = { 0 }
        },
        .src = {
            .addr_bytes = { 0 }
        },
        .type = 0xFFFF
    };
    //////////////////////////////////////////////////////////////////////

    // Compose the pattern
    struct rte_flow_item patterns[] = {
        {
            .type = RTE_FLOW_ITEM_TYPE_ETH,
            .spec = (void *) &flow_item_eth_type_ipv4,
            .last = NULL,
            .mask = (void *) &flow_item_eth_mask_type_ipv4,
        },
        {
            .type = RTE_FLOW_ITEM_TYPE_END,
            .spec = NULL,
            .last = NULL,
            .mask = NULL,
        }
    };

    //////////////////////////////////////////////////////////////////////
    // Flow Actions
    //////////////////////////////////////////////////////////////////////
    // Only core dispatching is currently supported
    struct rte_flow_action_queue queue_conf;
    queue_conf.index = RTE_BE16(queue_index);
    //////////////////////////////////////////////////////////////////////

    struct rte_flow_action actions[] =
    {
        {
            .type = RTE_FLOW_ACTION_TYPE_QUEUE,
            .conf = &queue_conf
        },
        {
            .type = RTE_FLOW_ACTION_TYPE_END,
            .conf = NULL
        }
    };

    //////////////////////////////////////////////////////////////////////
    // Validate Flow
    //////////////////////////////////////////////////////////////////////
    struct rte_flow_error error;
    /* Poisoning to make sure PMDs update it in case of error. */
    memset(&error, 0x11, sizeof(error));

    uint8_t port_id = 0;
    int ret = rte_flow_validate(port_id, &attr, patterns, actions, &error);
    if (ret < 0) {
        flow_rule_complain(&error);
        return false;
    }

    errh->message("Flow rule validated");                       // Up until
here everything works

    //////////////////////////////////////////////////////////////////////
    // Create Flow
    //////////////////////////////////////////////////////////////////////
    struct rte_flow  *flow = NULL;

    // Create a DPDK flow
    flow = rte_flow_create(port_id, &attr, patterns, actions, &error);
  // At this line I get segmentation fault
    if (flow == NULL) {
        flow_rule_complain(&error);
        return false;
    }


SEG FAULT:
Thread 1 "click" received signal SIGSEGV, Segmentation fault.
0x00000000006a8f74 in priv_flow_create_action_queue (flow=0x7fffffffcc90,
flow=0x7fffffffcc90, error=<optimized out>, action=0x7fffffffcd90,
priv=0x7ffbfffeb180)
    at /opt/dpdk/drivers/net/mlx5/mlx5_flow.c:1085
1085 rxq = container_of((*priv->rxqs)[action->queues[i]],


I hope this helps you to understand the issue.

Best regards,
Georgios

On Mon, Sep 18, 2017 at 6:20 AM, Nélio Laranjeiro <
nelio.laranjeiro@6wind.com> wrote:

> Hi Georgios,
>
> On Mon, Sep 18, 2017 at 05:26:08AM -0700, george.dit@gmail.com wrote:
> > Dear all,
> >
> > I am implementing my own DPDK application to program a Mellanox NIC (mlx5
> > driver) but I have some issues.
> > Specifically I compose a simple rule that matches only the Ethernet
> > header's type field using value 800 (Ethernet frames that encapsulate
> IPv4
> > packets).
> > Then I compose an action RTE_FLOW_ACTION_TYPE_QUEUE which redirects the
> > matched packets to queue index 0.
> >
> > Using test-pmd, this rule is: "flow create 0 ingress pattern eth type is
> > 800 / end actions queue index 0 / end" and I get a "Flow rule #0 created"
> > message.
> > In my application I compose the same rule using:
> >
> > PATTERNS
> >
> > struct rte_flow_item_eth flow_item_eth_type_ipv4 = {
> >         .dst = {
> >             .addr_bytes = { 0 }
> >         },
> >         .src = {
> >             .addr_bytes = { 0 }
> >         },
> >         .type = RTE_BE16(ETHER_TYPE_IPv4)                // from
> rte_ether.h
> >     };
> >
> >     struct rte_flow_item_eth flow_item_eth_mask_type_ipv4 = {
> >         .dst = {
> >             .addr_bytes = { 0 }
> >         },
> >         .src = {
> >             .addr_bytes = { 0 }
> >         },
> >         .type = 0xFFFF
> >  // match only the 'type' filed
> >     };
> >
> > struct rte_flow_item patterns[] = {
> >         {
> >             .type = RTE_FLOW_ITEM_TYPE_ETH,
> >             .spec = &flow_item_eth_type_ipv4,
> >             .last = NULL,
> >             .mask = &flow_item_eth_mask_type_ipv4,
> >         },
> >         {
> >             .type = RTE_FLOW_ITEM_TYPE_END,
> >             .spec = NULL,
> >             .last = NULL,
> >             .mask = NULL,
> >         }
> >     };
> >
> > ACTIONS
> >
> > struct rte_flow_action_queue queue_conf;
> > queue_conf.index = 0;
> >
> > struct rte_flow_action actions[] =
> >     {
> >         {
> >             .type = RTE_FLOW_ACTION_TYPE_QUEUE,
> >             .conf = &queue_conf
> >         },
> >         {
> >             .type = RTE_FLOW_ACTION_TYPE_END,
> >             .conf = NULL
> >         }
> >     };
> >
> > PROBLEM
> >
> > When I pass this rule to rte_flow_validate(...) it is successfully
> > validated, but rte_flow_create() crashes although it get the very same
> > arguments.
>
> Can you explain a little more what do you mean by "crash" ?
>
> > If I replace my Mellanox NIC with an Intel 82599 (using DPDK's ixgbe
> > driver), then the validate function returns error: "Caught error type 9
> > (specific pattern item): Not supported by L2 tunnel filter".
> > The error reported by the Intel driver is weird because there is no
> > tunneling.
> >
> > I guess some value assignments require strict format and are sensitive
> > (e.g., big/little endian) so I would really appreciate your help.
> >
> > Thanks in advance and best regards,
> > Georgios
>
> With so few informations it is not easy to help, from the code above I
> don't see anything wrong.  To help can you answer the following points:
>
>  - which version of DPDK are you using,
>  - which version of MLNX_OFED,
>  - Is it possible to share your snippet of code (just the pattern/action
>    allocation) ?
>
> Regards,
>
> --
> Nélio Laranjeiro
> 6WIND
>



-- 
Georgios Katsikas
Industrial Ph.D. Student
Network Intelligence Group
Decision, Networks, and Analytics (DNA) Lab
RISE SICS
E-Mail:  georgios.katsikas@ri.se

Re: [dpdk-users] DPDK application that sends rules to the NIC

[Nélio Laranjeiro]

On Mon, Sep 18, 2017 at 06:38:36AM -0700, george.dit@gmail.com wrote:
>    Hi Nelio,
>    Thanks for the prompt reply.
>    I tried not to overwhelm the e-mail but you are right, there is some
>    missing information.
>    DPDK version: 17.08
>    MLNX_OFED: MLNX_OFED_LINUX-3.4-2.0.0.0-ubuntu16.04-x86_64.tgz

According to the documentation, you should use Mellanox OFED version:
4.1 [1], can you test with the correct version?

On my side, I've tested your snippet by replacing queue_index variable
by 0.  It work perfectly, I suppose your issue comes from the wrong
verbs version.

Regards,

[1] http://dpdk.org/doc/guides/nics/mlx5.html

-- 
Nélio Laranjeiro
6WIND

Re: [dpdk-users] DPDK application that sends rules to the NIC

[george.dit]

Hi,

My bad, I didn't notice this detail.
I will update and post my new results.

Thanks,
Georgios

On Mon, Sep 18, 2017 at 7:55 AM, Nélio Laranjeiro <
nelio.laranjeiro@6wind.com> wrote:

> On Mon, Sep 18, 2017 at 06:38:36AM -0700, george.dit@gmail.com wrote:
> >    Hi Nelio,
> >    Thanks for the prompt reply.
> >    I tried not to overwhelm the e-mail but you are right, there is some
> >    missing information.
> >    DPDK version: 17.08
> >    MLNX_OFED: MLNX_OFED_LINUX-3.4-2.0.0.0-ubuntu16.04-x86_64.tgz
>
> According to the documentation, you should use Mellanox OFED version:
> 4.1 [1], can you test with the correct version?
>
> On my side, I've tested your snippet by replacing queue_index variable
> by 0.  It work perfectly, I suppose your issue comes from the wrong
> verbs version.
>
> Regards,
>
> [1] http://dpdk.org/doc/guides/nics/mlx5.html
>
> --
> Nélio Laranjeiro
> 6WIND
>



-- 
Georgios Katsikas
Industrial Ph.D. Student
Network Intelligence Group
Decision, Networks, and Analytics (DNA) Lab
RISE SICS
E-Mail:  georgios.katsikas@ri.se

Re: [dpdk-users] DPDK application that sends rules to the NIC

[george.dit]

Hi again,

I just wanted to confirm that after upgrading to OFED 4.1, my application
works.

Thanks again,
Georgios

On Mon, Sep 18, 2017 at 7:58 AM, <george.dit@gmail.com> wrote:

> Hi,
>
> My bad, I didn't notice this detail.
> I will update and post my new results.
>
> Thanks,
> Georgios
>
> On Mon, Sep 18, 2017 at 7:55 AM, Nélio Laranjeiro <
> nelio.laranjeiro@6wind.com> wrote:
>
>> On Mon, Sep 18, 2017 at 06:38:36AM -0700, george.dit@gmail.com wrote:
>> >    Hi Nelio,
>> >    Thanks for the prompt reply.
>> >    I tried not to overwhelm the e-mail but you are right, there is some
>> >    missing information.
>> >    DPDK version: 17.08
>> >    MLNX_OFED: MLNX_OFED_LINUX-3.4-2.0.0.0-ubuntu16.04-x86_64.tgz
>>
>> According to the documentation, you should use Mellanox OFED version:
>> 4.1 [1], can you test with the correct version?
>>
>> On my side, I've tested your snippet by replacing queue_index variable
>> by 0.  It work perfectly, I suppose your issue comes from the wrong
>> verbs version.
>>
>> Regards,
>>
>> [1] http://dpdk.org/doc/guides/nics/mlx5.html
>>
>> --
>> Nélio Laranjeiro
>> 6WIND
>>
>
>
>
> --
> Georgios Katsikas
> Industrial Ph.D. Student
> Network Intelligence Group
> Decision, Networks, and Analytics (DNA) Lab
> RISE SICS
> E-Mail:  georgios.katsikas@ri.se
>



-- 
Georgios Katsikas
Industrial Ph.D. Student
Network Intelligence Group
Decision, Networks, and Analytics (DNA) Lab
RISE SICS
E-Mail:  georgios.katsikas@ri.se