[dpdk-users] rte_flow MARK action with Mellanox 100G Card to offload flow tagging to HW

[dpdk-users] rte_flow MARK action with Mellanox 100G Card to offload flow tagging to HW

[Arvind Narayanan]

Hi,

I am trying to use rte_flow's MARK action to set hash.fdir.hi field with
whatever I set via rte_flow action. I adapt the flow_filtering example and
create another function which takes 3 actions. I have also checked out this
blog post <https://www.napatech.com/hw-acceleration-via-rte_flow/> and my
implementation is similar to theirs.

I am able to create the flow without any error, however when I pass the
packets, hash.fdir.hi field in rte_mbuf is not set nor is mb->ol_flags &
PKT_RX_FDIR_ID set. Am I missing anything here?

RTE_LOG(INFO, MM, "Index %d (%d.%d) found by HW\n", bufs[i]->hash.fdir.hi,
bufs[i]->hash.fdir.hi >> 1, bufs[i]->hash.fdir.hi & 1);

stdout:
MM: Index -1 (2147483647.1) found by HW

Dell R620 with Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (6 cores)
NIC: Mellanox ConnectX-5 EN 100G  (MCX516A-CDAT)
Only one socket is there.
HT is disabled.
isolcpus=1-5
DPDK 18.08
Ubuntu 16.04 LTS
Driver MLNX_OFED_LINUX-4.4-2.0.7.0-ubuntu16.04-x86_64 was installed

For more information, this is function I use to set the MARK field (or
install the rte_flow rule).

struct rte_flow *
generate_ipv4_flow_tag(uint16_t port_id, uint16_t rx_q,
                   uint32_t src_ip, uint32_t src_mask,
                   uint32_t dest_ip, uint32_t dest_mask, uint32_t tag,
                   struct rte_flow_error *error)
{
    struct rte_flow_attr attr;
    struct rte_flow_item pattern[MAX_PATTERN_NUM];
    struct rte_flow_action action[MAX_PATTERN_NUM];
    struct rte_flow *flow = NULL;
    struct rte_flow_action_queue queue = { .index = rx_q };
    struct rte_flow_action_mark mark = { .id = tag };
    struct rte_flow_item_eth eth_spec;
    struct rte_flow_item_eth eth_mask;
    struct rte_flow_item_ipv4 ip_spec;
    struct rte_flow_item_ipv4 ip_mask;
    int res;

    memset(pattern, 0, sizeof(pattern));
    memset(action, 0, sizeof(action));

    /*
     * set the rule attribute.
     * in this case only ingress packets will be checked.
     */
    memset(&attr, 0, sizeof(struct rte_flow_attr));
    attr.ingress = 1;

    /*
     * create the action sequence.
     * one action only,  move packet to queue
     */

    action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE;
    action[0].conf = &queue;
    action[1].type = RTE_FLOW_ACTION_TYPE_MARK;
    action[1].conf = &mark;
    action[2].type = RTE_FLOW_ACTION_TYPE_END;

    /*
     * set the first level of the pattern (eth).
     * since in this example we just want to get the
     * ipv4 we set this level to allow all.
     */
    memset(&eth_spec, 0, sizeof(struct rte_flow_item_eth));
    memset(&eth_mask, 0, sizeof(struct rte_flow_item_eth));
    eth_spec.type = 0;
    eth_mask.type = 0;
    pattern[0].type = RTE_FLOW_ITEM_TYPE_ETH;
    pattern[0].spec = &eth_spec;
    pattern[0].mask = &eth_mask;

    /*
     * setting the third level of the pattern (ip).
     * in this example this is the level we care about
     * so we set it according to the parameters.
     */
    memset(&ip_spec, 0, sizeof(struct rte_flow_item_ipv4));
    memset(&ip_mask, 0, sizeof(struct rte_flow_item_ipv4));
    ip_spec.hdr.dst_addr = htonl(dest_ip);
    ip_mask.hdr.dst_addr = dest_mask;
    ip_spec.hdr.src_addr = htonl(src_ip);
    ip_mask.hdr.src_addr = src_mask;
    pattern[1].type = RTE_FLOW_ITEM_TYPE_IPV4;
    pattern[1].spec = &ip_spec;
    pattern[1].mask = &ip_mask;

    /* the final level must be always type end */
    pattern[2].type = RTE_FLOW_ITEM_TYPE_END;

    res = rte_flow_validate(port_id, &attr, pattern, action, error);
    if (!res)
        flow = rte_flow_create(port_id, &attr, pattern, action, error);

    return flow;
}


Can anyone please help me with what I may be missing?

Thanks,

Arvind

Re: [dpdk-users] rte_flow MARK action with Mellanox 100G Card to offload flow tagging to HW

[Arvind Narayanan]

I was able to solve this issue. It was my mistake. Just in case if anyone
hits this issue, make sure you install the rule correctly (e.g. 5-tuple
fields are assigned properly in the rte_flow actions/patterns). I was
performing both rte_cpu_to_be_32( htonl() ) for src and dst ip addresses
instead of using them once. This resulted in successful rule installation,
however when packets came in, they didn't match any rule.

When packets did not match any rule, they were all going to queue 0 unless
you isolate the ingress traffic to only allow pkts that match rules created
by rte_flow.

Thanks,
Arvind


On Fri, Sep 21, 2018 at 2:20 PM Arvind Narayanan <webguru2688@gmail.com>
wrote:

> Hi,
>
> I am trying to use rte_flow's MARK action to set hash.fdir.hi field with
> whatever I set via rte_flow action. I adapt the flow_filtering example and
> create another function which takes 3 actions. I have also checked out this
> blog post <https://www.napatech.com/hw-acceleration-via-rte_flow/> and my
> implementation is similar to theirs.
>
> I am able to create the flow without any error, however when I pass the
> packets, hash.fdir.hi field in rte_mbuf is not set nor is mb->ol_flags &
> PKT_RX_FDIR_ID set. Am I missing anything here?
>
> RTE_LOG(INFO, MM, "Index %d (%d.%d) found by HW\n", bufs[i]->hash.fdir.hi,
> bufs[i]->hash.fdir.hi >> 1, bufs[i]->hash.fdir.hi & 1);
>
> stdout:
> MM: Index -1 (2147483647.1) found by HW
>
> Dell R620 with Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (6 cores)
> NIC: Mellanox ConnectX-5 EN 100G  (MCX516A-CDAT)
> Only one socket is there.
> HT is disabled.
> isolcpus=1-5
> DPDK 18.08
> Ubuntu 16.04 LTS
> Driver MLNX_OFED_LINUX-4.4-2.0.7.0-ubuntu16.04-x86_64 was installed
>
> For more information, this is function I use to set the MARK field (or
> install the rte_flow rule).
>
> struct rte_flow *
> generate_ipv4_flow_tag(uint16_t port_id, uint16_t rx_q,
>                    uint32_t src_ip, uint32_t src_mask,
>                    uint32_t dest_ip, uint32_t dest_mask, uint32_t tag,
>                    struct rte_flow_error *error)
> {
>     struct rte_flow_attr attr;
>     struct rte_flow_item pattern[MAX_PATTERN_NUM];
>     struct rte_flow_action action[MAX_PATTERN_NUM];
>     struct rte_flow *flow = NULL;
>     struct rte_flow_action_queue queue = { .index = rx_q };
>     struct rte_flow_action_mark mark = { .id = tag };
>     struct rte_flow_item_eth eth_spec;
>     struct rte_flow_item_eth eth_mask;
>     struct rte_flow_item_ipv4 ip_spec;
>     struct rte_flow_item_ipv4 ip_mask;
>     int res;
>
>     memset(pattern, 0, sizeof(pattern));
>     memset(action, 0, sizeof(action));
>
>     /*
>      * set the rule attribute.
>      * in this case only ingress packets will be checked.
>      */
>     memset(&attr, 0, sizeof(struct rte_flow_attr));
>     attr.ingress = 1;
>
>     /*
>      * create the action sequence.
>      * one action only,  move packet to queue
>      */
>
>     action[0].type = RTE_FLOW_ACTION_TYPE_QUEUE;
>     action[0].conf = &queue;
>     action[1].type = RTE_FLOW_ACTION_TYPE_MARK;
>     action[1].conf = &mark;
>     action[2].type = RTE_FLOW_ACTION_TYPE_END;
>
>     /*
>      * set the first level of the pattern (eth).
>      * since in this example we just want to get the
>      * ipv4 we set this level to allow all.
>      */
>     memset(&eth_spec, 0, sizeof(struct rte_flow_item_eth));
>     memset(&eth_mask, 0, sizeof(struct rte_flow_item_eth));
>     eth_spec.type = 0;
>     eth_mask.type = 0;
>     pattern[0].type = RTE_FLOW_ITEM_TYPE_ETH;
>     pattern[0].spec = &eth_spec;
>     pattern[0].mask = &eth_mask;
>
>     /*
>      * setting the third level of the pattern (ip).
>      * in this example this is the level we care about
>      * so we set it according to the parameters.
>      */
>     memset(&ip_spec, 0, sizeof(struct rte_flow_item_ipv4));
>     memset(&ip_mask, 0, sizeof(struct rte_flow_item_ipv4));
>     ip_spec.hdr.dst_addr = htonl(dest_ip);
>     ip_mask.hdr.dst_addr = dest_mask;
>     ip_spec.hdr.src_addr = htonl(src_ip);
>     ip_mask.hdr.src_addr = src_mask;
>     pattern[1].type = RTE_FLOW_ITEM_TYPE_IPV4;
>     pattern[1].spec = &ip_spec;
>     pattern[1].mask = &ip_mask;
>
>     /* the final level must be always type end */
>     pattern[2].type = RTE_FLOW_ITEM_TYPE_END;
>
>     res = rte_flow_validate(port_id, &attr, pattern, action, error);
>     if (!res)
>         flow = rte_flow_create(port_id, &attr, pattern, action, error);
>
>     return flow;
> }
>
>
> Can anyone please help me with what I may be missing?
>
> Thanks,
>
> Arvind
>
>