How to debug receiving error or "rte_eth_stats.ierrors" ...

How to debug receiving error or "rte_eth_stats.ierrors" ...

[Ruslan R. Laishev]

[-- Attachment #1: Type: text/html, Size: 372 bytes --]

Re: How to debug receiving error or "rte_eth_stats.ierrors" ...

[Pavel Vazharov]

[-- Attachment #1: Type: text/plain, Size: 1945 bytes --]

Few questions:
1. Does the sending NIC support IP and TCP/UDP checksum offloading? I mean,
if these flags set?
    struct rte_eth_dev_info dev_info;

    rte_eth_dev_info_get(cfg.nic_port_id_, &dev_info);
    constexpr auto rxcsum = DEV_RX_OFFLOAD_CHECKSUM;

    constexpr auto l3csum = DEV_TX_OFFLOAD_IPV4_CKSUM;

    constexpr auto l4csum = DEV_TX_OFFLOAD_TCP_CKSUM |
DEV_TX_OFFLOAD_UDP_CKSUM;
    dev_rx_csum_          = ((dev_info.tx_offload_capa & rxcsum) ==
rxcsum);
    dev_tx_csum_l3_       = ((dev_info.tx_offload_capa & l3csum) ==
l3csum);
    dev_tx_csum_l4_       = ((dev_info.tx_offload_capa & l4csum) == l4csum);

2. Do you "tell" the sending NIC to do the checksum calculations before
sending the packets? I mean, do you do something like this for outgoing
packets?
    if (offl.ip_csum) {

        pkt->ol_flags |= PKT_TX_IP_CKSUM | PKT_TX_IPV4;

        pkt->l2_len = RTE_ETHER_HDR_LEN;

        pkt->l3_len = ih_len;

    }

    if (offl.tcp_csum) {

        pkt->ol_flags |= PKT_TX_TCP_CKSUM;

        pkt->l2_len = RTE_ETHER_HDR_LEN;

        pkt->l3_len = ih_len;

    }

    if (offl.udp_csum) {

        pkt->ol_flags |= PKT_TX_UDP_CKSUM;

        pkt->l2_len = RTE_ETHER_HDR_LEN;

        pkt->l3_len = ih_len;

    }

3. Is the receiving side also a DPDK application? If it is, and if the NIC
there supports checksum offloading you can check the flags of the received
packets to see if the receiving NIC has detected checksum errors.
pkt->ol_flags & (PKT_RX_IP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD)

Hope some of the above helps.

On Thu, Dec 8, 2022 at 9:36 AM Ruslan R. Laishev <zator@yandex.ru> wrote:

> Hello !
>
> I wrote too small apps to send and receive  ethernet/ip/udp
> frame/packet/dg, so on received side I  see next situation:
> number of in errors is equally a number of received packets. The test
> packet I made manually,  set offloads  IP/UDP checkusm.
> Is there what I'm need to check additionally ?
>
>

[-- Attachment #2: Type: text/html, Size: 3769 bytes --]

Re: How to debug receiving error or "rte_eth_stats.ierrors" ...

[Ruslan R. Laishev]

[-- Attachment #1: Type: text/html, Size: 10467 bytes --]

Re: How to debug receiving error or "rte_eth_stats.ierrors" ...

[Pavel Vazharov]

[-- Attachment #1: Type: text/plain, Size: 4799 bytes --]

Hi,

I see you configure the device so I assume it has the capabilities to do
the checksum offloading.

However, do you "tell" the device to calculate the checksums for every
packet. I mean, point 2 from my previous email where certain flags are set
in the `ol_flags` for every Tx packet.
I think this is needed in order to tell the device whether you want it to
calculate the checksum for the given packet or you want it to skip the
checksum calculation.
You need something like this for every packet before calling
`rte_eth_tx_burst` for the packet(s).

pkt->ol_flags |= PKT_TX_IP_CKSUM | PKT_TX_IPV4 | PKT_TX_UDP_CKSUM;
pkt->l2_len = RTE_ETHER_HDR_LEN;
pkt->l3_len = <ip header length>; // ((hdr.version_ihl & 0x0F) * 4U);


These comments from lib/librte_mbuf/rte_mbuf_core.h explain the flags
/**

 * Bits 52+53 used for L4 packet type with checksum enabled: 00: Reserved,

 * 01: TCP checksum, 10: SCTP checksum, 11: UDP checksum. To use hardware

 * L4 checksum offload, the user needs to:

 *  - fill l2_len and l3_len in mbuf

 *  - set the flags PKT_TX_TCP_CKSUM, PKT_TX_SCTP_CKSUM or PKT_TX_UDP_CKSUM

 *  - set the flag PKT_TX_IPV4 or PKT_TX_IPV6

 */
/**

 * Offload the IP checksum in the hardware. The flag PKT_TX_IPV4 should

 * also be set by the application, although a PMD will only check

 * PKT_TX_IP_CKSUM.

 *  - fill the mbuf offload information: l2_len, l3_len

 */

On Thu, Dec 8, 2022 at 10:57 AM Ruslan R. Laishev <zator@yandex.ru> wrote:

> Hello, Paul!
>
> Thanks for the answer.
>
>
> I set offloads as follows:
> ...
>
> #define RTE_ETH_TX_OFFLOAD_IPV4_CKSUM       RTE_BIT64(1)
>
> #define RTE_ETH_TX_OFFLOAD_UDP_CKSUM        RTE_BIT64(2)
>
> #define RTE_ETH_TX_OFFLOAD_TCP_CKSUM        RTE_BIT64(3)
>
> ...
>
> *static* uint64_t s_offloads = {
>
> 		RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | RTE_ETH_TX_OFFLOAD_UDP_CKSUM | RTE_ETH_TX_OFFLOAD_TCP_CKSUM};
>
>
>
> 		*if* ( l_dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE )
>
> 			l_port_conf.txmode.offloads |= RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
>
>
>
> 		l_port_conf.txmode.offloads |= s_offloads;
>
>
> l_dev_info.tx_offload_capa = 0x000080bf
> l_port_conf.txmode.offloads = 0x0000000e - it's before rte_eth_dev_configure(
> ( ... &l_port_conf ...)
>
>
>
> The receiver - is  DPDK application. Do I'm need set offloads at receiver
> site to eliminate "ierrors" ?
>
>
>
>
>
> 08.12.2022, 10:54, "Pavel Vazharov" <freakpv@gmail.com>:
>
> Few questions:
> 1. Does the sending NIC support IP and TCP/UDP checksum offloading? I
> mean, if these flags set?
>     struct rte_eth_dev_info dev_info;
>
>     rte_eth_dev_info_get(cfg.nic_port_id_, &dev_info);
>     constexpr auto rxcsum = DEV_RX_OFFLOAD_CHECKSUM;
>
>     constexpr auto l3csum = DEV_TX_OFFLOAD_IPV4_CKSUM;
>
>     constexpr auto l4csum = DEV_TX_OFFLOAD_TCP_CKSUM |
> DEV_TX_OFFLOAD_UDP_CKSUM;
>     dev_rx_csum_          = ((dev_info.tx_offload_capa & rxcsum) ==
> rxcsum);
>     dev_tx_csum_l3_       = ((dev_info.tx_offload_capa & l3csum) ==
> l3csum);
>     dev_tx_csum_l4_       = ((dev_info.tx_offload_capa & l4csum) ==
> l4csum);
>
> 2. Do you "tell" the sending NIC to do the checksum calculations before
> sending the packets? I mean, do you do something like this for outgoing
> packets?
>     if (offl.ip_csum) {
>
>         pkt->ol_flags |= PKT_TX_IP_CKSUM | PKT_TX_IPV4;
>
>         pkt->l2_len = RTE_ETHER_HDR_LEN;
>
>         pkt->l3_len = ih_len;
>
>     }
>
>     if (offl.tcp_csum) {
>
>         pkt->ol_flags |= PKT_TX_TCP_CKSUM;
>
>         pkt->l2_len = RTE_ETHER_HDR_LEN;
>
>         pkt->l3_len = ih_len;
>
>     }
>
>     if (offl.udp_csum) {
>
>         pkt->ol_flags |= PKT_TX_UDP_CKSUM;
>
>         pkt->l2_len = RTE_ETHER_HDR_LEN;
>
>         pkt->l3_len = ih_len;
>
>     }
>
> 3. Is the receiving side also a DPDK application? If it is, and if the NIC
> there supports checksum offloading you can check the flags of the received
> packets to see if the receiving NIC has detected checksum errors.
> pkt->ol_flags & (PKT_RX_IP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD)
>
> Hope some of the above helps.
>
> On Thu, Dec 8, 2022 at 9:36 AM Ruslan R. Laishev <zator@yandex.ru> wrote:
>
> Hello !
>
> I wrote too small apps to send and receive  ethernet/ip/udp
> frame/packet/dg, so on received side I  see next situation:
> number of in errors is equally a number of received packets. The test
> packet I made manually,  set offloads  IP/UDP checkusm.
> Is there what I'm need to check additionally ?
>
>
>
>
> ---
> С уважением,
> Ruslan R. Laishev
> OpenVMS bigot, natural born system/network progger, C contractor.
> +79013163222
> +79910009922
>
>

[-- Attachment #2: Type: text/html, Size: 14136 bytes --]

Re: How to debug receiving error or "rte_eth_stats.ierrors" ...

[Ruslan R. Laishev]

[-- Attachment #1: Type: text/html, Size: 13956 bytes --]

Re: How to debug receiving error or "rte_eth_stats.ierrors" ...

[Pavel Vazharov]

[-- Attachment #1: Type: text/plain, Size: 5339 bytes --]

Glad it helped.

On Thu, Dec 8, 2022 at 11:31 AM Ruslan R. Laishev <zator@yandex.ru> wrote:

> Hello, Paul!
>
> Thanks a lot for u great help!
>
> I did changes in the code as u have advised and 'ierrors' went away!
>
>
> 08.12.2022, 12:17, "Pavel Vazharov" <freakpv@gmail.com>:
>
> Hi,
>
> I see you configure the device so I assume it has the capabilities to do
> the checksum offloading.
>
> However, do you "tell" the device to calculate the checksums for every
> packet. I mean, point 2 from my previous email where certain flags are set
> in the `ol_flags` for every Tx packet.
> I think this is needed in order to tell the device whether you want it to
> calculate the checksum for the given packet or you want it to skip the
> checksum calculation.
> You need something like this for every packet before calling
> `rte_eth_tx_burst` for the packet(s).
>
> pkt->ol_flags |= PKT_TX_IP_CKSUM | PKT_TX_IPV4 | PKT_TX_UDP_CKSUM;
> pkt->l2_len = RTE_ETHER_HDR_LEN;
> pkt->l3_len = <ip header length>; // ((hdr.version_ihl & 0x0F) * 4U);
>
>
> These comments from lib/librte_mbuf/rte_mbuf_core.h explain the flags
> /**
>
>  * Bits 52+53 used for L4 packet type with checksum enabled: 00: Reserved,
>
>  * 01: TCP checksum, 10: SCTP checksum, 11: UDP checksum. To use hardware
>
>  * L4 checksum offload, the user needs to:
>
>  *  - fill l2_len and l3_len in mbuf
>
>  *  - set the flags PKT_TX_TCP_CKSUM, PKT_TX_SCTP_CKSUM or
> PKT_TX_UDP_CKSUM
>  *  - set the flag PKT_TX_IPV4 or PKT_TX_IPV6
>
>  */
> /**
>
>  * Offload the IP checksum in the hardware. The flag PKT_TX_IPV4 should
>
>  * also be set by the application, although a PMD will only check
>
>  * PKT_TX_IP_CKSUM.
>
>  *  - fill the mbuf offload information: l2_len, l3_len
>
>  */
>
> On Thu, Dec 8, 2022 at 10:57 AM Ruslan R. Laishev <zator@yandex.ru> wrote:
>
> Hello, Paul!
>
> Thanks for the answer.
>
>
> I set offloads as follows:
> ...
>
> #define RTE_ETH_TX_OFFLOAD_IPV4_CKSUM       RTE_BIT64(1)
>
> #define RTE_ETH_TX_OFFLOAD_UDP_CKSUM        RTE_BIT64(2)
>
> #define RTE_ETH_TX_OFFLOAD_TCP_CKSUM        RTE_BIT64(3)
>
> ...
>
> *static* uint64_t s_offloads = {
>
> 		RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | RTE_ETH_TX_OFFLOAD_UDP_CKSUM | RTE_ETH_TX_OFFLOAD_TCP_CKSUM};
>
>
>
> 		*if* ( l_dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE )
>
> 			l_port_conf.txmode.offloads |= RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE;
>
>
>
> 		l_port_conf.txmode.offloads |= s_offloads;
>
>
> l_dev_info.tx_offload_capa = 0x000080bf
> l_port_conf.txmode.offloads = 0x0000000e - it's before rte_eth_dev_configure(
> ( ... &l_port_conf ...)
>
>
>
> The receiver - is  DPDK application. Do I'm need set offloads at receiver
> site to eliminate "ierrors" ?
>
>
>
>
>
> 08.12.2022, 10:54, "Pavel Vazharov" <freakpv@gmail.com>:
>
> Few questions:
> 1. Does the sending NIC support IP and TCP/UDP checksum offloading? I
> mean, if these flags set?
>     struct rte_eth_dev_info dev_info;
>
>     rte_eth_dev_info_get(cfg.nic_port_id_, &dev_info);
>     constexpr auto rxcsum = DEV_RX_OFFLOAD_CHECKSUM;
>
>     constexpr auto l3csum = DEV_TX_OFFLOAD_IPV4_CKSUM;
>
>     constexpr auto l4csum = DEV_TX_OFFLOAD_TCP_CKSUM |
> DEV_TX_OFFLOAD_UDP_CKSUM;
>     dev_rx_csum_          = ((dev_info.tx_offload_capa & rxcsum) ==
> rxcsum);
>     dev_tx_csum_l3_       = ((dev_info.tx_offload_capa & l3csum) ==
> l3csum);
>     dev_tx_csum_l4_       = ((dev_info.tx_offload_capa & l4csum) ==
> l4csum);
>
> 2. Do you "tell" the sending NIC to do the checksum calculations before
> sending the packets? I mean, do you do something like this for outgoing
> packets?
>     if (offl.ip_csum) {
>
>         pkt->ol_flags |= PKT_TX_IP_CKSUM | PKT_TX_IPV4;
>
>         pkt->l2_len = RTE_ETHER_HDR_LEN;
>
>         pkt->l3_len = ih_len;
>
>     }
>
>     if (offl.tcp_csum) {
>
>         pkt->ol_flags |= PKT_TX_TCP_CKSUM;
>
>         pkt->l2_len = RTE_ETHER_HDR_LEN;
>
>         pkt->l3_len = ih_len;
>
>     }
>
>     if (offl.udp_csum) {
>
>         pkt->ol_flags |= PKT_TX_UDP_CKSUM;
>
>         pkt->l2_len = RTE_ETHER_HDR_LEN;
>
>         pkt->l3_len = ih_len;
>
>     }
>
> 3. Is the receiving side also a DPDK application? If it is, and if the NIC
> there supports checksum offloading you can check the flags of the received
> packets to see if the receiving NIC has detected checksum errors.
> pkt->ol_flags & (PKT_RX_IP_CKSUM_BAD | PKT_RX_L4_CKSUM_BAD)
>
> Hope some of the above helps.
>
> On Thu, Dec 8, 2022 at 9:36 AM Ruslan R. Laishev <zator@yandex.ru> wrote:
>
> Hello !
>
> I wrote too small apps to send and receive  ethernet/ip/udp
> frame/packet/dg, so on received side I  see next situation:
> number of in errors is equally a number of received packets. The test
> packet I made manually,  set offloads  IP/UDP checkusm.
> Is there what I'm need to check additionally ?
>
>
>
>
> ---
> С уважением,
> Ruslan R. Laishev
> OpenVMS bigot, natural born system/network progger, C contractor.
> +79013163222
> +79910009922
>
>
>
>
> ---
> С уважением,
> Ruslan R. Laishev
> OpenVMS bigot, natural born system/network progger, C contractor.
> +79013163222
> +79910009922
>
>

[-- Attachment #2: Type: text/html, Size: 15418 bytes --]