Re: [dts] Packets reorder in the second pass on mempool

["Liu, Yong" <yong.liu@intel.com>]

Hi Javadi,
I think it's fine to send out issue to dev mailing list.
Maybe there's no reply from people in community is just because of can't reproduce your issue.
We need the NIC type and related rx/tx queue settings for reproduce your problem.

> -----Original Message-----
> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Masoud Moshref Javadi
> Sent: Saturday, July 25, 2015 6:04 AM
> To: dts@dpdk.org
> Subject: [dts] Packets reorder in the second pass on mempool
> 
> Hi
> 
> I'm new to dpdk and I hope you can solve my weird problem (I think I
> sent this message to dev list by mistake. sorry)
> 
> I see packet reordering correlated with the batchsize and mempool size.
> 
> I have a very simple setting of a sender and a receiver connected with a
> simple 10G switch.
> The sender sends udp packets at 14.88MPPS and just puts a 32bit sequence
> number in the udp packets.
> The receiver reads packets and expects the packets to be in order.
> However, the receiver sees packets in a reordered way like this (note
> that it starts from 7808 in different runs):
> 
> expected, seen , seen-expected
> 7808, 7936 = 128
> 7978, 8106 = 128
> 8145, 8273 = 128
> 8308, 8436 = 128
> 8448, 8320 = -128
> 8391, 8519 = 128
> 8576, 8448 = -128
> 8518, 8646 = 128
> 8704, 8576 = -128
> ....
> 
> The batchsize at sender is 128.
> The configuration of ports and mempools are similar to basicfwd.c in
> basic forwarding example of dpdk.
> Interestingly if I change the size of mempool, the beginning of packet
> reordering changes from 7808.
> 
> #define NUM_MBUFS 8192
> #define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)
> #define MBUF_CACHE_SIZE 250
> #define BURST_SIZE 128
> 
> mbuf_pool = rte_mempool_create("MBUF_POOL",
>                                          NUM_MBUFS * nb_ports,
>                                          MBUF_SIZE,
>                                          MBUF_CACHE_SIZE,
>                                          sizeof(struct
> rte_pktmbuf_pool_private),
>                                          rte_pktmbuf_pool_init, NULL,
>                                          rte_pktmbuf_init,      NULL,
>                                          1,
>                                          0);
> 
> The main sender loop is as follows:
> 
>    for (;pkts_num<target;) {
>                           /* Get burst of RX packets, from first port of
> pair. */
>                           rte_mempool_sc_get_bulk(mbuf_pool,
> (void**)&bufs, BURST_SIZE);
>                           /* Send burst of TX packets, to second port of
> pair. */
>                           for (i=0; i< BURST_SIZE; i++){
>                                   m = bufs[i];
>                                   m->data_len = 60;
>                                   m->pkt_len  = 60;
>                                   rte_memcpy((uint8_t *)m->buf_addr +
> m->data_off,bufpkt, 60);
>                                   uint32_t * data = (uint32_t
> *)(rte_ctrlmbuf_data(m) + data_offset);
>                                   *data = pkts_num+i;
>                           }
> 
>                           uint16_t nb_tx = 0;
>                           while (nb_tx < BURST_SIZE){
>                                   nb_tx+= rte_eth_tx_burst(port, 0, bufs
> + nb_tx, BURST_SIZE - nb_tx);
>                           }
>                           pkts_num += nb_tx;
> }
> 
> Regards