[dpdk-dev] rte_mbuf: documentation, meta-data, and inconsistencies

[dpdk-dev] rte_mbuf: documentation, meta-data, and inconsistencies

[daniel chapiesky]

rte_muf: Inconsistency in the programmer's guide or the code.....?
---------------------------------------------------------

>From the DPDK 1.7.0 programmer's guide we read:

    "For a newly allocated mbuf, the area at which the data begins in the
message buffer
    is RTE_PKTMBUF_HEADROOM bytes after the beginning of the buffer, which
is cache
    aligned."


In the file ./lib/librte_mbuf/rte_mbuf.c and function rte_pktmbuf_init() we
find:

    m->pkt.data = (char*) m->buf_addr + RTE_MIN(RTE_PKTMBUF_HEADROOM,
m->buf_len);

Where RTE_PKTMBUF_HEADROOM is configured to be 128 bytes from the file
./config/common_linuxapp:

    CONFIG_RTE_PKTMBUF_HEADROOM=128


Question 1:

    Does the above invocation of RTE_MIN() cause the programmer's guide
    to be inaccurate?

    (Saying "in practice it does not matter..." is not an answer)


Question 2:

    Why is "packet metadata" implicitly sharing the same bytes of the mbuf
headroom area,
    instead of being explicitly kept from being overwritten by a call to

        rte_pktmbuf_prepend(pkt, 100);

    presuming my metadata is at least 32 bytes long????

    The above command would place the packet data area starting at the last
4 bytes of my metadata....

    (Saying "you can change CONFIG_RTE_PKTMBUF_HEADROOM at compile time" is
not an answer because it
     effects all mbufs not just the one I with my meta-data...)


Question 3:

    Why do we write:

        #define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) +
RTE_PKTMBUF_HEADROOM)

        rte_mempool_create(s, 1024, MBUF_SIZE, 32,0,
                        rte_pktmbuf_pool_init, NULL,
                        rte_pktmbuf_init, NULL,
                        socketid, flags);


    instead of:

        #define MBUF_SIZE (2048 + sizeof(struct rte_mbuf))

        rte_mempool_create(s, 1024, MBUF_SIZE, 32,0,
                        rte_pktmbuf_pool_init, NULL,
                        rte_pktmbuf_init, NULL,
                        socketid, flags);

    and have rte_pktmbuf_init() enforce ( + RTE_PKTMBUF_HEADROOM )
automatically?


------------------------------

My reason for asking the above questions stems from my work on a Packet
Framework
Pipeline.

I came to understand that there really isn't an *explicit* declaration
of "allocate this much meta-data space for each packet".

The programmer's guide and mbuf.h describe of headroom and tailroom as a
place to edit
the packet data. An example would be to wrap the packet for tunnelling by
prepending
and appending the data area to be large enought to contain a new header and
checksum.

In fact, while the programmer's guide mentions packet meta-data a few
times, there is no
section which actually describes ****how to make and access**** your very
own packet meta-data.
This addition would be very nice.

The tables in the Packet Framework Pipeline examples all use keys injected
into the meta-data
of the mbuf at RX time to compare a rule against that key (explicitly
stating the "offset
into packet meta-data") and not allowing "offset into packet data".

I actually like this setup because it allows the meta-data key to be
securely analyzed
and copied from the packet data - keeping malformed packets out of the
decision making
process of the tables.

But, in the end, sharing the meta-data area with the packet headroom seems
to be a very
bad idea.

Sincerely,

Daniel Chapiesky

Re: [dpdk-dev] rte_mbuf: documentation, meta-data, and inconsistencies

[Matthew Hall]

On Thu, Aug 28, 2014 at 08:00:59PM -0400, daniel chapiesky wrote:
> But, in the end, sharing the meta-data area with the packet headroom seems
> to be a very
> bad idea.
> 
> Sincerely,
> 
> Daniel Chapiesky

You might have picked a good time to inquire about it as some of the Intel 
guys are making patches to clean up rte_mbuf during the last couple of weeks 
as we speak.

Matthew.

[dpdk-dev] rte_memzone: memzone_reserve_aligned_thread_unsafe failed when running two process at the same time.

[zimeiw]

hi,


Running primary process, it is successful.

$ sudo ./simple_mp -c 1 -n 1 --socket-mem=64  --proc-type=primary
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 0 on socket 0
EAL: Support maximum 64 logical core(s) by configuration.
EAL: Detected 2 lcore(s)
EAL: Setting up memory...
EAL: Ask a virtual area of 0x26000000 bytes
EAL: Virtual area found at 0x7fffd1200000 (size = 0x26000000)
EAL: Ask a virtual area of 0x2200000 bytes
EAL: Virtual area found at 0x7fffcee00000 (size = 0x2200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffcea00000 (size = 0x200000)
EAL: Ask a virtual area of 0x1a00000 bytes
EAL: Virtual area found at 0x7fffcce00000 (size = 0x1a00000)
EAL: Ask a virtual area of 0x2200000 bytes
EAL: Virtual area found at 0x7fffcaa00000 (size = 0x2200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffca600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffca200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffc9e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffc9a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffc9600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffc9200000 (size = 0x200000)
EAL: Requesting 32 pages of size 2MB from socket 0
EAL: TSC frequency is ~2793683 KHz
EAL: Master core 0 is ready (tid=f7fe4800)
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL:   0000:02:01.0 not managed by UIO driver, skipping
EAL: PCI device 0000:02:06.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL:   0000:02:06.0 not managed by UIO driver, skipping
proc primary
APP: Finished Process Init.

simple_mp >


Running secondary process, it is faied for memzone <RG_MP_log_history> already exists.
$ sudo ./simple_mp -c 1 -n 1   --proc-type=secondary --socket-mem=1
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 0 on socket 0
EAL: Support maximum 64 logical core(s) by configuration.
EAL: Detected 2 lcore(s)
EAL: Setting up memory...
EAL: Analysing 358 files
EAL: Mapped segment 0 of size 0x4000000
EAL: memzone_reserve_aligned_thread_unsafe(): memzone <RG_MP_log_history> already exists
RING: Cannot reserve memory
EAL: TSC frequency is ~2793681 KHz
EAL: Master core 0 is ready (tid=f7fe4800)
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL: Cannot find resource for device
EAL: PCI device 0000:02:06.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL: Cannot find resource for device
proc secandary
APP: Finished Process Init.

simple_mp >

Re: [dpdk-dev] rte_memzone: memzone_reserve_aligned_thread_unsafe failed when running two process at the same time.

[Zhang, Jerry]

Hi,
    'memzone <RG_MP_log_history> already exists' is not an error for the secondary DPDK process.

    DPDK EAL initialization creates a mempool for log history.  This mempool creation includes creating a ring based on the memzone named RG_MP_log_history.
    For the secondary DPDK process,  if the mempool for log history creation fails because the memzone RG_MP_log_history exits, it looks up the exited mempool and reuse it. 
    Please refer to the implementation of rte_eal_common_log_init() function.

    From the logs, it seems your secondary DPDK process has initialized successfully.
	
int
rte_eal_common_log_init(FILE *default_log)
{
	STAILQ_INIT(&log_history);

	/* reserve RTE_LOG_HISTORY*2 elements, so we can dump and
	 * keep logging during this time */
	log_history_mp = rte_mempool_create(LOG_HISTORY_MP_NAME, RTE_LOG_HISTORY*2,
				LOG_ELT_SIZE, 0, 0,
				NULL, NULL,
				NULL, NULL,
				SOCKET_ID_ANY, 0);
       
	if ((log_history_mp == NULL) && 
	    ((log_history_mp = rte_mempool_lookup(LOG_HISTORY_MP_NAME)) == NULL)){
		RTE_LOG(ERR, EAL, "%s(): cannot create log_history mempool\n",
			__func__);
		return -1;
	}

	default_log_stream = default_log;
	rte_openlog_stream(default_log);
	return 0;
}


-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of zimeiw
Sent: Saturday, August 30, 2014 7:23 AM
To: dev@dpdk.org
Subject: [dpdk-dev] rte_memzone: memzone_reserve_aligned_thread_unsafe failed when running two process at the same time.



hi,


Running primary process, it is successful.

$ sudo ./simple_mp -c 1 -n 1 --socket-mem=64  --proc-type=primary
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 0 on socket 0
EAL: Support maximum 64 logical core(s) by configuration.
EAL: Detected 2 lcore(s)
EAL: Setting up memory...
EAL: Ask a virtual area of 0x26000000 bytes
EAL: Virtual area found at 0x7fffd1200000 (size = 0x26000000)
EAL: Ask a virtual area of 0x2200000 bytes
EAL: Virtual area found at 0x7fffcee00000 (size = 0x2200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffcea00000 (size = 0x200000)
EAL: Ask a virtual area of 0x1a00000 bytes
EAL: Virtual area found at 0x7fffcce00000 (size = 0x1a00000)
EAL: Ask a virtual area of 0x2200000 bytes
EAL: Virtual area found at 0x7fffcaa00000 (size = 0x2200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffca600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffca200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffc9e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffc9a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffc9600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffc9200000 (size = 0x200000)
EAL: Requesting 32 pages of size 2MB from socket 0
EAL: TSC frequency is ~2793683 KHz
EAL: Master core 0 is ready (tid=f7fe4800)
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL:   0000:02:01.0 not managed by UIO driver, skipping
EAL: PCI device 0000:02:06.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL:   0000:02:06.0 not managed by UIO driver, skipping
proc primary
APP: Finished Process Init.

simple_mp >


Running secondary process, it is faied for memzone <RG_MP_log_history> already exists.
$ sudo ./simple_mp -c 1 -n 1   --proc-type=secondary --socket-mem=1
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 0 on socket 0
EAL: Support maximum 64 logical core(s) by configuration.
EAL: Detected 2 lcore(s)
EAL: Setting up memory...
EAL: Analysing 358 files
EAL: Mapped segment 0 of size 0x4000000
EAL: memzone_reserve_aligned_thread_unsafe(): memzone <RG_MP_log_history> already exists
RING: Cannot reserve memory
EAL: TSC frequency is ~2793681 KHz
EAL: Master core 0 is ready (tid=f7fe4800)
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL: Cannot find resource for device
EAL: PCI device 0000:02:06.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL: Cannot find resource for device
proc secandary
APP: Finished Process Init.

simple_mp >