DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] rte_mbuf: documentation, meta-data, and inconsistencies
@ 2014-08-29  0:00 daniel chapiesky
  2014-08-29  3:09 ` Matthew Hall
  2014-08-29 23:22 ` [dpdk-dev] rte_memzone: memzone_reserve_aligned_thread_unsafe failed when running two process at the same time zimeiw
  0 siblings, 2 replies; 4+ messages in thread
From: daniel chapiesky @ 2014-08-29  0:00 UTC (permalink / raw)
  To: dev

rte_muf: Inconsistency in the programmer's guide or the code.....?
---------------------------------------------------------

>From the DPDK 1.7.0 programmer's guide we read:

    "For a newly allocated mbuf, the area at which the data begins in the
message buffer
    is RTE_PKTMBUF_HEADROOM bytes after the beginning of the buffer, which
is cache
    aligned."


In the file ./lib/librte_mbuf/rte_mbuf.c and function rte_pktmbuf_init() we
find:

    m->pkt.data = (char*) m->buf_addr + RTE_MIN(RTE_PKTMBUF_HEADROOM,
m->buf_len);

Where RTE_PKTMBUF_HEADROOM is configured to be 128 bytes from the file
./config/common_linuxapp:

    CONFIG_RTE_PKTMBUF_HEADROOM=128


Question 1:

    Does the above invocation of RTE_MIN() cause the programmer's guide
    to be inaccurate?

    (Saying "in practice it does not matter..." is not an answer)


Question 2:

    Why is "packet metadata" implicitly sharing the same bytes of the mbuf
headroom area,
    instead of being explicitly kept from being overwritten by a call to

        rte_pktmbuf_prepend(pkt, 100);

    presuming my metadata is at least 32 bytes long????

    The above command would place the packet data area starting at the last
4 bytes of my metadata....

    (Saying "you can change CONFIG_RTE_PKTMBUF_HEADROOM at compile time" is
not an answer because it
     effects all mbufs not just the one I with my meta-data...)


Question 3:

    Why do we write:

        #define MBUF_SIZE (2048 + sizeof(struct rte_mbuf) +
RTE_PKTMBUF_HEADROOM)

        rte_mempool_create(s, 1024, MBUF_SIZE, 32,0,
                        rte_pktmbuf_pool_init, NULL,
                        rte_pktmbuf_init, NULL,
                        socketid, flags);


    instead of:

        #define MBUF_SIZE (2048 + sizeof(struct rte_mbuf))

        rte_mempool_create(s, 1024, MBUF_SIZE, 32,0,
                        rte_pktmbuf_pool_init, NULL,
                        rte_pktmbuf_init, NULL,
                        socketid, flags);

    and have rte_pktmbuf_init() enforce ( + RTE_PKTMBUF_HEADROOM )
automatically?


------------------------------

My reason for asking the above questions stems from my work on a Packet
Framework
Pipeline.

I came to understand that there really isn't an *explicit* declaration
of "allocate this much meta-data space for each packet".

The programmer's guide and mbuf.h describe of headroom and tailroom as a
place to edit
the packet data. An example would be to wrap the packet for tunnelling by
prepending
and appending the data area to be large enought to contain a new header and
checksum.

In fact, while the programmer's guide mentions packet meta-data a few
times, there is no
section which actually describes ****how to make and access**** your very
own packet meta-data.
This addition would be very nice.

The tables in the Packet Framework Pipeline examples all use keys injected
into the meta-data
of the mbuf at RX time to compare a rule against that key (explicitly
stating the "offset
into packet meta-data") and not allowing "offset into packet data".

I actually like this setup because it allows the meta-data key to be
securely analyzed
and copied from the packet data - keeping malformed packets out of the
decision making
process of the tables.

But, in the end, sharing the meta-data area with the packet headroom seems
to be a very
bad idea.

Sincerely,

Daniel Chapiesky

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dpdk-dev] rte_mbuf: documentation, meta-data, and inconsistencies
  2014-08-29  0:00 [dpdk-dev] rte_mbuf: documentation, meta-data, and inconsistencies daniel chapiesky
@ 2014-08-29  3:09 ` Matthew Hall
  2014-08-29 23:22 ` [dpdk-dev] rte_memzone: memzone_reserve_aligned_thread_unsafe failed when running two process at the same time zimeiw
  1 sibling, 0 replies; 4+ messages in thread
From: Matthew Hall @ 2014-08-29  3:09 UTC (permalink / raw)
  To: daniel chapiesky; +Cc: dev

On Thu, Aug 28, 2014 at 08:00:59PM -0400, daniel chapiesky wrote:
> But, in the end, sharing the meta-data area with the packet headroom seems
> to be a very
> bad idea.
> 
> Sincerely,
> 
> Daniel Chapiesky

You might have picked a good time to inquire about it as some of the Intel 
guys are making patches to clean up rte_mbuf during the last couple of weeks 
as we speak.

Matthew.

^ permalink raw reply	[flat|nested] 4+ messages in thread

* [dpdk-dev] rte_memzone: memzone_reserve_aligned_thread_unsafe failed when running two process at the same time.
  2014-08-29  0:00 [dpdk-dev] rte_mbuf: documentation, meta-data, and inconsistencies daniel chapiesky
  2014-08-29  3:09 ` Matthew Hall
@ 2014-08-29 23:22 ` zimeiw
  2014-08-30 14:22   ` Zhang, Jerry
  1 sibling, 1 reply; 4+ messages in thread
From: zimeiw @ 2014-08-29 23:22 UTC (permalink / raw)
  To: dev



hi,


Running primary process, it is successful.

$ sudo ./simple_mp -c 1 -n 1 --socket-mem=64  --proc-type=primary
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 0 on socket 0
EAL: Support maximum 64 logical core(s) by configuration.
EAL: Detected 2 lcore(s)
EAL: Setting up memory...
EAL: Ask a virtual area of 0x26000000 bytes
EAL: Virtual area found at 0x7fffd1200000 (size = 0x26000000)
EAL: Ask a virtual area of 0x2200000 bytes
EAL: Virtual area found at 0x7fffcee00000 (size = 0x2200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffcea00000 (size = 0x200000)
EAL: Ask a virtual area of 0x1a00000 bytes
EAL: Virtual area found at 0x7fffcce00000 (size = 0x1a00000)
EAL: Ask a virtual area of 0x2200000 bytes
EAL: Virtual area found at 0x7fffcaa00000 (size = 0x2200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffca600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffca200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffc9e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffc9a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffc9600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffc9200000 (size = 0x200000)
EAL: Requesting 32 pages of size 2MB from socket 0
EAL: TSC frequency is ~2793683 KHz
EAL: Master core 0 is ready (tid=f7fe4800)
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL:   0000:02:01.0 not managed by UIO driver, skipping
EAL: PCI device 0000:02:06.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL:   0000:02:06.0 not managed by UIO driver, skipping
proc primary
APP: Finished Process Init.

simple_mp >


Running secondary process, it is faied for memzone <RG_MP_log_history> already exists.
$ sudo ./simple_mp -c 1 -n 1   --proc-type=secondary --socket-mem=1
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 0 on socket 0
EAL: Support maximum 64 logical core(s) by configuration.
EAL: Detected 2 lcore(s)
EAL: Setting up memory...
EAL: Analysing 358 files
EAL: Mapped segment 0 of size 0x4000000
EAL: memzone_reserve_aligned_thread_unsafe(): memzone <RG_MP_log_history> already exists
RING: Cannot reserve memory
EAL: TSC frequency is ~2793681 KHz
EAL: Master core 0 is ready (tid=f7fe4800)
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL: Cannot find resource for device
EAL: PCI device 0000:02:06.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL: Cannot find resource for device
proc secandary
APP: Finished Process Init.

simple_mp >




^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dpdk-dev] rte_memzone: memzone_reserve_aligned_thread_unsafe failed when running two process at the same time.
  2014-08-29 23:22 ` [dpdk-dev] rte_memzone: memzone_reserve_aligned_thread_unsafe failed when running two process at the same time zimeiw
@ 2014-08-30 14:22   ` Zhang, Jerry
  0 siblings, 0 replies; 4+ messages in thread
From: Zhang, Jerry @ 2014-08-30 14:22 UTC (permalink / raw)
  To: zimeiw, dev

Hi,
    'memzone <RG_MP_log_history> already exists' is not an error for the secondary DPDK process.

    DPDK EAL initialization creates a mempool for log history.  This mempool creation includes creating a ring based on the memzone named RG_MP_log_history.
    For the secondary DPDK process,  if the mempool for log history creation fails because the memzone RG_MP_log_history exits, it looks up the exited mempool and reuse it. 
    Please refer to the implementation of rte_eal_common_log_init() function.

    From the logs, it seems your secondary DPDK process has initialized successfully.
	
int
rte_eal_common_log_init(FILE *default_log)
{
	STAILQ_INIT(&log_history);

	/* reserve RTE_LOG_HISTORY*2 elements, so we can dump and
	 * keep logging during this time */
	log_history_mp = rte_mempool_create(LOG_HISTORY_MP_NAME, RTE_LOG_HISTORY*2,
				LOG_ELT_SIZE, 0, 0,
				NULL, NULL,
				NULL, NULL,
				SOCKET_ID_ANY, 0);
       
	if ((log_history_mp == NULL) && 
	    ((log_history_mp = rte_mempool_lookup(LOG_HISTORY_MP_NAME)) == NULL)){
		RTE_LOG(ERR, EAL, "%s(): cannot create log_history mempool\n",
			__func__);
		return -1;
	}

	default_log_stream = default_log;
	rte_openlog_stream(default_log);
	return 0;
}


-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of zimeiw
Sent: Saturday, August 30, 2014 7:23 AM
To: dev@dpdk.org
Subject: [dpdk-dev] rte_memzone: memzone_reserve_aligned_thread_unsafe failed when running two process at the same time.



hi,


Running primary process, it is successful.

$ sudo ./simple_mp -c 1 -n 1 --socket-mem=64  --proc-type=primary
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 0 on socket 0
EAL: Support maximum 64 logical core(s) by configuration.
EAL: Detected 2 lcore(s)
EAL: Setting up memory...
EAL: Ask a virtual area of 0x26000000 bytes
EAL: Virtual area found at 0x7fffd1200000 (size = 0x26000000)
EAL: Ask a virtual area of 0x2200000 bytes
EAL: Virtual area found at 0x7fffcee00000 (size = 0x2200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffcea00000 (size = 0x200000)
EAL: Ask a virtual area of 0x1a00000 bytes
EAL: Virtual area found at 0x7fffcce00000 (size = 0x1a00000)
EAL: Ask a virtual area of 0x2200000 bytes
EAL: Virtual area found at 0x7fffcaa00000 (size = 0x2200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffca600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffca200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffc9e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffc9a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffc9600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fffc9200000 (size = 0x200000)
EAL: Requesting 32 pages of size 2MB from socket 0
EAL: TSC frequency is ~2793683 KHz
EAL: Master core 0 is ready (tid=f7fe4800)
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL:   0000:02:01.0 not managed by UIO driver, skipping
EAL: PCI device 0000:02:06.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL:   0000:02:06.0 not managed by UIO driver, skipping
proc primary
APP: Finished Process Init.

simple_mp >


Running secondary process, it is faied for memzone <RG_MP_log_history> already exists.
$ sudo ./simple_mp -c 1 -n 1   --proc-type=secondary --socket-mem=1
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 0 on socket 0
EAL: Support maximum 64 logical core(s) by configuration.
EAL: Detected 2 lcore(s)
EAL: Setting up memory...
EAL: Analysing 358 files
EAL: Mapped segment 0 of size 0x4000000
EAL: memzone_reserve_aligned_thread_unsafe(): memzone <RG_MP_log_history> already exists
RING: Cannot reserve memory
EAL: TSC frequency is ~2793681 KHz
EAL: Master core 0 is ready (tid=f7fe4800)
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL: Cannot find resource for device
EAL: PCI device 0000:02:06.0 on NUMA socket -1
EAL:   probe driver: 8086:100f rte_em_pmd
EAL: Cannot find resource for device
proc secandary
APP: Finished Process Init.

simple_mp >

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2014-08-30 14:18 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-08-29  0:00 [dpdk-dev] rte_mbuf: documentation, meta-data, and inconsistencies daniel chapiesky
2014-08-29  3:09 ` Matthew Hall
2014-08-29 23:22 ` [dpdk-dev] rte_memzone: memzone_reserve_aligned_thread_unsafe failed when running two process at the same time zimeiw
2014-08-30 14:22   ` Zhang, Jerry

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).