Re: [dpdk-dev] [PATCH 00/10] Make DPDK tailqs fully local

[Thomas Monjalon <thomas.monjalon@6wind.com>]

2014-06-20 16:42, Anatoly Burakov:
> This issue was reported by OVS-DPDK project, and the fix should go to
> upstream DPDK. This is not memnic-related - this is to do with
> DPDK's rte_ivshmem library.
> 
> Every DPDK data structure has a corresponding TAILQ reserved for it in
> the runtime config file. Those TAILQs are fully local to the process,
> however most data structures contain pointers to next entry in the
> TAILQ.
> 
> Since the data structures such as rings are shared in their entirety,
> those TAILQ pointers are shared as well. Meaning that, after a
> successful rte_ring creation, the tailq_next pointer of the last
> ring in the TAILQ will be updated with a pointer to a ring which may
> not be present in the address space of another process (i.e. a ring
> that may be host-local or guest-local, and not shared over IVSHMEM).
> Any successive ring create/lookup on the other side of IVSHMEM will
> result in trying to dereference an invalid pointer.
> 
> This patchset fixes this problem by creating a default tailq entry
> that may be used by any data structure that chooses to use TAILQs.
> This default TAILQ entry will consist of a tailq_next/tailq_prev
> pointers, and an opaque pointer to arbitrary data. All TAILQ
> pointers from data structures themselves will be removed and
> replaced by those generic TAILQ entries, thus fixing the problem
> of potentially exposing local address space to shared structures.
> 
> Technically, only rte_ring structure require modification, because
> IVSHMEM is only using memzones (which aren't in TAILQs) and rings,
> but for consistency's sake other TAILQ-based data structures were
> adapted as well.
> 
> As part of this patchset, rte_malloc is also fixed to properly support
> multiprocess malloc and free. Previously, if the memory was malloc'd
> and freed in different processes, this could lead to segmentation
> faults due to different heap pointers in malloc elements themselves.
> This is fixed by making shared config to be mapped at the same
> addresses in both primary and secondary processes, so that the heap
> pointers in malloc elements are always valid, whatever process is
> doing malloc or free.
> 
> The mapping address for the shared config is also now set with the
> base-virtaddr flag, mapping the config file just before the start
> address for the hugepages.
> 
> v2 changes:
> * fixed race conditions in *_free operations
> * fixed multiprocess support for malloc heaps
> * added similar changes for acl
> * rebased on top of e88b42f818bc1a6d4ce6cb70371b66e37fa34f7d
> 
> v3 changes:
> * fixed race reported by Konstantin Ananyev (introduced in v2)
> 
> v4 changes:
> * rte_mem_config mapping address is now also set by --base-virtaddr
> 
> Anatoly Burakov (10):
>   eal: map shared config into exact same address as primary process
>   eal: use --base-virtaddr for mapping rte_config as well
>   rte_tailq: change rte_dummy to rte_tailq_entry, add data pointer
>   rte_ring: make ring tailq fully local
>   rte_hash: make rte_hash tailq fully local
>   rte_fbk_hash: make rte_fbk_hash tailq fully local
>   rte_mempool: make mempool tailq fully local
>   rte_lpm: make lpm tailq fully local
>   rte_lpm6: make lpm6 tailq fully local
>   rte_acl: make acl tailq fully local

Applied for version 1.7.1.

Thanks
-- 
Thomas