From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <arybchenko@solarflare.com>
Received: from dispatch1-us1.ppe-hosted.com (dispatch1-us1.ppe-hosted.com
 [67.231.154.164]) by dpdk.org (Postfix) with ESMTP id C4C7A4C9F
 for <dev@dpdk.org>; Mon, 13 May 2019 14:22:39 +0200 (CEST)
X-Virus-Scanned: Proofpoint Essentials engine
Received: from webmail.solarflare.com (uk.solarflare.com [193.34.186.16])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits))
 (No client certificate requested)
 by mx1-us3.ppe-hosted.com (Proofpoint Essentials ESMTP Server) with ESMTPS id
 9A0BC9C0053; Mon, 13 May 2019 12:22:38 +0000 (UTC)
Received: from [192.168.38.17] (91.220.146.112) by ukex01.SolarFlarecom.com
 (10.17.10.4) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Mon, 13 May
 2019 13:22:32 +0100
To: Olivier Matz <olivier.matz@6wind.com>, "Eads, Gage" <gage.eads@intel.com>
CC: "dev@dpdk.org" <dev@dpdk.org>
References: <9184057F7FC11744A2107296B6B8EB1E68CB4370@fmsmsx101.amr.corp.intel.com>
 <20190513121424.7u7nwp5ziq4nhcjs@platinum>
From: Andrew Rybchenko <arybchenko@solarflare.com>
Message-ID: <6a33e840-c914-5264-2b3e-07da70ac2689@solarflare.com>
Date: Mon, 13 May 2019 15:22:29 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.6.1
MIME-Version: 1.0
In-Reply-To: <20190513121424.7u7nwp5ziq4nhcjs@platinum>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
X-Originating-IP: [91.220.146.112]
X-ClientProxiedBy: ocex03.SolarFlarecom.com (10.20.40.36) To
 ukex01.SolarFlarecom.com (10.17.10.4)
X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.5.1010-24610.003
X-TM-AS-Result: No-14.248200-8.000000-10
X-TMASE-MatchedRID: eVEkOcJu0F5q0U6EhO9EE4mfV7NNMGm+jLOy13Cgb4+qvcIF1TcLYOZL
 s8RREdbvzBaQKIWNzwFky3cBakrmpcL0fzzd/ey4E0Q83A2vD+uM45JzjPlCaCf2lTjn+XQELcR
 1EklbV//EYq+FLTs+9kIWTYkhGXQdQ2oRSAnLD1zil2r2x2PwtfioIsi7Sa0gu/jTz8Y/kepBjC
 V+ldsAAglYWnzcSFNXD4h3xoLsUa6p+DTe0bMYyRVXx+49zfW3dSHY4x0UjQnd8+wpe45s+GToV
 D6I8ePZehCPxJLdS+FC3UkrkIx+4neMiXfSpNCD/1aBDnIV4ikIgWIk/VdnUkS/boWSGMtdMwRy
 rAxy6J8hJ/ufappeEpGTpe1iiCJq71zr0FZRMbBWdFebWIc3VsRB0bsfrpPIfiAqrjYtFiRtx1N
 8maAj9ILHuv1FQhw82T8rG0h5SjzbyI6qUvs7oH7cGd19dSFd
X-TM-AS-User-Approved-Sender: Yes
X-TM-AS-User-Blocked-Sender: No
X-TMASE-Result: 10--14.248200-8.000000
X-TMASE-Version: SMEX-12.5.0.1300-8.5.1010-24610.003
X-MDID: 1557750159-AOHXlzRRdgm4
Subject: Re: [dpdk-dev] ***Spam*** Re: Mempool handler ops index allocation
	issue
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Mon, 13 May 2019 12:22:40 -0000

Hi Olivier,

On 5/13/19 3:14 PM, Olivier Matz wrote:
> Hi Gage,
>
> On Thu, May 09, 2019 at 10:19:55PM +0000, Eads, Gage wrote:
>> Hi all,
>>
>> I ran into a problem with a multi-process application, in which two processes assigned the same mempool handler ops index to *different* handlers. This happened because the two processes supplied the -d EAL arguments in different order, e.g.:
>>
>> sudo ./appA -dlibrte_mempool_bucket.so -dlibrte_mempool_ring.so --proc-type primary &
>> sudo ./appB -dlibrte_mempool_ring.so -dlibrte_mempool_bucket.so --proc-type secondary &
>>
>> The dynamic load order matters because the ops indexes are assigned in the order the library ctors are run. This can result in the different processes trying to use different handlers for the same mempool.
>>
>> I'm not aware of any requirement that the EAL argument order should match across processes, so I don't think this is a user error. This could also happen in static libraries if they linked the libraries in a different order - but that shouldn't occur if both applications are following the rules for building an external application (https://doc.dpdk.org/guides/prog_guide/dev_kit_build_system.html#building-external-applications).
>>
>> If you agree that this is an issue, I see a couple possible resolutions:
>>
>>
>> 1.       Add a note/warning to the mempool handlers section of the user guide (https://doc.dpdk.org/guides/prog_guide/mempool_lib.html#mempool-handlers)
>>
>> 2.       Modify rte_mempool_register_ops() so that built-in handlers (ring, stack, etc.) have fixed IDs. E.g. ring is always 0, stack is always 1, etc. These handlers could be identified by their name string. User-registered mempools would still be susceptible to this problem, though.
>>
>> Thoughts? Alternatives?
> What about this:
>
> - add a new table in a named memory zone that stores the association
>    between mempool_ops id and name (but not the ops pointers) of the
>    primary process.
>
> - change rte_mempool_register_ops() to have a specific behavior in case
>    of a secondary process: lookup in that table to get the id associated
>    to the name (fail if not found).
>
>
> On the other hand, using secondary processes always looked a bit dangerous
> to me (for several reasons), so adding a note in the programmer's guide
> (your proposal 1) is also fine to me.

Suggested solution looks good to me.

Thanks,
Andrew.

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
	by dpdk.space (Postfix) with ESMTP id D1701A00E6
	for <public@inbox.dpdk.org>; Mon, 13 May 2019 14:22:42 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
	by dpdk.org (Postfix) with ESMTP id BBD154CAB;
	Mon, 13 May 2019 14:22:40 +0200 (CEST)
Received: from dispatch1-us1.ppe-hosted.com (dispatch1-us1.ppe-hosted.com
 [67.231.154.164]) by dpdk.org (Postfix) with ESMTP id C4C7A4C9F
 for <dev@dpdk.org>; Mon, 13 May 2019 14:22:39 +0200 (CEST)
X-Virus-Scanned: Proofpoint Essentials engine
Received: from webmail.solarflare.com (uk.solarflare.com [193.34.186.16])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits))
 (No client certificate requested)
 by mx1-us3.ppe-hosted.com (Proofpoint Essentials ESMTP Server) with ESMTPS id
 9A0BC9C0053; Mon, 13 May 2019 12:22:38 +0000 (UTC)
Received: from [192.168.38.17] (91.220.146.112) by ukex01.SolarFlarecom.com
 (10.17.10.4) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Mon, 13 May
 2019 13:22:32 +0100
To: Olivier Matz <olivier.matz@6wind.com>, "Eads, Gage" <gage.eads@intel.com>
CC: "dev@dpdk.org" <dev@dpdk.org>
References: <9184057F7FC11744A2107296B6B8EB1E68CB4370@fmsmsx101.amr.corp.intel.com>
 <20190513121424.7u7nwp5ziq4nhcjs@platinum>
From: Andrew Rybchenko <arybchenko@solarflare.com>
Message-ID: <6a33e840-c914-5264-2b3e-07da70ac2689@solarflare.com>
Date: Mon, 13 May 2019 15:22:29 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.6.1
MIME-Version: 1.0
In-Reply-To: <20190513121424.7u7nwp5ziq4nhcjs@platinum>
Content-Type: text/plain; charset="UTF-8"; format="flowed"
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
X-Originating-IP: [91.220.146.112]
X-ClientProxiedBy: ocex03.SolarFlarecom.com (10.20.40.36) To
 ukex01.SolarFlarecom.com (10.17.10.4)
X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.5.1010-24610.003
X-TM-AS-Result: No-14.248200-8.000000-10
X-TMASE-MatchedRID: eVEkOcJu0F5q0U6EhO9EE4mfV7NNMGm+jLOy13Cgb4+qvcIF1TcLYOZL
 s8RREdbvzBaQKIWNzwFky3cBakrmpcL0fzzd/ey4E0Q83A2vD+uM45JzjPlCaCf2lTjn+XQELcR
 1EklbV//EYq+FLTs+9kIWTYkhGXQdQ2oRSAnLD1zil2r2x2PwtfioIsi7Sa0gu/jTz8Y/kepBjC
 V+ldsAAglYWnzcSFNXD4h3xoLsUa6p+DTe0bMYyRVXx+49zfW3dSHY4x0UjQnd8+wpe45s+GToV
 D6I8ePZehCPxJLdS+FC3UkrkIx+4neMiXfSpNCD/1aBDnIV4ikIgWIk/VdnUkS/boWSGMtdMwRy
 rAxy6J8hJ/ufappeEpGTpe1iiCJq71zr0FZRMbBWdFebWIc3VsRB0bsfrpPIfiAqrjYtFiRtx1N
 8maAj9ILHuv1FQhw82T8rG0h5SjzbyI6qUvs7oH7cGd19dSFd
X-TM-AS-User-Approved-Sender: Yes
X-TM-AS-User-Blocked-Sender: No
X-TMASE-Result: 10--14.248200-8.000000
X-TMASE-Version: SMEX-12.5.0.1300-8.5.1010-24610.003
X-MDID: 1557750159-AOHXlzRRdgm4
Subject: Re: [dpdk-dev] ***Spam*** Re: Mempool handler ops index allocation
	issue
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
Message-ID: <20190513122229.ZegB7Aug9GdTItU0ayyon8UChD_beSW_Xl8nhX2jA2k@z>

Hi Olivier,

On 5/13/19 3:14 PM, Olivier Matz wrote:
> Hi Gage,
>
> On Thu, May 09, 2019 at 10:19:55PM +0000, Eads, Gage wrote:
>> Hi all,
>>
>> I ran into a problem with a multi-process application, in which two processes assigned the same mempool handler ops index to *different* handlers. This happened because the two processes supplied the -d EAL arguments in different order, e.g.:
>>
>> sudo ./appA -dlibrte_mempool_bucket.so -dlibrte_mempool_ring.so --proc-type primary &
>> sudo ./appB -dlibrte_mempool_ring.so -dlibrte_mempool_bucket.so --proc-type secondary &
>>
>> The dynamic load order matters because the ops indexes are assigned in the order the library ctors are run. This can result in the different processes trying to use different handlers for the same mempool.
>>
>> I'm not aware of any requirement that the EAL argument order should match across processes, so I don't think this is a user error. This could also happen in static libraries if they linked the libraries in a different order - but that shouldn't occur if both applications are following the rules for building an external application (https://doc.dpdk.org/guides/prog_guide/dev_kit_build_system.html#building-external-applications).
>>
>> If you agree that this is an issue, I see a couple possible resolutions:
>>
>>
>> 1.       Add a note/warning to the mempool handlers section of the user guide (https://doc.dpdk.org/guides/prog_guide/mempool_lib.html#mempool-handlers)
>>
>> 2.       Modify rte_mempool_register_ops() so that built-in handlers (ring, stack, etc.) have fixed IDs. E.g. ring is always 0, stack is always 1, etc. These handlers could be identified by their name string. User-registered mempools would still be susceptible to this problem, though.
>>
>> Thoughts? Alternatives?
> What about this:
>
> - add a new table in a named memory zone that stores the association
>    between mempool_ops id and name (but not the ops pointers) of the
>    primary process.
>
> - change rte_mempool_register_ops() to have a specific behavior in case
>    of a secondary process: lookup in that table to get the id associated
>    to the name (fail if not found).
>
>
> On the other hand, using secondary processes always looked a bit dangerous
> to me (for several reasons), so adding a note in the programmer's guide
> (your proposal 1) is also fine to me.

Suggested solution looks good to me.

Thanks,
Andrew.