From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id A4535A052F;
Wed, 29 Jan 2020 13:31:39 +0100 (CET)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id 6B8981BFC6;
Wed, 29 Jan 2020 13:31:39 +0100 (CET)
Received: from EUR02-HE1-obe.outbound.protection.outlook.com
(mail-eopbgr10051.outbound.protection.outlook.com [40.107.1.51])
by dpdk.org (Postfix) with ESMTP id D92631BFC0
for <dev@dpdk.org>; Wed, 29 Jan 2020 13:31:37 +0100 (CET)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=Uo6QhQU0vToNezVRBkQvusJYVBEQL+TXKu6oiPus9DKVvOmNjhJdnn4L0bgNM7LHC4pVdB4davW2U0lpMQ5AOrBF9OqPdAImJVm096FJHS8hIIhEEVuCDR/P0bx6WwXfWgPcwPS8Bu43akFtTHY3r0QuMXxDnpPE9gZdF1VgXQXoaZdA1V6nGKwpLoUQZvZOOqog2DTNL5qHeSK0jMfGdI7cJYonTRHUIgj8B8XRDb1dLDOPs32hcz5LdW9e1ZC6iI5oFns6kT3XIl79HOl/vriOiJ3brlP0VHNF2E/qSZbJ3q/Mo8x32LIOKBhiXJ4Ut5dmk/Doti+4RhStj4T+FA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=jG1CHvcOMFHQzKNIT2UgsWhfpv/gGDe4A1Dlq/LyUWU=;
b=laeqpsPG2iSkKfKe8Um2HKYQ4GL40hGe3J/tA8hlcdmVasL7X/um3FtFN0VEDZzHpgZl/VS91GDmDvOv51l/nVcgdsL5ZhsjETXB0bGYy0McIpZ4Wb231hfRt6ePZEB8isuuHO/yTYOxyuTAdR00+cQBilr7oE7mc+VIjT+gSROVcmyimcROkqjbrttLUvVR6WxE2TRfB6n7aWmUNY7cX0FnldqR7AhSK3ABOoYqnQCP7WmuaXPb6Tmx1FqePrQVOsiXFI/rnfTj/YysQSkkBMPctXQ/TKTsvBAQvEokxzY2A7l8t1E5pp02y7NdQ7xoV869CHvEVW7AoBeRDB/T4Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com;
dkim=pass header.d=mellanox.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com;
s=selector1;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=jG1CHvcOMFHQzKNIT2UgsWhfpv/gGDe4A1Dlq/LyUWU=;
b=FR6a1hgE6YERKIQzSczACnGhfVHu8mYmORj57JWP58j+VR/fg9b9QH9P0IFv1ziyfurIfehn/OHXog+U6HBKWw9CwgK+QrG/bZ98UbOjdJf+IcbCCdpMZiQIsHp/4vuFdhM9ld/8+eSzNG2oWKME4Ro1K5chLxY42/y07GFoDAI=
Received: from AM4PR05MB3265.eurprd05.prod.outlook.com (10.171.188.154) by
AM4PR05MB3124.eurprd05.prod.outlook.com (10.171.186.13) with Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.2665.22; Wed, 29 Jan 2020 12:31:35 +0000
Received: from AM4PR05MB3265.eurprd05.prod.outlook.com
([fe80::68eb:ad79:71f4:110f]) by AM4PR05MB3265.eurprd05.prod.outlook.com
([fe80::68eb:ad79:71f4:110f%3]) with mapi id 15.20.2686.025; Wed, 29 Jan 2020
12:31:35 +0000
From: Slava Ovsiienko <viacheslavo@mellanox.com>
To: Slava Ovsiienko <viacheslavo@mellanox.com>, "dev@dpdk.org" <dev@dpdk.org>
CC: Matan Azrad <matan@mellanox.com>, Raslan Darawsheh <rasland@mellanox.com>,
Ori Kam <orika@mellanox.com>, Shahaf Shuler <shahafs@mellanox.com>,
"olivier.matz@6wind.com" <olivier.matz@6wind.com>,
"stephen@networkplumber.org" <stephen@networkplumber.org>,
"thomas@mellanox.net" <thomas@mellanox.net>
Thread-Topic: app/test: unit test problem with testing mbuf in forked child
process
Thread-Index: AdXWn/g6hmqkdvcxSviP0FVkeGGCZg==
Date: Wed, 29 Jan 2020 12:31:35 +0000
Message-ID: <AM4PR05MB32657F4EC029471A22542B6FD2050@AM4PR05MB3265.eurprd05.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is )
smtp.mailfrom=viacheslavo@mellanox.com;
x-originating-ip: [77.75.144.194]
x-ms-publictraffictype: Email
x-ms-office365-filtering-ht: Tenant
x-ms-office365-filtering-correlation-id: bd29a930-1199-46f3-6bf2-08d7a4b72e49
x-ms-traffictypediagnostic: AM4PR05MB3124:|AM4PR05MB3124:
x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtFwd
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <AM4PR05MB31245BBC8D08740384AF3E4AD2050@AM4PR05MB3124.eurprd05.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:3173;
x-forefront-prvs: 02973C87BC
x-forefront-antispam-report: SFV:NSPM;
SFS:(10009020)(4636009)(136003)(366004)(39860400002)(346002)(396003)(376002)(199004)(189003)(76116006)(66946007)(66446008)(52536014)(64756008)(66476007)(5660300002)(2906002)(66556008)(86362001)(26005)(110136005)(316002)(54906003)(478600001)(71200400001)(6506007)(55016002)(8936002)(53546011)(33656002)(8676002)(4326008)(9686003)(81166006)(81156014)(7696005)(186003);
DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR05MB3124;
H:AM4PR05MB3265.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en;
PTR:InfoNoRecords; A:1; MX:1;
received-spf: None (protection.outlook.com: mellanox.com does not designate
permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: D4BhTeaDe9FUVckrtuHZtTRjlG1nz0mybBkc1mj/vEYkWJlFL5kP+IZ3nGad7WjziSAorPSBAYUDTg6hDVWN6Pz3PZKZAGFyuqgROs7VF+TgLM1G5DSS1nFGh4fgjMKDokjxHABWQcQEPxQcN/AitfeU91N+cpXTZcHvteErrTp7MJYsta7hxD1XTSXmHpcdgWifEpbhlXjjNEUnilcoVD6ZQ0bacoAIES3qYGLi0rQr/VFKtuhc7Nz3zuDVQ08eMyYILL3zOSxlUxaOU5hV6UWEp3twmhRtzAejgu+kHyGxPNRD1+JnGNQJD0sM3j3/1zvPzkTbTVh0hCOwKJhooDnZBDnLYwD4oQVnAk4bmkhrOB1z2hTecSYqbEIM484V4hHpjSAa1odxY57xjpz0fuXYNs4fr6LNpb+20XcN/5aGAmmcAOeDp6cdzJNWO5Ce
x-ms-exchange-antispam-messagedata: CrRqL9xhoTivAj7ErxMDiD6HqQDgp6I5pzQ5n3hEOlV6qbikbpTdC29ob/BO6g8ZhUbjyECHnhhWL1Ze41pXXuDy+wNLxVHy+zs7S+L/BNkZeAHIZVp4lJS4SYuz0WSBCHT0q9zRbtDXn4s4O6QDxQ==
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: Mellanox.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bd29a930-1199-46f3-6bf2-08d7a4b72e49
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Jan 2020 12:31:35.8009 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: KjK8opv0h8MP8nAiCNVfavrAuOxyR/YppnCxDzv9en9GzGiMTsdmaev7lpwYPN257e/AKSAz0oLRYnAiEe3tZkr0sd2W9sHyGzirAqjDIVo=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3124
Subject: [dpdk-dev] app/test: unit test problem with testing mbuf in forked
child process
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
While working on the unit test for the mbuf with pinned external buffers I =
found that:
app/test/test_mbuf.c:
- test_failing_mbuf_sanity_check()
- verify_mbuf_check_panics(buf)
- fork()
- child process()
Does not work (either over regular mbufs, not pinned ones) in correct way.
The mbuf being tested is not mapped (in huge page) to child process and it =
always completes with seg fault.
The question - is this problem of my setup, or did we change the memory map=
ping model
and this unit test should be updated?
=09
With best regards, Slava
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Viacheslav Ovsiienko
> Sent: Friday, January 24, 2020 22:25
> To: dev@dpdk.org
> Cc: Matan Azrad <matan@mellanox.com>; Raslan Darawsheh
> <rasland@mellanox.com>; Ori Kam <orika@mellanox.com>; Shahaf Shuler
> <shahafs@mellanox.com>; olivier.matz@6wind.com;
> stephen@networkplumber.org; thomas@mellanox.net
> Subject: [dpdk-dev] [PATCH] app/test: add test for mbuf with pinned exter=
nal
> buffer
>=20
> This patch adds unit test for the mbufs allocated from the special pool w=
ith
> pinned external data buffers.
>=20
> The pinned buffer mbufs are tested in the same way as regular ones with
> taking into account some specifics of cloning/attaching.
>=20
> Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
> ---
> app/test/test_mbuf.c | 165
> ++++++++++++++++++++++++++++++++++++++++++++++-----
> 1 file changed, 150 insertions(+), 15 deletions(-)
>=20
> diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c index
> 61ecffc..ee2f2f0 100644
> --- a/app/test/test_mbuf.c
> +++ b/app/test/test_mbuf.c
> @@ -310,8 +310,17 @@
> return -1;
> }
>=20
> +static uint16_t
> +testclone_refcnt_read(struct rte_mbuf *m) {
> + return RTE_MBUF_HAS_PINNED_EXTBUF(m) ?
> + rte_mbuf_ext_refcnt_read(m->shinfo) :
> + rte_mbuf_refcnt_read(m);
> +}
> +
> static int
> -testclone_testupdate_testdetach(struct rte_mempool *pktmbuf_pool)
> +testclone_testupdate_testdetach(struct rte_mempool *pktmbuf_pool,
> + struct rte_mempool *clone_pool)
> {
> struct rte_mbuf *m =3D NULL;
> struct rte_mbuf *clone =3D NULL;
> @@ -331,7 +340,7 @@
> *data =3D MAGIC_DATA;
>=20
> /* clone the allocated mbuf */
> - clone =3D rte_pktmbuf_clone(m, pktmbuf_pool);
> + clone =3D rte_pktmbuf_clone(m, clone_pool);
> if (clone =3D=3D NULL)
> GOTO_FAIL("cannot clone data\n");
>=20
> @@ -339,7 +348,7 @@
> if (*data !=3D MAGIC_DATA)
> GOTO_FAIL("invalid data in clone\n");
>=20
> - if (rte_mbuf_refcnt_read(m) !=3D 2)
> + if (testclone_refcnt_read(m) !=3D 2)
> GOTO_FAIL("invalid refcnt in m\n");
>=20
> /* free the clone */
> @@ -358,7 +367,7 @@
> data =3D rte_pktmbuf_mtod(m->next, unaligned_uint32_t *);
> *data =3D MAGIC_DATA;
>=20
> - clone =3D rte_pktmbuf_clone(m, pktmbuf_pool);
> + clone =3D rte_pktmbuf_clone(m, clone_pool);
> if (clone =3D=3D NULL)
> GOTO_FAIL("cannot clone data\n");
>=20
> @@ -370,15 +379,15 @@
> if (*data !=3D MAGIC_DATA)
> GOTO_FAIL("invalid data in clone->next\n");
>=20
> - if (rte_mbuf_refcnt_read(m) !=3D 2)
> + if (testclone_refcnt_read(m) !=3D 2)
> GOTO_FAIL("invalid refcnt in m\n");
>=20
> - if (rte_mbuf_refcnt_read(m->next) !=3D 2)
> + if (testclone_refcnt_read(m->next) !=3D 2)
> GOTO_FAIL("invalid refcnt in m->next\n");
>=20
> /* try to clone the clone */
>=20
> - clone2 =3D rte_pktmbuf_clone(clone, pktmbuf_pool);
> + clone2 =3D rte_pktmbuf_clone(clone, clone_pool);
> if (clone2 =3D=3D NULL)
> GOTO_FAIL("cannot clone the clone\n");
>=20
> @@ -390,10 +399,10 @@
> if (*data !=3D MAGIC_DATA)
> GOTO_FAIL("invalid data in clone2->next\n");
>=20
> - if (rte_mbuf_refcnt_read(m) !=3D 3)
> + if (testclone_refcnt_read(m) !=3D 3)
> GOTO_FAIL("invalid refcnt in m\n");
>=20
> - if (rte_mbuf_refcnt_read(m->next) !=3D 3)
> + if (testclone_refcnt_read(m->next) !=3D 3)
> GOTO_FAIL("invalid refcnt in m->next\n");
>=20
> /* free mbuf */
> @@ -418,7 +427,8 @@
> }
>=20
> static int
> -test_pktmbuf_copy(struct rte_mempool *pktmbuf_pool)
> +test_pktmbuf_copy(struct rte_mempool *pktmbuf_pool,
> + struct rte_mempool *clone_pool)
> {
> struct rte_mbuf *m =3D NULL;
> struct rte_mbuf *copy =3D NULL;
> @@ -458,11 +468,14 @@
> copy =3D NULL;
>=20
> /* same test with a cloned mbuf */
> - clone =3D rte_pktmbuf_clone(m, pktmbuf_pool);
> + clone =3D rte_pktmbuf_clone(m, clone_pool);
> if (clone =3D=3D NULL)
> GOTO_FAIL("cannot clone data\n");
>=20
> - if (!RTE_MBUF_CLONED(clone))
> + if ((!RTE_MBUF_HAS_PINNED_EXTBUF(m) &&
> + !RTE_MBUF_CLONED(clone)) ||
> + (RTE_MBUF_HAS_PINNED_EXTBUF(m) &&
> + !RTE_MBUF_HAS_EXTBUF(clone)))
> GOTO_FAIL("clone did not give a cloned mbuf\n");
>=20
> copy =3D rte_pktmbuf_copy(clone, pktmbuf_pool, 0, UINT32_MAX); @@
> -1199,10 +1212,11 @@
> buf =3D rte_pktmbuf_alloc(pktmbuf_pool);
> if (buf =3D=3D NULL)
> return -1;
> + /*
> printf("Checking good mbuf initially\n");
> if (verify_mbuf_check_panics(buf) !=3D -1)
> return -1;
> -
> + */
> printf("Now checking for error conditions\n");
>=20
> if (verify_mbuf_check_panics(NULL)) {
> @@ -2411,6 +2425,120 @@ struct test_case {
> return -1;
> }
>=20
> +/*
> + * Test the mbuf pool with pinned external data buffers
> + * - Allocate memory zone for external buffer
> + * - Create the mbuf pool with pinned external buffer
> + * - Check the created pool with relevant mbuf pool unit tests */
> +static int test_pktmbuf_ext_pinned_buffer(struct rte_mempool *std_pool)
> +{
> +
> + struct rte_pktmbuf_extmem ext_mem;
> + struct rte_mempool *pinned_pool =3D NULL;
> + const struct rte_memzone *mz =3D NULL;
> +
> + printf("Test mbuf pool with external pinned data buffers\n");
> +
> + /* Allocate memzone for the external data buffer */
> + mz =3D rte_memzone_reserve("pinned_pool",
> + NB_MBUF * MBUF_DATA_SIZE,
> + SOCKET_ID_ANY,
> + RTE_MEMZONE_2MB |
> RTE_MEMZONE_SIZE_HINT_ONLY);
> + if (mz =3D=3D NULL)
> + GOTO_FAIL("%s: Memzone allocation failed\n", __func__);
> +
> + /* Create the mbuf pool with pinned external data buffer */
> + ext_mem.buf_ptr =3D mz->addr;
> + ext_mem.buf_iova =3D mz->iova;
> + ext_mem.buf_len =3D mz->len;
> + ext_mem.elt_size =3D MBUF_DATA_SIZE;
> +
> + pinned_pool =3D rte_pktmbuf_pool_create_extbuf("test_pinned_pool",
> + NB_MBUF, MEMPOOL_CACHE_SIZE, 0,
> + MBUF_DATA_SIZE, SOCKET_ID_ANY,
> + &ext_mem, 1);
> + if (pinned_pool =3D=3D NULL)
> + GOTO_FAIL("%s: Mbuf pool with pinned external"
> + " buffer creation failed\n", __func__);
> + /* test multiple mbuf alloc */
> + if (test_pktmbuf_pool(pinned_pool) < 0)
> + GOTO_FAIL("%s: test_mbuf_pool(pinned) failed\n",
> + __func__);
> +
> + /* do it another time to check that all mbufs were freed */
> + if (test_pktmbuf_pool(pinned_pool) < 0)
> + GOTO_FAIL("%s: test_mbuf_pool(pinned) failed (2)\n",
> + __func__);
> +
> + /* test that the data pointer on a packet mbuf is set properly */
> + if (test_pktmbuf_pool_ptr(pinned_pool) < 0)
> + GOTO_FAIL("%s: test_pktmbuf_pool_ptr(pinned) failed\n",
> + __func__);
> +
> + /* test data manipulation in mbuf with non-ascii data */
> + if (test_pktmbuf_with_non_ascii_data(pinned_pool) < 0)
> + GOTO_FAIL("%s: test_pktmbuf_with_non_ascii_data(pinned)"
> + " failed\n", __func__);
> +
> + /* test free pktmbuf segment one by one */
> + if (test_pktmbuf_free_segment(pinned_pool) < 0)
> + GOTO_FAIL("%s: test_pktmbuf_free_segment(pinned)
> failed\n",
> + __func__);
> +
> + if (testclone_testupdate_testdetach(pinned_pool, std_pool) < 0)
> + GOTO_FAIL("%s: testclone_and_testupdate(pinned) failed\n",
> + __func__);
> +
> + if (test_pktmbuf_copy(pinned_pool, std_pool) < 0)
> + GOTO_FAIL("%s: test_pktmbuf_copy(pinned) failed\n",
> + __func__);
> +
> + if (test_failing_mbuf_sanity_check(pinned_pool) < 0)
> + GOTO_FAIL("%s: test_failing_mbuf_sanity_check(pinned)"
> + " failed\n", __func__);
> +
> + if (test_mbuf_linearize_check(pinned_pool) < 0)
> + GOTO_FAIL("%s: test_mbuf_linearize_check(pinned) failed\n",
> + __func__);
> +
> + /* test for allocating a bulk of mbufs with various sizes */
> + if (test_pktmbuf_alloc_bulk(pinned_pool) < 0)
> + GOTO_FAIL("%s: test_rte_pktmbuf_alloc_bulk(pinned)
> failed\n",
> + __func__);
> +
> + /* test for allocating a bulk of mbufs with various sizes */
> + if (test_neg_pktmbuf_alloc_bulk(pinned_pool) < 0)
> + GOTO_FAIL("%s: test_neg_rte_pktmbuf_alloc_bulk(pinned)"
> + " failed\n", __func__);
> +
> + /* test to read mbuf packet */
> + if (test_pktmbuf_read(pinned_pool) < 0)
> + GOTO_FAIL("%s: test_rte_pktmbuf_read(pinned) failed\n",
> + __func__);
> +
> + /* test to read mbuf packet from offset */
> + if (test_pktmbuf_read_from_offset(pinned_pool) < 0)
> + GOTO_FAIL("%s: test_rte_pktmbuf_read_from_offset(pinned)"
> + " failed\n", __func__);
> +
> + /* test to read data from chain of mbufs with data segments */
> + if (test_pktmbuf_read_from_chain(pinned_pool) < 0)
> + GOTO_FAIL("%s: test_rte_pktmbuf_read_from_chain(pinned)"
> + " failed\n", __func__);
> +
> + RTE_SET_USED(std_pool);
> + rte_mempool_free(pinned_pool);
> + rte_memzone_free(mz);
> + return 0;
> +
> +fail:
> + rte_mempool_free(pinned_pool);
> + rte_memzone_free(mz);
> + return -1;
> +}
> +
> static int
> test_mbuf_dyn(struct rte_mempool *pktmbuf_pool) { @@ -2635,12
> +2763,12 @@ struct test_case {
> goto err;
> }
>=20
> - if (testclone_testupdate_testdetach(pktmbuf_pool) < 0) {
> + if (testclone_testupdate_testdetach(pktmbuf_pool, pktmbuf_pool) <
> 0) {
> printf("testclone_and_testupdate() failed \n");
> goto err;
> }
>=20
> - if (test_pktmbuf_copy(pktmbuf_pool) < 0) {
> + if (test_pktmbuf_copy(pktmbuf_pool, pktmbuf_pool) < 0) {
> printf("test_pktmbuf_copy() failed\n");
> goto err;
> }
> @@ -2731,6 +2859,13 @@ struct test_case {
> goto err;
> }
>=20
> + /* test the mbuf pool with pinned external data buffers */
> + if (test_pktmbuf_ext_pinned_buffer(pktmbuf_pool) < 0) {
> + printf("test_pktmbuf_ext_pinned_buffer() failed\n");
> + goto err;
> + }
> +
> +
> ret =3D 0;
> err:
> rte_mempool_free(pktmbuf_pool);
> --
> 1.8.3.1