From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <konstantin.ananyev@intel.com>
Received: from mga09.intel.com (mga09.intel.com [134.134.136.24])
by dpdk.org (Postfix) with ESMTP id D92746AAE
for <dev@dpdk.org>; Fri, 5 Dec 2014 02:10:32 +0100 (CET)
Received: from orsmga003.jf.intel.com ([10.7.209.27])
by orsmga102.jf.intel.com with ESMTP; 04 Dec 2014 17:09:21 -0800
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.04,691,1406617200"; d="scan'208";a="493926484"
Received: from irsmsx152.ger.corp.intel.com ([163.33.192.66])
by orsmga003.jf.intel.com with ESMTP; 04 Dec 2014 17:07:06 -0800
Received: from irsmsx105.ger.corp.intel.com ([169.254.7.144]) by
IRSMSX152.ger.corp.intel.com ([169.254.6.56]) with mapi id 14.03.0195.001;
Fri, 5 Dec 2014 01:10:29 +0000
From: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>
To: Jean-Mickael Guerin <jean-mickael.guerin@6wind.com>, "dev@dpdk.org"
<dev@dpdk.org>
Thread-Topic: [PATCH v2] ixgbe: don't override mbuf buffer length
Thread-Index: AQHQD+2aqEhfxV/HVEKDZVlTVgxRRZyAKs/g
Date: Fri, 5 Dec 2014 01:10:28 +0000
Message-ID: <2601191342CEEE43887BDE71AB977258213BCCB6@IRSMSX105.ger.corp.intel.com>
References: <1417716553-1506-1-git-send-email-jean-mickael.guerin@6wind.com>
In-Reply-To: <1417716553-1506-1-git-send-email-jean-mickael.guerin@6wind.com>
Accept-Language: en-IE, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [163.33.239.182]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Subject: Re: [dpdk-dev] [PATCH v2] ixgbe: don't override mbuf buffer length
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Fri, 05 Dec 2014 01:10:34 -0000
> -----Original Message-----
> From: Jean-Mickael Guerin [mailto:jean-mickael.guerin@6wind.com]
> Sent: Thursday, December 04, 2014 6:09 PM
> To: dev@dpdk.org
> Cc: Richardson, Bruce; Ananyev, Konstantin
> Subject: [PATCH v2] ixgbe: don't override mbuf buffer length
>=20
> The template mbuf_initializer is hard coded with a buflen which
> might have been set differently by the application at the time of
> mbuf pool creation.
>=20
> Switch to a mbuf allocation, to fetch the correct default values.
> There is no performance impact because this is not a data-plane API.
>=20
> Signed-off-by: Jean-Mickael Guerin <jean-mickael.guerin@6wind.com>
> Acked-by: David Marchand <david.marchand@6wind.com>
> Fixes: 0ff3324da2 ("ixgbe: rework vector pmd following mbuf changes")
> ---
>=20
> v2: check returned value of ixgbe_rxq_vec_setup
>=20
> lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 5 ++++-
> lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c | 19 ++++++++++++-------
> 2 files changed, 16 insertions(+), 8 deletions(-)
>=20
> diff --git a/lib/librte_pmd_ixgbe/ixgbe_rxtx.c b/lib/librte_pmd_ixgbe/ixg=
be_rxtx.c
> index 5c36bff..7994da1 100644
> --- a/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
> +++ b/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
> @@ -2244,7 +2244,10 @@ ixgbe_dev_rx_queue_setup(struct rte_eth_dev *dev,
> use_def_burst_func =3D check_rx_burst_bulk_alloc_preconditions(rxq);
>=20
> #ifdef RTE_IXGBE_INC_VECTOR
> - ixgbe_rxq_vec_setup(rxq);
> + if (ixgbe_rxq_vec_setup(rxq) < 0) {
> + ixgbe_rx_queue_release(rxq);
> + return (-ENOMEM);
> + }
> #endif
> /* Check if pre-conditions are satisfied, and no Scattered Rx */
> if (!use_def_burst_func && !dev->data->scattered_rx) {
> diff --git a/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c b/lib/librte_pmd_ixgbe=
/ixgbe_rxtx_vec.c
> index c1b5a78..f7b02f5 100644
> --- a/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c
> +++ b/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c
> @@ -732,17 +732,22 @@ static struct ixgbe_txq_ops vec_txq_ops =3D {
> int
> ixgbe_rxq_vec_setup(struct igb_rx_queue *rxq)
> {
> - struct rte_mbuf mb_def =3D { .buf_addr =3D 0 }; /* zeroed mbuf */
> + struct rte_mbuf *mb_def;
>=20
> - mb_def.nb_segs =3D 1;
> - mb_def.data_off =3D RTE_PKTMBUF_HEADROOM;
> - mb_def.buf_len =3D rxq->mb_pool->elt_size - sizeof(struct rte_mbuf);
> - mb_def.port =3D rxq->port_id;
> - rte_mbuf_refcnt_set(&mb_def, 1);
> + mb_def =3D rte_pktmbuf_alloc(rxq->mb_pool);
> + if (mb_def =3D=3D NULL) {
> + PMD_INIT_LOG(ERR, "ixgbe_rxq_vec_setup: could not allocate one mbuf");
> + return -1;
> + }
> + /* nb_segs, refcnt, data_off and buf_len are already set */
> + mb_def->port =3D rxq->port_id;
>=20
> /* prevent compiler reordering: rearm_data covers previous fields */
> rte_compiler_barrier();
> - rxq->mbuf_initializer =3D *((uint64_t *)&mb_def.rearm_data);
> + rxq->mbuf_initializer =3D *((uint64_t *)&mb_def->rearm_data);
> +
> + rte_pktmbuf_free(mb_def);
> +
> return 0;
> }
>=20
> --
> 2.1.3
As I said in another mail, I don't think it is a proper fix.
What we did here - just changed one assumption to another.
Current assumption - all mbuf obj_init() would setup buf_len in exactly the=
same manner as rte_pktmbuf_init() does:
buf_len =3D mp->elt_size - sizeof(struct rte_mbuf);
New assumption - all mbuf obj_init() would setup buf_len for all mbufs in t=
he pool to the same value.
Both assumptions, I believe, are not always correct.
Though, probably the new one would be true more often.
I still think the proper fix is not to update mbuf's buf_len field at ixgbe=
_rxq_rearm() at all.
We should just leave the original value unmodified.
Actually, while looking at ixgbe_rxq_rearm(), I don't see any reason why we=
need to update buf_len field.
It is not the data that need to be rearmed.
The fields that need to be rearmed are:
uint16_t data_off;
uint16_t refcnt
uint8_t nb_segs;
uint8_t port;
6B in total.=20
We probably would like to keep rearming as one 64bit load/store.=20
Though straight below them we have:
uint64_t ol_flags;
As RX fully override ol_flags anyway, we can safely overwrite first 2B of i=
t.
That would allow us to still read/write whole 64bits and avoid overwriting =
buf_len. =20
I am talking about something like patch below.
I admit that it looks not so pretty, but I think it is much safer and corre=
ct.
Konstantin
--- a/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c
+++ b/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c
@@ -79,13 +79,19 @@ ixgbe_rxq_rearm(struct igb_rx_queue *rxq)
/* Initialize the mbufs in vector, process 2 mbufs in one loop */
for (i =3D 0; i < RTE_IXGBE_RXQ_REARM_THRESH; i +=3D 2, rxep +=3D 2=
) {
__m128i vaddr0, vaddr1;
+ uintptr_t p0, p1;
mb0 =3D rxep[0].mbuf;
mb1 =3D rxep[1].mbuf;
/* flush mbuf with pkt template */
- mb0->rearm_data[0] =3D rxq->mbuf_initializer;
- mb1->rearm_data[0] =3D rxq->mbuf_initializer;
+ p0 =3D (uintptr_t)&mb0->data_off;
+ *(uint64_t *)p0 =3D rxq->mbuf_initializer;
+ p1 =3D (uintptr_t)&mb1->data_off;
+ *(uint64_t *)p1 =3D rxq->mbuf_initializer;
+
+ //mb0->rearm_data[0] =3D rxq->mbuf_initializer;
+ //mb1->rearm_data[0] =3D rxq->mbuf_initializer;
/* load buf_addr(lo 64bit) and buf_physaddr(hi 64bit) */
vaddr0 =3D _mm_loadu_si128((__m128i *)&(mb0->buf_addr));
@@ -732,6 +738,7 @@ static struct ixgbe_txq_ops vec_txq_ops =3D {
int
ixgbe_rxq_vec_setup(struct igb_rx_queue *rxq)
{
+ uintptr_t p;
struct rte_mbuf mb_def =3D { .buf_addr =3D 0 }; /* zeroed mbuf */
mb_def.nb_segs =3D 1;
@@ -739,7 +746,8 @@ ixgbe_rxq_vec_setup(struct igb_rx_queue *rxq)
mb_def.buf_len =3D rxq->mb_pool->elt_size - sizeof(struct rte_mbuf)=
;
mb_def.port =3D rxq->port_id;
rte_mbuf_refcnt_set(&mb_def, 1);
- rxq->mbuf_initializer =3D *((uint64_t *)&mb_def.rearm_data);
+ p =3D (uintptr_t)&mb_def.data_off;
+ rxq->mbuf_initializer =3D *(uint64_t *)p;
return 0;
}