From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <shahafs@mellanox.com>
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
(mail-eopbgr40085.outbound.protection.outlook.com [40.107.4.85])
by dpdk.org (Postfix) with ESMTP id 024C6201
for <dev@dpdk.org>; Thu, 1 Nov 2018 08:21:33 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com;
s=selector1;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=eLptmWH/zNZPK6EeA12hprJFUaksHOIhPn+P5mJ6fCA=;
b=IJSlxWop5425CqyNytWZlT4EA3zERhAUWoj0KuAuCJRZljXOHRINWIOaQbvSrUmaMuXtulvVGamhYUojoHl3g+Viv0kyvTgv1R+ORiFkiUDs4ddX0ApQHPAciQiJ3YA/3M3Hhx4dPibwMfXKhAt58weUiVfw/jkZtKT4qIdbKew=
Received: from DB7PR05MB4426.eurprd05.prod.outlook.com (52.134.109.15) by
DB7PR05MB4107.eurprd05.prod.outlook.com (52.134.107.32) with Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.1294.22; Thu, 1 Nov 2018 07:21:32 +0000
Received: from DB7PR05MB4426.eurprd05.prod.outlook.com
([fe80::80e:e6b:baf2:d973]) by DB7PR05MB4426.eurprd05.prod.outlook.com
([fe80::80e:e6b:baf2:d973%3]) with mapi id 15.20.1273.027; Thu, 1 Nov 2018
07:21:32 +0000
From: Shahaf Shuler <shahafs@mellanox.com>
To: Tom Barbette <barbette@kth.se>, "dev@dpdk.org" <dev@dpdk.org>, Ferruh
Yigit <ferruh.yigit@intel.com>, Thomas Monjalon <thomas@monjalon.net>, Andrew
Rybchenko <arybchenko@solarflare.com>, "olivier.matz@6wind.com"
<olivier.matz@6wind.com>
CC: Yongseok Koh <yskoh@mellanox.com>
Thread-Topic: [PATCH v4] mlx5: Support for rte_eth_rx_queue_count
Thread-Index: AQHUbgdFa7YpZOBUAEWIgl1+xlkwxaU0YpJwgASx7wCAAXMSEA==
Date: Thu, 1 Nov 2018 07:21:31 +0000
Message-ID: <DB7PR05MB4426685FAA18491A43449207C3CE0@DB7PR05MB4426.eurprd05.prod.outlook.com>
References: <1540653055-67051-1-git-send-email-barbette@kth.se>,
<DB7PR05MB4426813F13178E990F884916C3F20@DB7PR05MB4426.eurprd05.prod.outlook.com>
<1540976475938.69727@kth.se>
In-Reply-To: <1540976475938.69727@kth.se>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is )
smtp.mailfrom=shahafs@mellanox.com;
x-originating-ip: [193.47.165.251]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; DB7PR05MB4107;
6:w2ycGFVxxD44rTXxNJEmcNK7EjqYwgEdxo5Voos2rt+MDv9iNqTrpa9JXgcQkcixrwdqPZwoflioJ/NhXoiNzsPVxCLjxRoBhzRWLWs6ajVRlZNebXqmuLBSn1xQQuElxQJwK+v3/TH/tFHxbPWAkHscwuaS0eHaXbi8eAVSbH5r/+wtTS2ExMaGUraWsgIBkB5uuOCOVMpsirbeCabj+iu69+frEqgW9q18n/kgbeT5zj2X82BatjHVOtq4LLHpCqC/7VacrxI61AXgDp4Gi30WWmkMsCccfGfoO7QysV7ILXGT9DS70pUQ0lsOQZTB+ZlQ7J+/DE/0fuZDktLpGQOzzoKV+HAiACbh8elj15kl5u0b0I7HSIEw7+A1CGwLwiIwZ+nxG8INnr1ycwlJE6uk+fiM1BO/FV8T+aEQ4ZR17z2myOY1FKPIgVZ1d3dLtd3nVxRM8mx3RAktGhcCSg==;
5:0rYgkPcU+4N0rzk5/cwEVD9PNN1jb2iGLz/qXBxIrse3YUH5z9v/D/yuJWSqgRwiX2mFsMQau+m/l7orEFluwoCVlAP1EZD5FdlVhzIUCdhEF6I8uKUTN/EOMAsm3BVM62EfDHjlZY2AYAh2Rb2LX0EhXqcwCHKoxXTQVrulXxE=;
7:qgWctuFhDil5+trlXVrfd9FxPCPAb+SiMiw5z0mbd+t5Iv4+G48F1C6sASRLjVaP6NOJ549QXGe9BOTPbjkKneeYbLJtWSgv56q7bU+/uxQ0gI5x5cK2VYUtK3bEQyzMoDfDiy2902rZq0Ppc9+33w==
x-ms-exchange-antispam-srfa-diagnostics: SOS;
x-ms-office365-filtering-correlation-id: ee42da05-bd4e-4054-cf4b-08d63fcaa5ff
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: BCL:0; PCL:0;
RULEID:(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020);
SRVR:DB7PR05MB4107;
x-ms-traffictypediagnostic: DB7PR05MB4107:
x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr
x-microsoft-antispam-prvs: <DB7PR05MB410786618D81A91991C95AF2C3CE0@DB7PR05MB4107.eurprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(158342451672863);
x-ms-exchange-senderadcheck: 1
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0;
RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(3231382)(944501410)(52105095)(10201501046)(3002001)(6055026)(148016)(149066)(150057)(6041310)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123564045)(20161123562045)(201708071742011)(7699051)(76991095);
SRVR:DB7PR05MB4107; BCL:0; PCL:0; RULEID:; SRVR:DB7PR05MB4107;
x-forefront-prvs: 0843C17679
x-forefront-antispam-report: SFV:NSPM;
SFS:(10009020)(979002)(376002)(366004)(346002)(396003)(39860400002)(136003)(199004)(189003)(9686003)(71190400001)(55016002)(316002)(8936002)(26005)(5660300001)(66066001)(74316002)(110136005)(5250100002)(105586002)(71200400001)(81166006)(6436002)(2906002)(97736004)(68736007)(86362001)(102836004)(8676002)(6116002)(575784001)(81156014)(3846002)(2501003)(106356001)(7696005)(6506007)(76176011)(99286004)(107886003)(478600001)(33656002)(229853002)(486006)(25786009)(14454004)(4326008)(53936002)(6246003)(7736002)(305945005)(2900100001)(186003)(11346002)(446003)(256004)(476003)(969003)(989001)(999001)(1009001)(1019001);
DIR:OUT; SFP:1101; SCL:1; SRVR:DB7PR05MB4107;
H:DB7PR05MB4426.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en;
PTR:InfoNoRecords; A:1; MX:1;
received-spf: None (protection.outlook.com: mellanox.com does not designate
permitted sender hosts)
x-microsoft-antispam-message-info: vSVgJ8YodkBilghi5ImJUBa2R4Y/qBnEtLoKh5Bvv45DR55beCk5ljzoIzYv2sYjZ86h2VRzoVhDWDWcvJJ8KHJgYRDS8DghZ+oY2bGGYFsbTJq6Gl+iHBUgDt9esy+OBAvzAbez6M8FbnTIhmJzNSNQ5ges0uoTEo7TPzLgE88coNBcscW/lOn/XT2OVTmHLWvOfR2ni8iGm5N5kekcKdK6Jcxrh29+juEJemb6y1lwvQbVElqFIDKQYq1ok0b4vX4YAbZsnZqGYJdPcycL4G8lbF+EvnD4M90sMa/4KA/SISUMCznWAiO7L54sbBhyyI23mcgVmMuz5YG0HTg2hhcPOG6EdD55c5JjgChCW3c=
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: Mellanox.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ee42da05-bd4e-4054-cf4b-08d63fcaa5ff
X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Nov 2018 07:21:31.9612 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR05MB4107
Subject: Re: [dpdk-dev] [PATCH v4] mlx5: Support for rte_eth_rx_queue_count
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Thu, 01 Nov 2018 07:21:34 -0000
Wednesday, October 31, 2018 11:01 AM, Tom Barbette:
> Subject: RE: [PATCH v4] mlx5: Support for rte_eth_rx_queue_count
>=20
> Hi Shahaf,
>=20
> I don't see how rte_eth_rx_descriptor_status can actually give me the
> number of packets in the RX queue? It will tell me the status of a packet=
at a
> given offset, right?
It will tell you if in a given offset on the rxq you have a packet ready. I=
think it will fit your needs, see below.=20
>=20
> About the goal: we have a full view of a network (switches and servers), =
and
> we want to know where the queueing is happening. So queues from
> switches and servers are reported to the controller to deduce latency,
> congestion, ....
> On top of that, as CPU occupancy is somehow erratic with PMDs (ie we use
> approximations), the queuing helps to make better scheduling decisions
> about which servers could accept more load.
So how about the below heuristic using rte_eth_rx_descriptor_status.
Let's say you configure the rxq w/ N descriptors. Pick some threshold which=
will represent "this queue on this server has enough work, no need to send=
more", e.g. 3*N/4.=20
Monitor the rxq descriptor using rte_eth_rx_descriptor_status(port, rxq, 3*=
N/4). If you get RTE_ETH_RX_DESC_AVAIL, then the TH is not yet reached and =
the server can process more, otherwise if you get RTE_ETH_RX_DESC_DONE it m=
eans you should stop scheduling packets for this one.
You can also pick set of different TH to say queues is {not busy, partially=
busy, really busy} to deduce about the latency. But for the latency it is=
better to work w/ NIC host coherent clock + timestamps (like you implement=
ed on a different patch). =20
>=20
> Considering the advance of Smart NICs, being able to monitor NICs as any
> networking equipment in a network is of even more importance (if I still
> need to make that point?).
>=20
>=20
> Tom
>=20
>=20
>=20
>=20
> ________________________________________
> De : Shahaf Shuler <shahafs@mellanox.com> Envoy=E9 : dimanche 28 octobre
> 2018 10:37 =C0 : Tom Barbette; dev@dpdk.org; Ferruh Yigit; Thomas Monjalo=
n;
> Andrew Rybchenko; olivier.matz@6wind.com Cc : Yongseok Koh Objet : RE:
> [PATCH v4] mlx5: Support for rte_eth_rx_queue_count
>=20
> Hi Tom,
>=20
> Adding ethdev maintainers and Oliver as the author of the new API.
>=20
>=20
> Saturday, October 27, 2018 6:11 PM, Tom Barbette:
> > Subject: [PATCH v4] mlx5: Support for rte_eth_rx_queue_count
> >
>=20
> I have a more basic question.
> The rte_eth_rx_queue_count is a very old API, more or less from the
> beginning of DPDK.
> We progressed from then and a newer API to check the descriptor status
> was introduced @DPDK17.05 : rte_eth_rx_descriptor_status, see commit[1].
> There is also a plan to deprecate it once all drivers will support the ne=
w API.
>=20
> With the new API you can check the number of available descriptors on a
> similar way.
> So my question is, is the new API enough for your functionality? If not, =
what
> it is missing? I would prefer to improve the new one instead of starting =
to
> support the old one.
>=20
>=20
> [1]
> commit b1b700ce7d6fe0db9152f73e8e00394fc2756e60
> Author: Olivier Matz <olivier.matz@6wind.com>
> Date: Wed Mar 29 10:36:28 2017 +0200
>=20
> ethdev: add descriptor status API
>=20
> Introduce a new API to get the status of a descriptor.
>=20
> For Rx, it is almost similar to rx_descriptor_done API, except it
> differentiates "used" descriptors (which are hold by the driver and n=
ot
> returned to the hardware).
>=20
> For Tx, it is a new API.
>=20
> The descriptor_done() API, and probably the rx_queue_count() API coul=
d
> be replaced by this new API as soon as it is implemented on all PMDs.
>=20
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
>=20
>=20
> > This patch adds support for the rx_queue_count API in mlx5 driver
> >
> > Changes in v2:
> > * Fixed styling issues
> > * Fix missing return
> >
> > Changes in v3:
> > * Fix styling comments and checks as per Yongseok Koh
> > <yskoh@mellanox.com> comments. Thanks !
> >
> > Changes in v4:
> > * Fix compiling issue because of a line that disappeared in v3
> >
> > Signed-off-by: Tom Barbette <barbette@kth.se>
> > ---
> > drivers/net/mlx5/mlx5.c | 1 +
> > drivers/net/mlx5/mlx5_rxtx.c | 78
> > ++++++++++++++++++++++++++++++++++++++------
> > drivers/net/mlx5/mlx5_rxtx.h | 1 +
> > 3 files changed, 70 insertions(+), 10 deletions(-)
> >
> > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index
> > ec63bc6..6fccadd 100644
> > --- a/drivers/net/mlx5/mlx5.c
> > +++ b/drivers/net/mlx5/mlx5.c
> > @@ -375,6 +375,7 @@ const struct eth_dev_ops mlx5_dev_ops =3D {
> > .filter_ctrl =3D mlx5_dev_filter_ctrl,
> > .rx_descriptor_status =3D mlx5_rx_descriptor_status,
> > .tx_descriptor_status =3D mlx5_tx_descriptor_status,
> > + .rx_queue_count =3D mlx5_rx_queue_count,
> > .rx_queue_intr_enable =3D mlx5_rx_intr_enable,
> > .rx_queue_intr_disable =3D mlx5_rx_intr_disable,
> > .is_removed =3D mlx5_is_removed,
> > diff --git a/drivers/net/mlx5/mlx5_rxtx.c
> > b/drivers/net/mlx5/mlx5_rxtx.c index 2d14f8a..2126205 100644
> > --- a/drivers/net/mlx5/mlx5_rxtx.c
> > +++ b/drivers/net/mlx5/mlx5_rxtx.c
> > @@ -417,20 +417,17 @@ mlx5_tx_descriptor_status(void *tx_queue,
> > uint16_t offset) }
> >
> > /**
> > - * DPDK callback to check the status of a rx descriptor.
> > + * Internal function to compute the number of used descriptors in an
> > + RX queue
> > *
> > - * @param rx_queue
> > - * The rx queue.
> > - * @param[in] offset
> > - * The index of the descriptor in the ring.
> > + * @param rxq
> > + * The Rx queue.
> > *
> > * @return
> > - * The status of the tx descriptor.
> > + * The number of used rx descriptor.
> > */
> > -int
> > -mlx5_rx_descriptor_status(void *rx_queue, uint16_t offset)
> > +static uint32_t
> > +rx_queue_count(struct mlx5_rxq_data *rxq)
> > {
> > - struct mlx5_rxq_data *rxq =3D rx_queue;
> > struct rxq_zip *zip =3D &rxq->zip;
> > volatile struct mlx5_cqe *cqe;
> > const unsigned int cqe_n =3D (1 << rxq->cqe_n); @@ -461,12 +458,7=
3
> > @@ mlx5_rx_descriptor_status(void *rx_queue, uint16_t offset)
> > cqe =3D &(*rxq->cqes)[cq_ci & cqe_cnt];
> > }
> > used =3D RTE_MIN(used, (1U << rxq->elts_n) - 1);
> > - if (offset < used)
> > + return used;
> > +}
> > +
> > +/**
> > + * DPDK callback to check the status of a rx descriptor.
> > + *
> > + * @param rx_queue
> > + * The Rx queue.
> > + * @param[in] offset
> > + * The index of the descriptor in the ring.
> > + *
> > + * @return
> > + * The status of the tx descriptor.
> > + */
> > +int
> > +mlx5_rx_descriptor_status(void *rx_queue, uint16_t offset) {
> > + struct mlx5_rxq_data *rxq =3D rx_queue;
> > + struct mlx5_rxq_ctrl *rxq_ctrl =3D
> > + container_of(rxq, struct mlx5_rxq_ctrl, rxq);
> > + struct rte_eth_dev *dev =3D ETH_DEV(rxq_ctrl->priv);
> > +
> > + if (dev->rx_pkt_burst !=3D mlx5_rx_burst) {
> > + rte_errno =3D ENOTSUP;
> > + return -rte_errno;
> > + }
> > + if (offset >=3D (1 << rxq->elts_n)) {
> > + rte_errno =3D EINVAL;
> > + return -rte_errno;
> > + }
> > + if (offset < rx_queue_count(rxq))
> > return RTE_ETH_RX_DESC_DONE;
> > return RTE_ETH_RX_DESC_AVAIL;
> > }
> >
> > /**
> > + * DPDK callback to get the number of used descriptors in a RX queue
> > + *
> > + * @param dev
> > + * Pointer to the device structure.
> > + *
> > + * @param rx_queue_id
> > + * The Rx queue.
> > + *
> > + * @return
> > + * The number of used rx descriptor.
> > + * -EINVAL if the queue is invalid
> > + */
> > +uint32_t
> > +mlx5_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id) {
> > + struct priv *priv =3D dev->data->dev_private;
> > + struct mlx5_rxq_data *rxq;
> > +
> > + if (dev->rx_pkt_burst !=3D mlx5_rx_burst) {
> > + rte_errno =3D ENOTSUP;
> > + return -rte_errno;
> > + }
> > + rxq =3D (*priv->rxqs)[rx_queue_id];
> > + if (!rxq) {
> > + rte_errno =3D EINVAL;
> > + return -rte_errno;
> > + }
> > + return rx_queue_count(rxq);
> > +}
> > +
> > +/**
> > * DPDK callback for TX.
> > *
> > * @param dpdk_txq
> > diff --git a/drivers/net/mlx5/mlx5_rxtx.h
> > b/drivers/net/mlx5/mlx5_rxtx.h index 48ed2b2..c82059b 100644
> > --- a/drivers/net/mlx5/mlx5_rxtx.h
> > +++ b/drivers/net/mlx5/mlx5_rxtx.h
> > @@ -345,6 +345,7 @@ uint16_t removed_rx_burst(void *dpdk_rxq, struct
> > rte_mbuf **pkts,
> > uint16_t pkts_n); int
> > mlx5_rx_descriptor_status(void *rx_queue, uint16_t offset); int
> > mlx5_tx_descriptor_status(void *tx_queue, uint16_t offset);
> > +uint32_t mlx5_rx_queue_count(struct rte_eth_dev *dev, uint16_t
> > +rx_queue_id);
> >
> > /* Vectorized version of mlx5_rxtx.c */ int
> > mlx5_check_raw_vec_tx_support(struct rte_eth_dev *dev);
> > --
> > 2.7.4