From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40056.outbound.protection.outlook.com [40.107.4.56]) by dpdk.org (Postfix) with ESMTP id E6FB61B236 for ; Tue, 31 Oct 2017 14:25:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=9VWCIdQOIvhOGGaJgizsUFO4Hmz4IaeQSuZrUpynlW4=; b=IxMxJ04rYZnbfX3kmmG0V7c7rFS/jubnhu4CLlZr6SxBV9hveQT8nF1PsLzuDrQPguQP8HrylsuCTLhuScWIuQX+z4wzt6Z8BBkzmFAh1kB4wdc2VKfogO9GT0YbMJpJ1rXSF0vkipohvmTai6ypd5/9/zgcbDY7/YM10gMg2RA= Received: from DB5PR05MB1254.eurprd05.prod.outlook.com (10.162.157.140) by DB5PR05MB1254.eurprd05.prod.outlook.com (10.162.157.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.178.6; Tue, 31 Oct 2017 13:25:12 +0000 Received: from DB5PR05MB1254.eurprd05.prod.outlook.com ([fe80::69a8:7948:e0cb:c16e]) by DB5PR05MB1254.eurprd05.prod.outlook.com ([fe80::69a8:7948:e0cb:c16e%13]) with mapi id 15.20.0178.014; Tue, 31 Oct 2017 13:25:12 +0000 From: Ophir Munk To: Adrien Mazarguil , Matan Azrad CC: "dev@dpdk.org" , Olga Shern , "Thomas Monjalon" , Ophir Munk Thread-Topic: [PATCH v3 2/7] net/mlx4: associate MR to MP in a short function Thread-Index: AQHTUWb3aL5Mf57q70iqr2p3SaBNZKL8cnqAgAF7TrA= Date: Tue, 31 Oct 2017 13:25:12 +0000 Message-ID: References: <1508768520-4810-1-git-send-email-ophirmu@mellanox.com> <1509358049-18854-1-git-send-email-matan@mellanox.com> <1509358049-18854-3-git-send-email-matan@mellanox.com> <20171030142323.GY26782@6wind.com> In-Reply-To: <20171030142323.GY26782@6wind.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=ophirmu@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB5PR05MB1254; 6:iniqpRVQ0/nTPnxXamSHq81Jz5CY/DSEDlXB14DwgdsVOMJ17ZPqW6DVvBiB2C7rZ+DWblCKmZ4fZewJkmvmlGgrZGtd1R2VQi1u4zxOndZCgaSxwUB1Im/YW4QEuofAoFFS1JKd74r/99BhSIqJyLsWJ92RVTAv/qWkmuF4Aub8Th/r9k5dx9ry061KOcbr0yM8fA1PEsm2I6Haoyo3nD/+NAMeAFiudPnlUqjQhKXIsrUZxcgXv/MDfI8a4qxCFrFZJR1+oKhdTRQ8VrggIHkt+LDtAhXAjVa+JSApz1Wy2gv9LwE+Oo9KbHEzD0nLYVUtlZQyZIvsU3jEQiTLZ3x/AHYFhPNhBNPoQVQK2SE=; 5:lfmXs2gd2EakxwKGsQ4CwOAU+wwfrPga/YbZQwXOZ/gVuigTNLYjYP23xhC+6AdY1fwBBAlMs3SvjEUF3aCHPfugJd40YYyUQKp95ydvzrWqg9OeYlDyRzIwUwhMmnmsGx22Bxzwy/ZeDTPTzcNi+3LMhgTQvFyUA28JGn+xM+0=; 24:XA5AsnwWSHTN4bODLdVGWmo/KG/+ivyBGp5AYBSE9gjxBqmTZMTH3ZgvlR4KVgLekUCpB/EgWS8ntSdW+gbP/7WP1s5WZG1pAL+5fNUJtDc=; 7:3WUGHUpA43PuXuWeYTVtOVmZ9uDzUZCGOE7AtWsgKqBAu+DQQmifuCFkhHt4bdoCJBtfshQ928MaTCgVfdhb+2JmZRMQ9wWi9VvT+pF8MJOIrKDBiWMZlELxnLqElWlbuCXPmdqe8rbXuaDljVMd9rzJiyslDFNJ5Knj9TOWVoantB8g75tPCjnBM99fGC+9rlloKSlRbV89ZRqM8nu6nij6GNxuJkRTYRu42E3Qf0T0/Ss6cfJipfVnT/NZUWrt x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: 5bae3d0e-1823-4260-0218-08d52062d107 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(48565401081)(4534020)(4602075)(2017052603199); SRVR:DB5PR05MB1254; x-ms-traffictypediagnostic: DB5PR05MB1254: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-exchange-antispam-report-test: UriScan:; x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(3002001)(10201501046)(3231020)(93006095)(93001095)(100000703101)(100105400095)(6055026)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123560025)(20161123555025)(20161123562025)(20161123564025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:DB5PR05MB1254; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:DB5PR05MB1254; x-forefront-prvs: 04772EA191 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(346002)(376002)(39860400002)(199003)(24454002)(189002)(107886003)(97736004)(54906003)(2906002)(6246003)(110136005)(68736007)(106356001)(478600001)(105586002)(189998001)(33656002)(101416001)(99286003)(55016002)(66066001)(4326008)(14454004)(5660300001)(6506006)(7696004)(6636002)(25786009)(2900100001)(7736002)(305945005)(6436002)(316002)(229853002)(74316002)(86362001)(575784001)(5250100002)(8676002)(53546010)(53936002)(3280700002)(6116002)(102836003)(3846002)(8936002)(2950100002)(93886005)(81156014)(9686003)(81166006)(50986999)(3660700001)(76176999)(54356999); DIR:OUT; SFP:1101; SCL:1; SRVR:DB5PR05MB1254; H:DB5PR05MB1254.eurprd05.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5bae3d0e-1823-4260-0218-08d52062d107 X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Oct 2017 13:25:12.4668 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR05MB1254 Subject: Re: [dpdk-dev] [PATCH v3 2/7] net/mlx4: associate MR to MP in a short function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Oct 2017 13:25:16 -0000 Hi, Please see inline On Monday, October 30, 2017 4:23 PM, Adrien Mazarguil wrote: >=20 > On Mon, Oct 30, 2017 at 10:07:24AM +0000, Matan Azrad wrote: > > From: Ophir Munk > > > > Associate memory region to mempool (on data path) in a short function. > > Handle the less common case of adding a new memory region to mempool > > in a separate function. > > > > Signed-off-by: Ophir Munk >=20 > Thanks for moving that function out of the way, a few more comments > below. >=20 > By the way, commit log lines can be up to 75 characters wide, only the ti= tle > line is limited to 50. >=20 Commit message updated to be longer > > --- > > drivers/net/mlx4/mlx4.h | 2 ++ > > drivers/net/mlx4/mlx4_mr.c | 46 > +++++++++++++++++++++++++++++++++++ > > drivers/net/mlx4/mlx4_rxtx.c | 57 > > -------------------------------------------- > > drivers/net/mlx4/mlx4_rxtx.h | 33 ++++++++++++++++++++++++- > > 4 files changed, 80 insertions(+), 58 deletions(-) > > > > diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index > > e0a9853..70cf453 100644 > > --- a/drivers/net/mlx4/mlx4.h > > +++ b/drivers/net/mlx4/mlx4.h > > @@ -160,5 +160,7 @@ int mlx4_flow_ctrl_set(struct rte_eth_dev *dev, > > /* mlx4_mr.c */ > > > > struct ibv_mr *mlx4_mp2mr(struct ibv_pd *pd, struct rte_mempool *mp); > > +uint32_t mlx4_txq_add_mr(struct txq *txq, struct rte_mempool *mp, > > + uint32_t i); > > > > #endif /* RTE_PMD_MLX4_H_ */ > > diff --git a/drivers/net/mlx4/mlx4_mr.c b/drivers/net/mlx4/mlx4_mr.c > > index 9700884..ebf6547 100644 > > --- a/drivers/net/mlx4/mlx4_mr.c > > +++ b/drivers/net/mlx4/mlx4_mr.c > > @@ -56,6 +56,7 @@ > > #include > > > > #include "mlx4_utils.h" > > +#include "mlx4_rxtx.h" >=20 > Alphabetical order is preferred, or so I thought. Nothing seems to enforc= e > this in the coding rules, it's only a suggestion. Suggestion accepted >=20 > > > > struct mlx4_check_mempool_data { > > int ret; > > @@ -181,3 +182,48 @@ struct ibv_mr * > > rte_errno =3D errno ? errno : EINVAL; > > return mr; > > } > > + > > +/** > > + * Add memory region (MR) <-> memory pool (MP) association to txq- > >mp2mr[]. > > + * If mp2mr[] is full, remove an entry first. > > + * > > + * @param txq > > + * Pointer to Tx queue structure. > > + * @param[in] mp > > + * Memory pool for which a memory region lkey must be added >=20 > Missing "." Added "." >=20 > > + * @param[in] i > > + * Index in memory pool (MP) where to add memory region (MR) >=20 > Same here. Done >=20 > > + * > > + * @return > > + * Added mr->lkey on success, (uint32_t)-1 on failure. > > + */ > > +uint32_t mlx4_txq_add_mr(struct txq *txq, struct rte_mempool *mp, > > +uint32_t i) >=20 > Coding style again; return type must be on a separate line for definition= s (not > declarations). >=20 > > +{ > > + struct ibv_mr *mr; > > + > > + /* Add a new entry, register MR first. */ > > + DEBUG("%p: discovered new memory pool \"%s\" (%p)", > > + (void *)txq, mp->name, (void *)mp); > > + mr =3D mlx4_mp2mr(txq->priv->pd, mp); > > + if (unlikely(mr =3D=3D NULL)) { > > + DEBUG("%p: unable to configure MR, ibv_reg_mr() failed.", > > + (void *)txq); > > + return (uint32_t)-1; > > + } > > + if (unlikely(i =3D=3D RTE_DIM(txq->mp2mr))) { > > + /* Table is full, remove oldest entry. */ > > + DEBUG("%p: MR <-> MP table full, dropping oldest entry.", > > + (void *)txq); > > + --i; > > + claim_zero(ibv_dereg_mr(txq->mp2mr[0].mr)); > > + memmove(&txq->mp2mr[0], &txq->mp2mr[1], > > + (sizeof(txq->mp2mr) - sizeof(txq->mp2mr[0]))); > > + } > > + /* Store the new entry. */ > > + txq->mp2mr[i].mp =3D mp; > > + txq->mp2mr[i].mr =3D mr; > > + txq->mp2mr[i].lkey =3D mr->lkey; > > + DEBUG("%p: new MR lkey for MP \"%s\" (%p): 0x%08" PRIU32, > > + (void *)txq, mp->name, (void *)mp, txq->mp2mr[i].lkey); > > + return txq->mp2mr[i].lkey; > > +} > > diff --git a/drivers/net/mlx4/mlx4_rxtx.c > > b/drivers/net/mlx4/mlx4_rxtx.c index 4f899ff..f89df46 100644 > > --- a/drivers/net/mlx4/mlx4_rxtx.c > > +++ b/drivers/net/mlx4/mlx4_rxtx.c > > @@ -239,63 +239,6 @@ struct pv { > > } > > > > /** > > - * Get memory region (MR) <-> memory pool (MP) association from txq- > >mp2mr[]. > > - * Add MP to txq->mp2mr[] if it's not registered yet. If mp2mr[] is > > full, > > - * remove an entry first. > > - * > > - * @param txq > > - * Pointer to Tx queue structure. > > - * @param[in] mp > > - * Memory pool for which a memory region lkey must be returned. > > - * > > - * @return > > - * mr->lkey on success, (uint32_t)-1 on failure. > > - */ > > -uint32_t > > -mlx4_txq_mp2mr(struct txq *txq, struct rte_mempool *mp) -{ > > - unsigned int i; > > - struct ibv_mr *mr; > > - > > - for (i =3D 0; (i !=3D RTE_DIM(txq->mp2mr)); ++i) { > > - if (unlikely(txq->mp2mr[i].mp =3D=3D NULL)) { > > - /* Unknown MP, add a new MR for it. */ > > - break; > > - } > > - if (txq->mp2mr[i].mp =3D=3D mp) { > > - assert(txq->mp2mr[i].lkey !=3D (uint32_t)-1); > > - assert(txq->mp2mr[i].mr->lkey =3D=3D txq- > >mp2mr[i].lkey); > > - return txq->mp2mr[i].lkey; > > - } > > - } > > - /* Add a new entry, register MR first. */ > > - DEBUG("%p: discovered new memory pool \"%s\" (%p)", > > - (void *)txq, mp->name, (void *)mp); > > - mr =3D mlx4_mp2mr(txq->priv->pd, mp); > > - if (unlikely(mr =3D=3D NULL)) { > > - DEBUG("%p: unable to configure MR, ibv_reg_mr() failed.", > > - (void *)txq); > > - return (uint32_t)-1; > > - } > > - if (unlikely(i =3D=3D RTE_DIM(txq->mp2mr))) { > > - /* Table is full, remove oldest entry. */ > > - DEBUG("%p: MR <-> MP table full, dropping oldest entry.", > > - (void *)txq); > > - --i; > > - claim_zero(ibv_dereg_mr(txq->mp2mr[0].mr)); > > - memmove(&txq->mp2mr[0], &txq->mp2mr[1], > > - (sizeof(txq->mp2mr) - sizeof(txq->mp2mr[0]))); > > - } > > - /* Store the new entry. */ > > - txq->mp2mr[i].mp =3D mp; > > - txq->mp2mr[i].mr =3D mr; > > - txq->mp2mr[i].lkey =3D mr->lkey; > > - DEBUG("%p: new MR lkey for MP \"%s\" (%p): 0x%08" PRIu32, > > - (void *)txq, mp->name, (void *)mp, txq->mp2mr[i].lkey); > > - return txq->mp2mr[i].lkey; > > -} > > - > > -/** > > * Posts a single work request to a send queue. > > * > > * @param txq > > diff --git a/drivers/net/mlx4/mlx4_rxtx.h > > b/drivers/net/mlx4/mlx4_rxtx.h index 7d67748..b32fcc9 100644 > > --- a/drivers/net/mlx4/mlx4_rxtx.h > > +++ b/drivers/net/mlx4/mlx4_rxtx.h > > @@ -36,6 +36,7 @@ > > > > #include > > #include > > +#include > > > > /* Verbs headers do not support -pedantic. */ #ifdef PEDANTIC @@ > > -165,7 +166,6 @@ int mlx4_rx_queue_setup(struct rte_eth_dev *dev, > > uint16_t idx, > > > > /* mlx4_rxtx.c */ > > > > -uint32_t mlx4_txq_mp2mr(struct txq *txq, struct rte_mempool *mp); > > uint16_t mlx4_tx_burst(void *dpdk_txq, struct rte_mbuf **pkts, > > uint16_t pkts_n); > > uint16_t mlx4_rx_burst(void *dpdk_rxq, struct rte_mbuf **pkts, @@ > > -182,4 +182,35 @@ int mlx4_tx_queue_setup(struct rte_eth_dev *dev, > uint16_t idx, > > const struct rte_eth_txconf *conf); void > > mlx4_tx_queue_release(void *dpdk_txq); > > > > + >=20 > Extra empty line, however see below. >=20 Extra line removed > > +/** > > + * Get memory region (MR) <-> memory pool (MP) association from txq- > >mp2mr[]. > > + * Call mlx4_txq_add_mr() if MP is not registered yet. > > + * > > + * @param txq > > + * Pointer to Tx queue structure. > > + * @param[in] mp > > + * Memory pool for which a memory region lkey must be returned. > > + * > > + * @return > > + * mr->lkey on success, (uint32_t)-1 on failure. > > + */ > > +static inline uint32_t > > +mlx4_txq_mp2mr(struct txq *txq, struct rte_mempool *mp) { > > + unsigned int i; > > + > > + for (i =3D 0; (i !=3D RTE_DIM(txq->mp2mr)); ++i) { > > + if (unlikely(txq->mp2mr[i].mp =3D=3D NULL)) { > > + /* Unknown MP, add a new MR for it. */ > > + break; > > + } > > + if (txq->mp2mr[i].mp =3D=3D mp) { > > + assert(txq->mp2mr[i].lkey !=3D (uint32_t)-1); > > + assert(txq->mp2mr[i].mr->lkey =3D=3D txq- > >mp2mr[i].lkey); > > + return txq->mp2mr[i].lkey; > > + } > > + } > > + return mlx4_txq_add_mr(txq, mp, i); > > +} >=20 > I'm OK with keeping this function as a static inline defined in a header = file, > however I think mlx4_rxtx.h is not the right place for it given the remai= ning > MR-handling functions are declared by mlx4.h. You should move it to that = file > as well (likely below mlx4_txq_add_mr()). >=20 > Note: RTE_DIM() and unlikely() require rte_common.h, NULL requires > stddef.h. > Please note that moving mlx4_txq_mp2mr() to mlx4.h requires to "#include ml= x4_rxtx.h" in mlx4.h=20 (because of accessing struct txq fields). At the same time mlx4_rxtx.h requires to "#include mlx4.h" because it uses = definitions such as=20 MLX4_RSS_HASH_KEY_SIZE. End result is that mlx4.h and mlx4_rxtx.h include each other.=20 It does not compile without rewriting code (domino effect) which I don't t= hink is a desirable result=20 of this review (unless you specify otherwise). Therefore for v4 I have left mlx4_txq_mp2mr() in mlx4_rxtx.h =20 > While there, I think you should drop both assert() statements and the rel= iance > on assert.h because such statements shouldn't be part of a static inline > function not supposed to fail. Let's assume mp2mr[] is correctly updated > upstream. >=20 assert() - removed > (Also missing empty line here.) >=20 Empty line added > > #endif /* MLX4_RXTX_H_ */ > > -- > > 1.8.3.1 > > >=20 > -- > Adrien Mazarguil > 6WIND