From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00072.outbound.protection.outlook.com [40.107.0.72]) by dpdk.org (Postfix) with ESMTP id 61EC51B210 for ; Fri, 6 Oct 2017 03:11:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=o8wDg3jhhOjxq1SWcpeej9/QpnXnsb9pjZEnPj70Nrc=; b=yt2URKvsWAOLIQFjXOm+NAnedBig1mfxa+T37cM/M0rx+ha08Y+xvFvmYl81Pkj4bETl9DF3PtX2r34eW2oA9aldFqLi5xU1eCTynbeALE1B7Zj4/918iCJ5Ha/HmtPYlkk9rf/6DVXPg2GF1nygM+I6N2kvetJ3Ti5346VPbCs= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; Received: from yongseok-MBP.local (209.116.155.178) by AM5PR0501MB2033.eurprd05.prod.outlook.com (2603:10a6:203:1a::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Fri, 6 Oct 2017 01:11:36 +0000 Date: Thu, 5 Oct 2017 18:11:26 -0700 From: Yongseok Koh To: Nelio Laranjeiro Cc: dev@dpdk.org, adrien.mazarguil@6wind.com, ferruh.yigit@intel.com Message-ID: <20171006011125.GA19124@yongseok-MBP.local> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.2 (2016-11-26) X-Originating-IP: [209.116.155.178] X-ClientProxiedBy: BN6PR14CA0022.namprd14.prod.outlook.com (2603:10b6:404:79::32) To AM5PR0501MB2033.eurprd05.prod.outlook.com (2603:10a6:203:1a::19) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8641c8f9-7bb5-4b23-696b-08d50c57317b X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(48565401081)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:AM5PR0501MB2033; X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2033; 3:AfWYJVAWItuRqp2ZCPIUDYYCZJRVQ4Ff/imo1ayKFk5lnJ94bOChpaYTMe3Azcx9ou7unmL0Nk2AB85OjB2gr+w/RNpD5w9vX64aHMYrg+igwyguNQmmFgmLmf1T0zMdWdns+mZUJXgvwT5hFurTtmEA3Crjyulvusp/pWGwISgfIZgJ3tMXk5dUNgbex00nBGfvcYbwNV7djI6eRdv87za9c4g3JhuFAyHWQGN04B3wIYOoW8inB61BEoyQd2B+; 25:m9tIfVmUd7PKUP5jar6QOFshEiqB6HSYfrWw+KuJkc6VQPwGOpozWIsGewloXZpUQ2EshLq0u+uYrGaP2+NxiCkvFm/mSNCqXnrAubfUkA37g5T+6w59L+61jkWJMDEdIuNJabOm1+pgjME219L/dsRY8TWTDp3okUD+zyLLkOhn46YHzjYBZih4dkkorFfgjGE1II8XoYLzLefJUv/y7r5XbYWjzQxw8Q/37yeBA075gdt2ZEtAHULCHHVYUIqeHs9zasLLiAGuFappPaFiyh7i1bWH8ob8zYV3JeySE94XiedUacZdEv12TKHyLW3QG8J2sywTVeUft9gQfEfKNw==; 31:YzAUT8+hAxq0zTwsO/jFVPmm9elKmOn4pUOZUiuENBN/kDHVrJv2+2EYWH64ePyre0t1WPJCC6qbzZyi9n/fVSsjPYSD7pBfz6KTrn1S3wTIL17wq1dV/iiIEz4T3Sr3GgO9KT207uZbn+VHhUbfsT3J5mAHTflplDhwGc/suXFppUxWFvSaWV+9IXdAqjMScan3yhs6B43IlX8BP/eI3UfqwPS83HGsj/mDKO1XDNw= X-MS-TrafficTypeDiagnostic: AM5PR0501MB2033: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2033; 20:Zr6sUqWVeBzhXbudpLZOgeq/w1FPMJG+Qd2fJbYl5pfx+nXGONSQZSMxZLQ2H5gPgm8aERyGo1aT9JdCgIIolMixU05mdVf01f6xJZYuOl5c02u3vQFpRDaFff4hEKUTzYT9Uf75grFr4/H+Tw/RazsmgKsjfAy9WuSZTqOQBpiHvdSd+WwPnY+ToZRvd55PWCCvc11uG/4bGtaIJQTzPitJTU6a376GvFRJp3uJgZq3+VtAjFkvrzlb8chmvZBZBRcjpKv50KGsRNNaT1E21YMhSak03ehJYBcmFysuWTNDtuzEO9x6TldZRyfAZ4ugDaNBQcWWm0FAO4XznoP05jbFflnLx4K61R1o1YrVvFQGS9a0OCE5down/6ijhEmt7YSEvvJI2hYQf1bkvQze+uQzEi+rpCpnFr4VjOkK0NEV9bVCNKLDiDQKDuPSN23DNp9/Ck71TwYefbZGa3UE1+zK5WTiK9RFg0QZWGPd2Pa99tSKSDQp+4YtvFYwhAem; 4:q6qEWtaQ5BTASpEidxGoqhhd5CUsM7nZxuloCJwjlpF7bBwpexvxS73XSLRLCiZL/ekfdYmYIAlRgXET41ZUOUnYZUOIQ2Fgd1Is2Edr5SAHDVmRvD/Z3JLRLAwR9P7sSI2Jk3Te614eW/2QqZ3uegT6nTwys0eONSTaeDMc77IYQVsPejMrxMwtGMJh8bpZe04h6ORjLQRkFaNSukT2ZxirPD80//oWfTiUlY3/C82cIQNsRIKM0QCJxhdXIIOS X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(93006095)(93001095)(10201501046)(3002001)(100000703101)(100105400095)(6055026)(6041248)(20161123560025)(20161123558100)(20161123555025)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:AM5PR0501MB2033; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:AM5PR0501MB2033; X-Forefront-PRVS: 0452022BE1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(376002)(346002)(39860400002)(24454002)(189002)(199003)(5660300001)(66066001)(97736004)(316002)(23726003)(83506001)(6666003)(3846002)(6116002)(2906002)(1076002)(4326008)(16526018)(50986999)(58126008)(6506006)(101416001)(105586002)(54356999)(76176999)(229853002)(33656002)(98436002)(2950100002)(53936002)(6916009)(16586007)(106356001)(55016002)(6246003)(478600001)(8676002)(7736002)(8936002)(189998001)(50466002)(9686003)(47776003)(81156014)(81166006)(68736007)(25786009)(86362001)(305945005)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR0501MB2033; H:yongseok-MBP.local; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM5PR0501MB2033; 23:VojVozWe9X7Z6aoXByx0yyKP9uhNwBoQROzIozL?= =?us-ascii?Q?ST+LyO3/U2H0so4k6Uf34EoMob6tPZLDseZaji+va15BV8we18Y+U/WpnuWn?= =?us-ascii?Q?rMVgTeoVI0mKLC0TLF+xKVTgjw+kJtsITPCyl3S+GsMn5Xo0MGlm0zS2bXnW?= =?us-ascii?Q?ip159TFGKvvBkSjUseqepBNIEi1WmFyaihdwUUGn5gniN1xHfEbKSFjMoC5u?= =?us-ascii?Q?KbmgRPHIFq884qjKEO6fnaWTjZG15w6p68HBtjiTB07VEXVgHlqtbz89x8C4?= =?us-ascii?Q?2Rq7EJwpYYUErzajrIyQAjTtN84/GPJ7JmK/v0nxyTUFO/klfmJSHgS+8Etn?= =?us-ascii?Q?kDwKROFaiNXwudFm3em8JkD0y16Zdm8yv+JSI/Wu0xITV4bnUkMKJDmRA98D?= =?us-ascii?Q?TzddS4Q4UG+ZkP93pdkIAjahhi4HGeml7pyhLOgRhRH18OlgAvzqPMIOoCER?= =?us-ascii?Q?yoOgU+P5OlAyHgUcWJGexWyflKRMyZ7R4sQ3KY7kMy2klXrl2/UtHP5Ki2vk?= =?us-ascii?Q?TKUAnGIwPT7Y9OQTSCrh7cnY6wqhcVfSesHj8T86KIWlvIr9TnqINiK1IU0c?= =?us-ascii?Q?x0XbK9i9j+JajBAtjnk3o/8G7OV09oOSJyE8u2Qi/C1+FNeTCF9LjYnXnN7H?= =?us-ascii?Q?fM29gWnS0UKg2pflpZv96bVOSKae/NgIJbs5MvkbLIsY1oaq+zCe0adXsCC4?= =?us-ascii?Q?1Bf+xebXjd0B0cJ1ourcuo6dcKps+VElUGCiws4Hz4BHuidoh5Gt2loWe80y?= =?us-ascii?Q?DJVcYlRFMFpGdQOJItU4Fx0SYXiod6lZrMVZq0/GRQMbKHfbjMzwIW/pDg8b?= =?us-ascii?Q?jiIF41Yz56fVOMBqO/YA/JSqRRh+HqP8wMr9ElydvUE5CcWtKZ0J8raRiCpc?= =?us-ascii?Q?zJBemhJErF82zycLwQFpvxTkN1+eV2FjKyKMwVg6V74TN5aYb8BJsAs+O1aJ?= =?us-ascii?Q?igXTEt3fQHSxC/OcE09EpHWCei+OVqpOX+6nihq1E97dHhZwc0W3yOn2DE0s?= =?us-ascii?Q?eS1JXBRFuf19vaeh7Ttg5GQ49CWvFWQfytFPaA/ikhttKskleCjQ8bbTG8eo?= =?us-ascii?Q?fB1vZC0LSV5eaSL03ChKAlA5UQXF6yFufmaXfQg1pqV93DAvdGc2J8ECXdvk?= =?us-ascii?Q?XP2B2eBZBrfjPk8oHT/FK8g/evUICWn5ykh8JgghuRi/rOId1tKktizH/8kX?= =?us-ascii?Q?wqlxmctFxlykgFPph/2c4oq1RMT7LXYHk3Xmk?= X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2033; 6:nfWUYBbDutkE3m/LJqGIDxht91qaxD7Ar+8WdIpi8czBiGJqdZs6W69OxFar0uFYc+hFL/OWWXM0YYBqWf9LSD1CeYft6C4H9oQrB4SCkHr/InQKKyVQ+0IGOxd8nBMV8fTMq+crg/KQ5LbTzLfhN6rOgQF/9OG1rNsraC19nDNeXA0hHXej+y0xuHy3bpACOjUPQQRxPDmrjEvx2/SMdDN7FBu2v58lzjkmJoqnVpUeZjKdaXJvZwwU8TEU21EgwC6JNdXq43ar8Qlijx5vFZMXLaVPJEKN2Wx/iNQuwEc+8/xFGxWu7i9i5dFpCO0NFBHq0QREixodCzcYcRwXLg==; 5:AsPR4aMpgx/hxdUXzWBm2/lq/mPxZr916OTQj90iyWfv4+PbGnW9WBQAYXeLyq7Xm6Gq26iCyvtRem1Yq1iyfjKDvRPZwTKD74MYGKB0oZfR3wUHGu2L6U5BJM9GlOT0Y6dvENRcgcS68eBHwxnXbA==; 24:Y9orVb/3aRvvgTRjCxZNjacyiEOFS0c92Q5kIBwKSNAcI+9/7MPJElnKDSPJxoZpVMznqY/BIZGDwyjiPwGpxqCBjXTXDqBh+qcIOziIwjc=; 7:a779jhD2ArtgvDgiTVWyVVqQnGEK3eteaSZlNEYi4FyXrOdtVOfTaq+YDoKXIbnRR13n0aFt+lBH2vv8aOMKm4oxPIlBctOJIIH/YMqQ4Ce/spnVsubq0k/EC501EVvyBZwAJAuol3LS/bv+pf81QBCtGTk9oaSDSgg9eQXfDIKdo1HFYHN3RaJb9t0N2SoGn+X4LH6ZrnnW0pNZB667wjRNPBw55o5twz0pl9yGvrs= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Oct 2017 01:11:36.5277 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0501MB2033 Subject: Re: [dpdk-dev] [PATCH v2 09/30] net/mlx5: add reference counter on memory region X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Oct 2017 01:11:40 -0000 On Thu, Oct 05, 2017 at 02:49:41PM +0200, Nelio Laranjeiro wrote: [...] > @@ -180,12 +133,14 @@ mlx5_txq_mp2mr_reg(struct mlx5_txq_data *txq, struct rte_mempool *mp, > { > struct mlx5_txq_ctrl *txq_ctrl = > container_of(txq, struct mlx5_txq_ctrl, txq); > - struct ibv_mr *mr; > + struct mlx5_mr *mr; > > /* Add a new entry, register MR first. */ > DEBUG("%p: discovered new memory pool \"%s\" (%p)", > (void *)txq_ctrl, mp->name, (void *)mp); > - mr = mlx5_mp2mr(txq_ctrl->priv->pd, mp); > + mr = priv_mr_get(txq_ctrl->priv, mp); > + if (mr == NULL) > + mr = priv_mr_new(txq_ctrl->priv, mp); > if (unlikely(mr == NULL)) { > DEBUG("%p: unable to configure MR, ibv_reg_mr() failed.", > (void *)txq_ctrl); > @@ -196,20 +151,17 @@ mlx5_txq_mp2mr_reg(struct mlx5_txq_data *txq, struct rte_mempool *mp, > DEBUG("%p: MR <-> MP table full, dropping oldest entry.", > (void *)txq_ctrl); > --idx; > - claim_zero(ibv_dereg_mr(txq_ctrl->txq.mp2mr[0].mr)); > + priv_mr_release(txq_ctrl->priv, txq_ctrl->txq.mp2mr[0]); In this function, txq_ctrl->txq can be replaced with txq. [...] > @@ -564,26 +572,34 @@ mlx5_tx_mb2mr(struct mlx5_txq_data *txq, struct rte_mbuf *mb) > { > uint16_t i = txq->mr_cache_idx; > uintptr_t addr = rte_pktmbuf_mtod(mb, uintptr_t); > + uint32_t lkey; > > assert(i < RTE_DIM(txq->mp2mr)); > - if (likely(txq->mp2mr[i].start <= addr && txq->mp2mr[i].end >= addr)) > - return txq->mp2mr[i].lkey; > + if (likely(txq->mp2mr[i]->start <= addr && txq->mp2mr[i]->end >= addr)) > + return txq->mp2mr[i]->lkey; > for (i = 0; (i != RTE_DIM(txq->mp2mr)); ++i) { > - if (unlikely(txq->mp2mr[i].mr == NULL)) { > + if (unlikely(txq->mp2mr[i]->mr == NULL)) { > /* Unknown MP, add a new MR for it. */ > break; > } > - if (txq->mp2mr[i].start <= addr && > - txq->mp2mr[i].end >= addr) { > - assert(txq->mp2mr[i].lkey != (uint32_t)-1); > - assert(rte_cpu_to_be_32(txq->mp2mr[i].mr->lkey) == > - txq->mp2mr[i].lkey); > + if (txq->mp2mr[i]->start <= addr && > + txq->mp2mr[i]->end >= addr) { > + assert(txq->mp2mr[i]->lkey != (uint32_t)-1); > + assert(rte_cpu_to_be_32(txq->mp2mr[i]->mr->lkey) == > + txq->mp2mr[i]->lkey); > txq->mr_cache_idx = i; > - return txq->mp2mr[i].lkey; > + return txq->mp2mr[i]->lkey; > } > } > txq->mr_cache_idx = 0; > - return mlx5_txq_mp2mr_reg(txq, mlx5_tx_mb2mp(mb), i); > + lkey = mlx5_txq_mp2mr_reg(txq, mlx5_tx_mb2mp(mb), i); > + /* > + * Request the reference to use in this queue, the original one is > + * kept by the control plane. > + */ > + if (lkey != (uint32_t)-1) > + rte_atomic32_inc(&txq->mp2mr[i]->refcnt); If mp2mr is overflowed (i == RTE_DIM(txq->mp2mr)), then mp2mr[0] will be removed with shifting other slots and the new entry will be added at the end. But referencing txq->mp2mr[i] would be illegal - out of range. Thanks, Yongseok