From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80089.outbound.protection.outlook.com [40.107.8.89]) by dpdk.org (Postfix) with ESMTP id AD77A4C90; Wed, 1 May 2019 05:33:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=15OQ+KW5uNFqAp2HGkshY3I/3NX/i2diprVyhqPhNzo=; b=EXWdRRLyByDgMbuaEP25m5NiDK8iLYU+K/LwUKDqBbu87OZtQttapX5K3SheuTU32A2FQ+6jiKZkK/fvDSYDM2l/6m7O+6ekSwiVdrULoo1tSB2S1zb3JnUUi0WLWQpS7AHe5ys4J9zgLtAfotPzg0VaSAc8+UtcsvqNxPJV8rA= Received: from AM0PR08MB3379.eurprd08.prod.outlook.com (20.177.109.142) by AM0PR08MB3139.eurprd08.prod.outlook.com (52.134.93.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1856.10; Wed, 1 May 2019 03:33:28 +0000 Received: from AM0PR08MB3379.eurprd08.prod.outlook.com ([fe80::2d20:bb91:f5bf:256e]) by AM0PR08MB3379.eurprd08.prod.outlook.com ([fe80::2d20:bb91:f5bf:256e%6]) with mapi id 15.20.1856.008; Wed, 1 May 2019 03:33:28 +0000 From: Dharmik Thakkar To: "bugzilla@dpdk.org" CC: "dev@dpdk.org" , "zhongdahulinfan@163.com" , nd , Honnappa Nagarahalli Thread-Topic: [dpdk-dev] [Bug 261] DPDK 18.11 bug on rte_hash_free_key_with_position Thread-Index: AQHU/zOopXR3JVjlDkqTOg8it2nYcKZVnt+A Date: Wed, 1 May 2019 03:33:28 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Dharmik.Thakkar@arm.com; x-originating-ip: [2605:6000:eb05:8c00:a853:bed7:2e48:e791] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 94871c1d-a2a8-4166-e766-08d6cde5c6a4 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(4618075)(2017052603328)(7193020); SRVR:AM0PR08MB3139; x-ms-traffictypediagnostic: AM0PR08MB3139: x-ms-exchange-purlcount: 1 nodisclaimer: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-forefront-prvs: 00246AB517 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(346002)(39860400002)(136003)(376002)(396003)(199004)(189003)(6246003)(53936002)(83716004)(14444005)(33656002)(71200400001)(71190400001)(6116002)(7736002)(4326008)(25786009)(186003)(36756003)(91956017)(2906002)(76116006)(73956011)(66946007)(72206003)(1730700003)(81166006)(305945005)(8936002)(966005)(14454004)(82746002)(102836004)(6916009)(81156014)(46003)(486006)(66446008)(64756008)(66556008)(11346002)(256004)(66476007)(476003)(2616005)(446003)(478600001)(76176011)(68736007)(53546011)(6506007)(316002)(86362001)(6436002)(5640700003)(6512007)(6306002)(54906003)(229853002)(99286004)(5660300002)(2351001)(6486002)(2501003)(21314003); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR08MB3139; H:AM0PR08MB3379.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: 8CSFPM+RTqR3Ub8GrVu7T07JkRUBvdWxva+iiuSlCp6Q9+eR/9wo7DOFmaLrUXB/Zsq4WbPepgzV5PxXvvH+jupfgEsDGyETFdwaN5q9HKESxtFV6iXpxHIT2caIDdnxmJGxMYzWOwztLaQHeQ2NB4WtDP9UyVEqGLKapd1cnPIlXst+LbDlqTOZU3+SHKphFofkNZTIstdsXe8GW9bmi7l+zks2EkO6lvuRw6QM3uT3rii2+w9OP3PfjSierc66VqV3UpfpnMJqm7bCi/5XDMqgKWCN+iyhhhD8gXZyhpwk7z8eThrrwkoUkzBYUTnJRVZd4CH8hzOgCe9TNfAv44P/D1gFljUZhHezG0ngfO1CB2Gyx1OnVRn4njoHUsRaV2yEkJZBKopSnKEl93zgfhN7OyjIRjGWi/E4ubj0HJo= Content-Type: text/plain; charset="us-ascii" Content-ID: <297C20059D9E0D48ABBFB64F4C73DB11@eurprd08.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: 94871c1d-a2a8-4166-e766-08d6cde5c6a4 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 May 2019 03:33:28.1723 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3139 Subject: Re: [dpdk-dev] [Bug 261] DPDK 18.11 bug on rte_hash_free_key_with_position X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 03:33:30 -0000 I am taking a look at this bug. Will update ASAP. Did you run any test case= to detect the bug? Thank you! > On Apr 30, 2019, at 4:03 AM, bugzilla@dpdk.org wrote: >=20 > https://bugs.dpdk.org/show_bug.cgi?id=3D261 >=20 > Bug ID: 261 > Summary: DPDK 18.11 bug on rte_hash_free_key_with_position > Product: DPDK > Version: 18.11 > Hardware: All > OS: All > Status: CONFIRMED > Severity: normal > Priority: Normal > Component: other > Assignee: dev@dpdk.org > Reporter: zhongdahulinfan@163.com > Target Milestone: --- >=20 > First let's see the definition of rte_hash_free_key_with_position in DPDK > 18.11, as shown bellow:=20 >=20 > int __rte_experimental > rte_hash_free_key_with_position(const struct rte_hash *h, > const int32_t position) > { > RETURN_IF_TRUE(((h =3D=3D NULL) || (position =3D=3D EMPTY_SLOT)), -EIN= VAL); >=20 > unsigned int lcore_id, n_slots; > struct lcore_cache *cached_free_slots; > const int32_t total_entries =3D h->num_buckets * RTE_HASH_BUCKET_ENTRI= ES; >=20 > /* Out of bounds */ > if (position >=3D total_entries) > return -EINVAL; >=20 > if (h->use_local_cache) { > lcore_id =3D rte_lcore_id(); > cached_free_slots =3D &h->local_free_slots[lcore_id]; > /* Cache full, need to free it. */ > if (cached_free_slots->len =3D=3D LCORE_CACHE_SIZE) { > /* Need to enqueue the free slots in global ring. */ > n_slots =3D rte_ring_mp_enqueue_burst(h->free_slots, > cached_free_slots->objs, > LCORE_CACHE_SIZE, NULL); > cached_free_slots->len -=3D n_slots; > } > /* Put index of new free slot in cache. */ > cached_free_slots->objs[cached_free_slots->len] =3D > (void *)((uintptr_t)position); > cached_free_slots->len++; > } else { > rte_ring_sp_enqueue(h->free_slots, > (void *)((uintptr_t)position)); > } >=20 > return 0; > } >=20 > There are two issues for this API. >=20 > First, the input parameter 'position' is the key index of the hash table,= which > is returned by rte_hash_add_key_xxx or rte_hash_del_key_xxx. Take a glanc= e look > of rte_hash_del_key_with_hash for example, we see that it returns key_idx= - 1 > if entry found and removed successfully. Hence rte_hash_free_key_with_pos= ition > is not correct while it enqueues position into free_slots directly. It mu= st > increase position by one to get the right key index, before enqueues into > free_slots. >=20 > As comparision, remove_entry()enqueue key_idx directly, which is correct:= =20 >=20 > static inline void > remove_entry(const struct rte_hash *h, struct rte_hash_bucket *bkt, unsig= ned i) > { > unsigned lcore_id, n_slots; > struct lcore_cache *cached_free_slots; >=20 > if (h->use_local_cache) { > lcore_id =3D rte_lcore_id(); > cached_free_slots =3D &h->local_free_slots[lcore_id]; > /* Cache full, need to free it. */ > if (cached_free_slots->len =3D=3D LCORE_CACHE_SIZE) { > /* Need to enqueue the free slots in global ring. = */ > n_slots =3D rte_ring_mp_enqueue_burst(h->free_slot= s, > cached_free_slots->objs, > LCORE_CACHE_SIZE, NULL); > cached_free_slots->len -=3D n_slots; > } > /* Put index of new free slot in cache. */ > cached_free_slots->objs[cached_free_slots->len] =3D > (void *)((uintptr_t)bkt->key_idx[i]); > cached_free_slots->len++; > } else { > rte_ring_sp_enqueue(h->free_slots, > (void *)((uintptr_t)bkt->key_idx[i])); > } > } >=20 > Second, computation of total_entries is not correct. This should be the t= otal > number of key slots.The number of key slots is seen as rte_hash_create, s= ay > (params->entries + (RTE_MAX_LCORE - 1) *(LCORE_CACHE_SIZE - 1) + 1) when > use_local_cache is true, else (params->entries + 1) >=20 > struct rte_hash * > rte_hash_create(const struct rte_hash_parameters *params) > { > ... > if (params->extra_flag & RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD) { > use_local_cache =3D 1; > writer_takes_lock =3D 1; > } > ... > /* Store all keys and leave the first entry as a dummy entry for > lookup_bulk */ > if (use_local_cache) > /* > * Increase number of slots by total number of indices > * that can be stored in the lcore caches > * except for the first cache > */ > num_key_slots =3D params->entries + (RTE_MAX_LCORE - 1) * > (LCORE_CACHE_SIZE - 1) + 1; > else > num_key_slots =3D params->entries + 1; > ... > /* Populate free slots ring. Entry zero is reserved for key misses. */ > for (i =3D 1; i < num_key_slots; i++) > rte_ring_sp_enqueue(r, (void *)((uintptr_t) i)); > ... > } >=20 > --=20 > You are receiving this mail because: > You are the assignee for the bug. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id E38A0A0ACC for ; Wed, 1 May 2019 05:33:32 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BC1EF58FA; Wed, 1 May 2019 05:33:31 +0200 (CEST) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80089.outbound.protection.outlook.com [40.107.8.89]) by dpdk.org (Postfix) with ESMTP id AD77A4C90; Wed, 1 May 2019 05:33:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=15OQ+KW5uNFqAp2HGkshY3I/3NX/i2diprVyhqPhNzo=; b=EXWdRRLyByDgMbuaEP25m5NiDK8iLYU+K/LwUKDqBbu87OZtQttapX5K3SheuTU32A2FQ+6jiKZkK/fvDSYDM2l/6m7O+6ekSwiVdrULoo1tSB2S1zb3JnUUi0WLWQpS7AHe5ys4J9zgLtAfotPzg0VaSAc8+UtcsvqNxPJV8rA= Received: from AM0PR08MB3379.eurprd08.prod.outlook.com (20.177.109.142) by AM0PR08MB3139.eurprd08.prod.outlook.com (52.134.93.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1856.10; Wed, 1 May 2019 03:33:28 +0000 Received: from AM0PR08MB3379.eurprd08.prod.outlook.com ([fe80::2d20:bb91:f5bf:256e]) by AM0PR08MB3379.eurprd08.prod.outlook.com ([fe80::2d20:bb91:f5bf:256e%6]) with mapi id 15.20.1856.008; Wed, 1 May 2019 03:33:28 +0000 From: Dharmik Thakkar To: "bugzilla@dpdk.org" CC: "dev@dpdk.org" , "zhongdahulinfan@163.com" , nd , Honnappa Nagarahalli Thread-Topic: [dpdk-dev] [Bug 261] DPDK 18.11 bug on rte_hash_free_key_with_position Thread-Index: AQHU/zOopXR3JVjlDkqTOg8it2nYcKZVnt+A Date: Wed, 1 May 2019 03:33:28 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Dharmik.Thakkar@arm.com; x-originating-ip: [2605:6000:eb05:8c00:a853:bed7:2e48:e791] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 94871c1d-a2a8-4166-e766-08d6cde5c6a4 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(4618075)(2017052603328)(7193020); SRVR:AM0PR08MB3139; x-ms-traffictypediagnostic: AM0PR08MB3139: x-ms-exchange-purlcount: 1 nodisclaimer: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-forefront-prvs: 00246AB517 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(346002)(39860400002)(136003)(376002)(396003)(199004)(189003)(6246003)(53936002)(83716004)(14444005)(33656002)(71200400001)(71190400001)(6116002)(7736002)(4326008)(25786009)(186003)(36756003)(91956017)(2906002)(76116006)(73956011)(66946007)(72206003)(1730700003)(81166006)(305945005)(8936002)(966005)(14454004)(82746002)(102836004)(6916009)(81156014)(46003)(486006)(66446008)(64756008)(66556008)(11346002)(256004)(66476007)(476003)(2616005)(446003)(478600001)(76176011)(68736007)(53546011)(6506007)(316002)(86362001)(6436002)(5640700003)(6512007)(6306002)(54906003)(229853002)(99286004)(5660300002)(2351001)(6486002)(2501003)(21314003); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR08MB3139; H:AM0PR08MB3379.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: 8CSFPM+RTqR3Ub8GrVu7T07JkRUBvdWxva+iiuSlCp6Q9+eR/9wo7DOFmaLrUXB/Zsq4WbPepgzV5PxXvvH+jupfgEsDGyETFdwaN5q9HKESxtFV6iXpxHIT2caIDdnxmJGxMYzWOwztLaQHeQ2NB4WtDP9UyVEqGLKapd1cnPIlXst+LbDlqTOZU3+SHKphFofkNZTIstdsXe8GW9bmi7l+zks2EkO6lvuRw6QM3uT3rii2+w9OP3PfjSierc66VqV3UpfpnMJqm7bCi/5XDMqgKWCN+iyhhhD8gXZyhpwk7z8eThrrwkoUkzBYUTnJRVZd4CH8hzOgCe9TNfAv44P/D1gFljUZhHezG0ngfO1CB2Gyx1OnVRn4njoHUsRaV2yEkJZBKopSnKEl93zgfhN7OyjIRjGWi/E4ubj0HJo= Content-Type: text/plain; charset="UTF-8" Content-ID: <297C20059D9E0D48ABBFB64F4C73DB11@eurprd08.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: 94871c1d-a2a8-4166-e766-08d6cde5c6a4 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 May 2019 03:33:28.1723 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3139 Subject: Re: [dpdk-dev] [Bug 261] DPDK 18.11 bug on rte_hash_free_key_with_position X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190501033328.utvQNNcSRYtMfw3bYS5_-WnHXxNS3Q3ZSWuQ4xu2gK8@z> I am taking a look at this bug. Will update ASAP. Did you run any test case= to detect the bug? Thank you! > On Apr 30, 2019, at 4:03 AM, bugzilla@dpdk.org wrote: >=20 > https://bugs.dpdk.org/show_bug.cgi?id=3D261 >=20 > Bug ID: 261 > Summary: DPDK 18.11 bug on rte_hash_free_key_with_position > Product: DPDK > Version: 18.11 > Hardware: All > OS: All > Status: CONFIRMED > Severity: normal > Priority: Normal > Component: other > Assignee: dev@dpdk.org > Reporter: zhongdahulinfan@163.com > Target Milestone: --- >=20 > First let's see the definition of rte_hash_free_key_with_position in DPDK > 18.11, as shown bellow:=20 >=20 > int __rte_experimental > rte_hash_free_key_with_position(const struct rte_hash *h, > const int32_t position) > { > RETURN_IF_TRUE(((h =3D=3D NULL) || (position =3D=3D EMPTY_SLOT)), -EIN= VAL); >=20 > unsigned int lcore_id, n_slots; > struct lcore_cache *cached_free_slots; > const int32_t total_entries =3D h->num_buckets * RTE_HASH_BUCKET_ENTRI= ES; >=20 > /* Out of bounds */ > if (position >=3D total_entries) > return -EINVAL; >=20 > if (h->use_local_cache) { > lcore_id =3D rte_lcore_id(); > cached_free_slots =3D &h->local_free_slots[lcore_id]; > /* Cache full, need to free it. */ > if (cached_free_slots->len =3D=3D LCORE_CACHE_SIZE) { > /* Need to enqueue the free slots in global ring. */ > n_slots =3D rte_ring_mp_enqueue_burst(h->free_slots, > cached_free_slots->objs, > LCORE_CACHE_SIZE, NULL); > cached_free_slots->len -=3D n_slots; > } > /* Put index of new free slot in cache. */ > cached_free_slots->objs[cached_free_slots->len] =3D > (void *)((uintptr_t)position); > cached_free_slots->len++; > } else { > rte_ring_sp_enqueue(h->free_slots, > (void *)((uintptr_t)position)); > } >=20 > return 0; > } >=20 > There are two issues for this API. >=20 > First, the input parameter 'position' is the key index of the hash table,= which > is returned by rte_hash_add_key_xxx or rte_hash_del_key_xxx. Take a glanc= e look > of rte_hash_del_key_with_hash for example, we see that it returns key_idx= - 1 > if entry found and removed successfully. Hence rte_hash_free_key_with_pos= ition > is not correct while it enqueues position into free_slots directly. It mu= st > increase position by one to get the right key index, before enqueues into > free_slots. >=20 > As comparision, remove_entry()enqueue key_idx directly, which is correct:= =20 >=20 > static inline void > remove_entry(const struct rte_hash *h, struct rte_hash_bucket *bkt, unsig= ned i) > { > unsigned lcore_id, n_slots; > struct lcore_cache *cached_free_slots; >=20 > if (h->use_local_cache) { > lcore_id =3D rte_lcore_id(); > cached_free_slots =3D &h->local_free_slots[lcore_id]; > /* Cache full, need to free it. */ > if (cached_free_slots->len =3D=3D LCORE_CACHE_SIZE) { > /* Need to enqueue the free slots in global ring. = */ > n_slots =3D rte_ring_mp_enqueue_burst(h->free_slot= s, > cached_free_slots->objs, > LCORE_CACHE_SIZE, NULL); > cached_free_slots->len -=3D n_slots; > } > /* Put index of new free slot in cache. */ > cached_free_slots->objs[cached_free_slots->len] =3D > (void *)((uintptr_t)bkt->key_idx[i]); > cached_free_slots->len++; > } else { > rte_ring_sp_enqueue(h->free_slots, > (void *)((uintptr_t)bkt->key_idx[i])); > } > } >=20 > Second, computation of total_entries is not correct. This should be the t= otal > number of key slots.The number of key slots is seen as rte_hash_create, s= ay > (params->entries + (RTE_MAX_LCORE - 1) *(LCORE_CACHE_SIZE - 1) + 1) when > use_local_cache is true, else (params->entries + 1) >=20 > struct rte_hash * > rte_hash_create(const struct rte_hash_parameters *params) > { > ... > if (params->extra_flag & RTE_HASH_EXTRA_FLAGS_MULTI_WRITER_ADD) { > use_local_cache =3D 1; > writer_takes_lock =3D 1; > } > ... > /* Store all keys and leave the first entry as a dummy entry for > lookup_bulk */ > if (use_local_cache) > /* > * Increase number of slots by total number of indices > * that can be stored in the lcore caches > * except for the first cache > */ > num_key_slots =3D params->entries + (RTE_MAX_LCORE - 1) * > (LCORE_CACHE_SIZE - 1) + 1; > else > num_key_slots =3D params->entries + 1; > ... > /* Populate free slots ring. Entry zero is reserved for key misses. */ > for (i =3D 1; i < num_key_slots; i++) > rte_ring_sp_enqueue(r, (void *)((uintptr_t) i)); > ... > } >=20 > --=20 > You are receiving this mail because: > You are the assignee for the bug.