From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on0090.outbound.protection.outlook.com [104.47.2.90]) by dpdk.org (Postfix) with ESMTP id C78FE11F5 for ; Tue, 29 May 2018 14:05:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia.onmicrosoft.com; s=selector1-nokia-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kcmJKb2JjW5RoURN0vEB8dTqYcUqVJjpwcQDflR/kF4=; b=i55ZDFR6mR0eOFlTmbr7EWnpp6+Gt1oQVaixZMjGQVJLU/Z/Cy3dVinSmAGoWYEndWrp8RmJT2rTAunLeHEnSKjI510ZM2M//IuB9LtkQyLOt7GoOKbsFEDbStAvQfX6GAoKyrEndMdjRAS+MLK4K2TsSnIu7Lzkhnj6e1OTjx4= Received: from HE1PR07MB0844.eurprd07.prod.outlook.com (10.162.24.17) by HE1PR07MB3434.eurprd07.prod.outlook.com (10.170.247.141) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.820.11; Tue, 29 May 2018 12:05:43 +0000 Received: from HE1PR07MB0844.eurprd07.prod.outlook.com ([fe80::3916:d0b5:8158:1cf9]) by HE1PR07MB0844.eurprd07.prod.outlook.com ([fe80::3916:d0b5:8158:1cf9%5]) with mapi id 15.20.0820.010; Tue, 29 May 2018 12:05:42 +0000 From: "Elo, Matias (Nokia - FI/Espoo)" To: "users@dpdk.org" Thread-Topic: Calling rte_event_port_unlink() causes subsequent events to end up in wrong port Thread-Index: AQHT90Vd5uWv74ch90qcCvC1WVIGtg== Date: Tue, 29 May 2018 12:05:42 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: Apple Mail (2.3445.6.18) authentication-results: spf=none (sender IP is ) smtp.mailfrom=matias.elo@nokia.com; x-originating-ip: [131.228.32.181] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1PR07MB3434; 7:gNqxCIdQzRoXVXQ2DxWUrg8tBEWJBvQLM3ImAN3K06oNL4ld1vdhgCUvtCyGOTT2/95C/BUZKULYRsc9Hr0Vj8cMU8m++cE2xiKzKvHiAnVwpHwoFom0f4/OYkwzMGGJRTnRoNKH+aX4duJRNPUy7/a8OXb01WBSLrjfFakAEbcpN0p6C8CYMhROiRFQLpSVre+IiwRNR5vfYoJ+KcZGs2tyHMrbiRjJRSpTLOmWeIt87IiqgIzG2vFEkHiwEit5 x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(8989080)(5600026)(48565401081)(4534165)(4627221)(201703031133081)(201702281549075)(8990040)(2017052603328)(7193020); SRVR:HE1PR07MB3434; x-ms-traffictypediagnostic: HE1PR07MB3434: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(8121501046)(5005006)(3231254)(11241501184)(806099)(944501410)(52105095)(93006095)(93001095)(3002001)(10201501046)(6055026)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123558120)(20161123562045)(20161123564045)(6072148)(201708071742011)(7699016); SRVR:HE1PR07MB3434; BCL:0; PCL:0; RULEID:; SRVR:HE1PR07MB3434; x-forefront-prvs: 0687389FB0 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(39860400002)(366004)(396003)(39380400002)(376002)(346002)(199004)(189003)(5640700003)(14454004)(305945005)(57306001)(2501003)(83716003)(6486002)(66066001)(106356001)(486006)(105586002)(478600001)(2351001)(97736004)(2616005)(476003)(50226002)(6916009)(26005)(99286004)(82746002)(7736002)(3660700001)(5250100002)(8936002)(6436002)(186003)(102836004)(316002)(3280700002)(1730700003)(81166006)(5660300001)(8676002)(3846002)(6116002)(68736007)(81156014)(114624004)(53936002)(6512007)(25786009)(2906002)(33656002)(36756003)(2900100001)(6506007)(86362001); DIR:OUT; SFP:1102; SCL:1; SRVR:HE1PR07MB3434; H:HE1PR07MB0844.eurprd07.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: nokia.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: yET+ja+MAw8fcbQDtMHKayIDJCp0HOevmJgGcOSXRRbTXYkQEIAMIuZZ/qVPfShSxXkWsSgK+yPX/sxS7Usz6LdF2fABtYOccX7XZuwQGTFOyZ03+N0JJMgDIF8S/GoZ3yee743+r1kFR35yf2a9h1K7+Ts421jWh30lw7yiKCDpX3eSwkg+vft2LL+kyMg1gAjiXUsLfqji/VP1e+T84Rsan3LnDDmrlcFqejaIHcU= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-ID: <3A0073D818A96942AE5E602A26DBA502@eurprd07.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 328ff470-7a87-4a2a-97ed-08d5c55c8079 X-OriginatorOrg: nokia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 328ff470-7a87-4a2a-97ed-08d5c55c8079 X-MS-Exchange-CrossTenant-originalarrivaltime: 29 May 2018 12:05:42.4557 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR07MB3434 X-Mailman-Approved-At: Wed, 30 May 2018 10:34:00 +0200 Subject: [dpdk-users] Calling rte_event_port_unlink() causes subsequent events to end up in wrong port X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 May 2018 12:05:48 -0000 Hi, I'm seeing some unexpected(?) behavior after calling rte_event_port_unlink(= ) with the SW eventdev driver (DPDK 17.11.2/18.02.1, RTE_EVENT_MAX_QUEUES_PER= _DEV=3D255). Scenario: - Run SW evendev on a service core - Start eventdev with e.g. 16 ports. Each core will have a dedicated port. - Create 1 atomic queue and link all active ports to it (some ports may not be linked). - Allocate N events and enqueue them to eventdev - Next, each worker core does a number of scheduling rounds concurrently. E.g. uint64_t rx_events =3D 0; while(rx_events < SCHED_ROUNDS) { num_deq =3D rte_event_dequeue_burst(dev_id, port_id, ev, 1, 0); if (num_deq) { rx_events++; rte_event_enqueue_burst(dev_id, port_id, ev, 1); } } - This works fine but problems occur when doing cleanup after the first loop finishes on some core. E.g. rte_event_port_unlink(dev_id, port_id, NULL, 0); while(1) { num_deq =3D rte_event_dequeue_burst(dev_id, port_id, ev, 1, 0); if (num_deq =3D=3D 0) break; rte_event_enqueue_burst(dev_id, port_id, ev, 1); } - The events enqueued in the cleanup loop will ramdomly end up either back = to the same port (which has already been unlinked) or to port 0, which is not used (mapping rte_lcore_id to port_id). As far as I understand the eventdev API, an eventdev port shouldn't have to= be linked to the target queue for enqueue to work. Have I understoop something incorrectly or is there a bug in the SW scheduler? I can provide a simple test application for reproducing this issue. Below i= s an example rte_event_dev_dump() output when processing events with two cores (ports 2 and 3). The rest of the ports are not linked at all but somehow an= event ends up to port 0 stalling the system. Regards, Matias EventDev todo-fix-name: ports 16, qids 1 rx 908342 drop 0 tx 908342 sched calls: 42577156 sched cq/qid call: 43120490 sched no IQ enq: 42122057 sched no CQ enq: 42122064 inflight 32, credits: 4064 Port 0=20 rx 0 drop 0 tx 2 inflight 2 Max New: 1024 Avg cycles PP: 0 Credits: 0 Receive burst distribution: 0:-nan%=20 rx ring used: 0 free: 4096 cq ring used: 2 free: 14 Port 1=20 rx 0 drop 0 tx 0 inflight 0 Max New: 1024 Avg cycles PP: 0 Credits: 0 Receive burst distribution: 0:-nan%=20 rx ring used: 0 free: 4096 cq ring used: 0 free: 16 Port 2=20 rx 524292 drop 0 tx 524290 inflight 0 Max New: 1024 Avg cycles PP: 190 Credits: 30 Receive burst distribution: 0:98% 1-4:1.82%=20 rx ring used: 0 free: 4096 cq ring used: 0 free: 16 Port 3=20 rx 384050 drop 0 tx 384050 inflight 0 Max New: 1024 Avg cycles PP: 191 Credits: 0 Receive burst distribution: 0:100% 1-4:0.04%=20 rx ring used: 0 free: 4096 cq ring used: 0 free: 16 ... Port 15=20 rx 0 drop 0 tx 0 inflight 0 Max New: 1024 Avg cycles PP: 0 Credits: 0 Receive burst distribution: 0:-nan%=20 rx ring used: 0 free: 4096 cq ring used: 0 free: 16 Queue 0 (Atomic) rx 908342 drop 0 tx 908342 Per Port Stats: Port 0: Pkts: 2 Flows: 1 Port 1: Pkts: 0 Flows: 0 Port 2: Pkts: 524290 Flows: 0 Port 3: Pkts: 384050 Flows: 0 Port 4: Pkts: 0 Flows: 0 Port 5: Pkts: 0 Flows: 0 Port 6: Pkts: 0 Flows: 0 Port 7: Pkts: 0 Flows: 0 Port 8: Pkts: 0 Flows: 0 Port 9: Pkts: 0 Flows: 0 Port 10: Pkts: 0 Flows: 0 Port 11: Pkts: 0 Flows: 0 Port 12: Pkts: 0 Flows: 0 Port 13: Pkts: 0 Flows: 0 Port 14: Pkts: 0 Flows: 0 Port 15: Pkts: 0 Flows: 0 -- iqs empty --