From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C3A7A424F2; Mon, 4 Sep 2023 18:37:48 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5608D402E9; Mon, 4 Sep 2023 18:37:48 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 83120402C1 for ; Mon, 4 Sep 2023 18:37:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1693845466; x=1725381466; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=ItwuyaGiXHJrOnIHQL7Okf1+Z1xlIl4NytwwGiZg0ZM=; b=gnqhzZIidFyYkFqFzlpTGRtQ667hi7CdA1Q1cYcyWKbXXdR6hhpvOp0v 8C2F/gIAkCq9gTl5kfKMbzYCDfA9KI/N9NXSYkD7fEh2sKqD+aW1KP86L 9hQm+3rHQi3QgH/gdWTN8WZJuOqvh3yQoabv6DlS59lvklxVyJvz7lipQ as852ZIquj8Gpu7GxIqaQe+r0ORF0OyQnyM5Z3E6PhUgd3e9BJHnOuMFM D5HeMbyriKDHBkM+39J0u/mONtJjBSNUrcdxcOEzi4kWhvdGdQcRGc3YS U4QXWert9Hw2jGNZe+bzEvcGnsJdNxJdYiJyNxUjJfIxhMODP7GI3borU A==; X-IronPort-AV: E=McAfee;i="6600,9927,10823"; a="379346598" X-IronPort-AV: E=Sophos;i="6.02,226,1688454000"; d="scan'208";a="379346598" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Sep 2023 09:37:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10823"; a="734366814" X-IronPort-AV: E=Sophos;i="6.02,226,1688454000"; d="scan'208";a="734366814" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by orsmga007.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 04 Sep 2023 09:37:45 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 4 Sep 2023 09:37:44 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 4 Sep 2023 09:37:44 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27 via Frontend Transport; Mon, 4 Sep 2023 09:37:44 -0700 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.101) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.27; Mon, 4 Sep 2023 09:37:43 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TshCkpiuqRx9U5wm5hVl6C6vwKst+P45soYVvk9JT0tsf0MJ/tNZ/jv9O9VsahmIh/fEUluLBi6AzELXdLLkwS2NAfIkb1bvQ+TuIB+x4bV4oBl7CyYoTZIPhF9FjGPk+0G8a4IikEY7csW6MAzI+BhFsZQE0clRuwDtZJmdCjKvjcEGYwwBl9WiaSHtbv9+HvfS+0DtLAiixtK1l0F2H3BV1oUk5ZAnrEpdYK7vzF+P0ptizv0I2l3LXxcocxRtp7se4oo+gFsBzfxBQO6oovag7AlJ5CGabnmoAACL3MSpll+wT1pqx6qNash5B/MNuWKhob6YZAru3VIdf+D5og== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BhwNjR3uRJDskVtgjSE0/DIYXp+WgSCnCchE5LWD/Lo=; b=gONJQ0Koxt26KBWC5wxGSvtvLgOGhHq77qnGwav65/s75B/cTystYEsUJwzciMgsmHk+w3RIyiWbmt0jJUlnsaQfNW9nvs2fXzwtQXK8ybqnbb2kfzjDe7wlPed8H7yubpfc7njfps1xlAM68Dkj5mQi/kLs8EOnfjMHa/S2C/qKjCBJJFqfdri3K+pxiKW1Epsg1+jiBMz99QzpelpBcGtez4SHP0IYS7h42cwl4xVpwvL/XUGCDdFVeuzaODXI/CVEb/zcL+lgjeKa4dJWT6McwnoXGiseUq4tbQU9wfVF3rFg839n0AcElmIDzWXOaw/BeWEqd9FdiWHHAqjLuA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by SA1PR11MB5923.namprd11.prod.outlook.com (2603:10b6:806:23a::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.27; Mon, 4 Sep 2023 16:37:41 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::43d1:af60:464:347]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::43d1:af60:464:347%5]) with mapi id 15.20.6745.030; Mon, 4 Sep 2023 16:37:41 +0000 Date: Mon, 4 Sep 2023 17:37:34 +0100 From: Bruce Richardson To: Harry van Haaren CC: Subject: Re: [PATCH 2/2] event/sw: fix ordering corruption with op release Message-ID: References: <20230831164736.2472671-1-harry.van.haaren@intel.com> <20230831164736.2472671-2-harry.van.haaren@intel.com> Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <20230831164736.2472671-2-harry.van.haaren@intel.com> X-ClientProxiedBy: DUZPR01CA0233.eurprd01.prod.exchangelabs.com (2603:10a6:10:4b4::17) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|SA1PR11MB5923:EE_ X-MS-Office365-Filtering-Correlation-Id: e0c5494f-e1f1-44d8-1ea2-08dbad65421a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 18xXhc3pjiZj5hz9OpGxb/bcJjeNvNwv9rCCRuMfCwoPMTKr7sU8cqvdb6GmR3EK6YIfeX80MP7n22owKxVD7b4mywkwN+dsjfFdolJfk9/00NlKbAxVSewRr+8mnhiee5LcTv77yFrUTpmruX3dMcoPOwkWi0KNw+RvQFLXfzWmp7vAD490OOk1qoWT/LzlZBZk5yApTJmOKXaEXoOpAxOnD0SQfzwoCs/OOiU1lf2rxSgmltcm/WtE27IgyQ+CyNAf5JUi+hjiJZjHhlij+nchm7YAsb1C/EmunKIdef4oTwD4VE7EEKONEV9TW3ocdQXyE7MvCbQGji6zSUxeAlsotgGswQ2Abi++ZlUxp40PMfoK8BsDqQUfR1IVscz0PfQUqcDgRFsYI5o85sIL4bNp1VLprGcqVu1ppmB4xOsYiALOAgwhq4KbQIe7t7snTAR6OFQfdEDYkUnyaYw2xQFtYlcpQlhWtMmJ+yPRQ6DGHmfO4lNManhpOe1NFiZFCvPC59NsmwvriHE0hnjBI16+/mZKImWaM5bCnQzzUlHYABNTd0pgqZx+5bBm/1Jw X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(376002)(346002)(396003)(136003)(366004)(39860400002)(186009)(1800799009)(451199024)(66899024)(26005)(44832011)(6666004)(82960400001)(316002)(6636002)(83380400001)(86362001)(66946007)(66556008)(38100700002)(6512007)(66476007)(478600001)(41300700001)(8676002)(6862004)(4326008)(8936002)(5660300002)(6506007)(6486002)(2906002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?nHiscpjFOJRfrPsGGuM04N0CJPi9RzK717J63+pmvvPS6i96MIxAR9pRqHbu?= =?us-ascii?Q?SXzBzyw/Ec7JdVMcScbN09qFY7DUYLXH0atvx3+OZu2FnXfXyxrwaHwxoioP?= =?us-ascii?Q?q8sAmXwBuWOd2aPgNe7UpQa4d8wAC/icb2ABBjSKd/kslgidLY2ZyP1hdCgW?= =?us-ascii?Q?414C2UNnvVs7kOtDR6K3QGJGdC+QUwEbDgIxt/Fa37IuIXGLSLZUEVcw34n/?= =?us-ascii?Q?ePCgW6LQH4GbqqgkVRVL3xyvcHCmCz4ctxCYbDxAUt7YmtexnyssSfckwQEV?= =?us-ascii?Q?q8A+CYF60y6ZnOugEjhwCtnBHyE9//LWrwYnQr8bfwXImlruoBWQ4nJw+xgs?= =?us-ascii?Q?XRoRrj+Wl4+VF4fP1ObnKjU1V5s+N1h0synBSljMNQf+4oqxKxidwxvHAVz8?= =?us-ascii?Q?G8L01qubnps2NA98D36ldfSN2zWOnpmFJ7gl4BSaLk8iOMw/+Wacih7Kq/75?= =?us-ascii?Q?y3fapJyTimIXXtGI7OIF4EJMwwMOSUGe8rpfrHLXCmLXljiIvCC6dR20qxF7?= =?us-ascii?Q?S+cclvZWutnbmi2WmB/yeKn4iL7R9FJaSgJf75C1suSDDmO285zu1YxA4ESR?= =?us-ascii?Q?e2GbfDCOUq2Qm079pVFvHG6hTV35VMgObDgjPWaoRzBTvooB+I0pCQG8NlJk?= =?us-ascii?Q?kVxoe5REafO8g8gKnSDgeuJ6v/QbqTIWeTeT5t7ndx807F21wVR/a2/KdtPE?= =?us-ascii?Q?Zzn4y09sr0mu392o9OYwPh4Luh1KS2b4yz/M6VDnX5IDICmMkoX6VanScMng?= =?us-ascii?Q?rKzrix8stNrbAzs1JPIYSU5313l90J2LSTl0ipIc5+1Q9LL615fzgNHy5KSZ?= =?us-ascii?Q?Z60Wbowt8Nh2homDjVTZ9Q52VU/IzVos0V1gk8HC0SuxFICam7rCpa098slO?= =?us-ascii?Q?58S7FTVDXFbdn0Q3pROF10w7WpMnN+HdFaxtUaMW9QzlSocKqSIruDcvd3N1?= =?us-ascii?Q?qRBC84H0Kj1bW/EPv3UFNQHl2EH1yN8kB54MUTmP2p6P30qxa/+TLk2hwhUK?= =?us-ascii?Q?dDqtPTt4qV9F95WpUVtnYRoFs7PZLi0LkKTZBi7O8UfNFi37iZT+OoNLb51X?= =?us-ascii?Q?k6s87H6+aAtxgQ/9EDN2Vf9agQuzCwRWi3fkKXLVQugusuIt7GWY2vVPrGEv?= =?us-ascii?Q?7XyykR32sxdOnfJLmDMRyLX3hVhLiHqBmIjQjcea1Cu533G3+s56rgYOm+EL?= =?us-ascii?Q?ITRQgaXuc4P31ns2025XBLJVzyE5yGx911kIjyD4mUfIXxbDhoad0YrZiJHg?= =?us-ascii?Q?Tw8HQeQvfYMKYQubCJMuMPLb/85/tCukQwv/L+Vy70f4ULu9YaFl/h5XHyFv?= =?us-ascii?Q?P7X2nPX0TvWmvwdLdwyih47lF3lVzmC/ydG9UiSfh/Kxj8R+h7AQ6/WQ4HVD?= =?us-ascii?Q?9vyfEqcKrH5QymjZsT9tq0iJNgxu3+oxnnN0ZGn2sUtX+xNKWPEwAdHr3yoP?= =?us-ascii?Q?W/qpmENkeucLvhSemzbbiXs9p2aQdJzSfmzEk6E4uyjLzj+D/E7cZXy25F+k?= =?us-ascii?Q?yunhraedRrRv6OBmUTd9Ae11ujnWaZKJIB9OcWoNhZ+I4C7R/+aKKDasMrHi?= =?us-ascii?Q?NMLzcptAmSkwe3DWeH2HahQeX4Wa9i9Zf5JsGDyTVuSenuRK09fkhDGjTOPY?= =?us-ascii?Q?pw=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: e0c5494f-e1f1-44d8-1ea2-08dbad65421a X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Sep 2023 16:37:41.8228 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: OYZjoa8c7HB5FnJe20VxH5+oFCHJA2yHfrfU1XZFxILwHM2xP6aaiatvU5WS+d0423XZUlBf6t8jk0t0sH3jYlmjQuI9RIW662+9pf+N6sg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB5923 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, Aug 31, 2023 at 05:47:36PM +0100, Harry van Haaren wrote: > This commit changes the logic in the scheduler to always > reset reorder-buffer entries in the QE_FLAG_COMPLETE path, > and not just the QE_FLAG_VALID path. > > A release event is a COMPLETE but not VALID (no new event). > As a result, release events previously left the history-list > in an inconsistent state, and future events with op type of > forward could be incorrectly reordered. > > Signed-off-by: Harry van Haaren > --- > drivers/event/sw/sw_evdev_scheduler.c | 45 ++++++++++++++++----------- > 1 file changed, 27 insertions(+), 18 deletions(-) > Hi Harry, wondering if this fix might work as well, and offer a simpler alternative. We can instead zero all unspecified hist-list entries on write. /Bruce --- a/drivers/event/sw/sw_evdev_scheduler.c +++ b/drivers/event/sw/sw_evdev_scheduler.c @@ -90,8 +90,7 @@ sw_schedule_atomic_to_cq(struct sw_evdev *sw, struct sw_qid * const qid, sw->cq_ring_space[cq]--; int head = (p->hist_head++ & (SW_PORT_HIST_LIST-1)); - p->hist_list[head].fid = flow_id; - p->hist_list[head].qid = qid_id; + p->hist_list[head] = (struct sw_hist_list_entry){ .qid = qid_id, .fid = flow_id }; p->stats.tx_pkts++; qid->stats.tx_pkts++; @@ -162,8 +161,8 @@ sw_schedule_parallel_to_cq(struct sw_evdev *sw, struct sw_qid * const qid, qid->stats.tx_pkts++; const int head = (p->hist_head & (SW_PORT_HIST_LIST-1)); - p->hist_list[head].fid = SW_HASH_FLOWID(qe->flow_id); - p->hist_list[head].qid = qid_id; + const uint32_t fid = SW_HASH_FLOWID(qe->flow_id); + p->hist_list[head] = (struct sw_hist_list_entry){ .qid = qid_id, .fid = fid }; if (keep_order) rob_ring_dequeue(qid->reorder_buffer_freelist,