From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D5D0142A12; Thu, 27 Apr 2023 16:52:49 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7696A42D2D; Thu, 27 Apr 2023 16:52:49 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 3D31342B8C for ; Thu, 27 Apr 2023 16:52:47 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 33R8EN9u014668; Thu, 27 Apr 2023 07:52:46 -0700 Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2046.outbound.protection.outlook.com [104.47.66.46]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3q7nc39gkq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 27 Apr 2023 07:52:45 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TtmijNg8CUePApuSYfqCA+4RQtIrwNAppVhQ/RuhtcHovfFeOuTwwAqDP7aqyHQk9iG01AlGswCv6P9KQaWWF4QDkJ+NCRtyQxoR46Du4mqgkakhyOsWJsg4+ej7Vcu0sM27uYSQE0ee4N20uQ03wqiv/0o+TedY4WfYIFAMkYAvgOHY7D6cz/+V0CHpicRv+n9BO/Vlaf2n8cDe8IvvXpOjYUddlBZ32K6kmKaW6FLixczXKWZwTplYw/tkbrcqQ0ggj0Q5CNVf+GUNwlP4l7ZbksO2V/dOYvfc6c+ZyZDaw7MN4GX3Cc/xSXUd8i45N2EwqmFgHwNt6XdlCwzKkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QkB9geDGLa0tuCowZO5A5S5E53JpeJPWSJ4fTrhuxGU=; b=DuXLxKJ7ZfU/i/jTY9w7y1BMuUSseM9ovra3IhsEa6Rmm2AngOYWc8HKOjR1//z3A98HSPVe+kjI2RUIoenKFnLjA/35eNt6dUv7hrj5nfzLica6QiXvnZ75prnmcALY6PnxtW5w2JjNXxxn7Z/seVGR1jg6FOcIazK/IQOLLXvPAPaKIH30Cg527Iu4CAkKPkvk/tqUO+ESeh4e4gi5B8T3GmKIdfDuaReC4xjebfitXJ4qWxJpKdQHpsGGzlOKJqwmDl8XCl+/JbTLZuGusm1uho7nzs41dZQ8CKQv4HkFoKC79Cz64G5M7F4lAE8gl1oJNjKXdQCLeuNPkL6mug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QkB9geDGLa0tuCowZO5A5S5E53JpeJPWSJ4fTrhuxGU=; b=MAc0LK+xr/3GblGCKx3NpweXSwJUCrNwoxgv5uPphK429PW5ppEqjGfRBPA2Qrj0UwjV3qtj0yip9ZYi+ZeSFDVJtvX7t5nP460vD5+FLaDMZgqqC8FH4bp8upX8nUQac/MZ/7yZxb6gUYMV7c6FzBvDcTS0jvwF9sLGHsCUzyU= Received: from PH0PR18MB4086.namprd18.prod.outlook.com (2603:10b6:510:3::9) by CY4PR18MB0999.namprd18.prod.outlook.com (2603:10b6:903:a6::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6340.22; Thu, 27 Apr 2023 14:52:42 +0000 Received: from PH0PR18MB4086.namprd18.prod.outlook.com ([fe80::2e10:cd72:2bc4:b4bb]) by PH0PR18MB4086.namprd18.prod.outlook.com ([fe80::2e10:cd72:2bc4:b4bb%6]) with mapi id 15.20.6340.021; Thu, 27 Apr 2023 14:52:42 +0000 From: Pavan Nikhilesh Bhagavatula To: Zhirun Yan , "dev@dpdk.org" , Jerin Jacob Kollanukkaran , Kiran Kumar Kokkilagadda , Nithin Kumar Dabilpuram , "stephen@networkplumber.org" CC: "cunming.liang@intel.com" , "haiyue.wang@intel.com" Subject: RE: [EXT] [PATCH v5 09/15] graph: introduce stream moving cross cores Thread-Topic: [EXT] [PATCH v5 09/15] graph: introduce stream moving cross cores Thread-Index: AQHZY4XmSDIdXgDB90SF6TEbdiQvvq8/ULkg Date: Thu, 27 Apr 2023 14:52:42 +0000 Message-ID: References: <20230330061834.3118201-1-zhirun.yan@intel.com> <20230331040306.3143693-1-zhirun.yan@intel.com> <20230331040306.3143693-10-zhirun.yan@intel.com> In-Reply-To: <20230331040306.3143693-10-zhirun.yan@intel.com> Accept-Language: en-IN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-dg-ref: =?us-ascii?Q?PG1ldGE+PGF0IG5tPSJib2R5LnR4dCIgcD0iYzpcdXNlcnNccGJoYWdhdmF0?= =?us-ascii?Q?dWxhXGFwcGRhdGFccm9hbWluZ1wwOWQ4NDliNi0zMmQzLTRhNDAtODVlZS02?= =?us-ascii?Q?Yjg0YmEyOWUzNWJcbXNnc1xtc2ctMjcyMDI4NzUtZTUwYi0xMWVkLWI2Y2It?= =?us-ascii?Q?NGMwMzRmNWZhYTM2XGFtZS10ZXN0XDI3MjAyODc2LWU1MGItMTFlZC1iNmNi?= =?us-ascii?Q?LTRjMDM0ZjVmYWEzNmJvZHkudHh0IiBzej0iOTMwOSIgdD0iMTMzMjcwODA3?= =?us-ascii?Q?NTkzNTgzNjg4IiBoPSJ6aHkvRUZ3NXdwSjNtREdIcWs0K2w1Y1cyUnc9IiBp?= =?us-ascii?Q?ZD0iIiBibD0iMCIgYm89IjEiIGNpPSJjQUFBQUVSSFUxUlNSVUZOQ2dVQUFC?= =?us-ascii?Q?UUpBQUJJaEl2cEYzblpBYlhKZHFiNU5kZ3d0Y2wycHZrMTJEQU9BQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBSEFBQUFDa0NBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBRUFBUUFCQUFBQWFpNXhJQUFBQUFBQUFBQUFBQUFBQUo0QUFBQmhB?= =?us-ascii?Q?R1FBWkFCeUFHVUFjd0J6QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFFQUFBQUFBQUFBQWdBQUFBQUFuZ0FBQUdNQWRRQnpBSFFBYndCdEFG?= =?us-ascii?Q?OEFjQUJsQUhJQWN3QnZBRzRBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBUUFBQUFBQUFB?= =?us-ascii?Q?QUNBQUFBQUFDZUFBQUFZd0IxQUhNQWRBQnZBRzBBWHdCd0FHZ0Fid0J1QUdV?= =?us-ascii?Q?QWJnQjFBRzBBWWdCbEFISUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQkFBQUFBQUFBQUFJQUFBQUFBSjRBQUFC?= =?us-ascii?Q?akFIVUFjd0IwQUc4QWJRQmZBSE1BY3dCdUFGOEFaQUJoQUhNQWFBQmZBSFlB?= =?us-ascii?Q?TUFBeUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= x-dg-refone: =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUVBQUFBQUFBQUFBZ0FBQUFBQW5n?= =?us-ascii?Q?QUFBR01BZFFCekFIUUFid0J0QUY4QWN3QnpBRzRBWHdCckFHVUFlUUIzQUc4?= =?us-ascii?Q?QWNnQmtBSE1BQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFRQUFBQUFBQUFBQ0FBQUFBQUNlQUFBQVl3QjFBSE1BZEFC?= =?us-ascii?Q?dkFHMEFYd0J6QUhNQWJnQmZBRzRBYndCa0FHVUFiQUJwQUcwQWFRQjBBR1VB?= =?us-ascii?Q?Y2dCZkFIWUFNQUF5QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFCQUFB?= =?us-ascii?Q?QUFBQUFBQUlBQUFBQUFKNEFBQUJqQUhVQWN3QjBBRzhBYlFCZkFITUFjd0J1?= =?us-ascii?Q?QUY4QWN3QndBR0VBWXdCbEFGOEFkZ0F3QURJQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBRUFBQUFBQUFBQUFnQUFBQUFB?= =?us-ascii?Q?bmdBQUFHUUFiQUJ3QUY4QWN3QnJBSGtBY0FCbEFGOEFZd0JvQUdFQWRBQmZB?= =?us-ascii?Q?RzBBWlFCekFITUFZUUJuQUdVQVh3QjJBREFBTWdBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQVFBQUFBQUFBQUFDQUFBQUFBQ2VBQUFBWkFCc0FIQUFY?= =?us-ascii?Q?d0J6QUd3QVlRQmpBR3NBWHdCakFHZ0FZUUIwQUY4QWJRQmxBSE1BY3dCaEFH?= =?us-ascii?Q?Y0FaUUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= x-dg-reftwo: =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUJBQUFBQUFBQUFBSUFBQUFBQUo0QUFBQmtBR3dB?= =?us-ascii?Q?Y0FCZkFIUUFaUUJoQUcwQWN3QmZBRzhBYmdCbEFHUUFjZ0JwQUhZQVpRQmZB?= =?us-ascii?Q?R1lBYVFCc0FHVUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFFQUFBQUFBQUFBQWdBQUFBQUFuZ0FBQUdVQWJRQmhBR2tBYkFCZkFHRUFa?= =?us-ascii?Q?QUJrQUhJQVpRQnpBSE1BQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUF3QUFBQUFBQUFBQUFBQUFBUUFBQUFBQUFBQUNB?= =?us-ascii?Q?QUFBQUFDZUFBQUFiUUJoQUhJQWRnQmxBR3dBYkFCZkFIQUFjZ0J2QUdvQVpR?= =?us-ascii?Q?QmpBSFFBWHdCakFHOEFaQUJsQUhNQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQkFBQUFBQUFBQUFJQUFBQUFBSjRBQUFCdEFH?= =?us-ascii?Q?RUFjZ0IyQUdVQWJBQnNBRjhBY0FCeUFHOEFhZ0JsQUdNQWRBQmZBR01BYndC?= =?us-ascii?Q?a0FHVUFjd0JmQUdRQWFRQmpBSFFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUVBQUFBQUFBQUFBZ0FBQUFBQW5nQUFBRzBBWVFCeUFIWUFaUUJzQUd3?= =?us-ascii?Q?QVh3QjBBR1VBY2dCdEFHa0FiZ0IxQUhNQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB?= =?us-ascii?Q?QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFRQUFBQUFBQUFB?= =?us-ascii?Q?Q0FBQUFBQUE9Ii8+PC9tZXRhPg=3D=3D?= x-dg-rorf: true x-ms-publictraffictype: Email x-ms-traffictypediagnostic: PH0PR18MB4086:EE_|CY4PR18MB0999:EE_ x-ms-office365-filtering-correlation-id: 3f7fa061-3bf2-4d68-9fca-08db472f0df4 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: rgPancszBPXiG32JLXfgQa9g5Q9E6f6L3prBiCgks4t1JBKACzGb0ZNoNJe/TofV7uvmSkAlUwzxdu7jnAUfm11duG7zc5QJlaB1zVFHXnsKU6KhppKeNRnpciEX8yT6UeoEQz8xGx43r8CVRKApCGm2XdtnnoF5J0YQfoez/uR62T3oGSc327RaCeAU5vdR9YV6ETF8We7UUPMxkcn1DWyAMlcqagMRnuuCKMNaJst2g/IOd3Mu98zuBjguLE3Clf/b932GoYijsd6weycWbidZvnELDhMGBSveM7cPjgKiaYmNQiJjxYs8e6Nnwg5NOg888eTpmKt3MGRbhJa6CPuXb6ClWWCipc/NYZqtZ1yApjIx0rU/8Agcx9hs/zqpkSbW6IUDX77y1WS6LZOP8B7xaAg+EEz3nKcST7Khy9QaZNBp4DRHs7tdUVqwBVXItVNy9bw2tvDa0Fdc2DuNqP//ogHKrs+71xjWtuqbzqo17MQ0qcHl6WL1kJ41vgS5B565ijhR9Gv0fWs2yWZxLuGifKF1ucqqUiIQSPxJbup9ZfKim6nR7jn0sHgL1n7Y84V/8NSnHqIxPeX7yO1pXTTJjEtCAfgTTzP7UkMh9zBmXFyH3Q39m2UHk1VMX5QM x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR18MB4086.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(136003)(39860400002)(366004)(346002)(376002)(396003)(451199021)(7696005)(478600001)(83380400001)(186003)(6506007)(26005)(9686003)(38100700002)(38070700005)(86362001)(33656002)(122000001)(71200400001)(64756008)(66446008)(66476007)(66556008)(66946007)(76116006)(55016003)(4326008)(316002)(2906002)(8676002)(8936002)(52536014)(41300700001)(5660300002)(54906003)(110136005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?eckPR3knGy4qGLmzbphSrasvfEv/YBPYLhS4cbuQmvpsXYaH2JEXlspQWQtI?= =?us-ascii?Q?X+gFAt9JoT6aqCFWjVLJMmd0heEqVJNeiKgrVd1KNAVK2U4z0LfGAaSSiuFo?= =?us-ascii?Q?TKRe1ULIxEpnovM3YyoYukhYLBhBRCOY0OOhf4/fDyb+1zxaLwHRw3PbAZ5z?= =?us-ascii?Q?dCgLEPOx7bVyqFPzM0oTr8AqeAZhASm5ZngbDJWyrCkm2ld8RQwmezVbqK0Y?= =?us-ascii?Q?gv4Dx+FT9Tpmk5IoBvAn7DL+GhQIyce75r87T7uEEIdCG12ZuLeIj8lkCvsH?= =?us-ascii?Q?Lu1eAflV8c0ObH7T8YOmOVB7ZUdsj8Ho4UDIU/3XisNHXTWR/um9C4aHOJBX?= =?us-ascii?Q?BaShZ7NBtJyiINWp/Z77KPacjLMsa1PRXL19rIeyGkegeTrkgUcHVwyw360V?= =?us-ascii?Q?jseJBkgl5YnqYgAq2aFlcxdAUaeVwZ1TDXWMnWEegdMil/kTXFnT/hxHcX04?= =?us-ascii?Q?tZH6e5mrPefyguWeSiZCghbosJhe3jefU9oCKkGbFw9RjwGZE5SbT8o8hTlh?= =?us-ascii?Q?1JbxRNgdaRcxoNrkO51/j16d3fUcaSVutgeqG9PRgLBUSDvoF1EhWNpr3eRg?= =?us-ascii?Q?HPVrC/Gf0CX6eYeYqI3gvvqta00zeks9dnozDc+UDHg4RRfDykX++XkL4CNG?= =?us-ascii?Q?iKEjzcaaMDAeI9kmS4p3uSumZOYnFgA1m89gk7bF8OIjaefDp4VxMP25iFdQ?= =?us-ascii?Q?4rKCNON+Ws9owZRQh5zPa9qc2Kk/b/KI2ghU6EV0DwWZn+NwCUjbsMEq5S1d?= =?us-ascii?Q?BpUGZbuQDDaZ63F7uSzigMSMp+cJdDJu19p3kHoSW0Or2QOmYIZiyuPrYJik?= =?us-ascii?Q?G20KATFAA9b/dYPdErW8Dn6gQK2PLh4ihbz4NZZh8ClVPLqBV9la0Fc4ZoB0?= =?us-ascii?Q?wsV31Hx5mbHUihIiyLy8G6PbIkm9WDQATY29gDFCRoTxJBu4iHoUcsnsZ/Ki?= =?us-ascii?Q?rYe9s5ElawMGhQ+v1k6vOdG3Bdye0xt+Ku5+rO45t/AQ0Pcmw7HhD4wOhTIv?= =?us-ascii?Q?+ksp8umADK3a4YBWvvu/djVtC9sEZLd7pctw5F51er2II1exI6KGrTVXZZxJ?= =?us-ascii?Q?1Kszsuuzz0ERwREP4SGAY7ejl3oxP0BwPJqOc+cJoFsN3ZMG3QAwVL/1Wntf?= =?us-ascii?Q?r6T+VQX/3oMLNKQ2OrP/wjOY080rLzh8hoRZ9u1ZAa7+0g5UchTiwZiG4zcs?= =?us-ascii?Q?mXzXTJmouUtRSXt2ZcstG46EN5EkbWjIdW7N7rZWRw8O7wuzgqKDeZo8yOxd?= =?us-ascii?Q?YbBcR73wteIJ8t3vF9Ql0GrskpqOo5l7qfGVfzrr9GkSx1+p3wepRP0skO1W?= =?us-ascii?Q?9/uKn64YL7B5nx8Z1XKsInsTBiUqdmnoXs40mkzAZbEGlvmplLKIcSRYpNM8?= =?us-ascii?Q?ubo1eZCvn1k2XJLcHlwcN25kNcNuELf/FDo6gQeEaBGF3vBH/nTK3vdFpZZO?= =?us-ascii?Q?NYmBd/k4vjr2ASZbHseHAB8syT3mza7NdV6RMOjf5sPU9OtuSy+JkPPFnl/E?= =?us-ascii?Q?MGFC/+Z5ZFBg7zn9/Q0Aw6pRgRwuImtz6YboElz8kBtIXc1eULlndx5Lm3WA?= =?us-ascii?Q?pc1ADiC9VRzy/BDA757n5yTTbYQTUFMn4YhUGTrX?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: marvell.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PH0PR18MB4086.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3f7fa061-3bf2-4d68-9fca-08db472f0df4 X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Apr 2023 14:52:42.5941 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: HiA90CohIV2jqrMuBovPozwsa81PzbzVdtytlkqqPTFnkRf905VxpWU8t+yUxtwquKNaGhEyienh3NfvXcv6G6G9h6v586Z3jBXc4z7s9Bc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR18MB0999 X-Proofpoint-GUID: AQXBgK6uR4u7XxugYomfSRbIeRx_uCEj X-Proofpoint-ORIG-GUID: AQXBgK6uR4u7XxugYomfSRbIeRx_uCEj X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-27_07,2023-04-27_01,2023-02-09_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > This patch introduces key functions to allow a worker thread to > enable enqueue and move streams of objects to the next nodes over > different cores. >=20 > Signed-off-by: Haiyue Wang > Signed-off-by: Cunming Liang > Signed-off-by: Zhirun Yan > --- > lib/graph/graph_private.h | 27 +++++ > lib/graph/meson.build | 2 +- > lib/graph/rte_graph_model_dispatch.c | 145 > +++++++++++++++++++++++++++ > lib/graph/rte_graph_model_dispatch.h | 37 +++++++ > lib/graph/version.map | 2 + > 5 files changed, 212 insertions(+), 1 deletion(-) >=20 > diff --git a/lib/graph/graph_private.h b/lib/graph/graph_private.h > index b66b18ebbc..e1a2a4bfd8 100644 > --- a/lib/graph/graph_private.h > +++ b/lib/graph/graph_private.h > @@ -366,4 +366,31 @@ void graph_dump(FILE *f, struct graph *g); > */ > void node_dump(FILE *f, struct node *n); >=20 > +/** > + * @internal > + * > + * Create the graph schedule work queue. And all cloned graphs attached = to > the > + * parent graph MUST be destroyed together for fast schedule design > limitation. > + * > + * @param _graph > + * The graph object > + * @param _parent_graph > + * The parent graph object which holds the run-queue head. > + * > + * @return > + * - 0: Success. > + * - <0: Graph schedule work queue related error. > + */ > +int graph_sched_wq_create(struct graph *_graph, struct graph > *_parent_graph); > + > +/** > + * @internal > + * > + * Destroy the graph schedule work queue. > + * > + * @param _graph > + * The graph object > + */ > +void graph_sched_wq_destroy(struct graph *_graph); > + > #endif /* _RTE_GRAPH_PRIVATE_H_ */ > diff --git a/lib/graph/meson.build b/lib/graph/meson.build > index c729d984b6..e21affa280 100644 > --- a/lib/graph/meson.build > +++ b/lib/graph/meson.build > @@ -20,4 +20,4 @@ sources =3D files( > ) > headers =3D files('rte_graph.h', 'rte_graph_worker.h') >=20 > -deps +=3D ['eal', 'pcapng'] > +deps +=3D ['eal', 'pcapng', 'mempool', 'ring'] > diff --git a/lib/graph/rte_graph_model_dispatch.c > b/lib/graph/rte_graph_model_dispatch.c > index 4a2f99496d..a300fefb85 100644 > --- a/lib/graph/rte_graph_model_dispatch.c > +++ b/lib/graph/rte_graph_model_dispatch.c > @@ -5,6 +5,151 @@ > #include "graph_private.h" > #include "rte_graph_model_dispatch.h" >=20 > +int > +graph_sched_wq_create(struct graph *_graph, struct graph > *_parent_graph) > +{ > + struct rte_graph *parent_graph =3D _parent_graph->graph; > + struct rte_graph *graph =3D _graph->graph; > + unsigned int wq_size; > + > + wq_size =3D GRAPH_SCHED_WQ_SIZE(graph->nb_nodes); > + wq_size =3D rte_align32pow2(wq_size + 1); Hi Zhirun, We should introduce a new function `rte_graph_configure` which can help=20 application to control the ring size and mempool size of the work queue? We could fallback to default values if nothing is configured. rte_graph_configure should take a=20 struct rte_graph_config { struct { u64 rsvd[8]; } rtc; struct { u16 wq_size; ... } dispatch; }; This will help future graph models to have their own configuration. We can have a rte_graph_config_init() function to initialize the rte_graph_= config structure. > + > + graph->wq =3D rte_ring_create(graph->name, wq_size, graph->socket, > + RING_F_SC_DEQ); > + if (graph->wq =3D=3D NULL) > + SET_ERR_JMP(EIO, fail, "Failed to allocate graph WQ"); > + > + graph->mp =3D rte_mempool_create(graph->name, wq_size, > + sizeof(struct graph_sched_wq_node), > + 0, 0, NULL, NULL, NULL, NULL, > + graph->socket, MEMPOOL_F_SP_PUT); > + if (graph->mp =3D=3D NULL) > + SET_ERR_JMP(EIO, fail_mp, > + "Failed to allocate graph WQ schedule entry"); > + > + graph->lcore_id =3D _graph->lcore_id; > + > + if (parent_graph->rq =3D=3D NULL) { > + parent_graph->rq =3D &parent_graph->rq_head; > + SLIST_INIT(parent_graph->rq); > + } > + > + graph->rq =3D parent_graph->rq; > + SLIST_INSERT_HEAD(graph->rq, graph, rq_next); > + > + return 0; > + > +fail_mp: > + rte_ring_free(graph->wq); > + graph->wq =3D NULL; > +fail: > + return -rte_errno; > +} > + > +void > +graph_sched_wq_destroy(struct graph *_graph) > +{ > + struct rte_graph *graph =3D _graph->graph; > + > + if (graph =3D=3D NULL) > + return; > + > + rte_ring_free(graph->wq); > + graph->wq =3D NULL; > + > + rte_mempool_free(graph->mp); > + graph->mp =3D NULL; > +} > + > +static __rte_always_inline bool > +__graph_sched_node_enqueue(struct rte_node *node, struct rte_graph > *graph) > +{ > + struct graph_sched_wq_node *wq_node; > + uint16_t off =3D 0; > + uint16_t size; > + > +submit_again: > + if (rte_mempool_get(graph->mp, (void **)&wq_node) < 0) > + goto fallback; > + > + size =3D RTE_MIN(node->idx, RTE_DIM(wq_node->objs)); > + wq_node->node_off =3D node->off; > + wq_node->nb_objs =3D size; > + rte_memcpy(wq_node->objs, &node->objs[off], size * sizeof(void > *)); > + > + while (rte_ring_mp_enqueue_bulk_elem(graph->wq, (void > *)&wq_node, > + sizeof(wq_node), 1, NULL) =3D=3D 0) > + rte_pause(); > + > + off +=3D size; > + node->idx -=3D size; > + if (node->idx > 0) > + goto submit_again; > + > + return true; > + > +fallback: > + if (off !=3D 0) > + memmove(&node->objs[0], &node->objs[off], > + node->idx * sizeof(void *)); > + > + return false; > +} > + > +bool __rte_noinline > +__rte_graph_sched_node_enqueue(struct rte_node *node, > + struct rte_graph_rq_head *rq) > +{ > + const unsigned int lcore_id =3D node->lcore_id; > + struct rte_graph *graph; > + > + SLIST_FOREACH(graph, rq, rq_next) > + if (graph->lcore_id =3D=3D lcore_id) > + break; > + > + return graph !=3D NULL ? __graph_sched_node_enqueue(node, > graph) : false; > +} > + > +void > +__rte_graph_sched_wq_process(struct rte_graph *graph) > +{ > + struct graph_sched_wq_node *wq_node; > + struct rte_mempool *mp =3D graph->mp; > + struct rte_ring *wq =3D graph->wq; > + uint16_t idx, free_space; > + struct rte_node *node; > + unsigned int i, n; > + struct graph_sched_wq_node *wq_nodes[32]; > + > + n =3D rte_ring_sc_dequeue_burst_elem(wq, wq_nodes, > sizeof(wq_nodes[0]), > + RTE_DIM(wq_nodes), NULL); > + if (n =3D=3D 0) > + return; > + > + for (i =3D 0; i < n; i++) { > + wq_node =3D wq_nodes[i]; > + node =3D RTE_PTR_ADD(graph, wq_node->node_off); > + RTE_ASSERT(node->fence =3D=3D RTE_GRAPH_FENCE); > + idx =3D node->idx; > + free_space =3D node->size - idx; > + > + if (unlikely(free_space < wq_node->nb_objs)) > + __rte_node_stream_alloc_size(graph, node, node- > >size + wq_node->nb_objs); > + > + memmove(&node->objs[idx], wq_node->objs, wq_node- > >nb_objs * sizeof(void *)); > + memset(wq_node->objs, 0, wq_node->nb_objs * > sizeof(void *)); Memset should be avoided in fastpath for better performance as we anyway se= t wq_node->nb_objs as 0. > + node->idx =3D idx + wq_node->nb_objs; > + > + __rte_node_process(graph, node); > + > + wq_node->nb_objs =3D 0; > + node->idx =3D 0; > + } > + > + rte_mempool_put_bulk(mp, (void **)wq_nodes, n); > +} > + > int > rte_graph_model_dispatch_lcore_affinity_set(const char *name, unsigned > int lcore_id) > { > diff --git a/lib/graph/rte_graph_model_dispatch.h > b/lib/graph/rte_graph_model_dispatch.h > index 179624e972..18fa7ce0ab 100644 > --- a/lib/graph/rte_graph_model_dispatch.h > +++ b/lib/graph/rte_graph_model_dispatch.h > @@ -14,12 +14,49 @@ > * > * This API allows to set core affinity with the node. > */ > +#include > +#include > +#include > +#include > + > #include "rte_graph_worker_common.h" >=20 > #ifdef __cplusplus > extern "C" { > #endif >=20 > +#define GRAPH_SCHED_WQ_SIZE_MULTIPLIER 8 > +#define GRAPH_SCHED_WQ_SIZE(nb_nodes) \ > + ((typeof(nb_nodes))((nb_nodes) * > GRAPH_SCHED_WQ_SIZE_MULTIPLIER)) > + > +/** > + * @internal > + * > + * Schedule the node to the right graph's work queue. > + * > + * @param node > + * Pointer to the scheduled node object. > + * @param rq > + * Pointer to the scheduled run-queue for all graphs. > + * > + * @return > + * True on success, false otherwise. > + */ > +__rte_experimental > +bool __rte_noinline __rte_graph_sched_node_enqueue(struct rte_node > *node, > + struct rte_graph_rq_head *rq); > + > +/** > + * @internal > + * > + * Process all nodes (streams) in the graph's work queue. > + * > + * @param graph > + * Pointer to the graph object. > + */ > +__rte_experimental > +void __rte_graph_sched_wq_process(struct rte_graph *graph); > + > /** > * Set lcore affinity with the node. > * > diff --git a/lib/graph/version.map b/lib/graph/version.map > index aaa86f66ed..d511133f39 100644 > --- a/lib/graph/version.map > +++ b/lib/graph/version.map > @@ -48,6 +48,8 @@ EXPERIMENTAL { >=20 > rte_graph_worker_model_set; > rte_graph_worker_model_get; > + __rte_graph_sched_wq_process; > + __rte_graph_sched_node_enqueue; >=20 > rte_graph_model_dispatch_lcore_affinity_set; >=20 > -- > 2.37.2