From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5144043A72; Mon, 5 Feb 2024 14:28:26 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C0E8F402BE; Mon, 5 Feb 2024 14:28:25 +0100 (CET) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by mails.dpdk.org (Postfix) with ESMTP id EEA0F402A2 for ; Mon, 5 Feb 2024 14:28:23 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DB4AchkOa2UWq5Hkb4+t4B0w3xdno5lmyM7MmEcn/bBI3u4zhwCZBQv56xVa/GwjlfgW4P2FBRLcODyoHWzo+xgNjLqUR0E89qW/xDOvPqucfSue+w4G3JO+QE7LtzfY0LGdJ5VjK+wcu5fPD4nRUE1yx3P3pNzvA3PvWmoL9xNjAHA8kpFOXNCxTIIstlYuQFvtyX/t/Us6gfaaIDiWpBJdJtWElwfh1XcbAoOyVuFG5MMxjWSKbk+o4q+f1jxPWixg/dL98W2IiHj00yyPHt2eaAtJ06uiIEgJODjxsvjwfgIyDZ65NIaQa6Zh5ObRWnQOAam/uKlis3G+3wp0bQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cZ1J76fgDoTBoo9y43ADYDwEftGw1fWk1RH8hDqiM2A=; b=jl9M+bQ0H2lDHKIOWItK+sA+/Snbx8sm11kDFg8K6hyOsVj00o2g5i55U7mu5tQuG+JBOW3hMx6ynaMWz4NoITyDEXTrU4UG+S/9pgIH9OMe7p2yLiKHTq+OvjK5Wor+ZQXGSAUK+4ohsY74Fkt0C4DRy4DlpCFg9sbygHfOUUXCE6hLqwad5PZChYkw1aV/SK3qCtVJgtP1z+bDZCTJgWdwtiTPkdvaUfhJUvt/pqDUpJbGp0opDsfG8c9iHpgJb4C/XGHMqDM+qDHd4J96lMPnGd/o7u/qL+Ki1rio+Pj7rdrDuv+VlJb6B2e1nPf3AnwnapVGRU+tWm/fONRpsg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cZ1J76fgDoTBoo9y43ADYDwEftGw1fWk1RH8hDqiM2A=; b=NJGvFI6W9j1qTjlwsMkpsKYzfIExHP3i6M5LeEO218ysBvN7LvKMMIvWS7Wzrs5nSBharx5ufmgxbxSmw4qN/30WYS4Efjmk3EF47K9Gi11gPw0KNqxSglFLmF7dK9BFVS8fRpLq7QeSiv8pcfCmoUZzyPJPhlxcdfkWjVuNfYqMppK3KiKhMSwPaHcbrHokaFQ39bPwfZQxKMKjwCse4leCncG9107PpLKP2iJKNJyLVpbxg6sYm6TWGYsvuPYL5prK+IFZoj0BzaVTwj0SXW7d++avWiPOPSax9jqESD8OIU5EjGxagqlm3odEV+64t/STt2ojYDFbPLbL94ZllA== Received: from CO6PR12MB5396.namprd12.prod.outlook.com (2603:10b6:303:139::8) by CH0PR12MB5369.namprd12.prod.outlook.com (2603:10b6:610:d4::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.16; Mon, 5 Feb 2024 13:28:21 +0000 Received: from CO6PR12MB5396.namprd12.prod.outlook.com ([fe80::df56:6290:5b5a:e018]) by CO6PR12MB5396.namprd12.prod.outlook.com ([fe80::df56:6290:5b5a:e018%7]) with mapi id 15.20.7270.016; Mon, 5 Feb 2024 13:28:21 +0000 From: Suanming Mou To: "NBU-Contact-Thomas Monjalon (EXTERNAL)" CC: "ferruh.yigit@amd.com" , Ori Kam , Aman Singh , Yuying Zhang , Andrew Rybchenko , "dev@dpdk.org" Subject: RE: [PATCH v6 2/3] ethdev: add compare item Thread-Topic: [PATCH v6 2/3] ethdev: add compare item Thread-Index: AQHaVXDaBZq+jggkgkSjXxtfgPHx8LD7u70AgAAFmgA= Date: Mon, 5 Feb 2024 13:28:21 +0000 Message-ID: References: <20231214031227.363911-1-suanmingm@nvidia.com> <20240202004234.503097-1-suanmingm@nvidia.com> <20240202004234.503097-3-suanmingm@nvidia.com> <3205480.5fSG56mABF@thomas> In-Reply-To: <3205480.5fSG56mABF@thomas> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: CO6PR12MB5396:EE_|CH0PR12MB5369:EE_ x-ms-office365-filtering-correlation-id: ed4fdcc8-0073-4745-152a-08dc264e5269 x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 92i3je/aIAWLMOwmhgC8/f0JBXuGPD/BApyfAHIUPosy0xkzuYADU6oyGyryxu33tc9W+eIdNB/oHHsLgQfW8+IboBQd3OhFOOIpj9wRkw6zSF5iuvW8wLJp3/s73BTomc/Ab2mnz/qBSkd/IbLnfuxqnsE7ar/KY1IZg8NPpxGG8kaYhvzjJQ01X8RIqDBj8t8MR+CLQuGl78nJ5veHHfj59P95g1pdNF1YWZsQ/5UXkhyKjA4bCVI5lpX5sihtdvFDAGYI8igA3IhCFAs8eWI3aOCx73kk6t49kULrx65Q39cBExuN4D6qjYh2L3F9yvVmDfVEyR3Rp3zsmgNjMY242VGT8AkpEqN1mH5OJZG6ZixQ10UNofNN76wdKZ4tWBPXx/Z9FbEzsRkT8yRTstV3j0y/nziqNY1v+XVZZnc1k5fJ16oKOnnC7HQXaecyF2PA1ZC3u2zDPeFWwzhLgOWRn2p2B9KKDVFPd64sfS+imcwPjqDz90C1J1Yz671rlnpfKE9Xv23U6GrkG5QBlHMltf/lv77rP+YwvVWSaTeOr3VsMPFRwlNTkb96wb67oKt87KTo5jX09EQXsddYzGQP34Gm0scUJVb6qIBIRIpE2x5QL7RZLr6sPIkEcCtu x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CO6PR12MB5396.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(136003)(366004)(346002)(39860400002)(376002)(396003)(230922051799003)(64100799003)(1800799012)(451199024)(186009)(26005)(55016003)(41300700001)(316002)(54906003)(66946007)(38070700009)(64756008)(6916009)(66476007)(66556008)(66446008)(9686003)(53546011)(478600001)(7696005)(6506007)(83380400001)(71200400001)(38100700002)(122000001)(2906002)(5660300002)(76116006)(4326008)(33656002)(86362001)(8936002)(52536014)(8676002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?lguD1haHQ6wKob6bqeNm8ef4+wodAAMqSFSCvy0+CemlplvPwR9DsYDGiWL6?= =?us-ascii?Q?GJdXOCUMPstkEAqwKMTClgMrRRuwbrZmY6nNzQfIAukXprcAp0JSunrXgJdN?= =?us-ascii?Q?czPncYCrQ94vMQO/0lSUVAvMhkB7imHVpr9lzRvzgAgMS80XNKU1cl26v9zL?= =?us-ascii?Q?n6qO9tq8RPvv48RtvW9qOP0lAWmR0Zgiiood3jayGTG3VegNm94kUCvKKE1P?= =?us-ascii?Q?i/WFjyMD0jcyHc9UnPUjF6m00nWejFBVUUfyj9r4NDLvZyhsXv1/PcQvfnwx?= =?us-ascii?Q?Y4B+PQ/Prx/cv90t2RK/7GGb70jfoo/6oxxuGvcBjDGEO8ITD2Yy6blKzRgw?= =?us-ascii?Q?s1deAu48LG+pVWm/8iSqM1TWwbaPwGnDQFEy2WyDpwZizU4wja7cx1PuOf5/?= =?us-ascii?Q?5wwkfnH5d+mrP2soPdwCS6SfRTE6IjX3vmhiUBulM0Ahq5NOJVtcVbGC/56K?= =?us-ascii?Q?2ry5W6omdTVLeyN/9Sr00yDsZghp4A5+TiR8Qi1o3ZwHRnTQOgD5OtL4MmIV?= =?us-ascii?Q?9ZxEQFIbq2z9Hj/FclsUtH/q3e+eyqb1RQHU5pdDiddv7wW9jXg+7lt2eCOB?= =?us-ascii?Q?PaP7ZIssCOxHh8FIap2CJ9UxuHhQviOrtytn30zbF/4gZzghPvWjYWb3VlAj?= =?us-ascii?Q?Jq0FfcbQPRF4Fx5gnKqiwJYQDDm5ELCkba6yKHHskd6tnx4cOaljfuQo9t9v?= =?us-ascii?Q?jEBwg8zYUNEsizNzg9gWLuEzHR22ogQJolkf4jrdPsf+7B6sTzP0M+7G0luu?= =?us-ascii?Q?9Ggd0CN/SuHshfsDA5T7JmOrZDG8V/V0fCvgr2bAf+RRw7LQDN5o9KLBr9f5?= =?us-ascii?Q?eGL0+FxSMREeGs8BowpDQNLkDvN4AB5H+iWaU17Y+dBkp6Yhh3C0/XAH6/4h?= =?us-ascii?Q?ZPgphWWG5GVJLbX5owJvLePHGLLBaL5vceoNxl2vFMRKyrDyd+SArTag1fkm?= =?us-ascii?Q?/jwpm8p5wzYJmJf2bnfdElRhE7ySENPscwSCWU2Q7A7yA5PbFBraN8ydvEmO?= =?us-ascii?Q?XsGGvIS1e5A5Kpm3jdIzweYPKUbtTY1RFu0GD2xi9ILEnaAN9O2Ih5MgcS6r?= =?us-ascii?Q?MbYyeMyY94QxFerqSDb+GgM4XCSnd8yGvoN6bGGgkzmwdadWOMMC2QA2Ab3M?= =?us-ascii?Q?0VOyNQ8hoOin84keXDHitKDjC2zfmf2E8HJyt+vEakgz61P9fDL+e61cCgoJ?= =?us-ascii?Q?TO6oZLMsr+KxLn+7Jl/wzWIswxKlQqp1k9X3O7iq+WUdinbYDFBXAZGsMRfd?= =?us-ascii?Q?Akg7K1uakA62rRSty5695LNpcILw8esfx6ETXEsD4RNsk2qf2U5nDm0npwPS?= =?us-ascii?Q?FyBEKeq3YJ0Npfpix+rRcJJ4ItkMSnRYxyhOnqSQ1jzcs7SREYYvk+n3sBSj?= =?us-ascii?Q?6Wdw8dcviKftLoJ7KfuQHzsqxPd1wgxzeb0QU49tyA5yPJ5T6bBTBfXZX4tq?= =?us-ascii?Q?Rotbvsy6dGXpt8n4fewv9H4fxPx+5MqC7ZqhUI5uTSSfHsWwLRtpcK405nq9?= =?us-ascii?Q?G4aPEuBp1sj0D97d2yr/KiYQjG5rq1TaY+7LBVY1leV/rXElmthT5Vb59WCq?= =?us-ascii?Q?yv1ox+3BrdDpFvPQcdAOmYYe8oZZx0x9p4dKvxqw?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CO6PR12MB5396.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: ed4fdcc8-0073-4745-152a-08dc264e5269 X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Feb 2024 13:28:21.1554 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 2FNNRAKbHIvWF6nCDOZHI6szMlfDiDUgG0+5XKg9Onah6YkSzyUG+bqMSxta6Tq5Futz3GCG8tvqjgVXnBH/Mg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB5369 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Thomas, > -----Original Message----- > From: Thomas Monjalon > Sent: Monday, February 5, 2024 9:01 PM > To: Suanming Mou > Cc: ferruh.yigit@amd.com; Ori Kam ; Aman Singh > ; Yuying Zhang ; > Andrew Rybchenko ; dev@dpdk.org > Subject: Re: [PATCH v6 2/3] ethdev: add compare item >=20 > 02/02/2024 01:42, Suanming Mou: > > --- a/doc/guides/rel_notes/release_24_03.rst > > +++ b/doc/guides/rel_notes/release_24_03.rst > > +* **Added compare flow matching criteria.** >=20 > Suggestion: > "Added flow matching with various comparisons." > Not sure how to illustrate the idea. What about " Added flow matching with various comparison results." ? >=20 > > + Added ``RTE_FLOW_ITEM_TYPE_COMPARE`` to allow matching on compare >=20 > s/compare/comparison/ >=20 > > + result between the packet fields or value. >=20 > remove "the" >=20 >=20 > > --- a/lib/ethdev/rte_flow.h > > +++ b/lib/ethdev/rte_flow.h > > + /** > > + * Matches the packet with compare result. >=20 > Match packet with various comparison types. >=20 > > + * > > + * See struct rte_flow_item_compare. > > + */ > > + RTE_FLOW_ITEM_TYPE_COMPARE, > > }; > > > > /** > > @@ -2365,6 +2372,182 @@ static const struct rte_flow_item_ptype > > rte_flow_item_ptype_mask =3D { }; #endif > > > > +/** > > + * Field IDs for packet field. > > + * e.g. the packet field IDs used in > > +RTE_FLOW_ACTION_TYPE_MODIFY_FIELD > > + * and RTE_FLOW_ITEM_TYPE_COMPARE. > > + */ > > +enum rte_flow_field_id { > > + RTE_FLOW_FIELD_START =3D 0, /**< Start of a packet. */ > > + RTE_FLOW_FIELD_MAC_DST, /**< Destination MAC Address. > */ > > + RTE_FLOW_FIELD_MAC_SRC, /**< Source MAC Address. */ > > + RTE_FLOW_FIELD_VLAN_TYPE, /**< VLAN Tag Identifier. */ > > + RTE_FLOW_FIELD_VLAN_ID, /**< VLAN Identifier. */ > > + RTE_FLOW_FIELD_MAC_TYPE, /**< EtherType. */ > > + RTE_FLOW_FIELD_IPV4_DSCP, /**< IPv4 DSCP. */ > > + RTE_FLOW_FIELD_IPV4_TTL, /**< IPv4 Time To Live. */ > > + RTE_FLOW_FIELD_IPV4_SRC, /**< IPv4 Source Address. */ > > + RTE_FLOW_FIELD_IPV4_DST, /**< IPv4 Destination Address. */ > > + RTE_FLOW_FIELD_IPV6_DSCP, /**< IPv6 DSCP. */ > > + RTE_FLOW_FIELD_IPV6_HOPLIMIT, /**< IPv6 Hop Limit. */ > > + RTE_FLOW_FIELD_IPV6_SRC, /**< IPv6 Source Address. */ > > + RTE_FLOW_FIELD_IPV6_DST, /**< IPv6 Destination Address. */ > > + RTE_FLOW_FIELD_TCP_PORT_SRC, /**< TCP Source Port Number. > */ > > + RTE_FLOW_FIELD_TCP_PORT_DST, /**< TCP Destination Port > Number. */ > > + RTE_FLOW_FIELD_TCP_SEQ_NUM, /**< TCP Sequence Number. */ > > + RTE_FLOW_FIELD_TCP_ACK_NUM, /**< TCP Acknowledgment > Number. */ > > + RTE_FLOW_FIELD_TCP_FLAGS, /**< TCP Flags. */ > > + RTE_FLOW_FIELD_UDP_PORT_SRC, /**< UDP Source Port Number. > */ > > + RTE_FLOW_FIELD_UDP_PORT_DST, /**< UDP Destination Port > Number. */ > > + RTE_FLOW_FIELD_VXLAN_VNI, /**< VXLAN Network Identifier. */ > > + RTE_FLOW_FIELD_GENEVE_VNI, /**< GENEVE Network > Identifier. */ > > + RTE_FLOW_FIELD_GTP_TEID, /**< GTP Tunnel Endpoint Identifier. */ > > + RTE_FLOW_FIELD_TAG, /**< Tag value. */ > > + RTE_FLOW_FIELD_MARK, /**< Mark value. */ > > + RTE_FLOW_FIELD_META, /**< Metadata value. */ > > + RTE_FLOW_FIELD_POINTER, /**< Memory pointer. */ > > + RTE_FLOW_FIELD_VALUE, /**< Immediate value. */ > > + RTE_FLOW_FIELD_IPV4_ECN, /**< IPv4 ECN. */ > > + RTE_FLOW_FIELD_IPV6_ECN, /**< IPv6 ECN. */ > > + RTE_FLOW_FIELD_GTP_PSC_QFI, /**< GTP QFI. */ > > + RTE_FLOW_FIELD_METER_COLOR, /**< Meter color marker. */ > > + RTE_FLOW_FIELD_IPV6_PROTO, /**< IPv6 next header. */ > > + RTE_FLOW_FIELD_FLEX_ITEM, /**< Flex item. */ > > + RTE_FLOW_FIELD_HASH_RESULT, /**< Hash result. */ > > + RTE_FLOW_FIELD_GENEVE_OPT_TYPE, /**< GENEVE option type. */ > > + RTE_FLOW_FIELD_GENEVE_OPT_CLASS,/**< GENEVE option class. */ > > + RTE_FLOW_FIELD_GENEVE_OPT_DATA, /**< GENEVE option data. */ > > + RTE_FLOW_FIELD_MPLS, /**< MPLS header. */ > > + RTE_FLOW_FIELD_TCP_DATA_OFFSET, /**< TCP data offset. */ > > + RTE_FLOW_FIELD_IPV4_IHL, /**< IPv4 IHL. */ > > + RTE_FLOW_FIELD_IPV4_TOTAL_LEN, /**< IPv4 total length. */ > > + RTE_FLOW_FIELD_IPV6_PAYLOAD_LEN,/**< IPv6 payload length. */ > > + RTE_FLOW_FIELD_RANDOM /**< Random value. */ > > +}; >=20 > You should use spaces for aligmnent of the comments. Sorry, the original code is not using spaces.=20 Anyway, never mind, since I touched that, let me format them with spaces. >=20 > [...] > > +struct rte_flow_field_data { > > + enum rte_flow_field_id field; /**< Field or memory type ID. */ > > + union { > > + struct { > > + /** Encapsulation level and tag index or flex item > handle. */ > > + union { > > + struct { > > + /** > > + * Packet encapsulation level containing > > + * the field to modify. > > + * > > + * - @p 0 requests the default behavior. > > + * Depending on the packet type, it > > + * can mean outermost, innermost or > > + * anything in between. > > + * > > + * It basically stands for the > > + * innermost encapsulation level. > > + * Modification can be performed > > + * according to PMD and device > > + * capabilities. > > + * > > + * - @p 1 requests modification to be > > + * performed on the outermost packet > > + * encapsulation level. > > + * > > + * - @p 2 and subsequent values > request > > + * modification to be performed on > > + * the specified inner packet > > + * encapsulation level, from > > + * outermost to innermost (lower to > > + * higher values). > > + * > > + * Values other than @p 0 are not > > + * necessarily supported. > > + * > > + * @note that for MPLS field, > > + * encapsulation level also include > > + * tunnel since MPLS may appear in > > + * outer, inner or tunnel. > > + */ > > + uint8_t level; > > + union { > > + /** > > + * Tag index array inside > > + * encapsulation level. > > + * Used for VLAN, MPLS or TAG > types. > > + */ > > + uint8_t tag_index; > > + /** > > + * Geneve option identifier. > > + * Relevant only for > > + * > RTE_FLOW_FIELD_GENEVE_OPT_XXXX > > + * modification type. > > + */ > > + struct { > > + /** > > + * Geneve option type. > > + */ > > + uint8_t type; > > + /** > > + * Geneve option class. > > + */ > > + rte_be16_t class_id; > > + }; > > + }; > > + }; > > + struct rte_flow_item_flex_handle *flex_handle; > > + }; > > + /** Number of bits to skip from a field. */ > > + uint32_t offset; > > + }; > > + /** > > + * Immediate value for RTE_FLOW_FIELD_VALUE, presented in > the > > + * same byte order and length as in relevant rte_flow_item_xxx. > > + * The immediate source bitfield offset is inherited from > > + * the destination's one. > > + */ > > + uint8_t value[16]; > > + /** > > + * Memory address for RTE_FLOW_FIELD_POINTER, memory > layout > > + * should be the same as for relevant field in the > > + * rte_flow_item_xxx structure. > > + */ > > + void *pvalue; > > + }; > > +}; > > + > > +/** > > + * Expected operation types for compare item. > > + */ > > +enum rte_flow_item_compare_op { > > + RTE_FLOW_ITEM_COMPARE_EQ, /* Compare result equal. */ > > + RTE_FLOW_ITEM_COMPARE_NE, /* Compare result not equal. */ > > + RTE_FLOW_ITEM_COMPARE_LT, /* Compare result less than. */ > > + RTE_FLOW_ITEM_COMPARE_LE, /* Compare result less than or > equal. */ > > + RTE_FLOW_ITEM_COMPARE_GT, /* Compare result great than. > */ > > + RTE_FLOW_ITEM_COMPARE_GE, /* Compare result great than or > equal. */ > > +}; >=20 > It's a pity we cannot easily what changed because it moved. > Maybe make the move in a separate patch before changing it? Since you are the second one concern about that, I will split the move to a= separate patch before. >=20