From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BFF47A0350; Fri, 4 Feb 2022 18:17:16 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 531CD4013F; Fri, 4 Feb 2022 18:17:16 +0100 (CET) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 3899040041; Fri, 4 Feb 2022 18:17:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643995034; x=1675531034; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=NrQfdcpgvME80BQbv16WYw6kMh232JyXcNeAFaHAZtc=; b=oH2hk2dyHtM8ax5I+k6Gucnmq6EzlPdnzV8ziV5RUEpDqFF7JFCX6eYs 7CH0CJeMETZ3/9Q10kWfKJ/TilegW2WvHVUB62DA+W6XzGBg8JUvfslee wHeXkaCKWBusLYGycTnTHiL6/sxXzdRoE45WKTo2t3JxJaR3/YB2MKWSt KbnjDFgQ9qM51frIa/tSEIDFl0zJbADPZ2Ypd2Hk/MZwgiz99aNpmo3lv /t85xmceAplXIsx8/u106zWeuQf4KAidF8NTGp7UluobAdPGtQY9LkHnt aKT7KdGh0V/Tegp1ZJPJF1BM24GDlUZwLApvxam3xEIGRsLNH4cAD5B1c Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10248"; a="228376231" X-IronPort-AV: E=Sophos;i="5.88,343,1635231600"; d="scan'208";a="228376231" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2022 09:17:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,343,1635231600"; d="scan'208";a="498536940" Received: from fmsmsx606.amr.corp.intel.com ([10.18.126.86]) by orsmga002.jf.intel.com with ESMTP; 04 Feb 2022 09:17:12 -0800 Received: from fmsmsx609.amr.corp.intel.com (10.18.126.89) by fmsmsx606.amr.corp.intel.com (10.18.126.86) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Fri, 4 Feb 2022 09:17:12 -0800 Received: from fmsmsx601.amr.corp.intel.com (10.18.126.81) by fmsmsx609.amr.corp.intel.com (10.18.126.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Fri, 4 Feb 2022 09:17:11 -0800 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20 via Frontend Transport; Fri, 4 Feb 2022 09:17:11 -0800 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (104.47.73.44) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.20; Fri, 4 Feb 2022 09:16:45 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=V8eYxLg0eDez50+ClIdc/toIyqc+eyJc59vztG/O/x+zkl3zk9Gnqwkff3e7tDg2xG9jkuV4tntp6fOMB1IUKQWEQ0DpXwLUWFlwbfUDuurcnVnpIPrJ6pfrQkoe6DZk5onP4z4+K+ZAb1WgaCjM4DlmP5fS8PUJx518R/e6n4kqjI9I0gifcv1WfIji6Hivv+/K+V1UeV0dpGpGrU9yAzlGJAFUMgwSFdt4Z45yewioX5b8Rk9D3u5gpfPbeQbsIVQ549S+La7pdQOKB9PWbzQ8cNkwtezwyB0nVm5EfqktHOpSA0wsCo90JUffmYv4BmBGrfSqUbW1ImiOx7Vm3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nUhr2Bxl6rGEtdlxJsadDl7rmJXzY2I7oxfIZrcWcXI=; b=l7JbYtYSOlfpYpTFFYkoa7cBG4ntlr93MJLX6ACyOumm+tiAyQc/ohG6eVL4wKQgkuMw7OJUSnKBnhajnTFYOBLRwdW6/vgVDkH/x3Cek/JCPklL8oevFw9tOameZL+GQN/3Evz1BaEZCDekJkOHKwxcFs/iypzJa79hZVTZZ75suOiwx02QqKTZG0YWDEyYOfFzfPvsdcZYRP+k+6f9FJ7s4j9sXLEEp8nay6/EzbOqxY7QO6vvTa0ch/jAze1CYoF11YTAhS9OMpp5yDYmdqU04KROe7f9LZev4UfckmEh3BlFDdARfoEfvtxoKjp+QHAY/AubRCABbTlavvSLJA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none Received: from DM6PR11MB4491.namprd11.prod.outlook.com (2603:10b6:5:204::19) by BY5PR11MB4055.namprd11.prod.outlook.com (2603:10b6:a03:18b::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4951.12; Fri, 4 Feb 2022 17:16:40 +0000 Received: from DM6PR11MB4491.namprd11.prod.outlook.com ([fe80::8ccc:ed65:78fa:1b07]) by DM6PR11MB4491.namprd11.prod.outlook.com ([fe80::8ccc:ed65:78fa:1b07%4]) with mapi id 15.20.4951.012; Fri, 4 Feb 2022 17:16:40 +0000 From: "Ananyev, Konstantin" To: Luc Pelletier , "Richardson, Bruce" CC: "dev@dpdk.org" , "Li, Xiaoyun" , "stable@dpdk.org" Subject: RE: [PATCH v5] eal: fix unaligned loads/stores in rte_memcpy_generic Thread-Topic: [PATCH v5] eal: fix unaligned loads/stores in rte_memcpy_generic Thread-Index: AQHYC7iuO+/uD//7cU2PwhFiekEQFqyDvVrQ Date: Fri, 4 Feb 2022 17:16:39 +0000 Message-ID: References: <20220115194102.444140-1-lucp.at.work@gmail.com> <20220117153711.32829-1-lucp.at.work@gmail.com> In-Reply-To: <20220117153711.32829-1-lucp.at.work@gmail.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.200.16 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 58462642-7d26-41d8-9895-08d9e8021baa x-ms-traffictypediagnostic: BY5PR11MB4055:EE_ x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: q3LN5pCfYvdG8fpZxv7odsKGMLfIRzaZE8bZGYwhCaAko6XQFY0FtSGin0fEo/2hUvSqabN0P+KF0B0GA22LBdhvC5grvqTFOKnaWWTR5Yk8X1vsrJG0esquxg4htkjhZqCej0FVhbunVKl1kfWnbesBWtwQ1HIo2Xk0G1/2SL39T9qWWlE91EyX61qFChNVlbCrlVsyuHdHUOe1a0D6wI0UCbK/KSQ6Xe39OqQC/zIfthxEYGRjucEbN45I2HTL0I62n4Ypr/VzHMA8IEEWI6ft1LXTPIjWy6EJOOb4W7ih2jUHOvxHIOyWsXHzp2n+K5Fdw4fXtbFDBtTQhPWa5DqRtCpataMquu5TSH7SksE51FRMs47iMLyFTyowBhgMM8KX0l65eHMU6w0AS49qpkakvoMQiys6TGOI6J0dT6v0Bzd0aA+YS52UC3CiiuTKF3Z0C5d8xBS90rRsVnHrtAq9T07CKUCl7yaQz3JOTvN0DFF8me71hFM+/yeMI/QguJGedextYDPV+7B7G8c3/JZF36sGDVSCc0PHXwaBfzF1cfVcpRabv9o/T73qpzl7Po6tHM0w8oDsJ3y8PgvGOcESIy8X4I1TToys+Yrwa1SCkgGWk65fN4smXb1w+FV9BNiCXMhWbS/C/Eecz5OhOa52KToxhphzRD6/rY2yHTSxBYMbUoYZ257rNI5p/QHeVq32jA6nqNMbmj298LnHzw0PYyRCq8D6Kq7Hb9O38QQ= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB4491.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(122000001)(71200400001)(508600001)(2906002)(38070700005)(26005)(9686003)(186003)(54906003)(110136005)(7696005)(33656002)(316002)(6636002)(6506007)(55016003)(86362001)(66946007)(76116006)(8936002)(64756008)(66476007)(66556008)(82960400001)(4326008)(38100700002)(5660300002)(83380400001)(66446008)(52536014)(8676002)(41533002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?8Y7CLc+kDempopqFkWxJB+SeAScsF4h3Cxr9JDtfYe93tQQerFRjvFWl8bmQ?= =?us-ascii?Q?M6BiBG9ncRGklqCSElbDyC3CjQvN+Rmhs3PEeyAb4LIAvnzWta7/w96v7LJd?= =?us-ascii?Q?XlofT/fTKdQI0yIEBqK4AnRt7RSFNEDdMmfTScnull7ji8X6BMlhfF0nv02c?= =?us-ascii?Q?n8EwFHzQ7GHMC2rSKe6A0oxjsee28A9tKbVYNaE0748+hcMK4Wvl/fgEEEJo?= =?us-ascii?Q?nsnnGwOXfhgZBgr4+6vhL9wBFCo2MJlu0+c7V2ipVmffFyH/H1HTIb3Q88px?= =?us-ascii?Q?JK9ScUkW2ZxgHkfhXoVKJkuoRHtAK8OPbASGyybTE3Xw2HZxbDBfpfrftjnE?= =?us-ascii?Q?kqS7aZbs/nT7REH4VMPtzTj4sob9z1dSbJPZToaSk3e7AfphoBETJIHMBFl0?= =?us-ascii?Q?GkoiD0r41q5R/5QdLm2SzrA/AMSeoIhDCsFhZW0aPN08u2m+t84SjbLSWeMk?= =?us-ascii?Q?4fvBDlC7rRosITPrA32HbGRY+YcdGAhWbGpf4GZLEO6yHEb+tTEixVTrkAu4?= =?us-ascii?Q?QhSei54FsC3eY42Gh6AGSr6JAKInrsC34FybWaTVSpBbGLtNCdxvmwVbhtkx?= =?us-ascii?Q?p8SN7XTVTCDd8dIDFNWy16qJ25M7VGsuJrwyfkjTTufTCFa7xSzdgeSFomEs?= =?us-ascii?Q?teNsOP84QvrCbFG42y5kncGJ568z6a9q1R/psPUvw+9GnraySNeI7UqZTyki?= =?us-ascii?Q?fU0OeXGI+rTYUl9rhp/0bjUOaGdcZkWXAXO5reOwI4hlxaAzYUV+JojOMCpQ?= =?us-ascii?Q?0YDCxaljbx7v7+QvqycvmUrzAMwIU01UN4Xj+jjJ5oBtXz+fY+n7bJiDWuKk?= =?us-ascii?Q?o6cuOPQf6Xn0kUqq/X2z2X9ZOSLRmD2Hiy8huBmcQhZqn29zqn5kGhbN57k3?= =?us-ascii?Q?zJodRcxMf66dHPtlHSM1j2x5uK9ZpYnzw2rRh+iS00FtKoui4NsXGO+ZwiPE?= =?us-ascii?Q?gfwfSYTS4sVpYHux8nrs1Lp1oct0LUvfAWyyOpiqVhNfkJEKxpiuMmmn3ytE?= =?us-ascii?Q?ZwExnOg50/u3mXpFisfX+TnYxh5+xgQl4gvu20HqfZwOFpif9EKWrjhR7tLJ?= =?us-ascii?Q?BMsjQ7dY0cQCKFfUB8uXn7W0DF3y6hMQkZVX8Yw9/rAdqumco19o5E2TFEN8?= =?us-ascii?Q?Mwc5nIUH8xmCqrfiaoGShebbAVzrW9QtOoDBjV6AhdaSDkNYtJeqhjUd+Kx+?= =?us-ascii?Q?t5NsOY/avClAiLlMnP/cxFLlY9bZIujYHDlsLJZaFEwLHrOEw8LKCPJ5/EZk?= =?us-ascii?Q?zSNEQIrzboexAPINoeWmtiXcGITmwJWO1os+BUw13zEjt+h7ARIdcehn439l?= =?us-ascii?Q?j4w9hQGpi+1ydNMzr/CpGv/r69jDsNi6cPw5crDgJUfPTDIImMOwIJkGR/OR?= =?us-ascii?Q?E2zcA33C6j3h4C0wbkSPbkKpAC+9m0EyY7faHn92Bp2EF1hcn7MdNh6e0aqj?= =?us-ascii?Q?7xusvbgky5c0j0lq/hM1u0XyJ7xXhHOg6dSBhNdsHwUTjFKMTA7RsYxKD9f7?= =?us-ascii?Q?9pSwLV3H7xGTa2j5CLvkwD9TNTI5YSOgdkY7veMJbOAV6b6a1hzhrbrP+JX8?= =?us-ascii?Q?qmwBnbdQGS8ZOyhzuAN1XdPsSz4R88B+su50xdGdAiibc29EP27x2mNX7eNo?= =?us-ascii?Q?GR9NZ8MEVj0+GOtinvjPBofTBMS/pLThBFq6DqXS60rwhyR2Ys+RWRCuJIXZ?= =?us-ascii?Q?/FnHYg=3D=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4491.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 58462642-7d26-41d8-9895-08d9e8021baa X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Feb 2022 17:16:40.0285 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Ij7fop433+CemZpTEoYy/137+n61NjICKDCs5BeA2G4kBBpIzHdGU8H2JC6FcUL8eSynudqtPEJ7NEotfm+7WUS9T4XfvIWnBcjmWX0hkss= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR11MB4055 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > Calls to rte_memcpy_generic could result in unaligned loads/stores for > 1 < n < 16. This is undefined behavior according to the C standard, > and it gets flagged by the clang undefined behavior sanitizer. >=20 > rte_memcpy_generic is called with unaligned src and dst addresses. > When 1 < n < 16, the code would cast both src and dst to a qword, > dword or word pointer, without verifying the alignment of src/dst. The > code was changed to use a packed structure to perform the unaligned > load/store operations. This results in unaligned load/store operations > to be C standards-compliant. Still not sure we need to fix that: This is x86 specific code-path, and as I remember on x86 there are no penalties for unaligned access to 2/4/8 byte values.=20 Though I like introduction of rte_mov15_or_less() function -t helps with code dedup.=20 >=20 > Fixes: d35cc1fe6a7a ("eal/x86: revert select optimized memcpy at run-time= ") > Cc: Xiaoyun Li > Cc: stable@dpdk.org >=20 > Signed-off-by: Luc Pelletier > --- >=20 > Thanks to Stephen's pointer to look at the linux kernel, I was able to > find a way to perform the unaligned load/store using pure C code. The > new functions added to perform the load/store could likely be moved to a > different file and the code duplication could likely be eliminated by > using a macro. However, I will hold off on making these changes until I > get confirmation from maintainers that this technique is acceptable and > this is what we want to move forward with. >=20 > lib/eal/x86/include/rte_memcpy.h | 142 +++++++++++++++++-------------- > 1 file changed, 80 insertions(+), 62 deletions(-) >=20 > diff --git a/lib/eal/x86/include/rte_memcpy.h b/lib/eal/x86/include/rte_m= emcpy.h > index 1b6c6e585f..4e876d39eb 100644 > --- a/lib/eal/x86/include/rte_memcpy.h > +++ b/lib/eal/x86/include/rte_memcpy.h > @@ -45,6 +45,83 @@ extern "C" { > static __rte_always_inline void * > rte_memcpy(void *dst, const void *src, size_t n); >=20 > +static __rte_always_inline uint64_t > +rte_load_unaligned_uint64(const void *ptr) > +{ > + struct unaligned_uint64 { uint64_t val; } __rte_packed; > + return ((const struct unaligned_uint64 *)ptr)->val; > +} > + > +static __rte_always_inline uint32_t > +rte_load_unaligned_uint32(const void *ptr) > +{ > + struct unaligned_uint32 { uint32_t val; } __rte_packed; > + return ((const struct unaligned_uint32 *)ptr)->val; > +} > + > +static __rte_always_inline uint16_t > +rte_load_unaligned_uint16(const void *ptr) > +{ > + struct unaligned_uint16 { uint16_t val; } __rte_packed; > + return ((const struct unaligned_uint16 *)ptr)->val; > +} > + > +static __rte_always_inline void > +rte_store_unaligned_uint64(void *ptr, uint64_t val) > +{ > + struct unaligned_uint64 { uint64_t val; } __rte_packed; > + ((struct unaligned_uint64 *)ptr)->val =3D val; > +} > + > +static __rte_always_inline void > +rte_store_unaligned_uint32(void *ptr, uint32_t val) > +{ > + struct unaligned_uint32 { uint32_t val; } __rte_packed; > + ((struct unaligned_uint32 *)ptr)->val =3D val; > +} > + > +static __rte_always_inline void > +rte_store_unaligned_uint16(void *ptr, uint16_t val) > +{ > + struct unaligned_uint16 { uint16_t val; } __rte_packed; > + ((struct unaligned_uint16 *)ptr)->val =3D val; > +} > + > +/** > + * Copy bytes from one location to another, > + * locations should not overlap. > + * Use with unaligned src/dst, and n <=3D 15. > + */ > +static __rte_always_inline void * > +rte_mov15_or_less_unaligned(void *dst, const void *src, size_t n) > +{ > + void *ret =3D dst; > + if (n & 8) { > + rte_store_unaligned_uint64( > + dst, > + rte_load_unaligned_uint64(src)); > + src =3D ((const uint64_t *)src + 1); > + dst =3D ((uint64_t *)dst + 1); > + } > + if (n & 4) { > + rte_store_unaligned_uint32( > + dst, > + rte_load_unaligned_uint32(src)); > + src =3D ((const uint32_t *)src + 1); > + dst =3D ((uint32_t *)dst + 1); > + } > + if (n & 2) { > + rte_store_unaligned_uint16( > + dst, > + rte_load_unaligned_uint16(src)); > + src =3D ((const uint16_t *)src + 1); > + dst =3D ((uint16_t *)dst + 1); > + } > + if (n & 1) > + *(uint8_t *)dst =3D *(const uint8_t *)src; > + return ret; > +} > + > #if defined __AVX512F__ && defined RTE_MEMCPY_AVX512 >=20 > #define ALIGNMENT_MASK 0x3F > @@ -171,8 +248,6 @@ rte_mov512blocks(uint8_t *dst, const uint8_t *src, si= ze_t n) > static __rte_always_inline void * > rte_memcpy_generic(void *dst, const void *src, size_t n) > { > - uintptr_t dstu =3D (uintptr_t)dst; > - uintptr_t srcu =3D (uintptr_t)src; > void *ret =3D dst; > size_t dstofss; > size_t bits; > @@ -181,24 +256,7 @@ rte_memcpy_generic(void *dst, const void *src, size_= t n) > * Copy less than 16 bytes > */ > if (n < 16) { > - if (n & 0x01) { > - *(uint8_t *)dstu =3D *(const uint8_t *)srcu; > - srcu =3D (uintptr_t)((const uint8_t *)srcu + 1); > - dstu =3D (uintptr_t)((uint8_t *)dstu + 1); > - } > - if (n & 0x02) { > - *(uint16_t *)dstu =3D *(const uint16_t *)srcu; > - srcu =3D (uintptr_t)((const uint16_t *)srcu + 1); > - dstu =3D (uintptr_t)((uint16_t *)dstu + 1); > - } > - if (n & 0x04) { > - *(uint32_t *)dstu =3D *(const uint32_t *)srcu; > - srcu =3D (uintptr_t)((const uint32_t *)srcu + 1); > - dstu =3D (uintptr_t)((uint32_t *)dstu + 1); > - } > - if (n & 0x08) > - *(uint64_t *)dstu =3D *(const uint64_t *)srcu; > - return ret; > + return rte_mov15_or_less_unaligned(dst, src, n); > } >=20 > /** > @@ -379,8 +437,6 @@ rte_mov128blocks(uint8_t *dst, const uint8_t *src, si= ze_t n) > static __rte_always_inline void * > rte_memcpy_generic(void *dst, const void *src, size_t n) > { > - uintptr_t dstu =3D (uintptr_t)dst; > - uintptr_t srcu =3D (uintptr_t)src; > void *ret =3D dst; > size_t dstofss; > size_t bits; > @@ -389,25 +445,7 @@ rte_memcpy_generic(void *dst, const void *src, size_= t n) > * Copy less than 16 bytes > */ > if (n < 16) { > - if (n & 0x01) { > - *(uint8_t *)dstu =3D *(const uint8_t *)srcu; > - srcu =3D (uintptr_t)((const uint8_t *)srcu + 1); > - dstu =3D (uintptr_t)((uint8_t *)dstu + 1); > - } > - if (n & 0x02) { > - *(uint16_t *)dstu =3D *(const uint16_t *)srcu; > - srcu =3D (uintptr_t)((const uint16_t *)srcu + 1); > - dstu =3D (uintptr_t)((uint16_t *)dstu + 1); > - } > - if (n & 0x04) { > - *(uint32_t *)dstu =3D *(const uint32_t *)srcu; > - srcu =3D (uintptr_t)((const uint32_t *)srcu + 1); > - dstu =3D (uintptr_t)((uint32_t *)dstu + 1); > - } > - if (n & 0x08) { > - *(uint64_t *)dstu =3D *(const uint64_t *)srcu; > - } > - return ret; > + return rte_mov15_or_less_unaligned(dst, src, n); > } >=20 > /** > @@ -672,8 +710,6 @@ static __rte_always_inline void * > rte_memcpy_generic(void *dst, const void *src, size_t n) > { > __m128i xmm0, xmm1, xmm2, xmm3, xmm4, xmm5, xmm6, xmm7, xmm8; > - uintptr_t dstu =3D (uintptr_t)dst; > - uintptr_t srcu =3D (uintptr_t)src; > void *ret =3D dst; > size_t dstofss; > size_t srcofs; > @@ -682,25 +718,7 @@ rte_memcpy_generic(void *dst, const void *src, size_= t n) > * Copy less than 16 bytes > */ > if (n < 16) { > - if (n & 0x01) { > - *(uint8_t *)dstu =3D *(const uint8_t *)srcu; > - srcu =3D (uintptr_t)((const uint8_t *)srcu + 1); > - dstu =3D (uintptr_t)((uint8_t *)dstu + 1); > - } > - if (n & 0x02) { > - *(uint16_t *)dstu =3D *(const uint16_t *)srcu; > - srcu =3D (uintptr_t)((const uint16_t *)srcu + 1); > - dstu =3D (uintptr_t)((uint16_t *)dstu + 1); > - } > - if (n & 0x04) { > - *(uint32_t *)dstu =3D *(const uint32_t *)srcu; > - srcu =3D (uintptr_t)((const uint32_t *)srcu + 1); > - dstu =3D (uintptr_t)((uint32_t *)dstu + 1); > - } > - if (n & 0x08) { > - *(uint64_t *)dstu =3D *(const uint64_t *)srcu; > - } > - return ret; > + return rte_mov15_or_less_unaligned(dst, src, n); > } >=20 > /** > -- > 2.25.1