From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by dpdk.org (Postfix) with ESMTP id 1AD6723C for ; Wed, 18 Jul 2018 22:58:20 +0200 (CEST) Received: by mail-pg1-f196.google.com with SMTP id f1-v6so2526921pgq.12 for ; Wed, 18 Jul 2018 13:58:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=t/WKtPVtLl8tBWYH9vuX4sW7e1kAuPW9kDhUm9LljGc=; b=Qf9HsBDXPfd7ERsC9bWB/mrID+hUVvEr7a/UGbKtEN/BKlkq4r3nfBoB0/gbFZxRG5 NCHtGs0nDbTycWMxRvEESPZ0OzZRGkCqZk7FsC0MqI+FzsW9aabIZ6eakrO3dV42q8eL RgBUK57IuECgMsduWFKxP4BXtfkReka4/+G5zOyMVMyXr8erX6DdDmW7yyU9z7KtPIzn LdITRO2JTF0TnckWSVdJgTVWMeoDrSGZgc0gDE+XigIfUBrN24NhqzWpCRZNU2enwELR PJo4FkloTMXa+daeqEvGJG2B3ptUVxLfOgUoxA0hF3gacSOeFGMuaNH2UnX4SdZ/D8JX iPPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t/WKtPVtLl8tBWYH9vuX4sW7e1kAuPW9kDhUm9LljGc=; b=UDgE46V22OVzfOWTm45E1hIVcxgbStuhG87mdrSbNg7iIoaQHlUBOJ7Ia5wL4zs2WR lB5uTqbRXasBbr1cD2AbwJgOWBNXGqnlUALB3qYOgBWAfo/BYSky028V42DGryeTYiTN Hc5Avcb5ML5ijrdsCg2lA94y/FSAIxd2TowTJK8qJtA6iAOD7tHG1aJGqjcX8h/3T/f/ OgVfU0zT5H9Hcm/Z1gGQswyvjozf+7pWn5/HrKOiw2B34G5TtA+DWwrHhdbaR6haJ7fa XNPq5VLd6kopsNXvpY2CeFmjrm6cThiQWIkxWOCTqUOs1z54N9I7pUcKYk3tQp3WzF1m J+oA== X-Gm-Message-State: AOUpUlFjCa9PF7nlo1oxjuC7JL2+UNuYlmrBFNAXXAyBJrevlwuo0hBL mFYpsOwSkj6DRMcG9fj2jTyz+A== X-Google-Smtp-Source: AAOMgpd+3nrk9qENaKqBGns9sCY863d+kL0D5RdXSrtnEGiDztLuzUF549xI3dAOysqBME0uBwdZjQ== X-Received: by 2002:a62:1157:: with SMTP id z84-v6mr6737594pfi.66.1531947500146; Wed, 18 Jul 2018 13:58:20 -0700 (PDT) Received: from xeon-e3 (204-195-22-127.wavecable.com. [204.195.22.127]) by smtp.gmail.com with ESMTPSA id o65-v6sm6381221pfb.104.2018.07.18.13.58.19 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 18 Jul 2018 13:58:20 -0700 (PDT) Date: Wed, 18 Jul 2018 13:58:17 -0700 From: Stephen Hemminger To: Andrew Rybchenko Cc: "Burakov, Anatoly" , "dev@dpdk.org" , Sergio Gonzalez Monroy Message-ID: <20180718135817.66728c37@xeon-e3> In-Reply-To: References: <8bc76811-ac29-d7f2-e4c3-12b50fd44dba@solarflare.com> <58e5044c-3d13-9171-4168-b4d6b1d61927@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] Memory allocated using rte_zmalloc() has non-zeros X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jul 2018 20:58:21 -0000 On Wed, 18 Jul 2018 22:52:12 +0300 Andrew Rybchenko wrote: > On 18.07.2018 20:18, Burakov, Anatoly wrote: > > On 18-Jul-18 4:20 PM, Andrew Rybchenko wrote: =20 > >> Hi Anatoly, > >> > >> I'm investigating issue which finally comes to the fact that memory=20 > >> allocated using > >> rte_zmalloc() has non zeros. > >> > >> If I add memset just after allocation, everything is perfect and=20 > >> works fine. > >> > >> I've found out that memset was removed from rte_zmalloc_socket() some= =20 > >> time ago: > >> =20 > >> =C2=A0>>> =20 > >> commit b78c9175118f7d61022ddc5c62ce54a1bd73cea5 > >> Author: Sergio Gonzalez Monroy > >> Date:=C2=A0=C2=A0 Tue Jul 5 12:01:16 2016 +0100 > >> > >> =C2=A0=C2=A0=C2=A0=C2=A0 mem: do not zero out memory on zmalloc > >> > >> =C2=A0=C2=A0=C2=A0=C2=A0 Zeroing out memory on rte_zmalloc_socket is n= ot required anymore=20 > >> since all > >> =C2=A0=C2=A0=C2=A0=C2=A0 allocated memory is already zeroed. > >> > >> =C2=A0=C2=A0=C2=A0=C2=A0 Signed-off-by: Sergio Gonzalez Monroy=20 > >> > >> <<< > >> > >> but may be something has changed now that made above statement false. > >> > >> I observe the problem when memory is reallocated. I.e. I configure 7=20 > >> queues, > >> start, stop, reconfigure to 3 queues, start. Memory is allocated on=20 > >> start and > >> freed on stop, since we have less queues on the second start it is=20 > >> allocated > >> in a different way and reuses previously allocated/freed memory. > >> > >> Do you have any ideas what could be wrong? > >> > >> Andrew. > >> > >> =20 > > > > Hi Andrew, > > > > I will look into it first thing tomorrow. In general, we memset(0) on=20 > > free, and kernel gives us zeroed out pages initially, so the most=20 > > likely point of failure is that i'm not overwring some malloc headers=20 > > correctly on free. =20 >=20 > OK, at least now I know how it is supposed to work in theory. >=20 > The following region was allocated=C2=A0 (the second number below is poin= ter=20 > plus size) > ALLOC 0x7fffa3264080-0x7fffa32640b8 >=20 > Not zerod address is 16 bytes before: > (gdb) p/x ((uint64_t *)0x7fffa3264070)[0] > $4 =3D 0x4000000002 > (gdb) p/x ((uint64_t=C2=A0 *)0x7fffa3264070)[1] > $5 =3D 0x80 >=20 > then freed > FREE 0x7fffa3264080-0x7fffa32640b8 >=20 > but above values (gdb) are still the same > then it is allocated as the part of bigger memory chunk > ALLOC 0x7fffa3245b80-0x7fffa3265fd8 > which should contain zeros, but above values are still the same. >=20 > It is interesting that it looks like it was the first block freed on the= =20 > port stop. I'm not 100% sure since I've put printouts to my allocation=20 > wrapper, not EAL. >=20 > Many thanks, > Andrew. memset here is what is supposed to clear the data. struct malloc_elem * malloc_elem_free(struct malloc_elem *elem) { void *ptr; size_t data_len; ptr =3D RTE_PTR_ADD(elem, MALLOC_ELEM_HEADER_LEN + elem->pad); data_len =3D elem->size - elem->pad - MALLOC_ELEM_OVERHEAD; elem =3D malloc_elem_join_adjacent_free(elem); malloc_elem_free_list_insert(elem); elem->pad =3D 0; /* decrease heap's count of allocated elements */ elem->heap->alloc_count--; memset(ptr, 0, data_len); Maybe data_len is not correct either because of bug, or your application cl= obbered the malloc reserved regions in the element. More likely, gcc is incorrectly optimizing this away. https://wiki.sei.cmu.edu/confluence/display/c/MSC06-C.+Beware+of+compiler+o= ptimizations https://www.cryptologie.net/article/419/zeroing-memory-compiler-optimizatio= ns-and-memset_s/