[PATCH] bitops: fix issue in parallel atomic tests

[PATCH] bitops: fix issue in parallel atomic tests

[Mattias Rönnblom]

The macros generating the parallel test for atomic test-and-
[set|clear|flip] functions used a 64-bit reference word when assuring
no neighbouring bits were modified, even when generating code for the
32-bit version of the test.

This issue causes spurious test failures on GCC 12.2.0 (the default
compiler on for example Debian 12 "bookworm"), when optimization level
2 or higher are used.

The test failures do not occur with GCC 11, 12.3 and 13.2.

To the author, this looks like a promotion-related compiler bug in GCC
12.2.

Fixes: 35326b61aecb ("bitops: add atomic bit operations in new API")

Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
---
 app/test/test_bitops.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/app/test/test_bitops.c b/app/test/test_bitops.c
index 4200073ae4..681e984037 100644
--- a/app/test/test_bitops.c
+++ b/app/test/test_bitops.c
@@ -227,7 +227,7 @@ test_bit_atomic_parallel_test_and_modify ## size(void) \
 	bool expected_value = total_flips % 2; \
 	TEST_ASSERT(expected_value == rte_bit_test(&word, bit), \
 		"After %"PRId64" flips, the bit value should be %d", total_flips, expected_value); \
-	uint64_t expected_word = 0; \
+	uint ## size ## _t expected_word = 0; \
 	rte_bit_assign(&expected_word, bit, expected_value); \
 	TEST_ASSERT(expected_word == word, "Untouched bits have changed value"); \
 	return TEST_SUCCESS; \
@@ -275,7 +275,7 @@ test_bit_atomic_parallel_flip ## size(void) \
 	bool expected_value = total_flips % 2; \
 	TEST_ASSERT(expected_value == rte_bit_test(&word, bit), \
 		"After %"PRId64" flips, the bit value should be %d", total_flips, expected_value); \
-	uint64_t expected_word = 0; \
+	uint ## size ## _t expected_word = 0; \
 	rte_bit_assign(&expected_word, bit, expected_value); \
 	TEST_ASSERT(expected_word == word, "Untouched bits have changed value"); \
 	return TEST_SUCCESS; \
-- 
2.43.0

RE: [PATCH] bitops: fix issue in parallel atomic tests

[Morten Brørup]

> From: Mattias Rönnblom [mailto:mattias.ronnblom@ericsson.com]
> Sent: Sunday, 13 October 2024 13.57
> 
> The macros generating the parallel test for atomic test-and-
> [set|clear|flip] functions used a 64-bit reference word when assuring
> no neighbouring bits were modified, even when generating code for the
> 32-bit version of the test.
> 
> This issue causes spurious test failures on GCC 12.2.0 (the default
> compiler on for example Debian 12 "bookworm"), when optimization level
> 2 or higher are used.
> 
> The test failures do not occur with GCC 11, 12.3 and 13.2.
> 
> To the author, this looks like a promotion-related compiler bug in GCC
> 12.2.

I am curious about the compiler bug...

Did the bug occur when the most significant bit was set, so it sign related?

Maybe this will reveal something:

TEST_ASSERT(expected_word == word,
  "Untouched bits have changed value, %" PRIx ## size
  " should be %" PRIx64,
  word, expected_word);

> 
> Fixes: 35326b61aecb ("bitops: add atomic bit operations in new API")
> 
> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> ---

I took a deep look into this.

Regardless of what the compiler bug is,

Reviewed-by: Morten Brørup <mb@smartsharesystems.com>

Re: [PATCH] bitops: fix issue in parallel atomic tests

[Mattias Rönnblom]

On 2024-10-13 15:37, Morten Brørup wrote:
>> From: Mattias Rönnblom [mailto:mattias.ronnblom@ericsson.com]
>> Sent: Sunday, 13 October 2024 13.57
>>
>> The macros generating the parallel test for atomic test-and-
>> [set|clear|flip] functions used a 64-bit reference word when assuring
>> no neighbouring bits were modified, even when generating code for the
>> 32-bit version of the test.
>>
>> This issue causes spurious test failures on GCC 12.2.0 (the default
>> compiler on for example Debian 12 "bookworm"), when optimization level
>> 2 or higher are used.
>>
>> The test failures do not occur with GCC 11, 12.3 and 13.2.
>>
>> To the author, this looks like a promotion-related compiler bug in GCC
>> 12.2.
> 
> I am curious about the compiler bug...
> 
> Did the bug occur when the most significant bit was set, so it sign related?
> 

It seems to happen a lot more often than 1/32 times. Also, all involved 
types are unsigned.

If you set the optimization level to "1" (i.e., 
__attribute__((optimize("O"))) on the 
test_bit_atomic_parallel_test_and_modify32 function, the test passes on 
12.2.0.

> Maybe this will reveal something:
> 
> TEST_ASSERT(expected_word == word,
>    "Untouched bits have changed value, %" PRIx ## size
>    " should be %" PRIx64,
>    word, expected_word);
> 

Confusingly enough, the failing assertion is the one prior that assertion.

>>
>> Fixes: 35326b61aecb ("bitops: add atomic bit operations in new API")
>>
>> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
>> ---
> 
> I took a deep look into this.
> 
> Regardless of what the compiler bug is,
> 
> Reviewed-by: Morten Brørup <mb@smartsharesystems.com>
> 

Thanks.

I'm far from sure it's a compiler bug. Just look at the base rate: how 
often does the code you just wrote fail because of a bug in your code, 
and how often is the root cause to be found in the compiler or the 
standard libraries.

RE: [PATCH] bitops: fix issue in parallel atomic tests

[Morten Brørup]

> From: Mattias Rönnblom [mailto:hofors@lysator.liu.se]
> Sent: Sunday, 13 October 2024 17.20
> 
> On 2024-10-13 15:37, Morten Brørup wrote:
> >> From: Mattias Rönnblom [mailto:mattias.ronnblom@ericsson.com]
> >> Sent: Sunday, 13 October 2024 13.57
> >>
> >> The macros generating the parallel test for atomic test-and-
> >> [set|clear|flip] functions used a 64-bit reference word when
> assuring
> >> no neighbouring bits were modified, even when generating code for
> the
> >> 32-bit version of the test.
> >>
> >> This issue causes spurious test failures on GCC 12.2.0 (the default
> >> compiler on for example Debian 12 "bookworm"), when optimization
> level
> >> 2 or higher are used.
> >>
> >> The test failures do not occur with GCC 11, 12.3 and 13.2.
> >>
> >> To the author, this looks like a promotion-related compiler bug in
> GCC
> >> 12.2.
> >
> > I am curious about the compiler bug...
> >
> > Did the bug occur when the most significant bit was set, so it sign
> related?
> >
> 
> It seems to happen a lot more often than 1/32 times. Also, all involved
> types are unsigned.

OK. I was speculating that the compiler bug might be treating an unsigned as a signed, and somehow sign extending the most significant bit of a negative value into the higher bits when converting the type to a bigger type.

> 
> If you set the optimization level to "1" (i.e.,
> __attribute__((optimize("O"))) on the
> test_bit_atomic_parallel_test_and_modify32 function, the test passes on
> 12.2.0.
> 
> > Maybe this will reveal something:
> >
> > TEST_ASSERT(expected_word == word,
> >    "Untouched bits have changed value, %" PRIx ## size
> >    " should be %" PRIx64,
> >    word, expected_word);
> >
> 
> Confusingly enough, the failing assertion is the one prior that
> assertion.

Ahh... I misread your "promotion" suspicion as "type promotion", not instruction reordering.

> 
> >>
> >> Fixes: 35326b61aecb ("bitops: add atomic bit operations in new API")
> >>
> >> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> >> ---
> >
> > I took a deep look into this.
> >
> > Regardless of what the compiler bug is,
> >
> > Reviewed-by: Morten Brørup <mb@smartsharesystems.com>
> >
> 
> Thanks.
> 
> I'm far from sure it's a compiler bug. Just look at the base rate: how
> often does the code you just wrote fail because of a bug in your code,
> and how often is the root cause to be found in the compiler or the
> standard libraries.
> 

A strong argument for rootcausing exactly what the specific compiler gets wrong when compiling the code triggering the error.

Re: [PATCH] bitops: fix issue in parallel atomic tests

[David Marchand]

On Sun, Oct 13, 2024 at 2:06 PM Mattias Rönnblom
<mattias.ronnblom@ericsson.com> wrote:
>
> The macros generating the parallel test for atomic test-and-
> [set|clear|flip] functions used a 64-bit reference word when assuring
> no neighbouring bits were modified, even when generating code for the
> 32-bit version of the test.
>
> This issue causes spurious test failures on GCC 12.2.0 (the default
> compiler on for example Debian 12 "bookworm"), when optimization level
> 2 or higher are used.
>
> The test failures do not occur with GCC 11, 12.3 and 13.2.
>
> To the author, this looks like a promotion-related compiler bug in GCC
> 12.2.
>
> Fixes: 35326b61aecb ("bitops: add atomic bit operations in new API")
>
> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
> Reviewed-by: Morten Brørup <mb@smartsharesystems.com>

Tested-by: David Marchand <david.marchand@redhat.com>

Applied to fix the CI, thanks.


-- 
David Marchand

Re: [PATCH] bitops: fix issue in parallel atomic tests

[David Marchand]

On Sun, Oct 13, 2024 at 5:37 PM Morten Brørup <mb@smartsharesystems.com> wrote:
> > I'm far from sure it's a compiler bug. Just look at the base rate: how
> > often does the code you just wrote fail because of a bug in your code,
> > and how often is the root cause to be found in the compiler or the
> > standard libraries.
> >
>
> A strong argument for rootcausing exactly what the specific compiler gets wrong when compiling the code triggering the error.
>

I agree that root causing it is preferrable.
I merged this change in any case to resolve the false positives in CI.


-- 
David Marchand

Re: [PATCH] bitops: fix issue in parallel atomic tests

[Mattias Rönnblom]

On 2024-10-14 16:16, David Marchand wrote:
> On Sun, Oct 13, 2024 at 5:37 PM Morten Brørup <mb@smartsharesystems.com> wrote:
>>> I'm far from sure it's a compiler bug. Just look at the base rate: how
>>> often does the code you just wrote fail because of a bug in your code,
>>> and how often is the root cause to be found in the compiler or the
>>> standard libraries.
>>>
>>
>> A strong argument for rootcausing exactly what the specific compiler gets wrong when compiling the code triggering the error.
>>
> 
> I agree that root causing it is preferrable.
> I merged this change in any case to resolve the false positives in CI.
> 
> 

The change is reworking some pretty awkward, potentially incorrect (if 
GCC 12.2 is right and I am wrong) code. So it should be merged in any 
case. The reason the reference word was always 64 bit was just an 
oversight on my part.