DPDK patches and discussions
 help / color / Atom feed
* [dpdk-dev] Minimun value of RTE_MAX_LCORE
@ 2020-10-15 10:49 Juraj Linkeš
  2020-10-15 11:15 ` Bruce Richardson
  2020-10-15 11:30 ` Gaëtan Rivet
  0 siblings, 2 replies; 5+ messages in thread
From: Juraj Linkeš @ 2020-10-15 10:49 UTC (permalink / raw)
  To: dev

Hi dpdk devs,

Is there a constraint on how low RTE_MAX_LCORE can be? I'm implementing a discovery mechanism that sets RTE_MAX_LCORE according to the number of host cores, but I'm hitting errors when the values are low:
https://travis-ci.com/github/jlinkes/dpdk/jobs/399596828
Message: Found 2
cores
Message: Found 1
numa nodes

../app/test/test_rcu_qsbr.c:296:54: error: iteration 2 invokes undefined behavior [-Werror=aggressive-loop-optimizations]

../app/test/test_rcu_qsbr.c:315:55: error: array subscript is above array bounds [-Werror=array-bounds]

All VM jobs failed in that Travis build. Travis VMs only have 2 cores, so I tried to put a bound on the build. I set it to 4 and all jobs except GCC shared lib jobs passed, which still threw iteration 4 invokes undefined behavior error:
https://travis-ci.com/github/jlinkes/dpdk/jobs/400004089

../examples/performance-thread/l3fwd-thread/main.c:2338:34: error: iteration 4 invokes undefined behavior [-Werror=aggressive-loop-optimizations]

This happens for number of cores < 32 and looks like a limitation unique to l3fwd (with cores between 4 and 32 - I didn't see the error elsewhere).

Should I use the bound or are these legitimate errors? The fact that only GCC (and not clang) shared lib jobs failed is also suspicious.

Thanks,
Juraj


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-dev] Minimun value of RTE_MAX_LCORE
  2020-10-15 10:49 [dpdk-dev] Minimun value of RTE_MAX_LCORE Juraj Linkeš
@ 2020-10-15 11:15 ` Bruce Richardson
  2020-10-15 11:30 ` Gaëtan Rivet
  1 sibling, 0 replies; 5+ messages in thread
From: Bruce Richardson @ 2020-10-15 11:15 UTC (permalink / raw)
  To: Juraj Linkeš; +Cc: dev

On Thu, Oct 15, 2020 at 10:49:17AM +0000, Juraj Linkeš wrote:
> Hi dpdk devs,
> 
> Is there a constraint on how low RTE_MAX_LCORE can be? I'm implementing a
> discovery mechanism that sets RTE_MAX_LCORE according to the number of
> host cores, but I'm hitting errors when the values are low:
> https://travis-ci.com/github/jlinkes/dpdk/jobs/399596828 Message: Found 2
> cores Message: Found 1 numa nodes
> 
> ../app/test/test_rcu_qsbr.c:296:54: error: iteration 2 invokes undefined
> behavior [-Werror=aggressive-loop-optimizations]
> 
> ../app/test/test_rcu_qsbr.c:315:55: error: array subscript is above array
> bounds [-Werror=array-bounds]
> 
> All VM jobs failed in that Travis build. Travis VMs only have 2 cores, so
> I tried to put a bound on the build. I set it to 4 and all jobs except
> GCC shared lib jobs passed, which still threw iteration 4 invokes
> undefined behavior error:
> https://travis-ci.com/github/jlinkes/dpdk/jobs/400004089
> 
> ../examples/performance-thread/l3fwd-thread/main.c:2338:34: error:
> iteration 4 invokes undefined behavior
> [-Werror=aggressive-loop-optimizations]
> 
> This happens for number of cores < 32 and looks like a limitation unique
> to l3fwd (with cores between 4 and 32 - I didn't see the error
> elsewhere).
> 
> Should I use the bound or are these legitimate errors? The fact that only
> GCC (and not clang) shared lib jobs failed is also suspicious.
>
Four seems a reasonable lower bound to have, but 32 seems high. I agree
that the warning itself looks suspicious too.

However, looking at the code and the line in question, I think there is
indeed an app bug.  The loop goes up to MAX_ETHPORTS, but the tx_bufs array
being dereferenced is dimensioned by MAX_LCORES, which is why you hit
problems with lcore values below 32 (==MAX_ETHPORTS). Based on the use, I
suspect the structure definition may be wrong and the array should be
dimensioned for MAX_ETHPORTS - but I haven't dug into the code further than
that.

Regards,
/Bruce

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-dev] Minimun value of RTE_MAX_LCORE
  2020-10-15 10:49 [dpdk-dev] Minimun value of RTE_MAX_LCORE Juraj Linkeš
  2020-10-15 11:15 ` Bruce Richardson
@ 2020-10-15 11:30 ` Gaëtan Rivet
  2020-10-15 12:14   ` Juraj Linkeš
  1 sibling, 1 reply; 5+ messages in thread
From: Gaëtan Rivet @ 2020-10-15 11:30 UTC (permalink / raw)
  To: Juraj Linkeš; +Cc: dev

On 15/10/20 10:49 +0000, Juraj Linkeš wrote:
> Hi dpdk devs,
> 
> Is there a constraint on how low RTE_MAX_LCORE can be? I'm implementing a discovery mechanism that sets RTE_MAX_LCORE according to the number of host cores, but I'm hitting errors when the values are low:
> https://travis-ci.com/github/jlinkes/dpdk/jobs/399596828
> Message: Found 2
> cores
> Message: Found 1
> numa nodes
> 
> ../app/test/test_rcu_qsbr.c:296:54: error: iteration 2 invokes undefined behavior [-Werror=aggressive-loop-optimizations]
> 
> ../app/test/test_rcu_qsbr.c:315:55: error: array subscript is above array bounds [-Werror=array-bounds]
> 
> All VM jobs failed in that Travis build. Travis VMs only have 2 cores, so I tried to put a bound on the build. I set it to 4 and all jobs except GCC shared lib jobs passed, which still threw iteration 4 invokes undefined behavior error:
> https://travis-ci.com/github/jlinkes/dpdk/jobs/400004089
> 
> ../examples/performance-thread/l3fwd-thread/main.c:2338:34: error: iteration 4 invokes undefined behavior [-Werror=aggressive-loop-optimizations]
> 
> This happens for number of cores < 32 and looks like a limitation unique to l3fwd (with cores between 4 and 32 - I didn't see the error elsewhere).
> 
> Should I use the bound or are these legitimate errors? The fact that only GCC (and not clang) shared lib jobs failed is also suspicious.
> 
> Thanks,
> Juraj
> 

Hi,

I can see a CPU config setting it to 4, so it might be a valid value.
Not sure it would be the lower bound though.

However, I think the issue you get here shows why your discovery
mechanism is not great.  Most of the time, DPDK applications are not
built on their target machine: either due to CI (like your issue),
automatic packaging, cross-compilation for smartNIC, etc.

Platforms that would benefit from your discovery mechanism will define
RTE_MAX_LCORE explicitly, e.g. in config/arm/meson.build, line 34 and
further.

Regards,
-- 
Gaëtan

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-dev] Minimun value of RTE_MAX_LCORE
  2020-10-15 11:30 ` Gaëtan Rivet
@ 2020-10-15 12:14   ` Juraj Linkeš
  2020-10-16  6:06     ` Honnappa Nagarahalli
  0 siblings, 1 reply; 5+ messages in thread
From: Juraj Linkeš @ 2020-10-15 12:14 UTC (permalink / raw)
  To: Gaëtan Rivet; +Cc: dev



> -----Original Message-----
> From: Gaëtan Rivet <grive@u256.net>
> Sent: Thursday, October 15, 2020 1:31 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] Minimun value of RTE_MAX_LCORE
> 
> On 15/10/20 10:49 +0000, Juraj Linkeš wrote:
> > Hi dpdk devs,
> >
> > Is there a constraint on how low RTE_MAX_LCORE can be? I'm implementing a
> discovery mechanism that sets RTE_MAX_LCORE according to the number of
> host cores, but I'm hitting errors when the values are low:
> > https://travis-ci.com/github/jlinkes/dpdk/jobs/399596828
> > Message: Found 2
> > cores
> > Message: Found 1
> > numa nodes
> >
> > ../app/test/test_rcu_qsbr.c:296:54: error: iteration 2 invokes
> > undefined behavior [-Werror=aggressive-loop-optimizations]
> >
> > ../app/test/test_rcu_qsbr.c:315:55: error: array subscript is above
> > array bounds [-Werror=array-bounds]
> >
> > All VM jobs failed in that Travis build. Travis VMs only have 2 cores, so I tried to
> put a bound on the build. I set it to 4 and all jobs except GCC shared lib jobs
> passed, which still threw iteration 4 invokes undefined behavior error:
> > https://travis-ci.com/github/jlinkes/dpdk/jobs/400004089
> >
> > ../examples/performance-thread/l3fwd-thread/main.c:2338:34: error:
> > iteration 4 invokes undefined behavior
> > [-Werror=aggressive-loop-optimizations]
> >
> > This happens for number of cores < 32 and looks like a limitation unique to
> l3fwd (with cores between 4 and 32 - I didn't see the error elsewhere).
> >
> > Should I use the bound or are these legitimate errors? The fact that only GCC
> (and not clang) shared lib jobs failed is also suspicious.
> >
> > Thanks,
> > Juraj
> >
> 
> Hi,
> 
> I can see a CPU config setting it to 4, so it might be a valid value.
> Not sure it would be the lower bound though.
> 
> However, I think the issue you get here shows why your discovery mechanism is
> not great.  Most of the time, DPDK applications are not built on their target
> machine: either due to CI (like your issue), automatic packaging, cross-
> compilation for smartNIC, etc.
> 

It's not supposed to be great, just be a more sensible default for native builds. Users can still specify the cores on the command line if they like. Or do the default build (which will use predefined values).

> Platforms that would benefit from your discovery mechanism will define
> RTE_MAX_LCORE explicitly, e.g. in config/arm/meson.build, line 34 and further.
> 

I'm not sure what you mean. If they define it explicitly, then they can't benefit from the discoreved values. What we're exploring is using the discovered values for native builds and moving the statically defined RTE_MAX_LCORE and RTE_MAX_NUMA_NODES to cross files. Then we'll have native builds which better match the build machine and if anyone wants the target to be a particular SoC, they can use a cross file.

> Regards,
> --
> Gaëtan


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-dev] Minimun value of RTE_MAX_LCORE
  2020-10-15 12:14   ` Juraj Linkeš
@ 2020-10-16  6:06     ` Honnappa Nagarahalli
  0 siblings, 0 replies; 5+ messages in thread
From: Honnappa Nagarahalli @ 2020-10-16  6:06 UTC (permalink / raw)
  To: Juraj Linkeš, Gaëtan Rivet; +Cc: dev, nd, Honnappa Nagarahalli, nd

<snip>

> >
> > On 15/10/20 10:49 +0000, Juraj Linkeš wrote:
> > > Hi dpdk devs,
> > >
> > > Is there a constraint on how low RTE_MAX_LCORE can be? I'm
> > > implementing a
> > discovery mechanism that sets RTE_MAX_LCORE according to the number
> of
> > host cores, but I'm hitting errors when the values are low:
> > > https://travis-ci.com/github/jlinkes/dpdk/jobs/399596828
> > > Message: Found 2
> > > cores
> > > Message: Found 1
> > > numa nodes
> > >
> > > ../app/test/test_rcu_qsbr.c:296:54: error: iteration 2 invokes
> > > undefined behavior [-Werror=aggressive-loop-optimizations]
> > >
> > > ../app/test/test_rcu_qsbr.c:315:55: error: array subscript is above
> > > array bounds [-Werror=array-bounds]
> > >
> > > All VM jobs failed in that Travis build. Travis VMs only have 2
> > > cores, so I tried to
> > put a bound on the build. I set it to 4 and all jobs except GCC shared
> > lib jobs passed, which still threw iteration 4 invokes undefined behavior
> error:
> > > https://travis-ci.com/github/jlinkes/dpdk/jobs/400004089
> > >
> > > ../examples/performance-thread/l3fwd-thread/main.c:2338:34: error:
> > > iteration 4 invokes undefined behavior
> > > [-Werror=aggressive-loop-optimizations]
> > >
> > > This happens for number of cores < 32 and looks like a limitation
> > > unique to
> > l3fwd (with cores between 4 and 32 - I didn't see the error elsewhere).
> > >
> > > Should I use the bound or are these legitimate errors? The fact that
> > > only GCC
> > (and not clang) shared lib jobs failed is also suspicious.
2 would be the minimum (one for main and one for worker), not less than that. Please check [1], I have fixed both of them.

[1] https://patches.dpdk.org/patch/81028/

> > >
> > > Thanks,
> > > Juraj
> > >
> >
> > Hi,
> >
> > I can see a CPU config setting it to 4, so it might be a valid value.
> > Not sure it would be the lower bound though.
> >
> > However, I think the issue you get here shows why your discovery
> > mechanism is not great.  Most of the time, DPDK applications are not
> > built on their target
> > machine: either due to CI (like your issue), automatic packaging,
> > cross- compilation for smartNIC, etc.
> >
> 
> It's not supposed to be great, just be a more sensible default for native
> builds. Users can still specify the cores on the command line if they like. Or do
> the default build (which will use predefined values).
> 
> > Platforms that would benefit from your discovery mechanism will define
> > RTE_MAX_LCORE explicitly, e.g. in config/arm/meson.build, line 34 and
> further.
> >
> 
> I'm not sure what you mean. If they define it explicitly, then they can't
> benefit from the discoreved values. What we're exploring is using the
> discovered values for native builds and moving the statically defined
> RTE_MAX_LCORE and RTE_MAX_NUMA_NODES to cross files. Then we'll
> have native builds which better match the build machine and if anyone wants
> the target to be a particular SoC, they can use a cross file.
Agree. I think there is enough flexibility provided for different use cases.

> 
> > Regards,
> > --
> > Gaëtan


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, back to index

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-15 10:49 [dpdk-dev] Minimun value of RTE_MAX_LCORE Juraj Linkeš
2020-10-15 11:15 ` Bruce Richardson
2020-10-15 11:30 ` Gaëtan Rivet
2020-10-15 12:14   ` Juraj Linkeš
2020-10-16  6:06     ` Honnappa Nagarahalli

DPDK patches and discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ http://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/ public-inbox