DPDK usage discussions
 help / color / Atom feed
* [dpdk-users] Should ''shmget" not be used to consume hugepages in DPDK?
@ 2019-12-19 16:23 Byonggon Chun
  2019-12-19 21:06 ` Stephen Hemminger
  0 siblings, 1 reply; 4+ messages in thread
From: Byonggon Chun @ 2019-12-19 16:23 UTC (permalink / raw)
  To: users

Hi all.
I'm Kubernetes contributors and I'm working to make container isolation of
hugepages that allows us to set hugepages limit per container cgroup.
(At this point, limits are set on pod level cgroup even though we asked
hugepages as the container level resource)

I tore down testPMD and some parts of DPDK lib and I got a question after i
found there is no usage of "shmget" in DPDK.

My question is that Should "shmget" not be used to consume hugepages in
DPDK?
And here is following questions:
1) If we don't have to use "shmget", Why? Does it affect performance?
2) If I use "shmget" to get hugepages, should I call "mlock" syscall for it?

For more details, as I know there are three ways to consume hugepages in
kubernetes.
1) shmget with SHM_HUGETLB
2) mmap with hugetlbs filebacking
3) mmap with MAP_ANONYMOUS | MAP_HUGETLB

And I found that testPMD calls mlock syscall when it maps an anonymous
hugepages or external allocated hugepages.
https://github.com/DPDK/dpdk/blob/924e55fb340623f03fdf2ff7fbcfd78819d1db25/app/test-pmd/testpmd.c#L896
https://github.com/DPDK/dpdk/blob/924e55fb340623f03fdf2ff7fbcfd78819d1db25/app/test-pmd/testpmd.c#L916

Thanks.

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dpdk-users] Should ''shmget" not be used to consume hugepages in DPDK?
  2019-12-19 16:23 [dpdk-users] Should ''shmget" not be used to consume hugepages in DPDK? Byonggon Chun
@ 2019-12-19 21:06 ` Stephen Hemminger
  2019-12-20 12:42   ` Byonggon Chun
  0 siblings, 1 reply; 4+ messages in thread
From: Stephen Hemminger @ 2019-12-19 21:06 UTC (permalink / raw)
  To: Byonggon Chun; +Cc: users

On Fri, 20 Dec 2019 01:23:50 +0900
Byonggon Chun <byonggonchun@gmail.com> wrote:

> Hi all.
> I'm Kubernetes contributors and I'm working to make container isolation of
> hugepages that allows us to set hugepages limit per container cgroup.
> (At this point, limits are set on pod level cgroup even though we asked
> hugepages as the container level resource)
> 
> I tore down testPMD and some parts of DPDK lib and I got a question after i
> found there is no usage of "shmget" in DPDK.
> 
> My question is that Should "shmget" not be used to consume hugepages in
> DPDK?
> And here is following questions:
> 1) If we don't have to use "shmget", Why? Does it affect performance?
> 2) If I use "shmget" to get hugepages, should I call "mlock" syscall for it?
> 
> For more details, as I know there are three ways to consume hugepages in
> kubernetes.
> 1) shmget with SHM_HUGETLB
> 2) mmap with hugetlbs filebacking
> 3) mmap with MAP_ANONYMOUS | MAP_HUGETLB
> 
> And I found that testPMD calls mlock syscall when it maps an anonymous
> hugepages or external allocated hugepages.
> https://github.com/DPDK/dpdk/blob/924e55fb340623f03fdf2ff7fbcfd78819d1db25/app/test-pmd/testpmd.c#L896
> https://github.com/DPDK/dpdk/blob/924e55fb340623f03fdf2ff7fbcfd78819d1db25/app/test-pmd/testpmd.c#L916
> 
> Thanks.

shmget is a legacy Unix API and there is no point in using it.
For new applications libhugetlbfs is preferable. 

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dpdk-users] Should ''shmget" not be used to consume hugepages in DPDK?
  2019-12-19 21:06 ` Stephen Hemminger
@ 2019-12-20 12:42   ` Byonggon Chun
  2019-12-22 15:33     ` Byonggon Chun
  0 siblings, 1 reply; 4+ messages in thread
From: Byonggon Chun @ 2019-12-20 12:42 UTC (permalink / raw)
  To: Stephen Hemminger; +Cc: users

> shmget is a legacy Unix API and there is no point in using it.

Yeah, I agree with it,
I also prefer to use mmap with hugetlbfs in a DPDK container.

The reason why I started this mail thread is some DPDK users still use
shmget to consume hugepages, and I just wanted to find a good rationale to
convince them to use mmap.

But, at this point, I have only one rationale : shmget is a legacy UINIX
API.

On Fri, Dec 20, 2019 at 6:06 AM Stephen Hemminger <
stephen@networkplumber.org> wrote:

> On Fri, 20 Dec 2019 01:23:50 +0900
> Byonggon Chun <byonggonchun@gmail.com> wrote:
>
> > Hi all.
> > I'm Kubernetes contributors and I'm working to make container isolation
> of
> > hugepages that allows us to set hugepages limit per container cgroup.
> > (At this point, limits are set on pod level cgroup even though we asked
> > hugepages as the container level resource)
> >
> > I tore down testPMD and some parts of DPDK lib and I got a question
> after i
> > found there is no usage of "shmget" in DPDK.
> >
> > My question is that Should "shmget" not be used to consume hugepages in
> > DPDK?
> > And here is following questions:
> > 1) If we don't have to use "shmget", Why? Does it affect performance?
> > 2) If I use "shmget" to get hugepages, should I call "mlock" syscall for
> it?
> >
> > For more details, as I know there are three ways to consume hugepages in
> > kubernetes.
> > 1) shmget with SHM_HUGETLB
> > 2) mmap with hugetlbs filebacking
> > 3) mmap with MAP_ANONYMOUS | MAP_HUGETLB
> >
> > And I found that testPMD calls mlock syscall when it maps an anonymous
> > hugepages or external allocated hugepages.
> >
> https://github.com/DPDK/dpdk/blob/924e55fb340623f03fdf2ff7fbcfd78819d1db25/app/test-pmd/testpmd.c#L896
> >
> https://github.com/DPDK/dpdk/blob/924e55fb340623f03fdf2ff7fbcfd78819d1db25/app/test-pmd/testpmd.c#L916
> >
> > Thanks.
>
> shmget is a legacy Unix API and there is no point in using it.
> For new applications libhugetlbfs is preferable.
>

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dpdk-users] Should ''shmget" not be used to consume hugepages in DPDK?
  2019-12-20 12:42   ` Byonggon Chun
@ 2019-12-22 15:33     ` Byonggon Chun
  0 siblings, 0 replies; 4+ messages in thread
From: Byonggon Chun @ 2019-12-22 15:33 UTC (permalink / raw)
  To: dev, users

x-posting to dev mailing list.

Hi all.
I'm Kubernetes contributors and I'm working to make container isolation of
hugepages that allows us to set hugepages limit per container cgroup.
(At this point, limits are set on pod level cgroup even though we asked
hugepages as the container level resource)

I tore down testPMD and some parts of DPDK lib and I got a question after i
found there is no usage of "shmget" in DPDK.

My question is that Should "shmget" not be used to consume hugepages in
DPDK?
And here is following questions:
1) If we don't have to use "shmget", Why? Does it affect performance?
2) If I use "shmget" to get hugepages, should I call "mlock" syscall for it?

For more details, as I know there are three ways to consume hugepages in
kubernetes.
1) shmget with SHM_HUGETLB
2) mmap with hugetlbs filebacking
3) mmap with MAP_ANONYMOUS | MAP_HUGETLB

And I found that testPMD calls mlock syscall when it maps an anonymous
hugepages or external allocated
hugepages.https://github.com/DPDK/dpdk/blob/924e55fb340623f03fdf2ff7fbcfd78819d1db25/app/test-pmd/testpmd.c#L896https://github.com/DPDK/dpdk/blob/924e55fb340623f03fdf2ff7fbcfd78819d1db25/app/test-pmd/testpmd.c#L916

Thanks.





On Fri, Dec 20, 2019 at 9:42 PM Byonggon Chun <byonggonchun@gmail.com>
wrote:

> > shmget is a legacy Unix API and there is no point in using it.
>
> Yeah, I agree with it,
> I also prefer to use mmap with hugetlbfs in a DPDK container.
>
> The reason why I started this mail thread is some DPDK users still use
> shmget to consume hugepages, and I just wanted to find a good rationale to
> convince them to use mmap.
>
> But, at this point, I have only one rationale : shmget is a legacy UINIX
> API.
>
> On Fri, Dec 20, 2019 at 6:06 AM Stephen Hemminger <
> stephen@networkplumber.org> wrote:
>
>> On Fri, 20 Dec 2019 01:23:50 +0900
>> Byonggon Chun <byonggonchun@gmail.com> wrote:
>>
>> > Hi all.
>> > I'm Kubernetes contributors and I'm working to make container isolation
>> of
>> > hugepages that allows us to set hugepages limit per container cgroup.
>> > (At this point, limits are set on pod level cgroup even though we asked
>> > hugepages as the container level resource)
>> >
>> > I tore down testPMD and some parts of DPDK lib and I got a question
>> after i
>> > found there is no usage of "shmget" in DPDK.
>> >
>> > My question is that Should "shmget" not be used to consume hugepages in
>> > DPDK?
>> > And here is following questions:
>> > 1) If we don't have to use "shmget", Why? Does it affect performance?
>> > 2) If I use "shmget" to get hugepages, should I call "mlock" syscall
>> for it?
>> >
>> > For more details, as I know there are three ways to consume hugepages in
>> > kubernetes.
>> > 1) shmget with SHM_HUGETLB
>> > 2) mmap with hugetlbs filebacking
>> > 3) mmap with MAP_ANONYMOUS | MAP_HUGETLB
>> >
>> > And I found that testPMD calls mlock syscall when it maps an anonymous
>> > hugepages or external allocated hugepages.
>> >
>> https://github.com/DPDK/dpdk/blob/924e55fb340623f03fdf2ff7fbcfd78819d1db25/app/test-pmd/testpmd.c#L896
>> >
>> https://github.com/DPDK/dpdk/blob/924e55fb340623f03fdf2ff7fbcfd78819d1db25/app/test-pmd/testpmd.c#L916
>> >
>> > Thanks.
>>
>> shmget is a legacy Unix API and there is no point in using it.
>> For new applications libhugetlbfs is preferable.
>>
>

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, back to index

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-12-19 16:23 [dpdk-users] Should ''shmget" not be used to consume hugepages in DPDK? Byonggon Chun
2019-12-19 21:06 ` Stephen Hemminger
2019-12-20 12:42   ` Byonggon Chun
2019-12-22 15:33     ` Byonggon Chun

DPDK usage discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/users/0 users/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 users users/ http://inbox.dpdk.org/users \
		users@dpdk.org
	public-inbox-index users


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.users


AGPL code for this site: git clone https://public-inbox.org/ public-inbox