[PATCH] cmdline: configure input buffer size

[PATCH] cmdline: configure input buffer size

[Gregory Etelson]

DPDK defines cmdline input buffer size to 512 characters.
That buffer size can be too small for long application input.
For example, the following flow template API testpmd command is 444
bytes long:
```
flow queue 0 create 0 template_table 1000 \
  pattern_template 0 actions_template 0 postpone no \
  pattern eth / ipv4 / udp / end \
  actions modify_field op set dst_type tag dst_level 0 dst_offset 0 \
    src_type value src_value 0x31 width 32 /  \
    modify_field op set dst_type ipv4_src src_type value \
    src_value 10101010 width 32 / modify_field op add dst_type \
    ipv4_ttl dst_level 0 dst_offset 0 src_type value \
    src_value ff width 8 / count / jump group 100 / end
```

The patch introduces conditional `RDLINE_CUSTOM_BUF_SIZE` definition.
Application can set custom cmdline size during DPDK configuration:

`meson setup ... -Dc_args='-DRDLINE_CUSTOM_BUF_SIZE=4096' ...`

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
---
 lib/cmdline/cmdline_private.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/lib/cmdline/cmdline_private.h b/lib/cmdline/cmdline_private.h
index b64f363903..7908d963f9 100644
--- a/lib/cmdline/cmdline_private.h
+++ b/lib/cmdline/cmdline_private.h
@@ -17,7 +17,11 @@
 
 #include <cmdline.h>
 
+#ifndef RDLINE_CUSTOM_BUF_SIZE
 #define RDLINE_BUF_SIZE 512
+#else
+#define RDLINE_BUF_SIZE RDLINE_CUSTOM_BUF_SIZE
+#endif
 #define RDLINE_PROMPT_SIZE  32
 #define RDLINE_VT100_BUF_SIZE  8
 #define RDLINE_HISTORY_BUF_SIZE BUFSIZ
-- 
2.43.0

Re: [PATCH] cmdline: configure input buffer size

[Bruce Richardson]

On Wed, May 01, 2024 at 08:26:59AM +0300, Gregory Etelson wrote:
> DPDK defines cmdline input buffer size to 512 characters.
> That buffer size can be too small for long application input.
> For example, the following flow template API testpmd command is 444
> bytes long:
> ```
> flow queue 0 create 0 template_table 1000 \
>   pattern_template 0 actions_template 0 postpone no \
>   pattern eth / ipv4 / udp / end \
>   actions modify_field op set dst_type tag dst_level 0 dst_offset 0 \
>     src_type value src_value 0x31 width 32 /  \
>     modify_field op set dst_type ipv4_src src_type value \
>     src_value 10101010 width 32 / modify_field op add dst_type \
>     ipv4_ttl dst_level 0 dst_offset 0 src_type value \
>     src_value ff width 8 / count / jump group 100 / end
> ```
> 
> The patch introduces conditional `RDLINE_CUSTOM_BUF_SIZE` definition.
> Application can set custom cmdline size during DPDK configuration:
> 
> `meson setup ... -Dc_args='-DRDLINE_CUSTOM_BUF_SIZE=4096' ...`
> 
> Signed-off-by: Gregory Etelson <getelson@nvidia.com>
> ---
Basic question, what would be the impact of increasing the default from 512
to 1k or 2k? Do we get a large memory footprint increase, or is it just an
extra 1 or 2k of memory used?

/Bruce

Re: [PATCH] cmdline: configure input buffer size

[Etelson, Gregory]

Hello Bruce,

>> Application can set custom cmdline size during DPDK configuration:
>>
>> `meson setup ... -Dc_args='-DRDLINE_CUSTOM_BUF_SIZE=4096' ...`
>>
>> Signed-off-by: Gregory Etelson <getelson@nvidia.com>
>> ---
> Basic question, what would be the impact of increasing the default from 512
> to 1k or 2k? Do we get a large memory footprint increase, or is it just an
> extra 1 or 2k of memory used?
>
> /Bruce
>

Increasing the RDLINE_BUF_SIZE size will also increase application memory usage.
However, that memory increase was required, because application could not fit 
it's command line into the default buffer.

Applications that can run with the default RDLINE_BUF_SIZE are not affected by 
that patch.

Regards,
Gregory

Re: [PATCH] cmdline: configure input buffer size

[Stephen Hemminger]

On Wed, 1 May 2024 13:06:47 +0300 (IDT)
"Etelson, Gregory" <getelson@nvidia.com> wrote:

> > Basic question, what would be the impact of increasing the default from 512
> > to 1k or 2k? Do we get a large memory footprint increase, or is it just an
> > extra 1 or 2k of memory used?
> >
> > /Bruce
> >  
> 
> Increasing the RDLINE_BUF_SIZE size will also increase application memory usage.
> However, that memory increase was required, because application could not fit 
> it's command line into the default buffer.
> 
> Applications that can run with the default RDLINE_BUF_SIZE are not affected by 
> that patch.
> 
> Regards,
> Gregory


The buffer is transient so should have little impact.
Why not just use LINE_MAX from limits.h?

Re: [PATCH] cmdline: configure input buffer size

[Etelson, Gregory]

Hello Stephen,

>>> Basic question, what would be the impact of increasing the default from 512
>>> to 1k or 2k? Do we get a large memory footprint increase, or is it just an
>>> extra 1 or 2k of memory used?
>>>
>>> /Bruce
>>>
>>
>> Increasing the RDLINE_BUF_SIZE size will also increase application memory usage.
>> However, that memory increase was required, because application could not fit
>> it's command line into the default buffer.
>>
>> Applications that can run with the default RDLINE_BUF_SIZE are not affected by
>> that patch.
>>
>> Regards,
>> Gregory
>
>
> The buffer is transient so should have little impact.
> Why not just use LINE_MAX from limits.h?
>

The LINE_MAX value will be enough even for the template API testpmd commands.
If there will be no objections, I'll set LINE_MAX as the new 
RDLINE_BUF_SIZE.

Regards,
Gregory

Re: [PATCH] cmdline: configure input buffer size

[Bruce Richardson]

On Wed, May 01, 2024 at 06:56:37PM +0300, Etelson, Gregory wrote:
> Hello Stephen,
> 
> > > > Basic question, what would be the impact of increasing the default from 512
> > > > to 1k or 2k? Do we get a large memory footprint increase, or is it just an
> > > > extra 1 or 2k of memory used?
> > > > 
> > > > /Bruce
> > > > 
> > > 
> > > Increasing the RDLINE_BUF_SIZE size will also increase application memory usage.
> > > However, that memory increase was required, because application could not fit
> > > it's command line into the default buffer.
> > > 
> > > Applications that can run with the default RDLINE_BUF_SIZE are not affected by
> > > that patch.
> > > 
> > > Regards,
> > > Gregory
> > 
> > 
> > The buffer is transient so should have little impact.
> > Why not just use LINE_MAX from limits.h?
> > 
> 
> The LINE_MAX value will be enough even for the template API testpmd commands.
> If there will be no objections, I'll set LINE_MAX as the new
> RDLINE_BUF_SIZE.
> 
+1 for just upping the size.
Thanks.

[PATCH v2] cmdline: increase input buffer size

[Gregory Etelson]

DPDK defines cmdline input buffer size to 512 characters.
That buffer size can be too small for long application input.
For example, the following flow template API testpmd command is 444
bytes long:
```
flow queue 0 create 0 template_table 1000 \
  pattern_template 0 actions_template 0 postpone no \
  pattern eth / ipv4 / udp / end \
  actions modify_field op set dst_type tag dst_level 0 dst_offset 0 \
    src_type value src_value 0x31 width 32 /  \
    modify_field op set dst_type ipv4_src src_type value \
    src_value 10101010 width 32 / modify_field op add dst_type \
    ipv4_ttl dst_level 0 dst_offset 0 src_type value \
    src_value ff width 8 / count / jump group 100 / end
```

The patch increases cmdline input buffer size to the LINE_MAX value.

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
---
v2: set RDLINE_BUF_SIZE to LINE_MAX
    change the patch subject
---
 lib/cmdline/cmdline_private.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/cmdline/cmdline_private.h b/lib/cmdline/cmdline_private.h
index b64f363903..3d0168ce03 100644
--- a/lib/cmdline/cmdline_private.h
+++ b/lib/cmdline/cmdline_private.h
@@ -17,7 +17,7 @@

 #include <cmdline.h>

-#define RDLINE_BUF_SIZE 512
+#define RDLINE_BUF_SIZE LINE_MAX
 #define RDLINE_PROMPT_SIZE  32
 #define RDLINE_VT100_BUF_SIZE  8
 #define RDLINE_HISTORY_BUF_SIZE BUFSIZ
--
2.43.0

RE: [PATCH v2] cmdline: increase input buffer size

[Morten Brørup]

+CC: Bruce, Stephen

> From: Gregory Etelson [mailto:getelson@nvidia.com]
> Sent: Wednesday, 1 May 2024 19.18
> 
> DPDK defines cmdline input buffer size to 512 characters.
> That buffer size can be too small for long application input.
> For example, the following flow template API testpmd command is 444
> bytes long:
> ```
> flow queue 0 create 0 template_table 1000 \
>   pattern_template 0 actions_template 0 postpone no \
>   pattern eth / ipv4 / udp / end \
>   actions modify_field op set dst_type tag dst_level 0 dst_offset 0 \
>     src_type value src_value 0x31 width 32 /  \
>     modify_field op set dst_type ipv4_src src_type value \
>     src_value 10101010 width 32 / modify_field op add dst_type \
>     ipv4_ttl dst_level 0 dst_offset 0 src_type value \
>     src_value ff width 8 / count / jump group 100 / end
> ```
> 
> The patch increases cmdline input buffer size to the LINE_MAX value.

+ ... , which typically is 2048.

> 
> Signed-off-by: Gregory Etelson <getelson@nvidia.com>
> ---

With or without suggested patch description change,
Reviewed-by: Morten Brørup <mb@smartsharesystems.com>

Re: [PATCH v2] cmdline: increase input buffer size

[Bruce Richardson]

On Wed, May 01, 2024 at 08:26:54PM +0200, Morten Brørup wrote:
> +CC: Bruce, Stephen
> 
> > From: Gregory Etelson [mailto:getelson@nvidia.com]
> > Sent: Wednesday, 1 May 2024 19.18
> > 
> > DPDK defines cmdline input buffer size to 512 characters.
> > That buffer size can be too small for long application input.
> > For example, the following flow template API testpmd command is 444
> > bytes long:
> > ```
> > flow queue 0 create 0 template_table 1000 \
> >   pattern_template 0 actions_template 0 postpone no \
> >   pattern eth / ipv4 / udp / end \
> >   actions modify_field op set dst_type tag dst_level 0 dst_offset 0 \
> >     src_type value src_value 0x31 width 32 /  \
> >     modify_field op set dst_type ipv4_src src_type value \
> >     src_value 10101010 width 32 / modify_field op add dst_type \
> >     ipv4_ttl dst_level 0 dst_offset 0 src_type value \
> >     src_value ff width 8 / count / jump group 100 / end
> > ```
> > 
> > The patch increases cmdline input buffer size to the LINE_MAX value.
> 
> + ... , which typically is 2048.
> 
> > 
> > Signed-off-by: Gregory Etelson <getelson@nvidia.com>
> > ---
> 
> With or without suggested patch description change,
> Reviewed-by: Morten Brørup <mb@smartsharesystems.com>
> 
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

[PATCH v3] cmdline: increase input buffer size

[Gregory Etelson]

DPDK defines cmdline input buffer size to 512 characters.
That buffer size can be too small for long application input.
For example, the following flow template API testpmd command is 444
bytes long:
```
flow queue 0 create 0 template_table 1000 \
  pattern_template 0 actions_template 0 postpone no \
  pattern eth / ipv4 / udp / end \
  actions modify_field op set dst_type tag dst_level 0 dst_offset 0 \
    src_type value src_value 0x31 width 32 /  \
    modify_field op set dst_type ipv4_src src_type value \
    src_value 10101010 width 32 / modify_field op add dst_type \
    ipv4_ttl dst_level 0 dst_offset 0 src_type value \
    src_value ff width 8 / count / jump group 100 / end
```

The patch increases cmdline input buffer size to the LINE_MAX value,
which typically is 2048 bytes.

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Morten Brørup <mb@smartsharesystems.com>
---
v2: set RDLINE_BUF_SIZE to LINE_MAX
    change the patch subject.
v3: fix Windows compilation error.
---
 lib/cmdline/cmdline_private.h | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/lib/cmdline/cmdline_private.h b/lib/cmdline/cmdline_private.h
index b64f363903..f00b025ecb 100644
--- a/lib/cmdline/cmdline_private.h
+++ b/lib/cmdline/cmdline_private.h
@@ -11,13 +11,20 @@
 #include <rte_os_shim.h>
 #ifdef RTE_EXEC_ENV_WINDOWS
 #include <rte_windows.h>
+#ifndef LINE_MAX
+/**
+ * The LINE_MAX value is derived from POSIX.
+ * Windows environment may not have POSIX definitions.
+ */
+#define LINE_MAX 2048
+#endif
 #else
 #include <termios.h>
 #endif
 
 #include <cmdline.h>
 
-#define RDLINE_BUF_SIZE 512
+#define RDLINE_BUF_SIZE LINE_MAX
 #define RDLINE_PROMPT_SIZE  32
 #define RDLINE_VT100_BUF_SIZE  8
 #define RDLINE_HISTORY_BUF_SIZE BUFSIZ
-- 
2.43.0

Re: [PATCH v3] cmdline: increase input buffer size

[Gregory Etelson]

[-- Attachment #1: Type: text/plain, Size: 2500 bytes --]

Hello,

What is the status of that patch ?

Regards,
Gregory
________________________________
From: Gregory Etelson <getelson@nvidia.com>
Sent: Friday, May 3, 2024 07:27
To: Gregory Etelson <getelson@nvidia.com>
Cc: Ali Alnubani <alialnu@nvidia.com>; dev@dpdk.org <dev@dpdk.org>; Maayan Kashani <mkashani@nvidia.com>; Raslan Darawsheh <rasland@nvidia.com>; bruce.richardson@intel.com <bruce.richardson@intel.com>; stephen@networkplumber.org <stephen@networkplumber.org>; mb@smartsharesystems.com <mb@smartsharesystems.com>
Subject: [PATCH v3] cmdline: increase input buffer size

DPDK defines cmdline input buffer size to 512 characters.
That buffer size can be too small for long application input.
For example, the following flow template API testpmd command is 444
bytes long:
```
flow queue 0 create 0 template_table 1000 \
  pattern_template 0 actions_template 0 postpone no \
  pattern eth / ipv4 / udp / end \
  actions modify_field op set dst_type tag dst_level 0 dst_offset 0 \
    src_type value src_value 0x31 width 32 /  \
    modify_field op set dst_type ipv4_src src_type value \
    src_value 10101010 width 32 / modify_field op add dst_type \
    ipv4_ttl dst_level 0 dst_offset 0 src_type value \
    src_value ff width 8 / count / jump group 100 / end
```

The patch increases cmdline input buffer size to the LINE_MAX value,
which typically is 2048 bytes.

Signed-off-by: Gregory Etelson <getelson@nvidia.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Morten Brørup <mb@smartsharesystems.com>
---
v2: set RDLINE_BUF_SIZE to LINE_MAX
    change the patch subject.
v3: fix Windows compilation error.
---
 lib/cmdline/cmdline_private.h | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/lib/cmdline/cmdline_private.h b/lib/cmdline/cmdline_private.h
index b64f363903..f00b025ecb 100644
--- a/lib/cmdline/cmdline_private.h
+++ b/lib/cmdline/cmdline_private.h
@@ -11,13 +11,20 @@
 #include <rte_os_shim.h>
 #ifdef RTE_EXEC_ENV_WINDOWS
 #include <rte_windows.h>
+#ifndef LINE_MAX
+/**
+ * The LINE_MAX value is derived from POSIX.
+ * Windows environment may not have POSIX definitions.
+ */
+#define LINE_MAX 2048
+#endif
 #else
 #include <termios.h>
 #endif

 #include <cmdline.h>

-#define RDLINE_BUF_SIZE 512
+#define RDLINE_BUF_SIZE LINE_MAX
 #define RDLINE_PROMPT_SIZE  32
 #define RDLINE_VT100_BUF_SIZE  8
 #define RDLINE_HISTORY_BUF_SIZE BUFSIZ
--
2.43.0


[-- Attachment #2: Type: text/html, Size: 4685 bytes --]

Re: [PATCH v3] cmdline: increase input buffer size

[David Marchand]

On Fri, May 3, 2024 at 6:28 AM Gregory Etelson <getelson@nvidia.com> wrote:
>
> DPDK defines cmdline input buffer size to 512 characters.
> That buffer size can be too small for long application input.
> For example, the following flow template API testpmd command is 444
> bytes long:
> ```
> flow queue 0 create 0 template_table 1000 \
>   pattern_template 0 actions_template 0 postpone no \
>   pattern eth / ipv4 / udp / end \
>   actions modify_field op set dst_type tag dst_level 0 dst_offset 0 \
>     src_type value src_value 0x31 width 32 /  \
>     modify_field op set dst_type ipv4_src src_type value \
>     src_value 10101010 width 32 / modify_field op add dst_type \
>     ipv4_ttl dst_level 0 dst_offset 0 src_type value \
>     src_value ff width 8 / count / jump group 100 / end
> ```
>
> The patch increases cmdline input buffer size to the LINE_MAX value,
> which typically is 2048 bytes.
>
> Signed-off-by: Gregory Etelson <getelson@nvidia.com>
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> Reviewed-by: Morten Brørup <mb@smartsharesystems.com>

Applied, thanks.


-- 
David Marchand