[dpdk-dev] [PATCH 0/6] update dpdk-setup.sh

[dpdk-dev] [PATCH 0/6] update dpdk-setup.sh

[Ferruh Yigit]

This is an effort to update the dpdk-setup.sh script and keep it one
more year until whole functionality of it is replaced.

Other option is to remove it completely, please comment.

Ferruh Yigit (6):
  usertools/setup: remove make based build
  usertools/setup: remove inserting custom kernel modules
  usertools/setup: remove running built applications
  usertools/setup: remove hugepage functions
  usertools/setup: fix loading vfio module
  usertools/setup: move removal target to 21.11

 doc/guides/rel_notes/deprecation.rst |   2 +-
 usertools/dpdk-setup.sh              | 332 ++-------------------------
 2 files changed, 15 insertions(+), 319 deletions(-)

-- 
2.26.2

[dpdk-dev] [PATCH 1/6] usertools/setup: remove make based build

[Ferruh Yigit]

DPDK build system is switched to 'meson' based build, please check the
documentation for details: "doc/guides/linux_gsg/build_dpdk.rst"

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 usertools/dpdk-setup.sh | 74 +++--------------------------------------
 1 file changed, 4 insertions(+), 70 deletions(-)

diff --git a/usertools/dpdk-setup.sh b/usertools/dpdk-setup.sh
index 411bf2e07f5e..d1eb188692a6 100755
--- a/usertools/dpdk-setup.sh
+++ b/usertools/dpdk-setup.sh
@@ -36,53 +36,6 @@ q()
 	quit
 }
 
-#
-# Sets up environmental variables for ICC.
-#
-setup_icc()
-{
-	DEFAULT_PATH=/opt/intel/bin/iccvars.sh
-	param=$1
-	shpath=`which iccvars.sh 2> /dev/null`
-	if [ $? -eq 0 ] ; then
-		echo "Loading iccvars.sh from $shpath for $param"
-		source $shpath $param
-	elif [ -f $DEFAULT_PATH ] ; then
-		echo "Loading iccvars.sh from $DEFAULT_PATH for $param"
-		source $DEFAULT_PATH $param
-	else
-		echo "## ERROR: cannot find 'iccvars.sh' script to set up ICC."
-		echo "##     To fix, please add the directory that contains"
-		echo "##     iccvars.sh  to your 'PATH' environment variable."
-		quit
-	fi
-}
-
-#
-# Sets RTE_TARGET and does a "make install".
-#
-setup_target()
-{
-	option=$1
-	export RTE_TARGET=${TARGETS[option]}
-
-	compiler=${RTE_TARGET##*-}
-	if [ "$compiler" == "icc" ] ; then
-		platform=${RTE_TARGET%%-*}
-		if [ "$platform" == "x86_64" ] ; then
-			setup_icc intel64
-		else
-			setup_icc ia32
-		fi
-	fi
-	if [ "$QUIT" == "0" ] ; then
-		make install T=${RTE_TARGET}
-	fi
-	echo "------------------------------------------------------------------------------"
-	echo " RTE_TARGET exported as $RTE_TARGET"
-	echo "------------------------------------------------------------------------------"
-}
-
 #
 # Creates hugepage filesystem.
 #
@@ -456,28 +409,10 @@ unbind_devices()
 	sudo ${RTE_SDK}/usertools/dpdk-devbind.py -b $DRV $PCI_PATH && echo "OK"
 }
 
-#
-# Options for building a target. Note that this step MUST be first as it sets
-# up TARGETS[] starting from 1, and this is accessed in setup_target using the
-# user entered option.
-#
-step1_func()
-{
-	TITLE="Select the DPDK environment to build"
-	CONFIG_NUM=1
-	for cfg in config/defconfig_* ; do
-		cfg=${cfg/config\/defconfig_/}
-		TEXT[$CONFIG_NUM]="$cfg"
-		TARGETS[$CONFIG_NUM]=$cfg
-		FUNC[$CONFIG_NUM]="setup_target"
-		let "CONFIG_NUM+=1"
-	done
-}
-
 #
 # Options for setting up environment.
 #
-step2_func()
+step1_func()
 {
 	TITLE="Setup linux environment"
 
@@ -512,7 +447,7 @@ step2_func()
 #
 # Options for running applications.
 #
-step3_func()
+step2_func()
 {
 	TITLE="Run test application for linux environment"
 
@@ -526,7 +461,7 @@ step3_func()
 #
 # Other options
 #
-step4_func()
+step3_func()
 {
 	TITLE="Other tools"
 
@@ -538,7 +473,7 @@ step4_func()
 #
 # Options for cleaning up the system
 #
-step5_func()
+step4_func()
 {
 	TITLE="Uninstall and system cleanup"
 
@@ -562,7 +497,6 @@ STEPS[1]="step1_func"
 STEPS[2]="step2_func"
 STEPS[3]="step3_func"
 STEPS[4]="step4_func"
-STEPS[5]="step5_func"
 
 QUIT=0
 
-- 
2.26.2

Re: [dpdk-dev] [PATCH 1/6] usertools/setup: remove make based build

[Walsh, Conor]

Hi Ferruh,

Patch tested.

Thanks,
Conor.

> From: dev <dev-bounces@dpdk.org> On Behalf Of Ferruh Yigit
> Sent: Wednesday 25 November 2020 19:19
> To: Thomas Monjalon <thomas@monjalon.net>
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; dev@dpdk.org;
> techboard@dpdk.org; Stephen Hemminger
> <stephen@networkplumber.org>; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: [dpdk-dev] [PATCH 1/6] usertools/setup: remove make based build
> 
> DPDK build system is switched to 'meson' based build, please check the
> documentation for details: "doc/guides/linux_gsg/build_dpdk.rst"
> 
> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>

Tested-by: Conor Walsh <conor.walsh@intel.com>

> ---
>  usertools/dpdk-setup.sh | 74 +++--------------------------------------
>  1 file changed, 4 insertions(+), 70 deletions(-)
> 
> diff --git a/usertools/dpdk-setup.sh b/usertools/dpdk-setup.sh
> index 411bf2e07f5e..d1eb188692a6 100755
> --- a/usertools/dpdk-setup.sh
> +++ b/usertools/dpdk-setup.sh
> @@ -36,53 +36,6 @@ q()
>  	quit
>  }
> 
> -#
> -# Sets up environmental variables for ICC.
> -#
> -setup_icc()
> -{
> -	DEFAULT_PATH=/opt/intel/bin/iccvars.sh
> -	param=$1
> -	shpath=`which iccvars.sh 2> /dev/null`
> -	if [ $? -eq 0 ] ; then
> -		echo "Loading iccvars.sh from $shpath for $param"
> -		source $shpath $param
> -	elif [ -f $DEFAULT_PATH ] ; then
> -		echo "Loading iccvars.sh from $DEFAULT_PATH for $param"
> -		source $DEFAULT_PATH $param
> -	else
> -		echo "## ERROR: cannot find 'iccvars.sh' script to set up ICC."
> -		echo "##     To fix, please add the directory that contains"
> -		echo "##     iccvars.sh  to your 'PATH' environment variable."
> -		quit
> -	fi
> -}
> -
> -#
> -# Sets RTE_TARGET and does a "make install".
> -#
> -setup_target()
> -{
> -	option=$1
> -	export RTE_TARGET=${TARGETS[option]}
> -
> -	compiler=${RTE_TARGET##*-}
> -	if [ "$compiler" == "icc" ] ; then
> -		platform=${RTE_TARGET%%-*}
> -		if [ "$platform" == "x86_64" ] ; then
> -			setup_icc intel64
> -		else
> -			setup_icc ia32
> -		fi
> -	fi
> -	if [ "$QUIT" == "0" ] ; then
> -		make install T=${RTE_TARGET}
> -	fi
> -	echo "----------------------------------------------------------------------------
> --"
> -	echo " RTE_TARGET exported as $RTE_TARGET"
> -	echo "----------------------------------------------------------------------------
> --"
> -}
> -
>  #
>  # Creates hugepage filesystem.
>  #
> @@ -456,28 +409,10 @@ unbind_devices()
>  	sudo ${RTE_SDK}/usertools/dpdk-devbind.py -b $DRV $PCI_PATH
> && echo "OK"
>  }
> 
> -#
> -# Options for building a target. Note that this step MUST be first as it sets
> -# up TARGETS[] starting from 1, and this is accessed in setup_target using
> the
> -# user entered option.
> -#
> -step1_func()
> -{
> -	TITLE="Select the DPDK environment to build"
> -	CONFIG_NUM=1
> -	for cfg in config/defconfig_* ; do
> -		cfg=${cfg/config\/defconfig_/}
> -		TEXT[$CONFIG_NUM]="$cfg"
> -		TARGETS[$CONFIG_NUM]=$cfg
> -		FUNC[$CONFIG_NUM]="setup_target"
> -		let "CONFIG_NUM+=1"
> -	done
> -}
> -
>  #
>  # Options for setting up environment.
>  #
> -step2_func()
> +step1_func()
>  {
>  	TITLE="Setup linux environment"
> 
> @@ -512,7 +447,7 @@ step2_func()
>  #
>  # Options for running applications.
>  #
> -step3_func()
> +step2_func()
>  {
>  	TITLE="Run test application for linux environment"
> 
> @@ -526,7 +461,7 @@ step3_func()
>  #
>  # Other options
>  #
> -step4_func()
> +step3_func()
>  {
>  	TITLE="Other tools"
> 
> @@ -538,7 +473,7 @@ step4_func()
>  #
>  # Options for cleaning up the system
>  #
> -step5_func()
> +step4_func()
>  {
>  	TITLE="Uninstall and system cleanup"
> 
> @@ -562,7 +497,6 @@ STEPS[1]="step1_func"
>  STEPS[2]="step2_func"
>  STEPS[3]="step3_func"
>  STEPS[4]="step4_func"
> -STEPS[5]="step5_func"
> 
>  QUIT=0
> 
> --
> 2.26.2

[dpdk-dev] [PATCH 2/6] usertools/setup: remove inserting custom kernel modules

[Ferruh Yigit]

Custom DPDK kernel module insert support relies on make based build
variable 'RTE_TARGET' to find the location of the kernel modules, which
is not valid anymore.

Also 'igb_uio' kernel module moved to another git repository:
https://git.dpdk.org/dpdk-kmods/

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 usertools/dpdk-setup.sh | 91 +++++++----------------------------------
 1 file changed, 14 insertions(+), 77 deletions(-)

diff --git a/usertools/dpdk-setup.sh b/usertools/dpdk-setup.sh
index d1eb188692a6..bcf0c843678f 100755
--- a/usertools/dpdk-setup.sh
+++ b/usertools/dpdk-setup.sh
@@ -78,39 +78,6 @@ remove_igb_uio_module()
 	fi
 }
 
-#
-# Loads new igb_uio.ko (and uio module if needed).
-#
-load_igb_uio_module()
-{
-	if [ ! -f $RTE_SDK/$RTE_TARGET/kmod/igb_uio.ko ];then
-		echo "## ERROR: Target does not have the DPDK UIO Kernel Module."
-		echo "       To fix, please try to rebuild target."
-		return
-	fi
-
-	remove_igb_uio_module
-
-	/sbin/lsmod | grep -s uio > /dev/null
-	if [ $? -ne 0 ] ; then
-		modinfo uio > /dev/null
-		if [ $? -eq 0 ]; then
-			echo "Loading uio module"
-			sudo /sbin/modprobe uio
-		fi
-	fi
-
-	# UIO may be compiled into kernel, so it may not be an error if it can't
-	# be loaded.
-
-	echo "Loading DPDK UIO module"
-	sudo /sbin/insmod $RTE_SDK/$RTE_TARGET/kmod/igb_uio.ko
-	if [ $? -ne 0 ] ; then
-		echo "## ERROR: Could not load kmod/igb_uio.ko."
-		quit
-	fi
-}
-
 #
 # Unloads VFIO modules.
 #
@@ -171,30 +138,6 @@ remove_kni_module()
 	fi
 }
 
-#
-# Loads the rte_kni.ko module.
-#
-load_kni_module()
-{
-    # Check that the KNI module is already built.
-	if [ ! -f $RTE_SDK/$RTE_TARGET/kmod/rte_kni.ko ];then
-		echo "## ERROR: Target does not have the DPDK KNI Module."
-		echo "       To fix, please try to rebuild target."
-		return
-	fi
-
-    # Unload existing version if present.
-	remove_kni_module
-
-    # Now try load the KNI module.
-	echo "Loading DPDK KNI module"
-	sudo /sbin/insmod $RTE_SDK/$RTE_TARGET/kmod/rte_kni.ko
-	if [ $? -ne 0 ] ; then
-		echo "## ERROR: Could not load kmod/rte_kni.ko."
-		quit
-	fi
-}
-
 #
 # Sets appropriate permissions on /dev/vfio/* files
 #
@@ -416,32 +359,26 @@ step1_func()
 {
 	TITLE="Setup linux environment"
 
-	TEXT[1]="Insert IGB UIO module"
-	FUNC[1]="load_igb_uio_module"
-
-	TEXT[2]="Insert VFIO module"
-	FUNC[2]="load_vfio_module"
-
-	TEXT[3]="Insert KNI module"
-	FUNC[3]="load_kni_module"
+	TEXT[1]="Insert VFIO module"
+	FUNC[1]="load_vfio_module"
 
-	TEXT[4]="Setup hugepage mappings for non-NUMA systems"
-	FUNC[4]="set_non_numa_pages"
+	TEXT[2]="Setup hugepage mappings for non-NUMA systems"
+	FUNC[2]="set_non_numa_pages"
 
-	TEXT[5]="Setup hugepage mappings for NUMA systems"
-	FUNC[5]="set_numa_pages"
+	TEXT[3]="Setup hugepage mappings for NUMA systems"
+	FUNC[3]="set_numa_pages"
 
-	TEXT[6]="Display current Ethernet/Baseband/Crypto device settings"
-	FUNC[6]="show_devices"
+	TEXT[4]="Display current Ethernet/Baseband/Crypto device settings"
+	FUNC[4]="show_devices"
 
-	TEXT[7]="Bind Ethernet/Baseband/Crypto device to IGB UIO module"
-	FUNC[7]="bind_devices_to_igb_uio"
+	TEXT[5]="Bind Ethernet/Baseband/Crypto device to IGB UIO module"
+	FUNC[5]="bind_devices_to_igb_uio"
 
-	TEXT[8]="Bind Ethernet/Baseband/Crypto device to VFIO module"
-	FUNC[8]="bind_devices_to_vfio"
+	TEXT[6]="Bind Ethernet/Baseband/Crypto device to VFIO module"
+	FUNC[6]="bind_devices_to_vfio"
 
-	TEXT[9]="Setup VFIO permissions"
-	FUNC[9]="set_vfio_permissions"
+	TEXT[7]="Setup VFIO permissions"
+	FUNC[7]="set_vfio_permissions"
 }
 
 #
-- 
2.26.2

Re: [dpdk-dev] [PATCH 2/6] usertools/setup: remove inserting custom kernel modules

[Walsh, Conor]

Hi Ferruh,

Patch tested.

Thanks,
Conor.

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Ferruh Yigit
> Sent: Wednesday 25 November 2020 19:19
> To: Thomas Monjalon <thomas@monjalon.net>
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; dev@dpdk.org;
> techboard@dpdk.org; Stephen Hemminger
> <stephen@networkplumber.org>; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: [dpdk-dev] [PATCH 2/6] usertools/setup: remove inserting custom
> kernel modules
> 
> Custom DPDK kernel module insert support relies on make based build
> variable 'RTE_TARGET' to find the location of the kernel modules, which
> is not valid anymore.
> 
> Also 'igb_uio' kernel module moved to another git repository:
> https://git.dpdk.org/dpdk-kmods/
> 
> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>

Tested-by: Conor Walsh <conor.walsh@intel.com>

> ---
>  usertools/dpdk-setup.sh | 91 +++++++----------------------------------
>  1 file changed, 14 insertions(+), 77 deletions(-)
> 
> diff --git a/usertools/dpdk-setup.sh b/usertools/dpdk-setup.sh
> index d1eb188692a6..bcf0c843678f 100755
> --- a/usertools/dpdk-setup.sh
> +++ b/usertools/dpdk-setup.sh
> @@ -78,39 +78,6 @@ remove_igb_uio_module()
>  	fi
>  }
> 
> -#
> -# Loads new igb_uio.ko (and uio module if needed).
> -#
> -load_igb_uio_module()
> -{
> -	if [ ! -f $RTE_SDK/$RTE_TARGET/kmod/igb_uio.ko ];then
> -		echo "## ERROR: Target does not have the DPDK UIO Kernel
> Module."
> -		echo "       To fix, please try to rebuild target."
> -		return
> -	fi
> -
> -	remove_igb_uio_module
> -
> -	/sbin/lsmod | grep -s uio > /dev/null
> -	if [ $? -ne 0 ] ; then
> -		modinfo uio > /dev/null
> -		if [ $? -eq 0 ]; then
> -			echo "Loading uio module"
> -			sudo /sbin/modprobe uio
> -		fi
> -	fi
> -
> -	# UIO may be compiled into kernel, so it may not be an error if it can't
> -	# be loaded.
> -
> -	echo "Loading DPDK UIO module"
> -	sudo /sbin/insmod $RTE_SDK/$RTE_TARGET/kmod/igb_uio.ko
> -	if [ $? -ne 0 ] ; then
> -		echo "## ERROR: Could not load kmod/igb_uio.ko."
> -		quit
> -	fi
> -}
> -
>  #
>  # Unloads VFIO modules.
>  #
> @@ -171,30 +138,6 @@ remove_kni_module()
>  	fi
>  }
> 
> -#
> -# Loads the rte_kni.ko module.
> -#
> -load_kni_module()
> -{
> -    # Check that the KNI module is already built.
> -	if [ ! -f $RTE_SDK/$RTE_TARGET/kmod/rte_kni.ko ];then
> -		echo "## ERROR: Target does not have the DPDK KNI
> Module."
> -		echo "       To fix, please try to rebuild target."
> -		return
> -	fi
> -
> -    # Unload existing version if present.
> -	remove_kni_module
> -
> -    # Now try load the KNI module.
> -	echo "Loading DPDK KNI module"
> -	sudo /sbin/insmod $RTE_SDK/$RTE_TARGET/kmod/rte_kni.ko
> -	if [ $? -ne 0 ] ; then
> -		echo "## ERROR: Could not load kmod/rte_kni.ko."
> -		quit
> -	fi
> -}
> -
>  #
>  # Sets appropriate permissions on /dev/vfio/* files
>  #
> @@ -416,32 +359,26 @@ step1_func()
>  {
>  	TITLE="Setup linux environment"
> 
> -	TEXT[1]="Insert IGB UIO module"
> -	FUNC[1]="load_igb_uio_module"
> -
> -	TEXT[2]="Insert VFIO module"
> -	FUNC[2]="load_vfio_module"
> -
> -	TEXT[3]="Insert KNI module"
> -	FUNC[3]="load_kni_module"
> +	TEXT[1]="Insert VFIO module"
> +	FUNC[1]="load_vfio_module"
> 
> -	TEXT[4]="Setup hugepage mappings for non-NUMA systems"
> -	FUNC[4]="set_non_numa_pages"
> +	TEXT[2]="Setup hugepage mappings for non-NUMA systems"
> +	FUNC[2]="set_non_numa_pages"
> 
> -	TEXT[5]="Setup hugepage mappings for NUMA systems"
> -	FUNC[5]="set_numa_pages"
> +	TEXT[3]="Setup hugepage mappings for NUMA systems"
> +	FUNC[3]="set_numa_pages"
> 
> -	TEXT[6]="Display current Ethernet/Baseband/Crypto device settings"
> -	FUNC[6]="show_devices"
> +	TEXT[4]="Display current Ethernet/Baseband/Crypto device settings"
> +	FUNC[4]="show_devices"
> 
> -	TEXT[7]="Bind Ethernet/Baseband/Crypto device to IGB UIO
> module"
> -	FUNC[7]="bind_devices_to_igb_uio"
> +	TEXT[5]="Bind Ethernet/Baseband/Crypto device to IGB UIO
> module"
> +	FUNC[5]="bind_devices_to_igb_uio"
> 
> -	TEXT[8]="Bind Ethernet/Baseband/Crypto device to VFIO module"
> -	FUNC[8]="bind_devices_to_vfio"
> +	TEXT[6]="Bind Ethernet/Baseband/Crypto device to VFIO module"
> +	FUNC[6]="bind_devices_to_vfio"
> 
> -	TEXT[9]="Setup VFIO permissions"
> -	FUNC[9]="set_vfio_permissions"
> +	TEXT[7]="Setup VFIO permissions"
> +	FUNC[7]="set_vfio_permissions"
>  }
> 
>  #
> --
> 2.26.2

[dpdk-dev] [PATCH 3/6] usertools/setup: remove running built applications

[Ferruh Yigit]

Running application relies on make based build variable 'RTE_TARGET' to
find the location of the applications, which is not valid anymore.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 usertools/dpdk-setup.sh | 52 ++---------------------------------------
 1 file changed, 2 insertions(+), 50 deletions(-)

diff --git a/usertools/dpdk-setup.sh b/usertools/dpdk-setup.sh
index bcf0c843678f..759f6c096536 100755
--- a/usertools/dpdk-setup.sh
+++ b/usertools/dpdk-setup.sh
@@ -17,11 +17,6 @@ echo "--------------------------------------------------------------------------
 
 HUGEPGSZ=`cat /proc/meminfo  | grep Hugepagesize | cut -d : -f 2 | tr -d ' '`
 
-#
-# Application EAL parameters for setting memory options (amount/channels/ranks).
-#
-EAL_PARAMS='-n 4'
-
 #
 # Sets QUIT variable so script will finish.
 #
@@ -252,34 +247,6 @@ set_numa_pages()
 	create_mnt_huge
 }
 
-#
-# Run unit test application.
-#
-run_test_app()
-{
-	echo ""
-	echo "  Enter hex bitmask of cores to execute test app on"
-	echo "  Example: to execute app on cores 0 to 7, enter 0xff"
-	echo -n "bitmask: "
-	read Bitmask
-	echo "Launching app"
-	sudo ${RTE_TARGET}/app/test -c $Bitmask $EAL_PARAMS
-}
-
-#
-# Run unit testpmd application.
-#
-run_testpmd_app()
-{
-	echo ""
-	echo "  Enter hex bitmask of cores to execute testpmd app on"
-	echo "  Example: to execute app on cores 0 to 7, enter 0xff"
-	echo -n "bitmask: "
-	read Bitmask
-	echo "Launching app"
-	sudo ${RTE_TARGET}/app/testpmd -c $Bitmask $EAL_PARAMS -- -i
-}
-
 #
 # Print hugepage information.
 #
@@ -381,24 +348,10 @@ step1_func()
 	FUNC[7]="set_vfio_permissions"
 }
 
-#
-# Options for running applications.
-#
-step2_func()
-{
-	TITLE="Run test application for linux environment"
-
-	TEXT[1]="Run test application (\$RTE_TARGET/app/test)"
-	FUNC[1]="run_test_app"
-
-	TEXT[2]="Run testpmd application in interactive mode (\$RTE_TARGET/app/testpmd)"
-	FUNC[2]="run_testpmd_app"
-}
-
 #
 # Other options
 #
-step3_func()
+step2_func()
 {
 	TITLE="Other tools"
 
@@ -410,7 +363,7 @@ step3_func()
 #
 # Options for cleaning up the system
 #
-step4_func()
+step3_func()
 {
 	TITLE="Uninstall and system cleanup"
 
@@ -433,7 +386,6 @@ step4_func()
 STEPS[1]="step1_func"
 STEPS[2]="step2_func"
 STEPS[3]="step3_func"
-STEPS[4]="step4_func"
 
 QUIT=0
 
-- 
2.26.2

Re: [dpdk-dev] [PATCH 3/6] usertools/setup: remove running built applications

[Walsh, Conor]

Hi Ferruh,

Patch tested.

Thanks,
Conor.

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Ferruh Yigit
> Sent: Wednesday 25 November 2020 19:19
> To: Thomas Monjalon <thomas@monjalon.net>
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; dev@dpdk.org;
> techboard@dpdk.org; Stephen Hemminger
> <stephen@networkplumber.org>; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: [dpdk-dev] [PATCH 3/6] usertools/setup: remove running built
> applications
> 
> Running application relies on make based build variable 'RTE_TARGET' to
> find the location of the applications, which is not valid anymore.
> 
> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>

Tested-by: Conor Walsh <conor.walsh@intel.com>

> ---
>  usertools/dpdk-setup.sh | 52 ++---------------------------------------
>  1 file changed, 2 insertions(+), 50 deletions(-)
> 
> diff --git a/usertools/dpdk-setup.sh b/usertools/dpdk-setup.sh
> index bcf0c843678f..759f6c096536 100755
> --- a/usertools/dpdk-setup.sh
> +++ b/usertools/dpdk-setup.sh
> @@ -17,11 +17,6 @@ echo "-----------------------------------------------------------
> ---------------
> 
>  HUGEPGSZ=`cat /proc/meminfo  | grep Hugepagesize | cut -d : -f 2 | tr -d ' '`
> 
> -#
> -# Application EAL parameters for setting memory options
> (amount/channels/ranks).
> -#
> -EAL_PARAMS='-n 4'
> -
>  #
>  # Sets QUIT variable so script will finish.
>  #
> @@ -252,34 +247,6 @@ set_numa_pages()
>  	create_mnt_huge
>  }
> 
> -#
> -# Run unit test application.
> -#
> -run_test_app()
> -{
> -	echo ""
> -	echo "  Enter hex bitmask of cores to execute test app on"
> -	echo "  Example: to execute app on cores 0 to 7, enter 0xff"
> -	echo -n "bitmask: "
> -	read Bitmask
> -	echo "Launching app"
> -	sudo ${RTE_TARGET}/app/test -c $Bitmask $EAL_PARAMS
> -}
> -
> -#
> -# Run unit testpmd application.
> -#
> -run_testpmd_app()
> -{
> -	echo ""
> -	echo "  Enter hex bitmask of cores to execute testpmd app on"
> -	echo "  Example: to execute app on cores 0 to 7, enter 0xff"
> -	echo -n "bitmask: "
> -	read Bitmask
> -	echo "Launching app"
> -	sudo ${RTE_TARGET}/app/testpmd -c $Bitmask $EAL_PARAMS -- -i
> -}
> -
>  #
>  # Print hugepage information.
>  #
> @@ -381,24 +348,10 @@ step1_func()
>  	FUNC[7]="set_vfio_permissions"
>  }
> 
> -#
> -# Options for running applications.
> -#
> -step2_func()
> -{
> -	TITLE="Run test application for linux environment"
> -
> -	TEXT[1]="Run test application (\$RTE_TARGET/app/test)"
> -	FUNC[1]="run_test_app"
> -
> -	TEXT[2]="Run testpmd application in interactive mode
> (\$RTE_TARGET/app/testpmd)"
> -	FUNC[2]="run_testpmd_app"
> -}
> -
>  #
>  # Other options
>  #
> -step3_func()
> +step2_func()
>  {
>  	TITLE="Other tools"
> 
> @@ -410,7 +363,7 @@ step3_func()
>  #
>  # Options for cleaning up the system
>  #
> -step4_func()
> +step3_func()
>  {
>  	TITLE="Uninstall and system cleanup"
> 
> @@ -433,7 +386,6 @@ step4_func()
>  STEPS[1]="step1_func"
>  STEPS[2]="step2_func"
>  STEPS[3]="step3_func"
> -STEPS[4]="step4_func"
> 
>  QUIT=0
> 
> --
> 2.26.2

Re: [dpdk-dev] [PATCH 3/6] usertools/setup: remove running built applications

[Thomas Monjalon]

26/11/2020 15:02, Walsh, Conor:
> Hi Ferruh,
> 
> Patch tested.
> 
> Thanks,
> Conor.
> 
> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of Ferruh Yigit
> > Sent: Wednesday 25 November 2020 19:19
> > To: Thomas Monjalon <thomas@monjalon.net>
> > Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; dev@dpdk.org;
> > techboard@dpdk.org; Stephen Hemminger
> > <stephen@networkplumber.org>; Richardson, Bruce
> > <bruce.richardson@intel.com>
> > Subject: [dpdk-dev] [PATCH 3/6] usertools/setup: remove running built
> > applications
> > 
> > Running application relies on make based build variable 'RTE_TARGET' to
> > find the location of the applications, which is not valid anymore.
> > 
> > Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> 
> Tested-by: Conor Walsh <conor.walsh@intel.com>

Hi Conor,
If no comment on code below, you can remove the below lines from your reply.
Thanks


> 
> > ---
> >  usertools/dpdk-setup.sh | 52 ++---------------------------------------
> >  1 file changed, 2 insertions(+), 50 deletions(-)
> > 
> > diff --git a/usertools/dpdk-setup.sh b/usertools/dpdk-setup.sh
> > index bcf0c843678f..759f6c096536 100755
> > --- a/usertools/dpdk-setup.sh
> > +++ b/usertools/dpdk-setup.sh
> > @@ -17,11 +17,6 @@ echo "-----------------------------------------------------------
> > ---------------
> > 
> >  HUGEPGSZ=`cat /proc/meminfo  | grep Hugepagesize | cut -d : -f 2 | tr -d ' '`
> > 
> > -#
> > -# Application EAL parameters for setting memory options
> > (amount/channels/ranks).
> > -#
> > -EAL_PARAMS='-n 4'
> > -
> >  #
> >  # Sets QUIT variable so script will finish.
> >  #
> > @@ -252,34 +247,6 @@ set_numa_pages()
> >  	create_mnt_huge
> >  }
> > 
> > -#
> > -# Run unit test application.
> > -#
> > -run_test_app()
> > -{
> > -	echo ""
> > -	echo "  Enter hex bitmask of cores to execute test app on"
> > -	echo "  Example: to execute app on cores 0 to 7, enter 0xff"
> > -	echo -n "bitmask: "
> > -	read Bitmask
> > -	echo "Launching app"
> > -	sudo ${RTE_TARGET}/app/test -c $Bitmask $EAL_PARAMS
> > -}
> > -
> > -#
> > -# Run unit testpmd application.
> > -#
> > -run_testpmd_app()
> > -{
> > -	echo ""
> > -	echo "  Enter hex bitmask of cores to execute testpmd app on"
> > -	echo "  Example: to execute app on cores 0 to 7, enter 0xff"
> > -	echo -n "bitmask: "
> > -	read Bitmask
> > -	echo "Launching app"
> > -	sudo ${RTE_TARGET}/app/testpmd -c $Bitmask $EAL_PARAMS -- -i
> > -}
> > -
> >  #
> >  # Print hugepage information.
> >  #
> > @@ -381,24 +348,10 @@ step1_func()
> >  	FUNC[7]="set_vfio_permissions"
> >  }
> > 
> > -#
> > -# Options for running applications.
> > -#
> > -step2_func()
> > -{
> > -	TITLE="Run test application for linux environment"
> > -
> > -	TEXT[1]="Run test application (\$RTE_TARGET/app/test)"
> > -	FUNC[1]="run_test_app"
> > -
> > -	TEXT[2]="Run testpmd application in interactive mode
> > (\$RTE_TARGET/app/testpmd)"
> > -	FUNC[2]="run_testpmd_app"
> > -}
> > -
> >  #
> >  # Other options
> >  #
> > -step3_func()
> > +step2_func()
> >  {
> >  	TITLE="Other tools"
> > 
> > @@ -410,7 +363,7 @@ step3_func()
> >  #
> >  # Options for cleaning up the system
> >  #
> > -step4_func()
> > +step3_func()
> >  {
> >  	TITLE="Uninstall and system cleanup"
> > 
> > @@ -433,7 +386,6 @@ step4_func()
> >  STEPS[1]="step1_func"
> >  STEPS[2]="step2_func"
> >  STEPS[3]="step3_func"
> > -STEPS[4]="step4_func"
> > 
> >  QUIT=0
> > 
> > --
> > 2.26.2
> 
> 

[dpdk-dev] [PATCH 4/6] usertools/setup: remove hugepage functions

[Ferruh Yigit]

There is a specific tool to setup hugepages, with better support, no
need to have duplication.

The hugepage script is: './usertools/dpdk-hugepages.py'

Please try "./usertools/dpdk-hugepages.py -h" for more information about
the tool.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 usertools/dpdk-setup.sh | 145 +++-------------------------------------
 1 file changed, 9 insertions(+), 136 deletions(-)

diff --git a/usertools/dpdk-setup.sh b/usertools/dpdk-setup.sh
index 759f6c096536..1e36661e572e 100755
--- a/usertools/dpdk-setup.sh
+++ b/usertools/dpdk-setup.sh
@@ -15,8 +15,6 @@ echo "--------------------------------------------------------------------------
 echo " RTE_SDK exported as $RTE_SDK"
 echo "------------------------------------------------------------------------------"
 
-HUGEPGSZ=`cat /proc/meminfo  | grep Hugepagesize | cut -d : -f 2 | tr -d ' '`
-
 #
 # Sets QUIT variable so script will finish.
 #
@@ -31,36 +29,6 @@ q()
 	quit
 }
 
-#
-# Creates hugepage filesystem.
-#
-create_mnt_huge()
-{
-	echo "Creating /mnt/huge and mounting as hugetlbfs"
-	sudo mkdir -p /mnt/huge
-
-	grep -s '/mnt/huge' /proc/mounts > /dev/null
-	if [ $? -ne 0 ] ; then
-		sudo mount -t hugetlbfs nodev /mnt/huge
-	fi
-}
-
-#
-# Removes hugepage filesystem.
-#
-remove_mnt_huge()
-{
-	echo "Unmounting /mnt/huge and removing directory"
-	grep -s '/mnt/huge' /proc/mounts > /dev/null
-	if [ $? -eq 0 ] ; then
-		sudo umount /mnt/huge
-	fi
-
-	if [ -d /mnt/huge ] ; then
-		sudo rm -R /mnt/huge
-	fi
-}
-
 #
 # Unloads igb_uio.ko.
 #
@@ -182,79 +150,6 @@ set_vfio_permissions()
 	fi
 }
 
-#
-# Removes all reserved hugepages.
-#
-clear_huge_pages()
-{
-	echo > .echo_tmp
-	for d in /sys/devices/system/node/node? ; do
-		echo "echo 0 > $d/hugepages/hugepages-${HUGEPGSZ}/nr_hugepages" >> .echo_tmp
-	done
-	echo "Removing currently reserved hugepages"
-	sudo sh .echo_tmp
-	rm -f .echo_tmp
-
-	remove_mnt_huge
-}
-
-#
-# Creates hugepages.
-#
-set_non_numa_pages()
-{
-	clear_huge_pages
-
-	echo ""
-	echo "  Input the number of ${HUGEPGSZ} hugepages"
-	echo "  Example: to have 128MB of hugepages available in a 2MB huge page system,"
-	echo "  enter '64' to reserve 64 * 2MB pages"
-	echo -n "Number of pages: "
-	read Pages
-
-	echo "echo $Pages > /sys/kernel/mm/hugepages/hugepages-${HUGEPGSZ}/nr_hugepages" > .echo_tmp
-
-	echo "Reserving hugepages"
-	sudo sh .echo_tmp
-	rm -f .echo_tmp
-
-	create_mnt_huge
-}
-
-#
-# Creates hugepages on specific NUMA nodes.
-#
-set_numa_pages()
-{
-	clear_huge_pages
-
-	echo ""
-	echo "  Input the number of ${HUGEPGSZ} hugepages for each node"
-	echo "  Example: to have 128MB of hugepages available per node in a 2MB huge page system,"
-	echo "  enter '64' to reserve 64 * 2MB pages on each node"
-
-	echo > .echo_tmp
-	for d in /sys/devices/system/node/node? ; do
-		node=$(basename $d)
-		echo -n "Number of pages for $node: "
-		read Pages
-		echo "echo $Pages > $d/hugepages/hugepages-${HUGEPGSZ}/nr_hugepages" >> .echo_tmp
-	done
-	echo "Reserving hugepages"
-	sudo sh .echo_tmp
-	rm -f .echo_tmp
-
-	create_mnt_huge
-}
-
-#
-# Print hugepage information.
-#
-grep_meminfo()
-{
-	grep -i huge /proc/meminfo
-}
-
 #
 # Calls dpdk-devbind.py --status to show the devices and what they
 # are all bound to, in terms of drivers.
@@ -329,41 +224,23 @@ step1_func()
 	TEXT[1]="Insert VFIO module"
 	FUNC[1]="load_vfio_module"
 
-	TEXT[2]="Setup hugepage mappings for non-NUMA systems"
-	FUNC[2]="set_non_numa_pages"
-
-	TEXT[3]="Setup hugepage mappings for NUMA systems"
-	FUNC[3]="set_numa_pages"
+	TEXT[2]="Display current Ethernet/Baseband/Crypto device settings"
+	FUNC[2]="show_devices"
 
-	TEXT[4]="Display current Ethernet/Baseband/Crypto device settings"
-	FUNC[4]="show_devices"
+	TEXT[3]="Bind Ethernet/Baseband/Crypto device to IGB UIO module"
+	FUNC[3]="bind_devices_to_igb_uio"
 
-	TEXT[5]="Bind Ethernet/Baseband/Crypto device to IGB UIO module"
-	FUNC[5]="bind_devices_to_igb_uio"
-
-	TEXT[6]="Bind Ethernet/Baseband/Crypto device to VFIO module"
-	FUNC[6]="bind_devices_to_vfio"
-
-	TEXT[7]="Setup VFIO permissions"
-	FUNC[7]="set_vfio_permissions"
-}
-
-#
-# Other options
-#
-step2_func()
-{
-	TITLE="Other tools"
-
-	TEXT[1]="List hugepage info from /proc/meminfo"
-	FUNC[1]="grep_meminfo"
+	TEXT[4]="Bind Ethernet/Baseband/Crypto device to VFIO module"
+	FUNC[4]="bind_devices_to_vfio"
 
+	TEXT[5]="Setup VFIO permissions"
+	FUNC[5]="set_vfio_permissions"
 }
 
 #
 # Options for cleaning up the system
 #
-step3_func()
+step2_func()
 {
 	TITLE="Uninstall and system cleanup"
 
@@ -378,14 +255,10 @@ step3_func()
 
 	TEXT[4]="Remove KNI module"
 	FUNC[4]="remove_kni_module"
-
-	TEXT[5]="Remove hugepage mappings"
-	FUNC[5]="clear_huge_pages"
 }
 
 STEPS[1]="step1_func"
 STEPS[2]="step2_func"
-STEPS[3]="step3_func"
 
 QUIT=0
 
-- 
2.26.2

Re: [dpdk-dev] [PATCH 4/6] usertools/setup: remove hugepage functions

[Walsh, Conor]

Hi Ferruh,

Patch tested.

Thanks,
Conor.

> From: dev <dev-bounces@dpdk.org> On Behalf Of Ferruh Yigit
> Sent: Wednesday 25 November 2020 19:19
> To: Thomas Monjalon <thomas@monjalon.net>
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; dev@dpdk.org;
> techboard@dpdk.org; Stephen Hemminger
> <stephen@networkplumber.org>; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: [dpdk-dev] [PATCH 4/6] usertools/setup: remove hugepage
> functions
> 
> There is a specific tool to setup hugepages, with better support, no
> need to have duplication.
> 
> The hugepage script is: './usertools/dpdk-hugepages.py'
> 
> Please try "./usertools/dpdk-hugepages.py -h" for more information about
> the tool.
> 
> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>

Tested-by: Conor Walsh <conor.walsh@intel.com>

[dpdk-dev] [PATCH 5/6] usertools/setup: fix loading vfio module

[Ferruh Yigit]

script is checking the existing of the kernel module file, but in some
distros kernel modules are stored compressed, like as 'vfio-pci.ko.xz'.

Add wildcard to cover both compressed and not compressed cases.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 usertools/dpdk-setup.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/usertools/dpdk-setup.sh b/usertools/dpdk-setup.sh
index 1e36661e572e..e8667e094a53 100755
--- a/usertools/dpdk-setup.sh
+++ b/usertools/dpdk-setup.sh
@@ -62,7 +62,7 @@ load_vfio_module()
 {
 	remove_vfio_module
 
-	VFIO_PATH="kernel/drivers/vfio/pci/vfio-pci.ko"
+	VFIO_PATH="kernel/drivers/vfio/pci/vfio-pci.ko*"
 
 	echo "Loading VFIO module"
 	/sbin/lsmod | grep -s vfio_pci > /dev/null
-- 
2.26.2

[dpdk-dev] [PATCH 6/6] usertools/setup: move removal target to 21.11

[Ferruh Yigit]

There are still some functions in the script that not replaced
completely, give more time to complete removal of the tool.

Current functionality supported by the script is:
 ----------------------------------------------------------
 Step 1: Setup linux environment
 ----------------------------------------------------------
 [1] Insert VFIO module
 [2] Display current Ethernet/Baseband/Crypto device settings
 [3] Bind Ethernet/Baseband/Crypto device to IGB UIO module
 [4] Bind Ethernet/Baseband/Crypto device to VFIO module
 [5] Setup VFIO permissions

 ----------------------------------------------------------
  Step 2: Uninstall and system cleanup
 ----------------------------------------------------------
 [6] Unbind devices from IGB UIO or VFIO driver
 [7] Remove IGB UIO module
 [8] Remove VFIO module
 [9] Remove KNI module

 [10] Exit Script

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
 doc/guides/rel_notes/deprecation.rst | 2 +-
 usertools/dpdk-setup.sh              | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 96986fabd598..6ad3dafe1bd3 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -103,5 +103,5 @@ Deprecation Notices
 
 * dpdk-setup.sh: This old script relies on deprecated stuff, and especially
   ``make``. Given environments are too much variables for such a simple script,
-  it will be removed in DPDK 20.11.
+  it will be removed in DPDK 21.11.
   Some useful parts may be converted into specific scripts.
diff --git a/usertools/dpdk-setup.sh b/usertools/dpdk-setup.sh
index e8667e094a53..9f24e28c03fd 100755
--- a/usertools/dpdk-setup.sh
+++ b/usertools/dpdk-setup.sh
@@ -289,7 +289,7 @@ while [ "$QUIT" == "0" ]; do
 	OPTIONS[$OPTION_NUM]="quit"
 	echo ""
 	echo '--------------------------------------------------'
-	echo 'WARNING: This tool will be removed from DPDK 20.11'
+	echo 'WARNING: This tool will be removed from DPDK 21.11'
 	echo '--------------------------------------------------'
 	echo
 	echo -n "Option: "
-- 
2.26.2

Re: [dpdk-dev] [PATCH 0/6] update dpdk-setup.sh

[Burakov, Anatoly]

On 25-Nov-20 7:19 PM, Ferruh Yigit wrote:
> This is an effort to update the dpdk-setup.sh script and keep it one
> more year until whole functionality of it is replaced.
> 
> Other option is to remove it completely, please comment.

This script has been largely unmaintained for a long time, so my 
personal opinion is that i'd rather see it removed.

That said, i know for a fact that users (especially non-expert ones) 
rely on this script, so removing it without providing a replacement is 
not practical. This patchset IMO is a good first step towards bringing 
the script up to date, but more work is needed as current usability of 
the script is atrocious.

> 
> Ferruh Yigit (6):
>    usertools/setup: remove make based build
>    usertools/setup: remove inserting custom kernel modules
>    usertools/setup: remove running built applications
>    usertools/setup: remove hugepage functions
>    usertools/setup: fix loading vfio module
>    usertools/setup: move removal target to 21.11
> 
>   doc/guides/rel_notes/deprecation.rst |   2 +-
>   usertools/dpdk-setup.sh              | 332 ++-------------------------
>   2 files changed, 15 insertions(+), 319 deletions(-)
> 


-- 
Thanks,
Anatoly

Re: [dpdk-dev] [PATCH 0/6] update dpdk-setup.sh

[Thomas Monjalon]

27/11/2020 14:57, Burakov, Anatoly:
> On 25-Nov-20 7:19 PM, Ferruh Yigit wrote:
> > This is an effort to update the dpdk-setup.sh script and keep it one
> > more year until whole functionality of it is replaced.
> > 
> > Other option is to remove it completely, please comment.
> 
> This script has been largely unmaintained for a long time, so my 
> personal opinion is that i'd rather see it removed.
> 
> That said, i know for a fact that users (especially non-expert ones) 
> rely on this script, so removing it without providing a replacement is 
> not practical. This patchset IMO is a good first step towards bringing 
> the script up to date, but more work is needed as current usability of 
> the script is atrocious.

I take it as one more argument to remove as planned in 20.11.

Non-expert users should think before using this script.
If they absolutely want it back they will take it from old versions,
or they will complain, which is a kind of contribution,
a first step to think about something better.