DPDK patches and discussions
 help / color / mirror / Atom feed
* [PATCH] devtools: add script to check for non inclusive naming
@ 2023-03-31 20:08 Stephen Hemminger
  2023-04-03 14:47 ` [PATCH v2] " Stephen Hemminger
  0 siblings, 1 reply; 7+ messages in thread
From: Stephen Hemminger @ 2023-03-31 20:08 UTC (permalink / raw)
  To: dev; +Cc: Stephen Hemminger

Shell script to find use of words that not be used.
By default it prints matches.  The -q (quiet) option
is used to just count. There is also -l option
which lists lines matching (like grep -l).

Examples:
 $ ./devtools/check-naming.sh -q
Total files: 37 errors, 90 warnings, 1 suggestions

 $ ./devtools/check-naming.sh -q -l
Total lines: 121 errors, 255 warnings, 1 suggestions

Uses the word lists from Inclusive Naming Initiative
see https://inclusivenaming.org/word-lists/

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
 devtools/check-naming.sh  | 89 +++++++++++++++++++++++++++++++++++++++
 devtools/naming/tier1.txt |  8 ++++
 devtools/naming/tier2.txt |  1 +
 devtools/naming/tier3.txt |  3 ++
 4 files changed, 101 insertions(+)
 create mode 100755 devtools/check-naming.sh
 create mode 100644 devtools/naming/tier1.txt
 create mode 100644 devtools/naming/tier2.txt
 create mode 100644 devtools/naming/tier3.txt

diff --git a/devtools/check-naming.sh b/devtools/check-naming.sh
new file mode 100755
index 000000000000..2a25eaa2eef9
--- /dev/null
+++ b/devtools/check-naming.sh
@@ -0,0 +1,89 @@
+#! /bin/bash
+# SPDX-License-Identifier: BSD-3-Clause
+#
+# Produce a list of files with non-inclusive naming
+
+errors=0
+warnings=0
+suggestions=0
+quiet=false
+veborse=false
+lines='-l'
+
+print_usage () {
+    echo "usage: $(basename $0) [-l] [-q] [-v]"
+    exit 1
+}
+
+# Locate word list files
+selfdir=$(dirname $(readlink -f $0))
+words=$selfdir/naming
+
+# These give false positives
+skipfiles=( ':^devtools/naming/' \
+	    ':^doc/guides/rel_notes/' \
+	    ':^doc/guides/contributing/coding_style.rst' \
+	    ':^doc/guides/prog_guide/glossary.rst' \
+)
+# These are obsolete
+skipfiles+=( \
+	    ':^drivers/net/liquidio/' \
+	    ':^drivers/net/bnx2x/' \
+	    ':^lib/table/' \
+	    ':^lib/port/' \
+	    ':^lib/pipeline/' \
+	    ':^examples/pipeline/' \
+)
+
+#
+# check_wordlist wordfile description
+check_wordlist() {
+    local list=$words/$1
+    local description=$2
+
+    git grep -i $lines -f $list -- ${skipfiles[@]} > $tmpfile
+    count=$(wc -l < $tmpfile)
+    if ! $quiet; then
+	if [ $count -gt 0 ]; then
+	    if $verbose; then
+   		    echo $description
+		    echo $description | tr '[:print:]' '-'
+	    fi
+   	    cat $tmpfile
+	    echo
+	fi
+    fi
+    return $count
+}
+
+while getopts lqvh ARG ; do
+	case $ARG in
+		l ) lines= ;;
+		q ) quiet=true ;;
+		v ) verbose=true ;;
+		h ) print_usage ; exit 0 ;;
+		? ) print_usage ; exit 1 ;;
+	esac
+done
+shift $(($OPTIND - 1))
+
+tmpfile=$(mktemp -t dpdk.checknames.XXXXXX)
+trap 'rm -f -- "$tmpfile"' INT TERM HUP EXIT
+
+check_wordlist tier1.txt "Tier 1: Replace immediately"
+errors=$?
+
+check_wordlist tier2.txt "Tier 2: Strongly consider replacing"
+warnings=$?
+
+check_wordlist tier3.txt "Tier 3: Recommend to replace"
+suggestions=$?
+
+if [ -z "$lines" ] ; then
+    echo -n "Total lines: "
+else
+    echo -n "Total files: "
+fi
+
+echo $errors "errors," $warnings "warnings," $suggestions "suggestions"
+exit $errors
diff --git a/devtools/naming/tier1.txt b/devtools/naming/tier1.txt
new file mode 100644
index 000000000000..46532f2b234b
--- /dev/null
+++ b/devtools/naming/tier1.txt
@@ -0,0 +1,8 @@
+abort
+blackhat
+whitehat
+cipple
+master
+slave
+whitelist
+blacklist
diff --git a/devtools/naming/tier2.txt b/devtools/naming/tier2.txt
new file mode 100644
index 000000000000..cd4280d1625c
--- /dev/null
+++ b/devtools/naming/tier2.txt
@@ -0,0 +1 @@
+sanity
diff --git a/devtools/naming/tier3.txt b/devtools/naming/tier3.txt
new file mode 100644
index 000000000000..0f9586ff5bac
--- /dev/null
+++ b/devtools/naming/tier3.txt
@@ -0,0 +1,3 @@
+man.in.the.middle
+segregate
+tribe
-- 
2.39.2


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH v2] devtools: add script to check for non inclusive naming
  2023-03-31 20:08 [PATCH] devtools: add script to check for non inclusive naming Stephen Hemminger
@ 2023-04-03 14:47 ` Stephen Hemminger
  2023-04-03 23:08   ` Luca Boccassi
  0 siblings, 1 reply; 7+ messages in thread
From: Stephen Hemminger @ 2023-04-03 14:47 UTC (permalink / raw)
  To: dev; +Cc: Stephen Hemminger

Shell script to find use of words that not be used.
By default it prints matches.  The -q (quiet) option
is used to just count. There is also -l option
which lists lines matching (like grep -l).

Uses the word lists from Inclusive Naming Initiative
see https://inclusivenaming.org/word-lists/

Examples:
 $ ./devtools/check-naming-policy.sh -q
 Total files: 37 errors, 90 warnings, 2 suggestions

 $ ./devtools/check-naming-policy.sh -q -l lib/eal
 Total lines: 32 errors, 8 warnings, 0 suggestions

Add MAINTAINERS file entry for the new tool and resort
the list files back into to alphabetic order

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
v2 - fix typo in words
   - add subtree (pathspec) option
   - update maintainers file (and fix alphabetic order)

 MAINTAINERS                     |   8 ++-
 devtools/check-naming-policy.sh | 107 ++++++++++++++++++++++++++++++++
 devtools/naming/tier1.txt       |   8 +++
 devtools/naming/tier2.txt       |   1 +
 devtools/naming/tier3.txt       |   4 ++
 5 files changed, 125 insertions(+), 3 deletions(-)
 create mode 100755 devtools/check-naming-policy.sh
 create mode 100644 devtools/naming/tier1.txt
 create mode 100644 devtools/naming/tier2.txt
 create mode 100644 devtools/naming/tier3.txt

diff --git a/MAINTAINERS b/MAINTAINERS
index 8df23e50999f..b5881113ba85 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -83,26 +83,28 @@ Developers and Maintainers Tools
 M: Thomas Monjalon <thomas@monjalon.net>
 F: MAINTAINERS
 F: devtools/build-dict.sh
-F: devtools/check-abi.sh
 F: devtools/check-abi-version.sh
+F: devtools/check-abi.sh
 F: devtools/check-doc-vs-code.sh
 F: devtools/check-dup-includes.sh
-F: devtools/check-maintainers.sh
 F: devtools/check-forbidden-tokens.awk
 F: devtools/check-git-log.sh
+F: devtools/check-maintainers.sh
+F: devtools/check-naming-policy.sh
 F: devtools/check-spdx-tag.sh
 F: devtools/check-symbol-change.sh
 F: devtools/check-symbol-maps.sh
 F: devtools/checkpatches.sh
 F: devtools/get-maintainer.sh
 F: devtools/git-log-fixes.sh
+F: devtools/libabigail.abignore
 F: devtools/load-devel-config
+F: devtools/naming/
 F: devtools/parse-flow-support.sh
 F: devtools/process-iwyu.py
 F: devtools/update-abi.sh
 F: devtools/update-patches.py
 F: devtools/update_version_map_abi.py
-F: devtools/libabigail.abignore
 F: devtools/words-case.txt
 F: license/
 F: .editorconfig
diff --git a/devtools/check-naming-policy.sh b/devtools/check-naming-policy.sh
new file mode 100755
index 000000000000..90347b415652
--- /dev/null
+++ b/devtools/check-naming-policy.sh
@@ -0,0 +1,107 @@
+#! /bin/bash
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2023 Stephen Hemminger
+#
+# This script scans the source tree and creates list of files
+# containing words that are recommended to bavoide by the
+# Inclusive Naming Initiative.
+# See: https://inclusivenaming.org/word-lists/
+#
+# The options are:
+#   -q = quiet mode, produces summary count only
+#   -l = show lines instead of files with recommendations
+#   -v = verbose, show a header between each tier
+#
+# Default is to scan all of DPDK source and documentation.
+# Optional pathspec can be used to limit specific tree.
+#
+#  Example:
+#    check-naming-policy.sh -q doc/*
+#
+
+errors=0
+warnings=0
+suggestions=0
+quiet=false
+veborse=false
+lines='-l'
+
+print_usage () {
+    echo "usage: $(basename $0) [-l] [-q] [-v] [<pathspec>]"
+    exit 1
+}
+
+# Locate word list files
+selfdir=$(dirname $(readlink -f $0))
+words=$selfdir/naming
+
+# These give false positives
+skipfiles=( ':^devtools/naming/' \
+	    ':^doc/guides/rel_notes/' \
+	    ':^doc/guides/contributing/coding_style.rst' \
+	    ':^doc/guides/prog_guide/glossary.rst' \
+)
+# These are obsolete
+skipfiles+=( \
+	    ':^drivers/net/liquidio/' \
+	    ':^drivers/net/bnx2x/' \
+	    ':^lib/table/' \
+	    ':^lib/port/' \
+	    ':^lib/pipeline/' \
+	    ':^examples/pipeline/' \
+)
+
+#
+# check_wordlist wordfile description
+check_wordlist() {
+    local list=$words/$1
+    local description=$2
+
+    git grep -i $lines -f $list -- ${skipfiles[@]} $pathspec > $tmpfile
+    count=$(wc -l < $tmpfile)
+    if ! $quiet; then
+	if [ $count -gt 0 ]; then
+	    if $verbose; then
+   		    echo $description
+		    echo $description | tr '[:print:]' '-'
+	    fi
+   	    cat $tmpfile
+	    echo
+	fi
+    fi
+    return $count
+}
+
+while getopts lqvh ARG ; do
+	case $ARG in
+		l ) lines= ;;
+		q ) quiet=true ;;
+		v ) verbose=true ;;
+		h ) print_usage ; exit 0 ;;
+		? ) print_usage ; exit 1 ;;
+	esac
+done
+shift $(($OPTIND - 1))
+
+tmpfile=$(mktemp -t dpdk.checknames.XXXXXX)
+trap 'rm -f -- "$tmpfile"' INT TERM HUP EXIT
+
+pathspec=$*
+
+check_wordlist tier1.txt "Tier 1: Replace immediately"
+errors=$?
+
+check_wordlist tier2.txt "Tier 2: Strongly consider replacing"
+warnings=$?
+
+check_wordlist tier3.txt "Tier 3: Recommend to replace"
+suggestions=$?
+
+if [ -z "$lines" ] ; then
+    echo -n "Total lines: "
+else
+    echo -n "Total files: "
+fi
+
+echo $errors "errors," $warnings "warnings," $suggestions "suggestions"
+exit $errors
diff --git a/devtools/naming/tier1.txt b/devtools/naming/tier1.txt
new file mode 100644
index 000000000000..a0e9b549c218
--- /dev/null
+++ b/devtools/naming/tier1.txt
@@ -0,0 +1,8 @@
+abort
+blackhat
+blacklist
+cripple
+master
+slave
+whitehat
+whitelist
diff --git a/devtools/naming/tier2.txt b/devtools/naming/tier2.txt
new file mode 100644
index 000000000000..cd4280d1625c
--- /dev/null
+++ b/devtools/naming/tier2.txt
@@ -0,0 +1 @@
+sanity
diff --git a/devtools/naming/tier3.txt b/devtools/naming/tier3.txt
new file mode 100644
index 000000000000..072f6468ea47
--- /dev/null
+++ b/devtools/naming/tier3.txt
@@ -0,0 +1,4 @@
+man.in.the.middle
+segregate
+segregation
+tribe
-- 
2.39.2


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH v2] devtools: add script to check for non inclusive naming
  2023-04-03 14:47 ` [PATCH v2] " Stephen Hemminger
@ 2023-04-03 23:08   ` Luca Boccassi
  2023-04-04  2:17     ` Stephen Hemminger
  0 siblings, 1 reply; 7+ messages in thread
From: Luca Boccassi @ 2023-04-03 23:08 UTC (permalink / raw)
  To: Stephen Hemminger; +Cc: dev

On Mon, 3 Apr 2023 at 15:47, Stephen Hemminger
<stephen@networkplumber.org> wrote:
>
> Shell script to find use of words that not be used.
> By default it prints matches.  The -q (quiet) option
> is used to just count. There is also -l option
> which lists lines matching (like grep -l).
>
> Uses the word lists from Inclusive Naming Initiative
> see https://inclusivenaming.org/word-lists/
>
> Examples:
>  $ ./devtools/check-naming-policy.sh -q
>  Total files: 37 errors, 90 warnings, 2 suggestions
>
>  $ ./devtools/check-naming-policy.sh -q -l lib/eal
>  Total lines: 32 errors, 8 warnings, 0 suggestions
>
> Add MAINTAINERS file entry for the new tool and resort
> the list files back into to alphabetic order
>
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> ---
> v2 - fix typo in words
>    - add subtree (pathspec) option
>    - update maintainers file (and fix alphabetic order)

There's a json file on the website, how about downloading that on the
fly rather than storing a local copy that will go out of date?
https://inclusivenaming.org/word-lists/index.json

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH v2] devtools: add script to check for non inclusive naming
  2023-04-03 23:08   ` Luca Boccassi
@ 2023-04-04  2:17     ` Stephen Hemminger
  2023-04-04 22:00       ` Luca Boccassi
  0 siblings, 1 reply; 7+ messages in thread
From: Stephen Hemminger @ 2023-04-04  2:17 UTC (permalink / raw)
  To: Luca Boccassi; +Cc: dev

On Tue, 4 Apr 2023 00:08:30 +0100
Luca Boccassi <bluca@debian.org> wrote:

> On Mon, 3 Apr 2023 at 15:47, Stephen Hemminger
> <stephen@networkplumber.org> wrote:
> >
> > Shell script to find use of words that not be used.
> > By default it prints matches.  The -q (quiet) option
> > is used to just count. There is also -l option
> > which lists lines matching (like grep -l).
> >
> > Uses the word lists from Inclusive Naming Initiative
> > see https://inclusivenaming.org/word-lists/
> >
> > Examples:
> >  $ ./devtools/check-naming-policy.sh -q
> >  Total files: 37 errors, 90 warnings, 2 suggestions
> >
> >  $ ./devtools/check-naming-policy.sh -q -l lib/eal
> >  Total lines: 32 errors, 8 warnings, 0 suggestions
> >
> > Add MAINTAINERS file entry for the new tool and resort
> > the list files back into to alphabetic order
> >
> > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> > ---
> > v2 - fix typo in words
> >    - add subtree (pathspec) option
> >    - update maintainers file (and fix alphabetic order)  
> 
> There's a json file on the website, how about downloading that on the
> fly rather than storing a local copy that will go out of date?
> https://inclusivenaming.org/word-lists/index.json

Ok, but that would mean using python and would also mean that terms like
segreation which are not on the official list would not be caught

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH v2] devtools: add script to check for non inclusive naming
  2023-04-04  2:17     ` Stephen Hemminger
@ 2023-04-04 22:00       ` Luca Boccassi
  2023-04-05  1:23         ` Stephen Hemminger
  0 siblings, 1 reply; 7+ messages in thread
From: Luca Boccassi @ 2023-04-04 22:00 UTC (permalink / raw)
  To: Stephen Hemminger; +Cc: dev

On Mon, 2023-04-03 at 19:17 -0700, Stephen Hemminger wrote:
> On Tue, 4 Apr 2023 00:08:30 +0100
> Luca Boccassi <bluca@debian.org> wrote:
> 
> > On Mon, 3 Apr 2023 at 15:47, Stephen Hemminger
> > <stephen@networkplumber.org> wrote:
> > > 
> > > Shell script to find use of words that not be used.
> > > By default it prints matches.  The -q (quiet) option
> > > is used to just count. There is also -l option
> > > which lists lines matching (like grep -l).
> > > 
> > > Uses the word lists from Inclusive Naming Initiative
> > > see https://inclusivenaming.org/word-lists/
> > > 
> > > Examples:
> > >  $ ./devtools/check-naming-policy.sh -q
> > >  Total files: 37 errors, 90 warnings, 2 suggestions
> > > 
> > >  $ ./devtools/check-naming-policy.sh -q -l lib/eal
> > >  Total lines: 32 errors, 8 warnings, 0 suggestions
> > > 
> > > Add MAINTAINERS file entry for the new tool and resort
> > > the list files back into to alphabetic order
> > > 
> > > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> > > ---
> > > v2 - fix typo in words
> > >    - add subtree (pathspec) option
> > >    - update maintainers file (and fix alphabetic order)  
> > 
> > There's a json file on the website, how about downloading that on the
> > fly rather than storing a local copy that will go out of date?
> > https://inclusivenaming.org/word-lists/index.json
> 
> Ok, but that would mean using python and would also mean that terms like
> segreation which are not on the official list would not be caught

No need for python, it can be done with 'jq' very easily. Also there's
'segregate' which is close enough, it's tier 3. eg:

$ wget https://inclusivenaming.org/word-lists/index.json -q -O- | jq -r '.data[] | select ((.tier == "3")) | .term'
man-in-the-middle
Segregate

-- 
Kind regards,
Luca Boccassi

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH v2] devtools: add script to check for non inclusive naming
  2023-04-04 22:00       ` Luca Boccassi
@ 2023-04-05  1:23         ` Stephen Hemminger
  0 siblings, 0 replies; 7+ messages in thread
From: Stephen Hemminger @ 2023-04-05  1:23 UTC (permalink / raw)
  To: Luca Boccassi; +Cc: dev

On Tue, 04 Apr 2023 23:00:42 +0100
Luca Boccassi <bluca@debian.org> wrote:

> > 
> > Ok, but that would mean using python and would also mean that terms like
> > segreation which are not on the official list would not be caught  
> 
> No need for python, it can be done with 'jq' very easily. Also there's
> 'segregate' which is close enough, it's tier 3. eg:
> 
> $ wget https://inclusivenaming.org/word-lists/index.json -q -O- | jq -r '.data[] | select ((.tier == "3")) | .term'
> man-in-the-middle
> Segregate

Doing it in python allows for better UI. And makes it easier to do enhancements
like show tier 1 only, or add more words on command line, or exclude additional directories.

Feature creep can be fun...

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH] devtools: add script to check for non inclusive naming
       [not found] <0230331200824.195294-1-stephen@networkplumber.org>
@ 2023-04-19 15:00 ` Stephen Hemminger
  0 siblings, 0 replies; 7+ messages in thread
From: Stephen Hemminger @ 2023-04-19 15:00 UTC (permalink / raw)
  To: dev; +Cc: Stephen Hemminger

Script to find words that should not be used.
Really just a wrapper around git grep command.
By default it prints matches.

Uses the word lists from Inclusive Naming Initiative
see https://inclusivenaming.org/word-lists/

Examples:
$ ./devtools/check-naming-policy.sh -c
app/test-compress-perf/comp_perf_test_cyclecount.c:1
uapp/test-compress-perf/comp_perf_test_throughput.c:1
app/test-compress-perf/comp_perf_test_verify.c:1
app/test/test_common.c:1
...

$ ./devtools/check-naming-policy.py lib/pcapng
lib/pcapng/rte_pcapng.c:                /* sanity check that is really a pcapng mbuf */

$ ./devtools/check-naming-policy.py -l lib/eal
lib/eal/common/eal_common_memory.c
lib/eal/common/eal_common_proc.c
lib/eal/common/eal_common_trace.c
lib/eal/common/eal_memcfg.h
lib/eal/common/rte_malloc.c
lib/eal/freebsd/eal.c
lib/eal/include/generic/rte_power_intrinsics.h
lib/eal/include/generic/rte_rwlock.h
lib/eal/include/generic/rte_spinlock.h
lib/eal/include/rte_debug.h
lib/eal/include/rte_seqlock.h
lib/eal/linux/eal.c
lib/eal/windows/eal.c
lib/eal/x86/include/rte_rtm.h
lib/eal/x86/include/rte_spinlock.h
lib/eal/x86/rte_power_intrinsics.c

$ ./devtools/check-inclusive-naming -h
usage: check-inclusive-naming.py [-h] [-c] [-d] [-l] [-t {0,1,2,3}]
                                 [-x EXCLUDE] [--url URL]
                                 [paths ...]

Identify word usage not aligned with inclusive naming

positional arguments:
  paths                 files and directory to scan

options:
  -h, --help            show this help message and exit
  -c, --count           Show the nuber of lines that match
  -d, --debug           Debug this script
  -l, --files-with-matches
                        Show only names of files with hits
  -t {0,1,2,3}, --tier {0,1,2,3}
                        Show non-conforming words of particular tier
  -x EXCLUDE, --exclude EXCLUDE
                        Exclude path from scan
  --url URL             URL for the non-inclusive naming word list

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
v4 - fix python lint warnings and spelling

 MAINTAINERS                        |   1 +
 devtools/check-inclusive-naming.py | 137 +++++++++++++++++++++++++++++
 2 files changed, 138 insertions(+)
 create mode 100755 devtools/check-inclusive-naming.py

diff --git a/MAINTAINERS b/MAINTAINERS
index 8df23e50999f..141d70596020 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -89,6 +89,7 @@ F: devtools/check-doc-vs-code.sh
 F: devtools/check-dup-includes.sh
 F: devtools/check-maintainers.sh
 F: devtools/check-forbidden-tokens.awk
+F: devtools/check-inclusive-naming.py
 F: devtools/check-git-log.sh
 F: devtools/check-spdx-tag.sh
 F: devtools/check-symbol-change.sh
diff --git a/devtools/check-inclusive-naming.py b/devtools/check-inclusive-naming.py
new file mode 100755
index 000000000000..916d49aa2ecf
--- /dev/null
+++ b/devtools/check-inclusive-naming.py
@@ -0,0 +1,137 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2023 Stephen Hemminger
+#
+# This script scans the source tree and creates list of files
+# containing words that are recommended to avoided by the
+# Inclusive Naming Initiative.
+# See: https://inclusivenaming.org/word-lists/
+
+"""Script to run git grep to finds strings in inclusive naming word list."""
+
+import argparse
+import json
+import subprocess
+import sys
+from urllib.request import urlopen
+
+WORDLIST_URL = 'https://inclusivenaming.org/word-lists/index.json'
+
+# These give false positives
+dont_scan = [
+    'doc/guides/rel_notes/',
+    'doc/guides/contributing/coding_style.rst'
+    'doc/guides/prog_guide/glossary.rst'
+]
+
+
+def args_parse():
+    "parse arguments and return the argument object back to main"
+
+    parser = argparse.ArgumentParser(
+        description="Identify word usage not aligned with inclusive naming")
+    parser.add_argument("-c",
+                        "--count",
+                        help="Show the number of lines that match",
+                        action='store_true')
+    parser.add_argument("-d",
+                        "--debug",
+                        default=False,
+                        help="Debug this script",
+                        action='store_true')
+    parser.add_argument("-l",
+                        "--files-with-matches",
+                        help="Show only names of files with hits",
+                        action='store_true')
+    parser.add_argument("-n",
+                        "--line-number",
+                        help="Prefix with line number to matching lines",
+                        action='store_true')
+    # note: tier 0 is "OK to use"
+    parser.add_argument("-t",
+                        "--tier",
+                        type=int,
+                        choices=range(0, 4),
+                        action='append',
+                        help="Show non-conforming words of particular tier")
+    parser.add_argument('-x',
+                        "--exclude",
+                        default=dont_scan,
+                        action='append',
+                        help="Exclude path from scan")
+    parser.add_argument('--url',
+                        default=WORDLIST_URL,
+                        help="URL for the non-inclusive naming word list")
+    parser.add_argument('paths', nargs='*',
+                        help='files and directory to scan')
+
+    return parser.parse_args()
+
+
+def fetch_wordlist(url, tiers):
+    "Read list of words from inclusivenaming.org"
+
+    # The word list is returned as JSON like:
+    # {
+    # "data" :
+    #         [
+    #             {
+    #                 "term": "abort",
+    #                 "tier" : "1",
+    #                 "recommendation": "Replace when possible.",
+    # ...
+    with urlopen(url) as response:
+        entries = json.loads(response.read())['data']
+
+    wordlist = []
+    for item in entries:
+        tier = int(item['tier'])
+        if tiers.count(tier) > 0:
+            # convert minus sign to minus or space regex
+            pattern = item['term'].replace('-', '[- ]')
+            wordlist.append(pattern.lower())
+    return wordlist
+
+
+def git_args(args):
+    "Construct command line based on args"
+
+    # Default to Tier 1, 2 and 3.
+    if args.tier:
+        tiers = args.tier
+    else:
+        tiers = list(range(1, 4))
+
+    wordlist = fetch_wordlist(args.url, tiers)
+    if args.debug:
+        print(f"Matching on: {wordlist}")
+
+    cmd = ['git', 'grep', '-i']
+    if args.files_with_matches:
+        cmd.append('-l')
+    if args.count:
+        cmd.append('-c')
+    if args.line_number:
+        cmd.append('-n')
+    for word in wordlist:
+        cmd.append('-e')
+        cmd.append(word)
+    cmd.append('--')
+    # convert the dont_scan paths to regexp
+    for path in dont_scan:
+        cmd.append(f":^{path}")
+    cmd += args.paths
+    if args.debug:
+        print(cmd)
+    return cmd
+
+
+def main():
+    "decode command line arguments then run setup to run"
+
+    grep = subprocess.run(git_args(args_parse()), check=False)
+    sys.exit(grep.returncode)
+
+
+if __name__ == "__main__":
+    main()
-- 
2.39.2


^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2023-04-19 15:00 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-03-31 20:08 [PATCH] devtools: add script to check for non inclusive naming Stephen Hemminger
2023-04-03 14:47 ` [PATCH v2] " Stephen Hemminger
2023-04-03 23:08   ` Luca Boccassi
2023-04-04  2:17     ` Stephen Hemminger
2023-04-04 22:00       ` Luca Boccassi
2023-04-05  1:23         ` Stephen Hemminger
     [not found] <0230331200824.195294-1-stephen@networkplumber.org>
2023-04-19 15:00 ` [PATCH] " Stephen Hemminger

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).