[PATCH v2 0/4] Add ACVP tool

[PATCH v2 0/4] Add ACVP tool

[Brandon Lo]

v2:
* Added SPDX license identifier and copyright

This adds a tool for easy interaction with the ACVP API. It will
handle downloading test vectors, uploading the result, and fetching
the final verdict of those uploaded results.

Brandon Lo (4):
  tools: add acvp_tool
  tools: add default config file for acvp_tool
  tools: add requirements file for acvp_tool
  doc: add readme file for acvp_tool

 requirements.txt            |   7 +
 tools/acvp/README           |  71 ++++++++
 tools/acvp/__init__.py      |   0
 tools/acvp/acvp_config.json |  23 +++
 tools/acvp/acvp_tool.py     | 319 ++++++++++++++++++++++++++++++++++++
 tools/acvp/requirements.txt |   7 +
 6 files changed, 427 insertions(+)
 create mode 100644 tools/acvp/README
 create mode 100644 tools/acvp/__init__.py
 create mode 100644 tools/acvp/acvp_config.json
 create mode 100644 tools/acvp/acvp_tool.py
 create mode 100644 tools/acvp/requirements.txt

-- 
2.25.1

[PATCH v2 1/4] tools: add acvp_tool

[Brandon Lo]

This tool is used to interact with the ACVP API.

Signed-off-by: Brandon Lo <blo@iol.unh.edu>
---
 requirements.txt        |   7 +
 tools/acvp/__init__.py  |   0
 tools/acvp/acvp_tool.py | 319 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 326 insertions(+)
 create mode 100644 tools/acvp/__init__.py
 create mode 100644 tools/acvp/acvp_tool.py

diff --git a/requirements.txt b/requirements.txt
index f2a6844..48371a7 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,2 +1,9 @@
 git-pw==2.1.0
 whatthepatch==1.0.2
+certifi==2021.10.8
+charset-normalizer==2.0.10
+idna==3.3
+pyotp==2.6.0
+requests==2.27.1
+six==1.16.0
+urllib3==1.26.8
diff --git a/tools/acvp/__init__.py b/tools/acvp/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/tools/acvp/acvp_tool.py b/tools/acvp/acvp_tool.py
new file mode 100644
index 0000000..40d2f2f
--- /dev/null
+++ b/tools/acvp/acvp_tool.py
@@ -0,0 +1,319 @@
+#!/usr/bin/env python3
+
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2022 The University of New Hampshire
+
+import hashlib
+import sys
+import time
+import base64
+import argparse
+import os
+import json
+import logging
+from typing import Tuple, Optional, Any, Dict, List
+
+import pyotp
+import requests
+
+
+class ACVPProxy:
+    def __init__(self, cert_path: str, key_path: str, totp_path: str,
+                 config_path: str):
+        """ACVP Proxy used to abstract API calls.
+
+        @param cert_path: Path to the client certificate.
+        @param key_path: Path to the client key.
+        @param totp_path: Path to the one-time password seed.
+        @param config_path: Path to the configuration for the session.
+        """
+        self.cert: Tuple[str, str] = (cert_path, key_path)
+
+        if None in self.cert:
+            logging.error('Missing certificate/key file.')
+            sys.exit(1)
+
+        self.totp_path: str = totp_path
+        self.login_data: Optional[Dict[str, Any]] = None
+        self.session_data: Optional[Dict[str, Any]] = None
+
+        with open(config_path, 'r') as f:
+            self.config: Any = json.load(f)
+
+    def __get_totp(self) -> str:
+        """Get the current one-time password.
+
+        Uses the totp_path argument when instantiating to
+        read the TOTP seed.
+
+        @return: String containing the password.
+        """
+        with open(self.totp_path, 'r') as f:
+            seed = f.read().strip()
+        base64_seed = base64.b32encode(base64.b64decode(seed)).decode('utf-8')
+        totp = pyotp.TOTP(s=base64_seed, digits=8, digest=hashlib.sha256,
+                          interval=30)
+        return totp.now()
+
+    def __fetch_vector_set(self, url: str) -> Optional[Dict]:
+        """Fetch the vector set object from a URL.
+
+        @param url: URL of the vector set given by NIST.
+        @return: Dictionary of the response from the NIST API.
+        The returned dictionary comes without the session information.
+        """
+        logging.info(f'Fetching vector set {url}')
+        token = self.session_data['jwt']
+        while True:
+            response = requests.get(
+                f'{self.config["url"]}{url}',
+                cert=self.cert,
+                headers={'Authorization': f'Bearer {token}'}
+            )
+            if not response.ok:
+                logging.error(f'Failed to fetch vector set {url}')
+                logging.error(json.dumps(response.json(), indent=4))
+                return None
+
+            vector_set_json = response.json()[1]
+            if 'retry' in vector_set_json:
+                duration = vector_set_json['retry']
+                logging.info(f'Server says retry in {duration} seconds...')
+                time.sleep(duration)
+                continue
+
+            logging.info(f'Downloaded vector set {url}')
+            return vector_set_json
+
+    def login(self) -> bool:
+        """Log into the API server.
+
+        Uses the instance's current TOTP seed and certificate file paths
+        to authenticate with the API.
+
+        If successful, the access token of the account will be stored in
+        the instance.
+
+        @return: True if authentication succeeded, false otherwise.
+        """
+        response = requests.post(
+            url=f'{self.config["url"]}/acvp/v1/login',
+            json=[
+                {'acvVersion': '1.0'},
+                {'password': self.__get_totp()},
+            ],
+            cert=self.cert,
+        )
+
+        if not response.ok:
+            logging.error('Failed to log in.')
+            logging.error(json.dumps(response.json(), indent=4))
+            return False
+
+        self.login_data = response.json()[1]
+        # Renamed 'accessToken' to 'jwt' in the json object
+        # to stay consistent with libacvp
+        self.login_data['jwt'] = self.login_data.pop('accessToken')
+        return True
+
+    def register(self) -> Optional[List[Any]]:
+        """Register a new test session.
+
+        This requires the ACVPProxy instance to be authenticated (use .login).
+
+        @return: If registration succeeded, it will return the list
+        containing session information and vector sets.
+        """
+        if self.login_data is None:
+            logging.error('ACVP proxy cannot register a test session without '
+                          'logging in first.')
+            return None
+
+        response = requests.post(
+            url=f'{self.config["url"]}/acvp/v1/testSessions',
+            json=[
+                {'acvVersion': '1.0'},
+                {
+                    'isSample': False,
+                    'algorithms': self.config['algorithms']
+                }
+            ],
+            cert=self.cert,
+            headers={'Authorization': f'Bearer {self.login_data["jwt"]}'}
+        )
+
+        if not response.ok:
+            logging.error('Unable to register.')
+            logging.error(json.dumps(response.json(), indent=4))
+            return None
+
+        self.session_data = response.json()[1]
+        # Renamed 'accessToken' to 'jwt' in the json object
+        # to stay consistent with libacvp
+        self.session_data['jwt'] = self.session_data.pop('accessToken')
+        write_data = [self.session_data]
+
+        for url in self.session_data['vectorSetUrls']:
+            write_data.append(self.__fetch_vector_set(url))
+
+        return write_data
+
+    def fetch_verdict(self, vector_results: List[Any]) -> List[Any]:
+        """Fetch verdict for a list of vector sets.
+
+        @param vector_results: List of vector set dictionaries with answers.
+        @return: A list containing the session information and verdicts.
+        """
+        session_url = self.session_data['url']
+        write_data = [self.session_data]
+        for _, vector_set in vector_results:
+            vector_set_id = vector_set['vsId']
+            logging.info(f'Downloading verdict for vector set {vector_set_id}')
+            while True:
+                result = requests.get(
+                    f'{self.config["url"]}{session_url}'
+                    f'/vectorSets/{vector_set_id}/results',
+                    cert=self.cert,
+                    headers={
+                        'Authorization': f'Bearer {self.session_data["jwt"]}'
+                    }
+                )
+                version, result_json = result.json()
+                if 'retry' in result_json:
+                    duration = result_json['retry']
+                    logging.info(f'Vector set verdict not ready, waiting '
+                                 f'{duration} seconds...')
+                    time.sleep(duration)
+                    continue
+
+                write_data.append([version, result_json])
+                break
+        return write_data
+
+    def upload(self, vector_sets: List[Any]) -> bool:
+        """Upload the given vector sets.
+
+        @param vector_sets: List of vector set dictionaries with answers.
+        @return: True if uploading succeeded.
+        """
+        has_error = False
+        session_url = self.session_data['url']
+
+        for version, vector_set in vector_sets:
+            response = requests.post(
+                f'{self.config["url"]}{session_url}/vectorSets/'
+                f'{vector_set["vsId"]}/results',
+                json=[version, vector_set],
+                cert=self.cert,
+                headers={'Authorization': f'Bearer {self.session_data["jwt"]}'}
+            )
+
+            if not response.ok:
+                has_error = True
+                logging.error(f'Could not upload vector set response for '
+                              f'vector set ID {vector_set["vsId"]}.')
+                logging.error(json.dumps(response.json(), indent=4))
+                continue
+
+        return not has_error
+
+
+def main(request_path: Optional[str],
+         response_path: Optional[str],
+         verdict_path: Optional[str],
+         do_upload: bool,
+         config_path: str):
+
+    if request_path and response_path:
+        logging.error('You cannot use both a request and a response file.')
+        sys.exit(1)
+
+    if not any([request_path, response_path]):
+        logging.error('You must specify either a request or a response file.')
+        sys.exit(1)
+
+    proxy = ACVPProxy(
+        cert_path=os.getenv('ACVP_CERT_FILE'),
+        key_path=os.getenv('ACVP_KEY_FILE'),
+        totp_path=os.getenv('ACVP_SEED_FILE'),
+        config_path=config_path,
+    )
+
+    logging.info('Attempting to log in...')
+    if not proxy.login():
+        logging.error('Could not log in.')
+        sys.exit(1)
+    logging.info('Successfully logged in.')
+
+    if request_path:
+        logging.info('Creating a new test session and downloading vectors...')
+        test_session = proxy.register()
+        if not test_session:
+            logging.error('Could not create a new test session.')
+            sys.exit(1)
+
+        with open(request_path, 'w') as f:
+            json.dump(test_session, f, indent=4)
+    elif response_path:
+        logging.info('Using response file...')
+        with open(response_path, 'r') as upload_file:
+            upload_json: List[Any] = json.load(upload_file)
+        proxy.session_data = upload_json[0]
+
+        if do_upload:
+            logging.info('Uploading response file...')
+            if not proxy.upload(upload_json[1:]):
+                logging.error('Could not successfully upload results file.')
+                sys.exit(1)
+
+        if verdict_path:
+            logging.info('Fetching verdict...')
+            verdict = proxy.fetch_verdict(upload_json[1:])
+            if not verdict:
+                logging.error('Could not successfully fetch verdict file.')
+                sys.exit(1)
+            with open(verdict_path, 'w') as f:
+                json.dump(verdict, f, indent=4)
+
+    logging.info('Done')
+
+
+if __name__ == '__main__':
+    logging.basicConfig(level=logging.INFO)
+    parser = argparse.ArgumentParser(description='ACVP tool to fetch tests '
+                                                 'and upload results.')
+    parser.add_argument('--request', '-r',
+                        help='Path to download a request file. '
+                             'If specified, the tool start a new test session '
+                             'and download the test information into the '
+                             'given path.')
+    parser.add_argument('--response', '-s',
+                        help='Path of the response file. '
+                             'If specified, the tool will use this file '
+                             'to determine session information. '
+                             'This argument must be used when uploading '
+                             'results or downloading verdicts.')
+    parser.add_argument('--verdict', '-v',
+                        help='Download the verdict to the specified path. '
+                             'If this flag is set, the tool will download the '
+                             'verdict for the given response file '
+                             '(--response).')
+    parser.add_argument('--upload', '-u',
+                        help='Upload the given response file to the API. '
+                             'If this flag is set, the tool will upload the '
+                             'given response file (--response).',
+                        action='store_true')
+    parser.add_argument('--config', '-c',
+                        help='Path of the configuration file. '
+                             '(Default: acvp_config.json)',
+                        default='acvp_config.json')
+
+    args = parser.parse_args()
+
+    main(
+        request_path=args.request,
+        response_path=args.response,
+        verdict_path=args.verdict,
+        do_upload=args.upload,
+        config_path=args.config,
+    )
-- 
2.25.1

Re: [PATCH v2 1/4] tools: add acvp_tool

[Brandon Lo]

On Wed, Jan 26, 2022 at 1:16 PM Brandon Lo <blo@iol.unh.edu> wrote:
> diff --git a/requirements.txt b/requirements.txt
> index f2a6844..48371a7 100644
> --- a/requirements.txt
> +++ b/requirements.txt
> @@ -1,2 +1,9 @@
>  git-pw==2.1.0
>  whatthepatch==1.0.2
> +certifi==2021.10.8
> +charset-normalizer==2.0.10
> +idna==3.3
> +pyotp==2.6.0
> +requests==2.27.1
> +six==1.16.0
> +urllib3==1.26.8

Seems like this change contains some extra changes to the root
requirements file.
I will send a quick v3 so that it doesn't add any extra requirements
to labs that don't use FIPS validation.

-- 
Brandon Lo
UNH InterOperability Laboratory
21 Madbury Rd, Suite 100, Durham, NH 03824
blo@iol.unh.edu
www.iol.unh.edu

[PATCH v3 1/4] tools: add acvp_tool

[Brandon Lo]

This tool is used to interact with the ACVP API.

Signed-off-by: Brandon Lo <blo@iol.unh.edu>
---
v3:
* Removed changes to root requirements

v2:
* Added SPDX and copyright line

 tools/acvp/__init__.py  |   0
 tools/acvp/acvp_tool.py | 319 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 319 insertions(+)
 create mode 100644 tools/acvp/__init__.py
 create mode 100644 tools/acvp/acvp_tool.py

diff --git a/tools/acvp/__init__.py b/tools/acvp/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/tools/acvp/acvp_tool.py b/tools/acvp/acvp_tool.py
new file mode 100644
index 0000000..40d2f2f
--- /dev/null
+++ b/tools/acvp/acvp_tool.py
@@ -0,0 +1,319 @@
+#!/usr/bin/env python3
+
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2022 The University of New Hampshire
+
+import hashlib
+import sys
+import time
+import base64
+import argparse
+import os
+import json
+import logging
+from typing import Tuple, Optional, Any, Dict, List
+
+import pyotp
+import requests
+
+
+class ACVPProxy:
+    def __init__(self, cert_path: str, key_path: str, totp_path: str,
+                 config_path: str):
+        """ACVP Proxy used to abstract API calls.
+
+        @param cert_path: Path to the client certificate.
+        @param key_path: Path to the client key.
+        @param totp_path: Path to the one-time password seed.
+        @param config_path: Path to the configuration for the session.
+        """
+        self.cert: Tuple[str, str] = (cert_path, key_path)
+
+        if None in self.cert:
+            logging.error('Missing certificate/key file.')
+            sys.exit(1)
+
+        self.totp_path: str = totp_path
+        self.login_data: Optional[Dict[str, Any]] = None
+        self.session_data: Optional[Dict[str, Any]] = None
+
+        with open(config_path, 'r') as f:
+            self.config: Any = json.load(f)
+
+    def __get_totp(self) -> str:
+        """Get the current one-time password.
+
+        Uses the totp_path argument when instantiating to
+        read the TOTP seed.
+
+        @return: String containing the password.
+        """
+        with open(self.totp_path, 'r') as f:
+            seed = f.read().strip()
+        base64_seed = base64.b32encode(base64.b64decode(seed)).decode('utf-8')
+        totp = pyotp.TOTP(s=base64_seed, digits=8, digest=hashlib.sha256,
+                          interval=30)
+        return totp.now()
+
+    def __fetch_vector_set(self, url: str) -> Optional[Dict]:
+        """Fetch the vector set object from a URL.
+
+        @param url: URL of the vector set given by NIST.
+        @return: Dictionary of the response from the NIST API.
+        The returned dictionary comes without the session information.
+        """
+        logging.info(f'Fetching vector set {url}')
+        token = self.session_data['jwt']
+        while True:
+            response = requests.get(
+                f'{self.config["url"]}{url}',
+                cert=self.cert,
+                headers={'Authorization': f'Bearer {token}'}
+            )
+            if not response.ok:
+                logging.error(f'Failed to fetch vector set {url}')
+                logging.error(json.dumps(response.json(), indent=4))
+                return None
+
+            vector_set_json = response.json()[1]
+            if 'retry' in vector_set_json:
+                duration = vector_set_json['retry']
+                logging.info(f'Server says retry in {duration} seconds...')
+                time.sleep(duration)
+                continue
+
+            logging.info(f'Downloaded vector set {url}')
+            return vector_set_json
+
+    def login(self) -> bool:
+        """Log into the API server.
+
+        Uses the instance's current TOTP seed and certificate file paths
+        to authenticate with the API.
+
+        If successful, the access token of the account will be stored in
+        the instance.
+
+        @return: True if authentication succeeded, false otherwise.
+        """
+        response = requests.post(
+            url=f'{self.config["url"]}/acvp/v1/login',
+            json=[
+                {'acvVersion': '1.0'},
+                {'password': self.__get_totp()},
+            ],
+            cert=self.cert,
+        )
+
+        if not response.ok:
+            logging.error('Failed to log in.')
+            logging.error(json.dumps(response.json(), indent=4))
+            return False
+
+        self.login_data = response.json()[1]
+        # Renamed 'accessToken' to 'jwt' in the json object
+        # to stay consistent with libacvp
+        self.login_data['jwt'] = self.login_data.pop('accessToken')
+        return True
+
+    def register(self) -> Optional[List[Any]]:
+        """Register a new test session.
+
+        This requires the ACVPProxy instance to be authenticated (use .login).
+
+        @return: If registration succeeded, it will return the list
+        containing session information and vector sets.
+        """
+        if self.login_data is None:
+            logging.error('ACVP proxy cannot register a test session without '
+                          'logging in first.')
+            return None
+
+        response = requests.post(
+            url=f'{self.config["url"]}/acvp/v1/testSessions',
+            json=[
+                {'acvVersion': '1.0'},
+                {
+                    'isSample': False,
+                    'algorithms': self.config['algorithms']
+                }
+            ],
+            cert=self.cert,
+            headers={'Authorization': f'Bearer {self.login_data["jwt"]}'}
+        )
+
+        if not response.ok:
+            logging.error('Unable to register.')
+            logging.error(json.dumps(response.json(), indent=4))
+            return None
+
+        self.session_data = response.json()[1]
+        # Renamed 'accessToken' to 'jwt' in the json object
+        # to stay consistent with libacvp
+        self.session_data['jwt'] = self.session_data.pop('accessToken')
+        write_data = [self.session_data]
+
+        for url in self.session_data['vectorSetUrls']:
+            write_data.append(self.__fetch_vector_set(url))
+
+        return write_data
+
+    def fetch_verdict(self, vector_results: List[Any]) -> List[Any]:
+        """Fetch verdict for a list of vector sets.
+
+        @param vector_results: List of vector set dictionaries with answers.
+        @return: A list containing the session information and verdicts.
+        """
+        session_url = self.session_data['url']
+        write_data = [self.session_data]
+        for _, vector_set in vector_results:
+            vector_set_id = vector_set['vsId']
+            logging.info(f'Downloading verdict for vector set {vector_set_id}')
+            while True:
+                result = requests.get(
+                    f'{self.config["url"]}{session_url}'
+                    f'/vectorSets/{vector_set_id}/results',
+                    cert=self.cert,
+                    headers={
+                        'Authorization': f'Bearer {self.session_data["jwt"]}'
+                    }
+                )
+                version, result_json = result.json()
+                if 'retry' in result_json:
+                    duration = result_json['retry']
+                    logging.info(f'Vector set verdict not ready, waiting '
+                                 f'{duration} seconds...')
+                    time.sleep(duration)
+                    continue
+
+                write_data.append([version, result_json])
+                break
+        return write_data
+
+    def upload(self, vector_sets: List[Any]) -> bool:
+        """Upload the given vector sets.
+
+        @param vector_sets: List of vector set dictionaries with answers.
+        @return: True if uploading succeeded.
+        """
+        has_error = False
+        session_url = self.session_data['url']
+
+        for version, vector_set in vector_sets:
+            response = requests.post(
+                f'{self.config["url"]}{session_url}/vectorSets/'
+                f'{vector_set["vsId"]}/results',
+                json=[version, vector_set],
+                cert=self.cert,
+                headers={'Authorization': f'Bearer {self.session_data["jwt"]}'}
+            )
+
+            if not response.ok:
+                has_error = True
+                logging.error(f'Could not upload vector set response for '
+                              f'vector set ID {vector_set["vsId"]}.')
+                logging.error(json.dumps(response.json(), indent=4))
+                continue
+
+        return not has_error
+
+
+def main(request_path: Optional[str],
+         response_path: Optional[str],
+         verdict_path: Optional[str],
+         do_upload: bool,
+         config_path: str):
+
+    if request_path and response_path:
+        logging.error('You cannot use both a request and a response file.')
+        sys.exit(1)
+
+    if not any([request_path, response_path]):
+        logging.error('You must specify either a request or a response file.')
+        sys.exit(1)
+
+    proxy = ACVPProxy(
+        cert_path=os.getenv('ACVP_CERT_FILE'),
+        key_path=os.getenv('ACVP_KEY_FILE'),
+        totp_path=os.getenv('ACVP_SEED_FILE'),
+        config_path=config_path,
+    )
+
+    logging.info('Attempting to log in...')
+    if not proxy.login():
+        logging.error('Could not log in.')
+        sys.exit(1)
+    logging.info('Successfully logged in.')
+
+    if request_path:
+        logging.info('Creating a new test session and downloading vectors...')
+        test_session = proxy.register()
+        if not test_session:
+            logging.error('Could not create a new test session.')
+            sys.exit(1)
+
+        with open(request_path, 'w') as f:
+            json.dump(test_session, f, indent=4)
+    elif response_path:
+        logging.info('Using response file...')
+        with open(response_path, 'r') as upload_file:
+            upload_json: List[Any] = json.load(upload_file)
+        proxy.session_data = upload_json[0]
+
+        if do_upload:
+            logging.info('Uploading response file...')
+            if not proxy.upload(upload_json[1:]):
+                logging.error('Could not successfully upload results file.')
+                sys.exit(1)
+
+        if verdict_path:
+            logging.info('Fetching verdict...')
+            verdict = proxy.fetch_verdict(upload_json[1:])
+            if not verdict:
+                logging.error('Could not successfully fetch verdict file.')
+                sys.exit(1)
+            with open(verdict_path, 'w') as f:
+                json.dump(verdict, f, indent=4)
+
+    logging.info('Done')
+
+
+if __name__ == '__main__':
+    logging.basicConfig(level=logging.INFO)
+    parser = argparse.ArgumentParser(description='ACVP tool to fetch tests '
+                                                 'and upload results.')
+    parser.add_argument('--request', '-r',
+                        help='Path to download a request file. '
+                             'If specified, the tool start a new test session '
+                             'and download the test information into the '
+                             'given path.')
+    parser.add_argument('--response', '-s',
+                        help='Path of the response file. '
+                             'If specified, the tool will use this file '
+                             'to determine session information. '
+                             'This argument must be used when uploading '
+                             'results or downloading verdicts.')
+    parser.add_argument('--verdict', '-v',
+                        help='Download the verdict to the specified path. '
+                             'If this flag is set, the tool will download the '
+                             'verdict for the given response file '
+                             '(--response).')
+    parser.add_argument('--upload', '-u',
+                        help='Upload the given response file to the API. '
+                             'If this flag is set, the tool will upload the '
+                             'given response file (--response).',
+                        action='store_true')
+    parser.add_argument('--config', '-c',
+                        help='Path of the configuration file. '
+                             '(Default: acvp_config.json)',
+                        default='acvp_config.json')
+
+    args = parser.parse_args()
+
+    main(
+        request_path=args.request,
+        response_path=args.response,
+        verdict_path=args.verdict,
+        do_upload=args.upload,
+        config_path=args.config,
+    )
-- 
2.25.1

[PATCH v2 2/4] tools: add default config file for acvp_tool

[Brandon Lo]

This provides the user with a generic configuration to
get started. It will use the demo server and a basic AES-GCM
test.

Signed-off-by: Brandon Lo <blo@iol.unh.edu>
---
 tools/acvp/acvp_config.json | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100644 tools/acvp/acvp_config.json

diff --git a/tools/acvp/acvp_config.json b/tools/acvp/acvp_config.json
new file mode 100644
index 0000000..9339885
--- /dev/null
+++ b/tools/acvp/acvp_config.json
@@ -0,0 +1,23 @@
+{
+    "url": "https://demo.acvts.nist.gov",
+    "algorithms": [
+        {
+            "algorithm": "ACVP-AES-GCM",
+            "revision": "1.0",
+            "direction": ["encrypt"],
+            "keyLen": [128, 192, 256],
+            "tagLen": [128],
+            "aadLen": [0],
+            "ivGenMode": "8.2.2",
+            "ivGen": "internal",
+            "ivLen": [96],
+            "payloadLen": [
+                {
+                    "max": 65536,
+                    "min": 0,
+                    "increment": 256
+                }
+            ]
+        }
+    ]
+}
\ No newline at end of file
-- 
2.25.1

[PATCH v2 3/4] tools: add requirements file for acvp_tool

[Brandon Lo]

Adds the basic requirements for the acvp_tool script.

Signed-off-by: Brandon Lo <blo@iol.unh.edu>
---
 tools/acvp/requirements.txt | 7 +++++++
 1 file changed, 7 insertions(+)
 create mode 100644 tools/acvp/requirements.txt

diff --git a/tools/acvp/requirements.txt b/tools/acvp/requirements.txt
new file mode 100644
index 0000000..428f06c
--- /dev/null
+++ b/tools/acvp/requirements.txt
@@ -0,0 +1,7 @@
+certifi==2021.10.8
+charset-normalizer==2.0.10
+idna==3.3
+pyotp==2.6.0
+requests==2.27.1
+six==1.16.0
+urllib3==1.26.8
-- 
2.25.1

[PATCH v2 4/4] doc: add readme file for acvp_tool

[Brandon Lo]

This readme file contains instructions to set up
and use the acvp_tool.

Signed-off-by: Brandon Lo <blo@iol.unh.edu>
---
 tools/acvp/README | 71 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 71 insertions(+)
 create mode 100644 tools/acvp/README

diff --git a/tools/acvp/README b/tools/acvp/README
new file mode 100644
index 0000000..0cd3acc
--- /dev/null
+++ b/tools/acvp/README
@@ -0,0 +1,71 @@
+The ACVP tool is a general tool for interacting with the NIST ACVP API
+in order to test different cryptographic implementations.
+
+It produces machine-readable output for parsing in a CI environment.
+
+
+Requirements
+------------
+
+There are also packages you need to download from the requirements.txt file:
+* pyotp
+* requests
+
+The tool expects that you have all the credential files from NIST:
+* Client certificate (usually a .cer file from NIST)
+* Key file for the certificate
+* Time-based one-time password seed file (usually a .txt file from NIST)
+
+The path to each file must be stored in an environment variable:
+$ACVP_SEED_FILE  =  Path to the TOTP seed .txt file    (given by NIST).
+$ACVP_CERT_FILE  =  Path to the client .cer/.crt file  (given by NIST).
+$ACVP_KEY_FILE   =  Path to the certificate key file   (generated by user).
+
+If you do not have the required files from NIST, you must email them
+to create demo credentials.
+https://pages.nist.gov/ACVP/#access
+
+
+Setup
+-----
+
+After setting the environment variables as described in the
+"Requirements" section, you will need to edit the acvp_config.json file.
+
+The acvp_config.json file is expected to be a json object
+containing two keys: "url" and "algorithms"
+
+"url" must be the base URL string of the API you want to use.
+"algorithms" must be an array of algorithm objects as detailed in the
+ACVP API specification here:
+https://github.com/usnistgov/ACVP/wiki/ACVTS-End-User-Documentation
+
+Now you can use the acvp_tool.py script to register a test session,
+upload the results, and download the verdict.
+
+
+Usage
+-----
+
+To see all options available, use the --help flag.
+
+First, register and download a new test session with the tool:
+    acvp_tool.py --request $DOWNLOAD_PATH
+The file written to $DOWNLOAD_PATH will contain both the session information
+and the test vectors.
+
+You should use the DPDK FIPS validation example application to test
+the vectors in this file. The example application will generate
+the result file which is uploaded back to the ACVP API.
+
+After running tests with the vector file, you can submit the result:
+    acvp_tool.py --response $RESULT_PATH --upload
+where $RESULT_PATH is the path of the file containing the answers.
+
+Once you submit your results, you can do
+    acvp_tool.py --response $RESULT_PATH --verdict $VERDICT_PATH
+where $VERDICT_PATH is where you want to save the verdict information.
+The verdict file will contain the result of each test case submitted.
+
+You can also combine the options:
+    acvp_tool.py --response $RESULT_PATH --upload --verdict $VERDICT_PATH
-- 
2.25.1

[PATCH v3 0/4] Add ACVP tool

[Brandon Lo]

v3:
* Remove extra requirements in root requirements file

v2:
* Add SPDX and copyright line

This adds a tool for easy interaction with the ACVP API. It will
handle downloading test vectors, uploading the result, and fetching
the final verdict of those uploaded results.

Brandon Lo (4):
  tools: add acvp_tool
  tools: add default config file for acvp_tool
  tools: add requirements file for acvp_tool
  doc: add readme file for acvp_tool

 tools/acvp/README           |  71 ++++++++
 tools/acvp/__init__.py      |   0
 tools/acvp/acvp_config.json |  23 +++
 tools/acvp/acvp_tool.py     | 319 ++++++++++++++++++++++++++++++++++++
 tools/acvp/requirements.txt |   7 +
 5 files changed, 420 insertions(+)
 create mode 100644 tools/acvp/README
 create mode 100644 tools/acvp/__init__.py
 create mode 100644 tools/acvp/acvp_config.json
 create mode 100644 tools/acvp/acvp_tool.py
 create mode 100644 tools/acvp/requirements.txt

-- 
2.25.1

[PATCH v3 1/4] tools: add acvp_tool

[Brandon Lo]

This tool is used to interact with the ACVP API.

Signed-off-by: Brandon Lo <blo@iol.unh.edu>
---
v3:
* Removed changes to root requirements

v2:
* Added SPDX and copyright line

 tools/acvp/__init__.py  |   0
 tools/acvp/acvp_tool.py | 319 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 319 insertions(+)
 create mode 100644 tools/acvp/__init__.py
 create mode 100644 tools/acvp/acvp_tool.py

diff --git a/tools/acvp/__init__.py b/tools/acvp/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/tools/acvp/acvp_tool.py b/tools/acvp/acvp_tool.py
new file mode 100644
index 0000000..40d2f2f
--- /dev/null
+++ b/tools/acvp/acvp_tool.py
@@ -0,0 +1,319 @@
+#!/usr/bin/env python3
+
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2022 The University of New Hampshire
+
+import hashlib
+import sys
+import time
+import base64
+import argparse
+import os
+import json
+import logging
+from typing import Tuple, Optional, Any, Dict, List
+
+import pyotp
+import requests
+
+
+class ACVPProxy:
+    def __init__(self, cert_path: str, key_path: str, totp_path: str,
+                 config_path: str):
+        """ACVP Proxy used to abstract API calls.
+
+        @param cert_path: Path to the client certificate.
+        @param key_path: Path to the client key.
+        @param totp_path: Path to the one-time password seed.
+        @param config_path: Path to the configuration for the session.
+        """
+        self.cert: Tuple[str, str] = (cert_path, key_path)
+
+        if None in self.cert:
+            logging.error('Missing certificate/key file.')
+            sys.exit(1)
+
+        self.totp_path: str = totp_path
+        self.login_data: Optional[Dict[str, Any]] = None
+        self.session_data: Optional[Dict[str, Any]] = None
+
+        with open(config_path, 'r') as f:
+            self.config: Any = json.load(f)
+
+    def __get_totp(self) -> str:
+        """Get the current one-time password.
+
+        Uses the totp_path argument when instantiating to
+        read the TOTP seed.
+
+        @return: String containing the password.
+        """
+        with open(self.totp_path, 'r') as f:
+            seed = f.read().strip()
+        base64_seed = base64.b32encode(base64.b64decode(seed)).decode('utf-8')
+        totp = pyotp.TOTP(s=base64_seed, digits=8, digest=hashlib.sha256,
+                          interval=30)
+        return totp.now()
+
+    def __fetch_vector_set(self, url: str) -> Optional[Dict]:
+        """Fetch the vector set object from a URL.
+
+        @param url: URL of the vector set given by NIST.
+        @return: Dictionary of the response from the NIST API.
+        The returned dictionary comes without the session information.
+        """
+        logging.info(f'Fetching vector set {url}')
+        token = self.session_data['jwt']
+        while True:
+            response = requests.get(
+                f'{self.config["url"]}{url}',
+                cert=self.cert,
+                headers={'Authorization': f'Bearer {token}'}
+            )
+            if not response.ok:
+                logging.error(f'Failed to fetch vector set {url}')
+                logging.error(json.dumps(response.json(), indent=4))
+                return None
+
+            vector_set_json = response.json()[1]
+            if 'retry' in vector_set_json:
+                duration = vector_set_json['retry']
+                logging.info(f'Server says retry in {duration} seconds...')
+                time.sleep(duration)
+                continue
+
+            logging.info(f'Downloaded vector set {url}')
+            return vector_set_json
+
+    def login(self) -> bool:
+        """Log into the API server.
+
+        Uses the instance's current TOTP seed and certificate file paths
+        to authenticate with the API.
+
+        If successful, the access token of the account will be stored in
+        the instance.
+
+        @return: True if authentication succeeded, false otherwise.
+        """
+        response = requests.post(
+            url=f'{self.config["url"]}/acvp/v1/login',
+            json=[
+                {'acvVersion': '1.0'},
+                {'password': self.__get_totp()},
+            ],
+            cert=self.cert,
+        )
+
+        if not response.ok:
+            logging.error('Failed to log in.')
+            logging.error(json.dumps(response.json(), indent=4))
+            return False
+
+        self.login_data = response.json()[1]
+        # Renamed 'accessToken' to 'jwt' in the json object
+        # to stay consistent with libacvp
+        self.login_data['jwt'] = self.login_data.pop('accessToken')
+        return True
+
+    def register(self) -> Optional[List[Any]]:
+        """Register a new test session.
+
+        This requires the ACVPProxy instance to be authenticated (use .login).
+
+        @return: If registration succeeded, it will return the list
+        containing session information and vector sets.
+        """
+        if self.login_data is None:
+            logging.error('ACVP proxy cannot register a test session without '
+                          'logging in first.')
+            return None
+
+        response = requests.post(
+            url=f'{self.config["url"]}/acvp/v1/testSessions',
+            json=[
+                {'acvVersion': '1.0'},
+                {
+                    'isSample': False,
+                    'algorithms': self.config['algorithms']
+                }
+            ],
+            cert=self.cert,
+            headers={'Authorization': f'Bearer {self.login_data["jwt"]}'}
+        )
+
+        if not response.ok:
+            logging.error('Unable to register.')
+            logging.error(json.dumps(response.json(), indent=4))
+            return None
+
+        self.session_data = response.json()[1]
+        # Renamed 'accessToken' to 'jwt' in the json object
+        # to stay consistent with libacvp
+        self.session_data['jwt'] = self.session_data.pop('accessToken')
+        write_data = [self.session_data]
+
+        for url in self.session_data['vectorSetUrls']:
+            write_data.append(self.__fetch_vector_set(url))
+
+        return write_data
+
+    def fetch_verdict(self, vector_results: List[Any]) -> List[Any]:
+        """Fetch verdict for a list of vector sets.
+
+        @param vector_results: List of vector set dictionaries with answers.
+        @return: A list containing the session information and verdicts.
+        """
+        session_url = self.session_data['url']
+        write_data = [self.session_data]
+        for _, vector_set in vector_results:
+            vector_set_id = vector_set['vsId']
+            logging.info(f'Downloading verdict for vector set {vector_set_id}')
+            while True:
+                result = requests.get(
+                    f'{self.config["url"]}{session_url}'
+                    f'/vectorSets/{vector_set_id}/results',
+                    cert=self.cert,
+                    headers={
+                        'Authorization': f'Bearer {self.session_data["jwt"]}'
+                    }
+                )
+                version, result_json = result.json()
+                if 'retry' in result_json:
+                    duration = result_json['retry']
+                    logging.info(f'Vector set verdict not ready, waiting '
+                                 f'{duration} seconds...')
+                    time.sleep(duration)
+                    continue
+
+                write_data.append([version, result_json])
+                break
+        return write_data
+
+    def upload(self, vector_sets: List[Any]) -> bool:
+        """Upload the given vector sets.
+
+        @param vector_sets: List of vector set dictionaries with answers.
+        @return: True if uploading succeeded.
+        """
+        has_error = False
+        session_url = self.session_data['url']
+
+        for version, vector_set in vector_sets:
+            response = requests.post(
+                f'{self.config["url"]}{session_url}/vectorSets/'
+                f'{vector_set["vsId"]}/results',
+                json=[version, vector_set],
+                cert=self.cert,
+                headers={'Authorization': f'Bearer {self.session_data["jwt"]}'}
+            )
+
+            if not response.ok:
+                has_error = True
+                logging.error(f'Could not upload vector set response for '
+                              f'vector set ID {vector_set["vsId"]}.')
+                logging.error(json.dumps(response.json(), indent=4))
+                continue
+
+        return not has_error
+
+
+def main(request_path: Optional[str],
+         response_path: Optional[str],
+         verdict_path: Optional[str],
+         do_upload: bool,
+         config_path: str):
+
+    if request_path and response_path:
+        logging.error('You cannot use both a request and a response file.')
+        sys.exit(1)
+
+    if not any([request_path, response_path]):
+        logging.error('You must specify either a request or a response file.')
+        sys.exit(1)
+
+    proxy = ACVPProxy(
+        cert_path=os.getenv('ACVP_CERT_FILE'),
+        key_path=os.getenv('ACVP_KEY_FILE'),
+        totp_path=os.getenv('ACVP_SEED_FILE'),
+        config_path=config_path,
+    )
+
+    logging.info('Attempting to log in...')
+    if not proxy.login():
+        logging.error('Could not log in.')
+        sys.exit(1)
+    logging.info('Successfully logged in.')
+
+    if request_path:
+        logging.info('Creating a new test session and downloading vectors...')
+        test_session = proxy.register()
+        if not test_session:
+            logging.error('Could not create a new test session.')
+            sys.exit(1)
+
+        with open(request_path, 'w') as f:
+            json.dump(test_session, f, indent=4)
+    elif response_path:
+        logging.info('Using response file...')
+        with open(response_path, 'r') as upload_file:
+            upload_json: List[Any] = json.load(upload_file)
+        proxy.session_data = upload_json[0]
+
+        if do_upload:
+            logging.info('Uploading response file...')
+            if not proxy.upload(upload_json[1:]):
+                logging.error('Could not successfully upload results file.')
+                sys.exit(1)
+
+        if verdict_path:
+            logging.info('Fetching verdict...')
+            verdict = proxy.fetch_verdict(upload_json[1:])
+            if not verdict:
+                logging.error('Could not successfully fetch verdict file.')
+                sys.exit(1)
+            with open(verdict_path, 'w') as f:
+                json.dump(verdict, f, indent=4)
+
+    logging.info('Done')
+
+
+if __name__ == '__main__':
+    logging.basicConfig(level=logging.INFO)
+    parser = argparse.ArgumentParser(description='ACVP tool to fetch tests '
+                                                 'and upload results.')
+    parser.add_argument('--request', '-r',
+                        help='Path to download a request file. '
+                             'If specified, the tool start a new test session '
+                             'and download the test information into the '
+                             'given path.')
+    parser.add_argument('--response', '-s',
+                        help='Path of the response file. '
+                             'If specified, the tool will use this file '
+                             'to determine session information. '
+                             'This argument must be used when uploading '
+                             'results or downloading verdicts.')
+    parser.add_argument('--verdict', '-v',
+                        help='Download the verdict to the specified path. '
+                             'If this flag is set, the tool will download the '
+                             'verdict for the given response file '
+                             '(--response).')
+    parser.add_argument('--upload', '-u',
+                        help='Upload the given response file to the API. '
+                             'If this flag is set, the tool will upload the '
+                             'given response file (--response).',
+                        action='store_true')
+    parser.add_argument('--config', '-c',
+                        help='Path of the configuration file. '
+                             '(Default: acvp_config.json)',
+                        default='acvp_config.json')
+
+    args = parser.parse_args()
+
+    main(
+        request_path=args.request,
+        response_path=args.response,
+        verdict_path=args.verdict,
+        do_upload=args.upload,
+        config_path=args.config,
+    )
-- 
2.25.1

RE: [PATCH v3 1/4] tools: add acvp_tool

[Ali Alnubani]

> -----Original Message-----
> From: Brandon Lo <blo@iol.unh.edu>
> Sent: Wednesday, February 2, 2022 5:05 PM
> To: NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>
> Cc: ci@dpdk.org; Brandon Lo <blo@iol.unh.edu>
> Subject: [PATCH v3 1/4] tools: add acvp_tool
> 
> This tool is used to interact with the ACVP API.
> 
> Signed-off-by: Brandon Lo <blo@iol.unh.edu>
> ---
> v3:
> * Removed changes to root requirements
> 
> v2:
> * Added SPDX and copyright line
> 
>  tools/acvp/__init__.py  |   0
>  tools/acvp/acvp_tool.py | 319
> ++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 319 insertions(+)
>  create mode 100644 tools/acvp/__init__.py
>  create mode 100644 tools/acvp/acvp_tool.py
> 

Should acvp_tool.py have executable permissions?

[PATCH v3 2/4] tools: add default config file for acvp_tool

[Brandon Lo]

This provides the user with a generic configuration to
get started. It will use the demo server and a basic AES-GCM
test.

Signed-off-by: Brandon Lo <blo@iol.unh.edu>
---
 tools/acvp/acvp_config.json | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100644 tools/acvp/acvp_config.json

diff --git a/tools/acvp/acvp_config.json b/tools/acvp/acvp_config.json
new file mode 100644
index 0000000..9339885
--- /dev/null
+++ b/tools/acvp/acvp_config.json
@@ -0,0 +1,23 @@
+{
+    "url": "https://demo.acvts.nist.gov",
+    "algorithms": [
+        {
+            "algorithm": "ACVP-AES-GCM",
+            "revision": "1.0",
+            "direction": ["encrypt"],
+            "keyLen": [128, 192, 256],
+            "tagLen": [128],
+            "aadLen": [0],
+            "ivGenMode": "8.2.2",
+            "ivGen": "internal",
+            "ivLen": [96],
+            "payloadLen": [
+                {
+                    "max": 65536,
+                    "min": 0,
+                    "increment": 256
+                }
+            ]
+        }
+    ]
+}
\ No newline at end of file
-- 
2.25.1

[PATCH v3 3/4] tools: add requirements file for acvp_tool

[Brandon Lo]

Adds the basic requirements for the acvp_tool script.

Signed-off-by: Brandon Lo <blo@iol.unh.edu>
---
 tools/acvp/requirements.txt | 7 +++++++
 1 file changed, 7 insertions(+)
 create mode 100644 tools/acvp/requirements.txt

diff --git a/tools/acvp/requirements.txt b/tools/acvp/requirements.txt
new file mode 100644
index 0000000..428f06c
--- /dev/null
+++ b/tools/acvp/requirements.txt
@@ -0,0 +1,7 @@
+certifi==2021.10.8
+charset-normalizer==2.0.10
+idna==3.3
+pyotp==2.6.0
+requests==2.27.1
+six==1.16.0
+urllib3==1.26.8
-- 
2.25.1

[PATCH v3 4/4] doc: add readme file for acvp_tool

[Brandon Lo]

This readme file contains instructions to set up
and use the acvp_tool.

Signed-off-by: Brandon Lo <blo@iol.unh.edu>
---
 tools/acvp/README | 71 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 71 insertions(+)
 create mode 100644 tools/acvp/README

diff --git a/tools/acvp/README b/tools/acvp/README
new file mode 100644
index 0000000..0cd3acc
--- /dev/null
+++ b/tools/acvp/README
@@ -0,0 +1,71 @@
+The ACVP tool is a general tool for interacting with the NIST ACVP API
+in order to test different cryptographic implementations.
+
+It produces machine-readable output for parsing in a CI environment.
+
+
+Requirements
+------------
+
+There are also packages you need to download from the requirements.txt file:
+* pyotp
+* requests
+
+The tool expects that you have all the credential files from NIST:
+* Client certificate (usually a .cer file from NIST)
+* Key file for the certificate
+* Time-based one-time password seed file (usually a .txt file from NIST)
+
+The path to each file must be stored in an environment variable:
+$ACVP_SEED_FILE  =  Path to the TOTP seed .txt file    (given by NIST).
+$ACVP_CERT_FILE  =  Path to the client .cer/.crt file  (given by NIST).
+$ACVP_KEY_FILE   =  Path to the certificate key file   (generated by user).
+
+If you do not have the required files from NIST, you must email them
+to create demo credentials.
+https://pages.nist.gov/ACVP/#access
+
+
+Setup
+-----
+
+After setting the environment variables as described in the
+"Requirements" section, you will need to edit the acvp_config.json file.
+
+The acvp_config.json file is expected to be a json object
+containing two keys: "url" and "algorithms"
+
+"url" must be the base URL string of the API you want to use.
+"algorithms" must be an array of algorithm objects as detailed in the
+ACVP API specification here:
+https://github.com/usnistgov/ACVP/wiki/ACVTS-End-User-Documentation
+
+Now you can use the acvp_tool.py script to register a test session,
+upload the results, and download the verdict.
+
+
+Usage
+-----
+
+To see all options available, use the --help flag.
+
+First, register and download a new test session with the tool:
+    acvp_tool.py --request $DOWNLOAD_PATH
+The file written to $DOWNLOAD_PATH will contain both the session information
+and the test vectors.
+
+You should use the DPDK FIPS validation example application to test
+the vectors in this file. The example application will generate
+the result file which is uploaded back to the ACVP API.
+
+After running tests with the vector file, you can submit the result:
+    acvp_tool.py --response $RESULT_PATH --upload
+where $RESULT_PATH is the path of the file containing the answers.
+
+Once you submit your results, you can do
+    acvp_tool.py --response $RESULT_PATH --verdict $VERDICT_PATH
+where $VERDICT_PATH is where you want to save the verdict information.
+The verdict file will contain the result of each test case submitted.
+
+You can also combine the options:
+    acvp_tool.py --response $RESULT_PATH --upload --verdict $VERDICT_PATH
-- 
2.25.1

Re: [PATCH v3 0/4] Add ACVP tool

[Brandon Lo]

On Wed, Feb 2, 2022 at 10:04 AM Brandon Lo <blo@iol.unh.edu> wrote:
>
> v3:
> * Remove extra requirements in root requirements file
>
> v2:
> * Add SPDX and copyright line
>
> This adds a tool for easy interaction with the ACVP API. It will
> handle downloading test vectors, uploading the result, and fetching
> the final verdict of those uploaded results.
>
> Brandon Lo (4):
>   tools: add acvp_tool
>   tools: add default config file for acvp_tool
>   tools: add requirements file for acvp_tool
>   doc: add readme file for acvp_tool
>
>  tools/acvp/README           |  71 ++++++++
>  tools/acvp/__init__.py      |   0
>  tools/acvp/acvp_config.json |  23 +++
>  tools/acvp/acvp_tool.py     | 319 ++++++++++++++++++++++++++++++++++++
>  tools/acvp/requirements.txt |   7 +
>  5 files changed, 420 insertions(+)
>  create mode 100644 tools/acvp/README
>  create mode 100644 tools/acvp/__init__.py
>  create mode 100644 tools/acvp/acvp_config.json
>  create mode 100644 tools/acvp/acvp_tool.py
>  create mode 100644 tools/acvp/requirements.txt
>
> --
> 2.25.1
>

Adding Ali and Aaron to the discussion.

-- 
Brandon Lo
UNH InterOperability Laboratory
21 Madbury Rd, Suite 100, Durham, NH 03824
blo@iol.unh.edu
www.iol.unh.edu

RE: [PATCH v3 0/4] Add ACVP tool

[Ali Alnubani]

> -----Original Message-----
> From: Brandon Lo <blo@iol.unh.edu>
> Sent: Wednesday, February 2, 2022 5:05 PM
> To: NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>
> Cc: ci@dpdk.org; Brandon Lo <blo@iol.unh.edu>
> Subject: [PATCH v3 0/4] Add ACVP tool
> 
> v3:
> * Remove extra requirements in root requirements file
> 
> v2:
> * Add SPDX and copyright line
> 
> This adds a tool for easy interaction with the ACVP API. It will
> handle downloading test vectors, uploading the result, and fetching
> the final verdict of those uploaded results.
> 
> Brandon Lo (4):
>   tools: add acvp_tool
>   tools: add default config file for acvp_tool
>   tools: add requirements file for acvp_tool
>   doc: add readme file for acvp_tool
> 
>  tools/acvp/README           |  71 ++++++++
>  tools/acvp/__init__.py      |   0
>  tools/acvp/acvp_config.json |  23 +++
>  tools/acvp/acvp_tool.py     | 319
> ++++++++++++++++++++++++++++++++++++
>  tools/acvp/requirements.txt |   7 +
>  5 files changed, 420 insertions(+)
>  create mode 100644 tools/acvp/README
>  create mode 100644 tools/acvp/__init__.py
>  create mode 100644 tools/acvp/acvp_config.json
>  create mode 100644 tools/acvp/acvp_tool.py
>  create mode 100644 tools/acvp/requirements.txt
> 
> --
> 2.25.1

Acked-by: Ali Alnubani <alialnu@nvidia.com>

[PATCH v4 0/4] Add ACVP tool

[Brandon Lo]

v4:
* Make acvp_tool.py executable

v3:
* Remove extra requirements in root requirements file

v2:
* Add SPDX and copyright line

This adds a tool for easy interaction with the ACVP API. It will
handle downloading test vectors, uploading the result, and fetching
the final verdict of those uploaded results.

Brandon Lo (4):
  tools: add acvp_tool
  tools: add default config file for acvp_tool
  tools: add requirements file for acvp_tool
  doc: add readme file for acvp_tool

 tools/acvp/README           |  71 ++++++++
 tools/acvp/__init__.py      |   0
 tools/acvp/acvp_config.json |  23 +++
 tools/acvp/acvp_tool.py     | 319 ++++++++++++++++++++++++++++++++++++
 tools/acvp/requirements.txt |   7 +
 5 files changed, 420 insertions(+)
 create mode 100644 tools/acvp/README
 create mode 100644 tools/acvp/__init__.py
 create mode 100644 tools/acvp/acvp_config.json
 create mode 100755 tools/acvp/acvp_tool.py
 create mode 100644 tools/acvp/requirements.txt

-- 
2.25.1

[PATCH v4 1/4] tools: add acvp_tool

[Brandon Lo]

This tool is used to interact with the ACVP API.

Signed-off-by: Brandon Lo <blo@iol.unh.edu>
---
v4:
* Make acvp_tool.py executable

v3:
* Removed changes to root requirements

v2:
* Added SPDX and copyright line

 tools/acvp/__init__.py  |   0
 tools/acvp/acvp_tool.py | 319 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 319 insertions(+)
 create mode 100644 tools/acvp/__init__.py
 create mode 100755 tools/acvp/acvp_tool.py

diff --git a/tools/acvp/__init__.py b/tools/acvp/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/tools/acvp/acvp_tool.py b/tools/acvp/acvp_tool.py
new file mode 100755
index 0000000..40d2f2f
--- /dev/null
+++ b/tools/acvp/acvp_tool.py
@@ -0,0 +1,319 @@
+#!/usr/bin/env python3
+
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2022 The University of New Hampshire
+
+import hashlib
+import sys
+import time
+import base64
+import argparse
+import os
+import json
+import logging
+from typing import Tuple, Optional, Any, Dict, List
+
+import pyotp
+import requests
+
+
+class ACVPProxy:
+    def __init__(self, cert_path: str, key_path: str, totp_path: str,
+                 config_path: str):
+        """ACVP Proxy used to abstract API calls.
+
+        @param cert_path: Path to the client certificate.
+        @param key_path: Path to the client key.
+        @param totp_path: Path to the one-time password seed.
+        @param config_path: Path to the configuration for the session.
+        """
+        self.cert: Tuple[str, str] = (cert_path, key_path)
+
+        if None in self.cert:
+            logging.error('Missing certificate/key file.')
+            sys.exit(1)
+
+        self.totp_path: str = totp_path
+        self.login_data: Optional[Dict[str, Any]] = None
+        self.session_data: Optional[Dict[str, Any]] = None
+
+        with open(config_path, 'r') as f:
+            self.config: Any = json.load(f)
+
+    def __get_totp(self) -> str:
+        """Get the current one-time password.
+
+        Uses the totp_path argument when instantiating to
+        read the TOTP seed.
+
+        @return: String containing the password.
+        """
+        with open(self.totp_path, 'r') as f:
+            seed = f.read().strip()
+        base64_seed = base64.b32encode(base64.b64decode(seed)).decode('utf-8')
+        totp = pyotp.TOTP(s=base64_seed, digits=8, digest=hashlib.sha256,
+                          interval=30)
+        return totp.now()
+
+    def __fetch_vector_set(self, url: str) -> Optional[Dict]:
+        """Fetch the vector set object from a URL.
+
+        @param url: URL of the vector set given by NIST.
+        @return: Dictionary of the response from the NIST API.
+        The returned dictionary comes without the session information.
+        """
+        logging.info(f'Fetching vector set {url}')
+        token = self.session_data['jwt']
+        while True:
+            response = requests.get(
+                f'{self.config["url"]}{url}',
+                cert=self.cert,
+                headers={'Authorization': f'Bearer {token}'}
+            )
+            if not response.ok:
+                logging.error(f'Failed to fetch vector set {url}')
+                logging.error(json.dumps(response.json(), indent=4))
+                return None
+
+            vector_set_json = response.json()[1]
+            if 'retry' in vector_set_json:
+                duration = vector_set_json['retry']
+                logging.info(f'Server says retry in {duration} seconds...')
+                time.sleep(duration)
+                continue
+
+            logging.info(f'Downloaded vector set {url}')
+            return vector_set_json
+
+    def login(self) -> bool:
+        """Log into the API server.
+
+        Uses the instance's current TOTP seed and certificate file paths
+        to authenticate with the API.
+
+        If successful, the access token of the account will be stored in
+        the instance.
+
+        @return: True if authentication succeeded, false otherwise.
+        """
+        response = requests.post(
+            url=f'{self.config["url"]}/acvp/v1/login',
+            json=[
+                {'acvVersion': '1.0'},
+                {'password': self.__get_totp()},
+            ],
+            cert=self.cert,
+        )
+
+        if not response.ok:
+            logging.error('Failed to log in.')
+            logging.error(json.dumps(response.json(), indent=4))
+            return False
+
+        self.login_data = response.json()[1]
+        # Renamed 'accessToken' to 'jwt' in the json object
+        # to stay consistent with libacvp
+        self.login_data['jwt'] = self.login_data.pop('accessToken')
+        return True
+
+    def register(self) -> Optional[List[Any]]:
+        """Register a new test session.
+
+        This requires the ACVPProxy instance to be authenticated (use .login).
+
+        @return: If registration succeeded, it will return the list
+        containing session information and vector sets.
+        """
+        if self.login_data is None:
+            logging.error('ACVP proxy cannot register a test session without '
+                          'logging in first.')
+            return None
+
+        response = requests.post(
+            url=f'{self.config["url"]}/acvp/v1/testSessions',
+            json=[
+                {'acvVersion': '1.0'},
+                {
+                    'isSample': False,
+                    'algorithms': self.config['algorithms']
+                }
+            ],
+            cert=self.cert,
+            headers={'Authorization': f'Bearer {self.login_data["jwt"]}'}
+        )
+
+        if not response.ok:
+            logging.error('Unable to register.')
+            logging.error(json.dumps(response.json(), indent=4))
+            return None
+
+        self.session_data = response.json()[1]
+        # Renamed 'accessToken' to 'jwt' in the json object
+        # to stay consistent with libacvp
+        self.session_data['jwt'] = self.session_data.pop('accessToken')
+        write_data = [self.session_data]
+
+        for url in self.session_data['vectorSetUrls']:
+            write_data.append(self.__fetch_vector_set(url))
+
+        return write_data
+
+    def fetch_verdict(self, vector_results: List[Any]) -> List[Any]:
+        """Fetch verdict for a list of vector sets.
+
+        @param vector_results: List of vector set dictionaries with answers.
+        @return: A list containing the session information and verdicts.
+        """
+        session_url = self.session_data['url']
+        write_data = [self.session_data]
+        for _, vector_set in vector_results:
+            vector_set_id = vector_set['vsId']
+            logging.info(f'Downloading verdict for vector set {vector_set_id}')
+            while True:
+                result = requests.get(
+                    f'{self.config["url"]}{session_url}'
+                    f'/vectorSets/{vector_set_id}/results',
+                    cert=self.cert,
+                    headers={
+                        'Authorization': f'Bearer {self.session_data["jwt"]}'
+                    }
+                )
+                version, result_json = result.json()
+                if 'retry' in result_json:
+                    duration = result_json['retry']
+                    logging.info(f'Vector set verdict not ready, waiting '
+                                 f'{duration} seconds...')
+                    time.sleep(duration)
+                    continue
+
+                write_data.append([version, result_json])
+                break
+        return write_data
+
+    def upload(self, vector_sets: List[Any]) -> bool:
+        """Upload the given vector sets.
+
+        @param vector_sets: List of vector set dictionaries with answers.
+        @return: True if uploading succeeded.
+        """
+        has_error = False
+        session_url = self.session_data['url']
+
+        for version, vector_set in vector_sets:
+            response = requests.post(
+                f'{self.config["url"]}{session_url}/vectorSets/'
+                f'{vector_set["vsId"]}/results',
+                json=[version, vector_set],
+                cert=self.cert,
+                headers={'Authorization': f'Bearer {self.session_data["jwt"]}'}
+            )
+
+            if not response.ok:
+                has_error = True
+                logging.error(f'Could not upload vector set response for '
+                              f'vector set ID {vector_set["vsId"]}.')
+                logging.error(json.dumps(response.json(), indent=4))
+                continue
+
+        return not has_error
+
+
+def main(request_path: Optional[str],
+         response_path: Optional[str],
+         verdict_path: Optional[str],
+         do_upload: bool,
+         config_path: str):
+
+    if request_path and response_path:
+        logging.error('You cannot use both a request and a response file.')
+        sys.exit(1)
+
+    if not any([request_path, response_path]):
+        logging.error('You must specify either a request or a response file.')
+        sys.exit(1)
+
+    proxy = ACVPProxy(
+        cert_path=os.getenv('ACVP_CERT_FILE'),
+        key_path=os.getenv('ACVP_KEY_FILE'),
+        totp_path=os.getenv('ACVP_SEED_FILE'),
+        config_path=config_path,
+    )
+
+    logging.info('Attempting to log in...')
+    if not proxy.login():
+        logging.error('Could not log in.')
+        sys.exit(1)
+    logging.info('Successfully logged in.')
+
+    if request_path:
+        logging.info('Creating a new test session and downloading vectors...')
+        test_session = proxy.register()
+        if not test_session:
+            logging.error('Could not create a new test session.')
+            sys.exit(1)
+
+        with open(request_path, 'w') as f:
+            json.dump(test_session, f, indent=4)
+    elif response_path:
+        logging.info('Using response file...')
+        with open(response_path, 'r') as upload_file:
+            upload_json: List[Any] = json.load(upload_file)
+        proxy.session_data = upload_json[0]
+
+        if do_upload:
+            logging.info('Uploading response file...')
+            if not proxy.upload(upload_json[1:]):
+                logging.error('Could not successfully upload results file.')
+                sys.exit(1)
+
+        if verdict_path:
+            logging.info('Fetching verdict...')
+            verdict = proxy.fetch_verdict(upload_json[1:])
+            if not verdict:
+                logging.error('Could not successfully fetch verdict file.')
+                sys.exit(1)
+            with open(verdict_path, 'w') as f:
+                json.dump(verdict, f, indent=4)
+
+    logging.info('Done')
+
+
+if __name__ == '__main__':
+    logging.basicConfig(level=logging.INFO)
+    parser = argparse.ArgumentParser(description='ACVP tool to fetch tests '
+                                                 'and upload results.')
+    parser.add_argument('--request', '-r',
+                        help='Path to download a request file. '
+                             'If specified, the tool start a new test session '
+                             'and download the test information into the '
+                             'given path.')
+    parser.add_argument('--response', '-s',
+                        help='Path of the response file. '
+                             'If specified, the tool will use this file '
+                             'to determine session information. '
+                             'This argument must be used when uploading '
+                             'results or downloading verdicts.')
+    parser.add_argument('--verdict', '-v',
+                        help='Download the verdict to the specified path. '
+                             'If this flag is set, the tool will download the '
+                             'verdict for the given response file '
+                             '(--response).')
+    parser.add_argument('--upload', '-u',
+                        help='Upload the given response file to the API. '
+                             'If this flag is set, the tool will upload the '
+                             'given response file (--response).',
+                        action='store_true')
+    parser.add_argument('--config', '-c',
+                        help='Path of the configuration file. '
+                             '(Default: acvp_config.json)',
+                        default='acvp_config.json')
+
+    args = parser.parse_args()
+
+    main(
+        request_path=args.request,
+        response_path=args.response,
+        verdict_path=args.verdict,
+        do_upload=args.upload,
+        config_path=args.config,
+    )
-- 
2.25.1

[PATCH v4 2/4] tools: add default config file for acvp_tool

[Brandon Lo]

This provides the user with a generic configuration to
get started. It will use the demo server and a basic AES-GCM
test.

Signed-off-by: Brandon Lo <blo@iol.unh.edu>
---
 tools/acvp/acvp_config.json | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100644 tools/acvp/acvp_config.json

diff --git a/tools/acvp/acvp_config.json b/tools/acvp/acvp_config.json
new file mode 100644
index 0000000..9339885
--- /dev/null
+++ b/tools/acvp/acvp_config.json
@@ -0,0 +1,23 @@
+{
+    "url": "https://demo.acvts.nist.gov",
+    "algorithms": [
+        {
+            "algorithm": "ACVP-AES-GCM",
+            "revision": "1.0",
+            "direction": ["encrypt"],
+            "keyLen": [128, 192, 256],
+            "tagLen": [128],
+            "aadLen": [0],
+            "ivGenMode": "8.2.2",
+            "ivGen": "internal",
+            "ivLen": [96],
+            "payloadLen": [
+                {
+                    "max": 65536,
+                    "min": 0,
+                    "increment": 256
+                }
+            ]
+        }
+    ]
+}
\ No newline at end of file
-- 
2.25.1

[PATCH v4 3/4] tools: add requirements file for acvp_tool

[Brandon Lo]

Adds the basic requirements for the acvp_tool script.

Signed-off-by: Brandon Lo <blo@iol.unh.edu>
---
 tools/acvp/requirements.txt | 7 +++++++
 1 file changed, 7 insertions(+)
 create mode 100644 tools/acvp/requirements.txt

diff --git a/tools/acvp/requirements.txt b/tools/acvp/requirements.txt
new file mode 100644
index 0000000..428f06c
--- /dev/null
+++ b/tools/acvp/requirements.txt
@@ -0,0 +1,7 @@
+certifi==2021.10.8
+charset-normalizer==2.0.10
+idna==3.3
+pyotp==2.6.0
+requests==2.27.1
+six==1.16.0
+urllib3==1.26.8
-- 
2.25.1

[PATCH v4 4/4] doc: add readme file for acvp_tool

[Brandon Lo]

This readme file contains instructions to set up
and use the acvp_tool.

Signed-off-by: Brandon Lo <blo@iol.unh.edu>
---
 tools/acvp/README | 71 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 71 insertions(+)
 create mode 100644 tools/acvp/README

diff --git a/tools/acvp/README b/tools/acvp/README
new file mode 100644
index 0000000..0cd3acc
--- /dev/null
+++ b/tools/acvp/README
@@ -0,0 +1,71 @@
+The ACVP tool is a general tool for interacting with the NIST ACVP API
+in order to test different cryptographic implementations.
+
+It produces machine-readable output for parsing in a CI environment.
+
+
+Requirements
+------------
+
+There are also packages you need to download from the requirements.txt file:
+* pyotp
+* requests
+
+The tool expects that you have all the credential files from NIST:
+* Client certificate (usually a .cer file from NIST)
+* Key file for the certificate
+* Time-based one-time password seed file (usually a .txt file from NIST)
+
+The path to each file must be stored in an environment variable:
+$ACVP_SEED_FILE  =  Path to the TOTP seed .txt file    (given by NIST).
+$ACVP_CERT_FILE  =  Path to the client .cer/.crt file  (given by NIST).
+$ACVP_KEY_FILE   =  Path to the certificate key file   (generated by user).
+
+If you do not have the required files from NIST, you must email them
+to create demo credentials.
+https://pages.nist.gov/ACVP/#access
+
+
+Setup
+-----
+
+After setting the environment variables as described in the
+"Requirements" section, you will need to edit the acvp_config.json file.
+
+The acvp_config.json file is expected to be a json object
+containing two keys: "url" and "algorithms"
+
+"url" must be the base URL string of the API you want to use.
+"algorithms" must be an array of algorithm objects as detailed in the
+ACVP API specification here:
+https://github.com/usnistgov/ACVP/wiki/ACVTS-End-User-Documentation
+
+Now you can use the acvp_tool.py script to register a test session,
+upload the results, and download the verdict.
+
+
+Usage
+-----
+
+To see all options available, use the --help flag.
+
+First, register and download a new test session with the tool:
+    acvp_tool.py --request $DOWNLOAD_PATH
+The file written to $DOWNLOAD_PATH will contain both the session information
+and the test vectors.
+
+You should use the DPDK FIPS validation example application to test
+the vectors in this file. The example application will generate
+the result file which is uploaded back to the ACVP API.
+
+After running tests with the vector file, you can submit the result:
+    acvp_tool.py --response $RESULT_PATH --upload
+where $RESULT_PATH is the path of the file containing the answers.
+
+Once you submit your results, you can do
+    acvp_tool.py --response $RESULT_PATH --verdict $VERDICT_PATH
+where $VERDICT_PATH is where you want to save the verdict information.
+The verdict file will contain the result of each test case submitted.
+
+You can also combine the options:
+    acvp_tool.py --response $RESULT_PATH --upload --verdict $VERDICT_PATH
-- 
2.25.1