[PATCH v8 0/1] Add ACVP Tool

[PATCH v8 0/1] Add ACVP Tool

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

v1: https://mails.dpdk.org/archives/ci/2022-January/001599.html
v2: https://mails.dpdk.org/archives/ci/2022-January/001611.html
v3: https://mails.dpdk.org/archives/ci/2022-February/001636.html
v4: https://mails.dpdk.org/archives/ci/2022-April/001702.html
v5: https://mails.dpdk.org/archives/ci/2023-March/002112.html
v6: https://mails.dpdk.org/archives/ci/2023-March/002147.html
v7: https://mails.dpdk.org/archives/ci/2023-April/002159.html
v8:
  * squashed commits into one

Jeremy Spewock (1):
  tools: add acvp_tool

 tools/acvp/README           | 118 +++++++++++++
 tools/acvp/__init__.py      |   0
 tools/acvp/acvp_config.json |  52 ++++++
 tools/acvp/acvp_tool.py     | 319 ++++++++++++++++++++++++++++++++++++
 tools/acvp/requirements.txt |   7 +
 5 files changed, 496 insertions(+)
 create mode 100644 tools/acvp/README
 create mode 100644 tools/acvp/__init__.py
 create mode 100644 tools/acvp/acvp_config.json
 create mode 100755 tools/acvp/acvp_tool.py
 create mode 100644 tools/acvp/requirements.txt

-- 
2.40.0

[PATCH v8 1/1] tools: add acvp_tool

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This tool is used to interact with the NIST ACVP API. In combination with
the default configuration file provided, this tool has the ability to
retrieve Vector Sets from and submit answers to the ACVP demo API. Upon
submitting your answers, you will receive a verdict file that denotes
if they were correct.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
Signed-off-by: Brandon Lo <blo@iol.unh.edu>
---
 tools/acvp/README           | 118 +++++++++++++
 tools/acvp/__init__.py      |   0
 tools/acvp/acvp_config.json |  52 ++++++
 tools/acvp/acvp_tool.py     | 319 ++++++++++++++++++++++++++++++++++++
 tools/acvp/requirements.txt |   7 +
 5 files changed, 496 insertions(+)
 create mode 100644 tools/acvp/README
 create mode 100644 tools/acvp/__init__.py
 create mode 100644 tools/acvp/acvp_config.json
 create mode 100755 tools/acvp/acvp_tool.py
 create mode 100644 tools/acvp/requirements.txt

diff --git a/tools/acvp/README b/tools/acvp/README
new file mode 100644
index 0000000..e89828a
--- /dev/null
+++ b/tools/acvp/README
@@ -0,0 +1,118 @@
+The ACVP tool is a general tool for interacting with the NIST ACVP API
+in order to test different cryptographic implementations.
+
+It produces machine-readable output for parsing in a CI environment.
+
+Supported Algorithms
+--------------------
+* AES-CBC
+* AES-CMAC
+* AES-GMAC
+* HMAC-SHA-1
+* TDES-CBC
+* AES-CTR
+
+Requirements
+------------
+
+There are also python packages you need to download from the requirements.txt file:
+* pyotp
+* requests
+
+Along with these, you will also need to install the `nasm` package using your local package manager.
+
+The tool expects that you have all the credential files from NIST:
+* Client certificate (usually a .cer file from NIST)
+* Key file for the certificate
+* Time-based one-time password seed file (usually a .txt file from NIST)
+
+The path to each file must be stored in an environment variable:
+* $ACVP_SEED_FILE  =  Path to the TOTP seed .txt file    (given by NIST).
+* $ACVP_CERT_FILE  =  Path to the client .cer/.crt file  (given by NIST).
+* $ACVP_KEY_FILE   =  Path to the certificate key file   (generated by user).
+
+If you do not have the required files from NIST, you must email them
+to create demo credentials.
+https://pages.nist.gov/ACVP/#access
+
+
+Setup
+-----
+
+After setting the environment variables as described in the
+"Requirements" section, you will need to edit the acvp_config.json file.
+
+The acvp_config.json file is expected to be a json object
+containing two keys: "url" and "algorithms"
+
+"url" must be the base URL string of the API you want to use.
+"algorithms" must be an array of algorithm objects as detailed in the
+ACVP API specification here:
+https://github.com/usnistgov/ACVP/wiki/ACVTS-End-User-Documentation . In the case of the supported algorithms listed above, the only thing that will need to change in the config file is the `"algorithm"` field to match the name of the algorithm you would like to test.
+* In order to test AES-CTR you'll also have to remove the key `"ivGenMode"`
+
+Now you can use the acvp_tool.py script to register a test session,
+upload the results, and download the verdict.
+
+In order to run the DPDK sample application, there are a few libraries which must be installed:
+* Intel IPSec Multi-buffer (v1.3)
+```
+git clone https://github.com/intel/intel-ipsec-mb.git
+cd intel-ipsec-mb
+git checkout v1.3
+make -j 4
+make install
+```
+* FIPS Object Module
+```
+curl -o openssl-fips-2.0.16.tar.gz https://www.openssl.org/source/openssl-fips-2.0.16.tar.gz
+tar xvfm openssl-fips-2.0.16.tar.gz
+cd openssl-fips-2.0.16
+./config
+make
+make install
+```
+Usage
+-----
+### Interacting with ACVP API
+To see all options available, use the --help flag.
+
+First, register and download a new test session with the tool:
+
+    acvp_tool.py --request $DOWNLOAD_PATH
+The file written to $DOWNLOAD_PATH will contain both the session information and the test vectors.
+
+You should use the DPDK FIPS validation example application to test
+the vectors in this file. The example application will generate
+the result file which is uploaded back to the ACVP API.
+
+After running tests with the vector file, you can submit the result:
+
+    acvp_tool.py --response $RESULT_PATH --upload
+where $RESULT_PATH is the path of the file containing the answers.
+
+Once you submit your results, you can do
+
+    acvp_tool.py --response $RESULT_PATH --verdict $VERDICT_PATH
+where $VERDICT_PATH is where you want to save the verdict information.
+The verdict file will contain the result of each test case submitted.
+
+You can also combine the options:
+
+    acvp_tool.py --response $RESULT_PATH --upload --verdict $VERDICT_PATH
+
+### Using the DPDK FIPS Validation Example Application
+First, you have to make sure that you configure DPDK to build the FIPS sample application before you compile with ninja
+```
+#inside dpdk/
+meson --werror  -Dexamples=fips_validation build
+ninja -C build
+```
+Once this has finished, you can now run the sample application and validate the test vectors. In order to run this validation step, you have to supply a valid crypto device and either a `*.json` or `*.req` file with vectors for validation. You can use the virtual device `crypto_aesni_mb` provided by the Intel IPSec Multi-buffer library and pass the JSON file containing test vectors from the ACVP API using `--req-file`.
+
+Example usage:
+
+    #inside dpdk/
+    build/examples/dpdk-fips_validation --vdev crypto_aesni_mb -- --req-file aes-cbc-vectors.json --rsp-file aes-cbc-answers.rsp --cryptodev crypto_aesni_mb`
+
+The file path passed into `--rsp-file` will contain the validated vectors from the sample applications and can be passed to the ACVP API to receive a verdict on your results.
\ No newline at end of file
diff --git a/tools/acvp/__init__.py b/tools/acvp/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/tools/acvp/acvp_config.json b/tools/acvp/acvp_config.json
new file mode 100644
index 0000000..2d55d8d
--- /dev/null
+++ b/tools/acvp/acvp_config.json
@@ -0,0 +1,52 @@
+{
+    "url": "https://demo.acvts.nist.gov",
+    "algorithms": [
+        {
+            "algorithm": "ACVP-TDES-CBC",
+            "revision": "1.0",
+	    "keyingOption": [
+                1
+            ],
+	    "messageLength": [{"min": 0, "max": 65535, "increment": 1}],
+	    "capabilities": [
+		{
+		  "direction": ["gen", "ver"],
+		  "keyLen": [128],
+		  "msgLen": [
+			{
+			  "max": 65536,
+			  "min": 0,
+			  "increment": 256
+			}
+		  ],
+		  "macLen": [
+			{
+			  "min": 64,
+			  "max": 128,
+			  "increment": 8
+			}
+		  ]
+		}
+	    ],
+            "direction": ["encrypt"],
+            "keyLen": [128, 192, 256],
+	    "macLen": [
+			{
+			    "min": 80,
+			    "max": 160,
+			   "increment": 8
+			}
+	    ],
+            "tagLen": [128],
+            "aadLen": [0],
+            "ivGen": "internal",
+	    "ivGenMode": "8.2.2",
+            "ivLen": [96],
+            "payloadLen": [
+                128
+            ],
+	    "overflowCounter": true,
+	    "incrementalCounter": true
+        }
+    ]
+}
diff --git a/tools/acvp/acvp_tool.py b/tools/acvp/acvp_tool.py
new file mode 100755
index 0000000..40d2f2f
--- /dev/null
+++ b/tools/acvp/acvp_tool.py
@@ -0,0 +1,319 @@
+#!/usr/bin/env python3
+
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2022 The University of New Hampshire
+
+import hashlib
+import sys
+import time
+import base64
+import argparse
+import os
+import json
+import logging
+from typing import Tuple, Optional, Any, Dict, List
+
+import pyotp
+import requests
+
+
+class ACVPProxy:
+    def __init__(self, cert_path: str, key_path: str, totp_path: str,
+                 config_path: str):
+        """ACVP Proxy used to abstract API calls.
+
+        @param cert_path: Path to the client certificate.
+        @param key_path: Path to the client key.
+        @param totp_path: Path to the one-time password seed.
+        @param config_path: Path to the configuration for the session.
+        """
+        self.cert: Tuple[str, str] = (cert_path, key_path)
+
+        if None in self.cert:
+            logging.error('Missing certificate/key file.')
+            sys.exit(1)
+
+        self.totp_path: str = totp_path
+        self.login_data: Optional[Dict[str, Any]] = None
+        self.session_data: Optional[Dict[str, Any]] = None
+
+        with open(config_path, 'r') as f:
+            self.config: Any = json.load(f)
+
+    def __get_totp(self) -> str:
+        """Get the current one-time password.
+
+        Uses the totp_path argument when instantiating to
+        read the TOTP seed.
+
+        @return: String containing the password.
+        """
+        with open(self.totp_path, 'r') as f:
+            seed = f.read().strip()
+        base64_seed = base64.b32encode(base64.b64decode(seed)).decode('utf-8')
+        totp = pyotp.TOTP(s=base64_seed, digits=8, digest=hashlib.sha256,
+                          interval=30)
+        return totp.now()
+
+    def __fetch_vector_set(self, url: str) -> Optional[Dict]:
+        """Fetch the vector set object from a URL.
+
+        @param url: URL of the vector set given by NIST.
+        @return: Dictionary of the response from the NIST API.
+        The returned dictionary comes without the session information.
+        """
+        logging.info(f'Fetching vector set {url}')
+        token = self.session_data['jwt']
+        while True:
+            response = requests.get(
+                f'{self.config["url"]}{url}',
+                cert=self.cert,
+                headers={'Authorization': f'Bearer {token}'}
+            )
+            if not response.ok:
+                logging.error(f'Failed to fetch vector set {url}')
+                logging.error(json.dumps(response.json(), indent=4))
+                return None
+
+            vector_set_json = response.json()[1]
+            if 'retry' in vector_set_json:
+                duration = vector_set_json['retry']
+                logging.info(f'Server says retry in {duration} seconds...')
+                time.sleep(duration)
+                continue
+
+            logging.info(f'Downloaded vector set {url}')
+            return vector_set_json
+
+    def login(self) -> bool:
+        """Log into the API server.
+
+        Uses the instance's current TOTP seed and certificate file paths
+        to authenticate with the API.
+
+        If successful, the access token of the account will be stored in
+        the instance.
+
+        @return: True if authentication succeeded, false otherwise.
+        """
+        response = requests.post(
+            url=f'{self.config["url"]}/acvp/v1/login',
+            json=[
+                {'acvVersion': '1.0'},
+                {'password': self.__get_totp()},
+            ],
+            cert=self.cert,
+        )
+
+        if not response.ok:
+            logging.error('Failed to log in.')
+            logging.error(json.dumps(response.json(), indent=4))
+            return False
+
+        self.login_data = response.json()[1]
+        # Renamed 'accessToken' to 'jwt' in the json object
+        # to stay consistent with libacvp
+        self.login_data['jwt'] = self.login_data.pop('accessToken')
+        return True
+
+    def register(self) -> Optional[List[Any]]:
+        """Register a new test session.
+
+        This requires the ACVPProxy instance to be authenticated (use .login).
+
+        @return: If registration succeeded, it will return the list
+        containing session information and vector sets.
+        """
+        if self.login_data is None:
+            logging.error('ACVP proxy cannot register a test session without '
+                          'logging in first.')
+            return None
+
+        response = requests.post(
+            url=f'{self.config["url"]}/acvp/v1/testSessions',
+            json=[
+                {'acvVersion': '1.0'},
+                {
+                    'isSample': False,
+                    'algorithms': self.config['algorithms']
+                }
+            ],
+            cert=self.cert,
+            headers={'Authorization': f'Bearer {self.login_data["jwt"]}'}
+        )
+
+        if not response.ok:
+            logging.error('Unable to register.')
+            logging.error(json.dumps(response.json(), indent=4))
+            return None
+
+        self.session_data = response.json()[1]
+        # Renamed 'accessToken' to 'jwt' in the json object
+        # to stay consistent with libacvp
+        self.session_data['jwt'] = self.session_data.pop('accessToken')
+        write_data = [self.session_data]
+
+        for url in self.session_data['vectorSetUrls']:
+            write_data.append(self.__fetch_vector_set(url))
+
+        return write_data
+
+    def fetch_verdict(self, vector_results: List[Any]) -> List[Any]:
+        """Fetch verdict for a list of vector sets.
+
+        @param vector_results: List of vector set dictionaries with answers.
+        @return: A list containing the session information and verdicts.
+        """
+        session_url = self.session_data['url']
+        write_data = [self.session_data]
+        for _, vector_set in vector_results:
+            vector_set_id = vector_set['vsId']
+            logging.info(f'Downloading verdict for vector set {vector_set_id}')
+            while True:
+                result = requests.get(
+                    f'{self.config["url"]}{session_url}'
+                    f'/vectorSets/{vector_set_id}/results',
+                    cert=self.cert,
+                    headers={
+                        'Authorization': f'Bearer {self.session_data["jwt"]}'
+                    }
+                )
+                version, result_json = result.json()
+                if 'retry' in result_json:
+                    duration = result_json['retry']
+                    logging.info(f'Vector set verdict not ready, waiting '
+                                 f'{duration} seconds...')
+                    time.sleep(duration)
+                    continue
+
+                write_data.append([version, result_json])
+                break
+        return write_data
+
+    def upload(self, vector_sets: List[Any]) -> bool:
+        """Upload the given vector sets.
+
+        @param vector_sets: List of vector set dictionaries with answers.
+        @return: True if uploading succeeded.
+        """
+        has_error = False
+        session_url = self.session_data['url']
+
+        for version, vector_set in vector_sets:
+            response = requests.post(
+                f'{self.config["url"]}{session_url}/vectorSets/'
+                f'{vector_set["vsId"]}/results',
+                json=[version, vector_set],
+                cert=self.cert,
+                headers={'Authorization': f'Bearer {self.session_data["jwt"]}'}
+            )
+
+            if not response.ok:
+                has_error = True
+                logging.error(f'Could not upload vector set response for '
+                              f'vector set ID {vector_set["vsId"]}.')
+                logging.error(json.dumps(response.json(), indent=4))
+                continue
+
+        return not has_error
+
+
+def main(request_path: Optional[str],
+         response_path: Optional[str],
+         verdict_path: Optional[str],
+         do_upload: bool,
+         config_path: str):
+
+    if request_path and response_path:
+        logging.error('You cannot use both a request and a response file.')
+        sys.exit(1)
+
+    if not any([request_path, response_path]):
+        logging.error('You must specify either a request or a response file.')
+        sys.exit(1)
+
+    proxy = ACVPProxy(
+        cert_path=os.getenv('ACVP_CERT_FILE'),
+        key_path=os.getenv('ACVP_KEY_FILE'),
+        totp_path=os.getenv('ACVP_SEED_FILE'),
+        config_path=config_path,
+    )
+
+    logging.info('Attempting to log in...')
+    if not proxy.login():
+        logging.error('Could not log in.')
+        sys.exit(1)
+    logging.info('Successfully logged in.')
+
+    if request_path:
+        logging.info('Creating a new test session and downloading vectors...')
+        test_session = proxy.register()
+        if not test_session:
+            logging.error('Could not create a new test session.')
+            sys.exit(1)
+
+        with open(request_path, 'w') as f:
+            json.dump(test_session, f, indent=4)
+    elif response_path:
+        logging.info('Using response file...')
+        with open(response_path, 'r') as upload_file:
+            upload_json: List[Any] = json.load(upload_file)
+        proxy.session_data = upload_json[0]
+
+        if do_upload:
+            logging.info('Uploading response file...')
+            if not proxy.upload(upload_json[1:]):
+                logging.error('Could not successfully upload results file.')
+                sys.exit(1)
+
+        if verdict_path:
+            logging.info('Fetching verdict...')
+            verdict = proxy.fetch_verdict(upload_json[1:])
+            if not verdict:
+                logging.error('Could not successfully fetch verdict file.')
+                sys.exit(1)
+            with open(verdict_path, 'w') as f:
+                json.dump(verdict, f, indent=4)
+
+    logging.info('Done')
+
+
+if __name__ == '__main__':
+    logging.basicConfig(level=logging.INFO)
+    parser = argparse.ArgumentParser(description='ACVP tool to fetch tests '
+                                                 'and upload results.')
+    parser.add_argument('--request', '-r',
+                        help='Path to download a request file. '
+                             'If specified, the tool start a new test session '
+                             'and download the test information into the '
+                             'given path.')
+    parser.add_argument('--response', '-s',
+                        help='Path of the response file. '
+                             'If specified, the tool will use this file '
+                             'to determine session information. '
+                             'This argument must be used when uploading '
+                             'results or downloading verdicts.')
+    parser.add_argument('--verdict', '-v',
+                        help='Download the verdict to the specified path. '
+                             'If this flag is set, the tool will download the '
+                             'verdict for the given response file '
+                             '(--response).')
+    parser.add_argument('--upload', '-u',
+                        help='Upload the given response file to the API. '
+                             'If this flag is set, the tool will upload the '
+                             'given response file (--response).',
+                        action='store_true')
+    parser.add_argument('--config', '-c',
+                        help='Path of the configuration file. '
+                             '(Default: acvp_config.json)',
+                        default='acvp_config.json')
+
+    args = parser.parse_args()
+
+    main(
+        request_path=args.request,
+        response_path=args.response,
+        verdict_path=args.verdict,
+        do_upload=args.upload,
+        config_path=args.config,
+    )
diff --git a/tools/acvp/requirements.txt b/tools/acvp/requirements.txt
new file mode 100644
index 0000000..428f06c
--- /dev/null
+++ b/tools/acvp/requirements.txt
@@ -0,0 +1,7 @@
+certifi==2021.10.8
+charset-normalizer==2.0.10
+idna==3.3
+pyotp==2.6.0
+requests==2.27.1
+six==1.16.0
+urllib3==1.26.8
-- 
2.40.0

Re: [PATCH v8 1/1] tools: add acvp_tool

[Aaron Conole]

jspewock@iol.unh.edu writes:

> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> This tool is used to interact with the NIST ACVP API. In combination with
> the default configuration file provided, this tool has the ability to
> retrieve Vector Sets from and submit answers to the ACVP demo API. Upon
> submitting your answers, you will receive a verdict file that denotes
> if they were correct.
>
> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> Signed-off-by: Brandon Lo <blo@iol.unh.edu>
> ---

Hi Ali,

Please provide your ack and I will apply.

-Aaron

RE: [PATCH v8 1/1] tools: add acvp_tool

[Ali Alnubani]

> -----Original Message-----
> From: jspewock@iol.unh.edu <jspewock@iol.unh.edu>
> Sent: Monday, April 17, 2023 10:18 PM
> To: ci@dpdk.org
> Cc: Jeremy Spewock <jspewock@iol.unh.edu>; Brandon Lo
> <blo@iol.unh.edu>
> Subject: [PATCH v8 1/1] tools: add acvp_tool
> 
> From: Jeremy Spewock <jspewock@iol.unh.edu>
> 
> This tool is used to interact with the NIST ACVP API. In combination with
> the default configuration file provided, this tool has the ability to
> retrieve Vector Sets from and submit answers to the ACVP demo API. Upon
> submitting your answers, you will receive a verdict file that denotes
> if they were correct.
> 
> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> Signed-off-by: Brandon Lo <blo@iol.unh.edu>
> ---

Acked-by: Ali Alnubani <alialnu@nvidia.com>

Thanks,
Ali

Re: [PATCH v8 1/1] tools: add acvp_tool

[Aaron Conole]

Ali Alnubani <alialnu@nvidia.com> writes:

>> -----Original Message-----
>> From: jspewock@iol.unh.edu <jspewock@iol.unh.edu>
>> Sent: Monday, April 17, 2023 10:18 PM
>> To: ci@dpdk.org
>> Cc: Jeremy Spewock <jspewock@iol.unh.edu>; Brandon Lo
>> <blo@iol.unh.edu>
>> Subject: [PATCH v8 1/1] tools: add acvp_tool
>> 
>> From: Jeremy Spewock <jspewock@iol.unh.edu>
>> 
>> This tool is used to interact with the NIST ACVP API. In combination with
>> the default configuration file provided, this tool has the ability to
>> retrieve Vector Sets from and submit answers to the ACVP demo API. Upon
>> submitting your answers, you will receive a verdict file that denotes
>> if they were correct.
>> 
>> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
>> Signed-off-by: Brandon Lo <blo@iol.unh.edu>
>> ---
>
> Acked-by: Ali Alnubani <alialnu@nvidia.com>

Thanks, applied!

> Thanks,
> Ali