README.md

Creates a CycloneDX SBOM by way of tox.

The current implementation is limited to working with a pyproject.toml file and the tox configuration in a separate tox.ini file. It's tested against Python 3.10.

This is intended to be run in an independent tox environment with no dependencies. A good format for the tox environment in the tox.ini file is:

[testenv:sbom]
description = Generate Software Bill of Materials
constrain_package_deps = true
use_frozen_constraints = true
commands =
    python _custom_build/tox_cyclonedx.py

(assuming you place the file in your project under _custom_build/tox_cyclonedx.py). You would then generate the SBOM file by running tox run -e sbom.

You can customize the project configuration in your pyproject.toml file by including a [tool.cyclonedx] section with these values:

• debug - turn debug logging on or off (boolean).
• dependencies - CycloneDX pip dependency (array of strings).
• tox-config - tox.ini file location (string). Note: setup.cfg and pyproject.toml settings for tox is not currently supported.
• tox-env - testenv section in the tox.ini file (string).
• force-include - list of dependencies to add to the SBOM (array of strings). This must be a list of exact dependencies, so they each must have a == version marker.
• cyclonedx-cmd - override the default CycloneDX execution command (string). Use {requirements} as the replacement for the location of the input frozen requirements file, and {report} as the location of the generated report file.
• report-file - location of the output SBOM report file (string).

The default configuration is the same as specifying this in your pyproject.toml file:

[tool.cyclonedx]
debug = False
dependencies = [ "cyclonedx-bom>=3.11.2,<4" ]
tox-config = "tox.ini"
tox-env = "py"
report-file = "cyclonedx-sbom.xml"
force-include = []
cyclonedx-cmd = [
  "python",
  "-I",
  "-m",
  "cyclonedx_py",
  "-i",
  "{requirements}",
  "--requirements",
  "--format",
  "xml",
  "--output",
  "{report}",
]

The tool will install CycloneDX inside the requested tox environment during execution. Therefore, it's highly suggested to run this inside its own tox environment. The installation of CycloneDX follows the tox environment section's install_command. It will fall back and check a pip.conf or pip.ini in the current directory for trusted-host and index-url values.

The python code is released under the ISC license.

tox_cyclonedx.py

#!/usr/bin/python3

# ISC License
# Copyright 2023 Groboclown
#
# Permission to use, copy, modify, and/or distribute this software for any purpose
# with or without fee is hereby granted, provided that the above copyright notice
# and this permission notice appear in all copies.
#
# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
# REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
# AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,
# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
# LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
# OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
# PERFORMANCE OF THIS SOFTWARE.

"""Creates a CycloneDX SBOM by way of tox.

The current implementation is limited to working with a `pyproject.toml` file and
the tox configuration in a separate `tox.ini` file.  It's tested against Python 3.10.

This is intended to be run in an independent tox environment with no
dependencies.  A good format for the tox environment in the `tox.ini` file is:

```
[testenv:sbom]
description = Generate Software Bill of Materials
constrain_package_deps = true
use_frozen_constraints = true
commands =
    python _custom_build/tox_cyclonedx.py
```

You would then generate the SBOM file by running `tox run -e sbom`

You can customize the project configuration in your `pyproject.toml` file by
including a [tool.cyclonedx] section with these values:

    debug                   turn debug logging on or off (boolean)
    dependencies            CycloneDX pip dependency (array of strings)
    tox-config              tox.ini file location (string)
                              Note: setup.cfg and pyproject.toml settings for tox is
                              not currently supported.
    tox-env                 testenv section in the tox.ini file (string)
    force-include           list of dependencies to add to the SBOM (array of strings)
    cyclonedx-cmd           override the default CycloneDX execution command (string)
	                          Use "{requirements}" as the replacement for the location
							  of the input frozen requirements file, and "{report}"
							  as the location of the generated report file.
    report-file             location of the output SBOM report file (string)

"""

# Implementation note:
#   This file very explicitly only depends on PIP and built-in Python libraries.
#   This is also why this is a Gist, and not an installable package - having it as
#   an installable package would make it pollute the product frozen requirements file.
import configparser
import collections.abc
import os
import shlex
import subprocess
import sys

# This is a no-no, but makes life easier.
from pip._vendor import tomli


class Config:
    """Configuration settings."""

    __slots__ = (
        "pyproject_file",
        "initial_pip_install_cmd",
        "pip_freeze_cmd",
        "tox_file",
        "tox_env",
        "tox_env_dir",
        "cyclonedx_dependencies",
        "force_dependencies",
        "report_file",
        "debug",
        "installed_requirements_file",
        "initial_cyclonedx_cmd",
    )

    def __init__(self) -> None:
        self.pyproject_file = "pyproject.toml"
        self.initial_pip_install_cmd: List[str] = [
            sys.executable,
            "-I",
            "-m",
            "pip",
            "install",
            "{opts}",
            "{packages}",
        ]
        self.pip_freeze_cmd: List[str] = [
            sys.executable,
            "-I",
            "-m",
            "pip",
            "freeze",
            "-q",
            "--local",
            "--exclude-editable",
        ]
        self.tox_file = "tox.ini"
        self.tox_env = "py"
        self.tox_env_dir = os.path.join(".tox", "py")
        self.cyclonedx_dependencies: List[str] = [
            "cyclonedx-bom>=3.11.2,<4",
        ]
        self.force_dependencies: List[str] = []
        self.report_file = "cyclonedx-sbom.xml"
        self.installed_requirements_file = os.path.join(
            self.tox_env_dir, "installed-requirements.txt"
        )
        self.initial_cyclonedx_cmd: List[str] = [
            sys.executable,
            "-I",
            "-m",
            "cyclonedx_py",
            "-i",
            "{requirements}",
            "--requirements",
            "--format",
            "xml",
            "--output",
            "{report}",
        ]

    @property
    def cyclonedx_cmd(self) -> collections.abc.Sequence[str]:
        """Generate the cyclonedx command."""
        ret: list[str] = []
        for item in self.initial_cyclonedx_cmd:
            item = item.strip()
            if item:
                ret.append(
                    item.format(
                        requirements=self.installed_requirements_file,
                        report=self.report_file,
                    )
                )
        return ret

    def get_pip_install_cmd(
        self,
		opts: collections.abc.Sequence[str],
		packages: collections.abc.Sequence[str],
    ) -> collections.abc.Sequence[str]:
        """Get the pip install command."""
        ret: list[str] = []
        for item in self.initial_pip_install_cmd:
            if item == "{opts}":
                ret.extend(opts)
            elif item == "{packages}":
                ret.extend(packages)
            else:
                ret.append(item)
        return ret


DEBUG = [False]


def printerr(text: str) -> None:
    """Print to stderr."""
    sys.stderr.write(text + "\n")
    sys.stderr.flush()


def debug(text: str) -> None:
    """Debug message."""
    if DEBUG[0]:
        print(f"[DEBUG] {text}")


def load_config(config: Config) -> bool:
    """Updates the configuration based on the files.
    Returns False on error; error is reported by this function."""
    if not os.path.isfile(config.pyproject_file):
        printerr(f"Could not find {config.pyproject_file}.")
        return False

    ret = True

    debug(f"Loading pyproject file {config.pyproject_file}")
    with open(config.pyproject_file, encoding="utf-8") as f:
        pp_toml = tomli.loads(f.read())
    toml_cfg = pp_toml.get("tool", {}).get("cyclonedx")

    if "debug" in toml_cfg:
        val = toml_cfg["debug"]
        if isinstance(val, bool):
            DEBUG[0] = val
        elif isinstance(val, str):
            if val.lower() in ("true", "t", "y", "yes", "on"):
                DEBUG[0] = True
            elif val.lower() in ("false", "f", "n", "no", "off"):
                DEBUG[0] = False
            else:
                printerr(f"Invalid 'debug' configuration value: {val}")
                ret = False
        else:
            printerr(f"Invalid 'debug' configuration type: {type(val)}")
            ret = False

    if "dependencies" in toml_cfg:
        config.cyclonedx_dependencies.clear()
        deps = toml_cfg["dependencies"]
        if isinstance(deps, str):
            config.cyclonedx_dependencies.append(deps)
        elif isinstance(deps, collections.abc.Iterable):
            config.cyclonedx_dependencies.extend(deps)
        else:
            printerr(f"Invalid 'dependencies' configuration type: {type(deps)}")
            ret = False
    debug(f"Using CycloneDX dependencies {repr(config.cyclonedx_dependencies)}")

    if "tox-env" in toml_cfg:
        # NOTE: can tox.ini include a different .tox directory name?
        # If so, that requires another check to properly set the .tox_env_dir .
        env = toml_cfg["tox-env"]
        if isinstance(env, str):
            config.tox_env = env
            config.tox_env_dir = os.path.join(".tox", env)
        else:
            printerr(f"Invalid 'tox-env' configuration type: {type(env)}")
            ret = False
    debug(f"Using tox testenv:{config.tox_env}, env dir {config.tox_env_dir}")

    if "tox-config" in toml_cfg:
        cfg_file = toml_cfg["tox-config"]
        if isinstance(cfg_file, str):
            config.tox_file = cfg_file
        else:
            printerr(f"Invalid 'tox-config' configuration type: {type(cfg_file)}")
            ret = False
    if os.path.samefile(config.pyproject_file, config.tox_file):
        printerr(f"Tox environment in the {config.pyproject_file} is not supported")
        ret = False
    if not os.path.isfile(config.tox_file):
        printerr(f"Could not find tox configuration file {config.tox_file}")
        ret = False
    debug(f"Using tox configuration file {config.tox_file}")

    if "force-include" in toml_cfg:
        deps = toml_cfg["force-include"]
        config.force_dependencies.clear()
        if isinstance(deps, str):
            config.force_dependencies.append(deps)
        elif isinstance(deps, collections.abc.Iterable):
            config.force_dependencies.extend(deps)
        else:
            printerr(f"Invalid 'force-include' configuration type: {type(deps)}")
            ret = False
    for dep in config.force_dependencies:
        if "==" not in dep:
            printerr(f"Forced dependency '{dep}' must be an exact ('==') dependency.")
            ret = False
    debug(
	    f"Forcing the SBOM to include dependencies {repr(config.force_dependencies)}"
	)

    if "cyclonedx-cmd" in toml_cfg:
        cmd = toml_cfg["cyclonedx-cmd"]
        config.initial_cyclonedx_cmd.clear()
        if isinstance(cmd, str):
            config.initial_cyclonedx_cmd.extend(shlex.split(cmd))
        elif isinstance(cmd, collections.abc.Iterable):
            config.initial_cyclonedx_cmd.extend(cmd)
        else:
            printerr(f"Invalid 'cyclonedx-cmd' configuration type: {type(cmd)}")
            ret = False

    if "report-file" in toml_cfg:
        report_file = toml_cfg["report-file"]
        if isinstance(report_file, str):
            config.report_file = report_file
        else:
            printerr(f"Invalid 'report-file' configuration type: {type(report_file)}")
            ret = False

    loaded_install_cmd = False
    if os.path.isfile(config.tox_file):
        # Load in tox defined settings.
        tox_cfg = configparser.ConfigParser()
        tox_cfg.read(config.tox_file)
        section = f"testenv:{config.tox_env}"
        if section not in tox_cfg:
            section = "testenv"
        tox_test_env = tox_cfg.get(section, "install_command")
        if tox_test_env:
            config.initial_pip_install_cmd.clear()
            config.initial_pip_install_cmd.extend(shlex.split(tox_test_env))
            loaded_install_cmd = True
    if not loaded_install_cmd:
        pip_file = "pip.conf"
        if not os.path.isfile(pip_file) and os.path.isfile("pip.ini"):
            pip_file = "pip.ini"
        if os.path.isfile(pip_file):
            # Non-standard, but it's helpful.
            parser = configparser.ConfigParser()
            parser.read(pip_file)
            trusted_host = parser.get("global", "trusted-host", fallback="").strip()
            for item in trusted_host.split(" "):
                item = item.strip()
                if item:
                    config.initial_pip_install_cmd.append(f"--trusted-host={item}")
            index_url = parser.get("install", "index-url", fallback="").strip()
            for item in index_url.split(" "):
                item = item.strip()
                if item:
                    config.initial_pip_install_cmd.append(f"--index-url={item}")
    debug(f"Using pip install command {repr(config.initial_pip_install_cmd)}")

    if not os.path.isdir(config.tox_env_dir):
        printerr(f"Could not find tox environment directory {config.tox_env_dir}")
        ret = False

    # Hard-coded name and location; cannot be changed.
    config.installed_requirements_file = os.path.join(
        config.tox_env_dir, "installed-requirements.txt"
    )

    return ret


def get_requirements_name(line) -> str:
    """From a requirements line, get the package name"""
    line = line.strip()
    line_len = len(line)
    if line_len <= 0 or line[0] == "#":
        return ""
    end = line_len
    for pos in (
        line.find("="),
        line.find(">"),
        line.find("["),
        line.find("<"),
        line.find(","),
        line.find("~"),
        line.find(";"),
    ):
        if pos == 0:
            return ""
        if 0 < pos < end:
            end = pos
    return line[:end]


def get_pip_freeze_cmd(config: Config) -> list[str]:
    """Generate the freeze command."""
    ret = list(config.pip_freeze_cmd)
    filename = os.path.join(config.tox_env_dir, "constraints.txt")
    if os.path.isfile(filename):
        with open(filename, "r", encoding="utf-8") as fis:
            for line in fis.readlines():
                name = get_requirements_name(line)
                if name:
                    debug(f"Excluding package from SBOM: {name}")
                    ret.append("--exclude")
                    ret.append(name)
    return ret


def run_install_cyclonedx(config: Config) -> int:
    """Install cyclonedx.  Returns install exit code"""
    args = config.get_pip_install_cmd([], config.cyclonedx_dependencies)
    if not args:
        return 100
    debug(f"Install command: {repr(args)}")
    res = subprocess.run(args)
    return res.returncode


def run_pip_freeze(config: Config) -> int:
    """Run pip freeze, piping the generated requirements to the given file."""
    args = get_pip_freeze_cmd(config)
    if not args:
        return 100
    os.makedirs(os.path.dirname(config.installed_requirements_file), exist_ok=True)
    debug(f"Freeze command: {repr(args)}")
    with open(config.installed_requirements_file, "wb") as fos:
        res = subprocess.run(
            args,
            stdout=fos,
        )
    return res.returncode


def run_cyclonedx(config: Config) -> int:
    """Run cyclonedx over the requirements file."""
    os.makedirs(os.path.dirname(config.report_file), exist_ok=True)
    if os.path.isfile(config.report_file):
        # cyclonedx will fail if it already exists.
        os.unlink(config.report_file)
    args = config.cyclonedx_cmd
    debug(f"SBOM command: {repr(args)}")
    res = subprocess.run(args)
    return res.returncode


def clean_freeze_file(config: Config) -> int:
    """Clean up the freeze file."""
    with open(config.installed_requirements_file, "r", encoding="utf-8") as fis:
        contents = fis.read()
    with open(config.installed_requirements_file, "w", encoding="utf-8") as fos:
        extra_deps = list(config.force_dependencies)
        for line in contents.splitlines(keepends=True):
            if " @ " in line and "file:/" in line:
                # This is the local package that we're installing.  It must not
                # be in the SBOM.
                continue
            if line in extra_deps:
                extra_deps.remove(line)
            fos.write(line)
        for req in extra_deps:
            fos.write(f"{req}\n")
    return 0


def cli_main(args: collections.abc.Sequence[str], config: Config) -> int:
    """Run the command."""
    for arg in args[1:]:
        if arg in ("--help", "-h"):
            printerr(f"Usage: {args[0]} [--debug] [pyproject.toml]")
            return 0
        if arg in ("--debug", "-d"):
            config.debug = True
            continue
        config.pyproject_file = arg
    if not load_config(config):
        return 1

    if os.path.isfile(config.installed_requirements_file):
        # The temporary requirements file is placed in the tox virtual environment
		# directory.  This directory is removed everytime tox discovers dependency
		# changes, so it's a good marker for if the requirements have already been
		# installed.  We don't want to run this twice, because that will cause the
		# later-installed cyclonedx dependencies to affect the final freeze list.
        printerr("Skipping freeze generation; looks like it's already been run.")
    else:
        print("Generating requirements...")
        code = run_pip_freeze(config)
        if code != 0:
            printerr("Failed running pip freeze")
            return code
        code = clean_freeze_file(config)
        if code != 0:
            printerr("Failed preparing freeze file for analysis")
            return code

    print("Installing cyclonedx...")
    code = run_install_cyclonedx(config)
    if code != 0:
        printerr("Failed installing cyclonedx")
        return code

    print("Generating SBOM report...")
    code = run_cyclonedx(config)
    if code != 0:
        printerr("Failed running cyclonedx-bom")
        return code

    return 0


if __name__ == "__main__":
    sys.exit(cli_main(sys.argv, Config()))