..dep-edge-checking.md

Enforcement of allowed dep edges

the idea is to have testonly-esque checks

consider a project where we want to partition content (for an existing rule) into three categories:

• "open-source"; a.k.a. oss
• "proprietary"; a.k.a. prp
• "development-only"; a.k.a. dev

such that these categories are supersets: dev ⊃ prp ⊃ oss

i.e. the following dep edges are allowed:

• content of kind oss can depend on content of kind oss
• content of kind prp can depend on content of kind prp OR oss
• content of kind dev can depend on content of kind dev OR prp OR oss

and the following are not allowed:

• oss depending on prp OR dev
• prp depending on dev

more concretely:

cc_library(name = "external_lib")

cc_library(name = "oss1")           # kind = "oss"
cc_library(name = "proprietary1")   # kind = "prp"
cc_library(name = "dev1")           # kind = "dev"

cc_library(                         # kind = "oss"
    name = "oss",
    deps = [
        ":external_lib",            # fine, not tagged
        ":oss1",                    # okay
        ":proprietary1",            # !!! not allowed
        ":dev1",                    # !!! not allowed
    ],
)

cc_library(                         # kind = "prp"
    name = "proprietary",
    deps = [
        ":external_lib",            # fine, not tagged
        ":oss1",                    # okay
        ":proprietary1",            # okay
        ":dev1",                    # !!! not allowed
    ],
)

cc_library(                         # kind = "dev"
    name = "dev",
    deps = [
        ":external_lib",            # fine, not tagged
        ":oss1",                    # okay
        ":proprietary1",            # okay
        ":dev1",                    # okay
    ],
)

# and, ideally:
cc_library(name = "laundered", deps = [":dev1"])
cc_library(name = "oss2", deps = [":laundered"]) # !!! not allowed

using rule extensions + aspects

uhhh... didn't actually end up implementing this; I think environments are a better fit for a few reasons:

• applies to all rules; don't need to extend rules or add aspects to run using CLI flags
• better error messages? though this is something that you could have the aspects do with enough work
• select awareness: support for "refinements"

this would look like:

• an aspect that walks your dep graph checking that dep edges match the desired rules
• targets communicate their "kind" by any of:
  • rule extensions that have an extra attribute (non-configurable, we'd hope) or an extra provider
  • aspect_hints
  • using tags

using environments

for background on environments see:

• Kristina Chodorow's blog post
• this stackoverflow post's overview
• docs on:
  • the common compatible_with and restricted_to rule attributes
  • environment_group and environment (within the codebase overview)
• definitions for the environment and environment_group rules in the Bazel source code
• this source code comment on ConstraintSemantics within Bazel
• this github issue discussing the state of environments in Bazel and potential alternatives
• the --target_environment flag docs
• other misc undocumented attributes like:
  • restrictedTo and compatibleWith on rule definitions
    • these have no starlark-accessible equivalent (i.e. no corresponding attributes on rule(...))
    • buuuut we can fake it by mimicking the effects; i.e. setting the private (starts with $) attributes that the environments given to restrictedTo and compatibleWith are lowered onto the rule instantiation as
    • UPDATE: nevermind, this doesn't work; those attributes are only added if restrictedTo is set so we can't set them from the starlark side... need to just do the regular old macro thing
    • it's unfortunate because the behavior of restrictedTo/compatibleWith on rule declarations and how it interacts with restricted_to and compatible_with attrs specified on targets is pretty good and not trivial to replicate in the macro layer
  • default_restricted_to and default_compatible_with in package(...)

Tip

You can use package(default_restricted_to = [...]) and package(default_compatible_with = [...]) to set restricted_to/compatible_with for all targets in a package!

(these attributes to package are not documented; see here)

Important

The way default_restricted_to and default_compatible_with interact with the corresponding attributes set on targets is that restricted_to/compatible_with take precedence: no merging of values happens.

See here for details.

Tip

You can do a "restricted environment build" by setting environments to "check" targets against

i.e. --target_environment=//:oss_e1 will flag targets that cannot be built for OSS:

❯ bazel build //... --target_environment=//:oss_e1
INFO: Build option --target_environment has changed, discarding analysis cache.
ERROR: This is a restricted-environment build.

//:dev_e1_2 declares compatibility with:
  [//:dev_e1]
but does not support:
  //:oss_e1

//:prp_e1_2 declares compatibility with:
  [//:prp_e1, //:dev_e1]
but does not support:
  //:oss_e1

//:prp_e1_1 declares compatibility with:
  [//:prp_e1, //:dev_e1]
but does not support:
  //:oss_e1

//:dev_e1_1 declares compatibility with:
  [//:dev_e1]
but does not support:
  //:oss_e1

--target_environment=//:dev_e1 will flag targets that can't be built for dev (i.e. nothing)

--target_environment=//:oss_e1,//:prp_e1 will flag targets that can't be built for oss AND prp:

❯ bazel build //... --target_environment=//:oss_e1,//:prp_e1
INFO: Build option --target_environment has changed, discarding analysis cache.
ERROR: This is a restricted-environment build.

//:prp_e1_2 declares compatibility with:
  [//:prp_e1, //:dev_e1]
but does not support:
  //:oss_e1

//:dev_e1_2 declares compatibility with:
  [//:dev_e1]
but does not support:
  //:oss_e1
  //:prp_e1

//:prp_e1_1 declares compatibility with:
  [//:prp_e1, //:dev_e1]
but does not support:
  //:oss_e1

//:dev_e1_1 declares compatibility with:
  [//:dev_e1]
but does not support:
  //:oss_e1
  //:prp_e1

[!TIP]

refined environments

• selects: union (refinement)

TODO: restricted_to and compatible_with on rules?

• can get close with macros but not perfect
• fortunately: non-configurable so can check stuff in your macro

subtractive: restricted_to

see BUILD.bazel

additive: compatible_with

see BUILD.bazel

.envrc

use nix -p bazel

.gitignore

/.direnv
/bazel-*

BUILD.bazel

load(":e1.bzl", "oss_cc_binary")

# package(default_restricted_to = [":oss_e1", ":prp_e1", ":dev_e1"])

### environments: subtractive

# NOTE: the set relationships are: oss ⊃ prp ⊃ dev
#
# i.e. three valid env states for "producers":
#   - dev             -- content _classified_ as dev; usable only in dev
#   - dev, prp        -- content _classified_ as prp; usable in dev or prp
#   - dev, prp, oss   -- content _classified_ as oss; usable anywhere
#
# for consumers (i.e. `--target_environment`, rules with limitations) this is
# flipped:
#   - oss           -- accepts only content valid for oss (most restrictive)
#   - oss, prp      -- accepts oss and prp content
#   - oss, prp, dev -- accepts all content (most permissive)
#
# imo this is the modeling that makes sense
#   - it might be tempting to think about this the other way; i.e. `dev` content
#     can freely depend on `oss` and `prp` so we should use `oss + prp + dev`
#     as the environment list for `dev` content
#   - but ^ is incorrect; the key bit is that deps flow upwards: we're
#     constraining what a target can be composed into
#   - from that perspective it follows naturally that `dev` should only list the
#     `dev` environment since that's the only place it can be used

environment(name = "oss_e1")
environment(name = "prp_e1")
environment(name = "dev_e1")

environment_group(
    name = "privacy_e1",
    defaults = [
        ":oss_e1",
        ":prp_e1",
        ":dev_e1",
    ],
    environments = [":oss_e1", ":prp_e1", ":dev_e1"],
)

cc_library(name = "oss_e1_1") # no restrictions, can be used in all environments!
cc_library(name = "prp_e1_1", restricted_to = [":prp_e1", ":dev_e1"]) # not allowed to use this in `oss` environments...
cc_library(name = "dev_e1_1", restricted_to = [":dev_e1"]) # only allowed to use this in `dev` environments

cc_library(
    name = "oss_e1_2",

    # because we don't specify `restricted_to` here, inherits the default: can
    # be used for all environments
    deps = [
        ":oss_e1_1", # okay; can be used for all environments
        # ":prp_e1_1", # !!! not okay: can't be used for `oss`
        # ":dev_e1_1", # !!! not okay: can't be used for `oss` or `dev`
    ],
)

cc_library(
    name = "prp_e1_2",
    restricted_to = [":prp_e1", ":dev_e1"], # can't use this in `oss`...

    deps = [
        ":oss_e1_1", # okay; can be used for all environments
        ":prp_e1_1", # okay: can be used for all environments that this target can be used for (prp and dev)
        # ":dev_e1_1", # !!! not okay: can't be used for `prp`
    ],
)

cc_library(
    name = "dev_e1_2",
    restricted_to = [":dev_e1"], # can't use this in `oss` or `prp`, only `dev`

    deps = [
        ":oss_e1_1", # okay; can be used for all environments
        ":prp_e1_1", # okay: can be used for all environments that this target can be used for (dev)
        ":dev_e1_1", # okay: can be used for all environments that this target can be used for (dev)
    ],
)

# you can "craft" rules that are bound to a particular environment, kind of:
oss_cc_binary(
    name = "oss_only_e1",
    deps = [":oss_e1_2"] + select({
        # "//:privacy_e1": [":dev_e1_2"],
        "//conditions:default": [
            # ":dev_e1_2"
        ],
    }) + [
        # ":prp_e1_1", # !!! errors
        # ":dev_e1_1", # !!! errors
    ],

    restricted_to = [
        # ":prp_e1", # !!! errors; not allowed
        # ":dev_e1", # !!! errors; not allowed
        ":oss_e1", # warns; redundant
    ],
)

### environments: additive

# same as above except the defaults are just `dev` (most permissive environment)
#
# users need to tag content that's allowed in more constrained environments
# (`prp`, `oss`)
#
# it's a tradeoff:
#   - now users can't forget to restrict sensitive stuff that, i.e. shouldn't
#     make it into `oss` or `prp`
#   - but now _everything_ that's `oss` or `prp` (and all that it transitively
#     depends on!) has to be tagged as being `oss` or `prp`
#     + previously stuff that wasn't tagged was just assumed to be `oss`
#       (compatible with the least permissive environment)

environment(name = "oss_e2")
environment(name = "prp_e2")
environment(name = "dev_e2")

environment_group(
    name = "privacy_e2",
    defaults = [":dev_e2"], # by default, only compatible with dev
    environments = [
        ":oss_e2",
        ":prp_e2",
        ":dev_e2",
    ],
)

cc_library(name = "oss_e2_1", compatible_with = [":prp_e2", ":oss_e2"]) # compatible with all environments!
cc_library(name = "prp_e2_1", compatible_with = [":prp_e2"]) # only additional environment is `prp`
cc_library(name = "dev_e2_1") # only allowed to use this in `dev` environments

cc_library(
    name = "oss_e2_2",

    # need to specify the extra environments:
    compatible_with = [":prp_e2", ":oss_e2"],

    deps = [
        ":oss_e2_1", # okay; can be used for all environments
        # ":prp_e2_1", # !!! not okay: can't be used for `oss`
        # ":dev_e2_1", # !!! not okay: can't be used for `oss` or `dev`
    ],
)

cc_library(
    name = "prp_e2_2",
    compatible_with = [":prp_e2"], # can use this in `prp` (as well as dev, the default)

    deps = [
        ":oss_e2_1", # okay; can be used for all environments
        ":prp_e2_1", # okay: can be used for all environments that this target can be used for (prp and dev)
        # ":dev_e2_1", # !!! not okay: can't be used for `prp`
    ],
)

cc_library(
    name = "dev_e2_2",
    # only compatible with `dev` so don't need to set `compatible_with`

    deps = [
        ":oss_e2_1", # okay; can be used for all environments
        ":prp_e2_1", # okay: can be used for all environments that this target can be used for (dev)
        ":dev_e2_1", # okay: can be used for all environments that this target can be used for (dev)
    ],
)

e1.bzl

# NOTE: this does not work unfortunately; cannot set this attribute from the
# starlark side it appears
_ = '''
def oss_cc_binary(*, name, **kw):
    native.cc_binary(
        name = name,
        **(kw | {
            "$restricted_to": [Label("//:oss_e1")],
        })
    )
'''

_OSS_ENV = Label("//:oss_e1")
_PRP_ENV = Label("//:prp_e1")
_DEV_ENV = Label("//:dev_e1")

def oss_cc_binary(*, name, **kw):
    restricted_to = kw.pop("restricted_to", [])
    if restricted_to:
        filtered = []

        # cannot allow users to mark this target as accepting deps that aren't
        # oss
        #
        # this means disallowing adding `prp` or `dev`:
        #
        # (in practice this shouldn't actually matter? since there shouldn't
        # be deps other than OSS ones that have both oss + (prp OR dev))
        for env in restricted_to:
            env = native.package_relative_label(env)
            if env == _PRP_ENV or env == _DEV_ENV:
                fail(
                    "error in `restricted_to`: `oss_cc_binary` can only " +
                    "consume OSS content; you're not allowed to mark this " +
                    "target (" + name + ") as being compatible with ", env
                )
            elif env == _OSS_ENV:
                print("redundant environment in `restricted_to`:", env)
                continue
            filtered.append(env)

        restricted_to = filtered

    native.cc_library( # using `cc_library` for the sake of this example
        name = name,
        **(kw | {
            "restricted_to": [_OSS_ENV] + restricted_to,
        })
    )

MODULE.bazel