Cross-repository Maven

cross-repository-maven.md

Cross-repository Maven

Currently, Maven repositories follow a problematic centralised model. For example, the integrity of the software artifacts is checked by using digests that are located on the same directory as the artifact itself. Given that the verification of digital signatures currently implies additional work by the user, the vast majority of builds never check those signatures and rely on the digests.

This proposal describes how the implementers of Maven clients and repository managers could use a multi-repository model, which would give more choice to users while improving security.

This proposal only applies to releases, not snapshots. The basic idea is to deploy release artifacts to multiple repositories, then put the URLs of the other servers as metadata in the groupId/artifactId directory of each repository, and later use the additional Maven servers for increased reliability (e.g. using digests coming from a different server than the artifact).

This proposal is backwards-compatible with current Maven clients and servers, which could continue to operate as they do today.

This document assumes an Apache Maven configuration style, although developers of other Maven clients could easily extend it to their syntax.

Configuring repositories

In the <repositories> section of the POM or the settings.xml file, new configurations would be added: priority and prefixes (scope will be discussed later).

• priority can have the values (from more to less priority) main, normal (default) and metadata. Repositories with metadata priority will never be used to retrieve artifacts, only metadata (typically digests).

• prefixes would be used if that repository is intended to serve only artifacts with a given set of prefixes. prefix sub-elements would list the allowed prefixes, although it would be conceivable to have configurations to allow and disallow prefixes.

  If both the POM and settings contain a prefixes section for the same repository, the former is ignored.

  It is similar to the Remote Repository Filtering of Maven but located in the POM or settings.xml instead of the local repository. RRF is intended to speed up artifact resolution and not to express trust or policy. Therefore, the local prefixes.txt file could be easily overwritten by a remotely-supplied file.

Example:

     <repositories>
       <repository>
         <id>repo1</id>
         <name>Repository 1</name>
         <url>https://repo1.example.org/maven/</url>
         <!-- If no separate metadata server is found, this repo could be trusted
              for both artifacts and metadata (digests).
         -->
         <priority>main</priority>
       </repository>
       <repository>
         <id>repo2</id>
         <name>Repository 2</name>
         <url>https://repo.repo2.com/</url>
         <!-- Priority is 'main' but only applies when handling matching artifacts -->
         <priority>main</priority>
         <prefixes>
           <!-- Retrieve only artifacts matching the following prefixes -->
           <prefix>com.example.group</prefix>
           <prefix>com.example.foo.bar</prefix>
         </prefixes>
       </repository>
       <repository>
         <id>repo3</id>
         <name>Repository 3</name>
         <url>https://maven.repo3.net/</url>
         <!-- Deploy artifacts and digests, but when downloading artifacts
              trust only the metadata (digests and repositories.xml). -->
         <priority>metadata</priority>
       </repository>
       <repository>
         <id>repo4</id>
         <name>Repository 4</name>
         <url>https://repo.repo4.com/</url>
         <priority>normal</priority>
       </repository>
     </repositories>

Maven clients may configure Maven Central by default, giving it a main priority.

Note: Gradle already supports a content section which is very similar.

Distributed deployment process

The basic idea is that artifacts and/or digests coming from a single build would be deployed to all of the configured repositories, and a repositories.xml file would be created or updated at the artifact's groupId/artifactId directory in each server whenever that artifact (or its metadata) is available from additional repositories.

As the deployments would come from the same build, the artifacts and digests are guaranteed to be the same.

The repositories.xml file

For each artifact successfully deployed to a repository, whenever the files were deployed to more than one repository, the Maven client must create or update a repositories.xml file located in that artifact's groupId/artifactId directory, showing the url of the other repositories where the artifact is available, the latest release, as well as the date of last update (the date of the current deployment).

For the repo1 repository, the metadata file could have the following content:

<remote-repositories>
  <groupId>com.example.group</groupId>
  <artifactId>theartifact</artifactId>
  <repositories>
    <repository>
    <url>https://repo.repo2.com/</url>
      <release>6.2</release>
      <lastUpdated>20261203164753</lastUpdated>
    </repository>
    <repository>
    <url>https://maven.repo3.net/</url>
      <release>6.2</release>
      <lastUpdated>20261203164753</lastUpdated>
    </repository>
    <repository>
      <url>https://maven.outdated-repo.com/</url>
      <release>5.3.1</release>
      <lastUpdated>20250202112506</lastUpdated>
    </repository>
  </repositories>
</remote-repositories>

The value in the lastUpdated element should be the same as the one in the maven-metadata.xml file located in the same directory, at the time of update.

The Maven client must ignore any repository where the URL does not comply with the Repository URL syntax section. And should only use repositories that are independent from the target repository and from the others in the file (see Repository independence check.

The metadata must only contain the repositories where the artifact is actually available, and clients could use different strategies to achieve that:

• Leaving the repositories.xml metadata update until the end of the deployment (after the deployments to other servers finished successfully).
• Or updating it after the deployment, to remove one or more repositories where the deployment failed. The client must then restore the previous entry, if any.

Serialization should have Unix-style line endings (LF as the end of line).

Also, remember that Maven Central requires generating digests for any uploaded file.

Repository URL syntax

Any repository URL intended for use in the repositories.xml file must be rejected if it satisfies any of the following:

• The schema is different to https.

• The host is an IP address.

• The decoded path is longer than 256 characters or has any character other than ([0-9]|[a-z]|[A-Z]|-|_|/).

• Has an anchor (#).

Clients should also reject the URL if it has a query string (this aspect is left to feedback).

Repository independence check

The following procedures are suggested to check if two repositories with given URLs are independent:

• They do not resolve to the same IP address.

• The second-level domain name (or third level if there is a country-code second level domain, ccSLD) is different. Implementing this step rigorously would involve using a recent version of the public suffix list and clients may skip it.

For example:

• https://repo.maven.apache.org/maven2/ points to the same IP address as https://repo1.maven.org/maven2/ so they are not independent.

• repo.company.com and maven.company.co.uk are assumed to not be independent.

Clients may implement additional mechanisms for checking repository independence.

Repository cleanups

Repositories should have procedures to update the repositories.xml file when a repository closes or any other compelling reason arises. In that case the affected repositories should be commented out instead of just removing the relevant repository block. A comment should explain the reason for the exclusion, preferably pointing to a place with more information (like an issue tracker).

Example:

<repositories>
  <!-- Repository was closed, see https://issues.example.org/bug/1234 -->
  <!--
  <repository>
    <url>https://closed-repo.example.net/</url>
    <lastUpdated>20260615061232</lastUpdated>
  </repository>
  -->
</repositories>

Metadata-only deployments

Sometimes one may want to deploy only the metadata (including digests) to a repository: it could be a small server specialised in metadata, or a repository whose terms of use open a loophole for developer liability (in other language ecosystems, open source software developers have been targeted through package repositories and it is a matter of time before that happens with Java).

The scope element is proposed for that purpose. If the scope is metadata, only digests and metadata in general should be deployed to the remote repository. The default value is all.

The priority setting is used exclusively for artifact retrieval, while scope is only used to skip the artifacts in remote deployments.

In the pom.xml or settings.xml:

       <repository>
         <id>metadata-repo</id>
         <name>Metadata repository</name>
         <url>https://metadata-repo.example.com/</url>
         <!-- When downloading artifacts, trust artifacts (if any)
              and metadata -->
         <priority>normal</priority>
         <!-- Deploy only metadata, including digests -->
         <scope>metadata</scope>
       </repository>

In the above example our artifacts are not deployed to the repository but we trust the ones that are present (maybe uploaded by a trusted third party).

Post-deployment updates

Maven clients should be able to update the repositories.xml file in remote repositories after the deployment, to add or remove servers.

Adding a repository

To add a new repository R to a repositories.xml file located in a target repository T, outside of the context of an artifact deployment, follow these steps:

• Check that R matches Repository URL syntax.

• R and T are independent according to Repository independence check, otherwise skip.

• If R is already present in T's repositories.xml, either as a repository element node or as a commented-out repository, skip the update.

• Repeat the independence check for each repository element, abort if fails.

• If no artifact versions available in R are present in T, abort the update.

• For the latest versions of the artifact that are available in both R and T, download the digests from both servers. It is suggested to limit the downloads up to the latest 6 versions.

• If no digests are found at either R or T, abort the update.

• If any digest present in both R and T for the same artifact version, classifier and algorithm is different, abort the update.

• Use the DOM (or any equivalent procedure) of repositories.xml to add R as a repository child of the repositories element, with the release and the date for lastUpdated as found in R's maven-metadata.xml file.

• Serialize the resulting DOM with Unix-style line endings (LF as end of line) and upload the serialized document to T, replacing the previous one.

• Update the digests for the file (repositories.xml.sha1, etc).

It is discouraged to attempt adding a repository to a remote repositories.xml file unless reproducible builds were used, or the files were copied from the original repository.

Removing a repository

To remove a repository R from a repositories.xml file located in a target repository T, outside of the context of an artifact deployment, follow these steps (or any procedure with identical results):

• Obtain the DOM document of the target repositories.xml file from T.

• Find the repository block containing R in that document. If R is not present in T's repositories.xml, skip the update.

• Serialize that repository element with its content (the element and all childs with their content, although release may be ignored), with a layout consistent with repositories.xml. Prepend and append a Line Feed (LF).

• Create a comment node C with the above serialization as text.

• Replace the repository element by a comment node explaining the reason for removal. The reason must have been given as a command line parameter.

• Insert the comment node C as the next sibling of the comment node from the previous step.

• Serialize the resulting DOM with Unix-style line endings (LF as end of line) and upload the serialized document to T, replacing the previous one.

• Update the digests for the file (repositories.xml.sha1, etc).

Note: if the repository is being removed during the multi-repo deployment of the artifact, because the deployment was intended but failed, it must be removed as explained in the Distributed deployment section.

Artifact retrieval

Maven clients supporting distributed repositories must follow the following procedure when retrieving an artifact from a remote repository:

• Build a list of trusted repositories from the repositories configurations in settings.xml and pom.xml. If a repository has a content section that does not match the requested artifact, ignore it.

• If a repository has a prefixes.txt file that does not match the artifact group, omit that repository.

• If there is more than one repository with the same priority, they should be used in the order that they were specified (origin from settings.xml has precedence over pom.xml).

• The Maven client chooses a repository with the highest priority according to the precedence rules. If the highest priority is metadata, fail.

• Download the repositories metadata from repositories.xml in that artifact's directory in the repository.

• Check if the artifact with the desired version is available from the server.

• If the artifact is not available, repeat the process of obtaining repositories.xml and the artifact, with the remaining trusted repositories with normal or main priority (giving precedence to the latter).

• If no repository with main or normal priority in the trusted list has the artifact, fail.

• Once a repository containing the desired version of the artifact(s) was found, download the artifact(s) and digests. Check the artifact(s) with the digests, and at the end of this process the digests will be compared with the ones found on a different repository.

• From the repositories.xml metadata downloaded during the previous process obtain a list of repositories, excluding any URL that does not match the Repository URL syntax section.

• If the obtained list of relevant repositories contains any servers from the locally-specified trusted list, use the priority and precedence rules previously mentioned to choose one of them to retrieve a set of digests, which will be compared to the ones that were downloaded from the same server as the artifact.

• In case of error retrieving the digests, obtain the repositories.xml metadata from that server and fallback to other repositories in the trusted list (if they are listed in any of the repositories.xml files obtained so far).

• If necessary, use non-trusted servers obtained from the remote repositories.xml files (clients may let the user disable that behaviour through some configuration) to retrieve the digests. Clients must limit the depth of that repository discovery process.

• If that also fails and the artifact was downloaded from a repository with main priority, the build succeeds. Otherwise fail the build.

• Compare the two sets of digests obtained so far. If there is any mismatch, fail the build.

• Maven clients should provide a way for users to prevent certain repositories, provided by remote repositories.xml files, from being used, and should by default provide a separate list of known closed or malicious repositories, directly updatable with new versions of the software.

Version ranges and latest releases

Sometimes a version is specified as RELEASE and Apache Maven attempts to retrieve the latest version (also true when an open version range like [1.1,) is used). When discovering versions of artifacts across different repositories, Maven clients may use the lastUpdated date and the latest release from the repositories.xml file to find out which repositories are more up-to-date.

Security and privacy considerations

Security

Given that artifacts will only be downloaded from repositories specified in the settings.xml and pom.xml files, and remotely discovered servers would only be used to retrieve digests, the surface for attacks from a discovered malicious server would be small.

A theoretical concern would be that it could attempt to exploit vulnerabilities in the HTTP implementation of the Maven client, but those are unlikely to happen.

A malicious remote repository could also try to throttle the HTTP connections, but clients should configure reasonable timeouts. Clients should also limit the size of the files retrieved from a remotely discovered server.

The XML parsers used by the client must be configured to avoid XML attacks (the techniques used for preventing those attacks are well known, and outside of the scope of this document).

This proposal would result in an improvement of security, as the only integrity check that clients perform automatically is to verify the digests, and now they could be obtained from independent repositories. So in the event that a trusted repository was compromised, the artifacts could be checked with the digests from another server if available, and the build would fail.

Privacy

If server URLs are filtered according to the Repository URL syntax section, it is unlikely but not impossible that they could be used for identifying the client when it connects to the new server, especially if the repository URL was dynamically generated (by encoding a session identifier in the URL path), but that could barely have any privacy impact.