Snapshot testing in Velopack

README.md

How to do snapshot testing on OpenAPI spec

This is a simple writeup of how snapshot testing is done on the OpenAPI spec for Velopack API. Though the implementation can be easily applied to other services.

Setup the API project

The API project should include an OpenAPI endpoint. The most popular libraries for this are Swashbuckle, NSwag, and soon built-in support.

The API project should be configured to serve up the Open API specification. For Velopack, we are using NSwag. The setup in the API resembles the following:

builder.Services.AddOpenApiDocument(options =>
{
    options.Title = "Velopack Flow";
    options.Description = "Velopack's host solution for managing releases";

    //options.SchemaFilter<EnumSchemaFilter>();
    var securityScheme = new NSwag.OpenApiSecurityScheme
    {
        Name = "Auth",
        Type = NSwag.OpenApiSecuritySchemeType.OpenIdConnect,
        OpenIdConnectUrl = "https://velopackflow.b2clogin.com/velopackflow.onmicrosoft.com/tfp/B2C_1_velopack_signin///v2.0/.well-known/openid-configuration",
        In = NSwag.OpenApiSecurityApiKeyLocation.Header,
        Scheme = "Bearer",


        ExtensionData = new Dictionary<string, object?>()
        {
          // Setting x-tokenName to id_token will send response_type=token id_token and the nonce to the auth provider.
          // x-tokenName also specifies the name of the value from the response of the auth provider to use as bearer token.
          // See https://github.com/swagger-api/swagger-ui/issues/7561
          { "x-tokenName", "id_token" }
        },
    };
    options.AddSecurity(securityScheme.Name, securityScheme);
    options.OperationProcessors.Add(new OperationSecurityScopeProcessor(securityScheme.Name));
});
...
app.UseOpenApi();
app.UseSwaggerUi(options =>
{
    options.EnableTryItOut = true;
    options.DocExpansion = "list";
    options.DocumentTitle = "Velopack Flow";
    options.OAuth2Client = new()
    {
        UsePkceWithAuthorizationCodeGrant = true,
        ClientSecret = "",
        AppName = "Velopack API",
        Scopes =
        {
            "openid",
            "offline_access"
        }
    };
    options.PersistAuthorization = false;
    options.WithCredentials = true;
});

When running the API the Open API document is available on the route: http://localhost:5582/swagger/v1/swagger.json.

Setup the test project

For doing the snopshot testing, we are using Verify.Xunit. This NuGet package must be added to the test project.

We leverage WebApplicationFactory for integration testing the API project. To facilite its usage, the following base class is used for all of the integration tests to support setting up logging output to forward to xUnit.

public abstract class EndpointTestsBase : TempFileTestBase, IClassFixture<WebApplicationFactory>, IDisposable
{
    protected readonly WebApplicationFactory _factory;
    protected readonly ITestOutputHelper _testOutput;
    private bool _disposedValue;

    public EndpointTestsBase(WebApplicationFactory factory, ITestOutputHelper testOutput)
    {
        _factory = factory;
        _testOutput = testOutput;
        _factory.AddTestLogger(testOutput);
    }

    protected override void Dispose(bool disposing)
    {
        if (!_disposedValue)
        {
            if (disposing)
            {
                _factory.RemoveTestLogger(_testOutput);
            }
            _disposedValue = true;
        }
        base.Dispose(disposing);
    }
}

Then to perform the snapshot test, the test class is simply this:

public class PublicApiSurfaceTests : EndpointTestsBase
{
    public PublicApiSurfaceTests(WebApplicationFactory factory, ITestOutputHelper testOutput) 
        : base(factory, testOutput)
    {
    }

    [Fact]
    public async Task PublicApiIsUnchanged()
    {
        HttpClient client = _factory.CreateClient();

        string openApiSpec = await client.GetStringAsync("swagger/v1/swagger.json");

        await Verify(openApiSpec);
    }
}