Proposal: Add payload migration hooks to the Orleans serializer

orleans-serialization-migration.md

Proposal: payload migration hook for Orleans serialization

Problem

Orleans handles many additive/removal DTO changes well through field ids. The harder rolling-upgrade case is a semantic shape change where one deployed revision sends an aliased DTO shape while another deployed revision expects a different current shape.

Concrete cases:

• silo-to-silo or client-to-silo calls during blue/green and rolling deployments;
• responses crossing between different deployed versions;
• Orleans stream items, including persistent stream payloads already stored in an external broker such as Azure Event Hubs;
• nested DTOs inside any of the above.

Today the receiving code generally has to keep obsolete shapes in grain/observer APIs, add translation code at every boundary, write custom codecs, or synchronize deployment tightly enough that old payloads cannot arrive. The desired behavior is narrower: if Orleans can identify that the wire payload is OrderSubmittedV1 and the local call site expects OrderSubmittedV2, the serializer should be able to deserialize V1, run a registered migration, and return V2 to application code.

Deployment model

This proposal assumes the existing Orleans constraint that every active silo revision must know the serialized types which any other active revision may put on the wire. Migration does not remove that requirement. It lets each revision keep its business logic typed to the DTO shape it actually uses.

A typical rollout would be:

1. Revision A uses UserV1.
2. Revision B still uses UserV1, but adds UserV2 plus migrations in both directions between UserV1 and UserV2.
3. Revision B is deployed everywhere, so all active silos know both aliases and both codecs.
4. Revision C starts using UserV2 in grain/client/stream contracts.
5. Revision C can now run beside Revision B. B still writes/expects UserV1; C writes/expects UserV2; the serializer adapts payloads at the boundary.

The result is not arbitrary cross-version type discovery. It is a staged compatibility bridge which lets neighboring deployed revisions communicate without either revision's application code handling the other's UserVx shape.

Proposal

Add an opt-in migration step at the serializer boundary where an actual payload type is known and is not assignable to the expected type.

In effect:

1. Read the actual type using existing Orleans type metadata.
2. Deserialize the payload as that actual type.
3. If the value is assignable to the expected type, return it unchanged.
4. Otherwise, look for a registered actual -> expected migration.
5. If found, return the migrated value.
6. If not found, preserve the current failure behavior.

This should not change the wire format. Existing [Alias] metadata remains the stable discriminator for long-lived payloads. The normal same-type path should stay on the existing fast path.

Non-goals

• Do not solve grain interface or method routing/versioning. This proposal only adapts payload values after Orleans has selected a compatible invocation/stream delivery path.
• Do not replace storage-provider-specific state migration. The target scenario is Orleans-serialized payloads in RPC, responses, streams, and nested DTOs.
• Do not silently reinterpret same-type payload bodies by default. Brownfield data whose header says "expected type" but whose body is an old shape should require explicit opt-in, if supported at all.
• Do not require cyclic object-graph migration in the first version.

Developer model

One possible API shape:

public interface IMigrateFrom<TSource, TTarget>
    where TTarget : IMigrateFrom<TSource, TTarget>
{
    static abstract bool TryMigrateFrom(TSource source, out TTarget result);
}

public interface IMigrate<TSource, TTarget>
{
    bool TryMigrateFrom(TSource source, out TTarget result);
}

Example:

[GenerateSerializer, Alias("order-submitted-v1")]
public sealed record OrderSubmittedV1(...)
    : IMigrateFrom<OrderSubmittedV2, OrderSubmittedV1>
{
    public static bool TryMigrateFrom(
        OrderSubmittedV2 source,
        out OrderSubmittedV1 result)
    {
        result = new OrderSubmittedV1(...);
        return true;
    }
}

[GenerateSerializer, Alias("order-submitted-v2")]
public sealed record OrderSubmittedV2(...)
    : IMigrateFrom<OrderSubmittedV1, OrderSubmittedV2>
{
    public static bool TryMigrateFrom(
        OrderSubmittedV1 source,
        out OrderSubmittedV2 result)
    {
        result = new OrderSubmittedV2(...);
        return true;
    }
}

Static target-owned migrations are source-generator and trimming friendly. External IMigrate<TSource,TTarget> services cover cases where the target type is not owned by the application or the migration needs DI.

Runtime shape

The core primitive could be small:

• a migration registry keyed by (sourceType, targetType);
• generated registrations for static IMigrateFrom<TSource,TTarget> implementations;
• explicit registration for external migrators;
• one migration attempt when unexpected-type deserialization produces a non-assignable value.

Conceptual hook:

var source = DeserializeActualType(field.FieldType);

if (source is TExpected expected)
{
    return expected;
}

if (migrationRegistry.TryMigrate(field.FieldType, typeof(TExpected), source, out var migrated))
{
    return (TExpected)migrated!;
}

return CurrentFailureBehavior(source);

The important design point is not the exact helper location. It is that Orleans already has enough information at the field/value boundary to know both the encoded actual type and the local expected type. That boundary is where migration gives the most leverage across RPC arguments, responses, streams, and nested DTOs.

Compatibility expectations

• Every active silo revision must know all types and aliases which may be serialized by any other active revision.
• New DTO shapes should be introduced in a compatibility revision before any later revision starts using them in business logic.
• Compatibility revisions can continue writing old aliased DTOs while already knowing the new aliases and codecs.
• Revisions using the new DTO shape write current DTOs normally.
• Active revisions register migrations for the source/target pairs they may receive during the rollout.
• Applications keep source DTOs and migrations for at least the relevant message/data retention window.
• Missing or failed migrations should throw by default with source alias/type, target alias/type, and migrator details.

Observability

At minimum, expose logs or metrics for:

• migration attempted;
• migration succeeded;
• migration failed;
• no migration registered for a source/target pair;
• duplicate/conflicting migration registrations.

Tests worth adding

1. Deserialize an aliased old DTO as a current expected DTO using a static migration.
2. Do the same with an explicitly registered external migrator.
3. Verify an Orleans stream item written as V1 can be delivered to a V2 observer, including a persistent stream-provider-style payload.
4. Verify nested DTO migration.
5. Verify no migration lookup happens on the same-type fast path.
6. Verify unknown aliases and missing migrations preserve current failure semantics except for better diagnostics.
7. Verify source-generator/AOT registration without reflection scanning.
8. Verify a staged rollout case where one revision knows V1 and V2 but uses V1, while another revision uses V2.
9. Document or reject cyclic graph behavior in MVP tests.

Questions for the Orleans team

• Is the unexpected-actual-type path the right minimal hook, or is there a better serializer boundary for this?
• Should the API live in Orleans.Serialization or an optional package?
• Should Orleans support both static target-owned migrations and DI migrators, or start with only one?
• Should migration sources be required to have [Alias]?
• Should Orleans provide validation or diagnostics for "known but unused" compatibility types in staged rollouts?
• Should brownfield same-header/old-body payloads be in scope, or left to custom codecs/provider migration?
• Is non-cyclic DTO support an acceptable MVP constraint?

Suggested MVP

1. Add migration contracts and explicit registration APIs.
2. Build a source/target migration registry from codegen metadata and explicit registrations.
3. Attempt migration only when the actual decoded type is not assignable to the expected type.
4. Keep the wire format and same-type fast path unchanged.
5. Document alias requirements, retention-window guidance, failure behavior, and cyclic graph limitations.