opaque-sip.md

Pre-SIP: Transparent members for Opaque types

I think opaque type aliases is one of Scala 3's most interesting features. Many use cases for it has already been discussed. One oppertunity that I think would be quite useful, while at the same time being a very good fit with the rest of the language, is the ability to expose members of the underlying type as members on the opaque type.

For example: The current imlementation of IArray uses extension methods to export methods from Array:

extension (arr: IArray[Byte]) def apply(n: Int): Byte = arr.asInstanceOf[Array[Byte]].apply(n)
...
extension [T <: Object](arr: IArray[T]) def apply (n: Int): T = arr.asInstanceOf[Array[T]].apply(n)
extension [T](arr: IArray[T]) def apply (n: Int): T = arr.asInstanceOf[Array[T]].apply(n)

...

extension (arr: IArray[Byte]) def length: Int = arr.asInstanceOf[Array[Byte]].length
...
extension (arr: IArray[Object]) def length: Int = arr.asInstanceOf[Array[Object]].length
extension [T](arr: IArray[T]) def length: Int = arr.asInstanceOf[Array[T]].length

Given that re-exporting methods on the underlying type is probably a quite common use case, I think exposing a subset of the underlying type would be very useful.

People have proposed the ability to export methods before, but I've not seen a bigger discussion about it:

https://contributors.scala-lang.org/t/having-another-go-at-exports-pre-sip/2982/22?u=mbloms https://contributors.scala-lang.org/t/proposal-for-opaque-type-aliases/2947/54?u=mbloms

Syntax is always hard, for now I'm borrowing the new syntax @odersky introduced for givens/structural instances. Using that, a concrete way to extend opaque types to support this could look like this:

opaque type Nat with {
  def +(x: Nat): Nat
} <: Int = Int

opaque type IArray[+T] with {
  def apply(i: Nat): T
  def clone(): IArray[T]
  def length: Nat
} = Array[_ <: T]

Of course, the transparent members can't override the implementation of underlying members, but they should be able to override the type signatures in a compatible way.

The overriding type signature of a transparent member must be compatible with:

• The underlying type as seen from inside the scope of the definition
• The lower bound as seen from outside the scope of the definition
• The upper bound as seen from outside the scope of the definition

One way to guarantee this could be that given the definition

opaque type T with {def m(x: A): B} >: Lo <: Hi = Rep

• There must be a member definition of m in Rep where the erasure of m in T matches the erasure of m in Rep
• There must be a matching member definition of m in Lo, and m in Lo must subsume the definition of m in T
• If there is a matching definition of m in Hi, then m in T must subsume the definition of m in Hi

Where the definition of matching and subsumes is taken from the 5.1.3 and 3.5.2 respectively of the Scala Language Specification.

Join of types with transparent members

The fact that transparent members correspond to real members could be used to preserve them in joins:

trait C
  def children: List[C]

object opaques:
  opaque type A {
    def children: List[A]
  } = C
  opaque type B {
    def children: List[B]
  } = C

import opaques._

def x: A & B = ...
def xs: List[A & B] = x.children

def y: A | B = ...
def ys: List[A | B] = y.children

Normally, the join of A and B would be Any since A and B is unrelated to each other. With transparent members, the fact that children in A and B is owned by the same base class is known on the outside. In other words the ownership of a transparent method is transparent.

Using my own notation I express the join of A and B like this: The join of A | B is {this: opaque C => def children: List[A | B]}

Motivating examples

At first, it might not be obvious why this would be useful, but I would say there are

Subproposal - support export clauses:

Something something blah blah I want this:

opaque type IArray[+T] = Array[_ <: T] with {
  export this.{apply,clone,length}
}

And also this:

object Defs {
  opaque type Name = String

  extension (n: Name) {
    export n.length
  }
  def newName(s: String): Name = s
}

I think the problem with export clauses in general. For instance consider this example which do not use opaque types:
https://gist.github.com/mbloms/31f8c8c32cfb0dec2d636d5f55ad9499

Before I realised Builder has the mapResult method, my first thought was simply to make an anonymous builder and export methods from ArrayBuilder. If it wasn't for mapResult I would have had to redefine every method in Builder that return this explicitly.

Perhaps in this special case, this could be solved by attaching signatures in the export clause, but that would not match existing syntax. In any case, there are definitely cases where you don't necessarily want to blindly replace the original type with the opaque version.

I see what you mean. Indeed each usage of export must be checked if anything leaks or not, or has an appropriate/desired signature as a surface.

I think it's important to focus on what the most common use case for transparent members will be, and IArray isn't it. What I see as the most common use case -- and, of course, I might be wrong about it -- is addressing primitive obsession. It's replacing String and the AnyVal subclasses with opaque types without performance impact or the drudgery of implementing forwarding methods.

But that also means curating where the backing type is replaced on method type signatures, and where it is preserved and, for that matter, where the type signature needs to be changed outright. Some examples:

• contains on String takes a CharSequence, but it makes more sense for opaque types to take themselves instead.
• /(x: Int): Int on Int should become /(x: Opaque): Opaque, but /(x: Double): Double (etc) should not be exported at all.
• >>(n: Int): Int on Int should become >>(n: Int): Opaque.
• Any methods shouldn't be overloaded at all, which I suppose is the status quo.

Given that, I think the most common use cases can solved by the library instead. Provide OpaqueInt, OpaqueString et cetera traits covering the "primitives" (String being the outlier) with carefully curated method forwarding and that should take care of most use cases without any added complexity to the language.

How would this library solution be used in practice? Is OpaqueString a trait or is it some kind of annotation?

Anyway: personally I don't really see why it's a problem to explicitly define the type signature of contains of an opaque alias of String for instance. Isn't it clearer to have it explicitly written out?

What I'm mostly concerned is:

• using extension methods as forwarders incur some overhead
• they aren't part of the type so they don't work for structural types
• joins are weird: OpaqueInt | Int only have the members from Any
• they are more boilerplate than they have to be
• other stuff I can't think of at the moment

Getting rid of boilerplate is not the main objective here, it's a secondary desirable side effect.

I want the semantics of this proposal before I can start building the library

OpaqueString would be a trait or class. It would be used like this:

object Name extends OpaqueString
type Name = Name.Type

It would provide all of String methods with adjustments like contains(other: Name): Boolean instead of contains(other: CharSequence): Boolean. And, of course, it would not be a String, nor will its methods accept or return String unless appropriate.