Swift IP-address and Port API Proposal

IP-address-and-port-API-proposal.md

Introduction

This text is written for Requirements for IP address and port APIs Swift forums post.

I first go through prior arts regarding "IP-address + port" implementations. Then I draw conclusions, and at last, I propose an API Shape to define the scope of this core library.

Prior Art

Preface

What is a "CIDR" / "IPNetwork" / "Prefix"?

In the process of writing this post, I had to go through ensuring all details are correct and up to date, at which point I noticed there is a relatively new IETF Proposed Standard RFC 9911; Common YANG Data Types, in which the authors describe 2 family of types named as ip[v4/v6/]-prefix and ip[v4/v6/]-address-and-prefix, alongside lots of other concepts.

The definitions are currently as follows (some details excluded for brevity, indicated by ...):

 typedef ip-prefix {
    type union {
      type ipv4-prefix;
      type ipv6-prefix;
    }
    ...
  }

  typedef ipv4-prefix {
    type string {
      pattern
        '(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\.){3}'
      + '([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])'
      + '/(([0-9])|([1-2][0-9])|(3[0-2]))';
    }
    description
      "The ipv4-prefix type represents an IPv4 prefix.
       The prefix length is given by the number following the
       slash character and must be less than or equal to 32.

       A prefix length value of n corresponds to an IP address
       mask that has n contiguous 1-bits from the most
       significant bit (MSB) and all other bits set to 0.

       The canonical format of an IPv4 prefix has all bits of
       the IPv4 address set to zero that are not part of the
       IPv4 prefix.

       The definition of ipv4-prefix does not require that bits
       that are not part of the prefix be set to zero.  However,
       implementations have to return values in canonical format,
       which requires non-prefix bits to be set to zero.  This means
       that 192.0.2.1/24 must be accepted as a valid value, but it
       will be converted into the canonical format 192.0.2.0/24.";
  }

  typedef ipv6-prefix {
    ...
  }

  typedef ip-address-and-prefix {
    type union {
      type ipv4-address-and-prefix;
      type ipv6-address-and-prefix;
    }
    ...
  }

  typedef ipv4-address-and-prefix {
    type string {
      pattern
        '(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\.){3}'
      + '([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])'
      + '/(([0-9])|([1-2][0-9])|(3[0-2]))';
    }
    description
      "The ipv4-address-and-prefix type represents an IPv4
       address and an associated IPv4 prefix.
       The prefix length is given by the number following the
       slash character and must be less than or equal to 32.

       A prefix length value of n corresponds to an IP address
       mask that has n contiguous 1-bits from the most
       significant bit (MSB) and all other bits set to 0.";
  }

  typedef ipv6-address-and-prefix {
    ...
  }

As defined above, an ip[v4/v6/]-prefix:

The canonical format of an IPv4 prefix has all bits of
the IPv4 address set to zero that are not part of the
IPv4 prefix.

Which matches the notion of a IPNetwork that we discussed on Swift forums, and which we'll see more in the prior arts section, and counts the host bits as insignificant.

An ip[v4/v6/]-address-and-prefix on the other hand is analogous to a CIDR, where host bits can also be important.

Both ip[v4/v6/]-prefix and ip[v4/v6/]-address-and-prefix would hold two members in Swift. One an address, one a prefix (or the corresponding mask), but in ip[v4/v6/]-address-and-prefix, the address can be any address, while in ip[v4/v6/]-prefix, it must not care about any "non-prefix bits" (aka "host bits").

What you need to know for the text below is that when I mention ip-prefix and ip-address-and-prefix, I'm referring to the definitions above.

To put it in simpler words:

	is canonical ip-address-and-prefix ?	is canonical ip-prefix ?
192.168.1.0/24	Yes	Yes
192.168.1.98/24	Yes	No
2001:db8:0:cd30::/60	Yes	Yes
2001:db8:0:cd30:123:4567:89ab:cdef/60	Yes	No

Note that although some are not a canonical ip-prefix, all must still succeed as inputs to a ip-prefix type, just that such a type would not care about the extra "host bits".

Every canonical ip-prefix is a valid canonical ip-address-and-prefix. The reverse is not true.

In 192.168.1.98/24, the host bits are the trailing .98, since they are masked-out by the /24 mask. For less savvy readers /24 mask is essentially saying "keep the 24 first (most significant) bits, mask out the remaining 32-24=8 bits". The remaining 8 bits in the above case are the .98, which are called the "non-prefix bits" / "host bits".

The current CIDR implementation in swift-endpoint is analogous to a ip-address-and-prefix. As discussed in the Conclusion section, we might have to give CIDR a more standard name. Below you'll see what other libraries mean with IPNetwork / Prefix. Sometimes they disagree.

It's also worth being clear that there is no such requirement to follow Proposed Standard RFC 9911. RFC 9911 defines the concepts in a clear manner, so in my opinion it's worth following for that. In the text below you'll see that some libraries do get the concepts mixed up, or have previously gotten it mixed up and have since patched their implementation.

C (POSIX / BSD)

Reference: glibc manual; internet namespace

Contains in[6]_addr, sockaddr_in[6].

Contains sin6_flowinfo, sin6_scope_id fields.

While Linux's glibc is linked above, a lot of other platforms such as Darwin also include the common required members, thanks to POSIX compliancy.

Lots of other languages/libraries and APIs have been understandably inspired by these APIs and some closely follow form.

Swift

References: network, SwiftNIO

SwiftNIO

Contains SocketAddress, SocketAddress.IPv4Address, SocketAddress.IPv6Address, SocketAddress.UnixSocketAddress.

These are Swift wrappers for the C types.

IPv6Address contains flow-info and scope-id although it doesn't expose public setters for them and the getters need reaching through a C struct.

Notably IPv[4/6]Address types contain ports in themselves which is generally uncommon, except for Zig net modules which do the same.

Network.framework

Contains IPv4Address, IPv6Address, IPAddress, IPv6Address.Scope, NWInterface.

Contains NWEndpoint which contains even higher-level concepts of where to connect to, such as a URL.

NWEndpoint contains NWEndpoint.Port, NWEndpoint.Host. NWEndpoint.Host looks somewhat close to swift-endpoint's ConnectionTarget, but contains no port or Unix domain socket address. NWEndpoint.Host contains cases name(String, NWInterface?), ipv4(IPv4Address), ipv6(IPv6Address).

As visible, uses strings for domain / hostnames.

Contains TCP / UDP functionalities.

Rust

References: std::net, tokio::net, smol::net

Contains Ipv4Addr, Ipv6Addr, IpAddr, SocketAddrV4, SocketAddrV6, SocketAddr. SocketAddrV4 contains IPv4 + port. SocketAddrV6 contains IPv6 + port + flow-info + scope-id.

Contains Ipv6MulticastScope which is currently nightly-only (since a decade ago).

Contains no domain / hostname related types in stdlib.

Contains TCP and UDP stream/listeners.

There is a reference issue in the rust repo tracking convenience IP address methods: rust-lang/rust#27709.

Runtimes such as tokio and smol use the std::net primitives.

Zig

Reference: std.net, std.Io.net

std.net (legacy)

Contains Ip[4/6]Address, Address.

Ip[4/6]Address contain a port number as well.

Ip6Address contains flow-info and scope-id as well.

Address is an IP address or a Unix domain socket address.

Contains some more TCP / UDP functionalities.

std.Io.net

Contains HostName, Interface, Ip[4/6]Address, IpAddress, UnixAddress.

Ip[4/6]Address contain a port number as well.

Ip6Address contains flow-info and scope-id (named as interface) as well.

UnixAddress is a Unix domain socket address.

Contains some more TCP / UDP functionalities, including hostname resolution.

Contains no IDN (Internationalized Domain Names) functionalities or such in stdlib alongside the HostName support.

C++ (Boost ASIO)

Reference: Boost

Contains ip::address, ip::address_v[4/6], ip::network_v[4/6], ip::basic_endpoint.

ip::network_v[4/6] are ip-address-and-prefix (not ip-prefix) implementations. Notably, other implementations (Python and .NET's IPNetwork) where they call it a "Network", are ip-prefix implementations instead.

Contains more TCP / UDP functionalities, including hostname resolution.

Golang

Reference: net, net/netip

net (legacy ip implementations)

Contains loose types such as type IP []byte which is either 4 or 16 bytes. Contains:

type IPAddr struct {
	IP   IP
	Zone string // IPv6 scoped addressing zone
}

Uses raw strings for domain / hostnames.

Notably, does not contain any flow-info.

Contains lots more networking APIs.

netip

Contains Addr, AddrPort, Prefix.

AddrPort is a "socket address" containing an IPv4 or IPv6, including the zone and port.

Addr is AddrPort minus the port.

Prefix is a ip-address-and-prefix (explicitly not ip-prefix) implementation.

Notably, AddrPort / Addr do not contain any flow-info.

Contains no other types.

The API surface is something similar to what I'm envisioning for swift-endpoint. However, in swift-endpoint (and generally in Swift) we're being more explicit and clearly defining each sub-type before moving on to the next concept. Meaning that to mimic netip, we'd need to have the following types: AnyIPAddress, IPv4Address, IPv6Address, CIDR (likely to be renamed), Port, and some unimplemented ones such as IPEndpoint or similar.

JavaScript (nodejs)

Reference: net

Contains SocketAddress.

SocketAddress contains flow-label but not scope-id. To be clear, the relation between a flow-label and flow-info is as follows: flow-info (32 bits) = version (4 bits) + traffic-class (8 bits) + flow-label (20 bits).

Sometimes uses strings as ip addresses. Hostnames are strings as well.

Contains subnet-containment checks but through net.BlockList.

Python

Reference: internet category, socket category

internet category

Python's internet category is large in scope. What we are more interested in is the ipaddress module.

ipaddress contains class ipaddress.IPv4Address, IPv6Address, IPv[4/6]Network, IPv[4/6]Interface.

IPv[4/6]Interface are ip-address-and-prefix implementations, and IPv[4/6]Network are ip-prefix implementations, with minor divergences. Specifically, IPv[4/6]Interface preserves host bits and IPv[4/6]Network rejects them (raises a ValueError) by default and masks them when strict=False.

IPv6Address can contain its related scope-id, but not flow-info.

No specific domain / hostname types are exposed.

I personally see Python as a good reference for what users expect when they want things to just get out of the way and "just work".

socket category

Contains a socket tuple.

socket tuple for IPv6 is defined as (host, port, flowinfo, scope_id).

Contains more networking functionalities.

Java

Reference: java.net

Contains IDN, Inet4Address, Inet6Address, InetAddress, InetSocketAddress, InterfaceAddress, UnixDomainSocketAddress.

Inet6Address contains scope-id, but not flow-info.

Contains a bunch more HTTP-related types.

Contains Multicast scope classifiers such as isMCGlobal, isMCLinkLocal, isMCNodeLocal, isMCOrgLocal, isMCSiteLocal.

Uses strings for hostnames, but does contain IDN (Internationalized Domain Names) functionalities through the IDN class. The IDN implementation follows IDNA2003 which is obsoleted in favor of IDNA2008.

C# (.NET)

References: System.Net, IPNetwork, IPNetwork behavior change discussion, IPNetwork normalization implementation

Contains IPAddress, IPEndPoint, IPNetwork.

IPAddress contains scope-id, but not flow-info.

Starting with .NET 10, IPNetwork is an ip-prefix which closely follows Proposed Standard RFC 9911. The public documentation is stale and still describes the pre-.NET-10 behavior of rejecting non-zero host bits.

Contains a lot more networking functionalities.

Conclusion

DomainName / HostName / IDN

Domain / hostname is not a type that is exposed in most of the libraries above. This makes me conclude that swift-endpoint's DomainName type falls out of scope of a core "IP-address + port" library. DomainName's usage is not limited to DNS. For example TLS's SNI extension requires such concept. So we should ensure all libraries are able to use such a type without a bigger dependency such as a DNS library.

Note that DomainName and HostName would have their own differences and we'll have to decide the details.

Generally speaking, swift-endpoint itself as a package is a valid goal to follow, but for now, and as what is likely the intention of this discussion, we should focus on IP-address types and their close relatives. Likely we should migrate the exact APIs we want from swift-endpoint's implementations to another repository.

IPv6Address Multicast Scope

Network.framework has had IPv6Address.Scope since iOS 12. Rust appears not to be willing to commit to stabilize its Ipv6MulticastScope type. Overall I really like the idea of having such a type as well to not have to memorize each number and its related multicast scope meaning and such.

ip-prefix / ip-address-and-prefix / IPNetwork / CIDR

My conclusion is that an ip-prefix / ip-address-and-prefix type is likely a good choice to have, since implementing IP properties such as isLinkLocalUnicast conceptually rely on them although do not necessitate them.

So even if the Swift library does not contain an ip-prefix / ip-address-and-prefix type, it'll have to have an internal notion of it, at which point "code-size" concerns are weakened, and the question might become that "is having an ip-prefix / ip-address-and-prefix type actually useful?"; to which I'd answer: yes. These are well-established concepts, and if you're working with IP addresses, you're likely to bump into them.

I'd lean on the ip-address-and-prefix type's side for no other reason than it's lossless (keeps host bits) so for a general audience who might not 100% know the details and differences between ip-prefix and ip-address-and-prefix, it'll "just work".

As mentioned above, some language/libraries do have such a ip*prefix type as well, so it wouldn't be without precedent: C++ Boost, Python, Golang and .NET. Furthermore, a lot of higher level libraries (Rust's hyper-util and hickory-proto) have had to somehow find their way to such a type. All in all, I think an ip-prefix / ip-address-and-prefix type is too foundational and useful to not have.

About the naming, CIDR is fine but it's not what we intend it to be. Afterall, CIDR means "Classless Inter-domain Routing" and this "ip-address + port" library would likely want to not concern itself with "routing".

Port

Most if not all libraries above simply use some kind of integer for port. Given Swift's ExpressibleByIntegerLiteral functionality and Network.framework's prior art (NWEndpoint.Port), I think a cheap wrapper type around a port is a good addition. We can then expose a minimum set of port-specific functionalities, for example having static accessors for IANA-registered service ports for ease of use. This is the way swift-endpoint currently implements Port. See my prior post.

ConnectionTarget

Falls out of scope since it would need to be able to accept domain / hostnames. For now my conclusion is that it should be implemented near whatever DNS library we'll have, possibly a layer behind the DNS library. We can then use the DomainName / HostName type that the DNS library requires, as well as relying on that DNS library to resolve domain / hostnames. This will take some more work to be able to properly replace system resolver function (getaddrinfo), for example to implement hosts file parsing and respecting resolv.conf/host.conf / nsswitch.conf by default.

swift-dns currently contains a hosts-file parser but no resolv.conf parser. swift-dns's hosts-file functionality is not fully implemented so the parser is largely unused although it is properly implemented and tested. If we want to take such a route to use a pure-Swift resolver by default in the ecosystem behind the scenes (for example right in SwiftNIO), we should study Golang's experience with using a pure-Go resolver first: https://pkg.go.dev/net#hdr-Name_Resolution.

NAT64

Falls out of scope. Perhaps we should have a swift-routing library that exposes some of these concepts as Craig and Keely look interested in them.

There are very few NAT64-related functionalities in swift-endpoint. They should be removed so a proper NAT64 implementation can take care of them instead.

UnixDomainSocketAddress

Having this type in the proposed "ip-address and port" library can be controversial. Prior Art is split on it.

C / POSIX, SwiftNIO, Network.framework, Java, .NET and Zig agree on just having it around the final "SocketAddress" type. Rust's std::net::SocketAddr contains only IPv4 / IPv6. Go's new netip.AddrPort does not include it. It's defined as net.UnixAddr. C++'s Boost contains local::*_protocol::endpoint which is separate from the ip:: namespace. Nodejs accepts strings for a Unix domain socket address. net.SocketAddress does not include it. Python accepts it as string or bytes, and does not include a concrete "SocketAddress" type at all.

I'm personally in favor of having it in the proposed "ip-address + port" library. Can't think of another good place for it either. Like would we create another package just to add a UnixDomainSocketAddress type? Having it in the same target as wherever ConnectionTarget will live also looks a bit too far, although is not the worst idea.

Would be an enum which follows Linux Unix(7) and other platforms' documentations. Would provide some convenience methods and functionalities. While this sounds like a Unix-specific type, it practically isn't. Even Windows supports it since 2017. Few platforms like WASI don't support it but that shouldn't stop us.

IPEndpoint / SocketAddress

Most of these libraries contain this as "Socket Address" or with similar "Endpoint" namings. Apple does have a NWEndpoint type which sets precedent for "Endpoint" usage, but it contains much more than just a "Socket Address". Generally it looks like the libraries above agree more on the "Socket" naming while "Endpoint" would also be a valid name. This includes SwiftNIO's "Socket" namings which has taken lots of inspiration from Java. The exact API shape is TBD. Will certainly have to support scope-id, likely flow-info as well, for IPv6.

Byte Order

These libraries' public APIs generally look unbothered by byte-order concerns of interoping with C. In the sense that they don't have lots of byte-ordering related public APIs / arguments or such. I think we should generally follow suit, but make byte-orderings clear in docs. Especially via examples understandable to users less familiar with byte-orderings. Perhaps in a few places we might want to expose a byte-order argument or such. Exact API shape is TBD since I'll have to go through an implementation.

Proposed API Shape

All in all, I propose the following high-level types to exist in the core "ip address and port" library:

• IPv4Address, IPv6Address, AnyIPAddress, Port.
  • Currently implemented.
• IPv4AddressAndPrefix, IPv6AddressAndPrefix.
  • Currently implemented as CIDR<IPType>. I'll have to rework the API shape.
• IPv6Address.MulticastScope.
  • Currently unimplemented.
  • Would be a RawRepresentable value over the nibble value of a Multicast scope.
• UnixDomainSocketAddress.
• IPv4SocketAddress, IPv6SocketAddress, AnyIPSocketAddress.
  • Currently unimplemented.
  • IPv4SocketAddress will only contain the IP address and the port.
  • IPv6SocketAddress will additionally contain flowInfo and scopeID.
• AnySocketAddress.
  • Consists of IPv4SocketAddress, IPv6SocketAddress and UnixDomainSocketAddress.

Types that I'm undecided on:

• PrefixLength (for IPv[4/6]AddressAndPrefix).
• AnyIPAddressAndPrefix (IP-version-independent).

Note that nothing would be final even assuming everyone agrees on the API shape above. We'll still have to see how things work out in practice.