ILPv1 over ILPv4

ILPv1-over-ILPv4.md

ILPv1 over ILPv4

Motivation

ILPv4 makes some trade-offs compared to ILPv1. However, mixed in with these trade-offs are some - imho - pure improvements. This document describes what it would look like if I had to redesign ILPv1 given what we know now, but weren't willing to make some of the more controversial trade-offs. This work also revealed the surprising conclusion that ILPv1 transactional semantics (non-chunked or "universal mode quasi-atomic") are possible to achieve using ILPv4.

Differences between ILPv1 and ILPv4

There are two main differences that are relevant for this discussion:

• ILPv4 assumes that all parties in the chain of an ILP transaction understand ILP. ILPv1 is designed around "ledgers" which are systems that act like an ILP connector, but do not understand ILP.
• ILPv4 assumes that all payments are small. ILPv1 is designed to support larger payments as well, which requires support for additional features, such as liquidity curves to deal with different rates for differently sized payments.

Supporting ILPv1-style "ledgers" in ILPv4

As noted in the original ILPv4 ticket, it is possible to model an ILPv1 ledger as a connector whereby the sending party pre-funds with the connector and the connector post-funds with the receiving party. That way, both sides trust the connector.

We also have the concept of simulating a connector in the plugin for example in ilp-plugin-mini-accounts. Putting these ideas together, we can actually use an escrowing ledger like five-bells-ledger in ILPv4.

To use this ilp-plugin-bells-escrow, you would configure the hosting connector to settle with every payment. Then you would add the following behavior in the plugin. On the sending side, you would prepare on ILPv4-prepare, ignore sendMoney, return successfully on ledger fulfill and throw on ledger reject/cancel. On the receiving side you would emit data on ledger prepare and emit money on ledger fulfill.

Other escrowing ledgers such as blockchains could be used the same way.

Non-chunked payments in ILPv4

ILPv4 generally assumes that larger payments will be split into smaller chunks. However, this is only necessary when the maximum packet size is small. In order to get ILPv1 behavior, we simply need to configure connectors with a large packet size.

ILPv1 has various functionality related to handling different exchange rates for differently sized payments. This includes liquidity curves in the routing protocol and the quoting protocol ILQP. This functionality serves two important goals:

1. Performance improvements / quote caching
2. Quoting by destination amount

Performance

When we originally designed ILPv1, we anticipated that quoting would add significant overhead to the payment process. However, in practice, we found that even when doing a one hop payment over the fastest real-money ledger (XRP Ledger), the latency added due to quoting was negligible. As a result, I would say today that all of the caching functionality in the quoting and routing protocols was horribly overengineered and it is better to just quote ahead of every payment. The payment will take one or more orders of magnitude longer anyway and this removes a tremendous amount of complexity.

Quoting by destination amount

Quoting by destination amount is difficult without ILQP. It would be possible to create increasingly large prepared payments in an iterative process similar to chunked payments in ILPv4. However, we have to assume that preparing payments is slow and expensive in this context, so this would probably not work for a ILPv1 style network. (It is still an interesting option for doing non-chunked payments quoted by destination amount in ILPv4.)

Based on the considerations above, I would remove the quote-liquidity functionality from ILQP since it presents an unnecessary optimization and keep only the quote-by-source and quote-by-destination messages.

Assessing the viability

If implemented this way, I believe the resulting solution would be better than canonical ILPv1, but worse than canonical ILPv4. It retains many of ILPv4's architectural and protocol simplifications and contains the workarounds for legacy ledgers to the respective ledger plugins.

Writing this document lead to some interesting realizations for doing non-chunked payments in ILPv4. Iteratively preparing a payment in chunks (which all contain the same condition) is a novel idea which effectively provides a way for receivers to atomically receive a payment, yet allows the sender to "quote by destination amount" without needing ILQP.

It seems worthwhile further exploring the space between and around ILPv1 and ILPv4. That said, my strong recommendation is to focus our efforts on ILPv4 because it is overall the most promising approach and provides the greatest benefit to users: extremely fast and cheap payments, even when payment size is very small - i.e. the Internet of Value.

Our goal (which has not changed in my opinion) is to connect any sender with any receiver on any network. Our realization, in moving from v1 to v4 is that this is better achieved by overlaying a new interoperability layer over existing networks rather than trying to alter those existing networks to work with ILPv1.

In reality, what we have now is a true interoperability layer, whereas what we had before was architecturally broken (assuming the goal was an interoperability layer akin to IP). I honestly believe this and think it will help us move on with ILPv4 if we acknowledge it.

As you know, ILPv1 design had always sat badly with me, hence the many long debates about the right "layers" to carry the different ILP data elements. We made ILPv1 "look" like an interoperability layer but then made a lot of compromises when it came to the design. We couldn't adhere to proper encapsulation because it was impossible to implement a connector if we did. Data that we needed to inspect in both the sending and receiving ledger components had to be pulled out of the ILP packet where it should always have been.

The analogy with the IP stack actually slowed us down in this case because we wanted the ILP Packet to be carried in the ledger layer in the same way that an IP packet is carried by network protocols. But network protocols have very different properties to payment protocols, they have no request/response semantics for one so there is no need for state in the IP modules at each node, but critically they have very low latency. The delivery of an IP packet is only as fast as the networks it travels on, and in ILPv1 that was the same. For many use cases that was a problem. So to accommodate our request/response semantic and improve latency we needed to add a layer to the stack.

What is interesting is that we had already used ILPv4 when we used "dial-up ILP" to create virtual, conditional-payments aware, connections between connectors that had slow ledgers between them or ledgers that didn't support conditional payments. ILPv4 is just a realization that overlaying a new messaging layer on top of the ledger layer should be the norm, not the exception. In effect, ILPv4 is a network of connectors using only ILP-virtual and we have stopped trying to use other ledgers on the basis that they are so slow.

But can you use ILPv1 in an ILPv4 payment?

Yes (kind of). Two connectors absolutely can choose to use a ledger between them that supports conditional payments and only exchange ILP packets if the exchange is underwritten in that ledger using the same condition. But, that would be a bilateral decision with trade-offs.

Example:

Sender - [ILP Prepare] -> Connector A -[XRP Escrow Prepare]-> Connector B - [ILP Prepare] -> Receiver
Sender <- [ILP Prepare] - Connector A <-[XRP Escrow Fulfill]- Connector B <- [ILP Fulfill] - Receiver

where:

XRP Escrow Prepare is:

Connector A -[Create Escrow (Amount: X, Condition: Y, Timeout: Z, Recipient: Connector: B)]-> XRP Ledger
Connector A -[ILP Prepare (Amount: X, Condition: Y, Timeout: Z - t)]-> Connector B
Connector B -[Verify Escrow (Amount: X, Condition: Y, Timeout: Z, Recipient: Connector: B)] -> XRP Ledger
Connector B  also verifies t > safe limit to commit fulfillment to ledger

and XRP Escrow Fulfill is:

Connector A <-[ILP Fulfill (Fulfillment: Y,  data)]-> Connector B
XRP Ledger <-[Redeem Escrow (Fulfillment: Y)]-> Connector B

One of the trade-offs is that this link is much slower than simply passing the ILP Packet. For any sender that has low timeout requirements routes through Connector A and Connector B are simply not viable.

However, I think that this reduced trust requirement could allow for lower costs per payment, so in a use case where the amounts are larger and the sender is happy to use larger timeouts a route through Connector A and Connector B may be the best route.

Also, we shouldn't assume there won't be connectors whose business model requires them to hold trust-less accounts with their peers.

Some of the use cases we are exploring like micro-payments may not be viable over this route but others like Web Payments may be. Remember that for most retail payment networks the accepted user experience is payment that complete in the tens of seconds not milliseconds.

What I love about ILv4 is it doesn't preclude this scenario, it leaves it up to the emergent properties of the network to determine if this will be viable or not. We don't have to decide as the protocol designers, we've done the right thing by allowing options without compromising on interoperability.

I feel the same about payment size..

Is ILPv4 designed for small payments?

No. ILPv4 has absolutely no opinion on payment size. In my opinion, everyone who says it does has PSKv2 and their assumed usage of Ledger API v2 in the back of their minds. Happy to be proven wrong but that's my view.

Based on the use cases we are looking at, and our predictions around the emergent properties of the network, we have made assumptions about the maximum amount of liquidity that we believe connectors will make available per payment, and have optimized our transport layer design around those assumptions.

This has allowed us to design a very efficient transport protocol (PSKv2) that can discard the complexities of quoting along a route with possibly variable rates at different amounts. This was only possible after we accepted that quoting doesn't belong in the ILP protocol, it belongs in the transport protocol. (I still think there may be elements of ILQP that are useful for alternative transport protocols but they are not useful in PSKv2)

But, the PSKv2 design has trade-offs like uncertainty around exact delivery amounts (i.e. the ability to deliver an exact amount and transparency around the required send amount before making the payment).

I believe there is room for exploration around the use cases that can't live with that tradeoff, and the design of an alternative transport protocol that attempts to deliver exact amounts and provide the sender with an up-front quote before sending.

If we think our of the box there are a lot of ways of achieving this that don't just involve end-to-end quotes and forcing all connectors on a route to honor their quotes.

Example 1:

There are whole industries that exist to deal with certainty (and hedging). Perhaps the answer is not a new transport protocol but a way for an ILSP to offer the sender a guarantee that they will deliver the exact amount to the receiver and in the background they hedge against the risks of not being able to do that. In return the sender pays a small premium (included in the quote from the ILSP), but they get the desired user experience.

Example 2:

A third-party service can index the liquidity and rates at all of the connectors it knows and provide quoting as a service. Senders will ask this service how much they need to pay and to what initial connector. The SLA with the service will give the sender the certainty and transparency they need.

Zero-risk on a widely distributed system like Interledger will always come at a price. It's either in the form of "insurance" from a third-party, or time in the form of slow ledgers that offer atomicity for the executing of the payment. Even in ILPv1 there is not zero-risk, connectors still need to charge some spread to deal with the possibility they can't deliver a fulfillment and are left out of pocket.

TL:DR;

I believe we have a tendency to say things about ILP that are not factual but rather opinion based on the use cases we are focused on or the implementations we have built so far. There are no normative requirements in ILPv4 to send small amounts Nor is there a normative requirement to forward ILP Packets without a guarantee of payment from the peer that sent the packet.

This "guarantee" could be in the form of pre-payment from the sender or an, already prepared, conditional payment using the same condition and timeouts prepared in favor of the receiving connector (as described with XRP Ledger above).

ILPv1 is not an inter-networking protocol, despite trying to be one through use of the global ILP address space, but it does define some useful standards for conditional payments that, when used with ILPv4, can add additional guarantees at each hop where they are used and allow connectors to use a mutually trusted ledger instead of trusting each other directly.

If that is employed on all hops you have an ILPv1-like route using ILPv4.

Most importantly, I don't think we need to change ILP to solve the challenges that are not currently addressed in PSKv2.

Just as TCP has dominated the Internet as the preferred transport protocol for years, PSKv2 may dominate on the Interledger. But, the recent move toward new protocols built on UDP, suggests the same may happen with ILP and PSKv2 as the network looks for features that PSKv2 can't offer.

We have limited bandwidth and can't solve every use case. I agree with @justmoon that we should stay focused on ILPv4 as it is now and PSKv2 or similar transports with a view to doing something quite revolutionary.

That said, we should encourage exploration of other transport protocols to solve for different use cases and as I mentioned the other day I think there is a path to doing that requires minimal resources from our team.

Also, we shouldn't assume there won't be connectors whose business model requires them to hold trust-less accounts with their peers.

Can you clarify what you mean here?

• Does “trust-less” in this sense mean “secured”? As in an XRP payment channel?
• Or are you sayings the two connectors have no direct accounting relationships or agreements. Therefore the must settle through a trusted third party?

I initially read the sentence as the former but upon re-read the latter seems correct.

@adrian, I agree substantially with everything you said.

However, nothing written above really addresses why I think both the v1 and v4 models seem incomplete.
I’ll spend time writing up my thoughts today but the TL;DR is that in v4 we are still saying “connectors settle via trusted ledgers”.
I maintain strongly that characterizing some entities as Ledgers and others as Connectors was a faux pas (false step). Ledgerness is a facet of every entity as is connectorness. The concept of settlement is related to the topology of accounting relationships among three or more entities. It isn’t related to an intrinsic property of any specific entity.

In discussions of v4 I tend to hear statements like:

ILP standardizes the clearing relationships among connectors. ILP does not have to specify or standardize the settlement relationships between connectors. Connectors can work that out for themselves using the properties of any commonly shared ledger”

While I agree this is true, it still sounds incomplete to me when you dismiss any significance in the words connector and ledger.

If a bank (formerly a ledger) can implement the connector protocol, then two (formerly connectors) can settle by rippling a circular clearing payment through the bank. If Alice owes Connie, then a v4 payment routed (Connie -> Alice -> Bank -> Connie) could settle their account. The v4 payment brings Alice and Connie’s account balance back to zero, while synchronously moving that value to Connie’s bank account. There is no reason to presume that a bank implanting the v4 Connector protocol is going to be any slower than any random Joe implementing the same protocol.

Now, I’m not claiming that v4 as specified can’t do everything. That may very well turn out to be the case. The incompleteness I see is that we seem to be only discussing a subset of all possible payment transactions at this point. I’m also claiming that if it turns out that v4 can’t do something, then there exist a v5 that has only minor tweaks to v4 that can execute any type of payment identically.

ILPv4 is just a realization that overlaying a new messaging layer on top of the ledger layer should be the norm, not the exception.

Yes! That's the best summary I've heard!

---

You're right that a connector can act like a ledger, but to fully support the "trustless connectors, and rollback to the safety of the ledger" which at least that last version of ILPv1 provided, you would have to build this connector-that-acts-as-a-ledger on top of what I called ILPv4.1, otherwise the sender can't instruct the "ledger" which (next) "connector" they want to use.

I responded in that thread with an alternative that allows you to choose the next connector without any changes to ILPv4.

I don't think that that will be a common use case, but it is certainly nice that it is possible for a ledger + ledger plugin to offer this feature without any special support in ILP.

---

Another system ILPv4 misses is push distribution of exchange rate information. It was introduced in ilp-kit 3.0 and then removed again in ILPv2.

Push distribution of exchange rate information:

• Does not provide a meaningful benefit to overall payment latency

  I would refer to my point about performance from the doc above. Quoting is already so much faster than execution that optimizing it via things like push distribution of exchange rates provides no meaningful benefit.

  Picture the Seven Ledger Demo (~60s) and now picture it with 50% faster quoting (still ~60s). It's just not a meaningful difference in the context of ILPv1.

• Means that rates are less up-to-date than with quoting

  An exchange rate that is an hour old is necessarily less accurate than one that is only a few seconds old. That means I have to include more slippage in my payment. In ILPv1, the deliverable amount was visible, so slippage was lost as fees. (In ILPv4, connectors can still try to steal the slippage for themselves, but at least they don't see the amount under which the receiver will reject so it's impossible to steal all of it and stealing most of it risks a higher rate of failure.)

• Creates higher (possibly prohibitive) load on the routing layer

  Every node needs to tell every other node about new rates on a frequent interval, let's say once an hour. This is a substantial load for a routing protocol.

  In the BGP world, some people think that even a 24 hour update interval is too much:

  Given there are 800k routes in the default free zone, a timer of 24 hours (which would still make me uncomfortable in security terms), there would need to be 800k/24 hours updates per hour added to the load of every router in the Internet.

  I would echo this sentiment based on my own tests. Even with a tiny number of nodes (I tested 500 connectors on my laptop) it takes a couple of minutes to reach convergence. (The ilp-kit 3.0 version supports nowhere near 500 nodes.)

---

Iteratively preparing a payment in chunks (which all contain the same condition) is a novel idea

Isn't that basically the same as the binary search that PSK2 proposed to get an informational quote for a fixed destination amount?

Suppose we had 8-bit amounts.

Binary search would mean that I would quote a source of amount of b10000000. If the receiver rejects with "too high" I would try b01000000 next. If I get back "too low" I would try b11000000 next. And so on. This takes O(log n) time.

An immediate improvement would be interpolation search. In the case of a linear exchange rate and using linear interpolation, this improves the performance to O(log log n). The client could use polynomial interpolation to converge more quickly when the exchange rate is not linear.

What I was suggesting is an alternative for situations where you want a "quasi-atomic" payment, but don't need a prior informational quote. The way it would work is that the recipient would keep packets that undershoot the target amount prepared and respond using a zero-amount packet (or an out of band message) instead of a rejection. When a combination of packets that matches the target amount closely enough is prepared, the recipient fulfills all of them in one go.

What's neat is that this is parallelizable.

Suppose I'm the sender and my maximum source amount for this payment is 40 sender units. I'm trying to deliver a destination amount of 150 receiver units to the recipient. The fee policies between me and the receiver are non-linear.

I prepare a twenty unit (source amount) packet, a ten unit packet, a five unit packet, a three unit packet and two one unit packets. The receiver sees prepared packets with the amounts 180, 90, 45, 15, 5, and 5. Note that the exchange rate was not linear. The receiver fulfills the 90, 45 and 15 unit packets for a total of 150 units and rejects the rest.

The receiver can of course fulfill more than the destination amount, but that's not that different than them quoting a higher destination amount to begin with. Either way, they can't exceed the maximum source amount. Generally, in practice, we've seen two cases. 1) Where a human users reviews the quoted amount. In that case, to achieve atomicity we'd get an informational quote, so we would use interpolation search with unfulfillable packets and not this scheme. 2) Where a computer is making the purchase decision, e.g. ilp-curl. In that case, there is probably some maximum configured and otherwise it's up to the receiver to charge whatever they want. This protocol would work for this case. What stops the receiver from overcharging is repeat business. AWS can overcharge me for my S3 files anytime they like - they have my credit card. But if I notice they're charging more than their pricing page says I'll probably stop using them and they'll get shamed on Twitter.

And one more difference that also affects the game theoretical network dynamics, since ILPv4 uses forwarded payments, it requires test payments and/or more trust between nodes

ILPv1 also requires test payments because 1) the destination amount does not allow detection whether another connector is honoring its quotes (a malicious connector can selectively honor its quotes which is undetectable by its peers even when the destination amount is known) and 2) there are other reasons that ILP requires test payments, such as detecting high packet loss, high latency, high quoted rates, and so on.

Interesting proposals! Something like that could work, yes. Also reminds me of the bulk fulfill idea we talked about at some point.

I think short-term, we should just use ILPv4 for best-effort micropayments and not try to shoehorn quoting or escrow on top of it. So the Interledger payments travel on the messaging layer, and we just keep a vague concept of per-hop "underlying ledger" which underpins the trust relation at each hop, possibly with a payment channel layer inbetween.

Longer-term, Bob and Adrian made some interesting statements!