Computation seems to have two flavors, and we have been using "algorithm" as the catch-all word for all procedures capable of simulation on a Turing machine. The word "protocol" naturally showed up when we covered network computation during my academic CS training. We never used "protocol" in any other context.
I think this is a mistake
Let's reserve the word "algorithm" only for a certain specific flavor. Think of a Grandmaster chess player, looking at the entire board with all the pieces and positions. What they do depends on this knowledge and a cloud of knowledge around previous moves by them and the opponent. This is a great prototype for when to invoke the word "algorithm". In their naive first pass, algorithms resist parallelization. They depend on the computation of a previous step, and they have to be done step by step.
In Distributed Systems coursework, the motivation typically begins with the need for speed. We start to accept "good enough" solutions even for problems we know how to solve exactly, simply because the exact path is too slow.
When I first went through the curriculum, the subject wasn't framed as a lack of knowledge. It wasn’t a matter of not knowing how to solve a problem; it was the reality that we just needed it to happen faster. To achieve this, we "ballpark" the result by distributing the work across multiple computers. Since these nodes cannot know the global state, they are forced to work with only local state. They don't know what everyone else is doing, so they talk to one or two neighbors and otherwise mind their own business. The result is a high-quality approximation in under a picosecond, network lag notwithstanding.
Later, when we moved to NP-complete problems, the subject was still called "Approximation Algorithms." The word "protocol" was reserved strictly for network communication.
Reflecting on this now, it seems like one big ontological misunderstanding. We have always dealt with both global/sequential/exact computation and local/parallel/approximate compute. However, we never bothered to create a clear vocabulary for these different flavors.
To clarify: there are true approximation algorithms, which are mathematical methods used to find near-exact answers to NP-complete problems. Yet, in that same course, we frequently used "protocol-flavored" compute to achieve the same ends. There are cases where an algorithmic flavor simply won't work, and only a protocol will suffice. Conversely though, I cannot recall a case where a protocol flavor was inadequate.
My theory is that protocols have the luxury of defining what they call a "solution" and deciding when they are "done." Algorithms, on the other hand, can be trapped in time. They may not even know if or when they will halt, a classic challenge illustrated by the Halting Problem. These distinctions were never discussed in class, but in hindsight, a better name for that course would have been The Art of Approximation, Algorithms and Protocols. Altohugh most of these methods can go to arbitrary levels of decimals for precisions that it seems scandalous to call them approximation. But the spirit of the methods is definitely that of approximation.
Coming to AI, the backbone of it is widely understood as the back-probagation algorithm— what it is is basically mediocre differentiation in the lingo of this community. Each node only knows the values of a few connected nodes. When it wakes up it performs an action. It goes to sleep.
and BAM!! AI
Progress in AI has been a slow and methodical process of making protocols out of algorithms. In "Attention Is All You Need," the key contribution can be viewed this way: RNNs are like an army marching in a single line and time. You need to understand what happened before to understand what is happening now and next, which is the Markov process.
Transformers, however, traded off exactness. We no longer think of all history, but only inside the context window. Every word can calculate its attention score right away without having to wait for some previous score to be computed. Its context window can be packaged and sent off to its own GPU. So each compute is local, instead of being global or sequential as in an RNN. Massive compute is available, but massive time is not. So essentially a civilization-level breakthrough: by moving an algorithmic compute into a protocol type compute.
Looking ahead, we can think of AI progress the same way: what other algorithms are still left in the AI architecture and how can they be protocolized? For example, without doing a forward pass, you can't do a backward pass; at that level the compute is still sequential. Can we think of a way to do a good enough backward pass that doesn't have to wait for the forward pass?
Just spitballing here, some practitioners (grumpy algorithmists) would have spit their coffee at this point when I said backpass without the forward, But this is the sort of thinking that gets rewarded over and over again. It needs a certain level of epistemic courage to think good enough thoughts for long enough time.
(Some other collection of thoughts that came to me while writing the above...Protocols in Morality, Meaning making, Software, and Physical Building Architecture, etc.—all try to remove time. I use "time" and "state" interchangeably because that's how I think about them)
-
Bhagavad Gita: "What happened, happened well". This is not a dismissal of pain, but a way to clear the "cache of the soul" and make it stateless.
-
The Buddhist Protocol: Focusing on the breath, the immediate sensation, and the current act of compassion. Watch the "now" as it melts the "global state" of suffering and moves you toward freedom. It is computationally elegant. Present & Presence are the main motifs.
-
Functional Programming (My favorite style): This is essentially removing as much state as humanly possible from a program and gathering all that mess together in one file where it can torment us less.
-
Design principle of "Locality of Behavior" is closely aligned - can be found in programming and architecture. I am not an expert, but I am warming up to this in my most recent designs. Google will bring you to lot of good technical expositions on how to think about Lob.
-
Sabbath: Doing is to be replaced by being. Protocol inside a protocol.
-
Grace in Christianity: Sinfulness - you are the sum total of all your transgressions. Forgiveness is when you are allowed to wipe your logs clean, in an act of Grace. Protocol of confession to periodically clear statefulness/sinfulness.
-
Elizabeth Gilbert Ted talk about Showing up every day withouth paying attention to what happened previously or what will happen today is a personal discipline protocol of sorts. Lots of therapy and coaching protocols try to do this using an external loci. Meditation is doing it witout external locus - mostly using a plain protocol.
-
Tempo book by @Venkatesh Rao | UTC-8 explores protocols to make decisions without external time (algorithms) involved.
-
Quote from ZenArtMotorcycle: To live only for some future goal is shallow. It's the sides of the mountain which sustain life, not the top. Here's where things grow. Again, a state removal prototol. One more from the same book "Mountains should be climbed with as little effort as possible and without desire. The reality of your own nature should determine the speed. If you become restless, speed up. If you become winded, slow down. You climb the mountain in an equilibrium between restlessness and exhaustion" - best quote to read when you have a burnout. The next bullet explains why you burnout in the first place.
-
Coaching philosophy of Joe Hudson (famous SV coach): Focus on joy. If you are doing what you enjoy, you can get by only with your internal state guiding your life. Joy will help you find the way.
-
Pain is stateful. Joy is stateless. To be mindful is to be stateless.
-
"Doing time" is literally a punishment. Freedom is the absence of time from your mental experience, or being time itself. Tamil phrase to inform that someone just died: "they became time" on this day.
-
Suffering is the experience of Ego that looks at the big picture, keeps track of the various states of existence like material possessions and other artefacts of care of subjective experience. Ego death is when you realize that you don't/can't have the big picture and switch to a cog in the grand picture of creation type of existence.