Min Streams Interface Spec

simple-streams.md

Simple Streams Interface Spec

This spec describes a minimal stream interface meant for protocol implementors. Modules written to the interfaces in this spec can be used by a wide variety of project. The protocols will not need any extra dependencies themselves. It's just an interface to implement.

Simple Stream

A simple stream is just a function. It represents a pull-stream. It has the following signature:

// Implementing a stream.
function read(abort, callback) {
  // abort is a flag for signaling to the source that you're done with the stream.
  // if abort is truthy, and you're a filter, you usually want to pass it on the next time you call the source read.
  // if youe're a source, then stop reading and cleanup.  Call the callback with an end event when done.
    
  // To encode an item, have a falsy value for the first param and the item for the second.
  callback(null, item);
  
  // To encode the end of the stream, use the special `undefined` value for item.
  callback(null, undefined);
  // or simply
  callback();
  
  // If you wish to emit an error, pass a truthy value to the first parameter to the callback;
  callback(new Error("Oops, the monkey got out and ate all the banannas."));
});

// Consuming a stream
read(null, function (err, data) {
  // I got data!
});

The callback is allowed to be called before the read function returns (sync callback like in Array.prototype.forEach), but it won't always be sync because it might depend on some non-blocking I/O source. If you're doing any recursive loops over data, make sure to take this into account and not blow your stack.

Note: Dominic Tarr's pull-streams have a slightly different encoding for end events. Use an adapter for interop.

Filters

A protocol that translates between two stream formats is called a filter. The input and output events are a many-to-many relationship. Every input event may output zero or more output events.

Pull Filter

The easiest to understand filter is a pull filter. Modules implementing this spec export a function that accepts a read function and return a new read function.

// Implementing a pull filter
// input: strings
// output: uppercase string
function toupper(read) {
  return function (abort, callback) {
    // Forward abort through, we don't need to mess with it.
    read(abort, function (err, data) {
      // Forward errors and end of stream events throguh as well.
      if (data === undefined) {
        return callback(err);
      }
      callback(null, data.toUpperCase());
    });
  }
}

// Using the filter
var words = createWords(); // An imaginary stream source that emits strings
var uppercaseWords = toupper(words);
// Now consume the new stream instead and it will be the uppercase protocol.

An example of an echo server in working code https://github.com/creationix/moonslice-node/blob/master/testserver.js#L4

thanks!