We've adopted a laissez-faire attitude about communication between plugins. But could the node stream protocol be our unix pipe equivalent?
See How We Find Data
Dominic Tarr recalls, In node.js’s early days the website featured an example of a tcp echo server. This is the birth of node’s most important api, the Stream. But things were just beginning to take form. post
When Tarr discovered streams and realized they would be useful, he wrote event-stream which applied streams to a use similar to Functional Reactive Programming. He convinced substack and Max Ogden how useful streams were, and then they convinced everyone else.
Streams had been created by the core node developers, but they were then adopted by people who contributed greatly to node’s ecosystem. These two groups had slightly different interpretations of what a stream was.
If node streams teach us anything, it’s that it’s very difficult to develop something as fundamental as streams inside a “core”. You can’t change core without breaking things because things assume core and never declare dependencies.
Part I of Tarr's story suggests we haven't done anything wrong by not incorporating streams into wiki's core yet. Maybe part II will hint when the abstraction will be done enough to do so.
The npm async module solves many problems we ignored or dealt with by hand. Caolan McMahon's occasional posts offer direct and related advice.
Node Style and Structure ready for async. post
Tiny Immutables ready for async. post
Stream all the things, with backpressure. post
Doug McIlroy envisioned software development by component assembly and had largely defined what became pipes before asking unix developers to consider adding components to their work.
Typewritten summary of requirements as of 1964. post
McIlroy's retrospective on early years of unix. pdf
Nonuniform Signal Resampled anticipates signal computation in wiki without being wiki.