Home

Welcome to the flowgraph wiki!

The version of github.com/vectaport/flowgraph that was published in 2015 has been migrated to github.com/vectaport/fgbase. This new version of the flowgraph package is a higher level interface model of a flowgraph system, with an fgbase implementation.

github.com/vectaport/flowgraph --> GoDoc GitHub Wiki

github.com/vectaport/fgbase --> GoDoc GitHub Wiki

The flowgraph package allows for the construction of flow-based systems written in Go in the style of an asynchronous hardware chip design. This can be used to model asynchronous chip designs for design and testing purposes, and allows for the same design model to be used in both hardware and software. It can also be used for constructing flow-based systems on any multi-core or many-core processor supported by the Go environment (and across the cloud as well).

Flowgraphs are built out of hubs interconnected by streams. The hubs are implemented with goroutines that use select to wait on incoming data or back-pressure handshakes. The data and handshakes travel on streams implemented with channels of empty interfaces for forward flow (interface{}) and channels of empty structs for back-pressure (struct{}).

The user of this package is completely isolated from the details of using goroutines, channels, and select, and only has to provide the empty interface functions that transform incoming data into outgoing data as needed for the application under construction. It includes the ability to log each data flow and transformation at the desired level of detail for debugging and monitoring purposes.

But most of all, what this package offers is a straight-forward way to explore designing systems in the asynchronous scalable programming model that is currently fueling innovation in Silicon Valley and beyond. This works stems from the same technology that led to Wave Computing and Eta Compute, two new chip companies in pursuit of greater efficiency through processor and voltage scalability for today's most computationally intensive applications.

In the beginning Ken Thompson gave us pipes between processes, later extended to sockets with select on his summer trip to UC Berkeley. These are all you need to build (or model) scalable asynchronous processes without a centralized bottleneck, but because of the lack of explicit support for back-pressure the resultant systems can often be compared to an urban freeway without metered on-ramps, i.e. no built-in way to prevent gridlock by managing the ingestion of new traffic to the system. Pretty much the same can be said for Go channels -- they have no built-in back pressure by design. This package offers a solution for that.

This package also offers the thrill of creating systems that seemingly run by themselves, with no centralized sequence or storage, ingesting data and producing results on demand, and iterating over it with explicit looping feedback. If you are like me and have a fundamental fascination in autonomous distributed processing (as found all over nature), I think you'll find this package rewarding to use.

Currently the examples of using the flowgraph package are embedded in flowgraph_test.go. Other examples of using these concepts can be found in the underlying fgbase package as well in the earlier C++ implementation of much the same at http://ipl.sf.net