Overview

A Google search for "flow based programming" will turn up quite a few variations and implementations -- the simplest (which I've used a lot for rapid prototyping) is UNIX shell pipes. See J. Paul Morrison's work and Wikipedia for good background. An example embedded design (including a network configuration table) is at http://www.jmi.com/design.html. A Python framework that seems to have re-invented FBP independently is Kamaelia/Axon (too complicated already, but not yet production quality).

See FlowMessages for message format.

Ground Rules

Because there are so many ways FBP can be defined, we need to come up with a definition of what we mean by FBP in ISconf. Here's what we have so far -- we use the RFC 2119 definitions of SHOULD, MUST, MAY, etc.:

Tasks are the FBP processes, what Morrison calls Components, the things that modify, store, and analyze data. Because we have a microthread kernel available, we can spawn a great many tasks with very little overhead.

Messages are the FBP data, what Morrison calls Information Packets (IPs). We SHOULD pass by reference; within the same process, we can pass python objects around between tasks. We SHOULD use rpc822 message objects to carry messages; these are easily serialized so that we can pass them between processes as well.

Buses carry messages between tasks, like the traces on a circuit board. Tasks are like the integrated circuits and other components on the board.

Buses connect to tasks via one or more Pins; think electronic components again. Morrison calls these Ports -- we're changing the name to prevent confusion with TCP or UDP ports. A task MAY have zero or more input pins, and zero or more output pins, and any pin can serve as both input and output onto the same bus. Right now, a task MUST NOT have more than one read-pin connected to the same bus -- it's a limitation in kernel.event(), task.ready(), and the task event queue. Any bus MAY have multiple readers and multiple writers. In ISconf we implement buses in the isconf.kernel code -- right now it's kernel.event() et al, but this might change into a more FBP-specific function. XXX Add Bus class to Kernel, update this paragraph.

The netlist defines the connection of task pins to buses; Morrison calls this a network; again we change the name to prevent confusion. The netlist SHOULD be defined at a high level; at program startup, we should create all of the tasks and set up the netlist similar to the way it's done in http://www.jmi.com/design.html. Tasks MUST NOT know of each other -- tasks only know about messages and pins.

Buses are objects which MUST support a standard API; as we create each task, we pass it references to the buses it will use, like this:

def init():
    ...
    busA = Bus()
    busB = Bus()
    busC = Bus()
    ...
    kernel.spawn(taskfoo(pin1=busA, pin2=busB, pin3=busC))
    kernel.spawn(taskbar(pin1=busB, pin2=busA))
    ...

The tasks themselves don't refer to buses internally -- they only know their own pins, like this:

def taskfoo(pin1, pin2, pin3)
    while 1:
       msg = None
       # get a message
       yield pin1.rx(msg)
       # do processing
       ...
       # send a message
       kernel.tx(pin2,newmsg)

The API which a bus object supports MUST include these methods:

    # send a message to all readers currently connected to the bus
    numreaders = bus.tx(msg)
    # get the next unread message (blocking, with timeout)
    yield bus.rx(msg,timeout=None)

All data communication between tasks SHOULD take place via messages -- no global variables should be used. See, for example, http://c2.com/cgi/wiki?GlobalVariablesAreBad for guidance in case this rule needs to be modified. Also keep in mind that tasks MUST be able to be refactored to run in other processes, or even on other machines -- for privilege separation, re-juggling distributed algorithms, and so on. Global variables would impair that refactoring.

Nouns SHOULD be messages, verbs SHOULD be tasks. This makes sense when you think about it -- verbs "do something" to nouns. Compare this to conventional OO, where nouns are usually implemented as objects, and verbs are the methods on those objects.

XXX

This will all get a lot easier somewhere around Python2.5, when PEP 342 allows yield to return a value.

XXX

• • instead of one master rpc822 server, have each task:
    • • instantiate its own rpc server
      • register classes
      • listen on input alias(es) passed in at task creation
      • emit responses on output alias(es) specified at task creation
  • set up plugboard in init(), creating each task, telling it what alias(es) to listen on and which alias(es) to send output to
    • • ...eventually this can be controlled via a config file:

            [taskFoo]
            generator={generator_function}
            in1={alias}
            out1={alias}

• • never hardcode an alias name in a task
  • will likely need some shared storage space(s) for things like peers;
    • • pass references to these in messages
  • do not pass unknown messages through -- use exception outputs

implement multipart (e.g. rpc822 sigs) as nested message lists