UNFINISHED (But Working)
cluster is a tcl package which provides a simple light-weight framework for
providing inter-process and inter-machine discovery and communications.
cluster's evaluation hooks provide powerful extensibility, allowing the programmer to modify
the general behavior of a cluster to fit their security and/or communications handling needs.
Package Dependencies
- TclUDP Extension (v1.0.11 or higher)
Supported Platforms
- Linux / Unix
- Recommended tuapi
- OS X Initial Support
It is actually relatively easy to support other platforms or not require tuapi. Just need to modify a few of the initial procs to give data such as LAN IP's and MAC Address.
Note: All of the platform-specific functionality is implemented in utils/platform.tcl. It should not be difficult to add support for other platforms.
cluster aims to provide a lightweight modular framework for cluster discovery, communications,
and coordination. cluster members automatically discover each other and provide their preferred
protocols for communication.
In a local environment, it's as simple as having both run [cluster join] and awaiting
discovery. We can then pass messages and commands between our cluster members.
By default, our beacons will be set with a UDP TTL of 0. This means that our beacons
will only be heard locally on the system. We may set the ttl value used by our beacons
using the -remote configuration parameter. Anything higher than 0 will begin to open
our beacons further out from the machine.
When -remote is set to 0, communications will be ignored from any communications originating
from outside of our localhost. You may also use hooks to provide this type of security based on
the context of the cluster.
In order to make the cluster communication as fast and light-weight as possible, a special binary wire protocol is utilized (bpacket) which resembles Protocol Buffers which a few modifications. In general this should be anywhere from 30-60% smaller than using JSON or similar (depending on the type of packet being sent).
cluster provides hooks that allow you each member to intercept evaluation at different
parts of the communications process. This allows you to add security, new features, and/or
run tasks whenever needed. We do not automatically execute any code within your interp, it is
up to you to add such functionality if needed (examples below).
Each member in the cluster advertises what protocols it knows how to use as well as the priority of those protocols. Other members use this to establish direct channels of communication when needed. Should a protocol fail for any reason, the next will be attempted (and so on).
It is extremely easy to provide new protocols that cluster can utilize. Simply follow the general template provided by the included protocols. Out of the box we support UDP, TCP, and Unix Sockets.
Part of the cluster protocol provides capabilities for services to assist each other in
self-healing when something goes wrong with one of a services preferred protocol handlers
without its knowledge.
Below we see a simple example of using cluster where we simply join the cluster
and register a hook to inform us whenever a new service has been discovered.
Running this on two different shells (on the same system for now), you should
see the two shells discovered each other. In each example, the $service variable
is a reference to the service object which can be used to communicate with or get
further context about the service.
package require cluster
set cluster [cluster join -tags [list service_one]]
$cluster hook service discovered {
puts "New Service Discovered: $service"
}
# Enter the event loop and allow cluster to work
vwait _forever_
package require cluster
set cluster [cluster join -tags [list service_two]]
$cluster hook service discovered {
puts "New Service Discovered: $service"
}
# Enter the event loop and allow cluster to work
vwait _forever_Channels provide an additional layer of extensibility to your cluster communications. Members of the cluster will ignore any data they receive that are on channels they have not subscribed to.
When combined with hooks this provides us with the capability to add some simple add-ons to how our cluster will work.
For example, say we wanted to encrypt some of the data that was being transmitted, but only on a specific channel. This way only members which know the encryption key would attempt to read it.
All we would need to do is add a hook for the channel we want which would encrypt before sending and decrypt before reading. The "raw" key of a payload tells us to save raw bytes rather than utf-8 encoded strings.
$cluster hook channel 5 send {
if { [dict exists $payload data] } {
dict set payload raw [encrypt [dict get $payload data]]
dict unset payload data
}
}
$cluster hook channel 5 receive {
if {[dict exists $data raw]} {
dict set payload data [decrypt [dict get $payload raw]]
dict unset payload raw
}
}Queries allow us to send a command to the cluster and collect the results. They provide a special object that lives for a short period to coordinate the responses automatically for you.
package require cluster
set cluster [cluster join -tags [list service_one]]
set var "Hello,"
$cluster hook query {
if { ! [my local] } { throw error "Only Local can Query" }
uplevel #0 [list try $data]
}
$cluster hook service discovered {
puts "New Service Discovered: $service"
}
# Enter the event loop and allow cluster to work
vwait _forever_
package require cluster
set cluster [cluster join -tags [list service_two]]
set var "World!"
$cluster hook query {
if { ! [my local] } { throw error "Only Local can Query" }
uplevel #0 [list try $data]
}
$cluster hook service discovered {
puts "New Service Discovered: $service"
}
# Enter the event loop and allow cluster to work
vwait _forever_
package require cluster
set cluster [cluster join -tags [list service_three]]
# Our QueryResponse will be called with events that occur during
# the queries lifecycle.
proc QueryResponse { event } {
lassign $event action query
switch -- $action {
response {
# A Service has provided a response. Return the result so
# we can collect the results when completed.
return [$query result]
}
done {
# Our query is complete - all services have responded.
set results [$query results]
puts "Query Completed! Results Are:"
# Hello, World!
puts [dict values $results]
}
timeout {
# Our query has timed out
}
}
}
proc RunQuery {} {
# Query all the services on the localhost, collect the results, run QueryResponse
# for events, return the value of ::var
$::cluster query -collect -resolve localhost -command QueryResponse -query { set ::var }
}
# Give a few seconds for all the services to join, then run the query
after 5000 RunQuery
# Enter the event loop and allow cluster to work
vwait _forever_
The top-level API is used to join a given cluster.
All of the configuration options are optional. It is valid to simply call cluster join to utilize the default values.
| Argument Name | Type | Required | Default | Description |
|---|---|---|---|---|
| -address | IP | No | 230.230.230.230 | The Broadcast IP Address to use for the cluster |
| -port | Port | No | 23000 | The Broadcast Port to use for the cluster |
| -ttl | Seconds | No | 600 | How many seconds should a service live if unseen? |
| -heartbeat | MS | No | 120000 | At what interval should we send heartbeats to the cluster? |
| -channels | List | No | 0 1 2 | A list of communication channels that we should join |
| -remote | Integer | No | 0 | Should we listen outside of localhost? TTL. |
| -tags | List | No | A list of tags that we want to broadcast to the cluster. | |
| -protocols | List | No | t c | What protocols do we want to support for this member? |
| -$proto_id | Dict | Yes* | This is only required when providing custom protocols. |
More Information on each argument coming soon...
Once we have joined the cluster, we will use the reference that was returned to coordinate communication and requests.
Send a heartbeat to the cluster, informing them of your presence on the cluster. This is handled automatically but can be sent manually if desired. We have a few optional arguments that may be included.
More Information Coming Soon...
Reschedule the heartbeat so that it will occur after the given time. This is useful for when we want to cause a heartbeat to occur sooner than the normal time but dont want to accidentally dispatch tons of heartbeats when multiple services call it, etc.
Send a discovery probe to the cluster. This requests that all members of the cluster report to you. Services may report back at randomly delayed intervals. For the most part this should never really be required as it is handled internally.
This command allows you to broadcast a command to all members of the cluster. It expects a single argument (payload) which should be a properly formatted payload dict.
When we want to send data to our cluster, the [cluster send] command allows us to do so. It includes
various optional arguments that assist us with defining which services should receive the payload. Cluster
will determine from the final list of services what the best method will be to efficiently handle the dispatch.
Note: If multiple arguments are provided which resolve to services, the final list will be the merged result of unique services.
| Argument | Type | Description |
|---|---|---|
-services |
List | A list of services to send the message to. |
-resolver |
resolver | See [$cluster resolver] for help on available arguments |
-resolve |
resolve | See [$cluster resolve] for help on available arguments |
-broadcast |
Boolean | A Boolean as-to whether a UDP Multi-Cast should be used |
-protocols |
List | A list of protocols that should be attempted |
-channel |
Number | A channel to send the payload to |
-ruid |
String | A "Request Unique ID" to send with the payload |
-data |
String | Any data that should be sent with the payload |
Tip: If using -broadcast but also resolving services, the broadcast will include a filter so that only the given services will actually read the given payload. In general this type of logic is automatically handled for you.
More Information on each argument coming soon...
- -id
- -collect
- -resolve
- -command
- -query
- -timeout
This allows us to "search" for matching services which meet a specific criteria. This is used to aid in sending queries and events to the cluster.
# Resolve services by running a search against each $arg to return the
# filtered services which match every arg. Resolution is a simple "tag-based"
# search which matches against a services given tags.
# set services [$cluster resolve localhost my_service]More Information Coming Soon...
A more advanced version of resolve which allows us to add additional logic to the resolution process.
# resolver is a more powerful option than resolve which allows adding of some added logic.
# Each argument will define which services we want to match against.
#
# Additionally, we can specify boolean-type modifiers which will change the behavior. These
# are applied IN ORDER so for example if we run -match after -has then has will not use match
# but any queries after will.
#
# Modifiers:
# -equal (default)
# Items will use equality to test for success
# -match
# Items will use string match to test for success
# -regexp
# Items will use regexp to test for success on each item
# -has [list]
# The service must match all items in the list
# -not [list]
# The service must NOT match any of the items given
# -exact [list]
# The service must have every item in the list and no others
# -some [list]
# The service must match at least one item in the list
#
# Examples:
set services [ $cluster resolver -match -has [list *wait] -equal -some [list one two three] ]
More Information Coming Soon...
More Information Coming Soon...
When sent without arguments, this will respond with the current tags that have been sent to the members of the cluster. Otherwise we can use this command to add, remove, or replace the tags that we wish to associate ourselves with.
When tags are modified, they will be included on the next heartbeat which is sent to the cluster. Other than that, they are not included unless requested through discovery or direct queries.
Think of tags as a way of determining which members of the cluster are in various states and/or provide various services.
The command itself provides a syntax for modifying the current tags where each argument is taken and handled in the order given. Modifiers allow us to conduct various operations on the tags so that we can easily manipulate them if needed.
Note: tags are what we are generally using while resolving services during send and query operations.
| Modifier | Description |
|---|---|
-map |
Similar to [string map], this will replace a given tag with another if it exists. |
-mappend |
Similar to -map except that it will add the second tag even if the first does not exist. It will also only add the new tag if it does not already exist. |
-remove |
Removes the given tags if they exist. |
-replace |
Replaces the entire list of tags with the given tags then switches back to append for any further arguments given. |
-append (Default) |
(-lappend is synonymous), adds the given tag to the list of tags. |
$cluster tags tag0 tag1
# tag0 tag1
$cluster tags -append tag0 tag2 -map [list tag2 tag3] -remove tag3 -append tag4 tag5
# tag0 tag1 tag4 tag5
# In this example the -map does nothing since tag10 does not exist in our list of tags.
$cluster tags -map [list tag10 tag12]
# tag0 tag1 tag4 tag5
$cluster tags -mappend [list tag0 tag5] [list tag6 tag8] [list tag3 tag8]
# tag1 tag4 tag5 tag8
Tip: You can think of
[-mappend]as a shortcut for[$cluster tags -remove tag1 -append tag2]
Returns a list of references to each service that we currently know about.
Returns the number of services we currently know about.
Returns the current configuration object that our cluster and services utilize to coordinate their lifecycles.
Returns the current time-to-live value that is used. This determines how long we will allow a discovered service to remain in memory before removing it (if we have not received a heartbeat or communication from it).
There are times this value is reduced. When we attempt to open a channel with a service and it fails for any reason, we will dispatch a "ping" request. This will tell our cluster that we have failed to communicate with a service. All services will then expect a heartbeat from the given service or they will remove that service on their next evaluation (heartbeat).
More Information Coming Soon...
More Information Coming Soon...
More Information Coming Soon...
More Information Coming Soon...
More Information Coming Soon...
More Information Coming Soon...
More Information Coming Soon...
More Information Coming Soon...
More Information Coming Soon...
More Information Coming Soon...
More Information Coming Soon...
More Information Coming Soon...
More Information Coming Soon...
More Information Coming Soon...
More Information Coming Soon...
Hooks provide a means to customize the evaluation environment with short snippets of mutations and additions to the regular lifecycle of the package. Hooks are always evaluated within the hooks given context.
We can modify the behavior at each level by either changing values before they are sent or received, or throwing an error to cancel any further handling.