Distributed matrix inversion project for ECE 465 - Cloud Computing at Cooper Union, Spring 2014. Allows a client to specify a file of matrices to be processed by matrix inversion servers. Format for input files is specified below, and script to generate test data is included in this repo.
Features an application layer load balancer to determine the best server for a client to connect to for matrix inversion.
The major components of this project can be divided into three parts - the matrix client, the load balancer and the matrix inversion server. All of these are meant to run simultaneously and communicate using network sockets. As such, these can all be run on one machine, or distributed across multiple machines.
Additionally, some useful scripts for deployment and data generation can be found in the scripts directory, and are discussed further below.
In order to compile and run this project the general syntax is:
mvn clean compile exec:exec -P <profile name>
Where "profile name" is any of the following:
1. loadBalancer
2. matrixServer
3. matrixClient
In order for this project to function properly, the above profiles should be run in that order. That is, a histogram server requires a load balancer to properly function, and an image client requires a load balancer and a histogram server to function properly.
Hence, the load balancer should be run first. Then any matrix servers should be set up. At this point, image clients will be able to successfully connect and retrieve data.
Project properties (data source/output, port numbers, etc.) can be configured in the project.properties file located in src/main/resources. The properties required by this project are as follows:
On the load balancer, the client port is the port on which the balancer will listen for incoming connections from clients. This is also the connection on which it will respond to clients with a histogram server assignment. The host port is the port on which the load balancer listens for new histogram worker instances. The hostname is the ip address of url at which the load balancer can be reached by external clients or servers (localhost if this project is run locally).
On the matrix server, the client port is the port on which the server will listen for incoming client connections. If this project is run locally, this port must be different than the client port for the load balancer.
The matrix source/output files are relatively self-explanatory and should be specified relative to the base directory of this project.
Provided in the scripts folder is a python script titled 'generateData.py' which generates a file of matrices in the proper format for this project. The syntax to run this script is as follows:
./scripts/generateData.py
Further details are specified within the script comments.
This project was tested using data generated by the included script to demonstrate its performance potential and scalability.
The results of our testing are shown in the graphs below.
A fast deploy script is included with this project in the scripts directory. This features deployment to either Cooper Union ICE Lab servers (requires an ICE account) or AWS (requires an ssh token). Further details are described within the script comments.
- Christian Sherland
- Michael Scibor
- Ethan Lusterman