[WIP]: Added MySQL as a datastore #165
Conversation
Flags relating to databases Signed-off-by: Computroniks <mnickson@sidingsmedia.com>
Signed-off-by: Matthew Nickson <mnickson@sidingsmedia.com>
Signed-off-by: Matthew Nickson <mnickson@sidingsmedia.com>
Previously the GetAllocatedIPs function used it's own database connection. It now uses a pointer with the store.IStore interface to access the DB. Signed-off-by: Matthew Nickson <mnickson@sidingsmedia.com>
The DB schema has been changed to use a flatfile design in order to reduce the complexity of queries and data processing. As no relational features are required there is no point in using a relational schema. Signed-off-by: Matthew Nickson <mnickson@sidingsmedia.com>
Computroniks
changed the title
|
For some reason the Diff looks a bit weird for |
The specific datastore backend to use can now be set by using command line options or by using environment variables. The default datastore backend is still jsondb but mysql can now also be used as a backend. Environment variables have also been added to control settings relevant to the database. SQL queries are made by directly accessing the database/sql API. TLS is also supported. Signed-off-by: Matthew Nickson <mnickson@sidingsmedia.com>
Documentation has been added detailing how to use the different database backends available. A minor gramatical error was also corrected. Signed-off-by: Matthew Nickson <mnickson@sidingsmedia.com>
|
Well, fixed that issue. Turns out it was a line ending issue. I had forgotten to switch main.go from CRLF to LF when I coppied the files over to a linux machine without using git. |
|
Perhaps using a package like this for the DB migrations: https://github.com/golang-migrate/migrate |
|
Converted back to draft as need to implement #164 and a way to manage database migrations. |
Computroniks
changed the title
|
Sorry, I don't really have time to work on this currently so I will close this to save cluttering the PR list |
Overview
This PR partially resolves #75 as it adds support for using MySQL (and MariaDB) as a datastore backend. This PR allows a user to connect to the remote database by specifying a number of environment variables. Documentation has also been added in
README.mddetailing the environment variables that need to be set in order to use specific databases.Techinal Details
Below is a detailed description of some of the descisions that I made when creating this patch.
Database schema
The database uses a flatfile design as apposed to making use of some of the relational features of MySQL in order to reduce the complexity of the code and reduce the number of queries that must be sent to the server. As JSONDB does not support relations in this sense there is no point in over complicating the schema as this would only lead to a poorer performance. The schema its self is stored in the templates directory and loaded with rice. If the database is empty then the schema will be created. Currently no way has been implemented to allow for updates to the schema but this is something that should be considered in future if changes are going to be made to the database schema, as seen with #164.
Direct use of database/sql
I decided to make direct use of the database/sql API due to the added complexity of using a package such as SQLX or GORM. These packages added no real advantage due to the structure of the data (ip addresses being stored as an array of strings). To make good use of these packages then the structure of most of the models would have to be changed, resulting in an awful lot of code to be modified. Instead, I chose to use the API directly and convert the arrays of strings to a string seperated by an
arrayDelimiter. This allows the addresses to be easily parsed and inserted into structs by using the strings package.Future support
I am more than happy to continue to support this feature in future in case of any issues or if any modifications need to be added. If so then just send me a message and I will have a look. I am also thinking about implementing perhaps a PostgreSQL backend to cover the main SQL servers depending upon what others think. I don't think that their is any point in using SQLite though as there is already the JSONDB and no relational features are used.