Allow remote database configuration

[isabelcosta]

Description

• SQLAlchemy Database URI built from environment variables to enable different types of databases to be used. Example, would be using a SQLite db versus a MySQL remote database on Amazon RDS
• The database URI is defined by the developer in using these environment variables, that will be used to build this database URI {db_type}://{db_user}:{db_password}@{db_endpoint}/{db_name}:

DB_TYPE=<database_type>
DB_USERNAME=<database_username>
DB_PASSWORD=<database_password>
DB_ENDPOINT=<database_endpoint>
DB_NAME=<database_name>

• Added a new configuration similar to DevelopmentConfig but meant to run only on a local configuration using a SQLite database.
• Fixed and added tests according to the new import of the SQLAlchemy Database URI

Followed example for Database URI for a MySQL database from this blog post: https://medium.com/@rodkey/deploying-a-flask-application-on-aws-a72daba6bb80
I also checked the documentation.

Continuation of previous PR #89 . (I couldn't reopen on GitHub because the base branch was deleted, and I wasn't allowed to change the base branch)

Fixes #87

Type of Change:

• Code

Code/Quality Assurance Only

• Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Ran pre-existing tests and new ones with python -m unittest discover tests

Switched configurations between local and dev:

• For dev environment I set up the above-mentioned variables, with values from a test database from PostgreSQL as a Service called ElephantSQL (I used free plan). Slack message mentioning this service
• For local environment, I didn't have to set those variables, since the app is setup to create a local sqlite database.

Checklist:

• My PR follows the style guidelines of this project
• I have performed a self-review of my own code or materials
• I have commented my code or provided relevant documentation, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• Update requirements.txt

Code/Quality Assurance Only

• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[isabelcosta]

Although the work in this PR works, this probably needs some cleanup, because it has code I did during GSoC 2018. Please review this when you can @m-murad @ramitsawhney27

Also, commits are unsquashed, because I'm planning to squash in the end. Just so I know the history of the commits while working on this.

[isabelcosta]

Ups, I think I'm missing adding the environment variables on the deploy.sh script @m-murad right? This should be done on this PR.

[m-murad]

Yes, it should be done in this PR.

[isabelcosta]

done the deploy.sh change @m-murad

[isabelcosta]

I got this error today when I tried to register a new user on the Postgres db instance I created a while ago. Although the changes were done, it took a while to see it reflected and I got the internal server error 500.

[2020-02-16 22:59:45,970] ERROR in app: Exception on /register [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
psycopg2.OperationalError: SSL SYSCALL error: EOF detected


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.7/site-packages/flask_restplus/api.py", line 319, in wrapper
    resp = resource(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask/views.py", line 88, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_restplus/resource.py", line 44, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/dockerBuild/app/api/resources/user.py", line 250, in post
    result = DAO.create_user(data)
  File "/dockerBuild/app/api/dao/user.py", line 44, in create_user
    existing_user = UserModel.find_by_username(data['username'])
  File "/dockerBuild/app/database/models/user.py", line 117, in find_by_username
    return cls.query.filter_by(username=username).first()
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2825, in first
    ret = list(self[0:1])
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2617, in __getitem__
    return list(res)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2925, in __iter__
    return self._execute_and_instances(context)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2948, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
    context)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) SSL SYSCALL error: EOF detected
 [SQL: 'SELECT users.id AS users_id, users.name AS users_name, users.username AS users_username, users.email AS users_email, users.password_hash AS users_password_hash, users.registration_date AS users_registration_date, users.terms_and_conditions_checked AS users_terms_and_conditions_checked, users.is_admin AS users_is_admin, users.is_email_verified AS users_is_email_verified, users.email_verification_date AS users_email_verification_date, users.current_mentorship_role AS users_current_mentorship_role, users.membership_status AS users_membership_status, users.bio AS users_bio, users.location AS users_location, users.occupation AS users_occupation, users.organization AS users_organization, users.slack_username AS users_slack_username, users.social_media_links AS users_social_media_links, users.skills AS users_skills, users.interests AS users_interests, users.resume_url AS users_resume_url, users.photo_url AS users_photo_url, users.need_mentoring AS users_need_mentoring, users.available_to_mentor AS users_available_to_mentor \nFROM users \nWHERE users.username = %(username_1)s \n LIMIT %(param_1)s'] [parameters: {'username_1': 'catarina', 'param_1': 1}] (Background on this error at: http://sqlalche.me/e/e3q8)

[isabelcosta]

@m-murad and I tested this in a community open session today, and we got only the error from email setup to send emails (which is related to issue #233 ).

Regarding the problem mentioned in the comment above #207 (comment) can be explained by the server being disconnected sometimes from the PostgresSQL instance being used for development purposes. This can be caused by the "quality" of the instance (e.g.: not having a lot of memory, being used by multiple applications).

Some potential explanations (after googling):

• DatabaseError: SSL SYSCALL error: EOF detected psycopg/psycopg2#533
• psycopg2.OperationalError SSL SYSCALL error: EOF detected paris-saclay-cds/ramp-board#151
• https://stackoverflow.com/questions/24130305/postgres-ssl-syscall-error-eof-detected-with-python-and-psycopg

We decided to still merge this because using this free instance for remote database (even if the connection between database and server fails sometimes) can bring more benefits then the hassle of having the database deleted every time a new merge is done.

cc @mayburgos