Start Backend Test Dev

[mtreacy002]

Description

As a student,
I need create and populate data models classes,
so that I can move on to the next tasks specified in the First Coding Phase Week 1.

Mocks

N/A

Acceptance Criteria

Update [Required]

• must have enough pip dependencies (aka requirements.txt) at least to start the app
• must have remote server established as well as local server
• able to run project and see swagger UI
• must have .env.template
• must have migration scripts for initial migration
• must have .gitignore, .pylintrc, and .travis.yml
• must have functions if user opted to create and populate UserModel, MentorshipRelationModel, TasksListModel, TaskCommentModel, UserExtensionModel, PersonalBackgroundModel, OrganizationModel, ProgramModel, MentorshipRelationExtensionModel
• must have basic tests setup e.g. test_config.py and other simple tests

Definition of Done

• All of the required items are completed.
• Approval by all mentors.

Estimation

2 days

[mtreacy002]

Hi @anitab-org/bridgeintech-maintainers. I've created models for the BridgeInTech DB which consists of 2 schemas, bit schema and public (aka ms schema).
Here's the screenshot of the DB structure on pgadmin4

And here's the Swagger UI running on local server from running python run.py command (note that I haven't started any API routing here).

Here's the SQL statements for each tables:

MS Schema (public):

• users

• mentorship_relations

• tasks_list

• tasks_comments

bitschema

• users_extension

• personal_backgrounds

• organizations

• programs

• mentorship_relation_extension_model

Note: the _model at the end of this table just appear out of nowhere. In the code base, I use the exact same style as other tables as table name. Weird... 🤔. Maybe someone can lend me another pair of 👀 and spot why this happens?

I will commit and push the codes to my remote forked repo. But as this is only for a test, I probably won't open a PR until we finalise it 😉 . The branch is called test-schema.

Look forward to your feedback 😆

[mtreacy002]

PS: tomorrow I'm gonna work on migration script using Alembic and perhaps if time permits, will populate the tables with mock data. @ramitsawhney27, can I populate the data directly on pgadmin4 by running sql statements so not by placing the query inside the code (inside the code will get populated every time we're running the app)? This is coz I'm afraid the later will cause issue coz the data already exist.

Or, should I do it with test cases for db models? Coz this way the mock data will only get inserted while the test is running and will be removed at the end of each test.

Can you please advise?

[ramitsawhney27]

Dummy data insertion via pgadmin4 is good. We don't need tests as discussed in the weekly meeting.

[mtreacy002]

Notes to self:

The following changes need to be made to address datetime inconsistencies from mentorship-backend code base (discussed in mentorship-backend PR#556 and issue#608:

• app/database/models/user.py
  • register
    • current: time.time()
      recorded in: UTC+0
    • required: datetime.utcnow()
      recorded in: UTC+0

 direct impact: 
 * **app/schedulers/delete_unverified_users_cron_job.py**

 * **app/tests/cron_job_functions/test_delete_unverified_users_cron.py**

• app/api/dao/user.py
  • confirm_register
    • current: datetime.now()
      recorded in: local UTC (action user's UTC offset)
    • required: datetime.utcnow()
      to be recorded in: UTC+0

More to be added ....

[mtreacy002]

Update:
Hi @anitab-org/bridgeintech-maintainers. I've populated the MS+BIT tables with mock data using the code I just pushed to test-schema branch on my forked repo. I set it so the application will automatically populate the database when you run it.
You can test it by setting up your own postgresql and virtual environment locally (just like we set up development environment for Mentorship System). After you install dependencies (pip install -r requirements.txt) and run the app with python run.py command, the database will be auto-populated with dummy data. you can view them either with pgadmin4 or running query inside psql shell. Atm I'm working on a db_query_mock.py to auto-run few queries when running the application. Will let you know when I'm done.

Let me know if there's anything unclear on the setting up 😉

[mtreacy002]

Update:
Hi @anitab-org/bridgeintech-maintainers.
So, I've tried the following tests to see if BIT could send requests to MS API service.
I've placed the query in BIT codes (db_query_mock) to automatically send request POST /register when running application (as I haven't got BIT swagger UI set up properly for API endpoints)

I've done the tests with 3 different conditions:

• Test 1. Running both service locally (2 APIs + 2 DBs):

1. run MS API on local port 4000 with database sqlite local_data.db
2. run BIT API on local port 5000 with database postgresql bit_schema local database

• Test 2. MS API on Heroku, BIT on local port 5000 (2 APIs and 2 DBs)

1. connect to MS API on Heroku server (which means it has whatever database connected to Heroku as set in dev server - this would be AnitaB's elephantsql posgres MS, I pressume)
2. run BIT API as per above (running locally on port 5000 with postgresql bit_schema local database

• Test 3. Running both APIs locally with one target DB as what we want to achieve with this project (as last discussed on issue#10).

1. run MS API on local port 4000 with database target postgresql bit_schema
2. run BIT API on local port 5000 with database the same target database postgresql bit_schema

The results are as follows:

• Test 1 result (local MS API + local BIT API + 2 different DBs)

1. Successfully registered and new user data is recorded in MS sqlite local_data.db (not recorded in BIT postgresql bit_schema db)

• Test 2 result (heroku MS API + local BIT API + 2 different DBs)

1. Successfully registered and new user data is recorded in MS heroku db (not recorded in BIT postgresql bit_schema db)

as I don't have access to debugger console when connecting to heroku server, I took the below screenshot instead.
Right screenshot is successful login on MS heroku API whereas Left screenshot is successful login on MS local API (sqlite3 local_data.db).

• Test 3 result (MS local API + BIT local API with one target DB, postgresql bit_schema db.

This test 3 raised sqlalchemy error which I think caused by the non-existence BIT schemas inside the current code base of develop branch on MS (mentorship-backend repo). So regardless that the postgresql it's targeting as database already have BIT schemas in place.

@ramitsawhney27. Can you please advise how we should solve this issue? Thanks beforehand.

[ramitsawhney27]

You could discuss this with Isabel to see how this can be included in mentorship system.

[mtreacy002]

@isabelcosta, can you share your thoughts?

[mtreacy002]

Update:
Hi @anitab-org/bridgeintech-maintainers. I've just created and pushed the alembic script to support flask-migration to the test-schema branch.

We can also use it to do migration on mentorship-backend if we've decided to go ahead with incorporating bitschema in mentorship-backend code base.
Note: Since we have existing data in current db, extra care need to be taken so that the migration won't affect them (if we choose to keep these data). Not sure how (export import to csv?? 🤷 )

[mtreacy002]

This issue is no longer needed as the code change for it has been merged to PR#26 and mentor has reviewed and requested changes there so I (again) will close this