When querying the language model in ASL Signbank, Django expects a 'Global SB'-like table structure

[Woseseltops]

We don't really understand what's happening here, but we have a theory along these lines:

Normal situation

You define your table structure in Python as a Django model class. Django translates this to a database structure, stores this description somewhere in the database, and uses it to execute some SQL to make sure the Python class and the database actually match up. With the make-migrations command we can check if the Python class and the stored database structure description still match up.

Our alternative situation

The Language model is influenced by the model-translation plugin we use. Model translations adds some dynamic extra fields depends on what languages are in the database, so there no longer is a 1 to 1 relation between the Python class and the database structure.

What (we think) went wrong

For the ASL Signbank, Django complains it wants columns named nl en zh-hans, despite these languages not being defined anywhere in the ASL codebase. We think that this went wrong during a migration in the past, where we accidentally imported the Global Signbank database description in the ASL Signbank database.

Current solution

We manually made the ASL Signbank database match up with the weird expectations Django currently has. That is, we added the Dutch columns.

Better solution

Find where Django has stored the Global Signbank database description and correct this to the ASL situation.

(cc @vanlummelhuizen )

[vanlummelhuizen]

The migration that contains the related db update is https://github.com/Signbank/Global-signbank/blob/master/signbank/dictionary/migrations/0009_auto_20170802_1541.py

[Woseseltops]

Okay after looking through the ASL database for a looooooong time, I'm growing more and more confident that the theory above is wrong and the fact that the Language table wants a Dutch and Chinese column is itself not stored in the database somewhere.

But then, how can the ASL Signbank possibly know about the existence of Dutch and Chinese interface languages on another server? Two new theories:

Theory 1

The model-translation app looks at the locales in the repository:

(signbank) /var/www/signbank/repo/conf/locale$ ls
en_US  nl  zh_Hans

A counter argument to this theory is that I would also expect the need for an en_US column, which is not present in both Signbanks.

Theory 2

The whole complaint is unrelated to the workings of model-translation, and instead Django builds up the expected database schema by going through all migration files every single time a new migration is done.

If this is the case, all Signbanks need an identical database... in which case the current situation is already the desired end situation.

[Woseseltops]

Okay I've investigated some more by temporarily removing the Dutch and Chinese columns again, and I've discovered that my assumption that nothing would work without these columns is wrong. In fact, everything worked and all unit tests except the ones mentioned in #510 were green! I think the problem only pops when migrating, and then it kind of makes sense that Django goes over previous migrations; so theory 2, that the info is taken from migration files, is probably closest to the truth.

As for a solution, the matter is a lot less pressing now we know it will most likely only pop up during migrations. Still, it's weird that during this time we always need Dutch columns because that's in a migration file somewhere. A solution could be to 'reset' the Django migration history. Let's discuss next week @vanlummelhuizen .

PS
For later reference, this is the sqlite command to delete the two columns:

BEGIN TRANSACTION;
CREATE TABLE dictionary_language_backup AS SELECT id, name, name_en_US, 
language_code_2char, language_code_3char, description, name_en FROM dictionary_language;
DROP TABLE dictionary_language;
ALTER TABLE dictionary_language_backup RENAME TO dictionary_language;
COMMIT;

[Woseseltops]

Better idea by @vanlummelhuizen : make the hardcoded migration file https://github.com/Signbank/Global-signbank/blob/master/signbank/dictionary/migrations/0009_auto_20170802_1541.py dynamic .

[susanodd]

@Woseseltops is this still a problem?

I think this also applies to the empty signbank #1316

[vanlummelhuizen]

Better idea by @vanlummelhuizen : make the hardcoded migration file https://github.com/Signbank/Global-signbank/blob/master/signbank/dictionary/migrations/0009_auto_20170802_1541.py dynamic .

See also branch https://github.com/Signbank/Global-signbank/compare/make_installable. Perhaps I will look into it.

[vanlummelhuizen]

Ah, I did not see #1320 yet.