Issue 828: Refactor schema to allow smaller indexes

[DiegoPino]

GitHub Issue: (Islandora/documentation#828)

What does this Pull Request do?

Refactors Gemini (Fedora/Drupal URI Mapper) to use hash based primary keys (composite) in its DB.
This is still Silex until we move to Symfony.
I also added a Symfony/Console app. using Doctrine's Migrate model to allow Table creation (and keeping track of changes) using console app

What's new?

New DB table schema includes 4 new fields, fedora_hash, drupal_hash, dateCreated and dateUpdated

• Does this change require documentation to be updated? No, all the public part was left as-IS
• Does this change add any new dependencies? yep, run composer install (or update if you are unsure about your PHP version)
• Does this change require any other modifications to be made to the repository
  Totally. Since Gemini is a deal breaker/repo-cms superglue, you need to move your already existing mapped values to this new schema or drop and start from scratch. Manual procedure is quite simple. Just update your schema, then add hashes (sha512) and timestamps (now() is good enough) to existing values
• Could this change impact execution of existing code? No

How should this be tested?

Pull or apply patch
run composer install inside Gemini
Make sure your working and existing config.yaml file is still there and it works
run php bin/console list to see available commands
run php bin/console migrations:migrate which will trigger a Migration file. (fancy name for MYSQL SQL!)

Migration file was written for MYSQL 5.7, will work on 5.6 too. But if other DBs are used, a new migration schema will have to be generated

Additional Notes:

This was made in a hurry today since I did most of the fun stuff on Symfony4! and is mostly a backport to keep the ball rolling. Backport means i could have forgotten something, please test and give me all the feedback and concern, change requests you want.

If this goes in I will make the ansible pull too, which is really just running php bin/console migrations:migratewhich by the way could be automatized to run when composer install runs via composer.json file.

Interested parties

Tag (@seth-shaw-unlv @ajs6f @DigitLib @dannylamb @Islandora-CLAW/committers)

[codecov]

Codecov Report

Merging #46 into master will decrease coverage by 4.08%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master      #46      +/-   ##
============================================
- Coverage     61.15%   57.07%   -4.09%     
+ Complexity      146      122      -24     
============================================
  Files             8        5       -3     
  Lines           726      636      -90     
============================================
- Hits            444      363      -81     
+ Misses          282      273       -9

Impacted Files	Coverage Δ	Complexity Δ
Gemini/src/UrlMinter/UrlMinter.php

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3167ad1...0ee2ce0. Read the comment docs.

[dannylamb]

Sweet! Thanks @DiegoPino. I'll pull it down and give it a try as soon as I can.

[ajs6f]

Without a working install, I'm not in a position to test right now, but this looks good! One question: Most dbs include md5 and sha1 functions. Would it be better to use stored procedures for hashing to hide the impl in the db? There's good and bad to doing that, of course.

[DiegoPino]

@ajs6f totally, also timestamp update. This was a quick fix to keep Silex alive. I can do that of course. And sha1 is faster too (but collisions dunno, you have huge collections!)

[ajs6f]

Yeah, timestamp is the same way, so to speak. But don't worry about SI's overly-ample collections! We will do locally what we need to do, let's get CLAW ready for the bulk of its audience.

(Just FYI, we are hugely concerned about using multiple Drupal apps over one repo API, we must be able to do that, so we will probably need to do some fancy footwork anyway!)

[DiegoPino]

Yes, you missed a step. i left the table generation as a Symfony console task( see the “how to test” description of the pull) but it is basically this `php bin/console migrations:migrate` I’m on my phone, away from a computer but that is what triggers the table creation Let me know how that goes. Composer auto invocation of that command is something I would like to do as a second pull, but before I would like to discuss default behavior in case the table is already there and has values

[seth-shaw-unlv]

Yeah, I realized that just after I posted. We can adjust the playbook once we ensure this works.

[seth-shaw-unlv]

Tested CentOS last night and Ubuntu w/ the content-model-overhaul this morning. Everything looks good to me.

[DiegoPino]

@seth-shaw-unlv++ thanks!! I really appreciate all the feedback and quick test.