DDO Schema handling

[alexcos20]

Once we have validateDDO command:

• when adding new ddo (Metadata created), identify the right collection and store the ddo (indexer)
• when query, make sure all collections are used

[alexcos20]

from Slack:

Jamie:
[@alexcos20 ]
You mentioned previously that the schemas folder will end up containing gaia-x schemas and others. I'm wondering how we are supposed to know which schema a DDO is using? In the DDO we use the version number version: '4.1.0' rather than the schema name. So how do we know if this is supposed to use the op_ddo_v4.1.0 schema or gaiax_ddo_v4.1.0 schema?

Good issue :)
I would say for now, let's keep it as it is, as there are no known conflicts..
If in the future will detect some, we will create an additional json file in the folder, that looks like:

{
    "did:op":  {   
              "versions": [
                  { "4.1.0" :        //this is version for DDO + match did starts with did:op
                       { 
                           "validation_file":"op_4.1.0.schacl",
                           "mapping_file":"op_4.1.0.json",
                           "collection_name":"op_4.1.0"
                       }
                  },
                  { "5.1.0" :        //this is version for DDO + match did starts with did:op
                       { 
                           "validation_file":"op_5.1.0.schacl",
                           "mapping_file":"op_5.1.0.json",
                           "collection_name":"op_5.1.0"
                       }
                  }
              ]
     },
     "did:gaia-x":  {   
              "versions": [
                  { "1.0" :        //this is version for DDO + match did starts with did:gaia-x
                       { 
                           "validation_file":"gaia_x_1.0.schacl",
                           "mapping_file":"gaia_x_1.0.json",
                           "collection_name":"gaia_x_1.0"
                       }
                  },
                  { "2.0" :        //this is version for DDO + match did starts with did:gaia-x
                       { 
                           "validation_file":"gaia_x_2.0.schacl",
                           "mapping_file":"gaia_x_2.0.json",
                           "collection_name":"gaia_x_2.0"
                       }
                  },
             ]
     }
}

This will take care of all conflicts, and it's not changing our logic (now we are browsing the folder to take the files, instead we will have it predefined)

[jamiehewitt15]

We also need to ensure that a did does not exist in more than one collection, so if a ddo is updated to use a new schema it should be removed from all other schemas.

[alexcos20]

so the flow is the following:

• metaDataCreated/Update
  • check version. If we don't have a version in the DDO, discard it (we cannot validate it)
  • if valid, put it in corresponding collection, otherwise log error and discard it
  • when updated, and new_version != current_version, delete it from old collection and put updated version in new collection
• metaDataState:
  • if changes from active to disabled: delete it from old collection and put short version in "short_collection" collection
  • if changes from disabled to active: delete it from "short_collection" and put it in corresponding collection according to version

When doing query:
use all "version" collections + "short_collection"

[jamiehewitt15]

Merged in: #239