feat: Translation Api

[tomasciccola]

this should close #466

[tomasciccola]

I can't remember whats the rationale for omitting the actual translated value (called message) when hashing the doc. For now I'm using the whole doc for hashing, but there was probably a good reason for it

[gmaclennan]

I can't remember whats the rationale for omitting the actual translated value (called message) when hashing the doc. For now I'm using the whole doc for hashing, but there was probably a good reason for it

Yes the reason is so that when you edit the translated value, the docId does not change, and so that you can know the docId without knowing the translated value (when reading a translated value, when you have the other fields)

[tomasciccola]

I'm wondering how are we actually using the cache (when calling translationApi.index(doc). If, lets say, I translate a field to spanish (by calling translationApi.put(translation), the index would look like

Map(1) { 'es' => Set(1) { 'field' } }

Which signals that basically we have fields translated to spanish. I would think that we would use this Map on subsequent calls to translationApi.get(), but I don't actually know how we would use that?? (I would imagine this would remove some filters from the sql query, but I don't actually know...)

[tomasciccola]

Now that I think about it, shouldn't the #translatedLanguageCodeToSchemaNames, should be smth like

Map<languageCode,Map<schemaName,Map<fieldRef,message>>>

I know that it looks convoluted, but that way we're actually storing the translated values in memory that we can retrieve

[gmaclennan]

Now that I think about it, shouldn't the #translatedLanguageCodeToSchemaNames, should be smth like

Map<languageCode,Map<schemaName,Map<fieldRef,message>>>

I know that it looks convoluted, but that way we're actually storing the translated values in memory that we can retrieve

In translationApi.get():

get(opts) {
  const docTypeIsTranslatedToLanguage = this.#translatedLanguageCodeToSchemaNames.get(opts.languageCode)?.has(opts.schemaNameRef)
  if (! docTypeIsTranslatedToLanguage) return []

The vast majority of records in the database will not have any translation. This provides a shortcut to avoid an unnecessary database query, with a fairly small memory footprint (the size of this map is limited to the number of languages the user has translated for, by the number of schema types they have translated, so fairly small).

[tomasciccola]

owww ok yeah, that makes sense; its not to cache records, is to avoid queries for things we don't have translation for. I kinda knew that after our sync, but I guess I just forgot

[tomasciccola]

Shouldn't the index method be called every time we do a put ? So, in that sense, it should basically be a private method of the api - and internally run every time we call put - ?
Is there a case where we want to put a translation but not index it??

[tomasciccola]

I took the liberty of setting index as a private method and running it when calling put. There's smth that I may be missing, so that change could be reverted.
I'll probably create e2e tests for this, so that we can test creating translations for real records in a project (probably using the configs that we use for testing), but in terms of unit tests I dunno if there's stuff missing (there may be...)

[gmaclennan]

You're missing that there is more than one user/writer in a project 🙂

[tomasciccola]

You're missing that there is more than one user/writer in a project 🙂

HEh! That's what I was missing. I'll revert that and update the tests

[EvanHahn]

Suggested change

	if (doc) {
	await this.#dataType.update(doc.versionId, value)
	}
	await this.#dataType.update(doc.versionId, value)

[EvanHahn]

These lines have several issues, which (I think) I fixed in #590.

[EvanHahn]

LGTM. Thanks for addressing all of my comments!