[question]usage with meteor 1.3

[dagatsoin]

Hi,
I just found your package while looking around to have a CouchDB like behavior with sync offline/online and persistant data. I will use Cordova and Meteor 1.3 for my project.

Do you know any issues to use the alpha version of GroundDB with 1.3?

[ecerroni]

I was wondering about the same. Looking forward to get an answer here.

[dagatsoin]

Looking to the alpha 1.0 branch, there is no update since August '15? Is this project frozen?

[dagatsoin]

I simply tried:

var list = new Meteor.Collection(null);
var groundList = new Ground.Collection(list, 'list');

I got:
Ground.Collection requires a collection name [missing-name]

So I think it is not compatible yet.

[ecerroni]

Try
var list = new Mongo.Collection(null);

[dagatsoin]

Yep, it is what I did, I pasted the wrong part.

Plus, the package removes standard-minimifiers.js and as the last commit on the alpha branch was in November I don't think the library is up to date.

So for now, it is not usable in 1.3.

[ecerroni]

Too bad. It's a package I used for a mobile app with great success.

It's perfect for offline mobile apps. I hope the library will eventually be updated for Meteor 1.3

[dagatsoin]

I hope so much too! And I will attempt a nasty ping to @raix to know his plans about this :)

[raix]

Hi guys - working on it :)

[dagatsoin]

<3

[ecerroni]

Great! :)

[veeramarni]

Is the issue fixed for Meteor 1.3 version?

[dagatsoin]

Hi @raix do you need some help for the next grounddb version? I don't know if I have the level for collaborating on this but if you have got clear task todo I can look at it.

[raix]

@dagatsoin thanks sorry about the delay - solving this issue just got in my sprint - I'll try to outline things in issues/milestones so others can contribute and help out.

[dagatsoin]

Ok, count me in.

[raix]

Hey guys,

cc.: @dagatsoin

Take a look at 2.0.0-alpha.2 branch https://github.com/GroundMeteor/db/tree/grounddb-caching-2016 - review / feedback is welcome

I've decoupled the new ground db to be a pure caching db (at least for now) - The major reason is that it's more important to have better control of the cached data. Second reason is than you might want to swap out the source at some point.

Migration

Migration would probably look something like:

  // current "connection-driver"
  var foo = Ground.Collection('bar', {
    cleanupLocalData: false, // Manually trigger cleanup
  });

  Meteor.subscribe('bar', () => {
    foo.removeLocalOnly(); // Match grounded data with subscription
  });

  // current "ground db II"
  const bar = new Mongo.Collection('bar');
  const foo = new Ground.Collection('foo');
  foo.observeSource(bar.find()); // Cache bar or a subset of bar

  Meteor.subscribe('bar', () => {
    foo.keep(bar.find()); // Match grounded data with subscription or a subset
  });

  // You might also want to point bar finds to foo - this is a custom case depending
  // on implementation
  bar.find = function(...args) {
    return foo.find(...args);
  };
  bar.findOne = function(...args) {
    return foo.findOne(...args);
  };

Again the GroundDB II does not cache/resume outstanding method calls or try to be clever about things - but it supports localforage (localstorage/indexeddb/websql and SQLite on cordova) so it's non blocking.

Comparison with approximate 2.000 documents:

Current ground db: (Notice the white screen at first while ground db is blocking)

Ground db v2:

[mrlowe]

Hi @raix, I'm just showing up to the Meteor+GroundDB party and seeing this note about your next version, specifically the bit about no longer resuming outstanding methods. It's a little confusing to me, though. Suppose I want to insert an object into a collection. Can that be done securely without using a server method?

[dagatsoin]

@raix I am ready to dive in groundb. Question: how to manage storage system? I use Cordova and will surely use the SQLite plugin. So how to make sure that grounddb will use SQLite?

[dagatsoin]

And which SQLite plugin must we use? https://github.com/litehelpers/Cordova-sqlite-storage or https://github.com/nolanlawson/cordova-plugin-sqlite-2?

[atulmy]

@raix I'm building an app for a clinic and would require data offline. Its on Meteor 1.3. Which version should I go ahead with? Ground DB 1 or if 2 alpha is usable on production as well? Any help or alternatives would be useful!

[raix]

@dagatsoin the 2.0.0-rc.3 uses localforage - so you can install the SQLite localforage plugin and configure a localforage storage manually:

import localforage from 'localforage';
import Ground from 'meteor/ground:db';

const fooStorageAdapter = localforage.createInstance({
  driver: window.cordovaSQLiteDriver._driver, // Not tested - depends on the plugin
  name: 'foo',
  version: 1.0,
});
const foo = new Ground.Collection('foo', {
  storageAdapter: fooStorageAdapter,
});

foo.observeSource(bar.find());

@atulmy on Meteor 1.3 I would use 2.0.0-rc.3 - the old version might break

[dagatsoin]

hi @raix I don't get how to install the 2.0.0-rc.3. Currently I have downloaded the last commit in /packages/grounddb then I did meteor add ground:db@2.0.0-rc.3

The terminal displayed:

Changes to your project's package version selections:

dispatch:kernel                   added, version 0.0.6
dispatch:request-animation-frame  added, version 0.0.1
ground:db                         added, version 2.0.0-rc.3
ground:servertime                 added, version 2.0.0
raix:eventemitter                 added, version 0.1.3
raix:eventstate                   added, version 0.0.4

So I guess it is OK. But When I try to import it with import Ground from 'meteor/ground:db'; Webstorm does not find the path.

[dagatsoin]

and import localforage from 'localforage'; is also marked in red.
import localforage from 'localforage-cordovasqlitedriver/src';is not marked in red but seams weird to me.

Could be useful to know the installation steps.

Thx :)

EDIT: note I am using Typescript. I will look on the type definition file side.

[raix]

oh - there might be issues with the typescript/meteor import syntax - not sure if it's actually working.

the latest ground db is not using SQLite - it should add the npm version of localstorage however. ref

[dagatsoin]

You mean it does not use SQLite "directly"?
But it is faisable if I use localforage-cordovasqlitedriver?

[dagatsoin]

What is the best approach to abstract Mongo.collection and Ground.collection?
For exemple if I want to use the same entities collection in offline or online case. Do I need to handle offlineEntities.find() and onlineEntities.find() depending if the user is offline or not?

EDIT:
ok I saw your example above with

// You might also want to point bar finds to foo - this is a custom case depending
  // on implementation
  bar.find = function(...args) {
    return foo.find(...args);
  };
  bar.findOne = function(...args) {
    return foo.findOne(...args);
  };

[ecerroni]

Yep, resume of methods is very needed and it was what made V1 stand out, I mean perfect.

[ghost]

Hi @raix ,

ok , I get the idea of Ground:DB II, maybe that would make it easier to implement something like offline storage for React/GraphQL, more decoupled and adaptable.
But it also makes the interface a little more complex.

Would it be possible to have a sample/demo app or some more documentation then so we can understand the new functionality and implement it into our apps?

Regards

[raix]

@Der-Ede I'm not sure, theres already a nice implementation for redux https://github.com/rt2zz/redux-persist

At the moment I'm holding back just a bit on Meteor and GraphQL

[dagatsoin]

Hi @raix
I wonder if it is still possible to watch the Meteor.user() and keep it offline.
I did:

    export const GUser = new GroundCollection('user');

    GUser.observeSource(Meteor.users.findOne(Meteor.userId()));

But I have errors:

Exception in onLogin callback: TypeError: source.find is not a function
    at GroundCollection.observeSource (http://localhost:3000/packages/ground_db.js?hash=bc8ae55d470c7d2e1ee35f4648843a9e00419508:333:83)
    at http://localhost:3000/app/app.js?hash=02dc6c704b922bb5297e8c71d239f4cd1e743d97:791:27
    at runAndHandleExceptions (http://localhost:3000/packages/callback-hook.js?hash=fff7fdef0707c85900b21f766a4b6c65bf278ff4:162:24)
    at http://localhost:3000/packages/callback-hook.js?hash=fff7fdef0707c85900b21f766a4b6c65bf278ff4:169:12
    at http://localhost:3000/packages/accounts-base.js?hash=93b4e125b77b1eebf232c67314f674ce3d15f389:313:9
    at Hook.each (http://localhost:3000/packages/callback-hook.js?hash=fff7fdef0707c85900b21f766a4b6c65bf278ff4:138:15)
    at http://localhost:3000/packages/accounts-base.js?hash=93b4e125b77b1eebf232c67314f674ce3d15f389:312:25
    at http://localhost:3000/packages/underscore.js?hash=27b3d669b418de8577518760446467e6ff429b1e:794:19
    at loggedInAndDataReadyCallback (http://localhost:3000/packages/accounts-base.js?hash=93b4e125b77b1eebf232c67314f674ce3d15f389:434:5)
    at MethodInvoker._callback (http://localhost:3000/packages/meteor.js?hash=ae8b8affa9680bf9720bd8f7fa112f13a62f71c3:1105:22)

[raix]

@dagatsoin I think you might have to hijack the Meteor.users collection on the client, eg.:

  export const GUser = new GroundCollection('user');
  GUser.observeSource(Meteor.users.find());
  Meteor.users.find = function(args...) {
    return GUser.find(...args);
  }

[dagatsoin]

No, ever tried bit as as I log into:

Exception in delivering result of invoking 'login': MinimongoError: Modifier must be an object
    at MinimongoError (http://localhost:3000/packages/minimongo.js?hash=88217d643bc16fdf3505c6d4b2b8f5ddc400c49a:82:11)
    at Function.LocalCollection._modify (http://localhost:3000/packages/minimongo.js?hash=88217d643bc16fdf3505c6d4b2b8f5ddc400c49a:3100:11)
    at LocalCollection.update (http://localhost:3000/packages/minimongo.js?hash=88217d643bc16fdf3505c6d4b2b8f5ddc400c49a:818:21)
    at LocalCollection.upsert (http://localhost:3000/packages/minimongo.js?hash=88217d643bc16fdf3505c6d4b2b8f5ddc400c49a:855:15)
    at GroundCollection.upsert (http://localhost:3000/packages/ground_db.js?hash=bc8ae55d470c7d2e1ee35f4648843a9e00419508:644:61)
    at MeteorAccount.autoSaveUser (http://localhost:3000/app/app.js?hash=0ea2778e63f85ccc27ea0cd53c04c56b50c314fb:951:28)
    at http://localhost:3000/app/app.js?hash=0ea2778e63f85ccc27ea0cd53c04c56b50c314fb:919:27
    at userCallback (http://localhost:3000/packages/accounts-password.js?hash=26c5712042d53cb92bed4571735603904d1ec8bd:96:23)
    at http://localhost:3000/packages/accounts-base.js?hash=93b4e125b77b1eebf232c67314f674ce3d15f389:322:26
    at http://localhost:3000/packages/underscore.js?hash=27b3d669b418de8577518760446467e6ff429b1e:794:19

Or when I want to just keeping the current user with

Accounts.onLogin(()=> {
  GUser.keep(Meteor.users.findOne(Meteor.userId()));
});

I get

Exception in onLogin callback: TypeError: Cannot read property 'map' of undefined
    at http://localhost:3000/packages/ground_db.js?hash=bc8ae55d470c7d2e1ee35f4648843a9e00419508:454:22
    at Array.map (native)
    at Function._.map._.collect (http://localhost:3000/packages/underscore.js?hash=27b3d669b418de8577518760446467e6ff429b1e:167:56)
    at GroundCollection.keep (http://localhost:3000/packages/ground_db.js?hash=bc8ae55d470c7d2e1ee35f4648843a9e00419508:453:33)
    at http://localhost:3000/app/app.js?hash=fc1fa9b1adbe20fb82bdfc1c8450121e49514b1c:799:19
    at runAndHandleExceptions (http://localhost:3000/packages/callback-hook.js?hash=fff7fdef0707c85900b21f766a4b6c65bf278ff4:162:24)
    at http://localhost:3000/packages/callback-hook.js?hash=fff7fdef0707c85900b21f766a4b6c65bf278ff4:169:12
    at http://localhost:3000/packages/accounts-base.js?hash=93b4e125b77b1eebf232c67314f674ce3d15f389:313:9
    at Hook.each (http://localhost:3000/packages/callback-hook.js?hash=fff7fdef0707c85900b21f766a4b6c65bf278ff4:138:15)
    at http://localhost:3000/packages/accounts-base.js?hash=93b4e125b77b1eebf232c67314f674ce3d15f389:312:25
account.service.ts:75 METEOR USERID CHANGED rSBkx5hNdxDyhoMga
debug.js:41 Exception in onLogin callback: TypeError: Cannot read property 'map' of undefined
    at http://localhost:3000/packages/ground_db.js?hash=bc8ae55d470c7d2e1ee35f4648843a9e00419508:454:22
    at Array.map (native)
    at Function._.map._.collect (http://localhost:3000/packages/underscore.js?hash=27b3d669b418de8577518760446467e6ff429b1e:167:56)
    at GroundCollection.keep (http://localhost:3000/packages/ground_db.js?hash=bc8ae55d470c7d2e1ee35f4648843a9e00419508:453:33)
    at http://localhost:3000/app/app.js?hash=fc1fa9b1adbe20fb82bdfc1c8450121e49514b1c:799:19
    at runAndHandleExceptions (http://localhost:3000/packages/callback-hook.js?hash=fff7fdef0707c85900b21f766a4b6c65bf278ff4:162:24)
    at http://localhost:3000/packages/callback-hook.js?hash=fff7fdef0707c85900b21f766a4b6c65bf278ff4:169:12
    at http://localhost:3000/packages/accounts-base.js?hash=93b4e125b77b1eebf232c67314f674ce3d15f389:313:9
    at Hook.each (http://localhost:3000/packages/callback-hook.js?hash=fff7fdef0707c85900b21f766a4b6c65bf278ff4:138:15)
    at http://localhost:3000/packages/accounts-base.js?hash=93b4e125b77b1eebf232c67314f674ce3d15f389:312:25
debug.js:41 Exception in delivering result of invoking 'login': MinimongoError: Modifier must be an object
    at MinimongoError (http://localhost:3000/packages/minimongo.js?hash=88217d643bc16fdf3505c6d4b2b8f5ddc400c49a:82:11)
    at Function.LocalCollection._modify (http://localhost:3000/packages/minimongo.js?hash=88217d643bc16fdf3505c6d4b2b8f5ddc400c49a:3100:11)
    at LocalCollection.update (http://localhost:3000/packages/minimongo.js?hash=88217d643bc16fdf3505c6d4b2b8f5ddc400c49a:818:21)
    at LocalCollection.upsert (http://localhost:3000/packages/minimongo.js?hash=88217d643bc16fdf3505c6d4b2b8f5ddc400c49a:855:15)
    at GroundCollection.upsert (http://localhost:3000/packages/ground_db.js?hash=bc8ae55d470c7d2e1ee35f4648843a9e00419508:644:61)
    at MeteorAccount.autoSaveUser (http://localhost:3000/app/app.js?hash=fc1fa9b1adbe20fb82bdfc1c8450121e49514b1c:954:28)
    at http://localhost:3000/app/app.js?hash=fc1fa9b1adbe20fb82bdfc1c8450121e49514b1c:922:27
    at userCallback (http://localhost:3000/packages/accounts-password.js?hash=26c5712042d53cb92bed4571735603904d1ec8bd:96:23)
    at http://localhost:3000/packages/accounts-base.js?hash=93b4e125b77b1eebf232c67314f674ce3d15f389:322:26
    at http://localhost:3000/packages/underscore.js?hash=27b3d669b418de8577518760446467e6ff429b1e:794:19

[ilan-schemoul]

Personally meteor.user can be grounded (tough the doc would gain to be more clear about keep/observeSource etc.)

[ilan-schemoul]

@raix why do you say :

V1 listens to the localstorage events making updates in one tab also update in other tabs even if the app is offline, it's a feature that might be implemented in V2 later

In the description :

This version of GroundDB is a caching only storage - meaning it does not support resuming of method calls/cross tab updates etc.

Actually when I make changes in a tab, reactivity makes that my other tab are updated accordingly to the changes I made from one tab. When I read you first, I've been thinking that when an user make a change in one tab, others tabs would keep an old, not updated accordingly to the other tab, version but it's not the case.

[ilan-schemoul]

@raix can you explain why do you say that there's no sync between tabs, we're some to be confused

[raix]

If you disconnect the server tab updates will stop working

[raix]

what do you mean by "core principle" of Meteor?

[dagatsoin]

The behavior you described is not related with data persistence. It is related to network.
Having two tabs opened is like having two smartphones connecting to the same server. So if your server stops your smartphones also stops to be in sync. No matter If you use Meteor with/out ground collection. And this is the same for tabs.

[ilan-schemoul]

In anyway as methods doesn't work offline when tab are needed to be synced when offline ?

[ghost]

sorry to ask, but what is it with the tabs?
usually users/customers only have one tab open.
if you want to test different users and the synchronization between them i'd recommend just opening one tab in two different browsers so you can be sure not to sync just the internal local storage but that the syncing is done through the server.

[ilan-schemoul]

In my website people will be likely to have multiple tabs open.

[ghost]

@nitrobay: I was just thinking whether I open multiple tabs on a site and actually I do, but only on google search and on my cloud bookkeeping software, in the second case of which this could be omitted if the service was programmed better.
I mean you could always replace the need for different tabs by just adding bootstrap tabs to the top of your service/website emulating the different tabs of the browser.
One other principle of meteor is to serve for the 90-95% of use cases. In special cases you probably would have to roll your own (based on grounddb).
Especially testing and trying out the behavior of the local storage. Do the different tabs all write into one local storage or separate and how can this be controlled? I mean you surely don't want to have unpredictable behavior of different instances writing simultaneously into the local storage.
Does the grounddb demo function on different tabs?

[ilan-schemoul]

Yeah I'll definitely test this use case

[btafel]

Hi @dagatsoin any luck caching Meteor.user? Thanks