model.getCache throws 'Maximum call stack size exceeded' error

[threepointone]

This usually happens after making a request to an HttpDataSource. I have a standalone reproducible case (pardon the es6/7) -

import request from 'superagent';
import Router from 'falcor-router';
import falcor from 'falcor';

// a 'service' that fetches user details from github
function user(id){
  return new Promise((resolve, reject) =>
    request.get(`https://api.github.com/users/${id}`).end((err, res) =>
      err ? reject(err) : resolve(res.body)));
}

var model = new falcor.Model({
  source: new Router([{
    // define a single route corresponding to the above service
    route: 'users[{keys:ids}]',
    get: async function (pathSet){
      let responses = await Promise.all(pathSet.ids.map(user));
      return pathSet.ids.map((id, i) =>
        ({path: ['users', id],  value: responses[i]}));
    }
  }])
});

console.log(model.getCache()); // empty, all good

model.get(`users.threepointone['id', 'url', 'name']`)
  .subscribe(function(json){
    try {
      console.log(model.getCache());
    }
    catch (e){
      // throws error
      // Maximum call stack size exceeded
      console.error(e);
    }
  }, function(error){
    console.error(error);
  });

I see that the problem is happening in a _copyCache function, where it's recursively going through its internal cache, but following a parent key (and others) resulting in a circular reference?

[ThePrimeagen]

Hmm. This should be an easy fix. I will have to look into this shortly, if I am the one to fix it :)

---

Btw, @threepointone we are more than happy to receive a PR with associated unit test to confirm. If you already know the spot should be an easy fix an parent and other internal keys cache keys should be stripped before cloning.

[threepointone]

thanks, I'll try today!

[threepointone]

It looks like every object is tagged with a number of keys, could you explain how you prevent clashes with pre-existing keys?

The code in question is this bit here - I'm unsure whether I should remove any key named parent, since objects could come in with said key. also, how to deal with version?

[ThePrimeagen]

@threepointone Yes, those keys are actually something we were going to change anyways. Essentially what is going on here is that parent key is actually <30>parent where <30> is ascii code 30, record separating character. We are going to go with a $ one here for performance reasons, but for now you can always (which I thought it did) for the first character being that prefix character remove it for the depth recursing.

[ThePrimeagen]

@threepointone 0.x / master should have the fix. Could you verify that this fixes the issue for you? If, for whatever reason its not, please paste in the state of the cache before serialization.

[threepointone]

This error still persists. The above script still throws the same error.

[threepointone]

logged just before it executes _copyCache.

{ users: 
   { threepointone: 
      { login: 'threepointone',
        id: [Object],
        avatar_url: 'https://avatars.githubusercontent.com/u/18808?v=3',
        gravatar_id: '',
        url: [Object],
        html_url: 'https://github.com/threepointone',
        followers_url: 'https://api.github.com/users/threepointone/followers',
        following_url: 'https://api.github.com/users/threepointone/following{/other_user}',
        gists_url: 'https://api.github.com/users/threepointone/gists{/gist_id}',
        starred_url: 'https://api.github.com/users/threepointone/starred{/owner}{/repo}',
        subscriptions_url: 'https://api.github.com/users/threepointone/subscriptions',
        organizations_url: 'https://api.github.com/users/threepointone/orgs',
        repos_url: 'https://api.github.com/users/threepointone/repos',
        events_url: 'https://api.github.com/users/threepointone/events{/privacy}',
        received_events_url: 'https://api.github.com/users/threepointone/received_events',
        type: 'User',
        site_admin: false,
        name: [Object],
        company: null,
        blog: null,
        location: 'Bangalore, India',
        email: 'threepointone@gmail.com',
        hireable: true,
        bio: null,
        public_repos: 89,
        public_gists: 18,
        followers: 184,
        following: 3,
        created_at: '2008-07-29T12:17:53Z',
        updated_at: '2015-11-26T10:12:59Z',
        '\u001ekey': 'threepointone',
        '\u001eparent': [Circular],
        '\u001eversion': 1,
        'ツabsolutePath': [Object],
        '$size': 202 },
     '\u001ekey': 'users',
     '\u001eparent': [Circular],
     'ツabsolutePath': [ 'users' ],
     '$size': 202,
     '\u001eversion': 1 },
  '\u001eversion': 1,
  '$size': 202 }

[threepointone]

Specifically, it looks like some keys have ツ prefix, and some have \u001e?

[ThePrimeagen]

@threepointone You should re-pull master. That was during a transition of performance changes. We went from an untype-able character, to a type-able character. We found huge performance wins both on device and node by going from variable keys to static (using . separator).

I believe if you repull this problem should not persist. parent should be removed in the filter method here: https://github.com/Netflix/falcor/blob/master/lib/get/getCache.js#L45 We strip out internal keys. Could you retry this and validate for me? If take your example and make all internal keys ツ prefixed, then there is no issues going into _copyCache with that example.