"Server sent add for existing id" error

[bradvogel]

Ever since introducing fastrender to my app, I'm getting "Server sent add for existing id". Any guidance on what might be the cause or how I might track it down?

[arunoda]

This could be a fast-render issue.

You can use following techniques to debug this:

• First get familiar with our debugging commands: https://github.com/meteorhacks/fast-render#debugging
• Add fast-render support for one route at a time
• with that find out what's the publication doing causing that
• when you isolate the issue
• send me the output of FastRender.debugger.getPayloadJSON() and FastRender.debugger.getLogsJSON()

If possible, try to create a sample repo and send it to me. Then I can work on the issue, very quickly.

[wearhere]

Related issue/comment:

Meteor isnt designed to expect data in a collection before subscription is complete.

But that author suggests that you already worked around this at least to some extent, @arunoda?

[wearhere]

@arunoda could this be happening if we (I work with @bradvogel) subscribe to a particular collection in Router.configure and again in a particular route?

This Gist contains code excerpts from the relevant files.

I see two potential problems there:

• we run the subscription twice (it's not the same exact subscription, but it is the same record being returned, just with different fields)
• we run the subscription once without fastRender: true, in the global config (not sure why that is, maybe we had routes we didn't want to use FastRender for–@bradvogel?), and then with fastRender: true in the route.

Either/both of the above sound problematic @arunoda? I'm also happy to get you the debug logs you asked for, just thought you might immediately see one or both of these as the cause of the issue.

[bradvogel]

I found out what the issue was. I was updating a collection inside the rendered callback.

For others who might come across this issue, here's a (simplified) example that will trigger the exception:

router.js

Router.route('message', {
  path: '/:id',
  fastRender: true,
  waitOn: function() {
    return Meteor.subscribe('mymessage', this.params.id);
  },
  data: function(){
    return Messages.findOne(this.params.id);
  }
});

message.js:

Template.message.rendered = function(){
   // This will throw an exception.
   Messages.update(this.data.id, {
        lastopened: Date.now()
   });
}

The fix is to not touch the collection inside the rendered method because it's still running synchronously after fastRender has simulated the update coming through. I haven't looking into the details but I assume that's what's happening.

[bradvogel]

Reopening since this still looks like a fastrender bug. @arunoda ?

[arunoda]

This is interesting. Seems like something wrong going here.

I will try to reproduce and try to find a cure.
On 2014 දෙසැ 21, ඉරිදා at ප.ව. 6.15 Brad Vogel notifications@github.com
wrote:

Reopened #80 #80.

—
Reply to this email directly or view it on GitHub
#80 (comment).

[arunoda]

Yes. I can reproduce this. I can't think of a direct fix to Fast Render right now.

[landland]

hey @arunoda I am getting this after switching to Cloudflare. I've had to remove fast-render for the time being. Without Cloudflare I don't run into this with fast-render installed. Just thought I would provide this extra information in case that helps.

[bradvogel]

@arunoda Any tips on working around this? I'm seeing intermittently, even when the database update happens asynchronously after the template renders.

[bradvogel]

@arunoda I'd like to help with a fix for this. Would you mind quickly giving me a braindump of what the timing issue might be?

[arunoda]

Hey. Thanks for that. What do you mean by the timing issue.

This happens when you are doing a insert operation before the DDP connection comes live.

FastRender adds data to those collection when the page loads. That's the root cause for this issue.

As a quick fix, you can do these write ops using a method. But don't implement that method on the client. Define it only in the server.

[landland]

I get this error in a different use case, but it seems strange. If I have fast-render installed but I am not using it, meaning I do not have fastRender: true enabled in any of my routes, I still get this error. Does that sound possible, @arunoda ? I have to completely remove the package for the error to go away.

This is only after I switched to Cloudflare, enabled ssl and disabled websockets, and put the app behind a nginx load balancer. Even when I pause Cloudflare the issue persists. I also do not update any collections in Template rendered callbacks, but do subscribe to collections using a Session variable. Hope that helps.

[arunoda]

Not sure, how it relates to CF.
But your code might have some insert or update ops.

I will look for proper fix. But I can't figure out the root cause exactly.
On 2015 ජන 9, සිකු at පෙ.ව. 11.09 landland notifications@github.com wrote:

I get this error in a different use case, but it seems strange. If I have
fast-render installed but I am not using it, meaning I do not have fastRender:
true enabled in any of my routes, I still get this error. Does that sound
possible, @arunoda https://github.com/arunoda ? I have to completely
remove the package for the error to go away.

This is only after I switched to Cloudflare, enabled ssl and disabled
websockets, and put the app behind a nginx load balancer. Even when I pause
Cloudflare the issue persists. I also do not update any collections in
Template rendered callbacks, but do subscribe to collections using a
Session variable. Hope that helps.

—
Reply to this email directly or view it on GitHub
#80 (comment)
.

[bradvogel]

Thanks for the workaround @arunoda !

[landland]

You're right, @arunoda. I was updating Meteor.users and that was causing the issue. Interesting how this wasn't happening without SSL/disabling websockets. The workaround of updating via server side method works. Thanks man!

[SachaG]

Just wanted to say I also run into this in Telescope from time to time.

[shtefcs]

We are facing same issue on Telescope App.
.

The content is loading to long, it's take few minutes to showup/render. Do u have solutions for @arunoda , cause right now all things we did on Telescope drop in water.

[arunoda]

This happens when we receive a document from server which already exists.
But it should not slow down the content loading.
If you are receiving this all the times, there seems to be something wrong.
Is this with the latest telescope?

On Mon, Apr 20, 2015 at 5:40 PM Stefan Smiljkovic notifications@github.com
wrote:

We are facing same issue on Telescope App.

https://camo.githubusercontent.com/a00ecbd26f29a349800fe1f6c4d005d04c80084a/687474703a2f2f636f6e74656e742e73637265656e636173742e636f6d2f75736572732f736874656663732f666f6c646572732f4a696e672f6d656469612f33353336633338612d636234362d346562632d386531612d3338336339663462663332382f323031352d30342d32305f313430362e706e67
.

The content is loading to long, it's take few minutes to showup/render. Do
u have solutions for @arunoda https://github.com/arunoda , cause right
now all things we did on Telescope drop in water.

—
Reply to this email directly or view it on GitHub
#80 (comment)
.

[delgermurun]

@arunoda I'm calling method after content loaded. So it's takes so long time. If I disable fast-render it works fine.

[arunoda]

@delgermurun which method? I'd love to see a local re-production. Is this with telescope?

[SachaG]

Yes, on Telescope Meta with the latest Telescope master, I sometimes get this when submitting a comment. The comment goes through eventually, but it takes ~1 minute.

[SachaG]

Here is the code to call the comment submission method by the way:

https://github.com/TelescopeJS/Telescope/blob/master/client/views/comments/comment_form.js

And the submitComment method:

https://github.com/TelescopeJS/Telescope/blob/master/collections/comments.js#L206-L259

[delgermurun]

@arunoda it's on my private package and it's very simple method. Just check time and return instantly. I guess maybe, it's taking long time to connect server.

[arunoda]

Okay. May be some regression with some new changes in the core. I'll have a look at this.

[jelenajjo]

Tnx God I am not the only one getting this. My app become very slow and its take to much to load, same as above mention. If i remove fastrender app become like 5-10 slower. So really waiting for solution.

[delgermurun]

@arunoda I'm playing around this issue. I don't know how fast-render works. So I debug only from outside. First of all it happens very inconsistent, so it makes very hard to debug.

I reproduced on latest devel branch of Telescope. Logged in as an admin. I added method on server and calling it from post_body templates onRendered callback. It increases error occurrence.

I opened same post on 2 different tabs and refreshes them. So sometimes one works fine, other one raises error. Sometimes both fails or both works. How amazing is that.

I enabled log, here is some logs.

Log when throws error: https://gist.github.com/delgermurun/b96b9df994beae64b1d5
Here is log when works fine: https://gist.github.com/delgermurun/39f884fdf1834d0ba221

As you see, on those logs there is a lot difference. How that can be happen?

I'm refreshing same browser's (chrome/firefox) 2 tabs (next to each other :) ).

Seems like, if it is occurs first time, happens fairly often. What I mean is if you can produce error, it happens fairly consistent on this tab. If isn't it works fine consistently.

[markudevelop]

Interesting, I also get this error. I'm not sure if this fine to keep fast render and just ignore the error?

[shtefcs]

@arunoda Is there any news about this bug ?

[markshust]

I was receiving the same error, and confirmed at least in my testing that it's only happening with https://github.com/GroundMeteor/db

EDIT: never mind my stupidity, this was due to a circular ref

[SachaG]

Could the error be related to very large documents? I'm getting it a lot when posting on http://meta.telescopeapp.org, and I'm wondering if it's because my user object contains over 700 entries for its user.upvotedComments property.

[arunoda]

Hmm. I'll try to replicate with that.

On Fri, Jul 10, 2015 at 6:57 AM Sacha Greif notifications@github.com
wrote:

Could the error be related to very large documents? I'm getting it a lot
when posting on http://meta.telescopeapp.org, and I'm wondering if it's
because my user object contains over 700 entries for its
user.upvotedComments property.

—
Reply to this email directly or view it on GitHub
#80 (comment)
.

[arunoda]

Thanks sacha for giving me access to the telescope meta db. I found out the problem. Actually, there are a lot of stuff is happening.

But the root problem is, when there is a method call while fast-render doing it's stuff. So, the fix is to defer all Method calls until fast-render do it's job.
I think, I can do it inside fast-render.

[epeng5]

@arunoda any updates here? I'm running into this issue as well on production.

[artiparty]

All search results of my posts look like this in google:

I believe it is related to this issue as well, right?

[SachaG]

Any update on this, by any chance?

[artiparty]

@SachaG once I updated Telescope version this issue solved by itself 👌

[raduchiriac]

(Sorry I know my comment will be too specific to Telescope but...)
I updated to Telescope 0.25.7 and I started seeing this error, some posts weren't even rendering. I had to downgrade to 0.25.6 for now.

@shtefcs what version do you use for your Telescope website. Do you still see it?

@artiparty, really? it's actually the opposite. I wonder how can this be fixed.

[shtefcs]

@raduchiriac I deleted that website and telescope installation. I was bit customized.

[malagant]

I have the same problem with 0.25.7.
Should I downgrade? This error prevents user from using our site. So a quick clarification would be helpful. I can also provide some more input if needed.

[SachaG]

Apparently this only happens when logged in. I'll have to dig deeper to isolate it, but maybe I'll disable FR in the meantime.

[abhaytalreja]

@SachaG - could it be cause of telescopejs/telescope@452e521

Based on the above comments and comparing the commits I see that's the only change in the route.js that can impact it..

@arunoda - This is happening consistently with telescope 0.25.7

[abhaytalreja]

@SachaG and @arunoda - Confirmed, If i comment this line, we are no longer getting the error. - I have another collection which has that line and when i access that detail page.. i get the exception. https://github.com/TelescopeJS/Telescope/blob/452e521ee90e2eef68e37c4829500cd2979c92fb/packages/telescope-posts/lib/routes.js#L18

[arunoda]

This is happens when trying to use a method when FastRendering mostly. May be you can defer it a bit.
Until the connection to Meteor is made?

We can fix this from the FastRender side as well? For now I hope you can do this.

[SachaG]

@abhaytalreja oh, good find. I don't mind delaying it, how would we do that? Or would if (Meteor.isClient) {...} work?

[abhaytalreja]

@SachaG - Sorry for getting a little late.. i moved this increment to onRendered call of the last blaze component.

[tdarci]

@abhaytalreja... thanks for the fix. This problem has been dogging us. Where is this fixed (branch, pr, sha)? We are using 0.25.7

[abhaytalreja]

@tdarci - This issue is not resolved, i have just given the pointer and i fixed it on my local copy!

[tdarci]

@abhaytalreja ... thanks for the reply.
Is the issue due to trackRouteEntry(params._id) in routes.js?
I'd love to see your fix, if at all possible! Thanks.

[wietsevr]

@abhaytalreja Would love to see your solution as well. Thanks in advance!

[abhaytalreja]

@tdarci @wietsevr - Just delay the call trackRouteEntry(params._id); to this in your page load. The way i achieved it was move this line to render function of the template that loaded the last for that page.

[wietsevr]

@abhaytalreja Thanks. Works with me with Meteor.defer()