Error when using Download My Data or Export My Data #10777

bkraul · 2018-05-15T18:02:45Z

Description:

When attempting to export or download a user's data, the system errors out. No additional configuration has been done to User Data Download section, other than it is enabled.

Server Setup Information:

Version of Rocket.Chat Server:
Operating System: Ubuntu 18.04 LTS
Deployment Method(snap/docker/tar/etc): docker rocketchat/rocket.chat:latest
Number of Running Instances: 1

Rocket.Chat

Version	0.64.1
Database Migration	115
Database Migration Date	Mon May 07 2018 10:41:14 GMT-0500 (Central Daylight Time)
Installed at	Mon Jan 29 2018 16:02:11 GMT-0600 (Central Standard Time)
Uptime	8 days, 2 hours, 28 seconds
Deployment ID	JAELT9iyADPbKEGbG
PID	1
Running Instances	1
OpLog	Enabled

Commit

Hash	`54738c0`
Date	Thu May 3 13:37:12 2018 -0300
Branch	HEAD
Tag	0.64.1
Author	Rodrigo Nascimento
Subject	Merge pull request #10660 from RocketChat/release-0.64.1

Steps to Reproduce:

Go to User -> My Account -> My Data -> Download/Export My Data

Expected behavior:

The data should be made available for download.

Actual behavior:

System errors out with "Internal Server Error" message:

Relevant logs:

Exception while invoking method 'requestDataDownload' Error: ENOENT: no such file or directory, mkdir '/tmp/userData/*********'
    at Object.fs.mkdirSync (fs.js:885:18)
    at MethodInvocation.requestDataDownload (/app/bundle/programs/server/app/app.js:10356:10)
    at MethodInvocation.methodMap.(anonymous function) (packages/rocketchat_monitoring.js:2731:30)
    at MethodInvocation.methodsMap.(anonymous function) (/app/bundle/programs/server/packages/rocketchat_lib.js:2186:28)
    at maybeAuditArgumentChecks (/app/bundle/programs/server/packages/ddp-server.js:1877:12)
    at DDP._CurrentMethodInvocation.withValue (/app/bundle/programs/server/packages/ddp-server.js:902:126)
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1186:12)
    at DDPServer._CurrentWriteFence.withValue (/app/bundle/programs/server/packages/ddp-server.js:902:98)
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1186:12)
    at Promise (/app/bundle/programs/server/packages/ddp-server.js:902:46)
    at new Promise (<anonymous>:null:null)
    at Session.method (/app/bundle/programs/server/packages/ddp-server.js:875:23)
    at /app/bundle/programs/server/packages/ddp-server.js:754:85

The text was updated successfully, but these errors were encountered:

TwizzyDizzy · 2018-05-18T13:34:18Z

Can confirm this on 0.64.2-rc.2.

Download my data error

May 18 15:32:09 dev-chat RocketChat: #033[34mMeteor ➔ method#033[39m requestDataDownload -> userId: mFFfETbmt3gxcFk2G , arguments:  { '0': { fullExport: false } }
May 18 15:32:09 dev-chat RocketChat: Exception while invoking method 'requestDataDownload' Error: ENOENT: no such file or directory, mkdir '/tmp/userData/mFFfETbmt3gxcFk2G'
May 18 15:32:09 dev-chat RocketChat: at Object.fs.mkdirSync (fs.js:885:18)
May 18 15:32:09 dev-chat RocketChat: at MethodInvocation.requestDataDownload (/opt/rocketchat/bundle/programs/server/app/app.js:10630:10)
May 18 15:32:09 dev-chat RocketChat: at MethodInvocation.methodMap.(anonymous function) (packages/rocketchat_monitoring.js:2731:30)
May 18 15:32:09 dev-chat RocketChat: at MethodInvocation.methodsMap.(anonymous function) (/opt/rocketchat/bundle/programs/server/packages/rocketchat_lib.js:2261:36)
May 18 15:32:09 dev-chat RocketChat: at maybeAuditArgumentChecks (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:1877:12)
May 18 15:32:09 dev-chat RocketChat: at DDP._CurrentMethodInvocation.withValue (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:902:126)
May 18 15:32:09 dev-chat RocketChat: at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1186:12)
May 18 15:32:09 dev-chat RocketChat: at DDPServer._CurrentWriteFence.withValue (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:902:98)
May 18 15:32:09 dev-chat RocketChat: at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1186:12)
May 18 15:32:09 dev-chat RocketChat: at Promise (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:902:46)
May 18 15:32:09 dev-chat RocketChat: at new Promise (<anonymous>:null:null)
May 18 15:32:09 dev-chat RocketChat: at Session.method (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:875:23)
May 18 15:32:09 dev-chat RocketChat: at /opt/rocketchat/bundle/programs/server/packages/ddp-server.js:754:85

Export my data error

May 18 15:32:26 dev-chat RocketChat: #033[34mMeteor ➔ method#033[39m requestDataDownload -> userId: mFFfETbmt3gxcFk2G , arguments:  { '0': { fullExport: true } }
May 18 15:32:26 dev-chat RocketChat: Exception while invoking method 'requestDataDownload' Error: ENOENT: no such file or directory, mkdir '/tmp/userData/mFFfETbmt3gxcFk2G'
May 18 15:32:26 dev-chat RocketChat: at Object.fs.mkdirSync (fs.js:885:18)
May 18 15:32:26 dev-chat RocketChat: at MethodInvocation.requestDataDownload (/opt/rocketchat/bundle/programs/server/app/app.js:10630:10)
May 18 15:32:26 dev-chat RocketChat: at MethodInvocation.methodMap.(anonymous function) (packages/rocketchat_monitoring.js:2731:30)
May 18 15:32:26 dev-chat RocketChat: at MethodInvocation.methodsMap.(anonymous function) (/opt/rocketchat/bundle/programs/server/packages/rocketchat_lib.js:2261:36)
May 18 15:32:26 dev-chat RocketChat: at maybeAuditArgumentChecks (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:1877:12)
May 18 15:32:26 dev-chat RocketChat: at DDP._CurrentMethodInvocation.withValue (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:902:126)
May 18 15:32:26 dev-chat RocketChat: at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1186:12)
May 18 15:32:26 dev-chat RocketChat: at DDPServer._CurrentWriteFence.withValue (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:902:98)
May 18 15:32:26 dev-chat RocketChat: at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1186:12)
May 18 15:32:26 dev-chat RocketChat: at Promise (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:902:46)
May 18 15:32:26 dev-chat RocketChat: at new Promise (<anonymous>:null:null)
May 18 15:32:26 dev-chat RocketChat: at Session.method (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:875:23)
May 18 15:32:26 dev-chat RocketChat: at /opt/rocketchat/bundle/programs/server/packages/ddp-server.js:754:85

Cheers
Thomas

graywolf336 · 2018-05-19T02:05:53Z

@RocketChat/core This is an important piece of our GDPR and seeing as it is broke on instances, we need to get it fixed asap.

TkTech · 2019-03-12T18:51:03Z

Should probably be re-opened, still broken in 0.74.3 and 1.0.0-beta.0. 69 jobs in rockechat_export_operations with the oldest one from 2018-06-02. Cleared all old jobs, queued up a new one, same error.

This feature doesn't seem to have ever actually worked except for single server setups.

{ Error: ENOENT: no such file or directory, mkdir '/tmp/userData/7zfSTXWNPAdXaToex/partial'
    at Object.fs.mkdirSync (fs.js:885:18)
    at createDir (packages/rocketchat:user-data-download/server/cronProcessDownloads.js:33:6)
    at continueExportingRoom (packages/rocketchat:user-data-download/server/cronProcessDownloads.js:186:2)
    at exportOperation.roomList.forEach.exportOpRoomData (packages/rocketchat:user-data-download/server/cronProcessDownloads.js:418:5)
    at Array.forEach (<anonymous>)
    at continueExportOperation (packages/rocketchat:user-data-download/server/cronProcessDownloads.js:417:29)
    at cursor.forEach.exportOperation (packages/rocketchat:user-data-download/server/cronProcessDownloads.js:467:3)
    at SynchronousCursor.forEach (packages/mongo/mongo_driver.js:1107:16)
    at Cursor.(anonymous function) [as forEach] (packages/mongo/mongo_driver.js:887:44)
    at Object.processDataDownloads [as job] (packages/rocketchat:user-data-download/server/cronProcessDownloads.js:462:9)
    at packages/littledata_synced-cron.js:251:26
    at scheduleTimeout (packages/littledata_synced-cron.js:304:9)
    at packages/littledata_synced-cron.js:356:42
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1304:12)
    at packages/meteor.js:620:25
    at runWithEnvironment (packages/meteor.js:1356:24)
  errno: -2,
  code: 'ENOENT',
  syscall: 'mkdir',
  path: '/tmp/userData/7zfSTXWNPAdXaToex/partial' }

bkraul · 2019-03-13T17:30:19Z

I can confirm this on the latest docker develop version.

dhoeld · 2019-05-01T19:40:30Z

We are able to confirm this on Rocket.Chat version 0.74.3 too.

mghicas · 2019-05-03T17:14:46Z

Anyone have a fix for this?
On thing i noticed in my setup (3 docker containers on 3 different hosts) - One container has /tmp/userData created and the others done. Do you need to bind a host directory to /tmp/userData?

Also running 0.74.3

himpich · 2019-05-27T07:51:43Z

According to the logfiles the requestDataDownload Service is trying to create a "tempdir/partial" folder without creating the "tempdir" folder first (tempdir seems to be the userId?)

The file.server Source uses "mkdirp" instead of "fs" which creates parent directories as well. Switching to that method should solve at least the "mkdir" error

mghicas · 2019-06-06T15:05:52Z

User Data export in 1.1.1 (Docker Image at least) also appears to be broken.
When you select the export sand the internal scheduler kicks it off:

The directories are created
The error below is thrown
Email is never sent (but email is tested and working)

Setup is 3 containers on different hosts with an external MongoDB

From inside the docker container on node1:

#cd /tmp/userData
#find .
./userData
./userData/eq5ka6HvRJnaNg3Rp
./userData/eq5ka6HvRJnaNg3Rp/full
./userData/eq5ka6HvRJnaNg3Rp/full/assets
./userData/eq5ka6HvRJnaNg3Rp/partial
./userData/eq5ka6HvRJnaNg3Rp/partial/assets

Error messages in the logs:

Jun  2 03:45:00 rchat01 dockerd-current[9150]: #033[34mSyncedCron ➔ info#033[39m Starting "Generate download files for user data".
Jun  2 03:45:00 rchat01 dockerd-current[9150]: { Error: ENOENT: no such file or directory, mkdir '/tmp/userData/eq5ka6HvRJnaNg3Rp/full'
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at Object.fs.mkdirSync (fs.js:885:18)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at createDir (app/user-data-download/server/cronProcessDownloads.js:36:6)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at continueExportingRoom (app/user-data-download/server/cronProcessDownloads.js:189:2)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at exportOperation.roomList.forEach.exportOpRoomData (app/user-data-download/server/cronProcessDownloads.js:419:5)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at Array.forEach (<anonymous>)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at continueExportOperation (app/user-data-download/server/cronProcessDownloads.js:418:29)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at cursor.forEach.exportOperation (app/user-data-download/server/cronProcessDownloads.js:468:3)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at SynchronousCursor.forEach (packages/mongo/mongo_driver.js:1107:16)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at Cursor.(anonymous function) [as forEach] (packages/mongo/mongo_driver.js:887:44)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at Object.processDataDownloads [as job] (app/user-data-download/server/cronProcessDownloads.js:463:9)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at packages/littledata_synced-cron.js:251:26
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at scheduleTimeout (packages/littledata_synced-cron.js:304:9)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at packages/littledata_synced-cron.js:356:42
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1304:12)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at packages/meteor.js:620:25
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at runWithEnvironment (packages/meteor.js:1356:24)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:   errno: -2,
Jun  2 03:45:00 rchat01 dockerd-current[9150]:   code: 'ENOENT',
Jun  2 03:45:00 rchat01 dockerd-current[9150]:   syscall: 'mkdir',
Jun  2 03:45:00 rchat01 dockerd-current[9150]:   path: '/tmp/userData/eq5ka6HvRJnaNg3Rp/full' }
Jun  2 03:45:00 rchat01 dockerd-current[9150]: #033[34mSyncedCron ➔ info#033[39m Not running "Generate download files for user data" again.
Jun  2 03:45:00 rchat01 dockerd-current[9150]: #033[34mSyncedCron ➔ info#033[39m Not running "Generate download files for user data" again.
Jun  2 03:45:00 rchat01 dockerd-current[9150]: #033[34mSyncedCron ➔ info#033[39m Finished "Generate download files for user data".
Jun  2 03:45:00 rchat01 dockerd-current[9150]: #033[34mSyncedCron ➔ info#033[39m Not running "Generate download files for user data" again.
Jun  2 03:45:00 rchat01 dockerd-current[9150]: #033[34mSyncedCron ➔ info#033[39m Not running "Generate download files for user data" again.
Jun  2 03:45:00 rchat01 dockerd-current[9150]: #033[34mSyncedCron ➔ info#033[39m Not running "Generate download files for user data" again.

Hudell · 2019-06-06T16:19:47Z

Yup, #14645 fixes this.

0l-l0 · 2019-09-10T14:14:08Z

The same issue with the official docker images (latest, develop, release-candidate) from here.
The server's error log:

{ Error: ENOENT: no such file or directory, mkdir '/tmp/userData/EvFWaNTxwXrBP25zc/full'                                                                                                         
    at Object.fs.mkdirSync (fs.js:885:18)                                                                                                                                                        
    at createDir (app/user-data-download/server/cronProcessDownloads.js:36:6)                                                                                                                    
    at continueExportingRoom (app/user-data-download/server/cronProcessDownloads.js:189:2)
    at exportOperation.roomList.forEach.exportOpRoomData (app/user-data-download/server/cronProcessDownloads.js:419:5)
    at Array.forEach (<anonymous>)
    at continueExportOperation (app/user-data-download/server/cronProcessDownloads.js:418:29)
    at cursor.forEach.exportOperation (app/user-data-download/server/cronProcessDownloads.js:468:3)
    at SynchronousCursor.forEach (packages/mongo/mongo_driver.js:1107:16)
    at Cursor.(anonymous function) [as forEach] (packages/mongo/mongo_driver.js:887:44)
    at processDataDownloads (app/user-data-download/server/cronProcessDownloads.js:463:9)
    at app/user-data-download/server/cronProcessDownloads.js:479:3
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1304:12)
    at packages/meteor.js:620:25
    at runWithEnvironment (packages/meteor.js:1356:24)
  errno: -2,
  code: 'ENOENT',
  syscall: 'mkdir',
  path: '/tmp/userData/EvFWaNTxwXrBP25zc/full' }

bkraul · 2019-09-10T14:17:02Z

I can confirm this. This seems to not have yet been fully corrected.

eeowaa · 2019-10-23T14:28:26Z

My RocketChat server admin has confirmed that we are at the latest version yet cannot export. He pointed me to this ticket as indication that we are stuck. That is all the information I have right now but just wanted to add my suspicion that this issue is not actually fixed.

himpich · 2019-10-23T14:51:20Z

It is working on my end now. I configured the correct path and removed the export-jobs from the mongo db. New jobs ran through.

reetp · 2019-10-23T14:52:03Z

It is working on my end now. I configured the correct path and removed the export-jobs from the mongo db. New jobs ran through.

For others can you please post how you did that? Thanks.

himpich · 2019-10-23T15:01:17Z

start your mongo client and connect

use correct database:

rs0:PRIMARY> use rocketchat;
switched to db rocketchat

Verify there is some export operations data:

rs0:PRIMARY> db.rocketchat_export_operations.find( {} );

remove it:

rs0:PRIMARY> db.rocketchat_export_operations.find( {} );

re-verify that the table (or whatever it is called in mongodb) is now empty.

Good luck

reetp · 2019-10-23T15:24:15Z

remove it:

rs0:PRIMARY> db.rocketchat_export_operations.find( {} );

Not sure that is quite right !

reetp · 2019-10-23T16:29:55Z

Still not sure this is fixed. v 2.1.1 docker <corrected version - long day :-( >

First, it shouldn’t leave old exports blocking up the system but it clearly has.

Second, to try and get over this I deleted out the old keys as per the above comments. That throws errors as per this:

SyncedCron ➔ info Finished "Generate download files for user data".
SyncedCron ➔ info Not running "Generate download files for user data" again.
SyncedCron ➔ info Not running "Generate download files for user data" again.
SyncedCron ➔ info Not running "Generate download files for user data" again.
SyncedCron ➔ info Not running "Generate download files for user data" again.
Exception on find Error: Meteor code must always run within a Fiber. Try wrapping callbacks that you pass to non-Meteor libraries with Meteor.bindEnvironment.
    at Object.Meteor._nodeCodeMustBeInFiber (packages/meteor.js:1186:11)
    at Meteor.EnvironmentVariable.EVp.get (packages/meteor.js:1199:10)
    at Object.collection.(anonymous function) [as findOne] (packages/matb33_collection-hooks.js:132:37)

Full log: https://pastebin.com/Fq7q3N7t

I then tried adding one html and one json export for my user.

It puts the new file paths in the settings but does not use appear to use them - see pics.

One very unhappy test install right now :-(

graywolf336 added the type: bug label May 19, 2018

Hudell self-assigned this May 21, 2018

Hudell mentioned this issue May 21, 2018

[FIX] Internal Error when requesting user data download #10837

Merged

rodrigok closed this as completed in #10837 May 21, 2018

kaiiiiiiiii mentioned this issue Jun 4, 2018

"Internal server error" when downloading "my data" from the user Profile #10994

Closed

Hudell reopened this Mar 12, 2019

Hudell mentioned this issue May 27, 2019

[FIX] Error when using Download My Data or Export My Data #14645

Merged

sampaiodiego closed this as completed in #14645 Jun 21, 2019

reetp mentioned this issue Oct 24, 2019

[GDPR] Cron jobs for User data export wedged #15663

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error when using Download My Data or Export My Data #10777

Error when using Download My Data or Export My Data #10777

bkraul commented May 15, 2018

TwizzyDizzy commented May 18, 2018 •

edited

Loading

graywolf336 commented May 19, 2018 •

edited by geekgonecrazy

Loading

TkTech commented Mar 12, 2019

bkraul commented Mar 13, 2019

dhoeld commented May 1, 2019

mghicas commented May 3, 2019 •

edited

Loading

himpich commented May 27, 2019

mghicas commented Jun 6, 2019

Hudell commented Jun 6, 2019

0l-l0 commented Sep 10, 2019

bkraul commented Sep 10, 2019

eeowaa commented Oct 23, 2019

himpich commented Oct 23, 2019

reetp commented Oct 23, 2019

himpich commented Oct 23, 2019

reetp commented Oct 23, 2019

reetp commented Oct 23, 2019 •

edited

Loading

Error when using Download My Data or Export My Data #10777

Error when using Download My Data or Export My Data #10777

Comments

bkraul commented May 15, 2018

Description:

Server Setup Information:

Rocket.Chat

Commit

Steps to Reproduce:

Expected behavior:

Actual behavior:

Relevant logs:

TwizzyDizzy commented May 18, 2018 • edited Loading

graywolf336 commented May 19, 2018 • edited by geekgonecrazy Loading

TkTech commented Mar 12, 2019

bkraul commented Mar 13, 2019

dhoeld commented May 1, 2019

mghicas commented May 3, 2019 • edited Loading

himpich commented May 27, 2019

mghicas commented Jun 6, 2019

Hudell commented Jun 6, 2019

0l-l0 commented Sep 10, 2019

bkraul commented Sep 10, 2019

eeowaa commented Oct 23, 2019

himpich commented Oct 23, 2019

reetp commented Oct 23, 2019

himpich commented Oct 23, 2019

reetp commented Oct 23, 2019

reetp commented Oct 23, 2019 • edited Loading

TwizzyDizzy commented May 18, 2018 •

edited

Loading

graywolf336 commented May 19, 2018 •

edited by geekgonecrazy

Loading

mghicas commented May 3, 2019 •

edited

Loading

reetp commented Oct 23, 2019 •

edited

Loading