Issue with df2sd

[rahulmahajan511]

Below is my code logic:-
1-Read the large sas file(>15GB) using pyreadstat in chunk way.
2- Do some processing on each chunk.
3- Create a temp sas file of the chunk using df2sd
4- Combine the chunk into main sas file using sas submit by data step.

The above logic works fine for small file but when i try to run it on my actual (>15GB) sas file, it work fine for initial 2 million records but then df2sd just hangs....It just keep waiting at that step, either i have to interrupt the script or it ends up with error connectionreseterror winerror 10054

When i interrupt the script the logs are also not created for that step..Any idea why this would be happening and way around it

[tomweber-sas]

Hey @rahulmahajan511 I can help with this. Would you mind opening an Issue for this, since it is an issue more than a general discussion. https://github.com/sassoftware/saspy/issues Then we can handle it there! And, I'll need to see some information, so what's your config, for this connection? Can you submit your SASsession object and show the output from that, and can I see the code in question that's being executed.

Thanks,
Tom

[rahulmahajan511]

Thanks for the response, I suggested i have open the issue
#550 (comment)