Target pipeline logging options in a scheduled non interactive workflow

[al-obrien]

• I understand and agree to https://books.ropensci.org/targets/help.html.

I am having trouble finding the best way to have logging and progress messages accessible for monitoring while using targets when deployed in a non interactive context, such as using Posit Connect and refreshing the project every morning.

tar_poll(), and others in that vein, looked promising but they output to the console. As far as I'm aware, this would not be as easy to review mid progress on Posit Connect.

Is there an ideal approach in this situation, perhaps pairing with another logging tool, or an ability to push Target messages to a text file?

[joelnitta]

I haven't personally tried it, but I have heard that {logger} works well.

https://daroczig.github.io/logger/

There is a thread about it on the dslc.io slack.

[al-obrien]

Unfortunately I cannot seem to access that thread to see the specifics.

Upon further digging, it looks like a similar discussion on how to redirect messages from a pipeline was discussed for the predecessor, {drake}:

ropensci/drake#400 (comment).

Similar to that thread I'm also interested in getting messages specific to targets output that normally goes to the console when interactive. Is this as simple as putting sink(), or other logging type functions, somewhere in the _targets.R or just before calling tar_make() (or equivalent distributed versions)?

Not sure if this is something I missed, or if not, if any kind of official support (or recommended patterns) is planned for {targets}.

[wlandau]

tar_make() runs the pipeline in a callr background process, and this process can be configured through arguments callr_function and callr_arguments. The latter may allow you more control about how ordinary print() messages are handled, as well as stdout/stdin redirection.