dispatcher needs option to queue job but also observe log

[gschueler]

Right now there are two options when running commands and jobs with dispatch and/or run:

1. Queue the job. Spits back a URL where you can observe the output of the job.
2. Don't queue. Output comes back to the terminal.

We need an option where it's queued, but the output also comes back to the terminal (as if you went to the URL and followed the output). This is important because we want to allow users not to use the GUI but also use the logging capabilities of the rundeck server.

I would go as far as to say this should be the default behavior, although I am not adamant on that point.

original LH ticket

Original Creator: Moses Lei

[gschueler]

(by Greg Schueler at 2011-05-19T16:57:25-07:00)

this could perhaps be done by having the client side also tail (and parse) the log file that is produced by the server

[gschueler]

(by Moses Lei at 2011-05-20T14:02:01-07:00)

What about when you're running the command line tools from a different box? (This is possible if you just configure framework.properties the right way.)

[gschueler]

(by Noah Campbell at 2011-05-20T14:43:11-07:00)

But that would assume bi-directional communication on the remote machine?

[gschueler]

(by Greg Schueler at 2011-05-20T14:48:41-07:00)

if you use CLI tool or web API to run jobs/adhoc commands from a different
client, then it would require some new api ability to tail log output
remotely...

[gschueler]

(by Moses Lei at 2011-05-20T14:54:58-07:00)

Well, how does the output get to the web page? It's an AJAX call to refresh the latest log output, right? Why can't the CLI tool also use the same call? (This would require a polling from the client, though, which I don't like.)

Alternatively, to make it more realtime, you could just have a fairly simple API request that would just return the log in Transfer-Encoding: chunked form as RunDeck receives it. We should be able to unescape that and output it immediately to the terminal.

[gschueler]

(by Moses Lei at 2011-05-20T14:57:35-07:00)

(In general, I think our goal should be to allow someone to never use the GUI-- they should be able to use all of RunDeck's primary functionality just from the command line or the web service API.)

[gschueler]

(by Greg Schueler at 2011-05-20T15:12:21-07:00)

the new api ability would use the same underlying code. the ajax/json used
by the gui is not documented as an api.

the gui polls for new data. That is the simplest way to do this for the api.

if you want more realtime output, this requires the server to keep
connection open and the thread running

[gschueler]

(by Noah Campbell at 2011-05-20T15:20:59-07:00)

At some point yeah. Given the request/response nature of the webui, you'd have to have a way of changing to some stream mechanism, i.e. websockets. All in due time :)

[gschueler]

(by Moses Lei at 2011-05-20T16:05:48-07:00)

Aren't we thinking a little too complicated here? We don't need bi-di or websockets... I would imagine it would work like this:

Client sends GET or POST with these items: execution ID and either 1) log line to start from or 2) timestamp to start from, or indicate it should receive the log starting from "now". At that point the client doesn't need to do anything else; if it loses the connection it just needs to re-request from the last content it got. (Could possibly use HTTP Content-Range headers for this.)

At that point the server just needs to send a response (probably text/plain in the same format that logs are written to disk), keep the thread open and not close it until Success or Failure or other job ending event. At the end of the output, server should send a trailer header field with the Success/Failure/Killed status of the job and then close the connection.

If the client asks for the live log, but the job has already completed, server should return the output from where the client requested it, send the trailer header, and close the connection.

I don't think polling for latest data on interval would really be acceptable at the command line-- the experience should be equivalent to your executing "ssh server command"; i.e. the expectation would be that the output would come back in real time. In other words, the experience should be the same as using --noqueue, but with the execution logged on the server.

(As a matter of fact I don't think the polling is really that great for the web UI either, but the expectations for a web interface are different from the expectations for a command line tool, so it's more acceptable there. One way I can think of off the top of my head to accomplish realtime output for the Web UI would be to do the same thing as above, but have the client request the log in an <iframe>, which could be kept in a loading state until the job is done.)

Are the log files kept on disk updated in close-to-realtime? If so, we could accomplish this just by sending a redirect to the log file location, and HTTP could accomplish this for us.

[gschueler]

(by Moses Lei at 2011-05-20T16:07:27-07:00)

but have the client request the log in an <iframe>

[gschueler]

(by Greg Schueler at 2012-06-22T10:38:40-07:00)

see #368

[gschueler]

Added this feature (cc @mlei ) to the run, dispatch, and rd-queue tools:

run [job selection] --follow
rd-queue follow -e ID
dispatch [node selection] --follow -- [command]

You can use -f/--follow to follow output. If you add -q/--quiet, then the tool will wait until the execution finishes, and will exit with appropriate status (0 for success, non-zero otherwise).

If you use -r/--progress, then it will only print progress indicator chars, such as # (for jobs with a known expected duration) or "." for unknown duration.