Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Machine readable (json) output of --version #371

Closed
buhtz opened this issue Sep 7, 2022 · 10 comments
Closed

Machine readable (json) output of --version #371

buhtz opened this issue Sep 7, 2022 · 10 comments
Assignees
Labels
enhancement New feature or request

Comments

@buhtz
Copy link

buhtz commented Sep 7, 2022

Doing rsync --version displays on stdout

rsync  version 3.2.3  protocol version 31
Copyright (C) 1996-2020 by Andrew Tridgell, Wayne Davison, and others.
Web site: https://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, hardlink-specials, symlinks, IPv6, atimes,
    batchfiles, inplace, append, ACLs, xattrs, optional protect-args, iconv,
    symtimes, prealloc, stop-at, no crtimes
Optimizations:
    SIMD, asm, openssl-crypto
Checksum list:
    xxh128 xxh3 xxh64 (xxhash) md5 md4 none
Compress list:
    zstd lz4 zlibx zlib none

This is human- but not machine-readable. Parsing this is possible but not elegant. Would be nice to have a --json to print out something like this

{
    "version": "3.2.3",
    "protocol": "31",
    "Capabilities": [
        " 64-bit files", "64-bit inums", "64-bit timestamps"
    ],
    "Compress list": [
        "zstd", "lz4", "zlibx", "zlib", "none"
    }
}
@WayneD WayneD added the enhancement New feature or request label Sep 9, 2022
@WayneD WayneD self-assigned this Sep 9, 2022
@WayneD
Copy link
Member

WayneD commented Sep 9, 2022

Seems useful. I've done something quick in rsync patches to be refined for future inclusion.

@buhtz
Copy link
Author

buhtz commented Sep 9, 2022

Great to hear. Can you please show me an example output of it?

Or do you have a nightly build or a rep I can build from without adding patch diff filesß

@WayneD
Copy link
Member

WayneD commented Sep 10, 2022

$ ./rsync -VV

{
  "program": "rsync",
  "version": "3.2.6-10-gad53a9b5",
  "protocol": "31.0",
  "copyright": "(C) 1996-2022 by Andrew Tridgell, Wayne Davison, and others.",
  "url": "https://rsync.samba.org/",
  "capabilities": {
    "file_bits": 64, "inum_bits": 64, "timestamp_bits": 64,
    "long_int_bits": 64, "socketpairs": true, "symlinks": true,
    "symtimes": true, "hardlinks": true, "hardlink_specials": true,
    "hardlink_symlinks": true, "IPv6": true, "atimes": true,
    "batchfiles": true, "inplace": true, "append": true, "ACLs": true,
    "xattrs": true, "secluded_args": "optional", "iconv": true,
    "prealloc": true, "stop_at": true, "crtimes": false
  },
  "optimizations": {
    "SIMD_roll": true, "asm_roll": true, "openssl_crypto": true,
    "asm_MD5": false
  },
  "checksum_list": [
    "xxh128", "xxh3", "xxh64", "md5", "md4", "sha1", "sha256", "sha512", "none"
  ],
  "compress_list": [
    "zstd", "lz4", "zlibx", "zlib", "none"
  ],
  "daemon_auth_list": [
    "sha512", "sha256", "sha1", "md5", "md4"
  ],
  "license": "GPL3",
  "caveat": "rsync comes with ABSOLUTELY NO WARRANTY"
}

@buhtz
Copy link
Author

buhtz commented Sep 10, 2022

Thank you so much.

@WayneD
Copy link
Member

WayneD commented Sep 10, 2022

FYI, this will be in 3.2.7pre pretty soon, so it will be in the github builds and in the rsync binaries dirs soon.

@WayneD
Copy link
Member

WayneD commented Sep 10, 2022

In the latest git, see the script support/json-rsync-version as a way to get the json data from older rsyncs. Only tested back through 3.0.6 at the moment.

@WayneD WayneD closed this as completed Sep 10, 2022
@WayneD
Copy link
Member

WayneD commented Sep 10, 2022

FYI, I changed capabilities & optimizations into dictionaries so that someone doesn't need to scan a list for "no crtimes" vs "crtimes". That's now "crtimes": true|false

@buhtz
Copy link
Author

buhtz commented Sep 17, 2022

Just a sidenote:

Version 3.2.3 (Debian stable) still overs a -VV which return the same content as -V.
Version 3.1.3: does not offer a -VV

Makes a bit hard to detect the json-supporting version of rsync because I can not be sure that -VV gives me a json output. ;)

@WayneD
Copy link
Member

WayneD commented Sep 19, 2022

The output is a lot longer when -VV is a syntax error (and is on stderr). If the output doesn't start with a { then it's not the json.

The python script is provided as a way to get the json output no matter what the rsync version is. Install the script somewhere and run the equivalent of support/json-rsync-version /usr/bin/rsync (or pipe the --version --version output of an rsync into it) and you'll always get json (assuming that python3 is available on the host).

@buhtz
Copy link
Author

buhtz commented Sep 19, 2022

The output is a lot longer when -VV is a syntax error (and is on stderr). If the output doesn't start with a { then it's not the json.

Yes I know. And I have to add checks for that in my code now. And I also found out that -V is not provided by 3.1.3
In the end the behaviour of -V , -VV is very different between the rsync versions.

What I know so far now

  • rsync >= 3.2.6: -VV return a json
  • rsync <= 3.2.5 and > (somewhere near) 3.1.3: -VV return the same as -V
  • rsync <= (somewhere near) 3.1.3: -VV doesn't exists
  • rsync == 3.1.3 (Ubuntu 20 LTS) doesn't even know '-V'

The python script is provided

But I won't ship my application with such a script but implement that in my own code.. At the end I have to parse the output. In my current code I try to extract the rsync version in different ways.

  1. Use -VV and parse the output for an error message (-VV not exist)
    1. ...parse the output as json.
    2. ...parse the output via regex for the version string.
  2. If -VV is unknown try a usual --version and parse the output via regex for a version string.

Please feel free to use my code in replacement of your script. See _get_extern_version() and that section in collect_diagnostics() calling that function twice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants