ssh2john.py doesn't extract all keys from a keyfile

[mbrownnycnyc]

Hello folks,

I'm having an issue where I can't crack a format=ssh rsa key... in fact, it's one grabbed from the vuln machine from basic pentesting room on tryhackme.

I'm running kali on WSL2 on Windows 10.

┌──(kali㉿DESKTOP-3TJKPJG)-[~/tryhackme/01_basic_pentesting]
└─$ uname -a
Linux DESKTOP-3TJKPJG 5.10.16.3-microsoft-standard-WSL2 #1 SMP Fri Apr 2 22:23:49 UTC 2021 x86_64 GNU/Linux

I've built the latest from the repo:

┌──(kali㉿DESKTOP-3TJKPJG)-[~/tryhackme/01_basic_pentesting]
└─$ /home/kali/john_from_github/john/run/john --list=build-info
Version: 1.9.0-jumbo-1+bleeding-c470f5d8b 2021-12-28 11:55:38 +0100
Build: linux-gnu 64-bit x86_64 AVX2 AC OMP
SIMD: AVX2, interleaving: MD4:3 MD5:3 SHA1:1 SHA256:1 SHA512:1
CPU tests: AVX2
$JOHN is /home/kali/john_from_github/john/run/
Format interface version: 14
Max. number of reported tunable costs: 4
Rec file version: REC4
Charset file version: CHR3
CHARSET_MIN: 1 (0x01)
CHARSET_MAX: 255 (0xff)
CHARSET_LENGTH: 24
SALT_HASH_SIZE: 1048576
SINGLE_IDX_MAX: 32768
SINGLE_BUF_MAX: 4294967295
Effective limit: Max. KPC 32768
Max. Markov mode level: 400
Max. Markov mode password length: 30
gcc version: 11.2.0
GNU libc version: 2.32 (loaded: 2.32)
Crypto library: OpenSSL
OpenSSL library version: 0101010cf
OpenSSL 1.1.1l  24 Aug 2021
GMP library version: 6.2.1
File locking: fcntl()
fseek(): fseek
ftell(): ftell
fopen(): fopen
memmem(): System's
times(2) sysconf(_SC_CLK_TCK) is 100
Using times(2) for timers, resolution 10 ms
HR timer: clock_gettime(), latency 100 ns
Total physical host memory: 12717 MiB
Available physical host memory: 12357 MiB
Terminal locale string: en_US.UTF-8
Parsed terminal locale: UTF-8

I've verified that the passphrase is beeswax, which is located in rockyou, by using the key for auth with ssh -i.

However, i can't crack using john as follows:

┌──(kali㉿DESKTOP-3TJKPJG)-[~/tryhackme/01_basic_pentesting]
└─$ /home/kali/john_from_github/john/run/ssh2john.py ./linpeas/kay_id_rsa > ./john/kay_id_rsa_john_hash

┌──(kali㉿DESKTOP-3TJKPJG)-[~/tryhackme/01_basic_pentesting]
└─$ /home/kali/john_from_github/john/run/john --log-stderr ./john/kay_id_rsa_john_hash --wordlist=/usr/share/wordlists/rockyou.txt
Using default input encoding: UTF-8
0:00:00:00 Starting a new session
0:00:00:00 Loaded a total of 1 password hash
Loaded 1 password hash (SSH, SSH private key [RSA/DSA/EC/OPENSSH 32/64])
0:00:00:00 Cost 1 (KDF/cipher [0=MD5/AES 1=MD5/3DES 2=Bcrypt/AES]) is 0 for all loaded hashes
Cost 1 (KDF/cipher [0=MD5/AES 1=MD5/3DES 2=Bcrypt/AES]) is 0 for all loaded hashes
0:00:00:00 Cost 2 (iteration count) is 1 for all loaded hashes
Cost 2 (iteration count) is 1 for all loaded hashes
Will run 4 OpenMP threads
0:00:00:00 Command line: /home/kali/john_from_github/john/run/john --log-stderr ./john/kay_id_rsa_john_hash --wordlist=/usr/share/wordlists/rockyou.txt
0:00:00:00 - UTF-8 input encoding enabled
0:00:00:00 - Passwords will be stored UTF-8 encoded in .pot file
0:00:00:00 Autotune found best speed at OMP scale of 1
0:00:00:00 - Hash type: SSH, SSH private key (min-len 0, max-len 10 [worst case UTF-8] to 32 [ASCII])
0:00:00:00 - Algorithm: RSA/DSA/EC/OPENSSH 32/64
0:00:00:00 - Will reject candidates longer than 32 bytes
0:00:00:00 - Candidate passwords will be buffered and tried in chunks of 32
0:00:00:00 Proceeding with wordlist mode
0:00:00:00 - Wordlist file: /usr/share/wordlists/rockyou.txt
0:00:00:00 - memory mapping wordlist (139921507 bytes)
0:00:00:00 - No word mangling rules
Press 'q' or Ctrl-C to abort, almost any other key for status
0:00:00:00 - No stacked rules
0g 0:00:00:06 DONE (2022-01-09 22:30) 0g/s 2324Kp/s 2324Kc/s 2324KC/sa6_123..�*�7¡Vamos!�
0:00:00:06 Session completed
Session completed.

I came across an old issue and none of the suggestions (like arg order, or using single versus double arg declarations) solve this issue. Things seem to work fine... the wordlist is loading, john recognizes the format of the target, etc.

I feel like this is a bug, however, if I should post to the userlist let me know.

Thanks

kay_id_rsa.txt
kay_id_rsa_john_hash.txt

[magnumripper]

That keyfile holds four keys. The last two are identical. If I save them to separate files, the first one isn't recognized by john, the third (and fourth) are cracked as beeswax and the second remains uncracked.

[magnumripper]

Also, diffing the second keyfile against the third, they're identical except the second is missing 15 lines. So I take it that key is just damaged and thus uncrackable. The first file appears to be the same key as well, with less linefeeds.

I recall ssh2john has problems reading multiple keys from a single file. Don't we have an issue for that?

[magnumripper]

I confirmed that when given the OP file, ssh2john only extracts the second (damaged) key and silently ignores the others

[mbrownnycnyc]

Thank you. I'll work out extracting all keys and validating with openssh before feeding to ssh2john.

If there's a separate bug, is this slated for future release? To support multiple key extractions "automatically" would be a useful feature.

[mbrownnycnyc]

@magnumripper

In short, I realize what I did. I was using various methods to extract the key from linpeas output, and continuously appended keys. This was my fault, and if a output a single key, john works without issue. Although the feature of parsing many keys and subsequently cracking them would be useful, this is a PEBCAK bug that's been resolved.

Thank you!

[magnumripper]

Thanks for reporting back. Still, we should fix ssh2john so it doesn't just extract the first valid key - at least not without even warning. So I'll reopen this issue for that (I can't find any old issue... perhaps we discussed it at some point but forgot opening one).

[pradkrish]

Hello, is this issue still open? I am new to the project and willing to give this one a go, happy to get some guidance.

[solardiz]

@pradkrish Yes, the issue is still open as per @magnumripper's comment above, and we'd appreciate contributions to the project. Please feel free to work on this one - which I guess means implementing support for processing many keys at once or at least printing a warning when multiple keys are seen.

We also have many other open issues where we'd appreciate help.

[pradkrish]

Thanks. I would like to discuss the solution before beginning to work on it.

I am assuming we need to create one hash per key. After emitting a warning, shall we prompt the user to provide multiple hash filenames so we can save one hash per file. Or, If the user has provided only one hash file, we create as many hash files as the number of keys? Do you have any suggestions?

[solardiz]

We generally output the hashes to stdout - not explicitly to files - and this implies just outputting all hashes in one stream. The user of the script will generally have redirected stdout to a file. If so, they'll get all the hashes in one file, one hash per line. That's fine.

Also, if we do correctly process multiple keys and output multiple hashes, there's no need to print any warning - I only mentioned printing a warning if we somehow did not do the full thing right.

[pradkrish]

is it safe to assume that the key file may contain multiple key types, that is, one could be RSA, another could be DSA etc.?

[solardiz]

@pradkrish Yes, I guess we should support arbitrary combinations of key types.

FWIW, @mbrownnycnyc's kay_id_rsa.txt attached above contains multiple RSA keys (only RSA), but somehow the first one has missing linefeeds - maybe is just broken in that way, and shouldn't be correctly processed by us, but maybe we should then print warnings about keys/lines that look wrong and could not be processed.

[magnumripper]

Perhaps it'd be a good idea to prefix the output hash(es) with a virtual login field? For a single hash output, it could be just the input filename as in kay_id_rsa.txt and for 2nd hash on it could be suffixed by an index such as kay_id_rsa.txt_2 and so on.

[solardiz]

Prefixing with filename-index: makes sense to me. Should also avoid/replace any : embedded in the filename first, like we already do in some other scripts.

[pradkrish]

I am also refactoring the file ssh2john while working on this bug. To ensure I am not changing any behaviour, I would like to run some tests while refactoring. I could be wrong here but I don't see any tests specific to this function, is that the case?

[solardiz]

Please keep any refactoring in its own commit(s), separate from functional changes.

Unfortunately, we don't automatically test the *2john tools in our current CI setup (something you can contribute later). For now, you'll need to manually test the tool before vs. after your changes using sample inputs from https://github.com/openwall/john-samples/tree/main/SSH and I guess also from @mbrownnycnyc's original comment here.