Today we are releasing some statistics around deployment of Pulse Connect Secure versions in the wild. The hope is that by releasing these statistics we can help to highlight the risk around outdated versions of PCS, which are being actively exploited by malicious actors.
We have also shared the raw data with national CIRTs and National Cyber Security Center’s so that vulnerable organizations can be informed.
A Recent Timeline on PCS Exploitation
On 31st March 2021, the Ivanti Product Security Team (PSIRT) released a KB article, documenting a new tool for its Pulse Connect Secure product. This tool, named the “Pulse Connect Secure Integrity Tool”, is designed to allow administrators to verify the integrity of system files on a PCS device. The idea being that if a malicious actor has compromised a PCS device (for example, through exploitation of known vulnerabilities), they will typically modify system files to add backdoor functionality, or introduce webshells to the device. By scanning the filesystem for “known-good” file hashes, the tool can identify if critical system files have been modified, or if new malicious files are introduced to the filesystem. This tool was initially released for public download, before being later being restricted to licensed users via the my.pulsesecure.net portal.
On April 15th, CISA, NSA and FBI released a joint advisory stating that Russian Foreign Intelligence services were exploiting vulnerabilities in a number of internet facing devices, including Pulse Connect Secure. The advisory specifically stated CVE-2019-11510, an arbitrary file read (which can lead to RCE), was being abused.
On April 16th, Ivanti PSIRT released a follow-up article stating that they were aware of targeted exploitation attempts against some PCS customers, to exploit vulnerabilities addressed in 2019, and 2020. They advised that customers should run the integrity checker tool and review logs for unusual authentication attempts.
Later, on April 20th Ivanti PSIRT released a further advisory stating that a new vulnerability had been discovered in PCS, which allowed for authentication bypass and RCE. This was being tracked as CVE-2021-22893. At the time of writing, the PSIRT team were still working on an official patch for the vulnerability, but did provide a workaround file, which could be used to block the URLs used in the attack. The advisory also confirmed that the vulnerability affects PCS R9.0R3, R9.1R1 (and higher). Version 8.3R7.1 is not affected by the vulnerability.
On the same day, FireEye also released a comprehensive blog post stating that they had recently responded to a number of cases involving exploitation of PCS vulnerabilities. During these investigations, the FireEye researchers identified that attackers were not only exploiting previously patched vulnerabilities from 2019 and 2020, but also the zero day being tracked as CVE-2021-22893. After exploiting PCS devices, the actors would deploy a number of different backdoors and webshells which would provide perpetual authentication bypass and code execution capabilities to the attacker. The FireEye post contains further technical analysis and detection capabilities for these backdoors.
Understanding the PCS Landscape
Whilst at the time of writing, no patch for CVE-2021-22893 has been released, it is clear that actors are still successfully compromising organisations through older, patched vulnerabilities. With this in mind, we wanted to understand the landscape around deployed PCS versions across the internet, and provide some statistics.
In order to identify the deployed versions, we needed to come up with a version identification method. Previously this has been possible by requesting specific files on PCS, which contain the version number. However we wanted to come up with a passive method that could be used to identify the version using open sources such as Censys.
We started with a simple GET request to a PCS device. A request to / responds with a 302 redirect to /dana-na/auth/url_default/welcome.cgi. In the HTTP response from welcome.cgi we can see some links to static .js and .css files that contain what appears to be a SHA-256 hash:
<script src="/dana-na/css/ds_84f7ff2a650614312210f0bdd5c296cef9cb6e543f335151393ba5091b5411eb.js"></script>
Checking various versions of PCS we noticed that this value changes between versions. This is enough to fingerprint PCS versions using the response to a (redirected) GET request to the PCS home page. However, we still needed to map these hashes to specific PCS versions.
Going back to the Ivanti integrity checker tool (which is provided in encrypted form), we decrypted the .pkg file and found something very useful – a list of file hashes for every version of PCS supported by the tool:
➜ ls ps-pcs-sa-genericV2-b12255-integritycheck-package/installer/bom_files
package-bom-sha256-PCS-8.3.7.1-65025.6.txt package-bom-sha256-PCS-9.1.10.2-12179.1.txt package-bom-sha256-PCS-9.1.12-11727.1.txt package-bom-sha256-PCS-9.1.4.1-4967.1.txt package-bom-sha256-PCS-9.1.6-5801.1.txt package-bom-sha256-PCS-9.1.8.2-8511.1.txt package-bom-sha256-PCS-9.1.9.2-12181.1.txt
package-bom-sha256-PCS-9.1.1-1505.1.txt package-bom-sha256-PCS-9.1.11-11161.1.txt package-bom-sha256-PCS-9.1.2-2331.1.txt package-bom-sha256-PCS-9.1.4.2-5035.1.txt package-bom-sha256-PCS-9.1.7-6567.1.txt package-bom-sha256-PCS-9.1.8.4-12177.1.txt
package-bom-sha256-PCS-9.1.1.10-3271.1.txt package-bom-sha256-PCS-9.1.11.1-11915.1.txt package-bom-sha256-PCS-9.1.3-3535.1.txt package-bom-sha256-PCS-9.1.4.3-5185.1.txt package-bom-sha256-PCS-9.1.8-7453.1.txt package-bom-sha256-PCS-9.1.9-9189.1.txt
package-bom-sha256-PCS-9.1.10-10119.1.txt package-bom-sha256-PCS-9.1.11.3-12173.1.txt package-bom-sha256-PCS-9.1.4-4763.1.txt package-bom-sha256-PCS-9.1.5-5459.1.txt package-bom-sha256-PCS-9.1.8.1-7851.1.txt package-bom-sha256-PCS-9.1.9.1-9701.1.txt
➜ head ps-pcs-sa-genericV2-b12255-integritycheck-package/installer/bom_files/package-bom-sha256-PCS-8.3.7.1-65025.6.txt
1eedb1848e4dc293096ff61ba12a83dd4c876db8955e257d43e3bc91bdd0b357 ./boot/boot.b
19941514ca0be67f1dc9539015b9a3196db0595fc33003ef51f382ca13e5e390 ./boot/VERSIONThis included file names for the static .js and .css files, allowing us to map the file names to specific versions:
➜ grep -r 84f7ff2a650614312210f0bdd5c296cef9cb6e543f335151393ba5091b5411eb ps-pcs-sa-genericV2-b12255-integritycheck-package/installer/bom_files|head -n 2
ps-pcs-sa-genericV2-b12255-integritycheck-package/installer/bom_files/package-bom-sha256-PCS-9.1.8-7453.1.txt:eb17c096978a4053e620d9f1e644e5751f58096309fd6f90bf08f3ca44a1fdb0 ./root/home/webserver/htdocs/dana-na/themes/meeting/ds_nav_84f7ff2a650614312210f0bdd5c296cef9cb6e543f335151393ba5091b5411eb.css
ps-pcs-sa-genericV2-b12255-integritycheck-package/installer/bom_files/package-bom-sha256-PCS-9.1.8-7453.1.txt:4b43ea1cc9b963b978c719c5086c7495e06443a758c5002bb23b9093b95a6b44 ./root/home/webserver/htdocs/dana-na/themes/meeting/ds_ie_84f7ff2a650614312210f0bdd5c296cef9cb6e543f335151393ba5091b5411eb.cssAs can be seen in the above output, taking the hex string from the .js file returned from a GET request to welcome.cgi, we can then look this up in the integrity checker database to reveal the version (in this case, 9.1R8 build 7453).
This allowed us to build a database of known file names to PCS versions, and collect some statistics on versions deployed on the internet.
Statistics
Of note, we discovered that:
- 21047 PCS servers were analysed
- The most deployed version of PCS is 8.3R7.1 (build 65025)
- The second most deployed version of PCS is vulnerable to CVE-2020-8243 and CVE-2020-8260. Both of which are post-authentication RCEs
- 7.7% were running a version vulnerable to CVE-2019-11510, an unauthenticated file-read that allows credential dumping and RCE
- 42% were running a version vulnerable to CVE-2020-8243
- 53% were running a version vulnerable to CVE-2020-8260
- The largest number of vulnerable devices were located in the United States
- 16% of servers were running an EOL version of PCS. EOL versions do not receive patches and thus may also be exploitable to known CVEs. They will also not receive security updates in the future
- 13% were running a version older than 8.3R7.1
