Critical bug in ubiquitous Java framework sets off an internet cluster bomb
AWS, Red Hat, VMware, more affected with pre-auth RCE exploits circulating
First published Friday Dec 10, 20:28 BST. Updated multiple times since, incl. Monday Dec 13, 13:00 BST. Updated again Wednesday Dec 15 to confirm Log4j 2.16.0 is out and completely disables JNDI by default. DNote: A new vuln in Log4j 2.15.0 allocated CVE-2021-45046 has had its CVSS boosted from 3.7 to 9.0 after RCE was demonstrated.
A critical vulnerability in Log4j -- one of the most widely used logging frameworks in the entire Java ecosystem -- exposes swathes of popular software applications to easy exploitation, security experts warned on Friday.
Allocated CVE-2021-44228 and first reported by Alibaba Cloud Security team’s Chen Zhaojun, the vulnerability (dubbed "Log4Shell") is in widely used open source software Log4j. This, in turn, is present in many services as a dependency -- including enterprise applications and numerous cloud services, as the NCSC highlighted.
From AWS to Cisco, cloud providers to security software firms, IT teams were scrambling on Friday to identify if they were exposed, as the number of hosts actively exploiting the vulnerability climbed to over 150. Public exploit code (showing attackers how to use the vulnerability to access systems) circulated widely meanwhile.
The Dutch NCSC was tracking advisories from companies here.
The Swiss Gov't Computer Emergency Response Team was among those putting out lucid advisories.
French security researcher Nicolas Chatelain of TNP IT Security said the bug gave unauthenticated RCE as root on default installations of VMware's vCenter server -- as tested on 7.0.3, build 18778458 -- with VMware confirming multiple other impacted products including VMware Horizon and VMware Identity Manager. Thousands of other companies had either openly or more quietly admitted that they were exposed by Wednesday as attacks escalated.
Multiple products from companies as diverse as AWS, Splunk, Siemens and others were also exposed.
The critical Log4j vulnerability makes it possible for any attacker who can inject text into log messages or log message parameters into server logs that load code from a remote server; the targeted server then executes that code via calls to the Java Naming and Directory Interface (JNDI). As Sophos reported on Sunday: "JNDI interfaces with a number of network services, including the Lightweight Directory Access Protocol (LDAP), Domain Name Service (DNS), Java’s Remote Interface (RMI), and the Common Object Request Broker (CORBA)... Sophos has seen efforts to exploit LDAP, DNS and RMI, using a URL tagged to those services redirected to an external server."
One security expert, Greg Linares warned on Sunday that "based on what I've seen, there is evidence that a worm will be developed for this in the next 24 to 48 hours. Self propagating with the ability to stand up a self hosted server on compromised endpoints. In addition to spraying traffic, dropping files, it will have C2C.
"The Biggest hurdle appears to be implementing a JDK gadget to enable code execution on limited env. That is currently being researched by several groups," Greg Linares posted after reviewing attacker discussions.
https://twitter.com/Laughing_Mantis/status/1470168079137067008
Follow The Stack on LinkedIn
VMware noted in its advisory: "Organizations who do have perimeter security controls on their virtualization infrastructure management interfaces may still be in jeopardy. Ransomware gangs have repeatedly demonstrated to the world that they are able to compromise corporate networks while remaining extremely patient, waiting for a new vulnerability in order to attack from inside a network...Organizations may want to consider additional security controls and isolation between their IT infrastructure and other corporate networks..."
Cisco meanwhile was investigating the possibility that over 100 products were vulnerable. It is not alone. Atlassian said: "If a you have modified the default logging configuration to enable the JMS Appender functionality, remote code execution may be possible in the following products: Jira Server & Data Center; Confluence Server & Data Center; Bamboo Server & Data Center; Crowd Server & Data Center; Fisheye; Crucible.")
AWS said its cloud-based hardware security module CloudHSM was affected, as was Amazon OpenSearch. The former may require manual updates. The hyperscaler It added rules to improve detection and mitigation of risks arising from the issue (AWSManagedRulesKnownBadInputsRuleSet AMR in the AWS WAF service) for CloudFront, Application Load Balancer (ALB), API Gateway, and AppSyn customers late Friday.
The critical vulnerability in Log4j was first reported to Apache on November 24. An initial patch was easily bypassed and a secure/stable version (2.15.0) was issued December 9 but the horse had bolted. UPDATED Wednesday December 15: Log4j 2.15.0 also was discovered to be vulnerable, albeit less severely. Log4j 2.16.0 is out and completely disables JNDI by default. While this may cause some issues for many, it also mitigates risk.
Critical vulnerability in Log4j: Cluster bomb, cluster f***
Brian Fox, CTO of software security firm Sonatype said of the severity of the bug that "the combination of scope and potential impact here is unlike any previous component vulnerability I can readily recall."
(A good technical write-up of the vulnerability by Cloudflare CTO John Graham-Cumming is here.)
https://twitter.com/paradoxengine/status/1469396857273135107
In the case of Minecraft – one of those hit first -- attackers appear to have been able to get RCE on Minecraft Servers by simply pasting a short message into the chat box. Log4j is used on Management/Gateways by Postgresql among a host of other widely used software tools, including but not limited to Apache Struts2, Apache Solr, Apache Druid, Apache Flink and Apache Swift, the UK's NCSC noted, adding that "other large projects including Netty, MyBatis and the Spring Framework also make use of the library."
The agency said in its advisory: "If you are using the Log4j 2 library as a dependency within an application you have developed, ensure you update to version 2.15.0 or later. If you are using an affected third-party application, ensure you keep the product updated to the latest version. The flaw can also be mitigated in previous releases (2.10 and later) by setting system property 'log4j2.formatMsgNoLookups' to 'true' or removing the JndiLookup class from the classpath." Have pity on your IT team who saw a peaceful weekend ahead...
Multiple security firms and the community have rolled out helpful tools and tips as users scramble to patch.
These include Cybereason's "vaccine" (freely available on GitHub) which the security firm describes as "a relatively simple fix that requires only basic Java skills to implement and is freely available to any organization". (Bugcrowd CTO Casey Ellis noted that although helpful as a quick fix, the free tool's effectiveness is limited: "It does not work for versions prior to 2.10, requires a restart, and the exploit must fire properly in order to be effective, and even when it does run properly, it still leaves the vulnerable code in place.")
What's causing this critical Log4j vulnerability?
As per Apache's Log4j security guide: Apache Log4j2 <=2.14.1 JNDI features used in configuration, log messages, and parameters do not protect against attacker controlled LDAP and other JNDI related endpoints. An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers when message lookup substitution is enabled. From log4j 2.15.0, this behavior has been disabled by default. This issue was remediated in Log4J v2.15.0." (That this bug should exist at all left many howling.)
The Apache Logging Services team provided the following mitigation advice: "In previous releases (>=2.10) this behavior can be mitigated by setting system property "log4j2.formatMsgNoLookups" to “true” or by removing the JndiLookup class from the classpath (example: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class). Java 8u121 protects against RCE by defaulting "com.sun.jndi.rmi.object.trustURLCodebase" and "com.sun.jndi.cosnaming.object.trustURLCodebase" to "false".
"You can manually check for use of affected versions of Log4J by searching your project repository for Log4J use, which is often in a pom.xml file. Where possible, upgrade to Log4J version 2.15.0.
"If you are using Log4J v1 there is a migration guide available. Please note that Log4J v1 is End Of Life (EOL) and will not receive patches for this issue. Log4J v1 is also vulnerable to other RCE vectors and we recommend you migrate to Log4J 2.15.0 where possible. If upgrading is not possible, then ensure the -Dlog4j2.formatMsgNoLookups=true system property is set on both client- and server-side components."
Attackers were already dropping cryptominers as we published Friday. Worse will follow without prompt action. (Updated Monday December 13: Sophos has seen "scans for the vulnerability, exploit tests, and attempts to install coin miners. We have also seen attempts to extract information from services, including Amazon Web Services keys and other private data". According to Netlab 360 researchers, hackers are exploiting Log4Shell to install Mirai and Muhstik malware on vulnerable devices. The malware families install crypto miners and enable large-scale DDoS attacks. The attacks recorded by experts were directed at devices running Linux. Microsoft Threat Intelligence Center meanwhile says it has seen Log4shell also used to install Cobalt Strike beacons for upcoming malicious campaigns."
Hugops to all involved in getting fixes sorted.
