What CISOs need to know about the “3CX” software supply chain attacks

You may have seen news about a 3CX supply chain attack and quite a significant amount of early and sometimes confusing noise around what happened. Here is what we know about the 3CX supply chain attack, as of April 3, 2023. (The Stack has structured the below as a Q&A for an easy digest.)

What is 3CX?

3CX is a business communications provider that offers internet-based (VoIP) call, video and live chat services. It claims 12 million users in 190 countries.  Customers including a raft of blue chips like Avira, Chevron, CocaCola, Ikea, McDonalds, Mercedes Benz, the NHS, Schlumberger and numerous other well known brands.

Did 3CX get hacked?

Yes. 3CX has now confirmed a compromise. CEO Nick Galea said on March 30: “Early this morning we informed our partners and customers that our electron windows app shipped in Update 7, version numbers 18.12.407 & 18.12.416, included a severe security issue. We since learned that Electron Mac App version numbers 18.11.1213 shipped with Update 6, and 18.12.402, 18.12.407 & 18.12.416 in Update 7 have also been affected.

When did the 3CX supply chain attack start?

SentinelOne telemetry now sets the earliest infection attempt as March 8, 2023. Based on data recovered from GitHub, infrastructure used by the Windows variant was activated on December 7, 2022 and domains and web infrastructure used in the attacks were registered as early as November 2022, according to Volexity.

Was this a supply chain attack?

Yes. Customers and endpoint detection providers have detected malicious activity stemming from legitimate and signed software from 3CX. There is no evidence that this was the result of an upstream open source poisoning that also affected other software providers, despite earlier suggestions by 3CX CEO Nick Galea on a community forum that “this happened because of an upstream library we use became infected.”

So what’s the FFMPEG noise I’m hearing?

The company initially suggested that an FFmpeg library was compromised. FFmpeg (an open source provider of a suite of libraries and programs for handling multimedia files) has denied these claims. 3CX CISO Pierre Jordan added in March 30 blog: “The issue appears to be one of the bundled libraries that we compiled into the Windows Electron App via GIT” (without adding any further details.)

How did they get hit then?

We’re still awaiting an incident report.

Security firm ReversingLabs noted that the malicious FFmpeg files were signed with a legitimate certificate issued to 3CX. “Our analysis of the malicious update points either to a compromise of the 3CX development pipeline that resulted in malicious code being added during the build, or the possibility of a malicious dependency being served by a package repository,” adding the incident was likely caused by “the compromise of the repository from which the Electron application binaries were fetched during the build process.”

The short version: A 3CX repository likely was poorly protected and got compromised, allowing the attackers to bake malicious capabilities into its actual software that they could then use to attack customers downstream.

Should CISOs care about the upstream minutiae of this incident?

Yes. As ReversingLabs notes, “the manner in which the malicious code was added to the 3CXDesktopApp is of little value to 3CX customers who downloaded malicious code onto internal systems. However, it should be of intense interest to organizations engaged in software development, as it points to the need for increased scrutiny of compiled software images to detect malicious code, unexplained modifications or other discrepancies that may be a critical, early indicator of a supply chain compromise.”

OK, what can I do to secure MY build processes?

The Stack highly recommends reviewing the Secure Supply Chain Consumption Framework (S2C2F) for detailed guidance, including straightforward steps/pre-assessment guidance  to get IT leaders comfortable engaging with developers and engineers to inquire about their existing tools, capabilities, and workflows.

Does the incident have a CVE?

Yes. It has been allocated CVE-2023-29059

What does the malware do?

CrowdStrike, Palo Alto Networks, and SentinelOne were both among those to detect the attacks earlier and per the former, saw the compromised 3CXDesktopApp beaconing to actor-controlled infrastructure, deploy second-stage payloads, and, “in a small number of cases,[result in]  hands-on-keyboard activity.” Yikes.

More detail, please.

Well, it’s an “interesting multi-stage attack chain” per SentinelOne: “The 3CXDesktopApp application serves as a shellcode loader with shellcode executed from heap space. The shellcode reflectively loads a DLL, removing the “MZ” at the start. That DLL is in turn called via a named export ‘DllGetClassObject’ This stage will in turn download icon files from a dedicated Github repository... These ICO files have Base64 data appended at the end.

"That data is then decoded and used to download another stage. At this time, the DLL appears to be a previously unknown infostealer meant to interface with browser data, likely in an attempt to enable future operations as the attackers sift through the mass of infected downstream customers. The final stage implements infostealer functionality, including gathering system information and browser information from Chrome, Edge, Brave, and Firefox browsers. That includes querying browsing history and data from the Places table for Firefox-based browsers and the History table for Chrome-based browsers” the company adds.

How many 3CX customers have been hit as a result?

We don’t know and may never. Shodan searches the week the incident became public showed that there were 242,519 publicly exposed 3CX phone management systems online. One security vendor alone, Huntress, has sent out 2,783 incident reports where the 3CXDesktopApp.exe binary matches known malicious hashes and was signed by 3CX on March 13, 2023. That’s over a third of its pool of ~8,000 hosts running 3CX. The affected company has 600,000 customers. If Huntress’s sample of 34% of its pool running infected versions is a proxy for overall numbers, that could mean over 200,000 companies hit, but right now that is highly speculative.

How has 3CX handled this incident?

Badly, initially, in our view.

For at least a week prior to the event being made public, users had been warning that their endpoint protection software was triggering alerts about 3CX behaviour. One partner posted on a support forum on March 22 that SentinelOne was detecting “post-exploitation penetration framework or shellcode; indirect command was executed; code injection to other process memory space during the target process' initialization – but was told by at least one member of support staff on a community forum that “we have no control over their [customers’ antivirus] software and the decisions it makes so it's not exactly our place to comment…”

See also: Securing containers at scale: From runtime to remediation

One customer posted on Friday that “I have been in their forums all day providing assistance and have had hundreds contact me directly thanking me for help and the minute I challenge them for a Post Incident Report or ask if they have engaged security consultants, banned and told to f*ck off” — sharing an email from 3CX CEO Nick Galea that suggested they “please find another product and leave us to try and fix this matter”.

The company has since hired Mandiant and promised more detail on the incident, but security vendors and the security research community have been way ahead of it in terms of sharing IOCs, warning customers etc.

What can I do to build resilience to future supply chain attacks?

Per Jonathan Hencinski, VP, Security Operations, Expel, “there are some long term best-practices security teams can employ to stay ahead of future attacks (because let’s face it, they probably aren’t going away).

1. Know what “supply chain” means to you: “Supply chain” means different things to different orgs, but for many tech companies your supply chain is a long list of cloud-based services and applications that facilitate your day-to-day business. Figure out who or what is on that list, ASAP.
2. Make a backup plan: Plan for alternative providers should things go south. This isn’t to say you need backups for all your cloud services, but at least have a contingency plan for potentially rapid provider shifts in the event of a catastrophe.
3. Prioritize asset management: When you learn about a compromised major vendor or software repository, you must be able to effectively answer, “Are we impacted?” And answer it quickly.
4. Be creative and prepare for the worst: It’s not easy to dream up attacks like SolarWinds, or now 3CX. Plan tabletops and ask people around your company the hard questions, like: “What’s the worst thing that could happen?” Then develop incident response (IR) plans accordingly.

Can I get some IOCs?

Do contact your security partners. But per SentinelOne meanwhile:

Additional Mac Indicators

Follow The Stack on LinkedIn