As the industry rallies around the 3CX attack and secures deployed 3CX electron App instances, the root cause is being ignored — a compromised software supply chain. For 3CX — there is a mismatch between the source code they ingested and the packaged code they created. The threat was left of shift-left and so should the detection. Not just 3CX but almost all software companies could be compromised this way. We discuss the way forward in this blog.
Runtime Threat detection companies have done a commendable job in detecting the 3CX software supply chain attack. The broader ecosystem has reacted well by containing the attack — with vendors adding detections, revoking certificates, and 3CX itself rapidly releasing a new version of its software.
3CX CISO has apologized. CEO Nick Galea has asked all customers to uninstall their own product. This software supply chain attack is directly impacting a company with 600,000 customers and 12 million users. A global software brand has just been taken advantage of.
However, much needs to be understood, even as researchers write intricate blogs about how the attack worked and how you can step-through the attack using advanced tools. The fog will lift eventually as 3CX will investigate and release details.
Just wanted to point out that this is a really good line! Great writing!
The attack was executed by a known Lazarus threat actor group with known malicious C2 servers being contacted. Since the actor group was recognizable, EDR systems were able to flag suspicious activity from these applications early on, which brought the breaches to light.
At the same time, a sobering implication has been made apparent: multiple layers of security tools have failed.
First, 3CX was comprehensively compromised — whatever security tools they had — application security tools, malware scans, and SCA tools all were bypassed.
Second, all threat prevention tools — endpoint threat prevention products did not prevent the malware as it was deployed on millions of endpoints. The silence from network players is interesting for an attack that was detected by malicious network activity.
After the fact, when the attack became common knowledge, every runtime security tool claimed to detect and prevent malicious 3CX electron app versions.
The table below lays out a chronology of 3CX attack timeline. Effectively, the best estimate available is that the effort to compromise started Dec 7, 2022. The first infection was in January 2023 and the first detection was on March 29, 2023.
On March 30, 2023, the universal multi-media toolkit, makers of ffmpeg.dll announced the following on their twitter account:
They effectively said that their source code is clean. 3CX build systems as they built the tampered versions were compromised. This is similar to SolarWinds, but the industry must wait for 3CX to explain the gory details. The net of this is that the source code is clean, but the built bits are compromised: there is a mismatch between the source code and the packaged code. The threat was left of shift-left and so should the detection. This is the reason the detection is post compromise and too late for many customers of 3CX. If the attackers never used known C2 servers, even EDR would not have raised a flag. This is the nature of modern supply chain attacks and runtime security tools cannot detect them.
Basically, the plumbing in place at 3CX — like every enterprise examined — could not attest to the integrity of their software. SCA tools and App Sec tools do not do this kind of attestation.
Establishing software supply chain integrity is hard — that is why we founded Lineaje.
Software supply chain integrity develops a chain of trust from your shipping product to the built components included in it to the source code they were built from — whether by you or in an open-source project. And then, integrity is established for each component you depend on, and so on and so forth, until every transitive component’s integrity is detailed.
Lineaje bases the supply chain integrity attestation on our proprietary and amazing fingerprinting technology. This enables us to create a software supply chain of trust with each link having its own integrity rating from a shipping product down to its last leaf component that has no further dependencies. Our fingerprinting-centric LCAL methodology has a significant advantage over build system-centric attestation approaches. Lineaje uses what is accessible, specifically package and source code. If you are using open-source software for example, those are accessible to you. So why not use them for attestation? If you modify those dependencies, you still have access to both enabling deep fingerprinting-based attestation.
Lineaje enables component attestations without dependency on any tools. This attestation is at a component level. A key thing to understand is that each attestation level is for the component itself, not for its dependencies. A highest-level component of attestation does not mean that its dependencies have as high an attestation as the parent. This is also true for SLSA and NIST and an inherent weakness of component-level attestation. This understanding is critical for supply chain security.
Last, but not the least, LCAL levels identify the weakest links in the software supply chain. With chained LCAL levels, Lineaje can attest that what you shipped is what you built, what you built is what you sourced, and what is sourced is what was published by your dependency, what was published is what was sourced by them, and so on and so forth all the way upstream to the leaf transitive dependency — a provable end to end attested software supply chain.
Lineaje’s flagship product SBOM360 auto-attests all your software against these software integrity levels.
A LCAL-3 violation would have triggered a mismatch between source code and a built package from it.
Attacks like the 3CX highlight the inherent weakness in the software supply chain. It’s not that 3CX was less prepared than other software development groups are, it’s just that they are unfortunate enough to be targeted first. Anyone could be vulnerable to this same scenario. You need to look at Supply Chain Integrity Management Software like SBOM360. If you are at RSA, look for us at Lineaje.com.