Munawar Hafiz, CEO of OpenRefactory, writes about the benefits of proactive vulnerability management strategies, including the potential advantage of knowing about bugs before they become public CVEs. Edited by Charlie Bedard.
Three Vulnerabilities with Different Impacts
We encounter security vulnerabilities every day. Some we are able to dodge while others leave a major impact. We’ll look at three relatively recent and well-known vulnerabilities.
- The XZ Utils Backdoor Vulnerability (CVE-2024-3094) We were saved by just a few days from the XZ Utils Backdoor vulnerability having a major impact. Fortunately, the flawed XZ Utils code did not reach production Linux distributions. The problem primarily affected beta versions of Fedora, Debian, openSUSE, and Ubuntu. Had it reached a production Linux distribution, which it was destined to do in a few days, the impact would have been much more pronounced.
- The OpenSSH regreSSHion Vulnerability (CVE-2024-6387) We were not that lucky with the OpenSSH regreSSHion vulnerability. Because this was in OpenSSH, which is ubiquitous, this impacted all x86 machines. Despite the widespread need to update and patch OpenSSH, the exploitation of this vulnerability required extensive time and multiple connections, offering a longer window to address the issue compared to more urgent threats.
- The Log4Shell Vulnerability (CVE-2021-44228) In contrast, the Log4Shell vulnerability, discovered in December 2021 in the Apache Log4j library, had a profound impact on the cybersecurity landscape. This remote code execution flaw led to widespread exploitation, forcing organizations into frantic patching and causing significant economic and operational disruptions.
Understanding the Expanded Attack Surface
Modern software development heavily relies on open-source code, with 97% of applications utilizing these components. While open-source code accelerates development and innovation, it also vastly increases the attack surface, making systems more vulnerable. Open-source components, often maintained by small teams or individuals, may lack rigorous security audits and timely updates, complicating the security landscape.
The combined attack surface now includes both proprietary code and numerous open-source components, creating a complex environment for potential exploitation.
The Life of a Security Practitioner
Security practitioners face daily challenges in managing this expanded attack surface:
- Many Targets: Like the turtles going all the way down, open source libraries themselves depend on many other open source libraries. You need to measure and mitigate risks of all the dependencies, tens of thousands of them.
- Patch Management: If you are a security practitioner, you wake up every morning fearing the worst. You look at your Software Composition Analysis (SCA) tool dashboard and check if there is a dumpster fire that has started and needs your immediate attention. You have to stay informed about the latest security patches and updates released by the open-source community and promptly apply them to mitigate known vulnerabilities.
- Incident Response Planning: You need to constantly be developing and maintaining robust incident response plans to quickly address and mitigate the impact of any security incidents involving open-source components.
- Open Source Evaluation: When your team is considering using an open source component and you do not have a separate open source program office, often the task comes to you to evaluate the long term risk of depending on that software.
All of these tasks are a race against the clock. When a CVE is public, all SCA tools pick that up and it finds its way onto your dashboard. By that time, a patch may be available, but active exploits may also be rampant. The race is to patch your code before the attackers get here.
By the time the CVE for Log4Shell was public on December 10, 2021, not only was the exploit code available, but also active attack campaigns had been underway since November 20, 2021. That is why there was so much panic and so much cost involved with that vulnerability.
In the case of regreSSion, we seem to have more time on our hands. But we do not know if an alternative way has been found to exploit the vulnerability faster. That would be a game changer and we would have to catch up.
Before There Were Vulnerabilities, There Were Bugs
In two of the cases, Log4Shell and regreSSion, the bug in the code had been there for a long time before it was found. Log4Shell was discovered by the attackers first and we knew about it while playing catch up. The root cause was a feature that was wrongfully added into Log4j several years ago.
In the case of regreSSHion, the problem in the code had been there since October 2020 and it was only found in June 2024.
Many such unknown bugs are lurking in open source software. There are several steps before a privately reported bug becomes a publicly known vulnerability with a CVE:
-
- Initially, a security researcher or an end user discovers the bug and confidentially reports it to the maintainers.
- The maintainers then verify the validity of the issue and assess its severity.
- Maintainers work to develop a patch or workaround, balancing the urgency of the fix with the complexity of the code.
- Once a solution is ready, they coordinate with the CVE Numbering Authority (CNA) to assign a CVE identifier.
- The patch and associated CVE details are typically released simultaneously to allow users to mitigate the vulnerability promptly.
This process can take days to weeks, depending upon the responsiveness of the maintainers, the complexity of the bug, and the coordination required for a secure release. The goal is to ensure that fixes are available before detailed vulnerability information is made public, minimizing the window of opportunity for attackers to exploit the flaw.
The Benefits of Knowing the Future
What if we can take advantage of the time between an actual bug report and the CVE issuance?
The average time is about 4 months. That is a lot. Imagine that you know about a bug in your open source supply chain at the same time it is reported to the maintainer of the open source software. Also imagine that you have a fix available for the bug. This foresight allows for:
-
- Private Fork Creation: If you assess that the bug is critical, you have to determine if it reaches your code and can impact your attack surface. If there is an impact, you can create a private fork of the open source code and fix that bug in your fork. You use the private fork as your dependency instead of the public release. Later, when the CVE is public and an updated release is available you do not have to rush to a patch. You have built a moat. You can assess the risk in that updated release following a rigorous process and only then can switch to that.
- Engineering Resource Allocation: If the bug is critical and it impacts the attack surface but a fix is not available, you may spend engineering resources to create a fix. And then you create a private fork just like Action 1.
- Monitoring and Adoption: If the bug is critical and it impacts the attack surface but a fix is not available, you may alternatively monitor the progress of the bug fix and wait for the fix to be available. As soon as the fix is available, you adopt it. Because you have known about the problems, at least, the news does not catch you by surprise. You still have a good shot at mitigating the risk.
- Assessment of Benign Bugs: If you assess that the bug is benign and/or it does not impact you, you can ignore the information and carry on.
For Actions 1 and 2, proactively addressing these bugs can provide an average of four additional months to mitigate risks effectively.
The Clean Beach Project is opening up the opportunity to proactively respond in that time window before a CVE is public. This will reduce a lot of the chaos in the life of security practitioners and significantly reduce the risk of operating in a world where open source dependencies are inevitable. Stay tuned for more.
The Clean Beach Project has been incubated and supported through grants from the Alpha-Omega project.
