In the past 20 years, bug hunting has transformed from a hobby (or maybe even a felony) to a full-time profession for tens of thousands of talented software engineers around the globe. Thanks to the growth in private and public bug bounty programs, men and women with the talent can earn a good living by sniffing out flaws in the code for applications and – increasingly -physical devices that power the 21st century global economy. 

Asus ShadowHammer suggests Supply Chain Hacks are the New Normal

Bug Hunting Smart TVs To Supply Chain

What does that work look like and what platforms and technologies are drawing the attention of cutting edge vulnerability researchers? To find out we sat down with the independent researcher known as Sick Codes (@sickcodes). In recent months, he has gotten attention for a string of important discoveries. Among other things, he discovered flaws in Android smart television sets manufactured by the Chinese firm TCL and was part of the team, along with last week’s guest John Jackson, that worked to fix a serious server side request forgery flaw in a popular open source security module, NPM Private IP

Spotlight Podcast: How Machine Learning is revolutionizing Application Fuzzing

In this interview, Sick Codes and I talk about his path to becoming a vulnerability researcher, the paid and unpaid research he conducts looking for software flaws in common software and internet of things devices, some of the challenges and impediments that still exist in reporting vulnerabilities to corporations and what’s in the pipeline for 2021. 


As always,  you can check our full conversation in our latest Security Ledger podcast at Blubrry. You can also listen to it on iTunes and check us out on SoundCloudStitcherRadio Public and more. Also: if you enjoy this podcast, consider signing up to receive it in your email. Just point your web browser to securityledger.com/subscribe to get notified whenever a new podcast is posted. 

In this episode of the podcast (#200), sponsored by Digicert: John Jackson, founder of the group Sakura Samurai talks to us about his quest to make hacking groups cool again. Also: we talk with Avesta Hojjati of the firm Digicert about the challenge of managing a growing population of digital certificates and how  automation may be an answer.


Life for independent security researchers has changed a lot in the last 30 years. The modern information security industry grew out of pioneering work by groups like Boston-based L0pht Heavy Industries and the Cult of the Dead Cow, which began in Lubbock, Texas.

After operating for years in the shadows of the software industry and in legal limbo, by the turn of the millennium hackers were coming out of the shadows. And by the end of the first decade of the 21st century, they were free to pursue full fledged careers as bug hunters, with some earning hundreds of thousands of dollars a year through bug bounty programs that have proliferated in the last decade.

Despite that, a stigma still hangs over “hacking” in the mind of the public, law enforcement and policy makers. And, despite the growth of bug bounty programs, red teaming and other “hacking for hire” activities, plenty of blurry lines still separate legal security research from illegal hacking. 

Hacks Both Daring…and Legal

Still, the need for innovative and ethical security work in the public interest has never been greater. The Solar Winds hack exposed the ways in which even sophisticated firms like Microsoft and Google are vulnerable to compromised software supply chain attacks. Consider also the tsunami of “smart” Internet connected devices like cameras, television sets and appliances are working their way into homes and workplaces by the millions. 

Podcast Episode 112: what it takes to be a top bug hunter

John Jackson is the co -founder of Sakura Samurai, an independent security research group. 

What does a 21st century hacking crew look like? Our first guest this week is trying to find out. John Jackson (@johnjhacking) is an independent security researcher and the co-founder of a new hacking group, Sakura Samurai, which includes a diverse array of security pros ranging from a 15 year old Australian teen to Aubrey Cottle, aka @kirtaner, the founder of the group Anonymous. Their goal: to energize the world of ethical hacking with daring and attention getting discoveries that stay on the right side of the double yellow line.

Update: DHS Looking Into Cyber Risk from TCL Smart TVs

In this interview, John and I talk about his recent research including vulnerabilities he helped discover in smart television sets by the Chinese firm TCL, the open source security module Private IP and the United Nations. 

Can PKI Automation Head Off Chaos?

One of the lesser reported sub plots in the recent Solar Winds hack is the use of stolen or compromised digital certificates to facilitate compromises of victim networks and accounts. Stolen certificates played a part in the recent hack of Mimecast, as well as in an attack on employees of a prominent think tank, according to reporting by Reuters and others. 

Avesta Hojjati is the head of Research & Development at Digicert.

How is it that compromised digital certificates are falling into the hands of nation state actors? One reason may be that companies are managing more digital certificates than ever, but using old systems and processes to do so. The result: it is becoming easier and easier for expired or compromised certificates to fly under the radar. 

Our final guest this week, Avesta Hojjati, the  Head of R&D at DigiCert, Inc. thinks we’ve only seen the beginning of this problem. As more and more connected “things” begin to populate our homes and workplaces, certificate management is going to become a critical task – one that few consumers are prepared to handle.

Episode 175: Campaign Security lags. Also: securing Digital Identities in the age of the DeepFake

What’s the solution? Hojjati thinks more and better use of automation is a good place to start. In this conversation, Avesta and I talk about how digital transformation and the growth of the Internet of Things are raising the stakes for proper certificate management and why companies need to be thinking hard about how to scale their current certificate management processes to meet the challenges of the next decade. 


(*) Disclosure: This podcast was sponsored by Digicert. For more information on how Security Ledger works with its sponsors and sponsored content on Security Ledger, check out our About Security Ledger page on sponsorships and sponsor relations.

As always,  you can check our full conversation in our latest Security Ledger podcast at Blubrry. You can also listen to it on iTunes and check us out on SoundCloudStitcherRadio Public and more. Also: if you enjoy this podcast, consider signing up to receive it in your email. Just point your web browser to securityledger.com/subscribe to get notified whenever a new podcast is posted. 

A serious security flaw in a commonly used, but overlooked open source security module may be undermining the integrity of hundreds of thousands or even millions of private and public applications, putting untold numbers of organizations and data at risk.

A team of independent security researchers that includes application security professionals at Shutterstock and Squarespace identified the flaw in private-ip, a npm module first published in 2016 that enables applications to block request forgery attacks by filtering out attempts to access private IP4 addresses and other restricted IP4 address ranges, as defined by ARIN

The SSRF Blocker That Didn’t

The researchers identified a so-called Server Side Request Forgery (SSRF) vulnerability in commonly used versions of private-ip. The flaw allows malicious attackers to carry out SSRF attacks against a population of applications that may number in the hundreds of thousands or millions globally. It is just the latest incident to raise questions about the security of the “software supply chain,” as more and more organizations shift from monolithic to modular software application development built on a foundation of free and open source code.

Report: Cybercriminals target difficult-to-secure ERP systems with new attacks

According to an account by researcher John Jackson of Shutterstock, flaws in the private-ip code meant that the filtering allegedly carried out by the code was faulty. Specifically, independent security researchers reported being able to bypass protections and carry out Server-Side Request Forgeries against top tier applications. Further investigation uncovered a common explanation for those successful attacks: private-ip, an open source security module used by the compromised applications.

SSRF attacks allow malicious actors to abuse functionality on a server: reading data from internal resources or modifying the code running on the server. Private-ip was created to help application developers spot and block such attacks. SSRF is one of the most common forms of attack on web applications according to OWASP.

Black Box Device Research reveals Pitiful State of Internet of Things Security

The problem: private-ip didn’t do its job very well.

“The code logic was using a simple Regular Expression matching,” Jackson (@johnjhacking) told The Security Ledger. Jackson, working with other researchers, found that private-ip was blind to a wide number of variations of localhost, and other private-ip ranges as well as simple tricks that hackers use to obfuscate IP addresses in attacks. For example, researchers found they could send successful requests for localhost resources by obscuring those addresses using hexadecimal equivalents of private IP addresses or with simple substitutions like using four zeros for each octet of the IP address instead of one (so: 0000.0000.0000.0000 instead of 0.0.0.0). The result: a wide range of private and restricted IP addresses registered as public IP addresses and slipped past private-ip.

Private-IP: small program, BIG footprint

The scope of the private-ip flaws are difficult to grasp. However, available data suggests the component is very widely used. Jackson said that hundreds of thousands, if not millions of applications likely incorporate private-ip code in some fashion. Many of those applications are not publicly addressable from the Internet, but may still be vulnerable to attack by an adversary with access to their local environment.

Private-ip is the creation of developer Damir Mustafin (aka “frenchbread”), a developer based in the Balkan country of Montenegro, according to his GitHub profile, which contains close to 60 projects of different scopes. Despite its popularity and widespread use, private-ip was not a frequent focus of Mr. Mustafin’s attention. After first being published in August 2016, the application had only been updated once, in April 2017, prior to the most recent update to address the SSRF flaw.

A Low Key, High Distribution App

The lack of steady attention didn’t dissuade other developers from downloading and using the npm private-ip package, however. It has an average of 14,000 downloads weekly, according to data from GitHub. And direct downloads of private-ip are just one measure of its use. Fully 355 publicly identified npm modules are dependents of private-ip v1.0.5, which contains the SSRF flaws. An additional 73 GitHub projects have dependencies on private-ip. All told, that accounts for 153,374 combined weekly downloads of private-ip and its dependents. One of the most widely used applications that relies on private-ip is libp2p, an open source network stack that is used in a wide range of decentralized peer-to-peer applications, according to Jackson.

While the flaw was discovered by so-called “white hat” vulnerability researchers, Jackson said that it is almost certain that malicious actors knew about and exploited it -either directly or inadvertently. Other security researchers have almost certainly stumbled upon it before as well, perhaps discovering a single address that slipped through private-ip and enabled a SSRF attack, while failing to grasp private-ip’s role or the bigger flaws in the module.

In fact, private-ip may be the common source of a long list of SSRF vulnerabilities that have been independently discovered and reported in the last five years, Jackson said.”This may be why a lot of enterprises have struggled with SSRF and block list bypasses,” he said.

After identifying the problem, Jackson and his team contacted the developer, Damir Mustafin (aka “frenchbread”), looking for a fix. However, it quickly became clear that they would need to enlist additional development talent to forge a patch that was comprehensive. Jackson tapped two developers: Nick Sahler of the website hosting provider Square Space and the independent developer known as Sick Codes (@sickcodes) to come up with a comprehensive fix for private-ip. The two implemented the netmask utility and update private-ip to correctly filter private IP ranges and translate all submitted IP addresses at the byte level to catch efforts to slip encoded addresses past the filter.

Common Mode Failures and Software Supply Chain

Even though it is fixed, the private-ip flaw raises larger and deeply troubling questions about the security of software applications on which our homes, businesses and economy are increasingly dependent.

The greater reliance on open source components and the shift to agile development and modular applications has greatly increased society’s exposure to so-called “common cause” failures, in which a the failure of a single element leads to a systemic failure. Security experts say the increasingly byzantine ecosystem of open source and proprietary software with scores or hundreds of poorly understood ‘dependencies’ is ripe for such disruptions.

Sites like npm are a critical part of that ecosystem -and part of the problem. Created in 2008, npm is a package manager for the JavaScript programming language that was acquired by GitHub in March. It acts as a public registry of packages of open source code that can be downloaded and incorporated into web and mobile applications as well as a wide range of hardware from broadband routers to robots. But vetting of the modules uploaded to npm and other platforms is often cursory. Scores have been called out as malicious and an even greater number are quietly dropped from the site every day after being discovered to be malicious in nature.

Less scrutinized is low quality code and applications that may quickly be adopted and woven into scores or hundreds or thousands of other applications and components.

“The problem with (software) dependencies is once you identify a problem with a dependency, everything downstream is f**ked,” the developer known as Sick Codes told The Security Ledger. “It’s a house of cards.”

Patch Now

In the short term, organizations that know they are using private-ip version 1.0.5 or earlier as a means of preventing SSRF or related vulnerabilities should upgrade to the latest version immediately, Jackson said. Static application security testing tools can help identify whether private-ip is in use within your organization.

The bigger fix is for application developers to pay more attention to what they’re putting into their creations. “My recommendations is that when software engineers use packages in general or third party code, they need to evaluate what they’re using and where its coming from,” Jackson said.