GoKawiilI want to juxtapose two classes of people in the security industry on the defender side of security engineering for products. The first are the Security as Identity people. This goes back to the security-as-counterculture attitude, glamorized in Hackers and dating back to phone phreaking. It reflects Thompson’s “Workism”, where work is not just a way to earn a living, but a way to build a sense of identity, purpose, and community. In the case of security, this identity tends to be built around being different, seeing things other people do not, and being clever in an edgy and misunderstood way. Someone who is beyond merely a security professional, but has the persona of a Hacker. Stylish outsiders smarter than the system. Detectives finding insights others miss.
There’s nothing wrong with the hacker ethos. The idea of finding places where systems break down and work in unexpected or incorrect ways is obviously extremely valuable. Careers, conferences, and professional communities all reward vulnerability discovery, the more surprising the better. Similarly, the hacker aesthetic is what it is. At this point, it’s mainstream. Around 25,000 people attend DEF CON each year.
Valuing unexpected insights is not only a hacker thing. In the 2000s, the book Freakonomics was extremely popular. It also took the form of proposing surprising explanations for seemingly straightforward or otherwise unremarkable social phenomena. If you can find something unexpected, you must be very smart and cool! But should surprise really be a signal of intelligence? You can take that attitude even farther, and do a Paul Graham style “it turns out” misdirection. Instead of saying you believe in Y, say you expected X, investigated, and it turns out that Y is true. That way, you don’t have to justify Y, you just say X is false.
The identity you choose matters beyond aesthetics. Problem definitions inherently carry an associated worldview, and it’s very difficult to change a worldview, particularly if you’ve built your identity, community, and profession around that worldview. Evidence for solutions that address the underlying problem, but not in a way that validates the worldview, can be rejected. As a concrete example, Nordhaus and Shellenberger address this failure mode for climate change activism in their notorious “Death of Environmentalism” essay from 2004. Environmentalists wanted to get people to agree to protect “the environment” as a supposed “thing”, a special interest deserving of good, that should have technical policy interventions in order to advance it, specifically. This meant solutions needed to take the form of being related to “the environment”, like cap-and-trade and carbon emission limitations. Solutions that resulted in the desired outcome of less greenhouse gases, but were not sufficiently “environmental”, such as decreasing clean energy costs (even with increased usage), higher-yield agriculture (not just organic farm-to-table), and the idea that technological substitution can work better than behavioral modification, were rejected. Ironically, in the two decades since the controversial essay, we can see that many of these rejected solutions are the ones ultimately played out. Improvements in solar and battery technology, alongside electric cars and other renewables, many of which had government subsidies in the 2010s, did more to reduce greenhouse emissions than the policies that validated the accepted worldview of environmentalism in the 2000s.
In security, solutions that do not look like repeatedly discovering, exploiting, and patching technically impressive vulnerabilities may be overlooked if they do not satisfy the worldview of the Security as Identity person, particularly if they would rather act as a detective than as a mechanic. An attacker mindset and a coherent threat model are important to building secure systems. But while necessary, are they complete? And are they even a good starting point? We do not teach people to cook by telling them to “think like a chef”.
This brings us to the second class of person, the Security as Robustness people. It turns out, some people in security are primarily motivated by building correct systems that preserve their desired properties, including during failure conditions, edge cases, and inconsistencies. Security is part of building a well-engineered system that operates properly in all conditions, including in the face of an attacker.
To illustrate the difference, let’s look at how people think about vulnerability rewards programs (VRPs), also known as bug bounties. What’s the point of a VRP? At minimum, a VRP serves as a point of contact for security issues such that they can be patched in product before they are leveraged by attackers. If you run a sufficiently large product (or, in particular, a platform), you will have people contact you with security issues. At least some of the security reports will be valid, and require patches. A VRP is a standard process for this that forces the organization to have the ability to respond to both external and internal security bug reports, filed by people other than the individual feature teams working on some part of the product. It financially incentivizes participants to report bugs to you in a way that’s productive for you. There’s a very straightforward economic argument to say the VRP is successful so long as the number of bugs found increases faster than the relative payout prices. The more bugs found by the VRP, the better.
Beyond bug finding and patching, VRPs can and should be used to inform the strategy of the security team. The VRP is as close to a replacement for the role of the customer in the standard approach to product management as you can get. Attackers aren’t your users, and the full set of insecurity is an unknown unknown. While it won’t enumerate the entire attack surface, the VRP is a repeated and consistent way to gain insight into attacker methodologies and validate the work of the security team, such that you can bound the size and impact of the unknown unknowns. What types of bugs are showing up in the VRP, where in the product are they occurring, why are they happening, and what teams own them can all be inputs to a security roadmap that should be building towards a future where the bugs are rare. VRPs are also a two-way street. Vulnerability researchers react to incentives, so if the security team wants to validate a hypothesis to justify some security initiative, they can also tweak the rules and rewards in the VRP to incentivize submissions that either validate the hypothesis and can be used as collateral to justify the roadmap to other teams and leadership, or that show the hypothesis was incorrect. If the VRP results suggest the roadmap is not attached to reality, the security team can reject the hypothesis and try something different. One of the best things about a VRP is that what bug hunters look for is not constrained by the beliefs of the people who work on the product.
When the VRP is used as an input to a broader strategy, the overarching goal of a VRP is not to fully enumerate all bugs and thus have a secure product. The point of a VRP is to grapple with the insecurity. Arguably, dealing with bugs that come externally through the VRP is the worst possible way to prioritize those bugs, because it is inherently interrupt-driven and not capacity-managed by the teams that patch the bugs. Patching VRP bugs is another case of playing whack-a-mole. Actually securing something remains the responsibility of the security team, who should be making sure that either the bugs can’t happen (e.g. via safe coding), or that the bugs don’t matter anymore (e.g. through architectural changes that lower impact). This distinction with VRPs, between enumerating all bugs versus informing a strategy to reduce insecurity over time, is the litmus test for Security as Robustness.
There’s a polarity in that finding more issues even quicker is good, because it is better to find things and patch them than to never know there were problems, but it is also bad because it is evidence that your system is not working as intended. Finding problems is a diagnostic metric. It proves that your sensor is working. Tracking incidence of types of problems is an outcome metric. Ensuring that the same type of problem is not happening repeatedly over and over again is the actual sign of success.
... continue reading