*Music* Herald: The next talk is about how risky
is software you use. So you may be heard about Trump versus a Russian security
company. We won't judge this, we won't comment this, but we dislike the
prejudgments of this case. Tim Carstens and Parker Thompson will tell you a little
bit more about how risky the software is you use. Tim Carstens is CITL's Acting
Director and Parker Thompson is CITL's lead engineer. Please welcome with a very,
very warm applause: Tim and Parker! Thanks.
*Applause* Tim Carstens: Howdy, howdy. So my name is
Tim Carstens. I'm the acting director of the cyber independent testing lab. It's
four words there, we'll talk about all for today, especially cyber. With me today as
our lead engineer Parker Thompson. Not on stage or our other collaborators: Patrick
Stach, Sarah Zatko, and present in the room but not on stage - Mudge. So today
we're going to be talking about our work, the lead in. The introduction that was
given is phrased in terms of Kaspersky and all of that, I'm not gonna be speaking
about Kaspersky and I guarantee you I'm not gonna be speaking about my president.
Right, yeah? Okay. Thank you. *Applause*
All right, so why don't we go ahead and kick off: I'll mention now parts of this
presentation are going to be quite technical. Not most of it, and I will
always include analogies and all these other things if you are here in security
but not a bit-twiddler. But if you do want to be able to review some of the technical
material, if I go through it too fast you like to read if you're a mathematician or
if you are a computer scientist, our sides are already available for download at this
site here. We think our pal our partners at power door for getting that set up for
us. Let's let's get started on the real material here. Alright, so we are CITL: a
nonprofit organization based in the United States founded by our chief scientist
Sarah Zatko and our board chair Mudge. And our mission is a public good mission - we
are hackers but our mission here is actually to look out for people who do not
know very much about machines or as much as the other hackers do.
Specifically, we seek to improve the state of software security by providing the
public with accurate reporting on the security of popular software, right? And
so there was a mouthful for you. But no doubt, no doubt, every single one of you
has received questions of the form: what do I run on my phone, what do I do with
this, what do I do with that, how do I protect myself - all these other things
lots of people in the general public looking for agency in computing. No one's
offering it to them, and so we're trying to go ahead and provide a forcing function
on the software field in order to, you know, again be able to enable consumers
and users and all these things. Our social good work is funded largely by charitable
monies from the Ford Foundation whom we thank a great deal, but we also have major
partnerships with Consumer Reports, which is a major organization in the United
States that generally, broadly, looks at consumer goods for safety and performance.
But also partners with The Digital Standard, which probably would be of great
interest to many people here at Congress as it is a holistic standard for
protecting user rights. We'll talk about some of the work that goes into those
things here in a bit, but first I want to give the big picture of what it is we're
really trying to do in one one short little sentence. Something like this but
for security, right? What are the important facts, how does it rate, you
know, is it easy to consume, is it easy to go ahead and look and say this thing is
good this thing is not good. Something like this, but for software security.
Sounds hard doesn't it? So I want to talk a little bit about what I mean by
something like this. There are lots of consumer outlook and
watchdog and protection groups - some private, some government, which are
looking to do this for various things that are not a software security. And you can
see some examples here that are big in the United States - I happen to not like these
as much as some of the newer consumer labels coming out from the EU. But
nonetheless they are examples of the kinds of things people have done in other
fields, fields that are not security to try to achieve that same end. And when
these things work well, it is for three reasons: One, it has to contain the
relevant information. Two: it has to be based in fact, we're not talking opinions,
this is not a book club or something like that. And three: it has to be actionable,
it has to be actionable - you have to be able to know how to make a decision based
on it. How do you do that for software security? How *do* you do that for
software security? So the rest of the talk is going to go in three parts.
First, we're going to give a bit of an overview for more of the consumer facing
side of things for that we do: look at some data that we have reported on early
and all these other kinds of good things. We're then going to go ahead and get
terrifyingly, terrifyingly technical. And then after that we'll talk about tools to
actually implement all this stuff. The technical part comes before the tools. So
it just tells you how terrifyingly technical we're gonna get. It's gonna be
fun right. So how do you do this for software security: a consumer version. So,
if you set forth to the task of trying to measure software security, right, many
people here probably do work in the security field perhaps as consultants
doing reviews; certainly I used to. Then probably what you're thinking to yourself
right now is that there are lots and lots and lots and lots of things that affect
the security of a piece of software. Some of which are, mmm, you're only gonna see
them if you go reversing. And some of which are just you know kicking around on
the ground waiting for you to notice, right. So we're going to talk about both
of those kinds of things that you might measure. But here you see these giant
charts that basically go through on the left - on the left we have Microsoft Excel
on OS X on the right Google Chrome for OS X this is a couple years old at this point
maybe one and a half years old but over here I'm not expecting you to be able to
read these - the real point is to say look at all of the different things you can
measure very easily. How do you distill, it how do you boil it
down, right. So this is a the opposite of a good consumer safety label. This is just
um if you ever done any consulting this is the kind of report you hand a client to
tell them how good their software is, right? It's the opposite of consumer
grade. But the reason I'm showing it here is because, you know, I'm gonna call out
some things and maybe you can't process all of this because it's too much
material, you know. But I'm gonna call it some things and once I call them out just
like NP you're gonna recognize them instantly. So for example, Excel, at the
time of this review - look at this column of dots. What's this dots telling you?
It's telling you look at all these libraries -all of them are 32-bit only.
Not 64 bits, not 64 bits. Take a look at Chrome - exact opposite, exact opposite
64-bit binary, right? What are some other things? Excel, again, on OSX maybe you can
see these danger warning signs that go straight straight up the whole thing.
That's the the absence of major heat protection flags in the binary headers.
We'll talk about some what that means exactly in a bit. But also if you hop over
here you'll see like yeah yeah yeah like Chrome has all the different heat
protections that a binary might enable, on OSX that is, but it also has more dots in
this column here off to the right. And what do those dots represent?
Those dots represent functions, functions that historically have been the source of
you know if you call these functions are very hard to call correctly - if you're a
C programmer the "gets" function is a good example. But there are lots of them. And
you can see here the Chrome doesn't mind, it uses them all a bunch. And Excel not so
much. And if you know the history of Microsoft and the trusted computing
initiative and the SDO and all of that you will know that a very long time ago
Microsoft made the decision and they said we're gonna start purging some of these
risky functions from our code bases because we think it's easier to ban them
than teach our devs to use them correctly. And you see that reverberating out in
their software. Google on the other hand says yeah yeah yeah those functions can be
dangerous to use but if you know how to use them they can be very good and so
they're permitted. The point all of this is building to is that if you start by
just measuring every little thing that like your static analyzers can detect in a
piece of software. Two things: one, you wind up with way more data than you can
show in a slide. And two: the engineering process, the software development life
cycle that went into the software will leave behind artifacts that tell you
something about the decisions that went into designing that engineering process.
And so you know, Google for example: quite rigorous as far as hitting you know
GCC dash, and then enable all of the compiler protections. Microsoft may be
less good at that, but much more rigid in things that's were very popular ideas when
they introduced trusted computing, alright. So the big takeaway from this
material is that again the software engineering process results in artifacts
in the software that people can find. Alright. Ok, so that's that's a whole
bunch of data, certainly it's not a consumer-friendly label. So how do you
start to get in towards the consumer zone? Well, the main defect of the big reports
that we just saw is that it's too much information. It's a very dense on data but
it's very hard to distill it to the "so what" of it, right?
And so this here is one of our earlier attempts to go ahead and do that
distillation. What are these charts how did we come up with these? Well on the
previous slide when we saw all these different factors that you can analyze in
software, basically here's whose views that we arrive at this. For each of those
things: pick a weight. Go ahead and compute a score, average against the
weight: tada, now you have some number. You can do that for each of the libraries
and the piece of software. And if you do that for each of the libraries in the
software you can then go ahead and produce these histograms to show, you know, like
this percentage of the DLLs had a score in this range. Boom, there's a bar, right.
How do you pick those weights? We'll talk about that in a sec - it's very technical.
But the the takeaway though, is you know that you wind up with these charts. Now
I've obscured the labels, I've obscured the labels and the reason I've done that
is because I don't really care that much about the actual counts. I want to talk
about the shapes, the shapes of these charts: it's a qualitative thing.
So here: good scores appear on the right, bad scores appear on the left. The
histogram measuring all the libraries and components and so a very secure piece of
software in this model manifests as a tall bar far to the right. And you can see a
clear example at in our custom Gentoo build. Anyone here is a Gentoo fan knows -
hey I'm going to install this thing, I think I'm going to go ahead and turn on
every single one of those flags, and lo and behold if you do that yeah you wind up
with tall bar far to the right. Here's in Ubuntu 16, I bet it's 16.04 but I don't
recall exactly, 16 LTS. Here you see a lot of tall bars to the right - not quite as
consolidated as a custom Gentoo build, but that makes sense doesn't it right? Because
then you know you don't do your whole Ubuntu build. Now I want to contrast. I
want to contrast. So over here on the right we see in the same model, an
analysis of the firmware obtained from two smart televisions. Last year's models from
Samsung and LG. And here the model numbers. We did this work in concert with
Consumer Reports. And what do you notice about these histograms, right. Are the
bars tall and to the right? No, they look almost normal, not quite, but that doesn't
really matter. The main thing that matters is that this is the shape you would expect
to get if you were playing a random game basically to decide what security features
to enable in your software. This is the shape of not having a security program, is
my bet. That's my bet. And so what do you see? You see heavy concentration here in
the middle, right, that seems fair, and like it tails off. On the Samsung nothing
scored all that great, same on the LG. Both of them are you know running their
respective operating systems and they're basically just inheriting whatever
security came from whatever open source thing they forked, right.
So this is this is the kind of message, this right here is the kind of thing that
we serve to exist for. This is us producing charts showing that the current
practices in the not-so consumer-friendly space of running your own Linux distros
far exceed the products being delivered, certainly in this case in the smart TV
market. But I think you might agree with me, it's much worse than this. So let's
dig into that a little bit more, I have a different point that I want to make about
that same data set - so this table here this table is again looking at the LG
Samsung and Gentoo Linux installations. And on this table we're just pulling out
some of the easy to identify security features you might enable in a binary
right. So percentage of binaries with address space layout randomization, right?
Let's talk about that on our Gentoo build it's over 99%. That also holds for the
Amazon Linux AMI - it holds in Ubuntu. ASLR is incredibly common in modern Linux.
And despite that, fewer than 70 percent of the binaries on the LG television had it
enabled. Fewer than 70 percent. And the Samsung was doing, you know, better than
that I guess, but 80 percent is a pretty disappointing when a default Linux
install, you know, mainstream Linux distro is going to get you 99, right? And it only
gets worse, it only gets worse right you know?
RELRO support, if you don't know what that is that's ok but if you do, look abysmal
coverage look at this abysmal coverage coming out of these IOT devices very sad.
And you see it over and over and over again. I'm showing this because some
people in this room or watching this video ship software - and I have a message, I
have a message to those people who ship software who aren't working on say Chrome
or any of the other big-name Pwn2Own kinds of targets. Look at this: you can be
leading the pack by mastering the fundamentals. You can be leading the pack
by mastering the fundamentals. This is a point that really as a security field we
really need to be driving home. You know, one of the things that we're seeing here
in our data is that if you're the vendor who is shipping the product everyone has
heard of in the security field and maybe your game is pretty decent right? If
you're shipping say Windows or if you're shipping Firefox or whatever. But if
you're if you're doing one of these things where people are just kind of beating you
up for default passwords, then your problems are way further than just default
passwords, right? Like the house, the house is messy it needs to be cleaned,
needs to be cleaned. So the rest of the talk like I said we're going to be
discussing a lot of other things that amount to getting you know a peek behind
the curtain and where some of these things come from and getting very specific about
how this business works, but if you're interested in more of the high level
material - especially if you're interested in interesting results and insights, some
of which I'm going to have here later. But I really encourage you though to take a
look at the talk from this past summer by our chief scientist Sarah Zatko, which is
predominantly on the topic of surprising results in the data.
Today, though, this being our first time presenting here in Europe, we figured we
would take more of an overarching kind of view. What we're doing and why we're
excited about it and where it's headed. So we're about to move into a little bit of
the underlying theory, you know. Why do I think it's reasonable to even try to
measure the security of software from a technical perspective. But before we can
get into that I need to talk a little bit about our goals, so that the decisions and
the theory; the motivation is clear, right. Our goals are really simple: it's a
very easy organization to run because of that. Goal number one: remain independent
of vendor influence. We are not the first organization to purport to be looking out
for the consumer. But unlike many of our predecessors, we are not taking money from
the people we review, right? Seems like some basic stuff. Seems like some basic
stuff right? Thank you, okay. Two: automated, comparable, quantitative
analysis. Why automated? Well, we need our test results to be reproducible. And Tim
goes in opens up your software in IDA and finds a bunch of stuff that makes them all
stoped - that's not a very repeatable kind of a standard for things. And so we're
interested in things which are automated. We'll talk about, maybe a few hackers in
here know how hard that is. We'll talk about that, but then last we also we're
well acting as a watchdog - we're protecting the interests of the user, the
consumer, however you would like to look at it. But we also have three non goals,
three non goals that are equally important. One: we have a non goal of
finding and disclosing vulnerabilities. I reserve the right to find and disclose
vulnerabilities. But that's not my goal, it's not my goal. Another non goal is to
tell software vendors what to do. If a vendor asks me how to remediate their
terrible score, I will tell them what we are measuring but I'm not there to help
them remediate. It's on them to be able to ship a secure product without me holding
their hand. We'll see. And then three: non-goal, perform free security testing
for vendors. Our testing happens after you release. Because when you release your
software you are telling people it is ready to be used. Is it really though, is
it really though, right? *Applause*
Yeah, thank you. Yeah, so we are not there to give you a preview of what your score
will be. There is no sum of money you can hand me that will get you an early preview
of what your score is - you can try me, you can try me: there's a fee for trying
me. There's a fee for trying me. But I'm not gonna look at your stuff until I'm
ready to drop it, right. Yeah bitte, yeah. All right. So moving into this theory
territory. Three big questions, three big questions that need to be addressed if you
want to do our work efficiently: what works, what works for improving security,
what are the things that need or that you really want to see in software. Two: how
do you recognize when it's being done? It's no good if someone hands you a piece
of software and says, "I've done all the latest things" and it's a complete black
box. If you can't check the claim, the claim is as good as false, in practical
terms, period, right. Software has to be reviewable or a priori, I'll think you're
full of it. And then three: who's doing it - of all the things that work, that you
can recognize, who's actually doing it. You know, let's go ahead - our field is
famous for ruining people's holidays and weekends over Friday bug disclosures, you
know New Year's Eve bug disclosures. I would like us to also be famous for
calling out those teams and those software organizations which are being as good as
the bad guys are being bad, yeah? So provide someone an incentive to be maybe
happy to see us for a change, right. Okay, so thank you. Yeah, all right. So how do
we actually pull these things off; the basic idea. So, I'm going to get into some
deeper theory: if you're not a theorist I want you to focus on this slide.
And I'm gonna bring it back, it's not all theory from here on out after this but if
you're not a theorist I really want you to focus on this slide. The basic motivation,
the basic motivation behind what we're doing; the technical motivation - why we
think that it's possible to measure and report on security. It all boils down to
this right. So we start with a thought experiment, a gedankent, right? Given a
piece of software we can ask: overall, how secure is it? Kind of a vague question but
you could imagine you know there's versions of that question. And two: what
are its vulnerabilities. Maybe you want to nitpick with me about what the word
vulnerability means but broadly you know this is a much more specific question
right. And here's here's the enticing thing: the first question appears to ask
for less information than the second question. And maybe if we were taking bets
I would put my money on, yes, it actually does ask for less information. What do I
mean by that what do I mean by that? Well, let's say that someone told you all of the
vulnerabilities in a system right? They said, "Hey, I got them all", right? You're
like all right that's cool, that's cool. And if someone asks you hey how secure is
this system you can give them a very precise answer. You can say it has N
vulnerabilities, and they're of this kind and like all this stuff right so certainly
the second question is enough to answer the first. But, is the reverse true?
Namely, if someone were to tell you, for example, "hey, this piece of software has
exactly 32 vulnerabilities in it." Does that make it easier to find any of them?
Right, there's room for to maybe do that using some algorithms that are not yet in
existence. Certainly the computer scientists in here
are saying, "well, you know, yeah maybe counting the number of SAT solutions
doesn't help you practically find solutions. But it might and we just don't
know." Okay fine fine fine. Maybe these things are the same, but the my experience
in security, and the experience of many others perhaps is that they probably
aren't the same question. And this motivates what I'm calling here is Zatko's
question, which is basically asking for an algorithm that demonstrates that the first
question is easier than the second question, right. So Zatko's question:
develop a heuristic which can to efficiently answer one, but not
necessarily two. If you're looking for a metaphor, if you want to know why I care
about this distinction, I want you to think about some certain controversial
technologies: maybe think about say nuclear technology, right. An algorithm
that answers one, but not two, it's a very safe algorithm to publish. Very safe
algorithm publish indeed. Okay, Claude Shannon would like more information. happy
to oblige. Let's take a look at this question from a different perspective
maybe a more hands-on perspective: the hacker perspective, right? If you're a
hacker and you're watching me up here and I'm waving my hands around and I'm showing
you charts maybe you're thinking to yourself yeah boy, what do you got? Right,
how does this actually go. And maybe what you're thinking to yourself is that, you
know, finding good vulns: that's an artisan craft right? You're in IDA, you
know you're reversing old way you're doing all these things or hit and Comm, I don't
know all that stuff. And like, you know, this kind of clever game; cleverness is
not like this thing that feels very automatable. But you know on the other
hand there are a lot of tools that do automate things and so it's not completely
not automatable. And if you're into fuzzing then perhaps
you are aware of this very simple observation, which is that if your harness
is perfect if you really know what you're doing if you have a decent fuzzer then in
principle fuzzing can find every single problem. You have to be able to look for
it you have to be able harness for it but in principle it will, right. So the hacker
perspective on Zatko's question is maybe of two minds on the one hand assessing
security is a game of cleverness, but on the other hand we're kind of right now at
the cusp of having some game-changing tech really go - maybe you're saying like
fuzzing is not at the cusp, I promise it's just at the cusp. We haven't seen all the
fuzzing has to offer right and so maybe there's room maybe there's room for some
automation to be possible in pursuit of Zatko's question. Of course, there are
many challenges still in, you know, using existing hacker technology. Mostly of the
form of various open questions. For example if you're into fuzzing, you know,
hey: identifying unique crashes. There's an open question. We'll talk about some of
those, we'll talk about some of those. But I'm going to offer another perspective
here: so maybe you're not in the business of doing software reviews but you know a
little computer science. And maybe that computer science has you wondering what's
this guy talking about, right. I'm here to acknowledge that. So whatever you think
the word security means: I've got a list of questions up here. Whatever you think
the word security means, probably, some of these questions are relevant to your
definition. Right. Does the software have a hidden backdoor
or any kind of hidden functionality, does it handle crypto material correctly, etc,
so forth. Anyone in here who knows some computers abilities theory knows that
every single one of these questions and many others like them are undecidable due
to reasons essentially no different than the reason the halting problem is
undecidable,\ which is to say due to reasons essentially first identified and
studied by Alan Turing a long time before we had microarchitectures and all these
other things. And so, the computability perspective says that, you know, whatever
your definition of security is ultimately you have this recognizability problem:
fancy way of saying that algorithms won't be able to recognize secure software
because of the undecidability these issues. The takeaway, the takeaway is that
the computability angle on all of this says: anyone who's in the business that
we're in has to use heuristics. You have to, you have to.
All right, this guy gets it. All right, so on the tech side our last technical
perspective that we're going to take now is certainly the most abstract which is
the Bayesian perspective, right. So if you're a frequentist, you need to get with
the times you know it's everything Bayesian now. So, let's talk about this
for a bit. Only two slides of math, I promise, only two! So, let's say that I
have some corpus of software. Perhaps it's a collection of all modern browsers,
perhaps it's the collection of all the packages in the Debian repository, perhaps
it's everything on github that builds on this system, perhaps it's a hard drive
full of warez that some guy mailed you, right? You have some corpus of software
and for a random program in that corpus we can consider this probability: the
probability distribution of which software is secure versus which is not. For reasons
described on the computability perspective, this number is not a
computable number for any reasonable definition of security. So that's a neat
and so, for practical terms, if you want to do some probabilistic reasoning, you
need some surrogate for that and so we consider this here. So, instead of
considering the probability that a piece of software is secure, a non computable
non verifiable claim, we take a look here at this indexed collection of
probabilities. This is an infinite countable family of probability
distributions, basically P sub h,k is just the probability that for a random piece of
software in the corpus, h work units of fuzzing will find no more than k unique
crashes, right. And why is this relevant? Well, at the bottom we have this analytic
observation, which is that in the limit as h goes to infinity you're basically
saying: "Hey, you know, if I fuzz this thing for infinity times, you know, what's
that look like?" And, essentially, here we have analytically that this should
converge. The P sub h,1 should converge to the probability that a piece of software
just simply cannot be made to crash. Not the same thing as being secure, but
certainly not a small concern relevant to security. So, none of that stuff actually
was Bayesian yet, so we need to get there. And so here we go, right: so, the previous
slide described a probability distribution measured based on fuzzing. But fuzzing is
expensive and it is also not an answer to Zatko's question because it finds
vulnerabilities, it doesn't measure security in the general sense and so
here's where we make the jump to conditional probabilities: Let M be some
observable property of software has ASLR, has RELRO, calls these functions, doesn't
call those functions... take your pick. For random s in S we now consider these
conditional probability distributions and this is the same kind of probability as we
had on the previous slide but conditioned on this observable being true, and this
leads to the refined of the Siddall variant of Zatko's question:
Which observable properties of software satisfy that, when the software has
property m, the probability of fuzzing being hard is very high? That's what this
version of this question phrases, and here we say, you know, in large log(h)/k, in
other words: exponentially more fuzzing than you expect to find bugs. So this is
the technical version of what we're after. All of this can be explored, you can
brute-force your way to finding all of this stuff, and that's exactly what we're
doing. So we're looking for all kinds of things, we're looking for all kinds of
things that correlate with fuzzing having low yield on a piece of software, and
there's a lot of ways in which that can happen. It could be that you are looking
at a feature of software that literally prevents crashes. Maybe it's the never
crash flag, I don't know. But most of the things I've talked about, ASLR, RERO, etc.
don't prevent crashes. In fact a ASLR can take non-crashing programs and make them
crashing. It's the number one reason vendors don't enable it, right? So why am
I talking about ASLR? Why am I talking about RELRO? Why am i talking about all
these things that have nothing to do with stopping crashes and I'm claiming I'm
measuring crashes? This is because, in the Bayesian perspective, correlation is not
the same thing as causation, right? Correlation is not the same thing as
causation. It could be that M's presence literally prevents crashes, but it could
also be that, by some underlying coincidence, the things we're looking for
are mostly only found in software that's robust against crashing.
If you're looking for security, I submit to you that the difference doesn't matter.
Okay, end of my math, danke. I will now go ahead and do this like really nice analogy
of all those things that I just described, right. So we're looking for indicators of
a piece of software being secure enough to be good for consumers, right. So here's an
analogy. Let's say you're a geologist, you study minerals and all of that and you're
looking for diamonds. Who isn't, right? Want those diamonds! And like how do you
find diamonds? Even in places that are rich in diamonds, diamonds are not common.
You don't just go walking around in your boots, kicking until your toe stubs on a
diamond? You don't do that. Instead you look for other minerals that are mostly
only found near diamonds but are much more abundant in those locations than the
diamonds. So, this is mineral science 101, I guess, I don't know. So, for example,
you want to go find diamond: put on your boots and go kicking until you find some
chromite, look for some diopside, you know, look for some garnet. None of these
things turn into diamonds, none of these things cause diamonds but if you're
finding good concentrations of these things, then, statistically, there's
probably diamonds nearby. That's what we're doing. We're not looking for the
things that cause good security per se. Rather, we're looking for the indicators
that you have put the effort into your software, right? How's that working out
for us? How's that working out for us? Well, we're still doing studies. It's, you
know, early to say exactly but we do have the following interesting coincidence: and
so, here presented I have a collection of prices that somebody gave much for so-
called the underground exploits. And I can tell you these prices are maybe a little
low these days but if you work in that business, if you go to Cyscin, if you do
that kind of stuff, maybe you know that this is ballpark, it's ballpark.
Alright, and, just a coincidence, maybe it means we're on the right track, I don't
know, but it's an encouraging sign: When we run these programs through our
analysis, our rankings more or less correspond to the actual prices that you
encounter in the wild for access via these applications. Up above, I have one of our
histogram charts. You can see here that Chrome and Edge in this particular model
scored very close to the same and it's a test model, so, let's say they're
basically the same. Firefox, you know, behind there a little
bit. I don't have Safari on this chart because - this or all Windows applications
- but the Safari score falls in between. So, lots of theory, lots of theory, lots
of theory and then we have this. So, we're going to go ahead now and hand off to our
lead engineer, Parker, who is going to talk about some of the concrete stuff, the
non-chalkboard stuff, the software stuff that actually makes this work.
Thompson: Yeah, so I want to talk about the process of actually doing it. Building
the tooling that's required to collect these observables. Effectively, how do you
go mining for indicator indicator minerals? But first the progression of
where we are and where we're going. We initially broke this out into three major
tracks of our technology. We have our static analysis engine, which started as a
prototype, and we have now recently completed a much more mature and solid
engine that's allowing us to be much more extensible and digging deeper into
programs, and provide a much more deep observables. Then, we have the data
collection and data reporting. Tim showed some of our early stabs at this. We're
right now in the process of building new engines to make the data more accessible
and easy to work with and hopefully more of that will be available soon. Finally,
we have our fuzzer track. We needed to get some early data, so we played with some
existing off-the-shelf fuzzers, including AFL, and, while that was fun,
unfortunately it's a lot of work to manually instrument a lot of fuzzers for
hundreds of binaries. So, we then built an automated solution
that started to get us closer to having a fuzzing harness that could autogenerate
itself, depending on the software, the software's behavior. But, right now,
unfortunately that technology showed us more deficiencies than it showed
successes. So, we are now working on a much more mature fuzzer that will allow us
to dig deeper into programs as we're running and collect very specific things
that we need for our model and our analysis. But on to our analytic pipeline
today. This is one of the most concrete components of our engine and one of the
most fun! We effectively wanted some type of
software hopper, where you could just pour programs in, installers and then, on the
other end, come reports: Fully annotated and actionable information that we can
present to people. So, we went about the process of building a large-scale engine.
It starts off with a simple REST API, where we can push software in, which then
gets moved over to our computation cluster that effectively provides us a fabric to
work with. It makes is made up of a lot of different software suites, starting off
with our data processing, which is done by apache spark and then moves over into data
data handling and data analysis in spark, and then we have a common HDFS layer to
provide a place for the data to be stored and then a resource manager and Yarn. All
of that is backed by our compute and data nodes, which scale out linearly. That then
moves into our data science engine, which is effectively spark with Apache Zeppelin,
which provides us a really fun interface where we can work with the data in an
interactive manner but be kicking off large-scale jobs into the cluster. And
finally, this goes into our report generation engine. What this bought us,
was the ability to linearly scale and make that hopper bigger and bigger as we need,
but also provide us a way to process data that doesn't fit in a single machine's
RAM. You can push the instance sizes as you large as you want, but we have
datasets that blow away any single host RAM set. So this allows us to work with
really large collections of observables. I want to dive down now into our actual
static analysis. But first we have to explore the problem space, because it's a
nasty one. Effectively in settles mission is to process as much software as
possible. Hopefully all of it, but it's hard to get your hand on all the binaries
that are out there. When you start to look at that problem you understand there's a
lot of combinations: there's a lot of CPU architectures, there's a lot of operating
systems, there's a lot of file formats, there's a lot of environments the software
gets deployed into, and every single one of them has their own app Archer app
armory features. And it can be specifically set for one combination
button on another and you don't want to penalize a developer for not turning on a
feature they had no access to ever turn on. So effectively we need to solve this
in a much more generic way. And so what we did is our static analysis engine
effectively looks like a gigantic collection of abstraction libraries to
handle binary programs. You take in some type of input file be it ELF, PE, MachO
and then the pipeline splits. It goes off into two major analyzer classes, our
format analyzers, which look at the software much like how a linker or loader
would look at it. I want to understand how it's going to be loaded up, what type of
armory feature is going to be applied and then we can run analyzers over that. In
order to achieve that we need abstraction libraries that can provide us an abstract
memory map, a symbol resolver, generic section properties. So all that feeds in
and then we run over a collection of analyzers to collect data and observables.
Next we have our code analyzers, these are the analyzers that run over the code
itself. I need to be able to look at every possible executable path. In order to do
that we need to do function discovery, feed that into a control flow recovery
engine, and then as a post-processing step dig through all of the possible metadata
in the software, such as like a switch table, or something like that to get even
deeper into the software. Then this provides us a basic list of basic blocks,
functions, instruction ranges. And does so in an efficient manner so we can process a
lot of software as it goes. Then all that gets fed over into the main modular
analyzers. Finally, all of this comes together and gets put into a gigantic blob
of observables and fed up to the pipeline. We really want to thank the Ford
Foundation for supporting our work in this, because the pipeline and the static
analysis has been a massive boon for our project and we're only beginning now to
really get our engine running and we're having a great time with it. So digging
into the observables themselves, what are we looking at and let's break them apart.
So the format structure components, things like ASLR, DEP, RELRO.
basic app armory, that's going to go into the feature and gonna be enabled at the OS
layer when it gets loaded up or linked. And we also collect other metadata about
the program such as like: "What libraries are linked in?", "What's its dependency
tree look like – completely?", "How did those software, how did those library
score?", because that can affect your main software. Interesting example on Linux, if
you link a library that requires an executable stack, guess what your software
now has an executable stack, even if you didn't mark that. So we need to be owners
to understand what ecosystem the software is gonna live in. And the code structure
analyzers look at things like functionality: "What's the software
doing?", "What type of app armory is getting injected into the code?". A great
example of that is something like stack guards or fortify source. These are our
main features that only really apply and can be observed inside of the control flow
or inside of the actual instructions themselves. This is why control
photographs are key. We played around with a number of
different ways of analyzing software that we could scale out and ultimately we had
to come down to working with control photographs. Provided here is a basic
visualization of what I'm talking about with a control photograph, provided by
Benja, which has wonderful visualization tools, hence this photo, and not our
engine because we don't build their very many visualization engines. But you
basically have a function that's broken up into basic blocks, which is broken up into
instructions, and then you have basic flow between them. Having this as an iterable
structure that we can work with, allows us to walk over that and walk every single
instruction, understand the references, understand where code and data is being
referenced, and how is it being referenced.
And then what type of functionalities being used, so this is a great way to find
something, like whether or not your stack guards are being applied on every function
that needs them, how deep are they being applied, and is the compiler possibly
introducing errors into your armory features. which are interesting side
studies. Also why we did this is because we want to push the concept of what type
of observables even farther. Let's say take this example you want to be able to
take instruction abstractions. Let's say for all major architectures you can break
them up into major categories. Be it arithmetic instructions, data manipulation
instructions, like load stores and then control flow instructions. Then with these
basic fundamental building blocks you can make artifacts. Think of them like a unit
of functionality: has some type of input, some type of output, it provides some type
of operation on it. And then with these little units of functionality, you can
link them together and think of these artifacts as may be sub-basic block or
crossing a few basic blocks, but a different way to break up the software.
Because a basic block is just a branch break, but we want to look at
functionality brakes, because these artifacts can provide the basic
fundamental building blocks of the software itself. It's more important, when
we want to start doing symbolic lifting. So that we can lift the entire software up
into a generic representation, that we can slice and dice as needed.
Moving from there, I want to talk about fuzzing a little bit more. Fuzzing is
effectively at the heart of our project. It provides us the rich dataset that we
can use to derive a model. It also provides us awesome other metadata on the
side. But why? Why do we care about fuzzing? Why is fuzzing the metric, that
you build an engine, that you build a model that you drive some type of reason
from? So think of the set of bugs, vulnerabilities, and exploitable
vulnerabilities. In an ideal world you'd want to just have a machine that pulls out
exploitable vulnerabilities. Unfortunately, this is exceedingly costly
for a series of decision problems, that go between these sets. So now consider the
superset of bugs or faults. A fuzzer can easily recognize, or other software can
easily recognize faults, but if you want to move down the sets you unfortunately
need to jump through a lot of decision hoops. For example, if you want to move to
a vulnerability you have to understand: Does the attacker have some type of
control? Is there a trust boundary being crossed? Is this software configured in
the right way for this to be vulnerable right now? So they're human factors that
are not deducible from the outside. You then amplify this decision problem even
worse going to exploitable vulnerabilities. So if we collect the
superset of bugs, we will know that there is some proportion of subsets in there.
And this provides us a datasets easily recognizable and we can collect in a cost-
efficient manner. Finally, fuzzing is key and we're investing a lot of our time
right now and working on a new fuzzing engine, because there are some key things
we want to do. We want to be able to understand all of
the different paths the software could be taking, and as you're fuzzing you're
effectively driving the software down as many unique paths while referencing as
many unique data manipulations as possible. So if we save off every path,
annotate the ones that are faulting, we now have this beautiful rich data set of
exactly where the software went as we were driving it in specific ways. Then we feed
that back into our static analysis engine and begin to generate those instruction
out of those instruction abstractions, those artifacts. And with that, imagine we
have these gigantic traces of instruction abstractions. From there we can then begin
to train the model to explore around the fault location and begin to understand and
try and study the fundamental building blocks of what a bug looks like in an
abstract instruction agnostic way. This is why we're spending a lot of time on our
Fuzzing engine right now. But hopefully soon we'll be able to talk about that more
and maybe a tech track and not the policy track.
C: Yeah, so from then on when anything went wrong with the computer we said it
had bugs in it. *laughs* All right, I promised you a technical journey, I
promised you a technical journey into the dark abyss of as deep as you want to get
with it. So let's go ahead and bring it up. Let's wrap it up and bring it up a
little bit here. We've talked a great deal today about some theory. We've talked
about development in our tooling and everything else and so I figured I should
end with some things that are not in progress, but in fact which are done in
yesterday's news. Just to go ahead and make that shared here with Europe. So in
the midst of all of our development we have been discovering and reporting bugs,
again this not our primary purpose really. But you know you can't help but do it. You
know how computers are these days. You find bugs just for turning them on, right?
So we've been disclosing all of that a little while ago. At DEFCON and Black Hat
our chief scientist Sarah together with Mudge went ahead and dropped this
bombshell on the Firefox team which is that for some period of time they had ASLR
disabled on OS X. When we first found it we assumed it was a bug in our tools. When
we first mentioned it in a talk they came to us and said it's definitely a bug on
our tools or might be or some level of surprise and then people started looking
into it and in fact at one point it had been enabled and then temporarily
disabled. No one knew, everyone thought it was on. It takes someone looking to notice
that kind of stuff, right. Major shout out though, they fixed it immediately despite
our full disclosure on stage and everything. So very impressed, but in
addition to popping surprises on people we've also been doing the usual process of
submitting patches and bugs, particularly to LLVM and Qemu and if you work in
software analysis you could probably guess why.
Incidentally, if you're looking for a target to fuzz if you want to go home from
CCC and you want to find a ton of findings LLVM comes with a bunch of parsers. You
should fuzz them, you should fuzz them and I say that because I know for a fact you
are gonna get a bunch of findings and it'd be really nice. I would appreciate it if I
didn't have to pay the people to fix them. So if you wouldn't mind disclosing them
that would help. But besides these bug reports and all these other things we've
also been working with lots of others. You know we gave a talk earlier this summer,
Sarah gave a talk earlier this summer, about these things and she presented
findings on comparing some of these base scores of different Linux distributions.
And based on those findings there was a person on the fedora red team, Jason
Calloway, who sat there and well I can't read his mind but I'm sure that he was
thinking to himself: golly it would be nice to not, you know, be surprised at the
next one of these talks. They score very well by the way. They were leading in
many, many of our metrics. Well, in any case, he left Vegas and he went back home
and him and his colleagues have been working on essentially re-implementing
much of our tooling so that they can check the stuff that we check before they
release. Before they release. Looking for security before you release. So that would
be a good thing for others to do and I'm hoping that that idea really catches on.
*laughs* Yeah, yeah right, that would be nice. That would be nice.
But in addition to that, in addition to that our mission really is to get results
out to the public and so in order to achieve that, we have broad partnerships
with Consumer Reports and the digital standard. Especially if you're into cyber
policy, I really encourage you to take a look at the proposed digital standard,
which is encompassing of the things we look for and and and so much more. URLs,
data, traffic, motion and cryptography and update mechanisms and all that good stuff.
So, where we are and where we're going, the big takeaways here for if you're
looking for that, so what, three points for you: one we are building a tooling
necessary to do larger and larger and larger studies regarding these surrogate
security stores. My hope is that in some period of the not-too-distant future, I
would like to be able to, with my colleagues, publish some really nice
findings about what are the things that you can observe in software, which have a
suspiciously high correlation with the software being good. Right, nobody really
knows right now. It's an empirical question. As far as I know, the study
hasn't been done. We've been running it on the small scale. We're building the
tooling to do it on a much larger scale. We are hoping that this winds up being a
useful field in security as that technology develops. In the meantime our
static analyzers are already making surprising discoveries: hit YouTube and
take a look for Sara Zatko's recent talks at DEFCON/Blackhat. Lots of fun findings
in there. Lots of things that anyone who looks would have found it. Lots of that.
And then lastly, if you were in the business of shipping software and you are
thinking to yourself.. okay so these guys, someone gave them some money to mess up my
day and you're wondering: what can I do to not have my day messed up? One simple
piece of advice, one simple piece of advice: make sure your software employs
every exploit mitigation technique Mudge has ever or will ever hear of. And he's
heard of a lot of them. He's only gonna, you know all that, turn all those things
on and if you don't know anything about that stuff, if nobody on your team knows
anything about that stuff didn't I don't even know I'm saying this if you hear you
know about that stuff so do that. If you're not here, then you should be here.
Danke, Danke. Herald Angel: Thank you, Tim and Parker.
Do we have any questions from the audience? It's really hard to see you with
that bright light in my face. I think the signal angel has a question. Signal Angel:
So the IRC channel was impressed by your tools and your models that you wrote. And
they are wondering what's going to happen to that, because you do have funding from
the Ford foundation now and so what are your plans with this? Do you plan on
commercializing this or is it going to be open source or how do we get our hands on
this? C: It's an excellent question. So for the
time being the money that we are receiving is to develop the tooling, pay for the AWS
instances, pay for the engineers and all that stuff. The direction as an
organization that we would like to take things I have no interest in running a
monopoly. That sounds like a fantastic amount of work and I really don't want to
do it. However, I have a great deal of interest in taking the gains that we are
making in the technology and releasing the data so that other competent researchers
can go through and find useful things that we may not have noticed ourselves. So
we're not at a point where we are releasing data in bulk just yet, but that
is simply a matter of engineering our tools, are still in flux as we, you know.
When we do that, we want to make sure the data is correct and so our software has to
have its own low bug counts and all these other things. But ultimately for the
scientific aspect of our mission. Though the science is not our primary mission.
Our primary mission is to apply it to help consumers. At the same time, it is our
belief that an opaque model is as good as crap, no one should trust an opaque model,
if somebody is telling you that they have some statistics and they do not provide
you with any underlying data and it is not reproducible you should ignore them.
Consequently what we are working towards right now is getting to a point where we
will be able to share all of those findings. The surrogate scores, the
interesting correlations between observables and fuzzing. All that will be
public as the material comes online. Signal Angel: Thank you.
C: Thank you. Herald Angel: Thank you. And microphone
number three please. Mic3: Hi, thanks so some really
interesting work you presented here. So there's something I'm not sure I
understand about the approach that you're taking. If you are evaluating the security
of say a library function or the implementation of a network protocol for
example you know there'd be a precise specification you could check that
against. And the techniques you're using would make sense to me. But it's not so
clear since you've set the goal that you've set for yourself is to evaluate
security of consumer software. It's not clear to me whether it's fair to call
these results security scores in the absence of a threat model so. So my
question is, you know, how is it meaningful to make a claim that a piece of
software is secure if you don't have a threat model for it?
C: This is an excellent question and I anyone who disagrees is they should the
wrong. Security without a threat model is not security at all. It's absolutely a
true point. So the things that we are looking for, most of them are things that
you will already find present in your threat model. And so for example we were
reporting on the presence of things like a ASLR and lots of other things that get to
the heart of exploitability of a piece of software. So for example if we are
reviewing a piece of software, that has no attack surface
then it is canonically not in the threat model and in that sense it makes no sense
to report on its overall security. On the other hand, if we're talking about
software like say a word processor, a browser, anything on your phone, anything
that talks on the network, we're talking about those kinds of applications then I
would argue that exploit mitigations and the other things that we are measuring are
almost certainly very relevant. So there's a sense in which what we are measuring is
the lowest common denominator among what we imagine or the dominant threat models
for the applications. The hand-wavy answer, but I promised heuristics so there
you go. Mic3: Thanks.
C: Thank you. Herald Angel: Any questions? No raising
hands, okay. And then the herald can ask a question, because I never can. So the
question is: you mentioned earlier these security labels and for example what
institution could give out the security labels? Because as obviously the vendor
has no interest in IT security? C: Yes it's a very good question. So our
partnership with Consumer Reports. I don't know if you're familiar with them, but in
the United States Consumer Reports is a major huge consumer watchdog organization.
They test the safety of automobiles, they test you know lots of consumer appliances.
All kinds of things both to see if they function more or less as advertised but
most importantly they're checking for quality, reliability and safety. So our
partnership with Consumer Reports is all about us doing our work and then
publishing that. And so for example the televisions that we presented the data on
all of that was collected and published in partnership with Consumer Reports.
Herald: Thank you. C: Thank you.
Herald: Any other questions for stream. I hear a no. Well in this case people thank
you. Thank Tim and Parker for their nice talk
and please give them a very very warm hall round of applause.
*applause* C: Thank you. T: Thank you.