*32c3 preroll music*
Angel: I introduce Whitney Merrill. She is an attorney in the US
and she just recently, actually last week, graduated
to her CS masters in Illinois.
*applause*
Angel: Without further ado: ‘Predicting Crime In A Big Data World’.
*cautious applause*
Whitney Merrill: Hi everyone. Thank you so much for coming.
I know it´s been a exhausting Congress, so I appreciate you guys coming
to hear me talk about Big Data and Crime Prediction.
This is kind of a hobby of mine, I,
in my last semester at Illinois, decided to poke around
what´s currently happening, how these algorithms are being used and kind of
figure out what kind of information can be gathered. So, I have about 30 minutes
with you guys. I´m gonna do a broad overview of the types of programs.
I´m gonna talk about what Predictive Policing is, the data used,
similar systems in other areas where predictive algorithms are
trying to better society, current uses in policing.
I´m gonna talk a little bit about their effectiveness and then give you
some final thoughts. So, imagine,
in the very near future a Police officer is walking down the street
wearing a camera on her collar. In her ear is a feed of information
about the people and cars she passes alerting her to individuals and cars
that might fit a particular crime or profile for a criminal.
Early in the day she examined a map highlighting hotspots for crime.
In the area she´s been set to patrol the predictive policing software
indicates that there is an 82% chance of burglary at 2 pm,
and it´s currently 2:10 pm. As she passes one individual
her camera captures the individual´s face, runs it through
a coordinated Police database - all of the Police departments that use this database
are sharing information. Facial recognition software indicates that
the person is Bobby Burglar who was previously convicted of burglary,
was recently released and is now currently on patrole. The voice in her ear whispers:
50 percent likely to commit a crime. Can she stop and search him?
Should she chat him up? Should see how he acts?
Does she need additional information to stop and detain him?
And does it matter that he´s carrying a large duffle bag?
Did the algorithm take this into account or did it just look at his face?
What information was being collected at the time the algorithm
chose to say 50% to provide the final analysis?
So, another thought I´m gonna have you guys think about as I go
through this presentation, is this quote that is more favorable
towards Police algorithms, which is: “As people become data plots
and probability scores, law enforcement officials and politicians alike
can point and say: ‘Technology is void of the racist, profiling bias of humans.’”
Is that true? Well, they probably will point and say that,
but is it actually void of racist, profiling humans?
And I´m gonna talk about that as well.
So, Predictive Policing explained. Who and what?
First of all, Predictive Policing actually isn´t new. All we´re doing
is adding technology, doing better, faster aggregation of data.
Analysts in Police departments have been doing this by hand for decades.
These techniques are used to create profiles that accurately match
likely offenders with specific past crimes. So, there´s individual targeting
and then we have location-based targeting. The location-based,
the goal is to help Police forces deploy their resources
in a correct manner, in an efficient manner. They can be as simple
as recommending that general crime may happen in a particular area,
or specifically, what type of crime will happen in a one-block-radius.
They take into account the time of day, the recent data collected
and when in the year it´s happening as well as weather etc.
So, another really quick thing worth going over, cause not everyone
is familiar with machine learning. This is a very basic breakdown
of training an algorithm on a data set.
You collect it from many different sources, you put it all together,
you clean it up, you split it into 3 sets: a training set, a validation set
and a test set. The training set is what is going to develop the rules
in which it´s going to kind of determine the final outcome.
You´re gonna use a validation set to optimize it and finally
apply this to establish a confidence level.
There you´ll set a support level where you say you need a certain amount of data
to determine whether or not the algorithm has enough information
to kind of make a prediction. So, rules with a low support level
are less likely to be statistically significant and the confidence level
in the end is basically if there´s an 85% confidence level
that means there´s an 85% chance that the suspect, e.g. meeting the rule in question,
is engaged in criminal conduct. So, what does this mean? Well,
it encourages collection and hoarding of data about crimes and individuals.
Because you want as much information as possible so that you detect
even the less likely scenarios.
Information sharing is also encouraged because it´s easier,
it´s done by third parties, or even what are called fourth parties
and shared amongst departments. And here, your criminal data again
was being done by analysts in Police departments for decades, but
the information sharing and the amount of information they could aggregate
was just significantly more difficult. So,
what are these Predictive Policing algorithms and software…
what are they doing? Are they determining guilt and innocence?
And, unlike a thoughtcrime, they are not saying this person is guilty,
this person is innocent. It´s creating a probability of whether or not
the person has likely committed a crime or will likely commit a crime.
And it can only say something to the future and the past.
This here is a picture from one particular piece of software
provided by HunchLab; and patterns emerge here from past crimes
that can profile criminal types and associations, detect crime patterns etc.
Generally in this types of algorithms they are using unsupervised data,
that means someone is not going through and saying true-false, good-bad, good-bad.
There´s just 1) too much information and 2) they´re trying to do clustering,
determine the things that are similar.
So, really quickly, I´m also gonna talk about the data that´s used.
There are several different types: Personal characteristics,
demographic information, activities of individuals, scientific data etc.
This comes from all sorts of sources, one that really shocked me, was,
and I´ll talk about it a little bit in the future, but, is the radiation detectors
on New York City Police are constantly taking in data
and it´s so sensitive, it can detect if you´ve had a recent medical treatment
that involves radiation. Facial recognition and biometrics
are clear here and the third-party doctrine – which basically says
in the United States that you have no reasonable expectation of privacy in data
you share with third parties – facilitates easy collection
for Police officers and Government officials because they can go
and ask for the information without any sort of warrant.
For a really great overview: a friend of mine, Dia, did a talk here at CCC
on “The architecture of a street level panopticon”. Does a really great overview
of how this type of data is collected on the streets. Worth checking out
´cause I´m gonna gloss over kind of the types of data.
There is in the United States what they call Multistate Anti-Terrorism
Information Exchange Program which uses everything from credit history,
your concealed weapons permits, aircraft pilot licenses,
fishing licences etc. that´s searchable and shared amongst Police departments
and Government officials and this is just more information. So, if they can collect
it, they will aggregate it into a data base. So, what are the current uses?
There are many, many different companies currently
making software and marketing it to Police departments.
All of them are slightly different, have different features, but currently
it´s a competition to get clients, Police departments etc.
The more Police departments you have the more data sharing you can sell,
saying: “Oh, by enrolling you’ll now have x,y and z Police departments’ data
to access” etc. These here are Hitachi and HunchLab,
they both are hotspot targeting, it´s not individual targeting,
those are a lot rarer. And it´s actually being used in my home town,
which I´ll talk about in a little bit. Here, the appropriate tactics
are automatically displayed for officers when they´re entering mission areas.
So HunchLab will tell an officer: “Hey, you´re entering an area
where there´s gonna be burglary that you should keep an eye out, be aware”.
And this is updating in live time and they´re hoping it mitigates crime.
Here are 2 other ones, the Domain Awareness System was created
in New York City after 9/11 in conjunction with Microsoft.
New York City actually makes money selling it to other cities
to use this. CCTV-cameras are collected, they can…
If they say there´s a man wearing a red shirt,
the software will look for people wearing red shirts and at least
alert Police departments to people that meet this description
walking in public in New York City. The other one is by IBM
and there are quite a few, you know, it´s just generally another hotspot targeting,
each have a few different features. Worth mentioning, too, is the Heat List.
This targeted individuals. I’m from the city of Chicago. I grew up in the city.
There are currently 420 names, when this came out about a year ago,
of individuals who are 500 times more likely than average to be involved
in violence. Individual names, passed around to each Police officer in Chicago.
They consider the rap sheet, disturbance calls, social network etc.
But one of the main things they considered in placing mainly young black individuals
on this list were known acquaintances and their arrest histories.
So if kids went to school or young teenagers went to school
with several people in a gang – and that individual may not even be involved
in a gang – they’re more likely to appear on the list. The list has been
heavily criticized for being racist, for not giving these children
or young individuals on the list a chance to change their history
because it’s being decided for them. They’re being told: “You are likely
to be a criminal, and we’re gonna watch you”. Officers in Chicago
visited these individuals would do knock and announce with a knock on the door
and say: “Hi, I’m here, like just checking up what are you up to”.
Which you don’t need any special suspicion to do. But it’s, you know,
kind of a harassment that might cause a feedback,
back into the data collected.
This is PRECOBS. It’s currently used here in Hamburg.
They actually went to Chicago and visited the Chicago Police Department
to learn about Predictive Policing tactics in Chicago to implement it
throughout Germany, Hamburg and Berlin.
It’s used to generally forecast repeat-offenses.
Again, when training data sets you need enough data points to predict crime.
So crimes that are less likely to happen or happen very rarely:
much harder to predict. Crimes that aren’t reported: much harder to predict.
So a lot of these software… like pieces of software
rely on algorithms that are hoping that there’s a same sort of picture,
that they can predict: where and when and what type of crime will happen.
PRECOBS is actually a term with a plan
– the movie ‘Minority Report’, if you’re familiar with it, it’s the 3 psychics
who predict crimes before they happen.
So there’re other, similar systems in the world that are being used
to predict whether or not something will happen.
The first one is ‘Disease and Diagnosis’. They found that algorithms are actually
more likely than doctors to predict what disease an individual has.
It’s kind of shocking. The other is ‘Security Clearance’ in the US.
It allows access to classified documents. There’s no automatic access in the US.
So every person who wants to see some sort of secret cleared document
must go through this process. And it’s vetting individuals.
So it’s an opt-in process. But here they’re trying to predict who will
disclose information, who will break the clearance system;
and predict there… Here, the error rate, they’re probably much more comfortable
with a high error rate. Because they have so many people competing
for a particular job, to get clearance, that if they’re wrong,
that somebody probably won’t disclose information, they don’t care,
they just rather eliminate them than take the risk.
So I’m an attorney in the US. I have this urge to talk about US law.
It also seems to impact a lot of people internationally.
Here we’re talking about the targeting of individuals, not hotspots.
So targeting of individuals is not as widespread, currently.
However it’s happening in Chicago;
and other cities are considering implementing programs and there are grants
right now to encourage Police departments
to figure out target lists.
So in the US suspicion is based on the totality of the circumstances.
That’s the whole picture. The Police officer, the individual must look
at the whole picture of what’s happening before they can detain an individual.
It’s supposed to be a balanced assessment of relative weights, meaning
– you know – if you know that the person is a pastor maybe then
pacing in front of a liquor store, is not as suspicious
as somebody who’s been convicted of 3 burglaries. It has to be ‘based
on specific and articulable facts’. And the Police officers can use experience
and common sense to determine whether or not their suspicion…
Large amounts of networked data generally can provide individualized suspicion.
The principal components here… the events leading up to the stop-and-search
– what is the person doing right before they’re detained as well as the use
of historical facts known about that individual, the crime, the area
in which it’s happening etc. So it can rely on both things.
No court in the US has really put out a percentage as what Probable Cause
and Reasonable Suspicion. So ‘Probable Cause’ you need to get a warrant
to search and seize an individual. ‘Reasonable Suspicion’ is needed
to do stop-and-frisk in the US – stop an individual and question them.
And this is a little bit different than what they call ‘Consensual Encounters’,
where a Police officer goes up to you and chats you up. ‘Reasonable Suspicion’
– you’re actually detained. But I had a law professor who basically said:
“30%..45% seem like a really good number
just to show how low it really is”.You don’t even need to be 50% sure
that somebody has committed a crime.
So, officers can draw from their own experience to determine ‘Probable Cause’.
And the UK has a similar ‘Reasonable Suspicion’ standard
which depend on the circumstances of each case. So,
I’m not as familiar with UK law but I believe even that some of the analysis-run
‘Reasonable Suspicion’ is similar.
Is this like a black box? So, I threw this slide in
for those who are interested in comparing this US law.
Generally a dog sniff in the US falls under a particular set
of legal history which is: a dog can go up, sniff for dogs,
alert and that is completely okay.
And the Police officers can use that data to detain and further search
an individual. So is an algorithm similar to the dog which is kind of a black box?
Information goes out, it’s processed, information comes out and
a prediction is made. Police rely on the ‘Good Faith’
in ‘Totality of the Circumstances’ to make their decision. So there’s
really no… if they’re relying on the algorithm
and think in that situation that everything’s okay we might reach
a level of ‘Reasonable Suspicion’ where the individual can now pat down
the person he’s decided on the street or the algorithm has alerted to. So,
the big question is, you know, “Could the officer consult predictive software apps
in any individual analysis. Could he say: ‘60% likely to commit a crime’”.
In my hypo: Does that mean that the person
without looking at anything else detain that individual.
And the answer is “Probably not”. One: predictive Policing algorithms just
can not take in the Totality of the Circumstances. They have to be
frequently updated, there are things that are happening that
the algorithm possibly could not have taken into account.
The problem here is that the algorithm itself,
the prediction itself becomes part of Totality of the Circumstances,
which I’m going to talk about a little bit more later.
But officers have to have Reasonable Suspicion before the stop occurs.
Retroactive justification is not sufficient. So,
the algorithm can’t just say: “60% likely, you detain the individual
and then figure out why you’ve detained the person”. It has to be
before the detention actually happens. And the suspicion must relate
to current criminal activity. The person must be doing something
to indicate criminal activity. Just the fact that an algorithm says,
based on these facts: “60%”, or even without articulating
why the algorithm has chosen that, isn’t enough.
Maybe you can see a gun shaped bulge in the pocket etc.
So, effectiveness… the Totality of the Circumstances,
can the algorithms keep up? Generally, probably not.
Missing data, not capable of processing this data in real time.
There’s no idea… the algorithm doesn’t know,
and the Police officer probably doesn’t know the all of the facts.
So the Police officer can take the algorithm into consideration
but the problem here is: Did the algorithm know that the individual was active
in the community, or was a politician, or
that was a personal friend of the officer etc. It can’t just be relied upon.
What if the algorithm did take into account that the individual was a Pastor?
Now that information is counted twice and the balancing for the Totality
of the Circumstances is off. Humans here must be the final decider.
What are the problems? Well, there’s bad underlying data,
there’s no transparency into what kind of data is being used,
how it was collected, how old it is, how often it’s been updated,
whether or not it’s been verified. There could just be noise in the training data.
Honestly, the data is biased. It was collected by individuals in the US;
generally there’ve been several studies done that
black, young individuals are stopped more often than whites.
And this is going to cause a collection bias.
It’s gonna be drastically disproportionate to the makeup of the population of cities;
and as more data has been collected on minorities, refugees in poor neighborhoods
it’s gonna feed back in and of course only have data on those groups and provide
feedback and say: “More crime is likely to
happen because that’s where the data
was collected”. So, what’s an acceptable error rate, well,
depends on the burden of proof. Harm is different for an opt-in system.
You know, what’s my harm if I don’t get clearance, or I don’t get the job;
but I’m opting in, I’m asking to being considered for employment.
In the US, what’s an error? If you search and find nothing, if you think
you have Reasonable Suspicion based on good faith,
both on the algorithm and what you witness, the US says that it’s
no 4th Amendment violation, even if nothing has happened.
It’s very low error false-positive rate here.
In Big Data, generally, and machine-learning it’s great!
Like 1% error is fantastic! But that’s pretty large for the number of individuals
stopped each day. Or who might be subject to these algorithms.
Because even though there’re only 400 individuals on the list in Chicago
those individuals have been listed basically as targets
by the Chicago Police Department.
Other problems include database errors. Exclusion of evidence in the US
only happens when there’s gross negligence or systematic misconduct.
That’s very difficult to prove, especially when a lot of people view these algorithms
as a big box. Data goes in, predictions come out, everyone’s happy.
You rely and trust on the quality of IBM, HunchLab etc.
to provide good software.
Finally, some more concerns I have include feedback loop auditing
and access to data and algorithms and the prediction thresholds.
How certain must a prediction be – before it’s reported to the Police –
that the person might commit a crime. Or that crime might happen
in the individual area. If Reasonable Suspicion is as low as 35%,
and reasonable Suspicion in the US has been held at: That guy drives a car
that drug dealers like to drive, and he’s in the DEA database
as a possible drug dealer. That was enough to stop and search him.
So, are there Positives? Well, PredPol,
which is one of the services that provides Predictive Policing software,
says: “Since these cities have implemented there’s been dropping crime”.
In L.A. 13% reduction in crime, in one division.
There was even one day where they had no crime reported.
Santa Cruz – 25..29% reduction, -9% in assaults etc.
One: these are Police departments self-reporting these successes for…
you know, take it for what it is and reiterated by the people
selling the software. But perhaps it is actually reducing crime.
It’s kind of hard to tell because there’s a feedback loop.
Do we know that crime is really being reduced? Will it affect the data
that is collected in the future? It’s really hard to know. Because
if you send the Police officers into a community it’s more likely
that they’re going to affect that community and that data collection.
Will more crimes happen because they feel like the Police are harassing them?
It’s very likely and it’s a problem here.
So, some final thoughts. Predictive Policing programs are not going anywhere.
They’re only in their wheelstart.
And I think that more analysis, more transparency, more access to data
needs to happen around these algorithms. There needs to be regulation.
Currently, a very successful way in which
these companies get data is they buy from Third Party sources
and then sell it to Police departments. So perhaps PredPol might get information
from Google, Facebook, Social Media accounts; aggregate data themselves,
and then turn around and sell it to Police departments or provide access
to Police departments. And generally, the Courts are gonna have to begin to work out
how to handle this type of data. There’s not case law,
at least in the US, that really knows how to handle predictive algorithms
in determining what the analysis says. And so there really needs to be
a lot more research and thought put into this.
And one of the big things in order for this to actually be useful:
if this is a tactic that had been used by Police departments for decades,
we need to eliminate the bias in the data sets. Because right now
all that it’s doing is facilitating and continuing bias, set in the database.
And it’s incredibly difficult. It’s data collected by humans.
And it causes initial selection bias. Which is gonna have to stop
for it to be successful.
And perhaps these systems can cause implicit bias or confirmation bias,
e.g. Police are going to believe what they’ve been told.
So if a Police officer goes on duty to an area
and an algorithm says: “You’re 70% likely to find a burglar
in this area”. Are they gonna find a burglar because they’ve been told:
“You might find a burglar”? And finally the US border.
There is no 4th Amendment protection at the US border.
It’s an exception to the warrant requirement. This means
no suspicion is needed to commit a search. So this data is gonna go into
a way to examine you when you cross the border.
And aggregate data can be used to refuse you entry into the US etc.
And I think that’s pretty much it. And so a few minutes for questions.
*applause* Thank you!
Herald: Thanks a lot for your talk, Whitney. We have about 4 minutes left
for questions. So please line up at the microphones and remember to
make short and easy questions.
Microphone No.2, please.
Question: Just a comment: if I want to run a crime organization, like,
I would target the PRECOBS here in Hamburg, maybe.
So I can take the crime to the scenes
where the PRECOBS doesn’t suspect.
Whitney: Possibly. And I think this is a big problem in getting availability
of data; in that there’s a good argument for Police departments to say:
“We don’t want to tell you what our tactics are for Policing,
because it might move crime”.
Herald: Do we have questions from the internet? Yes, then please,
one question from the internet.
Signal Angel: Is there evidence that data like the use of encrypted messaging
systems, encrypted emails, VPN, TOR, with automated request to the ISP,
are used to obtain real names and collected to contribute to the scoring?
Whitney: I’m not sure if that’s being taken into account
by Predictive Policing algorithms, or by the software being used.
I know that Police departments do take those things into consideration.
And considering that in the US Totality of the Circumstances is
how you evaluate suspicion. They are gonna take all of those things into account
and they actually kind of have to take into account.
Herald: Okay, microphone No.1, please.
Question: In your example you mentioned disease tracking, e.g. Google Flu Trends
is a good example of preventive Predictive Policing. Are there any examples
where – instead of increasing Policing in the lives of communities –
where sociologists or social workers are called to use predictive tools,
instead of more criminalization?
Whitney: I’m not aware if that’s… if Police departments are sending
social workers instead of Police officers. But that wouldn’t surprise me because
algorithms are being used to suspect child abuse. And in the US they’re gonna send
a social worker in regard. So I would not be surprised if that’s also being
considered. Since that’s part of the resources.
Herald: OK, so if you have a really short question, then
microphone No.2, please. Last question.
Question: Okay, thank you for the talk. This talk as well as few others
brought the thought in the debate about the fine-tuning that is required
between false positives and preventing crimes or terror.
Now, it’s a different situation if the Policeman is predicting,
or a system is predicting somebody’s stealing a paper from someone;
or someone is creating a terror attack.
And the justification to prevent it
under the expense of false positive is different in these cases.
How do you make sure that the decision or the fine-tuning is not going to be
deep down in the algorithm and by the programmers,
but rather by the customer – the Policemen or the authorities?
Whitney: I can imagine that Police officers are using common sense in that,
and their knowledge about the situation and even what they’re being told
by the algorithm. You hope that they’re gonna take…
they probably are gonna take terrorism to a different level
than a common burglary or a stealing of a piece of paper
or a non-violent crime. And that fine-tuning
is probably on a Police department
by Police department basis.
Herald: Thank you! This was Whitney Merrill, give a warm round of applause, please!!
Whitney: Thank you! *applause*
*postroll music*
*Subtitles created by c3subtitles.de in the year 2016. Join and help us!*