Professional Documents
Culture Documents
Original decision = 3rd October 2017 and the Internal Review (IR) Decision = 30th
January 2018. All information highlighted within in bold represents additional
information and amended/augmented decisions made under the internal review. The
original decision technological capabilities and possible corruption harm and public
interest arguments applied under Section 31 of the Freedom of Information Act 2000,
have been disregarded at the IR decision stage. The response below relates to the
outcome of the internal review and decision.
Your request:
I am writing under the Freedom of Information Act 2000 to request the following
information around the Harm Assessment Risk Tool (HART) being used by Durham
constabulary.
– Details of the use of HART within an active policing environment (not previous
studies)
– The underlying algorithm of HART
– Weightings used within HART for the variables (predictors) within the system
– Guidance for police officers about the informing affected individuals that HART has
been used in a decision making process
The model Harm Assessment Risk Tool (HART) was created by a collaboration
between Dr G Barnes at the Department of Criminology at Cambridge University and
Durham Constabulary and is an ongoing experimental research programme. The
findings of this research programme will be published following formal and
academic evaluation.
The HART model is based on code ‘R’ software available via open source search –
www.r-project.org
The code ‘R’ statistical programming language and the Random Forest library within
the language were used to produce the HART model. All algorithmic responses use
the past as a model of what will take place in the future. In this instance the original
model used historical data from 104,000 custody events that occurred in Durham
Constabulary over the period 2008-2012. The model uses 34 Predictors: 29 are taken
from the detained person’s offending behaviour plus age, gender, two forms of post
code (outward code only) and a count of intelligence reports related to the detained
person. The historic data is set against the predictors to create a random forest of 509
separate classification and regression decision trees (CART). Each decision tree is a
model in and of itself and produces a forecast which is then used as one vote out of
509 votes. The overall forecast becomes the outcome with the most votes. The random
forest model does not require any traditional weighting of predictors. All predictors are
1
Information Rights and Disclosure
Professional Standards and Legal Services
Peterlee Police Office
St Aidans Way
Peterlee
County Durham
SR8 1QR
Tel. No: 0191 3752596
Web Site: www.durham.police.uk
E-mail: freedomofinformation@durham.pnn.police.uk
provided to the algorithm in a raw unaltered form. The original model is being refreshed
with more up to date historical data as part of the ongoing research programme.
The random forest model does not use, nor does it require, traditional
‘weighting’ of predictors, whereby specific importance is given to a particular
decision, predictor or historical data. All predictors are provided to the algorithm
in a raw unaltered form. The HART model is currently being refreshed with
updated historical data, within the ongoing research programme.
Durham Constabulary’s use of the HART model is limited to a triage tool to assist
support custody officers to identify offenders for consideration of being offered
diversion into the Checkpoint Programme. The HART model is part of an ongoing
experimental research programme in relation to the diversionary scheme. Checkpoint
is a diversion scheme. Details on the Checkpoint Programme have been published
and can be found on the following link:
https://www.durham.police.uk/Information-and-advice/Pages/Checkpoint.aspx
The tool is used to identify individuals who are eligible for consideration of
diversion into the Checkpoint Programme. Use of HART outcomes and the HART
model does not replace the decision making autonomy and obligations of the
custody officer and, whilst the randomised trial shows that HART outcomes are
one factor in decision making more generally, custody sergeants are exercising
their statutory decision making responsibilities.
In relation to Question 4: Guidance for police officers about the informing affected
individuals that HART has been used in a decision making process
Please see the document entitled ‘Script for Custody Officers’ attached to the covering
email. A copy of the attached script is given to the detained person and their legal
representative, if present.
I am not required by statute to release any further information by virtue of the following
exemptions under the Freedom of Information Act 2000:
2
Information Rights and Disclosure
Professional Standards and Legal Services
Peterlee Police Office
St Aidans Way
Peterlee
County Durham
SR8 1QR
Tel. No: 0191 3752596
Web Site: www.durham.police.uk
E-mail: freedomofinformation@durham.pnn.police.uk
Section 22A (1) (a) (i-iii) – Information intended for future publication
‘22A Research
(1) Information obtained in the course of, or derived from, a programme of
research is exempt information if— (a) the programme is continuing with a view
to the publication, by a public authority or any other person, of a report of the
research (whether or not including a statement of that information), and (b)
disclosure of the information under this Act before the date of publication would,
or would be likely to, prejudice—
(i) the programme,
(ii) the interests of any individual participating in the programme,
(iii) the interests of the authority which holds the information….’
Section 21 is an absolute exemption and does not require a public interest test. In this
case there are a number of academic articles available via open source searches on
the HART model and Checkpoint.
Section 22A is a qualified and class based exemption and requires a public interest
test.
Section 31 is a qualified and prejudice based exemption and requires the harm in
releasing information to be provided and a public interest test.
Section 40 is an absolute exemption, which does not require a public interest test,
but requires the balancing of the legitimate interests of the public against the
interests of the individual under the first Data Protection Principle (that of
‘fairness’).
This exemption applies because the right given under the Act to request official
information held by public authorities does not apply to the personal data of
third parties where disclosure of that information would not be fair to the
individual, and where there is no legitimate public interest in disclosure. In all
the circumstances of the case it has been determined that the duty to the
3
Information Rights and Disclosure
Professional Standards and Legal Services
Peterlee Police Office
St Aidans Way
Peterlee
County Durham
SR8 1QR
Tel. No: 0191 3752596
Web Site: www.durham.police.uk
E-mail: freedomofinformation@durham.pnn.police.uk
individual under the Data Protection Act 1998, and the public interest in
maintaining the exemption from disclosure of personal information held by the
force in such instances, outweighs the public interest in disclosure. In this
instance, personal information can only be disclosed to the individual
concerned. Releasing personal details to a person other than the data subject
would not only breach the data subject’s Data Protection rights it may also
breach the obligations placed on an authority under the European Convention
on Human Rights. In this case the personal data is sensitive personal data
pertaining to offenders’ custody event data and it would be unfair under the First
Data Protection Principle to release this personal data to the world at large.
Durham Constabulary acknowledges there have been many calls for new
regulation of algorithms, big data, artificial intelligences, machine learning, in
order to increase transparency and ensure accountability.
Releasing the HART algorithm could allow the model to be tested on other
datasets to demonstrate fairness of process.
In other jurisdictions where algorithms have been used, disclosure has been
provided. One example is Chicago Police Department’s release of information
surrounding its algorithm and Pro Publica’s (news organisation) algorithmic
bias analysis and findings.
The HART model and the Checkpoint Programme are both experimental ongoing
research in collaboration with Cambridge University. By the nature of such
experimental research there will be future developments as the process evolves. It is
anticipated that there will be future publications as academic papers and Force
information releases via its website on both aspects of the work.
4
Information Rights and Disclosure
Professional Standards and Legal Services
Peterlee Police Office
St Aidans Way
Peterlee
County Durham
SR8 1QR
Tel. No: 0191 3752596
Web Site: www.durham.police.uk
E-mail: freedomofinformation@durham.pnn.police.uk
The outcomes of the research programme are not yet known and are part of a
continuing research programme. Releasing information concerning the
programme prematurely and in a piecemeal fashion, would lead to misleading
information. Such misrepresentation would likely undermine individual
confidence in the diversion programme and have similar effect on public
confidence. The credibility of the research could also be damaged and detract
from the outcomes. During the implementation stage, this programme
demonstrated promising findings.
Given that the aim of the programme is to reduce offending and de-escalate
offending behaviours, there is a strong public interest in properly evaluating and
reporting on the outcomes. Reducing offending is a key objective of local
policing and national law enforcement.
In 2016, Checkpoint was recognised by the Howard League for Penal Reform
with a national award in support of the League’s principal aim of addressing the
underlying causes of offending. Checkpoint offers an alternative to prosecution
through the courts, by encouraging voluntary engagement with services
provided by a number of agencies. Please see the web link below for details of
this national award.
https://howardleague.org/community-awards/2016-community-awards-winners/
5
Information Rights and Disclosure
Professional Standards and Legal Services
Peterlee Police Office
St Aidans Way
Peterlee
County Durham
SR8 1QR
Tel. No: 0191 3752596
Web Site: www.durham.police.uk
E-mail: freedomofinformation@durham.pnn.police.uk
transparent at the point of being taken into account by a custody officer. The
human involvement in the decision making process in custody is paramount and
takes primacy.
Many factors are considered by a custody officer in arriving at any one decision
concerning a suspect in their care, HART forecasts are one factor. The
overriding principles for custody officer decision making are the bases of
necessity and proportionality.
http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedoc
ument/science-and-technology-committee/algorithms-in-
decisionmaking/oral/75798.html
The Government has recognised the calls for new regulation of algorithms, big
data, artificial intelligences, machine learning etc., in order to increase
transparency and ensure accountability. A House of Lords Select Committee
Inquiry on Artificial Intelligence was convened (‘What should the Government
do about artificial intelligence’) and the report from this Committee is now being
prepared after the conclusion of the recent oral evidence process. Also, the
Government convened a House of Commons Science and Technology
Committee Inquiry into ‘Algorithms in decision making’ and Durham
Constabulary volunteered/contributed to this committee in writing and was then
invited to give oral evidence on HART/Checkpoint on 12th December 2017. This
House of Commons Algorithms Inquiry is still ongoing and further oral evidence
sessions are ongoing. Durham Constabulary has stated publically its’ support
for national guidance in this subject area and also for an algorithmic harm
regulator, who would then be best placed to assess and adjudge any risks to the
rights and freedoms of a person. Please see below the web link to the Force’s
oral evidence.
http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedoc
ument/science-and-technology-committee/algorithms-in-
decisionmaking/oral/75798.html
Durham Constabulary supports the many observations made by the Alan Turing
Institute and by others, on the need for a regulatory body and/or another trusted
third party to oversee the use of algorithms in automated decision making or
6
Information Rights and Disclosure
Professional Standards and Legal Services
Peterlee Police Office
St Aidans Way
Peterlee
County Durham
SR8 1QR
Tel. No: 0191 3752596
Web Site: www.durham.police.uk
E-mail: freedomofinformation@durham.pnn.police.uk
Durham Constabulary would be prepared to reveal the HART algorithm and the
associated personal data and custody event datasets to an algorithmic
regulator.
HART will process personal data under Part 3 (Law Enforcement Provisions) of
the new Data Protection Act, as opposed to under Part 2 (GDPR).
HART could not be properly evaluated on other datasets. HART was specifically
written using Durham Constabulary’s custody data (sensitive personal data), for
the specific purpose of aiding decision making in the custody environment. This
would not be generalisable to any other Police Force or any other data
streams/data sets.
ii. The race bias concerns raised by Pro Publica were subsequently rebutted
by Northgate, the company that created the questioned Compas automated
decision making algorithm. Pro Publica subsequently rebutted Northgate’s
rebuttal, followed by the Wisconsin Supreme Court unanimously ruling in July
2016 that the State of Wisconsin could continue to legitimately use the Compas
algorithm (Wisconsin v Eric Loomis case).
The release of the full algorithm for the HART model could be used to identify the
technological capabilities within the Force and leave it open to possible corruption of
ICT systems by those intent on harming police capabilities. Such a breach would
impact on police resources through increased criminal activity. In turn this would impact
on the citizens of County Durham and Darlington. Knowledge of the full algorithm could
allow offenders to moderate their behaviour in order to avoid the due process of law
by seeking a diversionary schemes, such as Checkpoint. Improper use of part of the
triage system of identifying possible candidates for the Checkpoint Programme would
undermine the partnership approach to policing. This in turn would impact on public
expenditure for all agencies involved in the Checkpoint Programme. Ill-advised
speculation or publicity during the trial phases of the ongoing experiment would
disadvantage those who are already participating and would undermine those who
7
Information Rights and Disclosure
Professional Standards and Legal Services
Peterlee Police Office
St Aidans Way
Peterlee
County Durham
SR8 1QR
Tel. No: 0191 3752596
Web Site: www.durham.police.uk
E-mail: freedomofinformation@durham.pnn.police.uk
may be offered this opportunity to engage with agencies and a programme designed
to break the cycle of criminality. In turn this would undermine public and individual
confidence in Durham Constabulary.
8
Information Rights and Disclosure
Professional Standards and Legal Services
Peterlee Police Office
St Aidans Way
Peterlee
County Durham
SR8 1QR
Tel. No: 0191 3752596
Web Site: www.durham.police.uk
E-mail: freedomofinformation@durham.pnn.police.uk
Checkpoint has been recognised and has won a national award in 2016 from The
Howard League for Penal Reform, pursuant to its principal aim of addressing the
underlying causes of their offending. Please see the web link below for details
of this national award.
https://howardleague.org/community-awards/2016-community-awards-winners/
Releasing information in relation to the HART model would add to the public debate
on use of computers in decision making within the policing environment. Further
disclosure would demonstrate Durham Constabulary’s commitment to accountability
and transparency in relation to innovations around decision making and diversionary
schemes. It would further illustrate Durham Constabulary’s continuing commitment to
developing schemes, in partnership with other agencies, to reduce future criminal
behaviour by demonstrating the success of the selection processes that include, but
not exclusively, a computer assisted tool available to custody officers in the decision
making process by identifying moderate risk offenders who may benefit from the
Checkpoint Programme.
Durham Constabulary acknowledges there have been many calls for new
regulation of algorithms, big data, artificial intelligences, machine learning, in
order to increase transparency and ensure accountability.
Releasing the HART algorithm could allow the model to be tested on other
datasets to demonstrate fairness of process.
In other jurisdictions where algorithms have been used, disclosure has been
provided. One example is Chicago Police Department’s release of information
surrounding its algorithm and Pro Publica’s (news organisation) algorithmic
bias analysis and findings.
There is transparency around the use of HART and the standards expected
regarding custody officer decision making. The custody officer makes the
significant human custody/criminal justice decision and can be challenged via
applications through the courts and Criminal Justice system.
9
Information Rights and Disclosure
Professional Standards and Legal Services
Peterlee Police Office
St Aidans Way
Peterlee
County Durham
SR8 1QR
Tel. No: 0191 3752596
Web Site: www.durham.police.uk
E-mail: freedomofinformation@durham.pnn.police.uk
Very recently, further public domain information about HART and Checkpoint
has been given via the Force’s oral evidence, on 12th December 2017, at the
House of Commons Science and Technology Committee meeting on ‘Algorithms
in decision making’. Please see below the web link to the Force’s oral evidence.
http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedoc
ument/science-and-technology-committee/algorithms-in-
decisionmaking/oral/75798.html
HART will process personal data under Part 3 (Law Enforcement Provisions) of
the new Data Protection Act, as opposed to under Part 2 (GDPR).
ii. The race bias concerns raised by Pro Publica were subsequently rebutted
by Northgate, the company that created the questioned Compas automated
decision making algorithm. Pro Publica subsequently rebutted Northgate’s
rebuttal, followed by the Wisconsin Supreme Court unanimously ruling in July
2016 that the State of Wisconsin could continue to legitimately use the Compas
algorithm (Wisconsin v Eric Loomis case).
The public interest test is centred on whether information should be released to the
world at large and not just to you as the requester. The key test when considering the
public interest is to establish whether in all the circumstances of the request the public
interest in disclosing the information is not outweighed by that in maintaining the
10
Information Rights and Disclosure
Professional Standards and Legal Services
Peterlee Police Office
St Aidans Way
Peterlee
County Durham
SR8 1QR
Tel. No: 0191 3752596
Web Site: www.durham.police.uk
E-mail: freedomofinformation@durham.pnn.police.uk
exemptions.
I have considered the impact of releasing further information would have in relation to
the ongoing experimental research on the HART model and Checkpoint Programme.
It is important to recognise that the experimental use of the HART model is as one tool
in the assessment of a detainee for their potential inclusion in the experimental
Checkpoint Programme. Whilst there is considerable public interest in knowing about
the development of the HART model it would be premature to do so at present as this
is ongoing research and it is anticipated that findings will be published in the future as
explained above. It is recognised that releasing further information would add to the
public discourse on the use of computer assisted decision making in the policing
environment and in the diversionary schemes. However both HART and Checkpoint
Programme are developmental and as yet there is no definitive results of these
experiments to disclose, other than those already in the public domain.
Further I have considered the impact of releasing details of the algorithm as outlined
above. The police service utilise a number of information technology systems and to
release information that could lead those intent on harming the operational capabilities
of the police would not be in the public interest, as it would lead to increased crime and
impact on police resourcing. I have also considered the impact on offenders, on
Durham Constabulary, on the partnership approach to policing and the relationships
with partner agencies in relation to the delivery of the Checkpoint Programme and the
potential for changing criminal behaviour in those who agree to participate in the
programme.
Decision
Whilst the public interest considerations favouring disclosure carry weight, it is felt that
on balance the factors favouring non-disclosure are of greater importance. As the use
of HART and the Checkpoint Programme are still experimental and as such are subject
to development and change as the trial information on both is gathered and assessed
it would be premature to release further information which could lead to incorrect
assumptions or conclusions. In reaching my internal review decision, I can confirm
that I have disregarded the technological capabilities and possible corruption
harm arguments applied under Section 31 of the Freedom of Information Act
2000.
In accordance with Section 17 of the Freedom of Information Act 2000 this letter acts
as a Refusal Notice in relation to any further information.
Please note that Durham Constabulary’s response to your request is unique and
should not be used as a comparison with any other Force response you receive.
11
Information Rights and Disclosure
Professional Standards and Legal Services
Peterlee Police Office
St Aidans Way
Peterlee
County Durham
SR8 1QR
Tel. No: 0191 3752596
Web Site: www.durham.police.uk
E-mail: freedomofinformation@durham.pnn.police.uk
Yours sincerely
Leigh Davison,
Head of the Information Rights & Disclosure Unit.
12