Open science in

cognitive modeling
Caspar Addyman
c.addyman@bbk.ac.uk @BrainStraining
International Conference on Cognitive Modeling
University of Groningen
10 April 2015
To the extent possible under law, Caspar Addyman has waived all copyright and
related or neighboring rights to Open science in cognitive modeling. This work is

Overview
How I came to write a
Modeling Manifesto
What Ive done since
What we should all do next

Overview
How I came to write a
Modeling Manifesto
What Ive done since
What we should all do next
What I am going to do next

TRACX

French, Addyman & Mareschal, Psych.

Truncated Recursive Autoassocative

Chunk Extractor (TRACX)

t
a
h
W

T
A

A
L

?
B

(French, Addyman & Mareschal, Psych

http://leadserv.u-

http://github.com/YourBrain/TRACX-

Computational Modeling in Cognitive Science: A Manifesto for Change

Addyman & French, TOPICS, 2012

1.Let EVERYONE run your

simulations (easily)
2.Let motivated users adjust
parameters/ run their own
simulations
3.Let other modelers use your
code
4.Write good documentation
5.Clear mapping between model and
empirical data

Yes but..
Implementations are not
specifications
Cooper and Guest, Cognitive Systems
Research, 2013
1. Troubles with replicating modelling results
Guest and Cooper (2012) ICCM 11
2. Bit rot
a consequence of continuing changes in technology
code needs to be kept up to date.
3. Formal model specification
..more attention must be devoted to model specification
as an independent exercise to implementation.

Psychology PhD students

should know how to code.
Jay McClelland

It is true though that we dont

publish all our code right away. In
fact, some of our code gets tied up
in industrial projects which make it
hard to release.
Jurgen Schmidhuber,
Reddit AMA, March 2015

Playing ATARI with Deep Learning

Google DeepMind
Nature 26 Feb 2015
Released their code

Andrej Karpathy

http://convnetjs.com

Supporting information for

submissions in which software is
the central part of the paper:
The associated source code of the
software described by the paper. This should, as
far as possible, follow accepted community
standards and be licensed under a suitable
license such as BSD, LGPL, or MIT. Dependency
on commercial software such as Mathematica
and MATLAB does not preclude a paper from
consideration, although complete open source
solutions are preferred.
Documentation for running and installing the
software. For end-user applications, instructions
for installing and using the software are
prerequisite; for software libraries, instructions
for using the application program interface are
prerequisite.
A test dataset with associated control
parameter settings. Where feasible, results from

101 Innovations in Scholarly

Communication

Bianca Kramer & Jeroen Bosman(2015)

dx.doi.org/10.6084/m9.figshare.1286826

GAMIT-Net, CogSci 2014

http://github.com/YourBrain/GAMIT

Addyman & Mareschal, 2014

Caspar Addyman. (2014). GAMIT Model MATLAB Code.

Zenodo. dx.doi.org/10.5281/zenodo.9895

GitHub

Caspar Addyman. (2015). ContecCaspar Addyman. (2015). InfantPulse-Oximeter-in-Matlab: Version 1. Habituation-in-Matlab: Version 1.
Zenodo. 10.5281/zenodo.16277
Zenodo. 10.5281/zenodo.16627

iPython / Jupyter

Beaker
beakernotebook.com

Computational Modeling in Cognitive Science: A Manifesto for Change

1.Let EVERYONE run your

simulations (easily)
2.Let motivated users adjust
parameters/ run their own
simulations
3.Let other modelers use your
code
4.Write good documentation
5.Clear mapping between model and
empirical data

Computational Modeling in Cognitive Science: A Manifesto for Change

1.Let EVERYONE run your

simulations (easily)
2.Let motivated users adjust
parameters/ run their own
simulations
3.Let other modelers use your
code
4.Write good documentation

5.Clear mapping between

model and empirical data

 Try notebooks
Try Python
Try sharing

Questions for you

1. What is your
workflow?
Is it connected?
Is it shareable?
If not, why not?

2. How can we
narrow the gap
between models and
experiments?