MEMORY

> BRAIN VS THE COMPUTER

Memory management is an essential skill that every programmer develops

over time. To develop that skill you need to have a sense of how computer
memory works. Surprisingly, memory is not a simple thing. It is actually
quite complex and only a few experts actually understand all of the details.
In this post, I make an analogy between computer and human memory to
explain the basics of both. This leads me to dive into the core of both
computers and human minds as these are deeply intermingled.
As stated by wikipedia, memory refers to the physical devices used to store
programs (sequences of instructions) or data. Both computers and humans
have multiple forms of memory. I dont know whether human memory
inspired the design of computers but the similarities are striking as you are
going to see.

RAM or Random Access Memory and the

Hippocampus

RAM or Random Access Memory

When someone refers to computer memory, it usually means RAM. RAM

or Random Access Memory was designed to provide a quick access to series
of numbers. To limit access time, RAM is organized around arrays of
electronic components that allow very quick access to any location in the

memory. The main advantage of RAM is speed (typical RAM bandwidth today
is around 5-10 GBytes/s). The constraints on speed have historically limited
the overall size of available RAM on computers (today typical RAM memory is
around 8 GBytes). The main disadvantage of this form of memory is its
volatility : you turn off your computer and it is gone.

Where is the hippocampus in our brain?

Humans do have a form of RAM and it is called short-term memory.

Interestingly, short-term memory is usually associated with a very specific
region of the brain called thehippocampus. Short-term memory is the
memory you will use to recall immediate actions (like the series of words you
are actually reading).

The hippocampus by Camillo Golgi

We dont really understand how the hippocampus does it but I have always
been puzzled by this structure homogeneity. It is composed of billions of cells
that are very densely packed, like on an array as can be seen on the Golgi
staining in the attached image. Reminds you something?
It is traditionally thought that the memory content stored in the
hippocampus is transferred to another brain area for long term storage. This
process is supposedly happening when you are turned off (your sleep). Again
another similarity..

The hard drive and the cortex.

A hard drive from Western digital

As for the hippocampus, RAM storage was too volatile so engineers came out
with a complimentary form of storage for long term memory in computers :
the Hard Drive or HD. Hard drive have low bandwidth (usually 100 MBytes/s)
but can store much more (computer usually comes with 500 GBytes of hard
drive). This is where all of your data and programs is located.
Human long-term memory is presumably located in the cortex, exactly why
and how is a matter of intense research nowadays. I am sure you have
experienced that recollecting old memories can take quite some time
(sometimes minutes). This process is usually variable as it is when reading
from the hard drive. Hard drive can have very different read/write speed
depending on where the memory is stored.

Long term memory is supposedly transferred to the cerebral cortex via the hippocampus.

As for humans, when you put a computer to sleep, RAM memory is

transferred to the hard drive for long term storage. This process is reversed
when you turn it on to make the best usage of the fast RAM. I suppose the
same thing is happening when we wake up in the morning (in my case, this
process is especially slow).
So far so good. Nothing too fancy here as the distinction between RAM and
HD is usually known.
There are still 3 forms of computer memory that we need to understand, so
hold on.

CPU Cache and neuronal network attractors

Transferring memory from the RAM to the CPU cache

Computers have something called CPU cache. CPU or Central Processing

Unit is probably the main thing a computer have that we dont. This is the
center of your system where all calculations are being done. In essence a
CPU is just a ton of operators packed in a very tight space, they can combine
bits of information in very specific ways. Thanks to decades of work, CPU
operates at very very high frequencies, in the GHz range. Most CPUs are now
64 bits which means they can process 64 bits of information at each clock
cycle. Because the frequency is so high, it became essential to have a high
bandwidth to provide information to the CPU. If you check the numbers, it
appears that the RAM is not fast enough to feed these monsters (the order of
magnitude is 24 GBytes/s). Thats what the CACHE was designed for. The
total memory content of the CACHE is very small as we really want this thing
to be dead-fast (by small I mean in the MByte range). The CACHE is a very
little memory space packed at the close proximity of your CPU to limit access
times. There are multiple levels of cache (L1, L2 as depicted on the
associated image but there are more) to provide intermediate levels of
storage to make the CPU work smooth and organized.
Our brain is not clocked at such speeds. A typical neuron emits a new spike
at about 10-40 Hz. While it can go higher, we can say we have a 40 Hz clock
running in our skull. This is why our computer screens are only refreshed at
60 Hz. Why would you go faster, we dont see it

A neuronal network is a distributed cache memory

But as for more modern computer architecture, things are done in parallel in
our little head. With such a low clock speed, information is transferred very
slowly from one side of the head to the other so we cant afford to have a
single memory storage.
Indeed, Memory in the brain is distributed all other the place. I see the cortex
as a gigantic CACHE storage. Each bit of information that is processed wake
up the local neuronal network, reactivate its memory and compare this
memory with the provided input. The beauty of neuronal networks is that
both the program (what we usually script in Matlab in M-files) and the
associated memory (the program parameters or your grandmother face) can
be stored in the same place.

Protected memory and explicit memory

Human
memory
is
further
divided
into
two
called explicit and implicit memory. Explicit memory is the
consciously using, like when you remember where you left
the day before, while implicit memory is not necessarily

major
classes
memory you are
your doors keys
conscious : you

know how to walk but you dont have to think to walk. Still you had to learn
and store it somehow. I have no idea how this separation is done at the
neuronal level but I do know that something very similar exists in computers
and its called memory protection. Memory protection is built-in in any
modern OS. When you write a program that needs some memory for its
behavior, the OS allocates some space to it. It does so with some protection
mechanism to prevent the program from accessing any other memory. This
technique obviously prevent bugs and promote overall stability of your
system. It does seem to me that explicit and implicit memory fullfill the same
purpose : stability of our brain Operating System and protection of
important memories.

Memory swapping and writing

It sometimes happens that a program asks for more memory than what is
available in the RAM. When this happen, your operating system has no other
solution than to use whats available : your hard drive. In that case, your OS
will allocate some space on the hard drive for your program to work. This is
usually refers as swapping. Since hard drive are several order of
magnitude slower, this leads to great loss in performance. This is probably
the number one reason things can go slow within Matlab. Your program will
not be able to tell if its data is stored on the hard drive or in the RAM as this
process is seemingly transparent. I personally believe that if your program
start to swap, it is because your program was not designed properly. It is
your job to be able to work within the memory limits of your computer.
Swapping does not exactly exists for human mind as our mind was extremely
well designed to work within its memory limits. However, in some ways we
do swap all the time.
Indeed, what do you think you are doing when you take notes during a class?
The memory content of the class is too high for your brain to take it straight
like this. You need an intermediate data storage device to allow you to
assimilate the new data set. So you swap the data to a piece of paper using

a pen. In some way, we human compensated our lack of a swapping device

via writing..