Modeling Abstractions in Machine Learning

Ruiqiong Zhong1, Khaled Bizri 2

Abstract
The process of abstractions is reviewed in presentation of some of its different varieties. The
consequent broader involvement of the abstractions to include affect significant other processes
carried out by the mind is indicated offering extensions of the existing attempts at understanding
both, abstractions and some processes of the mind, almost suggesting that the existence of forms
of abstractions beyond that offered by the common understanding of the word itself.

An outline of the authors Theory of Mind (Zhong & Bizri, 2017) is presented and its fundamental
reliance on abstractions for reducing stimuli and constructing models are explored. The resulting
platform for further investigation and exploration of the minds processes in the context of machine
learning is outlined.

A description of the informatic model developed to conduct proving and validation of aspects of
the Theory of Mind is reported and directions for future promising research in this domain are
outlined.

On Abstraction and the Theory of Mind

The Theory of Mind proposes a meta-model for integrating the processes of the mind known in
cognitive psychology and cognitive neuroscience. Its primary goal is the provision of a platform
which accommodate further investigation of the mind, in particular through computational
procedures.

An exteroceptive stimulus and others generated by the brain, are reduced to an event. Events are
the smallest unit of memory. A sequence of events represents stages in cognition each of which is
processed and reduced according to the data received from the stimulus. This endows the event
with its environment and properties. Such presents a model which is stored and recalled upon the
recurrence of a principal event or events it accommodates. In turn, the model represents the other
unit of memory in addition to the event.

The reduction of events is accompanied by contributions solicited by the mind and leading to the
endowment of the events with subjective qualities derived from logical, emotional, procedural
and other values. The continuous communications and assessments between the event and other
parts of memory are essential components in the completion of the process of cognition.
1
Information School, Guangdong University of Foreign Studies, Quangzhou, China.
2
Stanford University, Stanford, California, USA

1
Models may be procedural, emotional or both. Most models carry an emotional property and that
include procedural models. Such qualities lead to reinforcement and endows the model with
further characteristics for the purpose of identification and recall.

Both events and models are subject to updates and even relegation to oblivion as they are recalled
reassessed through communications with various memories and in particular seeking the adherence
to the subjective logical dictates emanating from them.

Envelope Models
Dominant in the working memory is the presence of an envelope model. It provides a container
for the recalled components of memory. These super models accommodate recalled events and
models. The envelope represents principal aspects of the state of the mind at any moment, which
in turn affects the recall processes and color the recalled events and models with its own
environment and properties.

Like ordinary models, the envelope communicates with its currently resident components as well
as other parts of the mind, and it is through this process that recalls of events and models are made.
Indeed, the triggering mechanism mentioned above occurs as a reaction of the envelope to a
selected event reduced from the stimuli presented in the environment.

An envelope always exists in the working memory. It is replaced by the selection of an alternative
due to several factors, including biological ones affecting by chemical and the type of neural
configurations present at the time. Further work on the envelope model is necessary to determine
whether it is responsible for example, for what is commonly labeled as mood (Mayer, 1988;
Hume, 2012) or attitude, and the role it plays in influencing the structure and content of events
and consequent new models.

Attention
In this context, the envelope is the main source of general attention as opposed to the ordinary
models which result from the process of attention selection. The definition for attention (Triesman,
1960) acquires as such a mechanism for both types, the general one best described by simple
awareness. In that state, many stimuli are processes and ignored. The particular one which
deserved selection triggers the recall of associated events or models through a search for a
compatible model using as a key the particular event which caused the selection. A failed search
for the appropriate model causes the creation of a new model as is witnessed in processing an
event which carries a new learning environment for example.

The Lifecycles of Events and Models

In the theory, events and models represents fundamental memory units of perception, processing,
storage, recall, deployment and updates. Further, they are maintained, enriched, and updated as
competing events or model are perceived, constructed or are already stored during reflection and
consolidation. These processes take place throughout the lifecycle of the brain, in developmental
psychology (Wadsworth, 1996) and in several works in computational neuroscience (Churchland
and Sejnowski (2016).

2
Some of the events and models are however relegated to short memory and earmarked by the
envelope model for erasure and consequential disappearance from the knowledge base of the brain.

Event Reduction and Model Construction

In principal, the transition of an event into a component of model takes the schematic sequence
suggested in Figure 1. A stimulus is reduced to an event by inheriting its environment and
properties the two components which define the event.

Figure 1. The Sequence of Processing an Event

Attention itself may be construed as an activity of the envelope constraining perception to within
specific boundaries determined by the properties of the cascading stimuli. This defines an
important property of events in a model, that of association.

Such an envelope is in our theory, the model itself. The constraints referred to above are imposed
by the expectations of what the next event should convey for existing models, or the curiosity of
could follow in the case of construction of a new model.

That is, an essential part of the reduction of the stimulus to an event requires the detection of its
associations with previous event of events. Events which lack such an association arise, and are
either stored and removed from the attention envelope or indeed, the actual envelop, the model is
temporarily or permanently removed due to properties in the new event.

Abstractions
The term Abstraction commonly refers to the minds propensity to group several objects,
physical or ideal, which share specific dominant properties in relation to the intended group. The
theory present abstractions as a fundamental process of the mind, to the extent that we view it as
the foundation of a large and complex constructs than what the term itself usually evokes, (Lowe,
1995 and Saab, D. J. and V. U. Riss, 2009).

The process of constructing a model may appear divorced of the what is commonly understood by
abstractions, but the process itself creating a model is in itself an abstract process known to the
mathematicians and scientists in building their models of physical and ideal objects. There a
projective abstraction is performed, a top-down instead of a bottom-up process, Kelmke (1968),

3
where a consistent wrapper is created to accommodate objects which help explain a phenomenon
seen in more explicit form in in gestalt reasoning (Dumitru, 2016). Further, some models evolve
into abstract models when their constituent events are elevated to that level.

This view of abstractions offers as well, a practical interpretation of the basis of several processes
recognized in psychology, such as generalizations, "association," idealization, analogies
and "construction" even "projection," are each imbued with an abstract process of selecting
specific dominant attributes of events and including them in probable or veritable membership of
an abstract model grouped by psychologists under one of the abstract labels listed above.

There is no logical reason to assume that the minds ability at abstraction is limited to those
recognizable forms. Further, that the capacity at abstraction is identical among individuals or
indeed, mammals and birds as in the squirrel hoarding nuts or chickadees, nuthatches, jays and
some woodpeckers. Store or cache seeds for longer term consumption, (Hernsstein, 1968) with
birds and Vonk (2003) in the study of primates.

In this connection, it is noted that whether the mind possesses abstract templates from which it
proceeds to build its models or that it simply follows procedural steps, as some argue are logically
equivalent, favoring in our opinion that of a model.

Event and Model Abstractions

Events may be resolved by the mind into abstract events. The recurrence of events sharing
dominant properties, produce an abstract event which is recalled to accommodate a compatible
concrete event. The construction of an abstract event is often the result of instant or delayed
reflection or consolidation of a new event.

It is noted that the concrete events related to an abstract event are not necessarily eliminated and
are often recalled on the reduction of a stimulus. Abstract events are used more often in the
attempts to reduce events that have no compatible representation in memory.

The theory, particularly in its informatic implementation, presumes the existence of two-
directional links between the abstract event and each concrete instance of it, in much the same way
as the word Hot is related to the many instances of temperatures above that of the body, for
example.

The Informatic Infrastructure

The components of the informatic model are depicted in Figure 2. Stimuli are received and reduced
to events and models as sketched in the sequence diagram in Figure 1. A typical example used in
the validation of the model is that of seeing a friend in the crowd. The recalled model encapsulates
the abstract pattern of (safe) exchanges the mind may conduct with this external source of stimuli
associated with doses of input from emotional and other memories.

Events are stored in dynamic structures to accommodate their various specific data in the case of
an event elevated to an abstract level. Models are merely sets of two-directional links to the

4
events, one residing in the event structure and the other in the model itself. Their structure is
stored in the same manner as events in the repository.

Figure 2. The Structure of the Meta Model

The repository as such supports what tantamount an annotated glossary and thesaurus, supporting
the vocabulary of the meta-model gained from the various dialectical and alternate forms of
exchanges of the mind with the external world as well as the brain.

Recalling an event is performed using augmented search patterns3 The search pattern
accommodates lookups for compatible descriptors and produce through the procedure of nearest
neighborhood analysis the most likely corresponding event with a preset error margin.

Input is currently provided in the form of a structured json script (Jackson, 2016) using a special
vocabulary which simulates the perception of the interoceptive stimuli of the environment and the
properties of the stimulus and reduce it to human readable form to provide at the moment means
of proving hypotheses, replicating the sensory capabilities of the mind. It is hoped that this part
will be soon revised with the addition of digital sensory devices to enable higher forms of learning
by the system.

The Repository is currently built around the open-source Apache Solr search engine, modified to
support process of augmentation and validation. A thread continually resolve synonymic terms
which leads to the expansion of the vocabulary of the engine and variations in the presentation of
the stimuli.

3
In addition to the usual ranking of possible targets, the Augmented Search Engine conducts replacements of tokens
in the search string by alternatives to those gained in the preliminary results in order to explore differences prohibiting
possible matches. This provides a greater capability at detecting abstractions, according to the basic definition of the
term.

5
Results
Early results indicate that the system is able to produce its own abstraction of an object walking
on four and issuing particular sounds into dogs and cats reported as X89 and X77 in its own
nomenclature which appear to be different to still unresolved X76 a tiger and X91 a leopard of
which it only had two events each, to the extent that if the stimulus is the sound made by cat or
a dog, the system log display accurate identification.

Other investigations demonstrated the recall of events and models upon the repetition of a cascade
of stimuli as well as the display of frenzied effort seen in the system log to resolve an expected
stimuli in the mode against the unexpected arrival of another.

Work on the computing model is continuing both in two directions, the first relates to maintaining
the accommodation of the findings in Cognitive, Psychological and Philosophical domains
consistent with the adoption of the mathematical process for depicting the minds abstraction
process, and the second in implementation and tests of further levels in abstraction as suggested
by the results of the tests.

Summary
The informatic model for the Theory of Mind provided a platform for validation of current analysis
in replicating aspects of the minds process based on the segmentation of the process of knowledge
acquisition into its stimulus, event and models. This was attested to by the wider incorporation of
abstractions, leading to the machines own inclination to produce abstract events and in auto-
creating models based on associated events.

Early results validate the fundamental assumptions, while work on endowing the machine with
other and more complex processes derived from the Theory of Mind continues, particularly in the
directions of the envelope model and in enhancements to self-learning based on the principles
outlined in the Theory of Mind.

Models and events establish a platform for integration of findings and hypothesis in variety of
disciplines related to the understanding of the mind. Effort to replicate several of the more
established and replicable experimental work in cognitive neuroscience and computational
neuroscience (Scott Kelso, 1995 and Daw and Doya, 2006) form the next phase of this work, where
the foundation of the meta-mode, abstractions will be subject to further analysis and investigation.

References
Churchland, P. S. and T. J. Sejnowski (2016), Blending Computational and Experimental
Neuroscience, Nature Reviews Neuroscience, 17, 667668; doi:10.1038/nrn.2016.114.

6
Daw, N. D. and Doya, K (2006), The computational neurobiology of learning and reward, Current
Opinion in Neurobiology, 16:199204.

Dumitru ML and G. H. Joergensen (2016), Gestalt Reasoning with Conjunctions and Disjunctions.
PLoS ONE 11(3): e0151774. DOI: 10.1371/journal.pone.0151774.

Herrnstein, R. J., and Loveland, D. H. (1964). Complex Visual Concepts in the Pigeon, Science,
146, 549-551.

Hume, D. (2012) "Emotions and moods." In Organizational behavior, catalogue.pearsoned.co.uk:

258-297.

Jackson, W., (2016) JSON Quick Syntax Reference, Apress, ISBN 1484218639.

Lowe, E. J. (1995), The Metaphysics of Abstract Objects, The Journal of Philosophy, Vol 92, Issue
10: pp. 509-534.

Mayer, J. D. and Gaschke, Y. N. (1988), The Experience and Meta-Experience of Mood, Journal
of Personality and Social Psychology, Vol 55, No 1: pp. 102-111.

Saab, D. J. and V. U. Riss (2009), Logic and Abstractions as Capabilities of the Mind, European
Conference on Computing and Philosophy, E-CAP 09, Madrid, Spain.

Scott Kelso, J. A. (1995), Dynamic Patterns, The Self-Organization of Brain and Behavior,
Bedford Books, MIT Press.

Triesman, Anne M. (1960), Contextual Cues in Selective Listening, Quarterly Journal of

Experimental Psychology Vol. 12, Issue 4.

Zhong, R. and Bizri K., (2017) Computational Representation of the Theory of Mind, Computers
and Education Elsevier. Submitted January, 24, 2017.

Vonk, J., (2003) Gorilla (Gorilla gorilla) and orangutan (Pongo abelii) Understanding of First-
and Second-Order Relations, Animal Cognition, Springer 6:7786; DOI 10.1007/s10071-003-
0159-x

Wadsworth, B. (1966), Piaget's Theory of Cognitive and Affective Development: Foundations of

Constructivism, 5th ed., Longman Publishing.