This paper introduces the concept of Open Source Intelligence (OSINT) as an important

component for understanding human problem solving in the 21st century. OSINT is in many
ways the result of changing humaninformation relationships resulting from the emergence
and growing dominance of the Internet and the World Wide Web in everyday life. This paper
suggests that the Internet/Web changes the dynamic relationship between what Cattell and
Horn have identified as the two general factors of human intelligence: crystallized
intelligence and fluid intelligence. The Internet/Web open up new possibilities for accessing
information and transcending over-determined cultural intelligence in problem solving. This
offers fluid intelligence, which often trails off in adulthood, a new vitality across the lifespan.
But the diminishment of crystallized intelligence, and especially cultural intelligence, also
presents a number of important problems in maintenance of cohesive, social cooperatives.
The development of OSINT (using tools and ethos created by the Open Source movement of
the last few decades) offers both a framework for reaching beyond the boundaries of
traditional cultural intelligence and ways to create cooperative, open, problem solving
communities. The Internet/Web will continue to create confusion and fear as we move deeper
into this new age, but also presents extraordinary possibilities for augmenting human intellect
if we can understand it and learn to harness its potential.
The revolution in information technology is making open sources more accessible,
ubiquitous, and valuable. The International Intelligence Communities have seen open sources
grow increasingly easier and cheaper to acquire in recent years. Up to 80% of electronic data
is textual and most valuable information is often encoded in pages which are neither
structured, nor classified. The process of accessing all these raw data, heterogeneous for
language used, and transforming them into information is therefore inextricably linked to the
concepts of textual analysis and synthesis, hinging greatly on the ability to master the
problems of multilinguality. This paper describes SYNTHEMA SPYWatch, a content
enabling system for OSINT,which has been adopted by some Intelligence operative structures
in Italy to support the collection,processing, exploitation, production, dissemination and
evaluation cycle. By this system, operative officers can get an overview of great volumes of
textual data,which helps them discover meaningful similarities among documents and find all
related information.
The challenges of developing OSINT are its "volume, variety, veracity, and velocity."
Regarding volume, the Internet and other dynamic sources of information hold massive
amounts of material that must be managed, organized, and identified as relevant to
intelligence acquisition and analysis. The second challenge is the variety of packages in
which the information comes. These packages include text, imagery, video, audio, domestic
and international public records, news articles, scientific journals, dictionaries and
taxonomies, etc. Veracity is a third challenge of OSINT. This involves an assessment of the
accuracy and truthfulness of information, which consists of identifying and qualifying the
reliability of information sources. The fourth challenge of OSINT is velocity, which pertains
to the rapid production of new and changing information. This article outlines four phases for
managing the challenges of OSINT: acquisition, refinement, fusion and analytics, and access
and delivery. Acquisition involves identifying the sources of data that are likely to yield the
types of information being sought. Refinement includes several steps designed to filter the
base amount of data collected so that key elements are discovered, extracted, and processed.
Data fusion and analytics refers to the rapid merging and integration of many datasets, files,
and records. This involves technologies known as pattern analysis, data enrichment,
clustering, linking, and detecting obscure relationships. Analytic capabilities provide the user
with a filter that screens out all but the most useful information. The final phase, access and
delivery, involves disseminating or sharing the information in the most appropriate manner
with a broader community of interested parties.
Purpose The paper seeks to show how the increasingly popular use of data and information
acquired from open sources (OS) impacts competitive and marketing intelligence (C/MI). It
describes the current state of the art in analysis efforts of open source intelligence (OSINT) in
business/commercial enterprises, examines the planning and execution challenges
organizations are experiencing associated with effectively using and fusing OSINT in C/MI
decision-making processes, and provides guidelines associated with the successful use of
OSINT.
Design/methodology/approach This is a descriptive, conceptual paper that utilizes and
develops arguments based on the search of three unclassified bodies of literature in
competitive and marketing intelligence, intelligence processing and marketing analysis.
Findings Open sources are useful in marketing analyses because they can be easily
accessible, inexpensive, quickly accessed and voluminous in availability. There are several
conceptual and practical challenges the analyst faces in employing them. These can be
addressed through awareness of these issues as well as a willingness to invest resources into
studying how to improve the data gathering/analysis interface.
Practical implications Marketing analysts increasingly rely on open sources of data in
developing plans, strategy and tactics. This article provides a description of the challenges
they face in utilizing this data, as well as provides a discussion of the effective practices that
some organizations have demonstrated in applying and fusing open sources in their C/MI
analysis process.
Originality/value There are very few papers published focusing on applying OSINT in
enterprises for competitive and marketing intelligence purposes. More uniquely, this paper is
written from the perspective of the marketing analyst and how they use open source data in
the competitive and marketing sense-making process and not the perspective of individuals
specialized in gathering these data.
An ocean of data is available on the web. From this ocean of data, information can in theory
be extracted and used by analysts for detecting emergent trends (trend spotting). However, to
do this manually is a daunting and nearly impossible task. We describe a semi-automatic
system in which data is automatically collected from selected sources, and to which linguistic
analysis is applied to extract e.g., entities and events. After combining the extracted
information with human intelligence reports, the results are visualized to the user of the
system who can interact with it in order to obtain a better awareness of historic as well as
emergent trends. A prototype of the proposed system has been implemented and some initial
results are presented in the paper.