Crowd Knowledge as a Service - Conceptualization of

Crowdsourcing System

Vijay V. Patil,
Manager, Accenture -IDC
Email: vijay.v.patil@accenture.com
Phone: 09766531891

Crowd Knowledge as a Service - Conceptualization of

Crowdsourcing System

Abstract. Organizations are interested to start Crowd Knowledge as a Service model to

outsource tasks, which are traditionally performed by a small group of people, to globally
distributed talent for quicker and low cost solutions. This research paper attempts to gain a
better understanding of what crowdsourcing systems are and what typical architecture
aspects are considered in the development of such systems.
In this paper, the researcher conducted a literature review in the domain of crowdsourcing
systems. As a result, the researcher derived and presented requirements & functional
architecting aspects which need to be implemented in a crowdsourcing system. Examples of
successful crowdsourcing use cases are presented with benefits and challenges.

Keywords: crowdsourcing, crowdsourcing system, crowd Knowledge as a Service, Human

Processing Unit

1. INTRODUCTION

2.

3.
The crowd knowledge as a service
is not a new concept. Outsourcing is not
only limited to organisational bodies but
has taken a new shape. When companies
need solutions to seemingly insoluble
problems, they now can turn to online
marketplaces where they can find
innovative and creative problem solving
ideas.
4.
5.
With the number of people online
approaching 3 billion by 2016 and
projected to reach 5 billion by 2020, a new
workforce has emerged that leading
organizations are now harnessing.
Available on-demand, this workforce has
abundant capacity and the expertise and
knowledge to perform work from simple to
complex, and solve problems and grand
challenges. This innovative model, distinct
from traditional outsourcing, presents a
new opportunity to create a more
competitive market position. In 2011 the
crowdsourcing market growth rate was
75%, exceeding 2010s market growth of
53% (crowdsourcing.org, 2011)
6.
7. Jeff Howe coined the term
crowdsourcing in a seminal
article appearing in Wired
magazine (Howe, 2006), defines it
as ...the act of taking a job
traditionally performed by a
designated agent (usually an
employee) and outsourcing it to an
undefined, generally large group of
people in the form of an open call.
According to Howe (2009)
crowdsourcing works because
crowdsourcing draws from a global
pool of talent, much of which has
never before been tapped
effectively and crowdsourcing
allows genuine meritocracies to
emerge - where people are

acknowledged for the quality of

their ideas rather than for their
formal academic qualifications. All
that matters is the final product,
not the backgrounds of those who
contributed to it. Crowdsourcing
directly attracts this new breed of
amateurs; these are people who
are not primarily motivated by
money directly, although theyre
happy to make a few extra bucks if
the chance arises. These are also
the kind of people who are working
on their own dime and who
dedicate their leisure time to
something they feel passionate
about, something they love to do
rather than have to do.
8.
9. The second section of this paper
includes crowdsourcing system
functional requirements with
popular crowdsourcing site
examples. Section 3 contains high
level conceptual architecture of
crowdsourcing system with detail
description of each functional
component in it. Section 4 states
the step by step implementation of
crowdsourcing. Crowdsourcing
success stories are presented in
section 5. Benefits & Challenges of
crowdsourcing are listed in Section
6 &7.
10.Crowdsourcing System
requirement
11. Crowdsourcing is a process for
sourcing a task or challenge to a
broad, distributed set of
contributors. It Includes
mechanism to attract the desired
participants, stimulate relevant
contributions and select winning
ideas or solution. Crowdsourcing
System act as an agent which

distributes the crowdsourcing

tasks that are issued by the
requesters or employer to the
potential recipients, crowd. Human
brainpower and collective
intelligence are the main drivers of
Crowdsourcing System.
12.
13. 2.1 Crowdsourcing HPU: Human
Processing Unit
14.
15. Crowdsourcing core concept is
built on Human computation (HPU)
of channeling the vast Internet
population to perform tasks or
provide data towards solving
problems. Human computation is
an elastic human workforce which
provide on demand talent capacity
for crowdsourcing. Pay for the
performance & not for efforts, is a
great feature of crowdsourcing.
Market bears the cost of failure
and not a provider. Crowd sourcing
has the potential to consumerize
the innovation by collaborating
new ideas, products and services
to market.
16.
17.

18. Fig 1 - Crowdsourcing Human

Processing Unit (HPU)
19.
20. 2.2 Functional requirements
21.

Right Model Selection: Mainly

Crowdsourcing can be used in three
models, first model to create new
ideas for innovation called ideation
process; second model is to outsource
simple, least important micro tasks.
The third important model is
outsourcing skilled task which can be
better performed by selected skillful
crowd if task design is accurate
22.
Right Crowd selection: Create crowd
profiling base on expertise area, the
analysis of social activities, social
relationships, and socially shared
contents helps improving the
effectiveness of crowd profiling. Public
Profile information is generally less
effective than information about
resources that they directly create,
own or annotate (A. Bozzon, 2013).
23.
Provider should able to leverage crowd
to target questions by configuring
polls, discussion topics, open-ended
queries, ideas & tasks. Task explicit
purpose need to be communicated
clearly with target crowd
24.
Provide incentives, motivational
mechanism and simulation to
participants. Dont be too cheap,
crowdsourcing is cheap, but it follows
the same formula of other jobs, the
more you ask from the crowd, the
more you have to give in return
25.
Provider should able to View responses
in real time, which should allow
optimizing subsequent questions or
actions based on those responses
26.
Evaluation of ideas/tasks and validate
the ones which need to progress in a
consistent and seamless manner.
System should have a way of verifying
the results. When outsourcing to a
large crowd of non-professional
workers, the results can vary greatly.
27.
Crowdsourcing system should come
close to providing qualitative and
quantitative findings/outputs as in
traditional development. Also help to
increase productivity, reap the benefits

of an elastic workforce, and reduce

costs
28.
Create new strategies, opens up new
possibilities, accelerate go-to-market
times and improve work performance
29.
29.3
Popular crowdsourcing sites
30.
Amazons Mechanical Turk
(www.mturk.com/mturk/welcome)One
of the most popular crowdsourcing
sites, useful for simple tasks like
tagging photos, categorizing products,
etc.
InnoCentive (www.innocentive.com) is
for those looking to solve complicated
questions, but it charge accordingly.
The company claims to deliver
breakthrough ideas and solutions at
lower cost, in shorter time, and with
less risk than previously possible.
99Designs (www.99designs.com) A
crowdsourced marketplace for all sorts
of graphic designs, where buyers post
contests for anything from logo to
website design, and a large pool of
designers submit their designs and bid
on the project in an open competition.
Freelancer (www.freelancer.com)
Famous crowdsourcing site which
offers online jobs to freelancers from
around the world for everything from
design to web development to
programming. Companies or
individuals post projects for a set price,
where after freelancers bid on the jobs
and offer their services
Utest (www.utest.com) Utest is a
company that offers software testing
that relies on crowdsourcing. Software
makers use Utests service to test their
desktop or mobile software with large
amounts of users. The tests include
security testing, functional testing,
usability testing etc.
31.
32.Crowdsource System
Conceptual Architecture

33. As a result of literature review; the

researcher could derive three
major architecture layers listed in
below Fig.1 user management,
task management and contribution
management. Each layer further
consists of functional components
which need to be developed while
building crowdsource system (Lars
Hetmank, 2013).
34.
35.
36.

37.
38. Fig 1 - Conceptual architecture

39. 3.1 User management

40. User management functions to
register users, evaluate users, to
form user groups for different
purposes, and to establish
coordination mechanisms among
the users
Register User, A user profile may
record both the user identity of the
worker and of the requester. To
improve the trust between workers
and requesters, the crowdsourcing
identity may also be associated with
public profiles on social network sites.
The analysis of social activities, social
relationships, and socially shared
contents helps improving the
effectiveness of worker profiling
Evaluation can be done
before user start the first task or after
they have finished a task. The former
applies entry questions,
prequalification tasks or gold standard
data to determine the expertise or skill
level of a worker. The latter considers
acceptance and rejection decisions of
historic contributions. System should
do the automated evaluation based on

above criteria and allocate some score

or rank.
Group formation, Different
types of tasks may require different
amounts of people. Sometimes, only
one individual per task is needed; in
other cases a closed group which has
specialized skills is necessary to solve
the problem and again in some cases
the whole open community is asked to
find a solution.
Enable coordination,
mechanism to collaborate between
crowds or from crowd to provider to
ask any clarification and provider to
crowd to give the feedback.
41.
42. 3.2 Task management
43. Task management to handle the
incoming submissions of tasks and
their distribution to the crowd that
will solve the task
Design Task. The quality of
the contributions highly depends on
the task design. Cheat submissions
can be prevented if the task is defined
appropriately Task size base on type
like simple, complex , novel and
reward associated with it, submission
time, the degree of confidentiality and
target crowd need to be defined
clearly. Additionally, the provider user
profile and other contextual
information, such as the location or
time may be automatically assigned to
the task specification. This information
may support the interpretation of the
task by the crowd
Assign Task. Allocating the
right task to the right person at the
right time is a key for the success of
crowdsourcing projects. Intelligent task
routing, where workers are selected
based on the task specification and the
user profile. Before assigning a task to
the crowd make sure the worker has
sufficient skills and knowledge to
accomplish the task and choose an
appropriate time when the worker can
or is willing to work
Workflow: Coordinates among the
inputs and the outputs of independent
human or machine functions in order
to get an optimal result. To design
complex tasks with global

requirements and constraints.

Crowdsourcing workflow to combines
the issued task, the contributions of
the crowd and powerful crowdsourcing
algorithm
44.
45. 3.3 Contribution management
46. Contribution management heavily
relates to quality control and
contains functions that evaluate,
pre-process, combine and select
solutions of the crowd
Evaluation plays a central role in
providing feedback to the task solver
in order to increase quality as well as
in selecting the best result from a large
set of solutions. Source of evaluation
need to be identified, it can come
either from another expert from crowd
or from requester. Feedback can be
given simultaneously while the workers
are still involved in the task, or
asynchronously after the task is
completed.
Selection algorithm/methods may be
used to validate, detect cheat
submissions and to sustain quality of
the final result. Selection can be based
on majority decision approach or
taking help of control group by giving
multiple validation tasks as a request
to rate the submitted solution.
47.
48.Crowdsourcing Implementation
Steps
49. When seeking to engage external
talent, one of the first of many
questions one must first ask is:
Why are we doing this? What do
we hope that external talent can
achieve for us that our internal
talent cannot (or should not)
achieve, and how do we integrate
the two together? (Massolution,
2013)
50.

51.

52. Fig 2 Implementation steps

53.
54. Plan: Companies that want to
take advantage of crowdsourcing
need to identify the tasks that
would give them
55. the most benefit at an attractive
level of effort. Companies need to
identify tasks that fill that criterion.
A client-user may have a strong
sense of a problem that needs to
be solved and, can drill down to
the precise tasks that could benefit
from flexible supply and outputbased pricing. The user
organization also needs to consult
with all stakeholders who will be
affected by the efficiencies
crowdsourcing will bring.
56.
57. Design: Crowdsourcing normally
involves task redesign. The
traditional way of performing tasks
is often optimized for an in-house
workforce. But in crowdsourcing a
supplier is looking to match a task
design with an appropriate worker
as well as to allocate tasks in a
way that guarantees privacy and
security to the client. That might
mean breaking a task down in new
ways so that task elements can be
assigned to different workers
having more targeted skill set.
Naturally, then, an integral part of
the process is to build profiles of
the types of workers the tasks
need.
58.
59. Implement: The crowdsourcing
provider need to scale the solution
by recruiting a qualified crowd.

60. Before launch, the tasks are fully

bedded into the teams routines,
including the QA aspects. The
client dashboard is finalized and
tasks are released to the crowd.
Provider should monitor the crowd
and track the progress of task.
Appropriate progress & benefit
reports need to be published at
regular interval.
61.
62.Crowdsourcing Success Stories
63.Open Innovation Creates
Greeting Card Success Story
(Ideaconnection, 2013)
64.
65. United States based greeting cards
making company decided that it
would create a focus group on the
Internet and it would ask these
people questions about how they
liked the designs of cards and
whether they would pay this
amount of money for a greeting
card. But one of the things that
company did not expect was that
people started designing their own
greeting cards and sending them
to company. They were saying,
We do not like your greeting cards
for birthdays, here is a much
better idea, and company started
to agree. It found there is a whole
pool of talented artists and graphic
designers who, for no cost at all to
company, were sending it designs
for greeting cards. And company
said it was saving something like
US$50 million a year because its
customers were designing their
own greeting cards.
66. The customers started sending
ideas for a whole new range of
greeting cards, things that
company had never even thought
about. The customers were
improving the current product lines
of company; they were actually
creating new product lines at no
charge. Even if they had signed
away their intellectual copyright,
the customers still seemed to be
happy with the outcome because
they were getting the recognition

for designing a card. What

company did was to make sure the
person who contributed the idea
got his or her name on the back of
the card Designed by . . . of the
company consortium. This was
crowd interaction that was actually
creating new businesses for
company.
67. This small crowdsourcing story
shows how you can actually get to
a point where the customer is
working with you and for you, and
you are not merely seeing a
customer as just an object for a
sale
68.
69.Supporting better regulatory
compliance through a Captive
Crowd (Massolution, 2013)
70.
71. A well-known global software
company supplies ERP software to
its global client base. This software
is designed to allow users to
comply with local accounting
regulations, and has to be up-todate and accurate. Imagine an ERP
system that gives wrong or
outdated guidance on tax
compliance, for example. In
addition the software provider has
to provide the information for its
own compliance needs. The
traditional way of dealing with this
problem, which involves multicountry, multi-lingual regulatory
research, is to rely on
management consulting firms who
offer global coverage. They are an
expensive option, tend to delegate
to juniors, and are not necessarily
in a position to provide the best
data. However, to date, they have
been the only option.
72. Crowdsource system provider has
created a private crowd of
accounting and systems experts in
all the countries that the software
provider needed to cover. The
Crowsource system platform
allowed that new network of
experts to be assembled very
quickly, and to cross-validate

expert advice as part of the

workflow.

73.
74.Secure Data Entry (Massolution,
2013)
75.
76. The ABC Department of Revenue
processes over 10 million tax
returns a year and the quality of
essential work began suffering
during tax season, which in turn
led to increased workloads for
examiners in the later stages of
the tax cycle. However, the
addition of a second shift of
seasonal workers created further
financial and managerial strains.
Revenue department needed to
eliminate the cost and burden of
staffing and supervising seasonal
workers with a solution that would
also protect sensitive tax
information.
77. Crowdsourcing Provider data entry
solution, chosen by Revenue
department, offers a secure and
cost effective alternative to
staffing a full shift of seasonal data
entry personnel. With Snippet
technology, sensitive paper
documents and scanned, fullcontext form images never leave
the borders of Revenue
department. Breaks tax documents
into digital image Snippets.
78. The Snippets are assigned a
random ID, scrambled to
guarantee security, and sent over
a secure internet connection to
remote Cloud network. The
Snippet images are then
dispatched to data entry workers
who enter and verify the data
using a web browser with a secure
connection. Using the Secure Split
Technology, fields containing
sensitive information, such as
Social Security numbers and credit
card numbers, are further
separated into smaller Snippets.
All Secure Split Snippets are
distributed to different crowd
workers to protect further the
confidentiality of the data. To
ensure a 99.99% accuracy level,

each Snippet is validated by at

least two independent sources
before the data is returned to
Revenue department system.
79. The Revenue department now
handles the peak processing
demands of tax season without the
cost and burden of training and
supervising seasonal workers.
Document turnaround time has
improved 70%. In addition,
Revenue department has cut costs
and eased data processing
burdens across all department
operations while improving data
accuracy.
80.
81.Benefits of Crowdsourcing
Cost-effectiveness, crowd sourced can
be done at a fraction of the cost of
traditional onsite/offshore sourcing.
Quick Turn Around, The time it takes to
gather, execute, and analyze is
shorter.
Flexibility, as trends emerge in
findings, requester can easily adjust
their strategy to catch any shifts or
surprises.
Problems can be explored at
comparatively little cost and Payment
is by results (Output base pricing
model).
The organization can tap a wider range
of talent than might be present in its
own organization
Turn customers into designers and/or
marketers
82.
83.Challenges with Crowdsourcing
Quality, cheap labor can results less
credible product, compared to
professionals
Have to manage a large scale of
workers, which need more time for
management.
Challenge in crowd members
collaboration as they compete with
each other in nature
As no contract in most crowdsourcing
cases, risk of incomplete work and
reuse of same work for others may
results in Intellectual property leakage
Choosing what to crowdsource & what
to keep in-house can be done by the
complete work

84.
85.
86.CONCLUSION
87.
The purpose of this paper was to
gain a better understanding of how crowd
knowledge can be used as a service using
crowdsourcing system, what are critical
exceptions from crowdsource system and
typical design aspects have to be
considered in the development of such
systems.
88. Crowdsourcing is not a silver bullet
for commerce. Its not a magic pill
which will make all commercial
challenges fade into oblivion.
Rather, crowdsourcing harnesses
the power of todays
communication technologies to
liberate the potential which exists
in large pools of people. It will shift
the way work gets done. This
collaboration for open innovation
with online communities can put
companies ahead of the
competition and save millions of
dollars that is usually spent for
research and development. Smart
companies are learning that
openness is a force for growth and
competitiveness
89.
90.REFERENCES
91. A. Bozzon (2013). Choosing the
Right Crowd: Expert Finding in
Social Networks. EDBT/ICDT 13
March 18 - 22 2013

92.
93. Howe, Jeff (2009). Crowdsourcing:
Why The Power of The Crowd is
Driving The Future of Business.
New York: Three Rivers Press.
2009.

94.
95. Howe, Jeff (2006). The Rise of
96.

Crowdsourcing, Wired Online

Magazine. June 2006.

97. Lars Hetmank (2013).

Components and Functions of
Crowdsourcing Systems - A
Systematic Literature Review.
11th International Conference on

Wirtschaftsinformatik, 27th
February 01st March 2013,
Leipzig, Germany
98.
99. Massolution (2013). The Crowd in
the Cloud: Exploring the Future of
Outsourcing, January 2013,
www.massolution.com
100.
101.
102.
PwC Advisory People and
Change (2011). Harnessing the
power of crowdsourcing: Does your
company stand out in a crowd?
www.pwc.com/us/peopleandchang
e
103.

104.
Ideaconnection (2013).
www.ideaconnection.com/openinnovation-success
105.
106.
Amazons Mechanical Turk
(2013).
www.mturk.com/mturk/welcome
107.
108.
InnoCentive (2013).
www.innocentive.com
109.
110.
Freelancer (2013).
www.freelancer.com
111.
112.
99Designs (2013).
www.99designs.com
113.
114.
Utest (2013).
www.utest.com