Improving Communications Service Provider Performance With Big Data

Improving Communications Service Provider Performance
with Big Data

Architects Guide and Reference Architecture Introduction
ORACLE ENTERPRISE ARCHITECT URE WHITE PAPER
| APRIL 2015
Disclaimer
The following is intended to outline our general product direction. It is intended for information
purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any
material, code, or functionality, and should not be relied upon in making purchasing decisions. The
development, release, and timing of any features or functionality described for Oracles products
remains at the sole discretion of Oracle.
ORACLE ENTERPRISE ARCHITECTURE WHITE PAPER IMPROVING COMMUNICATIONS SERVICE PROVIDER PERFORMANCE WITH BIG DATA
Table of Contents
Executive Summary
Key Business Challenges
Where to Find Business Cases that Justify Projects
Establishing an Architectural Pattern
12
IT Operational ETL Efficiency
15
Oracle Products in the Information Architecture
16
Additional Data Management System Considerations
20
Extending the Architecture to the Internet of Things
22
Keys to Success
24
Final Considerations
26
ORACLE ENTERPRISE ARCHITECTURE WHITE PAPER IMPROVING COMMUNICATIONS SERVICE PROVIDER PERFORMANCE WITH
BIG DATA
Executive Summary
The ability to access, analyze, and manage vast volumes of data while rapidly evolving the Information
Architecture is increasingly critical to communications service providers. Service providers are looking
to improve business efficiency and performance while facing a number of challenges including
competition from other operators and over-the-top (OTT) players, the need to create new service
offerings to better diversify the business, and ongoing operational management cost challenges and
inefficiencies.
Faced with these challenges, service providers are looking to leverage the new generation of Big
Data analytics tools and techniques to exploit the untapped reservoirs of data that exist within their
network and IT systems. The fine-grained data that can be collected from network probes, intelligent
sensors and other data sources are the fuel that powers Big Data analytics and provides new insights
into ways to improve customer experience, optimize networks, drive greater operational efficiency, and
enable broader service offerings. External data sources, such as social media feeds, can also provide
important insight into the customer sentiment and preferences.
Today, communications service providers collect data from a wide variety of sources. These data
sources can include:
Call data records (CDRs) and internet usage data records from mobile phones, devices, customer
premise equipment, and machine to machine (M2M) / Internet of Things (IoT) sensors and gateways
Network and signaling data from network probes
Fault, performance and monitoring data from element management systems (EMS) and network
management systems (NMS)
Security, audit and other log files from AAA and intrusion detection systems
Transaction data from Operations Support Systems (OSS), Business Support Systems (BSS), Point
of Sale (PoS), Charging / Prepaid Systems, Dealer Management, CRM, ERP and other enterprise
systems
Social Media feeds
Web browsing patterns from both fixed and mobile networks, and deep packet inspection (DPI)
based internet records
Demographic and profile data from CRM and front/office subscriber management systems
Fine-grained mobile location data
However, the sheer volume, variety and velocity of the data available to communications service
providers has meant that traditional BI and data warehousing techniques, geared to ingest and
analyze structured data from enterprise IT systems, are limited in the scope and depth of analysis that
can be performed on these disparate sources of data.
1 | ORACLE ENTERPRISE ARCHITECTURE WHITE PAPER IMPROVING COMMUNICATIONS SERVICE PROVIDER PEFROMANCE WITH BIG DATA
As a result, many data feeds are under-exploited. For example, network data is commonly used for
operational monitoring in the Network Operations Center (NOC), but its sheer volume means that
much of it is usually discarded without further downstream analysis. The fine-grained network data
could be further mined or analyzed to predict future network equipment failures, optimize network
operations, and understand and improve customer experience.
As new network technologies are deployed such as 4G/LTE and NFV (Network Functions
Virtualization), and mobile internet usage, IoT and M2M applications, and OTT applications usage
grow, the data generated increases exponentially. In turn, the business analysts who crave more data
want to analyze these massive and diverse data volumes.
The increase in data velocity and sources naturally drives an increase in aggregate data. Business
analysts want more data to be ingested at higher rates, stored longer and want to analyze it faster. Big
Data solutions enable communications service providers to meet the challenges of gaining insights
from this data deluge and achieve competitive advantage through improved customer experience,
greater operational efficiency, and monetization of a service providers data assets.
This paper provides an overview for the adoption of Big Data and analytic capabilities as part of a
next-generation architecture that can meet the needs in the evolving communications industry.
This white paper also presents a reference architecture introduction. The approach and guidance
offered is the byproduct of hundreds of projects and highlights the decisions that Oracles customers
faced in the course of their architecture planning and implementations. Oracles advising architects
work across many industries and government agencies and have developed standardized
methodology based on enterprise architecture best practices. Oracles enterprise architecture
approach and framework are articulated in the Oracle Architecture Development Process (OADP) and
the Oracle Enterprise Architecture Framework (OEAF).
2 | ORACLE ENTERPRISE ARCHITECTURE WHITE PAPER IMPROVING COMMUNICATIONS SERVICE PROVIDER PERFORMANCE WITH BIG DATA
Key Business Challenges

Communications service providers exist in a rapidly changing marketplace. New carriers emerge as start-ups while
older smaller carriers often merge with larger carriers. Multi-national carriers grow networks, services, and offerings
in new parts of the world through these acquisitions. Non-traditional competitors, such as OTT video, chat,
streaming services, are also eroding revenues and ARPU (Average Revenue per User).
Communications service providers today provide far more than just networks that are carriers of voice, video, and
data communications. There is a growing portfolio of enterprise solutions, IT and cloud services, and consumer
offerings that comprise a growing proportion of a service providers revenues.
The growing number of sources and data volumes, and the value of a service providers data itself, also offer
opportunities to create new service offerings. For example, many communications companies now provide
networks for carrying data transmitted from devices that contain intelligent sensors (e.g. the Internet of Things) and
offer data centers as cloud-based platform offerings.
In order to understand the state of the infrastructure and opportunities for business growth, extensive data analysis
is required. The types of data used in these analyses can vary widely with much of it now coming from sensors and
other streaming data sources. By deploying Big Data Management Systems that include traditional data
warehouses and newer data reservoirs (featuring Hadoop and / or NoSQL Databases), broader types of data can be
analyzed to ensure that the business becomes more agile. A key challenge remains making sense of this growing
mountain of data in meaningful ways that will impact the business.
Figure 1 represents three primary drivers for Big Data and analytics projects in communications service providers,
as well as a number of specific business use cases being implemented in order to contain costs or generate more
revenue for the service providers. The following section highlights the three primary drivers in more detail.
Figure 1: Key drivers of Big Data and analytics projects
Understanding Customer Experience

A service subscriber engages with a communications service provider across multiple touch points (website, retail
store, dealer, mobile handset, etc.) and the quality of each interaction and seamlessness and consistency of the
overall experience will impact customer satisfaction. These interactions will influence whether the subscriber will
recommend the service provider to others. The trail of data generated at each touch point, such as the amount and
time of mobile internet usage, or the location, time and amount of a mobile subscribers prepaid recharge, can drive
more accurate customer profiling and segmentation. This enables analysts and marketers to better understand
subscribers needs and behaviors, and more significantly enables personalization and actions at the optimal point in
time, whether these actions are directed towards offers, retention or customer satisfaction.
In a highly competitive market, communications service providers have long understood that customers will seek out
the service plans and offerings that they perceive as providing the greatest value for their money. So, much of the
focus has been on understanding how and why customers might churn and formulating service offerings that align to
customer demands in order to retain them.
Such data has traditionally been analyzed in data warehouses consisting of relational databases. Today, some of
this activity is being moved to Hadoop clusters, partly due to the relatively low price point of Hadoop clusters and
partly due to its schema-less file system being ideal for predictive analytics workloads. Churn, segmentation and
marketing analytics can be enriched through the incorporation of Big Data sources, whether they relate to network
usage, social media feeds or federation of opt-in profile data from external data sources.
Fast Data and NoSQL technologies also enable communications service providers to better realize the value of their
predictive analytics. These technologies can enable timely actions such as providing real-time offers or sending
alerts at the optimal time and channel of interaction.
Improving Network Management and Operational Efficiency

The need for excellence in network operations and improving operational efficiencies in a communications service
providers network infrastructure is well understood. However, the cost of systems in the Network Operations
Center (NOC) has been high because of the large number of vendor-specific management systems and silos
required to manage a heterogeneous multivendor network environment. Rapid root cause analysis, failure
prediction, and optimization have been hampered by the lack of tools and cost effective information management
architectures to ingest and access network Big Data for analytics and discovery.
Network equipment and associated management systems (Element and Network Management Systems) generate
alarms, SNMP traps, performance counters, and data feeds used in managing network operations. Increasingly,
this data is gathered from network probes and sensors providing real-time information on the state of the network.
There is an opportunity to better understand when maintenance needs to occur sooner and, by monitoring the
changing state of key components, affording even greater efficiencies and cost savings. Predictive analytics
solutions deployed across Big Data Management Systems (including Hadoop) will likely become common practice
to increase overall reliability and reduce cost.
Network monitoring and operations tools in the Network Operations Center (NOC) or Service Assurance function in
a communications service provider have largely consisted of vertical and network technology-specific management
solutions from specialist vendors or network equipment providers (NEPs). These were historically built using
custom toolsets and proprietary data stores since the volume and velocity of network data precluded the use of
COTS BI and Analytics tools for many network monitoring functions. This resulted in a multitude of siloed systems
and technologies in the NOC.
COTS Big Data technologies, including fast data platforms for streaming analytics, are now readily available to meet
the stringent performance and scalability needs of network operations. A new generation of network monitoring and
analytics tools are being deployed that can provide a network and vendor agnostic point of consolidated data
collection, real time monitoring, and offline analytics of network data at lower total cost than proprietary and NEPbased tools. These tools also allow network data to be combined with other data sources (such as charging and
billing data, cost data, customer and social data) to augment the 360 degree customer view and enhance service
management, marketing, and sales for consumer and enterprise markets.
Big Data analysis of fine-grained network data across technologies, network equipment, and network probes can be
federated with data from social media, engineering and contact center trouble ticket logs, and CPE / handset data to
enable a far greater degree of predictive and root cause analysis of network and service faults.
Predictive analytics might also be deployed in data warehouse solutions today, such as when optimizing the routing
of repair vehicles, crews and supplies. Such routing is fundamental to providing the lowest possible cost of delivery
while assuring faster repairs.
Network data analytics becomes even more critical for Network Function Virtualization (NFV). The analytics solution
can make recommendations to the orchestration engine when virtual network functions should be scaled up or
down.
Providing New Services and Data Monetization

Todays communications networks carry video and data as well as voice. Most providers of basic land-line
communications services now also support mobile devices. While some also offer cable-TV networks, most are now
considering the growing impact of delivery of televised content over the internet to mobile devices connected to their
networks.
Some are also exploring offering extended data services. These offerings can include providing the
communications backbone for Internet of Things deployment used in transmitting data from intelligent sensors to
analytics engines. Others are also providing data processing services by offering cloud-based data centers linked
into their networks.
For example, many envision combining location, usage and demographic data to develop new revenue sources and
enhance existing revenue channels. They are doing so in a manner consistent with privacy standards as well as
transparency. They are shrouding individual identities in data through aggregation and other techniques.
Customers who understand the personal value of such new services (such as knowing where their children who are
carry mobile devices are located) accept such data capture. Providing clear opt-out options re-enforces trust in
these services.
The streaming nature of much of the data is leading to increased utilization of Hadoop clusters for analysis and as
data reservoirs. NoSQL database engines are sometimes scaled to provide high capacity data ingestion engines
due to the tremendous data volumes encountered in these kinds of applications.
Where to Find Business Cases that Justify Projects

Many existing business capabilities can be enhanced when more and varied data becomes part of the Information
Architecture.
The business case for investment in Big Data technologies can be developed based on technology drivers as well
as business drivers.
From a technology perspective, the sheer volume of network control plane and signaling data, usage data and
CDRs / xDRs (Call Detail Records / x Data Records), as well as transaction data often means that Hadoop is a costeffective way to augment an existing data warehouse. Hadoop can be deployed as a data reservoir for staging,
transformation and aggregation of data before the data is uploaded to the data warehouse. Such a deployment
scenario will offload workloads and data from the existing data warehouse, thus providing more headroom for
expansion, and more performance capacity in the data warehouse. This scenario can provide a quick return-oninvestment, especially as the rate of data growth accelerates with the increasing adoption of 3G, 4G/LTE, FTTx,
video streaming, IoT and NFV technologies.
Apart from technology-based return-on-investment, the combination of fine-grained network activity and usage data
coupled with new analytics techniques are leading to new business cases enabled through innovative Big Data
projects.
Examples of these new capabilities include:
The ability to load and store massive quantities of data to enable a collect now, examine later approach.
Typically, the network operations center (NOC) monitors network alarms and performance counters to
ensure network availability and quality of service. These streams of network management data can now
be cost-effectively stored and analyzed downstream to identify root causes of network issues, predict
future network problems, and pinpoint specific problems down to specific customers and locations. When
this data is mashed up with usage, billing, and customer data, insights can be gained into how service for
top tier customers or businesses can be improved. The wisdom of further investment in network
infrastructure to improve coverage in rural or underserved areas can be evaluated.
Streaming event processing and analytics can drive real time alerts and actions in other words fast data,
rather than Big Data. High velocity event streams can be monitored and analyzed to detect errors and
abnormal conditions. Operators can use COTS event processing technology to collect and analyze
network signaling data to monitor VoIP (Voice-over-IP) service quality and IP video streaming quality over
services such as YouTube. Such analysis can lead to creating a higher quality of experience (QoE) for
subscribers.
Big Data enables unstructured text and sentiment processing, whether from in-house data sources (such
as call center or field service logs), or external sources (such as social media). For example, some
communications service providers have noted that trending topics on social media can be correlated with
service outages in the network and / or handset and application.
A communications service providers Big Data (consisting of network data, fine-grained location data from
cellular and WIFI networks, and applications sending data from mobile handsets) can provide tremendous
value to subscribers and therefore can be monetized. In such scenarios, data privacy and confidentiality
requirements are well understood and followed.
Figure 2 illustrates many of these new capabilities.
Figure 2: New capabilities enabled in Big Data solutions
IT organizations at communications companies typically work with their lines of business to build solutions that
deliver the following when defining Big Data projects:
1)
Improved Marketing, Customer Acquisition, and Retention: Communications service providers spend
tremendous sums of money on advertising that highlight their service plans, capabilities, and quality of service.
Understanding which customers are at risk of changing carriers (churning) and taking proactive action to retain
those customers can be crucial in maintaining market share. Fierce competition between mobile carriers has
resulted in an arms race between providers customer analytics competencies, with ever more sophisticated
churn, profiling and segmentation algorithms and approaches being introduced to achieve additional lift and
revenue impact. Extending the 360 degree customer view with Big Data, possible by also analyzing 3rd party
audience data, social media feeds, ecommerce purchase history and other opt-in data, can result in reduced
wasted advertising spending and increased campaign conversion rates through improved targeting of
prospects. For example, users who have a history of making mobile purchases through offers delivered via the
mobile channel might be targeted. Extending the customer profile with Big Data derived from network usage,
social network analysis, and location and transaction data can further refine profiles and segmentation. Usage
location during the day and night might be used to identify the location of work and residence which can be
used when inferring profiles and segments during analysis. Identifying patterns of mobile usage using finegrained network data is also invaluable in behavioral segmentation of users. Big Data techniques enable the
mash-up and analysis of this fine-grained data. External, 2nd and 3rd party data sources can dramatically
increase the ability of data scientists to implement increasingly sophisticated models and analysis.
2)
Real time personalization of customer experience. Investments in profiling, segmentation, and analytics will
not drive greater campaign conversion or retention unless this rich customer profile is made available in a
timely manner at a point of customer interaction. Big Data technologies can be deployed to make this customer
profile information available with far less data latency than traditional approaches and to facilitate real time
personalization of customer interactions regardless of channel. Thus, it is possible to avoid upsell / cross-sell
and retention offers being activated based on out-of-date data as the optimal moment to present the offer might
have already passed. Campaigns and offers can be launched at the right time, using the right channel, to
engage the customer at the optimal moment on the customer journey. A promotional offer might be made for
fast mobile internet access when a handset is upgraded by a customer from an older 2G model to a 4G
Smartphone. Or, an offer might be made to an inbound roaming customer at the moment the handset is
powered on at the airport. Understanding the effectiveness of promotions such as these and making
adjustments in strategies can enable market share growth.
3)
Better Product Management: Offering the right products, including physical devices, networks, and service
plans, are an important part of maintaining and gaining market share. A key challenge facing marketers is how
to stimulate greater adoption of mobile internet and smart phone usage and how to incentivize users to move
from low-ARPU feature phones on 2G networks to 3G/4G smart phones. Another key challenge is tracking and
assessing the impact of OTT (Over-the-top) services, such as Skype, WhatsApp and WeChat on voice
and messaging services in order to drive new product strategies in response to these non-traditional
competitors. Big Data technologies enable social media feeds from Twitter, Facebook and other online
sources to be analyzed and correlated with sales, campaign and usage data. This data can provide insight into
customer sentiment and customer interest in service plans and new handset or product launches, and can also
be used to assess post launch satisfaction. Network usage data can provide additional richness to the analysis
of how new products and services are being used, by whom, and at what location in order to assess met and
unmet demand for services, network infrastructure, and retail locations.
4)
Cost Effective & Timely Supply Chain and Logistics Management: Timely delivery of parts, products,
equipment, and personnel are critical to optimally managing network maintenance. Timely delivery of products
and parts to communications providers retail stores that interact with consumers is another aspect of the
supply chain that must be managed.
5)
Network Management, Service Assurance and Customer Experience management: Centralized monitoring
and troubleshooting of the network, including where dropped calls and congestion occur, and rapid response in
fixing the problems can lead to higher customer service satisfaction which, in turn, can lead to new customers.
Network monitoring can also be used to determine where illegal access to service is occurring. Historically, the
Network Operations Center (NOC) has been focused on the operational management of networks to maintain
uptime and network quality. As network technologies become more complex and IT-centric with IP-layer
services, value-added services, computing services, service delivery platforms, and network functions
virtualization (NFV), there is greater complexity. This complexity can make ensuring quality of service less
predictable, as layers of virtualization, connectivity, and virtualized computing and cloud resources can
contribute to greater service latency and generate new peaks in resource consumption. Big Data technologies
are required for the real time analysis of this data and provide a reservoir for managing these larger data
volumes. It is no longer the network equipment provider (NEP) or the niche network performance solution
vendor who can provide the tools to effectively manage the network. In this new network environment, a
service and network-agnostic end-to-end information management architecture, leveraging Big Data storage
and analytics tools, is required to effectively manage and operate the network and cloud infrastructure for a
service provider.
6)
New Information as a Service Offering: Offering communications infrastructure for IoT topologies (such as
providing the communications networks for telematics solutions from motor vehicle manufacturers) or cloudbased software and services solutions can lead to new revenue streams. A communications service provider
holding information on subscribers and their network usage and location is in a unique position of owning data
that would be invaluable both for internal monetization (by providing insights into customers and network
usage) as well as for external monetization. The data, with appropriate controls for data privacy, is a treasure
trove for external businesses and public sector organizations keen to understand behavior and movement of
their customers and citizens. Communications service providers are aggregating data, shrouding sensitive
identify information, and offering this data as a paid information service to businesses and governments. For
example, this data might be used to determine the effectiveness of public transport services or where to open a
new retail outlet or restaurant. Developing and measuring the effectiveness of partnerships with content
providers can also lead to new sources of revenue.
7)
IT Operational Efficiency: Not unique to communications companies and rarely driven from the lines of
business (but a possible reason for embarking on extended architectures that include Hadoop) is the need to
move data staging and transformation to a schema-less platform for more efficient processing and leveraging of
IT resources. IT operational efficiency is often difficult to prove but is sometimes an initial justification that IT
organizations gravitate toward when deploying these types of solutions.
On the next page, we show a table that summarizes several typical business challenges in communications
companies and the opportunity for new or enhanced business capability when adding new analytic capabilities.
TABLE 1 COMMUNICATIONS COMPANIES, BUSINESS CHALLENGES & OPPORTUNITIES
FUNCTIONAL AREA
BUSINESS CHALLENGE
OPPORTUNITY
Marketing & Customer Acquisition
Maximize promotion effectiveness
Target offerings that match customer

requirements while maximizing revenue and
market share
Understand the effectiveness of promotions
sooner and adjust rapidly when competitive
threats occur
Customer Management &

Retention
Respond to changing customer demand

and offer superior service, prevent customer
churn
Align product offerings and plans to customer

usage and behavior
Offer valued add-on offerings designed to
grow customer revenue, loyalty, and
minimize churn
Provide level of service matching customer
expectations leading to positive customer
sentiment on social media and elsewhere
Real Time Personalization of

Customer Experience
Personalize offers to the customer at the

point of engagement
Launch campaigns and offers at the right

time, using the right channel, to engage the
customer at the optimal moment on the
customer journey
Product Management
Optimize rate structures and product mix in

offerings
Maximize product sales

Maximize product and services revenue
Gain market share
Network Operations
Assure quality of service and customer

experience as the complexity of network
and IT services increases.
Detect network and service failures

proactively
Use predictive analytics to determine
potential network and service faults and
issues
Enhance network management to support
new technologies, 3G, 4G/LTE, FTTx, VPN ,
OTT Video and Cloud/IT services
Provide enterprise and B2B service and
network management
Understand the nature of poor customer
experience, such as slow internet access,
poor VoIP voice quality, dropped calls and
network congestion in order to take corrective
action
Understand network failures quickly
responding with predictable time to delivery
of critical parts, products, equipment, and
qualified personnel minimizing downtime
Optimize network to match required levels of
service and gain customers and their loyalty
Understand where fraudulent access to
service is occurring and manage accordingly
Information as a Service
Offer platforms that complement strategies

of major corporate and government
customers
Offer aggregated data as a service regarding

customer location, demographics, and
preferences (while shrouding sensitive data)
to enterprises and public sector customers
Establish cloud platforms for customers
software hosting
Provide new software services in the cloud

Provide and manage communications
backbones for Internet of Things topologies
Provide and manage communications
backbones servicing video feeds
Establishing an Architectural Pattern

Figure 3 illustrates key components in a typical Information Architecture.
Data is acquired and organized as
appropriate and then analyzed to make meaningful business decisions. A variety of underlying platforms provide
critical roles. Management, security and governance are critical throughout and are always top of mind in
communications companies. These components are further described in the Information Architecture and Big
Data whitepaper posted at http://www.oracle.com/goto/ea.
Figure 3: Key Information Architecture Components
How do we determine which of these components should be part of the architecture to meet the needs of a specific
organization or company? If we create an information architecture diagram, and trace the data flow from the
sources to the application (end-user), we can build a logical configuration of the components to support the
functions.
The first step in defining a future state architecture is documenting the current state, its capabilities and any
functional gaps. Typically a current state data warehouse environment might look something like Figure 4.
Figure 4: Typical Current State Data Warehouse
The first gap that typically has to be closed is a need to provide a more agile reporting and analysis environment
where new data and ad-hoc reports are needed on an ongoing basis. Information and data discovery engines can
provide this type of capability. When information discovery is incorporated into the architecture it would look
something like the illustration in Figure 5.
Figure 5: Typical Introduction of Information Discovery
Now that were better able to analyze the data we have, the next step would be to explore bringing in new data and
new data tapes. These data sets might be internal, 3rd party, structured, unstructured or of unknown structure.
When storing data of unknown structure, the most efficient way to store data sets is often in a Hadoop-based data
reservoir. Initially, such projects are often considered experimental in organizations and therefore they might be
independent efforts separated from the traditional environments, as illustrated in Figure 6.
Figure 6: Typical Early Hadoop Environment separate from the Data Warehouse
The profile of the data such as how it is acquired, how it should be formatted, the frequency of updates and quality
of the data will help us put the right technology in place best suited for the particular situation. We need to
understand whether real-time or batch processing is appropriate. We should understand the periodicity of
processing required based on data availability. Below is a partial list of the characteristics that should be considered:
Processing Method prediction, analytics, query, ad-hoc reports
Format and Frequency external data feeds, real-time, continuous or periodic on-demand
Data Type web/social media, machine generated, human generated, biometric, legacy or internal, transactional
Consumer Application Web Browser, Intermediate processes, Enterprise Application
When business value is found in analyzing data in a Hadoop-based data reservoir, lines of business generally begin
to see a need to link data there to historical data stored in their data warehouse. For example, a business analyst
might want to compare historical transactions for a shipment stored in the data warehouse to sensor data tracking
that shipment in the data reservoir. Various linkages are often established as pictured in Figure 7.
Figure 7: Integration of Hadoop Infrastructure and Data Warehouse
We also added something new to Figure 7, a real-time analytics and recommendation engine. In many situations,
the latency inherent in the data movement pictured above means that the recommendation from analysis would
come too late to take action in near real-time. A way around this is to perform periodic advanced analytics in the
data reservoir and / or data warehouse and provide updates to a real-time recommendation engine that becomes
more fine-tuned through self-learning over time.
IT Operational ETL Efficiency

In Figure 7, you might have noticed a line pointing from the transactional sources to the Hadoop cluster. This is to
illustrate a popular ETL alternative, leveraging Hadoop as a data transformation engine.
Lets now consider the type of data typically stored in todays data warehouse. Such warehouses are typically
based on traditional relational databases using a schema on write data model. The data sources can vary, but the
structure of the data is determined before the data in imported into the data warehouse. In the example below there
are two data sources. These two data sources go through an ETL process to prepare the data to be loaded into the
warehouse.
Figure 8: Structured Data and the Data Warehouse
Extending the architecture can enable a more agile workflow by incorporating data sets for which there is not rigid
structure. This data model is best defined as schema on read. That is, we store the data without the traditional
ETL processing, as we dont know exactly how we want to access the data. In the example below we are using
multiple data sources with varying structures.
Figure 9: Unstructured Data, Distributed File Systems and Key Value Data Stores
These two environments should not be separate and unique. Building an integrated Information Architecture that
can handle data sets of known structure as well as unknown structure enables us to augment the capabilities of
existing warehouses as well as leverage data center best practices that are already in place.
Oracle Products in the Information Architecture

In Figure 10, we illustrate how key Oracle products could fit in the generic architecture diagram previously shown.
Figure 10: How Key Oracle Products Fit in the Generic Architecture
Oracle provides an integrated solution as part of its Oracle Communications Analytics suite of products. The
solution follows the reference architecture that was just defined. It leverages Big Data technologies as well as the
Oracle relational database, which is where the Oracle Communications Data Model (OCDM) resides. The suite
includes prebuilt adapters to feed data from source applications into the platform components, and a set of
communications specific analytic applications that are aligned to TM Forums eTOM operations framework and
which address many of the use cases discussed earlier in this document. Service providers can leverage the
prebuilt components for fast ROI and integrate them into their existing environments. Figure 11 illustrates the Oracle
Communications Analytics product suite.
Figure 11: Oracle Communications Analytics product suite
Defining an Information Architecture is all about linking it to a specific use case. For example, an Information
Architecture diagram that focuses on analysis of data on a mobile network to create special offers to subscribers
might look like Figure 12:
Figure 12: Analysis of data on a mobile network to determine where to deliver promotions offers
The various software capabilities required in a typical architecture might include these Oracle components:
Oracle Relational Database Management System (RDBMS): Oracle Database 12c Enterprise Edition is designed
for performance and availability, security and compliance, data warehousing and analytics, and manageability.
Key data warehousing options often include In-Memory, OLAP, the Advanced Analytics Option, and Partitioning.
Oracle Communications Data Model (OCDM): An enterprise-wide data model for the communications industry
covering the following business areas network, product management, cost and contribution, customer
management, provisioning and activation, revenue, marketing, and partner management.
Oracle Communications Analytics adapters and applications: Prebuilt components integrating data from Oracle
Communications applications, and used within communications industry analytic applications to address specific
business processes and use cases (e.g. Billing Analytics, Social Network Analytics, and Policy and Charging
Analytics).
Oracle Business Intelligence Enterprise Edition (OBIEE): A business intelligence platform that delivers a full range
of capabilities - including interactive dashboards, ad hoc queries, notifications and alerts, enterprise and financial
reporting, scorecard and strategy management, business process invocation, search and collaboration, mobile,
integrated systems management and more.
Oracle Real-time Decisions: A real-time recommendation engine.
Hadoop Distributed File System (HDFS): A scalable, distributed, Java based file system that is the data storage
layer of Hadoop. Ideal for storing large volumes of unstructured data.
Flume: A framework for populating Hadoop with data via agents on web servers, application servers, and mobile
devices.
Oracle Data Loader for Hadoop: A connectivity toolset for moving data between the Oracle RDBMS and the
Hadoop environment.
ODI: Oracle Data Integrator is a comprehensive data integration platform that covers all data integration
requirements: from high-volume, high-performance batch loads, to event-driven, trickle-feed integration
processes, to SOA-enabled data services.
Oracle Enterprise Metadata Management: Data governance and metadata management tool providing lineage
and impact analysis, and model versioning for business and technical metadata from databases, Hadoop,
business intelligence tools, and ETL tools.
Endeca: An information discovery tool and engine.
Oracle Big Data Discovery: A Hadoop-based information discovery tool.
Oracle Big Data SQL: An optimal solution for querying an Oracle Database on Exadata and combining the results
with data that also answers the query and resides on Oracles Big Data Appliance.
ORE: Oracle R Enterprise enables analysts and statisticians to run existing R applications and use the R client
directly against data stored in Oracle Database (Oracle Advanced Analytics Option) and Hadoop environments
Oracle Enterprise Manager: An integrated enterprise platform management single tool used to manage both the
Oracle structured and unstructured data environments and Oracle BI tools.
Oracle Essbase: An OLAP (Online Analytical Processing) Server that provides an environment for deploying
pre-packaged applications or developing custom analytic and enterprise performance management applications.
The software products listed above can be deployed in an integrated environment leveraging these engineered
systems:
Big Data Appliance (BDA): Eliminates the time needed to install and configure the complex infrastructure
associated with build-out of a Hadoop environment by integrating the optimal server, storage and networking
infrastructure in a rack.
Exadata: Streamlines implementation and management while improving performance and time to value for Oracle
relational database workloads by integrating the optimal server, storage and networking infrastructure.
Exalytics: Provides an in-memory server platform for Oracle Business Intelligence Foundation Suite, Endeca
Information Discovery, and Oracle Essbase.
Obviously, many variations in Oracle products and other products are possible when defining and deploying your
Information Architecture.
Additional Data Management System Considerations

In defining the Information Architecture, it is important to align the data processing problem with the most
appropriate technology.
When considering the choices you have in database management systems to include in an Information Architecture,
you might consider if the form of the incoming data or ACID properties or fast data availability is most important.
Other considerations should include manageability, interoperability, scalability, and availability. Of course, you
should also consider the skills present in your organization.
Some of the various data management technologies in a typical architecture include:
Relational Databases
Typically already in use at most companies, RDBMS are ideal for managing structured data in predefined schema.
Historically they excel when production queries are predictable. Support of dimensional models makes them ideal
for many business intelligence and analytics workloads. They frequently house cleansed data of known quality
processed through ETL workloads. Relational databases also excel at transactional (OLTP) workloads where read /
write latency, fast response time, and support of ACID properties are important to the business.
These databases can usually scale vertically via large SMP servers. These databases can also scale horizontally
with clustering software.
Example RDBMS Product: Oracle Relational Database
MOLAP Databases
Typically used for highly structured data, MOLAP databases are ideal when you know what queries will be asked
(e.g. facts and dimensions are predefined and non-changing) and performance is critical. These databases excel at
certain business intelligence and analytics workloads.
Example MOLAP Product: Oracle Essbase, Oracle Database OLAP Option
NoSQL Databases
NoSQL databases are without schema and are designed for very fast writes. Often, they are used to support high
ingestion workloads. Horizontal scale is most often provided via sharding. Java and Java scripting (JSON) are
commonly used for access in many of the commercial varieties.
NoSQL databases are sometimes described as coming in different varieties:
Key Value Pairs: These databases hold keys and a value or set of values. They are often used for very lightweight
transactions (where ACID properties may not be required), and where the number of values tied to a key change
over time.
Column-based: These databases are collections of one or more key value pairs, sometimes described as two
dimensional arrays, and are used to represent records. Queries return entire records.
Document-based: Similar to column-based NoSQL databases, these databases also support deep nesting and
enable complex structures to be built such that documents can be stored within documents.
Graph-based: Instead of structures like the previous types, these databases use tree-like structures with nodes and
edges connecting via relations.
Example NoSQL Database Product: Oracle NoSQL Database
Distributed File System

Not a database per se as the name would indicate, highly distributed file systems have the advantage of extreme
scalability as nodes are added and frequently serve as a data landing zones or data reservoirs for all sorts of data.
Read performance is typically limited by the individual node of the system when accessing data confined to that
node, however scalability to a huge number of nodes is possible driving massive parallelism. Write performance
scales well as data objects can be striped across nodes.
The most popular distributed file system used today is Hadoop. Given its role as a data reservoir, it is increasingly a
location for performing predictive analytics. SQL access is available via a variety of interfaces though various levels
of standards support are offered.
Example Distributed File System Product: Cloudera Hadoop Distribution (featuring the Cloudera Hadoop Distributed
File System and other features)
Big Table Inspired Databases

There is an emerging class column-oriented data stores inspired by Googles BigTable paper. These feature tunable
parameters around consistency, availability and partitioning that can be adjusted to prefer either consistency or
availability (given these are rather operationally intensive.
A typical use case might be where consistency and write performance are needed with huge horizontal scaling.
HBase (deployed on a Hadoop Distributed File System) in particular has been deployed to 1,000 node
configurations in production.
Example Big Table inspired Product: Cloudera Hadoop Distribution (Cloudera HBase)
Extending the Architecture to the Internet of Things

Thus far, weve focused on the analytics and reporting and related data management pieces of the Information
Architecture. Where communications companies monitor and take action on data from sensors, the architecture
discussion is extended to the Internet of Things. This extended architecture for data capture, security, and linkage
to the rest of the Information Architecture can require additional consideration. Figure 13 illustrates a typical
footprint:
Figure 13: Connected Devices in an Internet of Things Footprint
Items to the far right of Figure 13 have largely been previously discussed in this paper. Many of the other items
pictured are what Oracle typically describes as Fusion Middleware components. For example, much of the sensor
programming today takes place using Java. Security is extremely important since most would not want unidentified
third parties intercepting the data provided by the sensors. Applications closer to the sensors themselves are often
written using Event Processing engines to take immediate action based on pre-defined rules. There are also
various message routing, provisioning, and management aspects of such a solution.
Figure 14 illustrates a typical capability map of this architecture for communications service provider companies:
Figure 14: Connected Devices Capability Map
Sensors are increasingly used to monitor the state of exploration, production, transportation, and refining facilities
and equipment. The real-time monitoring enables proactive measures to be taken sooner enabling great efficiencies
and reducing potential environmental and safety risk.
Figure 15 illustrates some of the Oracle products aligned to the previously shown capability map:
Figure 15: Oracle Products aligned to Capability Map
Keys to Success
One of the most significant keys to success in a large project undertaking is to gain alignment between the business
needs and goals and with the IT architecture design and deployment plans. Key business sponsors must be
engaged and active in all phases.
Methodologies based on phased approaches are almost always the most successful. To start, youll need to
understand the current state and its gaps so that you can better understand how to build towards the future state.
You will need to modify the architecture as business needs change. Therefore, a common method to help assure
success is to deploy quickly in well scoped increments in order to claim success along the way and adjust the plan
as needed. A complete Information Architecture is never built overnight, but is developed over years from continued
refinement.
Figure 16 illustrates such an approach, beginning with defining an initial vision, then understanding critical success
factors and key measures tied to use cases, defining business information maps based on output required, linking
the requirements to a Technical Information Architecture, defining a Roadmap (including phases, costs, and
potential benefits), and then implementing. Of course, an implementation leads to a new vision and requirements
and the process continues to repeat. Pictured in the figure are some of the artifacts Oracle often helps deliver
during Enterprise Architecture engagements and Information Architecture Workshops.
Figure 16: Typical Methodology for Information Architecture Projects
Usability needs will drive many of your decisions. Business analysts will likely have a variety of business
requirements and possess a variety of analysis and technical skills. They could require solutions ranging from
simple reporting to ad-hoc query capability to predictive analytics. Youll need to match the right tools and
capabilities to the right users. One size does not usually fit all. While new features in the data management
platforms can provide more flexibility as to where you host the data for such solutions, the data types, volumes and
usage will usually determine the most optimal technology to deploy. A common best practice is to eliminate as
much movement of data as possible to reduce latency.
Data security and governance are also a key consideration. Communications companies want data to remain
private unless they specifically agree to share it. So securing access to the data, regardless of data management
platforms, tools, and data transmission methods used, is critical. Data governance needs regarding the meaning of
data as well as its accuracy and quality will often require close coordination with and among multiple lines of
business.
Finally, as fast time to implementation important to the success of any business driven initiative, you will want to
leverage reference architectures, data models and appliance-like configurations where possible. These can speed
up the design and deployment and reduce the risk of incomplete solutions and severe integration challenges.
Leveraging engineered systems and appliances where possible can simplify the architecture, reduce time to value
and improve architecture reliability.
Final Considerations
This paper is intended to provide an introduction to applying Information Architecture techniques for communications
service providers. These techniques guide the extension of current architecture patterns to meet new and varied
data sources that are becoming part of the information landscape. Oracle has very specific views regarding this
type of information architecture and can provide even more of the individual components than were described in this
paper.
The following diagram provides a conceptual future state that can encompass all types of data from various facets of
the enterprise:
Figure 17: Typical Conceptual Future State Diagram
A more detailed look at Business Analytics reference architectures appears in documents posted to the Oracle
Enterprise Architecture web site at http://www.oracle.com/goto/ITStrategies.
The following is a figure from one of the just referenced documents to give an idea as to the level of detail that might
be considered around information delivery and provisioning.
Figure 18: A more detailed Reference Architecture Diagram for Information Delivery and Provisioning
The architecture discussion can also lead to consideration on where to host and analyze the data (e.g. in the cloud
versus on-premise). Most communications companies choose to host data in the location where the data initially
lands with an eye on minimizing network data traffic while securing the data at rest and in motion. Once the data
lands, reporting and predictive analytics often take place in that data management system. Where some of the data
is stored in the cloud and some is on-premise, there should be careful consideration of the impact of network
bandwidth on analysis performance where data from both locations is required.
An additional consideration not addressed in this paper is the availability of skills needed by the business analysts
and the IT organization. A future state architecture evaluation should include an understanding as to the degree of
difficulty that a future state might create and the ability of the organization to overcome it.
The competitive nature of communications service providers will assure that those that take advantage of these new
data sources to augment what they know about their business will continue to be leaders. They will continue to
invent new and better business processes and efficiencies and they will do so by evolving their Information
Architecture in an impactful manner.
Oracle Corporation, World Headquarters
Worldwide Inquiries
500 Oracle Parkway
Phone: +1.650.506.7000
Redwood Shores, CA 94065, USA
Fax: +1.650.506.7200
CONNECT W ITH US
blogs.oracle/enterprisearchitecture
facebook.com/OracleEA
Copyright 2015, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only, and the
contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other
warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or
fitness for a particular purpose. We specifically disclaim any liability with respect to this document, and no contractual obligations are
formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any
means, electronic or mechanical, for any purpose, without our prior written permission.
twitter.com/oracleEAs
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.
oracle.com/EA
Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and
are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are
trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. 0415
April 2015
Oracle Enterprise Architecture White Paper Improving Communications Service Provider Performance with Big Data
Author: Robert Stackowiak, Cheng Kian Khor, Tony Velcich, Venu Mantha, Alan Manewitz, Tom Luckenbach Art Licht.

Improving Communications Service Provider Performance With Big Data

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Improving Communications Service Provider Performance With Big Data

Uploaded by

Copyright:

Available Formats

Improving Communications Service Provider Performance

with Big Data

Key Business Challenges

Where to Find Business Cases that Justify Projects

Establishing an Architectural Pattern

IT Operational ETL Efficiency

Oracle Products in the Information Architecture

Additional Data Management System Considerations

Extending the Architecture to the Internet of Things

Key Business Challenges

Figure 1: Key drivers of Big Data and analytics projects

Understanding Customer Experience

Improving Network Management and Operational Efficiency

Providing New Services and Data Monetization

Where to Find Business Cases that Justify Projects

Figure 2 illustrates many of these new capabilities.

Figure 2: New capabilities enabled in Big Data solutions

TABLE 1 COMMUNICATIONS COMPANIES, BUSINESS CHALLENGES & OPPORTUNITIES

Marketing & Customer Acquisition

Maximize promotion effectiveness

Target offerings that match customer

Customer Management &

Respond to changing customer demand

Align product offerings and plans to customer

Real Time Personalization of

Personalize offers to the customer at the

Launch campaigns and offers at the right

Optimize rate structures and product mix in

Maximize product sales

Assure quality of service and customer

Detect network and service failures

Offer platforms that complement strategies

Offer aggregated data as a service regarding

Provide new software services in the cloud

Establishing an Architectural Pattern

Data is acquired and organized as

Figure 3: Key Information Architecture Components

Figure 4: Typical Current State Data Warehouse

Figure 5: Typical Introduction of Information Discovery

Figure 7: Integration of Hadoop Infrastructure and Data Warehouse

IT Operational ETL Efficiency

Figure 8: Structured Data and the Data Warehouse

Oracle Products in the Information Architecture

Figure 11: Oracle Communications Analytics product suite

Additional Data Management System Considerations

Distributed File System

Big Table Inspired Databases

Extending the Architecture to the Internet of Things

Figure 13: Connected Devices in an Internet of Things Footprint

Figure 14: Connected Devices Capability Map

Figure 15: Oracle Products aligned to Capability Map

Figure 16: Typical Methodology for Information Architecture Projects

Figure 17: Typical Conceptual Future State Diagram

Oracle Corporation, World Headquarters

500 Oracle Parkway

Redwood Shores, CA 94065, USA

You might also like