Professional Documents
Culture Documents
By Dion Hinchcliffe on August 10, 2012 Its certainly one of the hottest new buzzwords in technology, yet the meaning of big data typically depends on whom you ask. Yet its also clear that big data, an important reformulation of how we store and process our digital information, continues to make a big splash as a major IT trend of this half-decade. Certainly the market estimates are optimistic, with Deloitte recentlypegging the size of the market at between $1.3-$1.5 billion this year, while IDC forecasts the industry will be whopping $16.9 billion by 2015. But these large numbers tend to obscure the fundamental changes that currently seem to be taking place under the rubric of big data. The first of these is the data-first ethos thats embodied by trying to tap into and process ground truth (by seeking out the best raw data) and then deriving insight from what is uncovered (domain-specific business intelligence), rather that trying to find data to support ones already-completed strategic decision making. One of the better known examples of data-first thinking is the famous Moneyball story, as told in the 2003 book by Michael Lewis, relating the story of how the Oakland As bucked tradition and switched to heavy data analysis to identify their highest performers, with considerable success. Though only one data point, this story and a growing list of others are leading many believe that data-first thinking may be the solution to many longstanding problems to help combat everything from crime and disease to pollution and poverty. Its also perhaps the key to resolving somewhat more mundane challenges in our businesses as well. The second major change is the shift away from the relational data model as the definitive standard for how to process information for the first time in over a generation. To be sure, the growing adoption in customer-facing technology of emerging platforms such as Hadoop and NoSQL-style databases, is still most prevalent in Web startups and consumer services. Yet the peta and exa-bytes of todays data volumes in many business contexts practically demands technologies that scale well in the face of unrelenting datasets and shrinking time scales that are growing exponentially. For a variety of reasons too long to enumerate here, the relational model has at long last encountering both a serious challenge to its hegemony as well as real challengers that can frequently do better at handling todays data volumes and types. And though many organizations will continue to use relational technology to create some of their big data solutions, its no longer the only option, particularly as the growth in unstructured data is now much faster than classical structured data. The third change is the move towards making big data a more operational component of the way organizations work and how externally-facing products function. While data scientists are often required to get the best outcomes, the results of their work are often applications or data appliances that are usable by just about anyone. Just like Google enabled the layperson to query the entire contents of the Web with a few keywords, the next generation of enterprise big data seems to be about connecting workers with the data landscape of their organizations in a way that doesnt typically require IT wizards in white robes. Thus business solutions based on big data technology must be a readily approachable end-user technology for the average line worker in order to have a sustained and meaningful business impact. Related: Bursting the Big Data Bubble
22% planned to have a big data solution in six months 17% planned to have a big data solution in 12 months 15% planned to have a big data solution in two years Admittedly, attendees of this particular conference were more likely than average to be adopters of big data, so these numbers are a little optimistic, even given that big data is a big tent for a great many technologies that handle large data volumes and analytics. However, the story becomes even more interesting when we look at specific sectors. For example, the insurance industry recently reported that 15-20% of insurers are actively preparing for big data solution. Government, one of the larger potential beneficiaries of big data according to the seminal McKinsey report on the subject, is itself experiencing relatively slow adoption, witha recent survey of public sector CIOs and IT managers reporting it will take three years to start processing their data this way. If we look at function, instead of industry, we can see that sales processes are likely poised to be revolutionized by big data. A recent analysis by CSO Insights reveals that 71% of companies expect big data to have a significant impact on sales, despite only 16% currently doing so, a gap that many organizations will clearly want to close. Related: The enterprise opportunity of Big Data: Closing the clue gap However activating on the large set of changes that big data entails will clearly happen incrementally, yet broadly, in most companies. Theres technology, process, infrastructure, and management that all has to be put into place, plus the hiring of data scientists that understand your business (or learn to), as well as such stillesoteric concepts such as DevOps, which will marry the operational aspects with the development aspects of big data to quickly solve business problems by applying data-first analysis combined with just-in-time R&D and deployment. In addition, companies will also have to deliver on a big data stack in the enterprise. This stack will invariably consist of the following components, designed out of a conglomerate of open source software, commercial applications, on-premises and cloud infrastructure, combined with data from just about everywhere. The visual above also depicts this notional big data stack and you can see another, more technology and product-specific view I pulled together last year.
to pick from. Ultimately, the biggest challenge will be in integrating big data effectively into updated and revised business processes. Thus again, change itself will be the large overall obstacle as technology out-paces the ability of most organizations to absorb it. This will likely push big data into the cloud for most organizations look for strategies to speed adoption, further hastening cloud-related migration of so much of IT. This may not be a bad thing. Read the complete article @ ZDNet | Enterprise Web 2.0 Blog RSS Posted in Featured Posts, Technology / Software | Tagged BigData, Data Warehousing, databases, deloitte, Emerging Tech, hadoop, NoSQL | 4 Responses