Chapter 21: The Virtual Data Warehouse

Traditional Data Warehousing

De facto information systems strategy for supporting analytical decision making. It offers one of the best ways to translate raw data and present it in ways useful to decision makers. Concerns:
1. Often costly 2. Requires high level of maintenance 3. Lengthy to deploy

The Supply-driven, Physical Data Warehouse

Traditional Process
1. Identifying source data
Existing data, and an Enterprise Model Consistent corporate terminology

2. Extracting, Transforming, and Storing data

Refresh rates, replication methods, maintenance, and adapting/summarizing data to ensure data quality.

3. Accessing the data

End user tools or proprietary tools. Is there a need for a high speed end-user query environment? How can changing requirements be met? How will new data be incorporated?

The Demand-driven Virtual Data Warehouse

Provides a way of accessing data from whatever storage location it resides in, and provide a semantic map that allows the end user to view it as virual-ized as to its specific location.

The Demand-driven Virtual Data Warehouse

Advantages:
1. Faster time to Market
Focus on end-user requirements to enable immediate access where you can.

2. Complementary to physical data warehouse implementations

Permits faster prototype implementation, and helps clarify both the type of information and the access requirements that the warehouse efforts will need to support. It will provide an additional real-time data view.

The Demand-driven Virtual Data Warehouse

Advantages:
3. Open and flexible architecture
An open environment using industry standards makes it easier to address new data sources, and potentially provides a wider selection of tools for the end user.

4. Reduced maintenance
By providing access to the data without burdening the user with where or how the data is stored (transparency layer), the infrastructure can be changed, without the user being aware of the changes

Data Marts and the functional warehouse

Departmental warehouse
-all the operational data required by the department to perform it's daily functions

Information Logistics
Data Collection
-establish what is the common data -data must have a single source for each attribute

Data Validation
-should be implemented as part of the collection process

Data Implementation
-each attribute is assigned to a specific maintenance system which is responsible to ensure the data is properly implemented

Information Logistics
Data Distribution
Cross dock data (place in staging area, instead of warehouse). -Short term inventory: each user is required to pick up fresh data daily. The data will spend minimal time in the staging area -The staging area contains no summary data (only the lowest level of detail). -No maintenance is done to data in the staging area. All maintenance to source data can only be performed via designated collection and validation processes. -The staging area is not responsible for data quality

Middleware
The connection between the end user and the desired source data Access multiple databases simultaneously without creating a performance bottleneck Remain flexible and scalable by maintaining context information between databases, without impact to the performance of end user tools. Standardized for use with broader spectrum of tools

Summary
Provide transparent access and flexibility to desktop based end users. Identify what data should be stored, what data should be staged, and what data should be accessed directly. Is it better to build a storage facility that provides fast data access or to allow the data to go directly against the operational system.