Keith Berry US-Analytics US-Analytics is an industry leading professional services firm focused on helping clients successfully establish and maintain long term Business Intelligence (BI) and Enterprise Performance Management (EPM) applications and programs. BI and EPM Strategy and Processes Custom and packaged BI and EPM Applications Oracle Infrastructure Managed Services and Hosting For over a decade, market leading companies have trusted US-Analytics to solve complex business problems, drive managerial excellence, and deliver operational agility. Learn more at Booth #107 or www.us-analytics.com 12+ Years Hyperion implementation experience Certified in HFM and Essbase Second year at Kscope First EA presentation in 2004 Background Tool for exporting HFM data to a relational database in star schema format Prior to 11.1.2.2, the only way to export data not in <Entity Currency> As of 11.1.2.2, fully integrated with Data Export Extended Analytics What it is Writes directly to a relational database (system to system) Exports metadata and data Flexible star schema format supports analysis and transformation Why use it? Setup Create target database/schema Separate from HFM application database Can have multiple Setup Target Databases Create UDL (Universal Data Link) file File containing target database connection information One per target database Copy to each HFM server Setup Target Databases Registers UDL with HFM server 11.1.2.1 and prior Start All Programs Oracle EPM System Foundation Services EPM System Configurator 11.1.2.2 Start All Programs Oracle EPM System Financial Management Extended Analytics DSN Configuration DSN will now appear on Data Extract screen Configure DSN Configure DSN 11.1.2.1 and prior EA and Data Export separate tasks Administration Extended Analytics 11.1.2.2 EA and Data Export fully integrated Consolidation Extract Data Create Export Step #1 Set POV Step #2 Set Extract Destination Type Database of flat file Extract Format Standard Data Warehouse Essbase Metadata Only Selected Metadata Only Template Extract Dynamic Accounts Calculated Data Derived Data Values generated by ZeroView settings Line Item Detail n/a Step #3 Set Options Schema Actions Create, Update or Delete Destination Database DSN configured earlier Table prefix ten alphanumeric characters must start with character no spaces no underscore Step #3 Set Options What Happens Create Database Creates new tables if not already present Clears Fact Table and re-exports data if tables already present and metadata has not changed Drops and rebuilds all tables if metadata has changed Update Database Adds or updates records in Fact Table for existing POV Does not affect existing data outside of POV Will not execute if metadata has changed or schema does not exist Delete Database Deletes all tables Doesnt indicate if tables existed in the first place What is Exported Table layout optimized for dimensional data Central fact table One field for each dimension One or more fields for data Dimension tables Dimension tables = look-up tables Has corresponding value for each value in the related Fact table field Dimension information in parent-child format Contains additional information about the dimension field such as description, user-defined fields, etc. Fact table and dimension tables can be joined to manipulate and aggregate the data along dimension lines Database topology looks like a star Star Schema Format Star Schema Format 2 2013 43 Actuals 84 Jan 3 USD 88 California 19 Net Income 2 84 88 3 19 43 $100 Actuals, 2013, Jan, California, USD, Net Income - $100 Fact Table Value Dim Table Period Dim Table Scenario Dim Table Year Dim Table Account Dim Table Entity Dim Table Table Sets Assign table prefix during setup HFM creates and populates tables during each export All tables created during an export share the same prefix Version exports via prefixes Fact Table ScenarioID YearID PeriodID ViewID EntityID ParentID (for Entity parent) ValueID AccountID ICPID Custom1ID Custom2ID, etc. dData (float) Each field contains a numeric ID which corresponds to the ID field of the appropriate dimension table ID numbers are stable across exports for the same set of metadata Primary Key for the fact table is the composite of all dimension fields Dimension Tables Standard export format Dimension Table (prefix_dimension name) ID Label ParentID ParentLabel Description IsShared IsLeaf The above fields are standard for all dimensions Additional fields available based on individual dimension characteristics Dimension tables also vary by export type Boolean values (IsShared, IsLeaf) stored as Integer (1=True, 0=False) Additional dimension fields Scenario (prefix_SCENARIO) UserDefined1 UserDefined2 UserDefined3 DefaultView (lookup to ID in View Dimension table) Year (prefix_YEAR) None Period (prefix_PERIOD) None View (prefix_VIEW) None Additional dimension fields Entity (prefix_ENTITY) UserDefined1 UserDefined2 UserDefined3 IsICP DefaultCurrency (lookup to ID in Value Dimension table) Entity Parent (prefix_PARENT) UserDefined1 UserDefined2 UserDefined3 IsICP DefaultCurrency (lookup to ID in Value Dimension table) Value (prefix_VALUE) None ICP (prefix_ICP) None Additional dimension fields Account (prefix_ACCOUNT) UserDefined1 UserDefined2 UserDefined3 IsCalculated IsConsolidated IsICP AccountType Custom (prefix_CUSTOMX) UserDefined1 UserDefined2 UserDefined3 IsCalculated SwitchSign SwitchType AggWeight SQL View creates data table view from source tables: Combined View Standard SQL provided in appendix Change table prefix Adjust number of customs Recommend all access to data through View Combined View Export types Metadata Only Selected Metadata Only Standard Data Warehouse Essbase Export types Metadata Only All metadata Fact table empty Standard export table format Selected Metadata Only Same as Metadata Only, but exports only members selected Standard Export One table per dimension Separate dimension tables for Entity and Parent Standard Export Child repeated in dimension table each time it appears in under a new parent in the source hierarchy Separate join needed for base data and parent adjustment ID and ParentID in Entity table ID in Entity table and ID in Parent table ENTITY Table Data Warehouse ParentID and ParentLabel are dropped from the dimension tables. The entity parent table (prefix_PARENT) is also dropped A second table is created for each dimension (prefix_dimension_PARENT) which holds the parent-child information Data Warehouse Can join Fact table to Entity dimension table without duplicating values More complex when parent information is required (three-table join) ENTITY Table ENTITY_PARENT Table Essbase Essbase A second table for each dimension (prefix_dimension_BASE) Parent entity table Essbase BASE table lists the base members under each parent in the dimension ENTITY Table ENTITY_BASE Table Understanding Data in HFM Eliminations/Adjustments HFM built to support multiple, alternate financial consolidations of the same base data Two levels of data: Data and adjustments in base entity No parent specified Currency specified in Value Dimension member Eliminations/adjustments in specific parent/child combinations Parent specified Named Value Dimension member Currency not stated, but in currency of parent Example France base entity located in two alternate hierarchies Europe parent in first hierarchy Corp parent in second hierarchy Three records loaded to France for 10K, 20K and 5K (total 35K) Application consolidated Automatic eliminations performed by HFM Europe total now 30K Corp total now 15K France Europe Europe.France France HFM Consolidation Corp Data for France under Corp parent Data for France under Europe parent France Corp.France France France Europe - 5K 35K HFM Consolidation Corp Data for France under Corp parent Data for France under Europe parent France -20K 35K 30K 15K Translation Default currency assigned to each member Translation occurs for child member when consolidated to parent with different currency Translation occurs before parent adjustments - USD - USD - CAD - EUR - GBP Example Data loaded to base members as follows: USA 50K USD Canada 40K CAD France 15K EUR UK 10K GBP Consolidation/translation performed - - EUR Translation - USD - USD - CAD - EUR - EUR - GBP Base data for each entity after consolidation/translation Designing Your Exports Assumptions Can aggregate in target system Target system needs base data in a single currency Export in Data Warehouse format Export only base members of Account, ICP, and Custom dimensions Separate base data export from adjustment data export Translation handled on case-by-case basis Recommendations Export in Data Warehouse Format Each member present only once in dimension tables Simple join with no duplicates ENTITY Table ENTITY_PARENT Table Export Base Members for Accounts, ICP and Customs In HFM, parent members in ICP, Account and Custom dimensions calculated dynamically in RAM as required Overhead for processing Overhead for virtual data into real Base data - Load to Entity Split Export Entity [Base] Value Named currency, eg. USD, CAD Adjustment data - Load based on Entity Parent (except named currency Adjs) - Create base members under parent in target system as needed - Identify currency by Default Currency of the parent Split Export Entity [Hierarchy] Named currency Adjs Value [Parent Adjs] [Eliminaton] [Contribution Adjs] Transform Define all HFM parent members with the same default currency Main or alternate hierarchy If alternate, additional processing time Force Translate High overhead Will not update during consolidation Eliminations cannot be force translated Special handling for <Parent Curr Adjs> Translate downstream SQL Target application Translation - Possible Solutions Other Good Things to Know Templates Templates can be created to save export parameters Cannot be shared between users Not part of LCM export Copy utility on HFM server Start All Programs Oracle EPM System Financial Management Utilities Recommend Taskflows Utility tables HFM_EA_EXTRACT Prefix (relational database prefix) AppName (HFM application name) Task (aggregation option) Dimension (ID number) dTimestamp (1900 date system) Tracks the time each table was last updated View provided to convert to readable form HFM_LOCK_ACCESS Tracks when schemas are being written to prevent simultaneous updates Task flows API Code Http Listener (new) see HFM Developers Guide Automation Other Considerations Correct upper level data depends on HFM being properly consolidated before export EA does not trigger consolidation or translation Data keeps the same sign it had in HFM Revenue, Liabilities positive Must accommodate if aggregating downstream Dimension IDs in order they have been added to the system Member ID Not always suitable for dimension builds Handy things you can do with EA. Approach Export all base data for top entities in two HFM applications to be compared Compare with database query Benefits Comprehensive comparison (all dimensions) Automated No maintenance Always captures data present in one application, but not the other Automated Reconciliation Why it Works Parent members in Account, ICP and Custom dimensions calculated dynamically in RAM Base data compare Assumptions Aggregation weights the same in both applications Entity value changes are not offset in a higher entity Automated Reconciliation Setup Create export template for first data set (prefix = TIE1) Execute export Create SQL data view Repeat steps for second data set (prefix = TIE2) Create comparison data view Subsequent Steps Rerun exports Run compare query Steps Sample Export Parameters Recon Compare Query Calculated Data Analysis Tools for Accessing the Data Oracle Oracle SQL Developer http://www.oracle.com/technetwork/developer-tools/sql- developer/overview/index.html SQL Server Microsoft SQL Server Management Studio Express http://www.microsoft.com/en-us/download/details.aspx?id=8961 MS Excel MS Access