BI Query Runtime Statistics

Purpose
Using the query runtime statistics, you can determine how much time the execution of certain user actions in the front end and in the analytic engine require. The system records the performance-critical parts of the processing (statistics events). It calculates the net times by calculating the runtime of an event using the difference between the start and end times (minus the times for other events called from within the event). The BI query runtime statistics incorporates the following areas, which clearly differ with relation to event processing: Front End and Calculation Layer of the Analytic Engine Description This area includes the front end and OLAP including BI integrated planning (Front End/OLAP). There is a large number of various events that are processed serially. Examples Front End: Display of Web items, building of entire page, data provider and data area provider command processing OLAP: Generation of queries, creation of cache entries, quantity conversion BI Integrated Planning: Writing of delta records, saving of data, execution of a planning function

Aggregation Layer of the Analytic Engine Description This area includes the data manager (Data Manager). There is a small number of various events that are processed in parallel. Examples MultiProviders, aggregate splits, database access times (particularly access to E and F tables), RFC times

The statistics data is stored in various tables. The data is merged with the InfoProviders for the technical content (see also Analysis of Statistics Data).

Implementation Considerations
In the maintenance for statistics properties, you can specify for each object for which statistics data is to be recorded (query, workbook, Web template), whether and at which granularity you want to record the statistics data. You can also switch the statistics on or off for all queries of an InfoProvider. (See Maintenance of Statistics Properties.)

Integration
Some BI accelerator tests in the analysis and repair environment work with statistics data (see Checking BI Accelerator Indexes (Transaction RSRV), tests: Propose Delta Index for Indexes, Compare Large Fact Tables with Fact Index). As a prerequisite, the statistics have to be switched on for the relevant InfoProvider.

Features
The following figure provides an overview of the process for a runtime reading with the BI query runtime statistics:

The runtime reading starts with the first user action (for example, with the initial execution of a Web template) and finishes when the session is ended by the user (log out). All times for this session are saved under the same SESSIONUID. This can involve numerous user actions (such as navigation steps, updating of Web template, planning). Each user action is defined as a step and generates a new STEPUID if it contains events relevant for the statistics. The times for the event are summarized under this UID. Events within a step are assigned to the relevant context (such as Front End, OLAP, Planning). The context is specified by the handle type. If events in the same context, that is, with the same handle type, are executed for various objects within a step, the system differentiates between them using the different handle IDs. Example 1: A Web template containing two queries is executed. This is a step. Both queries pass through the OLAP processor, for example, the Reading of Master Data event, which belongs to the handle type OLAP. The handle IDs are different: The first query is assigned the ID "1" and the second query is assigned the ID "2". Example 2: A Web template is executed. This involves displaying various Web items. This time is recorded under the Web Reporting Item Rendering event, which belongs to the handle type W3_I. Each new Web item is recorded under a new handle ID. A handle consists of the tuple from the handle type and the handle ID. A handle has only one object for which events are recorded. The object type is attached to the handle type. The object for the handle type OLAP is a query; the object for the handle type W3_I is a Web item; the object for the handle type W3_T is a Web template. The name of the object is also saved. If an event is executed multiply in the same context (that is, during the same session, in the same step, and with the same handle), the system cumulates the times for the event. For each event, it calculates the net time by subtracting from the runtime the times for other events called from within the event, if applicable.

In the event OLAP: Read Data (event 3100), a data request is sent to the data manager; the system therefore records event 9000. For event 3100, however, the system only logs the time before and after the data manager is called. In the area of the front end and calculation layers of the analytic engine ( FE/OLAP), the accumulated times of all events of a step can therefore be less than or equal to the overall runtime of the step. This does not apply in the area of the aggregation layer of the analytic engine ( Data Manager), since multiple processes, and thus multiple events, are executed in parallel. When a request is sent to the data manager in the context of a query execution from the OLAP area, the system records the event Data Manager (event 9000) for the handle type OLAP. It then records the exact times and data separately in the data manager statistics. The front end/OLAP and data manager data are then linked by the key from STEPUID and the handle (handle type, handle ID). Events and Handles Events and corresponding short descriptions for them are in the table RSDDSTATEVENTS. You can create new events using the table maintenance (transaction SM30). Statistics events have two forms: pure counter events and time events. For pure counter events, the indicator is selected in the Count Only column. The system does not record time, but cumulates an integer value for this event. Event 2525 counts the read accesses for the OLAP cache. You can therefore identify from this figure, or from the fact that this event does not exist, whether a query uses the OLAP cache. If the event is not a pure counter event, the system always records a time. It can also write a counter, if required. What the counter reveals in some cases, results from the Description of the event. The Start-End column has just one technical meaning in the way data is recorded. Handle types and corresponding short descriptions for them are in the table RSDDSTATHANDLTP. You can create new handle types using the table maintenance (transaction SM30).

Analysis of Statistics Data

Use
To analyze the statistics data, you have the following options: Using the technical content Using the query monitor (transaction RSRT1) Using the database tables or the predefined views

Features
Technical Content Some objects from the technical content are used for the BI query runtime statistics. Prerequisite to this is that you have activated the technical content (see Installing Business Content). Query Monitor (Transaction RSRT1) Using the statistics data, in the query monitor, you can perform an ad hoc analysis of the query runtime.
...

1. Select the required query and choose Execute + Debug. The Debug Options dialog box appears. 2. Select the indicator for the debug option Display Statistics Data and execute the query. 3. When the query is executed and is displayed with the various navigation steps, press F3 (Back). The Statistic Data for Query Runtime screen appears. The data for the area for the front end and calculation layer of the analytic engine ( Front End/OLAP), and the data for the area for the aggregation layer of the analytic engine (Data Manager) are displayed on two separate tab pages. Database Tables and Predefined Views The statistics data is distributed across various database tables. An analysis using two predefined views (RSDDSTAT_OLAP and RSDDSTAT_DM) is therefore the easiest. To do this, use the Data Browser (transaction SM16). The RSDDSTAT_OLAP view contains the data from the events from the areas for the front end and calculation layer of the analytic engine ( Front End/OLAP): Field SESSIONUID STEPUID HANDLEID HANDLETP EVENTID UNAME STEPTP STEPCNT UTIME CALDAY Description UID of the user session (a roll area) UID of user step Counter for the runtime object Type of runtime object ID (NUMC9) of the event from the RSDDSTATEVENTS table User name Type of step (RSDDSTATSTEPTP table) Ascending count of steps Time (type TIMS) from STARTTIME field Calendar day (type DATS) from STARTTIME field

RUNTIME INFOPROV OBJNAME OBJPROP

Duration of a step in seconds InfoProvider (when valid) Name of runtime object (such as query or Web template) Properties of the object encoded as CHAR10 Example for queries: Read mode (see RSRREADMODE data element) Mode for data integrity (see RRACTUALDATA data element) Delta cache on/off (see RRDELTACACHE data element) Partition mode (see RRSPPARTIONMODE data element) Cache mode (see RSRCACHEMODE data element) Persistence mode (see RSRPERSISTMODE data element)

STATLEVEL EVTIME EVCOUNT EVENTIDCNT STARTTIME

Statistics detail level (0, 1, 2) (Net) Runtime of event Counter for this event (not required for all events) Number of calls for this event (for internal use only) Start time of step in yyyymmddhhmmss,mmmuuun format.

The RSDDSTAT_DM view contains the data from the events from the area for the

aggregation layer and analytic engine (Data Manager): Field STEPUID HANDLEID HANDLETP DMUID ACCESSCNT UNAME UTIME CALDAY OBJNAME INFOPROV PARTPROV AGGREGATE TABLTP TIMEDMPREP Description UID of user step Counter for runtime object Type of runtime object UIP for the data manager access Data access counter (such as database, BI accelerator, RFC) during a data manager event User name Time (type TIMS) from STARTTIME field Calendar day (type DATS) from STARTTIME field Name of runtime object (such as query or Web template) InfoProvider If INPROV is a MultiProvider, PARTPROV specifies the InfoProvider contained in it Technical name of the aggregate or BIA index (if applicable) Type of fact table (F or E) if InfoCube or aggregate was accessed Preparation time of data access, for ACCESSCNT = 0 only, since not parallel

TIMEDMPOST TIMEREAD TIMESID TIMENAVATTR TIMEHIERARCHY DBSEL DBTRANS WP_ID STARTTIME

Postprocessing time for data, for ACCESSCNT = 0 only, since not parallel Data read time (for example, database, RFC) Time for calculation/determination of new SIDs Time for reading master data Time for hierarchy handling Number of records read from the database Number of transferred records ID of work process in which the (possibly parallel) data read access was executed Start time of step in yyyymmddhhmmss,mmmuuun format

Maintenance of Statistics Properties

Use
Within the scope of the BI query runtime statistics, the system can collect data for the following types of BI objects: BEx query BEx Web template BEx workbook InfoProvider (with restrictions) On the Maintenance of Statistics Properties screen (transaction RSDDSTAT), you can edit the statistics properties for individual objects as well as the default settings for the object types listed above. You can also specify settings for the data transfer process (DTP).

Integration
From the Data Warehousing Workbench screen, you can navigate to the maintenance for the statistics properties by choosing Tools Settings for BI Statistics.

Features
Changing Statistics Properties The various objects are located on the corresponding tab pages. You can use the sorting and filter functions for the table to preselect the objects to be changed.
...

1. Select the objects you want to change. 2. Select the required settings using the Settings selection list for the object. You can choose whether the statistics are to be switched on or off, or you can choose the default setting (D) for the object type. For queries, you can also specify the detail level for the statistics. 3. To apply the settings to the list, choose Replace Values. The system sets the relevant indicator in the Last Changed column. 4. Choose . Default Setting for Object Types Every new object initially has the default setting for its object type as the statistics property. To change this setting, choose Extras -> Change Default. The dialog box with the current default settings for the object type appears; you can change the default values here. All objects of this type with "D" in the Statistics On/Off column are then treated according to the new default setting when statistics are recorded. An exception to this rule is the Query object type: if no specific statistics property is set for a query, that is, the default value "D" is set, the system first determines the statistics property for the InfoProvider of the query. If the InfoProvider has the value "D" as the statistics property, the system reverts to the default setting for the query; otherwise, it uses the InfoProvider setting (with the OLAP detail level from the InfoProvider setting). A new query is created. It has the statistics setting "D". The default setting for the query is Statistics Off. The InfoProvider for the query, however, explicitly has the setting Statistics On (not as default setting). In this particular case, the system records statistics data for the query.

This derivation relates only to the InfoProvider on which the query is directly based. If this involves a MultiProvider, no additional derivation is made of the possible settings from the InfoProviders contained the MultiProvider. Statistics Detail Level for the Query Object Type For queries, you also have the option of selecting a detail level for the statistics data. You can choose from the following: 0 Aggregated Data: The system writes only one OLAP event (event 99999) for the query. This contains the cumulative times within the OLAP processing of the query. The system does not record data from the aggregation layer of the analytic engine ( Data Manager) or aggregation information. 1 Only Front End/Calculation Layer Data: The system records all OLAP events, but not separate data from the aggregation layer of the analytic engine ( Data Manager). The system writes only the general data manager event 9000 in the OLAP context as well as the aggregation information. 2 All: The system records all data from the area for the front end and calculation layer (Front End/OLAP) as well as data from the area for the aggregation layer ( Data Manager) and aggregation information. 9 No Data: The system does not record any data from the front end and calculation layer (Front End/OLAP), or from the aggregated event 99999. However, it does record data for the BEx Web templates and workbooks, depending on the setting. When you select the detail level, keep in mind that a very large amount of data is recorded when the system records data manager times. A Web template with four queries is executed. Each query is based on a MultiProvider that contains ten InfoProviders; each of these InfoProviders has data in the E table and F table. Due to specific query properties, the query is split into two parts. Each part is based on a different aggregate. This results in: 4 queries * 10 InfoProviders * 2 parts * 2 table types = 160 records in the RSDDSTAT_DM table view (plus the records from the area for the front end and calculation layer of the analytic engine (Front End/OLAP) in the RSDDSTAT_OLAP table view). InfoProvider Tab Page In addition to the default setting described for queries of an InfoProvider, you can also change the following statistics properties for an InfoProvider: Statistics data for aggregate processes (fill, roll up, change run, condense): You can switch the recording on or off. The recorded data is stored in the RSDDSTATAGGR table. Data from the aggregation layer (Data Manager) for the external BI read interface (RSDRI): You can switch the recording on or off. Since queries through the external read interface do not run through the OLAP processor, for this type of request, the system records only the event 9001 ( External Read Interface) with the handle type EXTN in the area for front end and calculation layer ( Front End/OLAP). Processing in the area for the aggregation layer ( Data Manager) does not differ from a "normal" BEx query; the relevant statistics can therefore be recorded for this.

For InfoProviders, the statistics detail level relates to the default setting for the queries of the InfoProvider only, and not to the processes listed above. Deleting Statistics Data Statistics data should generally be deleted when data is loaded to the InfoCubes of the technical content. If the technical content is not activated, or if the data is to be deleted from the statistics table for some other reason, you can also do this manually in the maintenance for the statistics properties. When you choose Delete Statistical Data, the dialog box for restricting the areas in which the statistics data is to be deleted appears. You can select multiple areas. Query Statistics Tables: The system deletes the data for the BI query runtime statistics. Statistic Logging Tables: The system deletes the logging data for the BI query runtime statistics. Aggregates/BIA Index Processes: See Statistics for the Maintenance Processes of a BI Accelerator Index. InfoCube Statistics (Delete, Compress): The system deletes the data of the InfoCube statistics that results when data is deleted from an InfoCube or when data requests of an InfoCube are compressed. DTP Stats: The system deletes the data for DTP statistics. Using the Up to Day (Incl.) field, you can enter a date up until which the system is to delete the statistics data. If you do not enter a date, all data is deleted. Since this can be executed with a command (TRUNCATE TABLE), (and not using selective deletion in the database), this version is considerably faster. By restricting to one day, packages of 1000 records only are always deleted from the tables; this is followed by a database Commit. This makes it possible to restart after a termination (resulting from a TIMEOUT, for example), without a need to redo previous work. You can also use the RSDDSTAT_DATA_DELETE program to delete data from the statistics tables. The dialog box mentioned above for restricting the areas from which the data is to be deleted, is also displayed when you use this program.

BI Administration Cockpit (New)

Technical Data Function Is In Release New Software Component Component: SAP NetWeaver Release: 2004s BW-BCT-TCT Technical Content Valid for all countries

Assignment to Application Component Country Setting

Use
You use the BI Administration Cockpit to perform administration tasks in BI more simply and quickly. You can call the BI Administration Cockpit from the BI Administration 1.0 business package in SAP Enterprise Portal. It supports BI administrators in monitoring statuses and optimizing performance by providing an overview of the objects and processes in BI systems. It provides BI administrators with a central point of access that focuses on critical situations and allows them to navigate to more detailed information as well as error handling and optimization applications. You can also use the new technical Content, on which the BI Administration Cockpit is based, to create additional evaluations and reports. The technical Content has been enhanced in the following areas: Complete redesign of query runtime statistics Enhancement of data-load statistics to include statistics for process chains and data transfer processes Current data-load status of process chains and processes Current status of loaded requests New InfoObjects, data flows and extractors have also been created. The new technical Content allows you to perform evaluations that the old technical Content did not offer.

Effects on Existing Data

The BI Administration Cockpit is based on new InfoProviders that are delivered with the technical Content. Essentially these represent an enhancement of the existing technical Content with one exception: The query runtime statistics are updated to other detailed tables, offering a more exact evaluation. As a result, the following InfoCubes for statistics are obsolete as of NW 2004s: 0BWTC_C02 and 0BWTC_C03. Statistics that you created before the upgrade can still be evaluated with these InfoCubes, while the new InfoProviders are only available for query-runtime statistics that are generated after upgrading to NW 2004s. It is not possible to migrate old statistics into new InfoCubes. The new InfoProviders are: For more highly aggregated query-runtime statistics: 0TCT_C01, 0TCT_VC01 and 0TCT_MC01 These replace InfoCube 0BWTC_C02. For more detailed query-runtime statistics: 0TCT_C02, 0TCT_VC02 and 0TCT_MC02 These replace InfoCube 0BWTC_C02.

 For data manager statistics:

0TCT_C03, 0TCT_VC03 and 0TCT_MC03 These replace InfoCube 0BWTC_C03. For data-load statistics of process chains and processes: 0TCT_C21, 0TCT_VC21 and 0TCT_MC21 For data-load statistics of data transfer processes: 0TCT_C22, 0TCT_VC22 and 0TCT_MC22 For data-load statistics of InfoPackages: 0TCT_C23, 0TCT_VC23 and 0TCT_MC23 These deliver essentially the same information as InfoCube 0BWTC_C05 but they use the new InfoObjects. The remaining new InfoProviders also use the new InfoObjects. For the current data-load status of process chains and processes: 0TCT_VC11 and 0TCT_MC11 For the current status of requests loaded to InfoProviders, InfoObjects that have been updated flexibly, and PSA tables: 0TCT_VC11 and 0TCT_MC11 For use in combining both: InfoCubes for histories (not suitable for current status) RemoteProviders for current data MultiProviders. Queries for the new MultiProviders are delivered with the corresponding Web templates. A workset of business package BI Administration 1.0is also delivered with portal pages and iViews which contain one BI Web template each. These are components of the BI Administration Cockpit. You can also execute the queries and Web templates (with limitations) without the portal.

Effects on Data Transfer

It is not possible to migrate old statistics into new InfoCubes.

Effects on Customizing
For the new statistics, you have to determine the objects for which you want to update the statistics and to what level of detail. You call this dialog from the Data Warehousing Workbench. It has changed completely from the previous statistics functionality. In addition, you can manually assign a priority to the queries, InfoProviders and process chains. This priority is evaluated in the statistics. This can be used to establish a ranking or to exclude objects from the display in a report.

Processing Queries
A query can be divided into sub-queries by the system. This division is performed in the following circumstances: The query contains cells with a constant selection or other definitions that cannot be read using a logical read operation. The query is defined on the basis of a MultiProvider and, due to the selection conditions, data has to be read from a number of InfoProviders. See, Dividing a MultiProvider Query into Sub-Queries. The query contains complex selections, such as calculated or restricted key figures with hierarchy conditions. In this case, the selections are split and accessed individually. This is similar to the process performed in order to use InfoCube aggregates. The query is defined on the basis of an InfoCube, both fact tables contain data, and the database view does not use both fact tables to read the InfoCube. If dividing the query results in more than one sub-query, the read operation is performed in parallel by default.

Parallel Processing
Several dialog work processes are required in order to execute queries in parallel. The maximum degree of parallelism determines the maximum number of work processes that are used for each query. This value is restricted to 6 by default. The maximum value can be changed to a value between 1 and 100 in the QUERY_MAX_WP_DIAG entry in table RSADMIN. The actual degree to which queries are executed in parallel depends on the load on the system at any given time and lies between 1 (sequential processing) and the maximum value. If the number of sub-queries is greater than the maximum level of parallelism, all existing sub-queries are divided between the work processes determined by the degree of parallelism. The results of all sub-queries are collected at a synchronization point and collated to form an overall result. In sequential processing, the sub-queries are processed one after another. The interim result is immediately passed on to the analytic engine.

Sequential Processing
In the following cases, the system chooses sequential (non-parallel) processing: In table RSADMIN, entry QUERY_MAX_WP_DIAG has value (column value) 1. The entire query consists of one sub-access only. The query is running in a batch process. The query was started from the query monitor (transaction RSRT) using various debug options (for example, SQL query display, execution plan display). See, Dividing a MultiProvider Query into Sub-Queries. The query requests non-cumulative key figures. Insufficient dialog processes are available when the query is executed. These are required for parallel processing. The query is configured for non-parallel processing. You want to save the result of the query in a file or a table. In Release SAP NetWeaver 2004s, the system can efficiently manage the large intermediate results produced by parallel processing. In previous releases, the system terminated when it reached a particular intermediate result size and

proceeded to read data sequentially. This is no longer the case. Therefore, the RSADMIN parameter that was used in previous releases for reading a MultiProvider sequentially is no longer used.

Partial Parallel Processing

Queries that are based on non-cumulative InfoCubes have a high memory consumption and cannot be processed in parallel. For this reason, all queries that are based on non-cumulative InfoCubes are divided into sub-queries that are separated from the other sub-queries. The normal sub-queries are processed in parallel first and the sub-queries that are based on the noncumulatives are processed sequentially afterwards.