SOLUTIONS

Module BI2-M3: ETL

Topic: Transaction Data ETL from Flat Files

Exercise 2 Load transactional data for reporting in three steps:

Source system to the PSA PSA to the DataStore object (<root>D1nnn) DataStore object to the InfoCube (<root>R1nnn)

Refer to the Transactional Data Flow Global Bike roadmap in the BI2-M3 ETL slides as you work through the exercise.

2010 SAP AG

Page 127

SOLUTIONS
Solution 2.1
Create and name a flat file DataSource for loading transactional data from the flat file Source System. The data given in three files BicycleCompTransDataAU.csv, BicycleCompTransDataDE.csv, BicycleCompTransDataUS.csv, which contain the data from Australia, Germany and United States. Assign <root>T1nnn as the technical name as well as the short, medium and long descriptions. Specify the necessary extraction information for the flat file, obtain field proposals, activate the DataSource and preview the results. Note: The steps are almost identical to those for the master data DataSources. Since all threes flat files are of identical structure, it is sufficient to create one single DataSource in your Application Component and choose Transaction Data as the data type.

Enter the descriptions, <root>T1nnn.

Choose the Extraction tab and select No DTP Allowed for Direct Access. This means that the data are loaded from the PSA. Specify the file location with one header row. Select CSV as the data format with a semi-colon data separator. Choose Direct Entry for the number format with no thousands separator and decimal separator (,).

2010 SAP AG

Page 128

SOLUTIONS

Select the Proposal tab and click on Load Example Data to view the data with proposed field names and attributes.

Click on the Field tab to adjust the formats as needed. All fields not currently in Internal SAP format must be changed to External. The CURR data type for currency key figures must be selected. Add CURRENCY in the cur/unit column, as needed. Additionally, for the date fields DAY, MONTH, YEAR, the data type NUMC (character with numeric values) is appropriate for further processing. Make sure that your length and decimal place columns match the example below .

2010 SAP AG

Page 129

SOLUTIONS

Save and Activate the DataSource. Accept the warning that YEAR, MONTH and DAY names are reserved.

Preview the data.

Solution 2.2
Create three InfoPackages (one per file), load the data into the PSA and check it. Right click the DataSource and select Create InfoPackage.

2010 SAP AG

Page 130

SOLUTIONS

Enter Load AU into PSA (<root>T1nnn) as the description for the data of Australia and click Save.

Click Save followed by Start on the Schedule tab to execute the load into the PSA. When finished (see Data was requested message), click on the monitor.

2010 SAP AG

Page 131

SOLUTIONS

Click on the PSA icon to view the data.

Select the first data package and continue.

2010 SAP AG

Page 132

SOLUTIONS

The first 1000 records in the PSA load are displayed.

Hint: Data records can be edited if necessary . Repeat these steps to create InfoPackages and load the DE and and the US file as well. It is convenient to use the copy function.

Be sure to use the Extraction tab to change the file name.

2010 SAP AG

Page 133

SOLUTIONS

Hint: If necessary data can be deleted form PSA as follows: Display DataSource and choose Manage PSA.

Loaded InfoPackages are shown and can be deleted

2010 SAP AG

Page 134

SOLUTIONS

Solution 2.3
Create a transformation and data transfer process to load your DSO from the PSA. View and activate your data in the DSO. Right click your DataSource and select Create Transformation.

Select DataStore Object as the Object Type and your DSO, <root>D1nnn.

2010 SAP AG

Page 135

SOLUTIONS

Some rules are proposed by the system.

To define additional rules click each field on the left and drag the arrow to map the transformation to the appropriate InfoObject on the right.

Note: When dragging YEAR and MONTH the system prompts a information similar to following which can be accepted.

2010 SAP AG

Page 136

SOLUTIONS
For the date field 0CALDAY a transformation rule has to be defined. The internal date format is YYYYMMDD whereas the datasource has day month and year in three separate fields. Drag the three separate date field on the left to 0CALDAY on the right and accept the information.

Double click the Rule symbol transformation rule using a formula

to change the rule.

Define the

The formula can be defined by concatenation of the fields YEAR, MONTH and DAY in this order which can be done by using the operator & since these fields are of type NUMC. This leads to the internal date format YYYYMMDD.

2010 SAP AG

Page 137

SOLUTIONS

Click the back button to return to the Rule Details. The formula is saved. Enter an appropriate name that describes the rule and Transfer the rule.

2010 SAP AG

Page 138

SOLUTIONS
A special icon indicates the rule type.

Activate the transformation. Accept the 0RECORDMODE notification to continue the activation process.

Right click your DataSource to create a Data Transfer Process (DTP) for the DSO load.

Accept the DTP creation as is.

2010 SAP AG

Page 139

SOLUTIONS

Change the extraction mode to Full and allow the extraction to occur from the PSA. Activate the DTP.

Execute the DTP.

2010 SAP AG

Page 140

SOLUTIONS

Accept the offer to open the monitor.

Since the data load might take a few seconds, you may want to refresh the screen. If the load is successful, you will see green indicators. Otherwise, you will need to investigate the problems.

2010 SAP AG

Page 141

SOLUTIONS
Click on bullseye icon in order to administer the DSO and view your data. The data you loaded into the DSO are considered New and are not yet Active for reporting purposes. Choose the Contents tab. Note that you can view either the new or active data. Select to view the new data.

Click on the Number of Entries and you will see the number of rows loaded from the PSA. Close the window.

Repeat the process for active data. Since this is your initial load into the DSO, there are no active data records.

2010 SAP AG

Page 142

SOLUTIONS

Select the Requests tab and click Activate.

Select your request in order to activate the new data and click Start.

2010 SAP AG

Page 143

SOLUTIONS

Use Refresh to determine when the activation is complete (Request ID disappears). Close the window to return to DSO maintenance.

Check the number of new data rows (now 0) and active data rows (equal to the DSO load).

2010 SAP AG

Page 144

SOLUTIONS

After checking the number of active data rows, request a list of the active DSO data.

You may view the first 200 rows of data in the DSO.

2010 SAP AG

Page 145

SOLUTIONS

Solution 2.4
Create a transformation and data transfer process to load your InfoCube from your DSO. View your data in the InfoCube as well as the data flow. Right click your InfoCube (<root>R1nnn) and select Create Transformation.

Specify the DataStore object type, provide the name of your DSO (<root>D1nnn) and create the transformation.

2010 SAP AG

Page 146

SOLUTIONS
All the rules are generated except those for 0CALMONTH (which comines yaer and month), MU0NETSA (Net Sales) and MU0COGS (Cost of Goods Sold). Click on 0CALDAY description on the left and drag rule to 0CALMONTH.

The icon indicates that the system automatically does a time conversion. Connect MU0DSCNT and MU0REVEN to MU1NETSA (0CURRENCY is automatically added by the system). Double click the rule to get the details.

Choose Formula as the rule type. Note that 0CURRENCY, MU0DSCNT and MU0REVEN appear as Source Fields.

2010 SAP AG

Page 147

SOLUTIONS

Enter the formula, /BIC/MU1REVEN - /BIC/MU1DSCNT, by double clicking each key figure to add it and clicking the minus sign (in order). Use the back arrow to leave the editor. The formula will be saved.

Click Transfer Values to accept the rule.

2010 SAP AG

Page 148

SOLUTIONS

Notice that a formula rule now exists for MU0NETSA.

Now define the transfer rule for MU0COGS (Cost of Goods Sold). This has to be calculated by Sales Quantitiy * Transfer Price where Transfer Price is an attribute of the material MU0MATRL.

2010 SAP AG

Page 149

SOLUTIONS

The lookup of this Transfer Price and the calculation has to be done by a rule of type Routine which uses the source fields MU0SLQTY and MU0MATRL. Connect MU0SLQTY and MU0MATRL to MU0COGS (0BASE_UOM is automatically added by the system). Double click the rule to get the details.

Choose Routine

2010 SAP AG

Page 150

SOLUTIONS

Add the source code to read the Transfer Price and calculate the result. Please note: Your code may differ since you have used your own InfoObjects for Material (<root>MANNN) and Sales Quantity (<root>QTYNNN).
SELECT SINGLE /BIC/MU0TRPRI FROM /BIC/PMU0MATRL INTO RESULT WHERE /BIC/MU0MATRL = SOURCE_FIELDS-/BIC/MU0MATRL AND OBJVERS = 'A'. RESULT = RESULT * SOURCE_FIELDS-/BIC/MU0SLQTY.

2010 SAP AG

Page 151

SOLUTIONS

Check the source code and save . This returns to the previous Window. Since Transfer Price and Cost of Goods Sold both are in fixed currency USD no currency conversion is necessary. Choose No Conversion.

2010 SAP AG

Page 152

SOLUTIONS

Accept the deletion of assigned unit fields.

Add an appropriate text to describe the rule and hit Transfer Values to finish the definition. The transformation mapping now reflects the routine rule. Activate the transformation .

2010 SAP AG

Page 153

SOLUTIONS

Right click on your InfoCube and create a DTP.

Accept the DTP as it is presented.

Select the full extraction mode and activate the DTP.

2010 SAP AG

Page 154

SOLUTIONS

Execute the DTP from the DSO into the reporting InfoCube. Accept the prompt to view the monitor (not shown).

The DTP completed successfully with all records loaded into the InfoCube. Note that data has been summarized since DSO is on a daily level whereas InfoCube has monthly granularity.

2010 SAP AG

Page 155

SOLUTIONS

Right click your InfoCube and display its contents (instructions for listing contents provided earlier). Note the Display Data Flow option. It will be used in the next step.

Choose

and select the fields shown in the output

below. Choose Execute . Notice that Net Sales and Cost of Goods Sold are calculated in the transformation.

Display master data of Material MU0MATRL and check that Cost of Goods Sold is calculated properly as Sales Quantity * Transfer Price.

2010 SAP AG

Page 156

SOLUTIONS

Right click your InfoCube again to display the data flow.

2010 SAP AG

Page 157