Data File Validation for File Based Loader

An Oracle White Paper

September 25 2013

Contents
Contents ........................................................................................................................................ 2
Preface .......................................................................................................................................... 3
Disclaimer ....................................................................................................................................................................... 3
Intended Audience .......................................................................................................................................................... 3

Introduction .................................................................................................................................. 4
What is File Based Loader? .............................................................................................................................................. 4
What is Data File Validator? ............................................................................................................................................ 4

Installing Data File Validator ........................................................................................................ 5

Configuring Data File Validator .................................................................................................... 5
Using Data File Validator Utility ................................................................................................... 9
Extending Data File Validator for Additional Validations ........................................................... 11
Appendix A XML Schema definition for the Configuration File ............................................... 12

2 | Data File Validation for File Based Loader

Preface
Disclaimer
The following is intended to outline our general product direction. It is intended for information
purposes only, and may not be incorporated into any contract. It is not a commitment to deliver
any material, code, or functionality, and should not be relied upon in making purchasing decisions.
The development, release, and timing of any features or functionality described for Oracles
products remains at the sole discretion of Oracle.

Intended Audience
Welcome to the white paper on Data File Validator for File Based Loader utility
This guide assumes you have knowledge of the following:
Oracle Fusion HCM File Based Loader

Getting Started with HCM File Based Loader (Doc ID 1578466.1)

Oracle Fusion Human Capital Management File-Based Loader V1.1 (Doc ID 1533860.1)

3 | Data File Validation for File Based Loader

Introduction
This document provides details on the Data File Validator for File Based Loader and the steps required
installing and configuring it. It also provides information on the various configuration options, reviewing
validation errors, and steps for extending the validation routines.

What is File Based Loader?

File Based Loader (FBL) enables users to bulk-load data to Oracle Fusion HCM from any data
source. Typically, you use File Based Loader for an initial upload of data and maintain the data in
Oracle Fusion HCM thereafter. Additionally, you can use File-Based Loader to load data after the
initial load; for example, you may want to upload changed data to Oracle Fusion HCM if you are
continuing to maintain the source data.
File Based Loader supports a predefined set of Business Objects to be uploaded to Oracle Fusion.
For each of the Business Objects supported in FBL, the data must be created in a DAT file and
each DAT file has a predefined format. You construct the heading row in the data file for each
Business Object type by concatenating the Datastore Attribute Names with pipe separators.
Heading rows must be in capital letters and the columns can appear in any order.
The following example shows a sample Business Unit DAT file.

The DAT files that contain the data provided by the users are validated and loaded on to the
Fusion HCM system using FBL. FBL performs a two-stage process to load the data to Fusion.
Load HCM Data (Import) - In this stage, the data is read from the DAT files and populated into a
set of staging tables. This process performs key resolutions such as cross reference id resolution
and foreign key resolutions and basic data validations on the file formats.
Load Batch Data(Load) - In this stage the data is read from the staging tables and validated against
the business rules validation in the web service. Users can view the validation errors in the user
interface and correct the errors and reload the data.

What is Data File Validator?

During the FBL data loading process, the validations are performed either in the Import phase or
the Load phase. Data File Validator is a utility that enables you to perform most of the data
formatting validations even prior to loading the data in Fusion. This utility can be run in the
source system after you have generated the DAT files. The utility generates an HTML output that
lists the validation errors. You can correct the errors in the DAT file based on the report output.

4 | Data File Validation for File Based Loader

Note: Oracle recommends you to run this utility on the zip file you have generated prior to the
actual load to minimize data validation errors.

Objectives
Data File Validator enables users the:

Ability to perform data validation prior to actual load of data to Fusion.

Minimize the time taken to identify and resolve data validation errors.

Installing Data File Validator

You can download the Data File Validator utility from My Oracle Support Document ID - 1587716.1.
Data File Validator for File Based Loader. The utility is a Java based tool that is available as a zip file
and can be downloaded locally. The utility requires Oracle Java 1.6 or a later version to be installed on
your local machine.
Extract the zip file to a folder. The extract folder contains the utility executable.
Windows Users: Run the FBLValidator.exe
Linux Users: java -jar fblvalidator.jar

Configuring Data File Validator

You require the following information to run the utility.
Configuration File - This contains the structure and validation definitions of the various validations
performed by the Validator tool. Oracle provides a reference configuration file which can be downloaded
from My Oracle Support Document ID - 1587716.1. Data File Validation for File Based Loader.
Configuration file is an XML file that contains metadata about the various Field and Record level
validations performed as a part of the validation process.
Field Level Validations

Data Type Validations - GUID, VARCHAR, DATE, NUMERIC, DECIMAL

Mandatory Checks - Not Null checks
Range Validations - Attribute value in a given range, e.g. Rating Value < 5
List Validations - Attribute value in static list, e.g. Active Status in (Active, Inactive)
Regular Expression Validations - Attribute value matches regular expression pattern, e.g.
Email address in <string>@<string> format
Lookup Validations (Optional) - Attribute value in the lookup file, e.g. Phone Type is
PHONE_TYPE lookup

5 | Data File Validation for File Based Loader

Record Level Validations

Column value Comparisons - Compare two attributes in an entity, e.g Start Date < End Date
Complex Validations using Groovy - Any multi-attribute procedural code, e.g. Conditional
mandatory checks, Attribute Combination validation etc
Leverage the existing validations that exist in Business Rule validations.

A sample section of the configuration file is given below:

In the sample, each business object validation is defined within a <business-entity> tag. <attribute>
indicates the various fields in the DAT file for the Business Object.
Type of Validation

Example Format

Data Type

<business-entity name="POSITION">
<attribute name="POSITION_ID" data-type="GUID" mandatory="true"/>

Mandatory Check

<business-entity name="POSITION">
<attribute name="POSITION_ID" data-type="GUID" mandatory="true"/>

Range Validation

<attribute name="STAR_RATING" data-type="NUMERIC" mandatory="false">

<attribute-validations>
<range-validation lower-limit="0" upper-limit="10" include-bounds="true"/>
</attribute-validations>
</attribute>

List Validation

<attribute name="ACTIVE_STATUS" data-type="VARCHAR" mandatory="true">

<attribute-validations>
<list-validation>
<list-value>A</list-value>
<list-value>I</list-value>
</list-validation>
</attribute-validations> </attribute>

6 | Data File Validation for File Based Loader

Regular Expression
Validation

<attribute name="EMAIL_ADDRESS" data-type="VARCHAR" mandatory="true">

<attribute-validations>
<regexp-validation pattern="[a-zA-Z0-9_.\-]+@[a-zA-Z0-9_.\-]+" description="Only
alphabets, numbers, hyphen, underscore, dot and one occurrence of @ are allowed in a email
address"/>
</attribute-validations>
</attribute>

Lookup Validation

<attribute name="HIRING_STATUS" data-type="VARCHAR" mandatory="false">

<attribute-validations>
<lookup-validation lookup-type="HIRING_STATUS"/>
</attribute-validations>

Groovy Validation

<groovy-validation description="Legal Entity is mandatory for person type other than EMP,
APL and CWK">
<code>
<![CDATA[ if (SYSTEM_PERSON_TYPE.equals("EMP") ||
SYSTEM_PERSON_TYPE.equals("APL") || SYSTEM_PERSON_TYPE.equals("CWK")) {
if (LEGAL_ENTITY_ID == null)
return true;
else
return false;
}
return true;
]]>
</code>
</groovy-validation>

Optionally, you can configure the following parameters:

Lookup File - This file provides the list of lookup values that you can use for validation. A sample
lookup file is provided as part of the tool.

Use the Configure Lookup Validation window to select the lookup file.

Select the Do not validate lookups option, if you do not want to validate the data.
Select the Use Lookup File to validate the data option and select the lookup.txt file to validate
the data.

7 | Data File Validation for File Based Loader

You can either choose to create the lookup file based on the format given below or you can use the SQL
query given below to run it against the Fusion database to generate the lookup.txt file from the Fusion
system.
Lookup file format:
<Lookup Type><tab><Lookup Code>
For example:
BLOOD_TYPE
BLOOD_TYPE
BLOOD_TYPE
BLOOD_TYPE
BLOOD_TYPE
BLOOD_TYPE
BLOOD_TYPE
BLOOD_TYPE
ADDRESS_TYPE
ADDRESS_TYPE
ADDRESS_TYPE
ADDRESS_TYPE
ADDRESS_TYPE
.

A+
AAB+
ABB+
BO+
O1
2
3
HOME
IAT

SQL to generate the Lookup file from Fusion :

SELECT

lookup_type
,lookup_code
FROM
fnd_lookup_values_vl
WHERE
lookup_type IN ('ADDRESS_TYPE','BARGAINING_UNIT_CODE','CMP_PROPOSAL_REASON'
,'BLOOD_TYPE','CMP_SALARY_BASIS','CONTACT'
,'CONTRACT_TYPE','EMAIL_TYPE','EMPLOYEE_CATG'
,'EMP_CAT','FREQUENCY','GRADE_PAY_RATE_TYPE'
,'HIRING_STATUS','HONORS','HOURLY_SALARIED_CODE'
,'HRT_CONTENT_SUPP_CODE','JOB_FUNCTION_CODE','MANAGER_LEVEL'
,'MARITAL_STATUS','MILITARY_RANK','NAME_TYPE'
,'ORG_TYPE','PAY_CREATOR_TYPE','PAY_ENTRY_TYPE'
,'PER_CITIZENSHIP_STATUS','PER_DUTIES_TYPE','PER_ETHNICITY'
,'PER_HIGHEST_EDUCATION_LEVEL','PER_LINKAGE_TYPE','PER_NATIONAL_IDENTIFIER_TYPE'
,'PER_PASSPORT_TYPE','PER_RELIGION','PER_SUPERVISOR_TYPE'
,'PER_VISA_PERMIT_STATUS','PER_VISA_PERMIT_TYPE','PHONE_TYPE'
,'PROBATION_PERIOD','QUALIFYING_UNITS','SECURITY_CLEARANCE'
,'SEX','SYSTEM_PERSON_TYPE','TITLE'
,'UNITS')
AND
enabled_flag = 'Y'
AND
sysdate BETWEEN start_date_active
AND
end_date_active
ORDER BY lookup_type
,lookup_code

Validation Error Threshold

If there are large volumes of data which need to be validated, then instead of the entire validation process
to complete to identify the errors, you can specify a threshold value to restrict the application to exit after
the validating the errors. The Data File Validator utility exits after reaching the threshold specified.
You can specify the maximum number of validation errors before the entire process is terminated
For example, if you specify 1000 as the validation error threshold value, then when the application reaches
1000 validation errors, the application exits after generating the validation report for the first 1000
validation errors.
8 | Data File Validation for File Based Loader

Using Data File Validator Utility

You can run the Data File Validator by specifying the following parameters:
Configuration File - Select the configuration file that contains the structure and validation
definitions. You can download the configuration file for your Fusion Release version from My
Oracle Support.
Data File / Zip File - Select the DAT file or zip file that you want to validate. You can choose to
provide the DAT file or the Zip file that contains multiple DAT files.
Optionally, select the Lookup file and specify the threshold values as described in the previous
section.
Click Validate to perform the validation. The validation HTML report file gets launched in a
browser.

9 | Data File Validation for File Based Loader

The HTML report of the utility displays the summary of each of the business objects being processed and
also provides a summary of the validation errors.
The output validation report is generated in the current folder. The report has two sections:
1. Detail section
2. Summary Section

You can view the validation errors highlighted in red color in the Details section as indicated below:

10 | Data File Validation for File Based Loader

You can view the consolidated list of validation errors in the Summary section.

Extending Data File Validator for Additional Validations

Users can extend the validation logic available in the Configuration file by creating a copy and adding
additional data validations to meet their business requirements. You can refer the Configuration section
for the details on the various validation options supported by Data File Validator utility.
Use the format as specified in Configuring Data File Validator section to add more validation.

11 | Data File Validation for File Based Loader

Appendix A XML Schema definition for the

Configuration File
<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:element name="fbl-configuration">
<xsd:complexType>
<xsd:sequence>
<xsd:element ref="zip-file-structure" minOccurs="1"
maxOccurs="1"/>
<xsd:element ref="business-entity" minOccurs="0"
maxOccurs="unbounded"/>
</xsd:sequence>
<xsd:attribute name="version" type="xsd:string" use="required"/>
</xsd:complexType>
</xsd:element>
<xsd:element name="zip-file-structure">
<xsd:complexType>
<xsd:sequence>
<xsd:choice minOccurs="0" maxOccurs="unbounded">
<xsd:element ref="folder" minOccurs="1" maxOccurs="1"/>
<xsd:element ref="data-file" minOccurs="1" maxOccurs="1"/>
</xsd:choice>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element name="folder">
<xsd:complexType>
<xsd:sequence>
<xsd:choice minOccurs="0" maxOccurs="unbounded">
<xsd:element ref="folder" minOccurs="1" maxOccurs="1"/>
<xsd:element ref="data-file" minOccurs="1" maxOccurs="1"/>
</xsd:choice>
</xsd:sequence>
<xsd:attribute name="name" type="xsd:string" use="required"/>
</xsd:complexType>
</xsd:element>
<xsd:element name="data-file">
<xsd:complexType>
<xsd:attribute name="name" type="xsd:string" use="required"/>
<xsd:attribute name="business-entity-ref" type="xsd:string"
use="required"/>
</xsd:complexType>
</xsd:element>
<xsd:element name="business-entity">
<xsd:complexType>
<xsd:sequence>
<xsd:element ref="attribute" minOccurs="0"
maxOccurs="unbounded"/>
<xsd:element ref="line-validations" minOccurs="0"
maxOccurs="1"/>
</xsd:sequence>
<xsd:attribute name="name" type="xsd:string" use="required"/>
</xsd:complexType>
</xsd:element>
<xsd:element name="attribute">
<xsd:complexType>
<xsd:sequence>
12 | Data File Validation for File Based Loader

<xsd:element ref="attribute-validations" minOccurs="0"

maxOccurs="1"/>
</xsd:sequence>
<xsd:attribute name="name" type="xsd:string" use="required"/>
<xsd:attribute name="data-type" type="DataType" use="required"/>
<xsd:attribute name="mandatory" type="xsd:boolean" default="false"/>
</xsd:complexType>
</xsd:element>
<xsd:element name="attribute-validations">
<xsd:complexType>
<xsd:sequence>
<xsd:choice minOccurs="0" maxOccurs="unbounded">
<xsd:element ref="range-validation" minOccurs="1"
maxOccurs="1"/>
<xsd:element ref="list-validation" minOccurs="1"
maxOccurs="1"/>
<xsd:element ref="regexp-validation" minOccurs="1"
maxOccurs="1"/>
<xsd:element ref="lookup-validation" minOccurs="1"
maxOccurs="1"/>
</xsd:choice>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element name="line-validations">
<xsd:complexType>
<xsd:sequence>
<xsd:choice minOccurs="0" maxOccurs="unbounded">
<xsd:element ref="compare-validation" minOccurs="1"
maxOccurs="1"/>
<xsd:element ref="groovy-validation" minOccurs="1"
maxOccurs="1"/>
</xsd:choice>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element name="range-validation">
<xsd:complexType>
<xsd:sequence/>
<xsd:attribute name="description" type="xsd:string" use="optional"/>
<xsd:attribute name="lower-limit" type="xsd:string" use="optional"/>
<xsd:attribute name="upper-limit" type="xsd:string" use="optional"/>
<xsd:attribute name="include-bounds" type="xsd:boolean"
use="optional"/>
<xsd:attribute name="disabled" type="xsd:boolean" use="optional"/>
</xsd:complexType>
</xsd:element>
<xsd:element name="lookup-validation">
<xsd:complexType>
<xsd:sequence/>
<xsd:attribute name="description" type="xsd:string" use="optional"/>
<xsd:attribute name="lookup-type" type="xsd:string" use="required"/>
<xsd:attribute name="disabled" type="xsd:boolean" use="optional"/>
</xsd:complexType>
</xsd:element>
<xsd:element name="regexp-validation">
<xsd:complexType>
<xsd:sequence/>
<xsd:attribute name="description" type="xsd:string" use="optional"/>
<xsd:attribute name="pattern" type="xsd:string" use="required"/>
<xsd:attribute name="disabled" type="xsd:boolean" use="optional"/>
13 | Data File Validation for File Based Loader

</xsd:complexType>
</xsd:element>
<xsd:element name="list-validation">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="list-value" type="xsd:string" minOccurs="1"
maxOccurs="unbounded"/>
</xsd:sequence>
<xsd:attribute name="description" type="xsd:string" use="optional"/>
<xsd:attribute name="disabled" type="xsd:boolean" use="optional"/>
</xsd:complexType>
</xsd:element>
<xsd:element name="compare-validation">
<xsd:complexType>
<xsd:sequence/>
<xsd:attribute name="description" type="xsd:string" use="optional"/>
<xsd:attribute name="attribute1" type="xsd:string" use="required"/>
<xsd:attribute name="attribute2" type="xsd:string" use="required"/>
<xsd:attribute name="condition" type="CompareCondition"
use="required"/>
<xsd:attribute name="disabled" type="xsd:boolean" use="optional"/>
</xsd:complexType>
</xsd:element>
<xsd:element name="groovy-validation">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="code" type="xsd:string" minOccurs="1"
maxOccurs="1"/>
</xsd:sequence>
<xsd:attribute name="description" type="xsd:string" use="required"/>
<xsd:attribute name="disabled" type="xsd:boolean" use="optional"/>
</xsd:complexType>
</xsd:element>
<xsd:simpleType name="CompareCondition">
<xsd:restriction base="xsd:string">
<xsd:enumeration value="GT"/>
<xsd:enumeration value="GE"/>
<xsd:enumeration value="LT"/>
<xsd:enumeration value="LE"/>
<xsd:enumeration value="EQ"/>
<xsd:enumeration value="NE"/>
</xsd:restriction>
</xsd:simpleType>
<xsd:simpleType name="DataType">
<xsd:restriction base="xsd:string">
<xsd:enumeration value="NUMERIC"/>
<xsd:enumeration value="DECIMAL"/>
<xsd:enumeration value="VARCHAR"/>
<xsd:enumeration value="DATE"/>
<xsd:enumeration value="GUID"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:schema>

14 | Data File Validation for File Based Loader

Data File Validation for File Based Loader

September 2013
Author: Vinod Viswanathan
Ullas Bhat
Swathi Mathur
Oracle Corporation
World Headquarters
500 Oracle Parkway
Redwood Shores, CA 94065
U.S.A.
Worldwide Inquiries:
Phone: +1.650.506.7000
Fax: +1.650.506.7200

Copyright 2013, Oracle and/or its affiliates. All rights reserved. This document is
provided for information purposes only and the contents hereof are subject to change
without notice. This document is not warranted to be error-free, nor subject to any other
warranties or conditions, whether expressed orally or implied in law, including implied
warranties and conditions of merchantability or fitness for a particular purpose. We
specifically disclaim any liability with respect to this document and no contractual
obligations are formed either directly or indirectly by this document. This document may
not be reproduced or transmitted in any form or by any means, electronic or mechanical,
for any purpose, without our prior written permission.
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names
may be trademarks of their respective owners.

oracle.com

15 | Data File Validation for File Based Loader