BI Migration - Daily Replica Process

1
 Katherine Louise A. Rogacion

July 6, 2016

Change Control

Author Date Version Comment

Katherine Louise A. Rogacion 6-Jul-2016 1.0 Initial Draft

28-Jul-2016 2.0 Updated for reduced daily replica. Only

GSDBA and PRODDBA schemas will be
replicated.

04-Aug-2016 3.0 Updated the list of files

2
Overview

The purpose of this document is to outline the steps for the daily GR replica process.

3
Daily GR Replica Process

1. The export runs at 6pm CEST (server time) on TRSEKA180 (10.42.8.180) server.

# Daily schema replica export

#00 11 * * * /home/oracle/scripts/export_gr_tables.sh > /home/oracle/scripts/export_gr_tables.log 2>&1

00 18 * * * /home/oracle/scripts/export_gr_tables_reduced.sh > /home/oracle/scripts/export_gr_tables_reduced.log 2>&1

Note: The job usually takes approximately 1.5 hours to run. To check the export process, run the command below on the
Linux command prompt.

$ ps -ef | grep expdp

oracle 30597 17636 0 15:13 ? 00:00:00 expdp parfile=expdp_gsdba_datacms2.par

oracle 2905 1767 0 15:39 pts/1 00:00:00 grep expdp

If it’s finished, you will get an email notification with the subject “Export of TRCOTXKA GR tables is now finished.”
When you do a ps -ef | grep expdp, you will not see a process corresponding to expdp. Example is shown below.
4
$ ps -ef | grep expdp

oracle 26998 1528 0 18:11 pts/0 00:00:00 grep expdp

There are 22 total dump files that are produced on /u01/export directory on TRSEKA180 (10.42.8.180) server. These
are:

-rw-r----- 1 oracle oinstall 131420160 Aug 3 18:19 trcotxka_gsdba_partitioned_tables_03.dmp.gz ok ok ok

-rw-r----- 1 oracle oinstall 2737184768 Aug 3 18:26 trcotxka_gsdba_partitioned_tables_02.dmp.gz ok ok ok

-rw-r----- 1 oracle oinstall 3884474368 Aug 3 18:36 trcotxka_gsdba_partitioned_tables_01.dmp.gz ok ok ok

-rw-r----- 1 oracle oinstall 756375552 Aug 3 19:09 trcotxka_gsdba_partitioned_tables_datacms2_02.dmp.gz

-rw-r----- 1 oracle oinstall 778190848 Aug 3 19:12 trcotxka_gsdba_partitioned_tables_datacms2_03.dmp.gz

-rw-r----- 1 oracle oinstall 265916416 Aug 3 19:15 trcotxka_gsdba_partitioned_tables_datacms2_01.dmp.gz

-rw-r----- 1 oracle oinstall 265334784 Aug 3 19:18 GR_Tables_2016_GSDBA_02.dmp.gz ok

-rw-r----- 1 oracle oinstall 463249408 Aug 3 19:21 GR_Tables_2016_GSDBA_01.dmp.gz ok

-rw-r----- 1 oracle oinstall 22134784 Aug 3 19:25 GR_Tables_2016_GSDBA_part2_02.dmp.gz ok

-rw-r----- 1 oracle oinstall 61390848 Aug 3 19:25 GR_Tables_2016_GSDBA_part2_01.dmp.gz ok

5
-rw-r----- 1 oracle oinstall 163840 Aug 3 19:25 GR_Tables_2016_GSDBA_wrkdatacallbacks_02.dmp.gz ok

-rw-r----- 1 oracle oinstall 1404928 Aug 3 19:26 GR_Tables_2016_GSDBA_wrkdatacallbacks_01.dmp.gz ok

-rw-r----- 1 oracle oinstall 4583424 Aug 3 19:26 trcotxka_gr_gsdba_tables_1.dmp.gz ok

-rw-r----- 1 oracle oinstall 679936 Aug 3 19:26 trcotxka_gr_gsdba_tables_2.dmp.gz ok

-rw-r----- 1 oracle oinstall 1871872 Aug 3 19:26 trcotxka_gr_gsdba_tables_3.dmp.gz ok

-rw-r----- 1 oracle oinstall 106496 Aug 3 19:28 trcotxka_gr_gsdba_tables_4_02.dmp.gz ok

-rw-r----- 1 oracle oinstall 22642688 Aug 3 19:28 trcotxka_gr_gsdba_tables_4_01.dmp.gz ok

-rw-r----- 1 oracle oinstall 318418944 Aug 3 19:30 trcotxka_gr_gsdba_tables_5_02.dmp.gz ok

-rw-r----- 1 oracle oinstall 1762783232 Aug 3 19:33 trcotxka_gr_gsdba_tables_5_01.dmp.gz ok

-rw-r----- 1 oracle oinstall 606208 Aug 3 19:33 trcotxka_gr_proddba_tables_1.dmp.gz ok

-rw-r----- 1 oracle oinstall 356352 Aug 3 19:34 trcotxka_gr_proddba_tables_2.dmp.gz ok

-rw-r----- 1 oracle oinstall 24797184 Aug 3 19:34 trcotxka_gr_proddba_tables_3.dmp.gz ok

oracle@10.240.70.19:/oracle/backup/daily_replica

c#d5zV

2. Compress the export data pump files that are located on /u01/export directory on TRSEKA180 (10.42.8.180) server. The
command to use is gzip.

6
Example:

$ gzip trcotxka_gsdba_partitioned_tables_03.dmp

3. Delete the export dump files on /oracle/backup/daily_replica in TCSVDB101 (10.240.70.19) server from the previous
day’s import. The command to delete is rm.

ls -lrt *.dmp

Example:

rm *.dmp --> This deletes all export dump files. The * wildcard is used to signify that all export dump files will
be deleted.

4. Transfer the export dump files from TRSEKA180 (10.42.8.180) to TCSVDB101 (10.240.70.19). Use the scp command to transfer
it.

Example:

7
scp trcotxka_gsdba_partitioned_tables_03.dmp.gz oracle@10.240.70.19:/oracleg/backup/daily_replica

oracle@10.240.70.19's password:

trcotxka_gsdba_partitioned_tables_03.dmp.gz 100% 427MB 1.1MB/s 06:31

Note: The oracle password in TCSVDB101 (10.240.70.19) server is c#d5zV.

5. When you have transferred the export dump file to TCSVDB101 (10.240.70.19), de-compress the files that are compressed.

Example:

[oracle@tcsvdb101 daily_replica]$ gunzip trcotxka_gr_gsdba_tables_02.dmp.gz

6. Before importing the tables to BOBJTEST database, you will have to drop the tables and the sequences. The script is
located in /home/oracle/scripts directory.

a. Run the script drop_proddba_gsdba_before_import.sh to drop the selected tables that will be imported.

8
 Example: nohup ./drop_proddba_gsdba_before_import.sh & --> This will run the script as a background process.

b. Run the script drop_sequences_bobjtest.sh to drop the sequences.

Example: ./drop_sequences_proddba_gsdba_bobjtest.sh

Note: Please review the log files that will be generated.

7. Import the schemas by running the scripts manually. The import scripts are located in /home/oracle/scripts directory
in TCSVDB101 (10.240.70.19) server.

-rwxr-xr-x 1 oracle oinstall 546 Jul 4 17:00 import_gsdba_partitioned_tables.sh

-rwxr-xr-x 1 oracle oinstall 553 Jul 4 17:06 import_gsdba_data_cms2.sh

-rwxr-x--- 1 oracle oinstall 1015 Jul 27 15:58 import_proddba_gsdba.sh

The reason why we are running the imports separately is due to the fact that Oracle uses the undo space in the undo
tablespace when running imports. The scripts have been broken down to different schemas to import one schema at a time
with the bigger tables and to keep the undo tablespace manageable as well.

9
 Example: nohup ./import_gsdba_partitioned_tables.sh & --> Run it as a background process.

Notes:

a. Delete the last nohup.out by running this command - rm nohup.out.

b. To see the progress of the import, enter the command below.

tail -f nohup.out --> This will automatically print the import progress on your screen on your putty session.

Example output of tail -f nohup.out

===================================

[oracle@tcsvdb101 scripts]$ tail -f nohup.out

Starting import at 07-06-16

Importing GR tables

Import: Release 11.2.0.1.0 - Production on Wed Jul 6 19:09:41 2016

10
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.

Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production

With the Partitioning option

Master table "SYSTEM"."SYS_IMPORT_SCHEMA_20" successfully loaded/unloaded

Starting "SYSTEM"."SYS_IMPORT_SCHEMA_20": system/******** directory=DAILY_REPLICA

dumpfile=trcotxka_gsdba_partitioned_tables_datacms2_%U.dmp parallel=3 SCHEMAS=GSDBA table_exists_action=APPEND
logfile=impdp_trcotxka_gsdba_datacms2_07-06-16.log

Processing object type SCHEMA_EXPORT/SEQUENCE/SEQUENCE

Processing object type SCHEMA_EXPORT/SEQUENCE/GRANT/OWNER_GRANT/OBJECT_GRANT

Processing object type SCHEMA_EXPORT/TABLE/TABLE

Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA

. . imported "GSDBA"."DATA_CMS2":"DATA_CMS2_2016_Q1" 995.5 MB 18719226 rows

c. You can ignore the errors on the import log file.

d. Another way to see the progress of the import is to enter the command below.

11
 ps -ef | grep impdp

[oracle@tcsvdb101 2016_07_06]$ ps -ef | grep impdp

oracle 2101 2099 0 17:01 pts/0 00:00:01 impdp directory=DAILY_REPLICA

dumpfile=trcotxka_TDC_ADM_partitioned_tables_%U.dmp parallel=3 schemas=TDC_ADM table_exists_action=APPEND
logfile=impdp_trcotxka_TDC_ADM_parttables_07-06-16.log

oracle 4231 1481 0 18:51 pts/2 00:00:00 grep impdp

e. The longest import is on the import of the schema TDC_ADM.

8. While import is running, it is important to monitor the flash recovery area on the server because the import process
generates archived redo log files. The flash recovery area for BOBJTEST database is located on
/backup/flash_recovery_area/BOBJTEST/BOBJTEST/archivelog. Just select the appropriate date so you can see where the
archived redo logs are being generated.

[oracle@tcsvdb101 archivelog]$ pwd

/backup/flash_recovery_area/BOBJTEST/BOBJTEST/archivelog

[oracle@tcsvdb101 archivelog]$ ls -lrt

total 8
12
drwxrwx--- 2 oracle oinstall 4096 Jul 6 16:19 2016_07_05

drwxrwx--- 2 oracle oinstall 4096 Jul 6 19:07 2016_07_06

[oracle@tcsvdb101 2016_07_06]$ pwd

/backup/flash_recovery_area/BOBJTEST/BOBJTEST/archivelog/2016_07_06

[oracle@tcsvdb101 2016_07_06]$

[oracle@tcsvdb101 2016_07_06]$ ls -lrt

total 2806140

-rw-rw---- 1 oracle oinstall 453736448 Jul 6 19:08 o1_mf_1_5063_cqtgqdmy_.arc

-rw-rw---- 1 oracle oinstall 512402944 Jul 6 19:11 o1_mf_1_5064_cqtgxgon_.arc

-rw-rw---- 1 oracle oinstall 512400896 Jul 6 19:11 o1_mf_1_5065_cqtgxvjr_.arc

-rw-rw---- 1 oracle oinstall 520711680 Jul 6 19:12 o1_mf_1_5066_cqtgyxfq_.arc

-rw-rw---- 1 oracle oinstall 467357696 Jul 6 19:12 o1_mf_1_5067_cqtgz9qw_.arc

-rw-rw---- 1 oracle oinstall 497292288 Jul 6 19:12 o1_mf_1_5068_cqth0p1o_.arc

To free up space, simply delete the .arc files under the appropriate directory. Example is
/backup/flash_recovery_area/BOBJTEST/BOBJTEST/archivelog/2016_07_06.

13
Steps to free up space on the flash recovery area:

a. cd /backup/flash_recovery_area/BOBJTEST/BOBJTEST/archivelog/2016_07_06 Note: Go to the directory of the current date.

b. rm *

c. export ORACLE_SID=BOBJTEST

d. rman target /

e. RMAN> crosscheck archivelog all;

RMAN> crosscheck archivelog all;

using target database control file instead of recovery catalog

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=83 device type=DISK

validation failed for archived log

archived log file name=/backup/flash_recovery_area/BOBJTEST/BOBJTEST/archivelog/2016_07_06/o1_mf_1_5063_cqtgqdmy_.arc

RECID=3314 STAMP=916513682

14
validation failed for archived log

Crosschecked 10 objects

f. RMAN> list expired archivelog all; --> This will show all expired archivelog files.

g. RMAN> delete expired archivelog all; --> This will list all expired archivelog files and prompt you with this question.
Do you really want to delete the above objects (enter YES or NO)? Simply type YES.

h. To verify that you have freed up space and reduced the space usage on the directory, login to SQL *Plus as the sysdba
user and run the query below.

select

name,

floor(space_limit / 1024 / 1024) "Size MB",

ceil(space_used / 1024 / 1024) "Used MB"

from v$recovery_file_dest;
15
9. When all the import scripts have been and run and import is completed, you will need to re-create the sequences on
BOBJTEST database.

a. Login to TRSEKA180 (10.42.8.180) server.

b. Ensure the ORACLE_SID is set to TRCOTXKA database by running the command echo $ORACLE_SID.

c. Connect to TRCOTXKA database on SQL *Plus as the sysdba user - sqlplus / as sysdba. You can also do this on SQL Developer.
Connect to TRCOTXKA using system user. The password is easyjet.

d. Run the query below. The results will be used to create the sequences.

select 'Create Sequence ' || sequence_owner || '.' ||sequence_name||

' increment by '||increment_by||

' start with '||last_number||

' maxvalue '||max_value||

decode(cycle_flag,'N',' NOCYCLE ',' CYCLE ')||

decode(cache_size,0,'NOCACHE ','CACHE '||cache_size) || ';'

from dba_sequences

where sequence_owner in ('GSDBA','PRODDBA')

16
order by sequence_owner, sequence_name;

e. Copy the results to a temporary file on your laptop. (I use notepad for this.)

f. On TCSVDB101 (10.240.70.19) - Copy the contents of the script to

/home/oracle/kaye/migration/create_sequence_from_trcotxka.sql. (I usually delete the file, then re-create it with the
same file name because the script calls this sql file.)

g. Delete the file drop_sequences_bobjtest_proddba_gsdba.log under /home/oracle/scripts directory.

h. Run the script drop_sequences_bobjtest.sh. The file is located in /home/oracle/scripts directory.

./drop_sequences_proddba_gsdba_bobjtest.sh

i. Delete the file create_sequences_bobjtest.log under /home/oracle/scripts directory.

j. Run the script create_sequences_bobjtest.sh. The file is located in /home/oracle/scripts directory.

10. Connect to BOBJTEST database as the sysdba user. Run the query below.

select count(*) from dba_objects where status = ‘INVALID’;

If the query returns a value of 660, then we’ll need to recompile the invalid objects by running the script utlrp.sql
Please see below.

SQL> @$ORACLE_HOME/rdbms/admin/utlrp.sql --> This takes a while to run.

17
 After recompiling, run the query below. It should return 146 invalid objects which should be fine.

select count(*) from dba_objects where status = ‘INVALID’;

11. Verify the number of tables imported to BOBJTEST database. The queries below should be run after the whole process.

select * from dba_objects

where owner = 'GSDBA'

and object_type = 'TABLE'

and object_name in

('APPROVED_TL','BOARTICLE','BOARTICLE_HISTORY','BO_HISTORY','CT_HISTORY','DATA_BO','DATA_CALLBACKS','DATA_CMS1','DAT
A_CMS1_ADJ','DATA_MANUAL','DATA_TCMS','DC_HISTORY','IMPORTLOG','PC_HISTORY','STAT','USC_HISTORY','USC_HISTORY_TCMS',
'BO_HISTORY','CA_HISTORY','CONSULTANT_TIME','DATA_ICC','DATA_TCA','DATA_TCMS_ADJ','DATA_TCS','DATA_TCS_ABSENCE','DAT
A_TCS_OUTBOUND','DATA_UPSELL','DC_HISTORY','EMPLOYEE_ABSENCE','FAILED_DATA_CALLBACKS','FTA_AGENT','FTA_DS_PRICE_HIST
ORY','FTA_FG_DATA','FTA_FORECAST_GROUP_HISTORY','FTA_SAVED','FTA_SG_DATA','FTA_SHRINKAGE','FTA_STAFF_GROUP_HISTORY',
'GRETL_DAILY_EXPECTATION','MTB_HISTORY','RW_CAE_HISTORY','SOCIAL_STAT','WRK_DATA_CALLBACKS','ABSENCE_CODES','ABSENCE
_CODES_TCS','ABSENCE_TYPES','BACKOFFICE','BILLING_TYPE','BO_ACTIONOUTCOME','BO_CATEGORY','BO_CHANNEL','BO_CLIENTPROD
UCT','BO_CLOSURE','BO_COMPLEXITY','BOBJ_MNG_USERS','BOBJ_RS_BUILT','BOBJ_RS_COUNTRY','BOBJ_RS_NAME','BOBJ_RS_REGION'
,'BOBJ_RS_SITE','BOBJ_TARG_ADMIN','BOBJ_TARG_LEVEL_NAMES','BOBJ_TARG_LEVELS','BOBJ_TARG_STATUS','BOBJ_TARG_TYPES',
18
'BOBJ_TARG_UNITS','BOBJ_TARG_VALUES','CALLBACK_ACCOUNTS','CONSULTANT_TASK','DURATION_CONTRACTS','FTA_DS_PRICE','FTA_
FORECAST_GROUP','FTA_STAFF_GROUP','GRETL_IMPORT_CONFIG','GRETL_LOG4NET','MAILBOX_TASK_DURATION','MAILBOXES',

'SOCIAL_EMPLOYEEUSERS','SOCIAL_FILEADMINISTRATORS','SOCIAL_SMADMINISTRATORS','TASKS','UPSELL_CATEGORY','UPSELL_CODES
','UPSELL_CODES_TCMS','USE_TCS')

order by object_name;

should return 83 tables

select * from dba_objects

where owner = 'PRODDBA'

and object_type = 'TABLE'

and object_name in

('COSTPRO_PROFILE','COSTPRO_RANGE','CURRENCY','EXT_COMPANY','TC_CONTRACT_TYPE','TC_COUNTRIES','TC_DEPARTMENTS',

'TC_EMPLOYEES','TC_GROUPS','TC_HRMS_EMPLOYEES','TC_HRMS_RELATIONSHIP','TC_REGIONS','TC_ROLES','TC_SITES','TC_VCF','V
_TC_EMPLOYEES');

should return 15 tables

19
12. Delete all the export dump files and log files that are dated for the current day in /u01/export directory in TRSEKA180
(10.42.8.180) server. This is in preparation for the next daily run of the export job.

cd /u01/export

ls -lrt

-rw-r--r-- 1 oracle oinstall 1859 Jul 6 15:58 GR_Tables_2016_TDCADM.log

-rw-r----- 1 oracle oinstall 2338816 Jul 6 15:58 GR_Tables_2016_TDCADM.dmp

-rw-r--r-- 1 oracle oinstall 6099 Jul 6 16:00 GR_Tables_2016_TCMS3DBA.log

-rw-r--r-- 1 oracle oinstall 16593 Jul 6 16:17 GR_Tables_2016_GSDBA.log

-rw-r----- 1 oracle oinstall 76867195 Jul 6 16:17 GR_Tables_2016_GSDBA_01.dmp.gz

-rw-r--r-- 1 oracle oinstall 5337 Jul 6 16:28 trcotxka_gr_gsdba_tables.log

-rw-r----- 1 oracle oinstall 47756766 Jul 6 16:28 trcotxka_gr_gsdba_tables_02.dmp.gz

-rw-r----- 1 oracle oinstall 211794686 Jul 6 16:28 trcotxka_gr_gsdba_tables_01.dmp.gz

-rw-r--r-- 1 oracle oinstall 2883 Jul 6 16:28 trcotxka_gr_proddba_tables.log

-rw-r----- 1 oracle oinstall 237568 Jul 6 16:28 trcotxka_gr_proddba_tables_02.dmp

-rw-r----- 1 oracle oinstall 7572501 Jul 6 16:28 trcotxka_gr_proddba_tables_01.dmp.gz

20
-rw-r--r-- 1 oracle oinstall 1532 Jul 6 16:29 trcotxka_gr_tcadba_tables.log

-rw-r----- 1 oracle oinstall 290816 Jul 6 16:29 trcotxka_gr_tcadba_tables.dmp

-rw-r--r-- 1 oracle oinstall 1988 Jul 6 16:29 trcotxka_gr_tcms3dba_tables.log

-rw-r----- 1 oracle oinstall 557056 Jul 6 16:29 trcotxka_gr_tcms3dba_tables_02.dmp

-rw-r----- 1 oracle oinstall 28632913 Jul 6 16:29 trcotxka_gr_tcms3dba_tables_01.dmp.gz

-rw-r--r-- 1 oracle oinstall 3264 Jul 6 16:29 trcotxka_gr_tdcadm_tables.log

-rw-r----- 1 oracle oinstall 274432 Jul 6 16:29 trcotxka_gr_tdcadm_tables_02.dmp

-rw-r----- 1 oracle oinstall 5219909 Jul 6 16:29 trcotxka_gr_tdcadm_tables_01.dmp.gz

-rw-r--r-- 1 oracle oinstall 1871 Jul 6 16:30 trcotxka_gr_tmdba_tables.log

-rw-r----- 1 oracle oinstall 77824 Jul 6 16:30 trcotxka_gr_tmdba_tables_02.dmp

-rw-r----- 1 oracle oinstall 5283840 Jul 6 16:30 trcotxka_gr_tmdba_tables_01.dmp

-rw-r--r-- 1 oracle oinstall 1539 Jul 6 16:30 GR_Tables_2016_GSDBA_wrkdatacallbacks.log

-rw-r----- 1 oracle oinstall 163840 Jul 6 16:30 GR_Tables_2016_GSDBA_wrkdatacallbacks_02.dmp

-rw-r----- 1 oracle oinstall 1413120 Jul 6 16:30 GR_Tables_2016_GSDBA_wrkdatacallbacks_01.dmp

13. Send an email notification that the daily replica is complete.

21
22