This document provides a summary of the Marathon Oil Company's Autosys Run Book which outlines the policies, processes, and procedures for supporting their Autosys R11 infrastructure. It includes information on the project infrastructure architecture, service level agreements, alert and incident management guidelines, roles and responsibilities, and event management guidelines. The document is intended to be used by the Cognizant Autosys R11 Support Team and provides an overview of the automation project and expectations for service delivery.
This document provides a summary of the Marathon Oil Company's Autosys Run Book which outlines the policies, processes, and procedures for supporting their Autosys R11 infrastructure. It includes information on the project infrastructure architecture, service level agreements, alert and incident management guidelines, roles and responsibilities, and event management guidelines. The document is intended to be used by the Cognizant Autosys R11 Support Team and provides an overview of the automation project and expectations for service delivery.
This document provides a summary of the Marathon Oil Company's Autosys Run Book which outlines the policies, processes, and procedures for supporting their Autosys R11 infrastructure. It includes information on the project infrastructure architecture, service level agreements, alert and incident management guidelines, roles and responsibilities, and event management guidelines. The document is intended to be used by the Cognizant Autosys R11 Support Team and provides an overview of the automation project and expectations for service delivery.
Marathon Oil Company Autosys R11 Infra Support Run book ersion 1!" Prepared #y $ %ast &pdated #y Re'ie(ed #y Appro'ed #y )ame Rajesh Sree&e+i Patil %ishan , S Role ,ea- Me-ber CO. DM Si*nature Rajesh Sree&e+i %ishan +ate ()(/)('0 /1)(') 0(0)('2 Controlled Copy Marathon Oil Autosys Run book Project ID: <'((((/0))/> <SCI.ID. > !er: '.(> Release ID: C3: Protected Pa3e ) o4 20 ,able of Contents "!" Introduction!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! - "!1 Ser'ice Operations .uide Scope!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! - "!/ #rief o'er'ie( of the pro0ect !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! - 1!" Infrastructure Architecture:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 1 /!" Ser'ice %e'el A*reements!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 2 3!" Alert Mana*ement .uidelines !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 3 -!" Introduction!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 1" -!1 Purpose!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 1" -!/ Scope!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 1" -!3 +efinitions and Acronyms!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 11 -!- S%A!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 11 -!4 Roles and Responsibilities !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 11 4!" 5'ent Mana*ement .uidelines!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 1- 3!/ Ser'ice Re6uest .uidelines !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 12 1!" Introduction!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 12 1!1 Purpose!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 12 1!/ Scope!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 12 1!3 +efinitions and Acronyms!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 12 2!" ,asks!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 17 2!1 Ser'ice re6uest !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 17 2!/ S%A!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 13 2!3 8orkin* on Ser'ice Re6uests!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! /1 3!3 Incident Mana*ement .uidelines!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! // -!" 5scalation Matri9!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! // -!1 Customer 5scalation Process!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! /3 7!" Capacity Mana*ement!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! /3 https:$$ch1(orkspaces$sites$S81"7$MOC:Autosys:Process:+ocuments$Shared ;/"+ocuments$MOC$<o(:to:*et:*raphs:from:Scom!doc9!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!/- 1!" #usiness Continuity Plan ,his document pro'ides the information about the Co*ni=ant>s plan for the +isaster Reco'ery ?+R@ to ensure the #usiness Continuity : !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!/- https:$$ch1(orkspaces$sites$S81"7$MOC:Autosys:Process:+ocuments$Shared ;/"+ocuments$MOC$MOC:1""""1-//1:#CP+ocument!pdf !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!/- 7!" Ano(led*e Mana*ement!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! /- 3!" Ma0or Incidents!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! /7 1"!" RCA!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! /3 Controlled Copy Marathon Oil Autosys Run book Project ID: <'((((/0))/> <SCI.ID. > !er: '.(> Release ID: C3: Protected Pa3e 2 o4 20 11!" Automation!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 3" 1/!" Roles and Responsibilities!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 31 13!" Appendi9!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 33 1-!" Chan*e %o*!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 33 Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e 0 o4 20 "!" Introduction ,his &ocu-ent contains Ser+ice O6erations 6olicies7 6rocesses an& 6roce&ures re8uire& to su66ort Marathon Oil Co-6any9s AutoSys R''.2 en+iron-ents. ,his Ser+ice O6erations :ui&e contains in4or-ation 6ertainin3 to the architecture7 an& s6eci4ications 4or the R''.2 AutoSys en+iron-ent alon3 ;ith ser+ice su66ort an& ser+ice &eli+ery acti+ities re8uire& to -aintain the en+iron-ent. ,his 3ui&e is inten&e& 4or use by the Co3ni<ant AutoSys R''.2 Su66ort ,ea- an& contains tea- role &e4initions7 res6onsibilities7 -eetin3 an& re6ortin3 re8uire-ents7 inci&ent -ana3e-ent 6roce&ures7 an& escalation 6rocesses. ,his &ocu-ent contains Ser+ice O6erations 6olicies7 6rocesses an& 6roce&ures re8uire& to su66ort the Co3ni<ant I, IS Mana3e& Ser+ices 6ort4olio. ,his &ocu-ent &escribes ho; I, IS Ser+ice Deli+ery -ana3es Batch =ob Monitorin3 an& job sche&ulin3 usin3 Autosys R'' 4or Marathon Oil Co-6any on an O6erational basis. 0.1 Ser'ice Operations .uide Scope "!1!1 8hat this document co'ers ,he 6ur6ose o4 the &ocu-ent is to 3i+e a brie4 o+er+ie; an& sco6e o4 the 6roject. ,his &ocu-ent also &iscusses the +arious 6lans ;hich are critical 4or the o6eration o4 the Marathon Autosys In4rastructure Su66ort 6roject7 an& the 6rocesses ;hich ha+e to be 4ollo;e& 4or the s-ooth 4unctionin3 o4 the 6roject. "!1!/ 8hat this document does not co'er ,his &ocu-ent &oes not atte-6t to instruct a user concernin3 the technolo3y co+ere&7 tools trainin3 6olicies or -onitorin3 syste-s trainin3. "!1!3 &ser 59pectations It is e>6ecte& that this &ocu-ent ;ill be use& by the Co3ni<ant Ser+ice Deli+ery tea- su66ortin3 Marathon9s AutoSys R''.2 en+iron-ents. ,his &ocu-ent &oes not co+er any s6eci4ic internal Co3ni<ant a&-inistrati+e acti+ities or issues. 0.2 #rief o'er'ie( of the pro0ect Marathon Oil Cor6oration is an inte3rate& international ener3y co-6any en3a3e& in e>6loration an& 6ro&uction? oil san&s -inin3? -arketin3 an& trans6ortation. O6eratin3 across the 3lobe 7 Marathon is a-on3 the ;orl&@s lea&in3 inte3rate& ener3y co-6anies # a66lyin3 inno+ation technolo3ies to &isco+er an& &e+elo6 +aluable ener3y resources7 6ro+i&in3 hi3h#8uality 6ro&ucts to the -arket6lace an& &eli+erin3 +alue to all o4 the co-6any@s stakehol&ers. Marathon Oil Co-6any re8ueste& Co3ni<ant to u63ra&e the Autosys in4rastructure 4ro- Autosys 0.A to Autosys R''. ,he u63ra&e 6roject ;as co-6lete& success4ully by Co3ni<ant. Co3ni<ant is 6ro+i&in3 )0>B Su66ort 4ro- Mi& Dece-ber )(''7 ;hich is been 6er4or-e& by Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e A o4 20 I, IS. Co3ni<ant -ana3es their Autosys in4rastructure7 ser+ice o6erations an& 4ocuses on o6erations i-6ro+e-ents. Cor this the Client re8uires an onshore an& o44shore tea- o4 Autosys A&-inistrators in Autosys technolo3y 4ro- Co3ni<ant. Autosys Syste- A&-inistrators ;ill 6ossess stron3 technical skills7 or3ani<ational in4luence an& 3oo& ;ritten an& +erbal co--unications skills to e44ecti+ely co--unicate across all le+els o4 sta44 an& users. DisDer job res6onsibilities ;ill inclu&e all o4 the 4ollo;in3# Autosys R'' in4rastructure a&-inistration Cirst 6oint o4 contact 4or all o6erational 6roble-s in+ol+in3 a66lication in4rastructure Ee 6ro+i&e Fe+el '7 Fe+el ) an& Fe+el2 su66ort 4or Autosys .nsure 6roble-s are -ana3e&resol+e& cra&le#to#3ra+e 6ro+i&in3 6reli-inary an& 4inal RCA ;ithin a3ree& SFA Pro+i&e a )0>B e44icient -onitorin3 an& su66ort to the Autosys R'' u63ra&e& en+iron-ent to kee6 the Autosys ser+ices on the 6ro&uction7 &e+elo6-ent an& 5A en+iron-ent 4ully 4unctional. ,he Co3ni<ant ,ea- -e-bers ;ill be sel4#-oti+ate&7 ;ith a han&s#on orientation an& ;ill ha+e the ability to 4unction both in&e6en&ently as a -e-ber o4 a tea-GsH or lea&er o4 a tea-. ,he can&i&ate ;ill ;ork un&er the &irect su6er+ision o4 the Project -ana3er. "!/!1 Scope of the pro0ect Sco6e o4 this 6roject is Autosys R'' a&-inistration7 roun& the clock e>ecute& 4ro- Co3ni<ant o44ice 6re-ises )0B. ,his 6roject su66orts the 4ollo;in3 en+iron-ents :
'. D.! GDe+elo6-ent en+iron-entH ). ,.S, G,est en+iron-entH 2. PROD GPro&uction en+iron-entH ,he 4ollo;in3 are consi&ere& to be the -ajor ser+ices 6ro+i&e& by I,IS Autosys In4rastructure su66ort tea-. A66lication -aintenance by 6er4or-in3 &e6loy-ents7 installations7 u63ra&es an& 6atches -ana3e-ent. Scri6tin3 an& auto-ation 4or any -anual a&-inistration relate& tasks Installation an& con4i3uration o4 Autosys co-6onents an& ser+ices Autosys cluster an& ;ork loa& -ana3e-ent Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e / o4 20 Creatin3 the %no;le&3e Re6ository ha+in3 all &ocu-ents u6 to &ate Docu-entin3 Stan&ar& O6eration Proce&ures GSOPH O6ti-i<e& all their -onitorin3 toolsGO6ti-i<e 4or in4rastructure7 ECCH I,IF Co-6liant su66ort -o&el an& -etrics base& -ana3e-ent ,his 6roject su66orts a )0>B -onitorin3 an& a&-inistration su66ort7 ;hich co-6rise o4 )0 Drs su66ort 4ro- O44shore startin3 4ro- (/:2( AM IS, to ne>t -ornin3 (/:2( AM IS, all B &ays a ;eek. Ee ha+e * Drs Su66ort 4ro- Onsite ,ea- ;hich ;orks startin3 4ro- 1.(( AM CS, to A.(( PM CS, . 1!" Infrastructure Architecture:
Sl! )o! Ser'ices in Ser'ice Catalo* Pro0ect Ser'ices ?as in SO8@ ariation from the standard ser'ices ' Autosys R'' A&-inistration an& su66ort Fe+el'7 Fe+el)7 an& Fe+el2 su66ort 4or Autosys R'' co-6onent Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e B o4 20 /!" Ser'ice %e'el A*reements Measure S%A S%A Operational Penalt Reportin* Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e * o4 20 Compliance ,ar*et +efinition y Clause ?if any@ Bre6uency Inci&ents Res6onse ,i-e P'<I 2( -in P)<I /( -in P2 <I * hrs P0 <I '/ hrs P'>I 1*.AJ P)>I 1*.AJ P2>I1*.AJ P0>I1*.AJ ,he ti-e taken to ackno;le&3e or res6on& the recei6t o4 a ticket yes Monthly Inci&ents Resolution ,i-e P'<I 0 hrs P)<I * hrs P2 <I B) hrs P0 <I B) hrs P'>I 1*.AJ P)>I 1*.AJ P2>I1*.AJ P0>I1*.AJ ,he ti-e taken to resol+e the ticket. ,his e>clu&es the Marathon Oil ;ait ti-e K inclu&es the res6onse ti-e yes Monthly Ser+ice Re8uest Res6onse ,i-e P'<I ' hr P)<I ) hrs P2 <I )0 hrs P0 <I )0 hrs P'>I 1/.AJ P)>I 1/.AJ P2>I1/.AJ P0>I1/.AJ ,he ti-e taken to ackno;le&3e or res6on& the recei6t o4 a ticket yes Monthly Ser+ice Re8uest Resolution ,i-e P'<I * hrs P)<I '/ hrs P2 <I 2) hrs P0 <I 0* hrs P'>I1A.'(J P)>I1A.'(J P2>I1A.'(J P0>I1A.'(J ,he ti-e taken to resol+e the ticket. ,his e>clu&es the Marathon Oil ;ait ti-e K inclu&es the res6onse ti-e yes Monthly A+ailability Autosys a+ailability >I1*.)2J Syste- A+ailability as 6er a3ree& ser+ice hours inclu&in3 a3ree& &o;n ti-e yes Monthly Ca6acity Mana3e-ent CP$ $sa3e J Dar& 6a3e 4aults Bytes o4 6hysical -e-ory <I B(J <IA 6er sec 4or ) hrs or -ore <I*(J Ser+er Ca6acity Mana3e-ent Ser+ice Fe+el as the nu-ber o4 instances7 across all Autosys Ser+ers yes Monthly Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e 1 o4 20 Co--itte& bytes o4 -e-ory &oes not e>cee& the total 6hysical -e-ory J o4 &isk ti-e busy Disk 8ueue len3th CP$ 8ueue len3th J o4 CP$ $sa3e Cor ' hr or -ore <I1AJ 4or ) hrs or -ore <I) 4or ) hrs or -ore <I0 4or ) hrs or -ore >I0(J 4or entire -onth Marathon Oil Satis4action $ser .>ecuti+e >I 0.( >I0.( Calculate& as the a+era3e o4 all nu-erical ratin3s recei+e& 4ro- sur+ey 6artici6ants &urin3 the Measure-ent Perio& yes Monthly Process co-6letion on ti-e L AutoSys R'' =ob Cailure Res6onse <I)( -in 1A.)A Res6on&s by ackno;le&3in3 the job 4ailure alertalar- in the Autosys O6ti-i<e 6ro&uct yes Monthly 3!" Alert Mana*ement .uidelines Collo;in3 &ocu-ent 6ro+i&es instructions re3ar&in3 &ili3ence -onitorin37 han&lin3 job alerts an& Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e '( o4 20 6roce&ure to han&le those alerts: 5'ent Mana*ement Procedure ?MOC@ -!" Introduction 4.1 Purpose Autosys team deals with the effective monitoring of the Production environment. Workload Control Center (WCC) generates and publishes all the alerts in Production and currently manual monitoring is happening. It is the first Point of contact for all the alerts. Autosys eam is responsible for monitoring the environment for alarms or conditions that re!uire special attention to avoid business impacting incidents or outages" and rapidly responding to ma#or incidents as they occur. 4.2 Scope In scope$ Point of contact for all the alerts% he Autosys team is the first point of contact for any middleware issues. he (#Operations Alerts) team will send an email regarding any alert. hese emails are tracked by a Autosys team team member. Follow a proactive approach: he Autosys team will have a proactive approach to any alert and escalate to resolve issues before it becomes critical. his approach will help to increase productivity. Immediate Action for PROD alerts$ Autosys team will provide immediate response to the production related alerts as it is very critical. &ased on the re!uirement Autosys team will also provide the diagnostics. Initiate bridge calls$ Autosys team is responsible for initiating bridge calls. Autosys team member needs to open the bridge call. Autosys team will be a part of the bridge call throughout the lifecycle of the incident and will provide the re!uired diagnostics. Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e '' o4 20 Out of scope$ he Autosys team will not be responsible for handling non%production alerts. Autosys team will not be responsible for initiating change re!uests. 4.3 +efinitions and Acronyms Ce; o4 the acrony-s an& or its &e4initions 6ro+i&e& here -ay not ha+e been use& in this &ocu-ent7 but has been 6ro+i&e& 4or easy re4erence to their &e4initions. -!3!1 Acronyms Acronm Description WCC Workload Control Center 4.4 S%A All alerts are classi4ie& into a Sin3le 6riority le+el base& on that ;e nee& to ackno;le&3e an& res6on& to the alerts an& in4or- the job o;ners about the alert recei+e& 4or the job 4ailures ;ithin )( -in 4ro- the ti-e alert ;as 3enerate&. ,his also inclu&es re-e&y ticket creation. 4.5 Roles and Responsibilities -!4!1 5'ent monitorin* team>s process ,he Autosys tea- ;ill recei+e alerts in -onitorin3 tool ECC. 5'ent Mana*ement C 8CC Monitorin* '. (ogin to WCC using through this )*( http$++mps,-./$0101+wcc2ain(ogin+ . Put username as your PIC code and password as your 23C windows password. ,. 3nce you login you will reach the 2ain ab. In the main screen there are many views" in that choose 4(3&A( 5I6W. Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e ') o4 20 /. After logging in to 4(3&A( 5I6W" choose Alerts tab. Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e '2 o4 20 7. In the Alerts tab we see the #ob failure alerts generated for the #obs that have failed. 8irst of all Acknowledge the alert as shown below. Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e '0 o4 20 9. Check the #ob contact list for the failed #obs to get the #ob owner:s details. -. Create ticket on user:s name and notify the same to user in ,1 minutes. .. While notifying failure to the user attach err logs in it. 0. 3nce the issue+alert is worked on and resolved" close the acknowledged alert in WCC. -!4!/ 5'ent Mana*er ,he Autosys tea- is the 4irst 6oint o4 contact 4or any -i&≤are K Database alerts. ,he -onitorin3 tools ;ill sen& an e-ail re3ar&in3 any alert. Autosys tea- ;ill 6ro+i&e initial su66ort 4or all -ajor an& -inor 6ro&uction alerts. It ;ill res6on& to the alerts ;ithin the SFA &e4ine& an& base& on the se+erity o4 the alert. 4!" 5'ent Mana*ement .uidelines 4!1!1 Pro0ect specific *uidelines for Minor Incident <andlin* by Autosys team ,he Autosys tea- shall recei+e the alertsinci&ents an& 6ro+i&e 4irst 6oint res6onse an& resolution. Inci&ent is o6ene& an& &etecte& by the Autosys tea-. I4 the ser+ice is restore&7 the ticket is close&. I4 not escalate& to F2 tea- . ,he 6ri-ary -etho& o4 escalation is +ia an e-ail. I4 no F2 Pro&uction Su66ort -e-ber is a+ailable7 contact can be -a&e +ia -obile 6hone. $ntil a res6onse has been recei+e&7 an issue shoul& be consi&ere& as non#escalate&. Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e 'A o4 20 On escalation F2 tea- shall ;ork on it an& 6ro+i&e resolution. 4!1!/ Pro0ect specific *uidelines for Ma0or Incident <andlin* by Autosys team A MA=OR inci&ent can occur in any one o4 the 4ollo;in3 t;o ;ays. MA=OR alert 4ro- Monitorin3 ,ools MA=OR inci&ent create& by O6erationsA66lication ,ea- MA=OR alert 4ro- Monitorin3 ,ools In the e+ent o4 a MA=OR alert7 Autosys tea- -e-ber has to 6er4or- the initial in+esti3ation an& create a Se+)Se+' inci&ent ticket &e6en&in3 u6on the business i-6act. I4 the MA=OR alert re8uires i--e&iate attention7 then Autosys tea- -e-ber has &o the 4ollo;in3 ,ea- has to o6en a bri&3e an& call the OnsiteF2 tea-. In case o4 a66lications or &atabases o;ne& by Mi&≤are or &atabase tea-7 other MEDB tea- -e-bers shoul& be in+ol+e&. Collect the re8uire& &ia3nostics Resol+e or escalate the inci&ent to F2 tea-. I4 restart o4 a Pro&uction resource is re8uire& an Inci&ent Mana3er an& a ,echnical Mana3er shoul& be a+ailable in the bri&3e. Once the inci&ent is resol+e&7 ;e can close the Ser+ice "o; ticket a4ter u6&atin3 the Proble- ticket in it. "ote: I4 there are any MA=OR alerts tri33ere& by the -onitorin3 tools are &ue to any sche&ule& restart or chan3e7 they can be close& a4ter 6ro+i&in3 the chan3e nu-ber. MADOR incident created by Operations$Application ,eam Autosys tea- -e-ber shoul& o6enjoin a bri&3e 4or any MA=OR inci&ent. Autosys tea- -e-ber has to join the bri&3e an& &o the 4ollo;in3: Per4or- initial in+esti3ation to analy<e i4 it9s an a66lication issue or in4rastructure issue. Collect the re8uire& &ia3nostics Resol+e or escalate the inci&ent to F2 tea-. I4 restart o4 a Pro&uction resource is re8uire& an Inci&ent Mana3er an& a ,echnical Mana3er shoul& be a+ailable in the bri&3e. Once the inci&ent is resol+e&7 either Autosys tea- -e-ber can close the Ser+ice "o; ticket or ask O6s to close the ticket a4ter u6&atin3 the Proble- ticket in it. Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e '/ o4 20 3!1 ,icketin* ,ool ?Remedy@: '. Fo3in to MOC Re-e&y usin3 the $rl as belo;? MOC#$RF L htt6s:ser+ice&eskn).nais-c.co-arsysshare&lo3in.js6Marsys4or-snar-&y)MOCNDe l6DeskSu66ort ). Pro+i&e the $serna-e an& Pass;or&. 2. Search 4or re-e&y tickets base& on :rou6 i.e. MOCCOA$,OSPSOS$PPOR, 0. Click on any 6articular re-e&y ID to +ie; the 4ull &etails o4 the ticket. A. I4 4oun& any tickets as status Q"e;R or QAssi3ne&R assi3n the ticket to the in&i+i&ual an& chan3e the status to QEork in Pro3ressR7 as ;ell as u6&ate Eork lo3 section accor&in3ly. /. ,o +ie; the SFA &etail7 click on the SFA Earnin3 Console link. ,hen7 select the 3rou6 i.e. MOCCOA$,OSPSOS$PPOR, B. Select the tickets liste& to +ie; the SFA &etail o4 res6ecti+e ticket. *. Chan3e the status as QEork In Pro3ressR once starte& ;orkin3 on the ticket an& u6&ate Eork Fo3 o4 the ticket as such. 1. I4 nee& in4or-ation 4or 6rocessin3 the ticket7 sen& an e-ail to the re8uestor ;ith the ticket &etails7 re8uestin3 the in4or-ation nee&e&. $6&ate the status o4 the ticket as QPen&in3 Re8uestor In4or-ationR an& u6&ate the Eork Fo3 as such. Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e 'B o4 20 '(. Once 3ot the re8uire& in4or-ation 4ro- the re8uestor7 u6&ate the ticket Eork Fo3 ;ith the in4or-ation an& chan3e the status o4 the ticket to QEork In Pro3ressR. ''. Once the ticket is 6rocesse&7 sen& an e-ail to re8uestor in4or-in3 the ticket co-6letion an& u6&ate the Eork Fo3 as such. Chan3e the status to QPen&in3 A66ro+alR '). Mark the ticket as QResol+e&R once con4ir-e& by the re8uestor about ticket resolution by an e-ail. $6&ate the Eork Fo3 an& Inci&ent Resolution o4 the ticket as such. '2. I4 not recei+e& any res6onse 4ro- the re8uestor a4ter the ste6 '' abo+e? sen& 2 4ollo; u6 e-ails 4or the con4ir-ation o4 the ticket closure. '0. Mark the ticket as QResol+e&R un&er 2#Strike Process i4 not recei+e& con4ir-ation e-ail 4ro- re8uestor a4ter Ste6 '2 abo+e. 3!/ Ser'ice Re6uest .uidelines his ;ervice *e!uest 2anagement document provides operational instructions to follow to manage the ;ervice *e!uests received in the environment $ Ser'ice Mana*ement Procedure 1!" 1!" Introduction 6.1 Purpose ,he 6ur6ose o4 this &ocu-ent is to 6ro+i&e in4or-ation about ser+ice re8uest an& e>ecution 6rocess in Marathon MOC .n+iron-ents. 6.2 Scope ,he sco6e o4 this &ocu-ent is li-ite& to Marathon MOC Ser+ice re8uests. 6.3 +efinitions and Acronyms Ce; o4 the acrony-s an& or its &e4initions 6ro+i&e& here -ay not ha+e been use& in this &ocu-ent7 but has been 6ro+i&e& 4or easy re4erence to their &e4initions. ,he ter- Q"on#Pro&uction Chan3e Re8uestR re4ers to the Ser+ice Re8uests as in I,IF !2. 1!3!1 Acronyms Acronym +escription MOC Marathon Petroleu- Co-6any I,IF In4or-ation ,echnolo3y In4rastructure Fibrary Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e '* o4 20 CI Con4i3uration Ite- RCC Re8uest 4or Chan3es 1!3!/ +efinitions ,erm +efinition "on Pro&uction It inclu&es De+elo6-ent7 ,est en+iron-ent Priority Priority in&icates the relati+e or&er in ;hich a series o4 ite-s # shoul& be a&&resse&. Proble- An $nkno;n un&erlyin3 cause o4 one or -ore inci&ents Inci&ent any e+ent ;hich is not 6art o4 the stan&ar& o6eration o4 a ser+ice an& ;hich causes7 or -ay cause7 an interru6tion to7 or a re&uction in7 the 8uality o4 that ser+ice. Chan3e ,he a&&ition7 -o&i4ication or re-o+al o4 a66ro+e& su66orte& or baseline& har&;are7 net;ork7 so4t;are7 a66lication7 en+iron-ent7 an& syste-7 &eskto6 buil& or associate& &ocu-entation. EI%I It is a tool 4or the &iscussion an& sharin3 the &ata. It 6ro+i&es all the &etails 4or "on 6ro&uction. 2!" ,asks 7.1 Ser'ice re6uest ,he Autosys R'' Su66ort tea- is res6onsible 4or -aintainin3 an& su66ortin3 the MOC Autosys en+iron-ent o4 Marathon. One o4 the acti+ities that the Su66ort tea- is e>6ecte& to 6er4or- is L ;orkin3 on Ser+ice Re8uests7 Ser'ice Re6uests are Re-e&y tickets that &o not re8uire an e>tension or chan3e in so4t;are 4unctionality or a chan3e in ;ork 6rocesses. Routine Ser+ice Re8uests 4ollo; a 6re&eter-ine& 6rocess 4or 4ul4ill-ent an& &o not re8uire analysis. Ser+ice Re8uests are routine an& re6etiti+e in nature an& not +ariable or a&hoc. Do;e+er7 Ser+ice Re8uests are not BA$ Chan3e Re8uests because these &o not re8uire any chan3es in co&e. Ser+ice Re8uests are 3o+erne& by strict SFAs L both 4or res6onse ti-e an& resolution ti-e. Res6onse ,i-e is &e4ine& as the ti-e bet;een ticket creation an& ticket acce6tance in Re-e&y. Due to this7 it is i-6erati+e that the Su66ort tea- kee6s -onitorin3 the Re-e&y ticket 8ueue on all ;ork&ays an& acce6t tickets as soon as they are create&. Res6onse ,i-e is &e4ine& as the ti-e it takes 4or a ticket to be -o+e& to Eork in Pro3ress. So7 the Su66ort tea- nee&s to ensure that they chan3e the status o4 a ticket to Eork in Pro3ress as soon as they start ;orkin3 on a ticket. Resolution ,i-e is &e4ine& as the ti-e bet;een ticket creation an& ticket closure G;hen status is chan3e& to Resol+e&H in Re-e&y. ,his &oes not inclu&e the ti-e a ticket s6en&s in +arious Pen&in3 statuses. So7 it is i-6erati+e that the ticket status is chan3e& to the a66ro6riate Pen&in3 status ;hene+er the Su66ort tea- is ;aitin3 4or in6uts 4ro- another tea- or the re8uestor. Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e '1 o4 20 7.2 S%A All ser+ice re8uests are classi4ie& into &i44erent 6riority le+els base& on their i-6ortance an& 6riority le+el 4or business7 users an& in4rastructure. ,hese re8uests ;ill be res6on&e& an& actione& ;ithin 4ollo;in3 ti-elines base& on their 6riority. ,hese SFAs are in 6rocess o4 bein3 -a66e& ;ith tasks in Ser+ice Catalo3. Priority Response S%A ?mins@ Resolution S%A ?mins@ P' 'A )0( P) 'A 0*( P2 '00( 02)( P0 '00( 02)( 2!/!1 Ser'ice Re6uest Classification ,his section in&icates ser+ice re8uest classi4ication ty6e an& criteria base& on the 6riority7 I-6act on business an& $r3ency. Ser'ice Re6uest is classified as follo(s! Incident Classification Criteria for classification Critical L P' All 6ro&uction ser+er re8uests a44ectin3 -ulti6le a66lications to be consi&ere& as P' Critical Major L P) All 6ro&uction ser+er re8uests a44ectin3 sin3le a66lication to be consi&ere& as P) -ajor Minor L P2 Ser+ice re8uests ;hich are to be consi&ere& as P2 Fittlenone L P0 All &e+elo6-ent ser+ice re8uests to be consi&ere& as P0 1!1!1 Ser'ice Re6uest Process Blo( Acce6t the Re-e&y ticket an& &eter-ine i4 it has been correctly classi4ie& as an Inci&ent or Ser+ice Re8uest. Mo+e the ticket to QEork in Pro3ressR status. Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e )( o4 20 In the case o4 a Re-e&y ticket is ;ron3ly raise& as hi3h se+erity ticket Gi.e.7 i4 the se+erity shoul& be lo;ere&H7 contact the Autosys Ser+ice Deli+ery Coor&inator to a66ro+e re&ucin3 the se+erity o4 the inci&ent. Contact the re8uestor +ia 6hone an& +ia e-ail Gco6yin3 AutosysSupport &istribution listH to let the- kno; that the ticket has been 6icke& u6 by the su66ort tea- -e-ber an& also to &iscuss &etails 6ertainin3 to the issue;ork the re8uestor nee&s resol+e&co-6lete&. I4 the su66ort tea- -e-ber is not able to reach the re8uestor +ia 6hone they ha+e to lea+e a +oice-ail in4or-in3 the re8uestor that the ticket has been assi3ne& to the su66ort tea- -e-ber. ,he ;ork lo3 in Re-e&y nee&s to be u6&ate& ;ith &etails 6ertainin3 to all co--unications -a&e ;ith the re8uestor. In the case o4 a Di3h Se+erity ticket7 Autosys Ser+ice Deli+ery Coor&inator GOnsiteO44shoreH7 Client Mana3er an& Co3ni<ant -ana3e-ent shoul& be co6ie& ;hen this initial e-ail is sent. Assess the ticket an& &o an initial analysis to &eter-ine ;hat cause& the inci&ent Gor in case o4 Ser+ice Re8uest ;hat coul& be &one to resol+e itH $6&ate the ticket ;orklo3 ;ith the initial analysis Sen& re3ular u6&ates to the re8uester +ia e-ail Gan& co6y the e-ail to the Re-e&y ;orklo3H. In case o4 hi3h se+erity tickets7 these e-ail u6&ates nee& to be sent at least once an hour 4or Se+erity ' tickets an& at least once e+ery ) hours 4or Se+erity ) tickets. Marathon an& Co3ni<ant -ana3e-ent shoul& be co6ie& on these e-ail u6&ates. I4 a66licable7 6ro+i&e a ;ork aroun& to restore the Marathon Fi+e Syste- or 6ro+i&e a 4i>7 in accor&ance ;ith the a66licable Ser+ice Fe+els. I4 the Inci&ent or Ser+ice Re8uest re8uires a chan3e7 4ollo; the a66licable Chan3e Mana3e-ent 6roce&ures. In4or- the re8uestor once the issue has been resol+e& an& 3ain their concurrence that the ticket can be -arke& resol+e& or close&. $6&ate the Resolution tab in Re-e&y ;ith the ste6s taken to resol+e the ticket. ,his is a use4ul %no;le&3e Mana3e-ent 6ractice an& ;ill hel6 cut &o;n the Resolution ti-e i4 a si-ilar issue ha66ens in the 4uture. 1!1!/ Roles and Responsibilities Sl! )o Acti'ity Responsibility Dri+in3 the e44iciency K e44ecti+eness o4 the ,icket Mana3e-ent 6rocess. Inclu&in3 re3ular au&its o4 tickets to ensure they are bein3 u6&ate& accurately7 ti-ely an& co-6letely G;ith s6ecial attention to the 8uality o4 in4or-ation containe& in the ;ork lo3.H O44shore O6erations Fea& Pro&ucin3 -ana3e-ent in4or-ation Mana3in3 the ;ork o4 inci&entSer+ice re8uest su66ort sta44 Monitorin3 the e44ecti+eness o4 ser+ice -ana3e-ent an& -akin3 reco--en&ations 4or i-6ro+e-ent Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e )' o4 20 Sl! )o Acti'ity Responsibility Inci&entser+ice re8uest re3istration Marathon Ser+ice &esk Routin3 inci&entser+ice re8uest throu3h su66ort 3rou6 ;hen inci&ents are not close& Initial su66ort K classi4ication Dan&lin3 inci&entser+ice re8uest escalation Co3ni<ant ,ea- Ser+ice re8uest in+esti3ation an& &ia3nosis Detection o4 6ossible 6roble- an& the assi3n-ent o4 the- to the 6roble- -ana3e-ent tea- 4or root cause analysis ,he resolution K reco+ery o4 assi3ne& Inci&ent Closure o4 ser+ice re8uest 7.3 8orkin* on Ser'ice Re6uests 'A. Fo3in to MOC Re-e&y usin3 the $rl as belo;? MOC#$RF L htt6s:ser+ice&eskn).nais-c.co-arsysshare&lo3in.js6Marsys4or-snar-&y)MOCNDe l6DeskSu66ort '/. Pro+i&e the $serna-e an& Pass;or&. 'B. Search 4or re-e&y tickets base& on :rou6 i.e. MOCCOA$,OSPSOS$PPOR, '*. Click on any 6articular re-e&y ID to +ie; the 4ull &etails o4 the ticket. '1. I4 4oun& any tickets as status Q"e;R or QAssi3ne&R assi3n the ticket to the in&i+i&ual an& chan3e the status to QEork in Pro3ressR7 as ;ell as u6&ate Eork lo3 section accor&in3ly. )(. ,o +ie; the SFA &etail7 click on the SFA Earnin3 Console link. ,hen7 select the 3rou6 i.e. MOCCOA$,OSPSOS$PPOR, )'. Select the tickets liste& to +ie; the SFA &etail o4 res6ecti+e ticket. )). Chan3e the status as QEork In Pro3ressR once starte& ;orkin3 on the ticket an& u6&ate Eork Fo3 o4 the ticket as such. )2. I4 nee& in4or-ation 4or 6rocessin3 the ticket7 sen& an e-ail to the re8uestor ;ith the ticket &etails7 re8uestin3 the in4or-ation nee&e&. $6&ate the status o4 the ticket as QPen&in3 Re8uestor In4or-ationR an& u6&ate the Eork Fo3 as such. )0. Once 3ot the re8uire& in4or-ation 4ro- the re8uestor7 u6&ate the ticket Eork Fo3 ;ith the in4or-ation an& chan3e the status o4 the ticket to QEork In Pro3ressR. )A. Once the ticket is 6rocesse&7 sen& an e-ail to re8uestor in4or-in3 the ticket co-6letion an& u6&ate the Eork Fo3 as such. Chan3e the status to QPen&in3 A66ro+alR )/. Mark the ticket as QResol+e&R once con4ir-e& by the re8uestor about ticket resolution by an e-ail. $6&ate the Eork Fo3 an& Inci&ent Resolution o4 the ticket as such. )B. I4 not recei+e& any res6onse 4ro- the re8uestor a4ter the ste6 '' abo+e? sen& 2 4ollo; u6 e-ails 4or the con4ir-ation o4 the ticket closure. )*. Mark the ticket as QResol+e&R un&er 2#Strike Process i4 not recei+e& con4ir-ation e-ail 4ro- re8uestor a4ter Ste6 '2 abo+e. Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e )) o4 20 3!3 Incident Mana*ement .uidelines his Incident 2anagement document provides operational instructions to follow to manage the Incidents happening in the environment$ https$++ch'workspaces+sites+;W'10+23C<Autosys<Process<=ocuments+;hared >,1=ocuments+23C+23C<Autosys<;upport<Incident<2anagement<Procedure< 5'.1.doc his C*I;I document provides instructions to follow in Critical ;ituation 2anagement $ https$++ch'workspaces+sites+;W'10+23C<Autosys<Process<=ocuments+;hared >,1=ocuments+23C+2arathonAutosysCrtisit23C.doc -!" 5scalation Matri9 Please re4er this sheet to reach out to the O44shore ,ea- in case o4 any issues : htt6s:ch';orks6acessitesSE'(*MOCOAutosysOProcessODocu-entsShare& J)(Docu-entsMOCShi4tContactDetails.>ls I4 you &o not 3et any res6onse7 6lease 4ollo; the belo; escalation -atri> to ha+e the issues a&&resse& : "a-e Role Business Mobile Pri-ary Contact ,ea- Me-ber' On Call Pri-ary N1' 1B21)/)((( N1' 1B21)/)((( Alternate Secon&ary Contact ,ea- Me-ber ) On Call O44shore N1' 1B21)/1((( N1' 1B21)/1((( ,ertiary Contact '. ;ourav Chattergee ). Dhara Chauhan 2. Ranjeeth %u-ar O44shore O6erations Fea& Onsite Fe+el) Onsite Fe+el2 .n3ineer ?@'%01%7/-,7111 6At B 9'7/-' 0@' .'/%,@-% ,07- 0@' .'/%,@-% ,/0. N1' B0(#/(0#0))( *1' *2)/2*0A/0 0@' ,1'%,07%'007 Pri-ary .scalation %ishan , S Deli+ery Mana3er N1'#*(#02/)0((( .>t L A'0)A1 N1' 11*/(211'2 Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e )2 o4 20 Contact Secon&ary .scalation Contact '. Puthanveettil" ;ivakumar ) ). ! =ero-e Cernan&e< Sr.Deli+ery Mana3er Sr.Deli+ery Mana3er .'/ ,@- ,000 0@' B'2#/)1#//(( )(' )*/ B0*) *1' )('#/*/#/A0* ,ertiary .scalation Contact %e&arnath Mukherjee Assoc. Director # I, IS ?@'%01%/1.1/0-7 N1' 1//221*'*B -!1 Customer 5scalation Process !en&or contact list K escalation -atri> is as 3i+en belo; )ame Or*ani=ational Role eEMail Phone Patrick Stel<er ;enior Information echnology ;pecialist 6bstel<erS-arathonoil.co- B'2#)1/ )/2B GOH =ulie Fo6e< Information echnology 2anager #slopeCDmarathonoil.com *1' .'/%,@-'@.' GOH *2)*'/A0A/ GMH Anne % Dunt Compliance =irector akhuntDmarathonoil.com .'/%,@-/@./ (3) .'/%71@%7,0, (2) 7!" Capacity Mana*ement Ee ensure that I, ca6acity 4or Marathon -eets current an& 4uture business re8uire-ents in a cost# e44ecti+e -anner. Ca6acity Mana3e-ent is a ser+er -ana3e-ent solution that e>a-ines a66lications7 ser+ers an& &atabases to hel6 I, -ana3ers an& a&-inistrators i&enti4y 6er4or-ance 6roble-s in critical business 6rocesses. It has the ability to ensure the ser+ers that host or rein4orce autosys a66lications run s-oothly an& 6er4or- ;ithin acce6table li-its. Per4or-ance Monitorin3 6ro+i&es inte3rate& +ie;s into a66lication7 ser+er an& &atabase a+ailability an& 6er4or-ance7 so the 6roble-s can be resol+e& be4ore they i-6act the business. ,his ensures increase& a66lication a+ailability an& re&uce& res6onse ti-es Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e )0 o4 20 ,his &ocu-ent e>6lains ho; ;e ha+e take the back u6 o4 the ra; &ata 4or Ca6acity -ana3e-ent : htt6s:ch';orks6acessitesSE'(*MOCOAutosysOProcessODocu-entsShare& J)(Docu-entsMOC.>tractin3OtheORa;O&ataO4orOCa6acityOMana3e-ent.&oc> ,his &ocu-ent e>6lains ho; to 3et 3ra6hs 4ro- SCOM 4or Ca6acity -ana3e-ent Re6ortin3 : https:$$ch1(orkspaces$sites$S81"7$MOC:Autosys:Process:+ocuments$Shared ;/"+ocuments$MOC$<o(:to:*et:*raphs:from:Scom!doc9 1!" #usiness Continuity Plan ,his &ocu-ent 6ro+i&es the in4or-ation about the Co3ni<ant9s 6lan 4or the Disaster Reco+ery GDRH to ensure the Business Continuity : https:$$ch1(orkspaces$sites$S81"7$MOC:Autosys:Process:+ocuments$Shared ;/"+ocuments$MOC$MOC:1""""1-//1:#CP+ocument!pdf 2!" Induction Plan ,his &ocu-ent 6ro+i&es in4or-ation on ho; a ne; joiner ;ill be traine& in&ucte& to suit to Su66ort Marathon en+iron-ent : htt6s:ch';orks6acessitesSE'(*MOCOAutosysOProcessODocu-entsShare& J)(Docu-ents:eneralOIn&uctionOPlanOMarathon.>ls 7!" Ano(led*e Mana*ement '. ,his &ocu-ent is a re4erence 3ui&e 4or Autosys usersa&-inistrators : htt6s:ch';orks6acessitesSE'(*MOCOAutosysOProcessODocu-entsShare& J)(Docu-entsMOCAutoSysORe4erenceO:ui&eOG$"ITH.6&4 ). Autosys R'' =ob sche&ulin3. ,his &ocu-ent e>6lains about the 6rocess in+ol+e& in settin3 u6 a job sche&ule :
R''O=obOSche&ulin3 .&oc> 2. ,his &ocu-ent 6ro+i&es the list an& &escri6tion o4 the Basic co--an&s usa3e in Autosys Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e )A o4 20 R'' : htt6s:ch';orks6acessitesSE'(*MOCOAutosysOProcessODocu-entsShare& J)(Docu-entsMOCAutosysOCo--an&s.66t> 0. ,his &ocu-ent e>6lains all the ste6s to be 4ollo;e& 4or Mi3ratin3 AutoSys jobs 4ro- 0.A to R'' : htt6s:ch';orks6acessitesSE'(*MOCOAutosysOProcessODocu-entsShare& J)(Docu-entsMOCAutoSysOR''O=obOMi3rationsOProcess.&oc> A. .>6lains the ste6s nee& to be 4ollo;e& 4or the SAP Eeeken& Maintenance Acti+ity :
Eeeken& Maintenance Proce&ure.&oc> /. =ob On boar&in3 re8uest ,his &ocu-ent e>6lains the &etails ;e ask 4ro- re8uestor 4or a ne; job
Aut osys R'' Su66ort #=ob On boar&in3 re8uest . &oc> B. Proce&ure 4or "e; job creation an& Pro-otion
=ob creat ion an& Pro-otion. &oc *. =ob O6ti-i<ation ,his &ocu-ent e>6lains all the ste6s to be 4ollo;e& 4or Inacti+e job cleanu6 6rocess
A$,OSPS R''# =ob o6t i-i<at ion. &oc> 1. =ob Contact Fist Collo;in3 &ocu-ent contains all the u6&ate& contact list an& restart 6roce&ure re8uire& in case o4 =ob Cailure: UUctsin-b6s4ubUAutosysR''U=obOContactUMOC Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e )/ o4 20 Ee kee6 on u6&atin3 the list ;hene+er there is a chan3e in job contact an& restart 6roce&ure. 5uarterly ;e re+ie; the &ata ;ith the list that ;e ha+e in ,ea- !ie; site htt6:;;).i.-oc.co-sitesit#ent6rs#a66l#inte3rat#co6FistsAutosysJ)(=obDoc J)(J)(SAPAllIte-s.as6> ,o o6en ,ea- !ie; site7 ;e nee& to lo3on to Marathon net;ork usin3 Citri>. he Contact details are available in below two 6Acel sheets in share as well !on"#AP"$ontact%&ls #AP"$ontacts'lat%&ls 8iles are saved in our share drive$ EEcyrrs-19EWorkgroups<=rive7EAutosys*''EprodE6A2E6; (nderstanding the file: )ob !ame$ column should have the name of the #ob as defined in Autosys. $onfirm #end *mail$ his column should be always have a value + or ! + Indicate of the user re,uested for the email notification for the -ob .Default will be + for all new -obs until the user e&plicitl re,uest to stop it) Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e )B o4 20 ! indicate that the user re,uested to stop the notification )ob Restart Instruction: hese instructions are internal to support which is re!uested by the user. Any instructions updated in this column will be sent to the support team onl and not to user Primar $ontact: 2ake sure where ever you have a primary contact use the user 8irst name in the primary contact column as it makes more easy to address in the mail 62 sends Primar $ontact *mail: his column should contact only emails and not other data should be stored #econdar $ontact: 2ake sure where ever you have a ;econdary Contact use the user 8irst name in the ;econdary Contact+=( name column as it makes more easy to address in the mail 62 sends If there happens to be multiple users as a contact for any #ob all the names and email I=:s should be comma separated and !O/ semicolon '(. Ste6s to 4ollo; in case o4 Autosys Database Ser+er una+ailability :
Aut osysODat abaseO Ser+erO$na+ailabilit y.&oc ''. Ste6s to 4ollo; in case o4 Autosys Sche&uler Do;n :
AutosysOSche&ulerOi ssue.&oc '). Ste6s to 4ollo; in case o4 ECC Dun3 Issue :
ECCOhun3Oissue. &o c '2. Ste6s to 4ollo; to resol+e .>it co&e .rror ')) an& ')2 Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e )* o4 20
Process to resol+e .>it co&e .rror ')) an& ')2.&oc> '0. Ste6s to 4ollo; to set u6 co--on e#-ail bo>
I nst ruct ions t o set u6 Co--on e#-ailbo>. &oc 'A. Autosys R'' Ser+er Ser+ices ,his &ocu-ent lists all the Autosys Ser+er Ser+ices that ;e -onitor
Aut osys R'' Ser+er Ser+ices. &oc> '/. Calen&ar creation: ,his &ocu-ent e>6lain ho; to createe&it calen&ars in Autosys. Eith the calen&ar 4ro- Duane Dinck7 ;e9+e to u6&ate the BE calen&ars as ;ell. htt6s:ch';orks6acessitesSE'(*MOCOAutosysOProcessODocu-entsShare& J)(Docu-entsMOCCreatin3J)(Calen&arJ)(&ocu-entation.&oc> 'B. MRO#MonthlyMaintenance ,his &ocu-ent e>6lains the ste6s to be 4ollo;e& &urin3 Monthly DP Maintenance on Autosys Ser+ers:
$6&at e& MRO#Monthly Maint enance. >ls> 3!" Ma0or Incidents ,his &ocu-ent lists the &etails about AutoSys R'' $n6lanne& Outa3e # PROD$C,IO" # )(th Au3ust Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e )1 o4 20 )('2: Aut osys R'' $n6lanne& Out a3eOPro&uctionO)(t h Au3ust )('2. &oc> 1"!"RCA Ee ca6ture root cause analysis 4or all the -ajor acti+ities that ha66en ;ithin the en+iron-ent Initial RCA shoul& be sub-itte& ;ithin A &ays o4 the issue an& 4inal RCA shoul& be co-6lete& by 'A th
&ay 4ro- &ate o4 issue. '. A$,OSPS DB 4ailure 12()(') '0:(( CS, MRO Root Cause Analysis A$,OSPS DB 4ailure (12()('). &oc> ;teps to follow if same issue happens again
Aut osysODat abaseO Ser+erO$na+ailabilit y.&oc ). ECC Outa3e (*'))(') '(:2( PM CS, https$++ch'workspaces+sites+;W'10+23C<Autosys<Process<=ocuments+;hared >,1=ocuments+23C+2*3>,1*oot>,1Cause>,1Analysis%WCC >,13utage.docA ;teps to follow if same issue happens again htt6s:ch';orks6acessitesSE'(*MOCOAutosysOProcessODocu-entsShare& J)(Docu-entsMOCECCOhun3Oissue.&oc /. 2P;,-..Fob8ailure eAit code',, ,9+1'+,1'/ https$++ch'workspaces+sites+;W'10+23C<Autosys<Process<=ocuments+;hared >,1=ocuments+23C+2*3>,1*oot>,1Cause>,1Analysis%2P;,-..>,1Fob >,18ailure>,1eAit>,1code>,1',,.docA ;teps to follow if same issue happens again Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e 2( o4 20 https$++ch'workspaces+sites+;W'10+23C<Autosys<Process<=ocuments+ ;hared>,1=ocuments+23C+Process>,1to>,1resolve>,16Ait>,1code >,16rror>,1',,>,1and>,1',/.docA 7. Fobs failed without logs<2P;,-.. 1@+1-+,1', 1@$11 A2 https$++ch'workspaces+sites+;W'10+23C<Autosys<Process<=ocuments+;hared >,1=ocuments+23C+2*3>,1*oot>,1Cause>,1Analysis%Fobs>,1failed >,1without>,1logs<2P;,-...docA ;teps to follow if same issue happens again https$++ch'workspaces+sites+;W'10+23C<Autosys<Process<=ocuments+ ;hared>,1=ocuments+23C+Process>,1to>,1resolve>,16Ait>,1code >,16rror>,1',,>,1and>,1',/.docA 9. Cluster heartbeat and &ur GICs failed on the server 2P;,-., @+,@+,1', ,/$/1 C; https$++ch'workspaces+sites+;W'10+23C<Autosys<Process<=ocuments+;hared >,1=ocuments+23C+=raft>,12*3>,1*oot>,1Cause>,1Analysis >,1'.,,-',.docA his is still under investigations that why the fabric failover configuration was missing -. =eadlock was observed in Autosys =& on 17+''+,1'/ 11$,9$'9 A2 C; https$++ch'workspaces+sites+;W'10+23C<Autosys<Process<=ocuments+;hared >,1=ocuments+23C+2*3>,1*oot>,1Cause>,1Analysis%17%''3utage% 8inal.docA 11!" Automation As 62 automation is complete" in case of #ob failure an automated email is triggered to Autosys eam that simplifies the monitoring process. As well an email is triggered to #ob owner. If there is any change or addition to #ob contact+ restart procedure" we make the changes in our contact list in eam 5iew as well internal share and then send the list to *an#eeth. He makes the changes at 62 level for automation and we seek confirmation within ,7 hours. &elow are the steps followed for #ob failure notification$ Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e 2' o4 20 ;end email notification to the users for every #ob failures if not sent by 62. Gotification email will tell us for which failure the #ob owner has been and not been notified. ,. *emedy tickets for the failures needs to be created. 6Aception$ Go remedy ticket needs to be created until asked by the #ob owner for the error I=ata 6Aception 6rrorJ for all ;AP%H*" ;AP%H*=8 and ;AP%H**P #obs. ;ee the attached email for more clarification. /. =ifferent remedy tickets to be created for different #ob failures. 6g$ If the #ob ;AP%AP%KLM and ;AP%AP%PN* fails at the same time" two different tickets need to be created. If two or more #obs fail within the same boA #ob then only one remedy ticket is enough. 7. If a #ob fails and there is already a remedy ticket eAisting" add the #ob failure to the eAisting ticket" no need to create a separate ticket. 9. Attach logs to the #ob failure ticket. -. A separate CI for #ob forcestart has been created" please use that. .. If a ticket comes to our !ueue with a wrong CI" please correct the CI. 0. Fob 8ailure Process for &W Fobs$ Autosys # SAP BE =ob Cailure P... 1/!"Roles and Responsibilities 1/!1 Operator F Production control Analyst le'el Checklists The following table provides various checklists that are applicable for the engagement. These will be reviewed on a periodic basis depending on engagement needs. 01%0%0 Operator 0 and Operator 1 Roles https$++ch'workspaces+sites+;W'10+23C<Autosys<Process<=ocuments+;hared >,1=ocuments+23C+3pr'<and<3pr,<*oles<and<*esponsibilities.AlsA 2%0%1 /ech members Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e 2) o4 20 Be a+ailable on ser+ice 6er shi4t an& on call roster :et tickets 4ro- Re-e&y an& Prioriti<e the- $n&erstan& the issues an& resol+e the- ;ithin SFA A&here to the ticket -ana3e-ent 6rocess inclu&in3 the ti-ely an& accurate u6&atin3 o4 ticket in4or-ation an& co--unications to the ticket re8uestor Docu-entation o4 RCAs ;ithin SFA Partici6atin3 in internal an& e>ternal au&its con&ucte& at Marathon Re6ort to O44shore O6erations Fea& on a &ay to &ay basis Partici6ate in re3ular tea- -eetin3s ,echnical ,rainin37 coachin3 an& -entorin3 o4 ,ea- -e-bers %B creation an& -aintenance 2%0%3 #hift lead 5ueue -ana3e-ent7 ticket assi3n-ent an& 4ollo;#u6 Incident Mana*er: Shi4t Fea& acts as an Inci&ent Mana3er an& he is accountable 4or o+erall coor&ination o4 assi3ne& Inci&ents an& SFA co-6liance7 co-6laint an& escalation 6oint 4or -ajor Inci&ent. Assi3n an& track the 6roble- Mana3e-ent 6rocess an& ensure the RCA9s are sub-itte& on ti-e ensure the SFA9s are -et Fea& the shi4t han&o+er calls an& han&o+er &ocu-ent to shi4t lea& o4 the ne>t shi4t ;ith &etails o4 all the u6&ates an& critical outa3es issues Assi3n an& track the tasks assi3ne& to the in&i+i&uals in the tea- Pre6are an& share &aily Shi4t re6orts 2%0%4 /eam 5ead ,o au&it ;ork lo3s in Re-e&y to ensure correctness o4 technical solution Be a+ailable on ser+ice 6er shi4t an& on call roster :et tickets 4ro- ,icketin3 tool an& Prioriti<e the- $n&erstan& the issues an& resol+e Partici6atin3 in internal an& e>ternal au&its con&ucte& at Marathon Re6ort to O44shore O6erations Mana3er on a &ay to &ay basis Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e 22 o4 20 ,rack 6ro3ress o4 tickets on a &aily basis Pro+i&e necessary technical 3ui&ance to the su66ort en3ineers to hel6 resol+e tickets ;ithin SFA Partici6ate in re3ular tea- -eetin3s an& &aily onsite#o44shore#onsite han&o+er .nsure Con4i3uration -ana3e-ent &ocu-entation is u6&ate& as re8uire&. Gat -ini-u- a4ter e+ery chan3e is co-6lete&H. Pre6are Release Plans in conjunction ;ith the Sr. Su66ort .n3ineer an& O6erational Fea&s 13!" Appendi9 1-!" Chan*e %o* Please note that this table nee&s to be -aintaine& e+en i4 se6arate Chan3e ite- status lo3 is -aintaine& in the tool &ocu-ent an& the +ersion history nee&s to be 3i+en here. But re4erence can be 3i+en to &etails o4 the chan3es i4 it is not 3i+en here. e rsi on ) u m be r Chan*es Made Dra4t Initial baseline create& on )* th Au3 )(') by "a3araj De3&e !'.( <Please refer the configuration control tool / change item status form if the details of changes are maintained separately if not the template given below needs to be followed> Pa*e )o! Chan*ed #y 5ffecti'e +ate Chan*es 5ffected ')7 '2 "a3araj 00)('2 A&&ition o4 Process &ocu-ents to %no;le&3e Mana3e-ent section '0 "a3araj *'()(' ) A&&ition o4 Process &ocu-ent 4or O6r ' an& O6r) Roles 1 "a3araj 2((1)( ') )(')('2 '. A&&ition o4 .+ent Mana3e-ent Proce&ure to Alert Dan&lin3 :ui&elines ). A&&ition o4 Ser+ice Desk Re4erence '0 Rajesh (A(B)( '2 A&&ition o4 =ob Contact at cyrr share Controlled Copy )on Production Chan*e Re6uest Procedure Project ID: '((((/00)/ 5,IM !er: '.) C3: Protected Pa3e 20 o4 20 '* Rajesh (A(B)( '2 A&&ition o4 =ob 4ailure 6rocess 4or BE jobs '' Rajesh (A'))( '2 A&&ition in .scalation Matri> L Inclu&e Ru&y. '/ Rajesh (A)B)( '2 $6&ate& MRO#Monthly Maintenance 'B Rajesh (A)B)( '2 RCA 4or Dea&lock obser+e& in Autosys DB )B Rajesh (*)')( '2 Major Inci&ent: $n6lanne& Outa3e in Pro&uction on )( th Au3 )('2 '' Rajesh ()(/)( '0 A&&ition in .scalation Matri> L Inclu&e Pat an& =ulie.