Using Database Statistics To Make Capacity Planning Decisions

Using Database Statistics to Make Capacity Planning Decisions
Geoff Mower
Senior Software Performance Engineer, Blackboard
Steve Feldman
Senior Director Software Performance Engineering and Architecture, Blackboard
Who Are We?
Geoff Mower- Senior Software Performance Engineer

5 years on the frontlines as a DBA
2 hour interview on a plane
Performance and Scalability for Outcomes & Community
Who Are We?
Steve Feldman
Blackboard since 2003 Performance Engineering from the start Platform Architecture in 2005 Security Engineering in 2010 Love my joblove my team. If you email me, I will respond.
@seven_seconds http://goo.gl/Z4Rq5
What We Want You to Do?
ask questions about your data
Mine your data
Ask follow-up questions
What are our obstacles?
Certain data points are transient

Logging will need to be created in certain situations to fill in missing data points.
See slide #45 & #46 for an example
What You Need

Permissions to run SQL scripts on your Blackboard Learn environment's database Basic SQL fundamentals Analysis of results Database Tools
Aquadata client & Server or Quests Toad
Skill and Knowledge Sets Open DB Documentation

Blackboard Learn, Release 9.1 Open Database Document
http://bit.ly/loW9hi
Asking the Right Questions
Collecting the Right Data
Organizing and Analyzing the Data
Making Decisions
Ask Follow-up Questions
Types of Data
Data Types Event Process User Static Transient
Table
ACTIVITY_ACCUMULATOR
Example
User navigation
COURSE_USERS COURSE_CONTENT
LEARN_admin.bb_instance
snapshot
User created entities
installation
Tracks who has an active session
SESSIONS
Areas of Capacity Planning

How can the Blackboard Learn schema help you capacity plan for your business requirements in regards to?
Administrative Planning Expense and Budget Management
Strategic Functional Decisions
Administrative Planning
To be able to restore individual courses, can we use the course archive tool as a backup function?
How will we know which courses need archiving? Frequency of course archive task? Will the archiving process impact user experience? Does the course archive tool have limitation? Location of course archives?
How will we know which courses need archiving?

Course entity creation and modification
Creation Tables
COURSE_CONTENTS GRADEBOOK_MAIN GRADEBOOK_GRADES QTI_ASI_DATA QTI_RESULT_DATA FORUM_MAIN MSG_MAIN
Modification Tables
COURSE_CONTENTS GRADEBOOK_MAIN GRADEBOOK_GRADES QTI_ASI_DATA QTI_RESULT_DATA FORUM_MAIN MSG_MAIN CALENDAR STAFFINFORMATION
Data Points: Course/Organizations to be archived SQL/Oracle

COURSE_ID Hist_101 Eng_252 Sci_139 Eng_429 Art_202 Eco_323 Number of New/Modified Course Entities 42 42 38 37 35 35
Making Decisions from the Data
How many new or modified course/organization entities are required for a course/organization to be archived? How many courses/organizations should we archive each time the archive task is execution? What is the maximum time between archive tasks?
Acting on Your Descion SQL/Oracle Hist_101, c:\Course_Archives\2011-07-09\ Eng_252, c:\Course_Archives\2011-07-09\ Sci_139, c:\Course_Archives\2011-07-09\ Eng_429, c:\Course_Archives\2011-07-09\ Create a JOB that would run the SQL script to generate a txt file containing the query results Create a schedule task that would run the batch_ImportExport.bat tool and direct it to the txt file created by the JOB in the previous step.
Expense and Budget Management
Asking the Right Questions Expense and Budget Management

How much do we need to budget for in order to fund our database requirements?

When are we going to hit our 95% disk spaced used threshold.

Available disk space issue
Talk to Network Security Manager (owner of the SAN)
We need more disk space

What did I miss?

When will we run out of space? How long do we expect the requested disk space to last us?
Facts vs. guess work

Collecting the Right Data Data Points: Database file size
Binary Data Files (.mdf) TempDB Transaction Logs (.ldf) Back-up/restore process

Organizing and Analyzing the Data Data Points: Database file size SQL/Oracle
Database Name BBLEARN_stats BBLEARN cms_files_inst BBLEARN cms_files_inst cms_files_courses cms_files_users msdb cms_files_library cms_files_orgs msdb cms_files_users cms_files_courses Physical Name F:\data\BBLEARN_stats_data.mdf F:\data\BBLEARN_data.mdf F:\data\cms_files_inst_data.mdf G:\logs\BBLEARN_log.ldf G:\logs\cms_files_inst_log.ldf F:\data\cms_files_courses_data.mdf F:\data\cms_files_users_data.mdf F:\data\MSSQL.1\MSSQL\DATA\MSDBData.mdf F:\data\cms_files_library_data.mdf F:\data\cms_files_orgs_data.mdf F:\data\MSSQL.1\MSSQL\DATA\MSDBLog.ldf G:\logs\cms_files_users_log.ldf G:\logs\cms_files_courses_log.ldf Size in MB 70338 45520 5704 3083 1361 1072 926 792 720 384 259 214 207

Organizing and Analyzing the Data Data Points: Table Row Count and Sizes SQL/Oracle
Table Name
ACTIVITY_ACCUMULATOR COURSE_USERS USERS COURSE_MAIN NAVIGATION_ITEM SYSTEM_TRACKING USER_ROLES
Row Count Reserved

377266616 84097056 3837931 303687 121623 1876 3550 4114 496024 139480 86024 2032 1936 544
Data
46905888 258272 96848 47760 608 1672 160
Index Size Unused

37183888 237496 42440 37984 1128 160 288 7280 256 192 280 296 104 96

Organizing and Analyzing the Data Data Points: Table growth rate (by year) of the Activity_Accumulator SQL/Oracle
120000000
100000000
80000000
60000000
40000000
20000000
0 2003 2004 2005 2006 2007 2008 2009 2010

Organizing and Analyzing the Data Data Points: ACTIVITY_ACCUMULATOR growth rate (by month) SQL/Oracle
12000000 10000000
8000000
6000000
4000000
2000000

Data Points: COURSE_USERS table growth rate (by month)
900000 800000 700000 600000 500000 400000 300000 200000 100000 0 August August August

Conclusion
Due to opening of school tasks we will see our available disk space decrease below 5% during the month of August. In order to make it 3 years we will need a minimum 120 GB.

Can our database support all classroom testing if it was online?

Study
Online & offline test taking populations The tests themselves Test taking behavior How testing impacts the database Locating when you will see resource contention

Historical Use of Online Testing What is the percent of students from the previous term that have completed a test online? SQL/Oracle
Online Offline

Previous Use Courses with Tests Enrollments with Tests Number of questions submitted in a term Peak Question Submission by hour 11,274 225,480 4,219,032 60,346 Full Adoption 55,504 1,110,080 25,694,266 297,094

Determine CPU and I/O cost of testing (SQL server)
Track the Stored Procedures performing the works SQL/Oracle
qti_result_data_cr update_gradebook_grade attempt_cr update_qti_res_bbmd_dtmod gb2_is_log_enabled gb2_attempt_logger

Track the Stored Procedures performing the works SQL/Oracle

Determine CPU and I/O cost of testing
Track the queries performing the work SQL/Oracle

Determine IO cost of all cached stored procedures SQL/Oracle Average IO
700 600 500 400 300 200 100 0

A reasonable understanding
How tests are currently used How tests will be used How much each question costs the database upon submission Baseline of databases IO and CPU resources Non-testing database transactions
Lack information on


Collecting the Right Data Determine the breaking point of CPU and I/O of testing
Create a CPU logging script SQL/Oracle
SQL server logging limitation
4 hours of CPU tracking Lacks IO counters
Create a IO logging script SQL/Oracle

Snapshot in time
Need to log the delta between the last logged IO metric and current query results
Create a Query logging script SQL/Oracle

Compares test question submission to all other transactions

Create a separate database from the Blackboard Learns databases (you can use the database server Blackboard Learn is running on) Create tables with in the new database that match the query results you wish to store Create a JOB that will run each query and insert the results into the corresponding table and database.
Capacity Planning is an art if done properly
or a disaster if not done at all.
Geoff.Mower@blackboard.com
@gvmower
@seven_seconds sevenseconds.wordpress.com
stephen.feldman@blackboard.com
http://goo.gl/Z4Rq5
Please provide feedback for this session by emailing DevConFeedback@blackboard.com.
The title of this session is: Using Database Statistics to Make Capacity Planning Decisions
Appendices
What courses meet the course archive criteria
declare @Last_Archive datetime declare @Datediff int declare @Number_of_New_Entities int set @Last_Archive = '2011-6-7' --Enter the last datetime the course archival process ran set @Datediff = 24 --Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours. set @Number_of_New_Entities = 1 --Enter the number of new course entities required for a course to be archived select course_id, CC.CC + GM.GM + GG.GG + QA.QA + A.A + ConM.CONFERENCE_MAIN + FM.FORUM_MAIN + MM.MSG_MAIN + C.C + S.S AS [Number of New/Modified Course Entities] from course_main cm left join (select crsmain_pk1, COUNT(cc.pk1) CC from course_contents cc left join course_main cm on cm.pk1 = cc.crsmain_pk1 where (cc.dtcreated > @Last_Archive or cc.dtmodified > @Last_Archive) and datediff(hour, cm.dtcreated, cc.dtcreated) > @Datediff group by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(gm.pk1) GM from gradebook_main gm left join course_main cm on cm.pk1 = gm.crsmain_pk1 where (date_added > @Last_Archive or date_modified > @Last_Archive) and datediff(hour, cm.dtcreated, gm.date_added) > @Datediff group by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(gg.pk1) GG from gradebook_main gm left join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1 left join course_main cm on gm.crsmain_pk1 = cm.pk1 where (gg.date_added > @Last_Archive OR gg.date_modified > @Last_Archive) and datediff(hour, cm.dtcreated, gg.date_added) > @Datediff group by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(qa.pk1) QA from qti_asi_data qa left join course_main cm on cm.pk1 = qa.crsmain_pk1 where (bbmd_date_added > @Last_Archive OR bbmd_date_modified > @Last_Archive) and datediff(hour, cm.dtcreated, qa.bbmd_date_added) > @Datediff group by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(a.pk1) A from attempt a left join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1 left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1 left join course_main cm on cm.pk1 = gm.crsmain_pk1 where (a.attempt_date > @Last_Archive or a.date_modified > @Last_Archive) and datediff(hour, cm.dtcreated, a.attempt_date) > @Datediff group by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1 left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAIN from conference_owner CO left join conference_main CON on CON.conference_owner_pk1 = CO.pk1 left join course_main cm on cm.pk1 = co.owner_pk1 where co.owner_table = 'COURSE_MAIN' and (CON.dtcreated > @Last_Archive OR CON.dtmodified > @Last_Archive) and datediff(hour, cm.dtcreated, con.dtcreated) > @Datediff group by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1 left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAIN from conference_owner CO left join conference_main CON on CON.conference_owner_pk1 = CO.pk1 left join forum_main fr on fr.confmain_pk1 = CON.pk1 left join course_main cm on cm.pk1 = co.owner_pk1 where owner_table = 'COURSE_MAIN' and (fr.dtcreated > @Last_Archive OR fr.dtmodified > @Last_Archive) and datediff(hour, cm.dtcreated, fr.dtcreated) > @Datediff group by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1 left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAIN from conference_owner CO left join conference_main CON on CON.conference_owner_pk1 = CO.pk1 left join forum_main fr on fr.confmain_pk1 = CON.pk1 left join msg_main mm on mm.forummain_pk1 = fr.pk1 left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1 left join course_main cm on cm.pk1 = co.owner_pk1 where owner_table = 'COURSE_MAIN' and (mm.dtcreated > @Last_Archive OR mm.dtmodified > @Last_Archive) and datediff(hour, cm.dtcreated, mm.dtcreated) > @Datediff group by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1 left join (select crsmain_pk1, COUNT(c.pk1) C from calendar c left join course_main cm on cm.pk1 = c.crsmain_pk1 where c.dtmodified > @Last_Archive and datediff(hour, cm.dtcreated, c.dtmodified) > @Datediff group by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(s.pk1) S from staffinformation S left join course_main cm on cm.pk1 = s.crsmain_pk1 where s.dtmodified > @Last_Archive and datediff(hour, cm.dtcreated, s.dtmodified) > @Datediff group by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1
set serveroutput on size 100000 ---Administrative Planning What courses meet the course archive criteria declare v_Last_Archive date; v_Datediff int; v_Number_of_New_Entities int; v_Directory varchar(256); v_File_Name varchar(256); Begin v_Last_Archive := TO_DATE('7-JUN-2011'); --Enter the last datetime the course archival process ran v_Datediff := 1 ; --this is the day --Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created ----since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours. v_Number_of_New_Enti ties := 1; --Enter the number of new course entities required for a course to be archived select 'usr\local\blackboard\Course_Archi ves \' | | TO_CHAR(SYSDATE, 'DD-Mon-YYYY') | | '\' into v_Directory FROM Dual; --Alter the string to build the directory you wish to store your archives dbms_output.enable(9999999); dbms_output.new_line();
FOR rec IN ( select course_id | | v_Directory as output from course_main cm left join (select crsmain_pk1, COUNT(cc.pk1) CC from course_contents cc left join course_main cm on cm.pk1 = cc.crsmain_pk1 where (cc.dtcreated > v_Last_Archive or cc.dtmodified > v_Last_Archive) and cm.dtcreated- cc.dtcreated> v_Datediff group by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(gm.pk1) GM from gradebook_main gm left join course_main cm on cm.pk1 = gm.crsmain_pk1 where (date_added > v_Last_Archive or date_modified > v_Last_Archive) and cm.dtcreated- gm.date_added> v_Datediff group by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(gg.pk1) GG from gradebook_main gm left join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1 left join course_main cm on gm.crsmain_pk1 = cm.pk1 where (gg.date_added > v_Last_Archive OR gg.date_modified > v_Last_Archive) and cm.dtcreated- gg.date_added> v_Datediff group by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(qa.pk1) QA from qti_asi_data qa left join course_main cm on cm.pk1 = qa.crsmain_pk1 where (bbmd_date_added > v_Last_Archive OR bbmd_date_modified > v_Last_Archive) and cm.dtcreated- qa.bbmd_date_added> v_Datediff group by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(a.pk1) A from attempt a left join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1 left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1 left join course_main cm on cm.pk1 = gm.crsmain_pk1 where (a.attempt_date > v_Last_Archive or a.date_modified > v_Last_Archive) and cm.dtcreated- a.attempt_date > v_Datediff group by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1 left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAIN from conference_owner CO left join conference_main CON on CON.conference_owner_pk1 = CO.pk1 left join course_main cm on cm.pk1 = co.owner_pk1 where co.owner_table = 'COURSE_MAIN' and (CON.dtcreated > v_Last_Archive OR CON.dtmodified > v_Last_Archive) and cm.dtcreated-con.dtcreated > v_Datediff group by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1 left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAIN from conference_owner CO left join conference_main CON on CON.conference_owner_pk1 = CO.pk1 left join forum_main fr on fr.confmain_pk1 = CON.pk1 left join course_main cm on cm.pk1 = co.owner_pk1 where owner_table = 'COURSE_MAIN' and (fr.dtcreated > v_Last_Archive OR fr.dtmodified > v_Last_Archive) and cm.dtcreated- fr.dtcreated > v_Datediff group by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1 left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAIN from conference_owner CO left join conference_main CON on CON.conference_owner_pk1 = CO.pk1 left join forum_main fr on fr.confmain_pk1 = CON.pk1 left join msg_main mm on mm.forummain_pk 1 = fr.pk1 left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1 left join course_main cm on cm.pk1 = co.owner_pk1 where owner_table = 'COURSE_MAIN' and (mm.dtcreated > v_Last_Archive OR mm.dtmodifi ed > v_Last_Archive) and cm.dtcreated- mm.dtcreated > v_Datediff group by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1 left join (select crsmain_pk1, COUNT(c.pk1) C from calendar c left join course_main cm on cm.pk1 = c.crsmain_pk1 where c.dtmodified > v_Last_Archive and cm.dtcreated- c.dtmodified > v_Datediff group by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(s.pk1) S from staffinformation S left join course_main cm on cm.pk1 = s.crsmain_pk1 where s.dtmodified > v_Last_Archive and cm.dtcreated- s.dtmodified > v_Datediff group by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1 where nvl(CC.CC,0) + nvl(GM.GM,0) + nvl(GG.GG,0) + nvl(QA.QA,0) + nvl(A.A,0) + nvl(ConM.CONFERENCE_MAIN,0) + nvl(FM.FORUM_MAIN,0) + nvl(MM.MSG_MAIN,0) + nvl(S.S,0) + nvl(C.C,0) >= v_Number_of_New_Entities ) LOOP dbms_output.put_line (rec.output); END LOOP; dbms_output.new_line();
end;
declare @Last_Archive datetime declare @Datediff int declare @Number_of_New_Entities int declare @Directory varchar(256) declare @File_Name varchar(256) set @Last_Archive = '2011-6-7' --Enter the last datetime the course archival process ran set @Datediff = 24 --Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours. set @Number_of_New_Entities = 1 --Enter the number of new course entities required for a course to be archived set @Directory = (SELECT ', c:\Course_Archives\' + CAST(CONVERT(date, GETDATE()) as varchar(10)) + '\') --Alter the string to build the directory you wish to store your archives select course_id + @Directory from course_main cm left join (select crsmain_pk1, COUNT(cc.pk1) CC from course_contents cc left join course_main cm on cm.pk1 = cc.crsmain_pk1 where (cc.dtcreated > @Last_Archive or cc.dtmodified > @Last_Archive) and datediff(hour, cm.dtcreated, cc.dtcreated) > @Datediff group by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(gm.pk1) GM from gradebook_main gm left join course_main cm on cm.pk1 = gm.crsmain_pk1 where (date_added > @Last_Archive or date_modified > @Last_Archive) and datediff(hour, cm.dtcreated, gm.date_added) > @Datediff group by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(gg.pk1) GG from gradebook_main gm left join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1 left join course_main cm on gm.crsmain_pk1 = cm.pk1 where (gg.date_added > @Last_Archive OR gg.date_modified > @Last_Archive) and datediff(hour, cm.dtcreated, gg.date_added) > @Datediff group by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(qa.pk1) QA from qti_asi_data qa left join course_main cm on cm.pk1 = qa.crsmain_pk1 where (bbmd_date_added > @Last_Archive OR bbmd_date_modified > @Last_Archive) and datediff(hour, cm.dtcreated, qa.bbmd_date_added) > @Datediff group by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(a.pk1) A from attempt a left join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1 left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1 left join course_main cm on cm.pk1 = gm.crsmain_pk1 where (a.attempt_date > @Last_Archive or a.date_modified > @Last_Archive) and datediff(hour, cm.dtcreated, a.attempt_date) > @Datediff group by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1 left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAIN from conference_owner CO left join conference_main CON on CON.conference_owner_pk1 = CO.pk1 left join course_main cm on cm.pk1 = co.owner_pk1 where co.owner_table = 'COURSE_MAIN' and (CON.dtcreated > @Last_Archive OR CON.dtmodified > @Last_Archive) and datediff(hour, cm.dtcreated, con.dtcreated) > @Datediff group by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1 left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAIN from conference_owner CO left join conference_main CON on CON.conference_owner_pk1 = CO.pk1 left join forum_main fr on fr.confmain_pk1 = CON.pk1 left join course_main cm on cm.pk1 = co.owner_pk1 where owner_table = 'COURSE_MAIN' and (fr.dtcreated > @Last_Archive OR fr.dtmodified > @Last_Archive) and datediff(hour, cm.dtcreated, fr.dtcreated) > @Datediff group by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1 left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAIN from conference_owner CO left join conference_main CON on CON.conference_owner_pk1 = CO.pk1 left join forum_main fr on fr.confmain_pk1 = CON.pk1 left join msg_main mm on mm.forummain_pk1 = fr.pk1 left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1 left join course_main cm on cm.pk1 = co.owner_pk1 where owner_table = 'COURSE_MAIN' and (mm.dtcreated > @Last_Archive OR mm.dtmodified > @Last_Archive) and datediff(hour, cm.dtcreated, mm.dtcreated) > @Datediff group by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1 left join (select crsmain_pk1, COUNT(c.pk1) C from calendar c left join course_main cm on cm.pk1 = c.crsmain_pk1 where c.dtmodified > @Last_Archive and datediff(hour, cm.dtcreated, c.dtmodified) > @Datediff group by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(s.pk1) S from staffinformation S left join course_main cm on cm.pk1 = s.crsmain_pk1 where s.dtmodified > @Last_Archive and datediff(hour, cm.dtcreated, s.dtmodified) > @Datediff group by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1 where ISNULL(CC.CC,0) + ISNULL(GM.GM,0) + ISNULL(GG.GG,0) + ISNULL(QA.QA,0) + ISNULL(A.A,0) + ISNULL(ConM.CONFERENCE_MAIN,0) + ISNULL(FM.FORUM_MAIN,0) + ISNULL(MM.MSG_MAIN,0) + ISNULL(S.S,0) + ISNULL(C.C,0) >= @Number_of_New_Entities
set serveroutput on size 100000 ---Administrative Planning What courses meet the course archive criteria declare v_Last_Archive date; v_Datediff int; v_Number_of_New_Entities int; v_Directory varchar(256); v_File_Name varchar(256); Begin v_Last_Archive := TO_DATE('7-JUN-2011'); --Enter the last datetime the course archival process ran v_Datediff := 1 ; --this is the day --Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created ----since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours. v_Number_of_New_Enti ties := 1; --Enter the number of new course entities required for a course to be archived select 'usr\local\blackboard\Course_Archi ves \' | | TO_CHAR(SYSDATE, 'DD-Mon-YYYY') | | '\' into v_Directory FROM Dual; --Alter the string to build the directory you wish to store your archives dbms_output.enable(9999999); dbms_output.new_line();
FOR rec IN ( select course_id | | v_Directory as output from course_main cm left join (select crsmain_pk1, COUNT(cc.pk1) CC from course_contents cc left join course_main cm on cm.pk1 = cc.crsmain_pk1 where (cc.dtcreated > v_Last_Archive or cc.dtmodified > v_Last_Archive) and cm.dtcreated- cc.dtcreated> v_Datediff group by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(gm.pk1) GM from gradebook_main gm left join course_main cm on cm.pk1 = gm.crsmain_pk1 where (date_added > v_Last_Archive or date_modified > v_Last_Archive) and cm.dtcreated- gm.date_added> v_Datediff group by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(gg.pk1) GG from gradebook_main gm left join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1 left join course_main cm on gm.crsmain_pk1 = cm.pk1 where (gg.date_added > v_Last_Archive OR gg.date_modified > v_Last_Archive) and cm.dtcreated- gg.date_added> v_Datediff group by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(qa.pk1) QA from qti_asi_data qa left join course_main cm on cm.pk1 = qa.crsmain_pk1 where (bbmd_date_added > v_Last_Archive OR bbmd_date_modified > v_Last_Archive) and cm.dtcreated- qa.bbmd_date_added> v_Datediff group by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(a.pk1) A from attempt a left join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1 left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1 left join course_main cm on cm.pk1 = gm.crsmain_pk1 where (a.attempt_date > v_Last_Archive or a.date_modified > v_Last_Archive) and cm.dtcreated- a.attempt_date > v_Datediff group by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1 left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAIN from conference_owner CO left join conference_main CON on CON.conference_owner_pk1 = CO.pk1 left join course_main cm on cm.pk1 = co.owner_pk1 where co.owner_table = 'COURSE_MAIN' and (CON.dtcreated > v_Last_Archive OR CON.dtmodified > v_Last_Archive) and cm.dtcreated-con.dtcreated > v_Datediff group by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1 left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAIN from conference_owner CO left join conference_main CON on CON.conference_owner_pk1 = CO.pk1 left join forum_main fr on fr.confmain_pk1 = CON.pk1 left join course_main cm on cm.pk1 = co.owner_pk1 where owner_table = 'COURSE_MAIN' and (fr.dtcreated > v_Last_Archive OR fr.dtmodified > v_Last_Archive) and cm.dtcreated- fr.dtcreated > v_Datediff group by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1 left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAIN from conference_owner CO left join conference_main CON on CON.conference_owner_pk1 = CO.pk1 left join forum_main fr on fr.confmain_pk1 = CON.pk1 left join msg_main mm on mm.forummain_pk 1 = fr.pk1 left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1 left join course_main cm on cm.pk1 = co.owner_pk1 where owner_table = 'COURSE_MAIN' and (mm.dtcreated > v_Last_Archive OR mm.dtmodifi ed > v_Last_Archive) and cm.dtcreated- mm.dtcreated > v_Datediff group by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1 left join (select crsmain_pk1, COUNT(c.pk1) C from calendar c left join course_main cm on cm.pk1 = c.crsmain_pk1 where c.dtmodified > v_Last_Archive and cm.dtcreated- c.dtmodified > v_Datediff group by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1 left join (select crsmain_pk1, COUNT(s.pk1) S from staffinformation S left join course_main cm on cm.pk1 = s.crsmain_pk1 where s.dtmodified > v_Last_Archive and cm.dtcreated- s.dtmodified > v_Datediff group by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1 where nvl(CC.CC,0) + nvl(GM.GM,0) + nvl(GG.GG,0) + nvl(QA.QA,0) + nvl(A.A,0) + nvl(ConM.CONFERENCE_MAIN,0) + nvl(FM.FORUM_MAIN,0) + nvl(MM.MSG_MAIN,0) + nvl(S.S,0) + nvl(C.C,0) >= v_Number_of_New_Entities ) LOOP dbms_output.put_line (rec.output); END LOOP; dbms_output.new_line();
end;
Expense and Budget Management Database File Size

SELECT DB_NAME(database_id) AS DatabaseName, Name AS Logical_Name, Physical_Name, (size*8)/1024 SizeMB FROM sys.master_files order by SizeMB desc
Expense and Budget Management Database File Size

select (select sum(bytes)/1024/1024 from dba_data_files)+ (select sum(bytes)/1024/1024 from dba_temp_files) "Size in MB" from dual;
Expense and Budget Management Table Row Count and Sizes

select IDENTITY( int ) AS pk1, name into Table_List from sys.tables CREATE TABLE Table_Results (name nvarchar(256), rows int, reserved nvarchar(256), data nvarchar(256), index_size nvarchar(256), unused nvarchar(256)) declare @pk1 int declare @table nvarchar(256) declare @end int set @pk1 = 0 set @end = (select top 1 PK1 from Table_List order by pk1 Desc) WHILE @pk1 < @end begin set @pk1 = @pk1 + 1 set @table = (select name from Table_List where pk1 = @pk1) insert into Table_Results exec sp_spaceused @table end select * from table_results drop table table_list drop table table_results
Expense and Budget Management Table Row Count and Sizes

Select OWNER, TABLE_NAME, TABLESPACE_NAME, NUM_ROWS, PCT_FREE , PCT_USED , (blocks*8/1024,2) as MB from dba_tables order by OWNER, TABLESPACE_NAME, TABLE_NAME;
Expense and Budget Management Table growth rate (by year)

/* depending on the table, the dtcreated could need to be changed to timestamp */ select YEAR(dtcreated), COUNT(*) from course_contents --Enter the table name here group by YEAR(dtcreated)
Expense and Budget Management Table growth rate (by year)

select TO_char(dtcreated,'YYYY') , COUNT(*) from course_contents --Enter the table name here group by TO_char(dtcreated,'YYYY') ;
Expense and Budget Management Table growth rate (by month)

/* depending on the table, the dtcreated could need to be changed to timestamp */ select YEAR(dtcreated), MONTH(dtcreated), COUNT(*) from course_main --Enter the table name here group by YEAR(dtcreated), MONTH(dtcreated)
Expense and Budget Management Table growth rate (by month)

select TO_char(dtcreated,'YYYY'), TO_char(dtcreated,'MM') , COUNT(*) from course_contents --Enter the table name here group by TO_char(dtcreated,'YYYY'), TO_char(dtcreated,'MM') order by TO_char(dtcreated,'YYYY'), TO_char(dtcreated,'MM') ;

What is the percent of enrollments from the previous term that have completed a test online?
declare @start_date datetime declare @end_date datetime declare @TotalPopulation int declare @TestingPopulation int set @start_date = '2010-9-1 00:00:00:000' set @end_date = '2011-9-1 00:00:00:000' set @TotalPopulation = (select Count(distinct users_pk1) from course_users cu left join course_main cm on cm.pk1 = cu.crsmain_pk1 where role = 'S' and dtcreated > @start_date and dtcreated < @end_date) set @TestingPopulation = (select Count(Distinct cu.users_pk1) from course_users cu left join gradebook_grade gg on gg.course_users_pk1 = cu.pk1 left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1 left join course_main cm on cm.pk1 = cu.crsmain_pk1 where gm.qti_asi_data_pk1 > 0 and gg.first_attempt_pk1 > 0 and dtcreated > @start_date and dtcreated < @end_date)
select @TestingPopulation as Online_Testing, @TotalPopulation - @TestingPopulation as Offline_Testing

What is the percent of enrollments from the previous term that have completed a test online?
set serveroutput on size 100000 declare v_start_date date; v_end_date date; v_TotalPopulation int; v_TestingPopulation int; Begin dbms_output.enable(9999999); dbms_output.new_line(); v_start_date := to_date('1-JAN-2010'); v_end_date := to_date('1-JAN-2012'); select Count(distinct users_pk1) into v_TotalPopulation from course_users cu left join course_main cm on cm.pk1 = cu.crsmain_pk1 where role = 'S' and dtcreated > v_start_date and dtcreated < v_end_date; select Count(Distinct cu.users_pk1) into v_TestingPopulation from course_users cu left join gradebook_grade gg on gg.course_users_pk1 = cu.pk1 left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1 left join course_main cm on cm.pk1 = cu.crsmain_pk1 where gm.qti_asi_data_pk1 > 0 and gg.first_attempt_pk1 > 0 and dtcreated > v_start_date and dtcreated < v_end_date; dbms_output.put_line (v_TestingPopulation || ',' || to_char(v_TotalPopulation - v_TestingPopulation)); --select v_TestingPopulation as Online_Testing, v_TotalPopulation - v_TestingPopulation as Offline_Testing; end;

How many tests have been completed per enrollment?
declare @enrollment_start datetime declare @enrollment_end datetime set @enrollment_start = '2010-9-1 00:00:00:000' set @enrollment_end = '2011-9-1 00:00:00:000' select NumberOfTests, COUNT(NumberOfTests) NumberOfEnrollments into #BinTests from ( select AllStudents.users_pk1, NumberOfTests = Case When TestingStudents.TheCount IS NULL THEN 0 When TestingStudents.TheCount > 0 THEN TestingStudents.TheCount END FROM ( select distinct users_pk1 from course_users where role = 'S' and enrollment_date > @enrollment_start and enrollment_date < @enrollment_end) AllStudents left join( select cu.users_pk1, COUNT(gg.pk1) as TheCount from course_users cu left join gradebook_grade gg on gg.course_users_pk1 = cu.pk1 left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1 where gm.qti_asi_data_pk1 > 0 and gg.first_attempt_pk1 > 0 and cu.enrollment_date > @enrollment_start and cu.enrollment_date < @enrollment_end group by cu.users_pk1) TestingStudents on AllStudents.users_pk1 = TestingStudents.users_pk1) TestBin group by NumberOfTests
declare @start int declare @end int declare @enrollments int set @start = 0 set @end = (Select top 1 NumberOfTests from #BinTests order by NumberOfTests desc) create table #Results( Number_Of_Tests int, Number_Of_Enrollments int)
While @start-1 < @end Begin
set @enrollments = (select NumberOfEnrollments from #BinTests where NumberOfTests = @start) insert into #Results (Number_of_Tests, Number_Of_Enrollments) values (@start, @enrollments) set @start = @start + 1 end
select Number_Of_Tests, Number_Of_Enrollments = Case When Number_Of_Enrollments IS NULL THEN 0 When Number_Of_Enrollments >= 0 THEN Number_Of_Enrollments End from #Results drop table #BinTests drop table #Results

How many tests have been completed per enrollment?
set serveroutput on size 100000 declare v_enrollment_start date; v_enrollment_end date; sql_stmt varchar(8000); v_start int; v_end int; v_enrollments int; Begin dbms_output.enable(9999999); dbms_output.new_line(); v_enrollment_start := to_date('2010-9-1 00:00:00', 'yyyy- mm-dd HH24:MI:SS'); v_enrollment_end := to_date('2011-9-1 00:00:00', 'yyyy- mm-dd HH24:MI:SS'); --sql_stmt := EXECUTE IMMEDIATE ' drop table BinTests'; sql_stmt :='' | | 'create table ' | | 'BinTests as ' | | 'select NumberOfTests , COUNT(NumberOfTests ) as NumberOfEnroll ments ' | | 'from ( ' || ' select AllStudents.users_pk1, nvl(TestingStudents.TheCount,0) NumberOfTests' || ' FROM ( ' || ' select distinct users_pk1 ' || ' from course_users ' || ' where role = ''S'' ' || ' and enrollment_date > ' | | 'to_date('''| | v_enrollment_start | | ''')' || ' and enrollment_date < ' | | 'to_date('''| | v_enrollment_end | | ''')' | | ' ) AllStudents ' || ' left join( ' || ' select cu.users_pk1, COUNT(gg.pk1) as TheCount ' || ' from course_users cu ' || ' left join gradebook_grade gg on gg.course_users_pk1 = cu.pk1 ' || ' left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1 ' || ' where gm.qti_asi_data_pk1 > 0 ' || ' and gg.first_attempt_pk1 > 0 ' || ' and cu.enrollment_date > ' | | 'to_date('''| | v_enrollment_start | | ''')' || ' and cu.enrollment_date < ' | | 'to_date('''| | v_enrollment_end | | ''')' || ' group by cu.users_pk1) TestingStudents ' || ' on AllStudents.users_pk1 = TestingStudents.users_pk1) TestBin ' | | 'group by NumberOfTests' ; dbms_output.put_line (sql_stmt ); EXECUTE IMMEDIATE sql_stmt ; EXCEPTION WHEN OTHERS THEN NULL; End; /
set serveroutput on size 100000
declare v_enrollment_start date; v_enrollment_end date; sql_stmt varchar(8000); v_start int; v_end int; v_enrollments int; Begin dbms_output.enable(9999999); dbms_output.new_line(); v_enrollment_start := to_date('2010-9-1 00:00:00', 'yyyy- mm-dd HH24:MI:SS'); v_enrollment_end := to_date('2011-9-1 00:00:00', 'yyyy- mm-dd HH24:MI:SS');
v_start := 0; Select NumberOfTests into v_end from (select NumberOfTests from BinTests order by NumberOfTests desc) WHERE ROWNUM =1; EXECUTE IMMEDIATE ' drop table Results'; EXECUTE IMMEDIATE ' create table Results ( Number_Of_Tests int, Number_Of_Enroll ments int)'; EXCEPTION WHEN OTHERS THEN NULL; END; /
set serveroutput on size 100000
declare v_enrollment_start date; v_enrollment_end date; sql_stmt varchar(8000); v_start int; v_end int; v_enrollments int; Begin dbms_output.enable(9999999); dbms_output.new_line(); v_enrollment_start := to_date('2010-9-1 00:00:00', 'yyyy- mm-dd HH24:MI:SS'); v_enrollment_end := to_date('2011-9-1 00:00:00', 'yyyy- mm-dd HH24:MI:SS');
While v_start-1 < v_end loop
select NumberOfEnroll ments into v_enrollments insert into Results (Number_of_Tests, Number_Of_Enroll ments) values ( v_start, v_enrollments ); v_start := v_start + 1; end loop;
from BinTests where NumberOfTests = v_start;
dbms_output.new_line(); FOR rec IN ( select Number_Of_Tests, nvl(Number_Of_Enroll ments,0) Number_Of_Enroll ments from Results)
LOOP dbms_output.put_line (rec.Number_Of_Tests | | ',' | | rec.Number_Of_Enrollments); END LOOP; dbms_output.new_line(); EXCEPTION WHEN OTHERS THEN NULL; --drop table BinTests --drop table Results END;

How many test questions were submitted at our peak hour?
SELECT count(pk1) TheCount, cast(DATEPART(YEAR, bbmd_date_added) as varchar(4)) + '-' + cast(DATEPART(MONTH, bbmd_date_added) as varchar(2)) from qti_result_data group by DATEPART(YEAR, bbmd_date_added), DATEPART(MONTH, bbmd_date_added) order by DATEPART(YEAR, bbmd_date_added), DATEPART(MONTH, bbmd_date_added)

SELECT count(pk1) TheCount, TO_char(bbmd_date_added,'YYYY') || '-' || TO_char(bbmd_date_added,'MM') from qti_result_data group by TO_char(bbmd_date_added,'YYYY'), TO_char(bbmd_date_added,'MM') order by TO_char(bbmd_date_added,'YYYY'), TO_char(bbmd_date_added,'MM');

SELECT top 25 count(pk1) TheCount, cast(DATEPART(YEAR, bbmd_date_added) as varchar(4)) + '-' + cast(DATEPART(MONTH, bbmd_date_added) as varchar(2)) + '-' + cast(DATEPART(DAY, bbmd_date_added) as varchar(2)) + ' ' + cast(DATEPART(HOUR, bbmd_date_added) as varchar(2)) + ':00' from qti_result_data group by DATEPART(YEAR, bbmd_date_added), DATEPART(MONTH, bbmd_date_added), DATEPART(DAY, bbmd_date_added), DATEPART(HOUR, bbmd_date_added) order by count(pk1) desc

select * from (SELECT count(pk1) TheCount, TO_char(bbmd_date_added,'YYYY') || '-' || TO_char(bbmd_date_added,'MM') || '-' || TO_char(bbmd_date_added,'DD') || '-' || TO_char(bbmd_date_added,'HH') from qti_result_data group by TO_char(bbmd_date_added,'YYYY'), TO_char(bbmd_date_added,'MM'), TO_char(bbmd_date_added,'DD') , TO_char(bbmd_date_added,'HH') order by count(pk1) desc ) where rownum <26;

Test question submission by hour of the day?
declare @start_date datetime declare @end_date datetime set @start_date = '2010-9-1 00:00:00:000' set @end_date = '2011-9-1 00:00:00:000' SELECT count(pk1) TheCount, cast(DATEPART(hour, bbmd_date_added) as varchar(4)) TheHour from qti_result_data where bbmd_date_added > @start_date and bbmd_date_added < @end_date group by DATEPART(HOUR, bbmd_date_added) order by DATEPART(HOUR, bbmd_date_added)

Test question submission by hour of the day?
set serveroutput on size 100000 declare v_start_date date; v_end_date date; Begin v_start_date := to_date('2010-9-1 00:00:00', 'yyyy-mm-dd HH24:MI:SS'); v_end_date := to_date('2011-9-1 00:00:00', 'yyyy-mm-dd HH24:MI:SS'); dbms_output.enable(9999999); dbms_output.new_line();
FOR rec IN ( SELECT count(pk1) TheCount, TO_char(bbmd_date_added,'HH24') TheHour from qti_result_data where bbmd_date_added > v_start_date and bbmd_date_added < v_end_date group by TO_char(bbmd_date_added,'HH24') order by TO_char(bbmd_date_added,'HH24')) LOOP dbms_output.put_line (to_char(rec.TheCount)); END LOOP; dbms_output.new_line();
end;

Test question Submission by day of the week?
declare @start_date datetime declare @end_date datetime set @start_date = '2010-9-1 00:00:00:000' set @end_date = '2011-9-1 00:00:00:000' SELECT count(pk1) TheCount, TheDay = Case when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 1 then 'Sunday' when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 2 then 'Monday' when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 3 then 'Tuesday' when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 4 then 'Wednesday' when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 5 then 'Thursday' when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 6 then 'Friday' when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 7 then 'Saturday' end
from qti_result_data where bbmd_date_added > @start_date and bbmd_date_added < @end_date group by DATEPART(DW, bbmd_date_added) order by DATEPART(DW, bbmd_date_added) asc

Test question Submission by day of the week?
--v_start_date := '2010-9-1 00:00:00:000' --v_end_date := '2011-9-1 00:00:00:000'
SELECT count(pk1) TheCount, to_char(to_date (bbmd_date_added, 'yyyy-mm-dd HH24:MI:SS'),'DAY') from qti_result_data where bbmd_date_added > to_date('2010-9-1', 'yyyy-mm-dd') and bbmd_date_added < to_date('2011-9-1', 'yyyy-mm-dd') group by to_char(to_date (bbmd_date_added , 'yyyy-mm-dd HH24:MI:SS'),'DAY') order by to_char(to_date (bbmd_date_added , 'yyyy-mm-dd HH24:MI:SS'),'DAY') asc

SELECT p.name AS[SP_Name], ps.total_worker_time AS[Total_Worker_Time], ps.total_physical_reads AS[Total_Physical_Reads], ps.total_logical_reads, ps.total_logical_writes AS[Total_Logical_Writes], (ps.total_logical_reads +ps.total_logical_writes) /ps.execution_count AS[TOTAL_IO], ps.execution_count, ISNULL(ps.execution_count/DATEDIFF(Second,ps.cached_time,GETDATE()),0)AS[Calls/Second], ps.total_elapsed_time, ps.last_worker_time, ps.total_elapsed_time/ps.execution_count AS[avg_elapsed_time], ps.last_elapsed_time, ps.last_execution_time, ps.cached_time, qs.text Query FROM sys.procedures AS p, sys.dm_exec_procedure_stats AS ps cross apply sys.dm_exec_sql_text (ps.sql_handle) AS qs WHERE p.[object_id] =ps.[object_id] and ps.database_id =DB_ID() --and p.name = 'qti_result_data_cr' -- Enter the name of the stored procedure ORDER BY ps.last_worker_time DESC;

Typically, you would view the AWR data through Oracle Enterprise Manager screens or AWR reports. However, you can view the statistics with the following views: V$ACTIVE_SESSION_HISTORY This view displays active database session activity, sampled once every second. See "Active Session History (ASH)". V$ metric views provide metric data to track the performance of the system The metric views are organized into various groups, such as event, event class, system, session, service, file, and tablespace metrics. These groups are identified in the V$METRICGROUP view. DBA_HIST views The DBA_HIST views contain historical data stored in the database. This group of views includes: DBA_HIST_ACTIVE_SESS_HISTORY displays the history of the contents of the in-memory active session history for recent system activity. DBA_HIST_BASELINE displays information about the baselines captured on the system DBA_HIST_DATABASE_INSTANCE displays information about the database environment DBA_HIST_SNAPSHOT displays information on snapshots in the system DBA_HIST_SQL_PLAN displays the SQL execution plans DBA_HIST_WR_CONTROL displays the settings for controlling AWR

SELECT SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1) Query, qs.execution_count, qs.total_logical_reads, qs.last_logical_reads, qs.total_logical_writes, qs.last_logical_writes, (qs.total_logical_reads +qs.total_logical_writes) /qs.execution_count AS[AVG_IO], qs.total_worker_time 'Total_CPU_Time', qs.last_worker_time 'Last_CPU_Time', qs.total_elapsed_time/1000000 total_elapsed_time_in_S, qs.last_elapsed_time/1000000 last_elapsed_time_in_S, qs.last_execution_time, qs.creation_time, qp.query_plan FROM sys.dm_exec_query_stats qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qt CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp WHERE SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1) LIKE '%qti%' -- Insert the query you are interested in finding ORDER BY qs.last_execution_time DESC

Typically, you would view the AWR data through Oracle Enterprise Manager screens or AWR reports. However, you can view the statistics with the following views: V$ACTIVE_SESSION_HISTORY This view displays active database session activity, sampled once every second. See "Active Session History (ASH)". V$ metric views provide metric data to track the performance of the system The metric views are organized into various groups, such as event, event class, system, session, service, file, and tablespace metrics. These groups are identified in the V$METRICGROUP view. DBA_HIST views The DBA_HIST views contain historical data stored in the database. This group of views includes: DBA_HIST_ACTIVE_SESS_HISTORY displays the history of the contents of the in-memory active session history for recent system activity. DBA_HIST_BASELINE displays information about the baselines captured on the system DBA_HIST_DATABASE_INSTANCE displays information about the database environment DBA_HIST_SNAPSHOT displays information on snapshots in the system DBA_HIST_SQL_PLAN displays the SQL execution plans DBA_HIST_WR_CONTROL displays the settings for controlling AWR

Determine I/O cost of question submission (SQL server)
SELECT p.name AS[SP_Name], (ps.total_logical_reads +ps.total_logical_writes) /ps.execution_count AS AVG_IO FROM sys.procedures AS p, sys.dm_exec_procedure_stats AS ps cross apply sys.dm_exec_sql_text (ps.sql_handle) AS qs WHERE p.[object_id] =ps.[object_id] and ps.database_id =DB_ID() and p.name = 'qti_result_data_cr' ORDER BY (ps.total_logical_reads +ps.total_logical_writes) /ps.execution_count DESC;

Determine I/O cost of question submission (SQL server)
please look at AWR report sear qti_result_data_cr

Determine IO cost of all cached stored procedures
SELECT p.name AS[SP_Name], (ps.total_logical_reads +ps.total_logical_writes) /ps.execution_count AS AVG_IO FROM sys.procedures AS p, sys.dm_exec_procedure_stats AS ps cross apply sys.dm_exec_sql_text (ps.sql_handle) AS qs WHERE p.[object_id] =ps.[object_id] and ps.database_id =DB_ID() ORDER BY (ps.total_logical_reads +ps.total_logical_writes) /ps.execution_count DESC;

Determine IO cost of all cached stored procedures
--look at AWR report set pagesize 60; column executions format 999,999,999; column Mem_used format 999,999,999; SELECT SUBSTR(owner,1,10) Owner, SUBSTR(type,1,12) Type, SUBSTR(name,1,20) Name, executions, sharable_mem Mem_used, SUBSTR(kept||' ',1,4) "Kept?" FROM v$db_object_cache WHERE type in ('PROCEDURE','PACKAGE BODY','PACKAGE') ORDER BY executions desc; HERE IS THE OUTPUT OF MEMORY.SQL: SQL> @memory

Determine total IO cost of all cached stored procedures
SELECT p.name AS[SP_Name], (ps.total_logical_reads +ps.total_logical_writes) AS TOTAL_IO FROM sys.procedures AS p, sys.dm_exec_procedure_stats AS ps cross apply sys.dm_exec_sql_text (ps.sql_handle) AS qs WHERE p.[object_id] =ps.[object_id] and ps.database_id =DB_ID() ORDER BY (ps.total_logical_reads +ps.total_logical_writes) DESC;

Determine total IO cost of all cached stored procedures
AWR report

Query that compares the test taking activity cost vs. to other activity
select 'QTI_IO' as TYPE, SUM(TOTAL_LOGICAL_READS + TOTAL_LOGICAL_WRITES) IO FROM (SELECT SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1) Query, qs.execution_count, qs.total_logical_reads, qs.last_logical_reads, qs.total_logical_writes, qs.last_logical_writes, (qs.total_logical_reads +qs.total_logical_writes) /qs.execution_count AS[AVG_IO], qs.total_worker_time 'Total_CPU_Time', qs.last_worker_time 'Last_CPU_Time', qs.total_elapsed_time/1000000 total_elapsed_time_in_S, qs.last_elapsed_time/1000000 last_elapsed_time_in_S, qs.last_execution_time, qs.creation_time, qp.query_plan FROM sys.dm_exec_query_stats qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qt CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp WHERE SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1) LIKE '%qti%' -- Insert the query you are interested in finding and SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1) NOT LIKE '%sys.dm%' --and qs.last_execution_time + .005 > GETDATE() )QTI_IO UNION select 'NON_QTI_IO' as TYPE, SUM(TOTAL_LOGICAL_READS + TOTAL_LOGICAL_WRITES) IO FROM (SELECT SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1) Query, qs.execution_count, qs.total_logical_reads, qs.last_logical_reads, qs.total_logical_writes, qs.last_logical_writes, (qs.total_logical_reads +qs.total_logical_writes) /qs.execution_count AS[AVG_IO], qs.total_worker_time 'Total_CPU_Time', qs.last_worker_time 'Last_CPU_Time', qs.total_elapsed_time/1000000 total_elapsed_time_in_S, qs.last_elapsed_time/1000000 last_elapsed_time_in_S, qs.last_execution_time, qs.creation_time, qp.query_plan FROM sys.dm_exec_query_stats qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qt CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp WHERE SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1) NOT LIKE '%qti%' -- Insert the query you are interested in finding and SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1) NOT LIKE '%sys.dm%' --and qs.last_execution_time + .005 > GETDATE() )NON_QTI_IO

Query that compares the test taking activity cost vs. to other activity
AWR report and v$sql, sqlarea on qti_result_data_cr

Query that joins the QTI tables (bbmd_data_added) with the cost of the queries associated with the insert
declare @start_date datetime declare @end_date datetime declare @question_io int set @start_date = '2010-9-1 00:00:00:000' set @end_date = '2011-9-1 00:00:00:000' set @question_io = (SELECT ps.last_logical_reads + ps.last_logical_writes Current_IO FROM sys.procedures AS p, sys.dm_exec_procedure_stats AS ps cross apply sys.dm_exec_sql_text (ps.sql_handle) AS qs WHERE p.[object_id] =ps.[object_id] and ps.database_id =DB_ID() and p.name = 'qti_result_data_cr')
SELECT count(pk1) * @question_io Test_Question_IO, cast(DATEPART(YEAR, bbmd_date_added) as varchar(4)) + '-' + cast(DATEPART(MONTH, bbmd_date_added) as varchar(2)) + '-' + cast(DATEPART(DAY, bbmd_date_added) as varchar(2)) TheDate from qti_result_data where bbmd_date_added > @start_date and bbmd_date_added < @end_date group by DATEPART(YEAR, bbmd_date_added), DATEPART(MONTH, bbmd_date_added), DATEPART(DAY, bbmd_date_added) order by DATEPART(YEAR, bbmd_date_added), DATEPART(MONTH, bbmd_date_added), DATEPART(DAY, bbmd_date_added)

Query that joins the QTI tables (bbmd_data_added) with the cost of the queries associated with the insert
AWR report and v$sql, sqlarea on qti_result_data_cr

Create a logging script
declare @ts_now bigint declare @start int declare @end int declare @Start_time datetime declare @End_time datetime declare @Bb_Activity int declare @QTI_RESULTS_Activity int set @ts_now = (SELECT cpu_ticks/(cpu_ticks/ms_ticks)FROM sys.dm_os_sys_info);
SELECT TOP(256) SQLProcessUtilization AS [SQL_Server_Process_CPU_Utilization], SystemIdle AS [System_Idle_Process], 100 - SystemIdle - SQLProcessUtilization AS [Other_Process_CPU_Utilization], DATEADD(ms, -1 * (@ts_now - [timestamp]), GETDATE()) AS [start_Time], DATEADD(minute, 1, DATEADD(ms, -1 * (@ts_now - [timestamp]), GETDATE())) AS [end_Time], IDENTITY( int ) AS TheCount into #cpu FROM (SELECT record.value('(./Record/@id)[1]', 'int') AS record_id, record.value('(./Record/SchedulerMonitorEvent/SystemHealth/SystemIdle)[1]', 'int') AS [SystemIdle], record.value('(./Record/SchedulerMonitorEvent/SystemHealth/ProcessUtilization)[1]', 'int') AS [SQLProcessUtilization], [timestamp] FROM (SELECT [timestamp], CONVERT(xml, record) AS [record] FROM sys.dm_os_ring_buffers WHERE ring_buffer_type = N'RING_BUFFER_SCHEDULER_MONITOR' AND record LIKE N'%<SystemHealth>%') AS x) AS y ORDER BY record_id DESC
set @start = 0 set @end = (select MAX(TheCount) from #cpu) CREATE TABLE [dbo].[#cpu_Activity]( [Activity_Accumulator_Activity] [int] NULL, [QTI_RESULTS_Activity] [int] NULL, [SQL_Server_Process_CPU_Utilization] [int] NULL, [System_Idle_Process] [int] NULL, [Other_Process_CPU_Utilization] [int] NULL, [start_Time] [datetime] NULL, [end_Time] [datetime] NULL, [TheCount] [int] NULL)
WHILE @start < @end BEGIN SET @start = @start + 1 SET @Start_time = (select Start_Time from #cpu where TheCount = @start) SET @End_time = (select End_Time from #cpu where TheCount = @start) set @Bb_Activity = (select COUNT(*) from activity_accumulator where timestamp > @Start_time and timestamp < @End_time) set @QTI_RESULTS_Activity = (select COUNT(*) from qti_result_data where bbmd_date_modified > @Start_time and bbmd_date_modified < @End_time) Insert into #cpu_Activity SELECT @Bb_Activity, @QTI_RESULTS_Activity, * from #cpu where thecount = @start
end select * from #cpu_activity drop table #cpu; drop table #cpu_activity;

AWR report, v$sql select ... rom activity_accmultor where event_type ='login';

CREATE TABLE [dbo].[#FILE_IO]( [name] [sysname] NULL, [num_of_bytes_read] [bigint] NOT NULL, [num_of_bytes_written] [bigint] NOT NULL, [num_of_reads] [bigint] NOT NULL, [num_of_writes] [bigint] NOT NULL, [size_on_disk_bytes] [bigint] NOT NULL ) ON [PRIMARY] declare @start int declare @end int set @start = 5 set @end = (select top 1 database_id from sys.master_files where database_id > 5 order by database_id desc) while @start < @end begin insert into #FILE_IO select name, num_of_bytes_read, num_of_bytes_written, num_of_reads, num_of_writes, size_on_disk_bytes from sys.dm_io_virtual_file_stats(@start,NULL) s left join sys.master_files f on s.database_id = f.database_id and s.file_id = f.file_id set @start = @start + 1 end select * from #FILE_IO drop table #FILE_IO

AWR report, v$sql select ... rom activity_accmultor where event_type ='login';

SELECT 'TEST QUESTION' Type, SUM(qs.execution_count) Execution_Count, SUM(qs.total_logical_reads) Total_Reads, SUM(qs.total_logical_writes) Total_Writes, SUM(qs.total_worker_time) Total_CPU FROM sys.dm_exec_query_stats qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qt CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp WHERE SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1) LIKE '%INSERT INTO QTI_RESULT_DATA%' --Enter a portion of the query text you are interested in and SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1) NOT LIKE 'SELECT SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,%' UNION SELECT 'OTHER TRANSACTIONS' Type, SUM(qs.execution_count) Execution_Count, SUM(qs.total_logical_reads) Total_Reads, SUM(qs.total_logical_writes) Total_Writes, SUM(qs.total_worker_time) Total_CPU FROM sys.dm_exec_query_stats qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qt CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp WHERE SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1) NOT LIKE '%INSERT INTO QTI_RESULT_DATA%' --Enter a portion of the query text you are interested in and SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1) NOT LIKE 'SELECT SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,%'

AWR report, and use v$sql, sqlarea to find INSERT INTO QTI_RESULT_DATA%'

Using Database Statistics To Make Capacity Planning Decisions

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Using Database Statistics To Make Capacity Planning Decisions

Uploaded by

Copyright:

Available Formats

Using Database Statistics to Make Capacity Planning Decisions

Who Are We?

Geoff Mower- Senior Software Performance Engineer

2 hour interview on a plane

Performance and Scalability for Outcomes & Community

Who Are We?

What We Want You to Do?

ask questions about your data

Mine your data

Ask follow-up questions

What are our obstacles?

Certain data points are transient

What You Need

Skill and Knowledge Sets Open DB Documentation

Asking the Right Questions

Collecting the Right Data

Organizing and Analyzing the Data

Ask Follow-up Questions

Data Types Event Process User Static Transient

Areas of Capacity Planning

Administrative Planning Expense and Budget Management

Strategic Functional Decisions

How will we know which courses need archiving?

Data Points: Course/Organizations to be archived SQL/Oracle

Expense and Budget Management

Asking the Right Questions Expense and Budget Management

Expense and Budget Management

Expense and Budget Management

Expense and Budget Management

What did I miss?

Facts vs. guess work

Expense and Budget Management

Expense and Budget Management

Expense and Budget Management

Row Count Reserved

Index Size Unused

Expense and Budget Management

0 2003 2004 2005 2006 2007 2008 2009 2010

Expense and Budget Management

Expense and Budget Management

Expense and Budget Management

Strategic Functional Decisions

Strategic Functional Decisions

Strategic Functional Decisions

Strategic Functional Decisions

Strategic Functional Decisions

Strategic Functional Decisions

Strategic Functional Decisions

Strategic Functional Decisions

Strategic Functional Decisions

Strategic Functional Decisions

Strategic Functional Decisions

Create a IO logging script SQL/Oracle

Create a Query logging script SQL/Oracle

Strategic Functional Decisions

Capacity Planning is an art if done properly

or a disaster if not done at all.

Please provide feedback for this session by emailing DevConFeedback@blackboard.com.

Expense and Budget Management Database File Size

Expense and Budget Management Database File Size

Expense and Budget Management Table Row Count and Sizes

Expense and Budget Management Table Row Count and Sizes

Expense and Budget Management Table growth rate (by year)