1

Table of Contents
Foreword .................................................................................................................. 8
Preface ...................................................................................................................... 9
Acknowledgements ................................................................................................ 10
1. Introduction ..................................................................................................... 11
1.1 Book structure ............................................................................................... 11
1.2 The subset of best practice rules considered ................................................. 11
1.2.1 Design ............................................................................................ 11
1.2.2 Implementation ......................................................................... 12
1.2.3 Usage ......................................................................................... 12
1.2.4 Optimization .............................................................................. 12
1.3 Related work: domain presentation................................................... 13
1.4 Technologies used ......................................................................................... 15
1.4.1 Oracle 11g Database Server .......................................................... 16
1.4.2 Other Oracle tools: Enterprise Manager, SQL Developer, Client, ODBC
driver 18
1.4.3 MS Access 2010 Forms, Menus, Libraries, and VBA classes and methods
21
1.4.4 ActiveX Data Objects (ADO) middleware .................................... 25
2 Database axioms and best practice rules ........................................................ 30
2

2.1 Database axioms ........................................................................................... 30
2.1.1 Design ............................................................................................ 30
2.1.2 Implementation .............................................................................. 30
2.1.3 Usage .............................................................................................. 31
2.1.4 Optimization .................................................................................. 31
2.2 Best practice rules ......................................................................................... 32
2.2.1 Design ............................................................................................ 32
2.2.2 Implementation in Oracle .............................................................. 34
2.2.3 Querying and manipulating ........................................................... 36
2.2.4 Optimization .................................................................................. 37
3 DBDetectiveOs architecture .......................................................................... 38
4 DBDetectiveOs database design and implementation ................................... 41
4.1 Design ........................................................................................................... 41
4.1.1 E-R Diagram and restrictions list .................................................. 41
4.1.2 Mathematical scheme .................................................................... 45
4.1.3 Relational scheme and associated non-relational constraints list .. 49
4.1.4 Sequences, triggers, db links, stored procedures, and views ......... 52
4.2 Implementation ............................................................................................. 53
4.2.1 Tables and constraints .................................................................... 53
4.2.2 Sequences, triggers, db links, stored procedures, and views ......... 55
3

5 DBDetectiveOs application design and implementation ............................... 58
5.1 Design ........................................................................................................... 58
5.1.1 Business Model and Use Cases ..................................................... 58
5.1.2 Menu, forms, and library design .................................................... 60
5.2 Implementation ......................................................................................... 63
5.2.1 Menu development ........................................................................ 68
5.2.2 Forms development ........................................................................ 69
5.2.3 Library development ...................................................................... 71
6 DBDetectiveOs Users Guide ........................................................................ 72
6.1 Installing, uninstalling, and configuration .................................................... 72
6.1.1 Oracle 11g (or newer) script execution for creating and initializing the
db 72
6.1.2 Installation and configuration of the MS Access 2010 to Oracle 11g
bridge 73
6.1.3 DBDetectiveO uninstallation ......................................................... 93
6.2 Starting and exiting DBDetectiveO ............................................................... 93
6.3 Managing server connections and database links ......................................... 94
6.3.1 Navigating between known servers and dbs metadata .................. 95
6.3.2 Inserting a new Oracle server connection ...................................... 97
6.3.3 Updating server connection ........................................................... 99
6.3.4 Deleting server connection .......................................................... 100
4

6.3.5 Inserting a new Oracle db link .................................................... 101
6.3.6 Updating Oracle db link .............................................................. 104
6.3.7 Deleting Oracle db link ................................................................ 105
6.4 Mining Oracle metacatalog data for DBDetectiveO investigations ........... 107
6.4.1 Setting the desired target db to investigate .................................. 107
6.4.2 Starting target db investigation .................................................... 108
6.5 Investigating violations of db best practice rules ....................................... 109
6.5.1 Displaying global statistics .......................................................... 109
6.5.2 Displaying main tables, columns, triggers, constraints, and indexes
metadata 110
6.5.3 Displaying Oracles constraint types codes and their meaning .. 112
6.5.4 Displaying DBDetectiveOs table types codes and their meaning113
6.5.5 Selecting, unselecting, and displaying desired mining results .... 113
6.6 Displaying mined metadata and storing semantic and decision one .......... 115
6.6.1 Displaying mined metadata on objects and their dependencies .. 116
6.6.2 Storing semantic and decision metadata in DBDetectives db .... 118
6.7 Investigating Oracle advisors recommendations and log ........................... 121
6.7.1 Oracle advisors log ...................................................................... 121
6.7.2 Oracle advisors recommendations .............................................. 121
7 Case study ..................................................................................................... 122
Conclusion and further work ................................................................................ 124
5

References ............................................................................................................ 126
Subject and abbreviation index ............................................................................ 130


8

Foreword
Unfortunately, as compared to investments and efforts that are put into database
applications, those put in designing and developing tools for asserting and improving
database quality are almost insignificant. Fortunately, among our best students there
are some that realize how important is database quality, the fact that the best way to
guarantee it is to follow from start (business analysis and scheme design) to end
(maintenance and fine-tuning) a set of best practice rules, and that university (and es-
pecially MSc. and PhD. dissertation projects) are excellent opportunities to work in
this field too.
I congratulate Miss Alina Iuliana Dicu for his excellent work in architecting, de-
signing, developing, testing, and documenting DBDetective for Oracle 1.0, as well as
for writing this book that should be of great interest for all practitioners in the data-
base field: most of the best practice rules that she selected for this first commercial
version of DBDetective are addressing general database design, manipulation, and
querying optimization topics, and even the Oracle implementation and fine tuning
ones are of broader interest, as almost all relational database management systems
technologies are very similar one to another.
It is a great pleasure for me to wish her great success first of all in her current PhD
studies, devoted to the same subject mining metadata for discovering possible viola-
tions of database design, implementation, usage, and optimization, storing database
architects, administrators, and owners decisions regarding such findings, and automa-
tically generating code and executing it for accordingly improving the quality of the
involved databases, as well as implementing her research results in even more po-
werful versions of DBDetective(O) (as Oracle might already be accompanied and/or
replaced by any other similar tool in subsequent versions of DBDetective), and sel-
ling more and more licenses of it!
I am convinced that most of the readers of this book will not only enjoy its clarity,
adequate structure and illustrations, but also the wealth of knowledge it contains,
both theoretical and technological.

Assoc. Prof. Dr. Christian Manca Bucharest, March 2014
Computer Science Department, Ovidius State University, Constana, Romania
Engineering in Foreign Languages Dept., POLITEHNICA Univ., Bucharest
9

Preface
Too often, databases (dbs) are very poorly designed, implemented, queried,
and manipulated. As a consequence, they allow storing implausible data, some-
times even do not accept plausible data, are very slow when data accumulates,
and need too much unneeded disk and memory space, as well as programming ef-
fort.
In order to be able to judge dbs quality, Professor Christian Mancas established
sets of db axioms and corresponding best practice rules (bprs) (see comprehensive
sets in [1] and complete ones in [2]) that he is teaching both for undergraduate (the ba-
sic ones) and graduate (the advanced ones) students.
DBDetective for Oracle (DBDetectiveO) is a tool for mining db metadata in
search for possible violations of some important such db bprs, designed for Oracle
dbs and developed by using Oracles PL/SQL and Microsofts (MS) Access Visual
Basic for Applications (VBA).
DBDetectives idea belongs to Professor Christian Mancas who included it in his
2013 list of MSc. Thesis Project proposals for the Bucharest POLITEHNICA Uni-
versitys Faculty of Engineering Taught in Foreign Languages Computer Science En-
glish Streams Software Engineering MSc. Program. I chose it because I was always
interested in dbs, especially in their quality and possible optimizations and also be-
cause at that time I was working with Oracle dbs. Essentially, this book was my dis-
sertation thesis elaborated under Professor Mancas supervision that I publicly defen-
ded end of July 2013.
My main contributions were the following:
research for and contrastive presentation of related work
design, development, test, case study, and documentation for violations detection
corresponding to the 23 best practice rules presented in section 1.1
research for and development of the bridge between Access 2010 and Oracle 11g,
using MS ODBC Driver Manager, Oracle ODBC Driver for Win64, and Oracle
Client 11g
this dissertation, also including DBDetectiveOs Users Guide.
Even if since a couple of months after successful graduation of the Software Engi-
neering MSc. with this thesis I had to switch to PostgreSQL and MS SQL Server, I am
still very happy that I worked for the architecture, design, and coding of DBDetec-
10

tiveO, especially because the vast majority of the considered db axioms and bprs per-
tain to general db design, implementation, usage, and optimization and even those that
might seem at a first glance to strictly address Oracle dbs, are in fact much more gene-
ral too (as corresponding Oracle technologies are conceptually very similar to those of
other relational db management systems (RDBMSs)).
This is why further work will include extending the scope of this tool to more and
more other RDBMSs, as the main difference between them is their metadata cata-
logues structure. The obvious main goal is also common, regardless of RDBMSs: to
provide db architects, programmers, and administrators with data necessary for im-
proving the quality of their dbs.
Acknowledgements
I am indebted to all of our wonderful Professors of this Software Engineering
Master, for their outstanding scientific and technological competence, passion, pa-
tience, and didactic skills. In particular, this work is based on knowledge obtained
from the lectures and labs of Advanced Databases (Prof. Christian Mancas), Pro-
gramming Paradigms (Prof. Dan Luca Serbanati and Andrei Vasilateanu), and
Human-Computer Interaction (Prof. Petre Dimo).
I am very grateful to Prof. Christian Mancas, especially for his enlightening and
beautiful lectures and labs, his rigorous, wide open, and in-depth approach to any
project, be it a university semester or a commercial one, as well as for his patient gui-
dance that he offered me throughout the whole process of preparing this dissertation
and for accepting to write the foreword to this book.
Thanks to my scientific coordinator, Prof. Christian Mancas, to the Department
Director, Prof. George Dragoi, and to the Faculty Dean, Prof. Adrian Volceanov, for
their approval to publish this book at Lambert Academic Publishing and to mention
that it is based on my MSc. thesis dissertation.
Thanks also to Lambert Academic Publishings editor Iuliana Oaserele for her
kind help in formatting and publishing this book.
11

1. Introduction
1.1 Book structure
This opening chapter includes both the motivation and the goals of this first
DBDetectiveO version, related work, and a succinct presentation of the technologies
used. Chapter 2 introduces the db axioms and corresponding bprs that were chosen to
fundament this first DBDetectiveO version. Chapter 3 presents DBDetectiveOs ar-
chitecture; chapter 4 documents its db (back-end) design and implementation, while
chapter 5 addresses same topics for the application (front-end); chapt er 6 con-
t ai ns its user guide; chapter 7 pr esent s a case st udy: t he results of its use
for a production db. The book ends with conclusion and further work, references,
and a subject and abbreviations index.
Essentials of DBDetectives motivation, related work, principles, and archi-
tecture, design, and usage of a prototype were described in [3]; essentials of DBDe-
tectiveOs 1.0 actual architecture, design, and development were published in [4].
1.2 The subset of best practice rules considered
This first version of DBDetective provides detection for the following 32
types of violations, presented in each category by descending order of their
negative impact on db performances:
1.2.1 Design
1. Fundamental not read-only tables having no not-null columns (except for the
primary key).
2. Keyless non-empty fundamental tables.
3. Fundamental tables (not corresponding to subsets) that dont have seman-
tic keys.
4. Tables having concatenated primary keys.
5. Tables having non-numeric primary key columns.
6. Oversized surrogate primary keys.
7. Tables having superkeys.
8. Temporary tables constraints.
9. Empty fundamental tables, their associated indexes and constraints.
10. Empty columns in non-empty fundamental tables, their associated inde-
xes, and constraints.
11. Tables on which no other interesting (i.e. not trigger only referring to the
12

corresponding table) object (e.g. view, stored procedure, etc.) depends
upon.
1.2.2 Implementation
12. User tables and/or indexes in system tablespaces.
13. Small (under x lines) non-cached fundamental tables.
14. Improperly typed foreign keys.
15. Concatenated foreign keys.
16. Foreign keys having non-numeric columns.
17. Over and under-sized foreign keys.
18. Concatenated indexes with wrong columns order.
19. Fundamental tables having migrated rows.
20. Normal (B-tree) indexes that should be of bitmap type (as their columns
have less than y % distinct values or more than z % null ones).
21. Bitmap indexes that should be of normal (B-tree) type (as their columns
have more than y % distinct values and less than z % null ones).
22. Tables with empty blocks.
1.2.3 Usage
23. Total number of sub-queries and each ones position in source code.
24. Total number of dynamic SQL executions and each ones position in the
source code.
25. Not parallel enabled functions and each ones position in source code.
26. Not result cache functions and each ones position in source code.
27. Not relies on functions and each ones position in source code.
28. Not compound triggers.
29. Not compound triggers without firing sequences.
1.2.4 Optimization
30. Tables to shrink.
31. Indexes without recent statistics gathered on them.
32. Indexes to be compressed (on tables having less than w lines).
The main advantages of the proposed approach are the following:
A much larger set of investigated possible db performance issues, including db
design ones.
Possibility to store, along with mined metadata, both additional semantic and
decision support data.
13

Possibility to automatically generate and execute correcting code.
Smaller and much faster dbs.
1.3 Related work: domain presentation
Even some medium complex relational db management systems (RDBMS),
like, for example, MS Access, offer tools for assessing db schemes quality. It is
true that, for example, since its 10th version, Oracle Dat abas e ( Or acl e)
provides advisor data and t hat in the current 11g version even a couple of par-
tially automated tools for exploiting them (see [5]; unfortunately, they are availa-
ble only in its Enterprise edition, not also in its Standard ones). They are:
Automatic SQL Tuning (ASQLT)
SQL Access Advisor (SQLAA), also including the Partition Advisor (PA)
Real-time SQL Monitoring (RTSQLM).
ASQLT offers better SQL profiling (by fixing potential regression after update
and verifying benefit through test execution), sensible defaults with flexible con-
figuration, and making the easy decisions, but under complete db administrator
(DBA) control; obviously, it is only easing DBAs tasks of improving SQL state-
ments execution performances.
RTSQLM is mainly an extension of ASQLT for parallel transactions: it guides
tuning efforts by monitoring and reporting any SQL execution that consumes
more than 5 sec. of CPU or I/O time and runs in parallel (parallel query PQ,
parallel data manipulation language (DML) statements PDML, and parallel
data definition language (DDL) statements PDDL), by monitoring each exe-
cution independently, with no performance impact, and by exposing statistics at
multiple levels (global and parallel execution, as well as plan operation).
SQLAA helps by suggesting best indexes types; note that, unfortunately, Oracle
provides only normal (that is B-type) indexes in its Standard editions, while the se-
cond type, bitmap, is only provided in its much more expensive Enterprise editions.
PA is the only one of great help from the db physical implementation point of
view: it detects whether a table, materialized view, or index should or should not
be partitioned (including hash and new interval on date and number partition
types) and, if partitioned, recommends when to add partition-wise joining too.
Moreover, Oracle made public a best practice rules guide (see [6]), which also
includes three of the ones considered by DBDetective too (see chapter 2 below):
14

reduce data contention
choose indexing techniques that are best for your application, and
use static SQL whenever possible.
Obviously, many of the db implementation, fine-tuning, querying, and mani-
pulating best practice rules from [1] and [2] are derived not only from Pr of es-
sor Chr i s t i an Mancas s experience, but also from Oracle documentation (e.g.
[7]).
Other similar, third party tools exist; for example, DB Optimizer, a member
of Embarcadero Technologies DB PowerStudio for Oracle [8], which, compared
to Oracles ones, only adds more fancy graphical tools and analyze SQL and
PL/SQL complex statements ( also including bind variables), for pinpointing in-
dexes that are used, not used, or needed but missing.
The most interesting recent theoretical paper on conceptual db schema opti-
mization ([9]) offers an impressive list of 54 references and a mathematical back-
ground for characterizing and detecting two classes of equivalence (one based on
the mathematical semantics of the conceptual schemas and the other one on
conceptual preference by humans) between schemas before and after each optimi-
zation step, based on rules, but no actual language for constraint and derivation
rule specifications is presented: it only announces that a tool for performing
schema optimization in a semi-automated way was currently being designed.
The latest paper in this field ([10]), presents (unfortunately using poor En-
glish) an optimization method of relational schemas based on the standard nor-
malization theory (but only up to 3NF, not 5NF, as theory is from long time
ago established), by eliminating partial and transitive functional dependencies;
obviously, its major drawback is that, in order to apply this method, you first
need the set of corresponding functional dependencies. Moreover, it has been pro-
ven since decades (see, e.g., [2]) that relational db design is not enough powerful,
as the relational data model ( RDM) is a purely syntactic type one, and that
db design has to be done by using higher conceptual level, semantic data models.
It is worth noting that db scheme optimization is of paramount importance:
all major RDBMS competitors are striving to invent, patent, and apply new
such methods in order to gain as many pleased customers as possible. For exam-
ple,
[11] presents a U.S. patent obtained for SAP: for answering queries more
15

quickly and efficiently on tables storing directed graphs (and especially
trees), path information is also stored for connecting each node to all
those with which it is connected.
[12] and [13] present two U.S. related patents (the first one still pending
approval) for IBM: a db optimizer collects statistics on which type of ap-
plications access dbs and possibly makes changes to their schemes based
on them; specifically, new redundant columns of a better suited data type
might be automatically added, filled with corresponding conversed data,
and maintained synchronized forever with the duplicated column base da-
ta; as a further refinement, the optimizer also detects when one type of
application accesses a column a percentage of time that exceeds a prede-
fined threshold level and is less than other such level and creates a new co-
lumn in the db such that data is present in both formats.
[14] internationally presented a mathematical data model (MDM, introduced
and presented in Romania some two decades earlier and fully presented in [2]).
[15], [16], and [17] present the most important details of it relevant to this the-
sis subject, as they are founding the semantic normal relational form into which
this model is translating its mathematic schemes (see [2] and [18]); db design
best practice rules of [1] and [2], which are obviously applicable to no matter
what other targeted RDBMS, are derived from this semantic normal form. Note
that a vast majority of the rest of these rules are also applicable on any other
RDBMS as well.
1.4 Technologies used
Very many companies are using for developing their products and/or hosting their
customers data the Standard One edition of the Oracles 11g db servers; conse-
quently, this was a strong argument for both considering Oracle flavored dbs imple-
mentation, usage, and optimization best practice rules, as well as developing
DBDetectives db and business logic (BL) tiers. Moreover, although their Oracle ser-
vers are generally installed on Linux/Unix platforms, front-end graphic user interface
(GUI) and BL software are mostly designed and developed for MS Windows operat-
ing systems (OSs) versions 7 and 8, generally on 64 bits. Consequently, all software
environments and tools used for DBDetective are on 64 bits for Windows 7 and 8 too.
16

1.4.1 Oracle 11g Database Server
The name Oracle comes from the code name of a CIA project for which the com-
panys founders had all worked on previously, while at the Ampex Corporation. By
revenue, Oracle is currently the third-largest software ma-
ker after Microsoft and IBM. The company is specialized in developing and market-
ing computer hardware systems and enterprise software products particularly its
own brands of RDBMSs, but also tools for db development and software systems
for enterprise resource planning (ERP), customer relationship management (CRM),
and supply chain management (SCM), as well as middlewares.
In 1978, eight years after E.F. Codd published his seminal paper [19] that founded
RDM, Oracle Version 1, written in assembly language, was launched on PDP-11 mi-
ni-computers, under the RSX-11 OS, in 128K of memory. Implementation separates
Oracles from users code. Version 1 was never officially released. Oracle 2 was the
first version of the Oracle db software that had a customer, in June 1979: the U.S.
Wright-Patterson Air Force Base.
In 1983, for portability reasons, Oracle 3 is released, rewritten in C: part of Oracle
Corporation's success from that moment onwards arose from using this programming
language, which eased porting to different OSs, as most of them support C. In 1984,
Oracle version 4 (introducing read consistency) as well as a first PC version (running
on an own MS-DOS version) were released. One year later, its fifth version was one
of the first RDBMS ever developed according to a server-client architecture. Two
years later, the PC version 5 starts running in the 286 protected mode. Same year, its
version 5.1 is released with support for distributed queries. In 1987, Oracle offers its
first Unix-based applications.
In 1988, Oracle 6 is released with support for row-level locking, hot backups, and
embedding the PL/SQL procedural language engine into the db. Stored program
blocks (e.g. procedures and triggers) were added to version 7, released in 1992, in
which users could also submit PL/SQL blocks for immediate execution in the server
from an environment such as SQL*Plus, or via SQL statements embedded in a host
program. Oracle 7 included separate PL/SQL engines in various client tools (such as
SQL*Forms and Reports) too, performance enhancements, administrative utilities,
application-development tools, security features, the ability to persist PL/SQL pro-
gram units in the db as stored procedures and triggers, and support for declara-
tive referential integrity.
17

In 1993, Oracle released its Cooperative Development Environment (CDE), which
bundled Oracle Forms, Reports, Graphics, and Book. Two years later, it released its
first 64-bit db servers versions. In 1996, it moved towards an open standards-based,
web-enabled architecture.
In 1997 Oracle 8 was released with SQL object technology, Internet technology,
and support for terabytes of data. One year later, the corporation released Oracle Ap-
plications 10.7 Network Computing Architecture (NCA) and Oracle 8i (where
i stands for Internet), first time for Linux platforms too. All the applications in the
business software were since running across the web in a standard web browser.
Moreover, in 1999, Oracle offered its first DBMS with XML support.
Oracle Database and Application Server 9i were released in 2000, including an
Internet File System (iFS), later re-branded as Oracle Content Management SDK and
a Real Application Cluster (RAC). In 2001, it became the first RDBMS to complete
the 3 terabyte Transaction Processing Performance Council ad-hoc decision support
benchmark (TPC-H) world record. One year later, 9i was declared the first RDBMS
to pass 15 industry standard security evaluations.
In 2004, Oracle 10g was released (where g stands for Grid, from Enterprise Grid
Computing [20], with a grid of interconnected Oracle db servers). Besides grids (for
which tools like Oracle Scheduler and Performance Monitor for Large Clusters are
provided), its main new features included globalization and Unicode support,
PL/SQL conditional compilation, an additional matrix math package, and obfuscation
of dynamically generated code, SQL Perl compatibility for regular expressions, a
XML and SQL Rule Manager for extending triggers capabilities of handling db
events, enhanced XQuery and XPath for XML processing, Secure Sockets Layer (SSL)
support (i.e. adding HTTPS to HTTP), Oracle Data Guard (ODG) tool for fast-start,
redo, and SQL apply failovers, Recovery Manager (RMAN) backup encryption, incre-
mental roll forward of db copies, unused block compression, and lot of Business In-
telligence (BI) improvements, like the MODEL clause of the SELECT statement de-
signed for analytic applications, incremental aggregation for OnLine Analytical Pro-
cessing (OLAP) cubes, and a decision tree algorithm for data mining, RAC enhance-
ments like a Load Balancing Advisory and clusterware file redundancy, as well as se-
curity enhancements (transparent and key management for data encryption) and bet-
ter MS Windows integration of the Oracle Data Provider for MS .NET technologies
(ODP.NET).
18

In 2005, (when Oracle released its first free database, Oracle Database
10g Express Edition (XE)) a serious security vulnerability in Oracle database pas-
sword management is published by two researchers of the University of London.

Ora-
cle Corporation replied that existing safeguards and following good industry practi-
ces were sufficient defenses and didn't close the underlying security hole until its re-
lease of the 11g version in 2007.
The current 11g version main new features are: PL/SQL functions and queries re-
sult cache, subprograms inlining, encryption, compression, and preprocessing for ex-
ternal tables, virtual columns, and SQL pivot and unpivot queries. Version 12c is still
postponed.
DBDetectiveOs db, including tables, db links, views, sequences, triggers, etc. was
developed in Oracle 11g SQL [21] and PL/SQL [22].
1.4.2 Other Oracle tools: Enterprise Manager, SQL Developer, Client,
ODBC driver
1.4.2.1 Oracle Enterprise Manager (OEM)
For creating and managing the DBDetective db, I used the Oracle Enterprise Ma-
nager (OEM) latest 11g Database Control web-based graphic console [23].
OEM uses by default the SYSMAN schema in an Oracle db as a super-administra-
tor account/repository. OEM provides comprehensive, flexible, easy-to-use monitor-
ing functionality that supports the timely detection and notification of impending IT
problems to the intended users. It offers the most comprehensive monitoring from
Oracle Database instances to Oracle Real Application Clusters to Oracle Applica-
tion Server Farms and Clusters. OEM Grid Control comes with a comprehensive set
of performance and health metrics that allow monitoring of key components in your
environment such as applications, application servers, dbs, as well as the back-end
components on which they rely, such as hosts, OSs and storage. Note that todays
OEM versions can also manage IBM DB2, MS SQL Server, and other RDBMSs dbs.
OEM for Grid Controls architecture has the following three components:
the collection agent (Oracle Management Agent OMA)
the aggregation agent (Oracle Management Server OMS)
the repository agent (Oracle Management Repository OMR)
OMA runs on the target host and collects information on the hardware, OS, and
applications that run on the target. OMS runs on one or two servers and collects the
data generated by OMAs. OMS pulls the information from OMAs and aggregates the
19

collections into the repository. OMS also acts as the user-interface by generating
web-pages for DBAs to view the status of systems and services. OMR comprises an
instance of the Oracle db that stores the data collected by OMS. Installers can make
OMR highly available or fault-tolerant by running it on an Oracle RAC instance
across multiple nodes.
The key features of OEM Grid Control are the following:
- System Monitoring
- Managing Groups
- Job System
- Information Publisher
- Compliance Management
- Extending Enterprise Manager
- Managing Targets
OEM performs much of its activity through intelligent agents which Oracle refers
to as Oracle Management Agents. These run as autonomous proxy processes on a
managed node and perform execution and monitoring tasks for OEM, communicating
using the (Secure) Hypertext Transfer Protocol (HTTP / HTTPS).
1.4.2.2 Oracle SQL Developer (OSQLD)
For developing DBDetectiveOs SQL views, triggers, sequences, db links, etc. I
used the Oracle SQL Developer (OSQLD) latest version 3.2.20.09 [24].
OSQLD is a free, Java Development Kit (JDK)-based Integrated Development
Environment (IDE) for (PL/)SQL programming in Oracle (but also IBM DB2, MS
Access, MS SQL Server, Oracle MySQL, Sybase Adaptive Server, and Teradata) dbs.
It supports automatic tabs, code insight, bracket matching and syntax coloring for
PL/SQL.
OSQLD components include the Oracle Web Agent (OWA or MOD_PL/ SQL), an
extension module for the Apache web server, which helps in setting up dynamic web-
pages from PL/SQL within OSQLD. Moreover, since its version 3, OSQLD also in-
cludes the SQL Developer Data Modeler (SQLDDM), which operates with and mo-
dels metadata that constituted before version 3 a separate (but integrated)
free counterpart of OSQLD.
20

1.4.2.3 Oracle Client (OC)
Besides its 11g db server that is both the back-end tier of DBDetectiveO and the
host of its input customer dbs, in order to bridge it with DBDetectiveOs front-end
developed in MS Access (see next subsection for explanations on this choice), I also
had to use Oracles 11g client (OC), for being able to use its Open DataBase Connec-
tivity (ODBC) driver.
OC provides the following three installation options [25]:
- Instant Client: Enables you to install only the shared libraries required by Ora-
cle Call Interface (OCI) applications that use the Instant Client (IC) feature.
This installation type requires much less disk space than the other Oracle Da-
tabase Client installation types.
IC includes Instant Client Light (ICL); you may want to use ICL if your ap-
plications will generate error messages in American English only. The advan-
tage of using ICL is that it has a smaller footprint than regular IC: its shared li-
braries, which your applications must load, are only 3032 MB as opposed to
the 216 MB that regular IC uses. Hence, your applications use less memory.
- Administrator: Enables applications to connect to an Oracle db on the local or
on a remote system. It also provides tools that let you administer an Oracle db
server.
- Runtime: Enables applications to connect to an Oracle db on the local or on a
remote system.
1.4.2.4 Oracle Open DataBase Connectivity Driver (OODBCD)
Microsoft Open Database Connectivity (ODBC) provides a common C program-
ming interface for applications to access data from RDBMS. Access to dbs is ma-
naged by the MS ODBC Driver Manager (MODBCDM). The driver manager pro-
vides the linkage between an ODBC application and an ODBC driver for a specific
RDBMS. Note that ODBC applications do not need to be re-linked or re-compiled in
order to change which RDBMS they are accessing.
Up to Access 2010, MS provided ODBC drivers for Oracle 8i. Since then, not
even this driver is provided anymore. Fortunately, the free Oracle ODBC Drivers
(OODBCD) provide access to all Oracle db versions for applications written using
the ODBC interface, including 11g, both for 32 and 64-bit OS platforms.
21

In Oracle 9i Release 2, Oracle ODBC even introduced full support for .NET via
the MS ODBC.NET data provider. With ODBC.NET, applications can access the
Oracle dbs from any .NET programming language.
1.4.3 MS Access 2010 Forms, Menus, Libraries, and VBA classes and
methods
As even Oracle db Standard One editions licenses are very expensive, amounts
which add to the cost of its needed hardware infrastructure, but also as Oracle is cur-
rently and from many years already a Java provider, very many companies are using
mostly free Java tools for developing their products (e.g. Maven-type MVC projects
of the Spring Tools Suite).
On the other hand, the following considerations lead me to using MS Access [26]
for developing DBDetectiveOs front-end GUI:
Many companies also use for internal purposes the MS Office 365 suite, which
includes Access 2010 too.
Oracle Forms and Reports have to be bought apart (and since very many years
already, although Oracle still develops them, they are de facto superseded by
the Java-based Oracle Application Express (OAE), which, unfortunately, is
even costlier), whereas Access provides very powerful forms, reports, and
charts; moreover, since its 2010 version, Access can web-publish any desktop
application developed with its technologies, via MS SharePoint.
As proved also by [27], Visual Basic for (db) Applications (VBA) [28] is much
more productive than Java: for example, in order to achieve same functionali-
ty, some 60 times more Java than VBA statements were needed.
Last but not least, coupling Oracle and Microsoft technologies is a challenge
per se, as these two software giants are increasingly fighting each other (espe-
cially since Microsoft bought the SQL Server rights) for monopolist suprema-
cy, rather than trying to cooperate for the sake of their few common db and
computer programming customers.
Consequently, even Professor Christian Mancas encouraged me to use MS Access
for developing DBDetectiveOs front-end GUI.
Redmond, Washington-based giant Microsoft is not only the software world lea-
der, dominating already for several decades the PC OS and office suites markets, but
also one of the worlds most valuable companies.
22

MS Access 1.0 was launched in 1992, after some 4 development years, both as a
competitor to that time PC market dominating Borlands Paradox and Aston-Tates
(then Borlands too) dBase (as Foxs FoxPro, the former third one, was bought the
same year by Microsoft, for incorporating its Fox's Rushmore query optimiza-
tion routines into Access: bitmap compressed index and and or, processed in main
memory buffers) and as a frontend framework for the SQL Server (at that time still
owned by Sybase Corporation: it was only in 1994 that Microsoft bought the rights
for it on MS Windows OSs).
Consequently, MS Access is not only a db engine, but a full RDBMS, which
helps not only in storing data, but also in db development (sophisticated queries),
frontend applications development (forms to control the data, reports, and charts), da-
ta analysis and more generally widely understood data manipulation. This tool can
be divided into the following two components:
MS Access Database Engine (ACE), formerly known as MS SQL JET, which
is a very good option for smaller dbs. However, it is not aimed to be competi-
tion for other db engines, not even for simpler ones, like innoDB or MyISAM
from Oracle MySQL, not to mention Oracle db, IBM DB2, PostgresSQL, Sy-
base Adaptive Server, MS SQL Server, etc. ACE is compact and flexible, but
has some significant limitations (see end of sub-section).
MS Access db development and data manipulation tools, providing queries
(views), forms (for data manipulation), reports, macros (for automation purpo-
ses), VBA code (stored in class and library modules, stemming from MS Visual
Basic 6, enhanced with db event-oriented programming capabilities), and a lot
of wizards (from linking and embedding external data, exporting ACE stored
data, creating and managing application menus, hiding pointer type foreign
keys data and replacing it with dynamic combo-boxes displaying correspond-
ing semantic keys data, publishing desktop applications to web, etc.).
MS Access is mostly valued for:
its connectivity features
- data stored by ACE can easily be migrated through Access to practically
any other db and vice-versa, as long as appropriate ODBC drivers are avai-
lable;
- the same goes for many other data formats, like Excel, Outlook, XML, PDF;
23

- not only data, but also views (queries), forms, and reports can be imported
from or exported to other Access dbs;
its really very open and quite scalable technology: Access is also used as MS
applications frontend for dbs managed by other RDBMS, especially MS SQL
Server, but not only (see above point too);
it can also use additional technologies like ActiveX components, which give in-
teresting options when we have to extend the basic features of our applications
(please note that we can write our own ActiveX components in, for example,
C++ or use thousands of components provided by third-parties);
Dynamically Linked Libraries (DLL) can also be used from VBA: this gives
even more power, as you can write your own libraries in almost any other lan-
guage available for MS Windows platforms, to enhance basic MS Access appli-
cation functionality;
since MS Access 2007, applications can integrate with MS Sharepoint (pre-
viously, in MS Access 2003, there were only the so-called Access Data Pages
(ADPs), which allowed creation of only very simple web applications).
Consequently, MS Access is like Oracle Forms, integrated with Oracle Reports,
simplified in some areas (VBA, Sharepoint integration, wizards, etc.), enhanced with
its own db engine oriented for smaller dbs and with connectivity to any ODBC-ready
db backend, together with very easy data link/import/export capabilities between ma-
ny different data source types (as the original concept of Access was for end users to
be able to access data from any source; note that Access was previously used by Mi-
crosoft for its communications program meant to compete mainly with Datastorm
Technologies ProComm, but, as this program proved to be a failure, it was dropped
and its name was later reused for the first companys db software).
As it can perform heterogeneous joins between data sets stored across different
platforms, Access is often used too to download (or link to) data from enterprise level
dbs for manipulation, analysis, and reporting locally.
Note that MS Access is a file server-based database: unlike clientser-
ver RDBMSs, Access did not implemented database triggers, stored procedures,
or transaction logging up to its 2010 version, which finally includes table-level trig-
gers and stored procedures too, built into the ACE data engine. Moreover, tables,
queries, forms, reports, and macros can now be developed specifically for web-based
24

application in Access 2010, as integration with Microsoft SharePoint 2010 is also
highly improved.
Also note that, since its beginnings, MS Access offered parameterized queries.
These queries, as well as Access tables can be referenced from other programming
platforms like VB6 and .NET, through Data Access Objects (DAO, only for 32-bits
platforms) or the newer ActiveX Data Objects (ADO, for both 32 and 64-bits) Com-
ponent Object Model (COM) type middleware libraries. Dually, from Access, VBA
can reference parameterized stored procedures via ADO.
The native Access Jet mdb db format also evolved over the years, from versions
1.0, 1.1, 2.0 (all first three still prone to data corruption for files larger than 10MB),
7.0 (Access 95, launched on Windows 95), and 97. Access 2000 ACE introduced a
new format, called accdb, which is not backward compatible with earlier versions of
Access and which was improved in the subsequent Access versions 2002, 2007, 2010,
and 2013.
Since its 95 version, MS Access can not only be bought alone, but is also included
in the upper, professional versions of the MS Office suites, together with MS Word,
Excel, Powerpoint, Outlook, Visio, etc. In previous versions of MS Access the ability
to distribute applications required the purchase of a Developer Toolkit; from Access
2010 the "Runtime Only" version is offered as a free download, making the distribu-
tion of royalty-free applications possible on Windows 7 and 8.
Current MS Access main limitations are the following:
maximum 2GB per db (dually, note that up to 1GB ones are processed in
internal memory, whenever available);
maximum 32 indexes per table;
maximum 10 columns per index, with a total of no more than 255 bytes;
except for long text (memo) and ActiveX object columns (which cannot be
sorted, indexed, grouped by, or even completely displayed in forms/reports,
but only in text boxes of maximum 64KB, although they may store up to
1GB ASCII or 2GB UNICODE data), the maximum row size is 4KB;
maximum 255 simultaneous user connections (with fast speed up to some
100 of them).
Note that, due to its (Object Linking and Embedding - OLE) linking capabilities,
although MS Access is designed for small (departmental or home) dbs under 2GB,
one can in fact store and process almost any amount of data, providing no table exce-
25

eds 2GB: each such table can be stored in a separate db and all of them may be linked
to a pure code one.
1.4.4 ActiveX Data Objects (ADO) middleware
Microsoft's ActiveX Data Objects (ADO) is a set of COM objects for accessing
data sources. ADO is a part of the MS Data Access Components (MDAC or Windows
DAC), which provides a middleware layer between programming languages and OLE
DB (a means of accessing data stores, whether they are dbs or not, in a uniform man-
ner) [29].
ADO allows developers to write programs that access data without knowing how
the db or any other data source is actually implemented; developers must be aware of
the data source type for connections only. Moreover, no knowledge of SQL is requi-
red to access a db through ADO, although ADO can be used to directly execute SQL
commands too. The obvious disadvantage of the latter is that it introduces a depen-
dency upon the type of db used. There is however an advantage too, as ADO automa-
tically translates between various SQL idioms.
Introduced in 1996, ADO is the successor to Microsoft's earlier object layers for
accessing data sources, including Remote Data Objects (RDO) and Data Access Ob-
jects (DAO). ADO is made up of four collections and twelve objects.
The four collections are the following:
Fields, containing a set of Field objects that can be used in either a Recordset
object or in a Record object. In a Recordset object, each of the Field objects
that make up the Fields collection corresponds to a column in that Record set
object. In a Record object, a Field can be an absolute or relative URL that
points into a tree-structured namespace (used for semi-structured da-
ta providers like the MS OLE DB Provider for Internet Publishing) or as a re-
ference to the default Stream object associated with that Record object.
Properties: an object can have as many Property objects as needed, which are
contained in the object's Properties collection.
Parameters: a Command object can have several Parameter commands to
change its predefined behavior; each of the Parameter objects are contained in
the Command object's Parameters collection.
Errors: all provider-created errors are passed to a collection of Error objects,
while the Errors collection itself is contained in a Connection object. When an
26

ADO operation creates an error, the collection is cleared and a new group of
Error objects is created in the collection.
The 12 objects are the following:
Connection: the connection object is ADO's connection to a data store via OLE
DB. The connection object stores information about the session and provides
methods of connecting to the data store. As some data stores have different
methods of establishing a connection, some methods may not be supported in
the connection object for particular OLE DB providers. A connection object
connects to the data store using its 'Open' method with a connection string
which specifies the connection as a list of key value pairs (for example:
"Provider='SQLOLEDB';DataSource='The SqlServer'; Initial Catalog='North-
wind';Integrated Security='SSPI';"), the start of which must identify the type of
data store connection that the connection object requires:
- an OLE DB provider (for example SQLOLEDB), using the syntax "pro-
vider =";
- a file name, using the syntax "file name=";
- a remote provider and server (see RDS), using the syntax "Remote provider
=" and "Remote server="; or
- an absolute URL, using the syntax "URL=".
Command: after the connection object establishes a session to the data source,
instructions are sent to the data provider via the command object. The com-
mand object can send SQL queries directly to the provider through the use of
the CommandText property, send a parameterized query or stored procedure
through the use of a Parameter object or Parameters collection or run a query
and return the results to a dataset object via the Execute method. There are
several other methods that can be used in the Command object relating to other
objects, such as the Stream, RecordSet or Connection objects.
Recordset: a recordset is a group of records and can either come from a base
table or as the result of a query on tables. The RecordSet object contains a
Fields collection and a Properties collection. The Fields collection is a set of
Field objects, which are the corresponding columns in the table/query. The
Properties collection is a set of Property objects, which defines a particular
functionality of an OLE DB provider. The RecordSet has numerous methods
and properties for examining the data that exists within it. Records can be up-
27

dated in the recordset by changing the values in the record and then calling on
the Update or UpdateBatch methods.
Immediate: the recordset is locked using the adLockOptimistic or adLock Pes-
simistic lock. The data are updated at the data source after the record is
changed and the Update method is called.
Batch: the recordset is locked using adLockBatchOptimistic and each time
Update is called the data are updated in a temporary buffer. Finally, when
UpdateBatch is called, the data are completely updated back at the data
source. This has the advantage of it all being done in memory, and if a problem
occurs then UpdateCancel is called and the updates are not sent to the data
source.
Transaction: if the OLE DB provider allows it, transactions can be used. To
start a transaction, programmers invoke the BeginTrans method and then do
the required updates. When they are all done, programmers invoke the Com-
mitTrans method. RollbackTrans can be invoked to cancel any changes
made inside the transaction and rollback the db to the state before the transac-
tion began.
Record: this object represents one record in the db and contains a collection of
fields. A RecordSet consists of a collection of Record objects.
Stream: a stream, mainly used in a RecordSet object, is a means of reading and
writing a stream of bytes. It is mostly used to save a recordset in an XML for-
mat, to send commands to an OLE DB provider as an alternative to the Com-
mandText object, and to contain the contents of a binary or text file.
Parameter: a parameter is a means of altering the behavior of a common piece
of functionality; for instance, a stored procedure might have different parame-
ters passed to it depending on what needs to be done: these are called parame-
terized commands.
Field: each Record object contains many fields and a RecordSet object has a
corresponding Field object also. The RecordSet object's Field object corres-
ponds to a column in the db table that it references.
Property: this object is specific to the OLE DB provider and defines an ability
that the provider has implemented. A property object can be either a built-in
property (a well-defined property implemented by ADO already and thus can-
28

not be altered) or can be a dynamic property defined by the underlying data
provider and can be changed.
Error: when an OLE DB provider error occurs during the use of ADO, an Error
object will be created in the Errors collection. Other errors do not go into an
Error object, however. For instance, any errors that occur when manipulating
data in a RecordSet or Field object are stored in a Status property.
Some basic steps are required in order to be able to access and manipulate data
using ADO:
1. Create a connection object to connect to the data source.
2. Create a recordset object in order to receive data in it.
3. Open the connection.
4. Populate the recordset by opening it and passing the desired table name, SQL
statement, or XML tag as a parameter to the open function.
5. Do all the desired searching/processing on the fetched data.
6. Commit the changes made to the data (if any) by using Update or Update-
Batch methods.
7. Close the recordset.
8. Close the connection.
The ADO 2.8 software development kit (SDK) consists of the following four com-
ponents:
1. ADO: its primary benefits are ease of use, high speed, low memory overhead,
and a small disk footprint. ADO supports key features for building client/ server and
web-based applications.
2. Multidimensional ADO (ADO MD): provides easy access to multidimensional
data from languages such as MS Visual Basic, C++, and J++. ADO MD extends
ADO to include objects specific to multidimensional data, such as the CubeDef and
Cellset objects. With ADO MD you can browse multidimensional schema, query a
cube, and retrieve the results. Like ADO, ADO MD uses an underlying OLE DB pro-
vider to gain access to data. To work with ADO MD, the provider must be a multidi-
mensional data provider (MDP) as defined by the OLE DB for OLAP specification.
MDPs present data in multidimensional views as opposed to tabular data providers
(TDPs) that present data in tabular views.
29

3. Remote Data Service (RDS) is a feature of ADO with which you can move data
from a server to a client application or web page, manipulate the data on the client,
and return updates to the server in a single round trip.
4. ADO Extensions for Data Definition Language and Security (ADOX) is an ex-
tension to the ADO objects and programming model, including objects for schema
creation and modification, as well as security. Because it is an object-based approach
to schema manipulation, you can write code that will work against various data
sources regardless of differences in their native syntaxes. ADOX is a companion li-
brary to the core ADO objects. It exposes additional objects for creating, modifying,
and deleting schema objects, such as tables and procedures. It also includes security
objects to maintain users and groups and to grant and revoke permissions on objects.

30

2 Database axioms and best practice rules
From the sets of db axioms and best practice rules presented in [2], only the fol-
lowing small subsets (see sub-sections 2.0 and 2.1) were considered for DBDetec-
tiveO 1.0.
2.1 Database axioms
2.1.1 Design
A1. Data plausibility axiom: any db instance should always store only plausible
data; implausible (garbage) data might be stored only temporarily, during updating
transactions (that is, any time before start and after end of such a transaction all data
should be plausible).
A2. Unique objects axiom: just like, generally, for sets elements, object sets do not
allow for duplicates (that is each object for which data is stored in a db should always
be uniquely identifiable through its corresponding data).
A3. Best possible performance axiom: db design, implementation, and optimiza-
tion should guarantee obtaining the maximum possible performance that is the
overall best possible execution speed for critical queries and updates, as well as the
best possible average execution speed for the non-critical ones.
A4. No constraints on redundant data axiom: no constraint should be enforced on
redundant data.
A5. Constraints discovery axiom: for any non-trivial and non-contradictory and
not implied (that is computable) restriction existing in the modeled sub-universe, any
db should enforce a corresponding constraint.
A6. Constraints optimality axiom: for any db scheme, no implied constraint
should be enforced and the constraint set should be enforceable in the minimum pos-
sible time (that is there should not exist another equivalent constraint set whose en-
forcement takes less time in the given context).
2.1.2 Implementation
A7. Constraints enforcement axiom: for each fundamental table, any of its con-
straints whose type is provided by the target RDBMS has to be enforced through that
RDBMS; all other db constraints should be enforced by all software applications
built on top that db.
A8. Rows uniqueness axiom: any fundamental table row should be always uni-
quely identifiable according to all existing uniqueness constraints in the correspond-
31

ing sub-universe and it should correspond to a unique object from the corresponding
objet set.
A9. No void rows axiom: for any fundamental table, at least one semantic funda-
mental column should not accept null values.
A10. Referential integrity axiom: any foreign key value should always reference
an existing value (dually, no db should ever contain dangling pointers).
A11. Primary keys axiom: any fundamental table should have a surrogate integer
primary key whose range cardinality should be equal to the maximum possible cardi-
nality of the corresponding modeled object set; except for tables corresponding to
subsets, primary keys should store auto generated values.
A12. Foreign keys axiom: any foreign key should be simple (that is not concatena-
ted), reference a surrogate integer primary key and have as range exactly the range of
the referenced primary key.
A13. Key Propagation Principle axiom: any mapping (that is many-to-one or one-
to-many relationship) between two fundamental object sets should be implemented
according to the Key Propagation Principle: a foreign key referencing its co-domain
(that is the one side) is added to the table corresponding to its domain (that is the
many side).
A14. No superkeys axiom: no superkey should ever be enforced; generally, no
other constraints than those declared in the corresponding conceptual db scheme
should ever be enforced.
2.1.3 Usage
A15. Relevant data and processing axiom: any query should consider and mini-
mally process, in each of its steps, only relevant data.
A16. Fastest possible manipulation axiom: at least in production environments,
data manipulations should be done with best possible algorithms and technologies, so
that processing speed be the fastest possible.
2.1.4 Optimization
A17. Reduce data contention: intelligently use hard disks, big files, multiple table-
spaces, partitions, and segments with adequate block sizes, separate user and system
data, avoid constant updates of a same row, etc., in order to always reduce data con-
tention to the minimum possible.
A18. Minimize db size: regularly shrink tablespaces, tables, and indexes for main-
taining high processing speeds and minimal db and backup sizes.
32

A19. Maximize use of RDBMS statistics: regularly gather and intelligently use sta-
tistics provided by RDBMS for fine-tuning dbs.
A20. Follow RDBMS advisors recommendations: regularly monitor and apply all
recommendations of RDBMS advisors.
2.2 Best practice rules
2.2.1 Design
BPR0 (Data plausibility). All actual constraints should be enforced in all funda-
mental (that is not temporary, not derived) tables, in order to guarantee their data
instances plausibility.
BPR1 (Surrogate primary keys). Each fundamental table should have an asso-
ciated primary surrogate key: an one-to-one integer property (range restricted
according to the maximum possible corresponding instances cardinality - see
BPR2 below), whose sole meaning should be unique identification of the corres-
ponding lines (this being the reason to refer to them as surrogate or syntactic keys
too).
Note that very rarely, by exception, such a surrogate key might also have a
semantic meaning: for example, rabbit cages may be labeled physically too by
the corresponding surrogate key values (and, obviously, no supplementary attri-
bute should then be added for unique identification, as it would redundantly du-
plicate the corresponding surrogate key).
Such minimal primary keys favor optimal foreign keys (see BPR5 below) with
least possible time for computing joins. If storage space is a drastic concern,
you might not add them to tables that are not referenced; otherwise, it is better
to always add them both for avoiding tedious supplementary DBA tasks when
they will become referenced too, as well as for homogeneity reasons. Obviously,
derived/computed tables, be them temporary or not, may have no primary keys.
Note that, not only theoretically, in RDM, but also in all RDBMS versions,
any key may be freely chosen as the primary one and that, actually, as a conse-
quence, unfortunately, the vast majority of existing dbs are using concatenated
and/or semantic, not only numeric, surrogate primary keys.
BPR2 (Instances cardinalities). Surrogate keys should always take values in
integer sets whose cardinalities should reflect maximum possible number of ele-
ments of the corresponding sets.
33

BPR3 (Semantic keys). Any fundamental (not temporary, not derived) table
corresponding to an object set that is not a subset (of another object set) should
have associated all of its corresponding semantic (candidate, ordinary) keys: ei-
ther one-to-one columns or minimally one-to-one column products. With ex-
tremely rare exceptions (see the rabbit cages example in BPR1 above), any such
non-subset table should consequently have at least one semantic key: any of its
lines should differ in at least one non-primary key column (note that there is
not only one NULL value, as, for example, Oracle and MS SQL Server errone-
ously assume in this context, but an infinite number of NULLs, all of them dis-
tinct!). Only subsets and derived/ computed ones might not have semantic keys.
Note that any table may have more than one semanti c key (and, according
to a theorem from [ 2], combi ni ng t hose publ i shed i n [16] and [17], a maxi-
mum number of keys equal to the combination of n taken [n /2] times, where n is
the total number of semantic, non-primary key columns of the table and [x] is
the integer part of x). Obviously, in order to reject implausible instances, all of
them (just like all of the other existing constraints) should always be included in
the corresponding conceptual models, schemes, and implementations.
Note that, for all subsets tables, all of their lines may always be uniquely
identified semantically too (through the subset surrogate primary key, see BPR1
above) by all of the keys of the corresponding superset table. Also note that, ob-
viously, subset tables too may have other semantic keys (not only their syn-
tactical primary ones).
BPR4 (No superkeys). We should never consider superkeys (i.e. one-to-one co-
lumn products that are not minimal, that is for which there is at least one co-
lumn that can be dropped and the remaining sub-product is also one-to-one;
equivalently, superkeys are those products that properly include keys, i.e. they
include at least one key without being equal to it), either conceptually, or, espe-
cially, for implementations. We should always stick to keys (see BPR3 above).
Note that, unfortunately, not only MS Access, SQL Server, Oracle, or MySQL
do not reject superkeys!
BPR5 (Foreign keys). Any foreign key should always reference the primary
key of the corresponding table. Hence, it should be a sole integer column: we
should never use concatenated columns, neither non-integer columns as foreign
34

keys. Moreover, their definitions should match exactly the ones of the corres-
ponding referenced primary keys (see BPR2 above).
Please note that, again, unfortunately, not only RDM, but almost all
RDBMSs too allow foreign keys, be them concatenated or not, to reference not
only primary keys (see BPR1 above), but anything else, including non-keys (pro-
vided that all of the corresponding columns belong to a same table).
Please also note that, most unfortunately, it is a widespread practice to de-
clare concatenated primary keys containing concatenated foreign keys for chains
of referencing tables; consequently, even if the root of such a chain has only a
column as its primary key, the next table in the chain should have a primary key
with arity of at least two, the third ones arity should be of at least three, etc.
BPR6 (At least one not null non primary key column per table). Any table should
have at least one not accepting nulls column, other than its primary key: what
would a line having only nulls (except for its syntactic surrogate key value)
stand for?
BPR7 (No constraints on not fundamental tables). Temporary and derived (compu-
ted) tables should not have any constraints enforced: as they are not fundamental,
they should be read-only for users; moreover, being computed from valid data with
valid expressions, their instances are always plausible. Consequently, adding con-
straints on them would only slow down processing speed and increase db size for no-
thing.
BPR8 (No superfluous fundamental tables or rows). Fundamental tables should be
useful. If, for example, the instances of such a table are always empty, then that table
is superfluous. Similarly, if the set of values (the image) of a column of a non-empty
fundamental table is always empty, that column is superfluous. A fundamental table
on which no object depends upon, except for its triggers, might also be superfluous.
2.2.2 Implementation in Oracle
BPR9 (Reduce data contention). Critical tables, (almost) static ones, core ones
(to the enterprise, being used by several applications), very large ones, tempora-
ry ones, all of the others, and indexes of any db should be placed in distinct ta-
blespaces (and all of them also distinct from the system ones) for optimal fine-
tuning (e.g. caching all frequently used small static tables in memory see BPR10
below, setting adequate block sizes, etc.), and thus performance.
35

Whenever several hard disks are available, tablespaces should cleverly exploit
them all (e.g. storing critical tables tablespace on the fastest one). Create asso-
ciated data files with auto extension enabled, rather than creating many small
ones. Separate user data from system dictionary data to reduce I/O contention.
As data contention can substantially hurt application performance, reduce it by
distributing data in multiple tablespaces and partitions, avoid constant updates of a
same row (e.g. to calculate balance), and run periodic reports instead.
BPR10 (Caching small frequently used tables). Always cache small, especially
lookup static (but not only) tables that are frequently used.
BPR11 (Use correct data types). Using incorrect data types might decrease the ef-
ficiency of the optimizer, hurt performance, and cause applications to perform unne-
cessary data conversions. Dont use strings to store dates, times or numbers. Ensure
that conditional expressions compare the same data types. Do not use type conversion
functions (such as TO_DATE or TO_CHAR) on indexed columns: use instead the
functions against the values being compared to a column.
BPR12 (Not adding unnecessary indexes). Indexes should not be added either
for small instances tables or for columns containing mostly nulls in which you do
not search for NOT NULL values. Obviously, there should not be more than one
index on same columns (although there are RDBMSs allowing it! Fortunately, for
example, Oracle does not.).
BPR13 (Concatenated indexes column order). For concatenated indexes, the
order of their columns should be given by the cardinality of their corresponding
duplicate values: the first one should have the fewest duplicates, whereas any
other one should have more duplicates than its predecessor and fewer than its
successor; columns having very many duplicates or NULLs should then be ei-
ther placed last, or even omitted from indexes.
BPR14 (Indexes best types). Normal (B-Trees) indexes should be used except for
small range of values, when bitmap ones should be used instead. Note, however,
that bitmap indexes are greatly improving queries, but are significantly slowing
down updates and that they are available only for Enterprise Oracle editions
and not for the Standard ones too.
BPR15 (Indexes effectiveness). For improving indexes effectiveness, DBAs
should regularly gather statistics on them. (Note that on some RDBMS e.g.
Oracle starting with 10g this can be scheduled and performed automatically.)
36

BPR16 (Avoid row chaining). If pctfree setting is too low, updates may force
Oracle to move rows (row migration). In some cases (e.g. when row length is
greater than db block size) row chaining is inevitable. When row chaining and
migration occur, each access of a row will require accessing multiple blocks, im-
pacting the number of logical reads/writes required for each command. Hence,
never decrease pctfree and always use largest block sizes possible.
2.2.3 Querying and manipulating
BPR17 (Use parallelism). Whenever possible, be it for querying and/or mani-
pulating, use parallel programming.
BPR18 (Avoid dynamic SQL). Whenever possible, avoid dynamic SQL and use
views and/or parameterized stored procedures instead; when it is absolutely ne-
eded, keep dynamicity to the minimum possible (i.e. keep dynamic only what
cannot be programmed otherwise and for the rest use static SQL) and prefer the
dynamic native one, introduced as an improvement on the DBMS_SQL API, as it is
easier to write and executes faster.
Note that DBMS_SQL is still maintained because of the inability of native dyna-
mic SQL to perform so-called "Method 4 Dynamic SQL", where the name/ number of
SELECT columns or the name/number of bind variables is dynamic.
Also note that native dynamic SQL cannot be used for operations performed on a
remote db.
Views and stored procedures have obvious advantages: not only they are al-
ready parsed, but they also have associated optimized execution plans stored
and ready to execute.
BPR19 (Avoid subqueries). Subqueries allow for much more elegant and close
to mathematical logic queries, but are generally less efficient than corresponding
equivalent join queries without subqueries (except for cases when RDBMSs opti-
mizers replace them with equivalent subqueryless queries).
BPR20 (Use result cache queries). For significant performance improvement of
frequently run queries with same parameter values, always use result cache que-
ries and query fragments: as their results are cached in memory (in the result
cache, part of the shared pool), there is no more need to re-evaluate them (tri-
vially, except for the first time and then only for each time when underlying da-
ta is updated).
37

BPR21 (Use compound triggers). Always use compound triggers instead of or-
dinary ones, not only for new code, but also by replacing existing ordinary ones,
whenever possible: they improve not only codding efficiency, but also processing
speed for bulk operations.
BPR22 (Set firing sequences). Always use, whenever necessary, setting the
triggers firing sequence (clause FOLLOWS), in order to control their execution or-
der.
BPR23 (Use function result cache). Whenever appropriate, use result cache
functions, which enhances corresponding code performance (e.g., according to Ora-
cle, with at least 40% and up to 100% for most of pure Oracle PL/SQL code).
2.2.4 Optimization
BPR24 (Regularly shrink tables and tablespaces). Whenever rows are deleted
from table instances, physical disk space they occupy is not freed, as, in fact, Oracle
only marks them for deletion. Not only to gain storing space, but especially to speed
up queries, you should regularly shrink involved tables and tablespaces.
BPR25 (Regularly gather statistics on indexes). For improving indexes effective-
ness, DBAs should regularly gather statistics on them (by invoking stored procedures
DBMS_STATS.GATHER_SCHEME_STATISTICS and DBMS_STATS.GATHER
_TABLE_STATISTICS). Note that, starting with 10g, this can be scheduled and per-
formed automatically by the GATHER_STATS_JOB of the Automatic Statistics Col-
lection tool. Moreover, you might use the COMPUTE STATISTICS clause of the
CREATE INDEX statement.
BPR26 (Use key-compressed indexes). Especially for unique multicolumn inde-
xes, key compression should always be used for obtaining smaller and faster indexes:
they eliminate repeated occurrences of key column prefix values, by sharing prefix
entries among all suffix entries in index blocks. Note that the COMPRESS clause can
be specified during either index creation or rebuild.
BPR27 (Follow Oracle advisors recommendations). Oracle advisors are almost
always right, so the best thing to do is to regularly monitor and do what they are sug-
gesting, preferably automatically; failed ones should especially be monitored and ne-
cessary actions should be taken a.s.a.p.

38

3 DBDetectiveOs architecture
The 3-tier architecture of DBDetectiveO is presented in figure 1:
Windows 7/8
GUI (MS Access 2010 Menus, Forms & Reports)
MS Access 2010
BL
Servers & dbs management (VBA) Investigations parameterization

Sequences, triggers, db links (SQL & PL/SQL) Views, stored procedures
Oracle 11g
RedHat Linux 6.2

DB (Oracle 11g)
Figure 1: DBDetectiveOs o ve r a l l architecture
A detailed DBDetectiveOs architecture is presented in figure 2.
The light GUI, developed in MS Access 2010, provides users with a simple,
three levels menu and forms/reports for displaying/printing mainly investigation
results, but also for accepting corresponding parameter values (e.g. the desired
thresholds for caching tables or compressing indexes), as well as managing Ora-
cle target servers and dbs (users) needed data (server names, connection strings,
dbs names, users accounts and passwords, IPs, etc.), and additional semantic deci-
sion support data; it also provides radio buttons for selecting desired detection op-
tions (e.g. full list of subsection 1.2 above or only partial ones) and buttons for
launching metadata mining, investigations, etc.

39

ODBDetective.accdb
MS Access 2010
Menu, Forms, Reports (GUI )
Parameters, servers and dbs data Servers, dbs, and
inserts/updates/deletes, GUI events investigation data
MS VBA (BL)
Servers management form class Investigation forms classes
DBs management form class

General library class


middlewares
MS ADO
MS ODBC Driver Manager
Oracle ODBC Driver for Win64
Oracle 11g Client
Workstation 1 Workstation i Workstation n

LAN LAN

VPN



Customer 1 VPN I nternet VPN Customer m


ORACLE DBDETECTI VEO (DB & BL) ORACLE
DB1.1 SQL and PL/SQL tables, views, triggers, sequences, db links, etc. DBm.1
DB1.k DBm.p
Figure 2: DBDetectiveOs detailed architecture
40

GUI-triggered events are handled by the VBA BL tier sub-layer, which is made up
of three class forms (for servers, dbs, and investigation options respectively) and a
library used by all of them. This sub-layer contains methods for adding, updating,
and deleting data on targeted servers and dbs, deleting no more needed investigation
data, adding (and even updating catalogue metadata) for the currently selected
db, etc.
As usual, especially when extended SQL is available (PL/SQL in this case), there
is also a RDBMS engine-based BL sub-layer, which, in this case, mainly includes de-
tection and investigation views, but also triggers, sequences, stored functions and
procedures, etc.
Between these two BL sub-layers, a four level middleware pack is needed
to bridge MS and Oracle, which comprises ADO (included in Access), MS
ODBC Driver Manager, Oracle ODBC Driver for Win64, and Oracle Client.
All MS Access 2010 layers of the solution are encapsulated in a pure code
(except for the table which drives the menu) concurrent db called DBDetec-
tiveO.accdb, which can be deployed on any number of workstations having
(VPN) access to an Oracle 11g server where the DBDetectiveOs SQL &
PL/SQL BL sub-layer, as well as its DB layer reside.
Through db links, DBDetectiveO can mine and import metacatalog data
from any number of other Oracle servers dbs too.

41

4 DBDetectiveOs database design and implementation
DBDetectiveOs db design and implementation were done according to the algo-
rithms presented in [2].
4.1 Design
For investigating possible violations of the best practice rules presented in section
1.1 above by Oracle db schemes, DBDetectiveO has to mine in the corresponding
server catalog for metadata on dbs (users), tablespaces, objects, their dependencies,
tables, views, columns, constraints, indexes, their columns, triggers, and PL/SQL
stored functions and procedures code. Fortunately, Oracle provides both DBA and
user versions of views for all such metadata, from which DBDetective extracts only
relevant columns into its own corresponding db tables. Besides the above corres-
ponding object sets, DBDetectiveO offers some dictionary-type ones (for decrypting
Oracle internal codifications), as well as a set of Oracle servers and one of dbs (just
like, for example, Oracles SQL Developer, for saving needed connections data).
4.1.1 E-R Diagram and restrictions list
Figure 3 presents the DBDetectiveOs structural Entity-Relationship (E-R) Dia-
gram (E-RD).
Note that, for graphical reasons, canonical injections are represented as
instead of c . Also note that, again for graphical reasons, although indexes are Ora-
cle objects too, in figure 3 this inclusion (INDEXES c OBJS) is not depicted.
Also note that, except for the properties of SERVERS and DBS, all other ones (ex-
cept for some few properties added for future extensions and *DB of OBJS, as well
as *Tablespace of OBJS, which are computed) are obtained through data mining, so
they are read-only: consequently, there are no restrictions on them.


42

*Server
SERVERS TABLESPACES DEPENDENCIES
Server *DB *Tablespace
TBL_TYPES DBS OBJS OBJ_TYPES
Type Type Package
TABLES TRIGGERS VIEWS PCK_BODIES METHODS
Table Column View Package Method
COLUMNS VIEWS_COLS *COLUMNS PACKAGES SOURCE_CODE

INDX_COLS CNSTR_COLS CONSTRAINTS
Type
INDEXES CNSTR_TYPES
Figure 3: DBDetectiveOs structural E- RD

Note too that the Package property of METHODS (which abstracts both stored
PL/SQL functions and procedures) is not compulsory (as they may be methods de-
fined outside PL/SQL packages). Perhaps the most important thing to note is that
Oracle metacatalogs are uni-server: although all of its g versions are supporting grids
of interconnected servers, there is no centralized metacatalog: consequently, DBDe-
tectiveO had to add to its db computed property *Server of TABLESPACES.
In fact, Oracle does not abstract PL/SQL stored function and procedures into a
same method set (so, for DBDetective, METHODS = FUNCTIONS PROCE-
DURES), neither differentiates between views and tables columns (although most of
the views ones are obtained from tables ones), nor, unfortunately, stores for each
PL/SQL source code line the method to whom it belongs, but only the corresponding
package body.

43

The associated restrictions list (RL) is the following one:

SERVERS (the set of Oracle db servers of interest)
RL1 (servers max. cardinality): 200
RL2 (servers domain ranges):
ServerName: ASCII(64)
HostIP: ASCII(16)
SID, ServiceName, Pw: ASCII(30)
Port: [1521, 1575] _ NAT(4)
SMode: {Dedicated, Shared}
Protocol: {TCP, IPC}
*ConnectionString: ASCII(255), computable according to the template:
(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=Protocol)(HO
ST= HostIP)(PORT= Port)))(CONNECT_DATA=(SERVER=SMode)(SERVICE
_NAME= SID/ServiceName)))
RL3 (servers mandatory properties): ServerName, HostIP, Port, SMode,
Protocol
RL4 (servers uniqueness restrictions): ServerName are HostIP are unique (there
may not be two servers with either same name or same host address); note that, gene-
rally, according to *ConnectionString definition, HostIP is not a key (but Protocol -
HostIP - Port - SMode - SID and Protocol - HostIP - Port - SMode - ServiceName
are instead), as several connections to a same server are allowed (for example with
different protocols and/or modes); obviously, DBDetectiveO only needs one per ser-
ver, which is why HostIP is unique for it (and, consequently, Protocol - HostIP -
Port - SMode - SID and Protocol - HostIP - Port - SMode - ServiceName are super-
keys).
RL5 (servers non-existence restrictions): whenever SID is null, ServiceName
may not be null and vice-versa (i.e. whenever ServiceName is null, SID may not be
null) (that is either SID or ServiceName should be specified, but not both of them)


44

DBS (the set of Oracle dbs of interest)
RL6 (dbs max. cardinality): 20000
RL7 (dbs domain ranges):
DB: {[A-Z], [0-9], _}(30)
Prefix(DB, 4) e {SYS_, ORA_} (Oracle names may not exceed 30 cha-
racters, which should be either capital letters, numbers, and/or underscore; moreover,
their prefixes may not be SYS_ or ORA_)
Pw: ASCII(30)
*Link_Name: ASCII(128), computable according to the template: DB & "#" &
Server
RL8 (dbs mandatory properties): DB, Server, Pw
RL9 (dbs uniqueness restrictions): Server and DB are unique (there may not be
two dbs with same name on a same server)

OBJ _TYPES (the quasi-static set of Oracle db object types of interest)
RL10 (object types max. cardinality): 32
RL11 (object types domain ranges):
ObjType: {Function, Index, Java Class, Package, Package Body, Pro-
cedure, Synonym, Trigger, Type, View}
RL12 (object types mandatory properties): ObjType
RL13 (object types uniqueness restrictions): ObjType is unique (there may not be
two Oracle object types with same name)

CNSTR_TYPES (the quasi-static set of Oracle db constraint types of interest)
RL14 (constraint types max. cardinality): 16
RL15 (constraint types domain ranges):
CnstrCode: {C, F, H, O, P, R, S, U, V}
CnstrType: {Check, REF, Hash, R/O, Primary, Referential,
Supplemental, Unique, View}
CnstrDescr: { Check constraint on a table, Constraint involving a REF
column, Hash expression, Read-only view, Primary key, Referential
integrity (foreign key), Supplemental logging, Unique constraint (non-pri-
mary superkey), Check constraint on a view}
RL16 (constraint types mandatory properties): CnstrCode, CnstrType
45

RL17 (constraint types uniqueness restrictions): CnstrCode, CnstrType,
CnstrDescr are unique (there may not be two Oracle constraint types with same
code, name, or description)

TBLS_TYPES (the quasi-static set of DBDetective table types)
RL18 (table types max. cardinality): 8
RL19 (table types domain ranges):
Code: {B, C, G, N, R, U, Y, l}
Comments: { To be reviewed, Critical table, Not used anymore: to be
dropped, Rarely used, Small table that can grow large, Used table,
Small table to be cached, System tables: null value}
RL20 (table types mandatory properties): Code, Comments
RL21 (table types uniqueness restrictions): Code and Comments are unique
(there may not be two DBDetective table types with same code or comment)
4.1.2 Mathematical scheme
By applying the algorithms for translating E-RDs and restriction lists into mathe-
matical schemes and for detecting and designing constraints to the above E-RD and
restrictions list, the following refined mathematical scheme is obtained:
SERVERS
S# [-100, 100], total
ServerName ASCII(64), total
HostIP ASCII(16), total
SID ASCII(30)
ServiceName ASCII(30)
Pw ASCII(30)
Port [1521, 1575] _ NAT(4), total
SMode {Dedicated, Shared}, total
Protocol {TCP, IPC}, total
*ConnectionString ASCII(255) (superkey, as HostIP is a key)
*ConnectionString (x) = (DESCRIPTION=(ADDRESS_LIST=(ADDRESS= &
(PROTOCOL= & Protocol(x) & )(HOST= & HostIP(x) & )(PORT= & Port(x)
46

& )))(CONNECT_DATA=(SERVER = & SMode(x) & )(SERVICE_NAME= &
iif(Isnull(SID(x)), ServiceName(x), SID(x)) & )))
Initial keys (according to RL4) are {S#, ServerName, HostIP}.
Results of applying the algorithm for keys detection are:
Candidate Key? Prime? Proof
SID No No There may be any number of servers having
same SID; SID cannot take part in any SER-
VERS key.
ServiceName No No There may be any number of servers having
same ServiceName; ServiceName cannot take
part in any SERVERS key.
Pw No No There may be any number of servers having
same Pw; Pw cannot take part in any SERVERS
key.
SMode No No There may be any number of servers having
same SMode; SMode cannot take part in any
SERVERS key.
Protocol No No There may be any number of servers having
same Protocol; Protocol cannot take part in
any SERVERS key.
Consequently, SERVERS does not have any other keys than the three initial ones.
C5 (according to RL5):
(xeSERVERS)((SID(x) e NULLS ServiceName(x) e NULLS) .
(ServiceName(x) e NULLS SID(x) e NULLS))


47

DBS
D# [-10000, 10000], total
DB {[A-Z], [0-9], _}(30), total
Pw ASCII(30), total
*Link_Name ASCII(128)
*Link_Name(x) = DB(x) & "#" & Server(x)
Server : DBS SERVERS, total
C7 (according to RL7): (xeDBS)(Prefix(DB(x), 4) e {SYS_, ORA_})
Initial keys (according to RL9) are {D#, DB - Server}.
Results of applying the algorithm for keys detection are:
Candidate Key? Prime? Proof
DB No Yes There may be any number of dbs having
same names (on different servers); according
to RL9, DB takes part in the Server - DB
key.
Pw No No There may be any number of dbs having
same Pw (even on a same server); Pw can-
not take part in any DBS key.
Server No No There may be any number of dbs on a same
server; according to RL9, Server takes part
in the Server - DB key.
DB - Server
Yes According to RL9, there may not be two dbs
having a same name on a same server.
Consequently, DBS does not have any other keys than the initial one.

OBJ _TYPES
OT# [1, 32] c NAT(2), total
ObjType {Function, Index, Java Class, Package, Package Body,
Procedure, Synonym, Trigger, Type, View} c ASCII(16), total


48

CNSTR_TYPES
CT# [1, 16] c NAT(2), total
CnstrCode {C, F, H, O, P, R, S, U, V} c ASCII(1), total
CnstrType: {Check, REF, Hash, R/O, Primary, Referential,
Supplemental, Unique, View} c ASCII(16), total
CnstrDescr { Check constraint on a table, Constraint involving a REF co-
lumn, Hash expression, Read-only view, Primary key, Referential integrity
(foreign key), Supplemental logging, Unique constraint (non-primary super-
key), Check constraint on a view} c ASCII(64), total

TBLS_TYPES
TT# [1, 8] c NAT(1), total
Code {B, C, G, N, R, U, Y, l} c ASCII(1), total
Comment {To be reviewed, Critical table, Not used anymore: to be drop-
ped, Rarely used, Small table that can grow large, Used table, Small table to
be cached, System tables: null value} c ASCII(32), total

Applying the E-RD cycles detection and analysis algorithm, the following seven
cycles are discovered in figure 3:
1. the cycle having nodes DEPENDENCIES and OBJS
is of commutative type, but with length two, corresponding to the binary homogene-
ous relation DEPENDENCIES, which should obviously be acyclic (as no Oracle ob-
ject may depend either directly or indirectly on itself); as this constraint is enforced
by Oracle, no other explicit constraint needs to be added to the math scheme;
2. the cycle having nodes OBJS, DBS, TABLESPACES, SERVERS
is of commutative type, should commute, and commutes by the definition of compu-
ted mappings *Server, *Tablespace, *DB (Server *DB = *Server *Tablespace, as
any Oracle db object belonging to a db should reside in a tablespace belonging to the
same Oracle db server as the corresponding db): no other explicit constraint needs to
be added to the math scheme;

49

3. the cycle having nodes TRIGGERS, COLUMNS, TABLES, OBJS
is of commutative type, but should not commute, as, trivially, any trigger attached to
a table column is a different object than that table: no other explicit constraint needs
to be added to the math scheme;
4. the cycle having nodes TRIGGERS, COLUMNS, VIEWS_COLS, *COLUMNS,
VIEWS, OBJS
is of commutative type, but should not commute, as, trivially, any trigger attached to
a table column is a different object than any view whose columns are based on that
table column: no other explicit constraint needs to be added to the math scheme;
5. the cycle having nodes TABLES, COLUMNS, VIEWS_COLS, *COLUMNS,
VIEWS, OBJS
is of commutative type, but should not commute, as, trivially, any table is a different
object than any view whose columns are based on that table columns: no other expli-
cit constraint needs to be added to the math scheme;
6. the cycle having nodes INDEXES, INDX_COLS, COLUMNS, TABLES, OBJS
(not depicted, but existing)
is of commutative type, but should not commute, as, trivially, any index (built on a
tables columns) is a different object than any view whose columns are based on that
tables columns: no other explicit constraint needs to be added to the math scheme;
7. the cycle having nodes INDEXES, INDX_COLS, COLUMNS, TRIGGERS,
OBJS (not depicted, but existing)
is of commutative type, but should not commute, as, trivially, any index (built on a
tables columns) is a different object than any trigger attached to a column of that
table: no other explicit constraint needs to be added to the math scheme.
4.1.3 Relational scheme and associated non-relational constraints list
By applying the algorithm for translating math schemes into correspondent rela-
tional ones and associated non-relational constraint lists to the math scheme above,
the results are the following:

50

SERVERS (S#, Server, HostIP)
S# ServerName HostIP Pw SID ServiceName Port SMode Protocol *ConnectionString
auton.,
[-100,
100]
ASCII(64) ASCII(16) ASCII(30) ASCII(30) ASCII(30) [1521, 1575] {D,S} {TCP,IPC} ASCII(255)
NOT
NULL
NOT NULL NOT
NULL
NOT NULL NOT NULL NOT NULL

DBS (D#, DB - Server)
D# Server DB Pw *LinkName
auton., [
-10000,
10000]
Im(SER
VERS.S#
)
{[A-Z],
[0-9],
_}(30)
ASCII(30) ASCII(128)
NOT
NULL
NOT
NULL
NOT NULL NOT
NULL

OBJ _TYPES (OT#, ObjType)
OT# ObjType
auton., [1, 32] ASCII(16)
NOT NULL NOT NULL

CNSTR_TYPES (CT#, CnstrCode, CnstrType,
CnstrDescription)
TT# CnstrCode CnstrType CnstrDescription
auton., [1,
16]
ASCII(1) ASCII(16) ASCII(64)
NOT
NULL
NOT NULL NOT NULL NOT NULL

TBLS_TYPES (TT#, Code, Comment)
TT# Code Comment
auton., [1, 8] ASCII(1) ASCII(32)
NOT NULL NOT NULL NOT NULL
51

Except for a few added columns, the following table schemes were automatically
created by executing SQL CREATE TABLE AS SELECT from Oracle views:
TBLSPACES(x, Server, Block_Size, Initial_Extent, Next_Extent, Min_Ex-
tents, Max_Extents, Max_Size, PCT_Increase, Min_ExtLen, Status, Contents,
Extent_Management, Allocation_Type, Segment_Space_Management, Def_Tab
_Compression, Retention, BigFile, Predicate_Evaluation, Encrypted, Compress_
For);
OBJ S(x, Owner, ObjectName, SubObject_Name, Object_ID, Data_Object
_ID, Object_Type, Status, Temporary, Generated, Secondary, NameSpace,
Edition_Name, DB);
DEPENDENCI ES(x, Object, Referenced_Object, Referenced_Link_Name,
Dependency_Type);
TABLES(x, Object, Tablespace, Cluster, IOT_Name, Status, PCT_Free, PCT
_Used, Ini_Trans, Max_ Trans, Initial_Extent, Next_Extent, Min_Extents, Max_
Extents, PCT_Increase, FreeLists, FreeList_Groups, Cardinal, Blocks, Empty_
Blocks, Avg_Space, Chain_Cnt, Avg_Row_Len, Avg_Space_ Freelist_Blocks,
Num_FreeList_Blocks, Degree, Instances, Cache, Table_Lock, Sample_Size, Par-
titioned, IOT_Type, Temporary, Secondary, Nested, Buffer_Pool, Flash_Cache,
Cell_Flash_Cache, Row_Movement, Global_Stats, User_Stats, Duration, Skip_
Corrupt, Monitoring, Cluster_Owner, Dependencies, Compression, Compress_
For, Dropped, Read_Only, Segment_Created, Result_Cache, Static, Fundamen-
tal, Used_in_SP);
TBL_COLUMNS(x, Object, Data_Type, Data_Length, Data_Precison, Data
_Scale, Nullable, Default_Length, Num_Distinct, Low_Value, High_Value, Num
_Nulls, Num_Buckets, Char_Col_Decl_Length, Avg_Col_Len, Char_Length, Char
_Used, Data_Type_Owner, Column_ID, Density, Sample_Size, Character_Set_
Name, Global_Stats, User_Stats, V80_FMT_Image, Data_Upgraded, Histogram,
Computed, Needs_Index, Index_Type, Join_Index);
CONSTRAI NTS(x, Object, Table, Constraint_Type, Index, R_Constraint,
Status, Validated, Delete_Rule, Bad, View_Related, Invalid, Rely, Generated,
Deferred, Deferrable);
CONSTR_COLS(x, Constraint, Column, Position);
I NDEXES(x, Object, Index_Type, Table, Uniqueness, Compression, Prefix_
Length, Tablespace, Ini_Trans, Max_Trans, Initial_Extent, Next_Extent, Min_
52

Extents, Max_Extents, PCT_Increase, PCT_Threshold, FreeLists, FreeList_
Groups, Include_Column, PCT_Free, BLevel, Leaf_Blocks, Distinct_Keys, Avg_
Data_Blocks_Per_Key, Avg_Leaf_Blocks_Per_Key, Clustering_Factor, Status,
Num_Rows, Sample_Size, Degree, Instances, Partitioned, Temporary, Genera-
ted, Secondary, Buffer_Pool, Flash_Cache, Cell_Flash_Cache, User_Stats, Global
_Stats, Duration, PCT_Direct_Access, Parameter, DomIdx_Status, DomIdx_
OpStatus, FuncIdx_Status, Join_Index, IOT_Redundant_PKey_Elim, Dropped, Visi-
bility, DomIdx_Management, ToBeAdded, Used, BitMap, ToReorder);
I NDEXES_COLS(x, Index, Column, Column_Position, Descend, Correct_Co-
lumn_Position);
TRI GGERS(x, Object, Trigger_Type, Triggering_Event, Triggering_Object,
Column, Referencing_Names, When_Clause, Status, Description, Action_Type,
CrossEdition, Before_Statement, Before_Row, After_Row, After_Statement, In-
stead_of_Row, Fire_Once, Apply_Server_Only);
VI EWS(x, Object, Text_Length, Type_Text, OID_Text_Length, OID_Text,
View_Type_Owner, View_Type, Superview, Editioning_View, Read_Only, Edi-
tion_Name);
SOURCE_CODE_LI NES(x, Name, Type, Line, Text).

The associated non-relational constraints list is the following:
C5 (according to RL5): (xeSERVERS)((SID(x) e NULLS ServiceName(x) e
NULLS) . (ServiceName(x) e NULLS SID(x) e NULLS))
C7 (according to RL7): (xeDBS)(Prefix(DB(x), 4) e {SYS_, ORA_})
4.1.4 Sequences, triggers, db links, stored procedures, and views
4.1.4.1 Autonumbers in Oracle
Unfortunately, Oracle does not provide autonumbers; in order to overcome this,
sequences and triggers have to be designed and implemented, one for each involved
table: for each insert in a such a table, if no value is specified for the corresponding
column, then the trigger assigns to it the current value of the corresponding sequence,
which is then incremented by one.
53

4.1.4.2 Implementing non-relational constraints in Oracle
Fortunately, non-relational constraints may be implemented in Oracle by also
using triggers. However, the above list of non-relational constraints is implemented
in DBDetectiveO in VBA, which can provide users with context sensitive and im-
mediate error messages (whilst Oracle is returning only context independent error
messages, which are not immediate, at cell level, but always delayed at the row one).
4.1.4.3 Connecting dbs in Oracle
Fortunately, in Oracle you may access and process data of any dbs from within
any other one, provided that a valid db link is defined in the host db for each such
remote ones. Note that remote dbs need not to be on the same Oracle db server
as the host one. DBDetectiveO fully exploits this feature for remotely mining from
its Oracle db any metacatalog data that its users desire to investigate.
4.1.4.4 Mining for remote metadata
Using Oracles db links, DBA and user views, as well as parameterized stored
procedures (the parameters being db links), DBDetectiveO is filling its db with all ne-
eded metadata for investigation, thus impacting very few (only in read-only mode,
only from the metacatalogs, only once) on the customers investigated Oracle remote
servers, dbs, and network bandwidth (typically, the size of the DBDetectiveOs db is
only some 200MB).
4.1.4.5 Investigating mined metadata
Mined metadata is investigated by DBDetectiveO with the aid of (hierarchies of)
views, built upon its db.
4.2 Implementation
4.2.1 Tables and constraints
All DBDetectiveOs tables and constraints were implemented according to the al-
gorithm for translating relational schemes and associated non-relational constraint
54

lists into Oracle 11g dbs, by using Oracle SQL Developer Edit Table Advanced op-
tion (see figure 4):

Figure 4: The advanced Edit window of the Oracle SQL Developer
Note that, unfortunately, for tables SERVERS and DBS whose instances are ma-
naged by the DBDetective MS Access application, their constraints had to be also im-
plemented in VBA: in order to avoid getting the Oracle context-insensitive corres-
ponding error messages, VBA is enforcing all these constraints too.
Also note that if you declare NOT NULL constraints in Oracle, when users are
trying to leave such a column cell null, unfortunately, corresponding VBA BeforeUp-
date trigger-type methods are not invoked, as ADO is first passing nulls to Oracle,
which is rejecting them, with the following standard Access not so fortunate error
message:

55

Consequently, the only way to give users context sensitive error messages in such
cases (and, moreover, to help them by undoing accidentally emptied cells) is not to
enforce NOT NULL constraints in Oracle, but only in VBA. This is why DBDetec-
tiveOs NOT NULL constraints are not enforced in Oracle.
Dually, but for the same reason of displaying context sensitive error messages and
undoing implausible data updates, as well as because of the fact that Oracle, ODBC,
and ADO error trapping in VBA is problematic, the rest of the DBDetectives relatio-
nal type constraints are enforced both in Oracle and VBA.
For example, there are times when Oracle or ODBC errors are not triggering ADO
and/or Access corresponding errors; much worse, there are even contexts (e.g. con-
necting to an Oracle db) when, dually, although there is no Oracle, ODBC, or Access
error, ADO is however reporting an error!
4.2.2 Sequences, triggers, db links, stored procedures, and views
All DBDetectiveOs sequences, triggers, db links, stored procedures, and views
were implemented by using Oracle SQL Developer (see figures 5, 6, 7, 8, and 9 be-
low).

Figure 5: The Oracle SQL Developer window: managing sequences
56


Figure 6: The Oracle SQL Developer window: managing triggers

Figure 7: The Oracle SQL Developer window: managing db links
57


Figure 8: The Oracle SQL Developer window: managing stored procedures


Figure 9: The Oracle SQL Developer window: managing views


58

5 DBDetectiveOs application design and implementation
5.1 Design
DBDetectiveOs application design was done according to the principles and me-
thodologies presented in [30], [31], [32], [33], and [34].
5.1.1 Business Model and Use Cases
It was decided that the best software life cycle that suits DBDetectives needs is
the Prototype Software Development Life Cycle, as there is only one developer:

Figure 10: Prototype Software Development lifecycle diagram
Using Software Prototyping, several DBDetective prototypes (that is incomplete
versions) were created, such as to receive feedback from my coordinating Professor
Christian Mancas for each of them.
DBDetective application is developed using Incremental Prototyping: the applica-
tion is built as separate prototypes that are merged at the end in an overall design.
5.1.1.1 Business Model
DBDetectives main purpose is to help stakeholders (owners, customers, manage-
ment, etc.) to better understand the business functions and also promote business im-
provements and/or innovation.

59

Business Actors
Actors roles played by external parts of the system (humans, hardware or soft-
ware systems).
The only human actor that will interact with the DBDetective application is the
User, who analyzes db schemes for detecting anomalies of their design, implementa-
tion, and usage, and provides statistics and decision support data, for improving dbs
performance by correcting their schemes in order to eliminate best practices viola-
tions.
Functions provided by the system
Non-functional requirements:
- Client dbs must be Oracle ones.
- Users computers must have MS Access 2010 or later, Oracle Client 11g or la-
ter, and Oracle ODBC driver for Windows (32 or 64 bits depending on its ope-
rating system platform) installed.
- Users should be trained to use the DBDetective application.
Functional requirements:
- The system has to mine each customer Oracle dbs metadata.
- The system has to save the interesting part of this metadata into the Oracle
DBDetectives db.
- The system has to provide SQL views and PL/SQL stored procedures for inves-
tigating metadata according to best practice rules from subsection 1.1.
- The system has to provide forms for presenting investigations results and also
for storing additional semantic and decision support data.

60

5.1.1.2 Use Cases
Here are the DBDetectiveO systems use cases (see figure 11):
Use Case 1: Add Server Connection;
Use Case 2: Add Database Link;
Use Case 3: Edit Server Connection;
Use Case 4: Edit Database Link;
Use Case 5: Set Database to Investigate;
Use Case 6: Mine Target db Metadata;
Use Case 7: Inspect Mined Metadata;
Use Case 8: Inspect Oracle Advisor Log;
Use Case 9: Inspect Oracle Advisor Recommendations;
Use Case 10: Investigate Target db Issues;
Use Case 11: Delete Database Link;
Use Case 12: Delete Server Connection.
5.1.2 Menu, forms, and library design
DBDetectiveOs menu is a very simple one, consisting only of three levels; figure
12 presents its architecture. Note that there are only two non-terminal nodes (to be
implemented as switchboards); all of the rest are leaf nodes (to be implemented as
forms); arrows indicate inclusion (of sub-forms in forms: for example, form Servers
& Dbs Management includes form DBs as a linked sub-form). Note that forms Ta-
bles, Columns, and Triggers and Constraints are provided both as standalone ones, as
well as sub-forms of the Table Details form; form Indexes (with its sub-form Index
Columns) is a sub-form of both Tablespaces, Indexes, and their Columns and Table
Details forms.
The standard switchboard type Access menu will be used, mainly with only GUI
improvements of its form, but also two programming ones: for exiting the application
Close DB (which leaves Access open) is replaced by Quit (which closes Access too)
and, for displaying forms with their corresponding instances too in datasheet mode,
Add is replaced by Edit, but with Datasheet format.
61


Figure 11: DBDetectiveOs use case diagram
Besides the switchboard form, only two other types of forms are needed too: those
for the investigation results and those for managing DBDetectiveOs own metadata.
The rationale of the former is that investigation data should be protected (so pre-
sented in read-only mode to users), while decision columns, which are added to the
investigation ones obtained through mining, have to be managed in read-write con-
trolled mode. Each investigation view (see 4.1.2 above) has one such form associated
to it.
62

DBDetectiveO Main Menu

Current DB Investigation Mined Metadata Oracle Advisors Recommendations Oracle Advisors Log Manage Servers&DBs
Submenu
Global Statistics Objects Dependencies DBs
Constraint Types Tablespaces Indexes Index Columns
Table Types Tables Columns Triggers
Investigation Constraints Constraint Columns
Results 40 Forms Views
Package Sequences
Table Details PL/SQL Code Lines

Figure 12: DBDetectiveOs menu architecture
63

On top of them, one investigation manager form only contains radio buttons for
allowing users to define their desired investigation patterns, as well as an associated
class whose methods are asking users for investigation parameters, are validating
their values, and are calling corresponding views and parameterized stored proce-
dures.
The rationale of the latter is that DBDetectives own metadata has to be managed
in read-write controlled mode. Apart from forms Servers & Dbs Management, Con-
straint Types, and Table Types, DBDetective also needs to manage in read-write con-
trolled mode decision data columns stored in about half of the investigation results ta-
bles (again, obviously, through their corresponding forms see subsection 6.6.2).
In order to avoid hard-codding and re-definitions, all commonly used constants
and methods are grouped in a General VBA library.
5.2 Implementation
DBDetectives menus, forms, and library were developed in MS Access 2010 and
VBA (which embeds SQL and ADO too). Before anything else, the bridge between
Access 2010 and Oracle 11g had to be installed and configured (see 6.1.2 below).
For creating ODBDETECTIVE.accdb, start MS Access, select "Blank database",
provide the filename (ODBDETECTIVE.accdb), and click on "Create":

Figure 13: MS Access 2010: creating a new db
64

A new db was created (with only one local table, Table1, opened in edit mode):

Figure 14: MS Access 2010: opening a newly created db
Then, go to the "External data" tab on the ribbon menu and select the "ODBC da-
tabase" button: the "Get External Data ODBC Database" window will be dis-
played with the two available options (select the second one):
Import source data into new tables in the current db (not only tables: views too
are imported as tables! those that cannot be successfully executed at import time are
saved in a Name AutoCorrect Save Failures table; moreover, not only views defi-
nitions are not imported, but no relationships between tables or constraints are either!
Consequently, this option is fitted only for OLAP-type Access processing), or
Link to the data source by creating linked tables: this is the option to select
normally:

65


Figure 15: MS Access 2010: creating links to tables stored remotely
Then, click the "OK" button: a "Select data source" window is displayed; change
the tab from "File data source" to "Machine data source" and select the ODBDE-
TECTIVE db DSN (see figure 16 below).

Figure 16: MS Access 2010: selecting the machine data source ODBC driver
After a while, we will see the "Link Tables" window, which presents all the tables
and views that resides on the Oracle 11g server:
66


Figure 17: MS Access Link Table Manager: browsing Oracle db tables and views
Obviously, as we don't need them all, select only the ones belonging to the
ODBDETECTIVE user (keep Ctrl pressed and click on each one of them; for the sake
of convenience, check the "Save password" checkbox too) and then click "OK":

Figure 18: MS Access Link Table Manager: selecting desired Oracle tables and views
We can ignore security messages if we are connected through a secure VPN con-
nection, by pressing the "Save Password" button each time:
67


Figure 19: MS Access Link Table Manager: saving Oracle db password
When the "Select Unique Record Identifier" window appears, select the column(s)
which should be treated as the primary key unique identifier (required if you want to
manipulate corresponding instances; otherwise, they are read-only!) for every table
and updatable view:

Figure 20: MS Access Link Table Manager: selecting Oracle table/view primary key
After a while, all ODBDETECTIVE schema tables and views are linked from MS
Access to the Oracle db server, via ODBC. We can see them all on the left "Tables"
pane; to open any of them, just double-click on its name:
68


Figure 21: MS Access: Tables pane with linked Oracle tables/views and a view instance
5.2.1 Menu development
For the application menu, the Access Switchboard Manager was used:

Figure 22: An example of DBDetectives main menu implementation in Access
The standard switchboard was then modified both graphically, and by changing its
embedded macro by replacing the standard Close DB command (which lets Access
open) with the Quit one (which also closes Access) for the menu Quit button:

69


Figure 23: An example of DBDetectiveOs main menu GUI enhancement in Access
5.2.2 Forms development
Similarly, for the forms, the Access Form Wizard was initially used, but both their
GUI and properties were enhanced, and associated VBA classes were added. Figure
24 presents an example of GUI enhancement, figure 25 one of properties enhance-
ments (LINK_NAME was declared read-only), and figure 26 presents, as an example,
the beginning of the VBA class associated to the form DBS (a sub-form of SERVERS).
70


Figure 24: An example of DBDetectiveOs Servers & dbs form GUI enhancement in Access

Figure 25: An example of DBDetectives DBS form properties enhancement in Access
71


Figure 26: An example of DBDetectives DBS form class VBA method
5.2.3 Library development
For developing DBDetectives General library, the VBA IDE was used too:

Figure 27: An example of DBDetectives General library VBA method
72

Please note that, generally, VBA programming for ODBC-connected back-ends is
more demanding than for MS Jet or ACME, or even SQL Server ones: for example,
re-querying and even refreshing are not automatic (but need explicit programming),
ADO, so Oracle too, are taking precedents in the events chain over VBA trigger-like
methods (which, normally, is not the same).
Oracle table schemes default values are not automatically copied into Access
forms, VBA object properties like NewRecord and OldValue are not available, etc.;
moreover, as Oracle does not provide auto-numbering, you should first define corres-
ponding Oracle sequences and triggers and then you should carefully program in
VBA all corresponding methods, as these Oracle trigger generated surrogate key va-
lues are only generated just before inserting new rows in Oracle tables (and are null
before), whereas Access auto-numbering generates them before saving any new data
in virtual memory, long before saving it to disk, in dbs.
6 DBDetectiveOs Users Guide
6.1 Installing, uninstalling, and configuration
DBDetectiveO is available on a CD containing both the DBDetective.sql Oracle
11g SQL script for creating and initializing its back-end db and the ODBDetec-
tive.accdb MS Access front-end application, as well as its present Users Guide.
Minimum hardware and software requirements are the following: at least 0.5GB
free available space for an Oracle 11g db server, in order to create and populate
DBDetectiveOs db; a 64-bit computer running 64-bit MS Windows 7/8 and Access
2010/2013, with at least 10MB of free disk space for storing ODBDetective.accdb
and another 1.5GB for the Oracle Client 11g and ODBC driver for Win64 (if not al-
ready installed).
6.1.1 Oracle 11g (or newer) script execution for creating and initializing
the db
For creating and initializing DBDetectiveOs Oracle db, you should, for example,
copy the DBDetective.sql file from the distribution CD to a new SQL tab of the
Oracle SQL Developer (see figure 28 below) and execute (run) it (which can also be
73

done from SQL*Plus). The Developer can freely be downloaded from the Oracle
website.

Figure 28: Creating DBDetectiveOs db with the Oracle SQL Developer
Note that, in order to create the db, you must have the CREATE USER system
privilege.
If everything is going ok, you will see lot of messages displayed in the Oracle
SQL Developer Results pane, but no error one, starting with:
user DBDETECTIVE created.
GRANT succeeded.
GRANT succeeded.
and ending with:
1 rows inserted.
6.1.2 Installation and configuration of the MS Access 2010 to Oracle 11g
bridge
To download Oracle Client software (see http://www.oracle.com/technet
work/database/enterpriseedition/downloads/112010-win64soft094461.html) first ac-
cept the OTN License Agreement and then use your OTN account (which can freely
be created). Choose your MS Windows version in our case, 64-bit (that is file
win64_11gR2_client.zip):
74


Figure 29: Downloading Oracle Client 11g
Then, extract corresponding zipped files in the folder in which you store your in-
stallation versions:

Figure 30: Unzipping Oracle Client 11g
Note that the Oracle Client is not certified for Windows 8, but, however, it is
working. Also note that Java JDK (not JRE) and the MS .NET platform are required
by this Oracle tool too (and will be downloaded and installed during the process, if
they are not already installed on your computer). Currently, only JDK 6 is supported;
however, the unsupported JDK 7 works too.
To start the actual installation, execute the unzipped setup.exe; first, a security
window will appear (click Yes), then, if your OS is Windows 8, another one will pop-
75

up, with the information that this version of Oracle Database 11gR2 Client is not
supported on MS Windows 8 (you can ignore it by clicking Yes):

Figure 31: Oracle Client 11g message box warning when installing on Windows 8
On the next screen, the Oracle Installer (OI) main window is displayed; select the
Administrator type of installation (step 1):

Figure 32: Main Oracle Client 11g installation window
On the next screen choose desired (e.g. English and Romanian) languages (step
2):
76


Figure 33: Select languages Oracle Client 11g installation window
Then, double-check the paths and, when ok, click the Next > button (step 3):

Figure 34: Specify Installation Location Oracle Client 11g installation window
As MS Windows 8 is not supported by this version of Oracle Database 11gR2
Client, step 4 (Perform Prerequisite Checks) is skipped by the OUI on Windows 8.
In step 5, please refer to the selected configuration; if everything is correct, click
the Finish button (note that your selections may be saved before, for your reference):
77


Figure 35: Summary Oracle Client 11g installation window
Installation process will be started (see the progress bar and status in step 6):

Figure 36: Install Product Oracle Client 11g installation window
When OI finishes, click the Close button:
78


Figure 37: Finish Oracle Client 11g installation window
Note that, depending on your configuration, you should allow OCI to download
and install missing additional required software (e.g. MS .NET 3.5 components).
Next, run the SQL Developer: select actual JDK installation path (I used JDK 6;
all versions are freely downloadable from the Oracle web site and one of them should
be installed already):

Figure 38: Oracle SQL Developer JDK path message box
Note that if you try using JDK 7, you will get the error:

Figure 39: Oracle SQL Developer installing JDK 7 error message box
However, there is a simple tweak to overcome it: just copy MSVCR100. dll from
the JDK directory to the root directory of SQL Developer:
79


Figure 40: Oracle SQL Developer JDK 7 error overcome: copy missing file

Figure 41: Oracle SQL Developer JDK 7 error overcome: paste missing file
80

When launching SQL Developer, there will be a message prompt informing that
this product is not certified to be run with JDK 7 and some issues can occur: you can
click on the Skip This Message Next Time check box and then on the Yes button:

Figure 42: Oracle starting SQL Developer JDK 7 error message box
With any of the JDK versions 6 or 7, the SQL Developer will then start:

Figure 43: Oracle SQL Developer starting window
81

Now, you should create and configure three basic connections in SQL Developer,
respectively for the following users: sys, system, and ODBDETECTIVE.
Firstly, do this for the sys user: right-click "Connections" node and select "New
Connection" from the context menu:

Figure 44: Oracle SQL Developer: creating a new db connection
The "New / Select Database Connection" window will be shown; fill the form
with the values similar to the ones presented below (refer to your actual configura-
tion):

Figure 45: Oracle SQL Developer: filling in/updating db Basic connection data
82

Then, click "Test". If there is a "Status : Success" message displayed over the
Help button, then everything was created correctly. Click the "Save" button and a
new connection called ODBDETECTIVEsys appears under the "Connections" node.
Please notice that you have to change the role ("Role" label on the form with the
related combo-box) for the sys user from "default" to "SYSDBA", while using the de-
fault "Basic" connection type (refer to the label "Connection Type" and related com-
bo-box).
Then, create a new connection for the system user and provide similar values to
the ones presented below (however, check your own configuration nuances). There
are only two important differences the first is connection name, the second is that
we are now using the "default" role:

Figure 46: Oracle SQL Developer: testing a new/updated db connection
After clicking the "Save" button, there will appear a new connection available un-
der the "Connections" node called ODBDETECTIVEsystem. Click then "Cancel" to
close the New/Select Database Connection window.
You have thus created and configured the "basic" type connections for the sys and
system users: this is required to unlock the ODBDETECTIVE user, who is locked by
default in each Oracle Database 11gR2 installation, and to set a password for it (in
our case: DBDETECTIVE).
Before creating the ODBDETECTIVE user, Oracles Transparent Network Sub-
strate (TNS) interface for connections between clients and server has to be confi-
gured. To configure the TNS interface, go to the following location (or similar in
your case properly modified to follow your actual configuration and installation di-
rectory):
83

D:\app\oracle.client\product\11.2.0\client_1\network\admin\
If there is a tnsnames.ora file, edit it; if not, then create a blank tns names.ora file
in Notepad (or any other text editor); then, add the following lines to it (of course,
taking care of your actual Oracle db server IP address, port, and service name) and
save the file in the above mentioned folder:
ODBDETECTIVE =
(DESCRIPTION =
(ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)
(HOST = 172.31.1.56)(PORT = 1521)
)
)
(CONNECT_DATA = (SERVICE_NAME = orcl))
)

Figure 47: Creating/editing an Oracle tnsnames.ora file
Then, in SQL Developer connect with the previously created ODBDETECTIVEsys
connection as a system user: right-click on corresponding connection name, then se-
lect "Connect", and provide password (if not stored in the connection data):
84


Figure 48: Oracle SQL Developer: current db connection SQL window
To unlock the ODBDETECTIVE user and assign to it a password ("DBDETEC-
TIVE"), execute the following SQL statement (once SQL statement is typed in the
editor, just press F9 Execute Statement or select the Play button from the toolbar):
ALTER USER ODBDETECTIVE IDENTIFIED BY
"DBDETECTIVE" ACCOUNT UNLOCK;


85

This should look like this:

Figure 49: Oracle SQL Developer: running SQL statements to unlock accounts
With the ODBDETECTIVE user unlocked, you can configure a connection for it
exactly the same way like for the sys and system user above, but of type TNS, not
"Basic" and select "ODBDETECTIVE" in the "Network Alias" combo-box. Of
course, the role should be "default", as this is not a SYSDBA user. Do this and then
click the "Test" button; if the connection test succeeded, click "Save" to store this
connection too:

Figure 50: Oracle SQL Developer: filling in/updating db TNS connection data
86

Then click "Connect" and try to execute (F9) a simple SQL statement (e.g.
SELECT * FROM CNSTR_TYPES;) in the editor window:

Figure 51: Oracle SQL Developer: running SQL statements to test newly created connection
We have successfully unlocked user ODBDETECTIVE and configured the TNS
interface on the client side. We also changed the connection type for ODBDETEC-
TIVE. We could do the same thing for sys and sysdba, but we can leave their connec-
tions with the previously defined "Basic" type too, as both methods work.
We're now almost ready to bridge MS Access 2010 via ODBC with Oracle Data-
base 11gR2, as all required components are installed and configured. Note that there
are three available ODBC driver types for Oracle Database: the Microsoft one, pro-
vided with MS Windows, the Oracle-made one, and third-party ones.
According to OTN, the MS ODBC driver for Oracle uses an old Oracle 7 version
of the OCI Application Programming Interface (API), which is obsolete for about 15
years already. This can easily explain the stability and performance issues we can
have with newer versions of the Oracle Client and/or db. Consequently, I chose the
Oracle one, which is automatically installed together with the Oracle Client 11g.
87

Finally, you can now double-click on ODBDETECTIVE.accdb, click on the
Linked Table Manager (on the External Data ribbon tab), click on the Select All but-
ton, then on the Always prompt for new location check-box, and then on the OK but-
ton:

Figure 52: MS Access Linked Table Manager: relinking DBDetectives table and views to your
Oracle db server
Then, click the "OK" button: a "Select data source" window is displayed; change
the tab from "File data source" to "Machine data source":

Figure 53: MS Access 2010: selecting the machine data source ODBC driver
88

Click the "New" button to create new data source for the Oracle db. A new win-
dow will be displayed:

Figure 54: MS Access 2010: creating a new ODBC machine data source
If you have Windows administrative rights, select "System Data Source (Applies
to this machine only)" instead of "User Data Source (Applies to this machine only)",
which is selected by default, in order to create a DSN that is available to all users on
your machine, as well as by any system-wide service.
If you do not have Windows system privileges, when selecting System, you get the
following error message:

Figure 55: MS Access 2010: ODBC System DSN warning message box
Clicking OK, you should then choose the User Data Source option on the follow-
ing window:
89


Figure 56: MS Access 2010: creating a new ODBC user type machine data source
Then click on "Next >" and select "Oracle in OraClient11g_home1" to choose the
Oracle-made ODBC driver (don't use the MS-made ODBC driver instead!):

Figure 57: MS Access 2010: selecting the Oracle ODBC Windows driver
Once the Oracle ODBC driver is selected, click on the "Next >" button. A summa-
ry window will be shown:
90


Figure 58: MS Access 2010: finishing creation of a new ODBC DSN
Click "Finish": the "Oracle ODBC Driver Configuration" window will open, in
which we have to provide correct information in order to establish a connection with
the ODBDETECTIVE schema of the Oracle db server:

Figure 59: Oracle ODBC Driver Configuration window
Fill it with the desired appropriate values:
91


Figure 60: Filling in the Oracle ODBC Driver Configuration window
Please note that we have to select "TNS Service Name", in order to use the TNS
ODBDETECTIVE service name for establishing the desired connection. Once all the
fields are filled with connection information, we can use the "Test Connection" but-
ton to check if it works; the "Oracle ODBC Driver Connect" window will be dis-
played:

Figure 61: Oracle ODBC Driver Connection window
92

On this window, we have to provide a password (recall that the ODBDETECTIVE
user was unlocked and the password "DBDETECTIVE" was set for it above) and then
click on "OK", which should connect Access to Oracle.
If the connection test was successful, click "OK" on the "Testing Connection"
window, to close it:

Figure 62: Oracle ODBC Driver Connection successful message box
Then, click "OK" on the "Select Data Source" one with the "ODBDETECTIVE
db" data source selected from the list of available machine data sources:

Figure 63: MS Access 2010: selecting the Oracle machine data source ODBC driver
93

When "Oracle ODBC Driver Connect" window appears, provide the password
("DBDETECTIVE") for ODBDETECTIVE user and then click on "OK":

Figure 64: Oracle ODBC Driver Connection: providing Oracle db password
Click on OK and then on the Close button of the Linked Table Manager:

Figure 65: MS Access Linked Table Manager successful refresh message
6.1.3 DBDetectiveO uninstallation
Uninstalling DBDetectiveO is extremely simple: you just have to delete (just like
any file) the ODBDetective.accdb front-end, drop the Oracle 11g back-end ODBDE-
TECTIVE user, and uninstall Oracle Client 11g from Windows.
6.2 Starting and exiting DBDetectiveO
In order to start DBDetectiveO, you have first to make sure that the corresponding
Oracle 11g server is up and running, that your computer is connected to it (via LAN
or the Internet, for security reasons being preferable to use a VPN connection; you
can check both of these two pre-requisites through the Oracle SQL Developer
ODBDETECTIVE connection: see 6.1.2 above), and then to double-click on the
ODBDetective.accdb; the following window will appear:
94


Figure 66: DBDetectiveOs application main switchboard menu window
You may then anytime exit DBDetectiveO by clicking on its last, Quit DBDetec-
tiveO button. Moreover, you can exit it too from the Current DB investigation and
Manage ORACLE servers and databases forms, by clicking their Quit buttons and
then confirming your request (that is clicking on the OK button of the message box):

Figure 67: DBDetectiveOs application quit confirmation message box
6.3 Managing server connections and database links
When clicking on the Manage ORACLE servers and databases button from the
main switchboard menu, either the following error message is displayed, if there is no
connectivity to the Oracle 11g server:

Figure 68: MS Access ODBC connection failure message box
or the corresponding form opens:
95


Figure 69: DBDetectiveOs application Manage known servers and users window
Any time you wish to quit DBDetectiveO you can click on the red Stop button, top
middle after the application name.
In its left third, data for the currently used Oracle db server is shown (and may be
edited, case in which, if updates are valid, the corresponding connection string, which
is read-only, is automatically computed). Right bottom, a sub-form displays data for
all known dbs on the current server (which is editable too, except for the link names
that are also automatically computed after any valid update of db names).
6.3.1 Navigating between known servers and dbs metadata
The currently displayed server may be easily switched to any other one known to
DBDetectiveO by opening the Choose desired Oracle server combo-box and se-
lecting another server:
96


Figure 70: DBDetectiveOs application Manage known servers and users: selecting another
current server connection
By clicking on the desired one, its data is displayed:

Figure 71: DBDetectiveOs application Manage known servers and users: navigation between
any two known server connections
97

6.3.2 Inserting a new Oracle server connection
To insert a new server metadata, click on the yellow starred Access New (blank)
record right arrow button in the bottom-left of the window and a blank new record is
provided:

Figure 72: DBDetectiveOs application Manage known servers and users: positioning on a new
server connection record
The localhost, TCP, 1521, and DEDICATED are default values that you
may modify when needed or let them as such when you agree on them; the server
name and host are compulsory and unique; for example, trying to give to this new
server the existing name Demo would result in the following error message:

Figure 73: DBDetectiveOs application Manage known servers and users: error message box
when trying to duplicate server connection names
Note that the protocol, the port, and the mode are mandatory too, that port should
be a natural between 1521 and 1575, and that either the SID or the service name
should be specified (but not both of them).
98

For example, trying to leave both SID and service name null results in getting the
following error message:

Figure 74: DBDetectiveOs application Manage known servers and users: error message box
when trying to leave blank both Oracle SID and Service Name
whereas trying to save an invalid port number results in getting the error message:

Figure 75: DBDetectiveOs application Manage known servers and users: error message box
when trying to enter an out of range Oracle port value
When all data is valid, the corresponding connection string is computed and the
new server metadata is saved into the SERVERS table of the Oracle DBDetectiveOs
db:

Figure 76: DBDetectiveOs application Manage known servers and users: automatic generation
of server connection string when saving inserted metadata
99

6.3.3 Updating server connection
For updating a server metadata you should first navigate to it (see 6.3.1 above)
and then modify any of its characteristics, by obeying the rules presented in 6.3.2
above; for example, immediately after saving a new 1522 port value for the Demo1
server, its corresponding connection string is re-computed and saved into the DBDe-
tectiveOs db:

Figure 77: DBDetectiveOs application Manage known servers and users: automatic generation
of server connection string when saving updated metadata
Note that saving current record data into the db is performed by Access (only if it
does not violate corresponding scheme constraints) either when the ribbon Save but-
ton from the Records group is clicked or when the form is closed or, simply, when
moving to another record (in this case, for example, either on any line of the Databa-
ses sub-form, or to a new line, or to the previous or next ones, or to the first or the
last ones, or to the one selected in the Choose desired Oracle server combo-box,
etc.).
Obviously, if any update is invalid, then it is rejected and saving does not occur;
for example, if trying to fill in both SID and the service name, the following error
message is displayed:
100


Figure 78: DBDetectiveOs application Manage known servers and users: confirmation
message box when replacing SID with a Service Name
6.3.4 Deleting server connection
For updating a server metadata you should first navigate to it (see 6.3.1 above)
and then ask for its deletion by clicking on the Delete Record button from the ribbon
Records group; if you cancel the confirmation message, nothing happens; if you con-
firm it, then,

Figure 79: DBDetectiveOs application Manage known servers and users: confirmation
message box when asking for deletion of a server connection
if there are no dbs on the current server (like it is the case for this newly Demo1, just
previously added), then its metadata is deleted and the cursor moves on the next one
(in this case, as Demo1 was the last one, this is the empty record one).
If the server has dbs, then the following second confirmation screen is displayed:

Figure 80: DBDetectiveOs application Manage known servers and users: confirmation
message box when deleting a server connection having db links
101

If cancelled, nothing happens; if confirmed, then if the currently under investiga-
tion db is not among the hosted dbs, all their corresponding metadata is deleted and
then the servers one is deleted too; if among the hosted dbs there is also the currently
under investigation one, then the following third confirmation screen is displayed:

Figure 81: DBDetectiveOs application Manage known servers and users: confirmation
message box when deleting a server connection having the db link to the currently under
investigation db
Obviously, if you click on Cancel, nothing happens; if you click on OK, then all
investigation data is deleted, then all corresponding dbs one is deleted too, followed
by the servers one.
6.3.5 Inserting a new Oracle db link
To insert a new Oracle db metadata, you should first select the desired Oracle ser-
ver (see 6.3.1), then click on the corresponding new record line of the Databases sub-
form and start typing desired db name and password (both being mandatory):

Figure 82: DBDetectiveOs application Manage known servers and users: inserting a new db
link
102

If, according to Oracle naming rules, the db name is invalid (i.e. using other cha-
racters than [1-9], [A-Z], and _, starting with ORA_ or SYS_, or having more
than 30 characters), then a corresponding error message is displayed; for example:

Figure 83: DBDetectiveOs application Manage known servers and users: error message box
when trying to use Oracle not accepted characters in db names

Figure 84: DBDetectiveOs application Manage known servers and users: error message box
when trying to use db names starting with SYS_ or ORA_

Figure 85: DBDetectiveOs application Manage known servers and users: error message box
when trying to use db names longer than 30 characters

103

When asking for saving valid data, corresponding link db name is automatically
computed (from the db name and the host server id, separated by #) and data is
saved (moreover, note that dbs are lexicographically ordered by their names):

Figure 86: DBDetectiveOs application Manage known servers and users: automatic generation
of a new db link when saving new db data
Moreover, a corresponding Oracle db link is created too:

Figure 87: DBDetectiveOs db: automatic creation of a new db link when saving new db data
104

6.3.6 Updating Oracle db link
To update Oracle db metadata, you should first select the desired Oracle server
(see 6.3.1), then click on the corresponding db record line of the Databases sub-form
and start typing desired db name and/or password new values.
If new values are valid (see 6.3.5 above), a new corresponding db link name is au-
tomatically re-computed and stored in the db along with the rest of updated metadata:


Figure 88: DBDetectivOes application Manage known servers and users: automatic generation
of a new db link when saving updated db data
Moreover, the old Oracle db link is deleted and another one is created with the
new name:
105


Figure 89: DBDetectiveOs db: automatic deletion of old db link and creation of a new one
when saving updated db data
6.3.7 Deleting Oracle db link
To delete Oracle db metadata, you should first select the desired Oracle server
(see 6.3.1), then click on the corresponding db record line of the Databases sub-form,
and then click on the Delete Record button from the ribbon Records group; if you
cancel the confirmation message, nothing happens; if you confirm it and the db is not
the currently under investigation one, then, its metadata is deleted,

Figure 90: DBDetectiveOs application Manage known servers and users: confirmation
message box when asking for deletion of a db link
and the cursor moves on the next one:
106


Figure 91: DBDetectives application Manage known servers and users: deleting a db link
Moreover, corresponding Oracle db link is deleted too:

Figure 92: DBDetectiveOs db: automatic deletion of db links from Oracle db too
If the db to delete is the currently under investigation one, then another confirma-
tion message is displayed; if you cancel your request, nothing happens;
107

.
Figure 93: DBDetectiveOs application Manage known servers and users: confirmation
message box when asking for deletion of the db link to the currently under investigation db
if you confirm it, all investigation data is deleted and then corresponding db one and
Oracle db link are deleted too.
6.4 Mining Oracle metacatalog data for DBDetectiveO investigations
6.4.1 Setting the desired target db to investigate
Any time you want to investigate another Oracle db, this db has to be known to
DBDetective (i.e. its own and its server metadata should be stored in the DBDetec-
tives db), you should then select the desired Oracle server (see 6.3.1), then click on
the corresponding db record line of the Databases sub-form, and then click on the Set
target database to investigate button, which is placed in the middle of the Manage
known servers & users form, above the Start target database investigation one.
If no db is selected, the following error message is displayed:

Figure 94: DBDetectiveOs application Manage known servers and users: error message box
when trying to set the current db to investigate with no db selected
If a db is selected, then a confirmation message is displayed:

Figure 95: DBDetectiveOs application Manage known servers and users: confirmation
message box when the selected db was set as the current one
108

6.4.2 Starting target db investigation
If you want to start investigation for the currently selected db (see 6.3.8 above), all
you have to do is to click on the Start target database investigation button, which is
placed in the middle of the Manage known servers & users form, under the Set target
database to investigate one.
If no target db was previously set, the following error message is displayed:

Figure 96: DBDetectives application Manage known servers and users: error message box when
trying to start mining for metadata without previously setting the desired db
If the set target db is the currently under investigation one, then the following con-
firmation message is displayed:

Figure 97: DBDetectiveOs application Manage known servers and users: confirmation
message box when asking to refresh currently under investigation metadata
If you click on Cancel, nothing happens; if you click on OK, then DBDetectiveO
is refreshing its investigation data (to take into account, for example, scheme and in-
stance updates that occurred since the previous analysis).
If the set target db is not the currently under investigation one, then the following
confirmation message is displayed:
109


Figure 98: DBDetectiveOs application Manage known servers and users: confirmation message
box when asking to replace currently under investigation metadata with the one of another db
If you click on Cancel, nothing happens; if you click on OK, then DBDetectiveO
is replacing its current investigation data with the one for the currently set target db.
6.5 Investigating violations of db best practice rules
When clicking on the Investigation button of the main menu, its window opens:

Figure 99: DBDetectiveOs application Investigations window
Any time you wish to quit DBDetective you can click on the red Stop button, top
middle, after the application name.
6.5.1 Displaying global statistics
If you want to check what is the db under investigation and/or inspect global in-
vestigation statistics, click on the Global Statistics button, which will open the fol-
lowing form:
110


Figure 100: DBDetectiveOs application Global Statistics window
6.5.2 Displaying main tables, columns, triggers, constraints, and indexes
metadata
If you click on the Table Details button, the following form is opened:
111


Figure 101: DBDetectiveOs application Table Details window
Main first table metadata is displayed in its header, while its tabs present cor-
responding metadata on columns and triggers, constraint, and indexes. You can select
any other table, from the Select Other Table combo-box:

Figure 102: DBDetectiveOs application Table Details window: selecting another table
To inspect currents table constraints, click on the Constraints tab; to open column
details sub-forms, click on the + to the left of desired constraints:
112


Figure 103: DBDetectiveOs application Table Details window: browsing table constraints and
their column details
To inspect currents table indexes, click on the Indexes tab; to open column details
sub-forms, click on the + to the left of desired indexes:

Figure 104: DBDetectiveOs application Table Details window: browsing table indexes and
their column details
6.5.3 Displaying Oracles constraint types codes and their meaning
As Oracle displays only its codes for constraint types, DBDetectiveO provides
their decoding too; in order to display it, just click on the Constraint Types button:
113


Figure 105: DBDetectiveOs application Oracle Constraint Types window
6.5.4 Displaying DBDetectiveOs table types codes and their meaning
You can display DBDetectiveOs table type codes and their meaning by clicking
on the Table Types button:

Figure 106: DBDetectiveOs application Table Types window
Please note that you can also display full table metadata, as well as metadata on
objects, dependencies, views, sequences, and PL/SQL code line from the Mined Me-
tadata submenu (see 6.6 below).
6.5.5 Selecting, unselecting, and displaying desired mining results
If you click on the Select All button, all 32 radio buttons, each one corresponding
to an investigation type, are selected:
114


Figure 107: DBDetectiveOs application Investigations window: selecting all 32 investigation
types
When you click on the Unselect All button, all 32 radio buttons are unselected
(just like when the forms open). By clicking (odd times to select, even ones to unse-
lect) on the corresponding issues radio buttons, you can also choose any desired com-
bination of them, like, for example:

Figure 108: DBDetectiveOs application Investigations window: selecting only some desired
investigation types
When you are satisfied with your selection, you can click on the Run Selected Op-
tions button, which will open all corresponding selected forms presenting desired in-
vestigation metadata (for example, not read-only fundamental tables not having at
least one non primary key column not accepting nulls):
115


Figure 109: DBDetectiveOs application example of investigation results
6.6 Displaying mined metadata and storing semantic and decision one
To display mined metadata and/or to store associated semantic and decision one,
you should click on the Mined metadata button of the main menu (see 6.2 above); the
following sub-menu is displayed:

116


Figure 110: DBDetectiveOs application Mined metadata submenu
6.6.1 Displaying mined metadata on objects and their dependencies
To display mined metadata on objects and dependencies between them, just click
its first button and then click on the + icon to the left of the name of desired objects,
in order to open their dependent objects sub-forms:

Figure 111: DBDetectiveOs mining results: metadata on objects and their dependencies
117

To display mined metadata on views, just click its fifth button:

Figure 112: DBDetectiveOs mining results: metadata on views
To display mined metadata on sequences, just click its sixth button:

Figure 113: DBDetectiveOs mining results: metadata on sequences
To display mined metadata on PL/SQL source code, just click its seventh button:

Figure 114: DBDetectiveOs mining results: metadata on PL/SQL source code lines
118

6.6.2 Storing semantic and decision metadata in DBDetectives db
Automatic violations detection can obviously be based only on syntactic crite-
ria. In order to make correct decisions for db schemes enhancements, its users
should analyze DBDetective output and, based on semantic knowledge, decide
whether or not to correct each of the syntactically discovered possible violations
(e.g. analyzed current instance may be a non-typical one, empty tables may be legiti-
mately be empty as they are, in fact, temporary ones, some tables instances might
grow much larger in certain contexts, etc.).
DBDetective 1.0 already provides users with the ability to store in its db additio-
nal, semantic metadata for easing subsequent decision taking on needed improve-
ments. For example, added columns Static, Fundamental, and Used_in_SP of
TABLES are storing whether or not tables are (quasi-)static (i.e. they are very ra-
rely updated, if any), actually storing fundamental or derived data, and used in
PL/SQL dynamic code, respectively; codes in column Cache may be decrypted by using
the Table Types button (see 6.5.5 below); moreover, a Comments column is provided too, for
storing as much relevant semantics on each table as needed and known.
To display mined metadata on tables, their columns, and associated triggers and/or
to store decision support data, just click the third button of the Mined metadata sub-
menu for opening the following window:

Figure 115: DBDetectiveOs mining results and decision support data: metadata on tables
119

Similarly, columns Computed, Needs Index, Index_Type, and Join_Index of
TBL_COLUMNS are storing whether or not columns are storing fundamental or
computed data, an index on them should be added, and if yes, of what type
(normal B-tree or bitmap), and, if bitmap, of what subtype (join or not), res-
pectively.
To display mined metadata on table columns and/or to store decision support data,
just click the third button of the Mined metadata sub-menu, and then click on the +
icon to the left of the name of the corresponding table, in order to open the tables co-
lumns sub-form:

Figure 116: DBDetectiveOs mining results and decision support data: metadata on columns
and triggers
Similarly too, columns ToBeAdded, Used, BitMap, and ToReverse of INDE-
XES are storing whether or not new indexes should be added or existing ones are
actually used or should be dropped because they are too expensive, or their type
should be changed, or their columns should be re-ordered; column CorrectCo-
lumnPosition of INDEXES_COLUMNS is obviously useful too.
To display mined metadata on indexes and/or to store decision support data, just
click the second button of the Mined metadata sub-menu, and then click on the +
icon to the left of the name of the corresponding tablespace, in order to open the inde-
xes sub-form; for opening desired index columns, click on the + icon to the left of
the name of the corresponding indexes:
120


Figure 117: DBDetectiveOs mining results and decision support data: metadata on indexes and
their columns
To display mined metadata on constraints and/or add/update/delete comments on
them (in the Comments column), just click the fourth button of the Mined metadata
sub-menu, and then click on the + icon to the left of the name of the corresponding
table, in order to open desired column constraints sub-forms:

Figure 118: DBDetectiveOs mining results and decision support data: metadata on constraints
and their columns

121

6.7 Investigating Oracle advisors recommendations and log
6.7.1 Oracle advisors log
To display mined metadata on Oracles corresponding advisors log just click the
fourth button of the Main DBDetectiveO menu (see 6.2):

Figure 119: DBDetectiveOs mining results: Oracles advisors log for the investigated db
6.7.2 Oracle advisors recommendations
To display mined metadata on Oracles corresponding advisors recommendations
just click the fifth button of the Main DBDetectiveO menu (see 6.2):

Figure 120: DBDetectiveOs mining results: Oracles advisors recommendations for the
investigated db
122

Unfortunately, as this happens generally, in this case too, the only recommenda-
tion types were to perform shrinking (a useful, but not at all an enough one) and to
enable row movements.
7 Case study
DBDetective was already used on several customer dbs with great results. For
example, such a db has a total of 3,860 user objects (and 5,649 dependencies be-
tween them) with only 116 being temporary and 191 automatically generated
indexes ones, out of which 586 tables (totaling 8,621 columns and 129,956,314
rows) interconnected by 415 foreign keys, with 66 temporary ones, and 2,842 in-
dexes (out of which 2,651 are explicitly defined); their instances plausibility is
enforced by 3,860 constraints, out of which 278 are unique ones, with 229 prima-
ry and 49 semantic keys, 3,167 are check constraints, the remaining 415 being re-
ferential integrities; there are also one view, 151 sequences, 19 triggers, 127 pac-
kages (containing 1,647 procedures and 467 functions) totaling 129,730 PL/SQL
lines.
Running all of the 32 DBDetectiveO options, discovered were the following:
180 empty non-temporary tables, on which there were defined 211 indexes
and 243 constraints;
89 fundamental not empty tables on which no interesting object (table,
PL/SQL code, etc.) depends on;
no cached table, although there were 168 such candidates for x = 1,000
and 58 for x = 100;
268 constraints on temporary tables (257 check, 10 primary, and one se-
mantic keys);
2 tables having migrated rows;
311 empty columns, out of which 161 were not VARCHAR (1 BLOB, 2
CLOB, 3 CHAR, 49 DATE, and 106 NUMBER), and on which defined
were 24 constraints (1 check, 1 unique, and 22 foreign keys) and 21 inde-
xes;
292 keyless non-empty fundamental tables;
40 tables having concatenated primary keys (3 quaternary, 8 ternary, and
29 double);
123

15 not numeric primary keys (11 VARCHAR2, 1 CHAR, and 3 DATE);
all 239 surrogate primary keys were oversized (and, what is worse, to the
maximum possible NUMBER(38) value!);
8 concatenated foreign keys (4 ternary and 4 double);
63 foreign keys having non-numeric columns; all 415 foreign keys were
oversized (again, at the maximum possible: NUMBER(38) !);
273 improperly typed foreign keys;
576 normal indexes that should have been bitmap instead (for y = 10 and
z = 90);
56 indexes to be compressed for w = 100 and other 37 for w = 1,000;
24 concatenated indexes (out of which 14 were unique ones with 6 for
primary keys!) totalizing 80 columns on non-empty tables having wrong
columns order;
740 subqueries;
317,542 dynamic SQL statements;
unfortunately too, there were no statistics gathered on indexes, no parallel
enabled functions, no result cache queries or functions, no compound trig-
gers, and no triggers with firing sequences either;
fortunately, there were no tables or indexes in system tablespaces, no ta-
ble having no not-null columns, no table to shrink, no superkeys, and no
bitmap indexes that should have been normal (as there were no bitmap
indexes at all).
According to this data (that you may see in the Global Statistics form,
see 6.5.1 above), t he customer decided to apply, in a first step, the following
changes to its db scheme:
- downsizing primary keys to corresponding tables double of maximum car-
dinals
- replacing concatenated primary keys with simple surrogate ones
- replacing concatenated foreign keys with single ones
- replacing all non-numeric primary keys with surrogate ones
- replacing all non-numeric foreign keys with corresponding numeric ones
- eliminating migrating rows
- shrinking all tables
- regularly gathering statistics on indexes.
124

Consequently, performance of its application improved by more than 2.5
times, out of which, shrinking and enabling row movement, the only Oracle advisors
recommendations, brought only some 5%.
Conclusion and further work
Unfortunately, most of the existing dbs are poorly designed, implemented, fine-
tuned, and used. For asserting and, especially, improving their quality, several sets of
best practice rules were proposed in this field. Considering a kernel crucial subset of
the ones introduced in [1] and [2], under the supervision of Prof. Christian Mancas, I
have architectured, designed, developed, tested, used, and documented a first version
of DBDetective for Oracle, a metadata mining tool for detecting violations by Oracle
dbs of corresponding dbs design, implementation, and usage best practice rules.
Implementation (hence documentation, testing, and usage) was done in Oracle
11g and MS Access 2010, under RedHat Linux 6.2 and 64-bit Windows 7/8, on Dell
servers and notebooks.
Several usages of DBDetective on some customers dbs proved its efficiency: one
of them is presented as a case study in this book too.
DBDetectives idea, related work, architecture, design, prototype implementation,
and usage were published in January 2013 [3]. Some details of its 1.0 version fully
documented here were previously published too [4].
A few of the DBDetectiveO f eat ur es ar e al s o pr ovi ded by Or acl e
t ool s , but onl y avai l abl e f or i t s ver y expens i ve Ent er pr i s e Edi -
t i ons . DBDetective is somewhat similar, for example, to DB Optimizer [8], but
has very many powerful additional features.
Automatic violations detection can obviously be based only on syntactic cri-
teria. In order to make correct decisions for db schemes enhancements, its users
should analyze DBDetectives output and, based on semantic knowledge, decide
whether or not to correct each of the syntactically discovered possible violations
(e.g. analyzed current instance may be a non-typical one, empty tables may be legiti-
mately be empty as they actually are temporary ones, some tables instances might
grow much larger in certain contexts, etc.).
125

DBDetective already allows for users to store some data on decisions they took af-
ter analyzing its findings, thus greatly simplifying the task to correct correspond-
ing detected violations.
Further improvements will include:
adding supplementary best practice rules
allow for more decision data to be stored
backup and restore investigation and decision data
automatically generate as much SQL scripts as possible for improving
Oracle dbs performances based on investigation and decision data
extensions to other RDBMSs than Oracle.


126

References
1. Mancas C. and Dicu A. I. Under gr aduat e Dat abas e Labs Not es
(2014). To be submitted to Lambert Publishing House, Saarbrucken, Germany.
2. Mancas C. Conceptual Data Modeling and Database Design. A Completely
Algorithmic Approach (2014). To be published by Apple Academic Press, NJ,
U.S.A.
3. Mancas C. and Dicu A.I. DBDetective: a metadata mining tool for
detecting violations of some Oracle database design, implementation,
querying, and manipulating best practice rules (2013). Seminar Se-
ries in Mathematics and Computer Science, Ovidius State Univer-
sity, Constanta, Romania, 26-43.
4. Mancas C. and Dicu A.I. Architecture, design, development, and
usage of DBDetective 1.0 (2013). Journal of Information Technology & Soft-
ware Engineering, Henderson, NV, 3:1, 1-13.
5. Oracle Corp. DBAs New Best Friend: Advanced SQL Tuning Features of
Oracle Database 11g (2007). www.oracle.com/technetwork/database/ma-
nageability/sqltunepresentati07130395.pdf
6. Oracle Corp. Guide for Developing High-Performance Database Appli-
cations (2010). An Oracle White Paper www.oracle.com/technetwork/data-
base/performance/perfguidewpfinal133229.pdf.
7. Oracle Corp. Oracle Database Performance Tuning Guide 11g Release
1 (11.1) (2008), http://docs.oracle.com/cd/B28359_01/server.
111/b28274.pdf.
8. Embarcadero Technologies, Inc. DB PowerStudio for Oracle (2012).
www.embarcadero.com/products/db-powerstudiofororacle.
9. PropernH.A. and Halpin T.A. Conceptual Schema Optimization Da-
tabase Optimization before sliding down the Waterfall (2004). Technical
Report 341, June 23, 2004 version, Dept. Comp. Sci., Univ. of Queensland,
Brisbane, Australia.
10. Dong Yu-Jie and Li Fu-Guo. Research on Optimization Strategy of
Relational Schema based on Normalization Theory (2010). Proc. of 3
rd
Intl. Symp. on Comp.Sci. and Computational Techn. ISCSCT10, Jiaozuo,
P.R.China, 41-43.
127

11. Kaiser M. et al. Data organization for database optimization (2005).
U.S. Patent application 0010606 A1, www.google.ro/patents/
US20050010606?dq=database+optimization.
12. Arnold J.A. et al. Database optimization apparatus and method
(2007). U.S. Patent application 0073644 A1, www.google.com/patents/
US20070073644.
13. Arnold J.A. et al. Database optimization apparatus and method
(2006). U.S. Patent number 7089260, www.google.com/patents/US7089260.
14. Mancas C. On Knowledge Representation Using an Elementary Ma-
thematical Data Model (2002). Proc. of the 1
st
IASTED Int. Conf. on Infor-
mation and Knowledge Sharing IKS 2002, Univ. of the U.S. Virgin Islands,
St. Thomas, USVI, USA, 206-211.
15. Mancas C. and Dragomir S. On the Equivalence between Entity-Re-
lationship and Functional Data Modeling (2003). Proc. of the 7
th
IASTED
Int. Conf. on Softw. Eng. and Appl. SEA 2003, Marina del Rey, CA, USA, 335-
340.
16. Mancas C. and Dragomir S. An Optimal Algorithm for Structural
Keys Design (2003). Proc. of the 7
th
IASTED Int. Conf. on Softw. Eng. and
Appl. SEA 2003, Marina del Rey, CA, USA, 328-334.
17. Mancas C. and Crasovschi L. An Optimal Algorithm for Computer-
Aided Design of Key Type Constraints (2003). Proc. of the 1
st
Balkan Conf.
in Informatics BCI 2003, Aristotle Macedonian Univ., Thessaloniki, Greece,
574-584.
18. Mancas C. and Mancas S. MatBase Relational Import Subsystem
(2006). Proc. of IASTED Datab. and Appl., DBA 2006, Innsbruck, Austria,
123-128.
19. Codd, E.F. A Relational Model of Data for Large Shared Data
Banks (1970). CACM 13:6, 377-387.
20. Oracle Corp. Oracle Real Application Cluster 10g the Founda-
tion for Enterprise Grid Computing (2005). An Oracle White Paper.
www.oracle.com/technetwork/database/enterprise-edition/oracle-real-application-
clusters-10-133762.pdf.
128

21. Oracle Corp. Oracle Database SQL Language Reference 11g
Release 1 (11.1) (2010). http://docs.oracle.com/cd/B28359_01/server.111/
b28286.pdf
22. Oracle Corp. Oracle Database PL/SQL Language Reference 11g
Release 1 (11.1) (2009). http://docs.oracle.com/cd/B2835901/appdev.111/
b28370.pdf
23. Oracle Corp. Oracle Database 2 Day DBA 11g Release 1 (11.1)
(2012). http://docs.oracle.com/cd/B28359_01/server.111/b28301. pdf
24. Oracle Corp. Oracle SQL Developer Users Guide Release 1.1
(2007). http://docs.oracle.com/cd/B28359_01/appdev.111/b31695.pdf
25. Oracle Corp. Oracle Database Client Quick Installation Guide 11g
Release 1 (11.1) for Microsoft Windows x64 (2008). http://docs.oracle.com/
cd/B28359_01/install.111/b32306/toc.htm
26. Microsoft Corp. Microsoft Access 2010 Product Guide (2010).
http://www.microsoft.com/en-us/download/confirmation.aspx?id=5829
27. Dicu, A. I. Programming Paradigms, Semester Project (2012). Bu-
charest POLITEHNICA University, Faculty of Engineering in Foreign
Languages, Computer Science English Stream, Software Engineering
Master.
28. Microsoft Corp. Microsoft VBA Language Reference for Office 2010 (2010).
http://msdn.microsoft.com/enus/library/office/gg264383(v= office.14).aspx
29. Henning T., Cooper R., Griffith G. L., and Dennison J. Microsoft Access 2010
Programming Reference (2010), Safari Books Online.
http://my.safaribooksonline.com/book/programming/microsoft-
access/9780470591666/using-ado-to-access-data/adding_ado_references
30. Mancas C. Conceptual Data and Knowledge Modeling and Querying. MSc.
Lectures and lab notes (2012), Mathematics and Computer Science Dept., Ovidius
University, Constanta, Romania.
31. Mancas C. Architecture, Design, and Development of Database Software Ap-
plications. MSc. Lectures and lab notes (2012), Mathematics and Computer Science
Dept., Ovidius University, Constanta, Romania.
32. Mancas C. Advanced Databases. MSc. Lectures and lab notes (2012), Faculty
of Eng. Taught In Foreign Lang., Computer Science & Information Technology En-
glish Stream, POLITEHNICA University, Bucharest, Romania.
129

33. Serbanati D.L. Programming Paradigms. MSc. Lectures and lab notes (2012),
Faculty of Eng. Taught In Foreign Lang., Computer Science & Information Techno-
logy English Stream, POLITEHNICA University, Bucharest, Romania.
34. Dimo P. Human Computer Interaction. MSc. Lectures and lab notes (2012),
Faculty of Eng. Taught In Foreign Lang., Computer Science & Information Techno-
logy English Stream, POLITEHNICA University, Bucharest, Romania.

130

Subject and abbreviation index
3-tier architecture, 38
Access Database Engine, 22
Access main limitations, 24
Access Runtime Only, 24
ACE. See Access Database Engine
ADO. See ActiveX Data Objects
ADO Extensions for Data Definition
Language and Security, 29
ADO MD. See Multidimensional ADO
ADOX. See ADO Extensions for Data
Definition Language and Security
ADP. See Access Data Pages
API. See Application Programming
Interface
ASQLT. See Automatic SQL Tuning
Automatic SQL Tuning, 13
BI. See Business Intelligence
bitmap index, 13
BL. See business logic
Business Logic, 40
CDE. See Cooperative Development
Environment
COM. See Component Object Model
Cooperative Development
Environment, 17
CRM. See customer relationship
management
DAO. See Data Access Objects
db. See database
DB Optimizer, 14
DB PowerStudio for Oracle, 14
DBA. See db administrator
DBDetectiveO. See DBDetective for
Oracle
DB-O-M-0.( Reduce data contention),
35
DDL. See data definition language
DLL. See Dynamically Linked
Libraries
DML. See data manipulation language
Entity-Relationship Diagram, 41
E-R. See Entity-Relationship
E-RD. See Entity-Relationship
Diagram
ERP. See enterprise resource planning
Graphic User Interface, 38
GUI. See graphic user interface
131

HTTP. See Hypertext Transfer
Protocol
HTTPS. See Secure Hypertext Transfer
Protocol
IC. See Instant Client
ICL. See Instant Client Light
IDE. See Integrated Development
Environment
iFS. See Internet File System
Instant Client, 20
Instant Client Light, 20
JDK. See Java Development Kit
MDAC. See MS Data Access
Components
MDM. See mathematical data model
MDP. See multidimensional data
provider
MOD_PLSQL. See Oracle Web Agent
MODBCDM. See Microsoft ODBC
Driver Manager
MS. See Microsoft
MS Office, 24
Multidimensional ADO, 28
MVC. See Model View Controller
NCA. See Network Computing
Architecture
normal index, 13
OAE. See Oracle Application Express
OC. See Oracle db client
OCI. See Oracle Call Interface
ODBC. See Open DataBase
Connectivity
ODG. See Oracle Data Guard
ODP.NET. See Oracle Data Provider
for MS .NET
OEM. See Oracle Enterprise Manager
OI. See Oracle Installer
OLAP. See OnLine Analytical
Processing
OLE. See Object Linking and
Embedding
OMA. See Oracle Management Agent
OMR. See Oracle Management
Repository
OMS. See Oracle Management Server
OODBCD. See Oracle ODBC Driver
Oracle Installer, 75
Oracle Management Agent, 18
Oracle Management Repository, 18
Oracle Management Server, 18
OS. See operating system
132

OSQLD. See Oracle SQL Developer
OWA. See Oracle Web Agent
PA. See Partition Advisor
Partition Advisor, 13
PDDL. See parallel data definition
language
PDML. See parallel data manipulation
language
PQ. See parallel query
RAC. See Real Application Cluster
RDBMS. See relational db
management system
RDM. See relational data model
RDO. See Remote Data Objects
RDS. See Remote Data Service
Real-time SQL Monitoring, 13
Remote Data Service, 29
restrictions list, 43
RL. See restrictions list
RL1 (servers max. cardinality), 43
RL10 (object types max. cardinality),
44
RL11 (object types domain ranges),
44
RL12 (object types mandatory
properties), 44
RL13 (object types uniqueness
restrictions), 44
RL14 (constraint types max.
cardinality), 44
RL15 (constraint types domain
ranges), 44
RL16 (constraint types mandatory
properties), 44
RL17 (constraint types uniqueness
restrictions), 45
RL18 (table types max. cardinality),
45
RL19 (table types domain ranges), 45
RL2 (servers domain ranges), 43
RL20 (table types mandatory
properties), 45
RL21 (table types uniqueness
restrictions), 45
RL3 (servers mandatory properties),
43
RL4 (servers uniqueness restrictions),
43
RL5 (servers non-existence
restrictions), 43, 46, 52
RL6 (dbs max. cardinality), 44
RL7 (dbs domain ranges), 44
RL8 (dbs mandatory properties), 44
133

RL9 (dbs uniqueness restrictions), 44,
47, 52
RMAN. See Recovery Manager
RTSQLM. See Real-time SQL
Monitoring
SCM. See supply chain management
SDK. See software development kit
SQL Access Advisor, 13
SQLAA. See SQL Access Advisor
SQLDDM. See SQL Developer Data
Modeler
SSL. See Secure Sockets Layer
TDP. See tabular data provider
TNS. See Transparent Network
Substrate
TPC-H. See Transaction Processing
Performance Council ad-hoc decision
support benchmark
Transparent Network Substrate, 82
VBA. See Visual Basic for
Applications
Visual Basic for (db) Applications, 21
VPN. See Virtual Private Network
Windows DAC. See MS Data Access
Components
XE. See Oracle DB Express Edition
134