Octree Consulting Document and

Database Management Systems

Explained

Agenda

Purpose
Principles
Technical details
Example
Questions and Answers

Purpose
To build a system designed to store all Mining &
Geology data and to reuse existing functionality
provided by Microsoft and similar systems that are
usually available already Windows File Shares, MS
Office, SQL Server, SSRS, mail services.

Purpose
To eliminate costs associated with off-the-shelf
software licensing and maintenance
To utilise common industry-standard technologies
systems are less human dependent, although a
skilled administrator(s) is still required
To reduce blackboxing - all macros, scripts, etc.
are integral part of the system and remain
property of and accessible to the client

Principles
Keep it simple, stupid (KISS ) automated
import of raw data, no double-handling or
software-specific data entry required.
Data Warehouse (DW) - convenient centralised
storage easy to generate reports
Single source of the truth (SSOT) - document
vs database record and data ownership,
database is the reflection of the files, dataflow
from file to database to report, paper copies
are the ultimate original.

Technical Details
Documents (files) official electronic
copies (usually with a corresponding
signed paper copy). The currently
supported files are Excel and plain text
files (.csv, .txt). These can be easily
imported directly into the SQL database
without any data (re-)entry. Currently in
M&G all forms that are being imported into
the SQL database are Excel 2010 forms
that are locked with a password to prevent
erroneous or malicious modifications to
the forms.

Technical Details
Documents Database synchronisation
two different strategies:
old - currently used for 90% of M&G
data
new - being developed in order to
simplify the data flow and improve the
times of the synchronisation process.

Technical Details
Old strategy
Once files have been added, modified or deleted on the
shared drive a dedicated user opens a special Excel
spreadsheet and presses the only available button with
caption Synchronisation. This action invokes a VB
macro script that converts the new and updated Excel
files into csv files and calls the SQL servers import
procedure. This procedure imports the csv files into
relational database structures and data become available
for immediate reporting. The synchronisation events are
recorded into an event log that can be used to
investigate corrupted or missing documents and other
issues.

Technical Details
Old Strategy Example TC Daily Works

Technical Details
New Strategy
A scheduled SQL server task runs every 5-10
minutes and identifies changes on the shared
drive. If any changes detected then the new and
updated files are accessed directly via OLE DB
drivers and their content stored in the database
tables. Data that belong to files that have been
removed from the file server is being deleted. The
synchronisation events are stored into an event
log table that can be used to investigate
corrupted or missing documents and other issues.

Technical Details
New Strategy only DB Admin User
Interface

Technical Details
Database schema buffer tables and data
warehouse tables:
Buffer tables - Raw file data stored with no
validation represents a mirror image of the
file data but already split into columns and
rows
Data warehouse tables specific tables with
optimised performance populated with the
good data only filtered out from the buffer
tables. All reports are generated on the fly from
these tables.

Reporting
various options, currently in M&G:
automated emails (sent on specific event and on
scheduled time of the day)
automated file exports daily mined ore & waste
report
Excel tables populated with data that is refreshed on
open or whenever needed used for daily tracking of
performance and EOM reporting
Web Reports interactive charts and tables. Based on
SQL Server Reporting Services (SSRS)
direct link from Geovia Surpac to the tables
containing geological data

Reports

Example
TC Daily Works example old
strategy

Example
Form A Incident Report - new
strategy

Q&A
Excel forms required is this a
problem?
Who (user groups) will use the
system?
What events and what actions from
database system perspective?
What kind of reports? Good starting
point to find out what we need in the
database tables.
Your questions?