Types of Data:

The term Data is used to describes pieces of information in computing.

- Structured Data:
This is data that has been organised with tags in a context. An example of a collection of
structured data is a database.

- Unstructured Data:
Data with no context is known as unstructured data. This is usually found in the form of text,
images and other data.

Ensuring Data Quality:

Validation:
Validation is very important when maintaining a database. It ensures that data of different
types are input into the database are entered in a specific format. For example, the current
date can be written in a number of different formats, but choosing one makes it easier to
store and query the data. At DNEG we use the ISO 86001 Date Notation, on the left is
incorrectly written dates with the ISO formatted date on the right:

Unvalidated Input Validated Input

19/09/2013-15:53:00 2013-09-19T15:53:00

2012-11-16 15:53:00 2012-11-16T15:53:00

T15:53:00 05/08/2017 2018-05-25T15:53:00

02/10/2019 2019-02-10T15:53:00

Verification:
Whether the data is being inputted manually or through an automated process there is a
potential for data to be input incorrectly. Usually, input errors are caused through human
error so one method of verifying this data is asking the user to enter the information twice.
The most common use of this is in password setup/reset systems.

Tables comprising of structured data are used in Relational Databases. These tables have
keys that reference the table they’re assigned to. The primary key is a unique id of a current
record in the table, A secondary key is an id assigned to a referenced table. I’ll explain
relational databases in the next section.
Relational Databases:

Relational databases consist of tables and rows, known as records, which are stored in a
table. They refer to a primary key which are per-row values that are a unique value assigned
per-row (usually a hash-code or an integer). These primary keys are referenced from other
tables and it allows for a greater level of complexity that isn’t possible in traditional
2-dimensional database types.

Database tables are created for various object types, in the context of visual effects, there
could be tables for shots, assets and asset versions. If an asset will only be specific to a
particular shot or sequence, they will be references to the shot or sequence as a primary
key.

Q​uerying a Database:
Shotgun provides an API which allows developers and TDs to construct queries that returns
the corresponding information back. The APIs make querying, editing or deleting entries
from the database much more user-friendly and less dangerous than if you were directly
interacting with it via MySQL.

Here’s a small example of a simple shotgun database query in python taken from my initial
training that returns a list of the current active shows that are in production.

from​ shotgun.common ​import​ conn

current_active_shows = conn.shn.find(
​"Project"​,
[[​"sg_status"​, ​"is"​, ​"Active"​]],
fields=[​"name"​, ​"sg_status"​]
)

This is the formatted output of that query:

[
{
​"sg_status"​: ​"Active"​,
​"type"​: ​"Project"​,
​"id"​: ​1​,
​"name"​: ​"SHOW_A"
},
{
​"sg_status"​: ​"Active"​,
​"type"​: ​"Project"​,
​"id"​: ​2​,
​"name"​: ​"SHOW_B"
}
]