MQL-to-SQL

a JSON-based query language

for RDBMS data access from AJAX applications

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 Welcome!
● Roland Bouman, Leiden, the Netherlands
● Ex-MySQL, ex-Sun Microsystems
● Web & BI developer at Strukton Rail
● Author
– “Pentaho Solutions”
– “Pentaho Kettle Solutions”
● http://rpbouman.blogspot.com,
@rolandbouman

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 Agenda
● The data access problem
● What's wrong with SQL?
● The Metaweb Query Language (MQL)
● MQL-to-SQL
● Demo
● Questions

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 Agenda
● The data access problem
● What's wrong with SQL?
● The Metaweb Query Language (MQL)
● MQL-to-SQL
● Demo
● Questions

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 The Data Access Problem
● Classical Client/Server
SQL
Resultset

Desktop Application
● Web application RDBMS

SQL
? Resultset

Browser HTTP
MQL-to-SQL: a JSON-based Query language for your RDBMS
Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 The Data Access Problem
● Build a service
– Protocol on top of HTTP: REST, RPC
– Data format: JSON, XML
● Advantage: controlled data access
● Disadvantage: controlled=limited

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 The Data Access Problem
● Build a query service
– Example: dbslayer

SQL SQL
JSON Resultset

● Advantage: unlimited data access

● Disadvantage: unlimited=uncontrolled
MQL-to-SQL: a JSON-based Query language for your RDBMS
Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 Agenda
● The data access problem
● What's wrong with SQL?
● The Metaweb Query Language (MQL)
● MQL-to-SQL
● Demo
● Questions

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 What's wrong with SQL?
● Nothing of course, it's great!!!
– very powerful
– expressive syntax
– declarative
– designed for the RDBMS
– portable
– well supported

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 What's wrong with SQL?
● SQL is very powerful
– ...way too powerful
● SQL has expressive syntax
– ...and too hard to parse and generate
● SQL is declarative
– ...but not enough
● SQL is designed for the RDBMS
– ...can't escape relational model

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 SQL not declarative enough
Language:
SELECT f.title language_name
, f.description
, f.release_year Spoken in
, f.length
, f.rating Film:
, a.first_name
, a.last_name title
, l.name description Actor:
FROM film f release_year
INNER JOIN language l length last_name
rating first_name
ON f.language_id = l.id
LEFT JOIN film_actor fa Has cast Starring in
ON f.id = fa.film_id
LEFT JOIN actor a Film_Actor:
ON fa.actor_id = a.id
Film Key
Actor Key
MQL-to-SQL: a JSON-based Query language for your RDBMS
Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 SQL not declarative enough
Language:
SELECT f.title language_name
, f.description
, f.release_year Spoken in
, f.length
, f.rating Film:
, a.first_name
, a.last_name title
, spoken_in.name description Actor:
FROM film f release_year
RELATE spoken_in language l length last_name
rating first_name
, has_cast film_actor fa
RELATE starring_in actor a Has cast Starring in

Film_Actor:

Film Key
Actor Key
MQL-to-SQL: a JSON-based Query language for your RDBMS
Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 SQL can't escape the RM
header
Header
– line 1a
– line 2b Split off line 1a
– multivalued attributes line 2b
– repeating groups
Unnormalized data First normal form (1NF)

??? Remove
redundancy
line 1a
header
line 2b

header line 1a
header line 2b SQL queries: 1
– JOIN operation 2
Denormalized data (1NF) Higher normal forms (>= 3NF)
MQL-to-SQL: a JSON-based Query language for your RDBMS
Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
SQL can't escape the RM
Language:
language_name

Film:
title
Category: description Actor:
release_year
name length last_name
rating first_name

Film_Category: Film_Actor:

Film Key Film Key

Category Key Actor Key
MQL-to-SQL: a JSON-based Query language for your RDBMS
Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 SQL can't escape the RM
SELECT f.*
, l.*, ol.*
, (SELECT *
FROM film_category fc
INNER JOIN category c
ON fc.category_id = c.id
WHERE fc.film_id = f.film_id)
, (SELECT *
FROM film_actor fa
INNER JOIN actor a
ON fa.actor_id = a.id
WHERE fa.film_id = f.film_id)
FROM film f
INNER JOIN language l
ON f.language_id = l.id
INNER JOIN language ol
ON f.original_language_id = ol.id
MQL-to-SQL: a JSON-based Query language for your RDBMS
Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 SQL can't escape the RM
SELECT f.*
, l.*, ol.*
, c.*, a.*
FROM film f
INNER JOIN language l
ON f.language_id = l.id
INNER JOIN language ol
ON f.original_language_id = ol.id
LEFT JOIN film_category fc
ON f.id = fc.film_id
LEFT JOIN category c
ON fc.category_id = c.id
LEFT JOIN film_actor fa
ON f.id = fa.film_id
LEFT JOIN actor a
ON fa.actor_id = a.id

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 How do we get rid of SQL?
● Object relational mapper?
● NoSQL?
– Document databases
– ACID
– Schema flexibility
● Can we preserve the RM?

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 Agenda
● The data access problem
● What's wrong with SQL?
● The Metaweb Query Language (MQL)
● MQL-to-SQL
● Demo
● Questions

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 The Metaweb Query Language
● MQL – rhymes with “pickle”
● Native query language of Freebase
– http://www.freebase.com/
– http://www.freebase.com/docs
● JSON over HTTP
– Natural fit for AJAX applications

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 Freebase MQL Services
● http://api.freebase.com/api/service/mqlread?
query={"query":[{Your MQL here}]}
● http://api.freebase.com/api/service/mqlread?
queries={"q1": {"query":[{Your MQL here}]},
"q2": {"query":[{Your MQL here}]}}
● http://api.freebase.com/api/service/mqlwrite

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 JSON
● Subset of JavaScript
– Object: set of property/value pairs
– Scalar: boolean, null, number, string
– Array: list of values
{
"type": "/sakila/film",
"film_id": 1,
"title": "ACADAMY DINOSAUR",
"language": {
"language_id": 1,
"language_name": "English"
},
"category":["Action","Adventure"]
}
MQL-to-SQL: a JSON-based Query language for your RDBMS
Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 JSON is great for AJAX
● Modern Web applications
– Asynchronous JavaScript and XML
– ...except it's JSON rather than XML
● 1 to 1 runtime representation
– XML requires mapping, DOM traversal
– JSON support in all modern browsers
● Cross-domain requests (JSONP)
– <script> injection

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 MQL Queries
● JSON-based
– Both Query and Result
● MQL is “query by example”
– MQL Query is an object template
– Finds objects matching the template
– Fills in value placeholders in template
● Query & Result symmetry
– Easy to parse, easy to generate

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 Basic MQL query
● Property/value pairs act as filter
● Value placeholders return data
Query: Result:
{ {
"type": "/sakila/film", "type": "/sakila/film",
"film_id": 1, "film_id": 1,
"title": null, "title": "ACADAMY DINOSAUR",
"language": { "language": {
"language_id": null, "language_id": 1,
"name": null "language_name": "English"
}, },
"category": [] "category":["Action","Adventure"]
} }
MQL-to-SQL: a JSON-based Query language for your RDBMS
Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 Meta: Objects & Properties
● Objects are bags of properties
● Properties
– name/value pairs
– Single- or multi-valued (array)
– Value can be scalar or object
● No explicit relationships
– Implicit: properties with object values

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 Meta: Types
● Types define a bag of properties
– Objects may have multiple types
– Types are grouped in domains
● In MQL, use the type property
– type: "/domainname/typename"
● Property has exactly 1 expected type

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 MQL Joins/subqueries
● Implicit via object-type properties
● Can't specify join conditions

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 MQL operators
● Property/value act as equals operator
● Explicit operators for advanced filters
● !=, <, <=, >=, >
– Have their usual meaning
● |= and !|= test for membership
– "one of" and "not one of"
● ~= for word pattern matching

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 MQL operators
● Operators follow the property name
● Operations not returned in the result
Query: Result:
{ {
"type": "/sakila/film", "type": "/sakila/film",
"film_id<": 2, "film_id": 1
"title~=": "ACADEMY", }
"language!=": "Danish",
"category|=": [
"Action", "Adventure"
],
"film_id": null
}
MQL-to-SQL: a JSON-based Query language for your RDBMS
Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 MQL property aliases
● Implement an AND operation
Query:
{
"type": "/sakila/film",
"film_id": null,
"title": null,
"1:category": "Action",
"2:category": "Adventure"
}

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 MQL directives
● Reserved property names
– limit
– return: count
– cursor
– * wildcard

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 Agenda
● The data access problem
● What's wrong with SQL?
● The Metaweb Query Language (MQL)
● MQL-to-SQL
● Demo
● Questions

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 MQL-to-SQL
● Implementation of mqlread service
– PHP using PDO database access
– MySQL, PostgreSQL, SQLite, Oracle
– http://code.google.com/p/mql-to-sql
– GPL
– Lacks many features, has many bugs
– don't use in hospitals and airplanes

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 MQL-to-SQL
● MQL domain maps to a SQL schema
● MQL type maps to a SQL table
● MQL property maps to
– SQL column
– Relationship (foreign key)
● Mappings defined in JSON file

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 MQL-to-SQL
index.php config.php
<?php <?php
include
'config.php'; $connection_file_name = '';

$metadata_file_name = '';

...code...

?> ?>
MQL-to-SQL: a JSON-based Query language for your RDBMS
Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 MQL-to-SQL metadata file
{
"domains": {
"sakila": {
"schema_name": "",
"types": {
"actor": {
"properties": {
"actor_id": {
"column_name": "actor_id",
"nullable": false,
"type": "/type/int"
},
"starring_in": {
"type": "/sakila/film_actor",
"direction": "referenced<-referencing",
"join_condition": [
{
"referencing_column": "actor_id",
"referenced_column": "actor_id"
},

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 Agenda
● The data access problem
● What's wrong with SQL?
● The Metaweb Query Language (MQL)
● MQL-to-SQL
● Demo
● Questions

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 Agenda
● The data access problem
● What's wrong with SQL?
● The Metaweb Query Language (MQL)
● MQL-to-SQL
● Demo
● Questions

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman
 Finally
● I need your feedback
● Love to see this inside a RDBMS
– drizzle plugin?
– MySQL plugin?
– PostgreSQL SP language?
● http://code.google.com/p/mql-to-sql/
● Online query editor

MQL-to-SQL: a JSON-based Query language for your RDBMS

Project: http://code.google.com/p/mql-to-sql/
Contact: http://rpbouman.blogspot.com - @rolandbouman