Intro to PostgreSQL

Mike Callahan WFO Louisville September 8, 2009

Topics

Relational database theor !ost"reS#L Re$eren%e ps&l Snoop S#L s nta'

S(L(C) *!+,)( -.S(R) -.)O +(L()( FROM CR(,)( )(M! ),/L( CO!0

Databases in AWIPS
Sources
Data

PostgreSQL

SQL
SQ L

D2D WHFS
Other ,ppli%ations

SQ

psql pg_...

Databases

Relational Theory

+ata are stored in "roups %alled 1tables1 or 1relations1 )ables %onsist o$ 1ro2s1 or 1re%ords1 and 1%olumns1 or 1$ields1 Ro2s are usuall identi$ied b a uni&ue 1ke 1 2hi%h ma be a sin"le %olumn or a "roup o$ %olumns Columns %an be linked to other tables 2ith similar %olumns 3ood desi"n o$ the database stru%ture or 1s%hema1 is %riti%al4

5,ll the data in a ro2 should be dependant on the ke , the 2hole ke , and nothin" but the ke 45

Working with databases

Most o$ the time, ou 2ill 2ork 2ith e'istin" databases 6o2ever, ou still need to kno2 the s%hema We 2ill %on%entrate on $our tasks
('tra%tin" data Chan"in" data -nsertin" data +eletin" data

Why PostgreSQL

S2it%hin" database server $rom 6! to Linu' !ost"reS#L is $ree, -n$ormi' $or Linu' is not !ost"reS#L is the most advan%ed open sour%e relational database, M S#L did not have tri""ers, !ost"reS#L did 7ersion on ,W-!S is 842, latest version is 848 +o%umentation is e'%ellent and is at 2224post"res&l4or"9do%s98429444 We 2ill use version 842 $or our e'amples

PostgreSQL Documentation

!re$a%e

3ives histor , %onventions, et%4 , &ui%k introdu%tion to S#L , des%ription o$ the standard and the !ost"reS#L additions +etails o$ ea%h %ommand

)utorial

S#L Lan"ua"e

Re$eren%e: S#L Commands

ps l
!"interacti#e"app$ps l%html

)he standard !ost"reS#L appli%ation to 2ork 2ith databases -t uses a superset o$ standard S#L -t %an be intera%tivel like +/a%%ess, or in s%ripts like s&l%md

&ommon ps l &ommand Line 'ptions

;,, ;;no;ali"n: set output to non;ali"ned, no paddin" ;% sql, ;;%ommand sql: e'e%ute the s&l %ommand and then e'it ;d name, ;;dbname name: name o$ database, same as name as the $irst non;option ar"ument ;$ name, ;;$ile name: use name as the sour%e o$ %ommands ;o name, ;;output name: put the output in name ;&, ;;&uiet: suppress 2el%ome messa"es ;t, ;;tuples;onl : suppress print %olumn names, result ro2 %ounters, et% ;<, ;;help: "et %ommand line help

&ommon ps l (eta$commands

=a: to""le output $ormat >ali"ned9unali"ned? =% name: %onne%t to a database name =%op table: %op a table to9$rom a $ile =d : list all tables >displa ? =d table: sho2 in$ormation about a table =e: edit the &uer bu$$er =": e'e%ute the &uer bu$$er >"o? =h command: displa help on command =i name: read name into &uer bu$$er >input?

&ommon ps l (eta$commands

=o name: send output to name =p: displa the &uer bu$$er =&: &uit the pro"ram =r: resets >%lears? the &uer bu$$er =t: to""le the output headers on9o$$ =2 name: 2rite the &uer bu$$er to name =@ command: e'e%ute the Linu' command =<: help on meta;%ommands

Interacti#e ps l

) pin" in a &uer loads the &uer bu$$er )he %ommand 2ill not run unless terminated 2ith a semi%olon >A? Meta;%ommands all start 2ith bla%kslash >=? ('it 2ith =&

Snoopy

Snoop is an intera%tive, 3*- $ront;end to ps&l -t 2as desi"ned $or the h dro database, but 2ill 2ork 2ith an ,W-!S database #ueries are in a s%rollin" 2indo2, data is in a separate s%rollin" 2indo2 6elp is available about tables, %olumns, ro2s, S#L s nta', and h dro database data di%tionar ,vailable on the ,W-!S L,+

Snoopy Screen Shot

Snoopy )elp

,vailable onl $or the h dro database

SQL *or PostgreSQL

!"interacti#e"s l$commands%html

S#L, pronoun%ed Bse&uelC, is the standard 2a o$ 2orkin" 2ith relational databases )he standard is 2ell;supported but 2eak so ever database mana"er adds %ommands to the standard -n these e'amples, S#L ke 2ords 2ill be C,!-),L-D(+ 6o2ever, %ase is .O) important e'%ept in strin"s

+asic Data Types in SQL

Chara%ter, 7ar%har: , se&uen%e o$ alphanumeri% %hara%ters en%losed in sin"le &uotesA a string 123 )o in%lude a sin"le &uote in a strin" double itA 'Mi e''s string' -nte"ers: , number 2ithout a de%imal pointA 12!" #2 Real: , number 2ith a de%imal pointA 3.1$1%" 2!.&& +e%imal, .umeri%: Fi'ed de%imal point, a%%urate but slo2 math /oolean: '()* or F+LS* in ps&l )R*( is displa ed as t, F,LS( is displa ed as $

Date"Time Data Types in SQL

+ate: )he 2ord $ollo2ed b a date en%losed in sin"le &uotesA D+'* 2&&%#2#1, Other $ormats are allo2ed but this is the standard4 )ime: )he 2ord $ollo2ed b a time en%losed in sin"le &uotesA '-M* &2.3$.&& Se%onds and %olons are optional4 )imestamp: /othA '-M*S'+MP 2&&$#1&#23 13.2&.&& -nterval: , time intervalA -/'*(0+L 2 1a2s" -/'*(0+L 12 3o4rs Man times ou in !ost3reS#L ou don1t need to t pe the des%riptor4 For e'ample: '2&&%#2#1,'" '&2.3$'" et%4

Special ,alues

)-M(S),M! E.OWF

)he %urrent time, date, or timestamp )he %urrent date or midni"ht timestamp )omorro2 date or midni"ht timestamp 0esterda date or midni"ht timestamp Midni"ht 00:00:00 *)C time

+,)( E)O+,0F

)-M(S),M! E)OMORROWF

+,)( E0(S)(R+,0F

)-M( E,LL/,LLSF

&ommon SQL &ommands

S(L(C)

('tra%t data $rom one or more tables Chan"e data in a table &ui%kl ,dd ne2 data to a table Remove data $rom a table Create a temporar table Send the %ontents o$ a table to a $ile or vi%e versa

*!+,)(

-.S(R) -.)O

+(L()( FROM

CR(,)( )(M! ),/L(

CO!0

S-L-&T

Most S#L 2ork involves %reatin" S(L(C) statements 0ou %annot harm data usin" a S(L(C) Command S(L(C) statements %an be simple but %an also be almost in%omprehensible S(L(C) statements are broken up into %lauses

FROM, W6(R(, 3RO*! /0, 6,7-.3, *.-O., -.)(RS(C), (GC(!), OR+(R /0, L-M-), FOR *!+,)(

We 2ill onl look at a $e2 simple &ueries

+asic S-L-&T

S(L(C)

List o$ %olumns to e'tra%t List o$ tables to 2ork 2ith Condition to limit sele%tion 3roupin" e'pression Condition to limit "roupin" Sort order .umber o$ ro2s to e'tra%t

FROM

W6(R(

3RO*! /0

6,7-.3

OR+(R /0

L-M-)

S-L-&T &ommand Structure

S*L*5' F(6M WH*(* 7(6)P 89
List o$ tables Condition ('pression

List o$ %olumns

H+0-/7

6(D*( 89
Sort order

L-M-'
.umber

Condition

+uilding a S-L-&T Query

0ou must kno2 the s%hema >stru%ture? o$ our database to kno2 the tables and the %olumns ou 2ant to e'tra%t Hno2 the %ondition ou 2ant to $ilter the sele%tion Hno2 the sort order ou 2ant to see the data, the order 2ill be unde$ined i$ not spe%i$ied

-.ample
Find all -+s and names in Hansas sorted b name

)ables and %olumns

lo%ation: lid, name )able names need to be in%luded onl i$ %olumn names are ambi"uous lo%ation4state I 1HS1 lo%ation4name

Condition

Order

S*L*5' li1" na:e F(6M lo;ation WH*(* state < '=S' 6(D*( 89 na:e>

/oining Two Tables

Retrieve the list o$ lo%ations, ids, and observers in HS sorted b observer1s last name

)ables and %olumns

lo%ation: lid, name observer: lastname lo%ation4lid I observer4lid and lo%ation4state I 1HS1 observer4lastname

Condition

Sort order

S*L*5' li1" na:e" lastna:e F(6M lo;ation" obser?er WH*(* lo;ation.li1 < obser?er.li1 +/D lo;ation.state < '=S' 6(D*( 89 lastna:e>

(ore on W)-R- &lause

Comparison operators

I, @I or JK, K, KI, J, JI ,.+, OR, .O)

Lo"i%al operators

Strin" %on%atenation 1a1 LL 1strin"1 /()W((. operator

W6(R( $s /()W((. M0 ,.+ N0 W6(R( state -. >1H01, 1-.1, 1O61? W6(R( $s -S .*LL

-. operator

-S .*LL test

(atching in &haracter &olumns

L-H(
*sed to set up sear%h tar"ets O ; Pero or more %hara%ters Q ; a sin"le %hara%ter

W6(R( lid L-H( 1QQQHM1 or lid L-H( 1QQQH21

*sed to set up !OS-G re"ular e'pression sear%hes

W6(R( lid R 1SHTM2U1 1S=2=d1 be%omes e1S==2==d1

*se es%ape notation $or ba%kslash e'pressions

Substrings

*se to e'tra%t part o$ a %hara%ter %olumn based on position >%ount $rom M?

S*/S)R-.3>%olumn FROM be"in FOR len"th?

('4 sho2 phone number o$ observers in the Hansas Cit HS Metro area %ode

S*L*5' na:e" 3p3one F(6M obser?er WH*(* S)8S'(-/7@3p3one F(6M 1 F6( 3A < 'B13'

S-L-&T tricks

)estin" an e'pression

S*L*5' S)8S'(-/7@'+85D*' F(6M 1 F6( 3A S*L*5' C F(6M lo;ation S*L*5' l.li1" lastna:e F(6M lo;ation l" obser?er o WH*(* l.li1<o.li1 +/D lastna:e L-=* 'aD' S*L*5' li1 +S Lo;ation_-D F(6M lo;ation... S*L*5' D-S'-/5' ;o4nt2 F(6M lo;ation S*L*5' li1" na:e F(6M lo;ation 6(D*( 89 2 ...6(D*( 89 li1 D*S5

3et all the $ields in a table

.ame short%ut

Rename %olumns

(liminate dupli%ate ro2s

Order short%ut

+es%endin" order

Date"Time"Timestamp 0unctions

,3(>timestamp, TtimestampU?

%al%ulate interval bet2een t2o timestamps, se%ond ar"ument de$aults to .OW SW6(R( ,3(>obstime? J -.)(R7,L 1V hours1 e'tra%t the unit $rom a timestamp %ommon units: %entur , da , de%ade, do2 >da o$ 2eek SunI0, MonIM, et%?, do >Wulian date?, hour, minute, month, &uarter, se%ond, timePone, 2eek, ear SW6(R( (G)R,C)>do2 $rom dat%rst? I X

(G)R,C)>unit $rom timestamp?

1roup 0unctions

CO*.)>?

-nstead o$ displa in" the ro2s, Yust %ount them S(L(C) CO*.)>Z? FROM observer Cal%ulate the avera"e o$ a %olumn Cal%ulate the ma'imum in a %olumn Cal%ulate the minimum in a %olumn Cal%ulate the sum o$ a %olumn

,73>?

M,G>?

M-.>?

S*M>?

The 1R'2P +3 &lause

When usin" the previous $un%tions, the 3RO*! /0 %lause tells 2hat to "roup We 2ant the hi"hest %rest at (7(R0 lo%ation, sorted b -+ -$ 2e don1t use 3RO*! /0 2e 2ill "et the hi"hest %rest o$ ,LL -+s instead o$ the hi"hest %rest at (7(R0 -+4

S*L*5' l.li1" M+E@stageA F(6M lo;ation l" ;rest ; WH*(* l.li1 < ;.li1 7(6)P 89 l.li1 6(D*( 89 l.li1>

The 2PDAT- &ommand

*sed to &ui%kl %han"e the values in a %olumn )his %ommand %an save a lot o$ 2ork but %an also be dan"erous 6as S(), FROM, and W6(R( %lauses 6ere is a tri%k:
Create the W6(R( %lause but use S(L(C) Z 0ou 2ill see the ro2s ou are "oin" to %han"e /uild rest o$ &uer

The 2PDAT- &lauses

S()

)he %olumn and the value ou 2ant to assi"n it , list o$ tables ou used in the S() %lause )he %onditions the data must mat%h

FROM

W6(R(

2PDAT- &ommand Structure

)PD+'*
)able

S*'

F(6M

WH*(*
List o$ tables Condition

Column and value

2sing 2PDAT

('4 9MX area %ode %han"es to 9M8 )able and %olumn

observer: hphone E9M8F LL S*/S)R-.3>hphone FROM 8? S*/S)R-.3>hphone FOR X? I 19MX1

7alue

Condition

Check WHERE Clause

S*L*5' C F(6M obser?er WH*(* S)8S'(-/7@3p3one F6( 3A < 'B13'

UPDATE rows

)PD+'* obser?er S*' 3p3one < 'B1$ FF S)8S'(-/7@3p3one F(6M $A WH*(* S)8S'(-/7@3p3one F6( 3A < 'B13'

The I4S-RT I4T' &ommand

-nsert data into a ne2 ro2 6as onl 7,L*(S %lause Can also use a S(L(C) &uer

The I4-SRT &lause

7,L*(S
, list o$ data that ou 2ant to insert -$ the order o$ the %olumns is not listed, it must mat%h the order de$ined in the s%hema For sa$et sake, al2a s list the %olumns

S(L(C)

, standard sele%t &uer , but the s%hema o$ the result must mat%h the tar"et

I4S-RT &ommand Structure

-/S*(' -/'6 0+L)*S
)able, %olumn list List o$ values

-or-

-/S*(' -/'6 S*L*5'

)able, %olumn list Sele%t &uer

2sing I4S-RT

Want to add a ne2 ro2 in datum datum s%hema is

lid ; lo%ation id ddate ; datum date elev ; elevation

Our values are ,/(HM, 2002;X;2X, X0NV4X8

-/S*(' -/'6 1at4: @li1" 11ate" ele?A 0+L)*S @'+8*=1'" '2&&2#3#23'" 3&!%.3$A>

2sing I4S-RT 5con6t7

Want to add data $rom a temporar table named Yunk into the hei"ht table Yunk and hei"ht have the same s%hema

-/S*(' -/'6 3eig3t S*L*5' C F(6M G4n >

The D-L-T- 0R'( &ommand

Remove ro2s 6as onl W6(R( %lause )here is no &ui%k undo so be %are$ul -$ ou a%%identall leave o$$ the W6(R( %lause, ,LL the ro2s 2ill be deleted@ 6ere is another tri%k:
Create the W6(R( %lause but use S(L(C) Z 0ou 2ill see the ro2s ou are "oin" to delete Chan"e S(L(C) Z to +(L()(

The D-L-T- &lause

W6(R(

)he %ondition the ro2s must mat%h

D-L-T- &ommand Structure

D*L*'* F(6M WH*(*
)able Condition

2sing D-L-T

Want to remove an observer to make a lo%ation ina%tive )he lo%ation -+ is C,.HM

Check WHERE Clause

S*L*5' C F(6M obser?er WH*(* li1 < '5+/=1'

DELETE row

D*L*'* F(6M obser?er WH*(* li1 < '5+/=1'

The &R-AT- T-(P TA+L&ommand

Creates a temporar table that %an be used in $uture &ueries )able is automati%all deleted on%e ps&l session is %ompleted4 -n Snoop , table is deleted at end o$ last statement in &uer 4 -t has a simple stru%ture

&R-AT- T-(P TA+L- &ommand Structure

+S
)able Sele%t &uer

+'* '*MP '+8L*

2sing &R-AT- T-(P TA+L

Want to %op "a"e readin"s $or the last three da s into a temp table, Yunk

5(*+'* '*MP '+8L* G4n +S S*L*5' C F(6M 3eig3t WH*(* +7*@obsti:eA H -/'*(0+L '3 1a2s'>

The &'P3 &ommand

)here is a &ui%k and eas 2a to %op a table to a $ile 0ou %ould pro%ess a table 2ithout 2orkin" in S#L )his is similar to the -n$ormi' *.LO,+ %ommand -t has man options but 2e 2ill onl look at a $e2 -t %ombines 2ell 2ith the CR(,)( )(M! ),/L( %ommand 0ou must be lo""ed on as postgres $or this %ommand to 2ork

&'P3 &ommand Structure

56P9
)able

F(6M '6 W-'H

File Options

-.ample &'P3 &ommands

56P9 obser?er '6 'obser?er.1at' W-'H D*L-M-'*( 'F'> Same as -n$ormi' *.LO,+ %ommand e'%ept de$ault delimiter is a ),/ 56P9 ;rest '6 ';rest.;s?' W-'H 5S0> 3ood 2a o$ movin" data into ('%el 56P9 obser?er F(6M 'obser?er.1at' W-'H D*L-M-'*( 'F'> Movin" data into a table

&ombining &R-AT- T-(P TA+L- and &'P3

We 2ant to %op toda 1s sta"es into a $ile, st"Qtd 4dat

5(*+'* '*MP '+8L* G4n +S S*L*5' C F(6M 3eig3t WH*(* +7*@obsti:eA H -/'*(0+L '1 1a2'> 56P9 G4n '6 stg_t12.1at>

In &onclusion

!ost"reS#L ma not be as popular as M S#L, it 2ill do 2hat 2e need it to do )he !ost"reS#L do%umentation is e'%ellent and available on;line as 6)ML or o$$;line as !+F ps&l is "ood to use in s%ripts Snoop is "ood to use intera%tivel )he best 2a to learn S#L is to pla 2ith it 0ou %annot harm data b S(L(C) &ueries