Awk

Awk
Mon Jan 20 15:36:59 UTC 2014

Part of the Unix tutorials And then there's My blog
Table of Contents
Why learn AWK?
Basic Structure
Executing an AWK scrit
Which shell to use with AWK?
!yna"ic #ariables
The Essential Syntax of AWK
Arith"etic Exressions
Unary arith"etic oerators
The Autoincre"ent an$ Auto$ecre"ent %erators
Assign"ent %erators
Con$itional exressions
&egular Exressions
An$'%r'(ot
Co""an$s
AWK Built)in #ariables
*S ) The +nut *iel$ Searator #ariable
%*S ) The %utut *iel$ Searator #ariable
(* ) The (u"ber of *iel$s #ariable
(& ) The (u"ber of &ecor$s #ariable
&S ) The &ecor$ Searator #ariable
%&S ) The %utut &ecor$ Searator #ariable
*+,E(AME ) The Current *ilena"e #ariable
Associati-e Arrays
Multi)$i"ensional Arrays
Exa"le of using AWK.s Associati-e Arrays
%utut of the scrit
/icture /erfect /&+(T* %utut
/&+(T* ) for"atting outut
Escae Se0uences
*or"at Secifiers
Wi$th ) secifying "ini"u" fiel$ si1e
,eft 2ustification
The *iel$ /recision #alue
Exlicit *ile outut
AWK (u"erical *unctions
Trigono"etric *unctions
Exonents3 logs an$ s0uare roots
Truncating +ntegers
Awk - A Tutorial and Introduction - by Bruce Barnett http://www.grymoire.com/Unix/Awk.html
1 of 233 25-02-2014 12:21
4&an$o" (u"bers
The ,otto scrit
String *unctions
The ,ength function
The +n$ex *unction
The Substr function
5AWK.s Tolower an$ Touer function
The Slit function
(AWK.s string functions
The Match function
The Syste" function
The 5etline function
The systi"e function
The Strfti"e function
User !efine$ *unctions
AWK atterns
*or"atting AWK rogra"s
En-iron"ent #ariables
A&5C ) (u"ber or argu"ents 6(AWK'5AWK7
A&5# ) Array of argu"ents 6(AWK'5AWK7
A&5+(! ) Argu"ent +n$ex 65AWK only7
*(& 6(AWK'5AWK7
%*MT 6(AWK'5AWK7
&STA&T3 &,E(5T8 an$ "atch 6(AWK'5AWK7
SUBSE/ ) Multi)$i"ensional array searator 6(AWK'5AWK7
E(#+&%( ) en-iron"ent -ariables 65AWK only7
+5(%&ECASE 65AWK only7
C%(#*MT ) con-ersion for"at 65AWK only7
E&&(% ) syste" errors 65AWK only7
*+E,!W+!T8S ) fixe$ wi$th fiel$s 65AWK only7
AWK3 (AWK3 5AWK3 or /E&,
Copyriht 1994!1995 "r#$e "arnett and %enera& '&e$tri$ Co(pany
Copyriht 2001!2004!2013 "r#$e "arnett
A&& rihts reser)ed
*o# are a&&o+ed to print $opies of this t#toria& for yo#r persona& #se! and
&in, to this pae! -#t yo# are not a&&o+ed to (a,e e&e$troni$ $opies! or
redistri-#te this t#toria& in any for( +itho#t per(ission.
/riina& )ersion +ritten in 1994 and p#-&ished in the 0#n /-ser)er
1ast Update: 2ed 3e- 6 04:02:35 '0T 2013
A+, is an e5tre(e&y )ersati&e prora((in &an#ae for +or,in on fi&es.
2e'&& tea$h yo# 6#st eno#h to #nderstand the e5a(p&es in this pae! p&#s
a s(iden.
The e5a(p&es i)en -e&o+ ha)e the e5tensions of the e5e$#tin s$ript as
part of the fi&ena(e. /n$e yo# do+n&oad it! and (a,e it e5e$#ta-&e! yo#
$an rena(e it anythin yo# +ant.
Why learn AWK?
7n the past 7 ha)e $o)ered grep and sed. This se$tion dis$#sses A28!
another $ornerstone of U97: she&& prora((in. There are three
)ariations of A28:
A28 ; the <)ery o&d= oriina& fro( AT>T
2 of 233 25-02-2014 12:21
9A28 ; A ne+er! i(pro)ed )ersion fro( AT>T
%A28 ; The 3ree 0oft+are fo#ndation's )ersion
/riina&&y! 7 didn't p&an to dis$#ss 9A28!
-#t se)era& U97: )endors ha)e rep&a$ed
A28 +ith 9A28! and there are se)era&
in$o(pati-i&ities -et+een the t+o. 7t
+o#&d -e $r#e& of (e to not +arn yo#
a-o#t the differen$es. 0o 7 +i&& hih&iht
those +hen 7 $o(e to the(. 7t is
i(portant to ,no+ than a&& of A28's
feat#res are in 9A28 and %A28. Most! if
not a&&! of 9A28's feat#res are in %A28.
9A28 ships as part of 0o&aris. %A28 does
not. ?o+e)er! (any sites on the 7nternet
ha)e the so#r$es free&y a)ai&a-&e. 7f yo#
#ser 1in#5! yo# ha)e %A28. "#t in
enera&! ass#(e that 7 a( ta&,in a-o#t
the $&assi$ A28 #n&ess other+ise noted.
2hy is A28 so i(portant@ 7t is an
e5$e&&ent fi&ter and report +riter. Many
U97: #ti&ities enerates ro+s and $o&#(ns
of infor(ation. A28 is an e5$e&&ent too& for
pro$essin these ro+s and $o&#(ns! and is
easier to #se A28 than (ost $on)entiona&
prora((in &an#aes. 7t $an -e
$onsidered to -e a pse#do;C interpretor!
as it #nderstands the sa(e arith(ati$
operators as C. A28 a&so has strin
(anip#&ation f#n$tions! so it $an sear$h
for parti$#&ar strins and (odify the
o#tp#t. A28 a&so has asso$iati)e arrays!
+hi$h are in$redi-&e #sef#&! and is a
feat#re (ost $o(p#tin &an#aes &a$,.
Asso$iati)e arrays $an (a,e a $o(p&e5
pro-&e( a tri)ia& e5er$ise.
7 +on't e5ha#sti)e&y $o)er A28. That is! 7
+i&& $o)er the essentia& parts! and a)oid
the (any )ariants of A28. 7t (iht -e too
$onf#sin to dis$#ss three different
)ersions of A28. 7 +on't $o)er the %9U
)ersion of A28 $a&&ed Aa+,.A 0i(i&ar&y! 7
+i&& not dis$#ss the ne+ AT>T A28 $a&&ed
Ana+,.A The ne+ A28 $o(es on the 0#n
syste(! and yo# (ay find it s#perior to
the o&d A28 in (any +ays. 7n parti$#&ar!
it has -etter dianosti$s! and +on't print
o#t the infa(o#s A-ai&in o#t near &ine ...A
(essae the oriina& A28 is prone to do.
7nstead! Ana+,A prints o#t the &ine it didn't
#nderstand! and hih&ihts the -ad parts
+ith arro+s. %A28 does this as +e&&! and
this rea&&y he&ps a &ot. 7f yo# find yo#rse&f
needin a feat#re that is )ery diffi$#&t or
i(possi-&e to do in A28! 7 s#est yo#
either #se 9A28! or %A28! or $on)ert
yo#r A28 s$ript into P'B1 #sin the Aa2pA
$on)ersion prora( +hi$h $o(es +ith
P'B1. P'B1 is a (ar)e&o#s &an#ae! and 7
#se it a&& the ti(e! -#t 7 do not p&an to
$o)er P'B1 in these t#toria&s. ?a)in
(ade (y intention $&ear! 7 $an $ontin#e
+ith a $&ear $ons$ien$e.
3 of 233 25-02-2014 12:21
Many U97:
#ti&ities
ha)e
strane
na(es.
A28 is one
of those
#ti&ities. 7t
is not an
a--re)iation
for
awk+ard.
7n fa$t! it
is an
e&eant
and si(p&e
&an#ae.
The +or,
AA28A is
deri)ed
fro( the
initia&s of
the
&an#ae's
three
de)e&opers:
A. Aho! ".
2.
8ernihan
and P.
2ein-erer.
Basic Structure
The
essentia&
oraniCation
of an A28
prora(
fo&&o+s the
for(:
pattern
D a$tion
E
The
pattern
spe$ifies
+hen the
a$tion is
perfor(ed.
1i,e (ost
U97:
#ti&ities!
A28 is &ine
oriented.
That is!
the pattern
spe$ifies a
test that is
perfor(ed
+ith ea$h
4 of 233 25-02-2014 12:21
&ine
read
as
inp#t.
7f
the
$ondition
is
tr#e!
then
the
a$tion
is
ta,en.
The
defa#&t
pattern
is
so(ethin
that
(at$hes
e)ery
&ine.
This
is
the
-&an,
or
n#&&
pattern.
T+o
other
i(portant
patterns
are
spe$ified
-y
the
,ey+ords
A"'%79A
and
A'9F.A
As
yo#
(iht
e5pe$t!
these
t+o
+ords
spe$ify
a$tions
to
-e
ta,en
-efore
any
&ines
are
read!
and
after
the
&ast
&ine
is
5 of 233 25-02-2014 12:21
read.
The
A28
prora(
-e&o+:
BEGIN { print "START" }
{ print }
END { print "STOP" }
adds
one
&ine
-efore
and
one
&ine
after
the
inp#t
fi&e.
This
isn't
)ery
#sef#&!
-#t
+ith
a
si(p&e
$hane!
+e
$an
(a,e
this
into
a
typi$a&
A28
prora(:
"'%79
D
print
A3i&eGt/+nerAE
D
print
HI!
AGtA!
H3E
'9F
D
print
A
;
F/9'
;A
E
7'&&
i(pro)e
the
s$ript
in
the
ne5t
6 of 233 25-02-2014 12:21
se$tions!
-#t
+e'&&
$a&&
it
A3i&e/+ner.A
"#t
&et's
not
p#t
it
into
a
s$ript
or
fi&e
yet.
7
+i&&
$o)er
that
part
in
a
-it.
?an
on
and
fo&&o+
+ith
(e
so
yo#
et
the
f&a)or
of
A28.
The
$hara$ters
AGtA
7ndi$ates
a
ta-
$hara$ter
so
the
o#tp#t
&ines
#p
on
e)en
-o#ndries.
The
AHIA
and
AH3A
ha)e
a
(eanin
si(i&ar
to
a
she&&
7 of 233 25-02-2014 12:21
s$ript.
7nstead
of
the
eihth
and
third
ar#(ent!
they
(ean
the
eihth
and
third
fie&d
of
the
inp#t
&ine.
*o#
$an
thin,
of
a
fie&d
as
a
$o&#(n!
and
the
a$tion
yo#
spe$ify
operates
on
ea$h
&ine
or
ro+
read
in.
There
are
t+o
differen$es
-et+een
A28
and
a
she&&
pro$essin
the
$hara$ters
+ithin
do#-&e
J#otes.
A28
#nderstands
spe$ia&
$hara$ters
fo&&o+
the
AGA
$hara$ter
8 of 233 25-02-2014 12:21
&i,e
AtA.
The
"o#rne
and
C
U97:
she&&s
do
not.
A&so!
#n&i,e
the
she&&
<and
P'B1=
A28
does
not
e)a&#ate
)aria-&es
+ithin
strins.
To
e5p&ain!
the
se$ond
&ine
$o#&d
not
-e
+ritten
&i,e
this:
Dprint
AHIGtH3A
E
That
e5a(p&e
+o#&d
print
AHI
H3.A
7nside
the
J#otes!
the
do&&ar
sin
is
not
a
spe$ia&
$hara$ter.
/#tside!
it
$orresponds
to
a
fie&d.
2hat
do
7
9 of 233 25-02-2014 12:21
(ean
-y
the
third
and
eiht
fie&d@
Consider
the
0o&aris
AK#srK-inK&s
;&A
$o((and!
+hi$h
has
eiht
$o&#(ns
of
infor(ation.
The
0yste(
L
)ersion
<0i(i&ar
to
the
1in#5
)ersion=!
AK#srK5-inK&s
;&!A
has
9
$o&#(ns.
The
third
$o&#(n
is
the
o+ner!
and
the
eihth
<or
nineth=
$o&#(n
in
the
na(e
of
the
fi&e.
This
A28
prora(
$an
-e
#sed
to
pro$ess
the
o#tp#t
of
the
A&s
;&A
10 of 233 25-02-2014 12:21
$o((and!
printin
o#t
the
fi&ena(e!
then
the
o+ner!
for
ea$h
fi&e.
7'&&
sho+
yo#
ho+.
Update:
/n
a
&in#5
syste(!
$hane
AHIA
to
AH9A.
/ne
(ore
point
a-o#t
the
#se
of
a
do&&ar
sin.
7n
s$riptin
&an#aes
&i,e
Per&
and
the
)ario#s
she&&s!
a
do&&ar
sin
(eans
the
+ord
fo&&o+in
is
the
na(e
of
the
)aria-&e.
A+,
is
different.
The
do&&ar
sin
(eans
11 of 233 25-02-2014 12:21
that
+e
are
referin
to
a
fie&d
or
$o&#(n
in
the
$#rrent
&ine.
2hen
s+it$hin
-et+een
Per&
and
A28
yo#
(#st
re(ener
that
AHA
has
a
different
(eanin.
0o
the
fo&&o+in
pie$e
of
$ode
prints
t+o
Afie&dsA
to
standard
o#t.
The
first
fie&d
printed
is
the
n#(-er
A5A!
the
se$ond
is
the
fifth
fie&d
<or
$o&#(n=
on
the
inp#t
&ine.
"'%79
D
5M5
E
12 of 233 25-02-2014 12:21
D
print
5!
H5E
Executing an AWK scrit
0o
&et's
start
+ritin
o#r
first
A28
s$ript.
There
are
a
$o#p&e
of
+ays
to
do
this.
Ass#(in
the
first
s$ript
is
$a&&ed
A3i&e/+ner!A
the
in)o$ation
+o#&d
-e
&s
;&
N
3i&e/+ner
This
(iht
enerate
the
fo&&o+in
if
there
+ere
on&y
t+o
fi&es
in
the
$#rrent
dire$tory:
3i&e
/+ner
a.fi&e
-arnett
another.fi&e
13 of 233 25-02-2014 12:21
-arnett
;
F/9'
;
There
are
t+o
pro-&e(s
+ith
this
s$ript.
"oth
pro-&e(s
are
easy
to
fi5!
-#t
7'&&
ho&d
off
on
this
#nti&
7
$o)er
the
-asi$s.
The
s$ript
itse&f
$an
-e
+ritten
in
(any
+ays.
The
C
she&&
)ersion
+o#&d
&oo,
&i,e
this:
#!/bin/csh -f
# Linux users have to change $8 to $9
awk '\
BEGIN { print "File\tOwner" } \
{ print $8, "\t", $3} \
END { print " - DONE -" } \
'
C&i$,
here
to
et
fi&e:
awk9exa"le:;csh
As
yo#
14 of 233 25-02-2014 12:21
$an
see
in
the
a-o)e
s$ript!
ea$h
&ine
of
the
A28
s$ript
(#st
ha)e
a
-a$,s&ash
if
it
is
not
the
&ast
&ine
of
the
s$ript.
This
is
ne$essary
as
the
C
she&&
doesn't!
-y
defa#&t!
a&&o+
strins
7
ha)e
a
&on
&ist
of
$o(p&aints
a-o#t
#sin
the
C
she&&.
0ee
To
Ten
reasons
not
to
use
the
C
shell
The
"o#rne
she&&
<as
does
15 of 233 25-02-2014 12:21
(ost
she&&s=
a&&o+s
J#oted
strins
to
span
se)era&
&ines:
OPK-inKsh
O
1in#5
#sers
ha)e
to
$hane
HI
to
H9
a+,
'
"'%79
D
print
A3i&eGt/+nerA
E
D
print
HI!
AGtA!
H3E
'9F
D
print
A
;
F/9'
;A
E
'
C&i$,
here
to
et
fi&e:
awk9exa"le:;sh
The
third
for(
is
to
store
the
$o((ands
in
a
fi&e!
and
e5e$#te
a+,
;f
fi&ena(e
16 of 233 25-02-2014 12:21
0in$e
A28
is
a&so
an
interpretor!
yo#
$an
sa)e
yo#rse&f
a
step
and
(a,e
the
fi&e
e5e$#ta-&e
-y
add
one
&ine
in
the
-einnin
of
the
fi&e:
OPK-inKa+,
;f
"'%79
D
print
A3i&eGt/+nerA
E
D
print
HI!
AGtA!
H3E
'9F
D
print
A
;
F/9'
;A
E
C&i$,
here
to
et
fi&e:
awk9exa"le:;awk
Chane
the
per(ission
+ith
the
chmod
$o((and!
17 of 233 25-02-2014 12:21
<i.e.
A$h(od
Q5
a+,Re5a(p&e1.a+,A=!
and
the
s$ript
-e$o(es
a
ne+
$o((and.
9oti$e
the
A;fA
option
fo&&o+in
'OPK-inKa+,
Aa-o)e!
+hi$h
is
a&so
#sed
in
the
third
for(at
+here
yo#
#se
A28
to
e5e$#te
the
fi&e
dire$t&y!
i.e.
Aa+,
;f
fi&ena(eA.
The
A;fA
option
spe$ifies
the
A28
fi&e
$ontainin
the
instr#$tions.
As
yo#
$an
see!
A28
$onsiders
&ines
that
start
+ith
a
AOA
to
-e
a
$o((ent!
18 of 233 25-02-2014 12:21
6#st
&i,e
the
she&&.
To
-e
pre$ise!
anythin
fro(
the
AOA
to
the
end
of
the
&ine
is
a
$o((ent
<#n&ess
its
inside
an
A28
strin.
?o+e)er!
7
a&+ays
$o((ent
(y
A28
s$ripts
+ith
the
AOA
at
the
start
of
the
&ine!
for
reasons
7'&&
dis$#ss
&ater.
2hi$h
for(at
sho#&d
yo#
#se@
7
prefer
the
&ast
for(at
+hen
possi-&e.
7t's
shorter
and
si(p&er.
7t's
19 of 233 25-02-2014 12:21
a&so
easier
to
de-#
pro-&e(s.
7f
yo#
need
to
#se
a
she&&!
and
+ant
to
a)oid
#sin
too
(any
fi&es!
yo#
$an
$o(-ine
the(
as
+e
did
in
the
first
and
se$ond
e5a(p&e.
Which shell to use with AWK?
The
for(at
of
A28
is
not
free;for(.
*o#
$annot
p#t
ne+
&ine
-rea,s
6#st
any+here.
They
(#st
o
in
parti$#&ar
&o$ations.
To
-e
pre$ise!
in
the
oriina&
A28
20 of 233 25-02-2014 12:21
yo#
$an
insert
a
ne+
&ine
$hara$ter
after
the
$#r&y
-ra$es!
and
at
the
end
of
a
$o((and!
-#t
not
e&se+here.
7f
yo#
+anted
to
-rea,
a
&on
&ine
into
t+o
&ines
at
any
other
p&a$e!
yo#
had
to
#se
a
-a$,s&ash:
OPK-inKa+,
;f
"'%79
D
print
A3i&eGt/+nerA
E
D
print
HI!
AGtA!
G
H3E
'9F
D
print
A
;
F/9'
;A
E
21 of 233 25-02-2014 12:21
C&i$,
here
to
et
fi&e:
awk9exa"le<;awk
The
"o#rne
she&&
)ersion
+o#&d
-e
OPK-inKsh
a+,
'
"'%79
D
print
A3i&eGt/+nerA
E
D
print
HI!
AGtA!
G
H3E
'9F
D
print
AdoneAE
'
C&i$,
here
to
et
fi&e:
awk9exa"le<;sh
+hi&e
the
C
she&&
+o#&d
-e
OPK-inK$sh
;f
a+,
'
"'%79
D
print
A3i&eGt/+nerA
EG
D
print
HI!
AGtA!
GG
H3EG
'9F
D
22 of 233 25-02-2014 12:21
print
AdoneAEG
'
C&i$,
here
to
et
fi&e:
awk9exa"le<;csh
As
yo#
$an
see!
this
de(onstrates
ho+
a+,+ard
the
C
she&&
is
+hen
en$&osin
an
A28
s$ript.
9ot
on&y
are
-a$,
s&ashes
needed
for
e)ery
&ine!
so(e
&ines
need
t+o.
<9ote
;
this
is
tr#e
+hen
#sin
o&d
a+,
<e..
on
0o&aris=
-e$a#se
the
print
state(ent
had
to
-e
on
one
&ine.
9e+er
A28's
23 of 233 25-02-2014 12:21
are
(ore
f&e5i-&e
+here
ne+&ines
$an
-e
added.=
Many
peop&e
+i&&
+arn
yo#
a-o#t
the
C
she&&.
0o(e
of
the
pro-&e(s
are
s#-t&e!
and
yo#
(ay
ne)er
see
the(.
Try
to
in$&#de
an
A28
or
sed
s$ript
+ithin
a
C
she&&
s$ript!
and
the
-a$,
s&ashes
+i&&
dri)e
yo#
$raCy.
This
is
+hat
$on)in$ed
(e
to
&earn
the
"o#rne
she&&
years
ao!
+hen
7
+as
24 of 233 25-02-2014 12:21
startin
o#t.
7
stron&y
re$o((end
yo#
#se
the
"o#rne
she&&
for
any
A28
or
sed
s$ript.
7f
yo#
don't
#se
the
"o#rne
she&&!
then
yo#
sho#&d
&earn
it.
As
a
(ini(#(!
&earn
ho+
to
set
)aria-&es!
+hi$h
-y
so(e
strane
$oin$iden$e
is
the
s#-6e$t
of
the
ne5t
se$tion.
!yna"ic #ariables
0in$e
yo#
$an
(a,e
a
s$ript
an
A28
e5e$#ta-&e
-y
(entionin
AOPK-inKa+,
;fA
25 of 233 25-02-2014 12:21
on
the
first
&ine!
in$&#din
an
A28
s$ript
inside
a
she&&
s$ript
isn't
needed
#n&ess
yo#
+ant
to
either
e&i(inate
the
need
for
an
e5tra
fi&e!
or
if
yo#
+ant
to
pass
a
)aria-&e
to
the
insides
of
an
A28
s$ript.
0in$e
this
is
a
$o((on
pro-&e(!
no+
is
as
ood
a
ti(e
to
e5p&ain
the
te$hniJ#e.
7'&&
do
this
-y
sho+in
a
si(p&e
A28
26 of 233 25-02-2014 12:21
prora(
that
+i&&
on&y
print
one
$o&#(n.
(%TE=
there
will
be
a
bug
in
the
first
-ersion;
The
n#(-er
of
the
$o&#(n
+i&&
-e
spe$ified
-y
the
first
ar#(ent.
The
first
)ersion
of
the
prora(!
+hi$h
+e
+i&&
$a&&
ACo&#(n!A
&oo,s
&i,e
this:
OPK-inKsh
O9/T'
;
this
s$ript
does
not
+or,P
$o&#(nMH1
a+,
'Dprint
H$o&#(nE'
C&i$,
here
to
et
fi&e
<-#t
-e
a+are
27 of 233 25-02-2014 12:21
that
it
doesn't
+or,=:
Colu"n:;sh
A
s#ested
#se
is:
&s
;&
N
Co&#(n
3
This
+o#&d
print
the
third
$o&#(n
fro(
the
ls
$o((and!
+hi$h
+o#&d
-e
the
o+ner
of
the
fi&e.
*o#
$an
$hane
this
into
a
#ti&ity
that
$o#nts
ho+
(any
fi&es
are
o+ned
-y
ea$h
#ser
-y
addin
&s
;&
N
Co&#(n
3
N
#niJ
;$
N
sort
;nr
28 of 233 25-02-2014 12:21
%nly
one
roble"=
the
scrit
$oesn.t
work;
The
)a&#e
of
the
A$o&#(nA
)aria-&e
is
not
seen
-y
A28.
Chane
Aa+,A
to
Ae$hoA
to
$he$,.
*o#
need
to
t#rn
off
the
J#otin
+hen
the
)aria-&e
is
seen.
This
$an
-e
done
-y
endin
the
J#otin!
and
restartin
it
after
the
)aria-&e:
OPK-inKsh
$o&#(nMH1
a+,
'Dprint
H'H$o&#(n'E'
C&i$,
here
to
et
fi&e:
Colu"n<;sh
29 of 233 25-02-2014 12:21
This
is
a
)ery
i(portant
$on$ept!
and
thro+s
e5perien$ed
prora((ers
a
$#r)e
-a&&.
7n
(any
$o(p#ter
&an#aes!
a
strin
has
a
start
J#ote!
and
end
J#ote!
and
the
$ontents
in
-et+een.
7f
yo#
+ant
to
in$&#de
a
spe$ia&
$hara$ter
inside
the
J#ote!
yo#
(#st
pre)ent
the
$hara$ter
fro(
ha)in
the
typi$a&
(eanin.
7n
the
C
&an#ae!
this
is
do+n
-y
p#ttin
a
-a$,s&ash
-efore
30 of 233 25-02-2014 12:21
the
$hara$ter.
7n
other
&an#aes!
there
is
a
spe$ia&
$o(-ination
of
$hara$ters
to
to
this.
7n
the
C
and
"o#rne
she&&!
the
J#ote
is
6#st
a
s+it$h.
7t
t#rns
the
interpretation
(ode
on
or
off.
There
is
rea&&y
no
s#$h
$on$ept
as
Astart
of
strinA
and
Aend
of
strin.A
The
J#otes
to&e
a
s+it$h
inside
the
interpretor.
The
J#ote
$hara$ter
is
not
passed
on
to
31 of 233 25-02-2014 12:21
the
app&i$ation.
This
is
+hy
there
are
t+o
pairs
of
J#otes
a-o)e.
9oti$e
there
are
t+o
do&&ar
sins.
The
first
one
is
J#oted!
and
is
seen
-y
A28.
The
se$ond
one
is
not
J#oted!
so
the
she&&
e)a&#ates
the
)aria-&e!
and
rep&a$es
AH$o&#(nA
-y
the
)a&#e.
7f
yo#
don't
#nderstand!
either
$hane
Aa+,A
to
Ae$ho!A
or
$hane
the
first
&ine
to
read
AOPK-inKsh
;5.A
32 of 233 25-02-2014 12:21
0o(e
i(pro)e(ents
are
needed!
ho+e)er.
The
"o#rne
she&&
has
a
(e$hanis(
to
pro)ide
a
)a&#e
for
a
)aria-&e
if
the
)a&#e
isn't
set!
or
is
set
and
the
)a&#e
is
an
e(pty
strin.
This
is
done
-y
#sin
the
for(at:
HDvariable:;defaultvalueE
This
is
sho+n
-e&o+!
+here
the
defa#&t
$o&#(n
+i&&
-e
one:
OPK-inKsh
$o&#(nMHD1:;1E
a+,
'Dprint
H'H$o&#(n'E'
C&i$,
here
to
33 of 233 25-02-2014 12:21
et
fi&e:
Colu"n>;sh
2e
$an
sa)e
a
&ine
-y
$o(-inin
these
t+o
steps:
OPK-inKsh
a+,
'Dprint
H'HD1:;1E'E'
C&i$,
here
to
et
fi&e:
Colu"n?;sh
7t
is
hard
to
read!
-#t
it
is
$o(pa$t.
There
is
one
other
(ethod
that
$an
-e
#sed.
7f
yo#
e5e$#te
an
A28
$o((and
and
in$&#de
on
the
$o((and
&ine
variableMvalue
this
)aria-&e
+i&&
-e
set
+hen
34 of 233 25-02-2014 12:21
the
A28
s$ript
starts.
An
e5a(p&e
of
this
#se
+o#&d
-e:
OPK-inKsh
a+,
'Dprint
H$E'
$MHD1:;1E
C&i$,
here
to
et
fi&e:
Colu"n@;sh
This
&ast
)ariation
does
not
ha)e
the
pro-&e(s
+ith
J#otin
the
pre)io#s
e5a(p&e
had.
*o#
sho#&d
(aster
the
ear&ier
e5a(p&e!
ho+e)er!
-e$a#se
yo#
$an
#se
it
+ith
any
s$ript
or
$o((and.
The
se$ond
(ethod
is
spe$ia&
to
A28.
Modern
A28's
35 of 233 25-02-2014 12:21
ha)e
other
options
as
+e&&.
0ee
the
co";unix;shell
*AA.
The Essential Syntax of AWK
'ar&ier
7
dis$#ssed
+ays
to
start
an
A28
s$ript.
This
se$tion
+i&&
dis$#ss
the
)ario#s
ra((ati$a&
e&e(ents
of
A28.
Arith"etic Exressions
There
are
se)era&
arith(eti$
operators!
si(i&ar
to
C.
These
are
the
-inary
operators!
+hi$h
operate
on
t+o
)aria-&es:
AWK Table :
Binary %erators
%erator Tye Meaning
Q Arith(eti$ Addition
; Arith(eti$ 0#-tra$tion
S Arith(eti$ M#&tip&i$ation
K Arith(eti$ Fi)ision
36 of 233 25-02-2014 12:21
T Arith(eti$ Mod#&o
Uspa$eV 0trin Con$atenation
Usin
)aria-&es
+ith
the
)a&#e
of
A4A
and
A3!A
A28
ret#rns
the
fo&&o+in
res#&ts
for
ea$h
operator
+hen
#sin
the
print
$o((and:
Exression &esult
4Q3 10
4;3 4
4S3 21
4K3 2.33333
4T3 1
4 3 43
There
are
a
fe+
points
to
(a,e.
The
(od#&#s
operator
finds
the
re(ainder
after
an
inteer
di)ide.
The
print
$o((and
o#tp#t
a
f&oatin
point
n#(-er
on
the
di)ide!
37 of 233 25-02-2014 12:21
-#t
an
inteer
for
the
rest.
The
strin
$on$atenate
operator
is
$onf#sin!
sin$e
it
isn't
e)en
)isi-&e.
P&a$e
a
spa$e
-et+een
t+o
)aria-&es
and
the
strins
are
$on$atenated
toether.
This
a&so
sho+s
that
n#(-ers
are
$on)erted
a#to(ati$a&&y
into
strins
+hen
needed.
Un&i,e
C!
A28
doesn't
ha)e
AtypesA
of
)aria-&es.
There
is
one
type
on&y!
and
it
$an
-e
a
strin
or
n#(-er.
The
$on)ersion
r#&es
38 of 233 25-02-2014 12:21
are
si(p&e.
A
n#(-er
$an
easi&y
-e
$on)erted
into
a
strin.
2hen
a
strin
is
$on)erted
into
a
n#(-er!
A28
+i&&
do
so.
The
strin
A123A
+i&&
-e
$on)erted
into
the
n#(-er
123.
?o+e)er!
the
strin
A123:A
+i&&
-e
$on)erted
into
the
n#(-er
0.
<9A28
+i&&
-eha)e
different&y!
and
$on)erts
the
strin
into
inteer
123!
+hi$h
is
fo#nd
in
the
-einnin
of
the
strin=.
39 of 233 25-02-2014 12:21
Unary arith"etic oerators
The
AQA
and
A;A
operators
$an
-e
#sed
-efore
)aria-&es
and
n#(-ers.
7f
:
eJ#a&s
4!
then
the
state(ent:
print
;5W
+i&&
print
A;4.A
The Autoincre"ent an$ Auto$ecre"ent %era
A28
a&so
s#pports
the
AQQA
and
A;;A
operators
of
C.
"oth
in$re(ent
or
de$re(ent
the
)aria-&es
-y
one.
The
operator
$an
on&y
-e
#sed
+ith
a
sin&e
)aria-&e!
and
$an
-e
40 of 233 25-02-2014 12:21
-efore
or
after
the
)aria-&e.
The
prefi5
for(
(odifies
the
)a&#e!
and
then
#ses
the
res#&t!
+hi&e
the
postfi5
for(
ets
the
res#&ts
of
the
)aria-&e!
and
after+ards
(odifies
the
)aria-&e.
As
an
e5a(p&e!
if
:
has
the
)a&#e
of
3!
then
the
A28
state(ent
print
5QQ!
A
A!
QQ5W
+o#&d
print
the
n#(-ers
3
and
5.
These
operators
are
a&so
assin(ent
operators!
41 of 233 25-02-2014 12:21
and
$an
-e
#sed
-y
the(se&)es
on
a
&ine:
5QQW
;;yW
Assign"ent %erators
Laria-&es
$an
-e
assined
ne+
)a&#es
+ith
the
assin(ent
operators.
*o#
,no+
a-o#t
AQQA
and
A;;.A
The
other
assin(ent
state(ent
is
si(p&y:
variable
M
arithmetic_expression
Certain
operators
ha)e
pre$eden$e
o)er
othersW
parenthesis
$an
-e
#sed
to
$ontro&
ro#pin.
The
state(ent
5M1Q2S3
4W
is
the
sa(e
as
42 of 233 25-02-2014 12:21
5
M
<1
Q
<2
S
3==
A4AW
"oth
print
o#t
A44.A
9oti$e
spa$es
$an
-e
added
for
reada-i&ity.
A28!
&i,e
C!
has
spe$ia&
assin(ent
operators!
+hi$h
$o(-ine
a
$a&$#&ation
+ith
an
assin(ent.
7nstead
of
sayin
5M5Q2W
yo#
$an
(ore
$on$ise&y
say:
5QM2W
The
$o(p&ete
&ist
fo&&o+s:
AWK Table <
Assign"ent
%erators
%erator Meaning
QM
Add
res#&t to
)aria-&e
;M
0#-tra$t
res#&t
fro(
)aria-&e
43 of 233 25-02-2014 12:21
SM
M#&tip&y
)aria-&e
-y res#&t
KM
Fi)ide
)aria-&e
-y res#&t
TM
App&y
(od#&o
to
)aria-&e
Con$itional exressions
The
se$ond
type
of
e5pression
in
A28
is
the
$onditiona&
e5pression.
This
is
#sed
for
$ertain
tests!
&i,e
the
if
or
while.
"oo&ean
$onditions
e)a&#ate
to
tr#e
or
fa&se.
7n
A28!
there
is
a
definite
differen$e
-et+een
a
-oo&ean
$ondition!
and
an
arith(eti$
e5pression.
*o#
$annot
$on)ert
a
-oo&ean
$ondition
44 of 233 25-02-2014 12:21
to
an
inteer
or
strin.
*o#
$an!
ho+e)er!
#se
an
arith(eti$
e5pression
as
a
$onditiona&
e5pression.
A
)a&#e
of
0
is
fa&se!
+hi&e
anythin
e&se
is
tr#e.
Undefined
)aria-&es
has
the
)a&#e
of
0.
Un&i,e
A28!
9A28
&ets
yo#
#se
-oo&eans
as
inteers.
Arith(eti$
)a&#es
$an
a&so
-e
$on)erted
into
-oo&ean
$onditions
-y
#sin
re&ationa&
operators:
AWK Table >
&elational
%erators
%erator Meaning
MM 7s eJ#a&
45 of 233 25-02-2014 12:21
PM
7s not
eJ#a& to
V
7s
reater
than
VM
7s
reater
than or
eJ#a& to
U
7s &ess
than
UM
7s &ess
than or
eJ#a& to
These
operators
are
the
sa(e
as
the
C
operators.
They
$an
-e
#sed
to
$o(pare
n#(-ers
or
strins.
2ith
respe$t
to
strins!
&o+er
$ase
&etters
are
reater
than
#pper
$ase
&etters.
&egular Exressions
T+o
operators
are
#sed
to
$o(pare
strins
to
re#&ar
e5pressions:
AWK Table ?
&egular
46 of 233 25-02-2014 12:21
Exression
%erators
%erator Meaning
X Mat$hes
PX
Foesn't
(at$h
The
order
in
this
$ase
is
parti$#&ar.
The
re#&ar
e5pression
(#st
-e
en$&osed
-y
s&ashes!
and
$o(es
after
the
operator.
A28
s#pports
e5tended
re#&ar
e5pressions!
so
the
fo&&o+in
are
e5a(p&es
of
)a&id
tests:
+ord
PX
K0TABTK
&a+ren$eR+e&,
X
K<oneNt+oNthree=K
An$'%r'(ot
There
are
t+o
-oo&ean
operators
that
$an
-e
#sed
+ith
$onditiona&
e5pressions.
That
47 of 233 25-02-2014 12:21
is!
yo#
$an
$o(-ine
t+o
$onditiona&
e5pressions
+ith
the
AorA
or
AandA
operators:
A>>A
and
ANN.A
There
is
a&so
the
#nary
not
operator:
AP.A
Co""an$s
There
are
on&y
a
fe+
$o((ands
in
A28.
The
&ist
and
synta5
fo&&o+s:
if
<
conditional
=
statement
Y
e&se
statement
Z
+hi&e
<
conditional
=
statement
for
<
expression
W
conditional
W
expression
=
statement
48 of 233 25-02-2014 12:21
for
<
variable
in
array
=
statement
-rea,
$ontin#e
D
Y
statement
Z
...E
variableMexpression
print
Y
expression-list
Z
Y
V
expression
Z
printf
format
Y
!
expression-list
Z
Y
V
expression
Z
ne5t
e5it
At
this
point!
yo#
$an
#se
A28
as
a
&an#ae
for
si(p&e
$a&$#&ationsW
7f
yo#
+anted
to
$a&$#&ate
so(ethin!
and
not
read
any
&ines
for
inp#t!
yo#
$o#&d
#se
49 of 233 25-02-2014 12:21
the
BEGIN
,ey+ord
dis$#ssed
ear&ier!
$o(-ined
+ith
a
exit
$o((and:
#!/bin/awk -f
BEGIN {
# Print the squares from 1 to 10 the first way
i=1;
while (i <= 10) {
printf "The square of ", i, " is ", i*i;
i = i+1;
}
# do it again, using more concise code
for (i=1; i <= 10; i++) {
printf "The square of ", i, " is ", i*i;
}
# now end
exit;
}
C&i$,
here
to
et
fi&e:
awk9rint9s0uares;awk
The
fo&&o+in
as,s
for
a
n#(-er!
and
then
sJ#ares
it:
#!/bin/awk -f
BEGIN {
print "type a number";
}
{
print "The square of ", $1, " is ", $1*$1;
print "type another number";
}
END {
print "Done"
}
C&i$,
here
to
et
50 of 233 25-02-2014 12:21
fi&e:
awk9ask9for9s0uare;awk
The
a-o)e
isn't
a
ood
fi&ter!
-e$a#se
it
as,s
for
inp#t
ea$h
ti(e.
7f
yo#
pipe
the
o#tp#t
of
another
prora(
into
it!
yo#
+o#&d
enerate
a
&ot
of
(eanin&ess
pro(pts.
?ere
is
a
fi&ter
that
yo#
sho#&d
find
#sef#&.
7t
$o#nts
&ines!
tota&s
#p
the
n#(-ers
in
the
first
$o&#(n!
and
$a&$#&ates
the
a)erae.
Pipe
A+$
;$
SA
into
it!
51 of 233 25-02-2014 12:21
and
it
+i&&
$o#nt
fi&es!
and
te&&
yo#
the
a)erae
n#(-er
of
+ords
per
fi&e!
as
+e&&
as
the
tota&
+ords
and
the
n#(-er
of
fi&es.
#!/bin/awk -f
BEGIN {
# How many lines
lines=0;
total=0;
}
{
# this code is executed once for each line
# increase the number of files
lines++;
# increase the total size, which is field #1
total+=$1;
}
END {
# end, now output the total
print lines " lines read";
print "total is ", total;
if (lines > 0 ) {
print "average is ", total/lines;
} else {
print "average is 0";
}
}
C&i$,
here
to
et
fi&e:
a-erage;awk
*o#
$an
pipe
the
o#tp#t
of
A&s
52 of 233 25-02-2014 12:21
;sA
into
this
fi&ter
to
$o#nt
the
n#(-er
of
fi&es!
the
tota&
siCe!
and
the
a)erae
siCe.
There
is
a
s&iht
pro-&e(
+ith
this
s$ript!
as
it
in$&#des
the
o#tp#t
of
A&sA
that
reports
the
tota&.
This
$a#ses
the
n#(-er
of
fi&es
to
-e
off
-y
one.
Chanin
&inesQQW
to
if
<H1
PM
Atota&A
=
&inesQQW
+i&&
fi5
this
pro-&e(.
9ote
the
53 of 233 25-02-2014 12:21
$ode
+hi$h
pre)ents
a
di)ide
-y
Cero.
This
is
$o((on
in
+e&&;+ritten
s$ripts.
7
a&so
initia&iCe
the
)aria-&es
to
Cero.
This
is
not
ne$essary!
-#t
it
is
a
ood
ha-it.
AWK Built)in #ariables
7
ha)e
(entioned
t+o
,inds
of
)aria-&es:
positiona&
and
#ser
defined.
A
#ser
defined
)aria-&e
is
one
yo#
$reate.
A
positiona&
)aria-&e
is
not
a
spe$ia&
)aria-&e!
-#t
a
f#n$tion
triered
54 of 233 25-02-2014 12:21
-y
the
do&&ar
sin.
Therefore
print
H1W
and
:M1W
print
H:W
do
the
sa(e
thin:
print
the
first
fie&d
on
the
&ine.
There
are
t+o
(ore
points
a-o#t
positiona&
)aria-&es
that
are
)ery
#sef#&.
The
)aria-&e
AH0A
refers
to
the
entire
&ine
that
A28
reads
in.
That
is!
if
yo#
had
eiht
fie&ds
in
a
&ine!
print
H0W
is
si(i&ar
55 of 233 25-02-2014 12:21
to
print
H1!
H2!
H3!
H4!
H5!
H6!
H4!
HI
This
+i&&
$hane
the
spa$in
-et+een
the
fie&dsW
other+ise!
they
-eha)e
the
sa(e.
*o#
$an
(odify
positiona&
)aria-&es.
The
fo&&o+in
$o((ands
H2MAAW
printW
de&etes
the
se$ond
fie&d.
7f
yo#
had
fo#r
fie&ds!
and
+anted
to
print
o#t
the
se$ond
and
fo#rth
fie&d!
there
are
t+o
+ays.
This
is
the
first:
#!/bin/awk -f
56 of 233 25-02-2014 12:21
{
$1="";
$3="";
print;
}
and
the
se$ond
#!/bin/awk -f
{
print $2, $4;
}
These
perfor(
si(i&ar&y!
-#t
not
identi$a&&y.
The
n#(-er
of
spa$es
-et+een
the
)a&#es
)ary.
There
are
t+o
reasons
for
this.
The
a$t#a&
n#(-er
of
fie&ds
does
not
$hane.
0ettin
a
positiona&
)aria-&e
to
an
e(pty
strin
does
not
de&ete
the
)aria-&e.
7t's
sti&&
there!
-#t
the
$ontents
has
-een
de&eted.
57 of 233 25-02-2014 12:21
The
other
reason
is
the
+ay
A28
o#tp#ts
the
entire
&ine.
There
is
a
fie&d
separator
that
spe$ifies
+hat
$hara$ter
to
p#t
-et+een
the
fie&ds
on
o#tp#t.
The
first
e5a(p&e
o#tp#ts
fo#r
fie&ds!
+hi&e
the
se$ond
o#tp#ts
t+o.
7n;-et+een
ea$h
fie&d
is
a
spa$e.
This
is
easier
to
e5p&ain
if
the
$hara$ters
-et+een
fie&ds
$o#&d
-e
(odified
to
-e
(ade
(ore
)isi-&e.
2e&&!
it
$an.
58 of 233 25-02-2014 12:21
A28
pro)ides
spe$ia&
)aria-&es
for
6#st
that
p#rpose.
*S ) The +nut *iel$ Searator #ariable
A28
$an
-e
#sed
to
parse
(any
syste(
ad(inistration
fi&es.
?o+e)er!
(any
of
these
fi&es
do
not
ha)e
+hitespa$e
as
a
separator.
as
an
e5a(p&e!
the
pass+ord
fi&e
#ses
$o&ons.
*o#
$an
easi&y
$hane
the
fie&d
separator
$hara$ter
to
-e
a
$o&on
#sin
the
A;3A
$o((and
&ine
option.
The
fo&&o+in
$o((and
+i&&
print
59 of 233 25-02-2014 12:21
o#t
a$$o#nts
that
don't
ha)e
pass+ords:
a+,
;3:
'Dif
<H2
MM
AA=
print
H1
A:
no
pass+ordPAE'
UKet$Kpass+d
There
is
a
+ay
to
do
this
+itho#t
the
$o((and
&ine
option.
The
)aria-&e
A30A
$an
-e
set
&i,e
any
)aria-&e!
and
has
the
sa(e
f#n$tion
as
the
A;3A
$o((and
&ine
option.
The
fo&&o+in
is
a
s$ript
that
has
the
sa(e
f#n$tion
as
the
one
60 of 233 25-02-2014 12:21
a-o)e.
#!/bin/awk -f
BEGIN {
FS=":";
}
{
if ( $2 == "" ) {
print $1 ": no password!";
}
}
C&i$,
here
to
et
fi&e:
awk9noassw$;awk
The
se$ond
for(
$an
-e
#sed
to
$reate
a
U97:
#ti&ity!
+hi$h
7
+i&&
na(e
A$h,pass+d!A
and
e5e$#ted
&i,e
this:
$h,pass+d
UKet$Kpass+d
The
$o((and
A$h,pass+d
;3:A
$annot
-e
#sed!
-e$a#se
A28
+i&&
ne)er
see
this
ar#(ent.
A&&
interpreter
s$ripts
a$$ept
one
and
on&y
one
61 of 233 25-02-2014 12:21
ar#(ent!
+hi$h
is
i((ediate&y
after
the
AOPK-inKa+,A
strin.
7n
this
$ase!
the
sin&e
ar#(ent
is
A;f.A
Another
differen$e
-et+een
the
$o((and
&ine
option
and
the
interna&
)aria-&e
is
the
a-i&ity
to
set
the
inp#t
fie&d
separator
to
-e
(ore
than
one
$hara$ter.
7f
yo#
spe$ify
30MA:
AW
then
A28
+i&&
sp&it
a
&ine
into
fie&ds
+here)er
it
sees
those
t+o
$hara$ters!
in
that
62 of 233 25-02-2014 12:21
e5a$t
order.
*o#
$annot
do
this
on
the
$o((and
&ine.
There
is
a
third
ad)antae
the
interna&
)aria-&e
has
o)er
the
$o((and
&ine
option:
yo#
$an
$hane
the
fie&d
separator
$hara$ter
as
(any
ti(es
as
yo#
+ant
+hi&e
readin
a
fi&e.
2e&&!
at
(ost
on$e
for
ea$h
&ine.
*o#
$an
e)en
$hane
it
dependin
on
the
&ine
yo#
read.
0#ppose
yo#
had
the
fo&&o+in
63 of 233 25-02-2014 12:21
fi&e
+hi$h
$ontains
the
n#(-ers
1
thro#h
4
in
three
different
for(ats.
1ines
4
thro#h
6
ha)e
$o&on
separated
fie&ds!
+hi&e
the
others
separated
-y
spa$es.
/9'
1
7
T2/
2
77
O0TABT
T?B'':3:777
3/UB:4:7L
37L':5:L
O0T/P
07:
6
L7
0'L'9
4
L77
The
A28
prora(
$an
easi&y
s+it$h
-et+een
these
for(ats:
#!/bin/awk -f
{
if ($1 == "#START") {
FS=":";
} else if ($1 == "#STOP") {
FS=" ";
} else {
#print the Roman number in column 3
print $3
}
}
64 of 233 25-02-2014 12:21
C&i$,
here
to
et
fi&e:
awk9exa"le>;awk
9ote
the
fie&d
separator
)aria-&e
retains
its
)a&#e
#nti&
it
is
e5p&i$it&y
$haned.
*o#
don't
ha)e
to
reset
it
for
ea$h
&ine.
0o#nds
si(p&e!
riht@
?o+e)er!
7
ha)e
a
tri$,
J#estion
for
yo#.
2hat
happens
if
yo#
$hane
the
fie&d
separator
+hi&e
readin
a
&ine@
That
is!
s#ppose
yo#
had
the
fo&&o+in
&ine
/ne
65 of 233 25-02-2014 12:21
T+o:Three:4
3i)e
and
yo#
e5e$#ted
the
fo&&o+in
s$ript:
#!/bin/awk -f
{
print $2
FS=":"
print $2
}
2hat
+o#&d
-e
printed@
AThreeA
or
AT+o:Three:4@A
2e&&!
the
s$ript
+o#&d
print
o#t
AT+o:Three:4A
t+i$e.
?o+e)er!
if
yo#
de&eted
the
first
print
state(ent!
it
+o#&d
print
o#t
AThreeA
on$eP
7
tho#ht
this
+as
)ery
strane
at
first!
-#t
after
p#&&in
o#t
so(e
hair!
,i$,in
the
de$,!
and
ye&&in
at
66 of 233 25-02-2014 12:21
(#se&f
and
e)eryone
+ho
had
anythin
to
do
+ith
the
de)e&op(ent
of
U97:!
it
is
int#iti)e&y
o-)io#s.
*o#
6#st
ha)e
to
-e
thin,in
&i,e
a
professiona&
prora((er
to
rea&iCe
it
is
int#iti)e.
7
sha&&
e5p&ain!
and
pre)ent
yo#
fro(
$a#sin
yo#rse&f
physi$a&
har(.
7f
yo#
$hane
the
fie&d
separator
before
yo#
read
the
&ine!
the
$hane
affects
+hat
yo#
read.
7f
yo#
$hane
it
67 of 233 25-02-2014 12:21
after
yo#
read
the
&ine!
it
+i&&
not
redefine
the
)aria-&es.
*o#
+o#&dn't
+ant
a
)aria-&e
to
$hane
on
yo#
as
a
side;effe$t
of
another
a$tion.
A
prora((in
&an#ae
+ith
hidden
side
effe$ts
is
-ro,en!
and
sho#&d
not
-e
tr#sted.
A28
a&&o+s
yo#
to
redefine
the
fie&d
separator
either
-efore
or
after
yo#
read
the
&ine!
and
does
the
riht
thin
ea$h
ti(e.
/n$e
yo#
68 of 233 25-02-2014 12:21
read
the
)aria-&e!
the
)aria-&e
+i&&
not
$hane
#n&ess
yo#
$hane
it.
"ra)oP
To
i&&#strate
this
f#rther!
here
is
another
)ersion
of
the
pre)io#s
$ode
that
$hanes
the
fie&d
separator
dyna(i$a&&y.
7n
this
$ase!
A28
does
it
-y
e5a(inin
fie&d
AH0!A
+hi$h
is
the
entire
&ine.
2hen
the
&ine
$ontains
a
$o&on!
the
fie&d
separator
is
a
$o&on!
other+ise!
it
is
a
spa$e.
?ere
69 of 233 25-02-2014 12:21
is
a
)ersion
that
+or,ed
+ith
o&der
)ersions
of
a+,:
#!/bin/awk -f
{
if ( $0 ~ /:/ ) {
FS=":";
} else {
FS=" ";
}
#print the third field, whatever format
print $3
}
C&i$,
here
to
et
fi&e:
awk9exa"le?;awk
?o+e)er!
this
-eha)ior
$haned
in
&ater
)ersions!
so
the
a-o)e
s$ript
no
&oner
+or,s.
2hat
happens
is
that
on$e
the
30
)aria-&e
is
$haned!
yo#
ha)e
to
re;e)a&#ate
the
fie&ds
-y
#sin
$!$:
#!/bin/awk -f
{
70 of 233 25-02-2014 12:21
if ( $0 ~ /:/ ) {
FS=":";
$0=$0
} else {
FS=" ";
$0=$0
}
#print the third field, whatever format
print $3
}
C&i$,
here
to
et
fi&e:
awk9exa"le?a;awk
This
e5a(p&e
e&i(inates
the
need
to
ha)e
the
spe$ia&
AO0TABTA
and
AO0T/PA
&ines
in
the
inp#t.
%*S ) The %utut *iel$ Searator #ariable
There
is
an
i(portant
differen$e
-et+een
print
H2
H3
and
print
H2!
H3
The
first
e5a(p&e
prints
o#t
one
fie&d!
and
the
se$ond
71 of 233 25-02-2014 12:21
prints
o#t
t+o
fie&ds.
7n
the
first
$ase!
the
t+o
positiona&
para(eters
are
$on$atenated
toether
and
o#tp#t
+itho#t
a
spa$e.
7n
the
se$ond
$ase!
A28
prints
t+o
fie&ds!
and
p&a$es
the
o#tp#t
fie&d
separator
-et+een
the(.
9or(a&&y
this
is
a
spa$e!
-#t
yo#
$an
$hane
this
-y
(odifyin
the
)aria-&e
A/30.A
7f
yo#
+anted
to
$opy
the
pass+ord
fi&e!
-#t
de&ete
the
en$rypted
pass+ord!
72 of 233 25-02-2014 12:21
yo#
$o#&d
#se
A28:
#!/bin/awk -f
BEGIN {
FS=":";
OFS=":";
}
{
$2="";
print
}
C&i$,
here
to
et
fi&e:
$elete9assw$;awk
%i)e
this
s$ript
the
pass+ord
fi&e!
and
it
+i&&
de&ete
the
pass+ord!
-#t
&ea)e
e)erythin
e&se
the
sa(e.
*o#
$an
(a,e
the
o#tp#t
fie&d
separator
any
n#(-er
of
$hara$ters.
*o#
are
not
&i(ited
to
a
sin&e
$hara$ter.
(* ) The (u"ber of *iel$s #ariable
7t
73 of 233 25-02-2014 12:21
is
#sef#&
to
,no+
ho+
(any
fie&ds
are
on
a
&ine.
*o#
(ay
+ant
to
ha)e
yo#r
s$ript
$hane
its
operation
-ased
on
the
n#(-er
of
fie&ds.
As
an
e5a(p&e!
the
$o((and
A&s
;&A
(ay
enerate
eiht
or
nine
fie&ds!
dependin
on
+hi$h
)ersion
yo#
are
e5e$#tin.
The
0yste(
L
)ersion!
AK#srK-inK&s
;&A
enerates
nine
fie&ds!
+hi$h
is
eJ#i)a&ent
to
the
"er,e&ey
AK#srK#$-K&s
;&A
$o((and.
74 of 233 25-02-2014 12:21
7f
yo#
+anted
to
print
the
o+ner
and
fi&ena(e
then
the
fo&&o+in
A28
s$ript
+o#&d
+or,
+ith
either
)ersion
of
A&s:A
#!/bin/awk -f
# parse the output of "ls -l"
# print owner and filename
# remember - Berkeley ls -l has 8 fields, System V has 9
{
if (NF == 8) {
print $3, $8;
} else if (NF == 9) {
print $3, $9;
}
}
C&i$,
here
to
et
fi&e:
owner9grou;awk
Fon't
foret
the
)aria-&e
$an
-e
prepended
+ith
a
AH.A
This
a&&o+s
yo#
to
print
the
&ast
fie&d
of
any
$o&#(n
OPK-inKa+,
75 of 233 25-02-2014 12:21
;f
D
print
H93W
E
C&i$,
here
to
et
fi&e:
rint9last9fiel$;awk
/ne
+arnin
a-o#t
A28.
There
is
a
&i(it
of
99
fie&ds
in
a
sin&e
&ine.
P'B1
does
not
ha)e
any
s#$h
&i(itations.
(& ) The (u"ber of &ecor$s #ariable
Another
#sef#&
)aria-&e
is
A9B.A
This
te&&s
yo#
the
n#(-er
of
re$ords!
or
the
&ine
n#(-er.
*o#
$an
#se
A28
to
on&y
e5a(ine
$ertain
&ines.
This
76 of 233 25-02-2014 12:21
e5a(p&e
prints
&ines
after
the
first
100
&ines!
and
p#ts
a
&ine
n#(-er
-efore
ea$h
&ine
after
100:
#!/bin/awk -f
{ if (NR > 100) {
print NR, $0;
}
C&i$,
here
to
et
fi&e:
awk9exa"le@;awk
&S ) The &ecor$ Searator #ariable
9or(a&&y!
A28
reads
one
&ine
at
a
ti(e!
and
-rea,s
#p
the
&ine
into
fie&ds.
*o#
$an
set
the
AB0A
)aria-&e
to
$hane
A28's
definition
of
a
A&ine.A
7f
77 of 233 25-02-2014 12:21
yo#
set
it
to
an
e(pty
strin!
then
A28
+i&&
read
the
entire
fi&e
into
(e(ory.
*o#
$an
$o(-ine
this
+ith
$hanin
the
A30A
)aria-&e.
This
e5a(p&e
treats
ea$h
&ine
as
a
fie&d!
and
prints
o#t
the
se$ond
and
third
&ine:
#!/bin/awk -f
BEGIN {
# change the record separator from newline to nothing
RS=""
# change the field separator from whitespace to newline
FS="\n"
}
{
# print the second and third line of the file
print $2, $3;
}
C&i$,
here
to
et
fi&e:
awk9exa"leB;awk
The
t+o
&ines
are
78 of 233 25-02-2014 12:21
printed
+ith
a
spa$e
-et+een.
A&so
this
+i&&
on&y
+or,
if
the
inp#t
fi&e
is
&ess
than
100
&ines!
therefore
this
te$hniJ#e
is
&i(ited.
*o#
$an
#se
it
to
-rea,
+ords
#p!
one
+ord
per
&ine!
#sin
this:
#!/bin/awk -f
BEGIN {
RS=" ";
}
{
print ;
}
C&i$,
here
to
et
fi&e:
onewor$9er9line;awk
-#t
this
on&y
+or,s
if
a&&
of
the
+ords
are
79 of 233 25-02-2014 12:21
separated
-y
a
spa$e.
7f
there
is
a
ta-
or
p#n$t#ation
inside!
it
+o#&d
not.
%&S ) The %utut &ecor$ Searator #ariab
The
defa#&t
o#tp#t
re$ord
separator
is
a
ne+&ine!
&i,e
the
inp#t.
This
$an
-e
set
to
-e
a
ne+&ine
and
$arriae
ret#rn!
if
yo#
need
to
enerate
a
te5t
fi&e
for
a
non;U97:
syste(.
#!/bin/awk -f
# this filter adds a carriage return to all lines
# before the newline character
BEGIN {
ORS="\r\n"
}
{ print }
C&i$,
here
80 of 233 25-02-2014 12:21
to
et
fi&e:
a$$9cr;awk
*+,E(AME ) The Current *ilena"e #ariabl
The
&ast
)aria-&e
,no+n
to
re#&ar
A28
is
A371'9AM'!A
+hi$h
te&&s
yo#
the
na(e
of
the
fi&e
-ein
read.
#!/bin/awk -f
# reports which file is being read
BEGIN {
f="";
}
{ if (f != FILENAME) {
print "reading", FILENAME;
f=FILENAME;
}
print;
}
C&i$,
here
to
et
fi&e:
awk9exa"leBa;awk
This
$an
-e
#sed
if
se)era&
fi&es
need
to
-e
parsed
-y
A28.
9or(a&&y
yo#
#se
standard
inp#t
81 of 233 25-02-2014 12:21
to
pro)ide
A28
+ith
infor(ation.
*o#
$an
a&so
spe$ify
the
fi&ena(es
on
the
$o((and
&ine.
7f
the
a-o)e
s$ript
+as
$a&&ed
Atestfi&ter!A
and
if
yo#
e5e$#ted
it
+ith
testfi&ter
fi&e1
fi&e2
fi&e3
7t
+o#&d
print
o#t
the
fi&ena(e
-efore
ea$h
$hane.
An
a&ternate
+ay
to
spe$ify
this
on
the
$o((and
&ine
is
testfi&ter
fi&e1
;
fi&e3
Ufi&e2
7n
this
$ase!
the
se$ond
82 of 233 25-02-2014 12:21
fi&e
+i&&
-e
$a&&ed
A;!A
+hi$h
is
the
$on)entiona&
na(e
for
standard
inp#t.
7
ha)e
#sed
this
+hen
7
+ant
to
p#t
so(e
infor(ation
-efore
and
after
a
fi&ter
operation.
The
prefi5
and
postfi5
fi&es
spe$ia&
data
-efore
and
after
the
rea&
data.
"y
$he$,in
the
fi&ena(e!
yo#
$an
parse
the
infor(ation
different&y.
This
is
a&so
#sef#&
to
report
synta5
errors
in
parti$#&ar
fi&es:
83 of 233 25-02-2014 12:21
#!/bin/awk -f
{
if (NF == 6) {
# do the right thing
} else {
if (FILENAME == "-" ) {
print "SYNTAX ERROR, Wrong number of fields,",
"in STDIN, line #:", NR, "line: ", $0;
} else {
print "SYNTAX ERROR, Wrong number of fields,",
"Filename: ", FILENAME, "line # ", NR,"line: ", $0;
}
}
}
C&i$,
here
to
et
fi&e:
awk9exa"leC;awk
Associati-e Arrays
7
ha)e
#sed
doCens
of
different
prora((in
&an#aes
o)er
the
&ast
20
years!
and
A28
is
the
first
&an#ae
7
fo#nd
that
has
asso$iati)e
arrays.
This
ter(
(ay
-e
(eanin&ess
to
yo#!
-#t
-e&ie)e
(e!
these
arrays
are
84 of 233 25-02-2014 12:21
in)a&#a-&e!
and
si(p&ify
prora((in
enor(o#s&y.
1et
(e
des$ri-e
a
pro-&e(!
and
sho+
yo#
ho+
asso$iati)e
arrays
$an
-e
#sed
for
red#$e
$odin
ti(e!
i)in
yo#
(ore
ti(e
to
e5p&ore
another
st#pid
pro-&e(
yo#
don't
+ant
to
dea&
+ith
in
the
first
p&a$e.
1et's
s#ppose
yo#
ha)e
a
dire$tory
o)erf&o+in
+ith
fi&es!
and
yo#
+ant
to
find
o#t
ho+
(any
fi&es
are
o+ned
-y
ea$h
85 of 233 25-02-2014 12:21
#ser!
and
perhaps
ho+
(#$h
dis,
spa$e
ea$h
#ser
o+ns.
*o#
rea&&y
+ant
so(eone
to
-&a(eW
it's
hard
to
te&&
+ho
o+ns
+hat
fi&e.
A
fi&ter
that
pro$esses
the
o#tp#t
of
ls
+o#&d
+or,:
&s
;&
N
fi&ter
"#t
this
doesn't
te&&
yo#
ho+
(#$h
spa$e
ea$h
#ser
is
#sin.
7t
a&so
doesn't
+or,
for
a
&are
dire$tory
tree.
This
reJ#ires
find
and
86 of 233 25-02-2014 12:21
xargs:
find
.
;type
f
;print
N
5ars
&s
;&
N
fi&ter
The
third
$o&#(n
of
A&sA
is
the
#serna(e.
The
fi&ter
has
to
$o#nt
ho+
(any
ti(es
it
sees
ea$h
#ser.
The
typi$a&
prora(
+o#&d
ha)e
an
array
of
#serna(es
and
another
array
that
$o#nts
ho+
(any
ti(es
ea$h
#serna(e
has
-een
seen.
The
inde5
to
-oth
arrays
are
the
sa(eW
yo#
87 of 233 25-02-2014 12:21
#se
one
array
to
find
the
inde5!
and
the
se$ond
to
,eep
tra$,
of
the
$o#nt.
7'&&
sho+
yo#
one
+ay
to
do
it
in
A28;;the
+ron
+ay:
#!/bin/awk -f
# bad example of AWK programming
# this counts how many files each user owns.
BEGIN {
number_of_users=0;
}
{
# must make sure you only examine lines with 8 or more fields
if (NF>7) {
user=0;
# look for the user in our list of users
for (i=1; i<=number_of_users; i++) {
# is the user known?
if (username[i] == $3) {
# found it - remember where the user is
user=i;
}
}
if (user == 0) {
# found a new user
username[++number_of_users]=$3;
user=number_of_users;
}
# increase number of counts
count[user]++;
}
}
END {
for (i=1; i<=number_of_users; i++) {
print count[i], username[i]
}
}
C&i$,
here
to
et
88 of 233 25-02-2014 12:21
fi&e:
awk9exa"leD;awk
7
don't
+ant
yo#
to
rea$
this
s$ript.
7
to&d
yo#
it's
the
+ron
+ay
to
do
it.
7f
yo#
+ere
a
C
prora((er!
and
didn't
,no+
A28!
yo#
+o#&d
pro-a-&y
#se
a
te$hniJ#e
&i,e
the
one
a-o)e.
?ere
is
the
sa(e
prora(!
e5$ept
this
e5a(p&e
that
#ses
A28's
asso$iati)e
arrays.
The
i(portant
point
is
to
noti$e
the
differen$e
in
siCe
-et+een
89 of 233 25-02-2014 12:21
these
t+o
)ersions:
#!/bin/awk -f
{
username[$3]++;
}
END {
for (i in username) {
print username[i], i;
}
}
C&i$,
here
to
et
fi&e:
count9usersE;awk
This
is
shorter!
si(p&er!
and
much
easier
to
#nderstand;;/n$e
yo#
#nderstand
e5a$t&y
+hat
an
asso$iati)e
array
is.
The
$on$ept
is
si(p&e.
7nstead
of
#sin
a
n#(-er
to
find
an
entry
in
an
array!
#se
anything
you
want.
An
asso$iati)e
array
in
an
array
90 of 233 25-02-2014 12:21
+hose
inde5
is
a
strin.
A&&
arrays
in
A28
are
asso$iati)e.
7n
this
$ase!
the
inde5
into
the
array
is
the
third
fie&d
of
the
A&sA
$o((and!
+hi$h
is
the
#serna(e.
7f
the
#ser
is
A-in!A
the
(ain
&oop
in$re(ents
the
$o#nt
per
#ser
-y
effe$ti)e&y
e5e$#tin
#serna(eYA-inAZQQW
U97:
#r#'s
(ay
&eef#&&y
report
that
the
I
&ine
A28
s$ript
$an
-e
rep&a$ed
-y:
91 of 233 25-02-2014 12:21
a+,
'Dprint
H3E'
N
sort
N
#niJ
;$
N
sort
;nr
Tr#e!
?o+e)er!
this
$an't
$o#nt
the
tota&
dis,
spa$e
for
ea$h
#ser.
2e
need
to
add
so(e
(ore
inte&&ien$e
to
the
A28
s$ript!
and
need
the
riht
fo#ndation
to
pro$eed.
There
is
a&so
a
s&iht
-#
in
the
A28
prora(.
7f
yo#
+anted
a
AJ#i$,
and
dirtyA
so&#tion!
the
a-o)e
+o#&d
-e
fine.
92 of 233 25-02-2014 12:21
7f
yo#
+anted
to
(a,e
it
(ore
ro-#st!
yo#
ha)e
to
hand&e
#n#s#a&
$onditions.
7f
yo#
a)e
this
prora(
an
e(pty
fi&e
for
inp#t!
yo#
+o#&d
et
the
error:
a+,:
#serna(e
is
not
an
array
A&so!
if
yo#
piped
the
o#tp#t
of
A&s
;&A
to
it!
the
&ine
that
spe$ified
the
tota&
+o#&d
in$re(ent
a
non;e5istin
#ser.
There
are
t+o
te$hniJ#es
#sed
to
93 of 233 25-02-2014 12:21
e&i(inate
this
error.
The
first
one
on&y
$o#nts
)a&id
inp#t:
#!/bin/awk -f
{
if (NF>7) {
username[$3]++;
}
}
END {
}
}
C&i$,
here
to
et
fi&e:
count9users:;awk
This
fi5es
the
pro-&e(
of
$o#ntin
the
&ine
+ith
the
tota&.
?o+e)er!
it
sti&&
enerates
an
error
+hen
an
e(pty
fi&e
is
read
as
inp#t.
To
fi5
this
pro-&e(!
a
$o((on
te$hniJ#e
is
to
(a,e
94 of 233 25-02-2014 12:21
s#re
the
array
a&+ays
e5ists!
and
has
a
spe$ia&
(ar,er
)a&#e
+hi$h
spe$ifies
that
the
entry
is
in)a&id.
Then
+hen
reportin
the
res#&ts!
inore
the
in)a&id
entry.
#!/bin/awk -f
BEGIN {
username[""]=0;
}
{
username[$3]++;
}
END {
if (i != "") {
}
}
}
C&i$,
here
to
et
fi&e:
count9users<;awk
This
happens
to
fi5
the
other
pro-&e(.
App&y
this
te$hniJ#e
and
yo#
+i&&
(a,e
yo#r
95 of 233 25-02-2014 12:21
A28
prora(s
(ore
ro-#st
and
easier
for
others
to
#se.
Multi)$i"ensional Arrays
0o(e
peop&e
as,
if
A28
$an
hand&e
(#&ti;
di(ensiona&
arrays.
7t
$an.
?o+e)er!
yo#
don't
#se
$on)entiona&
t+o;di(ensiona&
arrays.
7nstead
yo#
#se
asso$iati)e
arrays.
<Fid
7
e)en
(ention
ho+
#sef#&
asso$iati)e
arrays
are@=
Be(e(-er!
yo#
$an
p#t
anything
in
the
inde5
of
an
asso$iati)e
array.
7t
reJ#ires
a
different
+ay
to
96 of 233 25-02-2014 12:21
thin,
a-o#t
pro-&e(s!
-#t
on$e
yo#
#nderstand!
yo#
+on't
-e
a-&e
to
&i)e
+itho#t
it.
A&&
yo#
ha)e
to
do
is
to
$reate
an
inde5
that
$o(-ines
t+o
other
indi$es.
0#ppose
yo#
+anted
to
effe$ti)e&y
e5e$#te
aY1!2Z
M
yW
This
is
in)a&id
in
A28.
?o+e)er!
the
fo&&o+in
is
perfe$t&y
fine:
aY1
A!A
2Z
M
yW
Be(e(-er:
the
A28
strin
$on$atenation
operator
is
97 of 233 25-02-2014 12:21
the
spa$e.
7t
$o(-ines
the
three
strins
into
the
sin&e
strin
A1!2.A
Then
it
#ses
it
as
an
inde5
into
the
array.
That's
a&&
there
is
to
it.
There
is
one
(inor
pro-&e(
+ith
asso$iati)e
arrays!
espe$ia&&y
if
yo#
#se
the
for
$o((and
to
o#tp#t
ea$h
e&e(ent:
yo#
ha)e
no
$ontro&
o)er
the
order
of
o#tp#t.
*o#
$an
$reate
an
a&orith(
to
enerate
the
indi$es
98 of 233 25-02-2014 12:21
to
an
asso$iati)e
array!
and
$ontro&
the
order
this
+ay.
?o+e)er!
this
is
diffi$#&t
to
do.
0in$e
U97:
pro)ides
an
e5$e&&ent
sort
#ti&ity!
(ore
prora((ers
separate
the
infor(ation
pro$essin
fro(
the
sortin.
7'&&
sho+
yo#
+hat
7
(ean.
Exa"le of using AWK.s Associati-e Array
7
often
find
(yse&f
#sin
$ertain
te$hniJ#es
repeated&y
in
A28.
This
e5a(p&e
+i&&
de(onstrate
these
te$hniJ#es!
and
i&&#strate
the
po+er
and
e&ean$e
of
99 of 233 25-02-2014 12:21
A28.
The
prora(
is
si(p&e
and
$o((on.
The
dis,
is
f#&&.
2ho's
onna
-e
-&a(ed@
7
6#st
hope
yo#
#se
this
po+er
+ise&y.
Be(e(-er!
yo#
(ay
-e
the
one
+ho
fi&&ed
#p
the
dis,.
?a)in
reso&)ed
(y
(ora&
di&e((a!
-y
p&a$in
the
-#rden
sJ#are&y
on
yo#r
sho#&ders!
7
+i&&
des$ri-e
the
prora(
in
detai&.
7
+i&&
a&so
dis$#ss
se)era&
tips
yo#
+i&&
find
#sef#&
100 of 233 25-02-2014 12:21
in
&are
A28
prora(s.
3irst!
initia&iCe
a&&
arrays
#sed
in
a
for
&oop.
There
+i&&
-e
fo#r
arrays
for
this
p#rpose.
7nitia&iCation
is
easy:
#R$o#ntYAAZM0W
R$o#ntYAAZM0W
#R$o#ntYAAZM0W
a&&R$o#ntYAAZM0W
The
se$ond
tip
is
to
pi$,
a
$on)ention
for
arrays.
0e&e$tin
the
na(es
of
the
arrays!
and
the
indi$es
for
ea$h
array
is
)ery
i(portant.
7n
a
$o(p&e5
prora(!
it
$an
-e$o(e
$onf#sin
to
re(e(-er
101 of 233 25-02-2014 12:21
+hi$h
array
$ontains
+hat.
7
s#est
yo#
$&ear&y
identify
the
indi$es
and
$ontents
of
ea$h
array.
To
de(onstrate!
7
+i&&
#se
a
AR$o#ntA
to
indi$ate
the
n#(-er
of
fi&es!
and
ARs#(A
to
indi$ate
the
s#(
of
the
fi&e
siCes.
7n
addition!
the
part
-efore
the
ARA
spe$ifies
the
inde5
#sed
for
the
array!
+hi$h
+i&&
-e
either
A#A
for
#ser!
AA
for
ro#p!
A#A
for
102 of 233 25-02-2014 12:21
the
#ser
and
ro#p
$o(-ination!
and
Aa&&A
for
the
tota&
for
a&&
fi&es.
7n
other
prora(s!
7
ha)e
#sed
na(es
&i,e
#serna(eRtoRdire$toryY#serna(eZMdire$toryW
3o&&o+
a
$on)ention
&i,e
this!
and
it
+i&&
-e
hard
for
yo#
to
foret
the
p#rpose
of
ea$h
asso$iati)e
array.
')en
+hen
a
J#i$,
ha$,
$o(es
-a$,
to
ha#nt
yo#
three
years
&ater.
7')e
-een
there.
The
third
s#estion
is
103 of 233 25-02-2014 12:21
to
(a,e
s#re
yo#r
inp#t
is
in
the
$orre$t
for(.
7t's
enera&&y
a
ood
idea
to
-e
pessi(isti$!
-#t
7
+i&&
add
a
si(p&e
-#t
s#ffi$ient
test
in
this
e5a(p&e.
if (NF != 10) {
# ignore
} else {
etc"
7
p&a$ed
the
test
and
error
$&a#se
#p
front!
so
the
rest
of
the
$ode
+on't
-e
$&#ttered.
A28
doesn't
ha)e
#ser
defined
f#n$tions.
9A28!
%A28
and
P'B1
do.
104 of 233 25-02-2014 12:21
The
ne5t
pie$e
of
ad)i$e
for
$o(p&e5
A28
s$ripts
is
to
define
a
na(e
for
ea$h
fie&d
#sed.
7n
this
$ase!
+e
+ant
the
#ser!
ro#p
and
siCe
in
dis,
-&o$,s.
2e
$o#&d
#se
the
fi&e
siCe
in
-ytes!
-#t
the
-&o$,
siCe
$orresponds
to
the
-&o$,s
on
the
dis,!
a
(ore
a$$#rate
(eas#re(ent
of
spa$e.
Fis,
-&o$,s
$an
-e
fo#nd
-y
#sin
A&s
105 of 233 25-02-2014 12:21
;s.A
This
adds
a
$o&#(n!
so
the
#serna(e
-e$o(es
the
fo#rth
$o&#(n!
et$.
Therefore
the
s$ript
+i&&
$ontain:
siCeMH1W
#serMH4W
ro#pMH5W
This
+i&&
a&&o+
#s
to
easi&y
adapt
to
$hanes
in
inp#t.
2e
$o#&d
#se
AH1A
thro#ho#t
the
s$ript!
-#t
if
+e
$haned
the
n#(-er
of
fie&ds!
+hi$h
the
A;sA
option
does!
+e'd
ha)e
to
$hane
ea$h
fie&d
referen$e.
*o#
don't
+ant
to
106 of 233 25-02-2014 12:21
o
thro#h
an
A28
s$ript!
and
$hane
a&&
the
AH1A
to
AH2!A
and
a&so
$hane
the
AH2A
to
AH3A
-e$a#se
those
are
rea&&y
the
AH1A
that
yo#
6#st
$haned
to
AH2.A
/f
course
this
is
$onf#sin.
That's
+hy
it's
a
ood
idea
to
assin
na(es
to
the
fie&ds.
7')e
-een
there
too.
9e5t
the
A28
s$ript
+i&&
$o#nt
ho+
(any
ti(es
ea$h
$o(-ination
of
107 of 233 25-02-2014 12:21
#sers
and
ro#ps
o$$#r.
That
is!
7
a(
oin
to
$onstr#$t
a
t+o;part
inde5
that
$ontains
the
#serna(e
and
ro#pna(e.
This
+i&&
&et
(e
$o#nt
#p
the
n#(-er
of
ti(es
ea$h
#serKro#p
$o(-ination
o$$#rs!
and
ho+
(#$h
dis,
spa$e
is
#sed.
Consider
this:
ho+
+o#&d
yo#
$a&$#&ate
the
tota&
for
6#st
a
#ser!
or
for
6#st
a
ro#p@
*o#
$o#&d
re+rite
the
s$ript.
/r
108 of 233 25-02-2014 12:21
yo#
$o#&d
ta,e
the
#serKro#p
tota&s!
and
tota&
the(
+ith
a
se$ond
s$ript.
*o#
$o#&d
do
it!
-#t
it's
not
the
A28
+ay
to
do
it.
7f
yo#
had
to
e5a(ine
a
-aCi&&ion
fi&es!
and
it
ta,es
a
&on
ti(e
to
r#n
that
s$ript!
it
+o#&d
-e
a
+aste
to
repeat
this
tas,.
7t's
a&so
ineffi$ient
to
reJ#ire
t+o
s$ripts
+hen
one
$an
do
109 of 233 25-02-2014 12:21
e)erythin.
The
proper
+ay
to
so&)e
this
pro-&e(
is
to
e5tra$t
as
(#$h
infor(ation
as
possi-&e
in
one
pass
thro#h
the
fi&es.
Therefore
this
s$ript
+i&&
find
the
n#(-er
and
siCe
for
ea$h
$ateory:
'a$h
#ser
'a$h
ro#p
'a$h
#serKro#p
$o(-ination
A&&
#sers
and
ro#ps
This
is
+hy
7
ha)e
4
arrays
to
$o#nt
#p
the
n#(-er
of
fi&es.
7
don't
rea&&y
need
110 of 233 25-02-2014 12:21
4
arrays!
as
7
$an
#se
the
for(at
of
the
inde5
to
deter(ine
+hi$h
array
is
+hi$h.
"#t
this
does
(aa,e
the
prora(
easier
to
#nderstand
for
no+.
The
ne5t
tip
is
s#-t&e!
-#t
yo#
+i&&
see
ho+
#sef#&
it
is.
7
(entioned
the
indi$es
into
the
array
$an
-e
anythin.
7f
possi-&e!
se&e$t
a
for(at
that
a&&o+s
yo#
to
(ere
infor(ation
fro(
se)era&
arrays.
111 of 233 25-02-2014 12:21
7
rea&iCe
this
(a,es
no
sense
riht
no+!
-#t
han
in
there.
A&&
+i&&
-e$o(e
$&ear
soon.
7
+i&&
do
this
-y
$onstr#$tin
a
#ni)ersa&
inde5
of
the
for(
U#serV
Uro#pV
This
inde5
+i&&
-e
#sed
for
a&&
arrays.
There
is
a
spa$e
-et+een
the
t+o
)a&#es.
This
$o)ers
the
tota&
for
the
#serKro#p
$o(-ination.
2hat
a-o#t
the
other
three
arrays@
7
+i&&
112 of 233 25-02-2014 12:21
#se
a
ASA
to
indi$ate
the
tota&
for
a&&
#sers
or
ro#ps.
Therefore
the
inde5
for
a&&
fi&es
+o#&d
-e
AS
SA
+hi&e
the
inde5
for
a&&
of
the
fi&e
o+ned
-y
#ser
daemon
+o#&d
-e
Adae(on
S.A
The
heart
of
the
s$ript
tota&s
#p
the
n#(-er
and
siCe
of
ea$h
fi&e!
p#ttin
the
infor(ation
into
the
riht
$ateory.
7
+i&&
#se
I
arraysW
4
113 of 233 25-02-2014 12:21
for
fi&e
siCes!
and
4
for
$o#nts:
#R$o#ntY#ser
A
SAZQQW
R$o#ntYAS
A
ro#pZQQW
#R$o#ntY#ser
A
A
ro#pZQQW
a&&R$o#ntYAS
SAZQQW
#RsiCeY#ser
A
SAZQMsiCeW
RsiCeYAS
A
ro#pZQMsiCeW
#RsiCeY#ser
A
A
ro#pZQMsiCeW
a&&RsiCeYAS
SAZQMsiCeW
This
parti$#&ar
#ni)ersa&
inde5
+i&&
(a,e
sortin
easier!
as
yo#
+i&&
see.
A&so
i(portant
is
to
sort
the
infor(ation
in
an
order
that
is
#sef#&.
*o#
$an
try
to
for$e
114 of 233 25-02-2014 12:21
a
parti$#&ar
o#tp#t
order
in
A28!
-#t
+hy
+or,
at
this!
+hen
it's
a
one
&ine
$o((and
for
sort@
The
diffi$#&t
part
is
findin
the
riht
+ay
to
sort
the
infor(ation.
This
s$ript
+i&&
sort
infor(ation
#sin
the
siCe
of
the
$ateory
as
the
first
sort
fie&d.
The
&arest
tota&
+i&&
-e
the
one
for
a&&
fi&es!
so
this
+i&&
-e
one
of
the
first
115 of 233 25-02-2014 12:21
&ines
o#tp#t.
?o+e)er!
there
(ay
-e
se)era&
ties
for
the
&arest
n#(-er!
and
$are
(#st
-e
#sed.
The
se$ond
fie&d
+i&&
-e
the
n#(-er
of
fi&es.
This
+i&&
he&p
-rea,
a
tie.
0ti&&!
7
+ant
the
tota&s
and
s#-;tota&s
to
-e
&isted
-efore
the
indi)id#a&
#serKro#p
$o(-inations.
The
third
and
fo#rth
fie&ds
+i&&
-e
enerated
-y
the
inde5
of
the
array.
This
is
the
tri$,y
116 of 233 25-02-2014 12:21
part
7
+arned
yo#
a-o#t.
The
s$ript
+i&&
o#tp#t
one
strin!
-#t
the
sort
#ti&ity
+i&&
not
,no+
this.
7nstead!
it
+i&&
treat
it
as
t+o
fie&ds.
This
+i&&
#nify
the
res#&ts!
and
infor(ation
fro(
a&&
4
arrays
+i&&
&oo,
&i,e
one
array.
The
sort
of
the
third
and
fo#rth
fie&ds
+i&&
-e
di$tionary
order!
and
not
n#(eri$!
#n&i,e
the
first
t+o
fie&ds.
The
ASA
117 of 233 25-02-2014 12:21
+as
#sed
so
these
s#-;tota&
fie&ds
+i&&
-e
&isted
-efore
the
indi)id#a&
#serKro#p
$o(-ination.
The
arrays
+i&&
-e
printed
#sin
the
fo&&o+in
for(at:
for (i in u_count) {
if (i != "") {
print u_size[i], u_count[i], i;
}
}
O
7
on&y
sho+ed
yo#
one
array!
-#t
a&&
fo#r
are
printed
the
sa(e
+ay.
That's
the
essen$e
of
the
s$ript.
The
res#&ts
is
sorted!
and
7
$on)erted
the
spa$e
into
a
ta-
for
$os(eti$
118 of 233 25-02-2014 12:21
reasons.
%utut of the scrit
7
$haned
(y
dire$tory
to
#usr#ucb!
#sed
the
s$ript
in
that
dire$tory.
The
fo&&o+in
is
the
o#tp#t:
size count user group
3173 81 * *
3173 81 root *
2973 75 * staff
2973 75 root staff
88 3 * daemon
88 3 root daemon
64 2 * kmem
64 2 root kmem
48 1 * tty
48 1 root tty
This
says
there
are
I1
fi&es
in
this
dire$tory!
+hi$h
ta,es
#p
3143
dis,
-&o$,s.
A&&
of
the
fi&es
are
o+ned
-y
root.
2943
dis,
-&o$,s
-e&on
to
ro#p
staff.
There
119 of 233 25-02-2014 12:21
are
3
fi&es
+ith
ro#p
dae(on!
+hi$h
ta,es
#p
II
dis,
-&o$,s.
As
yo#
$an
see!
the
first
&ine
of
infor(ation
is
the
tota&
for
a&&
#sers
and
ro#ps.
The
se$ond
&ine
is
the
s#-;tota&
for
the
#ser
Aroot.A
The
third
&ine
is
the
s#-;tota&
for
the
ro#p
Astaff.A
Therefore
the
order
of
the
sort
is
#sef#&!
+ith
the
s#-;tota&s
-efore
the
indi)id#a&
entries.
120 of 233 25-02-2014 12:21
*o#
$o#&d
+rite
a
si(p&e
A28
or
rep
s$ript
to
o-tain
infor(ation
fro(
6#st
one
#ser
or
one
ro#p!
and
the
infor(ation
+i&&
-e
easy
to
sort.
There
is
on&y
one
pro-&e(.
The
#usr#ucb
dire$tory
on
(y
syste(
on&y
#ses
1I49
-&o$,sW
at
&east
that's
+hat
du
reports.
2here's
the
dis$repan$y@
The
s$ript
does
not
#nderstand
hard
&in,s.
This
(ay
not
-e
a
pro-&e(
121 of 233 25-02-2014 12:21
on
(ost
dis,s!
-e$a#se
(any
#sers
do
not
#se
hard
&in,s.
0ti&&!
it
does
enerate
ina$$#rate
res#&ts.
7n
this
$ase!
the
prora(
vi
is
a&so
e!
ex!
edit!
view!
and
2
other
na(es.
The
prora(
on&y
e5ists
on$e!
-#t
has
4
na(es.
*o#
$an
te&&
-e$a#se
the
&in,
$o#nt
<fie&d
2=
reports
4.
This
$a#ses
the
fi&e
to
-e
$o#nted
4
ti(es!
+hi$h
$a#ses
an
122 of 233 25-02-2014 12:21
ina$$#rate
tota&.
The
fi5
is
to
on&y
$o#nt
(#&tip&e
&in,s
on$e.
'5a(inin
the
&in,
$o#nt
+i&&
deter(ine
if
a
fi&e
has
(#&tip&e
&in,s.
?o+e)er!
ho+
$an
yo#
pre)ent
$o#ntin
a
&in,
t+i$e@
There
is
an
easy
so&#tion:
a&&
of
these
fi&es
ha)e
the
sa(e
inode
n#(-er.
*o#
$an
find
this
n#(-er
+ith
the
-i
option
to
ls.
To
sa)e
(e(ory!
+e
on&y
ha)e
to
re(e(-er
123 of 233 25-02-2014 12:21
the
inodes
of
fi&es
that
ha)e
(#&tip&e
&in,s.
This
(eans
+e
ha)e
to
add
another
$o&#(n
to
the
inp#t!
and
ha)e
to
ren#(-er
a&&
of
the
fie&d
referen$es.
7t's
a
ood
thin
there
are
on&y
three.
Addin
a
ne+
fie&d
+i&&
-e
easy!
-e$a#se
7
fo&&o+ed
(y
o+n
ad)i$e.
The
fina&
s$ript
sho#&d
-e
easy
to
fo&&o+.
7
ha)e
#sed
)ariations
of
this
h#ndreds
124 of 233 25-02-2014 12:21
of
ti(es
and
find
it
de(onstrates
the
po+er
of
A28
as
+e&&
as
pro)ide
insiht
to
a
po+erf#&
prora((in
paradi(.
A28
so&)es
these
types
of
pro-&e(s
easier
than
(ost
&an#aes.
"#t
yo#
ha)e
to
#se
A28
the
riht
+ay.
9ote
;
this
)ersion
+as
+ritten
for
a
0o&aris
-o5.
*o#
ha)e
to
)erify
if
ls
is
eneratin
the
riht
n#(-er
of
ar#(ents.
The
-g
125 of 233 25-02-2014 12:21
ar#(ent
(ay
need
to
-e
de&eted!
and
the
$he$,
for
the
n#(-er
of
fi&es
(ay
ha)e
to
-e
(odified.
U$ate$7
added
a
1in#5
)ersion
-e&o+
;
to
-e
do+n&oaded.
This
is
a
f#&&y
+or,in
)ersion
of
the
prora(!
that
a$$#rate&y
$o#nts
dis,
spa$e!
appears
-e&o+:
#!/bin/sh
find . -type f -print | xargs /usr/bin/ls -islg |
awk '
BEGIN {
# initialize all arrays used in for loop
u_count[""]=0;
g_count[""]=0;
ug_count[""]=0;
all_count[""]=0;
}
{
# validate your input
if (NF != 11) {
# ignore
} else {
# assign field names
inode=$1;
size=$2;
linkcount=$4;
user=$5;
126 of 233 25-02-2014 12:21
group=$6;
# should I count this file?
doit=0;
if (linkcount == 1) {
# only one copy - count it
doit++;
} else {
# a hard link - only count first one
seen[inode]++;
if (seen[inode] == 1) {
doit++;
}
}
# if doit is true, then count the file
if (doit ) {
# total up counts in one pass
# use description array names
# use array index that unifies the arrays
# first the counts for the number of files
u_count[user " *"]++;
g_count["* " group]++;
ug_count[user " " group]++;
all_count["* *"]++;
# then the total disk space used
u_size[user " *"]+=size;
g_size["* " group]+=size;
ug_size[user " " group]+=size;
all_size["* *"]+=size;
}
}
}
END {
# output in a form that can be sorted
for (i in u_count) {
if (i != "") {
print u_size[i], u_count[i], i;
}
}
for (i in g_count) {
if (i != "") {
print g_size[i], g_count[i], i;
}
}
for (i in ug_count) {
if (i != "") {
print ug_size[i], ug_count[i], i;
}
}
for (i in all_count) {
if (i != "") {
print all_size[i], all_count[i], i;
}
}
} ' |
# numeric sort - biggest numbers first
# sort fields 0 and 1 first (sort starts with 0)
# followed by dictionary sort on fields 2 + 3
sort +0nr -2 +2d |
# add header
(echo "size count user group";cat -) |
# convert space to tab - makes it nice output
# the second set of quotes contains a single tab character
tr ' ' ' '
127 of 233 25-02-2014 12:21
# done - I hope you like it
C&i$,
here
to
et
fi&e:
count9users>;awk
Be(e(-er
+hen
7
said
7
didn't
need
to
#se
4
different
arrays@
7
$an
#se
6#st
one.
This
is
(ore
$onf#sin!
-#t
(ore
$on$ise
#!/bin/sh
find . -type f -print | xargs /usr/bin/ls -islg |
awk '
BEGIN {
# initialize all arrays used in for loop
count[""]=0;
}
{
# validate your input
if (NF != 11) {
# ignore
} else {
# assign field names
inode=$1;
size=$2;
linkcount=$4;
user=$5;
group=$6;
# should I count this file?
doit=0;
if (linkcount == 1) {
# only one copy - count it
doit++;
} else {
# a hard link - only count first one
seen[inode]++;
if (seen[inode] == 1) {
doit++;
}
}
128 of 233 25-02-2014 12:21
# if doit is true, then count the file
if (doit ) {
# total up counts in one pass
# use description array names
# use array index that unifies the arrays
# first the counts for the number of files
count[user " *"]++;
count["* " group]++;
count[user " " group]++;
count["* *"]++;
# then the total disk space used
size[user " *"]+=size;
size["* " group]+=size;
size[user " " group]+=size;
size["* *"]+=size;
}
}
}
END {
# output in a form that can be sorted
for (i in count) {
if (i != "") {
print size[i], count[i], i;
}
}
} ' |
# numeric sort - biggest numbers first
# sort fields 0 and 1 first (sort starts with 0)
# followed by dictionary sort on fields 2 + 3
sort +0nr -2 +2d |
# add header
(echo "size count user group";cat -) |
# convert space to tab - makes it nice output
# the second set of quotes contains a single tab character
tr ' ' ' '
# done - I hope you like it
C&i$,
here
to
et
fi&e:
count9users;awk
?ere
is
a
)ersion
that
+or,s
+ith
(odern
1in#5
syste(s!
-#t
ass#(es
yo#
ha)e
+e&&;-eha)ed
fi&ena(es
<+itho#t
129 of 233 25-02-2014 12:21
spa$es!
et$!=:
count9users9new;awk
/icture /erfect /&+(T* %utut
0o
far!
7
des$ri-ed
se)era&
si(p&e
s$ripts
that
pro)ide
#sef#&
infor(ation!
in
a
so(e+hat
#&y
o#tp#t
for(at.
Co&#(ns
(iht
not
&ine
#p
proper&y!
and
it
is
often
hard
to
find
patterns
or
trends
+itho#t
this
#nity.
As
yo#
#se
A28
(ore!
yo#
+i&&
-e
desiro#s
of
$risp!
$&ean
for(attin.
To
a$hie)e
this!
yo#
(#st
(aster
the
printf
f#n$tion.
130 of 233 25-02-2014 12:21
/&+(T* ) for"atting outut
The
printf
is
)ery
si(i&ar
to
the
C
f#n$tion
+ith
the
sa(e
na(e.
C
prora((ers
sho#&d
ha)e
no
pro-&e(
#sin
printf
f#n$tion.
$rintf
has
one
of
these
synta$ti$a&
for(s:
printf
<
for(at=W
printf
<
for(at!
ar#(ents...=W
printf
<
for(at=
Ve5pressionW
printf
<
for(at!
ar#(ents...=
V
e5pressionW
The
parenthesis
and
se(i$o&on
are
optiona&.
7
on&y
#se
the
first
for(at
to
131 of 233 25-02-2014 12:21
-e
$onsistent
+ith
other
near-y
printf
state(ents.
A
print
state(ent
+o#&d
do
the
sa(e
thin.
$rintf
re)ea&s
it's
rea&
po+er
+hen
for(attin
$o((ands
are
#sed.
The
first
ar#(ent
to
the
printf
f#n$tion
is
the
for(at.
This
is
a
strin!
or
)aria-&e
+hose
)a&#e
is
a
strin.
This
strin!
&i,e
a&&
strins!
$an
$ontain
spe$ia&
es$ape
seJ#en$es
to
print
$ontro&
$hara$ters.
Escae Se0uences
132 of 233 25-02-2014 12:21
The
$hara$ter
AGA
is
#sed
to
Aes$apeA
or
(ar,
spe$ia&
$hara$ters.
The
&ist
of
these
$hara$ters
is
in
ta-&e
-e&o+:
AWK Table @
Escae Se0uences
Se0uence !escrition
Ga
A0C77 -e&&
<9A28K%A28
on&y=
G- "a$,spa$e
Gf 3or(feed
Gn 9e+&ine
Gr
Carriae
Bet#rn
Gt ?oriConta& ta-
G)
Lerti$a& ta-
<9A28 on&y=
Gddd
Chara$ter <1
to 3 o$ta&
diits= <9A28
on&y=
G5dd
Chara$ter
<he5ade$i(a&=
<9A28 on&y=
GUAny
other
$hara$terV
That
$hara$ter
7t's
diffi$#&t
to
e5p&ain
the
differen$es
+itho#t
-ein
+ordy.
?opef#&&y
7'&&
pro)ide
eno#h
e5a(p&es
to
133 of 233 25-02-2014 12:21
de(onstrate
the
differen$es.
2ith
9A28!
yo#
$an
print
three
ta-
$hara$ters
#sin
these
three
different
representations:
printf<AGt
G11G59GnA=W
A
ta-
$hara$ter
is
de$i(a&
9!
o$ta&
11!
or
he5ade$i(a&
09.
0ee
the
(an
pae
as$ii<4=
for
(ore
infor(ation.
0i(i&ar&y!
yo#
$an
print
three
do#-&e;J#ote
$hara$ters
<de$i(a&
34!
he5ade$i(a&
22!
or
o$ta&
42
=
#sin
printf("\"\x22\42\n");
*o#
sho#&d
noti$e
a
differen$e
-et+een
the
134 of 233 25-02-2014 12:21
printf
f#n$tion
and
the
print
f#n$tion.
$rint
ter(inates
the
&ine
+ith
the
%&S
$hara$ter!
and
di)ides
ea$h
fie&d
+ith
the
%*S
separator.
$rintf
does
nothin
#n&ess
yo#
spe$ify
the
a$tion.
Therefore
yo#
+i&&
freJ#ent&y
end
ea$h
&ine
+ith
the
ne+&ine
$hara$ter
AGn!A
and
yo#
(#st
spe$ify
the
separatin
$hara$ters
e5p&i$it&y.
*or"at Secifiers
The
po+er
of
the
printf
state(ent
&ies
in
the
for(at
spe$ifiers!
135 of 233 25-02-2014 12:21
+hi$h
a&+ays
start
+ith
the
$hara$ter
AT.A
The
for(at
spe$ifiers
are
des$ri-ed
in
ta-&e
6:
AWK Table B
*or"at Secifiers
Secifier Meaning
T$
A0C77
Chara$ter
Td
Fe$i(a&
inteer
Te
3&oatin
Point
n#(-er
<enineerin
for(at=
Tf
3&oatin
Point
n#(-er
<fi5ed point
for(at=
T
The shorter
of e or f!
+ith trai&in
Ceros
re(o)ed
To /$ta&
Ts 0trin
T5 ?e5ade$i(a&
TT 1itera& T
Aain!
7'&&
$o)er
the
differen$es
J#i$,&y.
Ta-&e
3
i&&#strates
the
differen$es.
The
first
&ine
states
Aprintf<T$GnA!100.0=AA
prints
a
Ad.A
136 of 233 25-02-2014 12:21
AWK Table C
Exa"le of for"at
con-ersions
*or"at #alue &esults
T$ 100.0 d
T$ A100.0A 1 <9A28@=
T$ 42 A
Td 100.0 100
Te 100.0 1.000000eQ02
Tf 100.0 100.000000
T 100.0 100
To 100.0 144
Ts 100.0 100.0
Ts A13fA 13f
Td A13fA 0 <A28=
Td A13fA 13 <9A28=
T5 100.0 64
This
ta-&e
re)ea&s
so(e
differen$es
-et+een
A28
and
9A28.
2hen
a
strin
+ith
n#(-ers
and
&etters
are
$o)erted
into
an
inteer!
A28
+i&&
ret#rn
a
Cero!
+hi&e
9A28
+i&&
$on)ert
as
(#$h
as
possi-&e.
The
se$ond
e5a(p&e!
(ar,ed
+ith
A9A28@A
+i&&
ret#rn
137 of 233 25-02-2014 12:21
AdA
on
so(e
ear&ier
)ersions
of
9A28!
+hi&e
&ater
)ersions
+i&&
ret#rn
A1.A
Usin
for(at
spe$ifiers!
there
is
another
+ay
to
print
a
do#-&e
J#ote
+ith
9A28.
This
de(onstrates
/$ta&!
Fe$i(a&
and
?e5ade$i(a&
$on)ersion.
As
yo#
$an
see!
it
isn't
sy((etri$a&.
Fe$i(a&
$on)ersions
are
done
different&y.
printf("%s%s%s%c\n", "\"", "\x22", "\42", 34);
"et+een
the
ATA
and
the
for(at
$hara$ter
$an
-e
fo#r
optiona&
pie$es
of
infor(ation.
7t
138 of 233 25-02-2014 12:21
he&ps
to
)is#a&iCe
these
fie&ds
as:
TUsinV
UCeroV
U+idthV.
Upre$isionVfor(at
7'&&
dis$#ss
ea$h
one
separate&y.
Wi$th ) secifying "ini"u" fiel$ si1e
7f
there
is
a
n#(-er
after
the
AT!A
this
spe$ifies
the
(ini(#(
n#(-er
of
$hara$ters
to
print.
This
is
the
width
fie&d.
0pa$es
are
added
so
the
n#(-er
of
printed
$hara$ters
eJ#a&
this
n#(-er.
9ote
that
this
is
the
(ini(#(
fie&d
siCe.
7f
the
139 of 233 25-02-2014 12:21
fie&d
-e$o(es
to
&are!
it
+i&&
ro+!
so
infor(ation
+i&&
not
-e
&ost.
0pa$es
are
added
to
the
&eft.
This
for(at
a&&o+s
yo#
to
&ine
#p
$o&#(ns
perfe$t&y.
Consider
the
fo&&o+in
for(at:
printf<A
TstTdGnA!
s!
d=W
7f
the
strin
AsA
is
&oner
than
I
$hara$ters!
the
$o&#(ns
+on't
&ine
#p.
7nstead!
#se
printf<AT20sTdGnA!
s!
d=W
As
&on
as
the
strin
is
140 of 233 25-02-2014 12:21
&ess
than
20
$hara$ters!
the
n#(-er
+i&&
start
on
the
21st
$o&#(n.
7f
the
strin
is
too
&on!
then
the
t+o
fie&ds
+i&&
r#n
toether!
(a,in
it
hard
to
read.
*o#
(ay
+ant
to
$onsider
p&a$in
a
sin&e
spa$e
-et+een
the
fie&ds!
to
(a,e
s#re
yo#
+i&&
a&+ays
ha)e
one
spa$e
-et+een
the
fie&ds.
This
is
)ery
i(portant
if
yo#
+ant
to
pipe
the
o#tp#t
141 of 233 25-02-2014 12:21
to
another
prora(.
Addin
infor(ationa&
headers
(a,es
the
o#tp#t
(ore
reada-&e.
"e
a+are
that
$hanin
the
for(at
of
the
data
(ay
(a,e
it
diffi$#&t
to
et
the
$o&#(ns
a&ined
perfe$t&y.
Consider
the
fo&&o+in
s$ript:
#!/usr/bin/awk -f
BEGIN {
printf("String Number\n");
}
{
printf("%10s %6d\n", $1, $2);
}
C&i$,
here
to
et
fi&e:
awk9exa"leF;awk
7t
+o#&d
-e
a+,+ard
<fori)e
the
$hoi$e
of
+ords=
to
add
a
ne+
$o&#(n
and
142 of 233 25-02-2014 12:21
retain
the
sa(e
a&in(ent.
More
$o(p&i$ated
for(ats
+o#&d
reJ#ire
a
&ot
of
tria&
and
error.
*o#
ha)e
to
ad6#st
the
first
printf
to
aree
+ith
the
se$ond
printf
state(ent.
7
s#est
#!/usr/bin/awk -f
BEGIN {
printf("%10s %6sn", "String", "Number");
}
{
printf("%10s %6d\n", $1, $2);
}
C&i$,
here
to
et
fi&e:
awk9exa"le:E;awk
or
e)en
-etter
#!/usr/bin/awk -f
BEGIN {
format1 ="%10s %6sn";
format2 ="%10s %6dn";
printf(format1, "String", "Number");
}
{
printf(format2, $1, $2);
}
143 of 233 25-02-2014 12:21
C&i$,
here
to
et
fi&e:
awk9exa"le::;awk
The
&ast
e5a(p&e!
-y
#sin
strin
)aria-&es
for
for(attin!
a&&o+s
yo#
to
,eep
a&&
of
the
for(ats
toether.
This
(ay
not
see(
&i,e
it's
)ery
#sef#&!
-#t
+hen
yo#
ha)e
(#&tip&e
for(ats
and
(#&tip&e
$o&#(ns!
it's
)ery
#sef#&
to
ha)e
a
set
of
te(p&ates
&i,e
the
a-o)e.
7f
yo#
ha)e
to
add
an
e5tra
spa$e
to
(a,e
thins
144 of 233 25-02-2014 12:21
&ine
#p!
it's
(#$h
easier
to
find
and
$orre$t
the
pro-&e(
+ith
a
set
of
for(at
strins
that
are
toether!
and
the
e5a$t
sa(e
+idth.
C?ainin
the
first
$o&#(ne
fro(
10
$hara$ters
to
11
is
easy.
,eft 2ustification
The
&ast
e5a(p&e
p&a$es
spa$es
-efore
ea$h
fie&d
to
(a,e
s#re
the
(ini(#(
fie&d
+idth
is
(et.
2hat
do
yo#
do
if
yo#
+ant
the
145 of 233 25-02-2014 12:21
spa$es
on
the
riht@
Add
a
neati)e
sin
-efore
the
+idth:
printf<AT;10s
T;6dGnA!
H1!
H2=W
This
+i&&
(o)e
the
printin
$hara$ters
to
the
&eft!
+ith
spa$es
added
to
the
riht.
The *iel$ /recision #alue
The
pre$ision
fie&d!
+hi$h
is
the
n#(-er
-et+een
the
de$i(a&
and
the
for(at
$hara$ter!
is
(ore
$o(p&e5.
Most
peop&e
#se
it
+ith
the
f&oatin
point
for(at
<Tf=!
-#t
s#rprisin&y!
146 of 233 25-02-2014 12:21
it
$an
-e
#sed
+ith
any
for(at
$hara$ter.
2ith
the
o$ta&!
de$i(a&
or
he5ade$i(a&
for(at!
it
spe$ifies
the
(ini(#(
n#(-er
of
$hara$ters.
[eros
are
added
to
(et
this
reJ#ire(ent.
2ith
the
Te
and
Tf
for(ats!
it
spe$ifies
the
n#(-er
of
diits
after
the
de$i(a&
point.
The
Te
AeQ00A
is
not
in$&#ded
in
the
pre$ision.
The
T
for(at
$o(-ines
the
$hara$teristi$s
of
the
Td
and
Tf
147 of 233 25-02-2014 12:21
for(ats.
The
pre$ision
spe$ifies
the
n#(-er
of
diits
disp&ayed!
-efore
and
after
the
de$i(a&
point.
The
pre$ision
fie&d
has
no
effe$t
on
the
T$
fie&d.
The
Ts
for(at
has
an
#n#s#a&!
-#t
#sef#&
effe$t:
it
spe$ifies
the
(a5i(#(
n#(-er
of
sinifi$ant
$hara$ters
to
print.
7f
the
first
n#(-er
after
the
AT!A
or
after
the
AT;!A
is
a
Cero!
then
the
syste(
adds
Ceros
+hen
148 of 233 25-02-2014 12:21
paddin.
This
in$&#des
a&&
for(at
types!
in$&#din
strins
and
the
T$
$hara$ter
for(at.
This
(eans
AT010dA
and
AT.10dA
-oth
adds
&eadin
Ceros!
i)in
a
(ini(#(
of
10
diits.
The
for(at
AT10.10dA
is
therefore
red#ndant.
Ta-&e
I
i)es
so(e
e5a(p&es:
AWK Table D
Exa"les of co"lex for"atting
*or"at #ariable &esults
T$ 100 AdA
T10$ 100 A dA
T010$ 100 A000000000dA
Td 10 A10A
T10d 10 A 10A
T10.4d 10.1234564I9 A 0010A
T10.Id 10.1234564I9 A 00000010A
T.Id 10.1234564I9 A00000010A
T010d 10.1234564I9 A0000000010A
Te 9I4.1234564I90 A9.I41235eQ02A
T10.4e 9I4.1234564I90 A9.I412eQ02A
T10.Ie 9I4.1234564I90 A9.I4123454eQ02A
Tf 9I4.1234564I90 A9I4.123454A
149 of 233 25-02-2014 12:21
T10.4f 9I4.1234564I90 A 9I4.1235A
T010.4f 9I4.1234564I90 A009I4.1235A
T10.If 9I4.1234564I90 A9I4.12345649A
T 9I4.1234564I90 A9I4.123A
T10 9I4.1234564I90 A 9I4.123A
T10.4 9I4.1234564I90 A 9I4.1A
T010.4 9I4.1234564I90 A000009I4.1A
T.I 9I4.1234564I90 A9I4.12346A
To 9I4.1234564I90 A1433A
T10o 9I4.1234564I90 A 1433A
T010o 9I4.1234564I90 A0000001433A
T.Io 9I4.1234564I90 A00001433A
Ts 9I4.123 A9I4.123A
T10s 9I4.123 A 9I4.123A
T10.4s 9I4.123 A 9I4.A
T010.Is 9I4.123 A0009I4.123A
T5 9I4.1234564I90 A3d-A
T105 9I4.1234564I90 A 3d-A
T0105 9I4.1234564I90 A00000003d-A
T.I5 9I4.1234564I90 A000003d-A
There
is
one
(ore
topi$
needed
to
$o(p&ete
this
&esson
on
printf.
Exlicit *ile outut
7nstead
of
sendin
o#tp#t
to
standard
o#tp#t!
yo#
$an
send
o#tp#t
to
a
na(ed
fi&e.
The
for(at
150 of 233 25-02-2014 12:21
is
printf<AstrinGnA=
V
AKt(pKfi&eAW
*o#
$an
append
to
an
e5istin
fi&e!
-y
#sin
AVV:A
printf<AstrinGnA=
VV
AKt(pKfi&eAW
1i,e
the
she&&!
the
do#-&e
an&e
-ra$,ets
indi$ates
o#tp#t
is
aen$e$
to
the
fi&e!
instead
of
written
to
an
e(pty
fi&e.
Appendin
to
the
fi&e
does
not
de&ete
the
o&d
$ontents.
?o+e)er!
there
is
a
s#-t&e
differen$e
-et+een
A28
and
the
she&&.
Consider
the
151 of 233 25-02-2014 12:21
she&&
prora(:
#!/bin/sh
while x=`line`
do
echo got $x >>/tmp/a
echo got $x >/tmp/b
done
This
+i&&
read
standard
inp#t!
and
$opy
the
standard
inp#t
to
fi&es
AKt(pKaA
and
AKt(pK-.A
3i&e
AKt(pKaA
+i&&
ro+
&arer!
as
infor(ation
is
a&+ays
appended
to
the
fi&e.
3i&e
AKt(pK-!A
ho+e)er!
+i&&
on&y
$ontain
one
&ine.
This
happens
-e$a#se
ea$h
ti(e
the
she&&
see
the
AVA
or
AVVA
$hara$ters!
it
opens
the
fi&e
for
+ritin!
$hoosin
152 of 233 25-02-2014 12:21
the
tr#n$ateK$reate
or
appendin
option
at
that
ti(e.
9o+
$onsider
the
eJ#i)a&ent
A28
prora(:
#!/usr/bin/awk -f
{
print $0 >>"/tmp/a"
print $0 >"/tmp/b"
}
This
-eha)es
different&y.
A28
$hooses
the
$reateKappend
option
the
first
ti(e
a
fi&e
is
opened
for
+ritin.
After+ards!
the
#se
of
AVA
or
AVVA
is
inored.
Un&i,e
the
she&&!
A28
$opies
a&&
of
standard
inp#t
to
fi&e
AKt(pK-.A
7nstead
of
a
strin!
so(e
153 of 233 25-02-2014 12:21
)ersions
of
A28
a&&o+
yo#
to
spe$ify
an
e5pression:
O
Ynote
to
se&fZ
$he$,
this
one
;
it
(iht
not
+or,
printf<AstrinGnA=
V
371'9AM'
A.o#tAW
The
fo&&o+in
#ses
a
strin
$on$atenation
e5pression
to
i&&#strate
this:
#!/usr/bin/awk -f
END {
for (i=0;i<30;i++) {
printf("i=%d\n", i) > "/tmp/a" i;
}
}
C&i$,
here
to
et
fi&e:
awk9exa"le:<;awk
This
s$ript
ne)er
finishes!
-e$a#se
A28
$an
ha)e
10
additiona&
fi&es
open!
154 of 233 25-02-2014 12:21
and
9A28
$an
ha)e
20.
7f
yo#
find
this
to
-e
a
pro-&e(!
&oo,
into
P'B1.
7
hope
this
i)es
yo#
the
s,i&&
to
(a,e
yo#r
A28
o#tp#t
pi$t#re
perfe$t.
AWK (u"erical *unctions
7n
pre)io#s
t#toria&s!
7
ha)e
sho+n
ho+
#sef#&
A28
is
in
(anip#&atin
infor(ation!
and
eneratin
reports.
2hen
yo#
add
a
fe+
f#n$tions!
A28
-e$o(es
e)en
(ore!
(((!
f#n$tiona&.
There
155 of 233 25-02-2014 12:21
are
three
types
of
f#n$tions:
n#(eri$!
strin
and
+hate)er's
&eft.
Ta-&e9
&ists
a&&
of
the
n#(eri$
f#n$tions:
AWK Table F
(u"eric *unctions
(a"e *unction #ariant
$os $osine %A28!A28!9A28
e5p '5ponent %A28!A28!9A28
int 7nteer %A28!A28!9A28
&o 1oarith( %A28!A28!9A28
sin 0ine %A28!A28!9A28
sJrt
0J#are
Boot
%A28!A28!9A28
atan2 Ar$tanent %A28!9A28
rand Bando( %A28!9A28
srand
0eed
Bando(
%A28!9A28
Trigono"etric *unctions
/h
6oy.
7
-et
(i&&ions!
if
not
doCens!
of
(y
readers
ha)e
-een
+aitin
for
(e
to
dis$#ss
triono(etry.
Persona&&y!
7
don't
#se
triono(etry
(#$h
156 of 233 25-02-2014 12:21
at
+or,!
e5$ept
+hen
7
o
off
on
a
tanent.
0orry
a-o#t
that.
7
don't
,no+
+hat
$a(e
o)er
(e.
7
don't
#s#a&&y
resort
to
p#ns.
7'&&
+rite
a
note
to
(yse&f!
and
after
7
sine
the
note!
7'&&
ha)e
(y
-oss
$osine
it.
(ow
sto
thatG
7
hate
ar#in
+ith
(yse&f.
7
a&+ays
&ose.
Thin,in
a-o#t
(ath
7
&earned
in
the
year
157 of 233 25-02-2014 12:21
2
".C.
<"efore
Co(p#ters=
see(s
to
$a#se
f&ash-a$,s
of
hih
s$hoo&!
pi(p&es!
and
<sh#dder=
ti(es
-est
&eft
forotten.
The
stress
of
re(e(-erin
those
days
(#st
ha)e
(ade
(e
foret
the
standards
7
nor(a&&y
set
for
(yse&f.
"esides!
no;one
appre$iates
o-t#se
h#(or
any+ay!
e)en
if
7
find
a$#te
+ay
to
say
it.
7
-etter
$hane
the
s#-6e$t
fast.
Co(-inin
h#(or
and
$o(p#ters
is
a
)ery
158 of 233 25-02-2014 12:21
serio#s
(atter.
?ere
is
a
9A28
s$ript
that
$a&$#&ates
the
triono(etri$
f#n$tions
for
a&&
derees
-et+een
0
and
360.
7t
a&so
sho+s
+hy
there
is
no
tanent!
se$ant
or
$ose$ant
f#n$tion.
<They
aren't
ne$essary=.
7f
yo#
read
the
s$ript!
yo#
+i&&
&earn
of
so(e
s#-t&e
differen$es
-et+een
A28
and
9A28.
A&&
this
in
a
thin
)eneer
of
de(onstratin
+hy
+e
&earned
triono(etry
in
the
159 of 233 25-02-2014 12:21
first
p&a$e.
2hat
(ore
$an
yo#
as,
for@
/h!
in
$ase
yo#
are
+onderin!
7
+rote
this
in
the
(onth
of
Fe$e(-er.
#!/usr/bin/nawk -f
#
# A smattering of trigonometry...
#
# This AWK script plots the values from 0 to 360
# for the basic trigonometry functions
# but first - a review:
#
# (Note to the editor - the following diagram assumes
# a fixed width font, like Courier.
# otherwise, the diagram looks very stupid, instead of slightly stupid)
#
# Assume the following right triangle
#
# Angle Y
#
# |
# |
# |
# a | c
# |
# |
# +------- Angle X
# b
#
# since the triangle is a right angle, then
# X+Y=90
#
# Basic Trigonometric Functions. If you know the length
# of 2 sides, and the angles, you can find the length of the third side.
# Also - if you know the length of the sides, you can calculate
# the angles.
#
# The formulas are
#
# sine(X) = a/c
# cosine(X) = b/c
# tangent(X) = a/b
#
# reciprocal functions
# cotangent(X) = b/a
# secant(X) = c/b
# cosecant(X) = c/a
#
# Example 1)
160 of 233 25-02-2014 12:21
# if an angle is 30, and the hypotenuse (c) is 10, then
# a = sine(30) * 10 = 5
# b = cosine(30) * 10 = 8.66
#
# The second example will be more realistic:
#
# Suppose you are looking for a Christmas tree, and
# while talking to your family, you smack into a tree
# because your head was turned, and your kids were arguing over who
# was going to put the first ornament on the tree.
#
# As you come to, you realize your feet are touching the trunk of the tree,
# and your eyes are 6 feet from the bottom of your frostbitten toes.
# While counting the stars that spin around your head, you also realize
# the top of the tree is located at a 65 degree angle, relative to your eyes
# You suddenly realize the tree is 12.84 feet high! After all,
# tangent(65 degrees) * 6 feet = 12.84 feet
# All right, it isn't realistic. Not many people memorize the
# tangent table, or can estimate angles that accurately.
# I was telling the truth about the stars spinning around the head, however.
#
BEGIN {
# assign a value for pi.
PI=3.14159;
# select an "Ed Sullivan" number - really really big
BIG=999999;
# pick two formats
# Keep them close together, so when one column is made larger
# the other column can be adjusted to be the same width
fmt1="%7s %8s %8s %8s %10s %10s %10s %10sn";
# print out the title of each column
fmt2="%7d %8.2f %8.2f %8.2f %10.2f %10.2f %10.2f %10.2fn";
# old AWK wants a backslash at the end of the next line
# to continue the print statement
# new AWK allows you to break the line into two, after a comma
printf(fmt1,"Degrees","Radians","Cosine","Sine",
"Tangent","Cotangent","Secant", "Cosecant");
for (i=0;i<=360;i++) {
# convert degrees to radians
r = i * (PI / 180 );
# in new AWK, the backslashes are optional
# in OLD AWK, they are required
printf(fmt2, i, r,
# cosine of r
cos(r),
# sine of r
sin(r),
#
# I ran into a problem when dividing by zero.
# So I had to test for this case.
#
# old AWK finds the next line too complicated
# I don't mind adding a backslash, but rewriting the
# next three lines seems pointless for a simple lesson.
# This script will only work with new AWK, now - sigh...
# On the plus side,
# I don't need to add those back slashes anymore
#
# tangent of r
(cos(r) == 0) ? BIG : sin(r)/cos(r),
# cotangent of r
(sin(r) == 0) ? BIG : cos(r)/sin(r),
# secant of r
(cos(r) == 0) ? BIG : 1/cos(r),
# cosecant of r
(sin(r) == 0) ? BIG : 1/sin(r));
161 of 233 25-02-2014 12:21
}
# put an exit here, so that standard input isn't needed.
exit;
}
C&i$,
here
to
et
fi&e:
trigono"etry;awk
9A28
a&so
has
the
ar$tanent
f#n$tion.
This
is
#sef#&
for
so(e
raphi$s
+or,!
as
ar$
tanent<aK-=
M
an&e
<in
radians=
Therefore
if
yo#
ha)e
the
:
and
*
&o$ations!
the
ar$tanent
of
the
ratio
+i&&
te&&
yo#
the
an&e.
The
atan%<=
f#n$tion
ret#rns
a
)a&#e
fro(
neati)e
pi
to
positi)e
162 of 233 25-02-2014 12:21
pi.
Exonents3 logs an$ s0uare roots
The
fo&&o+in
s$ript
#ses
three
other
arith(eti$
f#n$tions:
log!
exp!
and
s&rt.
7
+anted
to
sho+
ho+
these
$an
-e
#sed
toether!
so
7
di)ided
the
&o
of
a
n#(-er
-y
t+o!
+hi$h
is
another
+ay
to
find
a
sJ#are
root.
7
then
$o(pared
the
)a&#e
of
the
e5ponent
of
that
ne+
&o
to
the
-#i&t;in
sJ#are
root
f#n$tion.
7
163 of 233 25-02-2014 12:21
then
$a&$#&ated
the
differen$e
-et+een
the
t+o!
and
$on)erted
the
differen$e
into
a
positi)e
n#(-er.
#!/bin/awk -f
# demonstrate use of exp(), log() and sqrt in AWK
# e.g. what is the difference between using logarithms and regular arithmeti
# note - exp and log are natural log functions - not base 10
#
BEGIN {
# what is the about of error that will be reported?
ERROR=0.000000000001;
# loop a long while
for (i=1;i<=2147483647;i++) {
# find log of i
logi=log(i);
# what is square root of i?
# divide the log by 2
logsquareroot=logi/2;
# convert log of i back
squareroot=exp(logsquareroot);
# find the difference between the logarithmic calculation
# and the built in calculation
diff=sqrt(i)-squareroot;
# make difference positive
if (diff < 0) {
diff*=-1;
}
if (diff > ERROR) {
printf("%10d, squareroot: %16.8f, error: %16.14f\n",
i, squareroot, diff);
}
}
exit;
}
C&i$,
here
to
et
fi&e:
awk9exa"le:>;awk
*a+n.
This
e5a(p&e
isn't
too
e5$itin!
e5$ept
to
those
+ho
en6oy
nitpi$,in.
164 of 233 25-02-2014 12:21
'5pe$t
the
prora(
to
rea$h
3
(i&&ion
-efore
yo#
see
any
errors.
7'&&
i)e
yo#
a
(ore
e5$itin
sa(p&e
soon.
Truncating +ntegers
A&&
)ersion
of
A28
$ontain
the
int
f#n$tion.
This
tr#n$ates
a
n#(-er!
(a,in
it
an
inteer.
7t
$an
-e
#sed
to
ro#nd
n#(-ers
-y
addin
0.5:
printf<Aro#ndin
TI.4f
i)es
TIdnA!
5!
int<5Q0.5==W
&an$o" (u"bers
9A28
has
f#n$tions
that
165 of 233 25-02-2014 12:21
$an
enerate
rando(
n#(-ers.
The
f#n$tion
rand
ret#rns
a
rando(
n#(-er
-et+een
0
and
1.
?ere
is
an
e5a(p&e
that
$a&$#&ates
a
(i&&ion
rando(
n#(-ers
-et+een
0
and
100!
and
$o#nts
ho+
often
ea$h
n#(-er
+as
#sed:
#!/usr/bin/nawk -f
# old AWK doesn't have rand() and srand()
# only new AWK has them
# how random is the random function?
BEGIN {
# srand();
i=0;
while (i++<1000000) {
x=int(rand()*100 + 0.5);
y[x]++;
}
for (i=0;i<=100;i++) {
printf("%dt%d\n",y[i],i);
}
exit;
}
C&i$,
here
to
et
fi&e:
ran$o";awk
7f
yo#
e5e$#te
166 of 233 25-02-2014 12:21
this
s$ript
se)era&
ti(es!
yo#
+i&&
et
the
e5a$t
sa(e
res#&ts.
'5perien$ed
prora((ers
,no+
rando(
n#(-er
enerators
aren't
rea&&y
rando(!
#n&ess
they
#se
spe$ia&
hard+are.
These
n#(-ers
are
pse#do;
rando(!
and
$a&$#&ated
#sin
so(e
a&orith(.
0in$e
the
a&orith(
is
fi5ed!
the
n#(-ers
are
repeata-&e
#n&ess
the
n#(-ers
are
seeded
+ith
a
#niJ#e
)a&#e.
This
is
done
#sin
the
srand
f#n$tion
a-o)e!
+hi$h
is
$o((ented
o#t.
167 of 233 25-02-2014 12:21
Typi$a&&y
the
rando(
n#(-er
enerator
is
not
i)en
a
spe$ia&
seed
#nti&
the
-#s
ha)e
-een
+or,ed
o#t
of
the
prora(.
There's
nothin
(ore
fr#stratin
than
a
-#
that
o$$#rs
rando(&y.
The
srand
f#n$tion
(ay
-e
i)en
an
ar#(ent.
7f
not!
it
#ses
the
$#rrent
ti(e
and
day
to
enerate
a
seed
for
the
rando(
n#(-er
enerator.
The ,otto scrit
7
pro(ised
a
(ore
168 of 233 25-02-2014 12:21
#sef#&
s$ript.
This
(ay
-e
+hat
yo#
are
+aitin
for.
7t
reads
t+o
n#(-ers!
and
enerates
a
&ist
of
rando(
n#(-ers.
7
$a&&
the
s$ript
A&otto.a+,.A
#!/usr/bin/nawk -f
BEGIN {
# Assume we want 6 random numbers between 1 and 36
# We could get this information by reading standard input,
# but this example will use a fixed set of parameters.
#
# First, initialize the seed
srand();
# How many numbers are needed?
NUM=6;
# what is the minimum number
MIN=1;
# and the maximum?
MAX=36;
# How many numbers will we find? start with 0
Number=0;
while (Number < NUM) {
r=int(((rand() *(1+MAX-MIN))+MIN));
# have I seen this number before?
if (array[r] == 0) {
# no, I have not
Number++;
array[r]++;
}
}
# now output all numbers, in order
for (i=MIN;i<=MAX;i++) {
# is it marked in the array?
if (array[i]) {
# yes
printf("%d ",i);
}
}
printf("\n");
exit;
}
C&i$,
169 of 233 25-02-2014 12:21
here
to
et
fi&e:
lotto;awk
7f
yo#
do
+in
a
&ottery!
send
(e
a
post$ard.
String *unctions
"esides
n#(eri$
f#n$tions!
there
are
t+o
other
types
of
f#n$tion:
strins
and
the
+hat$ha(a$a&&its.
3irst!
a
&ist
of
the
strin
f#n$tions:
AWK Table :E
String *unctions
(a"e #ariant
inde5<strin!sear$h=
A28!
9A28!
%A28
&enth<strin=
A28!
9A28!
%A28
sp&it<strin!array!separator=
A28!
9A28!
%A28
s#-str<strin!position=
A28!
9A28!
%A28
s#-str<strin!position!(a5=
A28!
9A28!
%A28
s#-<ree5!rep&a$e(ent=
9A28!
%A28
170 of 233 25-02-2014 12:21
s#-<ree5!rep&a$e(ent!strin=
9A28!
%A28
s#-<ree5!rep&a$e(ent=
9A28!
%A28
s#-<ree5!rep&a$e(ent!strin=
9A28!
%A28
(at$h<strin!ree5=
9A28!
%A28
to&o+er<strin= %A28
to#pper<strin= %A28
asort<strin!YdZ= %A28
asorti<strin!YdZ= %A28
ens#-<r!s!h Y!tZ= %A28
strton#(<strin= %A28
Most
peop&e
first
#se
A28
to
perfor(
si(p&e
$a&$#&ations.
Asso$iati)e
arrays
and
triono(etri$
f#n$tions
are
so(e+hat
esoteri$
feat#res!
that
ne+
#sers
e(-ra$e
+ith
the
eaerness
of
a
$hain
s(o,er
in
a
fire+or,s
fa$tory.
7
s#spe$t
(ost
#sers
add
so(e
si(p&e
strin
f#n$tions
to
their
repertoire
on$e
they
171 of 233 25-02-2014 12:21
+ant
to
add
a
&itt&e
(ore
sophisti$ation
to
their
A28
s$ripts.
7
hope
this
$o&#(n
i)es
yo#
eno#h
infor(ation
to
inspire
yo#r
ne5t
effort.
There
are
fo#r
strin
f#n$tions
in
the
oriina&
A28:
index<=!
length<=!
split<=!
and
substr<=.
These
f#n$tions
are
J#ite
)ersati&e.
The ,ength function
2hat
$an
7
say@
The
length<=
f#n$tion
$a&$#&ates
the
&enth
of
a
strin.
7
often
#se
it
172 of 233 25-02-2014 12:21
to
(a,e
s#re
(y
inp#t
is
$orre$t.
7f
yo#
+anted
to
inore
e(pty
&ines!
$he$,
the
&enth
of
the
ea$h
&ine
-efore
pro$essin
it
+ith
if
<&enth<H0=
V
1=
D
.
.
.
E
*o#
$an
easi&y
#se
it
to
print
a&&
&ines
&oner
than
a
$ertain
&enth!
et$.
The
fo&&o+in
$o((and
$enters
a&&
&ines
shorter
than
I0
$hara$ters:
#!/bin/awk -f
{
if (length($0) < 80) {
prefix = "";
173 of 233 25-02-2014 12:21
for (i = 1;i<(80-length($0))/2;i++)
prefix = prefix " ";
print prefix $0;
} else {
print;
}
}
C&i$,
here
to
et
fi&e:
center;awk
The +n$ex *unction
7f
yo#
+ant
to
sear$h
for
a
spe$ia&
$hara$ter!
the
index<=
f#n$tion
+i&&
sear$h
for
spe$ifi$
$hara$ters
inside
a
strin.
To
find
a
$o((a!
the
$ode
(iht
&oo,
&i,e
this:
senten$eMAThis
is
a
short!
(eanin&ess
senten$e.AW
if
<inde5<senten$e!
A!A=
V
0=
D
printf<A3o#nd
a
$o((a
in
174 of 233 25-02-2014 12:21
position
GTdGnA!
inde5<senten$e!A!A==W
E
The
f#n$tion
ret#rns
a
positi)e
)a&#e
+hen
the
s#-strin
is
fo#nd.
The
n#(-er
spe$ified
the
&o$ation
of
the
s#-strin.
7f
the
s#-strin
$onsists
of
2
or
(ore
$hara$ters!
a&&
of
these
$hara$ters
(#st
-e
fo#nd!
in
the
sa(e
order!
for
a
non;Cero
ret#rn
)a&#e.
1i,e
the
length<=
f#n$tion!
this
is
#sef#&
for
$he$,in
for
proper
inp#t
$onditions.
175 of 233 25-02-2014 12:21
The Substr function
The
substr<=
f#n$tion
$an
e5tra$t
a
portion
of
a
strin.
/ne
$o((on
#se
is
to
sp&it
a
strin
into
t+o
parts
-ased
on
a
spe$ia&
$hara$ter.
7f
yo#
+anted
to
pro$ess
so(e
(ai&
addresses!
the
fo&&o+in
$ode
fra(ent
(iht
do
the
6o-:
#!/bin/awk -f
{
# field 1 is the e-mail address - perhaps
if ((x=index($1,"@")) > 0) {
username = substr($1,1,x-1);
hostname = substr($1,x+1,length($1));
# the above is the same as
# hostname = substr($1,x+1);
printf("username = %s, hostname = %s\n", username, hostname);
}
}
C&i$,
here
to
et
fi&e:
176 of 233 25-02-2014 12:21
e"ail;awk
The
substr<=
f#n$tion
ta,es
t+o
or
three
ar#(ents.
The
first
is
the
strin!
the
se$ond
is
the
position.
The
optiona&
third
ar#(ent
is
the
&enth
of
the
strin
to
e5tra$t.
7f
the
third
ar#(ent
is
(issin!
the
rest
of
the
strin
is
#sed.
The
s#-str
f#n$tion
$an
-e
#sed
in
(any
non;o-)io#s
+ays.
As
an
e5a(p&e!
it
$an
-e
#sed
to
$on)ert
#pper
177 of 233 25-02-2014 12:21
$ase
&etters
to
&o+er
$ase.
#!/usr/bin/awk -f
# convert upper case letters to lower case
BEGIN {
LC="abcdefghijklmnopqrstuvwxyz";
UC="ABCDEFGHIJKLMNOPQRSTUVWXYZ";
}
{
out="";
# look at each character
for(i=1;i<=length($0);i++) {
# get the character to be checked
char=substr($0,i,1);
# is it an upper case letter?
j=index(UC,char);
if (j > 0 ) {
# found it
out = out substr(LC,j,1);
} else {
out = out char;
}
}
printf("%s\n", out);
}
C&i$,
here
to
et
fi&e:
uer9to9lower;awk
5AWK.s Tolower an$ Touer function
%A28
has
the
toupper<=
and
tolower<=
f#n$tions!
for
$on)enient
$on)ersions
of
$ase.
These
f#n$tions
ta,e
strins!
so
yo#
$an
red#$e
the
a-o)e
s$ript
to
a
178 of 233 25-02-2014 12:21
sin&e
&ine:
#!/usr/local/bin/gawk -f
{
print tolower($0);
}
C&i$,
here
to
et
fi&e:
uer9to9lower;gawk
The Slit function
Another
+ay
to
sp&it
#p
a
strin
is
to
#se
the
split<=
f#n$tion.
7t
ta,es
three
ar#(ents:
the
strin!
an
array!
and
the
separator.
The
f#n$tion
ret#rns
the
n#(-er
of
pie$es
fo#nd.
?ere
is
an
e5a(p&e:
#!/usr/bin/awk -f
BEGIN {
# this script breaks up the sentence into words, using
# a space as the character separating the words
string="This is a string, is it not?";
search=" ";
n=split(string,array,search);
for (i=1;i<=n;i++) {
printf("Word[%d]=%s\n",i,array[i]);
179 of 233 25-02-2014 12:21
}
exit;
}
C&i$,
here
to
et
fi&e:
awk9exa"le:?;sh
The
third
ar#(ent
is
typi$a&&y
a
sin&e
$hara$ter.
7f
a
&oner
strin
is
#sed!
on&y
the
first
&etter
is
#sed
as
a
separator.
(AWK.s string functions
9A28
<and
%A28=
ha)e
additiona&
strin
f#n$tions!
+hi$h
add
a
pri(iti)e
0'F;&i,e
f#n$tiona&ity:
sub<=!
match<=!
and
gsub<=.
'ub<=
perfor(s
a
strin
s#-stit#tion!
&i,e
sed.
To
rep&a$e
180 of 233 25-02-2014 12:21
Ao&dA
+ith
Ane+A
in
a
strin!
#se
s#-<Ko&dK!
Ane+A!
strin=
7f
the
third
ar#(ent
is
(issin!
H0
is
ass#(ed
to
-e
strin
sear$hed.
The
f#n$tion
ret#rns
1
if
a
s#-stit#tion
o$$#rs!
and
0
if
not.
7f
no
s&ashes
are
i)en
in
the
first
ar#(ent!
the
first
ar#(ent
is
ass#(ed
to
-e
a
)aria-&e
$ontainin
a
re#&ar
e5pression.
The
sub<=
on&y
$hanes
the
first
181 of 233 25-02-2014 12:21
o$$#rren$e.
The
gsub<=
f#n$tion
is
si(i&ar
to
the
g
option
in
sed:
a&&
o$$#rren$e
are
$on)erted!
and
not
6#st
the
first.
That
is!
if
the
patter
o$$#rs
(ore
than
on$e
per
&ine
<or
strin=!
the
s#-stit#tion
+i&&
-e
perfor(ed
on$e
for
ea$h
fo#nd
pattern.
The
fo&&o+in
s$ript:
#!/usr/bin/nawk -f
BEGIN {
string = "Another sample of an example sentence";
pattern="[Aa]n";
if (gsub(pattern,"AN",string)) {
printf("Substitution occurred: %s\n", string);
}
exit;
}
C&i$,
here
to
et
fi&e:
182 of 233 25-02-2014 12:21
awk9exa"le:@;awk
print
the
fo&&o+in
+hen
e5e$#ted:
0#-stit#tion
o$$#rred:
A9other
sa(p&e
of
A9
e5a(p&e
senten$e
As
yo#
$an
see!
the
pattern
$an
-e
a
re#&ar
e5pression.
The Match function
As
the
a-o)e
de(onstrates!
the
sub<=
and
gsub<=
ret#rns
a
positi)e
)a&#e
if
a
(at$h
is
fo#nd.
?o+e)er!
it
has
a
side;effe$t
of
$hanin
the
strin
tested.
7f
yo#
don't
+ish
this!
yo#
183 of 233 25-02-2014 12:21
$an
$opy
the
strin
to
another
)aria-&e!
and
test
the
spare
)aria-&e.
9A28
a&so
pro)ides
the
match<=
f#n$tion.
7f
match<=
finds
the
re#&ar
e5pression!
it
sets
t+o
spe$ia&
)aria-&es
that
indi$ate
+here
the
re#&ar
e5pression
-eins
and
ends.
?ere
is
an
e5a(p&e
that
does
this:
#!/usr/bin/nawk -f
# demonstrate the match function
BEGIN {
regex="[a-zA-Z0-9]+";
}
{
if (match($0,regex)) {
# RSTART is where the pattern starts
# RLENGTH is the length of the pattern
before = substr($0,1,RSTART-1);
pattern = substr($0,RSTART,RLENGTH);
after = substr($0,RSTART+RLENGTH);
printf("%s<%s>%s\n", before, pattern, after);
}
}
C&i$,
here
184 of 233 25-02-2014 12:21
to
et
fi&e:
awk9exa"le:B;awk
1ast&y!
there
are
the
+hat$ha(a$a&&it
f#n$tions.
7
$o#&d
#se
the
+ord
A(is$e&&aneo#s!A
-#t
it's
too
hard
to
spe&&.
Farn
it!
7
had
to
&oo,
it
#p
any+ay.
AWK Table ::
Miscellaneous *unctions
(a"e #ariant
et&ine
A28!
9A28!
%A28
et&ine Ufi&e
9A28!
%A28
et&ine )aria-&e
9A28!
%A28
et&ine )aria-&e
Ufi&e
9A28!
%A28
A$o((andA N
et&ine
9A28!
%A28
A$o((andA N
et&ine )aria-&e
9A28!
%A28
syste(<$o((and=
9A28!
%A28
$&ose<$o((and=
9A28!
%A28
systi(e<= %A28
strfti(e<strin= %A28
strfti(e<strin!
ti(esta(p=
%A28
The Syste" function
185 of 233 25-02-2014 12:21
9A28
has
a
f#n$tion
system<=
that
$an
e5e$#te
any
prora(.
7t
ret#rns
the
e5it
stat#s
of
the
prora(.
if
<syste(<AK-inKr(
6#n,A=
PM
0=
print
A$o((and
didn't
+or,AW
The
$o((and
$an
-e
a
strin!
so
yo#
$an
dyna(i$a&&y
$reate
$o((ands
-ased
on
inp#t.
9ote
that
the
o#tp#t
isn't
sent
to
the
9A28
prora(.
*o#
$o#&d
send
it
to
a
fi&e!
and
open
that
186 of 233 25-02-2014 12:21
fi&e
for
readin.
There
is
another
so&#tion!
ho+e)er.
The 5etline function
A28
has
a
$o((and
that
a&&o+s
yo#
to
for$e
a
ne+
&ine.
7t
doesn't
ta,e
any
ar#(ents.
7t
ret#rns
a
1!
if
s#$$essf#&!
a
0
if
end;of;fi&e
is
rea$hed!
and
a
;1
if
an
error
o$$#rs.
As
a
side
effe$t!
the
&ine
$ontainin
the
inp#t
$hanes.
This
ne5t
s$ript
fi&ters
the
inp#t!
and
187 of 233 25-02-2014 12:21
if
a
-a$,s&ash
o$$#rs
at
the
end
of
the
&ine!
it
reads
the
ne5t
&ine
in!
e&i(inatin
the
-a$,s&ash
as
+e&&
as
the
need
for
it.
#!/usr/bin/awk -f
# look for a as the last character.
# if found, read the next line and append
{
line = $0;
while (substr(line,length(line),1) == "\\") {
# chop off the last character
line = substr(line,1,length(line)-1);
i=getline;
if (i > 0) {
line = line $0;
} else {
printf("missing continuation on line %d\n", NR);
}
}
print line;
}
C&i$,
here
to
et
fi&e:
awk9exa"le:C;awk
7nstead
of
readin
into
the
standard
)aria-&es!
yo#
$an
spe$ify
the
)aria-&e
to
188 of 233 25-02-2014 12:21
set:
et&ine
aR&ine
print
aR&ineW
9A28
and
%A28
a&&o+
the
getline
f#n$tion
to
-e
i)en
an
optiona&
fi&ena(e
or
strin
$ontainin
a
fi&ena(e.
An
e5a(p&e
of
a
pri(iti)e
fi&e
prepro$essor!
that
&oo,s
for
&ines
of
the
for(at
Oin$&#de
fi&ena(e
and
s#-stit#tes
that
&ine
for
the
$ontents
of
the
fi&e:
#!/usr/bin/nawk -f
{
# a primitive include preprocessor
if (($1 == "#include") && (NF == 2)) {
# found the name of the file
filename = $2;
while (i = getline < filename ) {
print;
}
} else {
print;
}
189 of 233 25-02-2014 12:21
}
C&i$,
here
to
et
fi&e:
inclu$e;nawk
9A28's
et&ine
$an
a&so
read
fro(
a
pipe.
7f
yo#
ha)e
a
prora(
that
enerates
sin&e
&ine!
yo#
$an
#se
A$o((andA
N
et&ineW
print
H0W
or
A$o((andA
N
et&ine
a-$W
print
a-$W
7f
yo#
ha)e
(ore
than
one
&ine!
yo#
$an
&oop
thro#h
the
res#&ts:
while ("command" | getline) {
cmd[i++] = $0;
}
for (i in cmd) {
printf("%s=%s\n", i, cmd[i]);
}
190 of 233 25-02-2014 12:21
/n&y
one
pipe
$an
-e
open
at
a
ti(e.
7f
yo#
+ant
to
open
another
pipe!
yo#
(#st
e5e$#te
$&ose<A$o((andA=W
This
is
ne$essary
e)en
if
the
end
of
fi&e
is
rea$hed.
The systi"e function
The
systime<=
f#n$tion
ret#rns
the
$#rrent
ti(e
of
day
as
the
n#(-er
of
se$onds
sin$e
Midniht!
Jan#ary
1!
1940.
7t
is
#sef#&
for
(eas#rin
ho+
&on
portions
191 of 233 25-02-2014 12:21
of
yo#r
%A28
$ode
ta,es
to
e5e$#te.
# how long does it take to do a few loops?
BEGIN {
LOOPS=100;
# do the test twice
start=systime();
for (i=0;i<LOOPS;i++) {
}
end = systime();
# calculate how long it takes to do a dummy test
do_nothing = end-start;
# now do the test again with the *IMPORTANT* code inside
start=systime();
for (i=0;i<LOOPS;i++) {
# How long does this take?
while ("date" | getline) {
date = $0;
}
close("date");
}
end = systime();
newtime = (end - start) - do_nothing;
if (newtime <= 0) {
printf("%d loops were not enough to test, increase it\n",
LOOPS);
exit;
} else {
printf("%d loops took %6.4f seconds to execute\n",
LOOPS, newtime);
printf("That's %10.8f seconds per loop\n",
(newtime)/LOOPS);
# since the clock has an accuracy of +/- one second, what is the error
printf("accuracy of this measurement = %6.2f%%\n",
(1/(newtime))*100);
}
exit;
}
C&i$,
here
to
et
fi&e:
awk9exa"le:C;gawk
The Strfti"e function
%A28
has
a
spe$ia&
f#n$tion
for
$reatin
strins
192 of 233 25-02-2014 12:21
-ased
on
the
$#rrent
ti(e.
7t's
-ased
on
the
strftime<3$=
f#n$tion.
7f
yo#
are
fa(i&iar
+ith
the
AQA
for(ats
of
the
date<1=
$o((and!
yo#
ha)e
a
ood
head;start
on
#nderstandin
+hat
the
strftime
$o((and
is
#sed
for.
The
systime<=
f#n$tion
ret#rns
the
$#rrent
date
in
se$onds.
9ot
)ery
#sef#&
if
yo#
+ant
to
$reate
a
strin
-ased
on
the
ti(e.
2hi&e
yo#
$o#&d
$on)ert
the
193 of 233 25-02-2014 12:21
se$onds
into
days!
(onths!
years!
et$.!
it
+o#&d
-e
easier
to
e5e$#te
AdateA
and
pipe
the
res#&ts
into
a
strin.
<0ee
the
pre)io#s
s$ript
for
an
e5a(p&e=.
%A28
has
another
so&#tion
that
e&i(inates
the
need
for
an
e5terna&
prora(.
The
f#n$tion
ta,es
one
or
t+o
ar#(ents.
The
first
ar#(ent
is
a
strin
that
spe$ified
the
for(at.
This
strin
$ontains
re#&ar
$hara$ters
and
spe$ia&
$hara$ters.
194 of 233 25-02-2014 12:21
0pe$ia&
$hara$ters
start
+ith
a
-a$,s&ash
or
the
per$ent
$hara$ter.
The
-a$,s&ash
$hara$ters
+ith
the
-a$,s&ash
prefi5
are
the
sa(e
7
$o)ered
ear&ier.
7n
addition!
the
strftime<=
f#n$tion
defines
doCens
of
$o(-inations!
a&&
of
+hi$h
start
+ith
AT.A
The
fo&&o+in
ta-&e
&ists
these
spe$ia&
seJ#en$es:
AWK Table :<
5AWK.s strfti"e
for"ats
Ta
The &o$a&e's
a--re)iated
+ee,day
na(e
TA
The &o$a&e's
f#&& +ee,day
na(e
T-
The &o$a&e's
a--re)iated
(onth na(e
T"
The &o$a&e's
f#&& (onth
na(e
195 of 233 25-02-2014 12:21
T$
The &o$a&e's
AappropriateA
date and ti(e
representation
Td
The day of the
(onth as a
de$i(a&
n#(-er
<01;;31=
T?
The ho#r
<24;ho#r
$&o$,= as a
de$i(a&
n#(-er
<00;;23=
T7
The ho#r
<12;ho#r
$&o$,= as a
de$i(a&
n#(-er
<01;;12=
T6
The day of the
year as a
de$i(a&
n#(-er
<001;;366=
T(
The (onth as
a de$i(a&
n#(-er
<01;;12=
TM
The (in#te as
a de$i(a&
n#(-er
<00;;59=
Tp
The &o$a&e's
eJ#i)a&ent of
the AMKPM
T0
The se$ond as
a de$i(a&
n#(-er
<00;;61=.
TU
The +ee,
n#(-er of the
year <0#nday
is first day of
+ee,=
T+
The +ee,day
as a de$i(a&
n#(-er
<0;;6=.
0#nday is day
0
T2
The +ee,
n#(-er of the
year <Monday
is first day of
+ee,=
T5
The &o$a&e's
AappropriateA
date
representation
196 of 233 25-02-2014 12:21
T:
The &o$a&e's
AappropriateA
ti(e
representation
Ty
The year
+itho#t
$ent#ry as a
de$i(a&
n#(-er
<00;;99=
T*
The year +ith
$ent#ry as a
de$i(a&
n#(-er
T[
The ti(e Cone
na(e or
a--re)iation
TT A &itera& T.
Fependin
on
yo#r
operatin
syste(!
and
insta&&ation!
yo#
(ay
a&so
ha)e
the
fo&&o+in
for(ats:
AWK Table :>
%tional 5AWK
strfti"e for"ats
TF
'J#i)a&ent to
spe$ifyin
T(KTdKTy
Te
The day of
the (onth!
padded +ith a
-&an, if it is
on&y one diit
Th
'J#i)a&ent to
T-! a-o)e
Tn
A ne+&ine
$hara$ter
<A0C77 13=
Tr
'J#i)a&ent to
spe$ifyin
T7:TM:T0
Tp
TB
'J#i)a&ent to
spe$ifyin
T?:TM
TT
'J#i)a&ent to
spe$ifyin
T?:TM:T0
197 of 233 25-02-2014 12:21
Tt
A TA"
$hara$ter
T,
The ho#r as a
de$i(a&
n#(-er
<0;23=
T&
The ho#r
<12;ho#r
$&o$,= as a
de$i(a&
n#(-er
<1;12=
TC
The $ent#ry!
as a n#(-er
-et+een 00
and 99
T#
is rep&a$ed -y
the +ee,day
as a de$i(a&
n#(-er
YMonday MM
1Z
TL
is rep&a$ed -y
the +ee,
n#(-er of
the year
<#sin 70/
I601=
T)
The date in
LM0 for(at
<e..
20;JU9;1991=
/ne
#sef#&
for(at
is
strfti(e<A
TyRT(R
TdRT?R
TMRT0A=
This
$onstr#$ts
a
strin
that
$ontains
the
year!
(onth!
day!
ho#r!
(in#te
and
se$ond
in
a
for(at
that
a&&o+s
$on)enient
198 of 233 25-02-2014 12:21
sortin.
7f
yo#
ran
this
at
noon
on
Christ(as!
1994!
it
+o#&d
enerate
the
strin
94R12R25R12R00R00
?ere
is
the
%A28
eJ#i)a&ent
of
the
date
$o((and:
#! /usr/local/bin/gawk -f
#
BEGIN {
format = "%a %b %e %H:%M:%S %Z %Y";
print strftime(format);
}
C&i$,
here
to
et
fi&e:
$ate;gawk
*o#
+i&&
note
that
there
is
no
e5it
$o((and
in
the
-ein
state(ent.
7f
7
+as
#sin
A28!
an
e5it
state(ent
199 of 233 25-02-2014 12:21
is
ne$essary.
/ther+ise!
it
+o#&d
ne)er
ter(inate.
7f
there
is
no
a$tion
defined
for
ea$h
&ine
read!
9A28
and
%A28
do
not
need
an
e5it
state(ent.
7f
yo#
pro)ide
a
se$ond
ar#(ent
to
the
strftime<=
f#n$tion!
it
#ses
that
ar#(ent
as
the
ti(esta(p!
instead
of
the
$#rrent
syste('s
ti(e.
This
is
#sef#&
for
$a&$#&atin
f#t#re
ti(es.
The
fo&&o+in
s$ript
$a&$#&ates
the
ti(e
one
+ee,
200 of 233 25-02-2014 12:21
after
the
$#rrent
ti(e:
BEGIN {
# get current time
ts = systime();
# the time is in seconds, so
one_day = 24 * 60 * 60;
next_week = ts + (7 * one_day);
format = "%a %b %e %H:%M:%S %Z %Y";
print strftime(format, next_week);
exit;
}
C&i$,
here
to
et
fi&e:
one9week9later;gawk
User !efine$ *unctions
3ina&&y!
9A28
and
%A28
s#pport
#ser
defined
f#n$tions.
This
f#n$tion
de(onstrates
a
+ay
to
print
error
(essaes!
in$&#din
the
fi&ena(e
and
&ine
n#(-er!
if
appropriate:
#!/usr/bin/nawk -f
{
if (NF != 4) {
error("Expected 4 fields");
} else {
print;
}
}
function error ( message ) {
if (FILENAME != "-") {
printf("%s: ", FILENAME) > "/dev/tty";
201 of 233 25-02-2014 12:21
}
printf("line # %d, %s, line: %s\n", NR, message, $0) >> "/dev/tty";
}
C&i$,
here
to
et
fi&e:
awk9exa"le:D;nawk
AWK atterns
7n
(y
first
t#toria&
on
A28!
7
des$ri-ed
the
A28
state(ent
as
ha)in
the
for(
pattern
D$o((andsE
7
ha)e
on&y
#sed
t+o
patterns
so
far:
the
spe$ia&
+ords
BEGIN
and
EN(.
/ther
patterns
are
possi-&e!
yet
7
ha)en't
#sed
any.
There
are
se)era&
reasons
for
this.
The
first
is
202 of 233 25-02-2014 12:21
that
these
patterns
aren't
ne$essary.
*o#
$an
d#p&i$ate
the(
#sin
an
if
state(ent.
Therefore
this
is
an
Aad)an$ed
feat#re.A
Patterns!
or
perhaps
the
-etter
+ord
is
$onditions!
tend
to
(a,e
an
A28
prora(
o-s$#re
to
a
-einner.
*o#
$an
thin,
of
the(
as
an
ad)an$ed
topi$!
one
that
sho#&d
-e
atte(pted
after
-e$o(in
fa(i&iar
+ith
the
-asi$s.
A
pattern
or
$ondition
is
si(p&y
an
203 of 233 25-02-2014 12:21
a--re)iated
test.
7f
the
$ondition
is
tr#e!
the
a$tion
is
perfor(ed.
A&&
re&ationa&
tests
$an
-e
#sed
as
a
pattern.
The
Ahead
;10A
$o((and!
+hi$h
prints
the
first
10
&ines
and
stops!
$an
-e
d#p&i$ated
+ith
Dif
<9B
UM
10
=
DprintEE
Chanin
the
if
state(ent
to
a
$ondition
shortens
the
$ode:
9B
UM
10
DprintE
"esides
re&ationa&
tests!
yo#
$an
a&so
204 of 233 25-02-2014 12:21
#se
$ontain(ent
tests!
i.
e.
do
strins
$ontain
re#&ar
e5pressions@
Printin
a&&
&ines
that
$ontain
the
+ord
Aspe$ia&A
$an
-e
+ritten
as
Dif
<H0
X
Kspe$ia&K=
DprintEE
or
(ore
-rief&y
H0
X
Kspe$ia&K
DprintE
This
type
of
test
is
so
$o((on!
the
a#thors
of
A28
a&&o+
a
third!
shorter
for(at:
Kspe$ia&K
DprintE
These
tests
$an
-e
$o(-ined
+ith
the
A9F
205 of 233 25-02-2014 12:21
<))=
and
/B
<**=
$o((ands!
as
+e&&
as
the
9/T
<+=
operator.
Parenthesis
$an
a&so
-e
added
if
yo#
are
in
do#-t!
or
to
(a,e
yo#r
intention
$&ear.
The
fo&&o+in
$ondition
prints
the
&ine
if
it
$ontains
the
+ord
A+ho&eA
or
$o&#(ns
1
and
2
$ontain
Apart1A
and
Apart2A
respe$ti)e&y.
<H0
X
K+ho&eK=
NN
<<H1
X
Kpart1K=
>>
<H2
X
Kpart2K==
DprintE
206 of 233 25-02-2014 12:21
This
$an
-e
shortened
to
K+ho&eK
NN
H1
X
Kpart1K
>>
H2
X
Kpart2K
DprintE
There
is
one
$ase
+here
addin
parenthesis
h#rts.
The
$ondition
K+ho&eK
DprintE
+or,s!
-#t
<K+ho&eK=
DprintE
does
not.
7f
parenthesis
are
#sed!
it
is
ne$essary
to
e5p&i$it&y
spe$ify
the
test:
<H0
X
K+ho&e=
DprintE
A
(#r,y
sit#ation
arises
+hen
a
si(p&e
)aria-&e
is
207 of 233 25-02-2014 12:21
#sed
as
a
$ondition.
0in$e
the
)aria-&e
N,
spe$ifies
the
n#(-er
of
fie&ds
on
a
&ine!
one
(iht
thin,
the
state(ent
93
DprintE
+o#&d
print
a&&
&ines
+ith
one
of
(ore
fie&ds.
This
is
an
i&&ea&
$o((and
for
A28!
-e$a#se
A28
does
not
a$$ept
)aria-&es
as
$onditions.
To
pre)ent
a
synta5
error!
7
had
to
$hane
it
to
93
PM
0
DprintE
208 of 233 25-02-2014 12:21
7
e5pe$ted
9A28
to
+or,!
-#t
on
so(e
0#n/0
syste(s
it
ref#sed
to
print
any
&ines
at
a&&.
/n
ne+er
0o&aris
syste(s
it
did
-eha)e
proper&y.
Aain!
$hanin
it
to
the
&oner
for(
+or,ed
for
a&&
)ariations.
%A28!
&i,e
the
ne+er
)ersion
of
9A28!
+or,ed
proper&y.
After
this
e5perien$e!
7
de$ided
to
&ea)e
other!
e5oti$
)ariations
a&one.
C&ear&y
this
is
#ne5p&ored
territory.
7
$o#&d
209 of 233 25-02-2014 12:21
+rite
a
s$ript
that
prints
the
first
20
&ines!
e5$ept
if
there
+ere
e5a$t&y
three
fie&ds!
#n&ess
it
+as
&ine
10!
-y
#sin
93
MM
3
@
9B
MM
10
:
9B
U
20
D
print
E
"#t
7
+on't.
/-s$#rity!
&i,e
p#ns!
is
often
#nappre$iated.
There
is
one
(ore
$o((on
and
#sef#&
pattern
7
ha)e
not
yet
des$ri-ed.
7t
is
the
210 of 233 25-02-2014 12:21
$o((a
separated
pattern.
A
$o((on
e5a(p&e
has
the
for(:
Kstart
K!KstopK
DprintE
This
for(
defines!
in
one
&ine!
the
$ondition
to
t#rn
the
a$tion
on!
and
the
$ondition
to
t#rn
the
a$tion
off.
That
is!
+hen
a
&ine
$ontainin
AstartA
is
seen!
it
is
printed.
')ery
&ine
after+ards
is
a&so
printed!
#nti&
a
&ine
$ontainin
AstopA
is
seen.
This
one
is
a&so
printed!
211 of 233 25-02-2014 12:21
-#t
the
&ine
after!
and
a&&
fo&&o+in
&ines!
are
not
printed.
This
trierin
on
and
off
$an
-e
repeated
(any
ti(es.
The
eJ#i)a&ent
$ode!
#sin
the
if
$o((and!
is:
{
if ($0 ~ /start/) {
triggered=1;
}
if (triggered) {
print;
if ($0 ~ /stop/) {
triggered=0;
}
}
}
The
$onditions
do
not
ha)e
to
-e
re#&ar
e5pressions.
Be&ationa&
tests
$an
a&so
-e
#sed.
The
fo&&o+in
prints
a&&
&ines
-et+een
20
and
40:
212 of 233 25-02-2014 12:21
<9BMM20=!
<9BMM40=
DprintE
*o#
$an
(i5
re&ationa&
and
$ontain(ent
tests.
The
fo&&o+in
prints
e)ery
&ine
#nti&
a
AstopA
is
seen:
<9BMM1=!KstopK
DprintE
There
is
one
(ore
area
of
$onf#sion
a-o#t
patterns:
ea$h
one
is
independent
of
the
others.
*o#
$an
ha)e
se)era&
patterns
in
a
s$riptW
none
inf&#en$e
the
other
patterns.
7f
the
fo&&o+in
s$ript
is
e5e$#ted:
9BMM10
DprintE
<9BMM5=!
<9BMM15=
DprintE
213 of 233 25-02-2014 12:21
K555K
DprintE
<9BMM1=!K9eLerMat$hThi0K
DprintE
and
the
inp#t
fi&e's
&ine
10
$ontains
A555!A
it
+o#&d
-e
printed
4
ti(es!
as
ea$h
$ondition
is
tr#e.
*o#
$an
thin,
of
ea$h
$ondition
as
$#(#&ati)e.
The
e5$eption
is
the
spe$ia&
BE5+(
and
E(!
$onditions.
7n
the
oriina&
A28!
yo#
$an
on&y
ha)e
one
of
ea$h.
7n
9A28
and
%A28!
yo#
$an
ha)e
se)era&
"'%79
or
'9F
a$tions.
214 of 233 25-02-2014 12:21
*or"atting AWK rogra"s
Many
readers
ha)e
J#estions
(y
sty&e
of
A28
prora((in.
7n
parti$#&ar!
they
as,
(e
+hy
7
in$&#de
$ode
&i,e
this:
O
Print
$o&#(n
3
print
H3W
+hen
7
$o#&d
#se
print
H3
O
print
$o&#(n
3
After
a&&!
they
reason!
the
se(i$o&on
is
#nne$essary!
and
$o((ents
do
not
ha)e
to
start
on
the
first
$o&#(n.
This
is
tr#e.
215 of 233 25-02-2014 12:21
0ti&&
7
a)oid
this.
*ears
ao!
+hen
7
started
+ritin
A28
prora(s!
7
+o#&d
find
(yse&f
$onf#sed
+hen
the
nestin
of
$onditions
+ere
too
deep.
7f
7
(o)ed
a
$o(p&e5
if
state(ent
inside
another
if
state(ent!
(y
a&in(ent
of
-ra$es
-e$a(e
in$orre$t.
7t
$an
-e
)ery
diffi$#&t
to
repair
this
$ondition!
espe$ia&&y
+ith
&are
s$ripts.
9o+adays
7
#se
emacs
to
do
the
for(attin
for
(e!
216 of 233 25-02-2014 12:21
-#t
10
years
ao
7
didn't
ha)e
this
option.
My
so&#tion
+as
to
#se
the
prora(
cb!
+hi$h
is
a
AC
-ea#tifier.A
"y
in$&#din
optiona&
se(i$o&ons!
and
startin
$o((ents
on
the
first
$o&#(n
of
the
&ine!
7
$o#&d
send
(y
A28
s$ript
thro#h
this
fi&ter!
and
proper&y
a&in
a&&
of
the
$ode.
En-iron"ent #ariables
7')e
des$ri-ed
the
4
spe$ia&
)aria-&es
in
A28!
and
217 of 233 25-02-2014 12:21
-rief&y
(entioned
so(e
others
9A28
and
%A28.
The
$o(p&ete
&ist
fo&&o+s:
AWK Table :?
Secial #ariables
#ariable /urose AWK (AWK 5AWK
*S
3ie&d
separator
*es *es *es
(*
9#(-er of
3ie&ds
*es *es *es
&S
Be$ord
separator
*es *es *es
(&
9#(-er of
inp#t
re$ords
*es *es *es
*+,E(AME
C#rrent
fi&ena(e
*es *es *es
%*S
/#tp#t fie&d
separator
*es *es *es
%&S
/#tp#t
re$ord
separator
*es *es *es
A&5C
O of
ar#(ents
*es *es
A&5#
Array of
ar#(ents
*es *es
A&5+(!
7nde5 of
AB%L of
$#rrent fi&e
*es
*(&
7np#t re$ord
n#(-er
*es *es
%*MT
/#p#t
for(at
<defa#&t
AT.6A=
*es *es
&STA&T
7nde5 of
first
$hara$ter
after
(at$h<=
*es *es
&,E(5T8
1enth of
strin after
(at$h<=
*es *es
SUBSE/
Fefa#&t
separator
+ith
(#&tip&e
s#-s$ripts in
array
<defa#&t
AG034A=
*es *es
218 of 233 25-02-2014 12:21
E(#+&%(
Array of
en)iron(ent
)aria-&es
*es
+5(%&ECASE
7nore $ase
of re#&ar
e5pression
*es
C%(#*MT
$on)ersion
for(at
<defa#&t:
AT.6A=
*es
E&&(%
C#rrent
error after
et&ine
fai&#re
*es
*+E,!W+!T8S
&ist of fie&d
+idths
<instead of
#sin 30=
*es
"79M/F'
"inary Mode
<2indo+s=
*es
179T
T#rns ;;&int
(ode onKoff
*es
PB/C793/
Array of
infor(aiton
a-o#t
$#rrent A28
prora(
*es
BT
Be$ord
ter(inator
*es
T':TF/MA79
Te5t do(ain
<i.e.
&o$a&iCation=
of $#rrent
A28
prora(
*es
0in$e
7')e
a&ready
dis$#ssed
(any
of
these!
7'&&
on&y
$o)er
those
that
7
(issed
ear&ier.
A&5C ) (u"ber or argu"ents 6(AWK'5AW
The
)aria-&e
A&5C
spe$ifies
the
n#(-er
of
219 of 233 25-02-2014 12:21
ar#(ents
on
the
$o((and
&ine.
7t
a&+ays
has
the
)a&#e
of
one
or
(ore!
as
it
$o#nts
its
o+n
prora(
na(e
as
the
first
ar#(ent.
7f
yo#
spe$ify
an
A28
fi&ena(e
#sin
the
A;fA
option!
it
is
not
$o#nted
as
an
ar#(ent.
7f
yo#
in$&#de
a
)aria-&e
on
the
$o((and
&ine
#sin
the
for(
-e&o+!
9A28
does
not
$o#nt
as
an
ar#(ent.
na+,
;f
220 of 233 25-02-2014 12:21
fi&e.a+,
5M14
%A28
does!
-#t
that
is
-e$a#se
%A28
reJ#ires
the
A;)A
option
-efore
ea$h
assin(ent:
a+,
;f
fi&e.a+,
;)
5M14
%A28
)aria-&es
initia&iCed
this
+ay
do
not
affe$t
the
A&5C
)aria-&e.
A&5# ) Array of argu"ents 6(AWK'5AWK
The
A&5#
array
is
the
&ist
of
ar#(ents
<or
fi&es=
passed
as
$o((and
&ine
ar#(ents.
A&5+(! ) Argu"ent +n$ex 65AWK only7
The
A&5+(!
spe$ifies
the
$#rrent
inde5
into
221 of 233 25-02-2014 12:21
the
A&5#
array!
and
therefore
A&5#HA&5+(!I
is
a&+ays
the
$#rrent
fi&ena(e.
Therefore
371'9AM'
MM
AB%LYAB%79FZ
is
a&+ays
tr#e.
7t
$an
-e
(odified!
a&&o+in
yo#
to
s,ip
o)er
fi&es!
et$.
*(& 6(AWK'5AWK7
The
*(&
)aria-&e
$ontains
the
n#(-er
of
&ines
read!
-#t
is
reset
for
ea$h
fi&e
read.
The
(&
)aria-&e
a$$#(#&ates
for
a&&
fi&es
read.
Therefore
if
yo#
e5e$#te
an
a+,
222 of 233 25-02-2014 12:21
s$ript
+ith
t+o
fi&es
as
ar#(ents!
+ith
ea$h
$ontainin
10
&ines:
na+,
'Dprint
9BE'
fi&e
fi&e2
na+,
'Dprint
39BE'
fi&e
fi&e2
the
first
prora(
+o#&d
print
the
n#(-ers
1
thro#h
20!
+hi&e
the
se$ond
+o#&d
print
the
n#(-ers
1
thro#h
10
t+i$e!
on$e
for
ea$h
fi&e.
%*MT 6(AWK'5AWK7
The
%*MT
)aria-&e
0pe$ifies
the
defa#&t
for(at
for
n#(-ers.
The
defa#&t
)a&#e
is
223 of 233 25-02-2014 12:21
AT.6.A
&STA&T3 &,E(5T8 an$ "atch 6(AWK'5AW
7')e
a&ready
(entioned
the
&STA&T
and
&,E(5T8
)aria-&es.
After
the
match<=
f#n$tion
is
$a&&ed!
these
)aria-&es
$ontain
the
&o$ation
in
the
strin
of
the
sear$h
pattern.
&,E(5T8
$ontains
the
&enth
of
this
(at$h.
SUBSE/ ) Multi)$i"ensional array searator 6(AW
'ar&ier
7
des$ri-ed
ho+
yo#
$an
$onstr#$t
(#&ti;
di(ensiona&
arrays
in
A28.
These
are
$onstr#$ted
-y
$on$atenatin
t+o
inde5es
toether
+ith
a
spe$ia&
224 of 233 25-02-2014 12:21
$hara$ter
-et+een
the(.
7f
7
#se
an
a(persand
as
the
spe$ia&
$hara$ter!
7
$an
a$$ess
the
)a&#e
at
&o$ation
:!
*
-y
the
referen$e
arrayY
:
A>A
*
Z
9A28
<and
%A28=
has
this
feat#re
-#i&t
in.
That
is!
yo#
$an
spe$ify
the
array
e&e(ent
arrayY:!*Z
7t
a#to(ati$a&&y
$onstr#$ts
the
strin!
p&a$in
a
spe$ia&
$hara$ter
-et+een
the
inde5es.
This
$hara$ter
is
the
225 of 233 25-02-2014 12:21
non;printin
$hara$ter
A034.A
*o#
$an
$ontro&
the
)a&#e
of
this
$hara$ter!
to
(a,e
s#re
yo#r
strins
do
not
$ontain
the
sa(e
$hara$ter.
E(#+&%( ) en-iron"ent -ariables 65AWK on
The
E(#+&%(
array
$ontains
the
en)iron(ent
)aria-&es
the
$#rrent
pro$ess.
*o#
$an
print
yo#r
$#rrent
sear$h
path
#sin
print
'9L7B/9YAPAT?AZ
+5(%&ECASE 65AWK only7
The
+5(%&ECASE
)aria-&e
is
nor(a&&y
Cero.
7f
yo#
set
it
to
non;Cero!
then
a&&
226 of 233 25-02-2014 12:21
pattern
(at$hes
inore
$ase.
Therefore
the
fo&&o+in
is
eJ#i)a&ent
to
Arep
;i
(at$h:A
"'%79
D7%9/B'CA0'M1WE
K(at$hK
DprintE
C%(#*MT ) con-ersion for"at 65AWK only
The
C%(#*MT
)aria-&e
is
#sed
to
spe$ify
the
for(at
+hen
$on)ertin
a
n#(-er
to
a
strin.
The
defa#&t
)a&#e
is
AT.6.A
/ne
+ay
to
tr#n$ate
inteers
is
to
$on)ert
an
inteer
to
a
strin!
and
$on)ert
the
strin
to
an
inteer
;
(odifyin
227 of 233 25-02-2014 12:21
the
a$tions
+ith
C%(#*MT:
a
M
12W
-
M
a
AAW
C/9L3MT
M
AT2.2fAW
$
M
a
AAW
Laria-&es
b
and
c
are
-oth
strins!
-#t
the
first
one
+i&&
ha)e
the
)a&#e
A12.00A
+hi&e
the
se$ond
+i&&
ha)e
the
)a&#e
A12.A
E&&(% ) syste" errors 65AWK only7
The
E&&(%
)aria-&e
des$ri-es
the
error!
as
a
strin!
after
a
$a&&
to
the
getline
$o((and
fai&s.
228 of 233 25-02-2014 12:21
*+E,!W+!T8S ) fixe$ wi$th fiel$s 65AWK on
The
*+E,!W+!T8S
)aria-&e
is
#sed
+hen
pro$essin
fi5ed
+idth
inp#t.
7f
yo#
+anted
to
read
a
fi&e
that
had
3
$o&#(ns
of
dataW
the
first
one
is
5
$hara$ters
+ide!
the
se$ond
4!
and
the
third
4!
yo#
$o#&d
#se
substr
to
sp&it
the
&ine
apart.
The
te$hniJ#e!
#sin
*+E,!W+!T8S!
+o#&d
-e:
"'%79
D37'1F27FT?0MA5
4
4AWE
D
printf<AThe
three
fie&ds
229 of 233 25-02-2014 12:21
are
Ts
Ts
TsGnA!
H1!
H2!
H3=WE
AWK3 (AWK3 5AWK3 or /E&,
This
$on$&#des
(y
series
of
t#toria&s
on
A28
and
the
9A28
and
%A28
)ariants.
7
oriina&&y
intended
to
a)oid
e5tensi)e
$o)erae
of
9A28
and
%A28.
7
a(
a
P'B1
#ser
(yse&f!
and
sti&&
-e&ie)e
P'B1
is
+orth
&earnin!
yet
7
don't
intend
to
$o)er
this
)ery
$o(p&e5
&an#ae.
Understandin
A28
is
)ery
#sef#&
+hen
230 of 233 25-02-2014 12:21
yo#
start
to
&earn
P'B1!
and
a
part
of
(e
fe&t
that
tea$hin
yo#
e5tensi)e
9A28
and
%A28
feat#res
(iht
en$o#rae
yo#
to
-ypass
P'B1.
7
fo#nd
(yse&f
oin
into
(ore
depth
than
7
p&anned!
and
7
hope
yo#
fo#nd
this
#sef#&.
7
fo#nd
o#t
a
&ot
(yse&f!
espe$ia&&y
+hen
7
dis$#ssed
topi$s
not
$o)ered
in
other
-oo,s.
2hi$h
re(inds
(e
of
so(e
$&osin
231 of 233 25-02-2014 12:21
ad)i$e:
if
yo#
don't
#nderstand
ho+
so(ethin
+or,s!
e5peri(ent
and
see
+hat
yo#
$an
dis$o)er.
%ood
&#$,!
and
happy
A28in...
/ther
of
(y
Uni5
she&&
t#toria&s
$an
-e
fo#nd
here;
/ther
she&&
t#toria&s
$an
-e
fo#nd
at
8einer.s
S8E,,$ora$o
and
Chris
*;
A;
2ohnson.s
Unix
Shell
/age
7'd
&i,e
to
than,
the
fo&&o+in
for
feed-a$,:
Ranjit Singh
Richard Janis Beckert
Christian Haarmann
Kuang He
Guenter Knauf
Chay Wesley
Todd Andrews
Lynn Young
232 of 233 25-02-2014 12:21
Chris Hall @_cjh
Erik Streb del Toro
-his
document
was
translated
by
troff%html
v"%.
on
'eptember
%%/
%.
and
then
manually
edited
to
make
it
compliant
with0
233 of 233 25-02-2014 12:21

Awk - A Tutorial and Introduction - by Bruce Barnett

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Awk - A Tutorial and Introduction - by Bruce Barnett

Uploaded by

Copyright:

Available Formats

Mon Jan 20 15:36:59 UTC 2014

You might also like