You are on page 1of 92

Linux Introduction

Requirements
Any Linux terminal application SecureCRT,
Absolute Telnet, Reflection, etc.
Access to Datalex VPN (F5 Edge)

What is an O/S

A layer of
abstraction
between the HW
and SW
A resource
coordinator
Virtual machine
Reactive system

Operating Systems

Definition

A collection of programs that integrate the hardware resources


of the computer and make those resources available to the
user in a productive, timely, and efficient manner

General Orientation

A Typical Terminal Session

Log in to identify yourself and gain access

Execute commands to do work

Log off to terminate your connection

Logging In and Out


Log in

LOGIN:
PASSWORD:

Do work

$ date
Mon Jul 30 09:56:10 EDT
2007

$ other commands

Log out
$ <CTRL> D
$ exit
$ logout

The ssh Command

Secure Shell (ssh)

Can only be used on LINUX servers and will bring you to the
current users shell (i.e. top user)

Syntax:

ssh <hostname>

ssh <userid>@<hostname>

Example:

ssh mh-p-tdp99

ssh top@cp-p-ui8

The Linux Shell

Started during login process

Command interpreter

Most common user interface

Normally executes user-written


and system programs

Called shell in Linux because it


surrounds the kernel

In Linux, users can write their own


shell

Command Line Format

Syntax
$ command [-options] [arguments]

Examples
$ date
Mon Jul 30 10:21:30 EDT 2007
$ echo hello
hello
$ echoHI
- bash: echoHI: not found.
$ ls -l
backup.log mine

yours

The Manual

Reference Manual

Very useful for looking up command syntax

Not designed as a tutorial

Not very useful for learning Linux

Reference only

Syntax:
man [command]
e.g. man echo
man ls
man ssh

Displaying Date and Time

The date command

displays the system date and time

Syntax: date

Sample Output

Mon Jul 30 04:35:27 EDT 2007

Clearing the Screen

The clear command

Clears the screen

Syntax: clear

Switching to another user

The su command

Switch to another user

Syntax: sudo su - <username>

Note: sudo is defined in root level, to grant access


to execute a certain action without having to type
the password root / owners password.

e.g. sudo su appuser


sudo su root

Navigating the File


System

What is a file system?

cabinet

= file system

drawer= directory

folder = directory

report = file

The Tree Structure


Cabinet

finance.dr
w

engineering.dr
w
assmbly.1

assmbly.2

feb.drw

jan.drw

week1

personnel.drw

week2

week3

march.drw

week1

= directory
= file

week2

The File System Hierarchy

The File System Hierarchy


contd
Static Files

Dynamic Files

/opt

/home

/usr/bin

/etc

/usr/sbin

/stand/vmunix

/usr/lib

/tmp

/usr/share

/dev

/usr/share/man

/mnt

/usr/local/bin

/var/mail

/usr/contrib/bin

/var/news

/sbin

/var/tmp

Path Names

name by which a file is


identified

always written as a
string of names
separated by slashes

Absolute:
1. /home/user3/f1
2. /home/user3/memo
3. /home/user3/memo/f1

Relative to /home/user3
1. f1
2. memo
3. memo/f1

4. /home/user1/f1

Relative to /home/user1
4. f1

Types of Pathnames
1.

2.

Full Pathname / Absolute Pathname

Gives the complete designation

Always starts at the top of the hierarchy (the root)

Always starts with a /

Not dependent on your current location

Always is unique across the entire hierarchy

Relative Pathname

Always starts at your current location in the hierarchy

Never starts with a /

unique relative to your current location only

Often shorter than the absolute path name

Some Special Directories


Dot (.) Represents your current directory

Example: current directory is /home


./user2/f1

represents

Dot Dot (..) Represents the directory immediately above


your current directory

Example: current directory is /home

/tmp

represents /
represents /tmp

Basic File System Commands

pwd

ls l or ll

Finds files

mkdir

Changes your location in the hierarchy to another directory

find

Sees what files and directories are under the current directory

cd

Displays the directory name of your current location in the directory

Creates a directory

rmdir

Removes a directory

The pwd Command

Syntax: pwd

print name of current/working directory

The cd Command

Syntax: cd [dir_pathname]

Note: When executed with no arguments, cd


command will return you to your home
directory

The ls Command

Syntax: ls [-adlFR] [pathname(s)]

-a List all files including those names which start with a dot (.) or
which are hidden

-d Lists characteristics of the directory, instead of the contents of the


directory

-l Provides a long listing that describes the attributes about each file,
including the type mode, number of links, owner, group, size (in bytes),
the modification date, and the name

-F Appends a slash (/) to each listed file that is a directory and an


asterisk (*) to each listed file that is executable

-R Recursively lists files in the given directory and in all subdirectories

-lh displays the files inside the given directory with their file in human
readable form.

Linux Shorthand
Commands
ls l or ll
ls ltrh or ll -trh

The mkdir and rmdir


Commands

Syntax: mkdir [-p] dir_pathname(s)


dir_pathname(s)

rmdir

-p creates intermediate directories if they do not already


exist

Examples:

mkdir fruit

mkdir fruit/apple fruit/grape

rmdir fruit/apple fruit/grape

rmdir fruit

mkdir p fruit/apple fruit/grape /home/Alex/hello/test

The find command


Syntax

The find command syntax is:

find {/where/to/look/up} criteria action

OR

find /dir/path/look/up criteria action

OR

find /dir/path/look/up -name "dir-name-here

e.g.
find / -iname matrixtdp.log

Exercise
1.

Login to top-t-tdp1.datalex.com machine

2.

Switch to appuser user

3.

Find the exercise directory user xels home directory.

4.

Go inside the directory

5.

Verify that you are in the right directory.

6.

List all files inside the exercise directory

7.

Create a directory named "mammals

8.

Remove the directory you created.

Answers
1.

Login to top-t-tdp1.datalex.com machine open absolute telnet > new >


connect

2.

Switch to appuser user sudo su - appuser

3.

Find the exercise directory user xels home directory. find / -iname
exercise

4.

Go inside the directory Verify that you are in the right directory. cd
/home/xel/exercise

5.

List all files inside the exercise directory ll or ls l

6.

Create a directory named "mammals mkdir mammals or mkdir p


/home/xel/exercise/mammals

7.

Remove the directory you created. rmdir mammals

Managing Files

What is a File?

A container of data or a link to a device.

Every file has a name and may hold data that resides on a
disk

There are several different types of files

Regular files

text, data, drawings

executables programs

Directories

Device files

File Characteristics

Managing file systems


commands

cat

less

head

tail

cp

mv

rm

zip/gzip

tar

touch

The cat Command

Concatenate and display the contents of file (s)

Syntax: cat [file]

Example:

$ cat file1

The less Command

Modified more command page up and down enabled as


well as search (by using /)

Syntax: less [filename]

Example:

$ less file1

Note: To see the next screen of text, press the


SPACE key. To see the next line, press the
ENTER key

The head Command

Displays the start of the file

Syntax: head [-n] [filename]

Example:

$ head -20 file1

Note: default number of row is 10 if not


defined

The tail Command

Display the end of the file

Syntax: tail [-nf] [filename]

f or F:

output appended data as the file grows

Example:

$ tail -20 file1

$ tail f /datalex/logs/tdp/jboss/matrixtdp.log

Note: default number of row is 10 if not


defined

The cp Command

Copies files

Syntax:

cp [-i] file1 new_file

Copy a file

cp [-i] file [file] dest_dir

cp r [-i] dir [dir] dest_dir Copy directories

Copy files to a directory

-i

will warn you if the destination file exists, and


require you to verify that the file should be copied
over

Note: A directory can be copied using the r (recursive)


option. When copying multiple files, the destination must
be a directory

The mv Command

Moves or renames files

Syntax:

mv [-i] file1 new_file

Rename a file

mv [-i] file [file] dest_dir Move files to a directory

mv [-i] dir [dir] dest_dir

Rename or move

-i will warn you if the destination file exists, and


require you to verify that the file should be
overwritten

The g(un)zip Command

Compress or Uncompress Files

gzip Compress a file


Syntax:
gzip filename
Example:
$ gzip testfile

gunzip Uncompress a
file
Syntax:
gunzip filename.gz
Example:
$ gunzip testfile.gz

The tar Command

Compress or Uncompress Directory

Syntax:
tar cvzf tarball
directory
Example:
$ tar cvzf dir.tar.gz
dir
c = create

Syntax:
tar xvzf tarball
Example:
$ tar xvzf dir.tar.gz
x = xtract

Summary
What can we do with a file?

ls
cat
less
cp
mv
mv
rm

look at the characteristics of a file


look at the contents of a file
look at the contents of a file, one screen at a time
make a copy of a file
change the name of a file or directory
move a file to another directory
remove a file

Exercise
1.

Cat information of flamingo

2.

Using less - get information of peacock

3.

Using less - go to the last line of peacock

4.

Display the first line of ostrich

5.

Display last 2 lines of ostrich (a line space counts as 1 line).

6.

Copy peacock and name it as peafowl

7.

Copy tuna outside its directory

8.

Move turtle from the amphibians directory to the reptile directory

9.

Get the Information for tortoise by searching through the turtle file.

10.

Rename lizard to snake

11.

Zip the peafowl file

12.

Unzip the peafowl file

13.

Remove the peafowl file

14.

Create an empty file in the mammals directory named dog

15.

Create a folder named animals under the exercise directory.

16.

Check what is the file size of the tarball created.

17.

Create a tarball of the whole mammals directory and move it inside the animals directory

Answers
1.

Cat information of flamingo Ans: cat flamingo

2.

Using less - get information of peacock Ans: less peacock

3.

Using less - go to the last line of peacock Ans: less peacock, inside less: Shift+G

4.

Display the first line of ostrich Ans: head -1 ostrich

5.

Display last 2 lines of ostrich (a line space counts as 1 line). Ans: tail -2 ostrich

6.

Copy peacock and name it as peafowl Ans: (inside birds directory) cp peacock peafowl

7.

Copy tuna outside its directory Ans: (inside fish directory) cd tuna ..

8.

Move turtle from the amphibians directory to the reptile directory Ans: (inside amphibians directory) cp turtle

9.

Get the Information for tortoise by searching through the turtle file. Ans: use less command (and / search tool)

10.

Rename lizard to snake Ans: (inside reptiles dir) mv lizard snake

11.

Zip the peafowl file Ans: (inside birds dir) gzip peafowl

12.

Unzip the peafowl file Ans: (inside birds dir) gunzip peafowl.gz

13.

Remove the peafowl file Ans: (inside birds dir) rm i peafowl

14.

Create an empty file in the mammals directory named dog Ans: (inside mammals dir) touch dog

15.

Create a folder named animals under the exercise directory. Ans: (inside exercise dir) mkdir animals

16.

Create a tarball of the whole mammals directory Ans: tar cvzf mammals.tar.gz mammals

17.

Move the tarball inside the animals directory Ans: (inside exercise dir) mv mammals.tar.gz animals/

18.

Check what is the file size of the tarball created. Ans: (inside anmials dir) ll h mammals.tar.gz

Vim Editor

What is Vim?

VIsual IMprovied editor

A screen-oriented text editor

Included with most Linux system distributions

Command driven

Categories of commands include

General administration

Cursor movement

Insert, Delete, Modify, Paste text

Starting a vim Session

Syntax:

vim [filename] Start a vim edit session of file

Examples:

$ vim funfile

All modifications are made to the copy of the file brought to


the memory

Modes

Command Mode

Last Line Mode

keystrokes interpreted as commands


ex and search inputs

Input Mode

vi input with keystrokes entered into the file

Ending a vim Session

ESC :wq!

save and quit

ESC :q!

quit without saving

Cursor Control Commands


Move
Move
Move
Move

left, one space = h


right, one space = l
down, one line = j
up, one line
= k

Forward, one screen =


Backward, one screen =
Up, half screen
=
Down, half screen
=

Ctrl f
Ctrl b
Ctrl u
Ctrl d

To go to a specific line = :## (where ## is the line number)


To go to the end of file = :$

Cursor Control Commands


contd
Go
Go
Go
H
Go
M
Go

to line n = :n
to end of line = G
to first line of the screen =
to middle of the screen =
to last line of screen = L

Move one word forward = W


Move one word backward = b
Move to the end of the word =
e
Go to the next sentence = )
Go to the previous sentence =
(
Go to the next paragraph = }
Go to the previous paragraph

Input Mode
Insert at the
beginning of the line = I

Open a line above the line


the cursor is on, ready
for inserting = O

Insert after the cursor = a

The quick | fox jumps over the


fence.
Insert before
the cursor = i

Open a line below the line


the cursor is on, ready
for inserting = o

Insert at the
end of the line = A

Deleting Text
Delete a no. of characters
to the left of the cursor = nX

Delete the whole line = dd

Delete a no. of characters


to the right of cursor = nx
Delete a no. of lines = ndd
Undo = u

Exercise
1.

Edit the salamander file under the amphibians directory


and write the following and save the file
Salamander is a common name of approximately 500 species of
amphibians.
They are typically characterized by a superficially lizard-like
appearance, with their slender bodies, short noses, and long tails

2.

Remove the second line.

3.

Undo the delete performed

4.

Save and exit from the file.

Answers:
Edit the salamander file under the amphibians directory and write the following and
save the file

1.

Salamander is a common name of approximately 500 species of amphibians.


They are typically characterized by a superficially lizard-like appearance, with their slender
bodies, short noses, and long tails
Ans:

vim salamander

Paste the following above.

Press [ESC] key

Type :wq! And press [ENTER]

2.

Remove the second line. Ans: Go to the 2nd line, type dd

3.

Undo the delete performed Ans: press u

4.

Save and exit from the file. Ans: press [ESC] key, type :wq!

Input Output
Redirection

Introduction

Input redirection

Allows you to get the input for a command from some file other
than the keyboard

Output redirection

Allows you to send the output of a command to some file other


than your terminal

stdin, stdout, and stderr

Input Redirection - <

Any command that reads its input from stdin can have its
input redirected to come from another file

Example:

$ cat < testfile

Note: Input redirection causes no change to the contents of the


input file

Output Redirection - > and


>>

Any command that reads its input from stdout can have its
output redirected to go to another file

Example:

$ cat f1> testfile

$ cat f1 >> testfile

e.g. tail F matrixtdp.log > new.log

Error Redirection 2> and


2>>

Any command that produces error messages to stderr can


have those messages redirected to another file

Example:

$ cp 2> testfile

$ cp 2>> testfile

Create/Overwrite
Create/Append

Exercise
1.

Read the contents of tuna and place the result inside the
isda file.

2.

Append the contents of tuyo to the isda file

3.

On the exercise directory catch the error of the command


cat something and put it inside the errors file.

Answers
1.

Read the contents of tuna and place the result inside the isda file. Ans: (inside
/home/xel/exercise/fish directory): cat tuna > isda

2.

Append the contents of tuyo to the isda file Ans: cat tuyo >> isda

3.

On the exercise directory catch the error of the command cat something and put it
inside the errors file.
Ans: cat something 2> errors

Filters

What Is a Filter?

Reads standard input and produces standard output

Filters the contents of the input stream or a file

Sends results to screen, never modifies the input stream of


file

Processes the output of other commands when they are


used in conjunction with output redirection

Examples: grep, sort, wc

The wc Command

Counts lines, words and characters in a file

Syntax:

wc [-lwc] [file]

line

word

character

Example:

$ wc testfile

$ wc l testfile

The grep Command

Pattern Matching

Syntax:

grep [-cinv] [-e] pattern [-e pattern] [file]

grep [-cinv] f patterns_list_file [file]


-c

only a count of matching lines is printed

-i

ignore the case of the letters in the pattern

-n

displays line numbers to each line displayed

-v

displays the lines which do not contain the pattern

Example:

$ grep v test testfile

$ grep test testfile

Exercise
1.

How many lines does frog have?

2.

How many words does ostrich have?

3.

Search for the dogs in alligator

4.

Search for the word Chinese in alligator using chinese as the search value.

5.

Display the lines of alligator without American

6.

How many lines in alligator have crocodile?

Answers
1.

How many lines does frog have? Ans: wc l frog

2.

How many words does ostrich have? Ans: wc w ostrich

3.

Search for the dogs in alligator Ans: grep dogs alligator

4.

Search for the word Chinese in alligator using chinese as the search value. Ans:
grep i chinese alligator

5.

Display the lines of alligator without American Ans: grep v American alligator

6.

How many lines in alligator have crocodile?


Ans: grep c crocodile alligator

Process Control

The ps Command

Displays information about processes currently running or


sleeping on the system

Syntax:

ps [ -efl ]

Examples:

ps Shows processes associated with user session

ps e

Shows every process running in system

ps f

Shows full listing

ps l

Shows long listing

Monitoring Processes
ps short report (processes associated with your terminal session)
$ ps
PID TTY

TIME COMMAND

3844 pts/te

0:00 ps

3769 pts/te

0:00 telnetd

3770 pts/te

0:00 sh

ps -f full listing
$ ps -f
UID

PID

PPID C

STIME TTY

TIME COMMAND

bb7169 3845 3770 1 16:17:23 pts/te


root

3769 2340 0 16:06:42 pts/te

bb7169 3770 3769 0 16:06:44 pts/te

0:00 ps -f
0:00 telnetd -b /etc/issue
0:00 sh

ps -e selects ALL processes


ps -ef full listing of ALL processes
ps aux
top

shows all processes with much more detailed information

display top CPU processes

Understanding Some
Information about a Process

PID

the process ID.

PPID

COMMAND

PR

NI
nice value. This ranges from 19 (nicest) to -20 (not nice to
others),
see nice(1). (alias nice). Initially created upon process
startup.

VIRT/VSZ length of memory area. virtual memory usage, it can


probably be
best described as the app's used address space every library the
app uses, every data it creates, everything is
included here.

the parent process ID. The process that created PID (child)
dynamic process priority.

RES/RSS
memory. In a

the process name

resident memory usage, i.e. what's actually in the


way it could be probably used for measuring
real memory usage of the app

Understanding Some
Information about a Process

SHR
shared memory. Potentionally shared memory. I.e. memory that
may be used not only by this particular app but also by some else.
And actually it seems to be the shared part of RES - SHR goes down if
the app will be swapped out, at least with recent kernels

S/STAT State of the Process

%CPU cpu utilization of the process in "##.#" format. Currently, it is


the CPU time used divided by the time the process has been running
(cputime/realtime ratio), expressed as a percentage. It will not add up
to 100% unless you are lucky. (alias pcpu).

%MEM ratio of the process's resident set size to the physical memory
on the machine, expressed as a percentage. (alias pmem).

TIME Total CPU time the task has used since it started. If cumulative
mode is on, this also includes the CPU time used by the process's
children which have died. You can set cumulative mode with the S
command line option or toggle it with the interactive command S. The
header line will then be changed to CTIME.

Understanding Process States


Here are the different values that the s, stat and state output specifiers (header "STAT" or "S") will
display to describe the state of a process.

Uninterruptible sleep (usually IO)

Running or runnable (on run queue)

Interruptible sleep (waiting for an event to complete)

Stopped, either by a job control signal or because it is being traced.

paging (not valid since the 2.6.xx kernel)

dead (should never be seen)

Defunct ("zombie") process, terminated but not reaped by its parent.

For BSD formats and when the stat keyword is used, additional characters may be displayed:

<

high-priority (not nice to other users)

low-priority (nice to other users)

has pages locked into memory (for real-time and custom IO)

is a session leader

is multi-threaded (using CLONE_THREAD, like NPTL pthreads do)

is in the foreground process group

The kill Command

Terminates ANY command

Including nohup and background processes

Syntax:

kill [-s signal_name] PID [PID...]

kill [-s signal_name] %Job#

Examples:

$ kill 1234

$ kill -9 1234 or kill s 9 1234 Totally terminates the process


(DO NOT DO THIS!)

$ kill -3 1234 quits the process and generates thread


dump/java stack trace (for java processes only)

Pipes

Introduction

Pipe is a useful feature

Capability of to link commands together

Able to filter the output of a command

Why Use Pipelines?


$ who >
temp_file
$ wc l <
temp_file
$ rm temp_file

who | wc -l

The | Symbol

used for linking two commands together

Output (stdout) of the command to the left of the | will be


used as the standard input (stdin) for the command to the
right

Syntax:

cmd | cmd

Examples:

$ ls l | less

$ ps ef |less

$ cat myfile |tail -20 |head -5

The cut Command

allows you to cut out columns or fields of text from


standard input or a file then send result to stdout

Syntax:

cut clist [file]

Cuts columns

cut flist [-dchar][-s][file] Cuts fields delimited by char

Examples:

$ echo abcdefghijklmno |cut c6-10

$ date |cut -c5-10

$ ps -ef |cut -c1-9,48-50

$ cat /etc/passwd |grep live3|cut f1,6 d:

$ cut f1,6 d: /etc/passwd |grep live3

Exercise
1.

Login to top machine, switch to root

2.

How many files does the /etc directory have?

3.

Look for processes related to appuser

4.

From #3 answer, how many lines does the output have?

5.

From the hostname command display top as the output instead of top1-tdp1.

6.

Using the /etc/passwd file, display only the first field delimited by colon
how many are outputted?

7.

How many files in /etc have the filename sudoers

Answers
1.

Login to top machine, switch to root Ans: sudo su - root

2.

How many files does the /etc directory have? Ans: ll /etc | wc -l

3.

Look for processes related to appuser Ans: ps ef | grep appuser

4.

From #3 answer, how many lines does the output have? Ans: ps ef |
grep appuser | wc -l

5.

From the hostname command display top as the output instead of top1-tdp1. Ans: hostname | cut d. f1

6.

Using the /etc/passwd file, display only the first field delimited by colon
how many are outputted? Ans: cat /etc/passwd | cut d: -f1| wc l

7.

How many files in /etc have the filename sudoers


Ans: ll /etc | grep sudoers | wc l

Remote file transfer

Remote file transfer


Using the following commands:

scp

ftp

sftp

The scp command


scp secured shell (ssh) copy
Used for transferring files via ssh between machines

Syntax:

scp pr [source file] hostname:[destination]

Note: the [destination] should always be an absolute path


name.

Examples:

$ scp pr /home/xe/file1.txt cp-i-tdp1:/home/appuser/

$ scp pr file1.txt bel-i-tdp1:/datalex/logs/jboss/archive/

FTP and SFTP

FTP File Transfer Protocol

SFTP Secure File Transfer Protocol

Used to transfer files from one machine to another.

Syntax:
To access ftp server:

ftp [username]@[ftp server hostname]

To access sftp server:

sftp [username]@[sftp server hostname]

e.g. sftp sup-75123@sftp.datalex.com

FTP and SFTP


Commands:

put

mput

bye

Tools:

FileZilla

WinSCP

FTP / SFTP Commands:


In Terminal:

sftp user@sftp.datalex.com

Password: Created account pw.

You can then put your preferred log to the sftp server.
Type:

sftp> put <filename>

For multiple files:

sftp> mput <filename1> <filename2> .. <filenameN>

You can then download the file via FileZilla:

FTP / SFTP Tools Filezilla

Open Filezilla

Host: sftp.datalex.com

Username: <SFTP Account username> e.g. sup-54321

Password: <SFTP Account password> e.g. R89nn6Rr

Then click "Quickconnect" button

FTP / SFTP Tools Filezilla


Now connected:

You can then easily drag and drop files from sftp dlx server to your
PC.