Unit 1 Connecting a Unix server to network and enabling services

1.1 Setting the hostname The host name is set up when the system is installed. But if not, or if you entered an incorrect name, you can set it with the command # hostname s gw.example.org

1.2 Assigning addresses to network interfaces The kernel detects the interfaces automatically when it start need to tell it what interfaces are connected to which networks and which address the system has on each network. if the network is a broadcast network such as an Ethernet, you need to specify a range of addresses, which can be reached directly on the network. This selection is performed with the network mask.

1.3 ifconfig utility The ifconfig utility is used to assign an address to a network interface and/or configure network interface parameters. The ifconfig utility must be used at boot time to define the network address of each interface present on a machine; Can be used at a later time to redefine an interface's address or other operating parameters. The command to configure an Ethernet interface with a network address is # ifconfig ep0 inet 220.147.37.5 netmask 255.255.255.0 up This command tells the interface to use the Internet protocols (the default) and up tells it to bring it up. The address is class C .

Point to point connection

# ifconfig tun0 139.130.136.133 139.130.136.129 netmask 255.255.255.255 Can be used to set up ppp Checking configuration
# ifconfig -a

-a flag lists the configuration of all interfaces

1.4 Daemons These are programs that are not designed to be run with continuous user input and they disconnect from the terminal at the first opportunity. There is a convention to name program that normally run as daemons with a trailing d. Exception - sendmail

Signals with daemons SIGTERM is a polite a way of killing a process. It will close all the files it may have opened and generally finish whatever it is doing at the time before shutting down. The process cannot ignore SIGKILL SIGHUP is used to send a hangupsignal to make daemon read a changed file

1.5 inetd The internet daemon This program should be run at boot time by /etc/rc. It then listens for connections on certain Internet sockets. When a connection is found on one of the sockets, it decides what service the socket corresponds to and invokes a program to service the request. After the program is finished, inetd continues to listen to the socket . inetd allows running one daemon to invoke several others. Inetd

Upon execution, inetd reads its configuration information from a configuration file which by default is /etc/inetd.conf. A typical entry in this file will be #ftp #telnet stream stram tcp tcp nowait nowait root /usr/libexec/ftpd ftpd l

root /usr/libexec/telnetd telnetd

Upon execution, inetd reads its configuration information from a configuration file which by default is /etc/inetd.conf. A typical entry in this file will be #ftp #telnet stream stram tcp tcp nowait nowait root /usr/libexec/ftpd ftpd l

root /usr/libexec/telnetd telnetd

Daemons started by inetd Telnetd Rshd Rlogind ftpd. Running daemons directly To run ftpd directly Add the following line in /etc/rc.local echo n starting local daemons: #put your local stuff here echo ftpd && ftpd -D Option D tells ftpd to run as a daemon However you should comment out the ftp line in /etc/inetd.conf by adding a hash mark (#) in front of it Either reboot or cause inetd to reread it in the configuration file

#ps waux | grep inetd

( to determine the pid number for inetd)

# kill 1 [pid]

(pid number of inetd

sshd The daemon program for ssh Secure shell is set of network connectivity tools used to access remote machines securely. ssh encrypts all traffic to effectively eliminate eavesdropping, connection hijacking and other network level attacks Enabling daemons rc command is used for autoreboot and daemon starting. When the system starts up /etc/rc first reads the description files /etc/defaults/rc.conf which contains the defaults for a number of configuration variables and /etc/rc.conf which contains your modification to the defaults. Starting daemons If the configuration file for a daemon process such as inetd or httpd is changed, the configuration file is to be reread. Start and restart will create an outage. Most daemons however are written to respond to SIGHUP signal. By sending a SIGHUP signal to the daemon processes, the file will be reread and the changes in the file will be implemented.

Unit 2 Installation of peripheral devices

2.1 Installation of a printer 2.2 Installation of a terminal 2.3 Exercises 2.4 Summary

Important commands in printing lpr adds a job to a print queue by copying the file into its spooling directory lpq Lists jobs that are currently in the printer queues. lprm Delete jobs. Nobody other than root can delete someone elses job from a print queue lpd The printer daemon responsible for sending data from spooling directory to a printer lpc Administrative interface to printer subsystem.

/etc/printcap file This file lists all devices serviced by the spooling system. Usually describes many common printers all commented out. Colons separate all individual items in a printcap entry /dev/lpt0 indicates device special file mx maximum file size mx # 10000 pl page length in lines pw page width in characters

Spooling directory This holds files destined for a particular printer until the daemon lpd can print them.

Spooling directories are conveniently located in /var/spool Each printer should have its own spooling directory

Terminals Terminals provide a convenient and low-cost way of accessing the UNIX system when you are not at the computer console or a connected network. Using a terminal attached to an unused serial port, you can log in and run any text program that you would normally run on the console. Dumb terminals Are specialised pieces of hardware that let you connect to computers over serial lines. Have the computational power to display send and receive text Examples : Digital Equipment Corporation VT 100 and Wyses WY-75. Init process Responsible for all process control and initialisation at system start up Reads the /etc/ttys file and start a getty process on the available terminals. Getty process is responsible for reading a login name and starting the login program. /etc/ttys file The first field normally specifies the name of the terminal special file as found in /dev The second field is the command to execute for this line, which is usually getty. The third field is the type of terminal usually connected to a tty line The fourth field specifies if the port should be enabled The final field is used to specify whether the port is secure. If it is secure, root login are allowed from this port.

Virtual terminals UNIX is a multitasking system. PC has only one screen. In Free BSD, virtual terminals solve this problem. In text mode you can change up to 12 different screens with the combinations of Alt key and a function key Use Alt-F1 to Alt-F12 By default 8 terminals are active. /dev/ttyv0 to /dev/ttyv7. The system console is /dev/ttyv0. Pseudoterminals FreeBSD offers an additional class of terminals called pseudo terminals. They come in pairs a master device also called a pty is used by processors which use the interface and has a name like /dev/ptyp0 or /dev/pty/0. The slave device looks like a terminal and has name like /dev/ttyp0. Major and minor device numbers Major device number is an index into a list of drivers and the minor number is a number, which the driver uses to distinguish the individual device Devices can be either block or character type On block devices , you can store a file system Eg: disk drive, floppy drive On character devices you cannot store a file system. Printer, terminal

Unit 3 Monitoring Security

examine the different methods of improving the security of UNIX system. Physical security Search path issues Plugging security holes due small mistakes Using groups more effectively Minimising the risks from freely available software Preventing theft and vandalism Limiting access to the console and cpu unit to prevent someone form crashing the system and rebooting to single user mode Controlling environment factors such as UPS, line conditioners, surge suppressors Restricting or monitoring other parts of the system like terminals, workstation cable Limiting access to backup tapes

Search path issues Place the current directory and bin sub directory of user at the end of the path list after the standard location of the commands Plugging security holes Do not use /tmp as any users home directory Think carefully about which users can be supplementary members of the group 0 and any other systems groups. Implications should be correctly understood. Roots umask should be 057 or more restrictive setting

Using groups more effectively Creating new groups SGID access User private groups

Minimising the risks from freely available software

freely available software should be checked for potential problems

Detecting problems Checking Password files Monitoring the file system Looking for SUID andInvestigate system activity Monitoring unsuccessful login attempts History on the root account. Configuring error logging SGID files Checking modification dates and inode numbers

Checking password files The password file should be examined regularly for potential account-level security problems. The master password file or shadow password file should also be checked

Accounts without passwords # grep ^[^:]*:: /etc/passwd # grep ^[^:]*:: /etc/master.passwd or

Detecting accounts without password Accounts with UID or GID of 0 can be located with grep #grep :00*: /etc/passwd

Checking new accounts New accounts created can be checked using a diff command # diff /etc/passwd /usr/local/bin/old/wps Where /usr/local/bin/old/wps is a copy of the password file stored in an obscure location

Checking ownership of password files #cd /etc ; ls l passwd master.passwd The passwd file should be readable and writeable by root and readable by group and others. The master.passwd file or the shadow password file should be readable and writeable by root only.

Monitoring file systems Checking file ownership and protection This will involve Checking the ownership and protection of important system configuration files Checking the ownership and protection of important directories Verifying the integrity of important binary files

Checking modification dates and inode numbers It is important to check modification times and inode numbers to find if any system files have been modified The following command checks inode numbers and modification times for files and directories starting with rc. # ls lsid /etc/rc*

Investigating system activity If you regularly monitor the system you will be able to check for security breaches It will be worthwhile checking for processes that remain idle for long periods of time, users logged in at unusual times or from unusual locations and the like. # ps auxww | head 3

Monitoring unsuccessful login attempts Repeated unsuccessful attempts for any user accounts can indicate someone trying to break into the system. In Free BSD you can see a list of events in /var/log/auth.log Some versions of UNIX provide some mechanism for logging all attempts to become superuser. Such logs can be very useful when trying to track down who did something untoward

History on the root account. Give root a shell that can support a history mechanism (the C shell or Korn shell ) Set the number of commands saved across login sessions to a large number in the roots initialisation file The different commands executed as root can be traced. C Copy the history file to some secure location periodically using the cron facility

Configuring error logging The syslog message logging facility provides a general way to specify where and how some types of system messages are saved syslogd collects messages sent by the various system processes and routes them to the final destination based on instructions given in its configuration file. /etc/syslog.conf

/etc/syslog Structure of the file Facility.level destination Facilities- kern, mail, lpr, daemon Severity level emerg, alert, crit, err, notice, info debug, none

Tracking user activities Use utilities that can determine what users have been doing on the system Will enable you to track down the cause of a security problem The last command displays data for each time a user is logged into the system. Last may also be followed by a list of usernames and/or terminal names

Unit 4 Monitoring system performance

Monitoring the system parameters Monitoring the system parameters Monitoring CPU activity Monitoring Memory use and paging activity Monitoring CPU activity UNIX provides the ability to monitor process execution Allows to a limited extent, specify their execution priorities Monitoring Memory A system should have adequate memory not just for the largest jobs but also for the overall mix of jobs run every day Paging and swapping are the means by which UNIX distributes available memory among current processes when their total memory needs exceed the amount of physical memory Swapping Swapping refers to writing an entire process to disk thereby freeing all its memory. A swapped out process should then be reread into memory from the disk when execution resumes Paging Paging involves moving sections of a processs memory in units called pages to disk in order to free up memory needed by that process or another one. A page fault occurs when a process needs a page of memory that is not resident and must be reread it from the disk Problem with paging The main problem occurs when there is not enough memory on the system for all the processes currently running An appreciable CPU time is spent for page faulting and the whole process is very inefficient.

Uptime command The uptime command will give a rough estimate of the system load. #uptime 3:25PM up 2 days, 2:41, 16 users , load average: 1.90, 1.43, 1.33

ps command The ps command gives a more complete picture of system activity. This utility gives a report summarising execution statistics for current processes. ps aux produces a table of all processes, arranged in the order of cpu usage.

ps aux command USER username of process owner PID Process ID % CPU % MEM VSZ RSS Estimated fraction of CPU consumed. Estimated fraction of system memory consumed. Virtual memory used in K Real memory used in K

TT, TTY Terminal port associate with process STAT Top utility This provides a continuous display of the system status and most active processes which it automatically updates every few seconds. -dcount show only count displays and then exit. A display is considered one update of the screen -stime set the delay between screen update in seconds. Default is 2 secs -Uusername Show only those processes owned by username. Current process state

Typical display Mem: 9220K Active, 1032K Inact, 3284K Wired, 1MB Cache, 2M Buf, 1320K Free Swap: 91M Total, 79M Free, 13% Inuse, 80K In, 104 K Out Priority for processes Unix has two priority levels Its nice number, which is the requested execution priority with respect to other processes. Root or the processors owner sets this. Its current execution priority, which is computed and dynamically updated by the operating system ( shown as PRI) Nice command Nice execute a utility at an altered scheduling

Syntax nice [ -n increment ] utility [argument] # nice n 5 date renice alter priority of running processes # renice 10 -u tjones Sets the nice value of all processes owned by tjones to 10 # renice n +10 -g users1 Increments the nice number of all processes of the group users1 by 10. # renice -20 567 Sets the nice number of process ID 567 to -20 Vmstat vmstat reports certain kernel statistics kept about process, virtual memory, disk, trap and cpu activity.

Unit 5 Downloading and installing programs in UNIX

Learn how to archive and compress files Use ftp to download files Set up the UNIX machine on the network Download a program and install it

Archiving and Compressing files Tar utility gzip and gunzip Tar Utility stores a number of files and directories into a single file. designed to read and write data on tape and the name is derived from tape archive. can be used for packaging directories and unpackaging them Options with tar command a Adds files to an existing tar file c Creates a new tar file -- delete Deletes a file from tar file f <filename> Name of output tar file t Lists contents of file v Produce verbose listing of file processed x Extracts files from tar file z applies compression to tar file using gzip and gunzip

gzip and gunzip gzip ( gnu zip) command is generally considered a good option for compressing files gunzip decompresses files compressed by gzip. Downloading files using ftp Used to download files directly to your computer from another computer on the Internet can be used on the command line of the console or a terminal window ftp name of a remote site. FTP download % ftp ftp.tenon.com % ftp 206.246.150.88 FTP commands ! Run a shell command ascii Specify text file downloads binary Specify binary file downloads bye Close the open connection and exit ftp cd nnn Change to directory nnn close Close the open connection and quit get file Download file from the current directory of the remote computer help List help topics for ftps commands ? gives a list of commands open Open connection on a remote computer pwd Print current working directory on the remote computer put Send (upload) a file from your computer to the remote computer

Using ftp to download files Use the ftp command from the command line followed by the name of a remote system. After ftp connects to the remote computer, the remote computer usually presents a login prompt. If you do not have an account, on the remote computer system, type anonymous at the Name prompt and press Enter. At the password prompt, type your e-mail address. Installation of a program Building programs with make command The make utility provides a scripting capability to automate the compilation and linking of files If the program source comes with a makefile and the documentation says the file supports Linux, making the program work on your system will be quite easy. Make command make <options> < make target > Make runs only on dependencies which means that certain action must be completed successfully before the system can carry out another action. This process is usually implemented to force make to build the program before initiating the install script It has built in defaults

Configure usually walks you through a script to establish the programs configuration and to configure other files before you can run make and make install. If the documentation says the program supports configure option, you can run it by typing make configure or make config depending upon your program

install

To build and install the program, use the make install command. This tells make to build the program and if everything is successful to install its executable and configuration files in the appropriate directories with the correct permissions ready for users on your system to use.

Clean Large program often leave a substantial number of intermediate files lying about in their source directories, which can consume a significant amount of space. To make sure that these are cleaned up or to clean up after a failed build of the program use the command make clean.

Unit 6 Building a custom Unix Kernel

Overview of Kernel There are three layers of the UNIX system. Hardware Operating System Kernel User level program Functions of Kernel Kernel manages resources in the UNIX system. It manages the processes, signals, virtual memory, file systems and network connections. All device drivers are linked into the kernel. It translates the request from processes to device driver instructions. Advantages of Custom Kernel Faster boot time. The kernel will probe only the hardware on your system. Less Memory usage Allows operating system upgrade and addition of new device drivers Configuration file The configuration file is contained in the directory /usr/src/sys/i386/conf and is named GENERIC Each line contains a keyword and one or more arguments An exhaustive list of options and more detailed explanation of the device lines is present in the LINT configuration file, located in the same directory as GENERIC. Mandatory fields machine i386 Type of cpu

ident GENERIC maxusers n

# Floating point support do not disable # Pseudo devices the number indicates how many units to allocate Configuration of custom kernel Make a list of your system hardware Get the components required to make a kernel for your system. If it is not already available, install it from the CDROM. Make a duplicate of the GENERIC kernel. You may call it MYKERNEL. The generic kernel will be available in /usr/src/sys/i386/conf directory Store a copy of the file MYKERNEL in another directory. Configuration of Kernel Build up a configuration file by making necessary changes to MYKERNEL. Run configuration of the files while you are in /usr/src/sys/i386 directory Run make depend Type # make on the command prompt Install the kernel. # make install The new kernel is installed in the boot directory and the old kernel is saved as kernel.old. Test, debug and reboot from the new kernel. Document the changes

Unit 7 System Initialisation in Linux

Setting initial run-level The initial run level in Centos Linux can be set by modifying the /etc/inittab file. The default setting for this Linux is level 5 which starts up the system in multi-user mode with full networking support with GUI. Common runlevels and what they mean 0 halt the systems 1- single user mode 2- multi-user mode but with no networking support 3- multi-user mode with full networking support 5 multi-user mode with full networking support and start the graphical user interface. id:5:initdefault: Entry in /etc/inittab shows the default setting in Centos. Respawning process in Linux you will learn how to configure a process so that it is respawned or restarted, if it is terminated for any reason. The most obvious practical application is to restart virtual consoles when users log out of them. The init process may be configured via the /etc/inittab file to run a process forever. 6:2345:respawn:/sbin/mingetty tty6 This line in /etc/inittab tells the init program to start the sixth virtual console (tty6) whenever Linux is running in run levels 2,3,4 and 5 and to start the program anytime it stops. 6:2345:once:/sbin/mingetty tty6 The login will not be spawned when a user logs out. It will be run only once

Once changes are made to /etc/inittab files, you need to type kill HUP 1 or kill -1 1 showkeys comand Shows the key code for any keys on the key board. Type showkeys on the command line and press the key to find its key code Page up ------ 104 Page down --- 109 Upper arrow --- 103

Unit 8 Managing Linux processes

Displaying parent/child relationships ps f you can see PID and PPID values. The PPID is the PID of the parent process Using the PPID values, you can see the parents of all processes ps eH e option selects all processes and the H option produces a process hierarchy display. Child process appears below their parents and is indented by two spaces. ps axjf The f option causes ps to use graphical characters ( \ and _) rather than spaces to show the parent/child relationships. pstree all the processes on the system arranged in a parent/child hierarchy. Customising ps output ps e -e option tells ps to display all processes. There are four columns: PID, TTY, TIME and CMD Other options -f full listing -j jobs formt j job control format l long listing s signal format v virtual memory format X i386 register format ( only with Intel processor-based system)

Selective listing of outputs ps eo pid,cmd | more Displays only PID and CMD columns

Unit 9 Common Administrative tasks in Linux

Configuring virtual consoles

Edit /etc/inittab file to include new virtual terminals Use the command kill -1 1 to let init know the changes Check whether these terminals are active using w command

Testing your logging In this section you will learn about the setting up logging options in /etc/syslog.conf file in Centos Linux You can test the syslog logging ( the file is called syslog) to see if it worked the way you intended when you created or modified the /etc/syslog.conf file.

Syslog file Entries in /etc/syslog.conf have the following format. Facility.level destination

Facility is the name of the subsystem sending the message, level is the severity level of the message and destination is the file, device computer of user name to send the message to

facilities kern The Kernel mail lpr The mail system The printer subsystem

daemon System Server processes. auth Login authentication system

Levels in the order of decreasing seriousness Emerg System Panic Alert Crit Err Serious error requiring immediate attention Critical errors like hard device errors Errors

Notice Non-critical messages Info Informative messages

Debug Extra Information helpful for tracking down problems None Ignore messages from this facility

Finding broken links and files with no owners If there are files with no owners it is likely that they will be allotted to new users when they are created. There can also be symbolic links that no longer point to a file. This section discusses techniques to find such files and links. The commands to delete unowned files and redundant symbolic links are discussed. To find all dangling links # symlinks r / | grep dangling To remove all dangling links # symlinks r d /

Understanding date and time stamps of files Three time stamps for files ctime mtime atime Helps administrators with housekeeping

How do you determine the time stamp for a file ls l gives mtime ls l -time=ctime gives ctime ls l --time=atime gives atime

How to create an alias to fine all the three time stamps alias dir=ls l -time=ctime | grep root ; ls l | grep root ; ls l -time=atime | grep root When we type dir all the three time stamps are displayed In Ex 9.4.1 , it should be entered while in /home/betty after creating a file file1 cat file1 changes the atime Modification of file changes all the three time stamps Except in cases where the file is opened and read, ctime changes for all instances such as the use of chmod, chown, chattr Study the option +d and +a for chattr

Touch command By default changes all the three time stamps Study the a and m options of touch

Unit 10 System Initialisation - Redefining Keyboard Lock/LED Operation

What will you learn in this unit? To control the state of Caps Lock, Num Lock and Scroll Lock function of your keyboard for your virtual consoles By default, these three key locks are off for all virtual consoles. You must press the lock keys to enable lock. For example you must press the Num Lock key to put the keyboard in the Num Lock state where the numeric keypads used for entering numbers rather than controlling the cursor. The state is indicated by a corresponding LED setleds D +num The Num Lock light or LED should turn on Num Lock is set back to its default state by running the reset program . You can also control the state of the associated lights or LEDs independent of the associated key lock. For example, you can turn on the caps Lock light or LED regardless of the state of the keyboards Caps Lock state. Eg : setleds L +caps turns on Caps Lock light regardless of the state of Caps Lock state

Use of setleds command You can set up the light to indicate that the user has mail, the file server cannot be reached, the user is reaching the limit of his disk space and so forth. You can run the program anytime you like from the command prompt or a script. You can also run the program in the system startup scripts to preset the locks to a known state.

Unit 11 File systems in UNIX File systems Logical and physical disk drives File system types Organisation of directories in UNIX Mounting a file system Unmounting a file system Shutting down a UNIX system

File system is a collection of files and directories on a disk or tape in standard UNIX file system format.

Logical drives A logical drive refers to a subdivision of physical disk, which you can refer to as if it were a disk by itself

Partitions in UNIX FreeBSD uses the term partition differently as compared to Microsoft. It refers to Microsofts partition as slices. Microsoft designates one slice as the primary Microsoft partition, the C: drive.

Unix partitioning Unix systems have their own form of partitioning and are not compatible with the Microsoft methods. UNIX implements their own partitioning within a single slice (Microsoft partition). BSD systems define up to 8 partitions per slice

Naming partitions in Free BSD The device entries are in the directory /dev The name of the driver comes next. It also indicates whether it is an IDE or SCSI drive The first disk is indicated by /dev/ad0. Next is the partition information. The strict slice name is specified by adding the letter s (for slice) and the slice number. /dev/ad0s1 indicates slice 1 in first disk. BSD systems name partitions by appending the letters a to h to the disk name. Thus the first Unix partition of the first slice (which is typically a root file system) would be called /dev/ad0s1a. Partition c is an exception. It represents the whole BSD disk where the FreeBSD resides.

File system types ufs is the UNIX Files System. cd9660 is ISO 9660 CD-ROM format with the so called Rock Ridge Extensions which enable UNIX-like file names to be used. nfs is the Network File System which is a means of sharing file systems across a network. Microsoft files can be accessed with msdos and ntfs file systems

Organisation of directories in Free BSD / root directory /sbin files System executable programs needed at start time - System administration

/bin executable programs of general use which are needed at start up time (binary files) /dev directory of device nodes /etc Contains many of the configuration files

/root home directory of user root /usr/bin common user binaries not required at start time /usr/sbin System administration programs not needed at start time /usr/share dictionaries, configuration files for user programs /usr/share/man manual pages /var a file system for log files , files which grow or change frequently /var/spool mail and printer queues /usr traditionally for users home directories but no longer. Second file system

/usr/local locally installed software /home home directories of the user

Mounting a file system Information on a disk slice or partition becomes available for user only after the kernel has been notified of its existence through the mount command Mounting a file system logically attaches the information stored in that file system to a specific directory that already exists.

Mounting a file system Each file system is itself a tree with a root directory. It can be attached or mounted on any directory in the UNIX tree. Mounting it on one directory versus another changes the path name of the files down to the root of the file system but has no effect on the structure of the files and directories within the file system.

Unmounting a file system For repair To save memory resident information about the file

If any part of the file system were used, UNIX would not unmount it.

Some useful commands df prints a summary of the free space in all the mounted file systems. du Shows the usage of the disk ( Disk Usage )

Shutting down a UNIX system Shutdown provides an automated shutdown procedure for super users to nicely notify users when the system is shutting down Use shutdown [+ time in minutes] command Options No options goes to single user mode Shutdown +1 sends to single user mode in 1 minute -h halt -r reboot shutdown -h now Shutdown and halt the system immediately

Single user mode This mode has only one user the person at the console The user must be at the console. There is no password protection, so physical security is the only security Not many processes are running generally- only init and shell you are using are active.

Generally the prompt is # Limited set of tools

Getting into single user mode Set boot flag in the boot program. Typically s represents single user mode. Boot -s Once operating system is running , use shutdown command

Getting out of single user mode CTRL-D will exit and take you into multi-user mode provided there are no problems with the boot process. Issue reboot or halt command

Use of a single user mode There is only one user on the system. There is less activity and less damage if a crash occurs. It is minimal mode that the system can boot if other file systems are corrupt. Allows the user to unmount the /usr and other normally busy file systems.

Unit 12 Unit 12 Compiling and linking with gcc

gcc compiler gcc is a C compiler developed by Free Software Foundation , which has developed a number of tools for use on a UNIX system It compiles the source code and assembles the assembly language code . The compiler produces and invokes the UNIX loader ld to produce an executable file. It also includes a compiler for the C++ language.

Fig 12.1 Stages of compilation of gcc

Stages of compilation Pre-processing It is this step which resolves directives like #define #include. A separate utility called cpp is used to do the pre-processing

Compilation This produces assembly language files from the input files. The assembler is usually invoked right away. So output is not normally saved in files. Assembly This takes the assembly language as input and produce object files with .o extensions. Linking In this final stage, the various .o modules are placed in their proper places in the executable files. The library files are also placed in the file. UNIX compilers perform this phase by internally invoking the linker, which is called Id.

options c implies Compile but not link -g Generates an expanded symbol table for debugging. This option enables debugging with GNU debugger gdb -v prints the compilers version number and completer details about how each pass is executed. -w Suppress all warning messages -W produce some additional warning messages about legal ( but questionable ) coding practices. -Wall Produces even more warning messages about questionable coding practices -Werror Make all warnings into errors. Do not attempt to produce an object file if a warning has occurred.

mainprg.c #include <stdio.h> void printheading (void), printmaxnum(int m); int funmax (int m, int n);

int main (void) { int a,b,max; printf("Enter two numbers\n"); scanf("%d%d", &a, &b ); printheading(); max=funmax(a,b); printmaxnum(max); return(0); }

util1.c #include <stdio.h> int funmax ( int m, int n) { if ( m > n) return (m); else return(n); } void notused(void) { printf ("\n not all functions have to be used \n"); return; }

util2.c #include <stdio.h> void printheading(void) { printf("This is a multiple source project \n"); return; } void printmaxnum ( int m) { printf("The larger number = %d\n", m); return; }

Unit 13

Awk and Sed utilities

Awk utility Awk utility is a powerful data manipulation programming language that allows the user to select lines of input based on specified criteria, take action on the data on these lines. Awk works on information arranged in a database file.

General structure of an awk command The general form of the awk command is as follows

command pattern {action} input file

awk /Chicken/ {print} groc_base

Selecting lies with specific patterns The patterns can be matched by operator (~) for a specific field. awk $1~/C/ groc_base selects all lines which have C in the first field.

The caret (^) in a regular expression forces a match at the beginning of a line or field awk $2~/^M/ groc_base selects all lines which have M as the first letter of second field.

The dollar ($) forces a match at the end of a line or field. awk $2~/eg$/ groc_base

selects all lines which have eg as the last part of a field

Printing a specific field To print a specific field, specify the first field as $1, second as $2 and so on awk /Veg/ {print $1 $3 } groc_base awk /Veg/ {print $1, $3 } groc_base awk $1~/^[CM]/{print $1, $2, $$3} groc_base

Selecting specific fields We can select records which satisfy a certain condition using equal to ( = =), greater than (>), less than (<) symbols. The logical operators are && ( and), || (or) , ! (not) awk $3 = = 1.20 groc_base prints the records for which the value of the third field is 1.20 awk $3 < 3.00 && $3 > 1.00 groc_base

Creating a command file We can create a file which contains the pattern and action. This file can be run with the awk command Create a file called print_dairy which contains the following line /Dairy/{print $1,$3} using cat command To run the program enter the following command. awk -f print_dairy groc_base (-f option tells awk to read a file for its program)

Improving the readability of files /Dairy/ { print $1 , $3 } /Dairy/ { name = $1 price = $3 print name, price }

Mathematical operations The words within print statement can be quoted and operations can be indicated in the statement. Operations can be indicated with the variables. The operations are addition (+), subtraction (-), multiplication (*), division (/)

Example of program with mathematical operations groc1 name=$1 type=$2 price=$3 qty=$4 taxable=$5 discount=0.1 saleprice=price - discount * price print name, type, saleprice, qty, taxable

To run groc1 use the command awk -f groc1 groc_base

BEGIN and END patterns The awk utility performs the action associated with BEGIN before it processes any of the lines of the data file. The END pattern works in a manner similar to BEGIN pattern except that awk takes action associated with it after it has processed the last of its input lines.

Example of begin and end patterns BEGIN { print The list of groceries purchased print -------------------------------------- print Name Type Unit price Quantity Taxable} {print} END { print NR, items were purchased}

Flow control in awk Awk uses the if, if-else, for and while loops of C language. We will have a simple example of a if- else loop here { if ( $3 < 1.00) print $1, $4 else print $3, $4 }

Sed editor UNIX filter Takes input from a file but its output is sent to standard output without altering the original file Syntax sed line specifier, editing instructions filenames

Line specifier You can give a line number or a range of line numbers..

-third line

5,10 - lines 5 through 10 10,$ - lines 10 through the last line You can specify a line for sed by providing a matching pattern enclosed in slashes. /large/ lines containing string large

/[Pp]at/ all lines containing Pat or pat You can combine two forms of line identification. 2, /talk/ from line 2 to the first mention of talk

Editing instructions s (substitute) p(print) d(delete) q (quit)

1i\ text 2a\ text appends the text after the line 2 inserts the text before the line 1

Options for sed -n suppresses the automatic printing of each processed line -f option followed by a space and filename takes editing commands from a named file sed -f edit speech Commands in edit file are applied to speech -e option allows the mixing of a command line editing command with commands in a file

Examples sedscript1 10 a\ January sed f sedscript1 months Appends January after the 10th line

Practice Exercise Create a file days Sunday Monday ------Thursday

Add Friday and Saturday after Thursday sed f addfile1 days Create the file addfile1 Add a heading for the file using a file addfile2 Days of the week

Unit 14 Testing sedscripts

The command % sed f sedscr testfile does not make the change in testfile. It sends to the screen the lines that were modified and the lines that were unchanged. You have to capture this output in a new file if you want to save it. % sed f sedscr testfile > newfile The output from the command should not be redirected to the input file. This would overwrite the input file. Use the diff program to point out the differences between the two files. % diff testfile newfile When the results are verified, a backup copy of the original input file can be made . you can use the mv command to overwrite the original with the new version.

for x do sed f sedscr $x > tmp.$x done The shell script testsed automates the process of saving the output of sed in a temporary file. It expects to find the script file sedscr in the current directory and applies these instructions to the input file named on the command line. The output is placed in a temporary file

The name of the file must be specified on the command line. The shell script saves the output in a temporary file with prefix tmp

% testsed filename Diff command You can also incorporate diff command into the shell script. add diff $x tmp.$x after the sed command Modify the file to incorporate the diff command. #!/bin/sh for x do echo "editing $x: \c" if test "$x" = sedscr then echo "not editing sedscript!" elif test -s $x then sed -f sedscr $x > ./tmp/$x if test -s ./tmp/$x then cmp -s $x ./tmp/$x && echo "file not changed: \c";\ cp ./tmp/$x $x ; echo "done" else echo "sed produced an empty file - check your sedscript." fi else echo "original file is empty" fi

done echo "all done" To run runsed, create a sed script named sedscr in the directory where you want to make the edits. Supply the name and names of the files to edit on the command line. %./ runsed chapter1 runsed invokes sed f sedscr on the named files one at a time and redirects the output to a temporary file. The program then tests this temporary file to make sure that the output was produced before copying it over the original The shell script incorporates testsed. The additional lines are intended to test for unsuccessful runs for instance when no output is produced. It compares the two files to see if changes were actually made or to see if an empty output file was produced before overwriting the original. You should used testsed first to verify your changes before actually making them permanent with runsed Create a new program testsedmod, which can do the same task as testsed when you enter the command % ./testsedmod sedscript1 months There should be a reference to sedscript1 and months inside the program as $1 and $2 A program prog1 echo $1 $2 $3 echo $1 $2 $3 echo $1 $2 echo $0 After exec permission is given Run the script ./prog1 one two three

What is the output? Answer one two three one two prog1