CS 345 Operating Systems BYU-Idaho

Adding a System Call to a Linux Kernel

Note:
This is a modification of the project at the end of chapter 2 of Operating System Concepts Seventh
Edition by Silberschatz, Galvin, and Gagne. Some of the text below is taken from that project
description.

Objectives

o Add a system call to a Linux kernel.

o Learn about some operating system internals
o Learn about system calls, Linux, compiling a Linux kernel, the make utility and more.
o Use a virtual machine (VM)

Requirements

o Add a helloworld system call to a Linux kernel carefully following the instructions below. To do this,
you will create a file and edit other kernel code files in the kernel source directory. You will also create
a program to test your system call.
o After adding the helloworld system call to a Linux kernel, youll add a second system call to the kernel
that writes a string to a file. When calling the system call you will give two arguments, a file name and
a string.
o Submit a .doc, .docx or .pdf file to I-Learn that contains the following items
o Your name, course name and date
o A conclusions section that includes:
if you were successful on the first part of the lab,
if you were successful on the second part of the lab,
problems encountered in doing this lab and what the solution to the problems were
insights and what you learned while doing this lab, and
comments about the lab and suggested improvements to the lab (modified instructions,
links, helps, etc.).
o A screenshot of/from the virtual machine (VM) showing the execution of the test program and
the last five lines of the /var/log/messages file. This screen shot should include having a
terminal window visible that shows your login name in the title bar, or some other personal
identifier. This screenshot may be created by using a snipping type of tool, or pressing:
PrtScn on the keyboard, or alt-PrtScn; using the alt key grabs just the active window.
o A second screenshot on the system showing the execution of the second system call and the
contents of the file written to by the system call. This screen shot should also include having
something visible that identifies you personally. To show the execution of the second system
call you should:
Remove the executable and the output file that you (probably) created while testing
Do a grep of the line out of your code that shows what contents you are writing to the
file that is being created
Do an ls to show that the executable and output file are not in your current working
directory
Compile and run your code that calls your new system call
Do an ls to show that the file is now in the directory
Do a cat on the newly created file to show its contents

Helps and additional comments

o Come back and re-read this section later if it doesnt make sense now.
o Part way through adding the system call, you might benefit from re-reading the background section.
o You will use gcc (not g++).
o Go back and re-read the Requirements section before submitting the .doc file.
 o Shutdown the virtual machine before closing the VMware Server Console.
o For a better understanding of system calls and the build process:
o man syscalls
o Take a look at a tutorial on the make command and Makefiles. One such tutorial is found at:
Norman Matloffs Unix and Linux Tutorial Center (Managing Modularity: Makefiles and Libraries
at: http://heather.cs.ucdavis.edu/~matloff/UnixAndC/CLanguage/Make.html.)
o To shutdown a UNIX/Linux system you may use the command: shutdown h 0

Setting up the Gentoo virtual machine

You will need to install VMPlayer or Virtual Box, use Hyper-V (Windows 10 Enterprise/Professional/Education)
or some other type of application/system that allows you to run a VM on your system. You will then need to get
a copy of the compressed Gentoo VM on your local system, unpack it and be able to run it.

On a Windows or Linux system you may install the free VMware Workstation Player by going to:
http://www.vmware.com/products/player/playerpro-evaluation.html.

On a Mac, install a free trial of VMware Fusion by going to https://www.vmware.com/products/fusion/fusion-

evaluation.html.

Next, you will need to download a copy of the compressed tar or zip file of the Gentoo Linux VM that is on the
CSEE servers onto your local system. The tar and zip file are in: /home/cs345/labSyscall. A copy of the
VM is also available on a flash drive that may be checked out from the Department administrative assistant.
This file may be downloaded using MobaXterm, WinSCP, on a Windows system or using an scp command
on Linux and Mac systems. For Windows systems, get the zip file. Note that it takes 10 minutes to get this
>500 MB file on a very fast Internet connection.

Once the tar or zip file is on your local system, extract the Gentoo VM directory using GUI or command line
tools. The extract/inflate/un-compress operation will take a minute or two to do. On Windows, right click the zip
file to get the extract command. A command line command on a Linux system would be:
cd someDirectory #someDirectory is where you saved the tar file
tar xjf Gentoo.tar.bz2

Starting your virtual machine

You will be running a Gentoo Linux system on a virtual machine through the VMware Player or other such VM
application/system. Start the VMware Player application.
Next, Open a Virtual Machine. Click this option and then browse to your uncompressed Gentoo VM directory
and open the Gentoo.vmx.

The Gentoo VM is now in your library of VMs and you may power on the virtual machine by selecting Play
virtual machine. You will next be asked if you moved or copied the VM, say that you copied it. In the Software
Updates window that will probably pop up, click Remind Me Later. As you do this, you will see a computer
booting up in the virtual environment. You are booting a virtual machine on a Gentoo Linux kernel.

To give the virtual machine input from the keyboard and mouse, click the window containing the virtual machine.
To get control of the mouse and keyboard back to use outside of the virtual machine, press ctrl-alt.

Notice that when booting your VM, you will come to a menu with two options. The first option is Gentoo-Good
and the second option is Gentoo-Student-Kernel. By default the second option should be highlighted. If you
ever destroy your student kernel, you can still get into your virtual machine by simply selecting Gentoo-Good at
this point. This will allow you to boot your virtual machine to make any needed changes to your kernel.

When the virtual machine is finished booting you will see a login prompt. Login as root using syscall as your
password. You are now in your virtual environment running Gentoo Linux and ready to start.
Some background (mostly taken from the book)
In this project, you will study the system call interface provided by the Linux operating system and how user
programs communicate with the operating system kernel via this interface. Your task is to incorporate a new
system call into the kernel, thereby expanding the functionality of the operating system.

A user-mode procedure call is performed by passing arguments to the called procedure either on the stack or
through registers, saving the current state and the value of the program counter, and jumping to the beginning of
the code corresponding to the called procedure. The process continues to have the same privileges as before.

System calls appear as procedure calls to user programs, but result in a change in execution context and
privileges. In Linux on the Intel 386 architecture, a system call is accomplished by storing the system call
number into the EAX register, storing arguments to the system call in other hardware registers, and executing a
trap instruction (which is the INT 0X80 assembly instruction). After the trap is executed, the system call number
is used to index into a table of code pointers to obtain the starting address for the handler code that implements
the system call. The process then jumps to this address and the privileges of the process are switched from
user to kernel mode. With the expanded privileges, the process can now execute kernel code that might include
privileged instructions that cannot be executed in user mode. The kernel code can then perform the requested
services such as interacting with I/O devices, perform process management and do other such activities that
cannot be performed in user mode.

The system call numbers for recent versions of the Linux kernel are listed in /usr/include/asm/unistd.h.
(For instance, __NR_close, which corresponds to the system call close() that is invoked for closing a file
descriptor, is defined as value 6.) The list of pointers to system call handlers for the Gentoo kernel you will be
modifying is stored in the file syscall_table.S under the heading ENTRY(sys_call_table). The order of
the entries in the system call table must be consistent with the system call number defined in the unistd.h file.
Adding a system call

Note: As you go through the following instructions take a little time to try to understand what you are doing
rather than just following instructions. You might also want to take a look at the Helps and additional
comments section of this handout.

Now that you are familiar with the various background tasks corresponding to booting a VMware virtual
machine, you can begin the process of adding a new system call to the Gentoo kernel on the virtual machine.
The system call will have limited functionality; it will simply transition from user mode to kernel mode, print a
message that is logged with the kernel messages, and transition back to user mode. We will call this the
helloworld system call. While it has only limited functionality, it illustrates the system call mechanism and sheds
light on the interaction between user programs and the kernel.

To add the helloworld system call to the Gentoo kernel, modify the following files in the kernel source code
(located under /usr/src/linux) :

1. /usr/src/linux/include/asm-i386/unistd.h
- Add a line after line 332: #define __NR_helloworld 325
- Change line 337 (it was line 336, but if you did the last step ): #define NR_syscalls 326

This is defining a new system call number for __NR_helloworld in the kernel source code and sets a
constant that gives the number of system calls currently defined in the kernel. Later, you will also edit a
unistd.h file that is not in the kernel source that is used when compiling user programs.

2. /usr/src/linux/include/linux/syscalls.h
- On line 614 add asmlinkage long sys_helloworld(void);

3. /usr/src/linux/arch/i386/kernel/syscall_table.S
- Add .long sys_helloworld at the end of this list.

The keyword .long denotes that the entry will occupy the same number of bytes as a data value of type
long.

4. /usr/src/linux/kernel/Makefile
- At the end of the obj-y section add helloworld.o

The Makefile in the kernel directory needs to know how to compile the helloworld.c file you will create next
when the build of the new kernel is done [that comes later]. Edit the file kernel/Makefile and add
helloworld.o to the definition of obj-y in that file. This is all that is needed to allow make [which takes its
commands from the various makefiles in the kernel source code] to compile helloworld.c. The
helloworld.c will be compiled to create the helloworld.o object file which will then be linked with many
other object files to create a new kernel. [You might want to read about make and what it does.]

Create the following file:

5. /usr/src/linux/kernel/helloworld.c
- This file should contain the following code:
#include <linux/linkage.h>
#include <linux/kernel.h>

asmlinkage long sys_helloworld()

{
printk(KERN_EMERG "Hello World!\n");
return 0;
}

This is the helloworld system call code. The printk() function is used to print messages to a kernel log file
and therefore may only be called from the kernel. The kernel messages specified in the parameter to
printk() are logged in the file /var/log/messages. The function prototype for the printk() call is
defined in /usr/src/linux/include/linux/kernel.h with code in
/usr/src/linux/kernel/printk.c.

Writing a test program to call the helloworld() system call

Now you will create a test program that calls the new helloworld system call. Modify the following file in the
Gentoo file system (this is outside of the kernel source code):
/usr/include/asm/unistd.h
- On line 325 add #define __NR_helloworld 325
- Change line 327 to #define NR_syscalls 326

A user program uses this header file to identify the helloworld system call. This unistd.h file and the one
in the kernel source must have the same value defined for __NR_helloworld.

Create the following file in roots home directory to test your new system call
/root/testMyCall.c
- This file should contain the following code:
#include <sys/syscall.h>
#include <linux/unistd.h>
#include <errno.h>

_syscall0(int, helloworld);

main()
{
helloworld();
}

As noted earlier, a system call is invoked by storing the appropriate value into a hardware register and
performing a trap instruction. Unfortunately, these are low-level operations that cannot be performed using C
language statements and instead require assembly instructions. Fortunately, Linux provides macros for
instantiating wrapper functions that contain the appropriate assembly instructions. The C code above uses the
_syscall0() macro to invoke your system call that you built into the Gentoo kernel.

The _syscall0 macro takes two arguments. The first specifies the type of the value returned by the system call;
the second argument is the name of the system call. The name is used to identify the system call number that
is stored in the hardware register before the trap instruction is executed. If your system call requires arguments,
then a different macro (such as _syscall2, where the suffix indicates the number of arguments) could be used to
instantiate the assembly code required for performing the system call.

Building the new kernel

Change directories to /usr/src/linux. From here type make, to execute the make file and start a kernel
build. Once a new kernel has been successfully built, type make install to have the system install the new
kernel for you.

Once everything is finished, it is time to reboot the system and load your new kernel that includes your system
call. To reboot, type reboot.

Once the system has rebooted, go to /root and compile your testMyCall.c program using: gcc
testMyCall.c.

Run the executable with ./a.out. You should see the text Hello World! displayed. Additionally,
/var/log/messages should have an entry added to it. Congratulations, you did it!!
To create the screenshot that you will submit in your tar file, execute the test program (./a.out) and do a:
tail -5 /var/log/messages to show the last five lines of the /var/log/messages file. Remember
that the screenshot is to include a terminal window, or something else, that displays your login name.

Doing it again
Now you have (hopefully) successfully built a kernel with a helloworld system call. You should indicate whether
you did or didnt in the conclusions section of your .doc document. The second part of this assignment is to
create another system call that writes a given string to a given file: write_file(char *filename, char
*string). In the kernel code, the function will be named sys_write_file( ... ).

Unfortunately, the library that you would normally use for this kind of file IO (<cstdio>) is not available to you
inside the kernel. This library is unavailable because its functions map to corresponding system calls which
perform the desired IO operations. However, since you are in kernel mode, you cannot make system calls into
the kernel. You will have to perform the file IO using routines compiled into the kernel.

To write a string to a file you may use the following:

#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/syscalls.h>
#include <linux/file.h>
#include <linux/fs.h>
#include <linux/fcntl.h>
#include <asm/uaccess.h>

asmlinkage long sys_write_file(char *filename, char *string)

{
struct file *file;
loff_t pos = 0;
int fd;

mm_segment_t old_fs = get_fs(); /* save the old file system view */

set_fs(KERNEL_DS); /* use the kernel file system view */

fd = sys_open(filename, O_WRONLY|O_CREAT, 0644);

if (fd >= 0)
{
sys_write(fd, string, strlen(string));
file = fget(fd);
if (file)
{
vfs_write(file, string, strlen(string), &pos);
fput(file);
}
sys_close(fd); /* close the file */
set_fs(old_fs); /* restore the old file system view */
return 0; /* a successful return value */
}
else
{
set_fs(old_fs); /* restore the old file system view */
return 1; /* return value: could not open or create file */
}
}

Create another version of testmycall.c to use your write_file system call instead of your helloworld system call.
(Lets say testmycall2.c.) You will have to use _syscall2 instead of _syscall0. Like:
 _syscall2( long, write_file, char*, filename, char*, string );

In calling your new system call, give it a name of some file in /root say /root/myFile to create/open and
write your string into.

Your new kernel should have two new system calls. Use a screenshot(s) to document their successful
operation. See the requirements section of the lab for what to show in the screenshot.

If things go wrong

Figure out what went wrong, fix the problem and document it in your documentation file.

If you panic your virtual machine there is hope of getting it back up and running! Often, the file system
check done as part of the boot process (discussed previously) will fix things for you automatically the next
time you try to boot your virtual machine. If things are not fixed automatically, you should be asked to hit
ctrl-d (to continue) or enter the root password. If you get this message, enter the root password. You'll be
put in a shell from which you may run commands to try to fix things. Usually, running the fsck command
with the -y option ( fsck y ) will do it for you. After running this fsck command, just exit the shell to
restart the boot process.

If you really mess up your virtual machine, you can start over by removing the Gentoo virtual machine from
the library and the disk. This is done by right clicking on the Gentoo VM in the startup screen of the
VMware and selecting Delete VM from Disk. Then, unpack the compressed tar file again and start over.
WARNING: If you do this you will lose any and all work you have done in the virtual environment!