Case Study 1: AMOEBA

History of Amoeba

Amoeba originated at the Vrije Universiteit, Amsterdam, The

Netherlands in 1981 as a research project in distributed and parallel
computing. It was designed primarily by Andrew S. Tanenbaum and
three of his Ph.D. students. By 1983, an initial prototype, Amoeba
1.0, was operational.

Starting in 1984, a second group was set up. This work used Amoeba
3.0, which was based on RPC. Using Amoeba 3.0, it was possible for
clients in Tromso to access servers in Amsterdam transparently, and
vice versa.

Research Goals

The primary goal of the project was to build a

transparent distributed operating system.
An important distinction between Amoeba and
most other distributed systems is that Amoeba has
no concept of a home machine.
A secondary goal of Amoeba is to provide a
testbed for doing distributed and parallel
programming.

The Amoeba System Architecture

Processor pool
X-terminals

File
server

Print server

The Amoeba Microkernel

Client

Microkernel
Process management
Memory management
Communication
I/O

Server

Thread

The Amoeba Servers

Amoeba is based on the client-server

model.
Probably the most important server is the
file server, known as the bullet server.
Another important server is the directory
server, also known as the soap server.

Objects and Capabilities

The basic unifying concept underlying all

the Amoeba servers and the services they
provide is the object.
Each object is managed by a server
process.
Objects are named and protected by
capabilities.

A capability in Amoeba
Bits

48

24

48

Server port

Object

Rights

Check

Capability

When the server creates an object, it

generates a capability.
port

object

1111 1111

C (random number)

If a client wants to create a restricted

capability, go through the following.
On subsequent operations, the client must
present the capability to identify the object.

Restricted capability
Capability
port

object 1111 1111

New rights mask

0000 0001
Exclusive OR
One-way function

Restricted capability
port

object 00000001 f(C XOR 00000001)

Standard Operations
Age

Perform a garbage collection cycle: starts a new garbage collection cycle to get rid of old objects
that are no longer accessible.

Copy

Duplicate the object and return a capability for the copy: it is a shortcut that makes it possible to
duplicate an object without actually transferring it. Without this operation, copying a file would
require sending it over the network twice: from the server to the client and then back again.

Detroy

Destroy the object and reclaim its storage: deletes the object

Getparams

Get parameters associated with the server: allow the system administrator to read and write
parameters that control server operation. For example, the algorithm used to choose processors can
be selected using this mechanism.

Info

Get an ASCII string briefly describing the object

Restrict

Produce a new, restricted capability for the object

Setparams

Set parameters associated with the server: same as Getparams

Status

Get current status information from the server

Touch

Pretend the object was just used: tells the server that the object touched is still in used.

Process Management in Amoeba

A process is an object in Amoeba. When a

process is created, the parent process is
given a capability for the child process.
The child can be suspended, restarted,
signaled, or destroyed.

Process management is handled at three different levels in

Amoeba.
1. At the lowest level are the process servers, which are
kernel threads running on every machine.
2. At the next level up we have a set of library procedures
that provide a more convenient interface for user
programs.
3. Finally, the simplest way to create a process is to use
the run server, which does most of the work of
determining where to run the new process.

Process Descriptor
Architecture = 386
Capability for exit status
Segment descriptors

Text
PC1
PC2

Shared data

Thread 1 Thread 2 Thead 3

PC1
PC2
PC 3
SP1
SP2
SP 3
2
3
1
SP1

Stacks
SP2

Private data

PC3
Segments

SP3

Library procedures

exec: to do an RPC with the specified

process server asking it to run the process.
getload: returns information about the CPU
speed, current load, and amount of memory
free at the moment.
stun: to suspend a child process. Two kinds
of stuns are supported: normal and
emergency.

Threads

When a process starts up, it has one thread.

During execution, the process can create
additional threads, and existing threads can
terminate.
Three methods are provided for threads to
synchronize: signals, mutexes, and
semaphores.
All threads are managed by the kernel.

Memory Management in Amoeba

Amoeba has an extremely simple memory model.

A process can have any number of segments it
wants to have, and they can be located wherever
it wants in the process virtual address space.
Segments are not swapped or paged, so a process
must be entirely memory resident to run.
Each segment is stored contiguously in memory.

Mapped segments
Process virtual
address space

D
T

Memory segments
S

Communication in Amoeba

Amoeba supports two forms of

communication: RPC, using point-to-point
message passing and group
communication.

Remote Procedure Call

1.
2.
3.

RPC Primitives:
get_request indicates a servers
willingness to listen on a port.
put_reply done by a server when it has a
reply to send.
trans send a message from client to
server and wait for the reply.

Group Communication in
Amoeba

CreateGroup

Create a new group and set its

parameters

JoinGroup

Make the caller a member of a group

LeaveGroup

Remove the caller from a group

SendToGroup

Reliably send a message to all members

of a group

ReceiveFromG Block until a message arrives from a

roup
group
ResetGroup

Initiate recovery after a process crash

The Amoeba Reliable Broadcast

Protocol
1.
2.
3.
4.

5.

The user process traps to the kernel, passing it the message.

The kernel accepts the message and blocks the user process.
The kernel sends a point-to-point message to the sequencer.
When the sequencer gets the message, it allocates the next
available sequence number, puts the sequence number in a
header field reserved for it, and broadcasts the message (and
sequence number).
When the sending kernel sees the broadcast message, it
unblocks the calling process to let it continue execution.

System Structure
Application programs
A
Kernel

A
S

Kernel

Sequencer enabled

A
Kernel
Sequencer disabled

Broadcast network

An example
Sequencer machine

A
M25

A
M

Last = 24

M25

M25

Last=24

Last=24history S
M25

M25
Request
for 24

A
Last=23 M25
buffered

Senders action for sending

The sender sends a message to the sequencer and starts a timer:

(a) the broadcast comes back before the timer runs out. (normal
case).
the sender just stops the timer.
(b) the broadcast has not come back before the timer expires.
(either the message or the broadcast has been lost).
the sender retransmits the message.
if the original message is lost, no harm is done.
if the sender missed the broadcast, the sequencer will
detect the retransmission as a duplicate and tell the
sender everything is all right.

(c ) The broadcast comes back before the timer

expires, but it is the wrong broadcast. This occurs
when two processes attempt to broadcast
simultaneously.
If message A gets to the sequencer first, and is
broadcast. A sees the broadcast and unblocks its
application program. However, B sees As
broadcast and realizes it has failed to go first. B
will accept As broadcast and wait.

Sequencers action

If a Request for Broadcast arrives:

(a) check to see if the message is a
retransmission. If so, inform the sender that the
broadcast has been done.
(b) if the message is new, assign the next
sequence number to it, and increment the
sequencer counter by 1.
The message and its identifier are stored in a
history buffer, and the message is then broadcast.

Senders action for receiving

When the sender receives a broadcast:

(a) if the sequence number is 1 higher than
the most recent one (normal case). No
broadcast has been missed.
(b) if the sequence number is more than 1
higher (a broadcast has been missed), the
sender will send a message to the
sequencer asking for the lost broadcast.

Management of the history buffer

If the history buffer fills up, if the sequencer knows that

all machines have received broadcasts, say, 0 through 23,
correctly, it can delete these from its history buffer. There
are several mechanisms to discover this information:
(a) each Request for Broadcast message sent to the
sequencer carriers a piggybacked acknowledgement, k,
meaning that all broadcasts up to and including k have
been correctly received.
(b) the sequencer can broadcast a Request for Status
message asking the number of the highest broadcast
received in sequence.

Two methods for doing reliable

broadcasting
2

S
2

1. Message sent to the sequencer

2. The sequencer broadcasts it

1.
2.

A broadcast M
S broadcasts Accept

In method 1, each message appears in full

on the network twice. Each user machine is
interrupted only once.
In method 2, the full message appears only
once on the network. Each machine is
interrupted twice.

Summary of the protocol

This protocol allows reliable broadcasting

to be done on an unreliable network in just
over two messages per reliable broadcast.
Each broadcast is indivisible, and all
applications receive all messages in the
same order, no matter how many are lost.

Fault tolerance

The protocol is also fault tolerant.

When a processor crashes, sooner or later, some
kernel will detect that the crashed machine are
not being acknowledged. The kernel will mark
the crashed processor as dead and initiates a
recovery.
In phase 1, one process is elected as coordinator.
In phase 2, the coordinator rebuilds the group and
brings all the other processes up to date.

The protocol is also fault tolerant

coordinator
(a)

Sequencer dies

coordinator

40

43

41

44

40

5
Dead sequencer

coordinator
(b)

40

43

41

44

40

new sequencer
(c )

44

44

44

44

44

(a) The sequencer crashes (b) A coordinator is selected (c) Recovery

How does the coordinator get any message it has

missed if the sequencer has crashed?
The solution lies in the value of k. If k is 0 (nonfault tolerant), only the sequencer maintains a
history buffer.
If k >0, k+1 machines maintain an up-to-date
history buffer. If k machines crash, there is still
one left to supply the coordinator with any
messages it needs.

How to implement?
In method 2, when the sequencer sees a message,
M, that was just broadcast, it does not
immediately broadcast an Accept message.
Instead, it waits until the k lowest-numbered
kernels have acknowledged that they have seen
and stored it. Now k+1 machines have stored M
in their history buffers.

The Fast Local Internet Protocol

Amoeba uses a custom protocol called

FLIP for actual message transmission. The
protocol handles both RPC and group
communication and is below them in the
protocol hierarchy.
FLIP is a network layer protocol.

RPC

Group
FLIP layer

The Bullet Server

Create

Create a new file; optionally commit it as

well

Read

Read all or part of a specified file

Size

Return the size of a specified file

Modify

Overwrite n bytes of an uncommitted file

Insert

Insert or append n bytes to an

uncommitted file

Delete

Delete n bytes from an uncommitted file

The Directory Server

Create

Create a new directory

Delete

Delete a directory or an entry in a directory

Append

Add a new directory entry to a specified

directory

Replace

Replace a single directory entry

Lookup

Return the capability set corresponding to a

specified name

Getmasks

Return the rights masks for the specified entry

Chmod

Change the rights bits in an existing directory

entry

The Replication Server

The Run Server
The Boot Server
The TCP/IP Server
Disk server
I/O server
A time-of-day server
A random number server
Swiss Army Knife server