Faculty of Computer Science

Real-Time Group

Diploma Thesis

Analyzing Real-Time Behavior of Flash Memories

Author: Daniel Parthey

Date of Birth: March 10, 1982 in Karl-Marx-Stadt
Email: daniel.parthey@informatik.tu-chemnitz.de

Date of Issue: October 2, 2006

Date of Submission: April 2, 2007
Supervisor: Dr. Robert Baumgartl
Task Description

Flash Memories have an increasing importance for the construction of mechanically robust embedded
computer systems and consumer electronics. For consumer applications, at least five different solutions
exist: Compact Flash (CF), Sony Memory Stick (MS), Secure Digital (SD), Multimedia Card (MMC)
and the xD-Picture Card. Of every type, different generations with different technical parameters (access
speed, capacity) exist.
Usually, the embedded controller of the medium is responsible for wear-levelling and error correction.
If no controller exists, the file system must take care of these aspects. Therefore, a number of specialized
file systems have been developed, among them the Journalling Flash File System (JFFS2) and the Yet
Another Flash File System (YAFFS2).
The aim of this thesis is to develop methods to characterize the timing of access operations for flash
memories of different types and with different file systems. Amongst other the work should present
detailed analysis of the following aspects:

• basic write and read access operations to different media types

• performance with random and sequential access patterns

• achievable and sustainable throughput including worst cases

• mount times for media of different capacity and file systems

• evolution of certain parameters for media of different age

• worst case behavior of different file systems

• influence of file system on performance and timing predictability

The term real-time signifies that not only an average value has to be obtained for every parameter, but
the worst case timing is interesting as well.

2
 Abstract

Flash memories are used as the main storage in many portable consumer electronic devices because they
are more robust than hard drives. This document gives an overview of existing consumer flash memory
technologies which are mostly removable flash memory cards. It discusses to which degree consumer
flash devices are suitable for real-time systems and provides a detailed timing analysis of some consumer
flash devices. Further, it describes methods to analyze mount times, access performance and timing
predictability of flash memories. Important factors which influence access timings of flash memories
are pointed out and different flash file systems are evaluated with regard to their suitability for real-time
systems. Some remaining problems of existing flash file system implementations concerning real-time
use are discussed.
Contents

1 Introduction 1

2 Basics 2
2.1 Flash Memory Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.2 Media Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.3 Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.4 Flash Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.5 File Systems for Flash . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3 State Of The Art 18

3.1 Storage Benchmark Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.2 Previous Benchmark Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4 Flanatoo - A Flash Analysis Tool 21

4.1 Basic Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.2 Time Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.3 Direct Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.4 Supported Benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5 Removable Flash Card Analysis 25

5.1 Test Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
5.2 Test Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
5.3 Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

6 VFAT Mount Times 35

6.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
6.2 Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
6.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

7 Real-Time Support for Flash File Systems 37

7.1 Problems of conventional flash file systems . . . . . . . . . . . . . . . . . . . . . . . . 37
7.2 Implementation Proposals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

i
8 Further Work 39
8.1 Collect more data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
8.2 Worst-case analysis of flash file systems . . . . . . . . . . . . . . . . . . . . . . . . . . 39
8.3 Real-time support in flash file systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

9 Conclusion 41

A Glossary 44
1 Introduction

Over the last decade, flash memories have become cheaper and continuously enjoyed increasing impor-
tance. They enable the user to quickly transfer data from one device to another without the need for a
computer network. They are compatible with a wide range of platforms because most of them support
the FAT/FAT32 file system which has become a standard for removable flash devices.
Flash based USB drives have almost completely replaced the conventional floppy disk because they
are small in size, easy to handle and have a much higher storage capacity.
Flash technology is superior to conventional random access memory because it is non-volatile. This
means that none of the already stored data is lost when power is turned off or a power failure occurs. In
contrast to hard disk drives, flash storage is silent, robust against shock, and the raw device is accessible
shortly after power is turned on because there are no moving parts inside of a flash memory. A hard disk
drive needs several seconds after power-on until its disks have spun up and its data is accessible [1].
With flash memory instead, no disks have to be spun up, no head has to be moved and there is no need
to wait for the disk to spin until the data appears under the head because flash memory is implemented
using transistor cells. This makes flash an interesting technology for building robust and fault-tolerant
embedded systems which need a quick startup, but this requires some further investigation on the proper-
ties of flash devices especially with regard to their suitability for real-time systems.
Chapter 2 gives an overview of the properties of common removable and non-removable flash tech-
nologies. It also describes interfaces which can be used to connect flash memory devices to a host.
Chapter 3 presents a collection of tools which can be used to analyse flash devices and file systems. Fur-
ther, it points to previous benchmarks which analysed the average read and write performance of many
different flash devices and card readers. Chapter 4 describes the usage and functionality of a tool which
was developed for this work to perform specific tests on flash cards. Chapter 5 describes the setup and
discusses the results of the benchmarks which were performed on different flash cards. It highlights the
most relevant tests and anomalies. In order to store files on flash devices, a file system is required, and
when it comes to flash file systems, the mount time is often an issue. So the mount times of the VFAT
file system, which is normally used on removable flash devices, were also measured. The results of these
tests are described in chapter 6. This chapter is directly followed by chapter 7 which explains the main
problems of conventional flash file systems and why they should not be used in hard real-time systems.
Chapter 7 also highlights issues which should be considered in the design of new real-time file systems
for flash devices. Chapter 8 summarizes work which still needs to be done and highlights topics for
further research.

1
2 Basics

In order to analyse the behavior and anomalies of flash memories, one has to understand the technology
behind them. The following sections give an overview of basic flash technologies and common types of
flash memories.

2.1 Flash Memory Technologies

Flash is a kind of electrically erasable read only memory (EEPROM). Its memory cells are based on a
metal-oxide-semiconductor field-effect transistor (MOSFET). These MOSFETs are extended by a so-
called floating gate between control gate and substrate which works as a trap for electric charge and
prevents leakage of electrons [2]. This makes the cell non-volatile and enables it to keep its state (1 or 0)
over a long period of time without any refresh or applied voltage. Such memory cells are grouped into
larger erase units (around 512 KBit). The erased logical state of each cell is one and write operations
can only switch single bits from one to zero by applying a negative voltage (10 to 13 Volts). If bits of an
erase unit need to be changed from zero to one, the whole erase unit must be cleared and set to logical
one. This is where the name flash comes from because clearing one erase unit is like a flash of light
erasing all the data in the unit [2].
Unlike other storage technologies, flash memory differentiates between read, write and erase. The
timing of raw flash chips can be characterized by three basic values: maximum read cycle time, maxi-
mum write cycle time and maximum block erase time. Other important characteristics which influence
performance are the flash type (NAND, NOR, etc.), the physical sector size, physical erase block size
and the total size of the flash chip. For complex flash devices like flash cards or USB drives, the number
of flash chips and the type of controller can also impact timing.

2.1.1 NOR Flash

Some of the first and smaller flash devices (up to 8 MiB) used NOR flash where the transistor cells
are connected in parallel (OR). Each bit can be separately accessed through a logic grid. This allows
for arbitrary write operations where bits can be changed from one to zero, until all cells of the whole
erase unit (usually 128 KiB) are set to zero because the cells do not interfere with each other. Being
linearly accessible makes NOR flash well suitable as execute in place (XIP) program memories for micro
processors. This is why NOR flash is still popular to boot embedded devices. The NOR technology can
rarely be found in removable flash devices because of its low storage density and relatively high cost per
chip [2], [3].

2
2.1.2 NAND Flash
The NAND flash technology is used in most current devices and connects the transistor memory cells in
series. This makes a higher data density and lower cost per chip possible. A deficiency of this approach
is that direct bit access is not possible. Any cell of a series which shall not be modified has to be
masked with an offset voltage. Therefore, the data is written to an internal register which takes care of
addressing the flash cell. Afterwards, a write command on the command bus of the flash device initiates
a data transfer from the register to the flash cell [2]. Because of electron leakage in neighbour cells
when writing a certain cell, NAND flash only allows a maximum number of about 10 write operations
to a single erase unit until the whole erase unit must be erased. Erase units in NAND flash are usually
divided into pages of 512 bytes. Each of these pages allows storing 16 extra bytes of out-of-band (OOB)
information, like error correction codes (ECC), erase counters, and other meta-information [3]. NAND
flash always requires ECC data since this technology is prone to page errors by design.
There are two different types of NAND flash which heavily affect the speed and capacity of flash
devices. A single level cell (SLC) can only store a single bit in its floating gate, the multi level cell
(MLC) technology allows storing several bits in one cell. This can either be accomplished by using
several floating gates or different levels of amperage to encode different states [2]. The x4 NAND flash
technology [4] from m-systems allows storing 4 bits in each flash cell.

2.1.3 OneNAND Flash

OneNAND [5] flash is a relatively new technology and has entered the market in 2003. It tries to combine
the advantages of NOR and NAND flash and eliminate the need for an expensive NOR flash chip. Each
OneNAND device contains a NAND flash chip, a NOR interface logic, and an SRAM buffer up to 5 KiB
size. This way the device provides a NOR interface to the outside world which allows transparent XIP.
Since the internal NAND flash allows high bandwidths up to 128 MB/s, OneNAND can reach much
higher sustained read and write performance in comparison to conventional NOR flash. Access latencies
might be better for NOR flash shortly after the device is powered on, but once the boot code is buffered
in SRAM, the OneNAND device is able to provide better access latencies than any NAND or NOR flash
chip. Since OneNAND flash has no moving parts and is very power-efficient, it will also be used in
hybrid hard disk drives to save power and store a certain amount of data while the disks are not running.

2.2 Media Types

There is a great variety of interfaces and protocols which provide access to the internal flash chips in
different ways. The following sections will describe the most common interfaces for removable and
embedded flash devices.

2.2.1 Smart Media (SM)

Smart Media cards are based on NAND flash memory compatible to the Toshiba TC58 series. They have
been introduced by Toshiba in 1995. Smart Media devices do not have an integrated controller, therefore
any additional features like wear levelling (as described in section 2.4.1) or flash file systems need to

3
be implemented in external hardware or software. Originally, they were called Solid State Floppy Disk,
being only a bit smaller than usual floppy disks. Later, they were renamed to Smart Media. With the help
of an adapter, the 22-pad-connector of Smart Media devices can be connected to a floppy cable. There
are also adapters which connect them to a PCMCIA slot or a Compact Flash Type II slot. A memory
card reader can connect them via a controller to the Universal Serial Bus or IEEE-1394 (FireWire) port.
A deficiency of the Smart Media technology is that its maximum capacity is limited to 128 MiB, which
resulted in its replacement by the xD-Picture Card.

2.2.2 xD-Picture Card (xD)

The xD-Picture Card is a removable flash memory card with a proprietary interface, which has been
introduced by Olympus and FujiFilm in 2003. It removes the capacity limit of Smart Media devices
by implementing a memory logic which supports a theoretical maximum capacity of 8 GB. Like Smart
Media devices, xD-Picture Cards do not have an integrated controller. In order to access data stored on
the internal NAND flash device through the 18-pin connector of the xD-Picture Card, the host device
must support the Single Level Cell or Multi Level Cell technology.
Currently, there are three types of xD-Picture Cards: The classic type without any label supports 16
to 512 MB, the Multi Level Cell type "M" supports 256 MB up to 2 GB, and the High Speed type "H"
supports capacities from 512 MB up to 1 GB.

2.2.3 Memory Stick (MS)

The Memory Stick has been developed by Sony in 1998 [6]. According to the Memory Stick Pro Speci-
fication [7] it has an integrated controller which implements an ECC error detection and correction
mechanism to detect bad blocks and correct bit errors which are both common for NAND flash. It does
not provide a default mass storage device interface, but a Memory Stick bus.
The following paragraphs will give a short overview and try to clarify the names of the different
Memory Sticks which can be found on the market.
The standard Sony Memory Stick is 50x21.5x2.8mm large [8]. It is narrower than a Smart Media or a
Compact Flash card and only supports capacities from 4 MB to 128 MB [9]. The interface provides one
serial data line with a bus clock of 20 MHz [10] which corresponds to 20 MBit/s and therefore limits the
maximum theoretical transfer rate to 2.5 MB/s.
The Sony Memory Stick Select (MS Select) was an attempt to duplicate the capacity of a Standard
Memory Stick by mechanically switching between two memories on one stick. Both halves of the stick
act like a single Memory Stick. The advantage of this approach is that it is supported by any device
which supports the standard Memory Stick, without the need for any new firmware or controller, but
because the user needed to switch back and forth between the two halves, it has not prevailed and was
superseded by superior technologies.
The introduction of the Memory Stick Pro (MSP) improved the maximum bandwidth and capacity by
implementing a new interface with 4 parallel lines with a bus clock of 40 Mhz. This allows a maximum
theoretical bandwidth of 160 Mbit/s or 20 MB/s [10]. The Memory Stick Pro specification [7] defines
a maximum possible capacity of 32 GB. The specification recommends the following layers for devices
which make use of Memory Sticks:

4
 • Application layer takes care of file contents

• File management layer handles the FAT file system and defines the logical device structure

• Protocol layer provides a serial or parallel interface to the stick and handles incoming commands

• Physical layer specifies physical and electrical properties of the Memory Stick bus

Only the lower three layers are covered by the specification. It is not possible to bypass the protocol layer
in order to access flash memory directly. Memory Stick Pro are always controlled through a command
interface provided by a controller. There are three types of commands:

• Transfer Protocol Commands (TPC) are used to control the memory stick and access the data
buffer and registers of the Memory Stick Pro.

• Memory Access Commands are used to access the flash memory and allow READ/WRITE access
to the user or information block area. This set of commands also includes an ERASE command
which deletes data from the current address in the user area and a STOP command which terminates
any of the above operations.

• Function Commands provide special functions to control the device. The FORMAT command
formats the whole device and the SLEEP command allows the host controller to put the device
into a low power consumption state.

Memory Stick
Interface
Controller
VSS
Vcc
BS
Register
Serial
DATA1
Interface
SDIO/
DATA0
Data Memory
DATA2 Buffer Interface Flash Memory Flash
INS
Sequencer Interface
Memory
DATA3
Parallel ECC
SCLK Interface
OSC
VCC Controller

VSS

Figure 2.1: Memory Stick Pro Scheme

Both Memory Stick and Memory Stick Pro are available in a smaller form factor of 31x20x1.6mm.
These versions have a suffix "Duo" and are called Memory Stick Duo (MSD) and Memory Stick Pro Duo
(MSPD). Both Duo variants are usually shipped with simple plastic adapters to fit them mechanically

5
into normal sized Memory Stick slots because their electrical interface is identical to the interface of
their larger counterparts.
Memory Stick Micro has been introduced in the beginning of 2006. It also uses the Memory Stick Pro
technology, but with a size of only 12.5x15x1.2mm it is even smaller than Memory Stick Pro Duo. It
allows high capacities up to 32 GB as well as high transfer rates up to 20 MB/s (like Memory Stick Pro)
and is additionally extremely small sized.

2.2.4 MultiMediaCard (MMC)

The MultiMediaCard (MMC) was developed by Ingentix and SanDisk in 1997. It can be accessed using
one of two different protocols which are described in the MultiMediaCard Specification [11]: The default
bus protocol is the MultiMediaCard bus and the other protocol is the Serial Peripheral Interface (SPI).
One of these two has to be selected when the card is powered on, and the protocol cannot be changed
without power-cycling the card.
The MMC bus protocol allows three different bus clock frequencies. The frequencies and bus protocols
supported by the host’s and card’s controllers have to be negotiated when the card is powered on [2].
All Multimedia Cards are backwards compatible with the x1 mode. This mode uses a single data line
running at a maximum clock speed of 20 MHz, which results in a theoretical maximum bandwidth of
2.5 MB/s. The MMC plus standard introduced the x4 mode using four parallel data lines running at a
clock frequency of at least 26 MHz (which allows a maximum bandwidth of at least 13 MB/s) and the
x8 mode using eight parallel data lines with a clock frequency of 52 MHz (which allows a maximum
bandwidth of 52 MB/s). These high-speed modes have been developed to provide a minimum bandwidth
for current multimedia applications like recording of video streams and for multi-shot digital cameras
which have to store a lot of data in a short time.
All MMC cards support a voltage of 3.3V. They require a minimum voltage of 2.7V and allow a
maximum voltage of 3.6V. Cards with a DV label are dual voltage cards [11] and can be run with only
1.8V in a range from 1.65V to 1.95V.
The standard form factor of MMC cards specifies dimensions of 24x32x1.4mm and a connector with
7 (for MMC) or 13 (for MMC plus) pins. The MMC plus standard allows higher transfer rates when
using the x4 or x8 bus protocol. Smaller form factors have been developed to fit the cards into small size
mobile devices. The reduced size (RS) form factor is a half-sized version of the standard MMC card with
dimensions of 24x18x1.4mm. Reduced size cards use the same connector as standard sized cards and
provide full electrical compatibility. A passive extender, which consists of a piece of plastic and a small
clip, can be used to put reduced size MMC cards into standard sized MMC slots. The MMC Mini form
factor (20x21.5x1.4mm) and the MMC Micro form factor (14x12x1.1mm) use 11 pins. MMC mobile
sounds like another form factor, but in fact it is just a short marketing term for MMC DV RS Plus, which
stands for a small reduced-size multimedia card with dual-voltage and high bandwidth support [2].
The MMC bus interface, as well as reading, writing and erasing of blocks, is handled by an internal
controller and completely independent from the flash technology used internally. The MMC standard
allows host software to work smoothly even with freshly developed MMC cards using new internal
technologies.
When data is written to an MMC card, the controller may induce a read after write and an ECC

6
error correction on newly erased units in order to verify if the data was written correctly. The internal
controller also implements an error management algorithm to correct and replace bits or even whole
sectors by reserved ones, which at least applies to the Samsung MMC device MC56U032NCFA [12].
MMC cards may have an automatic sleep mode to reduce power consumption when no command was
received for 5 milliseconds. This is why MMC cards are in sleep mode most of the time when they are
not being accessed. The sleep mode is transparent to the host, and the device automatically wakes up
when any command is received. Even though the duration between sleep mode and ready state might be
interesting to real-time developers, it is not mentioned in Samsungs data sheet [12].
The MMC specification [11] recommends that external MMC adapters should implement caching in
order to increase throughput by speeding up random access or read-modify-write operations, where a
cache can be very efficient on frequent updates. This recommendation indicates that MMC cards most
probably contain only a small cache or no cache at all.

2.2.5 Secure Digital Memory Card (SD)

The secure digital card was invented by SanDisk in 2001. SanDisk is a member of the SD Card Associa-
tion which defines today’s standards for secure digital cards. All secure digital cards have a standardized
9-pin connector and contain an integrated controller which implements the SD bus protocol, SD spe-
cific security features like secure digital music initiative (SDMI), and which does a simple form of wear
levelling (see section 2.4.1) by distributing write operations across the device.
In a flash memory overview article [2], the c’t magazine states that the controller provides up to three
different bus modes: The first one is the serial peripheral interface (SPI), the second one is a 1 bit SD
transfer mode and the last one is a 4 bit SD data transfer mode. While SPI mode and 1 bit SD data transfer
mode (x1) are mandatory for all cards, the 4 bit SD data transfer mode (x4) is optional for low speed
cards and mandatory only for SD high speed cards [13]. The x4 mode supports two clock frequencies, a
normal frequency of 25 MHz which allows a maximum theoretical bandwidth of 12.5 MB/s and a high
speed frequency of 50 MHz which allows a maximum bandwidth of 25 MB/s.
SD card slots are backwards compatible with MMC because the SD specification [14] is based on the
MMC standard. Therefore, MMC cards work in SD slots, but not the other way round. To prevent SD
cards from accidentally being put into an MMC slot, SD cards are usually thicker than MMC cards, but
there are SD card form factors which are only 1.4 mm thin. These cards will not work in an MMC slot,
in spite of the fact that they would mechanically fit in.
SD cards are specified [14] in three different form factors. The standard SD card has dimensions of
24x32x2.1mm, MiniSD cards are only 20x21.5x1.4mm large and the size of MicroSD is 11x15x1mm
which approximates the size of a fingertip. Therefore, MicroSD cards can be used in a wide range of
small embedded devices like modern mobile phones.
To provide a wide compatibility between devices supporting the SD protocol, the SD specification
requires devices to support for the FAT16 file system and the SD High Capacity (SDHC) specification
requires support for the FAT32 file system. Cards larger than 2 GByte which are not pre-formatted with
the FAT32 file system do not comply with the SDHC standard because SDHC devices must support
FAT32, but are not required to support any other file system [2].

7
2.2.6 Compact Flash (CF)
Compact Flash Cards were developed by SanDisk in 1994 as one of the first flash devices. They have
become a quasi standard for professional digital photography because of their high capacity, high speed
and strong mechanical robustness. Nevertheless, they are likely to be replaced by Secure Digital cards
because SD cards are smaller and becoming cheaper than CF cards. The interface of compact flash cards
is compatible to ATA and consists of 50 pins, of which 40 are an ATA connector and 10 are reserved for
alternative operating modes and the power supply.
There are two form factors for compact flash cards: Type I with dimensions of 42.8x36.4x3.3mm is
usually used for flash memories, because most digital cameras merely contain a thin CF slot. The thicker
type II with the dimensions 42.8x36.4x5.0mm is common for devices which need more space. Exam-
ples are devices like WLAN, ethernet and bluetooth adapters or barcode scanners which are connected
through the Compact Flash In/Out (CFIO) interface. Hard disks called Microdrives also require a Com-
pact Flash type II slot since they would not fit in the smaller type I slot. In fact, Microdrives do not
contain any flash memory and this is why they only shall be of minor interest in this document.
Four Compact Flash interface standards have been developed over time, and all of these standards are
fully compatible with each other. The CF 1.0 standard allows programmed I/O (PIO) mode 2 with a max-
imum transfer rate of 8.3 MByte/s, the CF 2.0 standard allows PIO 4 with a transfer rate of 16.6 MByte/s,
CF 3.0 allows using the UDMA-66 mode with a maximum theoretical transfer rate of 66 MByte/s, and the
current CF 4.0 standard [15] which supports UDMA-133 and a maximum transfer rate of 133 MByte/s
even exceeds the USB high-speed bandwidth of 400 MBit/s. Therefore, CF 4.0 based SanDisk Extreme
IV cards can only unfold their full performance when used in a card reader which allows a maximum
theoretical bandwidth of at least 100 MByte/s, e. g. a card reader supporting IEEE 1394b (FireWire-800
or FireWire-1600) [2].

2.2.7 Disk On Chip (DOC)

The DiskOnChip technology was developed by m-systems in 1997 [16] which has become a subsidiary
of SanDisk in 2006 [2]. These integrated devices with 32 pins contain NAND flash memory and a
microcontroller for bit error detection and correction (EDC). The internal controller also acts as glue
logic to make a small memory window of the NAND flash directly visible and addressable like NOR
flash [17]. This helps to reduce cost by omitting the NOR flash, but still allows booting from the device.
DiskOnChip devices are independent from the flash technology used internally because the controller
hides the specific NAND flash interface from the outside [18]. Specific information about mDOC devices
like the format version and the type of physical flash media can be extracted using the MS-DOS utility
dinfo provided by m-systems [19].

2.2.8 USB Flash Drive

USB flash drives contain a flash memory chip and an integrated controller placed together on a printed
circuit board. Since there is no proper standard for the shape or weight of USB flash drives, there are
various drives with different sizes, colours, controllers and flash memory technologies, but all have one
thing in common: All USB drives have a male USB connector and implement the USB mass storage

8
device class (MSC) which means that the drive exports a linear device which hides the physical block
structure, number of flash chips, erasure of blocks, flash addressing and wear levelling from the host
system. This indirection allows using the same generic mass storage drivers for all USB drives but
makes it more complicated to analyze the behavior of USB flash drives.
Performance of USB drives may be influenced by the USB chipset on the mainboard, but for modern
mainboards this is rarely the case, according to a c’t article about USB flash drives [20]. The USB
protocol overhead may also slow down the transfer. An important parameter for USB keydrives is the
number of flash chips used because this determines whether the drive can be run in dual-channel mode
with interleaved memory access (which can almost double the effective transfer rate) or needs to be run
in single-channel mode. Another bottleneck for USB keydrives is the integrated mass storage controller
because vendors tend to keep their devices as cheap as possible and therefore use cheap controllers.

2.2.9 Comparison of Flash Devices

The following table shortly summarizes the developer companies, interfaces and features of the most
common flash memory devices available. All of them are removable devices, except DiskOnChip (DOC).

Device Developer Since Type Size Controller Bus PINs

in mm
SM Toshiba 1995 NAND 45/37/0.76 none SM 22
xD Olympus 2003 MLC NAND 25/20/1.7 none xD 18
Fujifilm
MS Sony 1998 MLC NAND 50/21.5/2.8 ECC, MS (TPC) 10/11
31/20/1.6 Buffer
12.5/15/1.2
MMC Ingentix 1997 NAND 24/32/1.4 CRC MMC/SPI
SanDisk 24/18/1.4 7/10/13
12/14/1.1
SD SanDisk 2001 NAND 24/32/2.1 CRC, SD/SPI 9
20/21.5/1.4 ECC
11/15/1.0
CF SanDisk 1994 NAND 42.8/36.4/3.3 CRC, ATA 50
(rarely NOR) 42.8/36.4/5.0 ECC
DOC M-Systems 1997 NAND variable ECC SRAM 32
USB-Drive M-Systems 1998 NAND variable MSC USB (MSC) 4

Table 2.1: Comparison of flash memory interfaces

2.3 Interfaces
Card readers and flash drives may be connected to the host system in various ways. The following
sections will shortly summarize the most common interfaces and their capabilities.

9
2.3.1 USB - Universal Serial Bus
A popular interface which can be found in all modern PCs and some embedded devices is the universal
serial bus interface. The latest standard, USB 2.0, supports transfer rates up to 480 MBit/s. The actual
transfer rate of a transmission is influenced by transfer mode, bus transaction delays and USB protocol
overhead. The calculation of worst case transaction delays and protocol overhead can be obtained from
the USB specification [21]. Chapter 5 of the specification describes the four different transfer modes sup-
ported by USB: Control transfers are bidirectional, best-effort transfers where a packet has a maximum
payload size of 64 byte and every frame is acknowledged. These are intended to detect and configure
USB devices but not to transmit real payload data. Isochronous transfers use a synchronized clock to
deliver data in a timely manner in preference to correctness. This mode is best suited for multimedia
streams, which require data to be either delivered in time or discarded if it contained one or more er-
rors. Interrupt transfers provide a guaranteed maximum service period for transmission attempts, which
is useful for real-time applications. After the desired period, the transmission is either finished or has
failed, and it may be retried in the next period, to which the same guarantee applies. The last transfer
mode is called bulk transfer. This mode does not provide any timing guarantee, but tries to take advan-
tage of the highest bandwidth available. So the recommendation for real-time programmers is to make
use of either isochronous or interrupt transfer, but not bulk transfer.
The maximum theoretical transfer rate of USB 2.0 is high enough to access most of the currently
available flash memories at their maximum transfer rate. USB 2.0 can still be a bottleneck for fast flash
memory devices which comply with the Compact Flash 4.0 standard as explained in section 2.2.6. So
there is a need for alternative interfaces to connect high-speed card readers to a host system.

2.3.2 FireWire - IEEE-1394

Another high-speed serial bus is FireWire. It is hot-pluggable and well suited for transfer of multi-
media streams. The first IEEE-1394 standard [22] from 1995 already specified transfer rates of up to
400 MBit/s. According to chapter 3.6.4 of the specification, FireWire allows isochronous bus arbitration
and bandwidth allocation in order to guarantee a timely delivery of data. In 2000, the first amendment
IEEE-1394a [23] was added. The latest standard IEEE-1394b [24] appeared in 2002. This standard in-
troduced specifications for maximum theoretical transfer rates of up to 800 and 1600 MBit/s. In contrast
to USB 2.0, FireWire-800 and FireWire-1600 card readers are able to fulfill the bandwidth requirements
of storage cards following the Compact Flash 4.0 specification [15].

2.3.3 ATA
Compact Flash cards can be connected to an ATA port using a simple Compact Flash to ATA adapter and
communicate by using their True IDE I/O Transfer Function as specified in chapter 4.7 of the Compact
Flash specification [15]. It is important to note that the compact flash interface is hotpluggable, but the
ATA interface is not. So a compact flash may not be unplugged from a CF/ATA adapter while connected
to the ATA port of a running PC, otherwise the system might lock up.

10
2.3.4 Memory Technology Devices
Some flash memories, especially on-board devices and devices connected via serial peripheral interface,
can be directly accessed using an appropriate hardware driver. Such a driver can be either generic code
like the configurable MTD NAND driver which works out of the box, or a vendor customized driver
which can perform better on the specific flash chip [25].
Since neither block nor character devices differentiate between read, write and erase operations, Linux
supports a special Memory Technology Device (MTD), which provides a function set to access all fea-
tures of flash memories. The MTD subsystem in the operating system provides generic access to the
out-of-band and user area of flash memories while limiting the hardware specific driver part to the smal-
lest amount possible and keeping driver development simple. The MTD interface allows application
developers to create special flash file systems and block mapping layers like UBI (cf. section 2.4.3).
Software based on the MTD subsystem is independent from specific hardware while still being able to
access flash specific functions. There is a utility mtd_debug which uses the MTD subsystem to display
detailed information about the underlying memory technology device, e. g. the type of flash (NAND or
NOR) and the erase unit size. This information is gathered from the flash chip using the Common Flash
Memory Interface (CFI) [26].

2.3.5 Summary
Timings can only be guaranteed if all layers from the physical flash memory device layer to the applica-
tion layer specify and guarantee worst case timing constraints. Therefore, the data bus and its arbitration
and transfer modes need to be carefully selected in order to guarantee the bandwidth and latency that a
real-time application or memory device requires. Hard real-time systems additionally require real-time
drivers for the chosen interface which are able to guarantee maximum latencies.
Figure 2.2 shows a model of several possible flash architectures in which Linux is used as the operating
system. It illustrates the involved layers and their purposes in each architecture.

2.4 Flash Management

2.4.1 Wear Levelling
Flash devices can be rewritten only a limited number of times because the oxide layer in the memory
cells between control gate, floating gate and substrate gets worn out [2]. After a sufficient number of
erase cycles, blocks will become erroneous or statistically unreliable. Vendors guarantee a minimum
erase count from 10,000 up to 1,000,000 erase cycles, depending on the technology. In fact, torture tests
[27] have shown that modern OneNAND flash memories survive about 6 million erase cycles until the
first blocks become erroneous, which is about six times higher than vendors usually guarantee.
In order to prolong the lifetime of flash memories, a technique called "wear levelling" exists. It equally
distributes write operations across the whole device even if the same storage address is accessed again
and again. Wear levelling algorithms may move infrequently changed data (or static data) to already
worn out blocks in order to make use of relatively vital eraseblocks for data which has changed recently.
Paper [28] describes different strategies for wear levelling, but mainly concentrates on algorithms and

11
 Layers SM / xD Flash Card USB Drive MTD UBI

Application Application Application Application Application Application

Block Block Block Flash File UBI Flash FS
File System
File System File System File System System, or Block FS
Block UBI Block
Flash Management
OS Mapping Mapping
Linux MTD MTD
Device Block Device Block Device Block Device
Bad Block H. Bad Block H.
Driver MSC MSC MSC NAND, NOR NAND, NOR

Card Card
Driver NAND
Reader Protocol
Firm-
ware Flash Flash Flash
Flash
Management Management Management
Con-
troller Driver NAND NAND
NAND, NOR, NAND, NOR,
Flash Hardware NAND Flash NAND Flash NAND Flash
OneNAND OneNAND

Flash Management: Bad Block Handling + Wear Levelling + Block Mapping Algorithm (surrounded by black line)
UBI: Logical Volume Manager for Embedded Flash
MSC: Mass Storage Device Class (over USB or Firewire IEEE 1394)

Figure 2.2: Layer model of Linux flash memory architectures

data structures to increase flash lifetime and does not deal with real-time aspects.
Most vendors implement wear levelling techniques on the internal microcontroller of their flash memo-
ries, but keep source codes and algorithms secret. Smart Media and xD-Picture Cards do not provide
any wear levelling because they do not include a controller. The simplest approach to implement wear
levelling in software is to emulate a block device using a flash translation layer (FTL) as described
in section 2.4.2. Another approach is to develop specialized flash file systems which handle the wear
levelling problem and care about the eraseblock size and other specifics of the flash memory, but their
use only makes sense when no additional wear levelling mechanisms like flash translation layers or
hardware based wear levelling are in place. Stacking several wear levelling layers would only decrease
performance, so it should be implemented in exactly one layer.
DiskOnChip devices like SanDisk mDOC H3 implement the TrueFFS file system as firmware on the
integrated microcontroller. Even though SanDisk advertises all their DiskOnChip devices with buzz-
words like wear levelling, the mDOC H1 only implements the TrueFFS file system as a software driver
on the host system, in order to keep the hardware price low.

2.4.2 Flash Translation Layer

The flash translation layer (FTL) [29] is a bridge between a Linux block device and a memory technology
device. It is situated above the physical layer and is part of the PCMCIA specification because it was
designed for PCMCIA flash devices on laptops.
The FTL transparently maps blocks of a block device to physical erase units on the flash device. It
takes care of wear levelling and therefore allows the transparent use of traditional block-oriented file
systems on raw flash memories which do not provide any flash specific wear levelling mechanisms.
This block device approach is highly inefficient because no matter how little data is written, any write

12
access to the device requires erasing a whole flash erase unit. If the data units written are smaller than
one erase unit of the flash device, and the file system is not aware of the underlying hardware, it uses
the hardware in a very inefficient way. The FTL is now deprecated because of the above problems
and patent issues. It is not further developed and only supported by the Linux kernel for backward
compatibility [30]. Modern approaches use log-structured flash file systems instead, which directly take
care of wear levelling and other specifics of the underlying flash hardware. Some of these file systems
will be presented in section 2.5.

2.4.3 Unsorted Block Images

Unsorted block images (UBI) is a new logical volume and block management layer which relies on the
memory technology device layer. It hides physical flash type, wear levelling, bit flipping issues and bad
physical erase blocks (PEB) from upper layers and transparently provides generic UBI volumes. Such
a UBI volume is seen by the application or file system as consecutive logical erase blocks (LEB) which
support three flash operations: read, write and erase. On top of UBI volumes it is easier to implement
block device emulation layers (similar to FTL) or flash file systems because the upper layers do not have
to care about wear levelling or other specifics like the size of physical erase blocks. Further information
about UBI concepts can be found in the UBI design paper [31] and the UBI Frequently Asked Questions
[32].

2.4.4 Erase before Write

If a flash device makes use of the erase before write technique, any write access to a sector (512 bytes)
on a flash device, causes an erase unit (16 to 128 Kilobytes) to be erased [33]. This technique has good
timing predictability because the worst case delay for writing a sector is always the delay of erasing one
erase unit plus the delay of modifying the data in the sector.
MMC devices like the Samsung MultiMedia card MC56U032NCFA [12] implement the erase before
write technique. Section 2.6.5 (Memory Array Partitioning) of the corresponding data sheet explains
that these cards always perform an implicit erase before write. There are good reasons to do this because
NAND flash only allows a few write operations to a sector until it needs to be erased and flash generally
does not allow resetting single bits to logical one without erasing the whole unit.

2.5 File Systems for Flash

The following section will shortly describe file systems commonly used on flash devices. FAT and FAT32
are traditional block file systems which were originally designed for use on hard drives, but are now also
widespread in flash storage and implemented in embedded devices which provide mass storage slots.
The other file systems ExFAT, JFFS2, YAFFS2 and LogFS were specifically designed for use on flash
devices.

13
2.5.1 FAT/FAT32
The most common file system used on removable flash memories is the File Allocation Table (FAT or
FAT32) file system which is supported by many platforms and provides long filename support. For
compatibility reasons, most digital cameras and operating systems can read and write this file system
and it is used on a lot of removable flash devices, even though it does not care about wear levelling. This
is not an issue because most of today’s removable flash devices do wear levelling in firmware and can be
seen as a block-oriented mass storage device to the outside world.
FAT and FAT32 do not provide any access restrictions, therefore anyone with access to a FAT file
system can create, alter or delete any file on it. There are some security mechanisms defined in the flash
memory standards which work on the physical device layer, but these approaches have not become popu-
lar yet. There are also some memory devices (e. g. USB keydrives) which are bundled with software that
adds security on the file system layer by creating an encrypted and an unencrypted partition. This vendor-
specific software often runs only on a single operating system which nullifies the wide compatibility of
the FAT file system and makes these partitions unreadable on other operating systems or devices.
With regard to compatibility, FAT is the best choice for most modern flash memories which do internal
wear levelling, but when using the FAT file system on a flash memory device without any wear levelling
mechanism, the following problem occurs: The file system driver frequently changes the file allocation
table at the beginning of the device. This happens at least each time a file is created, resized or deleted.
If there is no wear levelling layer below the FAT file system, the same flash block will likely get updated
a lot of times and worn out quickly which will render the flash device unusable very soon. This is why
the FAT/FAT32 is often combined with software or firmware to provide a simple form of wear levelling
and equally distribute write operations across the physical blocks of the device. Any file system can then
work the same way as on other block devices because it sees the flash device as a logical block device
which indirectly accesses the real physical device through the wear levelling component.

2.5.2 ExFAT
The extended File Allocation Table [34] has been developed for mobile devices with non-removable NOR
or NAND flash memory. It is an improved version of the FAT file system which supports files larger than
4GB and allows the file system to be customized [35] for device specific parameters like the physical
erase block size of flash devices. Additionally, devices using an exFAT file system may optionally
implement a transaction-safe FAT file system (TFAT) [36] which increases reliability and allows crash
recovery on interrupted file system operations. It is not recommended to use TFAT on removable devices
in combination with desktop operating systems which do not know about TFAT because then the file
system is treated like FAT and file system operations are not transaction-safe. TFAT manages two copies
of the file allocation table and reroutes the FAT chain on file modifications. This causes a performance
decrease in comparison with FAT.

2.5.3 JFFS2
The Journalling Flash File System is a log-structured file system based on a simple log-structured file
system (LFS) which has been released by Axis Communications AB in Sweden in 1999 under the GNU

14
General Public License. JFFS only supported one inode type (raw inode) and therefore lacked support
for hard links. Later, an advanced, portable version named JFFS2, with several inode types (raw inode,
directory entry, clean marker) has been developed.
JFFS2 [3] is designed to work on NOR and NAND flash chips and provides error correction (ECC)
which is obligatory for NAND flash. The file system accesses flash devices through the Memory Tech-
nology Device (MTD) subsystem of Linux and takes care of wear levelling and bad block management.
This avoids the inefficiencies which would occur with a journalling file system on top of the flash trans-
lation layer, which is some kind of journalling file system by itself.
JFFS2 supports compression on the file system level. With file compression enabled, it is difficult to
predict the duration of file system operations. Additionally, it provides a background garbage collection
task which is triggered when necessary and reclaims unused erase units marked for deletion. Such
a background task may cause unpredictable delays or even deadlocks, which makes it impossible to
guarantee worst case delays for file operations on JFFS2. There is a problem that JFFS2 write access
operations will hang during garbage collection, which has been confirmed on the Linux MTD mailing
list by David Woodhouse, the maintainer of JFFS2 [37].
Another deficiency of JFFS2 is that it is only suitable for smaller flashes. It does not store a central
index on flash, but each JFFS2 node contains all index information about itself. At mount time, it
performs a multi-stage process including a full scan of the physical device in order to gather all inode
information and store it in RAM, which is much faster than flash. For all nodes, the CRC value is checked
and only the valid nodes are stored in inode cache structures, collected in a hash table [3]. The full scan
requires a lot of time and largely depends on the size and read performance of the flash device. It also
requires a lot of memory to hold the inode cache structures in RAM. Small embedded systems might
not have enough RAM to cache all inode information and successfully mount a large JFFS2 partition
containing multimedia files.
JFFS2 prolongs flash lifetime by implementing wear levelling and garbage collection, but yet noone
has proven the formal correctness of the JFFS2 garbage collection algorithm, and as long as it is not
proven to be deadlock-free, JFFS2 is not a good choice for real-time systems. The above-mentioned
problems make the JFFS2 file system unsuitable for hard real time systems which require big flashes
or have small RAM. On power-failure, the file system needs to be remounted and the complete device
scanned until the file system gets ready. This would make the file system unavailable for several seconds
or even minutes which might lead to missed deadlines.
Since JFFS2 does not scale well and only works on small flashes, it will likely be superseded by ad-
vanced flash file systems like JFFS3 [38] or LogFS [39] when they have reached production state. Both of
these file systems use hierarchical tree structures and are designed to work on large flashes, save memory
and aim to reduce mount time by implementing check point nodes which store accumulated information
about inodes. The design papers do not discuss real-time issues, but there are already extensions [40] to
JFFS3 and any further improvements are welcome on the linux-mtd mailing list.

2.5.4 LogFS
LogFS [39] is a new log-structured flash file system designed to be scalable and provide an efficient,
deterministic garbage collection algorithm which never uses more space than it reclaims. It aims to

15
mitigate the deficiencies of JFFS2 and is designed so that mount times do not linearly depend on device
size. The LogFS file system is still under heavy development, but in contrast to JFFS2, the relatively
simple garbage collection strategy of LogFS can be proven to be deadlock-free. Jörn Engel, the main
developer of LogFS, explained the garbage collection algorithm and its correctness in a talk [41] at the
linux.conf.au conference in Sydney 2007.
LogFS does not keep track of deleted nodes, it uses a tree whose root is stored in a journal in a few flash
erase units. To prevent the wearout of such essential journal blocks, they can be indirectly referenced
by another block which points to the actual journal. This pointer can be updated and the journal can be
moved around when the maximum erase count approaches.
The LogFS file system handles inodes like files, in fact LogFS stores all inodes in a file which is
referenced by the journal. When files are added or changed, it allocates new erase blocks, writes the
data and restructures the tree to reference the new data. This produces some garbage nodes which can be
collected afterwards by walking down the tree up to the level where data has been changed.

2.5.5 YAFFS2
YAFFS2 is the successor of YAFFS which is an acronym for Yet Another Flash Filing System. The initial
YAFFS version has been developed in 2002 by Charles Manning for the Aleph One company because the
first JFFS/JFFS2 versions were mainly developed for NOR flash and did not scale well (cf. section 2.5.3).
YAFFS2 has much code in common with YAFFS. It is a log-structured, tree-based file system which is
specifically designed for newer NAND flashes. The main focus lies on high performance, robustness and
data integrity. A Linux kernel patch which supports the YAFFS2 file system can be obtained from [42].
A YAFFS file system image can be created with the tool mkyaffsimg. Alternatively, a new YAFFS
file system can be directly created on NAND flash using the tool mkyaffs. This tool simply erases all
undamaged blocks of a NAND memory technology device. Any erased blocks are considered as empty
space in the file system. YAFFS obeys the requirements of the most restrictive NAND flashes and takes
care of writing the pages of each erase block in sequential order. YAFFS2 writes each page only once
before the block is erased. When the size of a file is reduced a new chunk called shrink header is
written. Such shrink headers allow marking previous blocks as dirty without the need to overwrite the
discarded chunks. In order to reduce wear, YAFFS2 does not erase blocks until they are full of data
chunks which are marked dirty. Mount time is improved by a feature called checkpointing which stores
accumulated file system meta-data on flash when the device is unmounted. This trade-off slows down
the unmount process, but allows reconstructing necessary file system information very quickly when the
device is inserted and mounted. Since JFFS2 does not support checkpointing, YAFFS2 usually has 5
times shorter mount times than JFFS2 [43], [44]. Nevertheless it should be noted that on power-loss,
no clean unmount could be done, and consequently no checkpoint data would be written. Paper [45]
describes how to improve crash recovery time for log-structured file systems like YAFFS by introducing
a RAM based log-record manager. The log-record manager maintains checkpoint information in RAM
and regularly updates check regions on flash. This helps to reduce the number of pages scanned at mount
time when a file system had not been cleanly unmounted before.
The YAFFS file system has a modular architecture which separates file system functions from flash
management functions. The YAFFS Direct Infrastructure (YDI) documentation [46] claims that ’YAFFS

16
has highly optimised and predictable garbage collection strategies’ and describes how YAFFS can be
integrated into real-time operating systems.

2.5.6 Summary
Most current flash file systems put great efforts in wear-levelling and robustness, but care less about
predictability, timing guarantees or real-time aspects. A flash file system which takes care of flash spe-
cific requirements, flash lifetime and strongly deterministic timing for read/write operations would be
desirable.
Regardless of the file system chosen, it is always recommended to mount flash devices with the
noatime option. This prevents the file system from updating file access timestamps when files are ac-
cessed read-only. The omission of write operations while reading leads to higher read performance,
reduces wear and therefore increases the lifetime of the flash device.

17
3 State Of The Art

This chapter discusses the capabilities of existing file system and device benchmarks.

3.1 Storage Benchmark Tools

3.1.1 bonnie and bonnie++
Bonnie [47] is a file system benchmark for Unix file systems, developed by Tim Bray in the C program-
ming language. It was designed to determine bottlenecks in file systems on server hard disks. Supported
benchmarks are the average sequential output and input (per character and block) and the maximum
number of random seeks per second. It uses lseek, read, getc, write and putc functions to access a single
file on the file system and tries to avoid caching effects. Nevertheless, it does not make use of direct
input/output kernel features because it primarily aims to benchmark the file system, rather than hard disk
performance. Bonnie++ [48] is an improved C++ version of Bonnie and was written by Russel Coker
in order to test the performance of file systems on database servers. It allows creating several files to
test file systems larger than 2 GB and adds creat, stat and unlink to the set of benchmarked file system
functions. These functions are used frequently on mail, news or proxy servers with thousands of files. It
also supports a multi-process mode for testing RAID or multi-core systems and a blocking write mode
calling sync after each write to emulate database servers which sync written data.

3.1.2 IOzone
IOzone [49] is a comprehensive file system benchmark available on many architectures. It supports a lot
of features, measures a variety of file operations and can be used to evaluate the overall performance of
different file systems. It provides gnuplot compatible output and supports synchronous operation which
flushes writes immediately to the storage device. Even though cache effects can be analysed with the
help of IOzone by varying the unit and file sizes, it displays only average values, so it would be hard to
detect rogue timing results with IOzone.
The original plan was to use IOzone for comparing the ext2 and jffs2 file systems on the BlackFin
BF537 board’s embedded 4MB flash memory technology device, but unfortunately, IOzone ran out of
memory when the cross-compiled generic target was executed under uClinux on the BlackFin board. On
desktop or server machines with plenty of RAM it should run well.

3.1.3 Tiobench
Tiobench [50] is a portable, threaded file system benchmark program. It is licensed under the GNU
General Public License and runs on any POSIX operating system which supports pthreads. Before

18
running the tool, a file system needs to be created. Each thread then creates a file in this file system to
measure read and write operations. Tiobench can be used to determine average and maximum access
latencies of read and write operations on different file systems and the average read and write throughput
at sequential or random offsets inside of files or raw block devices.

3.1.4 SiSoftware Sandra

The System ANalyser Diagnostic and Reporting Assistant (called Sandra) developed by SiSoftware in-
cludes some simple file system and removable storage/flash benchmark modules. These can be used
to measure average throughput and latencies, but they do not support detailed analysis of worst cases,
different positions or data patterns. This tool is proprietary software available for Windows only. Since
the source code of SiSoft Sandra is not publicly available, results of this tool are not widely trusted by
the community.

3.1.5 h2benchw
Heise has developed a disk benchmark [51] for DOS (h2bench) and for Windows (h2benchw) which
makes use of raw physical devices and measures the minimum, average and maximum access times as
well as the sustained read/write performance.

3.2 Previous Benchmark Results

As of this writing, the following flash benchmarks are available:

• A rough comparison report on usb flash drives, their prices, access times and average throughput
can be found at Tom’s Hardware [52].

• The CF/SD performance database [53] collects average performance data of various Compact
Flash and Secure Digital cards.

• Hans-Jürgen Reggel collects information about the compatibility of card readers and removable
flash memory cards from various vendors in his cardspeed project [54]. This project gives a broad
overview on which cards really provide the advertised bandwidth and which card reader is best
suited for which type of card. The results are based on the output of h2bench and a tool written by
himself.

• c’t magazine has also published an article [55] with comprehensive benchmark tables covering the
read and write throughput of many flash cards available, but these benchmarks also put their main
focus on average bandwidth.

3.3 Summary
Existing flash benchmark tools and reports usually compare average or best access times and data transfer
rates of different cards. Most of these benchmarks do not go into detail and often neglect the analysis

19
of special properties like anomalies, extreme cases or timeouts of flash memories and their interfaces.
Average bandwidth values are simply summarized in tables in order to compare the general performance
of certain devices and to make recommendations which ones to buy. This might be sufficient for most
end-users, but real-time developers will also be interested in worst-case results in order to estimate worst-
case execution time of flash access operations.
Most of the above disk benchmark tools specialize in hard disk anomalies related to moving parts, but
there are no moving parts in flash at all. Since flash is based on completely different technology, other
anomalies are to be expected and new tools are required to cover them. In order to implement some
special tests which were deemed relevant for flash memories, a tool named flanatoo was developed for
this thesis.

20
4 Flanatoo - A Flash Analysis Tool

This tool analyzes the behavior and the best, average and worst case performance of flash memories. Its
source code can be found in directory ./src/flanatoo of the enclosed tarball and can be compiled using
the make command. It provides features to perform a detailed analysis of a raw flash device: First, it
supports reading and writing at different positions to check whether performance depends on position.
Second, it allows writing different data patterns in order to check whether some patterns can be written
faster or slower than others. Third, it makes use of direct physical access to the device in order to bypass
caches. Finally, it records best, average and worst case times of access operations, to detect exceptionally
high delays and to get an impression of the worst case measurement results. The following sections will
describe the functionality in more detail.

4.1 Basic Use

The tool is called from the shell command line:

# ./flanatoo --<mode> --device <filename> [options]

Mode is the desired benchmark mode and filename is the raw block device on which the test will be run.
Both mode and filename are mandatory options.
The complete syntax and a list of available tests can be requested as follows:

# ./flanatoo --help

There are some additional parameters which may be applied to several tests: --max-blocksize
sets the maximum block size for tests where the flash is benchmarked using differently sized read/write
units. The --max-devsize option sets an upper size boundary on the device which may not be crossed
when reading or writing.

4.2 Time Measurements

In order to keep time measurements as precise as possible, the tool makes use of the processor’s time
stamp counter register (TSC) which can be read with assembler commands on all Intel compatible CPUs
since the Pentium. Since the time stamp counter only provides a number of ticks, but we want to know
the time passed, the CPU frequency needs to be determined. This is achieved by reading the time stamp
counter, waiting for 5 seconds using the sleep() function and then reading the time stamp counter again.
The sleep command may return shortly after 5 seconds, depending on the timer granularity of the clock
timer and other processes running, but the error is negligible in relation to the time span of 5 seconds.
It is assumed that the CPU runs at constant frequency and the difference between the two TSC values is

21
used to calculate the average number of processor cycles which passed per second. The result is stored
in variable options->cycles_per_second.
Another important parameter is the measurement overhead which is the minimum number of pro-
cessor cycles needed for the measurement itself. This measurement delay is determined in function
calibrate_tsc by running a loop with ten iterations and reading the time stamp counter twice in each
pass. The minimum number of cycles from all iterations is considered the minimum overhead of reading
the time stamp counter and is stored in variable options->rdtsc_overhead. The arithmetic mean of the
overhead is not used, because if only one pass requires much more time (e. g. if the process is inter-
rupted between the two read operations), this will have a great impact on the arithmetic mean of the
overhead, even though all the other values were much smaller. Unlike the arithmetic mean, the minimum
overhead represents the additional time required for each measurement and may be subtracted from any
measurement value.

4.3 Direct Access

In order to minimize cache effects in block device drivers or the file system, direct I/O access to the flash
device is requested using the O_DIRECT flag of the open system call. The O_NOATIME flag is passed
to prevent access times from being updated when the device is being read. In order to measure the real
physical write duration, it has to be ensured that the write system call does not return until the physical
write operation has finished, which is achieved with the O_SYNC flag. The O_RDWR option opens the
device in a mode which allows both read and write operations.

4.4 Supported Benchmarks

This section describes all benchmarks supported by the flash analysis tool.

4.4.1 Sustained Read

This benchmark is launched with the --read option. It determines the effective bandwidth of contin-
uous read access for different block sizes by sequentially reading a larger chunk of data starting at the
beginning of the device. It starts with 512 byte blocks and then increases the block size in powers of 2 up
to the maximum block size of 1 mebibyte if no other maximum block size is specified on the command
line. Each read operation is performed ten times. In order to mitigate CPU cache effects, the data buffer
is filled with random values before each read operation. The minimum, arithmetic mean and maximum
duration of the read operations for each block size are printed in milliseconds. Best, average and worst
case bandwidth are calculated using the current block size, number of blocks read and the total delay for
reading all blocks.

4.4.2 Sustained Write

The write benchmark is launched with the --write option and basically works the same way as the
sustained read benchmark, but it writes random data instead of reading data. Memory cache effects are

22
mitigated by filling random data into the buffer before it is written to the device. The O_SYNC flag
prevents an early return before each write operation is finished.

4.4.3 Classic Read Cache Test

This test is invoked with command line option --cache-read. It reads the same data block twice in
order to compare the time required for a first and a second access. The duration of both read operations is
stored in temporary variables. The program fills the memory buffer with random characters before each
read operation. This measure has been taken to reduce CPU cache effects because this test rather aims
to correlate differences between the two measurement results with a potential buffer cache in the card
reader or flash device. As above, each read sequence is repeated ten times and the block size is varied
from 512 bytes up to the maximum block size. For each block size, the best, average and worst case
time of the first and second read access are printed. These can be compared to investigate if there are
additional caches in the flash equipment.

4.4.4 Read-Ahead Cache Test

The --cache-read-ahead benchmark reads two sequential blocks and measures the duration of
reading the first and second block. The aim is to find out whether the external device has a read-ahead
cache. When such a cache is in place, a read operation would fetch unrequested subsequent data into
the cache and would allow sequential read requests to be responded to more quickly. Again, for each
block size the best, average and worst case times are printed. A calculation of an effective bandwidth
is not done in this benchmark because the measurement is limited to a single block and would not issue
significant results.

4.4.5 Write Cache Test

The write cache benchmark can be called with command line option --cache-write and works as
follows: Write several consecutive blocks of random data in order to thrash any probably existing write
cache. Then read a single block, write random data to it and read the block again. These two read
operations are measured and can be compared to find out whether the card reader or flash device has
a write cache. If a write cache was present in the external devices, the second read request would be
responded to more quickly than the first read request because the cache is dirty before the first read
request and data needs to be fetched from flash chip, but it already contains the correct data before the
second read request because the write cache was just updated. Additionally, a block from the write cache
is expected to be read faster than a block of the same size in the sustained read benchmark since the
cache will become dirty when the amount of sequentially read data exceeds the cache size.

4.4.6 Positioned Read

This benchmark can be invoked with the --positioned-read command line option. It divides
the flash device into ten parts of equal size and reads four 64 KiB blocks from the start of each part.
The measurement is repeated ten times for each position. Each time before reading a data block, the
target buffer is filled with random data in order to mitigate CPU caching effects. The output is the byte

23
offset of each part and the minimum, average and maximum number of milliseconds needed to read four
blocks from these positions. These results can be used to check if read operations are equally fast for all
positions.

4.4.7 Positioned Write

Command line option --positioned-write invokes this benchmark which is very similar to the
positioned read benchmark. Instead of reading the device, it writes four 64 KiB blocks at ten different
positions of the device. Each test is repeated ten times and the buffer is filled with random data before
writing any block. Afterwards, the byte offset of each position and the corresponding minimum, average
and maximum delay is printed in milliseconds. The results of this benchmark illustrate for a specific
flash device whether write performance is position-dependent.

4.4.8 Flash Test

This test can be run using the --flash-test command line option. It tries to determine the physical
erase block size by writing 1 MiB of 00...00 and FF...F0 patterns to the device and measures the delay
for each pattern and several block sizes. This test is based on the assumption that a physical erase block
is erased when a block consisting of 0xFF is written, but the block is not erased if there is at least one
zero contained in the written block. As long as the logical block size is less or equal to the size of a
physical erase block, the FF...F0 pattern would always include a 0x00 character in the range of a physical
erase block and no blocks would be erased. The same would apply to the 00...00 pattern which contains
no 0xFF at all. Now when the physical erase block size is exceeded, the 0xFF characters of the FF...00
pattern should cover a whole physical erase block and at least every second block should be erased. If
the above assumption was true, this would mean that writing the FF..00 pattern would be slower than for
smaller block sizes because additional erase operations would be required. Writing the 00..00 pattern
should be the same as for smaller block sizes.

4.4.9 Write Specific Data

The --write-specific-data test measures the influence of data patterns on flash write perfor-
mance. It writes four 64KB blocks of 16 patterns ranging from 0x00 to 0xFF. Each pattern is overwritten
with each of the same 16 patterns and the delay of the second write operation is measured. This results in
256 write combinations. Each combination is written 5 times and the minimum, average and maximum
delay is printed.

24
5 Removable Flash Card Analysis

5.1 Test Setup

Different flash memory cards were plugged into a HAMA 19-in-1 card reader, which is based on a
Genesys Logic GL819 chipset, and the flanatoo tool was used to perform the tests on each of the follow-
ing devices.

Identifier Device Size Vendor Product Code

CF8HP CF 8MB Hewlett Packard SDCFB-8 D0012JL
CF32MED CF 32MB Medion MD9464
CF64-1 CF 64MB Memory Solution FC 64 6Q62L
CF64-2 CF 64MB Memory Solution FC 64 5Q62L
CF64-3 CF 64MB Memory Solution FC 64 6Q62L
CF64-4 CF 64MB Memory Solution FC 64 5Q62L
CF256TOSH CF 256MB Toshiba THNCF256MMA
CF1GSAN CF Ultra II 1GB SanDisk BB05055KB
CF2GSAN CF Ultra II 2GB SanDisk BE05085JB
MD1GHIT CF II Microdrive 1GB Hitachi DSCM-11000
IBM 07N4071
MMC32 MMC 32MB - MC56U032NFCA-QC
Q6D650Q526
MMC512MOB MMC Mobile 512MB ExtreMemory MC2GH512DMCA-SA
S WAC572CA 615
MS16SONY MS 16MB Sony MSA-16A 0139S31
MS32PROSAN MS Pro Duo 32MB SanDisk -
NOR4ST Parallel NOR 4MB STMicroelectronics M29W320DB data sheet [56]
SD32MIT SD 32MB Mitsuca AA0403NV
SD512SAN SD 512MB SanDisk AX05278ATB
SD1GHAMA SD 1GB Hama F5D1GBUIM-9-R
S621W65M1944
B-07-DW266-01MP
USB128SEC USB Drive 128MB - -
USB512LEX USB Drive 512MB Lexar -

Table 5.1: List of benchmarked flash cards

25
 Only the card reader or USB drive were plugged into the USB 2.0 port of the host system and no other
USB devices were connected. The test system was an AMD Athlon64 3000+ machine with 1 GB of
RAM running a 2.6.18 Linux kernel. An exception where the card reader was not involved was the 4 MB
parallel NOR flash [56]. This chip is found on the BlackFin BF537 STAMP board and can be directly
accessed on the board through the MTD subsystem of uClinux. The Microdrive (MD1GHIT) contains a
small harddisk instead of flash memory, but due to its compact flash interface it can be plugged into the
card reader like a normal flash card and so it was included for comparison.

5.2 Test Performance

All cards were inserted into the card reader one after another. For each flash device a directory was
created and the script src/flanatoo/benchmark.sh was called to run the flanatoo benchmarks on the given
block device. In order to test the influence of different card readers, all tests on the compact flash CF64-1
card were performed twice, first with the HAMA 19-in-1 card reader and second with a no-name multi-
card reader.
The benchmark.sh script requires to be called with the filename of the flash block device to be tested.
Please note that the device will get overwritten. It then automatically stores the comma separated re-
sults into files in the current directory. This output directory is best located on a RAM disk to prevent
interference with hard disk activity. The benchmark can be run as follows:

RESULTS=Name-of-Flash-Device
DEVICE=/dev/sdb
cd /dev/shm
mkdir $RESULTS
cd $RESULTS
benchmark.sh $DEVICE

The output files were then copied from RAM disk to a non-volatile medium and converted into diagrams
using gnuplot and a configuration file which can be found in measurements/gnuplotrc:

cp -a $O $HOME
cd $HOME/$RESULTS
gnuplot gnuplotrc

The results can be found in the measurements subdirectory of the enclosed archive.

5.3 Test Results

5.3.1 Controller Hides Block Deletion
The flash test benchmark from section 4.4.8 was originally intended to determine the physical erase block
size of a device without the need to know the exact chip, but this did not work as intended since most de-
vices contain a flash controller which controls the deletion of blocks and hides them from the user. Such
a controller may also perform additional erase operations before any write operation (cf. section 2.4.4).

26
So the flash test is rather useless since the assumption that solely blocks filled with 0xFF trigger block
erases turned out to be false. Nevertheless, the next section shows that comparing the overall perfor-
mance for different block sizes can help to estimate the physical erase block size if the device cannot be
opened and the flash chip is not directly accessible.

5.3.2 Sensitivity to block size

Figure 5.1 illustrates that flash memories do not reach their full bandwidth until the block size used is
equal to or greater than the physical erase block (PEB) size of the medium. This block size typically lies
between 16 and 64 KB. Below the PEB size, the bandwidth approximately doubles when the access block
size is doubled, but when the PEB size is reached, the measured bandwidth converges at a maximum
sustained value. Most times a block is written, itself or another block needs to be erased beforehand in
order to reset the block to all 0xFF. The read bandwidth also shows this behavior, reading many small
blocks is much slower than reading a few large blocks. This might mean that the controller always reads
a whole erase unit from flash even when only one page is requested. Therefore, it is recommended to
access flash devices with a block size of at least their physical erase block size as specified by the flash
chip data sheet. Any flash file system or the flash block management layer should take care of buffering
data smaller than the size of a PEB and accessing only units of PEB size, otherwise even an access
operation with a very small amount of data will always take the time needed to read, write or erase a
whole physical erase block.

Figure 5.1: SanDisk 2 GB Compact Flash: Requires block size 64 KB to achieve full performance

27
5.3.3 Reading is faster than writing
When the logical block size is set to at least 64 KB, write bandwidth is lower than read bandwidth for
each of the devices. This confirms the claim that reading from flash memories is usually faster than
writing to them. Figure 5.1 shows that this is not always the case: In some exceptional cases reading
may be even slower than writing when block sizes below the physical erase block size are used.

5.3.4 Sensitivity to data patterns

The tests have shown that flash write performance significantly depends on the data being written. Since
flash memory stores data bits in separate memory cells, the time to physically store a data pattern depends
on how many bits need to be switched and thus how many memory cells have to be modified. If a bit
needs to be switched from logical zero to logical one, it will be required to use a clean erase unit with all
bits set to logical one or to erase the unit before writing, which has an impact on the physical block write
delay.
Writing blocks filled with 0xFF is an exceptional case for some removable flash devices. We can
divide the tested removable devices into three categories:

1. Writing 0xFF blocks is slightly (about 1 to 8ms) faster than writing any other data

2. Writing 0xFF blocks takes similar time as writing any other data

3. Writing 0xFF blocks is about 32ms slower than writing any other data

Category 1 - Writing 0xFF faster

Figure 5.2 illustrates that on the CF256TOSH card writing 0xFF blocks (below 158ms) on average is
slightly faster than writing any other data patterns, so it can be assigned to category 1. Sporadically, a
write operation requires about 270ms instead of about 160ms, no matter which data has been written.
This might be an indication of the device performing additional block erasures on some of the write
operations. The CF32MED, CF64-1, CF64-2, CF64-3, CF64-4, MMC32, MS16SONY, MS32PROSAN,
NOR4ST and SD32MIT devices all show similar characteristics: writing the 0xFF pattern produces the
shortest delay in comparison to writing any other patterns. Additionally, any of the write operations on
Multimedia Card MMC32 took between 193 and 196ms, no matter which data was written. Inside of
this range, the card shows characteristic patterns depending on which data was stored on the card before
the write operation started. Write operations were rather fast (193.0 to 194.2ms) when there was 0x00,
0x33, 0x66 or 0x99 on the card, while write operations on blocks filled with 0x11, 0x88, 0xBB or 0xEE
were rather slow (194.9 to 196ms). The speed of any other write combinations varied, but did not leave
the above-mentioned range. The delay for writing data to the embedded parallel NOR flash (NOR4ST)
strongly depends on both the previously stored and the newly written data patterns. Figure 5.3 shows
there are three groups of data patterns which produce different write delays. The rightmost data column
also reflects these three groups, but each of them has a write delay which is about 200ms smaller. This
means that when the device contains the 0xFF pattern before any new data is written, writing is about 5
percent faster.

28
Figure 5.2: Toshiba 256 MB Compact Flash: Writing 0xFF is on average faster than other data

Figure 5.3: ST M29W320DB: Writing is clearly faster when there was an 0xFF pattern before

29
 Devices MS16SONY (figure 5.4) and SD32MIT (figure 5.5) show a clear distinction between the
delays when writing 0xFF versus other data. Writing 0xFF to the 16 MB Sony Memory Stick was 8ms

Figure 5.4: Sony 16 MB Memory Stick: Writing 0xFF is clearly faster

faster than the worst case of other data, and writing four 0xFF blocks to the 32 MB Secure Digital Card
from Mitsuca was 5ms faster in contrast to the worst case of writing other data. The difference between
the best and worst case for writing data other than 0xFF to the SD32MIT device was only 7ms, which
clearly shows that writing blocks filled with 0xFF is an exceptional case and handled differently.
Since this behavior applies to a lot of the tested cards it seems to be common for flash memories. A
freshly erased physical block contains only 0xFF. It is therefore obvious that the 0xFF pattern is written
fastest because no memory cells need to be modified after an erase operation. The controller can simply
erase one physical block and leave it as is. Writing any other data would require to address and modify
memory cells which takes time. Since erase and write are two separate flash commands, a simple erase
command suffices to fill one or more blocks with 0xFF and the write command would not even need to
be issued.

Category 2 - Write speed varies

This category includes all devices where writing speed is not significantly determined by the written data
pattern. In contrast to the devices from category 1, these devices do not expose the internal flash behavior
to the outside world and do not show any significant timing difference between writing blocks filled with
0xFF and other blocks. The CF1GSAN compact flash card from SanDisk belongs to category 2 because
the delay for writing blocks filled with 0xFF does not significantly differ from blocks with other data

30
 Figure 5.5: Mitsuca 32 MB Secure Digital card: Writing 0xFF is clearly faster

patterns. However, this compact flash card shows another interesting write pattern: While most write
combinations require between 26.5 and 27.5ms, some data patterns have a better best case taking about
1 or 2ms less time, especially when the blocks contained 0x33, 0x77, 0xBB or 0xFF before the new
pattern was written. The CF2GSAN card also belongs to category 2. It behaves very deterministic and
writing any data always requires between 26.25 and 26.60 ms at position 0. However, there may be other
characteristics like sensitivity to position (cf. section 5.3.5), which can affect the worst case writing
time. Writing delays varied among devices with different interfaces, like a 8 MiB compact flash card
(CF8HP), a 512 MiB dual-voltage reduced-size MultimediaCard Plus (MMC512MOB), a 1 GiB Secure
Digital Card from Hama (SD1GHAMA), the 512 MiB Lexar USB drive (USB512LEX) and of course
the 1 GiB Microdrive (MD1GHIT) from Hitachi which will not show flash specific behavior because it
does not contain any flash memory at all.

Category 3 - Writing 0xFF slower

A few flash devices were writing four 0xFF blocks about 32ms slower than other data patterns, which
is the total opposite to category 1 devices where writing 0xFF blocks is faster than writing other data
patterns. Category 1 devices have still one thing in common with category 3 devices: write time for
0xFF blocks clearly contrasts with writing blocks containing different data patterns.
Slow writing of 0xFF blocks was observed on a 128 USB drive (USB128SEC) and on a 512 MB
Secure Digital Card from SanDisk (SD512SAN). Figure 5.6 illustrates that writing four 0xFF blocks to
the 512 Secure Digital Card required between 282 and 284ms while any other data patterns required only

31
250 to 252ms. A variability of only 2ms for 0xFF data as well as non-0xFF data but a difference of 32ms
between 0xFF and non-0xFF data indicates that additional internal activity is going on in the flash device
in the special case where blocks filled with 0xFF are written to the device.

Figure 5.6: SanDisk 512 MB Secure Digital Card: Writing 0xFF blocks requires 282-284ms

5.3.5 Sensitivity to Position

Another important characteristic is whether and how read/write performance depends on the position ac-
cessed on the device. About half of the devices yielded position dependent behavior and some interesting
anomalies.

Better performance at position zero

Several devices including CF256TOSH, CF1GSAN, MS32PROSAN, SD32MIT and SD512SAN show
a clearly better write performance at position 0 than on any other positions. This might be an indication
for optimizations in a region of the device where the file allocation table is usually stored. However,
there is also a device (Lexar USB drive USB512LEX) which writes data significantly slower to position
0, while write performance on all the other positions varies heavily.

Bad performance at certain positions

On the CF2GSAN and MS16SONY devices, the write delay of four 64KB blocks more than doubled
at the middle position of the device. This might indicate the existence of two memory banks which are

32
accessed sequentially and have to be switched back and forth. Writing to the middle of a flash device
might lead to worst case performance if there are several memory banks present. For hard real-time
systems, it is therefore recommended to take care of memory bank switching and to check whether
sequential access over several memory banks might have an impact on worst case. If direct access to
embedded flash memory chips was possible, several memory banks could be accessed in parallel using
several processing units. The problem with removable flash devices, however, is that the memory chips
are usually hidden behind a single controller which does not allow direct access to the memory chips.
The 512 MB MultiMedia Card mobile (MMC512MOB) shows another interesting effect. Figure 5.7
illustrates that above 256 MiB the read performance suddenly becomes worse. This indicates that the
device probably contains two memory banks, a fast and a slow one. Number, type and quality of memory
banks is definitely an issue, and position-dependent read-write-tests can help to identify such memory
bank related problems.

Figure 5.7: ExtreMemory 512 MB MMC mobile card: Worse read performance above 256 MB

Position dependent devices

Some of the devices do not show characteristic anomalies, but the write performance varies across the
device. For example, the write performance on the 1 GB Secure Digital card from Hama (SG1GHAMA)
extremely varied in a range from 100ms to 500ms.

33
Position independent devices

There was also a group of devices where position does not matter: Performance of reading/writing four
64 KiB blocks from or to the devices CF32MED, CF64-1, CF64-2, CF64-3, CF64-4, CF8HP, MD1GHIT,
MMC32 and USB128SEC is not significantly determined by the positions accessed.

5.3.6 Influence of Card Reader

The card reader comparison experiments have shown that the basic timing behaviors of the two card
readers look very similar, but the general performance of the no-name card reader is lower than that
of the HAMA card reader. The HAMA card reader uses the GL819 chipset. The corresponding data
sheet [57] from Genesys Logic does not provide worst case timing guarantees for write access to flash. It
only provides write cycle minimum time and read access maximum time. In hard real-time systems, it is
not recommended to use generic card reader chips for which the data sheet does not guarantee both worst-
case read and write cycle times. Nevertheless, the general behavior of the two tested card readers was
very similar and the access times did not differ significantly. Worst case access times can be estimated
using the data sheet and tools like flanatoo or any other physical block device benchmark which consider
worst-case times. This might suffice for using card readers in soft real-time systems where it does not
pose a problem when a few access operations miss the estimated worst case.

5.3.7 Conclusion
The above sections have shown that even two devices with the same interface may expose a totally
different timing because the behavior of removable flash memories strongly depends on the integrated
controller and flash chips. Devices from the same vendor might contain different hardware, but devices
from the same product line are likely to contain the same hardware, e. g. the four 64 MiB compact
flash cards have shown a very similar behavior in all the tests. Without having profound knowledge
of the firmware used, controller and flash chip, and without extensively measuring it, the general and
worst-case behavior of removable flash devices cannot be reliably predicted.
In order to determine best and worst-case behavior of flash memories without knowing the internal
hardware details, one needs to run at least the flash tests for writing different data patterns, writing data
at different positions and with different block sizes, in order to find out basic anomalies of a specific
device and to estimate the access block size required for optimal throughput. By looking at the test
results and graphics one can decide whether the specific device is suitable for real-time use and which
block size the application or file system should use.

34
6 VFAT Mount Times

Flash file systems usually use a virtual to physical block mapping which is stored in volatile random
access memory during runtime. On the flash device, each physical block keeps track of which logical
unit it belongs to. The logical to physical mapping information has to be rebuilt when the device is
plugged in and the file system is mounted. Depending on the type of file system and the size of the flash
device, this can result in a long delay when mounting the device. In contrast to flash file systems, the
VFAT file system should produce rather short and deterministic mount times because it was originally
not designed for flash and does not care about wear levelling or remapping of blocks. It simply stores its
file system information at the beginning of the device and relies on lower layers like the hardware to do
the necessary wear levelling.

6.1 Setup
Mount times of the VFAT file system were compared on different flash cards using the HAMA card
reader. The host and kernel setup was the same as for the removable flash card analysis which has been
described in chapter 5. The script src/scripts/fill-directory was used to create 30 files containing 1 MB
of random data from /dev/urandom. Then a VFAT file system was created on each flash device using the
script src/scripts/create-vfat. This script also copied the files containing random data to the device and
then unmounted it.

6.2 Measurements
The script src/scripts/mnt takes a device as the first argument and measures the time to mount it using
the time utility. The device is automatically mounted and unmounted 10 times, and between each pass
it waits for a key press. This allows the user to remove and re-insert the device before the script tries to
mount it again.

Device / Mount 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.

CF64-1 59 58 58 58 58 58 58 58 58 58
CF256TOSH 60 59 60 60 60 60 60 60 60 60
CF1GSAN 56 56 58 56 56 56 56 56 56 56
CF2GSAN 83 83 84 83 84 84 84 84 84 84
MMC32 114 114 114 114 114 114 114 114 116 114
MS32PROSAN 55 53 53 53 53 53 53 53 53 53
SD1GHAMA 28 28 28 31 28 28 28 28 28 28
USB128SEC 254 254 253 254 254 253 255 254 254 255

Table 6.1: Mount times of VFAT file system filled with 30 MB of data (in ms)
35
6.3 Discussion
Table 6.1 shows that mount times are very reproducible for each device (maximum difference between
best and worst case is 3ms), but mount times significantly differ between the devices, depending on their
general performance. There is no linear dependency between the VFAT mount times and the device size.
VFAT mount time significantly depends on the overall device performance, e. g. a relatively slow 128
MB Secure Pendrive (USB128SEC) requires about 255ms to mount a 128 MB VFAT file system, while
a modern 1 GB HAMA card (SD1GHAMA) on average requires only 28ms to mount a 1 GB VFAT
file system when both are filled with the same 30 MB of random data. Since the file allocation table
has a fixed size and read performance from flash is very deterministic, mount times are almost constant
for VFAT, the most frequently used file system on flash cards. However, the time between the card
insertion and the mount process depends on user interaction. During this time, the card reader and the
flash card get the chance to initialize themselves without being measured, which might lead to smaller
measurement values for mount times. The time between the re-insertion and the moment when the card
becomes readable should be added to the mount time, if it can be reliably measured. Also note that flash
file systems like JFFS2 require reading more data when mounting the device, which heavily increases
the total mount time.

36
7 Real-Time Support for Flash File Systems

If a real-time application requires working with files and the underlying layers do not support wear
levelling (like some embedded flash chips and controller-less flash cards when used without FTL or
UBI), a suitable real time flash file system becomes necessary. The following sections explain why most
standard flash file systems are not suitable for real-time use and make proposals which aspects a flash
file system developer should take care of.

7.1 Problems of conventional flash file systems

Flash file systems are commonly optimized for robustness, flash lifetime and performance, but not for
real-time aspects and deterministic behavior. One important problem is garbage collection, which often
runs as a background task as in JFFS2. As mentioned in section 2.5.3, when garbage collection is working
on a certain file and at the same time access to this file is requested, a resource conflict might occur
and cause unpredictable blocking delays. This is unacceptable in hard real-time systems. Therefore, a
deterministic, real-time compatible approach for garbage collection needs to be developed. High priority
real-time tasks should be able to perform normal read or write access operations while garbage collection
is done.
Some flash file systems (e. g. JFFS2) store a complete direct map of logical to physical blocks in RAM
and therefore have a huge memory footprint. They need to scan the whole device at mount time in order
to rebuild the logical to physical block mappings which leads to very long mount times. A checkpointing
feature which reduces mount times is already implemented in YAFFS2 and planned for JFFS3 [58].
The JFFS2 file system does not support synchronous access when mounted with mount -o sync
or on files opened with the O_SYNC flag, if write buffering is enabled [59]. Only the functions sync()
and fsync() are supposed to work. Benchmarks using the O_SYNC flag or mount -o sync will not
return reliable results for the performance of JFFS2 file systems, especially when the kernel options for
JFFS2 write buffering or file compression are enabled. In this case, file access functions might return
after the content has been successfully sent to the buffer or compression handler, but before the data has
actually been written to flash.

7.2 Implementation Proposals

In order to be resistant against power-failure and to support wear levelling, real-time flash file systems
should be log-structured, such that they do not always erase and update blocks but always try to write
new or modified data to free blocks. To prevent the device from running out of free blocks, a garbage
collection mechanism is required. In order to guarantee a timely file access in real-time systems, this

37
mechanism must guarantee a maximum time for garbage collection actions performed on each file sys-
tem operation. Paper [60] proposes design principles for real-time garbage collection on the block device
layer. These proposals should be combined with the principles of log-structured flash file systems. LogFS
includes a garbage collection mechanism with a bounded execution time which depends on the modified
node’s depth in the hierarchical file system tree. Further information about LogFS and its garbage collec-
tion algorithm can be found in Jörn Engel’s talk [41]. The LogFS garbage collector could possibly serve
as a basis for developing a deterministic real-time garbage collector for log-structured flash file systems.
As proposed in [61], section 4, any sophisticated flash file system should reduce its memory footprint
by storing block related metadata in the out-of-band area on flash, while only caching the topmost levels
of a hierarchical file system structure in RAM for quick access. The required amount of metadata in
the device’s out-of-band area should be minimized. For example, LogFS is designed to cache only a
small portion of metadata in RAM and store the rest on the flash device. This might be a good basis
for developing a real-time flash file system which supports big flashes, provides fast mount times, has a
small memory footprint and a predictable garbage collection.
In order to enable simultaneous read and write operations and to prevent data loss, paper [61] proposes
using two flash memory chips where information is always written to both chips one after another. One
of the chips is in read mode while the other chip is being written. When the chip has finished writing, the
chip is switched to read mode and the other one enters write mode. This method allows doing garbage
collection on the write-enabled chip while the application is reading from the other one. In order to get
enough time for garbage collection, the paper proposes constraining the inter-arrival time of requests.
The redundancy provided by the two chips improves reliability because data is always available twice,
and if a memory block contains defects, data can still be read from the other chip. If a write process is
half finished during power-failure, serial numbers ensure that the latest intact version of a block can be
found and recovered easily. The paper proposes the following crash-recovery strategy: When a page is
about to be modified, the page and its serial number are read from flash, then the page is modified in a
buffer and then the modified page is written back to flash with a higher serial number (modulus 4). When
the writing finished successfully, the direct map is updated to point to the new page and the old page
is marked dirty by clearing the dirty bit in the out-of-band area of the old page. One aspect the paper
misses is the indirect page table which maps each physical to a logical block. It is distributed across the
out-of-band areas of all flash pages. So the whole device is required to be scanned at mount time in order
to construct a direct page map and build up a consistent file system state in RAM. LogFS addresses this
issue by storing an index on flash in dedicated journal blocks so that it can quickly locate the flash blocks
containing a certain file.

38
8 Further Work

8.1 Collect more data

This work includes benchmarks on 18 removable flash cards as found in embedded systems like mobile
phones or digital cameras, a parallel NOR flash chip and a Microdrive. Unfortunately, the number of
devices tested does not suffice to get a general survey on all existing flash devices and their behavior.
Even flash cards with the same sort of interface behaved differently, so it would be very helpful to
analyse a wider range of devices in order to spot more commonalities and major differences between the
devices. It would be good if some of the anomalies or other conspicuous properties could be directly
associated with specific types of flash, number of chips, physical erase block sizes, vendor names etc.
In order to get a more comprehensive overview on removable (and non-removable) flash devices, we are
interested in benchmark results for more removable or embedded flash devices. The output should be in
CSV format and should be added a detailed description of the flash chips, any intermediate hardware,
like flash controller, buses or card reader, as well as the host device and operating system. Additional
information about on-board flash chips can be obtained using the mtd_debug utility from the mtd-tools
package. Further information about the tests can be found in chapter 4. To prevent spoiling the results, is
it helpful to remove as many external devices and kill as many processes as possible before running the
benchmark tool.

8.2 Worst-case analysis of flash file systems

Most of the existing flash file systems have not been specifically designed for real-time systems and
may produce unexpected delays in certain situations. Such worst-case situations might for instance be
caused by a dysfunctional underlying block layer, erroneous blocks, file system operations conflicting
with background garbage collection, or a lot of small files which heavily increases file system overhead.
Therefore, flash file systems require further investigation and improvements focused on worst case, la-
tency and real-time aspects. An extensive analysis of the source code and behavior of flash file systems
is required to find out and fix the main problems relevant for real-time projects. Porting projects like
IOzone or tiobench to embedded systems may help measuring the performance of different flash file
systems directly on the embedded flash.

8.3 Real-time support in flash file systems

It should be investigated whether it is possible to add real-time support to flash file systems like JFFS3
or LogFS which are currently in development. A real-time flash file system should be able to guarantee
upper time boundaries for any file system operation and should move worst case closer to the average

39
case. Besides, it should still maintain most of the advantages of modern log-structured flash file systems
like automatic wear-levelling, deadlock-free garbage collection, O(1) mount times and a small memory
footprint. If it turns out that it is not possible to achieve all these aims by changing an existing file system,
a new open source real-time flash file system needs to be designed which above all takes care of real-time
aspects, but also pays attention to flash specifics.

40
9 Conclusion

This thesis summarized existing flash technologies as well as tools and file systems in the flash memory
area. It described methods to analyse the real-time behavior and mount times of removable flash devices.
Flash file systems were discussed with regard to real-time aspects and proposals for further improvement
of file system modules were made. The author is not aware of any open source flash file system which
supports garbage collection with real-time guarantees. Among others, a predictable garbage collection
mechanism as proposed in [60] needs to be implemented as an integral part of a real-time flash file
system.
The tested flash memories have proven to be very predictable on read operations with fixed block
sizes while write performance tests showed a lot of anomalies. Each device is different and write access
time strongly depends on the technology around and inside the flash memory. The consequence is that
each series of removable flash memory devices intended for hard real-time systems needs to undergo
extensive performance and latency testing before going into production. This helps to detect anomalies
and to ensure that the device complies with the data sheet specification even in worst-case situations. In
comparison to hard disk drives, flash memories are clearly more suitable for real-time systems because
they are more robust and provide far better worst-case access latencies. For raw flash devices which do
not include a controller [56], vendors even guarantee maximum access times for program operations in
their data sheets. Access times measured in experiments with the M29W320DB flash reliably met the
maximum access times provided in the data sheet. Sensitivity to data patterns as described in section
5.3.4 is negligible as long as the worst case data pattern meets the data sheet specification. Therefore it
is worth to put further research into real-time file systems for embedded flash memories.

41
List of Tables

2.1 Comparison of flash memory interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . 9

5.1 List of benchmarked flash cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

6.1 Mount times of VFAT file system filled with 30 MB of data (in ms) . . . . . . . . . . . . 35

42
List of Figures

2.1 Memory Stick Pro Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2 Layer model of Linux flash memory architectures . . . . . . . . . . . . . . . . . . . . . 12

5.1 SanDisk 2 GB Compact Flash: Requires block size 64 KB to achieve full performance . 27
5.2 Toshiba 256 MB Compact Flash: Writing 0xFF is on average faster than other data . . . 29
5.3 ST M29W320DB: Writing is clearly faster when there was an 0xFF pattern before . . . 29
5.4 Sony 16 MB Memory Stick: Writing 0xFF is clearly faster . . . . . . . . . . . . . . . . 30
5.5 Mitsuca 32 MB Secure Digital card: Writing 0xFF is clearly faster . . . . . . . . . . . . 31
5.6 SanDisk 512 MB Secure Digital Card: Writing 0xFF blocks requires 282-284ms . . . . 32
5.7 ExtreMemory 512 MB MMC mobile card: Worse read performance above 256 MB . . . 33

43
A Glossary

ATA Advanced Technology Attachment

CF Compact Flash

CFI Common Flash Memory Interface

CFIO Compact Flash In/Out

DOC Disk On Chip

DV Dual Voltage

ECC Error Correcting Code

EDC Error Detection and Correction

EEPROM Electronically Erasable Programmable Read-Only Memory

exFAT Extended File Allocation Table

FAT File Allocation Table

FTL Flash Translation Layer

IDE Integrated Drive Electronics

JFFS2 Journalling Flash File System 2

LEB Logical Erase Block

LFS Log-Structured File System

MLC Multi Level Cell

MMC Multimedia Card

MOSFET Metal-Oxide-Semiconductor Field-Effect-Transistor

MSC Mass Storage Device Class

MSD Memory Stick Duo (small dimensions)

MS Memory Stick

MSPD Memory Stick Pro Duo (high speed and small dimensions)

44
MSP Memory Stick Pro (high speed)

MTD Memory Technology Device

NAND A type of flash where cell compounds are addressed through negated AND logic

NOR A type of flash where each cell is addressed through negated OR logic and supports XIP

OOB Out Of Band Area (16 Bytes additional space for each NAND flash page)

PCMCIA Peripheral Component MicroChannel Interconnect Architecture

PEB Physical Erase Block

PIO Programmed In/Out

RAID Redundant Array of Inexpensive Disks

RAM Random Access Memory

RS Reduced Size

SDHC Secure Digital High Capacity

SDMI Secure Digital Music Initiative

SD Secure Digital Card

SLC Single Layer Cell

SPI Serial Peripheral Interface

TPC Transfer Protocol Command

TSC Time Stamp Counter

UBI Unsorted Block Images

x1 1 Bit Secure Digital Data Transfer Mode

x4 4 Bit Secure Digital Data Transfer Mode

xD Extreme Digital Picture Card

XIP Execute In Place

YAFFS2 Yet Another Flash File System 2

YDI YAFFS Direct Infrastructure

45
Bibliography

[1] Booting Linux Really Fast, Daniel Parthey, University of Technology Chemnitz, April 2006
http://archiv.tu-chemnitz.de/pub/2006/0066

[2] Erinnerungskarten, Benjamin Benz, Heise, c’t 2006/23, page 136

http://www.heise.de/ct/06/23/136

[3] JFFS: The Journalling Flash File System, David Woodhouse, Red Hat Inc.
http://sources.redhat.com/jffs2/jffs2.pdf

[4] x4 Technology, M-Systems, August 2006

http://www.m-sys.com/site/en-US/Technologies/Technology/x4.htm

[5] OneNAND Product Brochure, Samsung Electronics, September 2006

http://www.samsung.com/Products/Semiconductor/Support/ebrochure/
memory/onenand_brochure_200609.pdf

[6] New Memory Stick Platform Strategy for The Broadband Era, Sony Press, January 2003
http://www.sony.net/SonyInfo/News/Press_Archive/200301/03-0110aE

[7] Memory Stick Pro Specification Summary, Memory Stick Developers’ Site Office
http://www.memorystick.org/eng/simplefmt/memorystick_pro_
specification_summary_non-licensee_e.pdf

[8] Shape of Memory Stick, Memory Stick Developers’ Site Office

http://www.memorystick.org/eng/aboutms/detail/configration.html

[9] Comparison of Memory Cards, Memory Stick Developers’ Site Office, September 2004
http://www.memorystick.org/eng/aboutms/detail/outline_spec.html

[10] Interface of Memory Stick, Memory Stick Developers’ Site Office

http://www.memorystick.org/eng/aboutms/interface.html

[11] MultiMediaCard Specification Summary, Multimedia Card Association, April 2005

http://www.mmca.org/compliance/buy_spec/MMCA_System_SummaryV41.
pdf

[12] Samsung MMC data sheet for MC56U032NCFA, Samsung, June 2004
http://www.samsung.com/Products/Semiconductor/FlashCard/MMC/
NormalMMC/FullSize/MC56U032NCFA/ds_mc56u032ncfa_rev09.pdf

46
[13] SD Specifications Part E1, Simplified SDIO Specification
SD Card Association, September 2006
http://www.sdcard.org/sdio/Simplified%20SDIO%20Card%
20Specification.pdf

[14] SD Specifications Part 1, Simplified Physical Layer Specification

SD Card Association, September 2006
http://www.sdcard.org/sd_memorycard/Simplified%20Physical%
20Layer%20Specification.PDF

[15] Compact Flash 4.0 Specification, Compact Flash Association, May 2006
http://www.compactflash.org/cfspc4_0.pdf

[16] History of M-Systems, M-Systems, September 2006

http://www.m-sys.com/site/en-US/Technologies/Innovation

[17] Types of NAND flash memory, Linux Memory Technology Devices

http://www.linux-mtd.infradead.org/doc/nand.html

[18] mDOC DiskOnChip Technology, M-Systems, October 2006

http://www.m-sys.com/site/en-US/Technologies/Technology/
DiskOnChip_Technology

[19] DiskOnChip Tools, M-Systems, August 2006

http://www.m-sys.com/site/en-US/Support/DeveloperZone/Software/
DiskOnChipTools.htm

[20] Speicherschwarm, Boi Feddern, Heise, c’t 2006/18, page 168

[21] USB 2.0 Specification, USB Implementers Forum Inc., April 2006
http://www.usb.org/developers/docs

[22] IEEE Standard 1394, IEEE Standards Board, August 1996

http://ieeexplore.ieee.org/search/wrapper.jsp?arnumber=526693

[23] IEEE Standard 1394a, IEEE Standards Board, 2000

http://ieeexplore.ieee.org/search/wrapper.jsp?arnumber=853984

[24] IEEE Standard 1394b, IEEE Standards Board, 2002

http://ieeexplore.ieee.org/search/wrapper.jsp?arnumber=1146719

[25] Mount costs too long, Charles Manning, Linux MTD mailing list, November 2006
http://lists.infradead.org/pipermail/linux-mtd/2006-November/
016778.html

[26] Common Flash Memory Interface Specification 2.0, Advanced Micro Devices, December 2001
http://www.amd.com/us-en/assets/content_type/DownloadableAssets/
cfi_r20.pdf

47
[27] Eraseblocks torture: OneNAND results, Artem Bityutskiy, December 2006
http://lists.infradead.org/pipermail/linux-mtd/2006-December/
017017.html

[28] Algorithms and Data Structures for Flash Memories

Eran Gal and Sivan Toledo, School of Computer Science, Tel-Aviv University, July 2004
http://www.cs.tau.ac.il/~stoledo/Pubs/flash-survey.pdf

[29] Understanding the Flash Translation Layer Specification, Intel Corporation, December 1998
http://www.intel.com/design/flcomp/applnots/29781602.pdf

[30] David Woodhouse: JFFS2, Richard Ibbotson, Linux-Magazine Issue 17, 2002
http://www.linux-magazine.com/issue/17/DavidWoodhouse_JFFS2.pdf

[31] UBI - Unsorted Block Images, Thomas Gleixner, Frank Haverkamp, Artem Bityutskiy, 2006
International Business Machines Corp.
http://www.linux-mtd.infradead.org/doc/ubidesign/ubidesign.pdf

[32] UBI Frequently Asked Questions, Linux Memory Technology Devices

http://www.linux-mtd.infradead.org/faq/ubi.html

[33] From HDD to flash, Esther Spanjer, M-Systems, May 2006

http://www.mil-embedded.com/articles/authors/spanjer

[34] Extended FAT file system, MSDN Library, Microsoft Corporation, February 2007
http://msdn2.microsoft.com/en-us/library/aa914353.aspx

[35] OEM Parameter Definition with exFAT, MSDN Library, Microsoft Corporation, February 2007
http://msdn2.microsoft.com/en-us/library/aa914663.aspx

[36] TFAT Overview, MSDN Library, Microsoft Corporation, February 2007

http://msdn2.microsoft.com/en-us/library/aa915463.aspx

[37] GC does not handle big syslog files, David Woodhouse, Linux MTD mailing list, September 2003
http://www.infradead.org/pipermail/linux-mtd/2003-September/
008545.html

[38] JFFS3 Design Issues, Artem B. Bityutskiy, November, 2005

http://www.linux-mtd.infradead.org/tech/JFFS3design.pdf

[39] LogFS - Finally a scalable flash file system

Jörn Engel, IBM & Robert Mertens, University of Osnabrück
http://wh.fh-wedel.de/~joern/logfs.pdf

[40] JFFS3 Plan Extension, Ferenc Havasi, Zoltán Sógor, Mátyás Majzik, University of Szeged
http://www.inf.u-szeged.hu/jffs2/jffs3-plan-extension-20060928.
pdf

48
[41] Garbage Collection in LogFS (Video), Jörn Engel, IBM
http://lca2007.linux.org.au/talk/91

[42] YAFFS2 File System, Aleph One Limited, 2006

http://www.aleph1.co.uk/yaffs/
http://www.aleph1.co.uk/cgi-bin/viewcvs.cgi/yaffs2.tar.gz

[43] YAFFS2 Specification and Development Notes, Aleph One Ltd., May 2005
http://www.aleph1.co.uk/node/38

[44] YAFFS patch, Comparison of JFFS2 and YAFFS2 mount times, Keisuke Yasui, Celinux developer
mailing list, April 2005
http://tree.celinuxforum.org/pipermail/celinux-dev/2005-April/
000924.html

[45] Efficient Initialization and Crash Recovery for Logbased File Systems over Flash Memory,
Tei-Wei Kuo, Li-Pin Chang, Chin-Hsien Wu, No, 21st ACM Symposium on Applied Computing
(ACM SAC), 2006
http://www.cis.nctu.edu.tw/~lpchang/papers/SAC_wu_sac06.pdf

[46] YAFFS Direct Infrastructure, Aleph One Ltd., July 2006

http://www.aleph1.co.uk/node/349

[47] Bonnie, Benchmark for Unix file systems, Copyright Tim Bray 1990-1996
http://www.textuality.com/bonnie

[48] Bonnie++, Improved C++ Version of Bonnie, Copyright Russell Coker 2000
http://www.coker.com.au/bonnie++

[49] IOzone File System Benchmark

http://www.iozone.org/

[50] Tiobench, Threaded I/O benchmark, Copyright Mika Kuoppala 1999-2000

http://sourceforge.net/projects/tiobench

[51] h2bench, Benchmark Utility, Heise

http://www.heise.de/software/default.shtml?prg=3787

[52] Comparing High Speed USB Flash Drives, Tom’s Hardware, May 2005
http://www.tomshardware.com/2005/05/20/data_transfer_on_the_run/

[53] CF/SD Performance Database, Rob Galbraith

http://www.robgalbraith.com/bins/multi_page.asp?cid=6007

[54] Cardspeed - Card Readers and Memory Cards, Hans-Jürgen Reggel

http://www.hjreggel.net/cardspeed

[55] In die Karten geschaut, Boi Feddern, Heise, c’t 2006/23, page 142

49
[56] M29W320DB Flash data sheet, STMicroelectronics
http://www.st.com/stonline/products/literature/ds/7876.pdf

[57] GL819 USB 2.0 Generation 3 Multi-I/F Card Reader Controller

Genesys Logic, September 2006
http://www.genesyslogic.com/econtents/product02.asp?SN=66

[58] The Journalling Flash File System, David Woodhouse, Red Hat Inc., October 2001
http://sources.redhat.com/jffs2/jffs2-slides-transformed.pdf

[59] JFFS2 as transactional FS, David Woodhouse, Linux MTD mailing list, March 2007
http://lists.infradead.org/pipermail/linux-mtd/2007-March/017654.
html

[60] Real-time garbage collection for flash-memory storage systems of real-time embedded sys-
tems, Li-Pin Chang, Tei-Wei Kuo, Shi-Wu Lo, November 2004, ACM Transactions on Embedded
Computing Systems, Volume 3, Issue 4, Pages 837-863
http://doi.acm.org/10.1145/1027794.1027801

[61] Real-time support of flash memory file system for embedded applications
Sudeep Jain & Yann-Hang Lee, April 2006, Department of Computer Science and Engineering,
Arizona State University, Tempe, AZ 85287, ISBN: 0-7695-2560-1
http://ieeexplore.ieee.org/search/wrapper.jsp?arnumber=1611716

50
Declaration of Authorship

I hereby declare that the whole of this diploma thesis is my own work, except where explicitly stated
otherwise in the text or in the bibliography. This work is submitted to Chemnitz University of Technology
as a requirement for being awarded a diploma in Computer Science ("Diplom-Informatik"). I declare that
it has not been submitted in whole, or in part, for any other degree.

51