MODULE ONE

FILE SYSTEM
In computing, a file system (or file system) is used to control how information is stored
and retrieved. Without a file system, information placed in a storage area would be one
large body of information with no way to tell where one piece of information stops and the
next begins.
By separating the information into individual pieces, and giving each piece a name, the
information is easily separated and identified. Taking its name from the way paper based
information systems are named, each piece of information is called a "file". The structure
and logic rules used to manage the groups of information and their names is called a "file
system".
There are many different kinds of file systems. Each one has different structure and logic.
Each one has different properties of speed, flexibility, security, size and more. Some file
systems have been designed to be used for specific applications. For example the ISO
9660 file system is designed specifically for optical disks.
File systems can be used on many different kinds of storage devices. Each storage device
uses a different kind of media. The most common storage device in use today is a hard
drive whose media is a disc that has been coated with a magnetic film. The film has ones
and zeros 'written' on it sending electrical pulses to a magnetic "read-write" head. Other
media that are used are magnetic tape, optical disc, and flash memory. In some cases, the
computer's main memory (RAM) is used to create a temporary file system for short term
use.
File systems are used to implement type of data store to store, retrieve and update a set
of files. "File system" refers to either the abstract data structures used to define files, or
the actual software or firmware components that implement the abstract ideas.
Some file systems are used on local data storage devices; others provide file access via
a network protocol (e.g. NFS, SMB, or 9P clients). Some file systems are "virtual", in that
the "files" supplied are computed on request (e.g. procfs) or are merely a mapping into a
different file system used as a backing store. The file system manages access to both the
content of files and the metadata about those files. It is responsible for arranging storage
space; reliability, efficiency, and tuning with regard to the physical storage medium are
important design considerations.

1.1 ASPECTS OF FILE SYSTEMS
1.1.1 Space management

Example of slack space, demonstrated with 4,096-byte NTFS clusters: 100,000 files, each 5
bytes per file, equals 500,000 bytes of actual data, but requires 409,600,000 bytes of disk
space to store
File systems allocate space in a granular manner, usually multiple physical units on the
device. The file system is responsible for organizing files and directories, and keeping
track of which areas of the media belong to which file and which are not being used. For
example, in Apple DOS of the early 1980s, 256-byte sectors on 140 kilobyte floppy disk
used atrack/sector map.
This results in unused space when a file is not an exact multiple of the allocation unit,
sometimes referred to as slack space. For a 512-byte allocation, the average unused space
is 256 bytes. For a 64 KB clusters, the average unused space is 32KB. The size of the
allocation unit is chosen when the file system is created. Choosing the allocation size based
on the average size of the files expected to be in the file system can minimize the amount
of unusable space. Frequently the default allocation may provide reasonable usage.
Choosing an allocation size that is too small may results in excessive overhead, if the file
system will contain mostly very large files.
File system fragmentation occurs when unused space or single files are not contiguous. As
a file system is used, files are created, modified and deleted. When a file is created the file
system allocates space for the data. Some file systems permit or require specifying an
initial space allocation and subsequent incremental allocations as the file grows. As files
are deleted the space they were allocated eventually is considered available for use by
other files. This creates alternating used and unused areas of various sizes. This is free
space fragmentation. When a file is created and there is not an area of contiguous space
available for its initial allocation the space must be assigned in fragments. When a file is
modified such that it becomes larger it may exceed the space initially allocated to it,
another allocation must be assigned elsewhere and the file becomes fragmented.

1.1.2 Filenames
A filename (or file name) is used to identify a storage location in the file system. Most file
systems have restrictions on the length of filenames. In some file systems, filenames are
not case sensitive (i.e., filenames such as FOO and foo refer to the same file); in others,
filenames are case sensitive (i.e., the names FOO and foo refer to two separate files).
Most modern file systems allow filenames to contain a wide range of characters from
the Unicode character set. Most file system interface utilities, however, have restrictions
on the use of certain special characters, disallowing them within filenames (the file system
may use these special characters to indicate a device, device type, directory prefix, or file
type). However, these special characters might be allowed by, for example, enclosing the
filename with double quotes ("). For simplicity, special characters are generally
discouraged within filenames.

1.1.3 Directories
File systems typically have directories (also called folders) which allow the user to group
files into separate collections. This may be implemented by associating the file name with
an index in a table of contents or an inode in a Unix-like file system. Directory structures
may be flat (i.e. linear), or allow hierarchies where directories may contain subdirectories.
The first file system to support arbitrary hierarchies of directories was used in
the Multics operating system.
[1]
The native file systems of Unix-like systems also support
arbitrary directory hierarchies, as do, for example,Apple's Hierarchical File System, and its
successor HFS+ in classic Mac OS (HFS+ is still used in Mac OS X), the FAT file system
in MS-DOS 2.0 and later and Microsoft Windows, the NTFSfile system in the Windows
NT family of operating systems, and the ODS-2 and higher levels of the Files-11 file system
in OpenVMS.
1.1.4 Metadata
Other bookkeeping information is typically associated with each file within a file system.
The length of the data contained in a file may be stored as the number of blocks allocated
for the file or as a byte count. The time that the file was last modified may be stored as the
file's timestamp. File systems might store the file creation time, the time it was last
accessed, the time the file's metadata was changed, or the time the file was last backed up.
Other information can include the file's device
type (e.g. block, character, socket, subdirectory, etc.), its owner user ID and group ID,
its access permissions and other file attributes (e.g. whether the file is read-
only, executable, etc.).
A file system stores all the metadata associated with the fileincluding the file name, the
length of the contents of a file, and the location of the file in the folder hierarchyseparate
from the contents of the file.
Most file systems store the names of all the files in one directory in one placethe
directory table for that directorywhich is often stored like any other file. Many file
systems put only some of the metadata for a file in the directory table, and the rest of the
metadata for that file in a completely separate structure, such as the inode.
Most file systems also store metadata not associated with any one particular file. Such
metadata includes information about unused regions -- free space bitmap, block
availability mapand information about bad sectors. Often such information about
a allocation group is stored inside the allocation group itself.
Additional attributes can be associated on file systems, such as NTFS, XFS, ext2, ext3, some
versions of UFS, and HFS+, using extended file attributes. Some file systems provide for
user defined attributes such as the author of the document, the character encoding of a
document or the size of an image.
Some file systems allow for different data collections to be associated with one file name.
These separate collections may be referred to as streams or forks. Apple has long used a
forked file system on the Macintosh, and Microsoft supports streams in NTFS. Some file
systems maintain multiple past revisions of a file under a single file name; the filename by
itself retrieves the most recent version, while prior saved version can be accessed using a
special naming convention such as "filename;4" or "filename(-4)" to access the version
four saves ago.

1.1.5 Utilities
The difference between a utility and a built-in core command function is arbitrary,
depending on the design of the operating system, and the memory and space limitations of
the hardware. For example, in Microsoft MS-DOS, formatting is performed by a utility and
simple file copying is a built-in command, while in the Apple DOS, formatting is a built-in
command but simple file copying is performed with a utility.
File systems include utilities to initialize, alter parameters of and remove an instance of
the file system. Some include the ability to extend or truncate the space allocated to the file
system.
Directory utilities create, rename and delete directory entries and alter metadata
associated with a directory. They may include a means to create additional links to a
directory (hard links inUnix), rename parent links (".." in Unix-like OS), and create
bidirectional links to files.
File utilities create, list, copy, move and delete files, and alter metadata. They may be able
to truncate data, truncate or extend space allocation, append to, move, and modify files in-
place. Depending on the underlying structure of the file system, they may provide a
mechanism to prepend to, or truncate from, the beginning of a file, insert entries into the
middle of a file or delete entries from a file.
Also in this category are utilities to free space for deleted files if the file system provides
an undelete function.
Some file systems defer reorganization of free space, secure erasing of free space and
rebuilding of hierarchical structures. They provide utilities to perform these functions at
times of minimal activity. Included in this category is the infamous defragmentation utility.
Some of the most important features of file system utilities involve supervisory activities
which may involve bypassing ownership or direct access to the underlying device. These
include high-performance backup and recovery, data replication and reorganization of
various data structures and allocation tables within the file system.
1.1.6 Restricting and permitting access
There are several mechanisms used by file systems to control access to data. Usually the
intent is to prevent reading or modifying files by a user or group of users. Another reason
is to ensure data is modified in a controlled way so access may be restricted to a specific
program. Examples include passwords stored in the metadata of the file or elsewhere
and file permissions in the form of permission bits, access control lists, or capabilities. The
need for file system utilities to be able to access the data at the media level to reorganize
the structures and provide efficient backup usually means that these are only effective for
polite users but are not effective against intruders.
Methods for encrypting file data are sometimes included in the file system. This is very
effective since there is no need for file system utilities to know the encryption seed to
effectively manage the data. The risks of relying on encryption include the fact that an
attacker can copy the data and use brute force to decrypt the data. Losing the seed means
losing the data.

1.1.7 Maintaining integrity
One significant responsibility of a file system is to ensure that, regardless of the actions by
programs accessing the data, the structure remains consistent. This includes actions taken
if a program modifying data terminates abnormally or neglects to inform the file system
that it has completed its activities. This may include updating the metadata, the directory
entry and handling any data that was buffered but not yet updated on the physical storage
media. Other failures which the file system must deal with include media failures or loss of
connection to remote systems.
In the event of an operating system failure or "soft" power failure, special routines in the
file system must be invoked similar to when an individual program fails.
The file system must also be able to correct damaged structures. These may occur as a
result of an operating system failure for which the OS was unable to notify the file system,
power failure or reset.
The file system must also record events to allow analysis of systemic issues as well as
problems with specific files or directories.
1.1.8 User data
The most important purpose of a file system is to manage user data. This includes storing,
retrieving and updating data.
Some file systems accept data for storage as a stream of bytes which are collected and
stored in a manner efficient for the media. When a program retrieves the data it specifies
the size of a memory buffer and the file system transfers data from the media to the buffer.
Sometimes a runtime library routine may allow the user program to define a record based
on a library call specifying a length. When the user program reads the data the library
retrieves data via the file system and returns a record.
Some file systems allow the specification of a fixed record length which is used for all
write and reads. This facilitates updating records.
An identification for each record, also known as a key, makes for a more sophisticated file
system. The user program can read, write and update records without regard with their
location. This requires complicated management of blocks of media usually separating key
blocks and data blocks. Very efficient algorithms can be developed with pyramid structure
for locating records.

1.1.9 Using a file system
Utilities, language specific run-time libraries and user programs use file system APIs to
make requests of the file system. These include data transfer, positioning, updating
metadata, managing directories, managing access specifications, and removal.
Another approach is to partition the disk so that several file systems with different
attributes can be used. One file system, for use as browser cache, might be configured with
a small allocation size. This has the additional advantage of keeping the frantic activity of
creating and deleting files typical of browser activity in a narrow area of the disk and not
interfering with allocations of other files. A similar partition might be created for email.
Another partition, and file system might be created for the storage of audio or video files
with a relatively large allocation. One of the file systems may normally be set read-only and
only periodically be set writable.
A third approach, which is mostly used in cloud systems, is to use "disk images" to house
additional file systems, with the same attributes or not, within another (host) file system
as a file. A common example is virtualization: one user can run an experimental Linux
distribution (using the ext4 file system) in a virtual machine under his/her production
Windows environment (usingNTFS). The ext4 file system resides in a disk image, which is
treated as a file (or multiple files, depending on the hypervisor and settings) in the NTFS
host file system.
Having multiple file systems on a single system has the additional benefit that in the event
of a corruption of a single partition, the remaining file systems will frequently still be
intact. This includes virus destruction of the system partition or even a system that will not
boot. File system utilities which require dedicated access can be effectively completed
piecemeal. In addition, defragmentation may be more effective. Several system
maintenance utilities, such as virus scans and backups, can also be processed in segments.
For example it is not necessary to backup the file system containing videos along with all
the other files if none have been added since the last backup. As of the image files, one can
easily "spin off" differential images which contain only "new" data written to the master
(original) image. Differential images can be used for both safety concerns (as a
"disposable" system - can be quickly restored if destroyed or contaminated by a virus, as
the old image can be removed and a new image can be created in matter of seconds, even
without automated procedures) and quick virtual machine deployment (since the
differential images can be quickly spawned using a script in batches)

1.1.10 Design limitations
All file systems have some functional limit that defines the maximum storable data
capacity within that system. These functional limits are a best-guess effort by the designer
based on how large the storage systems are right now and how large storage systems are
likely to become in the future. Disk storage has continued to increase at
near exponential rates (see Moore's law), so after a few years, file systems have kept
reaching design limitations that require computer users to repeatedly move to a newer
system with ever-greater capacity.
File system complexity typically varies proportionally with the available storage capacity.
The file systems of early 1980s home computers with 50 KB to 512 KB of storage would
not be a reasonable choice for modern storage systems with hundreds of gigabytes of
capacity. Likewise, modern file systems would not be a reasonable choice for these early
systems, since the complexity of modern file system structures would consume most or all
of the very limited capacity of the early storage systems.

1.2 TYPES OF FILE SYSTEMS
File system types can be classified into disk/tape file systems, network file systems and
special-purpose file systems.
1.2.1 Disk file systems
A disk file system takes advantages of the ability of disk storage media to randomly address
data in a short amount of time. Additional considerations include the speed of accessing
data following that initially requested and the anticipation that the following data may
also be requested. This permits multiple users (or processes) access to various data on the
disk without regard to the sequential location of the data. Examples include:
FAT (FAT12, FAT16, FAT32), exFAT, NTFS, HFS and HFS+, HPFS, UFS, ext2, ext3, ext4, XFS,
btrfs, ISO 9660, Files-11, Veritas File System, VMFS, ZFS, ReiserFS and UDF.
Some disk file systems are journaling file systems or versioning file systems.
1.2.2 Optical discs
ISO 9660 and Universal Disk Format (UDF) are two common formats that target Compact
Discs, DVDs and Blu-ray discs. Mount Rainier is an extension to UDF supported by Linux
2.6 series and Windows Vista that facilitates rewriting to DVDs.
Flash file systems: Flash file system
A flash file system considers the special abilities, performance and restrictions of flash
memory devices. Frequently a disk file system can use a flash memory device as the
underlying storage media but it is much better to use a file system specifically designed for
a flash device.
1.2.3 Tape file systems
A tape file system is a file system and tape format designed to store files on tape in a self-
describing form. Magnetic tapes are sequential storage media with significantly longer
random data access times than disks, posing challenges to the creation and efficient
management of a general-purpose file system.
In a disk file system there is typically a master file directory, and a map of used and free
data regions. Any file additions, changes, or removals require updating the directory and
the used/free maps. Random access to data regions is measured in milliseconds so this
system works well for disks.
Tape requires linear motion to wind and unwind potentially very long reels of media. This
tape motion may take several seconds to several minutes to move the read/write head
from one end of the tape to the other.
Consequently, a master file directory and usage map can be extremely slow and inefficient
with tape. Writing typically involves reading the block usage map to find free blocks for
writing, updating the usage map and directory to add the data, and then advancing the
tape to write the data in the correct spot. Each additional file write requires updating the
map and directory and writing the data, which may take several seconds to occur for each
file. Tape file systems instead typically allow for the file directory to be spread across the
tape intermixed with the data, referred to as streaming, so that time-consuming and
repeated tape motions are not required to write new data.
However, a side effect of this design is that reading the file directory of a tape usually
requires scanning the entire tape to read all the scattered directory entries. Most data
archiving software that works with tape storage will store a local copy of the tape catalog
on a disk file system, so that adding files to a tape can be done quickly without having to
rescan the tape media. The local tape catalog copy is usually discarded if not used for a
specified period of time, at which point the tape must be re-scanned if it is to be used in
the future.
IBM has developed a file system for tape called the Linear Tape File System. The IBM
implementation of this file system has been released as the open-source IBM Linear Tape
File System Single Drive Edition (LTFSSDE) product. The Linear Tape File System
uses a separate partition on the tape to record the index meta-data, thereby avoiding the
problems associated with scattering directory entries across the entire tape.
Tape formatting
Writing data to a tape is often a significantly time-consuming process that may take
several hours. Similarly, completely erasing or formatting a tape can also take several
hours. With many data tape technologies it is not necessary to format the tape before
over-writing new data to the tape. This is due to the inherently destructive nature of
overwriting data on sequential media.
Because of the time it can take to format a tape, typically tapes are pre-formatted so that
the tape user does not need to spend time preparing each new tape for use. All that is
usually necessary is to write an identifying media label to the tape before use, and even
this can be automatically written by software when a new tape is used for the first time.
1.2.4 Database file systems
Another concept for file management is the idea of a database-based file system. Instead
of, or in addition to, hierarchical structured management, files are identified by their
characteristics, like type of file, topic, author, or similar rich metadata.
IBM DB2 for i
[3]
(formerly known as DB2/400 and DB2 for i5/OS) is a database file system
as part of the object based IBM i
[4]
operating system (formerly known as OS/400 and
i5/OS), incorporating a single level store and running on IBM Power Systems (formerly
known as AS/400 and iSeries), designed by Frank G. Soltis IBM's former chief scientist for
IBM i. Around 1978 to 1988 Frank G. Soltis and his team at IBM Rochester have
successfully designed and applied technologies like the database file system where others
like Microsoft later failed to accomplish.
[5]
These technologies are informally known as
'Fortress Rochester' and were in few basic aspects extended from early Mainframe
technologies but in many ways more advanced from a technology perspective.
Some other projects that aren't "pure" database file systems but that use some aspects of a
database file system:
A lot of Web-CMS use a relational DBMS to store and retrieve files.
Examples: XHTML files are stored as XML or text fields, image files are stored as blob
fields; SQL SELECT (with optionalXPath) statements retrieve the files, and allow the
use of a sophisticated logic and more rich information associations than "usual file
systems".
Very large file systems, embodied by applications like Apache Hadoop and Google File
System, use some database file system concepts.
1.2.4 Transactional file systems
Some programs need to update multiple files "all at once". For example, a software
installation may write program binaries, libraries, and configuration files. If the software
installation fails, the program may be unusable. If the installation is upgrading a key
system utility, such as the command shell, the entire system may be left in an unusable
state.
Transaction processing introduces the isolation guarantee, which states that operations
within a transaction are hidden from other threads on the system until the transaction
commits, and that interfering operations on the system will be properly serialized with the
transaction. Transactions also provide the atomicity guarantee, that operations inside of a
transaction are either all committed, or the transaction can be aborted and the system
discards all of its partial results. This means that if there is a crash or power failure, after
recovery, the stored state will be consistent. Either the software will be completely
installed or the failed installation will be completely rolled back, but an unusable partial
install will not be left on the system.
Windows, beginning with Vista, added transaction support to NTFS, in a feature
called Transactional NTFS, but its use is now discouraged.
[6]
There are a number of
research prototypes of transactional file systems for UNIX systems, including the Valor file
system, Amino, LFS, and a transactional ext3 file system on the TxOS kernel, as well as
transactional file systems targeting embedded systems, such as TFFS.
[11]

Ensuring consistency across multiple file system operations is difficult, if not impossible,
without file system transactions. File locking can be used as a concurrency
control mechanism for individual files, but it typically does not protect the directory
structure or file metadata. For instance, file locking cannot prevent TOCTTOU race
conditions on symbolic links. File locking also cannot automatically roll back a failed
operation, such as a software upgrade; this requires atomicity.
Journaling file systems are one technique used to introduce transaction-level consistency
to file system structures. Journal transactions are not exposed to programs as part of the
OS API; they are only used internally to ensure consistency at the granularity of a single
system call.
Data backup systems typically do not provide support for direct backup of data stored in a
transactional manner, which makes recovery of reliable and consistent data sets difficult.
Most backup software simply notes what files have changed since a certain time,
regardless of the transactional state shared across multiple files in the overall dataset. As a
workaround, some database systems simply produce an archived state file containing all
data up to that point, and the backup software only backs that up and does not interact
directly with the active transactional databases at all. Recovery requires separate
recreation of the database from the state file, after the file has been restored by the backup
software.
1.2.5 Network file systems
A network file system is a file system that acts as a client for a remote file access protocol,
providing access to files on a server. Examples of network file systems include clients for
the NFS,AFS, SMB protocols, and file-system-like clients for FTP and WebDAV.
Shared disk file systems
A shared disk file system is one in which a number of machines (usually servers) all have
access to the same external disk subsystem (usually a SAN). The file system arbitrates
access to that subsystem, preventing write collisions. Examples include GFS2 from Red
Hat, GPFS from IBM, SFS from DataPlow, CXFS from SGI and StorNext from Quantum
Corporation.
1.2.6 Special file systems
A special file system presents non-file elements of an operating system as files so they can
be acted on using file system APIs. This is most commonly done in Unix-like operating
systems, but devices are given file names in some non-Unix-like operating systems as well.
1.2.7 Device file systems
A device file system represents I/O devices and pseudo-devices as files, called device files.
Examples in Unix-like systems include devfs and, in Linux 2.6 systems, udev. In non-Unix-
like systems, such as TOPS-10 and other operating systems influenced by it, where the full
filename or pathname of a file can include a device prefix, devices other than those
containing file systems are referred to by a device prefix specifying the device, without
anything following it.
1.2.8 Other special file systems
In the Linux kernel, configfs and sysfs provide files that can be used to query the
kernel for information and configure entities in the kernel.
procfs maps processes and, on Linux, other operating system structures into a
filespace.
1.2.9 Minimal file system / Audio-cassette storage
In the late 1970s hobbyists saw the development of the microcomputer. Disk and digital
tape devices were too expensive for hobbyists. An inexpensive basic data storage system
was devised that used common audio cassette tape.
When the system needed to write data, the user was notified to press "RECORD" on the
cassette recorder, then press "RETURN" on the keyboard to notify the system that the
cassette recorder was recording. The system wrote a sound to provide time
synchronization, then modulated sounds that encoded a prefix, the data, a checksum and a
suffix. When the system needed to read data, the user was instructed to press "PLAY" on
the cassette recorder. The system would listen to the sounds on the tape waiting until a
burst of sound could be recognized as the synchronization. The system would then
interpret subsequent sounds as data. When the data read was complete, the system would
notify the user to press "STOP" on the cassette recorder. It was primitive, but it worked (a
lot of the time). Data was stored sequentially, usually in an unnamed format, although
some systems (such as the Commodore PET series of computers) did allow the files to be
named. Multiple sets of data could be written and located by fast-forwarding the tape and
observing at the tape counter to find the approximate start of the next data region on the
tape. The user might have to listen to the sounds to find the right spot to begin playing the
next data region. Some implementations even included audible sounds interspersed with
the data.
1.2.10 Flat file systems
In a flat file system, there are no subdirectories.
When floppy disk media was first available this type of file system was adequate due to
the relatively small amount of data space available. CP/M machines featured a flat file
system, where files could be assigned to one of 16 user areas and generic file operations
narrowed to work on one instead of defaulting to work on all of them. These user areas
were no more than special attributes associated with the files, that is, it was not necessary
to define specific quota for each of these areas and files could be added to groups for as
long as there was still free storage space on the disk. The Apple Macintosh also featured a
flat file system, the Macintosh File System. It was unusual in that the file management
program (Macintosh Finder) created the illusion of a partially hierarchical filing system on
top of EMFS. This structure required every file to have a unique name, even if it appeared
to be in a separate folder.
While simple, flat file systems become awkward as the number of files grows and makes it
difficult to organize data into related groups of files.
A recent addition to the flat file system family is Amazon's S3, a remote storage service,
which is intentionally simplistic to allow users the ability to customize how their data is
stored. The only constructs are buckets (imagine a disk drive of unlimited size) and
objects (similar, but not identical to the standard concept of a file). Advanced file
management is allowed by being able to use nearly any character (including '/') in the
object's name, and the ability to select subsets of the bucket's content based on identical
prefixes.

1.3 FILE SYSTEMS AND OPERATING SYSTEMS
Many operating systems include support for more than one file system. Sometimes the OS
and the file system are so tightly interwoven it is difficult to separate out file system
functions.
There needs to be an interface provided by the operating system software between the
user and the file system. This interface can be textual (such as provided by a command line
interface, such as the Unix shell, or OpenVMS DCL) or graphical (such as provided by
a graphical user interface, such as file browsers). If graphical, the metaphor of the folder,
containing documents, other files, and nested folders is often used (see also: directory and
folder).
1.3.1 Unix-like operating systems
Unix-like operating systems create a virtual file system, which makes all the files on all the
devices appear to exist in a single hierarchy. This means, in those systems, there is
one root directory, and every file existing on the system is located under it somewhere.
Unix-like systems can use a RAM disk or network shared resource as its root directory.
Unix-like systems assign a device name to each device, but this is not how the files on that
device are accessed. Instead, to gain access to files on another device, the operating system
must first be informed where in the directory tree those files should appear. This process
is called mounting a file system. For example, to access the files on a CD-ROM, one must
tell the operating system "Take the file system from this CD-ROM and make it appear
under such-and-such directory". The directory given to the operating system is called
the mount point it might, for example, be /media. The /media directory exists on many
Unix systems (as specified in the Filesystem Hierarchy Standard) and is intended
specifically for use as a mount point for removable media such as CDs, DVDs, USB drives or
floppy disks. It may be empty, or it may contain subdirectories for mounting individual
devices. Generally, only the administrator (i.e. root user) may authorize the mounting of
file systems.
Unix-like operating systems often include software and tools that assist in the mounting
process and provide it new functionality. Some of these strategies have been coined "auto-
mounting" as a reflection of their purpose.
1. In many situations, file systems other than the root need to be available as soon as
the operating system has booted. All Unix-like systems therefore provide a facility
for mounting file systems at boot time. System administrators define these file
systems in the configuration file fstab (vfstab in Solaris), which also indicates
options and mount points.
2. In some situations, there is no need to mount certain file systems at boot time,
although their use may be desired thereafter. There are some utilities for Unix-like
systems that allow the mounting of predefined file systems upon demand.
3. Removable media have become very common with microcomputer platforms. They
allow programs and data to be transferred between machines without a physical
connection. Common examples include USB flash drives, CD-ROMs, and DVDs.
Utilities have therefore been developed to detect the presence and availability of a
medium and then mount that medium without any user intervention.
4. Progressive Unix-like systems have also introduced a concept
called supermounting; see, for example, the Linux supermount-ng project. For
example, a floppy disk that has been supermounted can be physically removed
from the system. Under normal circumstances, the disk should have been
synchronized and then unmounted before its removal. Provided synchronization
has occurred, a different disk can be inserted into the drive. The system
automatically notices that the disk has changed and updates the mount point
contents to reflect the new medium. Similar functionality is found on Windows
machines.
5. An automounter will automatically mount a file system when a reference is made to
the directory atop which it should be mounted. This is usually used for file systems
on network servers, rather than relying on events such as the insertion of media,
as would be appropriate for removable media.

1.3.2 Linux
Linux supports many different file systems, but common choices for the system disk on a
block device include the ext* family (such
as ext2, ext3 and ext4), XFS, JFS, ReiserFS and btrfs. For raw flash without a flash
translation layer (FTL) or Memory Technology Device (MTD), there is UBIFS, JFFS2,
and YAFFS, among others. SquashFS is a common compressed read-only file system.

1.3.3 Solaris
The Sun Microsystems Solaris operating system in earlier releases defaulted to (non-
journaled or non-logging) UFS for bootable and supplementary file systems. Solaris
defaulted to, supported, and extended UFS.
Support for other file systems and significant enhancements were added over time,
including Veritas Software Corp. (Journaling) VxFS, Sun Microsystems (Clustering) QFS,
Sun Microsystems (Journaling) UFS, and Sun Microsystems (open source, poolable, 128 bit
compressible, and error-correcting) ZFS.
Kernel extensions were added to Solaris to allow for bootable Veritas VxFS operation.
Logging or Journaling was added to UFS in Sun's Solaris 7. Releases of Solaris 10, Solaris
Express,OpenSolaris, and other open source variants of the Solaris operating system later
supported bootable ZFS.
Logical Volume Management allows for spanning a file system across multiple devices for
the purpose of adding redundancy, capacity, and/or throughput. Legacy environments in
Solaris may useSolaris Volume Manager (formerly known as Solstice DiskSuite.) Multiple
operating systems (including Solaris) may use Veritas Volume Manager. Modern Solaris
based operating systems eclipse the need for Volume Management through leveraging
virtual storage pools in ZFS.

1.3.4 OS X
OS X uses a file system that it inherited from classic Mac OS called HFS Plus, sometimes
called Mac OS Extended. HFS Plus is a metadata-rich and case-preserving but
(usually)case-insensitive file system. Due to the Unix roots of OS X, Unix permissions were
added to HFS Plus. Later versions of HFS Plus added journaling to prevent corruption of
the file system structure and introduced a number of optimizations to the allocation
algorithms in an attempt to defragment files automatically without requiring an external
defragmenter.
Filenames can be up to 255 characters. HFS Plus uses Unicode to store filenames. On OS X,
the filetype can come from the type code, stored in file's metadata, or the filename
extension.
HFS Plus has three kinds of links: Unix-style hard links, Unix-style symbolic
links and aliases. Aliases are designed to maintain a link to their original file even if they
are moved or renamed; they are not interpreted by the file system itself, but by the File
Manager code in userland.
OS X also supports the UFS file system, derived from the BSD Unix Fast File System
via NeXTSTEP. However, as of Mac OS X 10.5 (Leopard), OS X can no longer be installed on
a UFS volume, nor can a pre-Leopard system installed on a UFS volume be upgraded to
Leopard.
Newer versions of OS X are capable of reading and writing to the legacy FAT file
systems(16 & 32) common on Windows. They are also capable of reading the
newer NTFS file systems for Windows. In order to write to NTFS file systems on OS X
versions prior to 10.6 (Snow Leopard) third party software is necessary. Mac OS X 10.6
(Snow Leopard) and later allows writing to NTFS file systems, but only after a non-trivial
system setting change (third party software exists that automates this).

1.3.4 PC-BSD
PC-BSD is a desktop version of FreeBSD, which inherits FreeBSD's ZFS support, similarly
to FreeNAS. The new graphical installer of PC-BSD can handle / (root) on ZFS and RAID-
Z pool installs and disk encryption using Geli right from the start in an easy convenient
(GUI) way. The current PC-BSD 9.0+ 'Isotope Edition' has ZFS filesystem version 5 and ZFS
storage pool version 28.

1.3.5 Plan 9
Plan 9 from Bell Labs treats everything as a file, and accessed as a file would be (i.e.,
no ioctl or mmap): networking, graphics, debugging, authentication, capabilities,
encryption, and other services are accessed via I-O operations on file descriptors.
The 9P protocol removes the difference between local and remote files.
These file systems are organized with the help of private, per-process namespaces,
allowing each process to have a different view of the many file systems that provide
resources in a distributed system.
The Inferno operating system shares these concepts with Plan 9.

1.3.6 Microsoft Windows

Directory listing in a Windows command shell
Windows makes use of the FAT, NTFS, exFAT and ReFS file systems (the latter is only
supported and usable in Windows Server 2012; Windows cannot boot from it).
Windows uses a drive letter abstraction at the user level to distinguish one disk or
partition from another. For example, the pathC:\WINDOWS represents a
directory WINDOWS on the partition represented by the letter C. Drive C: is most
commonly used for the primary hard disk partition (since at the advent of hard disks
many computers had two floppy drives, A: and B:), on which Windows is usually installed
and from which it boots. This "tradition" has become so firmly ingrained that bugs exist in
many applications which make assumptions that the drive that the operating system is
installed on is C. The use of drive letters, and the tradition of using "C" as the drive letter
for the primary hard disk partition, can be traced to MS-DOS, where the letters A and B
were reserved for up to two floppy disk drives. This in turn derived from CP/M in the
1970s, and ultimately from IBM's CP/CMS of 1967.
1.3.6.1 FAT: File Allocation Table
The family of FAT file systems is supported by almost all operating systems for personal
computers, including all versions of Windows and MS-DOS/PC DOS and DR-DOS. (PC DOS
is an OEM version of MS-DOS, MS-DOS was originally based on SCP's 86-DOS. DR-DOS was
based on Digital Research's Concurrent DOS, a successor of CP/M-86.) The FAT file
systems are therefore well-suited as a universal exchange format between computers and
devices of most any type and age.
The FAT file system traces its roots back to an (incompatible) 8-bit FAT precursor
in Stand-alone Disk BASIC and the short-lived MDOS/MIDAS project.
[citation needed]

Over the years, the file system has been expanded from FAT12 to FAT16 and FAT32.
Various features have been added to the file system
including subdirectories, codepage support, extended attributes, and long filenames.
Third-parties such as Digital Research have incorporated optional support for deletion
tracking, and volume/directory/file-based multi-user security schemes to support file and
directory passwords and permissions such as read/write/execute/delete access rights.
Most of these extensions are not supported by Windows.
The FAT12 and FAT16 file systems had a limit on the number of entries in the root
directory of the file system and had restrictions on the maximum size of FAT-formatted
disks or partitions.
FAT32 addresses the limitations in FAT12 and FAT16, except for the file size limit of close
to 4 GB, but it remains limited compared to NTFS.
FAT12, FAT16 and FAT32 also have a limit of eight characters for the file name, and three
characters for the extension (such as .exe). This is commonly referred to as the 8.3
filename limit.VFAT, an optional extension to FAT12, FAT16 and FAT32, introduced
in Windows 95 and Windows NT 3.5, allowed long file names (LFN) to be stored in the
FAT file system in a backwards compatible fashion.
1.3.6.2 NTFS
NTFS, introduced with the Windows NT operating system, allowed ACL-based permission
control. Other features also supported by NTFS include hard links, multiple file streams,
attribute indexing, quota tracking, sparse files, encryption, compression, and reparse
points (directories working as mount-points for other file systems, symlinks, junctions,
remote storage links), though not all these features are well-documented.
]

1.3.6.3 exFAT
exFAT is a proprietary and patent-protected file system with certain advantages over
NTFS with regard to file system overhead.
exFAT is not backwards compatible with FAT file systems such as FAT12, FAT16 or
FAT32. The file system is supported with newer Windows systems, such as Windows
2003, Windows Vista, Windows 2008, Windows 7, Windows 8, and more recently, support
has been added for Windows XP. Support in other operating systems is sparse since
Microsoft has not published the specifications of the file system and implementing support
for exFAT requires a license.

1.3.7 OpenVMS
MVS [IBM Mainframe
Other file systems
The Prospero File System is a file system based on the Virtual System Model. The
system was created by Dr. B. Clifford Neuman of the Information Sciences Institute at
the University of Southern California.
RSRE FLEX file system - written in ALGOL 68
The file system of the Michigan Terminal System (MTS) is interesting because: (i) it
provides "line files" where record lengths and line numbers are associated as
metadata with each record in the file, lines can be added, replaced, updated with the
same or different length records, and deleted anywhere in the file without the need to
read and rewrite the entire file; (ii) using program keys files may be shared or
permitted to commands and programs in addition to users and groups; and (iii) there
is a comprehensive file locking mechanism that protects both the file's data and its
metadata.

1.4 LIMITATIONS
Converting the type of a file system
It may be advantageous or necessary to have files in a different file system than they
currently exist. Reasons include the need for an increase in the space requirements
beyond the limits of the current file system. The depth of path may need to be increased
beyond the restrictions of the file system. There may be performance or reliability
considerations. Providing access to another operating system which does not support
existing file system is another reason.
In-place conversion
In some cases conversion can be done in-place, although migrating the file system is more
conservative, as it involves a creating a copy of the data and is recommended.
[18]
On
Windows, FAT and FAT32 file systems can be converted to NTFS via the convert.exe
utility, but not the reverse. On Linux, ext2 can be converted to ext3 (and converted back),
and ext3 can be converted to ext4 (but not back), and both ext3 and ext4 can be converted
to btrfs, and converted back until the undo information is deleted. These conversions are
possible due to using the same format for the file data itself, and relocating the metadata
into empty space, in some cases using sparse file support.
Migrating to a different file system
Migration has the disadvantage of requiring additional space although it may be faster.
The best case is if there is unused space on media which will contain the final file system.
For example, to migrate a FAT32 file system to an ext2 file system. First create a new ext2
file system, then copy the data to the file system, then delete the FAT32 file system.
An alternative, when there is not sufficient space to retain the original file system until the
new one is created, is to use a work area (such as a removable media). This takes longer
but a backup of the data is a nice side effect.
Long file paths and long file names
In hierarchical file systems, files are accessed by means of a path that is a branching list of
directories containing the file. Different file systems have different limits on the depth of
the path. File systems also have a limit on the length of an individual filename.
Copying files with long names or located in paths of significant depth from one file system
to another may cause undesirable results. This depends on how the utility doing the
copying handles the discrepancy.

1.5 FILENAME
A filename (also written as two words, file name) is a name used to uniquely identify
a computer file stored in a file system. Different file systems impose different restrictions
on filename lengths and the allowed characters within filenames.
A filename may include one or more of these components:
host (or node or server) network device that contains the file
device (or drive) hardware device or drive
directory (or path) directory tree (e.g., /usr/bin, \TEMP, [USR.LIB.SRC], etc.)
file base name of the file
type (format or extension) indicates the content type of the file (e.g., .txt, .exe, .COM,
etc.)
version revision or generation number of the file
The components required to identify a file varies across operating systems, as does the
syntax and format for a valid filename.
Around 1962, the Compatible Time-Sharing System introduced the concept of a file (i.e.,
non-paper file).
Around this same time appeared the dot (period or full-stop) as a filename extension
separator, and the limit to three letter extensions might have come from RAD50 16-bit
limits.
[1]

Traditionally, filenames allowed only alphanumeric characters, but as time progressed,
the number of characters allowed increased. This led to incompatibility problems when
moving files from one file system to another.
[2]

Around 1995, VFAT, a variant of the FAT filesystem with an extended directory format,
was introduced in Windows 95 and Windows NT 3.5. It allowed mixed-case Unicode long
filenames (LFNs), in addition to classic "8.3" names.
In 1985, RFC 959 officially defined a pathname to be the character string which must be
entered into a file system by a user in order to identify a file.
[3]

OS X 10.3 marked Apple's adoption of Unicode 3.2 character decomposition, superseding
the Unicode 2.1 decomposition used previously. This change caused problems for
developers writing software for OS X.

1.6 FILE ATTRIBUTE
File attributes are metadata associated with computer files that define file system
behavior. Each attribute can have one of two states: set and cleared. Attributes are
considered distinct from other metadata, such as dates and times, filename
extensions or files system permissions. In addition to files, folders, volumes and other file
system objects may have attributes.
Traditionally, in MS-DOS and Microsoft Windows, there were four attributes: archive,
hidden, read-only and system. Windows has added new ones. Systems derived
from 4.4BSD-Lite, such asFreeBSD, NetBSD, OpenBSD, DragonFly BSD, and OS X, have sets
of "system" and "user" attributes; newer versions of the Linux kernel also support a set of
file attributes.

1.7 TYPES
1.7.1 DOS and Windows
Traditionally, in DOS and Microsoft Windows, files and folders accepted four attributes:
Archive: When set, it indicates that the hosting file has changed since the
last backup operation. Windows' file system sets this attribute on any file that has
changed. Backup software then has the duty of clearing it upon a successful backup.
Hidden: When set, indicates that the hosting file is hidden. MS-DOS commands
like dir and Windows apps like File Explorer do not show hidden files by default,
unless asked to do so.
[4]

System: When set, indicates that the hosting file is a critical system file that is
necessary for the computer to operate properly. MS-DOS and Microsoft Windows use
it to mark important system files. MS-DOS commands like dir and Windows apps like
File Explorer do not show system files by default even when hidden files are shown,
unless asked to do so.
Read-only: When set, indicates that a file should not be altered. Upon opening the file,
file system API usually does not grant write permission to the requesting application,
unless the application explicitly requests it. Read-only attributes on folders are usually
ignored.
As new versions of Windows came out, Microsoft has added to the inventory of available
attributes on the NTFS file system, including but not limited to:
Compressed: When set, Windows compresses the hosting file upon storage. For more
information, see NTFS File compression.
Encrypted: When set, Windows encrypts the hosting file upon storage to prevent
unauthorized access. For more information, see NTFS Encrypting File System (EFS).
Indexed: When set, Indexing Service or Windows Search do not include the hosting file
in their indexing operation.

1.7.2 4.4BSD-Lite and derivatives
In 4.4BSD and 4.4BSD-Lite, files and directories (folders) accepted four attributes that
could be set by the owner of the file or the superuser (the "User" attributes) and two
attributes that could only be set by the superuser (the "System" attributes):
(User) No-dump: When set, it indicates that the file or directory should not be saved
during a backup operation.
(User and System) Immutable: When set, indicates that the file or directory should not
be altered. Attempts to open the file for writing, create a file within the directory,
remove a file from the directory, rename a file within the directory, rename the file or
directory, or remove the file or directory will fail with a permissions error.
(User and System) Append-only: When set, indicates that the file should only be
appended to.
(User) Opaque: When set on a directory, indicates that the directory is opaque when
viewed through a union stack.
FreeBSD added some additional attributes, also supported by DragonFly BSD
(User and System) No-unlink: When set, indicates that the file or directory should not
be renamed or removed. Attempts to rename or remove the file or directory will fail
with a permissions error.
FreeBSD also supports
(System) No-archive: When set, indicates that the file or directory should not be
archived.
(System) Snapshot: When set, indicates that the file or directory is a snapshot file. This
attribute is maintained by the system, and cannot be set, even by the super-user.
whereas DragonFly BSD supports:
[10]

(User and System) No-history: When set, indicates that history should not be retained
for the file or directory.
(User) Swapcache: When set, indicates that clean filesystem data for the file, or for the
directory and everything underneath the directory, should be cached in swap space on
a solid-state disk.
(System) Swapcache: When set, indicates that clean filesystem data for the file, or for
the directory and everything underneath the directory, should not be cached in swap
space on a solid-state disk.
(System) Archived: When set, indicates that the file or directory may be archived.
NetBSD added an additional attribute, also supported by OpenBSD:
(System) Archived: When set, indicates that the file or directory is archived.
OS X added an additional attribute:
(User) Hidden: When set, indicates that the file or directory should not, by default, be
displayed in the GUI; (ls will display it, however).

1.8 NAMING FILES, PATHS, AND NAMESPACES
All file systems supported by Windows use the concept of files and directories to access
data stored on a disk or device. Windows developers working with the Windows APIs for
file and device I/O should understand the various rules, conventions, and limitations of
names for files and directories.
Data can be accessed from disks, devices, and network shares using file I/O APIs. Files and
directories, along with namespaces, are part of the concept of a path, which is a string
representation of where to get the data regardless if it's from a disk or a device or a
network connection for a specific operation.
Some file systems, such as NTFS, support linked files and directories, which also follow file
naming conventions and rules just as a regular file or directory would. For additional
information, see Hard Links and Junctions and Reparse Points and File Operations.
1.8.1 File and Directory Names
All file systems follow the same general naming conventions for an individual file: a base
file name and an optional extension, separated by a period. However, each file system,
such as NTFS, CDFS, exFAT, UDFS, FAT, and FAT32, can have specific and differing rules
about the formation of the individual components in the path to a directory or file. Note
that adirectory is simply a file with a special attribute designating it as a directory, but
otherwise must follow all the same naming rules as a regular file. Because the
term directory simply refers to a special type of file as far as the file system is concerned,
some reference material will use the general term file to encompass both concepts of
directories and data files as such. Because of this, unless otherwise specified, any naming
or usage rules or examples for a file should also apply to a directory. The term path refers
to one or more directories, backslashes, and possibly a volume name. For more
information, see the Paths section.
Character count limitations can also be different and can vary depending on the file
system and path name prefix format used. This is further complicated by support for
backward compatibility mechanisms. For example, the older MS-DOS FAT file system
supports a maximum of 8 characters for the base file name and 3 characters for the
extension, for a total of 12 characters including the dot separator. This is commonly
known as an 8.3 file name. The Windows FAT and NTFS file systems are not limited to 8.3
file names, because they have long file name support, but they still support the 8.3 version
of long file names.
1.8.2 Naming Conventions
The following fundamental rules enable applications to create and process valid names for
files and directories, regardless of the file system:
Use a period to separate the base file name from the extension in the name of a directory
or file.
Use a backslash (\) to separate the components of a path. The backslash divides the file
name from the path to it, and one directory name from another directory name in a path.
You cannot use a backslash in the name for the actual file or directory because it is a
reserved character that separates the names into components.
Use a backslash as required as part of volume names, for example, the "C:\" in
"C:\path\file" or the "\\server\share" in "\\server\share\path\file" for Universal Naming
Convention (UNC) names. For more information about UNC names, see the Maximum Path
Length Limitation section.
Do not assume case sensitivity. For example, consider the names OSCAR, Oscar, and oscar
to be the same, even though some file systems (such as a POSIX-compliant file system)
may consider them as different. Note that NTFS supports POSIX semantics for case
sensitivity but this is not the default behavior. For more information, see CreateFile.
Volume designators (drive letters) are similarly case-insensitive. For example, "D:\" and
"d:\" refer to the same volume.
Use any character in the current code page for a name, including Unicode characters and
characters in the extended character set (128255), except for the following:
o The following reserved characters:
< (less than)
> (greater than)
: (colon)
" (double quote)
/ (forward slash)
\ (backslash)
| (vertical bar or pipe)
? (question mark)
* (asterisk)
o Integer value zero, sometimes referred to as the ASCII NUL character.
o Characters whose integer representations are in the range from 1 through 31, except
for alternate data streams where these characters are allowed. For more information
about file streams, see File Streams.
o Any other character that the target file system does not allow.
Use a period as a directory component in a path to represent the current directory, for
example ".\temp.txt". For more information, see Paths.
Use two consecutive periods (..) as a directory component in a path to represent the parent
of the current directory, for example "..\temp.txt". For more information, see Paths.
Do not use the following reserved names for the name of a file:
CON, PRN, AUX, NUL, COM1, COM2, COM3, COM4, COM5, COM6, COM7, COM8, COM9,
LPT1, LPT2, LPT3, LPT4, LPT5, LPT6, LPT7, LPT8, and LPT9. Also avoid these names
followed immediately by an extension; for example, NUL.txt is not recommended. For
more information, see Namespaces.
Do not end a file or directory name with a space or a period. Although the underlying file
system may support such names, the Windows shell and user interface does not. However,
it is acceptable to specify a period as the first character of a name. For example, ".temp".
1.8.3 Short vs. Long Names
A long file name is considered to be any file name that exceeds the short MS-DOS (also
called 8.3) style naming convention. Typically, Windows stores long file names on disk as
special directory entries, which can be disabled system wide for performance reasons
depending on the particular file system. When you create a long file name, Windows may
also create a short 8.3 form of the name, called the 8.3 alias, and store it on disk also. This
8.3 aliasing can be disabled for a specified volume.

1.9 FILE SYSTEM STRUCTURE
The file system structure is the most basic level of organization in an operating system.
Almost all of the ways an operating system interacts with its users, applications, and
security model are dependent upon the way it organizes files on storage devices.
Providing a common file system structure ensures users and programs are able to access
and write files.
File systems break files down into two logical categories:
o Shareable vs. unsharable files
o Variable vs. static files
Shareable files are those that can be accessed locally and by remote
hosts; unsharable files are only available locally. Variable files, such as documents, can be
changed at any time; static files, such as binaries, do not change without an action from the
system administrator.
The reason for looking at files in this manner is to help correlate the function of the file
with the permissions assigned to the directories which hold them. The way in which the
operating system and its users interact with a given file determines the directory in which
it is placed, whether that directory is mounted with read-only or read/write permissions,
and the level of access each user has to that file. The top level of this organization is
crucial. Access to the underlying directories can be restricted or security problems could
manifest themselves if, from the top level down, it does not adhere to a rigid structure.
Memory Map
A memory map is a massive table, in effect a database, that comprises complete
information about how the memory is structured in a computer system. A memory map
works something like a gigantic office organizer. In the map, each computer file has a
unique memory address reserved especially for it, so that no other data can inadvertently
overwrite or corrupt it.
In order for a computer to function properly, its OS (operating system) must always be
able to access the right parts of its memory at the right times. When a computer first boots
up(starts), the memory map tells the OS how much memory is available. As the computer
runs, the memory map ensures that data is always written to, and read from, the proper
places. The memory map also ensures that the computer's debuggers can resolve memory
addresses to actual stored data.
If there were no memory map, or if an existing memory map got corrupted, the OS might
(and probably would) write data to, and read data from, the wrong places. As a result,
when data was read, it would not always pertain to the appropriate files or application
programs. The problem would likely start out small and unnoticeable, worsen with time,
and become apparent only after considerable damage had been done to stored data and
programs. In the end, some or all of the applications would fail to run, and many critical
data files would be ruined.

1.10 MEMORY-MAPPED FILE
A memory-mapped file is a segment of virtual memory which has been assigned a direct
byte-for-byte correlation with some portion of a file or file-like resource. This resource is
typically a file that is physically present on-disk, but can also be a device, shared memory
object, or other resource that the operating system can reference through a file descriptor.
Once present, this correlation between the file and the memory space permits applications
to treat the mapped portion as if it were primary memory.
Benefits
The primary benefit of memory mapping a file is increasing I/O performance, especially
when used on large files. For small files, memory-mapped files can result in a waste
of slack space
[1]
as memory maps are always aligned to the page size, which is mostly 4
KiB. Therefore a 5 KiB file will allocate 8 KiB and thus 3 KiB are wasted. Accessing memory
mapped files is faster than using direct read and write operations for two reasons. Firstly,
a system call is orders of magnitude slower than a simple change to a program's local
memory. Secondly, in most operating systems the memory region mapped actually is the
kernel's page cache (file cache), meaning that no copies need to be created in user space.
Certain application level memory-mapped file operations also perform better than their
physical file counterparts. Applications can access and update data in the file directly and
in-place, as opposed to seeking from the start of the file or rewriting the entire edited
contents to a temporary location. Since the memory-mapped file is handled internally in
pages, linear file access (as seen, for example, in flat file data storage or configuration files)
requires disk access only when a new page boundary is crossed, and can write larger
sections of the file to disk in a single operation.
A possible benefit of memory-mapped files is a "lazy loading", thus using small amounts of
RAM even for a very large file. Trying to load the entire contents of a file that is
significantly larger than the amount of memory available can cause severe thrashing as the
operating system reads from disk into memory and simultaneously writes pages from
memory back to disk. Memory-mapping may not only bypass the page file completely, but
the system only needs to load the smaller page-sized sections as data is being edited,
similarly to demand paging scheme used for programs.
The memory mapping process is handled by the virtual memory manager, which is the
same subsystem responsible for dealing with the page file. Memory mapped files are
loaded into memory one entire page at a time. The page size is selected by the operating
system for maximum performance. Since page file management is one of the most critical
elements of a virtual memory system, loading page sized sections of a file into physical
memory is typically a very highly optimized system function.
Drawbacks
The major reason to choose memory mapped file I/O is performance. Nevertheless there
can be tradeoffs. The standard I/O approach is costly due to system call overhead and
memory copying. The memory mapped approach has its cost in minor page faults - when a
block of data is loaded in page cache, but is not yet mapped into the process's virtual
memory space. In some circumstances, memory mapped file I/O can be substantially
slower than standard file I/O.
Another drawback of memory mapped files relates to a given architecture's address space:
a file larger than the addressable space can have only portions mapped at a time,
complicating reading it. For example, a 32-bit architecture such as Intel's IA-32 can only
directly address 4 GiB or smaller portions of files. This drawback is avoided in the case of
devices addressing memory when anIOMMU is present.
Common uses
Perhaps the most common use for a memory-mapped file is the process loader in most
modern operating systems (including Microsoft Windows and Unix-like systems.) When a
process is started, the operating system uses a memory mapped file to bring the
executable file, along with any loadable modules, into memory for execution. Most
memory-mapping systems use a technique called demand paging, where the file is loaded
into physical memory in subsets (one page each), and only when that page is actually
referenced. In the specific case of executable files, this permits the OS to selectively load
only those portions of a process image that actually needs to execute.
Another common use for memory-mapped files is to share memory between multiple
processes. In modern protected mode operating systems, processes are generally not
permitted to access memory space that is allocated for use by another process. (A
program's attempt to do so causes invalid page faults or segmentation violations.) There
are a number of techniques available to safely share memory, and memory-mapped file
I/O is one of the most popular. Two or more applications can simultaneously map a single
physical file into memory and access this memory. For example, the Microsoft Windows
operating system provides a mechanism for applications to memory-map a shared
segment of the system's page file itself and share data via this section.
Platform support
Most modern operating systems or runtime environments support some form of memory-
mapped file access. The function mmap(), which creates a mapping of a file given a file
descriptor, starting location in the file, and a length, is part of the POSIX specification, so
the wide variety of POSIX-compliant systems, such as UNIX, Linux, Mac OS X or OpenVMS,
support a common mechanism for memory mapping files. The Microsoft Windows
operating systems also support a group of API functions for this purpose, such
as CreateFileMapping().
The Boost C++ Libraries provide a portable implementation of memory-mapped files for
Microsoft Windows and POSIX-compliant platforms.
[8]
Another minimalistic and portable
implementation for Microsoft Windows and POSIX-compliant platforms, based on the
C++ Standard Template Library (STL), is available on SourceForge.
The Java programming language provides classes and methods to access memory mapped
files, such as FileChannel.
The D programming language supports memory mapped files in its standard library
(std.mmfile module).
Ruby has a gem (library) called Mmap, which implements memory-mapped file objects.
Since version 1.6, Python has included a mmap module in its Standard Library. Details of
the module vary according to whether the host platform is Windows or Unix-like.
For Perl there are a several modules available for memory mapping files on the CPAN,
such as Sys::Mmap and File::Map.
In the Microsoft .NET runtime, P/Invoke can be used to use memory mapped files directly
through the Windows API. Managed access (P/Invoke not necessary) to memory mapped
files was introduced in version 4 of the runtime (see Memory-Mapped Files). For previous
versions, there are third-party libraries which provide managed API's.

1.11 DISK SPACE MANAGEMENT METHODS
Disk Space Management tools provide data that system administrators need to
track disk space availability for designated systems, typically as part of Service Level
Agreement (SLA) measures. A disk space management tool can establish baseline
measures against which administrators can track ongoing capacity. A disk space
management tool makes it possible to collect metrics and monitor disk space availability
around the clock.
Operating system maintains a list of free disk spaces to keep track of all disk blocks which
are not being used by any file. Whenever a file has to be created, the list of free disk space
is searched for and then allocated to the new file. The amount of space allocated to this file
is then removed from the free space list. When a file is deleted, its disk space is added to
the free space list. In this section we will discuss two methods to manage free disk blocks.
Linked List
In this method all free disk blocks are linked together by each free block pointing to the
next free block. There must be another pointer pointing to the first free block of a linked
list which will be pointing to the second free block which would further point to the third
free block and so on.
This scheme is not very efficient since to traverse the list, we must read each block
requiring substantial time. The second disadvantage is additional memory requirement
for maintaining linked list of all free disk blocks. With a I K block and 16 bit disk block
number, each block on the free list holds the numbers of 512 free blocks. A 20M disk needs
a free list of maximum 40 blocks to hold all 20,000 disk block numbers.
ap
Most often the list of free disk spaces is implemented as a bit map or bit vector. Each block
is represented by a single bit. 0(zero) is marked as a free block whereas 1 is for allocated
block.
For example consider a disk (figure 4) where blocks 3,5,7,9,10,13,15,20 & 28 are free and
the rest of the blocks are allocated to a file. The bit map for free disk blocks would he 1 1 1
0 1 0 1 0 1 0 0 1 1 0 1 0 1 1 1 1 0 1 1 1 1 1 1 1 0 1. One advantage of this approach is that it
is a simple method and efficient to find n consecutive free blocks on the disk. But the
disadvantage is that extra disk space is required to store bit map. A disk with n blocks
require a bit map of n bits.

A 80 MB disk requires 80000 bits (1 BLOCK = 1 K byte) for the map. Compared to linked
list the bit map requires less space since it uses 1 bit per block, versus 16 bits in the linked
list model. Only if the disk is nearly full with the linked list it requires fewer blocks than
the bit map. In case there is enough main memory to hold bit map, this method is better
than linked list. If, however, only one block of memory can be spared for keeping track of
free disk blocks, and the disk is nearly full, then the linked list may be preferable.




















MODULE TWO
COMPUTER SECURITY
Computer security refers to the protection of all componentshardware, software, and
stored dataof a computer or a group of computers from damage, theft, or unauthorized
use. A computer security plan that is well thought out, implemented, and monitored makes
authorized computer use easy and unauthorized use or accidental damage difficult or
impossible.
Microsoft included security as part of the initial design specifications for Windows NT, and
it is pervasive in the operating system. The security model includes components to control
who accesses which objects (such as files and shared printers), which actions an
individual can take on an object, and which events are audited.

Unix security
Unix security refers to the means of securing a Unix or Unix-like operating system. A
secure environment is achieved not only by the design concepts of these operating
systems, but also through vigilant user and administrative practices.

2.1 SYSTEM RELIABILITY
The probability that a system will accurately perform its specified task under stated
environmental conditions.
Reliability can also be regarded as an attribute of any computer-related component
(software, or hardware, or a network, for example) that consistently performs according
to its specifications. It has long been considered one of three related attributes that must
be considered when making, buying, or using a computer product or component.
Reliability, availability, and serviceability - RAS, for short - are considered to be important
aspects to design into any system. In theory, a reliable product is totally free of technical
errors; in practice, however, vendors frequently express a product's reliability quotient as
a percentage. Evolutionary products (those that have evolved through numerous versions
over a significant period of time) are usually considered to become increasingly reliable,
since it is assumed that bug s have been eliminated in earlier releases. For example,
IBM's z/OS (an operating system for their S/390server series), has a reputation for
reliability because it evolved from a long line of earlierMVS and OS/390 operating system
versions.

2.2 DESIGN CONCEPTS
Permissions
A core security feature in these systems is the permissions system. All files in a typical
Unix-style file system have permissions set enabling different access to a file.
Permissions on a file are commonly set using the chmod command and seen through
the is command. For example:
-r-xr-xr-x 1 root wheel 745720 Sep 8 2002 /bin/sh
Unix permissions permit different users access to a file. Different user groups have
different permissions on a file.
More advanced Unix filesystems include the Access Control List concept which allows
permissions to be granted to multiple users or groups. An Access Control List may be used
to grant permission to additional individual users or groups. For example:
/pvr [u::rwx,g::r-x,o::r-x/u::rwx,u:sue:rwx,g::r-x,m::rwx,o::r-x]
In this example, which is from the chacl command on the Linux operating system, the
user sue is granted write permission to the /pvr directory.
User groups
Users under Unix style operating systems often belong to managed groups with specific
access permissions. This enables users to be grouped by the level of access they have to
this system. Many Unix implementations add an additional layer of security by requiring
that a user be a member of the wheel user privileges group in order to access
the su command.
Issues
Most Unix and Unix-like systems have an account or group which enables a user to exact
complete control over the system, often known as a root account. If access to this account
is gained by an unwanted user, this results in a complete breach of the system. A root
account however is necessary for administrative purposes, and for the above security
reasons the root account is seldom used for day to day purposes (the sudo program is
more commonly used), so usage of the root account can be more closely monitored.
Root access "as it should be" can be visualised by those familiar with the Superman stories
using the following analogy:
Using a root account is rather like being Superman; an administrator's regular user
is more like Clark Kent. Clark Kent becomes Superman for only as long as
necessary, in order to save people. He then reverts to his "disguise". Root access
should be used in the same fashion. The Clark Kent disguise doesn't really restrict
him though, as he is still able to use his super powers. This is analogous to using the
sudo program.
User and administrative techniques
Unix has many tools that can improve security if used properly by users and
administrators.
Passwords
Selecting a strong password and guarding it properly are probably the most important
things a user can do to improve Unix security. In Unix systems, the essential
information about users is stored under the file /etc/passwd. This file keeps track of
the users registered in the system and their main definitions. Passwords, or more
correctly, the hash of the password, can also be stored in the same place. The entries
in /etc/passwd occupy exactly one line each, and have the following form:
nickname:password_hash:UserID:GroupID:Complete_Name:home_dir:shell_bin

2.3 VULNERABILITY
In computer security, vulnerability is a weakness which allows an attacker to reduce a
system's information assurance.
Vulnerability is the intersection of three elements: a system susceptibility or flaw, attacker
access to the flaw, and attacker capability to exploit the flaw. To exploit a vulnerability, an
attacker must have at least one applicable tool or technique that can connect to a system
weakness. In this frame, vulnerability is also known as the attack surface.
Vulnerability management is the cyclical practice of identifying, classifying, remediating,
and mitigating vulnerabilities. This practice generally refers to software vulnerabilities in
computing systems.
A security risk may be classified as a vulnerability. The use of vulnerability with the same
meaning of risk can lead to confusion. The risk is tied to the potential of a significant loss.
Then there are vulnerabilities without risk: for example when the affected asset has no
value. A vulnerability with one or more known instances of working and fully
implemented attacks is classified as an exploitable vulnerability a vulnerability for
which an exploit exists. The window of vulnerability is the time from when the security
hole was introduced or manifested in deployed software, to when access was removed, a
security fix was available/deployed, or the attacker was disabledsee zero-day attack.
Security bug (security defect) is a narrower concept: there are vulnerabilities that are not
related to software: hardware, site, personnel vulnerabilities are examples of
vulnerabilities that are not software security bugs.
Constructs in programming languages that are difficult to use properly can be a large
source of vulnerabilities.

2.4 COMPUTER WORM
A computer worm is a standalone malware computer program that replicates itself in
order to spread to other computers. Often, it uses a computer network to spread itself,
relying on security failures on the target computer to access it. Unlike a computer virus, it
does not need to attach itself to an existing program. Worms almost always cause at least
some harm to the network, even if only by consuming bandwidth, whereas viruses almost
always corrupt or modify files on a targeted computer.
Many worms that have been created are designed only to spread, and do not attempt to
change the systems they pass through. However, as theMorris
worm and Mydoom showed, even these "payload free" worms can cause major disruption
by increasing network traffic and other unintended effects. A "payload" is code in the
worm designed to do more than spread the wormit might delete files on a host system
(e.g., the ExploreZipworm), encrypt files in a cryptoviral extortion attack, or send
documents via e-mail. A very common payload for worms is to install a backdoor in the
infected computer to allow the creation of a "zombie" computer under control of the worm
author. Networks of such machines are often referred to asbotnets and are very commonly
used by spam senders for sending junk email or to cloak their website's
address. Spammers are therefore thought to be a source of funding for the creation of such
worms, and the worm writers have been caught selling lists of IP addresses of infected
machines. Others try to blackmail companies with threatened DoS attacks.
Backdoors can be exploited by other malware, including worms. Examples
include Doomjuice, which can spread using the backdoor opened byMydoom, and at least
one instance of malware taking advantage of the rootkit and backdoor installed by
the Sony/BMG DRM software utilized by millions of music CDs prior to late 2005.
2.4.1 Protecting against dangerous computer worms
Worms spread by exploiting vulnerabilities in operating systems. Vendors with security
problems supply regular security updates , and if these are installed to a machine then the
majority of worms are unable to spread to it. If a vulnerability is disclosed before the
security patch released by the vendor, a zero-day attack is possible.
Users need to be wary of opening unexpected email, and should not run attached files or
programs, or visit web sites that are linked to such emails. However, as with
the ILOVEYOU worm, and with the increased growth and efficiency of phishing attacks, it
remains possible to trick the end-user into running malicious code.
Anti-virus and anti-spyware software are helpful, but must be kept up-to-date with new
pattern files at least every few days. The use of a firewall is also recommended.
In the AprilJune, 2008, issue of IEEE Transactions on Dependable and Secure Computing,
computer scientists describe a potential new way to combat internet worms. The
researchers discovered how to contain the kind of worm that scans the Internet randomly,
looking for vulnerable hosts to infect. They found that the key is for software to monitor
the number of scans that machines on a network sends out. When a machine starts
sending out too many scans, it is a sign that it has been infected, allowing administrators
to take it off line and check it for malware. In addition, machine learning techniques can be
used to detect new worms, by analyzing the behavior of the suspected computer.


2.4.2 Internet Worm
An Internet worm is type of malicious software (malware) that self-replicates and
distributes copies of itself to its network. These independent virtual viruses spread
through the Internet, break into computers, and replicate without intervention from and
unbeknownst to computer users.
Internet worms can be included in any type of virus, script or program. These worms
typically infect systems by exploiting bugs or vulnerabilities that can often be found in
legitimate software. Unlike Trojans or other viruses that require user intervention to
spread, Internet worms can spread on their own. This makes them extremely dangerous.
Internet worms are also known as computer worms.
Internet worms use various techniques to multiply over the Internet. Initial worms just
scanned local network hard drives and folders, and then inserted themselves into
programs.
In the 1990s, Internet worms came in the form of Visual Basic scripts that replicated on
computers running on Windows. These worms used the user's email to spread themselves
to all the addresses available in the user's address book.
In 2001, Internet worms began to exploit vulnerabilities in the Windows OS to infect
machines directly via the Internet. Later, Microsoft released automatic OS updates to
prevent this problem. Probably the most powerful Internet worm in terms of its scope was
the Code Red Worm, which scanned the Internet and attacked susceptible computers that
ran the Windows IIS Web server.
Internet worms are embedded in software and penetrate most firewalls and other forms
of network security. Anti-virus software applications combat worms along with other
forms of malware such as viruses

2.5 THIRTEEN SECURITY DESIGN PRINCIPLES
1) Secure the weakest link -- Spaf (that is, highly respected security expert Gene
Spafford of Purdue University) teaches this principle with a funny story. Imagine you are
charged with transporting some gold securely from one homeless guy who lives in a park
bench (well call him Linux) to another homeless person who lives across town on a steam
grate (well call her Android). You hire an armored truck to transport the gold. The name
of the transport company is "Applied Crypto, Inc." Now imagine youre an attacker who is
supposed to steal the gold. Would you attack the Applied Crypto truck, Linux the
homeless guy, or Android the homeless woman? Pretty easy experiment, huh? (Hint: the
answer is, "Anything but the crypto.")
As my co-author John Viega and I wrote back in our 2001 book, Building Secure Software,
"Security practitioners often point out that security is a chain; and just as a chain is only as
strong as the weakest link, a software security system is only as secure as its weakest
component." Attackers go after the weakest point in a system, and the weakest point is
rarely a security feature or function. When it comes to secure design, make sure to
consider the weakest link in your system and ensure that it is secure enough.
2) Defend in depth Author and consultant Kenneth van Wyk likes to call this one the
"belt and suspenders" approach. Redundancy and layering is usually a good thing in
security. Dont count on your firewall to block all malicious traffic; use an intrusion
detection system as well. If you are designing an application, prevent single points of
failure with security redundancies and layers of defense. From Building Secure Software,
"The idea behind defense in depth is to manage risk with diverse defensive strategies, so
that if one layer of defense turns out to be inadequate, another layer of defense will
hopefully prevent a full breach." It's a concept preached universally by information
security experts, and for good reason: it works.
3) Fail securely -- Make sure that any system you design does not fail "open." My favorite
story about this principle comes from the ill-fated Microsoft Bob product of
yesteryear. (Bob was the precursor of Clippy the paperclip.) According to legend, if you
failed to get your username and password right after three attempts, Bob would helpfully
notice and ask whether you wanted to pick a new password to use. Thanks Bob (said the
hacker)! Obviously a better default in this situation is to deny access.
From Building Secure Software, "Any sufficiently complex system will have failure modes.
Failure is unavoidable and should be planned for. What is avoidable are security problems
related to failure. The problem is that when many systems fail in any way, they exhibit
insecure behavior."
4) Grant least privilege -- When you do have to grant permission for a user or a process
to do something, grant as little permission as possible. Think about your Outlook
contacts. If you need someone to have access to your contacts to see some data, grant
them reader permission, but do not grant them edit permission. Or if you want a geekier
example, try this: most users of a system should not need root permission for their
everyday work, so dont give it to them. Bottom line, avoid unintentional, unwanted, or
improper uses of privilege by doling it out in a miserly fashion.
5) Separate privileges -- I once saw a system that divided its authentication front end
into an impressive number of roles with different degrees of access to the system. The
problem was that when a user of any role had to perform a back-end database action, the
software granted each user de-facto administrator privilege temporarily. Not good. Even
the lowliest intern could blitzkrieg the database.
Know that if an attacker is able to finagle one privilege but not a second, she may not be
able to launch a successful attack. Keep privilege sets apart.
6) Economize mechanism -- Complexity is the enemy of security engineering and the
friend of the attacker. Its just too easy to screw things up in a complicated system, both
from a design perspective and from an implementation perspective. The irony: Want to
see something complicated? Check out just about any piece of modern enterprise
software!
Do what you can to keep things simple. From Building Secure Software, "The KISS mantra
is pervasive: 'Keep It Simple, Stupid!' This motto applies just as well to security as it does
everywhere else. Complexity increases the risk of problems. Avoid complexity and avoid
problems."
7) Do not share mechanisms -- Should you plunk your inward-facing business
application on the public cloud? Probably not, according to this principle. Why have your
authentication system deal with random Internet traffic when you can limit it to
employees who you (supposedly) trust?
Heres a geekier example. If you have multiple users using the same components, have
your system create different instances for each user. By not sharing objects and access
mechanisms between users, you will lessen the possibility of security failure.
8) Be reluctant to trust -- Assume that the environment where your system operates is
hostile. Dont let just anyone call your API, and certainly dont let just anyone gain access
to your secrets! If you rely on a cloud component, put in some checks to make sure that it
has not been spoofed or otherwise compromised. Anticipate attacks such as command-
injection, cross-site scripting, and so on.
This principle can get tricky fast. From Building Secure Software, "One final point to
remember is that trust is transitive. Once you dole out some trust, you often implicitly
extend it to anyone the trusted entity may trust."
9) Assume your secrets are not safe -- Security is not obscurity, especially when it
comes to secrets stored in your code. Assume that an attacker will find out about as much
about your system as a power user, maybe more. The attackers toolkit includes
decompilers, disassemblers, and any number of analysis tools. Expect them to be aimed at
your system. Ever look for a crypto key in binary code? An entropy sweep can make it
stick out like a sore thumb. Binary is just a language.
10) Mediate completely -- Every access and every object should be checked, every
time. Make sure your access control system is thorough and designed to work in the
multi-threaded world we all inhabit today. Whatever you do, make sure that if
permissions change on the fly in your system, that access is systematically
rechecked. Dont cache results that grant authority or wield authority. In a world where
massively distributed systems are pervasive and machines with multiple processors are
the norm, this principle is a doozy to think about.
11) Make security usable -- If your security mechanisms are too odious, your users will
go to great length to circumvent or avoid them. Make sure that your security system is as
secure as it needs to be, but no more. If you affect usability too deeply, nobody will use
your stuff, no matter how secure it is. Then it will be very secure, and very near useless.
Spaf has always laughed at the line that mentions how the most secure system in the
world is one with its hard drive demagnetized that is buried in a 30 foot hole filled with
concrete poured around a Faraday grid. Such a system is, ahem, difficult to use.
12) Promote privacy -- Yeah, I know, everybody talks about privacy, but most people
dont actually do anything about it. You can help fix that. When you design a system, think
about the privacy of its ultimate users. Are you collecting personally identifiable
information (PII) just because somebody from the marketing team said to do so? Is it a
good thing to do? Do you store PII in a place where it can be compromised? Shouldnt that
be encrypted? Information security practitioners don't always have to provide the
answers to these privacy questions (that's what CIOs get paid for), but it's important for
infosec to put forth these kinds of questions if no one else does.
13) Use your resources -- If youre not sure whether your system design is secure, ask
for help. Architectural risk analysis is hard, but there are people who have been doing it
well for decades. Dont try to go it alone if you cant. And dont feel bad about asking for
help; this stuff is tricky.

2.6 PROTECTION MECHANISM
In computer science, protection mechanisms are built into a computer architecture to
support the enforcement of security policies. A simple definition of a security policy is "to
set who may use what information in a computer system". The access matrix model, first
introduced in 1971, is a generalized description of operating system protection
mechanisms. The separation of protection and security is a special case of the separation
of mechanism and policy.
Class java.security.ProtectionDomain
java.lang.Object
java.security.ProtectionDomain
public class ProtectionDomain
extends Object
This ProtectionDomain class encapulates the characteristics of a domain, which encloses
a set of classes whose instances are granted the same set of permissions.
In addition to a set of permissions, a domain is comprised of a CodeSource, which is a set
of PublicKeys together with a codebase (in the form of a URL). Thus, classes signed by the
same keys and from the same URL are placed in the same domain. Classes that have the
same permissions but are from different code sources belong to different domains.
A class belongs to one and only one Protection Domain.
Constructor Index
Constructor Description
ProtectionDomain(CodeSource,PermissionCollection) Creates a new
Constructor Index
Constructor Description
ProtectionDomain
with the given
CodeSource and
Permissions.
Method Index
Method Description
CodeSource getCodeSource()
Returns the CodeSource of this
domain.
PermissionCollection getPermissions()
Returns the permissions of this
domain.
boolean implies(Permission)
Check and see if this
ProtectionDomain implies the
permissions expressed in the
Permission object.
String toString()
Convert a ProtectionDomain to a
String.
Constructors
ProtectionDomain
public Protection Domain(CodeSource codesource,
PermissionCollection permissions)
Creates a new ProtectionDomain with the given CodeSource and Permissions. If the
permissions object is not null, then setReadOnly() will be called on the passed in
Permissions object.

Parameters
o codesource - the codesource associated with this domain
o permissions - the permissions granted to this domain
Methods
getCodeSource
public final CodeSource getCodeSource()
Returns the CodeSource of this domain.
Returns
o the CodeSource of this domain.
getPermissions
public final PermissionCollection getPermissions()
Returns the permissions of this domain.
Returns
o the permissions of this domain.
implies
public boolean implies(Permission permission)
Check and see if this ProtectionDomain implies the permissions expressed in the
Permission object.
Parameters
o permission - the Permission object to check.
Returns
o true if "permission" is a proper subset of a permission in this Protection Domain,
false if not.
toString
public String toString()
Convert a Protection Domain to a String.

2.7 ACCESS CONTROL LIST
An access control list (ACL), with respect to a computer file system, is a list of permissions attached to
an object. An ACL specifies which users or system processes are granted access to objects, as well as what
operations are allowed on given objects. Each entry in a typical ACL specifies a subject and an operation. For
instance, if a file has an ACL that contains (Alice, delete), this would give Alice permission to delete
the file.
ACL-based security models
When a subject requests an operation on an object in an ACL-based security model, the operating system first
checks the ACL for an applicable entry to decide whether the requested operation is authorized. A key issue
in the definition of any ACL-based security model is determining how access control lists are edited, namely
which users and processes are granted ACL-modification access.
Implementations
Many kinds of systems implement ACL, or have a historical implementation.
Filesystem ACLs
In the 1990s the ACL and RBAC models were extensively tested and used to administrate file permissions.
A filesystem ACL is a data structure (usually a table) containing entries that specify individual user or group
rights to specific system objects such as programs, processes, or files. These entries are known as access
control entries (ACEs) in the Microsoft Windows NT, OpenVMS, Unix-like, and Mac OS X operating systems.
Each accessible object contains an identifier to its ACL. The privileges or permissions determine specific
access rights, such as whether a user can read from, write to, or execute an object. In some implementations,
an ACE can control whether or not a user, or group of users, may alter the ACL on an object.
Most of the Unix and Unix-like operating systems (e.g. Linux, BSD, or Solaris) support POSIX.1e ACLs, based
on an early POSIX draft that was abandoned. Many of them, for example AIX,FreeBSD, Mac OS X beginning
with version 10.4 ("Tiger"), or Solaris with ZFS filesystem, support NFSv4 ACLs, which are part of the NFSv4
standard. There are two experimental implementations of NFSv4 ACLs for Linux: NFSv4 ACLs support
for Ext3 filesystem and recent Richacls, which brings NFSv4 ACLs support for Ext4 filesystem.
Networking ACLs
On some types of proprietary computer hardware (in particular routers and switches), an Access Control List
refers to rules that are applied to port numbers or IP Addresses that are available on a host or other layer 3,
each with a list of hosts and/or networks permitted to use the service. Although it is additionally possible to
configure Access Control Lists based on network domain names, this is generally a questionable idea
because individual TCP, UDP, and ICMP packets do not contain domain names. Consequently, the device
enforcing the Access Control List must separately resolve names to numeric addresses. This presents an
additional attack surface for an attacker who is seeking to compromise security of the system which the
Access Control List is protecting. Both individual servers as well as routers can have network ACLs. Access
control lists can generally be configured to control both inbound and outbound traffic, and in this context they
are similar to firewalls. Like Firewalls, ACLs are subject to security regulations and standards such as PCI
DSS.
SQL implementations
ACL algorithms have been ported to SQL and relational database systems. Many "modern" (2000's and
2010's) SQL based systems, like Enterprise resource planning and Content managementsystems, have used
ACL model in their administration modules.
Comparing with RBAC
The main alternative to the ACL model is the RBAC model. A "minimal RBAC Model", RBACm, can be
compared with an ACL mechanism, ACLg, where only groups are permitted as entries in the ACL. Barkley
(1997)
[8]
showed that RBACm and ACLg are equivalent.
In modern SQL implementations, ACL also manage groups and inheritance in a hierarchy of groups. So
"modern ACLs" can express all that RBAC express, and are notably powerful (compared to "old ACLs") in
their ability to express access control policy in terms of the way in which administrators view organizations.
For data interchange, and for "high level comparisons", ACL data can be translated to XACML.
[9]


















MODULE THREE
INPUT/OUTPUT
In computing, input/output or I/O (or informally, io or IO) is the communication
between an information processing system (such as a computer) and the outside world,
possibly a human or another information processing system. Inputs are the signals or data
received by the system, and outputs are the signals or data sent from it. The term can also
be used as part of an action; to "perform I/O" is to perform an input or output operation.
I/O devices are used by a person (or other system) to communicate with a computer. For
instance, a keyboard or a mouse may be an input device for a computer,
while monitors and printers are considered output devices for a computer. Devices for
communication between computers, such as modems and network cards, typically serve
for both input and output.
Note that the designation of a device as either input or output depends on the perspective.
Mouse and keyboards take as input physical movement that the human user outputs and
convert it into signals that a computer can understand. The output from these devices is
input for the computer. Similarly, printers and monitors take as input signals that a
computer outputs. They then convert these signals into representations that human users
can see or read. For a human user the process of reading or seeing these representations is
receiving input. These interactions between computers and humans is studied in a field
called humancomputer interaction.
In computer architecture, the combination of the CPU and main memory (i.e. memory that
the CPU can read and write to directly, with individual instructions) is considered the
brain of a computer, and from that point of view any transfer of information from or to
that combination, for example to or from a disk drive, is considered I/O. The CPU and its
supporting circuitry providememory-mapped I/O that is used in low-level computer
programming, such as the implementation of device drivers. An I/O algorithm is one
designed to exploit locality and perform efficiently when data reside on secondary storage,
such as a disk drive.
Interface
An I/O interface is required whenever the I/O device is driven by the processor. The
interface must have necessary logic to interpret the device address generated by the
processor. Handshaking should be implemented by the interface using appropriate
commands (like BUSY, READY, and WAIT), and the processor can communicate with an
I/O device through the interface. If different data formats are being exchanged, the
interface must be able to convert serial data to parallel form and vice-versa. There must be
provision for generating interrupts and the corresponding type numbers for further
processing by the processor if required.
A computer that uses memory-mapped I/O accesses hardware by reading and writing to
specific memory locations, using the same assembly language instructions that computer
would normally use to access memory.
Higher-level implementation
Higher-level operating system and programming facilities employ separate, more abstract
I/O concepts and primitives. For example, most operating systems provide application
programs with the concept of files. The C and C++ programming languages, and operating
systems in the Unix family, traditionally abstract files and devices as streams, which can be
read or written, or sometimes both. The C standard library provides functions for
manipulating streams for input and output.
In the context of the ALGOL 68 programming language, the input and output facilities are
collectively referred to as transput. The ALGOL 68 transput library recognizes the
following standard files/devices: stand in, stand out, stand errors and stand back.
An alternative to special primitive functions is the I/O monad, which permits programs to
just describe I/O, and the actions are carried out outside the program. This is notable
because the I/Ofunctions would introduce side-effects to any programming language, but
this allows purely functional programming to be practical.
Channel I/O
Channel I/O requires the use of instructions that are specifically designed to perform I/O
operations. The I/O instructions address the channel or the channel and device; the
channel asynchronously accesses all other required addressing and control information.
This is similar to DMA, but more flexible.



Port-mapped I/O
Port-mapped I/O also requires the use of special I/O instructions. Typically one or more
ports are assigned to the device, each with a special purpose. The port numbers are in a
separate address space from that used by normal instructions.

3.2 Device driver
In computing, a device driver is a computer program that operates or controls a
particular type of device that is attached to a computer.
[1]
A driver typically communicates
with the device through the computer bus or communications subsystem to which the
hardware connects. When a calling program invokes a routine in the driver, the driver
issues commands to the device. Once the device sends data back to the driver, the driver
may invoke routines in the original calling program. Drivers are hardware-dependent
and operating-system-specific. They usually provide the interrupt handling required for
any necessary asynchronous time-dependent hardware interface.
3.3 Kernel mode vs. user mode
Device drivers, particularly on modern Microsoft Windows platforms, can run in kernel-
mode (Ring 0 on x86 CPUs) or in user-mode (Ring 3 on x86 CPUs). The primary benefit of
running a driver in user mode is improved stability, since a poorly written user mode
device driver cannot crash the system by overwriting kernel memory. On the other hand,
user/kernel-mode transitions usually impose a considerable performance overhead,
thereby prohibiting user-mode drivers for low latency and high throughput requirements.
Kernel space can be accessed by user module only through the use of system calls. End
user programs like the UNIX shell or other GUI-based applications are part of the user
space. These applications interact with hardware through kernel supported functions.

3.4 APPLICATIONS
Because of the diversity of modern hardware and operating systems, drivers operate in
many different environments. Drivers may interface with:
printers
video adapters
Network cards
Sound cards
Local buses of various sortsin particular, for bus mastering on modern systems
Low-bandwidth I/O buses of various sorts (for pointing devices such
as mice, keyboards, USB, etc.)
Computer storage devices such as hard disk, CD-ROM, and floppy disk buses
(ATA, SATA, SCSI)
Implementing support for different file systems
Image scanners
Digital cameras
Common levels of abstraction for device drivers include:
For hardware:
Interfacing directly
Writing to or reading from a device control register
Using some higher-level interface (e.g. Video BIOS)
Using another lower-level device driver (e.g. file system drivers using disk drivers)
Simulating work with hardware, while doing something entirely different
For software:
Allowing the operating system direct access to hardware resources
Implementing only primitives
Implementing an interface for non-driver software (e.g., TWAIN)
Implementing a language, sometimes quite high-level (e.g., PostScript)
So choosing and installing the correct device drivers for given hardware is often a key
component of computer system configuration.

3.5 VIRTUAL DEVICE DRIVERS
Virtual device drivers represent a particular variant of device drivers. They are used to
emulate a hardware device, particularly in virtualization environments, for example when
a DOS program is run on a Microsoft Windows computer or when a guest operating
system is run on, for example, a Xen host. Instead of enabling the guest operating system
to dialog with hardware, virtual device drivers take the opposite role and emulate a piece
of hardware, so that the guest operating system and its drivers running inside a virtual
machine can have the illusion of accessing real hardware. Attempts by the guest operating
system to access the hardware are routed to the virtual device driver in the host operating
system as e.g., function calls. The virtual device driver can also send simulated processor-
level events like interrupts into the virtual machine.
Virtual devices may also operate in a non-virtualized environment. For example a
virtual network adapter is used with a virtual private network, while a virtual disk device
is used with iSCSI. A good example for virtual device drivers can be "Daemon Tools".
There are several variants of virtual device drivers..