High Memory in The Linux Kernel

Feature: High Memory In The Linux Kernel | KernelTrap http://kerneltrap.
org/node/2450
Search this site:
Forums News Blogs Features Site Login / Register
Home
Author
Feature: High Memory In The Linux Kernel Information
Submitted by Amit Shah on February 21, 2004 - 6:02am
high memory Linux memory memory management PAE RAM Amit Shah
Linux feature article
Offline
As RAM increasingly becomes a commodity, the prices drop and
computer users are able to buy more. 32-bit archictectures face certain Joined: Feb 28 2003
limitations in regards to accessing these growing amounts of RAM. To
better understand the problem and the various solutions, we begin with
an overview of Linux memory management. Understanding how basic
Navigation
memory management works, we are better able to define the problem, Create content
and finally to review the various solutions. Mailing list archives
Recent posts
This article was written by examining the Linux 2.6 kernel source code
for the x86 architecture types.
Overview of Linux memory management

32-bit architectures can reference 4 GB of physical memory (2^32).
Processors that have an MMU (Memory Management Unit) support the
concept of virtual memory: page tables are set up by the kernel which
map "virtual addresses" to "physical addresses"; this basically means
that each process can access 4 GB of memory, thinking it's the only
process running on the machine (much like multi-tasking, in which each
process is made to think that it's the only process executing on a CPU).
The virtual address to physical address mappings are done by the

kernel. When a new process is "fork()"ed, the kernel creates a new set Colocation donated by:
of page tables for the process. The addresses referenced within a

process in user-space are virtual addresses. They do not necessarily
map directly to the same physical address. The virtual address is
passed to the MMU (Memory Management Unit of the processor) Syndicate
which converts it to the proper physical address based on the tables
set up by the kernel. Hence, two processes can refer to memory
address 0x08329, but they would refer to two different locations in
memory.
The Linux kernel splits the 4 GB virtual address space of a process in

two parts: 3 GB and 1 GB. The lower 3 GB of the process virtual
address space is accessible as the user-space virtual addresses and
1 of 20 2/27/2011 9:23 PM
Feature: High Memory In The Linux Kernel | KernelTrap http://kerneltrap.org/node/2450
the upper 1 GB space is reserved for the kernel virtual addresses. This
is true for all processes. © 2001-2009 KernelTrap. |
Powered by Drupal. | Legal
statement.
+----------+ 4 GB
| |
| |
| |
| Kernel |
| | +----------+
| Virtual | | |
| | | |
| Space | | High |
| | | |
| (1 GB) | | Memory |
| | | |
| | | (unused) |
+----------+ 3 GB +----------+ 1 GB
| | | |
| | | |
| | | |
| | | Kernel |
| | | |
| | | Physical |
| | | |
|User-space| | Space |
| | | |
| Virtual | | |
| | | |
| Space | | |
| | | |
| (3 GB) | +----------+ 0 GB
| |
| | Physical
| | Memory
| |
| |
| |
| |
| |
+----------+ 0 GB
Virtual
Memory
The kernel virtual area (3 - 4 GB address space) maps to the first 1

GB of physical RAM. The 3 GB addressable RAM available to each
process is mapped to the available physical RAM.
The Problem
So, the basic problem here is, the kernel can just address 1 GB of
virtual addresses, which can translate to a maximum of 1 GB of
physical memory. This is because the kernel directly maps all available
kernel virtual space addresses to the available physical memory.
2 of 20 2/27/2011 9:23 PM
Solutions
There are some solutions which address this problem:
1. 2G / 2G, 1G / 3G split
2. HIGHMEM solution for using up to 4 GB of memory
1. 2G / 2G, 1G / 3G split
Instead of splitting the virtual address space the traditional way of 3G /

1G (3 GB for user-space, 1 GB for kernel space), third-party patches
exist to split the virtual address space 2G / 2G or 1G / 3G. The 1G /
3G split is a bit extreme in that you can map up to 3 GB of physical
memory, but user-space applications cannot grow beyond 1 GB. It
could work for simple applications; but if one has more than 3 GB of
physical RAM, he / she won't run simple applications on it, right?
The 2G / 2G split seems to be a balanced approach to using RAM

more than 1 GB without using the HIGHMEM patches. However, server
applications like databases always want as much virtual addressing
space as possible; so this approach may not work in those scenarios.
There's a patch for 2.4.23 that includes a config-time option of

selecting the user / kernel split values by Andrea Arcangeli. It is
available at his kernel page. It's a simple patch and making it work on
2.6 should not be too difficult.
Before looking at solutions 2 & 3, let's take a look at some more Linux
Memory Management issues.
Zones
In Linux, the memory available from all banks is classified into "nodes".
These nodes indicate how much memory each bank has. This
classification is mainly useful for NUMA architectures, but it's also used
for UMA architectures, where the number of nodes is just 1.
Memory in each node is divided into "zones". The zones currently

defined are ZONE_DMA, ZONE_NORMAL and ZONE_HIGHMEM.
ZONE_DMA is used by some devices for data transfer and is mapped

in the lower physical memory range (up to 16 MB).
Memory in the ZONE_NORMAL region is mapped by the kernel in the

upper region of the linear address space. Most operations can only
take place in ZONE_NORMAL; so this is the most performance critical
zone. ZONE_NORMAL goes from 16 MB to 896 MB.
To address memory from 1 GB onwards, the kernel has to map pages

from high memory into ZONE_NORMAL.
Some area of memory is reserved for storing several kernel data

structures that store information about the memory map and page
tables. This on x86 is 128 MB. Hence, of the 1 GB physical memory
3 of 20 2/27/2011 9:23 PM
the kernel can access, 128MB is reserved. This means that the kernel
virtual address in this 128 MB is not mapped to physical memory. This
leaves a maximum of 896 MB for ZONE_NORMAL. So, even if one has
1 GB of physical RAM, just 896 MB will be actually available.
Back to the solutions:
Since Linux can't access memory which hasn't been directly mapped
into its address space, to use memory > 1 GB, the physical pages
have to be mapped in the kernel virtual address space first. This means
that the pages in ZONE_HIGHMEM have to be mapped in
ZONE_NORMAL before they can be accessed.
The reserved space which we talked about earlier (in case of x86, 128
MB) has an area in which pages from high memory are mapped into
the kernel address space.
To create a permanent mapping, the "kmap" function is used. Since this

function may sleep, it may not be used in interrupt context. Since the
number of permanent mappings is limited (if not, we could've directly
mapped all the high memory in the address space), pages mapped this
way should be "kunmap"ped when no longer needed.
Temporary mappings can be created via "kmap_atomic". This function

doesn't block, so it can be used in interrupt context. "kunmap_atomic"
un-maps the mapped high memory page. A temporary mapping is only
available as long as the next temporary mapping. However, since the
mapping and un-mapping functions also disable / enable preemption,
it's a bug to not kunmap_atomic a page mapped via kmap_atomic.
3. HIGHMEM solution for using 64 GB of memory
This is enabled via the PAE (Physical Address Extension) extension of

the PentiumPro processors. PAE addresses the 4 GB physical memory
limitation and is seen as Intel's answer to AMD 64-bit and AMD x86-64.
PAE allows processors to access physical memory up to 64 GB (36
bits of address bus). However, since the virtual address space is just
32 bits wide, each process can't grow beyond 4 GB. The mechanism
used to access memory from 4 GB to 64 GB is essentially the same as
that of accessing the 1 GB - 4 GB RAM via the HIGHMEM solution
discussed above.
Should I enable CONFIG_HIGHMEM for my 1 GB RAM system?
It is advised to not enable CONFIG_HIGHMEM in the kernel to utilize

the extra 128 MB you get for your 1 GB RAM system. I/O Devices
cannot directly address high memory from PCI space, so bounce
buffers have to be used. Plus the virtual memory management and
paging costs come with extra mappings. For details on bounce buffers,
refer to Mel Gorman's documentation (link below).
For more information, see
4 of 20 2/27/2011 9:23 PM
Andrea Arcangeli's article on the original HIGHMEM patches in

Linux 2.3
Mel Gorman's VM Documentation for the 2.4 Linux Memory
Management subsystem
Linux kernel sources
Copyright (C) 2004, Amit Shah <amitshah@gmx.net>.

Add new comment
Excellent article
Anonymous on February 21, 2004 - 9:44am
It's great to see stuff like this on Kerneltrap. More please!
reply
Re: Excellent article

laudney on February 21, 2004 - 11:12am
The article is well-written and informative. For more details,
"understanding linux kernel 2.4 second edition" is definitely a
must.
-- laudney
reply
I wouldn't say well-written (

Anonymous (not verified) on April 28, 2006 - 6:23pm
I wouldn't say well-written (see other comments too)
Two quite serious things:
The physical memory bar (on the right) suggests that
1GB of
kernel-mapped virtual memory covers *first* 1GB of
physical
memory. This is not true. 1GB total, yes, but
that doesn't have to be the first 1GB of physical
memory.
This 1GB can be spread among physical pages located
*anywhere*
in the physical memory.
Similarly, sentence: "The kernel virtual area (3 - 4 GB
address space) maps to the first 1 GB of physical
RAM."
is just false.
Cheers!
reply
5 of 20 2/27/2011 9:23 PM
after a second thought.. plea

Anonymous (not verified) on April 28, 2006 -
6:40pm
after a second thought.. please disregard this
comment :)
reply
critical
I enjoy this type of critical review of the linux kernel and how
it most evolve.
reply
Clerify the doubt

Navi (not verified) on December 16, 2006 - 5:43am
What is the virtual memory ?
What is physical memory ?
If we write a code,when is the virtual adddress is created &
physical address is created?
What is memory mapping & I/O mapping & what is the use
of this ?
reply
PAE vs AMD64
"This is enabled via the PAE (Physical Address Extension)
extension of the PentiumPro processors. PAE addresses the 4
GB physical memory limitation and is seen as Intel's answer to
AMD 64-bit and AMD x86-64."
How can you claim PAE is an answer to AMD64 if it dates back
to PentiumPro ? AMD64 didn't even exist at that time.
reply
Re: PAE vs AMD64

This sentence in the article is totally incorrect and
misleading. PAE enables 32-bit OS to address >4G
physical memory by introducing a multi-level page table
scheme and enlarging page size from 4K to 4M. This has
nothing to do with AMD64.
But as a whole, the article is informative.
-- laudney
6 of 20 2/27/2011 9:23 PM
reply
PAE & Page Sizes

Anonymous on February 23, 2004 - 7:30pm
Actually, that's bullshit. If you want to use large pages
with PAE, you'll be using either 4k or 2M pages *OR*
*ONLY* 4M pages.
reply
Does the PSE (Page Size Exten

shesh moorti on February 23, 2004 - 9:13pm
Does the PSE (Page Size Extension) work only with
PAE ?
I thought 4M pagesize option would work without PAE
also.
#shesh
reply
Re:Does the PSE (Page Size

Exten
I think Page Size Extension is not dependent on
Page Table Extenesion.
On intel architecture Since PENTIUM(i586) has
PSE but not PTE. PTE was added on i686 i.e
PENTIUM PRO and P II/P III/ P4 etc
Regards,
Ameet
reply
I think the author meant "Pen

I think the author meant "Pentium Xeon" not Pentium Pro.
reply
not true
He's right, Pentium Pro's have a PAE register.
reply
7 of 20 2/27/2011 9:23 PM
Re: PAE vs AMD64

Anonymous on August 4, 2004 - 8:38pm
It is often SEEN as Intel's answer. (appearance that it is the
answer differs from reality that it's been around for a while)
reply
I have trouble understanding...

You say : "The kernel virtual area (3 - 4 GB address space)
maps to the first 1 GB of physical RAM. The 3 GB addressable
RAM available to each process is mapped to the available
physical RAM."
It seems to me that physical addresses are completely
sequential on PCs (there are no holes in the physical address
space). Thus how can the kernel map the userspace virtual
memory to physical RAM above 1 GB even if the machine has
less than 1 GB RAM ?
I must admit I have trouble understanding your explanations...
reply
Re: I have trouble understanding...

I think that the article is intended for computers with a lot of
memory (2GB - 4GB). AFAIK, the same idea is used for
computers with less memory, it's just that the numbers
scale down.
The 4GB virtual address space remains the same, however.
reply

This article is not clear in this part. The crucial point is:
physical memery is mapped != physical memory is used. It
means necessary 'page tables' have been set up so that the
physical memory is "addressable" from virtual memory
space, either process or kernel. The kernel uses its 3-4G
address space to map the physical memory from the very
start. This doesn't prevent processes from mapping certain
physical memory pages into their 1-3G address space, by
"malloc" or "brk" system calls.
This article should have mentioned that process can run in
"user mode" or "kernel mode". When in user mode, the
process can only access 1-3G address space, while in
"kernel mode", i.e. kernel runs in the process context and
8 of 20 2/27/2011 9:23 PM
executes on behalf of the process, all 4G address space

can be accessed. Besides, physical memory allocated to
processes can all be accessed by the kernel, while kernel
private data strutures stored in certain physical memory
pages is invisible to processes, otherwise any process can
mess up the kernel. That's why kernel needs to map all
physical memory into its address space, either directly (no
high mem) or selectively (high mem).
-- Laudney
reply
I agree to this comment.

Anonymous on October 31, 2004 - 8:00pm
This is much clear
reply
I'm not sure I understand you

I'm not sure I understand your concern.
The mapping of virtual addresses to physical addresses is
pretty arbitrary (with a granularity of 4k on x86). On a
machine with less than 1GB RAM, userspace virtual
memory with addresses above 1 GB are simply mapped to
physical addresses < 1 GB if they are mapped to RAM at
all.
It's not actually 3 GB of addressable RAM that is given to
each process (which is where you may be getting
confused). Rather, each process has 3 GB of address
space to work with. Some of the data in that address space
will probably be stored in physical RAM. Some of the data
in that address space will probably be stored on disk and
not in physical RAM. And what portion of physical RAM
each 4k chunk of virtual memory corresponds to (if any) is
pretty arbitrary in that 3 GB for processes.
I feel it is a mistake to consider the 3GB address spaces
given to processes as RAM. The 3GB address spaces are
abstractions that differ significantly from RAM, and from the
physical address space abstraction. The 896 MB portion of
the kernel address space can reasonably be considered
RAM (or rather whatever portion is mapped to RAM for
systems under 1 GB). This, I suspect, is very helpful. The
virtual and physical addresses differ, but the mapping
between the two is simple. But for systems > 1GB RAM,
there is not enough kernel virtual address to map to all the
RAM, and so some of the RAM needs to be treated
specially.
reply
9 of 20 2/27/2011 9:23 PM
thanks for clearing it up

"But for systems > 1GB RAM, there is not enough
kernel virtual address to map to all the RAM, and so
some of the RAM needs to be treated specially."
Ok, so basically the kernel is reinventing segmented
addressing in that case ?
reply
More like the kernel is reinv

More like the kernel is reinventing expanded
memory. Fortunately it's only for the kernel; user
processes don't have to worry about such hacks
until the 3MB mark. And unless I'm much mistaken,
on 64 bit systems most of these problems go away
(there may still be limitations when it comes to i/o).
reply

Amit Shah on February 23, 2004 - 4:21am
You're right in saying that physical addresses are
sequential. On an offtopic note, there are patches which
allow non-sequential physical memory regions: badram and
NUMA.
I've written: "The 3 GB addressable RAM available to each
process is mapped to the available physical RAM."
This means that the userspace application can grow to a
maximum of 3 GB. It is not necessary to have 3 GB of
physical memory to map it all. The kernel "swaps out" some
of the physical pages to make room for processes that
might need more RAM. If a process accesses a page that
has been swapped out, it is brought back in RAM (possibly
after swapping out some other page).
This is a wonderful concept, take a look at some OS theory
or i386 manuals for the complete description.
--
Amit Shah
http://amitshah.nav.to/
reply
That's not what I was objecti

That's not what I was objecting to. You said : "The
10 of 20 2/27/2011 9:23 PM
kernel virtual area (3 - 4 GB address space) maps to

the first 1 GB of physical RAM." In fact this is
misleading, since it gives the feeling that the whole first
GB of physical RAM is reserved for kernel use, which it
is certainly not (otherwise a machine with 1.5 GB RAM
would only give 512 MB max physical RAM to
userspace processes...). The ascii drawing is
misleading too.
reply
Re: I have trouble

understanding...
Amit Shah on February 23, 2004 - 11:49pm
it gives the feeling that the whole first GB of
physical RAM is reserved for kernel use
In fact, it is. The kernel has to have control over the

whole memory. It's the kernel's job to
allocate/deallocate RAM to processes.
What the kernel does is, maps the entire available
memory in its own space, so that it can access any
memory region. It then gives out free pages to
processes which need them.
The userspace processes cannot of its own
allocate a page to itself. It has to request the
kernel to give it some memory area. Once the
kernel has mapped a physical page in the
process's space, it can use that extra memory.
reply
"What the kernel does is,

map
"What the kernel does is, maps the entire
available memory in its own space, so that it
can access any memory region. It then gives
out free pages to processes which need
them."
Thanks, it's a lot clearer now! You should add
these precisions to your article.
reply
Re: I have trouble

understanding...
Anonymous on May 3, 2004 - 6:20am
This has always bugged me, so I appreciate
11 of 20 2/27/2011 9:23 PM
your explaination.
I don't understand why the kernel has to map
all available memory
into its its own space. It seems to me that it
should only
have to map the page tables corresponding to
the physical memory into its own space. As
long as it has data structures that track all
pages of physical memory why does it need to
access them. If the physical pages are
mapped into process space, then when the
process makes a system call, the kernel
should have access to copy any memory to
kernel space because it's in the context of the
calling process and can use its page tables.
Also, why have a 3:1 split when you can't map
3 GB of physical memory
anyway. I now you have 3 GB of virtual
address space, but what
good is it if you can't access physical memory.
I guess you could
have 896 MB of physcical memory and 2 GB
of swap, but I don't see the
rational behind this.
Thanks
Kevin
reply
Swap
I've got 2 question :
- Is the swap accounted as available RAM ? For instance, is
2Gb RAM + 4Gb Swap over the 4Gb limit ?
- The HIGHMEM I/O support is not explained (recent in 2.4) :
does the bouncing the article talk about remain true if this option
is set ?
reply
Re: Swap
1) Swap is not accounted as available RAM for this. There
is no limit on the sum of swap + RAM, only a limit on swap's
size (man mkswap says it is 2Gb). And with 2Gb of RAM
you must use HIGHEM support since it is more than 1 Gb of
RAM.
2) No idea about this, but the Documentation/Configure.help
seems to say you are right. But it seems that drivers must
do some work themselves to use it. Actually, for what I
could check the scsi driver support it: grep through the
kernel for "blk_nohighio". Only drivers/scsi/ contains
12 of 20 2/27/2011 9:23 PM
references to it and claims in comment to support it.

However, at the very beginning of 2.5 the block API was
completely changed; the 2.6 drivers using the new API
(almost all) can more easily support Highmem, if I'm not
wrong. See in the 2.6 kernel sources the file
Documentation/block/biodoc.txt.
reply
The article is misleading

Yenya on February 22, 2004 - 9:41am
Should I enable CONFIG_HIGHMEM for my 1 GB RAM

system?
It is advised to not enable CONFIG_HIGHMEM in the kernel to
utilize the extra 128 MB you get for your 1 GB RAM system.
I/O Devices cannot directly address high memory from PCI
space, so bounce buffers have to be used. Plus the virtual
memory management and paging costs come with extra
mappings. For details on bounce buffers, refer to Mel
Gorman's documentation (link below).
This is totally misleading. When you have 1GB RAM, (i.e.

CONFIG_HIGHMEM_4G should be used), there is no need of
bounce buffers. At least I don't know about a PCI device which
cannot access the full 4GB (or 32bits) of address space. Yes,
some devices do not support dual-address cycle (i.e. 64-bit
addressing on 32-bit bus), but this is not a problem when you
have 1G of RAM.
The actual solution differs depending on your load: if you have
lots of small tasks - such as HTTP server, you can safely modify
the 1GB/3GB split to something like 1.2GB/2.8GB. If you have
just few big tasks which may need full 3GB of virtual memory
(maybe the Oracle database, or one big number-crunching
process), you can use CONFIG_HIGHMEM_4G, which gives
you extra 128MB of physical memory which is still usable for
some parts of kernel or user-space processes.
I would definitely not suggest to omit those 128MB using the
standard
kernel without CONFIG_HIGHMEM on a system with 1GB
RAM.
-Yenya
reply
re: The article is misleading

Actually, I suspect that you might be misinformed. Follow
some of the links at the end of the article. For example, a
direct quote from one of them:
"I don't know if anyone has actually measured
13 of 20 2/27/2011 9:23 PM
the relative performance, but I'd expect the

answer to be the same as 2.4. There is a
small but measurable performance penalty for
enabling highmem which is higher than the
benefit of the extra 128 meg of memory you
get when you have 1G. If you have more than
1G it's better to enable highmem."
This sure seems to imply that the article is correct, and that
there is overhead associated with enabling highmem to get
access to the extra 128 megs... More, that the overhead is
probably not worth the gain. This sentiment is frequently
echo'd on the lkml...
reply
It depends on the type of the

Yenya on February 22, 2004 - 10:23am
It depends on the type of the load. If you have
processes which fit into 800MB of RAM and you rarely
do disk access, you surely don't need
CONFIG_HIGHMEM. If your system is I/O bounded,
you can safely waste few
CPU cycles in bookkeeping of another memory zone.
But the article is wrong in the sentence saying the PCI
device cannot access highmem directly on a 1GB
system.
-Yenya
reply
Overhead
Con Kolivas on February 22, 2004 - 5:09pm
The overhead of enabling the extra 128Mb used to
be great in 2.4, especially before the later additions
to the vm by -aa (around 2.4.23), so it was very
true that performance would suffer. It had nothing
to do with what the devices could access; it was
the extra structures put in place to use high mem.
2.6, however, has taken away a lot of that
overhead so it isn't a big performance hit.
reply
HIGHMEM
so from what i read : DONT USE 1GB RAM
even if u have it ?!
ironicaly i have 3 such machines already....
now what ?
One of them condtatnly crashing, now
14 of 20 2/27/2011 9:23 PM
no-HIGMEM kernel is ready

for test.
reply
Clarify recommendation
Con Kolivas on February 24, 2004 -
5:52am
I would recommend not using high mem
support with 1Gb ram and sacrifice 128Mb
unless you are running 2.6. If you are
running 2.6 it is fine to enable high mem
with 1Gb ram.
reply
HIGHMEM
Anonymous on February 27, 2004 -
8:01am
now I'm sure HIGHMEM was the
reason for the crashes..
w/o it my uptime passed 3 days...
not totaly sure yes, will see on
monday if it is still up..
reply
1GB without high mem

Anonymous on March 12, 2004 -
5:28am
This is how to modify
userspace/kernelspace from 3G/1G to
a 2.75G/1.25G split.
For a 2.6.3 kernel:
In linux/arch/i386/kernel/vmlinux.lds.S
change 0xC0000000 to 0xB0000000
on line 13.
In linux/include/asm-i386/page.h
change 0xC0000000 to 0xB0000000
on lines 118 and 120.
With these changes my system
reports 1034MB total memory instead
of 905MB.
reply
can this be done on

2.4
15 of 20 2/27/2011 9:23 PM
Anonymous on April 24, 2004 -

9:20am
can this be done on 2.4 ?
reply
This article gives the impres

This article gives the impression that if you have >1GB and you
don't enable highmem option then any memory above 1GB will
be unused. And if you do enable it then all memory above 1GB
will be accessed thru some small widnows - what a nonsense. I
wonder if the author actually knows what he is talking about.
reply
No - i think the article is c

D Norman (not verified) on April 19, 2005 - 6:19pm
No - i think the article is clear. The author is suggesting that
the part of each process' virtual address space reserved for
the Linux Kernel is the upper 1GB, and this is mapped
(presumably for all processes) to the bottom 1GB of
physical memory.
Thus, if the kernel needs to access some memory above
1GB physical address then it needs to map that page into
part of the kernel virtual address space. This 'kmap' does
exactly that. As he says, clearly there is a limited ability to
map physical memory to this kernel virtual address space.
It makes implementing a 'physical memory file' device driver
very easy, I would think, provided you are only interested in
the bottom 1GB of the physical memory.
Windows, for your information, uses a 2GB/2GB split by
default with an option to switch to 3GB/1GB, and has the
kernel part of the address space mapped at the top, like
linux. However, the kernel address space isn't mapped to a
specific place in physical memory, but is divided into
different sections with different characteristics.
reply
Please review your comments b

Anonymous (not verified) on November 2, 2005 - 1:47am
Please review your comments before committing it. All the
points of view given by the article are 100 precent correct.
reply
I think that you really don't
16 of 20 2/27/2011 9:23 PM
understand memory on linux

Anonymous (not verified) on July 19, 2006 - 1:07pm
Simply try it. Compile a kernel
with himem off, install on a system with greater than 1G.
You only get 1G-128M of memmory.
Now, run some benchmarks.
Then, compile the same kernel with himem on.
You get the full memory size avalible.
Run the same tests with the new kernel.
The tests on the himem kernel will be slower than the
non-himem kernel.
reply
What about 4G/0G?

shopov on February 23, 2004 - 3:07am
What about if we allow a process to have the complete 4G of
memory privately mapped to it (well, not 4G exactly, we take
into account the page tables, IDT, GDT, TSS overhead)? Then
the kernel must be accessed via a task gate, not an interrupt
gate, and I guess that performing a full-blown task switch is
much expensive than taking a regular interrupt switch (this does
not flush caches, does not reload/store registers, etc.). How
much more expensive it really is, does anyone know of some
benchmarks? What if the process does few and infrequent
syscalls, yet has a thirst for large amounts of RAM? What do
you think...
reply
Ingo's 4G/4G patch does just that

performance impact depends on what userland does, it's
between 0-20% or so i recall, check lkml for more.
reply
appeal
I would like to appeal against the decision to mark my earlier
message as a flamebait post (see post with the subject "This
article gives the impression..."). I don't understand why it was
considered as such. In the first sentence I state that the
impression this article gives is in fact incorrect (looking at other
posts it appears that I am not the only one who thinks so). In
the second sentence I questioned the author's understanding of
the subject. In particular, I didn't do any of the following:
17 of 20 2/27/2011 9:23 PM
1. attack author on some personal grounds

2. attack article on some unrelated to the subject grounds (e.g.
language, spelling, etc).
I, therefore, urge you to reconsider your decision.
reply
Re: appeal
neveripe on February 23, 2004 - 12:41pm
You haven't stated motivation for your impression.
reply
Re: appeal
Ah, I see, I haven't cited the article. Well, to correct my
mistake I would like to provide the following quotations.
I especially like the last one.
"So, the basic problem here is, the kernel can just
address 1 GB of virtual addresses, which can translate
to a maximum of 1 GB of physical memory. This is
because the kernel directly maps all available kernel
virtual space addresses to the available physical
memory."
"To address memory from 1 GB onwards, the kernel
has to map pages from high memory into
ZONE_NORMAL."
"Since Linux can't access memory which hasn't been
directly mapped into its address space, to use memory
> 1 GB, the physical pages have to be mapped in the
kernel virtual address space first. This means that the
pages in ZONE_HIGHMEM have to be mapped in
ZONE_NORMAL before they can be accessed."
reply
Re: appeal
Amit Shah on February 23, 2004 - 9:56pm
So what's your point? I see no contradictions in
these statements and I _know_ for a fact that they
are true.
reply
Re: appeal
Your article gives an idea (or maybe it just me
18 of 20 2/27/2011 9:23 PM
so stupid?) that one can't use more than 1GB

of memory without any of the proposed
'solutions'. When you say something like
"So, the basic problem here is, the kernel can
just address 1 GB of virtual addresses, which
can translate to a maximum of 1 GB of
physical memory."
don't you think it is natural to interpret as 'OS
(and thus any code that it is running) cannot
access more than 1GB of memory'?
If even the kernel cannot address more than
1GB than why would anyone think that any
ordinary process can?
And it would be so easy, really, to just clarify
that kernel is limited to 1GB of memory for *it's
own internal data* (and maybe
give some examples of such data, like process
descriptors, thread contexts, etc).
reply
Re: appeal
Amit Shah on February 24, 2004 -
11:20pm
interpret as 'OS (and thus any code that it
is running) cannot access more than 1GB
of memory'?
Yes, it's true that Linux can't access more

than 1 GB of physical memory by default.
This is the exact problem that's lead to this
article.
It's not right that any code cannot access
more than 1 GB as a logical conclusion
from this fact. You're confusing between
virtual memory addressing and physical
addressing.
If even the kernel cannot address more
than 1GB than why would anyone think
that any ordinary process can?
This is possible because of the virtual

memory. I would suggest you to go
through some i386 documentation for more
information.
reply
desiring a ram disk

bob engels (not verified) on September 5,
2006 - 2:44pm
19 of 20 2/27/2011 9:23 PM
Does anyone have or is it possible to construct

a ramdisk bigger than 4 gig. I have 32 gigs of
memory on my machine and would like to take
full advantage of it.
thanks
reply
1 2 next › last »
Comment viewing options
Select your preferred way to display the comments and click "Save
settings" to activate your changes.
20 of 20 2/27/2011 9:23 PM

High Memory in The Linux Kernel

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

High Memory in The Linux Kernel

Uploaded by

Copyright:

Available Formats

Feature: High Memory In The Linux Kernel | KernelTrap http://kerneltrap.

Search this site:

Forums News Blogs Features Site Login / Register

Overview of Linux memory management

The virtual address to physical address mappings are done by the

of page tables for the process. The addresses referenced within a

The Linux kernel splits the 4 GB virtual address space of a process in

The kernel virtual area (3 - 4 GB address space) maps to the first 1

There are some solutions which address this problem:

Instead of splitting the virtual address space the traditional way of 3G /

The 2G / 2G split seems to be a balanced approach to using RAM

There's a patch for 2.4.23 that includes a config-time option of

Memory in each node is divided into "zones". The zones currently

ZONE_DMA is used by some devices for data transfer and is mapped

Memory in the ZONE_NORMAL region is mapped by the kernel in the

To address memory from 1 GB onwards, the kernel has to map pages

Some area of memory is reserved for storing several kernel data

Back to the solutions:

2. HIGHMEM solution for using up to 4 GB of memory

To create a permanent mapping, the "kmap" function is used. Since this

Temporary mappings can be created via "kmap_atomic". This function

3. HIGHMEM solution for using 64 GB of memory

This is enabled via the PAE (Physical Address Extension) extension of

Should I enable CONFIG_HIGHMEM for my 1 GB RAM system?

It is advised to not enable CONFIG_HIGHMEM in the kernel to utilize

For more information, see

Andrea Arcangeli's article on the original HIGHMEM patches in

Copyright (C) 2004, Amit Shah <amitshah@gmx.net>.

Re: Excellent article

I wouldn't say well-written (

after a second thought.. plea

Clerify the doubt

Re: PAE vs AMD64

PAE & Page Sizes

Does the PSE (Page Size Exten

Re:Does the PSE (Page Size

I think the author meant "Pen

Re: PAE vs AMD64

I have trouble understanding...

Re: I have trouble understanding...

Re: I have trouble understanding...

executes on behalf of the process, all 4G address space

I agree to this comment.

I'm not sure I understand you

thanks for clearing it up

More like the kernel is reinv

Re: I have trouble understanding...

That's not what I was objecti

kernel virtual area (3 - 4 GB address space) maps to

Re: I have trouble

In fact, it is. The kernel has to have control over the

"What the kernel does is,

Re: I have trouble

references to it and claims in comment to support it.

The article is misleading

Should I enable CONFIG_HIGHMEM for my 1 GB RAM

This is totally misleading. When you have 1GB RAM, (i.e.

re: The article is misleading

"I don't know if anyone has actually measured

the relative performance, but I'd expect the

It depends on the type of the

no-HIGMEM kernel is ready