rthyndrtmu,g

On all computer systems memory is a scarce resource. No matter how much memory is available,

it never seems to be enough. It doesn’t seem so long ago that 256MB of RAM was considered suffi-

cient, but now 2GB of RAM is commonplace as a sensible minimum requirement even for desktop

systems, with servers usefully having significantly more.

From the earliest versions of the operating system, UNIX-style operating systems have had a very

clean approach to managing memory, which Linux, because it implements the X/Open specifica-

tion, has inherited. Linux applications, except for a few specialized embedded applications, are

never permitted to access physical memory directly. It might appear so to the application, but

what the application is seeing is a carefully controlled illusion.

Linux provides applications with a clean view of a huge directly addressable memory space.

Additionally, it provides protection so that different applications are protected from each other,

and it allows applications to apparently access more memory than is physically present in the

machine, provided the machine is at least well configured and has sufficient swap space.

Simple Memory Allocation

You allocate memory using the  malloc call in the standard C library:

#include <stdlib.h>

void *malloc(size_t size);

Notice that Linux (following the X/Open specification) differs from some UNIX implementations by

not requiring a special  malloc.h include file. Note also that the  size parameter that specifies the

number of bytes to allocate isn’t a simple  int , although it’s usually an unsigned integer type.

You can allocate a great deal of memory on most Linux systems. Let’s start with a very simple program,

but one that would defeat old MS-DOS-based programs, because they cannot access memory outside the

base 640K memory map of PCs.

Try It Out Simple Memory Allocation

Type the following program,  memory1.c :

#include <unistd.h>

#include <stdlib.h>

#include <stdio.h>

#define A_MEGABYTE (1024 * 1024)

int main()

{

char *some_memory;

int megabyte = A_MEGABYTE;

int exit_code = EXIT_FAILURE;

some_memory = (char *)malloc(megabyte);

if (some_memory != NULL) {

sprintf(some_memory, “Hello World\n”);

printf(“%s”, some_memory);

exit_code = EXIT_SUCCESS;

}

exit(exit_code);

}

When you run this program, it gives the following output:

$ ./memory1

Hello World

How It Works

This program asks the  malloc library to give it a pointer to a megabyte of memory. You check to ensure

that  malloc was successful and then use some of the memory to show that it exists. When you run the

program, you should see  Hello World printed out, showing that  malloc did indeed return the megabyte

256

Chapter 7: Data Management

of usable memory. We don’t check that all of the megabyte is present; we have to put some trust in the

malloc code!

Notice that because  malloc returns a  void * pointer, you cast the result to the  char * that you need.

The  malloc function is guaranteed to return memory that is aligned so that it can be cast to a pointer of

any type.

The simple reason is that most current Linux systems use 32-bit integers and 32-bit pointers for pointing

to memory, which allows you to specify up to 4 gigabytes. This ability to address directly with a 32-bit

pointer, without needing segment registers or other tricks, is termed a flat 32-bit memory model. This

model is also used in 32-bit versions of Windows XP and Vista. You should never rely on integers being

32-bit however, as an ever-increasing number of 64-bit versions of Linux are in use.

Allocating Lots of Memory

Now that you’ve seen Linux exceed the limitations of the MS-DOS memory model, let’s give it a more

difficult problem. The next program asks to allocate somewhat more memory than is physically present

in the machine, so you might expect  malloc to start failing somewhere a little short of the actual amount

of memory present, because the kernel and all the other running processes are using some memory.

Try It Out Asking for All Physical Memory

With  memory2.c , we’re going to ask for more than the machine’s physical memory. You should adjust

the define  PHY_MEM_MEGS depending on your physical machine:

#include <unistd.h>

#include <stdlib.h>

#include <stdio.h>

#define A_MEGABYTE (1024 * 1024)

#define PHY_MEM_MEGS 1024 /* Adjust this number as required */

int main()

{

char *some_memory;

size_t size_to_allocate = A_MEGABYTE;

int megs_obtained = 0;

while (megs_obtained < (PHY_MEM_MEGS * 2)) {

some_memory = (char *)malloc(size_to_allocate);

if (some_memory != NULL) {

megs_obtained++;

sprintf(some_memory, “Hello World”);

printf(“%s - now allocated %d Megabytes\n”, some_memory, megs_obtained);

}

else {

exit(EXIT_FAILURE);

}

}

exit(EXIT_SUCCESS);

}

257

Chapter 7: Data Management

The output, somewhat abbreviated, is as follows:

$ ./memory2

Hello World - now allocated 1 Megabytes

Hello World - now allocated 2 Megabytes

...

Hello World - now allocated 2047 Megabytes

Hello World - now allocated 2048 Megabytes

How It Works

The program is very similar to the previous example. It simply loops, asking for more and more memory,

until it has allocated twice the amount of memory you said your machine had when you adjusted the

define  PHY_MEM_MEGS . The surprise is that it works at all, because we appear to have created a program

that uses every single byte of physical memory on the author’s machine. Notice that we use the  size_t

type for our call to  malloc .

The other interesting feature is that, at least on this machine, it ran the program in the blink of an eye.

So not only have we apparently used up all the memory, but we’ve done it very quickly indeed.

Let’s investigate further and see just how much memory we can allocate on this machine with  memory3.c .

Since it’s now clear that Linux can do some very clever things with requests for memory, we’ll allocate

memory just 1K at a time and write to each block that we obtain.

Try It Out Available Memory

This is  memory3.c . By its very nature, it’s extremely system-unfriendly and could affect a multiuser

machine quite seriously. If you’re at all concerned about the risk, it’s better not to run it at all; it won’t

harm your understanding if you don’t:

#include <unistd.h>

#include <stdlib.h>

#include <stdio.h>

#define ONE_K (1024)

int main()

{

char *some_memory;

int size_to_allocate = ONE_K;

int megs_obtained = 0;

int ks_obtained = 0;

while (1) {

for (ks_obtained = 0; ks_obtained < 1024; ks_obtained++) {

some_memory = (char *)malloc(size_to_allocate);

if (some_memory == NULL) exit(EXIT_FAILURE);

sprintf(some_memory, “Hello World”);

}

megs_obtained++;

258

Chapter 7: Data Management

printf(“Now allocated %d Megabytes\n”, megs_obtained);

}

exit(EXIT_SUCCESS);

}

This time, the output, again abbreviated, is

$ ./memory3

Now allocated 1 Megabytes

...

Now allocated 1535 Megabytes

Now allocated 1536 Megabytes

Out of Memory: Killed process 2365

Killed

and then the program ends. It also takes quite a few seconds to run, and slows down significantly around

the same number as the physical memory in the machine, and exercises the hard disk quite noticeably.

However, the program has allocated, and accessed, more memory than this author physically has in his

machine at the time of writing. Finally, the system protects itself from this rather aggressive program and

kills it. On some systems it may simply exit quietly when  malloc fails.

How It Works

The application’s allocated memory is managed by the Linux kernel. Each time the program asks for

memory or tries to read or write to memory that it has allocated, the Linux kernel takes charge and

decides how to handle the request.

Initially, the kernel was simply able to use free physical memory to satisfy the application’s request for

memory, but once physical memory was full, it started using what’s called swap space. On Linux, this is

a separate disk area allocated when the system was installed. If you’re familiar with Windows, the Linux

swap space acts a little like the hidden Windows swap file. However, unlike Windows, there are no local

heap, global heap, or discardable memory segments to worry about in code — the Linux kernel does all

the management for you.

The kernel moves data and program code between physical memory and the swap space so that each

time you read or write memory, the data always appears to have been in physical memory, wherever it

was actually located before you attempted to access it.

In more technical terms, Linux implements a demand paged virtual memory system. All memory seen

by user programs is virtual; that is, it doesn’t actually exist at the physical address the program uses.

Linux divides all memory into pages, commonly 4,096 bytes per page. When a program tries to access

memory, a virtual-to-physical translation is made, although how this is implemented and the time it

takes depend on the particular hardware you’re using. When the access is to memory that isn’t physi-

cally resident, a page fault results and control is passed to the kernel.

The Linux kernel checks the address being accessed and, if it’s a legal address for that program, determines

which page of physical memory to make available. It then either allocates it, if it has never been written

before, or, if it has been stored on the disk in the swap space, reads the memory page containing the data

259

Chapter 7: Data Management

into physical memory (possibly moving an existing page out to disk). Then, after mapping the virtual

memory address to match the physical address, it allows the user program to continue. Linux applications

don’t need to worry about this activity because the implementation is all hidden in the kernel.

Eventually, when the application exhausts both the physical memory and the swap space, or when the

maximum stack size is exceeded, the kernel finally refuses the request for further memory and may pre-

emptively terminate the program