How memory allocation works on Linux

It is important to understand the details of Linux memory allocation, especially in the kernel and system architecture. Let’s take a deep dive into Linux memory allocation and understand what’s happening behind the scenes.

In a computer, to make a process executable, it needs to be placed in memory. To do this, the field must be allocated to the process in memory. Memory allocation is an important issue to pay attention to, especially in the kernel and system architecture.

Let’s take a closer look at Linux memory allocation and understand what’s going on behind the scenes.

How is memory allocation done?

Most software engineers don't know the details of this process. But if you are a systems programmer candidate, you should know more about it. When looking at the allocation process, it is necessary to make a small detail about Linux and the glibc library.

When applications need memory, they must request it from the operating system. This request from the kernel naturally requires a system call. You cannot allocate memory yourself in user mode.

**malloc()** series of functions are responsible for memory allocation in C language. The question to ask here is whether malloc() as a glibc function makes a direct system call.

There is no system call called malloc in the Linux kernel. However, there are two system calls for application memory requirements, they are brk and mmap.

Since you will be requesting memory in your application through glibc functions, you may want to know which system calls glibc is using at this time. The answer is both.

First system call: brk

Each process has a continuous data field. Through the brk system call, the program interrupt value that determines the data field limit is incremented and the allocation process is performed.

Although allocating memory using this method is very fast, it is not always possible to return unused space to the system.

For example, suppose you allocate five fields, each 16KB in size, to the brk system call through the malloc() function. When you complete the second of these fields, the related resource cannot be returned (deallocated) so that the system can use it. Because if you reduce the address value to show where the second field starts, and call brk, you will complete the freeing of the third, fourth, and fifth fields.

To prevent memory loss in this case, the malloc implementation in glibc monitors the allocated location in the process data field and then returns it to the system as specified by the free() function so that the system can use the free space for further memory allocation.

In other words, after allocating five 16KB areas, if you use the free() function to return the second area, and then request another 16KB area after a while, instead of expanding the data area through the brk system call, The previous address is back.

However, if the newly requested region is larger than 16KB, the data region will be expanded by allocating a new region via the brk system call because region 2 cannot be used. Although area number two is not used, the application cannot use it due to the different size. Because of scenarios like this, there's a thing called internal fragmentation, where you can rarely actually fully utilize all parts of the memory.

For better understanding, try compiling and running the following sample application:

#include 
#include 
#include 
int main(int argc, char* argv[])
{
        char *ptr[7];
        int n;
        printf("Pid of %s: %d", argv[0], getpid());
        printf("Initial program break   : %p", sbrk(0));
        for(n=0; nprintf("After 5 x 16kB malloc   : %p", sbrk(0));
        free(ptr[1]);
        printf("After free of second 16kB       : %p", sbrk(0));
        ptr[5] = malloc(16 * 1024);
        printf("After allocating 6th of 16kB    : %p", sbrk(0));
        free(ptr[5]);
        printf("After freeing last block        : %p", sbrk(0));
        ptr[6] = malloc(18 * 1024);
        printf("After allocating a new 18kB     : %p", sbrk(0));
        getchar();
        return 0;
}

When you run the application, you will get output similar to the following:

Pid of ./a.out: 31990
Initial program break   : 0x55ebcadf4000
After 5 x 16kB malloc   : 0x55ebcadf4000
After free of second 16kB       : 0x55ebcadf4000
After allocating 6th of 16kB    : 0x55ebcadf4000
After freeing last block        : 0x55ebcadf4000
After allocating a new 18kB     : 0x55ebcadf4000

The output of brk with strace is as follows:

brk(NULL)                               = 0x5608595b6000
brk(0x5608595d7000)                     = 0x5608595d7000

As you can see, 0x21000 has been added to the end address of the data field. You can understand this from the value 0x5608595d7000. So approximately 0x21000 or 132KB of memory is allocated.

There are two points to consider here. The first is to allocate more than specified in the example code. The other is which line of code causes the brk call that provides the allocation.

Address space layout randomization: ASLR

When you run the above sample applications one after another, you will see different address values ​​each time. Randomly changing the address space in this way greatly complicates the work of security attacks and increases the security of the software.

但是，在 32 位架构中，通常使用 8 位来随机化地址空间。增加位数将不合适，因为剩余位上的可寻址区域将非常低。此外，仅使用 8 位组合不会使攻击者的事情变得足够困难。

另一方面，在 64 位体系结构中，由于可以为 ASLR 操作分配的位太多，因此提供了更大的随机性，并且提高了安全程度。

Linux 内核还支持基于 Android 的设备，并且 ASLR 功能在 Android 4.0.3 及更高版本上完全激活。即使仅出于这个原因，也可以说 64 位智能手机比 32 位版本具有显着的安全优势。

通过使用以下命令暂时禁用 ASLR 功能，之前的测试应用程序每次运行时都会返回相同的地址值：

echo 0 | sudo tee /proc/sys/kernel/randomize_va_space

要将其恢复到以前的状态，在同一个文件中写入 2 而不是 0 就足够了。

第二个系统调用：mmap

mmap 是 Linux 上用于内存分配的第二个系统调用。通过 mmap 调用，内存中任何区域的空闲空间都映射到调用进程的地址空间。

在以这种方式完成的内存分配中，当您想使用前面 brk 示例中的 free() 函数返回第二个 16KB 分区时，没有机制可以阻止此操作。从进程的地址空间中删除相关的内存段。它被标记为不再使用并返回系统。

因为与使用 brk 相比，使用 mmap 的内存分配非常慢，所以需要分配 brk。

使用 mmap，内存的任何空闲区域都映射到进程的地址空间，因此在该进程完成之前，已分配空间的内容被重置。如果没有以这种方式进行重置，则属于先前使用相关内存区域的进程的数据也可以被下一个不相关的进程访问。这样就不可能谈论系统中的安全性。

Linux 中内存分配的重要性

内存分配非常重要，尤其是在优化和安全问题上。如上面的示例所示，不完全理解此问题可能意味着破坏系统的安全性。

甚至许多编程语言中存在的类似于 push 和 pop 的概念也是基于内存分配操作的。能够很好地使用和掌握系统内存对于嵌入式系统编程和开发安全和优化的系统架构都是至关重要的。

如果您还想涉足 Linux 内核开发，请考虑首先掌握 C 编程语言。

综上所述，Linux 中的内存分配是一个需要注意和理解的重要问题，特别是对于程序员、内核开发人员和系统架构师而言。熟练掌握内存分配可以提高软件性能和安全性，并在嵌入式系统编程和系统架构方面提供更好的支持。同时，C 编程语言的掌握也是涉足 Linux 内核开发的关键。

The above is the detailed content of How memory allocation works on Linux. For more information, please follow other related articles on the PHP Chinese website!