Example analysis of Redis's zmalloc function

Let’s look directly at the custom zmalloc function in the Redis source code (not the latest version). This function is used in exactly the same way as regular functions such as malloc. The difference lies in its internal implementation details.

void *zmalloc(size_t size) {

// Allocate memory;

void *ptr = malloc(size PREFIX_SIZE);

// Allocation failure throws exception;

If (!ptr) zmalloc_oom_handler(size);

// Can the system use the "malloc_size" function?

#ifdef HAVE_MALLOC_SIZE

Update_zmalloc_stat_alloc(zmalloc_size(ptr));

Return ptr;

#else

//Save the actual size of allocated data in the data field;

*((size_t*)ptr) = size;

// Calculate the memory usage after alignment and update the "used_memory" variable;

Update_zmalloc_stat_alloc(size PREFIX_SIZE);

// Return the initial position of the data body;

Return (char*)ptr PREFIX_SIZE;

#endif

}

In fact, the malloc function in the standard library can already automatically align the allocated memory, so the main purpose of the zmalloc method here is to accurately calculate the memory size allocated for each data storage. Each time memory is allocated, zmalloc will add an additional memory space of PREFIX_SIZE size to the allocated data memory size. This PREFIX_SIZE macro represents the maximum memory addressing space size (size_t) of the current system. Varies depending on the type of specific system. Here we can refer to this PREFIX_SIZE size space as the "data header" part of a storage unit.

The storage unit structure of the first version of Redis

As shown in the figure above, through the *((size_t*)ptr) = size; statement, Redis stores the actual allocated data block size in the first PREFIX_SIZE bytes of the currently allocated memory block, that is, in the data header, and in the following Binary data entities are actually stored in the memory space of "size" size. The function named update_zmalloc_stat_alloc here maintains a global variable named used_memory internally, which accumulates the memory size newly allocated each time. The function returns an offset pointer at the end, pointing to the data body part of the currently allocated memory. The specific implementation details of the update_zmalloc_stat_alloc function are as follows.

#define update_zmalloc_stat_alloc(__n) do {

size_t _n = (__n);

// Manual memory completion;

If (_n&(sizeof(long)-1)) _n = sizeof(long)-(_n&(sizeof(long)-1));

atomicIncr(used_memory, __n);

} while(0)

The important thing to note here is the line _n = sizeof(long)-(_n&(sizeof(long)-1));. The entire macro function first determines whether the memory size allocated this time is an integer multiple of sizeof(long) (64-bit machines correspond to 8-byte memory alignment; 32-bit machines correspond to 4-byte memory alignment), if not Then use the statement we gave before to add the corresponding placeholder space after the data segment to make up the number of bits to meet the memory alignment (4/8 bytes) requirements. The final atomicIncr function is used to update the global used_memory variable value while ensuring thread safety.

The process of memory release and memory allocation in this version of Redis is exactly the opposite. The code shown below is the implementation details of the corresponding "zfree" function. First, the function points to the first address of the data field containing the actual size of the data block through the (char*)ptr-PREFIX_SIZE statement (moving to a lower memory address), and then obtains the data through the *((size_t*)realptr) statement. The real memory size allocated by the block (excluding memory alignment areas). Finally, update the value of the global variable used_memory through the update_zmalloc_stat_free function and release the memory segment.

void zfree(void *ptr) {

#ifndef HAVE_MALLOC_SIZE

void *realptr;

size_t oldsize;

#endif

If (ptr == NULL) return;

#ifdef HAVE_MALLOC_SIZE

Update_zmalloc_stat_free(zmalloc_size(ptr));

free(ptr);

#else

realptr = (char*)ptr-PREFIX_SIZE;

Oldsize = *((size_t*)realptr);

Update_zmalloc_stat_free(oldsize PREFIX_SIZE);

free(realptr);

#endif

}

As shown below, if we look at the implementation details of the update_zmalloc_stat_free function, you will find that its execution process is similar to the previous update_zmalloc_stat_alloc function. By calculating the size of memory bytes that need to be supplemented and subtracting the corresponding size of memory space from the used_memory variable, the memory space usage can be accurately calculated.

#define update_zmalloc_stat_free(__n) do { \

size_t _n = (__n); \

If (_n&(sizeof(long)-1)) _n = sizeof(long)-(_n&(sizeof(long)-1)); \

atomicDecr(used_memory,__n); \

} while(0) 

The above is the detailed content of Example analysis of Redis's zmalloc function. For more information, please follow other related articles on the PHP Chinese website!