Describe the MEM_ROOT_PHP tutorial of MySQL in detail

Details of MySQL's MEM_ROOT

This article will explain in detail the MEM_ROOT structure that is widely used in MySQL. At the same time, it will omit the debug part of the information and only analyze the normal conditions in mysql. Use MEM_ROOT to do the memory allocation part.

Before the specific analysis, let’s first enumerate some macros used in the use of this structure:

<ol style="margin:0 1px 0 0px;padding-left:40px;" start="1" class="dp-css"><li>#define MALLOC_OVERHEAD 8 //分配过程中，需要保留一部分额外的空间<br /> </li><li>#define ALLOC_MAX_BLOCK_TO_DROP 4096 //后续会继续分析该宏的用途<br /></li><li>#define ALLOC_MAX_BLOCK_USAGE_BEFORE_DROP 10 //后续会继续分析该宏的用途<br /></li><li><br /></li><li>#define ALIGN_SIZE(A) MY_ALIGN((A),sizeof(double))<br /></li><li>#define MY_ALIGN(A,L) (((A) + (L) - 1) & ~((L) - 1))<br /></li><li><br /></li><li>#define ALLOC_ROOT_MIN_BLOCK_SIZE (MALLOC_OVERHEAD + sizeof(USED_MEM) + 8)<br /></li><li>/* Define some useful general macros (should be done after all headers). */<br /></li><li>#define MY_MAX(a, b) ((a) > (b) ? (a) : (b)) //求两个数值之间的最大值<br /></li><li>#define MY_MIN(a, b) ((a) < (b) ? (a) : (b)) //求两个数值之间的最小值</li></ol>

Let’s take a look at the information related to the MEM_ROOT structure:

<ol style="margin:0 1px 0 0px;padding-left:40px;" start="1" class="dp-css"><li>typedef struct st_mem_root<br /> </li><li>{<br /></li><li>USED_MEM *free; //free block link list的链表头指针<br /></li><li>USED_MEM *used;//used block link list的链表头指针<br /></li><li>USED_MEM *pre_alloc; //预先分配的block<br /></li><li>size_t min_malloc; //如果block剩下的可用空间小于该值，将会从free list移动到used list<br /></li><li>size_t block_size; //每次初始化的空间大小<br /></li><li>unsigned int block_num; //记录实际的block数量,初始化为4<br /></li><li>unsigned int first_block_usage; //free list中的第一个block 测试不满足分配空间大小的次数<br /></li><li>void (*error_handler)(void);//分配失败的错误处理函数<br /></li><li>} MEM_ROOT; </li></ol>

The following is the specific block information allocated.

<ol style="margin:0 1px 0 0px;padding-left:40px;" start="1" class="dp-css"><li>typedef struct st_used_mem<br /> </li><li>{ <br /></li><li>struct st_used_mem *next; //指向下一个分配的block<br /></li><li>unsigned int left; //该block剩余的空间大小<br /></li><li>unsigned int size; //该block的总大小<br /></li><li>} USED_MEM; </li></ol>

In fact, during the allocation process, MEM_ROOT manages used and free blocks through a doubly linked list:


The initialization process of MEM_ROOT is as follows:

<ol style="margin:0 1px 0 0px;padding-left:40px;" start="1" class="dp-css"><li>void init_alloc_root(MEM_ROOT *mem_root, size_t block_size,size_t pre_alloc_size __attribute__((unused)))<br /> </li><li>{<br /></li><li>mem_root->free= mem_root->used= mem_root->pre_alloc= 0;<br /></li><li>mem_root->min_malloc= 32;<br /></li><li>mem_root->block_size= block_size - ALLOC_ROOT_MIN_BLOCK_SIZE;<br /></li><li>mem_root->error_handler= 0;<br /></li><li>mem_root->block_num= 4; /* We shift this with >>2 */<br /></li><li>mem_root->first_block_usage= 0;<br /></li><li>} </li></ol>

During the initialization process, the block_size space is block_size-ALLOC_ROOT_MIN_BLOCK_SIZE. Because when the memory is not enough and needs to be expanded, the capacity is expanded through mem_root->block_num >>2 *block_size, so mem_root->block_num >>2 is at least 1, so during the initialization process mem_root->block_num=4 (Note :4>>2=1).

Let’s take a look at the specific steps of allocating memory:

<ol style="margin:0 1px 0 0px;padding-left:40px;" start="1" class="dp-css"><li>void *alloc_root(MEM_ROOT *mem_root, size_t length)<br /> </li><li>{<br /></li><li>size_t get_size, block_size;<br /></li><li>uchar* point;<br /></li><li>reg1 USED_MEM *next= 0;<br /></li><li>reg2 USED_MEM **prev;<br /></li><li><br /></li><li>length= ALIGN_SIZE(length);<br /></li><li>if ((*(prev= &mem_root->free)) != NULL)<br /></li><li>{<br /></li><li>if ((*prev)->left < length &&<br /></li><li>mem_root->first_block_usage++ >= ALLOC_MAX_BLOCK_USAGE_BEFORE_DROP &&<br /></li><li>(*prev)->left < ALLOC_MAX_BLOCK_TO_DROP)<br /></li><li>{<br /></li><li>next= *prev;<br /></li><li>*prev= next->next; /* Remove block from list */<br /></li><li>next->next= mem_root->used;<br /></li><li>mem_root->used= next;<br /></li><li>mem_root->first_block_usage= 0;<br /></li><li>}<br /></li><li>for (next= *prev ; next && next->left < length ; next= next->next)<br /></li><li>prev= &next->next;<br /></li><li>}<br /></li><li>if (! next)<br /></li><li>{ /* Time to alloc new block */<br /></li><li>block_size= mem_root->block_size * (mem_root->block_num >> 2);<br /></li><li>get_size= length+ALIGN_SIZE(sizeof(USED_MEM));<br /></li><li>get_size= MY_MAX(get_size, block_size);<br /></li><li><br /></li><li>if (!(next = (USED_MEM*) my_malloc(get_size,MYF(MY_WME | ME_FATALERROR))))<br /></li><li>{<br /></li><li>if (mem_root->error_handler)<br /></li><li>(*mem_root->error_handler)();<br /></li><li>DBUG_RETURN((void*) 0); /* purecov: inspected */<br /></li><li>}<br /></li><li>mem_root->block_num++;<br /></li><li>next->next= *prev;<br /></li><li>next->size= get_size;<br /></li><li>next->left= get_size-ALIGN_SIZE(sizeof(USED_MEM)); //bug:如果该block是通过mem_root->block_size * (mem_root->block_num >> 2)计算出来的，则已经去掉了ALIGN_SIZE(sizeof(USED_MEM)，这里重复了。<br /></li><li>*prev=next;<br /></li><li>}<br /></li><li><br /></li><li>point= (uchar*) ((char*) next+ (next->size-next->left));<br /></li><li>/*TODO: next part may be unneded due to mem_root->first_block_usage counter*/<br /></li><li>if ((next->left-= length) < mem_root->min_malloc)<br /></li><li>{ /* Full block */<br /></li><li>*prev= next->next; /* Remove block from list */<br /></li><li>next->next= mem_root->used;<br /></li><li>mem_root->used= next;<br /></li><li>mem_root->first_block_usage= 0;<br /></li><li>}<br /></li><li>} </li></ol>

The specific logic of the above code is as follows: 1. Check the free linked list to find a block that meets the space. If a suitable block is found, then: 1.1 Directly return the initial address of the block from size-left. Of course, during the free list traversal process, it will be judged that the left space in the first block in the freelist does not meet the space that needs to be allocated, and the block has been searched 10 times (ALLOC_MAX_BLOCK_USAGE_BEFORE_DROP) and does not meet the allocation length, and If the remaining space of the block is less than 4k (ALLOC_MAX_BLOCK_TO_DROP), the block will be moved to the used linked list. 2. If there is no suitable block in the free linked list, then: 2.1 Allocate the larger of mem_root->block_size * (mem_root->block_num >> 2) and length ALIGN_SIZE(sizeof(USED_MEM)) as the new block memory space. 2.2 Depending on the usage of the block, hang the block on the used or free linked list.
What needs to be noted here is the use of secondary pointers:

<ol style="margin:0 1px 0 0px;padding-left:40px;" start="1" class="dp-css"><li>for (next= *prev ; next && next->left < length ; next= next->next)<br /> </li><li>prev= &next->next;<br /></li><li>} </li></ol>

prev points to the address pointed to by next of the last block:

So replace the address of prev is the address of the new block, that is, the new block is added to the end of the free list: *prev=next;


Summary: MEM_ROOT uses a heuristic allocation algorithm for memory allocation. The greater the number of blocks, the larger the memory of a single block will be: block_size= mem_root->block_size * (mem_root->block_num >> 2).

http://www.bkjia.com/PHPjc/1083375.htmlwww.bkjia.comtruehttp: //www.bkjia.com/PHPjc/1083375.htmlTechArticleExplain in detail the MEM_ROOT of MySQL. This article will explain in detail the MEM_ROOT structure that is widely used in MySQL, and at the same time save Go to the debug part of the information and only analyze the use of M in mysql under normal circumstances...