Memory leak means that after the program applies for memory, it cannot release the allocated memory space. Memory overflow means that when the program applies for memory, there is not enough memory for the applicant to use; or it provides a storage space to store int data, but long data is stored, the result is that the memory is not enough, and an OOM error is reported. The accumulation of memory leaks will eventually lead to memory overflow.
The operating environment of this tutorial: windows7 system, java8 version, DELL G3 computer.
means that after the program applies for memory, it cannot release the applied memory space. A memory leak does not seem to occur. It has a big impact, but the consequence of accumulation of memory leaks is memory overflow.
means that when the program applies for memory, there is not enough memory for the applicant to use, or in other words, it gives you a piece of storage int Type data storage space, but you store long type data, then the result is that the memory is not enough, and an OOM error will be reported, which is the so-called memory overflow.
The accumulation of memory leaks will eventually lead to memory overflow
Memory overflow means that the memory space you want exceeds the space actually allocated to you by the system. At this time, the system is equivalent to being unable to meet your needs and will report a memory overflow error.
Memory leak means that you apply to the system to allocate memory for use (new), but do not return it (delete) after using it. As a result, you also lose the memory you applied for. It can no longer be accessed (perhaps you lost its address), and the system cannot allocate it to the required program again. It is equivalent to renting a cabinet with a key. After you store your things and lock the cabinet, you lose the key or fail to return the key. The result is that the cabinet cannot be used by anyone and cannot be trashed. The recycler recycles because it cannot find any information about him.
Memory overflow: A plate can only hold 4 fruits through various methods. You put 5 fruits, but it fell to the ground and could not be eaten. This is overflow. For example, if a stack is pushed when the stack is full, it will inevitably cause a space overflow, which is called an overflow. If the stack is empty, a space overflow will occur if the stack is empty, which is called an underflow. That is, the allocated memory is not enough to hold the sequence of data items, which is called a memory overflow. To put it bluntly, I can't bear that much, so I will report an error.
Common memory leaks. Code with memory leaks will be executed multiple times, causing a memory leak every time it is executed.
Occasional memory leaks. Code that causes memory leaks will only occur under certain circumstances or operations. Frequent and sporadic are relative. For certain circumstances, what is occasional may become common. So the testing environment and testing methods are crucial to detecting memory leaks.
One-time memory leak. The code that causes a memory leak will only be executed once, or due to algorithmic flaws, there will always be only one and only one block of memory leaked. For example, if memory is allocated in the constructor of a class, but the memory is not released in the destructor, the memory leak will only occur once.
Implicit memory leak. The program continuously allocates memory while it is running, but does not release the memory until the end. Strictly speaking, there is no memory leak here, because the program eventually releases all the requested memory. But for a server program that needs to run for days, weeks or even months, failure to release memory in time may also lead to the eventual exhaustion of all the system's memory. Therefore, we call this type of memory leak an implicit memory leak.
(1) Cause of memory overflow:
The amount of data loaded in the memory is too large, such as fetching too much data from the database at one time;
There are references to objects in the collection class. It is not emptied afterwards, making it impossible for the JVM to recycle;
There is an infinite loop in the code or the loop generates too many duplicate object entities;
used BUG in third-party software;
The startup parameter memory value is set too small
(2) Solution to memory overflow Solution:
The first step is to modify the JVM startup parameters and directly increase the memory. (Be sure to add the -Xms and -Xmx parameters.)
The second step is to check the error log to see if there are other exceptions or errors before the "OutOfMemory" error.
The third step is to walk through and analyze the code to find out where memory overflow may occur.
Focus on the following points:
Check whether there is a query to obtain all the data in the database query. Generally speaking, if one hundred thousand records are fetched into the memory at one time, it may cause a memory overflow. This problem is relatively hidden. Before going online, there was less data in the database and it was less likely to cause problems. After going online, there was more data in the database, and a single query may cause memory overflow. Therefore, try to use paging for database queries.
Check whether there are infinite loops or recursive calls in the code.
Check whether there is a large loop that repeatedly generates new object entities.
Check whether there is a query to obtain all data at once in the database query. Generally speaking, if one hundred thousand records are fetched into the memory at one time, it may cause a memory overflow. This problem is relatively hidden. Before going online, there was less data in the database and it was less likely to cause problems. After going online, there was more data in the database, and a single query may cause memory overflow. Therefore, try to use paging for database queries.
Check whether collection objects such as List and MAP are not cleared after use. Collection objects such as List and MAP will always have references to the objects, making these objects unable to be recycled by GC.
The fourth step is to use the memory viewing tool to dynamically view memory usage
##Ten scenarios of memory overflow
When the JVM is running, it first needs the class loader (classLoader) to load the bytecode file of the required class. After loading, it is handed over to the execution engine for execution. During the execution process, a period of space is needed to store data (analogous to CPU and main memory). The allocation and release process of this memory space is the runtime data area that we need to care about. Memory overflow occurs when the class loader is loading. Memory overflow is divided into two categories: OutOfMemoryError and StackOverflowError. The following lists 10 memory overflow situations and explains how memory overflow occurs through example code.1.java heap memory overflow
When a java.lang.OutOfMemoryError:Java heap space exception occurs, the heap memory overflows.1), problem description
2, java heap memory leak
1), problem descriptionMemory leaks in Java are some objects that are not It is used again by the application but cannot be recognized by garbage collection. Therefore, these unused objects still exist in the Java heap space indefinitely. The constant accumulation will eventually trigger java.lang.OutOfMemoryError. 2), Sample code When executing the above code, you may expect it to run forever without any problems, assuming that the pure caching solution only maps the underlying Scales to 10,000 elements instead of all keys already in the HashMap. However, elements will actually continue to be added because the key class does not override its equals() method. 
The corresponding solution is relatively simple: just rewrite the equals method:
3. Garbage collection timeout memory overflow
1), Problem description: When the application exhausts all available memory, the GC overhead limit exceeds the error, and the GC fails to clear it multiple times, java.lang will be triggered. OutOfMemoryError. When the JVM spends a lot of time executing GC, with little effect, and once the entire GC process exceeds the limit, an error will be triggered (the default jvm configuration GC time exceeds 98%, and the recycled heap memory is less than 2%).
2), sample code
3), solution
To reduce the object life cycle, try to quickly perform garbage collection .
4. Metaspace memory overflow
1), problem description
If the metaspace overflows, the system will throw java .lang.OutOfMemoryError: Metaspace. The reason for this abnormal problem is that the system has a lot of code or refers to a lot of third-party packages, or dynamic code generation and class loading are used, which results in a large memory footprint in the metaspace.
2), Sample code
3), Solution
By default, the size of the metaspace is only limited by local memory. However, for the sake of the performance of the whole machine, this item should be set as much as possible to avoid causing the service shutdown of the whole machine.
Optimize parameter configuration to avoid affecting other JVM processes
-XX:MetaspaceSize, initial space size, garbage collection will be triggered when this value is reached Perform type unloading, and the GC will adjust the value: if a large amount of space is released, the value will be appropriately lowered; if a small amount of space is released, the value will be appropriately increased if it does not exceed MaxMetaspaceSize.
-XX:MaxMetaspaceSize, the maximum space, there is no limit by default.
In addition to the above two size-specifying options, there are two GC-related attributes: -XX:MinMetaspaceFreeRatio. After GC, the percentage of the minimum Metaspace remaining space capacity is reduced to the allocated space. garbage collection. -XX:MaxMetaspaceFreeRatio, after GC, the percentage of the maximum Metaspace remaining space capacity is reduced to the garbage collection caused by freeing space.
Refer to third-party packages carefully
For third-party packages, be sure to choose carefully and remove unnecessary packages. This will not only help improve the speed of compilation and packaging, but also help improve the speed of remote deployment.
Focus on frameworks that dynamically generate classes
For frameworks that use a large number of dynamically generated classes, stress testing should be done to verify the dynamically generated classes If memory requirements are exceeded, an exception will be thrown.
5. Direct memory overflow
1) Problem description
When using allocateDirect() in ByteBuffer It will be used sometimes. Many javaNIO (like netty) frameworks are encapsulated as other methods. When this problem occurs, a java.lang.OutOfMemoryError: Direct buffer memory exception will be thrown.
Similar problems will occur if you directly or indirectly use the allocateDirect method in ByteBuffer without clearing it.
2), Sample code
3), Solution
If you often have similar operations, you can consider setting the parameters: -XX:MaxDirectMemorySize , and clear the memory in time.
6. Stack memory overflow
1), Problem description
When a thread executes a Java method, the JVM A new stack frame will be created and pushed to the top of the stack. At this time, the new stack frame becomes the current stack frame. When the method is executed, the stack frame is used to store parameters, local variables, intermediate instructions and other data.
When a method calls itself recursively, the data generated by the new method (which can also be understood as a new stack frame) will be pushed to the top of the stack. Each time the method calls itself, a copy will be made. The data of the current method is pushed to the stack. Therefore, each level of recursion requires the creation of a new stack frame. The result is that more and more memory in the stack will be consumed with recursive calls. If recursive calls themselves a million times, a million stack frames will be generated. This will cause stack memory overflow.
2), Sample code
3), Solution
If there are indeed recursive calls in the program and stack overflow occurs, you can increase the -Xss size can solve the problem of stack memory overflow. Recursive calls prevent the formation of an infinite loop, otherwise stack memory overflow will occur.
7. Memory overflow when creating a local thread
1) Problem description
The thread basically only occupies the memory area other than the heap, and also This error indicates that except for the heap, a memory area cannot be allocated for the thread. This is either because the memory itself is not enough, or the heap space is set too large, resulting in not much remaining memory, and due to the thread It takes up memory itself, so it is not enough.
2), sample code
3), solution
First check whether the operating system has a limit on the number of threads, use shell Threads cannot be created either. If this is the problem, you need to adjust the maximum number of files that the system can support.
In daily development, try to ensure that the maximum number of threads is controllable, and do not use the thread pool arbitrarily. It cannot grow without limit.
8. Memory overflow beyond the swap area
1) Problem description
During the startup process of the Java application, The required memory can be limited by -Xmx and other similar startup parameters. When the total memory requested by the JVM is greater than the available physical memory, the operating system begins to convert the content from memory to hard disk.
Generally speaking, the JVM will throw an Out of swap space error, which means that when the application fails to request the JVM native heap to allocate memory and the native heap is about to be exhausted, the error message contains the size of the allocation failure (in words section) and the reason for the request failure.
2), solution
Increase the size of the system swap area. I personally think that if the swap area is used, the performance will be greatly reduced. This method is not recommended and should be avoided in the production environment. The maximum memory exceeds the system's physical memory. Secondly, remove the system swap area and only use system memory to ensure application performance.
9. Array overflow memory overflow
1) Problem description Sometimes you will encounter this kind of memory overflow description Requested array size exceeds VM limit. Generally speaking, Java has a limit on the maximum size of an array that an application can allocate. However, the limit varies on different platforms, but it is usually between 1 and 2.1 billion elements. When the Requested array size exceeds VM limit error occurs, it means that the application is trying to allocate an array that is larger than the Java virtual machine can support. Before the JVM allocates memory for an array, it performs a platform-specific check: whether the allocated data structure is addressable on this platform.
2), sample code
The following is the code, that is, the array exceeds the maximum limit.
3), solution
Therefore, the array length must be within the length range allowed by the platform. However, this error is generally rare, mainly because the index of a Java array is of type int. The largest positive integer in Java is 2^31 - 1 = 2,147,483,647. And platform-specific limits can be very close to this number, for example: on my environment (64-bit macOS, running Jdk1.8) I can initialize arrays with lengths up to 2,147,483,645 (Integer.MAX_VALUE-2). If the length of the array is increased by 1 to reach nteger.MAX_VALUE-1, an OutOfMemoryError will occur.
10. System kills process memory overflow
1). Problem overview. Before describing the problem, first familiarize yourself with some knowledge of the operating system. : The operating system is built on the concept of processes. These processes work in the kernel. There is a very special process called "Out of memory killer". When the kernel detects that the system has insufficient memory, the OOM killer is activated, checks who currently occupies the most memory, and then kills the process.
Generally the Out of memory: Kill process or sacrifice child error will be triggered when the available virtual virtual memory (including swap space) is consumed to the point that the entire operating system is at risk. In this case, OOM Killer selects the "rogue process" and kills it.
2), Sample code
3), Solution
Although adding swap space can alleviate Java heap space exceptions, It is still recommended that the best solution is to upgrade the system memory so that the Java application has enough memory available, so this problem will not occur.
Summary: Through the above 10 types of memory overflow situations, everyone will know how to solve the problem when they actually encounter it. You should also remember the following in actual coding:
Third-party jar packages should be introduced with caution, and useless jar packages should be removed resolutely to improve compilation speed and system memory usage.
For large objects or a large number of memory applications, optimization must be carried out. Large objects must be processed in slices to improve processing performance and reduce the object life cycle.
Try to fix the number of threads to ensure that the memory occupied by threads is controllable. When a large number of threads are needed, the maximum number of open connections of the operating system must be optimized.
For recursive calls, the level of recursion must also be controlled, not too high and exceed the depth of the stack.
The bigger the memory allocated to the stack is not, the better, because the larger the stack memory, the more threads there are, and there is not much space left for the heap, and it is easy to throw OOM. There is generally no problem with the JVM's default parameters (including recursion).
Recommended related video tutorials: Java video tutorial
The above is the detailed content of What are memory leaks and memory overflows in java. For more information, please follow other related articles on the PHP Chinese website!
![[Web front-end] Node.js quick start](https://img.php.cn/upload/course/000/000/067/662b5d34ba7c0227.png)
