Home Database Mysql Tutorial MySQL数据库inset性能优化_MySQL

MySQL数据库inset性能优化_MySQL

May 27, 2016 pm 01:46 PM

我们在使用中MySQL的时候难免会遇到大批量数据inset的情况,通常最简单的方法就是写一个insert,然后通过循环给变量赋值,批量插入数据库:

                //save rddform
                for (int i = 0; i < rddformlist.Count; i++)
                {
                    string cmdText = "insert into rddform 
                    (ID,CreatedTime,ModifiedTime,CreatedBy,ModifiedBy,FormType) values 
                    (&#39;" + rddformlist[i].RddFormGuid + "&#39;,&#39;" + rddformlist[i].CreateTime + "&#39;,&#39;"
                     + rddformlist[i].ModifyTime + "&#39;,&#39;" + rddformlist[i].CreateBy + "&#39;,&#39;" 
                     + rddformlist[i].ModifyBy + "&#39;," + rddformlist[i].FormType + ")";
                    MySqlCommand mysqlcom = new MySqlCommand(cmdText, mysqlcon);
                    mysqlcom.Transaction = trans;  //绑定事务           
                    mysqlcom.ExecuteNonQuery();
                }
Copy after login

经过方法经过亲测,性能不太好,本地大概是一秒insert20条数据,云上大概是一秒insert3条数据。两千条数据本地大概就要一分半,云上大概十几分,显然速度太慢了。

现在我们改为多条数据合并插入,就是用一个insert语句插入所有的数据:

//save rddform
                for (int i = 0; i < rddformlist.Count; i++)
                {
                    strRddForm.Append("(&#39;" + rddformlist[i].RddFormGuid + "&#39;,&#39;" 
                    + rddformlist[i].CreateTime + "&#39;,&#39;" + rddformlist[i].ModifyTime 
                    + "&#39;,&#39;" + rddformlist[i].CreateBy + "&#39;,&#39;" + rddformlist[i].ModifyBy 
                    + "&#39;,&#39;" + rddformlist[i].FormType + "&#39;)");
                    if (i<rddformlist.Count-1)
                    {
                        strRddForm.Append(",");
                    }
                }
                MySqlCommand rddformcom = new MySqlCommand(strRddForm.ToString(), mysqlcon);
                rddformcom.Transaction = trans;  //绑定事务           
                rddformcom.ExecuteNonQuery();
Copy after login

亲测之后两千条本地真的是秒插,云上大概一秒多,这速度让人根本没有拒绝的理由,果断的把代码改了。当然速度可能根据个人机器以及网络环境因素有关,下面是网上牛人做的性能对比

注意事项:
 SQL语句是有长度限制,在进行数据合并在同一SQL中务必不能超过SQL长度限制,通过max_allowed_packet配置可以修改,默认是1M,测试时修改为8M。
 事务需要控制大小,事务太大可能会影响执行的效率。MySQL有innodb_log_buffer_size配置项,超过这个值会把innodb的数据刷到磁盘中,这时,效率会有所下降。所以比  较好的做法是,在数据达到这个这个值前进行事务提交。

以上就是MySQL数据库inset性能优化_MySQL的内容,更多相关内容请关注PHP中文网(www.php.cn)!


Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
2 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
2 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
2 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

How to optimize MySQL query performance in PHP? How to optimize MySQL query performance in PHP? Jun 03, 2024 pm 08:11 PM

MySQL query performance can be optimized by building indexes that reduce lookup time from linear complexity to logarithmic complexity. Use PreparedStatements to prevent SQL injection and improve query performance. Limit query results and reduce the amount of data processed by the server. Optimize join queries, including using appropriate join types, creating indexes, and considering using subqueries. Analyze queries to identify bottlenecks; use caching to reduce database load; optimize PHP code to minimize overhead.

Performance optimization and horizontal expansion technology of Go framework? Performance optimization and horizontal expansion technology of Go framework? Jun 03, 2024 pm 07:27 PM

In order to improve the performance of Go applications, we can take the following optimization measures: Caching: Use caching to reduce the number of accesses to the underlying storage and improve performance. Concurrency: Use goroutines and channels to execute lengthy tasks in parallel. Memory Management: Manually manage memory (using the unsafe package) to further optimize performance. To scale out an application we can implement the following techniques: Horizontal Scaling (Horizontal Scaling): Deploying application instances on multiple servers or nodes. Load balancing: Use a load balancer to distribute requests to multiple application instances. Data sharding: Distribute large data sets across multiple databases or storage nodes to improve query performance and scalability.

How to create a MySQL table using PHP? How to create a MySQL table using PHP? Jun 04, 2024 pm 01:57 PM

Creating a MySQL table using PHP requires the following steps: Connect to the database. Create the database if it does not exist. Select a database. Create table. Execute the query. Close the connection.

How to fix mysql_native_password not loaded errors on MySQL 8.4 How to fix mysql_native_password not loaded errors on MySQL 8.4 Dec 09, 2024 am 11:42 AM

One of the major changes introduced in MySQL 8.4 (the latest LTS release as of 2024) is that the &quot;MySQL Native Password&quot; plugin is no longer enabled by default. Further, MySQL 9.0 removes this plugin completely. This change affects PHP and other app

Performance optimization in Java microservice architecture Performance optimization in Java microservice architecture Jun 04, 2024 pm 12:43 PM

Performance optimization for Java microservices architecture includes the following techniques: Use JVM tuning tools to identify and adjust performance bottlenecks. Optimize the garbage collector and select and configure a GC strategy that matches your application's needs. Use a caching service such as Memcached or Redis to improve response times and reduce database load. Employ asynchronous programming to improve concurrency and responsiveness. Split microservices, breaking large monolithic applications into smaller services to improve scalability and performance.

How to delete data from MySQL table using PHP? How to delete data from MySQL table using PHP? Jun 05, 2024 pm 12:40 PM

PHP provides the following methods to delete data in MySQL tables: DELETE statement: used to delete rows matching conditions from the table. TRUNCATETABLE statement: used to clear all data in the table, including auto-incremented IDs. Practical case: You can delete users from the database using HTML forms and PHP code. The form submits the user ID, and the PHP code uses the DELETE statement to delete the record matching the ID from the users table.

How to set up MySQL connection pool using PHP? How to set up MySQL connection pool using PHP? Jun 04, 2024 pm 03:28 PM

Setting up a MySQL connection pool using PHP can improve performance and scalability. The steps include: 1. Install the MySQLi extension; 2. Create a connection pool class; 3. Set the connection pool configuration; 4. Create a connection pool instance; 5. Obtain and release connections. With connection pooling, applications can avoid creating a new database connection for each request, thereby improving performance.

The page is blank after PHP is connected to MySQL. What is the reason for the invalid die() function? The page is blank after PHP is connected to MySQL. What is the reason for the invalid die() function? Apr 01, 2025 pm 03:03 PM

The page is blank after PHP connects to MySQL, and the reason why die() function fails. When learning the connection between PHP and MySQL database, you often encounter some confusing things...

See all articles