Home Database Mysql Tutorial mysql大表更新sql的优化策略_MySQL

mysql大表更新sql的优化策略_MySQL

Jun 01, 2016 pm 01:26 PM
mysql project

bitsCN.com

问题sql背景:项目有6个表的要根据pid字段要写入对应的brand_id字段。但是这个其中有两个表是千万级别的。我的worker运行之后,线上的mysql主从同步立刻延迟了!运行了一个多小时之后,居然延迟到了40分钟,而且只更新了十几万行数据。问题sql如下:

<!-- 根据商品id更新品牌id -->	<update id="updateBrandIdByPid" parameterClass="com.jd.chat.worker.domain.param.UpdateBrandIdParam">		UPDATE $tableName$		SET brand_id = #newBrandId#		WHERE pid = #pid#			AND brand_id = 0	</update>
Copy after login
项目组的mysql专家帮我分析了下,因为pid字段没有索引,mysql引擎要逐行扫描出与传入的pid值相等的列,然后更新数据,也就是要扫描完1000W+行磁盘数据才能执行完这个sql。更严重的是,这个千万级的表里面有多少个不同的pid,我就要执行多少个这样的sql。

同事给我的建议的根据id字段进行sql代码层次的纵向分表。每次更新1000行的数据,这样mysql引擎就不用每次在扫全表了,数据库压力是之前的万分之一。而且id作为主键,是有索引的有索引,有索引能大大优化查询性能,优化后的sql如下:

<!-- 根据商品id更新品牌id -->    <update id="updateBrandIdByPid" parameterClass="com.jd.chat.worker.domain.param.UpdateBrandIdParam">        UPDATE $tableName$        SET brand_id = #newBrandId#        WHERE pid = #pid#            AND brand_id = 0            AND id BETWEEN #startNum# AND #endNum#    </update>
Copy after login
仅仅用了id限区间的语句,将一个千万级的大表代码层次上进行纵向切割。重新上线worker后,mysql主从没有任何延迟!而且经过监视,短短10分钟就更新了十几万数据,效率是之前的6倍!更重要的是数据库负载均衡,应用健康运行。

bitsCN.com
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
WWE 2K25: How To Unlock Everything In MyRise
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

MySQL: The Ease of Data Management for Beginners MySQL: The Ease of Data Management for Beginners Apr 09, 2025 am 12:07 AM

MySQL is suitable for beginners because it is simple to install, powerful and easy to manage data. 1. Simple installation and configuration, suitable for a variety of operating systems. 2. Support basic operations such as creating databases and tables, inserting, querying, updating and deleting data. 3. Provide advanced functions such as JOIN operations and subqueries. 4. Performance can be improved through indexing, query optimization and table partitioning. 5. Support backup, recovery and security measures to ensure data security and consistency.

Can I retrieve the database password in Navicat? Can I retrieve the database password in Navicat? Apr 08, 2025 pm 09:51 PM

Navicat itself does not store the database password, and can only retrieve the encrypted password. Solution: 1. Check the password manager; 2. Check Navicat's "Remember Password" function; 3. Reset the database password; 4. Contact the database administrator.

How to create navicat premium How to create navicat premium Apr 09, 2025 am 07:09 AM

Create a database using Navicat Premium: Connect to the database server and enter the connection parameters. Right-click on the server and select Create Database. Enter the name of the new database and the specified character set and collation. Connect to the new database and create the table in the Object Browser. Right-click on the table and select Insert Data to insert the data.

How to copy tables in mysql How to copy tables in mysql Apr 08, 2025 pm 07:24 PM

Copying a table in MySQL requires creating new tables, inserting data, setting foreign keys, copying indexes, triggers, stored procedures, and functions. The specific steps include: creating a new table with the same structure. Insert data from the original table into a new table. Set the same foreign key constraint (if the original table has one). Create the same index. Create the same trigger (if the original table has one). Create the same stored procedure or function (if the original table is used).

How to view database password in Navicat for MariaDB? How to view database password in Navicat for MariaDB? Apr 08, 2025 pm 09:18 PM

Navicat for MariaDB cannot view the database password directly because the password is stored in encrypted form. To ensure the database security, there are three ways to reset your password: reset your password through Navicat and set a complex password. View the configuration file (not recommended, high risk). Use system command line tools (not recommended, you need to be proficient in command line tools).

MySQL: Simple Concepts for Easy Learning MySQL: Simple Concepts for Easy Learning Apr 10, 2025 am 09:29 AM

MySQL is an open source relational database management system. 1) Create database and tables: Use the CREATEDATABASE and CREATETABLE commands. 2) Basic operations: INSERT, UPDATE, DELETE and SELECT. 3) Advanced operations: JOIN, subquery and transaction processing. 4) Debugging skills: Check syntax, data type and permissions. 5) Optimization suggestions: Use indexes, avoid SELECT* and use transactions.

How to execute sql in navicat How to execute sql in navicat Apr 08, 2025 pm 11:42 PM

Steps to perform SQL in Navicat: Connect to the database. Create a SQL Editor window. Write SQL queries or scripts. Click the Run button to execute a query or script. View the results (if the query is executed).

How to create a new connection to mysql in navicat How to create a new connection to mysql in navicat Apr 09, 2025 am 07:21 AM

You can create a new MySQL connection in Navicat by following the steps: Open the application and select New Connection (Ctrl N). Select "MySQL" as the connection type. Enter the hostname/IP address, port, username, and password. (Optional) Configure advanced options. Save the connection and enter the connection name.

See all articles