PostgreSQL的mysql_fdw使用_MySQL
要迁移Mysql的数据到PostgreSQL上来,有很多的办法,比如一些工具navicat,jmyetl等,但通常是要收费,而且效率和稳定性不一定好,最保守的办法是自己写一些脚本,当然也有一些开源的工具,比如要介绍的mysql_fdw,使用时本地需要建mysql软件和postgresql数据库,下面的测试例子是mysql和pg装在一起的。
mysql_fdw是一款基于BSD协议的开源工具,目前还不是postgresql内置,最新版本是1.0.1。
一、环境
CentOS 6.3
PostgreSQL 9.3.4
Host 10.1.11.73
二、Mysql的数据准备
mysql> create database db_kenyon;Query OK, 1 row affected (0.01 sec)mysql> create table tbl_kenyon(id int,vname varchar(48));Query OK, 0 rows affected (0.00 sec)mysql> insert into tbl_kenyon values(1,'test');Query OK, 1 row affected (0.00 sec)mysql> insert into tbl_kenyon values(2,'kenyon');Query OK, 1 row affected (0.00 sec)mysql> insert into tbl_kenyon values(null,'it/'s null');Query OK, 1 row affected (0.00 sec)mysql> insert into tbl_kenyon values(4,null);Query OK, 1 row affected (0.01 sec)mysql> select * from tbl_kenyon;+------+-----------+| id | vname |+------+-----------+| 1 | test || 2 | kenyon || NULL | it's null || 4 | NULL |+------+-----------+4 rows in set (0.00 sec)mysql> grant select on db_kenyon.tbl_kenyon to 'usr_kenyon'@'%' identified by '123456';Query OK, 0 rows affected (0.00 sec)mysql> flush privileges;Query OK, 0 rows affected (0.00 sec)mysql> select host,user,password from mysql.user;+-----------+------------+-------------------------------------------+| host | user | password |+-----------+------------+-------------------------------------------+| localhost | root | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 || db1 | root | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 || 127.0.0.1 | root | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 || ::1 | root | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 || % | usr_kenyon | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 |+-----------+------------+-------------------------------------------+5 rows in set (0.00 sec)
下载地址: http://pgxn.org/dist/mysql_fdw/
安装时使用postgres用户,参考README,有两步
make USE_PGXS=1
make USE_PGXS=1 install
安装完可能会有的异常,ERROR: could not load library "/home/postgres/lib/mysql_fdw.so": libmysqlclient.so.18: cannot open shared object file: No such file or directory 将mysql下的libmysqlclient.so.18文件拷贝到/home/postgres/lib下面或者做个软连接就可以了
1.创建extension扩展
postgres=# create extension mysql_fdw ;CREATE EXTENSION
postgres=# CREATE SERVER mysql_svr FOREIGN DATA WRAPPER mysql_fdw OPTIONS (address '10.1.11.73', port '3306');CREATE SERVER
postgres=# CREATE FOREIGN TABLE pg_mysql_tbl1 (id integer, name text) SERVER mysql_svr OPTIONS (table 'db_kenyon.tbl_kenyon');CREATE FOREIGN TABLEpostgres=# CREATE FOREIGN TABLE pg_mysql_tbl2 ( id integer, vname text) SERVER mysql_svr OPTIONS (query 'SELECT id, vname FROM db_kenyon.tbl_kenyon WHERE id2;');CREATE FOREIGN TABLE
postgres=# create user u_select ENCRYPTED PASSWORD '123456';CREATE ROLE
5.创建用户匹配关系(user mapping),用户为远程mysql的用户密码
CREATE USER MAPPING FOR u_select SERVER mysql_svr OPTIONS (username 'usr_kenyon', password '123456');
postgres=> select * from pg_mysql_tbl1 ; id | name ----+----------- 1 | test 2 | kenyon | it's null 4 | (4 rows)postgres=> select * from pg_mysql_tbl2; id | vname ----+------- 1 | test 4 | (2 rows)
postgres=# drop foreign table pg_mysql_tbl1;DROP FOREIGN TABLEpostgres=# drop foreign table pg_mysql_tbl2;DROP FOREIGN TABLEpostgres=# drop user mapping for u_select server mysql_svr ;DROP USER MAPPINGpostgres=# drop server mysql_svr ;DROP SERVERpostgres=# drop extension mysql_fdw ;DROP EXTENSION
五、总结
1.这个工具在ETL迁移Mysql到postgresql时比较有用,跨平台,跨数据库,而且简单方便
2.目前该工具还不是内置版本,引起的风险要注意,比如mysql和pg库端表字段后类型不一样,会产生乱码
六、参考:
1.http://pgxn.org/dist/mysql_fdw/
2.http://francs3.blog.163.com/blog/static/40576727201111211324599/?suggestedreading

Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics



Full table scanning may be faster in MySQL than using indexes. Specific cases include: 1) the data volume is small; 2) when the query returns a large amount of data; 3) when the index column is not highly selective; 4) when the complex query. By analyzing query plans, optimizing indexes, avoiding over-index and regularly maintaining tables, you can make the best choices in practical applications.

InnoDB's full-text search capabilities are very powerful, which can significantly improve database query efficiency and ability to process large amounts of text data. 1) InnoDB implements full-text search through inverted indexing, supporting basic and advanced search queries. 2) Use MATCH and AGAINST keywords to search, support Boolean mode and phrase search. 3) Optimization methods include using word segmentation technology, periodic rebuilding of indexes and adjusting cache size to improve performance and accuracy.

Yes, MySQL can be installed on Windows 7, and although Microsoft has stopped supporting Windows 7, MySQL is still compatible with it. However, the following points should be noted during the installation process: Download the MySQL installer for Windows. Select the appropriate version of MySQL (community or enterprise). Select the appropriate installation directory and character set during the installation process. Set the root user password and keep it properly. Connect to the database for testing. Note the compatibility and security issues on Windows 7, and it is recommended to upgrade to a supported operating system.

The difference between clustered index and non-clustered index is: 1. Clustered index stores data rows in the index structure, which is suitable for querying by primary key and range. 2. The non-clustered index stores index key values and pointers to data rows, and is suitable for non-primary key column queries.

MySQL is an open source relational database management system. 1) Create database and tables: Use the CREATEDATABASE and CREATETABLE commands. 2) Basic operations: INSERT, UPDATE, DELETE and SELECT. 3) Advanced operations: JOIN, subquery and transaction processing. 4) Debugging skills: Check syntax, data type and permissions. 5) Optimization suggestions: Use indexes, avoid SELECT* and use transactions.

In MySQL database, the relationship between the user and the database is defined by permissions and tables. The user has a username and password to access the database. Permissions are granted through the GRANT command, while the table is created by the CREATE TABLE command. To establish a relationship between a user and a database, you need to create a database, create a user, and then grant permissions.

MySQL and MariaDB can coexist, but need to be configured with caution. The key is to allocate different port numbers and data directories to each database, and adjust parameters such as memory allocation and cache size. Connection pooling, application configuration, and version differences also need to be considered and need to be carefully tested and planned to avoid pitfalls. Running two databases simultaneously can cause performance problems in situations where resources are limited.

Data Integration Simplification: AmazonRDSMySQL and Redshift's zero ETL integration Efficient data integration is at the heart of a data-driven organization. Traditional ETL (extract, convert, load) processes are complex and time-consuming, especially when integrating databases (such as AmazonRDSMySQL) with data warehouses (such as Redshift). However, AWS provides zero ETL integration solutions that have completely changed this situation, providing a simplified, near-real-time solution for data migration from RDSMySQL to Redshift. This article will dive into RDSMySQL zero ETL integration with Redshift, explaining how it works and the advantages it brings to data engineers and developers.
