About how Mysql metadata generates Hive table creation statement annotation scripts-Mysql Tutorial-php.cn

Home

Database

Mysql Tutorial

About how Mysql metadata generates Hive table creation statement annotation scripts

小云云

Dec 22, 2017 pm 02:18 PM

mysql how generate

Recently, when extracting data from Mysql and other relational databases into a Hive table, it is necessary to synchronize the comments in the MySQL table. The following script can generate hive table field comment modification statements. It mainly introduces to you the relevant information about how Mysql metadata generates Hive table creation statement annotation scripts. The article introduces it in great detail through sample code. It has certain reference learning value for everyone's study or work. Friends who need it can follow the instructions below. Let’s learn together.

Note: Other relational databases such as oracle can be implemented through the same idea, reading metadata and modifying the script syntax.

Use:

Execute the following statement in mysql metadata database: information_schema

SELECT CONCAT(&#39;alter table &#39;, TABLE_NAME, &#39; CHANGE COLUMN &#39;, COLUMN_NAME, &#39; &#39;, COLUMN_NAME, &#39; &#39;, DATA_TYPE, &#39; comment &#39;, &#39;"&#39;, COLUMN_COMMENT, &#39;"&#39;, &#39;;&#39;)
FROM (SELECT TABLE_NAME, COLUMN_NAME, CASE WHEN DATA_TYPE = &#39;varchar&#39; THEN &#39;string&#39; WHEN DATA_TYPE = &#39;int&#39; THEN &#39;int&#39; WHEN DATA_TYPE = &#39;tinyint&#39; THEN &#39;tinyint&#39; WHEN DATA_TYPE = &#39;decimal&#39; THEN &#39;double&#39; WHEN DATA_TYPE = &#39;datetime&#39; THEN &#39;string&#39; WHEN DATA_TYPE = &#39;timestamp&#39; THEN &#39;string&#39; WHEN DATA_TYPE = &#39;float&#39; THEN &#39;double&#39; WHEN DATA_TYPE = &#39;double&#39; THEN &#39;double&#39; WHEN DATA_TYPE = &#39;bigint&#39; THEN &#39;bigint&#39; END AS DATA_TYPE, COLUMN_COMMENT
FROM COLUMNS
WHERE TABLE_NAME = &#39;o_oms_statistic_profit&#39;
) t;

Copy after login

When extracting data from other relational databases such as Mysql into a Hive table, the comments in the mysql table need to be synchronized. The following script can generate a hive create table statement. Only the main field information of the hive table is generated, and other information needs to be added manually.

Execute the following statement in the mysql metadata database: information_schema

SELECT CONCAT(&#39;create table &#39;, TABLE_NAME, &#39;(&#39;, substring(column_info, 1, length(column_info) - 1), &#39;)&#39;, &#39; comment &#39;, &#39;"&#39;, TABLE_COMMENT, &#39;"&#39;, &#39;;&#39;)
FROM (SELECT TABLE_NAME, TABLE_COMMENT, group_concat(CONCAT(COLUMN_NAME, &#39; &#39;, DATA_TYPE, &#39; comment &#39;, &#39;"&#39;, COLUMN_COMMENT, &#39;"&#39;)) AS column_info
FROM (SELECT t1.TABLE_NAME, CASE WHEN t2.TABLE_COMMENT = NULL THEN t1.TABLE_NAME ELSE t2.TABLE_COMMENT END AS TABLE_COMMENT, COLUMN_NAME, CASE WHEN DATA_TYPE = &#39;varchar&#39; THEN &#39;string&#39; WHEN DATA_TYPE = &#39;int&#39; THEN &#39;int&#39; WHEN DATA_TYPE = &#39;tinyint&#39; THEN &#39;tinyint&#39; WHEN DATA_TYPE = &#39;decimal&#39; THEN &#39;double&#39; WHEN DATA_TYPE = &#39;datetime&#39; THEN &#39;string&#39; WHEN DATA_TYPE = &#39;timestamp&#39; THEN &#39;string&#39; WHEN DATA_TYPE = &#39;float&#39; THEN &#39;double&#39; WHEN DATA_TYPE = &#39;double&#39; THEN &#39;double&#39; WHEN DATA_TYPE = &#39;bigint&#39; THEN &#39;bigint&#39; END AS DATA_TYPE, CASE WHEN COLUMN_COMMENT = NULL THEN COLUMN_NAME ELSE COLUMN_COMMENT END AS COLUMN_COMMENT
FROM COLUMNS t1 JOIN TABLES t2 ON t1.TABLE_NAME = t2.TABLE_NAME
WHERE t1.TABLE_NAME = &#39;o_oms_statistic_profit&#39;
) t3
GROUP BY TABLE_NAME, TABLE_COMMENT
) t4;

Copy after login

Related recommendations:

What is MySQL metadata? Introduction to metadata and example code

Installation and simple testing of Hive based on MySQL metadata

[MySQL] Steps to obtain metadata

The above is the detailed content of About how Mysql metadata generates Hive table creation statement annotation scripts. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website

The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress images for free

Clothoff.io

AI clothes remover

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

How to fix KB5055523 fails to install in Windows 11?

4 weeks ago By DDD

How to fix KB5055518 fails to install in Windows 10?

4 weeks ago By DDD

Roblox: Grow A Garden - Complete Mutation Guide

2 weeks ago By DDD

Roblox: Bubble Gum Simulator Infinity - How To Get And Use Royal Keys

3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

How to fix KB5055612 fails to install in Windows 10?

3 weeks ago By DDD

Hot Tools

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Hot Topics

Java Tutorial

1663

CakePHP Tutorial

1420

Laravel Tutorial

1315

PHP Tutorial

1266

C# Tutorial

1239

Related knowledge

MySQL's Role: Databases in Web Applications Apr 17, 2025 am 12:23 AM

The main role of MySQL in web applications is to store and manage data. 1.MySQL efficiently processes user information, product catalogs, transaction records and other data. 2. Through SQL query, developers can extract information from the database to generate dynamic content. 3.MySQL works based on the client-server model to ensure acceptable query speed.

Laravel Introduction Example Apr 18, 2025 pm 12:45 PM

Laravel is a PHP framework for easy building of web applications. It provides a range of powerful features including: Installation: Install the Laravel CLI globally with Composer and create applications in the project directory. Routing: Define the relationship between the URL and the handler in routes/web.php. View: Create a view in resources/views to render the application's interface. Database Integration: Provides out-of-the-box integration with databases such as MySQL and uses migration to create and modify tables. Model and Controller: The model represents the database entity and the controller processes HTTP requests.

MySQL and phpMyAdmin: Core Features and Functions Apr 22, 2025 am 12:12 AM

MySQL and phpMyAdmin are powerful database management tools. 1) MySQL is used to create databases and tables, and to execute DML and SQL queries. 2) phpMyAdmin provides an intuitive interface for database management, table structure management, data operations and user permission management.

Solve database connection problem: a practical case of using minii/db library Apr 18, 2025 am 07:09 AM

I encountered a tricky problem when developing a small application: the need to quickly integrate a lightweight database operation library. After trying multiple libraries, I found that they either have too much functionality or are not very compatible. Eventually, I found minii/db, a simplified version based on Yii2 that solved my problem perfectly.

MySQL vs. Other Programming Languages: A Comparison Apr 19, 2025 am 12:22 AM

Compared with other programming languages, MySQL is mainly used to store and manage data, while other languages such as Python, Java, and C are used for logical processing and application development. MySQL is known for its high performance, scalability and cross-platform support, suitable for data management needs, while other languages have advantages in their respective fields such as data analytics, enterprise applications, and system programming.

Laravel framework installation method Apr 18, 2025 pm 12:54 PM

Article summary: This article provides detailed step-by-step instructions to guide readers on how to easily install the Laravel framework. Laravel is a powerful PHP framework that speeds up the development process of web applications. This tutorial covers the installation process from system requirements to configuring databases and setting up routing. By following these steps, readers can quickly and efficiently lay a solid foundation for their Laravel project.

MySQL for Beginners: Getting Started with Database Management Apr 18, 2025 am 12:10 AM

The basic operations of MySQL include creating databases, tables, and using SQL to perform CRUD operations on data. 1. Create a database: CREATEDATABASEmy_first_db; 2. Create a table: CREATETABLEbooks(idINTAUTO_INCREMENTPRIMARYKEY, titleVARCHAR(100)NOTNULL, authorVARCHAR(100)NOTNULL, published_yearINT); 3. Insert data: INSERTINTObooks(title, author, published_year)VA

MySQL: Structured Data and Relational Databases Apr 18, 2025 am 12:22 AM

MySQL efficiently manages structured data through table structure and SQL query, and implements inter-table relationships through foreign keys. 1. Define the data format and type when creating a table. 2. Use foreign keys to establish relationships between tables. 3. Improve performance through indexing and query optimization. 4. Regularly backup and monitor databases to ensure data security and performance optimization.

See all articles