A scheduled rerun written in Python to obtain database data

This article will share with you a method of regularly re-running to obtain database data based on python. It is very good and has reference value. Friends who need it can refer to it

Children's shoes that do big data often write scheduled tasks. When running data, due to the dependencies between tasks (usually the downstream depends on the data output of the upstream), data acquisition often fails, because many people will check the logs after discovering that the data fails, and then manually Go and perform your own tasks. Below I have implemented an automatic and repeated execution to retrieve data from the database. If it fails, it will automatically re-obtain it until the data is obtained.

Create a data table:

CREATE TABLE `testtable` ( 2 `id` int(11) unsigned NOT NULL AUTO_INCREMENT, 3 `name` varchar(20) NOT NULL, 4 PRIMARY KEY (`id`) 5 ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;

The data table is empty at the beginning. When the script retries for the third second, it will be inserted into the database. data.

The following is the implementation of the python code

#!/usr/bin/env python
 #-*- coning:utf-8 -*- 3 4 import MySQLdb 5 from time import sleep 6 7 class GetData(object): 8 def __init__(self): 9 self.conn = &#39;&#39; 10 self.host = &#39;127.0.0.1&#39; 11 self.port = 3306 12 self.user = &#39;root&#39; 13 self.passwd = &#39;123456&#39; 14 self.db = &#39;test&#39; 15 self.cnum = 5 #set retry number 16 17 def init_connect(self): 18 self.conn = MySQLdb.connect(host=self.host, user=self.user, passwd=self.passwd, db=self.db, port=self.port, 19 charset=&#39;utf8&#39;) 20 21 def get_data(self): 22 self.init_connect 23 cur = self.conn.cursor 24 sql = "select * from testtable" 25 cur.execute(sql) 26 rs = cur.fetchall 27 cur.close 28 self.conn.close 29 return rs 30 31 def run(self): 32 count = 1 33 while (count <= self.cnum): 34 rs = self.get_data 35 if len(rs) > 0: 36 print len(rs) 37 break 38 39 print count 40 sleep(10) 41 count += 1 42 43 if __name__ == &#39;__main__&#39;: 44 gd = GetData 45 gd.run

You can execute it manually. After the code is executed, At 3 seconds, execute the following sql

insert into testtable(`name`) values ('123'),('456'),('789'),('1111'),('3222'),('444');

The following is the script of the scheduled task

00 08 * * * cd /home/python/lsh_sync; python getdata.py >> getdata.log 2>&1

The above is a scheduled rerun written in Python to obtain database data introduced by the editor. I hope it will be helpful to you. If you have any questions, please leave me a message and the editor will reply to you in time. I would also like to thank you all for your support of the PHP Chinese website!

For more articles related to a scheduled rerun written in Python to obtain database data, please pay attention to the PHP Chinese website!