前言
也許有同學很迷惑:tornado不是標榜異步非阻塞解決10K問題的嘛?但是我卻發現不是torando不好，而是你用錯了.比如最近發現一個事情:某網站打開頁面很慢,伺服器cpu/內存都正常.網絡狀態也良好. 後來發現，打開頁面會有很多請求後端數據庫的訪問，有一個mongodb的數據庫業務api的rest服務.但是它的tornado卻用錯了,一步步的來研究問題:
說明
以下的例子都有2個url,一個是耗時的請求，一個是可以或者說需要立刻返回的請求,我想就算一個對技術不熟，從道理上來說的用戶， 他希望的是他訪問的請求不會影響也不會被其他人的請求影響
#!/bin/env python
import tornado.http
import
import tornado.httpopin
import
import tornado.optionsimport tornado.webimport tornado.httpclientimport timefrom tornado.options import defined given port", type=int)class SleepHandler(tornado.web.RequestHandler): def get(self): ")class JustNowHandler(tornado.web.RequestHandler): def get(self): self.write("i hope just now see you__ ornado.options. parse_command_line() app = tornado.web.Application(handlers=[ (r"/sleep", SleepHandler), (r"/justnow", J TPServer(app ) http_server.listen(options.port) tornado.ioloop.IOLoop.instance().start()假如你使用頁面請求或使用哪個工具先訪問http://localhost /sleep,再訪問http://localhost:8000/justnow.你會發現本來可以立刻返回的/jsutnow的請求會一直阻塞到/sleep請求完才返回.
這是為啥?為啥我的請求被/sleep請求阻塞了？如果平時我們的web請求足夠快我們可能不會意識到這個問題，但是事實上經常會有一些耗時的進程，意味著應用程式被有效的鎖定直至處理結束.
這是時候你有沒有想起@tornado.web.asynchronous這個裝飾器？但是使用這個裝飾器有個前提就是你要耗時的執行需要執行異步,比如上面的time.sleep,你只是加裝飾器是沒有作用的，而且需要注意的是Tornado默認在函數處理返回時關閉客戶端的連接,但是當你使用@tornado.web.asynchonous裝飾器時，Tornado永遠不會自己關閉連接，需要顯式的self.finish()關閉
我們大部分的函數都是阻塞的,例如上面的time.sleep其實tornado有個非同步的實作:
#!/bin/env python
import tornado.httpserver
default=8000, help="run on the given port", type=int)
class SleepHandler(tornado.web.RequestHandler):
@tornado.web.asynchronous
@tornado.web.asynchronous
yield tornado.gen.Task(tornado.ioloop.IOLoop.instance().add_timeout, time.time() + 5)
sel web .RequestHandler):
def get(self):
self.write("i hope just now see you")
if __name__ option;
app = tornado.web.Application(handlers=[
(r"/sleep", SleepHandler), (r"/justnow", JustNowHandler)])
http_server = tornado.httpserver.HTTPServer(app)
http_server.listen( options.port)
tornado.ioloop.IOLoop.instance().start()
這裡有個新的tornado.gen.coroutine裝飾器, coroutine是3.0之後新增的裝飾器.以前的辦法是用回調，還是看我這個例子:
class SleepHandler(tornado.web.RequestHandler): @tornado.web.asynchronous @tornado.web.asynchronous. .instance(). add_timeout(time.time() + 5, callback=self.on_response) def on_response(self): self.write("when i sleep 5s") self.finish()使用了callback, 但是新的裝飾器讓我們通過yield實現同樣的效果:你在打開/sleep之後再點擊/justnow， justnow的請求都是立刻返回不受影響.但是用了asynchronous的裝飾器你的耗時的函數也需要執行非同步
剛才說的都是沒有意義的例子，下面寫個有點用的:讀取mongodb資料庫數據，然後再前端按行write出來
from tornado.options defimport run on the given port", type=int)
# db其實就是test資料庫的遊標
db = motor.MotorClient().open_sync().test
class SleepHandler(BaseHandler): .asynchronous
@tornado.gen.coroutine
def get(self):
# 這一行執行還是阻塞需要時間的，我的
# 這一行執行還是阻塞需要時間的，我的# ) 。 _object( )
self.write('
%s' % message['a']))
def _on_response(self, message, error): if error: raise tornado.web.HTTPError(500, error)
𠟎 for i in message:
self.write('
%s' % i['a'])
else:
sel
def get(self): self.write("i hope just now see you ")if __name__ == "__main__": tornado.options.parse_command_line() app = tornado.web.Application(handlers=[Han (r"/真主這個耗時的東西不能異步的丟給某工具去執行而不阻塞我的請求呢?好吧，我也想到了:celery，正好github有這個東西:tornado-celery執行下面的程序首先你要安裝rabbitmq和celery:
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
import tornado.gen
import tornado.httpclient
import tcelery, tasks
import time
from tornado.options import define, options
int)
tcelery.setup_nonblocking_producer()
class SleepHandler(tornado.web.RequestHandler):
@tornado.web.asynch :
# tornado. gen.Task的參數是:要執行的函數, 參數
class JustNowHandler(tornado.web.RequestHandler):
def get(self):
==
tornado.options.parse_command_line() app = tornado.web.Application(handlers=[ http_server = tornado. httpserver.HTTPServer(app) http_server.listen(options.port) tornado.ioloop.IOLoop.instance().start()time
task是celery的任務定義的文件，包含我們說的time函數import time
from celery import Celery
celery = Celery("tasks", broker="amqp://guest:guest@localhost:5672")
celery.conf.CELERY_RESULT_BACKEND = "amqp"
@celery.task
def sleep(seconds):
time.sleep(float(seconds))
return seconds
if __name__ == "__main__":
celery.start()
Then start the celelry worker (otherwise, how will your task be executed? It is definitely needed A consumer takes it away):
celery -A tasks worker --loglevel=info
But the problem here may also be serious: our asynchronous non-blocking depends on celery, or the length of this queue, if the task If there are many, then you need to wait, which is very inefficient. Is there a way to change my synchronous blocking function to asynchronous (or be understood and recognized by tornado's decorator)?
#!/bin/env python
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
import tornado.httpclient
import tornado.gen
from tornado.concurrent import run_on_executor
# This concurrency library In python3, you need to install sudo pip install futures
from concurrent.futures import ThreadPoolExecutor
import time
from tornado.options import define, options
define("port", default=8000, help="run on the given port", type=int)
class SleepHandler(tornado.web.RequestHandler):
executor = ThreadPoolExecutor(2)
#executor is a local variable not global
@tornado.web.asynchronous
@tornado.gen.coroutine
def get(self):
# If the asynchronous execution you perform will return a value and continue to be called, you can do this (just for demonstration), otherwise just yield directly
res = yield self.sleep() I Self.write ("WHEN I SLEEP % S" % Res)
Self.finish () @Run_ON_EXECUTOR
DEF SLEEP (SELF):
Time.sleep (5) Return 5
Class JustNowHandler(tornado.web.RequestHandler):
def get(self):
self.write("i hope just now see you")
if __name__ == "__main__":
tornado.options.parse_command_line( )
app = tornado.web.Application(handlers=[
(r"/sleep", SleepHandler), (r"/justnow", JustNowHandler)])
http_server = tornado.httpserver.HTTPServer(app)
http_server.listen(options.port)
tornado.ioloop.IOLoop.instance().start()
