Introduction to the method of deploying web development in Python

This article mainly introduces several methods of deploying web development programs in Python, which has a good reference value. Let’s take a look at it with the editor

1. Fastcgi is supported through the flup module. The corresponding configuration instruction in nginx is fastcgi_pass

2. http, nginx uses proxy_pass to forward, this It is required that the back-end application must have a built-in http server that can handle high concurrency. In python's web framework, you can only choose tornado.

3, uwsgi, including 4 partsGroup into:

• ##uwsgi protocol

• web server built-in support protocol module

• Application server protocol support module

• Process control program

##nginx has built-in support for uwsgi protocol and uwsgi protocol starting from 0.8.4 Very simple, a 4-byte header + a body. The body can be a package of many protocols, such as http, cgi, etc. (marked by fields in the header).

The characteristic of uwsgi is its own process control program. It is written in C language and uses the natvie function. It is actually similar to spawn-fcgi/php-fpm. Therefore, uwsgi can support a variety of application frameworks, including (python, lua, ruby, erlang, go), etc.

4. mod_python, this is a built-in module of apache, and it relies heavily on the python compiled by mod_python. Version, used with apache, not recommended

5, cgi, this is too old, not recommended, and nginx does not support cgi mode, only lighttpd or apache

6, spawn-fcgi , this is the fastcgi multi-process management program, which comes with the lighttpd

installation

package. It has the same effect as flup. The difference is that flup is introduced at the python code level, and spawn-fcgi is an external program. spawn-fcgi is very versatile and can support code developed in any language, such as php, python, perl. As long as your code implements the fastcgiinterface, it can help you manage your process. 7. scgi, the full name is Simple Common Gateway Interface, is also an alternative version of cgi. The scgi protocol is very simple. I think it is similar to fastcgi, but it has not been widely promoted. The corresponding configuration command of nginx is scgi_pass. You can use it if you want, flup also supports it.

8. Gunicorn, a tool similar to uwsgi, is transplanted from the rails deployment tool (Unicorn). But the protocol it uses is WSGI, the full name is Python Web Server Gateway Interface, which is the official standard (PEP 333) defined in python2.5. It has good roots and is relatively simple to deploy. There are detailed tutorials on gunicorn.org/

9. mod_wsgi, a

module

of apache, also supports the WSGI protocol, code.google.com/p/modwsgi/

uwsgi

Install uwsgi

pip install uwsgi

Configuring uwsgiuwsgi has multiple configurations available:

1，ini 
2，xml 
3，json
4，yaml

Configuration Example

$ cat etc/uwsgi.ini 
[uwsgi]
socket = 127.0.0.1:9005
chdir = /Users/suoning/python_project/trunk/
wsgi-file = main.py
processes = 4
stats = 127.0.0.1:9000
daemonize = /tmp/uwsgiServer.log
pidfile = /tmp/uwsgi.pid
vacuum = true
log-maxsize = 50000000
disable-logging = true
callable = app
$

Detailed explanation of configuration parameters:

##Commonly used Options:

socket:

address and port number, for example: socket = 127.0.0.1:50000

processes:

The number of processes opened

workers:

The number of processes opened is equivalent to processes (the official website says it is spawn the specified number of workers / processes)

chdir:

Specify the running directory (chdir to specified directory before apps loading)

wsgi-file:

Load wsgi-file (load .wsgi file)

stats:

At the specified address, open StatusService (enable the stats server on the specified address)

threads:

Running threads. Due to the existence of GIL, I think this is really useless. (run each worker in prethreaded mode with the specified number of threads)

master:

Allow the master process to exist (enable master process)

daemonize:

Causes the process to run in the background and writes the log to the specified log file or udp server (daemonize uWSGI). In fact, the most commonly used method is to output the running records to a local file.

log-maxsize ：以固定的文件大小（单位KB），切割日志文件。 例如：log-maxsize = 50000000 就是50M一个日志文件。

pidfile ： 指定pid文件的位置，记录主进程的pid号。

vacuum ： 当服务器退出的时候自动清理环境，删除unix socket文件和pid文件（try to remove all of the generated file/sockets）

disable-logging ： 不记录请求信息的日志。只记录错误以及uWSGI内部消息到日志中。如果不开启这项，那么你的日志中会大量出现这种记录：

[pid: 347|app: 0|req: 106/367] 117.116.122.172 () {52 vars in 961 bytes} [Thu Jul 7 19:20:56 2016] POST /post => generated 65 bytes in 6 msecs (HTTP/1.1 200) 2 headers in 88 bytes (1 switches on core 0)

配置nginx

$ cat etc/nginx/servers/tongbupan.conf

server {
 listen  80;
 server_name localhost;

 location / {
  include uwsgi_params;
  uwsgi_pass 127.0.0.1:9005;
 }

 location /webstatic/ {
  expires 7d;
  add_header Cache-Control public;
  alias /Users/suoning/probject/python_project/webstatic/trunk/;
 }

}

$ 
$ nginx -t
nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
$ 
$ nginx -s reload
$

配置application

flask 示例

...
app = Flask('pan')
...

if name == 'main':
 # app.run(host='0.0.0.0', port=5000)
 app.run()

# 注意：变量app对应uwsgi配置文件uwsgi.ini中 callable = app

启动uwsgi

$ 
$ uwsgi --ini /usr/local/etc/uwsgi.ini
[uWSGI] getting INI configuration from /usr/local/etc/uwsgi.ini
$ 
$ ps -ef|grep uwsgi
11428  1 0 11:40下午 ??   0:01.23 uwsgi --ini /usr/local/etc/uwsgi.ini
11432 11428 0 11:40下午 ??   0:00.00 uwsgi --ini /usr/local/etc/uwsgi.ini
11433 11428 0 11:40下午 ??   0:00.00 uwsgi --ini /usr/local/etc/uwsgi.ini
11434 11428 0 11:40下午 ??   0:00.00 uwsgi --ini /usr/local/etc/uwsgi.ini
11435 11428 0 11:40下午 ??   0:00.00 uwsgi --ini /usr/local/etc/uwsgi.ini
11440 69240 0 11:40下午 ttys000 0:00.00 grep uwsgi
$ 
$ lsof -i tcp:9000
COMMAND PID USER FD TYPE    DEVICE SIZE/OFF NODE NAME
uwsgi 11428 suoning 28u IPv4 0x5583e11534d24e73  0t0 TCP localhost:cslistener (LISTEN)
$
$ lsof -i tcp:9005
COMMAND PID USER FD TYPE    DEVICE SIZE/OFF NODE NAME
uwsgi 11428 suoning 6u IPv4 0x5583e11535699e73  0t0 TCP localhost:9005 (LISTEN)
uwsgi 11432 suoning 6u IPv4 0x5583e11535699e73  0t0 TCP localhost:9005 (LISTEN)
uwsgi 11433 suoning 6u IPv4 0x5583e11535699e73  0t0 TCP localhost:9005 (LISTEN)
uwsgi 11434 suoning 6u IPv4 0x5583e11535699e73  0t0 TCP localhost:9005 (LISTEN)
uwsgi 11435 suoning 6u IPv4 0x5583e11535699e73  0t0 TCP localhost:9005 (LISTEN)
$

FCGI

参考：webpy.org/cookbook/fastcgi-nginx

配置Nginx

$ cat etc/nginx/servers/pan.conf

server {
 listen  80;
 server_name localhost;

 error_page 500 502 503 504 /50x.html;
 location = /50x.html {
  root html;
 }

 location / {
  fastcgi_param REQUEST_METHOD $request_method;
  fastcgi_param QUERY_STRING $query_string;
  fastcgi_param CONTENT_TYPE $content_type;
  fastcgi_param CONTENT_LENGTH $content_length;
  fastcgi_param GATEWAY_INTERFACE CGI/1.1;
  fastcgi_param SERVER_SOFTWARE nginx/$nginx_version;
  fastcgi_param REMOTE_ADDR $remote_addr;
  fastcgi_param REMOTE_PORT $remote_port;
  fastcgi_param SERVER_ADDR $server_addr;
  fastcgi_param SERVER_PORT $server_port;
  fastcgi_param SERVER_NAME $server_name;
  fastcgi_param SERVER_PROTOCOL $server_protocol;
  fastcgi_param SCRIPT_FILENAME $fastcgi_script_name;
  fastcgi_param PATH_INFO $fastcgi_script_name;
  fastcgi_pass 127.0.0.1:9005;
 }

 location /webstatic/ {
  expires 7d;
  add_header Cache-Control public;
  alias /Users/suoning/probject/python_project/webstatic/trunk/;
 }

}

$

配置application

简单示例

from flup.server.fcgi import WSGIServer
from pan import app

WSGIServer(
 app,
 bindAddress=(host, port),
 maxThreads=threads
).run()

生产环境示例

#!/usr/bin/env python
# -*- coding: utf-8 -*-
author = 'suoning'


import sys
import argparse
from flup.server.fcgi import WSGIServer
from lib.daemon import Daemon
from pan import app

APP_NAME = 'pan_platform'
APP_INST_NAME = '20170501'

parser = argparse.ArgumentParser(description=u'Run an pan FastCGI server')
parser.add_argument('command', type=str,
     help=u'command [start|stop|restart]',
     choices=['start', 'stop', 'restart'])
parser.add_argument('-p', '--port', type=int,
     help=u'port of this server', required=True)
parser.add_argument('-t', '--threads', type=int, default=50,
     help=u'max number of threads')
parser.add_argument('-host', '--host', default='0.0.0.0',
     help=u'Listen to the main clause')


class panPlatformDaemon(Daemon):
 def run(self):
  # 运行服务
  try:
   WSGIServer(
    app,
    bindAddress=(args.host, args.port),
    maxThreads=args.threads,
    umask=0111
   ).run()
  except:
   sys.stderr.write('oops')


def gen_pidfile(port):
 return '/var/run/%s_%s_%d.pid' % (APP_NAME, APP_INST_NAME, port)


if name == 'main':
 args = parser.parse_args()
 daemon = panPlatformDaemon(gen_pidfile(args.port))
 if 'start' == args.command:
  daemon.start()
 elif 'stop' == args.command:
  daemon.stop()
 elif 'restart' == args.command:
  daemon.restart()
 else:
  print "Unknown command"
  sys.exit(2)
 sys.exit(0)

fastcgi协议和http协议在代码部署中的的优劣对比

• fastcgi虽然是二进制协议，相对于http协议，并不节省资源。二进制协议，只能节省数字的表达，比如 1234567，用字符串表示需要7个Byte，用数字就是4个Byte，而字符串到哪里都一样

• fastcgi在传输数据的时候，为了兼容cgi协议，还要带上一堆cgi的环境变量，所以和http协议相比，用fastcgi传输数据并不省，反而多一些

• fastcgi 唯一的优点是，它是长连接的，用户并发1000个request，fastcgi可能就用10个 链接转发给后端的appplication，如果用http协议，那来多少给多少，会向后端appplication 发起1000个请求

• http代理转发方式，在面对超高并发的情况下会出问题，因为，tcp协议栈当中，port是int16整型 你本地新建一个connect，需要消耗一个端口，最多能到65536。外部并发几十万个请求，port池耗干，你的服务器只能拒绝响应了

CGI, FCGI, SCGI, WSGI 区别

WIKI Links:

CGI - en.wikipedia.org/wiki/Common_Gateway_Interface
FCGI - en.wikipedia.org/wiki/Fcgi
SCGI - en.wikipedia.org/wiki/SCGI
WSGI - en.wikipedia.org/wiki/Wsgi 

Other reference:

helpful.knobs-dials.com/index.php/CGI%2C_FastCGI%2C_SCGI%2C_WSGI%2C_servlets_and_such#FastCGI_and_SCGI

CGI = Common Gateway Interface

顾名思义，它是一种接口规范。该规范详细定义了Web服务器中运行的服务器代理程序，怎样获取及返回网页生成过程中，服务器环境上下文和HTTP协议中的参数名称，如大家所熟知的：REQUEST_METHOD，QUERY_STRING，CONTENT_TYPE等等。绝大部分的Web服务器程序，是以脚本的形式代理接受并处理HTTP请求，返回HTTP页面或响应。这些脚本程序，就是大家所熟知的PHP、ASP、JSP等等。

FCGI = Fast CGI

It is actually a variant of CGI in specific implementation. The design idea is to achieve the ultimate goal of improving Web service performance by reducing the communication overhead between the CGI agent program and the Web host service program. It can be seen that FCGI is not different from CGI in terms of specifications, but there are improvements in the specific implementation method: CGI's approach is that for each HTTP request, the Web host service program creates a new process to call the server script, correspondingly Request; FCGI's approach is to establish an independent FCGI service program process to communicate with the Web host service program process. Once the FCGI service process is started, it allocates resources, creates threads to respond to HTTP requests, and determines its own life cycle , thus greatly reducing the resource overhead made by the system to create processes. Modern popular web server programs, such as PHP and ASP.Net, are basically implementations of FCGI.

SCGI = Simple CGI

It is the product of FCGI after streamlining the data protocol and response process. Its design purpose is to adapt to the increasing number of HTTP requests based on AJAX or REST and make faster and more concise responses. And SCGI agrees that when the server returns a response to an HTTP protocol request, the HTTP connection is immediately closed. Therefore, it is not difficult to see that SCGI is more suitable for the "request-forget" communication model advocated by SOA in a general sense.

WSGI = Web Server Gateway Interface

This protocol is a patent of the Python language, which defines a set of universally applicable interfaces for communication between the Web service host program and the HTTP response agent program. It came about because Python programmers noticed that there was a serious coupling between Web frameworks and Web hosting server programs. For example, some frameworks were designed for Apache's mod_python. Therefore, WSGI defines a set of very low-level interfaces. Common Python web frameworks have implemented this protocol: such as CherryPy, Django, web.py, web2py, TurboGears, Tornado, Pylons, BlueBream, Google App Engine[dubious – discuss], Trac, Flask, Pyramid ,etc.

The above is the detailed content of Introduction to the method of deploying web development in Python. For more information, please follow other related articles on the PHP Chinese website!