This article is an idea that I came up with during my recent study of Node.js, and I would like to discuss it with everyone.
HTTP server for Node.js
It is very easy to implement an http service using Node.js. The simplest example is as shown on the official website:
1. Due to the single-threaded nature of Node.js, its robustness guarantee has relatively high requirements for developers.
2. There may be other http services on the server that have occupied port 80, and web services other than port 80 are obviously not user-friendly enough.
3.Node.js does not have much advantage in file IO processing. For example, as a regular website, it may need to respond to image and other file resources at the same time.
4. Distributed load scenario is also a challenge.
Therefore, using Node.js as a web service is more likely to be used as a game server interface and other similar scenarios. Most of them deal with services that do not require direct access by users and are only used for data exchange.
Node.js web service based on Nginx as front-end machine
Based on the above reasons, if it is a website-shaped product built using Node.js, the conventional usage is to place another mature http server on the front end of the Node.js web service, such as Nginx, which is most commonly used.
Then use Nginx as a reverse proxy to access the Node.js based web service. Such as:
location / {
proxy_pass http://127.0.0.1:1337;
}
location ~ .(gif|jpg|png|swf|ico|css|js)$ {
root /home/andy/wwwroot/yekai/static;
}
}
This will better solve the problems raised above.
Communicate using FastCGI protocol
However, the above-mentioned proxy method also has some disadvantages.
One possible scenario is that you need to control direct http access to the underlying Node.js web service. However, if you want to solve it, you can also use your own service or rely on firewall blocking.
The other reason is that the proxy method is a solution on the network application layer after all, and it is not very convenient to directly obtain and process the data that interacts with the client http, such as the processing of keep-alive, trunk and even cookies. Of course, this is also related to the capabilities and functional perfection of the proxy server itself.
So, I was thinking of trying another processing method. The first thing that came to mind was the FastCGI method that is now commonly used in php web applications.
What is FastCGI
Fast Common Gateway Interface/FastCGI is a protocol that allows interactive programs to communicate with web servers.
The background of FastCGI is to serve as an alternative to cgi web applications. One of the most obvious features is that a FastCGI service process can be used to handle a series of requests. The web server will pass the environment variables and the page request through a socket, such as The FastCGI process connects to the web server via a Unix Domain Socket or a TCP/IP connection. For more background knowledge, please refer to the Wikipedia entry.
FastCGI implementation for Node.js
Theoretically, we only need to use Node.js to create a FastCGI process, and then specify Nginx’s monitoring request to be sent to this process. Since both Nginx and Node.js are based on event-driven service models, "theoretically" they should be a natural solution. Let’s implement it ourselves below.
The net module in Node.js can be used to create a socket service. For convenience, we choose the unix socket method.
Slightly modify the configuration on the Nginx side:
var server = net.createServer();
server.listen('/tmp/node_fcgi.sock');
server.on('connection', function(sock){
console.log('connection');
sock.on('data', function(data){
console.log(data);
});
});
Then run (due to permission reasons, please ensure that the Nginx and node scripts are run by the same user or accounts with mutual permissions, otherwise you will encounter permission problems when reading and writing sock files):
node node_fcgi.js
When accessed in the browser, we see that the terminal running the node script receives the data content normally, such as this:
This proves that our theoretical foundation has achieved the first step. Next, we only need to figure out how to parse the content of this buffer.
FastCGI Protocol Basics
FastCGI records consist of a fixed-length prefix followed by a variable amount of content and padding bytes. The record structure is as follows:
version: FastCGI protocol version, now the default is 1 In the case of multiplexing and concurrency, just use 1 here
contentLength: content length, the maximum length here is 65535
paddingLength: padding length, the function is to fill long data to an integer multiple of 8 bytes, mainly Used to process data that remains aligned more efficiently, mainly due to performance considerations
reserved: reserved bytes, for subsequent expansion
contentData: real content data, which will be discussed in detail later
paddingData: padding data, anyway They are all 0, just ignore them.
For specific structure and description, please refer to the official website document (http://www.fastcgi.com/devkit/doc/fcgi-spec.html#S3.3).
Request section
It seems very simple, just parse it once and get the data. However, there is a pitfall here, that is, what is defined here is the structure of the data unit (record), not the structure of the entire buffer. The entire buffer is composed of one record and one record. It may not be easy to understand at first for those of us who are used to front-end development, but this is the basis for understanding the FastCGI protocol, and we will see more examples later.
var body = contentLength ? data.slice(end, contentLength) : null;
rcds.push([type, body, requestId]);
return arguments.callee();
}
}
//使用
sock.on('data', function(data){
getRcds(data, function(rcds){
})();
}
注意这里只是简单处理，如果有上传文件等复杂情况这个函数不适应，为了最简演示就先简便处理了。同时，也忽略了requestId参数，如果是多路复用的情况下不能忽略，并且处理会需要复杂得多。
接下来就可以根据type来对不同的记录进行处理了。type的定义如下：
接下来就可以根据记录的type来解析拿到真正的数据，下面我只拿最常用的FCGI_PARAMS、FCGI_GET_VALUES、FCGI_GET_VALUES_RESULT来说明，好在他们的解析方式是一致的。其他type记录的解析有自己不同的规则，可以参考规范的定义实现，我这里就不细说了。
FCGI_PARAMS、FCGI_GET_VALUES、FCGI_GET_VALUES_RESULT都是“编码名-值”类型数据，标准格式为：以名字长度，后跟值的长度，后跟名字，后跟值的形式传送，其中127字节或更少的长度能在一字节中编码，而更长的长度总是在四字节中编码。长度的第一字节的高位指示长度的编码方式。高位为0意味着一个字节的编码方式，1意味着四字节的编码方式。看个综合的例子，比如长名短值的情况：
对应的实现js方法示例：
if(body[j] >> 7 == 1){
valueLength = ((body[j ] & 0x7f) << 24) (body[j ] << 16) (body[j ] << 8) body[j ];
} else {
valueLength = body[j ];
}
var ret = body.asciiSlice(j, j nameLength valueLength);
name = ret.substring(0, nameLength);
value = ret.substring(nameLength);
params[name] = value;
j = (nameLength valueLength);
}
return params;
}
这样就实现了一个简单可获取各种参数和环境变量的方法。完善前面的代码，演示我们如何获取客户端ip：
Now we have understood the basics of the FastCGI request part. Next, we will implement the response part and finally complete a simple echo response service.
Response section
The response part is relatively simple. In the simplest case, you only need to send two records, which are FCGI_STDOUT and FCGI_END_REQUEST.
I won’t go into details about the specific content of the recorded entity, just look at the code:
function buffer0(len){
return new Buffer((new Array(len 1)).join('u0000'));
};
function writeStdout(data){
var rcdStdoutHd = new Buffer(8),
contendLength = data.length,
paddingLength = 8 - contendLength % 8;
rcdStdoutHd[0] = 1;
rcdStdoutHd[1] = TYPES.FCGI_STDOUT;
rcdStdoutHd[2] = 0;
rcdStdoutHd[3] = 1;
rcdStdoutHd[4] = contendLength >> 8;
rcdStdoutHd[5] = contendLength;
rcdStdoutHd[6] = paddingLength;
rcdStdoutHd[7] = 0;
return Buffer.concat([rcdStdoutHd, data, buffer0(paddingLength)]);
};
function writeHttpHead(){
return writeStdout(new Buffer("HTTP/1.1 200 OKrnContent-Type:text/html; charset=utf-8rnConnection: closernrn"));
}
function writeHttpBody(bodyStr){
var bodyBuffer = [],
body = new Buffer(bodyStr);
for(var i = 0, l = body.length; i < l; i = MaxLength 1){
function writeEnd(){
var rcdEndHd = new Buffer(8);
rcdEndHd[0] = 1;
rc dEndHd[2] = 0;
rcdEndHd[3] = 1;
rcdEndHd[4] = 0;
rcdEndHd[5] = 8;
rcdEndHd[6] = 0;
rcd EndHd[7] = 0;
return Buffer.concat([rcdEndHd, buffer0(8)]);
}
return function(data){
return Buffer.concat([writeHttpHead(), writeHttpBody(data), writeEnd()]);
};
In the simplest case, this allows a complete response to be sent. Modify our final code:
Copy the code
Comparison test
Finally, the question we need to consider is whether this solution is feasible? Some students may have noticed the problem, so I will post the simple stress test results first:
500 clients, running 20 sec.
Speed=22131 pages/min, 63359 bytes/sec.
Requests: 6523 susceed, 854 failed.
//Proxy mode:
500 clients, running 10 sec.
Speed=28752 pages/min, 73191 bytes/sec.
Requests: 3724 susceed, 1068 failed.
500 clients, running 20 sec.
Speed=26508 pages/min, 66267 bytes/sec.
Requests: 6716 susceed, 2120 failed.
//Direct access to Node.js service method:
500 clients, running 10 sec.
Speed=101154 pages/min, 264247 bytes/sec.
Requests: 15729 susceed, 1130 failed.
500 clients, running 20 sec.
Speed=43791 pages/min, 115962 bytes/sec.
Requests: 13898 susceed, 699 failed.
Postscript
If you are interested in continuing to play, you can check the source code of the example I implemented in this article. I have studied the protocol specifications in the past two days, and it is not difficult.
At the same time, I am planning to play uWSGI again, but the official said that v8 is already preparing to directly support it.
The game is very simple, please correct me if there are any mistakes.
![[Web front-end] Node.js quick start](https://img.php.cn/upload/course/000/000/067/662b5d34ba7c0227.png)
