Detailed introduction to http implementation in NODEJS

This article mainly introduces the technical process and detailed analysis of the http implementation of NODEJS. Friends who need it can refer to it.

1. Foreword

At present, HTTP protocol is the most widely used network protocol on the Internet, and it is also the one that front-end ER has the most contact with. kind of agreement. By reading the implementation of the http module in nodejs, you can have a deeper understanding of the HTTP protocol. The HTTP protocol is an application layer protocol based on the TCP protocol, and its implementation is inseparable from the TCP/IP protocol family. As for code implementation, the http module depends on the net module.

As shown in the figure below: In nodejs, http transmits data through the net module, and after obtaining the data, it relies on HTTP_PARSER to parse the data.

2. Source code

Start an HTTP service

Starting an HTTP service in nodejs is very simple, just instantiate one Server object, and listens to a certain port:

const Server = require('./libs/http').Server
const server = new Server( function(req, res) { 
 res.writeHead(200)
 res.end('hello world')
})
server.listen(9999)

SERVER class

The Server class inherits from net.Server and listens to the 'connection' event.

In the Server class, two main things are done: 1. Initialize the NET module and establish TCP network monitoring 2. Monitor its own request event

When the client request comes, Server The instance will first listen to the 'connection' event, establish a TCP connection and expose the socket object in the connectionListener. Next, the HTTP module interacts with the client through the socket object.

When a request arrives, the Server will trigger its own request event and call the requestListener method, which is the callback function passed in when creating the Server instance.

new Server( function(req, res) { 
 res.writeHead(200)
 res.end('hello world')
})

Note: The socket object is similar to an implementation of the TCP protocol, through which data can be exchanged with the client. Note: In the connectionListener function, the parser instance is also initialized and an onIncoming function is bound to it. HTTP Parser
The entire parsing process is carried out in connectionListener. The socket obtains the data pushed by TCP through the 'data' event.

When the socket obtains the data, it will first parse the data, that is: parser.excute (), the parsing tool is parser. It is worth mentioning that in order to reuse parser, the author obtained it from a 'FreeList pool'.

...
const parser = parsers.alloc() 
...
connectionListener（socket） { 
  socket.on('data', socketOnData)

  // TCP推入数据，parser进行解析
  function socketOnData(d) {
    ...
    const ret = parser.execute(d)
    ...
  }
}

1. When TCP data arrives, execute() first

2. Following the clues, we find that parser.excute is Excute (node_http_parser.cc). Excute is just an outsourcing, and the specific work is done by http_parser_excute (http_parser.c).

node_http_parser.cc is just a layer of packaging for http_parser.c. http_parser.c relies on the 7 externally exposed callback periodic functions to interact with node_http_parser.cc for data.

3. http_parser.c has only two types of callbacks: HTTP_CB and HTTP_DATA_CB. Through overloading, 8 periodic functions are registered in these two types of functions, as shown below:

4. Although http_parser registers 8 callback functions, node_http_parser.cc only exposes four periods to the outside world. Function:

parserOnHeaders

parserOnHeadersComplete

parserOnBody

parserOnMessageComplete

5. When http_parser.c parses to on_headers_complete, execute HTTP_CB( on_headers_complete) callback function, as shown in the figure:

The kOnHeadersComplete callback function will be executed within the function, that is: parserOnHeadersComplete function (common.js)

6. At this time, the request header parsing is basically completed, and then create An instance of IncomingMessage, and then wrap the request header data into the instance.
Execute the onIncoming callback function and pass the obtained IncomingMessage instance as a parameter.

function parserOnHeadersComplete (versionMajor, versionMinor, headers, method, url, statusCode, statusMessage, upgrade, shouldKeepAlive) { 
  ...
  parser.incoming = new IncomingMessage(parser.socket)
  parser.incoming.httpVersionMajor = versionMajor
  parser.incoming.httpVersionMinor = versionMinor
  parser.incoming.httpVersion = versionMajor + '.' + versionMinor
  parser.incoming.url = url
  ...
  skipBody = parser.onIncoming(parser.incoming, shouldKeepAlive)

}

7. In parserOnIncoming, create a ServerResponse instance.

With two instances of req and res, the request event monitored by the server is triggered.

When Server is instantiated, requestListener is used as a function parameter to monitor the request event.

8. Back to when the Server was created:

const server = new Server( function(req, res) { 
  var data = ''
  req.on('data', function(chunk){
    console.log('chunk: ' + chunk)
    data += chunk;
  })
  res.writeHead(200)
  res.end('hello world')
})

To sum up, after http_parser parses the header, the request event will be triggered.

Where should the body data be placed? In fact, the body data will be placed in the stream until the user uses the data event to receive the data. In other words, when the request is triggered, the body will not be parsed.

3. Process sorting

The complete http request is as follows: - The client initiates an HTTP request and first triggers the connection event on the server side. Establish a TCP link.

After receiving the connection event, the Server establishes a TCP connection, exposes the socket, and listens to the 'data' event through the socket; it initializes http-parser to prepare for subsequent parsing of data.

The HTTP request data reaches the server, and the parser executes the execute method to parse. After the request header is successfully parsed, the request event is triggered through a callback.

At this point, we have received the request for this http request in the Server callback function

4. Conclusion

Since many of the underlying libraries of nodejs are written in C/C, it is very inconvenient during the reading and debugging process. When I read the source code myself, I only focused on the JS part of the source code. For example, TCP's three-way handshake and four-way wave are not delved into its implementation details. The above analysis does not involve the analysis of http-body. For network requests with body, the actual situation is more complicated, and some details are not fully understood. When I summarize and share next time, I will try my best to fill in all the missing details.

The above is what I compiled for everyone. I hope it will be helpful to everyone in the future.

Related articles:

How to obtain node elements using JS

How to implement WebSocket function using NodeJS

About the actual usage of log4js in Express

The above is the detailed content of Detailed introduction to http implementation in NODEJS. For more information, please follow other related articles on the PHP Chinese website!