Why fastdfs needs to be combined with nginx

FastDFS is an open source distributed file system that manages files. Its functions include: file storage, file synchronization, file access (file upload, file download), etc., solving the problems of large-capacity storage and load balancing. It is especially suitable for online services based on files, such as photo album websites, video websites, etc.

When we use FastDFS to deploy a distributed file system, we use the FastDFS client API to upload, download, delete files, etc. operate. At the same time, HTTP services are provided through the FastDFS HTTP server. However, the HTTP service of FastDFS is relatively simple and cannot provide high-performance services such as load balancing. Therefore, the developer of FastDFS, Taobao architect Yu Qing, provides us with the FastDFS module used on Nginx (which can also be called Nginx of FastDFS). module). Its use is very simple.

FastDFS uses the Tracker server to store files on the Storage server, but servers in the same group need to copy files, which causes delays. Assume that the Tracker server uploads the file to 192.168.1.80, and the file The ID has been returned to the client. At this time, the background will copy the file to 192.168.1.30. If the copy is not completed, the client will use this ID to retrieve the file at 192.168.1.30, and an error will definitely occur. This fastdfs-nginx-module can redirect the connection to the source server to retrieve files, avoiding client errors due to replication delays.

FastDFS server has two roles: tracker and storage node. The tracker mainly does scheduling work and plays a load balancing role in access.

The storage node stores files and completes all functions of file management: storage, synchronization and providing access interfaces. FastDFS also manages the meta data of files. The so-called meta data of a file is the relevant attributes of the file, expressed in the form of key value pairs, such as: width=1024, where the key is width and the value is 1024. File meta data is a list of file attributes and can contain multiple key-value pairs.

FastDFS Principle

The storage nodes adopt a group method. A storage system consists of one or more groups. Files between groups are independent of each other. The sum of the file capacities of all groups is the file capacity of the entire storage system. A group can be composed of one or more storage servers. The files in the storage servers under a group are all the same. Multiple storage servers in the group play the role of redundant backup and load balancing (the storage capacity of a group It is the storage server with the smallest capacity in the group. Storage servers in different groups will not communicate with each other, but Storage servers in the same group will connect to each other for file synchronization).

When adding a server to a group, synchronization of existing files is automatically completed by the system. After synchronization is completed, the system automatically switches the newly added server to provide online services.

When the storage space is insufficient or about to be exhausted, groups can be added dynamically. Just add one or more servers and configure them as a new group, thus expanding the capacity of the storage system.

FastDFS has only two roles: Tracker server and Storage server. As the central node, the Tracker server's main function is load balancing and scheduling. The Tracker server records information such as grouping and Storage server status in the memory. It does not record file index information and takes up very little memory. In addition, when the client (application) and the Storage server access the Tracker server, the Tracker server scans the group and Storage server information in the memory and then gives a response. It can be seen from this that the Tracker server is very lightweight and will not become a system bottleneck.

The Storage server in FastDFS is usually called Trunk server or Data server in other file systems. Storage server directly uses the OS file system to store files. FastDFS does not store files in blocks. The files uploaded by the client correspond to the files on the Storage server one-to-one (the file identification in FastDFS is divided into two parts: the group name and the file name, both of which are indispensable)

For more Nginx related technical articles, please visit the Nginx usage tutorial column to learn!

The above is the detailed content of Why fastdfs needs to be combined with nginx. For more information, please follow other related articles on the PHP Chinese website!