Detailed explanation of the implementation process of multiple picture/file servers

Image/file server, as the name suggests, is used to store files. Some people use Alibaba Cloud's existing services, and some post files to the file server, and use an application on the file server to receive and save them. The methods are different. same. Let’s take a look at it with the editor.

Written in front

Image/file server, as the name suggests, is used to store files. Some people use Alibaba Cloud’s existing Services, some post files to a file server, and use an application on the file server to receive and save them. The methods vary. Veteran drivers are already playing bad games on various servers. Several problems when using multiple image/file servers are:

1. How to post files uploaded by the web to the file server, how fast it is, how much bandwidth it takes and how time-consuming it is.

2. How to distribute files evenly on each image server.

3. How to expand the new image server and ensure that the impact on the distributed content is minimized.

In sharing this article, I hope experienced drivers can give me some opinions, and on the other hand, I want to share it with new drivers and record their own journeys.

Determine the final goal in the first minute

1. Each file server not only provides storage functions, but also has external service capabilities to reduce the bandwidth usage and consumption of each Web server. Disk IO consumption.

2. Provide simple and easy horizontal expansion capabilities

3.AsyncAsynchronousIO ensures that threads are non-blocking.

4. The file reaches the image server quickly

Deny yourself in the third minute

1. Deny Take the Http Post image to Image server method

2. Deny the use of Hashmodulo algorithm

3. Deny the use of consistent hashing to avoid file movement after horizontal expansion of the server, even a small amount

Determine the implementation method in the fifth minute

1. Share the file server disk to a folder as a "network disk" between specific users, that is to say, the share Only the web server has permission to access the folder. You can refer to how to share a folder within the local area network or Alibaba Cloud shared folder.

2. Through the IIS virtual directory, point its physical path to the network disk. You know that shared folders use Tcp/Ip protocol and Netbios protocol to communicate.

3. Configure WebConfig to access the network disk identity (user name and password) - must be configured

4. The file shared in the first step is deployed on the file server in one of the WebApps under IIS file to expose the URL path. Because you know that IIS cannot obtain any content in the parent folder except by using file reading.

4. Determine the method of evenly allocating picture positions according to file names - the file suffix name method. The suffix name uses "file name-01" (server No. 1), and the suffix name uses "file name-02" (server No. 2). Server number), the random number is obtained using Random. Whatever the random number is, we have the server number. After multiple tests of 1 million random times, 1, 2, and 3 were obtained about 330,000 times respectively, ensuring balance. When expanding in the future, just open a few more Random numbers and there is no need to move files or reshash. And after horizontally expanding the number of servers, we hope to ensure that the three servers will no longer store files, and the Random range is adjustable. We have already saved the file name in the database or cache, and it has a suffix name. When returning the image URL to the front desk, we only need to spell out the file server IP or domain name based on the suffix name.

Follow the steps in the seventh minute

1. Deploy FileServerApp, its function is to expose the URL to the image, it is actually very simple, set up An Asp.Net empty project creates a folder to store pictures. And share the folder with the web server. See the deployment diagram in the last section for details.

2. Assume that you have implemented folder sharing

3. Create a new virtual directory under the IIS site. Note that the physical path points to the network Network path of the shared folder

##4. Under Webconfig

Add , IIS sets authentication for the virtual directory as the application pool identity.

5. Use

Server.MapPath to obtain the network path of FileServer2, and use your file reading and writing methods to read and write files in the directory.

6. It is recommended to use the Async method for writing files. Asynchronous IO will return to handle other things after you send the IO request packet to the network driver or disk driver. At some point in the future, the file is written successfully, the State machine restores your context and continues execution in a synchronous manner, which can greatly improve the throughput of your Web server.

Write the key points at the end

If there is still something vague, read here and you will understand it all

【Related recommendations】

1.ASP free video tutorial

2.ASP tutorial

3. Li Yanhui ASP basic video tutorial

The above is the detailed content of Detailed explanation of the implementation process of multiple picture/file servers. For more information, please follow other related articles on the PHP Chinese website!