Using AWS Elastic File System (EFS) in Go: A Complete Guide

With the widespread application of cloud computing technology and containerized applications, more and more enterprises are beginning to migrate applications from traditional physical servers to cloud environments for deployment and operation. Using high-performance storage systems in cloud environments is a very important issue, and AWS Elastic File System (EFS) is a powerful distributed file system that can provide high availability, high performance, serverless and scalability.

EFS can access and share files in real time from multiple EC2 instances and automatically scales to meet capacity and performance needs. In this article, we will take a deep dive into how to use AWS Elastic File System (EFS) in Go.

Environment settings

Before using EFS, you first need to set up the correct environment. We need an AWS account, AWS SDK for Go, and Go locale.

Install AWS SDK for Go

You can install AWS SDK for Go through the following command:

$ go get github.com/aws/aws-sdk-go/aws
$ go get github.com/aws/aws-sdk-go/aws/session

In order to verify whether the AWS SDK is installed correctly, you can write the following test program:

package main

import (
    "fmt"

    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
)

func main() {
    // Specify the AWS Region to use.
    sess, err := session.NewSessionWithOptions(session.Options{
        Config: aws.Config{
            Region: aws.String("us-west-2"),
        },
    })

    if err != nil {
        fmt.Println(err)
        return
    }

    // Create an S3 service client.
    s3 := NewS3(sess)

    // Verify the client by listing all buckets
    buckets, err := s3.ListBuckets(nil)
    if err != nil {
        fmt.Println(err)
        return
    }

    fmt.Println("Buckets:")
    for _, b := range buckets.Buckets {
        fmt.Printf("* %s created on %s
",
            aws.StringValue(b.Name), aws.TimeValue(b.CreationDate))
    }
}

If all goes well, the output will contain a list of AWS S3 buckets.

Create EFS file system

Before using EFS, you need to create a file system. Create an EFS file system by following these steps:

1. Log in to the AWS console.
2. Select EFS (Elastic File System) in the service list.
3. Click the Create File System button.
4. In the Create File System page, select the VPC and subnet (you need to select one of the public subnets to enable EFS to connect all EC2 instances).
5. In the Security Group section, select a security group that should allow all inbound and outbound traffic from the EC2 instance.
6. In the File System and Performance Mode section, select the default option.
7. Click Create file system.

When you create a file system, the system will automatically create an EFS-specific security group for you to allow all data traffic from the VPC. You can override this option with your own security group rules.

Install the EFS driver

In order to integrate your Go application with EFS, you need to install the AWS EFS driver. On Amazon Linux or RHEL, you can install the EFS driver by following these steps:

1. Execute the following command to install the EFS driver dependencies:

$ sudo yum install gcc libstdc++-devel gcc-c++ fuse fuse-devel automake openssl-devel git

2. Download And build the EFS driver:

$ git clone https://github.com/aws-efs-utils/efs-utils
$ cd efs-utils
$ ./build-deb.sh

3. Install the EFS driver:

$ sudo yum install ./build/amazon-efs-utils*rpm

4. Confirm that the EFS driver is installed correctly. This can be verified with the following command:

$ sudo mount -t efs fs-XXXXX:/ /mnt/efs

where fs-XXXXX is the ID of your EFS file system. If there are no error messages in the output, the installation is successful.

Using EFS

After installing the EFS driver and creating the EFS file system, you can connect your Go application to the EFS file system. The following are some best practices for using EFS:

In programs, use the EFS file system through the operating system standard library. In the Go language environment, you can use the syscall package or the os package to connect to the EFS file system.

The following is a sample program fragment to connect to EFS:

package main

import (
    "fmt"
    "os"
    "syscall"
)

func main() {
    // Mount EFS.
    if err := syscall.Mount("fs-XXXXX.efs.us-west-2.amazonaws.com:/", "/mnt/efs", "nfs4", 0, "rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2"); err != nil {
        fmt.Println("Unable to mount EFS file system:", err)
        os.Exit(1)
    }

    // List the files in the EFS file system.
    if files, err := os.ReadDir("/mnt/efs"); err != nil {
        fmt.Println("Unable to read files in EFS:", err)
    } else {
        fmt.Println("Files in EFS:")
        for _, file := range files {
            fmt.Println(file.Name())
        }
    }

    // Unmount EFS when done.
    if err := syscall.Unmount("/mnt/efs", 0); err != nil {
        fmt.Println("Unable to unmount EFS file system:", err)
        os.Exit(1)
    }
}

In the above code fragment, we use the system call to mount the EFS file system and list the files in it. At the end of the program, we used a system call to unmount the file system.

Since EFS is a RESTful API, it supports all standard file system operations, such as creating, reading, writing, and deleting files. In the Go language environment, you can use the functions of the os package to perform these operations.

The following is a sample program to create a file on EFS:

package main

import (
    "fmt"
    "os"
)

func main() {
    // Create a new file in EFS.
    if file, err := os.Create("/mnt/efs/myfile.txt"); err != nil {
        fmt.Println("Unable to create file:", err)
        os.Exit(1)
    } else {
        defer file.Close()
        fmt.Println("File created successfully.")
    }
}

In the above example, we use the Create function in the os package to create a new file on the EFS file system. This file must be closed before the program ends.

Conclusion

AWS Elastic File System (EFS) is a powerful distributed file system that provides high availability, performance, serverless and scalability. In the Go language environment, you can use the syscall package or the os package in the standard library to connect to the EFS file system and use all its functions. Through the guidance of this article, you should already have some best practices for using EFS, so that you can fully utilize the functions of EFS in the Go language environment.

The above is the detailed content of Using AWS Elastic File System (EFS) in Go: A Complete Guide. For more information, please follow other related articles on the PHP Chinese website!