How To Get an API Data and Store in AWS S3

This tutorial demonstrates how to retrieve data from the OpenWeather API using Python and store it in AWS S3. This straightforward method allows you to fetch and cloud-store API data for later use. Even if you're new to this, the steps are clearly outlined. For a different approach using React, see our article on fetching API data with React.

What You Will Learn:

This tutorial covers:

• Retrieving weather data from the OpenWeather API with Python.
• Setting up an S3 bucket for data storage.
• Uploading the fetched data to AWS S3.

Prerequisites:

Before starting, ensure you have:

• An AWS account (sign up here).
• A GitHub repository for your code (sign up here).
• A code editor (VS Code is recommended).

Step 1: Creating an AWS S3 Bucket

To store your data, create an S3 bucket:

1. Log in to your AWS account.
2. Search for "S3".
3. Click "Create bucket" and follow the instructions.
4. Choose a unique bucket name (e.g., my-weather-data).
5. Select a region.
6. Click "Create".

Step 2: Fetching Data from the OpenWeather API

Create an OpenWeather account.

Obtaining Your API Key:

1. Sign Up: Register on the OpenWeather website. Your API key will be available on the next page.
2. Locate Your API Key: The API key is usually found under an "API Key" tab.
3. Alternative Access: You can also find it in your profile settings.

Install the requests library:

<code class="language-bash">pip install requests</code>

Fetch weather data:

<code class="language-python">import requests
import json

api_key = 'YOUR_API_KEY'  # Replace with your key
city = 'London'

def get_weather_data():
    url = f'http://api.openweathermap.org/data/2.5/weather?q={city}&appid={api_key}'
    response = requests.get(url)
    return response.json()

weather_data = get_weather_data()
print(weather_data)</code>

This script retrieves current weather data for the specified city.

Step 3: Setting Up the AWS SDK for Python (Boto3)

Install Boto3:

<code class="language-bash">pip install boto3</code>

Configure your AWS credentials following the configuration guide. You'll need your Access Key ID and Secret Access Key.

Step 4: Uploading Data to AWS S3

Set up the S3 client:

<code class="language-python">import boto3

aws_access_key_id = 'YOUR_ACCESS_KEY'  # Replace
aws_secret_access_key = 'YOUR_SECRET_KEY'  # Replace
region_name = 'eu-west-2'  # Replace with your region

s3 = boto3.client('s3', aws_access_key_id=aws_access_key_id,
                   aws_secret_access_key=aws_secret_access_key,
                   region_name=region_name)</code>

Upload the data:

<code class="language-python">def upload_to_s3(data):
    bucket_name = 'my-weather-data'  # Replace with your bucket name
    file_name = 'weather_data.json'
    s3.put_object(Bucket=bucket_name, Key=file_name,
                  Body=json.dumps(data), ContentType='application/json')
    print('Upload successful!')

upload_to_s3(weather_data)</code>

How it Works:

The script uses requests to fetch JSON data and boto3 to upload it to your S3 bucket as weather_data.json.

Step 5: Verifying the Upload

Check your S3 Management Console to confirm the weather_data.json file is in your bucket.

Conclusion:

This tutorial showed how to fetch and store weather data from OpenWeather API in AWS S3 using Python. This is a valuable technique for managing and accessing API data in the cloud.

The above is the detailed content of How To Get an API Data and Store in AWS S3. For more information, please follow other related articles on the PHP Chinese website!