How to Download Files from the Web in Python 3?

Downloading Files from the Web in Python 3

Introduction

When creating programs that interact with the internet, often times it is necessary to download files from a web server. In Python 3, there are multiple ways to accomplish this task.

Python 3 Solution

The initially provided code encounters an error because the function expects a byte type for the URL argument, but the extracted URL from the JAD file is a string. To download a file when the URL is stored as a string, convert it to a byte type using the UTF-8 encoding:

<code class="python">import urllib.request

def downloadFile(URL=None):
    h = urllib.request.urlopen(URL.encode('utf-8'))
    return h.read()

downloadFile(URL_from_file)</code>

Alternative Solutions:

There are several alternative methods to download files from the web:

• urllib.request.urlopen: Obtain the contents of a web page by reading the response of urlopen:

  <code class="python">response = urllib.request.urlopen(URL)
  data = response.read() # a `bytes` object
  text = data.decode('utf-8') # a `str`</code>

• urllib.request.urlretrieve: Download and save a file locally:

  <code class="python">urllib.request.urlretrieve(URL, file_name)</code>

• urllib.request.urlopen shutil.copyfileobj: Offer a highly recommended and the most correct approach to file downloading:

  <code class="python">with urllib.request.urlopen(URL) as response, open(file_name, 'wb') as out_file:
    shutil.copyfileobj(response, out_file)</code>

• urllib.request.urlopen write to bytes object: Simpler option, but recommended only for small files:

  <code class="python">with urllib.request.urlopen(URL) as response, open(file_name, 'wb') as out_file:
    data = response.read() # a `bytes` object
    out_file.write(data)</code>

Compressed Data Handling

Finally, the extraction of compressed data on-the-fly is also possible:

<code class="python">url = 'http://example.com/something.gz'
with urllib.request.urlopen(url) as response:
    with gzip.GzipFile(fileobj=response) as uncompressed:
        file_header = uncompressed.read(64) # a `bytes` object</code>

The above is the detailed content of How to Download Files from the Web in Python 3?. For more information, please follow other related articles on the PHP Chinese website!