Talking to a Gmail POPerver with Python

POP is a relatively old protocol. The first version was specified in 1984. The version still in use today, POP3, was specified in 1996. To try it out, I went about connecting to a Gmail POP3 server.

The first step was looking up the POP3 settings -- what server to connect to, on what port. Google led me here, where I found the following information.

pop.gmail.com

Requires SSL: Yes

Port: 995

It mentions that SSL is required. This wasn't something I dealt with 25 years ago, when I was last messing around with POP. I was afraid it would be a headache, but it turned out to be no challenge whatsoever; with a little help from the Python docs, I arrived at this code.

import socket
import ssl

hostname = 'pop.gmail.com'
context = ssl.create_default_context()

with socket.create_connection((hostname, 995)) as sock:
    with context.wrap_socket(sock, server_hostname=hostname) as s:
        print(s.version())

It connects, and tells me what version of SSL is in use... or something. Great success! Time to start a conversation with the server.

Borrowing from the official RFC for POP3, here's an example POP3 conversation between a client and server/

C: <open connection>
S:    +OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
C:    USER mrose
S:    +OK mrose is a real hoopy frood
C:    PASS secret
S:    +OK mrose's maildrop has 2 messages (320 octets)
C:    STAT
S:    +OK 2 320
C:    LIST
S:    +OK 2 messages (320 octets)
S:    1 120
S:    2 200
S:    .
C:    RETR 1
S:    +OK 120 octets
S:    <the POP3 server sends message 1>
S:    .
C:    QUIT
S:    +OK dewey POP3 server signing off (maildrop empty)
C:  <close connection>

The first thing that happens is that the server sends a greeting to the client. Friendly. So I'll add code to receive a message from the server.

When you ask to receive data from a socket, you have to specify a buffer size. The docs recommend a power of 2, such as 4096. Many responses from the server will come through all at once. Some won't; sometimes a message from the server will be broken across sever reads, and the buffer may not be filled to its maximum even if there is more to come.

In the case of POP3, the way to tell if a message has come in completely depends on which message is coming in. Most of the time the server is sending a single line of text. (As we will see again later, these have a carriage return and line feed characters at the end of each line.) Certain messages that could have a much longer response use another way to show they're done: a period on a single line by itself.

import socket
import ssl

hostname = 'pop.gmail.com'
context = ssl.create_default_context()

with socket.create_connection((hostname, 995)) as sock:
    with context.wrap_socket(sock, server_hostname=hostname) as s:
        print(s.version())
        data = s.read(4096)
        print(data)

Run again and, we get a greeting. Another great success! Notice that the line ends with "rn" -- carriage return and line feed characters.

You have to pass a buffer size to the read method. It will then have a buffer that size available to read data from the server -- but there's no guarantee of how much data will come into the buffer at a time. This means that a protocol needs some way of specifying when a message is complete. There are numerous strategies possible. POP uses two: for all messages, lines are ended with rn. For a short (one line) message, this is all that's required. For multi-line responses, a period on a line by itself indicates that the message is complete.

TLSv1.3
b'+OK Gpop ready for requests from 2601:1c0:8301:b590:f408:d66a:3029:16ad dq2mb54750689ivb\r\n'

Now we need to start talking back to the server. Time to create an I/O (or O/I) loop; get some user input and send it to the server. Oops! I can't send a string directly; that gives me a TypeError. I need to convert the message to bytes. The string encode() method will do that (the default encoding of utf-8 works fine).

Only, when I run it -- oops again! Nothing happens when my message is sent to the server. Because I forgot that messages coming from the client also need to end with rn. Another tiny tweak gives us:

import socket
import ssl

hostname = 'pop.gmail.com'
context = ssl.create_default_context()

with socket.create_connection((hostname, 995)) as sock:
    with context.wrap_socket(sock, server_hostname=hostname) as s:
        print(s.version())
        while True:
            data = s.read(4096)
            print(data)
            msg = input() + "\r\n"
            s.send(msg.encode())

Great, now I can actually try to log in!

TLSv1.3
b'+OK Gpop ready for requests from 2601:1c0:8301:b590:f408:d66a:3029:16ad g4mb5147337iow\r\n'
USER grokprogramming
b'+OK send PASS\r\n'
PASS trustno1
b'-ERR [AUTH] Application-specific password required: https://support.google.com/accounts/answer/185833\r\n'

OK, so following that link takes me to a page where I can set up an application specific password. One potential stumbling block I came across: your account has to have two-factor authentication turned on in order for you to be able to make an application specific password, as far as I can tell. Why would I not have two-factor authentication turned on in the year of our Lorde 2024? I can't say. I do now.

Armed with an application specific password (mind you take out the spaces), I can log in! Then I'll issue the STAT command which will tell me how many messages I have, and their combined size. After that, I'll issue the LIST command, which will return a list of messages with an ID and a size for each one.

TLSv1.3
b'+OK Gpop ready for requests from 2601:1c0:8301:b590:f408:d66a:3029:16ad e18mb76868856iow\r\n'
USER grokprogramming
b'+OK send PASS\r\n'
PASS baygdsgkmihkckrb
b'+OK Welcome.\r\n'
STAT
b'+OK 263 14191565\r\n'
LIST
b'+OK 263 messages (14191565 bytes)\r\n1 2778\r\n2 2947\r\n3 6558\r\n4 9864\r\n5 35997\r\n6 45462\r\n7 45462\r\n8 63894\r\n9 11487\r\n10 74936\r\n11 74925\r\n12 11632\r\n13 32392\r\n14 74997\r\n15 51961\r\n16 15375\r\n17 46513\r\n18 21519\r\n19 15966\r\n20 27258\r\n21 28503\r\n22 35615\r\n23 86353\r\n24 280'

I've hit a bug in the code. The response for LIST spans multiple lines and, in this case, will require multiple buffer reads. The whole message will end with a period on a line by itself. Here I've received one buffer worth of the message, and now I'll have to hit return and send a blank message to the server in order for the code to advance to the next iteration of the loop and read from the buffer again.

I'll tweak the code so that the user always has the option of reading from the buffer again, or not. I'll also finally decode the incoming bytes from the server, so that the text renders nicer.

import socket
import ssl

hostname = 'pop.gmail.com'
context = ssl.create_default_context()

with socket.create_connection((hostname, 995)) as sock:
    with context.wrap_socket(sock, server_hostname=hostname) as s:
        print(s.version())
        while True:
            data = s.read(4096)
            print(data.decode())
            while input("more? y/[n]: ") == "y":
                data = s.read(4096)
                print(data.decode())
            msg = input("> ") + "\r\n"
            s.send(msg.encode())

And here's a full session including retrieving an email and sending the disconnect message.

> USER grokprogramming
+OK send PASS

more? y/[n]: 
> PASS trustno1
+OK Welcome.

more? y/[n]: 
> STAT
+OK 263 14191565

more? y/[n]: 
> LIST
+OK 263 messages (14191565 bytes)
1 2778
2 2947
3 6558
<...>
260 41300
261 114059
262 174321
263 39206
.

more? y/[n]: 
> RETR 1
+OK message follows
MIME-Version: 1.0
Received: by 10.76.81.230; Thu, 28 Jun 2012 20:21:50 -0700 (PDT)
Date: Thu, 28 Jun 2012 20:21:50 -0700
Message-ID: <CADBp03TWFOKcTOaK_0P7VV2GB+TZsoSd_W4G5nZKKs7pdk6cWQ@mail.gmail.com>
Subject: Customize Gmail with colors and themes
From: Gmail Team <mail-noreply@google.com>
To: Grok Programming <grokprogramming@gmail.com>
Content-Type: multipart/alternative; boundary=e0cb4e385592f8025004c393f2b4

--e0cb4e385592f8025004c393f2b4
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

To spice up your inbox with colors and themes, check out the Themes tab
under Settings.
       Customize Gmail =BB <https://mail.google.com/mail/#settings/themes>

Enjoy!

- The Gmail Team
[image: Themes thumbnails]

Please note that Themes are not available if you're using Internet Explorer
6.0. To take advantage of the latest Gmail features, please upgrade to a
fully supported
browser<http://support.google.com/mail/bin/answer.py?answer=3D6557&hl=3Den&=
utm_source=3Dwel-eml&utm_medium=3Deml&utm_campaign=3Den>
..

--e0cb4e385592f8025004c393f2b4
Content-Type: text/html; charset=ISO-8859-1

more? y/[n]: y

<html>
<font face="Arial, Helvetica, sans-serif">
<p>To spice up your inbox with colors and themes, check out the Themes tab
under Settings.</p>

<table cellpadding="0" cellspacing="0">
  <col style="width: 1px;"/>
  <col/>
  <col style="width: 1px;"/>
  <tr>
    <td></td>
    <td height="1px" style="background-color: #ddd"></td>
    <td></td>
  </tr>
  <tr>
    <td style="background-color: #ddd"></td>
    <td background="https://mail.google.com/mail/images/welcome-button-background.png"
        style="background-color: #ddd; background-repeat: repeat-x;
            padding: 10px; font-size: larger">
          <a href="https://mail.google.com/mail/#settings/themes"
            style="font-weight: bold; color: #000; text-decoration: none;
            display: block;">
      Customize Gmail &#187;</a>
    </td>
    <td style="ba
more? y/[n]: y
ckground-color: #ddd"></td>
  </tr>
 <tr>
    <td></td>
    <td height="1px" style="background-color: #ddd"></td>
    <td></td>
  </tr>
</table>

<p>Enjoy!</p>

<p>- The Gmail Team</p>

<img width="398" height="256" src="https://mail.google.com/mail/images/gmail_themes_2.png" alt="Themes thumbnails" />

<p><font size="-2" color="#999">Please note that Themes are not available if
you're using Internet Explorer 6.0. To take advantage of the latest Gmail
features, please
<a href="http://support.google.com/mail/bin/answer.py?answer=6557&hl=en&utm_source=wel-eml&utm_medium=eml&utm_campaign=en"><font color="#999">
upgrade to a fully supported browser</font></a>.</font></p>

</font>
</html>

--e0cb4e385592f8025004c393f2b4--
.

more? y/[n]: 
> QUIT
+OK Farewell.

more? y/[n]: 
> 

Yet another great success! I was able to log in to the POP3 server and retrieve a message. The script in its current state is pretty flexible, but it requires a lot of work from the user. I'll make a few final tweaks to make interacting with the POP3 server a little easier: if the user starts a message to the server with a "!" it will be stripped out, but the script will read in data from the server until it gets to a period on a line by itself -- in other words, for commands with long responses. No "!" and the script will read in a single line, looking for the \r\n characters.

import socket
import ssl

hostname = 'pop.gmail.com'
context = ssl.create_default_context()

def read_until(s, eom):
    # read into the buffer at least once
    data = s.read(4096)
    # continue reading until end of message
    while data[-len(eom):] != eom:
        data += s.read(4096)
    # return incoming bytes decoded to a string
    return data.decode()

def read_single_line(s):
    return read_until(s, b"\r\n")

def read_muli_line(s):
    return read_until(s, b"\r\n.\r\n")

with socket.create_connection((hostname, 995)) as sock:
    with context.wrap_socket(sock, server_hostname=hostname) as s:
        print(s.version())
        print(read_single_line(s))
        msg = input("> ")
        # empty msg will close connection
        while msg != "":
            if msg[0] == "!":
                msg = msg[1:]
                long = True
            else:
                long = False
            msg += "\r\n"
            s.send(msg.encode())
            if long:
                print(read_muli_line(s))
            else:
                print(read_single_line(s))
            msg = input("> ")
        s.close()

The above is the detailed content of Talking to a Gmail POPerver with Python. For more information, please follow other related articles on the PHP Chinese website!