Example analysis of the usage of Deferred objects in Python's Twisted framework

Deferred object structure
Deferred consists of a series of paired callback chains, each pair contains a callback for handling success (callbacks) and a callback for handling errors (errbacks). Initially, deffereds will consist of two empty callback chains. When adding callbacks to them they will always be added in pairs. When the result of the asynchronous processing returns, the Deferred will start and trigger the callback chain in the order in which they were added.
It may be easier to illustrate with examples. First, let’s take a look at addCallback:

from twisted.internet.defer import Deferred
  
def myCallback(result):
  print result
  
d = Deferred()
d.addCallback(myCallback)
d.callback("Triggering callback.")

Running it will get the following results:

Triggering callback.

In the above example, a deffered is created and its addCallback method is used to register a callback for handling success. d.callback will start deffered and call the callback chain. The parameters passed into the callback will also be received by the first function in each callback chain.
There is addCallback, and I think I can guess that the other wrong branch is addErrorback. Let’s look at an example:

from twisted.internet.defer import Deferred
  
def myErrback(failure):
  print failure
  
d = Deferred()
d.addErrback(myErrback)
d.errback(ValueError("Triggering errback."))

Running it will get the following results:

[Failure instance: Traceback (failure with no frames): <type &#39;exceptions.ValueError&#39;>: Triggering errback.]

It can be seen that Twisted will encapsulate errors in Failure.
It is worth noting that registered callbacks are always paired as mentioned before. When using the d.addCallback and d.addErrorback methods, we seem to just add a callback or an errback. In fact, in order to complete the creation of this level of callback chain, these methods will also register a pass-through for the other half. Remember that callback chains are always the same length. If you want to specify the callback and errback of this level of callback respectively. You can use the d.addCallbacks method:

d = Deferred()
d.addCallbacks(myCallback, myErrback)
d.callback("Triggering callback.")

So... let’s stop here today.

Advanced example
The next step should be something more practical, that is, put in Reactor. Let’s look at an example first:

from twisted.internet import reactor, defer
  
class HeadlineRetriever(object):
  def processHeadline(self, headline):
    if len(headline) > 50:
      self.d.errback(Exception("The headline ``%s'' is too long!" % (headline,)))
    else:
      self.d.callback(headline)
  
  def _toHTML(self, result):
    return "<h1>%s</h1>" % (result,)
  
  def getHeadline(self, input):
    self.d = defer.Deferred()
    reactor.callLater(1, self.processHeadline, input)
    self.d.addCallback(self._toHTML)
    return self.d
  
def printData(result):
  print result
  reactor.stop()
  
def printError(failure):
  print failure
  reactor.stop()
  
h = HeadlineRetriever()
d = h.getHeadline("Breaking News: Twisted Takes us to the Moon!")
d.addCallbacks(printData, printError)
  
reactor.run()

The above example receives a title and processes it. If the title is too long, an overlong error will be returned, otherwise it will be converted into HTML and returned.

Since the given title is less than 50 characters, executing the above code will result in the following return:

<h1>Breaking News: Twisted Takes us to the Moon!</h1>

One thing worth noting is that reactor is used above. callLater method, which can be used to create timed events to simulate an asynchronous request.

If we make the title very long, for example:

h = HeadlineRetriever()
d = h.getHeadline("1234567890"*6)
d.addCallbacks(printData, printError)

The result is expected:

[Failure instance: Traceback (failure with no frames): <type &#39;exceptions.Exception&#39;>: The headline ``123456789012345678901234567890123456789012345678901234567890&#39;&#39; is too long!]

Let’s take a look at the picture Trigger process:

The key points in Deferreds
1. Deferreds will be triggered when its callback or errback is called;
2. Deferreds can only be triggered once! If you try to trigger multiple times, an AlreadyCalledError exception will result;
3. Exceptions in the N-level callback or errback will be passed to the N+1-level errback; if there is no errback, an Unhandled Error will be thrown. If the Nth level callback or errback does not throw an Exception or return a Failure object, it will be processed by the N+1th level callback;
4. The result returned in the callback will be passed to the next level callback and serve as its first parameter;
5. If the error passed into errback is not a Failure object, it will be automatically wrapped once.

For more examples analyzing the usage of Deferred objects in Python’s Twisted framework, please pay attention to the PHP Chinese website for related articles!