Bruce Lee Will Block You!
Recently in the Twitterverse, I happened to notice a lot of linkshare regarding the blog post “Rabbits and Warrens”. And rightfully so, the author did an excellent job explaining AMQP. However, it fell short of explaining the importance of asynchronous programming when it comes to the python AMQP client libraries. The author did touch on the topic, Esteve Fernandez commented with an example on how to achieve the same results, but with txAQMP.
I want to highlight Esteve’s comment and the importance of using asynchronous client code. Dan recently posted how to install txAMQP and a quick guide to test it out. In his next post, he’ll be showing us more examples using txAMQP.
txAMQP (just like AMQP and all other related AMQP libraries) is in early stages of development. If you’re working on a project that is using the AMQP protocol and your message queue clients are written in python, there is a fairly compelling reason to use txAMQP – it is based on Twisted.
Twisted is an event-driven networking engine. Being event-driven is important when implementing AMQP in your project, because it creates a concurrency model using non blocking calls. Consider the following use case:
One process engages the amqp client to put a message on the queue. Microseconds later, a second request to the client api attempts to put a message. Now multiple that volume of by 10,000 or more.
If you are using a python client like py-amqplib, will only process each message once it finished the message before. This can create a “traffic jam”, if the volume is great enough.
[NOTE: Dmitriy Samovskiy wrote about non-blocking sockets for py-amqplib on his blog. He included an add-on to support non-blocking calls for py-amqplib. This has not been merged into the py-amqplib at the time of this writing.]
By using an event-based architecture (as created by Twisted), this allows txAMQP client to proceed to publish (or consume) messages without having to wait for the last message to finish processing. Basically if there’s someone knocking at the door (the event), txAMQP is going to answer. Whereas py-amqplib will not be able to answer the door because it is still with the last person. Because of Twisted, we can acheive a greater volume of messaging through concurrency/asynchronous programming.
Also noteworthy, txAMQP gives us support for using Thrift and can quickly implement any version specification for AMQP.
So I encourage all of you AMQP’ers & pythonistas to contribute to txAMQP.
