Building a WelcomeBot

Boston Python User Group, December 2014

Shauna Gordon-McKeon (@shauna_gm, shauna@openhatch.org, shauna on Freenode)

The background

The problem

The solution

So what is WelcomeBot?

So what is IRC?

So what is an application layer protocol?

-- What is WelcomeBot? A program written in Python which uses the socket library to connect to IRC servers, join IRC channels, and message IRC users. -- What is IRC? There's actually a few ways to answer that. One way is to think of IRC as a tool, or platform. In that way, you can think of IRC as something like AOL Instant Messenger, or AIM. I like comparing it to AIM because AIM was the first chat client that I ever used. It's a tool that allows you to talk to people, either in group chats, called channels, or in private messages. Another way to think of IRC is as a protocol. In fact, it's what's known as an application layer protocol. -- What's an application layer protocol? So in order to communicate with each other, nodes on a network need to agree on a basic syntax, an abstract structure, so that can establish when they're being talked to, who is talking to them, and neat things like that. There's different layers of abstraction. So for instance, there's IP, which is a very low level protocol. TCP, which is the transport layer protocol - you hear these things being referred to together as TCP/IP. And then, there's application level. Another application level protocol is HTTP, which our web browsers use, and SSH, which allows you to create secure encrypted connections. IRC is at that level of protocol.

Try it! @ bit.ly/ircbot-python

Socket Library

We start by creating a socket object, with some details about the connection we want to make.

ircsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

We then use this object to connect to the server on port 6667.

server = "irc.freenode.net" 
ircsock.connect((server, 6667))

Socket Library (cont)

We can now communicate using send and recv:

ircsock.send("PRIVMSG "+ channel +" :Hello!\n")
ircsock.recv(2048)

These sent messages can include a variety of IRC syntax:

ircsock.send("PONG :pingis\n")  
ircsock.send("JOIN "+ chan +"\n")
ircsock.send("NICK "+ botnick +"\n")

Basic Structure of WelcomeBot

(as seen at https://github.com/shaunagm/WelcomeBot/)

If newcomer joins, create a newcomer object with a datetime field.
If a non-newcomer speaks into the channel, silently record nickname and delete newcomer object.
Regularly check how long the newcomer has been around.
Once the newcomer has been around for longer than a set time (default is 60 seconds) greet them.

A mistake!

aka a learning experience

How do you listen and respond at the same time?

import queue, threading
def getIRC():
    q = Queue.LifoQueue()
    while True:
        ircmsg = ircsock.recv(2048) # receive data from the server
        ircmsg = ircmsg.strip('\n\r') # removing any unnecessary linebreaks.
        q.put(ircmsg)
t = Thread(target=getIRC)
t.daemon = True
t.start()

So, one of the first worries I had when trying to adapt this basic script to fit my needs was that the bot needs to be able to do a few things at once. It wants to receive IRC messages, but it also wants to be checking how long newcomers have been in the channel and welcoming them and processing them as needed. My first solution to this used the thread and queue libraries. I'm not going to go into a lot of detail here, but just briefly, you can see - this code creates a Last In, First Out queue, so processing things in the order its received. It's then wrapped in a thread, which means it can sort of hum along on repeat in the background while the bot does other stuff. So, from my perspective, this was great - the bot could take care of business while not missing a message.

Totally unnecessary

import select
ready_to_read, b, c = select.select([ircsock],[],[], 1)

Turns out that's all totally unnecessary. Operating systems, having been designed by reasonable people, recognize that you might want to do other things instead of waiting constantly by the phone for a call, and so they save a backlog of connections. Great! So I got rid of threading and queuing. Instead, I just needed to know when to access that backlog, that is - when a message had been received. Unix systems have a tool called select() which the python select library uses, which keeps track of when objects - sockets, but also files - have been changed. The select.select() function takes three parameters, which are lists of objects you want to check. The first is for things you want to read, the second is for things you want to write to, and the third is waiting for exceptional conditions - of course, we just want to read, so we use the first parameter. The last parameter is an optional timeout argument. Here, we're saying check, wait one second, and move on.

Learning experience for everyone

12 contributors, including 6 who are making their first open source contribution!

Saves known nicknames
Recognizes patterns of nicknames, eg "shauna__" as "shauna"
Responds to greetings and requests for more info
Best of all...

Tests!

class TestBotClass(unittest.TestCase):

    def setUp(self):
        self.bot = botcode.Bot()
        
    def test_known_nicks_setup(self):
        bot = botcode.Bot('test_nicks.csv')
        self.assertEqual(bot.known_nicks, [['Alice'], ['Bob']])

    def test_add_nick_underscore_removal(self):
        self.bot.known_nicks = [['Fluffy'], ['Spot']]
        self.bot.add_known_nick('Roger__')
        self.assertEqual(self.bot.known_nicks,[['Fluffy'], ['Spot'], ['Roger']])

    def tearDown(self):
        with open('test_nicks.csv', 'w') as csv_file:
            csv_file.write('Alice\nBob\n')

Thanks!

Guides

Ned (seriously, watch his Ned's Pycon 2014 talk on unit testing), Asheesh Laroia, and the kind folks at project night.

Contributors

jbertino, artLopez, aaparella, clarissavazquez, dcarrot2, ppegusii, sunu, BananaObserved, lkuper, kwurst and Aaron1011