A beginner’s guide to building a Chatbot

Part 1

Hello world. I’m someone who really enjoys writing and really enjoys coding — but so far, I’ve never considered writing about my code. But one of the perks of quarantine is having a lot of time, so I figured this was the best time to try it out.

So this is my first attempt at that. I am going to combine two of my favourite ways of coding to do this: the way I learnt to code in the CS106 program at Stanford, and all the online guides I use when working on projects.

1. I love the 106 program because it was less about the specifics of syntax and more about thinking through a problem by breaking it down into its component parts: small chunks which seem much more doable and easy to solve that huge behemoth projects that seem impossible. It’s a skill I value in coding as well as generally — and I’ve tried to apply that to this guide as much as possible.
2. I am always looking for guides online shamelessly — it’s how I’ve completed any project I’ve worked on. It’s absolutely crazy how many resources there are online and how many people have taken out the time to put them together. Whenever I get stuck on a detail, I turn to this huge online community, and I’ve never not found an answer. Which is crazy! No matter which language I’m working in, what kind of project I’m working on, there’s always somebody who took the time to write an explainer about it in some corner of the internet. Absolutely crazy and wonderful.

So, a brief explainer on what this project is about:

I want to build a Whatsapp chatbot to assist our customer service rep. This will be a very simple bot that will be initiated when a user texts us first. I am working on another article about pushing messages to customers (sending the first message as opposed to replying to messages) — but that will not be a part of this because the permissions required etc are very different.

Three main functionalities:

1. Responding to keywords in the users message based on a scheme in our google sheet. This is so that we can easily collaborate on the keyword: reply relationship, and change the language as a team (without needed to interact with any code).
2. Get the user’s order status from firebase: We do daily delivery of fresh traditional flowers. That means a lot of our messages are simply customers asking where their flowers are, and us responding with the proof of delivery picture showing that it’s outside their door. Pretty simple, and something that would be nice to automate, especially since our deliveries happen between 4–7am and that’s not a fun time for a customer service rep to be checking the database.
3. Transfer a conversation to a human: this is a pretty simple bot, which is fine because we don’t need something very sophisticated (right now). But that does mean that there will be times where the chatbot cannot go further and needs a human to intervene. That’s fine, because this is meant to assist our team. Also because nothing beats having a person on the other end of the line!

Why Whatsapp: because it’s the platform our customers are most comfortable with. It does make it a little harder because Whatsapp has a lot of rules and is relatively new — but that also makes it exciting.

Now, the ingredient list. Here’s what we’ll be using to build this bot:

1. Python: you should be reasonably familiar with this, at least enough to read the code I walk through and understand the syntax.
2. Twilio: This is what we use to connect to whatsapp etc. You’ll have to set up an account, connect to their sandbox (for testing) and use their API. We’ll walk through the latter stuff, but they have really good documentation (link) that walks you through the account set up etc.
3. Flask: it’s okay if this is unfamiliar. I had never used flask extensively before this project. One thing to note is that this project works, but in the development environment. I’m considering doing another article about how to put it into production (there’s a lot of things I’m putting off to future articles so this doesn’t get too bulky).
4. Ngrok: Another thing we need in the development environment. Another thing I knew little, if anything, about before this project. Didn’t pose a problem.
5. Firebase: This is where our company database is — we use the realtime database for our records and the storage for our proof of delivery pictures. I love Firebase because it’s so easy to use. You won’t need a lot of knowledge about it for this part — most of the firebase-y stuff is imported into this project from another program I wrote (which will be the topic of another article).
6. Google Sheets API: I mostly just love google products and it’s a huge bias I am totally okay with having. I use this so that the scheme for our keywords: replies and the log of messages can all be on google sheets — a.k.a available to the rest of my team in real time without them having to interact with code or do anything out of the ordinary. Also — this is so easy to use! Again, most (all) of the heavy lifting is in another program (yet another topic for yet another potential future post).

And here is the article I drew on heavily to make this project, linked here for the THIRD TIME in case you missed it: https://www.twilio.com/blog/build-a-whatsapp-chatbot-with-python-flask-and-twilio
I highly recommend reading it to understand the basic structure as well as how Twilio works, what Flask is and why we need ngrok.

Okay — that’s all the context we need. Now let’s dive into the code!

Here’s what the chatbot look’s like in its entirety. And this is minus the additional programs to connect to firebase and google sheet!

Don’t try to read this — you’ll only hurt your eyes.

I don’t know how you feel when looking at this, but I feel a little bit like this:

panic in the IDE

So let’s break it down:

So much better with colors

There are four main sections of the code, and the rest of this tutorial will break them down accordingly. Let’s check them out:

1. Imports

This is all of the building blocks that we need from other people/programs/libraries to help us build our code. Remember: you’re never starting from scratch! That’s the benefit of decomposing — if you break a problem into it’s small component parts, you’ll find that a lot of the small parts have already been solved!

2. Helper Functions

The main logic should be simple to read — that means a lot of the helper tasks need to be decomposed — a.k.a put into their own little functions that we call when we need them. This is the “art” part of the art and science of coding — good code it not only machine readable, it’s also human readable. That’s because it’s humans who will be using, maintaining and repurposing your code — so make it easier for them! Also for you — the easier to read your code is, the easier your life debugging will be. And believe me, you will be debugging a lot. I’ve tried to to that as much as possible here.

My rule of thumb is the same that I use for paragraphs — each paragraph should be about one idea. If you have a new idea, go onto the next paragraph. Similarly, each function does one main thing — if you find it’s doing more than one verb, it might be time to decompose.

3. Main Bot Logic

This is where the main action happens. I will break it all down further down in the article so stay tuned.

4. Boiler Code

Boiler plate code. Looks confusing — but just copy&paste it in and you’ll be good to go.

Think about this like labelling your notebook pages with the date and chapter heading. It’s just the context that allows you (or the computer in this case) find its place. You need not really understand it right now, just know that you need to put it in there. I’ll keep saying this because I’m a huge fan of not knowing something unless I have to.

Okay so let’s dive in a little bit more:

1. Imports:

Speaking to the internet

Twilio, date/time ft. some very special imports by me!

When I started out, I would look at projects online and blindly copy the imports and “pip install” my heart out. This is, while irresponsible, not that terrible of a strategy when starting out. But imports are probably the easiest thing in the world to understand, so I don’t know why I put it off for so long. It’s nice to know what each line in your code does, so the pictures above break it down for you.

Pay special attention the last picture, which I call “Rhea’s babies” — these are programs I made and custom functions from that. That’s why copy+pasting the imports section won’t work for this project — because you won’t have those files (example: dictsFromSheets.py) in your system.

2. Helper Functions

So we’ve got our imports in place — let’s now go to our helpers. Our helpers are our backbone. They do all the heavy lifting so our code can look pretty and succinct in the main method. Usually the way I approach helpers is that I write pseudo code (comments laying out the steps I need to get to the end goal) in my main method, and then under each comment I write the code, test and if it works, write the next bit of code for the next comment (/next step in my pseudo code). Then at regular intervals I start tucking those chunks of code that I know are working into helper functions.

This is not, strictly speaking, the recommended way to go about things — you’re supposed to pseudo code and then do your helper functions (and test iteratively along the way. Writing a bunch of code and then breaking it into helper functions is the retroactive way to do it.

But here’s my thing: the pre and post conditions of the helper functions matter. To elaborate — the way something is before you enter the helper function, and the way it is after you’re done with it. You have to make sure the work you do inside the helper method is passed back to the main logic AND you have to make sure the information you need is passed into the helper. I don’t know what I’ll need in the helper before I write it and test it a bunch of times! And I don’t want to rewrite the function header every time! It’s annoying. So I do it all in main where I have access to all the variables I need and I know the thing I’m changing which has to be passed back eventually — and once I know it works, THEN I take it out and put it into a helper method. And I only have to rewrite my header like once or twice — which is a win in my opinion.

I’m not saying you should develop my bad habits. I’m just saying there are reasons for my bad habits.

Anyway, back to the actual helpers in this project:

The above are pretty basic. They are the B actors who you need, but don’t necessarily need to fawn over.

These are the heavy lifters! Make sure you understand these ones because they’re actually interesting.

Whenever you see a green square in the main logic, you know we are using one of these helpers. You can scroll back here to check it out once you have the context of its use. This will give you a clearer picture of how they help.

And finally,

3. Main Bot Logic

Ta-da!

This is the main bot function — further broken up so some of the big bulky sections go under a title that explains essentially what they do. Though it seems long (unavoidable in the way I laid out my logic), the pseudo code essentially boils down to:

1. Set up all the data and variables you need.
2. For all your keywords:

— a. Figure out if this keyword, or any of its synonyms, are in the users message:

- — — — if yes:

— — — — pick a reply from your table of keyword: reply relationships. Do a little extra if the keyword if “STATUS” or “CALL”.

3. If you’re done searching all your keywords and the reply is STILL empty, then just set it to this default message to transfer to a human.

4. Add the reply to your history & log for good practise.

5. Wrap up your text reply in a message object, put it in the response box, and send it out!

It’s as simple as that! Anytime it doesn’t seem simple — come back to these 8 lines. Below I break down each line to match it to the corresponding code, and give you some details about the implementation.

1. Set up all the data and variables you need.

This above, deceptively short line of code sets up our data structures. It’s worth it to understand them a little bit further here.

Here’s how the google sheet that we get the data would looks like:

Sample

The first column is the keyword, the second is the reply for that keyword and the third counts the number of synonyms (that start from the next column).

That data gets into a data structure that looks technically like this:

d_message = {“Hi”:“Hello! I am Lakshmi…”, “call”: “Let me find an agent…”, “bye”: “Hope you have a good day!” …..}

d_synonym = {“Hi”:[“hi”,“hello”,“hey”,“good morning”…], “call”: [“call”,“phone”], “bye”: [“bye”,“goodbye”,“gbye”] …..}

For those wondering: d_message is a dictionary of string to string. d_synonym is a dictionary of string to list. We use dictionaries instead of lists because the look up is faster.

But here’s a slightly easier way to visualise it:

Data structures

When you call d_messages[“Hi”] you get back the reply “Hello, I am a chatbot.”

Now let’s move on to the rest of the set up we have to do:

Request + Twilio + Time

Getting the person’s order from our database + setting up variables

Notice our helpers are already coming into use big time. Now, let’s go on to the next part of our main bot logic: the part where we find the keyword and construct our reply.

See how we use the data structures? We are getting all the keywords in the first line, and the FOR method will iterate through them all. InMessage, if you remember from the helper section, uses the synonyms dictionary to check for all synonyms of the keyword in the message.

Now you’ll notice here that treat “STATUS” and “CALL” differently. Let’s explore those a little in detail.

This is because if a person asks for their status, we need to 1) pull out their order and 2) get the proof of delivery picture. That means we have three possible replies:

1. You’re order isn’t in the database
2. You’re order IS in the database BUT we haven’t gotten your proof of delivery picture yet (oops).
3. Here’s your order and here’s your proof of delivery picture. Yay!

The picture comes from Firebase and once again — that’s functionality for another article. But that’s why this part is a little long.

When a user asks for a call, we’ve got to contact a human. That’s why the explicit extra step.

And finally:

For every other keyword!

Now the next part of our main bot logic is putting this text reply + media into a message object, putting that in the response box and sending it to the user. We also add it to our logs for good practise. Here’s the code for that:

We’re done with our main bot logic!

Let’s recap:

In the broader context of our program:

We also went over the helpers:

And the imports:

And of course, the boiler plate:

And even though it took a long time — we converted that big chunk of code into something that (hopefully) makes a lot more sense!

That marks the end of our little coding journey.

(I mean, not completely because of all that extra stuff that links into this project that I still have to write about, but at least for this part of it.)

I haven’t written this with any particular level of coding in mind and oscillated pretty wildly throughout the text in how much I assumed you knew, but mostly because more than the code itself, this was an exercise in explaining how I think about code. And I hope this has helped/interested/entertained you. I hope that this gave you some insight into the structure of my project, what it looks like to break it up into tiny pieces and tackle those manageable chunk. Mostly, I hope you made it to the end of this long article!

I’d love to hear from you and get feedback — so do drop me a message and let me know how I can make this better/short/use more gifs in it. Thank you for reading!