How my Telegram bot “Vaccination finder” helped community and scaled for large scale users. Problems to Solutions — journey !
Motivation — Micro-Chat-Bots a future for human chores.
A month ago, I came across coWin open APIs provided by government of India for vaccination center lookup for third party integrations and it was available for public and protected use as per information security.
I saw many git repos and developers who implemented/consumed the same for their website integration using UI frameworks, python scripts and emailing CRON jobs using plain shell scripts and Java or many other technologies. But still there was a reachability issue in my opinion.
To solve that issue, I put some efforts and my knowledge around Telegram bots API and integrated coWin API. The target was search, lookup seamlessly, get notifications timely and easy to use.
Quick stats :
Users 350+
all notification sent (till 17 May 2021): 14k+
Demo video
Problem of Less reachability-
Reachability — who talks to whom? I mean your user have to check himself on websites most of the time. (Agree there are web push too — to go back to users) but how many users permit for the same ? 40% in my opinion.
Gmail auto sync is somehow turned off default, and not sure how many people checks email frequently or enabled notifications for email. Since they are too frequent from many sources and spams people are now fed up of them.
Next problem, Notification Frequency and API limit
There is one more problem via CoWIN API is imposed request limit of 100 per 5 minutes. But by luck or popularity I got more than 300 (and growing) users who want notifications on timely basis and with short interval. As you may know slots get booked too fast notification per 5 minutes is very slow, at best case you should get notified every minute but which is not possible for these many users. So, simple DB read and request approach does not work here.
Next is, User Experience and literacy
As most of the users are not familiar to bots concept but they need something helpful and easy to use. May be there are more but lets go deeper as and when we face them.
Getting Started
Bot Father — Amazing bot by telegram to create your own bot https://t.me/botfather helped to create a bot and get the bot created with all things like any normal user on Telegram can do, except talking to anyone directly (as bot may not have a device or contacts as such).
I created a project for basic TS code and all environment, then coding begun.
I got a token and then used it for node JS server implementation with some JS lib/scripts for telegram you can see them on my Github repo folder . I preferred typescript for scaling and updating the logic seamlessly using OOPs.
If your Identity is bot, How people will interact with your script?
Telegram supports concept of commands which helps you identify user wish/request and work according to it. By the way you have to tell bot father about list of supported commands by public. Once its fed, my bot started showing them in commands button as below.
you can check full demo with original bot directly on https://t.me/VaccineNotifier_IN_bot this link I will continue my journey here.
Initializing the bot
As you can see in code it needs token, I accept that via command line to make sure its secure and not pushed to git
Commands and reply buttons using keyboards.
Telegram support multiple commands and customized button based keyboards which can help you ease the user experience like below.
You can refer other commands on Github.
MongoDB to maintain minimum users information
Here any DB like MySQL or no-SQL can help but my favorite is MongoDB so I used that, since there is a minimum requirement if database so, Mongo is easy to implement and
Centralized Messages to track every sent message
I used unique code generation for every message I sent to users, to check status or any disputes and other information about it in future. So I brought another class just for sending any message from the application and imposed things I wanted or needed before message is sent.
In case user blocked the bot this class will remove that user immediately so that on next iteration we can save one network call for that user. In case user comes again he will be added back again to DB for processing.
How to handle load?
Group people of same district
Since there are many people in one district and we can serve their preference with single result set for that district, I used aggregate $group on mongo to avoid any duplication of districts calls.
Still there are many districts in the data base and we need available (request)capacity for people searching the centers using bot and then subscribed users. So I used,
Rabbit MQ — A queue that helps for producer consumer (task based) approach helped me to resolve the issue and manages independently and scalable for large scale. see more here
RabbitMQ is the most widely deployed open source message broker.
So, we need task based approach to make sure only one consumer can pick it up. So ideally you can deploy consumers on different terminals (at least now, its working now-ideally it should not as its same IP address). Still I have 2 consumers who are independent to work and depend on broker.
I used ack (acknowledged — done)and nack (negative acknowledgment — not done) to inform broker about task completion. This way it either mark task done and removes from queue or enqueue task for reprocessing respectively.
I am hoping to deploy the consumers to browsers using plain JS web socket client which can scale it more.
Stay tuned for more updates.
