Our Weekend Project saved lives and then it didn’t
Everything changed on 16th Feb 2021. For the first time in 6 months, there was a rise in the number of covid cases in India. In the span of 2 months, the cases peaked as much as the first wave. And within a month, almost four hundred thousand cases were reported every day. It was evident that the current medical resources available in the country are nowhere close to the required numbers.
The government started hosting public websites which had bed availability in various hospitals in a city. But these websites covered largely cities and metros. What about a hundred other cities and towns? There was a rise in volunteers who started recording the availability of beds, oxygen, and other covid related resources in spreadsheets. They worked round the clock verifying such information and kept these google sheets up to date.
In a crisis like this, one of the challenging things for a victim is to identify the availability of such resources, be it hospital beds, medicines, oxygen cylinders, and ventilators. Different people volunteer differently. We (a bunch of noob hackers) felt like we could contribute here using our combined knowledge of Data Science & Web Development to build a web app that could collate all the information and make it easy to search.
Everything here onwards is technical in nature.
The tech stack
This was a weekend project. This means we had to finish development and launch the site in less than 48 hours. Having said that, we still went ahead with technology that was appropriate for the job.
We wrote Python scripts that run at regular intervals. These scripts scrape the information from government websites and other verified user-hosted websites periodically and put it in our database. We also added google app scripts that scrape information from Spreadsheets and push the same information to the database.
Chose Go for the REST endpoints of the web app. We didn’t choose Go because it was fast or to take advantage of its concurrency. It’s just that to build a REST API Go was quickest in terms of time to develop. And it’s strongly typed nature ensured the data consistency.
The interesting part is the choice of language for our web client. We predicted a majority of the users are gonna be from mobile. And it was true, as per our google analytics almost 99% of users came from mobile. We didn’t want to build an android and ios app because it’s too time-consuming to build, there is a barrier of entry as someone has to install the app to use it, and most importantly we had to shell out money to pay for the app stores. But we still wanted something which can run on the web and also as an app. We evaluated developing a PWA (Progressive Web Application) but none of us knew how to. One of us had been working on Flutter for a while and told us that Flutter’s web implementation is now generally available. We took a gamble and chose to go with Flutter.
We debated on which database to use. We wanted to use MongoDB (NoSQL) but due to the nature of the data, we went ahead with MySQL (SQL).
The database design
When it comes to information catalog, especially with unstructured data fragmented across different sources like web pages and spreadsheets, it’s hard to come up with a database design. So we came up with something very simple.
If you look at the database schema, it has certain obvious design flaws. For example.,
- There is no foreign key constraint between the `places` and `cities` table. Despite there being a `cities` table.
- There is a hierarchical relationship between `cities` and `states` but we are recording `cityId` and `stateId` in `places` table.
- There are 2 separate fields `cityId` and `stateId` in `places` table despite there being `cityId` and `stateId` columns being present.
The reasons are rather simple actually. It’s mostly due to our data sources.
- There can be a hospital or another covid resource with information regarding the state but the city is not present.
- There can be a hospital or another covid resource with belongs to the city which is not present in our `cities` table.
We also wanted users to contribute. Users should be able to add information. Since there is no automated way to verify user-generated information, we show the verified flag as false.
Also when it comes to the `places` table there is no unique index. There is no way we can uniquely identify a place when dealing with multiple data sources. Hence on each run of a script, we delete the older data before adding new records.
The launch
We couldn’t finish the app in 48 hours. We underestimated the complexity involved when dealing with data from different data sources. We didn’t want to launch the app and misinform people. Also, we didn’t compromise on the experience with the app. We had to consider that the users are mostly friends and families of victims who are distressed. We chose to make the icons big enough so it’s easy to click. Kept font sizes big enough to read.
We bought the domain fightcorona.co.in. We wanted fightcorona.in but it was already taken. We deployed our code to an AWS EC2 instance. Used Nginx as our web server for serving both our web app as well as Golang’s REST API. We knew whatever we shipped in our first release was not good enough. It was just an MVP and there was a lot more that we could do. We didn’t have time to set up a CI/CD Pipeline and hence we just wrote shell scripts that build and ship the code to EC2 and restart the Nginx server.
We ensured there was an anonymous feedback option present in the app so users could directly communicate with us. Needless to say, there was an overwhelming amount of mixed feedback. Some appreciated the effort, informed us that they got a hospital bed using our app. Some criticized for not adding covering pan India information. Some just abused for no reason.
The end
Over the coming weeks, we shipped more and more features, refined our user flows, and added covered more places. The first few weeks we had hundreds of users and the site was shared organically among Whatsapp and Instagram. But when we look at our cohorts we realized people who visited the website never visited again after 2 days. It was a good sign because we genuinely don’t want people to use our app for a prolonged period. We want them to find whatever information they need instantly and just get better.
However, the daily usage metrics almost dropped to 10 users a day after a month. With our early feedback, we were confident that our app will help many. But there was a steady decline in usage. Though we succeeded in developing a solution we failed to reach that solution to the masses. I think most startups go through similar experiences. They build a really good product but they underestimate the amount of time and energy that needs to be spent in acquiring users.
One good news is that there was a steady decline in the number of cases from the second week of May and more and availability of hospital beds and medicines increased. In the end, we were glad that we could help a few hundred people during the tougher period the second wave. Now we are working on features that could potentially help thousands of people brace the impact of the third wave. Though we wish from the bottom of our hearts that this is the last of suffering we have to face.
Contributors: Arnab Majumdar, Makzink
