PlusPlus

Slack Bot for Scoring

rebuilt and improved by Redwerk

Incling
PlusPlusMelbourne, Australia

PlusPlus is a business entertainment startup that provides functionality for scoring all kinds of things and users in Slack.

All Customers
Product Development

PlusPlus came to us with a product in Python already developed, which needed a tech stack switch and stability improvements. We grasped the codebase in no time and started development from day 1.

Learn more
Startups & Innovation

Turning ideas into code, testing and releasing the results, improving and maintaining the projects is our contribution to happiness both of startup entrepreneurs and user communities.

Learn more

Challenge

PlusPlus is a chatbot for Slack made for entertainment. It allows to plus, minus and score things your friends say and do on Slack by addressing to objects and users with ‘@’. It has a global leaderboard and every action affects the general results.

The challenge for Redwerk team was to completely rewrite the existing bot, originally developed in Python, using Ruby as the core technology. We were also tasked to increase stability, make it possible to scale and optimize the overall performance of this high load project.

Solution

We’ve started with the most important part — code refactoring and fixing. The old version of the bot was written in Python and we needed to port it to Ruby, keeping in mind all the additional changes it might bring, fixing existing problems and implementing new functionalities. When the infrastructure was planned, we based our assumptions about the load on the average statistics of old Python bot usage: approx. 500-700 thousand user actions per minute. We extended this level up to ~1 million user actions/minute in order to have some reserve capacity and provided a set of stress tests on staging.

And here’s where another challenge appeared. It turned out that Slack limits testing to 1 message per second and doesn’t have the functionality to test high load projects. The actual live load of PlusPlus showed over 20-50 million actions per minute, and the existing infrastructure couldn’t support it. To cover the new demand, a major part of application logic had to be reworked.

Users and objects logic optimization

Redwerk’s dev team needed to carefully define and optimize any gaps and even the smallest leaks of memory and CPU usage. The actual project load sent millions of actions containing both @usernames and @things in a single line. Objects required more advanced logic than users, because they needed to be parsed, then checked if existed in database, created a new entry if didn’t, then updated score, leaderboard and the global trending. With millions of such actions, most of them happening within a single message, it overloaded the DB very quickly, as well as the memory and CPU.

Another high load task was to optimize score updates in the leaderboards, as every action had to influence the position in the global leaderboards of users and objects. The incredibly high amount of items that had to be parsed, checked, and updated, created additional challenges for development.

In order to optimise all these actions, our team developed functionality that uses multiple CPU cores on each server, along with a specific Bot Manager, which routes each bot to a specific node (Server + CPU core) — a kind of task which Amazon load balancer couldn’t handle. After that, we rebuilt request logic to decrease the number of requests.

We also switched from Ubuntu to CentOS. Ubuntu didn’t allow us to create additional connections to a database. CentOS in its turn provided more stable environment while working with MongoDB and supported the excessive action load. It also allowed us to re-configure AWS server and update OS core settings on nodes to improve the file descriptor and threads limits.

Need proper execution of your idea?

Contact Now

1m+

total users

500k+

active channels

4

people in dev team

5

months to complete

19,889

lines of code

Results

Redwerk team was able to rewrite PlusPlus bot completely. Now its infrastructure works stable and has enough workarounds and solutions to keep it live and restore it if needed. We’ve also created a mechanism to simplify scaling and add new servers quickly.

The solution was also provided with updated handover documentation and a training session for clients on how to handle and maintain the renewed system.

Visit plusplus.chat

PlusPlus

Ready to outsource software product development?

Contact Us
Redwerk developer's comment
Alex

Alex
Developer

The challenge for Redwerk team was to completely rewrite the existing bot, originally developed in Python, using Ruby as the core technology. We were also tasked to increase stability, make it possible to scale and optimize the overall performance of this high load project.

Impressed? Hire us

Contact Now

Other Case Studies

IT outsourcing case study - LikeLabs / Redwerk company
Russia
iPad app that allows visitors in a public place to create and share a review
IT outsourcing case study - Adfectious / Redwerk company
Romania
Mobile advertising system with advanced targeting and statistics
IT outsourcing case study - Cleanagents / Redwerk company
Germany
Mobile app for professional cleaning services