Almost a year ago, having my laptop and a sleeping bag in my backpack, I attended an AI-Hackathon in Germany. Right after the kick-off meeting at 9:00 AM I teamed up with two UX Designers and one Business Developer. We immediately started brainstorming to identify a potential project using open data and AI. Our first idea was to find a new particle in the CERN data or new physics. However, we dropped that idea real quick and decided to build a service for visually impaired people. The idea was basically to create an audiobook from any video content. Usually, Hackathons often aren’t long enough to create something entirely from scratch. Nevertheless, as I was working with Deep Learning models for quite some time, it wouldn’t take too long to recycle a couple of thousands line of code and wrap it around some video feed. Given my professional experience, my task was to develop the back-end of a minimalistic prototype within 24 hours while my teammates were focused on the user interface, presentation, and a bulletproof business case.
Without giving away too many details, the back-end was constructed based on three modules. The video module extracts keywords that are mapped to objects and atmospheric elements such as light saturation in the video frames. The audio module is mainly built on a speech-to-text algorithm with a topic extraction routine on top. The keywords extracted from the audio and video sequence are fed in a final step to the storytelling AI. Here, we create a narrative text that can be easily translated to audio. The funny thing about it is, that if we train our story AI on J. R. R. Tolkien’s books, you will receive a narrative text that sounds very similar to Tolkien’s writing style.
After 18 hours straight of intensive coding and data magic, we managed to produce a minimum valuable product possible.
Our prototype quickly draws the attention of the company’s CTO which was hosting the Hackathon as he was eager to learn more about the implemented methodology. The results of the other teams were stunning as well – every team produced cool prototypes and some committed to extending their work beyond the Hackathon as private projects. Overall, the Hackathon exceeded my expectations and to be honest, and I cannot tell how we won that event as the competition was strong a highly innovative. There is no universal solution to that, but one thing is sure: positive vibes, solid technical skills, and mainly great team-play are the most essential ingredients.