Over the last few decades, we have made giant leaps forward with machine learning technologies. We can now train machines to predict the next word that together forms a coherent sentence or pick out faces in camera footage. Specifically, in medical settings, ML can accelerate the drug development process by discovering alternative biomedical pathways for therapy.
With the improvement of these technologies, it is only natural to ask, “What can be built with these models?” And, secondly, “how do we do it?“
In a recent Medium blog post, Caleb Kaiser explains that we should not approach the latter question from a data science perspective, but rather an engineering one. Specifically, to address model building a new field is emerging: machine learning engineering.
ML engineering is the process of using analytical and data science knowledge and combining them with software engineering principles to create a product. How is this different from the work of a data scientist? Let’s examine the semantics.
Data scientists build models, providing a foundation in which ML engineers can use to apply the models to real world problems. In other words, data scientists build models that are able to automatically predict, learn, and improve from experience without being explicitly programmed. An ML engineer uses the models to build a production environment at scale. Thus, ML engineers have to work with data scientists, software engineers, data engineers, and a user interface/user experience (UI/UX) specialist to develop a production application. This is further exemplified below:
Source: towardsdatascience
For example, OpenAI’s GPT-2 is able to generate text that can mimic human writing by predicting the next word. This is the work of data scientists. AI Dungeon utilises the GPT-2 technology to generate multiple adventure stories for a dungeon crawl, a role-playing game that involves exploring and fighting in elaborate virtual dungeons. Players will input several texts, which the model then uses to generate a game layout. Building the dungeon crawler is the work of ML engineers.
However, it is not as easy as it sounds. This is because the software itself, GPT-2 is huge in size, at over 5GB, and utilises large resources, which may exhaust the server. A single text prediction can freeze its graphic processing unit (GPU) for extended periods of time. Scaling these problems to production-level models that can handle terabytes of real-time data, such as for AI Dungeon with over 1 million players, pose a difficult problem for machine learning engineers to solve. Engineers have to build a functional Application Programming Interface (API), implement rolling updates for the system to catch up with updated models, configure autoscaling to process incoming new data, as well as optimising costs. Simply put, ML engineers have to retrain, monitor, and batch predict the system, amongst other tasks.
The solution? Treating the challenges posed by ML engineering as engineering problems, not data science challenges. ML is in every corner of our society, from the music streaming services we use, ordering an Uber and obtaining an estimated time for the journey, to asking Siri what the weather is like on the weekends. We have been using ML services for years. As ML engineering becomes more accessible, not just for large technology companies and small start-ups, these inherent challenges can easily be solved as we learn from each other and build open source platforms for other engineers to optimise their models. In order to sustain the momentum and create breakthroughs in the biomedical field, we have to encourage ambitious engineers to create more applications that can accelerate diagnosis and treatment for patients.
Image credit: FreePix
