The past two years I have conducted an extensive literature and tool review to answer the question: “What should software engineers learn about building production-ready machine learning systems?”. During my research I noted that because the discipline of building production-ready machine learning systems is so new, it is not so easy to get the terminology straight. People write about it from different perspectives and backgrounds and have not yet found each other to join forces. At the same time the field is moving fast and far from mature. My focus on material that is ready to be used with our bachelor level students (applied software engineers, profession-oriented education), helped me to consolidate everything I have found into a body of knowledge for building production-ready machine learning (ML) systems. In this post I will first define the discipline and introduce the terminology for AI engineering and MLOps.
LINK
This chapter discusses how to build production-ready machine learning systems. There are several challenges involved in accomplishing this, each with its specific solutions regarding practices and tool support. The chapter presents those solutions and introduces MLOps (machine learning operations, also called machine learning engineering) as an overarching and integrated approach in which data engineers, data scientists, software engineers, and operations engineers integrate their activities to implement validated machine learning applications managed from initial idea to daily operation in a production environment. This approach combines agile software engineering processes with the machine learning-specific workflow. Following the principles of MLOps is paramount in building high-quality production-ready machine learning systems. The current state of MLOps is discussed in terms of best practices and tool support. The chapter ends by describing future developments that are bound to improve and extend the tool support for implementing an MLOps approach.
LINK
In my previous post on AI engineering I defined the concepts involved in this new discipline and explained that with the current state of the practice, AI engineers could also be named machine learning (ML) engineers. In this post I would like to 1) define our view on the profession of applied AI engineer and 2) present the toolbox of an AI engineer with tools, methods and techniques to defy the challenges AI engineers typically face. I end this post with a short overview of related work and future directions. Attached to it is an extensive list of references and additional reading material.
LINK
Recently, the job market for Artificial Intelligence (AI) engineers has exploded. Since the role of AI engineer is relatively new, limited research has been done on the requirements as set by the industry. Moreover, the definition of an AI engineer is less established than for a data scientist or a software engineer. In this study we explore, based on job ads, the requirements from the job market for the position of AI engineer in The Netherlands. We retrieved job ad data between April 2018 and April 2021 from a large job ad database, Jobfeed from TextKernel. The job ads were selected with a process similar to the selection of primary studies in a literature review. We characterize the 367 resulting job ads based on meta-data such as publication date, industry/sector, educational background and job titles. To answer our research questions we have further coded 125 job ads manually. The job tasks of AI engineers are concentrated in five categories: business understanding, data engineering, modeling, software development and operations engineering. Companies ask for AI engineers with different profiles: 1) data science engineer with focus on modeling, 2) AI software engineer with focus on software development , 3) generalist AI engineer with focus on both models and software. Furthermore, we present the tools and technologies mentioned in the selected job ads, and the soft skills. Our research helps to understand the expectations companies have for professionals building AI-enabled systems. Understanding these expectations is crucial both for prospective AI engineers and educational institutions in charge of training those prospective engineers. Our research also helps to better define the profession of AI engineering. We do this by proposing an extended AI engineering life-cycle that includes a business understanding phase.
LINK
Over the past three years we have built a practice-oriented, bachelor level, educational programme for software engineers to specialize as AI engineers. The experience with this programme and the practical assignments our students execute in industry has given us valuable insights on the profession of AI engineer. In this paper we discuss our programme and the lessons learned for industry and research.
MULTIFILE
Organizations feel an urgency to develop and implement applications based on foundation models: AI-models that have been trained on large-scale general data and can be finetuned to domain-specific tasks. In this process organizations face many questions, regarding model training and deployment, but also concerning added business value, implementation risks and governance. They express a need for guidance to answer these questions in a suitable and responsible way. We intend to offer such guidance by the question matrix presented in this paper. The question matrix is adjusted from the model card, to match well with development of AIapplications rather than AI-models. First pilots with the question matrix revealed that it elicited discussions among developers and helped developers explicate their choices and intentions during development.
MULTIFILE
Context:Rapid developments and adoption of machine learning-based software solutions have enabled novel ways to tackle our societal problems. The ongoing digital transformation has led to the incorporation of these software solutions in just about every application domain. Software architecture for machine learning applications used during sustainable digital transformation can potentially aid the evolution of the underlying software system adding to its sustainability over time.Objective:Software architecture for machine learning applications in general is an open research area. When applying it to sustainable digital transformation it is not clear which of its considerations actually apply in this context. We therefore aim to understand how the topics of sustainable digital transformation, software architecture, and machine learning interact with each other.Methods:We perform a systematic mapping study to explore the scientific literature on the intersection of sustainable digital transformation, machine learning and software architecture.Results:We have found that the intersection of interest is small despite the amount of works on its individual aspects, and not all dimensions of sustainability are represented equally. We also found that application domains are diverse and include many important sectors and industry groups. At the same time, the perceived level of maturity of machine learning adoption by existing works seems to be quite low.Conclusion:Our findings show an opportunity for further software architecture research to aid sustainable digital transformation, especially by building on the emerging practice of machine learning operations.
DOCUMENT
