From the course: DevOps for Data Scientists

Data science and software engineering

From the course: DevOps for Data Scientists

Start my 1-month free trial

Data science and software engineering

“

- [Instructor] DevOps is a set of practices based on software engineering and systems engineering. When we apply these techniques to data science, it helps to keep in mind how data science is similar to and different from application development. First, some definitions. Software engineering is a discipline dedicated to developing tools and techniques that allow us to build and use complex software systems. Data science is less about program development, and more about analyzing data. We do develop programs, which we call models. To develop those we use techniques from statistics and machine learning to get insights from our data. The product of data science is different from software engineering. In data science we build models. Models can be thought of as formulas or procedures for predicting values or analyzing datasets. These are implemented as programs, but they're not as generalized as other software engineering programming. Software engineering complements data science. When used together, they enable DevOps for data science. Some of the key areas of DevOps in the realm of data science are how we develop models, how we deploy models to production, how we keep production models running in a secure and reliable way, finally, how we can scale data science models to meet the volume of demand that we find in production environments.

Contents