Train Them in LAB

 

Stop Training Models in DEV. Train Them in LAB.

Congratulations, ML professionals of all kinds! Only 10 years after Andrew Ng taught everyone about gradient descent, our respectable profession has almost entirely moved away from using the PROD environment for everything. Pat yourself on the back.

Now shame on you! Because I glanced over your shoulder just now and saw you training a model in DEV. There’s a better way, people, and it’s called LAB.

Let’s talk for a minute about Environments. First off, let me provide a dense definition of Environment and then unpack it a bit. For the purpose of this blog, an Environment is a runtime that is configured to use a specific group of infrastructure resources for a particular use case. (This definition mashes a few different concepts together, namely runtime, resources, and use case. Here and here are some useful blog posts separating those concepts for you.)

For example, when you’re working in DEV and you start running your code, you’ve probably got your source control checked out to some sort of dev branch. And you’ve got a config file somewhere that points your code to the dev endpoints of any services you use.

Here