AutoML v.0.3.5: Assessor Speed-up and a Sneak Peek

Claudio Bruderer

Written by:
Claudio Bruderer (Product Manager at Modulos)

Our latest version of Modulos AutoML is out: v.0.3.5. With this version, we are adding major speed improvements to the assessor, which determines if a neural network has finished training. We are also preparing a new and improved workflow creation, which we are excited to share with our customers soon. We are including a sneak peek here.

Data Science Updates

Assessor Speed-Up

The training process phases each candidate solution passes through.

The Modulos AutoML solutions are trained iteratively. For neural networks in particular, the platform repeats this training process multiple times. This is controlled by the assessor, which decides whether the model needs to continue training for another epoch. If not, training stops and the final solution is packaged and made available to you.

In this release, we refined the assessor step. We use the median stopping rule to stop the training cycles, if the median of the validation scores stops improving. We now also take the variance of the scores into account with the goal of avoiding plateauing scores without significant improvements. These changes lead to a 10% to 70% speed-up when training neural networks (performance tested on identical machines and data, speed-ups vary depending on the ML task type). At the end of the training cycle we return the solution with the best validation score. Compared to previous Modulos AutoML versions, this allows you to generate more and better solutions within the same time frame.

Descriptions of Objectives

Redesigned descriptions of the objective selection in the workflow creation.

The configuration of the Machine Learning task – the workflow – by you as the domain expert is one of the crucial steps when tackling your use case with the AutoML process. During the workflow creation it is important to choose the appropriate objective, the measure which AutoML optimizes for. We standardized and expanded the descriptions of these objectives in the workflow creation to follow a clear design.

Platform Updates

Sneak Peek: Fully Redesigned Workflow Creation

An impression of the new workflow creation to be released soon.

We are happy to give you a sneak peek into what we are currently working on and are looking forward to sharing with you. We are redefining the entire workflow creation process. Besides giving it a fresh and sleek look, it will allow for more flexibility and dynamicity when configuring a Machine Learning task. With this new generation of the workflow creation, we will be able to provide you with even more exciting features in the near future. So, stay tuned!

Other Improvements and Bug Fixes

Plots in the README for solutions giving insights into the performance of a neural network on validation data.

We additionally made a range of fixes and improvements to AutoML, which include:

  • Refined the Kernel Density plot in the README for trained solutions for regression tasks. This figure allows you to visually assess a solution’s performance on the validation data. The scaling of the color map has been polished and we have expanded the range of datasets, for which it is applicable.
  • Updated the README to also include detailed instructions on how to update the AutoML platform to the latest version.
  • Made the F1 Score objective for binary classification more robust to accept a wider range of data.