Claudio Bruderer (Product Manager at Modulos)
The latest version of Modulos AutoML is out: v.0.3.4. With this version, we’re shipping some major new features to help you rapidly build better machine learning models to deploy to production while increasing the robustness of the platform.
Data Science Updates
Redesign of the Solution README
With every trained machine learning model (Solution), you get information on the setup of the workflow, the performance of the model, as well as how to install and deploy your Solution. These files have all been merged to a single .html-file for you to have everything at your fingertips.
More Insights into the Solutions
In the Insights section in the Solution README, we now include useful plots to quickly assess the performance of the model on your validation data. For this release, we have added a Kernel Density Plot for regression tasks. This plot allows you to quickly identify for which true values the model is performing well and for which it doesn’t do well. For neural networks (classification and regression tasks) we also include a plot of the training vs. validation score to assert that the model does not overfit.
New Objective: F1 Score for Binary Classification
We ship a new objective with this release:the F1 Score for binary classification. The F1 Score is a robust score for unbalanced datasets, where one class has significantly fewer samples than the other(s), and trades off the precision and recall of the classification task. Besides the existing general F1 Score objective, we now also include an F1 Score to cover the special case of binary classification (two classes with e.g. “0” and “1”).
Autopausing of Workflows
Since the space of applicable AI models, which themselves may have additional parameters to tune, is vast, there is a potentially infinite number of model configurations. Efficient and smart sampling of this space is one of the core tasks of Modulos AutoML. One of the side effects of this is that the workflow is also infinite and so your harddrive will eventually be filled up with Solutions. To prevent this, AutoML now automatically pauses all workflows to prevent this from happening.
Other Improvements and Bug Fixes
We additionally made a range of other fixes and improvements to AutoML, which include:
- The Jupyter notebook provided with the Solution has been upgraded and, among other things, also displays sample images for prediction tasks on images.
- We have revised the rule, which assesses whether a model has been trained long enough, to yield a good tradeoff between maximizing validation scores and minimizing runtime, while shielding against the dangers of overfitting.
- We have increased the default number of workers to match our smallest available Modulos AutoML license. This increases the number of parallel workflows you can run.
- We have straightened out how temporary files created by a task, which failed during one of the steps within a workflow, are handled. This increases the robustness of the platform and will eventually allow the platform to recover from errors.
- The icons across the platform have been homogenized for a more consistent Modulos look and feel.