Sacred

| *Every experiment is sacred*
| *Every experiment is great*
| *If an experiment is wasted*
| *God gets quite irate*

|pypi| |py_versions| |license| |rtfd| |doi|

|build| |coverage| |code_quality| |black|

Sacred is a tool to help you configure, organize, log and reproduce experiments. It is designed to do all the tedious overhead work that you need to do around your actual experiment in order to:

• keep track of all the parameters of your experiment
• easily run your experiment for different settings
• save configurations for individual runs in a database
• reproduce your results

Sacred achieves this through the following main mechanisms:

• Config Scopes A very convenient way of the local variables in a function to define the parameters your experiment uses.
• Config Injection: You can access all parameters of your configuration from every function. They are automatically injected by name.
• Command-line interface: You get a powerful command-line interface for each experiment that you can use to change parameters and run different variants.
• Observers: Sacred provides Observers that log all kinds of information about your experiment, its dependencies, the configuration you used, the machine it is run on, and of course the result. These can be saved to a MongoDB, for easy access later.
• Automatic seeding helps controlling the randomness in your experiments, such that the results remain reproducible.

Example

+------------------------------------------------+--------------------------------------------+ | Script to train an SVM on the iris dataset | The same script as a Sacred experiment | +------------------------------------------------+--------------------------------------------+ | .. code:: python | .. code:: python | | | | | from numpy.random import permutation | from numpy.random import permutation | | from sklearn import svm, datasets | from sklearn import svm, datasets | | | from sacred import Experiment | | | ex = Experiment('iris_rbf_svm') | | | | | | @ex.config | | | def cfg(): | | C = 1.0 | C = 1.0 | | gamma = 0.7 | gamma = 0.7 | | | | | | @ex.automain | | | def run(C, gamma): | | iris = datasets.load_iris() | iris = datasets.load_iris() | | perm = permutation(iris.target.size) | per = permutation(iris.target.size) | | iris.data = iris.data[perm] | iris.data = iris.data[per] | | iris.target = iris.target[perm] | iris.target = iris.target[per] | | clf = svm.SVC(C=C, kernel='rbf', | clf = svm.SVC(C=C, kernel='rbf', | | gamma=gamma) | gamma=gamma) | | clf.fit(iris.data[:90], | clf.fit(iris.data[:90], | | iris.target[:90]) | iris.target[:90]) | | print(clf.score(iris.data[90:], | return clf.score(iri