autofit.DirectoryPaths#

class DirectoryPaths[source]#

Bases: AbstractPaths

Manages the path structure for NonLinearSearch output, for analyses both not using and using the search API. Use via non-linear searches requires manual input of paths, whereas the search API manages this using the search attributes.

The output path within which the Paths objects path structure is contained is set via PyAutoConf, using the command:

from autoconf import conf conf.instance = conf.Config(output_path=”path/to/output”)

If we assume all the input strings above are used with the following example names:

name = “name” path_prefix = “folder_0/folder_1”

The output path of the NonLinearSearch results will be:

/path/to/output/folder_0/folder_1/name

If the PYAUTO_TEST_MODE environment variable is set, a test_mode segment is inserted directly after the output root:

/path/to/output/test_mode/folder_0/folder_1/name

This keeps smoke-test artefacts in a sibling tree so they cannot be picked up by a later real run with the same paths.

Parameters:

name (Optional[str]) – The name of the non-linear search, which is used as a folder name after the path_prefix. For searchs this name is the name.
path_prefix (Optional[PathLike]) – A prefixed path that appears after the output_path but before the name variable.
is_identifier_in_paths – If True output path and symlink path terminate with an identifier generated from the search and model
parent (Optional[AbstractPaths]) – The parent paths object of this paths object.
unique_tag (Optional[str]) – A unique tag for the search, used to differentiate between searches with the same name.
identifier (str) – A custom identifier for the search, if this is not None it will be used instead of the automatically generated identifier
image_path_suffix (str) – A suffix which is appended to the image path. This is used to differentiate between different image outputs, for example the image of the starting point of an MLE.

Methods

`completed`	Mark the search as complete by saving a file
`create_child`	Create a paths object which is the child of some parent paths object.
`for_sub_analysis`	Paths for an analysis which is a child of another analysis.
`is_object`	Is there a file pickles/{name}.pickle?
`load_array`
`load_fits`	Load an HDU from a fits file in the fits directory of the search.
`load_json`
`load_latent_samples`
`load_object`	Load a serialised object with the given name.
`load_samples`	Load samples from the database
`load_samples_info`
`load_samples_summary`	Load samples summary from the database.
`load_search_internal`	Load the internal representation of a non-linear search from a pickle or dill file.
`output_model_results`
`remove_object`	Remove the object with the given name from the pickles folder.
`remove_search_internal`	Remove the internal representation of a non-linear search.
`restore`	Copy files from the `.zip` file to the samples folder.
`save_all`
`save_array`	Save a numpy array as a csv file in the csvs directory of the search.
`save_fits`	Save an HDU as a fits file in the fits directory of the search.
`save_identifier`
`save_json`	Save a dictionary as a json file in the jsons directory of the search.
`save_latent_samples`	Write out the latent variables of the model to a file.
`save_object`	Serialise an object using dill and save it to the pickles directory of the search.
`save_parent_identifier`
`save_samples`	Save the final-result samples associated with the phase as a pickle
`save_samples_summary`	Save samples summary to the database.
`save_search_internal`	Save the internal representation of a non-linear search as dill file.
`save_summary`
`save_unique_tag`
`zip_remove`	Copy files from the sym linked search folder then remove the sym linked folder.

Attributes

`identifier`
`image_path`	The path to the image folder.
`is_complete`	Has the search been completed?
`is_grid_search`	Is this a grid search which comprises a number of child searches?
`non_linear_name`
`output_path`	The path to the output information for a search.
`parent`	The search performed before this search.
`profile_path`	The path to the profile folder.
`samples`	Load the samples associated with the search from the output directory.
`search_internal_path`	The path to the samples folder.

save_object(name, obj, prefix='')[source]#

Serialise an object using dill and save it to the pickles directory of the search.

Parameters:

name (str) – The name of the object
obj (object) – A serialisable object
prefix (str) – A prefix to add to the path which is the name of the folder the file is saved in.

save_json(name, object_dict, prefix='')[source]#

Save a dictionary as a json file in the jsons directory of the search.

Parameters:

name – The name of the json file
object_dict (Union[dict, list]) – The dictionary to save
prefix (str) – A prefix to add to the path which is the name of the folder the file is saved in.

load_json(name, prefix='')[source]#

save_array(name, array)[source]#

Save a numpy array as a csv file in the csvs directory of the search.

Parameters:

name (str) – The name of the csv file
array (ndarray) – The numpy array to save

load_array(name)[source]#

save_fits(name, fits, prefix='')[source]#

Save an HDU as a fits file in the fits directory of the search.

Parameters:

name (str) – The name of the fits file
fits – The HDUList to save
prefix (str) – A prefix to add to the path which is the name of the folder the file is saved in.

load_fits(name, prefix='')[source]#

Load an HDU from a fits file in the fits directory of the search.

Parameters:

name (str) – The name of the fits file
prefix (str) – A prefix to add to the path which is the name of the folder the file is saved in.

Return type:

The loaded HDU.

load_object(name, prefix='')[source]#

Load a serialised object with the given name.

e.g. if the name is ‘model’ then pickles/model.pickle is loaded.

Parameters:

name (str) – The name of a serialised object
prefix (str) – A prefix to add to the path which is the name of the folder the file is saved in.

Return type:

The deserialised object

remove_object(name)[source]#

Remove the object with the given name from the pickles folder.

Parameters:: name (str) – The name of a pickle file excluding .pickle

is_object(name)[source]#: Is there a file pickles/{name}.pickle?

property is_complete: bool#: Has the search been completed?

save_search_internal(obj)[source]#

Save the internal representation of a non-linear search as dill file.

The results in this representation are required to use a search’s in-built tools for visualization, analysing samples and other tasks.

load_search_internal()[source]#

Load the internal representation of a non-linear search from a pickle or dill file.

The results in this representation are required to use a search’s in-built tools for visualization, analysing samples and other tasks.

Return type:: The results of the non-linear search in its internal representation.

remove_search_internal()[source]#

Remove the internal representation of a non-linear search.

This deletes the entire search_internal folder, including a .pickle / .dill file containing the interal results and files with the timer values.

This folder can often have a large filesize, thus deleting it can reduce hard-disk use of the model-fit.

completed()[source]#: Mark the search as complete by saving a file

load_samples()[source]#: Load samples from the database

property samples#: Load the samples associated with the search from the output directory.

save_latent_samples(latent_samples)[source]#

Write out the latent variables of the model to a file.

Parameters:: latent_samples – Samples describing the latent variables of the model

save_samples(samples)[source]#: Save the final-result samples associated with the phase as a pickle

save_samples_summary(samples_summary, name='samples_summary')[source]#: Save samples summary to the database.

load_samples_summary()[source]#: Load samples summary from the database.

load_latent_samples()[source]#

load_samples_info()[source]#

save_all(search_config_dict=None, info=None)[source]#

property parent: AbstractPaths#: The search performed before this search. For example, a search that is then compared to searches during a grid search.

save_parent_identifier()[source]#

save_unique_tag(is_grid_search=False)[source]#

property is_grid_search: bool#: Is this a grid search which comprises a number of child searches?

create_child(name=None, path_prefix=None, is_identifier_in_paths=None, identifier=None)[source]#

Create a paths object which is the child of some parent paths object. This is done during a GridSearch so that results can be stored in the correct directory.

Parameters:

name (Optional[str])
path_prefix (Optional[str])
is_identifier_in_paths (Optional[bool]) – If False then this path’s identifier will not be added to its output path.
identifier (Optional[str])

Return type:

A new paths object

for_sub_analysis(analysis_name)[source]#

Paths for an analysis which is a child of another analysis.

The analysis name forms a new directory on the end of the original analysis output path.