autofit.Analysis#

class Analysis[source]#

Bases: ABC

Protocol for an analysis. Defines methods that can or must be implemented to define a class that compute the likelihood that some instance fits some data.

Methods

`compute_latent_samples`	Compute latent-variable samples for every posterior sample.
`compute_latent_variables`	Override to compute latent variables from the instance.
`fit_for_visualization`	Build the fit used by the visualizer.
`log_likelihood_function`
`make_result`	Returns the Result of the non-linear search after it is completed.
`modify_after_fit`	Overwrite this method to modify the attributes of the Analysis class before the non-linear search begins.
`modify_before_fit`	Overwrite this method to modify the attributes of the Analysis class before the non-linear search begins.
`modify_model`
`perform_quick_update`
`print_vram_use`	Print JAX VRAM use for a given batch size.
`save_attributes`
`save_results`
`save_results_combined`
`shared_state_from`	Optionally compute a per-evaluation object that is shared across the factors of a FactorGraphModel.
`with_model`	Associate an explicit model with this analysis.

Attributes

`LATENT_BATCH_MODE`
`LATENT_KEYS`
`supports_background_update`	Whether this analysis supports background quick updates.
`supports_jax_visualization`	Whether the visualizer can work directly with JAX arrays.

class Result#

Bases: AbstractResult

The result of a non-linear search.

The default behaviour is for all key results to be in the samples_summary attribute, which is a concise summary of the results of the non-linear search. The reasons for this to be the main attribute are:

It is concise and therefore has minimal I/O overhead, which is important because when runs are resumed

the results are loaded often, which can become very slow for large results via a samples.csv.

The output.yaml config files can be used to disable the output of the samples.csv file

and search_internal.dill files. This means in order for results to be loaded in a way that allows a run to resume, the samples_summary must contain all results necessary to resume the run.

For this reason, the samples and search_internal attributes are optional. On the first run of a model-fit, they will always contain values as they are passed in via memory from the results of the search. However, if a run is resumed they are no longer available in memory, and they will only be available if their corresponding samples.csv and search_internal.dill files are output on disk and available to load.

This object includes:

The samples_summary attribute, which is a summary of the results of the non-linear search.
The paths attribute, which contains the path structure to the results of the search on the hard-disk and

is used to load the samples and search internal attributes if they are required and not available in memory.

The samples of the non-linear search (E.g. MCMC chains, nested sampling samples) which are used to compute

the maximum likelihood model, posteriors and other properties.

The non-linear search used to perform the model fit in its internal format (e.g. the Dynesty sampler used

by dynesty itself as opposed to PyAutoFit abstract classes).

Parameters:

samples_summary (SamplesSummary) – A summary of the most important samples of the non-linear search (e.g. maximum log likelihood, median PDF).
paths (Optional[AbstractPaths]) – The paths to the results of the search, used to load the samples and search internal attributes if they are required and not available in memory.
samples (Optional[Samples]) – The samples of the non-linear search, for example the MCMC chains.
search_internal (Optional[object]) – The non-linear search used to perform the model fit in its internal format.
analysis (Optional[Analysis]) – The Analysis object that was used to perform the model-fit from which this result is inferred.

dict()#: Human-readable dictionary representation of the results

property model#

property projected_model: AbstractPriorModel#: Create a new model with the same structure as the previous model, replacing each prior with a new prior created by calculating sufficient statistics from samples and corresponding weights for that prior.

property samples: Samples | None#

Returns the samples of the non-linear search, for example the MCMC chains or nested sampling samples.

When a model-fit is run the first time, the samples are passed into the result via memory and therefore always available.

However, if a model-fit is resumed the samples are not available in memory and they only way to load them is via the samples.csv file output on the hard-disk. This property handles the loading of the samples from the samples.csv file if they are not available in memory.

Return type:: The samples of the non-linear search.

property search_internal#

Returns the non-linear search used to perform the model fit in its internal sampler format.

When a model-fit is run the first time, the search internal is passed into the result via memory and therefore always available.

However, if a model-fit is resumed the search internal is not available in memory and they only way to load it is via the search_internal.dill file output on the hard-disk. This property handles the loading of the search internal from the search_internal.dill file if it is not available in memory.

Return type:: The non-linear search used to perform the model fit in its internal sampler format.

class Visualizer#

Bases: object

should_visualize(paths, during_analysis=True)#

Whether a visualize method should be called perform visualization, which depends on the following:

1) If a model-fit has already completed, the default behaviour is for visualization to be bypassed in order to make model-fits run faster.

2) If a model-fit has completed, but it is the final visualization output where during_analysis is False, it should be performed.

3) Visualization can be forced to run via the force_visualization_overwrite, for example if a user wants to plot additional images that were not output on the original run.

If the analysis is running a database session visualization is switched off.

5) If PyAutoFit test mode is on visualization is disabled, irrespective of the force_visualization_overwite config input.

Parameters:: paths (AbstractPaths) – The paths object which manages all paths, e.g. where the non-linear search outputs are stored, visualization and the pickled objects used by the aggregator output by this function.
Return type:: A bool determining whether visualization should be performed or not.

static visualize(analysis, paths, instance, during_analysis)#

static visualize_before_fit(analysis, paths, model)#

static visualize_before_fit_combined(analyses, paths, model)#

static visualize_combined(analyses, paths, instance, during_analysis, quick_update=False)#

class Latent#

Bases: object

Latent-variable extension point, declared on an analysis as Analysis.Latent = MyLatent (mirrors Visualizer / Result).

Subclass and override keys and variables to define a catalogue of latent variables. All methods are @staticmethod and take the analysis as their first argument, so per-fit state (e.g. analysis.kwargs["magzero"]) is reachable without an instance lifecycle.

The base implementation is a backwards-compatibility shim: it reads the legacy Analysis.LATENT_KEYS and Analysis.compute_latent_variables, so existing analyses that override those continue to work without declaring a Latent. New code should subclass this class instead.

BATCH_MODE = None#

static keys(analysis)#: The ordered list of enabled latent names (back-compat: LATENT_KEYS).

static variables(analysis, parameters, model)#: Compute the latent values for one sample, returned as a tuple/dict positionally aligned with keys. Back-compat default delegates to analysis.compute_latent_variables.

LATENT_KEYS = []#

LATENT_BATCH_MODE = 'vmap'#

fit_for_visualization(instance)[source]#

Build the fit used by the visualizer.

Delegates to self.fit_from(instance). When use_jax=True, the profile evaluations inside fit_from dispatch to JAX via the decorator chain. The per-function JIT caches warm up on the first call and are reused on all subsequent quick updates.

compute_latent_samples(samples, batch_size=None)[source]#

Compute latent-variable samples for every posterior sample.

Thin wrapper around autofit.non_linear.analysis.latent.latent_samples_from, which reads which latents to compute (and how) from self.Latent (see Latent). Kept as a method for backwards compatibility — it is the entry point called by SearchUpdater._compute_latent_samples.

compute_latent_variables(parameters, model)[source]#

Override to compute latent variables from the instance.

Latent variables are expressed as a dictionary: {“name”: value}

More complex models can be expressed by separating variables names by ‘.’ {“name.attribute”: value}

Parameters:: instance – An instance of the model.
Return type:: The computed latent variables.

with_model(model)[source]#

Associate an explicit model with this analysis. Instances of the model will be used to compute log likelihood in place of the model passed from the search.

Parameters:: model – A model to associate with this analysis
Return type:: An analysis for that model

log_likelihood_function(instance, shared=None)[source]#

shared_state_from(instance)[source]#

Optionally compute a per-evaluation object that is shared across the factors of a FactorGraphModel.

This is the per-evaluation, cross-factor sibling of modify_before_fit. Where modify_before_fit runs once before sampling to precompute analysis state that does not depend on the model, shared_state_from runs once per likelihood evaluation (the model parameters change every sample) and computes state that is identical for every factor at the current point in parameter space.

When a FactorGraphModel evaluates its likelihood it calls this method on its lead factor’s Analysis. If the returned value is not None it is forwarded as the shared keyword argument to every factor’s log_likelihood_function, so that work which is identical for all factors (because they share model parameters) is computed once and reused rather than recomputed N times.

The default implementation returns None, meaning no state is shared and every factor’s log_likelihood_function runs exactly as it does without this mechanism. An Analysis opts in by overriding this method.

The returned object must be a valid JAX pytree of traced arrays when the fit is JIT-compiled: it is recomputed inside the jitted region each evaluation (it depends on the traced model parameters) and must not be memoised on the instance.

Correctness is the responsibility of the overriding Analysis: only return a shared object when the parameters it depends on really are shared across every factor. If they are not, return None and let each factor compute its own state.

Parameters:: instance – The model instance of the factor whose Analysis is acting as the lead.
Return type:: An object shared across all factors for this evaluation, or None for no sharing.

save_attributes(paths)[source]#

save_results(paths, result)[source]#

save_results_combined(paths, result)[source]#

modify_before_fit(paths, model)[source]#

Overwrite this method to modify the attributes of the Analysis class before the non-linear search begins.

An example use-case is using properties of the model to alter the Analysis class in ways that can speed up the fitting performed in the log_likelihood_function.

modify_model(model)[source]#

modify_after_fit(paths, model, result)[source]#

Overwrite this method to modify the attributes of the Analysis class before the non-linear search begins.

An example use-case is using properties of the model to alter the Analysis class in ways that can speed up the fitting performed in the log_likelihood_function.

make_result(samples_summary, paths, samples=None, search_internal=None, analysis=None)[source]#

Returns the Result of the non-linear search after it is completed.

The result type is defined as a class variable in the Analysis class. It can be manually overwritten by a user to return a user-defined result object, which can be extended with additional methods and attributes specific to the model-fit.

The standard Result object may include:

The samples summary, which contains the maximum log likelihood instance and median PDF model.
The paths of the search, which are used for loading the samples and search internal below when a search

is resumed.

The samples of the non-linear search (e.g. MCMC chains) also stored in samples.csv.
The non-linear search used for the fit in its internal representation, which is used for resuming a search

and making bespoke visualization using the search’s internal results.

The analysis used to fit the model (default disabled to save memory, but option may be useful for certain

projects).

Parameters:

samples_summary (SamplesSummary) – The summary of the samples of the non-linear search, which include the maximum log likelihood instance and median PDF model.
paths (AbstractPaths) – An object describing the paths for saving data (e.g. hard-disk directories or entries in sqlite database).
samples (Optional[SamplesPDF]) – The samples of the non-linear search, for example the chains of an MCMC run.
search_internal (Optional[object]) – The internal representation of the non-linear search used to perform the model-fit.
analysis (Optional[object]) – The analysis used to fit the model.

Returns:

The result of the non-linear search, which is defined as a class variable in the Analysis class.

Return type:

Result

property supports_background_update: bool#: Whether this analysis supports background quick updates.

property supports_jax_visualization: bool#: Whether the visualizer can work directly with JAX arrays.

perform_quick_update(paths, instance)[source]#

print_vram_use(model, batch_size)[source]#

Print JAX VRAM use for a given batch size.

Parameters:: batch_size (int) – The batch size to profile, which is the number of model evaluations JAX will perform simultaneously.