gmtorch_pse package

Submodules

gmtorch_pse._gaussian_mixture module

class gmtorch_pse._gaussian_mixture.GaussianMixture(n_components: int, n_dimensions: int, covariance_init: Tensor | None, weights_init: Tensor | None, means_init: Tensor | None, random_state: int | None, verbose: bool, verbose_interval: int, device: str = 'device')

Bases: object

Gaussian Mixture.

Representation of a Gaussian mixture model probability distribution. This class allows to estimate the parameters of a Gaussian mixture distribution.

Parameters:

• n_components (int, default=1) – The number of mixture components.

• n_dimensions (int, default=1)

• gaussian. (The number of dimensions of each)

• weights_init ((torch.Tensor) a 1D-Tensor containing the weights as floats, default=None) – The user-provided initial weights. If it is None, weights are initialized using the _initialize_parameters method.

• means_init (2D-Tensor with all the means of the components, default=None) – The user-provided initial means, If it is None, means are initialized using the _initialize_parameters method.

• verbose (bool, default=false) – Enable verbose output. If yes then it prints the current initialization and each iteration step(also the log probability and the time needed for each step). If false no output will be printed

• verbose_interval (int, default=10) – Number of iteration done before the next print.

Examples

#TODO add examples

_calc_bic()

Computes the Bayesian Information Criterion (BIC) for the current model.

The BIC is a criterion for model selection among a finite set of models. It introduces a penalty term for the number of parameters in the model to prevent overfitting. The model with the lowest BIC is generally preferred.

Returns:

The calculated BIC score (scalar).

Return type:

torch.Tensor

_calc_overfitting()

Calculates the penalty term representing model complexity.

This method counts the total number of free parameters (k) in the GMM and scales it by the natural logarithm of the sample size (n).

Returns:

The complexity penalty term.

Return type:

torch.Tensor

_check_parameters()

subroutine to check all parameters for GMM :returns: None

_e_step()

Calculates responsibility of each data point

_em()

Runs the Expectation-Maximization (EM) algorithm loop.

This method initializes the covariance matrices and iteratively performs the E-step and M-step to optimize the model parameters. It supports two fitting modes based on “self.d_fitting”:

1. Static Fitting: Runs for a fixed number of iterations defined by “self.em_count”.

2. Dynamic Fitting: Runs until the change in the Bayesian Information Criterion (BIC) is less than 1e-4 or a maximum of 1000 iterations is reached.

Returns:

• Returns the final BIC score (float) if dynamic fitting is enabled.

• Returns 0 (int) if static fitting is used.

Return type:

float or int

_initialize_parameters()

Initializes missing parameters and device settings.

This method sets up the initial state of gmm if parameters were not provided by the user:

weights: defaults to a uniform distribution (1/n_components). device: automatically detects the available accelerator (CUDA)

or defaults to CPU if not specified.

covariances: defaults to identity matrices for each component. means: defaults to zero vectors for each component. random state: sets the torch manual seed if an integer is provided.

_knn_means()

Runs the k-Means algorithm to initialize the cluster means.

This method serves as a heuristic initialization step. It starts with randomly initialized centroids and iteratively refines their positions by minimizing the Euclidean distance to the data points over a fixed number of iterations (‘self.k_count’).

The resulting centroids are stored in ‘self.means’ to be used as starting parameters for the Gaussian Mixture Model.

_m_step(responsibilities)

Performs the Maximization step (M-step) of the EM algorithm.

Updates the model parameters (mixture weights, means, and covariance matrices) by maximizing the expected log-likelihood based on the posterior probabilities (responsibilities) calculated in the E-step.

Parameters:

responsibilities (torch.Tensor) – A tensor of shape (n_samples, n_components) containing the soft assignments (posterior probabilities) of each data point to each cluster.

Side Effects:

Updates self.weights, self.means, and self.covariance in-place.

Implementation Details:

• Covariance Calculation: Uses a square-root weighting trick (sqrt_resp) before matrix multiplication. This improves numerical stability and allows efficient vectorization on the GPU.

• Regularization: Adds a small constant (EPSILON) to the diagonal of the covariance matrices to prevent singularities.

_new_means(assigned_mask: Tensor)

Updates the cluster centroids based on the assigned data points.

This performs the ‘maximization’ step of K-Means. It aggregates all data points assigned to a specific cluster and calculates their new mean center.

It utilizes vectorized operations (’index_add_’) for efficiency on the GPU and includes a safeguard against division-by-zero errors for empty clusters.

Parameters:

assigned_mask (torch.Tensor) – A tensor containing the cluster index (0 to n_components-1) for each data point.

_save_fitting_log()

conditioning(given_dimensions, given_values)

Calculates the conditional Gaussian Mixture Model (GMM) given specific observed values.

Mathematical Formulation (see “Pattern Recognition and Machine Learning” by Christopher Bishop (2006), pages 85f and “Probabilistic Machine Learning: An Introduction” by Kevin P. Murphy (2022), pages 87f):

Let A be the indices of the unknown dimensions and B be the indices of the observed dimensions.

1. New Means (Conditional Mean): mu_{A|B} = mu_A + Sigma_{AB} @ Sigma_{BB}^{-1} @ (x_B - mu_B)

2. New Covariances (Conditional Covariance): Sigma_{A|B} = Sigma_{AA} - Sigma_{AB} @ Sigma_{BB}^{-1} @ Sigma_{BA}

3. New Weights (Bayesian Update): w_{new} = softmax(log(w_{old}) + log(N(x_B | mu_B, Sigma_{BB})))

Parameters:

• given_dimensions (torch.Tensor) – A 1D-Tensor containing the indices of the fixed dimensions (x_B).

• given_values (torch.Tensor) – A 1D-Tensor containing the observed values for these dimensions.

Returns:

A new instance of GaussianMixture representing the distribution

over the remaining dimensions (x_A).

Return type:

GaussianMixture

expectation_average_of_means()

Calculates the global expected value (overall mean) of the Gaussian Mixture Model.

This method computes the weighted sum of the means of all components, representing the center of mass of the entire distribution.

Returns:

A 1D-Tensor of shape (n_dimensions,) containing the global expected mean.

Return type:

torch.Tensor

fit(dynamic_fitting=False, log_fitting=False)

Creates gaussian distributions for given datapoints.

Overwrites covariance, means and weights

classmethod from_default()

classmethod from_random(random_state: int, max_components=100, max_dimensions=100)

random_stateint, RandomState instance or None, default=None

Controls the random seed given to the method chosen to initialize the parameters (see ‘_initialize_parameters’). In addition, it controls the generation of random samples from the fitted distribution (see the method ‘sample’). Pass an int for reproducible output across multiple function calls.

classmethod from_user(n_components: int, n_dimensions: int, weights_init: Tensor, means_init: Tensor, covariances_init: Tensor)

generate_samples(n_samples=1, return_indices: bool = False)

Generates random samples from the fitted Gaussian Mixture Model.

The generation process works in two steps: 1. Determines which component k a sample belongs to, based on the distribution defined by

the mixture weights.

2. Generates the actual data point from the multivariate Gaussian distribution of the selected component k.

The resulting samples are shuffled to ensure random ordering, as the generation process initially groups samples by component.

Parameters:

• n_samples (int) – The total number of samples to generate. Must be a positive integer. Defaults to 1.

• return_indices (bool) – If True, the method returns a tuple containing the samples and the component indices that generated them. Defaults to False.

load_data_points(data_points)

marginalization(remaining_dimensions)

Calculates the marginalized GMM given certain dimensions that shall remain from the current GMM instance.

In a GMM there is no integration needed to marginalize a dimension, since removing certain rows/columns from the means and covariance tensor is mathematically identical to performing the integral.

Parameters:

remaining_dimensions (torch.Tensor) – A 1D-Tensor containing the dimensions that should persist

Returns:

marginalized GMM instance

Return type:

GaussianMixture

gmtorch_pse._gaussian_mixture._assert_finite(t: Tensor, variable_name: str) → None

Ensure that the given tensor does not contain NaN or infinite values.

Raises:

ValueError – If any element is NaN or +/-inf.

gmtorch_pse._gaussian_mixture._check_covariance(covariance: Tensor, n_components: int, n_dimensions: int)

A function to check entries of the covariance matrix. :param covariance: the covariance matrix :param n_components: number of gauss :param n_dimensions: dimensional count

Returns:

None

gmtorch_pse._gaussian_mixture._check_data_points(data_points: Tensor, n_dimensions: int)

A function to check the data_points for valid data. :param data_points: A 2D-Tensor with all the means of the components :param n_dimensions: the number of dimensions

Returns:

None

gmtorch_pse._gaussian_mixture._check_datatype(var: int, variable_name: str)

helper function to check constructor parameters of type int for valid content :param var: the variable that will be checked :param variable_name: the name of the variable for the error message.

Returns:

None

gmtorch_pse._gaussian_mixture._check_means(mean: Tensor, n_components: int, n_dimensions: int)

A function to check the means for valid data. :param mean: A 2D-Tensor with all the means of the components :param n_components: the number of components :param n_dimensions: the number of dimensions

Returns:

None

gmtorch_pse._gaussian_mixture._check_weights(weights: Tensor, n_components: int) → Tensor

Validate the weights for a GMM. :param weights: a 1D-Tensor containing the weights as floats. :type weights: torch.Tensor :param n_components: the expected number of components. :type n_components: int

Returns:

weights. If all weights add up to 1 it will return the original weights. If not, it will normalize weights and return the new weights.

Module contents