Evaluation Metrics (anglepy.metrics)¶

This module implements evaluation metrics for circular data.

anglepy.metrics.accuracy(y_pred, y_test, theta=0.5235987755982988)¶

Computes the threshold-based accuracy for angular predictions.

This metric evaluates the proportion of predictions that fall within a specified angular tolerance (theta) of the ground truth. It safely handles circular wrapping by calculating the shortest angular path (e.g., the distance between \(359^\circ\) and \(1^\circ\) is \(2^\circ\), not \(358^\circ\)) before evaluating against the threshold.

Parameters:

y_pred (torch.Tensor) – A tensor of predicted angles in radians.
y_test (torch.Tensor) – A tensor of ground truth angles in radians. Must be broadcastable to the shape of y_pred.
theta (float, optional) – The maximum allowed angular deviation (in radians) for a prediction to be considered “correct”. Defaults to \(\pi/6\) (30 degrees).

Returns:

The fraction of predictions that are within the theta threshold. Returns a value between 0.0 (no correct predictions) and 1.0 (all correct).

Return type:

float

Example

>>> # pi/6 is approx 0.523 radians (30 degrees)
>>> y_pred = torch.tensor([0.0, 3.14159, 6.28])     # [0, pi, ~2*pi]
>>> y_test = torch.tensor([0.2, 0.0, 0.0])          # [0.2, 0, 0]
>>> # Distances:
>>> # 1: |0.0 - 0.2| = 0.2 rad (<= 0.523) -> Correct
>>> # 2: |pi - 0.0| = 3.14 rad (> 0.523) -> Incorrect
>>> # 3: |~2*pi - 0| = ~0.0 rad (<= 0.523) -> Correct (due to wrapping)
>>> accuracy(y_pred, y_test, theta=torch.pi/6)

anglepy.metrics.angular_distance(a, b)¶: Calculates the shortest angular distance between two angles in radians.

anglepy.metrics.circular_crps(y_pred_ensemble, y_true)¶

Calculates the Circular Continuous Ranked Probability Score (CRPS) for an ensemble of predictions.

CRPS is a strictly proper scoring rule used to evaluate both the accuracy and calibration of probabilistic forecasts. This implementation adapts standard CRPS for circular or angular variables by using the shortest angular distance between points instead of the standard absolute difference (Gneiting, 2007).

The empirical circular CRPS is calculated as:

\[CRPS = \frac{1}{M} \sum_{i=1}^{M} d(\hat{y}_i, y) - \frac{1}{2 M^2} \sum_{i=1}^{M} \sum_{j=1}^{M} d(\hat{y}_i, \hat{y}_j)\]

where \(y\) is the true observation, \(\hat{y}_i\) is an ensemble member, \(M\) is the total number of ensemble members, and \(d(\cdot, \cdot)\) is the angular distance.

Parameters:

y_pred_ensemble (torch.Tensor or array-like) – The ensemble of predicted angles in radians. Expected shapes are \((N, M)\) for univariate predictions or \((N, D, M)\) for multivariate predictions, where \(N\) is the number of observations, \(D\) is the number of feature dimensions, and \(M\) is the number of ensemble members.
y_true (torch.Tensor or array-like) – The ground truth angles in radians. Expected shapes are \((N,)\) for univariate targets or \((N, D)\) for multivariate targets. It is automatically converted to a tensor and moved to the same device/dtype as y_pred_ensemble.

Returns:

A tensor containing the computed circular CRPS values. The shape exactly matches the shape of the input y_true (\((N,)\) or \((N, D)\)).

Return type:

torch.Tensor

Raises:

ValueError – If the number of observations (\(N\)) or the feature dimension size (\(D\)) of y_true does not match the corresponding dimensions of y_pred_ensemble.

Example

>>> # Univariate example: #Observations N=2, Ensemble size M=3
>>> y_true = torch.tensor([0.0, 3.14159])                  # True angles: 0, pi
>>> y_pred = torch.tensor([[0.1, -0.1, 0.0],               # Predictions close to 0
...                        [3.0, -3.0, 3.14159]])          # Predictions close to pi
>>> crps = circular_crps(y_pred, y_true)
>>> crps.shape
torch.Size([2])

>>> # Multivariate example: #Observations N=2, Features D=2, Ensemble size M=5
>>> y_true_mv = torch.zeros(2, 2)
>>> y_pred_mv = torch.zeros(2, 2, 5)
>>> crps_mv = circular_crps(y_pred_mv, y_true_mv)
>>> crps_mv.shape
torch.Size([2, 2])

anglepy.metrics.circular_mean_directional_error(y_pred, y_target, is_degrees=False)¶

Calculates the Circular Mean Directional Error between predicted and target angles.

This metric evaluates the accuracy of angular predictions by projecting the angular difference into a continuous cosine space. This naturally handles circular wrapping (e.g., the distance between 359 degrees and 1 degree). The resulting error is bounded between 0 (perfect alignment) and 2 (completely opposite directions).

The calculation is formally defined as:

\[1 - \frac{1}{N} \sum_{i=1}^{N} \cos(y_{\text{target}, i} - y_{\text{pred}, i})\]

Parameters:

y_pred (torch.Tensor) – A tensor of predicted angles.
y_target (torch.Tensor) – A tensor of ground truth target angles. Must be broadcastable to the shape of y_pred. It is automatically moved to the same device as y_pred.
is_degrees (bool, optional) – If True, indicates that the input tensors are in degrees and automatically converts them to radians before calculation. Defaults to False (assumes inputs are in radians).

Returns:

A scalar tensor containing the mean circular directional error.

Return type:

torch.Tensor

Example

>>> y_pred = torch.tensor([0.0, 3.14159])      # [0, pi]
>>> y_target = torch.tensor([0.0, 0.0])        # [0, 0]
>>> # cos(0 - 0) = 1, cos(0 - pi) = -1
>>> # mean(1, -1) = 0.0 -> 1.0 - 0.0 = 1.0
>>> circular_mean_directional_error(y_pred, y_target)
tensor(1.)

>>> # Using degrees
>>> y_pred_deg = torch.tensor([10.0, 350.0])
>>> y_target_deg = torch.tensor([10.0, 10.0])
>>> circular_mean_directional_error(y_pred_deg, y_target_deg, is_degrees=True)
tensor(0.0302)

anglepy.metrics.mean_absolute_angular_deviation(y_pred, y_target, is_degrees=False, return_degrees=True)¶

Calculates the Mean Absolute Angular Deviation (MAAD) between predicted and target angles.

This metric computes the shortest angular distance between two angles. It safely handles circular wrapping by projecting the angular difference into Cartesian coordinates (using sine and cosine) and using arctangent to map the error back to the \([-\pi, \pi]\) range. It then takes the mean of the absolute deviations.

Parameters:

y_pred (torch.Tensor) – A tensor of predicted angles.
y_target (torch.Tensor) – A tensor of ground truth target angles. Must be broadcastable to the shape of y_pred. It is automatically moved to the same device as y_pred.
is_degrees (bool, optional) – If True, indicates that the input tensors (y_pred and y_target) are in degrees. Defaults to False (radians).
return_degrees (bool, optional) – If True, the final computed mean deviation is returned in degrees. If False, it is returned in radians. Defaults to True.

Returns:

A scalar tensor containing the mean absolute angular deviation.

Return type:

torch.Tensor

Example

>>> # The naive difference between 350° and 10° is 340°.
>>> # The shortest angular path is actually 20°.
>>> y_pred = torch.tensor([350.0])
>>> y_target = torch.tensor([10.0])
>>> mean_absolute_angular_deviation(y_pred, y_target, is_degrees=True, return_degrees=True)
tensor(20.)

>>> # Using radians, returning radians
>>> y_pred_rad = torch.tensor([1.5 * 3.14159])  # ~270 degrees
>>> y_target_rad = torch.tensor([0.0])          # 0 degrees
>>> mean_absolute_angular_deviation(y_pred_rad, y_target_rad, is_degrees=False, return_degrees=False)
tensor(1.5708)  # ~pi/2 radians (90 degrees)

anglepy.metrics.mean_circular_crps(y_pred_ensemble, y_true, return_degrees=True)¶

Computes the scalar mean circular CRPS over all observations and response dimensions.

This function acts as a wrapper around circular_crps. It calculates the individual Continuous Ranked Probability Score (CRPS) for each observation and dimension, aggregates them into a single scalar mean, and optionally converts the results from radians to degrees.

Parameters:

y_pred_ensemble (torch.Tensor or array-like) – The ensemble of predicted angles in radians. Expected shapes are \((N, M)\) for univariate or \((N, D, M)\) for multivariate predictions.
y_true (torch.Tensor or array-like) – The ground truth angles in radians. Expected shapes are \((N,)\) or \((N, D)\).
return_degrees (bool, optional) – If True, converts both the final mean scalar and the raw CRPS tensor from radians to degrees before returning. Defaults to True.

Returns:

A tuple containing:

mean_crps (float): The aggregated mean circular CRPS across all observations and dimensions, returned as a standard Python float.
crps_tensor (torch.Tensor): The raw tensor of CRPS scores corresponding to each observation. Shape matches y_true.

Return type:

tuple

Example

>>> # #Observations N=2, Ensemble size M=3
>>> y_true = torch.tensor([0.0, 3.14159])                  # [0, pi]
>>> y_pred = torch.tensor([[0.1, -0.1, 0.0],               # Predictions close to 0
...                        [3.0, -3.0, 3.14159]])          # Predictions close to pi
>>> mean_score, raw_scores = mean_circular_crps(y_pred, y_true, return_degrees=True)
>>> type(mean_score)
<class 'float'>
>>> raw_scores.shape
torch.Size([2])

anglepy.metrics.med_err(y_pred, y_test, return_degrees=True)¶

Calculates the median absolute angular error between predicted and target angles.

This metric evaluates the central tendency of the prediction errors. It safely handles circular wrapping by computing the shortest angular path (e.g., the distance between \(2\pi\) and \(0.1\) radians is \(0.1\) radians) before finding the median across the entire batch. Using the median makes this metric robust to outliers compared to mean-based metrics.

Parameters:

y_pred (torch.Tensor) – A tensor of predicted angles in radians.
y_test (torch.Tensor) – A tensor of ground truth angles in radians. Must be broadcastable to the shape of y_pred. It is automatically moved to the same device as y_pred.
return_degrees (bool, optional) – If True, converts the final computed median error from radians to degrees. If False, returns the error in radians. Defaults to True.

Returns:

The median absolute angular error as a Python float.

Return type:

float

Example

>>> y_pred = torch.tensor([0.0, 3.14159, 6.28])     # [0, pi, ~2*pi]
>>> y_test = torch.tensor([0.1, 3.14159, 0.0])      # [0.1, pi, 0]
>>> med_err(y_pred, y_test, return_degrees=True)