Releases: lightning-uq-box/lightning-uq-box
v0.2.0
What's Changed
The 0.2 release includes both new methods, as well as bug fixes. It is especially nice that we have three new contributors who have fixed bugs and improved the documentation. The release notes below mention some of the new highlights as well as central improvements to the library. Thank you to all contributors!
If you come across these release notes and are interested in becoming involved, see this issue for an overview of other UQ methods, or just get in touch with any other comments or questions via a new issue.
New Methods
We have added new UQ methods as well as extended some methods to new tasks. See the table below for an update of the currently supported UQ method and task combinations.
- Add Img2Img Conformal Prediction by @nilsleh in #118
- Extend other available methods for Pixelwise Regression by @nilsleh in #119
- Add Variational Bayesian Last Layer by @nilsleh in #137
- Add UQ-Method Masked Ensemble (Masksembles) by @nilsleh in #157
- Add ZigZag UQ Method by @nilsleh in #161
- Add VAE UQ Method by @nilsleh in #163
- Add Mixture Density Network by @nilsleh in #179
- Add Density Layer as UQ Method by @nilsleh in #181
Bug Fixes / Improvements
- Remove n_ensemble_members in DeepEnsemble signatures by @codingS3b in #151
- Improve Toy Heteroscedastic Datamodule and Tutorials by @nilsleh in #147
- Fix GMM equations for
DeepEnsembleRegression
by @elcorto in #156 - Update Segmentation models to save predictions by @nilsleh in #120
- Fix the loss function in probabilistic UNet.js by @qqaazz800624 in #135
- Remove unused figure in plot_calibration_uq_toolbox() by @elcorto in #141
- Add EMA support for CARD by @nilsleh in #187
- Deep Ensemble Eval Mode Fix by @nilsleh in #172
- Laplace improvements by @nilsleh in #138
- Refactor Posthoc by @nilsleh in #12
Installation
Lightning UQ Box is now also available for installation via conda and spack, thanks to @adamjstewart.
Linters and CI
We have switched to ruff (replacing balck, flake8, isort, pydocstyle, and pyupgrade) as our main linting tool, added dependabot to manage dependencies, and began work on static type hint testing, thanks @adamjstewart.
- Improve Test Coverage by @nilsleh in #121
- Use ruff by @nilsleh in #131
- Add dependabot by @nilsleh in #189
- mypy: fix test type hints by @adamjstewart in #224
- mypy: fix viz type hints by @adamjstewart in #225
- mypy: fix more test type hints by @adamjstewart in #226
- mypy: fix dataset type hints by @adamjstewart in #227
- mypy: fix eval_utils type hints by @adamjstewart in #228
Citation
If you find this library useful for your work, you can now cite our preprint.
- Add citation information by @adamjstewart in #252
New Contributors
- @qqaazz800624 made their first contribution in #135
- @elcorto made their first contribution in #141
- @codingS3b made their first contribution in #151
Other Contributors
Full Changelog: v0.1.0...v0.2.0
v0.1.0
This is the first release of the Lightning-UQ-Box.
The following table shows the current status of implemented methods.
Single Forward Pass Methods
Uncertainty Quantification Method (UQ-Method) | Regression | Classification | Segmentation | Pixel Wise Regression |
---|---|---|---|---|
Quantile Regression (QR) | ✅ | ❌ | ❌ | ⏳ |
Deep Evidential (DE) | ✅ | ⏳ | ⏳ | ⏳ |
Mean Variance Estimation (MVE) | ✅ | ❌ | ❌ | ⏳ |
Approximate Bayesian Methods
Uncertainty Quantification Method (UQ-Method) | Regression | Classification | Segmentation | Pixel Wise Regression |
---|---|---|---|---|
Bayesian Neural Network VI ELBO (BNN_VI_ELBO) | ✅ | ✅ | ✅ | ⏳ |
Bayesian Neural Network VI (BNN_VI) | ✅ | ⏳ | ⏳ | ⏳ |
Deep Kernel Learning (DKL) | ✅ | ✅ | ❌ | ❌ |
Deterministic Uncertainty Estimation (DUE) | ✅ | ✅ | ❌ | ❌ |
Laplace Approximation (Laplace) | ✅ | ✅ | ❌ | ❌ |
Monte Carlo Dropout (MC-Dropout) | ✅ | ✅ | ✅ | ⏳ |
Stochastic Gradient Langevin Dynamics (SGLD) | ✅ | ✅ | ⏳ | ⏳ |
Spectral Normalized Gaussian Process (SNGP) | ✅ | ✅ | ❌ | ❌ |
Stochastic Weight Averaging Gaussian (SWAG) | ✅ | ✅ | ✅ | ⏳ |
Deep Ensemble | ✅ | ✅ | ✅ | ⏳ |
Generative Models
Uncertainty Quantification Method (UQ-Method) | Regression | Classification | Segmentation | Pixel Wise Regression |
---|---|---|---|---|
Classification And Regression Diffusion (CARD) | ✅ | ✅ | ❌ | ❌ |
Probabilistic UNet | ❌ | ❌ | ✅ | ❌ |
Hierarchical Probabilistic UNet | ❌ | ❌ | ✅ | ❌ |
Post-Hoc methods
Uncertainty Quantification Method (UQ-Method) | Regression | Classification | Segmentation | Pixel Wise Regression |
---|---|---|---|---|
Test Time Augmentation (TTA) | ✅ | ✅ | ⏳ | ⏳ |
Temperature Scaling | ❌ | ✅ | ⏳ | ❌ |
Conformal Quantile Regression (Conformal QR) | ✅ | ❌ | ❌ | ⏳ |
Regularized Adaptive Prediction Sets (RAPS) | ❌ | ✅ | ❌ | ❌ |