Skip to content

Latest commit

 

History

History
23 lines (17 loc) · 1.74 KB

bayesian_segnet.md

File metadata and controls

23 lines (17 loc) · 1.74 KB

June 2019

tl;dr: Estimate the variance of segmentation uncertainty with dropout inference samples. Use the mean for prediction. The idea is quite similar to TTA (test time augmentation).

Overall impression

The paper provides a practical way to evaluate the uncertainty (this is the epistemic uncertainty), at a cost at inference time. Refer to Bayesian DL for integration with aleatoric uncertainty.

Key ideas

  • Sampling with dropout performs better than weight averaging (normal dropout behavior during eval). Sampling dropout performs better than weight averaging after approximation with 6 samples. The performance saturates with 40 samples.
    • This comes at inference time cost, but is naively parallelizable.
  • The results also show that when the model predicts an incorrect label the model uncertainty is very high.
  • Class boundaries usually display high level of uncertainty.
  • Objects that are occluded or at a distance from the camera are are uncertain.
  • The uncertainty score is inversely proportional to occurrence and accuracy. The model is more confident about classes which are easier and occur more often.
  • The accuracy improves when we use a tighter threshold to filter out non-confident results. Uncertainty is an effective measure of accuracy.

Technical details

  • No need to use dropout layer after every layer. Get the optimal architecture first by test placing dropout in different places. Then keep using dropout during inference (variational inference).

Notes