Calculate map image classification efficacy (MICE) and other metrics using confusion matrix
Source:R/micer.R
miceCM.RdFor multiclass classification, returns a list object with the following items: $Mappings = class names; $confusionMatrix = confusion matrix where columns represent the reference data and rows represent the classification result; $referenceCounts = count of samples in each reference class; $predictionCounts = count of predictions in each class; $overallAccuracy = overall accuracy; $MICE = map image classification efficacy; $usersAccuracies = class-level user's accuracies (1 - commission error); $CTBICEs = classification-total-based image classification efficacies (adjusted user's accuracies); $producersAccuracies = class-level producer's accuracies (1 - omission error); $RTBICEs = reference-total-based image classification efficacies (adjusted producer's accuracies); $F1Scores = class-level harmonic mean of user's and producer's accuracies; $F1Efficacies = F1-score efficacies; $macroPA = class-aggregated, macro-averaged producer's accuracy; $macroRTBICE = class-aggregated, macro-averaged reference-total-based image classification efficacy; $macroUA = class-aggregated, macro-averaged user's accuracy; $macroCTBICE = class-aggregated, macro-averaged classification-total-based image classification efficacy; $macroF1 = class-aggregated, macro-averaged F1-score; $macroF1Efficacy = class-aggregated, macro-averaged F1 efficacy;
Arguments
- cm
confusion matrix as table object where rows define predictions and columns define reference labels.
- mappings
names of classes (if not provided, factor levels are used).
- multiclass
TRUE or FALSE. If TRUE, treats classification as multiclass. If FALSE, treats classification as binary. Default is TRUE.
- positiveIndex
index for positive case for binary classification. Ignored for multiclass classification. Default is 1 or first factor level.
Value
multiclass or binary assessment metrics in a list object. See details for description of generated metrics.
Details
For binary classification, returns a list object with the following items: $Mappings = class names; $confusionMatrix = confusion matrix where columns represent the reference data and rows represent the classification result; $referenceCounts = count of samples in each reference class; $predictionCounts = count of predictions in each class; $postiveCase = name or mapping for the positive case; $overallAccuracy = overall accuracy; $MICE = map image classification efficacy; $Precision = precision (1 - commission error relative to positive case); $precisionEfficacy = precision efficacy; $NPV = negative predictive value (1 - commission error relative to negative case); $npvEfficacy = negative predictive value efficacy; $Recall = recall (1 - omission error relative to positive case); $recallEfficacy = recall efficacy; $specificity = specificity (1 - omission error relative to negative case); $specificityEfficacy = specificity efficacy; $f1Score = harmonic mean of precision and recall; $f1Efficacy = F1-score efficacy;
Examples
#Multiclass example
data(mcData)
cmMC <- table(mcData$ref, mcData$pred)
miceCM(cmMC,
mappings=c("Barren", "Forest", "Impervious", "Low Vegetation", "Mixed Dev", "Water"),
multiclass=TRUE)
#> $Mappings
#> [1] "Barren" "Forest" "Impervious" "Low Vegetation"
#> [5] "Mixed Dev" "Water"
#>
#> $confusionMatrix
#> Reference
#> Predicted Barren Forest Impervious Low Vegetation Mixed Dev Water
#> Barren 75 13 10 63 1 1
#> Forest 7 20585 8 138 64 5
#> Impervious 59 62 196 34 75 0
#> Low Vegetation 46 617 33 2413 72 1
#> Mixed Dev 1 142 22 84 270 1
#> Water 6 21 12 1 2 158
#>
#> $referenceCounts
#> Barren Forest Impervious Low Vegetation Mixed Dev
#> 194 21440 281 2733 484
#> Water
#> 166
#>
#> $predictionCounts
#> Barren Forest Impervious Low Vegetation Mixed Dev
#> 163 20807 426 3182 520
#> Water
#> 200
#>
#> $overallAccuracy
#> [1] 0.9367144
#>
#> $MICE
#> [1] 0.7651436
#>
#> $usersAccuracies
#> Barren Forest Impervious Low Vegetation Mixed Dev
#> 0.4601227 0.9893305 0.4600939 0.7583281 0.5192308
#> Water
#> 0.7900000
#>
#> $CTBICEs
#> Barren Forest Impervious Low Vegetation Mixed Dev
#> 0.4559451 0.9300326 0.4540239 0.7290545 0.5098483
#> Water
#> 0.7886108
#>
#> $producersAccuracies
#> Barren Forest Impervious Low Vegetation Mixed Dev
#> 0.3865979 0.9601213 0.6975089 0.8829125 0.5578512
#> Water
#> 0.9518072
#>
#> $RTBICEs
#> Barren Forest Impervious Low Vegetation Mixed Dev
#> 0.3818514 0.7384867 0.6941081 0.8687298 0.5492225
#> Water
#> 0.9514884
#>
#> $f1Scores
#> Barren Forest Impervious Low Vegetation Mixed Dev
#> 0.4201680 0.9745071 0.5544554 0.8158918 0.5378486
#> Water
#> 0.8633879
#>
#> $f1Efficacies
#> Barren Forest Impervious Low Vegetation Mixed Dev
#> 0.4156219 0.8232649 0.5489642 0.7927871 0.5288035
#> Water
#> 0.8624267
#>
#> $macroPA
#> [1] 0.7394665
#>
#> $macroRTBUCE
#> [1] 0.6973145
#>
#> $macroUA
#> [1] 0.662851
#>
#> $macroCTBICE
#> [1] 0.6445859
#>
#> $macroF1
#> [1] 0.6990658
#>
#> $macroF1Efficacy
#> [1] 0.6699142
#>
#Binary example
data(biData)
cmB <- table(biData$ref, biData$pred)
miceMCResult <- miceCM(cmB,
mappings=c("Mined", "Not Mined"),
multiclass=FALSE,
positiveIndex=1)
print(miceMCResult)
#> $Mappings
#> [1] "Mined" "Not Mined"
#>
#> $confusionMatrix
#> Reference
#> Predicted Mined Not Mined
#> Mined 158 20
#> Not Mined 2 4820
#>
#> $referenceCounts
#> Mined Not Mined
#> 160 4840
#>
#> $predictionCounts
#> Mined Not Mined
#> 178 4822
#>
#> $positiveCase
#> [1] "Mined"
#>
#> $overallAccuracy
#> [1] 0.9956
#>
#> $mice
#> [1] 0.9289543
#>
#> $Precision
#> [1] 0.8876404
#>
#> $precisionEfficacy
#> [1] 0.8839248
#>
#> $NPV
#> [1] 0.8876404
#>
#> $npvEfficacy
#> [1] 0.8839248
#>
#> $Recall
#> [1] 0.9874999
#>
#> $recallEfficacy
#> [1] 0.9870866
#>
#> $Specificity
#> [1] 0.9874999
#>
#> $specificityEfficicacy
#> [1] 0.9870866
#>
#> $f1Score
#> [1] 0.9349112
#>
#> $f1ScoreEfficacy
#> [1] 0.9326617
#>