Skip to contents

Calculate reliability measures based on content analysis

Source: R/aux_fct.R
get_coder_metrics.Rd

This function calculates different reliability measures which are based on the empirical research method of content analysis.

Usage

get_coder_metrics(
  true_values = NULL,
  predicted_values = NULL,
  return_names_only = FALSE
)

Arguments

true_values

factor containing the true labels/categories.

predicted_values

factor containing the predicted labels/categories.

return_names_only

bool If TRUE returns only the names of the resulting vector. Use FALSE to request computation of the values.

Value

If return_names_only=FALSE returns a vector with the following reliability measures: #'

  • iota_index: Iota Index from the Iota Reliability Concept Version 2.

  • min_iota2: Minimal Iota from Iota Reliability Concept Version 2.

  • avg_iota2: Average Iota from Iota Reliability Concept Version 2.

  • max_iota2: Maximum Iota from Iota Reliability Concept Version 2.

  • min_alpha: Minmal Alpha Reliability from Iota Reliability Concept Version 2.

  • avg_alpha: Average Alpha Reliability from Iota Reliability Concept Version 2.

  • max_alpha: Maximum Alpha Reliability from Iota Reliability Concept Version 2.

  • static_iota_index: Static Iota Index from Iota Reliability Concept Version 2.

  • dynamic_iota_index: Dynamic Iota Index Iota Reliability Concept Version 2.

  • kalpha_nominal: Krippendorff's Alpha for nominal variables.

  • kalpha_ordinal: Krippendorff's Alpha for ordinal variables.

  • kendall: Kendall's coefficient of concordance W.

  • kappa2_unweighted: Cohen's Kappa unweighted.

  • kappa2_equal_weighted: Weighted Cohen's Kappa with equal weights.

  • kappa2_squared_weighted: Weighted Cohen's Kappa with squared weights.

  • kappa_fleiss: Fleiss' Kappa for multiple raters without exact estimation.

  • percentage_agreement: Percentage Agreement.

  • balanced_accuracy: Average accuracy within each class.

  • gwet_ac: Gwet's AC1/AC2 agreement coefficient.

If return_names_only=TRUE returns only the names of the vector elements.

See also

Other Auxiliary Functions: array_to_matrix(), calc_standard_classification_measures(), check_embedding_models(), clean_pytorch_log_transformers(), create_iota2_mean_object(), create_synthetic_units(), generate_id(), get_folds(), get_n_chunks(), get_stratified_train_test_split(), get_synthetic_cases(), get_train_test_split(), is.null_or_na(), matrix_to_array_c(), split_labeled_unlabeled(), summarize_tracked_sustainability(), to_categorical_c()