Skip to content
/ IRRsim Public

Simulate Inter-Rater Reliability Statistics

License

Notifications You must be signed in to change notification settings

jbryer/IRRsim

Repository files navigation

IRRsim: An R Package for Simulating Inter-Rater Reliability

R-CMD-check

This package provides functions and a shiny application to simulate inter-rater reliability statistics based on various scoring and response models. The initial motivation for this package is to understand the relationship between percent rater agreement and intraclass correlation.

Documentation is available at http://irrsim.bryer.org
Slide deck available at https://irrsim.bryer.org/IRRsim-Presentation.html

Install

The development version of IRRsim can be downloaded from Github using the remotes package:

remotes::install_github('jbryer/IRRsim')

Shiny App

Many of features of this package can be explored using the included shiny applications. The IRRsim_shiny() function will run the application.

IRRsim::IRRsim_demo()

IRRsim Shiny App

IRRsim Shiny App

Data

The analysis done in the vignette requires a large number of simulated datasets. Not only does it take a long time for those datasets to be generated, the file sizes exceed what is allowed in R packages. The data are stored as Github releases. See the data-releases/IRRsimulations.R R script for how these data files are generated. Alternatively, you can download the data using the following commands in R:

download_dir <- getwd()
download.file('https://github.com/jbryer/IRRsim/releases/download/IRRsimData/IRRsimData.rda', 
              paste0(download_dir, '/IRRsimData.rda'))
load(paste0(download_dir, '/IRRsimData.rda))