Get started

library(unsum)

Here is a brief walkthrough of using CLOSURE in unsum.

Call closure_generate() to run the CLOSURE algorithm. Enter mean, SD, and sample size that you read in a paper. For scale_min and scale_max, use the empirical minimum and maximum if available. Otherwise, use the more fundamental scale bounds, e.g., 1 and 7 for a 1-7 scale.

The mean and sd arguments must be strings to preserve trailing zeros. Note that CLOSURE can only be used if the values must be integers: e.g., a value can be 2 or 3, but not 2.5.

data <- closure_generate(
  mean = "3.5",
  sd = "2",
  n = 80,
  scale_min = 1,
  scale_max = 5
)

First create a plot of the mean sample found by CLOSURE. This gives us a sense of the overall results, which are quite polarized:

closure_plot_bar(data)

You can customize the plot, e.g., to show the sum of all samples found instead of the average sample, or only percentages, or different colors. See documentation at closure_plot_bar(). However, the default should be informative enough for a start.

Now let’s look at the results themselves:

data
#> $inputs
#> # A tibble: 1 × 7
#>   mean  sd        n scale_min scale_max rounding   threshold
#>   <chr> <chr> <dbl>     <dbl>     <dbl> <chr>          <dbl>
#> 1 3.5   2        80         1         5 up_or_down         5
#> 
#> $metrics
#> # A tibble: 1 × 5
#>   samples_initial samples_all values_all horns horns_uniform
#>             <int>       <int>      <int> <dbl>         <dbl>
#> 1              15       21861    1748880 0.653           0.5
#> 
#> $frequency
#> # A tibble: 5 × 4
#>   value f_average f_absolute f_relative
#>   <int>     <dbl>      <int>      <dbl>
#> 1     1     16.1      352051     0.201 
#> 2     2     10.2      222512     0.127 
#> 3     3      7.51     164251     0.0939
#> 4     4     10.2      222512     0.127 
#> 5     5     36.0      787554     0.450 
#> 
#> $results
#> # A tibble: 21,861 × 2
#>       id sample    
#>    <int> <list>    
#>  1     1 <int [80]>
#>  2     2 <int [80]>
#>  3     3 <int [80]>
#>  4     4 <int [80]>
#>  5     5 <int [80]>
#>  6     6 <int [80]>
#>  7     7 <int [80]>
#>  8     8 <int [80]>
#>  9     9 <int [80]>
#> 10    10 <int [80]>
#> # ℹ 21,851 more rows

• inputs records the arguments in closure_generate().

• metrics shows the number of possible samples that could have led to the reported summary statistics (samples_all) and the total number of all values found in them (values_all).

  Importantly, it also features an index of variation in bounded scales (horns). It ranges from 0 to 1, where 0 means no variability and 1 would be a sample evenly split between the extremes — here, 1, and 5 — with no values in between. The reference value horns_uniform shows which value horns would have if the mean sample was uniformly distributed. This is 0.5 because of the 1-5 scale. See horns() for more details.

  The actual horns value is 0.65, which is a moderate to high degree of variability in the abstract. However, in practice, 0.65 might be extremely high compared to theoretical expectations: if the sample should have a roughly normal shape, even the hypothetical 0.5 uniform value would be surprisingly high, let alone the 0.65 actual value.

• frequency shows the absolute and relative frequencies of values found by CLOSURE at each scale point. It also gives us the (absolute) frequency of values in the average sample that we saw in the plot above.

• results stores all the samples that CLOSURE found (sample). Each has a unique number (id).

See closure_generate() for more details.

In addition to the bar plot, unsum offers an ECDF plot for CLOSURE results:

closure_plot_ecdf(data)

What if you have a huge object with CLOSURE results that you want to save? Write it to disk with closure_write():

# Temporary folder just for this example --
# use a real folder on your computer instead!
fake_folder <- tempdir()

# Saving the path in a variable
path_new_folder <- closure_write(data, path = fake_folder)
#> ✔ All files written to:
#> /tmp/RtmpKHg2yD/CLOSURE-3_5-2-80-1-5-up_or_down-5/

This stores the results using the highly efficient Parquet format. It will only take a tiny fraction of a CSV file’s disk space.

In your later session, read the data in from the folder to get the same CLOSURE list back:

data_new <- closure_read(path_new_folder)

# Same as the original data:
waldo::compare(data, data_new)
#> ✔ No differences

A caveat: don’t modify the output of closure_generate() before passing it into other closure_*() functions. The latter need input with a very specific format, and if you manipulate the data between two closure_*() calls, these assumptions may no longer hold. Some checks are in place to detect alterations, but they may not catch all of them.