Common Rater Errors for Assessment in Pediatric Rare and Orphan Disease Trials

Carmen E. Sanchez, PhD, Senior Director of R&O at VeraSci presented “Common Rater Errors for Assessment in Pediatric Rare and Orphan Disease Trials.” at ISCTM’s 2018 autumn conference. The presentation investigated the impact of data quality review, by quantifying the number of scoring and administration errors typically found in a paper-based review of cognitive and adaptive behavior assessments, particularly tools that are sensitive to neurocognitive change in MPS disorders. Results showed that expert data review of cognitive and adaptive behavior assessments demonstrated the high frequency of administration and scoring errors committed by raters in R&O disease clinical trials. The impact of errors is considerable, leading to the inaccurate calculation of endpoints across the subtest and domain levels. A central rater serves a critical role in R&O disease trials to identify errors of scoring and correct them for accuracy. Common errors point to areas rater training could address. Specifically, VABS-II–The use of computer-based scoring programs to assist in score summation, BSID-III–Explicit training on the double basal guideline and its application to scoring, KABC-II–Provision of clear guidelines for scoring. Data quality review and remediation can have large positive effects on signal detection in R&O trials.