The National Standards Project
The National Standards Project: Promoting Evidence-based Practice
Susan M. Wilczynski, PhD, BCBA
National Autism Center
Randolph, Massachusetts, USA
Date First Published: April 2, 2007
Date Last Updated: April 2, 2007
The number of unproven treatments offered on the Internet, in magazines, or at conferences has grown dramatically in recent years in parallel with the striking increase in diagnosed cases of Autism Spectrum Disorders (ASD). Parents, educators, and service providers find themselves flooded with recommendations, but are often left feeling they have to make tough decisions without access to clear, concise information about the research-supported treatment options. The National Standards Project was developed in response to this urgent need for information. The National Standards Project is a collaborative effort to systematically review educational and behavioral treatment research involving individuals under the age of 22 in order to determine the strength of evidence supporting these approaches. In this way, parents and educators can weigh the quality and quantity of research supporting an intervention into their decision-making. Service providers from a diverse array of fields are compelled to use evidence-based practice.
The National Standards Project builds on previous efforts to develop evidence-based guidelines for the treatment of ASD. For example the National Research Council (NRC) brought together a multi-disciplinary subcommittee to examine scientific, theoretical, and policy literature with respect to the education and treatment of young children with autism. A useful set of recommendations resulted from this important work. Unfortunately, the scope of the NRC report was restricted to young children so conclusions about evidence-based practice with school-aged children and adolescents could not be drawn from this effort. Also, because the research reviewed was conducted prior to the year 2000 it necessarily can not reflect the latest research findings.
In 2005, the National Autism Center assembled a panel of nationally recognized scientists and practitioners with expertise in educational and behavioral interventions. These experts were convened to identify the best way to proceed with a systematic review of research that could be conducted in school and behavioral treatment settings. Following two plenary sessions and a series of follow-up meetings, the conceptual model for the National Standards Project was developed and several central concerns emerged. First, the methodology for reviewing the literature must be transparent. That is, interested consumers should be able to critically evaluate each step of the National Standards Project and be able to identify exactly what procedures were used and why. Second, the literature should be organized in several ways in order to best serve the people who need the information. Specifically, strength of evidence ratings should be available not only for comprehensive programs designed to address all core deficits associated with ASD, but also should address focused interventions designed to either decrease problem behaviors or increase developmentally appropriate skills. Third, independent scholars representing diverse fields of study and theoretical orientation should review the conceptual model for reviewing the literature so that the process for reviewing the literature reflects the comprehensive literature and fully respects the contributions made by many different disciplines to the treatment of ASD. Fourth, parents, educators, and service providers require information about many different interventions so the review could not be restricted only to those with the greatest quality and quantity of research. Thus, the scale reflecting the strength of evidence should be sufficiently broad so that the quality and quantity of research on emerging interventions is considered as well as interventions that enjoy a long history of empirical support. In addition, treatments that lack research support and interventions that have been discredited must also be evaluated using the same objective criteria used for all of the literature. Finally, the results of the National Standards Project must be disseminated in a user-friendly format so that all interested parties can consider the outcomes in their decision-making.
In the fall of 2006, the ASD literature reflecting interventions that could be implemented in school and behavioral treatment settings will be reviewed. Specifically, empirical intervention articles that have been published in peer-reviewed journals that involve individuals under 22 years of age will be evaluated individually using the Scientific Merit Rating Scale. The Scientific Merit Rating Scale evaluates the scientific rigor of individual studies and the extent to which conclusions about the effectiveness of a study can be established. Dimensions that are assessed include: (a) Design and Treatment Effects, (b) Measurement of the Dependent Variable, (c) Measurement of the Independent Variable, (d) Participant Ascertainment, and (e) Generality of Treatment Effects. Each dimension is scored on a six-point rating scale ranging from rigorous (5) to unsatisfactory (0). Expert panelists, many conceptual reviewers, and invited reviewers will undergo training to establish their reliability using this scale prior to beginning the systematic review.
The results of individual article ratings will be aggregated for both comprehensive programs and focused interventions and then compared against a Strength of Evidence Classification System. The Strength of Evidence Classification System includes a four-point continuum of research support (e.g., strongest, strong, and modest evidence as well as emerging findings) and two independent scores reflecting interventions that are (a) unestablished or (b) discredited. Interventions are rated ‘unestablished’ when they are based solely on poorly controlled studies, opinion, or conflicting results have been reported in the literature. Treatments are considered ‘discredited’ when well-controlled studies have demonstrated they are ineffective in changing the target behavior.
Once the Strength of Evidence Classification System has been used to rate comprehensive and focused interventions, the results will be documented and reviewed by a group of consumers (i.e., parents, educators, service providers) to ensure the outcomes are clearly described. This document will then be available at the National Autism Center website and will be disseminated to departments of education. In addition to strength of evidence ratings for interventions, consumers and scientists will be reminded that evidence-based practice reflects the integration of research findings with clinical judgment and patient values.
About the National Autism Center
The National Autism Center is a new nonprofit organization dedicated to supporting evidence-based practice for individuals with Autism Spectrum Disorders. The Center promotes best practices, serving as a vital source of information, training and services for families, practitioners and policy-makers. As its first initiative, the Center has brought renowned experts together to establish the National Standards Project. The National Standards will serve as a single, authoritative source of guide to help families, educators, practitioners, and policy-makers make informed decisions and choose effective treatments.
The founding and initial development of the National Autism Center has been sponsored by May Institute, a not-for-profit organization dedicated to providing comprehensive, research-validated services to children and adults with autism spectrum disorders, brain injury, mental retardation and behavioral healthcare needs for individuals throughout the country for over 50 years. The Center is also supported by donations and philanthropic support.