Program Implementation
Organizations may have limited resources or expertise to implement an assessment program in whole or in part. Specific components of the program can be managed by an external contractor. For instance, the administration of an assessment may require hundreds of test administrators in different locations for very specific periods; or scoring of an assessment may require many scorers needing to be expertly trained and monitored. In these cases, an external contractor may provide a more cost-effective, efficient and reliable solution for an organization than hiring a large number of people for short periods of time.
Report Development
Varying in length from a few pages to over 200 pages, these reports can summarize and interpret findings in many different formats (graphical, tabular or textual). The analysis can be based on data collected for the client or on existing datasets. They can cover an entire population (e.g., Canadian level) or specific sub-populations (e.g., a province, a school district or a school).
Quality Assurance
How can I be confident the results of large-scale student assessment are accurate, reliable, and valid?
If educators use the data/information to help improve student learning, and if other stakeholders rely on the results to understand how the education system is performing and make programming decisions, why should they trust the data? The answer is that confidence derives from a well-conceived and implemented quality assurance (QA) program.
As a basic quality assurance measure, all testing organizations prepare manuals meant to ensure that assessments are administered consistently according to the organization’s or jurisdiction’s guidelines. But as an organization, to what extent do you attend to other QA processes associated with test development, test administration, assessment monitoring, scoring/marking, database monitoring, data processing and data integrity, and reporting?
If you believe your assessment program may have QA gaps, there is a solution. The Quality Assurance Lab, a collaboration of RMJ Assessment and Consultation MEP Consulting Inc., delivers a wide range of QA services. Knowledgeable and experienced in best practices, this team offers organizations/jurisdictions auditing services to assess the adequacy of existing processes and is skilled in the design, development, and implementation of any QA needs. For more information, please visit our Web sites at: https://www.rmjassessment.com and http://consultationmepconsulting.com.
Training on Data Analysis
In some cases, organizations such as departments of Education or school districts may have data analysts but these may lack expertise in the analysis of large-scale assessment datasets. For instance, datasets such as CMEC’s Pan-Canadian Assessment Program (PCAP), OECD’s Programme for International Student Assessment (PISA) or IEA’s Progress in International Reading Literacy Study (PIRLS) and Trends in International Mathematics and Science Study (TIMSS) are complex and require an understanding of concepts such as weights, plausible values and standard errors for which specific training can be provided on request.