From Practice to Performance: Progress Learning’s SAT® Test Accuracy
Progress Learning, a leader in the development of SAT® and PSAT®-aligned practice tests, consistently prioritizes data-driven improvements to enhance the reliability and accuracy of its assessment tools. In the 2023-2024 school year (SY 23/24), we partnered with Aurora Public Schools in Colorado to administer six PSAT®/SAT® practice tests. These were strategically delivered at the beginning and middle of the school year, followed by the College Board’s official PSAT® and SAT® assessments at the end of the year. Using descriptive statistics, split-half reliability, and negative discriminator analysis, Progress Learning evaluated its test forms in relation to College Board assessments. This comprehensive analysis led to actionable insights, enabling Progress Learning to refine its assessments for the upcoming 2024-2025 school year.
Data Overview and Methodology
We analyzed over 1,500 student records per assessment, comparing results between Progress Learning and College Board (CB) versions of the SAT®, PSAT® 10, and PSAT® 8/9. The count of records, shown below, highlights the volume of data used in the analysis:
- SAT®: PL1 (1654), PL2 (1581), CB (1842)
- PSAT® 10: PL1 (1814), PL2 (1791), CB (1927)
- PSAT ®8/9: PL1 (1582), PL2 (2022), CB (2092)
This volume of data provided a robust foundation for evaluating form reliability, correlation, score characteristics, and the impact of negative discriminators, all aimed at improving the reliability and accuracy of future test forms.
Key Analyses and Findings
Progress Learning conducted multiple statistical analyses to assess its test forms and their alignment with College Board assessments, ensuring that students receive high-quality practice tests. These analyses include:
Form Reliability and Correlation: Progress Learning compared the correlations between its assessments and the College Board versions, as well as correlations between test versions. The correlations were strong across the board, with the highest being 0.89 between PL2 SAT® and CB SAT®, indicating close alignment. Areas for improvement were identified in the PSAT® 10 math section and PSAT® 8/9 reading and writing.
| Assessment | PL1 to CB | PL2 to CB | PL1 to PL2 |
|---|---|---|---|
| SAT | 0.87 | 0.89 | 0.87 |
| PSAT 10 | 0.86 | 0.85 | 0.87 |
| PSAT 8/9 | 0.81 | 0.77 | 0.77 |
Descriptive Statistics (Mean, Median, Standard Deviation):
Progress Learning examined score distributions by calculating means, medians, and standard deviations to highlight the differences between Progress Learning and College Board assessments. The mean score difference for the PSAT® 10 total score was -136, indicating that Progress Learning assessments were generally lower. However, adjustments to scoring tables have been implemented to reflect more accurate and reliable conversions.
Split-Half Reliability: To ensure consistency and parallelism between test forms, Progress Learning utilized split-half reliability analysis. Results confirmed that all forms met industry standards, with reliability coefficients ranging from 0.77 to 0.89. These findings further validated the structural soundness of Progress Learning’s test forms.
| Assessment | Reading/Writing | Math |
|---|---|---|
| PSAT 8/9 #1 | 0.87 | 0.85 |
| PSAT 10 #2 | 0.84 | 0.88 |
| SAT #2 | 0.89 | 0.87 |
Negative Discriminant Analysis:
Progress Learning identified and addressed items that negatively discriminated student performance. Through a detailed analysis of item-level response frequencies, poorly performing items were removed or revised. This effort resulted in significant improvements, with approximately 11% of PSAT® 8/9 and PSAT® 10 items being edited, moved, or swapped to increase overall form reliability.
Next Steps for Progress Learning
Progress Learning’s proactive approach to assessment design is anchored in continuous improvement. Based on this year’s data analysis, several key updates are planned for the 2024-2025 school year:
- Item-Level Swaps and Edits: Progress Learning will revise and replace items in the PSAT® 10 math section and the PSAT® 8/9 reading and writing sections to enhance test form correlation with College Board assessments.
- Scoring Table Modifications: Adjustments will be made to the scoring conversion tables to reflect more precise score calculations.
- Focus on Targeted Forms: While the overall reliability of Progress Learning forms is high, targeted improvements will focus on the PSAT® 10 and PSAT® 8/9 assessments to bring correlations in these areas up to the same level as the SAT® forms.
Conclusion
Progress Learning remains committed to providing high-quality, reliable, and SAT®-aligned assessments for the 2024-2025 school year. Through rigorous data analysis, including form reliability, correlations, split-half reliability, and negative discriminator analysis, Progress Learning has not only validated the strength of its assessments but also identified opportunities for targeted improvements. As a result, Progress Learning is well-positioned to offer students a stable and trusted suite of practice tests that mirror the structure and rigor of College Board assessments.
By continually refining its assessments, Progress Learning exemplifies its dedication to helping students succeed, positioning itself as a proactive and reliable leader in educational testing. Need accurate benchmark testing and prep for the SAT® for your school? Get in touch with Progress Learning today.
