About the College Board
We are a mission-focused organization that believes that investing in education is investing in the future. From our earliest days, we have devoted ourselves to educational opportunity and achievement. We have promoted innovation, equity, and excellence for generations of students. We are advocates for children and parents; we empower teachers and educations, and we are a strong presence in thousands of schools and communities across the country through programs and services - the STA, Advanced Placement (AP®) and Pre-AP are just a few. Our work falls broadly into four categories: College Readiness, College Connection & Success, Student Opportunities, and Advocacy.
Named by Fast Company as one of the most innovative education companies, the College Board is a mission-focused organization. This job requires a strong focus on improving educational opportunities and outcomes, particularly of disadvantaged students, in the context of a competitive business environment.
The Associate Psychometrician will provide psychometric support to College Board programs. Psychometric support would include undertaking or leading operational support and vendor support for ACCUPLACER, CLEP and digital offerings of the SAT Suite of Assessments, including but not limited to production of item and pool analysis reports using both classical and item response theory, item exposure and other test security analyses, performing equating (pre- and post-equating), standard setting, and review of test documentation materials. The Associate Psychometrician / Psychometrician may also design, conduct, and manage research efforts to support various programs, most notably the ACCUPLACER, CLEP and digital offerings of the SAT Suite of Assessments. Desirable candidates will have experience working with CAT, caMST and LOFT delivered tests, as well as have experience with automated test assemble and automated essay scoring.
What you'll do
- Provide psychometric support to College Board programs, including production of item and test analysis reports using both classical and item response theory models, test security analyses, supporting state level contracts and other general psychometric support.
- Conduct or lead equating, linking, scaling, standard setting and other psychometric analyses, using standard and customized psychometric analysis tools and statistical software packages.
- Design, conduct, and/or manage research projects to support College Board programs and/or to digital testing initiatives.
- Develop and contribute to the development and improvement of operational processes.
- Develop or design analysis processes to ensure the quality and accuracy of results.
- Design and develop software programs, documents and implement statistical and reporting systems.
- Contribute to or write technical and research reports involving psychometric analyses, test validity, and reliability.
- Respond to requests for data and data interpretation.
- Work collaboratively with program leads to sustain current programs and develop new initiatives.
- Work collaboratively with committees (e.g. exam committees and technical advisory committees).
- Participate in discussions, and contribute to the research, development and evaluation of the potential effects of changes in programs, including new testing modes and item types.
- Contribute or author memos, presentations, and research reports for both internal and external audiences to explain research results.
- Contribute to the profession through articles, presentations, seminars and participation in professional organizations.
Maintain regular contact with Executive Director and team members to present and discuss work plans and priorities and to report work status. Maintain close contact with program staff. Collaborate extensively with other staff in Psychometrics, the LEvR Division, and Assessment Design and Development.
Maintain contacts with vendors, consultants, institutions, and other external parties involved with College Board testing operations and research. Make presentations on behalf of the College Board to outside organizations.
- Doctorate in psychometrics, educational research, educational measurement or a related field is required. A minimum of 3 years relevant experience is required for the Associate Psychometrician level, and a 5-years minimum, and 7-years of relevant experience is required for the Psychometrician level.
- Demonstrated ability to work collaboratively with colleagues of various professional backgrounds.
- Experience in working with large datasets and assessment data.
- Demonstrated ability to design and conduct research studies and complex statistical analyses.
- Expertise with classical test theory and item response theory is required, as is experience with statistical analysis software (e.g., SAS, R, FLEXMirt, WINSTEPS, and Python.)
- Experience or expertise working with automated essay scoring, automated test assembly (preferable with ILOG/CPLEX)
- Experience conducting comparability studies across modes and device types.
- Excellent verbal and written communications skills, including technical presentation, public speaking, interpersonal, and writing and editing skills
More about you
- Excellent organizational, time management and consultation skills.
- Ability to work on several projects simultaneously.
- Strict attention to detail and extreme accuracy.
- Willingness and ability to mentor junior staff.
- Ability to work with an Agile mindset and experience using JIRA.
- Ability to work as part of a team with strong teamwork skills.
We offer an outstanding benefits package that includes up to four weeks of paid time off each year, comprehensive health insurance, a generous retirement savings plan, tuition reimbursement, ongoing professional development and training, and more.
Our mission is to clear a path for all students to own their future.
The College Board is committed to diversity in the workplace and is an Equal Opportunity Employer. The College Board participates in E-Verify, a service of DHS and SSA, where required. Please understand that only qualified applicants will be contacted.