The Psychometrician will provide psychometric support to large and small-scale programs at the College Board, Puerto Rico and Latin America (PRLA) Office. Psychometric support includes production of item and test analysis reports using both classical and item response theory models, item calibration, item-bank maintenance strategies, test security analyses, DIF analysis, performing equating, linking, scaling, standard setting, job-analysis, review of test documentation materials, and the assessment of pre-test and operational test configurations. The Psychometrician will design, conduct, and manage research efforts to support various assessments programs. This position requires initiative, creativity, advanced research skills and technical expertise.
What you will do:
The Psychometrician will provide psychometric support to College Board-PRLA programs, including the production of item and test analysis reports using both classical and item response theory models, test security analyses, and other general psychometric support. Perform equating, linking, scaling, standard setting and other psychometric analyses, using standard and customized psychometric analysis tools and statistical software packages. S/he will manage item and test-level analysis processes to ensure the quality and accuracy of results. Evaluate the appropriateness of psychometric procedures and the efficacy of psychometric-based research. In a cross-functional fashion, S/he will provide in-house psychometric consultation for test development, test administration, scoring, and reporting processes and practices, including developing and implementing action plans to support a high quality and efficient operation with sound measurement practices; and support state level contracts or institutional agreements with LATAM stakeholders. S/he will also:
- Design, conduct, and/or manage psychometric projects to support College Board-PRLA programs and/or to support state initiatives or institutional agreements with LATAM stakeholders.
- Design, conduct, document and maintain equating plans and scaling/linking procedures for current assessment and new developments.
- Design, program, document and implement statistical and reporting systems.
- Lead the integration of standard analysis tools or statistical software packages, and the development of customized psychometric solutions, and perform analyses of large datasets and assessment data.
- Design, develop, and document technical and operating procedures.
- Conduct item and test-level analysis of pretests, operational forms, and item-calibration
- Lead discussions, and contribute to the research, development, and evaluation of the potential effects of changes in programs, including new testing modes, item types and scoring procedures.
- Write and/or review technical manuals for assessments and research reports involving psychometric analyses, validity evidence, and reliability information.
- Work collaboratively with program directors and content specialist to sustain current programs and provide psychometric support for new initiatives.
- Work collaboratively with committees (e.g. test committees and technical advisory committees).
- Prepare memos, presentations, and research reports for both internal and external audiences to explain research results.
- Pro-actively work toward the improvement of current processes (e.g. improve measurement methodologies and assessment practices, shorter cycle time, reduced costs, greater customer satisfaction and quality control).
- Represent the College Board on assessment, psychometrics, and research issues with vendors and other internal and external constituents.
- Respond to requests for data and data interpretation.
- Perform any other duties or tasks as assigned or required.
- An ideal candidate will have a doctorate in psychometrics, educational research, educational measurement or a related field with a strong training in latent variable modeling techniques is required.
- 4-6 years of experience performing similar tasks, with demonstrated evidence of added responsibilities.
- Expertise with classical test theory, item response theory, and equating/linking procedures is required, as is experience with statistical analysis software (e.g., SAS, SPSS, BILOG, FLEXMirt, WINSTEPS, PARSCALE, Mplus, and R packages).
- Experience in working with large datasets and assessment data.
- Demonstrated ability to design and conduct psychometric analyses, research studies, and complex statistical analyses.
- Facility in at least two of the following areas: partial credit models, factor analysis, item analysis, scaling, standard setting, or item calibration.
- Proficiency in scripting language/programming and database querying skills (e.g., Python, R, SQL, MySQL)
- Demonstrated ability to work collaboratively with colleagues of various professional backgrounds.
More about you:
- Knowledge and adherence to the principles and standards for educational and psychological testing and other applicable professional standards and practices.
- Proven ability to analyze and propose solutions to measurement problems, and communicate those solutions effectively.
- Be well read and interested in continuing to read the literature, both journals and books, to keep up to date with current developments in the field.
- Excellent verbal and written communications skills, including technical presentation, public speaking, interpersonal, and writing and editing skills.
- Ability to provide operational and strategic technical guidance on the use of current and emerging technologies.
- Ability to work on several projects simultaneously with minimal supervision.
- Ability to work in a collaborative environment that emphasizes quality, rigor, problem-solving, and critical thinking.
- Excellent time management, organization and consultation skills.
- Strict attention to detail and extreme accuracy.
- Actively serve as a coach and mentor ensuring appropriate training for office staff regarding psychometric and research considerations in testing and best practices.
- Bilingual proficiency - Spanish & English.
We offer our employees an outstanding benefits package, which includes 4 weeks of paid time off, a generous retirement savings plan, tuition reimbursement and ongoing professional development and training.
The College Board’s mission is to connect students to college success and opportunity. We are a not-for-profit membership organization committed to excellence and equity in education. Among our best-known products are the SAT®, PSAT/NMSQT®, and Advanced Placement Program®. The College Board is committed to diversity in the workplace and is an Equal Opportunity Employer. The College Board participates in E-Verify, a service of DHS and SSA, where required. Please understand that only qualified applicants will be contacted.