• Senior or Principal Research Scientist

    Job Locations US-IA-Iowa City | US-Remote
    Posted Date 2 weeks ago(5/7/2018 5:49 PM)
    Job ID
    2018-1237
    # of Openings
    1
    Category
    Research
    Travel
    Up to 25% Travel
  • Overview

    ACT is a nonprofit organization helping people achieve educational and workplace success.  Our programs are designed to boost lifelong learning in schools and workplaces around the world. Whether it's guiding students along their learning paths, enabling companies to develop their workforce, fostering parent, teacher, and counselor understanding of student progress, guiding job seekers toward career success, or informing policymakers about education and workforce issues. ACT is passionate about making a difference in all we do.

     

    Learn more about working at ACT at act.org!

    Responsibilities

    Position objective: ACT is looking for a seasoned measurement professional to join our Assessment Transformation team. The Assessment Transformation team has as its mission to lead the research, design, and development of next generation assessments that facilitate adaptability and personalization by leveraging digital technologies, new measurement models, and digital data sources. This position will independently: provide psychometric support including test construction, field test design and item analysis, equating design and analysis, and score comparability research; incorporate the use of process data (e.g. response time, eye-tracking, log file data) to enhance assessment modeling and support for adaptive/personalized assessment feedback; conduct cognitive labs to provide validity evidence for response processes; conduct research on next generation psychometric models at scale.

     

    ESSENTIAL JOB RESPONSIBILITIES

     

    Typical work-related activities include:

    • Independently design and conduct psychometric and research analyses for ACT products and services
    • Interpret and synthesize results from psychometric analyses and make recommendations to ACT assessment product owners based on those results
    • Lead project teams, maintain quality control of all projects, and guide junior staff in completing assignments and acquiring skills
    • Create and maintain technical specifications and documentation for the psychometric processes and procedures related to the ACT and related assessment programs
    • Conduct cognitive labs, usability studies, and eye-tracking research to provide validity evidence based on response processes
    • Incorporate process data (e.g., response time, eye-tracking, log file data) into psychometric procedures to support adaptive and personalized assessment solutions
    • Provide input to technology teams and data engineering teams on capabilities needed for test delivery and scoring as well as data outputs and formats for analysis for both paper based and digital assessments
    • Present research study plans and results to the ACT Technical Advisory Committee for feedback
    • Provide support for proposal development in response to domestic RFP’s, international tenders, and similar requests for services
    • Collaborate closely with Test Development on development of overall assessment design and content specifications for current and new assessment products
    • Support the development of a research agenda to transition existing large scale educational assessments to new assessment modes and measurement models as well as the development of new assessments (both formative and summative) which utilize next generation psychometric methods
    • Participate within ACT Research and across ACT in creating an open, innovative, and collaborative environment
    • Increase visibility of ACT Research through staff publications, presentations, and presence across relevant national and international research communities

     

     

    Qualifications

    Minimum Qualifications

     

    Education:

    • Doctoral degree in educational measurement, educational psychology, quantitative psychology or related field

    Experience

    For Senior Research Scientist:

    • Six years of experience in the development and publishing of high-quality, large scale educational assessments

     

    For Principal Research Scientist:

    • Nine years of experience in the development and publishing of high-quality, large scale educational assessments

     

    For Both:

    • Demonstrated experience implementing program and project management methods for complex assessment programs (e.g., creating/managing schedules, creating/managing program documentation and analysis specifications, etc.)
    • Experience in working with technology delivered assessments including knowledge of processes for authoring, publishing, delivery, and data acquisition
    • Experience with transitioning large scale assessment programs to new modes of delivery and new measurement models
    • Experience in developing scoring methodologies and applying psychometric models for technology enhanced item types
    • Experience with developing proposal responses for state assessment programs
    • Experience applying Item Response Theory in support of field testing, test construction, and operational scaling and equating practices
    • Demonstrated experience working with IRT software packages including Bilog, FlexMIRT, Winsteps, etc.

    • Experience presenting to Technical Advisory Committees

    Options

    Sorry the Share function is not working properly at this moment. Please refresh the page and try again later.
    Share on your newsfeed

    Connect With Us!

    Not ready to apply? Connect with us for general consideration.