Assessment Blueprints
An assessment blueprint document defines the plan for one or more assessments. The blueprint sets expectations and testing objectives. It defines the standards being measured, the relative importance of those standards and may specify the depth of knowledge expected for each item. Blueprints are useful tools for ensuring that assessments are meet an expected standard of quality.
A robust assessment blueprint should:
- Determine the primary purpose of the assessment
- Identify the standards you will be assessing
- Identify the skills/competencies addressed within each standard
- Identify the levels of rigor of each skill
- Identify challenge styles to use for each skill.
Assessment Purpose
The purpose of the assessment is important to establish to guide the rest of the process. The purpose should be as tightly-scoped as possible; packing many goals into an assessment can dilute the clarity of the results or confuse candidates and harm completion rates.
In a hiring scenario, an assessment purpose might be drawn from a job role: to establish whether a candidate would be likely to have the skills necessary to fill a back-end engineer position, for example.
In an education scenario, an assessment purpose might be to ensure students retained information from a module. For example, an assessment might seek to establish student comfort with manipulating Pandas dataframes as part of a data science curriculum.
Assessment Standards
The standards you plan to assess help guide the difficulty, quantity and time limits of the challenges and quizzes that help you reach your goals for the assessment. This stage can help increase the granularity of your top-level goals.
Possible standards might include results-oriented language like "candidate writes SQL subqueries fluently" or "candidate can efficiently locate and fix a bug in a React component".
Standard Skills
Skills can help you identify the particular skills that a candidate might possess in order to accomplish a standard. For example, a debugging-oriented standard can be supported by a skill such as process efficiency. As above, you might use language like "candidate process efficiency" as a metric for measuring this particular skill. When completing a rubric, reviewers can rate the observed skill level based on the candidate's solution or responses.
Skill Rigor
This blueprint item offers the ability to compare candidates by specifying the expected rigor in a particular skill. This can help in choosing the difficulty of challenges designed to test each skill.
Challenge Style
Challenge styles are important to consider for assessment effectiveness. Variety and flow/sequence are likely to be important considerations. For example, an assessment with nothing but single-function, greenfield coding challenges may test redundant skills, tire the candidate unnecessarily and fail to generate signal on other facets of the candidate's skill repertoire such as debugging, writing and communication skills, test case development, refactoring, answering knowledge-based quiz questions and so forth. Planning for a healthy mixture of these challenge styles can increase the relevance of your results and increase candidate satisfaction and student engagement.
Check out our article on various challenge styles that can be utilized.