Principles of Learning

From Bunderson 2003, pp. 12-13 USER CENTERED DESIGN 1. Overall appeal. 2. Usability. The instrument will be easy to use, understandable, quick and efficient. 3. Perceived value to the target users, perceived positive consequences. CONTENT and CONSTRUCT evidence of validity 4. Content coverage and appropriateness. 5. Substantive processes -- The important but typically invisible mental processes used by those whom we would wish to score as more successful on an instrument, or affective attributes of persons such as their beliefs, attitudes, and values. It is only through theories of the cognitive, linguistic, affective or perhaps psychomotor processes that we can design appropriate questions or performance tasks to get at different degrees of these usually invisible processes. 6. Structure of the constructs. The starting number of questions or tasks is expected to collapse into a smaller number of separate unidimensional measurement scales. The scales we design should correspond with an...

" Quantitative psychologists have been quite happy with the idea of experimental design. Perhaps more of us can become comfortable with the principled design of both instruments and theory. Why not forthrightly seek to design and revise and experiment until we have evidence for all six aspects of construct validity, as well as evidence for desirable values, positive consequences, utility as ease of use, and all aspects of the unified validity model?" " Validity centered design as its current practitioners understand and use it is the beginning of a principled design process for designing and developing improved domain theories, the construct-linked measurement scales associated with them, and documenting the evidence for a validity argument. The validity argument is not accomplished all at one (and indeed, never ends), but is improved step by step as we complete work on each aspect of validity. It also includes planning for future activities to improve other aspects of t...

" Cronbach (1988) introduced the term “Validity Argument” after analogy to House’s (1977) notion of “the logic of evaluation argument”. Cronbach was in agreement with the complex but unified nature of validity, and its inseparable connection to values and consequences. Instruments cannot be validated themselves, and interpretations and uses change constantly. As a result, an instrument (and, we add, its associated theories), are never validated once and for all. There is an interplay between evidence and instrument features, and the instrument and theory evolve and are improved to reflect evidence and the correction of identified inadequacies. All we can do is continue to improve the argument for the theory, the construct-linked scales, the instrument, and its delivery system. We do this as Messick stated, through evidence and theoretical rationales.' (Bunderson 2003, p. 11 (pdf))