Principles of Learning

" Cronbach (1988) introduced the term “Validity Argument” after analogy to House’s (1977) notion of “the logic of evaluation argument”. Cronbach was in agreement with the complex but unified nature of validity, and its inseparable connection to values and consequences. Instruments cannot be validated themselves, and interpretations and uses change constantly. As a result, an instrument (and, we add, its associated theories), are never validated once and for all. There is an interplay between evidence and instrument features, and the instrument and theory evolve and are improved to reflect evidence and the correction of identified inadequacies. All we can do is continue to improve the argument for the theory, the construct-linked scales, the instrument, and its delivery system. We do this as Messick stated, through evidence and theoretical rationales.' (Bunderson 2003, p. 11 (pdf))

" Extensive research is often needed to understand the nature of the substantive process constructs that explain the ordering from lesser to increasingly able learners. This research may start with qualitative probes to get an initial idea of the domain constructs, followed by the discovery of qualitative order relationships, eventually leading to a qualitative model of the domain. The construction of many tasks – items, testlets, performance rubrics – is necessary, as it provides confirming evidence of theoretical 6 propositions about how many essentially unidimensional scales are needed to span the domain. An increasingly mature domain theory identifies a set of unidimensional scales that approximate equal interval scales, each giving a precise parameterization of the expertise space of a specific knowledge domain. It has a pool of real and possible tasks calibrated on the scales that can be used to measure accurately learner progress within the domain. When a domain theory is a...

" the general boundaries of the domain are initially established, but the process is not complete until a set of essentially unidimensional measurement scales is developed, which together define the scope and boundaries of the domain of learning and growth. Because perfect unidimensionality is impossible to achieve, we must settle for essential, or near, unidimensionality. Delineating a domain to a point where essential unidimensionality in each of a small set of domain-spanning scales is achieved is not always accomplished in practice, as it is often a time-consuming and lengthy process. Constructing a set of domain-spanning measurement scales elevates a good qualitative model of a domain to a theory." (Bunderson 2003, p. 5)

" Instruments are developed to assess learning progress and performance in domains of human knowledge, proficiency, and accomplishment. Domains are usually signified by familiar content or topical names, like calculus, American history, accounting, network engineering, and nursing. Topics signify categories of what people may know, but they do not describe what people can do, so other uses of language, such as objectives or work-models are needed." (Bunderson, 2003, p. 5 (pdf)

"A domain theory is a descriptive theory of the contents, substantive processes, and boundaries of a domain of human learning and growth that gives an account of construct-relevant sources of task difficulty; and conjointly, an account of the substantive processes operative at different levels of growth along the scale(s) that span the domain. Based on the constructs that account conjointly for difficulty and level of processing, and using measurement instruments linked to the constructs in the domain theory, testable predictions can be made about the relationships between tasks, processes, and locations along the scale(s)." (Bunderson 2003, p. 5 (PDF))

"...we need a domain theory to explain what is easy, what is somewhere between easy and hard, and what is hard. When we understand these things, we can measure, give meaningful feedback, and can teach people who desire to do so how to rapidly progress from lower to higher levels in either type of domain. These lead to the construction of learning maps that reflect progress up the domain scales, or reveal standing on a map of individual difference variables." (Bunderson, 2003, p. unknown --- pdf available online at http://www.edumetrics.org/papers/How_to_build_a_Domain_Theor.pdf )