The obsessive use of traditional standardized assessment approaches as valid and reliable solutions for educating and training the 21st Century Knowledge Citizen have significant problems and limitations for non-traditional learners with disabilities.
For example, existing instruments do not provide for demonstrations of a person’s behavioral understanding and emotional capacity for dealing with the changing social/functional demands of the workplace environment. They also lack provisions which can evaluate the problem solving and creativity needed to develop “what if’s” compensations, accommodations, modifications and learning strategies that address real-time workplace requirements.
This inability of the present system to offer more meaningful results unnecessarily limits skill focus, emphasizes “deficits and deficiencies”, and lack an applied real-world contextual basis. Not facing up to these shortcomings has consequences impacting the entire education reform movement’s hopes.
Typically, instructional & training needs are identified through the use of criterion-referenced ability assessments.
Of the many shortcomings associated with these instruments for the purpose of counseling, employment placement, and postsecondary education of adults with disabilities, there are three in particular that are problematic from my point of view and that drive alternative assessment choice and development.
First, standardized testing generates situations where the abilities measured are tested in formats not typical of contextual or “real-world” situations. Wiggins characterized the inadequacy of this testing approach as “removing what is central in intellectual competence: the use of judgment to recognize complex problems and use one’s discreet knowledge to solve them” (Wiggins,1989).
Second, standardized testing provides, at best, superficial information limited in relationship to the full potential of the person. Accurate measurement using standardized instruments falls short of examining the “whole” person because it provides limited understanding of the social, emotional, physical, aesthetic, and intellectual development that is unique to that individual (Ulrey & Ulrey, 1992).
Finally, standardized testing focuses on the content of test responses instead of the learning process used to formulate a response (Lazear, 1994). Living in an information age society, where the sheer volume of available knowledge is growing beyond our capability to acquire, make meaningful, store, and situationally select it, should promote efforts to look beyond mastering academic content as the primary benchmark of testing.
Increased attention should be given to designing instruments that show the processes associated with developing and utilizing the capacity to be a creative thinker and problem solver-how to adapt, transfer, and integrate classroom or workplace content into daily living/employment tasks.
In point of fact, just being able to recall information from multiple-choice, essay, or fill-in-the blank tests does not provide an accurate reading of a person with disabilities grasp of the material, nor does this alone guarantee success on the job, in which performance is evaluated through work quality and quantity, the benchmarks of practical application and mastery of content.
Global assessment efforts making progress, but coming up short too
The Programme for International Student Achievement (PISA)widely accepted as the “Gold standard” of international assessment for real world applications of critical thinking and problem solving, does not have measurements that can show student effectiveness in how to resolve conflicts with civility, bargaining & making compromises, or reliably & accurately assessing the many life situations which do not have clear, certain outcomes.
According to Ian Whitman, Head of Program in the Education Directorate of OECD, there are no efforts currently being undertaken by this body to develop tools which can assess how effectively a student deals with decision uncertainties such as gauging their risk-taking adversity/tolerance, or how to deal with the trade off’s & consequences of choices that are not clearly binary (black & white) situations.
These are the very conditions that confront all people each day- whether they are gamers trying to make the next level in Tour of Duty, or customer service representatives dealing with an out-of-control irate customer.
We do not develop curriculum, teach, or assess these types of situational experiences in either the secondary or higher educational systems.
We do spend many billions of dollars each year in staff training and development throughout the business and corporate communities across the country and world in an attempt to teach these types of “soft skills.”
With this thought in mind, what kind of assessments should we be developing that gets us close to measuring these capacities?
The need for authentic assessment being discussed in this post promotes the position for an abilities-based evaluation of such processes. Authentic assessments are realistically structured, taking into account real-world constraints typically encountered outside the learning environment (e.g., time, production-level requirements).
Academic design factors for authentic assessment require that testing occur in the context of actual learning situations that are relevant to students, business, and the community. By focusing on the mastery and measurement of true essential skills and competencies, which are supposed to be the basis of “real world” assessment, the ability to uncover students’ working knowledge, skillful use of academic/workplace competencies and transferability of applied learning into real-life situations is facilitated.
This method of assessment replaces the rewarding for retrieval of disconnected bits and pieces of information that are likely to be unrelated to the program of study, career development or workplace training as a whole.
The ability to creatively work through problems encountered on the job, to which there are many possible solutions, is valued at least as much as getting the “right” answer on a battery of timed paper-and-pencil examinations.
The structure of authentic assessment “examinations” emphasize individual progress in terms of discipline content, processes, and applications of materials-regarding both program requirements and the outside world. Evaluations are constructed using a developmental spiral to represent differing stages of achievement (e.g., content, process, product, skill level), resulting in tests worth taking and repeating until material is mastered.
Ultimately, “real world” academic and workplace assessments should accurately predict how an individual who is a nontraditional thinker knows, understands, perceives, learns, and processes information. Such an assessment measures abilities-based reflections of individual interests, capabilities, and aptitudes.
The combined results would be responsive to the demands of living in the adult community, acquiring and using workplace literacy, and identifying targeted employment or career objectives. This would be a good outcome for all people, not just those with special needs.