General
What is a test?
Are tests accurate?
Are tests always right?
Are tests fair?
How do you ensure tests are fair?
Why are some questions so personal?
Why do some questions seem pointless?
What are a test takers rights?
Is it alright to get the test beforehand?
How do I know if the test is being properly used?
Are tests updated?
What kind of training should a test user have?
Who interprets the results?
Is it possible to cheat on a test?
Who sets test standards?
How do tests help business?
What do tests measure in business?
Is it legal to use tests in hiring?
Why do companies rely on tests?
Do tests invade personal privacy?
Do tests prevent qualified people from hire or promotion?
Should test results be the only basis for hire or promotion?
What about testing for the disabled?
What types of tests are most common in schools?
How are these tests developed?
Are all standardized tests norm-referenced?
What about newly proposed performance assessments?
How do schools use individualized tests?
How can tests best help my child?
How can intelligence tests help students?
Do IQ tests discriminate against minorities?
Why is there so much controversy about IQ tests?
Testing in Clinical and Medical Settings
How do tests help mental help professionals and patients?
Can psychological tests attach labels on people?
Do clinical psychologists use different tests than shools or businesses?
Can a test prove someone is mentally disturbed?
What good are tests when someone has already been diagnosed?
Since the advent of diagnostics such as MRI, why do we need testing?
If you can't find an answer to your question, please send us an e-mail.
INTRODUCTION
Since the early 1920s, psychological and educational tests have come into widespread use as tools for professionals such as psychologists, counselors, teachers, and human resource managers in government, education, and industry. In North America today, there is scarcely a person over the age of ten who has not taken at least one test, whether it has been an achievement test, an IQ test, a personality test, or a measure of interest or aptitude in a particular field.
A principal reason for this growth in test use is that the results of professionally developed tests are more reliable and accurate assessments of human capabilities and behavior than those based on the observations of individuals alone, however well-informed or insightful those observations might be. Individual judgments about people tend to be subjective and vulnerable to error, no matter how hard one tries to be fair. Even skilled observers who pride themselves on their objectivity are apt to perceive people through a filter of unconsciously held biases, personal beliefs, incomplete knowledge, and temporary distractions.
By relying on personal observations alone, a teacher may view a slow learner in the classroom as being lazy; a psychologist may interpret eccentric behavior as indicative of a serious emotional disorder; or a human resources director may view a carelessly dressed person as incapable of handling a high-responsibility job and all three observers may be wrong with potentially serious consequences. Tests can help to reduce these types of misjudgments about individuals.
This does not mean that tests can serve as a replacement for the personal judgment of professionals. In fact, a test is usually only one source of information among several used to assist in making the most accurate and fair decisions possible. Professionally developed tests that are designed by experts, scored and interpreted by properly trained individuals, can help even the most experienced and knowledgeable decision-maker to construct a fairer and more accurate picture of an individual. With more accurate information, teachers can better encourage students who have different learning styles to learn more effectively, psychologists can better serve their clients by providing proper treatment based on more accurate diagnoses, and employers can increase the likelihood that they will hire the right person for any given position.
Today, there are literally hundreds of tests available to professionals. These may be used to measure a wide range of human characteristics and behavior. Some, such as mental abilities tests or personality inventories, may be quite comprehensive in scope. Others may be more narrowly focused aids used to diagnose problems such as alcohol abuse or to assess a job applicants mechanical ability.
Tests may have more than one use. For example, a neuropsychologist might use a cognitive test to track a patients rate of recovery from the effects of a stroke. A school psychologist could use the same test to assess a student for the possible presence of a learning disability. And a consulting psychologist could use the same test to assess a job applicants abstract reasoning ability when that skill is vital to effective job performance.
Despite the important and useful role that tests play in businesses, schools, clinics and hospitals, many people have questions about their origins and use. This is not surprising, since tests are not often widely understood by the public. Test questions are typically not supplied ahead of time, and their results are generally kept private. Tests can be highly sophisticated and complex instruments and are often based on many years of rigorous research. The following answers to some commonly asked questions about tests may help you better understand how tests are developed and how they are best used.
When we use the word test in this brochure, we are referring to professionally developed instruments such as those produced by ATP members. Professionally developed tests are constructed responsibly and are responsive to existing professional standards. It is important to emphasize that not every assessment that is promoted or represented as a test falls into this category.
A test begins as a set of questions. Experts in the appropriate field construct the questions to assess or measure a specific ability or characteristic. The questions are based on the developers knowledge of the ability or characteristic to be assessed by the test.
Next, the questions comprising the prospective test are administered to a large number (often thousands) of individuals in field studies, and the results are analyzed by measurement experts. The prospective test must prove to give the same accurate results time after time to be reliable, to be equally applicable to all groups who take it to be fair, and to measure usefully what it is stated to measure to be valid.
A prospective test that clears all these hurdles is still just a work-in-progress. Many tests are submitted to full scale standardization, where they are administered to a large sample of people under standardized (uniform) conditions. Samples of these kinds are selected to accurately reflect the national population or other target group and are referred to as the norm group or normative group. Additional studies using other large groups are conducted to further document the tests reliability, fairness, and validity. Results of these technical studies are collected in a test manual which guides test users in the proper use of the test. Finally, clear and easy-to-follow administration, scoring, and interpretation materials are created for test users.
A modern test represents a tremendous investment of time, talent, and money by the developers, technical experts, and the publisher. Creating or revising a comprehensive achievement, aptitude, personality or intelligence test can easily cost several million dollars.
While no assessment procedures are perfect, tests developed properly and used within the limits for which they were designed are highly accurate. Professionally developed tests are constructed on a foundation of research results and previous test outcomes that have often taken many years or decades to accumulate.
Tests can be easy to criticize, sometimes for the very reasons that they are so useful. For example, professional guidelines on test development require that the tests underlying research be described in the test users manual. This manual must include scientific evidence that makes clear both the strengths and weaknesses of the test. These days, there are few products that document their true characteristics, the strengths and weaknesses, so clearly and openly. The professionals who administer and use the test are responsible for understanding and evaluating the underlying characteristics of the test. You can ask them about these issues if you have questions.
Well-made tests, when used for their intended purposes, make fewer errors than other single evaluation procedures. However, test results are not perfect. No procedure that assesses people can be. The best decisions come when test results are combined with information from interviews, expert observations, ratings of past work, and so on. Even after considering all the information that is available, however, the people who make the most effective decisions will tend to be those who best understand how to incorporate the information provided by the test results.
WHY DO I SOMETIMES READ THAT TESTS ARE NOT FAIR, OR ARE NOT DOING WHAT THEY SHOULD?
When you hear someone say that tests are not performing well, the best question to ask in return is: Compared with what? Tests are often used to make difficult decisions about people. While no procedure that has people as its focus will be without error, tests do an excellent job of minimizing error.
More than seventy years of theoretical and practical research at the finest universities and laboratories in the United States have gone into refining test theory. When developing a test, reputable publishers or their authors conduct research that rests on this foundation and is responsive to the highest current professional standards.
Unless the critic can point to an alternative procedure that rests on a similar body of evidence and has proven to be equally reliable, fair, and valid, it is best to take the criticism with appropriate skepticism.
HOW DO TEST PUBLISHERS ENSURE THAT TESTS ARE FAIR?
Naturally, the first step is to instruct the people developing a test to avoid questions or content that appear unfair, stereotypical or biased toward any group that might take the test. Another step is to have a panel of experts review the questions for potential bias. Such a panel would include members of various groups that might be affected by bias in the tests results. Then, before it is published, the test may be given to individuals from different groups to see how well they perform. The data are evaluated to verify the work of the panel of experts, checking whether the test questions unfairly favor one group or other and whether the test predicts outcomes equally well for different groups. Test materials that do not pass these reviews are removed.
WHY ARE QUESTIONS ON SOME TESTS SO PERSONAL?
This will vary quite a bit from one kind of test to another. In some educational and business applications for example, certain questions may be inappropriate for example, questions about sexual preference or political or religious beliefs. Tests used in these settings must be carefully chosen and implemented with these restrictions in mind.
In some other applications, however, personal questions are necessary. On tests that are used for psychological counseling and treatment, questions are personal because peoples problems are personal. When people are having marital difficulties, for example, a counselor must seek answers to questions about sexual behavior, money management, arguments over childrens' religious training, occurrence of aggressive or violent behavior, drinking problems, and so on. In these instances tests are particularly useful. They let each person answer the questions privately and safely; they consistently ask all the questions so that important issues are not missed; and they allow answers to be compared with a large database to see how problematic or extreme the persons concerns may be.
In these cases it is important that consent be obtained from the person being tested before the test is administered. For minors, written consent must come from a parent or other legally responsible adult.
WHY DO SOME TEST QUESTIONS APPEAR SO SIMPLE OR POINTLESS?
Take as an example some statements that might appear on a test that assesses emotional distress. What does it mean if someone answers Yes to the statement: My thoughts are often jumbled. Anybody could feel like that one day and not the next. The truth is, a feeling that ones thoughts are jumbled does not mean much by itself. But now what if the person also answered Yes to the statements, It seems like I am always tired, I often feel guilty, and I usually wish people would just leave me by myself. Like the jumbled thoughts example, these are statements that, taken singly, each of us could probably have endorsed at some point or other. On the other hand, if an individual responded Yes to all these statements, a mental health providers attention might be drawn to potentially serious concerns.
Other kinds of tests will ask very different kinds of questions, yet there will often be the same sense that individual questions do not seem important. A test does two things with questions such as these. First, it groups many of them together in sets so that the meaning becomes focused and a single response or answer does not get overemphasized. Second, it documents how large numbers of carefully chosen people responded to these sets of questions. A tests power derives from the care that goes into assembling sets of questions and then studying how groups of people answer them. Separated from that background of research, the point is never clear in a single persons answer to a single question.
WHAT RIGHTS DOES A TEST TAKER HAVE?
First of all, a person (or the parents of a child) should voluntarily agree to a test administration, just as he or she would to sitting for an interview. It is also a right of a test taker to receive some information about the purpose of the test and how the results will be used. Further, it is the test takers right to have the results of the test kept confidential except for the test taker (or legal guardians), the experts who are authorized to give and interpret the test, and the individuals who are involved in the decision making process for which the test was administered.
IS IT ALL RIGHT TO GET THE TEST BEFOREHAND AND STUDY THE QUESTIONS?
No. Even on tests where there is no incentive to cheat, it is very important that the test questions remain unknown until the test is given. That's because a test is valid only if all persons take it under the same conditions that is, on a level playing field, so to speak. Any familiarity with the questions ahead of time may distort the results. Test publishers and responsible test givers go to great lengths to guard the integrity of the tests to ensure that they will be fair to all test takers.
The American Psychological Association and most state and provincial agencies for professional licensure have standards and codes of ethics on the proper use of tests. Publishers and users of educational tests also have strong policies on test use. Any person who has a concern about the misuse of test results should promptly call the appropriate professional agency.
ONCE A TEST IS PUBLISHED, IS IT EVER UPDATED?
Yes. Publishers try to ensure that the tests they publish do not become obsolete. They do this by periodically updating a test or extending its validation. For example, norm-referenced tests, which compare an individuals results to the results of a group of his or her peers, must be frequently updated since national norms may change over time.
WHAT KIND OF TRAINING SHOULD A TEST USER HAVE?
Persons who purchase and use tests should be qualified and trained in the application of the tests they will use. It is their responsibility to understand the nature of the test they are giving, to be aware of the research that validates the test, and to know the proper conditions under which the test should be given. They are also responsible for knowing how to interpret the results appropriately.
WHO INTERPRETS THE TEST RESULTS?
A trained professional such as a psychologist, counselor, or human resource manager typically interprets the results of professionally developed tests. Test publishers make available extensive aids for interpreting tests. These aids include manuals, printed reports, or interpretive guides that help test administrators report and apply the results properly. The publisher also provides the research results on which the test report or interpretations are based.
IS IT POSSIBLE FOR A TEST TAKER TO MANIPULATE OR CHEAT ON A TEST?
Although some people believe they can outsmart a test, it is seldom the case. First of all, most people don't cheat on a test if they are seeking some kind of help by taking the test. They realize that only candid answers will help professionals give them appropriate and effective assistance.
Tests that are used in settings where there may be an incentive to manipulate or cheat are constructed with checks and balances that identify inconsistent or inappropriate answers. These alert the test administrator that someone has attempted to answer questions deceptively. Test publishers and test users are also careful to maintain the security of tests that may be used in these situations, making sure that test materials are properly kept in secure places and are not released to non-professionals.
WHO SETS THE STANDARDS FOR TESTS?
Ultimately, the test publisher is responsible for the quality of the test. However, tests used in schools, businesses or other public settings are regulated by federal and state laws. Also, test developers and publishers are generally members of and support professional groups that have published standards for test development and use.
Questions About Testing in Business Settings
Business organizations use a variety of tests as aids in hiring, placing, or promoting employees. Tests can help human resource professionals find the right person for the right job as well as advance an employee along a career path that makes the best use of his or her talents and strengths. Tests can be also be used in putting together effective teams or work groups and in helping them to operate well together. Testing, when properly used, results in more motivated, productive, and satisfied workers and less friction and dissatisfaction in the work force.
WHAT KINDS OF THINGS DO TESTS MEASURE IN BUSINESS?
Employers sometimes use certain tests to help judge an applicants suitability for a particular kind of job or level of responsibility in other words, to match persons to the jobs for which they are best suited. Tests may also measure specific aptitudes such as mechanical skills or sales abilities, or characteristics such as trustworthiness.
Tests are also used to determine the training needs of organizations, their departments or sections, and employees. Test results can indicate whether individuals know how to do their jobs and which specific task need additional improvement. Human resource departments can use this information to target training needs and implement interventions that support increased productivity. Tests can also measure the effectiveness of training that has been conducted in the organization.
IS IT LEGAL TO USE TESTS IN HIRING?
Yes. Tests are just one of a number of procedures that have a legitimate place in making employment decisions. Certainly some kinds of questions for example, about age, religion, or other private and protected information may not be asked and tests used in hiring decisions must be designed with this in mind. But these concerns are not limited to tests. The same federal and state guidelines that apply to tests also apply to interviews, background checks, and so on.
WHY DO COMPANIES RELY ON TESTS?
Tests have several advantages over traditional interviews and other commonly used employee hiring and placement procedures. Tests are even-handed; they ask the same questions of everyone. Tests typically require less time than interviews, so they are more efficient in obtaining job-related information. Appropriate tests have been carefully screened to be fair and unbiased and not to ask for improper information. Tests allow the persons answers to be compared with hundreds or even thousands of other peoples answers to the same question under the same standard conditions. Finally, the decisions made from test results are based on research studies that prove their accuracy and effectiveness. No other procedure can make these claims.
DO TESTS INVADE PEOPLES PRIVACY?
It is possible that a misuse of a test could result in an invasion of privacy. For example, a clinical test intended for use in medical settings should not be used as a part of routine employment screening. But well-designed tests that are used for their intended purposes do not ask questions that invade a persons privacy.
Also it is true that people are sometimes unaware of what they are revealing about themselves by taking the test. However, taking a test is in many ways like answering questions during an interview. A person is equally unsure about inferences or interpretations made by an interviewer. The difference is that the questions on tests have been screened for fairness, are applied consistently to everyone, and that the resulting interpretations are based on research that has proved them to be reliable, fair, and valid. On the other hand, the opinions of an interviewer are just one persons views, however experienced that person may be, and interviewers can be distracted and influenced by prejudices of which they may not even be aware.
DO TESTS PREVENT QUALIFIED PEOPLE FROM GETTING HIRED OR PROMOTED?
Employment screening involves the selection of some people over others. Sometimes there will be more qualified people applying for a position than can be hired. A properly chosen test has an important place in selecting the candidate with the best fit to the position. Using other procedures without considering the results of a good test will tend to rule out more qualified people than will a procedure that includes an appropriate test.
SHOULD A PERSON BE HIRED OR PROMOTED ON THE BASIS OF THE RESULTS OF TESTS ALONE?
Even a battery of tests should not be the sole deciding factor in hiring or promotion. Properly used, tests are only one part of a process that includes other steps such as application forms, personal interviews, supervisor ratings, and background checks. Considered together, the results of these techniques can provide a more comprehensive picture of an individual to help an employer make the right decision for both the employee and the company.
WHAT ABOUT TESTING FOR THE DISABLED?
Appropriately developed and properly administered tests used in hiring new employees conform to legal requirements with regard to disabilities. Test publishers can, and often do, advise companies on ways to reasonably accommodate test takers with disabilities. Where it is appropriate and feasible, norms for specific disabled populations are developed.
Questions About Testing in Schools
WHAT TYPES OF TESTS ARE MOST OFTEN USED IN SCHOOLS?
The most common tests administered in schools are norm-referenced, group-administered achievement tests. These have been very widely used for many decades. The results are used to evaluate the achievement of children the skills they need to learn and the effectiveness of schools is in teaching these skills. Test results are also useful in helping teachers and parents learn what kinds of strengths and weaknesses a child may have in different subject areas.
These tests begin with a close collaboration between curriculum experts and measurement experts from major universities, school districts, and test publishers. These experts identify major academic skills and bodies of knowledge that children are expected to know and then they create appropriate test questions to assess the students skills and knowledge. The questions are then administered to a sample of thousands of children throughout the country that represent the national student population. The careful standardization of all materials and administration to well-defined comparison groups make it possible for test users to compare their results to those of all individuals, districts, or regions in the country.
ARE ALL STANDARDIZED TESTS NORM-REFERENCED?
There are also fair, reliable, and valid standardized tests that are not norm-referenced. For example, criterion-referenced tests go through the same initial process of collaboration between curriculum and measurement experts to identify necessary skills and knowledge and to develop test questions. Then the curriculum experts define what or how many questions should be answered correctly to demonstrate grade-level mastery. This is much like an individual teachers classroom test, but vastly enhanced through the input of widely recognized curriculum experts, standardization of the tests presentation, and verification of its stability, fairness, and accuracy so that results can be compared from one administration to another.
WHAT ABOUT PROPOSED NEW PERFORMANCE ASSESSMENTS?
Recently there have been efforts to create different tools for documenting a child's progress in school. The most common are called performance and portfolio assessments. These assessments involve observing the actual activity or behavior that is learned for example, giving a speech or doing a science experiment in the case of a performance assessment or assembling a record of the actual day-to-day class work a child does in a portfolio assessment. These approaches can communicate certain aspects of learning to teachers and parents. These techniques are not generally considered tests, however, since most lack a recognized basis for comparing results among children, classrooms, schools, and so on. Even if a basis of comparison were developed, it would be necessary to evaluate these tools for reliability, fairness, and validity before they could be adopted for the broader purposes that standardized tests serve.
HOW DO SCHOOLS USE INDIVIDUALIZED TESTS?
Schools use many different kinds of tests to help identify the individual factors that may stand in the way of a child's successful learning. Intelligence (or mental abilities) and achievement tests are among the most commonly used tests. Their results can show whether specific learning disabilities underlie school difficulties, rule out cognitive problems in cases where emotional disturbance is suspected, and identify the underachiever who may benefit from counseling or tutoring.
There are also many other kinds of tests that are not as well known yet are widely used. Developmental tests are used to screen children when they begin school so that their teachers will have a head start in understanding their special strengths and weaknesses. Speech and language specialists use tests of articulation and word usage to design and evaluate treatment programs for children with speech problems. Counselors and school psychologists use behavioral inventories to identify problems and help teachers develop proper behavior management techniques in classrooms. Ability tests are used to identify gifted and talented students who would benefit from an enriched learning program.
HOW CAN TESTS BEST HELP MY CHILD?
As with any tool, tests should be used with care and only when they suit the purpose. Individualized tests are administered when there is a need. Some signs of this need are continuing academic difficulty when parents and teachers efforts haven't worked; signs of emotional distress; and behavior problems beyond what is usual at the child's age. If problems like this are of concern, you should find a professional who can help you decide whether testing is worthwhile and determine which tests may produce the most helpful results. School psychologists and counselors know about these kinds of problems. Outside of schools, there are psychologists or counselors and child psychiatrists. Pediatricians often make referrals to appropriate testing specialists.
Sometimes a screening test may be given more widely for example, to all children entering kindergarten. Screening tests are specially constructed for such broad applications. What is more important, the recommendations provided by screening test results are limited they are often just a referral for more thorough testing if signs of problems do appear. The purpose of a screening test is to catch potential problems early. It is usually a good idea to have a screening test done if a school or medical care provider recommends it.
HOW DO INTELLIGENCE OR MENTAL ABILITIES TESTS HELP STUDENTS?
A child may do poorly in school for reasons unknown to a teacher or a school psychologist. An intelligence test can uncover valuable information about cognitive abilities that assist the teacher in helping the child develop his or her full potential. For example, intelligence tests in conjunction with other tests may reveal that a child may be having difficulty learning in school because of emotional problems or some undiagnosed physical disability rather than limited intelligence. Once the causes are identified, the child can receive appropriate assistance from medical, educational, or mental health specialists.
DO IQ TESTS DISCRIMINATE AGAINST MINORITIES?
Independent researchers have studied modern IQ tests and found that test results are valid and reliable for the major population groups in the U.S. When used properly with other instruments, these tests can help educators create the best learning situation for a child who is having difficult learning.
WHY IS THERE SO MUCH CONTROVERSY ABOUT IQ TESTS?
One kind of controversy arises when some people draw broad conclusions about population groups or the origins of mental ability based on results from intelligence tests. Intelligence tests are not designed to support such conclusions. These tests used within their own limits have been and continue to be very useful to educators in assessing the capacities of students to develop and learn.
Another controversy stems from concern about whether there is more than one kind of intelligence. Most measurement experts would agree that there are indeed more kinds of intelligence than are measured by conventional intelligence tests (or even can be measured by a single test). They would also agree that the kind of intelligence tested on modern IQ tests is nevertheless an important characteristic that has a proven value and a continuing place in helping schools to help children learn.
Questions About Testing in Clinical and Medical Settings
HOW DO TESTS HELP MENTAL HEALTH PROFESSIONALS AND THEIR PATIENTS?
Psychological tests enable mental health professionals to make diagnoses more reliably, validly, and quickly than they can from personal observation alone. Tests can uncover problems that a mental health professional may not detect until much later. This allows the clinician to focus on the appropriate treatment more quickly, thereby saving time and money for the patient or client. Once a course of treatment has begun, tests can help the clinician monitor the effectiveness of the treatment as it proceeds.
CAN PSYCHOLOGICAL TESTS ATTACH LABELS TO PEOPLE?
Tests are not designed to label people. In fact, responsible test manuals discourage any such linkage between results and labels. These manuals help users interpret the test results in a way that respects each individuals uniqueness. They also recommend that the test user base diagnoses or clinical decisions on several sources of information rather than on just a single test result.
However, some public groups such as insurance companies, legislatures, school boards, and human service agencies do attach labels to people. This happens when programs and treatments are standardized by categories. Thus, an individual seeking treatment or a child with learning problems often has to be diagnosed and labeled to fit into one of the categories in order to receive funding for the special help they need.
DO CLINICAL PSYCHOLOGISTS USE DIFFERENT TESTS FROM THOSE USED IN BUSINESSES OR IN SCHOOLS?
Sometimes. But most psychological tests, especially those measuring motor skills and mental aptitudes, can be useful in more than one field. A test for cognitive ability, for example, may be used in a school or at an employment office as well as by a mental health clinician. Also, some tests require high levels of training and expertise of test users for proper administration and interpretation.
CAN A TEST PROVE THAT SOMEBODY IS MENTALLY DISTURBED?
Tests alone cannot diagnose a person. They are tools used only by qualified and competent professionals, just as a physician may combine information from X-rays, blood tests, and personal examination of the patient in reaching a physical diagnosis.
WHAT GOOD ARE TESTS WHEN SOMEBODY HAS ALREADY BEEN DIAGNOSED WITH A PSYCHIATRIC DISORDER?
Psychological tests are often used to monitor a persons response to medications, which are used increasingly in treating a number of disorders, including depression, schizophrenia, and attention deficit disorder in children. The tests can help to track progress during a course of treatment to determine if a person is receiving the correct dosage, or is responding correctly to the medication. In a similar manner, tests can help monitor a persons response to other therapeutic treatments such as psychotherapy. Tests can also serve to confirm or reject potential diagnoses.
It's true that medical imaging techniques are now widely used to help physicians diagnose neurological illnesses. However, neuropsychological tests are useful for screening patients for signs of neurological disorder. They are also helpful, and far less expensive than computerized tomography (CT) scans and MRIs, for monitoring patients over long periods of recovery or rehabilitation. Through neuropsychological testing, a physician can measure the progress of stroke victims, for example, and determine whether a particular treatment is working properly.
For more information contact:
The
Association for Test Publishers
601
Pennsylvania Ave., NW
South Building, Suite 900
Washington D.C. 20004
866.240.7909
©1997 - 2003 Association of Test Publishers