©2002 Association of Test Publishers
Save or read as pdf file

PSYCHOLOGICAL TESTING IN THE WAKE OF 9 / 11

Invasion of privacy -- or rather, what the courts deem to be an invasion of privacy -- may be a whole new ball game since September 11th, theorized ATP General Counsel David Arnold, Ph.D., J.D., of  Reid London House ( a business of NCS Pearson), in his keynote address to the ATP General Meeting held February 4th, in Carlsbad, CA.

Arnold's address, Technology, Legal and Usage Issues: Psychological Testing in the Wake of 911, looked at how changed societal attitudes toward privacy, the practice of psychology, and psychological testing,  would impact the assessment industry.

"A year ago if you had been asked to remove your shoes or hand in your nail file before boarding an airplane, you would have considered it an invasion of privacy.  Now the public's expectations have changed," said Arnold.  And he guessed that courts’ expectations will change as well in regard to many types of tests, and that the public's attitude toward the use of many types of psychological tests will also change.

Arnold, who is educated both as an industrial psychologist and an attorney, explained that according to the courts, privacy is not an absolute right.  Courts will consider the perceived degree of invasiveness of a procedure, as well as the justification for the privacy invasion in rendering judgment.

To highlight his point, he used the example of a 1991 California case (Soroka v. Dayton Hudson, 235 Cal.App.3d 654) in which Target stores was sued over their use of the Minnesota Multiphasic Personality Inventory ® and California Psychological Inventory. Although the case never went to trial, both the trial court and an appellate court rendered opinions regarding whether the plaintiffs should be granted a preliminary injunction to halt Target’s use of the tests until the case was decided at trial.  While the trial court held that the use of these clinical assessments was justified by Target’s interest in hiring emotionally stable unarmed security guards, the appellate court held that Target’s justification wasn’t adequately compelling.  Simply stated, the court felt the tests were invasive and the nature of the job didn’t justify their use. 

Post 9-11, Arnold ventured a guess that the same appellate court, given the same case today in a climate of concern due to potential terrorism, would not have considered the tests to be quite so invasive.  Moreover, that the court would have probably been inclined to hold that  Target’s interests adequately justified invading its job applicants’ privacy.  Additionally, Arnold indicated that when it comes to putting federal marshals on airplanes with loaded guns to act as police officers, clinical screening and "invasive"
questions will be "imperative." And that this will have a trickle down effect to other jobs with less significant implications for security.

Arnold also detailed society's changed views toward psychology in general.  He pointed to  the New York City Police Department which last fall ordered all 55,000 employees to attend mental health counseling, and to the New York City Fire Department which assigned counselors to make rounds to all fire houses.  Arnold referred to such efforts as "Psychological First Aid" which, he noted, is becoming quite common and will, in turn, require a greater demand for test publishers’ tools.

As further evidence of this growing demand for testing tools, Arnold pointed to recent articles in  human resources, retail and security publications in which businesses were urged to “beef up” their screening procedures. He pointed to the Aviation and Transportation Security Act which  specifically states that applicants for security screening positions in airports need to demonstrate “no impairment due to drugs, sleep deprivation or alcohol as well as obtain a satisfactory score on a personnel selection examination.”

Arnold also reported an increased volume of calls into his office from potential test users and writers regarding the legal and technical  issues surrounding tests.  And he ventured a guess that this increased interest and activity was typical for many professionals within the assessment industry.

According to Arnold, employers have traditionally walked a fine line between invasion of privacy and negligent hiring claims.  “With the former, the employer is accused of delving too deeply into the applicant’s background, whereas with the latter, the claim is that the employer didn’t dig deep enough.  And in the wake of September 11th, this line has arguably moved towards allowing and expecting employers to delve deeper into a job applicant’s background, which may commonly include clinical personality testing,” Arnold reasoned.

Now, should tragedy strike, he said, the courts and the public will say, “you didn’t delve deeply enough. You didn’t go far enough, you didn’t do enough testing.”

ATP Member Business and Satisfaction Survey:     SALES are Up; SATISFACTION is High

Sales were up last year for most Certification/Licensure, Clinical and Educational test publishers. The highest percentage of change (more than 30-percent) was reported by publishers of Certification/Licensure tests. 

Those figures are according to ATP’s recently released Member Business and Satisfaction Survey which asked members to compare gross sales revenues in the year 2001 with their sales activity in 2000.
   
According to the survey,  sales slipped for many Industrial/Organizational test publishers as compared to 2000. And although a few companies did report modest increases, none of the increases topped 30 percent.
   
In Part II of the survey, members reported that their satisfaction with the Association remains high, with 53-percent of respondents reporting in as satisfied and another 41-percent reporting in as very satisfied.  A minority of respondents (six-percent) reported in as dissatisfied.
   
“We were pleased, of course, to see those numbers -- the very high satisfaction rate and the very low rate of dissatisfaction-- but even that six- percent is a rate we will explore and certainly not ignore,” said ATP Executive Director William G. Harris.

Members were also asked why they joined ATP.  The top three reasons given were networking, legislative advocacy and professional identity.  Other reasons cited were the conference, publications and insurance.
   
“Legislative Advocacy and Professional Identity have long been cited by members as top reasons for joining ATP since its inception as an Association,” noted Harris.  And he added, “The Board has always strived to be responsive to that mandate through their support of a strong advocacy program as well as the creation of professional guidelines.”

And, although respondents were given an array of choices regarding how they heard about ATP such as the website or conference advertising, an overwhelming majority of members reported that they heard about ATP through colleagues.

Harris noted that he was not surprised that word- of- mouth continues to be ATP's best recruitment process. “Our best recruiters have always been our own members,” he said.
   
As far as future direction is concerned, members gave high rankings of over 50-percent endorsing the publication of “White Papers” on web-based testing and test security,  digital publishing and copyright law and test prep issues.  More than 80-percent of respondents cited ATP’s pursuit of a comprehensive public relations program for the testing and assessment services industry and the pursuit of test publishing market research  as “Very Important”. 
   
Less than half the respondents said that continuation credit for conference attendees was important and respondents gave mixed reviews to distance learning -- with one third saying future distance learning opportunities were very important and a third saying they were not very important.  The remainder fell into the not important to not at all important range.

The ATP website received a passing level of satisfaction as did the Test Publisher newsletter, the format of the General Meeting and communication among Divisions. Respondents seemed especially satisfied with the combining of the General Meeting with the annual testing conference.
   
A myriad of responses were generated from the question,  “In the next 36 months what are the three most important issues or developments that the test publishing industry will face?”  Among the responses were: IDEA (Individuals with Disabilities Education Act) reauthorization; Internet testing;  Increased demand by clients for ISO 9000 or 17024 accreditation; Rise of Non-Professional Test Preparation or Brain Dump Sites; Test Security; Copyright; and Fighting Negativity in the Press.

“Overall, leading concerns from respondents centered on increased information and content in the form of white papers, an enhanced on-line journal and a clear, focused Public Relations effort.  As in the past, the goal of the ATP leadership will be to respond to these directives,” stated Harris.

For a complete copy of the Survey Results visit the ATP website at www.testpublishers.org.
 

ATP 2002 Conference Presentations Provide Glimpses of the Past, Present and Future for Computer- Based Testing

"The future ain't what it used to be" -- Yogi Berra

That was the quote with which Jerry Goldman, professor of  political science at Northwestern University concluded a keynote panel:  Showcase of Innovations in Computer-Based Testing, presented February 6, at the third annual Association of Test Publishers  conference held in Carlsbad, CA.   And it was an appropriate quote to sum up what the more than 450 conference attendees experienced in the varied content sessions, keynote addresses and sponsor-hosted workshops presented at the three day conference titled: Computer Based Testing: Assessment at Your Fingertips.

Goldman and his fellow presenters, James Olsen, Ph.D., Chief Scientist and Vice President, Alpine Media Corporation and Dick Devore, Ed.D., Simulations Development Leader for the American Institute for Certified Public Accountants (AICPA) Examination Team, were just one of the many presenting groups and individuals that gave conference attendees not just a glimpse of the future -- but an insiders’ view of what is happening now.

Olsen opened his presentation with a display of unique simulations being developed at Alpine Media which included a computer-based learning environment in which students could actually dismantle and reassemble three dimensional objects to better understand their properties and manipulate on-screen protractors and compasses as if they were hand-held.

Olsen's presentation was followed by Devore who shared the ongoing development of the AICPA’s goal to transition the Certified Public Accountant’s exam into a computer-based exam by 2003.  Devore demonstrated how testing innovations were being applied in order to provide test takers with practical demonstrations of  “real life” skills such as filling out actual W-2 and 1099 forms which can appear on screen alongside sets of different financial figures. 

And finally, Goldman introduced The AMbER model -- which is an acronym for Applied Multimedia-based Experimental Research.  Goldman took onlookers on a virtual journey into a courthouse with simulations so real, that a security guard halted progress at one point and demanded user identification and password information in order to continue.  Onlookers than entered a judge’s chamber, picked up a folder of information awaiting his review and moved on to the actual courtroom.  Once inside and seated it became apparent that a sentencing hearing was being conducted on a serious domestic dispute.  Statements were given by attorneys on both sides, as well as parties to the litigation.  The judge was then required to render a sentence.

The real interest however takes place when the test giver can then manipulate the courtroom scene -- changing the gender, attire, race and even speech patterns and presentation of the individuals. The potential is required for the scientist to compile statistics on the types and the severity of sentences that are handed down in comparison to the race, gender, dress or other variables that are presented.

In another keynote presentation on February 5, Jack Thorpe, Ph.D., Colonel, USAF ret., traced the history of computer simulation in the military and showed how the private civilian video game industry served to fuel innovation in the military and vice versa. Though the official title of the presentation was Distributed Simulation: The Need to Test for Teamwork Skills Using Technology-Based Assessments, Thorpe noted lightheartedly that an appropriate abridged title might be Mario Goes to War.  And he emphasized the strides that can take place when the government and private sector feed off each other in positive environments.

And in the closing keynote address, given by Willard R. Daggett, Ed.D., President of the International Center for Leadership in Education, Daggett described himself as knowing “just enough about testing and just enough about education to be dangerous."

Daggett, who is known worldwide for his efforts to modify education systems towards more rigorous and relevant skills and knowledge for all students, went on to describe his work as an allegory to the keyboard.  He travels with a portable, expandable key board which he likes to take out as a display.  He pointed out that the standard keyboard used today was created in 1919 -- and is an example of how a potentially flawed tool is continued to be used because the effort to make change is too great a prospect.  He presented the idea that many of the curricula and educational standards being used in the schools today are like that outmoded 1919 keyboard.
   
“There is a large gap between what we’re teaching and what kids are going to need when they leave school,” he said.
   
 “Do we teach knowledge for knowledge sake or do we teach knowledge for students to be able to apply it to real world situations - both predictable and unpredictable?” Daggett asked the audience.
   
He pointed, as an example, to the way that people communicate in email as opposed to the type of grammar being taught in school.
    
 “Schools need to look at what kids really need to know ...(as opposed to) what is on the test. And it is the responsibility of those in business and industry to communicate to the schools what kind of education and skills these students will need in the workplace.”
   
Daggett’s message to test publishers and test developers was succinct, “Success will go to those who figure out how to do two things: how to use common metrics and communicate them effectively;  and how to use good data to convince school districts what they need to take off the plate.”

ATP FOCUS ON...
   
The Department of Veterans Affair’s Professional Certification and Licensure Advisory Committee

Giving back to America and to the service people who have given so much of themselves to protect their country” -- that, said Sandra Winborne, is what her job as Certification Program Manager for the Defense Activity for Non-Traditional Education Support (“DANTES”) is all about.  And so, with that kind of supportive attitude, along with her years of expertise in military education, counseling services, and professional certification it was a natural fit for Winborne to assume the role of Chairperson for the
Department of Veterans Affairs’(“VA”) newly created Professional Certification and Licensure Advisory Committee (“PCLAC”).

“We are a hardworking Committee well grounded in what we believe,” asserted Winborne about the seven member Committee appointed in May of 2001.  The PCLAC  is charged with advising Secretary of Veterans Affairs, Anthony J. Principi, regarding the requirements of organizations offering licensing and certification tests as well as other related issues. The PCLAC  is sanctioned  under Public Law 106-419 (Section 122), which expands educational  opportunities for individuals on active duty or veterans with benefits under the Montgomery GI Bill.
   
The PCLAC is made up of representatives with expertise in areas of certification. licensure, testing, standards, assessment, analysis and education.  Another notable Committee member is ATP Executive Director, Dr. William G. Harris.
  
“Recommending G. for the Committee was one of the best things I ever did,” Winborne enthusiastically noted. “He brings real testing knowledge to the table. His understanding of testing is so broad when it comes to providing information for our consideration that assists the Committee members in reaching decisions.

G. Harris, along with Dr. Elizabeth Kolmstetter, Senior Director for Standards, Assessment and Certification for the National Skill Standards Board (“NSSB”) provides the psychometric framework for the Committee. Other members of the Committee include Lisa Lutz, Senior Policy Analyst at DynCorp I&ET, Michael Martin, Executive Director of the Commission for Certification in Geriatric Pharmacy and Michael Clark, General Manager of the National Institute for Certification in Engineering Technology.  Some members of the Committee also represent a coalition or quality standards organizations such as, the National Organization for Competency Assurance (NOCA) or the Coalition for Professional Certification (CPC).

Committee members, said Winborne, will serve for a minimum of two years.  In that time they hope to assist the VA State Approving Agencies who review certifying or licensing organizations, make recommendations for enhancements to the existing law, and handle other issues for advisement to the Secretary.
   
Winborne emphasized that ATP members need to become involved, “There is opportunity for companies to become involved in the process...they would be doing America a wonderful favor, while at the same time there is revenue potential in it for them (ATP members),” she noted.
   
Essentially, the law provides that service-connected individuals, including their dependents, spouses and/or surviving spouses, are eligible to receive up to $2,000 per test with no limit on the number of tests they can take.  They do not have to pass the test to receive the benefit and they can retake a test they fail.  Test takers can also apply to renew a license or certificate.
   
Winborne urged ATP members to visit the website at www.gibill.va.gov.  There, visitors can learn about the programs and fill out the paperwork necessary to participate.
   
 “You will be doing our nation's veterans a big favor if you participate in this program,” she said. She noted that the processing time for applying organizations varies from state to state and that organizations must meet certain guidelines.

“A test must be generally accepted within the industry and  the company must have offered the test for a minimum of two years,” Winborne said. And just as in her work with DANTES, Winborne emphasized the opportunity for companies to get involved not only for the revenue potential, but also for the opportunity “to give back to America.”

Association Notebook

Enhanced website...A website re-design committee is hard at work enhancing the ATP website with a new look and feel. Check it out at www.testpublishers.org.  And if your company is not listed twice on the website -- once in the Directory, and a second time in the Products Provided by ATP Members section, contact ATP webmaster Pete Wohlmut at webmaster@testpublishers.org. If you have forgotten your password contact Lauren Scheib at LScheibatATP@aol.com

The next ATP conference... will be held February 24 -25 at the Amelia Island Plantation Resort in Amelia Island, FL (Approx. 30 miles from the Jacksonville airport.)

The next board meeting of the Association of Test Publishers...will be held Friday, September 20 in Snoqualmie, WA. Members can have business placed on the agenda by emailing the ATP Board of Directors: laurens@testpublishers.org or wgharris@testpublishers.org

ATP Welcomes its newest members...Measured Progress, Inc., Hubbard & Hubbard, Inc., Professional Credential Services, Integral 7, Inc., The Oswald Group, G. Neil and the Graduate Management Admission Council (“GMAC”).

ATP Gratefully Acknowledges its sponsors for the 2002 Computer-Based Testing Conference:

PLATINUM
CAT*ASI • The Chauncey Group International • Educational Testing Service (“ETS”) • Galton Technologies, Inc. • NCS Pearson •  Prometric, a Thomson Learning business • Question Mark • TestOut Corporation

GOLD
ACT, Inc. • Applied Measurement Professionals (“AMP”) • Buros Institute for Assessment for Assessment Consultation and Outreach (“BIACO”) • i-asses.com • Scan-Optics, Inc. • Vantage Learning