Breaking News: On May 11, 2020, the Los Angeles Times reported that the University of California President recommended that the SAT be suspended for U C admissions. That is huge! Undoubtedly, as the U C goes, so will the nation.

Yet for some of us SAT-watchers, it was not “breaking” news. We have watched the SAT over the last many years slowly destroy itself as the testing gold standard. Let’s call it a self-induced suicide (an oxymoron, undoubtedly).

Sadly, as I see it, the SAT, which is used all over the world, has lost the right to call itself the gold standard in testing — a benchmark that is valid and reliable. A valid and reliable test is one that measures what it purports to measure and reports scores that others can rely on. There may have been a time when, if the SAT says it — then it is so. But, that seems to have ended a while ago. While the LA Times does not cite the lack of validity or reliability of scores, my sense is that the long sad saga that got us here has been part of the woodwork for a while. In short, these scores no longer are what they purport to be.

I see three main reasons for this decline. Most importantly, the test prep and the accommodations industries; and also, the slow drip of errors by the College Board, which sponsors the SAT, along the way.

First. The test prep industry. A bit of history.

The test prep industry was not always there. The SAT was developed in 1926 as an aptitude test to encourage and promote testing by students who may have been diamonds in the rough — underprivileged students who did not attend fancy schools or live in fancy homes. Like my immigrant brother, who scored 800 in math on the SAT in the 1960’s — a fact that changed his life. Over the SAT’s almost 100 years, we’ve come a long way from its initial purpose. It no longer tests “aptitude.” Instead, its goal is to predict freshman year grades in college.

NPR reported, “SAT prep was born in a Brooklyn basement in 1938, when Stanley H. Kaplan began tutoring students on how to prepare for standardized tests. Word spread, and soon students began arriving from all over the country to learn techniques for passing standardized tests. In the early 1970s, Kaplan expanded his business…., opening 70 centers by 1975. The Washington Post Company — owners of The Washington Post — bought the company from Kaplan in 1985.”

“SAT prep was big business, and…Kaplan Educational Centers, would become the largest such operation in the nation. Another company, The Princeton Review, was growing as well. Like Kaplan had in 1938, John Katzman began his company by tutoring a handful of New York students after he graduated college in 1981.”

There are other companies in the test-prep business and “[m]illions of students have spent have spent millions of dollars preparing for the SAT.”

For full article, please visit: https://www.pbs.org/wgbh/pages/frontline/shows/sats/test/history.html

For a long time, the College Board vigorously maintained that one can’t prepare for the SAT,students and parents who could afford to do so, doled out lots of money for test prep told another story. Other students were left behind. Eventually, even the College Board had to acknowledge that some students’ scores improved with test prep, and teamed up with Khan Academy to develop test prep material.

So what do today’s scores actually mean? Are they valid and reliable?

Second. The accommodations industry. In 2002, the College Board made a fateful decision — to stop ‘flagging’ test reports of students with disabilities who took the test under nonstandard conditions. Soon thereafter, the ACT followed suit. See my 2003 story about this decision at https://www.educationnext.org/disablingthesat/

The flag was designed to give score readers (such as college admissions officers) notice that the test had been given with extra time or other accommodations/ modifications. Therefore, the scores needed to be read with care as they were no longer standardized. Since the test was normed as a timed test, extra time modified its validity standards. Amazingly, ending the flags meant that nobody was notified of that fact. Slowly, over the years, the numbers of students taking the test with 50% or 75% or more time grew from 2% to close to 5%. And, quite predictably, most students who used extra time were vying for the small sliver of selective college seats. We began to see that the affluent used this avenue far more than disadvantaged students; for example, we started to see reports that at some exclusive prep schools 40% of their students had extended time.

It’s fair to ask what today’s scores actually mean. Are they still valid and reliable?

Over the years, in these two ways, the shine on the gold standard test — also called a common yardstick — has worn off. Today’s test results need to be read carefully — with a whopping grain of salt. Do we really know what they mean? Are they still reliable predictors?

Third. Over the years, the College Board has been confronted with controversy. In 2005, it recalibrated the SAT and added a writing component. Later, it changed the scoring, even as recently, SAT scores have declined. At another point, it tried to neutralize advantages of males over females in test results — even as females turned out to be better college freshmen students, belying the SAT’s stated purpose. Recently, it toyed with adding an “adversity score” which it dropped after facing an avalanche of criticism. Controversy continued about flaws in the “new” SAT, as well as the College Board’s handling of scoring errors that affected 5000 students. All the while, the rise of “test optional” college continued.

Drip. Drip. Drip. Perhaps not surprising, for an almost 100-year-old test.

Thus, in reading that U C President Janet Napolitano recommended that the SAT be dropped from admissions decisions (at least on a trial basis), we SAT-watchers were not surprised.

The next big question will be — assuming we still have colleges with admissions criteria after the pandemic, how will their admissions officers admit students in the future? Will the focus be on academic excellence and preparation or will it slide to other societal goals? You’re invited to join us watchers. Stay tuned.

This was originally posted on Medium

The provocative title of Greg Toppo’s piece in Education Next. How about untimed SATs for everyone, he posits. https://www.educationnext.org/support-builds-making-sat-untimed-for-everyone-possible-solution-gaming-the-system/

So interesting and so sad to me, as I ‘ve watched this unfortunate saga unfold since even before 2002. The SAT’s extra time story is the story that just won’t go away ever since the College Board made a total mess of if back in 2002/2003. That was when the CB (and shortly thereafter the ACT) stopped notifying readers by flagging (with an asterisk) test scores that were achieved under nonstandard conditions, such as extra time. Before then, these tests were indeed standardized timed tests. If a student received extended time, as was possible for a few — less than 2% of test takers — notice was provided to the readers of the test result reports — such as college admissions officers — that the score was obtained with a nonstandard accommodation; that is, the test was no longer standardized!

2002–3 opened the floodgates. As you’ll read below, Bruce Poch nailed that reality in Toppo’s article. I had predicted it back in 2003, in my own Education Next piece, “Disabling the SAT.” https://www.educationnext.org/disablingthesat/

Indeed, it was obvious for anyone who chose to look — to see!

In essence the CB and ACT invited students with disability labels to seek extra time on these tests — 50% 75% or 100% more time-and the gift was that nobody would know because the CB and ACT would no longer tell them! Standardized and nonstandardized testing would, henceforth, be lumped together in SAT and ACT reports!

We predicted then that this gift would lead to use/misuse of extra time, especially in mostly in wealthy communities. Besides my 2003 story, see Sam Abrams’ 2005 report, cited in Toppo’s piece. And now, it’s gone over the top as we’ve witnessed the criminalized extreme to which some parents have taken this in the “Varsity Blues” scandal.

While the CB’s Zachary Goldberg says that removing the flag was the right thing to do at the time, I disagree. It was the easy way out for this huge and powerful organization that was faced with the threat of a lawsuit at the time. They settled. Buckled. And in so doing, they wiped out validity from their prize product. They no longer kept the tests valid. This settlement confused parents, teachers and students — and still does. Over these almost 20 years, it became the story that they just can’t “disappear,” And tragically, as we all know, it has lead to unfair gaming of the system.

So how to fix it? Now, it appears that the wish among many seems to be to make these tests untimed. Sure, that may solve the gaming problem for now. But I say, careful what you wish for.

Sure, go ahead. Change the test — make it untimed —and while you’re at it, why not let students use their iPhones when taking the test or let them stand on their heads spitting wooden nickels to relax them…when taking it. Whatever. Sure, go ahead. And maybe create a new organization or company to create these tests.

But understand that you are creating and promoting different tests— no longer the SAT or ACT which are timed and standardized. At the very least, be honest about what you are doing. Sure, it may be a “Possible Solution to the Gaming Problem,” as the article’s tagline announces — but at what cost? And will it get us where we need and want to be?

Toppo quotes Ari Trachtenberg, a professor of electrical and computer engineering at BU, as questioning the accommodations practices. Accommodations are not rigorously analyzed or understood. Further, it’s not clear that giving more time to college students actually helps them in life or when they are faced with “high-pressure tech interviews, “ etc., etc., etc. Of course he’s right. As the article points out, the whole field of time-related accommodations is squishy at best.

Bruce Poch, a dean of admissions and executive director of college counseling at Chadwick School, is also quoted. He is spot on when he said the CB handed the keys to this problem to the world back in 2003.

Let’s analyze that a bit more. What I wrote about then and what is still true, is that the key to this mess is the CB’s refusal to tell us why the test is timed. Indeed Ruth Colker, a law professor at Ohio State University who is an advocate for untimed or extended-time SATs, writes that these tests should be untimed “unless they can show that the strict time limits are truly required for validity.” She is totally right, of course, in her focus on the key to this mess which has been here and ignored for almost 20 years!

Since 2002/3, many of us have made these arguments over the years . And yet, the beat goes on. More kids take the SATs and ACTs and these organization rake in more money.

The question to day is: WHY do we let the CB and ACT get away with just trying to maintain “test security” to keep their industry going? It’s maddening! Can’t they be put on the spot finally and answer the question: Why is the SAT timed? WHY? WHY? WHY?

We still don’t know. It’s more than time that the College Board level with the public and tell us.

This was originally posted on Medium

Go, Teen Vogue!

Here’s an important story in Teen Vogue about how the SAT and ACT are manipulated — and mostly, about how neither company (the College Board’s Educational Testing Service and the ACT)has taken decisive action to guarantee the validity of these tests. Little tweaks will not do it!

This is a good read! You’ll see me quoted toward the end of the piece (in bold), about the flawed accommodations policy of these two companies have implemented since 2003 ( basically, continuing to time these tests (for still unexplained reasons)… but not notifying test score users when extra time is given to some students) and how that has played out in the last 16 years. An unfortunate and sad tale, as I see it.

___________________________________________________________________

Why It’s So Easy to Cheat on College Admissions Tests Like the SAT

Schooled is a series that explores the nuances of the American education system by reporter Zach Schermele, an incoming freshman at Columbia University.

BY ZACH SCHERMELE, AUGUST 5, 2019

When the college admissions scandal was first unveiled by the Justice Department in March, the scintillating story now known as “Operation Varsity Blues” grabbed headlines worldwide. With competition for spots at elite schools growing ever more fierce, federal prosecutors alleged that wealthy parents paid between $15,000 and $75,000 to doctor their children’s standardized test scores. That was just the tip of the iceberg in that particular scandal, which was dramatic enough to get green-lit as a Lifetimemovie.

But to students in the thick of an increasingly cutthroat college admissions game — admission rates at top schools are still dropping to record or near-record lows — the pervasiveness of cheating on standardized tests was never a surprise.

“Going into my first SAT, a kid I knew asked me if he could cheat off me,” Caroline Skoglund, a rising senior at Darien High School in Connecticut, told Teen Vogue. Caroline refused. She had spent over 100 hours studying for the test, often working through dinner and sacrificing going out with her friends. It wasn’t her first encounter with students who wanted to cheat on arguably one of the most important tests in a high schooler’s life.

“I’ve heard kids walk out of the test and brag to their friends about how they copied answers from someone else,” she said. Caroline took the test four times before scoring in the 99th percentile — an achievement she’s proud of. But she also recognizes what she called a “frustrating” truth: “Many people simply cheat.”

Though some predicted Operation Varsity Blues would transform the way testing security is managed by the billion-dollar organizations that administer standardized tests, critics say real, substantive change has not been made in response. They say a lack of transparency from the largest test administrators in the U.S. — the College Board, which decides how the SAT and other tests are administered, as well as ACT — has led to uncertainty about how prevalent cheating really is on their exams.

One college student from California, who spoke on the condition of anonymity, told Teen Vogue that cheating on the SAT four years ago helped her get into a top-40 university. Because the exam was (and still is) administered across the U.S. on the same day, test-takers on the East Coast finished before she did, allowing her to find the answers on her phone during bathroom breaks.

“The East Coast kids are out and will start discussing questions on College Confidential,” she said, referring to an online college admissions forum. The student claimed proctors could neither stop her from using the restroom nor pat her down to feel for a cell phone — “so there was virtually no way [the proctor] could know.” Technology of any form is technically prohibited inside the testing center, and students’ scores can be canceled on any college admissions exam if proctors catch them using devices like cell phones during the test.

Students in different time zones still take College Board–sanctioned exams like the SAT, SAT subject tests, and AP tests within a similar window, meaning the time-zone vulnerabilities she used to cheat are still exploitable. Two different types of AP tests are usually given on the same date, secretly divided into two sets across testing centers, experts told Teen Vogue. According to Brooke Hanson, CEO and founder of SuperTutorTV, in cases with which she’s familiar, “all students nationwide get the same form of the test” on October, March, and May SAT test dates, with some exceptions for the “experimental” questions. Hanson spoke from personal experience and admitted individuals might get different tests if they’ve requested accommodations, and in general, cases involving accommodations, test-takers over the age of 18, and potential “foul play” may result in further scrutiny.

Students can take the ACT anywhere the test is offered in the U.S., and although the College Board would not independently verify similar rules for the SAT, testing experts told Teen Vogue that is still the case.

“Generally students can register for any SAT test given anywhere,” Hanson told Teen Vogue. “I’ve had students at camp in Michigan take the SAT in that state or if they’re out of town in NYC, take it there. I’ve also had international students fly to the U.S. to take the test here during the summer or on dates not offered abroad.”

Only proctors and sometimes test administrators, who are often strangers to students, are responsible for verifying their identities (unless an investigation is opened). In 2011, 20 teenagers involved in an alleged SAT cheating ring in Long Island were accused of criminal impersonation, scheming to defraud, and falsifying business records; many of them took the test outside of their own school district. In 2015, 15 Chinese students were accused of paying impostors to take tests, including the SAT, for them using fake passports at test centers in Pennsylvania. The lead defendant, who allegedly held an organizing role in the ring, later pleaded guilty; several others were deported. It is unclear whether any of the accused in either case were acquitted. But many of the exploitable rules around approved test-taking locations don’t appear to have verifiably changed.

And though high-stakes heists make big news, less sophisticated cases of cheating go virtually unnoticed. Another student, who also wished to remain anonymous, told Teen Vogue the ACT proctors at his school are “teachers who don’t really care enough to stop cheating.” According to the student, who currently attends a large public high school of several thousand students, copying from nearby answer sheets and whispering during the exam are common occurrences.

“AP exams are a joke,” he continued, referring to a test administered by the College Board and ETS. He said students are in a gym or classroom, “where every test form is the same, so cheating off the people near you is easy.”

“Parents are proctors and oblivious,” he said. The student’s school has not yet responded to Teen Vogue’s request for comment.

“The time restrictions on the ACT are way too difficult,” said another student, who told Teen Vogue he cheated on the ACT in June. The student spoke on the condition of anonymity and admitted using extra time during an ungraded portion of the test to flip back in his booklet and answer questions from the English section he hadn’t been able to finish.

“I don’t think time should be such a critical factor in a test like this, so that sort of justified cheating in my head,” he said. He claimed the extra time improved his score significantly.

Both the College Board and ACT do offer extended time to students who can show a medical history of a learning disability. Examples of affluent parents exploiting these “testing accommodations” with fraudulent doctor’s notes were widely cited in the Varsity Blues scandal, and a recent New York Times analysis found that 504 designations, which give test-takers extra time based on physical or mental impairments, are disproportionately common in rich communities. In a 2003 essay titled “Disabling the SAT,” education lawyer Miriam Kurtzig Freedman predicted that accommodations would lead to a phenomenon she called “diagnosis shopping.”

“These savvy parents now know that if they’re looking ahead to Billy or Johnny going to college and having to take the SAT or the ACT in junior year, they better get these [medical plans] in place early,” Freedman told Teen Vogue. “It’s totally dysfunctional, and it’s created by these organizations who are beholden to nobody.”

Freedman has been a vocal critic of the College Board and ACT especially after Operation Varsity Blues, calling them “extremely closed and nontransparent” organizations. Ed Colby, the senior director of media and public relations for ACT, said in a statement to Teen Vogue, “ACT takes test security very seriously, and we are continually working to enhance and enforce our test security measures.”

“We do not provide specific details about those measures for security reasons,” Colby said. A link to ACT’s website (provided by Colby) states, “We conduct extensive, and proactive analyses of our testing data in search of irregularities that could indicate misconduct.” ACT representatives also make unannounced visits to test centers and maintain a testing security hotline, according to the website.

A webpage created by the College Board after the Varsity Blues scandal erupted states the organization has since added personnel to its testing security team, among other measures. In a statement to Teen Vogue, College Board spokeswoman Jaslee Carayol would only specify one example of “an action [College Board] took in response” to Operation Varsity Blues, and added, “For security reasons, we cannot provide specifics about our approach.”

“In all but the rarest circumstances, students will take the test only at their schools or during a weekend session,” she said, referring to accommodation requests (students who do not request extra time on the test can still test anywhere in the U.S., according to testing experts). “If a student must take a test elsewhere, we will carefully confirm their need to test at an alternate location and the security of that location.”

The Educational Testing Service, which is hired by the College Board to develop and administer its exams, did not respond to Teen Vogue’s multiple requests for comment.

Whether many test-taking rules are followed, however, depends entirely on the people who proctor the tests, and while criteria do exist spelling out who can do the job, the College Board and ACT’s proctor requirements do not appear to have changed since one proctor and two test administrators were indicted as part of Operation Varsity Blues, the proctor pleading guilty. They were accused of playing various roles in falsifying students’ scores. The accused test administrators have pleaded not guilty.

“It might be smart for the test-makers to incorporate a stronger vetting process and require training,” Sam Pritchard, the director of College Prep Programs at Kaplan Test Prep, told Teen Vogue. “More oversight seems warranted in that it may prevent more cheating and also restore faith by test-takers and their parents, not to mention colleges who want to know that applicants’ scores are their own.”

Critics say that without testing representatives actively willing or knowledgeable enough to enforce the rules at the ground level, systemic cheating isn’t going to stop anytime soon.

“They certainly haven’t gone to the root cause of what their problem is,” Freedman said. “All they’re trying to do is clean up little messes here and there.”

This was originally posted on Medium