NIU professor offers solution to figure skating's ills
by Hank Brockett
3/1/02
In figure skating, that’s nearly an eternity. After all, in 1994, U.S. gold medalist Sarah Hughes was just turning 9 years old.
But Marilyn Looney’s research, performed after the 1994 Winter Olympics, directly dealt with just how figure skating was judged. And in a bit of forecasting that Miss Cleo would envy, the Nancy Kerrigan/Oksana Baiul judging controversy that year wasn’t the last time figure skating controversy would plague the sport.
Something changed, though. Looney, an NIU professor of kinesiology and physical education, watched as her theory on something called the Rasch model drew more and more attention during the recently concluded Winter Olympics. A pair of Canadians and Russians helped revive that dormant research.
“I’m hoping that the International Skating Union takes this opportunity to improve their system by looking at a number of different options,” Looney said. “Figure skating is a very popular sport, and I guess I would judge that by the number of events on television. They’re probably in danger of losing some of that fan support.”
The Canadian pair of Jamie Sale and David Pelletier originally finished second to Elena Berezhnaya and Anton Sikharulidze of Russia. However, after a French judge admitted to wrongdoing, both pairs were awarded gold medals in pairs figure skating. And even in Sarah Hughes’ victory, some Russian supporters claimed that judges were biased against Irina Slutskaya, the Russian silver medalist. Their protest went without further action.
Looney theorizes that the Rasch model might have detected the French judge’s errant scoring and possibly prevented much of the hoopla that followed.
The Rasch model supports the idea of objective measurement. In a real world example, the Rasch model is used when comparing the scoring of essay exams for standardized tests. The model helps factor out the severity of subjectivity and places the fate of a score out of the random chance of a hard or easy judge.
“The model assumes that the judges are going to be different,” she said. “Even if you take a group of people and train them and train them to try and judge exactly the same you’ll never come to perfect agreement. The model assumes that the judges are going to have some disagreement among each other. But the model also expects the judges to be consistent within themselves ... The model allows you to see when the judge breaks out of the pattern.”
Looney, who teaches classes on measurement and evaluation, first became interested in the Rasch model in 1995 and took her sabbatical to investigate the model further.
“I was interested in the model and then I was looking for some data that had some controversy,” she said.
Obviously, figure skating has complied.
Proposals offer positives and negatives
As the ISU examines various means of legitimizing a fragile judging process, Looney said the Rasch model could be applied almost in real time. As judges enter their scores into a computer, a software program could perform bias analysis.
The process still would need another step, though.
“You’d still have to have some policies in place, some decision-making rules so if you identify some bloc-voting some going on or some real errant scores, what are you going to do?” said Looney. “If you find this, what are you going to do next?”
In the wave of new ideas and proposed changes, Looney said one of the ISU’s recommendation -- which includes increasing the number of judges by five to 14 and randomly picking seven scores -- has its faults.
“Using a random selection process does not guarantee that a representative sample of judges from different cultural backgrounds will be selected,” she said.
That idea hinges on the idea that all judges look at the same thing. Judging always has shown particular subjective tastes toward either the technical or the artistic, depending on the judge.
“What you prefer in terms of music is probably different than what your parents prefer,” she said. “It’s just a matter of taste, not bias. So when judges give their scores, it’s going to be this cultural preference to what’s pleasing to them.”
If she had her way, Looney would propose changing some other judging aspects as well. For instance, she would pay the judges (who currently serve without paychecks) and provide nearly equal weight to the long and short programs that make up the figure skating events. Currently, the short program only makes up one-third of the final score.
Whether she influences figure skating policy or not, Looney’s research will continue on.
“At some point, I’d like to apply the model to the data from this (pairs) competition,” she said.