Challenging for experimental economists when labs close

Professor Erte Xiao

Right at the start of semester one the discussion started among the Monash Experimental Economists. The COVID-19 pandemic was closing down country after country and Professor Erte Xiao, Director of the Monash Laboratory for Experimental Economics (MonLEE) was worried how they would keep running the real-life experiments in the lab with students, so necessary to their research. Not long after, the lab was closed and experiments moved online.

“We had no choice but to switch to online platforms. Lata Gangadharan and I had a paper where the editor wanted us to run more experiments. Doing them online was the only way,” explains Professor Xiao.

As a starting point of the transition, MonLEE lab manager Nina Xue helped to create a protocol for how the online experiments would run: how to set them up, the change to a new appropriate software, how to recruit, pay the participants and keep track of the students online during the experiments.

The research pair consulted other experimental economists around the world for their experience in conducting the experiments online.

“The quality of data you collect from online experiments might not be as good as when you conduct experiments in a lab. You can’t fully control what students are doing online, which you could do in the lab," says Professor Xiao.

"It might mean that you have to recruit more participant to increase the sample size. On the other hand it might be cheaper to run experiments online so you can afford to do more of them.”

For longer and more complicated studies, it is difficult to maintain the students’ full attention and acquire reliable results. That is possibly one of the main reasons why not all experimental economists have resorted to online experiments.

“I think maybe people are completing unfinished projects this year and thinking more about ideas. Only about a third of the experimental economists at Monash were interested in online experiments when we sent out a survey,” she explains.

Like many of her colleagues, Professor Xiao is keen to get back in the lab with students once the pandemic is more under control. But she still thinks there will be a use for online studies in the future.

“They can work well on short, simple experiments. There are some procedures we can adopt to ensure anonymity and retain participants’ attention during the experiment,” she says.

In the future she would also like to collaborate more with other universities, even overseas, for larger online studies.

“If we can work this out and get interest from other universities we can share the resources. Collaborations become easier when it is all online. I guess that is something positive we have learned from this situation, it opens other doors and gives us more options.”

So what happened with the paper? “Well, we reported back to the editor that we’d run some more experiments and he was happy with that. Now we have to finish the revision – and we’ll see,” says Professor Xiao.