Menu
Statistics experts label ISP filtering trials unscientific

Statistics experts label ISP filtering trials unscientific

Trials for mandatory filtering would never be accepted in an academic statistics journal

The Federal Government’s ISP filter trials lack proper methodology and are not representative, according to experts in statistics and testing from two of Australia’s leading universities.

The criticisms come after two of the nine ISPs participating revealed only 15 of their customers, which in one case was 1 per cent of the total, chose to have their Internet filtered.

The vast majority of ISPs also used an opt-in system that requires users wanting to be filtered to request it.

“I would not have confidence in any of the results they find because of the way the sample has been constructed,” expert in statistics and senior lecturer at the Queensland University of Technology, Dr Daniel Johnson, said.

The professor contends a major reason why an opt-in system was selected was the difficulty in getting ISPs to force filtering on their users and said the result was a system without a strong basis in science.

“It’s probable that the results will be favourable for the survey because of the way they’ve collected data,” he said.

A good mandatory filter trial, Johnson said, would need a larger random sample set that couldn’t opt-out, rather than the opt-in system currently in place for most participating ISPs.

“Make people trial it for a week and then let them pull out if they need to because then at least you start with a more representative sample,” he said.

While the study was not easy to run it didn’t make the data gathered any more valid, Johnson said.

“I would expect that any professional in this area would state the shortcomings of their data in their findings,” he said.

Professor of statistics at Monash University and editor-in-chief of the International Journal of Forecasting, Professor Rob Hyndman, described an opt-in system as “about the worst way you can do it”.

“It would not be published in a statistical journal. It would never be of interest to statisticians because of the way it’s been conducted,” he said.

Hyndman went on to say while compulsory trials on randomised sample groups would be statistically ideal, the ethics of such an act would be questionable.

The Minister for the Department of Broadband, Communications and the Digital Economy (DBCDE), Senator Stephen Conroy, has previously stated the current trial’s success or failure will help form much of the government’s stance on mandatory nation-wide ISP filtering.

“We'll be guided by that trial. We've always said, consistently, we'll be guided by the trial,” the Minister said on the ABC’s Q&A program.

The company running the evaluation of the ISP filtering trial, Enex, was contacted for comment, but ARN was told due to a non-disclosure agreement between the company and the Federal Government, all queries on the ISP filtering trial had to be directed to the latter.

ARN contacted Senator Conroy’s office but did not receive a response by time of publication.

The Australian Bureau of Statistics was also contacted but the organisation declined to comment on the validity of the trial’s methodology.

Other leading tech experts have criticised the small number of participants and stated that while larger ISPs may be able to handle a national ISP filter, smaller providers may not.

The majority of the participating ISPs have revealed they are using Marshal8e6 filtering solutions as part of the trial.

Follow Us

Join the ARN newsletter!

Error: Please check your email address.

Tags ISP filter trials

Upcoming

Slideshows

IN PICTURES: VMworld 2015 Asia-Pacific and Japan party (+ 32 photos)

IN PICTURES: VMworld 2015 Asia-Pacific and Japan party (+ 32 photos)

VMware recently held an Asia-Pacific and Japan party for its partners in San Francisco following two days of keynotes and sessions. Whilst mingling and enjoying drinks and finger food, the partners were joined by VMware management who also took the opportunity to let their hair down to have some fun.

IN PICTURES: VMworld 2015 Asia-Pacific and Japan party (+ 32 photos)
IN PICTURES: VMworld 2015 sponsor and partner showcase (+41 photos)

IN PICTURES: VMworld 2015 sponsor and partner showcase (+41 photos)

VMware's sponsors and partners used the opportunity at VMworld 2015 to showcase some of their technologies. At an exhibition hall, these vendors educated those that popped by their stands on these solutions and addressed some of the issues surrounding mobility, datacentres, and the Cloud. SOme of the big names there included f5, Palo Alto Networks, HP, Intel, Samsung, and Symantec.

IN PICTURES: VMworld 2015 sponsor and partner showcase (+41 photos)

iasset.com is a channel management ecosystem that automates all major aspects of the entire sales, marketing and service process, including data tracking, integrated learning, knowledge management and product lifecycle management.

Show Comments