EDGE 2015 is starting in

Find out more EDGE 2015
Menu
Statistics experts label ISP filtering trials unscientific

Statistics experts label ISP filtering trials unscientific

Trials for mandatory filtering would never be accepted in an academic statistics journal

The Federal Government’s ISP filter trials lack proper methodology and are not representative, according to experts in statistics and testing from two of Australia’s leading universities.

The criticisms come after two of the nine ISPs participating revealed only 15 of their customers, which in one case was 1 per cent of the total, chose to have their Internet filtered.

The vast majority of ISPs also used an opt-in system that requires users wanting to be filtered to request it.

“I would not have confidence in any of the results they find because of the way the sample has been constructed,” expert in statistics and senior lecturer at the Queensland University of Technology, Dr Daniel Johnson, said.

The professor contends a major reason why an opt-in system was selected was the difficulty in getting ISPs to force filtering on their users and said the result was a system without a strong basis in science.

“It’s probable that the results will be favourable for the survey because of the way they’ve collected data,” he said.

A good mandatory filter trial, Johnson said, would need a larger random sample set that couldn’t opt-out, rather than the opt-in system currently in place for most participating ISPs.

“Make people trial it for a week and then let them pull out if they need to because then at least you start with a more representative sample,” he said.

While the study was not easy to run it didn’t make the data gathered any more valid, Johnson said.

“I would expect that any professional in this area would state the shortcomings of their data in their findings,” he said.

Professor of statistics at Monash University and editor-in-chief of the International Journal of Forecasting, Professor Rob Hyndman, described an opt-in system as “about the worst way you can do it”.

“It would not be published in a statistical journal. It would never be of interest to statisticians because of the way it’s been conducted,” he said.

Hyndman went on to say while compulsory trials on randomised sample groups would be statistically ideal, the ethics of such an act would be questionable.

The Minister for the Department of Broadband, Communications and the Digital Economy (DBCDE), Senator Stephen Conroy, has previously stated the current trial’s success or failure will help form much of the government’s stance on mandatory nation-wide ISP filtering.

“We'll be guided by that trial. We've always said, consistently, we'll be guided by the trial,” the Minister said on the ABC’s Q&A program.

The company running the evaluation of the ISP filtering trial, Enex, was contacted for comment, but ARN was told due to a non-disclosure agreement between the company and the Federal Government, all queries on the ISP filtering trial had to be directed to the latter.

ARN contacted Senator Conroy’s office but did not receive a response by time of publication.

The Australian Bureau of Statistics was also contacted but the organisation declined to comment on the validity of the trial’s methodology.

Other leading tech experts have criticised the small number of participants and stated that while larger ISPs may be able to handle a national ISP filter, smaller providers may not.

The majority of the participating ISPs have revealed they are using Marshal8e6 filtering solutions as part of the trial.

EDGE 2015:: For all the latest on EDGE 2015 including the keynote speakers visit the EDGE mini-site now

Follow Us

Join the ARN newsletter!

Error: Please check your email address.

Tags ISP filter trials

Upcoming

Slideshows

IN PICTURES: Intalock celebrates five years (+14 images)

IN PICTURES: Intalock celebrates five years (+14 images)

Data solutions specialist Intalock recently held its fifth anniversary party at X in Brisbane, which saw local resellers enjoy cocktails with a coy Casino Royale theme. Photos by Aaron Jamieson.

IN PICTURES: Intalock celebrates five years (+14 images)
IN PICTURES: Citrix Mobility Conference 2015 (+27 photos)

IN PICTURES: Citrix Mobility Conference 2015 (+27 photos)

Attendees pack the Hilton Hotel in Sydney to talk all things mobility and a bit of virtualisation for good measure. Partners and end users were able to get a glimpse of the new innovations from Citrix, demonstrations and meet other vendors in the Solutions Showcase.

IN PICTURES: Citrix Mobility Conference 2015 (+27 photos)

iasset.com is a channel management ecosystem that automates all major aspects of the entire sales, marketing and service process, including data tracking, integrated learning, knowledge management and product lifecycle management.

Show Comments