Enex Testlab: ISP filtering trial is up to scratch

Flawed sampling

I wonder of Exec TestLab would care to explain how they scientifically selected their random sample of filtering victims?

How could any supposedly reputable lab engage in a process with a fundamentally flawed population sampling method? Could they have subjugated their scientific principles just because the money was too good?

peer review?

"“Enex TestLab, as always, remains independent"

if they were truly independant they wouldn't have signed a non-dislcosure agreement.

"we apply peer-review and confidence factors to our analysis.” "

How can you apply peer review (ie: review by independant scholars) when you aren't allowed to release the results. The independant statisticians who have commented ARE the peers that shoud be reviewing the process, you can't have in-house peer review.

"To say, however, that the methodology that we have used in the ISP filter trial is lacking, or flawed is incorrect."

The independant peers who review work have already said it is flawed. oh dear, fail on so many levels it laughable.

Enex consider a test that uses opt-in, low user, low data volume testing, to be a sound methodology to test a system that will be mandatory, apply to every internet user in the country and high data volume sites.... yeah, right!!!
First time I've ever heard that testing 15 people at an ISP is the same as applying the test to 7 million internet users.
The TV ratings use higher numbers for testing and we all know how flawed those are.

Enex, how on Earth can you defend the trial? The testing framework released by the government shows how poorly the trial's being conducted. The methodology is incredibly poor and there's no proper scientific testing.

I hope Enex realise they've damaged their credibility by conducting such an obviously flawed trial. If censorship is mandated, then Enex should hang their heads in shame for helping the government to implement it.

It gets worse.

I've said before that Enex' position is thus:

"Since our testing methodologies are hidden behind Non-Disclosure Agreements and Commerical In Confidence, we're immune to peer review; ergo these academics can't possibly be talking about us; ergo our excrement still smells like roses."

But it gets even better! From the article, I quote:

"we apply peer-review and confidence factors to our analysis"

Enex are applying peer-review to themselves!? Did the definition of peer-review change last week??

I previously stated that Enex should choose their bedfellows carefully, because you never know what they'll catch from them. I guess we now know.

Congratulations, Enex, you are now as scientifically credible as a Creationist.

And they can prove pigs fly as well!

There was no published specification, testing methodology or design criteria for the trial. The testing ISPs were not required to conduct tests under identical conditions. How can Enex claim any academic credibility for any testing they may have done. In the time scales involved I'd even question what testing could have been done.

Four of the testing ISPs used the product of a single vendor. Any results could be interpreted as a simple test of a single product. I'd say Watchdog (possibly Marshal8e6) just got a free sales pitch written for them by a previously reputable academic organisation.

The fact that Enex was prepared to conduct their contracted works under a non disclosure agreement and allow a different organisation to write the report smacks of providing "correct" answers in exchange for payment. In fact, what did Enex actually do for the $887,000 noted on the DBCDE site?

Many of the critics of this proposal work in the same fields as Enex. Enex may live to regret "selling their soul for pieces of silver".

I would imagine that most of Enex's prospective customers are not as scientifically inept as Stephen Conroy, and are watching these events very cautiously. Their scientific credibility is falling apart as we speak.

NDA or not, "just trust us" doesn't justify dodgy science.

And even if they are a success, what does that tell us? That opt-in filters work. Fantastic, implement them and everyone but the 21st century book-burners will be happy.

What a waste of Time and Money

Stunning, this is the sillist waste of money I have ever seen, any person with reasonable knowledge can get round these silly filtering systems, I have proved how to at many companys over many years, because of the what has to allowed there is no to stop people from bypassing them, and who are they really trying to protect?

This is nothing more than another "fuelwatch" or "grocery watch" it will run for while waste a major amount of money get 100's of 1000"s of complaints then get shut down due to the inability to run it without major flaws appearing ALL the time.

What a waste, I acutally voted for these fools last time around, what a mistake that was.

This hole thing reeks, what would be the purpose of making a filtering test private in a democratic country.... other then bsing the public into proving the impossible is possible. Who could possibly benefit from that information?
Besides if its going to be implemented anyway why cant we all see the results of how badly we will be shafted?

Why Enex Cannot be Trusted

Enex says "you are not allowed to question our methodology" yet we, those affected most by it, are not allowed to know what that methodology is.

Welcome to Senator Conroy's world of censorship. This is a taste of what is to come.

To Enex I simply ask: Why should we trust you when you participate and perpetuate censorship?

The answer is simple: We cannot Enex nor anybody else who proposes to censor material under the shroud of secrecy.

Re: Enex tests

Dear Free Thinker, I'm intrigued by your gloating. What makes you think Exetel performed any sort of valid test? It seemed to me to be ill conceived, poorly executed, and swiftly forgotten.

Why do you think the current government "trial" has any sort of validity? For a start, there have been no published metrics for success, so anything can be made up after the fact with the intention of claiming any and all possible results as a success. This is consistent with a whitewash, not with valid scientific testing.

And as for conspiracies, when it is plainly obvious that the internet censorship proposal won't prevent child abuse, won't prevent bullying, won't improve anybody's morals, and won't do a thing to stop spam and virus attacks, you have to wonder what it can actually be used for. I think it's more reasonable to believe that it is intended as a mechanism to suppress free speech (on such topics as euthanasia and abortion) rather than any of the stated aims, since the stated aims cannot be achieved. Call this a conspiracy, if you like, but it's plain the minister is not being honest with us, or perhaps even with himself.

In short, your gleeful attitude seems misplaced. It is odd that you should be so cheerful when your tax dollars are being wasted on such a preposterous folly.

Laughable..

Free thinker? Funny, I thought the hall mark of free thinking is questioning the party line rather than lockstepping with it...

NDA? No transparency? Come on, if the tests were glowingly positive and performed under strict blind or double blind methodology, Conjob and Enex would be publishing the results for all to see...

The fact that the report has to be manufactured by someone other than the tester just proves that Enex's testing is not robust, not peer reviewable (how can you review what you can't see) and is, in short, not valid.

If this filter does come online, you 'free thinkers' will start wailing and gnashing your teeth and asking "WHY OH WHY DIDN'T SOMEONE TEST THIS PROPERLY AND KILL IT?!?!?". By then it'll be too late of course. Small consolation for those of us who don't believe the party line but if I'm going to hell in a hand basket, I'm going to make the most of it by laughing at those who expected to go to some sort of nanny state heaven...

A word on the Australian Internet censorship scandal

I've had a quick scan over Senator Stephen Conroy's infamous, long-awaited report on the efficacy of current Internet filtering technology and find it to be nothing short of scandalous. Without getting into the nitty gritty details (for example, how a filtering solution can achieve the impossible by improving rather than degrading the performance of encrypted, random transfers), it reads like it's a whitepaper for one of the various purveyors of censorship technology.

The cynic in me insisted I take a quick look at who these Enex Pty Ltd jabbers are anyway - who knows, they could be an industry lobby group for all we know. Sure enough, a quick look at their corporate client list reveals (based on some quick Google searching) over a dozen companies who make a living selling commercial censorship technology:
Anthology Solutions
Content Keeper Technologies
Content Watch
F-Secure Corporation
Internet Sheriff Technology
Manaccom
MessageLabs
NetBox Blue
Netgear
Netsweeper
PC Tools Software
Raritan (?)
Secure Computing Corporation (McAfee)
Symantec
Trend Micro
To put things in perspective, this represents around a quarter of their published client list, and that's not including half a dozen or so service providers that could arguably be thrown in with this bunch. Who in their right mind would risk upsetting one in four of their paying customers by writing a report critical of their products? And does anyone really believe that these vendors resisted the urge to apply pressure? Or that there were not personal relationships involved? I don't, not for a second. In my opinion this report was rigged from the outset to succeed, and in doing so deprive Australians of essential civil liberties.

The report itself is fatally flawed; the error margins are significant (e.g. "a conservative
+/-10 percent"), critical controls were missing (e.g. "as much as 40 percent of an internet service performance could be lost [due to factors outside of our control]"), outrageous assumptions were used (e.g. "performance impact is considered minimal if between 10 and 20 percent") and perhaps most importantly of all, it's creator has an obvious conflict of interest. I don't consider it to be worth the paper it's [not] printed on.

Another deeply concerning development is government grants that would encourage ISPs to go beyond the mandatory filters, despite all censorship systems tested reporting 2.5-3.5% false positive rates (that is, where innocuous/legitimate content is filtered). To put that in perspective, the best part of a billion legitimate pages would be improperly filtered (according to Wikipedia stats), or around 1 page in 30.

Speaking of Wikipedia, many of the systems are hybrid which means that hosts known to be clean would be ignored by IP (which is much more efficient). If, however, even a single page were problematic then the entire site (and all others sharing its' IPs) would be forced through a filtering proxy. This would affect some of the most popular sites on the Internet (such as Wikipedia and YouTube), not to mention other increasingly useful services like WikiLeaks (no doubt silencing such services is seen as a fringe benefit to our self-appointed censors). Need I remind you that similar filters in Britain caused severe problems for Wikipedia over a single CD cover only last year.

Another consideration that has not been covered anywhere near enough is the performance impact on cloud computing services. Web interfaces like Facebook, Twitter and Gmail are extremely sensitive to latency introduced by proxies and raw computing services like Amazon's S3 are sensitive to bandwidth limitations. Then you have the problem of platforms like Google App Engine, Google Sites & Microsoft Web Office which are both difficult to identify (they have many IPs which are not disclosed and difficult if not impossible to enumerate) and which host content for a massive number of customers. If even one person shares a document deemed obnoxious to their sensibilities then the performance will be reduced to unacceptable levels for everyone until it is removed (and then some).

It is my contention that censorship is completely incompatible with cloud computing, and that this alone is reason enough to scuttle it. In the mean time Electronic Frontiers Australia (EFA) has just landed themselves a new life member and I encourage anyone who cares about their future and that of their children to join as well (my friends in the USA may want to take a look at the EFF and Europeans the FFII).