In Which Your Obedient Serpent Defends the "Students"

[athelind]

There's been a bit of a kerfluffle about a recent study about students who fell for a hoax website about the Pacific Northwest Tree Octopus.

Frankly, the article linked above is a shoddy piece of science journalism. As eggshellhammer pointed out, it doesn't link to the original study. Even worse, in Your Obedient Serpent's eyes: it didn't specify the age level of the students. That's an important factor: a study about the critical thinking ability of kindergarten students has entirely different implications than the same study about a group of college undergraduates.

That in itself is an indication of a failure of critical thinking ability in would-be science journalists.

As it transpires, this study involved seventh-graders. The conclusion can thus be summarized as, "wow, you can con a 12-year-old into believing some crazy shit", which is hardly earth-shattering news. I'd say three-quarters of the contents of snopes.com is stuff that was repeated as gospel truth on the Bicentennial schoolyards of my twelfth year.

(I find the datum that students ignore search engines in favor of randomlytypinginaname.com to be much more startling, personally. Seriously, WTF?)

The other study mentioned in the University of Connecticut article suggests that this, in large measure, just reflects a need for improved emphasis on Internet search and access skills, and not some Terrible Crisis in Education. That's how the researchers seem to interpret it; the DANGER WILL ROBINSON! reactions were mostly imposed by the secondary sources. For my part, I was intrigued and, on some level, amused at the revelation that students who had difficulties with traditional literacy showed superior online reading facilities.

As for the details of the first study ... I'm going to be generous and completely ignore the implications of drawing broad conclusions from a sample group of twenty-five students in a single class. Let's assume, for the sake of argument, that this specific class is representative of the entire population of students in Connecticut. Let's take a look at two of the sited conclusions:

• All but one of the 25 rated the site as "very credible" ...

Let us, just for a moment, step out of the role of of the Know-It-All Grown-Up Who Knows This Site Is Patently Absurd Because There's No Such Thing. Let us remember that those reading this journal are likely to have at least five more years of formal education than the subjects of this study.

Yes, http://zapatopi.net/treeoctopus/ is "very credible".

"Credible" doesn't mean "true" or "accurate". It means "able to be believed", or "capable of persuading". The website has a professional presentation and a serious, convincing tone. The only obvious joke on the main page (aside from a deadpan link to sasquatch) is a reference to the organization "Greenpeas". The FAQ gets increasingly flippant and absurdist, but they avoid an overtly humorous tone for the main body.

Given that aquarium octopuses are well-known for getting out of their tanks and taking walks, and that there is at least one species of land-dwelling, arboreal hermit crabs, the idea of a "tree octopus" is just plausible enough to someone who knows just how weird and wacky life on Earth can get.

In science, "credibility" also means "reproducibility", and in this context, that extends to being able to find other corroborating sources.

This leads us to the second conclusion I want to examine:

• Most struggled when asked to produce proof - or even clues - that the web site was false ...

Hey, it's an exercise for the class! Let's check our own research and critical thinking abilities, shall we?

I'm curious to see what proofs (or even clues!) the folks reading this can come up with, above and beyond the flippant tone of the FAQ that I mentioned above. The Sasquatch link leads to an equally-deadpan page, of course.

Needless to say, "I just know there's no such thing" isn't a valid "proof"; in fact, it doesn't even rate as a "clue".

Answers will be graded!

---

Thanks ... and apologies ... to pseudomanitou for drawing my attention to this study and the reactions which followed. Please don't think I'm being an asshole for deconstructing this.

---

Update: eggshellhammer contacted the original author and scored a link to the original document. Yes, the sample group was larger than 25.

---

Comments

[pyat.livejournal.com]

Well-written, sir, and well deconstructed.

[jirris-midvale.livejournal.com]

Yes. Very yes.

The first place I heard about this was on some cranky old furry artist on FA's journal. It was him basically going 'HAW HAW THIS GENERATION IS SO STUPID' firmly cementing himself in the club of bitter old men who wear brushed denim pants up around their sternums.

I replied with this:
I think you missed the second half that talked about the continuing lack of schools being able to teach critical thinking skills. This isn't a problem with just young people. I've met many people throughout my life of all age groups who never even got the basic primer on that particular skillset. They never question their assumptions or look for fallacies in their own thought patterns and mental constructs. This lack of critical thinking leads to not questioning the sources of information and stereotyping entire groups of people based on limited experience with them. You're as likely to meet an old dude who watches nothing but golf and a news network who is convinced of utter bullshit as you are a 13 year old who assumes we've always had facebook.

Yeah, that's not a passive-aggressive as fuck reply about him using something he found on the internet that fit his assumptions that the internet makes kids dumb because they don't fact check or critically read what they find on the internet, no.

[pseudomanitou.livejournal.com]

Deconstructing any article I post is fair game -- so long as the topic doesn't become a game of blaming the messenger or uncritically dismissing the information via way of the messenger.

Again, for me, I think even kids at the age of 12 should be learning how to call bullshit. I was getting just that kind of an education from a public school system in Indiana, and I thank the gods I did get it.

In an age of information -- where information shapes so much democracy and value -- teaching critical thinking is more vital than math or English.

Proving the website is false would be tricky

[ab3nd.livejournal.com]

The assertions that it makes seem to mostly be that the tree octopus exists, and that it is in danger of going extinct.

To prove the second part false, I would have to prove that the tree octopus is not in danger of becoming extinct. That is, I would have to prove the nonexistence of threats to the tree octopus's continued existence.

To prove the first part false, I would have to prove that the tree octopus does not exist.

The simple way to prove nonexistence, assuming I can correctly identify things, is to look at all things, and if none of them can be identified as the thing that I am looking for, then I have proven that the thing does not exist. This will take an impractically long time, even if I just look at all the things in the Pacific Northwest.

Brute force being out, I must resort to something more clever. For example, I could assume the existence of the tree octopus and then derive a contradiction. Unfortunately, I don't have a handy proof that reality is logically consistent, nor do I have an example of something that is observed to be true, yet could not be true in a reality that includes tree octopi.

If someone could prove that, for example, the existence of tree octopi would result in all cheesecakes failing to set, and I had observed a set cheesecake, then I would know there were no tree octopi (or that something was wrong in the calibration of my cheesecake durometer).

The tautological case of "In a reality with tree octopi, there would be tree octopi" falls apart because I can't check all the things to see if at least one of them is a tree octopus, as above. Cases like "In a reality where there are tree octopi, there are no tree salmon" have a similar problem, because I have to check everything to make sure it isn't a tree salmon. In fact, even the cheesecake example fails because I cannot be certain that no cheesecake anywhere ever will never set without checking all of them.

In other words, disproof by looking at everything fails because I'm lazy, and disproof by finding contradictory properties of the observed system fails because I'm ignorant (of the properties of the system) AND lazy (about checking said properties).

But then, this only applies to those of us limited by inconveniences like mortality and the laws of physics. If I didn't have these problems, I could simply Go and Look, and then Reveal to you all The Way and The Light, A Truth for the Ages.

Of course, whether you believed me is up to you.

Further attempts at proof or disproof can be found here: http://en.wikipedia.org/wiki/Ontological_argument
Just remember to replace all instances of "God" with "tree octopus".

[castleclear.livejournal.com]

Dear Obedient Serpent, Thank you for teaching about Science as well as human credulity as related to reasoning and information. Your summary conclusion "wow, you can con a 12-year-old into believing some crazy shit" truly made me laugh outloud, as did the pleasant reintroduction to my vocabulary of the word kerfluffle. Best Wishes, sir.