In Which Your Obedient Serpent Defends the "Students"

[athelind]

There's been a bit of a kerfluffle about a recent study about students who fell for a hoax website about the Pacific Northwest Tree Octopus.

Frankly, the article linked above is a shoddy piece of science journalism. As eggshellhammer pointed out, it doesn't link to the original study. Even worse, in Your Obedient Serpent's eyes: it didn't specify the age level of the students. That's an important factor: a study about the critical thinking ability of kindergarten students has entirely different implications than the same study about a group of college undergraduates.

That in itself is an indication of a failure of critical thinking ability in would-be science journalists.

As it transpires, this study involved seventh-graders. The conclusion can thus be summarized as, "wow, you can con a 12-year-old into believing some crazy shit", which is hardly earth-shattering news. I'd say three-quarters of the contents of snopes.com is stuff that was repeated as gospel truth on the Bicentennial schoolyards of my twelfth year.

(I find the datum that students ignore search engines in favor of randomlytypinginaname.com to be much more startling, personally. Seriously, WTF?)

The other study mentioned in the University of Connecticut article suggests that this, in large measure, just reflects a need for improved emphasis on Internet search and access skills, and not some Terrible Crisis in Education. That's how the researchers seem to interpret it; the DANGER WILL ROBINSON! reactions were mostly imposed by the secondary sources. For my part, I was intrigued and, on some level, amused at the revelation that students who had difficulties with traditional literacy showed superior online reading facilities.

As for the details of the first study ... I'm going to be generous and completely ignore the implications of drawing broad conclusions from a sample group of twenty-five students in a single class. Let's assume, for the sake of argument, that this specific class is representative of the entire population of students in Connecticut. Let's take a look at two of the sited conclusions:

• All but one of the 25 rated the site as "very credible" ...

Let us, just for a moment, step out of the role of of the Know-It-All Grown-Up Who Knows This Site Is Patently Absurd Because There's No Such Thing. Let us remember that those reading this journal are likely to have at least five more years of formal education than the subjects of this study.

Yes, http://zapatopi.net/treeoctopus/ is "very credible".

"Credible" doesn't mean "true" or "accurate". It means "able to be believed", or "capable of persuading". The website has a professional presentation and a serious, convincing tone. The only obvious joke on the main page (aside from a deadpan link to sasquatch) is a reference to the organization "Greenpeas". The FAQ gets increasingly flippant and absurdist, but they avoid an overtly humorous tone for the main body.

Given that aquarium octopuses are well-known for getting out of their tanks and taking walks, and that there is at least one species of land-dwelling, arboreal hermit crabs, the idea of a "tree octopus" is just plausible enough to someone who knows just how weird and wacky life on Earth can get.

In science, "credibility" also means "reproducibility", and in this context, that extends to being able to find other corroborating sources.

This leads us to the second conclusion I want to examine:

• Most struggled when asked to produce proof - or even clues - that the web site was false ...

Hey, it's an exercise for the class! Let's check our own research and critical thinking abilities, shall we?

I'm curious to see what proofs (or even clues!) the folks reading this can come up with, above and beyond the flippant tone of the FAQ that I mentioned above. The Sasquatch link leads to an equally-deadpan page, of course.

Needless to say, "I just know there's no such thing" isn't a valid "proof"; in fact, it doesn't even rate as a "clue".

Answers will be graded!

---

Thanks ... and apologies ... to pseudomanitou for drawing my attention to this study and the reactions which followed. Please don't think I'm being an asshole for deconstructing this.

---

Update: eggshellhammer contacted the original author and scored a link to the original document. Yes, the sample group was larger than 25.

---

Comments

[pyat.livejournal.com]

Well-written, sir, and well deconstructed.

[jirris-midvale.livejournal.com]

Yes. Very yes.

The first place I heard about this was on some cranky old furry artist on FA's journal. It was him basically going 'HAW HAW THIS GENERATION IS SO STUPID' firmly cementing himself in the club of bitter old men who wear brushed denim pants up around their sternums.

I replied with this:
I think you missed the second half that talked about the continuing lack of schools being able to teach critical thinking skills. This isn't a problem with just young people. I've met many people throughout my life of all age groups who never even got the basic primer on that particular skillset. They never question their assumptions or look for fallacies in their own thought patterns and mental constructs. This lack of critical thinking leads to not questioning the sources of information and stereotyping entire groups of people based on limited experience with them. You're as likely to meet an old dude who watches nothing but golf and a news network who is convinced of utter bullshit as you are a 13 year old who assumes we've always had facebook.

Yeah, that's not a passive-aggressive as fuck reply about him using something he found on the internet that fit his assumptions that the internet makes kids dumb because they don't fact check or critically read what they find on the internet, no.

[pseudomanitou.livejournal.com]

Deconstructing any article I post is fair game -- so long as the topic doesn't become a game of blaming the messenger or uncritically dismissing the information via way of the messenger.

Again, for me, I think even kids at the age of 12 should be learning how to call bullshit. I was getting just that kind of an education from a public school system in Indiana, and I thank the gods I did get it.

In an age of information -- where information shapes so much democracy and value -- teaching critical thinking is more vital than math or English.

[zeeth_kyrah]

Which is why I agree that we should also be teaching statistics and system design as soon as possible. Critical thinking is meaningless if you can't take apart the stuff you're given.

[castleclear.livejournal.com]

Dear siege, I am grateful that the statistics courses I attended included about as much information on what statistics DON'T prove as well as what inferences might be drawn through statistical analyses. I would be grateful if you would be so kind as to recommend by reply post (or just LJ email me) recommended reading on system design, for mine was a very incomplete education. Best wishes, siege, for a beautiful day and an even better tomorrow.

P.S. I think that Civics needs also to be taught in the public schools again along with maturity-appropriate history. (e.g. grade schoolers possibly don't need to know about small pox infected blankets being given to native peoples and I recently met a 5 grader of mixed ethnicity who looked crestfallen when describing U.S. slavery when otherwise delightedly telling about what she was learning in school. She was quite animated talking about fractions.)

[zeeth_kyrah]

I find that the best way to learn system design is to design systems -- and I have a pretty broad definition of "system". Most people see this area as a business-oriented thing; but games are systems, and we speak much of the "political system", "economic system", "educational system" and so on. There is more than one design for these things, and I would suggest that to complete your understanding of system design, compare the systems in your daily life, describe their patterns, and see what makes them tick.

Not everyone can build a better system, but knowing how these systems are built, how they're supposed to work, and how they do work, gives a much better idea of what's currently right and wrong with the world at large. It also allows for a better grasp on how to make things better.

[castleclear.livejournal.com]

Dear Siege, Thank you. I've built various models as thought models in RP gaming, as well as learned to develop various work-arounds with respect to adapting from "relatively healthy, productive member os society" to "disabiity retirement with various chronic health ailments." Probably my best understanding of systems theory comes from the scientist character portrayed by Liv Ulllman in the film "MIND WALK."

[castleclear.livejournal.com]

I would say that the ability to reason is as vital as mathmematics and other languages, i.e., communication by symbolic representation (unless one is or intends to be a singularity/singular entity.) However, this is a minor quibble and my mind is open to the possibilty that reasoning is even more important than the ability to communicate, though that is difficult for me to comprehend since I am a member of a social animal species. Otherwise, I agree with you pseudomanitou and am grateful for the education you received.

Proving the website is false would be tricky

[ab3nd.livejournal.com]

The assertions that it makes seem to mostly be that the tree octopus exists, and that it is in danger of going extinct.

To prove the second part false, I would have to prove that the tree octopus is not in danger of becoming extinct. That is, I would have to prove the nonexistence of threats to the tree octopus's continued existence.

To prove the first part false, I would have to prove that the tree octopus does not exist.

The simple way to prove nonexistence, assuming I can correctly identify things, is to look at all things, and if none of them can be identified as the thing that I am looking for, then I have proven that the thing does not exist. This will take an impractically long time, even if I just look at all the things in the Pacific Northwest.

Brute force being out, I must resort to something more clever. For example, I could assume the existence of the tree octopus and then derive a contradiction. Unfortunately, I don't have a handy proof that reality is logically consistent, nor do I have an example of something that is observed to be true, yet could not be true in a reality that includes tree octopi.

If someone could prove that, for example, the existence of tree octopi would result in all cheesecakes failing to set, and I had observed a set cheesecake, then I would know there were no tree octopi (or that something was wrong in the calibration of my cheesecake durometer).

The tautological case of "In a reality with tree octopi, there would be tree octopi" falls apart because I can't check all the things to see if at least one of them is a tree octopus, as above. Cases like "In a reality where there are tree octopi, there are no tree salmon" have a similar problem, because I have to check everything to make sure it isn't a tree salmon. In fact, even the cheesecake example fails because I cannot be certain that no cheesecake anywhere ever will never set without checking all of them.

In other words, disproof by looking at everything fails because I'm lazy, and disproof by finding contradictory properties of the observed system fails because I'm ignorant (of the properties of the system) AND lazy (about checking said properties).

But then, this only applies to those of us limited by inconveniences like mortality and the laws of physics. If I didn't have these problems, I could simply Go and Look, and then Reveal to you all The Way and The Light, A Truth for the Ages.

Of course, whether you believed me is up to you.

Further attempts at proof or disproof can be found here: http://en.wikipedia.org/wiki/Ontological_argument
Just remember to replace all instances of "God" with "tree octopus".

Re: Proving the website is false would be tricky

[zeeth_kyrah]

The Tree Octopus is also claimed to live in a certain region and behave in a certain manner. Thus you could more easily search only that region (provided it is accessible to you) and discover whether any such creature exists there. Also note the photograph, which displays a conifer.. do conifers exist in that region? Thus a corollary test may be performed.

One way of searching for "all the things" is to use available references, especially those which are exhaustive (meaning they say they list All The Things, as Encyclopedia Britannica and Wikipedia do). If there is no tree octopus in those references, which may be more or less exhaustive, then one may safely assume that the tree octopus is either a newly-found species or is not an Earthly creature (one cannot say that a tree octopus does not exist elsewhere, but then, one cannot also yet say that life does or does not exist elsewhere using current scientific knowledge).

Adding to one's search for references by looking for scientific papers and websites outside of the single given site will also help to confirm or deny, as will looking over the other pages on the cited site itself.

Re: Proving the website is false would be tricky

[ab3nd.livejournal.com]

I am a bit of a nerd, so while I generally use "proof" the way most people use it (that is to mean "evidence sufficient to convince me"), in cases involving science or math, I tend to stick to the more formal use (that is, "a formal series of statements demonstrating a logical progression from accepted axioms to a conclusion").

Axiom 1: If Wikipedia lists all the things, then it lists me.
Axiom 1.1: I am a member of the set of all the things.

Axiom 2: If something is listed in Wikipedia, Wikipedia's search function will reveal it.
Axiom 2.1: Wikipedia's search function is without flaws.

Wikipedia's search function does not reveal a listing for me, therefore I am not listed in Wikipedia. Since I am not listed in Wikipedia, either I am not a member of the set of all things (contradicting axiom 2.1), Wikipedia's search function does not cover all of Wikipedia's content (contradicting axiom 2 and 2.1), or Wikipedia does not list all things (contradicting axiom 1).

But yeah, I'm generally content to look it up in a few places, maybe go check in person if it's not the other side of the country, and call it a day.

Re: Proving the website is false would be tricky

[athelind.livejournal.com]

Let's back away from the certainty implied by the verb "prove", and amend that to "provide evidence that suggests the site is false."

One thing that jumped out at me was the passage that listed reasons for the low populations of the Pacific Northwest Tree Octopus—and included among those reasons ... booming populations of its natural predators, including the bald eagle.

That's an assertion about a species that is well-known to be itself endangered, not a booming population. If I were taken in by the rest of the site, missed the "Greenpeas" reference, and wasn't sure about the veracity of the sasquatch links, that would have gotten my attention. It is also an assertion that is quickly and easily testable with just a quick Google search.

Of course, that same "quick Google search" on "Pacific Tree Octopus" or "Octopus paxarbolis" will immediately yield several sites asserting that the original site is a hoax, but that's so simple it's cheating.

[castleclear.livejournal.com]

Dear Obedient Serpent, Thank you for teaching about Science as well as human credulity as related to reasoning and information. Your summary conclusion "wow, you can con a 12-year-old into believing some crazy shit" truly made me laugh outloud, as did the pleasant reintroduction to my vocabulary of the word kerfluffle. Best Wishes, sir.