Quantification of Qualitative Data

Reading: Libarkin, Kurdziel, & Anderson 2007 College Student Conceptions of Geological Time and the Disconnect Between Ordering and Scale. J. Geoscience Ed 55(5) November

In this study Libarkin and co. continue their work on learning about how students learn. And importantly, learning what the students don't learn. They use questions from the Geoscience Concept Inventory (GCI) to evaluate a range of students from a range of colleges. One pretty neat thing they did was to give the students complete a timeline. The timeline had Earth Forms at the bottom and today at the top. They were asked to put in the following events in order and record the time that those events occurred:

1. First Life on Earth
2. Appearance of the Dinosaurs
3. Disappearance of the Dinosaurs
4. Appearance of Man

Most students did pretty good on the sequence of events, only a few did crazy things like have dinosaurs appear, then Man, then dinosaurs disappear. They did OK on getting the dates in. Nearly all students failed to get the scale correct, even when given the timeline on graph paper. They'd do things like have dinosaurs appear and die out in the middle of the time line, have humans appear way too far down the time line, etc. Interesting enough of a result on its own.

But then Libarkin et al went a step further and mapped the timeline results to a Ternary diagram.

A Ternary Diagram is basically a chart, like any data you can chart you find out the x-axis value, the y-axis value and plot a point where they intersect. Here, instead of two axes, there are three and they cross each other. The diagram itself looks like a triangle, the axes run through the corners of the triangle. The other neat thing about a Ternary diagram is that the three axes need to all sum together to make a whole, so a Timeline naturally lends itself to this sort of analysis.

The authors set out to plot each student's timeline on the Ternary Diagram. They determined the percentage of the total timeline for three intervals:

1. Earth Forms - First Life
2. First Life - Man
3. Man - Today

Each of these intervals make up an axis in the diagram (well, technically the percentage of the whole that each interval is makes up an axis). So now every student timeline has a value for each axis and can be plotted on the diagram. This allows the student timelines to be compared and analysed further. Typically, in geology, Ternary Diagrams are used to represent phases: phases of minerals, types of sediment, etc. A geological Ternary diagram will usually have several areas mapped out on it that correspond to, say, K-feldspar, or biogenic ooze, depending on the diagram. Here the authors feel that they've discerned a few 'phases' of student concepts. The creationist students tend to plot together, the students in an "Honors Natural History Field Course" mark out a phase, and between lies a wide space where other students fall into.

Fortunately, the point that represents reality, the scientific concensus on the issue, falls within the same 'phase' as the Honors Students. BUT, the Honors students phase is really large. Their answers aren't ALL over the place, they don't cross into Young Earth Creationist (YEC) territory, but a helluva lot of them are just completely wrong.

It might be better to think of the diagram as having three phases, a small circular phase around the correct answer, meaning that there is an agreed upon answer, and some error is allowed for amoung students. A second YEC phase, just like in the paper, and a third phase, everything else. Viewed that way, you can see that the Honors students DO NOT cluster in the "Correct" phase. The smallest spread (but also a very small sample size) is from "Students in a small elite private University in the Northeast (and they were enrolled in a gen ed bio coure)", and following that, to me it looks like a "Large State University in the SouthWest". So you might think that small elite Universities are faring better in their geoscience education, except that he "Honors College, Natural Science Field Course" students were from a "Large State University in the East".

As far as demographics go, they were typical of what I've seen in geoscience courses (keeping in mind that these weren't all students from geoscience courses in the first place), that is to say, a buncha white kids. Couple of other ethnicities represented, but no black students. Thats probably another issue best examined at another time.

I don't know if this is standard practice in the social sciences either but in the demographic section, one student abstained from reporting their ethnicity, and was counted as a Caucasian. For some reason I find that humorous; is it expected that non-whites will always fillout demographic information, or is it expected that only white people wouldn't bother to do it (or is that the same thing)?

By the way, most of the above blather about the procedure and results is distilled into its essence in the paper as:

"The timeline distance between three pairs of events, normalized to the sum of these distances, generates a ternary diagram from which conceptual zoning can be identified."

I think that the Lacodaemeons would've loved scientific writing.