Asking Better Questions: Ends and Means in eLearning

In a previous post I considered the possibility that much of what is presented as “Innovation” is anything but that. With access to some fairly new and attractive or otherwise popular products and armed with even a slight grasp of how to operate them it’s relatively easy to create an appearance of innovation. Simply put, if you can get your hands on some new gear, in even a short while you can present quite a convincing front.

Worse again, it is equally easy to generate what passes for proof; to an untrained eye you can make it look like your so-called innovation is creating some real differences. Any of these strategies can give you reams of what looks like convincing evidence:

  • Deliberately pick enthusiastic students or teachers and pile on the anecdotes that endorse the desired point of view. People who rely on system-one (more or less intuitive) reasoning are easily swayed by stories so it won’t be hard to capitalize on that to get some people talking about how innovative the project is.
  • Stage the project in a relatively well-off school or class and then compare the results from this highly-biased “treatment” group to the population in general. Very few will dig deep enough to see that the superior achievement results predated, and were independent of, the treatment.
  • Rely on manufacturer or vendor supplied “research” when crafting reports, proposals and press releases.
  • Bluff; just preface your claims with clauses like “decades of research shows…” and leave it at that. You might be surprised to see how few—if anyone—will call you out. Besides it will be relatively easy to portray those that do as kooks or curmudgeons.

That said, you could instead opt to take the more difficult path and strive for some real gains.

Notwithstanding the cynical tone of the opening of this post, it needs to be emphasized that emerging technologies should be welcomed, albeit guardedly, in all places of learning. I’ve come by this knowledge the hard way with ample personal experience in doing it both the right AND the wrong way. Lessons learned well generally involve first-hand experience and I have it, having done things for both the right and the wrong reasons–but generally having benefited from the experience in either case. It can be summed up succinctly: it’s best when you develop and refine an appropriate match between the technologies and the desired learning outcomes. This means, in particular, to start with the right sort of question:

  • Bad Question: How can (insert gadget name here) be used in the (insert subject name) classroom?
  • Better Question: What combination of equipment and methodology will foster better achievement in (insert subject name/outcome area)?

Notice the difference? Instead of placing the focus on the tools, place it on the learning.


You might say that, in the end, the two are the same. Yes, in both cases the goal is to do a better job. Take a closer look, though. Notice that the bad question is, in fact all wrong. First, by selecting a particular device it sets serious limits on what can be done. This can even lead to the selection of inferior methods. Consider this: A teacher wants to see if physics achievement can be improved through the use of tablets in the class after noticing that there are some good simulations available and asks, “how can I use tablets in the classroom?” With the best of intentions the approach is changed, replacing hands on activities with simulations. Now, while simulations are an excellent way to introduce topics, especially ones that cannot be done cheaply or safely, it makes little sense, when you think about it, to replace hands on activities involving motion, sound, electricity and light with simulations in which the only physical interaction is sliding a finger along a glass screen! After all, physics is all about interacting with the physical world! How ironic! If, instead the right question had been asked, no doubt the simulations would have been used but their use would have been balanced with follow up real-world interactions.

Second, the selection of a particular device sets in place a condition in which demonstrable improvements are expected. That’s nice, but what if it’s the case that the new technology is in fact inferior? You might suggest, “no problem, the report will show this.” Think about it, though, and be careful to layer in some human nature.

Consider again the previous case involving tablets and physics. Suppose that the unit of study was about current electricity and the tablets were used to explore the topic through simulations in which students constructed virtual circuits involving batteries, resistors, lamps, switches and meters to measure voltage and current instead of doing the same with the real thing. At the end of the unit the evaluation would be based on what could be measured, either online or using pencil and paper, and NOT on actually constructing the circuits.

How likely would it be that students would be able to do the same with real circuits? Not likely.

How likely is it that they would do about the same on a test? Very likely.

What’s the difference? In which class would it be more likely that you would find someone who could help you wire your basement? If, on the other hand, the right question had been asked, again, in all likelihood the simulations would have been utilized but their user would be balanced, blended with hands-on activities too.

Focusing instead on the learning will have two likely outcomes:

  • You will likely not get famous as “it’s” clearly about learning and not about you.
  • The project will show modest but useful results.

Whenever embarking on any effort to improve results in education it’s important to bear in mind one simple truth: you are not starting from scratch. The “traditional” methods that self-nominated reformers (most of whom have only limited classroom experience, other than the imagined stuff) so love to mock are in fact reasonably effective. The huge majority of people–those who’ve not been the beneficiaries of their enlightened practice but who have still managed to thrive nonetheless bear testament to that. Existing methods are, perhaps, not as good as they could be but are still nonetheless effective. Reformers should bear in mind that the traditional methods they so distain have several important advantages over proposed new ones. First, they are understood since, in all likelihood, existing practitioners not only use them now but will likely have been taught using them. More importantly, though, traditional methods have been refined from extensive classroom use. Proposed methods, by contrast are not well understood, raw and untested.

Far too often reformers boldly charge into classrooms armed with little more than vague ideas, shiny new equipment and an unhealthy combination of ignorance and arrogance. Students, parents, colleagues and administrators generally tolerate the ensuing activity since (a) it probably doesn’t interfere with them too much and (b) there is always the chance that some good might come of it. The proponent will usually get a little something—a write up in a journal, perhaps a trip to a conference, maybe even an award—but in the end the students will likely be left no better off and the effect on general classroom practice will be negligible.

It does not need to be that way, though. If, instead, the proponents asked the right question, one that focused on making some real improvement in student learning, then wins would be had all around. That is, better teaching and learning would result and, who knows, maybe the innovator’s career would get a boost anyway.