Asking Better Questions: Ends and Means in eLearning

In a previous post I considered the possibility that much of what is presented as “Innovation” is anything but that. With access to some fairly new and attractive or otherwise popular products and armed with even a slight grasp of how to operate them it’s relatively easy to create an appearance of innovation. Simply put, if you can get your hands on some new gear, in even a short while you can present quite a convincing front.

Worse again, it is equally easy to generate what passes for proof; to an untrained eye you can make it look like your so-called innovation is creating some real differences. Any of these strategies can give you reams of what looks like convincing evidence:

  • Deliberately pick enthusiastic students or teachers and pile on the anecdotes that endorse the desired point of view. People who rely on system-one (more or less intuitive) reasoning are easily swayed by stories so it won’t be hard to capitalize on that to get some people talking about how innovative the project is.
  • Stage the project in a relatively well-off school or class and then compare the results from this highly-biased “treatment” group to the population in general. Very few will dig deep enough to see that the superior achievement results predated, and were independent of, the treatment.
  • Rely on manufacturer or vendor supplied “research” when crafting reports, proposals and press releases.
  • Bluff; just preface your claims with clauses like “decades of research shows…” and leave it at that. You might be surprised to see how few—if anyone—will call you out. Besides it will be relatively easy to portray those that do as kooks or curmudgeons.

That said, you could instead opt to take the more difficult path and strive for some real gains.

Notwithstanding the cynical tone of the opening of this post, it needs to be emphasized that emerging technologies should be welcomed, albeit guardedly, in all places of learning. I’ve come by this knowledge the hard way with ample personal experience in doing it both the right AND the wrong way. Lessons learned well generally involve first-hand experience and I have it, having done things for both the right and the wrong reasons–but generally having benefited from the experience in either case. It can be summed up succinctly: it’s best when you develop and refine an appropriate match between the technologies and the desired learning outcomes. This means, in particular, to start with the right sort of question:

  • Bad Question: How can (insert gadget name here) be used in the (insert subject name) classroom?
  • Better Question: What combination of equipment and methodology will foster better achievement in (insert subject name/outcome area)?

Notice the difference? Instead of placing the focus on the tools, place it on the learning.


You might say that, in the end, the two are the same. Yes, in both cases the goal is to do a better job. Take a closer look, though. Notice that the bad question is, in fact all wrong. First, by selecting a particular device it sets serious limits on what can be done. This can even lead to the selection of inferior methods. Consider this: A teacher wants to see if physics achievement can be improved through the use of tablets in the class after noticing that there are some good simulations available and asks, “how can I use tablets in the classroom?” With the best of intentions the approach is changed, replacing hands on activities with simulations. Now, while simulations are an excellent way to introduce topics, especially ones that cannot be done cheaply or safely, it makes little sense, when you think about it, to replace hands on activities involving motion, sound, electricity and light with simulations in which the only physical interaction is sliding a finger along a glass screen! After all, physics is all about interacting with the physical world! How ironic! If, instead the right question had been asked, no doubt the simulations would have been used but their use would have been balanced with follow up real-world interactions.

Second, the selection of a particular device sets in place a condition in which demonstrable improvements are expected. That’s nice, but what if it’s the case that the new technology is in fact inferior? You might suggest, “no problem, the report will show this.” Think about it, though, and be careful to layer in some human nature.

Consider again the previous case involving tablets and physics. Suppose that the unit of study was about current electricity and the tablets were used to explore the topic through simulations in which students constructed virtual circuits involving batteries, resistors, lamps, switches and meters to measure voltage and current instead of doing the same with the real thing. At the end of the unit the evaluation would be based on what could be measured, either online or using pencil and paper, and NOT on actually constructing the circuits.

How likely would it be that students would be able to do the same with real circuits? Not likely.

How likely is it that they would do about the same on a test? Very likely.

What’s the difference? In which class would it be more likely that you would find someone who could help you wire your basement? If, on the other hand, the right question had been asked, again, in all likelihood the simulations would have been utilized but their user would be balanced, blended with hands-on activities too.

Focusing instead on the learning will have two likely outcomes:

  • You will likely not get famous as “it’s” clearly about learning and not about you.
  • The project will show modest but useful results.

Whenever embarking on any effort to improve results in education it’s important to bear in mind one simple truth: you are not starting from scratch. The “traditional” methods that self-nominated reformers (most of whom have only limited classroom experience, other than the imagined stuff) so love to mock are in fact reasonably effective. The huge majority of people–those who’ve not been the beneficiaries of their enlightened practice but who have still managed to thrive nonetheless bear testament to that. Existing methods are, perhaps, not as good as they could be but are still nonetheless effective. Reformers should bear in mind that the traditional methods they so distain have several important advantages over proposed new ones. First, they are understood since, in all likelihood, existing practitioners not only use them now but will likely have been taught using them. More importantly, though, traditional methods have been refined from extensive classroom use. Proposed methods, by contrast are not well understood, raw and untested.

Far too often reformers boldly charge into classrooms armed with little more than vague ideas, shiny new equipment and an unhealthy combination of ignorance and arrogance. Students, parents, colleagues and administrators generally tolerate the ensuing activity since (a) it probably doesn’t interfere with them too much and (b) there is always the chance that some good might come of it. The proponent will usually get a little something—a write up in a journal, perhaps a trip to a conference, maybe even an award—but in the end the students will likely be left no better off and the effect on general classroom practice will be negligible.

It does not need to be that way, though. If, instead, the proponents asked the right question, one that focused on making some real improvement in student learning, then wins would be had all around. That is, better teaching and learning would result and, who knows, maybe the innovator’s career would get a boost anyway.


15 thoughts on “Asking Better Questions: Ends and Means in eLearning

  1. I run the risk of sounding like a broken record [Pause .. it just occurred to me that anyone younger than, say, 30 or so has no idea what that phrase means! Boy, am I old.] but your ‘bad’ approach is quick-and-dirty and has some small chance of success while your ‘good’ approach really requires some serious thought. Many who find ourselves in the front lines have little time to develop grandiose strategies of teaching and learning (I’m sorry to say … but it’s true). It’s easier to grab the ‘front end’ and hope its application can assist in getting the message across in the classroom. Anything more thoughtful would take time away from all of the academic responsibilities I’ve mentioned here before. I don’t think my Chair or my Dean would look very positively upon it if I decided to support fewer students in courses of independent research because I was formulating a plan to improve the teaching of X in the classroom, using technologies A, B, and C. That’s not what I’m paid to do. I understand, too well, what you’re saying. And I agree entirely. But, unless there’s some sort of change in the winds from above (i.e., administrative winds, that is) … this isn’t going to happen. People will continue to apply bad ideas to classroom and laboratory teaching. Until faculty are recognized and rewarded for being good teachers … the bad ideas (quick fixes) will prevail at the cost of much better good ideas which take time to develop (as you, yourself, point out). Boy that sounds negative … but in my situation it has turned out to be the truth. Innovation takes time, and few of us have it. Perhaps down-the-road, at a time when Colleges and Universities come to the realization that they really have to be accountable for the ‘value added’ – what you are suggesting will not happen. D

    1. You are correct–unfortunately. I think the line of thinking that led me to make that suggestion around questions has come from two general frustrations that we share. First, after looking back on what is creeping towards four decades in education it seems to me that as far as teaching and learning are concerned we have not made as much headway as anyone expected. We have wasted a lot of time in pursuing the wrong things (think the half-century wasted on behaviourism, the few years wasted on “learning styles” and “differentiated instruction”) simply because the big names were pursuing them. Second, there’s no coordinated effort to try and ensure that individual campuses can cooperate and share ideas, especially around courses that are more or less the same from place to place.
      In the end, though, it leaves at that difficult place–instructors are spread far to thin to do much by way of the huge improvements.
      In addition It has been my own experience that trying to free up some instructor time to devote to content development is not as effective as it seems on the surface. Suppose, for example, an instructor is responsible for 4 or sections in a semester–a very heavy load that would leave essentially no time for research or development. Suppose further that the administration decides to take away one section, thus leaving three and then says to the instructor, “you are now 3/4 time instructor and you need to devote 1/4 of your time to development.” On the surface you would think that this would then result in a situation in which the instructor had the time and one in which one could expect some tangible products in the form of plans and student resources. It has been my experience, though, that when you do this the teaching load then swells to fill the remaining time. That’s because instructors never have the time to do the things they feel they should be doing and when given a little leeway, fill the gap with those things they already decided they should be doing. Yes, the students likely win owing to the extra attention and effort but there will be little left to show after the semester is over. It’s my opinion that if an instructor is asked to work on development then the time allotment needs to be full-time for the duration.

  2. I really liked this sentence: “You will likely not get famous as “it’s” clearly about learning and not about you.” – probably the essence of what makes a good teacher in any sort of learning setting, e- or not.
    I had just replied to a comment re my review of that QFT book I like so much – this author is a very good example: Not a famous professor that wants to show off experience and/or do “marketing” via publishing a book, but somebody relatively unknown in the community – except for writing a book that really puts students, understanding, and the subject as such first.
    Adding a self-critical remark I think I was never such a teacher. I enjoy educating experts who challenge me but I am probably not patient enough to explain things really well to people who have a learning style very different from my own or students who “have to” learn something because of formal requirements and who are not really intrinsically motivated. I totally don’t like keeping to any sort of agenda, including one I had made up myself – I rather jump on the most difficult question people throw at me and go off on a tangent.

    1. Yes, in the end universities have to work very hard on three different activities: research, teaching and outreach. These are three rather different activities and, unfortunately, the expectation is often that you have to be excellent at all three–completely unrealistic. One way that institutions get around this is by placing a large part of the teaching burden on adjuncts or per-course instructors. This is often a very good idea in the long run because (a) adjuncts are cheaper and (b) you can hire people who are very good at teaching regardless of the other two items. This is also bad, though because (a) adjuncts, then who are also highly qualified academics live a second-class life on that lower salary and (b) teaching and learning is never discussed at faculty council because the adjuncts are often not invited (that varies from institution to institution).

  3. Eric

    Hi Maurice. I hate to tell you that question two is really no better than question one. I think that you were endeavoring to say “better achievement with outcomes” or something to that effect. Better outcomes simply implies that the method and technology improve the outcome itself. Not exactly what you were trying to say … ?

  4. I wish I had saved the comment on the innovation post until I had read this. When I worked in public education one of my colleagues teaching kindergarten fought the imposition of technology in her classroom. Her response was that kids learned enough with Atari, Game Boy, and iPods (depending on the decade) and didn’t need formal instruction time to develop this skill. She is an incredibly gifted teacher, and highly effective. Her idea was that the kids themselves would drive innovation in the program, so long as individual educators had small enough classes to be able to observe and respond. In one year that I worked with her we had a unique set of learning needs in the classroom, and really had to work to bring a variety of learning tools and approaches to achieve the best results for the students. She had encouraged me to publish some of the things I had brought to the classroom (and the same can be said for the materials she has developed), but there was no time to think about this!

    1. That’s an important point: effective teaching is something that completely drains the teacher’s energies. It has to be that way, owing to the huge loads and even greater sets of needs. Sadly, then, most of the truly useful stories (and I am not referring to the stuff that merely looks good) often do not get told. One of the good movements that’s taken root over the past decade is the facilitation of online professional learning communities in which educators are able to share real strategies at times that work for them. They’ve been slow to take of but as the kids who have grown up socializing online become professional educators this, no doubt, will become mainstream. I look forward to that day.

      1. You make a good point. When I did my one year of private pre-school teaching, one of the resources that was passed on to me was a website a retired early educator was developing with her career’s worth of Kindergarten materials. The digital format made everything easy to adapt for younger children or a slightly different project need. But while a teacher is still in class, it is difficult for her/him to get on-line! It would be interesting to be a website facilitator for an educators’ group. I’d love to do something like this for pre-K and Kindergarten teachers. (It would be so sad to see some of the ideas of long-time teachers retire out of the system and disappear, because–as you say–they have not grown up in the on-line communities.)

  5. Well, if it’s not going to help me get famous, I’m not interested.
    On a serious note: I find myself, as usual, agreeing with everything you say, which makes it hard to leave a comment.

  6. I am becoming a cranky parent because in the classroom it is expected that our children need to bring in I-Pads. I believe at an elementary level they should be learning the fundamentals of math, writing, and learning to use the library. As much as playing games and researching on the I-Pad is a help sometimes I think it is a hindrance to the basics. Call me old-fashioned but innovation has its place just not necessarily always in the classroom.

    1. Yes, it always needs to come back to the fundamental question: what are the learning outcomes? As I’ve often mentioned, too often the various devices derail the essentials. Tablets have awesome potential, as do other classroom devices such as interactive whiteboards but they need to be used in conjunction with well-developed methods that put the learning in front.

Comments are Welcome!

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s