Athabasca University | AU Student/Staff Login | Invited Guest Login
Defaults matter
I have often written about the subtle and not-so-subtle constraints of learning management systems (LMSs) that channel teaching down a limited number of paths, and so impose implicit pedagogies on us that may be highly counter productive and dissuade us from teaching well - this paper is an early expression of my thoughts on the matter. I came across another example today.
When a teacher enters comments on assignments in Moodle (and in most LMSs), it is a one-time, one-way publication event. The student gets a notification and that's it. While it is perfectly possible for a dialogue to continue via email or internal messaging, or to avoid having to use such a system altogether, or to overlay processes on top of it to soften the hard structure of the tool, the design of the software makes it quite clear this is not expected or normal. At best, it is treated as a separate process. The design of such an assignment submission system is entirely about delivering a final judgement. It is a tacit assertion of teacher power. The most we can do to subvert that in Moodle is to return an assignment for resubmission, but that carries its own meanings and, on resubmission, still returns us to the same single feedback box.
Defaults are very powerful things that profoundly shape how we behave (e.g. see here, here and here). Imagine how different the process would be if the comment box were, by default, part of a dialogue, inviting response from the student. Imagine how different it would be if the student could respond by submitting a new version (not replacing the old) or by posting amendments in a further submission, to keep going until it is just right, not as a process of replacement but of evolution and augmentation. You might think of this as being something like a journal submission system, where revisions are made in response to reviewers until the article is acceptable. But we could go further. What if it were treated as a debugging process, using approaches like those in Bugzilla or Github to track down issues and refine solutions until they were as good as they could be, incorporating feedback and help from students and others on or beyond the course? It seems to me that, if we are serious about assignments as a formative means of helping someone to learn (and we should be), that's what we should be doing. There is really no excuse, ever, for a committed student to get less than 100% in the end. If students are committed and willing to persist until they have learned what they come here to learn, it is not ever the students' failure when they achieve less than the best: it is the teachers'.
This is, of course, one of the motivations behind the Landing. In part we built this site to enable pedagogies like this that do not fit the moulds that LMSs ever-so-subtly press us into. The Landing has its own set of constraints and assumptions, but it is an alternative and complementary set, albeit one that is designed to be soft and malleable in many more ways than a standard LMS. The point, though, is not that any one system is better than any other but that all of them embed pedagogical and process assumptions, some of which are inherently incompatible.
The solution is, I think, not to build a one-size-fits-all system. Yes, we could easily enough modify Moodle to behave the way I suggest and in myriad other ways (e.g. I'd love to see dialogue available in every component, to allow student-controlled spaces wherever we need them, to allow students to add to their own courses, etc) but that doesn't work either. The more we pack in, the softer the system becomes, and so the harder it is to operate it effectively. Greater flexibility always comes at a high price, in cognitive load, technical difficulty and combinatorial complexity. Moreover, the more we make it suit one group of people, the less well it suits others. This is the nature of monolithic systems.
There are a few existing ways to greatly reduce this problem, without massive reinvention and disruption. One is to disaggregate the pieces. We could build the LMS out of interoperable blocks so that we could, for instance, replace the standard submission system with a different one, without impacting other parts of the system. That was the goal of OKI and the now-defunct E-Framework although, in both cases, assembly was almost always a centralized IT management function and not available to those who most needed it - students and teachers. Neither have really made it to the mainstream. Sakai (an also-ran LMS that still persists) continues to use OKI technologies under the hood but the e-framework (a far better idea) seems dead in the water. These were both great ideas. There just wasn't the will or the money, and competition from incumbents like Moodle and Blackboard was too strong. Other widget-based methods (e.g. using Wookie) offer more hope, because they do not demand significant retooling of existing systems, but they are currently far from on the ascendent and the promising EU TENCompetence project that was a leader behind this seems moribund, its site offline.
Another approach is to use modules/plugins/building blocks within an existing system. However, this can be difficult or impossible to manage in a manner that delivers control to the end user without at the same time making it difficult for those that do not want or need such control, because LMSs are monoliths that have to address the needs of many people. Not everyone needs a big toolkit and, for many, it would actively make things worse if they had one. Judicious use of templates can help with that, but the real problem is that one size does not fit all. Also, it locks you in to a particular platform, making evolution dependent on designers whose goals may not align with how you want to teach.
Bearing that in mind, another way to cope with the problem is to use multiple independent systems bound by interoperability standards - LTI, OpenBadges or TinCan, for example. With such standards, different learning platforms can become part of the same federated environment, sharing data, processing, learning paths and so on, allowing records to be kept centrally while enabling incompatible pedagogies to run independently within each system. That seems to me to be the most sensible option right now. It's still more complex for all concerned than taking the easy path, and it increases management burden as well as replicating too much functionality for no particularly good reason. But sometimes the easy path is the wrong one, and diversity drives growth and improvement.
Jon Dron
I am a full professor and Associate Dean, Learning & Assessment in the School of Computing & Information Systems, and a member of The Technology-Enhanced Knowledge Research Institute at...
- On the importance of place
July 4, 2024 - 1:20pm
Jon Dron - At the end of this post a successful reader will be able to make better use of learning outcomes
May 9, 2024 - 1:08pm
Jon Dron - Hot off the press: Handbook of Open, Distance and Digital Education (open access)
January 4, 2023 - 9:55pm
Jon Dron - The importance of a good opening line
April 24, 2024 - 5:41pm
Jon Dron
Jon Dron
Tag cloud
Comments
Great post, Jon. I am constantly frustrated by the impacts of the business approaches taken to making decisions about LMS implementation, at the expense of freedom to meet instructional design needs. As an instructional developer, I'm frequently asked "How should I be using the LMS?" (we are a blended learning institution). My response, invariably, is that the teacher is asking the wrong question. They should be asking "What do I want and need to do?", from which we (the teacher/SME, and the ID) can start to identify the most appropriate tools. Some of those tools might reside in the LMS. In other cases, the LMS might confound our intended efforts, with little to no ability on our part to make the system work for us. Fortunately, I have the experience / skill to identify external tools, and ways to integrate them into to instructional design... but not every teacher/SME has that luxury. Nor do I always have the luxury of the time to go that extra mile! As you've noted, the more complex the layers you want to add, the more difficult it becomes to implement and manage. Scotty (Star Trek III) summed it up best when he said that "The more they overhaul the plumbing, the easier it is to stog up the drain!" Unfortunately, that's a reality we're going to have to live with for some time. It's hard to compete with the incumbent systems, and almost impossible for non-business entities to manage the development, implementation, and maintenance of a new platform.
Rob Power
Thanks Rob
Actually, I'm not sure the teachers' question is such a bad one. Terry Anderson talks about the dance between pedagogy and technology, but I think of it as being more like a machine (pedagogies - as in the methods and techniques for teaching - being technologies, as much as LMSs and course regulations). All the parts of that machine have to work together, and only a very few configurations actually work well. You can change any of the parts but, when you do, you have to look at the effects on others and the further consequences of doing so. In my example above, one way of circumventing the issue would be to adopt a controlling, teacher-dominated pedagogy: the methods would fit with the toolset much better. But that price would be too high for me - it would not fit with any of the other pedagogies I use at all, so the machine would have to be substantially rebuilt and, I believe, would not work as well as a result. At the very least it would be less flexible and less learner-oriented. Given that this wouldn't be acceptable to me, nor is abandoning the LMS altogether, it means I need workarounds. If I were stuck in Moodle I might use discussion forums for assignment submissions: this is messy and much too public (students would always get to see what others have done, regardless of whether it made sense to them or me) but that's another trade-off and, with some modification to the activities required so there is less potential for personal exposure, might just about work. Alternatively, I might ask for email or message submissions, but that would be a lot of work for all concerned and would limit the potential for sharing, so I would have to alter the peer-teaching parts to match, perhaps by adding more explicit instruction, as the cost of learner control. It's all a bunch of design tradeoffs, with each change altering the whole teaching machine in different ways. Luckily, I have the Landing, which solves these particular problems much better, albeit adding an extra layer of complexity for all parties. Aggregating technologies almost always softens the overall assembly and adds flexibility, but at the cost of extra effort and skill needed to make them operate and more opportunities for things to go wrong (I like the Scotty quote!). Another trade-off.
In an institution we have a vast range of technological constraints like timetables, policies, accreditation frameworks, rules etc, as well as social constraints like norms, social ties, and power hierarchies, as well as physical constraints like class sizes, availability, etc, to name but a tiny fraction, very few of which we have much control over. Often, technologies will be mandated or offer very limited configuration choices (e.g. classrooms, desks, chairs, LMSs, regulations, required textbooks, etc). And all of these things contribute to the teaching process, often far more than professional teachers. So 'what to I want and need to do?' is never a neutral question and pedagogy never comes first. In fact, it is often among the last things we think about and is usually a response to the constraints we work under rather than an overriding design driver. That's the nature of the beast. It might therefore be better to ask, for instance, 'what does this tool let me do?' or 'what does this tool prevent me from doing?' or 'what does this tool want me to do?', then to juggle and tinker with possible configurations until we find one that works. This is as true of methods and techniques for learning as it is for physically instantiated tools. You can weigh up the possibilities, find ways around constraints, discover new possibilities you never would have thought of without the constraints, as well as different opportunities provided by the tools that would have been impossible without them. For instance, though a few of us have flipped classrooms since long before the rise of the LMS, flipping is made so much easier now we have them: it was what Gibson would call an affordance but that I prefer to think of an adjacent possible, a propensity of of the system waiting to be discovered. It's a conversation with the tools, all of them, physical and otherwise, and they all play a role in the teaching process. We never teach alone and we are never the only teachers. I think this only causes problems when we don't pay enough attention to those other teachers, and so build teaching machines that don't work properly.
Jon
Thanks, for this, Jon.
I certainly agree on your pedagogy first approach, and I like the idea of a diversity of approaches for teachers and learners.
There are constraints we need to be cognizant of, however, the greatest of which are time and money.
Not all teachers want to take the time required to nurture learners to a final grade of 100%. I agree that it's certainly possible, but it does take time and effort, on the part of the teacher and learner.
In our case (AU) faculty may not have the time to devote to nurturing students, and we certainly don't have the budget to pay tutors to pursue such a learning model. In addition, not all students have the time or energy to devote to maximizing their learning opportunites--often 70%, or even a pass, is all they're looking for.
I think we need not only a diversity of technologies, but a diversity of pedagogies/assessment.
I've used the "contracting" assessment model in the past, where students contract for a particular grade based on the successful completion of a number of learning exercises--a potential grade of 90%, for instance, requires more learning exercises that one of 70%, but simply completing the learning exercises is not a guarantee of a contracted grade; the learning exercises must be SUCCESSFULLY completed.
I believe we should be offering those students who wish to optimize their learning an opportunity to do so, as well as providing opportunities to those who wish to gain minimal certification through a pass. It will certainly cost more, in terms of time and money, to work with students who are "maximizers," but perhaps this cost can be subsidized by ensuring teachers have a fair sprinkling of "minimizers."
Good conversation.
Thanks Derek
I agree, under our current course model this would not be very sustainable, at least taken to its ultimate conclusion. I do allow something a little like it in some of my own courses, inasmuch as students can continually refine what they have done based on (ungraded) feedback until it is better, though formally I only allow a fixed number of iterations in order to keep the workload manageable and the course contract dates are still a major constraint.
I like the contracting model, makes a lot of sense - it could be one way that we might alter the extraordinarily constraining technologies of course funding and tutor pay models. Another way might be to use a subscription model, a bit like WGU, in which students pay a fixed sum for ongoing support for as long as they wish and that pays for a certain amount of tutor time each time period (month/week), with summative assessment an optional extra as and when they feel ready. Or perhaps we could go for a totally flipped model, in which our course materials are entirely free and open (or at a very low subscription), but where students pay for help and assessment on demand, as and when needed. These sorts of approach, or something like them, are essential if we truly believe our mission is to educate. Alas, that is not our only mission. We (and all of academia) are awfully confused and conflicted thanks to our other mission to accredit, which is nearly always deeply anti-educational at its very core. The more we can separate those two roles, the better it will be for all concerned!
Jon
Agreed, there's a lot of things going on in the accreditation process, and not all of it relates to optimizing learning opportunities.
We're breaking free of some the disciplinary constraints these days, which were more concerned about policing entry into the various academic "guilds" or disciplines than learning. There are, of course, good reasons for disciplinary standards, but sometimes it's more about restricting membership rather than expanding.
This is why I think it's essential for AU to undertake a self-assessment to identify what it is we are committed to and how best we can accomplish it. Currently, we're trying to do a lot of very different things for a lot of very different people.
I don't believe we should try to be all things for all learners (the comprehensive research university (CRU) model). I know many of my colleagues don't agree, and see AU's "promotion" into the CRU pool as a good thing. But I don't think we'll ever be able to compete with UofA and UofC, or even UofL, for that matter, because we just don't have the numbers or resources. Moreover, I think AU can make a very important contribution in areas the CRUs don't. As faculty, we don't have to abandon research, but we can turn away from competing for research funding in time-consuming, mega competitions wherein others determine what is of importance for us to research--SSHRCC & NSERC, for example, toward research that is more pedagogically focused and meaningful for the mission we choose, and/or our discipline/field/profession.
There's nothing wrong with accreditation in principle: in fact, it can be a mighty good idea. It should just be dissociated entirely from learning and teaching. Learning for grades is about as useful as standing over someone with a big stick and forcing them to eat your choice of candy, with equally predictable results.
A self-assessment is a mighty good idea! I agree that we are spreading ourselves thin in research. With about 180 full-time faculty (compared with, say, the UofA, who have about 1700) combined with a very limited number of doctoral programs (a major problem we have to solve) we simply don't have the personnel. Thanks to our tutor model and paucity of doctoral programs, we are largely a de facto teaching university no matter how amazing our faculty might be: it's simple arithmetic. However, there is one crucial area where we excel mightily: we have by far the highest concentration of top quality thought-leading distance and online education researchers in the world, bar none. Our non-faculty distance/edtech researchers alone leave the UofA standing, and our faculty are positively stellar. That's precisely why I came. While we have some great pockets of research in other fields and I think we should nurture and cherish them, if for no other reasons than that passion for a subject feeds back into teaching and diversity is crucial to sustain creativity, we just don't have critical mass in any other area than online learning to sustain big projects or to claim eminence. Partnerships are not a bad idea though.
I'm all in favour of research on a shoestring and alternative sources of funding. SSHRC or NSERC are not the only fruit, and efforts to meet their demands take way too much time away from productive research.
Right now, FHSS is at ground zero, IT wise; other faculties would have to give up what they currently have to start something anew.
I've suggested to Cindy Ives in the past that FHSS is the place to start developing and refining a first class online learning environment, but we need some bodies and support.
FHSS is the perfect location to launch a new online learning environment from, and to establish an innovation unit.
We'd have to do some serious convincing, because it will require resources, but I think this is the ideal starting point to launch a serious effort to bring AU up-to-speed in terms of online provision and pedagogy.
First thing is to get FHSS innovating with existing systems, Moodle in particular. In the meantime, an innovation unit could be exploring how to improve/expand the existing system.
I'm just not sure we can create the will to make this happen.