What would be your perspective on exams as part of professional certifications? Would you consider exams appropriate or would you suggest other approaches?
Exams might be efficient but, most of the time, completely lack any authenticity. The ability to answer questions in a weird and unrealistic setting may have little correlation with the ability to perform your professional duties, unless they are usually performed under exam conditions. Most professionals perform their functions in a rich community, supported by the tools and information sources of their trade, in a specific context. Exams of the sort we usually provide in academia do not allow that and, if they do, are not the things I am complaining about. Portfolios or similar tools would usually be far better, especially mapped to evidence. Fairer, richer and more revealing. Viva voces would work in some circumstances. So would descriptive essays in others. So would evaluations by accredited peers of work done in a professional context. There is no hard and fast rule about what works in a given situation but, whatever system you use, it should show that the person being accredited can perform the job they are being accredited to do. Exams seldom show that. If you want to show that you are a computer professional, show that you can do that under the conditions computer professionals work under. If you claim to be a surgeon, prove it by performing surgery (supervised by real surgeons of course :-). And prove that you can do it again under different conditions.
I am fully in favour of summative assessment in most areas, but there are many ways to do that which work better than unseen written exams of the sort academia has been infested with for the past couple of hundred years. In the Western world, they were invented because it was really hard to prove geometric knowledge in a viva (China had them over a thousand years before that). They persisted and gained traction because they were cheap and easy for academics to administer and showed *some* correlation with actual skill. But they are a million miles from being a universal solution for all situations and the correlation with professional ability is not that strong. Many people fail them not due to lack of skill in the profession but due to lack of skill in taking exams, and vice versa. And they are not even that cheap in many situations. At Athabasca, for instance, they are incredibly expensive, stressful and time consuming, and attack the very people we most wish to support - typically, working people who, for one reason or another, did not follow the standard academic path when others did.
I don't want to get rid of them any more than I want to get rid of lectures or learning management systems or drill and practice approaches to teaching, though all have major and incontrovertible weaknesses. I just want us to choose when we use them wisely and reflectively. That will make them a lot rarer than they are.
I wonder if diaspora is going to have each person have a GPG key. I think usability is going to be the big issue with them in terms of where the "seed(?)" is hosted. They'll need free hosting providers of some kind. I should read more about it.
Yes, I think that's the plan Curtis: GPG is at the heart of it all. They suggest that most people will go for hosted, cloud-style solutions but that they will be able to run their own if they have the resources and interest, and (crucially) should be able to move from one provider to another with relative ease.
If we consider
a) crisp tags=tagging,
b) fuzzy tags= tagging(0...1) /* fuzzy quality */,
c) multi-valued tags=tagging((quality=0...1),/* fuzzy quality */ (value_to_learner=0...1) /* importance with fuzzy quality))
I had thought Intent would be identified in the fuzzy or multi-valued tag examples. As mentioned the cold start problem would be worse as the tags become more complex but through a combination of UI and adaption, personalization to decrease the overhead with tagging we should get better data of all three types.
I should note, that I thought of scalar tags as C, which effectively include the confidence or fuzziness of B for specific attributes. This way as a learner, I can also review the tagging meta-data created by more experienced.
I think the above comments are an example of how a continued discussion can begin to flush out an idea, push out and hopefully clarify points and when this has reached its natural end I wonder how this complete discussion might be tagged. I am not suggesting this conversation is anywhere near done - just an observation.
I am trying to think through how I might answer you Jon with respect to an example of an intent tag. I have not yet gathered a single piece of data and yet as I read through the prior contributions of learners in my course archive I am concerned as to how others may find and then perceive value and what they might need to assist them in possibly finding their way through the material. (Maybe I am just expressing doubt about my research) However I see great potential in this archival mass partly due to my perspective and partly because I am working hard to find this value. I think tagging is just one element in this process.
In reading through two years of course contributions I find rich discussions, great clarity of intent, as well as a lot of phatic asides. I think the phatic asides can be as valuable as the rich contributions and in most cases it is clear that they are just an aside comment but there are also a number of postings that have rich content but appear to wander somewhere else completely. The idea of an intent tag in this instance might be "an aside" or "off topic" or something more focussed on the actual wandering. I understand your concern about the metadata but maybe the idea of an intent-tag is less about the broader aggregation and could be more seen as a set of sub-tags. This does assume that the author used some clear crisp or fuzzy tags as well but I think of systems that use hierarchical naming schemes and maybe tagging needs to evolve into some form of multi-layered hierarchy. One layer (the top) that spoke to a more universal audience and sub-layers that spoke to a local audience.
@Stuart - I completely agree re sub-tags (we have work in progress to try to do that and my earlier systems used a 2-stage hierarchy of crisp tags then a further layer of fuzzy ones) but it becomes very complex in interface terms and massively increases the cold start problem when we do that. Each higher level in the hierarchy is a little tag system in itself in which a separate tag cloud develops.
I like the concept of phatic tags - it opens up a huge realm of other potential speech acts we might encapsulate as tags- performative tags (e.g. 'I agree', 'I do') for instance. I guess the value is particularly great if we are performing qualitative content analysis or want to visualise dialogue structures but I am always wary that every time we impose a constraint or affordance we are changing the system: in its weakest form it is a kind of social equivalent of Schroedinger's cat, in its strongest it can profoundly shape and form interactions. This can have value when we are strongly aware of what and why we are shaping it the way we do though, more often, we have tended to let it happen: for instance, the highly constraining form of the threaded discussion that is so ubiquitous and unreflectively implemented in much online learning. Giving people more control over the kind of tags and other fields used, as well as the ways things can interact, seems like a good idea.
Given the developing discussion here, it seems like it would be good to let people (collectively) create what would effectively be a structured database: to add fields that allow us to capture more structured data such as 'intent tags' or Glenn's more structured tags, or Eric's multi-valued tags. Once a particular kind of data entry field has been created by someone, anyone else could use it when adding resources (blogs posts, wiki pages, bookmarks etc) if they want. There is actually a forms plugin for Elgg that allows administrators to do something vaguely along those lines so it would not take too much to convert it into something a bit more social and let anyone play. Once you have defined a certain field type (e.g. 'intent', 'metatag' etc) you and others could re-use it in other forms. It would be hard to make it do more than act as a search field or maybe cluster things as behaviours are much harder to enable than data capture, but it might help to give a bit of hardness to the process if that's what people need.
@Eric-
That's an interesting distinction: so, are multi-valued tags (i) effectively multi-dimensional fuzzy tags, or do you mean (ii) that this is a fixed array of two elements that includes both the evaluation itself and a rating of the personal value of that evaluation to you? It's beginning to sound quite recursive!
If we are talking about multi-dimensional fuzzy tags then we might as well simply use fuzzy tags and allow people to select more than one at a time: that is a more flexible solution than an array. If I use a fuzzy tag of 'value to me' then I can achieve exactly the same result without the complexity.
If, on the other hand, it is a fixed array with two associated elements, the fuzzy tag itself and the value of the tagged resource to me as a learner then it gets more interesting but it is redundant: the fact that someone uses a tag in the first place carries with it an implication that the tag has some meaning or relevance - there would be no point in tagging something about which you care little. If I say that this is 'good for beginners' and, say, give it a rating of 5/5 it not only means that I think it is good for beginners but that I find this worth noting in the context I am using it, from my own perspective. If I give it a rating of 1/5 I am implying that 'good for beginners' is important to me but that this is hopeless in that context (but I might find other things of value about it). Why would I tag something using a quality that is unimportant to me? So, the fact that I have tagged it in the first place carries an implication of both the value of the resource and its relevance to me as a learner. It is of no interest to others how useful I found it as a separate concern from the tag itself. The interesting thing to others is why I found it valuable and how valuable I found it in that context, which is what fuzzy tags already give us.
And yes, the data become exponentially more sparse and the cold-start problem even colder.
Alas, we gain nothing from people who are more experienced as tags are thrown off along the learning journey as we learn: when they tagged things they were in the same boat as we are. Now they have learned what it was they sought to learn, they are unlikely to go back and change the weighting unless they are either remarkably altruistic or forced to do so by teachers.
All bets are off if we want to do some collaborative filtering or make use of a reputation system, where individual relevance does make a big difference and may well be worth recording, but there are gigantic cans of worms once we start down those paths about which many PhD theses have been written.
The Landing is a social site for Athabasca University staff, students and invited guests. It is a space where they can share, communicate and connect with anyone or everyone.
Unless you are logged in, you will only be able to see the fraction of posts on the site that have been made public. Right now you are not logged in.
If you have an Athabasca University login ID, use your standard username and password to access this site.
We welcome comments on public posts from members of the public. Please note, however, that all comments made on public posts must be moderated by their owners before they become visible on the site. The owner of the post (and no one else) has to do that.
If you want the full range of features and you have a login ID, log in using the links at the top of the page or at https://landing.athabascau.ca/login (logins are secure and encrypted)
Posts made here are the responsibility of their owners and may not reflect the views of Athabasca University.
We block sites that track your web browsing without your permission. If a link is greyed out, click once to enable sharing, once more to share.