Landing : Athabascau University

The Psychology of Hiring: Why Brainteasers Don't Belong in Job Interviews : The New Yorker

An interesting article that makes a very strightforward and obvious point, with some evidence, that brainteasers in job interviews do little more than demonstrate the candidate's ability to do brainteasers in job interviews. They do not predict success in the jobs they are filtering for. The parallel implications relating to typical exam processes and practices in educational systems are clear. 


  • A completely parallel implication is clear - where the question asked during the examination (etc.) is completely outside of the context of the course material it can't be of much use in evaluating the student in that material. There are questions or approaches that are not as easily recognized as being outside of the context.

    An example from a "teaching programming" course that I observed.

    The students were asked to write a program.

    On the surface this seems completely within context (actually - without the following constraints it is). During this exam they were allowed one book, no access to the Internet, and they were not allowed to ask for help from the professor or each other, whereas all the practice to that time, and perhaps more to the point their future tasks as employees, would allow all those additional resources (books, tutorials, interaction with mentors, peers, and the Internet as a whole). The professor's understandable concern was that the students would not do their own work, but it completely changed the environment the students were used to operating within. At least the context is much closer than the linked example.

    BTW the solution to something like this is not easy but it starts with having a relationship with the student(s) where the professor trusts them, and the student respects that trust. Other techniques include: the choice of task/question can be personalized, and some effort at cheating detection (not necessarily formal) as opposed to cheating prevention can be brought into play. (I'm sure that isn't the whole solution, but it's a start.)

    James Ronholm July 1, 2013 - 1:00pm

  • Yes - I have seen such things many many times. I inherited a course when I came to AU that did exactly that. I understand the fears that make people want to use exams but the exam is very seldom the answer to those fears. It's not so bad when the exam is in a more authentic setting and builds on coursework. At my previous institution we used to have all-day exams in a computer lab with full Internet access and the ability to talk to others, which built on coursework so everyone was doing something different. That's quite an expensive solution, however, and is still susceptible to impersonation and collusion fraud despite the presence of invigilators. It makes sense as a learning experience, though, because coding to deadlines and under high stress is a real-life problem most programmers face. 

    Jon Dron July 3, 2013 - 11:13am

These comments are moderated. Your comment will not be visible unless accepted by the content owner.

Only simple HTML formatting is allowed and any hyperlinks will be stripped away. If you need to include a URL then please simply type it so that users can copy and paste it if needed.