I've referred people to this work by Heinz Dreher many times over the past few years but usually forget where the original paper resides, so here is a bookmark to this interesting research. It provides a useful literature review that covers some of the earlier work on automated free-text assessment from the 1960s onwards but, more interestingly, reports on Heinz's own work on a system that helps markers and students by giving primitive but useful formative feedback.

I'm also fond of an older paper from the 1990s by Dave Whittington and Helen Hunt that compares a number of methods and suggests high correlations between human and automated markers, available at http://www.cs.washington.edu/education/courses/cse590D/04sp/papers/whittington-hunt.htm

In all instances it is, of course, pretty easy to game the system once you know the algorithms it is using. However, as a sanity and consistency check, automated systems are a useful tool.