One of my pet hates is the boring shiny store bought compliance e-learning, you know the kind – A page with a picture and 2 chunks of text and a next button, every few pages a handful of multiple choice questions that a monkey could answer to “check the learning”
Because of things like this I am not a big fan of the computer marked test (sometimes disparagingly called a “multiple guess test”) because it is something that is so easy to do badly.
I was going to cover computer tests later in the series but someone emailed me some fantastic research (Susanne Voelkel. Combining the formative with the summative: the development of a two-stage online test to encourage engagement and provide personal feedback in large classes. (2013) Research in Learning Technology 21: 19153) on the effect of tests so I am going to cover it now.
Feedback – The 2 key things about VLE/computer tests is that they A) mark themselves and B) Give instant feedback. The thing is that while A) will save you time you really really need to spend a lot of time on B) because when students have just got something wrong (or right) they are receptive to immediate feedback and you need to make sure that it is good and it deals with the mistake that they made – sadly yes this does mean you need to provide individual feedback for each wrong answer.
Motivation – as you can see from the research above, motivating students to take the test is key
- If you give them good tests with decent feedback students grades will improve, but not all students will take the tests (and thus we assume miss out on the benefits)
- If you make the test compulsory (ie counts towards the final mark) all students will take the test but you cant make the results available instantly, students don’t seem to get the same benefit from feedback if it is delayed.
- If you set a pre-test with feedback and students need to get a certain score before they can take the test that counts towards the final mark, then grades will improve and all students will take the tests. (most VLE’s will allow you to set this up – ask your friendly Learning Technologist how to do it)
Sadly again number 3 will take the most effort on the part of academics – but if you are going to do it, then you should do it properly.
Don’t over test your students – If there are lots of questions then students will not pay as much attention to individual question feedback, similarly if there is a big test every week students will lose interest.
Use the diagram to find X
If I was doing it I would have 1 or 2 big tests and the weekly questions I would ask in class using the flipped classroom method of Eric Mazur (watch this you tube for an idea of the kind of effect it can have in class)
Finally dont add questions that require a human to mark them in your VLE/Computer tests, because A) this will mean you spend time marking them and B) the feedback will not be instant and so will not be as effective.
More than just Content 
Joseph, Thank you so much for pointing me in the direction of the Voelkel article – it looks very useful. I am very interested in the points you make about the possible benefits of using VLE assessments which provide instant responses to students’ work, giving automated generic comments on their answers. One question arising from this which interests me is – is this really “feedback” as defined in the literature (eg in the REAP principles mentioned in ocTEL week 6)? Also, would students recognise this as “feedback”? (personally I think that the answers to both of these questions are “partly yes”!) Thanks again for an intersting posting, John Dermo
John – “partly yes”, I think you have answered your own question – it hits principle 3, 4, and 5 of the REAP Effective Assessment in a Digital Age (see page 15 of the pdf) and probably 12.
Students should recognise it as feedback, particularly if staff spend the time to make it good feedback. Also the logic underlying these automated testing is getting smarter, so will end up becoming more tailored to individual students.
However – Academics should make sure that there are other forms of assessment and feedback on their course (to pick up the other 9 REAP principles)
Pingback: Does weighing a pig make it smarter? #octel | More than just Content