Running to catch up I had some thoughts about last weeks #OcTEL, there was an interesting paper looking at some of the project that had failed and drawing 6 critical success factors for mobile web projects.
- The pedagogical integration of the technology into the course and assessment.
- Lecturer modelling of the pedagogical use of the tools.
- Creating a supportive learning community.
- Appropriate choice of mobile devices and web 2.0 social software.
- The need for technological and pedagogical support for matching the unique affordances of mobile web 2.0 with social constructivist learning paradigms.
- Creating sustained interaction that explicitly scaffolds the development of ontological shifts, that is the reconceptualisation of what it means to teach and learn within social constructivist paradigms, both for the lecturers and the students. The use of a structured and sustained intentional community of practice around each project was found to facilitate these ontological shifts.
Now we were meant to look back at one of our projects and use this to evaluate the key failures and successes of the project. I didnt do this!
Instead I looked at the 6 factors and thought, I think I can do better than that or at least explain it in a way that is easier to understand. A year or so back I developed “Gliddon’s Hierarchy of Technology Enhanced Learning”
You can work your way up the levels in this and use it to determine what the potential risks are to your project (just add them to your risk matrix as you consider them).
In week 3 of Octel we had to look at a few tools for content creation and say how they might be used. I chose Xerte because my Uni is looking at making this tool available to all staff in the near future.
Here is a brief video of my thoughts
The things that impressed me most with Xerte are the quality of question types which do allow you to provide something a bit more intellectually challenging than a multiple choice question and that the finished product looks very stylish without people having to spend lots of time getting it right.
In having to look at how a particular person would react to the concept of Learning Machines I decided to look at Illich http://donaldclarkplanb.blogspot.co.uk/search?q=Illich with his ideas for Deschooling Society and replacing it with learning webs based on 4 learning networks.
One of those learning networks was a Reference Service to Educational Objects, so could a teaching machine be considered an type of educational object? yes I think it could and if they were simply limited to that role then I think Illich would approve.
Here is a video of the “teaching machines” in action
For a modern version of the teaching machine try this https://www.oppia.org/explore/2 I think you will find that its a good example of using a “teaching machine” to teach you a concept. It is likely that access to this kind of object as well as books, articles and videos would be considered a good collection of Educational Objects.
The points where I think Illich would disapprove are firstly everyone sitting in school in rows working away at their own machine (their only autonomy the speed at which they progress through the defined curriculum), he would have wanted them selecting their own machine diets at a time and place of their own choosing. The second main point is that (I get the strong impression that) Skinner in the video appears to advocate that the machine learning will be all that a student needs to learn successfully rather than just one component (Illich has 4 learning networks in comparison!)
When the King of Macedonia wanted his son (Alexander the Great) educated he simply hired the cleverest man in the world. Sadly not only does this solution not scale very well it is considerably more expensive than most of us can afford (for a more in depth look at this idea see “What Would Socrates Think About MOOCs?”).
Machines on their own will not be sufficient. However as a tool used by competent teachers then learning machines can improve the ability to teach much in the same way that computers and humans are much better at chess than either humans or computers on their own (as detailed in this book).
For a more in depth look at the idea of teachers selecting particular tools to meet particular needs you might want to look at this blog post http://intelligentideation.wordpress.com/2014/05/07/1-3-teaching-machines-octel/ worth reading for the Gringotts philosophy.
In summary I think that Illich would approve of teaching machines as part of the solution but that a real education would be more complicated than that. This is a point I often come across, humans tend to like a simple answer or a silver bullet to solve their problem which leads to over hype (ie MOOCs will solve all the problems of education) but for complex systems there is almost never a simple answer and education is one of the most complex systems that exists. In education, its always more complicated than that.
In the final week of #octel we looked at evaluation and feedback and this was also a chance to look back at the course and see what we had learnt and if we had answered our big question.
Looking back over my Blog posts I feel that I have learnt a fair amount and have crystallised a few thoughts that I was playing around with, however looking back over the course materials I am not sure that what I learnt was always what the course had planned to teach me.
My question is – is that particularly a bad thing? and then with brilliant timing John Graves posted to the octel g+ group this speech from Stephen Downes, which reassures me that no its probably not a bad thing.
Looking back to my Big Question – “How do we get staff to engage with and create a culture of TEL that improves learning” have I come up with an answer? Well no I haven’t but I would be surprised if I did, I expect that question is likely to be my life’s work. I do however think I have a few more fragments of the answer, so I will take that as a success.
As for evaluating the octel course itself, was it a success? What about MOOCs in general how can their success be measured. We were given a nice ocTEL badge to say how brilliant we are, but was that what I took the course for?
I wonder if there is a danger in mistaking the qualification for the learning?
This is not so serious when its a course taken for interest such as ocTEL, but for a lot of education the stakes are much higher, as you will see from this article Riot after Chinese teachers try to stop pupils cheating, valuing the results much more than the process can cause problems. This article by Professor Sugata Mitra about Allowing the internet in exams would sidestep some of the cheating issues and is more likely to measure what students can do (as a result of their learning). So is the success the network – going back to the Downes speech earlier, I would say it probably is. In which case ocTEL was successful for those that engaged, we formed a community and learnt a bit and then moved on (some brief engagement and brief learning, some longer engagement and more learning – most with some level of success).
This is a fairly rambling post, but that is because I had a number of ideas bouncing around left over from the course – and that (to me) is probably the most important success.
“The greater danger for most of us lies not in setting our aim too high and falling short; but in setting our aim too low, and achieving our mark.”
― Michelangelo Buonarroti
As you have probably guessed I have gone “off piste” again this week. The #octel mooc is looking at project work, success/failures and risk management and how particularly when technology is involved you need to plan and plan well. Now I must admit that if you are unfamiliar with project planning then this will be really helpful to you – however at my University we are all over this topic, I do this stuff all the time.
In fact as an institution we are reaching the stage where our requirements and committees and approval routes are having a negative effect on our ability to be “Agile” and move at the speed required to keep current with modern technology as well as raising the cost of our projects.
So I am going to put the opposite argument and say we should put our effort into doing more things and expect some things to fail. Rather than putting so much effort into avoiding failure that we can only deliver a few things.
This TED talk explains why that is now the correct way to do this.
Oh – you should still do some project planing, so you know what it is you are trying to achieve! Also this is Education and thus affects peoples future, so if what you are doing is going to get people killed or cause them to fail the course – do avoid those risks! But if its a risk like a lesson going wrong – do it, Jump!
I should point out this is not a criticism of my institution – all institutions that are like us (big, existed a long time, successful, good reputation) will have similar problems. In fact we are taking steps to avoid the problem with an innovation fund so that we can just do stuff because it looks like a good idea. I want to call it the “It might fail, fund” but sadly a more sensible name will probably be chosen.
I was looking at the #octel video on the Saylor Foundation and my first thought was – oh there are no lecturers at all.
This made me realise that a fear that academics have been telling me about for years has finally happened. To learn more we conducted this interview with a frightened academic …
We had to write “elements of the … approach that you think could be applicable” and “problems you might anticipate with the approach” now I am a learning technologist and for the things I actually teach then Sue has already written an excellent post.
So instead I am going to think about how the approach would work for academics at the University I work for. To simply offer what Saylor offer; a well designed course of materials with no academic contact would not be acceptable to our students (particularly with no certification).
However used as part of a course I think the approach has potential. A well designed course which uses open content where possible that students can work their way through would work well with the flipped classroom model. Our students could study the readings and then discuss them and work on problems in lecture time (see Eric Mazur for an idea of how this works). Putting it online and sharing “CC by” would also be good for sharing, the academic community and the reputation of the University (I would have to check how happy the Uni would be with this approach!).
The value added of a University is the contact with brilliant minds (the lecturers and other students) how much better is that contact if it is a conversation rather than a boring monologue. Content can clearly be found on the net for free, so simply providing content will not be sufficient (even if the person providing it is brilliant – because there is also brilliant stuff for free on the net).
One of my colleagues told me they were planning a session on question design for students who had opted to create eLearning materials as part of their course. As we were looking at assessment and feedback on #octel this week, I asked them if they could send me their session.
Roberta was kind enough to record a quick video going over her session, explaining both what good question design involves and how she planned to deliver the session to the students.
Roberta Perli – Question Design
This week for #octel we need to”list and critique up to three different assessment approaches available on Technology Enhanced Learning courses”
The 3 I am going to use are
- Online tests with automatic feedback
- Online Essay submission with online marking
- Online peer assessment
Online tests with automatic feedback – done right these can be very useful, make sure students get the feedback as soon as possible. I talked a lot about these in my last post so I wont go into more detail here. One thing I missed was the importance of designing your questions well – I hope to feature a guest post from my teams assessment expert which covers how to design questions (post should appear by the end of the week). Also as the technology gets better the feedback can become more detailed and personalised (see this post about the automatic generation of personalized reporting)
Online Essay submission with online marking – at the really simple level if you stick to the same form as you would with paper (ie a student submits an essay, you write comments on it) then online has 2 advantages,
- Its more convenient for students to submit, they can “hand it in” from anywhere with an internet connection rather than physically going to an office
- Students (and staff) can look back at earlier essays done by the student and see feedback etc rather than searching through old paperwork for previous essays and feeback
However there are a number of advantages to online marking, if you use technology to be creative, this old case study from my departments website shows how use of audio and video feedback helps students.
Online Peer Assessment – you can do this offline but the paper shuffling is problematic and it can take much longer as students will have to physically hand over work. You do need to be careful online as well because having some students not do the inital essay can have repercussions further along the process. However it will ensure the students engage with the learning process giving them multiple passes over the material
- First pass – Writing the essay
- Second pass – reading the rubric/marking scheme the lecturer provides
- Third to Fifth pass – reading, considering and marking other students work (I like to ask them to mark 3 essays)
- Sixth pass – reading marks and comments from other students
- Seventh pass – reading what the lecturer thinks of the marking they did
- Eighth pass – reading the final marks of the lecturer on their work
This compares very well with the 2 passes done in a traditional “write essay/get marks” assessment.
So what about the smart pig? Well its a reference to the quote “Weighing a pig doesn’t make it heavier” which is often trotted out when someone wants to criticise testing and say it doesn’t help learning.
Now it is true that a lot of students get tested to within an inch of their lives, and sometimes it seems that the test are simply so that someone can make league tables of schools, check the teacher is doing their job etc. I will avoid going off into a rant about this and instead simply say – when you are testing your students, make sure they are learning from the experience, not just doing a test to get a mark.