Thursday 19 September 2013

The Difficulty of Literature Reviews

One of my worst nightmares when facing the PhD viva was that my examiner would point out some important piece of literature relating to my work that I have missed. I dreaded the question
 'I see that you have not referred to the works of XXX?'
or putting bluntly
'Why have you not referred to XXX in your thesis?'.
Luckily my examiners did not ask me this dreaded question.

Photo credit: Erik Abderhalden http://www.flickr.com/photos/soundfromwayout/2468912536/sizes/z/in/photostream/

Doing a literature review is difficult because you need to make boundaries and find relevant literature and not miss out on important work. This is where your supervisors help you by informing you key people in the field you should read about. But what if you still miss some important work?

Well there are several strategies you could go about:
1. First of all you can stop doing any literature reviews because anyway you will have to leave some work out of it - there is no way one can find all work and refer to them all (at least this is what I believe)
2. Secondly - you can go ahead putting your best effort and still if you fail to address some important work you can accept that fact if someone raise the issue (that is what my plan for PhD viva was)
3. Third - when you have finished your literature review, but you are yet to publicize it, if you find that there has been some new important papers published you could always acknowledge the fact that these papers have been published since you've finished your work
4. You could also do what you want and call it a literature review regardless of whether it refers to important work or not.

When I was doing MOOC systematic literature review (published in IRRODL) I tried my best to find all scholary literature I could find. Even then there are criticism that we have not included some work. But looking at Massive Open Online Courses and Online Distance Learning Review published by the Department of Business Innovations and Skills it seems that they have missed the one and only systematic literature review on MOOCs published so far. This made me question the effort being put into this report because had correct terms used in Google this would have appeared in the top 10 hits.

On the other hand, perhaps the authors knew about this work (systematic literature review) but found it not useful so did not refer to it? A quality systematic review in a topic gives you access to almost all relevant papers published in the period concerned. In my view our MOOC Systematic Review filled a huge gap by collating research relating to MOOCs and summarizing/analysing them. But perhaps I am too attached to my work and may be considering our work too highly? Perhaps it is time to think again?