Thursday, 19 September 2013

The Difficulty of Literature Reviews

One of my worst nightmares when facing the PhD viva was that my examiner would point out some important piece of literature relating to my work that I have missed. I dreaded the question
 'I see that you have not referred to the works of XXX?'
or putting bluntly
'Why have you not referred to XXX in your thesis?'.
Luckily my examiners did not ask me this dreaded question.

Photo credit: Erik Abderhalden

Doing a literature review is difficult because you need to make boundaries and find relevant literature and not miss out on important work. This is where your supervisors help you by informing you key people in the field you should read about. But what if you still miss some important work?

Well there are several strategies you could go about:
1. First of all you can stop doing any literature reviews because anyway you will have to leave some work out of it - there is no way one can find all work and refer to them all (at least this is what I believe)
2. Secondly - you can go ahead putting your best effort and still if you fail to address some important work you can accept that fact if someone raise the issue (that is what my plan for PhD viva was)
3. Third - when you have finished your literature review, but you are yet to publicize it, if you find that there has been some new important papers published you could always acknowledge the fact that these papers have been published since you've finished your work
4. You could also do what you want and call it a literature review regardless of whether it refers to important work or not.

When I was doing MOOC systematic literature review (published in IRRODL) I tried my best to find all scholary literature I could find. Even then there are criticism that we have not included some work. But looking at Massive Open Online Courses and Online Distance Learning Review published by the Department of Business Innovations and Skills it seems that they have missed the one and only systematic literature review on MOOCs published so far. This made me question the effort being put into this report because had correct terms used in Google this would have appeared in the top 10 hits.

On the other hand, perhaps the authors knew about this work (systematic literature review) but found it not useful so did not refer to it? A quality systematic review in a topic gives you access to almost all relevant papers published in the period concerned. In my view our MOOC Systematic Review filled a huge gap by collating research relating to MOOCs and summarizing/analysing them. But perhaps I am too attached to my work and may be considering our work too highly? Perhaps it is time to think again?


  1. When I last contributed to a systematic literature review it was very structured with an explicit protocol for which terms were used, sources examined, and criteria for inclusion/exclusion in the body of the paper. That at least answers "why wasn't xxx included". It does make the whole exercise very robotic and less creative though.

    Oh and I just looked for a systematic review of MOOC literature and your paper was in the #1 spot on Google :)

    1. Yes even after appearing as #1 in a Google search and its not referred to there must be something very wrong either in the paper or the person looking for information. As the author of the paper I would like to think latter.

  2. I have just looked at the newer version from @bisgovuk maybe the authors need to read up on literature reviews, there is an excellent book by Fink: Fink, A. (2010). Conducting Research Literature Reviews: From Internet to paper (3rd Edition ed.). Thousand Oaks, California: Sage.

    1. Luckily we had access to these sources. Must also acknowledge expertise from you and Andrew that went into the paper making it a worthwhile read for anyone interested in MOOC literature

  3. thanx now i have the link which i was looking for my research.
    12th five year plan education | mooc