Whenever I’m asked how to get started with research in open and distance learning, I have a one-word answer: PREST. Practitioner Research and Evaluation Skills Training (PREST) is a comprehensive set of resources produced by the Commonwealth of Learning and the International Research Foundation in Open Learning, and it is simply the best there is.

I was so keen on PREST when it first emerged that I wrote a review that was published in Distance Education (27, 1, pp. 123-128, 2006), and the rest of this post is my review. I don’t imagine that it’s reached a vast number of readers in the intervening years, so I’m posting it in the hope that those seeking guidance will hone in on this gem. And in case you want to put it online for your own organization, a Moodle version has been added.  … and did I say that it’s open access? Yes, this 1000+ page wonderful resource is free.

By the way, if the publishers of Distance Education have any objection to this action (from my limited understanding they won’t), just let me know and I’ll take it down.

Anyway, here’s the review (the pics weren’t in the original). Sorry it’s a bit long, so if you don’t need convincing just go straight to the materials – ‘PREST on!’ …

Pressed for Time? Try PREST!

First, I have to “declare interest” in relation to this review. I am an unashamed admirer of the Commonwealth of Learning (COL), and believe that it is an effective organization (especially given its small size—it “boxes above its weight”) that continues to make a genuine contribution to the advancement of distance education worldwide, especially for developing countries. And yes, I’ve done some work for them as a consultant over the years. I am even listed as a member of the Steering Group for these materials, even though my contribution was minimal.

You will not be surprised, therefore, that I am enthusiastic about some gems that have been recently added to COL’s treasure trove of resources in the form of the Practitioner Research and Evaluation Skills Training (PREST) series, published in collaboration with the International Research Foundation in Open Learning. As explained in the User Guide, the

materials aim to make available free research skills training in contexts with limited opportunities for staff development, and in particular, to help distance educators with limited research experience conduct principled and systematic research and evaluation.The resources have two objectives: to promote research and evaluation as a key part of professional development to be undertaken by all ODL practitioners rather than as the preserve of a privileged few; and to encourage ODL practitioners to become active producers rather than passive consumers of research and evaluation.

The core of the PREST materials is a set of six study modules, suitable either for self-study or for use in contexts where academic support is available. Judging by the suggested study hours for each module, the entire set of six would comprise a typical tertiary-level, one-semester subject (150 hours plus). The six modules are:

• A1: Doing educational research and evaluation in ODL
• A2: Planning research and evaluation
• A3: Getting and analysing quantitative data
• A4: Getting and analysing qualitative data
• A5: Mixed research methods
• A6: Reporting research to support or influence change.

The modules are presented in distance education format, and as such are exemplary. They conform to all that we know is needed for effective learning. Each module is divided into units, which in turn include the kind of format, devices, and structure with which we have become familiar: overviews, aims, learning outcomes, activities, feedback, references, readings, and so on. Mercifully, the readings are included, ensuring that the overall package is complete and obviates the need for any time-consuming pursuit of copyright clearances.

Unsurprisingly, I commenced my review by reading Module A1, prepared by Alan Woodley of the UK Open University. Woodley’s writing style is that of a mentor or critical friend, and this works especially well, ensuring that the materials are meaningful for both experienced researchers (try it—read and be reminded!) as well as for the novice researchers for whom the materials have been designed. I especially enjoyed the section, including the associated activities, in Unit 2 on the debate from Open Learning on what constitutes “research.” The feedback to the activities is thoughtful and incisive, and clearly not just thrown in as a design requirement. Another feature is the use of “pen portraits,” which comprise a series of seven small case studies of open and distance learning (ODL) practitioners and their research needs. Seven may be more than are really required, but they maximize the chances of having the reader identify with at least one of the characters and their contexts. Also of note is the attention paid to evaluation, which is bound to be high on the agendas of most emerging researchers.

Module A2, by Anna Robinson-Pant, continues in the same vein, dealing with planning aspects of research. As with the first module, the focus is on “doing,” with a strong mix of activities designed to both increase research skills and actively engage the reader in the process of developing a research proposal. Aspects of the case studies also carry through, helping to give the modules continuity and relevance. The activities work progressively through the steps in developing a proposal, so that in the final unit, readers are required to put the pieces together to form a complete proposal. In addition, the issues of funding and frameworks are not forgotten and sensible advice is provided on how to follow-up and monitor progress once the proposal is ready.

Alan Woodley is back again for Module A3, where aspects of quantitative data gathering and analysis are dealt with. Sensibly, the module does not delve deeply into statistical techniques, but remains focused and practical. A key feature is the inclusion of Microsoft Excel worksheets that form the basis of the bulk of the activities. In the interests of cohesion and continuity, one of the pen portraits from Module A1 is used as a case study to illustrate the rules of quantitative study. This module is necessarily quite long at 171 pages. It includes activities that encourage participants to undertake basic analysis of their own institutional data. The section in Unit 5 on the design and writing of good questions is especially helpful.

Module A4 moves on to the area of qualitative research, with Rob Walker as the principal author. As with Woodley’s units, Walker’s experience and skill shine through. This is evident in the use of “conversations” between a researcher and an adviser about, for example, the nature of qualitative research. The expected contents are included (use of case studies, interview skills, etc.), plus the bonus of a fascinating section in Unit 5 on visual research, a particular interest of the author. Justifiably (and you’ll have to read it to find out why), the topic of theory is left until the last unit, and it nicely rounds out and provides an underpinning for the overall material in this module.

Many researchers in ODL end up using mixed methods. Thus it is most welcome to find that Module A5 is devoted to this approach. Despite the admitted lack of quality exemplar material in the literature, the authors—Christine Spratt, Rob Walker, and Bernadette Robinson—have produced an excellent piece of work which would be appreciated by both expert and novice researchers. Theoretical underpinnings are briefly explored, and there is a nice summary review of quantitative and qualitative methods. The role of mixed methods in evaluation is also explored, including plenty of practical advice on developing an evaluation report.

Module A6 dovetails nicely with Module A5. It is on Reporting on research to support or influence change, which has been prepared by Janet Stuart and Colin Latchem. This is solid, sensible stuff that includes plenty of examples and activities to improve reporting skills. Particularly important is the final section on ensuring that the research effort isn’t wasted. This material both discusses how problems in implementation arise and how such problems can be overcome. As with a good proportion of the other units, there are some timely reminders for experienced researchers as well as for novices, including such general issues as bridging of the research–practice divide and more specific ones such as why the vice-chancellor doesn’t accept your research!

The core of the PREST materials (Part A) is supported by a further set of six handbooks on specific aspects of ODL research. These are:

• B1: Using programme monitoring in research and evaluation
• B2: Measuring outcomes
• B3: Cost and economics of open and distance learning
• B4: Programme evaluation and its role in quality assurance
• B5: Researching marginalised and disadvantaged learners
• B6: Researching tutoring and learner support.

The handbooks are structured similarly to the modules, and most are of about the same length and require about 20 hours of study. They also make use of the pen portraits introduced in the modules. The aim is either to provide follow-up material for those who complete the core modules, or to provide advice on a specific research area for those with at least a little research knowledge and experience. Admittedly, I have not studied the handbooks as closely as the core modules (there are, after all, 1,098 pages of material in total, not counting the readings!), but it appears that their potential value varies somewhat. B5 and B6 clearly cover areas that deserve particular attention, while B4 does not seem to add much to what has gone before and could perhaps be a little confusing for some readers, as its stance (e.g., on the relationship between research and evaluation) is at times at odds with the earlier modules. My view is that a shortened version could have been incorporated into B1. B3, on Costs and economics, is obviously a key concern for all researchers, and is a valuable inclusion. However, I wonder whether it should continue to be considered separately, or rather should cost concerns be built into all the materials so that researchers come to see this vital aspect of their work more inclusively.

Having provided a strong endorsement through this positive, though potted, overview of the PREST materials, do I have any negative comment? Yes of course, though these should be viewed more as minor quibbles than reflecting substantive issues. The edition I reviewed is “Version 1.0” and includes typos and errors which will no doubt be picked up and corrected over time (and isn’t it wonderful how much easier these can be done with electronic resources!). For example:

• Module A1, p. 57: Experiments or (not of) pilot schemes;
• Module A1, p. 63: two unnecessary bullets (6th and 10th);
• Module A2, p. 62: under the statement, “If you are preparing a proposal for funding, you may find the tips below useful,” there do not seem to be any “tips”; and
• Module A5, p. 40: in the list of stages, “collect the data” comes before “develop data collection instruments.”

These are all, of course, very minor quibbles. But while I’m on a roll, I might as well mention that although as a whole the layout and presentation is excellent, I am not a fan of mixing sans serif fonts (used for topic presentation) with serif fonts (used for activities). Correct me if I’m wrong, but my understanding is that serif fonts are easier to read for printed text, while sans serif fonts (such as Arial or Helvetica) are recommended for tables, diagrams, and reading onscreen.

The pen portraits are not used consistently throughout the materials—they seem to tail off somewhat in the middle of the core modules, then come back with a vengeance for the final one. This probably relates to the working relationship between the authors and coordinators/designers. Overall the coordinators/designers have done a great job in pulling it all together. They have created a structure within which, for the most part, the authors have been able to work. There will always be vague tensions between authors and editors, as the authors struggle at times to fit their own particular stance and approach to such a template. An apparent example is Rob Walker’s comment in the overview for Module A4, where he states that “[b]ecause of the limitations created by specifying objectives in behavioural terms, this module adopts a style and approach that is process-oriented rather than outcomes-driven.” This hints at a measure of discomfort with at least one specific feature of the materials. But overall, it seems that the project teams have worked together effectively, so that the general approach succeeds admirably.

What matters finally is not what reviewers in journals such as this have to say about these materials, but how they are used by the people for whom they are designed. It would be tragic, and I suspect bitterly disappointing to those who created them, if the PREST materials were ignored. So, if you have the opportunity, use these materials in your context; evaluate their effectiveness, and feed your critical comments back to the COL.

How might you do this? Well, naturally it depends on your context. If you work in a distance education setting and have no ready access to training in research, PREST provides probably the best learning materials that you are likely to find; if you know that you are a strongly committed independent learner who can study effectively in relative isolation, then the modules will provide you with a wonderful start to your personal research efforts; and if you have a close colleague with a bit more research experience who can discuss your studies and act as a critical friend, then so much the better.

If all this seems a bit daunting for you as an individual distance educator seeking to get more involved in research and evaluation, an alternative approach is to read the pen portraits (first introduced on p. 6 of Module A1), identify one that reasonably closely matches your circumstances and trace their “journey” through the materials. This is technically easy to do using the search function in Acrobat Reader (the software for reading PDF files). For example, suppose you find that you identify with the situation of and issues facing Kabir Shastry of the Open University of Udair, who wants to improve his research and publication record. You’ll find him mentioned a few times in Module A1, and then Module A6 includes consultant advice to Kabir on how to get into the journals.

More specific and potentially helpful is, however, the use of Kabir’s case in the first PREST Handbook (B1: Using programme monitoring in research and evaluation). Here, Neil Butcher skilfully traces a hypothetical investigation into the efficiency (or otherwise) of distance education systems, and introduces the stumbling blocks which typically face researchers such as Kabir, along with sensible ways of overcoming them and achieving realistic research goals. It is very instructive to follow Kabir’s journey (and do the relevant activities along the way!), especially if you find that you face institutional barriers in your efforts to undertake meaningful and relevant research. You may even end up using the example of Kabir’s case study to investigate the efficiency of your own institution!

If you are in the more luxurious position of being one of a group of distance educators, all of whom have an interest in research, you could think of forming a study group (something akin to a book club) to work through chosen parts of the materials collectively and collaboratively. Your discussions and reflections could be real (fortnightly/monthly meetings) or virtual (online discussion group or chat room). You would need to set specific goals to ensure your application, though, perhaps including the aim of eventual joint publication or research grant application.

The greatest good is most likely to be done, though, through systematic use of the PREST materials by open universities and research training bodies. The flexibility is there, through modularization, to use the materials in a variety of ways, from training workshops through to formal qualification as part of a distance education qualification. If you are putting together a programme for qualifying distance educators, then here is an opportunity to make your task significantly easier. If you are about to update some out-of-date research training materials, consider simply ditching them and pressing PREST into service. If your language of instruction is one other than English, then you might like to work collaboratively with the COL in translating PREST into, say, Spanish, French, or (and this could be potentially the most useful) Chinese.

Finally, as mentioned, once you’ve used the materials, evaluate their effectiveness (using the research skills you will develop) and send your feedback to the COL—they would love to hear from you and use your comments for ongoing improvements. Many of us have bemoaned the state of research in distance education; here’s a chance to do something positive to improve matters, especially in countries lacking the kinds of support that these training materials provide. Over to you!

photo credit: steven w via photopin cc