Developing an evaluation plan for an e-learning module to develop higher-order thinking skills and writing competences

2008-01-05 09:52:34

Laura Macháčková Henderson

This article presents the experience of developing an evaluation plan for an e-learning module designed to foster higher-order thinking skills and writing competences. The evaluation plan is under construction and while the author is not in a position to provide an example of good practice, this article hopes to share the experience of developing an evaluation plan for an e-learning module.


According to the literature, e-learning can provide opportunities for effective development of higher-order learning skills. Based on an interest in exploring educational opportunities for staff and students in teaching and learning for these skills, CZP UK is developing a one-semester e-learning module, “Critical approaches to globalisation: developing writing competences”. This module is being developed in the context of the European Commission funded project, Virtual Campus for a Sustainable Europe (VCSE). The goal of VCSE is to share experience in developing e-learning modules in the field of sustainable development and to open the modules to students from VCSE partner universities.

The students participating in the course will be at an advanced stage in their bachelor degree or at an early stage in their master degree, and will be based at universities across Europe and from a variety of disciplinary backgrounds. The module aims to foster skills in critical reflection, question formulation and multi-disciplinary and multi-cultural collaboration for developing and co-authoring texts. The module is currently under development and will be available in the spring semester 2008.

Purpose of evaluation

The goal of developing a module evaluation plan is to

Thus, the evaluation is conducted primarily for those developing the course and for providing evidence of good “value for money” to the funding body, but the students will be the main beneficiaries of the evaluation as the course developers will be able to adapt the course according to the information gained from the evaluation.

Evaluation process

Two phases of evaluation have been identified:

formative evaluation: conducted to obtain data from a pilot run of the course,
summative evaluation: conducted to measure the effect of developments resulting from formative evaluation and ascertain whether the module enables students to fulfil the learning goals of the course.

Time and other constraints will unfortunately not allow the pilot run of the module to occur in the same conditions as the first real life run of the module, but where appropriate it is planned to use the same methods and tools to allow for some, even if limited, basis of comparison.

Evaluation will be based on indicators addressing the following points:

In order to avoid feedback fatigue of respondents, we hope to vary the methods of evaluation rather than relying solely on online questionnaires, and ensure that the methods we use are targeted and concise. We aim to engage students in the process by informing them of the purpose of the evaluation activities and emphasising students’ potential benefit. There are five potential methods which we find interesting and potentially manageable within our resources:

The connections between proposed methods and the four areas of interest for evaluation are as follows:


Who is conducting the evaluation and what is their relationship to the participants?

This is an area of concern to address when planning the evaluation. There may be some evaluation methods which should be conducted by an external evaluator, i.e. not the course tutor (e.g. interviews), while there will be other methods which might be best conducted by the course tutor (e.g. textual data, quizzes and assessment of written work, confidence logs).


In thinking through the issues outlines above, the following resources were found to be helpful:

Evaluating e-learning, Centre for Academic Practice, University of Warwick, January 2004

Evaluating learning technology resources, Learning Technology Support Service, University of Bristol

Project Evaluation Toolkit Published by the University of Tasmania

Evaluation Cookbook published by the Learning Technology Dissemination Initiative

Flashlight Evaluation Handbook

Handbook for Learning-centred Evaluation of Computer-facilitated Learning Projects in Higher Education