New technology in higher education: understanding the innovation process

Peter Goodyear
Centre for Studies in Advanced Learning Technology, Dept. of Educational Research,
Lancaster University, Lancaster, England, LA1 4YL
March 1998
Invited keynote paper for the International Conference on 'Integrating Information and Communication Technology in Higher Education (BITE)', Maastricht, March 25-7 1998.

Abstract
This paper argues for a re-examination of our established ways of trying to understand ICT-based innovation in higher education. It suggests that a project-centered, rather than an environment-centered, approach to understanding innovative developments has obscured some key elements of what has been happening in higher education practice in recent years. It also makes it harder for us to carry out the complex co-ordinated tasks that are necessary for the productive design and management of learning environments in higher education, and to share our experiences and learn from each other in systematic ways. The paper outlines a case for an 'ergonomics of learning environments' ¬ an applied science through which we can both understand grassroots innovation and help design and manage better learning environments. Taking seriously the actuality of our students' work (a necessary precursor to the ergonomic approach) causes us to reflect on the nature of students' activity as apprentice knowledge workers. The paper offers an interpretation of some core purposes of learning and teaching in higher education by examining some aspects of knowledge work, and in particular by finding a central place for the development of epistemic fluency.

Introduction
In this paper I want to look back over 30-40 years of experience in the use of computers and communications technology in higher education to ask a number of related questions:
i) Why do we always seem to be on the point of moving from innovative projects to mainstream application?
ii) What are we trying to achieve through greater use of ICT in learning and teaching?
iii) What intellectual resources are available to us in trying to understand and promote better use of ICT in learning and teaching?
iv) What do we still need to know?

In sketching answers to these questions, I want to begin to develop the following argument. First, that there are two main forces shaping the actual use of ICT in higher education. On the one hand, we have waves of programmatic investment in technology-based innovation. Each of these waves has its own lifecycle. Each wave gains energy from ignoring some of what has happened in the past. Few waves have the energy to succeed in disseminating their innovative work such that it can secure a sustainable place in mainstream higher education practice. On the other hand, we have a steadily accelerating investment in the ICT infrastructure for learning and teaching (and for other academic and administrative purposes) - an investment which is both substantial and, in general, not justified on educational grounds. Associated with this growth of ICT infrastructure, we see a burgeoning use of ICT by students and staff alike, for a variety of purposes, only some of which seem to be aligned with what educational technologists believe to be the central benefits of ICT. Secondly, I want to argue that we are moving closer to a common view on what higher education is for. We need a common language for discussing educational aims and processes and for designing and managing learning environments. I want to suggest that we have opportunities now, which were largely unavailable to us in the last 30 years, for using a consensual view about higher education to articulate our beliefs about valuable educational processes and to plan the resources and infrastructure needed for their support. Thirdly, I suggest that we are held back by the lack of a high level conceptual framework within which we can locate and make sense of the various developmental activities in which we engage (as designers and managers of learning environments). A number of candidate frameworks exist, but I argue that none is adequate to our needs in higher education today. I try to remedy this by suggesting that we might best make progress by at least temporarily setting aside the claim that there is anything very special about the use of ICT in higher education. We can and should learn some lessons from the world outside academia. My argument concludes with some suggestions about what we stand to learn and how we might best start to learn it.
At the centre of my view about how we need to reconceptualise the design and management of learning environments in higher education is an idea which is coming to be called 'the ergonomics of learning environments'. In short, I argue that we should take seriously the idea that students in higher education are just another kind of knowledge worker. Consequently, we should try to design technology which is appropriate to their actual work rather than technology which embodies our teacher/managers' beliefs about what students should be doing. It is perfectly appropriate for us to have a view about what students should be doing. But we need to know what they actually do, in order to support them with better technology. Given how close are our relations with students, it is surprising how little we know about what they actually do. This opens up an intriguing and relatively unexplored territory for further research.
Innovation in higher education
In recent years it has become common for the more strident spokesmen of government and industry to chastise higher education for its conservatism, for its adherence to tradition and to outmoded and privileged practices. If one believed these spokesmen (they are usually men) then one would carry in one's head a vision of ivory towers, of elite leisurely institutions for the rich and underemployed - of institutions which have not learned to live in the hard, competitive world of global capitalism. I want to give no room for complacency. Nor do I believe that universities have yet found a way to become fully aligned with the needs of an open democratic society. However, the picture of elite, unchanging institutions populated by dusty, ineffective, unaccountable teachers simply will not do. Consider some evidence from the UK higher education sector.
1. In the last 35 years, the full-time student population has grown from 200,000 to over 1.1 million. In 1961, less than five per cent of UK school leavers went on into higher education. In 1997, that proportion had risen to 33% for the UK as a whole and to 45% for Scotland and Northern Ireland. .
2. This growth has been accompanied by a fall of 40% in resources per student in real terms. The increase in productivity in the UK HE sector has been three times the increase in productivity in the UK service sector.
3. The organisation of teaching methods has also been changing profoundly. Barnett (1997a), for example, has pointed to substantial lines of innovation around (a) the incorporation of work-based elements in students' learning (b) increasing use of study service, (c) a broadening of the forms of learning experience within the core curriculum, including action learning, experiential learning, role play, group learning, etc, (d) explicit attention to the development of transferable skills, (e) giving learners more autonomy in their learning, e.g. through the use of learning contracts, resource-based learning, etc. Silver (1998 ) talks about 'commitment to change' in even the most traditional of UK universities.
4. And in relation to ICT, there is evidence of substantial growth in academics' use of new technologies. In 1992, over a quarter of academics were using some form of computer-based learning in their courses (Laurillard et al, 1993). By 1997 this had risen to between 50 and 60% (Casey, 1997, table 2.10).

While the changes to be observed in UK higher education may not be uniformly found across Europe, I would want to argue that, in many European countries, the forces which have been stimulating change are very similar. Much of what has been happening is associated with global shifts from elite to mass higher education systems, increasing accountability and deteriorating per capita resources for the support of learning and teaching (Scott, 1995). This may seem an inauspicious background for innovation, yet it seems evident that energetic, ambitious and entrepreneurial staff within higher education are responding to adversity in innovative ways.
A vital source of support for such innovation comes in the form of programmes of investment organised at national and institutional levels. In the recent past in the UK, there has been a three phase national investment in the development of innovative ways of using ICT in universities: the Teaching and Learning Technology Programme (TLTP). TLTP is the largest coherent programme of investment in learning technology for higher education in the world. TLTP was launched by the then Universities Funding Council in February 1992. Phase 1 of the programme funded 43 projects at a combined cost of £22.5 million over the three year period 1992/3 to 1994/5. Phase 2 of the programme funded a further 33 projects at a combined cost of £11.25 million over the three year period 1993/4 to 1995/6. Transitional funding of £2 million was made available at the end of each Phase in order to allow some projects to move on to a self-supporting status. (35 of the 76 projects have been awarded transitional funding and are implementing their business plans.) Investment has also been made in the establishment of a Teaching and Learning Technology Support Network (TLTSN) based around nine institutional projects. In addition to the investment made through TLTP itself, the institutions involved in the various aspects of the programme, and the staff of those institutions, have made very substantial commitments of staff time and other resources. Phase 3 of TLTP is getting underway in the spring and summer of 1998. Some £10.5 million is expected to be available for projects, running between 1998/9 and 2000/1. The majority of these projects will be implementation-oriented and will seek to increase the use of existing TLTP materials rather than create new materials. TLTP aims 'to make teaching and learning more efficient by harnessing modern technology'. Its objectives include a commitment to widespread dissemination and take-up of TLTP products and their integration into courses and other forms of support for student learning. There is also a serious concern that all areas of the HE sector, all subject disciplines and all staff and students should reap the benefits of TLTP. Almost all of the Phase 1 and Phase 2 projects have delivered their products and are at or very near to completion.
A key question for TLTP concerns the extent of take up of its products by academics who have not been directly involved in the development projects. A major evaluation of TLTP carried out four years after the start of the programme (Coopers & Lybrand et al, 1996), reported that it was premature to gather data on take up. A study of the extent of take up of Phase 1 and Phase 2 products is currently underway (Spring 1998). We therefore have, as yet, very little systematic data on TLTP's success in dissemination. While it has directly funded the innovative activity of hundreds of staff, we simply do not know how far it has impacted on mainstream practice.
A concern about the dissemination of innovations beyond the core team who are directly funded to develop new resources is, of course, a recurring preoccupation for funding bodies and other concerned commentators. In the early 70's, the Carnegie Commission on Higher Education wrote:
'One of the great disappointments of the national effort to date is that for all the funds and effort thus far expended for the advancement of instructional technology, penetration of new learning materials and media into higher education has thus far been shallow' (Carnegie Commission on Higher Education, 1972, 47).
The UK National Development Programme in Computer Assisted Learning (NDPCAL), TLTP's forerunner in the 1970's, placed 'institutionalisation' at the heart of its agenda. NDPCAL was concerned that the projects it funded did not 'remain in the laboratory' but were actually rolled out into mainstream use in the institutions concerned. According to Hooper (1977), this could be said of 27 of the 35 projects funded under the programme. Involvement of staff other than those directly funded by the projects was also significant. NDPCAL projects directly funded about 100 staff. By the end of the programme (1976/7), a further 690 teaching staff were also involved in the use of NDPCAL materials (Hooper, 1977, 20-23). Probably about half of these were working in higher education . In the last 20 years then, the numbers of teaching staff in UK higher education making some use of computer assisted learning has risen from around 400 to between 45,000 and 55,000 (a hundred-fold increase).
What we do not know, of course, is much about
(a) the extent to which these teachers are using ICT in learning and teaching,
(b) how they are using it
(c) whether they are making much use of materials and other resources generated by development programmes like TLTP.

For example, we know from Casey's large scale national survey (1997) that of academics who have been teaching in the same university for the last five years, 59% claim to be making more use of multimedia than they were five years ago, and 51% claim to be making more use of 'interactive coursework'. From Callender's complementary survey of students in UK higher education, we know that 48% of students claim to have made some use of computer-based learning packages (Callender, 1997, Table 3.1). Among the things we don't know from these surveys is whether the use of ICT in learning accounts for 1%, 10% or 50% of an average student's learning experience. From an educational technologists point of view, it would be nice to know what kinds of learning resource are bundled up under the headings of 'multimedia' or 'interactive coursework'. Are students typically using resources which reflect the best of our pedagogical knowledge or are the resources low in interactivity and weak on pedagogy?

I want to suggest that, in the absence of firm quantitative evidence on these key matters, there are reasons for being cautious in assuming that programmatic investment in ICT for learning and teaching is delivering the goods. While failing to produce data on take up, the Coopers and Lybrand evaluation of TLTP did manage to review a range of TLTP Phase 1 & 2 courseware. From their observations, it is fair to infer that:
a) the best of the 76 projects have created materials in which there is very lively academic and commercial interest, but
b) the majority of projects have not drawn on existing knowledge about educational innovation, learning technology, pedagogy, HCI or courseware development methods (including methods of managing complex projects). For reasons which are not clear, a number of projects learned very little from the past, or from their own on-going challenges. In consequence they have created (at best) a core of TLTP products which are unlikely to enjoy wide take-up. The evaluators were particularly dismissive of a genre of courseware which appeared to have been characteristic of many TLTP projects - that of the electronic book or hypertext - which was characterised by low interactivity and shallow approaches to study.
Failure to learn from the past can sometimes be a desideratum for a new investment programme. If one pauses to consider the dynamics of the process through which the high-level decisions are made about whether or not to fund such a programme, it becomes clear that it will often be an advantage to portray the programme as a radical break with the past. 'More of the same' is not the kind of vision which persuades the guardians of public finance to open the purse strings. Two further elements can be added to this diagnosis. First, there is a high turnover in staff associated with educational technology development work. (There may be more longevity among educational technology researchers.) Most of the staff who do the development work on projects which are funded by periodic investment programmes are employed on short fixed term contracts. When the programme of investment ends, most of the staff have to find jobs somewhere else. By the time the next investment programme is approved, they see no advantage in returning from what may be a secure well paid job in industry to a more poorly paid fixed-term job in academia. Consequently, the possibilities for a continuous evolution of experience and know-how, within the field, become severely constrained (c.f. Goodyear, 1995). Secondly, there is some evidence of a deep scepticism about pedagogy among some of those more technically-minded academics (senior and junior) who have been active or influential in and around programmes such as TLTP. John Slater (1996), for example, refers to 'open hostility' and a role as 'lifelong Cassandra' occupied by some educational specialists. (See also Gardner, 1996).
My claim, then, is that some of the dynamics of programmatic funding militate against a continuous maturation of the field and against sustainable development. This goes beyond the obvious point that start-stop funding is no basis for continuity of effort. Rather, programmatic funding (a) attracts effort and attention towards apparently novel 'poles' of development and away from areas which had previously been the sites of accumulating expertise, and (b) creates new walls around a set of funded activities which inhibit interaction with centres of knowledge outside those walls.
I also want to make a secondary claim, that much of the evident growth in use of ICT in higher education has relatively little to do with the success or failure of programmes like NDPCAL or TLTP. Such programmes may colour the background against which growth in use of ICT is set ¬ such that most knowledgeable people in UK higher education can refer to the running of TLTP materials as a legitimate means of using ICT. But this is very different from saying that ICT is being bought primarily to run TLTP and other courseware. What then, is the underpinning rationale for higher education's very substantial ongoing investment in ICT?
Sustainable innovation
UK higher education is currently spending about £1 billion per year on ICT, or about 10% of its total turnover (see Table 1).
Table 1: Expenditure on ICT in UK Higher Education (Source: NCIHE (1997) Table 13.1 )


Million pounds
Central initiatives 60-70
HEI central spend 200-250
HEI departmental spend 400-550
HEI overheads 100-160
HEI courseware 20-50
Total 780-1080


This colossal sum is being found, in times of considerable financial adversity, without any clear or convincing educational rationale. As yet, only a minority of universities have an overarching ICT strategy which firmly links ICT investment to institutional goals (NCIHE, 1997, Chapter 13). Few universities have adopted methods of business process analysis, let alone business process re-engineering, to see how ICT may be used to obtain significant gains in efficiency or effectiveness (Ford et al, 1996). Rather, it has to be said, universities are buying and using ICT because it is there and because cognate areas of activity outside academia are buying it. Academic staff are 'knowledge workers' par excellence, and if knowledge workers need a desktop PC and Internet access, then so do academic staff. By extension, so do students. Recommendation 46 of the Dearing report (NCIHE, Chapter 13) says that universities should plan on the basis that all students will be required to have their own portable PC by the year 2005/6 and that student access to more highly powered networked workstations will need to be improved from the current 15:1 to around 8:1or 5:1 within the next 2-3 years.
The investment in ICT to date has not been driven by an unmet demand for computers on which to run the kinds of courseware being produced by TLTP. If anything, the reverse is true. When asked why they were making greater use of multimedia in their teaching, between 40 and 50% of academics said that it was because the technology was available (Casey, 1997, Table 2.11). Fewer than 30% said the main reason was that it would bring benefits to students.
It can be argued that the main driver for increasing use of ICT in higher education is that ICT has become a ubiquitous technology in the world of work. Higher education is just another workplace. Academics and students are just plain workers. It is not that a case has been proven that ICT can make higher education more efficient or effective (Hopkins, 1996). Rather, for those working in higher education to be denied access to the defining technology of the information age is simply unthinkable.
On this account, what is sustaining (and what is likely to continue to sustain) higher education's investment in ICT is not the winning of arguments about the relative efficacy of new educational technologies; rather, continuing innovation in the ICT infrastructure will be sustained by much more powerful forces shaping the acquisition and use of ICT in the world outside academia. Those of us who are concerned about how universities can make good use of ICT in support of student learning can draw some comfort and an important lesson from this fact. Whether or not we are able to demonstrate radical improvements in student learning on the basis of students' use of ICT, it is probable that universities (and students) will continue to invest in ICT. (This is a special comfort to those who have accepted Clark's argument that we can never demonstrate that improvements in student learning are attributable to media - see, for example, Clark, 1983, 1994.) The associated lesson is that we can probably predict the main ways in which students will use ICT on the basis of the main uses of ICT in other workplaces. That is, students will want to benefit from ICT by using it to wordprocess essays, reports and dissertations; to communicate using email and conferencing; to prepare presentations; to gather information using on-line databases and the WWW; and to learn to use the 'tools of the trade' of their academic discipline or intended profession.
This begs the question of whether there is any legitimate role for educational technologists - should we try to claim that we can make a distinctive and valuable contribution to helping improve students' learning or should we retire gracefully and let the currents of the mainstream of ICT sweep students where they will?
Before I try to answer this question, I want to take an important diversion to examine what I believe to be a growing consensus about the purposes of higher education. We cannot answer a question about the possibility of improving learning without being clear about what that learning consists of, and what it is for.
Purposes
Roughly speaking, there are three main interpretations of the purposes of higher education: the traditional academic, the vocationalist and the critical.
The traditional academic view
The traditional academic view (when it can be caught in the open, in these instrumentalist times) places value on the disinterested acquisition of academic knowledge. It holds that the acquisition of knowledge and understanding is intrinsically good and justifiable, whether or not that knowledge can subsequently be used to take action in the world beyond the university. Students are asked to become competent in academic discourse, with its heavy reliance on declarative conceptual knowledge, contemplative forms of analysis and use of textual (including mathematical) representations (Barnett, 1997). Implicitly or explicitly, it acts as if the aim were to induct students into the work and world of the academic and their discipline. Though its position as the dominant model for higher education is now seriously contested, it nevertheless underpins much of what goes on in educational technology. Diana Laurillard's influential book on Rethinking University Teaching: a framework for the effective use of educational technology (1993), for example, pays no sustained attention to other conceptions of what university education might be for. The main kind of learning outcome associated with this conception is the ability to recall declarative conceptual knowledge and deploy it in the construction of arguments, or in the solution of problems more generally. Laurillard draws our attention to a peculiar characteristic of academic knowledge: that it is 'articulated' knowledge, consisting of other peoples' descriptions of the world. This 'second order' quality distinguishes it from much of the 'first order' knowledge which we acquire through direct experience of the world. Laurillard holds that much of what cognitive psychology can tell us about learning is restricted to the acquisition of 'first order' knowledge through direct experience in naturally occurring environments. Cognitive psychology says little about learning in the artificial environments which higher education must construct in order to help students learn the descriptions of the world devised by others, and to help them in the processes of interpretation, reflection and exposition (Laurillard, 1987, 199-200; 1993; 28). I will return to this distinction shortly.
The vocationalist view (or the generic competence and employability view)
Through a variety of means and bodies, employers are pressing higher education to attend more closely to what they claim to need in the new graduates they wish to recruit. These demands may include the kinds of specialised technical knowledge acquired by some students on some courses but increasingly they refer to generic competencies (otherwise known as core skills or transferable skills). Frequently mentioned generic competencies include literacy, numeracy, communication, foreign language, leadership, teamworking and IT skills (e.g. Harvey & Mason, 1996; Assiter, 1995).
Lee Harvey's Quality in Higher Education survey data provide some of the most accessible up-to-date evidence on what U.K. employers feel they need, and are getting, from recent graduates (see e.g. Harvey & Mason, 1996, 17-23). In summary, employers:
• rarely rate specialised knowledge as a key factor determining whether they will hire a recent graduate, since this knowledge is unlikely to add enough to the organisation's expertise to affect its competitive edge, is liable to rapid obsolescence and is rarely in a form which the graduate can apply in the solution of important work-related problems
• value graduates' intellectual flexibility, powers of logical analysis, ability to conceptualise issues rapidly and to deal with large amounts of information but believe that graduates are insufficiently innovative, in part because they are insufficiently sensitive to the organisational implications of innovation
• feel that graduates are generally better able to deal with some of the basic requirements of working in a modern organisation (being dependable and highly motivated) than others (e.g. coping with pressure, managing time)
• believe that graduates are usually naive about organisational politics, industrial relations, knowing how to deal with people of different seniorities, and at recognising other peoples' motivations
• believe that graduates lack tact and are arrogant, especially in their relations with non-graduate working colleagues; though they are often excellent teamworkers
• comment very negatively about graduates' communication skills, especially about listening skills ('hearing what is meant as well as what is said'), oral presentation skills and the range of writing skills (especially the difficulties they have in writing persuasive cases). Employers did not rate IT skills as particularly valuable (29th of 62) and were generally very satisfied with graduates' IT skills (4th of 62).
Summarising this data and the results of similar studies of employers' expressed needs, Harvey and Knight (1996) conclude that organisations which recruit graduates are looking, above all else, for transformative potential. That is, they want new graduates entering their employ to have the capacity to transform their organisation, not merely to enhance its productivity and competitiveness along current lines. Elements of this transformative potential can be discerned in the six bullet points above and include willingness to learn; ability to deal with change and question assumptions; analytic, critical and problem-solving skills, as well as the knowledge and ideas (a 'fresh creative mind') brought to the organisation (Harvey & Mason, 1996, 14). But employers are clearly worried about graduates' difficulties in coming to terms with what might be called the culture of a modern organisation, such as organisations characteristic of growth sectors in a modern, knowledge-intensive, globally-oriented economy. I shall return to this point later, but for now I want to flag the idea that work in organisations, especially where much of that work is oriented to information, knowledge and communication (Boden, 1994), places some particularly difficult learning demands on new recruits and that some of these demands share some of the attributes which Laurillard ascribes to academic knowledge. In particular, learning other peoples' descriptions of the world is (I want to claim) an important element in both 'worlds'.
Critical being and reflexivity
This third conception is best articulated in the writing of Ronald Barnett (e.g. 1997a; 1997b). Rejecting 'academicist' and 'operational competence' conceptions, Barnett looks to a higher education 'fit for the 21st century'. He argues that individual reflexivity ('the capacity to go on interrogating one's taken-for-granted universe') is necessary for dealing with an essentially unknowable modern world. Higher education needs to respond by:
• supporting the student in their acquisition of discursive competence: offering a deep understanding of some discursive realm and an insight into what it is like to handle with confidence the concepts, theories and ideas of a field of thought, to handle complex ideas in communication with others
• encouraging self-reflexiveness: by framing the student's initiation into a field of thought such that they see its essential openness and how they may be actors in it
• encouraging informed but critical action: understanding the power and limitations of the field as a resource for action (Barnett, 1997a, 22-25).
A key part of Barnett's argument rests on a postmodernist conviction that we can have no certain knowledge of the world, and that consequently knowledge and skills become redundant or marginal (1997a, 29).
Towards a consensus on purpose
All three of these conceptions give an important place to declarative, conceptual academic knowledge: whether as a body of knowledge to be acquired for its own sake, or as a setting in which to develop transferable skills. One cannot learn to communicate without having something to communicate. One cannot be reflexive without having something to be reflexive about.
Beyond that, we see agreement about action in the world: whether constructing a logical and informed argument, solving the problems of professional practice, applying discursive knowledge in exploiting strategic organisational opportunities or in advancing one's social projects.
Indeed there are a number of convergences in these conceptions which tend to be obscured in their various critiques of each other. In particular, I want to draw out the theme of social and epistemological reflexivity, especially as it is coloured by what I shall refer to as contingent knowledge.
First, recall Laurillard's claim about the distinctiveness of academic knowledge: that learning in higher education involves the internalisation of second-order experience. It involves learning other peoples' descriptions of the world. Learning the economic theories of (say) Adam Smith and Karl Marx, or the structuralism of Althusser and Levi-Strauss involves a level of detachment from direct experience and contemplation of the 'real' world which (one can argue) distinguishes this process of learning from the process implicated in learning how gravity affects the flight of a ball which one is throwing on a cricket field. Now, my point is not to reject Laurillard's claim, but to extend it – to say that learning other peoples' descriptions of the world is not unique to academic learning but is also part and parcel of learning how to thrive in a complex, modern organisational culture. I acknowledge that some aspects of learning to be effective in a work organisation involve processes which are close to learning from direct experience of the world (as is the case in some kinds of valued learning within higher education - such as learning to use a piece of laboratory equipment or to drive a software package). But much of what distinguishes a novice from an expert worker in a complex organisation, I would claim, is an understanding of aspects of organisational culture – including knowledge objects created in and around the organisation, which are at best only loosely grounded in the 'real world'. An important class of such objects is what might be called the 'organisational fiction'. The imperatives of efficient, co-ordinated activity within complex organisations create a need for consensual and often implicit simplifications of such things as organisational goals, procedures and resources, or of interpretations of factors in the organisation's external environment, such as competitor behaviour. Such 'organisational fictions' have a strange relation to what we conventionally think of as truth and falsity. They are 'true', in so far as one normally needs to act as if they are true if one is to engage unproblemetically and productively in the day-to-day work and discourse of the organisation. That is, they are part of the taken-for-granted world in which effective action is set: though they need to be learned and they can be questioned. But their constructed status, their artificiality, their 'fictive' quality, is often made explicit. This gives to organisational fictions a status of being both true and false (if viewed from a relatively naive epistemology). A more sophisticated epistemology allows one to see that, from time to time, and task to task, one needs to make use of knowledge whose status depends heavily on context ('for the purposes of this project, it's important for us to act as if x was the case') and to understand the origins and power of those contextual factors ('the project manager believes that the client will accept that y meets the specification in this contract; to do more than y will eat into our profits).
In short, I am claiming that among the generic competencies which graduates will find most valuable if they are to succeed in modern, knowledge-intensive organisations, is a social and epistemological reflexivity which supports them in learning and using organisational fictions and other forms of contingent knowledge.
As a corollary, I would claim that the forms of experience necessary to developing such a reflexive capacity and disposition are not so far different from what either Barnett or Laurillard have in mind. Where I differ from Barnett (1997), is that I would insist that contingent knowledge of the kind I have described can still be central (not marginal), truth-like (not arbitrary) and complex enough to require significant effort in learning. It is all very well to insist on the production of critical graduates, but those graduates who cannot use organisational fictions or spend some of their time taking them for granted and getting on with the job will not progress far. Their transformative potential will remain just a potential.
Taking the argument back to the discussion of academic learning, the point is to recognise that students need to be helped in (a) coming to understand the ideas, propositions, theories, etc within any particular theoretical perspective, but also (b) to understand (in general terms) how that perspective took shape – indeed, to understand what a theoretical perspective is, to understand how theories and other ways of organising ideas within academia are generated, evolve and die. In part, an understanding of the sociology of knowledge production can help them acquire what might be called 'meta-theoretical' perspectives: viewpoints from which they can see how academic workers construct knowledge within their fields of enquiry.
Given the increasing dominance of knowledge work, and knowledge intensive organisations, in late modernity, a convergence between some of the imperatives of academic knowledge production and the exploitation of knowledge in the commercial world is to be expected (Scott, 1995, 140ff). I am arguing that we can capitalise on some of this convergence in shaping a higher education experience which promotes an integration of social and epistemological reflexivity.
Implications for learning
There is insufficient space here for pursuing the implications of this view across a representative range of educational opportunities. I will have to restrict myself to one clarificatory example, drawing on the work of Allan Collins. Morrison & Collins (1996, 109), referring to the multiplicity of ways in which it is possible to know about things in the modern world, advance an argument for 'epistemic fluency', which they describe as 'the ability to identify and use different ways of knowing, to understand their different forms of expression and evaluation, and to take the perspective of others who are operating within a different epistemic framework' .
Morrison and Collins introduce two associated constructs: epistemic forms and epistemic games. Epistemic forms are target structures which guide enquiry (e.g. lists, stage models, hierarchies, systems dynamics models, axiom systems). Epistemic forms are structures which help us construct meaning from the jumble of sense experiences. Epistemic games are 'sets of moves, constraints and strategies that guide the construction of knowledge around a particular epistemic form' (p108). Epistemic fluency is 'the ability to recognise and practice a culture's epistemic games, with their associated forms (target structures), goals, plans and strategies' (p114).
I find the idea of epistemic games appealing, because it offers a bridge between a somewhat abstract concern for epistemological reflexivity, on the one hand, and day-to-day educational practice, on the other. If we believe that epistemological reflexivity (as a stance) or epistemological fluency (as a practical accomplishment) are important, how can we help foster them? Through getting students to 'play' epistemic games.
'…you learn how to play [each] game … simply and only by playing these games with people who are already relatively more fluent than you are - and who, crucially, are willing to gradually pull you up to their level of expertise by letting you play with them.' (Morrison & Collins, 1996, 114).
We have space to pursue two implications of this idea a little further: how we might identify a core set of epistemic forms and games, and how we might see ICT underpinning a more productive engagement in epistemic games.
Table 2 lists 16 epistemic forms and games. Not all would seem to be of immediate application to every academic discipline. However, they are suggestive of how one might move towards a broad-based depiction of the variety of epistemic forms and games characteristic of any one community of practice (whether that community be academic or not). They also begin to suggest how one might conduct some useful investigative work in and around the learning experiences of one's students. A cognitive anthropology of learning environments would, inter alia, seek to uncover the dominant epistemic forms and games encountered by our students and to compare these with the practices of academics, and of knowledge-workers outside academia.
Table 2: Epistemic games (source: Morrison & Collins, 1996, 111)
Compare-and-contrast game
Spatial analysis game
Cost-benefit analysis game
Primitive elements game
Table or cross-product game
Hierarchy or tree structure game
Axiom-system game
Critical event analysis game
Cause-and-effect game
Problem-analysis game
Form-and-function game
Systems dynamics game
Aggregate behaviour game
Trend and cycles game
Situation-action game
Constraint system game

Stellan Ohlsson (1996) also draws on the idea of epistemic games in his inspirational account of how we ought to approach the construction of a scientific account of understanding. Ohlsson believes that cognitive science has scored a major success in its creation of an account of skill acquisition - of 'learning to do'. He believes that this success is largely attributable to a research strategy, outlined by Newell and Simon (1972), in which one first seeks to model the agent and only then seeks to model learning (as change in the agent). A vital element in a model of agency is a structural analysis of relevant types of task. In moving scientific attention from competence to understanding, a key part of the challenge is to identify relevant types of task (Ohlsson, 1996, 47-51). Ohlsson provides a list of epistemic tasks, drawing, in part, on the work of Collins. Table 3 reproduces this list of seven epistemic tasks, which Ohlsson claims to be complete (1996, 51).

Table 3: Epistemic tasks (source: Ohlsson, 1996, 51)
Epistemic task
Describing To fashion a discourse referring to an object or event such that a person who partakes of that discourse acquires an accurate conception of that object or event
Explaining E.g. in relation to some event, to fashion a discourse such that a person who partakes of that discourse understands why that event happened
Predicting To fashion a discourse such that a person who partakes of that discourse becomes convinced that such and such an event will happen
Arguing To state reasons for (or against) a particular position on some issue thereby increasing (or decreasing) the recipient's confidence that the position is right. Arguments are about what to believe and what to do
Critiquing (evaluating) To critique a cultural product is to fashion a discourse such that a person who partakes of that discourse becomes aware of the good and bad points of that product
Explicating To explicate a concept is to fashion a discourse such that a person who partakes of that discourse acquires a clearer understanding of its meaning
Defining To define a term is to propose a usage for that term.

Ohlsson's epistemic tasks, or Collins' epistemic forms and games, give us a way forward in linking specific learning activities to what I have claimed is emerging as a core consensual purpose for higher education – the fostering of epistemic fluency. Since we are specially interested in ICT, we can take another step forward. I want to claim that we primarily need ICT to support students in their engagement in epistemic games. ICT can do this in at least two ways. First, ICT can provide procedural facilitation for the use of important epistemic forms. Once we have identified the most important epistemic forms (for a course, discipline, profession or for commercial knowledge work), we can adapt existing ICT tools to support the use of these forms, and create libraries of examples, templates, etc. which students can learn to reuse (c.f. Pemberton et al., 1996). Secondly, we can use communications and collaboration technologies to allow students to participate in communities of practice, where the members of that community have differing degrees of experience in the playing of epistemic games (c.f. Lave and Wenger, 1991; Morrison and Collins, 1996). Such communities may include just students, or students and academics, or students and experienced professionals. ICT makes such networking far easier than it has been in the past. Both of these strategies argue for the adaptation of mainstream ICT rather than the creation of specialised courseware products as the primary site for investment.
An intellectual framework for designing and managing learning environments
I now want to address two related questions:
• given the changing and increasingly complex nature of higher education learning environments, what intellectual resources are available to those of us with professional responsibility for their design and management?
• how can we share those intellectual resources, so that there is the possibility of learning from one another?
The design and management of learning environments
The term 'learning environment' is very widely used yet rarely defined. The term has at least two main usages within the educational research literature. One connotes something rather small scale and relatively self-contained (as in a simulation-based learning environment). The other applies in a more holistic and macro-level way, connoting the totality of objects within which learning is situated and/or the totality of resources on which learners can draw. In this paper, I am primarily interested in the second, more holistic, of these usages (c.f. Noel Entwistle's writing about the factors influencing university student learning (e.g. Entwistle, 1996), Diana Laurillard's analysis of the peculiar characteristics of learning in 'unnatural' – that is, artificially constructed – environments (e.g. Laurillard, 1993), and also writing about the design of formal education settings generally (e.g. Greeno, Collins and Resnick, 1996, 26-33).
A distinction is sometimes made between the physical and the psycho-social learning environments (e.g. Fraser et al, 1992). This distinction needs some closer scrutiny if we are to be sure what we mean by 'learning environment'.
The physical environment consists of material, non-human things. These things can be relatively simple, such as a chair in a classroom, or relatively complex, such as a library building. The science of ergonomics has traditionally concentrated on the design and usability of relatively simple physical things.
According to Fraser et al (1992) the psycho-social environment consists of other human beings (singly or as grouped entities) and, by extension, their activities, including their discourse. The use of the term 'environment' in the writings of Entwistle and colleagues has more of the psycho-social than the physical in it.
Three points need to be made here. First, classing all other people as part of the environment is a peculiarly individualistic or psychologistic stance. I would argue for construing the learning environment as a place in which a community of learners does its work (c.f. Wilson, 1996, 5).
Secondly, a person's perception of a learning environment is not the same as the learning environment itself. I do not want to open a debate about mentalism:realism or about distributed cognition at this juncture. I want to make the point that we need to distinguish between the complex entity whose design and management concern us (i.e., the learning environment) and the variable, personal beliefs about that learning environment held by its inhabitants. These two are sometimes conflated. It is precisely because students vary in the ways that they interpret the environment that we need to maintain this distinction.
Thirdly, the incursion of ICT enriches, and increases the importance of, the physical environment. A process of reification translates human interaction into objects. For example, tutorial guidance which may once have been given in an informal and relatively unstructured manner by a teacher to his or her students becomes reified in an on-line study skills booklet. Or face-to-face discussion becomes mediated through electronic mail. Or the expository lecture is recorded on videotape or translated into an electronic book. As education ventures further into the use of open, distance and flexible learning techniques, so the physical environment of computer-mediated communications and networked learning resources displaces face-to-face interaction and reifies its content.
This brings us towards a definition. I want to define the learning environment as a complex set of nested structures which provide the physical setting for the work of a community of learners. This physical setting can include all sorts of learning resources, including what we conventionally think of as hardware and software but also other knowledge objects produced through interactions between members of the learning community. In contrast to writers like Entwistle, I do not include the diffuse atmospheric signals emanating from the way teachers, schools or academic departments present themselves to their students. I do, of course, agree that attention needs to be paid to the congruence between the tasks teachers set, the signals they give, the activities students undertake, the learning environment and its technology.

In trying to apply techniques from business process re-engineering to the design and management of university learning environments, Ford et al (1996) identified a number of key business processes, business objects and actors. A surprisingly large number of actors, with different roles, need to work together in the design and management of learning environments, especially where the learning environment has a significant ICT component and/or where rapid change needs to be managed. Figure 1 is a simplified representation of some of the main interactions and processes. Few of the members of Ford's team felt that their own institution managed these interactions and processes at all well. Academic departments would often engage in innovative developments without informing the library or information systems services. Institutional senior management would fail to link ICT planning with academic planning. Innovation most often required insulation from the broader institutional setting, rather than integration with mainstream activities and processes.

Figure 1: Designing and managing learning environments
































While such fragmentation in the design and management of learning environments is inexcusable now, it is likely to create insurmountable problems as the pace of change in higher education accelerates. Ways need to be found, urgently, of reducing fragmentation and supporting the development of effective co-operation and partnership between the various actors who have responsibility for the learning environment.
Intellectual resources for the design and management of learning environments
The design and management of learning environments, however fragmented, is a responsibility which weighs down hundreds of thousands of academic staff in the universities of Europe. In most international professions, there are methods of sharing experience and of abstracting general 'lessons learned' from particular events. Academics are very accustomed to doing this in their work as researchers, less so in their work as teachers. Few academics read the specialist literature on learning and teaching in higher education. Even if they did, it is hard to see exactly how the articles in that literature might directly inform their approach to teaching. Paradoxically, this is especially the case with those articles reporting fundamental research on student learning. It can be extremely hard to derive implications for one's own practice from leading edge research on student learning.
Of course, it is not just the academic teachers who need to play a part in designing and managing learning environments. As we have seen, many other kinds of staff are also involved. Some of these staff are even less likely to read the literature on learning and teaching, seeing themselves primarily as specialists in ICT or the management of library resources, and reportedly finding the language and literature of pedagogy impenetrable.
I want to argue that a major obstacle to the systematic sharing of experience is the lack of an appropriate framework for organising our thinking about what we are trying to achieve and for reflecting on, and describing, our progress. A scientific account of student learning can play a part in this, but I would argue that (a) it can only play a small part and (b) it is a misleading model for the kind of intellectual framework we need.
Frameworks and models
If we assume, as many have, that the primary gap in our practitioner knowledge comes from an inadequate scientific account of student learning, then it makes sense to invest more effort in fundamental research on student learning and in the dissemination of the results of that research. I do believe that we should be investing more of our national and international research budgets in work on student learning: exciting and useful progress is being made. But I feel that this needs complementary investment in research on the whole gamut of problems which need to be solved if we are to make significant improvements in our ability to design and manage learning environments. We do not yet understand how multidisciplinary teams of people in universities can best collaboratively design and manage an environment which is properly supportive of students' various activities. Given how badly we handle some of what appear to outsiders to be fundamental issues of environment design, it is sometimes a wonder that our students learn anything at all.
If we want to be able to learn from each other it is necessary to have an appropriate intellectual framework against which we can structure our experiences and render them mutually intelligible. A number of models exist, on which we can draw in identifying a usable framework. Each has something to contribute, but none, I argue, fully meets our needs. I shall review each of these models very briefly and locate them in the three-dimensional space portrayed in Figure 2. The three dimensions are meant to capture variations in setting, development method and information-gathering method. For conventional as much as logical reasons, parts of this three dimensional space are under-populated.
Figure 2: Project-oriented models
















Roughly speaking, there are three kinds of setting: experimental, quasi-experimental and naturalistic. Table 4 summarises some of their key features, especially with respect to their potential contribution to our collective understanding of innovative developments.

Table 4: Variations in setting
Experimental Quasi-experimental Naturalistic
Site Artificial (eg a psychology or usability lab.) Control group may be in natural site; experimental group in artificial Natural site
Students Specially selected subjects Usually 'normal' students, selected on a random or matched-pair basis Students who are taking their normal course
Researchers control over student tasks/activities High Medium Low (pedagogical rather than experimental control)
Data produced Usually focus on outcome measures; sometimes student satisfaction measures; sometime usability measures Usually focus on comparative outcome or satisfaction measures Usually restricted to outcome measures which draw on students' normal assessment work and students' end of course evaluation ratings; other measures may be too disruptive
Links between data and theory, hypotheses, design features, etc Tight Looser Often problematic
Generalisability of findings to 'real world' practice Often problematic because of the artificiality of the events studied May be easier, but still some significant problems (eg Hawthorne effects) May be problematic because of the uniqueness of the events
Ease of rolling out the innovation into real world practice Hard May be easier Roll out is part of the study process


Information is gathered by and about innovative developments in a variety of ways. Three easily recognisable approaches are experimental, summative evaluation and formative evaluation (Figure 2). In the experimental approach, everything is driven by the goal of gathering valid and reliable data which enable the experimenter to reject (or not reject) a hypothesis. Any developmental activity, such as creating a computer assisted learning program, is subservient to this hypothesis-testing and theory-building goal. This approach is normally deployed in an experimental or quasi-experimental setting. In the cases of both summative and formative evaluation, development rather than information-gathering is the primary goal. Summative evaluation is carried out at the end of a developmental process, in order to gather information about the success or otherwise of the developed product, resource or teaching method. It may be possible to link the findings of summative evaluation back to the theory or design decisions which (in some sense) underpinned the development - though this is often quite a problematic undertaking (Draper, 1997). This makes the findings of summative evaluation quite difficult to share, other than at the crudest level of whether the innovation failed catastrophically or not. Formative evaluation is carried out at key points during the developmental process in order to inform and improve the development process and improve the quality of the product. Draper (1997) uses the term 'integrative evaluation' to describe a formative evaluation which focuses on the integration of an innovative product (such as a CAL program) into its context of use. Formative evaluation tends to be very local in its scope: it is primarily concerned with, and is often at its best when sharply focussed on, a specific innovative product or activity. Again, the possibilities for generalising the lessons learned through formative evaluation can be quite limited, because of this pre-occupation with local concerns.
On the 'development methods' axis, two influential models are worth describing: linear instructional systems development (ISD) and iterative development (otherwise known as rapid prototyping). ISD has been the dominant paradigm for the development of complex instructional systems, especially in the USA, for many years (Tennyson, 1995; Braden, 1996). In its canonical form, it favours a linear derivation which works from an analysis of intended learning outcomes, through high level instructional design, detailed design, implementation (e.g. the authoring of a CAL program), testing and embedding. Various commentators have pointed out the difficulties with this model, not least that it often entails a high risk of producing an unusable product because checks on the validity of the understanding of requirements are left too late in the process. This has motivated the adoption of variants of the iterative development approach, in which a prototype (e.g. a mock-up of the interface to a CAL program) is produced as early in the project as possible, in order to validate and refine the project team's understanding of user requirements. Neither the ISD not the iterative development approach necessarily produces information (about the innovation concerned) from which those outside the project team can learn. Much depends on the forms of information gathering that accompany the innovation process (e.g. formative or summative evaluation activity). In principle, ISD with good formative evaluation can lead to more systematic accumulation of evidence about the quality of the various sub-processes and interim products involved. For example, it can cast light on better ways of doing detailed design work. The less disciplined activity normally associated with rapid prototyping does not always leave a clear trace of the team's thinking, from which others may learn.
I believe that most of the innovative projects described in our literature can be located in the space depicted by Figure 2. Indeed, the pragmatics of innovation and the gatekeeping work of journal editors and referees probably reduce the densely populated space down to two areas: quasi-experimental studies involving a cut-down linear ISD process and naturalistic case studies involving iterative development and formative evaluation. (Regions A and B on Figure 2, respectively.) Typically, innovative projects of type A stay in the laboratory, while those of type B require a protective niche within which they can be isolated from the complex constraints and processes of their larger host environment. As an international professional community, we can certainly learn something from both, or indeed all of, these kinds of projects. But we are missing two aspects which I believe are central to an understanding of large-scale change in higher education. The bulk of published work on ICT-based educational innovation has nothing to say about:
• the ecology of a university's learning environment(s), or
• how we might best engage in the complex, coordinated work activity which is entailed in designing and managing a university's learning environment(s) in a systematic way.
Our professional preoccupation with isolated innovative R&D projects has blinded us both to the real scale of change in universities and to the needs for addressing some of these institution-wide or system management issues. I now turn to an alternative framework which I believe can offer some ways into this increasingly urgent problem.
Towards an ergonomics of learning environments
Ergonomics is the study of the relationships between workers and their environment. I want to argue that we should:
i) shift our perspective from seeing students as empty knowledge containers to be filled by transmissive teaching, to seeing them as apprentice knowledge workers who learn through increasingly sophisticated and confident participation in communities of practice,
ii) see that the organisation of learning communities around key sets of epistemic games becomes our primary educational task,
iii) undertake open-minded enquiry into how our students actually do their work
iv) identify ways in which the learning environment (including its ICT) can be restructured and improved so as to provide better support for students' actual activity.

I believe adopting an ergonomist's perspective forces us to focus on the actuality of students' work, and not on some idealised view of how that work should be carried out. In this, I follow the French ergonomist Alain Wisner (1995, 597), in distinguishing between 'the prescribed work (the task) and the real work (the activity)'. This simple distinction, though a commonplace observation in everyday life, is often overlooked – whether in the implementation of computer systems in offices or in the design of supportive technology for student learning. The lessons are clearly being learned in the mainstream of information systems development and software engineering, as the growth of participative and user-centered methods testifies (eg Norman & Draper, 1986; Carroll, 1995). It is time that we sought to apply these insights in higher education. Of course, one can argue that it is permissible for teachers to insist on an identity between task and activity: to insist that students follow a prescribed route. While this may be valid in some contexts, it is clearly inappropriate for contexts in which we want students to take greater control of their own learning activity – whether this be through self-managed independent or group learning, through home-based study using conventional learning resources, or in some kind of 'virtual' classroom.
Implicit in what I have said so far is a distinction between the design of tasks and the design of technology. I think that these two have suffered, in the short history of educational technology, by being insufficiently distanced from each other. I need to explain this paradoxical view. In what might be called the traditional approach to task design and technology design, a set of beliefs derived from learning theory and knowledge of subject matter shapes the specification of instructional tasks. These task specifications are then embedded in instructional technology. The software directs and supports the learner in their response to the task. There is a tight link between task design and technology design: indeed, the main purpose of the technology is to present the task, and (perhaps) to assess the learner's response to the task. There is no clear conceptual separation between the prescribed work (the task) and the learner's actual response (their real activity), because there is an implicit belief that it is illegitimate for the learner's activity to diverge from the task.
What happens if we allow some conceptual separation between task and activity? The resources of learning theory and a knowledge of subject matter can still be used to help define worthwhile tasks. Thus we can continue to benefit from the achievements of cognitive science. And these resources are also used reflexively. Whereas a simple model of instructional design, or of instructional systems development (ISD), would suggest that task definition flows by a straightforward process of deduction from a learning needs analysis and the tenets of instructional design, a reflexive approach to ISD insists that some understanding of how students are likely to respond to tasks-as-set can and should influence the process of task definition. More striking, though, is the recognition that the main source of requirements and constraints for technology development come not from a specification of the learning task but from an understanding of students' actual activity. In this way, technology is built to support the actual needs of students in their work as learners, not to reify the task definitions of their teachers.
Finally, the ergonomic approach also encourages us to take a holistic or systemic approach to understanding the learning environment. This is part of the force of the 'environment' metaphor. It insists on the inter-relatedness of things, and emphasises the dangers of assuming that processes and their products will have only limited interactions (c.f. Hannafin & Hannafin, 1996). An ergonomics of learning environments necessarily takes a systemic approach: whether trying to understand how learning activities are proceeding in a learning environment, or trying to design and manage learning environments. A systemic approach emphasises the importance of attending to consistencies between activity and environment, between what students need to do and the technology that helps them do it.
Summary
In this paper, I have argued that we need a more clearly articulated high level framework against which to understand innovations in the use of ICT in higher education. Our dominant models (hardly frameworks) are project oriented, not environment oriented. Because of this, we tend to see the world from the viewpoint of those involved in innovative projects and their problems are foregrounded in our collective consciousness. Thus, on the one hand we lament the lack of take up of the results of these projects, yet on the other we have been blinded to a massive change in the use of ICT in higher education. An environment oriented framework is necessary if we are to gain a comprehensive understanding of real change and to coordinate the complex tasks which are involved in the proper design and management of learning environments. I have argued that higher education at the start of the 21st Century needs to draw strength from the growing importance of knowledge work in complex organisations and to recognise and build on convergencies between the use of knowledge in and outside academia. One way of translating this high-level aim into practice is through the idea of fostering epistemic fluency. Finally, I suggested that we stand better chances of learning from each other's innovative work if we demystify the use of ICT to support learning, casting the challenge instead in terms which would be very familiar outside higher education. We need to know what our students, as knowledge workers, actually do, and to design supportive technology with and around them. We need an ergonomics of learning environments to provide a framework for design, management and the communication of our experiences.
References
Assiter, A, ed, (1995) Transferable skills in higher education, Kogan Page, London..
Barnett, R (1997a) Towards a higher education for a new century, Institute of Education, University of London
Barnett, R (1997b) Higher education: a critical business, Open University Press, Buckingham.
Boden, D (1994) The business of talk: organizations in action, Polity Press, Cambridge
Braden, R (1996) The case for linear instructional design and development: a commentary on models, challenges and myths, Educational Technology (36,2) 5-23
Callender, C (1997)Full and part-time students in higher education: their experiences and expectations, Report 2 of the NCIHE (1997)
Carnegie Commission on Higher Education (1972) The fourth revolution: instructional technology in higher education, McGraw Hill, New York
Carroll, J. M. (1995). Scenario-Based Design: Envisioning Work and Technology in System Development. John Wiley & Sons, New York
Casey, B (1997) Academic staff in higher education: their experiences and expectations, Report 3 of the NCIHE (1997)
Clark, R (1983) Reconsidering research on learning from media, Review of Educational Research, (53) 445-459.
Clark, R (1994) Media will never influence learning, Educational Technology Research and Development (42) 21-29
Coopers & Lybrand et al (1996) Evaluation of the Teaching and Learning Technology Programme, Higher Education Funding Council for England, Bristol. (Summary available at http://back.niss.ac.uk/education/hefce/pub96/m21_96.html )
Draper, S (1997) Prospects for summative evaluation of CAL in higher education, Association for Learning Technology Journal (5,1) 33-9.
Entwistle, N (1996) Recent research on student learning and the learning environment, in Tait, J & Knight, P, eds, The management of independent learning, Kogan Page, London, 97-112
Ford, P, Goodyear, P, Heseltine, R, Lewis, R, Darby, J, Graves, J, Sartorius, P, Harwood, D, King, T. (1996) Managing change in higher education, Open University Press, Buckingham.
Fraser, B et al (1992) Assessment of the psychosocial environment of university science laboratory classrooms: a cross national study, Higher Education (24) 431-451
Gardner, N (1996) Avarice versus ideology: a perspective on TLTP, Active Learning (4) http://www.cti.ac.uk/publ/actlea/issue4/gardner/index.html
Goodyear, P (1995) Infrastructure for courseware engineering, in Tennyson, R & Barron, A. (eds) Automating Instructional Design: Computer-Based Development and Delivery Tools, Springer Verlag, Berlin, 11-31
Greeno, J., Collins, A. & Resnick, L. (1996) Cognition and learning, in Berliner, D, & Calfee, R., eds, (1996) Handbook of Educational Psychology, Simon & Schuster Macmillan, New York, 15-46
Hannafin, K & Hannafin, M (1996) The ecology of distance learning environments, Training Research Journal (1) 49-70.
Harvey, L & Knight, P (1996) Transforming higher education, Open University Press, Buckingham
Harvey, L & Mason, S (1996) A quality graduate, in Tait, J & Knight, P. eds, The management of independent learning, Kogan Page, London, 13-28
Hopkins, J (1996) Information technology and the information society in Europe: expectations and barriers to the exploitation of new media in the higher education and research sector, http://www.eaie.nl/ITHE/iteurope.html
Laurillard, D (1987) The different forms of learning in psychology and education, in Richardson, J et al., eds, Student learning: research in education and cognitive psychology, Open University Press, Milton Keynes, 198-207
Laurillard, D (1993) Rethinking university teaching: a framework for the effective use of educational technology, Routledge, London
Laurillard, D, Swift, B & Darby, J (1993) Academics' use of courseware materials: a survey, Association for Learning Technology Journal (1,1) 4-14
Lave, J & Wenger, E (1991) Situated learning: legitimate peripheral participation, Cambridge University Press, Cambridge.
Morrison, D & Collins, A (1996) Epistemic fluency and constructivist learning environments, In Wilson, B, ed, Constructivist learning environments, Educational Technology Press, Englewood Cliffs, New Jersey, 107-19
National Committee of Inquiry into Higher Education (1997), Higher Education in the Learning Society (the Dearing Report), HMSO, London http://www.leeds.ac.uk/educol/ncihe/
Norman, D & Draper, S, eds (1986 ) User Centered System Design, Lawrence Erlbaum Associates, Hillsdale NJ
Ohlsson, S (1996) Learning to do and learning to understand, in Reimann, P & Spada, H, eds, Learning in humans and machines, Pergamon, Oxford, 37-62
Pemberton, L, Shurville, S & Hartley, T (1996) Motivating the design for a computer assisted environment for writers in a second language, in Diaz de Illarraza Sanchez, A & Fernandez de Castro, I, eds, Computer aided learning and instruction in science and engineering, Springer, Berlin, 141-8
Perry, W (1968) Forms of intellectual and ethical development in the college years, Holt, Rinehart & Winston, New York.
Scott, P (1995) The meanings of mass higher education, SRHE/Open University Press, Buckingham
Slater, J (1996) The impact of TLTP, Active Learning (4) http://www.cti.ac.uk/publ/actlea/issue4/slater/index.html
Tennyson, R (1995) Instructional systems development: the fourth generation, In Tennyson, R & Barron, A (eds), Automating instructional design: computer-based development and delivery tools, 33-78
Wilson, B (1996) What is a constructivist learning environment?, p3-8 in Wilson, B, ed (1996) Constructivist Learning Environments, Educational Technology Press, Englewood Cliffs, NJ.
Wisner, A (1995a) Understanding problem building: ergonomic work analysis, Ergonomics (38,3) 595-605