MS4 level being good citizen -imperative- (1) (1).pdf
Theory And Methodology In Networked Learning
1. Theory and methodology in Networked learning
Gráinne Conole, The Open University
Positional paper for the Networked Learning Hotseat debate, January 2010
Introduction....................................................................2
Definitions.......................................................................3
Networked learning researchers’ birth disciplines................6
The nature of theory .......................................................8
Theoretical perspectives dominant discourses....................10
...................................................................................11
Cultural Historical Activity Theory.....................................................................11
Communities of Practice...................................................................................11
Actor Network Theory........................................................................................12
Cybernetics and systems thinking.....................................................................12
Methodological approaches..............................................12
Content analysis................................................................................................13
Ethnography......................................................................................................14
Case studies......................................................................................................14
Action research.................................................................................................14
Evaluation..........................................................................................................14
Research questions and grand challenges in the field..........16
The challenges of interdisciplinary research in networked
learning and strategies for fostering interdisciplinarity......22
Case studies...................................................................23
1
2. Case study: The agile development of a social networking site for education,
Cloudworks........................................................................................................24
Case study: A design and evaluation framework..............................................25
References.....................................................................27
Introduction
This paper is intended as an initial position paper to spark debate for the
th
networked learning hotseat scheduled for the week beginning 18 January 2010.
Each section will focus on a specific theme around theory and methods in
Networked Learning and will conclude with some questions for discussion. Given
the changing nature and contested nature of this field of research the paper
begins by providing a definition of some terms of the terms discussed. This
paper draws on a number of sources of data:
th
• Outputs from a TLRP TEL workshop on interdisciplinarity on 14
November 2008. Participants were all researchers involved in the TLRP
TEL research programme.
• A cloudscape on interdisciplinarity
http://cloudworks.ac.uk/cloudscape/view/1954
• Relevant research literature
• It also draws on a special interest group led by Martin Oliver in the early
nougties on theory and learning technology, which resulted in the
production of a special issue of JIME (http://www-
jime.open.ac.uk/2002/9/).
These will be combined with the outputs from the Networked Learning hotseat
discussions to provide an update positional paper for discussion on the TLRP
2
3. TEL programme website (http://www.tlrp.org/tel/) as part of a theme on
interdisciplinarity in TEL research.
Definitions
Research into the use of technology in an educational context had a long history
with changing labels over the years, each indicating evolving trends in the field
and emphasising different types of foci of inquiry. Commonly used terms include:
educational technology, learning technology, e-learning, Computer Supported
Collaborative Learning (CSCL) and more recently Technology-enhanced
Learning (TEL). Networked learning has a particular niche within this broader
family, as Goodyear (Goodyear 2005) contends:
The terms e-learning, web based learning and online learning now have
wide currency in education. I use the term networked learning to mean a
distinctive version of these approaches. I define networked learning as:
learning in which ICT is used to promote connections: between
one learner and other learners; between learners and tutors;
between a learning community and its learning resources
(Goodyear, Banks, Hodgson & McConnell, 2004).
The specific focus of this paper is on theories and methodologies in networked
learning. Many books have been written on research methods in Social Science.
Cohen et al. is one of the standard texts for educational research (Cohen et al.
2007). The Research Methods Knowledge Base
(http://www.socialresearchmethods.net/kb/) provides covers the entire research
process including: formulating research questions; sampling; measurement;
research design; data analysis; and, writing the research paper. It also
addresses the major theoretical and philosophical underpinnings of research
including: the idea of validity in research; reliability of measures; and ethics. The
ESRC National Centre for Research Methods (http://www.ncrm.ac.uk/) provides
a comprehensive site for collating research methods activities across the Social
3
4. Sciences, along with the latest in innovations in research methods. Early work
carried out by the centre included a review of research methods and generated a
typology of research methods (Beissel-Durrant 2004) which illustrates the rich
variety of research methods being used reflecting the breadth of different
epistemological perspectives in the field.
Oliver et al. (M. Oliver et al. 2007) argue that there are a range of different
epistemological positions adopted by researchers in the field and that these have
profound implications for how the field will be researched. They argue that this is
often explained in terms of the ‘paradigm debate’, and framed as a contrast
between qualitative and quantitative methods; although go on to qualify that this
is a rather crude distinction; i.e. qualitative data can be interpreted in a positivist
way and quantitative data can be used to yield understandings beyond the
specific numerical data. They argue that
‘we need to consider how different philosophical positions would interpret the
kinds of data generated by particular empirical methods. ‘Methodology’
describes this relationship, and must be understood separately from
‘methods’, which are the techniques used to collect and analyse data (This
will include things like interviews, questionnaires, observation etc.)
Methodology determines whether the implementation of particular methods is
successful or credible. Indeed, according to Agger, “methodologies can’t
solve intellectual problems but are simply ways of making arguments for
what we already know or suspect to be true” (Agger, 2004, p. 77).
To do this, methodology codifies beliefs about the world, reflecting ‘out there’
or ‘in here’ positions.
The view that knowledge is hard, objective and tangible will demand of researchers an observer role,
together with an allegiance to methods of natural science; to see knowledge as personal, subjective
and unique, however, imposes on researchers an involvement with their subjects and a rejection of the
ways of the natural scientist. To subscribe to the former is to be positivist; to the latter, anti-positivist.
(Cohen, Manion and Morrison, 2000: 6)
Such commitments and interests arise from historical, cultural and political
influences, which collectively shape traditions of research that provide the
4
5. context for current work (e.g. Conole, 2003). These have profound
implications for the topics that people study and the kinds of conclusions they
are willing to draw. (M. Oliver et al. 2007, p.9).
Therefore methods are the techniques used to collect and analyse data, whereas
methodology align with different epistemological beliefs and views of the world.
The term theory is contested and is used in a variety of different ways; here are
some definitions that are the closest to how it is used in a networked learning
research context:
• Theory, in the scientific sense of the word, is an analytic structure designed to
explain a set of empirical observations. A scientific theory does two things: 1.
it identifies this set of distinct observations as a class of phenomena, and 2.
makes assertions about the underlying reality that brings about or affects this
class. In the scientific or empirical tradition, the term "theory" is reserved for
ideas which meet baseline requirements about the kinds of empirical
observations made, the methods of classification used, and the consistency
of the theory in its application among members of the class to which it
pertains. These requirements vary across different scientific fields of
knowledge, but in general theories are expected to be functional and
parsimonious: i.e. a theory should be the simplest possible tool that can be
used to effectively address the given class of phenomena.
(http://en.wikipedia.org/wiki/Theory)
• A set of statements or principles devised to explain a group of facts or
phenomena, especially one that has been repeatedly tested or is widely
accepted and can be used to make predictions about natural phenomena.
(http://www.thefreedictionary.com/theory)
The relationship between theory and empirical data can be defined as follows:
Social research is theoretical, meaning that much of it is concerned with
developing, exploring or testing the theories or ideas that social researchers
5
6. have about how the world operates. But it is also empirical, meaning that it is
based on observations and measurements of reality -- on what we perceive
of the world around us. You can even think of most research as a blending of
these two terms -- a comparison of our theories about how the world
operates with our observations of its operation.
(http://www.socialresearchmethods.net/kb/naturres.php)
Networked learning researchers’ birth disciplines
Researchers at the TEL interdisciplinary workshop sited a broad range of ‘birth
disciplines’, including: Computer science, Plant science, Botany, Veterinary
science, Ethnology cultural studies, Psychology, HCI, Philosophy, Fine art, Moral
philosophy, Electronic engineering, Chemistry, History of art, AI, Geology, HPS,
International development education, Linguistics and AI, Philosophy, Sociology,
Maths and Physics. Authors involved in the ‘Contemporary perspectives in e-
learning research’ book (G. Conole & M. Oliver 2007) also came from a diverse
discipline background: Physical Sciences, Social Sciences, Critical Theory,
Education, Psychology, Computer Science, Philosophy and Management
Studies.
Clearly such diversity brings with it strengths; different theoretical perspectives
and methodologies; different interests in terms of the focus of inquiry and
research questions, but it also results in tensions - differences in definitions and
understandings and even fundamentally opposed epistemological beliefs.
Discussing the emergence of learning technology as a research field, Conole and
Oliver (G. Conole & M. Oliver 2002) note:
Learning technology is an inherently multidisciplinary field, and
stakeholders include of researchers from different fields (educational
research, cognitive psychology, instructional design, computer science,
etc) as well as teaching subject-experts who engage with it as 'end users'
or 'consumers'. This multi-disciplinarity is a common feature of emergent
6
7. research areas and, in one sense, is a strength. However, if we are to
capitalise on this richness of expertise, it is necessary to work towards a
clear theoretical underpinning that allows these diverse cultures to engage
with and develop the use of learning technology.
A recent cloud in cloudworks considers these benefits and tensions in more
detail (http://cloudworks.ac.uk/cloud/view/2806). Here is a summary of some of
the main arguments made to date:
• Some researchers recognise the underlying influence their ‘birth discipline’
has on their research approach. However others argued that their
perspectives around e-learning have been shaped far more by the
experiences they have had working in the area than by prior studies in an
unrelated discipline many years ago.
• The transition to an educational perspective for researchers originally fro a
Science background is very hard indeed, requiring a complete rethinking of
underlying epistemological beliefs. However having an understanding of both
Science and Social Science perspectives is incredibly useful. Similarly
transitional processes are evident from those coming into the research from
managerial or business backgrounds.
• Many researchers are drawn into research into the use of technologies in an
educational context from a practical perspective, i.e. what can these
technologies offer? What are the issues? This pragmatic stance is coupled
with a desire to understand and describe emergent theoretical perspectives.
• Irrespective of the theoretical and methodological lenses used to study
technological phenomena, the contextual and in particular the human
dimension is key
o Whether you call it Hermeneutics (Theology) or multiple
perspectives (Systems) doesn't matter - both recognise the situated
and contingent nature of anything involving people. Don't fall into
7
8. the trap of trying to make sense of TEL solely using the 'scientific'
paradigm.
• A number of points were made extrapolating key themes emerging from
research and practice. I) The focus needs to be on how technologies can
enhance the learning experience, and that it is important to remember that
good teaching and learning is possible without any technology. 2) The
teacher’s role is crucial, technology wont make a bad teacher good, 3) There
is no one size fits all solution. 4) Failure is useful, we can (and should) learn
from it.
The nature of theory
In the introduction to a special issue of JIME, Oliver provides an overview of the
position of theories in the emergent field of learning technologies in 2002 (M.
Oliver 2002):
I was struck by the diversity of theories that people were drawing upon,
and the very different ways in which they were using them. For some, a
theory was a touchstone, a guiding set of principles, the foundation on
which their work built. For others, theories were tools, and the important
thing was having the right one for the job. What, I wondered, was the right
way to use theory here? Should we believe in them, live them, and risk
being dogmatic — or should we be pluralistic, tied to none, and risk being
superficial?
The papers included in this issue are as varied and eclectic as the group
that contributed them. Approaches vary considerably — from theory as
tool, to theory as principle; from theory building, to theory using; from
disciplines as diverse as film studies, psychology, sociology and
education. So too do the topics — software tools, logic learning, metadata,
multimedia; an array of mainstream issues, and other gems besides. To
me, it is this diversity that makes this such an interesting area. It is
8
9. constantly challenging; always impossible to tell quite what perspective
might be brought to bear on your problem next.
Masterman and Manton considered the role of theory with respect to elearning
(Masterman & Manton 2009 and see also
http://cloudworks.ac.uk/cloud/view/1910) posing the following questions:
• What is the value of theory to teachers?
• What do we mean by theory?
• How has theory has been embedded into three areas?
They drew on Lawes work (Lawes, 2004), in particular the notion that theory
gives a framework of understanding that ultimately improves the quality of
practice and leads to the transformation of subjective experience. They argued
that theory could provide a glue between technology and practice. They then
went to make a distinction between theories, models and frameworks:
• Theories provide a means of understanding and predicting something
(Cook 2002). In the original article Cook expands this ‘A theory or model
can be used as a means for understanding and predicting some aspect of
an educational situation. Theories are not the same as models. A theory
can posses an explanatory power and can consist of a set of
...general assumptions and laws ... that are not themselves intended to be directly
(in)validated (for that, the theory must engender a model). Theories are foundational
elements of paradigms, along with shared problems and methods (Kuhn, 1962)
(Baker, 2000). ‘
• Model are abstract representations that helps us understand something
we can’t see or experience directly (Conole, Oliver et al., 2007)), models
include things like Kolb’s leaning cycle,
• A framework is a structure and vocabulary that supports the explication
of concepts and issues (G. Conole & M. Oliver 2002), such as Laurillard’s
Conversational Framework (Laurillard 2002).
9
10. They argue that theory is a cornerstone of professional practice… and an
antidote to technological determinism. However, teachers generally do not
consciously espouse formal theories and are driven by prior experience and
reflective practice.
Conole and Oliver (G. Conole & M. Oliver 2002) in discussing the range of
resources that can be used by practitioners to support decision making, identify
five types: tools, good practice, models, frameworks and templates/wizards. They
define models as
‘representations, usually of systems. These are frequently visual
representations, although formal models are more likely to be syntactic (or
derived from an underlying syntactic representation), often being defined
mathematically. Models may be tools, in that they can be used to carry out
analyses or may permit certain assumptions to be expressed. Equally,
however, they may be the object (i.e. purpose) of an activity, in that it may
be necessary to construct a model of a system in order to develop an
explicit understanding of how it works.’
And they go on to argue that there is a spectrum from templates/wizards through
to frameworks:
Aids to decision-making range from highly restrictive 'templates' or
'wizards', which provide high levels of support and step-by-step guidance
but little possibility of user-adaptation, through to 'theoretical frameworks',
which provide a context and scope for the work but leave the user to
devise their own strategy for implementation.
Theoretical perspectives dominant discourses
This section tries to articulate some of the main theoretical perspectives that are
evident in networked learning research. It is not intended to be comprehensive;
rather it aims to act as a starting point for discussion. At the TLRP TEL
interdisciplinary workshop the following range of theoretical perspectives were
10
11. listed: Social constructivism, Actor Network Theory, Constructivism, Critical
theory, Action research, Communities of practice – researchers and practitioners,
STS, Scientific enquiry, Conversational framework, Philosophy of technology,
Anthropological views on tools artefacts and technology, Activity theory. However
there was also a suggestion that we need to move beyond existing theories and
that in time new interdisciplinary theories might emerge from TEL work.
Cultural Historical Activity Theory
Socio-cultural perspectives are a predominate discourse in the field. In particular,
Cultural Historical Activity Theory (CHAT) (see the following edited collections
(Engeström et al. 1999) (Cole et al. 1997)(Daniels et al. 2007) has been used
extensively in Networked Learning particularly as a descriptive lense. A key idea
in CHAT is the notion of mediation by artifacts (Kuutti, 1991), which are broadly
defined ‘to include instruments, signs, language, and machines’ (Nardi, 1995). In
my own work I have drawn on this extensively in terms of exploration of the
range of mediating artefacts that can be used to support the learning design
process (Conole, 2008). Engestrom’s so-called ‘triangle’ representation
(Engestrom 2001) has been used extensively to described particular instances of
networked learning interventions, as it helps consider a focus on subject-object
with associated outcome supported through mediating tools in the context of a
wider community context and associated rules and divisions of labour (Joyes
2008) (Waycott et al. 2005) (Karasavvidis 2008).
Communities of Practice
Wenger’s notion of Communities of Practice (Wenger 1998) has been picked up
and used extensively in the field of Networked Learning;(G. Cousin & Deepwell
2005)(Guldberg & Pilkington 2006) (Breuleux et al. 1998)in some cases in
appropriately. It’s appeal is probably a combination of the fact that it is relatively
easy concept to grasp and that it offered a means of explaining some of the more
11
12. social-situated interactions arising in networked learning.
Actor Network Theory
Actor Nework Theory considers both people and technologies as Actants in a
connected network and in particular that it is the relationship between these
actants that is important. Although called a theory it doesn’t explain a
phenomenon but focuses more on why a network takes the form that it does. It is
much more interested in exploring how actor-networks get formed, hold
themselves together, or fall apart. It was developed by Callon (Callon 1999) and
Latour (Latour 2005) and also builds on the thinkings of Foucault (Fox 2000)
Cybernetics and systems thinking
Cybernetics and systems thinking provide a means of understanding complex
systems(Capra 1996) (Gharajedaghi 1999) and have been applied to a limited
extend in a networked learning context. Liber for example draws on the work of
Illich and Beer as a means of describing in modern learning environments and
systems (Liber 2004). Related work which also apply systems thinking include
the work of Friesen, Stankov et al. and Cantoni et al. (Friesen 2004) (Stankov et
al. n.d.)(Cantoni et al. 2004).
Methodological approaches
As the NCRM’s typology of research methods demonstrates there are a wide
range of research methods in use across the social sciences (Beissel-Durrant
2004). This section foregrounds some of the key methodological approaches
which have been predominant in networked learning. The choice of methodology
tends to reflect both the individual’s epistemological stance and their focus of
inquiry. Oliver et al. argue that
‘The kinds of data are available to e-learning researchers may suggest
particular kinds of interpretation. (M. Oliver et al. 2007)
This hints at the suggestion there is a complex inter-relationship between
research in the field and the affordances of the technologies themselves.
12
13. It is not possible to provide a comprehensive review of all the different
methodological approaches used in networking learning. Methodologies are
predominantly interpretive in nature; although experimental approaches are still
used extensively in North America. In terms of methods a range are evident –
interviews, focus groups, observation, surveys, student journals, video and audio
diaries, document analysis, and web tracking. In-depth case studies are popular,
as are large-scale surveys. The use of web tracking as a means of data
collection is still in its infancy but is a growing area of research.
Content analysis
Early research in the field was dominated by analysis of asynchronous
discussion forums. Coding schemes such as those developed by Henri, Garrison
et al. and Gunawadena et al. were used extensively. Henri (1991) identified
following five dimensions, which can be used to evaluate CMC: participative,
social, interactive, cognitive and metacognitive (Henri 1992). Garrison et al.
(2000) developed a 'community of learning' model which assumes that learning
occurs through the interaction of three core components: cognitive presence,
teaching presence, and social presence (Garrison et al. 2000). Gunawadena et
al. divided content into the types of cognitive activities the participant engaged
with (questioning, clarifying, negotiating, synthesising, etc), the types of
arguments they put forward, the resources used and any evidence of changes in
understanding (Gunawardena et al. 1997). There was a naïve assumption that
focusing on the content in the treaded messages was enough to capture the
whole event. Whereas in reality the level of detail/object of focus will naturally
have a significant impact on results and it was soon realised that taking account
of the broader context within which discussion forums were taken place was
important. Jones for example reports students simulating collaboration online
whilst co-present and seated around four computers (C. Jones 1999). A number
of approaches have been used to take account of the broader perspective. For
example, De Laat et al. use a multi-method approach using social network
analysis with content analysis and critical event recall (De Laat et al. 2007).
13
14. Social network analysis is used to visualise the social structures and dynamics of
the course, content analysis is used to identify the learning and teaching
processes and critical event recall is used to elicit teachers’ experiences and
perceptions.
Ethnography
Ethnography has been used extensively to study networking learning
phenomenon (Hodgson & Watland 2004) (Rice-Lively 1994). The approach is
qualitative based on ‘systematic description of human behaviour and
organisational culture based on first-hand observation’ (Howard 2002).
Case studies
Rich, situated case studies are a very popular and common form of studying
networked learning. A case study is an in-depth investigation/study of a single
individual, group, incident, or community (Yin 2009). The nature and scope of the
cases can vary significantly and the approach often overlaps with other
methodological approaches (such as action research, evaluation and
ethnography). Critics of the case study approach argue that the findings are not
generalisable or transferable. Proponents argue that the case-based approach
enables the researcher to gather a rich, contextual understanding of a situation in
context,
Action research
As might be expected given the educational nature of networked learning as a
research field, action research is often used as a methodological approach,
particularly by practitioners who are trialling out the use of technologies in their
classroom and want a framework within which to study the interventions.
Evaluation
The importance of evaluation has grown in recent years; as new learning
technologies emerge there is a need to evaluate how these are used to support
an increasingly diverse student population. The relationship between evaluation
and research more generally remains contested. Both processes may use the
same methods and study the same things. However, one way to distinguish them
14
15. is to consider how findings are used. If they are interpreted by an immediate,
local audience and used to support decision making, the study was probably an
evaluation; if findings are interpreted in terms of theories and are presented as a
contribution to knowledge, it was probably research. Oliver et al. contend that
pproaches in evaluation range from positivist approaches focussed upon
objective data collection (typically using quantitative methods) to interpretivist
ones more rooted in constructivism (typically using qualitative methodologies)
(Martin Oliver et al. 2007).
So which methodology should be used when, are some methodologies better
than others? Oliver considers how five different methodological approaches
(action research, behaviourist, activity theory-based, and a perspective based on
power) are used to tackle the same research problem. This provides a nice
illustration of how different theoretical perspectives would explain this situation
differently, and how each can contribute to our understanding of this field.
The following are some extracts from Cloudworks on different methodological
approaches or factors that influence different individuals’ approaches:
• I'm becoming particularly enamoured ATM with Content Analysis
(Krippendorff) it fits in well with my overall Systems perspective and
seems to me to be a useful tool in evaluating TEL as it focuses on
communication acts and meaning - which seems to me to be central to
any TEL 'pedagogy'. I'm also interested in the work done on Andragogy -
the transformative nature of adult learning and wonder whether some of
us shouldn't be moving away from pedagogical theories of TEL based on
the developmental psychology of children when we want to apply them to
adult learners ... (Diana Brewster)
• On a wider pedagogical level I do lean towards group based activities and
situated learning, as so much of my working life has been spent creating,
devising and refining ideas in goal oriented teams (Sacha van Straten)
15
16. • The methodologies I use span a wide range of disciplines, and combine
quantitative and qualitative research techniques, taking multiple
perspectives at different levels of analysis (micro, meso, macro). (Steven
Verjans)
Research questions and grand challenges in the field
Conole and Oliver articulate a set of research questions grouped according to
whether the focus predominantly on technological, pedagogical or organisational
issues. Despite the fact that these questions were generated in 2007 and were
referring to e-learning they provide a useful starting point for discussion what the
equivalent set is today for networked learning research.
Research focus Research and evaluation themes and questions
16
17. Pedagogical aspects
Understanding the learning process
What is effective pedagogy in terms of using learning technologies?
Will the use of ICT result in new forms of pedagogy?
What are students experiences of using technologies and which did they use and for what purpose?
What are the inherent affordances of different technologies?
In what ways can new technologies be used to support and enhance organisational learning?
What forms of collaborative activities were occuring and how can these be supported?
Are current teaching and assessment activities appropriate in an e-learning context?
How do current assessment practices enable the students to demonstrate what they had learnt and what
is the role of e-assessment?
Development
What are the design and development issues associated with the production of e-learning materials?
Who is involved in course development and what is their involvement?
How much time do developers and practitioners spend on course development?
What pedagogical models are course development teams using, how explicit are they and how effectively
do they translate in practice?
How are courses being design to address different learning styles and cognate levels?
What pedagogical models are being used and how explicit were these?
How well do the teaching and assessment methods map to the course learning aims and outcomes?
What assessment methods are used?
Delivery and integration
What are the best methods of integrating the use of ICT within the broader learning and teaching
context?
Are there pedagogical models underpinning different e-learning platforms and how do these influence the
way these systems is used?
How are different tools available within learning platforms being used to support learning?
What are students’ views of different learning systems?
How useful do students find e-learning resources?
How much did they use additional resources and the internet?
What are students’ experiences of online courses?
17
18. Standards and architecture
Underpinning technology What are the current trends in the development of underpinning standards and what are the associated
interoperability issues?
How interoperable are current tools, how well do the link with institutional systems such as student
records, finance, library, local VLEs and admission systems?
What research is being done into the development and testing of technical infrastructures and
architectures?
Tools and technologies
What are the new and emerging technologies and how can they be used to support learning and
teaching?
What learning platforms are being used and how do they compare?
What are the emerging new software and hardware systems and how might they be used?
What will be the impact of emergent mobile and smart technologies?
In what ways are in-built tracking mechanisms within e-learning systems giving rise to surveillance
issues?
Functionality and uses
How is technology constructing new forms of identity?
What are the new forms of power afforded by technologies and what are the implications?
What can we learn from in-built tracking and monitoring facilities and how might this knowledge be used?
How easy are different systems to navigate and use?
How can we better understand the different multiple forms of representation that new media now
provide?
What functionality of different tools is being used by tutors and students, for what purposes?
What do tutors and students think of different learning platforms and what are the perceived benefits and
limitations?
How much is multimedia being used in current practice and for what purposes?
What security mechanisms are available for authentication?
How are different forms of multimedia (images, audio, and video) being used to support learning and
teaching?
What do we know about the different characteristics of multimedia?
How usable are different learning platforms and how easy are they to navigate around?
18
19. Organisational issues
Users
What is our current understanding of how stakeholders (academics, support staff, administrators, senior
managers and students) work?
What mechanism and procedures are appropriate for developing shared knowledge banks of expertise
and information?
What are the different emerging roles and responsibilities associated with e-learning activities–
management, technical, research, dissemination, evaluation, training?
What are the different views of e-learning and its role amongst academics and support staff?
How are institutions dividing roles and responsibilities for e-learning and how much training and support
is staff getting?
Structures and processes
How can we better evolve map current institutional structures and skills and roles to capitilise on the
potential use of technologies within our organisations?
What do we understand about how institutions are currently structured in relation to implementation of e-
learning?
How do e-learning activities align with institutional courses and procedures?
How can we build a picture of what changes will be required to make the shift to using online learning
systems to support e-learning?
How is the knowledge gained from the development and delivery of e-learning courses being used to
guide e-learning practice more generally across institutions?
How is experience gained on one course been translated to other courses?
What institutional issues are arising as a result of e-learning activities?
What institutional support issues arose as a result of the development and what are the strategic
implications?
What quality assurance methods were developed and used?
What e-business models are being used in institutions?
Context and culture
What are the key organisational issues and challenges associated with implementing large-scale e-
learning initiatives?
How do we manage the bulk of existing materials and information on university web sites which have little
or no coherence and consistency?
How can we ensure that different stakeholders engage with e-learning in a meaningful way?
How can we manage the transition from existing practices and processes to effective use of new
19
20. Contextual factors
What are the legal and ethical issues (data protection, confidentiality etc) associated with the e-learning
How is plagiarism being detected and dealt with in an e-learning?
How are accessibility being addressed?
What are the ethical issues associated with e-learning?
What are the specific security issues associated with different learning platforms and technologies?
How students being authenticated and what are are the potential loopholes?
How are different institutions dealing with the issue of copyright and ownership of material
What gender differences are emerging in the use of different technologies and the ways in which e-
learning is being used?
What are the cultural and linguistic issues and how are these being addressed?
What subject discipline differences are evident in the use of the tools and the types of activities
associated with different courses?
How are the new disability laws being addressed in terms of e-learning activities?
In what ways is e-learning being used to promote widening participation?
How are special educational needs being addressed in e-learning?
In what ways is and might e-learning be used to support lifelong learning?
How does our e-learning developments compare with international developments?
At the TLRP TEL workshop the following areas of focus were listed: Cognitive
education, Creating research communities, Epistemology, Case-based learning,
Human computer interaction, Fieldwork across disciplines, Artificial intelligence,
People/communities, Educational research, Fluid learning objects, Personal
development, Fostering self-sustaining communities, Human learning and
judgment, Creative development, Field work across disciplines, Making a
sustainable permanent difference change.
At a TLRP TEL seminar (http://www.tlrp.org/tel.old/tel_events.html#sem2) Diana
Laurillard mapped the current dimensions and players in the field of technology
enhanced learning, in the UK and in the wider EU, building on the findings being
20
21. generated by the principal research funding bodies. She looked at how the TEL-
funded development proposals can be located within this developing research
space, and will consider the following questions:
* In which directions are they moving the field forward?
* How do they relate to other funded research in the field?
* How will they build on current issues and findings?
The seminar also considered the mechanisms and technologies available for
supporting the cumulation of knowledge from researchers and from practitioners
as action researchers in the field.
Figure 1: Laurillard's mapping of the TEL field
21
22. The challenges of interdisciplinary research in networked
learning and strategies for fostering interdisciplinarity
Networked Learning by nature is an interdisciplinary area; drawing on a wide
range of disciplines, theoretical perspectives and methodological approaches. Is
there anything distinctive about interdisciplinarity in this context as opposed to
interdisciplinarity more generally?
Alan Blackwell is co-director of Crucible – a centre for research in
interdisciplinary research.( http://www.crucible.cl.cam.ac.uk/), he has done
extensive research on interdisciplinarity. At the ESRC TEL workshop in
November 2009, he listed the following as ingredients for successfully fostering
interdisciplinarity:
• Leaders and founders of interdisciplines should resist convention and
maintain vision, while being mentors and coaches
• Freedom requires resource
• Collaborations grow in years not months
• Goals must offer serendipity not constraint
• Maintain and reward curiosity
• Understand work with and subvert structures – organisational, disciple, career
Along with suggestions for making it happen:
• Start small and move fast
• Bring creative and design practices to technology
• Facilitate encounters between communities
• Cheerfully transgress academic borders
• Engage with reflective social science
• Directly address public policy
More generally from my own experience in working in interdisciplinary teams,
one means of fostering interdisciplinarity is to create shared objects as the focus
of inquiry. The use of such ‘mediating artefacts’ in a project can act as trigger
22
23. points to discuss ideas around.
As a precursor to the current TLRP TEL programme a series of seminars were
run. Josie Taylor led one that focussed on interdisciplinarity
(http://www.tlrp.org/tel.old/tel_events.html#sem2) considering the following
questions:
* What is the difference between interdisciplinarity and multidisciplinarity?
* Does it matter?
* Can you engineer interdisciplinarity or can it only arise spontaneously?
* What about funding for interdisciplinary work?
In the TLRP TEL workhop one researcher questioned whether most
Interdisciplinary teams were really multi-disciplinary. Another person suggested it
was important to break down the space barrier – to create time and space for
sharing and imaging. Another argued for the need to create theoretical space.
Case studies
This section is intended to provide some case studies that aim to apply the
general discussions so for in particular context; to show how different theoretical
perspectives and methodological approaches are actually applied to real
research problems. I have included a couple of case studies from my own
research, but s hope that the Networked Learning hotseat discussions will
generate some additional examples. Each case study follows the following
template:
• The focus of research
• Theoretical perspectives
• Methodological approach adopted
23
24. • Key issues
Case study: The agile development of a social networking site for education,
Cloudworks
Cloudworks is a social networking site for sharing and discussing learning and
teaching ideas and designs. The original focus of the research was:
Given that there is a gap between the potential of technologies and how
they are used in practice, can general social networking and web 2.0
practices be harnessed and used to foster sharing and discussion in a
teaching context?
The theoretical perspective was predominately socio-cultural in nature, drawing
in particular on the notion of mediating artefacts. Of particular importance was
application of Engestrom’s notion of ‘social objects’. In the Cloudworks site, the
social objects are ‘clouds’, where a cloud can be anything to do with teaching
and learning (an idea, a discussion topic, a tool or a resource). The design and
development of Cloudworks is discussed in a recent Computers and Education
paper (G. Conole et al. 2008) and the initial theoreitical perspectives in an AJET
paper (G. Conole & J. Culver 2009). Recently we have begun to expand our
theoretical perspectives as discussed in a paper submitted to this year’s
Networked Learning Conference:(Alevizou, et al. n.d.)
Our initial theoretical perspectives on which the development of Cloudworks was based,
focussed around Engeström’s (2005) notion of 'social objects' in social networking and
Bouman et al’s . (2007) framework for 'sociality'. More recently we have started to explore
three additional frameworks and demonstrate how they are helping us with our
preliminary analyses of emerging activities on the site and in particular the insights they
provide into the dialogic interchanges and structures of involvement within the site. The
first framework is Goffman’s (1955;1963) notions of ‘face-work’ and ‘ritual performance’.
The second is Engeström’s (2001) idea of ‘expansive learning’. The third is the notion of
collective intelligence (Lévy, 1998; Jenkins, 2006). In this paper, we review a selection of
case studies from the site, and explore how the frameworks can be used to understand
then. We argue that these perspectives are useful in studying networked sociality
24
25. bounded in the context of learning, with wider implications for the matters of participation,
self-representation, and openness in education. We conclude with the methodological
frameworks that can support the further study of interaction, socialization and sharing into
higher education establishments and culture
Our methodological approach is essentially one based on agile development
using user-focused mixed method approach. Data is collected from a variety of
sources, including interviews, focus groups, workshops, observations, think aloud
protocols and web statistics.
Case study: A design and evaluation framework1
XDelia (Xcellence in Decision-making through Enhanced Learning in Immersive
Applications) www.xdelia.org, is a three-year pan-European project that uses
wearable sensors and serious games to investigate how people’s behavioural habits
and emotional states affect their financial decision making. The project combines
research skills and expertise of European partners from different methodological
traditions (experimental, economic, field research) who will work together to
achieve the project goals. Within this project we have developed a Design and
Evaluation framework that aims to help stakeholders of a inter-disciplinary research
project develop a shared understanding of project goals and methods by pooling
their knowledge of research approaches and methodologies. The Design and
Evaluation framework will provide a working collaborative model to capitalise on
the different approaches, using ongoing participatory evaluation to ensure the
development of an integrated set of research questions, optimum use of research
instruments and effective collaboration between the different disciplines.
The approach aligns with Patton’s utlilisation-focussed evaluation approach
(Patton 2008)and is informed by Cousins and Whitmore (1998) three dimensions
of collaborative inquiry; control of decision making, selection for participation and
depth of participation (Cousin & Whitmore 1998). To support the evaluation we are
developing a design and evaluation framework that will provide the vehicle to
ensure that comprehensive, ongoing evaluation is built into all facets of the
project and that evaluation findings feed back into the ongoing development
activities of the project in a timely manor.
It has been based on a participatory and iterative approach, which aims to be
1
The text here is taken from a conference presentation on this work(Clough et al.
2009)
25
26. ‘useful’ rather than rarified – i.e. formative evaluation that feeds into and informs
project activities as they occur throughout the project on an ongoing basis, rather
than a more removed summative evaluation which merely reports on project
activities towards the end of the project lifecycle. In addition to drawing out
specific instances that occur across the project, we want to explore a number of
underlying themes, some of which arose from the baseline interviews. For
example, the way in which complex inter-disciplinary projects of this kind are
coordinated can have a significant impact on how well the project works and the
extent to which overarching objectives are achieved. Similarly we want to
examine what kind of collaborative activities occurs in the project and the extent
to which they are successful or not. Finally what critical moments occur and how
do they steer subsequent project work? In keeping with the notion of being
participatory, iterative and ‘useful’ the Design and Evaluation framework
encourages partners to adopt a critically reflective approach to the evaluation
across the project – everyone is asked to reflect on what they are doing;
everyone is a researcher/reflector/evaluator. Figure 1 shows the relationship
between the design and evaluation sides of the framework in which each builds
upon and feeds into the other.
Figure 2: The X-Delia design and evaluation framework
26
27. References
Alevizou,, G. et al., Ritual performances, collective intelligence & expansive learning:
theoretical frameworks for analysing emerging activity patterns in Cloudworks. In
Aalborg, Denmark.
Beissel-Durrant, G., 2004. A Typology of Research Methods Within the Social Sciences.
Available at: http://eprints.ncrm.ac.uk/115/ [Accessed January 16, 2010].
Breuleux, A., Laferrière, T. & Bracewell, R.J., 1998. Networked learning communities in
teacher education. In Proceedings of SITE.
Callon, M., 1999. Actor-Network Theory.
Cantoni, V., Cellario, M. & Porta, M., 2004. Perspectives and challenges in e-learning:
towards natural interaction paradigms. Journal of visual languages and
computing, 15(5), 333–345.
Capra, F., 1996. The web of life, Anchor books New York.
Clough, G., Conole, G. & Scanlon, E., 2009. Behavioural Finance and Immersive Games:
A Pan-European Framework for Design and Evaluation. In Auckland.
Cohen, L., Manion, L. & Morrison, K., 2007. Research methods in education, Routledge.
Cole, M., Engeström, Y. & Vasquez, O.A., 1997. Mind, culture, and activity: Seminal
papers from the Laboratory of Comparative Human Cognition, Cambridge Univ
Pr.
Conole, G., 2008. Capturing practice, the role of mediating artefacts in learning design.
In In L. Lockyer, S. Bennett, S. Agostinhi and B. Harper Handbook of learning
designs and learning objects. IGI Global.
Conole, G. & Culver, J., 2009. Cloudworks: Social networking for learning design.
Australasian Journal of Educational Technology, 25(5). Available at:
http://www.ascilite.org.au/ajet/ajet25/conole.html [Accessed November 24,
2009].
Conole, G. et al., 2008. Cloudworks: social networking for learning design. Available at:
http://oro.open.ac.uk/13174/ [Accessed November 24, 2009].
Conole, G. & Oliver, M., 2007. Contemporary perspectives in e-learning research:
themes, methods and impact on practice. Available at:
http://oro.open.ac.uk/12148/ [Accessed November 24, 2009].
Conole, G. & Oliver, M., 2002. Embedding theory into learning technology practice with
27
28. toolkits. Journal of Interactive Multimedia Education, 8.
Cook, J., 2002. The Role of Dialogue in Computer-Based Learning and Observing
Learning: An Evolutionary Approach to Theory. Journal of Interactive
Multimedia Education, 5. Available at: http://www-jime.open.ac.uk/2002/5/.
Cousin, G. & Deepwell, F., 2005. Designs for network learning: A communities of
practice perspective. Studies in Higher education, 30(1), 57–66.
Cousin, J. & Whitmore, E., 1998. Framing participatory evaluation. New directions for
evalution, 5023.
Daniels, H., Cole, M. & Wertsch, J., 2007. The Cambridge Companion to Vygotsky 1st
ed., Cambridge University Press.
De Laat, M. et al., 2007. Online teaching in networked learning communities: A multi-
method approach to studying the role of the teacher. Instructional Science, 35(3),
257-286.
Engestrom, Y., 2001. Expansive learning at work: Toward an activity-theoretical
reconceptualization. Journal of education and work, 14(1), 133–156.
Engeström, Y., Punamäki-Gitai, R.L. & Miettinen, R., 1999. Perspectives on activity
theory, Cambridge University Press.
Fox, S., 2000. Communities Of Practice, Foucault And Actor-Network Therory. Journal
of Management Studies, 37(6), 853-868.
Friesen, N., 2004. Three objections to learning objects and e-learning standards. Online
education using learning objects, 59–70.
Garrison, D., Anderson, T. & Archer, W., 2000. Critical inquiry in a text-based
environment: Computer conferencing in higher education. Internet and Higher
Education, 2(2-3), 87-105.
Gharajedaghi, J., 1999. Systems thinking: Managing chaos and complexity: A platform
for designing business architecture, Elsevier.
Goodyear, P., 2005. Educational design and networked learning: patterns, pattern
languages and design practice. Australasian Journal of Educational Technology,
21(1), 82-101.
Guldberg, K. & Pilkington, R., 2006. A community of practice approach to the
development of non-traditional learners through networked learning. Journal of
Computer Assisted Learning, 22(3), 159–171.
28
29. Gunawardena, C., Lowe, C. & Anderson, T., 1997. Analysis of a global online debate
and the development of an interaction analysis model for examining social
construction of knowledge in computer conferencing. Journal of Educational
Computing Research, 17(4), 397-431.
Henri, F., 1992. Computer conferencing and content analysis. In In A.R. Kaye (Ed.)
Collaborative learning through computer conferencing: The Najaden Papers.
Berlin: Springer Verlag, pp. 116-136.
Hodgson, V. & Watland, P., 2004. Researching networked management learning.
Management learing, 35(2), 99-116.
Howard, P., 2002. Network Ethnography and the Hypermedia Organization: New Media,
New Organizations, New Methods. New Media and Society, 4(4), 550-574.
Jones, C., 1999. From the sage on the stage to what exactly? Description and the place of
the moderator in cooperative and collaborative learning. ALT-J, 7(2), 27-36.
Joyes, G., 2008. An Activity Theory Approach to Researching Tutors' Perceptions of
Effective Online Pedagogy. Educational Media International, 45(3), 231-243.
Karasavvidis, I., 2008. Activity Theory as a theoretical framework for the study of
blended learning: a case study. In Greece. Available at:
http://www.networkedlearningconference.org.uk/past/nlc2008/abstracts/Karasavv
idis.htm.
Latour, B., 2005. Reassembling the social: An introduction to actor-network-theory,
Oxford University Press, USA.
Laurillard, D., 2002. Rethinking university teaching, Routledge.
Liber, O., 2004. Cybernetics, e-learning and the education system. International Journal
of Learning Technology, 1(1), 127–140.
Masterman, L. & Manton, M., 2009. Pedagogic Theory and Pedagogic Planning in
Digital Worlds. In Brighton.
Oliver, M., 2002. JIME Special issue on theory for learning technologies: editorial.
Journal of Interactive Multimedia Education, 9.
Oliver, M. et al., 2007. Knowledge, society and perspectives on learning technology. In
In G. Conole and M. Oliver (Eds.) Contemporaryr perspectives in e-learning
research: themes, methods and impact on practice. London: RoutledgeFalmer,
pp. 21-37.
Oliver, M. et al., 2007. Evaluation. Available at: http://oro.open.ac.uk/15277/ [Accessed
29
30. November 24, 2009].
Patton, M., 2008. Utilization focussed evaluation 4th ed., Saint Paul, MN: Sage.
Rice-Lively, M.L., 1994. Wired Warp and Woof: An Ethnographic Study of a
Networking Class. Internet Research, 4(4), 20 - 35.
Stankov, S., Grubišić, A. & Žitko, B., E-learning paradigm & Intelligent tutoring
systems. Annual 2004 of the Croatian Academy of Engineering, 21–31.
Waycott, J., Jones, A. & Scanlon, E., 2005. An activity theory framework for analysing
PDAs as lifelong learning tools. Special issue of Learning Media and Technology,
30(2), 107-130.
Wenger, E., 1998. Communities of Practice. Learning, Meaning and Identity. Learning
in Doing: Social, Cognitive, and Computational Perspectives, Cambridge
University Press, Cambridge.
Yin, R., 2009. Case study research: design and methods 4th ed., London: Sage.
30
