Referencia:

Reference:

Rubio, Maria José (2003). Focus and models of evaluation of the e-learning. RELIEVE, v. 9, n. 2. http://www.uv.es/RELIEVE/v9n2/RELIEVEv9n2_1eng.htm.

 

This article in spanish (original Version)

Analysis / Análisis

 Hits:  Hit counter - Contador de visitas Visitas     

FOCUS AND MODELS OF EVALUATION OF THE E-LEARNING

(Enfoques y modelos de evaluación del e-learning)

In pdf format

by

En formatopdf

Article record

About the author

Print this article

María José Rubio (rubio@trivium.gh.ub.es)

Ficha del artículo

 Sobre la autora

Imprimir el artículo

 

Abstract

      The aim of this article is to present the state of arts about e-learning evaluation. Different evaluative approaches, different models, tools and experiencies are showed, in order to determine the quality of virtual learning. Two main approaches are showed: partial evaluation, with emphasis in some aspects of e-learning (materials, technology resources, teaching, etc.) and global evaluation, that uses management quality models and the practice of benchmarking.

Resumen

  El objetivo del artículo es presentar el estado de la cuestión sobre la evaluación del e-learning. Con este propósito se muestran los diferentes enfoques evaluativos, así como diferentes modelos, herramientas y experiencias encaminadas a determinar la calidad de la formación virtual, o e-learning. Se destacan dos enfoques principales, la evaluación de enfoque parcial, que enfatiza aspectos diversos del e-learning (los materiales, los recursos tecnológicos, la docencia, etc.) y la evaluación de enfoque global, que utiliza modelos de la gestión de la calidad y la práctica del benchmarking.

Keywords

Evaluation, quality, virtual education, virtual learning, e-learning

Descriptores

Evaluación, calidad, formación virtual, aprendizaje virtual. e-learning

 


Introduction


Comentar este artículo


Artículos relacionados:

Bo, Sáez  y Belloch (1999). Sistemas de evaluación de contenidos en teleformación: el proyecto C.F.V.


Volume 9, n 2


E-learning is a product more than those generated by the society of the information and the digital era in that it places an importance in the mark of the new teaching / learning models and of the learning throughout a lifetime in convergence with the possibilities that the information technologies and the communication offer to educational applications.

From the conceptual view point of e-learning, it is a term susceptible to different definitions and often interchangeable with others: on-line formation, on-line courses, virtual formation, teleformation, formation at distance, virtual campus. In the literal sense, in English, it means electronic learning; the learning is taking place through a half technological-digital one.

Rosenberg (2001) defines it as the use of the technologies based on internet to provide a wide unfolding of solutions in order to improve the acquisition of knowledge and abilities. The same author establishes three approaches that must complete you to be able to apply the term correctly (Rosenberg, 2001: 28-29) that:

a)          It takes place in the net, which would allow an immediate upgrade, storage and recovery, distribution and capacity of sharing content and the information,

b)          It arrives to the end user through a computer, using standard technological of Internet,

c)          It is centered in the widest vision in solutions for the learning that go beyond the traditional paradigms of the formation. 

However these approaches are strongly restrictive. The e-learning not only takes place through technological standards of the Internet, the platforms ad hoc is a more and more important element; and the material off line or downloable support the rest of the formation which is a primordial component while they don't improve the infrastructures of the communications in the own net.

From its erruption in the educational and formative world, the e-learning has not only generated important expectations of pedagogic character, but also of social and economic character, that together with the growing interest for the educational (González, 2000:53) quality in anyone of their manifestations and environments, makes that the necessity be imposed of developing appropriate evaluation models to the object and the different contexts in those that takes place.

 

1. Tendencies in the evaluation of the e-learning

The restlessness to evaluate e-learning is giving place to important initiatives and experiences at a world level guided to settle down standards that allow certifying of their quality.

Although the mechanisms of parametrization of the quality vary in so much function of the context (Tait, 1997), as of the own concept of quality (Harvey and Green, 1993), until the moment one can speak of two big tendencies in relation to the practices to evaluate the quality of the institutions and of the projects that use the e-learning as formative activity within their own (Sangrà, 2001) entity. The objective is centered mainly in looking for approaches and specific indicators that give answers to the questions that lays a foundation for the evaluation of the quality of the formation in specific environments, with specific means and is directed to people with a different profile to that of the traditional (in the case of the universities) pupil.

.        Partial focus. Centered mainly in some of the following aspects:

.. The formative activity

.. The formation materials

.. The technological platforms

.. The relationship cost / benefit 

.        Global focus. They are distinguished two tendencies:

.. The evaluation systems centered in models and/or norms of standard quality and total quality

.. Systems based on the practice of the benchmarking

 

1.1. The parcial focus evaluation

It is the evaluation centered in some of the considered elements of more interest inside a solution e-learning.

1.1.1. Evaluation of the formative activity

It is the process guided to evaluate a concrete action of formation, as it can be an on-line course, of more or smaller duration. The purpose of this evaluation is guided basically toward three aspects: to check the level of execution of the educational objectives, to improve the formative own action and to determine the return of the carried out investment.

Belanger and Jordan (2000:187) identify three main models in the evaluation of formative (adapted some of the traditional formation) actions.  Models that put the emphasis well in the diagnostic evaluation, before introducing the formative action, well in the final evaluation, once the formation has taken place.

Sistemic model of Vann Slyke et al. (1998)

The pattern provides a group of variables that interact like predictor factors of the success of the formative on-line action. From these they concentrate on the following:

Institutional characteristic

The institutional characteristics are related with the capacity of the organization to implement actions of e-learning, such as the objectives of the institution, the support infrastructure to the action, the economic capacity.

Characteristic of the addressees of the formation

These characteristics are related with the interests, expectations and the students' (self-sufficiency, personal management of the time, domain of the computer and attitude toward the technology, capacity for the resolution of problems.) abilities. He/she is the only model that presents the "characteristic variable of the pupil" like factor of success or failure of the on-line formation, although there are several authors that emphasize the individual differences of the users like element important predictor of the success of the virtual formation (Richardson, 2001; Oliver, 1998; Ramussen and Davidson, 1996).

Characteristic of the course

The characteristics of the course have to do with the capacity of the system of e-learning in relation to the necessities and teaching-learning methodologies for the course. For example, if the course requires a methodology based on the collaborative work in the virtual environment he/she should be able to facilitate it.

Characteristic of the formation at distance

These characteristics refer to the necessity of creating new models of accommodation from the users to the new environments, so that he/she assures their tranquility, comfort and learning easiness.

These variables should be studied for the on-line implementation of formative actions so that they adapt to the pupil, to the faculty, to the institution and the society. The students can finish this way, for example, by receiving a high interaction with the professor if they need it, the institution can increase the productivity among the educational ones or at social level it can improve the access to the education, the quality of life, the force of the work, etc.

 

Model of five levels of evaluation of Marshall and Shriver (in McArdle, 1999)

This model is centered in five action levels guided to assure the knowledge and competitions in the virtual student.

Teaching

This level is centered in the capacity of the educational one in the on-line formation to be projected through the technological (the electronic mail, the chat, the virtual classroom.) means, making use of appropriate communication abilities to that environment, such as the clarity in the writing of the messages, frequent intervention in the virtual classroom, immediacy and effectiveness in the answers to the messages of the pupil, appropriate appropriation of the resources that provides the technological environment.

Materials of the course

The evaluation of the materials should be carried out by the pupil with relationship to the level of difficulty, relevancy, interest or effectiveness.

Curriculum

The contents or the currículum of the course should be evaluated with a high level of analysis and for comparison with other curricula.

Modules of the courses

The modulation is a characteristic of the on-line courses that should equally be valued in relation to its structure and order.

Transfer of the learning

This last level tries to determine the grade in which the on-line course allows the participants to transfer the acquired knowledge to the work´s position.

The model combines different elements of the educational act; however, it puts a special emphasis in the educational one, as a dynamic factor of the formation in virtual environments. In fact we attend a renovated interest for the educational (Mateo, 2000a) quality that is transforming into a strategic factor of first magnitude. In the virtual environment the teacher must locate himself in a formative new space as a guide of the participant, being the interaction the base for the formative (Duart, 2001a) development. Although the standards of the levels of the faculty's performance are different in the on-line formation, many of the parameters considered in the present models of (Mateo, 2000b) educational evaluation can be the same.

 

Model of the four levels of Kirkpatrick (1994)

The pattern of Kirkpatrick has been and is broadly used in the evaluation of formative traditional actions, and currently they are several authors that recommend its adaptation and use in e-learning (Rosenberg, 2001, Mantyla, 2000, Belanger and Jordan, 2000).

The pattern is guided to evaluate the impact of a formative action through four levels: the reaction of the participants, the received learning, the reached transfer level and finally the resulting impact.

Reaction

It is without a doubt the type of evaluation more widely used in most of formation courses. It can be carried out through an opinion questionnaire, or in a more qualitative way by means of discussion groups. In the e-learning it covers a special interest if one keeps in mind that it is the only feed-back for the reaction of the users with which the agents of the course (in front of different elements, as the educational one, the materials, the contents, the environment, the learning, the transfer or the perception of the impact of the received formation) count.

Learning

This evaluation (in their formative version and/or sumative) tries to check the level of knowledge and abilities acquired by the pupil through tests (Mantyla, 2000: 267). Kirkpatrick (1999) recommends the use of a methodology quasi-experimental like strategy to be able to settle down in a more objective way the effectiveness of the course. Other authors on the other hand consider that rather than looking for for the effectiveness of the course, this evaluation should be used as a feedback method to improve it (Rosenberg, 2001). In any case, the challenge in the on-line formation with relationship to the evaluation of the learnings is in configuring strategies and validation systems that don't require any presence.

Transfer

The evaluation of the transfer consists on detecting if the acquired competitions with the formation are applied in the work environment and if they stay through time (better acting of the task, faster, less errors, change of attitude, etc.). To evaluate the transfer permits the demonstration the contribution from the formation to the improvement of people and the benefits that it contributes to the organization to determine their impact and profitability later. In spite of being a crucial evaluation in the managerial formation, 62% of the organizations only use it, according to completed investigations (EPISE, 2000, Pine grove et al., 1999). The instruments or more widely used strategies are observation, interviews of supervisors and even self-evaluation of the participants (Pine grove, 2002).

Impact

As in the previous level, this evaluation is also used mainly by the companies. Although traditionally the evaluation of the impact or of the results has been based on economics (to demonstrate a bigger number of sales, bigger productivity, less errors, quality of service, less reclamations.) approaches, these are not always possible to prove, Rosenberg (2001: 223) in that he/she affirms that it is "better to conform to the evidence than to claim tests. If when asking the manager of the company and all the supervisors if the recent introduction of solutions e-learning generates a bigger productivity, the great majority answer that yes, it serves as good evidence that a correlation exists among the solutions of e-learning and the mensuration of the business." The objective of the evaluation could also be to determine to what extent the lack of the formation can have a harmful impact in the organization. In the same way, it is necessary to notice that the impact of the formation not only takes place at an economic level, the generated knowledge, the innovation capacity that it produces or the dedication of the people are also important.

The main problem that all the models present indexed in this epigraph is that none of them clarify the evaluation indicators, neither the standards of evaluation or the processes or forms of obtaining evidence for each one of the evaluated elements.

1.1.2. Evaluation of the materials

The quality of the materials of the covered format forms a special significance in the non-present formation, being the main instrument of basic transmission of knowledge of which prepares the pupil. With the result that their evaluation has transformed into an evaluation that has been dedicated to bigger efforts.

The materials used in the e-learning can be textual, hipertextuals (or hipermedia [1]) or multimedia, and designed for their use online as much as off-line.

In the environment of the evaluation the line of evaluation of multimedia material is highlighted, also called evaluation of educational software, with a long tradition from the appearance of the first educational (EAO) computer applications. Under parameters of the evaluation of programs, the experiences related with the evaluation of material multimedia have become centered according to the objective of the evaluation and the agent appraiser in the evaluation of necessities, input, process, of product and/or of the results (chart 1). But particular form highlights the unfolding developed for the evaluation of the product guided to certify the quality of the materials and/or to facilitate the taking of decisions in its selection.

Cabero (2001:451-455) identifies three evaluation types with regard to the technological means in general: the evaluation of the means themselves (characteristic of the means), the comparative evaluation with another means and the didactic curricular (the behavior of the means in the teaching-learning context) evaluation; and three agent appraisers: the producers, the experts (in contents, instructive design.), and the users.

Numerous initiatives guided to design standard instruments of measures of quality for the evaluation of the technological educational materials exist. Among them they highlight for their magnitude: the Instructional Management Systems Project (http://ims.org) that gathers multinational American computer companies and educational institutions to define technological standards; or the Promoting Multimedia Acces Education and Training in European Society (http://www.perseus.tufts.edu), a project guided to define the formative quality of the multimedia material. In a more modest, but also prominent way, we find, by way of example: the Systemic Pattern of Quality of Software (Mendoza et al., 2001) guided to determine the quality of the software in low, he/she mediates and high; the project And-CumLaude (Rodríguez et al., 2001) in order to certify the quality of the educational material multimedia centered in approaches of quality of minima and of excellence; the method of Evaluation of Computerized Educational Materials based on the trial of experts (Galvis, 2000); or the scale of Cataloguing and Multimedia Evaluation SAMIAL (Navarrese, 1999) centered in different categories and guided to establish the quality of the material in excellent, high, correct and low.

 

Work of the evaluation

Objectives of the evaluation and agent appraisers

Approaches for the evaluation

Instruments of obtaining of information

Of necessities

To contribute information about the improvements that it can introduce to the new material 

Producers

I study other materials with similar objectives

Qualitative analysis

Of the input

To determine the capacities for the realization of the material

Producers

Software and hardware (of development and of the user)

Programming

Qualitative analysis

Of the process

To correct and to perfect the material during their development

Producers, potential users

Control of the contents

Functional control

Evaluation protocol

Of the product

To judge the quality of the material itself once finished and/or to make decisions for their use

External experts, selectors, producers

Disciplinary environment

Didactic environment

Technological environment

Evaluation protocol, interviews, discussion groups

Of the results

To determine the uses and the operation of the material in a teaching-learning context

To compare the material with others for effectiveness

Real users, investigators, selectors

The users' opinion

Utility

Relationship cost-benefits

 

Questionnaires, scales of attitude, aptitude tests

  Chart 1. Methodological perspectives of the evaluation of educational materials in technological support

 

Before instruments structured as those that we have just presented, more flexible proposals appear that lead to the identification of a series of approaches that appear so that it is the user, selector or appraiser that determines the moderation and reflection about the relevancy of their presence or absence. In this line he/she is the instrument promoted by European Academic Software Award (Baumgartner and Payr, 1997) developed around twelve approaches: accuracy, relevance, covering, interaction, learning, use, sailing, documentation, interface, use of the computer, adaptability and innovation. We also find numerous initiatives guided towards the adaptation of the ISO norms for the software and web material [2].

A general critique of the software evaluation is the low reliability and validity of many of the instruments designed for such an end report is circulated among the community of educators and directors, and the necessity of standard consensuar of quality whose presence is not to simply check mediating control lists, but through a more qualitative, contextual and intensive analysis of the material and on the part of different agent appraisers.

1.1.3. Evaluation of the technological platforms

The evaluation of the technological platforms is guided to value the quality of the virtual environment or virtual campus through which e-learning is implemented. The dimension and functionality of a virtual campus can vary substantial as it concerns giving support to a course (or courses) or to a whole institution, as in the case of the virtual universities.

In the market numerous standard platforms exist with possibilities of adaptation to the necessities of the different types of on-line formation, objectives of the same one and users, but can also be created ad hoc by the institution that adopts a solution e-learning. 

Acquired or created, the virtual campus is a dynamic element that evolves parallelly to the e-learning solution. From there the important thing is that he/she acquires the formative evaluation or the process guided to the progressive improvement of the virtual environment. This evaluation gains ground for the diagnostic evaluation, through which it decides the most appropriate strategy in function of the formative objectives and the necessities. The competition of the market of virtual platforms has forced to balance the services and capacities of the most important, so that it is no longer as important which is chosen but rather in improving it once implemented. Although it doesn't eliminate it the necessary stage of analysis of necessities, without the one which often are carried out in large technological investments without knowing for what it will be needed.

When determining the potential quality of a virtual campus, he/she should be able to generally determine that it is:

.. Stable and reliable

.. Tolerant to shortcomings

.. Standard in implementation of contents and technological resources

.. Agile and flexible

.. Current and intuitive to facilitate the interaction with the user

Most existing instruments for virtual evaluation (designed by the own distributors of platforms to establish comparisons, as well for independent agencies or universities) campus, they are guided to determine the characteristics of the same ones in function of a series of analysis (chart 2) categories.

At the base of these categories initiatives that models have elaborated exists a standard of quality of virtual platforms, among those that we highlight: Cybernetic Model for Evaluating Virtual of Learning Enviroments (Britain and Bark, 1999) guided towards the interrelation of the following aspects: negotiation resources, coordination, advising, individualization, self-organization and adaptation; or Quality Standards on the Virtual Campuses developed by the Virtual University of Pennsylvania, (http://www.vup.org/standards ), based on user's interface, the stocking, the software, the permits and licenses and the accessibility.

A more qualitative and more flexible model is the pattern ACTIONS of Bats (1999) that is specially guided to making decisions before introducing a specific technological method, and it allows evaluation of the advantages and inconveniences with relationship to five components (whose terms coincide with the initials of the pattern): access, costs, teaching and learning, interactivity and facility use, organizational issues, novelty and speed. It is the only initiative that introduces the estimate of economic variables, being applicable to any technology used by the e-learning.

 

Categories

Analysis

Cost

Hardware requirements and software

 

Characteristic

Development capacity

 

 Tools for the student

 

Tools for the instructor

 

Tools for the administrator

 

General and of extra services

Under what operating systems and navigators work, languages that it supports.

Technical, services and support that he/she offers.

Possibility to implement new functions, of carrying out reports, tests, support of VMRL to create a three-dimensional environment.

 Interaction that allows (synchronous / asynchronous, grupal / singular), access to resources (library, secretary, group work, evaluation.

What they allow him/her to make without necessity of programming in html (tests, contents, instructive design.)

What they allow him/her to make (to give authorizations, support to the user, to the educational one, I register, personalization of messages.)

  Chart 2.  Categories of analysis of technological platforms for e-learning

 

In spite of these and other experiences, we coincide with European Network on Intelligent Technologies for Smart Adaptive Systems (http://www.eunite.org) in that he/she still lacks to settle down and consensuar approaches at European level on the virtual campus, related with the establishment of standard of Acreditation for their quality, as well as orientations in the development of courses that you/they implement through these platforms, or the establishment of norms related with the copyright and the royalties, so that the evaluation can be carried out comparable categories.

 

1.1.4. Financial evaluation

One of the evaluations that is claiming more attention, especially on the part of the companies, is the one related to economic mensuration factors. The installation of a solution e-learning requires an initial important investment that is unjustifiable, from the financial point of view, if finally a return of this investment cannot be evidenced. It is what is denominated in economy ROI (Return On Investment). It is about a very simple formula (ROI = benefits / costs) to value the prospective return of an investment. Although behind the simplicity of concepts they hide many shades, especially when they are applied to the field of the formation, and still more than the e-learning.

The formation in the organizations produces benefits and it generates costs. The problem resides in how to measure or to determine the aquired benefits, beyond the positive value that represents for people and the own organization for itself. The challenge is not easy, since the most productive benefits in the formation are the most intangible and difficult of quantifying (satisfaction, initiative and leadership, and the abilities characteristic of people that configure the organization), while the most operative, although they produce short term results, they owe to the result of mechanical (increase of the productivity, saving of time.) knowledge Horton (2001).

In spite of the evident difficulties that it presents the financial evaluation of the formation and of the e-learning, experiences and proposals exist for their planning and execution, among those that highlight a taken model of Duart (2001b), inserted in the process of planning of the formative action. The pattern presents an orderly series of actions with a list of estimate indicators for each one of them.

The importance of the ROI like evaluation instrument resides in not exclusively attributing to the formation the derived benefits of the improvement of abilities and knowledge, but in being able to estimate as this improvement rebounds in the economic results of the organization, this way transforming it into an instrument to control the effectiveness and efficiency in the application of the investments.

 

 

Work

Evaluation instruments

Design

To define the objectives of the formative action

To edit, in agreement, objectives with the strategy of the institution and with the operative of the business unit that will implement it

Concrete and concise sentences that express how results hope to be achieved

To define the objectives of the participants' learning

To edit objectives of agreement learning with the participants

Commitment of each one of the participants with their objectives to the beginning of the formative action

Relationship chart between the learning objectives and the prospective results

To determine the formative modality (present, e-learning, dual, etc.)

To analyze what formative modality it can achieve the prospective objectives in a more efficient way

Comparative chart of advantages and inconveniences of the valued formation modalities

Valuation of the resistances or existent motivations in each modality

Valuation of the costs of introduction of a formative new modality in function of the prospective results

To determine the benefits of the formative action

To expose the prospective benefits, so much quantifiable as not, for the organization and for the participants of the modality of elected formation

To establish the prospective results

To expose the results in quantifiable data, also pointing out achievements and qualitative prospective benefits

To establish the prospective (clients / suppliers; internal or external) levels of satisfaction

It lists concrete prospective benefits expressed in percentages of improvement

It lists from internal benefits to the business unit, characteristics of the participants, clients and suppliers of the unit

It lists achievements expressed in short periods of time

Communication plan, internal and/or external to the own business unit that includes the objectives and the prospective achievements

To determine the costs of the formative action

To expose the costs of the elected modality, as much for the institution as for the business unit or the participants

It lists costs of the formative action

To calculate the ROI of the formative action

To analyze the benefits, the investments and the costs in function of the objectives and prospective results

Calculation of the ROI

Publication of the calculation of the ROI

Implementation

Development of the formative action

To develop the formative action of agreement with the foreseen planning

Collection of information during the process

Information of the process to the participants and the beneficiaries of the formative action

Correction of deviation errors on the planning

Evaluation

Evaluation of the formative action

To evaluate the action starting from the different instruments used

Analysis and diffusion of the results of the evaluation

Evaluation of the carried out and implemented design

To value improvements to carry out in the design of future formative actions starting from the omissions, weaknesses or strengths of the current design

It lists elements to incorporate, maintain or avoid in future designs of formative actions

 

 

 

 

 

 

  Chart 3. I model of evaluation of the ROI for solutions e-learning (Duart, 2001b)  

1.2. The evaluation of global focus

It is the evaluation that presently has the total set of elements that you/they intervene in an e-learning solution when establishing lines and criterion to negotiate or to evaluate their quality.

1.2.1. Evaluation and management of the quality

Sustained by positions of the systemic focus (Pérez Juste et al., 2000) and of the current paradigm of the complexity (López Rupérez, 1997) and based on the concept of continuous (Deming, 1981) improvement, the management of the quality is distinguished for its global and integral focus, being an organizational strategy and a management methodology that he/she makes all of the members of an organization participate with the fundamental object of improving its effectiveness, efficiency and functionality continually.

The installation of a system of quality in an organization comes based by the following principles (González, 2000: 66-68): 

.. A process guided to the satisfaction of the necessities and the addressees' expectations

.. It permanantely improves everything that the organization seeks to reach on the base of some clear and explicit objectives

.. The guarantee of the quality of the internal processes as measure to reach the quality of the product

.. The prevention rather than the supervision and detection of errors

Furthermore the importance of:

.. Leadership

.. Working in a team, the quality is the work of all

.. Systemic resolution of problems

.. Basing decision making on objective data

..The agility in the transmission of information

.. The formation of involved people

On the bases of these principles different tools have been generated, as the norms ISO and the evaluation models characteristic of TQM (Total Quality Management).

One of the most extended tools is the application of the norms ISO, that group of norms (of non prescriptive character) that you/they simply demand that an organization: a) defines and plans its processes, b) documents them in a correct way, c) checks its attitude, and d) guarantees the control and revision of the same.

The principles on which this group of norms is articulated are the following:

.. The organization possesses clear objectives of quality

.. Clear agreements exist among all the participants

.. The organization possesses the necessary resources to obtain the required level of quality

.. All the processes and systems are subjected to controls, with evaluations and modifications when it is convenient

.. Everything necessary to guarantee the quality of the document

.. The registrations of the quality allow  verification and confirmation of the quality guarantee

The interpretation for the terms of e-learning contexts can be found in chart 4, by way of proposal, and picking up the recommendations of Van De Berghe, W. (1997).

 

Term of ISO

Interpretation for an organization (educational or formative) that adopts a solution e-learning

Supplier

The institution or organization that imparts the teaching or formation

Clients

Pupil or participants

Product

Study, programs, currículum

Executive panels

General address / Center address / Principal / Committee

Contract

All the types of agreements with the clients: enrollment, access to the communication system, distribution of materials, advising, accreditation

Design

Definition of the specificities of the technological platform and their instructive and administrative capacities and the systems of security

Purchases

Acquisition of material and human resources and necessary services: faculty recruiting, software licenses, hardware purchase.

Processes

Administration process of the teaching or training: methodology (working in group, individual), use of the communication spaces, materials (on-line, off-line, multimedia, textual), evaluation (present or correspondence).

Inspection and tests

Evaluation of the formative action, faculty, materials, technology,.. on the part of the pupil or participants

Calibration

Validation of the used evaluation systems

  Chart 4. Equivalence of concepts of the norms ISO to the environment of the e-learning

 

The TQM, like EFQM (European Foundation Quality Management) or the more recently implanted IMC [3] (I Square of Integral Control) possess some implementation phases and a series of key areas or basic approaches of quality which articulate the pattern.

It is in all the cases of evaluation tools frequently used in the organizations and at the present time they are adapting to the contexts of virtual formation. But like some authors (Mateo, 2000b, Barberá, 2001) have noticed it is from next models to the management that to the educational processes. These models fundamentally emphasize the aspects of organizational management, the client's satisfaction, or relationship cost-benefits, aspects that are undoubtedly important, but insufficient in an activity whose nucleus is the teaching-learning.

Barberá (2001) proposes as dimensions that should be evaluated in the virtual formation: the scenario in which the educational action takes place (psycopedagogical foundations, general structures of the system, etc.); the proposals of the participants involved in the instruction (motivations, objectives and cognitives demands) process; the instructional (students' lists, of teachers, and of the same institution) agents; the intervention and the educational (organization of the educational activity, interaction patterns and virtual speech) interaction; and the same construction of knowledge (characteristic of the knowledge, dynamic and construction types).

Many of these dimensions are picked up through the practice of the benchmarking that is charging a great importance in the evaluation of the e-learning.

1.2.2. Practical of evaluation of the e-learning based on the benchmarking

Benchmarking is the process that allows a center or organization to be compared with another that obtains excellent results of quality, with the purpose of emulating it. In the e-learning environment this system seeks to offer tools and indications to improve the actions starting from the observation, comparison and cooperation based on the good practicals. Guidelines for Electronically Offered Degree and Certificate Programs of the Council of Regional Acredited Commissions of the United States and the Benvic Project “Benchmarking of Virtual Campuses” of the European Union are found in this line.

1.2.2.1. System benchmarking BENVIC

In the European Union the Benchmarking of Virtual Campuses Project (European Commission 2002) is being developed by eight university instituions of member countries in the framework of the MINERVA program since 2001. This project, well-known by the name BENVIC, is oriented towards the development and application of evaluation approaches to promote standards of quality in the virtual campus and in the general on-line training.

The objectives of the project are:

.. To develop, validate and establish a system of evaluation of experiences with virtual campus or e-learning solutions in the European context.

.. To promote a collaborative net to implement the evaluation through comparison and benchmarking.

.. To promote shared knowledge.

.. To develop a map of related competencies with the design and the implementation of virtual campus in order to help the institutions improve their practices and to achieve the quality for their e-learning solutions.

Phases of the project:

.        To define indicators of the quality of e-learning and the map of competencies

The characteristics of the adopted indicators are: a) they are oriented towards the change, b) they are contextual, c) they combine external and internal, subjective and objective, qualitative and quantitative benchmarking.

As large indicator areas the project defines the following:

.. Service to the student

.. Learning resources

.. Faculty support

.. Evaluation

.. Accessibility

.. Efficiency (related to the financial aspect)

.. Technological resources

.. Institutional execution

The different indicators of these areas are contained in three types of measures: of structural, practical and executive.

The map of competencies is a tool that allows the partner BENVIC to verify the level of abilities in its institution in relation to the handling of the virtual campus that possesses the faculty, the pupil, the technicians and the organization. Therefore it can estimate the difference between the required abilities and the available abilities. At the same time it facilitates a pedagogical audit whose results can be: an aid, a formation plan, sharing of knowledge, field certification, etc.

Four types of interrelated competencies have been considered:

.. The institutions own competencies

.. Common competencies for different sectors

.. Sectoral competencies, those adapted from a traditional campus to the virtual campus

.. Virtual competencies, related exclusively to the virtual campus

.        Inviting various institions to be self-evualted for the first time

The first institutions that participate continue on to integrate the inicial data into the database. There have been five case studies.

.        Determining benchmarking partners (demonstrating good performance)

Examples of good performance can already be established starting from the first experiences, they will serve as a comparison for other members.

.        Creating club BENVIC with a database of different institutions

The database is going to enlarge with the entrance of new institutions for self-evaluation .

 

Indicator: Service to the student

Structural Measures

Punctuation

(0-1-2)

How it has been implemented in their campus

1

Availability of systems and services for the support of the pedagogical communication between the pupil and the faculty (inter-intra)

 

 

n

 

 

 

 

Practice measures

Punctuation

(0-1-2)

How it has been implemented in their campus

n

The organization invests in resources to provide an effective and friendly interface for the pupil

 

 

 

Execution measures

Reached level

Comments

n

% of exams carried out on-line

 

 

 

Does he/she believe that some measure should be improved?

Does he/she believe that an important measure is lacking in the list?

Describe the measures to improve and the form of carrying them out

  Chart 5. Example of indicators belonging to a great category in the BENVIC Benchmarking system

 

Phases of the BENVIC Benchmarking process:

1. In the first phase, a representative of the virtual campus uses the indicators of the template to carry out the initial diagnosis in relation to indicators of the structure (available resources in the virtual campus to obtain the objectives which includes the human competencies, the platform, administration and management); the practice (how the virtual campus uses the resources in relation to the organization strategy in access and pedagogical design); and the execution (the impact of the results: learnings, cost-benefit, technological effectiveness).

2. In the second phase the institution applies the results from the initial diagnosis to the practice of the organization. Starting from here key areas that need to be improved are identified. The team examines the BENVIC database and identifies "benchmarking partners" of the BENVIC club, which are indentified as good examples in the areas of those indentified as deficient. The measured punctuations of the partners can be consulted in those areas and the form in which the benchmarking partners have solved them.

3. In the third phase the virtual campus is taken to a process of improvement and of benchmarking. The knowledge and the generated learning goes on to become part of the BENVIC benchmarking system.

 

Chart 1.  Application process of the BENVIC Benchmarking indicators

 

When the campus has been improved the administrators or partners enter the data again and they can go as far as transforming it into a new benchmarking that will serve as an example of good performance and comparison for new partners.

The BENVIC club system can work in the measure that is well administered and in the measure that different institutions are willing to share their own experiences and to be involved in plans of continuous improvement. It is one of the first systematic contributions to the elaboration of the standard of quality in on-line training.

1.2.2.2. Good Practices in the frame of Accreditation and of North American universities

In the North American context, the Regional Commissions for Accreditation have elaborated a guide of "good practical" in the superior education offered electronically: Best Practices for Electronically Offered Degree and Certified Programs (Middle States Commission on Higher Education, 2001). The development of the guide has relied on the initial collaboration of expert organizations in the field, and it is open to possible changes as the on-line own superior education evolves. These guides are being used for the Accreditation of university institutions that offer on-line training.

Those considered good practices have been divided into five components, each of which is directed to a particular area of the outstanding institutional activity for the electronic offer. These areas are:

.. Institutional context

.. Curriculum and training

.. Support for the faculty

.. Support for the pupil

.. Evaluation

Each component begins with a general description followed by numbered paragraphs directed to specific problems, describing the essential elements for quality. Next they list protocols with questions guided to determining the existence of those elements in the external or internal revisions of the on-line educational activities.

For the system of the practical good North American, several universities guides have elaborated guides or standards of quality of online training. Among them Virtual Michigan University is highlighted (http://standards.mivu.org/standards/ ) and its Quality Standards on on-line the Courses (2002), contained in four large areas: technology, usability, accessibility and instruction design. Or the Indiana University Center for Research on Learning and Technology and their Seven Principles [4] of Effective Teaching. A Practical for Online Evaluating Courses: good practicals that promote the relationship of the pupil with the faculty, good practicals that impel cooperation among students, good practical that favor active learning, the immediate, immediate feed-back, good practicals that emphasize time on taks, good practicals that generate high expectations and good practicals regarding diverse forms of learning.  We can find a complete list of all the universities that have formed good practice guides and standards of quality of e-learning on the Minnesota State Colleges and Universities page (www.oit.mnscu.edu/mitss/peerreview1.htm). 

1.2.3. Empirical studies of benchmarking

Among the empirical studies of the practice of the emphasized benchmarking, Quality On the Line developed by the Institut for Higher Education Policy in 2000, oriented to determining the decisive benchmarks of the quality in online training [5], and whose results reveal, in a similar way to the exposed experiences, 24 important and good practices:

1. Existence and installation of a documented technological plan that includes electronic measures of security.

2. Maximum reliability of the technological system.

3. Existence of a centralized system of support, to create and to maintain the educational infrastructure.

4. Use of guidelines in the design and development of the courses.

5. Upgrading and periodic revision of the materials.

6. Designing the teaching/learning process in a format that obligates the student to  involve himself/herself.

7.  Essential role of the pupil´s interaction with himself/herself and with the faculty, and to facilitate it through diverse means (e-mail, chat, in person, etc.).

8. Offering constructive answers to the consultations and contributions of the pupil, and doing it in a short time.

9. Teaching the pupil the appropriate methods of research.

10.  Before beginning the course, giving the pupil enough information to determine if they have the necessary motivation and appropriate resources.

11.  Giving complementary information about the course to the students, includying objective, concepts and ideas, and clearly specifying what results are expected from the program.

12.   Giving access to an adequate library that includes accessible resources through the internet.

13.  Setting expectations relating to handing in and corrections of activites.

14.   Giving the pupil enough information on the programs, including admission requirements, prices, books, accessories, technical requirements and support services.

15.  Giving the pupil information and training on how to obtain resources through databases, library nets, public services, news servicies and other sources.

16.  Offering the pupil easy access to technical support during the whole course, detailed instructions on the operation of used the technological means and practical sessions before the start of the course.

17.  Offering a quick and precise answer to the consultations directed to the support service for the pupil and preparing a structured system for attention to complaints.

18.  Giving technical assitance to the faculty and encouraging its use.

19.  Facilitating the faculty's transition from classroom strategies to on-line strategies.

20.  Maintaining support for the faculty during the whole course.

21.  Providing information to the faculty regarding how to solve problems derived from what the student can do with the data he/she receives.

22.  Forming a process of evaluation of the pedagogical effectiveness of the program, applying specific demands.

23.  Evaluating the effectiveness of the program with data of enrollment, costs and innovative and appropriate applications to the technology.

24.  Periodically revising the predicted learning results in order to guarantee that they are clear, useful and appropriate.

Other aspects did not turn out as outstanding, for example, the promotion of team work, the division of courses in modules or the consideration of different learning styles.

In short, high quality online training, according to this study, must be equally emphasized in a group of factors: an impeccable technical operation; an educational model to the client's measurement; good materials, good curriculum offering; independent support systems to solve technical (related to the computer system), scientific (related with the matter of the course) and pedagogical (related with the pattern of teaching learning) problems; trained and informed students and faculty; and a good internal organization.

Another study carried out in Swinburne University of Technology (Cashion and Palmieri, 2002) Quality in On-line Learning, identified 6 great dimensions related to good practicals (technology, support, adaptation to the objectives, materials, communication and evaluation), many of them coinciding with the previous ones. However we will find ourselves lacking, in general, the evaluation of the generation of knowledge and investigation in the university context. The virtual university should also be evaluated in relation to the structures that promote research, scientific fields and of specialization in those that research, diffusion of the results or the research staff itself (Continues them, 2001, mentioned by Sangrà, 2001: 9)

In short, the evaluation through bechmarking is formed by a selfevaluation process and comparison of some institutions in relation to what is considered good practice, be they real cases or theoretical definitions established by experts. 

By way of synthesis

Different perspectives and solutions try to give answer to the topic of the evaluation of the quality of the e-learning, some more economically based, others on the environment of the management, others more technologically ones and some pedagogically. However, until this moment one by itself cannot over all the necessities of a coarse and complex environment that embraces numerous variables and factors, and that has just begun. From different organizations, institutions and involved people, the need to create standards of quality, certify the quality and evaluate the quality of the online training in its different contexts and levels is demanded.  Furthermore, add the need to satisfy the training demands of the new society and to generate a quality culture and continuous improvement. From the methodological point of view it is also necessary to point out the importance of exploring models of evaluation of more qualitative orientation.

References

Belanger, F. y Jordan, D.H. (2000) Evaluation and Implementation of Distance Learning: technologies, tools and techniques. London. Idea Group Publishing.  

Barberá, E. (2001) (Coord.) La incógnita de la educación a distancia. Barcelona. Horsori.  

Bates, A.W. (1999) Research and Evaluation. En A.W. Bates, Managing Technological Change,    pp. 198-210. California. Jossey-Bass.  

Baumgartner, P. y Payr, S. (1997) Methods and practice of software evaluation. The case of the European Academic Software Award. Proceedig of ED-MEDIA 97 & ED-TELECOM 97, AACE.  

Britain, S.  y Liber, O. (1999) A framework for Pedagogical Evaluation of Virtual Learning Enviroments. University of Wales. Consultada el consultada el 5-mayo-2003 en http://www.leeds.ac.uk/educol/documents/00001237.htm  

Cabero, J. (2001). La evaluación e investigación sobre los medios de enseñanza. En J. Cabero, Tecnología Educativa. Diseño y utilización de medios de enseñanza, pp. 447-490. Barcelona. Paidós.  

Cashion, J. y Palmieri, P. (2002). Evaluation of Quality in On-line Learning. TAFE School of Social Sciences. Swinburne University of Technology. Consultada el consultada el 12-abril-2003 en http://www.tafe.swin.edu.au/ncver/  

Deming, W.E. (1981). Management and statistical techniques for quality and productivity. New York. New York University.  

Duart, J.M. (2001a). Avaluació de la qualitat docent en entorns virtuals d’aprenentatge. Consultada  el 5-mayo-2003 en http://www.uoc.edu/web/cat/art/uoc/0109041/duartmartin_imp.html  

Duart, J.M. (2001b). ROI i e-learning: més enllà de beneficis i costos. Consultada  el 15-mayo-2003 en http://www.uoc.edu/web/cat/art/uoc/duart0902/duart0902.html  

EPISE (2000). Evaluación de la formación en las empresas españolas. Barcelona. EPISE.  

European Commission DG Education and Culture (2002). Benchmarking of Virtual Campuses Project. Barcelona. Universitat Oberta de Catalunya.  

Galvis, A. (2000). Evaluación de materiales educativos computerizados por juicio de expertos. En   A. Galvis, Ingeniería de software educativo. Colombia. Universidad de los Andes.  

González, T. (2000). Evaluación y gestión de la calidad educativa. En T. González (Coord.)  Evaluación y gestión de la calidad educativa. Un enfoque metodológico, pp. 49-80. Málaga. Aljibe.  

Harvey, L. y Green, D. (1993). Defining quality. Assessment and Evaluation in Higher Education, 18 (1), pp. 9-34.  

Horton, W. (2001). Evaluating e-learning. California. ASTD (American Society for Training and Development).  

Institute for Higher Education Policy (2000). Quality on the line. Benchmarks for Succes in Internet-Based Distance Education. Washington. National Education Association.  

ISO/IEC 9126 (1991). Information technology. Software product evaluation. Montreal.  

Kaplan, R.S. y Norton, D.S. (1997). El cuadro de mando integral. Barcelona. Gestión 2000.  

Kirkpatrick, D.L. (1999). Evaluación de acciones formativas: los cuatro niveles. Barcelona. EPISE-Gestión 2000. 1ª edición 1994.  

León, J.A. (1998). La adquisición de conocimiento a través del material escrito: texto tradicional y sistemas de hipertexto. En C. Vizcarro y J.A. León (Ed.), Nuevas tecnologías para el aprendizaje. Madrid. Pirámide.  

López Rupérez, F. (1997). Complejidad y educación. Revista Española de Pedagogía, 206, pp. 103-112.   Mantyla, K. (2000) Evaluating Program Succes. En K. Mantyla, Distance Learning Yearbook, pp. 259-287. New York. McGraw-Hill.  

 Mateo, J. (2000a). La evaluación del profesorado y la gestión de la calidad de la educación. Hacia un modelo comprensivo de evaluación sistemática de la docencia. Revista de Investigación Educativa, 18 (1), pp. 7-34.  

Mateo, J. (2000b). La evaluación educativa, su práctica y sus metáforas. Barcelona:. Horsori.  

McArdle, G.E. (1999). Training Design and Delivery. Alexandria, VA. American Society for Training and Development.  

Mendoza, L., Pérez, M. y Rojas, T. (2001). Modelo sistémico para estimar la calidad de los sistemas de software (MOSCA). Acta Científica Venezolana. Universidad Simón Bolívar. Consultada  el 15-mayo-2003 en http://lisi.usb.ve/publicaciones  

Middle States Commission on Higher Education (2001). Interregional Guidelines for Electronically Offered Degree and Certificate Programs. Middle States Commission on Higher Education.  

Navarro, E. (1999). Evaluación de materiales multimedia. Comunicación y Pedagogía, 157, pp. 36-39.  

Oliver, M. (1998) (Ed.) Innovation in the Evaluation of Learning Technology. London. University of North London.  

Pérez Juste, R., López Rupérez, F., Peralta, M.D. y Municio, P. (2000). Hacia una educación de calidad. Gestión, instrumentos y evaluación. Madrid. Narcea.  

Pineda, P. (2002). Formació, transferència i avaluació: un triangle complex. Revista Econòmica de Catalunya, 44, pp. 79-89.  

Pineda, P. et al. (1999).. ¿Cómo se evalúa la formación en las organizaciones?. Capital Humano, octubre y noviembre, pp. 126-127. 

Promoting Multimedia Acces Education and Training in European Society (1999). Memorandum of Understanding. Consultada  el 10-junio-2003 en http://www.ecotec.com/sharedtetriss/news/mou1198/mou3-slidepresentation/  

Ramussen, K. y Davidson, G.V. (1996). Dimensions of learning styles and their influence on performance in hypermedia lessons. CD-ROM Proceedings from the anual ED-MEDIA/ED-TELECOM conference. Article nº 385.  

Richardson, J. (2001). An evaluation of Virtual Learning Enviroments and their learners: do individual differences effect perception of virtual learning enviroments. Interactive Educational Multimedia, 3, october. Consultada  el 25-abril-2003 en http://www.ub.es/multimedia/iem/  

Rodríguez, J.L., Rubio, M.J. y Escofet, A. (2001). Certificación de la calidad de materiales multimedia educativos. E-CumLaude. Universidad de Barcelona. Documento inédito.  

Rosenberg, M. (2001). E-learning: Estrategias para transmitir conocimiento en la era digital. Bogotá. McGraw-Hill Intramericana.  

Sangrà, A. (2001). La qualitat en les experiències virtuals d’educació superiorConsultada  el 5-mayo-2003 en http://www.uoc.edu/web/cat/art/uoc/0106024/sangra.html  

Sigalés, C. (2002). El potencial interactiu dels entorns virtuals d’ensenyament i aprenentatge en l’educació a distància. Consultada  el 5-mayo-2003 en http://www.uoc.edu/web/cat/art/uoc/sigales0102/sigales0102_imp.html

 

Silvio, J. (2000). La virtualización de la universidad. ¿Cómo podemos transformar la educación superior con al tecnología?. Caracas. IESALC/UNESCO.  

Tait, A. (Ed.)(1997).  Quality Assurance in Higher Education: Selected Case Studies. Vancouver. The Commonwealth of Learning.  

Van De Berghe, W. (1997). Aplicación de las Normas ISO 9000 a la enseñanza y la formación. Interpretación desde una perspectiva europea. Luxemburgo. CEDEFOP.  

Van Slyke, C. Kittner, M. y Belanger, F. (1998). Identifying Candidates for Distance education: A telecommuting perspective. Proceedings of the America’s Conference on Information Systems,   pp. 666-668. Baltimore.  

 

Referencias de portales 

European Network on Intelligent Technologies for Smart Adaptive Systems (EUNITE) (2003)  <http://www.eunite.org>  

Instructional Management Systems Project (IMS) (2000) <http://ims.org>  

MOU-Perseus project  (2000) <http://www.perseus.tufts.edu>

Quality Standards on the on-line Courses of Michigan Virtual University  (2002)  <http://standards.mivu.org/standards/>  

Quality Standards on the Virtual Campuses of the Virtual University Penn (2001) <http://www.vup.org/standards/>

 

[1] Hypertextual materials, or hypermedia, are especially appropriate in virtual learning environments because of the fact that they do not come from a sequential form; the user has applied an interactive process through which one can derive simultaneous information from different levels. The associations that permit a hipertextual system, in fact, are more similar to the form as the human being aquires a nonlinear interpretation of the knowledge (León, 1998). But this is only achieved if the material is of a technical and pedagogical quality.

[2] ISO/IEC 9001-Quality Systems-Model for quality assurance in design/development, product, installation and servicing. < http://www.iso90012000software.com/17025.html>

[3] From English, Balanced Scorecard, the first formulations appear at first from the Nineties and recently have displayed the entire model. (Kaplan y Norton, 1997).

[4] It explains fact of the seven defined principles by the American Association for Higher Education (http://www.aahe.org ), and about the principles in which José Silvio (2000:207) raises a series of doubts and questions.

[5] The study divided from 45 references or good practices identified throughout the bibliography that were relevant to the topic and institutions with experience en e-learning. In the study, six institutions participated that had important experience in distant education, are recognized as leaders within the field, are accredited by regional agencies and offer online teaching programs. The benchmarks were identified as much by the organization as by the student (the two principally implied agents in training and evaluation).

   


TO KNOW MORE / PARA SABER MÁS

This article has been cited in... / Este artículo ha sido citado en...

* Cabero, J. (2004). La investigación en tecnologías de investigación. Bordón, vol 56, n.3 y 4, pp. 443-468 [Párrafo añadido el 10 de Abril de 2007/ Paragraph added April 10th 2007].

* Lavigne, Gilles Organista, J y Aguirre, L (2006).Evaluación de la modalidad híbrida, presencial/en línea, por estudiantes de posgrado en educación. Actualidades Investigativas en Educación”, Volumen 6, “.1., 1-25. http://revista.inie.ucr.ac.cr/articulos/1-2006/archivos/evaluacion.pdf [Párrafo añadido el 10 de Abril de 2007/ Paragraph added April 10th 2007].

* Sandia, B., Montilva, J. y Barrios, J. (2005). Cómo evaluar recursos en línea. Educere, año 9, n. 31. 523-530. http://redalyc.uaemex.mx/redalyc/pdf/356/35603113.pdf  [Párrafo añadido el 10 de Abril de 2007/ Paragraph added April 10th 2007].

* Cokbaine, J & Casas, I. (2005). Un metamodelo OLAP para la evaluación de ambientes TEL. Un metamodelo OLAP para la evaluación de ambientes TEL. Rev. Fac. Ing. - Univ. Tarapacá, vol. 13 no. 1, 9-20.  Disponible en www.scielo.cl/pdf/rfacing/v13n1/art02.pdf [Párrafo añadido el 10 de Abril de 2007/ Paragraph added April 10th 2007].

* Centro Nacional Información y Comunicación Educativa (2004).  Estudio comparativo internacional sobre modelos estandarizados de evaluación y marcas de calidad de materiales educativos digitales. Madrid: Ministerio de educación y Ciencia, Serie Informes CNICE. http://ares.cnice.mec.es/informes/11/versionpdf.pdf [Párrafo añadido el 10 de Abril de 2007/ Paragraph added April 10th 2007].

* Moreno, Rafael, Martínez, Rafael J., Martín, Irene y Trigo, Eva (2005). Utilidad de las webs como herramienta auxiliar en la docencia universitaria presencial.   IberPsicología, 2005: 10.3. http://www.fedap.es/IberPsicologia/iberpsi10/congreso_lisboa/moreno2/moreno2.htm [Párrafo añadido el 10 de Abril de 2007/ Paragraph added April 10th 2007].

* Redondo, Sara (2006). Propuesta estructural de diseño de estándares de calidad pedagógica para entornos virtuales de formación. III Congreso ONLINE del Observatorio para la CiberSociedad Noviebre. http://www.cibersocietat.net/congres2006/gts/comunicacio.php?id=510&llengua=es  [Párrafo añadido el 10 de Abril de 2007/ Paragraph added April 10th 2007].

* Zepeda, M. S. (2005 ). Modelo de Evaluación de la Calidad en Docencia Universitaria en Educación a Distancia en UNITEC. Cognición, n. 3, 56-70.  http://www.cognicion.net/cognicion/files/cognicion3completa.pdf. [Párrafo añadido el 10 de Abril de 2007/ Paragraph added April 10th 2007].

* Buzón,  O.  (2005).   La  incorporación  de  plataformas  virtuales  a  la  enseñanza:  una  experiencia  de  formación  on-line  basada  en  competencias,  Revista  Latinoamericana  de  Tecnología  Educativa,  4  (1),  77-98.  [http://www.unex.es/didactica/RELATEC/sumario_4_1.htm].  http://158.49.119.99/crai/personal/relatec/VOL4_1/OlgaBuzon.pdf  [Párrafo añadido el 10 de Abril de 2007/ Paragraph added April 10th 2007].

* Chávez, F.J. y otros (2006). Evaluación educativa en las modalidades a distancia. Apertura. 6 (4). 44-55. http://redalyc.uaemex.mx/redalyc/pdf/688/68800405.pdf [Párrafo añadido el 10 de Abril de 2007/ Paragraph added April 10th 2007].

 

 


 

ABOUT THE AUTHOR SOBRE LA AUTORA

María José Rubio Hurtado (rubio@trivium.gh.ub.es). Associate profesor of the MIDE Department of the University of Barcelona.  She is a member of the Enseñanza y Aprendizaje Multimedia Research Group of the University of Barcelona´s Instituto de Ciencias de la Educación, where she has participated in different projects and investigations related to the field of ITC application to education and training.  Postal address: Edifici de Llevant, dpto. MIDE, Pesseig de la Vall d´Hebrón, 171, 08035 Barcelona.- Profesora titular del Departamento MIDE de la Universidad de Barcelona. Es miembro del Grupo de Investigación Enseñanza y Aprendizaje Multimedia del Instituto de Ciencias de la Educación de la Universidad de Barcelona, desde donde ha participado en diferentes proyectos e investigaciones relacionadas con el ámbito de las TIC aplicadas a la educación y la formación. Dirección postal: Edifici de Llevant, Dpt. MIDE, Passeig de la Vall d’Hebrón, 171, 08035 Barcelona. 


ARTICLE RECORD / FICHA DEL ARTÍCULO

Reference /

Referencia

Rubio, Maria José (2003). Focus and models of evaluation of the e-learning. RELIEVE, v. 9, n. 2. http://www.uv.es/RELIEVE/v9n2/RELIEVEv9n2_1eng.htm.

Title / Título

Enfoques y modelos de evaluación del e-learning. [Focus and models of evaluation of the e-learning]

Authors / Autores

María José Rubio 

Translators / Traductores

Michelle Nguyen, Miguel Bacigalupe

Review

Revista

Revista ELectrónica de Investigación y EValuación Educativa (RELIEVE), v. 9, n. 2

ISSN

1134-4032

Publication date /

Fecha de publicación

 2003  (Reception Date:  2003 Sept 16; Approval Date: 2003 Oct. 19  Publication Date: 2003 Oct. 22 )

Abstract /

Resumen

The aim of this article is to present the state of arts about e-learning evaluation. Different evaluative approaches, different models, tools and experiencies are showed, in order to determine the quality of virtual learning. Two main approaches are showed: partial evaluation, with emphasis in some aspects of e-learning (materials, technology resources, teaching, etc.) and global evaluation, that uses management quality models and the practice of benchmarking

 

El objetivo del artículo es presentar el estado de la cuestión sobre la evaluación del e-learning. Con este propósito se muestran los diferentes enfoques evaluativos, así como diferentes modelos, herramientas y experiencias encaminadas a determinar la calidad de la formación virtual, o e-learning. Se destacan dos enfoques principales, la evaluación de enfoque parcial, que enfatiza aspectos diversos del e-learning (los materiales, los recursos tecnológicos, la docencia, etc.) y la evaluación de enfoque global, que utiliza modelos de la gestión de la calidad y la práctica del benchmarking

Keywords

Descriptores

 Evaluación, calidad, formación virtual, aprendizaje virtual. e-learning

 Evaluation, quality, virtual education, virtual learning, e-learning

Institution /

Institución

Universidad de Barcelona (España)

Publication site /

Dirección

http://www.uv.es/RELIEVE 

Language / Idioma

 Español and English version (Title, abstract and keywords in English and Spanish)

 

Volumen 9, n.2

 

© Copyright, RELIEVE.  Reproduction and distribution of this articles it is authorized if the content is no modified and their origin is indicated (RELIEVE Journal, volume, number and electronic address of the document).

 

© Copyright, RELIEVE.  Se autoriza la reproducción y distribución de este artículo siempre que no se modifique el contenido y se indique su origen (RELIEVE, volumen, número y dirección electrónica del documento).

   

 

  [Principal]  [La Revista] [Artículos publicados]  [Publicar en RELIEVE]

 [Home Page]  [ About us]  [ Published Articles]  [To publish in RELIEVE]

Revista ELectrónica de Investigación y EValuación Educativa

  http://www.uv.es/RELIEVE

email: relieve@uv.es

Statistics    Estadísticas