Learning Management Systems (LMS) have acquired a significant relevance as support tools for learning (Britain and Liber, 1999; Melton, 2006; Ellis, 2009) in Higher Education, due to the fact that they allow us to manage contents, to establish synchronous and an asynchronous communication and to manage student assessment (Ross, 2008). However, these organisational changes entail a modification of the tutoring processes and types of activities, while they enable a constant and continuous monitoring of the students’ evolution (Antonenko, Toy and Niederhauser, 2004). Their main interest lies in the space that Information and Communication Technologies (ICT) take up in all sectors of the global society (Castells, 1999; Cohen and McCuaig, 2008). This fact opens up new fields of ICTrelated research, such as the one explored in this study.
We can consider an LMS as “a software system that combines a number of different tools that are used to systematically deliver content online and facilitate the learning experience around that content” (Weller, 2007, p. 5). These environments have evolved in the past few years and they have been used as a complement in diverse learning formats and contents, from a facetoface context to an exclusively virtual one (elearning), including mixed or blearning contexts (DeNeui and Dodge, 2006; Conrey and Smith, 2007; Vaughan, 2007). Their didactic contribution lies in the combination of elements that are specific to traditional teaching (information presentation, accessibility to materials, evaluation of student work) (Yueh and Hsu, 2008), and a series of additional elements that provide multiple communication pathways (including learningcentred social media) (Ellison, Steinfield and Lampe, 2007). Thus, this supports the adjustment of “traditional tools” to the new teachinglearning scenarios conditioned by the potential provided by the use of LMS (Pérez and Garcias, 2007). LMS entail innovative elements that have different characteristics from facetoface education and allow students to take a more active role (Silva Quiroz, 2011). Their integration in the different educational stages favours both the virtual learning and the studentteacher interaction. Consequently, the student’s role is redefined as an individual who generates and transmits knowledge on the net. To this end, the student needs a series of skills aimed at selfregulating the leaning process and favouring the construction of knowledge through information searching, selection, transformation and dissemination (Barberá and Badía, 2004). To attend to the characteristics of LMS and the need to promote an active role for the students, different freeaccess or commercialaccess platforms have been designed and implemented (MartínBlas and Serrano Fernández, 2008): Atutor, Claroline, ILIAS, Chamilo, Moodle, LRN, Teleduc, FLE3, Ganesha, etc. The institutions’ choice is based on criteria such as the users’ needs, the cost and the potential number of users (MartínBlas and Serrano Fernández, 2008). In the educational field, many universities have decided in favour of the Moodle platform. It is an open resource based upon pedagogycal principles (Cole y Foster, 2007; Goyal y Puhorit, 2010), which integrates diverse multimedia resources. For these very reasons, Moodle has become one of the most used LMS in Higher Education (Aydin and Tirkes, 2010; Saito and Ulbricht, 2012; Williams van Rooij, 2012). Moodle is presented as a platform that provides the necessary tools for virtual education (Aydin and Tirkes, 2010; Saito and Ulbricht, 2012; Williams Van Rooij, 2012). In addition, it promotes new learning, easing the access to the material in an organised way (Peat and Franklin, 2002). Thus, Moodle facilitates the development of the teachinglearning processes in elearning, blearning and facetoface education through elements such as interaction (Swan, Shea, Fredericksen, Pickett, Pelz and Maher, 2002), usability (Kirner and Saraiva, 2007) and social presence (Richardson and Swan, 2003). In this sense, it becomes clear that LMS improve learning outcomes (MartínBlas and SerranoFernández, 2009; Núñez et al., 2011; EscobarRodríguez and MongeLozano, 2012), and that teachers who use virtual resources increase student attention and participation, enabling a more meaningful learning process (Soyibo and Hudson, 2000). Other authors such as Steyaert (2005) prove that both LMS and the use of the internet, allow teachers to organise the contents by topics and to manage the organisation of the subject in a more efficient way, enabling a simpler visualisation of the syllabus (Peat and Franklin, 2002). Therefore, the aim of LMS in general, and Moodle in particular, entails interaction with the information and joint work between teachers and students. However, reality seems to be far from this aim, and these environments are often used as mere document repositories. In this context, the interest of studying the students’ perception of the virtual platform’s usefulness in Higher Education processes becomes apparent. With the aim of analysing and trying to collaborate in the improvement of the implementation of LMS, we have designed a scale for its real use in university contexts. It is a fact that the usefulness perceived by the user is extremely important for the success of a technological tool such as LMS (Davis, 1993; Friedrich and Hron, 2010; Sørebø, Halvari, Gulli and Kristiansen, 2009). Several studies based on the Technology Acceptance Model (Davis, Bagozzi and Warshaw, 1989), developed from the Theory of Reasoned Action (Ajzen and Fishbein, 1980; Rus, Pina, Sánchez and Martínez, 2011), which explains the current use of technologies based on the attitudes of the user towards the very use of the technologies, the usefulness and the perceived ease of use, show that there is a positive relationship between the use of a given technology and these three variables. Thus, people’s beliefs about an object will influence their attitude towards it. Therefore, the level and frequency of use of an LMS will be partly affected by the individual attitudes towards it. If we wish the LMS to be integrated in the teaching methods we need to promote adequate attitudes towards these tools. Based on the scarcity of studies in this field (that provide valid and reliable measuring instruments) that help to design and evaluate empirical models based on the Technology Acceptance Model, it seems useful to design a scale of the student’s perception of the usefulness of Moodle within the learning process. Therefore, the ultimate aim of this study is to create a tool that provides valuable information on the integration of Moodle in the university context and that allows for improvements in the teachinglearning environments that integrate LMS. MethodParticipants From the total population of university students enrolled in Educational Sciences degrees in the academic year 201112 we established a nonprobability accidental sample (Arnal, Rincón and Latorre, 1992) comprising 754 subjects. Thus, considering an infinite population, and supposing maximum variability (p=q=.5) and a ksigma=2, the error obtained for the sample is 3.64%. Variables and instruments The instrument consists of a scale designed to evaluate the students' perception of the usefulness of Moodle in the university teaching processes. We chose a survey study based on a quantitative instrument. After the content validation process (with judges), we obtained an instrument composed by 40 items (see table 1). It is a Likerttype scale (Morales, Urosa and Blanco, 2003), with four response options (0=not at all, 1=not much, 2=quite a lot, 3=a lot/very much). The items are distributed in five theoretical dimensions (Moore and Iida, 2010; Palmer and Holt, 2010; AlBusaidi and AlShihi, 2012):
Table 1: Questionnaire items about student perception on the usefulness of Moodle
Content validity is guaranteed through expert judges. The first draft, composed of 32 items, was assessed based on clarity and relevance criteria by eight experts in educational technology, three experts in research methodology and four university students. Based on these assessments we drew up the survey that was administered to the sample. Each expert judge had to indicate whether each of the 32 items assigned to the theoretical dimensions was adequate. In the case an item was deemed inadequate, the judge specified some recommended modifications. Given that we have a qualitative and multijudge measurement, we decided to calculate the concordance index and the Kappa index (Cohen, 1960) to check the level of interjudge agreement, considering each questionnaire item as an observed subject, in particular based on the calculation of the freemarginal multirather Kappa (Brennan and Perdiger, 1981). This index is recommended when the judges are unaware a priori of the number of cases they must assign to each category in the different observations, which is the case here (Brennan and Prediger, 1981). Thus, we obtained a concordance index of 83.75% and a Kappa index of .67, from which we can consider the existence of a good level of interjudge agreement (Landis and Koch, 1977). Procedure The first version of the questionnaire was designed in November 2011. After the judges’ assessment, the instrument went from 32 items to 40. The data collection was carried out between January and June 2012 through an online survey procedure. The resource that integrated the questionnaire guaranteed anonymity, which avoids social desirability bias. We tried to reduce the measurement errors which stemmed from the survey respondents by standardizing the conditions for administering the survey. Each survey taker was given precise instructions so that they would pass on systematic information to the participants. Given the low partial nonresponse index, which is not over 1.5% in any item (the highest frequency amounts to 10 lost values), considering in consequence that the effect of imputation is going to be minimum, we applied classical imputation techniques. On the other hand, taking into account the multivariate techniques used below, to avoid overestimation of the correlation coefficients among the variables, we decided to substitute the lost values with the unconditional mean imputation (Medina and Galván, 2007). Data analysis Even though, given the nature of the scales used in this study, it might be preferable to use alternative methods other than those based on Person’s correlation matrix, (LópezGonzález, PérezCarbonell and Ramos, 2011; LópezGonzález, 2012) or the use of the polychoric correlation matrix (Elosua Oliden and Zumbo, 2008), the vast scientific evidence developed in the ‘70s and ‘80s of the 20^{th} century (Hofacker, 1984; Labovitz, 1967, 1970; Morales Vallejo, 2006) suggests that the use of these methods entails small biases (Nunnally, 1978). In this study, given that “any answer codification, consistent with the conceptual order, does not distort the statistical conclusions to an acceptable degree” (Morales Vallejo, 2006, p. 39), we consider the Likerttype scale used as an interval scale, and we used Pearson’s correlation for the calculation of the matrices. However, we conducted a previous analysis to confirm this, which consisted in comparing Pearson’s correlation matrix and the polychoric correlation matrix obtained, confirming that the differences among the coefficients, always in favour of the polychoric correlation matrix, do not exceed in any case a 0.2 value, and in 87% of the cases they don’t reach a 0.1 value. We started by studying the interitem correlation between the groups of items within each dimensions. When we obtained excessively low or high correlation indexes we assessed their elimination by studying their theoretical importance and/or their colinearity with other items of the factor. We checked the previous assumptions of univariate and multivariate normality, homoscedasticity and non multi colinearity, with the aim of selecting the most suitable estimation method. Once checked, we studied the dimensional characteristics of each theoretical factor and the possible reduction of dimensions of their item groups through a Principal Components Analysis (Martorell, González, Ordóñez and Gómez, 2011). After that, we measured the reliability through Cronbach’s alpha, both for the whole of the scale and the groups of items that make up each principal component, and we also measured the reliability of the factors through the Composite Reliability Index (CRI). Finally, we studied both the convergent and the discriminant validity through the calculation of the Average Variance Extracted (AVE). Once the reliability and validity of the dimensions that make up the scale had been proven, we confirmed the model’s goodnessoffit through a confirmatory factor analysis (CFA). We studied the normed absolute fit indexes, such as Root Mean Square Error of Approximation (RMSEA), the Root Mean Square Residual (RMR) or the Goodness of Fit Index (GFI), and also the incremental fit indexes, including the Adjusted Goodness of Fit Index (AGFI), the Normed Fit Index (NFI) or the Relative Fit Index (RFI). The results of this study have been obtained by using the statistical programme IBM SPSS v.21, together with its extension AMOS, Excel y Epidat 3.1. ResultsItem analysis The study of itemelement correlations, considering as low correlations those below or close to .4 (Morales Vallejo, 2006), focuses its attention only on two items of the theoretical dimension contents (see table 2). Table 2: Totalelement statistics for each theoretical dimension
Firstly, item 06 is considered of theoretical relevance
because it belongs to the factor that addresses the
resources related to class presentations, so we did not
consider discarding it. Item 09, apart from being vague and
excessively generic in its definition, addressing generally
the interest of revising any kind of resource, slightly
overlaps with item 04. In addition, the perspective provided
by item 04 is closer to what we Regarding the rest of theoretical dimensions, we do not observe indexes below .4 or above .8. On the other hand, by analysing the interitem correlations for each dimension, we obtained acceptable correlation items for the most part, and in any case above .75. Previous assumptions
After checking the
colinearity multi colinearity assumption, we obtained
Variance Inflation Factor values below 2.5, and condition
indexes below 25 points for all items with regard to their
theoretical dimension. On the other hand, the value of the
correlation coefficient between the items of different
dimensions does not exceed Regarding normality and homoscedasticity, being CFA a multivariate technique, we must check both the univariate and the multivariate normality. The KolmogorovSmirnov test locates, in every case, the contrast statistic in the reject area of H_{0} (α=.05). Table 3 shows how all null hypotheses are rejected with a pvalue below .001. On the other hand, Mardia’s Coefficient (Mardia, 1970) results in a standardized score of 67.669 points, meaning that the joint distribution of the items is very far from the normal multivariate distribution. Thus, the normality of the data is not confirmed, and we will select an estimation method that does not imply this previous assumption. Table 3: Tests for the verification of normality and homoscedasticity
Principal Component Analysis Since principal component analysis does not demand the assumptions of normality and homoscedasticity (García Jiménez, Gil Flores, and Rodríguez Gómez 2000), we proceeded with its direct application on all items, forcing the extraction of 5 factors. With the model, a 50.9% of the total variance is explained, from which the first factor (in the initial solution, without rotating) explains a 30.3%. On the other hand, the sedimentation graph does not show evidences of the scale’s unidimensionality. Regarding the rotated solution, the configuration matrix (oblimin method) suggests that the theoretical assumptions on the dimensionality of the questionnaire can be correct, as shown in table 4. Table 4: Configuration matrix in the complete scale
Extraction method: Principal components analysis. Rotation method: Oblimin normalization with Kaiser. The three last items of the dimension activities have a saturation below .4 on the dimension itself. Despite this, we decided to keep them because we deemed them to be theoretically important. The same goes for the first item of the dimension learning. Likewise, we found that the dimensions learning and interaction contain shared information, saturating a good part of their items in the fourth dimension. After the content analysis if the items involved, we verified this similarity, taking into account that learning is, ultimately, a shared and social process (Vigostky, 1995). If we apply the principal component analysis dimension after dimension, we obtain unidimensional structures except for the dimension contents, where there are 2 principal components that explain 58.31% of the variance. By analysing the items of this dimension, we observe how the first five are related to general aspects of the platform’s content, and the other 3 refer to specific resources. In other cases, we obtained an explained variance of 43.4% (activities), 45.8% (assessment), 64.7% (interaction) and 56.1% (learning). Table 5 shows the correlation between the factors obtained in the exploratory factor analysis. The value of the coefficients indicates that the factors have shared information, which makes the rotation method the most suitable for this case. Table 5. Matrix of correlations between dimensions
Confirmatory Factor Analysis Given the evidences of the noncompliance with the normality assumption, and that the measurement scale for all the items is identical, we chose to use the estimation method of unweighted least squares (Bollen, 1989; Byrne, 2001; Kline, 2005). Choosing this method over other nonparametric ones responds to its analogue properties of OLS estimation (Lévy Mangin, 2006) and to its good behaviour towards the existence of low factorial loads (Ximénez and García, 2005), which is evidenced in the study. The implemented theoretical model is shown in figure 1. There are five first order factors, and a second order one, with the exception of the first order factor contents which, as explained before, is divided in two subdimensions.
Figure 1. Theoretical model of the two level CFA The parameters of the model, which estimation is presented in table 6, have factorial loads with scores on each factor exceeding .4 in every case, and in average they are above .6. Therefore we can conclude that every item properly contributes to its dimension. The square multiple correlations are generally low, which indicates there is a good part of the variance that is not explained by the factors. However, the CRI and Cronbach’s alpha obtained seem to indicate that the instrument has an acceptable reliability level, both at global and factor level (Lévy Mangin, 2006). On the other hand, the AVE values are below .5 in most of the factors, and therefore the validity (both convergent and discriminant) is not compromised (Kline, 2005). However, given the scarcity of scales that collect information on the different dimensions involved in the educational use of Moodle, this instrument represents a starting point in this research field. This way, the questions introduced in the questionnaire have a great amount of noise (nonexplained variance), valuable information contained by these variables, although it is placed in acceptable percentages, it is lower than desirable. Table 6: Questionnaire reliability and validity indexes
Regarding the model’s goodness of fit, we obtain indexes that show good fit (Bollen, 1989; Byrne, 2001; Kline, 2005; Lévy Mangin, 2006) both global (GFI=.985; RMSEA=0.47; RMR=.022) and incremental (AGFI=.981; NFI=.977; RFI= .976), by being below .05 in RMSEA and RMR and above .95 in the rest of the cases. As for the global fit, the results indicate that the model makes a satisfactory prediction of the data covariance matrix. The results regarding incremental fit suggest that the proposed model is adequate in comparison with the null model, and therefore the proposed relations have substantial weights. DiscussionMoodle is one of the more complete and adequate platforms for its implementation in Higher Education (Aydin and Tirkes, 2010; Saito and Ulbricht, 2012; Williams van Rooij, 2012). This fact is evidenced because Moodle provides three essential resources: the possibility of supplying contents and activities online, interactive assessment (Ross, 2008) and the flexible interaction and communication between the teacher and the students (Ellison et al., 2007). In this sense, both the use of Moodle and online materials and resources enhance and/or improve the learning outcomes (MartínBlas and SerranoFernández, 2009; Núñez et al, 2011; EscobarRodríguez and MongeLozano, 2012). The success of an LMS depends on many factors (Davis et al., 1989), among which is the user’s perceived usefulness of the LMS itself. In the scientific literature, we can locate a large number of studies which determine the keys of the success of an LMS. While these studies often contain student perception scales on the usefulness of the LMS, in many cases these scales don’t have the psychometric properties that would be necessary in order to ensure the reliability and validity of the collected information (Lin, 2008; Weaver, Spratt and Nair, 2008; Klobas and McGill, 2010; AlBusaidi and AlShihi, 2012). In many cases, the researchers only include ad hoc designed scales without an associated psychometric study (Ozkan and Koseler, 2009; Naveh, Tubin and Pliskin, 2010; Palmer and Holt, 2010; Rubin, Fernades and Avgerinou, 2013), or with a very simple and superficial study. In other cases, the sizes of the samples obtained in order to implement the psychometric study are small (Lin, 2008; Ozkan and Koseler, 2009; AlBusaidi and AlShihi, 2012) and largely limit its results. On the other hand, the dimensions included in these scales are vague and highly varied, depending on the interest of the specific research. This way, the present study overcomes many of these obstacles, because based on the set objectives, we designed a scale with content validity criteria, to subsequently validate the psychometric properties through the application of statistical techniques from a representative sample. Given the importance of the perceived usefulness as predictor variable of the success in the implementation of a technological tool such as the LMS (Davis, 1993; Sørebø et al., 2009; Friedrich and Hron, 2010), the described scale constitutes an instrument that is valid and reliable on a psychometric level, which allows the organisations and teachers to reflect on the weaknesses of Moodle as a complement of facetoface education. On the other hand, due to the amount and robustness of the empirical studies that verify the predictive value of the perception studies in the sphere of the Social Sciences (Eastman and Marzillier, 1984; Bandura and Locke, 2003; Rottinghaus, Larson, and Borgen, 2003; Valentine, Dubois and Cooper, 2004) in the field of the Higher Education (De Barros, 2012), the scale can be used as a measure of the quality and efficacy of the use of the institutional LMS in its every dimension, and it can be useful in the diagnostic and evaluation of the current state of the use of the platform. While the results obtained suggest that the scale adequately measures the users’ perceived usefulness of the LMS Moodle in the dimensions identified as key for all LMS (Moore and Iida, 2010; Palmer and Holt, 2010; AlBusaidi and AlShihi, 2012), which are the contents, activities, assessment, interaction and learning, we obtain variance indexes below the desirable ones in some cases. Given the inexistence of global scales with adequate psychometric properties, and this scale being a perception one, it is only logical to obtain these values. Using Pearson’s correlation matrix to carry out the analysis includes a certain bias relative to the metric of the variables (Elosua Oliden and Zumbo, 2008; Morales Vallejo, 2006),and it contributes to the obtainment of these low indexes, because in this case the value of the coefficients is underestimated in relation to the polychoric correlation matrix. Taking into consideration the underestimation of the real value of the correlation between the pairs of variables obtained here, an improvement of the goodness of fit indexes and the explained variance is to be expected with the application of more adequate techniques for Likerttype response scales when facing future research (LópezGonzález, PérezCarbonell and Ramos, 2011; LópezGonzález, 2012). Despite the importance of the weaknesses present in this research, this scale entails a starting point in future investigations for the development of scales adapted to subpopulations of university students from diverse areas of knowledge, whose dimensions will explain a bigger percentage of the variance. In this sense, it is worth remembering that the present study is restricted to the population of students from the area of Social Sciences, in particular from Educational Sciences, and that despite the large size of the sample, the nonprobabilistic sampling method can be a source of important biases. On the other hand, it is worth mentioning another weakness related to the context of this study, which focuses on the use of Moodle as a complement to facetoface education. In this way, the subjects involved in this study have been users of Moodle in facetoface contexts, and because of that we are unable to know how the scale would behave when adapted to blearning or elearning contexts. There is room for the possibility of adapting the scale in future studies in order to observe its behaviour in these other educational contexts. In conclusion, based on the evidences gathered and shown in the present research, we can conclude that this study constitutes a solid base for the development of coming investigations related with the analysis of the use of Moodle as an LMS in different educational institutions. ReferencesAjzen, I., & Fishbein, M. (1980). Understanding attitudes and predicting social behavior. Englewood Cliffs, New York: PrenticeHall. AlBusaidi, K. A., & AlShihi, H. (2012). Key factors to instructors’ satisfaction of learning management systems in blended learning. Journal of Computing in Higher Education, 24 (1), 1839. Antonenko, P., Toy, S., & Niederhauser, D. (2004). Modular ObjectOriented Dynamic Learning Environment: What Open Source Has to Offer. En Association for Educational Communications and Technology. Recuperado de http://www.eric.ed.gov/ERICWebPortal/detail?accno=ED485088 Arnal, J., Rincón, D. del, & Latorre, A. (1992). Investigación educativa: fundamentos y metodologías (1a. ed.). Barcelona: Labor. Aydin, C. C., & Tirkes, G. (2010). Open Source Learning Management Systems in Distance Learning. Turkish Online Journal of Educational Technology, 9 (2), 175184. Bandura, A., & Locke, E. A. (2003). Negative selfefficacy and goal effects revisited. The Journal of Applied Psychology, 88(1), 8799. Barberá, E. & Badia, A. (2004). Educar con aulas virtuales. Orientaciones para la innovación en el proceso de enseñanza y aprendizaje. Madrid: Antonio Machado Libros. Bollen, K. A. (1989). Structural equations with latent variables. New York: WileyInterscience. Brennan, R. L., & Prediger, D. J. (1981). Coefficient Kappa: Some uses, misuses, and alternatives. Educational and Psychological Measurement (41), 687699. Britain, S. & Liber, O. (1999). A Framework for Pedagogical Evaluation of Virtual Learning Environments. Recuperado de http://www.jtap.ac.uk/reports/htm/jtap041.html Byrne, B. (2001). Structural equation modeling with amos: basic concepts, applications, and programming. Oxford: Routledge. Castells, M. (1999). La Era de la información: economía, sociedad y cultura. La sociedad Red (Vol. 1). Madrid: Alianza Editorial. Cohen, D., & McCuaig, W. (2008). Three lectures on postindustrial society. Cambridge: MIT Press. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, (20), 3746. Cole, J. & Foster, H. (2007). Using Moodle: Teaching with the Popular Open Source Course Management System. London: O'Reilly. Conrey, F. R. & Smith, E. R. (2000). Attitude Representation: Attitudes as Patterns in a Distributed, Connectionist Representational System, Social Cognition, 25 (5), 718735. Davis, F. D. (1993). User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. International Journal of ManMachine Studies, 38 (3), 475487. Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User Acceptance of Computer Technology: A Comparison of Two Theoretical Models. Management Science, 35 (8), 9821003. De Barros, A. F. (2012). Características Psicométricas da Adaptacão Portuguesa do Perfil de AutoPercepção para Estudiantes Universitários  SPPCS. Revista Iberoamericana de Diagnóstico y Evaluación Psicológica, 1(33), 93110. DeNeui, D. L., & Dodge, T. L. (2006). Asynchronous Learning Networks and Student Outcomes: The Utility of Online Learning Components in Hybrid Courses. Journal of Instructional Psychology, 33 (4), 256259. Eastman, C., & Marzillier, J. S. (1984). Theoretical and methodological difficulties in Bandura’s selfefficacy theory. Cognitive Therapy and Research, 8 (3), 213229. Ellis, R.K. (2009). Learning Managament Systems. Alexandria: American Society for Training & Development (ASTD). Ellison, N.B., Steinfield, C., & Lampe, C. (2007). The benefits of facebok “friends”: Social capital and college students’ use of online social network sites. Journal of ComputerMediated Communication, 12(4), 11431168. Elosua Oliden, P. & Zumbo, B.D. (2008). Coeficientes de fiabilidad para escalas de respuesta categórica ordenada. Psicothema, 20(4), 896901. EscobarRodriguez, T., & MongeLozano, P. (2012). The acceptance of Moodle technology by business administration students. Computers & Education, 58 (4), 1085–1093. Friedrich, H. F., & Hron, A. (2010). Factors Influencing Pupils’ Acceptance of an ELearning System for Secondary Schools. Journal of Educational Computing Research, 42(1), 6378. García Jiménez, E., Gil Flores, J., & Rodríguez Gómez, G. (2000). Análisis factorial. Madrid: La Muralla. Gómez Rey, I.; Hernández García, E., & Rico García, M. (2009). Moodle en la enseñanza presencial y mixta del inglés en contextos universitarios. RIED. Revista Iberoamericana de Educación a Distancia, 12 (1), 169194. Goyal, E., & Purohit, S. (2010). Study of Using Learning Management System in a Management Course. SIES Journal of Management, 6(2), 1120. Hofacker, C.F. (1984). Categorical judgment scaling with ordinal assumptions. Multivariate Behavioral Research, 19, 91106. Kirner, T.G., & Saraiva, A.V. (2007). Software Usability Evaluation: an Empirical Study. Paper presented at Proceedings of the 9th International Conference on Enterprise Information Systems. Funchal, Portugal, 459465. Kline, R. (2005). Principles and practice of structural equation modeling. New York: Guilford Press. Klobas, J. E., & McGill, T. J. (2010). The role of involvement in learning management system success. Journal of Computing in Higher Education, 22(2), 114134. Labovitz, S (1970). The assignment of numbers to Rank order categories. American Sociological Review, 35, 315324. Labovitz, S. (1967). Some observations on measurement and statistics. Social Forces, 46, 151160. Landis J.R. & Koch, G. (1977). The measurement of observer agreement for categorical data. Biometrics (33), 159174. Lévy Mangin, J.P. (2006). Modelización con estructuras de covarianzas en ciencias sociales: temas esenciales, avanzados y aportaciones especiales. España: Netbiblo. Lin, Q. (2008). Student satisfactions in four mixed courses in elementary teacher education program. Internet and Higher Education, 11(1), 5359. LópezGonzález, E. (2012). Sugerencias para el análisis de Escalas con Métrica Delicada. Revista Iberoamericana de Evaluación Educativa, 5(1e), 84105. LópezGonzález, E., PérezCarbonell, A. & Ramos, G. (2011). Modelos complementarios al análisis factorial en la construcción de escalas ordinales: un ejemplo aplicado a la medida del Clima Social Aula. Revista de Educación, 354, 369397. Mardia, K. V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57(3), 519530. MartínBlas, T., & SerranoFernández, A. (2009). The role of new technologies in the learning process: Moodle as a teaching tool in Physics. Computers & Education, 52, 35–44. Martorell, C., González, R., Ordóñez, A. N. A., & Gómez, O. (2011). Estudio confirmatiorio del cuestionario de conducta antisocial (CCA) y su relación con variables de personalidad y conducta antisocial. Revista Iberoamericana de Diagnóstico y Evaluación Psicológica, 1(31), 97113. Medina, F., & Galván, M. (2007). Imputácion de datos: teoría y práctica. Santiago de Chile: Naciones Unidas, CEPAL, División de Estadística y Proyecciones Económicas. Melton, J. (2006). The LMS Moodle: A Usability Evaluation. Japan: Prefectural University of Kumamoto. Moore, K., & Iida, S. (2010). Students’ perception of supplementary, online activities for Japanese language learning: Groupwork, quiz and discussion tools. Australasian Journal of Educational Technology, 26(7), 966979. Morales Vallejo, P. (2006). Medición de actitudes en psicología y educación. Construcción de escalas y problemas metodológicos. Madrid: Universidad Pontificia de Comillas. Morales, P., Urosa, B., & Blanco, A. (2003). Construcción de escalas de actitudes tipo Likert. Madrid: La Muralla. Naveh, G., Tubin, D., & Pliskin, N. (2010). Student LMS use and satisfaction in academic institutions: The organizational perspective. Internet and Higher Education, 13(3), 127133. Nunnally, J.C. (1978). Psychometric theory. New York: McGrawHill. Núñez, J.C, Cerezo, R., Bernardo, A., Rosário, P., Valle, A., Fernández, E., & Suárez, N. (2011). Implementation of training programs in selfregulated learning strategies in Moodle format: Results of an experience in higher education. Psicothema 23(2), 274281 Ozkan, S., & Koseler, R. (2009). Multidimensional students’ evaluation of elearning systems in the higher education context: An empirical investigation. Computers & Education, 53(4), 12851296. Palmer, S., & Holt, D. (2010). Students’ perceptions of the value of the elements of an online learning environment: looking back in moving forward. Interactive Learning Environments, 18(2), 135151. Peat, M., & Franklin, S. (2002). Supporting Student Learning: The Use of Computerbased 9. Formative Assesment Modules. British Journal of Educational Technology, 33(5), 515523. Pérez i Garcias, A. (2006). Internet aplicado a la educación: aspectos técnicos y comunicativos. En J. Cabero (coord.) (2006). Nuevas tecnologías aplicadas a la educación. Madrid: Mc Graw Hill. Richardson, J., & Swan, K. (2003). Examining social presence in online courses in relation to students' perceived learning and satisfaction. Journal of Asynchronous Learning 6 (1), 2140. Ross, I. (2008). Moodle, la plataforma para la enseñanza y organización escolar. Ikastorratza, e Revista de Didáctica 2. Recuperado de http://www.ehu.es/ikastorratza/2_alea/moodle.pdf Rottinghaus, P. J., Larson, L. M., & Borgen, F. H. (2003). The relation of selfefficacy and interests: a metaanalysis of 60 samples. Journal of Vocational Behavior, 62(2), 221236. Rubin, B., Fernandes, R., & Avgerinou, M. D. (2013). The effects of technology on the Community of Inquiry and satisfaction with online courses. Internet and Higher Education, 17, 4857. Rus, T. I., Pina, F. H., Sánchez Y, J. J. M., & Martínez, O. L. (2011). Adaptación y validación de la escala de actitudes hacia el trabajo en desempleados mayores de 45 años. Revista Iberoamericana de Diagnóstico y Evaluación Psicológica, 2(32), 105122. Saito, D. S., & Ulbricht, V. R. (2012). Learning Managent Systems and FacetoFace Teaching in Bilingual Modality (Libras/Portuguese). IEEE Latin America Transactions, 10(5), 21682174. Silva Quiroz, J. (2011). Diseño y moderación de entornos virtuales de aprendizaje (EVA). Barcelona: UOC. Sørebø, Ø., Halvari, H., Gulli, V. F., & Kristiansen, R. (2009). The role of selfdetermination theory in explaining teachers’ motivation to continue to use elearning technology. Computers & Education, 53 (4), 11771187. Soyibo, K., & Hudson, A. (2000). Effects of Computerassisted Instruction (CAI) on 11th 8. Graders’ Attitudes to Biology and CAI and Understanding of Reproduction in Plants and Animals. Research in Science Technological Education, 18 (2), 191199. Steyaert, J. (2005). Web based higher education, the inclusion/exclusion paradox. Journal of Technology in Human Services, 23 (1/2), 6768. Swan, K., Shea, P., Fredericksen, E., Pickett, A., Pelz, W., & Maher, G. (2000). Building Knowledge Building Communities: Consistency, Contact and Communication in the Virtual Classroom. Journal of Educational Computing Research, 23(4), 35983. Valentine, J. C., Dubois, D. L., & Cooper, H. (2004). The relation between selfbeliefs and academic achievement: a metaanalytic review. Educational Psychologist, 39(2), 111133. Vaughan, N. (2007). Perspectives on Blended Learning in Higher Education. International Journal on ELearning, 6 (1), 8194. Vigostsky, L. (1995). Pensamiento y lenguaje. Buenos Aires: Paidós. Weaver, D., Spratt, C., & Nair, C. S. (2008). Academic and student use of a learning management system: Implications for quality. Australasian Journal of Educational Technology, 24 (1), 3041. Weller, M. (2007). Virtual Learning Environments: using, choosing and developing your VLE. New York: Routledge. Williams van Rooij, S. (2012). Opensource learning management systems: a predictive model for higher education. Journal of Computer Assisted Learning, 28 (2), 114–125. Yueh, H., & Hsu. S. (2008). Designing a learning management system to support instruction. Communications of the ACM, 51 (4), 59 63.


ARTICLE RECORD / FICHA DEL ARTÍCULO
Reference / Referencia 
OlmosMigueláñez, S., MartínezAbad, F., TorrecillaSánchez, E. M. & MenaMarcos, J. J. (2014). Psychometric analysis of a perception scale on the usefulness of Moodle in the University. RELIEVE, v. 20 (2), art. 1. DOI: 10.7203/relieve.20.2.4221 
Title / Título 
Psychometric analysis of a perception scale on the usefulness of Moodle in the University. [Análisis psicométrico de una escala de percepción sobre la utilidad de Moodle en la universidad]. 
Authors / Autores 
OlmosMigueláñez, S., MartínezAbad, F., TorrecillaSánchez, E. M. & MenaMarcos, J. J. 
Review / Revista 
RELIEVE (Revista ELectrónica de Investigación y EValuación Educativa), v. 20 n. 2 
ISSN 
11344032 
Publication date / Fecha de publicación 
2014 (Reception Date: 2014 May 03 ; Approval Date: 2014 July 29. Publication Date: 2014 September 30) 
Abstract / Resumen 
Because of the acquired relevance of learning management systems in higher education, and the spread of the use of the Moodle platform in many academic institutions, a scale of perceived usefulness of the Moodle in this context is designed, and the psychometric validity of the scale has been tested. The aim is to provide a reliable and valid instrument to measure the students’ perception about the usefulness of Moodle. The study obtained a sample of 754 subjects from the population of university students in fields of Educational Sciences. The results show that the scale evaluates the utility of the platform adequately in five dimensions: content, activities, assessment, interaction and learning. Finally, a discussion is developed about the usefulness of the scale to evaluate the usefulness of Moodle and to implement processes to improve its use in higher education institutions. Dada la importancia que los entornos virtuales de aprendizaje (learning management systems) han adquirido en la educación superior, y la generalización en el empleo de la plataforma Moodle en muchas instituciones universitarias, se diseña y se validan las cualidades psicométricas de una escala de utilidad percibida sobre el uso de Moodle. Se pretende aportar un instrumento válido y fiable que permita comprobar cuál es la percepción de los estudiantes sobre la utilidad de Moodle. De la población de estudiantes universitarios del ámbito de las Ciencias de la Educación, se obtiene una muestra de 754 sujetos. Los resultados manifiestan que la escala evalúa, adecuadamente, la utilidad de la plataforma en cinco dimensiones: contenidos, actividades, evaluación, interacción y aprendizaje. Finalmente, se discute sobre la utilidad de la escala para evaluar la utilidad de Moodle y para la implementación de procesos de mejora de su empleo en las instituciones de Educación Superior. 
Keywords / Descriptores 
Information and communication technologies, computer application, evaluation, factor analysis. Tecnología de la información y la comunicación, aplicación informática, evaluación, análisis factorial. 
Institution / Institución 
University of Salamanca (Spain) 
Publication site / Dirección 

Language / Idioma 
Español & English version (Title, abstract and keywords in English & Spanish) 
© Copyright, RELIEVE. Reproduction and distribution of this article is authorized if the content is no modified and its origin is indicated (RELIEVE Journal, volume, number and electronic address of the document).
© Copyright, RELIEVE. Se autoriza la reproducción y distribución de este artículo siempre que no se modifique el contenido y se indique su origen (RELIEVE, volumen, número y dirección electrónica del documento).
[ ISSN: 11344032 ]
Revista ELectrónica de Investigación y EValuación Educativa EJournal of Educational Research, Assessment and Evaluation
