This work is included in the strategic learning context and its assessment. Learning strategies are a multidimensional, polysemic and, occasionally, confusing construct which have led to numerous definitions (Ayala, Martínez & Yuste, 2004; Beltrán, 1993 and 2003; Bernad, 1999; Danserau, 1985; Kirby, 1984; Monereo, 1997; Monerero & Castelló, 1997; Nisbet & Shucksmith, 1987; Pozo, 1990; Weinstein & Danserau, 1985). Although it is true that, at times, emphasis has been placed on the cognitive and metacognitive aspects when it comes to conceptualising them (Danserau, 1985; Nisbet & Shucksmith, 1987; Kirby, 1984; Weinstein & Mayer, 1985), it is also true that the concept’s content has enriched to become more integrative, and includes affective-motivational and support elements.
From our point of view, learning strategies can be understood as an organised, conscious and intentional whole of what the student does to efficiently accomplish a learning objective in a given social context. Acting strategically implies wanting to learn efficiently, and wanting to design and carry out action plans that have been adapted not only to the targets foreseen, but also to the conditions of the context, by selecting and putting into practice efficient learning procedures, skills and techniques (García & Pintrich, 1993) whose effectiveness has to be assessed to amend that which needs amending. Learning strategies include affective-motivational and support elements (“wanting”, which involves the willingness and a suitable climate to learn): metacognitive elements (“making decisions and assessing them”, which involves students’ self-regulation) and cognitive elements (“being able”, which involves dealing with the strategies, skills and techniques related with information processing) (Abascal, 2003; Ayala, Martínez & Yuste, 2004; Corno, 1994; García & Pintrich, 1991; Gargallo, 2000; González Cabanach, Valle, Rodríguez, & Piñeiro, 2002; González-Pumariega, Núñez Pérez, González Cabanach & Valle, 2002; Monereo, 1997).
We are aware that the concept we propose is ample and eclectic. Nevertheless, we opt for this perspective instead of more restrictive ones because it is more integrative and enables a more complete strategies map design. All in all, it includes those concept elements that are currently considered fundamental: conscience, intentionality, managing different resources, self-regulation and connection with the context.
It is necessary to understand the concept in a dynamic perspective which emphasises the “strategic” use of the different procedures that are mobilised to learn. It no longer makes much sense to insist on the differences among macrostrategies, microstrategies and techniques, which has occasionally been the case (Kirby, 1984). It is now better to insist on the strategic use of the various components mobilised to learn with conscience, intentionality and flexibility, and towards a capacity for supervision and self-control, –basically, metacognitive performance. Strategic use implies “conditional” (Paris, Lipson & Wilxson, 1983) or strategic (Monereo, 1995) knowledge, which is added to declarative and procedural knowledge, that allows students to determine under which circumstances or conditions they use declarative and/or procedural knowledge and mobilise the appropriate attitudes to learn efficiently.
1.1. Our learning strategies classification
Research into learning strategies in a university setting, which is our particular case, needs good instruments to be developed for both the implementation of descriptive-explicative designs and the validation of programmes for teaching strategies. The lack of specific instruments for the university stage means that, occasionally, instruments which have been designed for other scenarios are used (Gil, Bernaras, Elizalde & Arrieta, 2009)
When it comes to assessing such strategies, it is essential to put together a suitable theoretical structure, that is a model or a “map”, as completely as possible to include the different strategies being mobilised for learning purposes without leaving any basic elements out.
With this global intention in mind, we have devised our own proposal which is indebted in part to the self-regulated learning model of Pintrich and Schrauben (1992). Their model was used to design the Motivational Strategies Learning Questionnaire - MSLQ (Pintrich, Smith, García & Mckeachie, 1991), which we will mention later (Gargallo, 2000). Our model is not only coherent with other authors’ contributions (Beltrán, 1993; Bernad, 1999; Gargallo, 1995; Justicia & Cano, 1993; Pozo, 1990; Román & Gallego, 1994; Pintrich, Smith, García & Mckeachie, 1991; Weinstein & Mayer, 1985; Weinstein, Palmer & Schulte, 1987), but also completes them (Table 1):
Table 1. Learning strategies classification (Gargallo, 2000)
From our viewpoint, our classification is integrative, is coherent with the learning strategies concept proposed, and covers the three basic dimensions of the human mind in relation to learning: willingness, capacity and autonomy (wanting, being able and deciding) (Beltrán, 2003; Beltrán, Pérez & Ortega, 2006; Weinstein, Husman & Dierking, 2002), without leaving anything out.
On the one hand, the Affective, Willingness and Support Strategies (Pintrich, Smith, García & Mckeachie, 1991; Roces, Tourón & González, 1995) carry weight in this classification, are all fundamental in learning and include the motivational and affective side (“wanting-willingness” is essential for “deciding-autonomy” and for “being able-capacity”) (Monereo, 1997; Pozo & Monereo, 1999). These strategies set up the process and help sustain effort. Our classification is based on and completes the model of Pintrich and De Grooth’s (1990) with its Motivational-Affective components that also include Value, Expectation and Affective Components.
On the other hand, the Metacognitive Strategies (deciding-autonomy) are sufficiently covered, and relate to the capacity for decision making, planning, self-assessing one’s performance and self-regulating.
We have opted to combine the two types of strategies (Affective and Metacognitive) with the Context Control ones in a first block; that is, the Affective, Support and Control Strategies by integrating the Affective-Motivational, Metacognitive, Context Control and Social Interaction Strategies since all these strategies do not directly address Information Processing, but set up and help implement the process. A second block includes the Cognitive Strategies that relate more directly with the process. Likewise, the Context Control and Social Interaction Strategies are included.
Finally, our classification includes the Information Processing Strategies (“being able-capacity”) contemplated in the traditional classifications in accordance with the information processing models (Acquisition, Elaboration, Organisation and Storage). These also include the Personalisation and Creativity Strategies (learning is more than retaining prepared and organised information as it implies recreating, critically repreparing, and making one’s own proposals), and the Retrieval, Transfer and Use Strategies, which tend to be missed in classifications of this kind (learning also involves efficiently using what one has learnt). Similarly, the Information Search, Collection and Selection Strategies are also incorporated, which are usually left out of the classifications found in the literature. In today’s information society, a strategies classification cannot evade this basic component.
1.2. Some of the questionnaires most widely used in our context and their limitations
An assessment should use the appropriate instruments to be coherent with this theoretical framework. However we found no such assessment which suitably covers the different strategies involved in learning. Some of those we analysed, which are the most widely used in our context, that is, research in the university setting, come up against other problems.
Thus the Spanish ACRA questionnaire by Román and Gallego (1994), which was validated in a non-university population (12-16-year olds) and has also been used in university populations in Spain, cannot simply be extrapolated to universities because the validation work done with university students does not prove the suitability of the questionnaire as it stands (Justicia & De la Fuente, 1999; De la Fuente & Justicia, 2003).
The underlying model of this instrument hypothesises that there are three main groups of Information Processing/Cognitive Learning Strategies (Acquisition, Encoding and Retrieval) and one fourth support strategy that includes both the Metacognitive and Socio-Affective Strategies. These four groups become four scales (Acquisition, Encoding, Retrieval and Support) that are subdivided into strategy types which, in turn, are subdivided into strategies. The questionnaire is made up of 84 items. The three Cognitive Strategies scales cover the fundamental processes of information processing: Attentional, Elaboration, Organisation, Repetition and Storage.
From our viewpoint, the structuring of the two first scales is debatable as the Storage Strategies are located in both the first scale, Acquiring Information, and the second, Encoding (the so-called Encoding or Storage, which is still questionable). The fourth scale, Support, in the Socio-Affective Strategies group includes both Intrinsic Motivation and Extrinsic Motivation in the same strategy, and both score in the same direction. This is conspicuous and gives the impression that Intrinsic Motivation and Extrinsic Motivation take the same value.
This instrument does not include any type of fundamental strategies like Information Search, Collection and Selection, and other important contemplated types are lacking. Therefore the Information Processing Strategies do not include Personalisation, Creativity and Transfer, while the Support Strategies exclude Tasks Values, Self-efficacy and Context Control.
On the other hand, the questionnaire includes excessively long items of doubtful intelligibility given the population it addresses. In our research works, we have been using this instrument in 12-16-year-old populations. There are several items that have proved difficult to understand. The same may be said of the university setting where we have used this instrument in the practical activities of a given course subject that we teach; again, students have had difficulties in understanding some items.
The authors mention good internal consistency in relation to the questionnaire scales, which varies between .73 and .87 (Cronbach’s alpha). However, they do not provide this consistency for the strategies that the scales are made up of.
On the other hand, the tests for adaptation to the university population, which have involved cutting down on items, do not corroborate the original theoretical structure in order to do construct validity tests (Justicia & De la Fuente, 1999). Indeed De la Fuente and Justicia (2003) found three dimensions/scales, but not four, which they identify as Cognitive and Control Strategies, Support Strategies and Study Habits, whose reliability varies between .54 and .85 (Cronbach’s alpha). In the cited work, the authors maintain that they continue investigating into the scale’s validity. So, it does not seem to be apt to use this instrument as it was designed in university populations.
Weinstein’s LASSI scale (1987), designed for the university population in the United States, has proved very popular in the Spanish research works done in the university setting, but it has its limitations.
From our point of view, one initial problem with the LASSI scale is that the author (Weinstein, 1987 and 1988; Weinstein, Zimmerman & Palmer, 1988) does not seem to have started with the classification that she herself devised and used in research in a generalised manner, after making a few changes once it had been published, to design her questionnaire. In the Cognitive Strategies part, this classification was designed in terms of the processing level and the cognitive control demanded which, at the most elemental level, includes Repetition Strategies, followed by Elaboration Strategies, and finally by Organization Strategies. A subdivision is created in all three cases so that these three cases are used with elemental or basic learning or for complex tasks. To these strategies, the following have been added: Regulation and Control Strategies -the use of metacognition- and Affective-Motivation Strategies.
The classification has its limits which most certainly derive from the time it was constructed, a time when these researchers were pioneers. When designing his questionnaire, Weinstein’s aim was to overcome the shortcomings of previous instruments. So, he wanted to focus on strategies relating to successful learning and on those that may be intervened in educational terms. He set out by focusing on the lack of works into the theme from which categories were created in order to set up an initial bank of 645 items. This rigorous questionnaire design process was concluded with a scale known worldwide which comprised 77 items and 10 scales: Attitudes, Motivation, Investing time, Anxiety, Concentration, Information Processing, Selecting Main Ideas, Study Aids, Self-checking, and Exam Strategies.
The developed process resulted in this instrument’s Megacognitive Strategies not appearing to be sufficiently streamlined. Besides, some fundamental Cognitive Strategies were overlooked, such as Information Search and Selection, as well as other very important processing strategies, for instance, Storage, Transfer, and Personalisation and Creativity (in fact, the Processing scale only includes the Elaboration and Organization Strategies). Likewise it is assumed that Motivational and Support aspects were also left out which, today, are believed essential; for example, Tasks Values, Attributions, Self-efficacy or Context Control, as well as suitable Social Interactions which are currently considered appropriate.
Besides, the questionnaire contains some excessively general item constructs, items that do not respond to true strategic activity. Then there are also items defined in terms of negative conducts which merely state what subjects do not do.
The reliability of the scale is good, and ranges between .68 and .86
Finally, the CEAM II questionnaire, a translated version of the MSLQ of Pintrich, Smith, García and Mckeachie (1991) that has been adapted to the Spanish university population (Roces, Tourón and González, 1995), also presents problems from the way we see it.
The MSLQ is based on the Self-regulated Learning Model of Mckeachie, Pintrich and collaborators (Mckeachie, Pintrich, Lin & Smith, 1986). This model includes several factors that affect learning. It emphasises cognitive factors, motivational factors and their relationships, and also the effect they have on students’ implication in both their learning and academic performance. The questionnaire is made up of 81 items arranged in two sections: a motivational one and a strategies one.
The first section has six subscales (Control Beliefs, Self-efficacy, Intrinsic Goal Orientation, Extrinsic Goal Orientation, Tasks Values and Anxiety in Exams) which are grouped as three dimensions (Expectations Components, Value Components and Affective Components). The second has nine subscales (Repetition, Elaboration, Organisation, Critical Thinking, Metacognition, Time and Place to study, Regulating Efforts, Learning with Others and Seeking Aid) which are grouped into two dimensions (Cognitive and Metacognitive Strategies, and Managing Resources Strategies).
The questionnaire includes a specifically motivational dimension, which is as important as the learning strategies dimension (processing), but it neither includes the Information Search, Collection and Selection strategies nor pays attention to the Megacognitive Strategies. However, important strategies, like Attributions, Interest and Physical State and State of Mind, are not assessed in the Motivational subscales. The Cognitive Strategies in this questionnaire do not include the strategies related with Memorising or Transfer.
Cronbach’s coefficient alpha for the internal consistency of the various MSLQ subscales ranges between .52 and .93. In Roces’ adapted version (Roces, González-Pienda, Núñez, González-Pumariega, García & Álvarez, 1999), the coefficients vary between .57 and .84.
1.3. The objectives of this work
Given the context, the objective of this work was to devise and validate a robust, well structured questionnaire that permits more complete information to be collected than the other cited models have done. For this purpose, we will include two scales in our design: one with Affective, Support and Control Strategies, and the other with Information Processing-related Strategies. The first scale will include subscales with Motivational Strategies, in which we will incorporate components that have not been included in the other questionnaires, and of Affective, Context Control, Social Interaction and Managing Resources Strategies, as well as Metacognitive Strategies (which will explicitly include the Planning, Self-assessment and Control/Self-regulation components that do come over clearly in the other questionnaires). The second scale will have an Information Search, Collection and Selection subscale. This subscale has not been included in any of the instruments analysed. Similarly, we will incorporate in this Information Processing and Use subscale the most relevant related processes: Acquisition, Elaboration, Organisation and Storage, without forgetting the Personalisation and Creativity Strategies, or the Information Transfer and Use Strategies.
We are well aware that the items covered in the instrument we design will not all include a reference to explicit action plans (conscious and intentional). This may lead the reader to interpret a contradiction between our conception of learning strategies and the development of specific questionnaire elements or items. This problem affects all the questionnaires analysed. The student’s use of the strategy, skill or the technique involved in the item will, or will not, confer it a strategic nature as it influences the student’s conscious, intentionality, adaptation to the context and self-regulating capacity. In this sense, the use of a contextualised questionnaire in different course subjects and with different teachers (as we have done it in one part of our research) may well provide insight into the differences in the usage, be it strategic or not, of the resources included in the instrument.
We will also design the questionnaire in order to avoid the intelligibility problems encountered in some of the instruments analysed.
We also intend to accomplish reliability that is similar to or higher than that of the instruments analysed.
The questionnaire we present in this work is known as CEVEAPEU (a Spanish acronym of: a Questionnaire to Assess Learning Strategies in University Students).
We believe that our contribution may prove relevant if we bear in mind that the design has been devised for the purpose of correcting the limitations mentioned in the other questionnaires and that CEVEAPEU was validated in a university population with a Spanish sample.
2. METHOD AND PROCEDURE
A test validation design has been used (Croker & Algina, 1986; Jornet & Suárez, 1996; Popham, 1990) in which a rigorous construction process has been followed, as set out in the Procedures section.
We have used two samples, one when the questionnaire was handed out (pilot phase) and the other for the definitive validation.
For the pilot phase, a representative sample of university students from the two public universities in the city of Valencia (East Spain) was formed: University of Valencia, General Study (UVEG from now onwards) and Polytechnic University of Valencia (UPV from now onwards). We have worked with these two universities in this research work. The baseline sample population was made up of undergraduate students from the two universities cited; a total number of 76,295 students. The sample was formed by stratified random sampling, and the strata were defined by the five main areas at UVEG (Experimental Sciences, Education, Humanities, Social Sciences and Health Sciences), the four at UPV Engineering, Architecture, Business Administration and Management, and Fine Arts) and by the undergraduate studies that made up the various degrees.
The sample ended with a total of 545 students studying 19 different degrees at 15 faculties or schools from both universities. Of these, 319 came from UVEG (58.5%) and 226 from UPV (41.5%). A 95% confidence level was achieved with a maximum error of 5%. Regarding genders, 208 were male (38.2%) and 337 were female (61.8%).
When the questionnaire was handed out in the definitive phase, work was done with a sample of 1127 university undergraduate students from three universities located in the city of Valencia: UVEG, UPV and the Catholic University of Valencia (UCV from now onwards). UCV, a private university, was included in the second year of this project after its official constitution in 2004.
Students were selected by 50 teachers at all three universities for other objectives in this research. We are aware that it would have been ideal to have done a representative sampling of the participating universities, but for reasons concerning opportunities and economising means, we used the available sample to perform the definitive validation. The learning strategies questionnaire was handed out to the students of these teachers for its definitive validation.
There were 25 groups of UVEG students in 3 Faculties, 14 groups of UPV students in 4 Higher Technical Schools, and 11 UCV groups in 3 Faculties.
We obtained a sufficiently varied and representative sample of the three participating universities: 1127 students in 50 groups of 10 Faculties or Schools studying 15 degrees. Of these, 648 (57.5%) came from UVEG, 268 (23.8%) from UPV and 211 (18.7%) from UCV; 322 were male (28.6%) and 805 were female (71.4%).
The final product is a questionnaire with 88 items arranged in two scales, six subscales and twenty-five strategies (Table 6). The items were designed as Likert-type scales, with five response options: totally disagree, disagree, undecided, agree and totally agree.
The initial theoretical questionnaire structure was designed based on the previously reported questionnaire (Table 2):
Table 2. Initial Questionnaire Structure
The three strategy types cited in the Introduction of this work (Affective-Willingness-Support, Metacognitive and Cognitive) were grouped into two scales. The first contained the Affective, Support and Control Strategies which included the Affective, Motivational and Support Strategies, that is, those responsible for the set up and maintenance of the process. The second scale incorporated the Information Processing-related Strategies, which included the Information Search and Selection Strategies plus the Information Processing and Use Strategies. Another alternative option would have been to maintain the Megacognitive Strategies as the independent scale; however, it was also appropriate when grouped with the Affective-Willingness-Support Strategies as it enabled two large scales with all the strategies, including the Support and Control Strategies in the first scale, which do not address Information Processing. This approach is similar to that followed in other instruments, such as the MSLQ.
As the table below shows, the two scales were set out as 6 subscales (4 from the first scale and 2 from the second), and as 23 strategies which, from our point of view, and according to the other proposals analysed, were those involved in learning.
With this theoretical structure, the research team designed an initial bank with 165 items.
Then, the questionnaire was analysed and assessed by 10 expert judges (Crocker & Algina, 1986; Roid & Haladyna, 1982; Jornet & Suárez, 1996). The judges assessed the construct validity and the content of the items, their intelligibility, and the construct validity of the questionnaire on a scale of 1 to 5.
The items with a mean below 4 points and those for which the judges’ assessments showed discrepancies (Kendall’s concordance test) were ruled out.
Having reduced the questionnaire to 147 items, we then went on to the pilot phase with the first sample of students. The instrument was completed by the students selected in their classroom and during class times. This involved a team member providing them with the exact instructions to answer the questionnaire. Normally, the teacher in charge of the class, who we contacted previously, remained in the classroom. Participation was voluntary. Students completed the questionnaire, and included identification and demographic data. Confidentiality was guaranteed. Similarly, they voluntarily signed an authorisation so the team could have access to their grade, which were collected at the end of the academic year.
The next stage involved data processing and the questionnaire validation phase (analysing the technical quality of the items: intelligibility, homogeneity coefficient and variation quotient; reliability-internal consistency analysis and construct validity). We also studied the predictive validity (correlations and multiple regression analyses).
Having reduced the questionnaire to 94 items, we then started work on the definitive stage with the second sample to redo all the aforementioned tests. At the end of the process, the definitive questionnaire included 88 items.
The results we present are from the definitive phase and relate to the final questionnaire.
3.1. The construct validity results
To confirm construct validity, we did principal components factorial analyses (PCAs). The aim of the PCAs was to reduce the original set of variables to a smaller series of non-correlated components that represent most of the information found in the original variables. Varimax rotation was used in all cases given the low correlations among the factors. We previously performed Bartlett’s sphericity test in all the scales and subscales, which revealed that the data obtained were suitable for the factorial analysis.
The results, which we now go on to present (Table 3), show a chi-squared value, which meant that the correlations matrix was not an identity matrix. Thus, the matrix of the data obtained was suitable for the factorial analysis. We also calculated the KMO index (the Kaiser-Meyer-Olkin measurement of sampling adequacy) which proved that the index was suitable.
Table 3. Bartlett’s sphericity test for the questionnaire scales and subscales
Several factorial analyses were done by firstly taking all the questionnaire items. The results obtained were not wholly satisfactory given the complexity of the construct which required more satisfactory approximations. Therefore, we decided to perform factorial analyses of the subscales. We did six factorial analyses, one for each subscale, which helped to filter the questionnaire and to reduce the number of items. These analyses also suitably corroborated the foreseen theoretical structure, as Table 2 shows. Then we did two new factorial analyses, one for each questionnaire scale: the Affective, Support and Control Strategies scale with 53 items, and the Information Processing-Related Strategies scale with 35 items. This section presents the results of these analyses. However, it is necessary to firstly point out that we used the Kaiser eigenvalues criterion of lambda>1 (1960) to specify the correct number of factors.
Results of the factorial analysis of the first scale, the Affective, Support and Control Strategies scale:
Table 4 presents the results of the factorial analysis of the principle components with varimax rotation which was chosen given the low correlations among the factors. These have been arranged downwardly in terms of the percentage of variance explained for each one. The criteria applied to locate an item in a factor was saturation being .400 or higher, and non-saturation being over .300 in the other factors.
We found that 15 factors explained 56.793% of the variance.
Table 4. Results of the factorial analysis of the first scale
Abbreviations. P.: Planning; I.S.: Social interaction skills and working classmates; A.E.P.: Self-efficacy and expectations; Ans.: Anxiety; E.F.A.: Physical state and state of mind; V.T.: Tasks value; C.C.: Context control; C./A.: Control/Self-regulation; A.I. Internal attributes; A.E.: External attributes; C.O.: Knowledge of objectives and assessment criteria; I.M.: Conception of intelligence being modifiable; M.E.: Extrinsic motivation; Auto.: Self-assessment; M.I.: Intrinsic motivation.
Results of the factorial analysis of the second scale, the Information Processing-related Strategies scale:
Table 5 offers the results of the factorial analysis of the principle components with varimax rotation. The criteria followed are those previously mentioned.
We found that 10 factors explained 61.26 % of the variance.
Table 5. Results of the factorial analysis of the second scale
Abbreviations: O. I.: organising information; P. y C.: Personalisation and creativity. Critical thinking; A.I.: Information acquisition; E.I.: Elaborating information; A.M.MN.: Storage. Memorisation. Use of mnemonic techniques. C.F.B.I.: Source knowledge and information searches: S.I.: Information selection; T.U.: Transfer. Use of information; A.S.R.: Storage. Simple repetition; M.R.: Managing resources to use information efficiently.
3.2. The internal reliability-consistency results
In order to determine the reliability of the questionnaire, we used Cronbach’s alpha coefficient. This is an internal consistency model based on the average inter-element correlation. The reliability of the whole questionnaire (88 items) was α= .897.
The results of the reliability of the questionnaire, scales and subscales are provided in Table 6.
Table 6. The scales, subscales and strategies of the questionnaire presented in terms of the initial theoretical structure designed for the questionnaire. Reliability data.
3.3. The predictive validity results
One of the most interesting aspects of research in this field is to determine to what extent using this questionnaire has helped, or not, to predict performance. If the learning strategies are tools we use to learn, then they must have some influence on academic performance. We did two tests to investigate predictive validity: correlations and multiple regression analyses.
Results of the correlations
We did Pearson’s correlations (product-time), a linear association measure, between the mean scores of the items of both scales and the six subscales and the mean scores of the grades obtained in five core/compulsory course subjects. We found positive correlations in all cases, and these were also significant, except for subscale 5: Information Search and Selection (Table 7).
Table 7. Correlations of the scales and subscales with mean grades
* Correlation is significant at the 0.05 level (bilateral) ** Correlation is significant at the 0.01 level (bilateral)
Results of the multiple regression analyses
We did three multiple regression analyses by selecting the mean grades of the items of the two scales for the first analysis, the six subscales for the second analysis and the twenty-five strategies for the third analysis as the predictive variables. Linear regression estimates the coefficients of the linear equation with one or more independent variables that best predict the value of the dependent variable. We used the mean grade of the five core/compulsory course subjects as the criterion variable in all cases to specify which strategies had a higher predictive value. The results which are included herein refer only to the third analysis in which the predictive variables are the grades of the 25 learning strategies because these results provide more detailed information. We had to be selective in this article for reasons of limited space.
We used the stepwise procedure to calculate the multiple correlation. This involves doing the corresponding ANOVA, calculating the regression coefficients, and also calculating the significance tests for the variables introduced into the regression equation until that time, as well as the beta coefficients and the partial correlations for the variables which had yet to be introduced. If any variable did not contribute significantly to the predicition, it was eliminated from the equation. This procedure was repeated until a regression equation in which all the variables contributed significantly to the prediction was obtained.
Table 8. Results of the multiple regression analyses. Predictors: scores of the 25 learning strategies. Criterion: grades
*** p<.001 ** p<.01
Of the twenty-five variables, five were introduced into the regression equation as they contributed significantly to prediction (Table 8). In order of predictive power, the five variables were: 1) Control/Self-regulation (a strategy of the third subscale, Metacognitive Strategies) (it explained 5% of the variance of the grades, where beta= 0.168, t= 3.295, p<.01); 2) Conception of Intelligence as being Modifiable (a strategy of the first subscale, Motivational Strategies) (it explained 2.5% of the variance of the grades, where beta= 0.140, t= 2.998, p<.01); 3) Source Knowledge and Information Searches (a strategy of the fifth subscale, Information Search and Selection Strategies) (it explained 2.4% of the variance of the grades, where beta= -0.312, t= -5.587, p<.001); 4) Informations Selection (a strategy of the fifth subscale, Information Search and Selection Strategies) (it explained 3.7% of the variance of the grades, where beta= 0.196, t= 3.835, p<.001); and 5) Elaboration (a strategy of the sixth subscale, Information Processing and Use Strategies) (it explained 2.9% of the variance of the grades, where beta= 0.207, t= 3.654, p<.001).
The R coefficient (of the multiple correlation) was 0.405 and the R2 determination coefficient was 0.164, which means that the multiple correlation of the dependent variable with the aggregate of the five predictors was 0.405, and explained 16.4% of the variance of the grades. The F in the ANOVA was 15.334 (p<0.001), thus showing a good level of prediction.
In the regression equation, we found a Metacognitive Strategy (it explained 5%), a Motivational one (it explained 2.5% of variance), two Information Search and Selection ones (which explained 6.1%) and one Information Processing and Use Strategy (that explained 2.9%). These questionnaire strategies had a higher predictive power on academic performance.
3.4. The temporal consistency-stability results
It was also our intention to analyse the extent to which the questionnaire maintained the stability of the results over time as this is also an indicator of instrument quality. To go about this, we correlated the results obtained at the different time points throughout the academic year, with an approximate 4-month interval, for the part of the sample for which we had two phases.
We found high correlation coefficients which were all significant (p<.01). The highest was found in subscale 2, Affective Components, while the lowest was encountered in subscale 1, Motivational Strategies (Table 9).
Table 9. Correlations between strategy scales and subscales in the pretest and post-test
** Correlation was significant at the 0.01 level (bilateral).
4. DISCUSSION AND CONCLUSIONS
Our basic objective was to devise and validate a solid, well structured learning strategies assessment questionnaire that enables greater information acquisition than those used formerly. We believe that we have fulfilled this objective.
We have constructed an instrument that not only collects sufficient information about the various elements making up the “Learning Strategies” construct, but also includes those that have not been considered in other traditionally used instruments (Bernad, 1999; Gargallo, 2000).
The CEVEAPEU offers a suitable construct validity that has been verified by an assessment made by a series of judges, as well as by factorial analyses. It is true that the factorial solutions for the two scales are probably not the most parsimonious from the strictly methodological viewpoint if we consider that many factors have been found which, occasionally, had a very low number of items.
Specifically, there are six factors with only two items: four in scale 1 (External Attributions, Knowledge of Objectives and Assessment Criteria, Conception of Intelligence Being Modifiable, and Extrinsic Motivation); and two in scale 2: Storage/Simple Repetition and Managing Resources to Use the Information Acquired). All these factors are relevant in the construct, and have been acknowledged in the literature (Pintrich, Smith, García & Mckeachie, 1991; Román & Gallego, 1994; Valle, González Cabanach, Núñez & González Pienda, 1998), and it was necessary to maintain them for an all-round assessment to be made. Besides, the internal consistency values found for them proved quite acceptable.
The results suitably adapt to the theoretical structure designed to devise the questionnaire and are, therefore, justifiable.
Furthermore, we wish to point out that the questionnaire has undergone minor modifications in terms of the initial theoretical structure that led to its construction.
In subscale 1, Motivational Strategies, seven strategies were included (Intrinsic Motivation, Extrinsic Motivation, Tasks Values, Perseverance in the Task, Attributions, Self-efficacy/Expectations and Conception of Intelligence Being Modifiable (Table 2). There were still seven strategies in the definitive validation, but the Perserverance in the Task Strategy disappeared in the factorial analysis, while the Attributions Strategies was divided into two: Internal Attributions and External Attributions (Table 6).
In subscale 3, Metacognitive Strategies, the three strategies were those initially foreseen: Knowledge, Planning and Assessment/Control/Self-regulation, but these three strategies were converted into four in the factorial analysis: Knowledge of Objectives and Assessment Criteria, Planning, Self-assessment and Control/Self-regulation.
Finally in subscale 6, Information Processing and Use Strategies, the seven strategies present in the initial theoretical structure (Acquisition, Elaboration, Organisation, Personalisation/Creativity/Critical thinking, Storage, Retrieval and Use) (Table 2) were converted into eight as the Storage Strategy was subdivided into two: Storage/Memorisation/Use of Mnemomic Techniques and Storage/Simple Repetition, while the Retrieval Strategy disappeared as some of its items were included in other strategies. Finally, the Use Strategy was also subdivided to form two new strategies: Transfer/Information Use, and Managing Resources to use the information acquired.
So, although these modifications were minor, they proved logical in the validation process in which the 23 strategies originally foreseen finally totalled to 25.
Moreover, the questionnaire offers robust internal consistency, just as the reliability analysis corroborates (Table 6). The reliability results of both scales are excellent, and those of the subscales are good. Furthermore, the reliability of the strategies in the same table is more than acceptable in the context of this research work if we bear in mind the number of items that the factors/strategies have (see the reliabilities of the ACRA, LASSI and MSLQ questionnaires: Román & Gallego, 1994; Pintrich, Smith, García & Mckeachie, 1991; Weinstein, 1987).
Besides, the predictive validity results on academic performance are aceptable from our point of view despite them not being high, and have been verified with correlations and multiple regression analyses.
The scale scores correlate positively and significantly with the students’ grades; the correlation values are higher for the Affective, Support and Control Strategies than for the Information Processing-related Strategies. All the subscale scores correlate positively and significantly, except subscale 5 which correlates positively but not significantly. The highest correlation values found are for the Metacognitive, Motivational and Information Processing/Use Strategies, and in that order. Therefore, the results reveal that there is a relationship between learning strategies and academic performance. The fact that the correlations do not have high values is coherent with the results obtained by the other researchers who performed correlations using other instruments (Camarero, Martín & Herrero, 2000; Pintrich, 1995; Pintrich & García, 1991; Roces et al., 1999).
The predictive levels we found by using the learning strategies scores to predict performance are no lower than those found by other authors who also resorted to the multiple regression technique for the psychometric studies of their scales, e.g., as Román and Gallego (1994) did for the ACRA scale. The highest predictive power on performance that we found corresponds to one Metacognitive strategy, Control/Self-regulation. It is followed by a Motivational strategy, Conception of Intelligence Being Modifiable, by two Information Search and Selection strategies (Source knowledge…. and Information Selection), and by one Information Processing/Use strategy (Elaboration). Therefore, two strategies correspond to scale 1, Affective, Support and Control Strategies, and three to Scale 2, Information Processing-related Strategies. It is necessary to consider that the percentage of variance explained is not excessively high, and that only 5 of the 25 strategies assessed were introduced into the regression equation when the “step-by-step” procedure was used. It would be worthwhile conducting new tests with larger and more representative population samples to confirm these results.
To finish the statistical section, the temporal consistency-stability results obtained are good and guarantee the questionnaire working well over time.
On the one hand, it is true that our study is not without its limitations. One such limitation is that the sample is not representative of the university population in Spain, but only in the Spanish city of Valencia. Therefore, it would be interesting to extend the questionnaire validation to a representative sample of the Spanish university population, which is a considerably complex task.
On the other hand, we are well aware of the limitations of self-report questionnaires, the type our questionnaire is. Some other limitations are a certain decontextualisation, an excessively generalist character, and the possibility of a subject responding in line with social desirability without being sincere enough. It is also a retrospective measure, that is, the student remembers information about his/her way of working, and not a direct measure taken at the time the task is carried out.
We also realise that making an all-round assessment of the theme of our work in terms of the objectives pursued in this research may require a complementary approach to centre more on contextual and procedural elements; that is, one that can be formulated with other procedures: observation, self-reports of the tasks done, thinking-out-loud protocols, interviews and assessment tasks, among others, to complete the information acquired by means of self-report-type questionnaires, like our own.
However, self-report instruments also have their advantages. For example, they are easily applied and relatively brief when information is to be acquired from large-sized samples, and this enables comparisons to be made among research works as the results are adequately “objective”. It is also necessary to remember that the questionnaires used by the researchers have been validated with methodological rigour, and that normative references have often been available.
Therefore these instruments provide researchers with relevant data to assess the various components of theoretical strategical learning models. They are also useful to not only make students aware of teaching programmes on learning strategies before starting them, which is still a somewhat unusual practice in the university setting (Hernández Pina, Rosário, Cuesta Sáez de Tejada, Martínez Clares and Ruiz Lara, 2006), but to also help them assess themselves. They can also be used as a tool to help teachers diagnose the skills and strategies that they must put into practice with their students. Besides, high school diploma and university students are more capable than younger students of offering complete and relatively objective information on their study methods and strategies (Mckeachie, Pintrich, Lin & Smith, 1986; Zimmerman & Martínez Pons, 1988).
Data from various studies provide evidence of the validity of the external criteria supporting such measures. For example, consistent associations between the grades obtained by students and the learning, motivation and performance indicators have been identified in the MSLQ validation studies. There are also data available reflecting the incidence that the learning strategies assessed with other instruements have on academic achievement, and even on students staying on at or dropping out of university (Cabrera, Bethencourt, Álvarez & González, 2006).
Therefore, we believe that there are more than enough arguments to employ such instruments for research work into the strategies used in the Spanish university setting. All in all, this instrument has been designed for a Spanish university population and has been validated with a Spanish population. The other instruments habitually used in Spain (LASSI and CEAM II) have not benefitted from a better quality validation or adaptation process than that which we have conducted in either the sample or the statistical procedures employed. Finally, LASSI and CEAM II have presented the limitations mentioned in the Introduction which CEVEAPEU has not.
Abascal, J. (2003). El sí mismo en los procesos de enseñanza-aprendizaje. En Mª.V. Trianes y J.A. Gallardo (Coords.), Psicología de la educación y del desarrollo (496-522). Madrid: Pirámide.
Ayala, C.L., Martínez, R. y Yuste, C. (2004). CEAM. Cuestionario de estrategias de aprendizaje y motivación. Barcelona: Instituto de Orientación Psicológica EOS.
Beltrán, J. (1993). Procesos, estrategias y técnicas de aprendizaje. Madrid: Síntesis/Psicología.
Beltrán, J. (2003). Estrategias de aprendizaje. Revista de Educación, 332, 55-73.
Beltrán, J., Pérez, L.F. & Ortega, M.I. (2006). CEA. Cuestionario de Estrategias de Aprendizaje. Madrid: TEA.
Bernad, J.A. (1999). Estrategias de aprendizaje. Madrid: Bruño.
Cabrera, L., Bethencourt, J.T., Álvarez, P. y González, M. (2006). El problema del abandono de los estudios universitarios. RELIEVE, 1 (2), http://www.uv.es/RELIEVE/v12n2/RELIEVEv12n2_1.htm. Consultado en 21 de abril de 2009.
Camarero, F., Martín, F. & Herrero, J. (2000). Estilos y estrategias de aprendizaje en estudiantes universitarios. Psicothema, 12 (4), 615-622.
Corno, L. (1994). Implicit teachings and self-regulated learning. Comunicación presentada en el Annual Meeting of the American Educational Research Association. New Orleans, LA, April, 4-8.
Crocker, J.C. & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart and Winston.
Danserau, D.F. (1985). Learning Strategy Research. En H.F. O’Neil (Ed.), Learning Strategies (pp. 209-240). Nueva York: Academic Press.
De la Fuente, J. & Justicia, F. (2003). Escala de estrategias de aprendizaje ACRA-Abreviada para alumnos universitarios. Revista Electrónica de Investigación Psicoeducativa y Psicopedagógica, 1 (2), 139-158.
Flavell, J.H. (1984). Desarrollo cognitivo. Madrid: Visor;
Flavell, J.H. (1989) Metacognition and cognitive monitoring: A new area of cognitive developmental inquiry. American Psychologist, 34, 906-911;
Flavell, J.H. & Wellman, H.M. (1977). Metamemory. En R. Vall y J.W. Hagen (Eds), Perspectives on the development of memory and cognition (3-33). Hillsdale, NJ.: Erlbaum.
García, T. & Pintrich, P.R. (1991). Student motivation and self-regulated learning. Comunicación presentada en el Annual Meeting of the American Educationa Research Association. Chicago, IL, April 3-7.
García, T. & Pintrich, P.R. (1993). Self-schemas, motivational strategies and self-regulated learning. Communication presented at Annual Meeting of the American Educational Research Association. Atlanta, GA, April 12-16.
Gil, P., Bernaras, E., Elizalde, L. M. & Arrieta, M. (2009). Estrategias de aprendizaje y patrones de motivación del alumnado de cuatro titulaciones del Campus de Gipuzkoa. Infancia y Aprendizaje, 32 (3): 329-341.
Gargallo, B. (1995). Estrategias de aprendizaje. Estado de la cuestión. Propuestas para la intervención educativa. Teoría de la Educación. Revista Interuniversitaria, 7, 53-75.
Gargallo, B. (2000). Procedimientos. Estrategias de aprendizaje. Su naturaleza, enseñanza y evaluación. Valencia: Tirant lo Blanch.
González Cabanach, R., Valle, A., Rodríguez, S. & Piñeiro, I (2002). Autorregulación del aprendizaje y estrategias de estudio. En J.A. González-Pienda, J.C. Núñez, L. Álvarez & E. Soler (Coord.), Estrategias de aprendizaje (17-38). Madrid: Pirámide
González-Pumariega, S., Núñez Pérez, J.C., González Cabanach, R. & Valle , A. (2002). El aprendizaje escolar desde una perspectiva psicoeducativa. En J.A. González-Pienda, R. González Cabanach, J.C. Núñez Pérez & A. Valle (Coords.), Manual de Psicología de la Educación (pp. 41-66). Madrid: Pirámide.
Hernández Pina, F., Rosario, P., Cuesta Sáez de Tejada, J.D. Martínez Clares, P. & Ruiz Lara, E. (2006). Promoción del aprendizaje estratégico y competencias de aprendizaje en estudiantes de primero de universidad. Revista de Investigación Educativa, 24 (2), 615-632.
Jornet, J. M. & Suárez, J. M. (1996). Pruebas estandarizadas y evaluación del rendimiento: usos y características métricas. Revista de Investigación Educativa, 14 (2), 141-163.
Justicia, F. y Cano, F. (1993). Concepto y medida de las estrategias y estilos de aprendizaje. En C. Monereo (Comp.), Las estrategias de aprendizaje: procesos, contenidos e interacción (pp.113-126). Barcelona: Domènech Ediciones.
Justicia, F. & De La Fuente, J. (1999). Análisis factorial de las escalas ACRA en una muestra de alumnos universitarios. Mente y Conducta en Situación Educativa. Revista electrónica del Departamento de Psicología. Universidad de Valladolid, 1 (1), 51-66.
Kaiser, H.F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141-151.
Kirby, J.R. (1984). Cognitive strategies and educational performance. Orlando:Academic Press.
Mckeachie, W.J., Pintrich, P.R., Lin, Y.G. & Smith, D. (1986). Teaching and learning in college classroom: A review of the research literature. An Arbor, MI: National Center for Research to Improve Postsecondary Teaching and Learning, The University of Michigan.
Monereo, C. (1995). De los procedimientos a las estrategias: implicaciones para el Proyecto Curricular Investigación y Renovación Escolar (IRES). Investigación en la escuela, 27, 21-38.
Monereo, C. (1997). La construcción del conocimiento estratégico en el aula. En Mª.L. Pérez Cabaní, La enseñanza y el aprendizaje de estrategias desde el curriculum (pp. 21-34). Gerona: Horsori.
Monereo, C. & Castelló, M. (1997). Las estrategias de aprendizaje. Cómo incorporarlas a la práctica educativa. Barcelona: Edebé.
Nisbet, J. y Shucksmith, J. (1987). Estrategias de aprendizaje. Madrid. Santillana.
Paris, S.G., Lipson, M.Y. y Wilxson, K.K. (1983). Becoming a strategic reader. Contemporary Educational Psychology, 8, 293-316.
Pintrich, P.R. (1995). Understanding self-regulated learning. New Directions for Teaching and Learning, 63, 3-12.
Pintrich, P.R & De Groot, E.V. (1990). Motivacional and self-regulated learning components of class-room performance. Journal of Educational Psychology, 82, 33-40.
Pintrich, P.R & García, T. (1991). Student goal orientation and self regulation in the class-room. En M. L. Maher, y P. R. Pintrich (Eds.), Advances in motivation and achievement (vol. 7) (pp. 371-402). Greenwich: CT, JAI Press.
Pintrich, P.R. & Schrauben, B. (1992). Student’s Motivational Beliefs and their Cognitive Engagement in Classroom academic Tasks. En D.H. Schunck y J. Meece (Eds.), Students Percepcions in the Classrrom (pp. 149-183). Hillsdale, N.J.: Lawrence Erlbaum.
Pintrich, P. R., Smith, D. A. F. García, T. & Mackeachie, W.J. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). Ann Arbor: Universidad de Michigan. Technical Report No. 91-B-004.
Popham, J. (1990). Modern educational measurement. Boston, MA: Allyn and Bacon.
Pozo J.I. (1990). Estrategias de aprendizaje. En C. Coll, J. Palacios y A. Marchesi, Desarrollo psicológico y educación, II. Psicología de la educació (pp. 199-221). Madrid: Alianza.
Pozo, J. I. y Monereo, C. (Coords.) (1999). El aprendizaje estratégico. Madrid: Santillana.
Roces, C., González-Pienda, J. A., Núñez, J. C., González-Pumariega, S., García, Mª. S. y Álvarez, L. (1999). Relaciones entre motivación, estrategias de aprendizaje y rendimiento académico en estudiantes universitarios. Mente y Conducta en Situación Educativa. Revista electrónica del Departamento de Psicología. Universidad de Valladolid, 1 (1), 41-50.
Roces, C., Tourón, J. y González, M. C. (1995). Validación preliminar del CEAM II (Cuestionario de estrategias de aprendizaje y motivación II). Psicológica, 16 (3), 347-366.
Roid, G. H. & Haladyna, T. M. (1982). A technology for test-item writing. Nueva York: Academic Press.
Román, J.M. & Gallego, S. (1994). ACRA. Escalas de estrategias de aprendizaje. Madrid: TEA.
Valle, A., González Cabanach, R., Núñez, J. & González-Pienda, J.A. (1998). Variables cognitivo-motivacionales, enfoques de aprendizaje y rendimiento académico. Psicothema, 10 (2), 393-412.
Weinstein, C. E. (1987). LASSI User’s Manual. Clearwater, FL: H&H and Publishing Company.
Weinstein, C. E. (1988). Assessment and training of student learning strategies. En R.R. Schmeck (Ed.), Learning strategies and learning styles (291-316). Nueva York: Plenum Press.
Weinstein, C.E. & Danserau, D.F. (1985): Learning strategies: the how of learning. En J.W. Segal et al., Thinking and learning strategies (pp. 125-142). Hillsdale: Erlbaum.
Weinstein, C.E., Husman, J. & Dierking, D. (2002). Self-Regulation Interventions with a focus on learning strategies. En M. Boekaerts, P.R. Pintrich & M. Zeinder; Handbook of Self-regulation (pp. 727-747). San Diego: Academic Press.
Weinstein, C.E. & Mayer, R.E. (1985). The teaching of learning strategies. En M. C. Wittrock, (Ed), Handbook of research on teaching (pp. 315-327). Nueva York: MacMillan.
Weinstein, C. E., Palmer, D. R. & Schulte, A.C. (1987). LASSI: Learning and Study Strategies Inventory. Clearwater, FL: Publishing Company.
Weinstein, C. E., Zimmerman, S.A. & Palmer, D. (1988). Assessing learning strategies: the design and development of LASSI. En C.E. Weinstein, E. T. Goetz y P.A. Alexander (Eds.), Learning and study strategies (25-40). San Diego: Academic Press Inc.
This work is part of the research work “Teaching and learning strategies in universities. Analysis of the incidence of fundamental variables on the ways students face learning” (code SEC2003-06787/PSCE). Approved by the Spanish Ministry of Science and Technology.
QUESTIONNAIRE TO ASSESS LEARNING STRATEGIES IN UNIVERSITY STUDENTS
( CEVEAPEU )
A) Details of the student answering the questionnaire:
Name and surname(s): _____________________________________________________ Date________
Faculty or School: _____________________________________________________________________
The degree you are studying: _______________________________________________
Sex: ¨ Male ¨ Female
Undergraduate course: ¨ First ¨ Second
Degree year: ¨ First ¨ Second ¨ Third ¨ Fourth ¨ Fifth
Age: ¨ 17-18 ¨ 19-20 ¨ 21-22 ¨ 23-24 ¨ 25-26 ¨ 27-28 ¨ 29 and above
Choice of career: ¨ My first choice ¨ My second choice ¨ My third choice
¨ My fourth choice ¨ Other options
Parents’ level of education:
¨ No studies ¨ No studies
¨ Primary education ¨ Primary education
¨ Secondary education ¨ Secondary education
¨ High school studies ¨ High school studies
¨ Undergraduate studies ¨ Undergraduate studies
¨ Graduate studies ¨ Graduate studies
¨ Doctoral degree ¨ Doctoral degree
Grades obtained in the previous academic year:
1.__________________ ¨Fail ¨Pass ¨Good ¨Outstanding ¨Honours
2.__________________ ¨Fail ¨Pass ¨Good ¨Outstanding ¨Honours
3.__________________ ¨Fail ¨Pass ¨Good ¨Outstanding ¨Honours
4.__________________ ¨Fail ¨Pass ¨Good ¨Outstanding ¨Honours
5.__________________ ¨Fail ¨Pass ¨Good ¨Outstanding ¨Honours
6.__________________ ¨Fail ¨Pass ¨Good ¨Outstanding ¨Honours
7.__________________ ¨Fail ¨Pass ¨Good ¨Outstanding ¨Honours
8.__________________ ¨Fail ¨Pass ¨Good ¨Outstanding ¨Honours
B) Responses to the items of this questionnaire: