Implementing interventions and assessing SRL

Planning

During the upcoming week we are going to be dwelling on the subject on how to implement effective interventions and how to assess SRL. So I’m anticipating that we will get information about how students can be helped to learn to regulate their learning and with it, practical suggestions for teachers how to teach and support SRL in students, all this grounded in the recent research about the interventions that have been found most useful and and effective. In addition, I anticipate that we are going to learn about different methods to assess SRL, and how we could apply that in our own studying and work.

My group also has to work on planning the teaching session the be held for the other course participants. So this week I really have to concentrate on that. I plan to attend the lecture about interventions and assessment of SRL and make the ICE notes, but I have to leave the articles to be read later on when I have finished all the other urgent tasks.

Three main points from the articles

Dignath, C., Buettner, G., Langfeldt, H-P. (2008). How can primary school students learn self-regulated learning strategies most effectively? A meta-analysis on self-regulation training programmes.

1. SRL among young children. A large amount of research has been conducted about how to make learning more efficient in the knowledge based society that requires new kind of competencies than before. During the last 30 years the construct of SRL has been developed to meet these new requirements. In SRL research the focus has been on how learning functions (focusing on the learner’s cognitive and motivational processes) as well as on how instructions function (focusing on the interaction between learner and instructor in a social environment).

Research has given empirical evidence that young children can and do engage in activities to self-regulate their learning. According to Dignath, Buettner and Langfeldt (2008) “the major advantage of training children how to self-regulate their learning is that during these first crucial years in school, students set up learning and self-efficacy attitudes, which are easier to change than when students have already developed disadvantageous learning styles and learning behaviour.”

Empirical studies have shown that students can develop strategies based on their experiences, but that strategy construction can also be guided to develop SRL strategies. However, instructing and training students about learning strategies at the first years of elementary school is still rare, thus leading to lack of metacognitive knowledge among young children.

2. Main effects of SRL interventions. The meta-analysis conducted by Dignath, Buettner and Langfeldt (2008) examined self-regulated learning interventions’ effectiveness on primary school students’ academic performance, strategy use, and motivation. It also investigated the effect of the different training characteristics on the effectiveness of the intervention.

The results of the meta-analysis showed that self-regulated learning training programmes have a positive effect on learning outcomes, strategy use, and motivation even for primary school childrenThe highest benefits from the analyzed interventions can be gained in mathematics performance (effects were higher for mathematics than for
reading and writing), motivational outcomes, as well as the use of cognitive and metacognitive strategies. These results revealed that young children benefit more in strategy use and motivation than primary school students in higher grades do.

The results of the meta-analysis conducted by Dignath, Buettner and Langfeldt differ to some extend compared to the findings of the meta-analysis conducted by Hattie et al. in 1996. In meta-analysis by Hattie et al. the highest effects of intervention was found to be on performance, and the lowest on motivation and study skills. In addition, academic performance attained the highest effect in the meta-analysis of Hattie et al., whereas the average effect size was higher for cognitive and metacognitive skills than for academic performance in the meta-analysis of  Dignath, Buettner and Langfeldt. These differences in the results could be explained by age differences, since the studies used in the meta-analysis by Hattie et al. were conducted with older students or adults.

3. Inferences drawn from the meta-analysis results by Dignath, Buettner and Langfeldt (2008) suggest that:

  1. A training programme should be based on social-cognitive theories
    • The interventions, which were based on social-cognitive theory or a combination of social-cognitive and metacognitive theories, led to the highest effect sizes, while interventions which were based on motivational theories reached only low effects. In the social-cognitive perspective, social factors play a significant role in cognitive development, which might be best suited to young children’s learning.
  2. A training programme should train cognitive (especially elaboration and problem solving strategies), metacognitive (especially planning strategies), and motivational strategies (especially feedback)
    • The highest effects were found for interventions that combined the instruction of different types of strategies. Interventions should be integrative and consider various different aspects of learning, including metacognitive and motivational aspects.
  3. A training programme should provide knowledge about strategy use and about its benefit
    • The highest effect sizes were found for interventions which provided students’ with knowledge about strategies and benefits of applying the strategies, or additionally also stimulated metacognitive reasoning.
    • Most effective training programmes provided students with feedback about their (strategic) learning. In addition, the instruction of action control strategies influenced positively students’ strategy use. This is related to the strategies of metacognitive reflection, and might be most efficient because it is very close to the concrete learning content and strategy use.
  4. Programme developers should emphasize the implementation of group work.
    • Effect sizes were significantly higher for interventions that did not train students by group work than for those that did. In the studies that were included in the meta-analysis only very little information about the implementation of group work in the learning settings were found. Thus, a possible reason for the negative effect of group work on training effects at primary school level might be that students were not used to working in groups and did not receive enough instruction about cooperative learning.
    • Primary school students may not yet have the competencies to work efficiently in groups and thus need instructions about cooperation. Therefore, researchers and trainers should give instructions of group work strategies before and during using this method.

Dignath, Buettner and Langfeldt  (2008) remind that there are several questions concerning the implementation of training programmes by teachers that remain unanswered and require further investigation. Also, further research should be made about the differences of instructing self-regulated learning to younger and older students in order to improve interventions and to make them suitable for the different needs of students at different ages.


Panadero, E., & Alonso-Tapia, J. (2013) Self-assessment: Theoretical and practical connotations. When it happens, how is it acquired and what to do to develop it in our students.

1. Self-assessment as part of self-regulation. Self-assessment can be viewed from two perspectives; firstly, it can be seen as instructional process that teachers initiate for students to assess their work in school, and secondyl, as a self-regulatory process exerted by the students themselves to evaluate their own learning process. Self-assessment is a self-reflective process through which students can evaluate their learning process (during and after it) and final products, and thus develop their learning skills based on their “lessons learned”.

According to Panadero “self-assessment is the qualitative assessment of the learning process, and of its final product, realized on the basis of pre-established criteria”. (Panadero and Alonso-Tapio, 2013) This refers to the fact that students should set goals based on the assessment criteria before starting the task or activity, thus it is easier to self-regulate during learning and also assess how the goals are reached. Assessment criteria can be 1. set by the teacher, 2.  set externally but accepted internally by the student, or 3. criteria can be set by the student.

Goals and assessment criteria should be clear because this can enhance student motivation, feeling of control over the tasks (knowledge of what is expected of them), and gives opportunity to better self-regulation.

2. How self-assessment is acquired. Self-assessing different tasks and phases in one’s learning is complex, and should thus be taught and supported by teachers.

The conditions needed for self-assessment are:

  • awareness of the value (usefulness for learning) of self-assessment,
  • access to criteria the assessment is based on (the students should know the criteria before starting the task, so they can set goals, monitor their progress during the execution, and assess their success after finishing the task),
  • tasks assessed need to be specific (if a task is not well-defined, it will complicate self-assessment).

The instructional aids to promote self-assessment are:

  • self-assessment modelling (watching a model, an “expert”, perform the task),
  • direct instructions and assistance (give instructions for and feedback on the student’s performance),
  • cues to help to know when to self-assess (by giving instructions and also modelling),
  • practice (training self-assessment in various contexts and tasks, e.g. in reading, writing and problem solving situations),
  • opportunities to review and improve (detecting mistakes, and being allowed to correct them, improves performance).

3. Instructional help for self-assessment. There are three major types of instructional help that promote self-assessment:

  1. Self-grading or self assessment without specified assessment criteria, in which students assess or grade their performance at the end of the tasks without assessment criteria.
  2. Self-assessment using rubrics. Rubric is a tool that explicitly states the assessment criteria, and thus helps the students to evaluate their performance against the criteria.
  3.  Self-assessment through using scripts, which are  a set of questions  organised systematically and focus on the process of performing a task.

In self-grading, when assessment criteria is not given in advance, the students self-assessment is not accurate and self-regulation skills are not likely to develop. By contracts, rubrics and scripts promote self-assessment as they give the assessment criteria based on which the students can assess their progress.

If students are given the assessment criteria before starting the task (not just after finishing it), the students have a better opportunity to set goals for themselves based on the assessment criteria, choose their learning strategies, monitor (assess) and modify the strategies during the execution of the task, and finally assess their success in reaching the goals after finishing the task, which also helps them to develop their learning skills and use them effectively in the next tasks to come.

ICE notes

ICE 1How can SRL be studied? SRL is an internal process, so how can it be reliably measured as we usually have to make inferences about SRL based on external cues (such as behavior). I would suspect that this is especially hard regarding young children as, compared to adults, they are not yet able (mature enough) to recognise the features of SRL in themselves and talk about these, as adults would. Though, there are of course differences among adult as well to what extend they are aware of SRL processes and thus able to discuss them.

Quite often young childrens’ behavior is observed to draw inferences about SRL, but how valid conclusions can one make about SRL based on external cues. Take for example the well-known Marshmallow test:

Surely one can see that some children are better able to resist the temptation of a marshmallow than others for a period of time. But what are the conclusion that one can draw from a test like this, done in a test environment, about the childrens’ self-regulation; are the children who can resist the marshmallow better self-regulators, and if so, does this transfer to other domains (such as reading, writing, or problem solving) in real life, and does this preserve over time?

Contrasting to observing behavior, SRL can be studied based on the study subjects’ own reports of their SRL through, for example, surveys and interviews. Some obvious problems that comes to my mind is that to what extend are the study subjects (even if they were aduls) able to recognise SRL processes in their own learning, and how truthfully they answer the questions. That is,  do the study subjects answer the questions based on how they view their SRL processes or rather how they would like to see themselves (the ideal).

So, SRL is not easy to measure and when conducting studies, it should be carefully considered that the study is as valid and reliable as possible. (A valid measure measures what it is supposed to, where as non-valid measure measures something that it was not supposed to, or is too narrow. Reliability has to do with the quality of measurement. In its everyday sense, reliability is the “consistency” or “repeatability” of the measures. Reliability problem surfaces when we are not able measure the true value but only the measured value that includes random error. ) Also, the sample size and composition should be meaningful, and it also needs to be considered whether the study is conducted in test environment or in “real world”.

My point above is, that one needs to be careful and critical when conducting research on SRL, and also when reading about and forming views of SRL based on different research made.

ICE 2There are different methods to measure SRL: self-report, observation of overt behavior, interview, thinking aloud protocols, traces of mental event and processes, learning diaries, case studies, which all have their good and bad sides. One should choose the research method(s) used that are in line with one’s research goals.

Lately I have been thinking about my thesis subject, and how I am going to collect the data for it. I’m interested in expertise, so my thesis will most likely be related to that. Though, I’m only at the beginning of my studies, and do not yet know so much about learning of expertise, so my plans for the thesis are not yet clearly defined. However, I think that the obvious choice for collecting the data is to interview (using semi-structured interview) different experts as I want to go deep in the experts’ minds in trying to understand the underlying factors of learning of expertise.

Of course, being able to interview only a limited number of experts for the thesis might pose a problem in the sense that to what extend the inferences made from the data can be generalised to and across other experts (external validity issue). However, I try to surpass some of the validity problems by choosing experts with different background variables (interview both men and women, from different age groups, and different domains of expertise). And what I really carefully need to consider, and do some studying about, is how to construct the semi-structured interview – what kind of questions need to be asked that will tap on the learning of expertise so that I would be really able to study and draw conclusions about it.

 Reflection

My planning for this solo phase succeeded in that I initially decided that due to schedule pressure (exams and group work in other courses) I will leave reading of the articles to the last week of studies before Christmas break. Prioritising and scheduling tasks helped me to concentrate on those ones that needed to be completed first before starting others.

One challenge I faced in this task was due to technical problems, namely, the bad connection when Mr. Ernerto Panadero was giving us a lecture online. Due to the nature of the subject that I find quite difficult (understanding different research methods and how to implement them), it was a pity that at some points it was really hard to follow the lecture. I could grab some ideas from the lecture, but some issues remained unclear to me, so I decided that I should look at the lecture slides again before writing ICE notes.

Overall, I think I have succeeded quite well on this course of Self-regulated learning. I know so much more about SRL now than I did at the beginning of the course, and I think I have started to see the big picture more and make connections between SRL, CL, and learning of expertise. Though, I’m sure that there is still a lot more to learn. 🙂

Considering my motivation and learning strategies, I’m quite happy how I did. There were some occasional drops in motivation, usually when faced with pressing schedules or issues (concepts or such) that were hard to understand. But, taking short breaks, breaking the tasks into smaller pieces, and scheduling helped to overcome motivational challenges. Also, as I find the subject of SRL to be very interesting and I want to understand it, it helped me to persevere in the face of challenges.

Reading many articles about SRL and writing about them to this blog really helped to understand what are the different factors that affect self-regulation and self-regulated learning (emotions, motivation, metacognition). I would have liked to spend some time in the classes to discuss the articles with the other students and the teachers. It would have been interesting to discuss what the others found to be the key points in the articles, and how they view SRL after reading the articles. Additionally, we could have discussed the issues we had trouble understanding, and hopefully this way the students and the teachers together could have helped to clarify those difficult issues.

In the future in my LET studies, I hope to further deepen my knowledge about SRL, CL, and learning of expertise. Now that I have gained knowledge about these issues, I hope I’m able to start forming relevant connections between them more and form a big picture about the features of an active and efficient self-regulated learner and learning, and what factors promote those in the best way.

References:

Dignath, C., Buettner, G., Langfeldt, H-P. (2008). How can primary school students learn self-regulated learning strategies most effectively? A meta-analysis on self-regulation training programmes. Educational Research Review, 3, pp. 101-129

Panadero, E., & Alonso-Tapia, J. (2013). Self-assessment: Theoretical and practical connotations. When it happens, how is it acquired and what to do to develop it in our students. Electronic Journal of Research in Educational Psychology, 11(2), 551-576.

Advertisements

About kaipihla

A student in Learning, Education and Technology Master's degree program in University of Oulu, Finland.
Video | This entry was posted in Self-regulated Learning and tagged , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s