A Study of Video-Mediated Opportunities for Self-Directed Learning in Required Core Curriculum

Improving a required course in our curriculum that has proven to be a challenge for our students was the focus of this study. Surveys of both students and instructors attempted to identify specific problem areas. Using the information from these surveys, the researchers developed a series of videos to explain vital course concepts and deployed these into the course sections. The purpose of the videos is to provide consistency across the multiple modalities in which we offer our courses (including online, classroom and via videoconferencing) and to improve overall student understanding. This project seeks to determine how supplemental content focusing on material identified as “difficult,” by students and instructors, can impact student performance. Challenges include the deployment of the videos across various modalities and obtaining sufficient student feedback.


Introduction
This study focuses on an attempt to improve student understanding of key concepts in a course which has gained a reputation for being difficult. Introduction to Research Methods (RSCH 202) is a class required in every program. It teaches students the basics of research, including specific assignments that deal with the following: composing an annotated bibliography; writing a literature review; developing a research question and testable hypothesis; and determining an appropriate methodology. Ideally, the class is taken at the sophomore level, as students have two prerequisites, one in English and the other in Mathematics. Students have a choice of English Composition or Technical Report Writing, either of which should provide the necessary writing foundation required to be successful in RSCH 202. English Composition even features an annotated bibliography assignment. Students may also choose from Statistics with Aviation Applications or Business Statistics, both of which should ready students for the components of the course which require a familiarity with quantitative analysis.
Regardless of this carefully established prerequisite structure, students struggle with RSCH 202, both academically and motivationally. The multi-disciplinary approach to the course appears to target two specific areas where students often display deficiencies: math and writing. Instructors consistently report that students seem to lack familiarity with concepts that should be covered in prerequisite courses. These concerns raise questions about the retention of prerequisite knowledge and the inability to properly apply those skills to RSCH 202. Additionally, and perhaps most importantly, there are misgivings that the prerequisite classes themselves do not adequately or accurately support the level of work required in RSCH 202. These academic concerns, coupled with a clear lack of learner engagement, make the course a challenge for both instructors and students.
A research team, consisting of the course monitor, as well as experienced instructors in both of the prerequisite areas, decided to explore possibilities for improving student success in the course. It was quickly determined that any assistance would need to be delivered electronically to ensure consistency of message and should target specific areas identified by students and instructors as particularly challenging. Furthermore, the team did not want to incorporate the material into the course itself, thereby giving an impression of additional "required" assignments but, instead, sought to create the sense that these tools were truly supplemental in nature. This distinction served two purposes: to determine student motivation in the course by observing whether students would choose to utilize optional material and to demonstrate whether self-perceived "problem areas" in the class matched initial student and faculty surveys (and anecdotal evidence).

Incorporating Technology in the Classroom
The incorporation of media-based learning tools has accelerated over that past several decades as technology has delivered new innovations into the classroom. Early improvements saw the whiteboard replace the traditional chalkboard. The 1980s were characterized by the use of overhead transparencies and videotapes, which gave way to videodiscs, CD-ROMS and DVDs in 1990s. Eventually, digital projectors and Microsoft PowerPoint became ubiquitous, and many classrooms now have smartboards and high-speed Internet connections. Individual computers, tablets and/or hand-held devices are now replete throughout dormitories, libraries, lecture halls and small classrooms alike.
Efficacy of Multimedia. Typically, the introduction of multimedia into the classroom setting or via independent study has been heralded as having a positive effect on learning and retention (Bagui, 1998;Barford & Weston, 1997;Fletcher, 2003;Kozma, 2001;Mayer, 2001;Schmid et al., 2014;Targamadze & Petrauskiene, 2010). Various studies have also suggested that student satisfaction and motivation are higher in courses that use multimedia materials (Astleitner & Wiesner, 2004;Yarbrough, 2001). More recent research by Rackaway (2012) specifically credited the higher levels of student engagement with the material facilitated by the incorporation of multimedia as having a positive effect on the achievement of learning objectives.
Conversely, several studies have questioned the efficacy of these instructional strategies. Janda's (1992) research comparing a teaching design including multimedia instruction with two alternative formats yielded mixed results. Student surveys indicated an overwhelming positive response from the students using the multimedia (i.e., they "liked" the video segments and reported that they helped them understand concepts). However, students in the multimedia sections scored lower on the course assessment measures than those in the traditional sections and likewise reported no increase in interest or sense of increased knowledge in the subject area in a follow-up questionnaire. These finding led Janda to conclude, "It is not enough to prove merely that students like multimedia applications. Researchers must also show that this teaching method contributes to learning -either in student self-reports of learning, or on tests of course content, or along some other dimensions of motivation or understanding" (p. 352). Janda's (1992) study was conducted in the early days of multimedia technology and thus reflects the incorporation of a novel and unfamiliar learning tool. He questions the return on investment aspect of this media as a learning supplement based on its high cost (at the time). Clearly, this is not the current state of affairs. The use of video tutorials is commonplace among academic disciplines as well as everyday life tasks and skills. Additionally, multimedia production has become highly accessible to even casual computer users both in terms of cost and ease of use.
Yet skepticism remains. Later, studies by Buzzell et al. (2002) using web-based tutorials yielded neutral findings when compared to traditional teaching methods. Borokhovski et al., (2015) in their recent meta-analysis of technology integration concluded that, "even development and introduction to educational practice of much more sophisticated computer tools and applications (what is often referred to as interactive, multi-functional, mobile, "smart" technology) have not impacted students' learning outcomes impressively enough to consider the issue unequivocally resolved in favor of proponents of educational technology (p. 1764). Rackaway (2012) blames the lack of attention and effort focused on determining best practices and effective use of multimedia in classrooms leading to haphazard and partial implementation for the lack-luster results of evaluative studies. As such, he calls for, "a pedagogical model that introduces a comprehensive multimedia toolkit geared to specific learning outcomes goals" (p. 190) in order to bolster multimedia as an effective teaching tool.
Potential Advantages. When used effectively and in line with best practices, Rackaway (2012) argues that multimedia provides for the accommodation of multiple learning styles that better engage students in course materials. Williams and Harkin (1999) also highlighted the advantage of reaching students with diverse learning styles and emphasized thoughtful inclusion of multimedia based on addressing specific learning goals, problem areas or deficiencies. Similarly, Ludwig, Daniel, Froman & Mathie (2004) called for its use to be based on intentional pedagogy with a clear and focused purpose. They further note various pedagogical rationale for incorporating media instruction including, to raise interest level, to enhance understanding, and to increase memorability. Specifically, they conclude that the incorporation of multi-media learning supplements can be particularly advantageous for learning complex topics and dynamic processes that unfold over time. Additionally, they argue that quality media material results in better cognitive encoding and subsequent easier retrieval, leading to increased comprehension and retrieval.
Low-Cost Educational Video. Video supplements have been widely used throughout classrooms to supplement traditional instructional materials (Bravo et al., 2011;Green et al., 2003). As noted previously, in the early days videos were difficult and resource-intensive to produce. However, more recently "low-cost educational videos," characterized by the ease of production, short length and limited focus, have emerged as popular instructional strategies (Bravo et al., 2011;Palmer, 2007;Simo et al., 2010). This video format has been implemented in traditional and blended courses as a supplement to existing course materials and serves as the main delivery format for the growing number of Massive Open Online Courses (MOOCs) (Brame, 2015). Various studies assessing the use of low-cost educational videos, aimed at supporting independent learning have concluded that they address the previous problems related to video inclusion and represent an effective learning tool (Allen & Smith, 2012;Hsin & Cigas, 2013;Kay, 2012;Lloyd and Robertson, 2012;Rackaway, 2012) Self-Directed Learning Self-directed learning has long been conceptualized as a pillar of higher education. Knowles (1975) first defined the concept as "a process in which individuals take the initiative with or without the help of others in diagnosing their learning needs, formulating goals, identifying human and material resources, selecting appropriate learning strategies and evaluating learning outcomes" (p. 18). Skiff and Beckendorf (2009) further broke down the strategy into components, including, identifying learning needs, planning learning, goals, discovering learning resources, implementing required learning tactics and strategies, and subsequently evaluating learning outcomes.
Specifically, various researchers concluded that incorporating reflection on their learning processes, evaluation of their depth of knowledge, and identification of areas that require further development, are important aspects of increasing and maintaining student engagement (Brown, 2004(Brown, -2005Nicol & Macfarlane-Dick, 2006). Herman (2012) also found that involving students in course content increases their intrinsic motivation and connection to the material.

Self-Directed vs. Self-Regulated Learning
Researchers Jossberger, Brandgruwel, Boshuizen and Van de Wiel (2010) further differentiate between the concepts self-directed learning, whereby learners determine their own goals, and self-regulated learning, whereby the learning activities to reach already established goals are managed by learners. Likewise Conradie (2014) considers self-directed learning to exist "at the macro level (i.e. planning of learning trajectory), while self-regulated learning is placed at the micro level (i.e. learning task level, self-controlled learning activities)" (p.255). Considering this distinction, it is selfregulated learning that can be most widely promoted in the context of established mandatory learning outcomes and curricular requirements (such as core and degree or program specific sequences of courses). Both self-directed and self-regulated learning embody a shift in responsibility from the teacher to the learner with regard to the control and management of the learning process. Zimmerman (1989) noted, "Students can be described as self-regulated to the degree that they are metacognitively, motivationally, and behaviorally active participants in their own learning process" (p. 329). Building on these notions, Ley and Young (2001) advocate for embedding selfregulation opportunities into learning environments to promote active engagement. Anderson (2001) has demonstrated that active learners tend to perform better than passive learners in hypermedia learning environments (such as media-rich on-line modalities). Thus, the incorporation of self-regulated media into existing curriculum should facilitate both increased student involvement and subsequently increased performance.

The Challenge of Student Apathy
Student apathy and disengagement constitute chronic challenges in higher education and represent the antithesis of the motivation that is both required for, and fueled by, self-regulation and self-direction. According to Zimmerman (2005), "A personal limitation that leads to dysfunctions in self-regulation is motivational; namely, the presence of apathy or disinterest. When a skill or its outcomes are not perceived as valuable, there is no incentive to self-regulate" (p. 27). Christman and House (2014) cite social and cultural shifts that have negatively impacted student motivation and the valuing of higher education, including transformation in family life, generational differences and the multitude of changes brought on by the advancement of technology. Thus, the challenge for educators becomes not only to construct specific course activities that allow for self-regulation and individual autonomy, but also to construct curriculum that is purposeful and relevant and to adequately convey this meaning to students.

Methodology and Findings
Design-based research (DBR) is a productive response to the long-standing criticism of educational research for its "weak link with practice" (van den Akker, Gravemeiher, McKenney & Nieveen, 2006). Anderson and Shattuck (2012) begin their review of the emerging research methodology with a synthesis of characteristics that define a quality DBR study:  Being situated in a real educational context  Focusing on the design and testing of a significant intervention  Using mixed methods  Involving multiple iterations  Involving a collaborative partnership between researchers and practitioners  Evolution of design principles  Practical impact on practice (p. 16-18) Bradley (2006, 2008) assert that DBR is the method for discovering "how promising instructional interventions might be implemented in classrooms to achieve valued pedagogical goals" (2006, p. 151).
Initial discussions among the research team were focused on designing an intervention for the course RSCH 202 -Introduction to Research Methods and the gathering of evidence that it had a practical impact in actual classes. Further discussion emphasized the need to involve practitioners outside of the research team, how multiple iterations would be necessary, and the mix of quantitative and qualitative research methods required to determine the details of the proposed intervention then investigate the significance of student outcomes. The commitment to documenting the iterative design principles (and sharing them with colleagues) turned a cross-disciplinary discussion into a DBR study.

Phase 1 -Topic Selection
The research literature reviewed above provides multiple calls for specific, thoughtful, and intentional selection of topics that benefit from the inclusion of multimedia content (Ludwig et al., 2004;Rackaway, 2012: Williams & Harkin, 1999. Anecdotal evidence from instructors suggests that students struggle to connect material from the pre-requisite courses -writing and statistics -to aid in understanding concepts or completing assignments in the RSCH 202 course. The research team included course developers for writing, statistics, and RSCH 202 as well as an education researcher from an unrelated discipline: physics. Although there was confidence that the troublesome concepts and assignments for the course were already known to the team, formal evidence-gathering was the natural first step. Participants. All of the 16 sections (328 active students) in the nine-week term beginning 19 October 2105 were invited to participate in a survey to inform our choices about where video intervention(s) might be most needed. Among the 53 respondents to the initial survey, 33 of them were taking the course in the asynchronous format. The remaining 20 respondents were enrolled in a section that met synchronously each week of the term, either in traditional classrooms or via the University's web conferencing platform, EagleVision (powered by Adobe Connect). The invitation to participate was repeated in the terms beginning 15 February 2016 and 21 March 2016 -collectively called "spring" in the data analysis. There were 81 active students in the four February sections and 328 in the sixteen March sections. The 84 respondents in the spring included 53 asynchronous sections and 31 in synchronous sections.
Survey design. To gather evidence for the topics that students found most difficult, a survey was available alongside the end-of-course evaluations and instructors were asked to post an announcement in the LMS requesting student participation. Instructors received similar questions regarding the ease/difficulty of various topics and assignments common to the synchronous and asynchronous sections -a total of 22 items on a Likert-style scale. Students were also asked to agree/disagree with the statement that their two pre-requisite courses had prepared them for the writing and statistical tasks in RSCH 202 as well as asking about the potential benefit of videos targeting the most difficult concepts in the course.
Topic selection. The fall responses were reasonably consistent with what the course developer had heard repeatedly from instructors and her own students in previous terms. The 16% response rate precluded statistical analysis beyond the frequency of "difficult" and "very difficult" responses to inform our selections. The literature review, the quantitative analysis, and the qualitative analysis assignments were the most difficult course tasks for students to complete. The difficult topics included reviewing research literature, statistical tests, qualitative research, and quantitative research.
Keeping in mind the goal of producing videos that are short and tightly focused, ten topics from the pre-requisite courses and three topics fundamental to the course under study were selected and team members assigned to create videos (Bravo et al., 2011;Palmer, 2007;Simo et al., 2010). The topics are listed below in the order they would be presented in the course: The distribution of these topics in the course schedule corresponded with at least one video for the first seven weeks of the nine-week term.

Phase 2 -Initial Video Intervention
The team produced the 13 videos to pilot in two sections during the term beginning 21 March 2016. Each video link came with another link to a short online survey to collect student feedback immediately after viewing the video.
Video production. Using screen capture software, the videos range in length from 2 to 12 minutes, most in the 6 to 7 minute range. All 13 slideshows use the same template for the sake of continuity in this course element. They are posted in an unlisted (limited access) channel on YouTube for the initial implementation.
Survey. Students were presented with four forced-choice questions: where they watched, which one(s) they watched, how much of the video they watched and rating their agreement that they were helpful for their understanding. Two open-response questions allowed them to write about what they found most helpful or what they thought needed improvement.
Participants. Two instructors -one asynchronous and one synchronous section -in the March term were recruited to post weekly announcements in the LMS with the links to videos and follow-up web surveys. Both expressed their interest in how the videos would help their students with difficulty concepts: "Thank you for these videos. Really helpful for summarizing" and "I think they're pretty good, especially for online courses." Students self-selected to participate in the surveys and response was quite low. There were 31 students between the two sections and we received only six responses throughout the term. Those responses all came from the synchronous section where the videos were shown during class time.

Phase 3 -Wide Video Intervention
For the remainder of the academic year, links to both videos and surveys went to all RSCH 202 instructors. There were a total of 24 sections taught by 16 instructors with 461 students enrolled. Only five sections (with unique instructors) and 108 enrolled students had regular announcements about the videos and surveys. Across those sections, only eight survey responses which appear to be from just two or three unique students. The continued lackluster engagement from students and the apparent lack of interest from instructors calls into question a number of our initial assumptions and conflicts with early information gathering.

Discussion and Conclusion
While self-directed learning has long been heralded as a critical element in enhancing and sustaining student motivation and engagement, mandated curriculum requirements, including standard learning outcomes, assessment measures and learning activities, limit the level of student autonomy for specific courses, especially at the macro level. Furthermore, courses that are considered more difficult, more work-intensive and less generally appealing (such as the RSCH 202 course) represent classes that students would not typically self-select, if given the choice. This presents difficulties in facilitating self-directed learning, or even self-regulated learning opportunities, that are reflected in the low response rate for this study. Without the full participation of students and instructors, we have no statistical evidence to draw conclusions about how the videos impact the perceptions regarding student understanding of scholarly research. Instructors added some anecdotal evidence that aligned with our literature review that the videos would be useful to student recall of pre-requisite material. However, most instructors during the study did not take the steps necessary for making the videos available to their students -the notable exception being the instructor who showed them during class. The challenge for further stages of this project will be to encourage students to take advantage of self-regulated learning opportunities embedded in the course that both appeal to students and address the major areas of difficulty.
The pilot study discussed here has entered a new phase from which we hope to be able to draw new conclusions. The supplemental videos are now embedded in the class, though they are still not required assignments. Additionally, the survey links have now been replaced with targeted questions on student end-of-course surveys. Ideally, these changes will result in additional use of these supplemental materials that will translate into fluency in the skills that will better ensure success in the course. While completely self-directed learning failed to produce the desired results in our pilot, we remain hopeful that the new self-regulated approach will alleviate some of the lingering concerns in RSCH 202, including the problematic performance in the essential skill areas and the general apathy that plagues the course.
Finally, the course itself is currently being reviewed as a whole. The course developer is actively considering other opportunities for self-regulated learning in the class. This study has prompted that thorough review and compelled the developer to ponder whether all current assignments are the best possible options for students to meet the course learning outcomes. RSCH 202 has the potential to be an exciting, dynamic course that introduces students to the fundamentals of scholarly research. Determining how to best use this opportunity to kindle an interest scholarship in will be an ongoing task, but it clearly begins with reinforcing the basic skills that serve as the underpinning for the course.