|
Longitudinal Evaluation of the Lockheed Martin/UCF Academy
In February 1999, the National Science Foundation awarded $934,081 for a four-year grant to evaluate the effectiveness and impact of the LMA. The grant affords investigators and NSF the time for a follow-up study of this program to examine the long-term effects of the original project primary funding ends. Longitudinal studies of academic programs are unusual because often, investigators have little interest in, or time for follow-up studies of their programs after the primary funding ends. Academic faculty have multiple demands for their time that seldom include looking back at previous work. Rather, the pressures in academe for continued external funding lead investigators to seek additional funds for a new project that capitalizes on what they have learned in the relatively short term of the original project. Thus, the long-term effects of the original projects remain uninvestigated and unknown.
The LMA longitudinal evaluation design calls for four waves of data collection and analysis with each wave corresponding approximately to one school year. The design looks somewhat like a funnel with a broad top and narrower ending. Wave one of the evaluation consisted of surveying 208 LMA graduates who have remained in the central Florida area with respect to their perceptions of changes encountered as a result of their experience in the LMA. Three focus groups were also held with LMA graduates who were classroom teachers, and one focus group was held consisting of graduates who held leadership positions. Furthermore, 55 graduates were observed in their schools for approximately one hour while they taught either a math or a science lesson.
During wave two, classroom observations continued, but they were more in-depth. Twenty-eight teachers were observed on a minimum of two different occasions. Likewise, 66 principals were surveyed with regarding their perceptions of LMA graduates.
The third wave (year) of data collection was completed in the 2001-2002 school year. Mini case-studies were conducted in the classrooms of 14 graduates of the LMA. Wave three observations were longer in duration, with each session lasting approximately three hours. Moreover, each of the 14 graduates were observed four or five times during a two-week period. At the time of this writing, wave 3 data was in the process of being analyzed, and thus much of this article sites data from waves one and two.
Classroom Environment
Cries for systemic reform in educational literature are loud and clear (Fullan, 1999; Eisner, 1999; Fullan & Heargreaves, 1996). Systemic reform must happen at multiple levels, that is the bottom and top must meet. The LMA supports reform on the classroom level (micro), the school level (meso), and at the university level (macro). Results of the LMA Longitudinal study illustrate that the LMA is helping to reform mathematics and science teaching in central Florida on multiple levels.
What may be the most important finding during the observational data collection is the exceptional class climates that existed in the vast majority of the classroom that were observed. The brain learns best when it is in a risk free climate. According to Caine and Caine (1991), the brain will downshift, that is it will revert to the lower functioning parts of the brain, if threatened. Situational downshifting is brought on in classrooms that have prespecified correct outcomes, personal meaning is limited and rewards and punishments are extrinsic. In contrast, optimal learning takes place when the student is in a state of relaxed alertness. This means that students and teachers need to be generally relaxed and working in an environment where they feel free enough to take risks.
The majority of the classroom observations took place in class climates that can easily be described as in a state of relaxed alertness. Indeed, 72 percent of the observations conducted in waves one and two were rated highly as a 4 or 5 on a 5 point scale for class climate. Overall, the teachers observed were very approachable to the students. The graduates obviously enjoyed teaching and their pleasant demeanor helped set the emotional tone for the class.
Collaboration is one important aspect of positive classroom cultures. Bruner (1986) and Vygotsky (1978) both conceive of learning as a social endeavor. Likewise, the social perspective of teaching and learning was commonly observed during this study. The majority of the lesson during waves one and two (70%) found the students working collaboratively. Students were active in their learning, talking with each other to make sense out of the learning activity in which they were engaged. Although student collaboration was very common, not all learning activities were designed for collaboration. Often teachers seat their students together in groups and call it collaborative learning. However, simply working together in groups does not ensure collaboration. Most students have learned to work alone on individualistic tasks. Upon close observation what is sometimes called collaborative learning is simply students sitting next to each other, but working on tasks individually. Still and all, the foundation for true collaborative learning is making meaning in a social group. Students are encouraged to talk about what they are learning, what they don't understand, how they are learning, and to help each other. Collaborative learning draws on students' natural inclination to talk with their peers. Collaborative learning channels student talk to positive academic dialogue. The data from this study indicate that while students frequently worked in groups on independent tasks, they still commonly talked about what they were doing and assisted each other. Although structured cooperative learning activities where each student is assigned a group role was not often observed, student collaboration was indeed common.
Goodlad's (1984) findings in A Place Called School are based on one of the most comprehensive studies of schooling completed. Goodlad paints a very unflattering picture of what happens inside American classrooms. He found that that the majority of classrooms were emotionally flat, neither positive nor negative places to be. The typical teacher stood in front of the class and droned on as students sat passively in their seats. There was little interaction between student and teacher. Goodlad found classroom environments to be marked by boredom. In contrast to findings of Goodland, this study reveals that the majority of LMA classrooms are relaxed and comfortable places to be. The teachers and the students in this study both seemed to enjoy being together in the classroom. Rarely did the observers find a teacher who stood in front of the class and dispensed knowledge as the students sat passively in their seats. Rather, 77 percent of the observations reveal that a collaborative relationship between the student and teacher was evident. The majority of the teachers were very approachable and exhibited excellent rapport with their students. The vast majority of the LMA teachers observed smiled and laughed with their students. They showed particular interest in students as people. Goodlad, on the other hand, found that most of the teacher student relationships observed for his study were not collaborative, neither were they abrasive nor joyous. He described the teacher-student relationship as characterized by "affective neutrality" (p.111). Goodlad also found that students' perceptions about whether or not their teacher was concerned about them was significantly related to the students' satisfaction with the class.
Likewise, respect was an overarching pattern found in the LMA teachers' classrooms. There was a high degree of respect for student ideas, questions and contributions noted by the observers (73 percent). The students were treated with respect, and discipline problems were handled accordingly. Indeed, it was rare for the observers to find a structured discipline plan in place. Instead, what the observers usually found was a community of learners where each individual was valued. Interaction in the classrooms was based on mutual respect. When there was a problem, the teacher typically handled it with care. For example, during one observation in an 8th grade science classroom, a student used a screwdriver to begin carving his name in the expensive black topped science table. The teacher moved the child into an office she shared with several other teachers. She handled the situation so subtly that most of the students were not even aware a student was sent from the room. After class was over, an aide escorted to the young man to the office. An administrator called the teacher on the phone to discuss the offense during her next class. While the teacher gave her attention to the phone conversation, the students in her room worked on their own, not taking advantage of the teacher being tied up on the phone to play around or act out in any way. This example illustrates two important findings from the class observations:
- Teachers handled behavior problems in a respectful and unobtrusive way, and
- Students understand the teachers' expectations and act accordingly without the teacher's direct supervision.
Lesson Design
The classroom climate and a standards-based curriculum appear to work together to produce classrooms that are full of eager, hard-working students. As mentioned earlier, the LMA is a standards-based master's degree. The foundation of the pedagogy taught in the LMA is the National Council of Teachers of Mathematics Principals and Standards (2000) and the National Science Teaching Standards (1996). During the course work at the University of Central Florida, the LMA teachers explore the standards and teach model-lessons based on the standards. Not surprisingly this focus is reflected in their classrooms, 100% percent of the observations in wave one were highly aligned with the standards, and 86% of wave two observations highly aligned with the standards.
During the three years of observational data collection teachers in the state Florida were under a great amount of stress associated with the high stakes assessment, Florida Comprehensive Assessment Test (FCAT), (Schmuddel, 2001). However, science was not a subject area that was assessed on FCAT, and thus many schools did not emphasize science instruction. Surprisingly, the observational data reveals that almost 40 percent of the elementary observations were in the area of science. Interestingly, 35 percent of the observed science lessons integrated mathematics to a high degreentegrating mathematics with science, the LMA graduates connected science and mathematics to real life experiences. According to Caine and Caine (1991) the brain learns easiest when learning is connected. They use the analogy of a map to illustrate this point. When a student learns a concept in rich, connected activities, then they have a neural map, so to speak. For example, when marbles are dropped onto a tray of flour to model what happens when meteors make craters, the learner has multiple connections. Most likely, the learner will have had previous experiences with both marbles and flour. Plus, if the learning activity is done collaboratively, then the relationship with other students and the dialogue that took place during the learning activity also supplies multiple paths to retrieve the information. Much like a map offers a traveler a variety of paths to travel if he/she can't find the way, experiential learning provides the learner multiple paths to travel to retrieve information. Overarching patterns from the focus groups and classroom observations illustrate that the students of LMA graduates commonly interact with a wide variety of resources to learn mathematics and science. This is in contrast with Weiss' findings published in 1994. Weiss surveyed 6,000 teachers with regards to their math and science teaching. The results indicated that about half of the elementary teachers and two thirds of intermediate teachers heavily emphasized learning facts and terms. The evaluation of LMA indicated that the most frequent purpose of the lessons observed in wave one (74%) and wave two (71%) was to develop conceptual understanding.
Leadership
Besides helping create a standards-based mathematics and science curriculum in the classrooms of the LMA graduates, one of the main goals of the LMA is to create a network of teacher leaders. Teacher leaders act as systemic reform agents. Data from the interviews and focus groups detail the many forms of leadership performed by the graduates of the LMA. In addition to the 17 percent of graduates that have been promoted to leadership positions, many more graduates take on informal leadership roles by exerting influence from inside their classrooms. This perception of leadership is well aligned with Maxwell (1998) who defines leadership as the ability to influence others. Providing professional development for other educators and serving on committees were the most commonly sited leadership activities (71% of respondents). In addition, results of the survey reveal that 99% of the respondents were professionally active outside of the classroom arena, for example presenting at professional conferences (31%) and authoring published articles (15%). One of the graduates was featured in a Scienceline PBS program on the inquiry teaching strategy. During this production, the graduate was interviewed and modeled teaching in his classroom. Furthermore, LMA was featured as crucial to his growth in inquiry teaching. Likewise, 20 LMA graduates have achieved the recognition of being Nationally Board Certified. This represents 6.04% of the LMA graduates, which compares to the state percentage of Nationally Board Certified teachers of 1.5%. Moreover, and not surprising, 91% of responding principals indicated that they would like more LMA graduates in their schools. Last, but certainly not least, the focus group data revealed that the graduates of the LMA are an empowered group of teachers with a strong voice. Both of these characteristics are applicable to leadership skills.
Characteristics of the Program
The evaluation that is being conducted on the LMA illustrates that it is meeting the dual goals of positively impacting public school classrooms in central Florida, while at the same time increasing the leadership capacity in its graduates. These findings beg the question: What makes the LMA effective? Focus group participants overwhelming cited cohort grouping as a crucial characteristic of the LMA.
Teachers progress through the LMA in cohorts of about 25 students. LMA teachers enroll in all of the master's classes together. The LMA professors give time and care to the creation of a community of learners within the cohort group. Many collaborative projects are assigned in the LMA, cooperative learning strategies are used extensively, and community-building activities are common. The work of Caine and Caine (1991) is drawn on by the LMA and a risk free environment is intentionally created to give the LMA teachers a safe place to ponder new ideas and try new pedagogies. The LMA adheres to the philosophy that learning is a social activity, and accordingly there is ample time for student conversation focused on mathematics and science teaching and learning. Feedback from the LMA classes of 2002 and 2003 both illustrate that class conversation plays an important part in learning. Teachers in these classes felt they learned a great deal from each other through participation in professional dialogue during the LMA classes. All three focus groups of teachers and the one focus group of graduates in formal leadership positions clearly portrayed the cohort group as the most powerful aspect of the program.
In addition, the professors of the LMA continuously model what they teach, thus pedagogy that is used in the LMA is reflected in the positive classroom environments and teaching practice found during the observations. While attending classes in the LMA, teachers encounter a collaborative community where they are encouraged to grow and take risks. Consequently, the majority of the graduates' classrooms observed in this study reflect the same values.
Daniels (2001) found that the LMA teachers' self-reported change hinged on the process of aligning beliefs with practice. Reflection is a critical component of the LMA. Teachers in the LMA keep dialogue journals with their professors. The goal of the dialogue journal is for the teachers to think deeply about what they are learning. In addition, the teachers are immersed in critical reading that encourages the teachers to begin to take a critical look at what they do in the classroom and why they do it Journaling offers the teachers a place in which to question their values and beliefs, the books they read for their classes, the classroom activities, and their practice. While analyzing the student journals, Daniels found that there was a disconnection between what the teacher said they believed about teaching and learning, and what they said they did in their classroom practice. She found the reflective aspect of the journaling process let the teachers acknowledge the divide and empowered them to begin to close it. In the journals the teachers examined their assumptions about teaching and learning and uncovered ways in which their personal biographies influenced their pedagogy. Thus, aligning beliefs about teaching and learning with classroom practice was crucial to teacher change in the LMA.
Summary
The longitudinal study of the LMA illustrates that the program is meeting its goals of systemic reform. On the micro level, classroom teachers are using standards-based teaching in the context of a respectful and collaborative environment to reform science and mathematics teaching. On the meso level, the LMA has helped many of its teachers become skillful leaders and thus influence others at the school and district level. The data clearly shows that a variety of teacher leaders have emerged from the LMA. Some of the leaders are in prominent and traditional leadership roles such a resource teachers and principals. Yet many of the LMA graduates lead quietly from their classrooms helping other teachers become better mathematics and science teachers. The University of Central Florida operates on the macro level working with local school systems and a private corporation (Lockheed-Martin) to reform education. Thus, the Lockheed Martin/UCF Academy for Mathematics and Science is an ongoing program that intentionally helps support systemic reform.
|
