In Search of Integration:
Mapping Math Instruction in
Pre-College Engineering Courses
When you hear Professor of Educational Psychology Mitchell Nathan talk about his group's research on pre-college engineering programs, one word in particular reverberates: integration.
Nathan is co-PI of the NSF-funded AWAKEN Project (Aligning educational experiences with WAys of Knowing ENgineering). As part of that work, he studies integration of mathematics and science content in Project Lead The Way (PLTW), a series of middle- and high-school pre-engineering courses. PLTW is used in 17% of American high schools and was praised as a "model curriculum" by the National Research Council in their call to arms Rising Above the Gathering Storm. Nathan reported on nearly two years' of work studying PLTW in a March 2 talk at the regular Learning Sciences Brown Bag, engaging in a bit of integration of his own as he weaved together a number of complementary strands of inquiry about how high school students learn engineering.
Part of the great potential for programs like PLTW, advocates note, is that they allow students to learn valuable math and science skills in context, via hands-on projects and other team-based engineering activities. While this model certainly stands to benefit recruiting, cognitive scientists who study the transfer of learning from one context to another caution that it can be difficult to realize.
"One thing we honed in on fairly early on is this idea of the degree to which concepts in the academic side of students' experience are explicitly integrated...with the engineering concepts," he said.
"When people are given parallel, independent information, learners don't necessarily make those connections for themselves." In other words, just because an instructor realizes that an engineering lesson is rooted in concepts from a student's math or physics class doesn't mean that the student shares that awareness.
Nathan studies this program in a research group that he leads jointly with co-PI and Professor of Educational Leadership and Policy Analysis Allen Phelps and that brings together graduate students from both the School of Education and the College of Engineering. The team examined PLTW curriculum according to a four-pronged scheme that included several quantitative and qualitative measures of the integration of mathematics.
Their approach was based on assessment theory by Andrew Porter, a UW-Madison trained educational psychologist who is now Dean of the University of Pennsylvania's Graduate School of Education. Porter writes about four types of curriculum: intended, enacted, assessed, and learned. He explains that to accurately measure the academic content of a course or courses, we need to study what students are supposed to be taught, what they're actually taught, what they're tested on, and what they take away from the course.
"Almost never are these four things going to be the same," Nathan said. Thus, the AWAKEN project's snapshot of PLTW includes each aspect of the curricula for several of the program's courses.
For their work on intended curricula, researchers combed through the PLTW's static course materials (lesson plans, suggested classroom activities, teaching training documents, etc.) and noted "any instance of a math principle, law, or formula that depicts how it is used to carry out or understand an engineering concept, task, or skill." They used standards published by the National Council of Teachers of Mathematics as a guide to identifying the mathematics material high school students are supposed to learn. Broken down into content standards (number and operation, algebra, geometry, etc.) and process standards (problem solving, reasoning and proof, communication, etc.) this document is a common touchstone in studies of intended curricula.
Nathan summarized the multi-layered process with the following question: "For each course, and then for each math standard, then for each element [lessons plans, training materials, etc.], and then with each subunit of the curriculum, how many times did we find places of explicit integration?"
The answer to that question, some integer X, was divided by the number of opportunities for such integration, N, to obtain an integration value. The group observed a wide variety of integration values among the three courses they studied, ranging from 2-13 percent for activities in the course Introduction to Engineering Design (IED), to 28-46 percent for activities in the course Principles of Engineering, and 50 percent or more in the course Digital Electronics. (The third course even included lessons in discrete mathematics, which are not part of the K-12 math standards.) Thus, it would seem that students who remain enrolled in a sequence of PLTW courses stand to benefit from increasing exposure to explicit mathematics concepts.
"This paints an interesting picture. There's definitely a progression over the years," Nathan said. "That may be somewhat problematic, because more people just take the first one or two [courses]."
Going on to study the enacted curriculum requires seeing how the intended paper curriculum actually influences classroom instruction. And that usually means video...lots of it.
"Whole programs of research and whole careers are devoted to exploring this question," Nathan said. Part of the problem is that there's so much variability in data about enacted curricula, from district to district, course to course--even from teacher to teacher and between sections taught by the same teacher. Nevertheless, studying enacted curricula is a valuable activity.
"Knowing the content of the enacted curriculum is important because what students are taught is a powerful predictor of student achievement on a test and helps explain a portion of the achievement gap between White, Black, and Hispanic students," Porter writes.
Nathan's group collected video data from two sections of IED taught by a single instructor. As part of their analytical process, they broke that data down into coherent clips. Each clip was assigned one or more codes that described important characteristics of the classroom moment it captured, including how class time was being apportioned, whether the instruction was conceptual or procedural, and--of course--whether any underlying math content was explicitly integrated.
Although the volume of video the group analyzed represents a much smaller proportion of the total enacted curriculum than in the analysis of the intended curriculum via course materials and published standards, the observations in this strand of the group's work agreed well with the pattern already identified. In the 34 distinct clips coded for math concepts, less than a third integrated those concepts explicitly. In the remaining cases, students were left to their own devices to connect the math concept to the engineering.
The most common missed opportunities involved the CAD tool students used to complete many of the course projects in IED. Various geometric transformations performed automatically by this software tended to go unidentified as mathematics as such and were usually explained instead as features of the modeling environment. (Followers of the history of CAD-criticism in engineering education may recognize in this example an early instance of the kind of uncritical black-box thinking that often gets blamed on the early introduction of analysis software in the curriculum. Civil engineer Henry Petroski devoted an entire chapter to this issue in his 1982 classic To Engineer Is Human.)
What does all this work studying PLTW have to teach college engineering faculty? Well, for starters, it's worth remembering that challenges to the transfer of learning don't just apply at the secondary level. Indeed, college instructors who long ago internalized the way general mathematical principles and formulas apply to the physical systems of their areas of expertise are vulnerable to losing touch with the many initial difficulties involved in forging those connections. This phenomenon is known in the education literature as the "expert blind spot."
Nathan's ultimate advice to PLTW instructors applies at the college level as well: Think carefully about how to design activities that form a "conceptual chain of inference" to help students explicitly connect the mathematical formalisms to "the more applied, hands-on aspects of the engineering work."
More generally, though, the highly integrated and theory-based curriculum assessment techniques deployed in the AWAKEN project can serve as models for all engineering educators. Even if individual course assessment at the college level cannot always be so systematic and thorough as in a specially funded project like AWAKEN, the benefits of coordinating different measurements of the different types of curricula stand to improve any formal or informal assessment project.
Editor's Notes: (1) Papers about both of the strands of Nathan's group's work described in this article have been accepted for the 2009 American Society for Engineering Education National Conference and Exposition in Austin, TX. These papers include as co-authors current and past UW-Madison graduate students Amy Prevost, Natalie Tran, Benjamin Stein, and Kyle Oliver and will be linked to below when they become available for online viewing via ASEE.org.
(2) Also referred to as "How People Learn Engineering," the AWAKEN project is helping fund TLI during spring 2009. Check back next month for coverage of the professional practice component of this work, performed by researchers from the COE's Engineering Professional Development Department and led by co-PI Sandra Courter.
For more information about the ideas in this article:
Website of the popular pre-college engineering program Project Lead The Way. Includes an explanation of its curriculum philosophy and a list of resources for participating schools and districts.
Introductory commentary by the National Council of Teachers of Mathematics. Includes links to the full listing for each content and process standard.
American Educational Research Association publication that includes Andrew Porter's work on curriculum assessment. Porter suggests an integrated approach that accounts for the inevitable differences between intended, enacted, assessed, and learned curricula.
Website of the NSF-funded research project of which Nathan's group is a part. Includes project overview, information about the collaborators, and links to related publications.