Work on “calibrating” the performance standards for use in New York City’s public schools began in February 1998 and continued through to the end of May 1999.

The work samples and commentaries form an essential element of the performance standards because they give concrete meaning to the words in the performance descriptions and show the level of performance expected by the standards. While the principal goal of the calibration process was to supplement the collection of student work samples used to illustrate standard-setting performances in the New Standards™ Performance Standards with work produced by students in New York City’s public schools, a group of New York City educators met with staff of the National Center on Education and the Economy and devised a plan to make the New York City Science Standards more valuable to educators in New York City by showing the relationships among science content standards and by showing conceptual development over the grade spans. To achieve these goals, a group of approximately sixty educators gathered for five days in the Spring of 1998 to serve as a Conceptual Planning Task Force.

The framework for the Science performance standards reflects New Standards partner representatives’ distillation of these several sources of guidance: Physical Sciences Concepts; Life Sciences Concepts; Earth and Space Sciences Concepts; Scientific Connections and Applications; Scientific Thinking; Scientific Tools and Technologies; Scientific Communication; Scientific Investigation.

Showing the relationships among science content standards
There are two widely used and respected national documents in science which provided the foundation for the work of New Standards: the National Research Council (NRC) National Science Education Standards (1996) and the American Association for the Advancement of Science (AAAS) Project 2061 Benchmarks for Science Literacy (1993). The AAAS analysis of the Benchmarks and the NRC Draft was helpful in seeing the substantial degree of agreement between the two documents. New Standards partner statements about standards and international documents, including the work of the Third International Mathematics and Science Study and the Organisation for Economic Co-operation and Development, were also used. Many of these sources, like the Benchmarks, give greater emphasis to technology and the applications of science than does the NRC.

As the amount of scientific knowledge explodes, the need for students to have deep understanding of fundamental concepts and ideas upon which to build increases; as technology makes information readily available, the need to memorize vocabulary and formulas decreases. There is general agreement among the science education community, in principle, that studying fewer things more deeply is the direction we would like to go. The choices about what to leave out and what to keep are hotly debated. There are 855 benchmarks and the content standards section of the NRC standards runs nearly 200 pages, so there are still choices to be made in crafting a reasonable set of performance standards.

The New Standards Science Standards carried the statement, “The Science standards are founded upon both the National Research Council’s National Science Education Standards and the American Association for the Advancement of Science’s Project 2061 Benchmarks for Science Literacy. These documents, each of which runs several hundred pages, contain detail that amplifies the meaning of the terms used in the performance descriptions.” The New York City Conceptual Planners said that the document would be much more useful if it showed, explicitly, the statements from these two documents and from New York State’s Learning Standards for Mathematics, Science, and Technology.

There were three reasons for making the correlations evident:

1. The New York City Science Standards should be self-sufficient; additional standards documents should not be necessary to understand what New York City students are expected to know and be able to do.
   
2. There is a tremendous degree of agreement among the content statements; teachers and others should be reassured that there are not divergent instructional demands.
   
3. In a small number of cases, the New York State Learning Standards are slightly different from the New York City Standards; since the State’s tests will be based on the State’s standards, these differences should be noted.
   

Thus, the set of related standards are presented so that educators can ascertain the extent of agreement for themselves.

Showing conceptual development over the grade spans
When the goal is deep understanding, it is necessary to revisit concepts over time. Students show progressively deeper understanding as they use the concept in a range of familiar situations to explain observations and make predictions, then use the concept in unfamiliar situations; as they represent the concept in multiple ways (through words, diagrams, graphs, or charts), and explain the concept to another person. The conceptual understanding standards make explicit that students should be able to demonstrate understanding of a scientific concept “by using a concept accurately to explain observations and make predictions and by representing the concept in multiple ways (through words, diagrams, graphs, or charts, as appropriate).” Both aspects of understanding—explaining and representing—are required to meet these standards.

For most people and most concepts, there is a progression from phenomenological to empirical to theoretical, or from a qualitative to a quantitative understanding. New Standards illustrated the progression using one important concept, density, and these student work samples: “Flinkers” at the elementary school level, “Discovering Density” at the middle school level, and “Density of Sand” at the high school level. The New York City Conceptual Planners said that it would be worthwhile to illustrate conceptual development for additional concepts. For each of the major areas of science, they selected five topics or concepts where they would focus the collection of student work. They also drafted tasks that teachers could use to elicit student work so that they could illustrate the variety of curricula that are used in New York City. In addition to density, the Physical Sciences concepts are acids and bases, heat and temperature, energy, force and motion, and chemical reactions. The Life Sciences concepts are interdependence, structure and function, change over time, responding to stimuli, and reproduction and heredity. The Earth and Space Sciences concepts are surface features, weather, rocks and soils, water cycle, and space.

While it was a challenge to find standards-setting work for every concept in the list at every grade level, it was possible to find eight sets of examples that illustrate conceptual development over time—from simple to complex, from descriptive to analytical, from familiar to unfamiliar. Eight such “storylines” are described below. Each storyline demonstrates a progressive level of understanding of a particular concept. As you review the work samples in each storyline, you'll note that some tasks show particular success at moving students into deeper levels of understanding.

Work produced by students in New York City’s public schools
The Conceptual Planners thus provided a framework for the work of the Calibration Task Force. All districts and high school superintendencies nominated representatives to complete the correlations and to collect work samples and meet regularly throughout the process to select the work to be included in this New York City edition.

Deciding what constitutes a standard-setting performance
The benchmarks against which these work samples were judged are the work samples that were selected for publication in the New Standards™ Performance Standards to illustrate standard-setting performances in relation to various parts of the standards. Those work samples were selected through a variety of strategies designed to tap the judgment of teachers and subject experts around the country about the “level of performance” at which the standards should be set at each of the grade levels: elementary, middle, and high school.

We define the elementary school level as being the expectations for student performance at approximately the end of fourth grade; middle school level as the expectations at approximately the end of eighth grade; and high school level as the expectations at approximately the end of tenth grade. We used the concept of grade level as our reference point because it is in common use and most people understand it. However, “at approximately the end of fourth grade,” for example, begs some questions. Do we mean the level at which our fourth graders currently perform? Or, do we mean the level at which our fourth graders might perform if expectations for their performance were higher and the programs through which they learn were designed to help them meet those higher expectations? And, do we mean the level at which the highest-achieving fourth graders perform or the level at which most fourth graders should perform?

We set the expectations for level of performance in terms of what we should expect of students who work hard in a good program; that is, our expectations assume that students will have tried hard to achieve the standards and they will have studied in a program designed to help them to do so. These performance standards are founded on a firm belief that the great majority of students can achieve them, providing they work hard, they study a curriculum designed to help them achieve the standards that is taught by teachers who are prepared to teach it well, and they have access to the resources they need to succeed. These conditions form an essential part of the New Standards Social Compact which underpins our belief that all students can and should be expected to meet high standards.

Some of the work samples included in the New Standards™ Performance Standards were also included in the Consultation Draft; some appeared in earlier drafts as well. The appropriateness of these work samples as illustrating standard-setting performances was the subject of extensive review, through discussions among the New Standards advisory committee for Science and through round-table discussions among experienced teachers and experts in Science. Some of the work samples included in earlier drafts did not pass the scrutiny of these reviews and were not included in the eventual publication. Many additional work samples were identified in the process of consultation and then subjected to the iterative process of review that was used to establish the level at which the standards should be set and the selection of work samples to be used to illustrate the meaning of the standards.

Selecting the work samples included in this New York City edition
The calibration group for the New York City edition of the performance standards followed a similar iterative process of review of collections of work samples to arrive at the selection that is included in this volume. Our goal was to identify candidate work samples for each part of the performance standards as the basis for selecting samples that would reflect the diversity of the communities that make up New York City and to demonstrate different approaches to producing standard-setting work, for example, student work that demonstrates conceptual understanding using the familiar format of expository writing as well as more innovative methods such as creating murals, models, and concept maps.

Districts supported the process by encouraging schools to provide samples of student work for review through their representatives on the group. We organized ourselves according to our expertise and experience at each of the grade spans and in each of the major areas of science and divided responsibility across the various parts of the standards. In this way, sub-groups developed expertise in relation to specific parts of the standards through the experience of reviewing work samples with reference to the relevant performance descriptions and to the work samples and commentaries published in the New Standards™ Performance Standards.

When the calibration working group met, we discussed the characteristics of the work samples collected. In some cases, work that was judged as nearly meeting the expectations for standard-setting work was returned to the students who had produced it with an invitation for revision and suggestions about the aspects of the work that would benefit from revision. These students returned revised work for further review.

At each stage of the process, review of the work collected to date helped sharpen our focus on the characteristics we needed to look for in the work we collected. Among the by-products of this process was our growing appreciation of the significance of the tasks or assignments that generate student work in influencing the quality of the product. Put simply, the work students produce generally reflects the assignment they have been given and the instruction on which the assignment is based. We are resolved to make this direct connection between standards and instruction the focus of our continuing efforts to assist all students to meet the expectations illustrated in the work samples in this volume.

Throughout the process, we had to remind ourselves continually that work that illustrates standard-setting performances is not the same as “best” work or “most exceptional” work. Some of the work samples we reviewed exceeded the expectations of the standards. Those work samples do not appear in this collection. We also had to remind ourselves that we were not trying to put together an anthology to celebrate the work students produce, valuable as such anthologies can be. Rather, our purpose was to identify samples of work that would help to give concrete meaning to the qualities described in the performance descriptions and establish the level of performance we should expect of work that is “good enough” to meet the standards. This meant that we chose some work samples over others because they provided clearer exemplification of the “bullet points” in the performance descriptions, even though some of the work we passed over unquestionably counted as “good” work.

We also learned that practice in making judgments about work in relation to the standards pays off. As the number of pieces of student work we had read and reviewed closely grew larger, we became clearer about the meaning of the bullet points in the performance descriptions and more confident of our judgment about the features that need to be demonstrated in work if it is to be considered standard setting. Some pieces of work that we judged to be candidates for inclusion in the collection early in the process did not rate among our judgments later on. Equally, there were some pieces of work that we rejected early in the process and later brought back to the table for further consideration.

Work produced by a diverse range of students
The work samples in this book reflect the diversity of backgrounds and experiences of the students studying in New York City’s public schools and the communities of which they are a part. The student work illustrating standard-setting performances in Science comes from schools throughout the city. The work comes from students with a wide range of cultural backgrounds, some of whom have a first language other than English or are studying in ESL or bilingual education programs.

In some cases, the diverse backgrounds and experiences of the students are evident in the work samples. In other cases, the students’ work reveals little about who they are. While we worked to ensure that the collection reflected the diversity of our students, we have not made specific reference to these characteristics in the commentaries that accompany the work samples. Work that illustrates a standard-setting performance is standard setting no matter who produced it. What unites the work samples is that they all help to illustrate the performance standards by demonstrating standard-setting performances for parts of one or more of the standards and demonstrate that all students can produce work that meets high expectations.

Genuine student work
In all cases, the work samples are genuine student work. While they illustrate standard-setting performances for parts of the science standards, many samples are not “perfect” in every respect. Some, for example, include imprecise language or graphic representations. Others have some spelling errors or awkward grammatical constructions. We think it is important that the standards are illustrated by means of authentic work samples and accordingly have made no attempt to “doctor” the work in order to correct these imperfections: the work has been included “warts and all.” Where errors occur, we have included a note drawing attention to the nature of the mistakes and commenting on their significance in the context of the work.

Resources
Reviewers of the New Standards edition have pointed out that our expectations are more demanding, both in terms of student time and access to resources, than they consider reasonable for all students. We acknowledge the distance between our goals and the status quo, and the fact that there is a tremendous disparity in opportunities between the most and least advantaged students. Indeed, the National Research Council included a program standard that delineates all of the resources–professional development, time, materials, adequate and safe space, and access to the world beyond the classroom–needed to achieve the National Science Education Standards. This program standard is reprinted in its entirety in the appendix.

In addition to taking advantage of existing school, district, and board resources, we think that there are two additional strategies that must be pursued to achieve our goals—making better use of existing, out-of-school resources and making explicit the connection between particular resources and particular standards.

Best practice in science has always included extensive inquiry and investigation, but it is frequently given less emphasis in the face of competing demands for student time and teacher resources. An elementary teacher faced with the unfamiliar territory of project work in science or a secondary teacher faced with the prospect of guiding 180 projects and investigations can legitimately throw up his or her hands and say, “Help!” There are many science-related organizations in New York City, such as the American Museum of Natural History, the Bronx Zoo, and the City Parks Foundation, that have science education on their agenda. Thus, we invited representatives of those organizations to participate in the calibration process and have incorporated examples of projects and investigations that are done outside of school to make clear that help is available. There is an extensive list of science-rich organizations, professional organizations for science teaching, and other resources to support teachers and students in the appendix.

All of the district, state, and national documents in science make explicit the need for hands-on experiences and using information tools. Thus, for example, Standard , Scientific Tools and Technologies, makes consistent reference to using telecommunications to acquire and share information. We know that more students have access to the Internet at home than at school, so this raises an equity issue. We feel that the best way to encourage schools to make sure that students’ access to information and ideas does not depend on what they get at home is to show several examples of work that was enhanced by use of the Internet and other technologies.