Back to Index

ork on “calibrating” the performance standards for use in New York City’s public schools began in October 1997 and continued through to the end of May 1998.

The work samples and commentaries form an essential element of the performance standards because they give concrete meaning to the words in the performance descriptions and show the level of performance expected by the standards. The principal goal of the calibration process was to supplement the collection of student work samples used to illustrate standard-setting performances in the New Standards™ Performance Standards with work produced by students in New York City’s public schools. To achieve this goal, all districts and high school superintendencies nominated representatives to join a “calibration” working group to collect work samples and meet regularly throughout the process to select the work to be included in this New York City edition.

Deciding what constitutes a standard-setting performance
The benchmarks against which these work samples were judged are the work samples that were selected for publication in the New Standards™ Performance Standards to illustrate standard-setting performances in relation to various parts of the standards. Those work samples were selected through a variety of strategies designed to tap the judgment of teachers and subject experts around the country about the “level of performance” at which the standards should be set at each of the grade levels: elementary, middle, and high.

We define the elementary school level as being the expectations for student performance at approximately the end of fourth grade; middle school level as the expectations at approximately the end of eighth grade; and high school level as the expectations at approximately the end of tenth grade. We used the concept of grade level as our reference point because it is in common use and most people understand it. However, “at approximately the end of fourth grade,” for example, begs some questions. Do we mean the level at which our fourth graders currently perform? Or, do we mean the level at which our fourth graders might perform if expectations for their performance were higher and the programs through which they learn were designed to help them meet those higher expectations? And, do we mean the level at which the highest-achieving fourth graders perform or the level at which most fourth graders should perform?

We set the expectations for level of performance in terms of what we should expect of students who work hard in a good program; that is, our expectations assume that students will have tried hard to achieve the standards and they will have studied in a program designed to help them to do so. These performance standards are founded on a firm belief that the great majority of students can achieve them, providing they work hard, they study a curriculum designed to help them achieve the standards that is taught by teachers who are prepared to teach it well, and they have access to the resources they need to succeed. These conditions form an essential part of the New Standards Social Compact which underpins our belief that all students can and should be expected to meet high standards.

Some of the work samples included in the New Standards™ Performance Standards were also
included in the Consultation Draft; some appeared in earlier drafts as well. The appropriateness of these work samples as illustrating standard-setting performances was the subject of extensive review, through discussions among the New Standards advisory committee for Mathematics and through round-table discussions among experienced teachers and experts in Mathematics. Some of the work samples included in earlier drafts did not pass the scrutiny of these reviews and were not included in the eventual publication. Many additional work samples were identified in the process of consultation and then subjected to the iterative process of review that was used to establish the level at which the standards should be set and the selection of work samples to be used to illustrate the meaning of the standards.

Selecting the work samples included in this New York City edition
The calibration group for the New York City edition of the performance standards followed a similar iterative process of review of collections of work samples to arrive at the selection that is included in this volume. Our goal was to identify candidate work samples for each part of the performance standards as the basis for selecting samples that would reflect the diversity of the communities that make up New York City and to demonstrate different approaches to producing standard-setting work, for example, student work that solves problems using the more familiar methods of formulas and procedures as well as more innovative methods such as developing original charts and diagrams. (See “Dart Board” for an example of innovative problem solving.)

Districts supported the process by encouraging schools to provide samples of student work for review through their representatives on the group. We organized ourselves according to our expertise and experience at each of the grade spans and divided responsibility across the various parts of the standards. In this way, sub-groups developed expertise in relation to specific parts of the standards through the experience of reviewing work samples with reference to the relevant performance descriptions and to the work samples and commentaries published in the New Standards™ Performance Standards.

When the calibration working group met, we discussed the characteristics of the work samples collected. In some cases, work that was judged as nearly meeting the expectations for standard-setting work was returned to the students who had produced it with an invitation for revision and suggestions about the aspects of the work that would benefit from revision. These students returned revised work for further review.

At each stage of the process, review of the work collected to date helped sharpen our focus on the characteristics we needed to look for in the work we collected. Among the by-products of this process was our growing appreciation of the significance of the tasks or assignments that generate student work in influencing the quality of the product. Put simply, the work students produce generally reflects the assignment they have been given and the instruction on which the assignment is based. We are resolved to make this direct connection between standards and instruction the focus of our continuing efforts to assist all students to meet the expectations illustrated in the work samples in this volume.

Throughout the process, we had to remind ourselves continually that work that illustrates standard-setting performances is not the same as “best” work or “most exceptional” work. Some of the work samples we reviewed exceeded the expectations of the standards. Those work samples do not appear in this collection. We also had to remind ourselves that we were not trying to put together an anthology to celebrate the work students produce, valuable as such anthologies can be. Rather, our purpose was to identify samples of work that would help to give concrete meaning to the qualities described in the performance descriptions and establish the level of performance we should expect of work that is “good enough” to meet the standards. This meant that we chose some work samples over others because they provided clearer exemplification of the “bullet points” in the performance descriptions, even though some of the work we passed over unquestionably counted as “good” work.

We also learned that practice in making judgments about work in relation to the standards pays off. As the number of pieces of student work we had read and reviewed closely grew larger, we became clearer about the meaning of the bullet points in the performance descriptions and more confident of our judgment about the features that need to be demonstrated in work if it is to be considered standard setting. Some pieces of work that we judged to be candidates for inclusion in the collection early in the process did not rate among our judgments later on. Equally, there were some pieces of work that we rejected early in the process and later brought back to the table for further consideration.

Work produced by a diverse range of students
The work samples in this book reflect the diversity of backgrounds and experiences of the students studying in New York City’s public schools and the communities of which they are a part. The student work illustrating standard-setting performances in Mathematics comes from schools throughout the city. The work comes from students with a wide range of cultural backgrounds, some of whom have a first language other than English or are studying in ESL or bilingual education programs.

In some cases, the diverse backgrounds and experiences of the students are evident in the work samples. In other cases, the students’ work reveals little about who they are. While we worked to ensure that the collection reflected the diversity of our students, we have not made specific reference to these characteristics in the commentaries that accompany the work samples. Work that illustrates a standard-setting performance is standard setting no matter who produced it. What unites the work samples is that they all help to illustrate the performance standards by demonstrating standard-setting performances for parts of one or more of the standards and demonstrate that all students can produce work that meets high expectations.

Genuine student work
In all cases, the work samples are genuine student work. While they illustrate standard-setting performances for parts of the mathematics standards, many samples are not “perfect” in every respect. Some, for example, include arithmetic errors. Others have some spelling errors or awkward grammatical constructions. We think it is important that the standards are illustrated by means of authentic work samples and accordingly have made no attempt to “doctor” the work in order to correct these imperfections: the work has been included “warts and all.” Where errors occur, we have included a note drawing attention to the nature of the mistakes and commenting on their significance in the context of the work.