FAQ

Teacher Data Reports will be available the second week of December.

Principals will obtain access to their reports after they have received training from their Network Leader and SAF. At the training, Network Leaders & SAFs will distribute login information to a password-protected website where principals can access their schools’ reports: http://www.nycteacherdatainitiative.com.

Principals can go directly to this website or access it through the Teacher Data Toolkit section of the Principal’s Portal (under the Get Your Reports menu):
http://intranet.nycboe.net/DOEPortal/Principals/HiringAndStaffDev/StaffDevelopment/TeacherDataToolkit.htm

Principals will distribute Teacher Data Reports to their teachers after principals have been trained. Principals can choose to share the reports with administrators in their schools so that they can determine together how to best use the reports to improve instruction at the school. A teacher’s report should not be disseminated to other teachers without the teacher’s permission.

Reports were generated for ELA & Math teachers who taught 4th-8th grade students in 2007-08, unless:

1. There were issues with a teacher’s course assignment data
2. The teacher did not have at least 6 students who took the state ELA or math tests in two consecutive years and were in that teacher’s course from Oct 31 – June 30
3. The teacher did not teach for the full year (Oct 31 – June 30)

Principals will receive the reports for all eligible teachers in their school during 2007-08 as well as reports for eligible teachers who are currently teaching in the principal’s school but taught in another DOE school in 2007-08.

No. Reports are only available to school administrators and will be shared with eligible teachers individually.

Yes, principals are free to share the Teacher Data Reports with assistant principals and other administrators. Principals are expected to arrange individual conversations with each teacher about his/her report, and in some cases those conversation should be led by or include an assistant principal.

Principals may ask teachers they are interviewing if they are willing to share their reports. It is up to the teacher to decide whether to share his/her report.

Principals have been and will continue to be explicitly instructed not to use Teacher Data Reports to evaluate their teachers. The DOE has standard processes in schools for teachers to raise issues or concerns.

No. Several large cities, and even some states, currently use Value-Added models to generate teacher data. States providing Value-Added teacher data in more than one district include: Ohio, North Carolina, Tennessee, Texas, and Florida. Cities providing Value-Added teacher data include: Houston, Dallas, Milwaukee, Charlotte/ Mecklenburg. Chicago and Washington, D.C. are currently developing Value-Added teacher data systems.

Principals in the pilot conducted by the DOE in 2007-08 overwhelmingly reported that the information in the Teacher Data Reports is useful for a variety of instructional improvement purposes like planning individual and group professional development. About 100 volunteer schools received Teacher Data Reports in the pilot phase. At the end of the year, 86% of pilot principals said they felt the information is useful to principals, and more than 75% of principals thought it is useful for teachers.

The Teacher Data initiative is funded by a grant from the Carnegie Foundation. The 2008-09 academic year is the last year of this 3-year grant.

Progress Reports and Teacher Data Reports have different objectives. Like the Teacher Data Reports, School Progress Reports were designed to help principals and teachers accelerate academic achievement for their students. However, School Progress Reports were also created to enable students, parents, and the public to hold the DOE and its schools accountable for student achievement and to determine eligibility for school rewards and consequences. In contrast, Teacher Data Reports are not to be used for evaluation purposes. Principals are not to use these reports in tenure determinations or in the annual rating process. Teacher Data Reports are intended to help teachers and principals pinpoint teachers’ strengths and weaknesses and devise strategies for improvement.

Further, while both School Progress Reports and Teacher Data Reports contain data about student gains on state tests, School Progress Reports provide information about how schools as a whole are doing while Teacher Data Reports provide information about how individual teachers’ efforts influence student learning.

Teacher Data Reports—along with periodic assessments, student in-class work and homework, classroom observations, ARIS data and knowledge management tools, and other school-generated data—were created to help school communities make decisions about where to focus instructional improvement efforts.

The DOE is providing Teacher Data Reports to help principals and teachers better understand how teacher efforts are influencing student progress. The reports can help answer questions about how teachers are influencing the performance of different groups of students (top third, bottom third, ELL, special education, male, female) in their classes over time.

Far too many City students perform below standards. Research has shown that teachers have substantial influence, both positive and negative, on students’ achievement. Teachers should be able to identify the effectiveness of their instruction and adjust it to meet their students’ needs. Principals should be able to identify school-wide instructional needs, select professional development opportunities, and facilitate the sharing of best practices that address these needs. By isolating individual teachers’ contributions to student progress, the Teacher Data Reports provide valuable information to school leaders and teachers about where to focus instructional improvement efforts.

The DOE is providing Teacher Data Reports as a tool for schools and teachers to use for instructional improvement.They are not be used to evaluate teachers.

The following factors were used to calculate each student’s predicted gain. A sophisticated multivariate regression analysis determined how much to weigh each factor:

For each section of the report, data is only presented if teachers had at least 6 eligible students in each category. For example, a teacher with 20 eligible students, 4 of whom are Special Education students, will receive data on her impact on all 20 students’ progress but will not receive information about her impact on the progress of her Special Education students.

A school summary report aggregates and summarizes data found on the reports for individual teachers in a principal’s school. Teacher-level reports provide greater detail about each teacher’s results in particular years and with particular subgroups of students.

The Teacher Data Reports are intended to give principals and teachers another lens through which to view teacher performance. Principals whose previous views are challenged by the results presented in the Teacher Data Reports should reflect on what factors in either his/her perspective or in the teacher’s performance caused this difference. By asking and answering more questions, principals and teachers can determine if any new strategies could be helpful to improve the effectiveness of an individual or group of teachers.

The “peer comparison” sections of the report are different from the “citywide comparison” sections in two ways. First, the predicted gain in all peer comparison calculations takes into account the teacher’s experience overall and in that grade and subject. Second, teachers are divided into 5 groups based on their average predicted gain, meaning they have similar student, classroom, and school characteristics. For peer comparisons, a teacher’s result is compared to the results only from classrooms in their group. There are several hundred teachers in each peer comparison group.

Statistically, a teacher’s contribution to students’ test score gains most likely lies near, but may not be exactly equal to, the percentile result on the report. All statistical measures contain some uncertainty, reflected by a range (also known as a confidence interval) around the percentile. The range provided in the Teacher Data Reports means you can be 95% certain that a teacher’s contribution falls in that range, most likely closer to the highlighted result than at the ends of the range.

Larger ranges indicate that there is more uncertainty about a teacher’s value-added percentile. The size of the range differs for each calculation based on a number of factors, including the number of students included in the calculation. Ranges will generally be larger when a teacher has taught for fewer years or had fewer eligible students.

 “Value-added” is a statistical term which, in the context of the Teacher Data Initiative, refers to how much an individual teacher contributes to his/her students’ gains on the state ELA and Math tests.

Teacher Data Reports calculate teachers’ “value-added” by controlling for more than 35 factors that can influence student achievement but that are outside of a teacher’s control, including students’ prior test scores, class size, and the percentage of students in the school with disabilities and living in poverty. These factors are used in a statistical model to predict each student’s test score gains in math and ELA from one year to the next. The average predicted gains for all of a teacher’s students are then compared to the students’ actual gains on the tests. The difference between the average predicted gain and average actual gain is the “value-added” result for the teacher.

For each section of a teacher’s report, the “percentile” shows the percentage of teachers who had value-added scores lower than the teacher’s.

The predicted gain, actual gain, and value-added data are in proficiency ratings, on a continuum from 1.00 to 4.50. A proficiency rating of 1.00 corresponds to the lowest score a student in Performance Level 1 can attain on the state ELA and Mathematics tests. A proficiency rating of 1.99 correspondents to the highest score a student can attain and still be at Performance Level 1. A proficiency rating of 4.50 corresponds to the highest score that can be attained on the test.

The predicted gain, actual gain, and value-added data represent the aspects of the achievement results in terms of proficiency ratings and the percentiles compare a teacher’s results to the results of other teachers in either a Citywide or “Peer” comparison group.

To generate predicted gains the model uses city and state tests results in grades 3-8 from 1998-99 to 2007-08. The model also includes data about student, classroom and school characteristics, as well as teacher experience. Percentile scores are only calculated for individual teachers for the last three years.

Schools were given the opportunity to report during data verification whether two teachers shared equally the responsibilities for one class. If schools reported this information the two teachers’ names will appear on the report. Since two teachers are responsible for the class, we are unable to adjust for teacher experience. Therefore, only citywide comparisons will appear on the report. We are currently unable to reliably measure the amount and impact of part-time push in/pull out teachers or extra-period tutoring.

Teachers who teach a group of students two years in a row at two different grade levels will receive separate reports for each eligible grade they taught.

Results of students who move from one class to another during the school year are not included in Teacher Data Reports.

 A panel of technical experts has approved the DOE’s value-added methodology. The DOE’s model has met recognized standards for demonstrating validity and reliability. Teachers’ value-added scores from the model are positively correlated with both School Progress Report scores and principals’ reported perceptions of teachers’ effectiveness, as measured by a research study conducted during the pilot of this initiative.

Statistical techniques are used to take into account the impact of the prior-year teacher.   Each student’s prior-year teacher affects that student’s predicted gain. For example, if a student had a prior-year teacher whose students all made large gains compared to similar students, the predicted gain for that student would increase.  If a student had a prior-year teacher whose students tended to have small gains, the predicted gain for that student would decrease.

Yes. The results reported in the Teacher Data Reports demonstrate high stability from year-to-year. Results for teachers in the top and bottom quintiles are particularly stable.

  A username and password were not assigned to schools without reports. (A school only needed one eligible teacher to receive a username and password.) Reports were generated for ELA & Math teachers who taught 4th-8th grade students in 2007-08, unless:

1. There were issues with a teacher’s course assignment data
2. The teacher did not have at least 6 students who took the state ELA or math tests in two consecutive years and were in that teacher’s course from Oct 31 – June 30
3. The teacher did not teach for the full year (Oct 31 – June 30)

If you think you should have reports (and a username and password) and do not, please contact the Teacher Data Initiative support desk at (212) 374-6646 or TeacherDataReports@schools.nyc.gov.

The primary reason why reports were not generated for teachers whom schools think should be “eligible” is because of issues with a teacher’s course assignment data.

To generate a report for a teacher, the DOE needed to have the following pieces of information for each teacher: 1) an accurate file number associated with each eligible course the teacher taught and 2) that course associated with each eligible student’s identification number.

Because of how these data were historically stored, the DOE data did not have these data available for some schools and teachers. All schools were asked to, and given instructions on how to, submit the necessary data as part of a teacher course assignment data verification process that occurred in October 2008. Reports could not be generated if schools did not submit all of the necessary data.

Schools were asked to verify and make necessary changes to this information as part of the teacher course assignment verification process that took place in October 2008. The data schools submitted during this process were used to generate the reports. The available data from the source systems was used to generate reports for schools that did not verify all or some of their data.

If you think the data in the reports does not match the verified data sent to the DOE as part of the teacher course assignment verification process that took place in October 2008, please submit specific information about the data error to the Teacher Data Initiative support desk at (212) 374-6646 or TeacherDataReports@schools.nyc.gov. We expect that schools will have an opportunity to verify data in the spring of 2009 for future Teacher Data Reports.

Principals will receive their username and password at their training from their SAF and Network Leader. Principals should contact their SAF or Network Leader if they did not receive (or lost) their username and password at the training. The support desk can only give a principal his/her username and password if it receives confirmation from the SAF/ Network Leader that the principal completed training.