|
| |
Accessing Reports
| |
Q:
How and when will Teacher Data Reports be distributed?
A:
Teacher Data Reports will be available the second week of December.
Principals will obtain access to their reports after they have received training from their Network Leader and SAF. At the training, Network Leaders & SAFs will distribute login information to a password-protected website where principals can access their schools’ reports: http://www.nycteacherdatainitiative.com.
Principals can go directly to this website or access it through the Teacher Data Toolkit section of the Principal’s Portal (under the Get Your Reports menu): http://intranet.nycboe.net/DOEPortal/Principals/HiringAndStaffDev/StaffDevelopment/TeacherDataToolkit.htm
Principals will distribute Teacher Data Reports to their teachers after principals have been trained. Principals can choose to share the reports with administrators in their schools so that they can determine together how to best use the reports to improve instruction at the school. A teacher’s report should not be disseminated to other teachers without the teacher’s permission.
|
Q:
Which teachers will receive reports?
A:
Reports were generated for ELA & Math teachers who taught 4th-8th grade students in 2007-08, unless:
- There were issues with a teacher’s course assignment data
- The teacher did not have at least 6 students who took the state ELA or math tests in two consecutive years and were in that teacher’s course from Oct 31 – June 30
- The teacher did not teach for the full year (Oct 31 – June 30)
Principals will receive the reports for all eligible teachers in their school during 2007-08 as well as reports for eligible teachers who are currently teaching in the principal’s school but taught in another DOE school in 2007-08.
|
Q:
Can parents and/or the public download Teacher Data Reports?
A:
No. Reports are only available to school administrators and will be shared with eligible teachers individually.
|
Q:
Can the assistant principal have conversations with teachers about their data reports?
A:
Yes, principals are free to share the Teacher Data Reports with assistant principals and other administrators. Principals are expected to arrange individual conversations with each teacher about his/her report, and in some cases those conversation should be led by or include an assistant principal.
|
Q:
Will principals have access to Teacher Date Reports when hiring teachers transferring from other schools?
A:
Principals may ask teachers they are interviewing if they are willing to share their reports. It is up to the teacher to decide whether to share his/her report.
|
|
| |
Initiative Background and Overview
| |
Q:
How can you be sure that principals won’t use the Teacher Data Reports to evaluate teachers?
A:
Principals have been and will continue to be explicitly instructed not to use Teacher Data Reports to evaluate their teachers. The DOE has standard processes in schools for teachers to raise issues or concerns.
|
Q:
Is New York City the first district to use a teacher “Value-Added” model?
A:
No. Several large cities, and even some states, currently use Value-Added models to generate teacher data. States providing Value-Added teacher data in more than one district include: Ohio, North Carolina, Tennessee, Texas, and Florida. Cities providing Value-Added teacher data include: Houston, Dallas, Milwaukee, Charlotte/ Mecklenburg. Chicago and Washington, D.C. are currently developing Value-Added teacher data systems.
|
Q:
Was the NYC DOE pilot of this initiative successful last year?
A:
Principals in the pilot conducted by the DOE in 2007-08 overwhelmingly reported that the information in the Teacher Data Reports is useful for a variety of instructional improvement purposes like planning individual and group professional development. About 100 volunteer schools received Teacher Data Reports in the pilot phase. At the end of the year, 86% of pilot principals said they felt the information is useful to principals, and more than 75% of principals thought it is useful for teachers.
|
Q:
How much will the Teacher Data Initiative cost?
A:
The Teacher Data initiative is funded by a grant from the Carnegie Foundation. The 2008-09 academic year is the last year of this 3-year grant.
|
Q:
How are the measurements in the Teacher Data Reports different from School Progress Reports?
A:
Progress Reports and Teacher Data Reports have different objectives. Like the Teacher Data Reports, School Progress Reports were designed to help principals and teachers accelerate academic achievement for their students. However, School Progress Reports were also created to enable students, parents, and the public to hold the DOE and its schools accountable for student achievement and to determine eligibility for school rewards and consequences. In contrast, Teacher Data Reports are not to be used for evaluation purposes. Principals are not to use these reports in tenure determinations or in the annual rating process. Teacher Data Reports are intended to help teachers and principals pinpoint teachers’ strengths and weaknesses and devise strategies for improvement.
Further, while both School Progress Reports and Teacher Data Reports contain data about student gains on state tests, School Progress Reports provide information about how schools as a whole are doing while Teacher Data Reports provide information about how individual teachers’ efforts influence student learning.
|
Q:
How do Teacher Data Reports relate to other DOE data tools?
|
Q:
Why is the DOE providing Teacher Data Reports?
A:
The DOE is providing Teacher Data Reports to help principals and teachers better understand how teacher efforts are influencing student progress. The reports can help answer questions about how teachers are influencing the performance of different groups of students (top third, bottom third, ELL, special education, male, female) in their classes over time.
Far too many City students perform below standards. Research has shown that teachers have substantial influence, both positive and negative, on students’ achievement. Teachers should be able to identify the effectiveness of their instruction and adjust it to meet their students’ needs. Principals should be able to identify school-wide instructional needs, select professional development opportunities, and facilitate the sharing of best practices that address these needs. By isolating individual teachers’ contributions to student progress, the Teacher Data Reports provide valuable information to school leaders and teachers about where to focus instructional improvement efforts.
|
Q:
How should Teacher Data Reports be used?
A:
The DOE is providing Teacher Data Reports as a tool for schools and teachers to use for instructional improvement.They are not be used to evaluate teachers.
|
|
| |
Reading and Interpreting Reports
| |
Q:
What specific factors are used to calculate predicted gains?
A:
The following factors were used to calculate each student’s predicted gain. A sophisticated multivariate regression analysis determined how much to weigh each factor:

|
Q:
Why are some sections of the Teacher Data Reports blank?
A:
For each section of the report, data is only presented if teachers had at least 6 eligible students in each category. For example, a teacher with 20 eligible students, 4 of whom are Special Education students, will receive data on her impact on all 20 students’ progress but will not receive information about her impact on the progress of her Special Education students.
|
Q:
How are Teacher Data Initiative school summary reports and teacher level reports different?
A:
A school summary report aggregates and summarizes data found on the reports for individual teachers in a principal’s school. Teacher-level reports provide greater detail about each teacher’s results in particular years and with particular subgroups of students.
|
Q:
The results on one of my teacher’s report contradict my belief about that teacher. What should I do if my views differ from the reported results?
A:
The Teacher Data Reports are intended to give principals and teachers another lens through which to view teacher performance. Principals whose previous views are challenged by the results presented in the Teacher Data Reports should reflect on what factors in either his/her perspective or in the teacher’s performance caused this difference. By asking and answering more questions, principals and teachers can determine if any new strategies could be helpful to improve the effectiveness of an individual or group of teachers.
|
Q:
What is the “peer comparison”?
A:
The “peer comparison” sections of the report are different from the “citywide comparison” sections in two ways. First, the predicted gain in all peer comparison calculations takes into account the teacher’s experience overall and in that grade and subject. Second, teachers are divided into 5 groups based on their average predicted gain, meaning they have similar student, classroom, and school characteristics. For peer comparisons, a teacher’s result is compared to the results only from classrooms in their group. There are several hundred teachers in each peer comparison group.
|
Q:
Why do Teacher Data Reports provide a range around the percentile result? How should I interpret a large range?
A:
Statistically, a teacher’s contribution to students’ test score gains most likely lies near, but may not be exactly equal to, the percentile result on the report. All statistical measures contain some uncertainty, reflected by a range (also known as a confidence interval) around the percentile. The range provided in the Teacher Data Reports means you can be 95% certain that a teacher’s contribution falls in that range, most likely closer to the highlighted result than at the ends of the range.
Larger ranges indicate that there is more uncertainty about a teacher’s value-added percentile. The size of the range differs for each calculation based on a number of factors, including the number of students included in the calculation. Ranges will generally be larger when a teacher has taught for fewer years or had fewer eligible students.
|
Q:
What is “value-added”?
A:
“Value-added” is a statistical term which, in the context of the Teacher Data Initiative, refers to how much an individual teacher contributes to his/her students’ gains on the state ELA and Math tests.
Teacher Data Reports calculate teachers’ “value-added” by controlling for more than 35 factors that can influence student achievement but that are outside of a teacher’s control, including students’ prior test scores, class size, and the percentage of students in the school with disabilities and living in poverty. These factors are used in a statistical model to predict each student’s test score gains in math and ELA from one year to the next. The average predicted gains for all of a teacher’s students are then compared to the students’ actual gains on the tests. The difference between the average predicted gain and average actual gain is the “value-added” result for the teacher.
|
Q:
What does the “percentile” represent?
A:
For each section of a teacher’s report, the “percentile” shows the percentage of teachers who had value-added scores lower than the teacher’s.
|
Q:
What units are “predicted gain”, “actual gain”, and “value-added” in? How should I interpret these numbers?
A:
The predicted gain, actual gain, and value-added data are in proficiency ratings, on a continuum from 1.00 to 4.50. A proficiency rating of 1.00 corresponds to the lowest score a student in Performance Level 1 can attain on the state ELA and Mathematics tests. A proficiency rating of 1.99 correspondents to the highest score a student can attain and still be at Performance Level 1. A proficiency rating of 4.50 corresponds to the highest score that can be attained on the test.
The predicted gain, actual gain, and value-added data represent the aspects of the achievement results in terms of proficiency ratings and the percentiles compare a teacher’s results to the results of other teachers in either a Citywide or “Peer” comparison group.
|
Q:
What data goes into the calculations on the Teacher Data Reports?
A:
To generate predicted gains the model uses city and state tests results in grades 3-8 from 1998-99 to 2007-08. The model also includes data about student, classroom and school characteristics, as well as teacher experience. Percentile scores are only calculated for individual teachers for the last three years.
|
|
| |
Special Situations
| |
Q:
How are CTT and other co-teaching situations handled?
A:
Schools were given the opportunity to report during data verification whether two teachers shared equally the responsibilities for one class. If schools reported this information the two teachers’ names will appear on the report. Since two teachers are responsible for the class, we are unable to adjust for teacher experience. Therefore, only citywide comparisons will appear on the report. We are currently unable to reliably measure the amount and impact of part-time push in/pull out teachers or extra-period tutoring.
|
Q:
How is looping handled?
A:
Teachers who teach a group of students two years in a row at two different grade levels will receive separate reports for each eligible grade they taught.
|
Q:
How are students who change classes / move during the school year handled?
A:
Results of students who move from one class to another during the school year are not included in Teacher Data Reports.
|
|
| |
The Value-Added Model
| |
Q:
Is the DOE’s value-added model reliable and valid?
A:
A panel of technical experts has approved the DOE’s value-added methodology. The DOE’s model has met recognized standards for demonstrating validity and reliability. Teachers’ value-added scores from the model are positively correlated with both School Progress Report scores and principals’ reported perceptions of teachers’ effectiveness, as measured by a research study conducted during the pilot of this initiative.
|
Q:
New York State tests are given in January (ELA) and March (Math). How is the fact that two classroom teachers teach student between the two test administrations handled?
A:
Statistical techniques are used to take into account the impact of the prior-year teacher. Each student’s prior-year teacher affects that student’s predicted gain. For example, if a student had a prior-year teacher whose students all made large gains compared to similar students, the predicted gain for that student would increase. If a student had a prior-year teacher whose students tended to have small gains, the predicted gain for that student would decrease.
|
Q:
Are the Teacher Data Reports stable from one year to the next (i.e., is there a strong relationship between a teachers score from one year to the next)?
A:
Yes. The results reported in the Teacher Data Reports demonstrate high stability from year-to-year. Results for teachers in the top and bottom quintiles are particularly stable.
|
|
| |
Troubleshooting
| |
Q:
My SAF/ Network Leader does not have a username and password for my school.
A:
A username and password were not assigned to schools without reports. (A school only needed one eligible teacher to receive a username and password.) Reports were generated for ELA & Math teachers who taught 4th-8th grade students in 2007-08, unless:
- There were issues with a teacher’s course assignment data
- The teacher did not have at least 6 students who took the state ELA or math tests in two consecutive years and were in that teacher’s course from Oct 31 – June 30
- The teacher did not teach for the full year (Oct 31 – June 30)
If you think you should have reports (and a username and password) and do not, please contact the Teacher Data Initiative support desk at (212) 374-6646 or TeacherDataReports@schools.nyc.gov.
|
Q:
One or more of my teachers met the criteria for receiving a report but one was not generated for him/her. What happened?
A:
The primary reason why reports were not generated for teachers whom schools think should be “eligible” is because of issues with a teacher’s course assignment data.
To generate a report for a teacher, the DOE needed to have the following pieces of information for each teacher: 1) an accurate file number associated with each eligible course the teacher taught and 2) that course associated with each eligible student’s identification number.
Because of how these data were historically stored, the DOE data did not have these data available for some schools and teachers. All schools were asked to, and given instructions on how to, submit the necessary data as part of a teacher course assignment data verification process that occurred in October 2008. Reports could not be generated if schools did not submit all of the necessary data.
|
Q:
There are errors in my data (e.g., a teacher is listed as teaching a subject he never taught, a teacher is listed as teaching more/fewer students than he actually taught). How should I proceed?
A:
Schools were asked to verify and make necessary changes to this information as part of the teacher course assignment verification process that took place in October 2008. The data schools submitted during this process were used to generate the reports. The available data from the source systems was used to generate reports for schools that did not verify all or some of their data.
If you think the data in the reports does not match the verified data sent to the DOE as part of the teacher course assignment verification process that took place in October 2008, please submit specific information about the data error to the Teacher Data Initiative support desk at (212) 374-6646 or TeacherDataReports@schools.nyc.gov. We expect that schools will have an opportunity to verify data in the spring of 2009 for future Teacher Data Reports.
|
Q:
I have not received/ lost my username and password. How do I get that information?
|
|
| |