Tennessee Exemplary Educators Program An External Change Agent
Tennessee Exemplary Educators Program An External Change Agent Approach to Using Data in State-Identified Schools Mike Jordan Deborah Williams Aaron Butler Tennessee Exemplary Educators 1 at Edvantia, Inc. formerly the Appalachia Educational Laboratory (AEL) Who are we? Edvantia is an education research and development not-for-profit corporation founded in
1966. We partner with educators, agencies, publishers, and service providers to improve learning and advance student success. What do we do? With core capabilities in research, evaluation, professional development, and technical assistance, Edvantia helps its clientpartners succeed in meeting federal and state mandates for high-quality teachers, research-based instructional practices, and school improvement. Tennessee Exemplary Educators 2
at Cookin with Data! Tennessee Exemplary Educators 3 at Whats on todays menu? An overview of the Tennessee Exemplary Educators program A discussion of what Exemplary Educators do to help schools use data effectively Tennessee Exemplary Educators 4
at The Appetizer: What is the Tennessee Exemplary Educators program? Tennessee Exemplary Educators 5 at What is the EE program? An initiative of TDEs Office of Accountability Drawn from research on role of external change agents in facilitating organizational change and school improvement Educators can learn best from other educators Tennessee
Exemplary Educators 6 at The EE Program Capitalizes on the experience and expertise of recently retired educators who have demonstrated track records of success in instructional or administrative roles Provides EEs with a continuous program of professional development for their new role Assigns EEs to provide individualized, on-site technical assistance in identified schools Tennessee Exemplary Educators 7 at The Work in High Priority Schools
School Improvement Planning Curriculum Alignment Best Practice Instruction Assessment Practices Tennessee Capacity Building Exemplary Educators 8 Reculturing at Where do data fit in? Most schools already have a wealth of data at their disposal EEs help schools use their data effectively to drive organizational and instructional processes Tennessee
Exemplary Educators 9 at District Directors Reflection Our state assigned two School Performance Coaches to work with our district. With the help of our improvement specialists, supervisors began to review the available data: state assessment and NCLB reports as well as individual student test and value-added data. One positive outcome of our year was the addition of a data management coordinator, a recommendation by the Performance Coaches. District Performance Insights, 2006 Tennessee Exemplary Educators 10 at
Savoring our Successes What outcomes are we seeing associated with the EE program? Tennessee Exemplary Educators 11 at Outcomes From 2001 through 2006, Tennessee identified 225 High Priority schools. 128 High Priority schools have achieved AYP for two consecutive years and come off the list (57%) 35 achieved AYP for the first year in 2007 and are eligible to come off the list in 2008. Tennessee
Exemplary Educators 12 at Results Year No. of High Priority Schools No. That No. That % That Achieved Achieved Achieved Good AYP AYP Standing
2003 2004 88 All TN Schools High Priority Schools 77 Target 50 2005 2006 Year Tennessee
Exemplary Educators 15 at In 2007, the TN EE program was recognized with a Top 50 Innovations in American Government Award from Harvard University. Tennessee Exemplary Educators 16 at Lets get into the kitchen! What recipe do EEs use to help schools use their data effectively?
Tennessee Exemplary Educators 17 at Why should cookin schools collect, analyze and use data? Simple plans work best. Our best plan is to arrange for teachers to analyze their achievement data, set goals, and then meet at least twice a month for 45 minutes or so. Mike Schmoker in Results Now Tennessee Exemplary Educators 18 at Why should cookin schools collect,
analyze and use data? A program or activity may have great merit and yet be of little worth to the organization simply because it does not coincide with identified needs or is not aligned with the organizations mission. Thomas Guskey in Leading Every Day Tennessee Exemplary Educators 19 at Cookin with Data Join us in our search for the right ingredients and a great Tennessee recipe for improved schools. Well add, stir, separate, and recombine data to cook up decisions for student success. Then well layer our findings into goals and action steps for a savory, satisfying School Improvement Plan!
Tennessee Exemplary Educators 20 at Heres what chefs need: To know the different types of data to be collected To understand how different types of data should be analyzed To understand how different types of data connect and interact To know how to ask the right questions to encourage schools to use data effectively Tennessee Exemplary Educators 21 at
What data should cookin schools collect, analyze, and use? 1. Demographic data Defined as statistical data that describes a population Tennessee Exemplary Educators 22 at Examples of Demographic Data
Grade distribution Length of school year Subgroup information Length of school day Enrollment data Tennessee Exemplary Educators 23 at What data should cookin schools collect, analyze, and use? 2. Perceptual data Type of data that deals with the opinions or views of peoplethe way
they understand something Tennessee Exemplary Educators 24 at Volunteer Elementary School Continuous School Improvement Questionnaire Results, 2005 Fall 2001 Spring 2002 Spring 2003 Spring 2004 Spring 2005 100
What data should cookin schools collect, analyze and use? 3. Process data Let these data guide your school toward change. Tennessee Exemplary Educators 26 at School Walkthrough Form School: ____________ Grade: ____ Teacher:____________ Date: ____________Time: _____ Subject: _____________ Learning Environment Observed Students purposefully engaged
Student work and rubrics displayed Classroom rules/consequences posted Daily/weekly goals/objectives posted Safe, clean, organized, and attractive YES NO YES YES NO Tennessee Exemplary Educators 27 at What data should cookin schools collect, analyze, and use? 4. Student Learning Data Measures of student learning help us understand how students are performing and
what students know as a result of instruction. Victoria Bernhardt, 2004 Tennessee Exemplary Educators 28 at Reading Performance Level 100 80 69 58 60 40 54 37
Tennessee Exemplary Educators 29 at ACTIVITY Over a three-year history, what has happened to the reading performance of students at this school? What would you tell the teachers? What would we need to concentrate on this year? Tennessee Exemplary Educators 30 at Mathematics Performance Level 100
14 0 2005 2006 2007 Tennessee Exemplary Educators 31 at ACTIVITY Over a three-year history, what has happened to the math performance of students at this school? What would you tell the teachers? What would we need to concentrate on this year?
Tennessee Exemplary Educators 32 at Why should a cookin school collect student learning data? To know if students have mastered skills To determine the knowledge students have gained To know if school processes are effective To know if students are ready to move to the next level/grade Tennessee Exemplary Educators 33 at
Types of student learning data TCAP achievement tests & writing assessment End-of-Course and Gateway exams Value-added scores Data from textbook & teacher-made assessments Samples of student work Data from student report cards Data from technology-generated assessments Tennessee Exemplary Educators 34 at
A menu of student learning data? From formative assessments From summative assessments From test item analyses From state web site From reports (TCAP, TVAAS, NCLB, etc.) From private foundation studies Tennessee Exemplary Educators 35 at
How should a school start cookin student learning data? Collect Organize Analyze Use Tennessee Exemplary Educators p. 12 36 at Raid the pantry! We need more ingredients! Norm-Referenced Tests Show how students do in relation to a norming group Criterion-Referenced Tests Show how students do in relation to learning standards Tennessee
Exemplary Educators 37 at Tennessee Cookbook for Curriculum Alignment The Dept. of Education published the Blueprint for Learning, a guide to the state curriculum to help teachers know what skills (SPIs) each student should have at each grade level. Tennessee Exemplary Educators 38 at Get out the pots & pansheres whats cookin in TN!
TCAP Achievement Tests Required in Grades 3-8 Criterion-referenced test questions correlate to the Blueprint/state standards Tennessee Exemplary Educators 39 at Get out the pots & pansheres whats cookin in TN! Gateway Exams Algebra I, Biology I, English II All students in these courses take the exam; exam grade counts as percentage of final grade. Graduation Requirement: Pass all 3 Tennessee
Exemplary Educators 40 at Get out the pots & pansheres whats cookin in TN! TCAP Writing Assessment Required in grades 5, 8, and 11 Scoring rubric: 0 to 6 Grade 5 35-minute Narrative Grade 8 35-minute Expository Grade 11 35-minute Persuasive Tennessee Exemplary Educators 41 at TN Writing Assessment
Score 3 Below Prof. 4 Proficient 5 Advanced # of boys scoring in each category 5 3 1 # of girls
scoring in each category 1 1 5 Tennessee Exemplary Educators 42 at Writing Activity What do we conclude from these writing assessment scores? What would we tell the teachers? What does the teacher need to do with classroom instruction?
Tennessee Exemplary Educators 43 at Get out the pots & pansheres whats cookin in TN! End-of-Course Exams All students enrolled in courses take the exam. Counts as percentage of final grade. Tennessee Exemplary Educators 44 at Raid the pantry! We need more ingredients!
Raw Score the number of questions that a student gets right on a test. Normal Curve Equivalent Scores equal interval scores based on the normal curve Range from 1 to 99 Can be averaged 40 NCE was the target for previously identified High Priority Schools Tennessee Exemplary Educators 45 at Raid the pantry! We need more ingredients! Scaled Scores Scores assigned by the testmakers on the basis of student achievement on tests; arbitrary scales Gain Scores The change or difference between two repetitions of the same test administered to the same students
Tennessee Exemplary Educators 46 at Scale Score Gain Form Class Average? Positive Gains? Negative Gains? One-Year Gains? SS Analysis? Purpose? Tennessee Exemplary Educators 47 at Scale Score Gains STUDENT READING
2007 SS MATH 2006 SS DIFF 2007 SS 2006 SS DIFF 652 663 -11 643 646
-3 687 642 45 638 638 0 668 632 36 647 585
62 633 627 6 650 642 8 643 611 32 593
583 10 600 578 22 506 587 -81 581 567 14 583
578 5 617 604 13 640 594 46 608 598 10
615 603 12 Tennessee Exemplary Educators 48 at Scale Score Gains Activity G4 Proficient - 457 G5 Proficient - 463 G4 Advanced G5 Advanced 507 517 Math SS Gain #1 - 457 #1 - 470
Exemplary Educators 49 at ACTIVITY Over a two-year history, what has happened to the math performance of students at this school? Which performance level is making the best gains? What does the teacher need to do with classroom instruction? Tennessee Exemplary Educators 50 at The Icing on the Cake Achieving AYP!
Tennessee Exemplary Educators 51 at AYP Essential Ingredients Grades 3-8 for 2008 Reading/Language Arts/Writing 89% Math 86% Grades 9-12 for 2008 Reading/Language Arts/Writing 93% Math 83% Tennessee Exemplary Educators 52 at Other Ingredients Numbers of students in each subgroup at each
school (TN uses N of 45) Numbers of 3-8 students below proficient Numbers of students who have not passed all three Gateway tests: English, math, and science Numbers of special education students who qualify for alternative assessments Tennessee Exemplary Educators 53 at Additional Indicators Participation 95% Graduation Rate 90%
Attendance 93% Tennessee Exemplary Educators 54 at Day Attendance 85 80 75 FALL DECEMBER FEBRUARY 70 65 60
2005 2006 Tennessee Exemplary Educators 55 at Graduation Rate: Calculating Graduation Track 1. Take the required graduation rate of 90% with 89.5% being the lowest allowable percentage 2. Subtract the schools baseline (2004) graduation rate (e.g., 89.5 69.5 = 20) 3. Divide the difference by 10 since there are 10 years from 2004 until 2014(20 / 10 = 2.0) Tennessee Exemplary Educators
2005 2006 2007 2008 30 20 10 0 Read/Lang Math Figure It Figure It Tennessee Exemplary Educators 60
at Classroom AYP # Students Read 89% SH 71% Math 86% SH 75% 15 14 11
13 12 20 18 15 18 15 25 23 18 22
19 Tennessee Exemplary Educators 61 at Projection Growth Model supplements the statutory AYP model uses individual student data to determine the percentages of students, by subgroup and subject area, who are projected to attain proficiency on the state assessment three years into the future uses 7th- and 8th-grade projections for 4th- and 5thgrade students and high school graduation exam projections for 6th- to 8th-grade students uses current-year scores for 3rd-grade students, students new to the state, and students who take alternative assessments Tennessee Exemplary Educators
62 at And now, its time for the check. School or System AYP Report What is contained in this report? What conclusions can we reach? Tennessee Exemplary Educators 63 at Tennessee
Exemplary Educators 64 at AYP REPORT ACTIVITY Which student subgroups have ns of 45? What is the participation rate (% tested) in 2007? Did the % Below Proficient in the All category improve from 2006 to 2007? If the goal is 79%, did this school make AYP? What note concerning disaggregated race/ethnicity data would you include in the School Improvement Plan? Tennessee Exemplary Educators 65 at
Lets look at another check Sample Report for an Elementary School www.state.tn.us/education Tennessee Exemplary Educators 66 at Get out the pots & pansheres whats cookin! TVAAS Not a test but a statistical system to analyze scores Measures growth in Scale Score points Goal: reach one years growth (100%) each year Can determine teacher effect Do special education students count in TVAAS? Tennessee Exemplary Educators
67 at Tennessee Exemplary Educators 68 at VALUE-ADDED ACTIVITY What grade level is not making gains in this school system over a 3-year average? Which one is likely to have problems if no improvement in gains are made next year? Which grade level is making the best gains? Tennessee Exemplary Educators 69 at
Tennessee Exemplary Educators 70 at VALUE-ADDED ACTIVITY Which group of students did not make gains last year (blue bar)? Over a three-year period (gold bar), which group made the best gains? Which group made the best gains last year? Which group has consistently made no gains? Tennessee Exemplary Educators 71 at
Tennessee Exemplary Educators 72 at Tennessee Exemplary Educators 73 at Tennessee Exemplary Educators 74 at STUDENT REPORT ACTIVITY
What would we conclude about this students history of gains (red line) over his elementary career? What is happening in middle school? Tennessee Exemplary Educators 75 at Tennessee Exemplary Educators 76 at Tennessee Exemplary Educators 77
at Tennessee Exemplary Educators 78 at Tennessee Exemplary Educators 79 at Servin Up Somethin Special Taking a Closer Look at the SIP Using the State Rubric Tennessee
Exemplary Educators 80 at ABCDE Five Components 1. All data 2. Beliefs, Mission, Vision 3. Curriculum, I, A, & O 4. Develop Action Plan 5. Evaluation of Plan Tennessee Exemplary Educators 81 at Component 1: Academic and Nonacademic Data Analysis
Variety of academic and nonacademic assessment measures Data collection and analysis Report Card data disaggregation Narrative synthesis of all data Prioritized list of targets Tennessee Exemplary Educators 82 at Servin Up the Meat & Potatoes Prioritized List of Goal Targets Exemplary List of Goal Targets: The list of goal targets matches data priorities and references the NCLB benchmarks. Tennessee Exemplary Educators
83 at Component 4: Action Plan Development Goals Action Steps Implementation Plan Tennessee Exemplary Educators 84 at SIP Monitoring Goal 1 The percent of students scoring proficient or above in the area of Reading/Language Arts will increase from 63% on the 2006 TCAP tests to 83% of the 2007 TCAP tests. Action Step
Date Monitored Date Monitored All teachers will implement the Balanced Literacy Model to build vocabulary by reading from a variety of texts and using content specific vocabulary during the daily literacy block. 12/5/2006 3/13/2007 Action Step 1 Status Action Step 1 Status Not implemented 0 Not implemented 0 Partial. implemented 7 Partial. implemented 1 Fully implemented
9 Fully implemented 18 Tennessee Exemplary Educators 85 at SIP Monitoring, continued Goal 1 The percent of students scoring proficient or above in the area of Reading/Language Arts will increase from 63% on the 2006 TCAP tests to 83% of the 2007 TCAP tests. Action Step Date Monitored Date Monitored All teachers will provide differentiated instruction and flexible grouping daily to prevent
deficiencies in reading through intervention and the grade level instructional focus. 12/5/2006 3/13/2007 Action Step 1 Status Action Step 1 Status Not implemented 2 Not implemented 0 Partial. implemented 9 Partial. implemented 5 Fully implemented 4 Fully implemented 14 Tennessee Exemplary Educators 86 at
Raid the pantry! We need more ingredients! Reporting Category Performance Indicator Shows percentage of test items answered correctly for an objective Shows level of mastery (for individual students or groups) of a test objective Can determine strengths and needs relative to test objective Can show mastery by school, teacher, & nation Tennessee Exemplary Educators 87 at More Entrees! RCPI Ratio Calculate by dividing school RCPI by State RCPI. RCPI Ratios determine strengths and needs by test objective. Ratios of 1.05 or higher may be considered strengths.
Ratios of .95 or lower may be considered needs. Use in SIP to target needs and develop action steps. Tennessee Exemplary Educators 88 at RCPI Analysis Computation Data Analysis Geometry System Avg 85
Exemplary Educators 89 at RCPI ACTIVITY Which math category does the teacher need to work on the most? In which category are students doing the best? Tennessee Exemplary Educators 90 at Relationships in the Data A school is truly using its data when people begin to look at the relationships in the data. This is illustrated by the intersection of different
types of data, how one influences another, and how patterns are created. Tennessee Exemplary Educators 91 at Servin Up Somethin Special A Smorgasbord of strategies! A buffet of bountiful data! Tennessee Exemplary Educators 92 at ARE YOU SMARTER THAN A (TN) 5TH GRADER?
$300 Question from Data Analysis Mrs. Blaylock weighed seven students in her class and found the following weights: (92, 84, 82, 88, 86, 90, 80) lbs What is the median weight of the seven students? A. 84 B. 86 C. 88 D. 90 Tennessee Exemplary Educators 93 at $300 Answer from Data Analysis(5.5.3) c. 86 lbs Tennessee
Exemplary Educators 94 at 5th Grade SPI Calendar Days Aug 1317 5 Aug 2024 5 Total 82 Read Lang
Writing Math 27 12 26 47 Tennessee Exemplary Educators 95 at Planning Form Grade_____________Date______________ Items Discussed/Reviewed:
__ Literacy __ Math __ Writing __ Social Studies __ Science __ Other What SPIs are the focus of this planning session? 1. What worked (5-10 minutes): Each team member reflects on how well the identified strategy (selected by the team at the previous meeting) worked. 2. Chief challenges (3-5 minutes): What is the most urgent instructional concern, problem, or obstacle to progress and better results? 3. Proposed solutions (8-10 minutes): Brainstorm solutions to these problems. 4. Action plan (10 minutes): Decide which solution or strategies might be best for the team to focus on and implement. Tennessee Exemplary Educators 96 at 4 Question Assessment 1. Estimate the temperature that is shown on this thermometer.
A. B. C. D. 50 40 35 30 2. The temperature on a thermometer is 32 F. Which of the following answers makes the most sense for the weather? A. B. C. D. Warm summer day August in Chattanooga Wintertime
Rainy summer day Tennessee Exemplary Educators 97 at Data on Display 1. Value of Collaboration: Teaching and learning are improved when teachers and administrators regularly collaborate to plan and assess instruction. 100 2. 80 70
60 50 40 30 20 10 0 Faculty and Staff Collaboration: Faculty and staff in my school work together to improve teaching and learning for their students. 100 3. 90 90
80 70 60 50 40 30 20 10 0 Faculty Trust: Faculty and staff in my school have confidence that there is no reason to be protective or careful around their peers. 100
Tennessee Exemplary Educators 100 at Getting Staff Perceptions Sept. 06 Dec. 06 Our staff is committed to recognizing student achievement and developing pride in success. 7.71 9.59 Our entire staff does a good job enforcing agreed-upon building rules and policies. 5.29
7.68 Our staff makes timely parent contacts regarding student discipline. 6.80 8.63 Our staff is visible and seen by the students in all building locations throughout the day. 7.5 9.28 Tennessee Exemplary Educators 101 at
Time Studies 8:15 8:30 8:45 9:00 9:15 9:30 9:45 10:00 Tennessee Exemplary Educators 102 at School Walkthroughs / Observations Tennessee Exemplary Educators 103
at Formative Assessments Number advanced, proficient, and below proficient Bubble students Identify SPIs for introduction or reteaching Trend analysis Tennessee Exemplary Educators 104 at JEOPARDY MATH Number s Algebra
Q $500 Q $500 Q $500 Q $500 Q $500 Tennessee Exemplary Educators 105 at ARE YOU SMARTER THAN A (TN) 3RD GRADER? What are the two missing symbols in this pattern? +, =, <, >, _, _, <, >, +, = a. <, > b. >, + c. +, =
d. None of the above Tennessee Exemplary Educators 106 at The Experts Are Among Us One of the reasons such teamwork and lesson study are so effective is that they tap into teachers existing capabilities and potential. Any faculty could begin improving performance, tomorrow morning, if they never attended another workshop in their lives. Mike Schmoker, Results Now (p. 114) Tennessee Exemplary Educators 107
at Get Cookin! It is not because things are difficult that we do not dare; it is because we do not dare that they are difficult. Lucius Amaeus Seneca Tennessee Exemplary Educators 108 at Bon Appetit! Tennessee Exemplary Educators 109
at For information about Edvantia research, products, or services, contact P.O. Box 1348, Charleston, WV 25325 Phone 304.347.0400 Fax 304.347.0487 [email protected] www.edvantia.org Edvantia is a nonprofit education research and development corporation, founded in 1966, that partners with practitioners, education agencies, publishers, and service providers to improve learning and advance student success. Edvantia provides clients with a range of services, including research, evaluation, professional development, and consulting. 2007 by Edvantia This presentation was developed under contract number GR 06-17396-00. Content was developed by Edvantia Research and Evaluation Specialist Aaron Butler and Tennessee Exemplary Educators Mike Jordan and Deborah Williams. Tennessee Exemplary Educators 110 at
Where is the Main Idea? The main idea can be found in three places: In the beginning, perhaps the first sentence. In the end, maybe the last sentence. Not stated, but implied (or inferred) through all the sentences. Find the...
Community Front Door team. HSCP. £30,000 = start by creating the skill base within GP practice admin staff . Starts as purely signposting to the appropriate service. As the CFD project grows, it can become a specific role; look to...
Source: BBC BiteSize. Living conditions in 19th Century cities. Pollution: coal was used to heat houses, cook food and heat water to produce steam to power machines in factories. The burning of coal created smoke, which led to terrible pollution...
Nonreductive Representationalism Phenomenal property <-> Phenomenally representing content C or: visually-phenomenally representing C. Further: content C may also presuppose the phenomenal Cf. projectivism, Shoemaker, Stoljar P-representing object as having quale Q P-representing object as disposed to cause quale Q. …
MapReduce computing frameworks like Hadoop and storage systems like HBASE and Cassandra provide low-cost, practical system foundations. Analysis also requires efficient algorithms, because "data in flight" may only be observed one time, so conventional storage-based approaches may not work.
Classification for High Dimensional Problems Using Bayesian Neural Networks and Dirichlet Diffusion Trees Radford M. Neal and Jianguo Zhang the winners of NIPS2003 feature selection challenge University of Toronto The results Their General Approach Use simple techniques to reduce the...
crystal array positioned parallel to the photon beam was used to detect the high energy gamma rays and provide information about the penetration depth of the gammas and the vertical divergence. Figure 1 shows an example of the data obtained...
Ready to download the document? Go ahead and hit continue!