1	Florida’s Professional Development System Evaluation Protocol Presentation of Protocol Standards Updates Florida Association for Staff Development Fall Leadership Conference, September 2006
2	Session Purposes Share results from the First Cycle Describe the Second Cycle Protocol System revisions Increase participation in and use of the Protocol System
3	Overview of the Protocol System
4	Major Shifts This Decade Tightened link between professional development and student achievement Increased concentration of professional development on teacher’s assignment Increased emphasis on skill and technique acquisition Increased use of technologies in delivery of training Application of instructional design research to professional development Emphasis on evaluating professional development effectiveness
5	Current Statutory Requirements (s. 1012.98 F.S.) Purpose: increase student achievement, enhance classroom instructional strategies that promote rigor and relevance throughout the curriculum, and prepare students for continuing education and the workforce. The system of professional development must align to the standards adopted by the state and support the framework for standards adopted by the National Staff Development Council.
6	Statutory Requirements (cont.) System must support School Improvement Plans through: Enhanced and differentiated instructional strategies to engage students in a rigorous and relevant curriculum based on state and local educational standards, goals, and initiatives; Increased opportunities to provide meaningful relationships between teachers and all students; and Increased opportunities for professional collaboration among and between teachers, guidance counselors, instructional leaders, postsecondary educators engaged in pre-service training for new teachers, and the workforce community.
7	Statutory Requirements (cont.) District plans must be aligned/linked with student and personnel needs and data sources: Analyses of student achievement data School discipline data School environment surveys Assessments of parental satisfaction Performance appraisal data School Improvement plans School inservice plans Other performance indicators
8	Statutory Requirements (cont.) District plans must provide: Inservice activities coupled with follow-up support appropriate to accomplish district-level and school-level improvement goals and standards Delivery of professional development by distance learning and other technology-based delivery systems to reach more educators at lower costs Continuous evaluation of the quality and effectiveness of professional development programs in order to eliminate ineffective programs and strategies and to expand effective ones.
9	Statutory Requirements (cont.) Collaborative development with: teachers teacher-educators of community colleges and state universities business and community representatives local education foundations consortia professional organizations
10	Statutory Requirements (cont.) Individual Professional Development Plans must: Be related to specific performance data for the students to whom the teacher is assigned. Define the inservice objectives and specific measurable improvements expected in student performance as a result of the inservice activity. Include an evaluation component that determines the effectiveness of the professional development plan.
11	Department of Education Shall disseminate, using web-based technology, research-based best practice methods by which the state and district school boards may evaluate and improve the professional development system. Report annually any school district that, in the determination of the department, has failed to provide an adequate professional development system.
12	New Searchable Database Department is developing a searchable database of highly rated practices Based on complete cycle of results Searchable by standard and district size Available on Department’s website
13	Results from First Cycle Mountains of Data
14	All 67 Districts Reviewed in First Cycle Districts selected based on: Size Geographic region
15
16	Evidence of Professional Development Impact Statistically significant moderate correlation between high ratings on standards and district’s evidence of increased student performance levels Significance level: .01 Correlation: .31 Evidence that good professional development programs are related to rising student achievement
17	Standards with Highest Average Ratings Content at all 3 levels Coordinated Records at all 3 levels District Level Leadership District Level Learning Strategies District Level Use of Technology in Delivery Coordination with School Improvement Plans Relevance of Professional Development
18	Standards with Lowest Average Ratings Action Research Web-based Resources and Assistance School Level Expenditures Evaluation Standards
19	Other Findings Highest ratings in Delivery Lowest ratings in Evaluation Medium districts received higher ratings than small districts
20
21
22	More First Cycle Findings Average rating for districts across all standards was 2.97 Range from 3.56 to 2.07 Median number of ratings below 2.0 (Marginal in the 4-point rating scales) was 4 Range from 0 to 26 represented a range of 0% to 39.4% of the 66 standards being rated below a “marginal” rating, with a median of 6.1%.
23	Summary of Highly Rated Practices First Cycle Summary Searchable Database
24	Charting the Course – Second Cycle
25	System Review and Revision Revision Team met on February 24^th Represented small/medium/large districts and consortia Experienced reviewers
26	Revision Team Laura Ahern Glades Naomi Boyer USF Todd Cluff Pasco Debbie Cooke Palm Beach Diane Cox Okaloosa Marlyn Dennison Pinellas Sandy Harrison Bay Sallie Jenkins Seminole Gary Moore Hardee Mike Todd Levy
27	Suggested Revisions Maintain Standards—revise Reviewer’s Guide, explanations, and rating rubrics Revise questions to probe more effectively Provide more examples of what is an excellent rating and what is unacceptable Clarify exit meeting process Drop SAC Chairs in the site visit/interviews Set aside time for Document Reviews Universities need to be more involved
28	Second Cycle Revamp Purpose remains Standards essentially the same Expansion of Reviewer’s Guide Increased detail in Rating Rubrics Small revisions in process New Protocol System document New Reviewer’s Guide Updated training system New searchable database on highest rated practices
29	Evaluation Protocol Purpose Ensure high quality district, school, faculty professional development Provide school districts with methods and protocols to conduct ongoing assessments on the quality of professional development
30	3-Year Review Cycle
31	Guided Discussions In smaller groups: Examine Protocol System Examine Reviewer’s Guide Discuss changes Use flipchart to identify questions or clarifying comments Report to whole group
32	Debrief Time
33	Reviewers are the Key!!! System depends on quality of reviewer judgments Reviewers represent the Department when onsite Your expertise is critical to the success of the whole Protocol System!!! Please participate!
34	Unlocking the Benefits of the Protocol System Improve the quality of professional development in all districts and schools Increase adherence to NSDC and Florida standards Identify best practices across the state Target professional development for impact Document effectiveness of professional development systems
35	Contacts/Follow-up Kathryn Hebda, Bureau Chief Phone: 850-245-0435 E-mail kathy.hebda@fldoe.org Henry Pollock, Program Director Phone 850-245-0546 E-mail: henry.pollock@fldoe.org Abigail Letcher, Program Specialist Phone: 850-245-0548 Email: abigail.letcher@fldoe.org