Continuous Improvement - Common Standard 4

Comprehensive Assessment Plan & Annual Assessment Projects

The education unit and its programs regularly assess their effectiveness in relation to the course of study offered, fieldwork and clinical practice, and support services for candidates.

Developing a Comprehensive Plan and Community of Practice: The School of Education (SOE) at Cal Poly is dedicated to being a learning organization that engages in intentional continuous improvement. We regularly and intentionally engage in reflection, collaboration, discussion protocols, and improvement projects driven by data, student learning artifacts, and stakeholder feedback. Grounded in Cal Poly’s signature pedagogy of Learn by Doing, our faculty work intersects theory and practice through scholarship and exploration of the cutting edge practices in partnership with K-12 Schools and local community colleges. Local partner districts serve as living laboratories in our collaborative efforts to improve educational opportunities for all learners. We recognize and approach comprehensive assessment as process of conducting research, using the data collection to identify and strengthen our impact on candidate preparedness, student learning, research theory, and our local community. 

In 2017, the SOE Coordinating Council meaningfully revised the unit's Comprehensive Assessment Plan and our unitwide learning themes, with an aim to create a coordinated and intentional approach to assessment that supports a more collective learning and improvement community. The SOE committed to an aim of developing more streamlined and intentional assessment practices across the SOE, responding to identified needs without increasing demands on limited time and personnel resources.  Drilling down into the needs and core principles of the unit, the SOE developed four major change ideas: to revise the unit's Learning Objectives to better exemplify shared principles across disciplines; to articulate a shared comprehensive improvement and assessment cycle; to create space for shared learning and improvement through a Community of Practice; and to develop a collaborative Wiki website of resources and dialogue. Over the course of the 2017-18 academic year, these changes took shape.

Currently, the SOE annually plans efforts to support shared assessment, continuous improvement, dialogue, and professional development, anchored by one of the unit's new Learning Themes. 

• Comprehensive Assessment Plan (PDF)
• Annual Assessment Plan 2018-19 - EmpowerED: Responding to Diverse Learning Needs (PDF)
• Annual Assessment Plan 2017-19 - InformED: Advancing Disciplinary Knowledge (PDF)

 

Data-Driven Continuous Improvement

Both the unit and its programs regularly and systematically collect, analyze, and use candidate and program completer data as well as data reflecting the effectiveness of unit operations to improve programs and their services.

The continuous improvement process includes multiple sources of data including 1) the extent to which candidates are prepared to enter professional practice; and 2) feedback from key stakeholders such as employers and community partners about the quality of the preparation.

Data Management and Analysis: The SOE staff recognizes that our programs and faculty members are best positioned to engage in continuous improvement when they have access to reliable and meaningful data about candidate learning. To support that need, the SOE staff has committed to engage in a coordinated improvement project to streamline unitwide data collection and data management. This improvement project includes identifying unitwide data needs, streamlining sources of data, determining roles and responsibilities for various data management tasks, and developing a plan for facilitating regular data-driven conversations across the SOE.

• Annotated List of Data Sources (PDF)
• Assessment System and Responsibilities (PDF)

Continuous Improvement Conversations & Efforts: As part of our coordinated Annual Assessment Plans (included above), and our commitment to continuous improvement, the School of Education seeks to engage in ongoing unitwide and programwide data-driven conversations. The SOE uses a Data Discussion Protocol to support more thoughtful and purposeful conversations. Our protocol was adopted from two research-based samples from The National School Reform Faculty and the School Reform Initiative.  These two templates were selected to meet the needs of the SOE because they: emphasize a preliminary conversation around a manageable chunk of data with targeted questions appropriate to our time constraints; focus on identifying data that is most valuable to our continuous improvement goals; identify root-causes and systemic issues at the unitwide and programwide level; improve data capacity and data integrity by developing shared understandings; and build collective buy-in across the unit.

• SOE Calendar of Quarterly Assessment Tasks & Data Discussions (PDF)
• SOE Data Discussion Protocol (PDF)
• Sample Data Protocol - edTPA Scores (PDF)
• Sample Data Protocol - Clinical Practice Observation Data (PDF)
• Sample Data Protocol - SSTEP Completer Survey (PDF)
• Sample Data Protocol - Employee Engagement Survey (PDF)
• Senior Project - Statistical Analysis of Observation Data (PDF) - During the 2017-18 academic year, the SOE utilized the services of an undergraduate senior from the Cal Poly statistics department as part of the senior thesis process.  Using anonymized data from student teaching observations conducted in the 2016-17 academic year, the student performed several statistical calculations to look for trends, statistical significance, and gaps in our data. The student was coached by the SOE Accreditation and Assessment Coordinator, as well the senior thesis advisor, over the course of the academic year.  Upon completion of the year, the student presented the data and findings to coordinators and members of the SOE faculty and made recommendations for the improvement of the SOE student-teaching program and the collection of student-teaching observation data. 

Collecting Feedback and Engaging Stakeholders

The continuous improvement process includes multiple sources of data including 1) the extent to which candidates are prepared to enter professional practice; and 2) feedback from key stakeholders such as employers and community partners about the quality of the preparation.

Collecting Feedback and Engaging Stakeholders: SOE Stakeholders have a variety of opportunities to engage in the assessment and feedback process, both formally (through scaled surveys and focus groups) and anecdotally (through collaboration and advisory opportunities).  

• SOE Advisory Board (PDF) - Established in the SOE Governance Document Article II. Section V, the SOE Advisory Board provides consultation and guidance for the overall direction and operation of the School of Education.
• Completer Focus Group (PDF) - Conducted on a quarterly basis to solicit more nuanced student feedback about how and why certain aspects of their program experience prepared them to be future teachers.
• Completer Survey (PDF) - Administered by EdQ to candidates in Multiple Subjects, Single Subjects, Special Education and Educational Leadership & Administration programs. Includes campus specific questions designed by SOE.
• 1-Year Out Survey - Administered by EdQ to 1-Year Out alumni to collect feedback on preparedness to enter the profession.
• Employer Survey - Administered by EdQ to principals who employed SOE alumni in their first year of teaching to collect feedback on preparedness to enter the profession.
• Alumni Survey (Sample) - Administered by SOE to alumni in their first 5-years after completion to collect feedback on preparedness and retention.
• Mentor Teacher Survey - Administered by CTC to mentor teachers who supervise a candidate in their final quarter of Clinical Practice.
• Cooperating Teacher Feedback Survey (Sample) - Administered by SOE to Cooperating Teachers who participate in co-teaching or grants programs.
• Grant Partner-District Surveys (Sample) - Administered by SOE to teachers and/or districts who participate in our grant funded programs to share feedback on how grant programs are supporting teacher preparation.
• Post Events & Workshops Surveys (Sample) - Administered by SOE to candidates, University Supervisors, teachers, and districts who participate in special events and workshops to improve teacher preparation programs.
• Employee & Partners Engagement Survey (Sample) - Administered by SOE to all full time and part time faculty, instructors, Clinical Practice Supervisors, staff, and campus partners.