The WPSD Survey is Deeply Flawed, and Shouldn't be Considered as a Source for Decision Making

WPSD Survey:  A lesson in poor survey methodology

A survey deeply flawed from its beginnings to its results.  This article dives into the November 2022 WPSD Survey and why it shouldn’t be considered as a reliable data source for decision making.   

·       “Stakeholder” survey not given to all stakeholders, excluded Merit Academy (public Charter School) and other families and taxpayers in the district. 

·       Survey rushed through in less than two weeks following the school board’s announcement to hire a new interim superintendent. Spent $8,200 to hire a company to host and deploy the survey. 

·       School administration including interim superintendent Tina Cassens wrote survey questions including questions about hiring a new superintendent.

·       Double dipping in the survey:  Each family received a survey for each child they had in the district, providing certain families with multiple responses.  In addition, each teacher with a child in the district received a survey, co-mingling staff and parent responses. This provided more weight to certain sub-groups. 

·       Former staff members were included in the surveys. 

·       Survey questions were written in a biased and leading way, missing context. 

·       Survey had a 54% response rate with certain respondents able to take the survey multiple times, while excluding other families. 

Much has been made of the WPSD’s November 2022 study results, but almost no conversation has been held around the methodology of the survey.  To start, the survey was labeled as a “Stakeholder” survey; however, the survey was not provided to all WPSD stakeholders. After complaints were received, it was changed to “The Family Survey”.  More complaints followed stating that the survey was not provided to all families in WPSD.  The survey on the WPSD website as of March 5th, 2023, is now labeled simply as “Survey Results”.  CORA results have provided a more in-depth review of the survey including the details behind the creation, cost, and deployment of the survey.  

Let’s begin with the creation of the survey as it was a very rushed process.  The district engaged a company called “Qualtrics” on Friday, November 4th to assist with the platform to deploy the survey.  In the email correspondence between WPSD Communications and Marketing Director, Lindsey Prahl and Qualtrics, Lindsey states in response to meeting the following Friday (11/11), “…Do you happen to have anything sooner, by chance?” “…we would like to get our first survey out next week.”  Del Garrick signed off on the order form from Qualtrics on November 7th for $8,200.  The survey was then deployed on November 14th.  The survey was rushed and turned around in less than two weeks following the School Board’s announcement on 10/26 to search for a new superintendent. 

Qualtrics was hired to host and deploy the survey, but not write the survey.  In fact, WPSD did not use a third party to write the survey, questions were written by the administration.  Lindsey Prahl states in the same email thread to Qualtrics, “Also, we feel comfortable building the survey…”.   Email threads also show that Tina Cassens, former Interim Superintendent has input on edits in the survey.  The survey includes questions around preferences of hiring a new superintendent or not, as well as ranking traits and priorities of a superintendent.  Note that at the time Tina Cassens was participating in the authoring of survey questions about whether or not to hire a new superintendent or keep the existing interim superintendents, Tina was surving as a interim superintendent and applying for the role of permanent superintendent. Conflict of interest? If the district was genuinely seeking quality results, why would you use a biased party to write the survey questions? This gives the appearance that the administration wanted to get feedback from select groups in a biased way to rig the interview process.

The survey was deployed on November 14th and was presented as a “stakeholder survey”.  The district website claims the survey was sent to “Parents/guardians completed a survey for each student in the District's public schools”.  The survey was intentionally not sent to Merit Academy parents or guardians which are in fact a part of the district’s public schools.  The “stakeholder” survey was also not open to families or taxpayers that do not send their children to a WPSD public school but live in the WPSD limits.  Again, why would the district funded by taxpayer dollars not want input on from all families in district, especially Merit Academy, one of their public schools?   

The egregious double dipping of the survey makes it an unreliable data source.  There was also no demographic information collected upon taking the survey.   The survey results do also not currently clearly define the way the survey was conducted or accurately define the populations polled in the outputs of the data. Parents and guardians completed a survey for each student in the district’s public schools, so if one family had 3 children in the district, they were able to take the survey 3 times.  In addition, each teacher with a child in the district also received a survey which comingled the parent and staff data.  A CORA request determined which staff and families were sent the survey, there were staff members on the list that were not employed by the district at the time as well, which means that that the staff survey data is contaminated. In total, there are a little less than 1,800 students enrolled in WPSD not including Merit, the survey had a 54% response rate. The survey deployment lacked controls, consistency, and properly weighted results.  Why would the district consider making decisions based on a survey that was unfairly weighted to certain subgroups and outright ignores other groups?

The survey questions themselves are oddly worded and very leading.  Many of the questions are framed in a way to clearly indicate an answer that would be preferable, without outlining any of the potential costs, opportunity costs of those survey options. Here are a few examples of questions taken directly from the survey:

Whole School Elementary Innovation Focus-Please prioritize the following list from most important to least important.

·       Comprehensive Education (current model)

·       STEAM (Science, Technology, Engineering, Arts, Math)

·       Environmental Education (outdoor education embedded focus)

·       Classical Education 

·       Fine Arts (ie. Music, art, theater embedded focus)

Comprehensive Education is highlighted as the “current model”.  Classical education doesn’t even receive a description.  Should be no surprise the “Current Model” received the highest responses in the survey. 

What is your preference for Middle School

·       Separate middle school and high school buildings

·       Combined middle school and high school in one building

·       No Preference 

Leading the witness here and providing zero context.  Of course, majority of respondents will not want to share a building.  In the case of WPSD, you have 1/2 full buildings draining resources and taxpayer dollars, so not providing context in misleading to say the least. No surprise that almost all respondents choose “Separate middle school and high school buildings”

What is your preference for Middle School:

·       Continuing to utilize the Summit Learning Platform to house curricular resources and track student progress.

·       Moving to alternative platforms for housing curriculum and tracking student progress

·       Moving to student textbooks and alternative platforms for tracking progress

·       No preference

This question is almost laughable.  They don’t provide any viable alternatives.  Virtually no one would choose to move away from something they know if not provided other viable options.  Survey results put Summit Learning Platform at the top, but clear it is a disliked platform. 

Please select your preference regarding the hiring of the Superintendent

·       Hire a new interim superintendent for the remainder of the 2022-2023 school year

·       Continue with already identified co-interim superintendents and hire a superintendent for the 2023-2024 school year

·       No preference

Phrased differently, should Del Garrick and Tina Cassens stay on as interim superintendents, or should we look elsewhere?  Clear why Del and Tina would want to have this question included.  Again, missing any sort of context to the question and the direction that district has been headed in with declining enrollment for 20+ years and subpar test scores. This was another opportunity for the administration to gain a bias response based on who was going to take the survey. 

This survey is flawed to say the least, yet certain individuals continue to site it as a reliable data source for making decisions in the district.  The district would be well served by removing the survey from the website and not using it to make future decisions.  Moving forward, if the district is looking to gather feedback from all stakeholders, they should consider more appropriate methods of writing and deploying a reliable scientific survey.