Understand Assessment Data with Three Metric Types
Assessment data comes in fast throughout the year and provides multiple opportunities for all stakeholders to learn about the performance of a school. With so many ways to slice the data, it can be difficult to identify the most important stories.
To make sure you get what you need from the final product, first develop guiding questions before you do any analysis. Ask yourself – what do you actually want to learn from the data. Some possible questions:
How are we doing compared to previous years?
Compared to the city or state?
Compared to neighborhood schools?
Compared to schools with similar subgroups?
Compared to accountability benchmarks?
Identifying what questions are most important to your school will help narrow down which data is most useful and drive better, more focused, outcomes.
With your defining questions in hand, your next step is to review the data that is available to help you answer those questions. Below we cover the three broad categories of data types.
1.) Descriptive Metrics: Ultimately, schools are held accountable for aggregate metrics, which are descriptive metrics that define accountability scores. These include metrics such as the percent of students meeting certain proficiency benchmarks, the median of the student growth percentile (MGP), or the percent of students chronically absent.
What percent of your students made this benchmark? How did that number compare to previous years? How do we compare to neighborhood or district schools? What percent of students met their growth goals? Answering these questions will provide a high-level overview of how the school performed and if it is making progress.
2.) Disaggregation Metrics: The next level of analysis is to disaggregate your data by student groups. This provides more detail about how the school is doing that might not be reflected in overall averages. For example, if a school serves a large ELL population, then it might be important to understand how ELL students are doing in the school compared to other ELL students in the city, or in relation to other students at the same school.
You can also look at data by groups that the school creates or student performance levels. For example, you can identify which grade levels are struggling and in need of more resources, or how well chronically absent students are performing on interim assessments. Answering questions at this level provides more detail about which groups are positively or negatively contributing to school performance.
3.) Distribution Metrics: Aggregates and subgroup data are only part of the total picture! If you only know the percent of students meeting a certain metric you might miss a key data insight. For example, two classes could have the same average score, but in one class there is a high percentage of both low performing students and high performing students, and in another class, the performance is spread evenly. The action needed to raise performance is very different in both of these classes. Looking at the number of students at each performance level is one example of how distribution metrics can provide more insights into assessment data.
Understanding these three broad metric types can help you narrow down the type of questions to ask about your school’s data. Think of these metrics as building blocks. Start with getting the high-level descriptive metrics right, then work on breaking down into subgroups. The next step would be to dive into distribution analysis. Following this method can help get to the heart of the story more quickly and design effective interventions to improve performance on overall descriptive metrics.
[This post contributed by Dan Wick, Data Specialist]