New York's ELA scores soared in 2025 and crashed in 2026. The numbers tell us it may be a mistake.

By Emily Oster

This newsletter typically features state education data releases; today, we’re taking a detour to talk about what is going on with the data from New York. Both because it’s an interesting story, and because it illustrates the value of really paying attention to the data details.

Last week, New York State Education Department (NYSED) released its preliminary 2026 assessment data to schools and school districts. As of this point, only the overall state numbers and New York City details have been made public; we expect full state-level data for 2026 to be released publicly in later this fall. The big headline from our initial look: after very large ELA gains in early grades in 2025, there were equally large declines in 2026.

For example: in grade 3, ELA proficiency rates went from 43% in 2024, to 54% in 2025 and then back to 43% in 2026. At least part of the huge increase in 2025 was publicly attributed to better phonics education, among other things, but the huge reversion in 2026 was attributed to … “we’re looking into it.”

The figure below shows all grades, in NY State and then NY City. The increases and declines are obvious in grades 3, 4 and 5.

The results across NYC schools look very similar – schools consistently increased a lot in 2025, then decreased about the same amount in 2026.

Let me state the obvious: data does not usually look like this. If you’re a reader of this Substack, you see the releases from states as we put them out. Outside of the pandemic, proficiency rates typically move by a couple of percentage points (or less) in a given year. Increases of 4 or 5 percentage points are notable. The huge gains in 2025 here are implausible, and they seem even more so with the decline in 2026.

The fact is, this data looks like someone made- and then fixed- a mistake. So I got curious – could we understand that mistake better? The detailed 2026 data isn’t out yet. But the mistake seems like it was made in 2025. And we do have the technical manuals from 2025 and the detailed data on scores by school, group and district. (Shout out here to the API from our Education Data Center – you, too, can make these graphs in a few minutes).

I dug into the data. Here’s what I learned.

The 2025 Pattern

Here’s a graph of 2025 ELA scores, by district, against 2024 scores for one example – grade 5 ELA.

With a couple of exceptions – largely, very small districts where the data is very noisy – every district increases by about the same amount. It looks like a clear level shift up. And when you divide up the data, you see similar things – there is a similar magnitude increase in the share proficient in the poorest quartile of districts and the richest, for example.

The data has the hallmarks of a simple change in cut scores – this is what it looks like if you just change the score at which proficiency is reached. But NY said they didn’t do that, and the 2026 data reversion suggests they didn’t.

But it also seems clear something happened that wasn’t quite right. Whatever happened here, it seems like it only affected grades 3, 4, and 5 (grade 6 also increased, but largely maintained to 2026) and only ELA scores.

So what was it?

With enough digging in the raw data and the technical files, we can get some sense. As background: every year, NY (and all states) give a slightly different version of their test. But they want these tests to be comparable across years. To make this happen, they begin with a set of anchor questions, which were included as part of the 2022 and 2023 testing.

These anchor questions are used to make the tests comparable across years.

This happens in two steps.

First, in every year, each question – new questions and anchor questions – are calibrated against this year’s students, so the average student sits at zero. You then have an estimate of how hard each question is for these kids. This doesn’t allow for comparisons over time, though.

Second, the system uses a procedure called Stocking-Lord. The anchor questions have been measured twice – once this year, and once in the past when they were banked. The procedure finds the stretch and shift of this year’s distribution that lines up this year’s anchors with the banked values, and then uses that stretch and shift to apply to all questions.

The shift part of this is a single number per grade, and it does two things – first, it credits students with more points if the test is harder, and then it also puts the whole cohort on a permanent comparable scale based on raw scores.

It’s that shift that, effectively, generates the change in test scores. These base scores are then converted to scale scores and to proficiency rates.

Based on the technical documentation and the detailed data, we can see where the error was (very likely) introduced in 2025 – the Stocking-Lord procedure step. The first step – within year difficulty calibration – doesn’t touch the year-to-year variation. And the last step, conversion to scale scores and proficiency, is right. The numbers that are fed into that procedure are shown in the documentation, and they map exactly to the change in scale scores and proficiency rates.

Somehow, in this intermediate step, an error was produced which (effectively) added an extra 0.3 standard deviation to grades 3, 4 and 5 in 2025. The amount added is very close in all three years, although not exact (0.305, 0.306, 0.311). The issue isn’t as simple as “someone added a number” – there are also some odd things in the estimated slopes out of this procedure, so it seems clear that something went wrong in the procedure.

Without more information, it is hard to know exactly what. NWEA is the vendor for the state, and they also do this for other states, which did not have this issue, and the issue seems confined to elementary grades in ELA, suggesting it’s not widespread. The most likely explanation – and here, this is really just informed speculation – is that someone used the wrong version of the anchor file in this procedure. Either an old version, or a trial version, or one with some kind of transformation of the data. This would possibly produce these errors, and assuming it was fixed in 2026, then it would be fixed.

I’m optimistic that the New York State Education Department is going to get to the bottom of this, since it’s definitely a knowable answer.

What’s the lesson?

I would say there are two lessons.

First: the fact that it’s possible to make so much progress on this question even without asking the people who did this is a testament to the power of having the detailed data at hand. And it’s a reason we need to keep making that data accessible and usable. Making the data available holds people to account.

Second: we should try to train ourselves to be more skeptical of results even if we wish they were true. I, as much as anyone, would love it if introducing phonics would increase proficiency rates by 12 percentage points in a single year. But that number isn’t realistic based on what we know about testing. And this should have been figured out, and fixed, last year.

Note: for those of you who want to go deeper, a technical appendix for this post is here.