Why do models get degraded after the initial release
I've been testing new models with varying effort levels, and I have a hunch about why AI labs reduce their new models' capabilities after the initial release.
In the first couple of weeks, the default effort is of the highest quality. After a few weeks, the same model performance moves to the highest effort level (i.e., max, ultra) due to token usage. Most people end up using the default effort level, hence get the feeling the quality went down.
The difference between the low level and the other levels (except the highest) is minimal in practice. You would actually rather use low for the most part and the highest for a few top problems.