0 to 60 most days😭😂

we're so back

actually laughed out loud😂 not what I meant!

for source code: github.com/skaltman/mod...

my feeling is "we'll benchmark whatever the default on the API is, don't make me swim through all of your reasoning settings" but we don't strictly do that atm

They are cheaper and don't do nearly as well on the eval as the other selected models! The -Pro version thinks for a while and does much better. :)

Thanks so much for that comment, that was the wrong model. Updated in the post!

Oh, that's awesome. I'll check it out.

I haven't tried. :)

The crunching of the ~2,000 tok prompt (and thus time to first token) seems to take much longer than the streaming of tokens once it begins! I've been surprised at the tok/s I've been seeing, even without spec dec.

Thanks for the shoutout!

Yes! Maybe I ought to have been clearer that that can be changed--the issue I unknowingly ran into was the value that'd be set to if I didn't set anything myself.