Earlier today I read an article about ChatGPT getting trounced by Chess on the Atari 2600 at the beginner’s setting.

Shit, I used AI to suggest a summer cocktail recipe and it thought bourbon was something you sliced and used as a garnish - I think we’re safe for now

So a company that is known for cheating on benchmarks to look better organises a benchmark and surprisingly passes it flawlessly… I am shocked!

This shits so stupid. LLMs aren’t good at math. They aren’t meant to be good at math. Stop trying to trick us into thinking theyre good at math. Use them for what theyre meant to do. Good Christ.

And not a single research paper was linked.