Jev on 770 AITA posts

That objection is raised in the thread and never answered. A commenter says using Reddit as ground truth risks evaluating on data the models may already have seen, which would undercut any accuracy comparison including the headline result.

The author's own summary calls the lead borderline once you account for how many comparisons were made, so the ranking is weak evidence either way. Seven setups over 770 posts is also few trials for a per-model margin.

The author says fast, but not the 40 to 200x range vendors quote. It was measured on one laptop in one evening, and the author asks you to treat the exact factor with caution, so read it as an order of magnitude.

No, which is a commenter's main complaint about the setup. Predicting a subreddit's verdict measures agreement with that community's labels rather than alignment, and nothing in the thread separates generalization from fitting those labels.

770 posts from Reddit's r/AmItheAsshole, where each of seven model setups had to predict the verdict Reddit actually gave. Jev came second of the seven setups.

No. The author states plainly that they have no affiliation with TypeSafe, and the code, logs and method are open so anyone can rerun the test.

A moderator asked the author not to post AI-generated or AI-edited text to Hacker News, citing the site guidelines and noting that some of the account's posts had been classified that way.