Method
Every model gets exactly the same prompts and settings in Season 1, so the scores can be compared with each other. The test itself is in Persian: it measures how well a model reads, writes and builds for Persian speakers.
- 01
Report card (40 points)
20 short questions: logic, street talk and Gen Z slang, literature, arithmetic, following instructions, and traps. Each question is asked 3 times and passes when at least 2 answers are right. Most questions are graded by code, not by a person or an AI.
- 02
Showcase (60 points)
8 interactive builds, each a single HTML file: the Simorgh arcade game, an ant colony, a 3D Azadi Tower, a santur, a Persian landing page, a Persian calendar, girih tiles and a shop. Each is worth 7.5 points and is checked against a list of 9 to 13 items.
- 03
Automated browser
Every build opens in a real browser that clicks and types, takes 4 screenshots, and measures things like errors, sound, frame rate and correct dates in code. The jury cannot overturn these results.
- 04
Jury
Items that need taste or understanding are judged by three jurors from three different companies: Gemini 3.8 Flash, Claude Haiku 5.5 and Jev. Each item is decided by majority and needs at least two firm votes. A juror who is unsure does not vote, and when the jury cannot decide, up to two reserve jurors from other companies weigh in. No juror judges a model from its own company.
- 05
Two runs per model
One run can move by a few points, because each showcase build is made only once. So every model takes the test twice and its official score is the mean of its runs, shown with the range. A model with only one run is marked “provisional”. If two runs are more than 8 points apart, a third run is taken. Each run’s builds and report card are on the model page.
- 06
Cost
The cost next to each model is what the model itself charged to answer the whole test, in US dollars (the Persian pages also show it in toman at the day’s free-market rate). The jury’s cost is kept apart, because it says nothing about the model. If the provider’s server fails mid-answer, the request is sent again and the failure is not counted against the model.
- 07
Mohsen’s take
My own score out of 10, shown apart from the MRMP score; it has no effect on the ranking.
- 08
Private questions
5 questions are never published, so later models cannot learn the answers in advance. For the same reason they are not run on free models.
- 09
Tiers
S from 90 up, A from 80, B from 65, C from 50, and D below 50.