ufakzeka-karar: An Open Turkish Typed-Decision Model with Order-Invariant Option Scoring
The paper’s own caution is that the released score was not blind after earlier HakemBench results were read.
ufakzeka-karar is a 182M-parameter Turkish typed-decision model that scores fixed answer options without generating text. Its option-scoring head is designed so answers do not change with option order, unlike a sequential baseline that flipped 2.3% to 2.8% of answers under reordering. On HakemBench v1.0’s open set, the released model ranks 7th of 16 with a 0.660 composite, or 6th when only four tracks are counted. The abstract flags that guardrail, moderation and customer-support results were shaped after reading test results. Weights and code are Apache-2.0. ArXiv · AI/CL/LG's note
ufakzeka-karar is a 182M-parameter Turkish typed-decision model that scores fixed answer options without generating text. Its option-scoring head is designed so answers do not change with option order, unlike a sequential baseline that flipped 2.3% to 2.8% of answers under reordering. On HakemBench v1.0’s open set, the released model ranks 7th of 16 with a 0.660 composite, or 6th when only four tracks are counted. The abstract flags that guardrail, moderation and customer-support results were shaped after reading test results. Weights and code are Apache-2.0. ArXiv · AI/CL/LG's note
score 4