--- title: "Ukraine Launches Ukrainian LLM Leaderboard: A National Exam for Neural Networks on Ukrainian Language Proficiency" description: "The Ministry of Digital Transformation and WINWIN AI launched the Ukrainian LLM Leaderboard — the first national ranking of large language models in Ukrainian. The top of the ranking is held by gemma-4, MamayLM, and lapa-v0.1.2-instruct." date: 2026-08-24T14:22:02.000Z lang: en url: https://xab.info/en/posts/ukrainian-llm-leaderboard-national-ai-ranking tags: [ukrainian-llm-leaderboard, artificial-intelligence, large-language-models, ukrainian-language, winwin-ai, hugging-face] publisher: "XAB.info" --- # Ukraine Launches Ukrainian LLM Leaderboard: A National Exam for Neural Networks on Ukrainian Language Proficiency ![Abstract visualization of a neural network made of glowing particles on a black background — a symbol of the Ukrainian LLM Leaderboard, a national exam for language models on Ukrainian knowledge](https://xab.info/media/2026/08/24/ukrainian-llm-leaderboard-natsionalnyi-reiting-ii/ukrainian-llm-leaderboard-natsionalnyi-reiting-ii-1.webp) ## 🎯 Key Points - WINWIN AI under the Ministry of Digital Transformation launched the Ukrainian LLM Leaderboard — the first national ranking of LLMs in Ukrainian - The methodology was developed by KSMU and lang-uk experts (Yuriy Panov, Dmytro Chaplinsky) and was previously applied when training Lapa LLM - Evaluation is conducted across four directions: machine translation, contextual QA, logic and reasoning, and instruction-following - Leaders of the ranking: google/gemma-4-26B-A4B-it, MamayLM (INSAIT), and lapa-v0.1.2-instruct - All results and datasets are open on Hugging Face; plans include modules for images, ethics, and cost The WINWIN AI Competence Center under Ukraine's Ministry of Digital Transformation, together with a group of researchers, has launched the Ukrainian LLM Leaderboard — the first national ranking of large language models that measures their performance specifically in the Ukrainian language. The initiative fills a systemic gap in the industry: until now, global leaderboards tested neural networks primarily in English, while Ukrainian competence was assessed indirectly, through machine translation, which masked specific errors, calques, and failures in understanding local context. Now the state has an objective basis for selecting technological solutions for government applications, and businesses have a tool for evaluating the profitability of integrating artificial intelligence into their product lines. ### Why Previous Approaches Did Not Work The key problem with older methodologies was that model quality in Ukrainian was not tested directly, but through a chain of machine translation. This approach concealed typical defects of the local language: calques from English, errors in grammatical structures, and the model's inability to correctly process specific vocabulary. The new evaluation rules were developed by experts from the Kyiv School of Management University (KSMU) and the lang-uk community — Yuriy Panov and Dmytro Chaplinsky — who have been training neural networks to understand Ukrainian for over a decade. It is separately noted that the same methodology has already been successfully applied when training the domestic Lapa LLM model, confirming its reproducibility and practical value. ### Methodology: Four Directions and Local Datasets The platform measures model effectiveness across several key directions, combining Ukrainian and international datasets adapted to the local context. In the machine translation block, evaluation is conducted at the level of individual sentences and entire paragraphs using the FLORES-200, LongFLORES, and WMT-22 datasets. The contextual question-and-answer section tests reading comprehension on the Belebele and SQuAD datasets. The "logic, knowledge, and reasoning" direction includes tests used by educational institutions and in the ZNO format (ZNO-Eval), as well as complex logical tasks from Winogrande, ARC Easy/Challenge, TriviaQA, and MMLU. Finally, instruction-following is assessed by the accuracy of executing complex requests within the IFEval benchmark. ### Who Is on Top: The Leaders of the Ranking According to the updated leaderboard data, the top positions are held by specialized and optimized models. Among the leaders by average rank, three solutions stand out. The google/gemma-4-26B-A4B-it model, using reasoning algorithms, demonstrates one of the highest scores in instruction-following and logical problem-solving. The MamayLM series (12B and 27B models based on Gemma-3 from the INSAIT Institute) shows strong results in machine translation and logical tests. The Ukrainian-language lapa-v0.1.2-instruct model from the developer community, in turn, occupies one of the best positions in the translation block by FLORES and WMT metrics. All results, codes, and datasets are published openly on the Hugging Face platform, allowing any researcher to reproduce the evaluation. ### Open Code and Future Plans The developers have announced an expansion of the Ukrainian LLM Leaderboard's functionality. The platform will gain modules for testing models' work with images, evaluating the ethics of responses, and analyzing the financial cost of using them in Ukrainian. Particular attention will be paid to the needs of the public sector: the ranking plans to include the assessment of processing regulatory texts, administrative procedures, and the security of handling personal data. Thus, the leaderboard is evolving from an academic quality measure into a full-fledged tool for procurement and regulatory policy in the field of artificial intelligence. ### Significance for the State and Business The practical value of the initiative goes beyond academic interest. For the state, the ranking becomes a transparent basis for selecting technological solutions when developing government digital services, where quality performance in Ukrainian is not an option but a mandatory requirement. For business, the tool allows for an objective assessment of the profitability of AI integration by comparing the accuracy and cost of models within a single methodology. The openness of the code and datasets further lowers the barrier to entry for developers and stimulates the formation of a local ecosystem oriented toward Ukrainian-language artificial intelligence. ## 🔍 Fact-Check Verification - [Ministry of Digital Transformation Forms AI Ranking by Ukrainian Language Proficiency: Who Is on Top](https://www.rbc.ua/ukr/news/mintsifri-sformuvalo-reyting-shi-znannyam-1787580856.html) - Подтверждает запуск рейтинга Минцифры по знанию украинского языка и наличие топа моделей. - [An Exam in Language Proficiency: Ukraine Launches Its First National Ranking of AI Models](https://delo.ua/ru/news/ekzamen-na-znanie-yazyka-v-ukraine-zapustili-pervyi-nacionalnyi-reiting-ii-modelei-471265/) - Подтверждает статус инициативы как первого национального рейтинга ИИ-моделей в Украине. ## ❓ FAQ ### Q: What is the Ukrainian LLM Leaderboard? **A:** It is the first national ranking of large language models, launched by the WINWIN AI center under Ukraine's Ministry of Digital Transformation. It objectively measures the quality of neural networks' performance in Ukrainian, unlike global rankings that are oriented toward English. ### Q: Who developed the evaluation methodology? **A:** The evaluation rules were developed by KSMU and lang-uk community experts — Yuriy Panov and Dmytro Chaplinsky — who have been training neural networks to understand Ukrainian for over a decade. The same methodology was used when training the Lapa LLM model. ### Q: Which models hold the top positions? **A:** Among the leaders by average rank are google/gemma-4-26B-A4B-it with reasoning algorithms, the MamayLM series from the INSAIT Institute, and the Ukrainian-language lapa-v0.1.2-instruct. All results and datasets are open on Hugging Face. ### Q: What is planned to be added to the ranking in the future? **A:** The developers plan modules for testing work with images, evaluating the ethics of responses, analyzing the financial cost of models, as well as assessing the processing of regulatory texts and the security of personal data for the public sector.