<p>TapTechNews June 5th news, advanced AI models like ChatGPT of OpenAI and GeminiUltra of Google, usually cost millions of US dollars to train them, and this cost is still rapidly rising.</p><p style="text-align: center;"><img src="https://img.taptechnews.com/news/2024/06/1717589934333262.png" /></p><p>As the computing demand increases, the cost of the computing power required to train them is also soaring. Therefore, AI companies are rethinking how to train these generative AI systems. In many cases, these strategies include reducing the computing cost under the current growth trajectory.</p><h2><strong>How is the training cost determined?</strong></h2><p>Stanford University, in cooperation with the research company EpochAI, estimated the training cost of AI models according to the cloud computing rent. The key factors analyzed by both sides include the training duration of the model, the utilization rate of hardware, and the value of training hardware.</p><p>Although many people guessed that the cost of training AI models is getting higher and higher, there is a lack of comprehensive data to support these claims. The <em>2024 AI Index Report</em> released by Stanford University is just one of the rare sources to support these claims.</p><p><strong>The constantly expanding training cost</strong></p><p>The following table shows the training cost of the main AI models adjusted for inflation since 2017:</p><p style="text-align: center;"><img src="https://img.taptechnews.com/news/2024/06/1717589936494160.png" /></p><p>Last year, the training cost of OpenAI's GPT-4 was estimated to be 78.4 million US dollars, much higher than the training cost of Google's PaLM (540B). Google's PaLM was launched only one year earlier than GPT-4, but the training cost was 12.4 million US dollars.</p><p>In contrast, the training cost of the early AI model Transformer developed in 2017 was 930 US dollars. This model plays a fundamental role in shaping the architecture of many large language models currently used.</p><p>The training cost of Google's AI model GeminiUltra is even higher, reaching an astonishing 191 million US dollars. As of the beginning of 2024, this model has exceeded GPT-4 in several indicators, the most notable being winning in the "Massive Multi-Task Language Understanding" (MMLU) benchmark test. This benchmark is an important ruler to measure the ability of large language models. For example, it is known for evaluating the knowledge and proficiency in problem-solving in 57 subject areas.</p><p><strong>Training future AI models</strong></p><p>In view of these challenges, AI companies are looking for new solutions to train language models to deal with the rising costs.</p><p>There are various methods, such as creating smaller models for performing specific tasks, while some other companies are experimenting with creating their own synthetic data to "feed" the AI systems. But so far, there has been no clear breakthrough in this regard.</p><p>For example, AI models using synthetic data sometimes "babble" and cause the so-called "model collapse".</p>