New Token-Based Pricing Model for Enhanced AI Training

Key insights

 

 

• Azure OpenAI Service has updated billing for fine tuning to a token-based model, replacing the previous hourly charge.
• New rates for various models include $0.0004/1K tokens for Babbage-002, $0.006/1K tokens for Davinci-002, and $0.008/1K tokens for both versions of GPT-35-Turbo.
• The shift to token-based pricing will significantly reduce costs for some training runs, exemplified by a training session that dropped from $67.50 to $1.08.
• Tools like OpenAI's tiktoken library facilitate easy token counting in training files, making cost estimation straightforward.
• The new pricing structure enables more experimentation and fine-tuning processes, likely enhancing model customization and effectiveness.

 

Exploring the Impact of Azure's Token-Based Billing on AI Model Fine Tuning

Azure's recent update to the OpenAI Service's billing approach marks a significant shift in how developers can manage and optimize AI training costs. By switching from an hourly rate to a token-based system, Azure has made it easier for users to predict and control their expenses, especially as AI model complexities and training needs evolve.

This change addresses a common challenge within the AI development community—predicting the cost of fine-tuning operations. Before, developers had to estimate the duration of training sessions without a clear way to gauge the associated costs accurately. Now, the focus on tokens rather than time allows for precision in planning and budgeting.

Moreover, the reduction in costs under the new model does not only encourage more frequent training sessions but also allows for broader experimentation with different parameters and larger datasets. This flexibility is crucial for developing more effective and specialized AI models that can meet specific user needs.

Ultimately, Azure's revised pricing structure is poised to foster a more vibrant and dynamic environment for AI research and development. By reducing financial barriers and simplifying cost calculations, Azure is enabling developers to push the boundaries of what's possible with AI, enhancing both the scope and quality of AI-driven solutions.

 

 

Read the full article Pricing Update: Token Based Billing for Fine Tuning Training 🎉

 

Pricing Update: Token Based Billing for Fine Tuning Training 🎉 We’re updating our billing for fine tuning with the Azure OpenAI Service to bill based on the number of tokens in your training file – instead of the total elapsed training time.

Model Previous Price New Price

• Babbage-002 $34 / hour $0.0004/1K tokens
• Davinci-002 $40 / hour $0.006/1K tokens
• GPT-35-Turbo (4k) $45 / hour $0.008/1K tokens
• GPT-35-Turbo (16K) $68 / hour $0.008/1K tokens
• GPT-4 $102 / hour $0.080/1K tokens

Why make this change? If you’ve fine-tuned before – or thought about it and decided against it – you may have struggled to estimate the cost. While it’s easy to guess relative time (longer files, more epochs take longer) there’s was no easy way to estimate an exact time. Now, it’s simple: count the number of tokens in your file, multiply that by the per token price, and the number of epochs and you’ve got a decent estimate for the overall costs.

In practice, this change will be a substantial discount for some training runs. For example, recently fine-tuned a GPT-35-Turbo model with a small dataset over 3 epochs. With hourly billing, this job took 1.5 hours and cost $67.50 -- but with the switch to token based billing, it costs just $1.08! The price difference isn’t quite as big for large files.

With these updates, experimenting with fine tuning should be easier than ever: you can easily estimate costs, and with lower and more transparent pricing you can train more models. More training runs allows you to assess the impact of different parameter combinations, data sets and more – and hopefully, create the best custom models for your use case.

Overview of Azure OpenAI Service's Billing Update

The recent updates to the Azure OpenAI Service's billing system represent a significant improvement for developers using AI tools. The move from an hourly rate to a token-based system means costs are now directly linked to the amount of data being processed, rather than the time it takes to process it. This change not only simplifies the estimation of costs but also makes it substantially cheaper, especially for smaller datasets.

The adjustment in billing structure encourages more frequent and diverse experimental training runs. As developers conduct more experiments with different configurations, they can optimize their AI models more effectively. This is crucial for enhancing the performance of AI applications across various scenarios.

Moreover, the introduction of token-based billing reflects a broader trend toward more transparency and flexibility in cloud computing costs. Developers can better budget their expenses and make more informed decisions about resource allocation. Overall, these enhancements to the Developer Tools facilitate a more cost-effective and efficient environment for innovating with AI technology.

 

## Questions and Answers about Microsoft 365

Keywords

Token Based Billing, Fine Tuning Training, Pricing Update, Cost of Token Based Training, Efficient Training Pricing, Billing Updates, Custom Training Cost, AI Model Training Pricing