Andy Norrie
What’s the cost of an AI image?
Image generation is commonly available right now, via ChatGPT, Bing, and more, but what is the real cost of AI image generation? Can you afford to run an AI model locally?
Starting with chatGPT
We ran a few ChatGPT tests today, and it produced a 1024×1024 image in around 20 seconds after entering the prompt. We can’t measure more than the basic time taken, so we need to run an AI model locally to learn more.
There are limited controls available on this version, but we can setup a similar system in our office that can (in theory) do the same job.
The cost of your own AI
To get our own ‘Quarch’ AI platform running, we took a lab test PC. This is a consumer grade system, running a decent Intel CPU from 2021 and a mid-tier GPU from 2020. The cost to build something like this is probably in the $1000-$2000 range.
For those wanting detail:
Gen5 Intel platform with i7-12900K, RTX 3060 Ti, 32GB RAM and a 2TB Gen5 M.2 SSD]
We used ComfyUI to run an AI image generation model, as it had a reputation for being fairly easy to get running.
The PC is hooked up via our AC Power Analysis Module, for high resolution capture of the power consumption.
With the PC idling in Windows 10, the power consumption is pretty stable, around 60–64 watts.
Generating Images
We used ComfyUI to run an AI image generation model. Given that this PC has a GPU from several years ago, I was expecting it to take a long time to generate an image. It turns out to be remarkably quick, though.
This is the profile for generating 3 images in sequence. We can see that the power rapidly rises to nearly 300 watts and remains there for basically the whole process.
The statistics show that each image takes around 15 seconds to generate; this is far faster than I was expecting. Clearly, ChatGPT is limiting the power used to generate images, as they will be running on vastly newer and more powerful hardware than we have here.
It also shows that image generation can be practical on ‘fairly’ low end hardware.
Each image takes about 3.8 kilojoules in total to generate. That is about 1 watt/hour.
1 watt/hour per image is 1000 images per KWh.
With the electric cost here today being about 35c/KWh, that gives us about 2,857 images for $1
Again, that is lower than I was expecting, although these are fairly simple images we are generating at 1024×1024 resolution.
CPU Image Generation
The big surge in AI recently has pushed NVIDIA shares to new highs. Another (slower) test shows the reason why. Here is the same image generation request, but running on the CPU instead of the GPU. We only ran one image, as it was so slow!
With CPU generation, the power consumption has a different profile, but the average power used was very similar, around 300 watts. There was a clear cycle to the energy use, with 5 second blocks of an increased level, then reducing for 8 seconds.
This is likely to be due to the way the workload is running. The CPU hit 90 °C at times but did not seem to be obviously throttling.
The time taken went from 15 seconds, up to 489 (8 minutes).
The total power used increased from 3.8 kilojoules up to 125.
That’s more than a 30x increase in both time and power consumption . Clearly, the NVIDIA GPU has an enormous benefit in this application.
Conclusion
I was expecting far slower performance from our in-house AI test, but this is actually really impressive to see running so well.
While there is a lot of concern over the energy consumption of AI models, we can see that it likely takes the AI far fewer resources to generate an image than it would for a human. Generating millions of junk images is still going to be a problem, of course, but the actual models are pretty efficient for what they do.
We can also see that the right processing hardware for the job makes an enormous difference.
If you really need an image of a boy with a dragon on his head, this is a pretty efficient way to get it!
Contact your local reseller or email support@quarch.com to get started!
