January: We’re going AI native, please use AI as much as possible.
March: Everyone needs to attend workshops on how to maximize AI productivity.
July: Some models are only available for special (properly justified) cases. Speak to your manager. Turns out that some people did everything (even commits) with top cost models.
August: Ok, we’re setting some limits on token usage, due to some people spending 1000% over the average.
September: we are creating additional frameworks and limits to AI use.
It took a bit of time for the actual AI cost/benefit to reach the top management and put some hard numbers in contrast with the hype machine, but it’s happening.
Our management is so “forward-looking” that we are still in the March stage: they want AI used everywhere, and added everywhere in the product.
Saying that sometimes you still need to use the brain for big decisions will make some people question your prompting skills publicly.
Hell, we even rebranded the whole company to be an “”“AI company”“”. I hate it there and I sense layoffs coming soon, but finding a job in this period is hard. Wish me luck.
Still in March over here. My management is pushing hard for us to start doing shit in copilot, and I’m humoring them, but I’m looking at current news with a very Ralph Wiggum “I’m in danger” mindset
Everyone who was paying attention even a little bit has known this would be the trajectory from the beginning. And we haven’t even gotten to the crazy price hikes per token that are coming.
And we haven’t even gotten to the crazy price hikes per token that are coming.
Why do you think that?
IMHO, API prices are likely to drop over time as we scale up and efficiencies are found, combined with general pricing pressure from the competition… SOTA models will continue to have SOTA prices, of course, but reasonable priced distilled and open weights models are already common.
A consequence of API prices dropping is that “heavily discounted” consumer plans will also adjust and won’t be so generous relative to API prices.
The biggest problem isn’t cost of running the models right now, the biggest problem is all the debt they built up on training and data center buildouts. That’s what they need to recoup and that’s what’s going to drive prices up. And once it’s recouped they’ll need to make pure profits.
And I hope that debt kills them because all the big AI companies are terrible in one way or another, and the eventual price disparity will make it uneconomical to use their models. Then we might return right back to development focusing on small models, not large ones.
IMHO, API prices are likely to drop over time as we scale up and efficiencies are found, combined with general pricing pressure from the competition… SOTA models will continue to have SOTA prices, of course, but reasonable priced distilled and open weights models are already common.
Some of this seems like wishful thinking to me.
“as we scale up” We already have large data centers, and, if anything, “scaling up” is just driving up expenses for debt and hardware. The real problem with AI, relative to the internet boom, is that the more you use it, the more expensive it is. The economies of scale to be taken advantage of are minimal.
I do think price of compute will come down, but exactly because people will be running their own open source models on their own hardware instead of Sam Altman to stack GPUs in a warehouse, but it will still be too expensive and inaccurate.
And we’re still headed for a financial cliff in this scenario.
Of course they want to turn the corner and ideally keep all their infra, but they might need to restructure their debts if/when the creditors come knocking.
BUT: they have a money printing machine with inference… Enterprise is where most of the money is made, as they pay (closer to) API prices… consumer & small business plans are heavily discounted in comparison, with mostly underutilized use-it-or-lose it plans. They’d be stupid if they were making an opex loss on inference.
As for why API prices are dropping? Competition is a bitch… but they seem willing and able to play the game for now.
Like a fraction of a percent of people will run their own models. It won’t influence price. Remember that coders/tech people are only a minority of users.
Oh, I see. I was mistaken. I thought tech companies were OpenAI’s major customers, and they certainly could make their own data centers for internal work, and that coders–that minority of users–were the only ones likely to pay full price for tokens, the rest of us wouldn´t be willing to pay full fare for a machine that is right most of the time sort of. Thanks for setting me straight on that.
Some large companies do it … but I think you greatly overestimate the willingness of large enterprises to build and maintain their own infra. Once a company has lost the competency (and many have), it’s hard to get it back. There are always pockets of people with the right skillset in a large org, but they mostly don’t want to do DC work anymore. Team/personal servers and beefy laptops, otoh…
They were supposed to do their IPO and have positive cash flow already. Time is running out before investors are getting antsy and there’s no more money available for scaling up.
They gobbled up the easy money… now they have to work harder for more, and yeah, they may have to slow down a bit and embrace fiscal responsibility… I expect to see restructuring of debts along the way, but I’d be very surprised to see any of the major US based players implode.
I feel like we’re co-workers, with how perfectly that timeline matched the day-job’s progression.
We’re at “did you take the seminar on looping that uses tokens like a firehouse so you can use less tokens? Why are you using too little (/not enough) tokens, and why is it not magically 10x-ing all of your (fucking ansible) config?” Or some shit.
C-Suit are herd animals. Same schools, same country clubs, same conference. They all are formed the same. And they don’t take advise from people who earn a few orders of magnitude less.
Soon, the guys upstairs will realise that the real savings are in replacing the middle management with agents, not replacing the workers. Fingers crossed.
I was a PM at a major now-hyperscaler and one of the engineers on my team was pretty vocal about PM not contributing.
He ended up leaving for a team that didn’t have one. A few years later I happened to be in his new building, he saw me, and came up with this huge grin on his face, “oh wow are you joining our team?”
You always hate the chick writing the specs until you don’t have one any more. 😂
January: We’re going AI native, please use AI as much as possible.
March: Everyone needs to attend workshops on how to maximize AI productivity.
July: Some models are only available for special (properly justified) cases. Speak to your manager. Turns out that some people did everything (even commits) with top cost models.
August: Ok, we’re setting some limits on token usage, due to some people spending 1000% over the average.
September: we are creating additional frameworks and limits to AI use.
It took a bit of time for the actual AI cost/benefit to reach the top management and put some hard numbers in contrast with the hype machine, but it’s happening.
Keep spending that shit left and right!
My company is at the July stage. Who knew giving people unrestricted access to Claude enterpise with no training would blow through budgets in days
Our management is so “forward-looking” that we are still in the March stage: they want AI used everywhere, and added everywhere in the product.
Saying that sometimes you still need to use the brain for big decisions will make some people question your prompting skills publicly.
Hell, we even rebranded the whole company to be an “”“AI company”“”. I hate it there and I sense layoffs coming soon, but finding a job in this period is hard. Wish me luck.
Still in March over here. My management is pushing hard for us to start doing shit in copilot, and I’m humoring them, but I’m looking at current news with a very Ralph Wiggum “I’m in danger” mindset
Everyone who was paying attention even a little bit has known this would be the trajectory from the beginning. And we haven’t even gotten to the crazy price hikes per token that are coming.
Why do you think that?
IMHO, API prices are likely to drop over time as we scale up and efficiencies are found, combined with general pricing pressure from the competition… SOTA models will continue to have SOTA prices, of course, but reasonable priced distilled and open weights models are already common.
A consequence of API prices dropping is that “heavily discounted” consumer plans will also adjust and won’t be so generous relative to API prices.
Because these companies started selling at a loss from the start to get people hooked. Neither Claude nor OpenAI are turning a profit right now.
The biggest problem isn’t cost of running the models right now, the biggest problem is all the debt they built up on training and data center buildouts. That’s what they need to recoup and that’s what’s going to drive prices up. And once it’s recouped they’ll need to make pure profits.
And I hope that debt kills them because all the big AI companies are terrible in one way or another, and the eventual price disparity will make it uneconomical to use their models. Then we might return right back to development focusing on small models, not large ones.
And I’m sure you’ve noticed how all the revenue and profit numbers that leak out ignore “one off” costs like training and data centre building.
But look how dangerous these models are. They need to be regulated so that only responsible closed model providers can produce them.
Just ignore that the responsible closed model providers are the ones opening their sandboxes and training their models on blackhat sources.
Some of this seems like wishful thinking to me.
“as we scale up” We already have large data centers, and, if anything, “scaling up” is just driving up expenses for debt and hardware. The real problem with AI, relative to the internet boom, is that the more you use it, the more expensive it is. The economies of scale to be taken advantage of are minimal.
I do think price of compute will come down, but exactly because people will be running their own open source models on their own hardware instead of Sam Altman to stack GPUs in a warehouse, but it will still be too expensive and inaccurate.
And we’re still headed for a financial cliff in this scenario.
Of course they want to turn the corner and ideally keep all their infra, but they might need to restructure their debts if/when the creditors come knocking.
BUT: they have a money printing machine with inference… Enterprise is where most of the money is made, as they pay (closer to) API prices… consumer & small business plans are heavily discounted in comparison, with mostly underutilized use-it-or-lose it plans. They’d be stupid if they were making an opex loss on inference.
As for why API prices are dropping? Competition is a bitch… but they seem willing and able to play the game for now.
If we’re talking about the survivability of AI as a technology, I agree.
If you’re talking about the viability of the hyperscaling business model, I think we are looking at wildly different sets of information.
Like a fraction of a percent of people will run their own models. It won’t influence price. Remember that coders/tech people are only a minority of users.
Oh, I see. I was mistaken. I thought tech companies were OpenAI’s major customers, and they certainly could make their own data centers for internal work, and that coders–that minority of users–were the only ones likely to pay full price for tokens, the rest of us wouldn´t be willing to pay full fare for a machine that is right most of the time sort of. Thanks for setting me straight on that.
Some large companies do it … but I think you greatly overestimate the willingness of large enterprises to build and maintain their own infra. Once a company has lost the competency (and many have), it’s hard to get it back. There are always pockets of people with the right skillset in a large org, but they mostly don’t want to do DC work anymore. Team/personal servers and beefy laptops, otoh…
They were supposed to do their IPO and have positive cash flow already. Time is running out before investors are getting antsy and there’s no more money available for scaling up.
They gobbled up the easy money… now they have to work harder for more, and yeah, they may have to slow down a bit and embrace fiscal responsibility… I expect to see restructuring of debts along the way, but I’d be very surprised to see any of the major US based players implode.
Both Anthropic & OpenAI have debt repayments coming up in 2027 that:
I feel like we’re co-workers, with how perfectly that timeline matched the day-job’s progression.
We’re at “did you take the seminar on looping that uses tokens like a firehouse so you can use less tokens? Why are you using too little (/not enough) tokens, and why is it not magically 10x-ing all of your (fucking ansible) config?” Or some shit.
Do we work at the same company??
I think this was the standard management reaction across all the companies that embraced AI as if it was the second coming of Jesus Christ.
C-Suit are herd animals. Same schools, same country clubs, same conference. They all are formed the same. And they don’t take advise from people who earn a few orders of magnitude less.
Fortunately, the people above them care about numbers, and these don’t add up.
They thought it was their ticket to fire everybody, and the more they made you use it the sooner they could fire you.
Soon, the guys upstairs will realise that the real savings are in replacing the middle management with agents, not replacing the workers. Fingers crossed.
That will just become a different kind of hell
I don’t doubt that for a second, my friend.
I was a PM at a major now-hyperscaler and one of the engineers on my team was pretty vocal about PM not contributing.
He ended up leaving for a team that didn’t have one. A few years later I happened to be in his new building, he saw me, and came up with this huge grin on his face, “oh wow are you joining our team?”
You always hate the chick writing the specs until you don’t have one any more. 😂