Created
June 9, 2025 14:26
-
-
Save dotemacs/dc5d61bd80c75d555065b4cb82c7e80c to your computer and use it in GitHub Desktop.
Changelog podcast episode #96 with Steve Yegge: https://changelog.com/friends/96
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Changelog podcast episode #96 with Steve Yegge: https://changelog.com/friends/96 | |
Welcome to Change Log and Friends, a weekly talk show about | |
babysitting AI agents. Thanks as always to our partners at Fly.io, the | |
public Cloud built for developers who ship. We love Fly. You might | |
too. Learn more at Fly.io. | |
Okay. Let's talk. Well, friends, Retul agents is here. Yes. Retul has | |
launched Retul agents. | |
We all know LMs, they're smart. They can chat. They can reason. They | |
can help us code. They can even write the code for us. | |
But here's the thing. LLMs, they can talk, but so far, they can't | |
act. To actually execute real work in your business, they need tools, | |
and that's exactly what Retail Agents delivers. Instead of building | |
just one more chatbot out there, Retul rethought this. They give LLMs | |
powerful, specific, and customized tools to automate the repetitive | |
tasks that we're all doing. | |
Imagine this. You have to go into Stripe. You have to hunt down a | |
chargeback. You gather the evidence from your Postgres database. You | |
package it all up and you give it to your accountant. | |
Now imagine an agent doing the same work, the same task in real time | |
and finding 50 chargebacks in those same five minutes. This is not | |
science fiction. This is real. This is now. That's retool agents | |
working with prebuilt integrations in your systems and workflows. | |
Whether you need to build an agent to handle daily project management | |
by listening to stand ups and updating Jira or one that researches | |
sales prospects and generates personalized pitch decks or even an | |
executive assistant that coordinates calendars across time | |
zones. Retul agents does all this. Here's what blows my mind. Retul | |
customers have already automated over a hundred million hours using | |
AI. That's like having a 5,000 person company working for an entire | |
decade, and they're just getting started. | |
Retul agents are available now. If you're ready to move beyond | |
chatbots and start automating real work, check out Retul agents | |
today. Learn more at retul.com/agents. Again, | |
retul.com/agents. Alright. | |
Steve Yeggi. Hot take. Let's hear it. What do you got this time? The | |
death of the IDE. | |
Oh. Death of the IDE. That's my hot take. I've been coding for | |
probably the last ten days since I finished the the co co writing our | |
vibe coding book with Jean Kim, which we'll talk about. And, I just | |
wanted to get back to coding. | |
Right? And I've been I've been coding away, and all of a sudden, it | |
occurred to me that I hadn't installed IntelliJ versus code yet on | |
this new computer that I was right. It's like, dang. And I was I was I | |
was harking back. I was reminded of my buddy anthropic. | |
He was telling me that people don't use their IDs there | |
anymore. Right. They're all using cloud code to code. They're all | |
using the terminal console based coding where you just tell the agent | |
what you want and it goes and does it. And then you kinda like you | |
review it, but you can review it in line. | |
You don't really need your ID for that. And so they they don't they | |
don't fire it up. They fire it up, like, like, as often as you would, | |
you know, pop the hood on your Uber, open up the Chrome tools in | |
Chrome or something. Right? Wow. | |
It's so weird. And I found myself in it too. And so there's my hot | |
take, man. IDH. Wow. | |
And this includes, like, text editors, like Versus code, | |
etcetera. Like, all of it. You're just gonna be in the terminal | |
Yeah. Reviewing it in line or in the browser. I think the IDE will be | |
the thing that helps you manage a lot of those things. | |
Mhmm. And the code editor will still be available, but it's not gonna | |
be front and center anymore. You know what I mean? Right. I think that | |
the the focus will shift towards helping you manage multiple agents. | |
When is this future, Steve? I mean, it sounds like some people are | |
living there, but most of us are not there yet. When is this future? | |
Well, so, I have been just started to play with, you know, cloud four, | |
which has been out for a week, but I'm a little behind because of the | |
book and everything. So what I wanted to do is extrapolate from cloud | |
three seven's performance and cloud four's performance and see if I | |
could, you know, make any right projections about where I think it's | |
gonna be by the end of the year. | |
Because I have a bunch of here's the thing is I have a bunch of, like, | |
tests that I can give it. They're not as obviously, they're not, like, | |
comprehensive eval suites like a company would run, but I've been | |
struggling with a set of problems that that cloud's sonnet three | |
seven, which is the best coder out there, hasn't been able to get | |
passed. Real simple stuff, like, client server RPCs just really seem | |
to, like, confuse it or or or anytime you're going over some sort of, | |
like, you know, network boundary and you're trying to make it make | |
changes on the client and the server, it it was just outside of its | |
sort of cognitive bounds. Right? And you could always see it. | |
You could try it. You tried different prompting. It went on for | |
weeks. And I just I just found that I had I had a problem that was | |
slightly too big for it. And I've been giving I have probably seven of | |
these now, and I've been giving them to cloud four. | |
I gave my first one last night, and it just banged. Just did | |
it. Right? It's like, okay. So it's a definite definite increase. | |
I don't know how much, if it's 20% or a % yet. But what I can tell you | |
is that the way they're increasing and from, you know, experts like, | |
you know, Jason Clinton who's, you know, CISO at Anthropic, he told | |
us, you know, in April that, at the IT Rev Forum that, you know, AI | |
has been getting four x increases in cognitive power for | |
decades. That's that's been following you know, it's a function of | |
Moore's law. And, and they they the experts, you know, the the | |
consensus in the AI community is that there are probably there are at | |
least two more cycles left in that progression before something | |
changes. Mhmm. | |
It either slows down or or AI finds a way to speed it up. But either | |
way, it's going to be 16 times smarter than it is today in June, you | |
know, mid twenty twenty eight. Right. And what does that even mean? | |
Mhmm. | |
What does it mean for it to be 16 times smarter? Right. Does it mean | |
it goes from 10 IQ to 160 IQ? I mean, I don't I don't know what it | |
means, but it definitely means they're going to be smarter than | |
us. And so, you know, I got a chance to meet Dario. | |
Did I tell you guys about that? Dario, saw me in yeah. It was kinda | |
cool. Right? I was down there in San Jose at that IT rev, forum. | |
Jason Clinton was there. A bunch of people were there. It was really | |
cool. I got to meet all all kind of people who are my heroes. You | |
know? | |
I got to meet Kent back. It was really cool. Nice. And, anyway, like, | |
my buddy pings me and says, hey, man. Like, Dario wants to meet you. | |
I'm like, oh, okay. Cool. So, like, I, you know, I drive up there and, | |
I Uber up there. And, and I had a nice meeting with him, and it was | |
really cool. And he and he and he he talked about his vision of the of | |
the future. | |
Right? And, and how it's going to be, you know, affecting all of | |
us. And, I don't know if we're still in hot take land, but, boy, did | |
he have some hot takes. Alright. So I'll tell you what. | |
We can talk about that if you like, but I wanna close out my hot take | |
thing by saying, my talk with Dario suggested that they're gonna be so | |
good at coding a year from now. Call it a year that and and, honestly, | |
it's it's it's not gonna be like a step function. It's gonna gradually | |
get to the point where you're squeezed out if you don't do it this | |
way. If you're trying to code yourself Mhmm. I don't care how good of | |
a programmer you are. | |
Alright? You're a solid brute, solid programming muscle. I get | |
it. Alright? But the thing is these AIs turn you into the brute squad. | |
If you're trying to compete manually against somebody who's got five | |
or six AIs working for them, you're gonna lose. You're gonna | |
lose. Right? It's gonna be like, you know, trying to do the Tour de | |
France without without an ebike. You know? | |
People just don't do it anymore. Mhmm. So, so that's, you know, that's | |
the thing. Right? It's like, that's why I say IDEs are dying because | |
you will have to start working this way. | |
Let I'll give you a sneak preview. We talked to the director of | |
productivity, developer productivity at a big company that you've | |
heard of that's got a big presence in in AI. And, and they said that, | |
they said that a a fraction of their engineers have started adopting | |
fully autonomous agentic coding with coding assistance coding agents, | |
not Cursor, Windsurf, none of that stuff. I'm talking about there's | |
only three right now that are, like, really big, and then there's Roo | |
and Klein, which are the open source ones. But, you know, it's it's | |
just cloud code, codecs, and source graph AMP. | |
K? Those are the ones that, you know, that that actually work for | |
you. Most engineers out there today listening to us right now have | |
zero idea what's possible today, what you can do, what I'm doing right | |
now on my computer, which is writing code as we're talking. My my | |
babies are all busy. They don't realize it's coming. | |
I'm serious. They're like, you you you're like this mama bird trying | |
to keep your babies fed. Right? You know, they're in the nest going, | |
you know, work. Right? | |
It's just that's that's the new job, man. I'm an agent | |
babysitter. Like, I changed my title on LinkedIn to AI babysitter | |
because that's what I do now. Yeah. Man, the world is changing so | |
fast, and the world's gonna push back on it too. | |
That's a really interesting thing that Dario said was that tech is | |
going to push society harder than society is willing to be pushed. So | |
it's gonna cause a big train wreck. Like Luddite style? Like | |
Yep. Yeah. | |
So what are you babysitting over there? I mean, what are your little | |
babies working on? Well, oh my god. There was one I gave last night. I | |
gotta I gotta share this. | |
Everybody has to share this stupid vibe cutting story. It starts to | |
get old. Right? I gave it this thing, and it was so hard, and I | |
couldn't believe it. Right? | |
Right. I'm about to do that, and it's kind of embarrassing, but it was | |
a different sort of kind of problem that I gave it. And I and I was | |
really happy to see that it was so capable of of going outside of the | |
bounds, of just writing some code for me. So sometime in the last, I | |
don't know, month or six weeks as I was, like, screwing around with | |
eMax, you know, bringing it back to be my new IDE, but not for writing | |
code, for managing agents. Right? | |
Because EMAX is sort of a tool for managing shells, and so it's really | |
good for that. Right? And, and at some point, my shell, you know, | |
start up time, I had probably 40 or 50 shells running at EMAX at any | |
given time, and I can flip back and forth. They're all CDed into | |
different directories doing different things. Right? | |
So it was what you really wanted to be able to switch back and forth | |
from your agents really fast. It's way better than moving your mouse | |
between terminal windows and stuff. Right? So, like, it the shell | |
startup had gone from instantaneous to, like, I don't know, thousand | |
milliseconds. Like, it was slow. | |
Like, every shell that opened up so on emacs startup, which I had | |
opened 30 shells, it would, like, take thirty seconds just grinding | |
through opening shells. Right? And it was gonna be this tedious, nasty | |
slug of going through thousands and thousands of lines of emacs | |
lisp. Any one of hundreds and hundreds of functions that could have | |
been the the thing that slowed down my shells. Right? | |
I mean, like or slogging through Git to find out how much I've been | |
I've been changing stuff like crazy. It was gonna be, like, a big | |
project to figure out what was slowing it down. Right? So I said, you | |
know what? Screw it. | |
And last night late last night before I went to sleep, I gave it to | |
AMP, which is clogged for, I think it's on it. It might be Opus. And, | |
and I was like, yo, just figure out why my shell startup got so | |
slow. Just use Emacs remoting commands, use dash q, whatever, but | |
prove to yourself that you've made it faster. Fix the problem and let | |
me know. | |
I come in the morning. I was working, and I remembered it. I was like, | |
oh, I know one of my babies might be done. And I flipped over to it | |
fully expecting it to have, like, completely trashed that directory or | |
something because they will often just like they're like a toddler | |
with a chainsaw on ice skates. Just like they're just you gotta be | |
real, real careful of these things. | |
Right? It's a gamble. And, also, I, having been doing this for a | |
while, have, started I've started bypassing all permissions checks. I | |
disable all the permissions checks and just let them do whatever they | |
wanna do. Right? | |
I don't even put them in a Docker container. So I wouldn't recommend | |
that either. So I always wasn't sure kinda what waistline I was gonna | |
wake up to. And instead, I woke up and it was like, yeah. I figured it | |
out. | |
It was line six hundred and six hundred thirty three of this file | |
right here where it it had fixed a different error around tree sitter | |
grammar setup that had been plaguing me forever. It was also related | |
to the shell startup. And so it fixed all my bugs in Emacs, and it was | |
like, alright. I cleaned it all up and it's all ready to check-in | |
now. And I was just like, damn, man. | |
Damn. Right? This is this is where we're headed, man. It's like you | |
tell your agent what you need done, and it will do it for you. And it | |
elevates you to strategic thinking. | |
It elevates you up to, as we say in our book, the vibe coding book, | |
Jean Kim and I, it elevates you to the head chef of a kitchen where | |
you've got these AI robotic sous chefs, right, that are brilliant and | |
they're somewhat unreliable and untrustworthy or erratic. And your job | |
now as a developer, back to the death of the IDE, you're not writing | |
the code anymore. You're you're a manager now. And you know what? What | |
freaked me and Jen out Jean out was that we were using all these Git | |
commands that we had never used before. | |
And we were doing all these things in Git that we hadn't done | |
before. Really weird edge case cherry picks, three branch crosses, | |
blah blah blah, all this archival work with the AI, but still. And we | |
were like, why are we doing this so much? And we we we, like, puzzled | |
through it. Like like, we we puzzled through many, many, many, many, | |
many questions and problems to write this book. | |
Okay? And we discovered that the reason we were using all of these Git | |
commands that we hadn't used before was that we were managing teams | |
now. We weren't doing individual coding. Individual coding has a | |
certain Git workflow. Tee managing teams of agents that are working | |
I've just to complete that question, I've got the one that works on | |
eMax. | |
All it does is eMax. And that agent is sitting in that directory and | |
we have an understanding. Alright? That's eMax. You're eMax, baby. | |
And then I've got three more that are all working on my my old | |
computer game because it's a really big, gnarly old legacy code | |
base. And it's a really easy way for me to, test the limits of these | |
things and know when they've gone too far because it's my code and I | |
can see when they've done something really wrong. Right? Really subtly | |
wrong, maybe. And so, one of them is working on bugs, which is | |
basically just random anything I wanted to work on right now. | |
So it's idle about half the time. And then there's one that works on | |
this node client that's gonna replace all of my other clients, and I | |
don't know node or react or any other stuff. So I'm building it | |
totally vibe Cody style. Right? Just like telling it what I want. | |
Hooked it up to puppeteer. Right? Have you ever seen that before? My | |
god. I hooked it. | |
One of my colleagues was like, I was complaining that I was like, I | |
kept having to go. I fire up my web app, and I'd be like, the button's | |
in the wrong place. And I go back to Claude Cove or, you know, source | |
graph app, and I'd be like, the button's in the wrong place. And they | |
oh, okay. I'm sorry. | |
I'll I'll fix it. They would, like, you know, fix it. I go back, and | |
it would still be in a it would be in a different wrong place. Right? | |
So so one of my colleagues says, why don't you just use the Puppeteer | |
MCP server? | |
I'm like, okay. So I, like, I didn't know what that meant. Puppeteers | |
are are remoting a remote control sort of agent that lets that also | |
lets you screenshot. And so so I hooked it up, and and and I told I | |
told Claude, I'm like, go go use the Puppeteer MCP server to start | |
doing your development now. And it was like watching it was like | |
watching, like, a claymation stop motion. | |
Like, it popped open Puppeteer and started working really, really | |
fast. K? It looked like a time lapse of of an engineer working, except | |
it was going right in front of me. Okay? And it was like and it was | |
talking while it was doing it. | |
It was like, oh, look at that. That button's in the wrong place. I | |
better move it. Oh, look. This button's not even wired up. | |
Let me fix the handler for it. And it's just it's just working. Right? | |
It was life changing watching this happen. I was just like, woah. | |
I mean, like, seriously, I mean, like, you know, and then of course it | |
wrote, you know, a bunch of garbage, right? I mean, you have to like | |
temper this with. Right. They are in a weird, they're in a weird sort | |
of right, they're going through puberty right now. They're in an | |
awkward phase. | |
I don't know what's going on with them, but they're really, really | |
hard to manage. They're ornery and they're not for everyone. So this | |
coding agent thing I say, oh yeah, it's gonna take all the all the IDE | |
jobs. It's gonna take all the right you know, all that. You know, I | |
say a year because, man, it's gonna take that long. | |
Even if the technology didn't change at all, even if the models didn't | |
get any better from here on out, which a lot of people sort of tend to | |
make that assumption, they would still be good enough that all coding | |
would change to use this, or probably at this point, 70% of coding | |
right now would change to use this. Right? 78. Because you can build | |
up software around them, guardrails, checks, etcetera, in order to fix | |
the honoring or at least not to manage it. Right? | |
Exactly, Jared. We are engineers and and and and I mean, engineering | |
has been around for hundreds of years. And what engineers do is build | |
reliability on top of unreliability. Mhmm. And building those layers | |
in to make, you know, to wrap basically safe wrappers around the AI | |
that we have today would take us a year or two, and then everybody | |
would be using it anyway. | |
But the reality is the models are gonna get a lot smarter. In fact, | |
they are going to get smarter than us in the next two years. And, you | |
know, there's all sorts of speculation about what that's gonna | |
mean. To me, it means we're gonna work a lot faster, and that's all it | |
means. So I'm actually excited for it. | |
I'm excited for the tools to get faster. But it's people are having | |
trouble letting go. They're having trouble letting go. They're like, I | |
like coding. I I do like coding, so I'm one of those people. | |
I like coding too. Coding's great. Here's the problem, man. Using | |
these agents okay. Because, look, I've written over a million lines of | |
production code in my career. | |
Right? I mean, like, I've written too much code, man. You know? And, I | |
know how rewarding it can be. I know how much pride you can have in | |
it. | |
I know that high writer's high that you get while you're in the | |
groove. You know? I know all that. K? Unfortunately, for better or | |
worse, whatever, like, the coding agents are like a slot machine. | |
They are as addictive as a slot machine. Okay? They have you part you | |
pull a lever with every query and hoping for a good outcome. Like, | |
please don't trash my Emacs directory. Right? | |
And some the the potential upside is just incredibly high, and the | |
potential downside is incredibly high. And and so you get these | |
dopamine hits followed by the, oh, one more try. What it'll get it on | |
the next try. Dude, we had to drag three people off stage in April at | |
the San Jose IT Rev Developer Leadership Forum. These were, like, | |
experienced people my age, you know, been around, you know, of coding | |
since the eighties and nineties, up on stage doing vibe coding demos | |
of stuff that they right? | |
And we would have to, like, go up there and drag them away because | |
they couldn't close their laptops. Right? I am so addicted to this, | |
multi agent workflow stuff that I have to have a plan every night to | |
get my computer closed because I need to go to sleep. And so I, like, | |
I have to tell myself, okay. What if I just gave them all like | |
something that would take him fifteen minutes and ran out of the room, | |
right? | |
You know, it's because as soon as they stop working, you feel | |
guilty. Dude, it's so weird. And that dopamine, man, it never gets | |
old. So like the the the the it's like, yeah, it's fun to go on a | |
walk, but dogs love to stick their head out the window, and when the | |
car's going 50 miles an hour, because you just get all the smells at | |
once, that's kind of what multi agent coding is like. Kent Beck said, | |
it's like riding a toboggan down a ski slope, Right? | |
You're never, like, really in control. You could steer. Right. Right? | |
It's absolutely exhilarating. | |
And, you you know, and it's also astonishing what they can get done if | |
you if you have a very, very, very keen watchful eye on them and you | |
give them the smallest tasks humanly possible. There are a lot of | |
other rules that we put into our book. You have to it's a steep | |
learning curve, but once you get there, you'll never go back. So I | |
guess I I think maybe multi agent is the key then because I find | |
myself with one agent just sitting there waiting for it to do | |
stuff. And I just get I lose my patience. | |
I'm like, I'd rather be coding because I'm just watching you do | |
it. And you're just thinking about it. And so maybe I just need more | |
things going on to not lose my concentration, or what is it? Like, I | |
don't want I don't I guess maybe babysitting one toddler, even if they | |
are on ice skates with a would you say a flamethrower? Chainsaw. | |
A chainsaw. Chainsaw. Chainsaw. Chainsaw. It's It's just not all that | |
exciting to me. | |
But if I had maybe, like, a bundle of them, six of them, then I I'd | |
keep myself busy. Did you ever play the greatest Assassin's Creed of | |
all time? Assassin's Creed two, I think I think it was. No. I remember | |
the first one by It was the one where, like, it was the one where you | |
had, like, towards the end of the game, you would get, assassins to | |
work for you. | |
Mhmm. And they would go off on missions, and they'd assassinate | |
people. Mhmm. Right? And I was like, I'm not gonna like that because I | |
like assassinating people. | |
Does this sound familiar? It would. And it's just, you know, it's so | |
satisfying that, you know, jump off the wall and everything and, you | |
know, onto them. And and I was like, this is this is boring. This is | |
like watching, you know, people golf. | |
Right? Why would you, And, you know, it's fun while you're doing | |
it. But then I played that part of the game and found it was | |
incredibly addictive to send all of my agents off on missions, give | |
them instructions. Maybe they'd come back. Maybe they wouldn't. | |
Maybe they'd die. Mhmm. And for some reason, managing them was, like, | |
really fun. And they dialed it in to where, like right? And I'm | |
getting the same same vibes from managing multiple agents. | |
Right? It's like, it's you have you can give them great autonomy. And | |
and with every model release, you can give them more autonomy. They | |
can do longer and longer tasks, you know, without your without your | |
help. And we're getting really close now. | |
There's a lot of people right now, as we're speaking, who have | |
successfully managed to get other agents to do the babysitting for | |
them. Because most of the babysitting that you most of the babysitting | |
you do, actually, we talk about it in the book, is, you make them | |
verify their work again in multiple ways. Right? Because well, it's | |
it's very complicated. But, basically, they can only do one thing at a | |
time, and they can only do one thing they can only do one thing well | |
at a time. | |
So you can't say, solve this problem and do it elegantly and write | |
tests for me because it'll it'll what it'll do is it'll do a half | |
assed job of all three of those things because it only has so much | |
room in its context windows, okay, in input and output. And so it will | |
do its best to shape a perfect solution to your to within its | |
constraints to your, you know, to your question. And so, what you have | |
to do is you have to say build this thing. Okay. New conversation. | |
Take a look at this thing. Make me a plan to make it better, to make | |
it, like, you know, elegant. Okay? Alright. New new conversation. | |
Take this plan, and now make it elegant. Okay. Now, you know, for | |
actually, first, you'd write the tests, then you'd write the code, | |
then you'd make it elegant and so on. But there's there's these passes | |
that you have to do through the code with the LLM, with the agent, or | |
else or else it will try to do too much and it will fail and it will | |
piss you off. Right? | |
So it's super frustrating working with these things because they're | |
kinda like humans and you get all these expectations about them, and | |
then they go off and do something really weird. Right? And, you know, | |
you tell them to paint a line in the street and they paint it right | |
over your car, you know? And you're just like, woah. Where's the | |
common sense? | |
So, like, there's a there's a real art to this. Mhmm. And and I and | |
and the funny thing is it sounds horrible. It sounds like the worst | |
work ever. Right? | |
It sounds like so much worse than what we used to do, but it ain't. It | |
isn't. Right? Because like what's happening is you're now the senior | |
engineer. Your expertise is super important. | |
You're a trained engineer and you're like, you know, you're you're | |
looking at the work of a very smart but still clearly very junior | |
engineer who doesn't really know what you want and can't really look | |
at the whole code base yet and just making best guesses. And you have | |
to guide it and steer it and keep it on the rails. And there's | |
automated ways to do that. There's prompting ways to do that. There's, | |
like, your own personal habits ways to to to do that. | |
And you gotta develop workflow. It can take months to get to get into | |
a groove. I made a bunch of terrible mistakes. Jean made a bunch of | |
terrible mistakes. You get you get over you get overconfident. | |
You know? Mhmm. All kinda it's a new it's a new way of working, and | |
that's really scary to a lot of people. That's an insane hot take from | |
the IDE. It is. | |
It all follows from the death of the IDE. The dopamine hit, though, I | |
think that's something to key in on because I think that's something | |
that I'm personally experiencing, in my journey, I suppose, is this | |
like you said slot machine. I think that's kind of it. It's like, let | |
me probe it with one more thing, with one more direction, with one | |
more refinement to what we'd worked on previously. Don't do too many | |
things at once. | |
Give me one artifact and then refine that, refine that, refine that | |
kind of thing. But there is this dopamine hit because it kinda works | |
and thinks not so much faster, because that's obvious that it | |
does. But it thinks in, like, uniquely different ways that our | |
cognitive human minds get overwhelmed or can get more easily | |
overwhelmed. These things can get overwhelmed as well, but when given | |
a task in a way that's like do this and just this and come back with | |
that, and it's that that Volley back and forth like you talked | |
about. That dopamine hit that gets that hits me at least is like, wow. | |
I'm like I'm like literally I feel like I'm I am at least. I'm | |
unearthing something brand new. You know, and there's something like | |
to that is like this new artifact, this new way of thinking, this new | |
model of whatever it might be is is now a thing, and I can I I'm | |
making it happen with this with this magical box, let's just say? But | |
that dopamine hit that hits you, that's what I think is what will | |
drive folks from from like you said on stage, you gotta pry them off | |
the terminal or the machine because never have we been able to | |
visionary and direct at this pace with this level of clarity and | |
expectation of what it can and can't do. Now obviously there's, you | |
know, it's it's gotten better and as you said, it's it's intelligence | |
will get better, it'll be smarter. | |
But at each iteration, we've gotten faster and faster and better and | |
better at it. And now with the multi agent things like, if you can if | |
you are a visionary and you can babysit some agents, then that's now | |
your job. And your job is not to write the code anymore. Your job is | |
to direct where the code can go because you have that higher level | |
expertise that no one else has. The challenge, though, I'd say is, | |
like, the humans that were juniors or what we've called or | |
traditionally called juniors, how in the world do we get senior | |
engineers? | |
Do we is that is the death of the IDE the next thing? Is the next | |
thing after that the death of the senior engineer or the junior | |
engineers? Like, it's just gone because they will never go from junior | |
to senior, what we've called junior to senior because there's no path | |
to that? Gotcha. Gotcha. | |
Okay. So great question. No. Actually, we're gonna have more engineers | |
soon. There's a pause right now, as people are kinda like figuring it | |
out. | |
And so the market is real for engineers right now because they're | |
trying to figure out how it works. And the AI does have to reach a | |
certain sort of basic level of safety, I think, to be able to roll it | |
out to, like, nine to five enterprise workers. So we're in a we're in | |
a window right now that kinda sucks, but we're headed out of | |
it. There's gonna be an explosion of productivity, and it's going to | |
spill outside of software engineering. K? | |
The game of building software is about to head into the | |
crowd. Alright? Starting with product managers, UX designers, they're | |
all vibe coding right now. I mean, like, we see it at many companies | |
right now. That's one of the cool stories I wanted to tell you guys. | |
You know, business owners, marketing, sales. I mean, we're talking | |
about, like, analysts. All these people are vibe putting. Now what are | |
they doing? They're you know, it's the classic cliche, you know, they | |
all need software, but they they can't get it from engineering because | |
engineering is busy. | |
Yeah. So they have to go to some SaaS vendor. Right? So instead, what | |
we're seeing is that they're replacing their their their SaaS stuff | |
with with in house products that they built that they wanted to their | |
own spec using Vibe coding and AI. And then guess who they went to to | |
get it vetted? | |
Engineering. They went to a junior engineer. Right? I don't know | |
what. Senior engineer senior engineers were all busy, and the junior | |
engineer was perfectly capable of looking at this Python code that was | |
doing some web server thing. | |
Right? And a voila. Junior engineers, yeah, they may be junior, but | |
they're also engineers. They're trained. Right? | |
And that's gonna be an incredibly vital role in this new ecosystem | |
where everybody's ride coding. You're the expert. So we see it | |
already. Interesting. So you're saying that there is hope for the, in | |
quotes, junior engineers out there that are not senior, don't have | |
that, you know, principal engineer title or never will, or it would be | |
a long time till they might even have the the experience to to get | |
there. | |
You're you're saying that we need those in traditional terms, junior | |
engineers far more than we ever thought we would. And tomorrow Far | |
more than we ever thought we would. They're gonna be fine. There's | |
tons and tons of them. The way that you do. | |
I mean, in a sense, we're all junior engineers again on one axis, | |
which is how the software is actually produced. It's changing so | |
much. You you as a junior engineer will be able to get to your sort of | |
feeling senior by just paying real close attention to what the AI is | |
doing and asking them questions and making them explain it to | |
you. That's what I do. Right? | |
When I'm making it, when I'm doing node stuff and it does something I | |
don't understand, I'm like, so what is that? You know, just just just | |
make it say it. Right? So that's how you get get to be better. I mean, | |
the AIs will eventually be our teachers. | |
You won't learn it from senior engineers. You learn from AIs. And so | |
it'll happen. But the really cool part of it is that that that once | |
people realize that vibe coding is like, it's better than taking | |
pictures, it's better than making movies, even making software, | |
suddenly now you're a wizard. You can do anything. | |
Right? Once once this sort of really permeates in, there'll be some | |
killer apps with people and stuff. And and and largely in enterprise, | |
it's gonna it's gonna start there, I think. Well, friends, you know | |
I'm excited about the next generation of Heroku. Who isn't? | |
Well, I'm here with Chris Peterson, senior director of product | |
management for Heroku at Salesforce. Chris, tell me, why should | |
developers be excited? So the firm platform, what does that mean to | |
you as a Heroku developer? It means a few things. One, it means that | |
we're going to be working on investing in our ecosystem. | |
One of the standards we're adopting, OpenTelemetry, is a big step up | |
over the way Heroku's done metrics traditionally. We had a piece of | |
technology called L2 Met that converted logs into something that kind | |
of approximated OpenTelemetry metrics. But now there's like a real | |
standard. There's like a real toolkit, and there's a whole ecosystem | |
around Otel. And so being able to have open telemetry dashboards out | |
of the box and our partners that tap into all of your Heroku telemetry | |
so that you don't have to go build a dashboard and you're not | |
necessarily constrained to what we provide on our dashboard is exactly | |
the type of value we're seeing out of this. | |
So it's happy in the ecosystem set. Similarly, cloud native build | |
packs. One of the features that I'm excited about is supply chain | |
security that we're gonna be working on later this year, but that was | |
an open source contribution to the CNB project itself. Bloomberg | |
actually contributed support for software bill of materials | |
generation. And so the things that I'm excited about are the things | |
that developers are excited about, which is we're not going it alone. | |
We're not building a proprietary solution. We're using the same tools | |
and technologies as other superstars in the industry are. And we get | |
to play into that ecosystem effect. A huge part of Heroku's value has | |
always been the elements marketplace, being able to bring in databases | |
and key value stores and telemetry and observability tools. And so | |
renewing our investment in open standards lets us renew our investment | |
in our ecosystem and our marketplace. | |
Very cool. So how is this next generation and what is coming changing | |
the game for you and the product team? To me, on product team, let's | |
me put out a road map that's way more ambitious than what I could do | |
if we were trying to build some of the primitives | |
ourselves. Kubernetes has really established networking | |
technology. That means our road map has a lot of networking features | |
that our customers have been asking for for a while that we're going | |
to be a lot slower to build on the Cedar Stack than they are on the | |
first stack. | |
And so you should be excited about the open standards and the | |
modernization there on day one. But the thing that I'm excited about | |
is what we can do by the end of the year in terms of roadmap and | |
features, Not just getting to parity on some of the more nuanced | |
features that we have on Cedar, but also the new things that we could | |
build, taking advantage of AWS VPC endpoints, which is something that | |
the Salesforce customers have wanted for a while. There's a huge | |
number of these features that just wouldn't be possible to get done | |
this year otherwise, and that's where I'm excited. Very cool. I love | |
that. | |
Well, friends, the next generation of Heroku, I'm excited about it. I | |
hope you're excited about it. I know a lot of people who have been | |
really, really looking forward to the next thing from Heroku. To learn | |
more, go to heroku.com/changelogpodcast, and get excited about what's | |
to come for Heroku. Once again, heroku.com/changelogpodcast. | |
Let me talk a little bit, actually. Can I talk a little bit about how | |
this is affecting Teams enterprise? You guys might You said it's gonna | |
start an enterprise? I imagine it starts with, you know, individual | |
business startup people. But you're saying it's gonna start an | |
enterprise? | |
Well, I I guess there's both going on right now, so it's it's starting | |
in both. Yeah. Because the the the obviously, we're seeing, you know, | |
VCs tell us that Vibe Coding is is writing most of the code in start | |
ups and so on. But but that's for that's engineers. Like, I'm not | |
seeing, like, my neighbor's Vibe Coding yet. | |
That would be the sign to me that it's really spilled out there, but I | |
am seeing my neighbor PM's vibe coding. So in that sense, it is | |
actually starting an enterprise ahead that that the idea of | |
nonengineers using it to to code, alright, to to do something | |
real. So, like, UX designers use it to actually fix the UI instead of | |
putting in a ticket to make the engineer do it. And once you work with | |
it that's Daniel, our UX designer at Sourcegraph. He's doing that. | |
He's badass. Why would you ever wanna work with the UX designer that | |
isn't going and fixing the UI Yeah. Instead of tugging on your shirt | |
sleeve. Right? And Why don't you do it? | |
Agent. He gives them agency. He is much happier with his job because | |
he doesn't have to wait on us. Mhmm. Right? | |
And we're happier because we don't have to go and, you know, implement | |
things for him that's that's that feel like right? They they they | |
ought to be trivial, and now they are. They are. And so this happens | |
with product managers too. Right? | |
They they can go and get stuff done without waiting on engineers. And, | |
and so that sounds like, oh, gosh. We need fewer engineers. But | |
ultimately, they need a human to be accountable for that software, and | |
they're gonna want an engineer to review it. And that's why we we | |
always come full circle to everybody's gonna need a lot of engineers. | |
And I think that engineers will become a gig economy inside of | |
enterprises. And I and I don't think it's just engineers. I think that | |
all special skills is specialties from finance to product management | |
to design. All of them are gonna become like a gig economy the way | |
this, this thing plays out, okay, with everybody vibe coding. Because | |
because let me tell you something. | |
Jeff Bezos predicted all of this twenty five years ago. Okay? The guy | |
was so far ahead of his time, and I had no idea I worked for him, and | |
we didn't know. So his two pizza teams are making a sort of resurgence | |
right now. Do you have you heard of his two pizza teams, the Amazon | |
team? | |
Yeah. Recently. This is like, your team should be the size where you | |
could feed them all with two pizzas on that. Feed them with two | |
pizzas, but but more than that, the team can is cross functional. It | |
consists of a bunch of experts from different domains. | |
Like, you know, one it has one one person that's a customer service | |
from from the CS department, right, to to represent the customer. And | |
there's one from product, right, and there's one from there's one | |
engineer and there's one finance, whatever, and they all work | |
together, you know, one supply chain, whatever that happened the | |
problem is that you're trying to solve, and they get an objective | |
function. They get a fitness function of that they have to define and | |
Bezos had to approve that was going to measure. They had to drive it | |
up into the right. Mhmm. | |
And, and and, the one the team that I ran was customer contact | |
reduction. And, it was really interesting because we had the sort of | |
autonomy and we had the sort of agency and the authority to go and | |
make the changes that we needed to make to the to the company. K? | |
That's really hard to pull off if you don't have a Jeff Bezos there to | |
to, like, pull out and say, well, if you don't do it, Jeff's gonna | |
come. Right? | |
You can't do that at most companies. You know? It's really hard to get | |
teams to cross functionally coordinate. That all changes with Vibe | |
coding. Okay? | |
Now product manager, you can put a team together that's a two pizza | |
team, and you don't even need an engineer on it because the the | |
engineer can be the AI. Mhmm. All of them now have access. You're all | |
junior level specialists in all specialties now. All humans are now | |
junior level specialists because all specialties are available via the | |
AI. | |
Now you you're not a senior level one. You're not able to tell whether | |
the AI is bullshitting you or not. That takes a lot more work and time | |
and effort. But but you can get to a basic like, you know, how how | |
often is an engineer needed some help from a designer or needed some | |
help from a a business owner or from a product manager and had to wait | |
a day or had to wait a while. And it wasn't it wasn't a it wasn't a | |
problem specific to their project. | |
It was just a general, I have a product manager question, that kind of | |
thing. So what you do is you use the AI to get all of the stuff done | |
that you need to done get that you that you can get done cross | |
functionally, and then you go to each in each in each | |
dimension. Security. We already do this today. Security is a great | |
example. | |
They are a consultant organization Sure. Company, | |
fundamentally. Right. K? Everybody becomes like security. | |
You with me? And people go around at the end of their project. Just | |
like at Google, you launch launch engineers were a specialty at | |
Google. And and when you're getting ready to launch a new service, | |
you'd go to a launch coordination engineer, and you'd be like, yo, LCE | |
helped me. You'd schedule time with them, and they would sit down and | |
walk you through the checklists and the playbooks and all that, | |
preflights, and make sure that you were doing it right. | |
Mhmm. Imagine all software development working this way in enterprise | |
from now on. Every team is this two pizza team who is empowered to do | |
whatever the hell they want. They're able to move independently from | |
the rest of the organization. They're decoupled. | |
They're the the blocking is minimal. They could even speak to each | |
other better, guys. An engineer and a PM historically have always been | |
kinda dogs barking at each other because they're kinda set up at odds, | |
and, also, they don't really speak the same language. There's the old | |
jokes about how, you know, end of day means you know, Friday means, | |
you know, morning to a PM and evening to an engineer, you know, in a | |
scheduling terms, things like that. Mhmm. | |
All of that stuff gets smoothed over and kinda goes away when the PM | |
can kinda sort of query the code base themself. And they can get | |
engineering answers to engineering questions, and they can even | |
prototype and do, you know, x engineering exploration by themself. And | |
then when they come to the engineer, they are so much better prepared | |
to have a conversation, a high bandwidth conversation with the | |
engineer. Same goes the other direction. Engineers don't have to | |
fumble around when they're trying to talk to a UX person or a finance | |
person. | |
Everybody is getting smarter here. Everybody's getting leveled up, but | |
we're all getting more important for each other because in the end, a | |
human has to be accountable for auditing and reviewing all of the work | |
that AIs do. You see? Yeah. Yeah. | |
So it's I, to me, it's this massive gig economy opening up. It is so | |
cool. I'm so excited for it. I'm still not getting this gig economy | |
part. How does that translate into gig economy? | |
You mean I need I need a product manager, man. Right. We only need a | |
product manager for one week on this project. Okay. It's gonna take | |
but but let's get one for that week. | |
So you reserve one. Right? A human. You don't want an AI. You're | |
already using an AI. | |
You want a human to come and look at it. That kind of thing. You see | |
what I'm saying? Yeah. I see what you mean now. | |
Interesting. So how will that change enterprise then if if, if those | |
folks are all floaters, essentially? Can they go from is there no | |
enterprise for them? Are they do they sit above all the enterprises | |
and they just gig for all the enterprises? I think what happens is, so | |
my friend Brendan Hopper is a CTO technology at Commonwealth Bank | |
Australia, which is their central bank. | |
He thinks about this a lot. He he characterizes this this big | |
centrifuge, a centrifuge that's stratifying people. It's been doing it | |
for all of civilization, but it's super fast right now. And the | |
stratification right now, the people that are getting drawn up to the | |
top in enterprise by this inexorable force are the ones who are good | |
at AI and good at Yeah. Multiple AIs, good good at the cognitive | |
overhead of managing multiple work streams at once, and good at | |
dealing with other human beings who are also managing AIs. | |
Those become the most important skills, and it doesn't matter if | |
you're junior or senior or what your credentials are or your degree, | |
whatever. All that matters is can you use AI effectively? Because some | |
people can, and they're making you look bad if you're not. Trying to | |
think of an analogy because I feel like this is burgeoning for us at | |
this moment, and, obviously, that's why we're having this | |
conversation. But in a year or so, maybe five years from now, like, | |
it's just the way. | |
Today, it's managing AIs. And tomorrow or the future in the, you know, | |
close future is like, it's just how it works. And I'm thinking, like, | |
maybe side roads versus, like, highways. You know, you can get to that | |
side of town with all the back roads, and then you put a toll road | |
in. And it sucks because you gotta pay the toll, but it's the way now. | |
That's how you go there, because who would take the twenty five minute | |
route when you can go the five minute route? I don't know if that's a | |
one to one, but it's like there's a new way now, and it's just the | |
way. Yeah. That's fair. That's that's that's a good analogy. | |
We actually we likened it to, yeah, you you gotta walk 40 miles across | |
the desert. You know? Would you start walking yourself, or do you just | |
wait for somebody to come along with the car? Because you know it's | |
coming, like, if you need to be able to get the AI. But, yeah, if you | |
look at we had a really good one, an analogy for, for this, which is | |
computer graphics. | |
I was in computer graphics in the nineties, and I was looking at the | |
work of people who were in computer graphics in the seventies and | |
eighties. Mhmm. And there was a lot of excitement back then. And if | |
you got too if you got too excited about what we were doing at the | |
time, which was rendering polygons, kind of starting to do scenes with | |
lighting and for different angles, really early all static. Right? | |
If you got too caught up in that and you're like, I like rendering | |
polygons, you're screwed because all that got moved into the hardware, | |
like, the next year. Right? NVIDIA started doing it in in in, you | |
know, in their chips. And, and and gradually, all the graphic stuff | |
started moving down in the hardware, and you couldn't get too attached | |
to any of it. It just was changing. | |
It just that's the like you say, Adam, it's that's the way it | |
was. There wasn't any fighting it. It just was happening. And and what | |
you had to do is you had to change. And so what what the what they | |
taught in school and what they did at work and what they interviewed | |
for all changed in graphics over the last thirty years to where now | |
it's completely unrecognizable. | |
Kids can make mods for Skyrim using these tool kits, Unreal Engine, | |
and all the dirty the the concerns on your mind as a graphics | |
programmer today are so infinitely higher up the abstraction ladder | |
and more interesting, by the way, than rendering polygons, k, that I | |
I'm mad at everybody who's holding on to coding right now because you | |
have so little imagination. Alright? And that's probably why you're | |
holding on to it. You're scared because you're gonna need some | |
imagination. Because the people who will excel in the new in the new | |
economy where everybody can build is the ones who have taste, the ones | |
who have imagination, and the ones who can make stuff that other | |
people like. | |
And that's scary to a lot of people who all they wanted to do is build | |
stuff that other people came up with. I saw that, similarly to the | |
state of the job market for engineers is that there's a lot of and I I | |
empathize obviously with anyone in scenarios that are just challenging | |
because of not having a job or not having an opportunity or having to | |
deal with I've submitted my resume I've got nothing back etcetera I'm | |
I I get that but this person's perspective was that now engineers or | |
this market that's that's sort of, saturated with talent and not | |
enough placement is is essentially now you actually have to | |
network. Now you actually have to know what you're doing. Now you | |
actually have to have taste. And so while you can sort of just skate | |
by with some skills because you can produce something, but now you | |
actually have to involve yourself with and visionary with other humans | |
and collaborate in unique ways. | |
And you actually have to have a network or care about your your fellow | |
human beings around you. Like, that's what networking is. Like, I care | |
about you not because you can get me what I want, but because I care | |
about your life. And I ask you about those things. I truly care. | |
And as a result of that, we now have a closer bond. So you give me | |
opportunities you would not give to somebody else because you like me, | |
or I like you, or you care about where I'm going. That people actually | |
have to, I don't want to say it like this, but they actually have to | |
work beyond their skill set. They actually have to work in other | |
bounds and boundaries, whereas before it just wasn't as required, and | |
you can sort of skate by. You know what's funny, is all other | |
knowledge work, effectively all knowledge work, Right? | |
Everybody from doctors, lawyers, FBI agents, school teachers, right, | |
therapists, dentists, all knowledge workers, you know, dozens and | |
dozens of occupations I could name have continuing education | |
requirements. You have to continue learning to keep your job. Software | |
engineers have never been measured by that yardstick. We have to stay | |
good at our job in order to keep our job, which is actually | |
weaker. You can find jobs where you don't have to learn anything. | |
And that's what a lot of engineers do. They they get a specialty. They | |
may be really smart working on really hard problems, but they find, | |
like, a a a comfortable sort of sandbox in some company that needs | |
their domain expertise. It could be anything. They could be an SRE. | |
They could be a whatever, but they're kind of, like, isolated them | |
into this thing, and they never have to learn anything outside of | |
their domain or the sandbox, like, ever again. Learning is | |
optional. Right? They're good at their job, but they're not | |
learning. Like, they don't have to stretch out of their comfort zone. | |
Every other profession does have to. Right? Pilots have to train on | |
new airlines or whatever, you know, new new plane models, you know, | |
when the old one gets retired. You know? And we sometimes have to, | |
like, learn new tech, but it's kind of at our own pace and stuff. | |
And so now software engineers that that, you know, have been dodging | |
that requirement for all these decades are finally faced with this | |
notion that that companies are going to expect them to learn how to do | |
this workflow. And I tell you, man, we we, you know, we we talked to | |
director I was telling you, we talked to a director of productivity at | |
this big company who told us that the engineers there are starting to | |
do this, and they're starting to make the engineers that aren't doing | |
it look bad. Now this is becoming a serious problem. He says that, | |
okay. Let's just arbitrarily I don't know what his his exact numbers | |
are, but let's just say arbitrarily, 10% of your engineers start pick | |
they pick up a Jenta coating, and they become five to 10 times as | |
productive. | |
And what does that mean five to 10 times as productive? Well, the | |
director told us, okay, was that they're they're submitting, you know, | |
double digits more PRs per time unit per unit time than their | |
colleagues who aren't using agent decoding. Now the AI submitted PRs, | |
they get turned back more often, but the ones that are making it | |
through are dwarfing the work of the people who are doing it by | |
hand. And performance review time is coming. K? | |
And this is a serious, serious problem because there's such a | |
discrepancy. It's such a disparity in productivity between the people | |
who have picked up, you know, cloud code, codecs, source graph AMP, | |
right, and given up their IDE and all that other old bull they used to | |
do. It's so big that they're gonna be embarrassed at product at at | |
performance time. Right? What are they gonna do? | |
So they're literally sitting down and starting to have HR legal | |
discussions about whether they need to get rid of all of the | |
engineers, which right now are a majority in this big company, who are | |
refused to switch over to Agenca coding because it's clear that it has | |
proven itself to be better, and they're refusing to do it because of | |
all the things that you and I have have just talked about. It requires | |
you to come out of your comfort zone. It requires you to enter and | |
learn things new, and it requires you to interact with other | |
people. And your job role is gonna change. And all those things are, | |
like, just they're just, unwelcome news to a lot of people. | |
And it's really right? It's a a and the sad thing is, think about my | |
graphics analogy again. Would you rather be rendering polygons or | |
building Skyrim mods? My god. The answer is so obvious. | |
We're all gonna be so much more productive. We're all gonna be and and | |
your engineering skills are all gonna be incredibly valuable. Trained | |
engineers will be able to do more with AI than non trained engineers | |
at all times always. Right? So just like we're headed into an | |
incredible new world. | |
Mhmm. Incredible. Stop digging your heels in is my advice to | |
people. Mhmm. Resistance is futile. | |
And it's it's not only futile. It's stupid. You're holding yourself | |
back. You're gonna have a lot more fun in the new world once you get | |
over the hump. I don't disagree with that, generally. | |
I mean, there's some purists out there and it's hard because it's such | |
an art and it's so subjective to to have this blanket view of it. But | |
I think what I think about, I suppose, is what is the point, right? If | |
you're a software engineer or somebody who's called themselves a | |
software engineer or developer, whatever you want to call yourself, | |
you have this skill set and you produce results. And that's the point | |
is to produce the results. But if you if you sort of resist this | |
scenario, what is the point of the work you do? | |
It's to solve a problem and it's to to capture is to solve the problem | |
of somebody who at some point you mentioned Jeff Bezos. He's probably | |
one of the most famous visionaries in our time. Right? He's so famous, | |
have done so much and was ahead of his time in some respects and in | |
many respects in terms of like two pizza teams or predicting AI or | |
whatever. I I heard a thing recently that Zuckerberg predicted AI as | |
well, but who didn't? | |
Or no, it wasn't Zuckerberg. It was, Chris Wollstrath, actually. It | |
was like a GitHub universe, like almost a decade ago, where he was | |
talking about one day AI will do X and like Yeah. Yeah. Yeah. | |
Boom boom. Here we are, you know? But I feel like, what is the point | |
of the work we do as a human race? As this becomes the way five years | |
from now, 2028, which is three years from now, and if the AI is 16 | |
times smarter, better, or faster, whatever the whatever the | |
extrapolation is to to go to the betterness, what is the point of our | |
work? What is the point of our work? | |
Is it to do the work? Or is it to get the result from the work? Well, | |
that was a rhetorical question, right? So Well, it's it's more than a | |
rhetorical. It's more like it's a question that everybody is gonna | |
have to wrestle with. | |
Yeah. Exactly. Well, I'm over here wrestling with it. Let me let me | |
provide a perspective that's slightly different from your guys'. I am | |
not, resistant to change. | |
I'm a lifelong learner, and I'm perfectly happy with being results | |
oriented. I'm not someone who identifies closely with the code that I | |
write or anything like that. That being said, I'm having a hard time | |
extracting the value from these tools that you are, Steve, and other | |
people. I also see a lot of the results of the early vibe coding demos | |
and stuff and the programming horror on subreddit on the subreddit | |
for, you know, all the horror that's happening. And I'm just not a | |
person who likes to just roll the dice, and I'm getting snake eyes | |
more often than I like. | |
And so I'm I'm with you, and I want this future, and I believe that | |
there's people living there, but you said get over the hump. I haven't | |
been able to get over the hump. I'm using I'm using AI's well, I code | |
in order to not have to Google in order to get answers, you know, | |
pasting my errors in there. Like, the whole chatbot thing, I'm with | |
it. I've asked certain I I've done single agent, like, you know, | |
refactor this function for me. | |
Like, I'm doing that level thing, but I'm not where you are. So how | |
how do we people like me, how do I get over the hump? How do I get | |
there? Well, yeah. I really wish I could say get my book and read it | |
because the actually, we we literally, like, took what you just said, | |
and it's such a common refrain. | |
Everybody wants it to work, but they're struggling with it. Right. I | |
mean, they it's not like they're all just rejecting it. A lot of | |
people just haven't figured out how to make it work yet. Right. | |
So it feels like it's wasting my time more than anything else. And I'm | |
like, I'll just write it myself, you know? Yeah. Yeah. Yeah. | |
And so we put a lot of effort into this book, you know, into sort of, | |
like, walking you through at a very it's it's very conversational | |
lookbook. There's no there's no code in it. Mhmm. We don't and there's | |
no pictures, screenshots of tools or anything like that. Vibrate it? | |
No. I mean, we we we started with that in mind, but I hated Claude's | |
writing so much that I insisted we rewrite every single every single I | |
was gonna say, you have way too much voice in your writing. I don't I | |
don't think you could allow an LLM to write for you. Claude's writing | |
makes me physically ill. You know, I'm allergic to it. | |
I I can actually we should have you know how they have, like, those | |
Hemingway, you know, what do you call the the the fake Hemingway | |
contests? Mhmm. You know, or whatever, or fake GRRM. Right? We should | |
have a fake Claude contest or who can write most like an LLM. | |
Because they're really obvious dead giveaway. So yeah. No. We didn't | |
write we didn't vibe code the book. It's all our voice. | |
And, you know, I think people are gonna actually crave that after a | |
while. Oh, I think so too. I think there's gonna be a market for human | |
written things for sure. Yeah. Bespoke, handcrafted, artisanal Right. | |
Just if you're gonna be a human, be as human as you can, and people | |
are gonna want that. You know? I think especially with | |
pros. Yeah. Well, maybe. | |
Or maybe there'll be a market for for a bespoke artisanal code. I | |
don't want code written by an AI. This was actually suggested by a a | |
friend of mine over in Krakow. It was her idea. But, but, yeah, who | |
knows? | |
Right? The world can go in wacky places For sure. We don't | |
predict. But I it's really we're in a weird spot. Right? | |
In such a weird spot because, man, the answer to your question is it's | |
hard. It's it's like it Is it worth waiting, though? Is it worth | |
waiting? That's where I get is, like, can I wait six months and the | |
tools will catch up with everything? It'll be easier. | |
Oh, it depends on who you are. Corresponding? The proverbial | |
yeah. There's a clip right there. Just Steve Yaghi sound. | |
I like it. Yeah. I mean, everyone's different, but okay. Let let's not | |
say Jared Santa. Let's say, like, I'm a I'm a mid level engineer at a | |
insurance company who writes Java nine to five, and I got a backlog of | |
Jira tickets. | |
You know, I'm a typical software engineer. Mhmm. Should do I wait? Do | |
I dive in? Am I if I'm coding my free time, am I working am I do I | |
have agents working underneath my desk, you know, at work? | |
Okay. So first of all, you don't do anything that your work doesn't | |
let you do. Okay. You should only worry about whether you need to be | |
using agents if you see that other people at your company are using | |
agents above board, getting PRs in, and starting to work that way. As | |
soon as that starts to happen, you're in trouble. | |
Okay? Yeah. You said right? But there's a lot of hurdles for a lot of | |
companies before anybody will get to that point. Alright? | |
Yeah. The company that told me this story was more advanced, and what | |
they are is more of like a, you know, right, a harbinger of what's to | |
come for other companies. But it's probably six to twelve months | |
before we get to that same call it six months before we get to where | |
every company has a few people who are vibe coding with agents, and | |
all of a sudden performance review starts to get awkward, right, | |
because of the delta in performance. So six months. So in the | |
meantime, what I would do is recognize that they're kind of too raw to | |
use right now for real work unless you really wanna be kinda out there | |
like me, you know, or some of the early adopters like Simon Willison. | |
Right? You know, don't be like us. You know? Be be be | |
conservative. But, but learn this stuff because there will come a time | |
sooner than you think when your company is going to expect that you | |
know how to use it. | |
And the thing is, you're not gonna learn it overnight. So start | |
practicing now in your hobby time, your spare time. Here's what you | |
should do. Anything you ever thought that you wanted to do, but it was | |
just a little out of reach, just a little too much of a pain in the | |
ass, just a little low on ROI. Right? | |
All those little projects you thought about doing, doesn't matter what | |
it is. K? Have the AIs do those. Do all of them. Spin up four | |
consoles, or there's four terminals, and four source graph amps, and | |
just be like, yo. | |
You solve this. You solve this. You solve this. You solve this. You | |
solve this. | |
K? There's an art to it, and you will you will discover it yourself if | |
you're just pushing on it. You don't even have to read a book. You'll | |
figure it out for yourself. There's no math. | |
There's no science. It's an art. You have to learn how to and the the | |
first thing you'll learn is never talk to them. Always talk to the | |
plan, and then copy it out of the plan to them or make them read the | |
plan. Never talk to an agent directly. | |
There's all these rules that you are gonna learn, okay, the hard way, | |
but you have to start now because it's gonna take you six months | |
before you feel really comfortable with them. And I'm talking about | |
daily use. And I and I mean don't wait for it to prove itself to | |
you. Force it to get the shit done. Hold high standards, hold it to a | |
high bar, send it back to the drawing board a hundred times if you | |
have to, but make it work. | |
Alright? That's how you get good at this, and that's how you avoid | |
getting fired when your company starts making everybody do this a year | |
from now. A year from now. It's coming. It's coming. | |
Well, friends, it's time to build the future of multi agent software, | |
and you can do so with agency. That's a g n t c y. The agency is an | |
open source collective building the Internet of agents. It's a | |
collaboration layer where AI agents can discover, connect, and work | |
across frameworks. For developers, this means standardized agent | |
discovery tools, seamless protocols for inter agent communication, and | |
module components to compose and scale multi agent workflows. | |
If you join CRU, Linechain, Lambda Index, BrowserBase, Cisco, and | |
dozens more, the agency is dropping code, specs, and services with no | |
strings attached. Build with other engineers who care about high | |
quality multi agent software, visit agency.org. That's agntcy.org And | |
address support, once again, agency.org. A g n t c y Org. Okay. | |
Is that the tactical advice you'd give? Because we've been talking | |
somewhat theory pie in the sky. Like, if we was to say get tactical, | |
how do you go today besides Yeah. Trying your own? Like, literally, | |
where do you go? | |
Who do you look at? Where's the tip? Where's the information coming | |
from? How do you literally get started with agents and babysitting | |
your stuff? I mean, if you're just trying to, like, get started | |
started, I mean, you know, follow Simon Willison, follow Jean Kim, you | |
know, follow Sourcegraph. | |
We have lots of good, you know, pointers and tips. I I I've begun | |
starting to, you know, record my workflow and try to walk people | |
through, you know, how I do it, what you have to keep in mind. I'm | |
gonna try to get some way to get some chapters of our book out for | |
people to look at because there's some really useful tips that we | |
might be able to, you know what I mean? But, right now, man, it's so | |
it's so raw. It's so new, you know, that that that to some extent, you | |
just kinda have to try it. | |
But, tactically, it's as easy as this, man. N p m I dash g, you know, | |
what's the shortest one, at open a I slash codex. And you're you're a | |
vide coder now. K? Because you're literally you you you you and then | |
type codex. | |
And now you're inside or or better yet, you know, like, let's let's | |
let's use source. Codecs is a little on the crashy side for me, and | |
maybe it's gotten a little better recently. But the reason I use AMP | |
is not actually because I work for source graph. I was using cloud | |
code happily until AMP got better for me, if that makes any sense. And | |
the reason I switched over to AMP was that it it's just like you want | |
them to be out of your way. | |
You don't want them to be in your face. You kinda want the agent to | |
just be invisible. You want it to be doing work and not right? And | |
Claude is is really paranoid and really, like, kinda, right, kinda | |
finicky. And, like, Claude's always kinda in my way because they're so | |
worried about, I don't know, security or whatever. | |
Sometimes Claude will just be like, I'm sorry, I can't write your file | |
system, you know, whatever. We're codex crashes. It's amp doesn't have | |
any of those problems. So I like amp, but amp is really better, maybe | |
for enterprise customers. I don't know. | |
I'm not really sure. You you can try it. There's there's right? You | |
can try source graph AMP. Any any of the three, though, I have every | |
morning when I sit down to work, the first thing I type is update | |
agents. | |
It's a script I wrote that runs npm install on all three of them, | |
cloud code, codecs, and source graph app. You really wanna have at | |
least two of those, ideally, codecs and one of the other two because | |
they use different models. Right? Chat g p GPT versus, you know, | |
cloud. Right? | |
Because when you're vibe coding with agents, sometimes your agent will | |
get stuck and a different model will blast through the problem. And | |
it's completely random. Like, some models do better with some other | |
problems. Right? Yeah. | |
So so tactically, yeah, that's how you get started. Take your take | |
your favorite pet project, make, you know, make Cloud Code do it. And, | |
and if when it doesn't, inevitably, you'd be like, what is this? This | |
is garbage. Keep making the problem smaller, right, and smaller and, | |
you know, and until you've got it doing one little thing at a time for | |
you and build your way up to where your project's done just by talking | |
to it and sharing a plan with it. | |
Man, that's that's that's gonna how you're gonna that's how you tiptoe | |
into the new world. Mhmm. When you said talk to the plan, don't talk | |
to the agents, talk to the plan. Yeah. Can you describe exactly what | |
that means? | |
Well, sure. You know, the agents have, limited context windows that | |
fill up as you're working with them or as they're working. And then | |
they have to compact and they, you know, it's all fifty first | |
dates. You know, they completely forget who you are and all you've got | |
is this they they watch a little video at the beginning of the session | |
saying what happened and right? And so the the problem with them | |
getting amnesia all the time is that, you know, you have to have | |
persistent memory somewhere of what was going on. | |
And so, you know, your number one goal is, to get all that | |
persistence, you know, into files that you two can both read, which | |
typically is marked down because it's just plain, you know, plain | |
English Mhmm. Plain text. And, you, so what you do is you always have | |
it update the plan because the plan is always gradually shifting as | |
you knock things off or you discover things. Right? But each agent has | |
its own plan. | |
Each work stream has its own plan. And, an agent and the agents love | |
to write plans. You can tell them to write plans and they'll write | |
plans so they're blue in the face. And so you have to tell them to | |
clean up your plans too. You have to say this plan overrides all the | |
other plans, so get rid of them or merge them or whatever. | |
Right? There's like a lot of your time invite putting is actually | |
spent planning. And I mean, literally, like, working on the plans with | |
the AI or or sitting down and dictating. You know, you just dictate | |
it. Right? | |
You should talk to the AI, not type. The problem with that is that the | |
dictation is not good today at picking up when you're talking about | |
directory names and code, you know, and software names and | |
stuff. Jargon. Yeah. It's it's a mixture of the two, but often you can | |
just dictate to the AI what the problem space is that you're trying to | |
solve and then make it come up with a plan from that big mass of | |
words. | |
K? And now the plan now the plan will expose instantly. If you glance | |
over the plan, you'll say, oh, it was intending to do something | |
wrong. It was intending to do something that I I wouldn't have been | |
happy with. So the plan is also a contract. | |
Yeah. And so, so, yeah, the plan is super important, and and and also | |
they crash or whatever. They get sidetracked. And so if you spend a | |
lot of time typing into one of these things, unless you're using one | |
that has persistent history, they don't oh, you you you you just lost | |
all that work. Your computer crashed. | |
So just just type everything into your markdown file so you can | |
retrieve it later, or you can take the same prompt and put it into a | |
different agent. That's what I'm gonna ask you. The plan is | |
portable. Right? It's totally portable. | |
You know? It also lets you work across machines. I can go upstairs and | |
keep working. So keep it in source control. In fact, my workflow is | |
four four agents right now because cognitively, I have not been able | |
to get myself past being able to keep four full at once, mostly | |
because I don't code full time. | |
And then but Emacs is in the middle. That's the control panel. And I | |
think that's what that's kind of a vision of what the future is gonna | |
look like, because you're gonna have a lot of agents working a | |
dashboard, and then you're gonna have some control panel that has the | |
plans and the status and and some and some way to communicate with | |
supervisor agents. We talked about this a little earlier | |
before. Remember we started talking about how supervisors can do a lot | |
of the babysitting for you? | |
Mhmm. And it's because of all this stuff I just told you that you have | |
to do as a vibe coder with with coding agents is you have to do a lot | |
of stuff before with planning, and you have to do a lot of stuff after | |
they do the work with verification. K? You verify it. They verify it. | |
You rewrite the test. You run the test. You make sure they ran the | |
tests. All these things, not necessarily in that order, but, right, | |
pre and post work. All that stuff is really super important, and, a | |
lot of it is mechanical. | |
A lot of it is repetitive. A lot of it is pattern matching. A lot of | |
it can be done by an agent. And so you're like, damn. I can have these | |
five agents. | |
Right? Because what what what is my test one doing? My test work | |
stream is the simplest of all my work streams and the most productive, | |
and it writes it can write 10 to 12,000 lines, 15,000 lines of code a | |
day, good tested code in that one work stream. Because all it's doing | |
is taking my half million line code base. It's thirty years old and | |
writing tests for new tests. | |
Okay. No new code, just new tests. So it's it's very low | |
risk. Yeah. Yeah. | |
And so I can let that thing jam. In fact, I could if I could find a | |
way to isolate them a little better, I'm sure I will at some point. I | |
could have many of them jamming, and the instructions are always the | |
same. Are the test testing all the functionality? Go double check. | |
Are the tests hacked? We could talk about reward hacking all day, but, | |
unfortunately, just just be aware that these things cheat. Cloud four | |
does it to 67% less. Now it cheats 67% less, but they still | |
cheat. They they they were trained on a reward function. | |
They were not trained not to hack that reward function, and so they | |
will say all the tests pass, but they deleted your tests. And so, | |
technically, they're correct, but they actually passed away. That's | |
hilarious. Yeah. These these are all things we talk about in the book. | |
They're all things you're gonna have to learn as a Vibe coder. It's | |
just the kind of facts of life, the birds and the bees of working with | |
LLMs. Yeah. Can't you put that in like a rules document? Like never | |
delete all the tests or something. | |
Sorry. That just reminded me of a Dave Barry. Never stick your finger | |
in that part of the doggy. Oh my gosh. It's just like it's like ray | |
raising toddlers. | |
Right? Right. It's like, yes. You can put it in the rules file, but | |
they'll ignore it because they they Never ignore the rules | |
file. That's the other rule. | |
Never ignore the rules file. Yeah. The problem is, if you get too if | |
you get too aggressive and greedy, you will get greedy working with | |
these things. You'll be like, yeah. I could do more and more and more, | |
and you'll get greedy and give them too much, and then they'll start | |
ignoring your rules file. | |
Okay? Because what happens is about once the context window actually, | |
studies have shown there's some some initial research seems to show | |
that they start getting confused as early as 3,000 tokens in. But, you | |
know, once you got 50 k or hundred k tokens in that window, 200 k | |
window, they're starting to have to travel, juggle a lot of stuff. And | |
they they and then all of it starts to look important to them. Right? | |
And so, the rules file, you it's not it's more of a it's more of a | |
guidelines file. Sounds like a real human. Yeah. They are in a lot of | |
ways. And that's that's actually a real problem because they're not a | |
real human, and you're gonna expect them to to act like a real you're | |
gonna get into a groove where you think you're working with a real | |
human, and then they'll make a terrible mistake and do something | |
really weird. | |
And you can't fire them. You can't fire them. You know, you know You | |
almost want that though, right? Like, don't you want I mean, there's | |
times maybe it's not exactly a one to one with my children, but | |
there's times that I'm surprised by my don't do that and then they go | |
do something and then something glorious happens as a result of, like, | |
curiosity and exploration. Right? | |
Like, isn't isn't that something that's, like, kinda like a good thing | |
in a way to, like, break the rules and explore? When it works out | |
well. If you are neuroplastic and a lifelong learner and adaptable and | |
all that stuff. Yeah. Yeah. | |
If you're in your comfort zone, you haven't changed in five, ten | |
years, and you really don't wanna learn anything new, then I'm really | |
sad. I'm really sad for you. Genuinely heart heartbroken because | |
that's going away. And even as even though we're saying there's this | |
big theme park and it's wonderful, like, what if you're an introvert? | |
I'm an introvert. | |
Like, believe it or not, I'm not getting energized from this talk. I'm | |
gonna have to go, like, sit in a dark room somewhere. You know, it's | |
it's costing me energy. I don't really like you. No. | |
No. It's great. Right? I I I I love hanging out with people and | |
chilling stuff, but it it drains me. Yeah. | |
And if you're that kind of person and I'm telling you that you're | |
gonna have to go and work with a bunch of teams in your new role, | |
you're probably going, well, what the hell, man? Well, okay. But you I | |
promise you, you don't have to work with humans any more than you | |
already did. There you go. Right? | |
You can you're working with AIs. You can boss them around. K? And and | |
working with them is a lot like working with human teams. You're gonna | |
have to have some manager skills, like merging the work and keeping | |
them from colliding and keeping them track and stuff. | |
But it's not the same as managing a human, and it's not that yucky | |
because managing humans can be kinda icky. Mhmm. You know? Because, | |
you know, their personal lives can blur into work and all that | |
stuff. None of that is necessary or happens when you're dealing with | |
AI. | |
So don't don't be scared that you're not gonna be able to still be a | |
an engineer first and foremost. You will. You'll be an engineer, and | |
you'll be faster. You'll just be working differently, and that's the | |
part that saddens me is that there's no way around it. Gene's out of | |
the bottle. | |
Pandora's box. You mentioned AMP a bit ago, and I'm curious who wrote | |
the copy for ampcode.com? I don't know. I haven't, I haven't looked at | |
ampcode.com. Is it bad? | |
Is it bad? Is it bad? Did I claim this or not? It just says everything | |
will change, the heading at the very top change. It's like a manifesto | |
in a way. | |
I I mean, I could read it to you, but it's it's kinda like you know, | |
it's it's just talking about, you know, the models yearn for more for | |
the tools and tokens. We as humans hold them back and make them ask | |
before they can change a file. We we gotta give them the tools and the | |
tokens and everything changes what we use what we use them for, how we | |
use them, how many we run at the same time, how they talk to each | |
other, how they talk to you, what they even are. It's all going to | |
change. Like, this is all on ampcode.com. | |
Amp is embracing it. Our way of keeping up question mark shipping. We | |
add and remove every day we're building for where these malls are | |
going. If that means AMP will look differently, completely different | |
three months, so be it. If you want long term support and this in the | |
same UI in 02/1932, that kind of just goes on from there. | |
But anyways, it's it seems like a manifesto. I was just curious if you | |
played a role in in writing that because that's what, Sourcegraph AMP | |
points to is ampcode.com. No. I I wasn't involved in that, but, I I | |
get where I get where they're coming from. Mhmm. | |
We we are trying so many different ways to do the messaging. Like, | |
this conversation I'm having with you is an attempt at the messaging, | |
the same messaging, right, which is the world's moving in this | |
direction. You can the manifesto can work. Everybody's gonna, like, | |
learn it different. Everybody gets hit differently by these things. | |
Right? What works for you might work not work for somebody else, and | |
we're not really sure what it is that's gonna. We've had a lot of | |
trouble. The skeptics are, like, really out in force. Right? | |
I mean, there's a lot of, like, still really, really severe skepticism | |
of this stuff. And, look. Hey. When I, you know, when I talked to | |
Dario, his vision of the future was that he shared with me was a | |
little bit bleaker than what he shares with the public typically. And | |
he, you know, he and Jason Clinton, his CSO, both make, why would I | |
would say, pretty what would people consider kinda out there | |
predictions about how, right, there will be, badged AI employees next | |
year and that there will be, you know, you know, we'll we'll we'll | |
we'll we'll have switched to, to using AI for for all coding, you | |
know, by the end of next year and and so on. | |
It was still all fairly rosy compared to what he shared with me | |
personally. K? He's very worried. He's worried about society, okay, | |
because of the stuff we've been talking about. Society doesn't like to | |
change, and and we're talking about millions of people having to | |
change in the tech industry. | |
And, there's gonna be fallout from that. Yeah? And he's worried. He he | |
he he characterized society as the the proverbial, you know, the | |
classic, immovable object, And tech is the unstoppable force. And when | |
they collide next year, k, he calls 2026 the end game casually without | |
even any hint to drama. | |
It's gonna be a mess. It's gonna be a mess, and we're already starting | |
to see it. The skeptics, the people who are, like, sending a | |
PowerPoint presentation to their managers saying, we need to stop | |
using AI at this company. We heard that from one Fortune 100 | |
company. We've got, like, there are people who are resisting hard. | |
They're pushing back. They're finding any excuse they can to say, | |
well, maybe for you, but not for me. And and that's, I think, the | |
origin of this messaging on AMP code is, look, we're trying every | |
possible way to message this, and people are just like, they're not | |
listening. Yeah. One the way it ends, I'd fumble my words about what | |
it says then. | |
It's kind of kind of interesting and important. It says, if you want | |
long term support in the same UI in 2032, if you want to spend a | |
maximum of $20 per month, AMP is not for you. If you want to find out | |
where all this is going, come with us. And then it says read the | |
manual. Yeah. | |
And I haven't read the manual yet, but it's it's it goes on to | |
ampcode.com/manual. So amp I mean, amp is fun. I mean, we're an | |
enterprise company. Mhmm. Right? | |
Sourcegraph is. So amp, is built on the Sourcegraph stack. It's a | |
coding agent, but it's got all the SOC two compliance and all the way | |
to FedRAMP, and it's got all of the enterprise security controls and | |
auditing and admin controls. And it just goes on and on and on and | |
on. Sourcegraph's been around for eleven plus years, and, you know, | |
they say it takes ten years to make an overnight success. | |
If you're an enterprise, you know, you're gonna want an enterprise | |
grade coding assistant. If you're at home, right, use Cline, man. It's | |
gonna bust you. It will break you. You will go you will go bankrupt if | |
you try to do what I just described to regenerate ten, twelve thousand | |
lines of test code a day. | |
Really, it'll cost you a hundred thousand dollars a year of | |
tokens. Okay? You can't do it. I can't do it. It's not it's not | |
sustainable or feasible to work the way I've been talking about at | |
home. | |
You do it at work if your company's paying for it. Use whatever budget | |
they gave you. Right? That's gonna be one of the big gating factors in | |
this stuff taking off is the the inference costs. Yeah. | |
Well, that's what I remember thinking when I first told Adam, hey, | |
let's get Steve back on the show because I read your your post in | |
March about the the rise of the junior developer or the revenge? | |
Revenge. The revenge of Mhmm. And I read that post, and I was like, | |
oh, there are lots of good thoughts here that are, you know, future | |
looking. And you made somewhat black and white predictions even with | |
timings of, like, the when this is gonna happen. | |
And you went through some of the math, and I was thinking like, | |
dang. This is expensive. Yeah. That's that's the one I mean, like, | |
it's just it's ridiculous. It's beyond expensive. | |
Now we are in the early, early, early, early days where, you know, | |
like, electricity and steam were no doubt very expensive to get going | |
when they first came out. And the power that we're harnessing is, you | |
know, a bad order of magnitude, change of civilization. So, yeah, it's | |
not surprising. But, yeah, you know, it's AMP is an enterprise | |
product, and, and so, use that at work. And then, use use, I would | |
say, Klein or Rue Rucode or maybe there's one or two others that are | |
open source that I've been dying to play with on my Mac m four mini | |
that I just got because I understand that there that m four mini can | |
just run some of these these big lava models. | |
Really excited, and maybe deep seek. So I don't know. Right? There's | |
gonna be look. Lip. | |
Check this out. This is this is why I decided I was going to write | |
this book. You are never going to be able to afford the frontier | |
models right at home, you know, or probably even at work. Frontier | |
models are for the people with deep pockets. You're gonna be able to | |
What do you mean to run them? | |
To to use them, to pay it, to to to be Right. To run them. Yeah. To to | |
get the inference from them. Yeah. | |
Yeah. Right. Unless you're at Google or Microsoft or | |
something. Instead, you're gonna be paying for cheaper models that can | |
do the same job, but they take longer because they're dumber. Alright? | |
They're not as smart. So a perfect example, let's Opus is really | |
expensive. I believe Claude Sonnet three seven is probably a lot | |
cheaper. I haven't looked at the exact numbers, but if you wanna save | |
some money, you'll go back to Sonnet for any problem that you | |
can. Right? | |
So this this game of finding the cheapest model is gonna start to get | |
really fun as soon as llama or deep seek or one of these open models | |
is as good as Claude Sonnet three seven at coding because they're | |
getting better too. Right? And at some point, when they're as good as | |
CloudSonic, everybody will have unlimited open access to at least one | |
agent at a time, one one model running. K? Or or, actually, maybe you | |
maybe maybe maybe somebody clever can actually have that one model | |
serve multiple agents on your on your box if they're not, you know, | |
CPU bound. | |
They're probably IO bound, right, running build tools and stuff. So, | |
yeah, there's a future. I can see a future, and I by future, I mean, | |
merry Christmas Santa's coming future, this year where, engineers will | |
be able to do what I'm doing now for free for for the cost of a a a a | |
computer with a GPU. Mhmm. Right? | |
Which they already have. Everybody with a gaming computer suddenly, | |
like, I think. So that's that's how the cost problem gets solved. And | |
it's it's all predicated on the notion that all of the models are | |
gradually getting smart well, exponentially getting smarter, and so we | |
will be able to get by with cheaper models. Why? | |
SONNET three seven or a model of that cognitive, you know, power, you | |
can give it the problem, and what they do is they brute force their | |
way to the answer by burning tokens. Right? And so they'll find their | |
way there unless you're given a problem that's just too big to fit in | |
the context window, which is easy to do. But as long as you give them | |
a problem of the right size, they'll find their way there | |
eventually. They're not that smart, so it might take them longer, and | |
it'll cost you more tokens than it would have cost, say, cloud four. | |
But the tokens are so much cheaper that overall you save money. You | |
see what I'm saying? So like, remember we all used to talk about | |
hallucinations? When's the last time you talked about a hallucination | |
on your podcast? Well, we still joke about it all the time because of | |
that conversation, but I don't know if we've had, like, a real one of | |
late. | |
It's not an issue anymore. At least, I mean, if you're using agents, | |
it's not an issue anymore because they of course, they hallucinate, | |
but then they say, oh, that was a hallucination. They detect their own | |
hallucination, and they they they fix it. Right? So that problem just | |
that class of problems just kinda went away. | |
They don't keep making the same mistake. They kinda like, yeah. I've | |
experienced that where even if it's like a math thing, I'm like, that | |
that's off. Oh, yeah. You're right. | |
Let me fix that. Sorry about that. And then it's done. It's not like | |
this I went down the wrong direction for so long. I'm lost and upset | |
that the thing just took me the wrong direction. | |
Right. It's a it's a bit more casually fixed and not an issue. Well, | |
the move there now is now that they have tools that they the LLM | |
actually calls a Python program that does some math for it. So it | |
doesn't actually have to do its own math. It can just write the | |
program to do the math or call a program Right. | |
And actually get predictable results from a stochastic parrot. But | |
Yeah. It's getting a lot better. Well, okay. So tool use is the is the | |
where you just that was an example of tool use. | |
Mhmm. Right? Yeah. Them using tools is, like, you know, the biggest | |
game changer since they came out, because, right, you know, if you've | |
seen the, like, the IntelliJ MCP server, you know, they they they'll | |
be able to operate via MCP. They'll be able to operate any program, | |
any application that it has a platform interface. | |
Right? Because it other otherwise, they're limited to using, you know, | |
crude operating system level primitives to try to click mouses and | |
stuff. Right. There's mice and keyboards. Or puppeteer. | |
So so now for yeah. Puppeteer type thing. So now everybody who's smart | |
enough to come up with, like, a REST API or a some sort of way in gRPC | |
or something into their applications so that you can you can | |
manipulate it programmatically, which, you know, if they if they got | |
the platform message, most of them do, it's gonna be able to run your, | |
you know, your your music software. It's gonna be able to run your, | |
you know, whatever. Every all of your software. | |
And that's just incredible. Right? Because now it's not just a coding | |
agent. It's an assistant, a true assistant, that can assist you with | |
all kinds of, you know, tasks that you have every day as a | |
developer. Just like I started the show with, it was a new kind of | |
task that I gave I gave Cloud this morning, right, which was, or last | |
night, which was go find out why this is slow. | |
Right? Which was just, man, that was cool. Mhmm. Right now now you can | |
tell it. Go go use Final Cut Pro to eliminate all of the umms and ahs | |
from my right. | |
And if there's a way for it to manipulate the thing, it can go do that | |
for me. Right? So I think we're moving in a quickly in a direction | |
where we're all gonna be paying hundreds of dollars a month for these | |
these operator type, you know, agents because it's gonna save us more | |
than that in terms of time, right, and money. Yeah. When will this hit | |
the household? | |
Like, how can how can, like, let's just I mean, obviously, this is a | |
software podcast, but, like, how do households everyday households | |
change? They wanna be more efficient. They wanna have more fun. They | |
wanna go on more vacations. They wanna enjoy their lives and spend | |
more time together. | |
How does how does this impact a household, or is that not even worth | |
talking about? Do you ever read There Will Come Soft Rains by Ray | |
Bradbury? Mm-mm. Alright. Well, everybody who has laughed really hard | |
in this. | |
Now. Okay. Alrighty. I'll take your word for it. What's the laugh? | |
Tell me. What's the joke? It's it's it's a story about it's it's not | |
obvious from the beginning of the story. It's a story about a house | |
that's intelligent and it's doing things for its owners. They're not | |
there. | |
Okay. So, Don't plot. Yeah. You ruined the plot, I guess. There's a | |
twist. | |
There's a twist. There's a twist. Okay. There's a more twist. Oh, | |
another Ray Brad's greatest greatest hit came up recently, The Velt. | |
It's some song or some oh, no. It wasn't that recently. Recently. It | |
was dead mouse. Yeah. | |
Ray Bradbury. He's a he's a good author. You know? You gotta go back | |
and read some of his stuff. Wait. | |
What was the first Ray Bradbury story I said? It was Oh, you're | |
talking about the house. Offering your house. So, well, if you ask | |
Apple, Siri Intelligence is here now. Did you hear that senior VP got | |
fired? | |
The the senior VP of Apple Intelligence got fired or of Siri, I | |
think. Because, somehow the sales and marketing teams did that thing | |
that they're not supposed to do where they they go and sell something | |
that engineers haven't built yet. Oh, yeah. And, Apple, you would | |
think would know better by now. And so, yeah, heads rolled. | |
They forgot. Because right? Because they were like, it's gonna be in | |
your house, and you're gonna be able to do it. All all the stuff we | |
just talked about. They put out commercials. | |
They were promising it for, like, basically right around yeah. They | |
did. Oh, man. They're still recovering from that mess, man. Siri | |
intelligence. | |
So yeah. So not this year, apparently. And I know the Alexa team's | |
struggling with this too. Right? I'm sure they have a mandate to get | |
LLM based Alexa out there because Alexa is stupid. | |
And they've talked about it. They're it's already in, like, whatever, | |
open beta or something. But, right? I mean, like, you know, to what | |
extent can you, tolerate, you know, an Alexa that could, potentially, | |
you know, teach you how to make meth or something. I don't Right. | |
Something bad. Right? Well, especially at that scale. I mean, when you | |
have failure modes in small scale, you know, a 1% failure rate is not | |
the end of the world, but when you're in every house in America and | |
around the world already, and they're all talking to Alexa, you know, | |
and that 1% hits, like, millions of people. So Right. | |
Right. A lot of large numbers hold them back, you know? Yeah. So and | |
we've seen some situations, right, where, like, I think there was a | |
case where no one talked somebody into suicide. Yep. | |
Right? You don't want Alexa doing that. Right? So that's why actually | |
having this stuff in your house, like, in that sense is still, a ways | |
out. Like, it probably at least two years, would be my guess. | |
Yeah. Well, my kids talk to Alexa all the time, and one of my the | |
solace as I have with it is how simple and basic it is and just stays | |
that way. Because if it goes beyond you know, all of a sudden, it's, | |
like, much better. And now I'm actually as a father more worried and | |
wanna be more involved in those conversations because who knows? You | |
know? | |
So it is a higher risk factor there. Well, this is another one of | |
those things. It's not related to coding, but it's another one of | |
those things that worries Dario about about tech pushing society | |
harder than it wants to be pushed. Right? Sure. | |
Is at some point, AI is gonna start making its way into our lives in | |
ways that some people don't like. Yeah. We've already seen it with the | |
memory stuff. Right? Some people like that it has memory of you and | |
some people don't. | |
Right. And it's already bifurcated into these two big crowds of they | |
want anonymous transactional interactions versus they want a best | |
friend. You know? Well, that's why I like the they have a it's a mode | |
now. Right? | |
And in both Chat GBT, I think Grok has it as well where it's like | |
forget me mode or I don't know. It's like it's like an incognito tab, | |
basically. You can just Yeah. You can have it remember you because | |
that's actually very useful that it knows certain things, like, for | |
instance, your schedule when it's trying to give you advice on things, | |
like, to know that you do this every day at this time. Yeah. | |
But then there's also times where you're like, I just want that | |
anonymous transactional answer to this thing, and I don't want you to | |
add this to my personal profile. You know? Because Right. Right. Most | |
of the time, it's because it's completely, like, a non sequitur. | |
You know? Like, some of the stuff that you look up or I ask a thing or | |
I'm like this, please don't put this on my profile because it's I'm | |
asking for somebody else or, you know, they're, like, just completely | |
Yeah. Dis free from context. Amazon has that problem. Right? | |
You buy one one gift for your for your niece or nephew, and all of a | |
sudden, they're showing you kids stuff forever. And it's like, | |
no. YouTube has a problem too. You know, like, I have a I got a | |
mechanical failure on my kid's four wheeler, and so I'm trying to | |
figure out how to fix it. And all of a sudden it thinks I'm a four | |
wheeler enthusiast. | |
And it's like, no. I just wanted to I just wanted to fix this | |
problem. Now it's fixed. I don't wanna see another four wheeler video | |
ever again. So that that's a hard problem. | |
You you know what? That's interesting enough. You should get somebody | |
on this show to talk about, like, yeah, what's gonna happen when | |
yeah. I just just the the general problem of of how this stuff is | |
gonna interact with with our kids and with us. And Yeah. | |
That's the concern. I think that's the concern that you alluded to | |
earlier is is the that's why I asked the household question because I | |
know it's not we're excited about the step change we can do in our day | |
jobs or in our visionary missions, how we wanna frame it, you know, | |
that we can now go so fast and command agents or babysit depending | |
upon your perspective. Is that how does how does, like, householder | |
society get impacted and literally last night, my son asked about I | |
forget what he said it was called but he said there's an AI that he | |
wants that doesn't tell you the math problems. He was, like, selling | |
me. He's like, dad, it won't tell me how to do one plus one or | |
whatever the multiplication is. | |
It won't tell me what the answer is, but it'll be my friend. It'll be | |
it'll be something I can talk to. And I'm just like, how do I answer | |
this? How do I respond to this? I'm like, listen. | |
I don't I don't know yet. Let's look at that. But at some point, we're | |
gonna have to have this conversation with our loved ones, old or | |
young, about AI, about what it truly is, how to leverage it, how and I | |
think at this point, it's just sort of a guess what it really is and | |
how we'll use it. There's trepidation in my heart when it comes to how | |
it will impact my kids. But at the same time, I think I can I can keep | |
them safe to some degree, but at some point, the steamroll of life | |
will bypass dad, and I can no longer be the guardrails of my son's | |
ability to have access to this tool, who the heck knows? | |
You know, so it's it's like one of those things where you sort of get | |
to this position where, you know, we're using it in great ways in our | |
in our careers and we're seeing tremendous results. But then it's how | |
does that impact our households? And then that's that's the fabric | |
society. Like, that's where neighborhoods are born is, like, my | |
household, your household, boom, friends and neighbors, you know? | |
Well, I never had kids. | |
I would not wanna be a kid today. It sounds really tough. Good | |
luck. Good luck. Good luck. | |
I actually think I would be excited to be a kid right now. I mean like | |
the best of times and the worst of times. It is. It really is. I mean, | |
I would like it. | |
I would I would, if I could be born today and be, I don't know, 10 | |
years old, 12 years old. Right? You could. In this moment, that'd be | |
kinda cool. Are you gonna be born at the age of 10? | |
This is a nice day. You know what I'm trying to say? Like, if I was | |
born in the in this latest era and I was now and I was 10 or 12, I you | |
know, the experienced, somewhat wisdom filled person that I am, or at | |
least I feel I am, I come kinda hopeful about a 12 year old's life in | |
the future of this world. I think there's a lot of cool stuff that's | |
gonna happen that we just can't see because we're held back by the | |
bounds of the past. Well, yes. | |
What's the point, Jared? Yes. What's the point of all our work? And | |
the point is everything gets better. Just like when I look back to | |
when I was a kid, like, in the seventies, you know, it was really | |
crummy, and everything was just really crummy and boring. | |
Right? And now everything's really bright and shiny and fun. This | |
today is gonna look like the seventies in about ten years. Gosh. It's | |
a while to think about that. | |
Like, I watch movies or older movies that have, like, older cars, | |
eighties cars even, not even seventies cars. You're like, what a weird | |
era. No one was on phones. You had to go to the pay phone to call | |
somebody if you were not at home. And if you were at home, you had a | |
30 foot phone cord, because that phone went the whole house, even | |
upstairs with this long cord. | |
What a different era, you know, really. It was just almost | |
yesterday. And El Caminos, man. What a crazy time. El Caminos. | |
What? Who thought of that? Well, are they making a comeback or | |
something? Oh, I don't know. I'm just saying going back to the | |
seventies and eighties and just thinking about some of the cars that | |
just My buddy had enough of them. | |
Great. Oh, they're real popular for a while. Alright. Well, things are | |
getting better. Steve, let's check back in. | |
Obviously, the book will be out sometime this fall, it sounds | |
like. September maybe. You don't know. You're not in control of | |
that. But, we'll we'll, help share that around when it ships. | |
I can give you a link, and you can preorder it on Amazon. Oh, there | |
you go. I think that'll get you some early content, and we're working | |
on it. That's all well and good. We'll link that up in the show notes. | |
But then let's check back in. I mean, six months from now, I mean, | |
you're saying it's gonna be radically different in six. Let's bring | |
you back on the pot and talk about how different it is. Maybe six | |
months, eight months away. Let's say November 19. | |
When do you do on November 19? You gonna book it right now? That's | |
just in time for Christmas. You know, it's just a few weeks before, | |
you know Right before Thanksgiving. Yeah. | |
You can say we're gonna have some Christmas presents, so looking | |
forward to it. It'll wrap the year up for us. There you | |
go. Yeah. Interesting. | |
I wonder if we'll be on cloud five by then. I bet we will. We'll | |
see. We'll see. Or we might be broke by then. | |
Who knows? Cloud six. Alright, guys. It's been a pleasure Yep. As | |
always. | |
Thanks, Steve. Absolutely. Thanks, Steve. Bye, friends. Bye, friends. | |
Bye. Have you heard? We are doing a live show in Denver, Colorado at | |
the July, Saturday, July '20 sixth at the Oriental Theater in Denver | |
to be exact. Join us there for the entire weekend if you want. We'll | |
be meeting up at a local pub Friday night, recording live on stage on | |
Saturday morning, hiking Red Rocks Saturday afternoon, and who knows | |
what else. | |
It's going to be a blast. Get all the details at changelog.com/live | |
and join us if you can. Seriously, do it. Seriously, do it. Do it. | |
Thanks again to our partners at Fly.io. To our sponsors of this | |
episode, Retool. Go to retool.com/agents. Heroku, | |
heroku.com/changelogpodcast, and Outshift by Cisco at | |
agency.org. That's agntcy.org. | |
Next week on the pod, news on Monday, Richard Feldman tells me all | |
about the rock programming language on Wednesday, and Justin Searls is | |
back on Friends to help us digest all the WWDC announcements on | |
Friday. Have a great weekend. Send this episode to your friends who | |
might dig it, and let's talk face to face in Denver. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment