From DevOps to AI-Native: rethinking software delivery

0:03 The usual joke that I make is that I got really, really, really bored about DevOps. Once people start using Copilot or using more of those AI tools in their software delivery, their kind of role is changing. And I kind of summarize that into four roles, four patterns. Hello everyone, welcome to this new episode of Tech.Rocks podcast. I'm Dimitri Baeli, co-founder of Tech.Rocks and member of the tech staff at Back Market. And I'm delighted to welcome Patrick Debois, advisor and current curator at AI Native Dev Community. Welcome to this series of episodes. It's a series for speakers for the next Tech.Rocks summit, which will be on December 1 and 2nd in 2025. At the Théâtre de Paris. So today I'm fortunate to welcome you, Patrick. How are you?

1:04 Thank you for having me. Et merci, bien, de pouvoir être ici. We'll continue in English, but that's fine. No, it's a real pleasure. I'm doing software factories for 20 years. So I've seen your name, I guess, 2009 and something like that, as inventor of a world. So what it is to invent a world, DevOps. It's too much credit, too much credit. It all happens by accident, so all good. But it's a good accident to invent the word. I'm jealous. But it's for sure more than that. Can you introduce yourself a bit? And we'll discuss later about your book. I think most people know me from my work around DevOps. Like you mentioned, having organized the first DevOps days and kind of subsequent events, I promoted and helped promoting that across the world. Then there was DevSecOps. And I think there's always a new kind of technology change happening in the industry where I like to be a little bit in the chaos and kind of work from there.

2:07 I try to be a little bit of a neutral voice, independent voice, and kind of share my stories and mostly. As my role currently as a curator at the AI Native Dev, it's like seeking out stories of like emerging tech and kind of like bringing it out in the world and learning. And I guess that's what makes me thick is learning and kind of how I do that is learning by sharing. So I get like new ideas back. So hence like ideal to be on a podcast like this. Yeah, and there's a small detail. Where are you from? Ah, je suis de Belgique. Ah, voilà. Intéressant. Europe, based in Europe, contributing to the world. Let's start a bit about the talk. You'll be speaking about AI in the software delivery world. So we know you about DevOps and continuous delivery. all those topics. We all know that AI is around and there's a switch for everybody to go into that topic.

3:09 What made you switch to AI? The usual joke that I make is that I got really, really, really bored about DevOps. And the thing is that if you look back at the number of talks we're having, whether it's continuous delivery and kind of that collaboration, and maybe it's called like platform teams right now, there hasn't been like major leap advancements in thinking. So for me, as I mentioned, the learning is important. So I kind of like stuck. Yes, you know, once in a while you see like a nuance and people are codifying this. And it's still interesting because new people are joining. But for me, it was a little bit more. And then kind of COVID hit. And I was like, what am I going to do? Like, what makes me thick? I explored the metaverse, building digital twins and playing again with new technology. Funnily enough, that led me to automating things in the gaming world. And there was already a little bit of AI. Not like the new AI of generative AI, but there were certain things.

4:10 And I also had once a startup around video and live streaming. So I was thinking about how can I do more of the graphical stuff and kind of automate the graphics. So again, a touch of AI in there. And that kind of all of a sudden there was the boom of generative AI, either from image. And then kind of for more general purposes on GPT. So that's kind of the journey that I went into AI. Since then, I think initially I thought, oh, this is cool. I can finally do AI without having to be like a machine learning expert that has to know a lot of mathematics. This is an integration game, like, you know, cobbling an API, making it work, doing a prompt. All things we're used to already in what we do, like whether it's called an API for this or something else. That was great. The initial focus I had was, oh, I see all these people doing cool stuff. And the first thing there was the Hello World with a chat interface. Everybody was doing this, right?

5:12 How can we actually make this robust? So I started thinking about, hey, how does testing look like for this? Kind of with evals and then how does observability of monitoring of kind of the new things look like? Trying to apply a little bit of the DevOps kind of like principles, but to the new technology, what would it look like? Accidentally coined again another term, open LLM tree, but as a side note, I get into that habit. But then I found that like any new technology, people, their first focus is making it work. It was not making it reliable. It was making it work. So whenever I got into a chat with somebody who was like, how do you do testing? Well, testing, we do that manually. It's fine. We'll do it. We'll do it later. So I didn't get a response on this. Just on that part, because I've met quite a lot of people within platform engineering scope refusing AI, saying it doesn't work, it's not designed and it's not structured enough, and you're the

6:17 ultra opposite. So you say, we have to find out how it works and how to use it. It's non-deterministic and that's kind of the biggest challenge for people kind of making it more reliable is If I was used to writing a test, you say, this is the input, this is the output. But then if I change the prompt with like even a comma or like a dot, it might give me a different result. And then how do I actually express a test of it being okay if I can't do a regular expression of looking for words and so on? So it has its own world, but people have made advancements on this. And then there was obviously the kind of the speed of kind of generating that stuff. So a lot of components. And so. I think my first wave for me personally on the AI, like sat more into AI platform engineering. So I kind of was working at a company at that time. I was just taking the principles of platform engineering, bootstrapping a new team around Gen AI, scaling this out to the developers, helping them with like blueprints,

7:23 a catalog, versioning, test frameworks, and so on, and kind of going from there. So that was, let's say for me, the first six to nine months of being in the AI space. And I learned all the things like RAG and evals and all basic principles that really helped me on getting started. It's still a challenge for everybody, like even security, that too. But what you see, sorry, is that there were a lot more tooling things coming and vendors coming into that space already. So that was a useful kind of thing to learn. So now we need a bit of... of some people to do synthesis about the scope and how to use it. And I guess you're part of that group of people. And that's what I get from the talk you'll be preparing for us in December. What is the scope of your talk? Well, the scope of my talk is my second wave. So first one was platform. Then I looked into what's the impact on the engineering structure a little bit with Conway's law.

8:28 The talk will be around what's the impact in the software delivery lifecycle. So the first one was putting AI inside of your products, how to make that reliable. Second thing, which my talk will be about, is how do you actually build more reliable using AI? So anything from code generation, co-pilots to agents, writing code, reviewing code, and kind of going into the world. So for me, that was a second wave. Like, hey. The testing was all cool, but I didn't have a domain myself. I have no hobbies besides computers and building stuff. So kind of the SDLC was my domain. So I kind of started like, how could it help? And that brought me back also into the automation mode where DevOps was a lot about automating stuff. And now with AI, I could also automate like a lot more with this. So it became very apparent that it was my domain. It was kind of an extension of where I was already doing things and kind of looking what it was there.

9:29 It also took a while for... the tools to catch up. So Cursor, or in Copilot, was probably first, but Cursor made a dent and kind of a surge in those toolings. And that was later, almost like a year later compared to your GPT. But that kind of gave a spark in that whole ecosystem. For me, it was a surprise that it works so well on CodePart. So it's logical after, but it's still a surprise that it works so well. So there's good chances of getting things done with that tooling. How do you summarize a bit the scope from your point of view? You expressed some patterns. Can we go through a bit of those? Sure. So what I noticed that when people start using Copilot or using more of those AI tools in their software delivery, their kind of role is changing. And I kind of summarize that into four roles, four patterns. One is somebody becomes more of a reviewer of the agents writing the code.

10:30 So I call that from producer to manager. You're a manager from all the agents. You look at what they do and you say, yes. It's almost like funny because that's like what ops does. You know, we receive the code, we'll have to push it live. So that was one. But then when we go back, before kind of building it, we want to explain how it needs to be done. So that was like the pattern from implementation to intent. So we're focusing less on the implementation details, but we're expressing like an architect, this is what we want to build more like specifications and diagrams and so on. So once we have that, we can have to know how Sorry, what is the best thing to build? Not how to build it, but what is the best thing? And then we start almost becoming a product owner who says, like, I'm going to discover what actually we need to build. So then you try like four different ways or four different solutions to do things. And you discover what is actually best for the end user, what you actually want. And then eventually you want to turn all that learnings across the different things into knowledge.

11:36 And that's kind of the fourth pattern that I described. So that's kind of where my talks, I will show a lot of examples on how tools are adapting to that pattern change, whether that is from specifications or review or discovering new ways to build the code to gathering knowledge. And that's kind of like the synthesis of my talk. And there will be lots of slides. I will be on averaging like two slides a minute. It's interesting. The thing I have in mind looking at that is you take the development activity. And you split it in four sub-activities that are moving and that can be helped by AI or by a new view of the tooling, and you split it and connect it to the other people we are working with. So it's interesting that it moves the barriers for DevOps. I remind, and I don't know if it's linked.

12:39 It is intentional. So definitely there. Moving barriers between ops and dev and maybe breaking walls. I would say more than moving barriers. It's breaking wall. You broke the wall between dev and ops. Here you're breaking more walls between dev and QA, dev and product. The data, I don't know how it's called. collecting the data, maybe the body, but it's a new thing. So it's interesting that we are breaking barriers with that system. So, okay. I think it's, you know, a typical example is the product owners can now code so they don't have to do the specifications. So like all these things and It feels like the self-servicing that kind of enabled this within DevOps, like I can do certain things with a certain level of automation, kind of now explodes into different fields, as you said, indeed. Well, in preparation of this talk, you submitted a pitch and we discussed a bit. And you said it will be probably too old quickly because everything changes.

13:42 How it is evolving currently? I've given, you know, it's almost like not a state of the union, but kind of like a progressing of the field. And I have to keep adapting slides every time. And like from the early beginnings, imagine AI was the order completion. Then came a chat and we're like, oh, chat, that's great. Oh, then we don't need to copy and paste. So it became chat inside of our IDE. And then it became agents. And then the agents became multiple agents. And then the agents became a swarm. So you see, like, every time you kind of say, oh, we have a nice pattern and I can work with this. Like, it's like a reshake of the way we work. And that's something unique, I think, to this evolution where. I think most iterations in the past were there's a new technology. And you see it refining itself almost like it's, you know, this is how we do things and then kind of immatures. I still feel like the pendulum is being shaken every couple of months on what it is and how we do it.

14:46 Like, will we still have an IDE? We're now heading to CLI. Maybe tomorrow we're going to the cloud. And so that's kind of why it's changing all the time. So it's kind of table flip every two weeks or two months. Is it going in circle or is it going somewhere? I think not yet. I think what's happening is it's in general, everything is grabbing more context. So that's definitely happening. Like, you know, just on, you know, what is in my code? What is in my chat? What is in between my chats? What is like in my ticket system? So that keeps on being expanded. And then the other kind of move is almost like, I'll have an IDE, but then I will have one agent, two agents, five agents. So I can't write it on my laptop. So it's heading to the cloud and then maybe how that's going to work. And then people talk about like Hive and kind of more like MOP agents working together. It's like diverging, but then the practice and the interface also keeps changing.

15:46 No, it's a crazy domain. You're not speaking about productivity. I feel like you're more focused on getting things working, so more on quality. Is that a conscious approach? There's a productivity paradox, but I'll first say... Everybody wants to measure productivity and the ROI of these tools, right? And there's many different ways. And like, you know, I do this task and now I can do it faster. Or I give it that one team and then I compare the two teams. And there's different ways of doing things. And I think it's not that I'm not interested in the productivity, but it is shifting is whatever speed I gain. in the code gen, I might have to spend in the review section. So it's hard to tell from that one thing where it's going and the quality. So I think I'm on the part where I just focus on how it's improving.

16:47 And then we ultimately can kind of talk about like... I really like that approach. You're not speaking about that is really interesting. Not wasting your time on productivity. It's a bit opinionated on my side, but I really like that. Feeding into the naysayers and the kind of the negatives or like AI is doing strange things. And I think my... Way of thinking is I want to almost overuse it to understand what the barrier is. So I know that's not the recommended way, but as a field, we need to kind of understand where it's good, like where it kind of breaks. And that's why I'm more focused on that. But I'll give you one tidbit on like a measurement is. You can say like, we did the thing so much faster, but okay. But what if I just, you know, have it do 20 things in the evening? It doesn't matter. I'm just like in the morning, I'm back. I have them done. So is that faster or not? Yes, it is. But it's not like the hourly increase. And then what are we measuring?

17:47 Is it almost like how accurate does the AI guess what we need to do is way more interesting. To kind of measure that. And I get the question quite often, like, do you actually, you know, Still believe the Dora metrics are important. Yes or no. So I'll briefly touch on that as well. And you answer yes, and then you discuss about something else. That's the fundamentals. Oh, well, we can make it quick. I just usually say, are you still delivering to production? And then they say, yes, okay, Dora metrics apply, done. I was expecting that answer. Yes, it applies. So let's move forward. What is your next question? Thank you. No, it's crazy on the topic here. And so you said you're working less on DevOps and more on software delivery. What the AI is changing on software delivery? I would say as a last question for today. First of all, I want to clarify, I get a question a lot, like what does it mean for DevOps folks? And what I find is that a dev will use it to build software and they can adapt this.

18:53 A DevOps person will likely get the eye through a vendor. They're not the ones building the stuff. So I can't see that much on it. It will happen. But the reason why I focus on the software delivery is if that process changes, the delivery will change. So if I focus now on the delivery with AI of the previous way of doing things, it's not that helpful. Of course, in the intermediate time, it's helpful. But when that kind of dev cycle changes from, you know, not having one agent, like 12 agents doing asynchronous coding, I think the CICD pipeline will completely change. And that's why I'm focusing on that part and the other part will follow automatically and testing will follow as well. But there's a shifting on the system. So you're moving also a barrier here, but not between people. It's more between systems. Yeah, it's almost like what we are to build multiple features at the same time, you know, do our multiple devs working on a CICD system and merging back in.

19:57 That same thing is happening actually on one. developers lab right now by having seven agents doing things in parallel. So in that way, it's almost like doing CICD and merging before it actually hits the central CICD merging. So it's even more left before it actually enters the CICD. We're already building and testing and approving and doing reviews more locally, but it's not the single death. And one of the questions that comes up quite often is, well, if AI created the code, is that my plus one on a pull request? Am I the plus one? And I think, you know, the point is, you still have to do the review. That's kind of like another part. So you can't just say, well, go off and do it and no PRs. But maybe it's indeed not like two people. But that's a matter of how your risk appetite is. If you don't want to take a risk, have seven people review it. If you're okay with the risk and your architecture has been changed to deal with the risk, That's fine, I guess.

21:00 And the way to test also. So yeah, some moving pieces on the system, but also on the ways of working, the tooling. So definitely need someone like you to do a good synthesis. I like the approach about keeping up to date because it's ever changing. So what we are discussing today will be... Caduc, I don't know the word in English for December, but it was really great to discuss with you. review on the topic. I will be there on the Tech.Rocks summit in December, 1st and 2nd of December. I hope you'll be there because you have a talk. I better be, right? Yeah, I know where you live. I look forward to all the feedback from these people and what they're experiencing in their own organizations and whether that applies or maybe conflicts. And that's even more interesting to see why it's not working. I'm sure every CTO is lost about the topic.

22:03 So it's not about opinions and convincing people. It's more enlightening people with your research and topics. One question I get, I bought Copilot for all my developers. Now what? That's basically what I want to answer with my talk. And your talk says buy 10 per developer because they will have multiple agents. So it would be good. Thank you. Please continue your research. We'll be happy to get your learnings by the time. And see you soon. It was the Tech.Rocks podcast, a new episode in the series for the summit. And we'll be happy to meet you really soon in December. Thanks for having me.