Resilience as a compound effect of software engineering and organizational structures

0:03 One of the problems that organizations sometimes have is being too reductive and thinking, oh, I can just tweak this dimension and it will impact all these other dimensions. But hey, guess what? They're all interconnected with each other. And then layered on top of that is what Rachel talked about, the Gerald Weinberg observation, and it's always a people problem. But it's always been a very changeable role, right? Like if you come into, if you don't like learning new things, please pick a different industry. Hello everyone, I'm Alain Buzzacaro, president of Instinct Pionnier, a coaching company. But today it is as a member of the Tech.Rocks curation team that I introduce myself to you. I'm delighted to be with you for this Tech.Rocks podcast, in which I have the pleasure of welcoming two internationally well-known tech leaders, Rachel Laycock, Hello, Rachel. Hi, Alain. How are you? Good. And Neal Ford. Hi, Alain. It's good to see you again after many years. And you both work for ThoughtWorks, that you will tell us a little bit more later.

1:09 And you will be presenting at the next Tech.Rocks Summit 2024 in December. And you will do a keynote that will be very well attended. All right, so let's start with you two. The first question is very obvious. Can you introduce yourself? Tell us what defines you or what drives you. Rachel, do you want to start? Sure. So I'm Rachel Laycock. I'm the Chief Technology Officer for ThoughtWorks. And what defines me is a very big question. But I think the thing that keeps me going and I guess motivates me and makes me get up in the morning to do this role is helping people grow and trying to leave things in a better state than I found them has always been a big motivator for me. So I'm a bit of an instigator for change, even though change is hard. And I then complain about all the change that I'm having to deal with. But it's always been a motivator for me. And I guess that was why I always. like kind of agile. And I think it was an XP practice. Maybe Neal probably will know, like the scouts rule of leave things better than you found them. I think that was the thing that I always got me excited about coding in my early days and agile.

2:14 But other than that, the other thing that gets me up in the morning is usually my kids. They usually wake me up. Then I have to think about, you know, what else I'm doing. All right, we'll dive into that later. How about you, Neal? When you come to see our keynote, Rachel does not normally sound like that. So her kids also give her bugs and make her sound much, much deeper voice than she normally does. So she sounds strange to me today. So thanks for having us, Alain. I've been at ThoughtWorks for almost 20 years, 19 years now. And I have sort of an unusual role. One of my titles is actually a meme wrangler, a meme being a viral unit of thought, like an internet meme, but it actually predates the idea of an internet meme. But that's where I do a lot of, write a lot of books, mostly these days about software architecture. Just published a headfirst software architecture with Mark Richards and Raj Gandhi. I speak at conferences around the world, and I'm a regular host of the ThoughtWorks Technology Podcast. And I still do professional services work as well.

3:15 I do a lot of advisory work for clients. They bring me perspective architectures, and I opine about them, either pro or con, which is good because it gives me a varied schedule, and I'm always doing something different every day. When I wake up, it doesn't, you know, it's not the same old thing every day. It's a brand new adventure every day. Well, actually, you mentioned that we met before, years ago, probably 15 years ago. And I remember when we met, the trendy topic was about agility. I mean, it's still there, but I mean, it morphed a little bit. But what has been important for you since then? What are the main things that you've been working on? You mentioned about architecture. So it's interesting that you mentioned agility because what we used to call agile engineering practices, we pretty much just call engineering practices now. So, you know, agility sort of won the day because it proved itself. You know, you get results when you do that. You know, eventually that kind of flows to the top. So I'm still passionately involved in agility, but mostly about the core reasons why it works, which are things like fast feedback.

4:24 So I'm constantly looking for ways on projects to increase feedback loops, make them faster and make them more robust. And that goes into the thing I've been spending most of my time on since I was talking about agility at that conference in Paris has been software architecture. Microservices came along and showed people that, hey, if it's done correctly, software architecture can be strategic from a business standpoint. And we can actually do things that other businesses can't do. And that revitalized a lot of interest in software architecture as a driver for really a holistic reorganization around. business and product thinking, and that encapsulates the move to the cloud, containerization. And so it's been quite an active adventure over the last 10 years in the software architecture space. It's been over 10 years since we're doing microservices, and it seems like it hasn't slacked off much since then. Welcome back on this. Rachel, you've been the CTO of FortWorks for a year and a half, I believe it is, or a year?

5:28 It's a year, believe it or not. A year. I don't know whether it's gone fast or slow. I can't decide. I'm very curious to know more about it. How did that happen and did it change your life? Yeah. So, I mean, how did it happen? It was kind of a natural progression, really. Like I was doing a lot of different technical leadership roles around ThoughtWorks. I'd been a head of tech in North America, which is like a CTO for a region. I'd run our enterprise modernization platforms and cloud service lines. So closely related to what Neal was talking around, around modernization and architecture and how much that's changing and helping clients get to the cloud and helping clients building new, modern, reusable, like user-centric platforms and things like that. So, yeah, I mean, when the role was offered to me, I knew it was going to be a big change because the scope is so much bigger. So before I was just focused on modernization and platforms and cloud, which is a lot of stuff. But now I have to think about our communities.

6:30 capabilities. I have to think about emerging technology, how, you know, the work that we do with our clients in that space, how can they leverage that? How can we bring like great software practices to that? But even in that space, which I consider like under the scope of technology strategy, I was fairly comfortable. What really threw me off and I guess threw off my life is we also went through a massive reorganization as a business in August, which was about two months into taking on the role. So it was already daunting, like owning the whole tech strategy for the organization, but then having to make even more change than I'd planned for, which meant introducing lots of new roles into the team, but also to support, you know, the new structures in the organization, because as a technology consulting firm and tech being at our core, we need to have technology leaders in all the right places. That's one of our core differentiators. It's what kind of makes us tick. It's what we're about. So I had to think about every new structure in the organization and what role did technology have to play in that structure. And then for people that are in existing roles, helping them settle into and like figure out how to navigate in the new structure.

7:36 And I have to say now I have enormous empathy for anyone that has to do any kind of org design, because I've always said in software and building software, the architecture is difficult and complex and there's always trade-off, but the people problems are always just so much more difficult because it's people. We have different needs and different desires, different motivations. And so when you're changing everybody's job or changing everything, around everybody's job, there's a lot of uneasiness. And managing that was probably the biggest challenge I've had to overcome and not having answers to people's questions, right? And this is something that, you know, Neal and I will talk about a little bit in the talk of how, you know, there is no perfect org structure. So, and there is no perfect role description. And even if you try to be collectively exhaustive, it almost makes it worse. Honestly. And so there's been a lot of lessons learned around that and helping people accept that and say, like, look, you know, just try working in the new structure and then let's kind of iterate on what we bring in some of the practices that we have in software delivery to

8:38 org design. But. Yeah, it's a big thing. I had to learn a lot really quickly about, you know, HR stuff that I didn't know about. You know, there's rules and regulations that are all different in different countries. So when you want to make changes, you can't make them all at the same time. There's different pace. That was the thing that had the biggest impact on my life, because when you you already know you have a steep learning curve coming and then you have to at the same time, it's it's very challenging. Well, and I think it's even worse because the role is CTO, and the CTO at ThoughtWorks is much more proactive than most other companies because it's kind of the holistic technology direction. And while all this other stuff was going on, at least a major technology disruption wasn't impacting our industry at the same time. Oh, wait, yeah, it was. AI was coming along while all this was also happening. So just to make Rachel's life even more interesting. That's a good point, Neal, actually. I'd forgotten about it. That one, because I feel like we've got that under control now. But yes, at the same time, so like last summer and fall, there was three big things going on at the same time of like, we need a new technology strategy.

9:41 AI is disrupting everything. We have to rethink things for ourselves, rethink things for our clients. And then we're going to reorg at the same time. But I guess because the AI one has now settled down where we have a solid strategy and that we're executing on it and we're doing work with clients and we're making progress because that feels more comfortable. I think I've put it. gone about that one. Oh, this is awesome. You're opening so many doors. I don't know where to jump first. But as you mentioned about all the disruption, as you know, the theme of this year for the Tech.Rocks Summit is resilience. Let's talk a little bit about that. And you started to mention that, Rachel. Explain to us what will be the topic of your session. What would you want to share with the audience? Neal, do you want to kick us off? Yeah, yeah. Well, Rachel and I have been talking about this quite a bit lately for obvious reasons. So, and we've been looking at this idea of resilience. We found a great dictionary definition that resilience is the ability for an organism or organization to withstand shocks and disruptions.

10:47 So that's a kind of a team-oriented resiliency. But there's also a software architecture perspective on resiliency, which has to do with availability and reliability and data integrity, et cetera. And we actually attack it from both directions, both as an architectural thing and as a systemic organizational impact. And what we gradually come to the realization is that you can't treat either of those independently. One of the problems that organizations sometimes have is being too reductive and thinking, oh, I can just tweak this dimension and it won't impact all these other dimensions. But hey, guess what? They're all interconnected with each other. And then layered on top of that is what Rachel talked about, the Gerald Weinberg observation. And it's all. It's a people problem, no matter what other things that you're working on. And so we do a deep dive on what does it really take in today's world to create a resilient organization, particularly when you get disruptions like we mentioned before, like AI. That's a really, really huge disruptive force.

11:48 How do organizations, so that's going to permutate your path forward. How resilient are you as an organization and how can you correct your path and incorporate the good parts of your organization, but incorporate the new stuff at the same time? That's a challenge that all these organizations are facing. And hopefully we have some advice based on lived experience, part of what Rachel's been living through. But we've been thinking about this for a while. So I think we have some observations and advice about this resiliency topic. As you mentioned, I want to quote you and you said it very fast. So I don't know if everybody picked it up, but you mentioned Gerald Denberg. And I want to really slow down here because it's probably one of the most influential writer of our world in software. So people don't forget to read all his book, Quality Software Management and so on. Rachel, do you want to talk about resilience a little bit with us also? And I'm very interested about what mentioned near the end about how do you make an information system resilient and on top of that introducing all the AI and generative AI things.

12:57 Yeah, it's an interesting one because what we're I've learned very quickly is that generative AI, because obviously AI in itself has been around and developing for a long time, but generative AI, which is basically exposing this capability to the masses, is impacting so many different parts of everybody's business, right? It can be how you face off your customers, how you deal with them, you know, whether you've got a call center or a chat bot, and that's just the simple stuff. It's how you build software is changing, right? The role of software engineers is, I believe, going to change, right? You've got this whole, you know, instead of, I mean, basically people are generating more and more code. That's one part of it. So we're going to turn into people reading code and being good at reading code and assessing and refactoring code as opposed to people that are good at writing code. And that's always been something that we've talked about at ThoughtWorks and Neal's talked about extensively is that you, you know, you read code more than you write it. And then there's like internal business systems as well.

13:58 So like for us, it's like changing how the sales force works, how knowledge management works. Right. And that's like a knowledge management. You know, it's a big problem in consulting, but it's a big problem in most organizations, actually, like getting the right access to the right information that's accurate. So when you have something disruptive like that, it really pokes holes in the challenges in your business. And you start to see areas where ultimately like some trade-off has been made and now you need to rethink potentially rethink that and I think one of the things that's really interesting is like yes Everything is a people problem, but ultimately, like no matter what design, whether it's a software architecture design or an org design, and as we know from Conway's law, these things are intimately linked. There is no like perfect structure, right? There's only what problems do you have right now and which ones is this structure going to solve? And then there is inevitable issues that that structure brings up and you and that's what you're dealing with. And often that's where you see those people problems, those community problems.

15:00 problems where you might even see like issues with the system itself, like its performance or whatever, like wherever you've created some kind of silo in the organization, you're creating some kind of communication gap, some kind of potential conflict. I mean, even when we talked about the org design at ThoughtWorks, it was like we talked about the idea of an intentional conflict where you know it's going to be there so you have to manage versus an unintentional conflict where you didn't predict that and it's not a behavior you want so you need to also like figure out whether you want to solve for that or not the intentional ones is like you already know about it you know if i silo this part of the organization and this part of the organization you're going to get different you'll get certain efficiencies right and i think this is really what the team topologies book is is kind of driving is that those small agile teams They don't scale. It's not even, they're not intended to. And so you have to start looking for ways to reorganize things around like whatever structures you have, but there's always trade-offs in that. And that, in order to make the system resilient, to me, it's about awareness.

16:02 It's about accepting, you know, accepting the trade-off, accepting there is no perfect org structure, there is no perfect. Software design, at least hopefully those things are aligned and you don't have them in conflict with each other. But then you identify like, what are the intentional conflicts? How do we want to know about it? And how do we want to fix that or not? How do we want to manage it? And then, and managing it, I think is where you start to create some of these cross-functional teams around areas that you critically need to work on or move faster in. So you might have silos in your organization, but you might You like, you know, in this area, we need to move faster. So this area, we're going to put together cross-functional teams. But that only goes so far. And then you need a way, you need signals that tell you that something unintentional is happening. And this is where Neal talks about in the technology, in the architecture, you have like fitness functions. And then in your organization, you need your own, I guess, people warning signs, like ways of monitoring what's happening and issues that you're having. And that could be ability to kind of end to end, get a client from a first conversation to a sale.

17:04 Right. And that's that's a common problem in consulting, but it's a common problem in so many different, you know, whether you're a product company or whatever, so many different organizations. Right. You want to get that client over the line. And if that slows down with the new structure you've put in place, then you want to be aware of that. Right. Because you might slow that down if you're putting in silos for efficiency. And this is the thing I think for the longest time we talked about silos, like don't do it. Terrible. Kind of like the monolith. Don't do it. It's terrible. It's like, actually, in some circumstances, it is the right thing. Very often what you're looking for in organizations and in software is not the best, but the least worst. It's the least worst combination of all these things because they all have interesting trade-offs. Let me give you one concrete example of exactly what Rachel's talking about. So she managed to touch on a lot of these things, but it's really about how you build resilient organizations in the modern world. And the fascinating thing to me about AI is not the obvious. parts, but it's the secondary effects. Rachel mentioned the kind of holistic effect across all the members of the teams, not just developers, but also other roles.

18:08 But the fascinating things for me are the secondary technical effects. So most of our listeners are probably familiar with the ThoughtWorks technology radar, which we produce twice a year. ThoughtWorks.com slash radar. And a couple of radars ago, we highlighted something that was called dependency injection checks for hallucinated dependencies. This is a great example of a second order effect because what happens is, let's say that you're using Gen AI to produce some code for you. Sometimes Gen AI, trying to be as hopeful as possible, hallucinates dependencies. What if a bad actor notices that every time you generate this kind of application, you get this incorrect dependency, and they put malicious code behind that dependency, and now projects that use AI are accidentally sucking in malicious code? Which brings back this idea of you need resilience at the architecture level, ways of checking your software build materials, tools like Dependabot and SNCC, fitness functions to run in your architecture, check those things.

19:11 But you need the organizational support in place as well, because as we've realized, and this gets back to the agile engineering stuff, more and more we're figuring out ways to automate things because we've realized. Modern software systems consist of hundreds of thousands or millions of tiny moving parts. And no person. can manage and govern all of that. You have to have automated checks built into those systems, particularly as we're accelerating things like hallucinated dependencies. We need to have better ways to guard against those new capabilities, which of course always come with new hazards. Well, that will make a very interesting talk. I'm sure people will be very interested to know about that. One thing I have a question, actually, that will be a free consulting here. Let's say that I work on an information. system that has quite a pretty decent technical depth. How can I try to, on top of that, make it even more resilient? Do you have strategies for this? For reducing technical debt, sure.

20:13 There are myriad strategies for finding. In fact, we have in my book, Software Architecture, the Hard Parts, we actually talk about software architecture migration and restructuring and some of the tools you can use to figure out just how much technical debt we have. So we always try to take an engineering approach to these things. And so the first thing I would do is try to run some actual metrics on the code, just how much technical debt is there. So from a structural standpoint, from a capability standpoint, what is the performance like? What should it be? You know, what are the drivers there? And then start slowly addressing the worst part. So this is the, you know, you're never going to fix the entire thing all at once, but you find the worst thing and fix it and then build test harnesses around that so that as you fix new things, you don't break what you've already fixed. And then slowly kind of encapsulate the important things in your code base and attack the things that need to change or need to be addressed, not necessarily try to eat the entire thing. But of course, that depends too on, you know, what's the ultimate purpose of this piece of software?

21:14 Am I just shoring it up to keep it alive for five years? Or is this the foundation for my company for the next 20 years? That's going to have a huge impact on how much time and effort that I put into, you know, addressing some of the things that have slowly accrued over time. But I think it's important to mention too that a lot of technical debt That is not a mistake made earlier. It was a perfectly good decision five years ago, but the ecosystem never stops changing and moving. You know, we make our decisions in architecture with the best information we have at the time, but no architect I know has a good working crystal ball. And sometimes you guess wrong about things. And so I think it's important to build the capability to make changes to your architecture into the architecture so that you're not as worried about predicting the future. If you build adaptability into the architecture, then when the inevitable change comes, you can adapt to it versus getting surprised by it. I think I was going to say that's the key. And that's something Neal and I have both talked about for years is you have to expect change. If you build a very fixed system, then you're building in technical debt by default.

22:18 But the other thing to realize is everybody has technical debt. So I would go so far as to say, especially now with more code getting generated, I have a hypothesis that it's potentially going to create more technical debt, that everyone needs to get good at how to manage technical debt and what to do about it. Right. To Neal's point, if you're shoring up the system, it's probably about monitoring and test harnesses and things like that. But if you're looking to this is a key part component of the system that actually we're going to build the rest of the business on for the next five years, that might be an area that you just need to rebuild that part. Right. And we're always cautious about full rebuild because people just go nuts for it. Right. It's way easier to just start again. But it's very problematic. But there are certain areas, and you have to be really clear about what that is, which components that is, that you might say, you know, if this is taking us into the future, then we should be using the latest and greatest technology and ways of working and approaches for this. And so if you want to learn more about that, I recommend reading the book, Building Evolutionary Architectures from Neal that you wrote with Rebecca Person and Patrick Kua.

23:28 I'm not sure how to pronounce his name. You got close. It's Patrick Kua. And we added actually our colleague, Promos Adalge, for the second edition because he's our resident expert on databases. And we've increasingly realized how important data is to software architecture. And so he was a co-wrote one of the chapters in the first edition, but we just made him a co-author for all of it in the second edition because you can't ignore data and the impacts of data and all the things that we're talking about. I would go so far as to say, because this is how we're already thinking about it, is the new generalist has data skills. So actually, that leads to my question is, what is a tech leader for you? And please explain to us, what is a ThoughtWorks tech leader? Because I believe not everybody knows ThoughtWorks. For some of us, it's a huge reference. It is the reference. But some of the auditor maybe don't know ThoughtWorks. Well, okay. This is also one of those big questions, what is a tech leader?

24:29 Because there's... Different scope, right? You can be a tech leader of a team that's, you know, that owns a few services or one service, or you can be a tech leader like Neal or like myself, which is, you know, a huge scope and a very different space. So the way I think about it is a tech leader is basically, it's a leader who knows tech. And then there's a scope element to that. But ultimately, your domain is tech and that's your expertise. And there are expertise within expertise of those areas. So I guess for me, it's like I really understand tech. consulting at a global scale, which is a very different context, but I have a background as a developer and a tech lead as a team and various other roles. But yeah, it's also changing, right? As I said, I think Gen AI is actually going to change technology, like developers roles, tech leadership roles. And other roles around that change software. But it's always been a very changeable role, right? Like if you come into, if you don't like learning new things, please pick a different industry.

25:32 Not only do you have to learn new things, it's getting faster and harder to learn and understand. And one of the great things about when we do the Tech Radar twice a year, the more and more things come onto it, we call them blips. More and more comes in all the time. And it's every six months. And we even got to a point where we just cleared it every six months and started again because we just kind of review the old stuff and look at new stuff at the same time. It's just too much stuff. And that's the one place that I, as the tech leader across like 10,000 people, can say, can get some idea of like what's going on and what are some of the themes. And one of the great things that Neal does is he helps us wrangle the themes. Like what are the themes that are coming out? What are we seeing? And that's kind of my... context. But yeah, I mean, it's about being a leader, but it's about being a very adaptable leader. Good. So if you have to help them, these technical leaders, what would be your advice, a book or a reference? Well, that's a big question, particularly if you're talking about leadership. And things like that. You know, we mentioned it before, Gerald Weinberg.

26:34 You know, there are a bunch of books about consulting that just are evergreen. You know, the Fred Brooks stuff is still, the Fred Brooks Mythical Man Month is cited a lot. I don't know how many people have actually read it. It's fascinating because it's a duplex book. It's half technical advice and half project management advice. And the technical advice is things like you should really consider using a higher level language than mainframe assembly for your project. I mean, things that are so obvious now, it's like, well, yeah, the project management advice could have been written yesterday. Because we still make the same mistakes that he was talking about then in Mythical Man Month. So I think that's, you know, knowing that stuff, knowing about people. And I think now more than ever, how to learn stuff is really important. Organizing your own personal information. This is one of my kicks within ThoughtWorks when I'm talking to brand new architects. Because architects... Will quickly drown in a sea of information, these information streams that pick up that they didn't realize were there when they were kind of sheltered from it as a developer or tech lead.

27:38 And so one of the things they have to get good at is personal information management with tools like, so there's a great book called How to Take Great Notes, which is actually a very, thin book, but it's great advice for knowledge management and, you know, putting together your own kind of personal database. And I think that's important because part of your job as you in the technology space, it's less and less about hardcore technical depth and expertise in narrow areas, but a lot more about breadth and understanding the impact of trade-offs and this versus that. And so being able to gather information and organize it, I think, is an important skill. Yeah, I mean, and Neal really covered a gamut there. And I was thinking about which books do I go back to over and over? And I would say there's one book that I go back to over and over. And people might think, well, this is only for a short period of time. But as I said, we're constantly learning, which means and things are constantly changing. And the org structures around us are constantly changing. So I actually think it's really valuable.

28:39 And it's called The First 90 Days. And it's a leadership book about, you know, the first 90 days in any leadership role. And I think getting good at, because it's all really about managing a change, a transition, and given all the transitions and change we're going through, I think it's a really good book that gives you a lot of advice on like how to think about it, right? How to model it, how to assess the situation you're in, how to build like relationships with your peers, with your leaders. And it's just a book I go back to when I change roles, when I have new people in the team, like I just go back to it over and over. It's just so useful. I agree with you. And also recently we've read with my team Accelerate from Nicole Frost-Green and all these books, we know they are here. Everything is written. The question is how to make sure that people understand all that and know all of that. If you have a secret, do you have a magic wand for this one? Hey, I'm an author. I just want to figure out how to get people to read books in 2024.

29:40 If we can figure that out, then I would be perfectly happy. Well, then people will come and see you and get inspired at Tech.Rocks Summit and probably will read your book. Rachel, Neal, thank you for the time spent with you. It's been so fast. It really makes me want to be part of it. Our listener will, of course, have the opportunity to interact with you there in Paris. So I remember everybody, it's the 2nd and the 3rd of December in the Fierce of Paris. Don't hesitate to take your seats now. Rachel, Neal, thank you so much. Thank you, Alain. Great to chat. Thank you.