The Architect Elevator

0:03 Il nous fait l'honneur d'être parmi nous. Gregor Hohpe, Enterprise Strategist pour AWS. Son talk portera sur The Architect Elevator. When we look at new technology, we're usually fascinated with all the new things we can do and all the new tools and products that we can provide or that we can take advantage of. But I feel we shouldn't just be looking at new technology and new products. We should also be looking at ourselves. And in my case, that means being an architect. And I'm convinced that being an architect in the age of cloud and machine learning and distributed systems is actually very different from what we might have thought an architect should be doing before. So what I want to share today is how we can see a completely new role as architects to become a change driver, to move out of the ivory tower and start riding the architect elevator.

1:03 So my name is Gregor. I'm an architect by heart, not always by title, but throughout most of my career, I have done things that architects would do. I've designed systems. I've built code. I informed executives. I got budget approvals. I get people informed. I launch products. And I actually also wrote quite a few books. Right now, I'm an enterprise strategist with Amazon Web Services. So I work with our large customers on their cloud strategy. Unsurprisingly, the material I write about and I think about is related to architects and the role of architects in architecture, and of course about cloud strategy, and also about distributed messaging systems. So today I want to focus on what is behind the architect elevator and why I believe that having a fresh look, not just at technology, but also at the role of architects is so important to successfully build systems in

2:04 today's day and age. Now, if we want to understand why the role of architects and architecture has changed so much, we first have to take one step back and zoom up. We first need to understand how the role of technology itself has changed, because that's what's driving the new role of architecture. I worked in very large enterprises. I was a chief architect in large organizations, and I learned how large IT organizations work. And it works like you have a certain as-is state, you make a business case for a project, you deliver a project, it takes you to the so-called to-be state, and then when the project is done, things go back to normal. We call that business as usual. Well, I used this for several billion euros worth of IT budget, and it worked fairly well. However, this model is based on two important assumptions.

3:05 The first assumption is that you know quite well what you're going to build, because that's how you need to define the business case. You need to be able to calculate the benefit you're going to generate. And the second assumption is that change is temporary. You package the change into a project, and when the project is done, you go back to business as usual. And business as usual implies that there are no more changes or very little changes. Now, in our modern so-called digital world, that is different. Both assumptions no longer hold true. In this world, we are experimenting. We're trying to find new business models. Now, and that makes it hard to define the business case because you're innovating. You might have hypotheses you're testing, but the proof is in the pudding. The proof is once you actually launch your experiment and see how the customers react to it. And you would have to be extremely lucky if your first experiment is the home run is the big hit.

4:12 So you should also be expecting to experiment continuously. So now change has become normal. And this is a massive change for the way that IT operates. On the left-hand model, I call this making digital copies. You take something that's already there and you use technology to make it better, more precise, more accurate, more economical. On the right-hand side, you're disrupting, you're building things that couldn't even exist without the technology. And that is a fundamentally different operating model between the economies of scale. You automate something that's there, the bigger your scale is, the higher your returns, versus the economies of speed, which is the faster you can experiment. the more good ideas you can put into production. Now, this is not a wordplay. The skills an organization needs to be successful on the left-hand side or successful on the right-hand side are completely different.

5:12 The left-hand side is about planning and execution. Sometimes I say guessing. Right on the right-hand side, it's about fast learning. And to underline how big this change really is, when I talk with customers, I realize that they even speak a different language. They use a different vocabulary. On the left-hand side, it's a classic IT vocabulary, right? We have a project proposal. It's going to take 18 months, 20 headcount, maybe a million euros. And then we buy hardware for another couple of hundred thousand euros, right? That's how people speak. Here's the business case. We're going to recover the cost in three years. Here's the benefit we're going to generate. Very classic IT speak. On the right-hand side, folks speak differently. So rather than about the project budget, they speak in burn rate. How much does the project cost per month? They talk about velocity. How much business value do we deliver per agile sprint? They don't buy hardware for a couple hundred thousand euros. Of course, they operate in the cloud because it's a consumption-based model.

6:18 So the vocabulary of this economy of speed is the vocabulary of rates. They don't speak in absolute numbers. They speak in cost per unit. Goli speaking, you might say they talk in the first derivative. So rather than trying to find the perfect use case, or let's say anticipate the perfect use case, they're more interested in cost of experimentation. How many tries do I have within a certain time period and a certain dollar amount? And that is a magic shift. So not only does IT start to speak in the first derivative, I actually believe architects live in the first derivative. And this is why in this environment, architecture is so important. So the exercise I did, and I think it's an exercise that's good for most people to try, is I try to think about when do you not need architects, right? What would be a situation where what you do isn't really needed, right?

7:22 It's a good exercise to do. Now, luckily for architects, well, that wasn't that easy, right? But in the end, the insight we came to was that if there's absolutely no rate of change, if nothing ever changes, you don't need a lot of architecture, right? So you just somehow get stuff running, and once it runs, it just sort of keeps on running. Luckily, this is a very poor assumption because change is everywhere, and change comes in all sorts of forms, right? You have changing scale requirements. You have scaling user needs. You have changing language and markets and technology evolutions. Change is just simply all. around you. So the good news here is that there's actually a lot of change and therefore a lot of need for architects. Now, when we say architects come in when change is at play, you can also see what we as architects have to deal with. We have to deal with the things that inhibit change. And generally, we call that friction.

8:22 Now, here's about in this picture, the only example where friction is a good thing. If you want to stop your car, you're happy that your brakes have friction. But you don't want to be driving around with your handbrake on. You want to remove the friction. And this is one of the most important things that architects can do. Now, friction is tricky because friction is everywhere and nowhere. There's generally no friction department. Sometimes people joke the legal department might be it, but at the end, they really just try to do their job and making sure the company doesn't get into trouble. So the real answer is friction is everywhere and it's between the parts. It's rarely in a single department. The second danger of friction is pushing harder is not going to reduce the friction. It's going to increase the friction. If you have a, let's say with the car metaphor, a small engine where the bearings are not really good, they're a little bit too tight, right? The engine isn't running as well as it should be like, oh, let me just like push a little harder.

9:23 Well, that's not going to reduce the friction. Basically, the whole thing will blow up. So we need to be smarter with dealing. with friction, right? It's not just about pushing harder because the friction is just going to go up as well. And the next thing we realize is that, you know, just more horsepower isn't going to do the trick because the friction is between the parts. And I see this particularly a lot. I work with a lot of customers who want to transform and speed up and become like the digital companies. And they say, well, tell me who we should hire, right? And basically what they're looking to do. They're looking to add horsepower to their workforce. Now, the reality is the organization probably looks just like this picture, right? How fast can these cars go? Well, unfortunately, I don't have one, but I know they can go upwards of 300 kilometers an hour. They can go extremely fast. Well, how fast are they going? Or maybe, I don't know, 25, 30, 40, if you're lucky.

10:24 Is putting more horsepower under the hood of these cars going to help? No, of course, it's not because the friction is in the system. So it's not a matter of, you know, do I need to hire people with more degrees or more certifications? No, it's a matter of getting the friction out of the system and letting people realize the horsepower that they actually have and get them out of the traffic jam. So that's what architects need to do when we live in the first derivative, get up the first derivative and reduce the friction. Now, sometimes people ask, like, okay, I get this, you know, economies of speed and reducing friction and thinking the first derivative, but maybe we're not in such a giant hurry. So I've worked for a very large insurance company. A lot of business has an annual cycle. So people might say, you know, this is all nice, but I don't need to push a thousand times a day my new software. Now, here's an important insight, and that is speeding up and reducing friction isn't just because you're in a hurry, but it's also because it allows you to work in very different ways.

11:30 One way that is, is what I call disposability. Architects, we like illities, like scalability and portability, things, maintainability. So here's a new one for you, and that is disposability. If your friction is low and you're highly automated and things go very quickly, you can dispose of things very easily because you can easily recreate them. And if you think about the first derivative of systems, like the change they need to undergo, scaling up and scaling down and deploying new things, being able to dispose of old items is a great asset that you could have. Because it reduces resource consumption and cost in the cloud, but it also makes sure you're always in a clean, defined state. So rather than patching things for the 12th time, you just dispose of them and recreate it. So removing friction isn't just about moving faster. It is also about working in very different ways. A good friend of mine, Martin Thompson, said we often look at the resource consumption of software.

12:35 Well, if you think in the first derivative, like architects do, you would also think about startup time. How long does it take for your software to ramp up? And then you will realize that has a big impact on system availability. Because the shorter the startup time, the faster you can fail over and the faster you can scale up. And ultimately, it makes your system more available because you reduce the time to recovery. or the time to scaling. So thinking in the first derivative has many, many advantages from working completely differently and being able to dispose things to increasing your system uptime. So that leads us to our third insight that yes, architects live in the first derivative. We like to move faster, but speeding up is more than just going faster. It allows you to work in very different ways. Now, I'm often asked what architects really do, like what is the value that we provide? Either I tell people we live in the first derivative, and if they're not confused enough, I show them this formula.

13:42 But this formula has actually a very important reason behind it. This is the Black-Scholes formula for options pricing. The gentleman got a Nobel Prize in economics, so we're not going to explain the formula, but we're going to explain why options are important and why architects sell options. Options are decisions that you can defer into the future. This is financial options. Generally, it's around buying shares, buying stock in a company. If you buy an option, you gain the right to acquire stock in the future, let's say one year from now. And this option has a great benefit because in one year, whether to buy the stock or not is becoming very easy. Let's say you have an option to buy a stock for $100. Well, in one year, One year, if the stock trades for more than $100, you use the option. You buy for $100, you have money in the bank. But if the stock trades for less than $100, then you don't exercise the option, right?

14:42 It's optional, right? And just either buy nothing or you buy it on the market. Either way, the decision has become extremely simple. And that's the power of options. It's like time travel. You defer the decision until a time when you have more information. When you have more information, the decision becomes very, very simple. Should you buy a stock today? You never know. Is it going to go up? Is it going to go down? Well, if you have the option, you find out whether it goes up and down, and you make the decision at the end of it. Now, as architects, we know what this looks like in a technology environment. One of the most difficult decisions we have to make is hardware sizing. I'm building an application. How much hardware do I need? It's maybe more difficult to guess even than the stock market. Now, there's an option we have, and the option is called horizontal scaling and elastic infrastructure. And with that option in place, you can defer the decision until you see how your application is running.

15:43 And just like with the financial option, the decision has become trivial. You need more hardware, you add more hardware. And if you have too much, you take hardware away. So these options have an enormous value, and that is the role that we as architects play. Now, I'm going to get into one small detail of the formula. And one thing that we know is the value of these options goes up with the level of uncertainty. So the more uncertainty we have, the more valuable it becomes to defer a decision. Well, it's logical. If nothing ever changes, making a decision today or making a decision tomorrow is probably roughly the same. But if things are highly uncertain, deferring the decision becomes more valuable. So if our kids... techs produce these options, like options like horizontal scaling, the uncertainty makes these options more valuable and therefore makes architecture more valuable. Last time I checked, we live in quite uncertain and turbulent times.

16:44 So this is a great pitch to explain why architecture is so important in our uncertain and hard to predict times. Now, interestingly, there's another word that we use in this context. Also starts with A and that is agile, right? Agile is also there to help us with uncertainty. If we knew everything up front, we just write it down, implement it, we wouldn't need to be agile. So to me, it's quite funny that sometimes people see agility and architecture as contrast, right? They see it as a conflict. I don't see this at all. If you're agile, the reason you're agile is because you live in a highly dynamic, uncertain world, and you want to be able to deal with that uncertainty. Well, this is exactly the reason you have architecture, because architecture gives you the options, allows you to defer decisions, and also allows you to better deal with the uncertainty. So the next time somebody comes to you and says, oh, I'm agile, I don't need a lot of architecture, you can say, no, it's exactly the opposite.

17:46 Both agility and architecture thrive of uncertainty. So they're actually best friends. So what we've learned here is the higher the uncertainty, like we have it today, the higher the value of the architecture options we produce, and therefore the higher the value of architecture. So it's actually great to be an architect in today's day and age. Now here's an architecture for you. We like architecture diagram. It's a layered system. Now, we like to layer a lot of systems. And as architects, we know why layering is good. It's a separation of concerns, clean dependencies, clean interfaces. You can remove layers easily because of that. Many good reasons why we use layering. It's one of the most popular architecture patterns. The business of architecture is the business of trade-offs. So there are also downsides of layering, right?

18:47 Some of the downsides are that you might have a lot of effort in translating, so there's extra complexity there, you know, data formats and things like that. You have higher runtime latency because of that. You might have the risk of change propagation. Like let's say you build a layered system, you might need to make a change in the front end and the backend for front end and the application logic and the persistence layer and the API layer, the database layer. We've all seen how a simple change can propagate through many layers. Now, the amazing thing about this is that all of what I just said applies to both technical systems, you know, front ends and back ends, but it also applies to organizations. Organizations have I've worked in very large insurance companies. They had many, many layers, and they enjoyed all the benefits on the left, right? That's the divided conqueror, interfaces, separation of concerns. But as the world is starting to move faster, they actually start to suffer from some of the downsides on the left.

19:50 The right, right? They have the telephone game where things go to too many layers and nobody knows in the end what was actually said. They suffer from the latency and many, many other things. So what we learn here is that layering helps you manage, but layering also introduces friction. We already learned that friction isn't our friend in this game. So here we come to a very important insight for architects. Complex organizational systems behave very similarly to complex technical systems. You know how layering works, you know the advantages and disadvantages, and now you can understand why some organizations are looking to de-layer because they're subject to the exact same forces that the technical system has. So now that means as an architect, you're much better qualified to be an organizational architect than you actually might have thought. And that's perfect because as a modern architect, you're going to work at the intersection between technology and the organization. If you want to bring a whole new way of working and a new architecture in the organization, you will not be successful if you don't also change the way the organization works.

20:59 So it's unavoidable that you need to cover both. But the good news is you're actually already well qualified to also bring organizational change. And that is really the key idea behind the architect elevator. Organizations have many layers and the layers hinder them in decision making and agility and they introduce friction. Smart people 20 years ago said, oh, why don't you get rid of some layers? Unfortunately, that doesn't work as easily. Just like in a skyscraper, you can't take just random floors out, right? The organization will not function. So what's the next best thing you do? You put more elevators in the building. You make it easier to connect through all the different floors. And I believe that architects are the folks that are best qualified to do this, right? You can connect the penthouse, the strategy with the technology, reality. In the engine room, you combine the organization and technology. So one supports the other. And you make sure that your stakeholders actually understand what the engine room is doing.

22:03 And that's what I invite architects to do. I invite architects to write this elevator. Yeah, what this looks like, right? I won't be able to go into all of these, right? But you need to make sure IT strategy and business strategy are aligned. You know, if you dig two tunnels and they don't meet, digging faster is not going to help. So this alignment is critical for everything else. You'll never be in a state that you want to be in, so you have to be a change driver. Complexity and friction are your biggest enemies, right? So you need to deal with complexity. You do that by explaining and demystifying things using models, we'll see. And then, of course, there's a lot of other good stuff down in JNG room, but there's usually plenty of material about that. So one more insight we're gaining here. The value of a modern architect isn't how illustrious your title is. Like a chief architect, that sounds really good. But if all you do is sit in the ivory tower and draw pictures, your value is actually Not that big. And if you're world's best Helm chart configurator, that is also great.

23:06 But if your projects never deliver any business value, that is also not that good. So in the end, it's about connecting the levels because that's where the organizations struggle the most. And that's where you make the biggest impact as an architect. And that's what writing the architect elevator means. And the more flows you can cover, the higher your value will be. Now, let's look quickly what that looks like, right? What does it look like to think like an architect? I don't think architect is something that's on your business card. It's really a way of thinking. And I have three ways in which I think architects think in a specific way. Well, the first thing that always comes to mind is that architects draw pictures. But when we talk about pictures, we usually believe architects draw like diagrams, boxes, and lines. But if you look at real life architects, they don't draw blueprints. The famous architects who design iconic buildings like this Guggenheim Museum here in the Bauhaus fame, they draw sketches.

24:09 And sketches are much more powerful than they might appear. You think like a little napkin sketch, I could have done this. The answer is no, you could not have, me neither. There is a lot of thought in here. It's a problem. Knowledge is the context of the site. It's at the oceanfront. It's a very horizontal layout. This is going to be a museum. It needs to have enough space. So there's actually a lot of thought. in here, but the sketch takes the noise out and amplifies what's important. And then all the blueprint drawing comes later. So, and if you're wondering whether this sketch is, you know, detached from reality, now it's absolutely not. Here's an overlay between the sketch and the real building. So we realized that great architects don't just draw blueprints, they also sketch. And sketching is great because it amplifies the essence of what you're going to build. without getting lost into the detail. And then the next thing that architects do is we can see things from more dimensions.

25:10 I'm sure you've been in discussions where people argue A versus B. It's like, oh, we need an API gateway, or we need a service mesh. Oh, we need a document database. No, we need a graph database, whatever it is. People always argue one versus the other, and they can do this for a long time. As an architect, often your job is to see more dimensions, because it might be just like in this cartoon, one of my favorite, where one person has a certain viewpoint and another person has another viewpoint. One person might be the development team and say, oh, we need to speed up software delivery. The other person says, no, we need to slow down software delivery because otherwise we'll have operational issues. Now, a role as your architect is to realize the two are not opposites of each other. You can have automated testing, shift left, DevSecOps, high levels of automation, run in the cloud, be serverless, like a million things that you can do to show like, hey, you can have a higher rate of delivery without causing operational issues, actually improving the operational footprint. And that's showing people that it's not one versus the other, that there's a bigger viewpoint.

26:14 You can see things from what I mentioned. It's one of the most satisfying maneuvers that you can do as an architect. And not only are we good at seeing things from different dimensions, we're also good at zooming in and zooming out. That's just like the elevator. When you ride down to other floors, it's not like a camera zoom. where you see the same thing a little bit bigger, you see very different things. Our IT world is very much like these fractals that I like to watch, these Mandelbrot sets, right? At the different layers, you see different things. And being able to zoom in and zoom that out is a very powerful maneuver for architects. So these are three ways that we as architects are able to think. We like to sketch. We like to see things on different dimensions. And we like to zoom in and zoom out. Now, when you talk about sketching and drawing pictures, what we're really doing is we're making models. Models are one of the most powerful tools we as architects have, especially for the elevator. I talked about battling complexity. Models, sketches are the best way to reduce complexity and bring out the essence.

27:19 Now, a common question is, people ask, well, that's great. What is the best model? What is the best sketch I should make? And there's an interesting answer, and that is there's not a single best model. The best model depends on which question you're looking to answer. So here are some classic models. These are models of a system we know pretty well. That's the system planet Earth. And which map is the best map? Well, it depends what question you're trying to answer. If you're looking to go for a hike or build a ski resort or avoid the flood zone when you're building your house, a topographical map is the absolute best map for that. Or if you're trying to drive from one city to the other as quickly as possible, you want the highway map. If you want to understand the elections, you need a political map. a map and if you happen to build distribution centers, you want a population density map. So what we learn here for architects is models are extremely powerful, but there isn't such a thing as here's my architecture, here's my model. The best model depends on what question you're looking to answer. Next time somebody says, show me architecture, it's a valid counter question to say, well, I'd be happy to do so, but let me know which question you want answered so I can give you the model that actually answers that question best.

28:32 And this is one of the most powerful maneuvers you have in the middle levels of the architect elevator, because you can explain things to people without having all the complexity, but still answer the questions in a meaningful way. Now, one level down in the elevator, what I mentioned is that you'll never be in the state that you want to be in because the world around you is moving. If you're standing still, you're essentially moving behind. It's like a train. You sit at the platform, the opposite train leaves. You feel like you're going backwards. That's the world we're living in. So as architects, you're also in charge of bringing change to your organization. As we said, you're already qualified to be an organizational designer. So let me share just one example of how understanding complex systems allows you to bring change. Now, the way in many large organizations, projects get funded is you make a project proposal, like we said before, and then you get to execute a project over a certain amount of time.

29:37 Now, the challenge is this system has some friction. There's project proposals to be written. There is steering committees to be had. There's budget reviews. There's just a lot of control checks, et cetera, that need to be done to execute a project. Let's say for argument's sake that the overhead to run a project is 100,000 euros. It would seem like a lot of money, but I know a lot of people who would say, oh, I want to work there because it's a lot less than we might have. So if that's the case, if there's 100,000 euros overhead to run a project, does it make sense to propose a project for 100,000 euros? No, it doesn't because you'd be doubling the cost, right? You'd be paying 100% tax, if you wish, right, with the overhead you have. So what is the right thing to do? Well, the right thing to do is to make a bigger project because the overhead doesn't go up that much. It's the same budget approval meetings and review meetings and steering committees, right?

30:38 You just make a million-dollar project, right? And you save the company money because the overhead is now just 10%. Of course, you know the next thing that's going to happen. Somebody says, well, all these projects that we're getting these days, they're all like millions of dollars. That's a lot of money. And they're right. It is a lot of money. So what are they going to do? Well, they're going to put in more detailed project plans, more detailed what-if scenarios, more budget reviews, more steering committees. They're going to put in more controls. And that's the right thing to do. You can see how your architect's thinking allows you to see that this actually goes into a somewhat vicious cycle. So we learn a couple of things here using our architect brain when we look at organizations. The one thing is you need to see the whole system. understand what's going on. There is no guilty party in this. This is not like debugging your code where you're trying to find the line of code, the typo, the syntax error, or fixing electronics where you're trying to find the broken transistor.

31:43 Everybody's doing the right thing, but the system is broken. So we as architects, need to use our system brain, zooming in, zooming out, seeing different angles to understand what's going on. Now, the good news is the cycle also works the other way around. So if you manage to reduce friction, right, people would make smaller projects, right? The smaller projects, you need fewer controls. Fuel controls also require less oversight or fuel controls create less friction. So that way the whole cycle turns into a positive cycle and things can actually speed up. So this is a great way of how you can use your technical architecture system thinking to organizations and actually bring. Change into your organization, which is necessary to be successful with your big architecture projects. Now, there's a lot more behind the Architect Elevator that I was able to share on this short presentation. So I invite you to have a look at the website, architectelevator.com, or the book, which is really based on my journey as the chief architect, you know, riding or being catapulted from the engine room up into the penthouse,

32:55 initially being very confused, but then ultimately realizing how I I can combine and make a connection between the penthouse and the engine room and start to bring organizational change and see architects and architecture in a whole different light. So with that, I thank you for your attention and we will follow up with a short question and answer session. Thank you so much.