Technology transformation: path to success

0:00 Nicolas, Grégory et Yevhen, sous vos applaudissements encore une fois, et sous un jingle qu'on va adorer. Merci Nicolas. Je vous laisse vous asseoir comme vous voulez. Nice to meet you. Thank you so much to be there. Have fun. You have 40 minutes, I think. Gigi, 40 minutes. Bonjour à tous. Maintenant, on va parler anglais. Merci à tous d'être présents. We're going to talk in English for the next 40 minutes. So I'm very honored and happy to be on stage with Yevhen and Grigory. Today, we are going to talk about a huge migration to the cloud. In a very difficult context, a war context. And so, Yevhen, the CISO of the company, and Grigory, the CTO of the company, they will tell us about their project, their migration, and how basically they did that.

1:06 Okay, so guys, very happy to have you on stage, and maybe we can start by presenting yourself briefly. Easy. Thank you for inviting and I appreciate to be here. Hello there. My name is Gregory, as already mentioned. I'm a CEO of Raiffeisen Bank Ukraine. And also 16 years, I've already fallen in love with engineering. I worked in a lot of industries. like starting from retail, then radio, then gambling, porn industry, and now I'm in a bank. And I love to do what I do. And today we will talk you our story. We will tell you our story about our cloud migration within three months with full-scale migration. Wasn't 2,000 servers, whatever. So, Zhen? Yeah, hello guys, hello guys. Actually, all you need to know about me that I didn't work for porn industry, at least just to balance this, so my name is Yevhen.

2:09 I'm working for security for more than 14 years, also in different industries, pharmaceutical, telecom, gambling, betting. And so on, and of course banks. So, yeah, we had an interesting story, and actually it continues. And also I want just to share for you a small context. So, Refazing Bank Ukraine is the biggest Ukrainian bank with foreign capital right now. So it's not state bank, it's kind of private. With more than 5,000 employees, it's approximately 5,600, with more than 3 million private clients, and very huge in terms of corporate. So we are leaders in corporate, actually. So, yeah. Yep, I hope this story will be interesting for you. Yeah, so let's start maybe just for everyone to understand the context. Maybe just can you describe the context and also why basically you had to migrate to this public cloud?

3:18 So we have obvious reason and not obvious one. Let's start from obvious. 24 February in our country, like war appears, I would say. And we didn't have any choice. We have to... We have to save our infrastructure. We have to serve our clients because money for our clients in the work condition is very important. And the obvious reason we have to do so. Not obvious reason is in early 2021, we started our... Technical transformation. We were super Super usual bank. We had a lot of Oracle stuff. We had a lot of IBM. We have just 60 engineers for entire bank who serve, as Zhenya mentioned, 3 million clients. And we were a bank with a culture of buyers, not the engineer one.

4:22 And we... Wanted to change it and we even we start to change it we hire right now it's 900 engineers we hired a lot of engineers we prepared new strategy and part of our strategy was migration to the cloud because we must change our first of all engineering culture and operation model and we We couldn't do so because of our legislation, our governments, which is the National Bank of Ukraine, not allow us to do so. before the war. But we did the massage during the last year, like during the 2021, we did massage for NBU. So just for everyone to understand, NBU is the National Bank of Ukraine. National Bank of Ukraine. We had a lot of conversation, we prepared them that sooner or later all banks should be in the cloud. And not only banks, but whatever, right now about banks.

5:26 We have to be in the cloud. And everything already was prepared to... To migration from the legislation side. As far as I remember, just law should be signed. And we waited until law will be signed, but war started earlier. So that is our background to migration. And maybe technically speaking, can you just give us a sense of the size of your infrastructure and the complexity of the project? Yeah, as I mentioned already, it was close to 2,000 servers. Regarding the data, it's easier to understand our size through the data. We have three petabytes of data. It's a really huge amount of data. Okay, and so focusing on tech also as well, so we say migrating to a public cloud, actually, I guess that you have plenty of different tech systems to migrate.

6:31 Like which public cloud did you decide to use? Oh, it was so easy for us, we choose AWS because They're leaders, first of all. And the second one, we already had landing zone. We are part of the big group, which is Raiffeisen Bank International. And we have like center of excellence who already prepared landing zone. What does it mean? It means that account already were created, network zone already was created. And maybe EAM policies already was created, and that's it. But it already was chosen before us. But we respect the choice, and if we have to do the second time, we will make the same decision. And we even add some complexity on top of that exercise because we didn't mention it yet, but we are multi-cloud and we're actually using Azure,

7:35 AWS, Oracle, and IBM Cloud. So it's not so easy. Yeah, and approximately we migrated to... 150 systems consist of different endpoints, different databases, and all our data right now operating in cloud and our processes operating in cloud. Okay, and so what about the technical challenges? Like the main one, I can imagine that the context has a lot of impact as well. So maybe can you elaborate a bit on that? Yeah, we have actually, we had view. And the biggest and not obvious one was direct connect from our data center to AWS. As I mentioned, 3 petabyte data should be migrated. But we didn't have internet connection from Ukraine to Frankfurt, to AWS region, which we are using for now. And instead of that, we built our own VPN channels. But maybe you know, maybe not.

8:37 But the VPN channel between the regular VPN channel between data center and AWS, it's just one gigabit throughput, which is... It will take you, I don't know, months, maybe much more than months to migrate 3TB data through the channel. So we used that VPN. We migrated data through the pure internet. We just built end-to-end encrypted connection between Bastion host and our data center, and we migrate less severe data, and we migrated through the regular internet. And then in two months, we finally built this direct connect. The funniest thing... but it's not funny, frankly speaking, we couldn't find any free engineer from a data center from the provider side who can build that direct connect physically because most of our men's were on a war. So, yeah, that's why I'm not sure that you'll face it with the same problem.

9:40 But the problem which you can face is, for example, we had a lot of Oracle databases. Yevhen mentioned 250 systems. It's approximately 500-something database, Oracle database mostly. And we used to use... Oracle Standard Edition, which doesn't support replication. It means you cannot keep your one node on your data center and second node in AWS, for example. For that, we have to migrate all our standard edition Oracle database to enterprise and just then to build that replication. And it took us... Additional one and a half months. It was the biggest challenge how to do it. And definitely we should think about the price of that decision because standard edition costs two or three times less than enterprise. But we negotiated with Oracle everything okay, but you should be ready if you have a standard edition Oracle, it can be problem for you.

10:41 And first point for the future summary, to make an immigration successfully and fast enough, you should have enough risk appetite. Without that, like even this migration could take like plus six months. If you decide, for example, to use strong encryption and wait until Direct Connect will be built, it will be a plus, a few months at least. That's an interesting point because, I mean, like, in the end, you are the security guy in the bank, so... You talk about risk. What was your approach regarding that? Actually, you know, it's easier to talk about something else, because if you talk about risks, the whole project is like one huge risk for the bank. It was not so easy even to start because, yeah, as Grisha mentioned, for a year before the war,

11:43 We negotiated with our national bank on how to do it, and it was not so easy because few banks want to store their data at the United States. Other banks want to do that at the European Union and different laws, as far as you know. And we prepared most of future papers, I mean legislation, to be approved by National Bank. I mean, we, like few banks, okay, so I coordinated that group. And after that, so the only one way how to start in our conditions at 24th of February 2022, I remember that meeting at 5 a.m. After rockets started falling. Like all board members in my room, And we asking each other, so now or never? We felt like that, actually. So in five minutes, we decided, okay, now.

12:44 And Grisha's team, our engineers team, started to do it. right now. In another 30 minutes we started this project, combined all engineers we had. It was approximately 180 people. And yeah, it was super fast. Without risk-taking, I believe we We could fail. And what about, yeah, so you're also in charge of business continuity. We can imagine that in such a context, it's a complex topic to handle. So what did you have to do regarding that? I mean, regarding the migration, having the right people working on the project, but also, yeah, basic continuity of the business, as I can imagine, it can be very hard. You know, first of all, in stressful conditions, it doesn't matter what is your position, what is your function, who are you,

13:45 what are you doing, are you DevOps, are you a C developer, are you a Java developer, are you a COBOL developer, are you a security guy, analyst, or QA? So you should do your best to support, first of all. So that's why we didn't care about any positions. So we collected just team in one room and we sit together. I mean, not literally in one room, because when war started, we even had, we have photos when our guys and girls work from half-destroyed houses. without roof, for example, literally, to support migration. So working from the shelters, it was not so easy because rockets were falling. It was super, you know, this tough phase of that war when everything starts and we were not ready as a country. Not fully ready. And so we decided to move forward anyway.

14:48 That's why we worked 24 by 7 for three months, literally 24 by 7. So it was like periods, you work for 12 hours, then you try to sleep. A little bit, at least. Then your colleague will support you during your sleep and vice versa. And in general, our biggest achievement that during that migration, our customers didn't feel anything. At all. So we had very few small downtimes during non-business hours. So for our clients it was super smooth, fancy. For businesses as well. For our business processes as well, but also as a business continuity part of that exercise, we spread our team across Europe and across Ukraine, because we understood that three months will be not enough, and we don't have such huge risk appetite to continue, for example, for five months or six months.

15:54 So we decided to spread. It's also like investment. So we pay for each and every employee, for accommodation, for food, for transfers and so on. So those who can go abroad, they went abroad. Those who should stay at Ukraine because of restrictions, as far as you know, like typically men from Ukraine cannot cross the border right now still. So, yeah, we made a lot of decisions, but the most important thing, like your board, and your decision makers, your shareholders, should have enough risk appetite in case of these emergency conditions to take all risks because it's a question of survival. It's not a question of how much money we will spend, maybe if something, vulnerability, threat, blah, blah, blah, and so on. It's a question, will you leave as a company? Will you support those 3 million clients?

16:55 Will they lose their money during a war? Will they have their money on their cards or cash? Will your ATMs work under interesting conditions like blackouts or not? So, yeah, it was also a big part of our exercise. I mean, business continuity management is not just a phrase for us. For us, it's like way of life right now. So all of us, we have power banks, power stations, solar panels, like additional electricity we can use. It's also part of our life. It was also part of precondition for decision making why we need GoCloud right now. Because for sure data centers is reliable enough. But still, imagine the whole city, like 4 million people without electricity for 7 days. So what should you do? And you cannot migrate to other country to continue work. So you need somehow to find electricity.

17:57 You should go to buy diesel. or petrol for your generators. You should have generators for hundreds kilowatts, you should have replacement, you should check everything if it's working, if some problems. So yeah, it's also huge. It's also huge. Super interesting. Let's stay on what you said earlier, Grigory. You said that you were a classic bank before actually the context, but also the decision that you made to migrate to a cloud. So I can imagine that you had people actually not really used to use AWS or whatever actually the cloud you decided to use. So in terms of team organization and maybe inspiration of, yeah, how did you manage to change people's mindset, but also organize them to work on that topic? Thank you for the good question. Actually, as I mentioned, in early 2021, we started our technical transformation, and we inspired by a book, maybe, not maybe, all of you know it, it's Team Topologies.

19:04 We were inspired by that book. We launched four different types of teams, which is like neighbor team, stream-aligned team, complicated system team and platform teams. So we knew that sooner or later we will migrate to some cloud. So in early 2021, we already started to hire guys, developers, DevOps, analysts, whatever, all all actually positioned, we started to hire just with experience in the cloud. It could be not a double yes, because we actually don't care. It must be the experience with clouds. So, and as I mentioned, we actually invest a lot of resources to enable teams whose goals was teach but not touch. It means... that you can use a neighbor's team which can come to you, teach your team how to leverage capabilities of the cloud, or actually it's not just cloud, it's how to build

20:14 cloud-native code, cloud business solution, cloud-native API, whatever. Our enablers team come to you, teach you to do so, teach you to leverage technologies and approaches, and then they do another job. And all of this have no sense if your security team is not on board. I mean, with that changes. So we as well hire a lot of engineers. We as well have our own development team. We as well have a team of DevOps. For example, I have 12 developers, 7 DevOpses. So security should be together with other IT teams. No chances to succeed without that. And I cannot just, if you ask me just one advice, what should you do to be successful in your technological transformation, transformation, whatever, like regular life, build the best relationship with your security team because they can be or your promoters or your destructors.

21:22 True. Security guys, especially from compliance, they can destroy any initiative because of high risks. So if they support you, most probably it will be much more easier to succeed. If they're not, sorry. I guess we can trust that you are still good friends, right? You are sitting next to each other, so it should be okay. Yeah, true. Okay. So let's focus maybe a bit more on the migration strategy. So we know that you have different kind of approaches, like doing that step by step. Doing like a big bang and one in a sudden you are on AWS. What was your strategy regarding that? I would say we didn't have any strategy. We had just tactical. I want to say that we as a serious company, we as a big bank, we definitely had a strategy which like super precisely written. But no, it wasn't like that. We just built a project team from the... We have L1 team who is responsible for, I don't know, any others, requests, whatever.

22:29 And that team become a project management team who is dealing with entire migration. We divide our migration to waves. It was five waves. It should be five ways, but frankly speaking, it's more look like two. But first wave was four times. It's about strategy. It's about strategy, yeah. And every day at 8 a.m., we, like as Zhenya mentioned, we met each other. Only one room, but it was virtual room, definitely. And we talked about our plans for today. It was every day it was tactical movement, tactical movement, no any strategy. Because strategy is more like, okay, it's our strategy for the next three to five years. But as you mentioned already, we spent just three months just because it was a project, not a product. And it was tactical decisions, not strategic one.

23:31 Okay. Yeah, it was big bang. It was, for biggest part of our bank, it was big boom. It was lift and shift. Like, we didn't change anything. We just move it. But for our cutting-edge technologies, we did refactoring. For example, our microservice platform was fully right for the AWS native services. For example, Kafka become MKS, Kubernetes become EKS, Radius Elastic Cache, OpenSearch was Elastic Search, whatever, I already forgot how they call it, DNS to R53, load balancer like F5 to ELB. That our cutting-edge technologies for these three months were refactored because of this neighbors team who already was hired with particular knowledge. Okay, and you said earlier, I think it's you, Yevhen, you say that We didn't really care about the budget because it was not the main topic and the main risk that we faced.

24:39 But for everyone in the room, we know that migrating to a cloud, in the end, we say we don't care about the budget, but it can cost a huge amount of money. So what about that? And especially your approach regarding FinOps, was it a topic at all or not? But maybe it is a topic now for you. So, yeah, can you elaborate on this? I can explain. Yeah, definitely we started our immigration because of war condition. We started our immigration. We didn't think about money at all. But in three months, we received the bill. $700,000. I remember that figure by heart because that figure showed that we screwed up. Because this is too much. I'll explain to you why. During the last five years, our cash flow for our data centers, our own data centers, was 5.6 million euros per year.

25:42 So we expect that cloud should be or the same or a little bit higher. For example, 6 million. 6 million euros we could afford. But if it's for one month, 600,000, it means that it's like 8 something million per year we couldn't afford. So we understood that we have to launch FinOps initiative. We knew that FinOps... Exists in the market, but we couldn't hire anyone because it's more like, I don't know, unicorn. It's easy to find unicorns and finops on the market. And we launched, again, our neighbor team spent, I don't know, maybe three weeks to read a lot of documentation, how to build more efficient For example, our biggest spending was for the disks. We used GP2 or GP3. We used a lot of GP2, which is outdated disk, and we used a lot of IO2 disk, which is five times more expensive than GP3.

26:47 But everything that you must know and everyone in your company must know. So our neighbors team. Found all the tips and tricks. And some systems, a little bit were factored. Some systems were just reinstalled. And our current spending is 300,000. Close to 300 something. And entire our bill for all clouds, which Evgeny already mentioned, and our data center, which still exists, because we have some VDI software for the branches, which is super old, and it's not worth it to migrate to any clouds. And entire our bill is less than 6 million euros per year. So clouds, expensive. It's true, but only if you are like, Only if you don't care about it. But if you care... Disclaimer, all prices are with VAT. Just if you're interested.

27:48 It's included VAT. Yeah, yeah, yeah. We have 20% VAT in our country, and I always calculate everything with our VAT. It's a tricky moment. Yes, yes. And we can... We can explain maybe a little bit, just a small workshop for two minutes, how to kill your migration, if you're interested in. It was our super huge fuck-up. So during these first three months, it was a video case. We have huge video infrastructure. We need it because we spread it across Ukraine. Ukraine is the biggest European country in terms of size. France is second. So actually... You should understand me. And we have a lot of branches. And those branches, we have recovered them by VDIs. Why VDIs? Because we are still using outdated technologies.

28:49 It's usually for big companies. It's a typical thing like legacy. So we have few applications running on those VDIs, so we decided to use VDIs for everybody. And actually we migrated those VDIs to the cloud. But migrated not lift and shift, but using AWS technology to set up those VDIs. Like workspaces or workplaces. You know, it was VDI VMware infrastructure in AWS. Yeah, yeah, yeah, initially. And then we checked and we... We realized that, guys, it's too much money. Like, we are not ready for them. So let's go back to the ground with those VDIs to save some money. When I mean some money, it's like 700K per month. And guess what? We forgot to switch off.

29:54 Because of these tactical meetings and so on, we just forgot. string to switch off VDIs. So yeah, we spent 600, 700k for months for nothing. So this is like small workshop, how to kill your migration. Actually, if you don't have war, if you don't have such conditions, usually spending 700k will kill any project for nothing. We just burn that money. But it was super good experience for us in terms of FinOps. I believe after that incident, we started to use a lot of products, helps us to calculate, to check anomalies. To adopt something new, like new type of instances, databases, and so on. And even guys from security also, sorry, I need to promote security because I'm from security. Guys from security also contribute inside this FinOps community.

30:58 So in general, we have approximately 150 AWS accounts. And for example, just security have approximately 12, 13, something like that. So each and every team using AWS account is responsible for their AWS account, including FinOps part. Yep. So after that incident, we started working hard and it's. For me, sounds and works amazing. I recommend All of you, if you're using cloud and still don't fall in love and don't check details, go deeply with FinOps, do that today, immediately. Because it will have immediate impact on your budgets and you can very easily visualize that and show your stakeholders, look guys, it works. And you can be better by small steps. It doesn't ask you to be like super senior guy from tomorrow.

32:04 You could be junior guy, but already save a few hundred bucks. It's cool. Yeah, that's an interesting learning. I mean, as a CTO, I can reflect on that. Like spending money is an important topic, especially now. Yeah, let's stay on security maybe with you, Yevhen. Considering the context, we can imagine that you have lots of threats to deal with and also maybe to increase the security awareness of your teams? What was your, I would not say strategy, because you told me that no strategy, so, but yeah, basically what did you do regarding that? Actually, it's not true. Like, definitely we have strategies. We are big guys from big bank. So, I mean, we have security strategy and clouds. Come on, come on, he's joking. Clouds were there. understanding of multi-cloud future setup were there. So we also started to indicate our engineers cloud topic.

33:09 But first of all, during that migration, yes, as I mentioned, trust is a key. And working together is a key as well. So the most tricky moment during the first few months was not to lost and not to leak personal data and financial data because we as well have regulations and they are the same strict as European Union has. I would say that the next huge topic for us during that few months of migration were how to establish flexible rules for the first six to 12 months. For our engineers on how to operate in those clouds. I mean access management, I mean encryption, I mean product security part. It was also not so easy because we started to create accounts, account after account after account after account.

34:13 It was huge. So the distributed responsibility model for security works for cloud. If you have good general rules like AWS allows you to set up strict rules for your mother account. and then reuse it for all, including that organization. I would say from security point of view, the most challenging for us was to be on the same page with development teams with IT teams to understand their pains and to help them with the same approach as Grisha mentioned. Like we met every day, like what's the problem on the table? Okay, I will tell you. I will take that. I will solve that. So it is what it is. I can just briefly add that it's kind of feedback for you, yeah? It's a good place. It is.

35:14 It is. Our security guys, as I mentioned, it is very important to build a good relationship because usually how security team or department look like, they look like a government. Who are saying what you should do, like you should be compliant with that, that, and that, and that, and that. The best relationship, when you build the best relationship, the security guys become contributor to your code, and they can work with you directly. They can work with this paradigm of shift left. They can automate everything, whatever you need. And for example, as I mentioned, Our main goal was change our operation model. And previously, just imagine from hello world to dev environment, you have to spend at least one week. For now, we have KPI 15 minutes and we achieve it. Achieve it because of. Because of security. Because we are a bank, the most of things we spend, it was for approving, security, whatever.

36:15 For now, 15 minutes instead of one week. That's an interesting point. So we are running a bit out of time. So maybe my last question. You had to change very rapidly because of the context, but you also said that We were planning to do that before, actually. So we can say that you transformed the organization, the operational model, and the infrastructure. Maybe what would you say about that and what are the benefits that you can get out of this migration and your basically transformation journey? Actually, the best thing I already mentioned is from week to 15 minutes to be deployed on dev. And that is important. Our current generation of developers can catch the inspiration during the routine. They just walk through the street and catch inspiration.

37:16 Okay, I know how to do that business or how to deal with that startup. They come home, they open laptop, and they're ready to work. Before it was impossible because they don't have access to infrastructure, to some runners, to whatever, everything. And now our operation model, like we have fully distributed model, operational, each engineer in the company has the same access right. For sure, it's like, let's say 99% has the same access, right? Because we have some security tokens. Let him think like that. It's okay. True, true. Truth is, not fully true, but for your regular work, for your usual work, you don't feel any, I would say, silences. I do definitely pain, but I mean walls, whatever. You are like security topic. Of course, security on a high level in our company.

38:18 But for us, as for developers, we don't feel any borders, any walls. That is the main things what we achieve. It's so hard to explain for one minute. Believe me, it's talk for four or five hours. But in a short period, I would say the biggest achievement in our cooperation model is 15 minutes from hello world to dev environment. And all of this would be impossible without part of our actual strategy and point from that called invest in people instead of technologies. Because it's super easy to operate portfolio of your vendors and it's super hard to raise inside your company culture. For one, two years, to raise developers, to raise engineers who are ready to deal with any task. Regardless, it's authentication, authorization models, some products, client banks, AWS, and so on. So it's the key.

39:19 That's for me, from my point of view. Okay, thanks a lot. So a quick wrap-up of what we see today. Like, we can see that we have two guys, like, they are C-level in a bank, but... Still, they know very, very well the tech behind and how to basically deal with the topics. Super interesting. Also, we learned about that you have a context. You can, even the worst possible context, you can leverage it actually to change, transform, etc. And that's, for me, one of the key points of your story. So thanks a lot for sharing that with us. And we can also see that Lots of pragmatism and I think for me it's a key learning also. of what we've discussed together, like pragmatism on security, how to adapt it to the context, pragmatism on the technical choices as well, and of course, super hard work in order to make it a success. So thanks a lot, Zegai, for all your insights. That was, for me, very inspiring. I hope you liked it in the room.

40:20 We have a couple of minutes for questions, so please just ask them. Thanks. So questions can be in English or in French, whatever you prefer. And yeah, please. Hello, thanks a lot for your feedback. Is there a specific reason why you didn't use AWS services Snowball to make easier for your migration? It's not available in Ukraine. Okay, good reason. Fair enough. And frankly speaking, it's not only because of that, but because, as Evgeny mentioned, we had just few downtimes during one hour and through the night. That's why Snowball required turning off your infrastructure, then you're waiting until this bus moved to data center, then everything will be connected, like one, two, three days.

41:32 We couldn't afford it. And as Jenny mentioned, our employees work without the roof just because one reason. We have to build strong relationship. Strong relationship with customers, they should trust economical system of Ukraine. If we downtime our bank, it will be disaster for people. People will don't have access to their money and they will not believe that Ukraine will survive. And now, like, we have our obligation was to to keep this feeling that everything, like we will survive, everything will be all right. And just one last one. Did you have a specific support from AWS toward your situation in the war? Or you were just a customer like any other? Yeah, we had... No, I would say that each, not specific, yeah, each company which want to build a relationship with AWS will have the same level of support.

42:46 I'm not sure that we have something special. It's like typical, like few chats, few architects to ask questions. And actually, that's it. But we cannot say that AWS didn't help us. They helped us, actually, helped us and continue to help. So I would personally recommend to having AWS as partners. So they are quite good. But no specific support. Thank you for sharing this experience. I believe at the beginning you mentioned you couldn't get a direct line built to transfer the data, and you ended up transferring it through the open internet using encryption. It is my imagination that governments are a few steps ahead of public encryption and are able to break that. Were you afraid at any point that the Russian government or any big player could intercept these petabytes of customer data?

43:53 Yes. Let me explain a little bit. Since the beginning of January 2022, Russian advanced persistent threats groups started to attack Ukrainian government and Ukrainian banks. So us as well. So we knew very good what is their tactics, how they are operating, knew all indicators, knew all mechanisms they use, vulnerabilities they use, and actually we were well prepared. So I'm not mentioning that, but we made a huge work to allow this data, parts of the data, because Grigory mentioned we didn't migrate super sensitive data, like your personal IDs, tax numbers, and so on. through public internet. So we divide those data from general data set and we were quite prepared for any possible scenario including data

45:01 So that's why partially our data moved like tokens, partially it was just encrypted, partially it was open, partially we set some very specific traps for possible groups. So, yeah, we did a lot, but we cannot explain everything we did because it's super massive. Thank you. Okay, unfortunately, time's up. So they will be with us this morning, especially during the breaks. So if you want to ask more questions, don't hesitate. And once again, thanks a lot. Thank you, guys. Thank you so much. Thank you. Thank you so much, guys.