0:00 A lot of what we do is problem solving, and not just on the how do I write the code part, but I think also being able to pinpoint where it goes wrong and you get a feel for that right? 0:09 When you've seen enough problems, we're under a lot of pressure to. Ship new stuff constantly. 0:14 That's the finish line. But I like to not think of it as a finish line. That's actually the starting line. That's where it gets interesting because that's where you Start learning. 0:21 If there's a system where the whole team that worked on it isn't there anymore, is that system legacy? 0:26 Oh yeah, definitely. No one wants to touch it. 0:28 Nobody wants to touch it right? You build some something you assume is wrong, and to be honest, usually what you build is wrong in some way. And then you optimize your feedback loop, and then you learn and you make sure that your code is easy to change, and then you change it. 0:40 Today we discuss how to build software and how to do that successfully. What are the skills that people bring to the table when they build it, and what does it look like from a system design perspective? 0:50 Joining me at the table is Nico Crane, Speaker Architect and Software Engineer over at Luminous. 0:56 And with over 30 years of experience, he's seen many organizations and built many successful projects. 1:02 So enjoy. You've had many roles starting from software engineer and then person that's responsible for cloud or agile methodologies, even engineering management architecture, and now back to software engineer. 1:17 What What keeps you coming back to software engineering in the first place? Good question. 1:21 So like if you, if you grow in your career, there's, you usually see this career letter, right, where you can have sort of the split in where you, you do the software engineer part, you're senior. And then at some point, either 1:34 you go more like architecture wise or more technical or more the, the people management side, right? 1:39 And a lot of people go into this management side, which I sometimes find a bit of a shame because we have all this talent which takes a lot of time to build up really hard. 1:47 It's a, it's a complicated job, right? 1:49 What we do and then we shift all those people into in management roles, right, which may not necessarily be there, their skill set, right? You also see people struggle with that, but it pays more. So it sort of feels like an 2:07 attractive path to follow. But on the other hand, the, the technical part is right. You there's a lot of stuff you can grow there. And I think in the end, both of the both of the sides you get into is sort of, yeah, the, the 2:22 fact that there's more of a human side to what you're doing, right? You're not just doing the technical part, you're also facilitating the people around you, right? If you want to be successful, lead what in whatever role, right, technical or, or 2:37 management, you need to help the people around you go faster and, and perform better and, and feel better, right? 2:44 Being happier. And so that human side becomes quite important, I think in both sides, at least if you want to increase the impact you make, Right. 2:56 And for me, that human side has always been there, I guess. 3:00 I mean, the the first job I had out of school, I immediately got an intern. And I mean, like, they probably saw something, I don't know. But I said, yeah, you're working now you get an intern. OK. 3:12 I just was the intern. OK. 3:16 So that was interesting. And I've been doing like coaching, helping others throughout my career. 3:23 It's just at a different scale at some point, right? 3:28 But I really like the technology as well, right? 3:31 I mean, I got into this because I like fiddling of like, like solving complicated problems. And I like just the technology side as well a lot. So, and I'm really good at that as well, right? I see I'm good at modelling stuff. I so I don't want to waste that 3:49 talent. At the same time, I see that I increase my impact by also doing other things. 3:55 So that's why I sort of right the, the sort of switching to engineering management at some point. 4:01 It was sort of natural career path right at the time. 4:06 Also the, the situation the company was in at that point was sort of a logical step for me. But then I right, the more I did it and the more I saw, right, at some point you get to a point where there is some kind of crisis and like, OK, we need to 4:24 fix this problem. We need the best people. 4:26 Yeah. So you, I'm a good guy, right? 4:29 So also, so I jump in and I helped out and I really realized at that point we were with three seniors tackling a super complicated problem. And we nailed it in three weeks, right? And so I was at that point back to, OK, I'm, I'm doing all this management stuff, but I'm 4:47 actually much better at the other stuff. 4:50 And like, I'm not a bad manager, but I'm better at the other stuff. So it makes sense to at least spend part of my time on that and also like it a lot. 5:02 And I also noticed that when I was like, I always call it with my feet in the mud, right? You pick up signals, right? 5:14 Almost going on within a team, what works, what doesn't work? 5:18 And when you're in a pure management role or a pure agile coaching role or something, I really missed that during that time, I, I tried to help teams, but somehow I didn't pick up the signals. And when I was in the team again, I immediately see, OK, I've been trying to solve XYZ 5:33 the last half year. And in three weeks, I, I solved the problem, right? And we found a much better solution together. So yeah, so I think that there's sort of combination being deeply involved with the technical stuff really helps you be able to do a better job as a like 5:54 socio technical engineer, right? So yeah, I think that's that combination is really strong. And I don't see a lot of people necessarily doing that. Like a lot of people seem to be very comfortable in there. I just. 6:07 I'm just. Contributor role, right? 6:11 I'm, I'm just doing my job at, at fixing go stuff. 6:16 But I think it's so much more rewarding if you can help the people around you so much better and create much more of an impact. In the projects that you've worked on from a hands on perspective, what are some of the skills that you've brought to the table that made in the 6:29 end that project successful or succeed? 6:33 So a bunch, obviously. So a big chunk is being able to quickly do problem solving. And it sounds very obvious, right? But a lot of what we do is problem solving and not just on the like how do I write the code part? But I think also especially on 6:51 when something goes wrong, being able to pinpoint where it goes wrong. And you get a feel for that right, when you've seen enough problems. 7:03 So I think that's the, the, the main skill, like being really good at problem solving. And that's something I think a lot of us I think are good at, but it's also something you can train and you can get better at. So I think that's the, the key 7:19 skill that that help there. It's a bit generic, right? 7:23 So problem solving is broad, but I think problem solving on the operational side and also having the interest on that part, not just on, I build something and it's done. 7:31 But understanding that when once it runs in production, right, it becomes more interesting because then you can learn if it actually works because it's usually different data or different behaviour or whatever. Yeah. 7:45 And then, then you can really, yeah, find better solutions for, for whatever you find, because the first solution is not the best one, right? The first thing you put in production is usually, even though you might think it's the 7:58 best solution, it's usually once it hits reality, you start finding out what what you really need because your users will see it and they say, oh, actually now that I see this, I actually want it this way. So that's one part of feedback, right? The other part is, well, maybe 8:14 it doesn't perform the way you expected it, or maybe, right, the data is different. So there's a lot of different aspects that become really interesting once you get that feedback loop going on. Problem solving. 8:25 You see it in the broadest sense of the word, as in we have to build something. How do we solve this on a technical sense? Something goes wrong in operations or in production. How do we solve that? 8:34 Users giving us feedback for something that we haven't built or built differently. How do we tackle that from that sense? I think, I think mainly the the fact that you you have to be really curious about how something works and not just assume that you know how it 8:49 works, but that it may behave differently than you intended. 8:54 And then being having a a sort of internal curiousness to, you know, dive deeper into something that you may based on assumptions, already know how it works, but you know, maybe you don't. How do you keep that curiosity high? Because for me it, it comes kind 9:11 of in, in peaks and in what the opposite is of Pico. 9:16 Yeah, I, I feel like I I've had it a lot, especially early in career. But then when things get be quite comfortable, then I feel like I need to step out of that comfort to get curious again. So it's an active curiosity for me. Yeah. 9:29 So sometimes it's come natural because it's just fun stuff that you're working on, right? Yeah. 9:33 And it's usually with new or, or exciting or innovative stuff. 9:38 And sometimes you have to trigger it, right? 9:40 And then you have to do exercises or, yeah, maybe, you know, maybe you, you have a, a way to surface some data that may not be what you expected. And then, right, your curiosity usually kicks in. Like if I look at I'm looking at 9:59 the actual data now and I see something that I didn't expect it and then right it it usually triggers something. 10:04 Gotcha. And the trick is usually what I see teams doing is right. You build something and then it's done and you build the next thing right. 10:13 You work on stories and then you pick up the next story when it's done, and it's done when it's deployed. 10:18 Or even worse, it's done when it's at the tester. 10:21 But well, it's a different topic, but so you try to add another aspect to it. It's not done when it's in production, is done when you've seen it work and when you've analyse it. And sometimes you need a bit of time for that, right? So usually you have a story, you 10:40 build it, you deploy it, and then at least come back to it like 2 year, 2 weeks later, look at the data. 10:47 And that's usually will trigger some curiosity. 10:50 But I don't see a lot of teams doing this, right? 10:53 We're under a lot of pressure to ship new stuff, yeah. 10:57 Constantly. That's the finish line. 10:59 And the finish line is sort of, but I like to not think of it as a finish line. It's just that the finish, like the finish when it's production, that's actually the starting line. Gotcha. 11:08 Right. That's where it gets interesting, because that's where you Start learning. 11:12 So the projects that you've worked on for Ingredients of Success, you mentioned people that are curious, people that are good at problem solving, tell me more what else is in there? You need a mix of people. 11:24 So that's really my my learning. Also again, very early on in my career, I was at what we then called full surface web. 11:38 I don't know, we built websites right back around 2000s and 11:44 state-of-the-art stuff back then in Netscape couldn't do much, but we made it do stuff you you couldn't imagine. 11:52 And the reason for that, that we made such amazing stuff is that it's not because we had good programmers or engineers, it's because we had a combination of really good designers and programmers, right? And designers, they thought of 12:07 really crazy ideas for what a website could look like. 12:11 I remember you built something for a Litanice which you know, it was totally yellow and the whole screen was yellow. 12:20 And they created custom chroming, what we call sort of custom window boundaries and and close buttons. 12:27 You couldn't do that back then, but we found a way to do it. 12:29 Like we made a window pop up, open in full screen and then we scaled down so we didn't have any Chrome and then we put our own close buttons and scrollers in there. 12:40 Crazy shit. And right, so it was the combination of the fact that the designers came up with these crazy ideas and the and then we as engineers, we didn't say, no, that's not how the browser works. 12:53 No, we said, interesting, that looks cool. 12:56 Is there a way we can do that? And then we tried it out and we found a way and we made it work, right. 13:02 So, and the designer could never get that working on their own, right? And we could never come up with that crazy idea or not because you would just build some crappy UI. But the combination was super cool. So that's really my learning at 13:14 that point is when you go alone, but you, you're never going to create super cool stuff. So you need different disciplines. So yeah, that's, that's really what I tried to do also in everything that I that I get the chance to create, right? I tried to make sure that we 13:32 have good engineers. We have both back end, front end or whatever we need mobile, you know, the combination of skills that we need a few people that are really specialized in some areas, a few people that are very broad because then they can 13:48 fill in all the gaps and people can learn from each other. 13:52 I think that's also an important aspect. 13:54 Let's get back to that and then, and then also make sure that like either you have like designers or UX Rs or those kind of people in your team, preferably sometimes the organization is not structured like that and they're in different teams, but then at least have very close 14:14 connections with them, right? Build up the personal connection and then make sure you involve them in a lot of stuff. 14:22 And then also like I had teams where we had, we were launching a new product. So and a really important part was doing marketing and doing like crazy marketing stuff, right? So we actually got a growth hacker on the team. And so you need to really figure 14:38 out what is it I need for this job that we're doing at this time and then make sure that the right people are there. 14:45 And either I've also been in a sort of a start up mode for quite a while. And then you don't have the luxury because you don't. Yeah, we didn't have the money to hire all this kind of people. I did get a UX person and some like broadly skilled people, engineers. 15:01 But then essentially we learned all this stuff ourself, right? 15:05 We did our own website build. We did our own marketing stuff. 15:09 We did have a sales guy on the team. 15:11 So then between the like, we were seven people between all of us. We had many different tests when we got acquired that later on, I was writing down the due to elegance documentation, like OK, how's the team? 15:25 What are the roles of the people? 15:26 I don't know. It's like at least like 5 to 8 roles behind every person, right? 15:31 So that was crazy. And so, yeah, so you can also do it all yourself. And no, don't be scared to do something. You're not the specialist in if and he's done and he's done, if you send it to some other team or other departments, I mean, they're bound to have other 15:50 stuff to do. They're not going to give it the priority you need. Yeah. 15:54 So maybe you just do it yourself, maybe involve them, but at least make sure that you can continue and do the stuff that you need to do. Yeah, I feel like I've, I've really enjoyed working in, I think, quite large teams where there was a diverse set of skills but also cultural 16:09 diversity. That was all the fun. 16:11 I learned a lot from that. I feel though, the team that I was most productive in was really just a team of me and two other colleagues where this was, I think quite early on in my career still. But I learned a lot. 16:23 And through actually forming those relationships, which were quite close. We could trust each other, we knew how to communicate with each other, and we could really just focus on making impact and getting getting shit done, which was really cool. The small teams are the most 16:34 fun, right? Yeah. 16:35 At least that because you have such a close connection. 16:37 Yeah. Yeah. 16:38 So 3-4 people, that's for me. Also the like, super nice to be in team like that. I'm not in that situation a lot anymore, but sometimes, yeah. But depends on what you need to do, right? If, if, if you have the skills within that team to do what you need to do. 16:55 I mean, that's brilliant. That's super fun. 16:57 But yeah, in a lot of cases you need a bit more people to get everything done. Also, sometimes the amount of work just is more and when you're smaller, you go faster. 17:07 But if you have more people. And you organise it, well, you can still go even faster and yeah, so or do more stuff or whatever. I felt a little bit selfish though, because I was the person that was the most early in my career. So I also felt like when we're 17:21 looking at a learning curve, I felt like I was learning the most. And I don't know how much they were learning from me as an individual. 17:27 And even though they said XY and ZI don't know if I believed them necessarily. But when you are in teams, do you feel like you still learn and grow from the people around you? How do you view that still? 17:38 Yeah. Obviously. 17:38 So I think that's a misconception, right? 17:39 So if you're the junior in the team, let's say. 17:44 I often get get people asking these like, yeah, but I then I'm pairing with these experienced guys and I'm learning how, but I'm not getting much back. I don't think it works that way because in my experience, the the more experienced people, 17:58 right, they also have a lot of baggage and they have a lot of ways of thinking in, in their own mindset, physiological, because it's shaped by their experience. 18:08 That's how our minds work, right? 18:11 And the. Let's say younger, fresher people. I always feel I have a lot of enthusiasm for stuff right And obviously depends on personality, but. That's what you what I typically see and that energy is usually contagious. 18:31 So that really adds to a team and can really revive. 18:36 I say A-Team that's maybe. Doesn't have too much energy anymore, right? If you put a few young people in there, that's usually a really house. 18:46 And then also like the other day, I was working with another guy in the in the team, like the youngest guy. 18:52 And I always try to take out these people and I learned a new shortcut. Right. 19:00 I saw him do something. I thought, wow, what did you do now? I used this shortcut. 19:05 Oh go. Like I'm doing that like 20 times a day. Like, why didn't I look that up? 19:11 It makes total sense to have that one. 19:13 So I immediately wrote it on a sticky, put it on my laptop actually, and then yeah. So learning a new shortcut. 19:20 So it's small things like that, but it's also, I think the fact that you don't have all the baggage in history then means that you have a very fresh view also on the problems that you face. So if you are in a teeny, you've worked on a piece of software for like 10 years and some 19:39 problem pops up, there's going to be an obvious solution that comes to mind because you know this software very well and there's going to be obvious solutions to that based on also the stuff you've done before. But maybe the stuff you've done 19:52 before is not the best for the current situation because the world around us changes. So the software needs to adapt as well, right? In these people with a fresh mind, they don't have that baggage, right? 20:02 They don't. Know how it's supposed to work or that it's supposed to work in a certain way or that it was thought of like 2 years back like this is how it should work maybe this is how it should work is not true anymore in today's world right? So that's critical view, or at 20:21 least someone shouting stuff that's not the mainstream or the obvious solution can trigger people think, wait a moment. 20:30 Actually in the current current scenario that we are in, maybe what you're saying is much better than the old stuff we thought of years back. And I see that in a lot of teams, right? There's legacy in pretty much every place you're running into today. 20:48 So there's a lot of this existing stuff in there and existing I DS and having just fresh I DS thrown in is super valuable. It I also feel like it needs a certain type of senior person or personality because if you build up seniority, then you build up experience. 21:07 So then the decisions you make, you feel like they are grounded in the experience that you have. So then for someone else to come in and be like, why don't we do XY and Z? 21:14 If you don't believe in that, you still have to be the person that can say, OK, let's validate this or let's not shut it down and let's still try or at least have a dialogue. 21:23 Yeah. So this is super important. 21:25 So this is something I, I realise I'm not sure when exactly, but when you get more experience, at some point I had this realization that I have my own experience, right? 21:35 And I have my own biases based on whatever I experienced. 21:40 So I. Really try to be very conscious on where the IDs that I have come from rather than the fact that especially when they come into a new team and they have a certain set up and I have I DS like how and our release management should should look like or how their branching 21:58 structure should be or whatever. Yeah, but that may not be suitable for that project. So I really try to force myself to have an open mind, listen a lot, hear what people are saying. And then maybe I still have an ID that that doesn't get shouted out and I'll throw it in. 22:14 But it I try to evaluate just like my ID is just as valuable as the IDs of the rest around me and me as a key, key skill, to be honest. And some people are very stubborn, right? And then they don't have that skill more or less of it. But trying to be very conscious 22:38 about it. Trying to be very, very open and open and very conscious on, on when. 22:46 You're. Putting an IDs out and whether they are fitting or not and also the IDs that you're hearing whether they are fitting or not. And I'm assuming your own ID is the best and it's so easy but it's so wrong, right? 23:03 Also assuming that your first own ID is not the best, right? 23:07 Probably the. 3rd or 4th is going to be better, so just forcing that continuous iteration on IDs. 23:16 Sort of doing constant like diverge, converge phases with the team, right is really important. 23:23 And then as an experienced person, you can put your ideas in there. It's fine as long as they're at the same level as the rest and as long as everybody gets hurt, 23:34 yeah. And then you you find much better solutions. What, what do you tell people that do not subscribe to this idea? 23:41 Because I do. But if I play devil's advocate, I can see people say, well, I'm senior and I've built up excellent years of experience. These people only have a couple and therefore this decision should hold more weight, right? 23:52 It's not a democracy. It cannot be kumbaya. 23:54 In the end, we are creating software. 23:56 It needs to be effective and valuable. 23:59 So I can see the other camp as well. 24:01 How do you convince them or how do you engage in that conversation? So there's two things. 24:06 So one is I would use tools like collaborative modelling or anything where you start the session with giving everybody the chance to silently write down the IDs they have, right? 24:21 You, you post the, the problem and then usually stickies or something, right? You, you write down a bunch of stuff that everybody in silence writes it down. 24:30 Everybody gets a chance to put their voice down, right? 24:34 There's always going to be people with a stronger voice in a group and with a less strong voice in a group, but their opinion is not necessarily stronger or less strong or their the value of their opinion, right. 24:46 So I want to get all this stuff out. 24:48 And then usually what happens is that the person, the stubborn person, they automatically get to see what the other people thought. And sometimes it works, right. 24:57 Sometimes they see, oh wow, that's actually quite a good solution and I didn't come up with it. 25:01 So it starts to to force them essentially into seeing that there's other solutions. So just it's like visualising servicing the solutions or multiple solutions helps. 25:13 The second thing, now I thought of something, but now I have to come find it back, right? It was about convincing people. 25:25 That don't subscribe to their ideas. 25:27 It's people have certain personalities. 25:31 So what I tried to do is just as I realised for myself that I have experienced coming from somewhere. 25:38 I understand that their stubbornness, their mindset, their way of thinking also comes from somewhere, right? 25:45 It's not that they decided to be like that. 25:48 It's the experience that they had that shaped them in that way. So that realization, realization, it's almost like empathy, right? 25:58 For the situation they're in. Like it's not their fault that they're thinking like that. It's just the experience that they had that made them think like that. 26:07 So then I have to come up with experience that I can give them that will change their way of thinking. 26:14 And so some example I just gave is one of those experiences, right? You could also be just talking to them about this, right? Making them understand that they're in the situation and that I accept and understand that they're in the situation of where it comes from, that it's 26:30 OK, but also that it's hurting the group in a certain way. 26:35 And if they understand that at some point, then well, usually you can get as you don't. Yeah, be careful not to get people right to to attack them, because then natural reaction is to push back, push back and you don't get anywhere. 26:50 So you need to just have an an honest conversation. 26:53 Gotcha. Yeah, we discussed a lot about the what the team brings to make a project quite successful. 27:00 But I'm curious from your perspective, looking at system design, design or systems thinking, what are the ingredients that the system should have to make a project in the end successful? It's interesting, right, because there's a lot of stuff that from architecture point of view, 27:14 right? You, you know that a good chase system should have. And on the other hand, the systems that are still there are not all looking like that, right? All the big legacy systems, right? They're not necessarily necessarily the nicest in the nicest shape, but they're there 27:37 for a reason, right? They run, they run, they survived. So I'm not sure if you ever heard of the stuff from Barry O'Reilly, it's called we'll get to it anyway. What what he says. 27:53 He is sort of a very, I think different and innovative view, I think on how architecture should work. 28:05 They're very different from what's currently mainstream and essentially says it's really hard to predict our system will evolve. We can't predict the future and we don't can't predict what's going to happen with a system a 28:23 year from now or next week. Even if you wanted to survive, We need to create elements that will allow it to survive. 28:31 And his theory, so he's been studying on the stuff and, and doing excellent talks on it. So his theory is on you sort of try to create a random simulation of just throwing stuff that could break the system at it. 28:48 See what the effect is, and then come up with some component you can add to the system to make it be able to survive that that issue, right? Or that that challenge that you send it. So he calls it stressors. 29:03 And then the theory is that the theory is that there's only so many positions that a system can end up in that that are going to be stable. There's a lot of like infinite number of possible positions that it could end up with, but a 29:19 lot of them are going to be unstable and it will fail. 29:22 It won't survive, but there's going to be some that will survive. And those are very sort of almost independent of the stuff you throw at it. 29:34 If you throw enough weird random stuff at it, you'll arrive at this unique point where it will survive. 29:41 And if you throw something really weird else at it, there's going to be so many components that make it survivable. 29:48 It will survive that weird thing as well. 29:51 So it has the fire breathing lizards example. 29:54 Like if you what if you have fire breathing lasers in your data centre, will your system survive? 29:59 So what do you need do you need to do to cope with that and all sort of really random stuff. And if you look into it, it like the stuff that comes out of these kind of sessions. 30:13 If you look at it this way, it's, it's really amazing how much weird stuff you can throw at it where it will survive. 30:20 So what does the system need to have to survive or a right to to be successful? Yeah, look at those legacy systems. Appreciate them a little bit more, even though I don't like necessarily working on them. 30:36 Like Greenfield's much nicer until you make it your own legacy, right. I've been there, right? 30:48 There must be something in those systems that's helping them survive, even though they're not perfect. 30:53 And then my approach is usually to find that and then also to try to think of more things that we could throw at it and find ways that it could break and then make sure that it's, it's more survivable after we after I'm I'm gone, right? 31:10 And that may be stuff like technical components, right, that you need to add. But I think in a lot of cases it's knowledge that you need to add some knowledge about how it works and not just the knowledge that you write down. 31:23 So the test that you write or the documentation rewrite. 31:27 And I don't do too much of that, but especially I think the knowledge in the, the minds of the teams usually like when you look at a system and, and it becomes legacy. 31:42 So what's the reason it will become legacy? 31:45 You and I. Yeah. 31:47 So what do you would you think that a system would make a system legacy? I don't know what makes a system legacy, but one of my theories is the reason why maybe how a system is designed and what we now call legacy, the systems we 31:59 don't really want to work in, they are running and they have solved the problem and they might not be at a stage where they are evolvable as quickly as Greenfield projects, but the fact that they have, they prove value. 32:13 Otherwise we wouldn't run and maintain them in production. 32:16 That's kind of their right of passage, that's their right of existence. That's why we still have them. 32:22 If they were not delivering value, then we don't need the systems. We would shut it down in the end. That for me is quite interesting because if that assumption, if that theory holds true, then the way we get to that, regardless of what you choose kind of early 32:37 in early stage projects, you need to get to a point where you deliver value, where you have proven value. 32:43 And then it's a stage of, OK, how evolvable is the system and how much do we want to evolve it to continuously add revenue. 32:50 That for me is kind of the balance in how you design systems. So you you put the hammer on the nail, right? So evolvability is a key aspect, right. So when a system becomes legacies because we we don't like, it becomes hard to change it, right? We don't like working in it. 33:11 And it's sometimes because we don't understand it because the code is someone else's code is because fixing bugs is super hard because you don't don't understand how it works, right? 33:26 Software, you already heard in the, in the word, it's software, It's not hardware, it's soft, it's flexible or supposed to be flexible, right? And that's why a lot of the stuff on DDD and code quality, it's all about improving the 33:41 code to be more readable, to be more understandable, to be commutative, to allow the system to evolve. 33:48 I think in the end, like the, the, the biggest ground principle behind all these things is making systems that are evolvable and easy to change. 33:59 So how often do you have a software system where you don't need to change anything, right? I mean, adding new features maybe at some point becomes less, but you still need to be able to fix bugs because there's going to be bugs, there's going 34:13 to be stuff that goes wrong, but also the world around the system, the organization around the system will evolve and change. So if the system stays the same way as it is and is unable to evolve with the organization around it, it become less and less valuable, right. 34:31 So you say exactly that the value, how the system is valuable, what that value, how valuable it is will diminish over time if it stands still. So evolvability is the key. 34:43 And I think what we what we look at things like quality, modularity, all those kind of things. 34:54 And in my opinion, something that's really overlooked a lot is this knowledge aspect, right? And then especially the knowledge in people's heads. So how much of A theory do you have ready on how things work? If you, if you worked on a 35:09 system very long, right, you know every corner of it, maybe not maybe you know a few corners, right, But you know, generally how it's structured. You know, it's, it's rough architecture, you know, if something breaks roughly where 35:22 it's going to be broken. And there's really nice story on, on this is not something new, right? 35:32 We've known this as an industry for very long. 35:34 And there's a guy called piece an hour who in the, I think 1985, he wrote a piece on programming theory building. 35:45 And he expressed exactly this, like the fact that if you look at how we look at software and the, the view back then was more like the production view. Like we have specifications going in and then we have programmers, which is like the 36:00 cogs in the machine and then outcomes code and then we can run it. And then that's it, right? 36:06 That's how the software production machine works. 36:09 And I think we still have a lot of feature factories today. 36:12 So sadly that's still how our industry works. 36:18 What now are noticed is that if you looked at actual projects that the real world didn't seem to work or match with that production view. Like if a like you saw a system where there were was a team of like engineers working on the 36:36 system day in, day out. And then they, it was installed at some, some customers and then there was support crews who were maintaining it on that location. The support teams, if there was something wrong, they never figured out how to fix stuff. 36:52 They always had to come back to the like core team and they were able to always fix it in like 10 minutes. 36:58 So what's the difference, right? Because they had like both teams had access to the same documentation, they had access to the source code, like so all the explicit knowledge, the stuff that you can write down, that was all there, right? 37:10 There was no difference in that. So that was not the differentiating factor. So the code and the documentation, the artefacts, the output of this software process was apparently not sufficient to be able to maintain, evolve and fix stuff, etcetera. 37:27 And then on the other hand, the people that had the writing knowledge, because they worked in it, they had the writing knowledge in their brain, the experience. 37:34 So it's like riding a bike. So there's this tested knowledge you when you learn to ride a bike, it's not by reading a book. It's really hard to learn too hard. Write it back. 37:43 It's something you need to experience. 37:45 You need to get a feel for the balance, right? 37:47 I can tell you just go sit on it, move your legs and then you don't move your arms too much. It's just feel the balance, right? It's you have to get it build up that feeling for that balance, right? 38:00 So this, that's a, that's a knowledge. 38:04 And as programmers, we build up this kind of tested knowledge for the systems that we work with, for the code that we work with. And when you have the knowledge, fixing a bug somewhere is super easy because you have this ready 38:18 theory of how things work. So if something doesn't work the way it's supposed to be, you know, well, it's probably in this area. So this was what now we saw, right? That there, there's this production view, but the code artefacts just not enough to be 38:34 able to work on a system. You need that theory of the programming theory. Programming is you have to, you have to have built a theory to be able to do this. 38:48 And the theory lives in your mind. 38:49 And it's not something you can say you, you pluck it out and you put it in someone else's mind. 38:53 How do you transfer that? You would transfer that by working together on the software, right? 38:57 So you have multiple engineers working on the system. 39:01 And when a new guy comes along, he'll have a lot of trouble figuring out how it works. But if he works in that team for a while, you'll learn and you'll pick it up and you'll learn how the system works. You get the the details. 39:12 At some point you build up this tested knowledge. 39:17 And we don't really, I don't think we are very aware of that this works this way, right? If you think of this theory building, it really matches. If I look at all the projects that I've been, it matches completely with what I've seen, right? So when I when I heard of the I 39:33 only heard of this like 2 years ago, I uncovered that this theory existed and this narrow guy existed, whatever. 39:43 So it really intrigued me. So I dove into it and tried to learn more about it because it matched so much with what I saw around me. And this is like knowledge we've had for a while and somehow it didn't reach me. 39:54 I don't know, like there's just so much we know from years back that's still relevant, apparently. 40:02 And yeah, so how do we become aware as a team, right, that this is something that's apparently happening and that being successful in software and being able to keep evolving, the software apparently needs human humans to interact, right, to 40:23 build up this knowledge together on a system before they can be able to work on a system. Then when you look at current, a lot of the projects that I come across, right there is people, especially in the current nationally getting getting worse, people going with pension, right? 40:39 So that's well deserved. But they have a lot of knowledge that then leaves other than there's people job hopping. 40:49 So that's knowledge going away. So there's a lot of lot of knowledge leaving the company, you could say constantly. 40:56 Yeah. And if you put the theory building view on top of like the legacy systems we see, like if there's a system where the whole team that worked in it isn't there anymore, it's a system legacy. 41:10 Oh yeah, definitely right. Because nobody knows how it works anymore. And it may not just be the the programmers that left, if the operational people that run it are no longer there and the system's still running, but nobody knows how to maintain it. It's going to fail and nobody 41:25 knows how to fix it. It's going to take a lot of time, no? One wants to touch it. 41:28 Nobody wants to touch it, right. So you, you can't change it, you can't upgrade it, whatever it's, it becomes really sort of a, a big risk. So it's a huge risk. 41:37 And still as an industry we don't seem to recognise it so much, even be aware of it. And also from AI would say from an architecture management point of view, it would be really I think beneficial to optimise the way we organise teams, the way 41:55 we grow and shrink teams, whatever, right, to keep at least the sort of the minimal tacit knowledge part available so it can survive. So the system is more than just the technology, right? And the code is the social technical system of the system and the code and the people that 42:12 work on it and the people that use it. 42:15 And if one of these goes away completely, yeah, the the system's sort of doomed. In the kind of current day and age, there's a lot of at least marketing buzz with regards to personal productivity with genetic tools and specifically your mention of retaining knowledge. 42:35 If the knowledge is gone, then the system quickly becomes legacy. I feel like the way of working that we now have has a risk. And I'm not saying it's it's 100% going to happen, but I feel like a bigger risk that people don't really build up the knowledge. 42:50 If they do it in a certain way, then the knowledge of the system will not necessarily build up. There is a different way of doing it where you specify requirements. 42:58 I feel like that's maybe the the future we will go into where you are very much conscious of knowledge that you capture somewhere and then you let something else build it. 43:07 But if you don't do that and it's kind of a black box, then that knowledge will never build up, which means likely it will become legacy quicker. So yeah, I had exactly the same observation. So also been thinking like, how 43:21 do we tackle that, right? Because it's this hygienic stuff is amazing. It's really cool. 43:27 It can really help us in a lot of ways. 43:29 But if we don't understand how to use the best, it's also going to hurt us a lot. And exactly the example you say, like write down better specifications. 43:42 The risk with that is that it's it's explicit knowledge, It's written down knowledge, It's not experienced knowledge. 43:49 So like Peter now would say, sorry, doesn't work. 43:56 So that's not I don't think that's a full solution. 44:00 I don't think it hurts, right? I think it's good to do that. 44:02 And that approach thing, I think better than just vibing all the way and then yallowing and seeing where it lands. 44:10 If you want to build something sustainable, right? 44:14 But you need another ingredient, right? 44:18 You need that. That's the knowledge part. 44:21 So how do you capture that? How do you transfer that? 44:23 How do you keep that alive? And there's a bunch of factors, I think that made it really hard in today's age. 44:30 So one is after COVID, a lot of people are working remote. 44:35 We're much more isolated than we were five years ago. 44:38 Yeah. So knowledge, this test and knowledge transfer is much harder, right? 44:45 You can, it works really well when you sit together and when you can just ask someone a question, whatever. 44:51 If you have to go into a Teams call to do the question, it's not the same, right? So that's already a factor. 44:58 I think that's a big risk that we have at the current moment. 45:02 The other factor like these decident tools makes it make it really easy to think that you have this bearing partner that can help you build something. So people also don't necessarily go work with a buddy anymore, much like a human buddy, but 45:15 they work with their AI buddy and the AI buddy. 45:19 The context is super important. Context like this test and knowledge is your your own context, right, your human context, your AI. But he also has a context, right, But it loses the context. It can't it can't grow too big 45:34 because then it's it's well, the results don't get too too bad, too good anymore. So unless we tackle that problem, right, and there's some idea of memory, right, which is sort of getting better and better, but the current state of 45:51 enginic tools, they're not going to help us much with this, right? They're just going to make it worse because they produce a lot of stuff. 45:59 They don't retain this tested knowledge. 46:03 I think it's also really hard for an agentic tool to capture an agentic tool or to capture this tested knowledge. 46:10 It would have to have a similar experience as we do, right? 46:13 It would have to live through the consequence of what it builds. So when you do something and you create it and you don't have a production issue, right? 46:23 You. Experienced that consequence, right, You'll learn from it. The current agenda tools, not a lot of them do this right. There's some tooling out there that looks at production stuff, super cool by the way, and tries to figure out what's going on, tries to to to fix essentially 46:44 there like do a root cause analysis, fix stuff, but there's not the learning and writing down the learnings part yet. 46:52 And I think with all the agile stuff, we've been getting better at learning, right? But we haven't done that in the agentic stuff yet. So yeah, I think that's a big challenge still there making sure that there's actual learning going on, actual like safe sandbox iteration with 47:10 feedback loops towards the agent again. 47:12 And not just building and then testing, but also actually running and seeing what it does in production, the whole chain. 47:20 And then you also introduce a lot of risk, right? 47:23 Because if you connect your agent to production to read logs and stuff, yeah, it might log some stuff that you don't want it to know. So it's not an easy thing to solve. But still, I think this is the thing that makes a human really good at having this built up 47:42 knowledge and working on systems effectively. 47:46 So if you want to get that with these agents as well, well, there's some stuff to do. Yeah, for sure. 47:52 One of the last thoughts I had was when you were discussing the evolvability element within a software project, because my question is mainly around timing. 48:02 I've been in projects where people have early on made conscious decisions to say we put down this specific architecture or this system design to be able to evolve in the future and that future might never come. 48:14 And if I as an individual within that project really experience kind of difficulties with regards to the speed and the productivity that I want to have due to these decisions that I don't know if will ever come, that leads to some frustration 48:28 sometimes. But then when that future comes, then it's quite nice because then you've covered a solid base that you can actually catapult on top of. 48:36 So this is for me quite a fine balancing act. 48:39 How you how have you balanced that in the past? 48:41 So. Definitely look into this stuff from Mary O'Reilly Right, because it's, it's good stuff if you want to find the right balance there, right? 48:50 It's it, it adds complexity if you add a lot of stuff early on that you don't need in the end, it's just complexity that you're dragging along and it makes yourself harder to change. 49:02 And I've definitely done that, right. 49:03 I mean, I've built super nice generic flexible components, whatever. And then I needed only a small piece of it. So that's, yeah, you'll learn from that. And my learning, like the stuff I build nowadays is always the minimal, just what I need now. 49:20 And I make really sure that I I make it in such a way that it's easy to change so that when that future comes, I can just change it. So I think that's the biggest lesson I've learned on that part is just optimize for what you know now. And I think Dave Farley says 49:41 this really well is right. You want to assume that you're going to be wrong like that the whatever you built is going to be wrong, and you're going to optimize to figure out what is wrong as soon as possible. So you built something, you 49:57 assume it's wrong. And to be honest, usually what you built is wrong in some way. And then you optimize your feedback loop and then you learn and you make sure that your code is easy to change and then you change it based on what you 50:11 learn and you iterate that as fast as you can or as fast as useful. But I think that's the the key, key ingredient there. So assuming you're wrong because that's already going to help you like N fall in the trap fund, we're going to build a super amazing fancy stuff that's going 50:28 to be able to deal with the future. 50:30 That assumption is already wrong, right? 50:31 Because you don't know the future. 50:33 So being able or being able to realize that it might still be a a good approach for the time being, whether maybe take out the stuff that you don't that you don't need right now. 50:47 Just do the stuff you. Need right now and then just learn. I like that a lot, yeah. 50:53 It made me reflect because the the easier to change for me that needs to then be explicit because my colleagues, for example, that added the complexity. 51:01 Their argument was also what it was. 51:02 This will be easy to change or rather kind of easier to build on top of because that's what they're optimizing for a solid base to catapult on top of. And I like that you mentioned that what we build, if we build it, build it simple. 51:15 By virtue of it being simple, it's easy to change. 51:18 So it can evolve from that aspect as well, which is kind of these two angles. And that's where I feel like a lot of my conversation went into because we both want the same thing. We both want a system that can evolve, but the approach is very different. 51:31 Yeah, that's quite. Interesting. 51:33 You may still end up with the solution that they came up with in the start. Yeah, right at some point later. 51:38 But it's just like with TDD, right? 51:40 When you do actual TD, you start with a very small, very wrong implementation and a test that makes it. 51:47 But then you add a test that makes it fail. 51:49 And then you figure out, OK, I need to change it a little bit to make the second test pass. And you build up bit by bit a piece of working software that does only whatever you wrote in your tests. And if you take the same 52:03 approach to your architecture, right, you do something very simple that you know will work for what you need now and then make sure that you can adapt it, evolve it. 52:11 And that's not always as easy with architecture, right? 52:14 Some, some people say like architecture is the, the hard decisions, right? The decisions that are hard to change. SO1 solution is to right make those decisions and then live with them. 52:29 The other solution that I really prefer is don't accept that and make the things that are hard that is hard to change, make it easy to change, right? So let's say database schema can be really hard to change. Well, then use tooling to make 52:42 it easy to change, like use some fly away or something else. 52:46 There's tooling enough to have an evolvable schema or maybe you don't use a relational database, but something that's easier to to change if you don't know the exact schema yet, right? 52:55 So there's a lot of solutions that people think I need an architecture decision record because this is a hard thing to change. Like maybe the programming language is a good thing to to write down. 53:09 Although even with the Genting tooling, maybe there's options. 53:12 Switch. Yeah, I have options nowadays. 53:13 So like stuff becomes easy to change. 53:16 So just make sure you make it easy to change and don't accept that it's hard. Yeah, definitely. 53:21 Thank you so much for coming on, Nikkei. 53:22 I really enjoyed this. This was great. 53:24 Thanks, it was a pleasure. All right, if you're still with us listening, let me know in the comments section what you thought of this episode and we'll see you in the next one.
Community discussion