In the first episode of the new season, I talk about Problem-Solving skills and strategies for developers. As we all have a common goal of solving problems with software, improving our skills and using various strategies are crucial.
I talk about a clear strategy to solve almost any problem we face. We can solve problems in five steps.
Find out the real problem
Clarify the problem and expectations
Simplify the problem
Create potential solutions
Merge and evaluate solutions
In five steps, you can solve many problems—big or small.
Transcript
[00:00:00] Candost: Hello, everyone. Welcome to the Software World with Candost, The new season, the third season, the first episode. And this episode, I'm going to talk about one of the things that we do as an engineer every single day. And that is: problem-solving.
[00:00:46] Our job as software engineers is to solve problems, business problems, customer problems, user problems, and many, many other problems that we focus on every day.
[00:00:58] Today, we are going to dive into problem solving and methodology, and systematic approach. How we approach the problem, how we analyze the problem, and how we create solutions to the problems that we face.
[00:01:12] So let's dive in. The most important part of solving a problem is actually understanding the problem. Like we work with clients who are not experts in our domains. That's a level of abstraction. Our clients don't need to know every single detail of engineering so that they can focus on only the parts that they need.
[00:01:35] It doesn't matter which team we work on. There's a vast language difference between stakeholders and the team. That's why extracting the exact problem and understanding it is crucial. There are tons of projects that went out of scope or delivered a different solution because the team didn't listen to the client, well.
[00:01:56] They assumed they knew better because they were the experts. If you take a look at it from another perspective in system design interviews, for example, it's precisely the same. If you don't analyze the problem that's given to you and understand it clearly, you won't be able to come up with the correct solution. So, our first step is analyzing the problem. Finding out the real problem.
[00:02:23] If your client or interviewer tells you that they want to scale their application to support 10 million active users, that is not a problem. That is a wish. That's what the client wants. That is not a problem. When we find that real problem, in the end, we might say they can't support 10 million simultaneous user requests coming into their servers. That is the problem. So we translate the wish into a problem. But that's not how we figure out the real problem, actually. That's our end goal. Right.
[00:03:00] So another example is if any team's problem is slow development velocity, and they want to speed up the engineering team, their problem is not the development velocity. If you ask business stakeholders, they can say that the slow engineering team is the most significant problem they have, but the underlying issue is most likely to be different.
[00:03:26] They might be slow because of the legacy and unmaintained systems or they didn't pay enough attention to improve the systems' operations. So engineers can't work efficiently and often they have to do many manual steps to complete a task. Hence errors happen in manual tasks and people spend more time fixing them.
[00:03:47] That's why our first goal is to identify the real problem. And often there isn't an easy way. It requires asking the right questions at the right moment. And it requires us to be. Good listener. But at the same time, not trusting our client's judgment and sometimes even our own judgment.
[00:04:11] So we need to start with asking questions. Asking questions is one of the most underrated strategies in software engineering. It doesn't matter if you are in an interview or in a client meeting, ask your questions with a goal to clarify the problem. Get as specific as possible in questions and stay as broad as necessary. Not more.
[00:04:41] Questions are the main elements of problem-solving. They pave the way for understanding the problem and the other person's needs. What people want and what they actually need often differ. And questions move these wants and needs closer and closer.
[00:05:01] Practicing asking skills asking questions skills is one of the fundamentals of the software engineering discipline. That's why we have StackOverflow, right? Like you ask questions on Stackoverflow. That's our fundamental. But how can you practice asking questions? That's another challenge, and many people actually don't even think about it.
[00:05:25] So I'm going to give a few suggestions on how to practice asking questions. Asking better questions.
[00:05:33] First, ask questions to understand why there is a problem. Try to identify the root cause. It's the same as trying to figure out a root cause of an incident. If you're trying to find the root cause of an incident, you ask questions. Can it be that one, can it be that one? What happens when we fixed this part? What happens when we roll back to this last deployment? What happens when we do this? What happens when we do that?
[00:06:03] That is identifying the root cause at least eliminating the options. That's why start with an intention to understand why even there is a problem. Let's use the example we used before about the scaling. The problem came to you as the project wants to support 10 million users. Okay. Cool. That's what they want. And we know that they can't support 10 million simultaneous users at this moment. So we know the problem. What we want to do is we need to ask questions. Let's ask a couple of questions to this problem. If we can get some clarification.
[00:06:44] Do they want to support 10 million users simultaneously? What does 10 million users mean? Do these 10 million people use the service at once or do they use it the whole day?
[00:06:57] It's a significantly different problem if 10 million users use the service at once in one hour period, then, if they are spread over the whole day. Our problem and solutions will differ according to that. Another question might be, do these 10 million people came to the service in waves, for example, peaking the service at certain times, maybe in the evenings, after work time or so. Or there are no peaks, but just the constant traffic coming in. Another question might be, do these users mostly read the information or do they also add some to the system, like how many redone right operations do we expect? That's one point that will change our whole system design.
[00:07:43] Another question might be, does the system require operational work from the staff? Like, do we need moderation for example? Do we want to build authorization levels and maybe another question might be, do we need authorization levels between 10 million users? Will there be a users who are moderators between those 10 million people? And also like, what are the security limitations we have to obey? What are the legal limitations we have? Do we want information encryption? If yes, then we need to ask many more questions then how do we approach the fault tolerance? These are all the questions that we need to ask to the client or if we're in an interview, our interviewer, right? If we don't know how to approach the fault tolerance we need to figure out, like, does the system need to handle failures automatically as much as it can with retry mechanisms or other strategies?
[00:08:38] Or is it okay to show users, certain error messages that they can understand? We need to ask all these questions and maybe even more depending on our timeline. And another suggestion from me would be ask with curiosity. Don't question, keep your humility and ask exploration questions.
[00:09:00] I have seen many, many people just questioning past decisions all the time. It's like, Hey, yeah, this was a past decision. I don't really know why people made this decision just doesn't make sense. Yes. It doesn't make sense now, but it made sense before like people make decisions according to the knowledge and vision in that moment. So they probably decided on specific strategies with the knowledge they had.
[00:09:25] Maybe even people who decided on these are not in this organization anymore. If you're in a big organization and maybe in a great organization that keeps decision documents then probably are lucky, and you can find your answers to your questions. But many organizations don't keep the comments like these.
[00:09:44] That's why you have to approach with curiosity to resurface the truth. Questioning the past decisions or just trash-talking won't have at all. Ask with an interest to collect data that may help you in your strategic decision later. Stay humble and stay in this exploration mode.
[00:10:04] And the last suggestion I have to finding our three problem is, there are no stupid questions. Every question you ask gives you the information you should consider. Or a path that you shouldn't follow. Each question teaches you something you didn't know. Sometimes you have to ask questions when you even know the answer, but you have to. Questions remove assumptions, which are the last things, these assumptions, we need in problem-solving. You might get rolling eyes as a response, you might annoy a few people with your questions, but it's all okay. You're not working for these people. You are working to solve a problem. You are working with them, not for them. Ignore the rolling eyes and keep your curiosity alive. So while asking these questions, like I mentioned before, get into the details of the problem. All information you collect will help you understand the problem, and find the optimal solution. Until you feel you have enough information, keep asking detailed questions. Get into the heart of the problem. Once you understand the problem, probably possible solutions will already begin to appear on your mind.
[00:11:23] Before you jump on the solutions. There are two more things you have to do.
[00:11:29] You need to clarify the problem and you need to clarify the expectations. Assumptions are the enemy of problem-solving. We form these assumptions to simplify our lives in general. So it's like a biological thing to do.
[00:11:43] However, when it comes to engineering, assumptions play a critical role in defining the success of the systems we are building. Once we understand the problem, we begin thinking we are aligned with the other site. This is more like a psychological thing. And even sometimes we think we know better because we are the experts, right.
[00:12:02] I've seen countless times engineers believe that they found the problem because of their analytical approach. But they skip aligning with the problem owner, their client, or interviewer. They just focus on solving a problem. But often that's the wrong one. It's not the problem to solve.
[00:12:21] That's why it's crucial to explain the problem as you understand it. We tend to bring our expertise to the table and showcase what we know. This temptation inside us nudges us to jump into unrelated solutions that seem like a solution.
[00:12:39] However, once we explain the problem as we understand it, the other side will give us either a confirmation or will correct our course. This piece, clarifying the problem, just explaining it solves many unexpected issues for the future.
[00:12:58] If I use the same example I mentioned before; maybe we can explain the problem with the information we have. Imagine we are designing the New York Times with a commenting function on. If we explain the problem, we might say, we want to scale the system to support 10 million people a day. These users will use the service throughout the day and we expect most of them only to read and a few of them to write.
[00:13:25] These people might use the service at once at certain events. And most of them will still read it instead of writing anything. Did I understand correctly? Did I get anything wrong? So, this is what we can say to just to clarify. You know, it just takes 10 seconds. But once we have the confirmation from the other side, now we can trust the clarity we brought to the problem, and we can go to the next step and clarify the expectations.
[00:13:57] Why expectations? Sometimes expectations can be different than it seems. You clarify the problem until now, but you might still not know what they want from you. Do they want you to advise on what they can do about the station? Or do they ask you to jump in and solve the problem by yourself? Like these kinds of things. Like asking about their expectations from you is a step that you shouldn't miss at all. It's also crucial to share your expectations. If you are working with a client and they ask you to solve their problems, you might expect some freedom on technical decisions, but if you don't tell them about it, they might block your halfway because you changed so many things.
[00:14:38] If you're in a system design interview, you can ask your interview her about their expectations besides designing the system. I mean, that part is kind of obvious that you're in an interview, you need to design the system, but you can still clarify some parts of their expectations.
[00:14:53] For example, they might expect you to use something in a certain way, in some part of the problem. Or they might limit you or give you some freedom. It's always better to clarify so that you can take precautious steps.
[00:15:06] Okay, we found out the real problem. We clarified the problem. We clarified the expectations. Now you think we are coming into the position where we can solve the problem, right? Yes! So after we analyzed the problem, our next step is of course solving it. That's like problem-solving. So the first part is the problem. The second part is solving. How can we solve the problem?
[00:15:32] The problems we face are often complex, but they don't have to be complicated. Once we clarify the issue we face, and the problem we see, we feel more comfortable solving it. The solutions are usually not easy, sometimes they are, but usually not. But they can be simple. Instead of solving a big, complex problem. We need to focus on smaller parts and solve them separately.
[00:16:00] Splitting the problem, simplifying it, and solving a smaller problem is much easier than focusing on the big, huge complex problem. You might have heard divide and conquer algorithms such as Quicksort and Merge Sort in computer science. We can approach the problem in a similar way.
[00:16:20] We can break the big problem into smaller chunks and work on them one by one. Let's use the same example again. Instead of thinking about a system to support both read and write operations, we first look at reading operations. When users read the same data, our problem becomes how to serve 10 million users the same piece of information simultaneously. Maybe under 500 milliseconds. Solving this problem is more straightforward than solving the overall situation. It's still a problem and the solution may not be easy. The solution may demand you to combine various strategies to succeed. But now, the problem is much simpler, and you can put your focus on this problem without thinking about the complex situation.
[00:17:09] Once you simplify the problem, you can go to the next step. Don't worry about the other parts of the problem. You just focus on one simple piece.
[00:17:19] As a next step, you need to create potential solutions. Now that you have a smaller problem. Solving it becomes much more manageable. Now it's time to shine a light on your urge and find solutions. Once you get into the solution mode, there are certain things to be aware of. Stay biased for action.
[00:17:39] Don't contemplate the ideation phase and get into the solution. If you're in a system design interview, for example, start drawing what you have in mind. If you're talking with a client, start to express your thoughts.
[00:17:56] Let's use the same example again: simultaneously serving 10 million requests per second, under 500 milliseconds. That's the problem. Caching might be one solution. Probably the first one that comes to mind. But there are more strategies we can find. We can use maybe bigger machines to handle more requests.
[00:18:15] Yet, probably that won't be enough. Like if he asked the right questions before, probably right now we have more information about the problem. For example, if we know our users' geographical distribution, we may prefer to serve our users from different machines that are closest to them.
[00:18:35] We can replicate our servers and use a load balancer to spread the load. Or we can combine both and use multiple servers, multiple big machines, and serve users from the nearest machines. All of these answers will depend on the questions we ask initially, and the answers we get.
[00:18:55] Within this solution-generating phase, push yourself to think of at least two more alternative decisions. This part is crucial. To alternative solutions. If you can only think of one solution, you're not thinking of the best one. If you find one alternative, what you have is a dilemma. You will probably make it a blind decision and miss at least a few perspectives.
[00:19:20] It's either/or if you have two solutions. Once you find the second alternative and have three options overall, you can compare your solution with the other two and recognize your solutions' pitfalls and trade-offs. So don't get stuck in one idea. Be open to changing your mind. Consistently rethink your overall approach.
[00:19:43] Once you discover any trade-offs, be flexible and determine the next path with your new information. Once you solved the small part of the problem you focused on, go to the next part of the problem. And go one by one, every small piece.
[00:19:59] The next step is merging and evaluating solutions. When you go to the next piece of the bigger problem and find a solution, stack that on the previous solutions and reconsider your previous steps. Think that like you are building a tower in Jenga. If you miss one of the three pieces, and you're already on the next level.
[00:20:19] Consider adding that missing piece. If the tower is not aligned well, it doesn't stay straight, and it's shaky, and you need to fix it. So you need to simply tweak your solutions that, put your hands around the tower and you need to strengthen the foundation. Right? Same as problem-solving, once you figured out a solution for this one piece and you are putting it on top of the previous piece, you need to correct the foundation. Until you finish the problem overall, repeat this process over and over again. Find a solution to one small part, and then put it onto the previous solution to see if they work together or not. If not fix that, improve that.
[00:21:01] So whenever you choose a solution over another, be aware of the trade-offs you're making. We tend to think we found the best approach possible in current circumstances and avoid seeing the disadvantages of our solution. When we judge the possible solutions, they're more analytical. However, when we get into the prediction of the outcome these tend to substitute to prediction with the evolution of our judgment and our intuitive predictions tend to be overconfident.
[00:21:31] Even though we create the solution as analytical as possible, considering all the options. Whenever we pick one, we are usually overweighing the value that we are going to get.
[00:21:44] So this is why we overlook the trade-offs. Once we are aware of this bias, we can correct our path and choose the best solution. So take a step back and look at your overall approach from time to time. Holistic thinking is one of the most powerful brain muscles that bring tremendous benefits.
[00:22:04] When we see how we are planning the overall solution, we can identify the pieces that don't fit into this plan, and the pieces that might create another set of issues.
[00:22:13] Now that we have a systematic approach to solving a problem, let's summarize this process. It looks like a lot, but actually, it's really less. First, find out the real problem by asking questions. There are no stupid questions. Whatever comes to your mind, ask with curiosity. Don't judge, the pest decisions. Try to understand them.
[00:22:33] Second, clarify the problem. Assumptions are the enemy of problem-solving. Explain the problem to your team, your client, interviewer, or whoever it is. Get confirmation from them that you understood clearly. And clarify expectations. Again, explain what they expect from you and what you expect from them, and get the confirmation.
[00:22:55] Third, simplify the problem. Slice the problem into smaller chunks. This way you will digest the problem more efficiently. This is the part we divide the problem.
[00:23:05] Fourth, create potential solutions. Now that you have many information and a simple problem, it's time to shine. Create solutions, but find at least two more alternatives to catch the trade-offs and bias early.
[00:23:20] This is the part we conquer the problem. Divide and conquer. Merge your solutions into your previous chunk solutions. Reevaluate your last chunks and check if your current solution fits the whole system. Approach holistically to the overall situation.
[00:23:37] In the end. You will have a solution that's created step-by-step. If you have smaller problems that you don't think this whole process might not be worthy of, try to do it as a brain exercise.
[00:23:50] You need to keep problem-solving muscles working to strengthen. You never know when you will need them. Once you practice this enough, this systematic approach will help you solve many many problems.
[00:24:02] And also after a while, it will become so intuitive. In five steps, you can solve any problem you face, either complex, or simple.
[00:24:14] And that's it. That's the end of today's episode.
[00:24:18] You can find all the show notes and the transcription of this podcast in the candost.blog, c a n d o s t.blog. And everything there. You can subscribe to the podcast and get the notifications on your email.
[00:24:32] You can also follow the show in your podcast app, and get all the notifications for the new episodes. If you want to reach me, you know where I am, I'm on Twitter @candosten, c a n d o s t e n.
[00:24:45] And I'm looking forward to meeting with you next time. Take care.
#27: Problem-Solving Skills & A Strategy for Software Engineers