The Psychology of Software Testing — Book Summary

Think this was always going to come sooner or later, and here it is. I’ve compilred my notes made around the book “ The Psychology of Software Testing” by John Stevenson.

The Psychology of Software Testing

Many books offer how to do testing but the idea behind this book was to look at the connection between psychology and software testing. Software testing is an area where a lot of human intervention is needed and with the help of this book the author wants to show how we can focus on our process of thinking even while testing our code. Over multiple chapters, the author covers different parts of the merger of psychology and software testing, like critical thinking, teamwork, creative work, and much more.

Chapter 1 Software Testing Biases

In the first chapter, we would be focusing on all the different biases which could cloud the process of software testing. A bias is any view, opinion, or belief that we form of outside words which shape our understanding of the world. We would talk about multiple biases and how we can prevent them.

Confirmation bias: Confirmation bias is when someone tends to notice and look out for facts that align with their belief and also to ignore facts and beliefs which contradict one’s belief. This is a problem while testing because a tester might assume that an area of code may be a low-risk one and might miss an important bug while testing. This would confirm with their view that the code is a low-risk code. The way any tester can prevent this is by having multiple testers go through the same code. Multiple people should not have the same bias and therefore should contradict at some point. Another way to prevent it is to do pair testing. If people are aware that there is a thing called confirmation bias then they can take steps to prevent it.

Cognitive dissonance: Cognitive dissonance is when we feel uncomfortable in our heads because of carrying conflicting ideas simultaneously. While testing if we have some expectation about how the software is supposed to work and if it gives some other result then we may change our expectations and that might be dangerous because we might miss some bugs. People might ignore bugs if they think that it does not match with the requirements as they are prone to different interpretations.

Conjunction effect: This bias comes up when 2 events are happening together or independently then the overlap of 2 events is given more chances of happening than the 2 individual ones independently. This happens because when we add more details to an event can make it look more plausible. This is related to testing because if we have multiple requirements in testing then we have to think about which one is more plausible. Some of the requirements and the test cases we come up with maybe clouded with this bias and therefore we need to apply mathematical probability to come up with the correct answer. Simply discussing the different requirements and thought processes with others help to prevent this fallacy.

Framing: Framing can be a problem when something is worded in a particular way and that leads to conjunction fallacy. How a requirement is framed could influence the development as well as testing of the code. Therefore as a tester, we need to be aware of this and while sharing the findings during testing make sure how the words are framed.

Illusory correlation: This happens when testers can correlate 2 independent events or outcomes when there is any evidence that there is no connection between them. This could be introduced in testing when testes working in isolation from hypotheses of how different software code is supposed to work in different conditions. One way to avoid this trap is not to work in isolation and involve multiple people in different stages while testing.

Cognitive illusion: This bias arises when we try to think something is real when it is not. In testing, this happens when we see something from only one point of view. Being aware of it and trying to test something from multiple points of view shall help in reducing this bias.

Patterns from nothing: This bias comes up when we try to find patterns where there is none. With testing, this could happen when the tester tries to find out patterns from the results but misses out on important information. One way to avoid this bias is to note down information and not try to find any patterns. Keep looking for more information and keep documenting it before analyzing the data.

These are just some of the biases that our brain has. We need to be aware of these biases and try to inculcate critical thinking into the testing process so that we can come up with a better testing approach.

Chapter 4 Quality: preference or perception

This chapter dives into the part of quality concerning the code being released and software testing. When we talk about quality assurance it is not that the testers are responsible for the quality of the code. That cannot be true because the code has already been developed. The testers are only responsible for making sure that the code has no problem and that it is meeting all the requirements. Therefore testing should be taken care of in the initial development phases and not really in the testing phase.

What is quality

Quality can be defined as a standard of something which is measured against some other thing of a similar kind. It is something that can be measured. The problem that arises with quality is that it can be subjective and can be defined by context. The tester can only provide information with regards to what’s working and what is not and therefore needs to know about his/her own bias before sharing the information. There needs to be a huge amount of communication and collaboration between the testing and the development team. We need to understand if the quality can be based on perception and preference.

Is this art?

An area where quality can be subjective is Art. But whenever we see some piece of Art we do not associate it with good quality and we should apply this same principle in software testing as well. A tester should be aware that what he/she may perceive as good enough may not be good enough for the customer. And therefore this should be used as a conversation initiator with the customer so that they get to know about their understanding of good quality code or software.

How much can you afford to lose if it fails?

Most of the management level people think about everything in terms of return of investment. The most frequent formula that the tool vendors use is the following

ROI = (Gains- Investment Costs)/Investment Costs.

But in testing terminology, we can also use

ROI Manual testing Cost — Automation Testing Cost

We like to see the numbers and accept them as they are but we must be aware that even they might be based on prediction biases. Therefore we should not directly believe those numbers simply because of the formulae. A good way to prevent this bias is to test whatever hypothesis we have, like if automating a test could save money or not. Imagine the worst that could happen and the risks which are associated with it.

Software quality of the future

The field of software development is changing dramatically with more companies are trying to push software in the shortest time possible. And with these rapid pushes, they also want to make sure that the software is of good quality. But with this rapid software development ad people becoming more tolerant of software bugs should our acceptance of software bugs and testing change? So as a tester we need to be aware of that line where we accept occasional acceptable bugs in the software but also not letting go of bugs that would lead to the software being rejected by the customers. It also seems that more companies are introducing constant DevOps deployments to tackle this problem and make sure that the software code is of good quality.

Security risks and backdoors

There is another side of the quality issue where an unintentional security risk can be present because of how the designer has designed the software code. And with so many of these codes being exposed to the internet, many of them have been exploited and hit the headlines. One good way to make sure this is covered while testing for any particular is to try to put yourself in the shoes of the person trying to exploit this vulnerability.

To summarize this chapter says that quality can be subjective and we cannot understand it fully without understanding the context in which we are testing it. Also understanding when an occasional bug is okay and when not is important.

Chapter 6: Being Creative

What is creativity

Creativity is coming up with new ideas and creativity is extremely important for both software development and testing. Creating ideas can be done by combining 2 or more different existing ideas.

The creative process model

Many different kinds of models have been developed to understand the creative process and most of them have been iterative. We would go through one of those models here. The model is called the ‘directed creativity cycle model’. The model has 4 key steps with a lot of actions. The first step is preparation when we do observation and analysis. The next step is imagination when we do the generation of new ideas and harvesting of those ideas. The third phase is development when we enhance those ideas and then evaluate them. The last step is action when we implement it and then live with those ideas.

Myths about creativity

There are many myths about creativity. Some of them are Eureka, Originality, Breed, Expert, incentive, Lone creator, Brainstorming, cohesive, constraint, mousetrap. We would be talking about some of the myths here as they would help us understand how they could be related to software testing. The breed myth says that some people are born creative. But this isn’t the case. Creative problems are not something an expert can solve every time, but in most cases, they require a diverse group of people. History also seems to tell us that there are people who have invented a lot of innovative products. But in most cases, these innovations are developed and not invented by a single person. These are just a few myths but we must understand how they can block our creative process.

Idea generation

We would talk about some of the tools and techniques that we can use to generate new ideas. The first is called SCAMPER. It is a checklist-based technique. SCAMPER means substituting for ideas, combining ideas, adapting to a change, modifying an idea, put to use, and reverse or rearrange ideas. There are multiple questions that you can ask at each part of the step.

Another way to foster creativity is to use testing heuristics. By heuristics, we mean to use triggers in our mind to represent objects and information. One of these heuristics is mnemonics. We can create multiple testing mnemonics which we can then use during the testing phase. Another tool that we can use for being more creative is to combine software testing heuristics and disruptions games. This could be something like gathering all the heuristics and making cue cards out of them. Then you can use them during planning sessions, testing phase, interviews, etc.

Nurture and promote innovation

Innovation can only be possible by being creative. Therefore leaders need to promote this culture of innovation and risk-taking. Critical thinking and coming up with new solutions should be a part of everyday jobs.

Build diversity

To eliminate many biases the best solution is to have a diverse team. Bringing diversity also brings in a fresh perspective and also gives it a better chance for new ideas to be generated. Leaders should therefore employ people who think differently.

We all need to practice and develop environments where creativity is promoted and new and exciting new solutions and tools are applied to be more creative.

Chapter 7: The importance of critical thinking

What is critical thinking?

Critical thinking is the process of rational thinking and understanding and logically connecting ideas. Usually, it means to have an independent set of thoughts where you apply your theories to analyze some data from observation. Most of our thinking is automatic and only when we try to break this pattern do we do any critical thinking.

Automatic thinking

Automatic thinking is our mind taking shortcuts to come up with quick decisions without actually thinking about it. This happens almost all the time with us as this reduces a lot of our mental processing. Therefore engaging your mind in any kind of critical thinking activity requires a lot of effort.

Characteristics of a critical thinker

A critical thinker is someone who has a passion for clarity and accuracy. They are skilled at understanding, deconstructing, and analyzing arguments. They are aware of the many biases and fallacies that can skew an argument. They are honest with themselves and can understand their limitations. They base their arguments on facts and figures.

Techniques to help critical thinking

The huh, really, so technique

Huh? I am understanding the questions and what is being asked?

Really? Do they have any facts or proof to support their understanding?

And? Is there any more information?

So? If you say something does it impact me or someone connected to me?

Ask these sets of questions and it would help you to understand the argument. These are just a single question but we can have multiple different kinds of questions at each step.

Rule of 3

When someone says or argues if you are not able to think of three different interpretations of it then you have not taught enough about it. By using the simple rule you are involving your mind to do critical thinking.

IDEAS as a 5 step critical thinking process

The first is to identify the problem at hand. The second is determining the required and relevant information needed. The third is to enumerate all the options and check their consequences. The fourth is to assess the outcome and the situation. The fifth part is to scrutinize and self-correct. One of the best ways to introduce critical thinking is to write down the ideas or arguments.

How can critical thinking be applied to testing

In the document review stage, you can use the heuristics and checklists to run through if the requirements are testable.

In the test planning, stage testers can use creative thinking as well as critical thinking to make sure the software is tested in different ways. Not having a lot of structure and following the flow is an easy way to do this as is brainstorming. To generate even more ideas try to come up with different “what if ?” questions.

During the test execution, you can use test framing as a critical thinking approach. Asking questions such as why are you running this test case? Why are you testing this requirement? These questions can help you to deploy your critical thinking skills.

In the test analysis phase, you have to apply self-assessment skills. Being self-critical and trying to find out deficiencies is an important step in test analysis. Apply critical thinking in thinking about the defects you have caught and if there is scope for improvement.

In the test reporting phase asking questions like can your delivery be interpreted in different ways or if the context is clear? helps to think clearly about how you report your findings.

The lesson we take from this chapter is that testing requires a lot of critical thinking. And if you are not deploying your critical or creative thinking then you are not even testing anything.

Chapter 8: Building passions and motivation

Testing is an area where you might encounter many testers who are passionate about their work. They are enthusiastic and eager to learn. Talk to them and try to understand where their passion stems from. Helping someone to become passionate about testing is to find something which piques their curiosity.

Passion and motivation

To become passionate about something you need to understand what motivates you. It is difficult to have passion without motivation. Motivation can help to build your passion as well as that for others.

Self-motivation

Asking yourself what motivates you and what you can do to explore that motivation further is a key step to finding motivation. Your motivation does not have to be work-related only, this could be anything traveling or any other activity. Once you find something a good idea is to write it somewhere publicly or privately

Team motivation

Many people think that people in any organization are always motivated by monetary rewards. That is true to some extent because monetary motivation is not everything. But some studies show that if you pay someone for things they are passionate about may demotivate them. Therefore it is important to find out what people like to do and then the stuff people would not want to do. Only introduce the monetary motivation for things they do not want to do. When thinking about testing, if it is scripted then your reward could be extrinsic and not intrinsic.

Motivation models

There are many different motivation models you can use, we are going to discuss a few below.

The first one is called Murray’s system of needs. In this model, we identify needs into one of the 5( ambition, materialistic power, affection, information) types. Some needs from one column can interrelate with one from another column. The model just says that everyone has a need and the value that we put on these secondary needs would determine their motivation.

Another one is Maslow’s hierarchy of needs. In this theory, the needs are divided in a pyramid form and it says that you need to fulfill the needs at the bottom of the pyramid before you move up the pyramid. At the bottom is the psychological needs, safety, then love and belonging. The last two are esteem and self-actualization respectively.

Relationship to software testing

Many in testing do not feel so much connected to the process of testing. Some people are very passionate about testing and there is a need to make that enthusiasm more contagious. Understanding what makes people motivated can help you to build a very strong team relationship. While hiring testers, knowing their passions and motivation could be the key between a good and a bad one.

Chapter 9: Testing and social science

What are social sciences?

Social science is a set of discipline which tells us how society thinks and works. It touches the human part. Their interactions and relationships. Social science can be important to understand how people interact with software. Below are some of the research strategies from social science that you can apply even in testing.

1. Emergent strategy: This states that as we learn more about anything our perspective about the topic as well as our strategy would change.
2. Grounded theory: In this, the tester could develop a theory even while testing. You start with some ideas and as you uncover more details those ideas change.

Qualitative coding

Qualitative coding in testing can provide many advantages like pattern recognition, theory forming, critical and systematic thinking, and more. Learning social science coding and applying it in testing could be a valuable tool.

Autistic software

More software is being developed which requires human interaction. And testing any software which does not have this interaction is easy. It gets difficult when the software has these interactions because the outcomes are not predicated. This software is good with numbers but has poor social skills. Therefore from a testing point of view, software developed should meet the needs of the user and not only the technological need of a software. Make them more sociable and interactable. If these measures are followed then the users would be less annoyed.

The human element

While testing most of the time testers follow a set of test cases and because they are a set of instructions they tend to forget the human element to testing. We also need to understand that everyone is different and apply this insight while testing and even during the development phase.

Note-taking

Note-taking is an important part of social science. And this also applies to software testing. If the notes are not detailed or accurate enough then the testing would not be good enough. There are many ways that a tester can take notes. We are going to discuss the Cornell Method here. In this method on the left-hand side you write cues and review, the big right part has the area for all the note-taking and in the bottom part, you have to summarize the notes.

In this chapter, we looked into some of the areas of social sciences. There are many more areas that cross over to testing. We learn about how we can do testing about being more human-centric.

Chapter 10: The positives and negatives of teamwork

Working with a group can be both positive and negative. In this chapter, we would visit both the positive and negative effects of teamwork and checkout ways to control negativity. Giving people a good working environment, recognition, and self-empowerment increase their teamwork.

Personality and behaviors

Knowing the personality of your team can be helpful. It is important to strike that balance of introverts and extroverts in a team. Making sure the personality types in a team do not clash is important. There should be an environment where people can voice their opinion and also feel strong enough to oppose some opinion if needed. There are many personality tests like MBTI etc which group people in certain categories but most of them are incorrect as they do not account for how complex people are and that they cannot be just labeled as a category. We are all unique and very complex and hence it is difficult to label us into one category.

Groupthink

One other misconception is that when a decision is made by a team then that decision is the best. This leads us to some of the disadvantages of working in a team.

The power of authority: If you are working in a team and another member of your team is your manager then you are going to be influenced by him/her. They might ask you to do something which might not be appropriate but you may not be able to disobey.

The power of the crowd: Sometimes strong people in a team can easily overpower someone’s opinions. They may also kill creative power in the team. If you need more creative ideas and discussions in a group then you may set some predefined goals before a session.

Tips for working in a team

Leading a team: If you are leading a team it is important that every member of the team feels valued. Providing an environment where it is easy to fail and learn is an important part of the innovation process. As a leader, you can act as a facilitator making sure the bad parts of teamwork do not run down the team. Give them a clear direction of expectations and be transparent with them.

Working in a team: In a team, it is important that you are seen as someone who adds some kind of value. Make sure you do not bring others down. Helping and collaborating with your team members is an important part of work. Look out for opportunities to share your knowledge but also be ready to learn. Make some time to talk to them and understand them on a personal level as well.