Written Bug Reports Are Usually Lousy

That’s unfortunately my experience.

This week I posted about this on my LinkedIn profile where I complained about a bug report I’d read before. The bug report went like this:

App doesn’t make the API calls that trigger feature X.

That was it. Written by a QA person.

I really have a hard time understanding such reports even when I’m not directly involved (maybe because of that, but in case of such bad reports, I doubt it plays a role).

I’m not saying this bug report is always wrong, but the situations when this is actually useful are, in my opinion, very limited. I think this would be mostly useful only if I’m sitting with another person in front of one screen, or at least close to each other, and we are developing and testing together. Then I can imagine I’d say this to announce my observation. We could then immediately build on top of that.

Since I read that report in Jira, the previous situation was likely not the case. Then what’s left is that this is rather an unhelpful bug report. But why exactly?

I hear you say something like: “because there are no steps, no version of the system, no assignee, and 5 other things that were recommended in the last article I read”. Well, maybe. But forget about it for now because doing a decent job in reporting bugs is not a memory exercise.

Doing a good job in reporting bugs means that when someone else (including you later) reads it, they will immediately be able and motivated to fix it.

That said, it means that you need to be clear on two things:

• why a bug should be fixed, why it is even a problem worth paying attention to
• what the problem is about

Do a good job in these two areas and bugs you report will get fixed (soon). And that’s the whole point — getting them fixed, not only reported.

For the sake of practice, let’s see the original bad bug report again:

App doesn’t make the API calls that trigger feature X.

So, what is wrong with it? Here are some of my observations:

• “App” might be a bit too general; this depends on the context, maybe there is only one app the team or company develops, then this might be ok; or maybe there is only one app that can trigger feature X, then it also might be ok. But there also might be a new member who doesn’t know all this. Being more specific is usually preferred.
• “App doesn’t make the API calls” — ok, this can perhaps be a title, but what’s important is why it doesn’t make the API calls; When did it change? What commit changed it? is it broken for all accounts/customers/…, or just in some specific situations? Does the app fail? Are there any logs that reveal an error? Are the calls really not made, or perhaps they are, but it fails later on the backend? What are the API calls? Since a plural is used, I assume more than one should be triggered.
• The bug report does not mention the scope of this problem — many problems are problems for just some customers, accounts, product plans, size of something, … Is this a general problem for all?
• There is no timeline — if it worked before and then stopped working, it’d be nice to know the timeline, both in terms of actual time and in terms of releases or commits; if it can be tracked down to a single commit, it’d help make the fix fast, e.g. by reverting some change.
• This bug report does not motivate anyone to do anything; if I read it, I might as well shrug and move on to something more interesting; in other words, why should I want to fix it?

That’s all nice but how to get better at bug reporting?

I think that you should start with a general attitude that might go something like this: “Am I making someone else’s job easier?” Because that’s your job as a tester as well — to provide information that makes someone else’s job easier.

More practical advice might be:

• watch BBST Bug Advocacy course, e.g. available online here: https://bbst.courses/videos/
• listen to developers — do you hear them say your bug reports are great source of info, great analysis of the problem, etc.? Do you hear them complain about your reports? Do they ask clarifying questions? Their behaviou can tell you a great deal about what you did right or wrong.
• Ask your fellow testers for feedback.
• Ask your fellow programmers for feedback.
• Report a bug and try to solve it in a week’s time — if you need to troubleshoot the problem again, you didn’t do a good job the first time.
• Read the report after yourself — simplify it the second time, shorten it the second time, fix typos, grammar issues, too long sentences, etc.

And also, if you see a bad report and have a good relationship with the person who wrote it, try talking to them about it. This might be tricky because people might think or feel it’s unsolicited advice. But it’s probably possible at least in some cases, and any small improvement metters.

I hope I’ll start seeing better reports in the future :)