Jan 042019

How to write a good test incident/anomaly/bug/defect report

You can read about IEEE 829-1998 documentation standards on wiki and ISTQB

Title/Description/Headline

  • Clear description of the bug (your bug title “is” your bug report).
  • Developer/reader should understand just from the title, without reading the entire report, what the bug is and where exactly it happens.
  • Bugs are migrated into the developer’s database
  • When short on time, TMs review and assign bugs to dev teams based on the title alone.
  • Other testers rely on titles when checking for duplicates.

Template: [Device/OS] – [Location] – [Bug Description]
Example: “iPhone 6+/iOS8.3 – Checkout – Cannot process PayPal payment”

Bug title format depends on a project. Possible data that can be included:

  • Device: mobile and tablet devices
  • OS: Windows 10, Linux Mint, Android 8.0
  • Browser: Firefox, Chrome, Safari
  • Name of the section/component where the Bug happened: “Shopping cart”, “Left navigation frame”, “Contact us form”, “About us page”
  • Short description of the problem: “Order button doesn’t work”, “Error message”, “Send button doesn’t work when clicked on”, “Employee photos don’t load”

Bug Category

    1. Crash: Application quits or closes unexpectedly while using the features with or without an error message.
      Sometimes there are “silent” crashes when the app does not close, but the log states there was a crash

      • Web – page hangs and doesn’t respond, ultimately resulting with an error, or the browser closes.
      • Computer – application freezes the device, hangs for a long time or closes abruptly.
      • Mobile – applications close abruptly with an error.
      • The app crashes when rotating the device on the home page (vs “The app crashes”)
      • “Silent crash” – app is unresponsive for some time (e.g. 70 sec), however when resumes functionality is not working as expected as a result of a crash (e.g. transactions are lost).

 

    1. Performance: Problematic slowness, hanging, or sluggish interface.
      • Slow navigation in the application
      • Features take longer to load than they should
      • Application reacts slowly when navigating throughout features.
      • Application or pages take too long to load.
      • Application freezes or becomes unresponsive.

 

    1. Functional: workflow failures; features don’t work as designed; unexpected or illogical application behavior; the end result differs from the expected result.
      • Broken page links;
      • A link doesn’t respond when clicked on;
      • A button leads to the wrong action;
      • A button does not respond when clicked on;
      • Search and filters return incorrect results;
      • Form validation issues, e.g. form validation accepts non-existent values
      • Audio issue: a video plays but the sound is not working
      • The user is not logged in after entering valid credentials and tapping “Sign in”

 

    1. Technical/Malfunction:
      Bugs that produce error messages or malfunctions of the application. Technical issues occur when communication between AUT (application under test) and server, backend (things processing data), or hardware does not connect/process requests properly, so often these are issues with:

      • Network connection problems which may occur between client and server sides,
      • Database,
      • Server side authentications,
      • Performance of the application
      • Application hangs for a long time,
      • Application crashes,
      • Application is not working and is getting crashed after launching it on non-compatible OS or browser,
      • Any error when trying to load a page,
      • JavaScript error,
      • After clicking on “Login” button, no action is performed and 500 server side error or 404 forbidden error is displayed in browser’s console window

 

    1. GUI/Visual/Display error: Layout and UI (user interface) distortions or mistakes.
      Any problem with the way the app/website looks.

      • Missing elements on a page;
      • A photo doesn’t load/picture is missing;
      • Page elements or content is misaligned;
      • Content does not fit the area it is in;
      • Inconsistent colors on a link, button or menu;
      • A button is overlapping with a text

 

  1. Content/Spelling/Grammar issue: a page with typos, grammatical errors and localization issues in the text of the website/app.
    • Localization issues where the wrong word is used in translations;
    • Spelling and grammatical errors;
    • Typos;
    • Inconsistent naming conventions, e.g., “Log in” vs “Login” or “Color” vs “Colour”
    • Capitalization errors, e.g., incorrect capitalization in titles;
    • Punctuation is used incorrectly in text. ( , . : ; ‘ ” )

Occurrence/Frequency

How frequently does the bug occur?

  • Always: bug is reproducible every single time (5 out of 5 times)
  • Occasionally: bug is not reproducible every time, but 3/5 or 4/5 of attempts
  • Sometimes: bug is reproducible 2 out of 5 times.
  • Hardly Ever: less than once in five attempts, but in multiple attempts you were able to reproduce the bug at least once.
  • Once: Couldn’t be reproduced no matter how often you tried.

Severity/Priority

Severity – the degree of impact (seriousness) a defect has on the product functionality.

Severity level is based on the technical aspect of the product, determined by the QA engineer as severity is driven by functionality.

Priority – the order (how soon) in which a defect should be fixed/resolved by a developer.

Priority status of defects is based on the customer requirements and is decided in consultation with the manager/client since priority is driven by business value.

    1. Critical
      • A critical bug is extremely rare
      • There are no workarounds
      • Show stopper, complete shut-down of the process, nothing can proceed further;
      • Mission critical bugs to the core functionality of the application;
      • The defect affects the system severely/very high impact/blocking issue and product cannot be used until it is fixed;
      • Defect absolutely must be fixed right away/NOW/ASAP;
      • This one bug would delay the launch/release of a product

       

        Examples:

      • Unable to log in;
      • Cannot check out or save my shopping cart;
      • An email client that has a bug that prevents users from sending/receiving emails,
      • Browser/App crash

       

    2. High/Major
      • Bugs that are related to the core functionality of the application
      • High impact, highly severe defect that causes the collapse of the system functionality.
      • However, certain parts of the system remain functional.
      • There are workarounds
      • Defect should be resolved ASAP, but don’t have to be fixed before product launch.
      • Issues can wait to be resolved in the first available patch/ next release after launch.

       

        Examples:

      • A flash demo doesn’t load properly
      • A bug tracking application that does not allow users to set a bug type/priority
      • A typo in the large text of the homepage banner

 

  1. Medium
    • Bugs that do not affect any critical user functionality
    • Not a big impact, bug causes some undesirable behavior, but the system is still functional
    • There are workarounds that allow users to accomplish the desired task
    • Should be fixed but not urgently.
    • Bug can wait until the next release.

     

      Examples:

    • A travel site that randomly fails to send email notifications
    • search results page that displays the right results, but formats incorrectly in Chrome browser etc.

     

  2. Low
    • Application is fully functional; bugs do not interfere with the core product functionality
    • Not a big impact, defects are just irritating annoyances (e.g. spelling or minor GUI problems)
    • There are workrounds/the workarounds are not needed to allow users to accomplish the desired task
    • Defects may or may not ever be fixed.
      Repair can be done once the more serious defects have been resolved

     

      Examples:

    • A misspelling in the middle of an interior page;
    • Text is displayed in the wrong location/overlaps;
    • The sitemap loads improperly in IE6.

Actions Performed/Steps

  • Numbered steps that clearly and concisely (minimum) describe all the steps required to reproduce the bug.
  • Never leave out information because it’s available in your screenshot or video.
  • Step 1 should always reference the test URL or launched app and be followed by all the steps needed to reproduce the issue (Go to the Test URL: http://testurl.com).
  • One exception to this would be using “Prerequisites”. This helps to minimize the steps that led to the bug.
    E.g., Instead of listing out every step to login in, start your steps with: “Prerequisite: User is logged in.”
    E.g., “Prerequisites – Set up an account using Facebook and be logged into that account.”
  • Be very careful with step minimization. Skipping straight to the URL where the problem occurred, or skipping any steps that might seem obvious can cause missing the very step that caused the bug.
  • While describing actions that led to the bug, don’t add any additional information: commentary, theories, or conclusions.

Expected Results (ER)

  • Explain what should happen, not what shouldn’t happen.
    E.g.: “The app shouldn’t crash” vs “The user is taken to XYZ screen.”
  • Should not be numbered or bulleted list

Actual Results (AR)

  • Sometimes actual result doesn’t match the result you expected. This can mean you have found the bug in the application, but sometimes it’s an incorrect/incomplete understanding of the appplication.
  • Should not be numbered or bulleted list
  • Describe what did happen, not what didn’t happen.
    E.g. “The user wasn’t taken to the XYZ screen.” vs “The user remained on the ABC screen.”

Attachments/Supporting Evidence

  • screenshots;
  • videos;
  • logs;
  • other bug attachments that can be of use to the developer.
    Each attachment should add to the value of the bug in at least one of three ways:

  • Proof that the bug exists (Not having the appropriate supporting evidence can lead to your bug being rejected);
  • Helps the customer to reproduce the bug;
  • Helps the developer resolve the bug.

Images/Screenshots

  • Use images to illustrate static issues.
  • Add an image even if you have a video.
  • Highlight the area(s) of interest in your image.

Videos

  • Use videos to illustrate dynamic issues where the steps are complex.
  • Videos can be the developer’s only way to observe a bug that they can’t reproduce.
  • Actions in the video should match the steps listed in the bug report. Video confirms your steps were accurate at the time the issue was created.
  • Videos should be trimmed to only show the bug.
    Show the minimum steps required to reproduce the bug.
  • Audio should be muted, except the audio related bug reports.
  • External/live videos that allow to see hand gestures (e.g. touching a button on the screen) can be more helpful than mirrored videos.

Log files

  • Console Logs give the developer a detailed snapshot of what’s going on behind the scenes.
  • Logs are extremely useful for the bugs that can’t be reproduced.
  • Exception Log Trace – extremely helpful for a crash.
  • Logs are plain text and therefore have a very small file size.
  • Use .txt file extension to save the logs.

Оставить комментарий

Your email address will not be published. Required fields are marked *