class: center ##Rapid Software Testing Part II ### Shing Lyu ??? top, middle, bottom left, center, right --- ###Agenda * How to execute tests * How to plan a test strategy for coverage * How to document the test process * How to report --- ##How to execute tests * Exploratory * Heuristics: applied, not followed ---  --- ##The Testing Story The backbone of Explorary Testing --- ##The Testing Story * Level 1: status of the product * Level 2: How you test it * Level 3: The value of the testing * Level 3+: Value of the stories --- If you are frustrated ## De-Focus * Find recent test patterns * Break that pattern * MFAT (multiple factors at a time) --- If you are Confused ##Focus * Simplify * Conserve States * Repeat * Return to known state * OFAT (one factor at a time) --- ##Disposable time * Learning * Thinking * Refine approaches --- ##Plunge in and Quit help you find a plan for a complex test job --- ## Test procedures * Test activity * Test cases * Test procedures * Configure * Operate * Observe * Evaluate --- ### To reach test _integrity_: Focus * known clean state * simple, deterministic * specified model * established and consisten lab procedures * specific perdictions and records * easy to reproduce --- ### To find _unexpected bug_: De-Focus * start from different states * complex actions * a variety of models * Question your procedure and tools * Open expectations * Tough test --- ###How do you test a whole product ### Use a diversified risk-based strategy --- ##Test Plan = Strategy + Logistics * Strategy: Test desin, choice of tests * Logistics: Resources allocation --- ## Making a strategy * Learn the product * Important potential problems * How to find those problem * How to find problem in general --- ##Heurisic Testing Model --- ##Diverse Half-Measures --- ## Value / Risk as a simplifying factor ###Find problems that mater --- ##Testing itself is #__risk analysis__ --- ## Cost as a Simplyfying Factor ### Quick and cheap tests --- ### Quick and Cheap tests * Clcik on Help * Monkey Test * Shoe Test * Blink Test * Resource Starvation * Crappy Configs * Multiple Instances * etc. --- ##Exactly repeat a test is impossible --- ### Should I repeat a test? * Cost vs. value * What can I gain from test I haven't performed * Try this: * Why ask the same question again? * Why ask the same question again? * Why ask the same question again? * Why ask the same question again? * Why ask the same question again? * Why ask the same question again? * Why ask the same question again? --- ##Exploiting Variation To Find More Bugs --- ###How do you record your work? ###Use concise, modular documents that help tell the testing story --- ##This should be documented... --- ##..._if_ and _when_ and _how_ it serves our purpose --- ###Problems of test documents * Distract us from real testing * Beuracratic format * Mixed information for different people in one report * No one reads it * etc. --- ####Concoise Documentation Minimizes Waste  --- ### Documenting Exploratory Tests * Automatic Logging (Recording) * Take Notes * Coverage * Oracles * Key procedures * Test Ideas * Bugs/Risks * Issues/Questions/Anomalies --- ### SBTM * Charter * Time Box; .footnote[60, 90*, 120 min] * Reviewable Result: .footnote[session sheet] * Debriefing: .footnote[coaching] --- ###Effective Reporting ### Learn to tell a compelling story that provokes the right questions --- ###Reporting Considerations * Reporter safety * Client * Rules * Significance of report * Subject of report * Other agents reporting * Medium * Precision and confindence level --- ### The Dashboard * Always visiable * "Do Not Erase" * Up-to-date ---  --- ###Areas * 15~30 top-level areas * Roughly equal value * Minimize overlap --- ### Test Effort * None * Start * Low * High * Pause * Blocked * Ship --- ### Test Coverage * 0 * 1: Sanity * 1+ * 2: Common & Critical * 2+ * 3: Complex Cases --- ### Quality Assessment * :-) * :-| * :-( * Comments: bug ID, reason for pause or blocking --- ###Time * Update 2-5 times/week or at each build * Progress & Reset --- ###Rapid testing is a personal discipline, it requires practive, but no one's permission --- ###Conclusion * Focus on the tester's mind * Deal with complexity * Tell a compelling testing story * Test skill through practice * Use heuristics * Be a service * Consider cost vs. value * Diversify the team and tatics * Manage the focus of your work * Conext driven