8/3/2019 SAST 2002-09-19 Managing ET
1/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 1
Managing Exploratory Testing
Parts of this presentation have been excerpted from:Rapid Software Testing, by James Bach,[email protected]
Black Box Software Testing, by Cem Kaner, [email protected]
High Speed WEB Testing, by Jarle Vga,[email protected]
Stle Amland, Amland Consulting,
Hulda Garborgsv. 2, 4020 STAVANGER, NORWAY
Mob:+47 905 28 930, Phone: +47 51580587, Fax: +4751585524
e-mail: [email protected] WEB: www.amland.no
SAST meeting September 19, 2002, Stockholm, Sweden
mailto:[email protected]:[email protected]:[email protected]:[email protected]://www.amland.no/http://www.amland.no/mailto:[email protected]:[email protected]:[email protected]:[email protected]8/3/2019 SAST 2002-09-19 Managing ET
2/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 2
Software Testing:
There is artistry. There is craft. There isscience. It is all threeas with all
technologies.
If you understand the science and havemastered the craft, then it is meaningful toinvestigate, espouse, and perfect the art.
Boris Beizer
8/3/2019 SAST 2002-09-19 Managing ET
3/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 3
Software Testing:
Testing is the process of executing a programwith the intent of finding errors.
Testing is an extremely creative andintellectually challenging task.
Glenford Myers inThe Art of Software Testing
8/3/2019 SAST 2002-09-19 Managing ET
4/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 4
Presentation Outline
What is Exploratory Testing?
How to do Exploratory Testing?
How to Manage Exploratory Testing?
High Speed WEB Testing Case Study
8/3/2019 SAST 2002-09-19 Managing ET
5/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 5
Different testing approachesSkeptical approaches
Analytical Approaches
Information-driven approaches
Time-honored but less effective approaches
Experiential and intuitive approaches:Lets think blue-sky, speculate and follow our intuition.
We have good hunches about where the bugs are lurking.
Lets jump in an explore the systems behavior hands-on, sowe can decide how to test it.
Lets find the important bugs fast, and worry about the testpaperwork later.
Ross Collard (2002)
8/3/2019 SAST 2002-09-19 Managing ET
6/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 6
What is Exploratory Testing?"Exploratory testing involves simultaneously learning,
planning, running tests, and reporting / troubleshootingresults."
Dr. Cem Kaner (2001)
"Exploratory testing is an interactive process of concurrentproduct exploration, test design and test execution.
To the extent that the next test we do is influenced bythe result of the last test we did, we are doing
exploratory testing.
James Bach, Satisfice (2001)
http://www.kaner.com/8/3/2019 SAST 2002-09-19 Managing ET
7/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 7
Said about eXtreme Programming
Agile software development is not conventional software developmentdone more quickly or done on tippie-toe. Agile software development issoftware done differently.
Ron Jeffries, (e-mail on agile-testing list, April 24, 2002)
proven (no single technique is new)application oriented
planned and disciplined
controllable and reliable
risk minimizingTwo sides of extreme programming:
for the developer: freedom, flexibility, fun
for the manager: controllability, reliability, high qualityMartin Lippert (University of Hamburg), ICSTEST 2002
8/3/2019 SAST 2002-09-19 Managing ET
8/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 8
The eXtreme Programmingand Exploratory Testing Analogy:
Agile software testing is not conventional (scripted) software testingdone more quickly or done on tippie-toe.
Exploratory Testing:proven (no single technique is new)
application oriented
planned and disciplined
controllable and reliable
risk minimizingTwo sides of Exploratory Testing:
for the tester: freedom, flexibility, fun
for the manager: controllability, reliability, high quality
8/3/2019 SAST 2002-09-19 Managing ET
9/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 9
ET vs. Scripted Testing
Fully
Scripted
Testing
Ad-hoc
Testing
AutomatedTests
Bug
Hunting
Exploratory Testing
Jarle Vga (2002)
8/3/2019 SAST 2002-09-19 Managing ET
10/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 10
What is Scripted Testing?Small (but realistic) example:
How to script and test this login?(Functional tests only not security!)
8/3/2019 SAST 2002-09-19 Managing ET
11/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 11
When to use Exploratory Testing? (1)There is little or no specifications and /or requirements
You have little or no domain knowledgeYou dont have time to specify, scriptand test
Uncertainty and Time Pressure!
8/3/2019 SAST 2002-09-19 Managing ET
12/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 12
When to use Exploratory Testing? (2)A common goal of exploration is to probeforweakareasof the program.
Test teams resource consumption per week:25% of the groups time developing new tests
50% executing old tests (including bugregression)
25% on exploratory testing
Cem Kaner (2001a)
8/3/2019 SAST 2002-09-19 Managing ET
13/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 13
Testing is done in Context1. The value of any practice depends on its context.2. There are good practices in context, but there
are no best practices.
3. People, working together, are the most importantpart of any project's context.
4. Projects unfold over time in ways that are often notpredictable.
5. The product is a solution. If the problem isn't solved, theproduct doesn't work.
6. Good software testing is a challenging intellectual process.7. Only through judgment and skill, exercised
cooperatively throughout the entire project,are we able to do the right things at the righttimes to effectively test our products.
http://www.context-driven-testing.com/
8/3/2019 SAST 2002-09-19 Managing ET
14/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 14
Presentation Outline
What is Exploratory Testing?
How to do Exploratory Testing?
How to Manage Exploratory Testing?
High Speed WEB Testing Case Study
8/3/2019 SAST 2002-09-19 Managing ET
15/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 15
Doing Exploratory TestingKeep your mission clearly in mind.
Keep notes that help you report what you did,
why you did it, and support your assessmentof product quality.
Keep track of questions and issues raised inyour exploration.
To supercharge your testing, pair upwith another tester and test the samething on the same computer at thesame time.
From Rapid Software Testing, copyright 1996-2002 James Bach
Charter/Mission
Notes, Risks, Issues,Questions and Errors
Execution:Pairs in Sessions
8/3/2019 SAST 2002-09-19 Managing ET
16/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 16
Charter: A clear mission for the sessionA charter may suggest what should be tested, how itshould be tested, and what problems to look for.
A charter is not meant to be a detailed plan.
General charters may be necessary at first:Analyze the Insert Picture function
Specific charters provide better focus, but take moreeffort to design:
Test clip art insertion. Focus on stress and flow techniques,and make sure to insert into a variety of documents. Wereconcerned about resource leaks or anything else that mightdegrade performance over time.
From Rapid Software Testing, copyright 1996-2002 James Bach
Charter/Mission
Notes, Risks, Issues,Questions and Errors
Execution:Pairs in Sessions
8/3/2019 SAST 2002-09-19 Managing ET
17/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 17
Taking NotesTest Coverage Outline/Matrix
Evaluation Notes
Risk/Strategy ListTest Execution Log
Issues, Questions & AnomaliesIt would be easier to test if you changed/added
How does work?Is this important to test? How should I test it?
I saw something strange
From Rapid Software Testing, copyright 1996-2002 James Bach
Charter/Mission
Notes, Risks, Issues,Questions and Errors
Execution:Pairs in Sessions
8/3/2019 SAST 2002-09-19 Managing ET
18/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 18
ET DocumentationPlanning:
Charter overview of what to test (plan)
Migh be a flip chart on the wall
Mission What are we looking for?
ExecutionNotes what happened during testing?
What did I do? Why did I do it?
Used to assess product quality after test.Data files input data used for testing
Bug reports enough details to recreate the test / bug
Track questions and Issues
Charter/Mission
Notes, Risks, Issues,Questions and Errors
Execution:Pairs in Sessions
8/3/2019 SAST 2002-09-19 Managing ET
19/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 19
Approaches / Styles of ETAt the heart of all ET styles:
Questionsand Questioning Skills
Characterize the styles with respect to each other:Do they focus on:
Method of questioning?
Method of describing or analysing the product?
The details of the product?
The patterns of use of the product?The environment in which the product is run?
To what extent would this style benefit from groupinteraction?
From Black Box Software Testing, copyright 19962002 Cem Kaner
8/3/2019 SAST 2002-09-19 Managing ET
20/55
8/3/2019 SAST 2002-09-19 Managing ET
21/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 21
Exploratory Testing SkillsGeneral systems modelling and dimensional analysis
Inferencing out loud
Experiment designTechnical story telling
Use of mnemonics and heuristics
De-biasing (individual or team)
Exploratory investigationRisk analysis
Bug advocacy
8/3/2019 SAST 2002-09-19 Managing ET
22/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 22
I thought skill is the ability to something,more or less.
Skill varies from person to person.Its distinct from talent and knowledge.
A technique, by contrast, is a way of doingsomething; a sort of recipe.
Skill belongs to a person, technique isuniversal.
James Bach, Satisfice Inc.
8/3/2019 SAST 2002-09-19 Managing ET
23/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 23
Heuristics (and rules) and Skills we relate to heuristics as a tool to apply; something that might
help us do the right thing in a given situation,
whereas we relate to a rule as something to
comply with; something that defines right behavior.
Using heuristics properly requires that you exercise
discretion and judgment, on some level; whereas judgment may get
in the way of rules. It's helpful to have contradictory heuristics,because that's like having a variety of advice available before
making a decision; whereas contradictory rules make compliance impossible.
8/3/2019 SAST 2002-09-19 Managing ET
24/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 24
Heuristic Risk Based TestingThe purpose of finding errors
Make a prioritised list of risks.
Perform testing that exploreseach risk
As risks evaporate and newones emerge, adjust your testeffort to stay focused on thecurrent crop. Ref.: James Bach
8/3/2019 SAST 2002-09-19 Managing ET
25/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 25
Summarizing ETThe object of your mission is to explore theMissouri river, & such principal streams of it,
as, by its course and communication with thewaters of the Pacific ocean...may offer themost direct & practicable water
communication across this continent for the
purposes of commerce.
- Thomas Jefferson's letter to Meriwether Lewis, June 1803
Robinson, H., Microsoft, Exploratory Modeling
8/3/2019 SAST 2002-09-19 Managing ET
26/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 26
Presentation Outline
What is Exploratory Testing?
How to do Exploratory Testing?
How to Manage Exploratory Testing?
High Speed WEB Testing Case Study
E i
8/3/2019 SAST 2002-09-19 Managing ET
27/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 27
The Test Session
1) Charter
2) Time Box3) Reviewable Result
4) Debriefing vs.
From Rapid Software Testing, copyright 1996-2002 James Bach
Charter/Mission
Notes, Risks, Issues,Questions and Errors
Execution:Pairs in Sessions
Debriefing
E ti
8/3/2019 SAST 2002-09-19 Managing ET
28/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 28
ET Execution - SessionsSession
Basic testing work unit
Uninterruptede-mail, meetings, telephone calls etc.
ReviewableA report should be produced
CharteredMission associated with this session; What arewe testing? What are we looking for?
Charter/Mission
Notes, Risks, Issues,Questions and Errors
Execution:Pairs in Sessions
Debriefing
8/3/2019 SAST 2002-09-19 Managing ET
29/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 29
Charter:
A clear mission for the sessionA charter may suggest what should be tested, how itshould be tested, and what problems to look for.
A charter is not meant to be a detailed plan.
General charters may be necessary at first:Analyze the Insert Picture function
Specific charters provide better focus, but take moreeffort to design:
Test clip art insertion. Focus on stress and flow techniques,and make sure to insert into a variety of documents. Wereconcerned about resource leaks or anything else that mightdegrade performance over time.
From Rapid Software Testing, copyright 1996-2002 James Bach
8/3/2019 SAST 2002-09-19 Managing ET
30/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 30
Time Box:
Focused test effort of fixed duration
Brief enough for accurate reporting.
Brief enough to allow flexible scheduling.
Brief enough to allow course correction.
Long enough to get solid testing done.Long enough for efficient debriefings.
Beware of overly precise timing.
Short: 60 minutes (+-15)Normal: 90 minutes (+-15)
Long: 120 minutes (+-15)
From Rapid Software Testing, copyright 1996-2002 James Bach
8/3/2019 SAST 2002-09-19 Managing ET
31/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 31
Reviewable Result: The session sheetCharter
#AREAS
Start Time
Tester Name(s)Breakdown
#DURATION
#TEST DESIGN AND EXECUTION
#BUG INVESTIGATION AND REPORTING#SESSION SETUP
#CHARTER/OPPORTUNITY
Data Files
Test Notes
Bugs#BUG
Issues#ISSUE
CHARTER-----------------------------------------------
Analyze MapMakers View menu functionality and
report on areas of potential risk.#AREAS
OS | Windows 2000Menu | ViewStrategy | Function TestingStrategy | Functional AnalysisSTART-----------------------------------------------5/30/00 03:20 pm
TESTER-----------------------------------------------Jonathan BachTASK BREAKDOWN-----------------------------------------------#DURATIONshort#TEST DESIGN AND EXECUTION65#BUG INVESTIGATION AND REPORTING 25#SESSION SETUP20
From Rapid Software Testing, copyright 1996-2002 James Bach
8/3/2019 SAST 2002-09-19 Managing ET
32/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 32
Debriefing:
Measurement begins with observationThe manager reviews session sheet to assurethat he understands it and that it follows theprotocol.
The tester answers any questions.
Session metrics are checked.
Charter may be adjusted.
Session may be extended.New sessions may be chartered.
Coaching happens.
From Rapid Software Testing, copyright 1996-2002 James Bach
Ch / i k Execution:
8/3/2019 SAST 2002-09-19 Managing ET
33/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 33
SummaryExploratory Test Management
Requires skills and experience practice
Be prepared! Improve your skills!Charter Define your mission
Take notes
Test in SessionsTrack Session Metrics
Charter/Mission
Notes, Risks, Issues,Questions and Errors
Execution:Pairs in Sessions
Debriefing
8/3/2019 SAST 2002-09-19 Managing ET
34/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 34
Presentation Outline
What is Exploratory Testing?
How to do Exploratory Testing?
How to Manage Exploratory Testing?
High Speed WEB Testing Case Study
8/3/2019 SAST 2002-09-19 Managing ET
35/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 35
High Speed WEB Testing*
A Case Study
With permission from Jarle Vga, XIT AS,[email protected],
Oslo, Norway* contribution to the book Software Quality and Testing in Internet Times, see references
8/3/2019 SAST 2002-09-19 Managing ET
36/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 36
Presentation OutlineAbout the WEB application
Preparation
Environment and people
Test Planning
Test execution
The web testing experience
Its on the Web. Go find the bugs before the client does!
8/3/2019 SAST 2002-09-19 Managing ET
37/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 37
Application: Basic functionalityWEB publishing tool:
Content Creator creates pages
Multi dimension / organisational viewsMulti language
Topic Manager have to approve eachtopic before publication
One company-One message
8/3/2019 SAST 2002-09-19 Managing ET
38/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 38
Preparation...DocumentationSystem documentation not available onlyhigh level standard product description
Changes on the test environmentConstant changes during planning period
Browser difficulties
Not possible to downgradeMicrosoft software, or toinstall two versions of InternetExplorer on the same PC
8/3/2019 SAST 2002-09-19 Managing ET
39/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 39
What to doFeatures to be tested:Functional test
Search facilities
Browser testsInternet Explorer 4.0 and up
Netscape 4.0 and up
Operating Systems:
Windows NTWindows 98
Windows 2000
would this be enough? And how to do it??
8/3/2019 SAST 2002-09-19 Managing ET
40/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 40
Who?12 testers
Different business areas
Different countriesMarketing and publishing people, a few ITliterates
2 test coordinatorsTest Environment manager
8/3/2019 SAST 2002-09-19 Managing ET
41/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 41
Test Environment20 PCs
5 browsers (versions)
3 Operating systems12 people
2 user groups simultaneously:Publishers / Content CreatorContent Approvers / Topic Manager
8/3/2019 SAST 2002-09-19 Managing ET
42/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 42
How to...
Test scripts - abandonedUnstable system
Unavailable documentationPair testingTo motiovate the tesers andovercome any problems related to
inexperienced peopleExploratory testing
Will depend on the testers ability toexplore the system to find bugs
8/3/2019 SAST 2002-09-19 Managing ET
43/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 43
The Test Plan (Chart)
Features to be testedImportant to be precise, this helps focusing the pair testing
Features not to be testedJust as important, used to limit the test
ApproachPreparations and requirements
No specific test script (but rather guidelines)
Use of role-play and test matrixes
Item pass/ fail criteriain order to judge the outcome of the test.
8/3/2019 SAST 2002-09-19 Managing ET
44/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 44
Test ExecutionMajor difficulties with the set-up of the test environment
Browsers not installed
Win2000 pcs not available
Functionality test according to guidelinesCreation of pages in the portal using role-play with pairs (ContentCreator/Topic Manager)
Using a matrix, where roles, OS and browsers were integrated.
Browser and op.sys. testMatrix with 5 browsers and 3 op.sys.
Yellow pages (search functionality)re-use of pages published
Exploratory testwhat happens if...
Th t i i di t h
8/3/2019 SAST 2002-09-19 Managing ET
45/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 45
ExecutionThe Matrix
GroupsUser group
1 2 3 4 5 6
Content Creator
Topic Manager
Group # Operating System
W-NT W98 W2000
Netscape 4 6 1
Netscape 6 3 4/6 5
Iexplore 4 1
Iexplore 5 4/5 2/3Browser
Iexplore 5.5 2 5 6
Day 1 10:00-12:00 13:00-15:00 15:00-16:30
Time
Day 2 09:00-17:00 09:00-17:00 13:00-16:00
The upper matrix indicate howthe groups worked on differentcombinations of browsers andOS. The lower how the groups
were divided into different usergroups at the start of the test, foreach change of operatingsystem they were alternated.
I.e. Group 3 started as ContentCreator working on NT with
Netscape 6, and moved to W98with Explorer 5 after lunch,working as Topic Manager.Different test guidelines was
developed for each combination,which gave us
OS*Browser = 3*5 = 15 differentguidelines.
8/3/2019 SAST 2002-09-19 Managing ET
46/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 46
Execution Exploratory pair testing
Plan increase coverage by using thetesters creativity
which is what happenedIt can be difficult to keep testers ontrack
but easy to motivate, and many bugsfound.
8/3/2019 SAST 2002-09-19 Managing ET
47/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 47
Execution Debriefing and Notes
Continous guiding / coordination required
Taking Notes did not seem to be a problem
(due to testers working in pairs)High number of duplicate error messages dueto no clear mission set for each pair
Debriefing takes time! Need to be planned
8/3/2019 SAST 2002-09-19 Managing ET
48/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 48
Acceptance Test SummaryApprox. 150+ bugs reported (incl. duplicates)
Mainly due to lack of limitations in the system, youcould do anything.
Developer accepted only approx. 60Corrupted pages: Web portal made inaccessible
Menu disappeared when usingsigns like (#*\/@ etc ) in
page titleCustomer: Severe safety riskDeveloper: Tester have misusedthe system
TestManager
8/3/2019 SAST 2002-09-19 Managing ET
49/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 49
Case Summary
The portal was NOT recommended forpilot production after the test.
Using pair- and exploratory methods
proved successful
Has happened since then:The two test coordinators scripted tests for the 60 bugs
fixed and participated in a 2 days re-test of the applicationApplication went live after this total of 4 days testing andhas been in production without significant problems for morethen a year
8/3/2019 SAST 2002-09-19 Managing ET
50/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 50
Lessons learned...(1)
Pair testingSpeeded up reporting errors
Excellent to promote creativity
Encouraged discussionDemanded a strong voice to keep everyonefocused!
Enables effective testing of short-lived, intense
projects...but has also proved very useful on many testprojects, also larger (i.e. at Microsoft, ref. J.Bach)
8/3/2019 SAST 2002-09-19 Managing ET
51/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 51
Lessons Learned...(2)
Exploratory testingRequires skilled testers
Requires some pre-set restraints, and thorough planning
Requires a system for briefing and reporting/ documenting(ref. J.Bach -Session based test management)
Requires guidance from test manager
..but when the requirements are fulfilled, it proved to bevery successful.
Easy to motivate
MANY deviations found
and it gave testers a greater feeling of participation
8/3/2019 SAST 2002-09-19 Managing ET
52/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 52
Learn moreExploratory Testing:
Rapid Software Testing, by James Bach,[email protected]
Black Box Software Testing, by Cem Kaner,
Amland Consulting; London, Copenhagen and Stockholm:Stockholm, October 30 31, 2002 (www.enea.se)
http://www.amland.no/training_schedule.htm
People Issues, test techniques, inspections:Grove Consultants, www.grove.co.uk
+ many, many more
mailto:[email protected]:[email protected]://www.grove.co.uk/http://www.grove.co.uk/mailto:[email protected]:[email protected]8/3/2019 SAST 2002-09-19 Managing ET
53/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 53
References/Bibliography (1)Amland, S., 1999. Risk Based Testing and Metrics, EuroSTAR '99Preceding, Barcelona, Spain, http://www.amland.no/articles
Argus, C. & Johnson, B., Ad Hoc Software Testing; Exploring theControversy of Unstructured Testing,
http://www.testingcraft.com/ad_hoc_testing.pdfBach, James, 1999a. Risk-Based Testing. How to conduct heuristic riskanalysis, Software Testing & Quality Engineering Magazine,November/December 1999, vol. 1, issue 6,http://www.stqemagazine.com. See also www.satisfice.com
Bach, James , 1999b. What is Exploratory Testing? Satisfice.com
Bach, Jonathan, 2000, Session Based Test Management, STQEMagazine, 11/00, www.satisfice.com
Beizer, Boris, Software Testing Techniques, Van Nostrand Reinhold.NY. 1990.
Collard, Ross, 2002, to be published, [email protected]
http://www.amland.no/articleshttp://www.testingcraft.com/ad_hoc_testing.pdfhttp://www.satisfice.com/http://www.satisfice.com/mailto:[email protected]:[email protected]://www.satisfice.com/http://www.satisfice.com/http://www.testingcraft.com/ad_hoc_testing.pdfhttp://www.amland.no/articles8/3/2019 SAST 2002-09-19 Managing ET
54/55
SAST 2002-09-19Managing ET & Case 2002 Amland Consulting 54
References/Bibliography (2)Kaner, C., Falk, Nguyen, 1999, Testing Computer Software, John Wiley & Sons
Kaner, C., 2000. Rethinking Software Metrics, Evaluating measurement schemes,Software Testing & Quality Engineering Magazine, March/April 2000, vol. 2,issue 2.
Kaner, C., 2001a, Black-Box Software Testing, training at Satisfice July 2001.
Kaner, C., Bach, J., Pettichord, B., 2001b, Lessons Learned in Software Testing,John Wiley & Sons; http://www.testinglessons.com/, ISBN: 0471081124
Kaner, C., Bach, J. 2001c, Exploratory Testing in Pairs, presentation atSTAREast, Orlando, FL, www.kaner.com
Keith, Geordie, 2002, All Hands on Deck: How we used our whole company totest, STQE Magazine, vol. 4, issue 4, July / August 2002.
Lyndsay, James and Eeden, Niel van, 2002, Adventures in Session-BasedTesting, paper at StickyMinds.com, will also be presented at EuroSTAR 2002
Vga, J., Amland, S., 2002. High Speed Web Testing, contribution to the book Software Quality and Testing in Internet Times, Editor: Dirk Meyerhoff et al,Springer-Verlag, 2002, ISBN: 3-540-42632-9
http://www.testinglessons.com/http://www.kaner.com/http://www.kaner.com/http://www.testinglessons.com/8/3/2019 SAST 2002-09-19 Managing ET
55/55
Stle Amland
Amland Consulting
Hulda Garborgsv. 2
N-4020 STAVANGERNorway
Phone: +47 905 28 930
Fax: +47 51 58 55 24
e-mail: [email protected]
WEB: www.amland.no
Contact Details
mailto:[email protected]://www.amland.no/http://www.amland.no/mailto:[email protected]Top Related