SlideShare a Scribd company logo
1 of 157
Download to read offline
Hypothesis Based Testing (HBT) Cookbook
© 2011-12, STAG Software Private Limited. All rights reserved.
STEM is the trademark of STAG Software Private Limited.
HBT is the intellectual property of STAG Software Private Limited.

This e-book is presented by STAG Software Private Limited www.stagsoftware.com




                                                                                 2
Hypothesis Based Testing (HBT) is a scientific personal test methodology that is unique in its approach to ensuring cleanliness of software. It is a
goal focused approach, commencing with setting up of cleanliness criteria, hypothesising potential defect types that can impede this, and then
performing activities to ensure that testing is purposeful and therefore effective and efficient. The central theme HBT is constructing a hypotheses
of potential defects that may be probable, and then scientifically proving that they do not indeed exist. The activities of testing like test strategy,
test design, tooling & automating become purposeful as these are focused on uncovering the hypothesised defect types ensuring that these
activities are done scientifically and in a disciplined manner.

HBT is based on sound engineering principles geared to deliver the promise of guaranteeing cleanliness. Its core value proposition is about
hypothesising potential defects that may be present in the software and then allow you to engineer a staged detection model to uncover the
defects faster and cheaper that other typical test methodologies.

HBT fits into any development methodology and weaves into your organisational test process. HBT is powered by STEMTM (STAG Test
Engineering Method) a collection of EIGHT disciplines of thinking. STEM provides the foundation for scientific thinking to perform the various
activities. It is personal scientific inquiry process that is assisted by techniques, principles and guidelines to decompose the problem, identify
cleanliness criteria, hypothesise potential defect types, formulate test strategy, design test cases, identify metrics and build appropriate automation.




                                                                                                                                                           3
Inspirations from nature
HBT has been inspired by certain ideas and these are discussed below. The inspirations have come from “Properties of matter”, “Fractional
distillation”, “Sherlock Holmes”, “Picture of baby growth”.


Properties of matter
                                                               Physical & Chemical properties of matter allow us to:
                                                               ... classify
                                      “affected by”            ... understand behaviours, interactions
                                                               ... enable checking purity


                                                               How can we use a similar train of thought to identify
                                                               “properties of cleanliness” and then “types of defects”?




                                                                                                         “Properties of the system”
                                                                              End user expectations         Cleanliness criteria




                                                                           Issues in specifications,
                                                                           structure, environment      Potential Defect Types (PDT)
                                                                                and behaviour

                                                                                                                                            4
Inspirations from nature


Fractional distillation
                                   This is a technique to separate mixtures that have components of different
                                   boiling points

                                   In software systems, there exists a variety of defect types that may be present in
                                   the system. How can we apply this thought process to optimally uncover the
                                   defects, by “fractionally distilling” them?

                                   Can we separate these types of defects on the basis of certain properties and
                                   optimally uncover the defects?




From : http://withfriendship.com




                                                                                                                    5
Inspirations from nature


Picture of baby growth
                                                                      The picture shows the health of the foetus/baby .
                                                                      This picture shows size, shape, parts and types of issues not
                                                                      present

                                                                      Seeking inspiration, can we depict the health of software system in
                                                                      a similar manner? Can we measure the ‘intrinsic quality’ at a
                                                                      stage?

                                                                      As we progressively evaluate in a staged manner, certain types of
                                                                      defects detected & removed and therefore quality grows.
                                                                      Can we chart this as “cleanliness index”?




Source :http://www.environment.ucla.edu/media/images/Fetal_dev5.jpg




                                                                                                                                            6
Inspirations

 Sherlock Holmes
                   Sherlock Holmes was person who applied deductive logic to solve
                   mysteries.

                   How can we see inspirations from Holmes to hypothesise the
                   types of defects that may be present and prove presence of these?




                                                                                       7
HBT - A Personal Scientific Test Methodology

Test methodologies focus on activities that are driven by a process which are powered by tools, yet successful outcomes still
depend a lot on experience.

Typically methodologies are at organisational level.


On the other hand HBT is a personal scientific methodology enabled by STEMTM , a defect detection technology to deliver
“Clean Software”




                                                                                                                                8
Scientific approach to detecting defects

Cleanliness criteria     What is the end user expectation of “Good Quality”?


Potential Defect Types   What types of issues can result in poor quality?


Evaluation Stage         When should I uncover them?


Test Types               How do I uncover them?


Test Techniques          What techniques to generate test cases?


Scenarios/Cases          What are the test cases? Are they enough?


Scripts                  How do I execute them?


Metrics & Management     How good is it? How am I doing?



                                                                               9
How is HBT different from other test methodologies?
The typical test methodologies in vogue have relied on strength of the process and the capability of the individual to ensure
high quality in the given cost and time constraints. They lack the scientific rigour to enable full cost optimisation and more
often rely on automation as the means to driving down cost and cycle time. For example, they do not provide a strong basis
for assessing the quality of test cases in terms of their defect finding potential and therefore improve effectiveness and
efficiency.

HBT on the other hand enables you to set a clear goal for cleanliness, derive potential types of defect and then devise a
“good net” to ensure that these are caught as soon as they get injected. It is intensely goal-oriented and provides you with a
clear set of milestones allowing you to manage the process quickly and effectively.


                                                                                                     Goal

                                                                                                            drives
                        la




                                                                                   T
                      ic




                                                                                   B
                   yp




                                                                                   H
                                                                                                  Activities
                  T




                                        Activities




                                                                                                                                defect detection
                             ......................................                    ......................................




                                                                                                                                  Powered by
                                                                      Powered by




                                                                                                                                  technology
                                                                      experience




                                                                                                                                    (STEM)
                             ......................................                    ......................................
                             ......................................                    ......................................
                             ......................................                    ......................................
                             ......................................                    ......................................
                             ......................................                    ......................................

                                                  hopefully
                                                  results in
                                            Goal
                                                                                                                                                   10
Hypothesis Based Testing - HBT 2.0
A Quick Introduction        Personal, scientific test methodology.
                                      SIX stage methodology powered by
                                      EIGHT disciplines of thinking (STEMTM).




                      Setup                Hypothesise
               Cleanliness Criteria    Potential Defect Types

 SUT
                                            Nine Stage
             Cleanliness Assessment
                                       Defect Removal Filter



                                                                                11
A quick introduction to HBT

                  SIX stages of DOING                        powered             EIGHT disciplines of THINKING
                                                                by

                                                                                         D8               D1
                S6                         S1
                                                                                   Analysis &      Business value
                Assess &         Understand
                                                                                   management      understanding
                ANALYSE          EXPECTATIONS
                                                                       D7                                                   D2
                                                                         Execution &                             Defect
                              D8 D1
                                                                         reporting            STEM Core          hypothesis
      Tooling            D7           D2
 S5                           STEM         Understand                                           32 core
      SUPPORT                                           S2
                         D6           D3   CONTEXT                                             concepts
                              D5 D4                                                                            Strategy &
                                                                            Visibility
                                                                                                               planning     D3
                                                                       D6
                Devise           Formulate
                                 HYPOTHESIS                                              Tooling   Test design
                PROOF


            S4                             S3                                            D5                D4




                     HBT                                     powered                        STEM
          Personal test methodology                             by               Defect detection technology


                                                                                                                                 12
D1 Business value understanding    D2     Defect hypothesis           D3    Test strategy & planning

     Landscaping                                                           Orthogonality principle
     Viewpoints                         EFF model                          Tooling needs assessment
     Reductionist principle             Defect centricity principle        Defect centred AB
     Interaction matrix                 Negative thinking                  Quality growth principle
     Operational profiling               Orthogonality principle            Techniques landscape
     Attribute analysis                 Defect typing                      Process landscape
     GQM

D4     Test design
                                                                      D5 Tooling
      Reductionist principle
     Input granularity principle                                           Automation complexity
     Box model                                                             assessment
     Behaviour-Stimuli approach                     32 core                Minimal babysitting principle
     Techniques landscape                          concepts                Separation of concerns
     Complexity assessment                                                 Tooling needs analysis
     Operational profiling
      Test coverage evaluation

     Visibility                           Execution & Reporting            Analysis & Management
D6                                 D7                                 D8


     GQM                                 Contextual awareness              Gating principle
     Quality quantification model         Defect rating principle           Cycle scoping



                                                                                                           13
Connecting HBT Stages to the
Scientific approach to detecting defects

   S1
         Cleanliness criteria             Potential defect types    S3
   S2




           Expectations             Staged & purposeful detection



                                                                    S4



                                           Complete test cases




  S6    Goal directed measures            Sensible automation       S5

                                                                         14
Clear baseline
                                                     Set a clear goal for quality
 Cleanliness criteria       Potential defect types
                                                     Example: Clean Water implies
                   S1, S2                            1.Colourless
                            Staged & purposeful      2.No suspended particles
                                                     3.No bacteria
                                 detection
                                                     4.Odourless
    Expectations
                                                     What information(properties) can be used to identify this?
                             Complete test cases
                                                     ... Marketplace,Customers, End users
                                                     ... Requirement(flows), Usage, Deployment
                                                     ... Features, Attributes
    Goal directed                                    ... Stage of development, Interactions
                            Sensible automation
     measures                                        ... Environment, Architecture
                                                     ... Behaviour, Structure




                                                                                                                  15
A goal focused approach to cleanliness
                                                      Identify potential defect types that can impede cleanliness
 Cleanliness criteria   Potential defect types   S3
                                                      Example:
                                                      Data validation
                                                      Timeouts
                        Staged & purposeful
                                                      Resource leakage
                             detection
                                                      Calculation
    Expectations                                      Storage
                                                      Presentation
                         Complete test cases          Transactional ...

                                                      Scientific approach to hypothesising defects is about looking at

    Goal directed                                     FIVE Aspects - Data, Logic, Structure, Environment & Usage
                        Sensible automation
     measures                                         from
                                                      THREE Views - Error injection, Fault proneness & Failure

                                                      Use STEM core concepts
                                                      > Negative thinking (Aspect)
                                                      > EFF Model (View)




                                                                                      “A Holmes-ian way of looking
                                                                                      at properties of elements”
                                                                                                                        16
Levels, Types & Techniques - STRATEGY
                                                                       NINE levels to Cleanliness

    Cleanliness criteria          Potential defect types
                                                                      L9   End user value
                                                            S4
                                      Staged & purposeful             L8   Clean Deployment
                                           detection

         Expectations                                                 L7   Attributes met

                                      Complete test cases             L6   Environment cleanliness

                                                                      L5   Flow correctness
         Goal directed                Sensible automation
          measures                                                    L4   Behaviour correctness

                                                                      L3   Structural integrity
                        L3
Quality Levels                              Test Techniques (T1-T4)
                         PDT7                                         L2   Input interface cleanliness
                                TT5
                         PDT6
                                TT4             TT3         T3
                L2       PDT5
                                                                      L1   Input cleanliness
                PDT4
                         TT3                    TT2
   L1           PDT3                                        T2
  PDT2        TT2                                TT1        T1
  PDT1        TT1


PDT: Potential Defect Types                                                                              17
Countable test cases &
                                                            Fault coverage

Countable test cases & Fault coverage
                                                      Use STEM Core concepts
 Cleanliness criteria   Potential defect types        > Box model
                                                      > Behaviour Stimuli approach
                                                      > Techniques landscape
                                                      > Coverage evaluation
                        Staged & purposeful
                             detection
                                                      to
    Expectations                                      - Model behaviour
                                                 S4   - Create behaviour scenarios
                         Complete test cases          - Create stimuli (test cases)

                                                      Irrespective of who designs, #scenarios/cases shall be same -
                                                      COUNTABLE
    Goal directed       Sensible automation                          Test Scenarios/Cases
     measures
                                                              R1                                          PDT1
                                                                                  TS    TC1,2,3
                                                              R2                                          PDT2
                                                                          TT
                                                              R3                  TS    TC4,5,6,7         PDT3

                                                             Requirements & Fault traceability

                                                      That test cases for a given requirement shall have the
                                                                    ability to detect specific types of defects
                                                                                           FAULT COVERAGE


                                                                                                                      18
Focused scenarios + Good Automation Architecture
                                                      Level based test scenarios yield shorter scripts that are
                                                      more flexible for change and easily maintainable.
 Cleanliness criteria   Potential defect types

                                                                L9     End user value
                        Staged & purposeful
                             detection                          L8     Clean Deployment

    Expectations
                                                                L7     Attributes met
                         Complete test cases
                                                                L6     Environment cleanliness
                                                 S5
                                                                L5     Flow correctness
    Goal directed       Sensible automation
     measures
                                                                L4     Behaviour correctness

                                                                L3     Structural integrity

                                                                L2     Input interface cleanliness

                                                                L1     Input cleanliness




                                                                                                                  19
“Cleanliness Index” - Improved visibility
                                                                                                        L4
 Cleanliness criteria            Potential defect types                                               PDT10   TT8

                                                                                                      PDT9    TT7

                                     Staged & purposeful                                        L3    PDT9    TT6




                                                                   Cleanliness
                                          detection
                                                                                               PDT8   TT5
     Expectations
                                                                                               PDT7
                                                                                                      TT4
                                     Complete test cases                                 L2    PDT6
                                                                                        PDT5
                                                                                               TT3
                                                                                 L1     PDT4
S6
     Goal directed                                                               PDT3
                                     Sensible automation
      measures                                                                   PDT2
                                                                                        TT2

                                                                                 PDT1   TT1
                    Quality report                                                              Stage
                CC1     CC2    CC3      CC4
       R1                                          Met
       R2
                                                   Not met
       R3                                                                             “Growth of a baby”
                                                   Partially met
       R4
       R5
                                                                                                                    20
HBT Stages
Six stages to produce clean software




                                       21
Six staged methodology to produce clean software
The act of validation in HBT consists of “SIX Stages of DOING”. It commences with first two stages focused on a scientific approach to
understanding of the customer expectations and the the context of the software. One of the key outcomes of the first two stages is
“Cleanliness Criteria” that gives a clear understanding of the expectation of quality. In the third stage, the Cleanliness Criteria and the
information acquired in the first two stages are used to hypothesise potential types of defects that are probable in the software. The fourth
stage consists of devising a proof to scientifically ensure that the hypothesised defects can be indeed be detected cost-efficiently. The fifth
stage focuses on building the tooling support needed to execute the proof. The last stage is about executing the proof and assessing if the
software does indeed meet the Cleanliness Criteria.

                                                                            Who are the customers, end users, what do they need, and
                                                                       S1
                 S6                      S1                                 what do they expect?

               Assess &         Understand
               ANALYSE                                                      What are the features of the system, what technologies are
                                EXPECTATIONS                           S2
                                                                            used, architecture?

                            D8 D1                                           What types of defects may be present?
                                    D2                                 S3
        Tooling        D7              Understand                           What types of fishes to catch?
   S5                       STEM
        SUPPORT                                   S2
                                    D3 CONTEXT
                       D6
                            D5 D4                                           What is strategy, plan, test scenarios/cases?
                                                                       S4
                                                                            Sherlock Holmes

                Devise         Formulate
                PROOF          HYPOTHESIS                                   What tools do I need to detect the defects?
                                                                       S5
                                                                            Boat in the fishing analogy
                S4                       S3
                                                                            How am I doing? How is quality?
                                                                       S6
                                                                            Fisherman
                                                                                                                                               22
Stage #1 : Understand EXPECTATIONS

                          The perception that end-users have of how well the product delivers the needs denotes the quality of the
    Understand the        software/system. "Needs" represent the various features that the software/system needs to have, to allow the
    marketplace for       end-user to fulfill his tasks effectively and efficiently. "Expectations" on the other hand represent how well the
      the system          needs are fulfilled.

                          The final software/system may be deployed in different marketplaces addressing the needs of various types of
     Understand the       customers. Hence it is imperative that we understand the various target markets (i.e marketplace) where the
  technology(ies) used    software or system will be deployed. There could be different types of customers in the marketplace. Hence it
                          is necessary to identify the various types of customers and then finally identify various types of end-users
                          present in the customer. What we have done now is to start from outward direction i.e marketplace and
                          adopt a customer/end-user centric view to understand the needs and expectations.
Understand deployment
    environment           Once we have identified the various types of customers and the corresponding end-users, we can move on to
                          understand the various technologies that make up the software or the system and also a deployment
                          environment. The intent is to get a good appreciation of the "construction components" and the target
                          environment of deployment. It is imperative that we should have a good understanding of the internal aspects
Identify end user types   and not merely the external aspects of the system.
& #users for each type
                          Now we're ready to go into a detailed analysis of the various types of end-users and the typical number of
                          users for each of these end-users. Subsequent to this, we need to identify the various business requirements
  Identify business       i.e. "needs" for each end-user.
requirements for each
      user type           At the end of the stage, the objective is to have a good understanding of the various end-users and their
                          needs paving the way to understanding expectations clearly.


                                                                                                                                             23
Needs & Expectations
                                                                         NEEDS
                                            Customers in    End users    Should write
                                                                         Should have a eraser
                                                               Kids
                                            Education                    EXPECTATIONS
                                                                         Should be attractive
                                                             Seniors
                                                                         Should be non-toxic
                                                                         Lead should not break easily

      Product                                                Artists
                                             Drawing
      e.g Pencil                                                         NEEDS
                                                           Draftsmen     Should write
                                                                         Should not need sharpening


                                                           Management    EXPECTATIONS
                                             Corporate                   Thickness should be consistent
                                                                         Variety of thickness should be
                                                           Engineering    available
                                                                         Variety of hardness should be
                                                             Admin        available


Needs typically features that allow to get the job done.
Expectations are how well the need is satisfied.

Remember Functional & Non-functional requirements ?

                                                                                                          24
What does “understanding” involve?
Good understanding of what is expected is key to effective testing. To accomplish this, it is imperative that we commence from understanding
who the various types of end users, their requirements and subsequently the expectations that they have from these. Having a deep domain
knowledge helps immensely. But what if I this is a domain that I am not very conversant with? Is there a scientific way to undertand?

Understanding is a non-linear activity, it is about identifying the various elements and establishing connections between these. In the process of
connecting the dots, missing information is identified, leading to intelligent questions. Seeking answers to these questions aids in deepening the
understanding.

                                                                              These are some of the elements that need to be understood.
                                                                              Some of the information elements are “external to the system” i.e.
                                                                              marketplace, customer types, end users, business requirements
                                                                              while some are “internal to the system” i.e. features, architecture,
                                                                              technology etc.

                                                                              Stage #1 (Understand EXPECTATIONS) focuses on “external
                                                                              information while Stage #2 (Understand CONTEXT) focuses on
                                                                              “internal information”.




                                                            “Good testing is about asking intelligent questions leading to deeper understanding.”
                                                                                                                                                     25
Information extracted & artefacts generated

   Information              At each stage, certain information is extracted, understood and transformed into artefacts useful to
                            perform effective & efficient testing.

    Marketplace


     Customers
                                                  Artefacts
                                                                            The key outcomes as demonstrated by the
                                                                            artefacts are:
     User types
                                             System overview                 ‣The big picture of the system
                                                                             ‣The various end users ascertained for
   Requirements                                                               different classes of customers in different
                       HBT
                                                                              marketplaces
                     Stage #1                   User type list
    Deployment                                                               ‣A list of business requirements for each
                                                                              type of end user.
   environment.
                                             Requirement map
    Technology


   Lifecycle stage
                      In Stage #1, the focus is on external
                      information that relate to marketplace,
    #Users/type       customers, end users andHBT Stage 1
                                                 business                     “Good understanding is key to effective
                      requirements. This stage is useful to get the           testing. Identifying who will use what is the
                      bigger picture of the system and its potential          beginning to become customer-focused”
                      usage and the how it is deployed.
Deliverables from Stage #1


                                Should contain a a good overview of the marketplace, the various types of customers, end-users types,
  System overview
                                deployment environment and technologies that will be used to build the system.


                               Should contain a list of the various types of users for different types of customers in various market
    User type list
                               segments.


                                Should contain a list of the business requirements and high level technical features mapped to the various
  Requirement map
                                individual types.




STEM Discipline applied in Stage #1
 The STEM discipline “Business value understanding” of STEM is applied in this stage of HBT.
 The two STEM core concepts of “Landscaping” and “Viewpoints” are useful in this stage to scientifically understand the expectations.




                                                                                                                                             27
Stage #2 : Understand CONTEXT
                            In this stage the objective is to understand the technical context in terms of the various features, their relative
    Identify technical      business value, the profile of usage and ultimately arrive at the cleanliness criteria. Note that at this stage, we
  features and baseline     are moving inward to get a better understanding of technical features of the system.
          them
                            Having identified the various business requirements mapped by each type of end-user, the next logical step is
                            to drill-down to the various technical features for each business requirement. It is important to understand
       Understand           the various technical features that constitute the entire system do not really work in isolation. Therefore it is
      dependencies          necessary to understand the interplay of the features i.e. understand the dependencies of a feature with other
                            features. Understanding this dependency is very useful at later stages of the life cycle, particularly to regress
 Understand profile of       optimally.
        usage
                            We now have a list of requirements and the corresponding technical features mapped by each end-user. We
                            are ready to proceed logically to understand the profile of usage of each of the features by the various end-
Identify critical success   users. To do this it is important to understand the typical in the maximum number of users for each user type
         factors            and then the volume of usage by each user for every technical feature. Since we already have a mapping
                            between the end-user type and the technical feature feature, all we have to do is to understand as to
                            approximately how many times this feature will be used by typical end-user of that end user type. The intent
Prioritize value of end     of this is to gain a deeper understanding of the usage profile to enable an effective strategy formulation at the
users(s) and features       later stage of HBT.

                            It is not only sufficient that the features work correctly, it is equally important that the various attributes of
 Ensure attributes are      the nonfunctional aspects of the various features are indeed met. Typically nonfunctional aspects of the
       testable             system are identified in the highest system level, and typically turn out to be fuzzy. Good testing demands that
                            each requirement is indeed testable. In HBT, attributes are identified for each key feature and then aggregated
                            to form the complete set of nonfunctional requirements. We will do this in two stages: firstly identifying the
   Setup cleanliness        critical success factors for the technical features and thereof the business requirement and then detailing the
       criteria             critical success factors to arrive at the nonfunctional requirements or attributes. Hence after figuring out the
                            usage profile, identify the success factors for each business requirement.
                                                                                                                                                  28
Stage #2 : Understand CONTEXT

Good testing is not about testing all features equally, it is about learning to focus more on those requirements/features that affect the customer
experience significantly. This does not imply that some requirements/features are less important than the others, it simply means that some
requirements/features are more important . Before we start detailing the various attributes, it is worthwhile to the prioritize the various
requirements /features and also various end-user types. To prioritize, start by prioritizing the various types of end users in terms of their
importance to the successful deployment of the final system. Subsequently rank the importance of each of the requirement/feature for each of
the end-user type. At the end of this exercise, we should have a very clear understanding of the business value of each requirement/feature. Note
that the understanding of usage profile comes in very handy here.

Now we are ready to derive the various attributes from the previously identified success factors and ensure that they are testable. A testable
requirement simply means that it is an unambiguously possible to state whether it failed all passed after executing it. In the context of attributes,
testability implies that each attribute does indeed have a clear measure/metric. Therefore it is necessary to identify the measures and the
expected value of the measures for each of the attribute.

Having identified the various technical features and the corresponding attributes, the usage profile in the ranking of the requirements/features,
we are now set to identify the various criteria that constitute the cleanliness of the intended software. Cleanliness criteria in HBT represents
testable expectations. Cleanliness criteria provides a very strong basis for ensuring a goal-focused testing. This allows one to identify potential
types of defects and then formulate an effective strategy in the complete set of test cases It is important that the cleanliness criteria is not vague
or fuzzy.




                                                                                                                                                         29
Information extracted & artefacts generated
                                                                         At each stage, certain information is extracted,
                                                       Artefacts         understood and transformed into artefacts useful
                                                                         to perform effective & efficient testing.
      Information
                                                      Feature list       The key outcomes as demonstrated by the
         Features                                                        artefacts are:
                                                  Value prioritization    ‣A clear list of technical features
                                                        matrix            ‣Ranking of features to focus on high risk
           Usage                                                           areas
                                                                          ‣Profile of usage
                                   HBT               Usage profile         ‣List of attributes
       Focus areas               Stage #2                                 ‣Feature interactions
                                                     Attributes list
                                                                          ‣Clarity of expectations outlined as
        Attributes                                                         “Cleanliness criteria”

                                                  Interaction matrix
       Interactions

                                                  Cleanliness criteria



 In Stage #2, the focus is on internal
 information that relate to technical features,
 their interactions, focus areas, attributes,
 architecture, technology.


                                                                                                                            30
Deliverables from Stage #2
       Feature list             Should contain the list of technical features, that forms the technical features baseline.

  Value prioritization
                                Should contain a set of users, requirements and features ranked by importance.
        matrix

      Usage profile              Should contain a the profile of various operations by various end users over time.

                                Should contain the key attributes stated objectively i.e. state expected value for all the measures
      Attributes list
                                for each attribute.

                                Should contain the which feature affects what. Note that this should list the interactions and not the details
   Interaction matrix
                                of interactions. The objective is to get a rapid understanding of the linkages.

   Cleanliness criteria           Should contain criteria that need to be met to ensure that the deployed system is indeed clean.



STEM Discipline applied in Stage #2
 The STEM discipline “Business value understanding” of STEM is applied in this stage of HBT.
 The STEM core concepts of “Interaction matrix”, “Operational profiling”, “Attribute analysis” and “GQM” are useful in this stage to
 scientifically understand the context.



                                                                                                                                                 31
Cleanliness criteria
Cleanliness criteria is a mirror of expectations, The intention is to come up with criteria that if met will ensure that system meets the
expectations of the the various end users. This is not be confused with “Acceptance criteria”, as “Acceptance criteria” is typically at a higher
level. Acceptance criteria is typically “extrinsic” in nature i.e. it describes aspects like long duration running, migration of existing data, clean
installation and running in the final deployment environment, delivering stated performance under real-life load conditions.

Cleanliness criteria represents the “intrinsic quality” i.e. what properties should the final system have to ensure that it is deemed clean?
Use the properties of the FIVE aspects of Data, Business logic, Structure, Environment, Usage as applied to your application to arrive at these
criteria specific to your application.

Note that the cleanliness criteria should both the the functional and non-functional requirements.




                                             The recommended style of writing Cleanliness criteria is:
                                             “That the system shall meet ....”

                                             Examples:
                                             That the system is able to handle large data (need to qualify large)
                                             That the system releases resources after use.
                                             That the system displays meaningful progress for long duration activities.
                                             That the system is able to detect inappropriate environment/configuration.




                                                                                                                                                        32
Stage #3 : Formulate HYPOTHESIS

 Having understood the expectations and the context resulting in the formulation of cleanliness criteria, we are ready to hypothesize
 the potential defects that could affect the cleanliness criteria. This is one of the important stages of HBT resulting in a clear
 articulation of the various types of defects and forms the basis for the remaining stages of HBT.

 The key idea is to use the external information like the feature’s behaviour, environment, attributes, usage and internal information like
 construction material i.e technology, architecture to hypothesize the potential defects that may be present in the software under
 construction. Also note that the history of the previous versions of the software or similar systems can also be used to construct and
 strengthen the hypothesis. Having hypothesized the potential defects, it is possible to scientifically construct a validation strategy and
 design adequate test cases, thereby ensuring that the final system to be deployed is indeed clean.

 The FIVE key aspects useful for constructing hypotheses of defects are: data, business logic, structure, environment and usage. This
 HBT stage allows us to follow a structured &scientific approach to the hypothesize the potential effects ensuring that we do not miss
 any.




                                                                                                                                              33
Stage #3 : Formulate HYPOTHESIS (continued)

                                               Firstly use the external information like data specification and business logic
  Identify potential faults for the five
                                               specification to identify the potential defects. The information related to data that could
  aspects - Data, Business logic, Structure,
                                               help are: data type, boundaries, volumes, rate, format, data interrelationships. The intent
  Environment, Usage
                                               should be to get into a "negative mentality" and think of what can go wrong with
                                               respect to all the information related to the data and then produce a list of potential
  Identify potential failures of the five       defects.
  aspects - Data, Business logic, Structure,
  Environment, Usage                           Now use the information related to the business logic to identify the potential effects.
                                               Business logic or the intended behaviour primarily transforms the various inputs i.e.
                                               input data to outputs that the user values. The intention is to identify potential
  Identify potential errors that could be      transformation losses. The information specific to business logic that is useful for
  injected in the five aspects - Data,          arriving at potential defects are : the various conditions and their linkages, values of
  Business logic, Structure, Environment,      conditions, exception handling conditions, access control and dependencies on the
  Usage                                        other parts of the software. Once again, the intent is to get into a "negative mentality",
                                               and identify erroneous business flows of logic.

  Now identify potential defects (PD) &        Up to now the focus has been on using external information like the specification of
  combine PDs, remove duplicate PDs            data and business logic to identify the potential defects. Now focus on the internal
                                               information like structure of the system and construction materials(i.e. language,
                                               technology) used to build the system to hypothesize potential defects. Structure at the
  Group similar PD to form Potential           highest level represents the deployment architecture while structure at the lowest level
  Defect Types (PDT)                           represents the structure of the code. Some of the structural information that could be
                                               useful to hypothesize are: flow of control, resource usage, distributed architecture,
                                               interfacing techniques, exception handling, timing information, threading, layering. As
 Map PDTs to the elements-under-test i.e.
features/requirements                          explained above, continue with the similar train of thought of examining these
                                               information with intent to identify potential problems in the structure.
                                                                                                                                             34
Stage #3 : Formulate HYPOTHESIS (continued)
Having identified potential defects using the behavioural and structural information, examine information related to environment and how they
can affect the deployed system. By environment, we mean the associated hardware and software on which the system is deployed and the
hardware, software and application resources used by the system. The objective is to examine carefully how these can affect the finally deployed
system. Some of the key information related to the environment that could be useful are: hardware/software versions, system access control,
application configuration information, speed of hardware (CPU, memory, hard disk, communication links), environment configuration information
(e.g. #handles, cache size etc), system resources (hardware, OS and other applications).

Up till now we have taken a fault-centric approach of looking for potential faults (aka defects) by examining external or internal information. In
addition to a fault-centric approach, we can also view the system from potential failure points and then identify the potential defects.
Additionally, it is also possible to examine the system from an error injection point of view. That is, understand the kinds of potential errors that
could be injected into the system to irritate the potential defects if any. The objective is to ensure that we have examined the system from all
three views (error centric, fault centric & failure centric) and thereby ensure that we have not missed any potential defects.

A failure centric approach demands that we wear an end-user hat and identify the potential failures that could cause business loss. The cleanliness
criteria formulated earlier could come in very handy as this would force us to think like a customer/end-user. What we trying to do is to ensure
that we have considered all the potential failures and therefore hypothesized the potential defects.

Now move to a user centric view to examine the various ways that an end-user could abuse the system by identifying the various ways errors
could be injected into the system. Not that an end user does not always connote a physical person, it could be another system that interacts with
the system via some interface. so examine the various points of interaction and look at the possibilities of their injection and then hypothesize the
potential defects that could get irritated by these errors. The kinds of information that could be useful here are: workflows, data access, interesting
ways of using the system, accessibility, environmental constraints faced by the physical end-user and potential deviant ways of using the system.

Then consolidate the potential defects and group similar ones into potential defect types (PDT). Finally map the PDTs to the various elements-
under-test i.e. feature/requirements. Now we have a clear notion as to what types of defects that we should look forward to uncovering in what
parts of the system.


                                                                                                                                                        35
Information extracted & artefacts generated
                                                                       At each stage, certain information is extracted,
                                                                       understood and transformed into artefacts useful
                                                                       to perform effective & efficient testing.

    Information


        Data


      Structure                                    Artefacts

                                                                               The key outcomes as demonstrated by the
    Environment                                                                artefacts are:
                       HBT                        PD catalog
                     Stage #3
                                                                                ‣List of potential defect types
    Business logic                                                              ‣Mapping between PDTs & the elements-
                                              Fault traceability                 under-test i.e. Feature/Requirement
                                                   matrix
        Usage


      Attributes


     Past defects      In Stage #2, the focus is on hypothesizing
                       PDTs using the FIVE aspects of Data, Business
                       logic, Structure, Environment & Usage from
                       THREE views - Error-centric, Fault-centric &
                       Failure-centric.
                                                                                                                          36
Deliverables from Stage #3

     PD catalog               Should contain the list of potential defects and the potential defect types

 Fault traceability
                              Should contain the mapping between the potential defect types/potential defects and features/requirements.
      matrix




STEM Discipline applied in Stage #3
 The STEM discipline “Defect hypothesis” of STEM is applied in this stage of HBT.
 The STEM core concepts of “Negative thinking”, “EFF model”, “Defect centricity principle” and “Orthogonality principle” are useful in this
 stage to scientifically hypothesize defects.




                                                                                                                                              37
Stage #4 : Devise PROOF (Part #1: Test Strategy & Planning)
HBT being a goal focused test methodology, the intent is about figuring out an optimal approach to detect the potential of defects in the
system. Therefore strategy in HBT is about staging the order of defect detection, identifying tests that are needed to uncover the specific
defect types and finally choosing test techniques best suited for each type of test.

Typically we have always looked at the levels of testing like unit, integration and system from the aspect of the “size” of entity-under test. Unit
test is typically understood as being done on the smallest component that can be independently tested. Integration test is typically understood
as being done once the various units have been integrated. System test is typically seen as the last stage of validation and is done on the whole
system.

What is not very necessarily very clear is the specific types of defects that are expected to be uncovered by each of these test levels. In HBT,
the focus shifts to specific types of defects to be detected, and therefore the act of detection is staged to ensure an efficient detection
approach.

In HBT, the notion is of quality levels, where each quality level represents a milestone towards meeting the final cleanliness criteria. In other
words each quality level represents a step in the staircase of quality. The notion is to ensure that the defects that can be caught earlier is
indeed caught. So the first step to formulation of strategy is to stage the potential defects and thereby formulating the various quality levels.

However in HBT, there are NINE pre-defined quality levels where the lowest quality level focuses on input correctness progressively going
onto the highest quality level to validate of the intended business value is indeed delivered.




                                                                                                                                                      38
Stage #4 : Devise PROOF (Part #1: Test Strategy & Planning)
   Understand scope
                              Having identified the various types of potential defect types to be detected at various levels, it is now
                              necessary to understand the specific types of tests needed to uncover these potential defects. In HBT each
                              test shall be intensely goal focused. This means that a type of test shall only uncover specific type of defects.
   Choose quality levels
                              The act of test type identification results in specific types of tests to be done at each of the quality levels.

                              Now that we know what types of defects need to be detected when and where what type of tests, we need
 Identify test types
                              to know how to design sufficient yet adequate test cases for each type of test. In HBT, a test technique is one
                              that allows us to design test cases. Based on the types of defects i.e. types of tests, we have to identify the
                              test technique(s) that is best suited for uncovering these types of the defects.
 Identify test techniques
                              Now we have a clearer idea of various types of defects, the levels of detection, types of tests and test
                              techniques., we are now ready to identify the optimal detection process best suited for design/execution of
 Identify detection process
                              test cases. The the act of identifying detection process also allows us to understand whether we need
                              technology support to be able to execute test cases and therefore pave the way for automation strategy.
 Identify tooling needs
                              At this point in time we have a strategy and are ready to develop the detailed test plan. Some of the key
                              elements of the test plan is the estimation of effort and time and formulating the various test cycles. In HBT
                              cycles are formulated first and then effort and time estimated.
 Formulate cycles
                              Finally potential risks that could come in the way of executing the test plan are identified and the risk
                              management plan put in place.
 Estimate effort
                              In summary, a strategy in HBT is a clear articulation of the quality levels, test types test techniques and
                              detection process model.
 Identify risks

                                                                                                                                                 39
Information extracted & artefacts generated
                                                                          At each stage, certain information is extracted,
                                                                          understood and transformed into artefacts useful
    Information                                                           to perform effective & efficient testing.

     Cleanliness
       criteria

        PDT

                                                  Artefacts
     Attributes
                                                                                  The key outcomes as demonstrated by the
                       HBT                      Test strategy                     artefacts are:
     Techniques
                     Stage #4                                                      ‣Test strategy
                                                                                   ‣Test plan
   Deployment env.                                Test plan

    Scope of work


     #Scenarios


        Risks
                        In Stage #4 (Part 1) the focus is on 1
                                                HBT Stage identifying
                        the quality levels, test types, test techniques
                        and the detection process.


                                                                                                                             40
Deliverables from Stage #4 (Part #1)

     Test strategy            Should contain the quality levels, test types, test techniques & detection process


       Test plan              Should contain the test effort estimate, cycle details and the potential risk & mitigation plan.




STEM Discipline applied in Stage #4 (Part #1)
 The STEM discipline “Strategy & Planning” of STEM is applied in this stage of HBT.
 The STEM core concepts of “Orthogonality principle”, “Quality growth principle”, “Defect centered activity breakdown” , “Cycle scoping” are
 useful in this stage to scientifically developing the strategy & plan.




                                                                                                                                               41
Stage #4 : Devise PROOF (Part #2: Test Design)
 The act of designing test cases is a crucial activity in the test life cycle. Effective testing demands that the test cases possess the power to
 uncover the hypothesized potential defects. It is necessary that the test cases are adequate and also optimal.

 In HBT the design is done level-wise and within each level test-type wise. Based on the level & type, the test entity may be different. The test
 design activity for an entity for a type of test at a quality level consists of two major steps, firstly to design test scenarios and then generate
 these test cases for each scenario. Test scenarios are designed entity-wise and therefore there is a built-in notion of requirements
 traceability. In addition to requirements traceability, it is expected that the test scenarios and corresponding test cases are traced to the
 potential types of defects that they are expected to uncover. This is termed “Fault traceability”.




                                                                                                                                                      42
Stage #4 : Devise PROOF (Part #2: Test Design)

  Identify test level to design   The act of test design commences with the identification of the quality level and then the specific type
  consider & identify entities    of test for which the test cases are to be designed. This allows us to identify the various test entities for
                                  which test cases have to be designed.

  Identify conditions & data      Having identified the test entities it is then required to identify the conditions that govern the business
                                  logic and the data elements that drive these conditions. Subsequent to this, build the behavioral model.

                                  Use the behavioral model to generate test scenarios. Then for every scenario, identify the data that
Model the intended behaviour      varies and then generate values for each data element. Finally combine the data values to generate the
semi-formally                     test cases.

                                  Since we have designed scenarios/cases entity-wise, requirements traceability is built-in i.e. the designed
Generate the test scenarios       scenarios/cases automatically trace to the entity (or requirement). Now map the scenarios/cases to the
                                  hypothesized PDTs to build the fault traceability matrix.

For each scenario, generate       Finally assess the test adequacy of the designed scenarios/cases by checking test breadth, depth &
test cases                        porosity.


Trace scenarios to PDT &
entity-under -test


Assess the test adequacy by
fault coverage analysis

                                                                                                                                                  43
Information extracted & artefacts generated
                                                                          At each stage, certain information is extracted,
                                                                          understood and transformed into artefacts useful
    Information                                                           to perform effective & efficient testing.


     Conditions
                                                  Artefacts
       Data
                                             Test scenarios &
       Logic                                       cases                         The key outcomes as demonstrated by the
                      HBT                                                        artefacts are:
                    Stage #4                   Requirements                       ‣Test scenarios & cases
     Structure                                                                    ‣Requirements traceability matrix
                                            traceability matrix
                                                                                  ‣Fault traceability matrix
        PDT                                  Fault traceability
                                                  matrix
   Defect escapes


     Attributes


                        In Stage #4 (Part 2), the focus is on designing
                        test scenarios/cases that can be proved to be
                        adequate and have the power to uncover the
                        hypothesized PDTs.

                                                                                                                             44
Deliverables from Stage #4 (Part #2)

   Test scenarios &
                                Should contain the test scenarios/cases for each entity for all types of tests at various quality levels
         cases

    Requirements
                                Should contain the mapping between the scenarios/cases and the entity-under-test
 traceability matrix


  Fault traceability
                                Should contain the mapping between the scenarios/cases and the PDTs
       matrix




STEM Discipline applied in Stage #4 (Part #2)
 The STEM discipline “Test design” of STEM is applied in this stage of HBT.
 The STEM core concepts of Reductionist principle, Input granularity principle, Box model , Behavior-Stimuli approach, Techniques landscape,
 Complexity assessment,Operational profiling, Test coverage evaluation are useful in design test scenarios/cases scientifically.




                                                                                                                                               45
Stage #4 : Devise PROOF (Part #3: Metrics Design)

                                    In this stage, the objective is to design measurements to manage the process of validation in an
  Identify progress aspects         effective and efficient manner. Since HBT is a good focused test methodology, it is necessary to device
                                    measurements that enable us to clearly show the progress towards this goal.

  Identify adequacy(coverage)       The measurements in HBT are categorized into progress related measures, test effectiveness
  aspects                           measures and system risk measures. Therefore it is necessary to identity the various aspects related
                                    to progress, effectiveness and the system health.

Identify progress aspects           Once the aspects are identified, key goals related to these are identified and then the metrics
                                    formulated. Finally it is necessary to understand when to measure and how to measure.


For each of the aspects identify
the intended goal to meet


For each of these goals, identify
questions to ask


To answer these questions,
identify metrics


Identify when you want to
measure and how to measure

                                                                                                                                             46
Information extracted & artefacts generated
                                                                             At each stage, certain information is extracted,
                                                                             understood and transformed into artefacts useful
                                                                             to perform effective & efficient testing.
    Information


   Quality aspects


  Progress aspects                                    Artefacts

                         HBT                                                         The key outcomes as demonstrated by the
   Process aspects     Stage #4                                                      artefacts are:
                                                   Metrics chart
                                                                                      ‣Chart of metrics that are goal-focused
  Organization goals


   When & how to
     measure




                           In Stage #4 (Part 3), the focus is on designing
                           metrics that will ensure that we stay on
                           course towards the goal.


                                                                                                                                47
Deliverables from Stage #4 (Part #3)

     Metrics chart             Should contain the list of metrics, collection frequency and a how this meets the goal.




STEM Discipline applied in Stage #4 (Part #3)
 The STEM discipline “Visibility” of STEM is applied in this stage of HBT.
 The STEM core concepts of GQM, Quality quantification model are useful in design metrics that are goal-focused.




                                                                                                                         48
Stage #5 : TOOLING support
  Perform tooling benefit         In this stage, the objective is to analyze the support that we need from tooling/technology to
  analysis                       perform the tests. Automation does always imply scripting that is typically automating the
                                 designed scenarios. It could also involve development of test bench, custom tooling to enable the
                                 system to be tested.
  Identify automation scope
                                 This stage of HBT allows you to identify the tooling needs, understand issues/complexity
                                 involved, perform cost-benefit analysis, evaluate existing tools for suitability/fitment and finally
Assess automation complexity     devising a good architecture that provides for flexibility/maintainability before embarking onto
                                 automation.
Identify the order in which
scenarios need to be automated


Evaluate tools


Design automation architecture


Develop scripts


Debug and baseline scripts



                                                                                                                                     49
Information extracted & artefacts generated
                                                                           At each stage, certain information is extracted,
   Information                                                             understood and transformed into artefacts useful
                                                     Artifacts
                                                                           to perform effective & efficient testing.
    Automation                                  Needs & benefits
     objectives
                                                   document

       Scope                                      Complexity
                                               assessment report                   The key outcomes as demonstrated by the
    Scenarios to                                                                   artefacts are:
     automate                                Tooling requirements
                                                                                    ‣The reason for tooling & automation
                       HBT                                                          ‣Challenges involved
                     Stage #5                                                       ‣Requirements of tooling
  Scenario fitness
                                                                                    ‣Scope of tooling & automation
                                                Automation scope                    ‣Architecture of automation
 Technologies used                                                                  ‣Automated scripts
                                                   Automation
                                                   architecture
     Tool info.
                                                Tooling & Scripts
  Complexity info.



                        In Stage #5, the focus is on identifying tooling
                        requirements and building automated scripts
                        that is delivers value & ROI.
                                                                                                                              50
Deliverables from Stage #5
  Needs & benefits            Should contain the technical & business need for automation.
     document

    Complexity
                             Should contain the technical challenges of automation
 assessment report

Tooling requirements         Should contain the requirements expected out of automation


  Automation scope           Should contain scope of automation

     Automation
                             Should contain the architecture adopted to building tooling/scripts
     architecture

  Tooling & Scripts          The actual tools/scripts for performing automated testing




STEM Discipline applied in Stage #5
 The STEM discipline “Tooling” of STEM is applied in this stage of HBT.
 The STEM core concepts of Automation complexity assessment, Minimal babysitting principle, Separation of concerns, Tooling needs analysis
 are useful in adopting a disciplined approach to tooling & automation and deliver the ROI..

                                                                                                                                             51
Stage #6 : Assess & ANALYZE
   Identify test cases/scripts   This stage is where you execute the test cases, record defects, report to the team and take
         to be executed          appropriate action to ensure that the system/application is delivered on time with the requisite
                                 quality.
   Execute test cases, record
          outcomes


       Record defects


 Record learnings from the
  activity and the context


 Record status of execution



 Analyze execution progress


Quantify quality and identify
      risk to delivery

   Update strategy, plan,
  scenarios, cases/scripts

                                                                                                                                    52
Information extracted & artefacts generated
                                                                      At each stage, certain information is extracted,
                                              Artifacts               understood and transformed into artefacts useful
                                                                      to perform effective & efficient testing.
                                        Execution status
                                            report


  Information                              Defect report              The key outcomes as demonstrated by the
                                                                      artefacts are:
   Execution                                                           ‣Report of test execution & progress
                                         Progress report               ‣Defect report
  information
                  HBT                                                  ‣Report on cleanliness aka quality
     Defect     Stage #6                                               ‣Learnings from execution resulting in
  information                          Cleanliness report               improved strategy, scenarios & cases
                                                                       ‣Any other key learnings
    Context                             Updated strategy,
                                        plan, scenarios &
                                              cases


                                          Key learnings



                    In Stage #6, the focus is on ensuring a
                    disciplined execution, intelligent analysis and
                    continuous learning to ensure that the goal is
                    reached.
                                                                                                                         53
Deliverables from Stage #6
   Execution status           Should contain the status of test execution
       report


     Defect report            Should contain defect information


    Progress report           Should contain progress of execution and thereof the cycle


  Cleanliness report          Should contain the cleanliness index and how well the cleanliness criteria has been met

   Updated strategy,
                              Updated strategy, plan, scenarios, cases based on learnings from execution
   plan, scenarios &
         cases


     Key learnings            Key observations/learnings that could be useful in the future



STEM Discipline applied in Stage #6
 The STEM disciplines of “Execution & reporting” and “Analysis and management” of STEM are applied in this stage of HBT.
 The STEM core concepts of Contextual awareness, Defect rating principle, Gating principle, Cycle scoping enable a disciplined execution,
 fosters continual learning and stay focused on the goal.


                                                                                                                                            54
STEM Disciplines




                   55
Discipline #1 : Business value understanding

How to                                 This discipline enables one to understand the system, create a baseline of features, attributes and
Understand a system                    finally expectations. This discipline consists of SEVEN tools, each of which uses certain STEM core
                                       concepts to ensure these are done in a scientific and disciplined manner.
Landscaping | Viewpoints
                                       Good quality implies meeting expectations. This requires that we understand expectations in
How to
                                       additions to the needs as delivered by the requirements. Understanding the intended business
Create a functional baseline
                                       value to delivered is key to this.
Viewpoints | Reductionist principle

How to
Create an attribute baseline

Viewpoints | Reductionist principle

How to                                How to
Identify focus areas                  Understand interdependencies

Value prioritisation | Viewpoints     Interaction matrix

How to                                How to
Understand usage                      Baseline expectations

Operational profiling | Viewpoints     Goal-Question-Metric | Viewpoints

                                                                                                                                             56
Baseline provides the basis for future work

What is to be tested needs to be clear.

Remember Functional & Non-functional requirements?




 Functional Baseline                                 Attribute Baseline
 Consists of list of features to be tested.          The non-functional aspects.
 Essentially a agreed upon list of features.         Agreed upon attributes & their values.




                                                                                              57
Tools in D1 -Business value understanding
                               STEM Core
Tools                                                                                    Description
                               Concepts
                                                        System is viewed as a collection of information elements that are
How to                         Landscaping
                                                        interconnected. This tool enables you to come up with intelligent questions to
Understand a system            Viewpoints
                                                        understand the various information elements and their interconnections.
                                                        Commencing from an external view of end users, various use cases
How to                         Viewpoints               (requirements) are identified and then technical features that constitute the
Create a functional baseline   Reductionist principle   use cases. This tool enables you to clearly setup a functional baseline that is
                                                        used as a basis for strategy, plan, design, tooling, reporting & management.
                                                        In addition to functional correctness, it is imperative that the attributes are
How to                         Attribute analysis
                                                        met,. This tool enables you to identify the attributes and ensure that these are
Create an attribute baseline   Viewpoints
                                                        testable.
                                                        All requirements/features are not equally valued by the end users. This tool
How to                         Viewpoints
                                                        allows you to rank the end users, requirements, features thereby enabling
Identify focus areas           Value prioritisation
                                                        prioritisation of testing based on the risk and perceived value.
                                                        Understanding the real life usage profile is about knowing what operations,
How to                         Viewpoints               #concurrent operations, rate of arrival are in progress at a point in time. This
Understand usage               Operational profiling     tool allows to arrive at the closer to reality potential usage profile of the
                                                        system to ensure effective non-functional tests.
                                                        Understanding how a feature/requirement affects/is-dependent on other
How to
                               Interaction matrix       feature/requirements is useful to understand impact & re-testing effort. This
Understand interdependencies
                                                        tool allows you to rapidly understand the interdependencies.
How to                         Viewpoints
                                                    This tool allows to derive cleanliness criteria that reflect the expectations.
Baseline expectations          Goal-Question-Metric
                                                                                                                                           58
Customers & End Users
                                            Customers in                      End users

                                                                                 Kids
                                            Education

                                                                               Seniors


      Product                                                                  Artists
                                             Drawing
      e.g Pencil
                                                                             Draftsmen



                                                                            Management
                                             Corporate
                                                                            Engineering

                                                                                Admin

A product or an application may be sold in different market places made up of different kinds of customers.
Each class of customer may have different types of end users who use the product.
It is important to understand that each end user may have different needs & expectations.

Testing is about ensuring that the product will indeed satisfy the variety of needs & expectations
                                                                                                              59
Needs & Expectations
                                                                         NEEDS
                                            Customers in    End users    Should write
                                                                         Should have a eraser
                                                               Kids
                                            Education                    EXPECTATIONS
                                                                         Should be attractive
                                                             Seniors
                                                                         Should be non-toxic
                                                                         Lead should not break easily

      Product                                                Artists
                                             Drawing
      e.g Pencil                                                         NEEDS
                                                           Draftsmen     Should write
                                                                         Should not need sharpening


                                                           Management    EXPECTATIONS
                                             Corporate                   Thickness should be consistent
                                                                         Variety of thickness should be
                                                           Engineering    available
                                                                         Variety of hardness should be
                                                             Admin        available


Needs typically features that allow to get the job done.
Expectations are how well the need is satisfied.

Remember Functional & Non-functional requirements ?

                                                                                                          60
Customer Profile
        Customer #1                  Customer #2                  Customer #3                   Customer #4




 Different customers have different types of end users, and differing number of users for type of end user.


                                                                                                              61
Customer Profile & Usage
                                    How many                 What does each one use?
    What types
                                    users                    What is order of importance?
    of users
                                                             What is the usage frequency?



                                                                                                F1

                                                                                                F2

                                                                                                F3

                                                                                                F4
                                                                                                                   System
                                                                                                F5

                                                                                                F6

                                                                                                F7

                                                                                                F8



 Different end users may use the system differently in terms of what they use, frequency of usage and how they value each each feature.

                                                                                                                                          62
Business Value



Ultimately end users need the system to do their job
BETTER, FASTER, CHEAPER and deliver value to their customers.

Understand that it is about “business value” of system - how does the system
help my business to do BETTER, FASTER, CHEAPER.




                                                                               63
Discipline #2 : Defect hypothesis




                                    64
Discipline #2 : Defect hypothesis

How to                            This discipline enables one to hypothesise potential defect types that may be present in the system
Hypothesise defects               under test and setup a clear goal approach to detection/prevention. Goal focused approach implies
                                  that we map the hypothesised potential defect types (PDT) to the elements-under-test i.e feature/
Negative thinking | EFF model |   requirements.
Defect centricity principle
                                  This discipline consists of TWO tools, each of which uses certain STEM core concepts to ensure
                                  these are done in a scientific and disciplined manner.
How to
Setup goal-focus
                                  Hypothesis is done by scientifically examining certain properties of the system and can be
                                  complemented by ones experience.
Orthogonality principle




                                                                                                                                        65
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook
Hypothesis Based Testing (HBT) Cookbook

More Related Content

What's hot

Chaos Engineering with Kubernetes
Chaos Engineering with KubernetesChaos Engineering with Kubernetes
Chaos Engineering with KubernetesArun Gupta
 
Identity and Access Management: The First Step in AWS Security
Identity and Access Management: The First Step in AWS SecurityIdentity and Access Management: The First Step in AWS Security
Identity and Access Management: The First Step in AWS SecurityAmazon Web Services
 
Managing-the-Risks-of-LLMs-in-FS-Industry-Roundtable-TruEra-QuantU.pdf
Managing-the-Risks-of-LLMs-in-FS-Industry-Roundtable-TruEra-QuantU.pdfManaging-the-Risks-of-LLMs-in-FS-Industry-Roundtable-TruEra-QuantU.pdf
Managing-the-Risks-of-LLMs-in-FS-Industry-Roundtable-TruEra-QuantU.pdfQuantUniversity
 
Internship Job Description
Internship Job DescriptionInternship Job Description
Internship Job DescriptionJason Shmeltzner
 
How to Become an IAM Policy Ninja
How to Become an IAM Policy NinjaHow to Become an IAM Policy Ninja
How to Become an IAM Policy NinjaAmazon Web Services
 
An Overview of Machine Learning on AWS
An Overview of Machine Learning on AWSAn Overview of Machine Learning on AWS
An Overview of Machine Learning on AWSAmazon Web Services
 
AI for Software Engineering
AI for Software EngineeringAI for Software Engineering
AI for Software EngineeringMiroslaw Staron
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
 
Emotion Recognition using Image Processing
Emotion Recognition using Image ProcessingEmotion Recognition using Image Processing
Emotion Recognition using Image Processingijtsrd
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceAmazon Web Services
 
So you want to be Well-Architected?
So you want to be Well-Architected?So you want to be Well-Architected?
So you want to be Well-Architected?Amazon Web Services
 
Facial emotion recognition
Facial emotion recognitionFacial emotion recognition
Facial emotion recognitionAnukriti Dureha
 
AI Powered Conversational Interfaces
AI Powered Conversational InterfacesAI Powered Conversational Interfaces
AI Powered Conversational InterfacesAmazon Web Services
 
AWSome Day Online Conference 2019 - Module 3 AWS Security.pdf
AWSome Day Online Conference 2019 - Module 3 AWS Security.pdfAWSome Day Online Conference 2019 - Module 3 AWS Security.pdf
AWSome Day Online Conference 2019 - Module 3 AWS Security.pdfAmazon Web Services
 
Best practices with Microsoft Graph: Making your applications more performant...
Best practices with Microsoft Graph: Making your applications more performant...Best practices with Microsoft Graph: Making your applications more performant...
Best practices with Microsoft Graph: Making your applications more performant...Microsoft Tech Community
 

What's hot (20)

Chaos Engineering with Kubernetes
Chaos Engineering with KubernetesChaos Engineering with Kubernetes
Chaos Engineering with Kubernetes
 
Identity and Access Management: The First Step in AWS Security
Identity and Access Management: The First Step in AWS SecurityIdentity and Access Management: The First Step in AWS Security
Identity and Access Management: The First Step in AWS Security
 
Managing-the-Risks-of-LLMs-in-FS-Industry-Roundtable-TruEra-QuantU.pdf
Managing-the-Risks-of-LLMs-in-FS-Industry-Roundtable-TruEra-QuantU.pdfManaging-the-Risks-of-LLMs-in-FS-Industry-Roundtable-TruEra-QuantU.pdf
Managing-the-Risks-of-LLMs-in-FS-Industry-Roundtable-TruEra-QuantU.pdf
 
QlikView vs Qlik Sense
QlikView vs Qlik SenseQlikView vs Qlik Sense
QlikView vs Qlik Sense
 
AWS Technical Essentials Day
AWS Technical Essentials DayAWS Technical Essentials Day
AWS Technical Essentials Day
 
Internship Job Description
Internship Job DescriptionInternship Job Description
Internship Job Description
 
Introduction to Amazon EKS
Introduction to Amazon EKSIntroduction to Amazon EKS
Introduction to Amazon EKS
 
How to Become an IAM Policy Ninja
How to Become an IAM Policy NinjaHow to Become an IAM Policy Ninja
How to Become an IAM Policy Ninja
 
An Overview of Machine Learning on AWS
An Overview of Machine Learning on AWSAn Overview of Machine Learning on AWS
An Overview of Machine Learning on AWS
 
AI for Software Engineering
AI for Software EngineeringAI for Software Engineering
AI for Software Engineering
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Elastic Block Storage (EBS)
Elastic Block Storage (EBS)Elastic Block Storage (EBS)
Elastic Block Storage (EBS)
 
Introduction to Chaos Engineering
Introduction to Chaos EngineeringIntroduction to Chaos Engineering
Introduction to Chaos Engineering
 
Emotion Recognition using Image Processing
Emotion Recognition using Image ProcessingEmotion Recognition using Image Processing
Emotion Recognition using Image Processing
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 
So you want to be Well-Architected?
So you want to be Well-Architected?So you want to be Well-Architected?
So you want to be Well-Architected?
 
Facial emotion recognition
Facial emotion recognitionFacial emotion recognition
Facial emotion recognition
 
AI Powered Conversational Interfaces
AI Powered Conversational InterfacesAI Powered Conversational Interfaces
AI Powered Conversational Interfaces
 
AWSome Day Online Conference 2019 - Module 3 AWS Security.pdf
AWSome Day Online Conference 2019 - Module 3 AWS Security.pdfAWSome Day Online Conference 2019 - Module 3 AWS Security.pdf
AWSome Day Online Conference 2019 - Module 3 AWS Security.pdf
 
Best practices with Microsoft Graph: Making your applications more performant...
Best practices with Microsoft Graph: Making your applications more performant...Best practices with Microsoft Graph: Making your applications more performant...
Best practices with Microsoft Graph: Making your applications more performant...
 

Similar to Hypothesis Based Testing (HBT) Cookbook

My presentation erin da802
My presentation   erin da802My presentation   erin da802
My presentation erin da802nida19
 
I/O chapter 4
I/O chapter 4I/O chapter 4
I/O chapter 4Roi Xcel
 
HBT a revolutionary approach to testing software
HBT a revolutionary approach to testing softwareHBT a revolutionary approach to testing software
HBT a revolutionary approach to testing softwarebhowmickk
 
Diagnosing behavioral problems and perception
Diagnosing behavioral problems and perceptionDiagnosing behavioral problems and perception
Diagnosing behavioral problems and perceptionEui Jung Hwang
 
Five Steps to Excellence
Five Steps to ExcellenceFive Steps to Excellence
Five Steps to ExcellencePMHaas
 
Evaluation Of Competency For Stand Trial
Evaluation Of Competency For Stand TrialEvaluation Of Competency For Stand Trial
Evaluation Of Competency For Stand TrialMichelle Robertson
 
Analysis And Criticisms Of The Rorschach
Analysis And Criticisms Of The RorschachAnalysis And Criticisms Of The Rorschach
Analysis And Criticisms Of The RorschachDani Cox
 
Ethics and Integrity by Tommy Seah- A Value Added Audit on Environment
Ethics and Integrity by Tommy Seah- A Value Added Audit on EnvironmentEthics and Integrity by Tommy Seah- A Value Added Audit on Environment
Ethics and Integrity by Tommy Seah- A Value Added Audit on EnvironmentTommy Seah
 
Rapid software testing
Rapid software testingRapid software testing
Rapid software testingSachin MK
 
Exploratory Testing in an Agile Context
Exploratory Testing in an Agile ContextExploratory Testing in an Agile Context
Exploratory Testing in an Agile ContextElisabeth Hendrickson
 
Exploratory Testing Explained
Exploratory Testing ExplainedExploratory Testing Explained
Exploratory Testing ExplainedTechWell
 
Competency models types and techniques
Competency models types and techniquesCompetency models types and techniques
Competency models types and techniquesThe Blockchain Academy
 
Testing Simulation Tools And Simulations.pdfTesting Simulation Tools And Simu...
Testing Simulation Tools And Simulations.pdfTesting Simulation Tools And Simu...Testing Simulation Tools And Simulations.pdfTesting Simulation Tools And Simu...
Testing Simulation Tools And Simulations.pdfTesting Simulation Tools And Simu...Tammy Lacy
 
Talent Circle 4 students
Talent Circle 4 studentsTalent Circle 4 students
Talent Circle 4 studentsGIORGIO MAGGI
 
Psychological Tests And Psychological Assessment
Psychological Tests And Psychological AssessmentPsychological Tests And Psychological Assessment
Psychological Tests And Psychological AssessmentRachel Quintana
 

Similar to Hypothesis Based Testing (HBT) Cookbook (20)

My presentation erin da802
My presentation   erin da802My presentation   erin da802
My presentation erin da802
 
I/O chapter 4
I/O chapter 4I/O chapter 4
I/O chapter 4
 
Human Assessment of Ontologies
Human Assessment of OntologiesHuman Assessment of Ontologies
Human Assessment of Ontologies
 
HBT - A Revolutionary Approach to Testing Software
HBT - A Revolutionary Approach to Testing SoftwareHBT - A Revolutionary Approach to Testing Software
HBT - A Revolutionary Approach to Testing Software
 
HBT a revolutionary approach to testing software
HBT a revolutionary approach to testing softwareHBT a revolutionary approach to testing software
HBT a revolutionary approach to testing software
 
Diagnosing behavioral problems and perception
Diagnosing behavioral problems and perceptionDiagnosing behavioral problems and perception
Diagnosing behavioral problems and perception
 
Five Steps to Excellence
Five Steps to ExcellenceFive Steps to Excellence
Five Steps to Excellence
 
Evaluation Of Competency For Stand Trial
Evaluation Of Competency For Stand TrialEvaluation Of Competency For Stand Trial
Evaluation Of Competency For Stand Trial
 
Analysis And Criticisms Of The Rorschach
Analysis And Criticisms Of The RorschachAnalysis And Criticisms Of The Rorschach
Analysis And Criticisms Of The Rorschach
 
Ethics and Integrity by Tommy Seah- A Value Added Audit on Environment
Ethics and Integrity by Tommy Seah- A Value Added Audit on EnvironmentEthics and Integrity by Tommy Seah- A Value Added Audit on Environment
Ethics and Integrity by Tommy Seah- A Value Added Audit on Environment
 
Rapid software testing
Rapid software testingRapid software testing
Rapid software testing
 
Exploratory Testing in an Agile Context
Exploratory Testing in an Agile ContextExploratory Testing in an Agile Context
Exploratory Testing in an Agile Context
 
Exploratory Testing Explained
Exploratory Testing ExplainedExploratory Testing Explained
Exploratory Testing Explained
 
Test management
Test managementTest management
Test management
 
Competency models types and techniques
Competency models types and techniquesCompetency models types and techniques
Competency models types and techniques
 
Testing Simulation Tools And Simulations.pdfTesting Simulation Tools And Simu...
Testing Simulation Tools And Simulations.pdfTesting Simulation Tools And Simu...Testing Simulation Tools And Simulations.pdfTesting Simulation Tools And Simu...
Testing Simulation Tools And Simulations.pdfTesting Simulation Tools And Simu...
 
Talent Circle 4 students
Talent Circle 4 studentsTalent Circle 4 students
Talent Circle 4 students
 
Fundamentals of testing
Fundamentals of testingFundamentals of testing
Fundamentals of testing
 
Psychological Tests And Psychological Assessment
Psychological Tests And Psychological AssessmentPsychological Tests And Psychological Assessment
Psychological Tests And Psychological Assessment
 
Lesson 19
Lesson 19Lesson 19
Lesson 19
 

More from STAG Software Private Limited

Application Scenarios of "doSmartQA -Smart Probing Assistant"
Application Scenarios of "doSmartQA -Smart Probing Assistant"Application Scenarios of "doSmartQA -Smart Probing Assistant"
Application Scenarios of "doSmartQA -Smart Probing Assistant"STAG Software Private Limited
 
Setting a clear baseline (How to test an user story #2)
Setting a clear baseline (How to test an user story #2)Setting a clear baseline (How to test an user story #2)
Setting a clear baseline (How to test an user story #2)STAG Software Private Limited
 
Question to Understand (How to test an User Story #1)
Question to Understand (How to test an User Story #1)Question to Understand (How to test an User Story #1)
Question to Understand (How to test an User Story #1)STAG Software Private Limited
 
Deliver Superior Outcomes Using HBT Visualization Tool
Deliver Superior Outcomes Using HBT Visualization ToolDeliver Superior Outcomes Using HBT Visualization Tool
Deliver Superior Outcomes Using HBT Visualization ToolSTAG Software Private Limited
 
Hypothesis Based Testing – Application and Adaptation for testing Enterprise ...
Hypothesis Based Testing – Application and Adaptation for testing Enterprise ...Hypothesis Based Testing – Application and Adaptation for testing Enterprise ...
Hypothesis Based Testing – Application and Adaptation for testing Enterprise ...STAG Software Private Limited
 
Think better using “Descriptive-Prescriptive” Approach
Think better using “Descriptive-Prescriptive” ApproachThink better using “Descriptive-Prescriptive” Approach
Think better using “Descriptive-Prescriptive” ApproachSTAG Software Private Limited
 

More from STAG Software Private Limited (20)

Application Scenarios of "doSmartQA -Smart Probing Assistant"
Application Scenarios of "doSmartQA -Smart Probing Assistant"Application Scenarios of "doSmartQA -Smart Probing Assistant"
Application Scenarios of "doSmartQA -Smart Probing Assistant"
 
Choked by technical debt?
Choked by technical debt?Choked by technical debt?
Choked by technical debt?
 
Are your quality metrics insightful?
Are your quality metrics insightful?Are your quality metrics insightful?
Are your quality metrics insightful?
 
Weighed down by automation?
Weighed down by automation?Weighed down by automation?
Weighed down by automation?
 
Covid19 and Clean Code Part 2 - Process & Criteria
Covid19 and Clean Code Part 2 - Process & CriteriaCovid19 and Clean Code Part 2 - Process & Criteria
Covid19 and Clean Code Part 2 - Process & Criteria
 
Seven Thinking Tools to Test Rapidly
Seven Thinking Tools to Test RapidlySeven Thinking Tools to Test Rapidly
Seven Thinking Tools to Test Rapidly
 
How to test less and accomplish more
How to test less and accomplish moreHow to test less and accomplish more
How to test less and accomplish more
 
Is regression hindering your progression?
Is regression hindering your progression?Is regression hindering your progression?
Is regression hindering your progression?
 
The Power of Checklist
The Power of ChecklistThe Power of Checklist
The Power of Checklist
 
The power of checklist
The power of checklist The power of checklist
The power of checklist
 
Webinar - 'Test Case Immunity’- Optimize testing
Webinar - 'Test Case Immunity’- Optimize testing Webinar - 'Test Case Immunity’- Optimize testing
Webinar - 'Test Case Immunity’- Optimize testing
 
Design Scientifically (How to test a user story)
Design Scientifically (How to test a user story)Design Scientifically (How to test a user story)
Design Scientifically (How to test a user story)
 
Setting a clear baseline (How to test an user story #2)
Setting a clear baseline (How to test an user story #2)Setting a clear baseline (How to test an user story #2)
Setting a clear baseline (How to test an user story #2)
 
Question to Understand (How to test an User Story #1)
Question to Understand (How to test an User Story #1)Question to Understand (How to test an User Story #1)
Question to Understand (How to test an User Story #1)
 
Language shapes the way you think
Language shapes the way you thinkLanguage shapes the way you think
Language shapes the way you think
 
Deliver Superior Outcomes Using HBT Visualization Tool
Deliver Superior Outcomes Using HBT Visualization ToolDeliver Superior Outcomes Using HBT Visualization Tool
Deliver Superior Outcomes Using HBT Visualization Tool
 
Hypothesis Based Testing – Application and Adaptation for testing Enterprise ...
Hypothesis Based Testing – Application and Adaptation for testing Enterprise ...Hypothesis Based Testing – Application and Adaptation for testing Enterprise ...
Hypothesis Based Testing – Application and Adaptation for testing Enterprise ...
 
Are Your Test Cases Fit For Automation?
Are Your Test Cases Fit For Automation?Are Your Test Cases Fit For Automation?
Are Your Test Cases Fit For Automation?
 
Think better using “Descriptive-Prescriptive” Approach
Think better using “Descriptive-Prescriptive” ApproachThink better using “Descriptive-Prescriptive” Approach
Think better using “Descriptive-Prescriptive” Approach
 
Improving Defect Yield - a three step approach
Improving Defect Yield - a three step approachImproving Defect Yield - a three step approach
Improving Defect Yield - a three step approach
 

Recently uploaded

The basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxThe basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxheathfieldcps1
 
Diploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfDiploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfMohonDas
 
Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.EnglishCEIPdeSigeiro
 
How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17Celine George
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRATanmoy Mishra
 
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxAUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxiammrhaywood
 
Philosophy of Education and Educational Philosophy
Philosophy of Education  and Educational PhilosophyPhilosophy of Education  and Educational Philosophy
Philosophy of Education and Educational PhilosophyShuvankar Madhu
 
UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE
 
Practical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxPractical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxKatherine Villaluna
 
Ultra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxUltra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxDr. Asif Anas
 
5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...CaraSkikne1
 
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptxSandy Millin
 
Patterns of Written Texts Across Disciplines.pptx
Patterns of Written Texts Across Disciplines.pptxPatterns of Written Texts Across Disciplines.pptx
Patterns of Written Texts Across Disciplines.pptxMYDA ANGELICA SUAN
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...raviapr7
 
How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17Celine George
 
Benefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationBenefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationMJDuyan
 
HED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfHED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfMohonDas
 
Human-AI Co-Creation of Worked Examples for Programming Classes
Human-AI Co-Creation of Worked Examples for Programming ClassesHuman-AI Co-Creation of Worked Examples for Programming Classes
Human-AI Co-Creation of Worked Examples for Programming ClassesMohammad Hassany
 
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptxClinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptxraviapr7
 

Recently uploaded (20)

The basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxThe basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptx
 
Diploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfDiploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdf
 
Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.
 
How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
 
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxAUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
 
Philosophy of Education and Educational Philosophy
Philosophy of Education  and Educational PhilosophyPhilosophy of Education  and Educational Philosophy
Philosophy of Education and Educational Philosophy
 
UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024
 
Practical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxPractical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptx
 
Ultra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxUltra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptx
 
5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...
 
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
 
Patterns of Written Texts Across Disciplines.pptx
Patterns of Written Texts Across Disciplines.pptxPatterns of Written Texts Across Disciplines.pptx
Patterns of Written Texts Across Disciplines.pptx
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...
 
How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17
 
Prelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quizPrelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quiz
 
Benefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationBenefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive Education
 
HED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfHED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdf
 
Human-AI Co-Creation of Worked Examples for Programming Classes
Human-AI Co-Creation of Worked Examples for Programming ClassesHuman-AI Co-Creation of Worked Examples for Programming Classes
Human-AI Co-Creation of Worked Examples for Programming Classes
 
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptxClinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
 

Hypothesis Based Testing (HBT) Cookbook

  • 2. © 2011-12, STAG Software Private Limited. All rights reserved. STEM is the trademark of STAG Software Private Limited. HBT is the intellectual property of STAG Software Private Limited. This e-book is presented by STAG Software Private Limited www.stagsoftware.com 2
  • 3. Hypothesis Based Testing (HBT) is a scientific personal test methodology that is unique in its approach to ensuring cleanliness of software. It is a goal focused approach, commencing with setting up of cleanliness criteria, hypothesising potential defect types that can impede this, and then performing activities to ensure that testing is purposeful and therefore effective and efficient. The central theme HBT is constructing a hypotheses of potential defects that may be probable, and then scientifically proving that they do not indeed exist. The activities of testing like test strategy, test design, tooling & automating become purposeful as these are focused on uncovering the hypothesised defect types ensuring that these activities are done scientifically and in a disciplined manner. HBT is based on sound engineering principles geared to deliver the promise of guaranteeing cleanliness. Its core value proposition is about hypothesising potential defects that may be present in the software and then allow you to engineer a staged detection model to uncover the defects faster and cheaper that other typical test methodologies. HBT fits into any development methodology and weaves into your organisational test process. HBT is powered by STEMTM (STAG Test Engineering Method) a collection of EIGHT disciplines of thinking. STEM provides the foundation for scientific thinking to perform the various activities. It is personal scientific inquiry process that is assisted by techniques, principles and guidelines to decompose the problem, identify cleanliness criteria, hypothesise potential defect types, formulate test strategy, design test cases, identify metrics and build appropriate automation. 3
  • 4. Inspirations from nature HBT has been inspired by certain ideas and these are discussed below. The inspirations have come from “Properties of matter”, “Fractional distillation”, “Sherlock Holmes”, “Picture of baby growth”. Properties of matter Physical & Chemical properties of matter allow us to: ... classify “affected by” ... understand behaviours, interactions ... enable checking purity How can we use a similar train of thought to identify “properties of cleanliness” and then “types of defects”? “Properties of the system” End user expectations Cleanliness criteria Issues in specifications, structure, environment Potential Defect Types (PDT) and behaviour 4
  • 5. Inspirations from nature Fractional distillation This is a technique to separate mixtures that have components of different boiling points In software systems, there exists a variety of defect types that may be present in the system. How can we apply this thought process to optimally uncover the defects, by “fractionally distilling” them? Can we separate these types of defects on the basis of certain properties and optimally uncover the defects? From : http://withfriendship.com 5
  • 6. Inspirations from nature Picture of baby growth The picture shows the health of the foetus/baby . This picture shows size, shape, parts and types of issues not present Seeking inspiration, can we depict the health of software system in a similar manner? Can we measure the ‘intrinsic quality’ at a stage? As we progressively evaluate in a staged manner, certain types of defects detected & removed and therefore quality grows. Can we chart this as “cleanliness index”? Source :http://www.environment.ucla.edu/media/images/Fetal_dev5.jpg 6
  • 7. Inspirations Sherlock Holmes Sherlock Holmes was person who applied deductive logic to solve mysteries. How can we see inspirations from Holmes to hypothesise the types of defects that may be present and prove presence of these? 7
  • 8. HBT - A Personal Scientific Test Methodology Test methodologies focus on activities that are driven by a process which are powered by tools, yet successful outcomes still depend a lot on experience. Typically methodologies are at organisational level. On the other hand HBT is a personal scientific methodology enabled by STEMTM , a defect detection technology to deliver “Clean Software” 8
  • 9. Scientific approach to detecting defects Cleanliness criteria What is the end user expectation of “Good Quality”? Potential Defect Types What types of issues can result in poor quality? Evaluation Stage When should I uncover them? Test Types How do I uncover them? Test Techniques What techniques to generate test cases? Scenarios/Cases What are the test cases? Are they enough? Scripts How do I execute them? Metrics & Management How good is it? How am I doing? 9
  • 10. How is HBT different from other test methodologies? The typical test methodologies in vogue have relied on strength of the process and the capability of the individual to ensure high quality in the given cost and time constraints. They lack the scientific rigour to enable full cost optimisation and more often rely on automation as the means to driving down cost and cycle time. For example, they do not provide a strong basis for assessing the quality of test cases in terms of their defect finding potential and therefore improve effectiveness and efficiency. HBT on the other hand enables you to set a clear goal for cleanliness, derive potential types of defect and then devise a “good net” to ensure that these are caught as soon as they get injected. It is intensely goal-oriented and provides you with a clear set of milestones allowing you to manage the process quickly and effectively. Goal drives la T ic B yp H Activities T Activities defect detection ...................................... ...................................... Powered by Powered by technology experience (STEM) ...................................... ...................................... ...................................... ...................................... ...................................... ...................................... ...................................... ...................................... ...................................... ...................................... hopefully results in Goal 10
  • 11. Hypothesis Based Testing - HBT 2.0 A Quick Introduction Personal, scientific test methodology. SIX stage methodology powered by EIGHT disciplines of thinking (STEMTM). Setup Hypothesise Cleanliness Criteria Potential Defect Types SUT Nine Stage Cleanliness Assessment Defect Removal Filter 11
  • 12. A quick introduction to HBT SIX stages of DOING powered EIGHT disciplines of THINKING by D8 D1 S6 S1 Analysis & Business value Assess & Understand management understanding ANALYSE EXPECTATIONS D7 D2 Execution & Defect D8 D1 reporting STEM Core hypothesis Tooling D7 D2 S5 STEM Understand 32 core SUPPORT S2 D6 D3 CONTEXT concepts D5 D4 Strategy & Visibility planning D3 D6 Devise Formulate HYPOTHESIS Tooling Test design PROOF S4 S3 D5 D4 HBT powered STEM Personal test methodology by Defect detection technology 12
  • 13. D1 Business value understanding D2 Defect hypothesis D3 Test strategy & planning Landscaping Orthogonality principle Viewpoints EFF model Tooling needs assessment Reductionist principle Defect centricity principle Defect centred AB Interaction matrix Negative thinking Quality growth principle Operational profiling Orthogonality principle Techniques landscape Attribute analysis Defect typing Process landscape GQM D4 Test design D5 Tooling Reductionist principle Input granularity principle Automation complexity Box model assessment Behaviour-Stimuli approach 32 core Minimal babysitting principle Techniques landscape concepts Separation of concerns Complexity assessment Tooling needs analysis Operational profiling Test coverage evaluation Visibility Execution & Reporting Analysis & Management D6 D7 D8 GQM Contextual awareness Gating principle Quality quantification model Defect rating principle Cycle scoping 13
  • 14. Connecting HBT Stages to the Scientific approach to detecting defects S1 Cleanliness criteria Potential defect types S3 S2 Expectations Staged & purposeful detection S4 Complete test cases S6 Goal directed measures Sensible automation S5 14
  • 15. Clear baseline Set a clear goal for quality Cleanliness criteria Potential defect types Example: Clean Water implies S1, S2 1.Colourless Staged & purposeful 2.No suspended particles 3.No bacteria detection 4.Odourless Expectations What information(properties) can be used to identify this? Complete test cases ... Marketplace,Customers, End users ... Requirement(flows), Usage, Deployment ... Features, Attributes Goal directed ... Stage of development, Interactions Sensible automation measures ... Environment, Architecture ... Behaviour, Structure 15
  • 16. A goal focused approach to cleanliness Identify potential defect types that can impede cleanliness Cleanliness criteria Potential defect types S3 Example: Data validation Timeouts Staged & purposeful Resource leakage detection Calculation Expectations Storage Presentation Complete test cases Transactional ... Scientific approach to hypothesising defects is about looking at Goal directed FIVE Aspects - Data, Logic, Structure, Environment & Usage Sensible automation measures from THREE Views - Error injection, Fault proneness & Failure Use STEM core concepts > Negative thinking (Aspect) > EFF Model (View) “A Holmes-ian way of looking at properties of elements” 16
  • 17. Levels, Types & Techniques - STRATEGY NINE levels to Cleanliness Cleanliness criteria Potential defect types L9 End user value S4 Staged & purposeful L8 Clean Deployment detection Expectations L7 Attributes met Complete test cases L6 Environment cleanliness L5 Flow correctness Goal directed Sensible automation measures L4 Behaviour correctness L3 Structural integrity L3 Quality Levels Test Techniques (T1-T4) PDT7 L2 Input interface cleanliness TT5 PDT6 TT4 TT3 T3 L2 PDT5 L1 Input cleanliness PDT4 TT3 TT2 L1 PDT3 T2 PDT2 TT2 TT1 T1 PDT1 TT1 PDT: Potential Defect Types 17
  • 18. Countable test cases & Fault coverage Countable test cases & Fault coverage Use STEM Core concepts Cleanliness criteria Potential defect types > Box model > Behaviour Stimuli approach > Techniques landscape > Coverage evaluation Staged & purposeful detection to Expectations - Model behaviour S4 - Create behaviour scenarios Complete test cases - Create stimuli (test cases) Irrespective of who designs, #scenarios/cases shall be same - COUNTABLE Goal directed Sensible automation Test Scenarios/Cases measures R1 PDT1 TS TC1,2,3 R2 PDT2 TT R3 TS TC4,5,6,7 PDT3 Requirements & Fault traceability That test cases for a given requirement shall have the ability to detect specific types of defects FAULT COVERAGE 18
  • 19. Focused scenarios + Good Automation Architecture Level based test scenarios yield shorter scripts that are more flexible for change and easily maintainable. Cleanliness criteria Potential defect types L9 End user value Staged & purposeful detection L8 Clean Deployment Expectations L7 Attributes met Complete test cases L6 Environment cleanliness S5 L5 Flow correctness Goal directed Sensible automation measures L4 Behaviour correctness L3 Structural integrity L2 Input interface cleanliness L1 Input cleanliness 19
  • 20. “Cleanliness Index” - Improved visibility L4 Cleanliness criteria Potential defect types PDT10 TT8 PDT9 TT7 Staged & purposeful L3 PDT9 TT6 Cleanliness detection PDT8 TT5 Expectations PDT7 TT4 Complete test cases L2 PDT6 PDT5 TT3 L1 PDT4 S6 Goal directed PDT3 Sensible automation measures PDT2 TT2 PDT1 TT1 Quality report Stage CC1 CC2 CC3 CC4 R1 Met R2 Not met R3 “Growth of a baby” Partially met R4 R5 20
  • 21. HBT Stages Six stages to produce clean software 21
  • 22. Six staged methodology to produce clean software The act of validation in HBT consists of “SIX Stages of DOING”. It commences with first two stages focused on a scientific approach to understanding of the customer expectations and the the context of the software. One of the key outcomes of the first two stages is “Cleanliness Criteria” that gives a clear understanding of the expectation of quality. In the third stage, the Cleanliness Criteria and the information acquired in the first two stages are used to hypothesise potential types of defects that are probable in the software. The fourth stage consists of devising a proof to scientifically ensure that the hypothesised defects can be indeed be detected cost-efficiently. The fifth stage focuses on building the tooling support needed to execute the proof. The last stage is about executing the proof and assessing if the software does indeed meet the Cleanliness Criteria. Who are the customers, end users, what do they need, and S1 S6 S1 what do they expect? Assess & Understand ANALYSE What are the features of the system, what technologies are EXPECTATIONS S2 used, architecture? D8 D1 What types of defects may be present? D2 S3 Tooling D7 Understand What types of fishes to catch? S5 STEM SUPPORT S2 D3 CONTEXT D6 D5 D4 What is strategy, plan, test scenarios/cases? S4 Sherlock Holmes Devise Formulate PROOF HYPOTHESIS What tools do I need to detect the defects? S5 Boat in the fishing analogy S4 S3 How am I doing? How is quality? S6 Fisherman 22
  • 23. Stage #1 : Understand EXPECTATIONS The perception that end-users have of how well the product delivers the needs denotes the quality of the Understand the software/system. "Needs" represent the various features that the software/system needs to have, to allow the marketplace for end-user to fulfill his tasks effectively and efficiently. "Expectations" on the other hand represent how well the the system needs are fulfilled. The final software/system may be deployed in different marketplaces addressing the needs of various types of Understand the customers. Hence it is imperative that we understand the various target markets (i.e marketplace) where the technology(ies) used software or system will be deployed. There could be different types of customers in the marketplace. Hence it is necessary to identify the various types of customers and then finally identify various types of end-users present in the customer. What we have done now is to start from outward direction i.e marketplace and adopt a customer/end-user centric view to understand the needs and expectations. Understand deployment environment Once we have identified the various types of customers and the corresponding end-users, we can move on to understand the various technologies that make up the software or the system and also a deployment environment. The intent is to get a good appreciation of the "construction components" and the target environment of deployment. It is imperative that we should have a good understanding of the internal aspects Identify end user types and not merely the external aspects of the system. & #users for each type Now we're ready to go into a detailed analysis of the various types of end-users and the typical number of users for each of these end-users. Subsequent to this, we need to identify the various business requirements Identify business i.e. "needs" for each end-user. requirements for each user type At the end of the stage, the objective is to have a good understanding of the various end-users and their needs paving the way to understanding expectations clearly. 23
  • 24. Needs & Expectations NEEDS Customers in End users Should write Should have a eraser Kids Education EXPECTATIONS Should be attractive Seniors Should be non-toxic Lead should not break easily Product Artists Drawing e.g Pencil NEEDS Draftsmen Should write Should not need sharpening Management EXPECTATIONS Corporate Thickness should be consistent Variety of thickness should be Engineering available Variety of hardness should be Admin available Needs typically features that allow to get the job done. Expectations are how well the need is satisfied. Remember Functional & Non-functional requirements ? 24
  • 25. What does “understanding” involve? Good understanding of what is expected is key to effective testing. To accomplish this, it is imperative that we commence from understanding who the various types of end users, their requirements and subsequently the expectations that they have from these. Having a deep domain knowledge helps immensely. But what if I this is a domain that I am not very conversant with? Is there a scientific way to undertand? Understanding is a non-linear activity, it is about identifying the various elements and establishing connections between these. In the process of connecting the dots, missing information is identified, leading to intelligent questions. Seeking answers to these questions aids in deepening the understanding. These are some of the elements that need to be understood. Some of the information elements are “external to the system” i.e. marketplace, customer types, end users, business requirements while some are “internal to the system” i.e. features, architecture, technology etc. Stage #1 (Understand EXPECTATIONS) focuses on “external information while Stage #2 (Understand CONTEXT) focuses on “internal information”. “Good testing is about asking intelligent questions leading to deeper understanding.” 25
  • 26. Information extracted & artefacts generated Information At each stage, certain information is extracted, understood and transformed into artefacts useful to perform effective & efficient testing. Marketplace Customers Artefacts The key outcomes as demonstrated by the artefacts are: User types System overview ‣The big picture of the system ‣The various end users ascertained for Requirements different classes of customers in different HBT marketplaces Stage #1 User type list Deployment ‣A list of business requirements for each type of end user. environment. Requirement map Technology Lifecycle stage In Stage #1, the focus is on external information that relate to marketplace, #Users/type customers, end users andHBT Stage 1 business “Good understanding is key to effective requirements. This stage is useful to get the testing. Identifying who will use what is the bigger picture of the system and its potential beginning to become customer-focused” usage and the how it is deployed.
  • 27. Deliverables from Stage #1 Should contain a a good overview of the marketplace, the various types of customers, end-users types, System overview deployment environment and technologies that will be used to build the system. Should contain a list of the various types of users for different types of customers in various market User type list segments. Should contain a list of the business requirements and high level technical features mapped to the various Requirement map individual types. STEM Discipline applied in Stage #1 The STEM discipline “Business value understanding” of STEM is applied in this stage of HBT. The two STEM core concepts of “Landscaping” and “Viewpoints” are useful in this stage to scientifically understand the expectations. 27
  • 28. Stage #2 : Understand CONTEXT In this stage the objective is to understand the technical context in terms of the various features, their relative Identify technical business value, the profile of usage and ultimately arrive at the cleanliness criteria. Note that at this stage, we features and baseline are moving inward to get a better understanding of technical features of the system. them Having identified the various business requirements mapped by each type of end-user, the next logical step is to drill-down to the various technical features for each business requirement. It is important to understand Understand the various technical features that constitute the entire system do not really work in isolation. Therefore it is dependencies necessary to understand the interplay of the features i.e. understand the dependencies of a feature with other features. Understanding this dependency is very useful at later stages of the life cycle, particularly to regress Understand profile of optimally. usage We now have a list of requirements and the corresponding technical features mapped by each end-user. We are ready to proceed logically to understand the profile of usage of each of the features by the various end- Identify critical success users. To do this it is important to understand the typical in the maximum number of users for each user type factors and then the volume of usage by each user for every technical feature. Since we already have a mapping between the end-user type and the technical feature feature, all we have to do is to understand as to approximately how many times this feature will be used by typical end-user of that end user type. The intent Prioritize value of end of this is to gain a deeper understanding of the usage profile to enable an effective strategy formulation at the users(s) and features later stage of HBT. It is not only sufficient that the features work correctly, it is equally important that the various attributes of Ensure attributes are the nonfunctional aspects of the various features are indeed met. Typically nonfunctional aspects of the testable system are identified in the highest system level, and typically turn out to be fuzzy. Good testing demands that each requirement is indeed testable. In HBT, attributes are identified for each key feature and then aggregated to form the complete set of nonfunctional requirements. We will do this in two stages: firstly identifying the Setup cleanliness critical success factors for the technical features and thereof the business requirement and then detailing the criteria critical success factors to arrive at the nonfunctional requirements or attributes. Hence after figuring out the usage profile, identify the success factors for each business requirement. 28
  • 29. Stage #2 : Understand CONTEXT Good testing is not about testing all features equally, it is about learning to focus more on those requirements/features that affect the customer experience significantly. This does not imply that some requirements/features are less important than the others, it simply means that some requirements/features are more important . Before we start detailing the various attributes, it is worthwhile to the prioritize the various requirements /features and also various end-user types. To prioritize, start by prioritizing the various types of end users in terms of their importance to the successful deployment of the final system. Subsequently rank the importance of each of the requirement/feature for each of the end-user type. At the end of this exercise, we should have a very clear understanding of the business value of each requirement/feature. Note that the understanding of usage profile comes in very handy here. Now we are ready to derive the various attributes from the previously identified success factors and ensure that they are testable. A testable requirement simply means that it is an unambiguously possible to state whether it failed all passed after executing it. In the context of attributes, testability implies that each attribute does indeed have a clear measure/metric. Therefore it is necessary to identify the measures and the expected value of the measures for each of the attribute. Having identified the various technical features and the corresponding attributes, the usage profile in the ranking of the requirements/features, we are now set to identify the various criteria that constitute the cleanliness of the intended software. Cleanliness criteria in HBT represents testable expectations. Cleanliness criteria provides a very strong basis for ensuring a goal-focused testing. This allows one to identify potential types of defects and then formulate an effective strategy in the complete set of test cases It is important that the cleanliness criteria is not vague or fuzzy. 29
  • 30. Information extracted & artefacts generated At each stage, certain information is extracted, Artefacts understood and transformed into artefacts useful to perform effective & efficient testing. Information Feature list The key outcomes as demonstrated by the Features artefacts are: Value prioritization ‣A clear list of technical features matrix ‣Ranking of features to focus on high risk Usage areas ‣Profile of usage HBT Usage profile ‣List of attributes Focus areas Stage #2 ‣Feature interactions Attributes list ‣Clarity of expectations outlined as Attributes “Cleanliness criteria” Interaction matrix Interactions Cleanliness criteria In Stage #2, the focus is on internal information that relate to technical features, their interactions, focus areas, attributes, architecture, technology. 30
  • 31. Deliverables from Stage #2 Feature list Should contain the list of technical features, that forms the technical features baseline. Value prioritization Should contain a set of users, requirements and features ranked by importance. matrix Usage profile Should contain a the profile of various operations by various end users over time. Should contain the key attributes stated objectively i.e. state expected value for all the measures Attributes list for each attribute. Should contain the which feature affects what. Note that this should list the interactions and not the details Interaction matrix of interactions. The objective is to get a rapid understanding of the linkages. Cleanliness criteria Should contain criteria that need to be met to ensure that the deployed system is indeed clean. STEM Discipline applied in Stage #2 The STEM discipline “Business value understanding” of STEM is applied in this stage of HBT. The STEM core concepts of “Interaction matrix”, “Operational profiling”, “Attribute analysis” and “GQM” are useful in this stage to scientifically understand the context. 31
  • 32. Cleanliness criteria Cleanliness criteria is a mirror of expectations, The intention is to come up with criteria that if met will ensure that system meets the expectations of the the various end users. This is not be confused with “Acceptance criteria”, as “Acceptance criteria” is typically at a higher level. Acceptance criteria is typically “extrinsic” in nature i.e. it describes aspects like long duration running, migration of existing data, clean installation and running in the final deployment environment, delivering stated performance under real-life load conditions. Cleanliness criteria represents the “intrinsic quality” i.e. what properties should the final system have to ensure that it is deemed clean? Use the properties of the FIVE aspects of Data, Business logic, Structure, Environment, Usage as applied to your application to arrive at these criteria specific to your application. Note that the cleanliness criteria should both the the functional and non-functional requirements. The recommended style of writing Cleanliness criteria is: “That the system shall meet ....” Examples: That the system is able to handle large data (need to qualify large) That the system releases resources after use. That the system displays meaningful progress for long duration activities. That the system is able to detect inappropriate environment/configuration. 32
  • 33. Stage #3 : Formulate HYPOTHESIS Having understood the expectations and the context resulting in the formulation of cleanliness criteria, we are ready to hypothesize the potential defects that could affect the cleanliness criteria. This is one of the important stages of HBT resulting in a clear articulation of the various types of defects and forms the basis for the remaining stages of HBT. The key idea is to use the external information like the feature’s behaviour, environment, attributes, usage and internal information like construction material i.e technology, architecture to hypothesize the potential defects that may be present in the software under construction. Also note that the history of the previous versions of the software or similar systems can also be used to construct and strengthen the hypothesis. Having hypothesized the potential defects, it is possible to scientifically construct a validation strategy and design adequate test cases, thereby ensuring that the final system to be deployed is indeed clean. The FIVE key aspects useful for constructing hypotheses of defects are: data, business logic, structure, environment and usage. This HBT stage allows us to follow a structured &scientific approach to the hypothesize the potential effects ensuring that we do not miss any. 33
  • 34. Stage #3 : Formulate HYPOTHESIS (continued) Firstly use the external information like data specification and business logic Identify potential faults for the five specification to identify the potential defects. The information related to data that could aspects - Data, Business logic, Structure, help are: data type, boundaries, volumes, rate, format, data interrelationships. The intent Environment, Usage should be to get into a "negative mentality" and think of what can go wrong with respect to all the information related to the data and then produce a list of potential Identify potential failures of the five defects. aspects - Data, Business logic, Structure, Environment, Usage Now use the information related to the business logic to identify the potential effects. Business logic or the intended behaviour primarily transforms the various inputs i.e. input data to outputs that the user values. The intention is to identify potential Identify potential errors that could be transformation losses. The information specific to business logic that is useful for injected in the five aspects - Data, arriving at potential defects are : the various conditions and their linkages, values of Business logic, Structure, Environment, conditions, exception handling conditions, access control and dependencies on the Usage other parts of the software. Once again, the intent is to get into a "negative mentality", and identify erroneous business flows of logic. Now identify potential defects (PD) & Up to now the focus has been on using external information like the specification of combine PDs, remove duplicate PDs data and business logic to identify the potential defects. Now focus on the internal information like structure of the system and construction materials(i.e. language, technology) used to build the system to hypothesize potential defects. Structure at the Group similar PD to form Potential highest level represents the deployment architecture while structure at the lowest level Defect Types (PDT) represents the structure of the code. Some of the structural information that could be useful to hypothesize are: flow of control, resource usage, distributed architecture, interfacing techniques, exception handling, timing information, threading, layering. As Map PDTs to the elements-under-test i.e. features/requirements explained above, continue with the similar train of thought of examining these information with intent to identify potential problems in the structure. 34
  • 35. Stage #3 : Formulate HYPOTHESIS (continued) Having identified potential defects using the behavioural and structural information, examine information related to environment and how they can affect the deployed system. By environment, we mean the associated hardware and software on which the system is deployed and the hardware, software and application resources used by the system. The objective is to examine carefully how these can affect the finally deployed system. Some of the key information related to the environment that could be useful are: hardware/software versions, system access control, application configuration information, speed of hardware (CPU, memory, hard disk, communication links), environment configuration information (e.g. #handles, cache size etc), system resources (hardware, OS and other applications). Up till now we have taken a fault-centric approach of looking for potential faults (aka defects) by examining external or internal information. In addition to a fault-centric approach, we can also view the system from potential failure points and then identify the potential defects. Additionally, it is also possible to examine the system from an error injection point of view. That is, understand the kinds of potential errors that could be injected into the system to irritate the potential defects if any. The objective is to ensure that we have examined the system from all three views (error centric, fault centric & failure centric) and thereby ensure that we have not missed any potential defects. A failure centric approach demands that we wear an end-user hat and identify the potential failures that could cause business loss. The cleanliness criteria formulated earlier could come in very handy as this would force us to think like a customer/end-user. What we trying to do is to ensure that we have considered all the potential failures and therefore hypothesized the potential defects. Now move to a user centric view to examine the various ways that an end-user could abuse the system by identifying the various ways errors could be injected into the system. Not that an end user does not always connote a physical person, it could be another system that interacts with the system via some interface. so examine the various points of interaction and look at the possibilities of their injection and then hypothesize the potential defects that could get irritated by these errors. The kinds of information that could be useful here are: workflows, data access, interesting ways of using the system, accessibility, environmental constraints faced by the physical end-user and potential deviant ways of using the system. Then consolidate the potential defects and group similar ones into potential defect types (PDT). Finally map the PDTs to the various elements- under-test i.e. feature/requirements. Now we have a clear notion as to what types of defects that we should look forward to uncovering in what parts of the system. 35
  • 36. Information extracted & artefacts generated At each stage, certain information is extracted, understood and transformed into artefacts useful to perform effective & efficient testing. Information Data Structure Artefacts The key outcomes as demonstrated by the Environment artefacts are: HBT PD catalog Stage #3 ‣List of potential defect types Business logic ‣Mapping between PDTs & the elements- Fault traceability under-test i.e. Feature/Requirement matrix Usage Attributes Past defects In Stage #2, the focus is on hypothesizing PDTs using the FIVE aspects of Data, Business logic, Structure, Environment & Usage from THREE views - Error-centric, Fault-centric & Failure-centric. 36
  • 37. Deliverables from Stage #3 PD catalog Should contain the list of potential defects and the potential defect types Fault traceability Should contain the mapping between the potential defect types/potential defects and features/requirements. matrix STEM Discipline applied in Stage #3 The STEM discipline “Defect hypothesis” of STEM is applied in this stage of HBT. The STEM core concepts of “Negative thinking”, “EFF model”, “Defect centricity principle” and “Orthogonality principle” are useful in this stage to scientifically hypothesize defects. 37
  • 38. Stage #4 : Devise PROOF (Part #1: Test Strategy & Planning) HBT being a goal focused test methodology, the intent is about figuring out an optimal approach to detect the potential of defects in the system. Therefore strategy in HBT is about staging the order of defect detection, identifying tests that are needed to uncover the specific defect types and finally choosing test techniques best suited for each type of test. Typically we have always looked at the levels of testing like unit, integration and system from the aspect of the “size” of entity-under test. Unit test is typically understood as being done on the smallest component that can be independently tested. Integration test is typically understood as being done once the various units have been integrated. System test is typically seen as the last stage of validation and is done on the whole system. What is not very necessarily very clear is the specific types of defects that are expected to be uncovered by each of these test levels. In HBT, the focus shifts to specific types of defects to be detected, and therefore the act of detection is staged to ensure an efficient detection approach. In HBT, the notion is of quality levels, where each quality level represents a milestone towards meeting the final cleanliness criteria. In other words each quality level represents a step in the staircase of quality. The notion is to ensure that the defects that can be caught earlier is indeed caught. So the first step to formulation of strategy is to stage the potential defects and thereby formulating the various quality levels. However in HBT, there are NINE pre-defined quality levels where the lowest quality level focuses on input correctness progressively going onto the highest quality level to validate of the intended business value is indeed delivered. 38
  • 39. Stage #4 : Devise PROOF (Part #1: Test Strategy & Planning) Understand scope Having identified the various types of potential defect types to be detected at various levels, it is now necessary to understand the specific types of tests needed to uncover these potential defects. In HBT each test shall be intensely goal focused. This means that a type of test shall only uncover specific type of defects. Choose quality levels The act of test type identification results in specific types of tests to be done at each of the quality levels. Now that we know what types of defects need to be detected when and where what type of tests, we need Identify test types to know how to design sufficient yet adequate test cases for each type of test. In HBT, a test technique is one that allows us to design test cases. Based on the types of defects i.e. types of tests, we have to identify the test technique(s) that is best suited for uncovering these types of the defects. Identify test techniques Now we have a clearer idea of various types of defects, the levels of detection, types of tests and test techniques., we are now ready to identify the optimal detection process best suited for design/execution of Identify detection process test cases. The the act of identifying detection process also allows us to understand whether we need technology support to be able to execute test cases and therefore pave the way for automation strategy. Identify tooling needs At this point in time we have a strategy and are ready to develop the detailed test plan. Some of the key elements of the test plan is the estimation of effort and time and formulating the various test cycles. In HBT cycles are formulated first and then effort and time estimated. Formulate cycles Finally potential risks that could come in the way of executing the test plan are identified and the risk management plan put in place. Estimate effort In summary, a strategy in HBT is a clear articulation of the quality levels, test types test techniques and detection process model. Identify risks 39
  • 40. Information extracted & artefacts generated At each stage, certain information is extracted, understood and transformed into artefacts useful Information to perform effective & efficient testing. Cleanliness criteria PDT Artefacts Attributes The key outcomes as demonstrated by the HBT Test strategy artefacts are: Techniques Stage #4 ‣Test strategy ‣Test plan Deployment env. Test plan Scope of work #Scenarios Risks In Stage #4 (Part 1) the focus is on 1 HBT Stage identifying the quality levels, test types, test techniques and the detection process. 40
  • 41. Deliverables from Stage #4 (Part #1) Test strategy Should contain the quality levels, test types, test techniques & detection process Test plan Should contain the test effort estimate, cycle details and the potential risk & mitigation plan. STEM Discipline applied in Stage #4 (Part #1) The STEM discipline “Strategy & Planning” of STEM is applied in this stage of HBT. The STEM core concepts of “Orthogonality principle”, “Quality growth principle”, “Defect centered activity breakdown” , “Cycle scoping” are useful in this stage to scientifically developing the strategy & plan. 41
  • 42. Stage #4 : Devise PROOF (Part #2: Test Design) The act of designing test cases is a crucial activity in the test life cycle. Effective testing demands that the test cases possess the power to uncover the hypothesized potential defects. It is necessary that the test cases are adequate and also optimal. In HBT the design is done level-wise and within each level test-type wise. Based on the level & type, the test entity may be different. The test design activity for an entity for a type of test at a quality level consists of two major steps, firstly to design test scenarios and then generate these test cases for each scenario. Test scenarios are designed entity-wise and therefore there is a built-in notion of requirements traceability. In addition to requirements traceability, it is expected that the test scenarios and corresponding test cases are traced to the potential types of defects that they are expected to uncover. This is termed “Fault traceability”. 42
  • 43. Stage #4 : Devise PROOF (Part #2: Test Design) Identify test level to design The act of test design commences with the identification of the quality level and then the specific type consider & identify entities of test for which the test cases are to be designed. This allows us to identify the various test entities for which test cases have to be designed. Identify conditions & data Having identified the test entities it is then required to identify the conditions that govern the business logic and the data elements that drive these conditions. Subsequent to this, build the behavioral model. Use the behavioral model to generate test scenarios. Then for every scenario, identify the data that Model the intended behaviour varies and then generate values for each data element. Finally combine the data values to generate the semi-formally test cases. Since we have designed scenarios/cases entity-wise, requirements traceability is built-in i.e. the designed Generate the test scenarios scenarios/cases automatically trace to the entity (or requirement). Now map the scenarios/cases to the hypothesized PDTs to build the fault traceability matrix. For each scenario, generate Finally assess the test adequacy of the designed scenarios/cases by checking test breadth, depth & test cases porosity. Trace scenarios to PDT & entity-under -test Assess the test adequacy by fault coverage analysis 43
  • 44. Information extracted & artefacts generated At each stage, certain information is extracted, understood and transformed into artefacts useful Information to perform effective & efficient testing. Conditions Artefacts Data Test scenarios & Logic cases The key outcomes as demonstrated by the HBT artefacts are: Stage #4 Requirements ‣Test scenarios & cases Structure ‣Requirements traceability matrix traceability matrix ‣Fault traceability matrix PDT Fault traceability matrix Defect escapes Attributes In Stage #4 (Part 2), the focus is on designing test scenarios/cases that can be proved to be adequate and have the power to uncover the hypothesized PDTs. 44
  • 45. Deliverables from Stage #4 (Part #2) Test scenarios & Should contain the test scenarios/cases for each entity for all types of tests at various quality levels cases Requirements Should contain the mapping between the scenarios/cases and the entity-under-test traceability matrix Fault traceability Should contain the mapping between the scenarios/cases and the PDTs matrix STEM Discipline applied in Stage #4 (Part #2) The STEM discipline “Test design” of STEM is applied in this stage of HBT. The STEM core concepts of Reductionist principle, Input granularity principle, Box model , Behavior-Stimuli approach, Techniques landscape, Complexity assessment,Operational profiling, Test coverage evaluation are useful in design test scenarios/cases scientifically. 45
  • 46. Stage #4 : Devise PROOF (Part #3: Metrics Design) In this stage, the objective is to design measurements to manage the process of validation in an Identify progress aspects effective and efficient manner. Since HBT is a good focused test methodology, it is necessary to device measurements that enable us to clearly show the progress towards this goal. Identify adequacy(coverage) The measurements in HBT are categorized into progress related measures, test effectiveness aspects measures and system risk measures. Therefore it is necessary to identity the various aspects related to progress, effectiveness and the system health. Identify progress aspects Once the aspects are identified, key goals related to these are identified and then the metrics formulated. Finally it is necessary to understand when to measure and how to measure. For each of the aspects identify the intended goal to meet For each of these goals, identify questions to ask To answer these questions, identify metrics Identify when you want to measure and how to measure 46
  • 47. Information extracted & artefacts generated At each stage, certain information is extracted, understood and transformed into artefacts useful to perform effective & efficient testing. Information Quality aspects Progress aspects Artefacts HBT The key outcomes as demonstrated by the Process aspects Stage #4 artefacts are: Metrics chart ‣Chart of metrics that are goal-focused Organization goals When & how to measure In Stage #4 (Part 3), the focus is on designing metrics that will ensure that we stay on course towards the goal. 47
  • 48. Deliverables from Stage #4 (Part #3) Metrics chart Should contain the list of metrics, collection frequency and a how this meets the goal. STEM Discipline applied in Stage #4 (Part #3) The STEM discipline “Visibility” of STEM is applied in this stage of HBT. The STEM core concepts of GQM, Quality quantification model are useful in design metrics that are goal-focused. 48
  • 49. Stage #5 : TOOLING support Perform tooling benefit In this stage, the objective is to analyze the support that we need from tooling/technology to analysis perform the tests. Automation does always imply scripting that is typically automating the designed scenarios. It could also involve development of test bench, custom tooling to enable the system to be tested. Identify automation scope This stage of HBT allows you to identify the tooling needs, understand issues/complexity involved, perform cost-benefit analysis, evaluate existing tools for suitability/fitment and finally Assess automation complexity devising a good architecture that provides for flexibility/maintainability before embarking onto automation. Identify the order in which scenarios need to be automated Evaluate tools Design automation architecture Develop scripts Debug and baseline scripts 49
  • 50. Information extracted & artefacts generated At each stage, certain information is extracted, Information understood and transformed into artefacts useful Artifacts to perform effective & efficient testing. Automation Needs & benefits objectives document Scope Complexity assessment report The key outcomes as demonstrated by the Scenarios to artefacts are: automate Tooling requirements ‣The reason for tooling & automation HBT ‣Challenges involved Stage #5 ‣Requirements of tooling Scenario fitness ‣Scope of tooling & automation Automation scope ‣Architecture of automation Technologies used ‣Automated scripts Automation architecture Tool info. Tooling & Scripts Complexity info. In Stage #5, the focus is on identifying tooling requirements and building automated scripts that is delivers value & ROI. 50
  • 51. Deliverables from Stage #5 Needs & benefits Should contain the technical & business need for automation. document Complexity Should contain the technical challenges of automation assessment report Tooling requirements Should contain the requirements expected out of automation Automation scope Should contain scope of automation Automation Should contain the architecture adopted to building tooling/scripts architecture Tooling & Scripts The actual tools/scripts for performing automated testing STEM Discipline applied in Stage #5 The STEM discipline “Tooling” of STEM is applied in this stage of HBT. The STEM core concepts of Automation complexity assessment, Minimal babysitting principle, Separation of concerns, Tooling needs analysis are useful in adopting a disciplined approach to tooling & automation and deliver the ROI.. 51
  • 52. Stage #6 : Assess & ANALYZE Identify test cases/scripts This stage is where you execute the test cases, record defects, report to the team and take to be executed appropriate action to ensure that the system/application is delivered on time with the requisite quality. Execute test cases, record outcomes Record defects Record learnings from the activity and the context Record status of execution Analyze execution progress Quantify quality and identify risk to delivery Update strategy, plan, scenarios, cases/scripts 52
  • 53. Information extracted & artefacts generated At each stage, certain information is extracted, Artifacts understood and transformed into artefacts useful to perform effective & efficient testing. Execution status report Information Defect report The key outcomes as demonstrated by the artefacts are: Execution ‣Report of test execution & progress Progress report ‣Defect report information HBT ‣Report on cleanliness aka quality Defect Stage #6 ‣Learnings from execution resulting in information Cleanliness report improved strategy, scenarios & cases ‣Any other key learnings Context Updated strategy, plan, scenarios & cases Key learnings In Stage #6, the focus is on ensuring a disciplined execution, intelligent analysis and continuous learning to ensure that the goal is reached. 53
  • 54. Deliverables from Stage #6 Execution status Should contain the status of test execution report Defect report Should contain defect information Progress report Should contain progress of execution and thereof the cycle Cleanliness report Should contain the cleanliness index and how well the cleanliness criteria has been met Updated strategy, Updated strategy, plan, scenarios, cases based on learnings from execution plan, scenarios & cases Key learnings Key observations/learnings that could be useful in the future STEM Discipline applied in Stage #6 The STEM disciplines of “Execution & reporting” and “Analysis and management” of STEM are applied in this stage of HBT. The STEM core concepts of Contextual awareness, Defect rating principle, Gating principle, Cycle scoping enable a disciplined execution, fosters continual learning and stay focused on the goal. 54
  • 56. Discipline #1 : Business value understanding How to This discipline enables one to understand the system, create a baseline of features, attributes and Understand a system finally expectations. This discipline consists of SEVEN tools, each of which uses certain STEM core concepts to ensure these are done in a scientific and disciplined manner. Landscaping | Viewpoints Good quality implies meeting expectations. This requires that we understand expectations in How to additions to the needs as delivered by the requirements. Understanding the intended business Create a functional baseline value to delivered is key to this. Viewpoints | Reductionist principle How to Create an attribute baseline Viewpoints | Reductionist principle How to How to Identify focus areas Understand interdependencies Value prioritisation | Viewpoints Interaction matrix How to How to Understand usage Baseline expectations Operational profiling | Viewpoints Goal-Question-Metric | Viewpoints 56
  • 57. Baseline provides the basis for future work What is to be tested needs to be clear. Remember Functional & Non-functional requirements? Functional Baseline Attribute Baseline Consists of list of features to be tested. The non-functional aspects. Essentially a agreed upon list of features. Agreed upon attributes & their values. 57
  • 58. Tools in D1 -Business value understanding STEM Core Tools Description Concepts System is viewed as a collection of information elements that are How to Landscaping interconnected. This tool enables you to come up with intelligent questions to Understand a system Viewpoints understand the various information elements and their interconnections. Commencing from an external view of end users, various use cases How to Viewpoints (requirements) are identified and then technical features that constitute the Create a functional baseline Reductionist principle use cases. This tool enables you to clearly setup a functional baseline that is used as a basis for strategy, plan, design, tooling, reporting & management. In addition to functional correctness, it is imperative that the attributes are How to Attribute analysis met,. This tool enables you to identify the attributes and ensure that these are Create an attribute baseline Viewpoints testable. All requirements/features are not equally valued by the end users. This tool How to Viewpoints allows you to rank the end users, requirements, features thereby enabling Identify focus areas Value prioritisation prioritisation of testing based on the risk and perceived value. Understanding the real life usage profile is about knowing what operations, How to Viewpoints #concurrent operations, rate of arrival are in progress at a point in time. This Understand usage Operational profiling tool allows to arrive at the closer to reality potential usage profile of the system to ensure effective non-functional tests. Understanding how a feature/requirement affects/is-dependent on other How to Interaction matrix feature/requirements is useful to understand impact & re-testing effort. This Understand interdependencies tool allows you to rapidly understand the interdependencies. How to Viewpoints This tool allows to derive cleanliness criteria that reflect the expectations. Baseline expectations Goal-Question-Metric 58
  • 59. Customers & End Users Customers in End users Kids Education Seniors Product Artists Drawing e.g Pencil Draftsmen Management Corporate Engineering Admin A product or an application may be sold in different market places made up of different kinds of customers. Each class of customer may have different types of end users who use the product. It is important to understand that each end user may have different needs & expectations. Testing is about ensuring that the product will indeed satisfy the variety of needs & expectations 59
  • 60. Needs & Expectations NEEDS Customers in End users Should write Should have a eraser Kids Education EXPECTATIONS Should be attractive Seniors Should be non-toxic Lead should not break easily Product Artists Drawing e.g Pencil NEEDS Draftsmen Should write Should not need sharpening Management EXPECTATIONS Corporate Thickness should be consistent Variety of thickness should be Engineering available Variety of hardness should be Admin available Needs typically features that allow to get the job done. Expectations are how well the need is satisfied. Remember Functional & Non-functional requirements ? 60
  • 61. Customer Profile Customer #1 Customer #2 Customer #3 Customer #4 Different customers have different types of end users, and differing number of users for type of end user. 61
  • 62. Customer Profile & Usage How many What does each one use? What types users What is order of importance? of users What is the usage frequency? F1 F2 F3 F4 System F5 F6 F7 F8 Different end users may use the system differently in terms of what they use, frequency of usage and how they value each each feature. 62
  • 63. Business Value Ultimately end users need the system to do their job BETTER, FASTER, CHEAPER and deliver value to their customers. Understand that it is about “business value” of system - how does the system help my business to do BETTER, FASTER, CHEAPER. 63
  • 64. Discipline #2 : Defect hypothesis 64
  • 65. Discipline #2 : Defect hypothesis How to This discipline enables one to hypothesise potential defect types that may be present in the system Hypothesise defects under test and setup a clear goal approach to detection/prevention. Goal focused approach implies that we map the hypothesised potential defect types (PDT) to the elements-under-test i.e feature/ Negative thinking | EFF model | requirements. Defect centricity principle This discipline consists of TWO tools, each of which uses certain STEM core concepts to ensure these are done in a scientific and disciplined manner. How to Setup goal-focus Hypothesis is done by scientifically examining certain properties of the system and can be complemented by ones experience. Orthogonality principle 65