FIT4004 Lecture Notes - Lecture 4: Fuzz Testing, Security Testing, Integration Testing
15 Jun 2018
School
Department
Course
Professor
Week 4 - Software testing strategies
Software testing: consists of the dynamic verification that a program provides expected
behaviors on a finite set of test cases , suitably selected from the usually infinite
execution domain.
What’s not testing: (things you don’t run)
●Formal methods
●Inspections
Testing objectives:
●Functional correctness
○Perfection is infeasible
○Eliminate as many faults as possible
○Improve reliability as much as possible
●Performance: testing to ensure that software meets resource utilization
requirements(eg. clock or CPU time) *usually automated
●Security: (Hardest things to provide as a developer, and hardest to assess from a
QA perspective)
○“Fuzz testing”: bombarding the system under test with random, or
randomly-modified data
○Vulnerability scanners that systematically test applications against known
bugs in non-updated libraries and infrastructure
○Pen testing: will examine not only the software, but also the
organizational context in which it is deployed
●Usability
●Reliability - How regularly will it fail to perform as specified
●Robustness - Determines how often things break. Failures are injected into the
system (for instance, by shutting down part of a system in the middle of an
operation) and the behaviour of the system is monitored to ensure that the
consequences of the failures are acceptable.
●User acceptance: Is where the system as a whole is tested to ensure it meets
users’ high-level business requirements. In the "verification and validation"
spectrum, this is very much at that validation end.
●Regression - repeating some testing of a system to ensure that software
behaviour hasn't changed (except in some desirable way) after modification.
●User Acceptance Testing:
○Is the last phase of the software testing process
○The most important peer group to include in UAT testing are “real” end
users of your software. Every role and stakeholder group should be
included.
○to make sure it can handle required tasks in real-world scenarios,
according to specifications.
○Meets the user’s needs, which may be different to specifications.
Test Levels:
1. Unit testing
2. Integration testing
3. System Testing
●Functional testing
●GUI testing/Usability testing
●Security testing
●Performance testing
Functional correctness testing
How to select test cases:
●Exploratory testing (ad-hoc)
○Used in industry
○Not useful for unit testing
○Issues with repeatability
○Iterative: firstly ‘go with ur gut’ then gradually develop mental model of
system
●Systematic approaches
○Structural (white-box)
■Insufficient - if functionality is missing, won’t pick it up as testing’s
based on code
■Easier to mechanically assess for adequacy
○Functional (black-box)
■Random testing
■Equivalence Partitioning (subdomain testing)
Black-box testing techniques:
