Testen von Software - Übung
last update: 24.06.2004
Vorlesung Testen im Grossen
Disclaimer: Das Skript, diente mir als Notiz für die Vorlesung.
Es gibt keine Garantie, dass es fehlerfrei ist.
Die benutzten Quellen sind auf der letzten Seite angegeben.
Testen im Grossen(pdf)
Allgemeines zur Übung
Zum erfolgreichen Absolvieren der Übung Testen von Software sind folgende 3 Kriterien zu erfüllen:
- kontinuierliche und aktive Teilnahme an den Übungsveranstaltungen
- Präsentation eines Testwerkzeuges
- Schreiben einer Kurzzusammenfassung über das referierte Testwerkzeug
Für die Zusammenfassung sollen folgende Templates genutzt werden:
Zusammenfassung Testwerkzeug (MS Word Version)
Zusammenfassung Testwerkzeug (OpenOffice Version)
Termine
Tools
Program checker
www.splint.org
Splint is a tool for statically checking C programs for security vulnerabilities
and coding mistakes. With minimal effort, Splint can be used as a better lint.
If additional effort is invested adding annotations to programs,
Splint can perform stronger checking than can be done by any standard lint.
http://artho.com/jlint/
A Java Programm checker.
Jlint will check your Java code and find bugs, inconsistencies and synchronization problems by doing data flow analysis and building the lock graph.
http://findbugs.sourceforge.net/
FindBugs is a program to find bugs in Java programs.
It looks for instances of "bug patterns" --- code instances that are likely to be errors.
http://pmd.sourceforge.net/
is a Java source code analyzer. It finds unused variables, empty catch blocks, unnecessary object creation, and so forth.
PMD scans Java source code and looks for potential problems. It has plugins for JEdit, JBuilder, Netbeans/Forte/Sun ONE, IntelliJ IDEA, Maven, Ant, Eclipse, Gel, and Emacs.
Regressiontest
http://expect.nist.gov/
programmed dialogue with interactive programs
http://www.gnu.org/software/dejagnu/
DejaGnu is a framework for testing other programs. Its purpose is to provide a single front end for all tests. Think of it as a custom library of Tcl procedures crafted to support writing a test harness. A Test Harness is the testing infrastructure that is created to support a specific program or tool.
Junit
http://www.junit.org
JUnit is a regression testing framework written by Erich Gamma and Kent Beck. It is used by the developer who implements unit tests in Java.
GUI Test
http://sourceforge.net/projects/jfcunit/
http://jfcunit.sourceforge.net
jfcUnit enables developers to write test cases for Java Swing based applications. It provides support for:
1.Obtaining handles on Windows/Dialogs opened by the Java code.
2.Locating components within a component hierarchy that occur within the containers found above.
3.Raising events on the found components, e.g. clicking a button, typing text in a TextComponent.
4.Handling testing of components in a thread safe manner.
Since version 2.0, jfcUnit provides XML Recording and playback. This allows users to quickly and automatically generate/edit scripts to drive the testing. The XML API's are open and allow for developers to define there own XML tag handlers. See the example JFCUnit via XML.
http://abbot.sourceforge.net/
The Abbot framework provides automated event generation and validation of Java GUI components, improving upon the very rudimentary functions provided by the java.awt.Robot class (A Better 'Bot). The framework may be invoked directly from Java code or accessed without programming through the use of scripts. It is suitable for use both by developers for unit tests and QA for functional testing.
http://www.ph-ludwigsburg.de/mathematik/personal/spannagel/jacareto/
Java Capture & Replay Toolkit - Jacareto is a capture&replay framework for Java applications and applets. It allows you to easily create your own capture&replay tools.
X11::GUITest / Win32::GuiTest
http://sourceforge.net/projects/x11guitest/
http://triumvir.org/articles/guitest-wd-mag/
http://www.cpan.org/authors/id/E/ER/ERNGUI/
X11::GUITest is a Perl package intended to facilitate the testing of GUI applications by means of user emulation. It can be used to test and interact with GUI applications which have been built (Xlib, GTK, etc.) upon the X toolkit. (GUItest of Ped?)
http://pounder.sourceforge.net
Pounder is a utility for automating Java GUI tests. It allows developers to dynamically load GUI- Components , record scripts, and then use those scripts in a test harness. Pounder differentiates itself from other utilities by allowing you to examine the results of a test run in source, while maintaining a separate GUI script that can be re-recorded if necessary.
Testmanagement
www.bugzilla.org
Bugzilla is a bug- or issue-tracking system. Bug-tracking systems allow individual or groups of developers effectively to keep track of outstanding problems with their product. Bugzilla was originally written by Terry Weissman in a programming language called TCL, to replace a rudimentary bug-tracking database used internally by Netscape Communications. Terry later ported Bugzilla to Perl from TCL, and in Perl it remains to this day. Most commercial defect-tracking software vendors at the time charged enormous licensing fees, and Bugzilla quickly became a favorite of the open-source crowd (with its genesis in the open-source browser project, Mozilla). It is now the de-facto standard defect-tracking system against which all others are measured.
Profiling
Material:
Buch Programmierwerkzeuge S. 223 - 228
http://www.gnu.org/software/binutils/manual/gprof-2.9.1/gprof.html
Material:
IX- Artikel 04/2004 S 42.
http://gd.tuwien.ac.at/languages/java/GoToJava2/html/k100298.html
Tools:
Jrat: jrat.sourceforge.net
Hpjmeter: www.hp.com/products1/unix/java/hpjmeter
Hyades: www.eclipse.org/hyades
JunitPerf: http://clarkware.com/software/JUnitPerf.html
Code Coverage
http://gcc.gnu.org/onlinedocs/gcc-3.0/gcc_8.html
gcov is a test coverage program. Use it in concert with GNU CC to analyze your programs to help create more efficient, faster running code. You can use gcov as a profiling tool to help discover where your optimization efforts will best affect your code.
Material:
Buch: Zeller/Krinke, Programmierwekzeuge S.229 ? 235
jcoverage
(nicht getested)
www.jcoverage.com
Material:
Artikel Linuxjournal http://www.linuxjournal.com//article.php?sid=5673
Artikel IX 04/2004: Offenbar unsicher
Tools:
Flawfinder: http://www.dwheeler.com/flawfinder
Rats http://www.securesw.com/download_rats.htm
PSCAN http://www.striker.ottawa.on.ca/~aland/pscan/ (historically)
bfbtester http://bfbtester.sourceforge.net/
distributed Testing
http://qat.sourceforge.net/
QAT was developed to ease the issues encountered by having to perform Quality Assurance tests across a variety of hardware and software combinations. The QAT tool can be divided into two main sections, the Agent, responsible for actually running each test or group of tests, and the Harness, which is responsible for test selection, management, result and agent co-ordination.
http://www.codesourcery.com/qm/qmtest
CodeSourcery's QMTest provides a cost-effective general purpose testing solution that allows an organization to implement a robust, easy-to-use testing program tailored to its needs. QMTest's extensible architecture allows it to handle a wide range of application domains: everything from compilers to graphical user interfaces to web-based applications.
Memory Checks
http://www.gnu.org/software/checker/checker.html
Checker is a tool which finds memory errors at runtime. Its primary function is to emit a warning when the program reads an uninitialized variable or memory area, or when the program accesses an unallocated memory area.
http://freshmeat.net/projects/leaktracer/
LeakTracer is a small tool for checking a C++ program for memory leaks. To use LeakTracer, run your program using the provided LeakCheck script. It uses the LD_PRELOAD feature to "overlay" some functions on top of your functions (no recompile needed). LeakTracer uses gdb to print out the exact line where the memory was allocated and not freed - this of course means you have to free all dynamically allocated data. LeakTracer also overrides the global operator new and operator delete.
http://valgrind.kde.org/
is a GPL'd system for debugging and profiling x86-Linux programs. With the tools that come with Valgrind, you can automatically detect many memory management and threading bugs, avoiding hours of frustrating bug-hunting, making your programs more stable. You can also perform detailed profiling to help speed up your programs.
Systematic Software Testing
http://cm.bell-labs.com/who/god/verisoft/
is a tool for systematically exploring the state spaces of systems composed of several concurrent processes executing arbitrary code written in any language. The state space of a system is a directed graph that represents the combined behavior of all the components of the system. Paths in this graph correspond to sequences of operations (scenarios) that can be observed during executions of the system. VeriSoft systematically explores the state space of a system by controlling and observing the execution of all the components, and by reinitializing their executions. It searches for coordination problems (deadlocks, divergences, etc.) between concurrent components, and for violations of user-specified assertions. VeriSoft can always guarantee a complete coverage of the state space up to some depth; hence, all possible executions of the system up to that depth are guaranteed to be covered.
http://www.softwareautomation.com/
a visual environment for Java(TM)/C/C++/VB software testing,
quality assurance, documentation, and maintenance on Unix/Linux/Windows