doc: fix link to fate.libav.org
[libav.git] / doc / fate.texi
CommitLineData
1ebbdda1
LB
1\input texinfo @c -*- texinfo -*-
2
3@settitle FATE Automated Testing Environment
4@titlepage
5@center @titlefont{FATE Automated Testing Environment}
6@end titlepage
7
8@top
9
10@contents
11
12@chapter Introduction
13
14FATE provides a regression testsuite embedded within the Libav build system.
15It can be run locally and optionally configured to send reports to a web
16aggregator and viewer @url{http://fate.libav.org}.
17
18It is advised to run FATE before submitting patches to the current codebase
19and provide new tests when submitting patches to add additional features.
20
21@chapter Running FATE
22
23@section Samples and References
24In order to run, FATE needs a large amount of data (samples and references)
25that is provided separately from the actual source distribution.
26
27To inform the build system about the testsuite location, pass
28@option{--samples=<path to the samples>} to @command{configure} or set the
29@var{SAMPLES} Make variable or the @var{FATE_SAMPLES} environment variable
30to a suitable value.
31
32The dataset is available through @command{rsync}, is possible to fetch
33the current sample using the straight rsync command or through a specific
34@ref{Makefile target}.
35
36@example
37# rsync -aL rsync://fate-suite.libav.org/fate-suite/ fate-suite
38@end example
39
40@example
41# make fate-rsync SAMPLES=fate-suite
42@end example
43
44
45@chapter Manual Run
46FATE regression test can be run through @command{make}.
47Specific Makefile targets and Makefile variables are available:
48
49@anchor{Makefile target}
50@section FATE Makefile targets
51@table @option
52@item fate-list
53List all fate/regression test targets.
54@item fate-rsync
55Shortcut to download the fate test samples to the specified testsuite location.
56@item fate
57Run the FATE test suite (requires the fate-suite dataset).
58@end table
59
60@section Fate Makefile variables
61@table @option
62@item V
63Verbosity level, can be set to 0, 1 or 2.
64@table @option
65 @item 0
66 show just the test arguments
67 @item 1
68 show just the command used in the test
69 @item 2
70 show everything
71@end table
72@item SAMPLES
73Specify or override the path to the FATE samples at make time, it has a
74meaning only while running the regression tests.
75@item THREADS
76Specify how many threads to use while running regression tests, it is
77quite useful to detect thread-related regressions.
018f39ef
AK
78@item CPUFLAGS
79Specify a mask to be applied to autodetected CPU flags.
1ebbdda1
LB
80@end table
81
82@example
018f39ef 83 make V=1 SAMPLES=/var/fate/samples THREADS=2 CPUFLAGS=mmx fate
1ebbdda1
LB
84@end example
85
86@chapter Automated Tests
87In order to automatically testing specific configurations, e.g. multiple
88compilers, @command{tests/fate.sh} is provided.
89
7b089b79
DEP
90This shell script builds Libav, runs the regression tests and prepares
91a report that can be sent to @url{http://fate.libav.org/} or directly
92examined locally.
1ebbdda1
LB
93
94@section Testing Profiles
95The configuration file passed to @command{fate.sh} is shell scripts as well.
96
97It must provide at least a @var{slot} identifier, the @var{repo} from
98which fetch the sources, the @var{samples} directory, a @var{workdir} with
99enough space to build and run all the tests.
100Optional submit command @var{fate_recv} and a @var{comment} to describe
101the testing profile are available.
102
103Additional optional parameter to tune the Libav building and reporting process
104can be passed.
105
106@example
107slot= # some unique identifier
108repo=git://git.libav.org/libav.git # the source repository
109samples=/path/to/fate/samples
110workdir= # directory in which to do all the work
111fate_recv="ssh -T fate@@fate.libav.org" # command to submit report
112comment= # optional description
113
114# the following are optional and map to configure options
115arch=
116cpu=
117cross_prefix=
118cc=
119target_os=
120sysroot=
121target_exec=
122target_path=
123extra_cflags=
124extra_ldflags=
125extra_libs=
126extra_conf= # extra configure options not covered above
127
128#make= # name of GNU make if not 'make'
129makeopts= # extra options passed to 'make'
130#tar= # command to create a tar archive from its arguments on
131 # stdout, defaults to 'tar c'
132@end example
133
134@section Submitting Reports
135In order to send reports you need to create an @command{ssh} key and send it
136to @email{root@@libav.org}.
137The current server fingerprint is @var{a4:99:d7:d3:1c:92:0d:56:d6:d5:61:be:01:ae:7d:e6}