| ======================================== | |
| Compiler-rt Testing Infrastructure Guide | |
| ======================================== | |
| .. contents:: | |
| :local: | |
| Overview | |
| ======== | |
| This document is the reference manual for the compiler-rt modifications to the | |
| testing infrastructure. Documentation for the infrastructure itself can be found at | |
| :ref:`llvm_testing_guide`. | |
| LLVM testing infrastructure organization | |
| ======================================== | |
| The compiler-rt testing infrastructure contains regression tests which are run | |
| as part of the usual ``make check-all`` and are expected to always pass -- they | |
| should be run before every commit. | |
| Quick start | |
| =========== | |
| The regressions tests are in the "compiler-rt" module and are normally checked | |
| out in the directory ``llvm/projects/compiler-rt/test``. Use ``make check-all`` | |
| to run the regression tests after building compiler-rt. | |
| REQUIRES, XFAIL, etc. | |
| --------------------- | |
| Sometimes it is necessary to restrict a test to a specific target or mark it as | |
| an "expected fail" or XFAIL. This is normally achieved using ``REQUIRES:`` or | |
| ``XFAIL:`` and the ``target=<target-triple>`` feature, typically with a regular | |
| expression matching an appropriate substring of the triple. Unfortunately, the | |
| behaviour of this is somewhat quirky in compiler-rt. There are two main | |
| pitfalls to avoid. | |
| The first pitfall is that these regular expressions may inadvertently match | |
| more triples than expected. For example, ``XFAIL: target=mips{{.*}}`` matches | |
| ``mips-linux-gnu``, ``mipsel-linux-gnu``, ``mips64-linux-gnu``, and | |
| ``mips64el-linux-gnu``. Including a trailing ``-`` such as in | |
| ``XFAIL: target=mips-{{.*}}`` can help to mitigate this quirk but even that has | |
| issues as described below. | |
| The second pitfall is that the default target triple is often inappropriate for | |
| compiler-rt tests since compiler-rt tests may be compiled for multiple targets. | |
| For example, a typical build on an ``x86_64-linux-gnu`` host will often run the | |
| tests for both x86_64 and i386. In this situation ``XFAIL: target=x86_64{{{.*}}`` | |
| will mark both the x86_64 and i386 tests as an expected failure while | |
| ``XFAIL: target=i386{{.*}}`` will have no effect at all. | |
| To remedy both pitfalls, compiler-rt tests provide a feature string which can | |
| be used to specify a single target. This string is of the form | |
| ``target-is-${arch}`` where ``${arch}}`` is one of the values from the | |
| following lines of the CMake output:: | |
| -- Compiler-RT supported architectures: x86_64;i386 | |
| -- Builtin supported architectures: i386;x86_64 | |
| So for example ``XFAIL: target-is-x86_64`` will mark a test as expected to fail | |
| on x86_64 without also affecting the i386 test and ``XFAIL: target-is-i386`` | |
| will mark a test as expected to fail on i386 even if the default target triple | |
| is ``x86_64-linux-gnu``. Directives that use these ``target-is-${arch}`` string | |
| require exact matches so ``XFAIL: target-is-mips``, | |
| ``XFAIL: target-is-mipsel``, ``XFAIL: target-is-mips64``, and | |
| ``XFAIL: target-is-mips64el`` all refer to different MIPS targets. |