| <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"> |
| <HTML |
| ><HEAD |
| ><TITLE |
| >The files DejaGnu produces.</TITLE |
| ><META |
| NAME="GENERATOR" |
| CONTENT="Modular DocBook HTML Stylesheet Version 1.76b+ |
| "><LINK |
| REL="HOME" |
| TITLE="DejaGnu" |
| HREF="book1.html"><LINK |
| REL="UP" |
| TITLE="Running Tests" |
| HREF="c401.html"><LINK |
| REL="PREVIOUS" |
| TITLE="Runtest" |
| HREF="x428.html"><LINK |
| REL="NEXT" |
| TITLE="Customizing DejaGnu" |
| HREF="c848.html"></HEAD |
| ><BODY |
| CLASS="SECT1" |
| BGCOLOR="#FFFFFF" |
| TEXT="#000000" |
| LINK="#0000FF" |
| VLINK="#840084" |
| ALINK="#0000FF" |
| ><DIV |
| CLASS="NAVHEADER" |
| ><TABLE |
| SUMMARY="Header navigation table" |
| WIDTH="100%" |
| BORDER="0" |
| CELLPADDING="0" |
| CELLSPACING="0" |
| ><TR |
| ><TH |
| COLSPAN="3" |
| ALIGN="center" |
| >DejaGnu: The GNU Testing Framework</TH |
| ></TR |
| ><TR |
| ><TD |
| WIDTH="10%" |
| ALIGN="left" |
| VALIGN="bottom" |
| ><A |
| HREF="x428.html" |
| ACCESSKEY="P" |
| ><<< Previous</A |
| ></TD |
| ><TD |
| WIDTH="80%" |
| ALIGN="center" |
| VALIGN="bottom" |
| >Running Tests</TD |
| ><TD |
| WIDTH="10%" |
| ALIGN="right" |
| VALIGN="bottom" |
| ><A |
| HREF="c848.html" |
| ACCESSKEY="N" |
| >Next >>></A |
| ></TD |
| ></TR |
| ></TABLE |
| ><HR |
| ALIGN="LEFT" |
| WIDTH="100%"></DIV |
| ><DIV |
| CLASS="SECT1" |
| ><H1 |
| CLASS="SECT1" |
| ><A |
| NAME="OUTPUTFILES" |
| ></A |
| >The files DejaGnu produces.</H1 |
| ><P |
| >DejaGnu always writes two kinds of output files: summary |
| logs and detailed logs. The contents of both of these are |
| determined by your tests.</P |
| ><P |
| >For troubleshooting, a third kind of output file is useful: |
| use <TT |
| CLASS="OPTION" |
| >--debug</TT |
| > to request an output file showing |
| details of what <SPAN |
| CLASS="PRODUCTNAME" |
| >Expect</SPAN |
| > is doing |
| internally.</P |
| ><DIV |
| CLASS="SECT2" |
| ><H2 |
| CLASS="SECT2" |
| ><A |
| NAME="SUM" |
| ></A |
| >Summary File</H2 |
| ><P |
| >DejaGnu always produces a summary output file |
| <TT |
| CLASS="FILENAME" |
| >tool.sum</TT |
| >. This summary shows the names of |
| all test files run; for each test file, one line of output from |
| each <B |
| CLASS="COMMAND" |
| >pass</B |
| > command (showing status |
| <I |
| CLASS="EMPHASIS" |
| >PASS</I |
| > or <I |
| CLASS="EMPHASIS" |
| >XPASS</I |
| >) or |
| <B |
| CLASS="COMMAND" |
| >fail</B |
| > command (status |
| <I |
| CLASS="EMPHASIS" |
| >FAIL</I |
| > or <I |
| CLASS="EMPHASIS" |
| >XFAIL</I |
| >); |
| trailing summary statistics that count passing and failing tests |
| (expected and unexpected); and the full pathname and version |
| number of the tool tested. (All possible outcomes, and all |
| errors, are always reflected in the summary output file, |
| regardless of whether or not you specify |
| <TT |
| CLASS="OPTION" |
| >--all</TT |
| >.)</P |
| ><P |
| >If any of your tests use the procedures |
| <B |
| CLASS="COMMAND" |
| >unresolved</B |
| >, <B |
| CLASS="COMMAND" |
| >unsupported</B |
| >, |
| or <B |
| CLASS="COMMAND" |
| >runtested</B |
| >, the summary output also |
| tabulates the corresponding outcomes.</P |
| ><P |
| >For example, after <B |
| CLASS="COMMAND" |
| >runtest --tool |
| binutils</B |
| >, look for a summary log in |
| <TT |
| CLASS="FILENAME" |
| >binutils.sum</TT |
| >. Normally, DejaGnu writes this |
| file in your current working directory; use the |
| <TT |
| CLASS="OPTION" |
| >--outdir</TT |
| > option to select a different |
| directory.</P |
| ><DIV |
| CLASS="EXAMPLE" |
| ><A |
| NAME="AEN780" |
| ></A |
| ><P |
| ><B |
| >Example 1. Here is a short sample summary log</B |
| ></P |
| ><TABLE |
| BORDER="0" |
| BGCOLOR="#E0E0E0" |
| WIDTH="100%" |
| ><TR |
| ><TD |
| ><PRE |
| CLASS="SCREEN" |
| > Test Run By rob on Mon May 25 21:40:57 PDT 1992 |
| === gdb tests === |
| Running ./gdb.t00/echo.exp ... |
| PASS: Echo test |
| Running ./gdb.all/help.exp ... |
| PASS: help add-symbol-file |
| PASS: help aliases |
| PASS: help breakpoint "bre" abbreviation |
| FAIL: help run "r" abbreviation |
| Running ./gdb.t10/crossload.exp ... |
| PASS: m68k-elf (elf-big) explicit format; loaded |
| XFAIL: mips-ecoff (ecoff-bigmips) "ptype v_signed_char" signed C types |
| === gdb Summary === |
| # of expected passes 5 |
| # of expected failures 1 |
| # of unexpected failures 1 |
| /usr/latest/bin/gdb version 4.6.5 -q |
| </PRE |
| ></TD |
| ></TR |
| ></TABLE |
| ></DIV |
| ></DIV |
| ><DIV |
| CLASS="SECT2" |
| ><H2 |
| CLASS="SECT2" |
| ><A |
| NAME="LOG" |
| ></A |
| >Log File</H2 |
| ><P |
| >DejaGnu also saves a detailed log file |
| <TT |
| CLASS="FILENAME" |
| >tool.log</TT |
| >, showing any output generated by |
| tests as well as the summary output. For example, after |
| <B |
| CLASS="COMMAND" |
| >runtest --tool binutils</B |
| >, look for a detailed |
| log in <TT |
| CLASS="FILENAME" |
| >binutils.log</TT |
| >. Normally, DejaGnu |
| writes this file in your current working directory; use the |
| <TT |
| CLASS="OPTION" |
| >--outdir</TT |
| > option to select a different |
| directory.</P |
| ><DIV |
| CLASS="EXAMPLE" |
| ><A |
| NAME="AEN790" |
| ></A |
| ><P |
| ><B |
| >Example 2. Here is a brief example showing a detailed log for |
| <SPAN |
| CLASS="PRODUCTNAME" |
| >G++</SPAN |
| > tests</B |
| ></P |
| ><TABLE |
| BORDER="0" |
| BGCOLOR="#E0E0E0" |
| WIDTH="100%" |
| ><TR |
| ><TD |
| ><PRE |
| CLASS="SCREEN" |
| > Test Run By rob on Mon May 25 21:40:43 PDT 1992 |
| |
| === g++ tests === |
| |
| --- Running ./g++.other/t01-1.exp --- |
| PASS: operate delete |
| |
| --- Running ./g++.other/t01-2.exp --- |
| FAIL: i960 bug EOF |
| p0000646.C: In function `int warn_return_1 ()': |
| p0000646.C:109: warning: control reaches end of non-void function |
| p0000646.C: In function `int warn_return_arg (int)': |
| p0000646.C:117: warning: control reaches end of non-void function |
| p0000646.C: In function `int warn_return_sum (int, int)': |
| p0000646.C:125: warning: control reaches end of non-void function |
| p0000646.C: In function `struct foo warn_return_foo ()': |
| p0000646.C:132: warning: control reaches end of non-void function |
| |
| --- Running ./g++.other/t01-4.exp --- |
| FAIL: abort |
| 900403_04.C:8: zero width for bit-field `foo' |
| --- Running ./g++.other/t01-3.exp --- |
| FAIL: segment violation |
| 900519_12.C:9: parse error before `;' |
| 900519_12.C:12: Segmentation violation |
| /usr/latest/bin/gcc: Internal compiler error: program cc1plus got fatal signal |
| |
| === g++ Summary === |
| |
| # of expected passes 1 |
| # of expected failures 3 |
| /usr/latest/bin/g++ version cygnus-2.0.1 |
| </PRE |
| ></TD |
| ></TR |
| ></TABLE |
| ></DIV |
| ></DIV |
| ><DIV |
| CLASS="SECT2" |
| ><H2 |
| CLASS="SECT2" |
| ><A |
| NAME="DEBUGFILE" |
| ></A |
| >Debug Log File</H2 |
| ><P |
| >With the <TT |
| CLASS="OPTION" |
| >--debug</TT |
| > option, you can request |
| a log file showing the output from |
| <SPAN |
| CLASS="PRODUCTNAME" |
| >Expect</SPAN |
| > itself, running in debugging |
| mode. This file (<TT |
| CLASS="FILENAME" |
| >dbg.log</TT |
| >, in the directory |
| where you start <B |
| CLASS="COMMAND" |
| >runtest</B |
| >) shows each pattern |
| <SPAN |
| CLASS="PRODUCTNAME" |
| >Expect</SPAN |
| > considers in analyzing test |
| output.</P |
| ><P |
| >This file reflects each <B |
| CLASS="COMMAND" |
| >send</B |
| > command, |
| showing the string sent as input to the tool under test; and |
| each <SPAN |
| CLASS="PRODUCTNAME" |
| >Expect</SPAN |
| > command, showing each |
| pattern it compares with the tool output.</P |
| ><DIV |
| CLASS="EXAMPLE" |
| ><A |
| NAME="AEN805" |
| ></A |
| ><P |
| ><B |
| >Example 3. The log messages begin with a message of the form</B |
| ></P |
| ><TABLE |
| BORDER="0" |
| BGCOLOR="#E0E0E0" |
| WIDTH="100%" |
| ><TR |
| ><TD |
| ><PRE |
| CLASS="SCREEN" |
| > expect: does {<SPAN |
| CLASS="SYMBOL" |
| >tool output</SPAN |
| >} (spawn_id <SPAN |
| CLASS="SYMBOL" |
| >n</SPAN |
| >) |
| match pattern {<I |
| CLASS="EMPHASIS" |
| >expected pattern</I |
| >}? |
| |
| </PRE |
| ></TD |
| ></TR |
| ></TABLE |
| ></DIV |
| ><P |
| >For every unsuccessful match, |
| <SPAN |
| CLASS="PRODUCTNAME" |
| >Expect</SPAN |
| > issues a |
| <I |
| CLASS="EMPHASIS" |
| >no</I |
| > after this message; if other patterns |
| are specified for the same <SPAN |
| CLASS="PRODUCTNAME" |
| >Expect</SPAN |
| > |
| command, they are reflected also, but without the first part of |
| the message (<I |
| CLASS="EMPHASIS" |
| >expect... match pattern</I |
| >).</P |
| ><P |
| >When <SPAN |
| CLASS="PRODUCTNAME" |
| >Expect</SPAN |
| > finds a match, the |
| log for the successful match ends with <I |
| CLASS="EMPHASIS" |
| >yes</I |
| >, |
| followed by a record of the <SPAN |
| CLASS="PRODUCTNAME" |
| >Expect</SPAN |
| > |
| variables set to describe a successful match.</P |
| ><DIV |
| CLASS="EXAMPLE" |
| ><A |
| NAME="AEN820" |
| ></A |
| ><P |
| ><B |
| >Example 4. Here is an excerpt from the debugging log for a |
| <SPAN |
| CLASS="PRODUCTNAME" |
| >GDB</SPAN |
| > test:</B |
| ></P |
| ><TABLE |
| BORDER="0" |
| BGCOLOR="#E0E0E0" |
| WIDTH="100%" |
| ><TR |
| ><TD |
| ><PRE |
| CLASS="SCREEN" |
| > send: sent {break gdbme.c:34\n} to spawn id 6 |
| expect: does {} (spawn_id 6) match pattern {Breakpoint.*at.* file |
| gdbme.c, line 34.*\(gdb\) $}? no |
| {.*\(gdb\) $}? no |
| expect: does {} (spawn_id 0) match pattern {return} ? no |
| {\(y or n\) }? no |
| {buffer_full}? no |
| {virtual}? no |
| {memory}? no |
| {exhausted}? no |
| {Undefined}? no |
| {command}? no |
| break gdbme.c:34 |
| Breakpoint 8 at 0x23d8: file gdbme.c, line 34. |
| (gdb) expect: does {break gdbme.c:34\r\nBreakpoint 8 at 0x23d8: |
| file gdbme.c, line 34.\r\n(gdb) } (spawn_id 6) match pattern |
| {Breakpoint.*at.* file gdbme.c, line 34.*\(gdb\) $}? yes |
| expect: set expect_out(0,start) {18} |
| expect: set expect_out(0,end) {71} |
| expect: set expect_out(0,string) {Breakpoint 8 at 0x23d8: file |
| gdbme.c, line 34.\r\n(gdb) } |
| epect: set expect_out(spawn_id) {6} |
| expect: set expect_out(buffer) {break gdbme.c:34\r\nBreakpoint 8 |
| at 0x23d8: file gdbme.c, line 34.\r\n(gdb) } |
| PASS: 70 0 breakpoint line number in file |
| </PRE |
| ></TD |
| ></TR |
| ></TABLE |
| ></DIV |
| ><P |
| >This example exhibits three properties of |
| <SPAN |
| CLASS="PRODUCTNAME" |
| >Expect</SPAN |
| > and |
| <SPAN |
| CLASS="PRODUCTNAME" |
| >DejaGnu</SPAN |
| > that might be surprising at |
| first glance:</P |
| ><P |
| ></P |
| ><UL |
| ><LI |
| STYLE="list-style-type: disc" |
| ><P |
| >Empty output for the first attempted match. The |
| first set of attempted matches shown ran against the output |
| <I |
| CLASS="EMPHASIS" |
| >{}</I |
| > --- that is, no |
| output. <SPAN |
| CLASS="PRODUCTNAME" |
| >Expect</SPAN |
| > begins |
| attempting to match the patterns supplied immediately; often, |
| the first pass is against incomplete output (or completely |
| before all output, as in this case).</P |
| ></LI |
| ><LI |
| STYLE="list-style-type: disc" |
| ><P |
| >Interspersed tool output. The beginning of |
| the log entry for the second attempted match may be hard to |
| spot: this is because the prompt <I |
| CLASS="EMPHASIS" |
| >{(gdb) }</I |
| > |
| appears on the same line, just before the |
| <I |
| CLASS="EMPHASIS" |
| >expect:</I |
| > that marks the beginning of the |
| log entry.</P |
| ></LI |
| ><LI |
| STYLE="list-style-type: disc" |
| ><P |
| >Fail-safe patterns. Many of the patterns |
| tested are fail-safe patterns provided by |
| <SPAN |
| CLASS="PRODUCTNAME" |
| >GDB</SPAN |
| > testing utilities, to reduce |
| possible indeterminacy. It is useful to anticipate potential |
| variations caused by extreme system conditions |
| (<SPAN |
| CLASS="PRODUCTNAME" |
| >GDB</SPAN |
| > might issue the message |
| <I |
| CLASS="EMPHASIS" |
| >virtual memory exhausted</I |
| > in rare |
| circumstances), or by changes in the tested program |
| (<I |
| CLASS="EMPHASIS" |
| >Undefined command</I |
| > is the likeliest |
| outcome if the name of a tested command changes).</P |
| ><P |
| >The pattern <I |
| CLASS="EMPHASIS" |
| >{return}</I |
| > is a |
| particularly interesting fail-safe to notice; it checks for an |
| unexpected <B |
| CLASS="KEYCAP" |
| >RET</B |
| > prompt. This may happen, |
| for example, if the tested tool can filter output through a |
| pager.</P |
| ><P |
| >These fail-safe patterns (like the debugging log itself) |
| are primarily useful while developing test scripts. Use the |
| <B |
| CLASS="COMMAND" |
| >error</B |
| > procedure to make the actions for |
| fail-safe patterns produce messages starting with |
| <I |
| CLASS="EMPHASIS" |
| >ERROR</I |
| > on standard output, and in the |
| detailed log file.</P |
| ></LI |
| ></UL |
| ></DIV |
| ></DIV |
| ><DIV |
| CLASS="NAVFOOTER" |
| ><HR |
| ALIGN="LEFT" |
| WIDTH="100%"><TABLE |
| SUMMARY="Footer navigation table" |
| WIDTH="100%" |
| BORDER="0" |
| CELLPADDING="0" |
| CELLSPACING="0" |
| ><TR |
| ><TD |
| WIDTH="33%" |
| ALIGN="left" |
| VALIGN="top" |
| ><A |
| HREF="x428.html" |
| ACCESSKEY="P" |
| ><<< Previous</A |
| ></TD |
| ><TD |
| WIDTH="34%" |
| ALIGN="center" |
| VALIGN="top" |
| ><A |
| HREF="book1.html" |
| ACCESSKEY="H" |
| >Home</A |
| ></TD |
| ><TD |
| WIDTH="33%" |
| ALIGN="right" |
| VALIGN="top" |
| ><A |
| HREF="c848.html" |
| ACCESSKEY="N" |
| >Next >>></A |
| ></TD |
| ></TR |
| ><TR |
| ><TD |
| WIDTH="33%" |
| ALIGN="left" |
| VALIGN="top" |
| >Runtest</TD |
| ><TD |
| WIDTH="34%" |
| ALIGN="center" |
| VALIGN="top" |
| ><A |
| HREF="c401.html" |
| ACCESSKEY="U" |
| >Up</A |
| ></TD |
| ><TD |
| WIDTH="33%" |
| ALIGN="right" |
| VALIGN="top" |
| >Customizing DejaGnu</TD |
| ></TR |
| ></TABLE |
| ></DIV |
| ></BODY |
| ></HTML |
| > |
