| <!--#set var="title" value="Developer's Guide" --> |
| <!--#include virtual="/incl/header.incl" --> |
| <h1 class="title">HLVM Developer's Guide</h1> |
| <div class="warning">CAUTION: This document is a work in progress.</div> |
| <ol> |
| <li><a href="#intro">Introduction</a></li> |
| <li><a href="#build">Build System</a> |
| <ol> |
| <li><a href="#scons">About SCONS</a></li> |
| <li><a href="#config">Configuring</a></li> |
| <li><a href="#rules">Build Rules</a></li> |
| <li><a href="#sanity">Sane Build Environment</a></li> |
| </ol> |
| </li> |
| <li><a href="#style">Coding Style</a></li> |
| </ol> |
| |
| <div class="author"> |
| <p>Author: <a href="mailto:rspencer@reidspencer.com">Reid Spencer</a>.</p> |
| </div> |
| |
| <h2><a name="intro">Introduction</a></h2> |
| <div class="text"> |
| <p>This document is a manual for developers who wish to contribute to the HLVM |
| project. It covers topics such as coding style, the build system and project |
| rules. All the <a href="http://llvm.org/docs/DeveloperPolicy.html">policies |
| of the LLVM project</a> apply here as well. This document just contains |
| things specific to HLVM.</p> |
| </div> |
| |
| <h2><a name="build">Build System</a></h2> |
| <div class="text"> |
| <p>This section describes the HLVM build system.</p> |
| </div> |
| |
| <h3><a name="scons">About SCONS</a></h3> |
| <div class="text"> |
| <p>The HLVM project uses the <a href="http://www.scons.org/">scons</a> |
| software construction tool to orchestrate its builds. This gives us superior |
| dependency checking and a much more flexible tool for developing the build |
| system. We started with "make" but were convinced of SCons superiority after |
| trying it for a week. If you're not familiar with it, please read up on it |
| and get a general understanding. You don't need detailed understanding because |
| all you will most likely need to do is follow the instructions in this |
| section.</p> |
| </div> |
| |
| <h3><a name="config">Configuring</a></h3> |
| <div class="text"> |
| <h4>Configuration On Every Build</h4> |
| <p>Unlike some other build systems (e.g. autoconf/automake), the |
| configuration and construction phases of building are not separated with |
| scons. The configuration parameters are checked every time you build. While |
| you might think that will slow things down, it doesn't. The configuration |
| information is cached and proceeds quite quickly once your configuration is |
| stable. If something changes in your environment, only those pieces affected |
| will get re-configured. This saves a lot of time and hassle by telling you of |
| any configuration errors on the very next build.</p> |
| <h4>The Options Cache</h4> |
| <p>Configuration parameters are specified on the scons command line. The |
| values of these parameters are stored in the <i>options cache</i> which is |
| simply a file named .options_cache in the root source directory. If the file |
| doesn't exist, it will be created the first time you run scons. On subsequent |
| runs, the values are loaded from this cache and used unless new values are |
| given on the command line. In this way, you only need to specify your options |
| once and then they "stick" from that point forward.</p> |
| <h4>Configuration Options</h4> |
| <p>HLVM has a variety of options to assist with configuring HLVM for your |
| environment and build choices. Each option has a default that will work in |
| many environments but might need adjusting for your environment. The default |
| assumes various software packages are in <tt>/usr/local</tt> and that a fully |
| debuggable version of HLVM should be built. The table below provides the |
| full list of options.</p> |
| <table> |
| <tr><th>Option</th><th>Description</th><th>Default</th></tr> |
| <tr> |
| <td>assertions</td> |
| <td>Include assertions in the code. This can be enabled or disabled separate |
| from the setting for debug.</td> |
| <td>1</td></tr> |
| <tr> |
| <td>debug</td> |
| <td>Build with debug options turned on. This includes #ifdef/#endif code as |
| well as instructing the compiler to include debug information (-g)</td> |
| <td>1</td></tr> |
| <tr> |
| <td>inline</td> |
| <td>Cause inline code to be inline. This causes inline functions to really |
| be inline. Its off by default for better debuggability.</td><td>0</td> |
| </tr> |
| <tr> |
| <td>optimize</td> |
| <td>Build object files with optimization. This turns on compiler |
| optimization (-O3) to make the code efficient.</td><td>0</td></tr> |
| <tr> |
| <td>profile</td> |
| <td>Generate profiling aware code. This causes the compiler to construct the |
| code so that it can be profiled (-pg on Unix).</td><td>0</td></tr> |
| <tr> |
| <td>small</td> |
| <td>Generate smaller code rather than faster. Instructs the code to favor |
| smaller object files and executables than faster ones. Might be useful for |
| memory constrained platforms.</td> |
| <td>0</td></tr> |
| <tr> |
| <td>strip</td> |
| <td>Strip executables of their symbols. This can significantly reduce the |
| size of the executable. Useful for release preparation when trying to |
| minimize the size of a tarball or RPM package.</td> |
| <td>0</td></tr> |
| <tr> |
| <td>prefix</td> |
| <td>Specify where to install HLVM. This is the root directory of where HLVM |
| will be installed when you run <tt>scons install</tt></td> |
| <td>/usr/local</td></tr> |
| <tr> |
| <td>confpath</td> |
| <td>Specify additional configuration dirs to search. This is a unix style |
| path (colon separate) of directories to include in the search for software |
| packages. These directories will be searched first. If you put your |
| software in a non-standard place like /proj (Reid) or /more (Saem) then |
| you'll want to use this option.</td> |
| <td>{empty}</td></tr> |
| <tr> |
| <td>with_llvm</td> |
| <td>Specify where LLVM is installed.</td> |
| <td>/usr/local</td></tr> |
| <tr> |
| <td>with_apr</td> |
| <td>Specify where apr is installed.</td> |
| <td>/usr/local/apr</td></tr> |
| <tr> |
| <td>with_apru</td> |
| <td>Specify where apr-utils is installed.</td> |
| <td>/usr/local/apr</td></tr> |
| <tr> |
| <td>with_libxml2</td> |
| <td>Specify where LibXml2 is installed.</td> |
| <td>/usr/local</td></tr> |
| <tr> |
| <td>with_gperf</td> |
| <td>Specify where the gperf program is installed.</td> |
| <td>/usr/local/bin/gperf</td></tr> |
| <tr> |
| <td>with_llc</td> |
| <td>Specify where the LLVM compiler is installed.</td> |
| <td>/usr/local/bin/llc</td></tr> |
| <tr> |
| <td>with_llvmdis</td> |
| <td>Specify where the LLVM disassembler is installed.</td> |
| <td>/usr/local/bin/llvm-dis</td></tr> |
| <tr> |
| <td>with_llvmas</td> |
| <td>Specify where the LLVM assembler is installed.</td> |
| <td>/usr/local/bin/llvm-as</td></tr> |
| <tr> |
| <td>with_llvmgcc</td> |
| <td>Specify where the LLVM C compiler is installed.</td> |
| <td>/usr/local/bin/llvm-gcc</td></tr> |
| <tr> |
| <td>with_llvmgxx</td> |
| <td>Specify where the LLVM C++ compiler is installed.</td> |
| <td>/usr/local/bin/llvm-g++</td></tr> |
| <tr> |
| <td>with_llvmar</td> |
| <td>Specify where the LLVM bytecode archiver is installed.</td> |
| <td>/usr/local/bin/llvm-g++</td></tr> |
| <tr> |
| <td>with_llvm2cpp</td> |
| <td>Specify where the LLVM llvm2cpp program is installed.</td> |
| <td>/usr/local/bin/llvm2cpp</td></tr> |
| <tr> |
| <td>with_runtest</td> |
| <td>Specify where DejaGnu runtest program is installed.</td> |
| <td>/usr/local/bin/runtest</td></tr> |
| <tr> |
| <td>with_doxygen</td> |
| <td>Specify where the doxygen program is installed.</td> |
| <td>/usr/local/bin/doxygen</td></tr> |
| <tr> |
| <td>with_xsltproc</td> |
| <td>Specify where the XSLT processor is installed.</td> |
| <td>/usr/local/bin/xsltproc</td> |
| </tr> |
| </table> |
| <h4>Configuration Prompts</h4> |
| <p>When you first run <tt>scons</tt> against HLVM, if you did not specify the |
| <tt>confpath</tt> option or the configuration code cannot otherwise find a |
| package it needs, you will be prompted to enter the applicable path names |
| manually. You only need to do this once as the values you enter will be |
| remembered in the <tt>.options_cache</tt> file. These prompts will repeat |
| until you enter a path in which the package is properly recognized. If you |
| wish to abort this, just enter <tt>exit</tt> or <tt>quit</tt> at any prompt. |
| </p> |
| </div> |
| |
| <h3><a name="rules">Build Rules</a></h3> |
| <div class="text"> |
| <p>This subsection describes the various build rules and extensions to SCons |
| that HLVM uses.</p> |
| <h4>The <tt>build</tt> python module</h4> |
| <p>For convenience, and to unclutter the SConscript files, the top level |
| <tt>build</tt> directory contains a python module that provides the build |
| facilities for HLVM. This module manipulates an SCons environment to set the |
| variables and define the builder rules for HLVM. The public interface to this |
| library is in the <tt>hlvm.py</tt> module. No other module should be imported |
| into the SConscript files.</p> |
| <h4>HLVM Builder Rules</h4> |
| <p>HLVM requires some specialized build rules beyond what SCons provides. |
| While general construction of static libraries, shared libraries, and |
| executables use the standard SCons builders, several more are defined in the |
| <tt>build</tt> python module. The table below describes the HLVM specific |
| builders.</p> |
| <table> |
| <tr><th>Builder</th><th>Source</th><th>Target</th><th>Description</th></tr> |
| <tr> |
| <td>RNGTokenizer</td> |
| <td>Relax/NG Schema (.rng)</td> |
| <td>*Tokenizer.h, *Tokenizer.cpp, *TokenHash.cpp</td> |
| <td>This builder scans a Relax/NG Schema for element, attribute, and |
| enumerated value names. It then uses these tokens to build a perfect |
| hash function (via <tt>gperf</tt>) for quickly recognizing the tokens |
| and converting it to an enumeration value. This is provided to make |
| recognizing the element and attributes names of an XML document faster. |
| </td> |
| </tr> |
| <tr> |
| <td>RNGQuoteSource</td> |
| <td>Relax/NG Schema (.rng)</td> |
| <td>C++ Strings For Inclusion (.inc)</td> |
| <td>This builder converts a Relax/NG schema into a set of strings that can |
| be #included into a C++ program as a literal string. It strips out |
| comments, annotations, and converts special characters into the C++ |
| equivalent. The purpose here is to allow a single source for the schema |
| to be incorporated directly into the software so that no file I/O is |
| necessaroy to access the schema.</td> |
| </tr> |
| <tr> |
| <td>Bytecode</td> |
| <td>C++ Source (.cpp)</td> |
| <td>LLVM Bytecode (.bc)</td> |
| <td>This builder compiles a C++ source file and produces an equivalent |
| LLVM bytecode file. This allows us to incorporate C++ source into the |
| Runtime as linkable bytecode.</td> |
| </tr> |
| <tr> |
| <td>BytecodeArchive</td> |
| <td>LLVM Bytecode (.bc)</td> |
| <td>LLVM Bytecode Archive (.bca)</td> |
| <td>This builder collects a group of LLVM Bytecode files together and |
| creates a bytecode archive from them using <tt>llvm-ar</tt></td> |
| </tr> |
| <tr> |
| <td>Cpp2LLVMCpp</td> |
| <td>C++ Source (.cxx)</td> |
| <td>C++ Source (.cpp)</td> |
| <td>This builder uses the <tt>llvm2cpp</tt> to turn source C++ into |
| equivalent C++ calls against the LLVM Intermediate Representation (IR) |
| to construct the same program or program fragment. The construction |
| environment variable LLVM2CPPFLAGS can be set to control what options |
| are passed to <tt>llvm2cpp</tt>. See the <tt>llvm2cpp</tt> manual page |
| for details. Examples are in the <tt>hlvm/CodeGen/SConscript</tt> |
| file.</td> |
| </tr> |
| <tr> |
| <td>Check</td> |
| <td>n/a</td> |
| <td>n/a</td> |
| <td>This builder is a little special. It is used to run the DejaGnu test |
| suite. It is aliased on the command line as the target "check" so that |
| the <tt>scons check</tt> command will invoke this build.</td> |
| </tr> |
| <tr> |
| <td>Doxygen</td> |
| <td>n/a</td> |
| <td>doxygen.tar.gz</td> |
| <td>This builder runs the doxygen documentation generator on the source |
| code and builds a gzipped tar file containing doxygen's output. This |
| allows the HLVM API documentation, generated from source code comments, |
| to be packged into a single file.</td> |
| </tr> |
| </table> |
| </div> |
| |
| <h3><a name="sanity">A Sane Build Environment</a></h3> |
| <div class="text"> |
| <p>Building HLVM is no small feat. It depends on a lot of software that is |
| quite version dependent. To bring a little sanity to the process, here is a |
| step-by-step procedure we know to work.</p> |
| <h4>Build Separation</h4> |
| <p>In building HLVM, you'll be installing compilers and library that may |
| already exist on your system. You don't want to overwrite your system versions |
| of these things or it will wreak havoc on your system. So, we suggest that you |
| start with a fresh directory. In the discussion that follows, we'll call it |
| <tt>/proj</tt> (that's what Reid uses). But, it could be anything you want, |
| as long as its new. <tt>~/hlvm</tt> would work just as well.</p> |
| <p<em class="warning">IMPORTANT: </em>Choosing the disk location for these |
| builds should not be taken lightly. You will need upwards of 40GB of storage |
| on a 32-bit machine to build all this software. Don't assume your home |
| directory has enough space!</p> |
| <p>Once you've found a suitable location for HLVM, create the following |
| directory structure:</p><pre> |
| cd /proj |
| mkdir gcc libxml2 apr apru gperf scons install</pre> |
| <p>In the following sections you will build each of these packages and install |
| them into <tt>/proj/install</tt> which will keep it separate from anything |
| else in your system.</p> |
| <h4>Build GCC 3.4.6</h4> |
| <p>First, start with obtaining GCC 3.4.6. This will be the compiler that you |
| use for all the remaining compilations. Use the following commands to obtain, |
| build and install GCC 3.4.6:</p><pre> |
| cd /proj/gcc |
| mkdir build |
| svn co svn://gcc.gnu.org/svn/gcc/tags/gcc_3_4_6_release src |
| cd ../build |
| ../src/configure --prefix=/proj/install |
| make |
| make install</pre> |
| <h4>Set Environment</h4> |
| <p>Now that you have gcc installed in a separate location, you will need to |
| change your environment to ensure that that version of gcc is the one used in |
| subsequent builds. Details vary from platform to platform, but on Linux, the |
| following should work:</p><pre> |
| export PATH=/proj/install/bin:$PATH |
| export LD_LIBRARY_PATH=/proj/install/lib:$LD_LIBRARY_PATH</pre> |
| <p>The essential point is to change your environment so that programs and |
| libraries installed into /proj/install will be found first. You should do this |
| in any shell environment in which you'll be building HLVM related software. |
| </p> |
| <h4>Build gperf</h4> |
| <p>This package is used for generating perfect hash functions. It is used by |
| HLVM for fast recognition of XML element and attribute names. Its easy and |
| fast to build:</p><pre> |
| cd /proj/gperf |
| wget http://mirrors.kernel.org/gnu/gperf/gperf-2.7.2.tar.gz |
| tar zxf gperf-2.7.2.tar.gz |
| mkdir build |
| cd build |
| ../gperf-2.7.2/configure --prefix=/proj/install |
| make |
| make install</pre> |
| <p>You can probably use any version after 2.7. We know it works with 2.7.2 and |
| 3.0.1.</p> |
| <h4>Build libxml2</h4> |
| <p>This package provides all XML services for HLVM. It is part of GNome and |
| many other packages and quite stable. It should build quickly and easily for |
| you. Use these commands:</p><pre> |
| cd /proj/libxml2 |
| wget ftp://xmlsoft.org/libxml2/libxml2-2.6.24.tar.gz |
| tar zxf libxml2-2.6.24.tar.gz |
| mkdir build |
| cd build |
| ../libxml2-2.6.24/configure --prefix=/proj/install |
| make |
| make install</pre> |
| <h4>Build apr</h4> |
| <p>The Apache Portable Runtime is a portability layer used within the Apache |
| HTTP Server. Although it is still undergoing active development, stable |
| releases are available. HLVM uses APR for portability in the runtime. Build |
| APR with the following commands:</p> |
| <pre> |
| cd /proj/apr |
| wget http://mirror.olnevhost.net/pub/apache/apr/apr-1.2.7.tar.gz |
| tar zxf apr-1.2.7.tar.gz |
| mkdir build |
| cd apr-1.2.7 |
| ./buildconf |
| cd ../build |
| ../apr-1.2.7/configure --prefix=/proj/install --enable-debug \ |
| --enable-threads --enable-other-child |
| make |
| make install</pre> |
| <h4>Build apr-util</h4> |
| <p>The apr-util package is some additional utilities that go with APR. Build |
| apr-util with the following commands:</p> |
| <pre> |
| cd /proj/apru |
| wget http://mirror.olnevhost.net/pub/apache/apr/apr-util-1.2.7.tar.gz |
| tar zxf apr-1.2.7.tar.gz |
| mkdir build |
| cd apr-util-1.2.7 |
| ./buildconf |
| cd ../build |
| ../apr-util-1.2.7/configure --prefix=/proj/install --enable-debug \ |
| --enable-threads --enable-other-child |
| make |
| make install</pre> |
| <h4>Build LLVM</h4> |
| <p>For now, you must build LLVM from the CVS repository. Although LLVM is |
| actively being developed, it is generally stable and this is safe. If you get |
| tempted to use a release tarball, it will fail. HLVM depends on post-1.7 |
| features of LLVM. When you build LLVM, use the "tools-only" target. This will |
| avoid building the "runtime" portion of LLVM which was necessary for an older |
| version of llvm-gcc (v3). Since we'll be using llvm-gcc4, this is unnecessary |
| and will eliminate some chicken-and-egg type problems.</p> |
| <pre> |
| cd /proj/llvm |
| cvs -d :pserver:anon@llvm-cvs.cs.uiuc.edu:/var/cvs/llvm login |
| <return> |
| cvs -z3 -d :pserver:anon@llvm-cvs.cs.uiuc.edu:/var/cvs/llvm co llvm |
| mkdir build |
| cd build |
| ../llvm/configure --prefix=/proj/install |
| make tools-only |
| make install |
| make ENABLE_OPTIMIZED=1 OPTIMIZE_OPTION=-O2 tools-only</pre> |
| <h4>Build llvm-gcc4</h4> |
| <p>Apple provides a Subversion repository for llvm-gcc4. You need this latest |
| version because HLVM depends on some of the features. You might find this one |
| a bit tricky. See the README.LLVM file in the top source directory for |
| additional help and late breaking news.</p> |
| <pre> |
| cd /proj/llvm-gcc4 |
| svn co svn://anonsvn.opensource.apple.com/svn/llvm cfe |
| mkdir build install |
| cd build |
| ../cfe/configure --prefix=/proj/llvm-gcc4/cfe/install \ |
| --enable-llvm=/proj/llvm/build --enable-languages=c,c++ --disable-threads \ |
| --program-prefix=llvm- |
| make |
| make install</pre> |
| <p>Note that the <tt>--disable-threads</tt> option is a temporary workaround |
| until LLVM's PR822 (supporting weak-external linkage) is implemented. This |
| should be fixed by September, 2006 at which time --disable-threads won't be |
| needed any more.</p> |
| <p><em class="warning">IMPORTANT: </em>You <em>MUST</em> install llvm-gcc4 |
| into its own installation area. Do <em>NOT</em> be tempted to install it into |
| <tt>/proj/install</tt>. If you do, it will overwrite your gcc 3.4.6 |
| installation and you will have corrupted your build environment. |
| <i>You've been warned.</i></p> |
| <h4>Install SCons</h4> |
| <p>As noted earlier, HLVM uses the SCons tool for its builds. You need version |
| 0.96.92. Install it like this:</p> |
| <pre> |
| cd /proj/scons |
| wget http://internap.dl.sourceforge.net/sourceforge/scons/scons-0.96.92.tar.gz |
| tar zxf scons-0.96.92.tar.gz |
| python config.py install</pre> |
| <p>Alternatively, you could use one of the SCons packages, such as:</p> |
| <pre> |
| cd /proj/scons |
| wget http://internap.dl.sourceforge.net/sourceforge/scons/scons-0.96.92-1.noarch.rpm |
| rpm --install scons-0.96.92-1.noarch.rpm</pre> |
| |
| <h4>Build HLVM</h4> |
| <p>Note in the following that if you've actually used <tt>/proj</tt> as your |
| build area then you don't need to provide any arguments to <tt>make</tt> as |
| these paths are the default. Also note that the arguments are only needed the |
| first time you build HLVM. Subsequently, these options will be remembered. See |
| the description of SCons above.</p> |
| <pre> |
| cd /proj/hlvm |
| svn co svn://hlvm.org/hlvm hlvm |
| cd hlvm |
| make debug MYMODE=Debug MYPREFIX=/proj/install MYPATH=/proj/install \ |
| MYLLVMGCC=/proj/llvm/cfe/install/bin</pre> |
| </div> |
| |
| <h2><a name="style">Coding Style</a></h2> |
| <div class="text"> |
| <p>Contributions to HLVM must meet the following Coding Style |
| requirements:</p> |
| <ul> |
| <li>Each file must start with the comment section common to other files in |
| HLVM. This comment section starts with a line describing the file, followed |
| by the HLVM title line, and a statement about the author and the |
| copyright holder (UIUC).</li> |
| <li>Source code that is to be part of an HLVM library or executable must be |
| written in C++. We know this isn't the greatest language in the world, but |
| it is the most compatible for meeting HLVM's integration and platform |
| support requirements. Its also necessary for integration with LLVM.</li> |
| <li>Utility programs should be written in python, especially if they are |
| build related.</li> |
| <li>Files must not exceed 80 columns in width</li> |
| <li>Indentation is 2 spaces (no tabs).</li> |
| <li>Doxygen documentation must be provided in header files, at the very |
| least.</li> |
| </ul> |
| </div> |
| |
| <!--#include virtual="/incl/footer.incl" --> |