[ORC] Refactor visit-members in StaticLibraryDefinitionGenerator. (#141546) This refactor was motivated by two bugs identified in out-of-tree builds: 1. Some implementations of the VisitMembersFunction type (often used to implement special loading semantics, e.g. -all_load or -ObjC) were assuming that buffers for archive members were null-terminated, which they are not in general. This was triggering occasional assertions. 2. Archives may include multiple members with the same file name, e.g. when constructed by appending files with the same name: % llvm-ar crs libfoo.a foo.o % llvm-ar q libfoo.a foo.o % llvm-ar t libfoo.a foo.o foo.o While confusing, these members may be safe to link (provided that they're individually valid and don't define duplicate symbols). In ORC however, the archive member name may be used to construct an ORC initializer symbol, which must also be unique. In that case the duplicate member names lead to a duplicate definition error even if the members define unrelated symbols. In addition to these bugs, StaticLibraryDefinitionGenerator had grown a collection of all member buffers (ObjectFilesMap), a BumpPtrAllocator that was redundantly storing synthesized archive member names (these are copied into the MemoryBuffers created for each Object, but were never freed in the allocator), and a set of COFF-specific import files. To fix the bugs above and simplify StaticLibraryDefinitionGenerator this patch makes the following changes: 1. StaticLibraryDefinitionGenerator::VisitMembersFunction is generalized to take a reference to the containing archive, and the index of the member within the archive. It now returns an Expected<bool> indicating whether the member visited should be treated as loadable, not loadable, or as invalidating the entire archive. 2. A static StaticLibraryDefinitionGenerator::createMemberBuffer method is added which creates MemoryBuffers with unique names of the form `<archive-name>[<index>](<member-name>)`. This defers construction of member names until they're loaded, allowing the BumpPtrAllocator (with its redundant name storage) to be removed. 3. The ObjectFilesMap (symbol name -> memory-buffer-ref) is replaced with a SymbolToMemberIndexMap (symbol name -> index) which should be smaller and faster to construct. 4. The 'loadability' result from VisitMemberFunctions is now taken into consideration when building the SymbolToMemberIndexMap so that members that have already been loaded / filtered out can be skipped, and do not take up any ongoing space. 5. The COFF ImportedDynamicLibraries member is moved out into the COFFImportFileScanner utility, which can be used as a VisitMemberFunction. This fixes the bugs described above; and should lower memory consumption slightly, especially for archives with many files and / or symbol where most files are eventually loaded.
Welcome to the LLVM project!
This repository contains the source code for LLVM, a toolkit for the construction of highly optimized compilers, optimizers, and run-time environments.
The LLVM project has multiple components. The core of the project is itself called “LLVM”. This contains all of the tools, libraries, and header files needed to process intermediate representations and convert them into object files. Tools include an assembler, disassembler, bitcode analyzer, and bitcode optimizer.
C-like languages use the Clang frontend. This component compiles C, C++, Objective-C, and Objective-C++ code into LLVM bitcode -- and from there into object files, using LLVM.
Other components include: the libc++ C++ standard library, the LLD linker, and more.
Consult the Getting Started with LLVM page for information on building and running LLVM.
For information on how to contribute to the LLVM project, please take a look at the Contributing to LLVM guide.
Join the LLVM Discourse forums, Discord chat, LLVM Office Hours or Regular sync-ups.
The LLVM project has adopted a code of conduct for participants to all modes of communication within the project.