[X86][regcall] Rework struct classification for non-Windows x86-64 targets (#187134)

Currently, when `X86_64ABIInfo::classifyRegCallStructTypeImpl`
classifies a struct argument or return value as direct, it leaves the
LLVM IR coerce type unspecified, implicitly relying on
`CodeGenTypes::ConvertType` to eventually construct a default IR type
based on the struct's layout. This conversion is neither stable nor
guaranteed to adhere to the ABI's classification rules.

Instead, rewrite `classifyRegCallStructTypeImpl` to construct an
explicit sequence of coerce types, using the existing field
classification to obtain a coerce type for each member of the struct.
Also, rename the function to `passRegCallStructTypeDirectly` and return
a boolean instead, so that now `classifyRegCallStructType` is the only
place that computes `ABIArgInfo`.

This rewrite also fixes several other issues with the `X86_64ABIInfo`
implementation of `__regcall`:

* Empty structs are now ignored instead of being misclassified as
direct.
* Arrays are now classified specially based on the element type, since
`X86_64ABIInfo::classifyArgumentType` ignores standalone array types.
* SSE registers used for return values are now correctly reused for
arguments, matching the 64-bit Windows behavior.

Since this is an ABI change, it has the potential to cause
incompatibilities with `__regcall` code compiled by earlier versions of
Clang. Specifically:

* Because SSE return registers can now be reused as argument registers,
functions will now pass more floating point arguments in SSE registers.
* `_Complex float` struct fields are now passed in one SSE register
instead of two.

Fixes #62999
Fixes #98635
6 files changed
tree: d095e93c0d09baeea160c47d885d501f994e96ff
  1. .ci/
  2. .github/
  3. bolt/
  4. clang/
  5. clang-tools-extra/
  6. cmake/
  7. compiler-rt/
  8. cross-project-tests/
  9. flang/
  10. flang-rt/
  11. libc/
  12. libclc/
  13. libcxx/
  14. libcxxabi/
  15. libsycl/
  16. libunwind/
  17. lld/
  18. lldb/
  19. llvm/
  20. llvm-libgcc/
  21. mlir/
  22. offload/
  23. openmp/
  24. orc-rt/
  25. polly/
  26. runtimes/
  27. third-party/
  28. utils/
  29. .clang-format
  30. .clang-format-ignore
  31. .clang-tidy
  32. .git-blame-ignore-revs
  33. .gitattributes
  34. .gitignore
  35. .mailmap
  36. CODE_OF_CONDUCT.md
  37. CONTRIBUTING.md
  38. LICENSE.TXT
  39. pyproject.toml
  40. README.md
  41. SECURITY.md
README.md

The LLVM Compiler Infrastructure

OpenSSF Scorecard OpenSSF Best Practices libc++

Welcome to the LLVM project!

This repository contains the source code for LLVM, a toolkit for the construction of highly optimized compilers, optimizers, and run-time environments.

The LLVM project has multiple components. The core of the project is itself called “LLVM”. This contains all of the tools, libraries, and header files needed to process intermediate representations and convert them into object files. Tools include an assembler, disassembler, bitcode analyzer, and bitcode optimizer.

C-like languages use the Clang frontend. This component compiles C, C++, Objective-C, and Objective-C++ code into LLVM bitcode -- and from there into object files, using LLVM.

Other components include: the libc++ C++ standard library, the LLD linker, and more.

Getting the Source Code and Building LLVM

Consult the Getting Started with LLVM page for information on building and running LLVM.

For information on how to contribute to the LLVM project, please take a look at the Contributing to LLVM guide.

Getting in touch

Join the LLVM Discourse forums, Discord chat, LLVM Office Hours or Regular sync-ups.

The LLVM project has adopted a code of conduct for participants to all modes of communication within the project.