[offload][OpenMP] Add atomic cross-team reductions (#209298)

Regular cross-team reductions have two phases: the intra-team reduction
and the inter-team reduction. Atomic cross-team reductions replace the
second phase with a atomic instruction which is used by the main thread
of each team to directly fold the result of the intra-team reduction
into the final result. Since this requires a combination of "data type"
and "combine operation" for which an atomic instruction is available,
only some (but very common) reductions can be transformed to atomic
reductions. In cases where multiple reductions are performed on the same
construct, the atomic path is only taken if all reductions can be
transformed. Otherwise, we fall back to the regular cross-team reduction
using a buffer with per-team slots. This is not strictly necessary, but
hybrid reductions would induce more complexity with questionable
benefit.

Selecting an atomic path might not be the best option for every
situation, which is why it is not enabled by default. Instead, it can be
enabled via `-fopenmp-target-atomic-reduction`. Note that enabling the
atomic path will not *force* atomic reductions. They will only be
applied if possible, as described above.

The performance (measured with https://github.com/ro-i/xteam-test @
c71339705091500f731e2a39f247d2660bacbdce, array size 177,777,777) is up
to +15% faster (aka, more throughput) for supported reductions on a
gfx942, with no noticeable regressions.
Example:
- sum reduction, type double: +10.22% faster
- sum reduction, type uint: +15.57% faster
- sum reduction, type ulong: +13.31% faster

On a gfx90a, there is little to negative benefit:
- sum reduction, type double: -4.32% faster (aka, slower)
- sum reduction, type uint: +3.08% faster
- sum reduction, type ulong: +1.68% faster

Claude assisted with this patch.

GitOrigin-RevId: 29a956e40a1f617887ecfeb02ae844102c4a5a48
2 files changed