[llvm] [Support][Windows][DTLTO] Expand short 8.3 form system temp dirs (PR #175196)

[Ben Dunbobbin via llvm-commits]

bd1976bris wrote:

> I don't understand the direct connection between path normalization vs. distributed build system. Are you trying to turn that into a unified location on all machines? But what if user doesn't have write permission to the directory? Or user is expecting to write to a different drive/partition?

A distribution system needs to be able to schedule compilation jobs onto remote nodes. To execute on a remote node, some sort of sandboxing/path-virtualization is required so that when a remote compilation loads from a specific path, it gets the same data that it would see on the machine from which the compilation job originated. On Linux (I'm unsure about the Mac side of things - Sorry), this is often provided by a distinct component, e.g. https://github.com/containers/bubblewrap, or a technology layer such as NFS. However, on Windows, both of these functions are typically combined under one system. See for example, Incredibuild™'s description of "Virtualized Distributed Processing™" in their [documentation](https://www.incredibuild.com/wp-content/uploads/2020/10/Incredibuild-Technology-Overview-v-2.pdf), or Unreal™'s UBA [documentation](https://dev.epicgames.com/documentation/en-us/unreal-engine/horde-unreal-build-accelerator-and-remote-compilation-tutorial-for-unreal-engine), which talks about "Lightweight Virtualization".

Windows supports legacy short 8.3 form names in the filesystem (see: https://en.wikipedia.org/wiki/8.3_filename). You might have seen paths like: `C:\Users\hansc\DOWNLO~1\SCRAPI~1.MP4`. These style of paths are particularly problematic: they tend to defeat sandboxing and path-virtualization mechanisms, and they introduce other usability issues, such as breaking reliable tab completion in the Windows command prompt.

> Only allow distributed thinLTO as standalone command from clang (no compilation job allowed) so user has to make sure the inputs/outputs to the task is accessible on all environments they care about.
Just overwrite TMPDIR in the build system to somewhere acceptable.
Have some alternative clang driver flag to ask clang to plan intermediate outputs to a specific directory.

Thanks. It's certainly the case that the user can adjust their build system or environment to prevent this style of paths from causing an issue along the lines that you suggest. However, one of the important features of DTLTO is that the user doesn't need to make such adjustments - the distribution should just work. Hence the requirement for normalization.

Having said all that I don't necessarily have to remove short 8.3 form paths by normalization `TMP/TEMP/TMPDIR`. I could implement the normalization in DTLTO specific code. Part of my motivation for this PR is that I think it would be nice for all users of LLVM based software on Windows if short 8.3 form paths weren't present. As stated earlier though - I can accept that no one has felt the need to make this change in the years of Windows support. It therefore seems likely that DTLTO is the only substantial justification for this so adjusting the `TMP/TEMP/TMPDIR` behaviour for all use-cases on Windows (this PR) might not be justifiable.

https://github.com/llvm/llvm-project/pull/175196