[llvm-dev] Debug info for CUDA code
Alexey Bataev via llvm-dev
llvm-dev at lists.llvm.org
Wed Feb 27 10:09:54 PST 2019
Hi Char, it looks like they have some kind of buffer overflow bug in
their ptxas tool.
-------------
Best regards,
Alexey Bataev
27.02.2019 13:08, treinz пишет:
> Hi Alexey,
>
> I submitted the bug report to nvidia. While they are working on it,
> can you share some insight in what could potentially cause this? I
> just want to get a sense if such a bug require significant amount of
> work to fix, which can help me make some decision moving forward with
> my project.
>
> Thanks,
> Char
>
> At 2019-02-27 03:19:02, "Alexey Bataev" <a.bataev at outlook.com> wrote:
>
> Hi Char, yes, this line causes the crash. You don't need .cu files
> (I don't know what files are you talking about, clang does not
> generate .cu files). You need just this .s file to reproduce the
> error and nothing else.
>
> -------------
> Best regards,
> Alexey Bataev
>
> 26.02.2019 13:06, treinz пишет:
>> Hi Alexey,
>>
>> Just want to make sure I understand what you said because I'm not
>> familiar with the llvm pipeline, it's this line:
>>
>> /net/gs/vol3/software/modules-sw/cuda/10.0/Linux/RHEL6/x86_64/bin/ptxas"
>> -m64 -g --dont-merge-basicblocks --return-at-end -v --gpu-name
>> sm_75 --output-file
>> /tmp/60663577.1.login.q/testparticles-4fd988.o
>> /tmp/60663577.1.login.q/testparticles-1d20c4.s
>>
>> that segfaults, where the testparticles-1d20c4.s is generated by
>> the command:
>>
>> /home/aznb/.linuxbrew/Cellar/llvm/HEAD-48aefa8/bin/clang-9" -cc1
>> -triple nvptx64-nvidia-cuda -aux-triple x86_64-unknown-linux-gnu
>> -S -disable-free -disable-llvm-verifier -discard-value-names
>> -main-file-name testparticles.cpp -mrelocation-model static
>> -mthread-model posix -mdisable-fp-elim -no-integrated-as
>> -fuse-init-array -fcuda-is-device -mlink-builtin-bitcode
>> /net/gs/vol3/software/modules-sw/cuda/10.0/Linux/RHEL6/x86_64/nvvm/libdevice/libdevice.10.bc
>> -target-feature +ptx61 -target-cpu sm_75 -dwarf-column-info
>> -debug-info-kind=limited -dwarf-version=2 -debugger-tuning=gdb -v
>> -resource-dir
>> /home/aznb/.linuxbrew/Cellar/llvm/HEAD-48aefa8/lib/clang/9.0.0
>> -internal-isystem
>> /home/aznb/.linuxbrew/Cellar/llvm/HEAD-48aefa8/lib/clang/9.0.0/include/cuda_wrappers
>> -internal-isystem
>> /net/gs/vol3/software/modules-sw/cuda/10.0/Linux/RHEL6/x86_64/include
>> -include __clang_cuda_runtime_wrapper.h -isystem
>> /home/aznb/.linuxbrew/Cellar/boost/1.68.0_clang/include/boost-1_68
>> -I /home/aznb/mycodes/scbmc -I
>> /home/aznb/mycodes/scbmc/build_cuda_clang_debug/_deps/kokkos-src/core/src
>> -I
>> /home/aznb/mycodes/scbmc/build_cuda_clang_debug/_deps/kokkos-src/containers/src
>> -I
>> /home/aznb/mycodes/scbmc/build_cuda_clang_debug/_deps/kokkos-src/algorithms/src
>> -I
>> /home/aznb/mycodes/scbmc/build_cuda_clang_debug/_deps/kokkos-build
>> -I /home/aznb/mycodes/scbmc/../thrust -I
>> /net/gs/vol3/software/modules-sw/cuda/10.0/Linux/RHEL6/x86_64/include
>> -I
>> /home/aznb/mycodes/scbmc/build_cuda_clang_debug/_deps/kokkosutils-src
>> -I
>> /home/aznb/mycodes/scbmc/build_cuda_clang_debug/_deps/alias-src
>> -I
>> /home/aznb/mycodes/scbmc/build_cuda_clang_debug/_deps/ziggurat-src
>> -I
>> /home/aznb/mycodes/scbmc/build_cuda_clang_debug/_deps/distribution-src
>> -I
>> /home/aznb/mycodes/scbmc/build_cuda_clang_debug/_deps/generic-src
>> -internal-isystem
>> /home/aznb/.linuxbrew/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/../../../../include/c++/5.5.0
>> -internal-isystem
>> /home/aznb/.linuxbrew/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/../../../../include/c++/5.5.0/x86_64-unknown-linux-gnu
>> -internal-isystem
>> /home/aznb/.linuxbrew/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/../../../../include/c++/5.5.0/backward
>> -internal-isystem
>> /home/aznb/.linuxbrew/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/../../../../include/c++/5.5.0
>> -internal-isystem
>> /home/aznb/.linuxbrew/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/../../../../include/c++/5.5.0/x86_64-unknown-linux-gnu
>> -internal-isystem
>> /home/aznb/.linuxbrew/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/../../../../include/c++/5.5.0/backward
>> -internal-isystem /usr/local/include -internal-isystem
>> /home/aznb/.linuxbrew/Cellar/llvm/HEAD-48aefa8/lib/clang/9.0.0/include
>> -internal-externc-isystem /include -internal-externc-isystem
>> /usr/include -internal-isystem /usr/local/include
>> -internal-isystem
>> /home/aznb/.linuxbrew/Cellar/llvm/HEAD-48aefa8/lib/clang/9.0.0/include
>> -internal-externc-isystem /include -internal-externc-isystem
>> /usr/include -O0 -std=c++17 -fdeprecated-macro
>> -fno-dwarf-directory-asm -fdebug-compilation-dir
>> /home/aznb/mycodes/scbmc/build_cuda_clang_debug -ferror-limit 19
>> -fmessage-length 0 -fobjc-runtime=gcc -fcxx-exceptions
>> -fexceptions -fdiagnostics-show-option -o
>> /tmp/60663577.1.login.q/testparticles-1d20c4.s -x cuda
>> /home/aznb/mycodes/scbmc/testparticles.cpp
>>
>> Both of these commands were shown in the error log. How do I
>> replace my original cpp files with the two *.cu files
>> automatically generated by llvm in the above command to reproduce
>> the bug?
>>
>> Thanks,
>> Char
>>
>>
>> At 2019-02-26 09:22:35, "Alexey Bataev" <a.bataev at outlook.com> wrote:
>>
>> Hi Char, unfortunately I cannot help you with these crashes,
>> because they caused not by clang /LLVM, but the NVidia's
>> PTX-to-SASS translator ptxas. It is ptxas who crashes and I
>> can't do anything about this. You need to report the bug to
>> NVidia, but you need to manually run the ptxas tool with the
>> generated .s file. Sometimes it helps to compile the code
>> sequentially, not using the parallel builds. Plus, those
>> crashes may disappear from time to time.
>>
>> Best regards,
>> Alexey Bataev
>>
>> 25 февр. 2019 г., в 19:59, treinz <treinz at 163.com
>> <mailto:treinz at 163.com>> написал(а):
>>
>>> Hi Alexey,
>>>
>>> Thanks for the great work! The version I checked out works
>>> most of the time. But I do encounter crashes sometimes. I
>>> can't file a bug report on https://bugs.llvm.org/ because I
>>> don't have an account. I sent an email
>>> to bugs-admin at lists.llvm.org
>>> <mailto:bugs-admin at lists.llvm.org> for an account already
>>> but I haven't heard back. Meanwhile, can you take a look at
>>> the issue? I'm attaching the bug report files here -- the
>>> clang_error.log has the verbose output from clang and the
>>> other files are generated automatically by clang upon the
>>> crash. I'm using the above-mentioned commit
>>> 48aefa8877713616802d9c90b63f86fe50b9a101 from github mirror.
>>> The bug is reproducible on my machine with a nvidia RTX2080
>>> card (sm=Turing75).
>>>
>>> Thanks,
>>> Char
>>>
>>> At 2019-01-24 03:25:39, "Alexey Bataev"
>>> <a.bataev at outlook.com <mailto:a.bataev at outlook.com>> wrote:
>>>
>>> Yes, this one.
>>>
>>> -------------
>>> Best regards,
>>> Alexey Bataev
>>>
>>> 23.01.2019 14:17, treinz via llvm-dev пишет:
>>>> Hi Alexey,
>>>>
>>>> Thanks for working on this. I assume the last commit
>>>> you mentioned is:
>>>>
>>>> commit 48aefa8877713616802d9c90b63f86fe50b9a101 (HEAD
>>>> -> master, origin/master, origin/HEAD)
>>>> Author: Alexey Bataev <a.bataev at hotmail.com>
>>>> Date: Wed Jan 23 18:59:54 2019 +0000
>>>>
>>>> [DEBUGINFO, NVPTX] Enable support for the debug
>>>> info on NVPTX target.
>>>>
>>>> Enable full support for the debug info.
>>>>
>>>> Differential revision: https://reviews.llvm.org/D46189
>>>>
>>>> git-svn-id:
>>>> https://llvm.org/svn/llvm-project/llvm/trunk@351974
>>>> 91177308-0d34-0410-b5e6-96231b3b80d8
>>>>
>>>> I'll check this one out.
>>>>
>>>> Thanks,
>>>> Char
>>>>
>>>> At 2019-01-24 03:01:18, "Alexey Bataev"
>>>> <a.bataev at outlook.com> wrote:
>>>>
>>>> Hi Char, I found the problem, for some reason the
>>>> last patch was applied correctly. Just committed
>>>> the fixed version. Tried to compile axpy.cu,
>>>> everything works.
>>>>
>>>> -------------
>>>> Best regards,
>>>> Alexey Bataev
>>>>
>>>> 23.01.2019 13:37, treinz пишет:
>>>>> Hi Alexey,
>>>>>
>>>>> I tried the b7195a6 from the llvm github mirror,
>>>>> which does include your commit D46189
>>>>> <https://reviews.llvm.org/D46189> (see below). But
>>>>> it can't compile the example program
>>>>> on https://llvm.org/docs/CompileCudaWithLLVM.html#invoking-clang
>>>>> <https://llvm.org/docs/CompileCudaWithLLVM.html#invoking-clang.> --
>>>>> the error is:
>>>>>
>>>>> [aznb at asp testcuda]$ clang++ axpy.cu -g -o axpy
>>>>> --cuda-gpu-arch=sm_61 -lcudart_static -ldl -lrt
>>>>> -pthread -v
>>>>> clang version 9.0.0
>>>>> (https://git.llvm.org/git/clang.git
>>>>> 4566ed0d7c076ad061168f7c6cb5fa3a793614a8)
>>>>> (https://git.llvm.org/git/llvm.git
>>>>> b7195a6d8d9064c4fac6a7b91826de80df394f69)
>>>>> Target: x86_64-unknown-linux-gnu
>>>>> Thread model: posix
>>>>> InstalledDir: /home/aznb/.linuxbrew/bin
>>>>> Found candidate GCC installation:
>>>>> /home/aznb/.linuxbrew/bin/../lib/gcc/x86_64-unknown-linux-gnu/5.5.0
>>>>> Found candidate GCC installation:
>>>>> /usr/lib/gcc/x86_64-redhat-linux/4.4.4
>>>>> Found candidate GCC installation:
>>>>> /usr/lib/gcc/x86_64-redhat-linux/4.4.7
>>>>> Selected GCC installation:
>>>>> /home/aznb/.linuxbrew/bin/../lib/gcc/x86_64-unknown-linux-gnu/5.5.0
>>>>> Candidate multilib: .;@m64
>>>>> Selected multilib: .;@m64
>>>>> Found CUDA installation:
>>>>> /net/vol3/software/modules-sw/cuda/10.0/Linux/RHEL6/x86_64,
>>>>> version 10.0
>>>>> "/home/aznb/mycodes/testcuda -ferror-limit 19
>>>>> -fmessage-length 0 -pthread -fobjc-runtime=gcc
>>>>> -fcxx-exceptions -fexceptions
>>>>> -fdiagnostics-show-option -o
>>>>> /tmp/59224042.1.login.q/axpy-560f45.s -x cuda axpy.cu
>>>>> clang -cc1 version 9.0.0 based upon LLVM 9.0.0svn
>>>>> default target x86_64-unknown-linux-gnu
>>>>> ignoring nonexistent directory "/include"
>>>>> ignoring nonexistent directory "/include"
>>>>> ignoring duplicate directory
>>>>> "/home/aznb/.linuxbrew/bin/../lib/gcc/x86_64-unknown-linux-gnu/5.5.0/../../../../include/c++/5.5.0"
>>>>> ignoring duplicate directory
>>>>> "/home/aznb/.linuxbrew/bin/../lib/gcc/x86_64-unknown-linux-gnu/5.5.0/../../../../include/c++/5.5.0/x86_64-unknown-linux-gnu"
>>>>> ignoring duplicate directory
>>>>> "/home/aznb/.linuxbrew/bin/../lib/gcc/x86_64-unknown-linux-gnu/5.5.0/../../../../include/c++/5.5.0/backward"
>>>>> ignoring duplicate directory "/usr/local/include"
>>>>> ignoring duplicate directory
>>>>> "/home/aznb/.linuxbrew/Cellar/llvm/HEAD-b7195a6/lib/clang/9.0.0/include"
>>>>> ignoring duplicate directory "/usr/include"
>>>>> #include "..." search starts here:
>>>>> #include <...> search starts here:
>>>>> /home/aznb/.linuxbrew/Cellar/llvm/HEAD-b7195a6/lib/clang/9.0.0/include/cuda_wrappers
>>>>> /net/vol3/software/modules-sw/cuda/10.0/Linux/RHEL6/x86_64/include
>>>>> /home/aznb/.linuxbrew/bin/../lib/gcc/x86_64-unknown-linux-gnu/5.5.0/../../../../include/c++/5.5.0
>>>>> /home/aznb/.linuxbrew/bin/../lib/gcc/x86_64-unknown-linux-gnu/5.5.0/../../../../include/c++/5.5.0/x86_64-unknown-linux-gnu
>>>>> /home/aznb/.linuxbrew/bin/../lib/gcc/x86_64-unknown-linux-gnu/5.5.0/../../../../include/c++/5.5.0/backward
>>>>> /usr/local/include
>>>>> /home/aznb/.linuxbrew/Cellar/llvm/HEAD-b7195a6/lib/clang/9.0.0/include
>>>>> /usr/include
>>>>> End of search list.
>>>>> "/net/vol3/software/modules-sw/cuda/10.0/Linux/RHEL6/x86_64/bin/ptxas"
>>>>> -m64 -g --dont-merge-basicblocks --return-at-end
>>>>> -v --gpu-name sm_61 --output-file
>>>>> /tmp/59224042.1.login.q/axpy-1ebb06.o
>>>>> /tmp/59224042.1.login.q/axpy-560f45.s
>>>>> ptxas /tmp/59224042.1.login.q/axpy-560f45.s, line
>>>>> 290; fatal : Parsing error near '{': syntax error
>>>>> ptxas fatal : Ptx assembly aborted due to errors
>>>>> clang-9: error: ptxas command failed with exit
>>>>> code 255 (use -v to see invocation)
>>>>>
>>>>> Maybe it's related to the illegal assembly code
>>>>> mentioned in https://reviews.llvm.org/D46189 Any
>>>>> idea what's going on? I'm including other people
>>>>> to see if they have any
>>>>> suggestion. https://reviews.llvm.org/D46189 also
>>>>> mentioned that this is fixed but has the fixed
>>>>> been merged in the trunk yet?
>>>>>
>>>>> BTW, github mirror's b7195a6 does include your
>>>>> commit about the debug info on nvptx:
>>>>>
>>>>> commit d77902a49a06890b97e33515d346c6e4df6b756b
>>>>> Author: Alexey Bataev <a.bataev at hotmail.com>
>>>>> Date: Tue Jan 22 17:43:37 2019 +0000
>>>>>
>>>>> [DEBUGINFO, NVPTX] Enable support for the
>>>>> debug info on NVPTX target.
>>>>>
>>>>> Summary: Enable full support for the debug info.
>>>>>
>>>>> Reviewers: echristo
>>>>>
>>>>> Subscribers: jholewinski, aprantl,
>>>>> JDevlieghere, llvm-commits
>>>>>
>>>>> Differential Revision:
>>>>> https://reviews.llvm.org/D46189
>>>>>
>>>>> git-svn-id:
>>>>> https://llvm.org/svn/llvm-project/llvm/trunk@351846
>>>>> 91177308-0d34-0410-b5e6-96231b3b80d8
>>>>>
>>>>> Thanks,
>>>>> Char
>>>>>
>>>>> 在 2019-01-23 07:59:12，"Alexey Bataev"
>>>>> <a.bataev at outlook.com> 写道：
>>>>>
>>>>> Hi Char, I don't know how often it is updated,
>>>>> but most probably it is already updated.
>>>>>
>>>>> Best regards,
>>>>> Alexey Bataev
>>>>>
>>>>> 22 янв. 2019 г., в 17:46, treinz
>>>>> <treinz at 163.com <mailto:treinz at 163.com>>
>>>>> написал(а):
>>>>>
>>>>>>
>>>>>> Hi Alexey,
>>>>>>
>>>>>> Thank you for letting me know. Can I assume
>>>>>> the the github mirror is bumped to 9.0 and
>>>>>> should contain the commit you mentioned? Just
>>>>>> want to make sure I'm looking at the right repo.
>>>>>>
>>>>>> Thanks,
>>>>>> Char
>>>>>>
>>>>>>
>>>>>> 在 2019-01-23 01:46:44，"Alexey Bataev"
>>>>>> <a.bataev at outlook.com
>>>>>> <mailto:a.bataev at outlook.com>> 写道：
>>>>>>
>>>>>> Hi, debug info for the NVPTX is
>>>>>> committed. It is not a part of 8.0 RC1,
>>>>>> unfortunately, only in 9.0 trunk.
>>>>>>
>>>>>> -------------
>>>>>> Best regards,
>>>>>> Alexey Bataev
>>>>>>
>>>>>> 14.12.2018 17:08, Alexey Bataev пишет:
>>>>>>> Actually, we need to commit just 2 more
>>>>>>> patches to enable basic debug info
>>>>>>> support for NVPTX. So, I hope, it is
>>>>>>> going to be the part of the next 8.0
>>>>>>> release. It won't be backported to 7.0
>>>>>>> definitely.
>>>>>>> Sure, I will let you know when the
>>>>>>> patches are committed to the trunk, no
>>>>>>> problems.
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Alexey Bataev
>>>>>>>
>>>>>>> 14 дек. 2018 г., в 16:55, treinz
>>>>>>> <treinz at 163.com <mailto:treinz at 163.com>>
>>>>>>> написал(а):
>>>>>>>
>>>>>>>> Are you planning to release this as
>>>>>>>> soon as it's ready or you want to make
>>>>>>>> it into a major release? Is it possible
>>>>>>>> to let me know (maybe by replying to
>>>>>>>> this thread) once the code is ready? I
>>>>>>>> know sometimes it takes a while to get
>>>>>>>> things in the major release. I greatly
>>>>>>>> appreciate your work on this!
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Char
>>>>>>>>
>>>>>>>> 在 2018-12-15 05:19:50，"Alexey Bataev"
>>>>>>>> <a.bataev at outlook.com
>>>>>>>> <mailto:a.bataev at outlook.com>> 写道：
>>>>>>>>
>>>>>>>> Hi Char, sorry, you need to wait a
>>>>>>>> little bit. It is almost ready, but
>>>>>>>> not yet. For now, use nvcc if you
>>>>>>>> need debug info for Cuda.
>>>>>>>>
>>>>>>>> -------------
>>>>>>>> Best regards,
>>>>>>>> Alexey Bataev
>>>>>>>>
>>>>>>>> 14.12.2018 14:33, treinz пишет:
>>>>>>>>> Hi Alex, Eric and Valentin,
>>>>>>>>>
>>>>>>>>> Thanks for the information. I
>>>>>>>>> don't mean to push this but I'm in
>>>>>>>>> desperate need of debugging some
>>>>>>>>> cuda code. I'm not familiar with
>>>>>>>>> the llvm internal but it sounds
>>>>>>>>> like there's at least line info
>>>>>>>>> now, right? If so, can you point
>>>>>>>>> me to a branch of llvm that can
>>>>>>>>> help tracing the bug down to
>>>>>>>>> certain line of code. I believe my
>>>>>>>>> bug is simply a write/read out of
>>>>>>>>> array boundary.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Char
>>>>>>>>>
>>>>>>>>> 在 2018-12-05 07:15:28，"Alexey
>>>>>>>>> Bataev" <a.bataev at hotmail.com> 写道：
>>>>>>>>>
>>>>>>>>> Hi, yes, these patches are
>>>>>>>>> required to have tne basic
>>>>>>>>> debug info support. At least 2
>>>>>>>>> more patches are required to
>>>>>>>>> be able to get values of the
>>>>>>>>> variables without additional
>>>>>>>>> manual data type casting in
>>>>>>>>> cuda-gdb.
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>> Alexey Bataev
>>>>>>>>>
>>>>>>>>> 4 дек. 2018 г., в 18:06, Eric
>>>>>>>>> Christopher
>>>>>>>>> <echristo at gmail.com
>>>>>>>>> <mailto:echristo at gmail.com>>
>>>>>>>>> написал(а):
>>>>>>>>>
>>>>>>>>>> Adding Alexey here who has
>>>>>>>>>> been driving this effort in llvm.
>>>>>>>>>>
>>>>>>>>>> There are about 5 patches
>>>>>>>>>> waiting on my review:
>>>>>>>>>>
>>>>>>>>>> -:
>>>>>>>>>> https://reviews.llvm.org/D54320
>>>>>>>>>> -:
>>>>>>>>>> https://reviews.llvm.org/D46189
>>>>>>>>>> -:
>>>>>>>>>> https://reviews.llvm.org/D51554
>>>>>>>>>> -:
>>>>>>>>>> https://reviews.llvm.org/D46061
>>>>>>>>>> -:
>>>>>>>>>> https://reviews.llvm.org/D45784
>>>>>>>>>>
>>>>>>>>>> After which I think we're good.
>>>>>>>>>>
>>>>>>>>>> -eric
>>>>>>>>>>
>>>>>>>>>> On Mon, Dec 3, 2018 at 6:29
>>>>>>>>>> PM Valentin Churavy via
>>>>>>>>>> llvm-dev
>>>>>>>>>> <llvm-dev at lists.llvm.org
>>>>>>>>>> <mailto:llvm-dev at lists.llvm.org>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> AFAIK there is work under
>>>>>>>>>> way from IBM (couple of
>>>>>>>>>> PRs open), but yes Clang
>>>>>>>>>> 7.0 doesn't produce the
>>>>>>>>>> necessary information.
>>>>>>>>>>
>>>>>>>>>> -V
>>>>>>>>>>
>>>>>>>>>> On Fri, Nov 30, 2018,
>>>>>>>>>> 14:58 treinz via llvm-dev
>>>>>>>>>> <llvm-dev at lists.llvm.org
>>>>>>>>>> <mailto:llvm-dev at lists.llvm.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>> I found this
>>>>>>>>>> http://lists.llvm.org/pipermail/llvm-dev/2017-November/118871.html
>>>>>>>>>> when googling about
>>>>>>>>>> compiling CUDA code
>>>>>>>>>> using llvm. Is it
>>>>>>>>>> still the case that
>>>>>>>>>> one can't step into
>>>>>>>>>> CUDA kernel code
>>>>>>>>>> compiled by llvm in
>>>>>>>>>> cuda-gdb? I'm using
>>>>>>>>>> clang 7.0.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Char
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> LLVM Developers
>>>>>>>>>> mailing list
>>>>>>>>>> llvm-dev at lists.llvm.org
>>>>>>>>>> <mailto:llvm-dev at lists.llvm.org>
>>>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> LLVM Developers mailing list
>>>>>>>>>> llvm-dev at lists.llvm.org
>>>>>>>>>> <mailto:llvm-dev at lists.llvm.org>
>>>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>> <bugreport.tar.bz2>
>>
>>
>>
>>
>>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190227/8033c72c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190227/8033c72c/attachment.sig>
More information about the llvm-dev
mailing list