<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
tt
{mso-style-priority:99;
font-family:"Courier New";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle20
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="RU" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Hi Doru,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Thank you for the detailed description of your changes.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">> Regarding your proposal, from your slides I understand that you perform a partial linking step as the first action for all object files and/or static libraries
given as input. So this clang invocation:</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">> clang++ -L. -labc test.cpp -o test</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">> would result in the same compilation steps as the current Clang version performs because the initial stage of partial linking would have no work to do (since there are no object
files present to be partially linked).</span><span lang="EN-US"><br>
<br>
</span><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">No, the intent is to change offload link step to always operate with fat objects and libs, so the compilation part of
the action graph for that command should produce temporary fat object which is then passed to the partial linking. I have not described that in slides, but I agree that it is probably not obvious and should have been mentioned.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">> Another question I have is regarding the "ld -r" box in your slides.</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">> How does ld -r work with "bundled" objects? Your diagram seems to imply that ld -r does the concatenation of all device images out of the box. Is this accurate?</span><span lang="EN-US"><br>
<br>
</span><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Actually the technique that is currently used by the clang-offload-bundler tool for creating bundled (or fat) objects
is very similar to what you are going to do for bundling NVPTX and host objects. The difference is in the initial “wrapper” that is created for the device code – you are using C++ structure, while clang-offload-bundler uses LLVM bitcode file. The bundler tool
creates a temporary LLVM IR which contains a global initialized array holding the device object, and this array is allocated in a section with predefined name (the name includes offload target triple). This “wrapper” bitcode file is then compiled for the host
and then partially linked against the host object.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">So technically fat/bundled object is just a host ELF object which has one or more additional ELF section containing device
object as data (one extra section per each offloading target). Linker concatenates sections with the same name while linking multiple objects files, so the result of partial linking will have the same named ELF sections holding device code concatenated from
all input (fat) objects.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Clang-offload-bundler would need to extract each device object individually while doing unbundling operation on the partially
linked object. A possible way to enable this would be creating one more section holding the device object size in addition to existing section with device object in fat/bundled object. That would allow clang bundler to get sizes of device objects that were
concatenated by partial linking.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Serguei<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><a name="_____replyseparator"></a><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Gheorghe-Teod Bercea [mailto:Gheorghe-Teod.Bercea@ibm.com]
<br>
<b>Sent:</b> Thursday, August 16, 2018 6:15 AM<br>
<b>To:</b> Dmitriev, Serguei N <serguei.n.dmitriev@intel.com><br>
<b>Cc:</b> 'cfe-dev@lists.llvm.org' <cfe-dev@lists.llvm.org>; Jonas Hahnfeld <hahnjo@hahnjo.de><br>
<b>Subject:</b> RE: [cfe-dev] [OpenMP] offload support for static libraries<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">Hi Serguei,</span><span lang="EN-US"><br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">Thanks a lot for the proposal.</span><span lang="EN-US"><br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">My proposal reworks a little bit the way the OpenMP-NVPTX toolchain creates device object files: the device specific part of the object is "wrapped" in an NVLINK-friendly C++
structure that is then compiled for the host. The result is a host object file with a device part which NVLINK can detect (D.o). The D.o object file is then partially linked against the host object file H.o and thus we obtain HD.o. This is required because
compilation is required to produce a single output object file (when doing "-c -o" for example). HD.o can now be passed to NVLINK directly or put in a static library and then passed to NVLINK. Either way, NVLINK will be able to detect the device part (due
to the special wrapping that we did previously) without the need to "unbundle" the object file (prior to passing it to NVLINK).</span><span lang="EN-US"><br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">The reason why the clang-offload-bundler is not involved in this is because we are using the standard object format for the object file that the OpenMP-NVPTX toolchain outputs
so there's no need for a custom format in this case. The partial linking step is required to put together the host and device object files and to ensure that only one object file is produced even if we actually invoked two toolchains (one for host and one
for the device).</span><span lang="EN-US"><br>
<br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">Regarding your proposal, from your slides I understand that you perform a partial linking step as the first action for all object files and/or static libraries given as input.
So this clang invocation:</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">clang++ -L. -labc test.cpp -o test</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">would result in the same compilation steps as the current Clang version performs because the initial stage of partial linking would have no work to do (since there are no object
files present to be partially linked).</span><span lang="EN-US"><br>
<br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">Another question I have is regarding the "ld -r" box in your slides.</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">How does ld -r work with "bundled" objects? Your diagram seems to imply that ld -r does the concatenation of all device images out of the box. Is this accurate?</span><span lang="EN-US"><br>
<br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">Thanks a lot,</span><span lang="EN-US"><br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif">--Doru</span><span lang="EN-US"><br>
<br>
</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
</span><span lang="EN-US"><br>
<br>
<br>
<br>
</span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">From: </span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif">"Dmitriev, Serguei N" <</span><a href="mailto:serguei.n.dmitriev@intel.com"><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif">serguei.n.dmitriev@intel.com</span></a><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif">></span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">To: </span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif">Jonas Hahnfeld <</span><a href="mailto:hahnjo@hahnjo.de"><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif">hahnjo@hahnjo.de</span></a><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif">></span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">Cc: </span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif">"'cfe-dev@lists.llvm.org'" <</span><a href="mailto:cfe-dev@lists.llvm.org"><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif">cfe-dev@lists.llvm.org</span></a><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif">>,
Doru Bercea <</span><a href="mailto:gheorghe-teod.bercea@ibm.com"><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif">gheorghe-teod.bercea@ibm.com</span></a><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif">></span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">Date: </span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif">08/15/2018 04:27 PM</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">Subject: </span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial",sans-serif">RE: [cfe-dev] [OpenMP] offload support for static libraries</span><span lang="EN-US"><o:p></o:p></span></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" noshade="" style="color:#A0A0A0" align="center">
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span lang="EN-US"><br>
<br>
<br>
</span><tt><span lang="EN-US" style="font-size:10.0pt">Hi Jonas,</span></tt><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""><br>
<br>
<tt>I guess this patch implements the proposal which Doru presented on the "OpenMP / HPC in Clang / LLVM Multi-company" meeting. As I remember he suggested to eliminate use of clang-offload-bundler tool when offload target is NTVPTX by replacing bundling operation
with partial linking of host and device objects, and then relying of the NVPTX linker to perform the unbundling operation at link phase. Based on Doru's explanations NVPTX linker "knows" how to extract device parts from such objects, so the explicit unbundling
operation in not required. Doru, please correct me if my understanding is not fully accurate. Doru's proposal definitely achieves the same goal for NVPTX offloading target (i.e. enables offload in static libraries), but it is NVPTX specific and cannot be extended
to other offloading targets (at least that is how it looked like when Doru described it).</tt><br>
<br>
<tt>I propose slightly different solution which I think should work for any generic OpenMP offload target (it was also discussed on the OpenMP multi-company meeting). In general case we have to use clang-offload-bundler because we cannot assume that device
object(s) can be bundled with the host object by performing partial linking of host and device objects. So bundling and unbundling operation will still be done by the clang-offload-bundler tool. The main part of my suggestion is adding partial linking of fat
objects (created by offload bundler tool) and static libraries (which are composed of fat objects) and only after that do the unbundling operation on the partially linked object (followed by the appropriate link actions for all offloading devices and then
for the host). This would guarantee that device parts of fat objects from static libraries will participate in the device link actions, and thus would enable offloading for static libraries.</tt><br>
<br>
<tt>Thanks,</tt><br>
<tt>Serguei</tt><br>
<br>
<tt>-----Original Message-----</tt><br>
<tt>From: Jonas Hahnfeld [</tt></span><a href="mailto:hahnjo@hahnjo.de"><tt><span lang="EN-US" style="font-size:10.0pt">mailto:hahnjo@hahnjo.de</span></tt></a><tt><span lang="EN-US" style="font-size:10.0pt">]
</span></tt><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""><br>
<tt>Sent: Tuesday, August 14, 2018 1:52 PM</tt><br>
<tt>To: Dmitriev, Serguei N <</tt></span><a href="mailto:serguei.n.dmitriev@intel.com"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">serguei.n.dmitriev@intel.com</span></a><tt><span lang="EN-US" style="font-size:10.0pt">></span></tt><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""><br>
<tt>Cc: 'cfe-dev@lists.llvm.org' <</tt></span><a href="mailto:cfe-dev@lists.llvm.org"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">cfe-dev@lists.llvm.org</span></a><tt><span lang="EN-US" style="font-size:10.0pt">>; Doru Bercea <</span></tt><a href="mailto:gheorghe-teod.bercea@ibm.com"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">gheorghe-teod.bercea@ibm.com</span></a><tt><span lang="EN-US" style="font-size:10.0pt">></span></tt><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""><br>
<tt>Subject: Re: [cfe-dev] [OpenMP] offload support for static libraries</tt><br>
<br>
<tt>This proposal has already been proposed for NVPTX in </tt></span><a href="https://reviews.llvm.org/D47394"><tt><span lang="EN-US" style="font-size:10.0pt">https://reviews.llvm.org/D47394</span></tt></a><tt><span lang="EN-US" style="font-size:10.0pt">, adding
Doru.</span></tt><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""><br>
<br>
<tt>Cheers,</tt><br>
<tt>Jonas</tt><br>
<br>
<tt>On 2018-08-14 18:43, Dmitriev, Serguei N via cfe-dev wrote:</tt><br>
<tt>> PROBLEM OVERVIEW</tt><br>
<tt>> </tt><br>
<tt>> OpenMP offload functionality is currently not supported in static </tt><br>
<tt>> libraries. Because of that an attempt to use offloading in static </tt><br>
<tt>> libraries ends up with a fallback execution of target regions on the </tt><br>
<tt>> host. This limitation clearly has significant impact on OpenMP offload </tt>
<br>
<tt>> usability.</tt><br>
<tt>> </tt><br>
<tt>> An output object file that is created by the compiler for offload </tt><br>
<tt>> compilation is a fat object. Such object files besides the code for </tt><br>
<tt>> the host architecture also contains code for the offloading targets </tt><br>
<tt>> which is stored as data in ELF sections with predefined names. Thus, a </tt>
<br>
<tt>> static library that is created from object files produced by offload </tt><br>
<tt>> compilation would be an archive of fat objects.</tt><br>
<tt>> </tt><br>
<tt>> Clang driver currently never passes fat objects directly to any </tt><br>
<tt>> toolchain. Instead it performs an unbundling operation for each fat </tt><br>
<tt>> object which extract host and device parts from the object. These </tt><br>
<tt>> parts are then independently processed by the corresponding target </tt><br>
<tt>> toolchains. However, current implementation does not assume that </tt><br>
<tt>> static archives may also be composed from fat objects. No unbundling </tt><br>
<tt>> is done for static archives (they are passed to linker as is) and thus </tt>
<br>
<tt>> device parts of objects from such archives get ignored.</tt><br>
<tt>> </tt><br>
<tt>> SUGGESTED SOLUTION</tt><br>
<tt>> </tt><br>
<tt>> It seems feasible to resolve this problem by changing the offload link </tt>
<br>
<tt>> process - adding an extra step to the link flow which will do a </tt><br>
<tt>> partial linking (ld -r) of fat objects and static libraries as shown </tt><br>
<tt>> on this diagram</tt><br>
<tt>> </tt><br>
<tt>> [Fat objects] \ / [Target1 link] \</tt><br>
<tt>> </tt><br>
<tt>> [Partial linking] - [Unbundling] - [TargetN link] - </tt><br>
<tt>> [Host link]</tt><br>
<tt>> </tt><br>
<tt>> [Static libs] / \--- Host part --/</tt><br>
<tt>> </tt><br>
<tt>> (You can also look at the .pdf file on this link </tt><br>
<tt>> </tt></span><a href="https://drive.google.com/file/d/1ZTNoB-Ghin1BTaiZ312FMSRS6rISDtlr/view"><tt><span lang="EN-US" style="font-size:10.0pt">https://drive.google.com/file/d/1ZTNoB-Ghin1BTaiZ312FMSRS6rISDtlr/view</span></tt></a><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""><br>
<tt>> ?usp=sharing [1] for illustrations for the suggested change)</tt><br>
<tt>> </tt><br>
<tt>> Linker will pull in all necessary dependencies from static libraries </tt><br>
<tt>> while performing partial linking, so the result of partial linking </tt><br>
<tt>> would be a fat object with concatenated device parts from input fat </tt><br>
<tt>> objects and required dependencies from static libraries. These </tt><br>
<tt>> concatenated device objects will be stored in the corresponding ELF </tt><br>
<tt>> sections of the partially linked object.</tt><br>
<tt>> </tt><br>
<tt>> Unbundling operation on the partially linked object will create one or </tt>
<br>
<tt>> more device objects for each offloading target, and these objects will </tt>
<br>
<tt>> be linked by corresponding target toolchains the same way as it is </tt><br>
<tt>> done now. Offload bundler tool would require enhancements to support </tt><br>
<tt>> unbundling of multiple concatenated device objects for each offloading </tt>
<br>
<tt>> target.</tt><br>
<tt>> </tt><br>
<tt>> Host link action can be changed to use host part of the partially </tt><br>
<tt>> linked object while linking the final image.</tt><br>
<tt>> </tt><br>
<tt>> Do you see any potential problems in the proposed change?</tt><br>
<tt>> </tt><br>
<tt>> Links:</tt><br>
<tt>> ------</tt><br>
<tt>> [1]</tt><br>
<tt>> </tt></span><a href="https://drive.google.com/file/d/1ZTNoB-Ghin1BTaiZ312FMSRS6rISDtlr/view"><tt><span lang="EN-US" style="font-size:10.0pt">https://drive.google.com/file/d/1ZTNoB-Ghin1BTaiZ312FMSRS6rISDtlr/view</span></tt></a><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""><br>
<tt>> ?usp=sharing _______________________________________________</tt><br>
<tt>> cfe-dev mailing list</tt><br>
<tt>> </tt></span><a href="mailto:cfe-dev@lists.llvm.org"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New"">cfe-dev@lists.llvm.org</span></a><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""><br>
<tt>> </tt></span><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev"><tt><span lang="EN-US" style="font-size:10.0pt">http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</span></tt></a><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New""><br>
<br>
</span><span lang="EN-US"><br>
<br>
<o:p></o:p></span></p>
</div>
</body>
</html>