<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1776557634;
mso-list-type:hybrid;
mso-list-template-ids:-1888848006 1306051156 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:\F0D8;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;
mso-fareast-font-family:Calibri;
mso-bidi-font-family:"Times New Roman";}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Sean Silva wrote:<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-family:Wingdings"><span style="mso-list:Ignore">Ø<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]>Roughly what I'm saying is instead of "check(..., expected)" instead do "print(...)". Then compare what is printed with the output produced by the program when compiled with the reference toolchain. This approach seems more useful…<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:.25in">Good point. In fact the tests do have a verbose mode where they print one-line-per-check, even when they pass, giving details including the line number in the test. That can be saved as the ‘golden master’ to compare
against future compilers, or possibly even to modify the test to pass on your compiler. This does not work in every case, but we plan to improve that.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-family:Wingdings"><span style="mso-list:Ignore">Ø<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]>Suppose that to extend the test suite to MS ABI…<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">I wouldn’t be that ambitious. The checking code relies on the basic concepts like IA64 vtbls and VTTs. As they exist now, most tests will probably fail to link with MSVC, or crash badly. It will be significant
amount of work to extend this to MS ABI.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">David Majnemer wrote:<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-size:12.0pt;font-family:Wingdings"><span style="mso-list:Ignore">Ø<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:12.0pt;font-family:"Times New Roman","serif"">I do hope that the intent is not to ensure that clang has bug-for-bug compatibility with earlier versions. We've fixed many bugs over time which would necessitate
ABI breaks from earlier versions of clang.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="font-size:12.0pt;font-family:"Times New Roman","serif"">We do not presume to impose any such restriction on future changes, even though many compiler vendors maintain such bug-to-bug compatibility.
If clang undergoes an ABI breaking change, the tests will have to change. We have tackled similar changes already internally.<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Nico Weber wrote:<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-family:Wingdings"><span style="mso-list:Ignore">Ø<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]>1.) How much of your 400 test files is from the manual abi method, how much from your test case generator?<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">About 20% are hand written<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-family:Wingdings"><span style="mso-list:Ignore">Ø<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]>2.) Test runtime probably scales fairly well with the number of cores, right? You say it takes one hour to run on one core, so an 8-core system should take less than 10 minutes?<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">Yes. It scales very well. Memory or disk contention may make it less than linear, but still very well.<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-family:Wingdings"><span style="mso-list:Ignore">Ø<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]>3.) How many bugs did you find with this so far?<o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in">I believe there were, but I can’t give a number since I was not directly involved in reporting them. Perhaps Paul Robinson or Katya can answer that.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Sunil Srivastava<o:p></o:p></p>
<p class="MsoNormal">SN Systems / Sony Computer Entertainment<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>