<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:p="urn:schemas-microsoft-com:office:powerpoint" xmlns:a="urn:schemas-microsoft-com:office:access" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" xmlns:b="urn:schemas-microsoft-com:office:publisher" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:c="urn:schemas-microsoft-com:office:component:spreadsheet" xmlns:odc="urn:schemas-microsoft-com:office:odc" xmlns:oa="urn:schemas-microsoft-com:office:activation" xmlns:html="http://www.w3.org/TR/REC-html40" xmlns:q="http://schemas.xmlsoap.org/soap/envelope/" xmlns:rtc="http://microsoft.com/officenet/conferencing" xmlns:D="DAV:" xmlns:Repl="http://schemas.microsoft.com/repl/" xmlns:mt="http://schemas.microsoft.com/sharepoint/soap/meetings/" xmlns:x2="http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ppda="http://www.passport.com/NameSpace.xsd" xmlns:ois="http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir="http://schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" xmlns:udc="http://schemas.microsoft.com/data/udc" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sub="http://schemas.microsoft.com/sharepoint/soap/2002/1/alerts/" xmlns:ec="http://www.w3.org/2001/04/xmlenc#" xmlns:sp="http://schemas.microsoft.com/sharepoint/" xmlns:sps="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:udcs="http://schemas.microsoft.com/data/udc/soap" xmlns:udcxf="http://schemas.microsoft.com/data/udc/xmlfile" xmlns:udcp2p="http://schemas.microsoft.com/data/udc/parttopart" xmlns:wf="http://schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:dsss="http://schemas.microsoft.com/office/2006/digsig-setup" xmlns:dssi="http://schemas.microsoft.com/office/2006/digsig" xmlns:mdssi="http://schemas.openxmlformats.org/package/2006/digital-signature" xmlns:mver="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns:mrels="http://schemas.openxmlformats.org/package/2006/relationships" xmlns:spwp="http://microsoft.com/sharepoint/webpartpages" xmlns:ex12t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:ex12m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:pptsl="http://schemas.microsoft.com/sharepoint/soap/SlideLibrary/" xmlns:spsl="http://microsoft.com/webservices/SharePointPortalServer/PublishedLinksService" xmlns:Z="urn:schemas-microsoft-com:" xmlns:tax="http://schemas.microsoft.com/sharepoint/taxonomy/soap/" xmlns:tns="http://schemas.microsoft.com/sharepoint/soap/recordsrepository/" xmlns:spsup="http://microsoft.com/webservices/SharePointPortalServer/UserProfileService" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:st="" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
@font-face
{font-family:"MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
{font-family:"Comic Sans MS";
panose-1:3 15 7 2 3 3 2 2 2 4;}
@font-face
{font-family:"\@MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
@font-face
{font-family:"Trebuchet MS";
panose-1:2 11 6 3 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:37052384;
mso-list-template-ids:-962793166;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1
{mso-list-id:384377877;
mso-list-template-ids:41039352;}
@list l1:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l1:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l2
{mso-list-id:433137821;
mso-list-template-ids:-1385394748;}
@list l2:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.25in;
mso-level-number-position:left;
margin-left:.25in;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:.75in;
mso-level-number-position:left;
margin-left:.75in;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l2:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.25in;
mso-level-number-position:left;
margin-left:1.25in;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l2:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.75in;
mso-level-number-position:left;
margin-left:1.75in;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l2:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.25in;
mso-level-number-position:left;
margin-left:2.25in;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l2:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.75in;
mso-level-number-position:left;
margin-left:2.75in;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l2:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.25in;
mso-level-number-position:left;
margin-left:3.25in;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l2:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.75in;
mso-level-number-position:left;
margin-left:3.75in;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l2:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.25in;
mso-level-number-position:left;
margin-left:4.25in;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l3
{mso-list-id:570778280;
mso-list-template-ids:354473590;}
@list l3:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l3:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l3:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l3:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l3:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l3:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l3:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l3:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l4
{mso-list-id:881135066;
mso-list-type:hybrid;
mso-list-template-ids:-714419476 -485601718 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l4:level1
{mso-level-text:"%1\.\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l4:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l4:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l4:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l4:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l4:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l4:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l4:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l4:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l5
{mso-list-id:1393457428;
mso-list-template-ids:-1915696824;}
@list l5:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l5:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l5:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l5:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l5:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l5:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l5:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l5:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l5:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l6
{mso-list-id:1792552507;
mso-list-template-ids:2086582614;}
@list l6:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l6:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l6:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l6:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l6:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l6:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l6:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l6:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l6:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l7
{mso-list-id:1890259573;
mso-list-type:hybrid;
mso-list-template-ids:137934714 1978429072 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l7:level1
{mso-level-text:"%1\.\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:12.0pt;
font-family:"Times New Roman","serif";
color:windowtext;
mso-ansi-font-style:normal;}
@list l7:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l7:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l7:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l7:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l7:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l7:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l7:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l7:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi Ted,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I’ll attempt to answer your questions:<o:p></o:p></span></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-indent:-.25in;mso-list:l7 level1 lfo8">
<![if !supportLists]><span style="mso-list:Ignore">1.)<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>What's the "<b><span style="color:#0B5394">official</span></b>" evaluation formula on branch coverage? Sorry I can't find any compelling documents to describe the accurate algorithm on this issue. I just thought gcov/lcov won't consider <i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">how
many times the decision evaluated to False.<o:p></o:p></span></i></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in;text-indent:-.25in;mso-list:l7 level2 lfo8">
<![if !supportLists]><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><span style="mso-list:Ignore">a.<span style="font:7.0pt "Times New Roman"">
</span></span></span></i><![endif]>[Alan]: I share your concern here. My reference is the ISO26262 standard for functional safety, which defines branch coverage as “<i>percentage of branches of the control flow of a computer program executed during a test.
Note 1: 100% branch coverage implies 100% statement coverage., Note 2: An if-statement always has two branches – condition true and condition false – independent of the existence of an else-clause</i>”<i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p></o:p></span></i></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.5in;text-indent:-1.5in;mso-text-indent-alt:-9.0pt;mso-list:l7 level3 lfo8">
<![if !supportLists]><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><span style="mso-list:Ignore"><span style="font:7.0pt "Times New Roman"">
</span>i.<span style="font:7.0pt "Times New Roman""> </span></span></span></i><![endif]>Basically, condition
<b>true</b> and condition <b>false</b> for every decision in a program. In contrast, MC/DC applies to
<i>single condition outcomes</i> that affect a decision. <i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p></o:p></span></i></p>
<p class="MsoListParagraph" style="margin-left:1.0in;text-indent:-.25in;mso-list:l7 level2 lfo8">
<![if !supportLists]><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><span style="mso-list:Ignore">b.<span style="font:7.0pt "Times New Roman"">
</span></span></span></i><![endif]>[Alan] Your GCOV output is producing branch information for each condition of the decision, and each of those branches corresponds to the true and false evaluation of each condition. The problem with GCC/GCOV is that, because
of the way it instruments, it’s not obvious how branches correspond to the source code.
<i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p></o:p></span></i></p>
<p class="MsoListParagraph"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></i></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l7 level1 lfo8"><![if !supportLists]><span style="mso-list:Ignore">2.)<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Are there any existing post-processing tools based on clang/llvm source based coverage<i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p></o:p></span></i></p>
<p class="MsoListParagraph" style="margin-left:1.0in;text-indent:-.25in;mso-list:l7 level2 lfo8">
<![if !supportLists]><span style="mso-list:Ignore">a.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>[Alan] I’m not aware of them, but others may know<o:p></o:p></p>
<p class="MsoListParagraph" style="margin-left:1.0in"><o:p> </o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l7 level1 lfo8"><![if !supportLists]><span style="mso-list:Ignore">3.)<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>How to disable STL (or other) noise in gcov branch coverage?<o:p></o:p></p>
<p class="MsoListParagraph" style="margin-left:1.0in;text-indent:-.25in;mso-list:l7 level2 lfo8">
<![if !supportLists]><span style="mso-list:Ignore">a.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>[Alan] I think this is a perennial problem – gcov reports hidden branches that are created, but the data isn’t necessarily useful from a source-code coverage perspective.
<o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">-Alan<o:p></o:p></span></p>
<p class="MsoNormal"><a name="_MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></a></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> LoveZhi Xie [mailto:crazyforzhi1987@gmail.com]
<br>
<b>Sent:</b> Monday, May 4, 2020 3:24 AM<br>
<b>To:</b> Phipps, Alan<br>
<b>Cc:</b> llvm-dev@lists.llvm.org; Vedant Kumar<br>
<b>Subject:</b> Re: [EXTERNAL] [llvm-dev] How to get branch coverage by using 'source-based code coverage'<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Hi, Alan<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks for making it clear. But I was more confused now :(<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I tested on a simple program and used both gcov and lcov to get branch coverage.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">The code and build commands as below:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><b><i><u>Example simple.cc</u></i></b><o:p></o:p></p>
</div>
<blockquote style="margin-left:30.0pt;margin-right:0in">
<div>
<p class="MsoNormal"><span style="font-family:"Trebuchet MS","sans-serif";color:#0B5394">#include <string></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Trebuchet MS","sans-serif";color:#0B5394">// If not comment this line, the branch coverage won't reach to 100%</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Trebuchet MS","sans-serif";color:#0B5394">// #include <iostream></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Trebuchet MS","sans-serif";color:#0B5394">int main(int argc, const char* argv[]) {</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Trebuchet MS","sans-serif";color:#0B5394"> std::string str = "foo";</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Trebuchet MS","sans-serif";color:#0B5394"> str = argv[1];</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Trebuchet MS","sans-serif";color:#0B5394"> if (str == "foo" || str == "bar") {</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Trebuchet MS","sans-serif";color:#0B5394"> int a = 5;</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Trebuchet MS","sans-serif";color:#0B5394"> a = 7;</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Trebuchet MS","sans-serif";color:#0B5394"> } else {</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Trebuchet MS","sans-serif";color:#0B5394"> int b = 6;</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Trebuchet MS","sans-serif";color:#0B5394"> b = 4;</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Trebuchet MS","sans-serif";color:#0B5394"> }</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Trebuchet MS","sans-serif";color:#0B5394"> return 0;</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Trebuchet MS","sans-serif";color:#0B5394">}</span><o:p></o:p></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><b><i><u><span style="font-family:"Trebuchet MS","sans-serif";color:black">Coverage build commands</span></u></i></b><o:p></o:p></p>
</div>
<blockquote style="margin-left:30.0pt;margin-right:0in">
<div>
<p class="MsoNormal"><span style="font-family:"Comic Sans MS";color:#0B5394">#!/bin/bash</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Comic Sans MS";color:#0B5394">g++ -o simple -fno-exceptions -fno-inline -fprofile-arcs -ftest-coverage simple.cc</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Comic Sans MS";color:#0B5394">./simple "foo"</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Comic Sans MS";color:#0B5394">./simple "bar"</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Comic Sans MS";color:#0B5394">./simple "hello1"</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Comic Sans MS";color:#0B5394">./simple "hello2"</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Comic Sans MS";color:#0B5394">./simple "hello3"</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Comic Sans MS";color:#0B5394">./simple "hello4"</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Comic Sans MS";color:#0B5394">./simple "hello5"</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Comic Sans MS";color:#0B5394">./simple "hello6"</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Comic Sans MS";color:#0B5394">./simple "hello7"</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Comic Sans MS";color:#0B5394">./simple "hello8"</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Comic Sans MS";color:#0B5394">lcov --rc lcov_branch_coverage=1 --capture --directory . --output-file
<a href="http://simple.lcov.info">simple.lcov.info</a></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Comic Sans MS";color:#0B5394">genhtml
<a href="http://simple.lcov.info">simple.lcov.info</a> --function-coverage --branch-coverage --output-directory lcov_out/</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Comic Sans MS";color:#0B5394">gcov -b -c simple.gcno</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</blockquote>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Both gcov and lcov will give a 100% branch coverage report:<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><img border="0" width="237" height="87" id="_x0000_i1025" src="cid:image002.png@01D62303.1255DF60" alt="cov1.png"><span style="font-family:"MS Gothic""> </span><img border="0" width="414" height="67" id="_x0000_i1026" src="cid:image004.png@01D62303.1255DF60" alt="cov2.png"><o:p></o:p></p>
</div>
</div>
<blockquote style="margin-left:30.0pt;margin-right:0in">
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</blockquote>
<div>
<p class="MsoNormal">My questions:<o:p></o:p></p>
</div>
<div>
<ul type="disc">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l1 level1 lfo1">
What's the "<b><span style="color:#0B5394">official</span></b>" evaluation formula on branch coverage?<o:p></o:p></li></ul>
<div>
<p class="MsoNormal"> Sorry I can't find any compelling documents to describe the accurate algorithm on this issue.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> I just thought gcov/lcov won't consider <i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">how many times the decision evaluated to False.</span></i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> P</span></i>lease see the gcov file on my example: simple.cc. There are 6 branches in total which are all hit<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> So the result would be 6/6 * 100% = 100%<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></i><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <img border="0" width="418" height="235" id="_x0000_i1027" src="cid:image006.jpg@01D62303.1255DF60" alt="cov3.png"><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<ul type="disc">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l3 level1 lfo2">
Are there any existing post-processing tools based on clang/llvm source based coverage?<o:p></o:p></li></ul>
</div>
</div>
<div>
<p class="MsoNormal"> I just saw a <a href="https://github.com/zero9178/C-Cpp-Coverage-for-CLion">
clion plugin</a> at github which may generate branch coverage by using region coverage data. But this depends on clion<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> which is not easy to implement automation test.<o:p></o:p></p>
</div>
<div>
<ul type="disc">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l6 level1 lfo3">
How to disable STL (or other) noise in gcov branch coverage?<o:p></o:p></li></ul>
<div>
<p class="MsoNormal"> Sorry, this is not clang/llvm question. But I am still looking forward to some suggestions. For example, if I add iostream header back to<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"> simple.cc, then we won't get a 100% branch coverage. This may because iostream will introduce some global destruction branch.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <img border="0" width="472" height="94" id="_x0000_i1028" src="cid:image008.jpg@01D62303.1255DF60" alt="cov4.png"><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> Or if it's impossible, can we think clang region coverage should be a best practice
<i>in C++ testing world</i>??<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"MS Gothic""> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"MS Gothic""> </span>AFAIK, chromium is using region coverage now.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"MS Gothic""> </span><img border="0" width="472" height="145" id="_x0000_i1029" src="cid:image010.jpg@01D62303.1255DF60" alt="cov5.png"><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">FYI.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Ted Xie<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Phipps, Alan <<a href="mailto:a-phipps@ti.com">a-phipps@ti.com</a>>
<span style="font-family:"MS Gothic"">于</span>2020<span style="font-family:"MS Gothic"">年</span>5<span style="font-family:"MS Gothic"">月</span>4<span style="font-family:"MS Gothic"">日周一</span>
<span style="font-family:"MS Gothic"">上午</span>12:54<span style="font-family:"MS Gothic"">写道：</span><o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">+ Vedant Kumar</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi Ted!</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I would not say that clang region coverage is branch coverage(‘aka decision coverage’). I would
say that it may be possible to extrapolate branch decision coverage based on region coverage. For example, consider the following:</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">[region0]</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">if (decision) {</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="font-size:11.0pt;font-family:Consolas;color:#1F497D"> [region1]</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in">
<span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">}</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Region coverage will tell you that whether [region1] was executed, and this will be enough to tell
you that the branch decision evaluated to True at least once. However, this will not tell you how many times the decision evaluated to False, which is also important when considering branch coverage. In this case, you can calculate this value by subtracting
the number of times [region0] was executed from the number of times [region1] was executed.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">What I am implementing for branch condition coverage will do this for you but will provide more granular
True/False counts for individual conditions that comprise all Boolean expression (which also generate branches), and will tally all of these cases as branches that are either covered or not covered.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I hope that helps.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">As for the future goals, I do plan on addressing what’s required for MC/DC, but that is work that
I have not yet planned.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thanks!</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">-Alan Phipps</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a name="m_7189132425558485252__MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> LoveZhi Xie [mailto:<a href="mailto:crazyforzhi1987@gmail.com" target="_blank">crazyforzhi1987@gmail.com</a>]
<br>
<b>Sent:</b> Sunday, May 3, 2020 11:12 AM<br>
<b>To:</b> Phipps, Alan<br>
<b>Cc:</b> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<b>Subject:</b> Re: [EXTERNAL] [llvm-dev] How to get branch coverage by using 'source-based code coverage'</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi, Alan<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Really very excited to receive your email and sorry to be slow replying, it has been exceptionally busy over the last few days ;(<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Your explanation made the problem clear to me. So gcov branch coverage should be called condition coverage and clang region coverage<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">is branch coverage in fact(also known as
<b><span style="color:#274E13">decision/C1</span></b><span style="color:black">)</span>, right?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">And llvm/clang will support all the following coverage criteria in future<o:p></o:p></p>
</div>
<div>
<ul type="disc">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo4">
Line coverage<o:p></o:p></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo4">
Function coverage<o:p></o:p></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo4">
Branch coverage/region coverage<o:p></o:p></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo4">
Condition coverage<o:p></o:p></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo4">
Modified Condition/Decision Coverage<o:p></o:p></li></ul>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">If there is any misunderstanding here, please point it out.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks,<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Ted Xie<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Phipps, Alan <<a href="mailto:a-phipps@ti.com" target="_blank">a-phipps@ti.com</a>>
<span style="font-family:"MS Gothic"">于</span>2020<span style="font-family:"MS Gothic"">年</span>4<span style="font-family:"MS Gothic"">月</span>28<span style="font-family:"MS Gothic"">日周二</span>
<span style="font-family:"MS Gothic"">下午</span>10:14<span style="font-family:"MS Gothic"">写道：</span><o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi Ted,</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Presently, branch coverage is not supported in clang source-based code coverage, but I am working
on implementing branch condition coverage right now. It will track True/False branches for each leaf-level condition, including within Boolean expressions comprised of logical operators (“&&”, “||”). Technically, this condition-based coverage is more granular
than how “branch coverage” is often defined (tracking on that control flow decisions allow for all regions to be hit), and is closer to the granularity that GCOV provides, but it’s anchored to the source code, won’t count “hidden branches”, and isn’t negatively
impacted by optimization (although the reverse isn’t true: coverage instrumentation may impact optimization).</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">My work is a step closer to getting us toward Modified Condition/Decision Coverage (MC/DC). I hope
to upstream my work within the next few months, so unfortunately it won’t help you in the short-term. Region coverage may be enough for what you need if you can ascertain based on the coverage that control-flow decisions have evaluated to both true and false,
but I’ll let others comment here with suggestions.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">-Alan Phipps</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a name="m_7189132425558485252_m_1180875836774759"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> llvm-dev [mailto:<a href="mailto:llvm-dev-bounces@lists.llvm.org" target="_blank">llvm-dev-bounces@lists.llvm.org</a>]
<b>On Behalf Of </b>LoveZhi Xie via llvm-dev<br>
<b>Sent:</b> Sunday, April 26, 2020 11:36 AM<br>
<b>To:</b> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<b>Subject:</b> [EXTERNAL] [llvm-dev] How to get branch coverage by using 'source-based code coverage'</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi, llvm/clang experts<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I need to get the branch coverage for some testing code. But i found gcov can't give a expected coverage which may<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">count some 'hidden branch' in (See <a href="https://stackoverflow.com/questions/42003783/lcov-gcov-branch-coverage-with-c-producing-branches-all-over-the-place" target="_blank">stackoverflow
answer</a>). Instead, I turn to use clang and the 'source-based code coverage' feature<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">may be a right choice. But i can't find anything to describe branch coverage explicitly on the
<a href="https://clang.llvm.org/docs/SourceBasedCodeCoverage.html" target="_blank">
official site</a>. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">So how to use clang to get branch coverage by utilizing 'source-based...' feature? And is
<b><i>region coverage </i></b>equivalent to branch coverage?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">(I tested on a simple program and region coverage seems pretty close to branch coverage)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Any help is highly appreciated<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Ted Xie<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</body>
</html>