<html>
<head>
<base href="https://bugs.llvm.org/">
</head>
<body><table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Bug ID</th>
<td><a class="bz_bug_link
bz_status_NEW "
title="NEW - encodeULEB128() buffer overflow cause tblgen "stack smashing detected" crash"
href="https://bugs.llvm.org/show_bug.cgi?id=37447">37447</a>
</td>
</tr>
<tr>
<th>Summary</th>
<td>encodeULEB128() buffer overflow cause tblgen "stack smashing detected" crash
</td>
</tr>
<tr>
<th>Product</th>
<td>new-bugs
</td>
</tr>
<tr>
<th>Version</th>
<td>trunk
</td>
</tr>
<tr>
<th>Hardware</th>
<td>PC
</td>
</tr>
<tr>
<th>OS</th>
<td>Windows NT
</td>
</tr>
<tr>
<th>Status</th>
<td>NEW
</td>
</tr>
<tr>
<th>Severity</th>
<td>enhancement
</td>
</tr>
<tr>
<th>Priority</th>
<td>P
</td>
</tr>
<tr>
<th>Component</th>
<td>new bugs
</td>
</tr>
<tr>
<th>Assignee</th>
<td>unassignedbugs@nondot.org
</td>
</tr>
<tr>
<th>Reporter</th>
<td>kaz-marukawa@xr.jp.nec.com
</td>
</tr>
<tr>
<th>CC</th>
<td>llvm-bugs@lists.llvm.org
</td>
</tr></table>
<p>
<div>
<pre>Created <span class=""><a href="attachment.cgi?id=20301" name="attach_20301" title="patch to fix described problem">attachment 20301</a> <a href="attachment.cgi?id=20301&action=edit" title="patch to fix described problem">[details]</a></span>
patch to fix described problem
When I tried to implement a backend for a CPU using 64 bit wide instructions,
tblgen crashes very often with "stack smashing detected" error.
I drilled down it and found tblgen calls encodeULEB128() with 64 bit value
for input and 8 bytes buffer for output sometimes. This encodeULEB128()
encodes a given value into buffer using SDNV (Self-Delimiting Numeric Value)
format. This format requires 10 bytes buffer at maximum (abs(64/7) = 10).
For example, 0x8000000000000000 input is converted into "80 80 80 80 80 80
80 80 80 01".
Once I applied attached patch, tblgen works fine with my modified backend.
Please review this patch and fix the bug. Thanks.
In this patch, modification of TableGen/FixedLenDecoderEmitter.cpp is what
above crash causes. Modification of ProfileData/InstrProf.cpp is what I
found by grep and this may not be needed.</pre>
</div>
</p>
<hr>
<span>You are receiving this mail because:</span>
<ul>
<li>You are on the CC list for the bug.</li>
</ul>
</body>
</html>