<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi Reid,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thanks for the feedback.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">This feature exists primarily for testing (verifying that the MCJIT/RuntimeDyld implementation doesn’t prevent remote execution), so my approach was to do the
simplest thing that worked. I did look at the Support/Program.h stuff and it definitely didn’t seem as simple as rolling my own. Obviously I may feel differently once I see what the buildbots do with this.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">On the other hand, this feature is likely to see secondary use as a reference implementation for remote execution. I suppose that warrants at least a comment
referencing the Program.h stuff. I’ll also take another look and see how much work would be involved in switching over to use that.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">-Andy<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Reid Kleckner [mailto:rnk@google.com]
<br>
<b>Sent:</b> Wednesday, October 02, 2013 11:49 AM<br>
<b>To:</b> Kaylor, Andrew<br>
<b>Cc:</b> llvm-commits<br>
<b>Subject:</b> Re: [llvm] r191843 - Adding out-of-process execution support to lli.<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">I don't think lli should be rolling it's own process creation and pipe code. Have you looked at Support/Program.h? It will use the preferred posix_spawn() call on Mac, and other things that you would otherwise forget.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">If the "redirects" interface isn't good enough, feel free to augment it.<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Wed, Oct 2, 2013 at 1:12 PM, Andrew Kaylor <<a href="mailto:andrew.kaylor@intel.com" target="_blank">andrew.kaylor@intel.com</a>> wrote:<o:p></o:p></p>
<p class="MsoNormal">Author: akaylor<br>
Date: Wed Oct 2 12:12:36 2013<br>
New Revision: 191843<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=191843&view=rev" target="_blank">
http://llvm.org/viewvc/llvm-project?rev=191843&view=rev</a><br>
Log:<br>
Adding out-of-process execution support to lli.<br>
<br>
At this time only Unix-based systems are supported. Windows has stubs and should re-route to the simulated mode.<br>
<br>
Thanks to Sriram Murali for contributions to this patch.<br>
<br>
<br>
<br>
Added:<br>
llvm/trunk/tools/lli/ChildTarget/<br>
llvm/trunk/tools/lli/ChildTarget/CMakeLists.txt<br>
llvm/trunk/tools/lli/ChildTarget/ChildTarget.cpp<br>
llvm/trunk/tools/lli/ChildTarget/Makefile<br>
llvm/trunk/tools/lli/ChildTarget/Unix/<br>
llvm/trunk/tools/lli/ChildTarget/Unix/ChildTarget.inc<br>
llvm/trunk/tools/lli/ChildTarget/Windows/<br>
llvm/trunk/tools/lli/ChildTarget/Windows/ChildTarget.inc<br>
llvm/trunk/tools/lli/RemoteTargetExternal.cpp<br>
llvm/trunk/tools/lli/RemoteTargetExternal.h<br>
llvm/trunk/tools/lli/RemoteTargetMessage.h<br>
llvm/trunk/tools/lli/Unix/<br>
llvm/trunk/tools/lli/Unix/RemoteTargetExternal.inc<br>
llvm/trunk/tools/lli/Windows/<br>
llvm/trunk/tools/lli/Windows/RemoteTargetExternal.inc<br>
Modified:<br>
llvm/trunk/test/ExecutionEngine/MCJIT/remote/simpletest-remote.ll<br>
llvm/trunk/test/ExecutionEngine/MCJIT/remote/stubs-remote.ll<br>
llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-common-symbols-remote.ll<br>
llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-data-align-remote.ll<br>
llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-fp-no-external-funcs-remote.ll<br>
llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-global-init-nonzero-remote.ll<br>
llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-ptr-reloc-remote.ll<br>
llvm/trunk/tools/lli/CMakeLists.txt<br>
llvm/trunk/tools/lli/Makefile<br>
llvm/trunk/tools/lli/RemoteTarget.cpp<br>
llvm/trunk/tools/lli/RemoteTarget.h<br>
llvm/trunk/tools/lli/lli.cpp<br>
<br>
Modified: llvm/trunk/test/ExecutionEngine/MCJIT/remote/simpletest-remote.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/ExecutionEngine/MCJIT/remote/simpletest-remote.ll?rev=191843&r1=191842&r2=191843&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/ExecutionEngine/MCJIT/remote/simpletest-remote.ll?rev=191843&r1=191842&r2=191843&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/test/ExecutionEngine/MCJIT/remote/simpletest-remote.ll (original)<br>
+++ llvm/trunk/test/ExecutionEngine/MCJIT/remote/simpletest-remote.ll Wed Oct 2 12:12:36 2013<br>
@@ -1,4 +1,4 @@<br>
-; RUN: %lli_mcjit -remote-mcjit %s > /dev/null<br>
+; RUN: %lli_mcjit -remote-mcjit -mcjit-remote-process=lli-child-target %s > /dev/null<br>
; XFAIL: mips<br>
<br>
define i32 @bar() {<br>
<br>
Modified: llvm/trunk/test/ExecutionEngine/MCJIT/remote/stubs-remote.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/ExecutionEngine/MCJIT/remote/stubs-remote.ll?rev=191843&r1=191842&r2=191843&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/ExecutionEngine/MCJIT/remote/stubs-remote.ll?rev=191843&r1=191842&r2=191843&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/test/ExecutionEngine/MCJIT/remote/stubs-remote.ll (original)<br>
+++ llvm/trunk/test/ExecutionEngine/MCJIT/remote/stubs-remote.ll Wed Oct 2 12:12:36 2013<br>
@@ -1,4 +1,4 @@<br>
-; RUN: %lli_mcjit -remote-mcjit -disable-lazy-compilation=false %s<br>
+; RUN: %lli_mcjit -remote-mcjit -disable-lazy-compilation=false -mcjit-remote-process=lli-child-target %s<br>
; XFAIL: mips<br>
<br>
define i32 @main() nounwind {<br>
<br>
Modified: llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-common-symbols-remote.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-common-symbols-remote.ll?rev=191843&r1=191842&r2=191843&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-common-symbols-remote.ll?rev=191843&r1=191842&r2=191843&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-common-symbols-remote.ll (original)<br>
+++ llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-common-symbols-remote.ll Wed Oct 2 12:12:36 2013<br>
@@ -1,4 +1,4 @@<br>
-; RUN: %lli_mcjit -remote-mcjit -O0 -disable-lazy-compilation=false %s<br>
+; RUN: %lli_mcjit -remote-mcjit -O0 -disable-lazy-compilation=false -mcjit-remote-process=lli-child-target %s<br>
; XFAIL: mips<br>
<br>
; The intention of this test is to verify that symbols mapped to COMMON in ELF<br>
<br>
Modified: llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-data-align-remote.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-data-align-remote.ll?rev=191843&r1=191842&r2=191843&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-data-align-remote.ll?rev=191843&r1=191842&r2=191843&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-data-align-remote.ll (original)<br>
+++ llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-data-align-remote.ll Wed Oct 2 12:12:36 2013<br>
@@ -1,4 +1,4 @@<br>
-; RUN: %lli_mcjit -remote-mcjit -O0 %s<br>
+; RUN: %lli_mcjit -remote-mcjit -O0 -mcjit-remote-process=lli-child-target %s<br>
; XFAIL: mips<br>
<br>
; Check that a variable is always aligned as specified.<br>
<br>
Modified: llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-fp-no-external-funcs-remote.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-fp-no-external-funcs-remote.ll?rev=191843&r1=191842&r2=191843&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-fp-no-external-funcs-remote.ll?rev=191843&r1=191842&r2=191843&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-fp-no-external-funcs-remote.ll (original)<br>
+++ llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-fp-no-external-funcs-remote.ll Wed Oct 2 12:12:36 2013<br>
@@ -1,4 +1,4 @@<br>
-; RUN: %lli_mcjit -remote-mcjit %s > /dev/null<br>
+; RUN: %lli_mcjit -remote-mcjit -mcjit-remote-process=lli-child-target %s > /dev/null<br>
; XFAIL: mips<br>
<br>
define double @test(double* %DP, double %Arg) {<br>
<br>
Modified: llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-global-init-nonzero-remote.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-global-init-nonzero-remote.ll?rev=191843&r1=191842&r2=191843&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-global-init-nonzero-remote.ll?rev=191843&r1=191842&r2=191843&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-global-init-nonzero-remote.ll (original)<br>
+++ llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-global-init-nonzero-remote.ll Wed Oct 2 12:12:36 2013<br>
@@ -1,4 +1,4 @@<br>
-; RUN: %lli_mcjit -remote-mcjit %s > /dev/null<br>
+; RUN: %lli_mcjit -remote-mcjit -mcjit-remote-process=lli-child-target %s > /dev/null<br>
; XFAIL: mips<br>
<br>
@count = global i32 1, align 4<br>
<br>
Modified: llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-ptr-reloc-remote.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-ptr-reloc-remote.ll?rev=191843&r1=191842&r2=191843&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-ptr-reloc-remote.ll?rev=191843&r1=191842&r2=191843&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-ptr-reloc-remote.ll (original)<br>
+++ llvm/trunk/test/ExecutionEngine/MCJIT/remote/test-ptr-reloc-remote.ll Wed Oct 2 12:12:36 2013<br>
@@ -1,4 +1,4 @@<br>
-; RUN: %lli_mcjit -remote-mcjit -O0 %s<br>
+; RUN: %lli_mcjit -remote-mcjit -O0 -mcjit-remote-process=lli-child-target %s<br>
<br>
@.str = private unnamed_addr constant [6 x i8] c"data1\00", align 1<br>
@ptr = global i8* getelementptr inbounds ([6 x i8]* @.str, i32 0, i32 0), align 4<br>
<br>
Modified: llvm/trunk/tools/lli/CMakeLists.txt<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/CMakeLists.txt?rev=191843&r1=191842&r2=191843&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/CMakeLists.txt?rev=191843&r1=191842&r2=191843&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/tools/lli/CMakeLists.txt (original)<br>
+++ llvm/trunk/tools/lli/CMakeLists.txt Wed Oct 2 12:12:36 2013<br>
@@ -1,6 +1,8 @@<br>
<br>
set(LLVM_LINK_COMPONENTS mcjit jit interpreter nativecodegen bitreader asmparser irreader selectiondag native instrumentation)<br>
<br>
+add_subdirectory(ChildTarget)<br>
+<br>
if( LLVM_USE_OPROFILE )<br>
set(LLVM_LINK_COMPONENTS<br>
${LLVM_LINK_COMPONENTS}<br>
@@ -21,4 +23,5 @@ add_llvm_tool(lli<br>
lli.cpp<br>
RecordingMemoryManager.cpp<br>
RemoteTarget.cpp<br>
+ RemoteTargetExternal.cpp<br>
)<br>
<br>
Added: llvm/trunk/tools/lli/ChildTarget/CMakeLists.txt<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/ChildTarget/CMakeLists.txt?rev=191843&view=auto" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/ChildTarget/CMakeLists.txt?rev=191843&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/tools/lli/ChildTarget/CMakeLists.txt (added)<br>
+++ llvm/trunk/tools/lli/ChildTarget/CMakeLists.txt Wed Oct 2 12:12:36 2013<br>
@@ -0,0 +1,3 @@<br>
+add_llvm_tool(lli-child-target<br>
+ ChildTarget.cpp<br>
+ )<br>
<br>
Added: llvm/trunk/tools/lli/ChildTarget/ChildTarget.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/ChildTarget/ChildTarget.cpp?rev=191843&view=auto" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/ChildTarget/ChildTarget.cpp?rev=191843&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/tools/lli/ChildTarget/ChildTarget.cpp (added)<br>
+++ llvm/trunk/tools/lli/ChildTarget/ChildTarget.cpp Wed Oct 2 12:12:36 2013<br>
@@ -0,0 +1,241 @@<br>
+#include "llvm/Config/config.h"<br>
+<br>
+#include "../RemoteTargetMessage.h"<br>
+#include <assert.h><br>
+#include <map><br>
+#include <stdint.h><br>
+#include <string><br>
+#include <vector><br>
+<br>
+using namespace llvm;<br>
+<br>
+class LLIChildTarget {<br>
+public:<br>
+ void initialize();<br>
+ LLIMessageType waitForIncomingMessage();<br>
+ void handleMessage(LLIMessageType messageType);<br>
+<br>
+private:<br>
+ // Incoming message handlers<br>
+ void handleAllocateSpace();<br>
+ void handleLoadSection(bool IsCode);<br>
+ void handleExecute();<br>
+ void handleTerminate();<br>
+<br>
+ // Outgoing message handlers<br>
+ void sendChildActive();<br>
+ void sendAllocationResult(uint64_t Addr);<br>
+ void sendLoadComplete();<br>
+ void sendExecutionComplete(uint64_t Result);<br>
+<br>
+ // OS-specific functions<br>
+ void initializeConnection();<br>
+ int WriteBytes(const void *Data, size_t Size);<br>
+ int ReadBytes(void *Data, size_t Size);<br>
+ uint64_t allocate(uint32_t Alignment, uint32_t Size);<br>
+ void makeSectionExecutable(uint64_t Addr, uint32_t Size);<br>
+ void InvalidateInstructionCache(const void *Addr, size_t Len);<br>
+ void releaseMemory(uint64_t Addr, uint32_t Size);<br>
+<br>
+ // Store a map of allocated buffers to sizes.<br>
+ typedef std::map<uint64_t, uint32_t> AllocMapType;<br>
+ AllocMapType m_AllocatedBufferMap;<br>
+<br>
+ // Communication handles (OS-specific)<br>
+ void *ConnectionData;<br>
+};<br>
+<br>
+int main() {<br>
+ LLIChildTarget ThisChild;<br>
+ ThisChild.initialize();<br>
+ LLIMessageType MsgType;<br>
+ do {<br>
+ MsgType = ThisChild.waitForIncomingMessage();<br>
+ ThisChild.handleMessage(MsgType);<br>
+ } while (MsgType != LLI_Terminate &&<br>
+ MsgType != LLI_Error);<br>
+ return 0;<br>
+}<br>
+<br>
+// Public methods<br>
+void LLIChildTarget::initialize() {<br>
+ initializeConnection();<br>
+ sendChildActive();<br>
+}<br>
+<br>
+LLIMessageType LLIChildTarget::waitForIncomingMessage() {<br>
+ int32_t MsgType = -1;<br>
+ if (ReadBytes(&MsgType, 4) > 0)<br>
+ return (LLIMessageType)MsgType;<br>
+ return LLI_Error;<br>
+}<br>
+<br>
+void LLIChildTarget::handleMessage(LLIMessageType messageType) {<br>
+ switch (messageType) {<br>
+ case LLI_AllocateSpace:<br>
+ handleAllocateSpace();<br>
+ break;<br>
+ case LLI_LoadCodeSection:<br>
+ handleLoadSection(true);<br>
+ break;<br>
+ case LLI_LoadDataSection:<br>
+ handleLoadSection(false);<br>
+ break;<br>
+ case LLI_Execute:<br>
+ handleExecute();<br>
+ break;<br>
+ case LLI_Terminate:<br>
+ handleTerminate();<br>
+ break;<br>
+ default:<br>
+ // FIXME: Handle error!<br>
+ break;<br>
+ }<br>
+}<br>
+<br>
+// Incoming message handlers<br>
+void LLIChildTarget::handleAllocateSpace() {<br>
+ // Read and verify the message data size.<br>
+ uint32_t DataSize;<br>
+ int rc = ReadBytes(&DataSize, 4);<br>
+ assert(rc == 4);<br>
+ assert(DataSize == 8);<br>
+<br>
+ // Read the message arguments.<br>
+ uint32_t Alignment;<br>
+ uint32_t AllocSize;<br>
+ rc = ReadBytes(&Alignment, 4);<br>
+ assert(rc == 4);<br>
+ rc = ReadBytes(&AllocSize, 4);<br>
+ assert(rc == 4);<br>
+<br>
+ // Allocate the memory.<br>
+ uint64_t Addr = allocate(Alignment, AllocSize);<br>
+<br>
+ // Send AllocationResult message.<br>
+ sendAllocationResult(Addr);<br>
+}<br>
+<br>
+void LLIChildTarget::handleLoadSection(bool IsCode) {<br>
+ // Read the message data size.<br>
+ uint32_t DataSize;<br>
+ int rc = ReadBytes(&DataSize, 4);<br>
+ assert(rc == 4);<br>
+<br>
+ // Read the target load address.<br>
+ uint64_t Addr;<br>
+ rc = ReadBytes(&Addr, 8);<br>
+ assert(rc == 8);<br>
+<br>
+ size_t BufferSize = DataSize - 8;<br>
+<br>
+ // FIXME: Verify that this is in allocated space<br>
+<br>
+ // Read section data into previously allocated buffer<br>
+ rc = ReadBytes((void*)Addr, DataSize - 8);<br>
+ assert(rc == (int)(BufferSize));<br>
+<br>
+ // If IsCode, mark memory executable<br>
+ if (IsCode)<br>
+ makeSectionExecutable(Addr, BufferSize);<br>
+<br>
+ // Send MarkLoadComplete message.<br>
+ sendLoadComplete();<br>
+}<br>
+<br>
+void LLIChildTarget::handleExecute() {<br>
+ // Read the message data size.<br>
+ uint32_t DataSize;<br>
+ int rc = ReadBytes(&DataSize, 4);<br>
+ assert(rc == 4);<br>
+ assert(DataSize == 8);<br>
+<br>
+ // Read the target address.<br>
+ uint64_t Addr;<br>
+ rc = ReadBytes(&Addr, 8);<br>
+ assert(rc == 8);<br>
+<br>
+ // Call function<br>
+ int Result;<br>
+ int (*fn)(void) = (int(*)(void))Addr;<br>
+ Result = fn();<br>
+<br>
+ // Send ExecutionResult message.<br>
+ sendExecutionComplete((int64_t)Result);<br>
+}<br>
+<br>
+void LLIChildTarget::handleTerminate() {<br>
+ // Release all allocated memory<br>
+ AllocMapType::iterator Begin = m_AllocatedBufferMap.begin();<br>
+ AllocMapType::iterator End = m_AllocatedBufferMap.end();<br>
+ for (AllocMapType::iterator It = Begin; It != End; ++It) {<br>
+ releaseMemory(It->first, It->second);<br>
+ }<br>
+ m_AllocatedBufferMap.clear();<br>
+}<br>
+<br>
+// Outgoing message handlers<br>
+void LLIChildTarget::sendChildActive() {<br>
+ // Write the message type.<br>
+ uint32_t MsgType = (uint32_t)LLI_ChildActive;<br>
+ int rc = WriteBytes(&MsgType, 4);<br>
+ assert(rc == 4);<br>
+<br>
+ // Write the data size.<br>
+ uint32_t DataSize = 0;<br>
+ rc = WriteBytes(&DataSize, 4);<br>
+ assert(rc == 4);<br>
+}<br>
+<br>
+void LLIChildTarget::sendAllocationResult(uint64_t Addr) {<br>
+ // Write the message type.<br>
+ uint32_t MsgType = (uint32_t)LLI_AllocationResult;<br>
+ int rc = WriteBytes(&MsgType, 4);<br>
+ assert(rc == 4);<br>
+<br>
+ // Write the data size.<br>
+ uint32_t DataSize = 8;<br>
+ rc = WriteBytes(&DataSize, 4);<br>
+ assert(rc == 4);<br>
+<br>
+ // Write the allocated address.<br>
+ rc = WriteBytes(&Addr, 8);<br>
+ assert(rc == 8);<br>
+}<br>
+<br>
+void LLIChildTarget::sendLoadComplete() {<br>
+ // Write the message type.<br>
+ uint32_t MsgType = (uint32_t)LLI_LoadComplete;<br>
+ int rc = WriteBytes(&MsgType, 4);<br>
+ assert(rc == 4);<br>
+<br>
+ // Write the data size.<br>
+ uint32_t DataSize = 0;<br>
+ rc = WriteBytes(&DataSize, 4);<br>
+ assert(rc == 4);<br>
+}<br>
+<br>
+void LLIChildTarget::sendExecutionComplete(uint64_t Result) {<br>
+ // Write the message type.<br>
+ uint32_t MsgType = (uint32_t)LLI_ExecutionResult;<br>
+ int rc = WriteBytes(&MsgType, 4);<br>
+ assert(rc == 4);<br>
+<br>
+<br>
+ // Write the data size.<br>
+ uint32_t DataSize = 8;<br>
+ rc = WriteBytes(&DataSize, 4);<br>
+ assert(rc == 4);<br>
+<br>
+ // Write the result.<br>
+ rc = WriteBytes(&Result, 8);<br>
+ assert(rc == 8);<br>
+}<br>
+<br>
+#ifdef LLVM_ON_UNIX<br>
+#include "Unix/ChildTarget.inc"<br>
+#endif<br>
+<br>
+#ifdef LLVM_ON_WIN32<br>
+#include "Windows/ChildTarget.inc"<br>
+#endif<br>
<br>
Added: llvm/trunk/tools/lli/ChildTarget/Makefile<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/ChildTarget/Makefile?rev=191843&view=auto" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/ChildTarget/Makefile?rev=191843&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/tools/lli/ChildTarget/Makefile (added)<br>
+++ llvm/trunk/tools/lli/ChildTarget/Makefile Wed Oct 2 12:12:36 2013<br>
@@ -0,0 +1,17 @@<br>
+##===- tools/lli/Makefile ------------------------------*- Makefile -*-===##<br>
+#<br>
+# The LLVM Compiler Infrastructure<br>
+#<br>
+# This file is distributed under the University of Illinois Open Source<br>
+# License. See LICENSE.TXT for details.<br>
+#<br>
+##===----------------------------------------------------------------------===##<br>
+<br>
+LEVEL := ../../..<br>
+TOOLNAME := lli-child-target<br>
+<br>
+include $(LEVEL)/Makefile.config<br>
+<br>
+LINK_COMPONENTS :=<br>
+<br>
+include $(LLVM_SRC_ROOT)/Makefile.rules<br>
<br>
Added: llvm/trunk/tools/lli/ChildTarget/Unix/ChildTarget.inc<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/ChildTarget/Unix/ChildTarget.inc?rev=191843&view=auto" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/ChildTarget/Unix/ChildTarget.inc?rev=191843&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/tools/lli/ChildTarget/Unix/ChildTarget.inc (added)<br>
+++ llvm/trunk/tools/lli/ChildTarget/Unix/ChildTarget.inc Wed Oct 2 12:12:36 2013<br>
@@ -0,0 +1,141 @@<br>
+//===- ChildTarget.inc - Child process for external JIT execution for Unix -==//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+//<br>
+// Implementation of the Unix-specific parts of the ChildTarget class<br>
+// which executes JITed code in a separate process from where it was built.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+#include <unistd.h><br>
+#include <stdio.h><br>
+#include <stdlib.h><br>
+#include <sys/mman.h><br>
+<br>
+namespace {<br>
+<br>
+struct ConnectionData_t {<br>
+ int InputPipe;<br>
+ int OutputPipe;<br>
+<br>
+ ConnectionData_t(int in, int out) : InputPipe(in), OutputPipe(out) {}<br>
+};<br>
+<br>
+} // namespace<br>
+<br>
+// OS-specific methods<br>
+void LLIChildTarget::initializeConnection() {<br>
+ // Store the parent ends of the pipes<br>
+ ConnectionData = (void*)new ConnectionData_t(STDIN_FILENO, STDOUT_FILENO);<br>
+}<br>
+<br>
+int LLIChildTarget::WriteBytes(const void *Data, size_t Size) {<br>
+ return write(((ConnectionData_t*)ConnectionData)->OutputPipe, Data, Size);<br>
+}<br>
+<br>
+int LLIChildTarget::ReadBytes(void *Data, size_t Size) {<br>
+ return read(((ConnectionData_t*)ConnectionData)->InputPipe, Data, Size);<br>
+}<br>
+<br>
+// The functions below duplicate functionality that is implemented in<br>
+// Support/Memory.cpp with the goal of avoiding a dependency on any<br>
+// llvm libraries.<br>
+<br>
+uint64_t LLIChildTarget::allocate(uint32_t Alignment, uint32_t Size) {<br>
+ if (!Alignment)<br>
+ Alignment = 16;<br>
+<br>
+ static const size_t PageSize = getpagesize();<br>
+ const size_t NumPages = (Size+PageSize-1)/PageSize;<br>
+ Size = NumPages*PageSize;<br>
+<br>
+ int fd = -1;<br>
+#ifdef NEED_DEV_ZERO_FOR_MMAP<br>
+ static int zero_fd = open("/dev/zero", O_RDWR);<br>
+ if (zero_fd == -1)<br>
+ return 0;<br>
+ fd = zero_fd;<br>
+#endif<br>
+<br>
+ int MMFlags = MAP_PRIVATE |<br>
+#ifdef HAVE_MMAP_ANONYMOUS<br>
+ MAP_ANONYMOUS<br>
+#else<br>
+ MAP_ANON<br>
+#endif<br>
+ ; // Ends statement above<br>
+<br>
+ uint64_t Addr = (uint64_t)::mmap(0, Size, PROT_READ | PROT_WRITE, MMFlags, fd, 0);<br>
+ if (Addr == (uint64_t)MAP_FAILED)<br>
+ return 0;<br>
+<br>
+ // Align the address.<br>
+ Addr = (Addr + Alignment - 1) & ~(uintptr_t)(Alignment - 1);<br>
+<br>
+ m_AllocatedBufferMap[Addr] = Size;<br>
+<br>
+ // Return aligned address<br>
+ return Addr;<br>
+}<br>
+<br>
+void LLIChildTarget::makeSectionExecutable(uint64_t Addr, uint32_t Size) {<br>
+ // FIXME: We have to mark the memory as RWX because multiple code chunks may<br>
+ // be on the same page. The RemoteTarget interface should be changed to<br>
+ // work around that.<br>
+ int Result = ::mprotect((void*)Addr, Size, PROT_READ | PROT_WRITE | PROT_EXEC);<br>
+ if (Result != 0)<br>
+ InvalidateInstructionCache((const void *)Addr, Size);<br>
+}<br>
+<br>
+/// InvalidateInstructionCache - Before the JIT can run a block of code<br>
+/// that has been emitted it must invalidate the instruction cache on some<br>
+/// platforms.<br>
+void LLIChildTarget::InvalidateInstructionCache(const void *Addr,<br>
+ size_t Len) {<br>
+<br>
+// icache invalidation for PPC and ARM.<br>
+#if defined(__APPLE__)<br>
+<br>
+# if (defined(__POWERPC__) || defined (__ppc__) || \<br>
+ defined(_POWER) || defined(_ARCH_PPC)) || defined(__arm__)<br>
+ sys_icache_invalidate(const_cast<void *>(Addr), Len);<br>
+# endif<br>
+<br>
+#else<br>
+<br>
+# if (defined(__POWERPC__) || defined (__ppc__) || \<br>
+ defined(_POWER) || defined(_ARCH_PPC)) && defined(__GNUC__)<br>
+ const size_t LineSize = 32;<br>
+<br>
+ const intptr_t Mask = ~(LineSize - 1);<br>
+ const intptr_t StartLine = ((intptr_t) Addr) & Mask;<br>
+ const intptr_t EndLine = ((intptr_t) Addr + Len + LineSize - 1) & Mask;<br>
+<br>
+ for (intptr_t Line = StartLine; Line < EndLine; Line += LineSize)<br>
+ asm volatile("dcbf 0, %0" : : "r"(Line));<br>
+ asm volatile("sync");<br>
+<br>
+ for (intptr_t Line = StartLine; Line < EndLine; Line += LineSize)<br>
+ asm volatile("icbi 0, %0" : : "r"(Line));<br>
+ asm volatile("isync");<br>
+# elif defined(__arm__) && defined(__GNUC__)<br>
+ // FIXME: Can we safely always call this for __GNUC__ everywhere?<br>
+ const char *Start = static_cast<const char *>(Addr);<br>
+ const char *End = Start + Len;<br>
+ __clear_cache(const_cast<char *>(Start), const_cast<char *>(End));<br>
+# elif defined(__mips__)<br>
+ const char *Start = static_cast<const char *>(Addr);<br>
+ cacheflush(const_cast<char *>(Start), Len, BCACHE);<br>
+# endif<br>
+<br>
+#endif // end apple<br>
+}<br>
+<br>
+void LLIChildTarget::releaseMemory(uint64_t Addr, uint32_t Size) {<br>
+ ::munmap((void*)Addr, Size);<br>
+}<br>
<br>
Added: llvm/trunk/tools/lli/ChildTarget/Windows/ChildTarget.inc<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/ChildTarget/Windows/ChildTarget.inc?rev=191843&view=auto" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/ChildTarget/Windows/ChildTarget.inc?rev=191843&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/tools/lli/ChildTarget/Windows/ChildTarget.inc (added)<br>
+++ llvm/trunk/tools/lli/ChildTarget/Windows/ChildTarget.inc Wed Oct 2 12:12:36 2013<br>
@@ -0,0 +1,41 @@<br>
+//=- ChildTarget.inc - Child process for external JIT execution for Windows -=//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+//<br>
+// Non-implementation of the Windows-specific parts of the ChildTarget class<br>
+// which executes JITed code in a separate process from where it was built.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+// The RemoteTargetExternal implementation should prevent us from ever getting<br>
+// here on Windows, but nothing prevents a user from running this directly.<br>
+void LLIChildTarget::initializeConnection() {<br>
+ assert(0 && "lli-child-target is not implemented for Windows");<br>
+}<br>
+<br>
+int LLIChildTarget::WriteBytes(const void *Data, size_t Size) {<br>
+ return 0;<br>
+}<br>
+<br>
+int LLIChildTarget::ReadBytes(void *Data, size_t Size) {<br>
+ return 0;<br>
+}<br>
+<br>
+uint64_t LLIChildTarget::allocate(uint32_t Alignment, uint32_t Size) {<br>
+ return 0;<br>
+}<br>
+<br>
+void LLIChildTarget::makeSectionExecutable(uint64_t Addr, uint32_t Size) {<br>
+}<br>
+<br>
+void LLIChildTarget::InvalidateInstructionCache(const void *Addr,<br>
+ size_t Len) {<br>
+}<br>
+<br>
+void LLIChildTarget::releaseMemory(uint64_t Addr, uint32_t Size) {<br>
+}<br>
<br>
Modified: llvm/trunk/tools/lli/Makefile<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/Makefile?rev=191843&r1=191842&r2=191843&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/Makefile?rev=191843&r1=191842&r2=191843&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/tools/lli/Makefile (original)<br>
+++ llvm/trunk/tools/lli/Makefile Wed Oct 2 12:12:36 2013<br>
@@ -10,6 +10,8 @@<br>
LEVEL := ../..<br>
TOOLNAME := lli<br>
<br>
+PARALLEL_DIRS := ChildTarget<br>
+<br>
include $(LEVEL)/Makefile.config<br>
<br>
LINK_COMPONENTS := mcjit jit instrumentation interpreter nativecodegen bitreader asmparser irreader selectiondag native<br>
<br>
Modified: llvm/trunk/tools/lli/RemoteTarget.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/RemoteTarget.cpp?rev=191843&r1=191842&r2=191843&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/RemoteTarget.cpp?rev=191843&r1=191842&r2=191843&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/tools/lli/RemoteTarget.cpp (original)<br>
+++ llvm/trunk/tools/lli/RemoteTarget.cpp Wed Oct 2 12:12:36 2013<br>
@@ -13,13 +13,44 @@<br>
//===----------------------------------------------------------------------===//<br>
<br>
#include "RemoteTarget.h"<br>
+#include "RemoteTargetExternal.h"<br>
#include "llvm/ADT/StringRef.h"<br>
#include "llvm/Support/DataTypes.h"<br>
#include "llvm/Support/Memory.h"<br>
#include <stdlib.h><br>
#include <string><br>
+<br>
using namespace llvm;<br>
<br>
+// Static methods<br>
+RemoteTarget *RemoteTarget::createRemoteTarget() {<br>
+ return new RemoteTarget;<br>
+}<br>
+<br>
+RemoteTarget *RemoteTarget::createExternalRemoteTarget(std::string &ChildName) {<br>
+#ifdef LLVM_ON_UNIX<br>
+ return new RemoteTargetExternal(ChildName);<br>
+#else<br>
+ return 0;<br>
+#endif<br>
+}<br>
+<br>
+bool RemoteTarget::hostSupportsExternalRemoteTarget() {<br>
+#ifdef LLVM_ON_UNIX<br>
+ return true;<br>
+#else<br>
+ return false;<br>
+#endif<br>
+}<br>
+<br>
+<br>
+////////////////////////////////////////////////////////////////////////////////<br>
+// Simulated remote execution<br>
+//<br>
+// This implementation will simply move generated code and data to a new memory<br>
+// location in the current executable and let it run from there.<br>
+////////////////////////////////////////////////////////////////////////////////<br>
+<br>
bool RemoteTarget::allocateSpace(size_t Size, unsigned Alignment,<br>
uint64_t &Address) {<br>
sys::MemoryBlock *Prev = Allocations.size() ? &Allocations.back() : NULL;<br>
<br>
Modified: llvm/trunk/tools/lli/RemoteTarget.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/RemoteTarget.h?rev=191843&r1=191842&r2=191843&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/RemoteTarget.h?rev=191843&r1=191842&r2=191843&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/tools/lli/RemoteTarget.h (original)<br>
+++ llvm/trunk/tools/lli/RemoteTarget.h Wed Oct 2 12:12:36 2013<br>
@@ -41,7 +41,9 @@ public:<br>
///<br>
/// @returns False on success. On failure, ErrorMsg is updated with<br>
/// descriptive text of the encountered error.<br>
- bool allocateSpace(size_t Size, unsigned Alignment, uint64_t &Address);<br>
+ virtual bool allocateSpace(size_t Size,<br>
+ unsigned Alignment,<br>
+ uint64_t &Address);<br>
<br>
/// Load data into the target address space.<br>
///<br>
@@ -51,7 +53,9 @@ public:<br>
///<br>
/// @returns False on success. On failure, ErrorMsg is updated with<br>
/// descriptive text of the encountered error.<br>
- bool loadData(uint64_t Address, const void *Data, size_t Size);<br>
+ virtual bool loadData(uint64_t Address,<br>
+ const void *Data,<br>
+ size_t Size);<br>
<br>
/// Load code into the target address space and prepare it for execution.<br>
///<br>
@@ -61,7 +65,9 @@ public:<br>
///<br>
/// @returns False on success. On failure, ErrorMsg is updated with<br>
/// descriptive text of the encountered error.<br>
- bool loadCode(uint64_t Address, const void *Data, size_t Size);<br>
+ virtual bool loadCode(uint64_t Address,<br>
+ const void *Data,<br>
+ size_t Size);<br>
<br>
/// Execute code in the target process. The called function is required<br>
/// to be of signature int "(*)(void)".<br>
@@ -72,24 +78,29 @@ public:<br>
///<br>
/// @returns False on success. On failure, ErrorMsg is updated with<br>
/// descriptive text of the encountered error.<br>
- bool executeCode(uint64_t Address, int &RetVal);<br>
+ virtual bool executeCode(uint64_t Address,<br>
+ int &RetVal);<br>
<br>
/// Minimum alignment for memory permissions. Used to seperate code and<br>
/// data regions to make sure data doesn't get marked as code or vice<br>
/// versa.<br>
///<br>
/// @returns Page alignment return value. Default of 4k.<br>
- unsigned getPageAlignment() { return 4096; }<br>
+ virtual unsigned getPageAlignment() { return 4096; }<br>
<br>
/// Start the remote process.<br>
- void create();<br>
+ virtual void create();<br>
<br>
/// Terminate the remote process.<br>
- void stop();<br>
+ virtual void stop();<br>
<br>
RemoteTarget() : ErrorMsg(""), IsRunning(false) {}<br>
- ~RemoteTarget() { if (IsRunning) stop(); }<br>
+ virtual ~RemoteTarget() { if (IsRunning) stop(); }<br>
<br>
+ // Create an instance of the system-specific remote target class.<br>
+ static RemoteTarget *createRemoteTarget();<br>
+ static RemoteTarget *createExternalRemoteTarget(std::string &ChildName);<br>
+ static bool hostSupportsExternalRemoteTarget();<br>
private:<br>
// Main processing function for the remote target process. Command messages<br>
// are received on file descriptor CmdFD and responses come back on OutFD.<br>
<br>
Added: llvm/trunk/tools/lli/RemoteTargetExternal.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/RemoteTargetExternal.cpp?rev=191843&view=auto" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/RemoteTargetExternal.cpp?rev=191843&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/tools/lli/RemoteTargetExternal.cpp (added)<br>
+++ llvm/trunk/tools/lli/RemoteTargetExternal.cpp Wed Oct 2 12:12:36 2013<br>
@@ -0,0 +1,162 @@<br>
+//===---- RemoteTargetExternal.cpp - LLVM out-of-process JIT execution ----===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+//<br>
+// Implementation of the RemoteTargetExternal class which executes JITed code<br>
+// in a separate process from where it was built.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+#include "llvm/Config/config.h"<br>
+<br>
+#include "RemoteTarget.h"<br>
+#include "RemoteTargetExternal.h"<br>
+<br>
+#include "llvm/ADT/StringRef.h"<br>
+#include "llvm/Support/DataTypes.h"<br>
+#include "llvm/Support/Memory.h"<br>
+#include "llvm/Support/Program.h"<br>
+#include "llvm/Support/raw_ostream.h"<br>
+#include <string><br>
+<br>
+using namespace llvm;<br>
+<br>
+bool RemoteTargetExternal::allocateSpace(size_t Size, unsigned Alignment,<br>
+ uint64_t &Address) {<br>
+ SendAllocateSpace(Alignment, Size);<br>
+ Receive(LLI_AllocationResult, Address);<br>
+ return false;<br>
+}<br>
+<br>
+bool RemoteTargetExternal::loadData(uint64_t Address, const void *Data, size_t Size) {<br>
+ SendLoadSection(Address, Data, (uint32_t)Size, false);<br>
+ Receive(LLI_LoadComplete);<br>
+ return false;<br>
+}<br>
+<br>
+bool RemoteTargetExternal::loadCode(uint64_t Address, const void *Data, size_t Size) {<br>
+ SendLoadSection(Address, Data, (uint32_t)Size, true);<br>
+ Receive(LLI_LoadComplete);<br>
+ return false;<br>
+}<br>
+<br>
+bool RemoteTargetExternal::executeCode(uint64_t Address, int &RetVal) {<br>
+ SendExecute(Address);<br>
+<br>
+ Receive(LLI_ExecutionResult, RetVal);<br>
+ return false;<br>
+}<br>
+<br>
+void RemoteTargetExternal::stop() {<br>
+ SendTerminate();<br>
+ Wait();<br>
+}<br>
+<br>
+void RemoteTargetExternal::SendAllocateSpace(uint32_t Alignment, uint32_t Size) {<br>
+ int rc;<br>
+ uint32_t MsgType = (uint32_t)LLI_AllocateSpace;<br>
+ rc = WriteBytes(&MsgType, 4);<br>
+ assert(rc == 4 && "Error writing message type.");<br>
+<br>
+ uint32_t DataSize = 8;<br>
+ rc = WriteBytes(&DataSize, 4);<br>
+ assert(rc == 4 && "Error writing data size.");<br>
+<br>
+ rc = WriteBytes(&Alignment, 4);<br>
+ assert(rc == 4 && "Error writing alignment data.");<br>
+<br>
+ rc = WriteBytes(&Size, 4);<br>
+ assert(rc == 4 && "Error writing size data.");<br>
+}<br>
+<br>
+void RemoteTargetExternal::SendLoadSection(uint64_t Addr,<br>
+ const void *Data,<br>
+ uint32_t Size,<br>
+ bool IsCode) {<br>
+ int rc;<br>
+ uint32_t MsgType = IsCode ? LLI_LoadCodeSection : LLI_LoadDataSection;<br>
+ rc = WriteBytes(&MsgType, 4);<br>
+ assert(rc == 4 && "Error writing message type.");<br>
+<br>
+ uint32_t DataSize = Size + 8;<br>
+ rc = WriteBytes(&DataSize, 4);<br>
+ assert(rc == 4 && "Error writing data size.");<br>
+<br>
+ rc = WriteBytes(&Addr, 8);<br>
+ assert(rc == 8 && "Error writing data.");<br>
+<br>
+ rc = WriteBytes(Data, Size);<br>
+ assert(rc == (int)Size && "Error writing data.");<br>
+}<br>
+<br>
+void RemoteTargetExternal::SendExecute(uint64_t Addr) {<br>
+ int rc;<br>
+ uint32_t MsgType = (uint32_t)LLI_Execute;<br>
+ rc = WriteBytes(&MsgType, 4);<br>
+ assert(rc == 4 && "Error writing message type.");<br>
+<br>
+ uint32_t DataSize = 8;<br>
+ rc = WriteBytes(&DataSize, 4);<br>
+ assert(rc == 4 && "Error writing data size.");<br>
+<br>
+ rc = WriteBytes(&Addr, 8);<br>
+ assert(rc == 8 && "Error writing data.");<br>
+}<br>
+<br>
+void RemoteTargetExternal::SendTerminate() {<br>
+ int rc;<br>
+ uint32_t MsgType = (uint32_t)LLI_Terminate;<br>
+ rc = WriteBytes(&MsgType, 4);<br>
+ assert(rc == 4 && "Error writing message type.");<br>
+<br>
+ // No data or data size is sent with Terminate<br>
+}<br>
+<br>
+<br>
+void RemoteTargetExternal::Receive(LLIMessageType ExpectedMsgType) {<br>
+ int rc;<br>
+ uint32_t MsgType;<br>
+ rc = ReadBytes(&MsgType, 4);<br>
+ assert(rc == 4 && "Error reading message type.");<br>
+ assert(MsgType == ExpectedMsgType && "Error: received unexpected message type.");<br>
+<br>
+ uint32_t DataSize;<br>
+ rc = ReadBytes(&DataSize, 4);<br>
+ assert(rc == 4 && "Error reading data size.");<br>
+ assert(DataSize == 0 && "Error: unexpected data size.");<br>
+}<br>
+<br>
+void RemoteTargetExternal::Receive(LLIMessageType ExpectedMsgType, int &Data) {<br>
+ uint64_t Temp;<br>
+ Receive(ExpectedMsgType, Temp);<br>
+ Data = (int)(int64_t)Temp;<br>
+}<br>
+<br>
+void RemoteTargetExternal::Receive(LLIMessageType ExpectedMsgType, uint64_t &Data) {<br>
+ int rc;<br>
+ uint32_t MsgType;<br>
+ rc = ReadBytes(&MsgType, 4);<br>
+ assert(rc == 4 && "Error reading message type.");<br>
+ assert(MsgType == ExpectedMsgType && "Error: received unexpected message type.");<br>
+<br>
+ uint32_t DataSize;<br>
+ rc = ReadBytes(&DataSize, 4);<br>
+ assert(rc == 4 && "Error reading data size.");<br>
+ assert(DataSize == 8 && "Error: unexpected data size.");<br>
+<br>
+ rc = ReadBytes(&Data, 8);<br>
+ assert(DataSize == 8 && "Error: unexpected data.");<br>
+}<br>
+<br>
+#ifdef LLVM_ON_UNIX<br>
+#include "Unix/RemoteTargetExternal.inc"<br>
+#endif<br>
+<br>
+#ifdef LLVM_ON_WIN32<br>
+#include "Windows/RemoteTargetExternal.inc"<br>
+#endif<br>
<br>
Added: llvm/trunk/tools/lli/RemoteTargetExternal.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/RemoteTargetExternal.h?rev=191843&view=auto" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/RemoteTargetExternal.h?rev=191843&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/tools/lli/RemoteTargetExternal.h (added)<br>
+++ llvm/trunk/tools/lli/RemoteTargetExternal.h Wed Oct 2 12:12:36 2013<br>
@@ -0,0 +1,118 @@<br>
+//===----- RemoteTargetExternal.h - LLVM out-of-process JIT execution -----===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+//<br>
+// Definition of the RemoteTargetExternal class which executes JITed code in a<br>
+// separate process from where it was built.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+#ifndef LLI_REMOTETARGETEXTERNAL_H<br>
+#define LLI_REMOTETARGETEXTERNAL_H<br>
+<br>
+#include "llvm/Config/config.h"<br>
+<br>
+#include "llvm/ADT/SmallVector.h"<br>
+#include "llvm/ADT/StringRef.h"<br>
+#include "llvm/Support/DataTypes.h"<br>
+#include "llvm/Support/Memory.h"<br>
+#include <stdlib.h><br>
+#include <string><br>
+<br>
+#include "RemoteTarget.h"<br>
+#include "RemoteTargetMessage.h"<br>
+<br>
+namespace llvm {<br>
+<br>
+class RemoteTargetExternal : public RemoteTarget {<br>
+public:<br>
+ /// Allocate space in the remote target address space.<br>
+ ///<br>
+ /// @param Size Amount of space, in bytes, to allocate.<br>
+ /// @param Alignment Required minimum alignment for allocated space.<br>
+ /// @param[out] Address Remote address of the allocated memory.<br>
+ ///<br>
+ /// @returns False on success. On failure, ErrorMsg is updated with<br>
+ /// descriptive text of the encountered error.<br>
+ virtual bool allocateSpace(size_t Size,<br>
+ unsigned Alignment,<br>
+ uint64_t &Address);<br>
+<br>
+ /// Load data into the target address space.<br>
+ ///<br>
+ /// @param Address Destination address in the target process.<br>
+ /// @param Data Source address in the host process.<br>
+ /// @param Size Number of bytes to copy.<br>
+ ///<br>
+ /// @returns False on success. On failure, ErrorMsg is updated with<br>
+ /// descriptive text of the encountered error.<br>
+ virtual bool loadData(uint64_t Address, const void *Data, size_t Size);<br>
+<br>
+ /// Load code into the target address space and prepare it for execution.<br>
+ ///<br>
+ /// @param Address Destination address in the target process.<br>
+ /// @param Data Source address in the host process.<br>
+ /// @param Size Number of bytes to copy.<br>
+ ///<br>
+ /// @returns False on success. On failure, ErrorMsg is updated with<br>
+ /// descriptive text of the encountered error.<br>
+ virtual bool loadCode(uint64_t Address, const void *Data, size_t Size);<br>
+<br>
+ /// Execute code in the target process. The called function is required<br>
+ /// to be of signature int "(*)(void)".<br>
+ ///<br>
+ /// @param Address Address of the loaded function in the target<br>
+ /// process.<br>
+ /// @param[out] RetVal The integer return value of the called function.<br>
+ ///<br>
+ /// @returns False on success. On failure, ErrorMsg is updated with<br>
+ /// descriptive text of the encountered error.<br>
+ virtual bool executeCode(uint64_t Address, int &RetVal);<br>
+<br>
+ /// Minimum alignment for memory permissions. Used to seperate code and<br>
+ /// data regions to make sure data doesn't get marked as code or vice<br>
+ /// versa.<br>
+ ///<br>
+ /// @returns Page alignment return value. Default of 4k.<br>
+ virtual unsigned getPageAlignment() { return 4096; }<br>
+<br>
+ /// Start the remote process.<br>
+ virtual void create();<br>
+<br>
+ /// Terminate the remote process.<br>
+ virtual void stop();<br>
+<br>
+ RemoteTargetExternal(std::string &Name) : RemoteTarget(), ChildName(Name) {}<br>
+ virtual ~RemoteTargetExternal() {}<br>
+<br>
+private:<br>
+ std::string ChildName;<br>
+<br>
+ // This will get filled in as a point to an OS-specific structure.<br>
+ void *ConnectionData;<br>
+<br>
+ void SendAllocateSpace(uint32_t Alignment, uint32_t Size);<br>
+ void SendLoadSection(uint64_t Addr,<br>
+ const void *Data,<br>
+ uint32_t Size,<br>
+ bool IsCode);<br>
+ void SendExecute(uint64_t Addr);<br>
+ void SendTerminate();<br>
+<br>
+ void Receive(LLIMessageType Msg);<br>
+ void Receive(LLIMessageType Msg, int &Data);<br>
+ void Receive(LLIMessageType Msg, uint64_t &Data);<br>
+<br>
+ int WriteBytes(const void *Data, size_t Size);<br>
+ int ReadBytes(void *Data, size_t Size);<br>
+ void Wait();<br>
+};<br>
+<br>
+} // end namespace llvm<br>
+<br>
+#endif // LLI_REMOTETARGETEXTERNAL_H<br>
<br>
Added: llvm/trunk/tools/lli/RemoteTargetMessage.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/RemoteTargetMessage.h?rev=191843&view=auto" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/RemoteTargetMessage.h?rev=191843&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/tools/lli/RemoteTargetMessage.h (added)<br>
+++ llvm/trunk/tools/lli/RemoteTargetMessage.h Wed Oct 2 12:12:36 2013<br>
@@ -0,0 +1,45 @@<br>
+//===---- RemoteTargetMessage.h - LLI out-of-process message protocol -----===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+//<br>
+// Definition of the LLIMessageType enum which is used for communication with a<br>
+// child process for remote execution.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+#ifndef LLI_REMOTETARGETMESSAGE_H<br>
+#define LLI_REMOTETARGETMESSAGE_H<br>
+<br>
+namespace llvm {<br>
+<br>
+// LLI messages from parent-to-child or vice versa follow an exceedingly simple<br>
+// protocol where the first four bytes represent the message type, the next<br>
+// four bytes represent the size of data for the command and following bytes<br>
+// represent the actual data.<br>
+//<br>
+// The protocol is not intended to be robust, secure or fault-tolerant. It is<br>
+// only here for testing purposes and is therefore intended to be the simplest<br>
+// implementation that will work. It is assumed that the parent and child<br>
+// share characteristics like endianness.<br>
+<br>
+enum LLIMessageType {<br>
+ LLI_Error = -1,<br>
+ LLI_ChildActive = 0, // Data = not used<br>
+ LLI_AllocateSpace, // Data = struct { uint_32t Align, uint_32t Size }<br>
+ LLI_AllocationResult, // Data = uint64_t AllocAddress (in Child memory space)<br>
+ LLI_LoadCodeSection, // Data = uint32_t Addr, followed by section contests<br>
+ LLI_LoadDataSection, // Data = uint32_t Addr, followed by section contents<br>
+ LLI_LoadComplete, // Data = not used<br>
+ LLI_Execute, // Data = Address of function to execute<br>
+ LLI_ExecutionResult, // Data = uint64_t Result<br>
+ LLI_Terminate // Data = not used<br>
+};<br>
+<br>
+} // end namespace llvm<br>
+<br>
+#endif<br>
<br>
Added: llvm/trunk/tools/lli/Unix/RemoteTargetExternal.inc<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/Unix/RemoteTargetExternal.inc?rev=191843&view=auto" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/Unix/RemoteTargetExternal.inc?rev=191843&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/tools/lli/Unix/RemoteTargetExternal.inc (added)<br>
+++ llvm/trunk/tools/lli/Unix/RemoteTargetExternal.inc Wed Oct 2 12:12:36 2013<br>
@@ -0,0 +1,91 @@<br>
+//=- RemoteTargetExternal.inc - LLVM out-of-process JIT execution for Unix --=//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+//<br>
+// Implementation of the Unix-specific parts of the RemoteTargetExternal class<br>
+// which executes JITed code in a separate process from where it was built.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+#include <unistd.h><br>
+#include <stdio.h><br>
+#include <stdlib.h><br>
+#include <sys/wait.h><br>
+<br>
+namespace {<br>
+<br>
+struct ConnectionData_t {<br>
+ int InputPipe;<br>
+ int OutputPipe;<br>
+<br>
+ ConnectionData_t(int in, int out) : InputPipe(in), OutputPipe(out) {}<br>
+};<br>
+<br>
+} // namespace<br>
+<br>
+namespace llvm {<br>
+<br>
+void RemoteTargetExternal::create() {<br>
+ int PipeFD[2][2];<br>
+ pid_t ChildPID;<br>
+<br>
+ pipe(PipeFD[0]);<br>
+ pipe(PipeFD[1]);<br>
+<br>
+ ChildPID = fork();<br>
+<br>
+ if (ChildPID == 0) {<br>
+ // In the child...<br>
+<br>
+ // Close the parent ends of the pipes<br>
+ close(PipeFD[0][1]);<br>
+ close(PipeFD[1][0]);<br>
+<br>
+ // Use our pipes as stdin and stdout<br>
+ if (PipeFD[0][0] != STDIN_FILENO) {<br>
+ dup2(PipeFD[0][0], STDIN_FILENO);<br>
+ close(PipeFD[0][0]);<br>
+ }<br>
+ if (PipeFD[1][1] != STDOUT_FILENO) {<br>
+ dup2(PipeFD[1][1], STDOUT_FILENO);<br>
+ close(PipeFD[1][1]);<br>
+ }<br>
+<br>
+ // Execute the child process.<br>
+ char *args[1] = { NULL };<br>
+ int rc = execv(ChildName.c_str(), args);<br>
+ if (rc != 0)<br>
+ perror("Error executing child process: ");<br>
+ }<br>
+ else {<br>
+ // In the parent...<br>
+<br>
+ // Close the child ends of the pipes<br>
+ close(PipeFD[0][0]);<br>
+ close(PipeFD[1][1]);<br>
+<br>
+ // Store the parent ends of the pipes<br>
+ ConnectionData = (void*)new ConnectionData_t(PipeFD[1][0], PipeFD[0][1]);<br>
+<br>
+ Receive(LLI_ChildActive);<br>
+ }<br>
+}<br>
+<br>
+int RemoteTargetExternal::WriteBytes(const void *Data, size_t Size) {<br>
+ return write(((ConnectionData_t*)ConnectionData)->OutputPipe, Data, Size);<br>
+}<br>
+<br>
+int RemoteTargetExternal::ReadBytes(void *Data, size_t Size) {<br>
+ return read(((ConnectionData_t*)ConnectionData)->InputPipe, Data, Size);<br>
+}<br>
+<br>
+void RemoteTargetExternal::Wait() {<br>
+ wait(NULL);<br>
+}<br>
+<br>
+} // namespace llvm<br>
\ No newline at end of file<br>
<br>
Added: llvm/trunk/tools/lli/Windows/RemoteTargetExternal.inc<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/Windows/RemoteTargetExternal.inc?rev=191843&view=auto" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/Windows/RemoteTargetExternal.inc?rev=191843&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/tools/lli/Windows/RemoteTargetExternal.inc (added)<br>
+++ llvm/trunk/tools/lli/Windows/RemoteTargetExternal.inc Wed Oct 2 12:12:36 2013<br>
@@ -0,0 +1,32 @@<br>
+//= RemoteTargetExternal.inc - LLVM out-of-process JIT execution for Windows =//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+//<br>
+// Definition of the Windows-specific parts of the RemoteTargetExternal class<br>
+// which is meant to execute JITed code in a separate process from where it was<br>
+// built. To support this functionality on Windows, implement these functions.<br>
+//<br>
+//===----------------------------------------------------------------------===//<br>
+<br>
+namespace llvm {<br>
+<br>
+void RemoteTargetExternal::create() {<br>
+}<br>
+<br>
+int RemoteTargetExternal::WriteBytes(const void *Data, size_t Size) {<br>
+ return 0;<br>
+}<br>
+<br>
+int RemoteTargetExternal::ReadBytes(void *Data, size_t Size) {<br>
+ return 0;<br>
+}<br>
+<br>
+void RemoteTargetExternal::Wait() {<br>
+}<br>
+<br>
+} // namespace llvm<br>
<br>
Modified: llvm/trunk/tools/lli/lli.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/lli.cpp?rev=191843&r1=191842&r2=191843&view=diff" target="_blank">
http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/lli/lli.cpp?rev=191843&r1=191842&r2=191843&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/tools/lli/lli.cpp (original)<br>
+++ llvm/trunk/tools/lli/lli.cpp Wed Oct 2 12:12:36 2013<br>
@@ -41,6 +41,7 @@<br>
#include "llvm/Support/PluginLoader.h"<br>
#include "llvm/Support/PrettyStackTrace.h"<br>
#include "llvm/Support/Process.h"<br>
+#include "llvm/Support/Program.h"<br>
#include "llvm/Support/Signals.h"<br>
#include "llvm/Support/SourceMgr.h"<br>
#include "llvm/Support/TargetSelect.h"<br>
@@ -83,6 +84,18 @@ namespace {<br>
cl::desc("Execute MCJIT'ed code in a separate process."),<br>
cl::init(false));<br>
<br>
+ // Manually specify the child process for remote execution. This overrides<br>
+ // the simulated remote execution that allocates address space for child<br>
+ // execution. The child process resides in the disk and communicates with lli<br>
+ // via stdin/stdout pipes.<br>
+ cl::opt<std::string><br>
+ MCJITRemoteProcess("mcjit-remote-process",<br>
+ cl::desc("Specify the filename of the process to launch "<br>
+ "for remote MCJIT execution. If none is specified,"<br>
+ "\n\tremote execution will be simulated in-process."),<br>
+ cl::value_desc("filename"),<br>
+ cl::init(""));<br>
+<br>
// Determine optimization level.<br>
cl::opt<char><br>
OptLevel("O",<br>
@@ -481,30 +494,50 @@ int main(int argc, char **argv, char * c<br>
// Everything is prepared now, so lay out our program for the target<br>
// address space, assign the section addresses to resolve any relocations,<br>
// and send it to the target.<br>
- RemoteTarget Target;<br>
- Target.create();<br>
+<br>
+ OwningPtr<RemoteTarget> Target;<br>
+ if (!MCJITRemoteProcess.empty()) { // Remote execution on a child process<br>
+ if (!RemoteTarget::hostSupportsExternalRemoteTarget()) {<br>
+ errs() << "Warning: host does not support external remote targets.\n"<br>
+ << " Defaulting to simulated remote execution\n";<br>
+ Target.reset(RemoteTarget::createRemoteTarget());<br>
+ } else {<br>
+ std::string ChildEXE = sys::FindProgramByName(MCJITRemoteProcess);<br>
+ if (ChildEXE == "") {<br>
+ errs() << "Unable to find child target: '\''" << MCJITRemoteProcess << "\'\n";<br>
+ return -1;<br>
+ }<br>
+ Target.reset(RemoteTarget::createExternalRemoteTarget(MCJITRemoteProcess));<br>
+ }<br>
+ } else {<br>
+ // No child process name provided, use simulated remote execution.<br>
+ Target.reset(RemoteTarget::createRemoteTarget());<br>
+ }<br>
+<br>
+ // Create the remote target<br>
+ Target->create();<br>
<br>
// Trigger compilation.<br>
EE->generateCodeForModule(Mod);<br>
<br>
// Layout the target memory.<br>
- layoutRemoteTargetMemory(&Target, MM);<br>
+ layoutRemoteTargetMemory(Target.get(), MM);<br>
<br>
// Since we're executing in a (at least simulated) remote address space,<br>
// we can't use the ExecutionEngine::runFunctionAsMain(). We have to<br>
// grab the function address directly here and tell the remote target<br>
// to execute the function.<br>
// FIXME: argv and envp handling.<br>
- uint64_t Entry = (uint64_t)EE->getPointerToFunction(EntryFn);<br>
+ uint64_t Entry = EE->getFunctionAddress(EntryFn->getName().str());<br>
<br>
DEBUG(dbgs() << "Executing '" << EntryFn->getName() << "' at 0x"<br>
<< format("%llx", Entry) << "\n");<br>
<br>
- if (Target.executeCode(Entry, Result))<br>
- errs() << "ERROR: " << Target.getErrorMsg() << "\n";<br>
+ if (Target->executeCode(Entry, Result))<br>
+ errs() << "ERROR: " << Target->getErrorMsg() << "\n";<br>
<br>
- Target.stop();<br>
- } else {<br>
+ Target->stop();<br>
+ } else { // !RemoteMCJIT<br>
// Trigger compilation separately so code regions that need to be<br>
// invalidated will be known.<br>
(void)EE->getPointerToFunction(EntryFn);<br>
<br>
<br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@cs.uiuc.edu">llvm-commits@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits</a><o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>