[LLVMdev] load bytecode from string for jiting problem
Willy WOLFF
willy.wolff at etu.unistra.fr
Wed Mar 19 14:58:47 PDT 2014
all of:
----
// cout << "lsr: " << lsr << "\n";
llvm::MemoryBuffer* mbjit =
llvm::MemoryBuffer::getMemBufferCopy (sr);
------
string lsr = sr.str();
// cout << "lsr: " << lsr << "\n";
llvm::MemoryBuffer* mbjit =
llvm::MemoryBuffer::getMemBuffer (lsr);
-------
string lsr = sr.str();
// cout << "lsr: " << lsr << "\n";
llvm::MemoryBuffer* mbjit =
llvm::MemoryBuffer::getMemBufferCopy (lsr);
have same result as invalid bit code.
The result of valgrind, effectively, i have invalid reads in the parseBitcodeFile:
==536== Conditional jump or move depends on uninitialised value(s)
==536== at 0x501FE3: llvm::BitstreamCursor::Read(unsigned int) (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536== by 0x501A19: llvm::BitcodeReader::ParseBitcodeInto(llvm::Module*) (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536== by 0x50AEC8: llvm::getLazyBitcodeModule(llvm::MemoryBuffer*, llvm::LLVMContext&) (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536== by 0x50B295: llvm::parseBitcodeFile(llvm::MemoryBuffer*, llvm::LLVMContext&) (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536== by 0x4F1231: blah_runtime_hook (runtime.cpp:348)
==536== by 0x4F46C2: ??? (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536== by 0x4F2B60: main (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536==
==536== Invalid read of size 8
==536== at 0x501FE8: llvm::BitstreamCursor::Read(unsigned int) (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536== by 0x501A19: llvm::BitcodeReader::ParseBitcodeInto(llvm::Module*) (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536== by 0x50AEC8: llvm::getLazyBitcodeModule(llvm::MemoryBuffer*, llvm::LLVMContext&) (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536== by 0x50B295: llvm::parseBitcodeFile(llvm::MemoryBuffer*, llvm::LLVMContext&) (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536== by 0x4F1231: blah_runtime_hook (runtime.cpp:348)
==536== by 0x4F46C2: ??? (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536== by 0x4F2B60: main (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536== Address 0x0 is not stack'd, malloc'd or (recently) free'd
==536==
==536==
==536== Process terminating with default action of signal 11 (SIGSEGV)
==536== Access not within mapped region at address 0x0
==536== at 0x501FE8: llvm::BitstreamCursor::Read(unsigned int) (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536== by 0x501A19: llvm::BitcodeReader::ParseBitcodeInto(llvm::Module*) (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536== by 0x50AEC8: llvm::getLazyBitcodeModule(llvm::MemoryBuffer*, llvm::LLVMContext&) (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536== by 0x50B295: llvm::parseBitcodeFile(llvm::MemoryBuffer*, llvm::LLVMContext&) (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536== by 0x4F1231: blah_runtime_hook (runtime.cpp:348)
==536== by 0x4F46C2: ??? (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
==536== by 0x4F2B60: main (in /home/willy/blah_test_script/new_blah/simple_scev_dynamic_array)
--
Willy WOLFF
On 19 Mar 2014, at 22:11, Vikas Bhargava wrote:
> Hi Willy,
> If the disassembly of the module works fine, then there is nothing wrong with the module.
>
> Stream uses the memorybuffer that you pass in parseBitcodeFile. If what Will is saying is true, there is something wrong with your code in "3:", i.e.:
>
> MemoryBuffer* mbjit = MemoryBuffer::getMemBuffer (sr.str());
> LLVMContext& context = getGlobalContext();
> ErrorOr<Module*> ModuleOrErr = parseBitcodeFile (mbjit, context);
> if (error_code EC = ModuleOrErr.getError())
> {
> std::cout << ModuleOrErr.getError().
> message() << "\n";
> assert(false);
> }
>
> Can you post how you modified it in your second reply? For debugging purpose, you can simply use MemoryBuffer::getMemBufferCopy() and not worry about validity of stringref or null-termination. Also, you can run your program through valgrind and check for any invalid reads.
>
> HTH
> Vikas.
> =======
>
>
>
> On Wed, Mar 19, 2014 at 10:32 AM, Willy WOLFF <willy.wolff at etu.unistra.fr> wrote:
> I mad the change, and still have the problem.
>
> I investigate more the source code of llvm.
>
> First, I change isRawBitcode function to print the content of the parameter like this:
> original: http://llvm.org/docs/doxygen/html/ReaderWriter_8h_source.html#l00081
>
> inline bool isRawBitcode(const unsigned char *BufPtr,
> const unsigned char *BufEnd) {
> // These bytes sort of have a hidden message, but it's not in
> // little-endian this time, and it's a little redundant.
> errs()<< "isRawBitcode output:\n";
> for (int i = 0; i < 4; i++)
> errs() << BufPtr[i] << "\n";
> if (BufPtr != BufEnd )
> errs() << "BP != BE ok\n";
> if (BufPtr[0] == 'B')
> errs() << "B ok\n";
> if (BufPtr[1] == 'C')
> errs() << "C ok\n";
> if (BufPtr[2] == 0xc0)
> errs() << "0xc0 ok\n";
> if (BufPtr[3] == 0xde)
> errs() << "0xde ok\n";
>
> return BufPtr != BufEnd &&
> BufPtr[0] == 'B' &&
> BufPtr[1] == 'C' &&
> BufPtr[2] == 0xc0 &&
> BufPtr[3] == 0xde;
> }
>
>
> Second, I change ParseBitcodeInto as this:
> original: http://llvm.org/docs/doxygen/html/BitcodeReader_8cpp_source.html#l01971
> ...
> errs() << "parsebitcodeinto sniff the signature\n";
> uint32_t bvar = Stream.Read(8);
> errs() << "B :" << bvar << "\n";
> if (bvar != 'B') {
> errs() << "B :" << bvar << "\n";
> return Error(InvalidBitcodeSignature);
> }
>
> if (Stream.Read(8) != 'C') {
> errs() << "C\n";
> return Error(InvalidBitcodeSignature);
> }
> if ( Stream.Read(8) != 0xc0 ) {
> errs() << "0xc0\n";
> return Error(InvalidBitcodeSignature);
> }
> if ( Stream.Read(8) != 0xde ) {
> errs() << "0xde\n";
> return Error(InvalidBitcodeSignature);
> }
> // if (Stream.Read(8) != 'B' ||
> // Stream.Read(8) != 'C' ||
> // Stream.Read(4) != 0x0 ||
> // Stream.Read(4) != 0xC ||
> // Stream.Read(4) != 0xE ||
> // Stream.Read(4) != 0xD
> // ) {
> ...
>
>
>
> The output of the code is :
>
>
> isRawBitcode output:
> B
> C
>
>
> BP != BE ok
>
> B ok
> C ok
> 0xc0 ok
> 0xde ok
>
> parsebitcodeinto sniff the signature
> B :37
> B :37
>
>
>
>
> It's possible that Stream object is not correctly initialized?
>
>
> On 03/13/2014 06:37 PM, Will Dietz wrote:
> On Thu, Mar 13, 2014 at 9:02 AM, Willy WOLFF <willy.wolff at etu.unistra.fr> wrote:
> Hello,
>
> I having a weird problem while writing a bytecode module to a string,
> and after read/parse it for unsing on a jit.
>
> I write a pass to export function to module, and put this module inside
> a global variable.
> I use WriteBitcodeToFile for this.
> For debuging, after this write, I try to load the exported module with
> parseBitcodeFile.
> This two step works.
>
>
>
> After, while the compiled program is running, I try to read and parse
> this global variable for jiting the function.
>
> 1) I read the global variable with
> StringRef sr (gv, gv_length);
>
> 2) I manually test this bytecode by
> (inspired by inline bool isRawBitcode(const unsigned char *BufPtr,
> const unsigned char *BufEnd) at
> http://llvm.org/docs/doxygen/html/ReaderWriter_8h_source.html#l00067)
> if (sr.str()[0] == 'B')
> std::cout << "B ok\n";
> if (sr.str()[1] == 'C')
> std::cout << "C ok\n";
> if (sr.str()[2] == (char) 0xc0)
> std::cout << "0xc0 ok\n";
> if (sr.str()[3] == (char) 0xde)
> std::cout << "0xde ok\n";
>
> 3) I try to parse the gv by
> MemoryBuffer* mbjit = MemoryBuffer::getMemBuffer (sr.str());
>
> Not sure if this is your issue, but should be fixed anyway:
>
> The std::string created by "sr.str()" ends its lifetime in this
> statement, and MemoryBuffer for efficiency reasons
> avoids copying data it doesn't have to (like StringRef) so will be
> referencing the freed memory.
>
> To resolve this:
> * Pass MemoryBuffer your StringRef directly
> * Use getMemBufferCopy()
> * Preserve the result of sr.str() into a stack variable and pass that
> to getMemoryBuffer() instead.
>
> As a final note, check if your bitcode buffer "string" is
> null-terminated or not. If not, be sure to be careful and
> do things like informing MemoryBuffer that this is the case.
>
> Hope this helps,
> ~Will
>
> LLVMContext& context = getGlobalContext();
> ErrorOr<Module*> ModuleOrErr = parseBitcodeFile (mbjit, context);
> if (error_code EC = ModuleOrErr.getError())
> {
> std::cout << ModuleOrErr.getError().message() << "\n";
> assert(false);
> }
>
>
>
>
> This is the execution result:
> B ok
> C ok
> 0xc0 ok
> 0xde ok
> Invalid bitcode signature
>
>
>
> Ok is not working :/
> But why ???
>
>
>
> For debuging, between 2) and 3), I export the readed module and write to
> a file on my hard drive,
> and try llvm-dis, and the dissasembly of the module works.
>
> Wath's wrong? Any idea for solve this problem?
>
> Thanks you very much.
>
> Regards,
> Willy
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140319/eef13914/attachment.html>
More information about the llvm-dev
mailing list