[LLVMdev] Question about SimplifyXorInst

[John Criswell]

On 7/20/11 8:24 AM, Xueying ZHANG wrote:
> Hi all,
>
> I am master student in Edinburgh, UK. I am doing my MSc project with
> LLVM compiler and I have to modify LLVM to implement the StackGuard
> with a XOR random Canary. However, I am not familiar with LLVM.

This sounds similar to the StackProtector feature that LLVM already has.
>
> My problem is that I want to XOR the random canary word with the
> return address which are both 32 bits. I found a method called
> SimplifyXorInst(Value *, Value *, const TargetData *,const
> DominatorTree *, unsigned) but I do not know whether it is right
> method and how to use it. Could you give me some advice or sample code
> about it?

I have no idea what SimplifyXorInst() does, but since the address of the 
return address is not visible at the LLVM IR level (to the best of my 
knowledge), it is probably not what you want.

The most straightforward thing to do, I think, is to write a 
MachineFunctionPass that adds prologue/epilogue code during code 
generation to do the xor'ing of the return address.

I haven't written a MachineFunctionPass, so I can't really help with any 
details.  My best suggestion is to find the StackProtector code in LLVM 
and see how it works.  The doxygen docs are also valuable 
(http://llvm.org/doxygen/hierarchy.html).

-- John T.
>
> Thank you
> Best wishes,
> Ying
>