[llvm-dev] adding prefixes to certain instructions x86 -- where to start?

[carr27 via llvm-dev]

I'm trying to make a security sandbox.  For example, lets say my program 
has a LoadInst in the LLVM IR and I know I want to confine the address 
range this LoadInst is accessing.  Maybe that LoadInst gets emitted as a 
MOV machine code instruction by the backend. During execution an 
attacker could potentially control the operands of the MOV instruction 
through some exploit, but usually he cannot modify the 
instructions/prefixes because the code is not writable. So the prefix 
can potentially let me confine the attacker to an address range even if 
he controls the instruction operands.

I hope that makes some sense.  If someone knows of a different approach 
-- a very light weight sandbox implemented in LLVM I'd be interested 
looking into it.

Thanks,
Scott

On 09/11/2015 10:55 AM, mats petersson wrote:
> What is it you are ACTUALLY trying to do?
>
> In other words, why would you want a different address size... 
> Understanding that would probably help provide a better answer (I have 
> absolutely no idea how to solve the actual question, but I suspect 
> understanding what the overall goal is will help a whole lot)
>
> --
> Mats
>
> On 11 September 2015 at 15:38, Scott A. Carr via llvm-dev 
> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>
>     Hello,
>
>     I’ve been using LLVM IR passes for my research for about a year
>     now, but for my next step I think I might have to dig into a
>     backend.  I'm hoping someone could give me a pointer on how to get
>     started.
>
>     The thing I would like to do is add and override address-size
>     override prefix [1] to a given x86-64 instruction.  I’m hoping I
>     can do something like:
>
>     1) Mark some IR instructions with metadata in my pass
>     2) Hack the backend to look for my metadata, and if found add the
>     prefix when the machine instruction is emitted
>
>     Does this seem feasible?  Does the LLVM x86 backend currently have
>     the capability of adding instruction prefixes and could someone
>     please point out where I should look in the code for it?
>
>     Thanks,
>     Scott A. Carr
>     PhD Student
>     Purdue University CS
>
>     [1]
>     http://wiki.osdev.org/X86-64_Instruction_Encoding#Operand-size_and_address-size_override_prefix
>     _______________________________________________
>     LLVM Developers mailing list
>     llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>     http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150911/e0366b71/attachment.html>