[LLVMdev] MemoryBuffer

[Gordon Henriksen]

NUL-termination is an optimization, in particular for clang's parser,  
where it's cheaper to check for '\0' to signal EOF than it is to  
compare pointers. Its presence doesn't move the end iterator (just  
makes it dereference-able) and in no way impacts your ability to use  
iterator pairs to delimit the binary content of the file. This feature  
does require file contents be copied into local memory (rather than  
mapped in) if the file is exactly a multiple of the system page size.

On 2009-09-24, at 16:44, OvermindDL1 wrote:

> On Thu, Sep 24, 2009 at 2:32 PM, Chris Lattner <clattner at apple.com>  
> wrote:
>>
>> On Sep 24, 2009, at 1:23 PM, OvermindDL1 wrote:
>>
>>> I was writing something using MemoryBuffer, and while looking  
>>> through
>>> its code I came across line 59:
>>> assert(BufEnd[0] == 0 && "Buffer is not null terminated!");
>>> I am curious if the MemoryBuffer only supports non-binary, non-null
>>> embedded files, or if it supports binary as well.  I do not see
>>> anything inherently not expecting binary files except for that one
>>> line, so I am curious as to the purpose of that assert?
>>
>> Your question is difficult to understand.  MemoryBuffer doesn't  
>> care what is
>> in the file.  It works fine if you have embedded nul's in the  
>> file.  It just
>> requires the mapped image to have a nul at the end.
>
> I am mostly just curious as to why it requires a nul at the end, seems
> odd when specifying two pointers for a begin/end style interface.  I
> am mostly just worried because a library I use returns such iterators
> that work in Memory Buffer (they are const char*'s as well), but the
> end pointer is one-past-the-end, and may point into invalid memory
> (say if it was at the end of a virtual page), thus could cause an
> access violation.  To prevent that I would need to copy it, and with a
> sizable chunk of memory that it may be, can cause the program to
> freeze for a sec.  It would also be nice to not require that as you
> could create a MemoryBuffer that references some *part* of an existing
> MemoryBuffer that does not have a nul as the defreferenced end pointer
> that you are giving in it.  Mostly I just do not understand why this
> restriction exists, so I am just looking for comprehension.