[flang-dev] RFC: Porting SourceFile to using LLVM file handling facilities

[David Truby via flang-dev]

Hi Steve,

After a bit more research I discovered that llvm has a WriteableMemoryBuffer class that effectively does exactly what we are already doing (with a more C++y and cross platform interface) and with which we can still perform the newline normalisation. 
I propose that in the interests of getting things moving faster, we set aside whether the newline normalisation is necessary or not and just replicate exactly the current behaviour with this mechanism. I’m happy to work on that.

Does that sound like a good way forward on this?

Thanks
David Truby

> On 20 Feb 2020, at 07:53, Steve Scalpone <sscalpone at nvidia.com> wrote:
> 
> Hi David, 
> 
> Normalizing the line terminator simplifies character processing for low cost.  After consulting with the author, I can't say how difficult it would be to add generalized handling of line terminators afterwards as we abandoned that approach early on.  Note there are other line terminators in addition LF and CR LF, although they are becoming more rare these days.
> 
> - Steve
> 
> On 2/19/20, 6:32 AM, "flang-dev on behalf of David Truby via flang-dev" <flang-dev-bounces at lists.llvm.org on behalf of flang-dev at lists.llvm.org> wrote:    
> 
>    Hi All,
> 
>    Does anyone have any insight into this? I’m wondering if it is safe to rework this section of the code and remove the carriage return rewriting, but I don’t want to start until I understand why it’s there to start with.
> 
>    Thanks
>    David Truby
> 
>> On 29 Jan 2020, at 16:03, David Truby via flang-dev <flang-dev at lists.llvm.org> wrote:
>> 
>> Hi All,
>> 
>> I’ve been looking at SourceFile as the main user of posix file handling functions and how this could be ported to use LLVM’s file handling facilities. I noticed that we are mmapping for sufficiently large files and just reading small ones, which seems like a good fit for the llvm::MemoryBuffer class (which does the same thing).
>> Does this seem like a sensible route to take to anyone that might be more familiar with the code than me?
>> 
>> One thing that I noticed however is that the input file is being rewritten (in memory) to remove carriage returns. Does someone more familiar with the preprocessor or parser than me know if this is because we can’t handle carriage returns later? If so, how difficult would it to be to add handling for them (just treating them as whitespace seems appropriate)?
>> I think we should avoid rewriting the input file if possible as it requires reading the whole thing one more time than is strictly necessary.
>> 
>> Thanks
>> David Truby
>> _______________________________________________
>> flang-dev mailing list
>> flang-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/flang-dev
> 
>    _______________________________________________
>    flang-dev mailing list
>    flang-dev at lists.llvm.org
>    https://lists.llvm.org/cgi-bin/mailman/listinfo/flang-dev
> 
> 
> 
> -----------------------------------------------------------------------------------
> This email message is for the sole use of the intended recipient(s) and may contain
> confidential information.  Any unauthorized review, use, disclosure or distribution
> is prohibited.  If you are not the intended recipient, please contact the sender by
> reply email and destroy all copies of the original message.
> -----------------------------------------------------------------------------------