[llvm-dev] Soundness in clang SA

Sat Aug 18 06:19:12 PDT 2018

Of course you can make any static analysis reports just some very specific
bug(s) that are already known, so the answer to your question is yes. But that's
probably not what you were asking about.

If you want the analysis to only report genuine bugs, you
may want to consider using fuzzers instead, like AFL or libFuzzer.
They are not guaranteed not to report false positives, as the environment
under which the subject program is fuzzed may be different from the actual
deployment environment, but at least they may be closer to what you
have in mind.

Best,
Andrew

On Friday, 17 August 2018, 11:50:18 PM GMT+8, via llvm-dev <llvm-dev at lists.llvm.org> wrote: 

Message: 3
Date: Fri, 17 Aug 2018 17:05:19 +0530
From: Siddharth Shankar Swain via llvm-dev <llvm-dev at lists.llvm.org>
To: llvm-dev <llvm-dev at lists.llvm.org>, Clang Dev
    <cfe-dev at lists.llvm.org>
Subject: [llvm-dev] Soundness in clang SA
Message-ID:
    <CAMkbrzJ40jL4aJNB_z5BjOeS=0KBQTmA-0_5Z-Jqp8VEH_HeTg at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi all,

Is it possible to develop a checker  or some feature in clang SA which will
only have perfect soundness property ( if we don't care about completness
property )  i.e if the analyzer says X is a genuine bug then X is really a
genuine bug. Whatever  bug  it reports are all genuine but it doesn't
report all genuine bugs. Please guide.

Thanks,
Siddharth
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180817/90524fad/attachment-0001.html>

------------------------------

Message: 4
Date: Fri, 17 Aug 2018 13:46:24 +0000
From: Troy Johnson via llvm-dev <llvm-dev at lists.llvm.org>
To: Hal Finkel <hfinkel at anl.gov>, Bekket McClane
    <bekket.mcclane at gmail.com>
Cc: "llvm-dev at lists.llvm.org" <llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] alias.scope and local restricted C pointers
Message-ID:
    <CY4PR11MB1397062B18744566C67323CAC63D0 at CY4PR11MB1397.namprd11.prod.outlook.com>

Content-Type: text/plain; charset="windows-1252"

Thanks to both of you for clarifying.

Bekket, local restrict is pretty common in my experience, at least with our users.  It would be good for it to work as expected.

Hal, I appreciate the difficulty in doing the based-on analysis, but it needs to be understood that a C compiler was never really expected to do that analysis!  Being able to do it would be great, of course, but it is quite daunting and was not the original intent of restrict.  I have in fact worked with the original author of the restrict keyword and proposed WG14/N2260 to clarify its usage and intent.

Basically, the based-on terminology was the least bad way anyone thought of at the time to describe the semantics.  The belief was that users would supply restrict on multiple pointers of the same type whenever none of them aliased, then use those pointers directly.  Doing the based-on analysis is similar to what's required to usefully interpret a single restrict-qualified pointer in isolation, and that's considered unreasonably hard.

So I think this problem is quite solvable in the common (and recommended) case of the user specifying restrict liberally and accessing data directly via those pointers.

The use case that I normally see is a bunch of local restricted pointers initialized once and indexed by some increasing integer.  The compiler would honor restrict if the user outlined the code into a separate function and the pointers were parameters instead.  Sometimes that is the recommended workaround, but it shouldn't be necessary.

I'm happy to discuss futher with whomever works on the related parts of Clang and LLVM.  I am not familiar with who that might be.

-Troy
________________________________
From: Hal Finkel <hfinkel at anl.gov>
Sent: Thursday, August 16, 2018 10:31:52 PM
To: Bekket McClane; Troy Johnson
Cc: llvm-dev at lists.llvm.org
Subject: Re: [llvm-dev] alias.scope and local restricted C pointers

On 08/16/2018 07:52 PM, Bekket McClane via llvm-dev wrote:

On Aug 16, 2018, at 4:41 PM, Troy Johnson via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:

Concerning slide 16 of https://llvm.org/devmtg/2017-02-04/Restrict-Qualified-Pointers-in-LLVM.pdf

Specifically “Currently, LLVM only supports restrict on function arguments, although we have a way to preserve that information if the function is inlined.”

Is that statement still accurate?

Yes, correct (actually I was just working on restrict, no_alias and alias.scope attributes). The inliner also propagates them correctly.

It would seem that https://llvm.org/docs/LangRef.html#noalias-and-alias-scope-metadata should be sufficiently general to honor C’s restrict qualifier on local pointers,
but it does not appear that Clang uses this part of LLVM’s IR for that purpose today and thus local restricts are ignored.

I think that’s correct, but I haven’t come out with any scenarios regarding local variables/memory that can _not_ be solved by AA. As BasicAA is able to solved most of the local cases, including malloc and some memory intrinsics.

int *restrict x = some_external_function();
int *restrict y = some_other_external_function();

This is one of the fundamental use cases for restrict and BasicAA has nothing to offer in this regard. In other words, it's a mechanism for encoding an interface contract.

-Hal

Best
Bekket

Thanks,
Troy

_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

--
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180817/add2df4f/attachment-0001.html>

------------------------------

Message: 5
Date: Fri, 17 Aug 2018 15:15:04 +0100
From: Jeremy Morse via llvm-dev <llvm-dev at lists.llvm.org>
To: Guang Chen <cgcgbcbc at 163.com>
Cc: llvm-dev at lists.llvm.org
Subject: Re: [llvm-dev] Use LLVM C API for reading metadata
Message-ID:
    <CAB+eix6Cqr9oL4cQCGiD52fW_+ec5AM4O2=i9r=+vqEtyky6CQ at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Guang,

The upcoming 7.0 release has more debug-info exposed by the C API,
including reading line and column information from DILocation's, as seen
here [0].

As for determining types, I believe all debug metadata attached directly to
instructions are DILocations, but navigating further into the debug info
structures isn't directly exposed by the C API.

[0]
https://github.com/llvm-mirror/llvm/blob/1d13545e10c5574a40821115d10c54c16f7c47be/include/llvm-c/DebugInfo.h#L396

--

Thanks,

Jeremy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180817/bad06925/attachment-0001.html>

------------------------------

Message: 6
Date: Fri, 17 Aug 2018 10:30:47 -0500
From: Hal Finkel via llvm-dev <llvm-dev at lists.llvm.org>
To: LLVM Dev <llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] local restrict - again
Message-ID: <aa45a162-a2de-d446-c5cc-073038f16c0b at anl.gov>
Content-Type: text/plain; charset="utf-8"

On 06/28/2018 10:59 AM, Hal Finkel wrote:
> Hi, Jeroen,
>
> We should move these conversations to llvm-dev so that they don't get
> missed and others can contribute. Can I cc the list?

[+llvm-dev] - Jeroen consented.

>
> Doing this will mean that the logic for when to remove dead llvm.noalias
> intrinsics will become more complicated. That might be worthwhile. So
> long as we can answer the basic question:
>
>   Given two memory accesses with the same noalias scope metadata, is one
> pointer derived from a noalias intrinic with the right scope and the
> other not so derived? We don't know this full derivation chain up front,
> and the chain can grow over time (after we do store-to-load forwarding
> in GVN or similar), so we'd need to have something in the pipeline that
> would add this metadata, but this certainly might all be workable.
>
> I think that this is worth exploring.
>
> Thanks again,
>
> Hal
>
>
> On 05/30/2018 09:46 AM, Jeroen Dobbelaere wrote:
>> Hi Hal,
>>
>> we are currently investigating if we can reduce the local restrict issues by making use of a 'side channel'.
>>
>> The rough idea is to start from a base form coming from clang, where we have something like:
>>
>>  ...
>>  %p = ...
>>  %rp =llvm.noalias %p , !scope.., !noalias ...
>>  ...
>>  store %rp, 10, !scope..., !noalias ...
>>  ...
>>
>>
>> and convert it (early, with an llvm-ir pass) to a form like:
>>  ...
>>  %p = ...
>>  %rp =llvm.noalias %p , !scope.., !noalias ...
>>  ...
>>  store %p, 10, !side_gep %rp, !scope..., !noalias ...  //@ Introduction of '!side_gep %rp, and removing the 'direct dependency' on %rp for the store address
>>  ...  
>>
>> This second form, is the version where we can deduce restrictness (through the 'side_gep')
>> We believe that this might help us fix the possible issues with LSR
>> We also think that a variant of this can help us to implement support for a restrict pointer that is member of a struct (and as an extension, for proposal n4150).
>>
>> Do you expect any issue with adding such a 'side channel' metadata to load/store instructions ?
>>
>> Thanks,
>>
>> Jeroen Dobbelaere
>>
>>
>>
>>
>>

-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory

------------------------------

Message: 7
Date: Fri, 17 Aug 2018 15:49:38 +0000
From: Emanuele Del Sozzo via llvm-dev <llvm-dev at lists.llvm.org>
To: cszide <cszide at 163.com>, "llvm-dev at lists.llvm.org"
    <llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] Replication -O3 optimizations manually
Message-ID:
    <VI1PR0802MB215708C341312DABDBB31690E53D0 at VI1PR0802MB2157.eurprd08.prod.outlook.com>

Content-Type: text/plain; charset="utf-8"

Hi Zide,

I think I found the right way to reach my goal.

I used the following command:

clang -O3 -Xclang -disable-llvm-optzns main.c -S -emit-llvm -o main.ll

to generate an IR file enriched by all the metadata that otherwise wouldn't be generated with -O0. Moreover, -disable-llvm-optzns flag ensures that none of the optimization passes has been applied yet to the IR.

In this way, I can replicate -O3 result by applying the optimization passes using opt. Apparently, those metadata are necessary to fully optimize the code.

I hope that this may help you too.

Best regards

Emanuele Del Sozzo

________________________________
From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Stefano Cherubin via llvm-dev <llvm-dev at lists.llvm.org>
Sent: Friday, August 17, 2018 11:44:50 AM
To: llvm-dev at lists.llvm.org; cszide
Subject: Re: [llvm-dev] Replication -O3 optimizations manually

Hi Zide,

the scope of opt is limited to the LLVM-IR, which is meant to be always target independent.
In order to apply backend optimizations you need to lower the representation to something closer to the machine-level.
I would suggest you to measure performance on machine code, not LLVM-IR.
To this end, please refer to the setup Emanuele is using.

However, I may not have properly understood your test.
lli is the LLVM-IR interpreter and it is meant more for functional testing rather than performance testing.
Are you comparing the performance of machine code generated by clang -O3 against the performance of lli optimized_IR.bc ?

Best regards,

Stefano Cherubin

On Friday, 17 August 2018, 03:55:52 CEST, cszide <cszide at 163.com> wrote:

Hi, Stefano
I also have the problem as described by Emanuele. You say that clang schedules target-independent and target-dependent passes.
However, when I use lli to execute bitcode generated by opt with -O3 or with the same optimization passes as -O3, the performance are still different.
So, are there some special operations by -O3 option? I read the source code of opt, but I cannot find the reason.

Best regards
Zide

At 2018-08-16 22:13:14, "Stefano Cherubin via llvm-dev" <llvm-dev at lists.llvm.org> wrote:
Hello Emanuele,

When you provide the optimization level -O3 to the clang driver, it does not simply schedule a sequence of passes to be run on the intermediate representation.
Indeed, it schedules target-independent and target-dependent passes.
Moreover, IIRC, the optimization level is also used in the later stages of the code generation to apply target-dependent optimizations (i.e. vectorizer).

The most common use case when someone wants to test its own pass/work within the LLVM toolchain is the following
- use clang to generate a LLVM-IR file
- use opt to run your desired pass / pass sequence and output another LLVM-IR file
- use clang -O3 to compile to executable machine code

However, with this approach you will run the passes on the LLVM-IR twice.
There are use cases when this could invalidate your results.
As opt stops at LLVM-IR level, I would suggest you to use also other LLVM tools to run individually the backend stages / sequence of passes which cannot be run by opt (such as llc / llvm-mc).
An extensive list of tools/commands you can use is available at [0].
For your specific case, I would suggest you to have a look at this restricted schema [1].

Yet there is another way to get into even fine grain detail.
You can check which are the clang DriverActions you are running with a given command line. See [2].
>From that point you can rebuild the exact whole sequence of commands that the clang driver triggers.

If you can provide more details about what is your use case (measure performance, pass development and testing, flag selection, phase ordering), we can suggest the most suitable approach.

Kind regards,

Stefano Cherubin

[0] http://llvm.org/docs/CommandGuide/
[1] https://github.com/skeru/LLVM-intro/blob/master/img/03/toolchain.pdf
[2] https://clang.llvm.org/docs/DriverInternals.html#driver-stages

On Thursday, 16 August 2018, 12:46:04 CEST, Emanuele Del Sozzo via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:

Hello llvm-dev,

my name is Emanuele and I am an intern in ARM. As part of the project I am doing here, I would like to manually replicate the optimizations that LLVM applies when I type -O3. In other words, I would like to know what are the compilation flags/passes that -O3 triggers.

I noticed that GCC reports, on its website, all the flags that are enforced by -O3 (https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html), but I wasn't able to find something similar within LLVM documentation. On the other hand, I found that this command displays all the optimization passes applied by opt when -O3 flag is on:

llvm-as < /dev/null | opt -O3 -disable-output -debug-pass=Arguments

I tried to apply the same optimization passes through opt, but, even though the performance are similar, the resulting binary is slower than the one generated using -O3 (also the binaries differ, of course).

Again, I found this other command that does something similar (it lists the sequence of optimization passes applied):

clang -O3 -mllvm -debug-pass=Arguments file.c

In this case, the performance are still different and some of the optimization passes listed in the last block of passes (e.g. -machinemoduleinfo, -stack-protector, etc.) are unknown to opt.

Said that, my question is: how can I find out what optimization passes/flags -O3 enforces in order to manually apply the same optimizations and have, hopefully, the same binary and performance?

I am currently using LLVM version 5.0.2.

Thank you for both your help and your time!

Best regards

Emanuele

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180817/0cc0448b/attachment.html>

------------------------------

Subject: Digest Footer

_______________________________________________
llvm-dev mailing list
llvm-dev at lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

------------------------------

End of llvm-dev Digest, Vol 170, Issue 53
*****************************************