[llvm-dev] llvm bug 11944

Tue Apr 27 11:54:58 PDT 2021

Hi,
As on adopting their ms extensions here, id guess as much as i hate them,
theyre in the right here and nux world isat fault.

As on eliminating deep asserts i dont know.

As on run time checkecks i feel aware of them and if you can mix and match
them and they work 100% no problem here.

Best regards,
Pawel Kunio

wt., 27.04.2021, 02:01 użytkownik David Blaikie <dblaikie at gmail.com>
napisał:

> On Mon, Apr 26, 2021 at 4:45 PM pawel k. <pawel.kunio at gmail.com> wrote:
> >
> > Hello,
> > Yep I meant sal. Id think we coud have printfy sallike annottation added
> to llvm libc at least for emulation of planned c++ extension about precond
> postcond constraints that need to hold for calls to lib not to give
> warnings or not to err. We could try to catch common risks if not
> errors/holes early via such a way.
>
> Possibly - as you say, it's something Microsoft's found useful to
> have, though hasn't really been picked up in the unix-y world.
>
> > Not sure im 100% right here but if preconds/postconds can be moved fully
> to compile time, we could try to get rid of many checks from libc code and
> for example move them to asserts in debug only etc.
>
> Maybe - though being able to build existing code that might rely on
> these checks is pretty important, so that might not be a feasible
> direction.
>
> > You may see by now already im enthusiast of solution of sw devel moving
> in bit rusty direction like linux kernel plans to.
> >
> > Another side question if we got or could use having solution(s) that big
> redmond supplier has for catching many
> >
> > pointer access/mem errors/leaks/usebeforealloc/useafterfree/global and
> stack vars usedbefore init
> >
> > kind of errors with 100% rate assumed all paths are taken. It is runtime
> checks only though and might need extending interfaces of malloc free new
> delete either via syntax extension or without if assisted by compiler.
>
> You're asking if the LLVM project has tools for finding those kinds of
> bugs? The sanitizers catch some of these at runtime (ASan for
> accessing memory that you don't have access to (buffer overrun, etc),
> MSan for accessing uninitialized memory, LSan for leaks, etc) and some
> can be caught at compile time (Clang has a check for variables used
> before they are initialized, for instance).
>
> - Dave
>
> >
> > Best regards,
> > Pawel Kunio
> >
> > pon., 26.04.2021, 18:30 użytkownik David Blaikie <dblaikie at gmail.com>
> napisał:
> >>
> >> On Fri, Apr 23, 2021 at 9:10 PM pawel k. <pawel.kunio at gmail.com> wrote:
> >> >
> >> > Hello,
> >> > To clarify on stdlib/libc.
> >>
> >> For what it's worth, there are several implementations of libc - GNU
> >> libc and LLVM libc (well, the latter is relatively new/incomplete)
> >> would be
> >>
> >> >  I like way ms has annotations in comments of their stdlib/libc.
> >>
> >> Ah, you're talking about SAL? (
> >>
> https://docs.microsoft.com/en-us/cpp/c-runtime-library/sal-annotations?view=msvc-160
> >> )
> >>
> >> >  It is used by compiler as config script to something similar to
> static analyzer/warningsmodule both release and debug and as base for
> genning runtime checks in debug only.
> >> >
> >> > It helps to catch suspicious or wrong calls to stdlib/libc.
> >> >
> >> > If You were interested in something similar, id be willing to help
> here. You wouldnt have to write a new subcheker/subanalyzer per libc
> function.
> >>
> >> Yeah, there's some general attribute annotations (like for
> >> printf-style functions, for instance). Not sure if there are
> >> attributes for general array+arraybound parameters, for instance.
> >>
> >> - Dave
> >>
> >> >
> >> > Best regards,
> >> > Pawel Kunio
> >> >
> >> > pt., 23.04.2021, 21:11 użytkownik David Blaikie <dblaikie at gmail.com>
> napisał:
> >> >>
> >> >> On Fri, Apr 23, 2021 at 12:29 AM pawel k. <pawel.kunio at gmail.com>
> wrote:
> >> >> >
> >> >> > Hello,
> >> >> > Thank You for kind friendly warning. Just to get it right. It
> might be too tough or too pointless for someone to integrate it?
> >> >>
> >> >> Yeah, my rough guess is that it's a challenging problem - one that
> >> >> many LLVM developers care about (many uses of LLVM as a library where
> >> >> the use of global constructors adds to the application's startup
> time)
> >> >> and have tried to fix to varying degrees at various times, but
> haven't
> >> >> managed it.
> >> >>
> >> >> > As of from ones you proposed i should either stick to warning
> point misreporting bugs or something on analyzer or tidy. Id say complex
> algos connected with optimizations skipping opportunity etc I could try to
> catch them and measure or benchmark them or maybe write tests for but fir
> hardcore c++ development i could need some time to get back in full shape.
> >> >>
> >> >> Actually middle end optimizations can sometimes be some of the
> simpler
> >> >> code to fix/play with - so I wouldn't rule out playing with/modifying
> >> >> them, if it interests you.
> >> >>
> >> >> > I was thinking of another side project that should be doable. I
> could try to get the snapshot of libc api and try to parse it and use it as
> config script to what kind of issues look for in analyzer.
> >> >>
> >> >> Not sure I'm following here - are you referring to GNU libc, or
> LLVM's
> >> >> recent libc project? & which analyzer?
> >> >>
> >> >> Clang's Static Analyzer ( https://clang-analyzer.llvm.org/ ) - if
> >> >> you're talking about that, and thinking of finding bugs /in/ the
> >> >> analyzer (rather than using the analyzer to find bugs in other code)
> >> >> by running it on some existing codebase (like libc, GNU or LLVM's)
> and
> >> >> reporting what you find - yes, that could be useful. (though realize
> >> >> there's probably lots of already known bugs, so sometimes bug finding
> >> >> isn't the bottleneck/where there's a problem)
> >> >>
> >> >> > Thus emulating planned c++ feature of postconditions preconditions
> checker.
> >> >> > If i reget my rolodex, i might be able to get in touch with few
> whitehats who could provide data what kind of holes via which api calls are
> most common and generally what kind of attacks misusing api are possible if
> that would help on this one.
> >> >>
> >> >> Yeah, if there are new static analysis checks you've got a use for,
> >> >> could try implementing them as either Clang warnings, clang-tidy
> >> >> checks, or Clang Static Analyzer checks.
> >> >>
> >> >> - Dave
> >> >>
> >> >> >
> >> >> > Best regards,
> >> >> > Pawel Kunio
> >> >> >
> >> >> > pt., 23.04.2021, 04:59 użytkownik David Blaikie <
> dblaikie at gmail.com> napisał:
> >> >> >>
> >> >> >> On Thu, Apr 22, 2021 at 7:47 PM pawel k. <pawel.kunio at gmail.com>
> wrote:
> >> >> >> >
> >> >> >> > Hello,
> >> >> >> > Sorry about my bad english and wrecked communications line. I
> meant 11944 bug from llvm database. As on our system, we had similar issue
> in sense on embedded project around lte and 5g base stations. Archs finally
> decided global compound constructors are forbidden due to startup overhead
> and call order unpredictability.
> >> >> >> >
> >> >> >> > Thank You for mentioning warning enabling flag. That should
> help me pinpoint all interesting uses of this construct.
> >> >> >>
> >> >> >> Sure thing - oh, the other thing you might find interesting,
> regarding
> >> >> >> the issues with global constructor ordering problems - there's a
> >> >> >> dynamic tool to help find those:
> >> >> >>
> https://github.com/google/sanitizers/wiki/AddressSanitizerInitializationOrderFiasco
> >> >> >>
> >> >> >> > I will try to lookup how these global constructors are done in
> llvmlib. What i saw as general guideline along which commandline opts
> component is built i generally like. I dont see yet fully where is issue in
> moving it to runtime. Id think we could try to use some pattern like
> commandline opts tables from gcc but nit sure yet whether keep much in
> globals or store within specific apps/tools classes.
> >> >> >> >
> >> >> >> > In one of first phases id think of checking buildflags of clang
> what targets etc are enabled and register all supported commands and maybe
> prescan command line for further flags which enable or disable other flags.
> >> >> >> >
> >> >> >> > I will try to build llvm lib with this warning enabled to see
> all opts uses and learn the api etc. I need to understand furst completely
> based on what logic macros and flags which flags blocks should be
> registered and later kept or disabled if we need to scan commandline in
> multiple passes etc.
> >> >> >>
> >> >> >> I will say I wouldn't readily suggest this as the /best/ place to
> >> >> >> start in LLVM - if you're interested in getting into the project
> >> >> >> generally, without any particular preference for which part -
> given
> >> >> >> this has been thought about a fair bit by folks throughout the
> >> >> >> project's life, the amount of design work/time involved by core
> >> >> >> developers will probably be non-trivial to figure out the right
> path
> >> >> >> before the mechanical work can begin. Fair warning.
> >> >> >>
> >> >> >> Usually I suggest starting with clang warnings (or
> >> >> >> clang-format/clang-tidy these days) bugs, since they can be fairly
> >> >> >> narrow/sometimes relatively isolated to fix. Source location bugs
> or
> >> >> >> fixits (warnings that point to the wrong part of the code, or
> errors
> >> >> >> that don't offer a fixit hint when they could) can be interesting
> >> >> >> learning experiences about the Clang Abstract Syntax Tree/source
> >> >> >> location information.
> >> >> >>
> >> >> >> But if you're more interested in LLVM proper - probably middle-end
> >> >> >> misoptimizations (probably easier to fix bugs/miscompiles than
> fixing
> >> >> >> missed optimizations - the latter are probably more likely to be
> >> >> >> deeply involved work/changes to make an optimization more
> powerful for
> >> >> >> some reason).
> >> >> >>
> >> >> >> > Ill try to learn all i can first from code and gathered
> warnings and if questions arise ill ring back with them and if not ill try
> to propose some solution after fully testing it on current testsuite and
> all targets etc.
> >> >> >>
> >> >> >> Yep, always happy to answer questions/provide pointers.
> >> >> >>
> >> >> >> - Dave
> >> >> >>
> >> >> >> >
> >> >> >> > Best regards,
> >> >> >> > Pawel Kunio
> >> >> >> >
> >> >> >> >
> >> >> >> > pt., 23.04.2021, 02:30 użytkownik David Blaikie <
> dblaikie at gmail.com> napisał:
> >> >> >> >>
> >> >> >> >> [It might be helpful if you used a few more words & fully
> explained
> >> >> >> >> what you're referring to - I'm having a hard time following
> your
> >> >> >> >> emails in this abbreviated writing style]
> >> >> >> >>
> >> >> >> >> You looked at an existing bug (do you have a link to it, or
> bug number
> >> >> >> >> on the bugs.llvm.org database) related to the use of global
> >> >> >> >> constructors in the LLVM codebase? Or in another codebase
> using LLVM?
> >> >> >> >>
> >> >> >> >> You had similar issues on another codebase you worked on where
> you
> >> >> >> >> weren't allowed to use any non-trivial global constructors?
> >> >> >> >>
> >> >> >> >> & you're proposing creating a checker or other tool to help
> find these
> >> >> >> >> cases? Or proposing using an existing tool for finding such
> things?
> >> >> >> >>
> >> >> >> >> Clang does have a warning for this already, I believe:
> >> >> >> >> -Wglobal-constructors. But, yes, the LLVM codebase isn't
> remotely
> >> >> >> >> ready for that and it's not been a high enough priority for
> anyone to
> >> >> >> >> really clean it up - mostly because the main use of global
> >> >> >> >> constructors is in the LLVM command line argument handling
> code - so
> >> >> >> >> it's a non-trivial design/redesign/refactoring effort to
> figure out
> >> >> >> >> the right new design for that and make all the changes
> necessary to
> >> >> >> >> migrate to such a design. (after that there'd probably be a
> bunch of
> >> >> >> >> smaller more incremental changes to cleanup global
> constructors and
> >> >> >> >> get the codebase to have no -Wglobal-constructors warnings,
> then we
> >> >> >> >> could turn on the warning to ensure we didn't regress)
> >> >> >> >>
> >> >> >> >>  - Dave
> >> >> >> >>
> >> >> >> >> On Thu, Apr 22, 2021 at 4:50 PM pawel k. via llvm-dev
> >> >> >> >> <llvm-dev at lists.llvm.org> wrote:
> >> >> >> >> >
> >> >> >> >> > Hello,
> >> >> >> >> > I took glimpse of PR llvm and or clang has ie lib cpp
> compound constructing global vars which makes objects get constructed in
> random order and with nonzero startup cpu/time overhead.
> >> >> >> >> >
> >> >> >> >> > In one of rather mighty embedded projects for 4g enodeb and
> later 5g base stations we had similar issue. It was forbidden to complex
> construct global vars.
> >> >> >> >> >
> >> >> >> >> > We could fancy clang syntax checker having option to early
> detect and track those so we could now how many are still left if any. I
> would disencourage running it on testsuite though as there were many false
> positives reported there.
> >> >> >> >> >
> >> >> >> >> > Happy if that helps any.
> >> >> >> >> >
> >> >> >> >> > Best regards,
> >> >> >> >> > Pawel Kunio
> >> >> >> >> > _______________________________________________
> >> >> >> >> > LLVM Developers mailing list
> >> >> >> >> > llvm-dev at lists.llvm.org
> >> >> >> >> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210427/e21930a6/attachment.html>