[compiler-rt] r189479 - Make lint checker script more robust

[Will Dietz]

Hi Alexey,

Awesome on all points! #1 and #2 directly address my issue and #3 is
an extra bonus :).

Thanks, have a good one!

~Will





On Thu, Aug 29, 2013 at 6:39 AM, Alexey Samsonov <samsonov at google.com> wrote:
> Hi Will!
>
> I've made a few improvements in r189577-189578:
> 1) now *san symbols are re-generated only if the runtime has changed.
> 2) code style checks are now only run as a part of testsuite for sanitizer
> tools (not when you run make -j20)
> 3) I've added a bit of parallelism: the execution time of check_lint.sh
> dropped from 13s to 4s on my machine.
>
>
> On Wed, Aug 28, 2013 at 8:12 PM, Will Dietz <w at wdtz.org> wrote:
>>
>> Hi Alexey, llvm-commits@,
>>
>> Is it possible to avoid the use of "ALL" as the dependency here?
>>
>> Presently the lint checking, gen_dynamic_list.py, and dfsan_abilist
>> targets
>> use this causing these targets to be executed unconditionally.
>>
>> This causes around 16s to be used on my machine, even
>> on a fully up-to-date build.  It's not a big deal for me personally,
>> but 16s for each build for everyone with compiler-rt in their build tree
>> does add up.
>>
>> Is there some accurate list of dependencies that can be used
>> or an alternative method for executing these targets only
>> when something has changed?
>>
>> FWIW 'ninja -d explain' shows the following on a fully up-to-date build:
>> ---------------------------------------
>> $ ninja -d explain
>> ninja explain: output
>> projects/compiler-rt/lib/asan/CMakeFiles/clang_rt.asan-x86_64-symbols
>> doesn't exist
>> ninja explain:
>> projects/compiler-rt/lib/asan/CMakeFiles/clang_rt.asan-x86_64-symbols
>> is dirty
>> ninja explain: projects/compiler-rt/lib/asan/clang_rt.asan-x86_64-symbols
>> is dirty
>> ninja explain: output
>> projects/compiler-rt/lib/sanitizer_common/CMakeFiles/SanitizerLintCheck
>> doesn't exist
>> ninja explain:
>> projects/compiler-rt/lib/sanitizer_common/CMakeFiles/SanitizerLintCheck
>> is dirty
>> ninja explain:
>> projects/compiler-rt/lib/sanitizer_common/SanitizerLintCheck
>> is dirty
>> ninja explain: output
>> projects/compiler-rt/lib/ubsan/CMakeFiles/clang_rt.ubsan-x86_64-symbols
>> doesn't exist
>> ninja explain:
>> projects/compiler-rt/lib/ubsan/CMakeFiles/clang_rt.ubsan-x86_64-symbols
>> is dirty
>> ninja explain:
>> projects/compiler-rt/lib/ubsan/clang_rt.ubsan-x86_64-symbols
>> is dirty
>> ninja explain: output
>>
>> projects/compiler-rt/lib/ubsan/CMakeFiles/clang_rt.ubsan_cxx-x86_64-symbols
>> doesn't exist
>> ninja explain:
>> projects/compiler-rt/lib/ubsan/CMakeFiles/clang_rt.ubsan_cxx-x86_64-symbols
>> is dirty
>> ninja explain:
>> projects/compiler-rt/lib/ubsan/clang_rt.ubsan_cxx-x86_64-symbols
>> is dirty
>> ninja explain: output
>> projects/compiler-rt/lib/tsan/rtl/CMakeFiles/clang_rt.tsan-x86_64-symbols
>> doesn't exist
>> ninja explain:
>> projects/compiler-rt/lib/tsan/rtl/CMakeFiles/clang_rt.tsan-x86_64-symbols
>> is dirty
>> ninja explain:
>> projects/compiler-rt/lib/tsan/rtl/clang_rt.tsan-x86_64-symbols
>> is dirty
>> ninja explain: output
>> projects/compiler-rt/lib/msan/CMakeFiles/clang_rt.msan-x86_64-symbols
>> doesn't exist
>> ninja explain:
>> projects/compiler-rt/lib/msan/CMakeFiles/clang_rt.msan-x86_64-symbols
>> is dirty
>> ninja explain: projects/compiler-rt/lib/msan/clang_rt.msan-x86_64-symbols
>> is dirty
>> ninja explain: projects/compiler-rt/lib/dfsan/CMakeFiles/dfsan_abilist is
>> dirty
>> ninja explain: projects/compiler-rt/lib/dfsan/dfsan_abilist is dirty
>> ---------------------------------------
>>
>> Thank you for your time,
>>
>> ~Will
>>
>> On Wed, Aug 28, 2013 at 7:15 AM, Alexey Samsonov <samsonov at google.com>
>> wrote:
>> > Author: samsonov
>> > Date: Wed Aug 28 07:15:03 2013
>> > New Revision: 189479
>> >
>> > URL: http://llvm.org/viewvc/llvm-project?rev=189479&view=rev
>> > Log:
>> > Make lint checker script more robust
>> >
>> > Modified:
>> >     compiler-rt/trunk/lib/sanitizer_common/CMakeLists.txt
>> >     compiler-rt/trunk/lib/sanitizer_common/scripts/check_lint.sh
>> >
>> > Modified: compiler-rt/trunk/lib/sanitizer_common/CMakeLists.txt
>> > URL:
>> > http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/sanitizer_common/CMakeLists.txt?rev=189479&r1=189478&r2=189479&view=diff
>> >
>> > ==============================================================================
>> > --- compiler-rt/trunk/lib/sanitizer_common/CMakeLists.txt (original)
>> > +++ compiler-rt/trunk/lib/sanitizer_common/CMakeLists.txt Wed Aug 28
>> > 07:15:03 2013
>> > @@ -104,7 +104,8 @@ endif()
>> >  if(UNIX)
>> >    set(SANITIZER_LINT_SCRIPT
>> > ${CMAKE_CURRENT_SOURCE_DIR}/scripts/check_lint.sh)
>> >    add_custom_target(SanitizerLintCheck ALL
>> > -    COMMAND LLVM_CHECKOUT=${LLVM_MAIN_SRC_DIR} SILENT=1
>> > ${SANITIZER_LINT_SCRIPT}
>> > +    COMMAND LLVM_CHECKOUT=${LLVM_MAIN_SRC_DIR} SILENT=1 TMPDIR=
>> > +      ${SANITIZER_LINT_SCRIPT}
>> >      DEPENDS ${SANITIZER_LINT_SCRIPT}
>> >      COMMENT "Running lint check for sanitizer sources..."
>> >      VERBATIM)
>> >
>> > Modified: compiler-rt/trunk/lib/sanitizer_common/scripts/check_lint.sh
>> > URL:
>> > http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/sanitizer_common/scripts/check_lint.sh?rev=189479&r1=189478&r2=189479&view=diff
>> >
>> > ==============================================================================
>> > --- compiler-rt/trunk/lib/sanitizer_common/scripts/check_lint.sh
>> > (original)
>> > +++ compiler-rt/trunk/lib/sanitizer_common/scripts/check_lint.sh Wed Aug
>> > 28 07:15:03 2013
>> > @@ -31,12 +31,12 @@ SANITIZER_INCLUDES_LINT_FILTER=${COMMON_
>> >  cd ${LLVM_CHECKOUT}
>> >
>> >  EXITSTATUS=0
>> > -LOG=$(mktemp)
>> > +LOG=$(mktemp -q)
>> >
>> >  run_lint() {
>> >    FILTER=$1
>> >    shift
>> > -  if [ "${SILENT}" == "1" ]; then
>> > +  if [[ "${SILENT}" == "1" && "${LOG}" != "" ]]; then
>> >      ${CPPLINT} --filter=${FILTER} "$@" 2>>$LOG
>> >    else
>> >      ${CPPLINT} --filter=${FILTER} "$@"
>> > @@ -97,7 +97,8 @@ for FILE in $FILES; do
>> >  done
>> >
>> >  if [ "$EXITSTATUS" != "0" ]; then
>> > -  cat $LOG | grep -v "Done processing" | grep -v "Total errors found"
>> > +  cat $LOG | grep -v "Done processing" | grep -v "Total errors found" \
>> > +    grep -v "Skipping input"
>> >  fi
>> >
>> >  exit $EXITSTATUS
>> >
>> >
>> > _______________________________________________
>> > llvm-commits mailing list
>> > llvm-commits at cs.uiuc.edu
>> > http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>
>
>
>
> --
> Alexey Samsonov, MSK