<div dir="ltr">Just an update,<div><br></div><div>At this point I've submitted and reverted this probably 6+ times, and I don't have time to be blocked by this anymore. </div><div><br></div><div>I'm looking into alternatives for now, including ways of creating a new top-level git repo such as <a href="https://git.llvm.org/git/codeview-tests.git/">https://git.llvm.org/git/codeview-tests.git/</a>.<br></div><div><br></div><div>I wish there were another way, but at this point I've been working on appeasing this for close to 2 months and I need some control over the bits that determine whether I'm able to make forward progress.</div><div><br></div><div>If and when someone on the Apple side has some time to make this work with their build infrastructure, I will be happy to revisit. <br></div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Nov 13, 2017 at 4:44 PM Zachary Turner <<a href="mailto:zturner@google.com">zturner@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Great! It's close to the end of the day, so I'll submit tomorrow to make sure everything has a chance to go fully green again to ensure I get failure emails if it breaks.</div><br><div class="gmail_quote"><div dir="ltr">On Mon, Nov 13, 2017 at 4:43 PM Adrian Prantl <<a href="mailto:aprantl@apple.com" target="_blank">aprantl@apple.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space">I can confirm that that fixes the issue!<div><br></div><div></div></div><div style="word-wrap:break-word;line-break:after-white-space"><div>— adrian</div></div><div style="word-wrap:break-word;line-break:after-white-space"><div><br><div><br><blockquote type="cite"><div>On Nov 13, 2017, at 4:38 PM, Zachary Turner <<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>> wrote:</div><br class="m_-4179398862014117493m_3700085524307750446Apple-interchange-newline"><div><div dir="ltr">Yea I also just found it. Try adding this code in the bottom of debuginfo-tests/<a href="http://lit.cfg.py/" target="_blank">lit.cfg.py</a><div><br></div><div><div>lit.util.usePlatformSdkOnDarwin(config, lit_config)</div></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Nov 13, 2017 at 4:38 PM Adrian Prantl <<a href="mailto:aprantl@apple.com" target="_blank">aprantl@apple.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Ha! Found it. *Somebody* is setting an SDKROOT variable in the environment. Can you find the code that would do this?<br>
<br>
— adrian<br>
<br>
> On Nov 13, 2017, at 4:30 PM, Adrian Prantl via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br>
><br>
> Yes I can reproduce this locally. It looks like we are not passing an -isysroot (pointing to the SDK) to clang but it isn’t clear what lit magic would expand this.<br>
><br>
> -- adrian<br>
><br>
>> On Nov 13, 2017, at 3:30 PM, Zachary Turner <<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>> wrote:<br>
>><br>
>> Yea I'm preparing a revert right now. Does it happen for you when you run debuginfo-tests locally?<br>
>><br>
>> On Mon, Nov 13, 2017 at 3:28 PM Adrian Prantl <<a href="mailto:aprantl@apple.com" target="_blank">aprantl@apple.com</a>> wrote:<br>
>> Since this is causing all of our internal CI to back up, could you please revert your two changes, so we can make sure that they were actually responsible, and can work on a fix for this? Let me know how we can help investigate this.<br>
>><br>
>> -- adrian<br>
>><br>
>><br>
>>> On Nov 13, 2017, at 3:25 PM, Adrian Prantl via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br>
>>><br>
>>> The first build where a test fails with similar symptoms has just one blamelist entry:<br>
>>><br>
>>> <a href="http://green.lab.llvm.org/green/job/clang-stage1-configure-RA/40391/" rel="noreferrer" target="_blank">http://green.lab.llvm.org/green/job/clang-stage1-configure-RA/40391/</a><br>
>>><br>
>>> • Update <a href="http://test_debuginfo.pl/" rel="noreferrer" target="_blank">test_debuginfo.pl</a> script to point to new tree location. (detail/ViewSVN)<br>
>>> by zturner<br>
>>> -- adrian<br>
>>><br>
>>>> On Nov 13, 2017, at 3:21 PM, Zachary Turner <<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>> wrote:<br>
>>>><br>
>>>> On the other hand this file hasn't changed recently, but I have no way to test this as it uses the LLDB code path, which only runs on OSX.<br>
>>>><br>
>>>> On Mon, Nov 13, 2017 at 3:19 PM Zachary Turner <<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>> wrote:<br>
>>>> I might be missing something, but this doesn't look like me?<br>
>>>><br>
>>>> <a href="http://green.lab.llvm.org/green/job/clang-stage1-configure-RA/40478/consoleFull#-42777206a1ca8a51-895e-46c6-af87-ce24fa4cd561" rel="noreferrer" target="_blank">http://green.lab.llvm.org/green/job/clang-stage1-configure-RA/40478/consoleFull#-42777206a1ca8a51-895e-46c6-af87-ce24fa4cd561</a><br>
>>>><br>
>>>> PASS: debuginfo-tests :: dbg-arg.c (34886 of 40729)<br>
>>>> PASS: debuginfo-tests :: ctor.cpp (34887 of 40729)<br>
>>>> PASS: debuginfo-tests :: ctor.cpp (34888 of 40729)<br>
>>>> PASS: debuginfo-tests :: aggregate-indirect-arg.cpp (34889 of 40729)<br>
>>>> PASS: debuginfo-tests :: aggregate-indirect-arg.cpp (34890 of 40729)<br>
>>>> PASS: debuginfo-tests :: dbg-arg.c (34891 of 40729)<br>
>>>> PASS: debuginfo-tests :: asan.c (34892 of 40729)<br>
>>>> PASS: debuginfo-tests :: asan.c (34893 of 40729)<br>
>>>> PASS: debuginfo-tests :: asan-blocks.c (34894 of 40729)<br>
>>>> PASS: debuginfo-tests :: asan-blocks.c (34895 of 40729)<br>
>>>> PASS: debuginfo-tests :: nested-struct.cpp (34896 of 40729)<br>
>>>> PASS: debuginfo-tests :: nested-struct.cpp (34897 of 40729)<br>
>>>> PASS: debuginfo-tests :: forward-declare-class.cpp (34898 of 40729)<br>
>>>> PASS: debuginfo-tests :: forward-declare-class.cpp (34899 of 40729)<br>
>>>> FAIL: debuginfo-tests :: foreach.m (34900 of 40729)<br>
>>>><br>
>>>> ******************** TEST 'debuginfo-tests :: foreach.m' FAILED ********************<br>
>>>><br>
>>>> Script:<br>
>>>> --<br>
>>>> /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/clang-build/bin/clang --target=x86_64-apple-darwin15.6.0 -O0 -g /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/llvm/tools/clang/test/debuginfo-tests/tests/foreach.m -c -o /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/clang-build/tools/clang/test/debuginfo-tests/Output/foreach.m.tmp.o<br>
>>>> /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/clang-build/bin/clang --target=x86_64-apple-darwin15.6.0 /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/clang-build/tools/clang/test/debuginfo-tests/Output/foreach.m.tmp.o -o /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/clang-build/tools/clang/test/debuginfo-tests/Output/foreach.m.tmp.out -framework Foundation<br>
>>>> /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/llvm/utils/<br>
>>>> <a href="http://test_debuginfo.pl/" rel="noreferrer" target="_blank">test_debuginfo.pl</a><br>
>>>> /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/llvm/tools/clang/test/debuginfo-tests/tests/foreach.m /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/clang-build/tools/clang/test/debuginfo-tests/Output/foreach.m.tmp.out<br>
>>>> --<br>
>>>> Exit Code: 1<br>
>>>><br>
>>>> Command Output (stdout):<br>
>>>> --<br>
>>>> Debugger output was:<br>
>>>> imported lldb from: "/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Versions/A/Resources/Python"<br>
>>>> error: foreach.m.tmp.out debug map object file '/Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/clang-build/tools/clang/test/debuginfo-tests/Output/foreach.m.tmp.o' has changed (actual time is 0x5a0a2528, debug map time is 0x5a0a2526) since this executable was linked, file will be ignored<br>
>>>> > break 25<br>
>>>> SBBreakpoint: id = 1, file = '', line = 25, exact_match = 0, locations = 0<br>
>>>> > r<br>
>>>> success<br>
>>>> > po thing<br>
>>>> = <could not resolve type><br>
>>>> > quit<br>
>>>><br>
>>>> --<br>
>>>> Command Output (stderr):<br>
>>>> --<br>
>>>><br>
>>>> /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/llvm/tools/clang/test/debuginfo-tests/tests/foreach.m:11:11: error: expected string not found in input<br>
>>>><br>
>>>> // CHECK: aaa<br>
>>>> ^<br>
>>>> /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/clang-build/tools/clang/test/debuginfo-tests/Output/foreach.m.gdb.output:1:1: note: scanning from here<br>
>>>> imported lldb from: "/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Versions/A/Resources/Python"<br>
>>>> ^<br>
>>>> /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/clang-build/tools/clang/test/debuginfo-tests/Output/foreach.m.gdb.output:2:211: note: possible intended match here<br>
>>>> error: foreach.m.tmp.out debug map object file '/Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/clang-build/tools/clang/test/debuginfo-tests/Output/foreach.m.tmp.o' has changed (actual time is 0x5a0a2528, debug map time is 0x5a0a2526) since this executable was linked, file will be ignored<br>
>>>><br>
>>>><br>
>>>><br>
>>>> On Mon, Nov 13, 2017 at 3:17 PM Adrian Prantl <<a href="mailto:aprantl@apple.com" target="_blank">aprantl@apple.com</a>> wrote:<br>
>>>> It looks like the bots are still red?<br>
>>>><br>
>>>> — Adrian<br>
>>>><br>
>>>><br>
>>>>> On Nov 10, 2017, at 3:14 PM, Zachary Turner <<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>> wrote:<br>
>>>>><br>
>>>>> Wasn't quite fixed, but it got a lot further this time. This time there was still an issue in the <a href="http://test_debuginfo.pl/" rel="noreferrer" target="_blank">test_debuginfo.pl</a> script regarding a hardcoded path to the llgdb.py script. I think I never encountered this locally because this codepath only happens on Darwin, and I was testing on Linux.<br>
>>>>><br>
>>>>> (As an aside, ugh... Perl...)<br>
>>>>><br>
>>>>> Regardless, this should be fixed in r317949, and hopefully that's the last of the issues. I have to run for a couple of hours, but I can check on this again in a bit. But I strongly suspect it will be fixed now.<br>
>>>>><br>
>>>>> On Fri, Nov 10, 2017 at 2:51 PM Adrian Prantl <<a href="mailto:aprantl@apple.com" target="_blank">aprantl@apple.com</a>> wrote:<br>
>>>>><br>
>>>>>> On Nov 10, 2017, at 2:50 PM, Zachary Turner <<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>> wrote:<br>
>>>>>><br>
>>>>>> I checked in a fix for that already, sorry for the trouble. I’m waiting for it to cycle<br>
>>>>><br>
>>>>> awesome. Thanks!<br>
>>>>><br>
>>>>> -- adrian<br>
>>>>><br>
>>>>>> On Fri, Nov 10, 2017 at 2:49 PM Adrian Prantl <<a href="mailto:aprantl@apple.com" target="_blank">aprantl@apple.com</a>> wrote:<br>
>>>>>> It looks like this broke green dragon:<br>
>>>>>><br>
>>>>>> <a href="http://green.lab.llvm.org/green/job/clang-stage1-configure-RA/40383/console" rel="noreferrer" target="_blank">http://green.lab.llvm.org/green/job/clang-stage1-configure-RA/40383/console</a><br>
>>>>>><br>
>>>>>> llvm-lit: /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/llvm/projects/libcxx/utils/libcxx/test/config.py:173: note: Adding environment variables: {'DYLD_LIBRARY_PATH': '/Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/clang-build/./lib', 'LIBCXX_FILESYSTEM_DYNAMIC_TEST_ROOT': '/Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/clang-build/projects/libcxx/test/filesystem/Output/dynamic_env'}<br>
>>>>>><br>
>>>>>> llvm-lit: /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/llvm/utils/lit/lit/llvm/config.py:332: note: using clang: /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/clang-build/bin/clang<br>
>>>>>> llvm-lit: /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/llvm/utils/lit/lit/util.py:379: note: using SDKROOT: '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk'<br>
>>>>>><br>
>>>>>> llvm-lit: /Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/llvm/utils/lit/lit/TestingConfig.py:101: fatal: unable to parse config file '/Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/llvm/tools/clang/test/debuginfo-tests/<a href="http://lit.cfg.py/" rel="noreferrer" target="_blank">lit.cfg.py</a>', traceback: Traceback (most recent call last):<br>
>>>>>> File "/Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/llvm/utils/lit/lit/TestingConfig.py", line 88, in load_from_path<br>
>>>>>> exec(compile(data, path, 'exec'), cfg_globals, None)<br>
>>>>>> File "/Users/buildslave/jenkins/workspace/clang-stage1-configure-RA/llvm/tools/clang/test/debuginfo-tests/<a href="http://lit.cfg.py/" rel="noreferrer" target="_blank">lit.cfg.py</a>", line 36, in <module><br>
>>>>>> config.test_source_root = os.path.join(config.debuginfo_tests_src_root, 'tests')<br>
>>>>>> AttributeError: TestingConfig instance has no attribute 'debuginfo_tests_src_root'<br>
>>>>>><br>
>>>>>> FAILED: CMakeFiles/check-all<br>
>>>>>><br>
>>>>>> -- adrian<br>
>>>>>><br>
>>>>>> > On Nov 10, 2017, at 1:00 PM, Zachary Turner via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br>
>>>>>> ><br>
>>>>>> > This is in as of r317925. I'm keeping an eye out for failure notifications. I may or may not need help diagnosing if something does go wrong (although I'm keeping my fingers crossed)<br>
>>>>>> ><br>
>>>>>> > On Thu, Nov 9, 2017 at 4:05 PM Zachary Turner <<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>> wrote:<br>
>>>>>> > Since it's towards the end of the day already, I'll put this in tomorrow morning around 9 or 10, to make sure I'm around to fix anything that arises (or revert).<br>
>>>>>> ><br>
>>>>>> ><br>
>>>>>> ><br>
>>>>>> > On Thu, Nov 9, 2017 at 2:53 PM Mike Edwards <<a href="mailto:medwards@apple.com" target="_blank">medwards@apple.com</a>> wrote:<br>
>>>>>> > Hi Zach,<br>
>>>>>> > Thanks for doing this extra work to make this lower impact for the rest of us. Let’s give it a try and see what happens.<br>
>>>>>> ><br>
>>>>>> > -Mike<br>
>>>>>> ><br>
>>>>>> ><br>
>>>>>> ><br>
>>>>>> >> On Nov 9, 2017, at 13:37, Zachary Turner <<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>> wrote:<br>
>>>>>> >><br>
>>>>>> >> Hi all, I think I've addressed all the concerns here, and I believe there should be no immediate impact to the current workflow. with that said, I plan to commit this either later today or early tomorrow if there are no other concerns.<br>
>>>>>> >><br>
>>>>>> >> On Tue, Nov 7, 2017 at 12:19 PM Zachary Turner <<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>> wrote:<br>
>>>>>> >> I tested this out, and AFAICT nothing will change. It will continue to just work if you have it checked out under clang/tests. It's a bit hard to construct this configuration locally since it requires moving some files around, and applying half of a CL here and half of a CL there. But, AFAICT it works.<br>
>>>>>> >><br>
>>>>>> >> I'm happy to send you some patches if you want to try them locally and confirm.<br>
>>>>>> >><br>
>>>>>> >> I'd like to print out a CMake warning if it detects the tree under clang/test and just mention that the workflow is deprecated. Any objections?<br>
>>>>>> >><br>
>>>>>> >> On Mon, Nov 6, 2017 at 1:49 PM Mike Edwards <<a href="mailto:medwards@apple.com" target="_blank">medwards@apple.com</a>> wrote:<br>
>>>>>> >> Thank you Zach.<br>
>>>>>> >><br>
>>>>>> >><br>
>>>>>> >>> On Nov 6, 2017, at 13:37, Zachary Turner <<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>> wrote:<br>
>>>>>> >>><br>
>>>>>> >>> I’m going to spend a little time seeing if i can make the change invisible to the bots so they will continue to work as they do today. Will report back after I’ve explored that a bit<br>
>>>>>> >>> On Mon, Nov 6, 2017 at 1:35 PM Mike Edwards <<a href="mailto:medwards@apple.com" target="_blank">medwards@apple.com</a>> wrote:<br>
>>>>>> >>>> I'm honestly not opposed to this idea. It just seems a shame to do this for purely logistical reasons if most people agree that the "right" place for debuginfo-tests is outside of the clang tree.<br>
>>>>>> >>><br>
>>>>>> >>> I totally understand what you are saying here and will just add that sometimes being part of a larger community means being willing to do things, sometimes, not exactly the “right” way, due to logistical reasons. I am not opposed to what you would like to do, I’m just furrowing my brow at the timeframe in which to do it.<br>
>>>>>> >>><br>
>>>>>> >>>><br>
>>>>>> >>>> That said, I'd still like to hear from ChrisM and MikeE about why it will take so long, because on the surface it seems like a low-impact move.<br>
>>>>>> >>><br>
>>>>>> >>> Past experience has taught me, anything I think is going to be simple and quick to fix, rarely ever turns out that way. While there will be a significant amount of work to change the way our bots work here at Apple, the work is not impossible to accomplish. Given the choice, I would of course prefer an approach such as Paulr has suggested. The ability to run things in parallel for a time provides for a much lower impact change on the entire community. I think this approach may also give us some time to decide where the debuginfo-test should fit in the new mono-repo. It would be a bummer to do the work necessary to make this change, only to discover we would have to do it differently in the not too distant future to accommodate the new mono-repo.<br>
>>>>>> >>><br>
>>>>>> >>> Zach, I do not want to be a blocker here. I just want to make sure we have explored all of the options to make sure we are not missing a lower impact approach. I also want to make sure we are not doing something that could wait until we migrate to the mono-repo next year.<br>
>>>>>> >>><br>
>>>>>> >>> Thanks,<br>
>>>>>> >>> Mike<br>
>>>>>> >><br>
>>>>>> ><br>
>>>>>> > _______________________________________________<br>
>>>>>> > LLVM Developers mailing list<br>
>>>>>> > <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
>>>>>> > <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
>>>>>><br>
>>>>><br>
>>>><br>
>>><br>
>>> _______________________________________________<br>
>>> LLVM Developers mailing list<br>
>>> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
>>> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
>><br>
><br>
> _______________________________________________<br>
> LLVM Developers mailing list<br>
> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
<br>
</blockquote></div>
</div></blockquote></div><br></div></div></blockquote></div></blockquote></div>