[lldb-dev] Buildbot log retention.

[Galina Kistanova]

Hello Vince,

Unfortunately, that's not that easy. Buildbot keeps all the builds statuses
and all the build steps statuses along with the actual build logs. This
information gets "pickled" - i.e. serialized to the file system and then
deserialized back when consumed, for instance by the web UI, with some
caching on top. Everything gets loaded before any filter could be applied.

I have changed the settings to keep the logs for all the statuses we keep
(currently 200). This is safe and relatively "cheap" change at the "cost"
of some room on a hard drive.

I will also play with the history horizon to see how the response time
would change depending on various settings, It might be some sweet spot
there we can get to.

> what about allowing it for specific builders via a configuration setting
override?

Zachary, please feel free to propose the patch. So far we have just few
LLDB builders, and selectively increasing the horizon just for them, should
not hurt much.

Other possible move would be upgrading the build master hardware for a
faster disk subsystem.

Thanks

Galina




On Sat, Mar 14, 2015 at 9:20 PM, Vince Harron <vince at nethacker.com> wrote:

> The debian builder only has logs back ~29 hours ago, build 4197.  The
> latest build is 4249.  We're retaining less than 60 builds, is that
> expected?
>
>
>
> On Wed, Mar 11, 2015 at 9:25 AM, Zachary Turner <zturner at google.com>
> wrote:
>
>> Even if it's an issue to change this for everything, what about allowing
>> it for specific builders via a configuration setting override?
>>
>> On Wed, Mar 11, 2015 at 9:15 AM Vince Harron <vince at nethacker.com> wrote:
>>
>>> Most of the logs won't ever be accessed, they'll just be sitting
>>> somewhere on the filesystem.  As long as you have plenty of disk space, I
>>> don't anticipate any problems.
>>>
>>>
>>>
>>> On Tue, Mar 10, 2015 at 11:26 AM, Galina Kistanova <gkistanova at gmail.com
>>> > wrote:
>>>
>>>> Hi Vince,
>>>>
>>>> Unfortunately, this is not as easy as we'd want.
>>>> We are using Mac-mini as the buildmaster, and even if there is enough
>>>> room on the disk, it might  be not enough performance to deal with 5-10
>>>> times more data.
>>>>
>>>> I'll look onto this.
>>>>
>>>> Thanks
>>>>
>>>> Galina
>>>>
>>>>
>>>> On Fri, Mar 6, 2015 at 9:35 PM, Vince Harron <vince at nethacker.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> Can we increase the number of logs retained by the lldb buildbots?
>>>>> It's currently holding on to ~200 builds per configuration.  I'd like to
>>>>> see that increase to 1000 or even 2000.
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Vince
>>>>>
>>>>>
>>>>>
>>>> _______________________________________________
>>> lldb-dev mailing list
>>> lldb-dev at cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150316/f9dd58a6/attachment.html>