[LLVMdev] MmapAllocator

Steven Noonan steven at uplinklabs.net
Sun Aug 8 15:58:52 PDT 2010


Correct, it does at the moment. I noted this on the code review.

- Steven

On Sun, Aug 8, 2010 at 2:50 PM, Alistair Lynn <arplynn at gmail.com> wrote:
> Hi Steven-
>
> Nice, but will this not break Windows? From an initial glance over your patch, it seems to assume the existence of mmap() in some form or other.
>
> Alistair
>
> On 8 Aug 2010, at 03:05, Steven Noonan wrote:
>
>> Hi folks,
>>
>> I've been doing work on memory reduction in Unladen Swallow, and
>> during testing, LiveRanges seemed to be consuming one of the largest
>> chunks of memory. I wrote a replacement allocator for use by
>> BumpPtrAllocator which uses mmap()/munmap() in place of
>> malloc()/free(). It has worked flawlessly in testing, and reduces
>> memory usage quite nicely in Unladen Swallow.
>>
>> The code is available for review here. I'd appreciate feedback if
>> there's an interest in integrating this into LLVM trunk:
>> http://codereview.uplinklabs.net/1905049
>>
>> Here are the results of our memory utilization tests. The 'Mem max'
>> numbers aren't particularly revealing though, so take a look at the
>> graphs. I think the spambayes benchmark was one of the most
>> interesting.
>>
>> ### 2to3 ###
>> Mem max: 39008.000 -> 38904.000: 1.0027x smaller
>> Usage over time: http://tinyurl.com/3axczjc
>>
>> ### bzr_startup ###
>> Mem max: 11996.000 -> 11984.000: 1.0010x smaller
>> Usage over time: http://tinyurl.com/2ucnbhb
>>
>> ### call_method ###
>> Mem max: 11632.000 -> 11544.000: 1.0076x smaller
>> Usage over time: http://tinyurl.com/22r6y9r
>>
>> ### call_method_slots ###
>> Mem max: 10908.000 -> 10820.000: 1.0081x smaller
>> Usage over time: http://tinyurl.com/3yqb2r4
>>
>> ### call_method_unknown ###
>> Mem max: 11216.000 -> 11152.000: 1.0057x smaller
>> Usage over time: http://tinyurl.com/3ypudfj
>>
>> ### call_simple ###
>> Mem max: 10692.000 -> 10540.000: 1.0144x smaller
>> Usage over time: http://tinyurl.com/2a62cbv
>>
>> ### django ###
>> Mem max: 21600.000 -> 20672.000: 1.0449x smaller
>> Usage over time: http://tinyurl.com/35rclpd
>>
>> ### float ###
>> Mem max: 15904.000 -> 15852.000: 1.0033x smaller
>> Usage over time: http://tinyurl.com/2vokmep
>>
>> ### hg_startup ###
>> Mem max: 7000.000 -> 7012.000: 1.0017x larger
>> Usage over time: http://tinyurl.com/3x4wneu
>>
>> ### iterative_count ###
>> Mem max: 9992.000 -> 9908.000: 1.0085x smaller
>> Usage over time: http://tinyurl.com/24dy7ql
>>
>> ### nbody ###
>> Mem max: 13552.000 -> 13240.000: 1.0236x smaller
>> Usage over time: http://tinyurl.com/23dstyu
>>
>> ### normal_startup ###
>> Mem max: 5380.000 -> 5396.000: 1.0030x larger
>> Usage over time: http://tinyurl.com/2fh7cmv
>>
>> ### nqueens ###
>> Mem max: 12832.000 -> 12756.000: 1.0060x smaller
>> Usage over time: http://tinyurl.com/29whema
>>
>> ### pickle ###
>> Mem max: 6856.000 -> 6844.000: 1.0018x smaller
>> Usage over time: http://tinyurl.com/3az5v6y
>>
>> ### pickle_dict ###
>> Mem max: 6848.000 -> 6836.000: 1.0018x smaller
>> Usage over time: http://tinyurl.com/2bkjdoh
>>
>> ### pickle_list ###
>> Mem max: 6836.000 -> 6824.000: 1.0018x smaller
>> Usage over time: http://tinyurl.com/23llzct
>>
>> ### regex_compile ###
>> Mem max: 39176.000 -> 38536.000: 1.0166x smaller
>> Usage over time: http://tinyurl.com/33wylgu
>>
>> ### regex_effbot ###
>> Mem max: 12340.000 -> 12084.000: 1.0212x smaller
>> Usage over time: http://tinyurl.com/37u84z4
>>
>> ### regex_v8 ###
>> Mem max: 33596.000 -> 33828.000: 1.0069x larger
>> Usage over time: http://tinyurl.com/397hyfm
>>
>> ### richards ###
>> Mem max: 12760.000 -> 12680.000: 1.0063x smaller
>> Usage over time: http://tinyurl.com/25n3wkl
>>
>> ### rietveld ###
>> Mem max: 29008.000 -> 28636.000: 1.0130x smaller
>> Usage over time: http://tinyurl.com/25uu4x3
>>
>> ### slowpickle ###
>> Mem max: 14096.000 -> 13804.000: 1.0212x smaller
>> Usage over time: http://tinyurl.com/2wzxmu2
>>
>> ### slowspitfire ###
>> Mem max: 94292.000 -> 93992.000: 1.0032x smaller
>> Usage over time: http://tinyurl.com/2wo4lrs
>>
>> ### slowunpickle ###
>> Mem max: 11620.000 -> 11516.000: 1.0090x smaller
>> Usage over time: http://tinyurl.com/26pw2cr
>>
>> ### spambayes ###
>> Mem max: 35896.000 -> 35860.000: 1.0010x smaller
>> Usage over time: http://tinyurl.com/2dhtkeb
>>
>> ### startup_nosite ###
>> Mem max: 4876.000 -> 4868.000: 1.0016x smaller
>> Usage over time: http://tinyurl.com/3agcts8
>>
>> ### threaded_count ###
>> Mem max: 10048.000 -> 9972.000: 1.0076x smaller
>> Usage over time: http://tinyurl.com/322hltw
>>
>> ### unpack_sequence ###
>> Mem max: 11252.000 -> 11244.000: 1.0007x smaller
>> Usage over time: http://tinyurl.com/34lsbqd
>>
>> ### unpickle ###
>> Mem max: 6872.000 -> 6860.000: 1.0017x smaller
>> Usage over time: http://tinyurl.com/2aeqeua
>>
>> ### unpickle_list ###
>> Mem max: 6860.000 -> 6844.000: 1.0023x smaller
>> Usage over time: http://tinyurl.com/36q766k
>>
>>
>>
>>
>> And to gauge the performance impact, I also ran the speed tests. It
>> seems using mmap()/munmap() has very little performance impact in
>> either direction, so that's good:
>>
>> ### 2to3 ###
>> 35.590589 -> 35.824554: 1.0066x slower
>>
>> ### bzr_startup ###
>> Min: 0.157976 -> 0.155976: 1.0128x faster
>> Avg: 0.167575 -> 0.168924: 1.0081x slower
>> Not significant
>> Stddev: 0.00334 -> 0.00716: 2.1463x larger
>> Timeline: http://tinyurl.com/39thymp
>>
>> ### call_method ###
>> Min: 0.878663 -> 0.884666: 1.0068x slower
>> Avg: 0.887148 -> 0.888667: 1.0017x slower
>> Not significant
>> Stddev: 0.02062 -> 0.02074: 1.0058x larger
>> Timeline: http://tinyurl.com/2fm39l2
>>
>> ### call_method_slots ###
>> Min: 0.872706 -> 0.867387: 1.0061x faster
>> Avg: 0.877261 -> 0.872754: 1.0052x faster
>> Significant (t=2.510615)
>> Stddev: 0.01523 -> 0.01586: 1.0410x larger
>> Timeline: http://tinyurl.com/3x84s9m
>>
>> ### call_method_unknown ###
>> Min: 1.031445 -> 1.028433: 1.0029x faster
>> Avg: 1.039063 -> 1.034296: 1.0046x faster
>> Not significant
>> Stddev: 0.03708 -> 0.03702: 1.0016x smaller
>> Timeline: http://tinyurl.com/395gevs
>>
>> ### call_simple ###
>> Min: 0.594110 -> 0.589934: 1.0071x faster
>> Avg: 0.606276 -> 0.594366: 1.0200x faster
>> Significant (t=5.874137)
>> Stddev: 0.01760 -> 0.01752: 1.0049x smaller
>> Timeline: http://tinyurl.com/2a2zv56
>>
>> ### django ###
>> Min: 0.997650 -> 0.993266: 1.0044x faster
>> Avg: 0.999423 -> 0.995495: 1.0039x faster
>> Significant (t=18.075408)
>> Stddev: 0.00093 -> 0.00122: 1.3050x larger
>> Timeline: http://tinyurl.com/28oa6wo
>>
>> ### float ###
>> Min: 0.102826 -> 0.102910: 1.0008x slower
>> Avg: 0.110088 -> 0.110280: 1.0017x slower
>> Not significant
>> Stddev: 0.02758 -> 0.02762: 1.0015x larger
>> Timeline: http://tinyurl.com/2w6ol8d
>>
>> ### hg_startup ###
>> Min: 0.045993 -> 0.044993: 1.0222x faster
>> Avg: 0.053388 -> 0.053510: 1.0023x slower
>> Not significant
>> Stddev: 0.00250 -> 0.00258: 1.0322x larger
>> Timeline: http://tinyurl.com/2ec392w
>>
>> ### iterative_count ###
>> Min: 0.157216 -> 0.156526: 1.0044x faster
>> Avg: 0.166971 -> 0.166897: 1.0004x faster
>> Not significant
>> Stddev: 0.06835 -> 0.07249: 1.0604x larger
>> Timeline: http://tinyurl.com/2g9agwl
>>
>> ### nbody ###
>> Min: 0.443087 -> 0.464941: 1.0493x slower
>> Avg: 0.456435 -> 0.475809: 1.0424x slower
>> Not significant
>> Stddev: 0.05609 -> 0.05523: 1.0156x smaller
>> Timeline: http://tinyurl.com/2wd6z8r
>>
>> ### normal_startup ###
>> Min: 0.438015 -> 0.437763: 1.0006x faster
>> Avg: 0.438425 -> 0.438810: 1.0009x slower
>> Not significant
>> Stddev: 0.00024 -> 0.00274: 11.6231x larger
>> Timeline: http://tinyurl.com/34nunk3
>>
>> ### nqueens ###
>> Min: 0.693033 -> 0.698259: 1.0075x slower
>> Avg: 0.698948 -> 0.704770: 1.0083x slower
>> Not significant
>> Stddev: 0.02644 -> 0.02590: 1.0208x smaller
>> Timeline: http://tinyurl.com/39ydyjs
>>
>> ### pickle ###
>> Min: 1.654750 -> 1.669246: 1.0088x slower
>> Avg: 1.660298 -> 1.673813: 1.0081x slower
>> Significant (t=-17.007317)
>> Stddev: 0.00391 -> 0.00403: 1.0298x larger
>> Timeline: http://tinyurl.com/36zz8yk
>>
>> ### pickle_dict ###
>> Min: 1.859310 -> 1.862217: 1.0016x slower
>> Avg: 1.864953 -> 1.863408: 1.0008x faster
>> Significant (t=2.269051)
>> Stddev: 0.00300 -> 0.00377: 1.2590x larger
>> Timeline: http://tinyurl.com/32kz4l6
>>
>> ### pickle_list ###
>> Min: 1.059003 -> 1.045209: 1.0132x faster
>> Avg: 1.065780 -> 1.048728: 1.0163x faster
>> Significant (t=21.791102)
>> Stddev: 0.00413 -> 0.00368: 1.1223x smaller
>> Timeline: http://tinyurl.com/27rpxol
>>
>> ### regex_compile ###
>> Min: 0.828427 -> 0.832179: 1.0045x slower
>> Avg: 0.890980 -> 0.894830: 1.0043x slower
>> Not significant
>> Stddev: 0.26185 -> 0.26180: 1.0002x smaller
>> Timeline: http://tinyurl.com/38c3z8m
>>
>> ### regex_effbot ###
>> Min: 0.162540 -> 0.162873: 1.0020x slower
>> Avg: 0.167092 -> 0.167389: 1.0018x slower
>> Not significant
>> Stddev: 0.02830 -> 0.02830: 1.0001x smaller
>> Timeline: http://tinyurl.com/33r7s5y
>>
>> ### regex_v8 ###
>> Min: 0.164368 -> 0.163174: 1.0073x faster
>> Avg: 0.417027 -> 0.416113: 1.0022x faster
>> Not significant
>> Stddev: 0.86580 -> 0.86190: 1.0045x smaller
>> Timeline: http://tinyurl.com/3yabd5v
>>
>> ### richards ###
>> Min: 0.352872 -> 0.353289: 1.0012x slower
>> Avg: 0.354989 -> 0.355434: 1.0013x slower
>> Not significant
>> Stddev: 0.00543 -> 0.00549: 1.0108x larger
>> Timeline: http://tinyurl.com/36weagp
>>
>> ### rietveld ###
>> Min: 0.693800 -> 0.695649: 1.0027x slower
>> Avg: 0.984689 -> 0.984075: 1.0006x faster
>> Not significant
>> Stddev: 0.36204 -> 0.36497: 1.0081x larger
>> Timeline: http://tinyurl.com/2wfw2z5
>>
>> ### slowpickle ###
>> Min: 0.772514 -> 0.757389: 1.0200x faster
>> Avg: 0.821586 -> 0.805534: 1.0199x faster
>> Not significant
>> Stddev: 0.17998 -> 0.18494: 1.0275x larger
>> Timeline: http://tinyurl.com/37rb8d5
>>
>> ### slowspitfire ###
>> Min: 1.022256 -> 1.023268: 1.0010x slower
>> Avg: 1.023244 -> 1.024326: 1.0011x slower
>> Significant (t=-6.305389)
>> Stddev: 0.00022 -> 0.00119: 5.3064x larger
>> Timeline: http://tinyurl.com/29h96r4
>>
>> ### slowunpickle ###
>> Min: 0.384167 -> 0.380310: 1.0101x faster
>> Avg: 0.410666 -> 0.409001: 1.0041x faster
>> Not significant
>> Stddev: 0.08844 -> 0.09114: 1.0305x larger
>> Timeline: http://tinyurl.com/286o5wt
>>
>> ### spambayes ###
>> Min: 0.417142 -> 0.398649: 1.0464x faster
>> Avg: 0.598665 -> 0.574443: 1.0422x faster
>> Not significant
>> Stddev: 0.58158 -> 0.57470: 1.0120x smaller
>> Timeline: http://tinyurl.com/23ozclq
>>
>> ### startup_nosite ###
>> Min: 0.333111 -> 0.332463: 1.0019x faster
>> Avg: 0.338470 -> 0.335088: 1.0101x faster
>> Significant (t=3.325272)
>> Stddev: 0.00886 -> 0.00500: 1.7726x smaller
>> Timeline: http://tinyurl.com/23ob2vy
>>
>> ### threaded_count ###
>> Min: 0.181111 -> 0.182459: 1.0074x slower
>> Avg: 0.214138 -> 0.211476: 1.0126x faster
>> Not significant
>> Stddev: 0.14987 -> 0.14531: 1.0314x smaller
>> Timeline: http://tinyurl.com/2dkupks
>>
>> ### unpack_sequence ###
>> Min: 0.000217 -> 0.000218: 1.0033x slower
>> Avg: 0.000223 -> 0.000222: 1.0048x faster
>> Significant (t=11.155116)
>> Stddev: 0.00002 -> 0.00001: 3.0782x smaller
>> Timeline: http://tinyurl.com/26tzfco
>>
>> ### unpickle ###
>> Min: 1.187517 -> 1.173118: 1.0123x faster
>> Avg: 1.212250 -> 1.178640: 1.0285x faster
>> Significant (t=9.457625)
>> Stddev: 0.02461 -> 0.00508: 4.8419x smaller
>> Timeline: http://tinyurl.com/2djz64c
>>
>> ### unpickle_list ###
>> Min: 1.106733 -> 1.086683: 1.0185x faster
>> Avg: 1.147736 -> 1.129444: 1.0162x faster
>> Significant (t=4.530626)
>> Stddev: 0.01237 -> 0.02573: 2.0800x larger
>> Timeline: http://tinyurl.com/2fctpe9
>>
>> Any thoughts?
>>
>> - Steven
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>




More information about the llvm-dev mailing list