6 There's currently a nasty memory leak in
7 [haskell-ide-engine](https://github.com/haskell/haskell-ide-engine) which leaks
8 all the cached information about a module from GHC. It seems to only occur on
9 some platforms, but on platforms unfortunate to be affected by it, it means that
10 a sizeable portion memory is leaked every time the user types.
11 For a small module of about 60 lines, this was 30-ishMB.
13 During this year's ZuriHac, Matthew Pickering, Daniel Gröber and I tried to get
14 this sorted out once and for all: It ended up taking the entire weekend!
17 The first step in the investigation began with figuring out what exactly was
18 leaking: Prior to ZuriHac, I had heap profiled HIE with a sample session. To do
19 this I had to build the executable with profiling enabled: This adds some extra
20 information regarding closures and "call sites", and causes your executable to
21 be linked with one of the versions of the RTS that was built with
26 # or in our case, for cabal projects
27 cabal v2-install :hie --enable-profiling
30 GHC comes with a bunch of different RTS libraries, each built with a
31 different combination of features.
34 ls `ghc --print-libdir`/rts | grep rts
35 libHSrts-ghc8.6.5.dylib
37 libHSrts_debug-ghc8.6.5.dylib
39 libHSrts_l-ghc8.6.5.dylib
42 libHSrts_thr-ghc8.6.5.dylib
44 libHSrts_thr_debug-ghc8.6.5.dylib
46 libHSrts_thr_l-ghc8.6.5.dylib
51 `thr` stands for threading, `debug` for debug, `l` for ??? and `p` for
52 profiling. If you pass GHC `-v`, you should see
53 `-lHSrts_thr_p` or equivalent when linking: We're bringing in the RTS built with
54 threading and profiling enabled.
56 Now we can profile our executable by running it with specific flags passed to
57 the RTS. To see what's available, we ran `hie +RTS --help`
60 hie: -h<break-down> Heap residency profile (hp2ps) (output file <program>.hp)
61 hie: break-down: c = cost centre stack (default)
63 hie: d = closure description
64 hie: y = type description
66 hie: b = biography (LAG,DRAG,VOID,USE)
72 Now that we know what is leaking, the question turns to where is it leaking.
73 The next steps are taken from [Simon Marlow's excellent blog
74 post](http://simonmar.github.io/posts/2018-06-20-Finding-fixing-space-leaks.html),
75 in which he details how he tracked down and fixed multiple memory leaks in GHCi.
77 The trick is to create a weak pointer -- a pointer that the garbage collector
78 ignores when looking for retainers. Point it to the offending type and store it
79 alongside the original, strongly referenced one.
80 Whenever you know the object should have been dellocated, force garbage
81 collection by calling `performGC` and dereference the weak pointer. If the
82 pointer is null, it was released, otherwise something is still holding onto it:
83 the object is being leaked!
89 (lldb) e findPtr((P_)myobjptr & ~0b111, 1)
90 0x420bbdec50 = hie-plugin-api-0.10.0.0-inplace:Haskell.Ide.Engine.GhcModuleCache.UriCache(0x420bbc04f9, 0x420fdff691, 0x420fdfd39a, 0x10cca094a, 0x420bbc0539)
91 0x420fdfd150 = WEAK(key=0x420fdfd39a value=0x420fdfd39a finalizer=0x10c56ba41)
92 0x420fdfd150 = WEAK(key=0x420fdfd39a value=0x420fdfd39a finalizer=0x10c56ba41)
96 -debug -g -fwhole-archive-hs-libs