Dat/Beaker caches gigabytes of data (Windows only)

This is just a thread to help explain why Dat (and Beaker) can consume a huge amount of disk space on Windows (ReFS/NTFS file systems). Skip to the last paragraph for a “fix”.

Dat uses sparse files — files with large sections of null bytes — to power partial downloads (e.g. a website but without all the Dat archive’s version history). However, the implementation is lacking support for NTFS-style sparse files as used in Windows.

This problem can cause a 200 kilobyte page you’ve downloaded through Beaker to take up megabytes or even gigabytes on disk (the full size of the Dat archive even though you’ve only downloaded 200 kB!)

It means browsing the web with Beaker can quickly fill up a small storage drive. This can also affect writing performance on hard drives (I don’t believe flash storage is affected) as huge files filled with null-bytes are unnecessarily written to disk.

This problem only affects Windows and doesn’t affect macOS or Linux unless sparse files have been disabled in the file system.

To free up space: Open Settings in Beaker, click Dat network activity, Cache, and click Clear cache. You may also want to remove some seeded (or trashed) Dat archives to free up space.

2 Likes

Thanks for the #protip! Saved me a couple hundred megabytes, which is pretty significant on my dinky hard drive.