build takes high memory usage

[tfoote]

We've documented jekyll taking ~6GB of RAM, which is enough to run our build agents out of memory if they're running with other processes.

From ros-infrastructure/ros_buildfarm_config#123 (comment)

http://build.ros.org/job/doc_rosindex/5/console

The build of the index took 60% of the RAM on an 8GB machines

This caused 2 consecutive build agents under load to fall over due to OOM errors.

I monitored memory during the build and have notes here:
https://gist.github.com/tfoote/9af6d68ebe4dc077691c2d03718617e9

There's discussion of potential mitigation techniques here:
ros-infrastructure/ros_buildfarm_config#123 (comment)

Related issues from Jekyll w/o any resolution:
jekyll/jekyll#560
jekyll/jekyll#1140

[tfoote]

Comment from #207 (comment)

We're also doing our own pagination where there is a plugin for this that might be more efficient: https://rubygems.org/gems/jekyll-paginate-v2 it's known to be noteably better than the older default pagination

I found a good discussion of performance here: https://boris.schapira.dev/2018/11/jekyll-build-optimization/ though it's a little bit dated.

Also similar recommendations here: https://forestry.io/blog/how-i-reduced-my-jekyll-build-time-by-61/

Liquid-c looks to be still active: https://github.com/Shopify/liquid-c

include-cached might be valuable too: https://github.com/benbalter/jekyll-include-cache