Food for thought: supporting only the latest GHC version

[pepeiborra]

What would happen if HLS HEAD only supported the latest GHC version and we kept branches to back port fixes to previous releases? I am not suggesting that we make this change now, but it's worth sketching for long term planning.

Extended example of how this would work:

• HLS versioning would be changed to match GHC versioning. E.g. HLS 9.6 would support only GHC 9.6.
• Branches would be used for back ports. Only bug fixes, not new features or plugins.
• Tooling would automatically download the matching HLS version for the GHC install.

The benefit would be removing all GHC Compat stuff and related CPP. This would:

• greatly help with HLS sustainability
• facilitate GHC upgrades
• enable a not-so-hypothetical future where a version of HLS is bundled with GHC.
• encourage people to upgrade to the latest GHC version in order to get the latest HLS features and plugins

But it would also have serious downsides and obstacles:

• Possibly no version of HLS would be fully featured, in terms of plugins,
• Many plugins would never build in the HEAD branch,
• cabal install Hackage builds would not work due to head.hackage

This is a thought that came up in discussions with @david-christiansen during ICFP.

[michaelpj]

One question: do you meant that HLS HEAD develops against GHC HEAD, or HLS HEAD develops against the latest relased GHC? The latter would be much less painful, and I think get us most of the benefit of only working on a single major GHC version. Initial support would look a bit like it does today: we have to wait a bit after the release of GHC X until enough of the ecosystem works that we can actually build HLS, then we can switch to it.

Targeting GHC HEAD doesn't seem feasible given how many dependencies HLS has. head.hackage is good and all, but we'd be expecting a lot more from it than it is currently capable of.

---

I think in principle targeting the latest released GHC only could work if we were willing to backport a lot (and do releases from old branches!). I think we would have to do that because:

• We can't really say "you won't get most bugfixes unless you're on the latest GHC". Many bugfixes would apply cleanly, so backporting wouldn't be too bad.
• We usually don't have full compatibility of all plugins with GHC X by the time GHC X+1 is released. So in many cases plugin compatibility changes would have to target old branches.
• If our support for older versions of GHC is long-lived, then the old branches would also have to be long-lived. Backporting gets increasingly hard as things age, since it becomes more common that you want to backport a patch that relies on some refactoring commit or similar that you didn't backport. Or you have to backport really aggressively.

In fact, I think the old branches would in practice have to be active development branches. So fundamentally what this would do is move us from having one branch with some CPP, to multiple branches with no CPP. But then the contribution workflow is maybe worse: I don't love CPP but I really don't love the idea of having to land patches on multiple branches that might have diverged.

And in practice we don't have that much CPP: a quick count shows 94 files with CPP out of 1318 in the repository.

[pepeiborra]

And in practice we don't have that much CPP: a quick count shows 94 files with CPP out of 1318 in the repository.

How are you counting the files in the repository? I get 196 relevant files, which means more than half have CPP:

$ find . -not  -path "*test*" -not -path "./dist-newstyle*" -not -path "./bench/*" -not -path "./hie-compat/*" -name "*.hs" | wc -l
     196

[pepeiborra]

Tracking the latest released GHC would also let us remove all compatibility code, right? I just can't see us building HLS with GHC HEAD without either radically slimming down HLS or... IDK what.

Yes, fair enough. Tracking the latest GHC release is good enough as it removes all the compat. code and makes upgrading much simpler

[pepeiborra]

In fact, I think the old branches would in practice have to be active development branches. So fundamentally what this would do is move us from having one branch with some CPP, to multiple branches with no CPP. But then the contribution workflow is maybe worse: I don't love CPP but I really don't love the idea of having to land patches on multiple branches that might have diverged.

Yes, this is absolutely true, and indeed multiple development branches is worse than a single one with CPP. To avoid this we would need to move plugins out of the HLS repo, to allow them to be developed independently. But this would require hot code loading and I'm struggling to picture a user friendly deployment model for it.

[July541]

Just for a reminder, we only have four plugins that are not supported for 9.2 while 9.4 was released, dropping these plugins for 9.2 is acceptable I think.

That means it doesn't matter if we only support the latest GHC version. But I'm not keen on following GHC HEAD, it will make HLS bind with GHC deeply. Maybe we can switch to the newer one when the latest RC is available.

[michaelpj]

How are you counting the files in the repository? I get 196 relevant files, which means more than half have CPP:

Thank you, I'm being an idiot and counting dist-newstyle 🤦 I take it back, we have a lot of CPP :(

To avoid this we would need to move plugins out of the HLS repo, to allow them to be developed independently.

I'm not 100% sure I follow, but is the idea that then the plugin repos would need to have per-HLS release branches and manage their own backports that way, along with releasing off old release branches? That might be viable, although it would make the plugin maintenance workflow pretty ugly also.

[kokobd]

During the upgrade to GHC 9.2 and 9.4, a lot of CPP were added here and there. For the initial support of a GHC version, that's good enough, as it really takes a lot of work. But maybe we need to invest more time in cleaning up the existing CPPs and try our best to move them into their own modules.
For cases where a compatibility module is not possible (exact print changes in ghc 9.2 for example), we can move the code that directly uses those incompatible APIs into a standalone module and use something like if impl(ghc >= 9.2) in *.cabal file to choose between different module for different ghc versions.

[July541]

Off-topic: I'm reluctant to use CPP due to hls-hlint-plugin has poor support, and it will make code mess.

[michaelpj]

I'm coming around to this a bit more over time. It really would make our lives a lot easier.

A few more thoughts:

• We'd need to make our support for installing old-but-compatible releases of HLS for older versions of GHC very slick. It's pretty good with the vscode extension and ghcup, but I'm not sure how this would work for other packagers, who I think still assume that they can install the latest version of HLS and get a HLS+wrapper that works for all versions.
  • Generally, we'd have to rethink the wrapper. Maybe we could even drop it!
• @fendor pointed out that it's much less painful to consign people on old GHCs to old versions of HLS, since HLS is now pretty functional. Users of 8.10.7 being stuck on HLS 1.8.0 forever wouldn't be so bad.
• I'm still worried about the plugin story. Suppose you have a plugin that works fine on GHC X, but can't build on GHC X+1 due to a dependency issue. You want to do a version-independent bugfix to it. Where do you do that? You can't really do it on the main branch, since it doesn't build. So... you do it on the old-version branch and forward-port it on faith? Very unclear.
• Only having to produce bindists for a single version of GHC when we do a release would be very nice.