Add ADR for porting router and router-api to PostgreSQL

[chao-xian]

Rendered page

Trello

[leenagupte]

This should be in content-store. The gds-api-adapters gem is just helper methods, it shouldn't have "business" logic in it.

[leenagupte]

So I'd split this into two steps. One to add helper methods to gds-api-adapter to call p-router-api (this still needs a better name!) and the next step to call the new helper methods from content store.

[chao-xian]

We did consider doing it in content-store, but felt it much easier to implement and manage (temporarily) in the gem.

Good point on the splitting of steps.

[Tetrino]

@chao-xian the suggestion from me to do the logic in the API (as small as that logic is) was based on the theory that it's one solid point of entry for everything. @leenagupte if content store is the only location that actually accesses these helper calls, we could do it there.

However to clarify, there is no business logic as such in the API adaptor in this plan, we simply append a call to another new psql helper method inside the existing one (as is the plan).

[leenagupte]

Content Store is the only place that calls router / router-api. You'd be making a bit of a mess in gds-api-adapters if you try to have the router helpers call two places as you'd need to do some work to get it to figure out what the endpoint is.

[KludgeKML]

My opinion is that either the API or the content store way of doing it would be fine, but the question of where/when to report errors probably makes the decision.

Basically, if we do this splitting we need to be able to know about the following error cases:

• Neither router-api nor p-router-api got the update (PANIC)
• both got it (FINE)
• router-api got it but not p-router-api (FINE, WE HAVEN'T SPUN UP P-ROUTER / OH, WE HAVE TO INVESTIGATE)
• p-router-api got it but not router-api (UH, WHAT? / FINE, WE'VE TURNED OFF OLD ROUTER)

[chao-xian]

Great point @KludgeKML thanks!

[chao-xian]

@aldavidson has suggested running a proxy to duplicate the requests and to be able to log - which is a good idea it seems

[leenagupte]

@aldavidson has suggested running a proxy to duplicate the requests and to be able to log - which is a good idea it seems

So a new app that routes requests to both versions of router-api? And the proxy would be called by content store?

[chao-xian]

@leenagupte yes he's suggesting a new app, and we can use that same proxy to duplicate our calls too

[chao-xian]

Link to @aldavidson 's spike alphagov/content-store#1062

[aldavidson]

Hi both - yes, we're looking at a simple proxy in front of two copies of content-store (one running Mongo, one running Postgres) that would forward all requests to both, and log any differences in the responses.

That will let us dual-run for a while, and build up a decent history of database backups before switching. Also lets us manage the switchover in phases (Mongo primary, PG secondary -> PG primary, Mongo secondary -> just PG -> no more proxy). It's based on the approach taken by The Guardian when they ported their CMS from Mongo to PG.

If you're considering the same, we'll hopefully be able to write the proxy in such a way that it's usable for both migrations

[chao-xian]

See also alphagov/govuk-rfcs#158

[chao-xian]

We've started some spikes:

• https://github.com/alphagov/router-postgres
• https://github.com/alphagov/router-api-postgres

[chao-xian]

See also spike from Al alphagov/content-store#1062

[chao-xian]

See also from Al:

• https://github.com/alphagov/content-store-proxy
• [Do Not Merge] Spike dual run content store on postgresql govuk-docker#656

[leenagupte]

This sounds good. I think the only thing missing is any risks or consequences and any other approaches considered.

[leenagupte]

@chao-xian Sorry, it occurred to me later that the Decision and the outline were mislabelled.

[chao-xian]

Completed work by @aldavidson ready to be provisioned on platform:

• https://github.com/alphagov/content-store-proxy
• https://github.com/alphagov/content-store-on-postgresql
• [DO NOT MERGE] WIP - Substitute proxy hostnames govuk-docker#660