kong taking longer than expected to reload configurations after local cache purge

[shishirkh]

We've observed that after a local cache is purged in the Kong data plane pods, the following log entries are generated:

2024/08/10 08:32:00 [notice] 1262#0: *179271 [lua] init.lua:259: purge(): [DB cache] purging (local) cache, context: ngx.timer
2024/08/10 08:32:00 [notice] 1262#0: *3 [lua] init.lua:259: purge(): [DB cache] purging (local) cache, context: ngx.timer

After this purge, we've noticed that it takes longer than expected for Kong to reload the configurations. During this period, all requests received by Kong error out until the data plane is able to fetch the configurations from the control plane again.

• What could be causing this delay in configuration reload?
• Are there any known issues with cache purging in this version of Kong?
• Could there be an issue with the communication between the data plane and control plane during this period?

Looking for insights and suggestions on how to troubleshoot or resolve this issue.

[shishirkh]

Adding a few more data:

• CPU and memory of all kong pods (data plane, control plane) are within acceptable limits.

• Our kong configuration does not involve any plugin or custom logic, we are using this kong just for routing calls to different microservices.

• This kong has been stable but recently this issue has come up. No Kong configuration like nginx parameters has been changed recently. Hence, configuration-related issues are unlikely the culprit here.

[ProBrian]

Hi @shishirkh, What's the version of Kong you're using?

[ProBrian]

Hi @bungle , could you help to make sure the purging of cache leading to longer time on config reloading is as expected?

[sharavara]

we have the same issue

time="2024-09-13T06:34:46Z" level=info msg="successfully synced configuration to Kong" update_strategy=InMemory url="https://localhost:8444"
2024/09/13 06:34:46 [notice] 1259#0: *4 [lua] init.lua:240: purge(): [DB cache] purging (local) cache, context: ngx.timer
2024/09/13 06:34:46 [notice] 1259#0: *4 [lua] init.lua:240: purge(): [DB cache] purging (local) cache, context: ngx.timer
2024/09/13 06:34:46 [notice] 1260#0: *151 [lua] init.lua:240: purge(): [DB cache] purging (local) cache, context: ngx.timer
2024/09/13 06:34:46 [notice] 1260#0: *151 [lua] init.lua:240: purge(): [DB cache] purging (local) cache, context: ngx.timer

kong/kubernetes-ingress-controller:2.11
kong:3.3