High CPU constantly (OpenBSD vmm machine)

[gonzalo-]

Background information

• Dendrite version or git SHA: 0.13.5
• SQLite3 or Postgres?: postgresql-server-16.2
• Running in Docker?:
• go version: go-1.22.1
• Client used (if applicable): Element

Description

• What is the problem: 90% of the CPU usage constantly.
• Who is affected: Since 0.13.5 I can see this problem
• How is this bug manifesting: Just right after start the service.
• When did this first appear: Never, well if I stop it :)

Steps to reproduce

I just upgrade the package, the previous version has no issue.

I got some logs from PPROF here

Disabled presence{enable_inbound,enable_outbound} doesn't fix the issue.

[S7evinK]

The provided pprof log is a bit difficult to read.
Can you get a CPU profile using wget -O 'profile.pprof' 'http://localhost:65432/debug/pprof/profile?seconds=10' (change port accordingly)?

Also, if you can, try updating to 0.13.7.

[gonzalo-]

@S7evinK thanks for replied:

Profile file here

I will try to update to the latest.

[S7evinK]

From what I can tell so far, the majority is spent in runtime/sys_openbsd3.go, which seems to be called from bleve (the fulltextsearch engine we use) and the underlying database bbolt

[gonzalo-]

@S7evinK any recommendation or possible fix?

[gonzalo-]

@S7evinK I can confirm that with 0.13.7 still the same behaviour.

[S7evinK]

Assuming you have fulltext search enabled, can you try disabling it? i.e. by setting sync_api.search.enabled to false in your Dendrite config.

[gonzalo-]

@S7evinK yes, that drops the CPU usage to 2%