Error during preprocessing: The database file is locked

[resolritter]

Describe the bug

When I try to use rga, for example as rga foo /home/user, it fails with the following message:

/home/user/foo.pdf adapter: poppler
Error: during preprocessing
Caused by:
    0: run_adapter(/home/user/foo.pdf)
    1: connect_pragmas
    2: Rusqlite("database is locked")
    3: database is locked
    4: Error code 5: The database file is locked

To work around the bug I had to do rm -rf ~/.cache/ripgrep-all. After that, rga foo /home/user began to work normally.

To Reproduce

I don't know

Operating System and Version

x86_64 Arch Linux

uname -a: Linux user 6.4.12-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 24 Aug 2023 00:38:14 +0000 x86_64 GNU/Linux

Output of rga --version

ripgrep-all 1.0.0-alpha.5

[Ma27]

So far I only encountered this after switching from 0.9.6 to 1.0.0-alpha.5 and after deleting the cache it was fine for me as well. If that only occurs after the upgrade it may be an option to clear the cache for users coming from an older release @phiresky ?

[phiresky]

Yeah this can only appear in 1.0 plus since it's an sqlite error and sqlite wasn't used before.

Did this error happen on the first run after upgrading or later? Did this error persist after one run is finished? Deleting the cache shouldn't really affect it since it's (usually) an error that happens if parallel processes write to the cache at the same time.

[Ma27]

On the ~first run of rga in a directory with PDF files (that contained the string I searched for) I encountered this error. After finding this issue I applied the "workaround" and the issue disappeared, but given your explanation this might've been luck then %)

[Rojetto]

I'm having the same issue on x86_64 Windows, also ripgrep-all 1.0.0-alpha.5. It usually happens when running rga on a big directory. I also notice a rogue instance of rga-preproc.exe hanging around in the task manager afterwards, and I can only delete the cache (which doesn't seem to help) after force closing that process.

[phiresky]

I also notice a rogue instance of rga-preproc.exe hanging around in the task manager afterwards @Rojetto

Is there any really huge file in that directory that might be taking multiple minutes to process? rga exits before the rga-preproc process?

[phiresky]

It would be good to hear any more detail from someone encountering this. I've changed the code around this a bit but also I haven't really been able to reproduce this myself.

[Rojetto]

I'm on 0.10.6 now and tried again, same issue unfortunately.

Is there any really huge file in that directory that might be taking multiple minutes to process? rga exits before the rga-preproc process?

No, there isn't. All of the files are a few MB max, no single file takes multiple minutes.

In my specific use-case, I have a bunch of ZIP files (with a custom extension, like JAR) which contain XML files whose content I want to search. I'm running rga.exe --rga-accurate -g '*.piz' "SEARCH_STRING". Error message that is output thousands of times is

path\foo.piz: preprocessor command failed: '"path\\rga-preproc" "path\\foo.piz"':
-------------------------------------------------------------------------------
path\foo.piz adapter: postprocprefix
Error: during preprocessing

Caused by:
    0: run_adapter(path\foo.piz)
    1: Rusqlite("database is locked")
    2: database is locked
    3: Error code 5: The database file is locked