Random read error on google cloud.

[stefangroha]

I am using SeqLib to read in bam/cram files on a cloud instance. After compiling the library with gcloud support, most of the time reading in bam files works, however at seemingly random times and for random files (always different when rerunning script), I get the following error:

[E::hts_open_format] Failed to open file : Operation not permitted

The next time I run it it does fine with that same file and other parallel instances are doing fine with that file as well. Just retrying in the same instance gives me the same error repeatedly.

Could there be an issue reading in that file while another instance is reading in the same file? Does htslib set some lock on the file?

Any input or ideas how to debug are greatly appreciated!

[jkbonfield]

Could this be the same issue as samtools/samtools#1622? Does it happen mainly after having been running for a while?

Our assumption there is it is a connection timeout and that we need to add support for automatically reconnecting.

[stefangroha]

That's possible. It does happen mainly if it's been running for a while or for larger files..

[jkbonfield]

Ok, let's hope when we fix the other issue (ongoing) then your issues also get fixed. We'll let you know when a new version is available to test, or you can monitor the other discussion if you wish.

[stefangroha]

Great, thanks so much! I will monitor the discussion.