Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use canonical path for calls to ZarrService #57

Merged
merged 4 commits into from
Jul 7, 2023
Merged

Use canonical path for calls to ZarrService #57

merged 4 commits into from
Jul 7, 2023

Conversation

dgault
Copy link
Member

@dgault dgault commented Jun 27, 2023

This PR is open for the purpose of testing the OMERO symlink issues.

Although we do not want to be using canonical paths within the reader, it seems that doing so when calling jzarr, does seem to resolve the issues and exceptions seen. I would expect a better long term solution would be to resolve this either at the JZarr level or within the ZarrService which call JZarr.

@dgault
Copy link
Member Author

dgault commented Jun 27, 2023

Unit test failures, investigating now

@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#535. See the console output for more details.
Possible conflicts:

--conflicts

@joshmoore
Copy link
Member

Ah, @dgault, looks like we will need to start excluding or merging things.

@snoopycrimecop
Copy link
Member

snoopycrimecop commented Jul 5, 2023 •
edited
Loading

Conflicting PR. Removed from build BIOFORMATS-push#29. See the console output for more details.
Possible conflicts:

--conflicts Conflict resolved in build BIOFORMATS-push#537. See the console output for more details.

@ome ome deleted a comment from snoopycrimecop Jul 5, 2023
@joshmoore joshmoore mentioned this pull request Jul 5, 2023
Closed
@joshmoore
Copy link
Member

The exception is gone, but for some reason the files are still not being detected. For /tmp/b.ome.zarr which is a symlink to /tmp/a.ome.zarr:

(omero) [jamoore@pilot-idr0125-omeroreadwrite ~]$ omero import -f --depth=20  /tmp/b.ome.zarr/
Using OMERO.java-5.6.3-346-5b8271d-ice36-b236
2023-07-05 09:34:00,594 285        [      main] INFO          ome.formats.importer.ImportConfig - OMERO.blitz Version: 5.6.3-SNAPSHOT
2023-07-05 09:34:00,611 302        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 6.13.1-SNAPSHOT revision: c86fa185c70978f4d9eeacf5177c4fc0c3fa6343 date: 5 July 2023
2023-07-05 09:34:00,699 390        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2023-07-05 09:34:01,041 732        [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 20 Metadata Level: MINIMUM
2023-07-05 09:34:01,716 1407       [      main] INFO      ome.formats.importer.ImportCandidates - 17 file(s) parsed into 0 group(s) with 5 call(s) to setId in 668ms. (675ms total) [0 unknowns]

as opposed to /tmp/a.ome.zarr:

(omero) [jamoore@pilot-idr0125-omeroreadwrite ~]$ omero import -f --depth=20  /tmp/a.ome.zarr/
Using OMERO.java-5.6.3-346-5b8271d-ice36-b236
2023-07-05 09:33:14,949 330        [      main] INFO          ome.formats.importer.ImportConfig - OMERO.blitz Version: 5.6.3-SNAPSHOT
2023-07-05 09:33:14,968 349        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 6.13.1-SNAPSHOT revision: c86fa185c70978f4d9eeacf5177c4fc0c3fa6343 date: 5 July 2023
2023-07-05 09:33:15,049 430        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2023-07-05 09:33:15,445 826        [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 20 Metadata Level: MINIMUM
2023-07-05 09:33:16,004 1385       [      main] INFO      ome.formats.importer.ImportCandidates - 17 file(s) parsed into 1 group(s) with 1 call(s) to setId in 555ms. (559ms total) [0 unknowns]
#======================================
# Group: /tmp/a.ome.zarr/OME/METADATA.ome.xml SPW: false Reader: loci.formats.in.ZarrReader
/tmp/a.ome.zarr/OME/METADATA.ome.xml
/tmp/a.ome.zarr/OME/.zgroup
/tmp/a.ome.zarr/OME/.zattrs
/tmp/a.ome.zarr/.zgroup
/tmp/a.ome.zarr/.zattrs
/tmp/a.ome.zarr/0/0/.zarray
/tmp/a.ome.zarr/0/0/0/0/0/0/0
/tmp/a.ome.zarr/0/1/.zarray
/tmp/a.ome.zarr/0/1/0/0/0/0/0
/tmp/a.ome.zarr/0/.zgroup
/tmp/a.ome.zarr/0/.zattrs
/tmp/a.ome.zarr/1/0/.zarray
/tmp/a.ome.zarr/1/0/0/0/0/0/0
/tmp/a.ome.zarr/1/1/.zarray
/tmp/a.ome.zarr/1/1/0/0/0/0/0
/tmp/a.ome.zarr/1/.zgroup
/tmp/a.ome.zarr/1/.zattrs

@joshmoore
Copy link
Member

From a private message to David, showinf was showing this for a symlinked directory:

filename = /tmp/b.ome.zarr/.zattrs
************ invalid used files list ************
Used files = []

@joshmoore
Copy link
Member

With the new commit:

filename = /tmp/b.ome.zarr/.zattrs
Used files:
	/tmp/b.ome.zarr/OME/METADATA.ome.xml
	/tmp/b.ome.zarr/OME/.zgroup
	/tmp/b.ome.zarr/OME/.zattrs
	/tmp/b.ome.zarr/.zgroup
	/tmp/b.ome.zarr/.zattrs
	/tmp/b.ome.zarr/0/0/.zarray
	/tmp/b.ome.zarr/0/0/0/0/0/0/0
	/tmp/b.ome.zarr/0/1/.zarray
	/tmp/b.ome.zarr/0/1/0/0/0/0/0
	/tmp/b.ome.zarr/0/.zgroup
	/tmp/b.ome.zarr/0/.zattrs
	/tmp/b.ome.zarr/1/0/.zarray
	/tmp/b.ome.zarr/1/0/0/0/0/0/0
	/tmp/b.ome.zarr/1/1/.zarray
	/tmp/b.ome.zarr/1/1/0/0/0/0/0
	/tmp/b.ome.zarr/1/.zgroup
	/tmp/b.ome.zarr/1/.zattrs

and

(omero) [jamoore@pilot-idr0125-omeroreadwrite jars]$ omero import -f --depth=20  /tmp/b.ome.zarr/
Using OMERO.java-5.6.3-346-5b8271d-ice36-b236
2023-07-06 09:29:18,640 299        [      main] INFO          ome.formats.importer.ImportConfig - OMERO.blitz Version: 5.6.3-SNAPSHOT
2023-07-06 09:29:18,656 315        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 6.13.1-SNAPSHOT revision: c86fa185c70978f4d9eeacf5177c4fc0c3fa6343 date: 5 July 2023
2023-07-06 09:29:18,725 384        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2023-07-06 09:29:19,074 733        [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 20 Metadata Level: MINIMUM
2023-07-06 09:29:19,620 1279       [      main] INFO      ome.formats.importer.ImportCandidates - 17 file(s) parsed into 1 group(s) with 1 call(s) to setId in 541ms. (546ms total) [0 unknowns]
#======================================
# Group: /tmp/b.ome.zarr/OME/METADATA.ome.xml SPW: false Reader: loci.formats.in.ZarrReader
/tmp/b.ome.zarr/OME/METADATA.ome.xml
/tmp/b.ome.zarr/OME/.zgroup
/tmp/b.ome.zarr/OME/.zattrs
/tmp/b.ome.zarr/.zgroup
/tmp/b.ome.zarr/.zattrs
/tmp/b.ome.zarr/0/0/.zarray
/tmp/b.ome.zarr/0/0/0/0/0/0/0
/tmp/b.ome.zarr/0/1/.zarray
/tmp/b.ome.zarr/0/1/0/0/0/0/0
/tmp/b.ome.zarr/0/.zgroup
/tmp/b.ome.zarr/0/.zattrs
/tmp/b.ome.zarr/1/0/.zarray
/tmp/b.ome.zarr/1/0/0/0/0/0/0
/tmp/b.ome.zarr/1/1/.zarray
/tmp/b.ome.zarr/1/1/0/0/0/0/0
/tmp/b.ome.zarr/1/.zgroup
/tmp/b.ome.zarr/1/.zattrs

👍

@joshmoore joshmoore mentioned this pull request Jul 6, 2023
Closed
@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#31. See the console output for more details.
Possible conflicts:

--conflicts

@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#27. See the console output for more details.
Possible conflicts:

--conflicts

@joshmoore joshmoore merged commit 7f2c1de into ome:main Jul 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@joshmoore joshmoore joshmoore approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@dgault @snoopycrimecop @joshmoore