Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ivirshup/census builder spatial #1245

Draft
wants to merge 65 commits into
base: main
Choose a base branch
from
Draft

Ivirshup/census builder spatial #1245

wants to merge 65 commits into from

Conversation

ivirshup
Copy link
Collaborator

@ivirshup ivirshup commented Jul 12, 2024 •
edited
Loading

TODO

  • Update source datasets links to ones which use schema v5.1.0
  • Update builder
    • Don't include normalized layer
    • Figure out what set of genes to use
    • populate spatial collection
      • Add a scene per ingested dataset
    • Add obs_scene table
    • Add var_scene table
    • Filter out objects with .uns["spatial"]["is_single"]==False (current filtering code seems to rely only on columns in obs)
  • Tests
    • Check equivalence of written data to input anndata
      • loc
      • images

prathapsridharan and others added 30 commits June 7, 2024 15:59
@prathapsridharan
Add setup instructions to integrate soma spatial
56bd44b
@prathapsridharan
Install git in census builder docker container
76c2a8a
@prathapsridharan
Add back pyarrow pin on builder for testing
a38c215
@prathapsridharan
Modify builder pins to test builder unit tests on GH
8ae84af
@prathapsridharan
Remove cellxgene_census package dependency for testing
b330183
@prathapsridharan
Pin to tiledbsoma commit to test builder unit tests
6655196
@prathapsridharan
Pin to tiledbsoma git commit for 1.9.5 to test
e91b2fa
@prathapsridharan
Unpin pyarrow in builder to test builder unit tests
4115601
@prathapsridharan
Pin to tiledbsoma git commit for 1.10.2 to test
1c7f985
@prathapsridharan
Pin to tiledbsoma git commit for 1.11.1 to test
4aa689e
@prathapsridharan
Pin tiledbsoma to 16f481f - head of spatial branch
327cee6
@prathapsridharan
Pin pyarrow back to 15.0.2
97ef790
@prathapsridharan
Pin tiledbsoma to fc5f8e7 to fix census builder tests
3567dc7
@prathapsridharan
Add comments to notebook
ba5ac8f
@prathapsridharan
Create notebook to demo census object creation
b448027
@prathapsridharan
Use absolute file path for contents of manifest file
0f1f20a
@prathapsridharan
Add "EFO:0010961" to the list of allowed assays
5c81aaa
@prathapsridharan
Add comments for clarity in pyproject.toml
ce4b525
@prathapsridharan
Fix filepaths in notebook
1f654b6
@prathapsridharan
Make census builder run without errors on spatial datasets
4712e95
@prathapsridharan
Add census_data and census_spatial collections
e7e6b07
@prathapsridharan
Update notebook
44c1109
@prathapsridharan
Pin tiledbsoma to commit 5069714 for latest spatial
1854f41
@prathapsridharan
Update tiledbsoma spatial notebook
14d682e
@prathapsridharan
Update tiledbsoma spatial notebook
bf0fdea
@prathapsridharan
Update tiledbsoma spatial notebook
ebbe1b1
@prathapsridharan
Pin tiledbsoma to commit 69d699e for latest spatial
ff73498
@prathapsridharan
Pin tiledbsoma to commit 9eb540f for latest spatial
a5c21a1
@prathapsridharan
Update tiledbsoma spatial notebook
2cef2bb
@ivirshup
Use latest commit from tiledbsoma spatial branch
03afeb4
@ivirshup
First attempt at writing images, current in progress
e4d2bf2
@ivirshup
Fix up writing of images
229379c
@ivirshup
Start test suite
953b710
@ivirshup
Write locations
61b901e
@ivirshup
Write obs_scene
833260d
@ivirshup
silence mypy
48cd6e3
@ivirshup
sorta update test
8499603
@ivirshup
update notebooks
8f8adcd
@ivirshup
Fix builds on CI by using hatch as build tool
3e504de
@ivirshup
Get build running for visium
cf9be22
@ivirshup
Add tests for images and obs_scene
0a257f9
@ivirshup
add notebook so I can change branches
a6989f0
@ivirshup
Allow manifests to contain s3 uris, add manifest folder
cb891e8
@ivirshup
Add manifest for slideseq
f39300b
@ivirshup
Merge branch 'main' into ivirshup/census-builder-spatial
41c7c18
@ivirshup
Add slide-seqv2 to allowed assays
d33c7dd
@ivirshup
Branch to allow for datasets that don't have associated images
9ca4327
@ivirshup
Use dask to write images + pin tiledbsoma to working commit
07e6e30
@ivirshup
Merge branch 'main' into ivirshup/census-builder-spatial
d6a1ab6
@ivirshup
Add support for scene class
cd96482
@ivirshup
Bump to newer commit on tiledbsoma branch
4a4316d
@ivirshup
Update pinned tiledbsoma version (not quite working)
6c86a75
@ivirshup
Merge branch 'main' into ivirshup/census-builder-spatial
5b9f70a
@ivirshup
Merge branch 'main' into ivirshup/census-builder-spatial, plus update…
8e6fa76 
… test datasets
@ivirshup
Merge branch 'main' into ivirshup/census-builder-spatial
1058956
@ivirshup
Fix write_obs/var_dataframe for cases where that dataframe is None
f9f72b8
@ivirshup
Working again! (mostly)
ef720da
@ivirshup
Add tiledb dep back
4241c9d
@ivirshup
update manifests
a4b34c1
@ivirshup ivirshup linked an issue Oct 31, 2024 that may be closed by this pull request
Draft Census schema for support of Visium and Spatial data #1092
Open
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Draft Census schema for support of Visium and Spatial data
2 participants
@ivirshup @prathapsridharan