Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement shifts and tilts as properties #15

Merged
merged 21 commits into from
Oct 30, 2024

Conversation

jat255
Copy link
Member

@jat255 jat255 commented Oct 29, 2024

Following up from our chat last week... This PR does a few things:

  • Converts shifts and tilts for stacks into properties on the CommonStack class
    • These are implemented as a couple new signals TomoTilts and TomoShifts
    • Done primarily so we can use the _additional_slicing_targets functionality from HyperSpy to take care of inav and isig slicing of these properties automatically
    • These values are dumped to the metadata and loaded back in as properties when saving/loading to hdf5
  • Removes the create_stack() method in favor of using the TomoStack() constructor directly
  • Implements full coverage of the entire application via tests (including CUDA code)
  • Adds a missing cupy dependency in the optional [gpu] group (optional, since CuPy is not supported on MacOS)
  • Bumps version to 0.9.1

A note about astra and CIL... I tried to downgrade our astra dependency to 2.1.X to be compatible with CIL, which worked, but caused lots of other packaging grief because the earlier versions of astra are not on PyPI, so things would only work with conda in that case. It also required downgrading numpy to < 2.0. We can revisit if needed, but I'm hoping CIL may be able to update their dependency to the latest version without too much trouble.

Remaining todos after this PR:

  • rig up hyperspy extension functionality
  • publish packages to PyPI and conda-forge

Here is the test output on my CUDA system:

 ❯ poetry run ./run_tests.sh 
Using etspy/tests/.coveragerc-cuda
=============================================================================================== test session starts ================================================================================================
platform linux -- Python 3.10.12, pytest-8.3.3, pluggy-1.5.0
rootdir: /home/jat/git_repos/etspy
configfile: pyproject.toml
testpaths: etspy/tests
plugins: cov-5.0.0
collected 246 items

etspy/tests/test_align.py ...............................                                                                                                                                                    [ 12%]
etspy/tests/test_base.py ......................................................................................................................                                                              [ 60%]
etspy/tests/test_cuda.py ...........                                                                                                                                                                         [ 65%]
etspy/tests/test_datasets.py ......                                                                                                                                                                          [ 67%]
etspy/tests/test_io.py ............................                                                                                                                                                          [ 78%]
etspy/tests/test_recon.py ..................                                                                                                                                                                 [ 86%]
etspy/tests/test_simulation.py ..............                                                                                                                                                                [ 91%]
etspy/tests/test_utils.py ...................                                                                                                                                                                [ 99%]
etspy/tests/test_version.py .                                                                                                                                                                                [100%]

================================================================================================= warnings summary =================================================================================================
etspy/tests/test_simulation.py::TestModifications::test_add_noise_gaussian
  /home/jat/git_repos/etspy/etspy/tests/test_simulation.py:110: RuntimeWarning: divide by zero encountered in divide
    stack.data[1:].mean(axis=(1,2)) / stack.data[1:].std(axis=(1,2))

etspy/tests/test_simulation.py::TestModifications::test_add_noise_poissanian
  /home/jat/git_repos/etspy/etspy/tests/test_simulation.py:124: RuntimeWarning: divide by zero encountered in divide
    stack.data[1:].mean(axis=(1,2)) / stack.data[1:].std(axis=(1,2))

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

---------- coverage: platform linux, python 3.10.12-final-0 ----------
Name                  Stmts   Miss  Cover   Missing
---------------------------------------------------
etspy/__init__.py        33      0   100%
etspy/align.py          369      0   100%
etspy/api.py             10      0   100%
etspy/base.py           632      0   100%
etspy/datasets.py        18      0   100%
etspy/io.py             236      0   100%
etspy/recon.py          215      0   100%
etspy/simulation.py     124      0   100%
etspy/utils.py          174      0   100%
---------------------------------------------------
TOTAL                  1811      0   100%
Coverage HTML written to dir etspy/tests/htmlcov
Coverage XML written to file etspy/tests/coverage.xml

=================================================================================== 246 passed, 2 warnings in 102.41s (0:01:42) ====================================================================================
>>> elapsed time 1m44s

update version to dev version; add script to run tests with conditional
coverage depending on presence of CUDA.
  - store tilts/shifts in stack.tilts and stack.shifts properties
  - create Signal1D subclasses for TomoTilts and TomoShifts with shape validation and custom slicers
  - remove io.create_stack and implement in CommonStack constructor
  - override copy and deepcopy for CommonStack so tilts and shifts are copied
  - override save for CommonStack so tilts and shits are stashed in the metadata when writing to disk
- add more tests for tilts and shifts
- get to 100% coverage
- fix CommonStack signal constructor
- better support for multiframe stacks
- fix save_movie dimensions
- fix plot_sinos due to new tilts property
- make run_test.sh script remove existing coverage reports
make workflow choice of conda environment condtional
@AndrewHerzing
Copy link
Collaborator

Tested Windows install via yml with conda on a non-CUDA machine. All tests are passing.

Tested Linux install via yml with conda on a CUDA-capable machine. All tests are passing.

@AndrewHerzing AndrewHerzing merged commit 2496f3b into usnistgov:master Oct 30, 2024
15 checks passed
@jat255 jat255 deleted the property_api branch October 30, 2024 00:59
Comment on lines +5 to +6
- nvidia
- defaults
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the nvidia and defaults channels necessary here. In the past, defaults and conda-forge had binaries incompatibilities and I suspect that this is still the case. The cudatoolkit stack in conda-forge is in very good shape and works well, using conda-forge only would avoid any potential issue.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was originally copied from the astra instructions: https://astra-toolbox.com/docs/install.html#linux-windows-using-conda-for-python and since it worked, I did not change it. I'll play around with it in my upcoming changes and loosen it to just conda-forge if that works.

resources/etspy-gpu.yml Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants