Handle releases for commits and pull requests #20
Conversation
.github/get_latest_index.py
Outdated
| def get_latest_idc_release_version(self, view_id): | ||
| print("Getting latest IDC release version...") | ||
| view = self.client.get_table(view_id) | ||
| latest_idc_release_version=int(re.search(r"idc_v(\d+)", view.view_query).group(1)) | ||
| return latest_idc_release_version |
There was a problem hiding this comment.
After browsing https://console.cloud.google.com/marketplace/product/bigquery-public-data/nci-idc-data?project=just-experience-186619
I was able to browser through the table bigquery-public-data.idc_current.dicom_all_view:
That said, I couldn't see the version information we are exacting there. Do you know where are these organized ?
There was a problem hiding this comment.
JC, it is a long story. idc_current BQ dataset contains views defined by queries that point to the latest version of the IDC release, which in turn are stored in the datasets named as bigquery-public-data.idc_v<release_number>, so currently the latest one is idc_v17. There is a known bug with the underlying mechanism from Google that we use that does not expose the actual query behind the view. In the past, you could find it in the Details section of the view. Unfortunately, we do not have any estimate from Google on when this would be fixed. Here's the tracker, but unfortunately it is not public - we will try to make it public: https://issuetracker.google.com/issues/324112186.
Fortunately, this is not a stopper, since we can list all of the idc_v* datasets to get the latest version. See relevant discussion in https://github.com/ImagingDataCommons/idc-index/issues/11#issuecomment-1946308527 - @vkt1414 is going to work on this after we are done with the current time-sensitive activity that is a priority (should be done in 1-2 weeks).
cc: @bcli4d for awareness
There was a problem hiding this comment.
Thanks for the explanation, that makes sense, and we can indeed see the versioned table listed:
There was a problem hiding this comment.
@jcfr I actually found a better way to get the version info. Sorry I'm behind but I'm planning to work on this gha soon.
There was a problem hiding this comment.
@vkt1414 yes, this is indeed a more elegant and I agree a better approach!
|
Closing this PR as there is CI/CD already now |
This gha will address task 2 of ImagingDataCommons/idc-index-data#2