You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In discussion with @johanneskoester it came up that the metadata.pipeline_interfaces section could be considered looper-specific. There are several other looper-specific sections (which I've already divided out in the docs), but this one is unique because it's part of the metadata section.
What about collecting all of these things into a new looper section of the project config?
The rationale is, each tool that reads a PEP could specify an optional section and do what it wants there. Then you just need to define that section in your PEP if you're using that tool. If it made sense, there could equivalently be introduced a snakemake section that could be encode app-specific project configuration (if any exists).
So I guess what I'm proposing is to group the sections for pipeline_args, pipeline_config, and compute(maybe?), plus the attribute metadata.pipeline_interfaces, into a new section called looper. I guess the point of this is to have a home for metadata.pipeline_interfaces. I guess metadata.results_subdir and metadata.submission_subdir also belong in this class of attributes, so they would follow pipeline_interfaces.
Just a thought. We can also just leave it how it is, which is optional and under metadata.
I really like that grouping and separation. I'm fine with it being called looper and think that makes the most sense for now since that's the primary consumer of a pep, but I think that we could even call it something like engine, consumer, or executor with the idea that a pep could also be used by a looper analogue that handles submission of samples to pipelines as defined by a pep.
Yep, makes sense those would be under looper. I think that as you point out, a pep should not necessarily be restricted in what sections it have and could allow for any tool that uses a pep to extend it as long as the minimum is there.
In discussion with @johanneskoester it came up that the
metadata.pipeline_interfaces
section could be considered looper-specific. There are several other looper-specific sections (which I've already divided out in the docs), but this one is unique because it's part of themetadata
section.What about collecting all of these things into a new
looper
section of the project config?The rationale is, each tool that reads a PEP could specify an optional section and do what it wants there. Then you just need to define that section in your PEP if you're using that tool. If it made sense, there could equivalently be introduced a
snakemake
section that could be encode app-specific project configuration (if any exists).So I guess what I'm proposing is to group the sections for
pipeline_args
,pipeline_config
, andcompute
(maybe?), plus the attributemetadata.pipeline_interfaces
, into a new section calledlooper
. I guess the point of this is to have a home formetadata.pipeline_interfaces
. I guessmetadata.results_subdir
andmetadata.submission_subdir
also belong in this class of attributes, so they would followpipeline_interfaces
.Just a thought. We can also just leave it how it is, which is optional and under
metadata
.What do you think, @afrendeiro, @vreuter ?
The text was updated successfully, but these errors were encountered: