You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To address #841 (comment) I suggest we add a raw_training_scores accessor function, following the established pattern for feature_importances and training_losses.
@davnn Are you happy with the name of this function? I'm thinking here of consistency with training_losses.
Step 1. Prepare a bunch of PR's to be rolled out after testing, all called scores:
Add reports_raw_training_scores trait to StatisticalTraits, defaulting to false, as here for feature_importances (but need to also add to list of all traits at top of file)
Add raw_training_scores(model, fitresult, report) stub to MLJModelInterface (in model_api.jl), as here; fallback to return nothing.
Overload MLJModelInterface.raw_training_scores(mach::Machine) following this patternand re-export raw_training_scores (the reports_raw_training_scores trait will be automatically re-exported in MLJBase and MLJ - no action required)
In MLJ: re-export raw_training_scores
Step 2
@blaom to check MLJTestIntegration runs with no new surprises (and maybe adds some detector-specific testing)
Step 3
Merge and rollout the scores PRs and tags new releases, taking care to bump compats accordingly. Probably @ablaom to do.
Update MLJ model API docs
Step 4
Roll out implementations for packages that already report
scores (in, eg, their report); @davnn to complete this checklist:
OutlierDetection.jl
?
The text was updated successfully, but these errors were encountered:
To address #841 (comment) I suggest we add a
raw_training_scores
accessor function, following the established pattern forfeature_importances
andtraining_losses
.@davnn Are you happy with the name of this function? I'm thinking here of consistency with
training_losses
.Here are the steps (adapted from #747).
Step 1. Prepare a bunch of PR's to be rolled out after testing, all called
scores
:reports_raw_training_scores
trait to StatisticalTraits, defaulting tofalse
, as here forfeature_importances
(but need to also add to list of all traits at top of file)raw_training_scores(model, fitresult, report)
stub to MLJModelInterface (in model_api.jl), as here; fallback to returnnothing
.MLJModelInterface.raw_training_scores(mach::Machine)
following this pattern and re-exportraw_training_scores
(thereports_raw_training_scores
trait will be automatically re-exported in MLJBase and MLJ - no action required)raw_training_scores
Step 2
MLJTestIntegration
runs with no new surprises (and maybe adds some detector-specific testing)Step 3
scores
PRs and tags new releases, taking care to bump compats accordingly. Probably @ablaom to do.Step 4
scores (in, eg, their report); @davnn to complete this checklist:
The text was updated successfully, but these errors were encountered: