[KeyPoints] - KeyPoints doesn't work like Detections filtering

[SkalskiP]

Bug

When trying to filter KeyPoints by confidence like this:

from inference import get_model

pitch_detection_model = get_model(model_id="football-field-detection-f07vi/14", api_key=ROBOFLOW_API_KEY)

result = pitch_detection_model.infer(frame, confidence=0.3)[0]
keypoints = sv.KeyPoints.from_inference(result)

keypoints[keypoints.confidence > 0.5]

I get this:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
[<ipython-input-21-6a7425a97626>](https://localhost:8080/#) in <cell line: 1>()
----> 1 keypoints[keypoints.confidence > 0.5]

[/usr/local/lib/python3.10/dist-packages/supervision/keypoint/core.py](https://localhost:8080/#) in __getitem__(self, index)
    556             xy=self.xy[index],
    557             confidence=self.confidence[index] if self.confidence is not None else None,
--> 558             class_id=self.class_id[index] if self.class_id is not None else None,
    559             data=get_data_item(self.data, index),
    560         )

IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed

This is a symptom of a much broader problem - the KeyPoints object does not allow the skeleton of different objects to have different lengths.

I managed to work around this problem like this, but it's a very hacky solution.

keypoints = sv.KeyPoints.from_inference(result)
filter = keypoints.confidence > 0.5
keypoints.xy = keypoints.xy[filter][np.newaxis]
keypoints.confidence = keypoints.confidence[filter][np.newaxis]

Additional

• Note: Please share a Google Colab with minimal code to test the new feature. We know it's additional work, but it will speed up the review process. The reviewer must test each change. Setting up a local environment to do this is time-consuming. Please ensure that Google Colab can be accessed without any issues (make it public). Thank you! 🙏🏻

[onuralpszr]

@SkalskiP do we want to filter "keypoint" group (ex: I have 7 detected humans I want to filter last 4 human keypoints and left other 3 humans based on filter) or just points itself because filtering points itself visually can break. Also in case of "shortages" should we fill values in something like negative values for prevents np.newaxis change requirement

[LinasKo]

@onuralpszr At which point would the newaxis cause issues? Would one of the dimensions collapse?

[onuralpszr]

@onuralpszr At which point would the newaxis cause issues? Would one of the dimensions collapse?

My thinking was, Not using "newaxis" at all case and also curious about solution because we also talked about this in earlier how to manage because "filtering points" cause issue also key points can be incomplete (half body detection or semi visible cases)

[LinasKo]

I see your points. Filtering is ultimately a skeleton-level operation and is almost guaranteed to invalidate a [N x Skeleton] array, if we want to continue keeping everything in a single block.

[onuralpszr]

I see your points. Filtering is ultimately a skeleton-level operation and is almost guaranteed to invalidate a [N x Skeleton] array, if we want to continue keeping its keypoints in a single block.

To prevent that "we need" some sort of fillers because I need them all same but don't do anything, Because in picture It could be okay but in videos I can see problems like "occlusions" would be first.

[web-ho]

@SkalskiP Shouldn't the class_id, confidence and class_name have the same shape as xy.shape[:2] and shouldn't they be arrays for keypoints detected for each object.