Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add transcript + metadata processing #15

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

Add transcript + metadata processing #15

wants to merge 14 commits into from

Commits on Aug 5, 2024

  1. add script

    @markus583 @kdu4108
    markus583 authored and kdu4108 committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    d9fb2bb View commit details
    Browse the repository at this point in the history

  2. minor cleanup

    @markus583 @kdu4108
    markus583 authored and kdu4108 committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    0221846 View commit details
    Browse the repository at this point in the history

  3. update comments (euler + TODO)

    @markus583 @kdu4108
    markus583 authored and kdu4108 committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    fd9666b View commit details
    Browse the repository at this point in the history

  4. add metadata; "shard-"

    @markus583 @kdu4108
    markus583 authored and kdu4108 committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    26dc422 View commit details
    Browse the repository at this point in the history

  5. add dataset option

    @markus583 @kdu4108
    markus583 authored and kdu4108 committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    266b739 View commit details
    Browse the repository at this point in the history

  6. adapt to whisper; adapt dirs

    @markus583 @kdu4108
    markus583 authored and kdu4108 committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    856d245 View commit details
    Browse the repository at this point in the history

  7. mv into folder

    @markus583 @kdu4108
    markus583 authored and kdu4108 committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    1511020 View commit details
    Browse the repository at this point in the history

  8. add splitter

    @markus583 @kdu4108
    markus583 authored and kdu4108 committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    83af75d View commit details
    Browse the repository at this point in the history

  9. adapt paths + langs

    @markus583 @kdu4108
    markus583 authored and kdu4108 committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    e3e5fd4 View commit details
    Browse the repository at this point in the history

  10. rename

    @markus583 @kdu4108
    markus583 authored and kdu4108 committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    691eece View commit details
    Browse the repository at this point in the history

  11. Modify train_val_test_split to invert from modality/train to train/mo… 

    …dality structure
    @kdu4108
    kdu4108 committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    74e1038 View commit details
    Browse the repository at this point in the history

  12. Change v2d to metadata and transcript filepaths. Transcript is untest… 

    …ed because we don't have the container working yet to use whisper (cry)
    @kdu4108
    kdu4108 committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    6a51d47 View commit details
    Browse the repository at this point in the history

  13. Remove shard- from v2d_to_metadata, transcript

    @kdu4108
    kdu4108 committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    90fd7c8 View commit details
    Browse the repository at this point in the history

  14. Validate that inputdir is a filtered raw subdir

    @kdu4108
    kdu4108 committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    1168fa0 View commit details
    Browse the repository at this point in the history