Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: Remove torchtext dependency #4038

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

WIP: Remove torchtext dependency #4038

wants to merge 4 commits into from

Commits on Oct 25, 2024

  1. Removed torchtext from NGramTokenizer

    m.habedank committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    c27d123 View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2024

  1. Refactored SentencePieceTokenizer 

    See: ludwig-ai#4032
    m.habedank committed Oct 26, 2024
    Configuration menu
    Copy the full SHA
    8e90e70 View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2024

  1. removed torchtext

    m.habedank committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    92a3ec0 View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2024

  1. Rewrote tests

    m.habedank committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    01f308e View commit details
    Browse the repository at this point in the history