This repo implements UniTok, a unified visual tokenizer well-suited for both generation and understanding tasks. It is compatiable with autoregressive generative models (e.g. LlamaGen), multimodal ...
A comprehensive PyTorch-based framework for pretraining and fine-tuning Qwen language models with custom datasets and tokenizers. Optimized for memory efficiency, featuring automatic batch sizing, ...