Overview
This model is built on Qwen2.5-Coder-1.5B-Instruct and has been fine-tuned exclusively on the official Roblox Luau corpus. It is designed to assist developers with code generation, completion, and understanding of Luau patterns commonly used in Roblox game development.
Key Features
-
Fine-tuned on quality-filtered Luau code from the official Roblox corpus
-
Optimized for Roblox-specific APIs and development patterns
-
Trained on TPU v5e-8 infrastructure
-
Includes interactive Colab notebook for immediate testing
Technical Details
The model uses Low-Rank Adaptation with target modules q_proj and v_proj, trained on filtered data (20-5000 character code samples) with a 90/10 train-validation split. Training was conducted with a batch size of 4, gradient accumulation of 32 steps, and a learning rate of 3e-5 using cosine annealing.
Access
Model: umjunsik1323/Qwen2.5-Coder-1.5B-roblox · Hugging Face Interactive Demo: Available via Google Colab (link in model card)
The model is released under Apache 2.0 license and is ready for integration into development workflows.
Feedback and contributions from the community are welcome.