A 518-million-parameter Llama-style language model, pretrained from random weights on a legal and financial corpus. Give it the start of a sentence and it continues in the legal register.
This is a base model, not a chatbot. It was trained on next-token prediction only, so it continues text rather than answering questions. Prompt it with the opening of a sentence and watch it complete the thought.
The honest quality metric is held-out validation perplexity: 7.91 (lower is better). It speaks the legal register fluently but does not know facts reliably — at this scale, grounded facts need retrieval (RAG).
| Class | LlamaForCausalLM |
| Layers | 24 |
| Hidden size | 1,280 |
| Attention | 20 heads · dim 64 · full MHA |
| Feed-forward | SwiGLU · inner 3,456 |
| Positional | RoPE · θ 10,000 |
| Norm | RMSNorm · ε 1e-5 |
| Context | 1,024 tokens |
| Vocabulary | 32,768 |
| Embeddings | tied input/output |
| Objective | Next-token pretraining |
| Pretraining tokens | 2.08B |
| Trainable here | none — imported base |
| Pretrained by | Dr. Sreedath Panat |
our cost begins at fine-tuning — the base is imported.
| Stage | Detail | Cost |
|---|---|---|
| RAFT fine-tune | trained on this base; no site of its own | $1.74 |
| Evaluation (shared) | 13 versions on 500 held-out questions, this model's share | $1.48 |
| Total | $3.22 |
Figures are Modal GPU usage (time × rate) across this model's lineage; shared datasets are charged at this model's share. Whether base pretraining is included is stated above — it is for the models pretrained here, and excluded for imported bases. Evaluation-derived metrics come from an independent blind-judge harness on a frozen, decontaminated held-out set. Serving is billed separately and scales to zero.