Local Inference Models
2026-08-08
One of the most exciting thing in AI is not just that powerful models can solve a fair number of tasks in a single attempt. It is that more of these models can now run locally.
Here are some local models I currently use:
- Whisper: I use it with my personal assistant when I cannot be bothered to type. It generally works very well.
- GLM-OCR: This is an excellent model for turning screenshots of equations or text from PDFs and images into copyable text. Here is my script.
- Qwen3.6: I used it for coding, but these days I prefer Gemma4, especially after the latest fixes to the chat template.
- Gemma4: I use it for my personal assistant and simple coding tasks.
Today, I experimented with MiniMax-H3. I asked Fable, using its xhigh effort setting, to write an optimized inference script. MiniMax-H3 fits easily in the VRAM of my RTX 3090 and takes about 22 minutes to generate a 124-frame video at 864x480 resolution.
I am looking forward to trying Qwen3.8 when it is released.