Jafar Uruç

Podman & Ubuntu

2026-08-16T00:00:00Z

I've always liked the idea of rootless containerization. I think it's even more crucial now that we have agents running all the time on our machines.

I learned about Podman before the LLM era from Podman in Action, which I found useful. One of the most annoying things about Podman, though, is using it on Ubuntu. The version available through apt is outdated, so you have to build it from source if you want a newer release.

Well, now that we have AI, I decided to take a stab at it and see if I could get it to build easily so I could use it on my servers or as a sandbox for my agents.

Credit goes to podman-static for open-sourcing his repo. My agent definitely took a lot of things from it.

I pushed it to my dotfiles. Here is how to build and install it:

git clone https://github.com/JafarAbdi/myconfigs.git --depth 1
cd ./myconfigs/podman
just build
just install

And ta-da! You now have a fully static Podman build that you can copy to any machine, and it will work out of the box.

I need to write a blog post about how much I love musl and zigbuild and how much easier they make my life by making fully static binaries possible.

Trying out Nemotron 3.5 Lightning 30B

2026-08-14T00:00:00Z

NVIDIA Nemotron 3.5 Lightning 30B A3B GGUF

llama serve \
     -hf ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF:Q4_0 \
     -ngl auto \
     -c 262144 -np 1 \
     -ctk q8_0 -ctv q8_0 \
     -fa on --fit on \
     --spec-type draft-mtp \
     --spec-draft-n-max 3 \
     --jinja --kv-unified \
     --temp 1.0 --top-p 0.95 --top-k 0 --min-p 0 \
     --reasoning on --reasoning-format deepseek \
     --no-reasoning-preserve \
     --host 0.0.0.0 --port 8080 \
     --alias nemotron-3.5-lightning

I asked it to build a Rubik's Cube game:

Prompt

Anyway, Gemma4 31B is still my favorite and most useful model.

AI failure mode #1: shelling out to Python

2026-08-14T00:00:00Z

Sometimes I look at code generated by clankers and think, WTF? I’ll start recording these failures in blog posts.

result = subprocess.run(
    ["uv", "run", "python", "-m", ...],
    cwd=_REPO_ROOT,
    check=False,
)

It's shelling out through uv just to start another Python process 🤷‍♂️ to get a function’s output rather than importing that file and calling the function directly...

Another instance I have seen a lot is when it tries to use ros2 cli interface rather than just importing ros2cli or using rclpy. This is an actual piece of code that was pushed...

try:
    completed = subprocess.run(
        ["ros2", "topic", "echo", "--once", "--no-arr", topic],
        stdout=subprocess.DEVNULL,
        stderr=subprocess.DEVNULL,
        timeout=...,
        check=False,
    )
except subprocess.TimeoutExpired:
    return 1

You can search github to see how much slop is being pushed.

Muse Glimmer 30B

2026-08-10T00:00:00Z

Meta released the weights for its Muse Glimmer model, following up on Muse Spark 1.2. One of my hobbies these days is trying out different models and wasting electricity and a few brain cells on them.

The server configuration below follows Meta's Muse Glimmer 30B model card.

llama serve \
  -hf meta-models/Muse-Glimmer-30B-GGUF:17gb \
  --spec-type draft-dflash \
  --spec-draft-n-max 15 \
  -ngl all -ngld all \
  -c 131072 -np 4 \
  -fa on --fit on --jinja \
  --temp 1.0 --top-p 0.95 --top-k 64 --min-p 0 \
  --host 0.0.0.0 --alias muse-glimmer --kv-unified --reasoning-preserve

llama serve \
  -hf meta-models/Muse-Glimmer-30B-GGUF:dynamic \
  --spec-type draft-dflash \
  --spec-draft-n-max 15 \
  -ngl all -ngld all \
  -c 131072 -np 4 \
  -fa on --fit on --jinja \
  --temp 1.0 --top-p 0.95 --top-k 64 --min-p 0 \
  --host 0.0.0.0 --alias muse-glimmer --kv-unified --reasoning-preserve

Performance

17gb: 57.95 tokens/s, using about 20 GB of VRAM with a 128K-token context.
dynamic: 56.84 tokens/s, using about 23 GB of VRAM with a 128K-token context.

In initial testing, Muse Glimmer is noticeably faster than Gemma 4, uses less memory, and comfortably handles four agents in parallel. I also did not notice much difference between the 17gb and dynamic variants. The former may be slightly faster, but I could not tell in practice.

I will try Muse Glimmer for a few days to see whether it can replace Gemma 4 and Qwen 3.6 in my workflow. For reference, here is my current Gemma 4 configuration:

llama serve \
  -hf ggml-org/gemma-4-31B-it-GGUF:Q4_0 \
  --spec-type draft-mtp \
  --spec-draft-n-max 4 \
  -ngl all -ngld all \
  -c 65536 -np 2 \
  -fa on --fit on --jinja \
  --temp 1.0 --top-k 64 \
  --reasoning on \
  --chat-template-kwargs '{"preserve_thinking":true}' \
  --host 0.0.0.0 \
  --port 8080 \
  --alias gemma4 \
  --kv-unified

I Don't Want Smarter Models

2026-08-09T00:00:00Z

Well... at least for now.

I really enjoy listening to The Standup Pod while vibe coding things that I regret two hours later. Anyway, I was listening to Offsite: Pi | Standup #67.

Quoting Dillon Mulroy:

I don't want smarter models. I don't want models are getting like RL on these like long horizon tasks. Like I don't use agents that way. I haven't seen agents be successful that way. They're okay. Let me backtrack. They're good for the long horizon stuff for like debugging and triaging.

I have the same feeling. I still miss the models we had between February and April 2026. Here is a plot of how I felt:

For me, their usefulness peaked during that period. Newer models remain useful and powerful, but they surprise me less.

Long-horizon agents have worked beautifully for me when debugging. I used to flex my ability to debug and SSH into machines to find issues. Holy cow, these models are so good at it. They know all the right commands, and when I give a model a way to receive feedback, it helps me fix all kinds of bugs, including hardware issues.

Outside debugging and triaging, however, I haven't had much success with agents handling very long-horizon tasks without my interaction.

The newer models also feel less steerable. Fable/Opus 5 always thinks it's better than you and takes ages to do a task, only to do everything except what you asked it to do. GPT-5.6 Sol feels more steerable, but it still sometimes runs for more than 50 turns. My guess is that they tried to make these models tackle long, autoresearch style runs, but now the models overrun tasks. I would rather have an autoresearch plugin that I can control than models that default to long runs. But that is just me 🤷‍♂️

Hello, world!

2026-08-08T00:00:00Z

One of the main reasons I had friction with blogging was the existing workflow. Until now, I had been using Jekyll with the al-folio theme. While it's amazing and beautiful, the workflow never clicked for me. Anyway, now with LLMs around, I asked my best friend Claude to build a suckless blog in simple Python.

The solution is very opinionated and built for my own constraints: two Python files, ~250 LOC. On top sit an unnecessary pi extension (.pi/extensions/title.ts) to feel good about myself, plus a proofread skill (.claude/skills/proofread/SKILL.md) to make sure I didn't ship a stupid typo. All posts will be written entirely by me: no AI slop bullshit.

Let's see how it goes!

Local Inference Models

2026-08-08T00:00:00Z

One of the most exciting thing in AI is not just that powerful models can solve a fair number of tasks in a single attempt. It is that more of these models can now run locally.

Here are some local models I currently use:

Whisper: I use it with my personal assistant when I cannot be bothered to type. It generally works very well.
GLM-OCR: This is an excellent model for turning screenshots of equations or text from PDFs and images into copyable text. Here is my script.
Qwen3.6: I used it for coding, but these days I prefer Gemma4, especially after the latest fixes to the chat template.
Gemma4: I use it for my personal assistant and simple coding tasks.

Today, I experimented with MiniMax-H3. I asked Fable, using its xhigh effort setting, to write an optimized inference script. MiniMax-H3 fits easily in the VRAM of my RTX 3090 and takes about 22 minutes to generate a 124-frame video at 864x480 resolution.

I am looking forward to trying Qwen3.8 when it is released.

Using Pixi as a Development Environment for ROS 2

2025-08-23T00:00:00Z

When starting a new ROS 2 project with multiple team members, establishing a reproducible development environment is crucial for success. Without it, you'll inevitably face the classic "works on my machine" problem, where code that runs perfectly for one developer fails mysteriously for another.

What is RoboStack?

RoboStack brings the Robot Operating System (ROS) to any platform - Linux, macOS, and Windows - using the Conda package manager. Unlike traditional ROS installations that are Linux-specific and require system-level packages, RoboStack packages ROS distributions (like Humble and Jazzy) as Conda packages that can be installed without root access.

This approach enables easy integration of ROS with machine learning libraries like PyTorch and Flax in the same environment, making it perfect for robotics research that combines traditional robotics with modern AI workflows.

Enter Pixi: A Unified Development Solution

One approach I used in the past was to create a Docker image that encapsulated all the dependencies and configurations needed for the project. This worked well for ensuring consistency across machines, but it also introduced significant complexity and frustration (see Appendix: Docker Challenges for details).

In this post, I'll introduce pixi as a modern alternative for creating reproducible ROS 2 development environments.

What Pixi Brings to the Table

Three tools in one:
1. Task runner (Replaces Makefile)
2. Dependency management (combines conda and PyPI through uv) with a lock file
3. Multi-environments management.
ROS 2 integration through RoboStack project.
Simple multi-distro support: Easy switching between ROS 2 distributions.
Python integration: Ensuring ROS 2 and Python dependencies are compatible.

For comprehensive documentation, visit pixi.sh.

Getting Started: Initial Setup

Begin by initializing Pixi in your project directory:

pixi init

Next, configure the appropriate channels. The channel selection depends on your target ROS 2 distribution. You can either manually edit your pixi.toml file or use the CLI:

# Add RoboStack channel for your desired ROS 2 distribution
pixi workspace channel add https://prefix.dev/robostack-jazzy

This will update your pixi.toml file with:

channels = ["conda-forge", "https://prefix.dev/robostack-jazzy"]

Replace robostack-jazzy with your desired ROS 2 distribution (e.g., robostack-humble for Humble).

ROS 2 Base Dependencies

Add ROS 2 dependencies using the CLI:

# Add one of the following based on your project needs:
pixi add ros-jazzy-ros-core        # Minimal core functionality
pixi add ros-jazzy-ros-base        # Standard base installation (recommended)
pixi add ros-jazzy-desktop         # Desktop environment with GUI tools
pixi add ros-jazzy-desktop-full    # Complete desktop installation

Pro Tip: Use the prefix.dev package explorer to browse available packages and their dependencies. For CLI users, run: pixi search --channel https://prefix.dev/robostack-jazzy ros-jazzy-desktop

For common development tools, add them with:

pixi add colcon-common-extensions colcon-mixin cmake ninja mold sccache

Dependency Management Strategy

I'm not aware of a tool that automatically extracts dependencies from package.xml files and updates the pixi.toml file. Hence, you will need to either manually add the required packages to your pixi.toml through the pixi add <package-name> command or by creating a script that parses package.xml files and updates the pixi.toml file accordingly with pixi add $(my-cool-vibe-coded-script).

Task Automation

Define your tasks using the CLI:

pixi task add build "colcon build"
pixi task add test "colcon test" --depends-on build

Execute tasks with simple commands:

pixi run test  # Automatically runs build first, then test

Advanced Task Features Worth Exploring: - Task arguments for parameterized execution - Task caching for improved performance

Multi-Environment Support

One of Pixi's most powerful features is seamless multi-environments support. You can set this up using CLI commands:

# Setup Humble environment
pixi workspace channel add https://prefix.dev/robostack-humble --feature humble
pixi workspace environment add humble --feature humble
pixi add ros-humble-ros-base --feature humble

# Setup Jazzy environment
pixi workspace channel add https://prefix.dev/robostack-jazzy --feature jazzy
pixi workspace environment add jazzy --feature jazzy
pixi add ros-jazzy-ros-base --feature jazzy

This creates the following configuration in your pixi.toml:

[feature.jazzy.dependencies]
ros-jazzy-ros-base = "*"

[feature.humble.dependencies]
ros-humble-ros-base = "*"

[environments]
humble = { features = ["humble"] }
jazzy = { features = ["jazzy"] }

Switch between environments effortlessly:

pixi shell -e jazzy   # Enter Jazzy environment
pixi shell -e humble  # Switch to Humble environment

You can also create specialized environments for different purposes:

# Setup linting environment
pixi add pre-commit --feature lint
pixi workspace environment add lint --feature lint --no-default-feature
pixi task add lint "pre-commit run --all-files" --feature lint

pixi run lint

This eliminates the need for multiple Docker containers.

PyPi dependencies

Pixi uses uv for managing PyPI dependencies alongside Conda packages. To add PyPI packages, use:

pixi add --pypi numpy scipy matplotlib

It also supports extras. For example, to add Flask with async support:

pixi add --pypi flask[async]

This will update your pixi.toml file with:

[pypi-dependencies]
flask = { version = ">=3.1.2, <4", extras = ["async"] }

If you want to add a source python package, you can do so by specifying the path:

[pypi-dependencies]
my_cool_library = { path = "./my_cool_library", editable = true, extras = ["dev"] }

Cross-Platform Support

Most of the RoboStack packages are available in linux-64, linux-aarch64, osx-64, osx-arm64, and win-64. If you are lucky enough and all your other dependencies are available on these platforms, you can use Pixi to create a cross-platform ROS 2 development environment.

Streamlined CI/CD Workflows

This part is what I'm mostly excited about when using Pixi. Pixi ensures that:

Local development environments match CI exactly
Failed tests can be debugged locally with identical conditions
No more "works on my machine" scenarios

While tools like industrial_ci remain valuable, Pixi's reproducibility makes debugging significantly more straightforward.

Example Implementation

For a practical demonstration, explore this complete ROS 2 project using Pixi: pixi_ros2_example.

The project showcases: - Dual support for Humble and Jazzy distributions - Clean, self-contained structure - Automated CI/CD integration

Project Structure:

project_root/
├── .github/workflows/ci.yaml    # CI/CD configuration
├── pixi.toml                    # Project configuration
├── pixi.lock                    # Dependency lock file
├── colcon_defaults.yaml         # Colcon settings
├── build/                       # Build artifacts
├── install/                     # Installation directory
├── pkg1/                        # ROS 2 package 1
├── pkg2/                        # ROS 2 package 2
└── ...

Bonus: Git Worktree Integration

I'm a big fan of git-worktree. With this approach, I can easily manage multiple branches of the same repository without needing to clone it multiple times or stash changes when switching branches. This is particularly useful for ROS 2 development, where you might want to work on multiple features or bug fixes simultaneously.

Simply running git worktree add ../worktree_dirname branch_name creates a new directory with the specified branch checked out, allowing you to work on it independently.

Current Limitations and Considerations

Synchronizing dependencies from package.xml: You will need to manually add dependencies from package.xml to the pixi.toml file. This is a bit of a hassle, but manageable.
ros-testing packages: Currently, RoboStack does not provide packages from the ros-testing repository.
Stability concerns: Pixi is still not stable, and some features are experimental. For example, in previous versions of pixi, having an environment variable in the activation section of pixi.toml would cause the value to be expanded, but it stopped working in v0.51.0 and was reverted in v0.52.0. This can lead to unexpected behaviors. Fortunately, you can pin the pixi version in your pixi.toml file to avoid such issues using requires-pixi.

[activation.env]
# In v0.51.0 the expansion is not performed, so the value is literally "$CONDA_PREFIX/my_env_var"
MY_ENV_VAR="$CONDA_PREFIX/my_env_var"

Specialized hardware support: For Jetson machines, NVidia provides custom builds for PyTorch/TorchVision that require specific handling. Using standard PyPI/Conda packages would cause segfaults or crashes. The best solution I'm aware of for this specific case is still using Docker.

Update: Experimental Build Backends for ROS Packages

Austin Gregg-Smith and Silvio Traversaro told me about an exciting new feature in Pixi, it now includes experimental build backends that can build ROS 2 packages directly from source code.

See ros_workspace for an example of how to use this feature.

Conclusion

As someone who has been working in robot learning where projects require a mix of Python and C++ codebases alongside complex dependency management spanning both ROS 2 and ML frameworks, Pixi has been a game-changer. The ability to seamlessly manage conda-forge packages for ROS 2 components alongside PyPI packages for machine learning dependencies in a single, reproducible environment has dramatically reduced development friction and allowed me to iterate much faster.

Appendix: Advanced Pixi Usage

Beyond ROS 2, I've successfully used Pixi to build complex projects like OMPL, MuJoCo, MuJoCo MPC, and llama.cpp from source - you can find these configurations in my pixi_workspaces repository.

Appendix: Docker Challenges in ROS 2 Development

Having worked with Docker for ROS 2 development, the typical workflow involves three files: a Dockerfile with several stages (base image with ROS 2 installed, dependency installation, build stage to compile ROS packages, and final dev stage with necessary tools), a docker-compose file defining services like the dev environment and CI/CD testing that mount the workspace and set environment variables, and finally a Makefile that orchestrates the build and run commands.

However, this approach has several significant drawbacks:

Reproducibility issues: To truly ensure reproducibility, you need to pin every dependency version in the Dockerfile: system & ROS 2 packages (via apt), Python packages (via pip). While ROS 2 provides a snapshot repository after each release and distro (snapshot repository), you still need to create custom scripts to scrape package.xml files and requirements.txt files to generate lock files containing all dependencies and their versions. As far as I know, no tool does this automatically for ROS 2 packages. You also need to ensure PyPI dependencies are compatible and won't cause runtime errors due to version mismatches.
Slow build times: If you are not careful with your docker build's context, you can end up with invalidating the Docker cache for every build, leading to long build times and slow development cycles.

Typical xkcd developer waiting for Docker to build a ROS 2 workspace.

Image size bloat: Docker images become massive, especially when adding CUDA and other heavy dependencies
Disk space consumption: Docker images can take up a lot of disk space, just run docker system df to see how much space is consumed.

Heaviest objects in the universe: neutron stars, black holes, and Docker images. (Source: Bluesky post)

Development Workflow Friction: This might be specific to robot learning + ROS 2 workflows. It's often much simpler to create a virtual environment with uv or poetry for ML code, install dependencies directly, and run your models. However, this approach creates two separate environments: one for ROS 2 packages and another for machine learning code. This split leads to integration challenges when you finally want to deploy your trained model on the robot. (Note that Conda isn't a viable solution here either, as both apt-provided ROS 2 and Conda modify environment variables in conflicting ways.)

Acknowledgments

Thanks to Sam Pfeiffer and Sebastian Castro for reviewing this post and providing helpful feedback.