Pull an open-weight LLM. Get it converted, quantized, benchmarked, served, and packaged for Apple Silicon — in one CLI.

silicon-forge pull qwen3:0.6b        # fetch from Hugging Face → convert to MLX → quantize (4-bit default)
silicon-forge bench qwen3:0.6b       # measure cold start, warm tok/s, peak memory — on YOUR machine
silicon-forge serve qwen3:0.6b       # local OpenAI-compatible API, works with any OpenAI client
silicon-forge package qwen3:0.6b --target mlx      # portable, self-contained MLX bundle
silicon-forge package qwen3:0.6b --target core-ai  # Core AI: honest status + verified manual path (see below)
silicon-forge catalog                # everything you've pulled, benchmarked, packaged

Apple's recommended path for bringing large open models on-device is: prepare and quantize with MLX, then deploy through Core AI (announced WWDC 2026 as the successor to Core ML for LLM-scale workloads). The open-weight ecosystem (Qwen, Llama, Mistral, Gemma, …) moves fast — but wiring one of those models into that pipeline today means hand-rolling conversion, quantization, benchmarking, and packaging yourself.

silicon-forge is that missing tool: what Ollama is for llama.cpp-style local serving, this aims to be for the MLX → Core AI pipeline — with a local, plain-JSON benchmark catalog of what your hardware actually measured as its empirical core.

uv tool install silicon-forge   # or: pip install silicon-forge

Requires Apple Silicon (M-series) and Python ≥ 3.10. From source: uv sync && uv run silicon-forge --help.

Real output from silicon-forge bench smollm2:135m on an Apple M2 (16 GB, macOS 26.5.2), SmolLM2-135M at 4-bit:

✓ load: 0.21s
✓ cold start: first token in 0.08s, 272.3 tok/s
✓ warm (3 runs): 293.2 tok/s generation, 2163.0 tok/s prompt
✓ peak memory: 0.16 GB
recorded in ~/.silicon-forge/catalog.json

Your numbers will differ — that's the point. Every value is measured on the machine that runs it; nothing in this project fabricates or estimates a benchmark number.

Model names are Ollama-style aliases (qwen3:0.6b, smollm2:135m, mistral:7b, …) or any raw Hugging Face repo id (Qwen/Qwen3-0.6B). Gated repos (Llama, Gemma) need an accepted license and HF_TOKEN. Quantization: --quant 2/3/4/6/8/none.

Every bench run appends to a plain JSON file you own (~/.silicon-forge/catalog.json):

• model, quantization, mlx/mlx-lm versions used
• device identity read from the system — chip, unified memory, OS — never typed by hand
• cold start and warm tokens/sec recorded separately, never averaged together — engines with one-time compile costs look slow in a blended average while running fast at steady state; the shape is the data point
• peak memory during inference
• a free-text observations field (--observations "...") for what a number can't capture: thermal throttling mid-run, backend quirks, surprises

No telemetry. No upload. No server. Share it yourself if and when you want to. The schema is versioned and documented in docs/catalog-schema.md; any JSON parser in any language can consume it without silicon-forge.

silicon-forge serve exposes the standard OpenAI API — /v1/chat/completions (streaming and non-streaming) and /v1/models — implemented on the Python standard library. Anything that speaks that protocol works with zero glue code: curl, the official OpenAI SDKs, LiteLLM, LangChain, or local-first-llm's generic openai-compatible backend. See examples/curl-serve.

That's the integration policy for this whole portfolio: tools compose through shared open protocols and open formats, never through code dependencies on each other.

Core AI is weeks old and still in developer beta. What we verified against Apple's current documentation (2026-07-02): models ship as .aimodel assets; LLM export goes through per-architecture recipes in Apple's coreai-models repo starting from the original PyTorch weights (there is no generic MLX-weights → .aimodel path yet); iOS targets additionally require xcrun coreai-build compile (Xcode 27). Community testing (apple-silicon-llm-bench) shows export output still shifting between OS betas.

So package --target core-ai currently runs a real toolchain preflight on your machine and prints the verified manual pipeline with sources — an explicit, honestly-labeled "not yet", rather than guessed export code presented as working. --target mlx is fully functional and produces bundles that Foundation Models can already consume on-device via MLXLanguageModel.

• mlx-lm — the foundation this project is built on for conversion, quantization, and generation. A dependency, credited prominently, not a competitor.
• Ollama — the UX inspiration; pull/serve by model name intentionally echoes it. Ollama focuses on llama.cpp-style local chat serving; it has no Core AI or app-shippable-bundle story.
• coremltools — Apple's PyTorch → Core ML converter; no by-model-name CLI, no benchmark catalog, no Core AI target.
• apple-silicon-llm-bench (MIT) — the closest prior art for Core AI benchmarking on real hardware and a valuable pipeline-shape reference. That project is a benchmark comparison; this one is a pull/convert/serve/catalog toolkit.

uv sync
uv run pytest              # fast unit suite — no GPU, no network, no weights
uv run pytest -m integration   # real pull→bench→serve (Apple Silicon + network)
uv run ruff check . && uv run ruff format --check .

See CONTRIBUTING.md — including the dependency policy (two runtime dependencies, both justified) and the project's non-negotiables: standalone by design, measured values only, no telemetry.

MIT