Last updated 08-09-2026
Category:
Reviews:
Join thousands of AI enthusiasts in the World of AI!
WoolyAI
WoolyAI is developer software for running multi-model agentic AI workloads on existing GPU hardware and emerging ASIC accelerators. The company sells three products: GPU Runtime Software for concurrent vLLM serving on NVIDIA and AMD GPUs, a Private Inference Stack for DGX Spark clusters, and a portable software stack that helps ASIC vendors support PyTorch, vLLM, and SGLang.
Where generic inference servers treat each model as a separate deployment, WoolyAI focuses on memory-aware scheduling across models that share one GPU or one small cluster. Its runtime adds VRAM swap, priority control, and model weight deduplication so agent workflows with planners, verifiers, and tool-use models do not each need a dedicated GPU stack.
The product targets ML platform teams, AI infrastructure engineers, and chip vendors who need private multi-model serving without oversized data-center GPU footprints. Benchmarks on the site cite 93.31 tok/s decode for Nemotron 3 Nano Omni 30B on a 2x DGX Spark setup and up to 1.77x higher prefill throughput versus native concurrent vLLM on an H100.
GPU Runtime adds VRAM swap and priority scheduling under existing vLLM servers on NVIDIA and AMD GPUs
DGX Spark Inference Stack serves DeepSeek, Gemma, and Nemotron models through one OpenAI-compatible endpoint
Internal benchmarks report 93.31 tok/s C4 decode for Nemotron 3 Nano Omni 30B on 2x DGX Spark
Priority-0 models stay at 0.89 to 0.94x native single-model token generation while background models run
ASIC software stack brings PyTorch, vLLM, and SGLang support to emerging AI chip vendors
24B and 27B model swap-in TTFT measured at about 2.4 seconds on an H100 80GB in WoolyAI tests
Runs multiple vLLM workloads on one GPU with priority control instead of separate deployments per model
DGX Spark inference stack targets lower-cost private multi-agent serving than large data-center GPU farms
Published internal benchmarks include specific tok/s and prefill numbers across named models
ASIC enablement stack covers PyTorch, vLLM, and SGLang for chip vendors
No public pricing; every plan requires a demo request
Benchmark numbers come from WoolyAI internal tests, not independent third-party reviews
Product documentation lives on a separate docs site with limited on-site pricing detail
What does WoolyAI build?
WoolyAI builds AI compute software for multi-model agentic inference. Its products include GPU Runtime Software, a Private Inference Stack for DGX Spark clusters, and an ASIC enablement stack for PyTorch, vLLM, and SGLang.
Which frameworks does WoolyAI support?
WoolyAI supports PyTorch, vLLM, and SGLang across its GPU Runtime and inference server products. Its ASIC software stack is designed to port those frameworks to emerging accelerator hardware.
What hardware does WoolyAI target?
WoolyAI targets NVIDIA and AMD GPUs for its runtime software and DGX Spark clusters for its private inference stack. It also builds software stacks for ASIC AI chip vendors.
How does WoolyAI pricing work?
WoolyAI does not publish plan prices on its website. Teams request a demo or technical discussion through woolyai.com to get product and pricing details.
What benchmark results does WoolyAI report?
WoolyAI reports 93.31 tok/s decode for Nemotron 3 Nano Omni 30B on 2x DGX Spark and up to 1.77x higher prefill throughput versus native concurrent vLLM on an H100 in internal tests.
Who should use WoolyAI?
WoolyAI is aimed at ML platform teams running agentic workflows and chip vendors that need modern framework support. It fits teams that want private multi-model serving without deploying a separate GPU stack per model.