Nvidia connects home PCs into a local AI cloud for free

You no longer need one expensive graphics card

Nvidia has announced the Personal AI Router (PAIR), a software tool that allows users to pool the computing power of multiple devices on a local network to run AI models. The goal is to enable larger LLMs (Large Language Models), such as Meta's Llama 3 or Alibaba's Qwen, to run on hardware that would not be able to handle it individually.

A step towards decentralization for enthusiasts

Interestingly, PAIR supports not only Nvidia RTX graphics cards but also Apple silicon (M1 and newer). Users can thus connect a Windows gaming PC with a MacBook to run a demanding model together via platforms like Ollama, LM Studio, or Jan.

Where it hits the reality of cables

While Nvidia promises smooth running, the physics of home Wi-Fi cannot be fooled. Network latency will be a real problem when combining multiple smaller GPUs, especially when generating long texts where transfer speed is key. It will require fast ethernet so it's not more waiting than working.

Testing interest in community clusters

If the tool catches on, it could mark the beginning of an era of community clusters, where instead of renting cloud servers, people simply assemble power from unused hardware around the house. The next logical step would then be to securely share power with neighbors, but Nvidia is far from there yet.

Lilith's verdict

Nvidia has just made sure that local models on the Mac will no longer be a closed island. Instead of a hardware war, it chose the software assimilation of foreign silicon.