Glossary

GPU & VRAM

In one line: A GPU is the chip that does the heavy lifting for AI, and VRAM is its onboard memory — the spec that decides which AI models your machine can run at all.

What it actually is

A GPU (graphics processing unit) was originally built to draw video-game graphics, but the same design — thousands of small processors working in parallel — turns out to be exactly what AI models need. When people say a business “runs AI locally,” they mean the model is loaded onto a GPU inside a machine they own, instead of on a cloud provider’s servers.

VRAM (video RAM) is the memory built onto the graphics card. An AI model has to fit inside it: if the model is bigger than your VRAM, it either won’t load or will run so slowly it’s unusable. That’s why VRAM — not speed, not brand, not price — is the first spec to check.

The tiers in plain terms

• 12GB VRAM — entry level. Runs smaller AI models: a capable private chatbot and document Q&A.
• 16GB VRAM — the sweet spot for small businesses. Runs mid-size models good enough for real drafting and customer-service work.
• 24–32GB VRAM — runs the strongest open-source models, approaching cloud-AI quality on your own hardware.

Why it matters for a small business

Owning a GPU means your customer data, contracts, and internal documents never leave your premises — useful for PDPA-sensitive work — and heavy daily AI use stops accruing monthly cloud bills. The trade-off is upfront cost and maintenance. The rule of thumb in 2026: if you’d use AI heavily every day, a card pays for itself against cloud rental in months; if your use is occasional, renting cloud GPUs stays cheaper.

Related terms

Large Language Model (LLM), Generative AI, Retrieval-Augmented Generation (RAG)