Inside a Private AI Server: A Hardware Guide to Mac Studio, NVIDIA DGX Spark, and Custom Builds

A private AI server for an office carries four workloads at once, and sizing means understanding all four rather than chasing a single benchmark number.

• Model inference: running the local language model that composes answers and drafts. This is the heavyweight, and it is what the memory discussion below is about.
• Embeddings: fingerprinting every document chunk, and every incoming question, with a smaller model. Light per item, constant in the background.
• The index and watcher: storing the searchable memory of your documents and re-indexing files as they change. This is storage and steady I/O, not raw compute.
• The workspace itself: the chat interface, accounts, permissions, and audit logging your team actually touches. Modest, but always on.

The variable that moves the sizing most is concurrency: five people asking occasional questions is a different machine than twenty people using it as their daily research tool, plus overnight agents batch-processing documents while the office sleeps. This is why we size hardware after the discovery prototype, when we have measured your real usage pattern instead of guessing at it.

If you remember one technical fact from this guide, make it this one: local AI lives and dies on memory. Two numbers govern everything.

Capacity decides which models you can run at all. A model’s billions of parameters have to fit in memory the GPU can reach: roughly, a well-compressed 70B-parameter model wants 40-plus gigabytes, and bigger models want proportionally more. Too little memory and the model simply does not load; there is no slow-but-working fallback worth using.

Bandwidth decides how fast answers stream. Generating each word requires sweeping the model’s weights through the processor, so tokens per second track memory bandwidth more closely than any other spec. This is why a machine with fast unified memory can feel quicker in chat than a nominally more powerful one that moves data slowly.

Both of the small machines below use unified memory: one large pool shared by CPU and GPU, so the GPU can use nearly all of it for the model. Traditional GPU builds instead have dedicated VRAM on the card: faster per gigabyte, but you buy capacity in expensive card-sized increments. That trade-off is essentially the whole hardware decision, and one more 2026 reality sharpens it: the industry-wide RAM shortage has raised prices and even deleted high-memory configurations mid-year, which makes right-sizing more valuable than it has ever been.

Apple’s Mac Studio is the machine nobody notices, which for an office is a feature. It starts at $2,499 with the M4 Max and 36GB of unified memory; the M3 Ultra configuration at $5,299 carries 96GB and moves data at 819 GB/s, the highest memory bandwidth you can buy at this price and footprint. It is silent under load and draws power like a desktop, not a server.

In practice, a 96GB M3 Ultra runs 70B-class open-weight models comfortably with room left for the embedding model, the index, and the workspace. For a small team whose core use case is private chat, document Q&A with citations, and drafting, that is genuinely enough machine, and the office-friendliness is unmatched: it sits on a shelf, makes no noise, and needs no special power or cooling.

Two caveats. First, the ceiling: Apple discontinued the 128GB, 256GB, and 512GB memory options during this year’s RAM squeeze, so 96GB is the maximum on a new unit, and there is no adding more later; you size at purchase. Second, the ecosystem: local AI tooling on Apple silicon is good and improving, but the deepest tooling still lives in NVIDIA’s CUDA world, which matters if your roadmap includes heavier custom workloads.

The DGX Spark is NVIDIA’s answer to exactly this market: a personal AI machine the size of a thick hardback book. It launched last October at $3,999 and currently sells for roughly $4,400 to $4,700 after memory-shortage price increases. Inside is the GB10 Grace Blackwell chip: a 20-core ARM CPU and a Blackwell GPU sharing 128GB of unified memory, with a 4TB SSD and about a petaflop of AI compute.

The 128GB pool is the headline. It clears the next tier of open-weight models, NVIDIA rates it for models up to roughly 200 billion parameters, which buys headroom for stronger reasoning, larger context, or several models resident at once: a chat model, an embedding model, and a specialist for extraction, all loaded together. It also speaks CUDA natively, so the widest range of local-AI tooling runs first-class, and when an office outgrows one unit, two Sparks can be linked to run larger models still.

The trade-offs are the mirror image of the Mac’s. Its memory bandwidth is well below the M3 Ultra’s, so single-stream chat can feel less snappy than raw specs suggest, particularly on smaller models where the Mac shines. It is a specialist appliance running NVIDIA’s Linux-based stack, which is exactly right for a dedicated AI server and less familiar for a generalist IT team to administer. For document-heavy offices that want larger models resident, though, it is the box we point at most often, which is why it anchors the middle column of the comparison below.

Past a certain team size, the constraint stops being "which model fits" and becomes "how many people can use it at once." That is when we spec a custom workstation or short-depth rack server around dedicated NVIDIA GPUs: 24 to 96-plus gigabytes of VRAM depending on cards, paired with fast NVMe storage for the index and enough system RAM to keep everything fed.

Dedicated VRAM is the fastest memory in this guide, and serious serving software can batch many simultaneous users through one GPU efficiently. A custom build is also the only expandable option: add a second GPU, more storage, or redundant power later. The costs are the obvious ones: more money, more noise and heat than either small box, and a preference for a closet or rack over a shelf. This is the right option for offices heading toward role-based workflows, overnight agents, and dozens of daily users, the command-center pattern described on our service page.

Read the columns as stages as much as alternatives. Plenty of engagements start on a Mac Studio or a DGX Spark proving the core use case, then grow into a custom build when workflows and agents earn their place; the documents, index design, and workspace move with you. What we help you avoid is the expensive version of enthusiasm: buying the third column to serve the first column’s workload, in the most expensive RAM market in years.

The honest sizing method is the one we run in discovery: prototype your actual use case on sample documents, measure the models and concurrency it really needs, then buy. The economics of that decision, including where an owned server crosses over against per-seat subscriptions, get their own article: what on-premise AI actually costs.

The server is the visible purchase, but what makes it a private system is the unglamorous engineering around it. When we deploy, the machine goes on its own isolated network segment with general outbound internet blocked at the firewall: no telemetry, no phoning home, no route out for your data. Your team reaches it over TLS at an internal address like ai.yourcompany.internal, with individual accounts and role-based permissions.

• A UPS: a battery backup so a power blip does not corrupt the index or interrupt an overnight job. Cheap insurance for any always-on machine.
• On-site backups: the index, chat history, and configuration back up to your own equipment, never to a third-party cloud bucket.
• Full-disk encryption: the machine holds your most sensitive documents in indexed form; it is encrypted at rest like the laptops are.
• Sensible placement: a Mac Studio or Spark lives happily on a shelf in a lockable room; a GPU build wants the network closet. Physical access is access.
• A maintenance path: updates and model upgrades happen on-site or over a client-authorized, time-limited channel that is logged and disabled outside approved windows. Never a standing public endpoint.

None of this is exotic IT, but all of it has to be done deliberately, and it is included in how we build and deploy these systems. If you want the deeper pipeline view of what the machine spends its days doing, that is covered in how on-premise RAG works.