Plan a GPU server around your AI workload
Define memory, isolation, storage and serving requirements for image generation and LLM inference before comparing GPU configurations.
Budget VRAM beyond model weights
Record model size, precision, context length, batch size and target concurrency. Allow room for KV cache, runtime overhead and temporary tensors. Quantization changes both memory requirements and model behaviour; validate it with representative inputs.
Confirm the allocation model
Ask for the GPU model and count, dedicated or shared VRAM, passthrough method, CPU allocation and administrator access. A vCPU count does not establish dedicated physical cores. Multiple GPUs do not imply a particular interconnect topology.
Validate the driver and framework together
Check the selected operating system, driver, CUDA runtime and framework versions. Use nvidia-smi to inspect device and driver information; its CUDA version is not proof of an installed CUDA toolkit. Run a minimal inference test before importing the full workload.
Make ComfyUI state reproducible
Manage models, inputs, outputs and custom-node dependencies separately. Save workflow JSON and pin node versions. Confirm what survives reboot, reinstallation and instance termination; back up irreplaceable outputs independently of the server.
Put vLLM and Ollama behind controlled access
Bind inference services to a private interface or localhost and expose them through an authenticated TLS gateway. Measure time to first token, generation rate, queueing and peak VRAM at the context lengths and concurrency you need. A single-request developer test is not a production capacity estimate.
Specify a four- or eight-GPU quote
Provide the model, parallelism strategy, VRAM per GPU, interconnect requirements, CPU, RAM, storage, transfer volume, rental term and budget. Have the quote specify installation scope, storage persistence and the acceptance workload so configurations can be compared on the same basis.