Private AI & LLM Infrastructure

On-Premise LLM Deployment & AI Infrastructure

AICubes designs, deploys and operates private large language model infrastructure for organisations that cannot send their data to public AI services. We size GPU clusters, install and tune the serving stack, and connect models to your applications through secure, OpenAI-compatible APIs.

Inference runs on NVIDIA H200 GPUs using vLLM, a widely adopted open-source inference engine. Your prompts, documents and outputs never leave infrastructure you control.

Get a quoteBook a consultationWhatsApp us
01Why on-premise

Why run LLMs on your own infrastructure

  • 01

    Data sovereignty

    Prompts, documents and outputs stay inside your network and jurisdiction.

  • 02

    Confidentiality

    No sensitive content is sent to third-party AI services, and nothing is used to train someone else’s model.

  • 03

    Predictable cost

    Fixed infrastructure instead of per-token fees, which matters for heavy internal use.

  • 04

    Control and choice

    Choose, evaluate and switch open-weight models on your own timeline, subject to their licences.

02Private AI & LLM Infrastructure

What we deliver

  • 01

    GPU infrastructure sizing

    Model and workload analysis to size NVIDIA H200 servers for your target models, context lengths and concurrent users.

  • 02

    vLLM serving stack

    Installation and tuning of vLLM with tensor and expert parallelism, continuous batching, prefix caching and FP8 where supported.

  • 03

    Model selection and evaluation

    Side-by-side evaluation of open-weight models on your own Arabic and English tasks before you commit to one.

  • 04

    Secure API gateway

    OpenAI-compatible endpoints with authentication, rate limits, usage quotas and full request logging.

  • 05

    RAG and fine-tuning

    Retrieval over your documents and, where it adds value, parameter-efficient fine-tuning on your data.

  • 06

    Monitoring and operations

    GPU, latency and throughput dashboards, alerting, model updates and capacity planning.

06NVIDIA H200 · vLLM

NVIDIA H200 and vLLM

We standardise on NVIDIA H200 GPUs for large-model inference and vLLM as the serving engine, sized to your models, context lengths and number of users.

NVIDIA H200 at a glance

GPU memory
141 GB HBM3e per GPU
Memory bandwidth
4.8 TB/s per GPU
Server configurations
NVIDIA HGX H200 systems with 4 or 8 GPUs
Precision
FP8 Tensor Cores (Hopper architecture)

Source: NVIDIA H200 product specifications (H200 SXM).

What vLLM provides

  • PagedAttention memory management for efficient use of GPU memory
  • Continuous batching for high throughput under concurrent load
  • Tensor, pipeline and expert parallelism across GPUs and nodes
  • OpenAI-compatible API server for straightforward integration
  • FP8 quantisation, prefix caching and speculative decoding where supported

Reference architecture

A typical deployment keeps every component inside your environment.

  1. 01Users and applicationsAssistants, search, chatbots, analytics and internal tools
  2. 02API gatewaySingle sign-on, role-based access, quotas and audit logging
  3. 03Retrieval layerVector database and connectors to your document stores
  4. 04Inference clustervLLM on NVIDIA H200 servers
  5. 05OperationsGPU, latency and usage monitoring with alerting

Public websites and cloud-hosted applications never run GPU inference themselves: they reach the private cluster only through authenticated, encrypted connections that you control.

07Open-weight models

Supported open-weight model families

We evaluate and deploy models from the families below. Each one is published with open weights and supported by vLLM; the right choice depends on your tasks, languages, hardware and licence requirements.

DeepSeek

ModelParametersInputsContextLicenceTypical H200 footprint
DeepSeek-V4-ProvLLM recipe caps context at ~800K tokens on a single H200 node.1.6T MoE · 49B activeText1MMITPermissive8× H200 (1 node)
DeepSeek-V4-Flash284B MoE · 13B activeText1MMITPermissive4× H200 per replica
DeepSeek-V3.2671B MoE · 37B activeText160KMITPermissive8× H200 (FP8)
DeepSeek-R1 (R1-0528)671B MoE · 37B activeText128KMITPermissive8× H200 (FP8)

Gemma · Google

ModelParametersInputsContextLicenceTypical H200 footprint
Gemma 4 (E2B, E4B, 12B, 26B-A4B, 31B)Audio input on E2B, E4B and 12B.2B–31B dense & MoEText, image, audio128K–256KApache-2.0Permissive1× H200
Gemma 3 (1B–27B)Commercial use permitted; Gemma Prohibited Use Policy applies and must be passed on to users.1B–27B denseText, image32K–128KGemma Terms of UseConditions apply1× H200

Llama 4 · Meta

ModelParametersInputsContextLicenceTypical H200 footprint
Llama 4 Scout (17B-16E)“Built with Llama” attribution; services above 700M monthly users need a licence from Meta.109B MoE · 17B activeText, image10MLlama 4 Community LicenseConditions apply1× H200 (FP8) · 2–4× (BF16)
Llama 4 Maverick (17B-128E)400B MoE · 17B activeText, image1MLlama 4 Community LicenseConditions apply8× H200 (FP8)

MiniMax

ModelParametersInputsContextLicenceTypical H200 footprint
MiniMax-M3Commercial use requires attribution and notice to MiniMax; written authorisation above US$20M annual revenue.~428B MoE · ~23B activeText, image1MMiniMax Community LicenseConditions apply8× H200 (BF16)
MiniMax-M2.7Commercial use requires prior written authorisation from MiniMax.230B MoE · ~10B activeText192KMiniMax Non-Commercial LicenseNon-commercial4× H200 (FP8)
MiniMax-M2.5Commercial use permitted with “MiniMax M2.5” shown in the product UI; prohibited-use policy applies.230B MoE · ~10B activeText192KMiniMax Model LicenseConditions apply4× H200 (FP8)

Mistral · Mistral AI

ModelParametersInputsContextLicenceTypical H200 footprint
Mistral Small 4 (119B-2603)119B MoE · ~6B activeText, image256KApache-2.0Permissive2× H200 (FP8)
Mistral Large 3 (675B-2512)675B MoE · 41B activeText, image256KApache-2.0Permissive8× H200 (FP8)

Kimi · Moonshot AI

ModelParametersInputsContextLicenceTypical H200 footprint
Kimi K3Vendor guidance targets Blackwell-class systems; on H200 it needs 2+ nodes. Model-as-a-service providers above US$20M revenue need a separate agreement.2.8T MoE · 104B activeText, image1MKimi K3 LicenseConditions applyMulti-node only (2+ H200 nodes)
Kimi K2.6Attribution required above 100M monthly users or US$20M monthly revenue.1T MoE · 32B activeText, image256KModified MITConditions apply8× H200 (INT4)
Kimi K2.51T MoE · 32B activeText, image256KModified MITConditions apply8× H200 (INT4)

Nemotron 3 · NVIDIA

ModelParametersInputsContextLicenceTypical H200 footprint
Nemotron 3 Ultra (550B-A55B)550B hybrid MoE · 55B activeText1MOpenMDW-1.1PermissiveMulti-node (2 H200 nodes, BF16)
Nemotron 3 Super (120B-A12B)120B hybrid MoE · 12B activeText1MNVIDIA Nemotron Open Model LicenseConditions apply2× H200 (FP8) · 4× (BF16)
Nemotron 3 Nano (30B-A3B)30B hybrid MoE · 3.5B activeText1MNVIDIA Nemotron Open Model LicenseConditions apply1× H200

Qwen · Alibaba Cloud

ModelParametersInputsContextLicenceTypical H200 footprint
Qwen3.8-27B27B denseText, image256KApache-2.0Permissive1× H200
Qwen3.8-Flash-NextModel-as-a-service and AI assistant businesses need a separate commercial licence.176B MoE · 6B activeText, image256KQwen Community License 1.0Conditions apply4× H200 (FP8)
Qwen3.8-2.4T-A95BServed with vLLM nightly builds. Model-as-a-service and AI assistant businesses above US$50M revenue need a separate licence.2.4T MoE · 95B activeText256KQwen3.8-Max LicenseConditions applyMulti-node (~32× H200, FP8)
Qwen3.6 (35B-A3B, 27B)27B–35B dense & MoEText, image256KApache-2.0Permissive1× H200
Qwen3.5 (0.8B–397B-A17B)Up to 397B MoE · 17B activeText, image256KApache-2.0Permissive1× H200 (small) · 8× H200 (397B)

Step · StepFun

ModelParametersInputsContextLicenceTypical H200 footprint
Step-3.7-Flash~198B MoE · 11B activeText, image256KApache-2.0Permissive8× H200 (1 node)
Step-3.5-Flash~197B MoE · 11B activeText256KApache-2.0Permissive4× H200 (BF16)

GLM · Z.ai

ModelParametersInputsContextLicenceTypical H200 footprint
GLM-5.2Full 1M context is documented on B200-class hardware; single-node H200 deployments use a shorter context.743B MoE · 39B activeText1MMITPermissive8× H200 (FP8)
GLM-5.1744B MoE · 40B activeText198KMITPermissive8× H200 (FP8)
GLM-5744B MoE · 40B activeText198KMITPermissive8× H200 (FP8)

Footprints are indicative for the published precision, based on vLLM recipes and vendor guidance; context length and concurrency change requirements. 4-bit (NVFP4/MXFP4) variants target newer GPU generations and are not assumed on H200. Licence terms are set by each model provider — we review them with you for your intended use.

08How a private LLM deployment runs

How a private LLM deployment runs

  1. 01

    Workload assessment

    Use cases, users, languages, data sources and security requirements.

  2. 02

    Model evaluation

    Candidate models benchmarked on your own Arabic and English tasks.

  3. 03

    Infrastructure sizing

    GPU, storage and network specification, with procurement support.

  4. 04

    Deployment and hardening

    Installation, access control, logging and performance tuning.

  5. 05

    Integration

    Connection to applications, retrieval sources and identity systems.

  6. 06

    Operations

    Monitoring, model updates and capacity planning as usage grows.

03Private AI & LLM Infrastructure

Typical use cases

  • Internal assistants for policy, HR and procurement knowledge
  • Private coding assistants for development teams
  • Document summarisation and drafting for ministries and legal teams
  • Back-end LLM for chatbots, avatars and RAG applications
  • Air-gapped AI for defence, security and critical infrastructure
04Private AI & LLM Infrastructure

How we deliver

  • On-premise, private cloud or hybrid

    Deployed where your data is allowed to live — your data centre, an in-country cloud, or a mix.

  • Arabic and English

    Interfaces, models and documentation designed for bilingual users from the start.

  • Integrated with your systems

    Connected to existing databases, identity providers, ERPs and line-of-business applications.

  • Supported after launch

    Monitoring, model updates and a named engineering contact once the system is live.

Frequently asked questions

No. Once models are downloaded and verified, the system can run fully offline. Updates are delivered through your approved change process.

It depends on the model, precision, context length and number of concurrent users. Mid-sized models run on a single H200; the largest open-weight models need an 8-GPU node or more. We size the cluster after a short workload assessment.

Yes. Applications talk to a standard API, so you can evaluate and switch models — subject to their licences — without rewriting your applications.

It depends on the task. Several of the families listed are multilingual and handle Arabic well, but quality varies by domain and style. We benchmark candidate models on your own Arabic content before recommending one.

Licences differ. Many are under permissive licences such as Apache-2.0 or MIT; others carry conditions such as attribution, revenue thresholds or non-commercial terms. We review the licence for your intended use before deployment, and your legal team makes the final decision.

05Services

AI Document Processing, OCR & RAG Systems

Arabic and English OCR, structured data extraction and enterprise search that answers questions with citations to your documents.

Learn more

Database AI & Natural Language Querying

Ask questions of enterprise databases in Arabic or English and get answers, tables and charts — governed by your access rules.

Learn more

AI Chatbots & WhatsApp Automation

Arabic and English assistants on WhatsApp, web and mobile that answer questions, complete tasks and hand off to staff when needed.

Learn more

Plan your next AI system with an engineer.

Tell us what you need to build. We respond within one business day with next steps, questions and — where the scope is clear — an indicative quote.

Get a quoteBook a consultationWhatsApp us