| Data storage | Local disk / MinIO (S3-compatible) / PostgreSQL | - No cloud bill, and data never leaves your hardware
- MinIO speaks the S3 API, so code moves to the cloud unchanged later
- PostgreSQL is a solid home for structured data
|
| Notebooks | JupyterLab or VS Code | - Free, and runs on any laptop or server
- Full control over packages and extensions
- Works offline
|
| GPU training | A local NVIDIA GPU, or rented GPUs (RunPod, Lambda, Vast.ai) | - An owned GPU has no hourly cost after purchase
- GPU rental marketplaces are often cheaper than the big clouds
- Choose the exact GPU model you need
|
| ML platform & registry | MLflow (tracking + model registry) | - Open-source, vendor-neutral experiment tracking and registry
- The same MLflow API works locally and on Databricks or Azure ML
- Easy to self-host with Docker
|
| Serverless serving | Docker container on a small VM (or Fly.io / Render) | - Simple and cheap for low or steady traffic
- A container runs the same anywhere
- Predictable flat monthly price
|
| GPU serving | Triton Inference Server / vLLM / BentoML on a GPU box | - Request batching squeezes the most throughput out of a GPU
- vLLM is a leading engine for serving LLMs fast
- No per-request fees
|
| Batch predictions | Cron or Prefect/Airflow job running a Python script | - Just a scheduled Python script, easy to understand
- No vendor lock-in
- Runs on servers you already have
|
| Pipelines | Prefect, Dagster or Airflow | - Open source with large communities
- Workflows are plain Python
- Run locally or move to any cloud later
|
| Vector database | Qdrant, Chroma or PostgreSQL + pgvector | - Free and open source
- Chroma is great for prototypes; Qdrant scales to production
- pgvector reuses an existing PostgreSQL database
|
| LLM access | Ollama or vLLM serving an open-weights model (Llama, Qwen, Mistral) | - Prompts and documents never leave your network
- No per-token charges, only hardware costs
- Pick, and even fine-tune, any open-weights model
|
| Monitoring | Prometheus + Grafana + Evidently AI | - Industry-standard open-source monitoring
- Evidently produces ready-made drift and quality reports
- Build a dashboard for any metric
|
| Privacy controls | Everything stays on your hardware; encrypt disks and restrict network access | - Full control over where data lives
- Works in air-gapped environments
- You set the encryption and access rules (and own the security work)
|