TL;DR: Build the infrastructure layer your AI actually needs—so models move from experiments to reliable, scalable production systems.
AI and ML projects fail for a lot of reasons — but infrastructure is one of the most common and most underestimated. Models don’t reach production because the data pipeline isn’t reliable. Training runs are slow and expensive because the compute isn’t configured correctly. ML engineers spend time debugging infrastructure instead of improving models. And when something finally does reach production, nobody is confident it will stay up.
I’m Amit Malhotra, a Principal GCP Architect based in Toronto with 20+ years in IT and 6+ years hands-on with Google Cloud. My focus is the infrastructure and platform engineering layer that AI, ML, and data teams depend on — the GKE clusters, Vertex AI pipelines, BigQuery architecture, Terraform foundations, and secure data infrastructure that make ML workloads reliable, reproducible, and production-ready. I’m an infrastructure and platform specialist, not a data scientist — and that distinction matters. I design the platform your ML team runs on so they can focus on the models rather than the infrastructure.
Every AI/ML platform engagement I run is guided by the SCALE Framework — my structured GCP architecture methodology. For AI and data platforms, the most critical pillars are Security by Design (protecting training data and model artifacts), Automation with Terraform (reproducible compute environments), and Elastic Scalability (infrastructure that scales with training and inference workloads without manual intervention).
The infrastructure problems that slow down AI and ML teams are specific — and often invisible until they’re causing real pain:
ML pipelines that aren’t reproducible — training runs that produce different results because the compute environment, data version, or library dependencies aren’t controlled, making it impossible to reliably reproduce experiments or debug production issues
Data pipelines that are brittle and manual — ingestion jobs that require intervention when they fail, no data quality validation, and no clear lineage between raw data and training datasets
Vertex AI or GKE compute that’s expensive and idle — GPU node pools running 24/7 when they’re only needed for training runs, with no autoscaling strategy and no cost visibility per experiment or team
No separation between training and serving infrastructure — model training and inference running on the same compute with no isolation, creating performance interference and making it difficult to scale serving independently
Data access controls that don’t reflect data sensitivity — training data in GCS buckets accessible to the entire organisation, no column-level security in BigQuery, and no audit trail for who accessed what training data
Model artifacts and experiments not tracked or versioned — trained models stored inconsistently, no experiment tracking, and no reliable way to roll back to a previous model version when a deployment goes wrong
ML engineers doing infrastructure work — your ML team spending time provisioning Vertex AI pipelines, debugging GKE node issues, and managing Terraform state instead of building and improving models
Reliable, Secure Infrastructure So Your ML Team Can Focus on Models
The right infrastructure for an AI/ML platform is invisible to the ML engineers using it — environments provision automatically, training jobs run reliably, data is where it’s supposed to be, and serving infrastructure scales with demand. My job is to design and build that platform layer so your ML team doesn’t have to think about it.
I work with your engineering and ML teams to understand how your data flows, how your models are trained and served, and what the platform needs to support — then design and implement the GCP infrastructure foundation that makes it reliable. For more on how I approach platform engineering work, see the MLOps / GenAI Platforms service page.
What I design and implement for AI, ML and data platforms on GCP:
Vertex AI pipeline infrastructure — Vertex AI Pipelines setup, Vertex AI Workbench configuration, custom training job architecture on GKE, and compute autoscaling so GPU and TPU resources are available when needed and not running when they’re not. See the MLOps / GenAI Platforms service.
BigQuery data platform architecture — dataset organisation, table partitioning and clustering for cost and performance, column-level security, and integration with data pipeline tooling
GCS data lake design — bucket hierarchy, lifecycle policies, data versioning, IAM-based access control per data classification, and integration with Vertex AI training pipelines
GKE infrastructure for ML workloads — GPU node pools with autoscaling, node taints and tolerations for workload isolation, resource quotas per team, and spot VM strategy for training cost reduction
Terraform-driven ML infrastructure — all Vertex AI resources, GKE node pools, BigQuery datasets, and GCS buckets version-controlled and reproducible across environments. See the full GCP Architecture & Modernization service.
Data security and access controls — IAM model for data access by sensitivity level, VPC Service Controls to prevent data exfiltration, Cloud Audit Logs for data access, and CMEK encryption for sensitive training data
MLOps CI/CD pipelines — automated pipeline for model training, evaluation, and deployment with quality gates, integrated with DevSecOps practices so model deployments go through the same security and quality controls as application code
What Your AI/ML Platform Looks Like With the Right GCP Infrastructure
The measure of good ML infrastructure is how much it gets out of the way of your ML team. Here’s what that looks like in practice:
Training runs are reproducible — the same code, data version, and compute environment produces consistent results, making experiments reliable and debugging tractable
ML engineers focus on models, not infrastructure — compute provisions automatically, data is where it’s supposed to be, and pipelines run without manual intervention
GPU and TPU costs are controlled — autoscaling node pools and Spot VM strategy means compute scales with actual training demand rather than running idle between jobs
Data is secure and access-controlled — training data, model artifacts, and pipeline outputs are protected by IAM policies, encrypted at rest, and auditable — meeting enterprise customer and compliance requirements
Models reach production reliably — a CI/CD pipeline for model deployment with quality gates and rollback capability means production deployments are controlled events, not manual uploads
Serving infrastructure scales with demand — inference endpoints on GKE or Vertex AI scale automatically with request volume, without manual node pool management
What Your AI/ML Platform Looks Like With the Right GCP Infrastructure
The measure of good ML infrastructure is how much it gets out of the way of your ML team. Here’s what that looks like in practice:
Training runs are reproducible — the same code, data version, and compute environment produces consistent results, making experiments reliable and debugging tractable
ML engineers focus on models, not infrastructure — compute provisions automatically, data is where it’s supposed to be, and pipelines run without manual intervention
GPU and TPU costs are controlled — autoscaling node pools and Spot VM strategy means compute scales with actual training demand rather than running idle between jobs
Data is secure and access-controlled — training data, model artifacts, and pipeline outputs are protected by IAM policies, encrypted at rest, and auditable — meeting enterprise customer and compliance requirements
Models reach production reliably — a CI/CD pipeline for model deployment with quality gates and rollback capability means production deployments are controlled events, not manual uploads
Serving infrastructure scales with demand — inference endpoints on GKE or Vertex AI scale automatically with request volume, without manual node pool management
Building AI or ML Infrastructure on GCP? Let’s Talk About Your Platform.
AI and ML infrastructure is a specialism — the compute, data, and security requirements are different from standard application platforms, and getting the architecture wrong creates compounding problems for your ML team. I work with AI and data teams who need the GCP infrastructure layer designed and built properly so they can focus on what they’re actually there to do.
I start with a free 30-minute architecture review — a direct conversation about your current GCP setup, your ML workloads, and what the platform needs to support. You work with me directly, not a delivery team. Book a free architecture review here.
Let’s Talk
Based in Toronto (EST), working with engineering teams across Canada & USA
Tell me about your current platform and where you're trying to get to. I'll respond with thoughts, not a proposal.
