Joining with no platform
When I first stepped in, there were no platform standards. No agreed way for work to enter the system, no agreed way to run it, no agreed way to know it broke. The first months went into defining those three things, because every feature after them depends on them.
The workload had a natural shape. Customer images go through processing, analysis and report generation, and every report belongs to a subscription, so the AI pipeline syncs with billing. That shape became the architecture: an API tier for customer‑facing requests, a queue tier for heavy AI work, and workers sized independently of the web tier. Celery and Redis carry the queues, Kubernetes autoscales the workers, and FastAPI plus Django handle the synchronous surface.
What shipped first
Three platform‑wide rules shipped before any new AI feature:
- Observability. Every service emits metrics and traces, Sentry catches exceptions, and dashboards exist before the incident does.
- Deployment automation. One path from merge to production, no manual steps on release day.
- Reliability budgets. Queue depth, error rate and p95 latency have named owners, not shared blame.
Boring by design. These rules are why we could add LLM features weekly without first asking whether the platform would hold.
AI processing at scale
Vision and language workloads are heavy and can fail for reasons outside our control: provider timeouts, rate limits, cold GPUs. The queue tier absorbs that. Workers scale on queue depth, tasks retry with back‑off, and a failed report costs a retry instead of a support ticket. Distributed microservices keep the heavy jobs off the request path, which is what keeps customer‑facing latency flat while processing volume grows.
Around the pipelines sit the rest of the platform: MLflow tracks models and experiments, Prometheus, Grafana and Sentry watch the services, and GitHub Actions runs the merge to production path on AWS and Azure with Terraform managing the infrastructure.
Evaluation and QA
Every change to a model, prompt or pipeline stage runs against a golden set of images with known outcomes, and regressions past tolerance fail the build. Automated checks catch numbers; people catch reports that read wrong, so each release candidate also gets a sampled human read against a short rubric.
This is slower than shipping on a good demo, and it is the reason AI releases stopped being incident responses. Full detail lives in Evaluation and QA at Qoves.
Owning the decisions
As founding engineer I am the primary technical decision‑maker for architecture, cloud and the AI platform. That clarity removed meetings: architecture choices, provider choices, scaling policy and cost trade‑offs are mine to make and mine to defend with dashboards. The rule I would give any founder hiring their first backend engineer is the same: write down who decides, and give that person the data to prove the decisions work.
Where it stands
Two years in, the platform runs high‑volume AI workloads with queue‑driven processing, autoscaling on Kubernetes, and evaluation gates in front of every release. The product surface grew around it: image analysis, report generation, subscriptions, and the customer APIs, all on the standards defined in the first months. If you are building something similar, the write‑ups on the blog cover the parts other engineers ask about most: the founding platform in Founding backend engineer at Qoves Studio, and report pipelines in From image analysis to a report customers trust.
