Qoves Studio sells AI-driven facial analysis to consumers. I joined in October 2024 as founding backend engineer and inherited no platform standards. The first months were spent defining them: how work enters the system, how it runs, and how we know it broke.
The shape of the workload#
Customer images go through AI processing, analysis, and report generation, then the report syncs with billing because reports belong to subscriptions. The natural split was:
- an API tier for customer-facing requests,
- a queue tier for heavy AI work,
- workers sized independently of the web tier.
Celery and Redis carry the queues, Kubernetes autoscales the workers, and FastAPI plus Django handle the synchronous surface.
@app.task(bind=True, max_retries=3)
def generate_report(self, analysis_id: str) -> None:
analysis = Analysis.objects.get(pk=analysis_id)
try:
report = pipeline.run(analysis)
except TransientError as exc:
raise self.retry(exc=exc)
report.publish()The retry policy matters more than the pipeline. Vision and LLM calls fail for reasons outside our control: provider timeouts, rate limits, cold GPUs. Retries with backoff are the difference between a bad minute and a support ticket.
Standards before features#
Three platform-wide rules shipped before any new AI feature:
- Observability. Every service emits metrics and traces; Sentry catches exceptions; dashboards exist before the incident.
- Deployment automation. One path from merge to production, no manual steps on release day.
- Reliability budgets. Queue depth, error rate and p95 latency have owners.
These are boring. They are also why we could add LLM features weekly without asking whether the platform would hold.
What I would tell a founder hiring their first backend engineer#
Write down who decides. I am the primary technical decision maker for architecture, cloud and AI platform at Qoves, and that clarity removed a lot of meetings. Give your first backend engineer the decision right and the dashboards to prove the decisions work.