MLA 013 Customer Facing Tech Stack
Jan 02, 2021

General website setup discussion (client, server, database) in preparation for hosting your machine learning model (MLOPs).

Show Notes

Client

There are many popular front-end frameworks (Angular, Vue, etc), but I recommend React.

Server

First, get as far as you possibly can with serverless frameworks, like AWS Amplify or GCP Firebase. Or if you prefer to use individual serverless components rather than an all-in-one package, see AWS Serverless (which are the components used underneath Amplify's hood).

When you've hit the ceiling on Amplify and need custom server code, still try serverless. AWS Lambda lets you write individual routes as Node/Python functions.

Finally, after you've hit the serverless ceiling (Amplify handling most leg-work, Cognito handling authentication, Lambda for one-off routes) and you really need custom server code for edge-cases, do the following.

  1. Pick a server framework. I recommend Node.js + Express.js (JavaScript, strong concurrency & performance, super popular); or FastAPI (if you prefer to stick to Python; but it's less popular / performant).
  2. Containerize your server code in Docker. Deploy this Dockerfile to ECR
  3. Use that container to run a Fargate cluster. You may also need Route53 (domains) and ELB (domain->fargate load balancing).

mla-012 for more information.

Database, Job-Queues, Sessions

  1. Popular databases are Postgres, MySQL, SQL Server, and MongoDB. I recommend Postgres.
  2. For in-memory session-management, and real-time pub/sub, use Redis.
  3. For job-queueing (sending work-orders to your ML server), use either RabbitMQ or SQS; I recommend SQS. Use the wrapper library Celery to interface with these technologies.

All that said, the main reason I like Postgres over its competition is it can replace 2 & 3 for you. You can run a job-queue via Postgres's select for update feature, and pub/sub via listen/notify.