MLA 017 AWS Local Development Environment

Welcome back to Machine Learning applied. Today we're gonna be talking about developing within the AWS environment. In other words, using AWS as your local development environment. Now, before we get started, let's reflect back on a prior episode about Docker. I did an episode where I said, you can package up an environment and its dependencies and the project's source code into a Docker container and deploy that docker container to the cloud. What we do is we write a Docker file, literally called Docker file with a capital D. At the top of that docker file, you specify the operating system you're gonna be using, and then within the Docker file, you're gonna specify any number of operating system level packages you want to install like FFM, PEG or Cuda Co DNN, and you might install some PIP packages. You can either directly inline the Docker file, say PIP install X, Y, and Z. Or you can have a requirements text file that gets copied into the Docker container, and then that thing gets kicked off with a PIP install of the requirements text. And then what you'll see in the Docker files is copy some host directory to the container directory all capitals. COPY copy space. The location of the source code on your computer relative path. So if you're working within your project route and at your project route, there are a handful of miscellaneous files. And then within the source directory, that's actually the source code for your project, all your Python files. And then within the Docker container, it's expected to be running out of the forward slash app directory. What you'll see is copy forward slash source space forward slash app. And what that will do is it will take your Python files from the source directory and copy them into the app directory of the docker container. Then you'll typically have a Docker composed Yml file, and that file will spin up a bunch of different Docker containers. It will handle the orchestration of multiple Docker files at a small scale. If you really want to be deploying lots of Docker containers at scale and handle the networking of them all together, you'll be using Kubernetes, not Docker compose. Docker composes for very simplistic environments, like on local hosts and Kubernetes is for large scale deployment of Docker containers to the cloud. But we're not talking about Kubernetes here. We'll talk about that in a later episode. It's sort of mutually exclusive to the type of DevOps that I'm talking about in this episode. In the last two episodes, local hosts, you have a Docker composed file. It spins up a bunch of docker containers and it handles the networking of those containers to each other. Now, as you're developing on local host against your Docker containers, what you'll see in those Docker composed files is volume mounts mounting. It will take your local source directory and it will mirror it into the running Docker container. So the copy command will actually copy the files into the Docker container one time Snapshot Fire and forget. But the volume directive will mount your local directory into your Docker container at that destination, and then that way you can actually develop within your Docker container on local hosts. As your programming in pie charm on your local host, your changes to your source files get mirrored to those source files location in the Docker container. So it makes editing your files on local hosts while you're developing a breeze. But that copy directive is the the thing that will actually be used when you actually. Build your docker container. You run docker build, and then you push that up to the cloud. That copy directive is what takes a snapshot of your files in the source directory it. It copies them over to that location in the docker container one time, builds the docker container and then pushes that to the cloud. So the copy command is a one-time deal, the finalization of copying your files over into the docker container. But the volume directive allows you to mirror your development environment into the Docker container so that you can edit within your Docker container on your local environment. Now, this seems like a lot of weird docker stuff before I getting into AWS, but you'll see why there's an analogy to this Docker and Docker composed stuff when we're talking about developing on AWS in your local environment. The other reason I'm bringing up Docker again in this episode is that Docker is still gonna be used quite extensively in AWS and in this episode when I talk about using AWS as your local environment, Docker is used very extensively, all over the cloud, all over the internet, all over AWS. Even when you're using their managed services like AWS Lambda, AWS, SageMaker, ECS, fargate, these are all things that you can use Docker within, and it is recommended that if you have that option that you do, so for example, an AWS Lambda, which lets you deploy single Python functions to the cloud as either a rest endpoint or some python function that you'll be calling one-offs within your text stack. Well, you can write your Python function and provide along with it a requirements text file and deploy that as a lambda function. But if you have a lot of requirements in your requirements text file, it is preferred that instead you package this all up as a Docker container and you deploy your Lambda function as a docker container. And that's for various technical reasons. One is that if you deploy Lambda functions in the traditional way, which is actually to build those PIP requirements on local hosts, zip them, put them into a S3 bucket as a zip file, and then you deploy your Lambda function from there, which is pretty sloppy in my opinion. And also, there's a file size cap, and if you have a lot of requirements, you're gonna exceed that cap very. Easily and oftentimes the default Lambda environment, say for example, the Python 3.8 Lambda environment doesn't have the same sort of operating system set up that is needed for a lot of these PIP installs. NumPy being a huge one. NumPy doesn't work out of the box in the default Lambda environment, so getting it working in a default Lambda environment is kind of a pain in the neck. Instead, just dockerize your app and you'll use that Docker container as your Lambda function because installing NumPy within a Docker container, well, it's all done for you. That's the magic of Docker containers. They're siloed environments at the operating system level. So a bunch of docker buildups. So now let's talk about AWS. Here's a pain point that I suffer from when I want to deploy no to the cloud. Well, I have a Docker composed Yml file on local host. It has a Postgres docker container, a fast API Docker container, and a client docker container. And the server is able to communicate effectively with the database just fine because they're sharing the same network bridge. It just accesses the port on local host 5, 4, 3, 2, and the client is able to access the server because again, the containers, the fast API Docker container sets up all of the proxying of network requests and exposing the ports to local hosts on my behalf. And so the client hits the server, the server hits the database. And then database back to server, back to client. Fine and dandy on local host. But when it comes time to deploy this to the server, that's not how AWS operates. AWS is much more complex than that. In order to get my server Docker container in the cloud, I have to first push the Docker container to ECR Elastic Container Registry on AWS. Then I need to set up AWS Fargate, which is a sub-service of ECS Elastic Container Services on AWS for hand. ECS is for Docker stuff is for deploying your Docker containers. In an AWS managed environment, I have to pull from ECR to Fargate and then set up my whole fargate stuff. Okay, so step one was ECR. Step two is setting up Fargate. Step three I. Is getting network requests to my Fargate container. Now, how do I do that? On AWS, you'll be using elastic load balancer A LB, or you'll be using API gateway. And then both of those services need to be set up for handling network requests, proxying, HTTP requests on down to your Docker container. And those services have to be tied to a domain name. That domain name is gonna come from AWS Route 53. And if you want SSL, if you want H-T-T-P-S, then you're also gonna need to use the A-W-S-A-C-M service, Amazon Certificates Manager. My database is gonna be deployed to RDS. That's relational database service, and exposing that database to my Docker container on Fargate is no small ordeal. I have to set everything up within an A-W-S-V-P-C, that's a virtual private cloud. I need a private subnet, a public subnet with a internet gateway. The server needs to be placed within the private subnet and in order to have outbound traffic, it also needs a NA gateway. The database needs to be within the private subnet and also needs to have database subnets associated with it. All the parts within this stack need to have security groups set up so that they can communicate with each other, and then the part that is exposed to the internet, either application load balancer or API gateway, is gonna be within the public subnet. So that was a whirlwind. Did that confuse you? Good. That's the point I'm trying to make is that everything you do on local host is almost an entirely different language to how you're gonna be putting this into the cloud, how you're gonna be putting this on AWS. And so what I offer to you to do instead is stop developing on local host in a Docker composed file, unprepared for AWS when that time comes, and that time will come instead, develop everything directly within A AWS. Develop your entire environment from the get go in an AWS tech stack. Now, how the heck would you do this? You have your local computer and then AWS is the cloud That's for hosting things in the cloud in in a production or a staging environment. What do you mean develop in AWS I'm developing on local host and AWS is the internet. That doesn't make sense. Well, it kind of doesn't make sense, but there are ways to do this, and that's what we're gonna be discussing here. There are a number of ways to do this, and so I'm gonna tackle them bit by bit. The first is the most obvious way to handle this, and that is to set up your environment in the cloud. In AWS first what you'll do is you'll go on AWS console, you'll create an account, go into the console, and then the first step is to go into A-W-S-V-P-C-A-V-P-C, or a virtual private cloud is sort of the entry points to all your AWS stuff. It's what encapsulates your services together into a single unit and allows them to communicate with each other through networking and keeps them private from the internet and safe and secure. And then you'll decide how you're gonna expose that VPC to the internet as the case may be. So for example, if you want a rest server or a web sockets server or GraphQL, you'll set up your VPC by way of security groups and subnets and internet gateways and nat gateways and all these things such that one port is exposed to the internet and strung up to some service. That service may be EC2 or ECS or Fargate or Lambda. So you go and you create your VPC, you set up your subnets and your security groups. Then you go on over to the IM console, and you set up your users. And you set up your user and some permission stuff. Then you head on over to AWS Lambda, and then from within Lambda you can write your Python code as one-off Python functions. And then you tie that up to API gateway and you expose API gateway to the internet. And what's cool about Lambda is that there's actually a code environment on the Lambda console. You can actually write your code for your Lambda functions in the web browser using their Python, IDE, or their JavaScript, IDE. So this is option number one. Option one. Number one is not having a local environment whatsoever. You write all your code in the cloud in these Lambda function handlers because there is an. IDE on Lambda that lets you edit your co, your Python code and then deploy these lambda functions, test them, et cetera. There's also an AWS service called Cloud nine. It's an entire IDE. It sort of replaces Adam on the local host or pi charm on local hosts. Now cloud nine does not hold a candle to to pie charm. Let's be clear, cloud nine is meant for simplistic editing of code in the cloud on AWS, but it is another way to handling this option. One I'm discussing, which is not having a local host whatsoever, and instead doing everything in the browser on your AWS console. So cloud nine, you can. Get clone a GI repository from GitHub and you can edit your files on Cloud nine. And it keeps track, of course of changed files from Git. So then you can, after you make those changes, get, commit and push those changes. And Cloud nine is strung up to other AWS services like ECR and Lambda, so that it lets you run your services for testing your Python functions. So if you were serious about editing your code in the cloud, you would not be using the Lambda code editor as your bread and butter. You'd be using cloud nine to edit your Python files and string those up to Lambda within Cloud Nine's functionality for connecting your code to Lambda. The Lambda Code editor is more for quick edits to your Lambda functions. Then of course per the last two episodes I discussed that SageMaker Studio has I Python notebooks in the studio environment. So you can edit and run and test and deploy your machine learning code in the SageMaker studio. It's using a different IDE than the cloud nine IDE. It's using I Python, these Jupyter Notebooks. So Amazon didn't write the Jupyter Stack, it's just what's commonly used by data science and machine learning engineers. So they just set up an environment on SageMaker so that you can use the tools that you're familiar with, namely Jupyter Notebooks, I Python notebooks. So if you're writing, testing, running. Deploying your machine learning code. You can do that all in the cloud on SageMaker studio in I Python notebooks. And if you're doing the same for web app, for server application code, you can do that on cloud nine. Now that's a cool option. Developing everything in the cloud. There's a handful of benefits there. One is that you're not tied to your computer, you're not stuck with a specific operating system supporting some functionality that you need. Or let's say you have a work computer and a home computer and you want to be able to pop back into your development environment. Well, it's in the cloud, so it's gonna be propagated from your desktop at work to your desktop at home. 'cause it's just running in a web browser or something. I faced recently, actually, my computer died, my Mac died, and so I had to transfer my environment over to my pc and that was kind of a pain in the butt. If I had set this all up in Cloud nine and SageMaker Studio, I wouldn't have to deal with that. The downside of this setup is that Cloud Nines and SageMaker Studio, these are not enterprise grade IDs. These are not super powerful IDs, and a lot of times we really like our local host environment. We like the tooling that we have on our computer. I love pie charm, and I love data grip. Data grip. It's for database management. It's a jet Brains IDE for data. I can browse the tables, I can look at the table metadata, I can run SQL queries, I can sort edit in line, et cetera. And then pie Charm is my preferred IDE for Python development. It is just wonderfully powerful Cloud nine and SageMaker Studio. Don't hold a candle to pie charm when it comes to just raw python editing capabilities. So I'd still rather use these things and therefore I don't use option one that I just discussed. Instead, let's move on to option two. Option two is that you set up everything in the cloud. Like I just mentioned. You have your VPC and your I Am roles and your I Am user. You have your RDS database, your Lambda functions. You have your ECR, Docker, CONT docker repository, API gateway, blah, blah, blah. But when you want to edit your code on local host and run some unit tests against services in your tech stack on AWS. You have to take one step, and that is to connect your local computer to your AWS Tech Stacks, VPC, by way of something called a client, VPN. So you can set up something called a client, VPN. And what that does is it allows you to connect your local computer over a tunnel, an internet tunnel, to your A-W-S-V-P-C, and then now when you want to connect to a database in your VPC on AWS your RDS database, you can do so because you are operating within the VPC. You wouldn't otherwise be able to do that because your database wants to be contained within a private VPC. It doesn't want to be exposed to the internet. You only want your database to be accessible to services running as servers, whether as Lambda functions. Or on Fargate or ECS or EC2. And those servers also want to be within the private subnet of your VPC and only accessible to the internet by way of the internet gateway on the public subnet. And that is tied into either API gateway or application load balancer. So you don't have access to these resources, these services on local host, but you can get access to them if you connect to your VPC by way of a client, VPN. So what would you do? Well, you would write your Lambda functions as Python functions. You would test those functions. Now, these functions may be making requests to a database on RDS, or they may be. Grabbing secret access keys using AWS Secrets Manager. That's one way to store private data, like database username and database password. These are not things you wanna store in a config json file or as environment variables. These are things you want to store in a Secrets Manager, which handles secret rotation and encryption and all these things automatically for you. Another thing that you don't really think of until you're starting to push your stuff to AWS, so your Python function can access your RDS database and your Secrets Manager Secrets and maybe SQS and SNS and these other AWS services. And you can write your Python function and you can run a unit test against that Python function and it will work because you're operating within the VPC of the deployed AWS environment running in the cloud. And then when it comes time to actually deploy those functions, then you do the step of packaging up those Python functions into zip files or docker containers and pushing them up to AWS Lambda. I. Then finally, the third option. The last option is there is a service called Local Stack. Local Stack. It is a open source project that replicates the AWS Tech stack, and it's all running on Docker containers. On local hosts, you have a Docker composed Yml file that spins up a whole bunch of local stack services, and those services are running on local host, not on AWS, on local host, and they are fake AWS services. So you have a local Stack S3 service. So what you do is you'd go into your Docker compose Yml file, you'd enable the S3 service. You'd say Docker compose upd. Local stack will spin up a Docker container whose end points that you'll be accessing with your project. Replicate everything that a W S's S3 service offers. Same for ECS and ECR and code deploy and code pipeline and RDS in every service. And almost every service that AWS offers, that's actually quite an undertaking. That's a really powerful service and I'm surprised that it works as well as it does that they're accounting for all of these endpoints that you're making. Calls to the AWS Stack by way of REST calls or CLI calls the A-W-S-C-L-I or BODO three calls. This local Stack project tries to replicate all AWS services and all calls that can be made to those services and also replicate how those services run all on local host using Docker containers. It's pretty impressive and it's a daunting and overwhelming undertaking. There's gotta be some loose ends, I imagine, within the local Stack project. I haven't used it. Extensively, but the amount to which I have used it, it has worked surprisingly well. So I'm gonna keep using it for the time being. But one thing to note is that it has a free tier and a paid tier. And the free tier is all the typical serverless stuff like Lambda and S3 and DynamoDB. And the paid tier is all the rest of the stuff like RDS and A CM and Route 53 and all these things. And it's 15 bucks a month I, if I recall correctly. And so if you're gonna be using Local Stack extensively for setting up your AWS stack, you'll definitely want to use their pro version. The benefit of using local stack over an actually deployed AWS stack is one cost savings. Because if you're deploying an AWS stack and you're getting into that stack from local host by way of a client VPC, then running these services in the cloud may cost you quite a lot. For example, RDS, the database service, that's kind of an expensive service, where previously I was running my Postgres database as a local docker container. Now I'm running it actually as a hosted RDS database on AWS. The main reason being that I want to develop against what I'll actually be interfacing with in the real world when I'm developing against a local. Docker container hosting Postgres. Well that's really easy to work with. It bypasses all the stuff that I would need to set up in the cloud. So instead, I deploy an RDS instance to A WSI client VPN, into that VPC. And I actually have to make sure that my IM policies and my subnets and my security groups are all set up correctly so that I can connect to the database. So it's making sure I do things right the first time. Measure twice, cut once, and running that RDS database is a little bit expensive. It's not terrible, but it's a little bit expensive. But if instead I ran all that. On local hosts using local stack, which is using Docker containers, but the environment is set up such that it acts the way RDS would act in the cloud. So I still do need to set up those VPCs and subnets and security groups in order to connect to my local RDS database. Then I'm saving the cost of running that database in the cloud. I'm paying 15 bucks a month for the pro version, but that's still cheaper than what I'd be paying for. Spinning up an RDS instance multiple times throughout the week in the cloud, and it all runs a lot faster because it's running on local host. There's low latency in making these network requests to local hosts as opposed to the cloud. Connecting and interfacing with these services will be a lot faster on local hosts as opposed to client VPNing into A VPC. And if I make changes to my AWS stack by way of Terraform, for example, which I'm gonna be talking about in a bit here, then deploying those changes is a lot faster on local host using local stack than it is on AWS. When you actually kick off infrastructure changes on AWS, it's actually like bringing online hardware and moving things around from one physical location to another physical location so that your architecture changes get reflected in the cloud before it's available to you to now VPN back into to test your changes. Doing that all on local hosts through Local Stack, it just brings all the Docker containers down and brings them back up the way it needs to. And it's really fast. It's really easy to deploy your infrastructure changes using Terraform on local hosts with local Stack. Now I brought up before the idea of mounting your source directory into your Docker container as opposed to copying the source into the Docker container for a build, there is essentially an equivalent of that for local stack. There's a way in local stack for mounting your local host source files to your Lambda functions that are being deployed as Lambda REST endpoints in your local stack infrastructure. And that really is the key here to treating AWS as a local development environment because otherwise, without being able to mount your code to Lambda functions that are as. Quote unquote deployed. You're gonna have to basically write some code and then deploy it to Lambda. Did it work? No. Okay. Tweak that code. Deploy it to Lambda. Did it work? Man, that's gonna be a really painful process. So another huge benefit of local Stack is that it allows you to mount your local Python functions to your quote unquote deployed Lambda functions. And that way you can continue to enlighten and edit your Lambda functions in pie charm while developing, running, and testing your code. So those are your three options We have Develop everything in the cloud. You don't even use an IDE, you use Cloud nine and SageMaker Studio. That's option one. Option two is spin everything up in the cloud, but develop on local host. But while developing on local host, you're still connecting to services in the cloud. You're not connecting to running Docker instances on local host know you're connecting to running AWS services. Or Docker containers running on AWS services, and you are connecting to those by way of a client, VPN. And then the third option is that you're running everything on local hosts in local Stack, which replicates the AWS Tech stack. And now you can access it all on local hosts and you can actually connect to it, presumably with a client VPC as well. So if you wanted to set up your client VPC on local hosts to local stack so that you can do that again in your cloud environment, when it comes time, you want to just do a few quick tests against your deployed environment before kicking everything off. One other major benefit of developing against AWS first is that you become acquainted with the offerings of AWS and you start to realize that there are a lot more offerings on AWS that can replace things that you would otherwise use in your project as PIP packages, for example. And that will actually get the job done better, more securely and hosted. And that if you had developed everything on local host in a Docker container, you might not have known about, you might have foregone. And then when it comes time to deploy to the cloud, you wish you would've used that AWS service instead. So for example, in the SageMaker episodes I talked about using Data Wrangler to transform your features and impute missing data and all these things. Well, if I had known about Data Wrangler when I was writing a lot of my machine learning code in the past, I would not have done that custom by hand in Pandas because data wrangler's handling of it is gonna be more robust, simpler to set up and scalable. Importantly, with data coming in from some stream of the data lake I'm gonna be using. Eventually deploying no fee. I'm gonna need that data pipeline to scale and the way it's written now is not scalable. So I will be transferring my custom feature transformation code to data pipeline so that it can scale so that there's various points in the pipeline I can tap into. For example, getting the journal embeddings at different entry points of my machine learning architecture. I wish it'd have known about that first 'cause I wouldn't have written it the way I did. I assumed that this needed to be done in Python. Another component being SageMaker experiments, their hyper parameter optimization capabilities. I would rather had I known, have used experiments so that I can distribute and scale running my hyper parameter optimization jobs rather than the way I handle them all on one computer at present in my GPU based docker container. Well, it doesn't stop at SageMaker. There's a whole bunch of services on AWS that can replace components of what would be, for example, your Python server. So another Docker container in no is the entire web server. The whole thing is running on Fast API and fast API handles a whole lot of things for you. And there are third party plugins for FAST API for handling other things. One example is the proxying of requests. Well, API gateway or application load balancer will do that automatically for you. You don't need to be aware of like Engine X and how we're moving HTP requests from the internet to your container. Another thing is actually load balancing. Or for example, let's say you want some rest endpoint of your server to be throttled so that any one user on the internet can't hit that endpoint a million times a minute. Well, API Gateway has that built in. You can just click a check checkbox and say, throttle this route. We only want one user to be able to hit this endpoint X amount of times per second. Where previously I was using a fast API plugin for throttling specific rest routes. The plugin is called Slow API get it fast. API. Let's slow them down. Well, I'm gonna gut that and I'm gonna use API Gateway's built-in handling of throttling. Then finally another example is the storing of user accounts. This one's really important. Currently I'm using a fast API plugin that handles storing user accounts to the database, their username and hashed password and email invalidating them as necessary, sending, forgot passwords if necessary, sending a activation code, and I have to string all that up myself. I have to specify those off routes as throttled using slow API and I have to send up a JWT. Jot we call it, which is sort of the authentication token that's commonly used in the web development space. When you log into a user account on the server, a jot gets sent down to the client that gets stored somewhere like A-H-T-T-P only cookie on the web browser, and then you send up that jot to your server to authenticate rest requests for that user. And you also have to handle invalidating that jot if the user changes their password, you have to handle expiring that jot every five minutes or so, and then requesting a refresh token and all this stuff. All that stuff around authentication took me weeks even with the plugins and tooling available to me from the fast API ecosystem, and I'm just not comfortable handling authentication on my own. Even with it being managed as open source projects, I would rather there be a service dedicated to user account storage and authentication and all of the security and the emails of new account and forgot password and activation and invalidation and all these things. I want a service that does this automatically for me. And lo and behold, AWS has a service called Kognito. I did not know that. I didn't know it until I started putting a lot of my stuff on AWS. If I had known that in advance, I would have developed the user account system in Kognito. Now I am migrating the user account system from Fast API to Kognito. And so one benefit had I been developing against AWS first instead of a docker based local environment, is that I probably would have discovered this service early on in the phase, and I. Then I would've measured twice and cut once. In fact, as I'm going through my server module by module, endpoint by endpoint, what I'm finding myself doing is gutting the entire server stack the entire server stack, and instead moving everything into single AWS Lambda functions. So previously I had one big old docker container with a bajillion python functions all as fast API. Rest endpoints. Well now with all of the tooling of available to me by way of API gateway, API Gateway's web sockets support their authentication and authorization handling by way of kognito and the authorization header, hitting API gateway's jot validator. I can just write these functions as single Python functions without any tooling, you know, like database connection, pool management and stuff like this, and instead leave it to AWS to handle all the meta tooling that I would otherwise be handling myself by deploying each function as single Lambda rest endpoints. In a way, a lot of AWS's offerings sort of replace your, not only your infrastructure stack, which is obvious because that's what AWS is, is cloud hosting, thereby infrastructure, but also replacing a lot of your modules and plugins and frameworks that you would be using at the server or hosting level. Now setting up your environment, the traditional way is to go onto the AWS console in the web browser, and then you click around, you set up a VPC, an IM user, an RDS database, some Lambda functions, string everything together, put it all behind API gateway. Clicking around and typing in things this way is not manageable. Very unmanageable. And it becomes more and more unmanageable over time. And you want to be able to track these change sets, these changes you make to your stack in some way that you can replicate in the future. So for example, if you are spinning up a tech stack, an infrastructure, we call it on AWS, for setting up a database and API gateway in Lambda functions, and you're doing this to get away from your local docker composed yml file. And you previously had your Docker composed Yml file in Git on GitHub. Well then future users and future, you will miss the tracking of infrastructure changes in Git. And you would have to sort of describe to a, a future user how to set up your tech stack with clicks. Keeping track of and managing your infrastructure in the web console is a non-starter. It is not the way to handle managing your infrastructure on AWS. Instead, what we use is a concept called infrastructure as code or infrastructure in code. And there are a number of projects out there for this. Chef and Ansible, but a very popular one is called Terraform, T-E-R-R-A-F-O-R-M, Terraform, that's the one I use. And Terraform lets you write code that sets up your tech stack on AWS, it's infrastructure as code. And then that way you can. Put your Terraform files into Git, into your GitHub repository, and so that when you move from one computer to the next computer or another user comes on board and they want to set up the same infrastructure for testing the code and running that project, they can just run Terraform in it and then Terraform apply and it will take those Terraform files and it will set up the infrastructure on AWS. So what you do with Terraform. Is that you? The first step is to make sure you have your A-W-S-I-A-M credentials set up on local host, and it will use those I am credentials to make sure it can spin up services in the cloud. It has the right permissions to create services, managed services on your behalf on AWS, and then you write these Terraform files. One might be for setting up your IM roles. Another one might be for setting up your Lambda functions. Another one for your RDS database, another one for your VPC, your security groups, your subnets, your client VPN, for setting up your client VPN, and your. Certificates on a CM for S-S-L-H-T-P-S and your Route 53 domain name and all these things. You use some off the shelf sample Terraform files, you modify them to your liking, and then you run Terraform and it and Terraform apply and it will set up your entire infrastructure, your entire tech stack on AWS in the cloud. And then you can connect to that tech stack with your client. VPN. Then at the end of the day, when you're done with your development, when you're done coding against this stack, you can run Terraform Destroy, and it will bring all of that infrastructure offline. So if it weren't for Terraform or CDK, or Serverless framework or one of these other things that I'll be discussing in a bit these infrastructure as code projects, if it weren't for these projects, I would not be doing this episode because it would not be worth managing the setting up of your infrastructure all through the web ui. Because if that's what you had to do anyway, if you were stuck with stringing together your entire infrastructure on AWS with clicks and typing in web forms, then the pain point that I mentioned at the beginning of this episode of your local environment being different than your cloud environment, well that would be a moot point. You have to set up your cloud environment the hard way anyway. But because we have this infrastructure as code project like Terraform, we can orchestrate the development of our cloud environment in code and then we can put ourselves into that environment and that way we, like I said, we measure twice cut once we set up our environment the way it. It's intended that we can develop against, we do the hard work first, and then we can replicate that environment to a staging environment, to a development environment, to a testing environment, and then finally to production. And Terraform is not the only option out there. Like I said, Ansible and Puppet and Chef, these are all popular frameworks as well. Other frameworks, a little bit more popular and modern that I see in use today. There's Amazon CDK, that is the Amazon Cloud development kit. It's essentially like Terraform. It is a infrastructure as code framework written by AWS for AWS, for deploying your services to the cloud and then taking them offline if you need in the future. Now, why the hell would I not use CDK? I'm championing everything AWS in this in the last two episodes. And CDK is written by AWS for AWS. Why the heck would I not be using CDK and instead I'm using Terraform? Well, there's a couple of reasons and they're a little bit more personal for my project. One is that at present, currently, CDK is a little bit newer than Terraform. And one of the limitations I find with CDK is that it does not support non-managed services like cloud native services, for example. So one sticking point I got hung up on when I was working on a project. At, so one point I got stuck on when I was working on a project at depth was that I wanted to deploy a chat bot. And I did this by way of AWS Lex. AWS Lex is a chat bot service. It lets you set up a bot with a handful of conversation flows and any number of responses to given utterances, and then the slots, the named entities that were gonna be pulling out of the user's responses. So if you had a customer service bot and it said, hi, how can I help you today? And the person said, credit card help, it would pull out credit card as a slot. Well setting all this up, this is not a service that you manage like EC2 or Fargate in the traditional sense on AWS. This is a cloud native offering. Cloud native means these rest calls or these Bodo three calls that you make, and you don't have to manage the running and hosting of your servers. CDK does not support currently November, 2021 does not support setting up a Lex bot, but there's a bunch of stuff you need to do to set up a Lex bot. Sure, it's not a a managed service, but there is still stuff you need to do to set it up. You gotta set up these example conversations. And the types of slots we want to pull out and stuff. And so Terraform supports that, but CDK does not. Terraform supports a lot of services, a lot more services than most of the other infrastructures code projects out there that I've found, including most of the cloud native and serverless capabilities on AWS and Azure and GCP. So that's the main reason I chose Terraform over CDK. Another reason lots of people choose Terraform over CDK is that Terraform is universal. It works on GCP and Microsoft Azure and other cloud services. Other cloud hosting providers, whereas CDK is just for AWS. The benefit of CDK is that it's easier to use and it's Amazon first, so first class citizen. So CDK will continue to improve at a rapid clip, supporting more and more AWS services and tying into them very effectively, maybe in a way such that we're terraform because it covers such a wide territory, would give any one AWS service, a little bit of second class support. Just by the nature of it being such a big project and a big undertaking. CDK would be a little bit more fine tuned and dialed in and first class support for these AWS services. I don't know that that really is something you need to worry about. I have not hit. Any shortcomings on AWS with Terraform at present, it supports everything I've ever needed it to support in a very efficient and bug-free capacity. But the biggest thing I see, the biggest difference I see is that Terraform is very verbose. It is complex. There's a huge learning curve where CDK is really dialed in and sleek and very efficient and streamlined. It's less code. CDK is less code than Terraform, but Terraform covers more services. Take your pick. It's up to you. I'm not gonna champion one over the other, but I personally use Terraform. And then there's this thing called the Serverless framework, which is a really popular infrastructure. This code. Framework, but it really mostly supports serverless technologies. It's really dialed in for AWS Lambda and Dynamo DB and API gateway, and less so for RDS. For example, RDS is not a serverless service. It is a managed, hosted database in the cloud, and so the serverless framework is less likely to be compatible with these non serverless services and more compatible with the serverless services like AWS Lambda and DYNAM O db. So that's another reason that I choose Terraform over serverless. Now finally, we talked about local stack hosting your entire AWS Tech stack on local host Terraform and CDK are compatible with Local Stack. So you write code in your Terraform files that tells Terraform we're actually pointing to local host, not to the cloud. You've set up a whole bunch of endpoints inside your Terraform file, you say S3 goes to local host, dynamo DB goes to local host API Gateway goes to local host, and then the next time you run Terraform apply, it actually runs it against your local stack environment and it works. It actually spins up a local host environment using Terraform infrastructures code the way it would do against the actual AWS cloud. And then again, a huge benefit of terraform, of infrastructure as code generally is that if you wanna make a modification to your tech stack, you run Terraform apply, and then it applies that edit. Just that edit, it just applies that modification and it will propagate modifications through your tech stack as the case may be, which is super powerful. So for example, if you make a modification to your VPC or to a subnet within the VPC or a security group within some subnet. Well, if you were to do that in the browser, you would have to then go to all your services, every single service that needs to be made aware of that modification, and you would have to then click into that service and then adjust for that modification as needed. And some things are gonna slip through the cracks. You're gonna forget about some services that need to be made aware of those modifications. Terraform will not. If you make a modification to some deployed service in your infrastructure, and Terraform is aware that that is a dependency of other services in your infrastructure, it will first make the modification to that service, let's say VPC, and then it will propagate the necessary modifications through your entire infrastructure to those other services. It knows how to make. Only the modifications necessary without doing anything too destructive. Another example of non-destructive modifications that Terraform is cool at is if you write multiple Lambda functions on local host, and let's just talk about Lambda here. This is really powerful. This will showcase how efficient Terraform is. Let's say you have five Lambda functions, five Python functions on local hosts. Maybe a handful of them are web server functions. A handful are one-off functions that you want to call from within some other area of your stack. Let's say one function is actually Aron job, so you can. Connect your local host to your A-W-S-V-P-C using client VPN so that you can run these Python functions that are eventually to become Lambda functions. You can run them on local host, maybe you're running these functions in a pie test suite, a unit test suite that's gonna call these various functions. And then it's able to be run within your VPC, so you have access to the services that you need within your AWS infrastructure. And then after you've run your tests against your functions, your content with these functions, and now you want to deploy these to lambda. Terraform will have these Lambda module chunks within your Terraform file where you point that block to that file and it will handle automatically when you run Terraform apply. It will handle packaging that up as a zip file, putting it on S3 if it has to deploying a Lambda function and then. If you have specified tying that up to API gateway and setting up the necessary permissions. So that's cool. You're gonna bypass a whole lot of the steps that you would've had to take, like packaging it up as a zip file, putting it on S3, and then deploying to Lambda from S3. That's a three step process that you're able to skip just by using Terraform. But then the next step is, let's say that you modify one of those functions only and you leave the rest alone. You run some unit tests. Cool, we're back in business. And Terraform detects that only that file has been changed based on some hashing algorithm of like the modification date of that file on disc. And so it only deploys the necessary changes to a single Lambda function, not to the other Lambda functions. And if necessary, it may propagate some additional changes, let's say, to API gateway where it's tied to that specific Lambda function. So everything I'm saying about developing against AWS on local host as opposed to a Docker Yml file is only really feasible if you're using infrastructure as code like Terraform and is made especially feasible at least a lot quicker and saves some money if you're using the local stack framework. But I, I haven't tested local stack super thoroughly. So as far as the amount of coverage of capabilities of AWS that they handle, your mileage may vary, but it's worth trying out first. Of course we're in the machine learning applied podcast, so within your VPC, connected through the client VPN as you are, you have access to SageMaker, you can kick off SageMaker training jobs. So previous episode we talked about using SageMaker Studios I Python notebooks to write your machine learning code so that you can kick off machine learning jobs within the SageMaker environment. Well, you can opt to create a SageMaker studio project. You can set that all up, either in the web browser, in AWS's console using SageMaker Studio. Or you can orchestrate the infrastructure setup of SageMaker through Terraform instead, and then you can connect to your SageMaker environment within your client VPN, so that you can kick off training jobs and all this stuff on local hosts rather than being on SageMaker studio. Now, why would you want to do that? Well. Let's say you're doing the rest of your web development and service development per the way I've described in this episode. Well, maybe you don't wanna mix it up too much. So we want to keep our machine learning development on the same kind of environment. Client VPN, connected to our VPC so that we could kick off SageMaker jobs, but two, so that you could use pie charm. I don't like Ipy notebooks personally, I don't like Jupiter nearly as much as I like JetBrains pie charm. It's such a powerful IDE. So if I'm able to continue to use my IDE for developing my machine learning code on local host, but that I'm able to connect to the SageMaker environment so I can still run SageMaker training jobs and inference jobs and experiments with hyper parameter optimization and all that stuff on local hosts, then that's a win-win. And of course you still get all the bells and whistles of SageMaker. You can still then go into SageMaker studio on the web and look at the feature importance is that get spit out from Shap in autopilot or in your experiments or in your training job, the model monitor, clarify all these things. So you still get all the output based on the features that are provided by SageMaker, but you can simply do your development on local host. So that's how you can develop against AWS on local hosts. Measure twice, cut once. Start building your project on AWS as if you are running these as like test databases or test servers. Connect to it with a client, VPN, so that you're within the environment to run your test code. And that way when it comes time for deploying your stack, you're not gonna get sideswiped by all the setup you'd normally have to do. Do this all in Terraform infrastructure as code, or CDK, or Serverless framework, because managing your infrastructure in the web interface of AWS or Azure or GCP is not gonna happen. Simply not gonna happen for complex infrastructures and give local stack a try. So you might be able to do all this stuff on local host. Now, like I said, this is all a bit mutually exclusive to an alternative type of infrastructure orchestration systems, one of which is Kubernetes. I'm actually gonna be talking to somebody at depth here soon to give me a rundown on how Kubernetes and Kube flow, especially, for example, for machine learning pipeline orchestration, compares to cloud-hosted managed services the way I'm describing in this episode. They're entirely different ways of handling infrastructure Kubernetes. What Kubernetes does is you give it a bunch of Docker files, so all of your services are written as docker files, okay? Rather than using or relying on the managed services that are offered on AWS or Azure or GCP, you are relying instead on open source projects contained in docker containers. So for example, in AWS, if you want a message queue, you'd use SQS. Well, the open source equivalent of SQS, there's something called zero MQ, and there's rabbit mq. So you could use a rabbit MQ docker container, and then for the server you'd use either AWS Lambda. Or Fargate, a hosted server, but I might use AWS Lambda for running a server on AWS. Well, you would actually host your server maybe as a dockerized fast, API container. Kubernetes takes all these containers, strings them all together, handles all the networking between them. Puts them all into its own version of A VPC and then deploys all of these services to EC2 instances if you're gonna be running this on AWS or the equivalent of instances on GCP or Microsoft Azure. So Kubernetes is intended for really relying on docker and open source projects to string all of your services together. You're not gonna be using AWS as your backbone, really. You're not gonna be tapping into the various services on AWS. You are gonna be hosting it on AWS with Kubernetes or GCP, or Azure, wherever you want to host it. But the services themselves and the way they communicate with each other and the way they scale up and down, that's all managed by Kubernetes. Not by AWS, it's all open source stuff that handles itself. And then you deploy the Kubernetes stack to AWS. And Kubernetes has what's called this master server on the control plane that does all the orchestration of your services with respect to each other. And it will do as smartly and as cheap as it can. So for example, if you have multiple services and they all expect some number of RAM and CPU and three of them are running on one EC2 container that actually has extra ram and. CPU available, then it will spin up a docker container on that EC2 instance rather than spinning up a new EC2 instance. So anyway, all that's to say that Kubernetes is an entirely different, mutually exclusive fashion of deploying your architecture to the cloud. One, which relies heavily on Docker files and open source projects. And what it does is it has this master server that keeps dibs on who's who within the stack. In order to decide whether we're going to scale up or scale down certain containers within the stack. As opposed to Terraform or CDK or Serverless framework, which is infrastructures code that is actually managing services that are offered by AWS. These are not necessarily open source offerings. They're not necessarily Docker containers, though they can be. Terraform manages your. Stack on AWS's cloud offerings by AWS. So they're mutually exclusive. Kubernetes, actually, it comes with a cost. There's a fee, a Kubernetes fee to run your control plane in the cloud. So that's one downside of Kubernetes when upside is that it's sort of this universal solution. So you can move your Kubernetes stack off of AWS and onto GCP and A-W-S-G-C-P and Azure. They all support Kubernetes. They all have a service that supports Kubernetes. It's, it's not just that, it's supported out of the box. AWS is service that supports Kubernetes is called EKS, and by running your Kubernetes cluster in the EKS service, that's it is in EKS, that you actually incur that fee to run your clusters control plane. And the reason I'm bringing up Kubernetes here is that it is another alternative to infrastructure as Code. Terraform is a project for infrastructure as code, and so is Kubernetes, but that they handle deploying your stack to the cloud in totally different ways. And another reason I'm bringing it up is because, as I mentioned, I'm going to be interviewing somebody from depth on how they use Kubernetes to deploy machine learning pipeline to the cloud by way of cube flow. And so rather than using SageMaker, this person at depth is actually using Kubernetes to orchestrate the entire data stack. Everything I discussed in the last two episodes of ingesting your Data and transforming the Data Imputing Knolls feature, store everything at scales, microservices, data pipelines, training, monitoring, debugging, and then deploying this person rather than using Sage Makers. Hosted offerings on AWS is using cube flow's, open source containerized offerings in order to string together the entire end-to-end data pipeline in a universally deployable and open source fashion, which has its perks, no doubt. And so I will be reporting back here soon on how things like cube flow on Kubernetes compared to managed services like SageMaker and AWS, all orchestrated through Terraform. So hope to get you that episode in the near future, and I'll see you then.