Did you know that the average S A start up spends 100 and $16,000 a year on the cloud
This makes sense. Since for less than the price of one employee's salary, you can have a scalable application that is simple and cheap
Amazing. Every word of what you just said was wrong except something's wrong
It's actually a million 100 and $60,000 a year. And for that
you get a service that still recommends a certification so that you can learn how to use it properly
How do we live in a world where we have subject matter experts for setting up
Aws isn't the whole point of paying for the cloud that it's supposed to be painless
Using this information. I've come up with this elaborate graph explaining how Jeff Bezos became the world's richest man
As you can see, venture capitalists, give all their money to start ups and start ups
give all their money to Aws. Simple ECR ECs R DS A LB
Cloudwatch, LAMDA S3 and even github actions. In a lot of cases
you can achieve very similar results with nothing more than Linux, Git Docker and A few other open source tools in this video
we're going to deploy a website for the cost of a domain name. We're going to deploy a server
a database ytics all with http S zero downtime deployments and continuous delivery
We will be using simple and boring tools that you probably already have installed on your computer and also wanna discuss common opinions on self
hosting and why I think we've become too dependent on cloud providers without good reason
Obviously, I'm not saying that everyone should always self host everything. But I'm advocating for always using the simplest tool to solve the problem you
actually have. There is no one size fits all solution and anyone telling you so is probably a salesperson understanding these simple tools is more
valuable scalable and cheaper than locking yourself into a vendor. The first thing you need is a computer
My weapon of choice is a 2017 entry level Macbook Pro. I left it charging in a drawer for too long and the battery swelled
up. So I ripped it out, breaking the keyboard in the process and now have the equivalent of a desktop computer
But with a screen and terrible thermal performance, you can and should use an actual computer
But this is a good example of how easy it can be to get started. Assuming you're not a masochist
you want to install Linux. Most of the internet generally recommends ubuntu or Debian with people generally saying that ubuntu is friendlier while Debian more
stable, but everyone is wrong. I'm going to use N OS. It's declarative repeatable and has a metric ton of packages
And when I come back to an abandoned project, two years later, I can just look at one configuration file to see how the machine
was set up. I flashed Nix Os on Aboot USB using Belia er and followed the steps to install on my computer's main partition
I would recommend using the guy installer even for a headless system since it makes the initial configuration so much easier and you can still
select headless during the process. The first two things you want to do are connect to a local network and enable SSH like all things
in NI I can do both of these from a single configuration file. While I'm here
I'm also going to install Git docker T Max and Vim. I refuse to use Nano more than once per computer
I also want to ensure that the firewall is enabled and only allow connections on port 8443
Since this machine will be on the public internet, now, I can stop this computer in a drawer and use SSH to connect to it
from an actual computer with a working keyboard. Elastic container service is a fully managed container orchestration server
So that helps you easily deploy, manage and scale containerized applications. So in other words
ECs is just docker since I've already installed docker. We can replace all of ecs using just the docker run command in the cloud
We usually have to build our image and push it to a registry. But here we can just build and run the images locally to
do that though. We're gonna need to clone the project onto this computer. So let's create some SSH keys which I'll add to my github
account. I also create a services directory to store my repositories in and clone
My project, my project contains a Docker file for my. This is a two stage Docker file
The first stage uses a Gradel base image to build my Coopland project
This could be done outside of Docker, but I do it this way so that I don't need to manage Gradel versions on the host
machine. The second stage takes the output from the build stage and runs the executable two stage builds
allow me to have a smaller runtime image. Let's build this Docker image using Docker build
This takes a few minutes. I'm telling you this computer is begging for death and then we can run it using Docker run
I'm going to open up the port that the service runs on so that I can make sure the website is working using my local
network. This landing page doesn't really have anything interesting, but this service also contains the HTML to Scotland DS L tools that are used to
write service side html directly in Scotland, we now have the website working locally
but we want it to be globally available and preferably, I'd like it not to be flagged as malware
So we're gonna have to implement http S this is most commonly done using a reverse proxy
A reverse proxy is a separate service that maps requests to different services
For example, we could serve static files or web server depending on the URL path domain pointing
A reverse proxy to static files is a very simple replacement for file storage
Ideally, your reverse proxy would also handle middleware like caching and HGTPS certificates
The most common tool for this is Engen X. It's battle tested and everywhere and very good
But Engen X does much more than just reverse proxy and I don't need all those features
I also hate having to manually set up http S certificates. So I'm going to use caddy instead
Paddy is less common but it has dead simple syntax for reverse proxy and automatic http S
So the perfect fit for what I need. Here's my caddy file for this project. You can see I route everything from the base URL
to the main server in anything that comes through the ytics subdomain
I map to my ytics service, which we'll be setting up later since I will be running caddy on the same Docker network as my
other services. I can reference the container name directly instead of using manual IP addresses
This will be very helpful when implementing zero downtime deployments later I could build and run another docker file for caddy
But since I want everything to be on the same network, I'll be using dot Compose. Docker compose allows you to build and run multiple
Docker images using a single YAML file. This is a good opportunity to show all the containers that I'm running for this service
The first container is my main server. This is the container we manually built and ran before
If I specify the docker file and context, I can use Docker compost to both build and run my containers
I could even publish all of them using this exact same file. If I wanted to, my main server uses S QR Light to persist
data. I'll be using a post press container for some ytics. But S QR Light is a right choice for my server since my application
isn't multi nodal more on that later. This configuration lets me run my server using my local docker file
I don't need to expose any ports for this particular container since it will be accessed by caddy through the docker network
The next container is caddy. I'm using an existing image instead of my own Docker file and providing my caddy file using a volume in
docker compose I expose ports for http and http S since all the requests of a host machine will be going through this container
After this, I have a few containers to run plausible, plausible is an open source alternative to Google ytics that can be self hosted
To be honest, it's probably a little overkill for my use case, but it gives me a nice U I with secure login
So I'm happy with it for now. The point here is that you can pick and choose from any open source tools to run alongside
your main containers and give you additional features. This can range from ytics to databases to telemetry and so on
Plausible is just something that's suited my needs. Plausible includes a couple of databases including a progress one
I can add as many services as I want to this compose file and build using Docker compose build
This will create all the necessary images to run my containers. I can then use Docker compose up to run the whole stack
Blogs can be accessed directly through Docker. I use T mugs to scroll through them
And if I need to do some ytics, I can output them and use the normal cli tools I have like org grab and JQ
to execute queries. Now that my service is up and running, it is technically accessible but it won't be available on port 80 or 443
on my local network. This is because caddy expects a request to be coming from the domain I specify in the caddy file
I just need to set up DNS on the service I use to purchase my domain name and point it to my server and my
website is now accessible as previously mentioned. This application isn't multi nodal
This is because I have no requirement for this to be multi nodal. I don't mind if this has short outages
I only have one spare computer. This is not a limitation with self hosting though
And you can host multiple nodes if you have multiple computers. Docker has built in technologies to go from a single container to multi nodal
services, some of which we have already covered. If all you need is a single container
you can build and run it using the Docker cli this is the first example we showed if you want to run multiple containers on
a single computer, Docker compose, lets you configure that using a single YAML file
This is how I'm choosing to run my services. If you want to run multiple containers across multiple computers
you can use Dockers swarm to communicate between the computers and load balance between them
And if for some reason, you need more orchestration than that, you can reach for certis
Obviously, even if you are using Dockers across multiple computers, you will still need a single database
This could something like a single post re instance running on your master node which is backed up to the other machines or a dedicated
nass with road enabled. This is something I'd like to explore more in the future
Like I mentioned, Docker compose is well suited for my needs since I am running on a single machine
The only problem is that deployments are not zero downtime. I don't want it to go down every time I have a deployment
especially once we have continuous deployment set up, I used to have another dependency for this
But now I just do it directly with compose. You can use the scale command to scale up to two containers and then remove the
old one allowing for zero downtime deployments. This works well because as you remember
caddy is pointing directly to the service name so it will automatically route traffic to the new service
When the old one becomes unavailable, I just need to run caddy reload to make sure that it stays happy across multiple deployments
I've created a simple shell script to automate my deployment steps. It just pulls the latest version from GIT builds for dock images using compose
and then scales up to two tasks. I sleep for 30 seconds since it's easier than doing a health check
And then I scale the old container down. I have an extra command to ensure that only one server container is running once deployment is
finished and another command to reload caddy. After every deployment, this will update any changes I made to the caddy file
Also with zero downtime. This script is great for manual deployments. But what I want is a Cron job that ensures the latest version has
always been deployed. If I ran this script every minute, it would never stop deploying
So I have another script called deploy if changed. It's a simple script that fetches the latest upstream and only runs for deploy script
found any changes between my local and remote stashing if there are uncommitted
local changes. Now, I want to run this using a Cron job. I create a directory called automation in my home directory
And in here I create one last shell script. This script will do three things
It will cd into the correct directory using its absolute path. It outputs all the logs to a dedicated file instead of standard output and
it uses a log file to prevent simultaneous deployments from canceling each other
My potato takes a few minutes to run the build but I want my chrome job to run very frequently
Add a few issues where I go into a weird state when deployed multiple times in quick succession
using a lock file will only run the command when no one else is currently accessing the file
Since the changes are only pulled when there is no lock file being used, any new changes will wait for the current deployment to be
finished before redeploying. The next version. The final thing we need to do is create a chrome job that executes the script
First, I need to install Chrome, which I can do by editing my configuration file
You can add a new chrome job by running chrome tab dash E specifying the user
All I want is a that runs every minute and runs the automation deploy script
I don't use github actions or any other deployment tools for a simple repository with a single target
There is little reason to use anything more than a few Bash scripts. It's dead simple to maintain because it has very few dependencies and
it works really well for what I needed to do. It's also true. Continuous deployment head of main is in production
Always I know this because G and Bash are enforcing it. There are a few common arguments against doing something like this and while some
do hold weight, I want to discuss them and why I tend to disagree. I have two rules that lend me down this journey
avoid premature optimization and avoid unnecessary abstractions. Although the second rule is probably a subset of the first
but I choose to explicitly remind myself to avoid unnecessary abstractions because as software engineers
we are drawn to them. The first argument people tend to have is that the cloud is scalable
I know very few people that genuinely need to scale 10 or 100 times overnight
I certainly don't. And for a few reasons, the first one is that I don't want explosive growth
I don't think it's healthy. I want slow and steady growth with people that stick around
If this particular website did blow up, I prefer it to crash rather than to scale
I would only want it to scale if I'm sure that the traffic was not malicious and I'm sure that my conversion numbers could healthily
cover costs. The truth is that getting huge overnight where short downtime would be a deal breaker is not a very common situation and I'm
not ready to accept vendor lock in just to be ready for an event that is very unlikely to happen and has very little negative
impact even if it does. Another thing people say is that the cloud is easier to get started
I've often found that things that are easier in software tend to be easy because there are an abstraction around something very complicated
While things that tend to be hard are actually simple under the hood. I would much prefer an engine with no cover if it meant
that I could tweak it than an engine, I don't understand which probably wasn't even designed for my use case
I agree that in some cases, it's easier to get started with the cloud, but progress can grind to a halt when you need to
do something unexpected or unintended. At the end of the day, you are using a black box designed by someone else
If a black box has no input for what you want to change or if you are unfamiliar with the abstractions provided
then you're out of luck. I will always choose simple tools over easy ones. Simplicity is hard
but it pays dividends in the long run. The last thing people will say is that the cloud is worldwide and this is true if
you want to serve dynamic content, meaning content that cannot be cached by CD N with low latency to multiple regions or directly to a
regions, you don't have access to, then you need to use a cloud. But I've actually never seen anyone use multiple regions on the clouds
for their services. So I don't know how common this actually is. Obviously there are some cases where the needs of a product are different
and the cloud is the best solution. In that case, by all means use it. I just don't like the idea that because fortune 500
companies do things a certain way, we should all follow suit. Your product is probably not a Fortune 500
And the chances are it never will be. We've defaulted to doing things a certain way even when the advocated way is overkill for our
needs. Cloud providers have convinced many people to accept their marketing material as the absolute truth
Maybe we should take a second to rethink our actual requirements before playing in these walled gardens
I didn't get to go deep into all these topics some because they just weren't requirements I need right now if you are looking to
do this but need to take it to the next step. I will check out Linux KVM Docker Swarm
Nas and raid these technologies will let you spin up redundant highly available services
Thank you for taking the time to watch and I hope that you've found some of this helpful. I really want to thank everyone for
watching a void of the end. It makes me so unbelievably happy to have people interested in the random rabbit holes that go down
There's another video to watch here if you're interested. Ok, bye