Self-hosted GitLab for a small team: from manual deploys to a pipeline

TL;DR - Deploys used to be a manual process that could take days to get scheduled and done. I built the CI/CD pipeline on our self-hosted GitLab with Docker, and now a merge to the main branch is a release. Running GitLab yourself is fine for a small team if you treat it like production: backups with the secrets file, runners away from the server, and upgrades on a calendar.

When I joined, getting a change to production meant someone building it by hand, copying it to a server and hoping nothing was different from last time. A release could take days between “it’s ready” and “it’s live”. Small fixes waited for big ones.

The company already ran GitLab on its own infrastructure, with a small team using it. Moving to a SaaS wasn’t a priority, so I built on what we had.

The pipeline

Every service gets the same pipeline: build, test, build a Docker image, push it to the registry, deploy to ECS. The image built once is the image that goes to staging and then to production. Nothing is rebuilt between environments.

flowchart LR
  MR[Merge request] --> Build[Build and test]
  Build --> Image[Docker image]
  Image --> Registry[(Registry)]
  Registry --> Staging[Deploy staging]
  Staging --> Prod[Deploy production]

Versioning and the changelog are part of the release, not something someone remembers to do later. Every release has a version and a list of what changed, which support reads before talking to a customer.

What running GitLab yourself requires

• Runners separate from the GitLab server. A heavy build should never slow down the web UI or a git push.
• Disk. The registry and job artifacts grow much faster than the repos. Set expire_in on artifacts and a cleanup policy on the registry, or the disk becomes a weekly alert.
• Backups with the secrets file. The GitLab backup doesn’t include gitlab.rb and gitlab-secrets.json. Without the secrets file, a restore loses CI variables and two-factor settings. Back them up separately and test a restore on a throwaway instance.
• Upgrades on a calendar. GitLab has required upgrade stops, and you can’t skip them. Small monthly upgrades are much less scary than one big jump a year.

What changed

Releases stopped being events. A fix goes out the same day it’s merged, and rolling back is deploying the previous image. That did more for the team’s confidence than any process document.