Maciej & Code

GitLab CI/CD Series Appendix II: Git submodules, better triggers & conclusions

Maciej Zwierzchlewski -

Even though several posts about GitLab CI/CD posts have appeared here, there are still some tips I have left in me. I think that these will be the final two, so there's also a "review" of GitLab CI/CD at the end of this post. Let's get to it, then!

Git submodules

If your GitLab repository contains Git submodules that are private, the CI/CD will have a problem when trying to clone them. The job will fail with a Host key verification failed error.

Now as scary as it sounds, there is a very easy fix for this issue. The following commands:

git config --global url."https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com/".insteadOf "git@gitlab.com:"
git submodule sync && git submodule update --init

will make git pull the submodule but using HTTP instead of SSL.

The CI_JOB_TOKEN is a built-in GitLab CI/CD variable that provides access based on the permissions of the user that causes the job to run. So if you have access to the submodule, the job will too!

Simply add the commands to the before_script or script sections of your job, and that's it - problem solved.

There are other solutions listed at Getting GitLab CI to clone private repositories, but this is by far the cleanest one. If you fear the token being leaked somehow, it is possible to restrict its permissions. Read more about that at Limit GitLab CI/CD job token access.

Better triggers

Can one trigger be better than another one?

I'm not sure, but it sure can be better suited to your workflow.

For example, all the job definitions in the GitLab CI/CD Series posts had the when: manual property. That means a user had to manually trigger every job for it to run. The workflow constraints made it unnecessarily difficult to do a deployment from another branch than the default one.

But all of that can be customised. Your CI/CD setup should fit you, not the other way around.

So let's alter the CI/CD configuration created in the GitLab CI/CD Series to respect the following conditions:

  • a pipeline should:
    • be created for:
      • a new or updated merge request
      • new commits on the default branch
      • when the Run pipeline button in the GitLab UI is clicked
  • the build jobs should:
    • run automatically when part of:
      • merge request pipeline
      • default branch pipeline
    • wait for a manual trigger when the Run pipeline button in GitLab UI is clicked
  • the deploy jobs should:
    • always wait for a manual trigger
    • be skipped (not created) in merge request pipelines

That might seem like a long list of conditions, but it's all simple. When it comes to merge requests, we want to validate that the code actually builds and prevent accidental deployments from them. The code on the main branch should always be built and ready to be deployed (deploy jobs created with manual triggers). It should also be possible to build and deploy code from any branch by using the GitLab UI. Of course, the rules that were there previously (regarding changes in specific directories) should still apply, but only for the merge request pipelines.

All that text for just a little bit of YAML:

...

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
    - if: $CI_PIPELINE_SOURCE == 'web'

build-frontend:
  ...
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
      changes:
        - 'client/**/*'
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
    - if: $CI_PIPELINE_SOURCE == 'web'
      when: manual
  ...

build-backend:
  ...
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
      changes:
        - 'server/**/*'
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
    - if: $CI_PIPELINE_SOURCE == 'web'
      when: manual
  ...

.deploy-rules:
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: manual
    - if: $CI_PIPELINE_SOURCE == 'web'
      when: manual

.deploy-backend:
  ...
  rules:
    - !reference [.deploy-rules, rules]
  ...

.deploy-frontend:
  ...
  rules:
    - !reference [.deploy-rules, rules]
  ...

The rules engine is really powerful, and this is just the tip of the iceberg when it comes to possibilities. It takes some time to get familiar with all the concepts, but once you do it, the sky is the limit. You can get more familiar with the available options by reading the Choose when to run jobs article from GitLab's documentation.

GitLab CI/CD conclusions

As this is probably the last post related to GitLab CI/CD that will appear on this blog, I wanted to share my opinion of the service. It might help someone make the right decision.

Starting with the good stuff - I love that the whole configuration is a git-tracked YAML file. You can look up how and when it changed, which is always a nice thing. It can also be neatly split into several files to keep things tidy, and aside from all the stuff you can do with YAML, GitLab added several additional features to make sure the setup is easy to maintain.

A great advantage over some other CI/CD systems is the integration between the CI/CD and the repository itself. This means that you get some great features, like merging after the pipeline succeeds and advanced trigger rules out of the box.

What's also great is the different job execution modes. There's the old-school mode of running the commands on the runner host system directly. Nowadays, it's cool to run your builds inside a docker container (not without reason!), and I love that the docker executor is a first-class citizen in GitLab CI/CD. It makes the setup so easy and consistent. Apart from the two mentioned modes, GitLab supports a number of others, so there should be something for everyone.

Moving over to the things I liked a bit less. The thing that was a big surprise is just how terrible the shared runners are. I would definitely not consider using GitLab CI/CD if you're not planning on setting up your own GitLab Runner (😉). They are horribly slow.

The other thing that rubbed me the wrong way is the lackluster variable support. Yes, they're there, but hidden deep in the project settings. As a point of reference, I've always loved Octopus Deploy's approach to variables - they can be grouped and looked up quickly. What I liked the most was that each release has a variable snapshot attached to it so that you can always see what variable values were used for a particular release. Here, in GitLab CI/CD, they feel a lot like an afterthought and burying them in the project settings is not helping that feeling.

Side note: I have to give praise where it's deserved, though - the scope wildcards are great!

The other thing that made it difficult to make good use of all the features GitLab has was the seemingly small community around it. The official documentation is very detailed but in my opinion, lacks real-world examples, and there's often little information on how the different features work with each other. That's often when the community comes in and fills the gaps with articles or videos. Unfortunately, it's not the case here - there are a couple of useful articles and some great StackOverflow answers, but most often you are on your own.

Now, before the next two points, I want to point out that I know that GitLab is a company that can do whatever they want and however they want.

That said, my next complaint is GitLab's attitude towards user-reported problems and suggestions. I won't list the responses here directly, but examples are not difficult to find. What's great is that you can get responses directly from the developers who made the product. But as developers, they are often very protective of their child. And on one hand, not being swayed by nitpickers is something to be admired. But on the other hand - some real issues are not addressed at all, and users are forced to make use of workarounds to achieve something that's standard in most other CI/CD platforms.

The last thing is not a complaint or downside but an important fact that you might consider when deciding on CI/CD platform. As GitLab is a business, they have to make decisions that make them money. And you can actually build a really great CI/CD workflow using the Free plan. The biggest downside that might be a deal-breaker to some is that there is no way to restrict deployments to production environments without upgrading to the Premium tier (that's $19/user/month as of writing this post). While for most companies, the cost is negligible, if you're just starting out you might not be able to afford it, so keep that in mind.

Even though there's much more text talking about the negative aspects of GitLab's CI/CD platform, I've been really enjoying it once I managed to set it up, so it fits my workflow. There are some small things that could be improved, but overall it's a pretty great product (if you host your own runners 😉).

Cover photo by Alessandro Sacchi on Unsplash