Given the necessity to create infrastructure throughout a number of environments whereas guaranteeing standardization and efficient monitoring, it turns into essential to provision these environments securely. To realize this, adopting an immutable infrastructure strategy, the place environments are provisioned as code, is important.
The aim of this text is to reveal a doable strategy to attaining this by utilizing GitLab’s buildings to implement templates and requirements, Terraform to use and keep requirements throughout servers, and Ansible for software program provisioning and configuration, using a shared roles mannequin throughout repositories. To handle the state of machines with Terraform, we use MinIO, because it allows this implementation on-premises.
Structure Design
Step 1
The method all the time begins with submitting a standardized concern, specifying the stack mannequin for use, whether or not firewall permissions are wanted, and whether or not it’s a brand new setup or only a useful resource improve.
Step 2
The operator evaluations the problem and begins the method. All conversations and time spent are logged throughout the concern.
Step 3
A brand new venture is initiated in GitLab, based mostly on the infrastructure mannequin that will probably be created. This venture is positioned throughout the corresponding group in GitLab, the place it inherits the mandatory atmosphere variables for standardized infrastructure creation.
Step 4
When the venture is created, you solely must specify the IPs for the infrastructure to be provisioned within the atmosphere specified within the concern (KVM, VMware). After planning with Terraform, the required assets are created, together with including labels if wanted, for Veeam to carry out backups based mostly on label insurance policies. Upon completion, the state of the created infrastructure is saved in a bucket.
Step 5
The subsequent step includes executing customary duties for all servers, akin to figuring out them, updating packages, putting in essential utilities, and registering the host in Zabbix for primary monitoring of the working system and the stack. Relying on the useful resource group, the suitable entry keys are assigned to the accountable groups. For instance, DBAs obtain entry keys for database servers.
Step 6
Based mostly on the chosen mannequin, the method of putting in and configuring your entire stack is carried out. Equally, customers are created, and credentials are registered in Vault when essential.
Step 7
With the appliance now operating within the new atmosphere, particular monitoring for every stack may be carried out, registering the brand new server in Consul. Prometheus, in flip, identifies the place it wants to gather info from. Every stack has its monitoring dashboard already configured, various solely by the identify of the venture that was created.
Step 8
The brand new infrastructure is delivered to the requester. Within the case of databases, credentials are supplied straight in Vault.
Mission Construction
The folder construction in GitLab is organized as follows:
- /infrastructure/: The primary group, the place world atmosphere variables and default values must be saved
- /infrastructure/gitlab-models: Pipeline fashions, the place we have now two primary tasks
- ansible-pipelines: A venture devoted to sustaining the stacks and the composition of roles.
Within the picture above, we see an instance of frequent duties. Within the construction, it’s positioned on the path:/infrastructure/gitlab-models/ansible-pipelines/common-task/provision.yml
- terraform-pipelines: Pipelines for the obtainable infrastructure fashions, akin to vSphere, KVM, AWS, and many others.
Within the picture above, we have now an instance of a pipeline that resides throughout the terraform-pipelines group, akin to kvm-terraform-pipeline.yml. As we will see, it’s a GitLab CI mannequin supposed to be prolonged in a stack pipeline.
- /infrastructure/templates: On this group, we have now the bootstrap tasks, which will probably be used to create the stack fashions.
- /infrastructure/provision/ansible/roles: On this venture, we have now the Ansible roles solely, permitting us to centralize and replace the roles in an remoted method.
- /infrastructure/dependencies-iac: This repository incorporates the platform’s dependencies, akin to Dockerfiles for Terraform and Ansible, guaranteeing that the variations of the mandatory instruments and libraries usually are not altered.
- /infrastructure/modules/: The modules created for Terraform are saved on this repository, with every venture having its respective folder.
- /infrastructure/on-premise/: This group is the place the created infrastructures will probably be maintained, and segmented by atmosphere, information middle, stack, and venture. Within the picture, we will see the hierarchy of teams and subgroups all the way down to the ultimate venture. At every of those ranges, we will override the variable values related to the teams.
How To Use a Platform
To simplify the usage of the platform, we created a repository known as issues-ops, the place we offer a difficulty template that may be chosen based mostly on particular wants. This manner, the infrastructure request is recorded proper from the beginning.
As soon as the problem is created, the DevSecOps staff can start organising the atmosphere. To do that, they merely must navigate to the suitable group, on this case, infrastructure/on-premise/staging/dc1/loadbalancer/nginx, and create a brand new venture based mostly on a template. They need to then present the identify of the venture to be created and assign the mandatory variables.
Inside every template, the .gitlab-ci.yml file required for atmosphere creation is already configured. Within the case of NGINX, it’s arrange on this format.
On this setup, each the infrastructure creation templates and the Ansible templates are included, guaranteeing that the default roles are already built-in into these tasks. Moreover, we offer steps to increase the mannequin. If extra roles have to be put in, you possibly can merely add the corresponding block, enabling a modular, building-block strategy to configuration.
Within the picture beneath, we see the pipeline that ran the requested atmosphere creation. You’ll discover that authorized_keys and frequent had been executed, regardless that they weren’t explicitly declared within the .gitlab-ci.yml. It’s because we have now customary roles coming from the imported Ansible template, guaranteeing that the default roles are utilized throughout all tasks.
Conclusion
The infrastructure platform has vastly contributed to sustaining and implementing requirements as a result of it requires a predefined mannequin to be deliberate, examined, carried out, and made obtainable as a template earlier than any new infrastructure may be created. This course of ensures that every time we have to provision assets in an atmosphere, we’re establishing constant requirements, versioning these environments, and guaranteeing they are often reliably reconstructed if essential.
One of many primary challenges is maintaining the fashions up-to-date and validated, particularly as purposes evolve and working system variations change. It’s essential to keep in mind that when utilizing infrastructure as code, all modifications must be made via it, guaranteeing correct configuration versioning and atmosphere immutability. Failing to take action could trigger the platform to revert the atmosphere to its outlined state, doubtlessly overriding handbook modifications.
The mannequin proposed on this article is flexible, and relevant to each on-premises and multi-cloud environments, making it an efficient resolution for hybrid infrastructures.