9 docs tagged with "foss"

While Concourse has an elegant architecture, a clean 'everything as code' approach and features iconic pipeline visualisations, it has failed to gain significant traction and remains a niche tool. The continued small user community, poor documentation and esoteric choice of Garden as the container run time instead of Docker presents operational challenges. For new deliverables we recommend other tools over Concourse, although there is not an urgent need to migrate existing projects away from it.

The ELK stack (and its EFK variant) provides a mature self-hosted solution for log collection, aggregation and inspection. ELK/EFK is a good choice for on-premise systems, but for cloud-hosted systems we usually find that native log aggregation solutions such as AWS CloudWatch Logs or Azure Monitor logs are a better choice. These give adequate functionality for most use cases without the operational overhead of running a self-hosted log aggregation system, and with pricing which is attractive in most cases compared to self-hosted options.

GitLab provide a suite of products which work well together but can be used separately; the two primary components are a git wrapper, similar to GitHub, and the well-regarded GitLab CI CI/CD system, which is a good default choice if running within your own infrastructure.

Grafana has become the de facto standard Open Source graphing and visualisation tool for systems monitoring and is used by many of our teams. It is a highly versatile tool, and while it is commonly used with Prometheus, it can also be used to visualise metrics from cloud platform monitoring systems such as Azure Monitor and AWS CloudWatch. Grafana also has a rich plugin system for panel components to help you represent your metrics the way you want.

Jekyll is a static site generator written in Ruby that builds and creates static sites from a source file set.

Jenkins is one of the most widely used CI/CD systems we come across. Despite its long history and large user base, the common consensus is that there are now far better alternatives than Jenkins. Jenkins is a workable solution for clients already invested in it, but we do not consider it a default choice for greenfield work. While plugins can add features such as containerised builds and pipelines definitions as code, these have to be configured carefully to work reliably and it is hard to avoid a cobbled-together feel. Newer tools which have been built with features such as these in mind tend to offer a cleaner and more reliable solution with less effort.

Apache Kafka is an open source distributed streaming platform, famous for its ability to handle very high message rates with low latency. While Kafka is wildly popular, we recommend caution due to operational complexity, and consider other tools to be better default choices.

Kubernetes is an open source container orchestration platform, highly regarded for its flexibility and scalability. While Kubernetes is often considered the de facto standard for container orchestration, we recommend careful consideration before adopting due to its operational complexity. In many cases, native cloud platform orchestration options such as ECS may prove to be a better fit, especially for relatively small or simple deployments.

Over recent years, Prometheus has become established across our teams as a firm favourite for capturing metrics, usually used with Grafana for visualisation. It is our default choice for monitoring on premise systems and to supplement cloud platform metrics systems such Azure Monitor and Application Insights or AWS CloudWatch Metrics. It has a highly versatile model, which has led to a rich ecosystem of exporters_ which provide a way for Prometheus to collect metrics from almost any source, including infrastructure, databases and message queues, and application/business metrics. Prometheus also supports alerting via the alert manager component using a built-in query syntax and rules engine.