The Federaliser

Federaliser is a powerful, extendable data integration tool designed to collect, normalise, and export data from multiple sources into a structured format. It enables seamless compatibility with Prometheus, OpenMetrics, and Telegraf, making it ideal for monitoring and analytics use cases.

What Does Federaliser Do?

Federaliser bridges the gap between disparate data sources and modern monitoring platforms. It can:

• Collect data from databases (MySQL, MSSQL, PostgreSQL/Redshift), APIs (JSON, XML, CSV), system commands, and file-based sources.
• Convert and normalise data into a structured format.
• Export data in either JSON (for Telegraf) or Prometheus/OpenMetrics format.
• Act as an intermediary layer, making any data source accessible for metrics dashboards.

How Federaliser Works

1. Data Collection

Federaliser retrieves data from different sources via:

• Database queries (SQL-based systems)
• Web APIs (RESTful JSON/XML endpoints)
• System commands (executing scripts or CLI tools)
• File-based inputs (reading logs, CSV, or structured files)

2. Data Normalisation

• Extracts relevant data fields based on user configuration.
• Converts structured data into a unified schema.
• Applies filtering and transformation rules as needed.

3. Data Export

Federaliser supports two primary output formats:

• JSON Format (for Telegraf) – Suitable for time-series ingestion.
• Prometheus/OpenMetrics Format – Directly ingested by Prometheus.

Using Federaliser with Grafana

Integration via Prometheus

• Federaliser transforms data into Prometheus-compatible metrics.
• Prometheus scrapes the endpoint exposed by Federaliser:

  - job_name: "federaliser"
    static_configs:
      - targets: ["your-federaliser-host"]
  

• Grafana queries Prometheus to visualise the data in dashboards.

Integration via Telegraf

• Federaliser provides Telegraf-compatible JSON output.
• Telegraf ingests data using the inputs.http plugin:

  [[inputs.http]]
    urls = ["http://your-federaliser-host/your-endpoint"]
    method = "GET"
    response_timeout = "5s"
  

• Data is then forwarded to InfluxDB, Prometheus, or other time-series databases.
• Grafana visualises the data from Telegraf’s storage backend.

Why Use Federaliser?

• Flexible Data Source Support – Works with databases, APIs, files, and scripts.
• Real-time Metrics Collection – Converts diverse data into time-series metrics.
• Seamless Integration – Easily integrates with Prometheus, Telegraf, and Grafana.
• Scalable & Extensible – Custom handlers and exporters can be added for new formats.

Federaliser simplifies data ingestion, normalisation, and export, making it an essential tool for monitoring and analytics workflows.

More information, including install and setup documentation can be found right here.