Monitoring & Alerting

Fugoku provides built-in monitoring with real-time metrics, customizable dashboards, and flexible alerting — all included at no extra cost.

Access monitoring at: console.fugoku.com/monitoring

Metrics Collection

Automatic Metrics

Every resource automatically emits metrics:

Resource	Metrics	Frequency	Retention
Servers/VMs	CPU, RAM, Disk I/O, Network, Disk usage	10s	30 days
Databases	CPU, RAM, Connections, Query latency, Cache hit ratio, Replication lag	10s	30 days
Volumes	Read/write IOPS, Throughput, Latency, Capacity	10s	30 days
Networks	Bandwidth, Packets, Errors, Drops	10s	30 days
Databases	Connections, Query latency, Cache hit ratio, Replication lag	10s	30 days
Load Balancers	Requests, Latency, Errors, Active connections	10s	30 days

Custom Metrics

Push application metrics via StatsD/DogStatsD protocol:

# Via StatsD (UDP port 8125)
echo "myapp.requests:1|c" | nc -u -w0 localhost 8125
echo "myapp.latency:150|ms" | nc -u -w0 localhost 8125
echo "myapp.errors:1|c" | nc -u -w0 localhost 8125

DogStatsD tags supported:

echo "api.requests:1|c|#service:api,env:prod,endpoint:/users" | nc -u -w0 localhost 8125

Dashboards

Built-in Dashboards

Dashboard	Resources	Key Metrics
Infrastructure Overview	All resources	CPU, RAM, Disk, Network, Cost
Server/VM Detail	Single server/VM	CPU, RAM, Disk I/O, Network, Processes
Database Detail	Single database	Connections, Query latency, Cache hit, Replication lag
Database Cluster	Database + replicas	Replication lag, Failover events
Network	VPC, VLANs, IPs	Bandwidth, Packets, Errors, DDoS
Kubernetes	Cluster + nodes	Pods, CPU, Memory, Pod restarts
GPU Clusters	GPU nodes	GPU utilization, Memory, Temperature
Billing	Project/Organization	Spend by service, Daily trend, Forecast

Custom Dashboards

Create dashboards tailored to your team:

Console: Monitoring → Dashboards → Create Dashboard

Features:

Drag-and-drop widgets
Time range: 1h to 90d, relative or absolute
Variables: Template variables for server, region, project
Annotations: Deploy markers, incident markers
Sharing: Team, organization, or public link
Export: PDF, PNG, JSON

Widget	Use Case
Time Series	Line/area charts for metrics over time
Stat	Single value with sparkline (current CPU, cost)
Table	Top N resources by metric
Heatmap	Latency distribution, CPU by host
Gauge	Single value with thresholds (disk usage %)
Bar Gauge	Horizontal bar with thresholds
Logs	Live log tail with filters
Text/Markdown	Documentation, runbooks, links

Alerting

Alert Rules

Create alerts on any metric with flexible conditions.

Alert Rule Components

Component	Description
Name	Descriptive name (e.g., "High CPU on production web servers")
Metric	Metric to evaluate (CPU, memory, disk, custom)
Condition	`>`, `>=`, `<`, `<=`, `==`, `!=`
Threshold	Value to trigger (e.g., `80` for 80%)
Duration	How long condition must persist (1m, 5m, 15m, 1h)
Aggregation	`avg`, `max`, `min`, `sum`, `count`, `p50`, `p95`, `p99`
Group by	Group by label (server, region, project)
Severity	`critical`, `warning`, `info`

Create Alert

Console: Monitoring → Alerts → Create Alert

CLI:

# CPU alert
fugoku alerts create \
  --name "High CPU on production web servers" \
  --metric cpu \
  --condition ">" \
  --threshold 80 \
  --duration 5m \
  --aggregation avg \
  --group-by server \
  --severity critical \
  --notify email:ops@company.com,slack:#alerts

# Disk space
fugoku alerts create \
  --name "Disk space critical" \
  --metric disk_usage \
  --condition ">" \
  --threshold 90 \
  --duration 5m \
  --severity critical \
  --notify email:ops@company.com

# Database replication lag
fugoku alerts create \
  --name "Database replication lag" \
  --metric replication_lag \
  --condition ">" \
  --threshold 30 \
  --duration 2m \
  --group-by database \
  --severity warning \
  --notify email:dba@company.com

Alert States

State	Description
OK	Condition not met
Pending	Condition met, waiting for duration
Firing	Condition met for full duration
Resolved	Condition no longer met after firing

Alert Routing

Channel	Configuration
Email	One or more addresses, custom subject/template
Slack	Webhook URL, channel, username, icon
PagerDuty	Integration key, severity mapping
Opsgenie	API key, team, tags
Webhook	Custom HTTP endpoint, HMAC signature
SMS	Phone numbers (via Twilio)
Voice Call	Phone numbers (via Twilio)

Notification Templates

Customize alert notifications:

{{#if firing}}
🔴 **ALERT: {{alert.name}}**
{{else}}
✅ **RESOLVED: {{alert.name}}**
{{/if}}

**Metric:** {{metric}}
**Value:** {{value}} (threshold: {{threshold}})
**Resource:** {{labels.resource}}
**Severity:** {{severity}}
**Started:** {{starts_at}}
{{#if firing}}
**Runbook:** {{alert.runbook_url}}
{{/if}}

Alert Grouping & Deduplication

Group by: Group alerts by label (server, region, project)
Repeat interval: Re-notify every N minutes while firing (default: 60m)
Auto-resolve: Auto-resolve after condition clears (configurable)

Log Management

Structured Logging

All platform logs available in real-time:

Log Source	Retention	Access
Server/VM console	30 days	Console, CLI, API
API audit logs	7 years (WORM)	Console, API, Stream
Firewall logs	90 days	Console, API, Stream
Database logs	30 days	Console, CLI
Load balancer logs	30 days	Console, API
Kubernetes audit	90 days	Console, API

Log Streaming

Stream logs to your SIEM in real-time:

Console: Monitoring → Logs → Stream

CLI:

# Stream to Splunk
fugoku logs stream create \
  --name splunk \
  --destination splunk \
  --hec-token $SPLUNK_HEC_TOKEN \
  --filter 'resource_type=server'

# Stream to Datadog
fugoku logs stream create \
  --name datadog \
  --destination datadog \
  --api-key $DD_API_KEY

Supported destinations: Splunk, Datadog, Elastic, Sumo Logic, Sentinel, Loki, Custom HTTP

Log Queries

# Search logs
fugoku logs query \
  --start 2024-01-15T00:00:00Z \
  --end 2024-01-15T23:59:59Z \
  --filter 'resource_type=server AND message:error' \
  --limit 100

Distributed Tracing (OpenTelemetry)

Enable distributed tracing for microservices:

Instrumentation

# Python (OpenTelemetry)
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(
    OTLPSpanExporter(endpoint="https://tracing.fugoku.com:4317")
))
trace.set_tracer_provider(provider)

Trace Context Propagation

# Headers automatically injected
traceparent: 00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01
tracestate: fugoku=project:prod,server:web-1

View Traces

Console: Monitoring → Traces

Service map
Trace timeline
Span details (latency, errors, attributes)
Service graph with latency heatmap

Synthetic Monitoring

Proactive endpoint monitoring from global locations.

HTTP Checks

fugoku synthetic create \
  --name "API Health Check" \
  --url https://api.myapp.com/health \
  --method GET \
  --expected-status 200 \
  --expected-body '{"status":"ok"}' \
  --interval 60s \
  --locations ashburn-1,chicago-1,london-1 \
  --alert-on-failure 3

Browser Checks (Playwright)

fugoku synthetic create \
  --name "Login Flow" \
  --type browser \
  --script @playwright-script.js \
  --interval 300s \
  --locations ashburn-1,london-1 \
  --alert-on-failure 1

Cost Monitoring

Real-time Cost Tracking

Current month spend — By service, project, resource
Daily breakdown — Day-over-day trends
Forecast — ML-based monthly forecast
Anomaly detection — Unusual spend spikes

Budget Alerts

fugoku billing alert create \
  --name "Monthly budget 80%" \
  --project production \
  --threshold-percent 80 \
  --notify email:finance@company.com

Cost Allocation

Dimension	Example
Project	`production`, `staging`, `dev`
Team	`backend`, `frontend`, `data`
Environment	`prod`, `staging`, `dev`
Service	`compute`, `storage`, `network`, `database`
Resource	Individual server, volume, IP

API Reference

The monitoring API is available under the admin endpoints. These require admin-level permissions.

Metrics

# Query metrics
GET /v1/admin/monitoring/metrics?metric=cpu&resource_type=server&resource_id=srv-abc&start=1h&aggregation=avg

Alerts

# List alerts
GET /v1/admin/monitoring/alerts?status=firing

Services

# List monitored services
GET /v1/admin/monitoring/services

Note: Full alert CRUD (create, update, delete), dashboards, log streaming, and tracing endpoints are available via the console and CLI. The API currently exposes read-only monitoring endpoints under /v1/admin/monitoring/.

Best Practices

Alert Design

Alert on symptoms, not causes — "High error rate" not "Database CPU high"
Actionable alerts — Every alert should have a clear runbook
Severity alignment — Critical = page, Warning = ticket, Info = log
Avoid alert fatigue — Tune thresholds, use grouping
Test alerts — Trigger test alerts monthly

Dashboard Design

One purpose per dashboard — Infra overview, DB deep-dive, etc.
Consistent time ranges — Link dashboards for correlated investigation
Use variables — Template variables for server, region, project
Set refresh intervals — 10s for real-time, 1m for historical

Log Management

Structured logging — JSON with consistent fields
Sampling — Sample high-volume debug logs
Correlation IDs — Trace requests across services
Retention policies — Hot (30d), Warm (1yr), Cold (7yr)

Cost Optimization

Right-size metrics — Don't collect 1s resolution for everything
Downsample old data — 10s → 1m → 1h automatically
Archive cold data — Move to object storage after 30 days
Monitor monitoring costs — Alert on metric ingestion volume

Getting Help

Documentation: docs.fugoku.com/monitoring
Support: support@fugoku.com
Status: status.fugoku.com
Community: discord.gg/fugoku

Next Steps:

Monitoring & Alerting

On this page