// docs / api reference
API & Webhook Reference
The agent exposes a small REST API and a set of webhook endpoints that monitoring tools POST to. This reference lists every endpoint, the payloads each source sends, how they authenticate, and copy-paste curl examples to send a test alert.
REST endpoints
| Method | Endpoint | Description |
|---|---|---|
| POST | /webhooks/splunk | Receive Splunk alerts |
| POST | /webhooks/sensu | Receive Sensu alerts |
| POST | /webhooks/pagerduty | Receive PagerDuty events |
| GET | /alerts | List alerts |
| GET | /alerts/{id} | Get alert details |
| GET | /sops | List all SOPs |
| GET | /sops/{id} | Get SOP details |
| POST | /sops/reload | Reload SOPs from disk |
| POST | /sops/discover | Discover SOPs from all sources (background) |
| POST | /sops/discover/github | Discover from GitHub repos |
| POST | /sops/discover/confluence | Discover from Confluence spaces |
| POST | /sops/{id}/activate | Activate an SOP |
| POST | /sops/{id}/deactivate | Deactivate an SOP |
| GET | /health | Health check |
?repos=org/a&repos=org/b. The plain /sops/discover runs in the background and returns immediately; the per-source endpoints run synchronously and return the SOPs they generated. See the SOP Authoring guide.Webhook authentication
Each webhook source authenticates differently:
| Source | Auth mechanism | Header |
|---|---|---|
| Sensu | Bearer token (SENSU_WEBHOOK_TOKEN) | Authorization: Bearer <token> |
| Splunk | Bearer token (SPLUNK_WEBHOOK_TOKEN) | Authorization: Bearer <token> |
| PagerDuty | HMAC-SHA256 signature (PAGERDUTY_WEBHOOK_SECRET) | X-PagerDuty-Signature: v1=<hmac> |
For local testing without HMAC verification, leave PAGERDUTY_WEBHOOK_SECRET empty and the signature check is skipped.
Sensu webhook payload
curl -X POST http://localhost:8000/webhooks/sensu \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <your-sensu-token>" \
-d '{
"entity": {
"name": "api-backend-1",
"entity_class": "agent",
"namespace": "prod",
"labels": {"service": "api-backend", "environment": "prod"}
},
"check": {
"name": "cpu_high",
"status": 2,
"output": "CPU usage is 95%",
"labels": {"severity": "critical"}
},
"timestamp": 1704067200
}'
Splunk webhook payload
curl -X POST http://localhost:8000/webhooks/splunk \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <your-splunk-token>" \
-d '{
"search_name": "Alert - api-backend - High CPU - prod",
"app": "search",
"owner": "admin",
"results_link": "https://splunk.example.com/results/123",
"result": {
"host": "api-backend-1",
"_raw": "CPU usage exceeded threshold",
"_time": "2024-01-01T12:00:00Z"
},
"service": "api-backend",
"environment": "prod",
"severity": "critical",
"signal": "cpu_high"
}'
PagerDuty webhook payload
Without HMAC verification (local testing):
curl -X POST http://localhost:8000/webhooks/pagerduty \
-H "Content-Type: application/json" \
-d '{
"messages": [{
"event": "incident.triggered",
"incident": {
"id": "P123ABC",
"incident_number": 123,
"title": "High CPU on api-backend in prod",
"status": "triggered",
"urgency": "high",
"html_url": "https://pagerduty.com/incidents/P123ABC",
"service": {"id": "PSVC123", "name": "api-backend"}
}
}]
}'
With an HMAC signature:
SECRET="your-pagerduty-webhook-secret"
BODY='{"messages":[{"event":"incident.triggered","incident":{"id":"P123","incident_number":1,"title":"Test","status":"triggered","urgency":"high","html_url":"https://pd.com/1","service":{"id":"S1","name":"test"}}}]}'
SIGNATURE="v1=$(echo -n "$BODY" | openssl dgst -sha256 -hmac "$SECRET" | awk '{print $2}')"
curl -X POST http://localhost:8000/webhooks/pagerduty \
-H "Content-Type: application/json" \
-H "X-PagerDuty-Signature: $SIGNATURE" \
-d "$BODY"
After sending a test alert, confirm it landed with curl http://localhost:8000/alerts.
Rate limiting
Webhook endpoints are rate limited. A 429 response indicates the limit has been reached; the response includes a Retry-After header. You can verify it by sending requests rapidly:
for i in $(seq 1 10); do
curl -s -o /dev/null -w "%{http_code}\n" \
-X POST http://localhost:8000/webhooks/sensu \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <your-token>" \
-d '{"entity":{"name":"test","entity_class":"agent","namespace":"prod","labels":{"service":"test","environment":"prod"}},"check":{"name":"test","status":2,"output":"test","labels":{"severity":"warning"}},"timestamp":1704067200}'
done