Connection Issues
Clients can't connect, or connections drop intermittently. Work through the checks below to isolate the problem.
Where failures happen
Quick checks
Section titled “Quick checks”Observe JSON endpoints require an operator session or accepted credential by default; the examples below use a bearer token. For trusted local evaluation only, you may opt out with monitoring.observe.auth.enabled=false.
# Traffic readinesscurl http://localhost:8080/readyz
# Channel statuscurl -s -H "Authorization: Bearer <token>" http://localhost:8080/observe/channels | jq
# Database connectivitycurl -s -H "Authorization: Bearer <token>" http://localhost:8080/observe/doctor | jq '.checks[] | select(.name | startswith("db"))'Symptom: LDAP connection refused
Section titled “Symptom: LDAP connection refused”Check 1: Is LDAP enabled?
Section titled “Check 1: Is LDAP enabled?”curl -s -H "Authorization: Bearer <token>" http://localhost:8080/observe/channels | jq '.ldap'Expected: enabled: true, running: true
If disabled: Confirm channels.ldap.listen contains at least one port — the
LDAP channel auto-enables when one or more listen entries are configured.
Check 2: Correct port?
Section titled “Check 2: Correct port?”curl -s -H "Authorization: Bearer <token>" http://localhost:8080/observe/channels | jq '.ldap.bindings[].actualPort'The actual port may differ from the configured port when using ephemeral ports.
Check 3: Firewall
Section titled “Check 3: Firewall”Verify traffic is allowed to the LDAP port:
nc -zv identity-scribe-host 10389Symptom: REST API returns 503
Section titled “Symptom: REST API returns 503”Check 1: Is the service starting?
Section titled “Check 1: Is the service starting?”curl http://localhost:8080/startedzIf not started: Check logs for startup errors.
Check 2: Is the database connected?
Section titled “Check 2: Is the database connected?”curl -s -H "Authorization: Bearer <token>" http://localhost:8080/observe/doctor | jq '.checks[] | select(.name | startswith("db"))'Check 3: Can the service receive traffic?
Section titled “Check 3: Can the service receive traffic?”curl http://localhost:8080/readyzIf readiness fails, the pod should not receive traffic yet. To check whether long-running initial ingest and index work is fully complete, inspect convergence:
curl -s -H "Authorization: Bearer <token>" http://localhost:8080/observe/convergence | jqSymptom: /api returns 404, or the UI shows no entries
Section titled “Symptom: /api returns 404, or the UI shows no entries”This is usually a disabled, hidden, or wrong-socket endpoint — not a connectivity fault.
/apireturning404while/api/entries/...answers is expected in production: interactive documentation is hidden by default.- The Operator UI showing no entries usually means the REST channel is off, the UI and REST are on different sockets, the session is unauthenticated, or initial sync has not converged.
The full decision path — with the exact settings and curl checks — is in API Access & Networking — Troubleshooting.
Symptom: Intermittent timeouts
Section titled “Symptom: Intermittent timeouts”Check 1: Pool exhaustion
Section titled “Check 1: Pool exhaustion”curl -s http://localhost:8080/metrics \ | grep -E 'scribe_(db_connections_(pending|active)|db_pool_pressure|query_permit_(pressure|queue))'If scribe_db_connections_pending > 0 frequently or scribe_db_pool_pressure stays above 0.8, increase the relevant database pool size.
If scribe_query_permit_pressure approaches 1.0 or scribe_query_permit_queue grows, queries are waiting on concurrency permits before they reach the database.
Check 2: Source LDAP issues
Section titled “Check 2: Source LDAP issues”Check ingest lag:
curl -s -H "Authorization: Bearer <token>" http://localhost:8080/observe/stats/ingest | jq '.entry_types[].lag_seconds'High lag points to source LDAP problems.
Check 3: LDAP TLS handshake failures
Section titled “Check 3: LDAP TLS handshake failures”If an LDAPS connection fails during TLS negotiation, the log may include a unified ldap.ssl recommendation with the
protocols and cipher suites that completed the handshake probe:
ldap.ssl { ca = "/etc/scribe/tls/ldap-ca.pem" protocols = [TLSv1.2, TLSv1.3] cipher-suites = [ TLS_AES_256_GCM_SHA384 TLS_RSA_WITH_AES_256_GCM_SHA384 ]}If the diagnostic says certificate verification was reached, the protocol and cipher are compatible; confirm ldap.ssl.ca
points to the CA that issued the LDAP server certificate. If no compatible pair is found, check LDAPS versus StartTLS,
the port, and whether the LDAP server requires ldap.ssl.cert and ldap.ssl.key.
Check 4: Network timeouts
Section titled “Check 4: Network timeouts”Increase timeouts if the network is slow:
ldap.so-timeout = 30000database.query-http-acquisition-timeout = 10sDatabase connection checklist
Section titled “Database connection checklist”| Check | Command |
|---|---|
| PostgreSQL running | pg_isready -h localhost -p 5432 |
| Credentials correct | Check SCRIBE_DATABASE_USER/PASSWORD |
| SSL required | Set ?sslmode=require in URL |
| Network reachable | nc -zv db-host 5432 |
| Pool not exhausted | Check scribe_db_connections_pending and scribe_db_pool_pressure |
Related
Section titled “Related”- Deployment — System requirements
- Configuration — Connection settings
- Error Handling — Error codes, retries, and support workflow