Deployment

Deploy IdentityScribe on Docker, Kubernetes, or bare metal. This guide covers installation, configuration, and verification for each deployment model.

1. Download the appropriate executable for your platform

2. Create minimal identity-scribe.conf file:

   # Database connection
   database {
     url = "jdbc:postgresql://localhost:5432/identity_scribe"
     user = "identity_scribe"
     password = "secure-password"
   }
   
   # LDAP backend (source directory to sync FROM)
   ldap {
     url = "ldap://your-ldap-server:389"
     bind-dn = "cn=admin,ou=sa,o=system"
     bind-password = "your-password"
   }
   
   # Entry types to sync
   transcribes {
     user {
       ldap {
         base = "ou=users,o=data"
         filter = "(objectClass=Person)"
       }
     }
   }
   
   # Enable REST API
   channels.rest {
     enabled = true
   }

   See config/minimal.conf for a complete example with environment variable overrides, or config/recommended.conf for a production-ready configuration with all channels.

3. Run: ./identity-scribe

4. Verify: curl http://localhost:8080/api/users | jq

When Identity Scribe starts, a styled startup banner displays operator-critical runtime information. The exact sections shown depend on your configuration — for example, Telemetry only appears when OTLP export is enabled, and Channels lists only the channels you’ve configured.

 identity-scribe  v1.2.3 ready in 847ms

┃ Mode        production
┃ License     ACME Corp — expires 2027-01-15 (365d)

┃ Config
┃   /etc/scribe/identity-scribe.conf
┃   environment variables

┃ Database    postgresql://db.example.com:5432/scribe (ssl)
┃ LDAP        ldaps://dc01.example.com:636

┃ Transcribes
┃   users     ou=users,dc=example,dc=com  (objectClass=person)
┃   groups    ou=groups,dc=example,dc=com (objectClass=group) [disabled]

┃ Telemetry
┃   Traces    http://localhost:4317 (grpc)
┃   Metrics   http://localhost:4317 (grpc)

┃ Monitoring
┃   Observe   http://localhost:9001/observe
┃   Health    http://localhost:9001/{livez,readyz,startedz,healthz}
┃   Metrics   http://localhost:9001/metrics

┃ Site
┃   Home      http://localhost:8080/
┃   Docs      http://localhost:8080/docs
┃   Ref       http://localhost:8080/ref
┃   UI        http://localhost:8080/ui

┃ Channels
┃   REST API  http://localhost:8080/api
┃   LDAP      ldap://localhost:10389

The banner is organized into sections:

• Header: Application name, version, and startup time
• Context: Application mode (production/development), license expiration status, and readonly warning (if enabled)
• Config: Configuration file paths and sources (environment variables if used)
• Data Layer: Database URL, LDAP backend URLs, and transcribe details (base DN, filter, disabled status)
• Telemetry: OTLP push endpoints for distributed tracing and metrics
• Monitoring: Pull endpoints for observability (observe, health, prometheus)
• Site: Operator-facing portal URLs (landing page, documentation, ref redirects, UI) — shown when site is enabled
• Channels: Enabled application channels — may include REST API, GraphQL, MCP, LDAP, or Identity Hub depending on configuration

The license expiration status is color-coded for quick visual assessment:

When running in readonly mode (SCRIBE_READONLY=true), a yellow warning is displayed:

┃ Readonly    true

Database and LDAP URLs are sanitized before display — credentials are never shown in the banner. Only the scheme, host, port, and database/path are displayed.

The banner uses ANSI colors for better readability. Color support is automatically detected based on:

1. FORCE_COLOR — explicit override (1/true = on, 0/false = off)
2. NO_COLOR — if present and not empty, disables colors (no-color.org)
3. TERM=dumb — no color support
4. Console/TTY presence — colors enabled when running interactively
5. CI environments — colors enabled in GitHub Actions, GitLab CI, CircleCI, Travis, Buildkite, and generic CI=true

# Disable colors (follows no-color.org standard)
NO_COLOR=1 ./identity-scribe

# Force colors on (even when piped or in non-TTY)
FORCE_COLOR=1 ./identity-scribe | tee output.log

# Force colors off (overrides everything)
FORCE_COLOR=0 ./identity-scribe

The formatting (box characters, alignment) is preserved regardless of color settings.

Create an identity-scribe.conf file based on the examples above or use the provided templates:

• config/minimal.conf — Bare minimum: database, LDAP backend, one transcribe, REST channel
• config/recommended.conf — Production-ready: all channels (LDAP, REST, GraphQL, MCP), observability, indices

• Goal: deterministic ordering that matches application-side sorting (virtual attributes, truncation-aware ordering) across environments.

• Recommended for production (multi-language content): ICU collation und-x-icu for platform-consistent Unicode ordering. Use TEMPLATE template0 to guarantee the specified collation/ctype is applied (template1 may carry a different locale or extensions).

  CREATE DATABASE "identity_scribe"
    TEMPLATE template0
    ENCODING = 'UTF8'
    LC_COLLATE = 'und-x-icu'
    LC_CTYPE   = 'und-x-icu'
    CONNECTION LIMIT = -1;

• Alternative (performance-focused / ASCII-only): C collation for maximum speed; use only if locale-aware ordering is not required and keep it consistent across all environments. TEMPLATE template0 is likewise recommended to ensure the desired collation/ctype is applied.

  CREATE DATABASE "identity_scribe"
    TEMPLATE template0
    ENCODING = 'UTF8'
    LC_COLLATE = 'C'
    LC_CTYPE   = 'C'
    CONNECTION LIMIT = -1;

For detailed slow query analysis, enable the pg_stat_statements extension:

-- Run as superuser after database creation
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

This requires shared_preload_libraries = 'pg_stat_statements' in postgresql.conf (restart required). Managed PostgreSQL services (AWS RDS, Azure, GCP) typically provide a UI setting for this.

Once enabled, IdentityScribe’s monitoring UI and /observe/stats/queries endpoint show slow query statistics automatically. See Enabling pg_stat_statements for provider-specific instructions.

For secure connections, add SSL configuration:

ssl {
  ca = "ca-bundle.pem"
  cert = "cert.pem"
  key = "key.pem"
  password = "optional-key-password"

  protocols = [
    "TLSv1.3"
    "TLSv1.2"
  ]
}

usage: identity-scribe
-c,--config <file>      use config file (supported are conf, json,
                        properties) (default: enviroment 'config.file'
                        which defaults to 'identity-scribe.conf')
-d,--debug              print debugging information
-D <arg>                key=value to set as property
-f,--logfile <file>     use given file, System.out, or System.err for log
                        (default: System.out)
-h,--help               print this message
-k,--license <file>     use license file (default: enviroment
                        'license.file' which defaults to
                        'identity-scribe.lic' relative to config file)
-l,--loglevel <level>   one of trace, debug, info, warn, error, fatal
                        (default: info)
-p,--printconfig        print the resolved config and exit
-q,--quiet              be extra quiet
-r,--readonly           Readonly mode — only channels are started
-s,--sync               Force re-sync of all entries
-t,--trace              print tracing information
-v,--version            print the version information and exit

Dev vs Prod Example:

# Development (enables leak detection, verbose errors)
export SCRIBE_APP_MODE=development

# Production (disables dev features)
export SCRIBE_APP_MODE=production

• SCRIBE_READONLY or identity-scribe -DREADONLY=...
• SCRIBE_TRANSCRIBE_ENABLED or identity-scribe -DTRANSCRIBE_ENABLED=...
• SCRIBE_TRANSCRIBE_<uppercase(type)>_ENABLED or identity-scribe -DTRANSCRIBE_<uppercase(type)>_ENABLED=...
• SCRIBE_CONFIG_FILE or identity-scribe -DCONFIG_FILE=...
• SCRIBE_LOG_FILE or identity-scribe -DLOG_FILE=...
• SCRIBE_LOG_LEVEL or identity-scribe -DLOG_LEVEL=...

Compatible with:

• Windows: Server 2019/2016/2012/2008 R2, Windows 11/10/8/7 SP1+
• macOS: Big Sur, Catalina, Mojave, Monterey
• Linux: Ubuntu, Debian, RHEL, CentOS, Amazon Linux, Oracle Linux, SLES, OpenSUSE, Alpine

• PostgreSQL:
  • Primary: 18 (mainline support + mainline CI gating)
  • Fallback: 17 (fully supported + mainline CI gating)
  • Deprecated compatibility: 15/16 (supported for now; reduced CI depth; removal timeline will be announced)
• Supports Plain, SSL, and Mutual-SSL connections

Note: PostgreSQL 15/16 are deprecated for compatibility. While still supported, they receive reduced test coverage and are not recommended for new deployments. The removal timeline will be set during the next major planning cycle or when PostgreSQL 15 approaches upstream EOL.

Recommended upgrade path: PostgreSQL 15/16 → 17 → 18

PostgreSQL 17 and 18 provide significant benefits for IdentityScribe’s workload:

• PostgreSQL 17: Improved JSON/value handling, faster sequential scans, better I/O performance, enhanced monitoring (pg_stat_io), and VACUUM improvements
• PostgreSQL 18: Asynchronous I/O (AIO), UUIDv7 support, skip scans for partition queries, virtual generated columns, parallel GIN builds, and faster pg_upgrade

Upgrade Checklist (17 → 18):

1. Pre-upgrade:

   • Review PostgreSQL 18 release notes for breaking changes
   • Backup your database: pg_dump -Fc identity_scribe > backup.dump
   • Verify IdentityScribe version compatibility (check changelog)
   • Test upgrade on a staging environment first

2. Upgrade:

   • Stop IdentityScribe service
   • Run pg_upgrade (PostgreSQL 18 includes faster upgrade process)
   • Verify database integrity: psql -c "SELECT version();"
   • Start IdentityScribe service

3. Post-upgrade:

   • Monitor application logs for errors
   • Verify LDAP connectivity and search operations
   • Check Prometheus metrics for performance improvements
   • Review pg_stat_io (PostgreSQL 17+) for I/O insights

Monitoring PostgreSQL 17/18 Features:

• pg_stat_io (PostgreSQL 17+): Monitor I/O statistics for better performance insights
• AIO (PostgreSQL 18): Automatic asynchronous I/O improves partition scan performance
• Skip scans (PostgreSQL 18): Optimizes partition queries for better performance

• Uses maximum 80% of available physical memory
• Example: 4GB RAM = 3.2GB max usage, 32GB RAM = 25.6GB max usage

• IO-bound application
• Multiple cores beneficial for parallel processing

Metrics are exposed on the HTTP server (default port 8080) in Prometheus format:

• Access: http://localhost:8080/metrics
• Health check: http://localhost:8080/observe/health/ready (or http://localhost:8080/readyz)
• Status check: http://localhost:8080/observe/status

Note: The bundled monitoring stack (monitoring/docker, monitoring/helm) uses a separate internal socket on port 9001 for monitoring. This is an example of named socket separation — the default configuration uses port 8080 for all endpoints.

Monitor thread utilization and concurrency:

• executor_active_threads{name="TaskExecutor"} – Number of active threads in the task executor
• scribe.concurrency – Configured concurrency limit (total threads available for processing tasks)
• JVM Thread Metrics – Standard JVM thread metrics via Micrometer:
  • jvm_threads_live – Current number of live threads
  • jvm_threads_peak – Peak number of live threads
  • jvm_threads_daemon – Number of daemon threads

Example Prometheus Query:

# Current active threads
executor_active_threads{name="TaskExecutor"}

# Thread utilization ratio
executor_active_threads{name="TaskExecutor"} / scribe_concurrency

Dashboard Recommendations:

• Panel: Line graph showing executor_active_threads over time
• Alert: If thread utilization consistently exceeds 90%, consider increasing concurrency settings

Monitor database connection pool utilization for Batch, System, and Channel pools:

• hikari.connections.active{pool="Batch|System|Channel"} – Number of active connections in use
• hikari.connections.idle{pool="Batch|System|Channel"} – Number of idle connections available
• hikari.connections.pending{pool="Batch|System|Channel"} – Number of threads waiting for a connection
• hikari.connections{pool="Batch|System|Channel"} – Total pool size (active + idle)

Example Prometheus Queries:

# Active connections per pool
hikari_connections_active{pool="Batch"}
hikari_connections_active{pool="System"}
hikari_connections_active{pool="Channel"}

# Pool utilization percentage
hikari_connections_active{pool="Batch"} / hikari_connections{pool="Batch"} * 100

# Connections waiting (indicates pool saturation)
hikari_connections_pending{pool="Batch"}

Tuning Guidelines:

• Under-sizing signs: If pending connections are consistently high or active equals maximum for extended periods, consider increasing pool size
• Over-sizing signs: If idle connections consistently exceed 50% of maximum, consider reducing pool size to free database resources
• Virtual thread workloads: With virtual threads, many more tasks can be queued than concurrency, but each task still uses one connection. Monitor actual connection utilization rather than task count

Dashboard Recommendations:

• Panel: Stacked area chart showing active, idle, and pending connections per pool
• Alert: If pending connections > 5 for more than 1 minute, investigate pool saturation

Monitor lock contention to validate concurrency optimizations:

• scribe.locks.wait_time – Time spent waiting to acquire a lock (Timer)
• scribe.locks.hold_time – Time a lock is held (Timer)
• scribe.locks.contention – Count of failed tryLock() attempts (Counter)
• scribe.locks.active – Current number of active lock acquisitions per stripe (Gauge)

Example Prometheus Queries:

# Mean lock wait time
rate(scribe_locks_wait_time_seconds_sum[5m]) / rate(scribe_locks_wait_time_seconds_count[5m])

# Mean lock hold time
rate(scribe_locks_hold_time_seconds_sum[5m]) / rate(scribe_locks_hold_time_seconds_count[5m])

# Contention rate (contention count / total acquisitions)
rate(scribe_locks_contention_total[5m]) / rate(scribe_locks_wait_time_seconds_count[5m]) * 100

# Active lock acquisitions per stripe
scribe_locks_active

Tuning Guidelines:

• High contention: If contention rate > 5% of acquisitions, consider increasing concurrency settings or stripe count multipliers
• High wait time: If mean wait time > 100ms under normal load, investigate lock contention patterns
• Stripe counts automatically scale with concurrency settings; manual adjustment typically not needed unless workload patterns are unusual

Dashboard Recommendations:

• Panel: Line graph showing mean wait time and hold time over time
• Panel: Bar chart showing contention rate by stripe
• Alert: If contention rate > 5% for more than 5 minutes, investigate lock contention

Monitor search and transcription latency:

• scribe.tasks.throttled – Number of tasks throttled due to concurrency limits (Counter)
• scribe.tasks.throttle.wait – Time spent waiting due to throttling (Timer)
• Search pipeline metrics – Granular metrics for SQL compilation, execution, and result processing (search.sql.*)
• Virtual attribute metrics – Compilation cost and value fan-out (search.virtual_attributes.*)
• Commit wait time – eventstore.commit.wait (Timer)

Example Prometheus Queries:

# Throttling rate
rate(scribe_tasks_throttled_total[5m])

# Mean throttle wait time
rate(scribe_tasks_throttle_wait_seconds_sum[5m]) / rate(scribe_tasks_throttle_wait_seconds_count[5m])

# Search latency percentiles
histogram_quantile(0.95, rate(search_sql_execution_seconds_bucket[5m]))
histogram_quantile(0.99, rate(search_sql_execution_seconds_bucket[5m]))

Dashboard Recommendations:

• Panel: Line graph showing throttle rate over time
• Panel: Heatmap showing latency distribution (p50, p95, p99)
• Alert: If throttle rate > 10/second for more than 1 minute, consider increasing concurrency settings

LDAP operation metrics use these symbols:

Tune GC and memory with these options:

-XX:MaximumHeapSizePercent=80
-XX:MaximumYoungGenerationSizePercent=30
-XX:±CollectYoungGenerationSeparately
-XX:MaxHeapFree=67108864

The application automatically calculates optimal thread pool size:

• Formula: Threads = CPU_Cores * Utilization * (1 + Wait/Compute)
• Default: CPU cores * 1.0 * (1 + 1.5)
• Database connections: Thread pool size + 2

# Paged results (RFC 2696)
ldapsearch -H "ldap://127.0.0.1:10389" -x -E '!pr=3/prompt' -s sub -b "ou=users,o=data" "(cn~=mayer)" 1.1

# Server side sorting (RFC 2891)
ldapsearch -H "ldap://127.0.0.1:10389" -x -E '!sss=-sn/givenName/cn' -s sub -b "ou=users,o=data" "(cn~=mayer)" sn givenName cn

# Virtual list views (ldapv3-vlv-09)
ldapsearch -H "ldap://127.0.0.1:10389" -x -E 'vlv=0/4/1/0' -E '!sss=-sn/givenName/cn' -s sub -b "ou=users,o=data" "(cn~=mayer)" sn givenName cn

1. Connection Failures

   • Check network connectivity
   • Verify SSL certificate validity
   • Confirm firewall rules
   • Validate database connection parameters

2. Performance Issues

   • Monitor thread pool utilization
   • Check database connection pool
   • Review GC logs
   • Verify sufficient memory allocation

3. Memory Problems

   • Adjust heap size settings
   • Monitor memory usage with Prometheus
   • Check for memory leaks with periodic restarts if needed

Configure logging levels in your configuration file:

log {
  level = "info"  # Default level (inherits to all components)
  file = "identity-scribe.log"

  # Component-specific levels (inherit from log.level by default)
  SuperVisor = ${log.level}   # Startup/shutdown, orchestration
  Ingest = ${log.level}       # Transcription pipeline, event store
  Monitoring = ${log.level}   # Wide-log output, observability
  License = ${log.level}      # License verification
  Config = ${log.level}       # Configuration parsing
}

Environment variable overrides:

SCRIBE_LOG_LEVEL=info          # Global level
SCRIBE_LOG_SUPERVISOR=debug    # Verbose startup/shutdown
SCRIBE_LOG_INGEST=debug        # Verbose transcription tracing
SCRIBE_LOG_MONITORING=off      # Disable wide-log output

Monitor application health:

# Status check
curl -s http://localhost:8080/observe/status | jq

# Health check
curl http://localhost:8080/observe/health/ready

# Metrics
curl http://localhost:8080/metrics

See HTTP Server Configuration for socket binding options.

For additional support, please contact your support representative or refer to your support agreement documentation.