All versions since v2.3.0 (2025-05-15)

v2.3.0 (2025-05-15)

Fixes

• Improved initial data synchronization speed and prevented Out-Of-Memory errors by implementing a high-watermark memory management strategy.
• Eliminated LDAP message size limits to prevent failures with large LDAP server responses.
• Optimized Database connection pool handling to prevent failed acquisitions.
• Enhanced memory and CPU usage metrics for more accurate monitoring.
• Improved performance of health and healthy check responses.
• Streamlined logging levels and messages for better clarity and reduced noise.

Breaking

• Removed built-in Prometheus metrics for PostgreSQL to mitigate excessive database load. For production environments, using the dedicated Prometheus Exporter is now recommended.

v2.4.0 (2025-06-25)

Features

• Faster and More Reliable Lookups: New database indexes have been added to speed up directory entry searches, especially for large datasets.
• Binary LDAP Entry Serialization: Storing LDAP entries in a binary format, for faster processing during sync and retrieval
• Optimized SQL Query Performance: SQL queries are now dynamically constructed for highly selective searches, ensuring PostgreSQL always uses the most efficient query plan.
• Initial Sync Performance: Improved initial synchronization speed by removing unnecessary locking.
• Sync State Query Optimization: Enhanced synchronization speed by optimizing queries that determine the state of entries

Fixes

• Service Info: Improved scribe service info to include the starting state as a healthy state, preventing health check failures during slow startups.
• Health Check: Shows the correct service name in the health check response.
• Monitoring: Suppressed noisy error logs for client disconnects (e.g., “Broken pipe”, “Connection reset”) on metrics and health endpoints. These are now logged at debug level instead of error, reducing log noise from normal client disconnects.
• Scribe Service: Improved error handling and logging in case of disconnects for scribe service.
• Scribe Service: After a restart, the scribe service will now continue with the incremental sync instead of the initial sync.

Breaking

• Attribute Sets Removed: Attribute sets are no longer supported and should be removed from the configuration

When removing attributeSets from your configuration, make sure to add all attributes that were previously only listed in attributeSets to the main attributes configuration. This ensures that all attributes are still observed and processed as before.

# Before (deprecated)
attributeSets = [
 "uid, mail, displayName"
]
attributes = "cn sn"

# After (supported)
attributes = """
 cn, sn
 uid, mail, displayName
"""

• Removed Prometheus Metrics: This version removes the following Prometheus metrics
• scribe.entries.refresh.cache.count
• scribe.entries.refresh.cache.time

v2.5.0 (2025-06-26)

Features

• LDAP Socket Tuning: Added support for tuning the LDAP socket settings using environment variables and configuration properties, with optimized defaults for production environments.
• Monitoring: Use dynamic balanced thread pool for metrics scraping to improve performance.

Fixes

• Query Syntax Correction: Removed invalid USE INDEX hint from LDAP presence filters.
• Prometheus Tag Standardization: All scribe_events_count metrics now use a consistent set of tag keys (event, op, target) to comply with Prometheus requirements. For event types where op or target are not applicable, the value "none" is used. This applies to event=add, event=move, and event=delete. This resolves meter registration errors and ensures reliable metric collection and querying.

v2.6.0 (2025-07-21)

Features

• LDAP Entry Reconciliation: Automatic reconciliation of entries in the database with the LDAP server.

  • The reconciliation is triggered by a maintenance task that runs periodically if either the interval or cron is set. See the reference.conf for the configuration options and defaults.
  • One reconciliation run is triggered after the initial sync is complete and the continuous search is started, regardless of the schedule, to ensure that all deletions during down time are caught.
  • Enable scheduled reconciliation if:
    • Your LDAP server has unreliable persistent search delete notifications
    • You need guaranteed consistency checks for compliance
    • You experience frequent network partitions
  • New Prometheus metrics for reconciliation:
    • scribe_reconciliation_entries_verified_total (counter): Number of entries verified as present in LDAP during reconciliation.
    • scribe_reconciliation_entries_deleted_total (counter): Number of entries deleted (synthetic deletes emitted) during reconciliation.
    • scribe_reconciliation_duration_seconds (summary/timer): Total time taken for a full reconciliation run.
    • scribe_reconciliation_last_run_timestamp_seconds (gauge): Unix timestamp of the last completed reconciliation run.

• Maintenance Task Scheduling: Added support for scheduling regular maintenance tasks

  • Please check the reference.conf for the configuration options and defaults.
  • The following services support maintenance tasks:
    • Database: for vacuuming and re-indexing the database instead of running these during startup
      • New Prometheus metrics for database maintenance:
        • database_maintenance_duration_seconds (timer): Duration of each maintenance run.
        • database_maintenance_failed_total (counter): Number of failed maintenance runs.
        • database_maintenance_last_run_timestamp_seconds (gauge): Unix timestamp of the last attempted maintenance run.
    • Scribe: for automatice reconciliation of entries in the database with the LDAP server.
      • New Prometheus metrics for scribe maintenance: are descibed above

• Intelligent Data Compression: Automatic compression of LDAP entry data to reduce storage footprint and network transfer sizes.

  • System automatically selects optimal compression algorithm based on data characteristics
  • Completely transparent to applications - no configuration changes required
  • Up to 70% reduction in storage usage for typical LDAP entries
  • New Prometheus metrics track compression effectiveness and performance

• LDAP Identity Resolution Reliability: Enhanced the way the system matches and updates LDAP entries, ensuring that even in rare cases—such as after directory restores or renames—entries are always correctly identified and updated.

  • Added extra safeguards to prevent rare synchronization issues, further protecting data integrity during both initial and ongoing syncs.
  • These improvements make the synchronization process more robust and reliable, especially in complex or high-throughput environments.

• Monitoring: The metrics are now scraped internally and cached at a regular interval to prevent slow responses.

  • Added a new monitoring.prometheus.scrapeInterval configuration property to control the interval at which the metrics are internally scraped. The default is 15 seconds.

  monitoring.prometheus.scrapeInterval = 15s

Fixes

• License Verification: Improved license verification to prevent the service from crashing in case of failed license verification due to connectivity issues to third party services like LDAP or databases. The service will now retry up to 5 times.
• Attribute Change Descriptions: Fixed an issue where change logs and event descriptions could use lowercased attribute names instead of the configured casing. All change tracking now consistently uses the attribute casing as defined in your configuration, ensuring clarity and compatibility with downstream systems.
• Operational Attribute Handling: Resolved a bug that could falsely report the removal of certain attributes (such as createTimestamp, modifyTimestamp, or entryUUID) during entry reconciliation. The system now correctly ignores unobserved operational attributes when comparing entries, preventing spurious removal events and ensuring only relevant changes are tracked.

Breaking Changes

• Metrics Optimization: This release includes significant changes to metrics that may impact existing dashboards and monitoring configurations.

  • Performance Impact
    • ~50% fewer metrics: Reduced from ~1,600 to ~800 total metrics
    • Memory optimization: Rolling windows with 5-minute expiry to reduce memory usage and provide more accurate metrics
    • Reduced cardinality: Service transitions, LDAP searches optimized
    • Better precision: Summary percentiles vs histogram approximations
    • Lower storage: Fewer time series to store and query, reducing storage requirements
  • Monitoring Recommendations
    • Update dashboards to use new metric formats before upgrading
    • Test queries in staging environment first to ensure the new metrics are working as expected
    • Review alerts that depend on histogram buckets to ensure they are still working as expected
    • Monitor memory usage after upgrade (should decrease)

  Detailed Changes

  • Service Transition Metrics

    Before:

    service_transition_seconds{service="X",from="new",to="starting",...}
    service_transition_seconds{service="X",from="starting",to="running",...}
    service_transition_seconds{service="X",from="running",to="failed",...}

    After:

    service_transition_seconds{service="X",type="startup",...}
    service_transition_seconds{service="X",type="restart",...}
    service_transition_seconds{service="X",type="failure",...}

    Changes:

    • Removed: from and to tags (high cardinality)
    • Added: type tag with meaningful categories (startup, restart, failure, shutdown)
    • Filtered: Startup noise transitions (new → starting)
    • Percentiles: Reduced from 5 to 2 percentiles (0.5, 0.95)

    Migration:

    • Update dashboards to use type instead of from/to tags
    • Categories: startup, restart, failure, shutdown

    # Old query
    service_transition_seconds{from="starting",to="running"}
    
    # New query
    service_transition_seconds{type="startup"}

  • LDAP Search Metrics

    Before: Histogram with 50+ buckets

    channel_ldap_search_time_seconds_bucket{...,le="0.001"} 0
    channel_ldap_search_time_seconds_bucket{...,le="0.002"} 1
    [... 50+ buckets ...]

    After: Summary with optimized percentiles

    channel_ldap_search_time_seconds{...,quantile="0.5"} 0.043
    channel_ldap_search_time_seconds{...,quantile="0.95"} 0.051
    channel_ldap_search_time_seconds{...,quantile="0.99"} 0.052

    Changes:

    • Format: Histogram → Summary
    • Percentiles: Reduced to 0.5, 0.95, 0.99 (high cardinality optimization)
    • Buckets: Removed all histogram buckets (50+ → 0)

    Migration:

    • Replace histogram_quantile() with direct quantile label access
    • Update SLI/SLO calculations to use summary percentiles

    # Old query
    histogram_quantile(0.95, rate(channel_ldap_search_time_seconds_bucket[5m]))
    
    # New query
    channel_ldap_search_time_seconds{quantile="0.95"}

  • Processing Time Metrics

    Before: Histogram with SLO buckets

    scribe_processing_time_seconds_bucket{entryType="user",phase="diffing",le="0.001"} 5
    scribe_processing_time_seconds_bucket{entryType="user",phase="diffing",le="0.002"} 25
    [... many buckets ...]

    After: Summary with full percentiles

    scribe_processing_time_seconds{entryType="user",phase="diffing",quantile="0.5"} 0.004
    scribe_processing_time_seconds{entryType="user",phase="diffing",quantile="0.75"} 0.007
    scribe_processing_time_seconds{entryType="user",phase="diffing",quantile="0.9"} 0.015
    scribe_processing_time_seconds{entryType="user",phase="diffing",quantile="0.95"} 0.050
    scribe_processing_time_seconds{entryType="user",phase="diffing",quantile="0.99"} 0.487

    Changes:

    • Format: Histogram → Summary
    • Percentiles: Added 0.5, 0.75, 0.9, 0.95, 0.99 (standard set)
    • Buckets: Removed all histogram buckets

    Migration:

    • Update queries to use quantile labels instead of histogram_quantile()
    • SLIs can now use direct percentile values

    # Old query
    histogram_quantile(0.90, rate(scribe_processing_time_seconds_bucket[5m]))
    
    # New query
    scribe_processing_time_seconds{quantile="0.9"}

  • New Entry Codec Metrics

    Added new compression and entropy metrics:

    scribe_entry_encode_bytes{codec="lz4|zstd|none",kind="raw|compressed",quantile="0.5"} 1440
    scribe_entry_decode_bytes{codec="lz4|zstd|none",kind="raw|compressed",quantile="0.5"} 1440
    scribe_entry_encode_compression{codec="lz4|zstd",le="5.0"} 4
    scribe_entry_encode_entropy{le="25.0"} 137598

    Features:

    • Encode/Decode sizes: Track compression effectiveness by codec
    • Compression percentage: Business-critical buckets at 85% and 95% thresholds
    • Entropy analysis: Optimized buckets for compression decision making
    • Percentiles: Standard set 0.5, 0.75, 0.9, 0.95, 0.99

v2.7.0 (2025-11-14)

Features

• Virtual Attributes: Compute attribute values on-demand instead of storing them, solving performance and storage issues with massive multi-valued attributes (100k+ values).

  • Define attributes using LDAP filter expressions with variable substitution
  • Two patterns supported:
    • Cross-reference: Compute relationships to other entries (e.g., finding all groups a user belongs to via groupMembership)
    • Self-reference: Filter based on current entry’s attributes (e.g., computing userLevel based on departmentNumber)
  • Zero storage overhead and always up-to-date
  • Transparent to LDAP clients - virtual attributes work in searches, filters, and sorting
  • Optimized SQL queries leverage existing partitioning and indexing

  Configuration Example:

  ldap {
    virtualAttributes {
      # Cross-reference: Find all groups where this user is a member
      groupMembership {
        filter = "(member={{current.entryDN}})"
        value = "{{other.entryDN}}"
      }
  
      # Self-reference: Compute user level based on department
      userLevel {
        filter = "(departmentNumber=Executive)"
        value = "senior"
      }
    }
  }
  
  transcribes = [
    {
      type = "user"
      ldap {
        attributes = """
          cn, sn, mail, groupMembership, userLevel
        """
        # groupMembership and userLevel will be computed virtually
      }
    }
  ]

  Use Cases:

  • Massive multi-valued attributes like equivalentToMe on roles (prevents timeouts and database bloat)
  • Reverse lookups without storing redundant data (e.g., “show me all groups this user belongs to”)
  • Computed attributes based on entry’s own properties (e.g., access levels, flags)

  See reference.conf for complete documentation and additional examples.

Fixes

• Checkpoint Advancement Under Load: Checkpoint timestamps now advance incrementally during continuous LDAP synchronization. Previously, checkpoints only updated when the system was idle, causing delays in offline delete detection and reconciliation. This fix improves responsiveness and ensures timely change detection even under sustained traffic.
• Reduced Monitoring Overhead: Simplified Prometheus metric buckets for compression and entropy monitoring, reducing time series by ~40-50%. This lowers memory usage and speeds up dashboard queries while maintaining all actionable thresholds. Existing dashboards remain compatible with no changes required.
• LDAP Connection Monitoring: Fixed an issue where LDAP connection metrics could show negative values in Prometheus monitoring. Connection tracking is now more accurate and reliable.
• LDAP Search Performance: Improved search filter processing for extended DN match filters, ensuring more accurate results.
• Enhanced Logging: Better diagnostic logging when operations are forwarded to the source LDAP server. Logs now include more context about entry types to help with configuration and troubleshooting.
• Query Optimization: Improved performance of LDAP filter processing by optimizing queries for common operational attributes, reducing unnecessary database overhead.
• SQL Function Optimization: Replaced PL/pgSQL functions with SQL functions to improve performance and reduce database overhead.
• SQL Index Optimization: Gather statistics for function based indexes to help the query planner to use the right indexes.
• Prometheus Metrics Restructuring: Reorganized LDAP forwarding metrics to ensure reliable metric collection and prevent registration errors. This change improves monitoring reliability but requires dashboard updates.

🚧 Unreleased Latest

Major release with REST and GraphQL APIs, MCP integration for AI assistants, OpenTelemetry-based observability, and a ready-to-run monitoring bundle. Multiple breaking changes to config keys, endpoints, and metrics. Plan an upgrade window.

Highlights

• Documentation site — Browse guides, reference, and APIs at /docs/
• REST API + OpenAPI UI (Scalar) with ETag-based conditional requests — see the REST Channel Guide
• GraphQL API + GraphiQL — Flexible query API with Relay Node interface and dynamic schema generation — see the GraphQL Channel Guide
• MCP Channel — AI-ready integration for Cursor, Claude Desktop, VS Code, Windsurf — query data and monitor health from your IDE — see the MCP Channel Guide
• Turnkey monitoring bundle shipped (Grafana dashboards + Docker Compose + Helm chart) — see the Monitoring Guide
• OpenTelemetry-first observability with metrics inventory and migration mapping — see Observability
• Operational endpoints — /observe/* for status, health, doctor/pressure, stats, and channel discovery — see Monitoring Guide
• Query performance improvements (planner v2 + backpressure under load) — see also Backpressure Under Query Load
• Rule-based log filtering — suppress noise with glob patterns and LDAP-style filters — see Observability: Wide Logs
• FleX query language — flexible filter syntax for runtime queries and config — see Filters Reference

Breaking Changes

If you’re upgrading an existing deployment, start here: Upgrading IdentityScribe.

Environment Variable Prefix

BREAKING: Environment variable prefixes have been standardized.

• IDENTITY_SCRIBE_* → SCRIBE_*
• SCRIBE_TELEMETRY_* → SCRIBE_*

Migration:

# Old
IDENTITY_SCRIBE_READONLY=true
SCRIBE_TELEMETRY_TRACES_ENABLED=true

# New
SCRIBE_READONLY=true
SCRIBE_TRACES_ENABLED=true

See Upgrading: env prefix.

Logger Configuration Consolidated

BREAKING: Logger configuration has been consolidated to 5 canonical loggers with standardized environment variable overrides. Old logger config keys are removed.

New canonical loggers:

All loggers inherit from log.level by default. Set log.Monitoring = "off" to disable wide-log output entirely.

Migration:

# Old
log.EventStore = "debug"
log.Settings = "warn"

# New
log.Ingest = "debug"
log.Config = "warn"

# New environment variables
SCRIBE_LOG_INGEST=debug
SCRIBE_LOG_CONFIG=warn
SCRIBE_LOG_MONITORING=off  # Disable wide-log output

See Upgrading: logging.

HTTP Configuration Restructured

BREAKING: All HTTP channels now use a unified socket model. Old monitoring and REST listener config keys are removed.

New configuration model:

• Default socket binds to localhost:8080 (safe-by-default)
• Named sockets defined in http.sockets.* for network separation
• Channels reference sockets by name: channels.rest.socket, monitoring.socket

Migration:

# Old
monitoring {
  hostname = "localhost"
  port = 9001
}

# New (simple: everything on one port)
http {
  port = 8080
  host = "0.0.0.0"
}

# New (separated: monitoring on internal port)
http {
  port = 8080
  host = "0.0.0.0"
  sockets.internal { port = 9001, host = "localhost" }
}
monitoring.socket = "internal"

See HTTP Server Guide for migration details and Upgrading: HTTP sockets.

Endpoint Path Changes

BREAKING: Status endpoints have been renamed. No aliases are provided.

Migration: Update monitoring tools, dashboards, and Kubernetes probes to use the new paths. Use /observe/health/ready or /readyz for readiness-only checks.

Note: /observe now serves interactive OpenAPI documentation for the Observe endpoints.

See Upgrading: endpoints.

Health Endpoint Format

BREAKING: Health endpoints now return MicroProfile Health JSON format.

// Old /healthy response
{"healthy": true, "ready": true, "sync": {...}}

// New /observe/health/ready response
{"status": "UP", "checks": [{"name": "ingest", "status": "UP"}]}

Use /observe/health for combined MicroProfile health status.

Migration for Kubernetes:

# Old
livenessProbe:
  httpGet:
    path: /healthy

# New
livenessProbe:
  httpGet:
    path: /livez
readinessProbe:
  httpGet:
    path: /readyz
startupProbe:
  httpGet:
    path: /startedz

See Upgrading: endpoints.

Metric Naming Standardization

BREAKING: All metrics now use the scribe.* prefix with dot notation. Prometheus auto-converts to snake_case (scribe_*).

Migration: Update Prometheus queries and Grafana dashboards. See Observability for the full migration mapping.

Common renames:

See Upgrading: metrics names.

Virtual Attribute Syntax

BREAKING: Virtual attribute placeholders renamed for consistency.

• {{current.*}} → {{self.*}}

Migration: Update virtual attribute configurations. Startup fails with instructions if legacy syntax is found.

# Old
filter = "(member={{current.entryDN}})"

# New
filter = "(member={{self.entryDN}})"

See Upgrading: virtual attributes.

Query Scoping Validation

BREAKING: Broad searches that could span multiple entry types must now include an explicit type constraint. Ambiguous queries are rejected early with a clear, actionable error.

Migration:

• Add a type constraint (for example entryType or a type-identifying condition such as objectClass) to any broad queries.
• If you rely on wildcard/broad bases, test queries in staging and update filters/config accordingly.

See Upgrading: query scoping.

Error Code Naming

BREAKING: Error codes are now standardized to SCREAMING_SNAKE_CASE. If clients match on error codes, update them to the new values.

Migration: See Failures for HTTP/REST error codes, headers, and response schema.

Support workflow: When reporting issues, include Error-Id, error.timestamp, and (if present) Trace-Id and your Correlation-Id. See “Reporting Issues to Support” in the Failures guide.

See Upgrading: error codes.

Transcribes Configuration Syntax

BREAKING: The transcribes configuration now uses object syntax where the key is the transcribe type:

transcribes {
  user {
    ldap { base = "ou=users,o=data", filter = "(objectClass=Person)" }
  }
  group {
    ldap { base = "ou=groups,o=data", filter = "(objectClass=groupOfNames)" }
  }
}

Legacy array syntax still works but logs a deprecation warning. Transcribes are processed alphabetically for deterministic behavior.

See Upgrading: transcribes.

Deprecations

Configuration Key Format

Configuration keys have been standardized to kebab-case. Legacy camelCase and underscore_case keys still work but log deprecation warnings.

Action: Update configuration files. Deprecated keys will be removed in a future major release. See the Configuration Reference for all current keys.

What’s New

Documentation Site

IdentityScribe now includes an embedded documentation site at /docs (on-prem) with /ref/* deep links into config/telemetry/error references, and publishes the same content as a public reference at scribe.identity-hub.io.

Improved 404 Handling

Unmatched HTTP routes now consistently return 404 Not Found instead of 500 Internal Server Error. A catch-all handler ensures clean 404 responses for any path that doesn’t match a registered route.

REST API Channel

A new REST/JSON channel provides HTTP access to directory data with OpenAPI documentation.

• OpenAPI 3.1.0 specification with an interactive UI (Scalar) at /api
• Collection endpoints at /api/entries/{type} with filtering, sorting, and pagination
• Single entry lookup at /api/entries/{type}/{id} — accepts Global ID, UUID, UOID, or DN identifiers
• Global ID field — entries include a Relay-style id field (base64url("{type}:{base36(uoid)}")) when operational attributes are selected
• Multi-format filters — FleX, JSON, SCIM (RFC 7644), or LDAP (RFC 4515) syntax auto-detected
• OpenAPI content negotiation — JSON, YAML, or HTML (Scalar UI) at /api based on Accept header
• JSON mapping — attribute names preserve requested alias/case; wildcard selection (* / +) uses canonical catalog names
• Parameters — list-type parameters (fields, sort, include) accept JSON arrays/objects in addition to comma/whitespace-separated strings
• Conditional requests (LOOKUP) — ETag + If-None-Match → 304 for efficient polling (ETags are weak validators because representations vary by fields)
• Code samples — embedded curl, JavaScript, TypeScript, and Go examples in the OpenAPI spec

# REST binds to the unified HTTP server (`http {}`).
http {
  host = "0.0.0.0"
  port = 8080
}

channels.rest {
  enabled = true
  # socket = "api"  # Optional: bind REST to a named socket
}

Or via environment: SCRIBE_REST_ENABLED=true SCRIBE_HTTP_PORT=8080

See the REST Channel Guide for endpoints, parameters, and examples.

• REST Conditional Requests: Added Last-Modified response header and If-Modified-Since request support for timestamp-based cache validation. ETag remains the preferred validator. RFC 7232 precedence is enforced: If-None-Match takes precedence over If-Modified-Since.

• REST Cache-Control Headers: Added explicit Cache-Control defaults:

  • Lookup: private, max-age=0, must-revalidate (enables conditional requests for authenticated/PII data)
  • Search: no-store (prevents caching of streaming/parameterized results)

• REST Prefer Header (RFC 7240): Added Prefer: handling=strict|lenient support for attribute validation:

  • handling=lenient (default): Unknown attributes in fields/sort/filter are silently ignored
  • handling=strict: Unknown attributes cause 400 Bad Request with INVALID_ARGUMENT code listing all unknowns
  • Responses include Preference-Applied header when handling is explicitly requested

• CORS Defaults: Default expose-headers now includes Cache-Control and Vary so browser clients can read caching headers.

GraphQL Channel

A new GraphQL channel provides flexible query access to directory data and change history with dynamic schema generation.

• Dynamic schema generated from entry type configuration — types, connections, and query fields
• Relay Node interface — all types implement Node with id: ID!; polymorphic lookup via node(id: ID!)
• Collection queries with filtering, sorting, and cursor pagination
• Connection counts — request estimated or exact totals via count(mode: ESTIMATED|EXACT) on entry and change connections (count-only queries skip data fetch). See GraphQL connection counts.
• Single entry lookup — accepts Global ID, UUID, UOID, or DN
• Change history API — global changes feed and per-entry changes field with filtering by time, type, and attributes
• Temporal lookup — query entries at any point in time via the at argument
• GraphiQL UI — interactive query builder at /graphql with schema explorer
• Automatic Persisted Queries (APQ) — reduce bandwidth with query caching
• Query security limits — depth and complexity limits protect against DoS

Enable via environment: SCRIBE_GRAPHQL_ENABLED=true

See the GraphQL Channel Guide for query patterns and configuration.

MCP Channel (Model Context Protocol)

AI coding assistants (Cursor, Claude Desktop, VS Code, Windsurf) can now query IdentityScribe directly from your IDE.

• Data MCP (/mcp) — Search entries, browse change history, look up schemas, and access reference docs
• Observe MCP (/observe/mcp) — Check health, diagnostics, and golden signals without leaving your editor

Enable via SCRIBE_MCP_ENABLED=true and SCRIBE_MONITORING_MCP_ENABLED=true.

See the MCP Channel Guide for setup, one-click install badges, and the full tool reference.

API Discovery

• API Catalog at /.well-known/api-catalog (RFC 9727) lists all API channels and schema URLs
• Schema endpoints — /api.json, /api.yaml, /graphql.sdl, /graphql.json for programmatic access
• GraphQL error codes now include both code (IdentityScribe-specific) and graphqlCode (standard) for client compatibility
• GraphQL performance — history queries only load requested fields (diff, patch, merge, meta) from the database

Rule-Based Log Filtering

Suppress log noise from fast/successful operations using glob patterns and LDAP-style filters:

monitoring.log.rules = [
  { action = exclude, where = "(&(scribe.result=ok)(duration.seconds<=50ms))" }
  { action = include, where = "(duration.seconds>=5s)" }
]

Rules evaluate first-match-wins. Unmatched operations fall back to monitoring.log.sample-rate.

See Observability: Wide Logs for syntax and examples.

FleX Query Language

FleX is a flexible filter syntax for runtime queries and configuration filters. Supports natural prefixes (status is active), lenient whitespace, logical combinators, IN lists, attribute groups, and all SCIM operators.

See the Filters Reference for syntax and examples.

OpenAPI for Operational Endpoints (/observe)

IdentityScribe now serves interactive OpenAPI documentation for the /observe/* endpoints at /observe (Scalar UI).

• OpenAPI 3.1 spec: JSON at /observe?format=json, YAML at /observe?format=yaml
• Scalar API Reference: HTML UI at /observe
• Status JSON: /observe/status (see Breaking Changes)

Configuration inherits from the root openapi {} section with per-endpoint overrides:

openapi {
  enabled = true
  ui {
    enabled = "auto"  # UI: enabled in dev/test, disabled in prod
    theme = "default"
  }
}

# Example override for Observe
monitoring.observe.ui.theme = "purple"

UI defaults are app.mode-aware (auto) for both REST and Observe. Developer tools and the test request button are also auto-gated; override via openapi.ui.* or channels.rest.ui.* (env: SCRIBE_OPENAPI_UI_*, SCRIBE_REST_UI_*).

Portal and Operator UI

Built-in web interface for operators:

• Portal (/) — System status, golden signals, and navigation to all features
• Entries (/ui/entries) — Browse and search entries with point-in-time navigation
• Changes (/ui/changes) — Change feed with filtering and diff viewing
• Observe (/ui/observe) — Health dashboard with pressure gauges and JVM metrics

See the Monitoring Guide for all views and operational workflows.

Change History API

Two new REST endpoints provide access to change history:

• GET /api/changes — Global change feed across all entries
• GET /api/entries/{type}/{id}/changes — Change history for a specific entry

Both endpoints support time range filtering (since/until), event type filtering (type), attribute change filtering (affects), and standard pagination. Response shapes include nodes, edges, patch, merge, meta, and data options.

See the interactive API documentation at /api for full parameter details and examples.

Point-in-Time Lookup

Entry lookups now support temporal queries via the at parameter:

# Get entry state at a specific timestamp
GET /api/entries/users/{id}?at=2024-04-22T08:43:50Z

# Get entry state from 1 hour ago
GET /api/entries/users/{id}?at=1h

# Get entry state at a specific event (using cursor from change history)
GET /api/entries/users/{id}?at=AAFntW9XAAAAAAApQwAA

Use with the Change History API to navigate between versions.

Entry Verification Timestamp

Added verifiedTimestamp for tracking when an entry was last verified during reconciliation.

• Accepted names: lastVerifiedAt, lastSeenVerified, verified_at, verifiedAt, verifiedTimestamp
• LDAP name: verifiedTimestamp (matches createTimestamp / modifyTimestamp)
• REST JSON: preserves the requested alias/case for attribute keys
• Type: ISO-8601 in REST, GeneralizedTime in LDAP
• Delegation: not delegated to upstream LDAP backends

Unified HTTP Server

All HTTP traffic now runs through a single WebServer with named sockets.

• AIMD-based concurrency limiting with auto-derived limits from global concurrency setting
• Request limits — max-payload-size and max-in-memory-entity per socket
• Connection limits — max-tcp-connections and max-concurrent-requests per socket
• CORS support — per-socket CORS with magic origins (same-origin, host, origin, *) and explicit origin lists
• Gzip compression enabled by default
• Graceful shutdown with configurable drain period
• HTTP/2 support via SCRIBE_HTTP2_ENABLED
• PII-safe telemetry defaults — common sensitive headers and client addresses are redacted in HTTP telemetry/logs by default

http {
  request-limits {
    max-payload-size = 10MiB     # or unset for unlimited
    max-in-memory-entity = 128KiB
  }
  connection-limits {
    max-tcp-connections = 1000   # or unset for unlimited
    max-concurrent-requests = 500
  }
  cors {
    enabled = true
    allow-origins = ["same-origin"]  # or explicit: ["https://app.example.com"]
    allow-credentials = false
  }
}

See the HTTP Configuration Reference for the http {} configuration block.

OpenTelemetry-Native Observability

Migrated from Micrometer to OpenTelemetry SDK.

• Prometheus at /metrics (OTel-backed)
• OTLP export for traces and metrics to collectors like Grafana Alloy
• Histogram exemplars link metrics to traces in Grafana
• Wide event logs — one structured log per operation with configurable sampling
• Pressure metrics — single-number health indicators (<1 = healthy, ≥1 = contention)

monitoring {
  traces.enabled = true
  exemplars = "traceBased"
}

See the Monitoring Guide for setup and Observability for metrics inventory.

Monitoring Bundle

IdentityScribe now ships a ready-to-run monitoring bundle (Grafana dashboards, Docker Compose, and a Helm chart). It’s included in the distribution under monitoring/.

See the Monitoring Guide for setup and deployment options.

MicroProfile Health Endpoints

Health checks now follow MicroProfile Health 4.0 with Kubernetes-friendly aliases.

Operational Endpoints

New endpoints for operational visibility:

Readiness probes now incorporate index build status; use /observe/indexes to diagnose readiness delays.

Notes:

• /observe/hints is available only when hint persistence is enabled.
• /observe/signatures requires EXPLAIN sampling to be enabled.
• /observe/doctor and /observe/pressure require metrics recording to be enabled.

Connection Hints

Tune PostgreSQL session parameters per-channel for different workload profiles.

database.connection-hints {
  statement-timeout = 60s
  work-mem = "256MB"
}

channels.rest.connection-hints {
  statement-timeout = 30s  # Override for REST only
}

Environment variables: SCRIBE_REST_STATEMENT_TIMEOUT, SCRIBE_LDAP_WORK_MEM, etc.

Virtual Attribute Delegation

Virtual attributes can now be forwarded to upstream LDAP backends when they support the attribute.

ldap.virtualAttributes {
  memberOf {
    filter = "(&(entryType=group)(member={{self.entryDN}}))"
    value = "{{other.entryDN}}"
    delegatable = true  # Forward to backend
  }
  securityEquals {
    # ...
    delegatable = "equivalentToMe"  # Rewrite name when delegating
  }
}

Virtual Attribute Type Scoping

Virtual attributes can now be scoped to specific entry types via the types field:

ldap.virtual-attributes {
  memberOf {
    types = ["user"]  # Only expose on user type
    filter = "(&(entryType=group)(member={{self.entryDN}}))"
    value = "{{other.entryDN}}"
  }
}

Omit types to expose on all types (backward compatible). Transcribe-level VAs are automatically scoped to their transcribe’s type. REST strict mode (Prefer: handling=strict) and GraphQL schema validation enforce type scoping at runtime.

Note: Transcribe-level VAs are now scoped to their defining transcribe by default. See the LDAP Configuration Reference for details.

Query Planner v2

The SQL generation engine has been rewritten for better PostgreSQL performance.

• Smart index utilization with truncation-aware patterns for long text values
• Stable query plans through padded IN-lists (up to 8 values) and array operators for larger sets
• Virtual attribute support including nested cross-references
• Cursor-based pagination with efficient forward/backward paging

Backpressure Under Query Load

Under high query load, IdentityScribe applies backpressure to protect the database:

• HTTP channels (REST/GraphQL) fail fast with 503 UNAVAILABLE and Retry-After when the system is busy
• LDAP blocks up to the configured time limit (rather than failing fast)

Tune via database.query-http-acquisition-timeout (see Database Configuration).

Transcription Throughput Tuning

Transcription uses a bounded worker pool with a bounded queue to reduce memory pressure under load.

• Configure per-transcribe throughput with transcription.workers and transcription.queue-capacity (defaults are {auto})
• See the Transcribes Configuration Reference for tuning guidance and defaults

Sortable Index Configuration

Attributes listed in indices.sortable now receive sortable indexes even if they are single-valued, improving ORDER BY performance.

If you previously relied on automatic filtering of single-valued attributes, re-check your index configuration (see the Configuration Reference).

Platform Support (PostgreSQL + Toolchain)

• PostgreSQL: 18 is the primary target, 17 is fully supported, and 15/16 are deprecated compatibility targets

• Native binary: the distributed artifact is a native executable (no Java runtime required to run)

• Building from source: the build toolchain targets Java 25 / GraalVM 25

• 50-60% faster startup through parallel service initialization

• Non-blocking index builds via CREATE INDEX CONCURRENTLY

Logging and Operator Experience

Improved startup banner, log formatting, and sampling behavior.

Startup banner now shows mode, license, config sources, backend URLs, channel bindings, and portal URLs. Printed after services are running, so port bindings are accurate:

 identity-scribe  v1.2.3 ready in 847ms

┃ Mode        production
┃ License     ACME Corp — expires 2027-01-15 (365d)
┃ Readonly    true

┃ Config
┃   /etc/scribe/identity-scribe.conf
┃   environment variables

┃ Database    postgresql://db.example.com:5432/scribe (ssl)
┃ LDAP        ldaps://dc01.example.com:636

┃ Channels
┃   REST API  http://localhost:8080/api
┃   LDAP      ldap://localhost:10389

Other improvements:

• PRETTY format uses ANSI colors for result status (green/yellow/red) and attribute prefixes. Auto-detected; disable with NO_COLOR=1.
• Exception bypass — operations with exceptions now bypass log sampling (same as warnings)
• Third-party loggers — HikariCP, Flyway, and others are configurable via SCRIBE_LOG_* env vars

See the Configuration Guide for log format options.

Failure Handling and Support

The Failures guide includes a “Reporting Issues to Support” checklist, including which IDs/headers to include (Error-Id, Trace-Id, Correlation-Id).

Log Noise Reduction

Reduced log verbosity during initial sync and routine operations:

• Default noise gates — Hints.* and Metrics.* suppressed unless slow or failing
• Queue wait threshold — monitoring.log.queue-wait-threshold (default: 90s) controls when warnings emit
• Segment leak suppression — automatically suppressed during initial sync
• LDAP search events — short-lived segments instead of long-lived parent, eliminating leak warnings

Fixes

• Graceful shutdown — HikariPool connections are now soft-evicted before pool closure, eliminating timeout warnings
• Timezone handling — Fixed timestamp drift in non-UTC JVM timezones; JDBC connections now set PostgreSQL session timezone to UTC
• LDAP virtual attribute delegation — Virtual attributes are forwarded correctly during backend delegation instead of being rejected
• Virtual attribute warnings — Fixed a false warning for self-referencing virtual attributes; the entryType warning now only triggers for cross-referencing templates
• Pagination metadata — Cursor pagination now correctly indicates whether more results exist before/after the current page
• Backward paging — Backward navigation streams efficiently without buffering entire pages
• Time limit enforcement — Search time limits now cancel database work promptly
• Query cancellation — Failed or timed-out requests cancel remaining database work immediately