Version History

All versions All versions forward

Version v2.6.0 (2025-07-21)

Features

LDAP Entry Reconciliation: Automatic reconciliation of entries in the database with the LDAP server.
- The reconciliation is triggered by a maintenance task that runs periodically if either the interval or cron is set. See the reference.conf for the configuration options and defaults.
- One reconciliation run is triggered after the initial sync is complete and the continuous search is started, regardless of the schedule, to ensure that all deletions during down time are caught.
- Enable scheduled reconciliation if:
  - Your LDAP server has unreliable persistent search delete notifications
  - You need guaranteed consistency checks for compliance
  - You experience frequent network partitions
- New Prometheus metrics for reconciliation:
  - scribe_reconciliation_entries_verified_total (counter): Number of entries verified as present in LDAP during reconciliation.
  - scribe_reconciliation_entries_deleted_total (counter): Number of entries deleted (synthetic deletes emitted) during reconciliation.
  - scribe_reconciliation_duration_seconds (summary/timer): Total time taken for a full reconciliation run.
  - scribe_reconciliation_last_run_timestamp_seconds (gauge): Unix timestamp of the last completed reconciliation run.
Maintenance Task Scheduling: Added support for scheduling regular maintenance tasks
- Please check the reference.conf for the configuration options and defaults.
- The following services support maintenance tasks:
  - Database: for vacuuming and re-indexing the database instead of running these during startup
    - New Prometheus metrics for database maintenance:
      - database_maintenance_duration_seconds (timer): Duration of each maintenance run.
      - database_maintenance_failed_total (counter): Number of failed maintenance runs.
      - database_maintenance_last_run_timestamp_seconds (gauge): Unix timestamp of the last attempted maintenance run.
  - Scribe: for automatice reconciliation of entries in the database with the LDAP server.
    - New Prometheus metrics for scribe maintenance: are descibed above
Intelligent Data Compression: Automatic compression of LDAP entry data to reduce storage footprint and network transfer sizes.
- System automatically selects optimal compression algorithm based on data characteristics
- Completely transparent to applications - no configuration changes required
- Up to 70% reduction in storage usage for typical LDAP entries
- New Prometheus metrics track compression effectiveness and performance
LDAP Identity Resolution Reliability: Enhanced the way the system matches and updates LDAP entries, ensuring that even in rare cases—such as after directory restores or renames—entries are always correctly identified and updated.
- Added extra safeguards to prevent rare synchronization issues, further protecting data integrity during both initial and ongoing syncs.
- These improvements make the synchronization process more robust and reliable, especially in complex or high-throughput environments.
Monitoring: The metrics are now scraped internally and cached at a regular interval to prevent slow responses.
- Added a new monitoring.prometheus.scrapeInterval configuration property to control the interval at which the metrics are internally scraped. The default is 15 seconds.
```
monitoring.prometheus.scrapeInterval = 15s
```

Fixes

License Verification: Improved license verification to prevent the service from crashing in case of failed license verification due to connectivity issues to third party services like LDAP or databases. The service will now retry up to 5 times.
Attribute Change Descriptions: Fixed an issue where change logs and event descriptions could use lowercased attribute names instead of the configured casing. All change tracking now consistently uses the attribute casing as defined in your configuration, ensuring clarity and compatibility with downstream systems.
Operational Attribute Handling: Resolved a bug that could falsely report the removal of certain attributes (such as createTimestamp, modifyTimestamp, or entryUUID) during entry reconciliation. The system now correctly ignores unobserved operational attributes when comparing entries, preventing spurious removal events and ensuring only relevant changes are tracked.

Breaking Changes

Metrics Optimization: This release includes significant changes to metrics that may impact existing dashboards and monitoring configurations.
- Performance Impact
  - ~50% fewer metrics: Reduced from ~1,600 to ~800 total metrics
  - Memory optimization: Rolling windows with 5-minute expiry to reduce memory usage and provide more accurate metrics
  - Reduced cardinality: Service transitions, LDAP searches optimized
  - Better precision: Summary percentiles vs histogram approximations
  - Lower storage: Fewer time series to store and query, reducing storage requirements
- Monitoring Recommendations
  - Update dashboards to use new metric formats before upgrading
  - Test queries in staging environment first to ensure the new metrics are working as expected
  - Review alerts that depend on histogram buckets to ensure they are still working as expected
  - Monitor memory usage after upgrade (should decrease)
Detailed Changes
- Service Transition Metrics
  
  Before:
```
service_transition_seconds{service="X",from="new",to="starting",...}
service_transition_seconds{service="X",from="starting",to="running",...}
service_transition_seconds{service="X",from="running",to="failed",...}
```
  After:
```
service_transition_seconds{service="X",type="startup",...}
service_transition_seconds{service="X",type="restart",...}
service_transition_seconds{service="X",type="failure",...}
```
  Changes:
  - Removed: from and to tags (high cardinality)
  - Added: type tag with meaningful categories (startup, restart, failure, shutdown)
  - Filtered: Startup noise transitions (new → starting)
  - Percentiles: Reduced from 5 to 2 percentiles (0.5, 0.95)
  Migration:
  - Update dashboards to use type instead of from/to tags
  - Categories: startup, restart, failure, shutdown
  Terminal window
```
# Old query
service_transition_seconds{from="starting",to="running"}

# New query
service_transition_seconds{type="startup"}
```
- LDAP Search Metrics
  
  Before: Histogram with 50+ buckets
```
channel_ldap_search_time_seconds_bucket{...,le="0.001"} 0
channel_ldap_search_time_seconds_bucket{...,le="0.002"} 1
[... 50+ buckets ...]
```
  After: Summary with optimized percentiles
```
channel_ldap_search_time_seconds{...,quantile="0.5"} 0.043
channel_ldap_search_time_seconds{...,quantile="0.95"} 0.051
channel_ldap_search_time_seconds{...,quantile="0.99"} 0.052
```
  Changes:
  - Format: Histogram → Summary
  - Percentiles: Reduced to 0.5, 0.95, 0.99 (high cardinality optimization)
  - Buckets: Removed all histogram buckets (50+ → 0)
  Migration:
  - Replace histogram_quantile() with direct quantile label access
  - Update SLI/SLO calculations to use summary percentiles
  Terminal window
```
# Old query
histogram_quantile(0.95, rate(channel_ldap_search_time_seconds_bucket[5m]))

# New query
channel_ldap_search_time_seconds{quantile="0.95"}
```
- Processing Time Metrics
  
  Before: Histogram with SLO buckets
```
scribe_processing_time_seconds_bucket{entryType="user",phase="diffing",le="0.001"} 5
scribe_processing_time_seconds_bucket{entryType="user",phase="diffing",le="0.002"} 25
[... many buckets ...]
```
  After: Summary with full percentiles
```
scribe_processing_time_seconds{entryType="user",phase="diffing",quantile="0.5"} 0.004
scribe_processing_time_seconds{entryType="user",phase="diffing",quantile="0.75"} 0.007
scribe_processing_time_seconds{entryType="user",phase="diffing",quantile="0.9"} 0.015
scribe_processing_time_seconds{entryType="user",phase="diffing",quantile="0.95"} 0.050
scribe_processing_time_seconds{entryType="user",phase="diffing",quantile="0.99"} 0.487
```
  Changes:
  - Format: Histogram → Summary
  - Percentiles: Added 0.5, 0.75, 0.9, 0.95, 0.99 (standard set)
  - Buckets: Removed all histogram buckets
  Migration:
  - Update queries to use quantile labels instead of histogram_quantile()
  - SLIs can now use direct percentile values
  Terminal window
```
# Old query
histogram_quantile(0.90, rate(scribe_processing_time_seconds_bucket[5m]))

# New query
scribe_processing_time_seconds{quantile="0.9"}
```
- New Entry Codec Metrics
  
  Added new compression and entropy metrics:
```
scribe_entry_encode_bytes{codec="lz4|zstd|none",kind="raw|compressed",quantile="0.5"} 1440
scribe_entry_decode_bytes{codec="lz4|zstd|none",kind="raw|compressed",quantile="0.5"} 1440
scribe_entry_encode_compression{codec="lz4|zstd",le="5.0"} 4
scribe_entry_encode_entropy{le="25.0"} 137598
```
  Features:
  - Encode/Decode sizes: Track compression effectiveness by codec
  - Compression percentage: Business-critical buckets at 85% and 95% thresholds
  - Entropy analysis: Optimized buckets for compression decision making
  - Percentiles: Standard set 0.5, 0.75, 0.9, 0.95, 0.99