Database

PostgreSQL database connection and pool configuration.

Collation guidance:

• Recommended (multi-language prod): ICU collation “und-x-icu” for consistent

ordering across platforms. Use TEMPLATE template0 to guarantee your collation/ctype apply (template1 may carry different locale or extensions).

CREATE DATABASE "identity_scribe"
  TEMPLATE template0
  ENCODING = 'UTF8'
  LC_COLLATE = 'und-x-icu'
  LC_CTYPE   = 'und-x-icu'
  CONNECTION LIMIT = -1;

• Performance-focused / ASCII-only prod: choose “C” for maximum speed; keep

it consistent across all environments.

CREATE DATABASE "identity_scribe"
  TEMPLATE template0
  ENCODING = 'UTF8'
  LC_COLLATE = 'C'
  LC_CTYPE   = 'C'
  CONNECTION LIMIT = -1;

Connection pool architecture

Identity Scribe uses three separate connection pools to isolate different workloads:

• Batch pool - Used by: Transcription tasks (main workload) - Size: Configured via maxPoolSize (default: concurrency + 5, max: concurrency * 1.5) - Each transcription task uses one connection for its lifetime

• System pool - Used by: System-level operations (commit coordination, migrations, maintenance, entry preparation) - Size: Configurable via systemPoolSize (optional) - Default: max(transcribeCount + 4, concurrency / 4), clamped between 2 and maxPoolSize / 2 - Scales with both number of transcribes and concurrency level - Isolates system operations from transcription tasks

• Channel pool - Used by: LDAP channel services (IdentityHub, LDAP channels) - Size: Configurable via channelPoolSize (optional) - Default: concurrency, minimum 2 - Can experience heavy LDAP traffic with multiple channels - Isolates channel operations from batch and commit workloads

Total maximum connections (when all pools are fully utilized): = Batch Pool + System Pool + Channel Pool = maxPoolSize + systemPoolSize + channelPoolSize

Example with default settings (concurrency = 16, 2 transcribes):

• Batch Pool: 16 + 5 = 21 connections (default maxPoolSize)
• System Pool: max(2 + 4, 16/4) = max(6, 4) = 6 connections
• Channel Pool: 16 connections
• Total Maximum: 21 + 6 + 16 = 43 connections

Example with maxPoolSize explicitly set to 50 (concurrency = 16, 2 transcribes):

• Batch Pool: 50 connections
• System Pool: max(2 + 4, 16/4) = 6 connections (capped at 50/2 = 25)
• Channel Pool: 16 connections
• Total Maximum: 50 + 6 + 16 = 72 connections

Tuning guidelines:

• Monitor pool metrics: Track hikari.connections.active, hikari.connections.idle,

and hikari.connections.pending for each pool

• Signs of under-sizing: If pending connections are consistently high or active equals

maximum for extended periods, consider increasing pool size

• Signs of over-sizing: If idle connections consistently exceed 50% of maximum,

consider reducing pool size to free database resources

• High concurrency workloads: Many more tasks can be queued than

concurrency, but each task still uses one connection. Monitor actual connection utilization rather than task count.

Size of the channel connection pool. This pool handles LDAP channel operations (IdentityHub, LDAP channels).

Default: concurrency

Minimum: 2

Channels can experience heavy LDAP traffic, especially with multiple active channels. Increase for high-throughput read-heavy workloads.

Example: With concurrency = 16:

Default = 16 connections

Priority: SCRIBE_DATABASE_CHANNEL_POOL_SIZE > config

database.channel-pool-size = ${?SCRIBE_DATABASE_CHANNEL_POOL_SIZE}

Connection Hints (channel query defaults)

Session-level hints applied to channel queries (REST, LDAP, GraphQL, gRPC). All keys are unset by default (PostgreSQL defaults apply). Channels can override these defaults in their own connection-hints section.

These hints control PostgreSQL session parameters for query execution. They are applied when opening a connection and restored when the query completes.

Lock acquisition timeout.

PostgreSQL will abort any statement that waits longer than this for a lock. Format: HOCON duration (e.g., 5s, 30s, 1m)

Priority: SCRIBE_DATABASE_LOCK_TIMEOUT > config

database.connection-hints.lock-timeout = ${?SCRIBE_DATABASE_LOCK_TIMEOUT}

PostgreSQL session flags (GUCs) applied to every query connection. Shape: typed object (changed in v3.0). Each key maps to a PostgreSQL GUC. Omitted keys fall back to the recommended default for the current PostgreSQL version.

These values are applied automatically on all query connections. Setting them server-level in postgresql.conf avoids per-connection overhead; the keys here are exposed for override/debugging only.

Fast-start bias for cursor execution.

database.connection-hints.session-flags.cursor-tuple-fraction = ${?SCRIBE_DATABASE_SESSION_FLAGS_CURSOR_TUPLE_FRACTION}

Effective I/O concurrency. Accepts ssd (=128), hdd (=2), or a raw integer.

database.connection-hints.session-flags.effective-io-concurrency = ${?SCRIBE_DATABASE_SESSION_FLAGS_EFFECTIVE_IO_CONCURRENCY}

Pinned to PostgreSQL 17 default to prevent excessive memory use on cursor and sort workloads. Higher values can cause hash operations to spill to disk, degrading performance.

database.connection-hints.session-flags.hash-mem-multiplier = ${?SCRIBE_DATABASE_SESSION_FLAGS_HASH_MEM_MULTIPLIER}

PG JIT compilation.

database.connection-hints.session-flags.jit = ${?SCRIBE_DATABASE_SESSION_FLAGS_JIT}

Partition-wise aggregate planning.

database.connection-hints.session-flags.partitionwise-aggregate = ${?SCRIBE_DATABASE_SESSION_FLAGS_PARTITIONWISE_AGGREGATE}

Partition-wise join planning.

database.connection-hints.session-flags.partitionwise-join = ${?SCRIBE_DATABASE_SESSION_FLAGS_PARTITIONWISE_JOIN}

Controls how PostgreSQL caches query plans across prepared-statement reuse. Values: auto, force_custom_plan, force_generic_plan.

database.connection-hints.session-flags.plan-cache-mode = ${?SCRIBE_DATABASE_SESSION_FLAGS_PLAN_CACHE_MODE}

Random page cost. Accepts ssd (=1.1), hdd (=4.0), or a raw number.

database.connection-hints.session-flags.random-page-cost = ${?SCRIBE_DATABASE_SESSION_FLAGS_RANDOM_PAGE_COST}

PostgreSQL memory size for sort/hash operations.

Units: B, kB, MB, GB, TB.

database.connection-hints.session-flags.work-mem = ${?SCRIBE_DATABASE_SESSION_FLAGS_WORK_MEM}

Statement execution timeout (prevents runaway queries). PostgreSQL will abort any statement that takes longer than this duration. Format: HOCON duration (e.g., 30s, 1m, 500ms)

Priority: SCRIBE_DATABASE_STATEMENT_TIMEOUT > config

database.connection-hints.statement-timeout = ${?SCRIBE_DATABASE_STATEMENT_TIMEOUT}

Allows conservative estimates for predicates without exact row-count support.

database.continuation.eligibility.allow-conservative-estimates = true

Requires at least one selective predicate before continuation safeguards can engage.

database.continuation.eligibility.require-selective-predicate = true

Enables additional safety limits for selected paginated queries. When enabled, selected paginated queries keep per-query work within the configured limits while preserving normal cursor and size-limit semantics.

Priority: SCRIBE_DATABASE_CONTINUATION_ENABLED > config

database.continuation.enabled = ${?SCRIBE_DATABASE_CONTINUATION_ENABLED}

Sets safety limits for qualifying paginated queries. These values do not change public page-size semantics.

Absolute safety cap for qualifying paginated queries. Keep this at or above max-candidates; the default uses the same cap.

Priority: SCRIBE_DATABASE_CONTINUATION_WINDOW_ABSOLUTE_SCAN_CAP > config

database.continuation.window.absolute-scan-cap = ${?SCRIBE_DATABASE_CONTINUATION_WINDOW_ABSOLUTE_SCAN_CAP}

Maximum safety limit for qualifying paginated queries. The default is 32 times the minimum safety limit.

Priority: SCRIBE_DATABASE_CONTINUATION_WINDOW_MAX_CANDIDATES > config

database.continuation.window.max-candidates = ${?SCRIBE_DATABASE_CONTINUATION_WINDOW_MAX_CANDIDATES}

Minimum safety limit for qualifying paginated queries. The effective limit also honors the requested page size.

Priority: SCRIBE_DATABASE_CONTINUATION_WINDOW_MIN_CANDIDATES > config

database.continuation.window.min-candidates = ${?SCRIBE_DATABASE_CONTINUATION_WINDOW_MIN_CANDIDATES}

Minimum safety-limit multiplier relative to the requested page size.

Priority: SCRIBE_DATABASE_CONTINUATION_WINDOW_MIN_LIMIT_MULTIPLE > config

database.continuation.window.min-limit-multiple = ${?SCRIBE_DATABASE_CONTINUATION_WINDOW_MIN_LIMIT_MULTIPLE}

Safety factor used when setting limits for eligible queries.

Priority: SCRIBE_DATABASE_CONTINUATION_WINDOW_SAFETY_FACTOR > config

database.continuation.window.safety-factor = ${?SCRIBE_DATABASE_CONTINUATION_WINDOW_SAFETY_FACTOR}

Fraction of rows modified since the last ANALYZE that also triggers a background statistics refresh, regardless of age. A value of 0.01 means that 1 % row churn is enough to mark the cached statistics as stale.

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_CHURN_BUDGET_RATIO > config

database.filter-selectivity.churn-budget-ratio = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_CHURN_BUDGET_RATIO}

Controls how the planner handles queries with multiple equality conditions (e.g. &(sn=Schmidt)(givenName=Paul)). Higher values make the planner more willing to use an optimized strategy for such queries. Lower values make it more conservative, preferring a general-purpose strategy. If directory searches combining multiple equality filters are slow, increasing this value may help. If individual conditions each match many entries, decreasing it prevents the planner from choosing an expensive strategy.

Default: 2.0

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_GATE_CONJUNCTION_CROSSOVER > config

database.filter-selectivity.gate-conjunction-crossover = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_GATE_CONJUNCTION_CROSSOVER}

Absolute upper limit for the adaptive substring threshold. No matter how large the directory, the effective threshold never exceeds this value. Increase this to require longer patterns on very large directories before using the fast path. Decrease to cap the threshold on the largest entry types.

Default: 20

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_GATE_LIKE_ADAPTIVE_CEILING > config

database.filter-selectivity.gate-like-adaptive-ceiling = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_GATE_LIKE_ADAPTIVE_CEILING}

Minimum pattern length before a substring search can use the fast trigram-index path. The effective threshold adapts to directory size — larger directories require slightly longer patterns before using the fast path — but never exceeds gate-like-adaptive-ceiling. When the query has additional filter conditions beyond the substring search, the effective threshold is lowered, making it easier to use the fast path (since those extra filters can narrow results early).

Default: 4

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_GATE_LIKE_BASE_PATTERN_LENGTH > config

database.filter-selectivity.gate-like-base-pattern-length = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_GATE_LIKE_BASE_PATTERN_LENGTH}

Reference entry count for the adaptive substring threshold. The system compares each entry type’s actual row count against this reference to compute a scale factor. Larger directories get a proportionally higher threshold, which prevents very broad substring patterns from using the fast path. Decrease this value to make the threshold more sensitive to directory size. Increase it to reduce sensitivity. Useful when your directory is uniformly large across all entry types and you want a consistent threshold.

Default: 100000

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_GATE_LIKE_CARDINALITY_REFERENCE > config

database.filter-selectivity.gate-like-cardinality-reference = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_GATE_LIKE_CARDINALITY_REFERENCE}

When enabled (default), substring searches on the sort column use the dedicated trigram-index fast path. The planner can use an optimized plan shape that avoids per-row probes, which can significantly improve performance for selective substring searches on attributes used for sorting. Disable to force all substring searches on sort columns to use the standard processing path. Useful as a safety valve if the fast path causes unexpected behavior in your environment.

Default: true

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_GATE_LIKE_SORT_ATTR_ENABLED > config

database.filter-selectivity.gate-like-sort-attr-enabled = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_GATE_LIKE_SORT_ATTR_ENABLED}

Selectivity threshold for range filters on sortable attributes (for example, sn >= ‘M’). When a range condition is estimated to match fewer than this fraction of entries, the planner may use handling tuned for selective range queries. Range selectivity is estimated from PostgreSQL statistics refreshed during IdentityScribe maintenance. Monitor the metric scribe.directory.plan.sort_index_range_dispatch.total to verify selective range handling decisions. Increase this threshold to allow selective handling for more range queries. Decrease it to keep default handling unless a range query is highly selective. Set to 0.0 to disable selective range handling.

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_GATE_SORTABLE_RANGE_FRACTION > config

database.filter-selectivity.gate-sortable-range-fraction = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_GATE_SORTABLE_RANGE_FRACTION}

Prefix search optimisation — measures actual match counts for new prefix patterns before choosing an execution strategy. Results are cached so only the first search for a new prefix pattern requires a measurement; all subsequent searches use the cached result.

Maximum number of measurement results to keep in cache. Each entry covers one (entry type, attribute, prefix) combination. Most deployments have far fewer than 1000 distinct prefix patterns in regular use. Increase if you serve a large number of distinct search prefixes and observe frequent cache churn.

Default: 1000

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_PROBE_CACHE_MAX_SIZE > config

database.filter-selectivity.probe.cache-max-size = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_PROBE_CACHE_MAX_SIZE}

How long a cached measurement result is considered valid. Cached results are also invalidated earlier when the underlying data changes significantly (controlled by stale-mod-ratio). Decrease if attribute distributions change frequently; increase to reduce measurement frequency.

Default: 1h

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_PROBE_CACHE_TTL > config

database.filter-selectivity.probe.cache-ttl = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_PROBE_CACHE_TTL}

Whether the prefix search optimisation is enabled. When enabled, the system measures how many entries match a prefix search before choosing an execution path. The measurement is cached and reused on all subsequent requests — only the first search for a previously unseen prefix pattern triggers a measurement run. Disable only if you need to force the conservative execution path for all prefix searches. Disabling requires a restart.

Default: true

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_PROBE_ENABLED > config

database.filter-selectivity.probe.enabled = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_PROBE_ENABLED}

How long to cache database statistics used to detect stale measurement results. Shorter values detect data changes faster but increase catalog query frequency.

Default: 1m

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_PROBE_PARTITION_STATS_CACHE_TTL > config

database.filter-selectivity.probe.partition-stats-cache-ttl = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_PROBE_PARTITION_STATS_CACHE_TTL}

How long to pause measurements for a pattern after repeated failures. After several consecutive failures for the same pattern, new measurement attempts are suspended until this window elapses. Raise if transient database errors cause premature suspension; lower for faster recovery after a database issue is resolved.

Default: 30s

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_PROBE_FAILURE_COOLDOWN > config

database.filter-selectivity.probe.probe-failure-cooldown = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_PROBE_FAILURE_COOLDOWN}

Maximum number of prefix pattern measurements that may run simultaneously. Measurements run using system pool connections. Decrease to limit connection pressure on small or shared database servers; increase on high-concurrency systems with spare system pool capacity.

Default: 4

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_PROBE_MAX_CONCURRENT > config

database.filter-selectivity.probe.probe-max-concurrent = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_PROBE_MAX_CONCURRENT}

Fraction of entries changed since the last statistics update that triggers an early cache refresh for affected prefix patterns. A value of 0.10 means 10% row churn invalidates the cached measurement and triggers a fresh one on the next request. Decrease to refresh more aggressively after bulk imports; increase to tolerate more churn before re-measuring.

Default: 0.10 (10 % row churn)

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_PROBE_STALE_MOD_RATIO > config

database.filter-selectivity.probe.stale-mod-ratio = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_PROBE_STALE_MOD_RATIO}

Maximum time allowed for a single prefix pattern measurement. If the measurement does not complete within this time, the system falls back to the conservative execution path for that request. Raise only if you see frequent measurement timeouts on large or heavily loaded databases.

Default: 1s

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_PROBE_TIMEOUT > config

database.filter-selectivity.probe.timeout = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_PROBE_TIMEOUT}

Maximum time allowed for a background statistics query. If the query does not complete within this budget the server uses conservative handling for that request and retries on the next query.

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_REFRESH_TIMEOUT > config

database.filter-selectivity.refresh-timeout = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_REFRESH_TIMEOUT}

Age after which cached attribute statistics are considered stale and refreshed in the background. Stale statistics are still served immediately while the refresh runs asynchronously.

Priority: SCRIBE_DATABASE_FILTER_SELECTIVITY_STALE_AGE_HORIZON > config

database.filter-selectivity.stale-age-horizon = ${?SCRIBE_DATABASE_FILTER_SELECTIVITY_STALE_AGE_HORIZON}

Self-tuning substring-search learning. Per-attribute thresholds are learned from live traffic and updated continuously at runtime. Set enabled = false to fall back to fully static behaviour with the defaults below. See /docs/operations/database-tuning/ for tuning guidance.

How long match-count tier boundaries are reused before being refreshed from table statistics. Boundaries are also refreshed on large table-size changes.

Default: 24h

Priority: SCRIBE_DATABASE_GATE_LEARNING_BUCKET_BOUNDARIES_FROZEN_AFTER > config

database.gate-learning.bucket.boundaries-frozen-after = ${?SCRIBE_DATABASE_GATE_LEARNING_BUCKET_BOUNDARIES_FROZEN_AFTER}

Maximum number of tracked attribute-query combinations retained in memory. Least-recently-used entries are evicted when the cap is reached.

Default: 10000

Priority: SCRIBE_DATABASE_GATE_LEARNING_BUCKET_LRU_CAP > config

database.gate-learning.bucket.lru-cap = ${?SCRIBE_DATABASE_GATE_LEARNING_BUCKET_LRU_CAP}

Match-count tier boundaries that partition observed result sizes into learning tiers. Must contain exactly 6 values in ascending order; larger values widen each tier.

Default: [0.1, 0.3, 1.0, 3.0, 10.0, 30.0]

Priority: SCRIBE_DATABASE_GATE_LEARNING_BUCKET_MATCH_COUNT_MULTIPLIER_DEFAULTS > config

match-count-multiplier-defaults = [0.1, 0.3, 1.0, 3.0, 10.0, 30.0]

database.gate-learning.bucket.match-count-multiplier-defaults = ${?SCRIBE_DATABASE_GATE_LEARNING_BUCKET_MATCH_COUNT_MULTIPLIER_DEFAULTS}

Row-count tier boundaries used during cold start, before table statistics are available. Values represent table sizes (in rows) that separate tiers.

Default: [100000, 1000000, 10000000]

Priority: SCRIBE_DATABASE_GATE_LEARNING_BUCKET_N_BOUNDARIES_DEFAULT > config

n-boundaries-default = [100000, 1000000, 10000000]

database.gate-learning.bucket.n-boundaries-default = ${?SCRIBE_DATABASE_GATE_LEARNING_BUCKET_N_BOUNDARIES_DEFAULT}

Fraction of rows modified since the last statistics refresh that marks an entry’s match-count tiers as stale and triggers a refresh.

Default: 0.10

Priority: SCRIBE_DATABASE_GATE_LEARNING_BUCKET_STALE_MOD_RATIO > config

database.gate-learning.bucket.stale-mod-ratio = ${?SCRIBE_DATABASE_GATE_LEARNING_BUCKET_STALE_MOD_RATIO}

Time after which an idle entry (no new observations) is evicted from memory. Persisted learned state is retained until normal maintenance pruning removes stale rows.

Default: 7d

Priority: SCRIBE_DATABASE_GATE_LEARNING_BUCKET_TTL > config

database.gate-learning.bucket.ttl = ${?SCRIBE_DATABASE_GATE_LEARNING_BUCKET_TTL}

Observation count needed to advance from COLD to LOW confidence. Raise if cold-start decision changes are too frequent; lower to learn faster.

Default: 50

Priority: SCRIBE_DATABASE_GATE_LEARNING_CONFIDENCE_COLD_THRESHOLD > config

database.gate-learning.confidence.cold-threshold = ${?SCRIBE_DATABASE_GATE_LEARNING_CONFIDENCE_COLD_THRESHOLD}

Number of consecutive refresh cycles with a stable learned threshold required before an entry is treated as HIGH confidence.

Default: 5

Priority: SCRIBE_DATABASE_GATE_LEARNING_CONFIDENCE_HIGH_MIN_STABLE_CYCLES > config

database.gate-learning.confidence.high-min-stable-cycles = ${?SCRIBE_DATABASE_GATE_LEARNING_CONFIDENCE_HIGH_MIN_STABLE_CYCLES}

Maximum relative drift allowed while an entry stays at HIGH confidence. Lower values make HIGH confidence harder to retain.

Default: 0.30

Priority: SCRIBE_DATABASE_GATE_LEARNING_CONFIDENCE_HIGH_RESET_THRESHOLD > config

database.gate-learning.confidence.high-reset-threshold = ${?SCRIBE_DATABASE_GATE_LEARNING_CONFIDENCE_HIGH_RESET_THRESHOLD}

Observation count needed to reach HIGH confidence.

Default: 500

Priority: SCRIBE_DATABASE_GATE_LEARNING_CONFIDENCE_HIGH_THRESHOLD > config

database.gate-learning.confidence.high-threshold = ${?SCRIBE_DATABASE_GATE_LEARNING_CONFIDENCE_HIGH_THRESHOLD}

Observation count needed to advance from LOW to MEDIUM confidence.

Default: 200

Priority: SCRIBE_DATABASE_GATE_LEARNING_CONFIDENCE_LOW_THRESHOLD > config

database.gate-learning.confidence.low-threshold = ${?SCRIBE_DATABASE_GATE_LEARNING_CONFIDENCE_LOW_THRESHOLD}

Time after which an entry with no new observations is reset to COLD. Tighten if your traffic patterns shift frequently; loosen for more stable workloads.

Default: 7d

Priority: SCRIBE_DATABASE_GATE_LEARNING_CONFIDENCE_SAMPLE_STALE_AFTER > config

database.gate-learning.confidence.sample-stale-after = ${?SCRIBE_DATABASE_GATE_LEARNING_CONFIDENCE_SAMPLE_STALE_AFTER}

Static cold-start threshold for same-attribute OR queries that mix substring and exact-match arms.

Default: 3.0

Priority: SCRIBE_DATABASE_GATE_LEARNING_DEFAULTS_SAME_ATTRIBUTE_MIXED_OR > config

database.gate-learning.defaults.same-attribute-mixed-or = ${?SCRIBE_DATABASE_GATE_LEARNING_DEFAULTS_SAME_ATTRIBUTE_MIXED_OR}

Static cold-start threshold for single substring or prefix queries.

Default: 6.0

Priority: SCRIBE_DATABASE_GATE_LEARNING_DEFAULTS_SINGLE_SUBSTRING > config

database.gate-learning.defaults.single-substring = ${?SCRIBE_DATABASE_GATE_LEARNING_DEFAULTS_SINGLE_SUBSTRING}

Static cold-start threshold for OR queries with multiple substring or prefix arms.

Default: 15.0

Priority: SCRIBE_DATABASE_GATE_LEARNING_DEFAULTS_SUBSTRING_OR > config

database.gate-learning.defaults.substring-or = ${?SCRIBE_DATABASE_GATE_LEARNING_DEFAULTS_SUBSTRING_OR}

Whether self-tuning is active. When false, static defaults are used for substring-search decisions. No background measurement queries run and no learned state is persisted.

Default: true

Priority: SCRIBE_DATABASE_GATE_LEARNING_ENABLED > config

database.gate-learning.enabled = ${?SCRIBE_DATABASE_GATE_LEARNING_ENABLED}

Alert threshold: percentage of decisions per hour that later appear to have chosen the slower path.

Default: 2.0

Priority: SCRIBE_DATABASE_GATE_LEARNING_GUARDRAIL_MISROUTE_REGRET_PCT > config

database.gate-learning.guardrail.misroute-regret-pct = ${?SCRIBE_DATABASE_GATE_LEARNING_GUARDRAIL_MISROUTE_REGRET_PCT}

Alert threshold: percentage of decisions per hour that ignore the learned value because it drifted too far from the static default.

Default: 5.0

Priority: SCRIBE_DATABASE_GATE_LEARNING_GUARDRAIL_OUTSIDE_DELTA_BAND_PCT > config

database.gate-learning.guardrail.outside-delta-band-pct = ${?SCRIBE_DATABASE_GATE_LEARNING_GUARDRAIL_OUTSIDE_DELTA_BAND_PCT}

How often learned threshold state is flushed to the database so it survives restarts.

Default: 5m

Priority: SCRIBE_DATABASE_GATE_LEARNING_PERSISTENCE_FLUSH_INTERVAL > config

database.gate-learning.persistence.flush-interval = ${?SCRIBE_DATABASE_GATE_LEARNING_PERSISTENCE_FLUSH_INTERVAL}

Maximum number of persisted-state rows written per second during a flush cycle. Limits write amplification on busy databases.

Default: 500

Priority: SCRIBE_DATABASE_GATE_LEARNING_PERSISTENCE_FLUSH_RATE_LIMIT > config

database.gate-learning.persistence.flush-rate-limit = ${?SCRIBE_DATABASE_GATE_LEARNING_PERSISTENCE_FLUSH_RATE_LIMIT}

Minimum confidence tier required before a learned threshold is applied. Entries at COLD or LOW confidence always fall back to the static default.

Valid values: COLD, LOW, MEDIUM, HIGH.

Default: “MEDIUM”

Priority: SCRIBE_DATABASE_GATE_LEARNING_PREDICATES_CONFIDENCE_GATE > config

database.gate-learning.predicates.confidence-gate = ${?SCRIBE_DATABASE_GATE_LEARNING_PREDICATES_CONFIDENCE_GATE}

Maximum relative deviation between the learned threshold and the static default before the system falls back to the static default. Prevents runaway learning.

Default: 0.5 (50 %)

Priority: SCRIBE_DATABASE_GATE_LEARNING_PREDICATES_DELTA_BAND > config

database.gate-learning.predicates.delta-band = ${?SCRIBE_DATABASE_GATE_LEARNING_PREDICATES_DELTA_BAND}

Minimum measured improvement required before IdentityScribe can enable an automatically learned faster range-search path. Until enough evidence is available, range searches use the standard safe path.

Default: 0.20 (20 %)

Priority: SCRIBE_DATABASE_GATE_LEARNING_PREDICATES_RANGE_MIN_IMPROVEMENT > config

database.gate-learning.predicates.range-min-improvement = ${?SCRIBE_DATABASE_GATE_LEARNING_PREDICATES_RANGE_MIN_IMPROVEMENT}

Minimum number of recent matching range searches required before IdentityScribe can enable an automatically learned faster path. Until enough evidence is available, range searches use the standard safe path.

Default: 3

Priority: SCRIBE_DATABASE_GATE_LEARNING_PREDICATES_RANGE_MIN_OBSERVATIONS > config

database.gate-learning.predicates.range-min-observations = ${?SCRIBE_DATABASE_GATE_LEARNING_PREDICATES_RANGE_MIN_OBSERVATIONS}

Fraction of recent queries that must show suboptimal decisions before reverting to default.

Default: 0.25 (25 %)

Priority: SCRIBE_DATABASE_GATE_LEARNING_PREDICATES_REGRET_RATE_THRESHOLD > config

database.gate-learning.predicates.regret-rate-threshold = ${?SCRIBE_DATABASE_GATE_LEARNING_PREDICATES_REGRET_RATE_THRESHOLD}

Minimum relative change in the learned threshold required before behaviour changes. Suppresses churn from small incremental adjustments.

Default: 0.15 (15 %)

Priority: SCRIBE_DATABASE_GATE_LEARNING_PREDICATES_STABILITY_BAND > config

database.gate-learning.predicates.stability-band = ${?SCRIBE_DATABASE_GATE_LEARNING_PREDICATES_STABILITY_BAND}

Reserved burst detection window. Parsed today for forward compatibility; current releases do not yet enable burst mode.

Default: 60s

Priority: SCRIBE_DATABASE_GATE_LEARNING_PROBE_BURST_DETECTION_WINDOW > config

database.gate-learning.probe.burst.detection-window = ${?SCRIBE_DATABASE_GATE_LEARNING_PROBE_BURST_DETECTION_WINDOW}

Reserved burst sustain window. Parsed today for forward compatibility; current releases do not yet enable burst mode.

Default: 5m

Priority: SCRIBE_DATABASE_GATE_LEARNING_PROBE_BURST_SUSTAIN_WINDOW > config

database.gate-learning.probe.burst.sustain-window = ${?SCRIBE_DATABASE_GATE_LEARNING_PROBE_BURST_SUSTAIN_WINDOW}

Reserved burst trigger threshold. Parsed today for forward compatibility; current releases do not yet enable burst mode.

Default: 10

Priority: SCRIBE_DATABASE_GATE_LEARNING_PROBE_BURST_THRESHOLD_BUCKETS > config

database.gate-learning.probe.burst.threshold-buckets = ${?SCRIBE_DATABASE_GATE_LEARNING_PROBE_BURST_THRESHOLD_BUCKETS}

Probe interval for entries at COLD confidence (recently observed; not yet stable).

Default: 30s

Priority: SCRIBE_DATABASE_GATE_LEARNING_PROBE_CADENCE_COLD > config

database.gate-learning.probe.cadence.cold = ${?SCRIBE_DATABASE_GATE_LEARNING_PROBE_CADENCE_COLD}

Probe interval for entries at HIGH confidence.

Default: 1h

Priority: SCRIBE_DATABASE_GATE_LEARNING_PROBE_CADENCE_HIGH > config

database.gate-learning.probe.cadence.high = ${?SCRIBE_DATABASE_GATE_LEARNING_PROBE_CADENCE_HIGH}

Probe interval for entries at LOW confidence.

Default: 3m

Priority: SCRIBE_DATABASE_GATE_LEARNING_PROBE_CADENCE_LOW > config

database.gate-learning.probe.cadence.low = ${?SCRIBE_DATABASE_GATE_LEARNING_PROBE_CADENCE_LOW}

Probe interval for entries at MEDIUM confidence.

Default: 15m

Priority: SCRIBE_DATABASE_GATE_LEARNING_PROBE_CADENCE_MEDIUM > config

database.gate-learning.probe.cadence.medium = ${?SCRIBE_DATABASE_GATE_LEARNING_PROBE_CADENCE_MEDIUM}

Maximum number of concurrent background measurement queries used for learning.

Default: 4

Priority: SCRIBE_DATABASE_GATE_LEARNING_PROBE_MAX_CONCURRENT > config

database.gate-learning.probe.max-concurrent = ${?SCRIBE_DATABASE_GATE_LEARNING_PROBE_MAX_CONCURRENT}

Reserved for a future burst scheduler that temporarily raises probe concurrency when many new attributes appear at once. Parsed today for forward compatibility; current releases still use max-concurrent as the active limit. Must be >= max-concurrent.

Default: 8

Priority: SCRIBE_DATABASE_GATE_LEARNING_PROBE_MAX_CONCURRENT_BURST > config

database.gate-learning.probe.max-concurrent-burst = ${?SCRIBE_DATABASE_GATE_LEARNING_PROBE_MAX_CONCURRENT_BURST}

How long table statistics are cached per (entry_type, attribute) before refreshing.

Default: 1h

Priority: SCRIBE_DATABASE_GATE_LEARNING_PROBE_PG_STATS_CACHE_TTL > config

database.gate-learning.probe.pg-stats-cache-ttl = ${?SCRIBE_DATABASE_GATE_LEARNING_PROBE_PG_STATS_CACHE_TTL}

Per-probe query timeout. Timed-out samples are ignored or down-weighted so the system can fall back to the static default. Must be between 1s and 10s.

Default: 5s

Priority: SCRIBE_DATABASE_GATE_LEARNING_PROBE_TIMEOUT > config

database.gate-learning.probe.timeout = ${?SCRIBE_DATABASE_GATE_LEARNING_PROBE_TIMEOUT}

Penalty value recorded for a timed-out measurement query, as a multiple of probe.timeout. Higher values make timeouts push the system back toward static defaults more aggressively.

Default: 2.0

Priority: SCRIBE_DATABASE_GATE_LEARNING_PROBE_TIMEOUT_PENALTY_MULTIPLIER > config

database.gate-learning.probe.timeout-penalty-multiplier = ${?SCRIBE_DATABASE_GATE_LEARNING_PROBE_TIMEOUT_PENALTY_MULTIPLIER}

Maintenance scheduling for tasks like vacuuming, re-indexing, etc.

Define a cron expression (unix cron format) to run the maintenance task at a specific time Default not set, eg no cron window

Priority: SCRIBE_DATABASE_MAINTENANCE_CRON > config

cron = ‘0 5 * * *’ # 5am daily

database.maintenance.cron = ${?SCRIBE_DATABASE_MAINTENANCE_CRON}

If enabled, maintenance tasks will be scheduled Defaults to enabled unless readonly is set to true

Priority: SCRIBE_DATABASE_MAINTENANCE_ENABLED > config

database.maintenance.enabled = ${?SCRIBE_DATABASE_MAINTENANCE_ENABLED}

Used after interval, to check if system is idle enough to run maintenance Only a well-known set of threshold metrics is supported (no labels/selectors):

• scribe_ingest_tasks_active
• scribe_ingest_task_pressure
• scribe_ingest_queue_pressure
• scribe_db_connections_active
• scribe_ldap_connections_active
• scribe_query_permit_pressure

Arbitrary metric names are NOT supported. These values are queried directly from internal metrics (not Prometheus).

Priority: SCRIBE_DATABASE_MAINTENANCE_HARD_THRESHOLDS > config

Defaults to:

hard-thresholds = {
scribe_db_connections_active = Math.ceil(Math.pow(Math.log(max-pool-size + channel-pool-size + system-pool-size), 2)) # DB pool load, eg many connections are active
}

database.maintenance.hard-thresholds = ${?SCRIBE_DATABASE_MAINTENANCE_HARD_THRESHOLDS}

Runs based on system load metrics, at least every 36 hours The default value for interval is 36 hours, if no interval or cron is set

Priority: SCRIBE_DATABASE_MAINTENANCE_INTERVAL > config

database.maintenance.interval = ${?SCRIBE_DATABASE_MAINTENANCE_INTERVAL}

Used after half of interval, to check if system is idle enough to run maintenance Only a well-known set of threshold metrics is supported (no labels/selectors):

• scribe_ingest_tasks_active
• scribe_ingest_task_pressure
• scribe_ingest_queue_pressure
• scribe_db_connections_active
• scribe_ldap_connections_active
• scribe_query_permit_pressure

Arbitrary metric names are NOT supported. These values are queried directly from internal metrics (not Prometheus).

Priority: SCRIBE_DATABASE_MAINTENANCE_SOFT_THRESHOLDS > config

Defaults to:

soft-thresholds = {
scribe_db_connections_active = Math.ceil(Math.log(max-pool-size + channel-pool-size + system-pool-size)) # DB pool load, eg not many connections are active
}

database.maintenance.soft-thresholds = ${?SCRIBE_DATABASE_MAINTENANCE_SOFT_THRESHOLDS}

Ratio applied to the directory size estimate to dynamically compute the PostgreSQL statistics target before each maintenance refresh. Computation: target = max(100, min(10000, ceil(estimated_entries * ratio))) Set to 0 or leave absent to disable the dynamic target (skip SET entirely).

Default: 0.00005 (5e-5), which yields targets of ~100–1,000 for directories

with 2M–20M entries.

Priority: SCRIBE_DATABASE_MAINTENANCE_STATISTICS_TARGET_RATIO > config

database.maintenance.statistics-target-ratio = ${?SCRIBE_DATABASE_MAINTENANCE_STATISTICS_TARGET_RATIO}

Maximum size of the batch connection pool. This pool is used by transcription tasks - each task uses one connection for its lifetime.

Default: concurrency + 5

Minimum: 5

Maximum: concurrency * 1.5 (rounded up) When setting this value, remember that the total connection count includes:

• This batch pool (max-pool-size)
• System pool (configurable via system-pool-size, default: max(transcribeCount + 4, concurrency / 4))
• Channel pool (configurable via channel-pool-size, default: concurrency)

Example: With max-pool-size = 50, concurrency = 16, and 2 transcribes:

Total max connections = 50 (batch) + 6 (system) + 16 (channel) = 72 connections

Priority: SCRIBE_DATABASE_MAX_POOL_SIZE > config

database.max-pool-size = ${?SCRIBE_DATABASE_MAX_POOL_SIZE}

What to do when transcribe types are found in the database but not in the current config. Orphaned types have data in the entries table but no matching entry in the transcribes configuration block. Values: warn — (default) log a startup warning listing the orphaned types; continue normally. error — abort startup with an error message; requires cleanup or re-addition of the type. ignore — silently skip the check; preserves the pre-3.x behavior.

Priority: SCRIBE_DATABASE_ORPHANED_TRANSCRIBES_BEHAVIOR > config

database.orphaned-transcribes-behavior = ${?SCRIBE_DATABASE_ORPHANED_TRANSCRIBES_BEHAVIOR}

Priority: SCRIBE_DATABASE_PASSWORD > config

database.password = ${?SCRIBE_DATABASE_PASSWORD}

Prepared Statement Caching (advanced tuning)

Controls server-side prepared statements and per-connection caching. These improve performance by reusing query plans across executions. Memory footprint: poolSize * prepared-statement-cache-size

Example: 35 connections * 8 MiB = ~280 MiB

prepare-threshold: executions before server-side prepare (1 = immediate) prepared-statement-cache-queries: max cached statements per connection prepared-statement-cache-size: memory limit per connection (HOCON memory size syntax)

database.prepare-threshold = 1

database.prepared-statement-cache-queries = 256

database.prepared-statement-cache-size = 8 MiB

Query translation behavior

Controls how the planner emits SQL for query patterns. Distinct from connection-hints.session-flags (PostgreSQL session GUCs) above — these settings affect SQL shape, not session parameters.

Tiebreaker for sort orderings on long-value single-valued attributes. When the value_match index uses truncated(value) (200-char prefix), values sharing the same truncated prefix may tie. The tiebreaker controls how those ties are resolved.

• deterministic (default): order tied values by uoid only. RFC 2891 §2 compliant

(server may return a deterministic order; spec is silent on tied-bucket alphabetical ordering). Skips heap fetches; ~2× faster on large corpora than alphabetical (lower bound — long-value buckets may show a higher cliff).

• alphabetical: order tied values by full alphabetical value, then uoid. Adds a

heap fetch per tied bucket. Use when byte-exact alphabetical ordering on long-value attributes (≥200 chars, e.g. description) is required. Multi-valued sort orderings always materialize the full value via aggregate subquery and are unaffected by this setting.

Priority: SCRIBE_DATABASE_QUERY_SORT_LONG_VALUE_TIEBREAKER > config

database.query.sort.long-value-tiebreaker = ${?SCRIBE_DATABASE_QUERY_SORT_LONG_VALUE_TIEBREAKER}

Query Connection Limiter

Timeout for HTTP/GraphQL queries when the system is busy. When all connections are in use, HTTP and GraphQL queries wait up to this duration before returning 503 Service Unavailable with a Retry-After header. LDAP queries block up to their query time limit instead.

Default: 5 seconds

Priority: SCRIBE_DATABASE_QUERY_HTTP_ACQUISITION_TIMEOUT > config

database.query-http-acquisition-timeout = ${?SCRIBE_DATABASE_QUERY_HTTP_ACQUISITION_TIMEOUT}

Retry policy for acquiring database connections. Unset fields inherit from the root retry block.

Initial delay between retries while waiting for database connections

Priority: SCRIBE_DATABASE_RETRY_INITIAL_DELAY > config

database.retry.initial-delay = ${?SCRIBE_DATABASE_RETRY_INITIAL_DELAY}

Randomized jitter added to each delay to avoid thundering herds

Priority: SCRIBE_DATABASE_RETRY_JITTER > config

database.retry.jitter = ${?SCRIBE_DATABASE_RETRY_JITTER}

Set to >0 to cap retries by attempt count instead of duration (0 = unlimited)

Priority: SCRIBE_DATABASE_RETRY_MAX_ATTEMPTS > config

database.retry.max-attempts = ${?SCRIBE_DATABASE_RETRY_MAX_ATTEMPTS}

Maximum delay between retries

Priority: SCRIBE_DATABASE_RETRY_MAX_DELAY > config

database.retry.max-delay = ${?SCRIBE_DATABASE_RETRY_MAX_DELAY}

Maximum time spent retrying before surfacing a timeout (leave unset to retry indefinitely)

Priority: SCRIBE_DATABASE_RETRY_MAX_DURATION > config

database.retry.max-duration = ${?SCRIBE_DATABASE_RETRY_MAX_DURATION}

Sort-index table backfill.

Controls how many sortable attribute sort-index tables are backfilled in parallel at startup. Backfill runs transparently in the background; the service remains fully operational during the backfill window. Cursor pagination on affected attributes falls back to the standard pipeline (correct, no speedup) until backfill completes.

Default: auto-scaled from the system connection pool size using max(1, min(4, systemPoolSize - 2)).

This reserves at least 2 system pool connections for maintenance, health checks, and DDL. On tiny deployments where systemPoolSize < 3, the auto-scaled default is 1. Ceiling: 4 (values above 4 are clamped at startup with a WARN). Explicitly configured values above systemPoolSize - 2 are also clamped with a WARN. Increase to 4 on high-spec systems to shorten the backfill window. Decrease to 1 to reduce backfill read pressure on the database during initial sync.

Priority: SCRIBE_DATABASE_SORT_INDEX_BACKFILL_MAX_CONCURRENT > config > auto-scaled default

Override: sort-index-backfill.max-concurrent = <1..4> (defaults to auto-scale formula above)

database.sort-index-backfill.max-concurrent = ${?SCRIBE_DATABASE_SORT_INDEX_BACKFILL_MAX_CONCURRENT}

Readiness verification timeout.

Bounds the final readiness check after a sort-index table backfill attempt. Backfill batches keep the shorter maintenance statement timeout; this longer bound is for large partitions where readiness verification can legitimately take longer. If verification times out, the sort index remains out of query planning, health details report it as degraded, and the background worker retries with backoff.

Default: 10 minutes

Priority: SCRIBE_DATABASE_SORT_INDEX_BACKFILL_READINESS_TIMEOUT > config > 10m

database.sort-index-backfill.readiness-timeout = ${?SCRIBE_DATABASE_SORT_INDEX_BACKFILL_READINESS_TIMEOUT}

defaults to the root ssl configuration

The location of the root certificate for authenticating the server. File containing the root certificate when validating server (mode = “verify-ca” or “verify-full”). Default will be the file “root.crt” in “$HOME/.postgresql” (*nix) or “%APPDATA%\postgresql” (windows).

Priority: SCRIBE_DATABASE_SSL_CA > config > ssl.ca

database.ssl.ca = ${?SCRIBE_DATABASE_SSL_CA}

The location of the client’s SSL certificate File containing the SSL Certificate. Default will be the file “postgresql.crt” in “$HOME/.postgresql” (*nix) or “%APPDATA%\postgresql” (windows).

Priority: SCRIBE_DATABASE_SSL_CERT > config > ssl.cert

database.ssl.cert = ${?SCRIBE_DATABASE_SSL_CERT}

Must be set to ‘true’ to enable SSL for the database

Priority: SCRIBE_DATABASE_SSL_ENABLED > config

database.ssl.enabled = ${?SCRIBE_DATABASE_SSL_ENABLED}

The location of the client’s PKCS#8 SSL key. File containing the SSL Key. Default will be the file “postgresql.pk8” in “$HOME/.postgresql” (*nix) or “%APPDATA%\postgresql” (windows).

Priority: SCRIBE_DATABASE_SSL_KEY > config > ssl.key

database.ssl.key = ${?SCRIBE_DATABASE_SSL_KEY}

The password for the client’s ssl key.

Priority: SCRIBE_DATABASE_SSL_PASSWORD > config > ssl.password

database.ssl.password = ${?SCRIBE_DATABASE_SSL_PASSWORD}

Size of the system connection pool. This pool handles commit coordination, migrations, maintenance, and entry preparation.

Default: max(transcribeCount + 4, concurrency / 4)

Minimum: 2

Maximum: max-pool-size / 2 The default scales with both the number of configured transcribes and concurrency level to handle workloads where multiple transcribes perform system operations simultaneously.

Example: With 4 transcribes and concurrency = 16:

Default = max(4 + 4, 16/4) = max(8, 4) = 8 connections

Priority: SCRIBE_DATABASE_SYSTEM_POOL_SIZE > config

database.system-pool-size = ${?SCRIBE_DATABASE_SYSTEM_POOL_SIZE}

https://www.prisma.io/dataguide/postgresql/short-guides/connection-uris https://jdbc.postgresql.org/documentation/use/#connecting-to-the-database Multiple servers can be specified, separated by a comma, eg “postgres://server1,server2” Options can be set in the URL or as separate properties

Priority: SCRIBE_DATABASE_URL > config

url = “postgres:”${user.name}

database.url = ${?SCRIBE_DATABASE_URL}

Priority: SCRIBE_DATABASE_USER > config

user = ${user.name}

database.user = ${?SCRIBE_DATABASE_USER}