> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ditto.live/llms.txt
> Use this file to discover all available pages before exploring further.
# Swift Release Notes
# Ditto SDK 5.1 — Faster Local Queries. More Capable Query Engine.
The new Ditto SDK 5.1 delivers major improvements across five areas. Read the top highlights below, and continue scrolling for the full release details including 77 platform and 112 SDK-specific improvements.
**[1. Performance: Faster Queries, Lower Memory Use](#1-performance-faster-queries-lower-memory-use)**
* Evictions are 53.6× faster and deletes are 43.0× faster on average.
* A full-collection `COUNT(*)` fell from 149.03 ms to 0.89 ms in Ditto's lab testing. This query is 167× faster on average.
**[2. Query Engine: New Ditto Query Capabilities](#2-query-engine-new-ditto-query-capabilities)**
* `JOIN` combines local collections in a single `SELECT`, removing the need to run separate queries and merge their results in application code.
* `ADVISE` recommends indexes for a query without running it or reading a document.
**[3. Security: Expanded Certificate Revocation](#3-security-expanded-certificate-revocation)**
* Revocations travel peer to peer and reach devices that were offline when the certificate was revoked.
* Enforcement is on by default: revoked peers are refused, and matching active connections are terminated.
**[4. Reliability: Production Diagnostics and Recovery](#4-reliability-production-diagnostics-and-recovery)**
* Support bundles carry `config_snapshot.json`, the effective configuration at capture time.
* Nine new counters under `ditto.network.dsoq.*` surface Ditto Sync over QUIC (DSOQ) protocol failures in production.
**[5. Transport at Scale: Multicast Beta](#5-transport-at-scale-multicast-beta)**
* Each update is published once to the mesh instead of once per peer, increasing the number of devices that can sync in a mesh.
* Opt-in beta capability in the Swift, Kotlin/KMP, Flutter, and Rust SDKs, with group encryption and peer-to-peer fallback. Support will expand to additional SDKs in the future.
**[Upgrading to 5.1 and rollback](#upgrading-to-5-1)**
* Upgrading to Ditto 5.1 changes the on-disk index format.
* If you need to downgrade from Ditto 5.1 or later, migrate through Ditto 5.0.2+. You can also migrate through Ditto 4.14.6+.
**Swift Specific Changes**
* [Swift Specific Changes](#platform-highlights)
**Full Changelog**
* [Full Changelog](#full-changelog)
***
# 1. Performance: Faster Queries, Lower Memory Use
Ditto 5.1 delivers a major leap in local query performance. Applications can load data faster, complete writes sooner, and react to changes more quickly—even as their local datasets and workloads grow.
The improvements span nearly every kind of local data operation: selects, indexed reads, inserts, updates, deletes, evictions, aggregations, and observers. In practice, this means more responsive user experiences, less time waiting for data operations, and greater capacity on the same device hardware.
These gains come from improvements throughout the local data path. Ditto performs less decoding and allocation, finds documents more directly, executes mutations more efficiently, avoids unnecessary observer work, and reduces full database scans during subscription changes. An opt-in relaxed durability mode can further reduce disk synchronization for rebuildable sync metadata without changing the durability of application documents.
## Android retail benchmark
The gains are broad rather than limited to one optimized query path. In testing on an Orion O6 Android device, Ditto 5.1 was faster in 70 of 71 measured scenarios, with one result too close to call. The 72-scenario suite modeled an offline-first retail application with approximately 93,000 documents across seven synced collections.
| DQL operation | Geometric-mean speedup |
| -------------- | ---------------------: |
| Aggregation | 4.2× |
| Delete | 43.0× |
| Evict | 53.6× |
| Indexed select | 1.4× |
| Insert | 1.2× |
| Select | 1.5× |
| Update | 18.5× |
These results compare median runtimes from Ditto 5.0.3 and Ditto 5.1.0. Performance varies by device, data, indexes, and query mix, so test representative workloads on your target hardware.
### Dramatically faster document counts
One of the largest individual improvements is full-collection document counting. Ditto 5.1.0 adds an optimized path for `COUNT(*)`, reducing the benchmark's median execution time from 149.03 ms to 0.89 ms—approximately **167× faster**. Counts with a filter also improved by approximately **4.4×**.
| Query | Ditto 5.0.3 | Ditto 5.1.0 | Speedup |
| ---------------------- | ----------: | ----------: | ------: |
| Full-collection count | 149.03 ms | 0.89 ms | 167× |
| Count with a condition | 60.85 ms | 13.89 ms | 4.4× |
These improvements accelerate queries such as `SELECT COUNT(*) FROM tasks` and `SELECT COUNT(*) FROM tasks WHERE status = 'open'`, making it substantially faster to calculate totals for dashboards, backlog checks, pagination, and application status displays.
## Lower memory use
In a separate Android workload that grew a collection from 3,000 to 30,000 documents, Ditto 5.1.0 delivered approximately 2.1× as many observer results while using less memory than Ditto 5.0.3.
| Memory measurement | Ditto 5.0.3 | Ditto 5.1.0 | Improvement |
| ------------------ | ----------: | ----------: | ----------: |
| Median Total PSS | 271.4 MB | 233.5 MB | 14.0% lower |
| Median native heap | 180.5 MB | 121.4 MB | 32.7% lower |
| Peak Total PSS | 469.5 MB | 400.6 MB | 14.7% lower |
Total PSS estimates the process's physical RAM footprint; native heap covers Ditto's Rust core. Tombstone cleanup, bulk mutations, and retained disconnected sync sessions are also bounded more carefully to reduce peak memory in high-volume deployments.
***
# 2. Query Engine: New Ditto Query Capabilities
## Join collections locally
Small Peer `SELECT` statements can now [join multiple local collections](/dql/select#joins). This removes the need to coordinate separate queries and merge their results in application code.
```sql DQL theme={null}
SELECT task._id, task.title, project.name
FROM tasks AS task
JOIN projects AS project ON task.projectId = project._id
WHERE task.status = 'open'
```
Joins use data already present in the local store. They do not fetch missing data from peers, and they are not supported in sync-subscription queries. The inner collection normally requires an appropriate index; use [`ADVISE`](/dql/advise) when you need an index recommendation.
## Create composite indexes
Small Peers now support [composite indexes](/dql/indexing#composite-index) over multiple fields. A single composite index can accelerate queries that repeatedly filter or sort by the same combination of fields—for example, tenant and time, status and assignee, or location and category.
The following index is designed for queries that filter tasks by `status` and sort them by `createdAt`:
```sql DQL theme={null}
CREATE INDEX IF NOT EXISTS status_created_idx
ON tasks (status, createdAt DESC)
```
It can improve queries such as:
```sql DQL theme={null}
SELECT * FROM tasks
WHERE status = 'open'
ORDER BY createdAt DESC
```
Field order matters. Put fields used in equality filters first, followed by fields used for range filters or sorting. Composite indexes can also include array and object values. If you are unsure which fields to index, run [`ADVISE`](/dql/advise) against the query to get an index recommendation.
## Find the right indexes with ADVISE
[`ADVISE`](/dql/advise) turns index optimization into a guided workflow. Prefix a query with `ADVISE` and Ditto analyzes its execution plan without running the query or reading any documents. The response explains why an index would help and provides a ready-to-run `CREATE INDEX` statement.
For example, advise a query that filters tasks by estimated effort:
```sql DQL theme={null}
ADVISE SELECT * FROM tasks WHERE estimateHours > 8
```
Ditto identifies the range predicate and recommends an index on `estimateHours`:
```json theme={null}
{
"advice": {
"statement": "select * from tasks where estimateHours > 8",
"suggestedIndexes": [
{
"collection": "tasks",
"reason": "range predicates on `estimateHours`",
"statement": "CREATE INDEX IF NOT EXISTS adv_tasks_estimateHours ON default:`tasks` (`estimateHours` ASC)"
}
]
}
}
```
Run the suggested statement yourself, or use `ADVISE AND PROVISION` to create the recommended indexes automatically. `ADVISE` can also recommend composite and covering indexes for more complex filters, sorting, projections, and joins. It is currently available on Small Peers.
## Return changed documents with RETURNING
[`RETURNING`](/dql/returning) lets an `INSERT`, `UPDATE`, `DELETE`, `EVICT`, or `TOMBSTONE` statement return data from the documents it changed. Applications can receive the affected data immediately instead of issuing a second query.
For example, update matching documents and return their IDs and new values in one operation:
```sql DQL theme={null}
UPDATE tasks
SET status = 'done'
WHERE projectId = 'proj-42' AND status = 'open'
RETURNING _id, status
```
`RETURNING` is especially valuable with deletes and evictions because it can return document contents before they are removed. It also supports projections, expressions, aliases, and aggregates such as `RETURNING COUNT(*) AS removed`.
Ditto 5.1 also adds [`INSERT ... SELECT`](/dql/insert#insert-from-a-select-statement) for creating documents directly from query results.
## Control long-running requests
Two new [system parameters](/dql/alter-system) help identify and control expensive DQL requests:
| System parameter | Default | Behavior |
| ------------------------------- | ------: | ------------------------------------------------------------------------- |
| `DQL_SLOW_REQUEST_WARN_SECONDS` | `60` | Logs request details at the threshold and repeats while the request runs. |
| `DQL_REQUEST_TIMEOUT_SECONDS` | `0` | Cooperatively cancels requests that exceed the configured limit. |
Set either parameter to `0` to disable it. Request history can also [filter by request type or explicit profiling requests](/dql/virtual-collections#request-history).
***
# 3. Security: Expanded Certificate Revocation
Certificate revocation information now propagates securely from Big Peer to Small Peers and from peer to peer throughout the mesh. As peers connect, they share the latest revocation information. Peers that were disconnected when a certificate was revoked receive the updated revocation state when they rejoin the mesh.
Revocation enforcement is enabled by default. Peers reject new connections that present a revoked certificate and terminate matching active connections when a revocation arrives. This isolates revoked clients and prevents them from reconnecting through another peer in the mesh.
***
# 4. Reliability: Production Diagnostics and Recovery
* Support bundles now include `config_snapshot.json`, which records the effective `DittoConfig`, transport configuration, system parameters, and SDK version.
* A configurable Unix `debug_socket` enables DQL diagnostics against a running Small Peer.
* SQLite metrics distinguish the application data store from replication metadata databases on supported Unix platforms.
* Corrupted per-peer replication metadata is reset and rebuilt automatically without affecting application documents.
***
# 5. Transport at Scale: Multicast Beta
Ditto's peer-to-peer model establishes a session between every pair of peers, so the total connection count grows as O(N²) with the size of the mesh. Beyond a certain mesh size, connection maintenance becomes the dominant cost even with carefully tuned LAN configuration.
Multicast sync takes a different approach. Devices join a shared multicast group rather than pairing off, which drops the connection count from O(N²) to O(N). There is one group membership per device.
Replication work is also optimized. A sender would typically transmit an update once per device in the mesh, which is O(N). Multicast enables the sender to publish a single broadcast to the group, which is approximately O(1).
The transport is built on reliable multicast (NORM, RFC 5740) combined with Ditto's data reconciliation, so peers recover missed data and catch up after joining or reconnecting.
When multicast is configured and available, it becomes the preferred replication path. Existing peer-to-peer transports remain active as automatic fallback for peers the group cannot reach. Documents and attachments both replicate over multicast, including repair of missing attachment data.
Multicast is a beta capability in Ditto 5.1. It ships in the core SDK as an opt-in feature rather than a part of the standard build. It is available in the Swift, Kotlin/KMP, Flutter, and Rust SDKs.
Contact Ditto support or your Ditto representative before deploying multicast in production to understand the current beta limitations.
***
# Upgrading to 5.1
Ditto 5.1 has undergone extensive backward-compatibility and rollback testing to ensure production deployments can safely return to supported earlier SDK versions when needed.
## Tested rollback compatibility
Ditto 5.1 changes the on-disk index format. If you need to downgrade from Ditto 5.1 or later, migrate through Ditto 5.0.2+. You can also migrate through Ditto 4.14.6+. These versions recognize the updated index format and automatically revert it to the format understood by earlier versions.
During the downgrade, composite indexes are replaced with single-field indexes, one for each of the composite index's keys.
See [index migration and downgrade behavior](/dql/indexing#migration) for details. As with any production upgrade, validate the procedure with representative application data before deployment.
***
# Platform Highlights
## Swift-Specific Changes
The Swift SDK in Ditto 5.1 improves concurrency behavior and diagnostics:
* Store observers now deliver callbacks on the queue specified by the `deliverOn` parameter.
* The store observation handler type aliases are now `@Sendable`.
* Exceptions thrown by application callbacks preserve their original call stack, so crash reporters attribute them to the throwing code instead of to Ditto.
* Custom identity providers no longer stall the SDK during the first credential refresh.
DittoSwiftTools adds `DiskUsageInspectorView`, a SwiftUI diagnostic view for inspecting live disk usage, growth trends, health thresholds, per-collection rankings, and document-size distributions.
### Platform changes
* Intel (`x86_64`) Mac support and tvOS support are removed.
* DittoSwiftTools now requires iOS 15+, Mac Catalyst 15+, or macOS 12+.
* The bundled `DittoSwiftPresenceViewer` is removed. Use `DittoPresenceViewer` from [DittoSwiftTools](https://github.com/getditto/DittoSwiftTools) instead.
* The `CBOR` enum and obsolete CBOR-specific store errors are deprecated.
***
# Full Changelog
## Swift Specific Changelog
**Added**:
* `DittoMulticastBetaConfig` and `DittoPeerToPeer.multicastBeta` for configuring the beta reliable UDP multicast transport. The transport is disabled by default and currently supported on iOS and macOS. On iOS, changes requested while sync is active take effect after sync is stopped and successfully started again, when Ditto validates the required platform configuration. (#SDKS-4471)
* `DittoConnectionType.multicast` enum option representing beta reliable UDP multicast connections. The transport is currently supported on iOS and macOS. (#SDKS-4471)
* `DiskUsageInspectorView` in DittoSwiftTools: a SwiftUI diagnostic view that shows live total disk usage, health status against an adjustable threshold, growth-rate trend, and projected time to threshold. Opt-in scans add per-collection rankings and a per-collection document-size histogram. (#SUPP-284)
**Changed**:
* DittoSwiftTools minimum platform requirements are raised to iOS 15+, Mac Catalyst 15+, and macOS 12+ (previously iOS 14+, Mac Catalyst 14+, and macOS 11+). The tvOS 15+ requirement is unchanged. (#SDKS-3141)
* The `DittoStoreObservationHandler`, `DittoStoreObservationHandlerWithSignalNext`, and `DittoSignalNext` type aliases are now `@Sendable`. (#SDKS-3856)
**Fixed**:
* Uncaught exceptions thrown inside SDK callback closures are now re-raised with the original call stack preserved, so crash reporters attribute the crash to the throwing code instead of reporting a `SIGABRT` inside Ditto. (#SDKS-3483)
* Custom identity providers using `DITIdentityProvider` no longer stall the SDK the first time a credential refresh is requested. The refresh request is released immediately so the peer is not blocked. (#SDKS-3623)
* `DittoStore.registerObserver` now dispatches its callback on the queue specified by the `deliverOn` parameter, resolving a crash when used with Swift concurrency. (#SDKS-3856)
**Deprecated**:
* `DittoPeerOS.tvOS`. The case is retained so peers running older SDK versions on tvOS can still be identified in the presence graph. (#SDKS-3944)
* The `CBOR` enum and the `DittoError.StoreErrorReason` cases `invalidDocumentStructure(cbor:)` and `nonStringKeyInDocument(key:)`. The SDK no longer produces these; they will be removed in a future version. (#SDKS-4009)
**Removed**:
* Support for Intel (`x86_64`) Macs. (#SDKS-2924)
* The `DittoSwiftPresenceViewer` framework target and podspec. Use the `DittoPresenceViewer` product from [DittoSwiftTools](https://github.com/getditto/DittoSwiftTools) via Swift Package Manager instead. (#SDKS-3141)
* tvOS platform support. (#SDKS-3944)
## 5.1.0 Common Changelog
**Performance**:
* The Document Sync protocol performs fewer full database scans after local subscription changes. (#DS-1043)
* Store observers no longer re-query unrelated collections when writes occur. (#QE-1116)
* Expired tombstones are cleaned up in bounded batches, configurable through the `tombstone_reap_batch_size` system parameter, to reduce peak memory in high-delete workloads. (#SPO-663)
**Added: Query engine and DQL**
* `debug_socket` system parameter for Unix socket-based DQL query access in Small Peer. Enables runtime-configurable query debugging and diagnostics via `ALTER SYSTEM SET debug_socket = '/path/to/socket'`. (#21822)
* Automatic logging of diagnostic information for long-running DQL requests, controlled by the new `DQL_SLOW_REQUEST_WARN_SECONDS` system parameter. (#QE-1038)
* `DQL_REQUEST_TIMEOUT_SECONDS` system parameter to limit the execution time of DQL requests. Requests exceeding the limit are stopped and complete with an error. (#QE-1044)
* A new [`request_history` qualifier](/dql/virtual-collections#request-history) for capturing requests that explicitly request profiling information. When enabled, requests that include `PROFILE` or the `#profile` directive are recorded. (#QE-1059)
* A new [`request_history` qualifier](/dql/virtual-collections#request-history) for filtering captured requests by `requestType`. (#QE-1060)
* Common trigonometric DQL scalar functions. (#QE-1088)
* Additional array-processing DQL scalar functions. (#QE-1089)
* `EXECUTE FUNCTION` statement for running DQL functions with side effects in a controlled environment. (#QE-1092)
* A new query-crate `Item` type that carries all value types the query engine uses, in aid of performance and Data Manipulation Language operations. (#QE-496)
* New DQL scalar functions to estimate object size and serialize values as JSON strings. (#QE-539)
* The ability to filter system parameters in `SHOW ALL` command output using a `LIKE` predicate. (#QE-554)
* Composite indexes, and indexes over array and object values, on Small Peers. This changes the on-disk index format and requires planning before a downgrade. (#QE-604)
* The [`ADVISE`](/dql/advise) statement, which recommends indexes for a query. (#QE-723)
* DQL `SELECT` statements can now join multiple local collections. (#FEAT-392)
**Added: Sync and replication**
* Garbage collection for Document Sync sessions now limits the number of disconnected sessions that are retained, even when the TTL is not exceeded. (#DS-1065)
* Info-level logging when sync scopes block remote subscriptions. (#DS-346)
* `disable_replication_gc_on_evict` system parameter (default: `false`). When set to `true`, calls to `evict()` no longer trigger immediate per-peer metadata cleanup; the periodic background replication GC continues to reclaim metadata for disconnected peers once they exceed the TTL (\~7 days by default). Intended as an opt-in escape hatch for deployments where eviction-time filesystem work contributes to write-path latency. (#QE-686)
* Automatic recovery when a peer's internal replication metadata is corrupted on startup. Ditto resets the affected sync metadata and resumes syncing without changing application documents. (#SPO-1081)
* `DITTO_RELAXED_SYNC_METADATA_DURABILITY`, an opt-in system parameter that reduces disk synchronization for rebuildable replication metadata without changing application-document durability. (#SPO-1081)
* New warn-level log triggered by multiple consecutive Small Peer resets during sync. (#SPO-52)
* New warn-level log when post-eviction session cleanup runs too frequently within a sliding window, signaling that excessive evictions may cause sync overhead on connected peers. (#SPO-640)
**Added: Networking and transports**
* Devices running the SDK now share certificate revocation lists and check them when connecting to peers. (#20088)
* `DITTO_PEER_CERTIFICATE_REVOCATION_CHECK_ENABLED` system parameter to control revocation checking. Revocation checks are enabled by default; set this parameter to `false` to disable them. (#20935)
* UDP support for NGN over the Wi-Fi Aware transport. (#21318)
* `transports_websocket_watchdog_interval_secs` system parameter for adjusting the WebSocket client watchdog interval. (#21885)
* `transport_websocket_connect_timeout` system parameter for adjusting the WebSocket connection timeout. (#23523)
**Added: Diagnostics and storage**
* Nine new metrics counters under `ditto.network.dsoq.*` for detecting NGN protocol-level failures in production: handshake failures, TLV decode errors, unknown type counts, dropped unreliable datagrams, implicit stream refusals, a connection lifecycle pair, and endpoint connect failures. (#NETW-998)
* Support bundles now include a `config_snapshot.json` file containing the customer's effective configuration at bundle-generation time, including the `DittoConfig`, transport settings, system parameters, and SDK version information. (#SDKS-3130)
* Per-database-role labels on SQLite storage metrics. Fsync count, fsync latency, and WAL size are attributed to either the application store or the replication metadata databases. The metrics are available on supported Unix platforms, including Linux and Android. (#SPO-1081)
* The `DITTO_SQLITE3_MAX_CONNECTIONS` system parameter can now be set as low as `16` (previously the minimum was `32`). (#SPO-668)
**Changed: Query engine and DQL**
* The DQL `PROFILE` response and the `system:request_history`, `system:active_requests`, and `system:shared_statements` virtual collections now expose the database identifier under `database_id` instead of `app_id`, aligning with v5 naming. (#22603)
* The DQL `UPDATE` statement has moved from a fixed query plan to the new execution operator model. (#QE-162)
* DQL `DELETE` statements can now include a `RETURNING` clause and may use a `USE IDS` clause to specify documents to delete by ID. (#QE-203)
* DQL `INSERT` statements can now include a `RETURNING` clause and may source documents from a `SELECT` statement. (#QE-368)
* DQL strings now accept all JSON escape sequences by default. (#QE-436)
* The DQL query engine now uses a standardized item type across query consumers. (#QE-497)
* DQL array and object transformation can now process the source contents in a nested fashion with the `WITHIN` keyword. (#QE-551)
* DQL transformation of an object to an array no longer requires a name-variable binding, simplifying the syntax. (#QE-551)
* Automatic generation of an ID scan in the query engine planner from `_id` field filters now considers `IN` filters in addition to simple equality filters. (#QE-553)
* The query engine now uses a lower-overhead buffer implementation for exchanging values. (#QE-724)
* `dql_enable_remote_full_syntax` now defaults to `true`, enabling authorized Remote Query and debug-socket sessions to run mutation and `ALTER SYSTEM` statements. Set it to `false` to restrict those sessions to `SELECT` and `SHOW`. (#SPO-970)
**Changed: Sync, networking, and security**
* JWT validation failures now log all non-sensitive claims (issuer, version, audience, timestamps) to aid support debugging without requiring customers to share JWT tokens. (#21207)
* Mesh connection limits are enforced per transport instead of through a shared Wi-Fi budget. Behavior at capacity is configurable, and rejected peers use exponential backoff to avoid retry storms. (#NETW-1586)
* Peer certificate revocation checks are enabled by default. Revoked certificates are rejected on new connections, matching active connections are terminated, and Big Peer propagates revocations to connected Small Peers. (#NETW-2055)
* `replication_session_request_timeout_secs` and `blob_session_request_timeout_secs` remain accepted for backward compatibility but no longer have an effect. (#SPO-869)
**Changed: Storage and durability**
* Document and diff encoding now preserves the existing format when writing to the transaction log, and defaults new writes to RKYV when no format has been selected. (#21224)
* Document insert and update operations now return an error when a document exceeds the configured hard size limit (5 MB by default). (#SPO-1003)
* The default value of the `DITTO_SQLITE3_MAX_CONNECTIONS` system parameter is now 32, down from 60. (#SPO-668)
**Changed: Diagnostics and logging**
* Warning log messages now use "store observer" terminology instead of "live query" to match DQL API naming. (#20974)
* The default number of subscription queries displayed in `__small_peer_info` increased from 16 to 32. (#21085)
* Structured peer ID fields now render resolvable peer-key prefixes instead of legacy suffixes; field names are unchanged. (#SPO-1090)
* Peer session discontinuity errors now suggest a potential cause. (#SPO-7)
**Fixed: Query engine and DQL**
* Legacy `find()` / `subscribe()` queries that combine `||` or `&&` with a comparison that hits a type mismatch (for example, `(age > 18) || (subscription == 'premium')` against a document where `age` is a string) no longer silently drop matching rows. The other side of the `||` / `&&` now correctly resolves the expression, matching the JMESPath specification and the DQL evaluator's behavior. (#23570)
* The DQL `substr` scalar function's negative index calculation, and the `lpad` and `rpad` scalar functions, now all correctly handle multi-byte characters. (#QE-1021)
* The DQL scalar function `object_content` now correctly processes the `nesting` configuration value `"only"` when passed as part of an object. It was already handled correctly when passed directly. (#QE-1021)
* Small Peer write transactions are processed in arrival order, preventing indefinite write starvation under competing operations. (#QE-1090)
* The underlying containers for the query engine's `active_requests` and `request_history` virtual collections have been changed to prevent the rare possibility of delays in concurrent read and write requests. (#QE-564)
* DQL statements now handle quoted namespace and data source values correctly. (#QE-727)
* `BETWEEN` index spans now include the upper bound. (#QE-896)
**Fixed: Networking and transports**
* Wi-Fi Aware transport connection recovery when a peer's screen turns off and back on. (#21454)
* `Peer::trigger_disconnect_all_peers` now correctly disconnects NGN-only transports such as UDP, ensuring all active peer-to-peer connections are properly closed. (#NETW-1100)
* Mesh initialization now validates that the `replication_over_ngn` system parameter is only enabled when `network_enable_ngn` is also enabled, preventing sync failures due to misconfigured NGN settings. (#NETW-1176)
* LAN peers discovered over UDP/NGN via mDNS are no longer dispatched to the TCP transport, eliminating repeated futile TCP connection attempts (and their timeout log noise) against peers that only speak UDP. (#NETW-1488)
* Bluetooth LE duplicate connections no longer interrupt active peer-to-peer sync during connection handoff. (#NETW-2199)
* mDNS now re-advertises with the current interface addresses when they change (interface up or down, address renewal, link-local address appearing or disappearing) instead of caching the initial set. (#NETW-936)
* The cloud WebSocket URL is now included in the transport config published to `__small_peer_info`, and is preserved when the transport config is later updated. (#SPO-389)
* The "repeatedly failed to connect to peer" error is now logged once per outage instead of on every retry. (#SPO-988)
**Fixed: Sync, storage, and lifecycle**
* A crash (`SIGABRT`) that could occur during Ditto shutdown. (#22065)
* A bug in internal subscription bookkeeping caused long-connected Document Sync sessions to become disabled due to spurious capacity errors. (#SPO-1011)
* The `DITTO_SQLITE3_SYNCHRONOUS`, `DITTO_SQLITE3_CACHE_SIZE`, `DITTO_SQLITE3_MMAP_SIZE`, `DITTO_SQLITE3_TEMP_STORE`, `DITTO_SQLITE3_WAL_AUTOCHECKPOINT`, and `DITTO_SQLITE3_FULLFSYNC` tuning parameters now take effect on the connections that execute reads and writes. Previously these values had no effect on the connections serving the actual workload, so the configured tuning was effectively ignored. (#SPO-1053)
* Collection scans no longer fail when encountering a corrupted or undecodable document. Corrupted documents are now skipped, allowing queries and observers to continue operating even if local storage contains unreadable records. (#SPO-1064)
* Connection flapping during `TransactionTooLarge` errors is prevented by transitioning the session into the Disabled state. (#SPO-259)
**Removed**:
* Spurious `dsoq.cbor` CBOR warning log during auth client initialization. (#21646)
* The experimental history-tracking feature. Applications upgrading from previous SDK versions that had this enabled can reclaim disk space by running `EVICT FROM __history` after upgrade. (#SPO-929)
## 5.0.3 Common Changes
**Added**: a new system parameter transports\_websocket\_watchdog\_interval\_secs to adjust WebSocket client watchdog. (#21885)
**Fixed**: Corrected a formalisation bug causing DQL statement filters to never match, that was affecting observers using projections. (#QE-1056)
**Fixed**: Corrected a hang when accessing `system:data_sync_info`. (#QE-1095)
## 5.0.2 Common Changes
**Added**: a new timeout for WebSocket connect that is adjustable by system parameter transport\_websocket\_connect\_timeout. (#21866)
**Changed**: Mesh chooser now enforces per-transport connection limits instead of a shared WiFi radio budget, preventing one transport (e.g. TCP) from starving others (e.g. AWDL, WiFi Aware). When a transport is at capacity, the behavior for new inbound connections is configurable: reject, drop oldest, drop newest, or accept. Peers at capacity reject inbound connections with a new `ConnectError::AtCapacity` variant and back off exponentially to avoid retry storms. (#NETW-1586)
**Added**: Automatic downgrading of the SP store from V3 to V2 on start-up. (#QE-595)
**Added**: Automatic downgrading of the SQLite3 schema from V3 to V2 allowing for downgrading from version 5.1. (#QE-595)
**Added**: `disable_replication_gc_on_evict` system parameter (default `false`). When set to `true`, calls to `evict()` no longer trigger immediate per-peer metadata cleanup; the periodic background replication GC continues to reclaim metadata for disconnected peers once they exceed the TTL (\~7 days by default). Intended as an opt-in escape hatch for deployments where eviction-time filesystem work contributes to write-path latency. (#QE-686)
**Added**: Optional background task to reclaim unused space in the Small Peer store and record space usage metrics. (#QE-779)
**Changed**: The way query profile timing and count data is recorded to reduce overheads. (#QE-811)
**Fixed**: A bug in internal subscription bookkeeping caused long-connected Document Sync sessions to become disabled due to spurious capacity errors. (#SPO-1011)
**Added**: The option to set the `DITTO_SQLITE3_MAX_CONNECTIONS` parameter lower than `32`, down until `16` (#SPO-668)
**Changed**: The default value of `SQLITE3_MAX_CONNECTIONS` parameter to 32, down from 60 (#SPO-668).
**Changed**: The replication\_session\_request\_timeout\_secs and blob\_session\_request\_timeout\_secs system parameters are no longer functional; the timeouts have been removed. They are accepted for backwards compatibility but have no effect. (#SPO-869)
## 5.0.1 Common Changes
**Removed**: Spurious `dsoq.cbor` CBOR warning log during auth client initialization. (#21646)
**Fixed**: Crash due to unignored SIGPIPE signal in the React Native SDK on iOS. (#22006)
**Added**: Garbage collection for document sync sessions now imposes limits on the number of disconnected sessions that will be retained, even if the TTL is not exceeded. (#DS-1065)
**Fixed**: the query engines erroneously build index spans not including the higher end for the BETWEEN operator. Fixed to include it. (#QE-896)
# 5.0 - Built for Speed and Developer Experience
Ditto 5.0 brings significant performance improvements and a developer experience redesigned for usability. Your applications run faster with optimized queries and sync, while modern APIs and simplified patterns eliminate complexity at every turn.
Automatic optimizations make your apps more responsive with zero code changes
Start building immediately without upfront type definitions
Clear, modern patterns replace confusing legacy APIs
DQL handles all data operations consistently across platforms
Query system metrics and configuration directly with DQL
Faster builds, lower memory usage, cleaner codebase
Advanced query features, networking improvements, and enhanced reliability
***
## Our Most Rigorously Tested Release
Ditto 5.0 has undergone extensive validation in our internal mesh lab, ensuring reliability across real-world scenarios before reaching production.
**Testing scope:**
* **Multi-device mesh scenarios** - Validated with up to 40 concurrent devices across iOS, Android, and budget hardware
* **Cross-platform compatibility** - Tested heterogeneous meshes mixing iOS, Android, and version combinations
* **Extended reliability testing** - 24-hour continuous operation tests validating memory stability and connection resilience
* **Real-world datasets** - From small 1MB datasets to large 50MB+ product catalogs
* **Network conditions** - LAN, BLE, WiFi Aware, AWDL, and mixed transport scenarios
* **Lifecycle edge cases** - Background/foreground transitions, network partitions, and recovery scenarios
Every test validates performance, memory management, data integrity, and sync convergence across the scenarios that matter most to production deployments.
***
## In This Release
This is a major version with breaking changes, but maintains backwards compatibility with v4 deployments (4.11+). Migrate at your own pace, and upgrade to v4.14 first for the smoothest transition.
**DQL for All Data Operations:**
* [DQL for All Data Operations](#dql-for-all-data-operations)
**Core Capabilities:**
* [Simplified Initialization & Configuration](#simplified-initialization-%26-configuration)
* [Schema-Free Data Modeling](#schema-free-data-modeling)
* [New DQL Query Features](#new-dql-query-features)
**Performance & Platform:**
* [DQL Query Performance Improvements](#dql-query-performance-improvements)
* [Data Sync Performance Improvements](#data-sync-performance-improvements)
* [Local System Observability](#local-system-observability)
* [Additional Improvements](#additional-improvements)
**Upgrading to v5:**
* [Migration Guidance](#migration-guidance)
* [Terminology Updates](#terminology-updates)
* [Breaking Changes](#breaking-changes)
**Swift Specific Changes**
* [Swift Specific Changes](#swift-specific-changes)
**Full Changelog**
* [Full Changelog](#full-changelog)
***
# DQL for All Data Operations
## DQL for All Data Operations
Ditto 5.0 completes the transition to DQL (Ditto Query Language) as the single API for all data operations. The legacy query builder has been removed.
### What Changed
* **Legacy query builder removed**: All `store.collection()` methods and fluent query APIs are no longer available
* **Full feature parity**: Every legacy operation has a DQL equivalent
* **Single query language**: DQL handles reads, writes, subscriptions, and observers
* **Works everywhere**: Same syntax across mobile, server, and web
* **SQL-familiar**: Standard SQL patterns for immediate productivity
### Why One Query Language
A unified query language means one implementation to maintain, faster feature delivery, and consistent behavior across all platforms. New capabilities and optimizations benefit every SDK simultaneously.
### Migration Path
All legacy query operations have direct DQL equivalents:
**Update Operations:**
```swift Swift - DQL (v5) theme={null}
try await store.execute(
"UPDATE cars SET miles = 50000 WHERE _id = 'abc123'"
)
```
```swift Swift - Legacy Query Builder (v4) theme={null}
store.collection("cars")
.findByID("abc123")
.update { doc in
doc["miles"].set(50000)
}
```
**Observers:**
```swift Swift - DQL Observer (v5) theme={null}
store.registerObserver(
query: "SELECT * FROM cars WHERE miles > 100000"
) { result in
// handle changes
}
```
```swift Swift - Legacy Observer (v4) theme={null}
store.collection("cars")
.find("miles > 100000")
.observe { docs, event in
// handle changes
}
```
Complete migration examples for all legacy query patterns are available in the [Legacy to DQL Migration Guide](/dql/legacy-to-dql-adoption).
***
# Core Improvements
## Simplified Initialization & Configuration
Ditto 5.0 introduces a completely redesigned initialization flow built around the new `DittoConfig` pattern. This replaces the previous `Identity`-based approach with a clearer, more predictable setup that aligns with modern best practices.
### What's New
The new configuration system provides:
* **Unified configuration object**: All initialization parameters are set through `DittoConfig` factory methods
* **Fallible initialization**: Explicit error handling during setup catches configuration issues early
* **Asynchronous patterns**: Native async/await support where appropriate for each platform
* **Simplified authentication**: Clearer authorization client that's easier to understand and implement
### What This Solves
The previous initialization flow required multiple unintuitive settings due to legacy compatibility concerns. For example, developers had to set `disableCloudSync = true` to connect to a Big Peer, which was confusing and non-obvious.
The new pattern consolidates all configuration into a single, coherent flow that makes the relationship between settings explicit and easier to reason about.
### Accessing Configuration at Runtime
After initializing Ditto, you can access the configuration object to retrieve settings like the database ID. This replaces methods like `getAppID()` from v4:
```javascript theme={null}
// Example: Access database ID
const config = ditto.config; // or ditto.getConfig() depending on SDK
const databaseID = config.databaseID;
```
Each SDK provides access to the config object with language-appropriate naming conventions. See the SDK-specific migration guides for exact syntax.
### Availability
The `DittoConfig` pattern was introduced as an option in v4.12 and becomes the only initialization method in v5.0.
Migration guides for each SDK are available in the v5 documentation. If you're currently on v4.11 or later, the migration path is straightforward.
### Migration Example
The initialization flow has changed from `Identity`-based to `DittoConfig`-based patterns. Here's how to migrate:
```swift Swift - V5 theme={null}
let config = DittoConfig(
databaseID: "your-database-id", // was: appID
connect: .server(url: URL(string: "REPLACE_ME_WITH_YOUR_URL")!)
)
let ditto = try await Ditto.open(config: config)
// Set up authentication
ditto.auth?.expirationHandler = { ditto, secondsRemaining in
ditto.auth?.login(token: "your-token", provider: .development) { clientInfo, error in
// Handle auth result
}
}
try ditto.sync.start() // was: ditto.startSync()
```
```swift Swift - V4 theme={null}
let identity = DittoIdentity.onlinePlayground(
appID: "your-app-id",
token: "your-token"
)
let ditto = Ditto(identity: identity)
try ditto.startSync()
```
### Accessing Configuration at Runtime
After initialization, you can access your Ditto configuration to retrieve settings like the database ID:
```swift theme={null}
let config = ditto.config
let databaseID = config.databaseID // Replaces ditto.appID from v4
```
## Schema-Free Data Modeling
Ditto 5.0 transforms the developer experience by making DQL strict mode disabled by default. This eliminates the need to define collection schemas or CRDT types upfront, allowing you to insert nested objects and data structures without pre-defining types.
### What's New
With strict mode disabled by default, Ditto changes how objects are stored and synchronized:
**Objects default to MAP type instead of REGISTER type**
This fundamental change means:
* **Field-level sync**: Ditto syncs individual field changes instead of replacing entire objects
* **Automatic type inference**: No need to pre-define collection schemas or CRDT types
* **Nested structures**: Insert complex JSON-like documents without type definitions
* **Concurrent updates merge**: When peers update different fields simultaneously, both changes are preserved
For example, if two peers update different fields in the same object concurrently, both changes sync successfully rather than one overwriting the other.
### What This Solves
This change provides a more simplified data management structure with two key benefits:
* **Add-wins behavior on objects**: When peers create or modify objects, additions are preserved rather than overwritten
* **Field-level delta sync on all nodes**: Every field in your document syncs independently, reducing bandwidth and enabling fine-grained conflict resolution throughout the entire document structure
### New Customers
Schema-free data modeling is enabled by default in Ditto 5.0. No action is needed—simply start building with automatic type inference and field-level sync.
### Migrating Customers
If you're migrating from Ditto 4.X, set `DQL_STRICT_MODE = true` to ensure your application behavior remains the same:
```sql theme={null}
ALTER SYSTEM SET DQL_STRICT_MODE = true
```
This maintains the same data modeling semantics you're currently using. Once your application is stable on v5, follow the strict mode migration guide and work with the Ditto CX team to migrate to schema-free data modeling and take advantage of the improved developer experience.
### Migration Considerations
Strict mode is a **local configuration setting** on each device that controls how DQL interprets and writes data structures.
**How it works across peers:**
* Data **syncs successfully** between peers regardless of different strict mode settings
* Each peer interprets data based on its **own** strict mode setting when reading/writing
* With `DQL_STRICT_MODE=false`: Objects are inferred as MAPs (field-level merging)
* With `DQL_STRICT_MODE=true`: Objects without explicit type definitions are treated as REGISTERs (whole object replacement)
**Impact on data:**
* Collections/documents created, modified, or read while strict mode is **disabled** will use inferred types (objects → MAPs by default)
* Collections/documents created, modified, or read while strict mode is **enabled** use default REGISTER type and require explicit definitions for other types
**Best practices for mixed deployments:**
* If mixing settings, explicitly define MAP types in collection definitions on peers with strict mode enabled
Learn more about strict mode, cross-peer synchronization, and troubleshooting in the [DQL Strict Mode documentation](/dql/strict-mode).
### How It Works
The key difference is whether you need to explicitly define MAP types for objects:
```swift DQL_STRICT_MODE=false (Default in v5) theme={null}
// Objects are automatically inferred as MAPs - no definition needed
try await store.execute("""
INSERT INTO products
DOCUMENTS ({
_id: '123',
name: 'Widget',
metadata: {
manufacturer: 'Acme Corp',
warehouse: 'East'
}
})
""")
// Query without type definitions
try await store.execute("""
SELECT * FROM products WHERE _id = '123'
""")
```
```swift DQL_STRICT_MODE=true (Default in v4) theme={null}
// Must explicitly define MAP types to use objects
try await store.execute("""
INSERT INTO COLLECTION products (metadata MAP)
DOCUMENTS ({
_id: '123',
name: 'Widget',
metadata: {
manufacturer: 'Acme Corp',
warehouse: 'East'
}
})
""")
// Query with explicit MAP definition
try await store.execute("""
SELECT * FROM COLLECTION products (metadata MAP)
WHERE _id = '123'
""")
```
## New DQL Query Features
Ditto 5.0 introduces several new DQL syntax features that expand query capabilities and make complex queries easier to write.
### CASE Statements
Add conditional logic to queries with CASE expressions:
```sql Simple CASE theme={null}
-- Match a field against multiple values
SELECT CASE a
WHEN 1 THEN 'one'
WHEN 2 THEN 'two'
ELSE 'neither'
END
FROM test
```
```sql Searched CASE theme={null}
-- Use conditional expressions
SELECT CASE
WHEN a = 1 THEN 'one'
WHEN a = 2 THEN 'two'
ELSE 'neither'
END
FROM test
```
### BETWEEN Expressions
The `BETWEEN` operator provides a shorthand for defining an inclusive numeric range for an expression result.
**Syntax:**
```sql theme={null}
BETWEEN AND
```
**Example:**
```sql theme={null}
SELECT * FROM test WHERE a BETWEEN 1 AND 10
```
This is equivalent to `(a >= 1 AND a <= 10)`.
**Order matters**: The order of expressions is important. Reversing the terms implies an inverse range - `BETWEEN 1 AND 10` and `BETWEEN 10 AND 1` are NOT equivalent.
### Array & Object Search Syntax
Test elements within arrays or objects using `ANY`, `EVERY`, or `ANY AND EVERY` operators:
**Syntax:**
```sql theme={null}
(ANY|EVERY|ANY AND EVERY) [name:] value (IN|WITHIN) expression SATISFIES condition END
```
* **IN** - Searches the array/object directly
* **WITHIN** - Searches recursively through nested structures
* **ANY** - Returns true if at least one element matches
* **EVERY** - Returns true if all elements match (empty arrays/objects pass)
* **ANY AND EVERY** - Like EVERY, but empty arrays/objects fail
```sql Arrays theme={null}
-- Check if any element in an array equals 2
SELECT ANY x IN [1,2,3] SATISFIES x = 2 END
FROM system:dual
-- Check if any nested element equals 2
SELECT ANY x WITHIN [1,[2],3] SATISFIES x = 2 END
FROM system:dual
-- Check if all elements equal 1 (or are arrays)
SELECT EVERY x WITHIN [1,[1],1]
SATISFIES x = 1 OR type(x) = 'array' END
FROM system:dual
```
```sql Objects theme={null}
-- Check if any field value equals 2
SELECT ANY x IN {'a':1,'b':2,'c':3} SATISFIES x = 2 END
FROM system:dual
-- Check if any nested field value equals 2
SELECT ANY x WITHIN {'a':1,'b':{'d':2},'c':3} SATISFIES x = 2 END
FROM system:dual
```
```sql Practical Examples theme={null}
-- Find documents where first array element is less than zero
SELECT * FROM test
WHERE ANY n:v IN test.array_field SATISFIES n = 0 AND v < 0 END
-- Find documents where any nested field is NULL
SELECT * FROM test
WHERE ANY v WITHIN test.details SATISFIES v IS NULL END
-- Find orders where items were modified after order date
SELECT * FROM orders o
WHERE ANY v IN o.items SATISFIES v.modified > o.order_date END
```
### Array & Object Transformation Syntax
Create new arrays and objects by transforming existing data structures:
**Array Transformation Syntax:**
```sql theme={null}
ARRAY valueExpr FOR [name:] value IN source [WHEN condition] END
```
**Object Transformation Syntax:**
```sql theme={null}
OBJECT nameExpr:valueExpr FOR name:value IN source [WHEN condition] END
```
```sql Array Transformations theme={null}
-- Transform array elements to objects with index
ARRAY {"index":i,"val":v} FOR i:v IN [1,2,3] WHEN v%2 = 0 END
-- Result: [{"index":1,"val":2}]
-- Convert numbers to strings, excluding value 1
ARRAY cast(v,'string') FOR v IN [1,2,3] WHEN v != 1 END
-- Result: ["2","3"]
-- Transform string characters with conditional logic
ARRAY CASE WHEN i%2 = 0 THEN "foo" ELSE "bar" END
FOR i:v IN split("hello","") END
-- Result: ["foo","bar","foo","bar","foo","bar"]
-- Extract values from object into array
ARRAY v FOR n:v IN {"a":"one","b":"two"} END
-- Result: ["one","two"]
```
```sql Object Transformations theme={null}
-- Transform object keys to uppercase, filter by value length
OBJECT upper(n):v FOR n:v IN {"a":"one","b":"two","c":"three"}
WHEN len(v) = 3 END
-- Result: {"A":"one","B":"two"}
-- Convert array to object with generated field names
OBJECT "field_"||cast(i,"string"):v FOR i:v IN [1,2,3] END
-- Result: {"field_0":1,"field_1":2,"field_2":3}
-- Filter object by value type
OBJECT n:v FOR n:v IN {"a":1,"b":[],"c":2}
WHEN type(v) != 'array' END
-- Result: {"a":1,"c":2}
```
**Key behaviors:**
* If `source` evaluates to `MISSING`, the result is `MISSING`
* For arrays: if `source` is not an array/object, the result is `NULL`
* For objects: duplicate field names will overwrite previous values
* Elements/fields where `valueExpr` evaluates to `MISSING` are excluded from the result
### Extended String Literals
Support for escape sequences in strings:
```sql theme={null}
-- Use escape sequences in string literals
SELECT * FROM messages
WHERE content = 'Line 1\nLine 2\tTabbed'
-- [PLACEHOLDER: Add more escape sequence examples if needed]
```
### Hexadecimal Numeric Constants
Additional numeric literal formats:
```sql theme={null}
-- Use hexadecimal literals
SELECT * FROM config
WHERE flags = 0xFF
-- [PLACEHOLDER: Add more hex literal examples if needed]
```
***
# Performance & Platform
## DQL Query Performance Improvements
Your applications will feel noticeably faster and more responsive. Data queries complete faster, delivering a snappier user experience whether users are searching, filtering, or loading content. These performance gains happen automatically—no code changes required.
### Query Planner Enhancements
The DQL query planner has been enhanced to recognize more optimization opportunities:
* **Automatic ID scan conversion**: Equality filters on `_id` fields automatically convert to ID scans, bypassing index lookups when exact document IDs are known
* **Deferred document fetching**: Query planner can defer fetching full documents until after sorting and applying offset/limit when index access supports it
* **Improved covering index support**: More scenarios where the query planner can satisfy queries entirely from index data without retrieving documents
* **Index-only queries**: Additional cases where queries can be answered using only index scans
### Streaming Query Execution
The query engine now uses streaming interfaces internally:
* **Reduced memory overhead**: Results are streamed rather than fully materialized where possible
* **DISTINCT operator streaming**: DISTINCT queries now stream results, reducing memory usage for large result sets
* **Improved operator inlining**: Query operators can be inlined into producers for better performance
### Shared Statement Cache
Ditto 5.0 introduces a shared statement cache that stores and reuses compiled query plans:
* **Plan reuse**: Compiled query plans are cached and reused for identical or similar statements, eliminating redundant parsing and planning overhead
* **Automatic validation**: Cached plans are automatically verified and invalidated when collection schemas or system directives change
* **Dynamic sizing**: The cache automatically resizes based on workload patterns
* **Improved large statement performance**: Particularly benefits complex queries and larger statements by avoiding expensive re-compilation
This optimization is especially impactful for applications that execute the same queries repeatedly, such as real-time dashboards or frequently-accessed data views.
These improvements are automatic - no code changes required. Your existing DQL queries will benefit from the enhanced query planner.
## Data Sync Performance Improvements
Building on the performance improvements delivered in v4.13 & v4.14, Ditto 5.0 further optimizes data synchronization through tiered blob storage and protocol enhancements, making sync operations faster and more efficient.
### What's New
Version 5.0 introduces:
* **Tiered blob storage**: Smaller sync updates avoid unnecessary disk I/O by using optimized storage tiers
* **Improved session handling**: Better avoidance of session resets on reconnection when in-flight updates were lost
* **Optimized fsync policy**: Document sync avoids forcing files to disk by default, decreasing I/O and improving latency
### Impact
Applications upgrading to v5.0 will experience:
* **Faster sync operations**: Reduced disk I/O overhead leads to quicker data synchronization
* **Lower latency**: Optimized file handling decreases sync latency across the board
* **Better reconnection handling**: Fewer redundant updates after temporary disconnections
* **Improved efficiency**: Reduced disk operations lower memory and CPU pressure during sync
## Local System Observability
Gain unprecedented visibility into how Ditto operates on your devices. Query real-time metrics, inspect runtime configuration, and monitor system health directly using DQL—giving you deeper insights than ever before to debug issues, optimize performance, and understand your application's behavior.
### New Virtual Collections
**`system:metrics`**
* Query performance metrics and diagnostics in real-time
* Access counters, timers, and other operational metrics via DQL
* Monitor system health and performance without external tools
**`system:system_info`**
* Query `peer_key`, `database_id`, and configuration settings
* Inspect runtime configuration and system parameters
* Useful for debugging and operational awareness
**`system:shared_statements`**
* Inspect the query plan cache
* View cached statements and their execution plans
* Supports DELETE operations to clear specific cached statements
* Helps optimize query performance and troubleshoot query planning
**Local-Only Collections**: System collections are local to each peer and are **not replicated** across the mesh. To query these collections on remote peers, use [Remote Query](https://docs.ditto.live/cloud/common/operations/remote-query) from the Ditto Portal.
### Usage Example
```sql theme={null}
-- Query system information
SELECT * FROM system:system_info
-- View cached query statements
SELECT * FROM system:shared_statements
-- Access metrics
SELECT * FROM system:metrics
WHERE metric_name = 'sync.documents_synced'
-- Clear a specific cached statement
DELETE FROM system:shared_statements
WHERE statement_id = 'abc123'
```
These collections provide unprecedented visibility into Ditto's internal state, making it easier to monitor, debug, and optimize your applications.
## Additional Improvements
### Reliability & Error Handling
* **Enhanced diagnostics**: Improved logging when peers receive data that cannot be deserialized
* **Recovery mechanisms**: Additional recovery paths for document deserialization errors
* **Smart log levels**: Connection failures start at warning level, escalate to error only after repeated failures
* **Panic messages**: Filtered to remove internal Rust machinery frames for improved readability
### Networking Improvements
* **Graceful shutdown**: Network connections close cleanly when Ditto is stopped
* **Faster disconnection detection**: When a peer crashes, Ditto stops attempting to connect within 15 seconds (previously up to 75 minutes)
* **mDNS improvements**: More reliable mDNS discovery, configurable service names, better address filtering
* **BLE improvements**: Fixed connection issues on Android 9 and earlier devices
* **WebSocket BYOD support**: Bring Your Own Discovery now supports WebSocket connections
* **Connection cleanup**: Fixed deadlock where devices could fail to establish new P2P connections until restarted
### DQL Engine Improvements
Beyond the query performance improvements detailed above, v5 includes:
* **Better error messages**: Improved parser error messages for invalid DQL syntax
* **Transaction safety**: Fixed deadlock scenarios in concurrent transactions
* **Index correctness**: Fixed issues where index scans could yield incorrect results on document deletion
### Logging & Diagnostics
* **Better disk utilization**: On-disk logs resume writing to incomplete files, making better use of available space
* **Compressed size limits**: Log file limits now apply to compressed size, significantly increasing retention
* **Explicit flushing**: Logs explicitly flushed before aborting due to panic
* **Virtual collections**: New `system:metrics` and `system:system_info` collections for DQL access to metrics and system information
### Platform Support
* **Linux aarch64**: Kotlin SDK now supports ARM64 Linux (Raspberry Pi, AWS Graviton, etc.)
* **Swift 6**: Full Swift 6 support with Sendable conformance
* **16KB alignment**: React Native Android meets Google Play's November 2025 requirement
***
# Upgrading to v5
### Swift Migration Guide
Upgrading to Ditto 5.0 requires updating your initialization code and migrating from legacy query APIs to DQL. The migration process involves:
* Updating from `DittoIdentity` to `DittoConfig`-based initialization
* Replacing legacy query builder operations with DQL statements
* Migrating collection observers to DQL observers
* Updating authentication patterns
For comprehensive migration instructions, code examples, and best practices, see the [Swift v4 to v5 Migration Guide](/sdk/latest/migration-guides/swift-v4).
## Terminology Updates
Ditto 5.0 updates terminology across the platform to align with industry standards and reduce confusion.
### Database ID (formerly App ID)
* `appID` → `databaseID` in all configuration methods
* `getAppId()` → `getConfig().databaseId` in SDK APIs
* Portal and documentation updated to use "Database ID" terminology
**Why this matters**: The term "App ID" caused confusion, particularly for mobile developers who associate "app" with the mobile application itself rather than the Ditto database instance. "Database ID" more accurately describes what the identifier represents: a unique identifier for your Ditto database that persists across all clients.
### Ditto Server (formerly Ditto Cloud)
* `isConnectedToDittoCloud` → `isConnectedToDittoServer` in presence APIs
* Documentation updated to use "Ditto Server" terminology
**Why this matters**: This clarifies that the property indicates connection to any Big Peer (Ditto Server), not just those running in Ditto's cloud service. This is more accurate for deployments using self-hosted Big Peers.
### Migration
Update your code to use the new terminology:
```swift Swift - New Terminology (v5) theme={null}
// Database ID
let config = DittoConfig(
databaseID: "your-id",
connect: .server(url: URL(string: "REPLACE_ME_WITH_YOUR_URL")!)
)
// Presence API
if peer.isConnectedToDittoServer {
// handle connection
}
```
```swift Swift - Old Terminology (v4) theme={null}
// App ID
let identity = DittoIdentity.onlinePlayground(
appID: "your-id",
token: "your-token"
)
// Presence API
if peer.isConnectedToDittoCloud {
// handle connection
}
```
The actual ID values and functionality remain unchanged - only the parameter and property names have been updated.
## Breaking Changes
Ditto 5.0 is a major version release that removes deprecated APIs and legacy features. For migration guidance, see [Migration Guidance](#v5-migration-guidance).
### Removed APIs
#### Legacy Query Builder (All SDKs)
All legacy query builder APIs have been removed:
* `store.collection()` → Use DQL `INSERT`, `UPDATE`, `EVICT` statements
* `collection.find()` → Use DQL `SELECT` queries
* `collection.findById()` → Use DQL with `_id` filter
* Live queries → Use DQL observers with `store.registerObserver()`
* Write transactions → Use `store.transaction()` with DQL
#### Legacy Initialization (All SDKs)
* `Identity` classes and all subclasses removed
* `Ditto(identity:, persistenceDirectory:)` constructors removed
* Use `DittoConfig` factory methods and `Ditto.open()` instead
#### Sync Methods Moved
* `ditto.startSync()` → `ditto.sync.start()`
* `ditto.stopSync()` → `ditto.sync.stop()`
* `ditto.isSyncActive` → `ditto.sync.isActive`
#### Other Removals
* `disableSyncWithV3()` - no longer needed, v3 sync removed entirely
* `AttachmentToken` - use dictionary variant
* Transport diagnostics APIs - obsolete, removed
* Various deprecated presence properties (`queryOverlapGroup`, `meshRole`, etc.)
* Emoji log level headings - setting had no effect, removed
### Behavioral Changes
Several default behaviors have changed in v5:
* **DQL strict mode**: Now defaults to `false` - no schema definitions required, automatic CRDT type inference
* **String literals in DQL**: Double quotes now delimit strings (not identifiers) for JSON compatibility
* **Subscription queries**: Reject `LIMIT` and `ORDER BY` unless `DQL_RESTRICT_SUBSCRIPTION=false`
* **Observer ordering**: Observers require explicit `ORDER BY` clause for stable ordering
* **WebSocket sync**: Disabled by default in new `TransportConfig` instances - must explicitly enable
* **Document IDs**: `null` is no longer allowed as a document ID
These behavioral changes may affect existing code. Review your DQL queries and subscription logic when migrating to v5.
### SDK Size Reduction
The removal of legacy APIs has reduced SDK footprint by approximately 25%, resulting in:
* Smaller application binary sizes
* Reduced memory usage
* Faster SDK initialization
* Simpler maintenance and debugging
***
# Swift Specific Changes
## Swift-Specific Changes
The Swift SDK has additional platform-specific changes in v5.0 beyond the common breaking changes.
### Removed Platform Support
**Removed Platform Support:**
* **tvOS** - No longer supported in v5.0
* **visionOS** (beta) - No longer supported in v5.0
If you have concerns about this change, please reach out to Ditto customer support.
### Platform Requirements
**Minimum Platform Versions:**
* **iOS**: 15.0 (previously 13.0)
**Swift 6 Compatibility:**
* Full support for Swift 6 strict concurrency checking
* All public classes marked as `final`
* Most public types marked as `Sendable`
* Introduced dedicated typealiases:
* `DittoTransactionScope` for `DittoStore.transaction()` scope closures
* `DittoAttachmentFetchEventHandler` for `DittoStore.fetchAttachment()` event handlers
### Removed
**CocoaPods Support:**
* CocoaPods distribution has been completely removed starting with v5.0
* Use Swift Package Manager (SPM) or XCFrameworks for integration
* CocoaPods is effectively end-of-life and no longer maintained
**ObjC SDK:**
* The deprecated DittoObjC SDK has been removed
* No longer required as a dependency for DittoSwift
**Other Removals:**
* `DittoExperimental.jsonByTranscoding(cbor:)` method
* `DittoTransportSnapshot` - Leftover from previously removed APIs
### API Changes
**Query Arguments:**
* `DittoSyncSubscription.queryArguments` no longer guaranteed to be strictly equal to original arguments due to serialization roundtrip
* `DittoStoreObserver.queryArguments` no longer guaranteed to be strictly equal to original arguments due to serialization roundtrip
* Added `queryArgumentsCBORData` and `queryArgumentsJSONData` properties for custom decoding
* Added overloads accepting `Codable`-conforming arguments for subscriptions and observers
**Attachment APIs:**
* Added `DittoStore.newAttachment(data:)` - Create attachments from in-memory data without file URL
* Changed `DittoAttachment.len` type from `Int` to `UInt64`
**Disk Usage APIs:**
* Renamed `DiskUsage` to `DittoDiskUsage`
* Renamed `DiskUsageItem` to `DittoDiskUsageItem`
* Renamed `DiskUsage.DiskUsageObserverHandle` to `DittoDiskUsageObserver` and made it top-level
**Peer & Connection Properties:**
* Renamed `DittoPeer.peerKeyString` to `DittoPeer.peerKey`
* Renamed `DittoConnectionRequest.peerKeyString` to `DittoConnectionRequest.peerKey`
* Renamed `DittoConnection.peerKeyString1` to `DittoConnection.peer1`
* Renamed `DittoConnection.peerKeyString2` to `DittoConnection.peer2`
* Renamed `DittoPeer.osV2` to `DittoPeer.os`
**Authentication APIs:**
* Converted `DittoAuthenticationRequest` from Objective-C typealias to pure Swift class
* Converted `DittoAuthenticationSuccess` from Objective-C typealias to pure Swift class
* Renamed `DittoAuthenticationRequest.appId` to `DittoAuthenticationRequest.databaseID`
### Bug Fixes
* Fixed iOS mDNS occasionally being very slow to operate (#18682)
* Fixed mDNS failing to advertise/browse with correct info (#NETW-883)
***
# Full Changelog
## Swift Specific Changes
**Fixed**:
* iOS mDNS occasionally very slow to operate (#18682)
* mDNS failed to advertise/browse with correct info (#NETW-883)
**Changed**:
* The `queryArguments` property on `DittoSyncSubscription`s is no longer guaranteed to be strictly equal to the original arguments passed when registering the sync subscription. This is due to a serialization roundtrip, which may affect equality checks, particularly for non-primitive values. If you want to decode query arguments into a specific type, then use the `queryArgumentsCBORData` or `queryArgumentsJSONData` properties now available on `DittoSyncSubscription` instances and decode things as required (#17003)
* The `queryArguments` property on `DittoStoreObserver`s is no longer guaranteed to be strictly equal to the original arguments passed when registering the store observer. This is due to a serialization roundtrip, which may affect equality checks, particularly for non-primitive values. If you want to decode query arguments into a specific type, then use the `queryArgumentsCBORData` or `queryArgumentsJSONData` properties now available on `DittoStoreObserver` instances and decode things as required (#17003)
* `DittoAttachment.len` type from `Int` to `UInt64` (#18587)
* Minimum deployment targets are now macOS 12.0 and iOS 15.0 (#19096)
* Renamed `DiskUsage` to `DittoDiskUsage` to match the rest of the APIs (#19351)
* Renamed `DiskUsageItem` to `DittoDiskUsageItem` to match the rest of the APIs (#19351)
* Renamed `DiskUsage.DiskUsageObserverHandle` to `DittoDiskUsageObserver` and made it top-level to match the rest of the APIs (#19351)
* `DittoAuthenticationRequest` and `DittoAuthenticationSuccess` converted from Objective-C typealiases to pure Swift classes (#19683)
* `DittoAuthenticationRequest.appId` property renamed to `DittoAuthenticationRequest.databaseID` to match the updated terminology in `DittoConfig` (#19683)
* `DittoPeer.peerKeyString` property renamed to `DittoPeer.peerKey` (#SDKS-1186)
* `DittoConnectionRequest.peerKeyString` property renamed to `DittoConnectionRequest.peerKey` (#SDKS-1186)
* `DittoConnection.peerKeyString1` property renamed to `DittoConnection.peer1` (#SDKS-1186)
* `DittoConnection.peerKeyString2` property renamed to `DittoConnection.peer2` (#SDKS-1186)
* The default value of `DittoTransportConfig.listen.websocketSync` is now `false`. It must be explicitly set to `true` to enable websocket sync (#SDKS-1641)
* Renamed `DittoPeer.isConnectedToDittoCloud` to `DittoPeer.isConnectedToDittoServer` to reflect updated terminology (#SDKS-2189)
* `DittoPeer.osV2` property has been renamed to `DittoPeer.os`. The corresponding `DittoPeer` initializer parameter has also been updated accordingly
* Upgraded DittoSwift SDK to support Swift 6 (#17934)
* Marked all public classes as `final` (#17934)
* Marked most public types as `Sendable` (#17934)
**Added**:
* `queryArgumentsCBORData` and `queryArgumentsJSONData` properties to `DittoSyncSubscription` instances. If you want to decode query arguments into a specific type, then use these and decode things as required (#17003)
* An overload of `DittoSync`'s `registerSubscription` method that takes an `arguments` value that is `Codable`-conforming (#17003)
* `queryArgumentsCBORData` and `queryArgumentsJSONData` properties to `DittoStoreObserver` instances. If you want to decode query arguments into a type you can specify then use these and decode things as required (#17026)
* An overload of `DittoStore`'s `registerObserver` method that takes an `arguments` value that is `Codable`-conforming (#17026)
* `DittoStore.newAttachment(data:)` to create attachments directly from in-memory data, without requiring a file URL (#18587)
* Introduced dedicated typealias `DittoTransactionScope` for the `scope` closure of `DittoStore.transaction()` (#17934)
* Introduced dedicated typealias `DittoAttachmentFetchEventHandler` for the `onFetchEvent` closure of `DittoStore.fetchAttachment()` (#17934)
**Deprecated**:
* `DittoError.TransportErrorReason` and `DittoError.transportError` for future removal (#1969)
**Removed**:
* `Ditto.disableSyncWithV3()` as Ditto v5 no longer supports syncing with v3 (#19340)
* `DittoError.MigrationErrorReason` and its only `disableSyncWithV3Failed` case, since v3 sync is no longer available in Ditto v5 (#19340)
* The deprecated DittoObjC SDK has been removed and is not required as a dependency for DittoSwift anymore (#17923)
* CocoaPods support. Starting with version 5.0, DittoSwift is no longer distributed via Cocoapods. Ditto has moved fully to SwiftPM, as Cocoapods is effectively end-of-life and no longer maintained. Please integrate DittoSwift via SwiftPM going forward (#19635)
* `Ditto.transportDiagnostics()` and `DittoTransportDiagnostics`, as these APIs are now obsolete (#SDKS-812)
* `DittoTransportSnapshot`, a leftover from previously deprecated and already removed APIs (#SDKS-812)
* `DittoExperimental.jsonByTranscoding(cbor:)` method (#SDKS-855)
* All deprecated APIs
* `DittoAttachmentToken` class is obsolete and has been removed. Please use the dictionary variant instead
## 5.0.0 Common Changelog
**Performance**:
* Improved the underlying representation of Ditto documents for better performance (#17509)
* Document synchronization between peers now batches updates more efficiently, reducing processing time for large document sets (#19273)
* Document sync avoids sending large, redundant updates after reconnecting a long-dormant session between two peers that are otherwise well-synced with the mesh (#DS-433)
* Improved avoidance of large redundant doc sync updates after session reconnect, based on the number of diffs sent in the initial post-reconnect update (#DS-475)
* Faster initial sync when processing rkyv-encoded document diffs (#DS-773)
* Faster eviction of indexed documents when using rkyv (#DS-774)
* Improved performance of initial index generation when using rkyv as a document format (#DS-774)
* Reduced redundant replication GC work during rapid eviction bursts by debouncing and coalescing compatible GC tasks (#CORE-1466)
* Synchronization protocol enhanced to better avoid resetting sessions on reconnect if in-flight updates were lost (#DS-820)
* Document Sync now uses tiered blob store for update file storage by default, allowing smaller updates to avoid unnecessary disk I/O (#DS-836)
* Document sync implementation avoids forcing outbound update files to disk by default, decreasing disk I/O and improving sync latency (#DS-921)
* Observers to avoid sort by id on non order by query / add limit to sort operator (#QE-261)
* Improved out-the-box performance for larger statements (#QE-377 & QE-378)
* Improved the performance of IN-list evaluation for large lists of static values (#QE-386)
**Fixed**:
* An issue with BLE on some Android 9 and earlier devices that prevented connection establishment (#17760)
* Multiple concurrent DQL transactions can no longer possibly lead to a deadlock (#17816)
* A bug where x509 refresh could speed up uncontrollably (#17947)
* A bug where a peer could get stuck with incorrect connection information (#17967)
* Network connections close gracefully when Ditto is stopped (#18053)
* The `system:data_sync_info` collection may briefly report sync immediately after connecting (#18137)
* An issue where peers could fail to connect other local peers via mDNS on macOS (#18488)
* A bug that meant that document id indexes were not created (#18512)
* A bug where forced TCP connections would be retried more frequently than expected (#19727)
* A bug where mDNS registration would fail with `NamingConflict` (#20411)
* mDNS discovery should support TCP and UDP independently (#20490)
* Connection manager now correctly cleans up orphaned connecting state when tasks are cancelled during shutdown (e.g., during auth refresh), preventing "Already connecting" errors on reconnection (#20377)
* A deadlock could occur where a device would fail to establish new P2P connections until restarted (#20810)
* A race condition where `fetchAttachment` could permanently fail to find locally-created attachments due to stale in-memory cache entries (#21146)
* A bug that caused small peers to re-upload remotely requested files (e.g. logs) on every startup (#CORE-810)
* High latency when write transactions trigger evictions (#CORE-1453)
* Live queries being unable to use indices (#DS-447)
* A rare scenario where an attachment fetch could be delayed by up to 60 seconds (#DS-461)
* Deadlock in sync session metadata cleanup caused cascading lockups for doc sync and/or local doc store access (#DS-485)
* Post eviction cleanup of disconnected document sync sessions now retains metadata for non-evicted documents (#DS-487)
* Document sync session metadata became unlinked after session reset (#DS-509)
* DQL SELECT queries on indexed collections could deadlock if executed in an explicit transaction (#DS-648)
* DQL queries using indexes could yield wrong results upon document deletion (#DS-851)
* Inconsistent internal hashing of Document IDs could cause failure to sync documents (#DS-944)
* Premature connection cleanup causing duplicate connections (#NETW-1021)
* When another peer crashes, Ditto will stop attempting to connect to it in under 15 seconds. Previously, connection attempts would occur for up to 75 minutes at 5 second intervals (#NETW-856)
* Changed live queries to use the new streaming interface for query execution (#QE-261)
* Use of DISTINCT with ORDER BY & OFFSET/LIMIT (#QE-281)
* In BP profile directive does not produce path in collection scan operator profile (#QE-294)
* Handling of references to group keys that include array element selection (#QE-306)
* Use deterministic summaries for documents created as part of observer evaluation (#QE-310)
* Removed fabricated descriptor information from collection scans in profile/explain information (#QE-315)
* Performance of the DQL parser for large statements (#QE-321)
* DQL duration function floating-point rounding errors (#QE-402)
* Incorrect association of terms following an `IN expr` clause (#QE-418)
* Executing index scans in observers preserves document ids to provide consistent ordering (#QE-429)
* The query planner misses using a Filter operator if collection scan filter pushdown is enabled (#QE-437)
* Small peer collection scan handles offset and limit incorrectly (#QE-438)
* Changed DQL planner index selection process to correctly handle predicate paths that haven't been formalised avoiding incorrect index selection (#QE-456)
* The DQL query planner will no longer generate index scans on fields which have been specified in a COLLECTIONS clause as the variant specified may differ from the indexed variant. This prevents incorrect query results arising from the mismatch (#QE-475)
* The small peer DQL query planner to not produce index scans when DQL\_STRICT\_MODE is set to true. This avoids incorrect results from filters on fields where the latest variant is not REGISTER (#QE-478)
* The store SQLITE logging callback has been changed to not report schema change errors avoiding flooding the logs with spurious warnings from SQLITE internal operations (#QE-479)
* A panic in the DQL parser when an invalid JSON directive is used prior to the PROFILE keyword (#QE-490)
* -9223372036854775808 is now parsed and handled as an integer value by DQL (#QE-499)
* Queries in the legacy language might fail to surface documents with settable counters (#QE-512)
* Documentation for on-disk logger incorrectly stated logs are retained for 3 days; actual retention is 15 days (#SDKS-1726)
* FFI resource cleanup now safely handles null context pointers in release callbacks, preventing potential null pointer dereferences during Ditto shutdown (#SDKS-2744)
* Attachment callback dispatch now properly handles thread pool dispatch failures by retaining context to prevent use-after-free errors when the callback pool is unavailable (#SDKS-2744)
* Transport watchdog no longer logs spurious "Address already in use" errors when restarting TCP/HTTP servers. The watchdog now waits for previous server tasks to fully exit and release their sockets before attempting to rebind ports, preventing EADDRINUSE races when servers are invalidated during e.g. app backgrounding (#SPO-127)
* Attachment garbage collection now removes empty shard directories, preventing inode leaks (#SPO-158)
* Bug which meant a Ditto starting log failed to be logged (#SPO-626)
* Android devices that fail to acquire an IP address will now continue to sync over LAN (#TRAN-725)
* A long-running on connecting callback no longer causes `ReceiveTimeout` failures in connectivity (#TRAN-729)
**Changed**:
* `network_enable_multihop` system parameter renamed to `network_enable_ngn`, gating NGN features which include multihop (#17882)
* Enabled opentelemetry tracing in core (#18478)
* Connection failures now use smart log levels, starting at warning and escalating to error only after repeated failures (#18779)
* mDNS now only advertises connectable addresses based on TCP server binding address. TCP Client now attempts to connect to all addresses in parallel (#18930)
* The default value of `ENABLE_ATTACHMENT_PERMISSION_CHECKS` store configuration parameter to `false` (#18931)
* mDNS service name is now configurable via the `transports_mdns_service_name` SystemParameter (default: `_http-alt`), ensuring both TCP and UDP transports use the same service name (#20289)
* Panic messages now filter out internal Rust machinery frames (backtrace capture, panic handling, runtime startup) for improved readability, with an environment variable `DITTO_PANIC_WITH_FULL_STACK_TRACE=1` available to show complete stack traces when needed (#20401)
* On-disk logs now make better use of available disk space by resuming writing to incomplete files (#CORE-561)
* On-disk log file limits now apply to the compressed size, significantly increasing log retention (#CORE-729)
* Write transaction diagnostic logs now include the blocking transaction's current operation, elapsed time, and queue depth (#CORE-1449)
* More information is logged at debug level around peer GC and eviction progress including what remote peers are being deleted (#CORE-1455)
* Significantly lowered the maximum values of some system parameters governing the on-disk rotating file logger's behavior (#CORE-848)
* More information is logged when a Ditto peer receives data that it cannot deserialize due to a hash mismatch (#CORE-897)
* We now make an additional attempt to recover from some kinds of document deserialization errors, which may reduce errors or crashes due to deserialization (#CORE-916)
* Document sync protocol now supports rkyv-encoded diffs, for more efficient initial synchronization (#DS-531)
* TCP server is no longer auto-enabled when TCP is disabled in config, NGN and UDP is enabled with LAN discovery (#NETW-1039)
* Mutators are preserved for Big Peer V4 (#QE-126)
* Disallow null as an id (#QE-202)
* DQL\_STRICT\_MODE default to false (#QE-267)
* Planning for statements using DISTINCT projections, in some circumstances (#QE-282)
* system:vitals outputs timers as sub-document (#QE-312)
* DQL queries run via observers now require that the user provides stable ordering themselves via a suitable ORDER BY clause (#QE-427)
* Double quotes delimit strings, not identifiers (JSON compatibility) (#QE-44)
* DQL query planner now recognises additional index access plans that eliminate the need for document retrieval (#QE-449)
* Added the key sorting direction to the `system:indexes` virtual collection output (#QE-467)
* The DQL planner to automatically convert equality filters on the `_id` field to ID scans, improving performance by skipping index use when exact document IDs are known up front (#QE-469)
* The DQL query planner can now create a query plan that defers fetching full documents until after sorting and application of offset and limit, if the index access portion of the plan can support it. This means performance improvements for queries with affected plans (#QE-473)
* Revised the Query Engine DISTINCT operator to stream results reducing memory overhead (#QE-474)
* The DQL planner to consider additional cases where an index may cover a query leading to improved performance in those scenarios (#QE-491)
* The DQL Query Engine planner can now produce specialised higher performance plans for Ditto Server `COUNT(*)` queries that include simple filters (#QE-510)
* The Query Engine DQL planner can now generate index-access plans against Ditto Server improving performance queries with filters that can be applied when scanning an index and where combining multiple index scans is beneficial (#QE-511)
* The Query Engine intersect scan operator can now stop before all inputs have completed, once it has been established that no further complete intersections can be produced, improving the performance where the number of values produced by each input differs greatly (#QE-530)
* Added specialisation of simple queries to take advantage of small peer indexing (#QE-266)
* Added support for microseconds to duration scalar functions (#QE-396)
* Removed the Parseable query trait to minimise the impact of the query parser maintenance on the monorepo (#QE-421)
* System collection `system:ditto_metrics` renamed to `system:metrics` for consistency with naming conventions. Existing queries using `system:ditto_metrics` will need to be updated to use `system:metrics` (#SDKS-2653)
* Ditto shutdown logging promoted to info level (#SPO-626)
**Added**:
* `ENABLE_ATTACHMENT_PERMISSION_CHECKS` ALTER SYSTEM parameter to be set to `false` to avoid certain rare cases of attachment fetcher hanging (#18016)
* Bring Your Own Discovery now supports WebSocket connections (#18965)
* `system:metrics` virtual collection for DQL access to metrics (#MESHCON-53)
* Introduced new entries to the `system:system_info` collection for `peer_key` and `database_id` (#19435)
* `DATA_SYNC_ENABLED` system parameter which, while set to `false`, halts the data sync machinery, but without loss of network connectivity (allowing for operations such as remote query) (#19643)
* `DITTO_USE_TIERED_BLOB_STORE_DOC_SYNC` system parameter env var to improve the performance of doc sync in heavy mesh scenarios, at the expense of sync chatter overhead upon restart (#19847)
* The new `transport_tcp_connect_timeout` system parameter which shortcuts long platform-based TCP connection timeouts (#20782)
* Explicit log flushing before aborting due to panic (#CORE-723)
* `system:system_info` virtual collection for DQL access to system info (#CORE-751)
* small\_peer\_info subscription queries now include arguments (#DS-1027)
* system:system\_info subscription queries now include arguments; query location moved from key to value.query (#DS-1027)
* Improved logging for document deserialization errors (#DS-568)
* System parameters that can be used to configure sqlite pragmas (#DS-682)
* System parameter `doc_sync_outbound_update_fsync_policy` to govern the use of fsyncs when creating doc sync update files (#DS-817)
* Channel open retries and watchdog to tear down VirtualConnections that never open a channel, preventing resource exhaustion from stalled peers (#NETW-1098)
* A system parameter `transports_dns_sd_backend` for choosing the mDNS backend implementation (#NETW-940)
* Ability to specify documents to insert in arrays in DQL (#QE-118)
* USE IDS LIST syntax (#QE-119)
* Support for settable counters to DQL (#QE-130, QE-393, QE-394, QE-395)
* Specialisation of COUNT(\*) DQL queries on the big peer (#QE-138)
* Subscription queries reject LIMIT and ORDER by unless DQL\_RESTRICT\_SUBSCRIPTION is set to false (#QE-218)
* INTERSECT and UNION (index) scans (#QE-230\_and\_240)
* Covering index scans (#QE-239)
* Support for the BETWEEN DQL expression (#QE-26)
* Streaming interfaces for query execution (#QE-262)
* Array & object search syntax to DQL (#QE-265)
* Ability to inline consumers into producer operators (#QE-272)
* Consolidated results across subservers for queries against ACTIVE\_REQUESTS, REQUEST\_HISTORY and VITALS (#QE-273)
* Array transformation DQL syntax (#QE-274)
* Object transformation syntax to DQL (#QE-276)
* Promethus metrics to the DQL Query engine (#QE-296)
* Automatic generation of USE IDS clause from \_id equality predicates when possible (#QE-313)
* A shared statement cache and virtual collection (#QE-316)
* Configurable concurrent request limit to the Query engine (#QE-317)
* Periodic dumping of request history cache entries to the log (SP only) (#QE-322)
* The ability for the shared statement to store, verify, amend and use existing plans for qualifying statements (#QE-324)
* PROFILE keyword as the equivalent to the #profile directive (#QE-336)
* CASE statement (#QE-41)
* Support for settable counters in INSERT DQL statements (#QE-406)
* The statement caches invalidates existing statements if the default directives change (#QE-407)
* Support for extended string literals that can contain escape sequences in DQL (#QE-410)
* Support for hexadecimal numeric constants in DQL statements (#QE-413)
* Statement cache resizing (#QE-414)
* The ability for Remote Query to handle DELETE, EVICT, TOMBSTONE DQL statement (#QE-441)
* Enable sending all statements via the SYNC CONTEXT statement (#QE-447)
* Support for DQL DELETE against the `system:shared_statements` virtual collection (#QE-450)
* Publish sync scopes in small peer info document (#SPO-276)
**Removed**:
* The static path option has been removed from TransportConfig. Please use attachments to serve content instead (#TRAN-256)
* Deprecated fields in presence have been removed across all SDKs (`queryOverlapGroup`, `meshRole`, `approximateDistanceInMeters`, `rssi`) (#TRAN-680)
## Related topics
- [Swift Quickstart](/sdk/latest/quickstarts/swift.md)
- [C++ Release Notes](/sdk/latest/release-notes/cpp.md)
- [C# Release Notes](/sdk/latest/release-notes/c-sharp.md)