Protocol and Limits
Python and Node.js share one Rust core, so these limits apply to both unless a row names a binding.
Protocol
| Boundary | Behavior in 0.1.8 |
|---|---|
| Handshake | Sends user:password, requests IPC capability 6, and accepts any reply of 1 or higher. A closed connection during the handshake is reported as an authentication failure. |
| Byte order | Little-endian only. A big-endian message is rejected. |
| Message size | Uses the 40-bit length field of the q header, so a message can be up to 1,099,511,627,775 bytes (240 − 1) in either direction. |
| Message types | sync() sends synchronous requests (type 1) and asyn() sends asynchronous messages (type 0). receive() returns the next incoming message of any type. |
| Arguments | At most 8 per call. A {...} expression called with arguments is sent as a lambda. |
| Nesting | Values may nest at most 64 levels deep, in both directions. |
| Socket | TCP with TCP_NODELAY. Every address the host name resolves to is tried within the connect timeout. |
Compression
| Direction | Behavior |
|---|---|
| Responses from q | q IPC compression modes 1 and 2 are decompressed. A compressed message is rejected if its declared size could not come from the bytes received. Each 17 compressed bytes expand to at most 2,056, so the bound is below 121 times. |
| Requests to q | A request with arguments is compressed only when it is at least 10,000,000 bytes long and the host is not localhost or 127.0.0.1. String-only queries are never compressed. |
serialize_as_ipc_bytes6, serializeAsIpcBytes6 | Compress only when asked to, and only messages of at least 2,000 bytes |
| All compression | The compressed form is used only when it is less than half the original size. Otherwise the message is sent uncompressed. |
Memory
- No fixed size limit on results. A result is bounded by available memory and by the 40-bit length field.
- Reading is paced by delivery. A declared message length reserves at most 32 MiB up front. After that the buffer grows by up to eight times, and only once the server has filled the previous reservation. Memory committed to a message therefore stays within eight times the bytes actually received.
- Allocation failures are errors. Every allocation whose size comes from wire data or file data is fallible. A result too large for the machine returns an error rather than ending the Python interpreter or the Node.js worker. This holds from 0.1.6 onward. 0.1.5 covered the large allocations only.
- Broken frames close the connection. If xqdb cannot read a frame, its header is invalid, or it cannot be decompressed, the call fails and xqdb closes the socket. A complete frame that fails to decode, such as an unsupported type, text that is not valid UTF-8 under the strict policy, or a q error, raises an error and leaves the connection open.
Binary table files
| Boundary | Behavior in 0.1.8 |
|---|---|
| File kinds | A table or keyed table saved as a single file. Directories, which hold splayed tables and partitioned databases, are rejected. |
| q file compression | LZ4 (algorithm 4) only. Other algorithms are rejected. |
| Compression bounds | A declared decompressed size is rejected if the file's own LZ4 blocks could not produce it. Each block allows the smaller of its block size and 255 times its compressed bytes. |
| Size | Read whole into memory, with no fixed limit |
| Paths (Node.js) | Windows UNC and device paths are rejected |
Connections
| Boundary | Python | Node.js |
|---|---|---|
| Socket timeout | Whole seconds, 0 (the default) for none | 1 second to 24 hours, default 30 seconds. Milliseconds are rounded up to whole seconds. |
| Retries | Retries sync, asyn, and receive after I/O errors, with 1, 2, 4… second sleeps, and re-sends the request | Retries only explicit connect() after XQDB_IO, with no delay |
| Concurrency | One thread at a time per Q. The GIL is released during I/O and decoding. | One running command plus 8 waiting per Q, then XQDB_BACKPRESSURE |
| Per-call size | No separate limit | Expression up to 64 MiB, and arguments up to 64 MiB after native snapshot accounting |
| TLS | rustls, TLS 1.2 and 1.3, operating system trust store, hostname verification, no client certificates | Same |
Supported platforms
| Package | Platforms |
|---|---|
xqdb on PyPI | cp310-abi3 wheels for win_amd64, manylinux_2_28_x86_64, and macosx_11_0_arm64, plus a source distribution |
@xbbg/xqdb on npm | win32-x64-msvc, linux-x64-gnu (glibc 2.28 or newer), and darwin-arm64 native packages |
Details: Connections and TLS, Timeouts and Retries, Node.js Workers and Backpressure, and Files and IPC Bytes.