Changelog
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
Unreleased
Changed
- In-house gRPC transport replaces
tonicon the MDDS server-streaming path. The SDK now drivesh2directly: prost encode → length-prefix frame → HTTP/2 DATA → response stream → trailers parse, with no tower stack, no boxed bodies, and noasync-traitdyn dispatch. New public modulethetadatadx::grpc::*exposesChannel,ChannelPool,ChannelLease,ServerStreaming,Codec,Status,DecoderPool,DecoderHandleand the matching error types (ChannelError,CodecError,StatusParseError,DecoderPoolError,DecoderSubmitError). Error::Transportpayload changed fromtonic::transport::ErrortoString. Pattern matches against the wrapped tonic type no longer compile; consumers that key on transport-level failures match the string variant directly.ChannelPool::next()now returns aChannelLease<'a>instead of&'a Channel. The lease pre-reserves an in-flight slot on the picked channel synchronously so concurrent burst dispatches (join_all-style) observe each reservation immediately and route around loaded channels. The lease derefs to&Channel; the typicalpool.next().server_streaming(...).awaitshape stays unchanged, but callers that bind the lease to aletto thread it acrossawaitpoints pass&leaseinto a function that takes&Channel(Deref coercion does the projection).Status::from_trailersnow tolerates malformedgrpc-messagetrailers per the gRPC HTTP/2 spec. The parser percent-decodes (RFC 3986,%HHescapes only — invalid escapes like%2Xare passed through literally bypercent-encoding); if the decoded bytes are valid UTF-8 they become the message. If percent-decode produces non-UTF-8 bytes (e.g.%FF), the parser falls back to the raw header bytes interpreted as UTF-8. If those bytes are also non-UTF-8 (opaque header value) the message is empty. Previously the parser returnedStatusParseError::MessageNotUtf8on any non-UTF-8 path, which a spec-conformant client must not surface — a malformed message must not invalidate a parsedgrpc-status. Concrete examples:Hello%20world→"Hello world";%FF→"%FF"(raw fallback); opaque non-UTF-8 →"".
Added
MddsConfig::decoder_threadsandMddsConfig::decoder_ring_sizecontrol the dedicated decoder pool that runs zstd decompress + protobuf decode off the tokio reactor.decoder_threads = 0auto-sizes tomin(channels, available_parallelism / 2);decoder_ring_sizemust be a power of two>= 64. Both fields are required when constructingMddsConfigliterally — production code should useMddsConfig::production_defaults().DecoderHandle::submitnow returnsResult<oneshot::Receiver<DecodeResult>, DecoderSubmitError>. Submits made after a worker-thread panic poisoned the pool fail fast withDecoderSubmitError::Poisonedrather than parking the caller on a dead consumer ring.ChannelErrorvariant routing: connection-level h2 failures —GOAWAY(either direction), IO failure on the h2 transport, peer shutdown, and open-phase connection drops (failures observed onready()/send_request()/send_data()while admitting the stream) — now surface asChannelError::ConnectionClosed. TheConnectionClosedDisplay string changed from"h2 connection closed by GOAWAY: ..."to"h2 connection closed: ..."to reflect the broader scope.ChannelError::H2Streamis now scoped strictly to per-streamRST_STREAM(any reason code) and h2 library-detected stream-scoped protocol errors. Callers that key retry / recycle policy offChannelErrormay need to remap branches: anything that previously matchedH2Streamfor connection-level decisions now belongs on theConnectionClosedarm.
Removed
tonicdependency removed. Theinhouse-grpcfeature flag is also gone — the in-house transport is the only path. Direct uses oftonic::transport::Channel,tonic::Status, ortonic::Streamingthroughthetadatadxre-exports are no longer available.MddsClient::stubwas removed. Internal call sites now reach the generated stubs throughproto::beta_theta_terminal::*directly; the field was crate-private but listed here for transparency for any caller relying onpub(crate)access.GrpcStatusKind::from_code()was renamed toGrpcStatusKind::from_u32()to match the wire type. The enum repr is nowu32(wasi32) so match-arms keyed on integer literals continue to compile; explicit casts may need to be removed.StatusParseError::MessageNotUtf8was removed. Malformedgrpc-messageno longer fails the trailers parse (see the Changed entry onStatus::from_trailers); exhaustive matches onStatusParseErrorneed to drop the variant. The replacement is best-effortStatus::message()with raw / empty fallback.
Migration
- Replace any
tonic::transport::Errorpattern onError::Transportwith the string payload, e.g.Error::Transport(msg) if msg.contains("..."). - Replace
GrpcStatusKind::from_code(n)withGrpcStatusKind::from_u32(n). - When constructing
MddsConfigfield-by-field, adddecoder_threads: 0, decoder_ring_size: 256(or callMddsConfig::production_defaults()). - Update
matcharms onStatusParseError— drop theMessageNotUtf8branch. The parser now returns a usableStatuswhosemessage()may be empty for non-UTF-8 trailer bytes. pool.next()callers that bind the result across anawaitmust keep the lease alive for the dispatch window. Pattern:let lease = pool.next(); stub_fn(&lease, req).await?;— storing the lease in a local rather than as a temporary inside the same expression keeps the pre-dispatch reservation committed until the open path's own in-flight token takes over.DecoderHandle::submitnow returnsResult<_, DecoderSubmitError>. Update callers fromlet rx = handle.submit(r); rx.awaittolet rx = handle.submit(r)?; rx.await(or surface thePoisonedvariant as a transport-level error as the SDK's owndecode_chunkhelpers do).
[10.0.0] - 2026-05-09
Semver-honest version bump for the v9.1.0 surface. The v9.0.x → v9.1.0 wave introduced 12 major API breaks per cargo-semver-checks (subscribe_*-family removal, polymorphic subscribe(spec), Contract::option arity change, Error enum reshape, FpssData::* contract_id removal in favour of typed Arc<Contract>, ThetaDataDx → ThetaDataDxClient rename, mdds::decode::v3 → mdds::decode::dual_type_columns module rename, IntoOptionSpec trait removal, FpssConnectArgs field additions, FpssConfig.queue_depth removal, flat TdxFpssControl → typed per-variant structs, Go SDK removal). Rust semver classifies that diff as a major bump.
v10.0.0 is the v9.1.0 surface with no further code changes — bumping the version number to align with semver discipline. The audit chain on the v9.1.0 wave (cargo fmt, clippy --workspace -- -D warnings, test --workspace, deny check, generate_sdk_surfaces --check, npm test 19/19, C++ CMake build, Python wheel + pytest) all passed; no findings remain after the closeout chain.
Changed
- Project version: 9.1.0 → 10.0.0 across
crates/thetadatadx,crates/tdbedependents,ffi,tools/{cli,mcp,server},sdks/{python,typescript}. tdbe stays at 0.13.1 (no API change in this bump). All standalone Cargo.lock files re-locked.
Migration from v9.1.0
No source-level changes required. Update the version pin:
| Surface | v9.1.0 | v10.0.0 |
|---|---|---|
Cargo.toml | thetadatadx = "9" | thetadatadx = "10" |
pyproject.toml / requirements.txt | thetadatadx>=9.1.0,<10 | thetadatadx>=10.0.0,<11 |
package.json | "thetadatadx": "^9.1.0" | "thetadatadx": "^10.0.0" |
| C++ pin | cargo build --release -p thetadatadx-ffi from v9.1.0 tag | v10.0.0 tag |
Added
- Fluent contract-first streaming API.
Contract::stock("AAPL"),Contract::option("SPY", "20260620", "550", "C"), and thecontract.quote()/.trade()/.open_interest()methods return a typedSubscriptionvalue. Full-stream subscriptions come fromSecType::Option.full_trades()/SecType::Option.full_open_interest(). The new polymorphicclient.subscribe(Subscription),client.subscribe_many([...]),client.unsubscribe(Subscription), andclient.unsubscribe_many([...])onThetaDataDxClient(Rust),ThetaDataDxClient(Python pyclass),ThetaDataDxClient(TypeScript napi), andtdx::UnifiedClient/tdx::FpssClient(C++) accept that value type directly. - Polymorphic C ABI: new
tdx_unified_subscribe/tdx_unified_unsubscribe/tdx_fpss_subscribe/tdx_fpss_unsubscribetake aTdxSubscriptionRequestpayload; one entry point handles every per-contract or full-stream variant. AsyncThetaDataDxClientPython class — async-only sibling ofThetaDataDxClient. Attribute access is restricted to*_asynchistorical methods plus the streaming lifecycle helpers; the synchronous historical surface raisesAttributeErrorso callers that opt into the async path do not accidentally block on a sync method.thetadatadx::preludeRust module — re-exportsCredentials,ThetaDataDxClient,Contract,Subscription,SecTypeExt,SecType, etc. for a one-import fluent path.
Changed
- Public client name: previous unified-client struct name is gone (no alias, no compat shim). Every binding ships only
ThetaDataDxClient(Rust struct, Python pyclass, TypeScript napi class). - Python streaming:
client.streaming(on_event)context manager is the recommended path; the bound session forwards every publicThetaDataDxClientmethod through__getattr__, so the new polymorphicsubscribe/unsubscribeare reachable on the session with zero hand-listed mirror.
Removed
Hard break — the typed subscribe / unsubscribe surface is gone. Every typed subscribe_* / unsubscribe_* and subscribe_option_* entry on the public client (Rust, Python, TypeScript, C++) plus the matching typed C ABI entry points (tdx_unified_subscribe_*, tdx_fpss_subscribe_*, tdx_unified_unsubscribe_*, tdx_fpss_unsubscribe_*, including the option-overload variants) have been deleted. Replacement is the polymorphic subscribe(Subscription) / unsubscribe(Subscription) / subscribe_many([...]) / unsubscribe_many([...]) paths.
Migration map (documentation only — no compat layer ships):
| Removed | Wave K replacement |
|---|---|
Rust: client.subscribe_quotes(&c) | client.subscribe(c.quote()) |
Rust: client.subscribe_trades(&c) | client.subscribe(c.trade()) |
Rust: client.subscribe_open_interest(&c) | client.subscribe(c.open_interest()) |
Rust: client.subscribe_full_trades(SecType::Option) | client.subscribe(SecType::Option.full_trades()) |
Rust: client.subscribe_full_open_interest(SecType::Option) | client.subscribe(SecType::Option.full_open_interest()) |
Rust: client.subscribe_all(&c) (quotes + trades batcher) | client.subscribe_many(vec![c.quote(), c.trade()]) |
Python: tdx.subscribe_quotes("AAPL") | tdx.subscribe(Contract.stock("AAPL").quote()) |
Python: tdx.subscribe_option_trades("SPY", e, k, r) | tdx.subscribe(Contract.option("SPY", expiration=e, strike=k, right=r).trade()) |
Python: tdx.subscribe_full_trades("OPTION") | tdx.subscribe(SecType.OPTION.full_trades()) |
TS: tdx.subscribeQuotes("AAPL") | tdx.subscribe(ContractRef.stock("AAPL").quote()) |
TS: tdx.subscribeFullTrades("OPTION") | tdx.subscribe(SecType.option().fullTrades()) |
C ABI: tdx_unified_subscribe_quotes(h, sym) | tdx_unified_subscribe(h, &TdxSubscriptionRequest{...}) |
C ABI: every tdx_*_subscribe_* / tdx_*_unsubscribe_* typed entry point | tdx_*_subscribe / tdx_*_unsubscribe (polymorphic) |
C++: fpss.subscribe_quotes("AAPL") | fpss.subscribe(tdx::Contract::stock("AAPL").quote()) |
[9.1.0] - 2026-05-07
Single-queue SSOT for the FPSS streaming pipeline (closes #513). The prior topology composed two queues — the LMAX Disruptor ring plus an internal crossbeam_channel::bounded(8192) — with a per-tick FpssEvent::clone between them. The start_streaming path now invokes the user callback directly from the Disruptor consumer thread.
Migration from v9.0.x
The flat TdxFpssControl { kind, id, detail } C ABI envelope is replaced by one typed #[repr(C)] struct per FpssControl::* Rust variant. Old code dispatched on event.control.kind then read event.control.id / event.control.detail; new code dispatches on event.kind then reads the matching event.<variant> payload. Field-by-field mapping for every control variant:
| v9.0.x (flat envelope) | v9.1.0 (typed struct) |
|---|---|
kind == TDX_FPSS_CONTROL && control.kind == 0; control.detail | kind == TDX_FPSS_LOGIN_SUCCESS; login_success.permissions |
kind == TDX_FPSS_CONTROL && control.kind == 1; control.id, control.detail | kind == TDX_FPSS_CONTRACT_ASSIGNED; contract_assigned.id, contract_assigned.contract |
kind == TDX_FPSS_CONTROL && control.kind == 2; control.id, control.detail | kind == TDX_FPSS_REQ_RESPONSE; req_response.req_id, req_response.result |
kind == TDX_FPSS_CONTROL && control.kind == 3 | kind == TDX_FPSS_MARKET_OPEN |
kind == TDX_FPSS_CONTROL && control.kind == 4 | kind == TDX_FPSS_MARKET_CLOSE |
kind == TDX_FPSS_CONTROL && control.kind == 5; control.detail | kind == TDX_FPSS_SERVER_ERROR; server_error.message |
kind == TDX_FPSS_CONTROL && control.kind == 6; control.detail (formatted) | kind == TDX_FPSS_DISCONNECTED; disconnected.reason (i32 RemoveReason) |
kind == TDX_FPSS_CONTROL && control.kind == 8; control.id, control.detail | kind == TDX_FPSS_RECONNECTING; reconnecting.reason, reconnecting.attempt, reconnecting.delay_ms |
kind == TDX_FPSS_CONTROL && control.kind == 9 | kind == TDX_FPSS_RECONNECTED |
kind == TDX_FPSS_CONTROL && control.kind == 10; control.detail | kind == TDX_FPSS_ERROR; error.message |
kind == TDX_FPSS_CONTROL && control.kind == 11; control.id, control.detail (hex) | kind == TDX_FPSS_UNKNOWN_FRAME; unknown_frame.code, unknown_frame.payload, unknown_frame.payload_len |
kind == TDX_FPSS_CONTROL && control.kind == 12 | kind == TDX_FPSS_UNKNOWN_CONTROL |
kind == TDX_FPSS_CONTROL && control.kind == 13 | kind == TDX_FPSS_CONNECTED |
kind == TDX_FPSS_CONTROL && control.kind == 14; control.detail (hex) | kind == TDX_FPSS_PING; ping.payload, ping.payload_len |
kind == TDX_FPSS_CONTROL && control.kind == 15 | kind == TDX_FPSS_RECONNECTED_SERVER |
kind == TDX_FPSS_CONTROL && control.kind == 16 | kind == TDX_FPSS_RESTART |
Field-by-field mapping for the data-variant contract_id removal and the hidden internal-only RawData / Empty variants:
| v9.0.x | v9.1.0 |
|---|---|
event.quote.contract_id (i32, wire-internal) | event.quote.contract.symbol (and expiration / strike / is_call for options) — same for trade, open_interest, ohlcvc |
Rust: FpssData::Quote { contract_id, contract, .. } | FpssData::Quote { contract, .. } (id removed) |
Python: event.contract_id | event.contract.symbol |
TypeScript: event.quote.contract_id | event.quote.contract.symbol |
C: event.quote.contract_id | event.quote.contract.symbol (NUL-terminated, may be null pre-ContractAssigned) |
C++: event.quote.contract_id | event.quote.contract.symbol |
FpssEvent::RawData { code, payload } matched on user callback | Removed; truncated FIT frames bump thetadatadx.fpss.decode_failures and never reach the callback. Unrecognised wire codes still surface as FpssControl::UnknownFrame { code, payload } (typed control variant). |
FpssEvent::Empty ring-slot placeholder visible to user code | Removed; ring slots use the crate-private FpssEventInternal::Empty, filtered before user delivery. |
Numeric values of TdxFpssEventKind renumber alphabetically; reach for the symbolic names (TDX_FPSS_LOGIN_SUCCESS in C, FpssLoginSuccessEvent in Go) — they are stable across the rename. C++ consumers using tdx::Fpss<Variant> aliases get the same borrowed-pointer ownership rules as before: pointers are valid only for the duration of the user callback. Python and TypeScript consumer code does not change.
Changed
- FPSS control events surface as typed-per-variant classes across every language binding — Python, TypeScript, AND the C / C++ / Go FFI surface — mirroring the Rust
FpssControlenum one-for-one. Replaces the previous flattenedSimpleevent type and the flatTdxFpssControl { kind, id, detail }C ABI. Python users dispatch viamatch event: case LoginSuccess(permissions=p): ... case Disconnected(reason=r): ...; TypeScript users dispatch via the discriminated union'skindfield with one typed payload per variant (event.loginSuccess,event.disconnected,event.reconnecting, ...). C consumers dispatch viaevent->kindinto the matchingevent-><variant>payload (event->login_success.permissions,event->disconnected.reason,event->reconnecting.{reason, attempt, delay_ms}, ...). C++ consumers read the same fields through the re-exportedtdx::Fpss<Variant>aliases; Go consumers read the matchingevent.<Variant>pointer (event.LoginSuccess.Permissions,event.Disconnected.Reason, ...). TheTdxFpssEventKindenum gains one discriminant per control variant — numeric values renumber alphabetically; symbolic names (TDX_FPSS_LOGIN_SUCCESS,FpssLoginSuccessEvent, etc.) are stable. Schema bumped to version 5 (crates/thetadatadx/fpss_event_schema.toml); generated outputs regenerated; codegen idempotency check enforced in CI. See the v9.0.x → v9.1.0 migration table for the old→new field mapping. ThetaDataDxClient::start_streamingnow invokes the user callback directly from the LMAX Disruptor consumer thread, with each invocation wrapped instd::panic::catch_unwind. There is exactly ONE queue between the TLS reader and the user callback (the Disruptor ring); the per-tickFpssEvent::cloneshim inclient.rsis gone.dropped_event_count()keeps the same public signature but now reportsProducer::try_publishfailures (ring-buffer overflow when the consumer falls behind) instead ofcrossbeam_channel::Fullrejections.- The TLS reader uses
Producer::try_publishfor every data event so a slow user callback can never block the reader. Handshake-time control frames (Connected,Ping,LoginSuccess,Reconnecting,Disconnected) keep the originalpublishsemantics so wire-order ordering relative toLoginSuccessis preserved. tdx_unified_freeandtdx_fpss_freenow apply the drain barrier internally before destroying the handle._freecalls the equivalent ofstop_streaming(orshutdownfor FPSS) and then polls the drain flag with a 5-second timeout; on overrun it logs atracing::error!and proceeds. Callers no longer need to call_await_drainbefore_freeto keep the callbackctxalive. The C++ wrapper'sFpssClientmove-assign now invokestdx_fpss_await_drainbetweentdx_fpss_shutdownand releasing the stagedstd::functionstorage, closing an analogous use-after-free window.- The
FpssConfigtuning knobstimeout_ms,connect_timeout_ms, andping_interval_msare now wired into the runtime. Previous releases shipped these as no-op fields whose values were ignored — the FPSS pipeline used hardcoded protocol-level constants (READ_TIMEOUT_MS,CONNECT_TIMEOUT_MS,PING_INTERVAL_MS) for every connection. Each knob now flows throughFpssConnectArgsto the connection (TCPconnect_timeout), framing (mid-frame stall budget + I/O loop overall deadline), and ping-heartbeat layers, and validates its range at config-load time:timeout_ms[100, 60_000],connect_timeout_ms[1_000, 60_000],ping_interval_ms[100, 300_000].DirectConfig::validatenow returnsResult<Self, Error>; the production / dev / stage presets remain infallible by construction. The redundant pre-DisruptorFpssConfig::queue_depthknob (and itsfpss_queue_depth()accessor) is removed: the post-SSOT pipeline has exactly one queue (the Disruptorring_size), so a separate event-channel-depth knob is dead. TOML configs that set[fpss] queue_depth = ...should switch toring_size. - The cross-language response-shape agreement validator (
scripts/validate_agreement.py) now consumes a TypeScript shape manifest alongside the Python / CLI / C++ runtime artifacts. The TS SDK emits its public-surface field set fromindex.d.tsviasdks/typescript/scripts/emit_validator_manifest.mjs; the diff engine treats shape-only artifacts as field-presence-only (values do not contribute to value-vs-value diffs, status-PASS entries do not fold into runtime status disagreements). Any TypeScript public-surface drift relative to the runtime SDKs now surfaces as a pre-merge agreement failure rather than going unnoticed until a downstream consumer hit the missing / extra field. - Repo hygiene pass. Root tree trimmed to standard institutional shape (matches the databento-rs layout): moved
ROADMAP.md→docs/, movedconfig.default.toml→crates/thetadatadx/, deleted unusedcliff.toml. Architecture ADRs inlined into source-code comments at their relevant locations;docs/architecture/removed. Generated SDK files moved to_generated/subdirectories under each SDK (sdks/python/src/_generated/,sdks/typescript/src/_generated/) so hand-written code leads the public surface listing — the SSOT codegen for cross-language parity is preserved unchanged, only the output paths moved.docs/java-parity-checklist.mdremoved (historical artifact; parity is shipped). - Typed
Errorenum:Error::Decode,Error::Decompress,Error::Config, andError::Grpcnow carry structuredkindfields (DecodeErrorKind,DecompressErrorKind,ConfigErrorKind,GrpcStatusKind) instead of bareStringpayloads. Callers can pattern-match on the kind for programmatic recovery without parsing error messages — e.g. distinguish aDecodeErrorKind::TruncatedRow { row_idx, expected_columns, actual_columns }from aProtobuf(String)codec failure, or branch onGrpcStatusKind::DeadlineExceededvs.Unauthenticatedwithout re-implementing thetonic::CodeDebug-string mapping.From<tonic::Status>populatesError::Grpc { kind: GrpcStatusKind::from_code(s.code()), .. }so the retry classifier and Python exception mapper share the same typed dispatch path. Migration: replaceif let Error::Decode(msg) = err { ... }withif let Error::Decode { kind, message } = err { match kind { DecodeErrorKind::TruncatedRow { .. } => ..., _ => ... } };Error::Config(format!(...))constructions becomeError::config_invalid(field, message)/Error::config_out_of_range(field, value, min, max)/Error::config_missing(field)etc. (helper constructors onError). The Pythonto_py_errmapper preserves its existingthetadatadx.SchemaMismatchError/RateLimitError/SubscriptionErrorleaf classes — only the internal dispatch switched from string-comparingstatusto matching the typedkind.
Fixed
- Round-3 review caught two pull-iter regressions left over from the Wave-M iterator surface. (a) The
EventIteratorterminal predicate keyed off the rawclient.shutdownflag, whichstop_streaming()flipped BEFORE the Disruptor consumer thread had finished pushing the tail of in-flight events into the iterator'sArrayQueue. Any caller pollingnext_timeoutbetween those two moments saw an empty queue + asserted shutdown and returnedClosed, dropping tail events on the floor. Replaced with a dedicatediter_closed: Arc<AtomicBool>flag flipped by a drop guard captured inside the Disruptor consumer closure — the guard fires only when the producer is dropped at io_loop exit, which only happens after the consumer thread has joined and every in-flight event has been pushed. Soak testcrates/thetadatadx/src/fpss/streaming_soak_tests.rs::iter_does_not_false_eof_during_drainpins the contract: 100 pre-queued tail events with the global shutdown asserted MUST surface asReadybefore the iterator signalsClosed. (b) Six docs files taught a non-existentevents()API on the client; corrected to the actual public entries (start_streaming_iter()Rust/Python/C++;startStreamingIter()TypeScript;streaming_iter()context manager on Python). - External multi-model audit: pull-iter
next_timeoutconflated timeout with terminal close.EventIterator::next_timeout()now returns a typed three-stateNextEventenum (Ready/Timeout/Closed) instead ofOption<FpssEvent>. Pre-fix,Noneoverloaded "deadline expired on a quiet-but-live stream" with "upstream shut down + queue drained", which propagated into every binding:- C ABI
tdx_fpss_event_iter_nextreturned-1(terminal) on a quiet live stream, so C consumers saw false EOF. - C++
EventIterator::ended_latched on the false EOF; the STLfor (const auto& e : iter)adapter terminated on the first timeout instead of re-polling. - Python
__next__retried indefinitely afterstop_streaming()because every 50 ms slice returnedNone(timeout-shaped) and the loop never observed a terminal signal. - TypeScript async
next()had the same defect: the Promise spun forever once the upstream queue closed. Fixed by liftingNextEventthrough the C ABI's three-state return (0ready /1timeout /-1closed), only latching the C++ wrapper'sended_on-1, looping the C++ STL adapter on1, raisingStopIterationfrom Python onClosed, and resolving the TS promise tonullonClosed. Soak testscrates/thetadatadx/src/fpss/streaming_soak_tests.rs::iter_returns_timeout_then_event_on_quiet_then_active_streamanditer_returns_closed_after_stop_streamingpin both branches; Pythontests/test_iter_mode.py::test_iter_terminates_after_stopasserts the Pythonfor event in iterator:loop exits within 1 s ofstop_streaming(). The blockingIterator::nextimpl onEventIterator(no timeout) is unchanged —Nonethere unambiguously means terminal because that path blocks until either an event arrives or the queue closes.
- C ABI
- Docs site lead-pages still referenced the pre-Wave-K
ThetaDataDxAPI class name and the removedsubscribe_quotes/subscribe_trades/subscribe_option_quotesper-kind methods.docs-site/docs/getting-started/installation.md,docs-site/docs/getting-started/quickstart.md, anddocs-site/docs/streaming/reconnection.mdnow lead withThetaDataDxClientplus the unified contract-firstclient.subscribe(contract.quote())/client.subscribe(contract.trade())API across Rust, Python, TypeScript, and C++. Event-payload examples switched from the removedevent.contract_id(wire-internal) toevent.contract.symbol. Migration tables in the CHANGELOG / release notes intentionally retain the old names — those document the rename, not the post-rename API. - External audit: single-slot drain barrier could falsely report quiescence under stacked lifecycle transitions. The
prev_drainedslot tracked only the most recently retired session's flag, so astart → stop → start → stopsequence in which the earlier session was still draining when the later one retired silently lost the earlier flag.await_drain()then returnedtruebased on the latest generation while the earlier callback could still be firing on the FFIctx— a use-after-free vector under reconnect-storm scenarios. The slot is now aMutex<Vec<Arc<AtomicBool>>>; every retired session's flag is pushed onto the Vec, andawait_drain()/tdx_*_freewalk the full set, lazily GC'ing flags that have flipped. Mirrored on the FFI handle'sprev_drainedfield. Regression coverage:crates/thetadatadx/src/fpss/streaming_soak_tests.rs::multi_gen_drain_waits_for_all_retired_sessionsdrives three realFpssClientinstances back-to-back with slow callbacks and asserts the barrier waits for every generation. - WS payload now carries
unresolved_contract_idfor pre-ContractAssignedticks. Pre-Wave-G the WS bridge surfaced the wire-internal numeric id; post-removal of the publiccontract_idfield, ticks that arrived before the matchingContractAssignedframe serialised as an emptyContractenvelope with no diagnostic channel for operators to correlate. The decoder now builds an unresolved-contract sentinel whosesymbolis__pending:<id>(the canonicalsec_type == SecType::Unknowncheck still gates consumer code paths); the WS formatter detects the prefix, emitscontract: {"status": "pending"}, and surfaces the parsed wire id as a top-levelunresolved_contract_idinteger. The public SDK callback signature is unchanged —__pending:is a diagnostic payload, not a stable identifier. - WS
/subscribeoption path now runs the canonical Gregorian validator. The Wave Htdbe::time::is_valid_yyyymmddcalendar check ran on the historical / REST surfaces but not on the WS option-subscribe path, which only applied the cheapis_valid_yyyymmdd_rangebounds check. Impossible dates like20260230(Feb 30),20260431(Apr 31), or20251301(month 13) leaked through. Both gates now run; the bounds check is the precheck, the calendar validator is the real gate. - Python and TypeScript bindings: stop / shutdown clear the registered callback. The unified C API preserves the callback across stop/reconnect, but the high-level bindings deliberately diverge:
stop_streaming()andshutdown()clear the stored callback, so a subsequentreconnect()raises until the caller re-registers viastart_streaming(callback). Documented on every affected method (stop_streaming,shutdown,reconnecton both bindings) so the explicit-handoff model is no longer surprising. stop_streaming()race that could resurrect streaming after stop returned. TheArcSwapslot acceptedStopped → Live, so an in-flightstart_streaming*()that began beforestop_streaming()could install a freshLiveslot AFTER stop observedStopped. Eachstop_streamingnow bumps anAtomicU64generation counter; eachstart_streaming*()snapshots the counter at entry, and theinstall_livercu closure refuses to install when the snapshot no longer matches. Regression tests incrates/thetadatadx/src/client.rs::testspin both branches of the gate.- MDDS
validate_dateaccepted impossible Gregorian dates (00000000,20260230,19990431,21010101). The shape-only check (length + ASCII digits) is now followed by a real calendar check via the newtdbe::time::is_valid_gregorian_date/is_valid_yyyymmddvalidator (year ∈ 1900..=2100, valid month, day-of-month including the 4 / 100 / 400 leap rule). Theflatfiles::request::validate_datehelper routes through the same canonical validator. - FPSS
Contract::optionand OCC-21 parsing accepted impossible expirations silently. Both paths now defer to the same canonical Gregorian validator as MDDS, so dates like Feb 30, Apr 31, or00000000fail at construction with an explicit error naming the offending input. - Silent
SystemTime::now()failure in the FPSS frame decoder. A clock skew beforeUNIX_EPOCHused to silently producereceived_at_ns = 0; the path now logs a rate-limitedtracing::warn!(targetthetadatadx::fpss::decode, every 1024 failures) and falls back to0only after surfacing the condition to operators. Same treatment for the WS server'ssonic_rs::to_stringfailure path: a newjson_serialize_failurescounter is exposed alongside the existingbroadcast_droppedcounter onGET /v3/system/fpss/status, and the failure path emits a rate-limitedtracing::error!. - Self-join deadlock when the user callback calls
stop_streaming(). With the consumer-thread dispatch in place, a callback that drops the lastArc<FpssClient>(which is whatThetaDataDxClient::stop_streaming()does internally) used to block onFpssClient::Drop'sio_handle.join(). The I/O thread's exit path drops the Disruptor producer, anddisruptor::Producer::dropjoins the consumer thread — the very thread running the callback.Dropnow captures the consumer thread'sThreadIdon first dispatch (OnceLock), detects the self-join case, and detaches the join onto a helper thread namedfpss-shutdown-detach. Cleanup completes asynchronously:is_streaming()flips tofalseimmediately on theLive → Stoppedswap, BEFORE the helper has joined; the newThetaDataDxClient::await_drain/tdx_*_await_drainbarrier (see Added) is the way to confirm full quiescence — i.e. that the previous user callback has stopped firing.crates/thetadatadx/src/fpss/streaming_soak_tests.rs::callback_triggered_stop_does_not_self_joindrives the realFpssClientthrough this path under a 5-second watchdog, andcallback_triggered_stop_then_await_drain_completesasserts the new barrier returnstruewithin budget and no further callback invocations happen after it returns. - Round-2 review caught two follow-up gaps. (a) The non-blocking C ABI poll path (
tdx_fpss_event_iter_next(.., 0)) was still collapsing timeout + closed viatry_next()returningOption<FpssEvent>. Fixed by promotingEventIterator::try_next()to also return the typedNextEventenum (symmetric withnext_timeout); the FFI now drives off the typed shape uniformly, so a C client polling afterstop_streaming()sees rc-1(terminal) instead of rc1(timeout) forever. The C++ wrapper'stry_next()calls the C ABI directly withtimeout_ms = 0and latchesended_only on rc-1. The Pythontry_nextand TypeScripttryNextkeep theirOption<…>public surfaces by collapsing bothTimeoutandClosedtoNone/null(single- state non-blocking polling stays the documented contract). New soak testcrates/thetadatadx/src/fpss/streaming_soak_tests.rs::iter_try_next_returns_closed_after_drainpins the contract:try_next()returnsClosed(notTimeout) once the queue is drained on a stopped session, and stays sticky on subsequent calls. (b) Front-door docs-site pages still taught the pre-Wave-K API. Updateddocs-site/docs/index.md,docs-site/docs/api-reference.md,docs-site/docs/getting-started/{authentication,first-query,streaming}.md, anddocs-site/docs/streaming/{index,connection,events}.mdto useThetaDataDxClient(Rust / Python / TypeScript) plustdx::UnifiedClient(C++), the polymorphicclient.subscribe(contract.quote())/client.subscribe(sec_type.full_trades())API, theclient.start_streaming_iter()/client.streaming_iter()pull-iter idiom, andevent.contract.symbolon data events. Migration tables in this CHANGELOG and the v9.1.0 release notes intentionally retain the pre-Wave-K names — those document the rename, not the post-rename surface.
Added
Flat-files ecosystem coverage across the tools surface. The
tdxCLI gains aflatfilesubcommand group (quotes,trades,trade_quote,ohlc,open_interest,eod, the fourstock_*equivalents, and the genericrequestarm). Each subcommand takes a singleYYYYMMDDdate plus--format csv|jsonland-o/--outputflags; missing-ostreams the bytes to stdout. (closes #433)REST server adds
GET /v3/flatfile/{sec_type}/{req_type}?date=...&format=...andPOST /v3/flatfile/requestroute handlers. The bytes ride a chunked response body (tokio_util::io::ReaderStream) so even hundred-MB blobs do not pin server memory;Content-Typeistext/csv; charset=utf-8for CSV orapplication/x-ndjson; charset=utf-8for JSONL. Flat files are batch downloads, not streaming subscriptions, so the WebSocket surface is unchanged. (closes #432)MCP server exposes eleven flat-file tools mirroring the Rust convenience methods:
tdx_flatfile_request(generic) plustdx_flatfile_option_quote/_trade/_trade_quote/_ohlc/_open_interest/_eodand the fourtdx_flatfile_stock_*shortcuts. Each tool writes the decoded blob to disk and returns the path so the LLM client can hand the file off to a downstream consumer that already speaks CSV / JSONL. (closes #431)Cross-language utility helpers (
condition_name,condition_description,is_cancel,updates_volume,quote_condition_name,quote_condition_description,is_firm,is_halted,exchange_name,exchange_symbol,sequence_signed_to_unsigned,sequence_unsigned_to_signed) now exposed in every binding. Python surfaces them asthetadatadx.util.*; TypeScript as theUtilclass with camelCase methods (Util.conditionName(0)); C++ as inline wrappers in thetdx::util::*namespace; the C ABI astdx_condition_name,tdx_exchange_name,tdx_sequence_signed_to_unsigned, etc. The Rust source-of-truth tables intdbe::{conditions, exchange, sequences}drive every binding directly — no language-specific duplication of the lookup data. (closes #424)docs-site dedicated FLATFILES section under
docs-site/docs/flatfiles/(overview, quickstart, API reference) with code samples in Python, TypeScript, C++, thetdxCLI, the REST server, and the MCP server. Wired into the VitePress sidebar alongside Real-Time Streaming. (closes #441)The query-builder docs page now covers FLATFILES request construction alongside the per-contract MDDS builder, including the parameter table, every snippet shape, and the bandwidth caveats. (closes #442)
ROADMAP gains a Binding Coverage Matrix tracking which features are exposed in each SDK / tool surface (Rust, Python, TypeScript, C, C++, MCP, CLI, REST/WS). Wave O flips the FLATFILES-tools and cross-language-utils rows to shipped. (closes #446)
Pull-iter delivery mode restored (was deleted in v8.0.30; now back as a sibling to push-callback). Adds
ThetaDataDxClient::start_streaming_iter()returning athetadatadx::EventIteratorin Rust;start_streaming_iter()/with tdx.streaming_iter() as it:returning the same iterator on Python (for event in it:);startStreamingIter()returning an async-iterableEventIteratornapi class on TypeScript (for await (const event of iter)); andtdx_unified_start_streaming_iter/tdx_fpss_event_iter_next/tdx_fpss_event_iter_close/tdx_fpss_event_iter_freein the C ABI plus a move-onlytdx::EventIteratorwith STL-iterator adapters in the C++ wrapper.The Disruptor consumer thread
force_pushes each event into acrossbeam_queue::ArrayQueuesized to match the ring; the user thread drains the queue under one lock acquisition per batch. On the Python binding this collapses N per-event GIL acquires into one acquire across the whole drain, which is the dominant throughput cost for tuple-build / deque-append integrators —streaming_throughput.rs::pyo3_iter_next_drainmeasures ~4.6 Melem/s vs. ~1.1 Melem/s for the equivalent push-callback shape (pyo3_deque_append), a 4.1× win on the same per-event Python work.Push-callback (
start_streaming(callback)) remains the recommended low-latency default; pull-iter is for high-throughput batch processing where amortising the lock cost dominates. Backpressure semantics match the callback path: when the iterator falls behind and the queue saturates, the consumer drops the new event and increments the samedropped_event_count()counter callbacks already surface. Mode is chosen at start; push and pull are mutually exclusive on a given client. Switch by stopping streaming and starting again.ThetaDataDxClient::panic_count()andFpssClient::panic_count()— new public methods that snapshot the count of user-callback panics caught by the Disruptor consumer'scatch_unwindboundary. Each panic is also surfaced viatracing::error!with targetthetadatadx::fpss::io_loop.ThetaDataDxClient::await_drain(timeout)— Rust quiescence barrier. Polls the previous streaming session's drain flag (set after the I/O thread + Disruptor consumer have joined) and returnstruewhen the previous user callback is guaranteed to have stopped firing. Pair withstop_streaming/reconnect_streamingfrom a thread other than the consumer thread when the application needs to free a captured context, replace the callback closure, or otherwise depend on full quiescence.tdx_unified_await_drain(handle, timeout_ms)andtdx_fpss_await_drain(handle, timeout_ms)— C ABI mirror ofawait_drain. Returns1once the previous Disruptor consumer thread has joined,0on timeout. Required betweentdx_*_stop_streaming/_reconnect/_shutdownand freeingctx; the FFIctxlifetime contract is now explicit that stop / reconnect are asynchronous on the consumer side.FpssClient::drained_flag()— exposes the sharedArc<AtomicBool>the higher-level barrier polls; useful for binding-layer code that wants to wire its own quiescence semantics.Python
await_drain(timeout_ms) -> boolandwith tdx.streaming(callback) as session:context manager. TypeScriptawaitDrain(timeoutMs)andawait using session = await tdx.streaming(callback)(TC39 explicit resource management). Both auto-callstop_streaming+await_drainon scope exit, mirroring the C++ RAII destructor lifecycle. The boundsessionproxies everysubscribe_*/unsubscribe_*call to the underlying client (Python__getattr__, TypeScriptProxy) so the streaming surface stays a single source of truth rooted in the Rust crate. Drain timeouts emit aRuntimeWarning(Python) orconsole.warn(TypeScript) without masking exceptions raised inside the body.RingSizeError(TooSmall { provided, minimum }/NotPowerOfTwo { provided, suggested }) — surfaced throughError::ConfigfromFpssClient::connectso a misconfigured buffer budget fails closed at construction with the offending value and the nearest valid size (ADR-002).crates/thetadatadx/src/fpss/streaming_soak_tests.rs— four soak tests (slow callback, panicking callback, callback-triggered stop, burst overload) plus the await-drain quiescence and free-blocks- until-drain tests, all exercising the consumer-thread wiring without a live FPSS connection. Lives inside the crate (rather thantests/) so the harness constructor stays#[cfg(test)]-only.Python SDK: "Streaming buffering" section in
sdks/python/README.mddocumenting thecollections.deque(Pattern A, default) andqueue.Queue(Pattern B, cross-thread blocking) consumer patterns.Vendor failure-mode resilience: capture+replay test harness against recorded FPSS bytes (
tests/replay_capture.rs), mid-frame TLS disconnect injection (tests/midframe_disconnect.rs), reconnect-storm test (tests/reconnect_storm.rs), vendor schema-drift coverage (tests/vendor_schema_drift.rs), property-based frame-decoder fuzz target (tests/decode_fuzz_property.rs), callback-watchdog API- slow-callback counter + rate-limited tracing warn (
tests/callback_watchdog.rs).
- slow-callback counter + rate-limited tracing warn (
ThetaDataDxClient::set_slow_callback_threshold(Duration)andThetaDataDxClient::slow_callback_count() -> u64(mirrored onFpssClient) — opt-in observability for user callbacks that exceed a wall-clock threshold. The Disruptor consumer measures every callback's elapsed time and increments a counter when over budget; atracing::warn!fires rate-limited per 1024 over-budget events to avoid log amplification. Observability only — Rust cannot safely cancel arbitrary user code, so the watchdog never kills the consumer.Duration::ZEROdisables the timer path entirely.Flat-file SDK parity across Python, TypeScript, and C++. New
tdx.flat_files.*(Python) /tdx.flatFiles.*(TypeScript) /tdx.UnifiedClient::flat_files()(C++) namespace returning a row-list with.to_arrow()/.to_pandas()/.to_polars()/.to_list()(Python) /.toArrowIpc()/.toJson()(TypeScript) /.to_arrow_ipc()(C++) terminals plus a genericrequest(sec_type, req_type, date)dispatcher andflatfile_to_pathraw-bytes helper. The dynamic schema (columns determined at runtime by(SecType, ReqType)) is implemented as hand-written thin wrappers overcrates/thetadatadx/src/flatfiles/arrow.rs::rows_to_arrowrather than through the SSOT codegen pipelines (build_support/sdk_surface/,build_support/endpoints/), which target static-schema surfaces only.
Removed
contract_id: i32removed from everyFpssData::*variant across every binding (Rust, Python, TypeScript, C, C++). The wire-internal numeric id the FPSS server assigns is no longer surfaced on data events; consumers readevent.contract.symbol(or otherContractfields —expiration,strike,is_call) for identity. Code that needs an id-keyed map builds it from theFpssControl::ContractAssignedevent stream. Thefpss_event_schema.tomlSSOT bumps toversion = 5; every generated binding regenerates without acontract_idfield.FpssEvent::{RawData, Empty}no longer in the public type. The decoder filters truncated FIT payloads onto thethetadatadx.fpss.decode_failuresmetric counter, and ring-buffer pre-allocation slots use the crate-privateFpssEventInternallayout-compatible companion enum (#[repr(C, u8)]shared discriminants — seeevents::FpssEventInternal::as_public). The Disruptor consumer reborrows&FpssEventfrom&FpssEventInternalzero-clone, so the H1-era hot-path cost is preserved.tools/server/AppState::contract_mapand thews::contract_mapmap-and-relookup pattern are deleted: the contractArcrides on the event itself, eliminating the reconnect/market-close TOCTOU race on the WS bridge.- Go SDK (
sdks/go/) deleted end-to-end. The previous half-state (Go files shipped, but no CI / no live validation / not advertised on the Rust core README badges) was SSOT drift; the C ABI inffi/remains the supported integration path for any third-party C / C++ consumer. The unusedbuild_support/ticks/go.rsgenerator is also removed. crates/thetadatadx/src/fpss/dispatcher.rsand its public exports. Panic isolation, drop counting, and consumer-thread invariants now live on the Disruptor consumer inio_loop.crossbeam-channelruntime dependency onthetadatadx.expert-modeandtest-harnessCargo features onthetadatadx. The C ABI no longer exposestdx_*_set_inline_callback; the queued and inline paths shared the same Disruptor-consumer pipeline post-#513, so the parallel entry points were theatre. The test-harness constructor (for_self_join_test) is now#[cfg(test)]-only and lives alongside the soak tests inside the crate.IntoOptionSpecsealed trait.Contract::option(symbol, expiration, strike, right)returns to its explicit four-argument form; callers holding wire-format integer triples useContract::option_raw(symbol, expiration, is_call, strike_raw)instead.FpssConnectArgs::Defaultimpl. The previous impl manufactured empty-string credentials inside aOnceLockso callers could spread..Default::default(); that produced a struct that could not actually connect. UseFpssConnectArgs::new(&creds, &hosts)and override the optional fields explicitly.
Changed (continued)
- TypeScript CI now runs
npm teston every advertised platform (Linux, macOS, Windows) instead of gating to Linux only. CI parity with the Python and Rust matrices: every platform we ship a prebuilt addon for has its tests run on that platform. README.md,crates/thetadatadx/README.md,docs/architecture.md,sdks/README.md,docs-site/docs/streaming/events.mdno longer advertise Go SDK support; the Rust install examples inREADME.md,crates/thetadatadx/src/frames/mod.rs, anddocs-site/docs/getting-started/{quickstart,installation}.mdnow showthetadatadx = "9".FpssClient::connectnow rejects a non-power-of-tworing_sizewithError::Configrather than silently rounding to the next power of two (ADR-002). Default configs (131_072) are unchanged.Contract::option(symbol, expiration, strike, right)reverts to the explicit four-argument signature; the wire-format integer triple constructor moves toContract::option_raw(...).
Performance
Per-event cost on the
start_streamingpath drops by removing theevent.clone()+ intermediate-channeltry_send+ drain-thread wakeup hop that previously sat between the Disruptor consumer and the user callback.Microbenchmark methodology (
crates/thetadatadx/benches/streaming_channels.rs): each variant retriesProducer::try_publishon overflow until exactlyEVENTS_PER_ITER(= 100_000) successful publishes have landed per Criterion sample, and the consumer closure (or trampoline) increments adelivered_events: AtomicU64. The retry-on-overflow loop guaranteesdelivered == EVENTS_PER_ITERby construction, soThroughput::Elements(EVENTS_PER_ITER)is exact and the reported figures are per-DELIVERED-event cost. Earlier revisions divided wall-clock by attempt count, which silently understated cost when the consumer fell behind. Adebug_assert_eq!(delivered, EVENTS_PER_ITER)per iteration cross-checks the invariant in debug builds (it is a no-op in release-mode bench runs).Indicative numbers from
cargo bench --bench streaming_channels -- --quickon a recent x86-64 Linux laptop (Criterion median, native release build, throughput per delivered event):disruptor_consumer_panic_isolated(live SSOT path:Producer::try_publish+ Disruptor consumer +catch_unwind): ≈ 1.46 ms / 100k ≈ 14.6 ns / delivered event (≈ 68 Melem/s).disruptor_consumer_no_catch_unwind(same pipeline without the panic boundary): ≈ 1.47 ms / 100k ≈ 14.7 ns / delivered event (≈ 68 Melem/s) —catch_unwindcost is below Criterion's noise floor onEmptyevents.disruptor_cross_thread(production-shape topology: producer on a worker thread, consumer on the Disruptor's own thread): ≈ 1.49 ms / 100k ≈ 14.9 ns / delivered event (≈ 67 Melem/s).direct_callback(prospective TLS-reader-direct path modelled viaBox<dyn Fn>adapter, no ring, no consumer thread): ≈ 533 µs / 100k ≈ 5.3 ns / delivered event (≈ 188 Melem/s).
Run
cargo bench --bench streaming_channelsfor per-machine numbers; the absolute values are sensitive to CPU model and governor settings, so the SDK ships the methodology and the variants rather than locking in figures that age out with each hardware refresh. The earlier "1.13 ns / event" figure for the Disruptor variants was an artefact of dividing wall-clock by attempt count, not delivered events; the corrected number above reflects per-callback-delivery cost.
[9.0.2] - 2026-05-07
Added
- Property-based tests on five hot paths via
proptest = "1.5"(dev-dependency only; no public-API surface change).crates/tdbe/src/codec/fie.rs: encoder/decoder round-trip on the full FIE alphabet (excluding'n', the documented terminator nibble), strict-vs-panicking-encoder agreement, and per-character nibble round-trip.crates/tdbe/src/codec/fit.rs: single-row FIT round-trip via a cfg(test) encoder againstFitReader::read_changes,decode_fit_buffer_bulkagreement on the same byte stream, andflush_digitsnon-negative-monotonicity on partial digit runs.crates/tdbe/src/greeks.rs: Black-Scholes invariants over the market range(spot, strike) in [0.01, 10000.0],rate in [-0.05, 0.20],div_yield in [0.0, 0.10],tte in [1/365, 5.0],iv in [0.001, 5.0]-- put-call parity (tolerance scaled tonorm_cdfapproximation error), call/put delta bounds, vega non-negativity, gamma non-negativity.crates/tdbe/src/time.rs:civil_to_epoch_daysmonotonicity over 1970..=2099,eastern_offset_msreturns exactly EST or EDT for every timestamp in 2000..=2099, and DST cutover sanity at the spring-forward / fall-back boundaries across 1990..=2099 (covers both pre- and post-2007 rule windows).crates/thetadatadx/src/fpss/protocol/contract.rs:Contract -> to_bytes -> from_bytesround-trip for stocks and options (expiration 2000..=2099, strike 1..=99_999_999, right in {C, P}, root 1..=6 ASCII uppercase), OCC-21 string parser round-trip, and compositeOCC-21 -> Contract -> bytesround-trip pinning the parser and wire codec against each other.
[9.0.1] - 2026-05-07
Changed
- Stripped 76 per-line Java reverse-engineering breadcrumbs across the FPSS, MDDS, and config trees; ADR-001 (
docs/architecture/ADR-001-java-terminal-parity.md) is now the single anchor for that work, referenced from one module-header line in each affected file. - Split
crates/thetadatadx/src/fpss/io_loop.rsintoio_loop/{mod.rs, login.rs, ping.rs}so the login handshake and ping heartbeat live next to the main I/O loop without sharing a 1,072-line file. - Removed
docs/public-api-redesign.md; v9.0.0 shipped the redesign it described, and a one-line note indocs/architecture/README.mdrecords that the planning prose for shipped surfaces is intentionally not preserved. - Declared
rust-version = "1.88"on every workspace[package]. CI'sLintmatrix grows a1.88axis on Linux so dependency bumps that raise the rustc requirement fail before release;README.mdadds aRequirementssection. - Added a
Semver checkCI job that runsobi1kenobi/cargo-semver-checks-action@v2against thev9.0.0tag on every PR.CONTRIBUTING.mddocuments the public-API stability policy and the local invocation.
[9.0.0] - 2026-05-07
Breaking
Contract::optionis now polymorphic;Contract::option_rawis gone. A new sealedIntoOptionSpectrait accepts either(&str, &str, &str)(human-friendly: expiration / strike / right) or(i32, bool, i32)(wire-format integer triple). Callers pass one tuple instead of four loose arguments, and the wire-format constructor moves under the same method name.rust// Before: let c = Contract::option("SPY", "20261218", "60", "C")?; let c = Contract::option_raw("SPY", 20261218, true, 60_000); // After: let c = Contract::option("SPY", ("20261218", "60", "C"))?; let c = Contract::option("SPY", (20261218, true, 60_000))?;FpssClient::connecttakes oneFpssConnectArgsstruct instead of seven loose arguments. The struct exposescreds,hosts,ring_size,flush_mode,policy,derive_ohlcvc.Defaultand anew(creds, hosts)shortcut cover the common path.rust// Before: FpssClient::connect(&creds, &hosts, 4096, FpssFlushMode::default(), ReconnectPolicy::default(), true, handler)?; // After: let args = FpssConnectArgs::new(&creds, &hosts); FpssClient::connect(args, handler)?;Wire-internal
contract_id: i32removed from every public surface. Dropped from Rust (ThetaDataDxClient::contract_map,ThetaDataDxClient::contract_lookup,FpssClient::contract_map,FpssClient::contract_lookup), C ABI (tdx_unified_contract_map,tdx_unified_contract_lookup,tdx_fpss_contract_map,tdx_fpss_contract_lookup,tdx_contract_map_array_free,TdxContractMapArray,TdxContractMapEntry), Python (contract_map(),contract_lookup()), TypeScript (contractMap(),contractLookup()), and C++ (FpssClient::contract_map,FpssClient::contract_lookup). Users identify contracts by(symbol, expiration, right, strike); the wire id stays inside the reader-thread cache and is delivered alongside every event viaFpssControl::ContractAssigned { id, contract }for callers that still need to maintain their own id→contract map.rust// Before: let map = client.contract_map()?; if let Some(c) = client.contract_lookup(id)? { ... } // After: build the map yourself from the event stream. client.start_streaming(|event| { if let FpssEvent::Control(FpssControl::ContractAssigned { id, contract }) = event { my_map.insert(*id, Arc::clone(contract)); } })?;pub mod protois nowpub(crate). Generated protobuf types are wire-internal. Bindings that needDataTable/DataValueList/ResponseData/Price/data_value::*go through the newthetadatadx::wirere-export, which surfaces only the types offline-decode harnesses actually need.rust// Before: use thetadatadx::proto::{DataTable, ResponseData}; // After: use thetadatadx::wire::{DataTable, ResponseData};FPSS submodules
connection,framing,dispatcher,ringreduced topub(crate). Onlyprotocolremains a public submodule offpss.Frame,read_frame,write_frameare surfaced as items atthetadatadx::fpss::for benchmark consumers; everything else (TLS connect, ring-buffer wait strategies, dispatcher internals) is now crate-private.rust// Before: use thetadatadx::fpss::framing::{read_frame, write_frame, Frame}; // After: use thetadatadx::fpss::{read_frame, write_frame, Frame};
Added
IntoOptionSpecsealed trait + impls for(&str, &str, &str)and(i32, bool, i32)— seefpss::protocol::IntoOptionSpec.FpssConnectArgsstruct +FpssConnectArgs::new(creds, hosts)shortcut.thetadatadx::wiremodule — the supported re-export surface for the generated protobuf payload types (DataTable,DataValueList,DataValue,ResponseData,Price,CompressionAlgo,CompressionDescription,data_value).
Changed
- Comprehensive public-API discipline sweep.
auth::{creds, nexus, session}reduced topub(crate); user-facing types (Credentials,AuthResponse,AuthUser,SessionToken,authenticate,authenticate_at) re-exported atthetadatadx::auth::*and the crate root. Everypub fn/pub structreachable from a public path was audited; internal helpers (TLS connect entry points, ring-size constants, framing reader idle predicates) are now crate-private. tdbe0.12.10 → 0.13.0 (eastern-time + json_canon + conditions codegen surface expansion warrants the minor bump).
Removed
Contract::option_raw(folded intoContract::optionviaIntoOptionSpec).ThetaDataDxClient::contract_map,ThetaDataDxClient::contract_lookup,FpssClient::contract_map,FpssClient::contract_lookup.- C ABI:
tdx_unified_contract_map,tdx_unified_contract_lookup,tdx_fpss_contract_map,tdx_fpss_contract_lookup,tdx_contract_map_array_free,TdxContractMapArray,TdxContractMapEntry. - Python SDK:
ThetaDataDxClient.contract_map,ThetaDataDxClient.contract_lookup. - TypeScript SDK:
ThetaDataDxClient.contractMap,ThetaDataDxClient.contractLookup. - C++ SDK:
FpssClient::contract_map,FpssClient::contract_lookup. pub mod proto(nowpub(crate); consumers usethetadatadx::wire).pub mod fpss::{connection, framing, dispatcher, ring}(nowpub(crate); surfaces preserved as items atfpss::root where needed).- Dead helpers removed:
fpss::connection::connect_to,fpss::framing::FrameReadState::is_idle,fpss::ring::DEFAULT_RING_SIZE.
[8.0.37] - 2026-05-07
Added
- Typed
SubscriptionTierenum (Free,Value,Standard,Pro) replacing rawOption<i32>onMddsClient.max_concurrent_requests(self)codifies the2^tiersemaphore semantics;from_wire(i32)decodes the wire byte (returningNonefor unknown values rather than silently coercing). Re-exported asthetadatadx::SubscriptionTier. The wire-sideauth::nexus::AuthUserkeeps its rawOption<i32>fields so deserialization stays infallible for unknown future tiers; the typed enum is the post-decode in-memory shape callers see.
Changed
- Streaming state machine collapsed into a single
ArcSwap<StreamingSlot>.ThetaDataDxClient's prior 3-field state (Mutex<Option<FpssClient>>,Mutex<Option<StreamingDispatcher>>,AtomicBool was_streaming) is now oneArcSwapof anIdle/Live/Stoppedenum. Read paths (is_streaming,connection_status,with_streaming, every per-subscription forwarder) collapse to one atomic load. Lifecycle paths retain serial semantics through an rcu-CAS install and an atomic swap-to-Stopped. Addsarc-swap1.7 to runtime deps. - Layout moves:
- Top-level mdds-specific modules relocated under
mdds/(endpoint.rs→mdds/endpoint_args.rs,macros.rs→mdds/macros.rs,registry.rs→mdds/registry.rs,validate.rsmerged intomdds/validate.rs,wire_semantics.rs→mdds/wire_semantics.rs). Re-exports preserved at crate root for back-compat (thetadatadx::endpoint::*,thetadatadx::EndpointMeta,thetadatadx::ENDPOINTS). unified.rs→client.rs(filename = primary type name).frames.rs+frames_generated.rs→frames/{mod,generated}.rs.tdbe::types::*_generated.rssegregated undertdbe::types::generated/.
- Top-level mdds-specific modules relocated under
Fixed
(LOW 3.2)
extract_*_columnreturn type left asVec<Option<T>>— iterator conversion deferred. The three helpers are public surface exercised by benches, integration tests, the macro-driven list endpoints, and the Polars / Arrow column projections; switching toimpl Iterator<Item = Option<T>>would force every caller to deal with the iterator shape and lose the missing-header early-return the warn-log path relies on.(LOW 3.9)
Drop::droponThetaDataDxClientdocuments its idempotency invariant. TheIdle/Live/Stoppedstate machine guarantees the FPSS / dispatcher shutdown sequence runs at most once acrossstop_streaming+Drop.(LOW 3.10)
#[allow(dead_code)]removed fromflatfiles/framing.rs::msg. Every one of the tenu16wire-code constants is genuinely used byflatfiles::requestandflatfiles::session; the attribute was a false positive.Refs #500.
[8.0.36] - 2026-05-07
Changed
crates/thetadatadx/src/decode.rs(2177 LoC) split into 7 modules undermdds/decode/{error,headers,transport,extract,cell,v3}. Pure structural refactor; public API unchanged viamdds::decode::*re-exports.Eastern-time + DST primitives lifted to
tdbe::time.eastern_offset_ms,march_second_sunday_utc,november_first_sunday_utc,april_first_sunday_utc,october_last_sunday_utc,civil_to_epoch_days,timestamp_to_ms_of_day,timestamp_to_date— single canonical module reused by mdds, fpss, flatfiles. tdbe 0.12.9 → 0.12.10.crates/thetadatadx/src/fpss/protocol.rs(1613 LoC) split into 4 modules underfpss/protocol/.mod.rskeeps constants and re-exports;contract.rsholdsContract+ 6 constructors +Display+FromStr+ OCC-21 parser;wire.rsholds payload builders / parsers;subscription.rsholdsSubscriptionKind.crates/thetadatadx/src/config.rs(1396 LoC, 30 flat fields) refactored into 7 nested typed sub-configs.DirectConfignow containsmdds,fpss,reconnect,retry,auth,metrics,runtime. Field-read accessors preserved onDirectConfigfor back-compat (config.mdds_host()etc still work). Field-write callers must migrate to nested form (config.fpss.queue_depth = ...). Addsmdds.connect_timeout_secs(default 10s, covers prior LOW finding).crates/tdbe/src/conditions.rs(2749 LoC) refactored to TOML-driven codegen. Source-of-truth atcrates/tdbe/data/{trade,quote}_conditions.toml(149 + 75 entries).crates/tdbe/build.rsreads the TOMLs and emitscrates/tdbe/src/conditions/tables_generated.rswith compile-time const arrays. Public surface unchanged; newcondition_tables_pintest pins 12 known entries against the const arrays for round-trip protection.Refs #500.
[8.0.35] - 2026-05-07
Documentation
- Sweep stale
root/exp_datereferences across the doc tree. Post-#484 (8.0.28) follow-up:docs/api-reference.md,docs/macro-guide.md,docs/architecture.md,docs/java-parity-checklist.md,docs-site/docs/api-reference.md,docs-site/docs/streaming/{connection,events}.md,docs-site/docs/historical/option/list/{roots,contracts}.md,sdks/cpp/README.md— Rust SDK references rewritten to use the post-#484symbol/expirationvocabulary. Closes #503.
[8.0.33] - 2026-05-07
Added
- Seven Architecture Decision Records under
docs/architecture/: ADR-001 (Java terminal parity sourcing), ADR-002 (FPSS ring power-of-two capacity), ADR-003 (MDDS2^tierconcurrent-request mapping), ADR-004 (Eastern-time DST cutover), ADR-005 (OCC-21 century scope, expires 2099-12-31), ADR-006 (FPSS reconnect policy with rate-limit-aware backoff), ADR-007 (flatfiles MDDS SPKI pin rotation policy). tdbe::json_canon— JSON canonicalisation (non-finite f64 tonull) is now atdbesubmodule. Existingjson_canon::*callers in the CLI, MCP, and server crates migrate totdbe::json_canon::*.
Changed
ParsedRight::from_wire_byteis now wired at the fouris_call/is_putmagic-number sites incrates/tdbe/src/types/tick.rs. Theright == 67/right == 80raw integer comparisons go through the typed parser; behaviour is identical, the magic numbers are gone.
Removed
crates/json_canonworkspace member folded intotdbe::json_canon. External consumers should switch totdbe = "0.12.9"and importtdbe::json_canon::{canonicalize, canonicalize_and_serialize, finite_or_null}.- Internal version references in source comments (
v8.0.10,v7.2.0,v8.0.3,v8.0.2,v6.0.1+). Semantic content preserved; release metadata pruned out of code paths where it adds no maintenance value. - Banned-vocabulary sweep on full-stream subscription wording in source doc comments, README.md, and ROADMAP.md. Replaced with
full-stream/full-type. CHANGELOG history is intentionally untouched.
[8.0.32] - 2026-05-06
Fixed
- StreamingDispatcher drain loop now catches user-callback panics and continues serving subsequent events. New
panic_countdiagnostic counter exposed alongsidedropped_count. Previously a panic in user code (Rust closure / PyO3 callable / napi ThreadsafeFunction / C extern fn) silently killed the dispatcher thread; onlyshutdown()surfaced it. - FPSS C ABI handle state machine.
tdx_fpss_set_callback/_inline_callback/_reconnect/_shutdownenforce the public contract: at most one registration per handle, shutdown is terminal, post-shutdown ops return -1 with a cleartdx_last_error()string. Previously the contract was documented but unenforced. - C ABI
ctxlifetime contract documented. The public header now statesctxmust outlive registration until shutdown / free returns (or a successful unified re-registration). Queued vs inline thread-affinity also documented. - Python
dropped_event_count()doc + test corrected to reflect the actual reset-on-reconnect / zero-after-stop semantics. The counter lives on the StreamingDispatcher and resets when the dispatcher is rebuilt (matches the TypeScript binding). - Dispatcher
droppedcounter is now strictly queue-full. TheDisconnectedvariant ofTrySendError(rare; happens only during shutdown races) feeds a new separatedisconnected_countso the user-facing drop metric isn't inflated by lifecycle noise. - TypeScript
index.d.tsregenerated so the JS-visible doc comment matches the Rust source — the counter resets on reconnect.
Changed
- Unified C ABI
set_callbackafter stop is REPLACEMENT. Documented explicitly inthetadx.hand the rustdoc: contrary to the FPSS one-shot rule, the unified high-level path supports stop + re-register as a normal user flow (this is whatreconnect_streamingis built on). The contract divergence is intentional.
[8.0.31] - 2026-05-06
tdbe
tdbe::right::ParsedRight::from_wire_byte(byte: i32) -> Option<Self>—const fndecoder for the FPSS wirerightbyte (67for'C',80for'P'). Inverse of the existingas_wire_byte(). Removes the rationale for downstream tick decoders to re-type the67/80magic numbers at every trust boundary; round-trip property test confirmsfrom_wire_byte(self.as_wire_byte().unwrap()) == Some(self)for every variant where the forward direction is defined. Patch bump tdbe 0.12.8 → 0.12.9.
[8.0.30] - 2026-05-06
This release closes #482: the entire FPSS streaming stack — Rust core, C ABI, Python, TypeScript, and C++ — moves to a callback-driven delivery model backed by a single StreamingDispatcher SSOT. Bundles PR #489 (dispatcher core), #490 (C ABI), #492 (Python), #493 (TypeScript), and #494 (C++ wrapper).
Breaking
Python
tdx.next_event(timeout_ms)REMOVED. Replaced withtdx.start_streaming(callback). The dispatcher thread acquires the GIL viaPython::with_gilto call the user's Python callable. The internalstd::sync::mpscshim insdks/python/src/streaming_methods.rsand the wrapper'srx/EventRx/ closure-localdropped_eventscounter are gone; the Python binding now wires straight through the SSOTStreamingDispatcher. Drop counter is exposed viatdx.dropped_event_count()(forwarded tothetadatadx::ThetaDataDxClient::dropped_event_count). The Python binding deliberately does NOT exposestart_streaming_inline: GIL acquisition can block, and a slow Python callback on the FPSS reader thread would fill the kernel TCP receive buffer and trigger a vendor-side disconnect.Migration:
python# Before tdx.start_streaming() while True: event = tdx.next_event(100) if event: process(event) # After def handler(event): process(event) tdx.start_streaming(callback=handler)TypeScript
tdx.nextEvent(timeoutMs)REMOVED. Replaced withtdx.startStreaming(callback). The dispatcher thread routes events through napi-rsThreadsafeFunctionto the Node main thread; the user's JS callback runs there, decoupled from the FPSS reader. Migration:typescript// Before tdx.startStreaming(); while (true) { const event = await tdx.nextEvent(100); if (event) process(event); } // After tdx.startStreaming((event) => process(event));The
droppedEvents()getter is renamed todroppedEventCount()and now forwards to the SSOTStreamingDispatcherso the value matches every other binding.std::sync::mpscis gone fromsdks/typescript/. The TypeScript binding deliberately does NOT expose astart_streaming_inlineopt-in: Node's libuv requires JS callbacks on the main thread, andThreadsafeFunction's internaluv_async_tqueue is the only safe path.C ABI streaming:
tdx_unified_next_event,tdx_fpss_next_event,tdx_unified_start_streaming, andtdx_fpss_event_freeREMOVED. Replaced with a callback-only surface that wires through the SSOTStreamingDispatcher:tdx_unified_set_callback(handle, fn, ctx)/tdx_fpss_set_callback(handle, fn, ctx)— queued: events flowFPSS reader -> bounded(8192) crossbeam queue -> dispatcher drain thread -> user fn. The reader never blocks on user code; overflow events are dropped and counted viatdx_*_dropped_events.tdx_unified_set_inline_callback(handle, fn, ctx)/tdx_fpss_set_inline_callback(handle, fn, ctx)— inline: user fn fires directly on the FPSS reader thread (microsecond-budget contract, identical semantics tostart_streaming_inline).
tdx_fpss_connectnow defers the FPSS TLS connection until the firstset_callback/set_inline_callbackcall (callback registration and connect are atomic). Allstd::sync::mpscusage and the poll-based receive path have been removed fromffi/src/streaming.rs.C++ wrapper: the poll-based
tdx::FpssClient::next_eventand the owningFpssEventPtr/FpssEventDeletertypes are gone. Event delivery is now exclusively callback-driven through the newset_callback/set_inline_callbackmethods (see Added).
Changed
- New
StreamingDispatchercore incrates/thetadatadx/src/fpss/ dispatcher.rs. Lock-freecrossbeam_channel::bounded(8192)queue between FPSS reader thread and a dedicated dispatcher thread that drains the queue and invokes the user-registered Rust callback. Existingstart_streaming(callback)API now wires through this dispatcher transparently — callers see no behavior change. Reader thread never blocks on user code; queue overflow drops events with a counter.
Added
start_streaming_inline(callback)— power-user opt-in Rust API. Callback fires directly from the FPSS reader thread, bypassing the dispatcher. Trade: zero queueing overhead (~12 ns/event vs 58 ns for the dispatcher path) but slow callbacks block the reader and cause vendor disconnects. Documented contract: callback must return within microseconds.- C++ wrapper callback API.
tdx::FpssClient::set_callback (std::function<void(const FpssEvent&)>)for the default queued path;set_inline_callbackfor power-user opt-in directly on the FPSS reader thread. Both wrap the C ABItdx_fpss_set_callback/tdx_fpss_set_inline_callbackshipped in this release. Thefpss_smokeexample is restored on the callback path.
[8.0.29] - 2026-05-06
Removed
Go SDK. The cgo bridge between Rust's C ABI and Go's runtime carries per-call overhead that masks the upstream throughput this SDK is engineered for. Users who need Go bindings can build their own cgo wrapper against the unchanged C ABI in
crates/ffi/— header atsdks/cpp/include/thetadx.h, all FFI types and free fns exported astdx_*symbols.Closes #481.
[8.0.28] - 2026-05-06
Breaking
**
Contract,OptionContract,FlatFileRow, andIndexEntryrenameroottosymbolandexp_datetoexpirationto match the v3 vendor surface documented in the v2 → v3 migration guide. The wire codec is unchanged —Contract::to_bytes/Contract::from_bytesstill serialize the field asrootperContract.javaparity, and the FLATFILES decoder still resolves both v2 (root) and v3 (symbol) response columns through the existingdecode::HEADER_ALIASES. Per-language renames:- Rust (
thetadatadx::fpss::protocol::Contract,tdbe::types::tick::OptionContract,thetadatadx::flatfiles::FlatFileRow):Contract.root→Contract.symbolContract.exp_date→Contract.expirationContract::stock(root)→Contract::stock(symbol)Contract::index(root)→Contract::index(symbol)Contract::rate(root)→Contract::rate(symbol)Contract::option(root, exp_date, …)→Contract::option(symbol, expiration, …)Contract::option_raw(root, exp_date, …)→Contract::option_raw(symbol, expiration, …)OptionContract.root→OptionContract.symbolFlatFileRow.root→FlatFileRow.symbol
- Python (
thetadatadx.Contract,thetadatadx.OptionContract):contract.root/contract.exp_date→contract.symbol/contract.expiration;OptionContract(root=…)constructor keyword →symbol=…. - TypeScript (
Contract,OptionContract):contract.root/contract.expDate→contract.symbol/contract.expiration. - Go (
thetadatadx.Contract,thetadatadx.OptionContract):c.Root/c.ExpDate→c.Symbol/c.Expiration. - C++ (
OptionContract,TdxContract,TdxOptionContract):c.root/c.exp_date/c.has_exp_date→c.symbol/c.expiration/c.has_expiration. - C ABI:
TdxContract.root→TdxContract.symbol,TdxContract.exp_date→TdxContract.expiration,TdxContract.has_exp_date→TdxContract.has_expiration,TdxOptionContract.root→TdxOptionContract.symbol. - FLATFILES CSV / JSONL: contract-prefix headers and JSON keys change from
root,expiration,strike,right,…tosymbol,expiration,strike,right,…. Stock blobs go fromroot,…tosymbol,…. The vendor's response columns are unchanged; only the SDK's emitted file headers change. - REST / WebSocket / MCP outputs in
tools/serverandtools/mcpemit"symbol"/"expiration"keys on every contract payload (option lists, FPSS event contracts, FLATFILES rows).
- Rust (
Changed
- Workspace 8.0.27 → 8.0.28, tdbe 0.12.7 → 0.12.8. The tdbe bump rides the regenerated
OptionContract.symbolfield incrates/tdbe/src/types/tick_generated.rs; every other change ships as patch deltas off the existing v8 line per repo policy. tools/cliraw column header forOptionContractissymbolinstead ofroot, sourced fromtick_schema.toml::fieldso future schema renames flow through the CLI without a helper edit.
[8.0.27] - 2026-05-06
Changed
- polars 0.52 -> 0.53. Adopts the new
DataFrame::new(height: usize, columns: Vec<Column>)signature incrates/thetadatadx/build_support/ticks/rust_frames.rs(the existingn = self.len()binding feeds the newheightargument). Re-runs ofgenerate_sdk_surfacesproduceframes_generated.rswith the updated call form. Closes #464.
Fixed
crates/tdbe/src/conditions.rstrade condition 61 renamed from a third-party product mark toPRICEVOLUMEADJ. Same scrub pattern as the v8.0.26 exchange-code-0 rename. Single tracked-source occurrence;rg 'NANEX'now clean. Description unchanged. Closes #476's sibling, filed as #480.
[8.0.26] - 2026-05-05
Breaking
GreeksTickremoved in every language and on the C ABI. The full union now ships asGreeksAllTickand the per-order endpoints return typed subsets. Callers update imports and method return types -- no forwarding shim. Renames:- Rust / Python / TypeScript / Go / C++
GreeksTick->GreeksAllTick(full union returned byoption_*_greeks_allandoption_*_greeks_eod). - C ABI free fn
tdx_greeks_tick_array_free->tdx_greeks_all_tick_array_free. - Endpoints now returning
GreeksFirstOrderTick:option_snapshot_greeks_first_order,option_history_greeks_first_order,option_history_trade_greeks_first_order. - Endpoints now returning
GreeksSecondOrderTick:option_snapshot_greeks_second_order,option_history_greeks_second_order,option_history_trade_greeks_second_order. - Endpoints now returning
GreeksThirdOrderTick:option_snapshot_greeks_third_order,option_history_greeks_third_order,option_history_trade_greeks_third_order. GreeksAllTickaddsbid,ask,underlying_ms_of_day,underlying_pricecolumns the upstream OpenAPI publishes but the legacyGreeksTickdid not carry. Field offset of every existing Greek shifts by 16 bytes (bid + ask) on the FFI mirror; rebuild any binary that links the C struct.
- Rust / Python / TypeScript / Go / C++
Added
- Per-endpoint typed Greeks structs. The vendor's
option_*_greeks_first_order,_second_order,_third_orderendpoints emit strict subsets of the full Greek column set. The SDK now exposesGreeksFirstOrderTick(delta / theta / vega / rho / epsilon / lambda + bid/ask + IV pair + underlying snapshot),GreeksSecondOrderTick(gamma / vanna / charm / vomma / veta + bid/ ask + IV pair + underlying snapshot), andGreeksThirdOrderTick(speed / zomma / color / ultima + bid/ask + IV pair + underlying snapshot). Each per-order endpoint returnsVec<<Type>>directly -- no zero-default columns leak from one subset into another. - New C ABI free symbols:
tdx_greeks_all_tick_array_free,tdx_greeks_first_order_tick_array_free,tdx_greeks_second_order_tick_array_free,tdx_greeks_third_order_tick_array_free. Matching FFI array types emitted on every binding (TdxGreeksAllTickArray,TdxGreeksFirstOrderTickArray, etc.). - New header alias
underlying_ms_of_day->underlying_timestampincrates/thetadatadx/src/decode.rs::HEADER_ALIASESso the wire Timestamp -> ms-of-day conversion flows through the standardrow_numberpath on every Greeks endpoint. - Per-field
offset_of!layout assertions incrates/tdbe/src/types/tick.rs::layout_asserts. Field-offset drift (e.g. swapping two same-size fields) sneaks pastsize_of/align_ofchecks alone -- the new asserts pin every observable Rust field offset that the C / Go FFI mirrors index into.
Changed
- Generated
tdbe::types::tickstruct definitions.crates/tdbe/src/types/tick_generated.rsis now emitted bygenerate_sdk_surfacesfromtick_schema.toml. The hand-writtentick.rskeepsimpl_contract_id!macro applications, theTradeTickflag helpers, andOptionContract::is_call/is_put-- everything else flows from the schema. Adding a new tick type means adding one[types.X]row. - Schema-driven C++ layout asserts and Go FFI sizes.
sdks/cpp/include/tick_layout_asserts.hpp.incandsdks/go/tick_ffi_sizes_generated.gonow compute every struct's size + alignment from the schema (viatick_ffi_size_and_align) rather than each emitter dispatching on the type name. Adding a tick type totick_schema.tomlproduces the size/align pair, the C++static_assert, and the Gounsafe.Sizeoftest entry without anybuild_support/ticks/{cpp,go}.rsedit. OpenInterestTickandTradeQuoteTickgained the missingalign = 64directive intick_schema.toml. The schema now matches the#[repr(C, align(64))]declared on the correspondingtdbetypes -- the schema-derived FFI size used to under-count by the alignment rounding (32/144 vs 64/192) before reaching the C++ layout assert.- Exchange code 0 in
crates/tdbe/src/exchange.rsrenamed from a third-party product mark to neutral SIP terminology (Composite). Symbol staysCOMP; wire byte 0 still resolves to the same array slot. Closes #476.
Fixed
option_*_greeks_*_orderno longer spamsexpected column header not foundwarnings.crates/thetadatadx/src/decode.rs::find_headeremitted atracing::warn!every time a generated parser asked for an optional column that was absent from the wire response. The Greeks family splits the column set across the wire —_greeks_first_orderships seven Greeks,_greeks_second_orderships five, and_greeks_third_orderships four — but the sharedGreeksTickschema carries the full 23-Greek union. Callingoption_snapshot_greeks_third_ordertherefore produced eight warn lines per response (zomma, color, ultima, d1, d2, dual_delta, dual_gamma, vera, …) before any user-visible decoding finished. The warn is now atracing::trace!so the diagnostic is still reachable viaRUST_LOG=thetadatadx=tracefor genuine schema-drift investigations, but stays out of stderr on routine subset calls. Required-column drift continues to surface as a typedError::MissingRequiredHeaderfrom the generated parser. Closes #472.
Changed
- TOML-driven render map collapses 19 hand-coded helper match arms into single-key lookups. Every per-language binding name a renderer needs for one tick type — Rust direct-client return type, generated parser fn, Go struct + converter, FFI array struct + free fn + output variant + header-return type, C++ value type, six Python converters (dict, columnar, pyclass-list, pyclass-list-class, vec-to-pylist, slice-arrow), TypeScript class + class-vec converter, and the Python pyclass struct name — moves into
[types.X.render]blocks incrates/thetadatadx/tick_schema.toml. The 20 helper functions that previously enumerated those names by hand (build_support/endpoints/helpers.rs::direct_return_typeand friends, plusbuild_support/ticks/mod.rs::pyclass_name) become single HashMap lookups against aOnceLock-cached load of the schema. Adding a tick type now requires one TOML row -- no helper edits. The generated SDK surfaces are byte-identical againstmainbecause the TOML rows reproduce the names the helpers previously hardcoded. - Per-endpoint vendor-schema column lists for the four Greeks families pinned and documented in
tick_schema.toml::GreeksTickagainst the upstream OpenAPI capture inscripts/upstream_openapi.yaml. TheGreeksTickstruct itself is unchanged — every Greeks endpoint still returnsVec<GreeksTick>and the union layout is the same — but the schema doc-comment now spells out which Greeks each endpoint publishes, why the others zero-default, and where the per-endpoint vendor schema is captured. The codegen pickup is doc-only; no SDK surface drift. - Three new unit tests in
crates/thetadatadx/src/decode.rs::testsdriveparse_greeks_ticksagainst the_first_order,_second_order, and_third_orderwire shapes (column lists pinned to upstream OpenAPI). Each test asserts bit-exact decoded values for the wire-present columns and0.0defaults for the documented gaps, so a future regression offind_headerback totracing::warn!— or any column-list drift in either direction — surfaces as a behavioural test failure rather than as live log spam. tdbe0.12.6 → 0.12.7.
[8.0.25] - 2026-05-05
Fixed
- Windows
ERROR_IO_PENDING(os error 997) no longer trips a fatal FPSS read error. On Windows the overlapped socket layer surfaces in-flight reads asERROR_IO_PENDINGinstead ofWSAEWOULDBLOCK. Ruststdmaps raw OS error 997 toErrorKind::Uncategorized, so the existingWouldBlock | TimedOutmatches incrates/thetadatadx/src/fpss/io_loop.rs::is_read_timeoutand the two retry arms incrates/thetadatadx/src/fpss/framing.rs(pre-header and mid-payload) treated it as fatal — Python users on Windows sawFPSS read error error=IO error: Overlapped I/O operation is in progress. (os error 997)spam followed by a reconnect storm. A newis_transient_readhelper inframing.rsmatchesWouldBlock,TimedOut, andraw_os_error() == Some(997); all three sites delegate to it so the I/O loop drains queued commands and retries the way it does on Linux and macOS. Closes #469.
Changed
tdbe0.12.5 → 0.12.7.
[8.0.24] - 2026-05-04
Added
tdbe::greeks::vera— public free function exposing the DvegaDr formula-K * exp(-r*T) * T * sqrt(T) * phi(d2)so callers can pull the single Greek without computing the full bundle.tdbe::greeks::compute_full_bundle_with_iv(s, x, v, r, q, t, is_call)— fullGreeksResultcomputation that skips the bisection IV solver and uses a caller-supplied volatility. Tier-0 intermediates are shared across every Greek in the bundle; ~2× faster than 17+ individual per-Greek calls. Typical use case is the IV-cache hot path. Takesis_call: boolrather than&str rightbecause callers in this path have already parsed the side;all_greeksandimplied_volatilitykeep the&str rightsurface.
Changed
tdbe0.12.4 → 0.12.5.
[8.0.23] - 2026-05-01
Fixed
- REST + MCP no longer return empty bodies on serialisation failure.
tools/server/src/handler.rsandtools/mcp/src/main.rspreviously swallowedsonic_rs::to_stringerrors viaunwrap_or_default(), producing a200 OKwith""(REST) or a successful but emptytools/callresult (MCP) when a tick payload contained a non-finite f64 cell. The REST handler now surfaces the failure as a structured500carrying the underlying error message in the existing JSON envelope, and the MCP handler returns a JSON-RPC-32603Internal Error. The cross-language non-finite f64 -> JSONnullcanonicalisation rule (previously inlined intools/cli/src/main.rsasraw_f64) now lives in the newcrates/json_canoncrate and is shared by CLI, REST, and MCP so all three frontends produce byte-identical output for the same payload. - FPSS WebSocket broadcast queue is now bounded.
tools/server/src/ws/broadcast.rspreviously usedtokio::sync::mpsc::unbounded_channel, which could grow without bound if the broadcast task lagged behind the Disruptor consumer thread. The channel is now bounded at 65_536 slots and the callback usestry_sendwith explicitFull/Closedarms; drops are accounted on a newAppState::record_fpss_broadcast_dropAtomicU64counter, exposed viaGET /v3/system/fpss/statusasbroadcast_dropped, and warn-logged once per 1024 drops to surface back-pressure without flooding stderr. ThetaDataDxClient::reconnect_streamingnow fails explicitly on partial re-subscription. The re-subscribe loop incrates/thetadatadx/src/unified.rspreviously logged failures viatracing::warn!and returnedOk(()), hiding partial reconnects from programmatic callers. The loop now collects every failed(SubscriptionKind, Contract)pair and, when the list is non-empty, returns the newError::PartialReconnect { failed }variant. The per-failuretracing::warn!lines stay for operational visibility.- Version metadata drift cleared.
sdks/cpp/CMakeLists.txt(8.0.9->8.0.23) and the workspace comment inCargo.toml(7.x->8.0.x) now match the rest of the v8 line. Banned vocabulary purged fromSECURITY.md, the[8.0.8]changelog entry, and the dropped-events Python test.
Added
crates/json_canon— a tiny shared crate exposingfinite_or_null,canonicalize, andcanonicalize_and_serializefor non-finite f64 -> JSONnullcollapse plus surfacedsonic_rs::Erroron serialisation failure. Pulled in bytools/cli,tools/server, andtools/mcp.- New
Error::PartialReconnect { failed: Vec<(SubscriptionKind, Contract)> }variant inthetadatadx::errorand a newContract::full_type_marker(sec_type)constructor used to encode a failed full-type subscription inside the structured failure list. AppState::record_fpss_broadcast_dropandAppState::fpss_broadcast_droppedon the REST server, and a newbroadcast_droppedfield onGET /v3/system/fpss/status.- A pytest CI step in
.github/workflows/python.ymlthat runspytest sdks/python/tests/after the wheel install. The existing import smoke is kept as a separate step.
Changed
tools/cli/src/main.rsraw_f64is now a thin delegation tojson_canon::finite_or_null.tools/server/src/format.rsrender_csv_valuecanonicalises before serialising and emits a<csv-render-error: …>sentinel rather than an empty cell on serialisation failure.tdbebumped to0.12.4to keep all member crates on a fresh patch line for the 8.0.23 release.
[8.0.22] - 2026-05-01
Fixed
fpss::accumulator::change_price_typenow matches the JVM terminal'sPriceCalcUtils.changePriceTypebyte-for-byte. v8.0.21 widened the multiplication throughi64and returned the unscaled input on overflow, which broke parity with the Java reference (Javaint * intsilently wraps under two's-complement). The rescale now usesi32::wrapping_mul, reproducing the JVM's exact wire bits in both debug and release. Tests pin the wrapping result to manually-computed Java-equivalent values (e.g.2_148 * 10^6 → -2_146_967_296).decode::row_number_i64clampsprice_typeto0..=19so the same wire cell decodes identically throughrow_number_i64androw_price_f64(the latter routes throughtdbe::Price::new, which has clamped to that range since it was introduced). Under the clamped contract,i32::MAX * 10^9 ≈ 2.15e18is well belowi64::MAX, so scale-up cannot overflow and the previousPrice overflowing i64error path is no longer reachable.
Added
crates/thetadatadx/tests/flatfiles_synthetic_golden.rs— a deterministic decoder-only golden test that builds a synthetic FLATFILES blob (header + INDEX + FIT-encoded DATA) in Rust and pins the CSV writer's output byte-for-byte. Runs in plaincargo testwith no live wire and no env var, giving CI a hard regression gate on the FIT decoder, INDEX walker, and price formatter on every push.
Changed
- Documentation references to "22 Greeks" updated to "23 Greeks" to reflect the
verafield added in v8.0.21. Touches thetdbeREADME + module docs, thethetadatadxREADME tick-types table, and the Python and C++ SDK READMEs.
[8.0.21] - 2026-04-30
Fixed
fpss::accumulator::change_price_typeno longer overflowsi32on rescale. Multiplications by10^N(up to10^9) widened toi64mid-arithmetic; rescales whose result still does not fiti32return the original price unchanged and emit atracing::warn!event with(price, price_type, new_price_type)rather than silently saturating or panicking. A live BRK.A wire integer in cents (71_396_865) rescaled toprice_type=4is the canonical trigger.decode::row_number_i64no longer routesPricecells throughf64. Large integer fields delivered asPricecells now decode with i64-native scaling (checked_pow/checked_mul), preserving every ULP past2^53. Scale-ups that overflowi64surface asDecodeError::TypeMismatch { expected: "i64-fitting Price", observed: "Price overflowing i64" }rather than a saturatedf64 as i64.
Changed
tdbe::greeks::all_greeksandtdbe::greeks::implied_volatilitynow returnResult<_, tdbe::Error>. Both helpers previously panicked whenrightdid not parse as a single side. They now returntdbe::Error::Configfor unrecognised or wildcard rights. Every in-repo call site (ffi,tools/cli,tools/mcp,sdks/python,crates/tdbe/benches) was updated. Direct callers of these helpers must add?or.expect().tdbe::greeks::GreeksResultgained avera: f64field computed insideall_greeks. Vera (a.k.a. DvegaDr) is the textbook cross-sensitivity of vega to the risk-free rate: `-K * exp(-r*T) * T- sqrt(T) * phi(d2)
. The downstreamTdxGreeksResult` C-ABI struct, the C++/Go/Python SDK Greeks structs, and the CLI/MCP output objects all carry the new field.
- sqrt(T) * phi(d2)
Added
crates/thetadatadx/tests/flatfiles_byte_match.rsgained a second test,option_eod_csv_byte_matches_vendor, that pulls OPTION/EOD for20260428and byte-matches against a vendor reference CSV pointed to by a new env var,THETADATADX_REFERENCE_EOD_CSV. The EOD path exercises the CSV price formatter end-to-end against vendor output — the existing OPEN_INTEREST byte-match did not, since OPEN_INTEREST has no price columns. The test skips when the reference CSV is missing; the doc comment documents the regeneration recipe.live.ymlgained acargo test --features live-tests --test flatfiles_byte_matchstep in the livesmokejob. It skips gracefully when the reference CSV is not provisioned for the runner.
[8.0.20] - 2026-04-30
Fixed
- FLATFILES price decoding was off by powers of ten across every output format. The CSV / JSONL writers and the typed in-memory
FlatFileRowreturn path were dividing the wire integer by10^Nwhere N was read directly from the row'sPRICE_TYPEcolumn. The vendor convention isreal_price = value * 10^(price_type - 10)(see [tdbe::types::price::Price]), so forprice_type = 8(cents) the correct factor is0.01(i.e.value / 10^2), notvalue / 10^8— off by 10^6. Effect: optionbid/ask/pricecolumns came out near-zero (e.g.1.9e-6instead of1.90), and the CSV{:.4}-formatted output rounded those to0.0000. Every consumer of the flat-file pipeline was affected. - The CSV price formatter no longer hardcodes 4 fractional digits. Rust's default
f64Display now preserves the full IEEE-754 precision the wire decoder produced, so micro-priced contracts survive the on-disk round-trip.
Changed
flatfiles::writer::price_divisor(private API) replaced withprice_type_for_row+decode_price. Both new helpers route price decoding throughtdbe::types::price::Price::to_f64(), which is the authoritative implementation of the ThetaData price convention.- The
OPTION/OPEN_INTERESTbyte-match integration test still passes — open-interest rows have no price columns, so the test never exercised the broken formatter. A new unit test (decode_price_uses_vendor_semantics) locks the corrected behaviour, andfmt_price_preserves_full_precisionasserts that micro-priced rows do not round to zero.
[8.0.19] - 2026-04-30
Changed
tools/mcpreplacedArc<RwLock<Option<ThetaDataDxClient>>>withArc<OnceCell<ThetaDataDxClient>>. The JSON-RPC handler no longer holds a read guard across awaited tool execution;OnceCell::getis lock-free.tools/cliget_arg()now usesunreachable!()with an explicit invariant comment. All call sites declare the argument with clap'srequired(true), so theNonebranch indicates a clap config bug, not user input.
Added
crates/tdbe::FitRows::get()returnsOption<&[i32]>for caller-supplied indices. The existingFitRows::row()keeps its panic-on-OOB contract with a clearer message.
[8.0.18] - 2026-04-30
Fixed
- Workspace version drift:
ffi,tools/cli,tools/server, andtools/mcpwere pinned at 8.0.15 while the SDK crates moved through 8.0.16 → 8.0.17. Every Rust crate, every npmpackage.json, and the TypeScriptpackage-lock.jsonnow report a single 8.0.18 surface. Contract::to_bytes()no longer panics on caller input. NewContract::validate()returns a typedResult<(), Error::Config>; newContract::try_to_bytes()is the fallible encoder.build_subscribe_payload()now returnsResult<Vec<u8>, Error>, andFpssClient::{subscribe, unsubscribe}validate before encoding. The reconnect re-subscribe loop logs and skips invalid contracts instead of failing the whole reconnect. Net: malformed roots flow back asError::Configto every binding instead of crashing the process.SessionToken::refreshno longer holds atokio::Mutexacross the Nexusauthenticate_at(...).await. Replaced withtokio::RwLock<Inner>for state plus a separatetokio::Mutex<()>for refresh dedup. Concurrentsnapshot()/current_uuid()readers continue against the previous (still-valid) UUID throughout.
[8.0.17] - 2026-04-30
Added
- FLATFILES — third public surface alongside MDDS and FPSS. Pulls one whole-universe INDEX + DATA blob per
(SecType, ReqType, date)tuple fromnj-{a,b}.thetadata.us:12000over a TLS PacketStream protocol distinct from MDDS gRPC and FPSS streaming. Server identity pinned to the production keypair viaMddsSpkiVerifier. Login: CREDENTIALS + VERSION → SESSION_TOKEN + METADATA, with PING heartbeats during auth tolerated and terminal login errors short-circuiting host retry. The raw download path uses asynctokio::fswith a 1 MB BufWriter; decode + write run ontokio::task::spawn_blockingso FPSS / MDDS tasks on the same runtime do not stall. crates/thetadatadx::flatfilesmodule:framing,mdds_spki,session,request,index,decode,decoded,decoded_row,format,types,writer,datatypesubmodules.- Three free-function entry points:
flatfile_request,flatfile_request_decoded,flatfile_request_raw. Mirror methods on the unifiedThetaDataDxClientclient. Convenience methods for the option / stock ×{open_interest, trade_quote, trade, quote, eod}matrix. - Public types:
FlatFileFormat::{Csv, Jsonl},SecType,ReqType,FlatFileRow,FlatFileValue,FlatFilesUnavailableReason. crates/thetadatadx/examples/flatfile_demo.rsend-to-end CLI example.crates/thetadatadx/tests/flatfiles_byte_match.rslive integration test (live-testsfeature gate) that byte-matches CSV output against the vendor terminal jar.
Changed
tdbe0.12.0 → 0.12.1. Republishes the SSOT-generator enum surface (Interval,RequestType,Version) sothetadatadxpublish resolves on crates.io.
Notes
- Cross-language coverage of FLATFILES (CLI, MCP, REST/WS server, FFI, Python, TypeScript, Go, C++) is tracked in the issue tracker; the Rust core is shipped today. See
ROADMAP.mdfor the binding coverage matrix.
[8.0.16] - 2026-04-30
Added
thetadatadx::utilsnamespace exposesconditions,exchange,sequencesfor tick post-processing without a separatetdbedependency.- Re-exports at the
thetadatadxcrate root for every tick struct returned by an SDK method (CalendarDay,EodTick,GreeksTick,InterestRateTick,IvTick,MarketValueTick,OhlcTick,OpenInterestTick,OptionContract,PriceTick,QuoteTick,TradeQuoteTick,TradeTick), the enums named on those structs (DataType,Interval,RateType,RemoveReason,RequestType,Right,SecType,StreamMsgType,StreamResponseType,Venue,Version),Price, and the offline Greeks helpers (all_greeks,implied_volatility,GreeksResult).
Changed
ROADMAP.mdaligned with the 2026-04-20 validator run (127 PASS / 7 subscription-tier-blocked / 0 FAIL) and the 2026-04-29 / 04-30 FLATFILES live run.
[8.0.15] - 2026-04-24
Fixed
- Linux wheel tag moved from
manylinux_2_38tomanylinux_2_17so the publishedthetadatadx-*-manylinux_2_17_x86_64.whlinstalls on every glibc 2.17+ runtime (CentOS 7 / RHEL 7+ / Ubuntu 18.04+ / Debian 10+ / Google Colab / Databricks). The v8.0.14 wheel was built onubuntu-latest(now Ubuntu 24.04 / glibc 2.38), which silently gated every older environment —pip install thetadatadxwould fall through to the sdist and fail the source build because Rust is not available on most hosted Python runtimes.
Changed
.github/workflows/python.ymlLinux wheel step now usesPyO3/maturin-action@v1withmanylinux: '2014'(glibc 2.17 toolchain inside a Docker container). macOS and Windows continue to build natively on their matrix runners.
[8.0.14] - 2026-04-23
Fixed
- Re-publish the v8.0.13 chain to crates.io and GitHub Releases. The v8.0.13 tag CI failed on the
Extended Surfacesdocs-consistency gate because the squash merge of #412 captured an intermediate branch state (top-levelCHANGELOG.mdhad the final wording while the mirroreddocs-site/docs/changelog.mdstill had the pre-cp wording). PyPI and npm published v8.0.13 successfully; crates.io and the GitHub Release did not. v8.0.14 re-publishes everything from the synced main tip. No behavior change vs v8.0.13.
[8.0.13] - 2026-04-23
Fixed
- Mid-stream chunk header drift in the MDDS response accumulator was silently masked:
MddsClient::collect_stream/for_each_chunkwould keep the first chunk'sheadersand pile subsequent chunks' rows underneath, even if a later chunk carried a different non-empty header set. A server-side schema change mid-response would therefore surface as silent data corruption instead of an error. Both paths now compare the saved first-chunk schema against every non-empty chunk header set and raise a newDecodeError::ChunkHeaderDrifton mismatch (P13 from the external bench handoff).
Added
decode::DecodeError::ChunkHeaderDrift { chunk_index, first, chunk }variant.
Known
option_at_time_quote0.67× vs vendor (bench handoff §8 #1). The v8.0.5 uniformmdds_query_field_exprrule that empties the top-levelexpirationfield on any option query carrying aContractSpecmay have flipped this specific endpoint into a slower server-side path. Needs a bench-validated per-endpoint override inendpoint_surface.toml. Not fixed in this release because a speculative generator carve-out without bench re-validation would risk regressing the other option endpoints that benefit from the current rule.option_history_greeks_eod0.704× vs vendor (bench handoff §8 #2). Persistent across v8.0.0 / v8.0.4 / v8.0.10. Likely server- side per-contract aggregation path rather than a wire-shape issue; needs proto-level diff against the otheroption_history_greeks_*endpoints (which are DX wins at 4-6× faster).
[8.0.12] - 2026-04-23
Removed
scripts/test_drift_injection.sh+ theFPSS drift injectionCI job (.github/workflows/ci.yml). The test was designed when the C++static_assert(offsetof)guards inthetadx.hppwere hand-maintained against a Rust-generated C struct layout. v8.0.11 moved both sides under the same SSOT generator, so swapping a field infpss_event_schema.tomlregenerates the C struct and the assert value in lockstep and the assertion can no longer fail. Removed rather than kept as a misleading safety net;regen_byte_identicalcovers generator consistency and the assertions still fire at C++ compile time against hand-committed C header corruption.
8.0.11 - 2026-04-23
Added
endpoint_surface.tomlnow declares the endpoint-surface enums used byright,venue,interval,rate_type,request_type, andversion. The generator emits the Rusttdbeenums, Python enum pyclasses, and the TypeScript napi string enums from the same TOML variant lists.- Go now gets generator-owned FFI drift artifacts for every checked size and offset:
endpoint_ffi_sizes_generated.go,tick_ffi_sizes_generated.go,fpss_ffi_sizes_generated.go,ffi_layout_generated_test.go, andfpss_ffi_offset_checks_generated.go. - C++ now gets generator-owned layout assertion includes:
tick_layout_asserts.hpp.incfromtick_schema.tomlandfpss_layout_asserts.hpp.incfromfpss_event_schema.toml. .github/release-notes/v8.0.11.mdrecords the SSOT refactor and local verification plan for this release.
Changed
crates/tdbe/src/types/enums.rsnow includes generator-emitted endpoint-surface enums instead of hand-maintainingRight,Venue,Interval,RateType,RequestType, andVersion.sdks/python/src/coerce.rsnow includes generator-emitted enum pyclasses instead of hand-maintaining thestring_enum!block.sdks/go/tick_ffi_mirrors.gono longer embeds hand-maintained expected sizes or FPSS offset literals; it consumes generator-owned constants and offset tables.sdks/go/ffi_layout_test.gohas been replaced by the generatedsdks/go/ffi_layout_generated_test.go, so the Go tick-layout drift detector now reads its expected values from TOML-derived generation.sdks/cpp/include/thetadx.hppnow includes generated layout assertion fragments instead of hand-maintainingstatic_assert(sizeof(...))andstatic_assert(offsetof(...))blocks.- Live docs and READMEs no longer hardcode endpoint, tick-type, or tool counts; they describe the generated surface instead.
- Release metadata bumps
8.0.10 -> 8.0.11acrossthetadatadx,thetadatadx-ffi,thetadatadx-cli,thetadatadx-server,thetadatadx-mcp,thetadatadx-py, andthetadatadx-napi. TypeScript package metadata, loader version guards, and the checked-in OpenAPI version now match8.0.11. tdbestays at0.12.0.
8.0.10 - 2026-04-23
Added
endpoint_surface.tomlnow carries upstream-verified defaults for every builder-bound optional param that the ThetaData OpenAPI spec documents as optional with a server-side fallback:venue = "nqb",rate_type = "sofr",version = "latest",exclusive = true,use_market_value = false,underlyer_use_nbbo = false. These flow through theparsed_endpoint!macro as the initial builder value, so callers that omit the field hit the same wire payload the official Python library produces — no per-endpoint runtime fallback needed.- Parameter descriptions in the SSOT now enumerate accepted values for
venue,rate_type,version,exclusive,use_market_value, andunderlyer_use_nbbo, which propagates into the per-language generator outputs (Rust docstrings, Goendpoint_options.go, C++endpoint_options.hpp.inc, Python builder docstrings). - SSOT defaults now cover
right = "both",strike = "*", andinterval = "1s". The option contract endpoints no longer requirerightandstrikeas positional Rust method arguments; callers set concrete values through the existing options builder fields when they need to override the server defaults. - Python bindings expose module-level
Right,Venue,Interval,RateType,RequestType, andVersionstring enum classes. Enum constrained parameters accept either plain strings or those enum objects. - TypeScript declarations expose matching literal-union types and const companions for
Right,Venue,Interval,RateType,RequestType, andVersion.
Changed
- The
venue=nqbdefault moved from a runtime constant (wire_semantics::DEFAULT_STOCK_VENUE) into the SSOT, makingendpoint_surface.tomlthe single source of truth for every parameter default across every emitter. The generator's query- assembly path now wraps default-bearingStrfields inSome(...)when marshalling into the proto request, keeping the wire shape identical to the previous release. collapse_redundant_wiresin the build-time mode matrix now reads per-endpoint SSOT defaults instead of the hardcodedvenue=nqbbranch, so future additions to the default set automatically collapse their redundantwith_<name>validator cells.- Release metadata bumps 8.0.9 -> 8.0.10 across every Rust crate (
thetadatadx,thetadatadx-ffi,thetadatadx-cli,thetadatadx-server,thetadatadx-mcp,thetadatadx-py,thetadatadx-napi), every TypeScript package (sdks/typescriptroot plus the three platform subpackages undersdks/typescript/npm/), the TypeScript native binding version guard insdks/typescript/index.js, and the OpenAPI contract indocs-site/public/thetadatadx.yaml. tdbestays at0.12.0; the encoding crate is untouched.- Rust, Python, TypeScript, Go, and C++ endpoint surfaces now project proto
repeated string symbolendpoints as bulk-capable symbol inputs. Singular-symbol wire endpoints remain singular. - Python historical date parameters (
date,expiration,start_date,end_date) acceptstr,datetime.date, ordatetime.datetime. Python time parameters (start_time,end_time,min_time,time_of_day) acceptstrordatetime.time. - TypeScript historical date and time parameters accept either
stringor JavaScriptDatevalues at the native binding boundary.
8.0.9 - 2026-04-23
Fixed
- The TypeScript package lock now matches
package.jsonfor version, license, Node engine, and platform optional dependency pins. - The requested repo-root
scripts/regen_byte_identical.shgate now delegates to the checked-in generator determinism harness, and the docs consistency and tier badge scripts are executable. - User-facing docs and release notes no longer point at deleted
thetadatadxmodules or removed FPSS shortcut APIs. CHANGELOG.mdanddocs-site/docs/changelog.mduse only the Keep-a-Changelog section buckets and avoid banned performance phrasing.
Changed
- Release metadata now points at
8.0.9across Rust crates, the TypeScript root package and platform packages, the TypeScript native binding version guard, the C++ package metadata, and the checked-in OpenAPI contract. - Every Rust crate version bumps
8.0.8 -> 8.0.9:thetadatadx,thetadatadx-ffi,thetadatadx-cli,thetadatadx-server,thetadatadx-mcp,thetadatadx-py,thetadatadx-napi. sdks/typescript/package.jsonand every platform subpackage undersdks/typescript/npm/bump to8.0.9so the npm dependency graph stays coherent.tdbestays at0.12.0; this patch is metadata, docs, and tooling hygiene only.
8.0.8 - 2026-04-23
Follow-up patch to v8.0.7. Addresses the review findings surfaced against the code-strip release: rustdoc breakage inside tdbe, TypeScript loader and subpackage versions drifting from the root package, a [8.0.7] changelog section that accidentally absorbed v8.0.6 content, stale references to removed modules, and a handful of doc inaccuracies around DataFrame terminals and SDK parameter names. No behaviour changes; every item is documentation, packaging metadata, or tooling hygiene.
Fixed
crates/tdbe/src/codec/fit.rs— broken intra-doc link onFitReader's module-level docstring now resolves via[FitReader::read_changes].crates/tdbe/src/right.rs— five redundant explicit link targets on[Error::Config]references dropped; rustdoc resolves the bare path against the in-scopeuse crate::error::Error.sdks/typescript/index.js— native-binding version guard now compares against'8.0.8'(was stale sentinel'8.0.0'). Mismatched binaries are caught whenNAPI_RS_ENFORCE_VERSION_CHECKis set.sdks/typescript/package.json—optionalDependenciespin each platform subpackage to8.0.8(was8.0.4). The three published subpackages (thetadatadx-linux-x64-gnu,thetadatadx-darwin-arm64,thetadatadx-win32-x64-msvc) bump from8.0.7to8.0.8in lockstep.CHANGELOG.md/docs-site/docs/changelog.md— v8.0.6 content (snapshot fast-path, Rustframesmodule) split back out of the v8.0.7 section into a standalone[8.0.6]entry; the### Changedbucket on v8.0.6 was renamed### Changedto stay within the Keep a Changelog vocabulary.docs/api-reference.md— two references to the oldtdbeerror module repointed totdbe::error.docs/java-parity-checklist.md— stale normalization-module path updated tomdds/endpoints.rs, the current home ofnormalize_intervalafter the v8.0.7 fold.crates/thetadatadx/src/wire_semantics.rs— stale normalization-module parenthetical removed from the module docstring.docs-site/docs/api-reference.md— DataFrame-terminals section narrowed:.to_pandas()/.to_polars()/.to_arrow()are available on the<TickName>Listlist-wrapper return types; snapshot-fast-path endpoints return a plainlist[TickClass]and do not carry the chainable terminals.sdks/python/README.md,sdks/go/README.md,sdks/cpp/README.md— parameter-name tables now use the canonical SSOT names (expiration,start_date,end_date) instead of theexp,start,endshorthand.
Changed
docs-site/docs/.vitepress/config.ts—vite.build.chunkSizeWarningLimitraised to1500kB. The docs site bundles Mermaid and Vue chunks that exceed the default 500 kB threshold; the warning was non-actionable.deny.toml— unused license allowances pruned from[licenses].allow; remaining entries carry a short comment explaining why each is there.cargo deny checknow produces zero warnings.
8.0.7 - 2026-04-23
Code-strip release. No new features. Every item removes dead or near-dead code, narrows module visibility, or consolidates parallel FFI surfaces. tdbe bumps to 0.12.0 (public module removed).
Removed
- MDDS normalization forwarding layer over
crate::wire_semantics. The three wire canonicalizers (normalize_expiration,wire_strike_opt,wire_right_opt) stay atcrate::wire_semantics; the MDDS-scopednormalize_interval,normalize_time_of_day, andcontract_spec!macro move next to their generated consumers incrates/thetadatadx/src/mdds/endpoints.rs. fpss::session::reconnect— 90 LOC public function, zero callers.ThetaDataDxClient::reconnect_streamingremains the reconnect entry point.reconnect_delayis kept (used byfpss::decode).- The crate-local right-parser re-export shim was removed.
parse_right,parse_right_strict, andParsedRightstay at the crate root via a directpub use tdbe::right::*. - The unreachable retry helper trio and the crate-level
#![allow(dead_code)]attribute that masked them were removed.StatusClassmoved intomacros.rsas a private enum. crates/tdbe/src/errors.rs— folded intotdbe::error. The two used items (HTTP_STATUS_CODE_KEY,error_from_http_code) are now reachable attdbe::error::*; the unusederror_namehelper and theerrorsmodule itself are gone.- 24
FpssClient/ThetaDataDxClientper-security shortcut methods (and their unsubscribe twins). Callers use theContract-takingsubscribe_quotes/subscribe_trades/subscribe_open_interestmethods directly. - 61
MddsClient::<endpoint>_with_deadlinesibling methods on every list endpoint. Per-call deadlines route throughEndpointArgs::with_timeout_ms(FFI / Python / TS / Go / C++) or the builder.with_deadline(Duration)setter on parsed endpoints. SDK generators now wrap the bare call intokio::time::timeoutlocally instead of calling the deleted_with_deadlinevariant. - 61
tdx_<endpoint>(no-options) FFI entry points. The C++ SDK already calls thetdx_<endpoint>_with_optionsvariants, so the plain-name declarations insdks/cpp/include/thetadx.hand the hand-written historical FFI wrappers are gone. pub use prostat thethetadatadxcrate root. Downstream consumers that needprost::Message(sdks/python) now pull it in as a direct dependency pinned to the same=0.14.3version.MddsClient::raw_query,MddsClient::raw_query_info,MddsClient::channel— zero callers anywhere in the tree.
Changed
pub mod unifiedandpub mod registrynarrowed topub(crate). The documented types (ThetaDataDxClient,SubscriptionInfo,ConnectionStatus,EndpointMeta,ParamMeta,ParamType,ReturnType,ENDPOINTS, plusby_category,find,param_type_to_json_type,CATEGORIESfor the CLI / MCP tools) stay public viapub use.DirectConfig::production_defaultsnarrowed topub(crate); the only caller outsideconfig.rsis in-crate (observability.rs).crates/tdbebumps to0.12.0(breaking:pub mod errorsremoved). The publicThetaDataErrorstruct,error_from_http_codefn, andHTTP_STATUS_CODE_KEYconst are still reachable at the newtdbe::error::*path.- FFI surface consolidated: every SDK — C++, Go, Python, TypeScript — now calls the
tdx_<endpoint>_with_optionsentry points. The plain-name FFI entry points are no longer exported.
8.0.6 - 2026-04-23
Snapshot-endpoint latency fast-path on the Python binding and new opt-in Rust frames module. Reduces residual latency on the 5 flagged snapshot / calendar endpoints (stock_snapshot_ohlc, stock_snapshot_quote, stock_snapshot_market_value, calendar_on_date, calendar_open_today), and brings chainable .to_polars() / .to_arrow() DataFrame ergonomics to Rust consumers behind opt-in Cargo features so polars and arrow stay out of the default dep graph.
Added
- Rust
framesmodule —TicksPolarsExt/TicksArrowExtextension traits behindpolars/arrow/framesCargo features. Chain.to_polars()/.to_arrow()off a decoder-owned&[tick::T]in Rust the same way Python users chain off<TickName>List. Per-tick-type impls are generator-emitted fromtick_schema.tomlintocrates/thetadatadx/src/frames_generated.rs(new file), covering every entry —CalendarDay,EodTick,GreeksTick,InterestRateTick,IvTick,MarketValueTick,OhlcTick,OpenInterestTick,OptionContract,PriceTick,QuoteTick,TradeQuoteTick,TradeTick. Column-shape SSOT with the Python slice_arrow path: both generators readtick_schema.tomland apply the same field-type → Arrow-dtype mapping, soticks.as_slice().to_polars()?in Rust produces the same DataFrame schema (column order, dtypes, theQuoteTick.midpointvirtual column, the contract-idexpiration/strike/righttail, theOptionContract.righti32 → string projection) astdx.stock_history_eod(...).to_polars()in Python. Dep footprint stays opt-in:polars = ["dep:polars"],arrow = ["dep:arrow-array", "dep:arrow-schema"],frames = ["polars", "arrow"]; polars pins to0.46withdefault-features = false(no lazy, no parquet, no SQL, no compute kernels) andarrow-array/arrow-schemapin to58.1.0matchingsdks/python/Cargo.tomlso the repo sees a single major version of the arrow family. Opt-in form:thetadatadx = { version = "8", features = ["polars"] }.
Changed
- Snapshot-kind endpoints now return plain
list[TickClass]instead of the<TickName>Listwrapper. Applies to every endpoint withsubcategory = "snapshot"or"snapshot_greeks"inendpoint_surface.toml, plus everycategory = "calendar"+kind = "parsed"entry — 20 endpoints total: 4stock_snapshot_*, 11option_snapshot_*(OHLC, trade, quote, open_interest, market_value, + 5 greeks variants + 1 IV variant), 3index_snapshot_*, 3calendar_*. The<T>Listallocation cost was pure overhead on the latency-sensitive path — callers never chain.to_polars()on a 1-row calendar result. Classification is entirely TOML-driven viahelpers::is_snapshot_endpoint; no hand-curated allowlist, so adding a new snapshot-kind endpoint to the TOML automatically opts it into the fast path on the next generator run. Return-type annotation changes (list[CalendarDay]instead ofCalendarDayList); positional args and kwargs on the public pymethod signature are unchanged. - Snapshot pymethods now dispatch via a new
run_blocking_snapshothelper — boundedtokio::time::timeoutinstead of the 100 ms signal-check ticker.run_blocking'stokio::select!poll loop taxed every sub-100 ms call with 1-5 ms of first-tick jitter in the worst case.run_blocking_snapshotdrops the ticker entirely:py.detach { runtime().block_on(tokio::time::timeout(5s, fut)) }. The 5-second upper bound is a liveness safeguard — every observed production snapshot call completes in <200 ms, so the bound adds zero steady-state cost. Ctrl+C is still honoured after the future resolves or the timeout fires. Emitted by the generator only whenis_snapshot_endpointis true; parsed / list / streaming endpoints keep the existingrun_blockingpath unchanged. run_blockingsignal-check poll cadence reduced from 100 ms to 20 ms. Drops the worst-case select-wait on short parsed-kind calls from ~100 ms to ~20 ms.Python::check_signals()is ~1 µs per call so driving the ticker 5× as often has negligible steady-state cost. Long-running endpoints see no behavioural change beyond a slightly finer-grained Ctrl+C cancellation window. One-line constant edit insdks/python/src/lib.rs; the matching doc-comment is updated.README.md/sdks/python/README.md— positioning refreshed. Dropped the old small snapshot / calendar latency caveat now that the fast-path reduces overhead on every measured endpoint. Added a feature-gated Rust DataFrame quickstart example showingthetadatadx = { version = "8", features = ["polars"] }plus the chainedticks.as_slice().to_polars()?call site.- Generator-emitted snapshot fast-path converters (
<tick>_vec_to_pylist) insdks/python/src/tick_classes.rs. One helper per snapshot-return tick type (9 total:calendar_days_vec_to_pylist,ohlc_ticks_vec_to_pylist,quote_ticks_vec_to_pylist,trade_ticks_vec_to_pylist,market_value_ticks_vec_to_pylist,open_interest_ticks_vec_to_pylist,iv_ticks_vec_to_pylist,greeks_ticks_vec_to_pylist,price_ticks_vec_to_pylist); one helper per tick type that is NOT reached by any snapshot endpoint is suppressed at generation time to avoid dead-code. Emission is gated on a TOML-derived set computed by the newendpoints::snapshot_return_typeshelper — adding a snapshot endpoint of a new tick type toendpoint_surface.tomlautomatically opts its converter into emission on the next generator run. Row-building body reusespyclass_from_tick_exprfrom the<TickName>List.to_list()path so both surfaces emit byte-identical pylist contents.
8.0.5 - 2026-04-22
Endpoint performance fixes discovered during a pre-release performance review. Four regressions on the MDDS wire surface, all converging on one generator-level asymmetry: the Rust request builder was sending a different wire shape than the request contract on option endpoints, and on a subset of calls that difference tipped the server into an enumeration slow-path. No behaviour changes on the returned tick data, no signature changes on the SDK surface.
Fixed
option_list_dates— duplicate expiration field removed from the request wire shape. The v3OptionListDatesRequestQueryproto carries both aContractSpec(whoseexpirationis the contract identity) and a top-levelstring expirationfield (a vestigial wire field that predatesContractSpec). The generator was populating both with the same canonicalized date, which forced the server onto a per-contract enumeration path. Fixed inbuild_support/endpoints/render/mdds.rs::mdds_query_field_expr: when the query message also carries aContractSpec, the top-levelexpirationfield now emitsString::new()to match the request contract. Same one-line generator rule covers every option query message that carries both fields; no hand-written per-endpoint edits.option_at_time_quote— duplicate expiration field removed from the at-time quote path. The same top-levelexpirationduplicate that bottleneckedoption_list_datesalso penalized the at-time-quote path on dense option chains. Same generator-level fix applies:expirationonOptionAtTimeQuoteRequestQuerynow emitsString::new().option_history_greeks_eod— wire-shape parity restored on the wide-schema path. Same fix as the two items above; greeks-EOD sent the duplicateexpirationfield through the same code path.ContractSpec.strike/ContractSpec.right— wildcard sentinels now marshal as literal"*"/"both"on the wire. The previouswire_strike_opt/wire_right_optmapping reinterpreted the SDK-surface wildcards ("","*","0"for strike;"both"for right) as proto-unset optional fields. Upstream request examples populate these fields literally; the v3 server treats an unset optional as "enumerate every strike / right for this contract" (slow path) and an explicit"*"/"both"as "chain-wide lookup" (fast path). Both helpers now always returnSome(...)with the canonical wildcard literal. No signature changes on the SDK surface; callers continue to pass"*"/"both"unchanged.
Changed
README.md/sdks/python/README.md— positioning corrected to measured v8.0.4 bench numbers. Dropped legacy headline claims from v8.0.0-era measurements and replaced them with endpoint-specific, reproducible notes. Small snapshot / calendar calls are no longer described as speedups because network round-trip time dominates those calls.- 8.0.2 slice-direct Arrow narrative scoped to builder terminals. The 8.0.2 changelog bullet ("
.arrow()/.pandas()/.polars()feed decoder-ownedVec<tick::T>straight into Arrow column builders, peaking RSS at about the tick payload") described the builder-terminal path. The<Type>List.to_polars()non-builder terminal also reaches the slice-direct converter (slice_arrow::<tick>_slice_to_arrow_table), but the column-builder pass holds both the decoder-owned slice and the column vectors in memory simultaneously. The narrative in bothCHANGELOG.mdanddocs-site/docs/changelog.mdnow scopes the memory note to the implementation path that provides it.
8.0.4 - 2026-04-22
Pre-release review hotfixes on the Python binding. Four silent bugs on the hand-written pyo3 glue — Gregorian date validation, Python logging-hierarchy normalization, async GIL contention on heavy convert paths, and interpreter-finalization safety on Python 3.13+. No behaviour changes on the generated endpoint surface; every fix is confined to the hand-written utility files the endpoint generator layers depend on.
Fixed
sdks/python/src/chunking.rs—Ymd::from_yyyymmddaccepted Gregorian-impossible dates. The hand-rolled parser range-checked month1..=12and day1..=31independently, so20230229(Feb 29 in a non-leap year),20240231(Feb 31),20240431(Apr 31) and every other calendar-invalid combination slipped through.to_ordthen silently normalized the bogus day to a neighbouring valid one, producing wrong chunk boundaries when the 365-day auto-chunk helper split a range starting or ending on an impossible date. The validator now delegates tochrono::NaiveDate::parse_from_str(_, "%Y%m%d"), which enforces leap-year and month-length rules from the canonical Gregorian tables.chronois adopted as a new direct dep onsdks/python/Cargo.toml(pinned to=0.4.44,default-features = false,alloconly) and is already a transitive dep via the tzdb chain pulled in bythetadatadx, so the crate graph does not gain any new package. Covered by 12 new tests: Feb 29 in 2023/2024/1900/2000, Feb 30, Feb 31, Apr 31, Jun/Sep/Nov 31, month 0/13/99, day 0, and end-to-end rejection throughsplit_date_range.sdks/python/src/logging_bridge.rs— Rusttracingtargets were passed tologging.getLoggerwith::separators. Rusttracingemits targets as::-separated module paths (thetadatadx::auth::nexus,thetadatadx::fpss::io_loop, …). Python's stdliblogginghierarchy is.-separated. Consequence:logging.getLogger("thetadatadx").setLevel(logging.DEBUG)did NOT propagate tothetadatadx::auth::nexusevents — Python treated those as unrelated top-level loggers with no parent-level filtering. The v8.0.2 release notes' claim that parent-levelsetLevelfilters Rust-side events was therefore false. Fixed by rewritingtarget.replace("::", ".")in theLayer::on_eventhook before callinglogging.getLogger(...). Covered by one new test pinning the transformation on the canonical targets plus a Python-level test that exercises the fullgetLogger → setLevel → isEnabledForhierarchy propagation with both the post-fix (normalized) and pre-fix (unnormalized) names.sdks/python/src/async_runtime.rs—spawn_awaitableran the convert closure on the tokio runtime worker under GIL contention. The helper's inner async block wrappedconvertinPython::attach(|py| convert(py, value))directly inside thefuture_into_pybody, so heavy convert work (e.g. building a 955 237-rowQuoteTickListpyclass) parked the runtime worker for the duration of the Python-object build. Two concurrent*_asynccalls on the same worker serialized end-to-end on the GIL even though tokio had other workers free. Fixed by offloading the convert closure totokio::task::spawn_blocking, which is tokio's designated lane for synchronous / long-running work — the runtime worker is free to service other endpoints while the current call synthesizes its Python payload on a blocking-pool thread. Join-error handling routes panics throughJoinError::into_panic()to aPyRuntimeErrorso the shape of the awaitable's error surface is unchanged. The module-level docstring walks through why option A (returnT: IntoPyObjectand let pyo3-async-runtimes handle materialization) was rejected — the 122 generator-emitted callsites inhistorical_methods.rsand the matching templates inbuild_support/endpoints/render/python.rsall pass typed pyclass-wrapper helpers (strings_to_string_list,trade_ticks_to_pyclass_list, …) that aren't plainIntoPyObjectimpls onVec<T>; routing the existing convert closures to the blocking pool resolves the contention with zero ripple to the helper surface. Covered by a new wall-clock test that fires two concurrentspawn_awaitablecalls with 100 ms convert closures and asserts the combined elapsed time is less than 1.5× single-task (pre-fix serial behaviour would be ~ 2×).sdks/python/src/logging_bridge.rs—Python::attachcould panic during interpreter finalization on Python 3.13+. A background Rust thread emitting atracingevent during CPython teardown would callPython::attach, which panics when the interpreter is mid-finalization (documented pyo3 behaviour, sharpened on 3.13+). The panic took down the process before the layer's existingErr(_) => returnguard could swallow the resulting logger error. Fixed by switching toPython::try_attach(pyo3 0.28 API), which returnsNonewhen the interpreter is unavailable (finalizing, not initialized, or mid-GC traversal) and lets us silently drop the event. Shutdown-time event loss is an acceptable tradeoff vs. a crash during interpreter exit. Covered by a new test assertingtry_attachreturnsSomeon the live-interpreter path (the regression guard — a revert to plainattachwould lose the finalization-safety property) and by a documentation note in the module docstring's "Threading model" section.
8.0.3 - 2026-04-22
Python-UX polish: DataFrame conversion is now a chain on the returned list (tdx.stock_history_eod(...).to_polars()). The free-function and client-method to_polars(ticks) / to_arrow(ticks) / to_pandas(ticks) / to_dataframe(ticks) entry points are removed hard — there is now exactly one surface for converting tick data into a DataFrame.
Changed
- Chained DataFrame conversion on every list-returning endpoint. Every endpoint wraps its result in a typed
<ReturnType>Listpyclass (EodTickList,TradeTickList,QuoteTickList, …, plusStringList,OptionContractList,CalendarDayListfor non-tick list returns). The wrapper exposes.to_polars(),.to_arrow(),.to_pandas(),.to_list()and the list protocol. Usage istdx.stock_history_eod(...).to_polars()— no intermediate variable, no free-function round-trip. Builder terminals collapse from four parallel.list()/.arrow()/.pandas()/.polars()methods to a single.list()whose return carries the same chained terminals.
Removed
- Free-function and client-method conversion helpers removed.
thetadatadx.to_polars(ticks),thetadatadx.to_arrow(ticks),thetadatadx.to_pandas(ticks),thetadatadx.to_dataframe(ticks)and the identically-named methods on the client handle are deleted. Consumers migrate by chaining the terminal off the endpoint return value (tdx.stock_history_eod(...).to_polars()in place ofthetadatadx.to_polars(tdx.stock_history_eod(...))). One path, one SSOT, one place to audit.
Changed
- Generator-emitted
_asyncmethods delegate to aspawn_awaitablehelper mirroring the syncrun_blockingpattern. One call per emit replaces the open-codedpyo3_async_runtimes::tokio::future_into_py(...)+Python::attach+map_err(to_py_err)scaffolding that every_asyncmethod previously inlined.sdks/python/src/historical_methods.rssheds ~599 lines of duplicated plumbing. - Docs-site restructure. Deleted the standalone benchmark page, the migration-from-thetadata guide, the five per-language
quickstart/*.mdfiles, and the separate async-python narrative. Replaced with a unified code-group quickstart exposing Rust / Python / TypeScript / Go / C++ via language tabs so one page stays in sync across SDKs.
8.0.2 - 2026-04-21
Bigger than a typical patch: ships a P0 decode-correctness fix alongside a feature-additive wave across the Rust SDK and the Python bindings. Every surface added here is backward-compatible — no method signatures change, no types are removed, no client code needs to migrate. The patch-level version reflects that existing callers continue to compile unchanged; the additive surface opens new opt-in paths.
Fixed
- P11 —
stock_history_trade_quote/option_history_trade_quotesilently returnedOk(vec![])on non-empty responses. The v3 MDDS server emits the combined-row pair astrade_timestamp/quote_timestamp;tick_schema.tomldeclared them asms_of_day/quote_ms_of_daywith no aliases.find_headerfailed both required-header guards and the parser short-circuited before decoding any row. Added aliasesms_of_day↔trade_timestamp,quote_ms_of_day↔quote_timestamp,date↔trade_timestamp. Verified against a fresh prod capture: AAPLstock_history_trade_quotenow returns 955 237 rows, SPY option returns 98. Captured-response regression fixtures ship for seven endpoints (stock_history_trade_quote,option_history_trade_quote,stock_history_eod,option_history_greeks_all,option_history_trade,option_snapshot_ohlc,calendar_open_today) so the same class of schema drift fails at PR time next release. - Decoder audit —
parse_<tick>_ticksguard no longer drops rows on schema drift. Generator template and the hand-writtenparse_option_contracts_v3now raiseDecodeError::MissingRequiredHeaderwhen theDataTablecarries rows but declares none of the expected columns. Empty responses continue to returnOk(vec![])(a holiday with no trades remains a legitimate outcome). Walked everyVec::new()/unwrap_or_default()call-site indecode.rsandfpss/decode.rs— the remaining ones are intentional soft-fail accessors (bench / macro) or per-event nibble buffers, flagged as such in the audit report.
Added
- Async Python surface — every historical endpoint gains an
_asynccompanion.client.stock_history_eod_async(...)returns an awaitable built onpyo3_async_runtimes::tokio::future_into_py. Sync and async paths share the sameOnceLock<tokio::runtime::Runtime>singleton — one runtime, one connection pool, one request semaphore. - Fluent builders —
tdx.<endpoint>_builder(...)returns a per-endpoint#[pyclass]with chainable setters and.list()/.arrow()/.pandas()/.polars()terminals plus_asynccompanions. Builder holdsArc<thetadatadx::ThetaDataDxClient>so every terminal drives the original client without re-authenticating. decode_response_bytes(endpoint, chunks)— generator-emitted#[pyfunction]that feeds recordedVec<&[u8]>proto::ResponseDataframes through the Rust decoder and returns the typed pyclass list, so external parity benches can attribute wall-clock cost between network and decode without an MDDS round-trip. Auto-wired for every endpoint that has a typed decoder.- Layered exception hierarchy —
thetadatadx.ThetaDataErrorroot plus nine leaves:AuthenticationError,InvalidCredentialsError,SubscriptionError,RateLimitError,SchemaMismatchError,NetworkError,TimeoutError,NoDataFoundError,StreamError.to_py_errmaps everythetadatadx::Errorvariant (plus gRPC status strings) onto the correct leaf.#[non_exhaustive]catch-all. - Python logging bridge —
tracing_subscriber::Layerthat forwards everytracingevent tologging.getLogger(target).log(...). Filter-first viaisEnabledFor(level)so default WARN loggers pay a single bool check per event with no formatting. Installed at module init. - Slice-based Arrow fast path on builder terminals —
.arrow()/.pandas()/.polars()(and their_asynccompanions) feed the decoder-ownedVec<tick::T>straight into the Arrow column builders, skipping the pyclass-list double-buffer. The<Type>List.to_polars()terminals on the typed-list wrapper also reach this slice-direct path; the column-builder pass holds the decoder-owned slice and the column vectors simultaneously. Schema is bit-identical to the pyclass-list path so downstream consumers alias either source interchangeably. (Language narrowed from the initial memory-footprint claim in v8.0.5 — see that entry.) RetryPolicy— initial_delay 250 ms, max_delay 30 s, max_attempts 5, full jitter by default. Retries only onUnavailable/DeadlineExceeded/ResourceExhausted. Unit-tested backoff math, jitter bounds, and thedisabled()shortcut.- Session auto-refresh —
auth::SessionTokenholds the session UUID behind atokio::sync::Mutex+ monotonic version counter. OnUnauthenticatedthe retry loop snapshots the token, re-auths via Nexus, swaps the UUID in place, and retries exactly once. A second 401 fails permanently. Concurrent 401s dedupe into a single Nexus round-trip via version-check short-circuit. - Environment-variable config matrix —
DirectConfig::production()layers env vars on the hardcoded defaults:THETADATA_MDDS_HOST,THETADATA_MDDS_PORT(upstream-compat), plus DX extensionsTHETADATA_NEXUS_URL,THETADATA_FPSS_HOST,THETADATA_FPSS_PORT,THETADATA_CLIENT_TYPE. Precedence: explicit builder setter > env var > hardcoded default. - Optional
metrics-prometheuscargo feature — pullsmetrics-exporter-prometheusand wires an HTTP/metricslistener onDirectConfig::metrics_port. Exporter starts insideThetaDataDxClient::connectso the first RPC counter is already covered. Feature-gated; default build stays dep-free. - Vendor docstring lift — 60 endpoint docstrings threaded through
endpoint_surface.toml→ model → parser → generator so sync / async / builder variants share one SSOT. Attribution recorded indocs/ATTRIBUTION.md. split_date_range(start, end)— pure Rust 365-day-window splitter exposed asthetadatadx.split_date_rangefor tooling and the auto-chunk pre-flight. Tested on single-day, exact boundary, multi-year contiguity, leap-day, and invalid input.- Capture fixtures — seven
tests/fixtures/captures/<endpoint>.{pb.zst,meta.toml}pairs anchor expected row counts, exact server header lists, and first-row field values.tests/test_decode_captures.rsfeeds each fixture through the samedecode_data_table→ tick-parser path theMddsClientuses and asserts three invariants per fixture. Two regression guards ensureMissingRequiredHeaderfires on non-empty schema drift and empty responses still returnOk(vec![]).
Changed
- Regenerated SDK surfaces —
historical_methods.rs,tick_arrow.rs,decode_bench.rsrebuilt off the merged generator. Byte-identical check passes. - Parser generator raises
MissingRequiredHeaderon schema drift — the generatedparse_<tick>_tickstemplate no longer silently returnsOk(vec![])when a required column is absent on a non-emptyDataTable. Empty responses continue to pass through unchanged.
8.0.1 - 2026-04-21
Fixed
tdbebumped to 0.11.0 to publish the newSecType::Unknownvariant to crates.io — the 8.0.0 release addedSecType::Unknown(empty-contract sentinel) but kepttdbeat0.10.0.cargo publish --verifyforthetadatadx 8.0.0pulledtdbe = 0.10.0from the registry, which does not containUnknown, and failed withE0599. Thethetadatadx,ffi,cli,mcp,server,py, andnapicrates bump to8.0.1so all three ecosystems (crates.io, PyPI, npm) end up on matching, publishable versions. npm and PyPI had already published 8.0.0 successfully; crates.io 8.0.0 was never materialized.- FPSS handshake surfaces every typed control frame —
wait_for_logincollectsConnected(code 4),Ping(code 10),ReconnectedServer(code 13), andRestart(code 31) frames that arrive beforeMETADATAinto an ordered buffer; the I/O loop drains the buffer onto the event bus before emittingLoginSuccessso user callbacks see the exact wire order. Previously all typed control frames exceptConnectedwere silently dropped by the handshake's trace-and-continue branch. Applies to the initial login AND the reconnect-path login. - Reconnect-path login short-circuits on permanent rejection —
LoginResult::Disconnected(reason)during the reconnect handshake now consultsreconnect_delay(reason)as the single source of truth for "no retry will fix this" and exits the I/O loop withshutdown = true+ aFpssControl::Disconnectedevent. Previously bad credentials burnedMAX_RECONNECT_ATTEMPTS(5) cycles ofReconnecting/Disconnectednoise before giving up. - Mid-frame reader yields to the command drain on a bounded budget —
FrameReadStatethreads partial-frame progress acrossread_frame_intocalls. A newMID_FRAME_DRAIN_WINDOW_MS = 200(4× the 50 ms drain cadence) caps the total wall time spent retrying a partial frame before the reader yields control to the I/O loop, which drains outbound commands and re-enters the reader with the preserved state. Previously a trickling sender could block heartbeats / user writes for up toREAD_TIMEOUT_MS(10 s) because the per-stall deadline reset on every successful byte. Contract::from_straccepts 1..=16-char roots —validate_rootwidens from1..=6to1..=16chars, matching the wire-codec upper bound inContract::to_bytes()/Contract::from_bytes().from_str/to_bytes/from_bytesnow round-trip symmetrically; the wire is the ground truth. Round-trip coverage for every length 7..=16 added.- Auth email redacted across
Debugand tracing —AuthResponse::Debug,AuthUser::Debug, and theauthenticate()tracing line that previously renderedemail = %creds.emailnow emit<redacted>/ a prefix-onlyali...@example.comform. Full emails no longer land in panic output, structured logs, or crash dumps. - Credentials parsing pipeline wraps every transient in
Zeroizing—from_filereads the file intoZeroizing<String>so the on-disk password bytes are wiped on drop;parse()/new()wrap the intermediate owned passwordStringinZeroizingbefore assigning to the struct. A panic or early-return between allocation and struct construction still wipes the plaintext on unwind. Completes the coverage the 8.0 release notes claimed; the previous implementation zeroed only the finalCredentials.passwordfield. - Empty-contract sentinel documentation unified —
FpssData::{Quote,Trade,OpenInterest,Ohlcvc}docstrings now promotecontract.sec_type == SecType::Unknownas the canonical check for the empty-contract placeholder (matchingfpss::decode's guidance).root.is_empty()is retained as a secondary mention but no longer the primary documented check -- it was brittle against future root-charset relaxations.
8.0.0 - 2026-04-21
Major release. Three headline groups land in one pass:
- FPSS events now carry a parsed
Arc<Contract>(#389). EveryFpssData::{Quote,Trade,OpenInterest,Ohlcvc}replaces thesymbol: Arc<str>field withcontract: Arc<Contract>, and thecontract_maplifts fromHashMap<i32, Contract>toHashMap<i32, Arc<Contract>>. Decoded events carry the full typed contract (root,sec_type,exp_date?,is_call?,strike?) at refcount cost rather than a bare symbol string; every language SDK exposes a matching typedContract.SecType::Unknownis added as the sentinel for not-yet-assigned contract IDs so exhaustive matches stay sound. impl FromStr for Contractplus historical FPSS subscribe shortcuts (#389)."AAPL".parse::<Contract>()?yields a stock contract;"SPY 260417C00550000".parse::<Contract>()?parses the OCC 21-char option identifier (2000–2099 scope, trim-tolerant 20-char pad, every parse failure returnsError::Configwith the offending input).FpssClientandThetaDataDxClientgained per-security subscribe and unsubscribe shortcuts — one-liners over the underlying typed subscribe machinery.- FPSS control codes 4 / 10 / 13 / 31 decode into typed variants (#389).
FpssControl::{Connected, Ping { payload }, ReconnectedServer, Restart}replace theUnknownFramefallthrough these codes used to hit. TheRestartarm clears delta decode state so subsequent ticks no longer decode against a stale baseline. FFI kind tags grow 13..=16; every SDK mirrors the new constants.
Removed
FpssData::{Quote,Trade,OpenInterest,Ohlcvc}::symbolremoved (#389) — migrate toevent.contract.rootfor the symbol string; option fieldsexp_date,strike,is_callare now direct attribute access oncontract.FpssControl::ContractAssigned { contract: Contract }→{ contract: Arc<Contract> }(#389) — pattern matches that bind by value must bind byArc<Contract>and clone viaArc::cloneif owned value was previously expected.contract_lookup()/contract_map()returnArc<Contract>/HashMap<i32, Arc<Contract>>(#389) — was by-valueContract/HashMap<i32, Contract>before. Call-site fix: drop one layer of.clone().Restart(code 31) andConnected(code 4) frames no longer arrive asUnknownFrame(#389) — handlers matching onFpssControl::UnknownFrame { code: 4 | 10 | 13 | 31, .. }need updated arms or a fallthrough on the new typed variants.SecType::Unknownvariant added totdbe::types::enums::SecType(#389) — exhaustivematchstatements without a wildcard arm must add a branch.FpssData::{Quote,Trade,OpenInterest,Ohlcvc}no longerderive(Clone)on the Python SDK pyclasses (#389) —Py<Contract>needs a GIL token for cloning; the derive was dead code (events flow one-way from Rust to Python).
Changed
- License switched to Apache-2.0 across every
Cargo.toml,package.json,pyproject.toml, and the top-levelLICENSE.deny.tomlallowlist cleaned up accordingly. - Top-level
README.mdrewritten as a professional SDK landing page: tagline, highlights, per-SDK quickstart (Rust / Python / TypeScript / Go / C++), architecture diagram, Java parity note. Neutral technical framing throughout. docs/java-parity-checklist.mdadded as the single source of truth for Java terminal parity — feature-by-feature table (parity / deviation / partial) covering wire protocol, authentication, control events, reconnection, FPSS streaming, tick decoding, Greeks, validation, and intentional improvements over the Java terminal. Three earlier stand-alone documents (docs/jvm-deviations.md,docs/java-class-mapping.md, and a prior protocol-archaeology note) folded in.- Internal
docs/dev/design notes removed (no longer load-bearing). DirectClientrenamed toMddsClient(#383) — the historical-data gRPC client now carries the name of the service it actually speaks to (MDDS = Market Data Delivery Service).use thetadatadx::DirectClientcall sites break; update touse thetadatadx::MddsClient. TheDirectConfigassociated config type keeps its name. High-level consumers ofThetaDataDxClient(Python / TypeScript / Go / C++ / Rust facade) are unaffected.crates/thetadatadx/src/direct.rssplit intocrates/thetadatadx/src/mdds/module (#383) — 732-line monolith broken into six concern-separated files (client,endpoints,endpoint_arg_ext,normalize,validate,mod). Pure move; wire behavior unchanged; all 304 workspace tests pass.crates/thetadatadx/proto/external.protorenamed tomdds.proto(#385) — the proto file described only MDDS (BetaEndpoints) messages; the filename now reflects that.tonic::include_proto!("beta_endpoints")and every downstream Rust import resolve unchanged (package declaration drove the module name, not the filename).build.rs,proto_parser, generated-header strings,MAINTENANCE.md,CONTRIBUTING.md,ROADMAP.md, and everydocs/reference updated (17 files, 51 lines).fpss_event_schema.tomlschema version bumped 2 → 3 (#389) — carries the new nestedContractcolumn type for every data-event variant. Every SDK Contract type (Python pyclass, TypeScript#[napi(object)], Go struct with*int32/*boolpointer optionals, C/C++ typedef withhas_*tagged-optional flags, Rust FFI#[repr(C)] TdxContractwithCString-backed root pointer) is generator-emitted from the updated schema.
Added
- Parsed
Arc<Contract>on every FPSS data event (#389) —FpssData::{Quote,Trade,OpenInterest,Ohlcvc}::contract: Arc<Contract>replaces the formersymbol: Arc<str>. Option events now exposeevent.contract.exp_date,.strike,.is_callwithout a second lookup; stock events readevent.contract.root. Refcount-only per-event clone. Mirrorsnet.thetadata.fpssclient.Contractfrom the Java terminal without the JSON round-trip.contract_lookupandcontract_mapreturnArc<Contract>/HashMap<i32, Arc<Contract>>on every SDK. impl FromStr for Contract(#389) —"AAPL".parse::<Contract>()?yields a stock contract (1..=6 ASCII A-Z,.permitted);"SPY 260417C00550000".parse::<Contract>()?parses the OCC 21-char institutional option identifier (6-byte root right-padded with spaces, 6-byte YYMMDD century-adjusted to 2000–2099 YYYYMMDD, single-byteC/P, 8-byte strike in thousandths of a dollar). 20-byte inputs are tolerated with a trailing-space pad. Parse failures returnError::Confignaming the offending input and the specific failure (length, root charset, expiration digits, right byte, strike digits).- Historical FPSS subscribe shortcuts (#389) — per-security subscribe and matching unsubscribe counterparts were added on
FpssClientandThetaDataDxClient. Each wraps theContractbuilder plus the typedsubscribe/unsubscribecall into one line; no duplicate request-ID or frame-build machinery. - Typed decoding of FPSS control codes 4 / 10 / 13 / 31 (#389) —
FpssControl::Connected(4),FpssControl::Ping { payload }(10),FpssControl::ReconnectedServer(13 — server-side ack, distinct from the client-side auto-reconnectReconnectedvariant), andFpssControl::Restart(31) replace theUnknownFramefallthrough these codes used to hit. TheRestartarm clears delta decode state so subsequent ticks no longer decode against a stale baseline. FFITdxFpssControlkind tags grow 13..=16; GoFpssCtrl*constants mirror them. Contracttype surfaced on every language SDK (#389) — Python pyclass (Py<Contract>embedded in each event, cloned viaclone_ref(py)), TypeScript#[napi(object)], Go struct with*int32/*boolpointer optional fields, C/C++ typedef withhas_*tagged-optional flags, Rust FFI#[repr(C)] TdxContractwith aCString-backedrootpointer.Contract.sec_type == SecType::Unknownis the sentinel for not-yet-assigned contract IDs; every SDK exposes the new variant.thetadatadx.to_arrow(ticks) -> pyarrow.Table(#379) — new public Python entry point that returns the Arrow table directly, for users wiring DuckDB / Arrow-Flight / cuDF / polars-arrow pipelines without a pandas or polars roundtrip. Requirespip install thetadatadx[arrow](pyarrow only).hint=kwarg onto_arrow/to_dataframe/to_polars(#380) — optionalhint: strnames the tick pyclass (e.g.hint="EodTick") so the Arrow schema is materialised even when the input list is empty. Previous empty-list calls returned a zero-column table; downstream pipelines asserting a fixed schema now get the right columns on empty market-hours windows.- Generated
#[new]constructors on every tick pyclass (#379) —EodTick(ms_of_day=1, volume=1_000_000, ...),OhlcTick(...),TradeTick(...), etc. All fields are keyword-only with zero / empty-string defaults, so test fixtures and user-side data construction are possible from Python (previously pyclass instances could only be produced by Rust endpoints). AllGreekspyclass (#378) —all_greeks(...)now returns a frozenAllGreekspyclass with 22#[pyo3(get)]f64 fields (value / iv / delta / gamma / theta / vega / rho plus every second- and third-order Greek) and a__repr__showing the six most-referenced values. Replaces the untyped 22-keyPyDictthat was the sole remaining dict-typed public return in the Python SDK.__repr__on every FPSS event pyclass (#380) —Ohlcvc,Quote,Trade,OpenInterest,Simple,RawDatanow render up to six live field values at the Jupyter / print boundary (matching the pattern already on tick pyclasses). OpaqueVec<u8>payloads andreceived_at_nsskipped as noise.dropped_events()counter on every streaming SDK (#377) —Arc<AtomicU64>hoisted ontoThetaDataDxClientsurvives reconnect and is exposed astdx.dropped_events() -> int(Python),tdx.droppedEvents(): bigint(TypeScript),client.DroppedEvents() uint64(Go),client.dropped_events() -> uint64_t(C++),tdx_fpss_dropped_events(handle)/tdx_unified_dropped_events(handle)(FFI). Previously silentlet _ = tx.send(buffered)call-sites now bump the counter and emittracing::debug!on targetthetadatadx::sdk::streaming.POST /v3/system/shutdownendpoint onthetadatadx-server(#377) — graceful shutdown over a privileged route gated by a per-startup random UUIDX-Shutdown-Tokenheader (constant-time compared viasubtle::ConstantTimeEq). Prints the token to stderr at startup only; never into structured logs. Dedicated governor allows one attempt per hour, burst 3. Method isPOST(notGET) so the action is neither cached nor prefetched.
Changed
DataFrame adapter migrated to Apache Arrow columnar pipeline (#379) —
to_dataframe(ticks)/to_polars(ticks)/to_arrow(ticks)build a singlearrow::RecordBatchin Rust and hand it to pyarrow via the Arrow C Data Interface (zero-copy at the pyo3 boundary). pandas 2.x aliases the numeric columns in place; polars consumes viapolars.from_arrow. At 100k x 20EodTickrows wall-clock drops from ~300-500 ms (legacy dict-of-lists) to ~8 ms — substantially. SSOT preserved: Arrow schema + converters are generated fromtick_schema.toml; no hand-maintained Arrow code.Per-endpoint DataFrame convenience wrappers removed (#379) — the four per-endpoint
stock_history_{eod,ohlc,trade,quote}Rust-tick-slice fast-path helpers onThetaDataDxClientwere deleted. The unified recipe is one extra line with identical performance:pythonticks = client.stock_history_eod("AAPL", "20240101", "20240301") df = thetadatadx.to_dataframe(ticks) # Arrow-backed, zero-copy on pandas 2.x pdf = thetadatadx.to_polars(ticks) # Arrow-backed, zero-copy table = thetadatadx.to_arrow(ticks) # DuckDB / cuDF / Arrow-FlightSingle code path, single generator, single test surface — 100% SSOT restored on the Python DataFrame surface.
Deleted
sdks/python/src/tick_columnar.rs(the old PyDict-based emission) (#379) — replaced end-to-end by the generator-emittedsdks/python/src/tick_arrow.rs.pip install thetadatadx[pandas]/[polars]now pullpyarrow>=14.0alongside the DataFrame library;pip install thetadatadx[arrow]is the pyarrow-only extras bundle.
Changed
Historical endpoints now return
list[TickClass]instead of a columnardict[str, list](#364 / #365). The 53 tick-returning historical methods (list endpoints returning scalarVec<String>— symbols, dates, expirations, strikes — are unchanged) in the Python SDK (stock_history_eod,option_history_trade,calendar_*, ...) now return a Python list of typed pyclass objects —EodTick,TradeTick,QuoteTick,OhlcTick,TradeQuoteTick,OpenInterestTick,MarketValueTick,GreeksTick,IvTick,PriceTick,CalendarDay,InterestRateTick,OptionContract. Brings the Python SDK into line with Rust core, TypeScript, Go, and C++ FFI. Migration:python# before ticks = tdx.stock_history_eod("AAPL", "20240101", "20240301") close = ticks["close"][i] # string key, silent typo failures # after ticks = tdx.stock_history_eod("AAPL", "20240101", "20240301") close = ticks[i].close # attribute access, typedto_dataframe(ticks),to_polars(ticks), andto_arrow(ticks)transparently pivot the new shape into a pandas / polars frame or apyarrow.Table.
Changed
- C++
TdxFpssEventfield order realigned with Rust + Go (#376) — the hand-writtenTdxFpssEventinsdks/cpp/include/thetadx.hdeclared{ kind, quote, trade, open_interest, ohlcvc, control, raw_data }while the Rust generator (and the Go C header) emits{ kind, ohlcvc, open_interest, quote, trade, control, raw_data }. Everyevent->quote.*/event->trade.*/event->ohlcvc.*access in existing C++ consumers was reading from the wrong offset — data corruption with no compile-time signal.thetadx.hnow#includes the generator-emittedfpss_event_structs.h.inc(byte-identical to the Go C header) andthetadx.hppgainsstatic_assert(offsetof / sizeof)covering every field of everyTdxFpss*struct. Any future drift is compile-fatal. - Go
FpssControlDatarenamed toFpssControl,FpssOpenInterest*→FpssOpenInterest(#376) — Go-idiomatic naming on the mirror struct set. Callers referencing the old names will fail to compile; rename one-for-one. The nested field names onFpssEvent(ev.RawData.Code,ev.RawData.Payload) are unchanged.
Changed
thetadatadx::directmodule removed; replaced bythetadatadx::mdds— the 732-line flatsrc/direct.rsis split into a concern-separatedsrc/mdds/module that mirrors the existingfpss/layout:client.rs(struct + connect),stream.rs(gRPC response helpers),validate.rs(param validators),normalize.rs(wire-format canonicalizers +contract_spec!macro),endpoints.rs(generatedinclude!sites). The generator modulebuild_support/endpoints/render/direct.rsis renamed torender/mdds.rsand now emitsmdds_*_generated.rsintoOUT_DIR; the template directorytemplates/direct/is renamed totemplates/mdds/. "MDDS" is the actual upstream gRPC service name — "direct" conveyed nothing.DirectClientrenamed toMddsClient— the struct inside the (now)mdds/module takes its module's name. Re-exported at the crate root asthetadatadx::MddsClient.ThetaDataDxClientstillDeref<Target = MddsClient>s, so every historical endpoint method is reached unchanged via the unified client.
Changed
thetadatadx-server: governor layer is now outermost, rate-limited traffic short-circuits first (#377) — axum.layer(X).layer(Y)makes Y the outer wrapper, so the previousConcurrencyLimit → BodyLimit → Governororder had the per-IP limiter innermost. Every rate-limited request still consumed a concurrency permit and ran the body-length check before being rejected. Reordered so the governor runs first; body-limit and concurrency gates are only touched by allowed traffic.thetadatadx-server:PeerIpKeyExtractoron the REST + WS routers (#377 / #378) — the per-IP rate limiter now keys on the real TCP socket source instead of the forwarded-header-trusting extractor used before. The server defaults to127.0.0.1without a trusted reverse proxy in front, so trustingX-Forwarded-For/X-Real-IP/Forwardedlet a local attacker cycle fake IPs and bypass the per-IP rate limit. Module doc comment spells out the deployment policy.thetadatadx-server:BoundedQuery<N>extractor caps query-string params during parse (#378) — the previous check ran after axum'sQuery<HashMap<String, String>>had already parsed the entire query string into a HashMap, so a?a=1&b=2&...flood still allocated MB+ before hitting the count check.BoundedQuery<32>counts&-delimited pairs on the raw URI beforeserde_urlencoded::from_str, rejects over-limit with 400, and caps HashMap capacity.thetadatadx-server: WS subscribe + every REST validator now runensure_no_control_chars+ per-field length caps (#377) — symbol / root ≤ 16, expiration == 8 (YYYYMMDD), strike ≤ 10, right == 1, date == 8, venue ≤ 8. Returns 400 with a descriptive error, never 500. Unknown query-param names surface the real name in the error instead of an opaque"parameter"fallback.thetadatadx-server: REST global concurrency limit 256, per-IP governor 20 rps / burst 40, body limit 64 KiB, WS text-frame cap 4 KiB (#377 / #378) — explicit layers on both routers. Legitimate subscribe commands are <200 B; 4 KiB is generous for pathological clients.thetadatadx-server: shutdown rate limit fixed — one token per hour, burst 3 (#377 follow-up) —per_second(3600)treats the argument as "requests per second", so the "3 attempts per hour" config was actually allowing ~3600 rps. Switched to.period(Duration::from_secs(3600)); constant renamed toSHUTDOWN_REPLENISH_PERIOD.thetadatadx-server: hot-pathString::cloneeliminated on FPSS TOCTOU contract map (#378) — the broadcast path now holdsHashMap<i32, Arc<Contract>>instead ofHashMap<i32, Contract>; mpsc channel carries(FpssEvent, Option<Arc<Contract>>). Hot-path clone is anArcrefcount bump instead of aStringallocation. Micro-bench (100k lookups): 26 ns/op → 22 ns/op, zero hot-path heap allocations. Regression testarc_contract_clone_is_refcount_bump_not_string_allocassertsArc::as_ptrequality to prevent future regressions.- TypeScript
const enum FpssEventKindremoved (#376) — the generated enum broke downstream consumers with"isolatedModules": trueintsconfig.json(all modern Vite / esbuild / ts-jest / Next.js setups).FpssEvent.kindis nowpub kind: &'static strwith a#[napi(ts_type = "'ohlcvc' | 'open_interest' | 'quote' | 'trade' | 'simple' | 'raw_data'")]override. Zero-allocation preserved; discriminated-union narrowing unchanged. go.modtoolchain bumped to 1.23 (#378) — Go 1.21 released mid-2023; CI matrix already runs 1.23. Node.jsengines.nodebumped from">= 18"to">= 20"(Node 18 EOL 2025-04-30).pastecrate replaced bypastey(#377) — upstreampastewas archived on 2024-10-07 (RUSTSEC-2024-0436).pastey = "0.2.1"is the actively-maintained successor; API compatible (::paste::paste!→::pastey::paste!). Single call-site incrates/thetadatadx/src/macros.rs.
Fixed
- FFI boundary catches Rust panics (#380) — zero
catch_unwindexisted across the FFI crate before this change. A Rust panic crossing anextern "C"boundary on Rust 1.81+ aborts the host process — C / Go / Python / C++ callers died with no way to recover. Newffi_boundary!macro wraps every extern body instd::panic::catch_unwind(AssertUnwindSafe(|| { ... })). Panic payloads are downcast to&'static strthenString, routed totracing::error!on targetthetadatadx::ffi::panic, written to the thread-localLAST_ERRORslot via the existingset_error, and the fn returns the caller-declared default (ptr::null_mut()/-1/0/ sentinel-empty-array). Coverage: 145 productionextern "C"functions wrapped — 84 inffi/src/lib.rsplus 61 in the generatedffi/src/endpoint_with_options.rs. Generator-emitted so future regeneration preserves parity. Regression tests atffi/tests/panic_boundary.rs. - Python
next_event(timeout_ms)honours Ctrl+C within 100 ms (#380) — previously the generator emitted a singlerecv_timeout(Duration::from_millis(timeout_ms))with the GIL released for the full user-supplied timeout (up to 5 minutes), so Ctrl+C was swallowed for the duration of the wait.build_support/sdk_surface.rsnow emits a 100 ms polling loop that callsPython::check_signals()?per iteration and returns on deadline. ThetaDataDxClient::newconstructor is cancellable (#380) — swappedrun_in_tokio_blockingforrun_blocking(py, async { connect(...).await })so a TLS / auth handshake hang stays Ctrl+C-interruptible.- FPSS TLS: SPKI pinning replaces
NoVerifier(#377) —PinnedVerifierparses the leaf cert viax509-parser, computes SHA-256 over the SubjectPublicKeyInfo DER bytes, and constant-time compares (subtle::ConstantTimeEq) against the capturedFPSS_SPKI_SHA256(verified identical across prodnj-a:20000/nj-b:20000, dev:20200, stage:20100— single keypair across every FPSS environment). Rejects withCertificateError::NotValidForNameon hostname mismatch (allowlist) orRustlsError::General("FPSS SPKI pin mismatch: ...")on pin mismatch.verify_tls12_signature/verify_tls13_signaturedelegate to rustls' proper signature verification. Previously any on-path attacker terminating TLS tonj-a.thetadata.us:20000could present any cert and harvest the plaintextStreamMsgType::Credentialsframe. - Password
Zeroizing<String>(#377) —Credentials.passwordwrapped inzeroize::Zeroizing<String>. Every clone (ThetaDataDxClient,io_loop, reconnect re-serialise) now wipes the backing buffer on drop. Core dump //proc/<pid>/memno longer recovers the password afterCredentialsdrops.Deref<Target = str>means call-sites are unchanged. - CSV formula injection defused on
thetadatadx-serverexports (#377) —escape_csv_fieldnow prefixes cells whose first byte is=,+,-,@, or\twith a single-quote'and encloses in CSV quotes. Defuses=cmd|'/C calc'!A1,@SUM(A1:A10),+1+cmd|...etc from executing in Excel downloads. Regression test covers all five payload shapes. - FPSS
io_loop: Java-parity mid-frame read retry with per-read deadline reset (#370) — previously a mid-frame read timeout desynced the decoder. The client now retries transparently with the per-read deadline reset, matching the Java terminal's reconnect behaviour. - WS subscribe strike / expiration use
i32::try_from(#377) — client-supplied expiration / strike no longer silently narrow viaas i32. ReturnsREQ_RESPONSE { response: "ERROR", ... }with a descriptive message on overflow. Validatesexpagainst[19000101, 21000101]YYYYMMDD bounds andstrike > 0before building the FPSS frame (#378). validate_generic_namedsanitises parameter names in error messages (#377 / follow-up) — ANSI escape sequences / control chars in a user-supplied param name can no longer escape into terminal-rendered log output. Names are passed throughsanitize_param_name(ASCII alphanumeric +_+-).- Shutdown token constant-time compare (#377) —
tools/server/src/state.rs::validate_shutdown_tokenswapped==forsubtle::ConstantTimeEq::ct_eq. Timing oracle on UUID prefix closed. - Reconnect-path write errors are surfaced, not masked (#377) —
crates/thetadatadx/src/fpss/io_loop.rshadlet _ = write_raw_frame_no_flush(...)silently dropping write failures on reconnect command-drain. Nowtracing::warn!witherror = %e, frame_code = ?frame.code. - FFI reconnect paths surface resubscribe errors (#378) — unified + FPSS reconnect paths previously silent-dropped resubscribe errors; now
tracing::warn!witherror,kind, and contract context. - Python
Credentials.__repr__redacts the email (#377 / #378) — wasCredentials(email="user@example.com"); email leaked into Jupyter, pytest output, and crash logs. NowCredentials(email=<redacted>). Matches the redactedDebugimpl incrates/thetadatadx/src/auth/creds.rs. - CSV headers union across rows (#376) —
tools/server/src/format.rsseeded column keys from the first row only; mixed-type queries (index rows withoutexpiration/strike/rightahead of option rows with them) silently dropped those columns. Headers now union across every row viaBTreeSet(sorted for free). - FPSS
Simplecontrol events carryevent_type+ nullabledetail/id(#378) — OpenAPIControlvariant was documenting the internal numerickind: int32, which no SDK surfaces. Aligned to the client-facing shape (kind: "simple"+event_typeenum + nullabledetail/id+received_at_ns). - Python
greeks.pyexample + README quick-start use attribute access onAllGreeks(#380) —g['iv']/g['delta']dict subscripts would have crashed at runtime becauseAllGreeksis a frozen pyclass without__getitem__. Rewritten tog.iv,g.delta, etc. - Typed
list[TickClass]examples across every endpoint page (#378) — ~50 files underdocs-site/docs/historical/had stale dict-key Python examples (subscript access on the old columnar shape). Switched to attribute access on the typed pyclass surface.scripts/fpss_smoke.py/scripts/fpss_soak.pylikewise switched from dict subscript on streaming events to attribute access (both scripts are wired into live CI).
Security
- FPSS TLS authenticity anchored on captured SPKI pin, no longer trust-on-first-use (#377) — see
Fixedabove. Cert rotation tolerated as long as the keypair stays; expiry sidestepped entirely (current ThetaData leaf expired 2024-01-12). Six new tests cover captured-leaf positive, hostname mismatch rejection, malformed-cert rejection, and openssl fingerprint reproducibility. - Cargo-deny advisory / licence / drift gates in CI (#377) — new
.github/workflows/security-audit.ymlruns RustSecaudit-checkon PR + push + weekly Monday 03:00 UTC cron + manual dispatch. Newcargo-denyjob reads policy fromdeny.toml(advisories deny, licences allowlist, bans duplicates warn, sources crates.io only). Newdrift-injectionjob runsscripts/test_drift_injection.shwhich flipsbid↔askin the FPSS schema, regenerates, and verifies the C++static_assert(offsetof)guards fail the cmake build.
Changed
Generator audit cleanup (#380) —
PYTHON_TICK_ARROW_DIRECT_TYPESconstant +render_python_tick_arrow_batch_fn(~70-line emitter) were orphaned by the*_dfremoval in #379 and survived only because of the module-level#![allow(dead_code)]umbrella. Deleted. The trait-drivenpyclass_list_to_arrow_tablepath is the sole public DataFrame entry point, backed by<T as ArrowFromPyclassList>::read_batch.render_python_tick_arrowdoc rewritten to describe the two still-emitted surfaces (arrow_schema_for_qualname+pyclass_list_to_arrow_table).clippy::type_complexityon a 4-tuple insdk_surface.rscleared via aMethodShape<'a>alias.Go layout regression:
TestTickFieldOffsetscovers every tick mirror field (#376) — the previousffi_layout_test.goonly asserted total structsizeof; same-size field reorders (e.g. swapping two i32 slots) passed the test while silently corrupting data. FPSS mirror types were not tested at all. cgo-typed FPSS offset asserts moved intotick_ffi_mirrors.go::init()(Go forbids cgo in_test.go).Full stale-data sweep + i64 widening across every doc surface (#375 / #378) —
OhlcTick/EodTickvolume + count widened fromi32toi64(#372 on the Rust side). Docs updated acrossdocs/api-reference.md,docs-site/docs/api-reference.md,docs-site/public/thetadatadx.yaml, and every per-endpoint page. Stale14 tick typesreferences corrected to 13.[Unreleased]compare link fixed fromv7.2.0...HEADtov7.3.1...HEAD; missingv7.2.1/v7.3.0/v7.3.1tag compares added.Toml crate metadata warning silenced (#377) —
toml = "1.1.2+spec-1.1.0"→toml = "1.1.2"in both[dependencies]and[build-dependencies]. Everycargo buildinvocation no longer warns about ignored semver metadata.Workspace manifest consolidated via
[workspace.package]+[workspace.lints](#384) — duplicateedition/license/authors/repository/homepage/rust-versionremoved from every memberCargo.tomland hoisted to the workspace root; each member inherits viax.workspace = true. A new[workspace.lints.rust]table denies the rustcwarningsgroup (matching CI's-D warnings) and promotesunsafe_op_in_unsafe_fnto deny alongside;[workspace.lints.clippy]deniesclippy::all. Every member crate opts in via[lints] workspace = true. Versions intentionally stay per-crate becausetdbeships on a0.xtrack independent of the7.xSDK line.tools/server/src/ws.rssplit intotools/server/src/ws/module (#384) — 1044 lines reorganised intoupgrade·session·subscribe·broadcast·contract_map·format·mod. Visibility tightened frompub(crate)topub(super)where external visibility wasn't needed. Pure move; every server unit / integration test passes.ffi/src/lib.rssplit into topic modules (#384) — 4054 lines reorganised intotypes/auth/historical/streaming/utility/error/panic. Theffi_boundary!macro moves topanic.rsand is#[macro_use]'d fromlib.rs. ABI byte-for-byte identical:nm -D --defined-onlylists the same 211tdx_*symbols on bothcdylibandstaticlibbefore and after the split. Downstream C / C++ / Go / Node consumers see zero difference.Three largest code generators split by render target (#384) —
build_support/endpoints/sdk_surface.rs(2905 LoC),ticks.rs(2094 LoC), andfpss_events.rs(1551 LoC) broken into concern-separated sub-modules:sdk_surface/{spec,common,python,typescript,go,cpp,mcp,cli}.rsticks/{schema,parser,cli_headers,python_arrow,python_classes,typescript,go}.rsfpss_events/{schema,common,buffered,python,typescript,ffi_rust,ffi_c,go_structs}.rs
A regen byte-identical harness (
crates/thetadatadx/tests/regen_byte_identical.sh) hashes every generated artifact before + after a clean rebuild and fails on any drift. Verified: 450 files, zero diff.50 multi-line
format!(r#"..."#)templates externalised into.tmplfiles (#386) — Rust generators no longer carry embedded Python / TypeScript / Go / C++ source as raw string literals. Templates loaded viainclude_str!and rendered through the existingformat!machinery (named positional args). No new runtime dependency (no tera / handlebars / askama). LoC reductions on the offender files:sdk_surface/cpp.rs-35%,fpss_events/ffi_rust.rs-33%,fpss_events/buffered.rs-38%,ticks/parser.rs-33%..gitattributesextended to pin every.tmpltoeol=lfso Windows checkouts can't leak CRLF intoinclude_str!output. Regen byte-identical harness confirms zero drift across 49 generated artifacts.
7.3.1 - 2026-04-16
Added
- npm pre-built native binaries for Linux x64, macOS arm64, Windows x64 (#335) --
npm install thetadatadxnow works without a Rust toolchain. Platform-specific packages (thetadatadx-linux-x64-gnu,thetadatadx-darwin-arm64,thetadatadx-win32-x64-msvc) are selected automatically viaoptionalDependencies. Unsupported platforms get a clear error message at import time. CI publishes all platform packages via GitHub Actions with OIDC provenance.
7.3.0 - 2026-04-16
Added
- TypeScript/Node.js SDK via napi-rs (#332) -- native addon exposing all 61 historical endpoints, 20+ streaming methods, and 13 tick types to Node.js 18+. Every method, type, and streaming dispatch is SSOT-generated from the same TOML surface that drives Python, Go, and C++. TypeScript type definitions included. CI builds and smoke-tests on every PR. npm publish workflow coming in a follow-up.
Fixed
- FPSS auto-reconnect now re-subscribes all active contracts (#333) -- the
io_loopreconnect path authenticated successfully but never re-sent subscription frames, so data stopped flowing after an involuntary disconnect.active_subsandactive_full_subsare now shared viaArc<Mutex<...>>between the client and the I/O thread; after reconnect login, every active subscription is re-sent before draining the command channel. - Unrecognized FPSS frame codes now emitted as
UnknownFramewith raw bytes -- previously logged at trace level and silently dropped, so users had no visibility into unexpected server frames. Now surfaced asFpssControl::UnknownFrame { code, payload }with hex-encoded wire bytes in the Python and TypeScript SDKs. - Python and TypeScript SDKs explicitly map
Reconnecting,Reconnected, andMarketClosecontrol events -- these previously fell through to the catch-all"unknown_control"label, which was confusing in soak-test logs. - FFI + Go SDK now expose
UnknownFramewith raw payload bytes -- the C FFI bridge mapsUnknownFrameto kind 11 with the hex-encoded payload in the detail field (was kind 99 with no detail). Go SDK adds theFpssCtrlUnknownFrameconstant and a complete control-kind enum for all 11 event types. All four SDKs (Python, TypeScript, Go, C++) now surface unrecognized server frames consistently.
Changed
active_subs/active_full_subspromoted toArc<Mutex<...>>(#333) -- subscription tables are now shared between theFpssClientand theio_loopthread so the reconnect path can read them without a command-channel round-trip. Snapshots are cloned before writing frames to avoid holding the lock during I/O.
7.2.1 - 2026-04-16
Fixed
- Greek and IV decoders regressed by v7.2.0 strict decode -- every Greek endpoint (
option_snapshot_greeks_*,option_history_greeks_*) returnedDecode failed: column N: expected Number, got Priceon live payloads. The v7.2.0 tightening routed everyf64tick column throughrow_float, which accepts onlyNumbercells, but the v3 MDDS server legitimately sends Greeks and implied-volatility values asPrice-encoded cells (matching Java'sPojoMessageUtils.dataValue2ObjectPRICE → BigDecimal arm).f64columns now decode throughrow_price_f64and accept bothPriceandNumbercells. Regression surfaced on live run 24520486541. - Bulk option-chain validator cells timed out at 60 s --
all_strikes_one_expandbulk_chaincells onoption_history_ohlc,option_history_quote,option_history_trade_quote,option_history_greeks_first_order,option_history_greeks_implied_volatility, andoption_at_time_quotelegitimately stream a full-chain payload that does not fit in the 60-second per-cell budget. The CLI / Python / Go / C++ validators now apply a 180-second deadline to bulk-chain / all-strike modes and keep the 60-second baseline for every other cell.
7.2.0 - 2026-04-16
Added
- Per-request deadlines and async cancellation (#298) -- every historical endpoint now accepts
with_timeout_ms(u64)orwith_deadline(Instant)on its builder and a matchingWithTimeoutMs/WithDeadlineoption in the Go SDK, C FFI, Python SDK, and C++ SDK. Underlying implementation routes throughtokio::time::timeouton the gRPC future, so cancellation is cooperative and frees server-side work promptly. Python surfaces a newTimeoutErrorclass distinct fromThetaDataErrorso callers can catch slow endpoints without swallowing other failures. - New
tdbe::error::DecodeErrorenum (#325) -- per-cell decoding errors now carry structured{ column, expected, observed }context instead of a generic string. Folds cleanly intothetadatadx::Error::Decodeat theDirectClientboundary. tdbe::codec::fit::FitRows-- a typed container replacing the previousVec<Vec<i32>>return from the bulk FIT decoder. Exposesrow(i)anditer()for column-major access without per-row heap allocations, materially reducing FPSS decode allocation pressure in sustained streaming.- Live parameter-mode matrix validator (#287, #288, #290, #291) -- every SDK release validator (
scripts/validate_cli.py,scripts/validate_python.py,sdks/go/validate.go,sdks/cpp/examples/validate.cpp) now runs one test per(endpoint, mode)pair instead of one per endpoint. Modes are emitted by the endpoint generator from the wire shape:- List endpoints: one
basicmode. - Stock / index / calendar / rate endpoints: one
concretemode. - Option
ContractSpecendpoints (29 endpoints): six modes each --concrete,concrete_iso,all_strikes_one_exp,all_exps_one_strike,bulk_chain,legacy_zero_wildcard. - Per-optional-parameter coverage: every optional builder parameter gets its own
with_<param>cell, plus a compoundall_optionalscell. Compound pairs likestart_time+end_timecollapse into a singlewith_intraday_windowcell. - Streaming endpoints remain exercised by
scripts/fpss_smoke.py/fpss_soak.py.
- List endpoints: one
- Upstream-derived tier and wildcard maps (#290, #291) -- dropped hand-maintained
endpoint_min_tierandendpoint_supports_expiration_wildcardmatch statements in favor of generator-time lookups against a pinned upstream OpenAPI snapshot. The parser fails closed on three drift classes: missingx-min-subscription, zero-endpoint snapshots, and unknownexpirationvariants. Surfaced and corrected one stale label (option_snapshot_market_valuewasvalue, upstream saysstandard). - Cross-language agreement check (#290, #291) --
scripts/validate_agreement.pyloads per-language validator artifacts atartifacts/validator_<lang>.jsonand asserts every(endpoint, mode)cell present in at least two SDKs agrees onstatusandrow_count.scripts/validate_release.shruns CLI -> Python -> Go -> C++ -> agreement in order. - Structured field-level diff in validator output (#293) -- the release validator now emits per-field diffs instead of opaque equality failures, so drift between SDKs is traceable without re-running.
- Per-cell 60-second timeout on every validator -- every cell is bounded by a hard 60-second timeout with language-specific hygiene (daemon thread + queue on Python,
packaged_task+_Exiton C++, goroutine + timeout-channel + deferred-close gate on Go,subprocess.run(timeout=60)on CLI). - Public API redesign charter (#282) --
docs/public-api-redesign.mdlays out the layered ergonomic facade plan (canonical parity layer, handwrittenhistorical/realtime/analyticsfacades, typed value foundations, compatibility window). The streaming category is namedrealtimeto avoid overloading the meanings oflivein CI and run-mode contexts.
Changed
- SDK surface is now fully declarative TOML (#300) -- every generated method signature, optional-parameter shape, streaming dispatch, FFI wrapper, Python binding, Go function, and C++ method is projected from
sdk_surface.toml,endpoint_surface.toml, andtick_schema.toml. Adding or changing a method is a TOML edit plusgenerate_sdk_surfaces, with no hand-editing of per-language glue. parse_*_ticks,parse_option_contracts_v3,parse_calendar_days_v3now returnResult<Vec<T>, DecodeError>(#325) -- the generated and hand-written row-decoders previously returnedVec<T>and silently coalesced per-cell type mismatches to zero. Mismatches now propagate asDecodeError::TypeMismatch { column, expected, observed }which folds intoError::Decodeat theDirectClientboundary. This is a Rust-caller-visible breaking change for anyone reaching pastDirectClient::*into the free functions; the SDKResult<Vec<T>, Error>shape users actually call is unchanged, so no ABI / FFI / Python / Go / C++ contract moves.Contract::optionnow returnsResult(#324) -- constructing an option contract from user-supplied strings can now surface invalidexpiration/strike/rightinput through?instead of panicking on malformed callers.- FIT decoder exposes
FitRows(tdbe 0.10.0) -- bulk decode returns a dedicated type instead ofVec<Vec<i32>>. Callers who passed the old nested-vec shape into downstream helpers need to switch toFitRows::row()/iter(). Error::Decodedisplay text now reads "Decode failed: ..." (was "Protobuf decode failed: ...") -- the variant now carries both protobuf deserialization errors and post-decode per-cell type-mismatch failures, so the old label was misleading.build_support/endpoints.rssplit into a focused module tree (#294) -- what was one 2700-line file is nowhelpers,model,modes,parser, andrender/{build_out,cli_validate,cpp,direct,ffi,go,python}underbuild_support/endpoints/. Public behavior is unchanged; discovering where a code-gen step lives is now a two-click navigation instead of a search.- Generator templates moved to
include_str!(#296, #301) -- every remainingpush_str(...)emitter inbuild_supportis now aninclude_str!of a.tmplfile underbuild_support/endpoints/render/templates/. Each generated language has its own template directory (cpp/,direct/,ffi/,go/,python/). Editing a generated code shape no longer requires editing a Rust string literal with embedded Rust syntax. - Test-mode fixtures now live in TOML (#295) -- per-mode test-fixture values were previously a Rust match statement; they are now in
sdk_surface.tomlunder[test_modes.<mode>]. The generator reads them and emits identical code. scripts/check_tier_badges.pylive-fetches upstreamopenapiv3.yaml(#280) -- removedscripts/upstream_tiers.jsonand pulls the authoritativex-min-subscriptionmap at check time, with 4 retries + exponential backoff and fail-closed on exhaustion. Eliminates the manual snapshot-refresh drift vector.- Validator tier gating is server-driven -- the four live matrix validators no longer depend on a client-side
VALIDATOR_ACCOUNT_TIERenv var. Every cell is attempted;PermissionDenied/subscriptionerrors from the server classify asSKIP: tier-permissionwith the declared min_tier echoed, and real bugs continue to surface asFAIL. Wildcard-expiration modes (all_exps_one_strike,bulk_chain,legacy_zero_wildcard) are suppressed on the 7 endpoints upstream binds toexpiration_no_star, because the v3 server rejects*for those. - Full-vocabulary wildcard support for option contract parameters (#284) --
validate_expirationaccepts*,YYYYMMDD, andYYYY-MM-DD; newvalidate_strikeaccepts*/0/ empty (wildcard) or a positive decimal.direct::wire_strike_optanddirect::wire_right_optmap wildcard sentinels toNonesoContractSpecleaves the field unset on the proto, matching what the server documents. Live-verified against production across 64 parameter-mode combinations. A full option chain's open interest for QQQ now returns all 10,158 rows in ~1s (a single bulk call), down from a 34-expiration serial loop (~22s). tdbebumped to 0.10.0 -- carries theFitRowsshape change and theDecodeErrorenum (both public-surface breaking under 0.x rules).
Fixed
- FPSS client is now
Sync-safe (#324) -- the internal read/write halves and session state are now properly guarded so sharing anFpssClientacross threads is sound. Previously a latent data race existed on reconnection bookkeeping. Markedunsafe impl Syncwith the exact invariants documented inline. - Python streaming deadlock on shutdown (#324) --
next_event()now releases the GIL before blocking on the ring buffer, and shutdown coordinates with the blocking reader so Ctrl+C interrupts streaming loops cleanly instead of hanging. - Python Ctrl+C interruptibility (#324) -- long-running gRPC calls now release the GIL and cooperate with Python's signal handling, so Ctrl+C returns control to the interpreter without waiting for the server.
- FFI
CStringinterior-NUL swallowing (#303, #324) -- string outputs across the C ABI now surfaceCString::newfailures viatdx_last_errorinstead of silently truncating at the embedded NUL byte. Callers that previously saw empty strings on malformed input now see a diagnosable error. - gRPC
Statusparsing propagates ThetaData error codes (#303) -- the server's numeric error codes are now extracted from theStatustrailers and surfaced by name, so failures likeINVALID_SYMBOLread asINVALID_SYMBOLinstead of the raw integer. - Protobuf
DataValuetype coercion (#303) -- mixedPrice/Numberencoding on OHLC cells is normalized consistently across all endpoints; previously a minority of Greeks rows decoded as zero when the server encoded them differently from the cell type hint. - Go TLS error-channel races on reconnect (#324) -- closing an FPSS TLS connection concurrently with an in-flight read no longer produces a spurious send-on-closed-channel panic on Go. The error channel is now drained with a select-default rather than assuming the receiver is still alive. CGo callbacks are also pinned to the calling OS thread to keep the TLS session's thread-local state consistent.
- Subscription drop on lock poison (#324) -- active FPSS subscriptions used to silently vanish if a panic poisoned the internal state mutex; the subscription tables now recover via
.into_inner()so reconnection still finds the intended subscriptions. - Float → i32 overflow and panic on invalid strike input (#324) -- strike parsing now bounds-checks the implied i32 representation before conversion, returning a structured error instead of panicking on a pathological user input (e.g.
"999999999.99"). - Greeks recomputation avoided on unchanged inputs (#324) -- the Black-Scholes call path memoizes on the common
(spot, strike, vol, rate, t)tuple so the analytics endpoints no longer recompute identical Greeks on back-to-back rows. - FIT decoder allocator thrash (#324) -- the bulk FIT decoder now reuses a single backing buffer through
FitRowsinstead of allocating per-row, cutting sustained streaming allocation rate by roughly an order of magnitude on busy symbols. - Double string allocation on
Contractclone (#324) --Contractnow wraps its symbol inArc<str>so cloning into per-subscription bookkeeping does not copy the byte buffer twice. - JSON serialization moved off the FPSS I/O thread (#324) --
next_eventnow returns typed structs and the serialization step is only paid at the FFI boundary when the caller asks for JSON, keeping the streaming hot path allocation-free. parse_rightno longer panics on unrecognized input (#324) -- the canonical right parser returns a structured error for unknown vocabulary instead of panicking, so a single malformed row can no longer take down the decoder.- Unset
DataValueoneof fails loud in every strict decoder (#326) --parse_option_contracts_v3(expiration, right),parse_calendar_days_v3(date, type, open, close), and the generator-emitted EOD helpers plus contract-id injectedexpiration/rightused to treat aDataValuewhosedata_typeoneof was unset as a legitimate null and coalesce to0. They now returnDecodeError::TypeMismatch { observed: "Unset" }, matchingrow_number/row_date/row_float/row_text/row_number_i64/row_price_f64and the Java terminal's default arm.NullValueis still coalesced (legitimate null); only the wire-anomaly path changes. - Option contract wildcard rejection (#284) -- before this release the SDK had no working path to the server's bulk-chain mode:
*was rejected client-side byvalidate_expiration, and0was rejected server-side. The SDK vocabulary now covers the full cross-product the server accepts. - Validator tier detection drift (#289) -- dropped the static tier gate that classified legitimate server responses as SKIP. The runtime permission fallback still catches drift between docs and the wire (for example,
interest_rate_history_eodbeing labelledfreeon docs but gated higher by the server). - CI unbroken on
main(#299) -- fixed atimeout_msTOML field mismatch and made the Go pin-test CRLF-robust. - FPSS internal visibility tightening --
active_subsandactive_full_subsare nowpub(in crate::fpss)rather thanpub(super), keeping per-contract and full-stream subscription state visible only to thefpssmodule tree. The reconnect-delay tests also now assert against theTOO_MANY_REQUESTS_DELAY_MS/RECONNECT_DELAY_MSconstants instead of hard-coded millisecond literals, so the tests cannot drift from the real protocol values.
Security
- Session token no longer leaks via
Debug(#324) --AuthResponse'ssession_tokenfield is now redacted in itsDebugimpl. Previously atracing::debug!("{auth:?}")would write the bearer token into logs. Credentials were already redacted; this closes the parallel leak on the response side.
Changed
- Generator bloat cleanup (#302) -- stripped roughly 1,500 lines of ceremony, over-abstraction, and redundant tests across
build_support/and the SDK layers. Behavior identical, surface identical, just less to read. fpss/mod.rssplit into focused submodules (#327) -- what was a 2,143-line single file is nowaccumulator,decode,delta,events,io_loop,session, and a slimmod.rsundersrc/fpss/. Each submodule owns one responsibility; public behavior is unchanged.- Per-cell rationale + redundancy audit in tests (#297) -- generated test cells now carry a one-line rationale in the comment, so deleted or merged cells leave an obvious trail for reviewers.
- Consolidated CI workflow cleanup (#323) -- shared the Rust-dep setup across jobs via a reusable composite action (
.github/actions/setup-rust-deps), removed duplicated workflow steps, and narrowedliveto manual dispatch so routine CI stays deterministic. - Python abi3 smoke CI no longer rebuilds the wheel (#304) -- the smoke job now reuses the wheel built earlier in the pipeline, cutting the job's runtime materially.
7.1.0 - 2026-04-14
Removed
- Greeks utilities now take
right: &strinstead ofis_call: bool(#278) --tdbe::greeks::all_greeksandtdbe::greeks::implied_volatilityaccept the same permissive vocabulary as the rest of the SDK ("C"/"P","call"/"put", case-insensitive) via the canonicalparse_right_strict. Panics with a descriptive message on unrecognised input or theboth/*wildcards. The signature change cascades to the Python SDK (right: str), Go SDK (right string), C++ SDK (const std::string& right), C FFI ABI (tdx_all_greeks/tdx_implied_volatilitytakeconst char* right), thetdx greeks/tdx ivCLI subcommands, and the MCPall_greeks/implied_volatilitytool input schemas. The low-level per-Greek primitives (value,delta,theta, ...) continue to take rawbool— they are pure-math helpers not in scope. Motivation: consistency withContract::option,normalize_right, andvalidate_rightso callers stop flipping between"C"strings andtruebools in the same session. tdbebumped to 0.9.0 -- breaking public signature change ingreeks.thetadatadx,thetadatadx-ffi,thetadatadx-cli,thetadatadx-mcp,thetadatadx-server,thetadatadx-py, and the C++ SDK (CMake project) bumped to 7.1.0 -- downstream version bumps to carry the breaking FFI ABI change.
Changed
thetadatadx::rightis now a thin re-export oftdbe::right(#278) -- the canonicalrightparser moved into the pure-datatdbecrate sotdbe::greekscould reuse it withouttdbereverse-depending onthetadatadx. Public API (parse_right/parse_right_strict/ParsedRightwith all four projections) is unchanged at thethetadatadx::rightpath. The error type now returnstdbe::error::Error::Configinstead ofthetadatadx::error::Error::Config; aFrom<tdbe::error::Error> for thetadatadx::Errorconversion is provided so?inthetadatadx-returning functions keeps working.- Top-level re-exports for offline Greeks (#278) --
thetadatadx::{all_greeks, implied_volatility, GreeksResult}now re-export fromtdbe::greeksso SDK consumers can avoid reaching into thetdbecrate directly. Docs preferuse thetadatadx::all_greeks;. - Centralized
rightparsing (#270) -- newthetadatadx::rightmodule exposesparse_right/parse_right_strictreturning aParsedRightenum that carries every downstream representation (MDDS lowercase string, FPSSis_callbool, short-form"C"/"P", FPSS wire byte).normalize_rightindirect.rs,validate_rightinvalidate.rs, andContract::optioninfpss/protocol.rsall route through it. - OpenAPI YAML aligned with upstream ThetaData (#270) --
right-paramenum indocs-site/public/thetadatadx.yamlextended to[call, put, both, C, P, c, p, CALL, PUT, Call, Put, "*"]to match what the server actually accepts (strict superset of upstream's[call, put, both]). Responserightstaystype: stringwith a note documenting the current"C"/"P"output shape.
Fixed
- Silent put-default on invalid
rightinContract::option(#270) -- previouslyContract::option(..., "xyz")silently constructed a put contract because the parser only checked for call forms. Now panics with a descriptive message, consistent with the existing strike/expiration panic style.
Changed
- Every Greeks example in the docs-site, READMEs, Python example, and notebooks updated to pass
right: "C"/right="C"/right: "C"instead ofis_call: true. - Note added to
docs-site/docs/api-reference.mdanddocs/api-reference.mdclarifying that the low-level per-Greek primitives still takeis_call: bool, while the user-facing aggregates takeright: &str. - Corrected 31 subscription-tier badges across
docs-site/docs/historical/**/*.md(#276) -- audit against ThetaData's canonicalopenapiv3.yaml(x-min-subscriptionfield) found 31 of 57 endpoint docs advertised the wrong subscription tier. Fixed against upstream truth. - Renamed misnamed doc file (#276) --
historical/option/at-time/ohlc.mdactually documented theoption_at_time_quoteendpoint; renamed toquote.md, fixed the nav link indocs-site/docs/.vitepress/config.ts, and updated the sole inbound reference inhistorical/option/index.md. - New
scripts/check_tier_badges.py(#276) -- validates every<TierBadge>in the historical docs againstscripts/upstream_tiers.json, a checked-in snapshot of ThetaData's authoritativex-min-subscriptionmap (with_sourceand_captured_atkeys for traceability). Wired intoscripts/check_docs_consistency.pyso the existingExtended SurfacesCI job gates tier drift automatically. No network calls at CI time. - Deleted orphan docs-site pages (#272) -- removed top-level single-page versions (
getting-started.md,historical.md,historical/{stock,option,index-data,calendar}.md,streaming.md,tools/index.md) superseded by the subdirectory navigation. Added a## Client Modelsection todocs-site/docs/streaming/index.mdthat makes the per-SDK split (Rust/Python unifiedThetaDataDxClient, Go/C++ standaloneFpssClient) unmistakable. RemovedignoreDeadLinks: truefromdocs-site/docs/.vitepress/config.tsso future link rot fails the VitePress build. - Sidebar landings for Historical Data and Tools sections (#274) -- added
link:fields on both top-level sidebar entries so clicking the section headers lands on the category overview. Created a newtools/index.mdoverview describing the CLI / MCP / REST Server trio.
7.0.0 - 2026-04-14
Removed
SnapshotTradeTickdeleted from all layers -- removed from Rust core, FFI, Python, Go, and C++ SDKs. Dead type that was never returned by any endpoint.- FFI options use explicit
has_*flags -- replaced NaN/-1sentinel-based optional fields withhas_exclusive,has_max_dte,has_strike_range,has_annual_dividend, etc. C, Go, and C++ consumers must check the companionhas_*i32 flag (0 = unset, 1 = set) before reading the value. generate_sdk_surfacesrestored as the checked-in surface authority -- the standalone codegen binary is required again and is the canonical way to regenerate and verify generated SDK/FFI/tool surfaces from TOML.- Streaming endpoints generated from TOML -- hand-written streaming endpoint blocks in
direct.rsreplaced by TOML-driven codegen. Method signatures unchanged but internal dispatch is generated. - Endpoint, utility, FPSS wrapper, and tick projection surfaces are spec-driven -- Rust, FFI, Python, Go, C++, CLI, and MCP now project their generated public surfaces from
endpoint_surface.toml,sdk_surface.toml, andtick_schema.toml. - Removed the misleading per-contract
subscribe_option_full_*/unsubscribe_option_full_*FPSS methods from the C FFI, Go SDK, and C++ SDK. Per-contract streams usesubscribe_option_*; full-stream subscriptions remainsubscribe_full_*by security type. - Python FPSS option subscription helpers now take
(symbol, expiration, strike, right)to match Rust, Go, and C++ argument order. - Go/C++
contract_mapAPI replaced --ContractMapJSON()/contract_map_json()removed; replaced with typedContractMap()/contract_map()returningmap[int32]string/std::map<int32_t, std::string>. Callers of the old JSON variant will fail to compile.
Removed
public-api-redesign.mdand README reference.migration-from-rest-ws.mdand navigation/index references.- 1,134 lines of commented-out legacy Python methods.
- obsolete claim that
generate_sdk_surfaceshad been removed.
Changed
- Workspace version bumped from 6.0.0 to 7.0.0.
tdbebumped from 0.7.0 to 0.8.0.tdbe@0.7.0was yanked from crates.io because it shipped with a brokenMarketValueTickschema (five stale fundamental fields); the 0.8.0 release carries the correctedmarket_bid/market_ask/market_pricelayout.- Docs consistency checker now points at correct generated files.
FpssControl::LoginSuccess { permissions }documented as opaque diagnostic metadata.- Public endpoint and utility surfaces now project optional request parameters consistently across Rust, Python, Go, C++, CLI, MCP, and REST from the checked-in specs.
- Python now exposes
reconnect()on the unified streaming client, matching the existing Go/C++ FPSS reconnect capability. time_of_dayaccepts both legacy millisecond strings and formatted wall-clock inputs such as9:30,09:30:00, and09:30:00.000, then normalizes to canonicalHH:MM:SS.SSS.- Release validation and live smoke harnesses were added and the GitHub live workflow was narrowed to manual dispatch so routine CI stays deterministic.
Fixed
market_valueendpoints now decodePricecells correctly instead of returning zeroed prices.- Release validation, generated Python/Go validators, and cross-platform CLI validation now use valid fixtures and treat legitimate empty responses correctly.
- C++ tick ABI layout now matches the aligned Rust FFI structs, fixing multi-element array stepping bugs.
- Windows Go FFI builds now use the correct GNU-targeted Rust artifacts when building with CGo on GitHub runners.
- Docs and OpenAPI now reflect the real at-time contract and strike wildcard semantics.
- Docs consistency checker no longer references deleted
migration-from-rest-ws.md. cargo fmtapplied tobuild_support/endpoints.rs.
6.0.1 - 2026-04-06
Removed
- All tick price fields changed from
i32tof64-- prices are decoded during parsing. Users accesstick.bid,tick.price,tick.opendirectly asf64. No moreprice_typeor_f64()helpers. price_typeremoved from all public APIs -- historical ticks, FPSS streaming events, FFI, Python, Go, C++.strike_price_typeremoved --strikeis nowf64on all tick structs.- All
_f64()and_price()helper methods removed --bid_f64(),get_price(),open_price(),trade_price(),midpoint_price(),midpoint_value(),strike_price()no longer exist. - FPSS streaming events: prices are
f64--FpssData::Quote,Trade,Ohlcvcexposef64fields directly. Noprice_type. No_f64dual fields. Contract::option()takes 4 strings --Contract::option("SPY", "20260417", "550", "C")instead of(root, i32, bool, i32). Matches the MDDS historical API experience.- Python SDK:
subscribe_option_*takes(symbol, exp_date, right, strike)as strings. Removedprice_raw,bid_raw,price_typefrom dicts. - Go SDK: removed
RightRaw,StrikePriceType,PriceRaw,BidRaw/AskRaw/OpenRaw/etc.,PriceToF64(). - C++ SDK: all price fields are
double. Removedtdx::price_to_f64(),tdx::bid_f64(),tdx::open_f64(), etc. - CLI:
price_typecolumn removed from all table output.
Added
QuoteTick.midpoint-- pre-computed(bid + ask) / 2.0at parse time.Contract::option_raw()-- raw wire-format constructor for the drop-in REST/WS server.- Go FFI layout tests -- compile-time
unsafe.Sizeofassertions for all 12 C-mirror structs. - WebSocket zero-copy fan-out -- per-client
mpsc<Arc<str>>, JSON serialized once. - Server
--no-ohlcvcflag -- disable OHLCVC bar derivation from trades. - CLI price formatting -- preserves up to 6 meaningful decimals, trims trailing zeros.
Fixed
tools/serverandtools/mcpcompilation -- updated for f64 migration (were excluded from workspace, broke silently).- Go FFI struct padding -- 8 structs had incorrect tail padding causing memory corruption on multi-element arrays.
OptionContractmissingDebug + Clonederives -- accidentally removed during refactor.- Server dead match arm -- removed v2 parameter fallback code.
Changed
- All 60+ endpoint pages updated: f64 fields, no
price_type, no_f64()helpers. - All SDK READMEs updated (Rust, Python, Go, C++).
- Streaming docs rewritten for f64 events.
- OpenAPI spec purged of
price_type. - JVM deviations doc: new sections for FPSS f64 streaming and
Contract::optionclean API. - Internal docs (architecture, api-reference, endpoint-schema) updated.
- README now explicitly warns that FPSS is not yet production-ready due to the upstream framing issue tracked in
#192.
5.4.0 - 2026-04-05
Removed
start_streaming_no_ohlcvc()removed -- useDirectConfig::derive_ohlcvc(false)instead. (#129)- Go SDK:
SnapshotTradeTicktype removed (was dead code after FFI cleanup).
Added
DirectConfig::derive_ohlcvc(bool)-- config-driven OHLCVC opt-out, replaces duplicate method. (#129)- REST server drop-in replacement --
--email/--password,--config,--fpss-regionCLI args./v3/system/statusendpoint. Startup banner. (#128) - Error suppression 5s after STOP -- matches Java terminal behavior. (#124)
- Auth retry on transient errors -- 3 attempts, 2s delay, network errors only. (#125)
- Config validation -- clamps queue_depth (16-1M), window_size (64-1024) with warnings. (#126)
- Password character warning -- on INVALID_CREDENTIALS disconnect. (#127)
- Clippy pedantic zero warnings --
#[must_use], inlined format args, numeric separators,try_fromcasts, error docs. No blanket suppression. (#131)
Fixed
- Zero
#[allow(dead_code)]in entire project. - Go SDK dangling extern for removed
TdxSnapshotTradeTickArray. - Doc comment typo
100_0000->1_000_000. - Test warning on unused
#[must_use]return. - All
#[allow]annotations have reason comments.
5.3.1 - 2026-04-04
Added
- FPSS auto-reconnect with configurable policy:
Auto(default, matches Java terminal),Manual,Custom(fn). New control events:Reconnecting,Reconnected. (#119) - Trade/quote condition descriptions with special-case annotations (e.g.,
*update last if only trade).
Fixed
- Greeks returned all zeros on intraday endpoints (
greeks_first_order,greeks_iv, etc.). The v3 server sends Greeks as Price-encoded cells;row_float()now decodes them. (#118) expiration=0on wildcard EOD -- contract ID extraction now handles ISO date text ("2024-01-31" -> 20240131). (#117)implied_volatility->implied_volheader alias added for v3 server column name.- Raw strike encoding in docs -- replaced "500000" with "500" (dollar amounts) across 37 files.
"EOD"removed from docs -- v3 uses"TRADE"/"QUOTE"only.- Options examples rewritten to use wildcard bulk queries instead of per-strike loops.
5.3.0 - 2026-04-04
Removed
- Go SDK:
EodTick,OhlcTick,TradeTick,QuoteTick,TradeQuoteTick,PriceTick,SnapshotTradeTickgain additional fields (raw prices, ext_conditions, price_type).Rightis nowstring("C"/"P") withRightRaw int32for raw access. - Python SDK: trade dicts gain
ext_condition1..4. Quote/OHLC/EOD/TradeQuote dicts gain raw price and detail fields. - Rust:
normalize_right()maps"C"->"call","P"->"put","*"->"both"for v3 server.
Added
tdbe::exchange-- 78 exchange codes with O(1) lookup:exchange_name(),exchange_symbol(). (#112)tdbe::conditions-- 149 trade conditions + 75 quote conditions with semantic flags (cancel, volume, high, low, last). (#112)tdbe::sequences-- FPSS sequence tracking with wrapping-aware gap detection. (#112)tdbe::error-- 14 ThetaData HTTP error codes mapped to human-readable names. gRPC errors now include the ThetaData error name. (#113)- OHLC price normalization --
row_price_value_normalized()andchange_price_type()handle mixed price_types across OHLC fields. (#106) - Greeks from Price cells --
row_float()decodes Price-typed cells.implied_volheader alias. (#106) - Calendar v3 parser -- handles text dates, text times, and type codes from v3 server. (#109)
normalize_right()-- maps C/P/* to call/put/both for v3 server. GoRightStr()helper. (#111)- Full SDK parity -- Python and Go SDKs now expose every field from every Rust tick type.
- Latency physics documentation -- speed-of-light calculations, colocation guidance, Mermaid diagrams.
Fixed
- 37% of OHLC intraday bars had wrong prices -- mixed price_type per cell caused 10x errors. (#106)
- All Greeks returned 0.0 -- server sends Greeks as Price cells, not Number cells. (#106)
option_list_contractsreturned 0 -- v3 server uses "symbol" not "root", ISO dates, text right. (#97)- Calendar endpoints returned zeros -- v3 text format mismatch. (#109)
- Dev server FPSS crashes -- binary Error frames and unknown codes handled gracefully. (#85)
PriceToF64Go formula wrong -- wasvalue / 10^pt, corrected tovalue * 10^(pt-10).- Python
greeks_tick_to_dictmissing 15 fields -- now has all 24.
Changed
- 14 documentation fixes across 13 files
- Mermaid diagrams replacing ASCII art in VitePress docs
- Latency physics section with speed-of-light calculations per geography
- 3 new JVM deviations documented
- v3 migration guide compliance verified
5.2.1 - 2026-04-04
Fixed
option_list_contractsreturned 0 contracts. The v3 MDDS server sendssymbol(notroot), ISO date strings (not YYYYMMDD integers), andPUT/CALLtext (not integer codes). Addedroot->symbolheader alias and a v3-aware parser. (#97)- Dev server FPSS replay boundary corruption handled gracefully. Binary Error frames are silently skipped. Unknown message codes are skipped with bounded retry (5 consecutive = framing corruption -> clean disconnect). (#85)
5.2.0 - 2026-04-04
Removed
- Go SDK: price fields on public structs are now
float64(decoded). Rawint32values available as*Rawfields.PriceTyperemoved from public structs. - Go FPSS events:
FpssQuote.Bid/Ask,FpssTrade.Price,FpssOhlcvc.Open/High/Low/Closeare nowfloat64. Raw values as*Rawfields. - Rust FPSS events:
FpssData::Quote,Trade,Ohlcvcgain pre-decoded*_f64fields (bid_f64,price_f64, etc.).
Added
- Rust
_f64()convenience methods on all tick types:price_f64(),bid_f64(),ask_f64(),open_f64(),high_f64(),low_f64(),close_f64(),midpoint_f64(). (#95) - Go pre-decoded f64 prices on all public structs and FPSS events. Users get
tick.Priceasfloat64ready to use. (#95) - C++
tdx::price helpers -- 17 inline functions for f64 price decoding on all tick types. - FFI FPSS events gain
*_f64fields (bid_f64,ask_f64,price_f64,open_f64,high_f64,low_f64,close_f64) pre-computed during event construction.
Fixed
- Go
PriceToF64formula wasvalue / 10^ptinstead ofvalue * 10^(pt-10). All FPSS streaming prices would have been wrong. (#95)
5.1.1 - 2026-04-03
Fixed
tdbedependency bumped to 0.2.0 for crates.io publish (0.1.x was yanked). No code changes.
5.1.0 - 2026-04-03
Removed
- FPSS FFI events now use
#[repr(C)]typed structs instead of JSON serialization.tdx_fpss_next_eventandtdx_unified_next_eventreturn*mut TdxFpssEvent(a flat tagged struct with quote, trade, open interest, OHLCVC, control, and raw_data variants). Free withtdx_fpss_event_free. (#82) - C++ SDK:
FpssClient::next_event()returnsFpssEventPtr(RAII unique_ptr toTdxFpssEvent). - Go SDK:
FpssClient.NextEvent()returns*FpssEventwith typed Go structs. - Streaming event prices are now raw integers with
price_type(matching the wire format). Callers decode withPrice::new(value, price_type).to_f64()ortdx::price_to_f64(value, price_type). serde_jsonremoved from FFI crate dependencies -- zero JSON crosses the FFI boundary.
Added
- Contract identification on all 10 option tick types --
expiration,strike,right,strike_price_typefields populated by the server on wildcard queries. Helper methodsstrike_price(),is_call(),is_put(),has_contract_id()on all 10 tick types viaimpl_contract_id!macro. (#84) - 8-field trade tick support -- FPSS dev server sends abbreviated 8-field trade ticks; production sends 16-field.
decode_tick()now auto-detects the field count from the first absolute tick per contract and dispatches to the correct index mapping. (#86) #[repr(C)]FPSS event structs in all SDKs --TdxFpssQuote,TdxFpssTrade,TdxFpssOpenInterest,TdxFpssOhlcvc,TdxFpssControl,TdxFpssRawDatawith taggedTdxFpssEventwrapper. (#82)FfiBufferedEventwith owned backing storage for safe cross-threadSendof pointer-containing structs.- Go SDK:
FpssQuote,FpssTrade,FpssOpenInterestData,FpssOhlcvc,FpssControlDataGo structs mirroring Rust#[repr(C)]layout. - C++ SDK:
FpssClientclass with RAIIFpssEventPtrfor streaming. - Python SDK:
greeks_tick_to_dictnow emits all 24 fields (was 8). (#92) tdbe: contract ID fields andimpl_contract_id!macro on all 10 tick types.
Fixed
- 9 stale JSON references in FFI doc comments, FFI README, Go README, docs-site API reference, and macro guide -- all now correctly describe typed structs. (#92)
- Python SDK
greeks_tick_to_dictmissing 16 fields (vanna, charm, vomma, veta, speed, zomma, color, ultima, d1, d2, dual_delta, dual_gamma, epsilon, lambda, vera, date). (#92) - Go SDK README documented
ActiveSubscriptions()return type asjson.RawMessage-- actually returns[]Subscription. (#92) - docs-site Go streaming example said "returns json.RawMessage or nil" -- now says "*FpssEvent or nil".
5.0.2 - 2026-04-03
Fixed
- OHLCVC accumulator
volumeandcountfields widened fromi32toi64to prevent integer overflow on high-volume symbols during dev server replay. (#80)
5.0.1 - 2026-04-03
Fixed
FpssClient::connect()now usesDirectConfig::fpss_hostsinstead of hardcoded production servers.dev()andstage()configs now correctly connect to their respective FPSS servers. (#77)- Removed dead
SERVERSconstant fromprotocol.rs
5.0.0 - 2026-04-02
Removed
- Builder pattern on all 61 endpoints -- methods return builders with
IntoFuture.start_time/end_timeare now builder methods, not positional params. All optional proto params exposed as chainable setters. received_at_ns: u64added to everyFpssDatavariant (Quote, Trade, OpenInterest, Ohlcvc)DirectConfig::dev()now uses actual ThetaData dev FPSS servers (port 20200, infinite replay) instead of production with reduced buffers
Added
- Builder pattern -- all endpoints return chainable builders. Zero noise for simple calls, all optional proto params discoverable via autocomplete.
received_at_ns-- nanosecond receive timestamp on every FPSS event for latency measurementtdbe::latency::latency_ns()-- DST-aware wire-to-application latency computationFpssFlushMode--Batched(default, matches Java) orImmediate(lowest latency)- Metrics --
metricscrate integration. Counters/histograms on all gRPC, FPSS, and auth operations. Zero overhead when no backend installed. - Config file --
DirectConfig::from_file()behindconfig-filefeature flag. TOML format matching v3 terminal. DirectConfig::stage()-- staging FPSS servers (port 20100)- 3 FPSS methods in all SDKs --
subscribe_full_open_interest,unsubscribe_full_trades,unsubscribe_full_open_interest - Cross-platform CI -- Format, Lint, Test, FFI Build on Ubuntu + macOS + Windows
- Macro guide --
docs/macro-guide.mdfor contributors - DST pre-2007 safety net -- handles old US DST rules (April-October) for pre-2007 dates
unsubscribe_option_open_interestin Python SDK (was missing)- Go
FpssClient-- complete standalone streaming client wrapper (sdks/go/fpss.go)
Fixed
- 30 documentation findings from production audit (version pins, method tables, CHANGELOG, SECURITY)
- 14 public methods missing doc comments on
ThetaDataDxClient - Python SDK
lock().unwrap()changed to poison recovery - Legacy
config.default.propertiesremoved (v2 artifact)
4.5.0 - 2026-04-02
Removed
- FFI:
#[repr(C)]typed struct arrays replace JSON -- all 60 data endpoints now return native struct arrays across the FFI boundary. C++ and Go SDKs read fields directly, zero JSON serialization. FPSS streaming events remain JSON (variable schemas). - C++
OptionContractnow usesstd::string root(wasconst char*) - Go SDK gains 9 previously missing Greeks endpoints
Added
- DST-aware timezone conversion --
eastern_offset_ms()correctly handles EST/EDT transitions using US Energy Policy Act 2005 rules. Historical data from November-March now has correct ms_of_day values. (#32) - gRPC flow control config --
DirectConfiggainedmdds_window_size_kbandmdds_connection_window_size_kb, wired into tonic channel builder. (#36) - Go SDK:
OptionSnapshotGreeksFirstOrder,OptionSnapshotGreeksSecondOrder,OptionSnapshotGreeksThirdOrder,OptionHistoryGreeksFirstOrder/SecondOrder/ThirdOrder,OptionHistoryTradeGreeksFirstOrder/SecondOrder/ThirdOrder(#39) - Go SDK:
SnapshotTradeTicktype and converter - Go SDK:
Verafield onGreeksTick - FFI: 13 typed tick array types (
TdxEodTickArray,TdxOhlcTickArray, etc.) withfrom_vec/free - FFI:
TdxStringArrayfor list endpoints,TdxOptionContractArrayfor contracts - C++ header:
thetadx.hwith all#[repr(C)]struct definitions and function signatures
Fixed
- Timezone hardcoded UTC-4 -- was producing ms_of_day shifted +1 hour for all Nov-Mar historical data. Now DST-aware with 5 unit tests. (#32)
- EOD parser divergent alias system -- unified to shared
find_header(). (#34) - reconnect_wait_ms -- changed from 1000 to 2000 to match Java terminal. (#35)
- C++ OptionContract use-after-free -- root string was dangling after array free. Now deep-copies to
std::string. (#39) - Active subscriptions not cleared on explicit shutdown --
shutdown()clears, involuntary disconnect preserves for reconnect. (#38) - Mermaid diagram syntax in architecture.md (#30)
Changed
- Price type per-row variation as known limitation in jvm-deviations.md (#37)
- FPSS ring buffer capacity monitoring as known limitation
4.4.0 - 2026-04-02
v3 MDDS DataTable parsing (Timestamp cells), DST-aware timezone, gRPC flow control, header aliases for EOD. See v4.5.0 for cumulative details.
4.3.0 - 2026-04-02
Added
start_timeandend_timeparameters exposed on all 25 endpoints that support time filtering. PassSome("04:00:00")for pre-market,Some("20:00:00")for extended hours, orNonefor RTH defaults (09:30:00-16:00:00). Affects stock history/snapshot/at-time, option history, and index history endpoints.
Fixed
- Version pins in README and getting-started docs updated to
"4.2" - Default venue
"nqb"(NASDAQ Best) documented in jvm-deviations.md
4.2.0 - 2026-04-01
Fixed
- Interval conversion: MDDS server accepts preset shorthand (
1m,5m,1h), not raw milliseconds.normalize_interval()now converts"60000"->"1m","300000"->"5m", etc. Sub-second presets supported:"100"->"100ms","500"->"500ms". Users can pass either milliseconds or shorthand directly. - Default start_time/end_time: the Java terminal defaults these to
"09:30:00"and"16:00:00". Our SDK left them as None, causing"Invalid time format: Expected hh:mm:ss.SSS"on trade/quote/greeks endpoints. Now defaults to RTH. - extract_text_column: now handles Number and Price DataTable values.
option_list_strikeswas returning 0 results because strikes come as Number values, not Text. - FPSS TLS certificate: ThetaData's FPSS servers have certificates expired since Jan 2024. Skip certificate verification for FPSS connections (matching Java terminal behavior).
Added
100ms, 500ms, 1s, 5s, 10s, 15s, 30s, 1m, 5m, 10m, 15m, 30m, 1h
4.1.2 - 2026-04-01
Interval format conversion (later superseded by shorthand normalization in v4.2.0).
4.1.1 - 2026-04-01
Fixed
- PyPI publish workflow: add
skip-existing: trueto prevent duplicate upload failures on tag re-push
4.1.0 - 2026-04-01
Added
subscribe_full_open_interest(sec_type)-- full-stream open interest subscription (was missing, Java terminal has it)unsubscribe_full_trades(sec_type)-- full-stream trade unsubscribe (was missing)unsubscribe_full_open_interest(sec_type)-- full-stream OI unsubscribe (was missing)reconnect_streaming(handler)onThetaDataDxClient-- saves active subscriptions, stops streaming, restarts with new handler, re-subscribes all per-contract and full-type subscriptions automaticallyactive_full_subscriptions()accessor for full-type subscription trackingdocs/java-class-mapping.md-- complete enumeration of all 588 Java terminal classes with Rust equivalents or justification for exclusion
Fixed
- DNS hostname resolution in FPSS connection --
SocketAddr::parse()replaced withToSocketAddrsto resolve hostnames likenj-a.thetadata.us(was silently failing)
Changed
- Greeks operator precedence (veta, speed, zomma, color, dual_gamma) -- Java decompiler may have lost parenthesization, Rust follows textbook Black-Scholes formulas
- FPSS ring buffer capacity monitoring -- documented as known limitation (disruptor-rs v4 has no fill-level API)
4.0.0 - 2026-04-01
Removed
tdbecrate extracted -- all data types, codecs, greeks, price, enums, and flags moved to standalonetdbecrate with zero networking dependencies. Users must addtdbeas a dependency and change imports:use tdbe::{Price, TradeTick, EodTick}.thetadatadxno longer exportstypes/,codec/,greeks.rs. These modules live intdbe.
Added
tdbecrate (crates/tdbe/) -- pure data-format crate. Single dependency (thiserror). Contains:- 14 hand-written tick structs (no build.rs codegen)
- FIT/FIE nibble codecs
- Price fixed-point encoding
- 22 Black-Scholes Greeks + IV solver
- All enums (SecType, DataType, StreamMsgType, etc.)
- Error types (Decode, Encode, Conversion, Io)
- Flags module (trade conditions, price flags, volume types)
- 6 criterion benchmarks
- Interactive Query Builder on docs site -- 13 real-world recipes (GEX, vol surface, option chains, live trade tape, etc.) with symbol autocomplete, dynamic dates, and copy-paste code generation for Rust and Python
- Inline credential construction -- all SDK examples now show both
from_file("creds.txt")andCredentials::new("email", "password")patterns - serde_json vs sonic_rs benchmark (
bench_json) -- criterion benchmark covering FPSS events, REST responses, DataTable serialization, and JSON parsing
Fixed
- Query builder syntax highlighter regex cross-contamination (visible
class="hl-string"in rendered code)
Changed
- Tick types in
tdbeare hand-written (noinclude!(), notick_schema.tomlcodegen). IDE-navigable, visible in source. - Magic numbers in
TradeTickimpl replaced withtdbe::flags::named constants - Documentation updated across 17+ files for new import paths
3.2.2 - 2026-03-30
Fixed
- Cleaned git history and consolidated documentation commits.
- Added contributor workflow documentation (conventional commits, pre-commit checks).
3.2.0 - 2026-03-30
Added
- Fully typed returns for all 61 endpoints - 9 new tick types (
TradeQuoteTick,OpenInterestTick,MarketValueTick,GreeksTick,IvTick,PriceTick,CalendarDay,InterestRateTick,OptionContract). All 31 endpoints that returned rawproto::DataTablenow return typedVec<T>. Theraw_endpoint!macro has been removed entirely. Zero raw protobuf in the public API. - TOML-driven codegen -
tick_schema.tomlis the single source of truth for all tick type definitions and DataTable column schemas.build.rsgenerates Rust structs and parsers at compile time. Adding a new column = one line in the TOML. - Proto maintenance guide (
proto/MAINTENANCE.md) - step-by-step instructions for ThetaData engineers to add columns, RPCs, or replace proto files. - 10 new parse functions in
decode.rs(includingparse_eod_ticksmoved from inline indirect.rs) - All downstream consumers updated: FFI (9 new JSON converters), CLI (9 new renderers), Server (9 new sonic_rs serializers), MCP (9 new serializers), Python SDK (9 new dict converters)
- Crate README (
crates/thetadatadx/README.md) and FFI README (ffi/README.md) - Python SDK: polars support documented (
pip install thetadatadx[polars])
Fixed
- Comprehensive documentation sweep - every doc page, README, notebook, and example file audited against the actual source code. Fixed fabricated homepage examples, wrong C++ include paths (
thetadatadx.hpp->thetadx.hpp), staleclient.variable names, missing typed return annotations, wrong Pythonall_greeks()parameter name, version pins (3.0->3.1),for_each_chunksignature in API reference, and incorrect license in footer. - Parameter/response display redesign - replaced flat markdown tables with vertical card layout across 60 endpoint documentation pages.
- Root README streamlined with navigation table (removed 90-line endpoint listing)
- Notebook 105: fixed event kinds and removed raw payload access pattern
- OpenAPI yaml: fixed license, GitHub URLs, removed DataTable response types
3.1.0 - 2026-03-27
Fixed
- Go SDK: price encoding was fundamentally wrong -
priceToFloat()used a switch-case instead ofvalue * 10^(price_type - 10). Every price returned by the Go SDK was incorrect. Now matches Rust exactly. - Python docs: streaming examples used wrong event key - streaming-event dict access changed from the legacy
typekey to the canonicalkindkey across README and all docs-site pages. Price::new()no longer panics in release -assert!replaced withdebug_assert!+clamp(0, 19)withtracing::warn!. A corrupt frame no longer crashes production.- C++
FpssClient: added missingunsubscribe_quotes()- was present in FFI but missing from C++ RAII wrapper. - FFI FPSS: mutex poison safety - all 12
.lock().unwrap()calls replaced with.unwrap_or_else(|e| e.into_inner()). Prevents undefined behavior (panic acrossextern "C") on mutex poisoning. Credentials.passwordvisibility - changed frompubtopub(crate)withpassword()accessor. Prevents accidental credential logging by downstream code.- WebSocket server: added OPEN_INTEREST + FULL_TRADES dispatch - previously silently dropped.
- C++ SDK type parity -
MarketValueTickexpanded from 3 to 7 fields,CalendarDayaddedstatus,InterestRateTickaddedms_of_day. - Python README: removed ghost methods -
is_authenticated()andserver_addr()were listed but did not exist. - Root README: stock method count - "Stock (13)" corrected to "Stock (14)".
3.0.0 - 2026-03-27
Removed
- Unified
ThetaDataDxClientclient — single entry point replacingDirectClient+FpssClient. Connect once, auth once. Historical available immediately, streaming connects lazily. DirectClientremoved from crate root re-exports — still accessible asthetadatadx::direct::DirectClientbut all methods available viaThetaDataDxClient(Deref)FpssClientremoved from crate root re-exports — usetdx.start_streaming(handler)instead- Python SDK:
DirectClientandFpssClientclasses removed. UseThetaDataDxClientonly.
Added
ThetaDataDxClient::connect(creds, config)— one auth, gRPC channel ready, no FPSS yettdx.start_streaming(handler)— lazy FPSS connection on demand (readsderive_ohlcvcfrom config)tdx.stop_streaming()— clean shutdown of streaming, historical stays alivetdx.is_streaming()— check if FPSS is active- All 61 historical methods via
Deref<Target = DirectClient> - All streaming methods (subscribe/unsubscribe) directly on
ThetaDataDxClient - FFI:
tdx_unified_connect(),tdx_unified_start_streaming(),tdx_unified_stop_streaming() - Server: graceful
stop_streaming()on shutdown
Fixed
- Server shutdown now calls
stop_streaming()before notifying waiters - Python SDK: removed duplicate method definitions (DirectClient + ThetaDataDxClient had same methods)
2.0.0 - 2026-03-27
Added
tdxCLI (tools/cli/) — command-line tool with all 61 endpoints + Greeks + IV. Dynamically generated from endpoint registry.cargo install thetadatadx-cli- MCP Server (
tools/mcp/) — Model Context Protocol server giving LLMs instant access to 64 tools (61 endpoints + ping + greeks + IV) over JSON-RPC stdio. Works with Cursor and every other MCP-compatible client. - REST+WS Server (
tools/server/) — drop-in replacement for the Java terminal. v3 API on port 25503, WebSocket on 25520 with real FPSS bridge. sonic-rs JSON. - VitePress documentation site (
docs-site/) — 33 pages covering API reference, guides, SDK docs, wire protocol internals. Deployed to GitHub Pages.
Removed
- FpssEvent split —
FpssEvent::Quote { .. }is nowFpssEvent::Data(FpssData::Quote { .. }). Control events areFpssEvent::Control(FpssControl::*). Migration: wrap your match arms. - OHLCVC derivation opt-in/out —
connect()still derives OHLCVC (default). SetDirectConfig::derive_ohlcvctofalseto disable for lower overhead on full trade streams. - FpssClient is fully sync — no tokio in the streaming path. LMAX Disruptor ring buffer. Callback API:
FnMut(&FpssEvent).
Added
- Endpoint registry — auto-generated from proto at build time. Single source of truth consumed by CLI, MCP, server. 61 endpoints.
- Repo reorganization —
tools/cli/,tools/mcp/,tools/server/(wascrates/*) - sonic-rs — SIMD-accelerated JSON in CLI, MCP, and server (replaces serde_json)
- Zero-alloc FPSS hot path — reusable frame buffer, tuple return (no Vec per frame), pre-allocated decode buffer, wrapping_add for delta parity
- Full SDK parity — all FPSS methods (subscribe_full_trades, contract_lookup, active_subscriptions, etc.) exposed in Python, Go, C++, FFI
- Full trade stream docs — explains the server's quote+trade+OHLC bundle behavior
- v3 REST API — server routes match ThetaData's OpenAPI v3 spec (was v2)
- 43 benchmarks — 10 per-module bench files covering every hot path
Fixed
- SIMD FIT removed — was 2.2x slower than scalar (regression). Pure scalar now.
- Server trade_greeks routes — 5 option history trade_greeks endpoints were silently dropped due to subcategory mismatch in path generation
- Audit findings (hot-path) — hot-path allocations, wrapping_add, BufWriter, find_header fallback, DATE marker handling, MCP sanitization, Price dedup
- Audit findings (server/CLI) — server security (CORS, shutdown auth), CLI expect(), MCP JSON-RPC validation, stale docs
- Auth response parsing — subscription fields are integers not strings
Changed
- FPSS frame read: zero-alloc (reusable buffer)
- FPSS decode: zero-alloc (tuple return, pre-allocated tick buffer)
- Delta: wrapping_add (matches Java, no branch)
- Required column validation (skip rows on missing headers, no garbage parse)
- 43 criterion benchmarks across all modules
1.2.2 - 2026-03-26
Added
- Polars support in Python SDK:
pip install thetadatadx[polars] to_polars(ticks)function converts tick dicts directly to polars DataFrame viapolars.from_dicts()- Optional dependency groups:
[pandas],[polars],[all]for both
Fixed
- Multi-platform Python wheels — now builds for Linux, macOS, and Windows (was Linux-only)
- Source distribution (sdist) included for pip build-from-source fallback
- Auth response parsing: subscription fields are integers (0-3), not strings — fixes connection failures
1.2.1 - 2026-03-26
Fixed
- Auth: subscription fields are integers — Nexus API returns
"stockSubscription": 0(int), not strings. Fixes"failed to parse Nexus API response"error on connect. - Multi-platform Python wheels — CI now builds for Linux + macOS + Windows (was Linux x86_64 only). Fixes
"no matching distribution found"for macOS/Windows users. - Source distribution — sdist included so
pip installcan build from source when no pre-built wheel matches. - Removed hallucinated "row deduplication" from docs (was never implemented, would have dropped real trades).
1.2.0 - 2026-03-26
Added
- OHLCVC-from-trade derivation —
OhlcvcAccumulatorderives OHLCVC bars from trade ticks in real time. Only emitsFpssEvent::Data(FpssData::Ohlcvc { .. })after a server-seeded initial bar, matching the Java terminal's behavior. Subsequent trades update open/high/low/close/volume/count incrementally. - FpssEvent split:
FpssData+FpssControl— the monolithicFpssEventenum is now a 3-variant wrapper:Data(FpssData)for market data (Quote, Trade, OpenInterest, Ohlcvc),Control(FpssControl)for lifecycle events (LoginSuccess, Disconnected, MarketOpen, etc.), andRawDatafor unparsed frames. This enablesmatcharms that handle all data without touching control flow, and vice versa — an intentional improvement not present in Java. - Streaming
_streamendpoint variants —stock_history_trade_stream,stock_history_quote_stream,option_history_trade_stream,option_history_quote_streamprocess gRPC response chunks via callback without materializing the full response in memory. Ideal for endpoints returning millions of rows. - Slab-recycled zstd decompressor — thread-local
(Decompressor, Vec<u8>)pair reuses the working buffer across calls. The internal slab retains its capacity, avoiding allocator pressure for repeated decompressions of similar-sized payloads. - 148 tests — new tests for OHLCVC accumulator, FpssEvent split, and streaming endpoints.
Fixed
18 correctness and protocol-conformance fixes from a full audit against the Java terminal:
FPSS Protocol
- FPSS contract ID is FIT-decoded — CONTRACT message contract IDs are now FIT-decoded (matching the Java terminal), not read as raw big-endian i32. Previously produced wrong contract-to-symbol mappings.
- Delta off-by-one fixed —
apply_deltasfield indexing corrected; previous implementation could shift all fields by one position, corrupting tick data. - Delta state cleared on START/STOP — per-contract delta accumulators are now reset when the server sends START (market open) or STOP (market close), matching Java behavior. Previously, stale deltas from the previous session leaked into the next session's ticks.
- ROW_SEP unconditional reset — ROW_SEP (0xC) now unconditionally resets the field index to SPACING (5), matching the Java FIT reader. Previously this was conditional, which could produce misaligned fields.
- Credential sign-extension — credential length fields are now read as unsigned, matching Java's
readUnsignedShort(). Previously, passwords longer than 127 bytes could produce a negative length. - Flush only on PING — the FPSS write buffer is now flushed only when sending PING messages, matching Java's batching behavior. Previously, every write triggered a flush, increasing syscall overhead and wire chattiness.
- Ping 2000ms initial delay — the first PING is now delayed by 2000ms after authentication, matching the Java terminal's
Thread.sleep(2000)before entering the ping loop. Previously, pings started immediately.
MDDS / gRPC Protocol
null_valueadded to DataValue proto — theDataValueoneof now includes anull_valuevariant (bool), matching the server's proto definition. Previously, null cells were silently dropped during deserialization."client": "terminal"in query_parameters — all gRPC requests now include"client": "terminal"in thequery_parametersmap, matching the Java terminal. Previously this field was omitted.- Dynamic concurrency from subscription tier —
mdds_concurrent_requestsis now derived from theAuthUserresponse's subscription tier (2^tier), matching the Java terminal's concurrency model. The config field still allows manual override. - Unknown compression returns error —
decompress_responsenow returnsError::Decompressfor unrecognized compression algorithms instead of silently treating the data as uncompressed. - Empty stream returns empty DataTable —
collect_streamnow returns an emptyDataTable(with headers, zero rows) when the gRPC stream contains no data chunks, instead of returningError::NoData. Callers can check.data_table.is_empty(). - gRPC flow control window — the gRPC channel now configures
initial_connection_window_sizeandinitial_stream_window_sizeto match the Java terminal's Netty settings, preventing throughput bottlenecks on large responses.
Auth / User Model
- Per-asset subscription fields in AuthUser —
AuthUsernow includesstock_tier,option_tier,index_tier, andfutures_tierfields from the Nexus auth response, enabling per-asset-class concurrency and permission checks. - Auth 401/404 handling — Nexus HTTP responses with status 401 (Unauthorized) or 404 (Not Found) are now treated as invalid credentials, matching the Java terminal's behavior. Previously these could surface as generic HTTP errors.
Observability
- Column lookup warns instead of silent fallback —
extract_*_columnfunctions now emit awarn!log when a requested column header is not found in the DataTable, instead of silently returning a vec ofNones. This makes schema mismatches immediately visible in logs.
Greeks
- 6 Greeks formula fixes — operator precedence corrections across 6 Greek functions to match Java's evaluation order. All formulas now produce bit-identical results to the Java terminal for the same inputs.
VeraDataType code (166) — second-order GreekVeraadded to theDataTypeenum, completing the full set of second-order Greeks (vanna, charm, vomma, veta, vera, sopdk).
Security
- Contract wire format fix — contract binary serialization now matches the Java terminal exactly. Previous versions could produce incorrect wire bytes for option contracts, causing subscription failures or wrong contract assignments. This was a protocol-level bug; upgrading to 1.2.x is strongly recommended.
Changed
- Slab-recycled zstd — thread-local decompressor reuses its working buffer, eliminating per-chunk allocation overhead.
- Streaming
_streamendpoints — process gRPC responses chunk-by-chunk without materializing the full DataTable in memory.
See TODO.md (as of the 1.2.0 release) for the production readiness checklist and performance roadmap.
1.1.1 - 2026-03-26
Added
mdds_concurrent_requestssemaphore on DirectClient — configurable limit on in-flight gRPC requests (default 2), exposed viaDirectConfig.mdds_concurrent_requests- Streaming
for_each_chunkmethod on DirectClient — process gRPC response chunks via callback without materializing the full response in memory - Pre-allocation hint in
collect_stream— usesoriginal_sizefromResponseDatato pre-allocate the decompression buffer, reducing reallocations - Horner-form
norm_cdf— replaced Abramowitz & Stegun polynomial approximation with Zelen & Severo Horner-form evaluation (~1e-7 accuracy, fewer multiplications) - Python SDK: FPSS streaming —
FpssClientclass withsubscribe(),next_event(), andshutdown()methods for real-time market data in Python - Python SDK: pandas DataFrame conversion —
to_dataframe()function plus per-endpoint DataFrame convenience methods on DirectClient (later superseded in #379 by the unifiedto_dataframe(ticks)Arrow-backed path); install withpip install thetadatadx[pandas] - FFI crate: FPSS support — 7 new
extern "C"functions for FPSS lifecycle (fpss_connect,fpss_subscribe_quotes,fpss_subscribe_trades,fpss_subscribe_open_interest,fpss_next_event,fpss_shutdown,fpss_free_event) - Go SDK: FPSS streaming —
FpssClientGo struct wrapping the FFI FPSS functions - C++ SDK: FPSS streaming —
FpssClientC++ RAII class wrapping the FFI FPSS functions
Fixed
- Version bump for crates.io/PyPI publish (v1.1.0 tag was re-pushed during history restore)
Changed
- All TODO performance items now complete: streaming iterator (
for_each_chunk), optimizednorm_cdf(Horner-form), concurrent request semaphore (mdds_concurrent_requests)
1.1.0 - 2026-03-26
Added
- All 61 endpoints via declarative macro (was 19 hand-written) — covers every v3 gRPC RPC: stock, option, index, interest rate, calendar
- All 61 endpoints in every SDK — Python, Go, C++, C FFI all match Rust core
- Zero-allocation FPSS path — fully sync I/O thread + LMAX Disruptor ring buffer (
disruptor-rsv4), no tokio in the streaming hot path - Cache-line aligned tick types —
#[repr(C, align(64))]on TradeTick, QuoteTick, OhlcTick, EodTick - Cached QueryInfo template — no per-request String allocation
- Precomputed DataTable column indices — O(1) per row, not O(headers)
- pow10 lookup tables for Price comparison and conversion
#[inline]on all hot-path functions (FIT decode, Price ops, tick accessors)- Reusable thread-local zstd decompressor — no fresh allocation per chunk
- Criterion benchmarks — fit_decode, price_to_f64, price_compare, all_greeks, fie_encode
- AdaptiveWaitStrategy — 3-phase spin/yield/hint tuned for ~100us FPSS tick intervals
Changed
- Authenticated against real Nexus API (session established)
- Retrieved 25,341 stock symbols from MDDS
- Retrieved 42 AAPL EOD ticks (Jan-Mar 2024) with correct OHLCV data
- Retrieved 2,010 SPY option expirations
- Retrieved 13,160 index symbols
- Calendar endpoint returned valid data
client_type = "rust-thetadatadx"accepted by server
1.0.1 - 2026-03-26
Changed
- Renamed crate from
thetadxtothetadatadx(crates.io + PyPI) - Renamed repository from
thetadxtoThetaDataDxClient - Changed license metadata
- Updated top-level README
- README updated with GitHub callouts (NOTE, TIP, IMPORTANT, WARNING, CAUTION)
- Fixed PyPI package description (was empty — added readme field to pyproject.toml)
1.0.0 - 2026-03-26
Added
- DirectClient for MDDS gRPC — all 60 gRPC RPCs exposed as 61 typed endpoint methods (stock/option/index/rate/calendar: list, history, snapshot, at-time, greeks) via declarative
define_endpoint!macro - FpssClient for FPSS streaming — real-time quotes, trades, open interest, OHLC via TLS/TCP with heartbeat and manual reconnection
- Auth module — Nexus API authentication (email/password → session UUID)
- FIT/FIE codec — nibble-based tick compression/decompression (ported from Java)
- Greeks calculator — full Black-Scholes: 22 Greeks + IV bisection solver with precomputed shared intermediates and edge-case guards (t=0, v=0)
- All tick types — TradeTick, QuoteTick, OhlcTick, EodTick, OpenInterestTick, SnapshotTradeTick, TradeQuoteTick with fixed-point Price encoding
- 80+ DataType enum codes — quotes, trades, OHLC, all Greek orders, dividends, splits, fundamentals
- Proto definitions — extracted via runtime FileDescriptor reflection from ThetaData Terminal v202603181 (endpoints.proto + v3_endpoints.proto)
- Runtime configuration —
DirectConfigwith all JVM-equivalent tuning knobs contract_lookup(id)onFpssClientfor single-entry hot-path lookupFpssEvent::Errorvariant for surfacing protocol parse failures- Date parameter validation on all
DirectClientmethods async-zstdfeature flag for optional streaming decompression- Python SDK (PyO3/maturin) — wraps the Rust crate, not a reimplementation
- Go SDK — CGo FFI bindings over the C ABI layer
- C++ SDK — RAII C++ wrapper over the C header
- C FFI crate (
thetadatadx-ffi) — stableextern "C"ABI for all SDKs - Documentation — architecture (Mermaid), API reference, Java parity checklist
- CI/CD — GitHub Actions (fmt, clippy, test, FFI build, crates.io publish, PyPI publish, GitHub Release)
- Project infrastructure — CHANGELOG, CONTRIBUTING, SECURITY, CODE_OF_CONDUCT, clippy.toml, cliff.toml, rust-toolchain.toml, LICENSE
Security
- Credential
Debugredaction — passwords never appear in debug output AuthRequestdoes not deriveDebug(prevents password in error traces)- Session UUID redaction — bearer tokens logged at
debug!level only, first 8 chars assert!on FPSS frame size limits — enforced in release builds- Unified TLS via rustls for all connections (MDDS gRPC + FPSS TCP + Nexus HTTP)
- Timeouts on all network operations (auth 10s/5s, gRPC keepalive, FPSS connect, FPSS read 10s)
- 7 credential/account errors treated as permanent disconnect (no futile reconnect loops)
- Contract root length validated before wire serialization
- FIT decoder uses i64 accumulator with i32 saturation (no silent overflow)
- Price type range enforced with
assert!in release builds