Flat Files

Flat files deliver the whole universe for one date in one call — every option contract or every stock for a given (security type, request type, date). Use them for daily ETL and backtests that need everything; use the per-contract reference endpoints when you need one contract fast; use streaming for live data.

Pull a file

The flat_files namespace exposes one method per (security type, request type) pair: option_quote, option_trade, option_trade_quote, option_ohlc, option_open_interest, option_eod, stock_quote, stock_trade, stock_trade_quote, stock_eod — plus a generic request(sec_type, req_type, date).

use thetadatadx::flatfiles::{FlatFileFormat, ReqType, SecType};

// Vendor-format file straight to disk (bounded memory):
tdx.flatfile_option_quote("20250303", "quotes.csv", FlatFileFormat::Csv).await?;

// Decoded rows in memory:
let rows = tdx.flatfile_request_decoded(SecType::Option, ReqType::Quote, "20250303").await?;

rows = tdx.flat_files.option_quote("20250303")
df = rows.to_polars()          # or .to_pandas() / .to_arrow() / .to_list()

# Or write the vendor-format file straight to disk (bounded memory):
tdx.flatfile_to_path("OPTION", "QUOTE", "20250303", "quotes.csv", "csv")

const rows = tdx.flatFiles.optionQuote('20250303');
const ipc = rows.toArrowIpc();   // feed into apache-arrow `tableFromIPC`

auto rows = client.flat_files().option_quote("20250303");
auto ipc = rows.to_arrow_ipc();  // std::vector<uint8_t>

curl 'http://127.0.0.1:25503/v3/flatfile/option/quote?date=2025-03-03&format=csv' -o quotes.csv

The server streams the response body in chunks, so downloads of any size run in bounded memory. The same surface is available from the CLI: tdx flatfile quotes 20250303 --format csv -o quotes.csv.

Size guidance

Flat files are large — a whole-universe option-quote day commonly exceeds 100 MB and tens of millions of rows.

• The decoded-rows path materializes everything in memory before returning; reserve it for machines with headroom.
• The to-disk path (flatfile_to_path, the HTTP route, the CLI) keeps peak memory bounded and is the right default for ETL.
• Transient download failures retry automatically with backoff; tune attempts and backoff via the flatfiles_* configuration fields.