The BigQuery Storage Read API
The Storage Read API streams table data out of BigQuery in parallel, in Arrow or Avro, with column and row filters, for Spark, pandas and other engines. It is billed per TiB read with 300 TiB a month free, reads from a snapshot, and can't read views or external tables directly.
The Storage Read API is the fast way to get table data out of BigQuery into another program. Instead of paging through query results, a client opens a read session on a table and reads it as several parallel streams of Apache Arrow or Avro rows. It is what the Spark BigQuery connector, the Python client’s to_dataframe(), BigQuery DataFrames and many BI and ML tools use under the hood. You rarely call it yourself, but its pricing and limits apply whenever those tools read BigQuery data.
How it works
- The client creates a read session on a table, and can choose a subset of columns, a simple row filter, a snapshot time and a maximum number of streams.
- BigQuery answers with the schema and one or more stream IDs. To read the whole table, the client must read every stream.
- Each stream is read with
ReadRows, independently, so many workers can read in parallel. A stream can be split further for more parallelism.
Column selection and the row filter run inside BigQuery storage, so only the selected data crosses the network.
Consistency
A session reads the table as it was at one moment, by default when the session was created. Rows written afterwards aren’t included, and all streams see the same snapshot, so a distributed job reads a consistent table. You can ask for an earlier snapshot time, within the time travel window. A session lasts at least 6 hours.
Cost
| Price (US, October 2026) | |
|---|---|
| Data read | $1.10 per TiB |
| Free each month | 300 TiB per billing account |
| Reading the temporary table of a query result | free, and doesn’t count toward the 300 TiB |
| Network egress | billed separately if the data leaves the region |
Reads from Google’s serverless Spark environments aren’t charged for streaming reads. Compared with querying, reading a whole table through the Storage Read API is much cheaper than SELECT * at $6.25 per TiB on on-demand pricing, and needs no slots.
What it can’t read
- Logical and materialized views. The API reads storage, and views have none. Run a query over the view and read its result table instead; the Spark connector does this for you when you enable it.
- External tables. Use BigLake or Iceberg tables. See BigLake and Iceberg.
- Row access policies with subqueries. Row-level and column-level security are enforced on Storage Read API reads, but a row access policy whose filter uses a subquery isn’t compatible with the API; only simple filters are. See Row-level security.
Permissions
The caller needs read access to the table (BigQuery Data Viewer) and permission to create read sessions in the project that pays (bigquery.readsessions.create, getData, update; the BigQuery Read Session User role).
When to use it, and when not
| Need | Use |
|---|---|
| Load a table or a large query result into pandas, Spark or an ML pipeline | the Storage Read API (through your client library) |
| Copy a table to files in Cloud Storage | an extract job, which is free on the shared pool |
| A few thousand rows for an application | a normal query |
| Write data into BigQuery | the Storage Write API |
In Python, client.query(sql).to_dataframe() uses the Storage Read API automatically when the google-cloud-bigquery-storage package is installed, which makes downloading large results many times faster.