Streaming data into BigQuery
The Storage Write API streams rows into BigQuery so they can be queried within seconds. Its gRPC interface is the cheaper, exactly-once option with 2 TiB a month free; the REST interface bills every row as at least 1 KB and blocks DML on recent rows for 30 minutes.
Streaming writes rows into a table continuously, a few at a time, and makes them queryable within seconds. It is how events, logs and change feeds land in BigQuery when a nightly load job is too late. Streaming isn’t free like batch loading, and the two interfaces of the Storage Write API differ a lot in price and guarantees.
Two interfaces
| gRPC (Storage Write API) | REST (Storage Write API REST, formerly insertAll) | |
|---|---|---|
| Price | $0.025 per GiB, first 2 TiB a month free | $0.01 per 200 MiB ($0.05 per GiB), each row counted as at least 1 KB |
| Delivery | at-least-once (default stream) or exactly-once (with offsets) | at-least-once, with an optional insertId per row |
| Formats | protocol buffers, Apache Arrow (JSON through the client libraries) | JSON |
UPDATE/DELETE/MERGE on recent rows | allowed (not inside a multi-statement transaction) | not for 30 minutes after a row is written |
| Throughput per project | 3 GB/s in us and eu, 300 MB/s elsewhere | 1 GB/s in us and eu, 300 MB/s elsewhere |
Prices are for the US as of October 2026. Google recommends gRPC for new work, and the numbers agree: it is half the price per GB before the free tier, and much cheaper for small rows, which the REST interface rounds up to 1 KB each. 10 million 200-byte events cost the same on REST as 10 GB.
Streams and delivery guarantees
With gRPC you write to a stream:
- The default stream needs no setup and many writers can share it. Rows are visible to queries as soon as they are written. Delivery is at-least-once: if a write is retried after a network error, the row can land twice.
- A committed stream that your application creates makes rows visible immediately too, but with offsets: the client numbers each append, and BigQuery never writes the same offset twice. That gives exactly-once delivery.
- A pending stream holds rows until you commit it, and the commit makes them all visible at once. Several workers can each write a pending stream and commit them together. It is a streaming alternative to a batch load.
For most pipelines, the default stream with idempotent processing downstream (deduplicate by an event ID when you read) is enough. Use committed streams with offsets when duplicates are unacceptable and you control the writer.
What streamed rows mean for queries and DML
- New rows are part of every query of the table, so dashboards on a streamed table see data within seconds.
- Rows streamed through the REST interface can’t be changed with
UPDATE,DELETE,MERGEorTRUNCATEfor 30 minutes. Older rows in the same table can. See INSERT, UPDATE and DELETE. - Streamed data can take up to 90 minutes to be included in a table copy.
- Streaming doesn’t count toward the 1,500 table modifications a day that limit load jobs, so it suits frequent small writes.
Limits
| Limit | Value |
|---|---|
| Concurrent gRPC connections per project | 5,000 in a region, 20,000 in a multi-region |
| Throughput per gRPC connection | at least about 1 MB/s; open more connections for more |
AppendRows request size (gRPC) | 20 MB |
| Row size and request size (REST) | 10 MB each; 50,000 rows per request, 500 recommended |
| Data committed at once from pending streams | 10 TB in us/eu, 1 TB elsewhere |
Streaming, loading or DML
| Need | Use |
|---|---|
| Rows queryable within seconds | Storage Write API |
| Files arriving every hour or day | a batch load job, which is free |
| A few changes from an application now and then | INSERT / MERGE statements |
| Mirroring a database’s inserts, updates and deletes | Change data capture |