Loading Parquet into BigQuery
BigQuery reads the schema from a Parquet file, so a load needs no schema. The traps are a LIST that arrives as nested list.element structs, a nanosecond timestamp that becomes an INTEGER, and a UINT64 above 2^63 that fails the load.
To load a Parquet file into BigQuery, run bq load --source_format=PARQUET mydataset.mytable file.parquet. BigQuery takes the column names and types from the file, so you pass no schema and no --autodetect. The same works from SQL with LOAD DATA, from the console, and from Querylab.io’s Import tab. A batch load is free; you pay only to store the table. Most loads just work. The ones that don’t are almost always one of five things: a LIST column that arrives wrapped in list.element, a nanosecond timestamp that arrives as an integer, a MAP that arrives as an array of key and value records, a DECIMAL too big for NUMERIC, or a UINT64 above 2^63.
Everything below was tested with real load jobs, on files written with pyarrow and DuckDB. The error messages are the ones BigQuery returned.
Loading the file
From a local file or Cloud Storage with bq:
bq load --source_format=PARQUET \
mydataset.events \
'gs://my-bucket/events/2026-03-*.parquet'
From SQL. LOAD DATA reads from Cloud Storage only, and the options use underscores:
LOAD DATA INTO mydataset.events
FROM FILES (
format = 'PARQUET',
uris = ['gs://my-bucket/events/2026-03-*.parquet'],
enable_list_inference = true,
decimal_target_types = ['BIGNUMERIC']
);
Neither example is run here; both need a bucket and a dataset. A load appends by default. --replace or LOAD DATA OVERWRITE replaces the table. Wildcards work for Cloud Storage only, and a local file loads one per command. The bq load flags and the LOAD DATA options in the example are the same settings under different names: --parquet_enable_list_inference, --decimal_target_types and --parquet_enum_as_string become enable_list_inference, decimal_target_types and enum_as_string. BigQuery accepted all three option names in a dry run.
How Parquet types map
This is what each Parquet type became in BigQuery with no flags. Each row was a separate load.
| In the Parquet file | Becomes | Notes |
|---|---|---|
| INT8, INT16, INT32, INT64 | INTEGER | All 64-bit in BigQuery. |
| UINT8, UINT16, UINT32 | INTEGER | 4000000000 as a UINT32 loaded fine. |
| UINT64 up to 2^63 - 1 | INTEGER | 5 loaded. |
| UINT64 above 2^63 - 1 | load fails | Value 9223372036854775813 outside range for column. Valid range [-9223372036854775808, 9223372036854775807]; in column 'u' |
| FLOAT, DOUBLE | FLOAT | NaN, Infinity and -0.0 load; -0.0 came back as 0. |
| float16 | BYTES | Two raw bytes, not a number. |
| BOOLEAN | BOOLEAN | |
| STRING, dictionary-encoded STRING | STRING | Dictionary encoding is invisible. |
| BINARY, fixed-length binary | BYTES | Includes UUIDs written as 16 bytes. |
| DATE32, DATE64 (pyarrow) | DATE | |
| TIME in milliseconds or microseconds | TIME | |
| TIME in nanoseconds | INTEGER | 36001000005000, nanoseconds since midnight. |
| TIMESTAMP in milliseconds or microseconds, UTC-adjusted or not | TIMESTAMP | A timestamp without a timezone is read as UTC. |
| TIMESTAMP in nanoseconds, UTC-adjusted or not | INTEGER | 1772359200123456000. No error, no warning. |
| INT96 | TIMESTAMP | Microseconds kept: 10:00:00.123456. |
| DECIMAL, any precision | NUMERIC | Depends on the values, not on the declared precision. See below. |
| LIST | RECORD with a list field | Unless list inference is on. See below. |
| MAP | RECORD with a key_value field | Always, even with list inference on. |
| Nested struct | RECORD | Nesting is kept. |
A few of these differ from what you would guess.
Nanosecond timestamps are not timestamps. Any writer that keeps nanosecond precision produces this, such as a pyarrow timestamp('ns') column. The load succeeds and the column is an INTEGER. The fix is below. INT96, millisecond and microsecond timestamps are fine. In pyarrow, pq.write_table(table, path, coerce_timestamps="us", allow_truncated_timestamps=True) writes microseconds, and that file loaded as TIMESTAMP with 10:00:00.123456.
A timestamp without a timezone arrives as TIMESTAMP, read as UTC. A pyarrow timestamp('us') (no tz) loaded as TIMESTAMP with the wall-clock time unchanged: 10:00:00.123456 became 2026-03-01T10:00:00.123456Z. A UTC-adjusted column loaded to the same value. If the values are local wall-clock times, you probably want a DATETIME column. Pass a schema (next section).
Declared decimal precision doesn’t matter. We loaded decimal(10,2), decimal(38,10), decimal(40,10), decimal(50,2) and decimal(76,38) columns holding 1.5, and all became NUMERIC. NUMERIC holds 38 digits with 9 after the decimal point. A load fails only when a value doesn’t fit:
| Value in the file | Default result |
|---|---|
1234567890123456789012345678.1234567891, decimal(38,10) | fails: The value for column 'd' is out of valid NUMERIC range: Value will lose precision after scaling down to NUMERIC type; input length: 16; scale: 10 |
12345678901234567890123456789012345678.12, decimal(40,2) | fails: Value is out of range after scaling to NUMERIC type |
1.12345678901234567890123456789012345678, decimal(76,38) | fails: same as the first, with scale: 38 |
The error message ends with “Consider specifying decimal target types”. With --decimal_target_types=BIGNUMERIC, all three loaded exactly (BIGNUMERIC), and --decimal_target_types=STRING loaded the 38-fraction-digit value as a string. The flag takes one type; NUMERIC,BIGNUMERIC is rejected by bq (value should be one of <NUMERIC|BIGNUMERIC|STRING>). A decimal with a scale of 0 and small values (decimal(9,0)) came in as NUMERIC with 15, not as INTEGER.
A LIST arrives wrapped in list.element
This is the one that surprises most people. We loaded a Parquet column of type list<string> holding ['a', 'b']. BigQuery created a RECORD named tags with one REPEATED RECORD field named list, and that has one field named element:
{ "tags": { "list": [{ "element": "a" }, { "element": "b" }] } }
That is the Parquet file’s physical layout (tags > list > element) copied as it is. Every query then has to write tags.list[OFFSET(0)].element or unnest tags.list. The same happens to a list of structs (items.list[].element.sku) and to a list of lists (element.list[].element). Files written by DuckDB and by pyarrow both have this layout.
Turn on list inference and BigQuery removes the wrapper:
bq load --source_format=PARQUET --parquet_enable_list_inference \
mydataset.orders orders.parquet
With it, tags is a plain STRING column in REPEATED mode, and items is a REPEATED RECORD with sku and qty. Here is what changed and what failed, each in its own load:
| File | Without inference | With --parquet_enable_list_inference |
|---|---|---|
['a','b'] and [] | tags.list[].element; [] becomes {"list": []} | tags as ARRAY<STRING>: ["a","b"] and [] |
a NULL list | tags is NULL | an empty array [] |
['a', NULL, 'b'] | loads, with {"element": null} | fails: Array cannot have a null element; error in writing field tags |
[[1,2],[3]] | loads as nested list.element structs | fails: list of lists is not supported with LIST logical type inference enabled |
[{sku, qty}, ...] | items.list[].element.sku | items as ARRAY<STRUCT<sku STRING, qty INT64>> |
a MAP | m.key_value[] with key and value | the same: inference doesn’t touch maps |
So inference is not a safe default. A list that contains a NULL element, or a list of lists, loads without the flag and fails with it. A BigQuery array can’t hold a NULL or another array directly (see Arrays), which is where both failures come from. If you can’t turn inference on, unwrap the structure in SQL:
-- Without list inference a Parquet LIST arrives as tags.list[].element. WITH loaded AS ( SELECT 'A-100' AS order_id, STRUCT([STRUCT('gift' AS element), STRUCT(CAST(NULL AS STRING)), STRUCT('rush')] AS list) AS tags UNION ALL SELECT 'A-101', STRUCT([STRUCT('rush' AS element)] AS list) ) SELECT order_id, ARRAY(SELECT t.element FROM UNNEST(tags.list) AS t WHERE t.element IS NOT NULL) AS tags FROM loaded ORDER BY order_id
| Row | order_id | tags |
|---|---|---|
| 1 | A-100 | [gift, rush] |
| 2 | A-101 | [rush] |
The NULL element is dropped by the WHERE, which also makes this the way to clean a list before you use it.
A map has no inference flag at all. It always arrives as m.key_value, an array of key and value records:
-- A Parquet MAP arrives as m.key_value[] with key and value fields. WITH loaded AS ( SELECT 'sensor-1' AS device, STRUCT([STRUCT('temp' AS key, 21 AS value), STRUCT('humidity', 40)] AS key_value) AS readings UNION ALL SELECT 'sensor-2', STRUCT([STRUCT('temp' AS key, 19 AS value)] AS key_value) ) SELECT device, (SELECT kv.value FROM UNNEST(readings.key_value) AS kv WHERE kv.key = 'humidity') AS humidity FROM loaded ORDER BY device
| Row | device | humidity |
|---|---|---|
| 1 | sensor-1 | 40 |
| 2 | sensor-2 | null |
For other ways to reshape such data, see Nested and repeated data.
Fixing a column type at load time
If you pass --schema, BigQuery reads only the columns you list, and converts when it can. We found:
| Schema you pass | File column | Result |
|---|---|---|
t:DATETIME | timestamp, microseconds, any timezone | 2026-03-01T10:00:00.123456; the UTC wall clock, not your local time |
i:TIMESTAMP | INT64 1700000000123 (milliseconds) | loads, but as microseconds: 1970-01-20 16:13:20.000123 |
t:TIMESTAMP | timestamp in nanoseconds | fails: Invalid timestamp nanoseconds value 1772359200123456000 of logical type TIMESTAMP_NANOS |
t:STRING | any timestamp | fails: Parquet column 't' has type INT64 which does not match the target cpp_type STRING_PIECE. |
s:TIMESTAMP | a string column | fails: Parquet column 's' has type BYTE_ARRAY which does not match the target cpp_type INT64. |
i16:FLOAT | an INT32 column | fails: Parquet column 'i16' has type INT32 which does not match the target cpp_type DOUBLE. |
i8:INTEGER | a file with ten columns | loads one column and ignores the rest |
t:INTEGER,x:STRING | a file with no x column | x is NULL for every row |
Two practical points. A schema is a way to pick columns and to ask for DATETIME, not a general cast: a Parquet type that doesn’t match the target type fails the load. And a column that is already INTEGER can be turned into a timestamp in SQL after the load, which is how to handle nanoseconds:
-- A TIMESTAMP(NANOS) column loads as INTEGER, in nanoseconds since the epoch. WITH loaded AS ( SELECT 1 AS event_id, 1772359200123456000 AS created_at UNION ALL SELECT 2, 1772359260000000000 ) SELECT event_id, created_at, TIMESTAMP_MICROS(DIV(created_at, 1000)) AS created_ts FROM loaded ORDER BY event_id
| Row | event_id | created_at | created_ts |
|---|---|---|---|
| 1 | 1 | 1772359200123456000 | 2026-03-01 10:00:00.123456 UTC |
| 2 | 2 | 1772359260000000000 | 2026-03-01 10:01:00 UTC |
DIV(created_at, 1000) turns nanoseconds into microseconds, which TIMESTAMP_MICROS expects. The nanosecond digits below the microsecond are lost.
The millisecond case in the table is the same trap in reverse. BigQuery reads an integer loaded into a TIMESTAMP column as microseconds, so a Unix epoch in milliseconds lands in January 1970:
-- An INT64 loaded into a TIMESTAMP column is read as microseconds. SELECT 1772359200123 AS ms_since_epoch, TIMESTAMP_MICROS(1772359200123) AS misread_as_micros, TIMESTAMP_MILLIS(1772359200123) AS read_as_millis
| Row | ms_since_epoch | misread_as_micros | read_as_millis |
|---|---|---|---|
| 1 | 1772359200123 | 1970-01-21 12:19:19.200123 UTC | 2026-03-01 10:00:00.123000 UTC |
Load such a column as INTEGER and use TIMESTAMP_MILLIS().
Column names
Parquet doesn’t restrict column names, but BigQuery does. What we loaded:
| Column name | Result |
|---|---|
user-id, a b, 1x, größe | loads, with the name as it is |
Name and name in one file | fails: Duplicated field name 'name'. Column names are case-insensitive. |
@t, a.b | fails: Field name '@t' is not supported by the current character map. Please change your field name or use character map V2 to let the system modify the field names. |
_TABLE_x | fails: Invalid field name "_TABLE_x". Field names are not allowed to start with the (case-insensitive) prefixes _PARTITION, _TABLE_, _FILE_, _ROW_TIMESTAMP, __ROOT__, _COLIDENTIFIER, and __DREMEL_PK_MERGED_STRUCT_ |
| 301 characters | fails: Invalid field name "xxx...". Fields must contain only...; the limit is 300 |
| an empty name | fails: Parquet file contains invalid empty field name in schema |
a nested field named a-b | loads |
The error for @t mentions “character map V2”. That is the --column_name_character_map option; we did not test it. Names with spaces, dashes and a leading digit load without it, so the older rule that names must start with a letter and use only letters, digits and underscores does not describe what a load accepts today. Rename awkward columns in the writer, or alias them in a query after loading.
Appending a file whose schema differs
A load into an existing table checks the file’s schema against the table’s. We loaded id INT64, name STRING first, then appended other files:
| Appended file | Result |
|---|---|
an extra column extra | fails: Provided Schema does not match Table ... Cannot add fields (field: extra) |
the same, with --schema_update_option=ALLOW_FIELD_ADDITION | loads; old rows get NULL in extra |
id as a string | fails: Field id has changed type from INTEGER to STRING |
name missing | loads; name is NULL for the new rows |
ID and Name in capitals | loads into the existing id and name columns |
A missing column is fine because Parquet columns are NULLABLE. A column written as required in Parquet arrives as REQUIRED (we loaded one), and appending a nullable file to that table fails: Field id has changed mode from REQUIRED to NULLABLE. That needs --schema_update_option=ALLOW_FIELD_RELAXATION.
The Parquet docs say a wildcard load can’t mix files with different schemas, and that if BigQuery has to pick a schema it uses the alphabetically last file. We could not test a wildcard load locally, so we only quote the docs here.
Empty and broken files
- A Parquet file with a schema and zero rows loads: the table is created with the columns and has 0 rows.
- A zero-byte file fails:
Input file is not in Parquet format.
Compression and limits
Compression is inside the Parquet file, so file.parquet stays one file. Files written with Snappy, gzip, zstd, Brotli and LZ4 all loaded. Google’s list of codecs is GZip, LZO, LZ4_RAW, Snappy and ZSTD, which doesn’t include Brotli; ours loaded anyway, but don’t rely on that.
| Limit | Value |
|---|---|
| Size of one load job | 15 TB (not applied with a reservation) |
| Files in one load job | 10,000,000 |
| Columns per table | 10,000 |
| Load jobs per table per day | 1,500, failed ones included |
| Load job run time | 6 hours |
Row size, to avoid resourcesExceeded | 50 MB |
Two input guidelines from Google: use row groups of at least 16 MiB, because smaller ones slow the load, and with more than 100 columns consider a page size below the 1 MiB default.
ORC
ORC loads the same way: bq load --source_format=ORC. Our results:
| In the ORC file | Becomes |
|---|---|
| int, string, float, boolean, date, binary | INTEGER, STRING, FLOAT, BOOLEAN, DATE, BYTES |
| decimal(10,2) | NUMERIC; --decimal_target_types=BIGNUMERIC gave BIGNUMERIC |
| timestamp (no timezone), nanoseconds | TIMESTAMP, microseconds kept; --schema t:DATETIME also works |
timestamp with local timezone (TIMESTAMP_INSTANT) | fails: Unaccepted ORC type kind: 18 |
| list of string | STRING in REPEATED mode, with no list.element wrapper |
| map | REPEATED RECORD with key and value |
| struct | RECORD |
Unlike Parquet, ORC lists and maps need no flag. Zero rows loads. A column of ORC timestamps with a timezone doesn’t load at all, so write them without one.
What Querylab.io does
The Import tab accepts a .parquet file, reads its schema in the browser, and shows the detected columns before anything is written. When you confirm, it uploads the file unchanged in a load job, together with the schema from the preview, so a column you changed in the preview is the type you get. It does not set parquetOptions, so list inference is off. See Import.
What it costs
A batch load on the shared slot pool is free. You pay to store the table, and for the Parquet file in Cloud Storage while it sits there. See Storage pricing. If you only need to query the files now and then, an external table reads Parquet in place without a load. The rest of the loading workflow, including partition loads, is in Loading data into BigQuery.