Docs/Writing data/Loading Parquet
Writing data

Loading Parquet into BigQuery

BigQuery reads the schema from a Parquet file, so a load needs no schema. The traps are a LIST that arrives as nested list.element structs, a nanosecond timestamp that becomes an INTEGER, and a UINT64 above 2^63 that fails the load.

To load a Parquet file into BigQuery, run bq load --source_format=PARQUET mydataset.mytable file.parquet. BigQuery takes the column names and types from the file, so you pass no schema and no --autodetect. The same works from SQL with LOAD DATA, from the console, and from Querylab.io’s Import tab. A batch load is free; you pay only to store the table. Most loads just work. The ones that don’t are almost always one of five things: a LIST column that arrives wrapped in list.element, a nanosecond timestamp that arrives as an integer, a MAP that arrives as an array of key and value records, a DECIMAL too big for NUMERIC, or a UINT64 above 2^63.

Everything below was tested with real load jobs, on files written with pyarrow and DuckDB. The error messages are the ones BigQuery returned.

Loading the file

From a local file or Cloud Storage with bq:

bq load --source_format=PARQUET \
  mydataset.events \
  'gs://my-bucket/events/2026-03-*.parquet'

From SQL. LOAD DATA reads from Cloud Storage only, and the options use underscores:

LOAD DATA INTO mydataset.events
FROM FILES (
  format = 'PARQUET',
  uris = ['gs://my-bucket/events/2026-03-*.parquet'],
  enable_list_inference = true,
  decimal_target_types = ['BIGNUMERIC']
);

Neither example is run here; both need a bucket and a dataset. A load appends by default. --replace or LOAD DATA OVERWRITE replaces the table. Wildcards work for Cloud Storage only, and a local file loads one per command. The bq load flags and the LOAD DATA options in the example are the same settings under different names: --parquet_enable_list_inference, --decimal_target_types and --parquet_enum_as_string become enable_list_inference, decimal_target_types and enum_as_string. BigQuery accepted all three option names in a dry run.

How Parquet types map

This is what each Parquet type became in BigQuery with no flags. Each row was a separate load.

In the Parquet fileBecomesNotes
INT8, INT16, INT32, INT64INTEGERAll 64-bit in BigQuery.
UINT8, UINT16, UINT32INTEGER4000000000 as a UINT32 loaded fine.
UINT64 up to 2^63 - 1INTEGER5 loaded.
UINT64 above 2^63 - 1load failsValue 9223372036854775813 outside range for column. Valid range [-9223372036854775808, 9223372036854775807]; in column 'u'
FLOAT, DOUBLEFLOATNaN, Infinity and -0.0 load; -0.0 came back as 0.
float16BYTESTwo raw bytes, not a number.
BOOLEANBOOLEAN
STRING, dictionary-encoded STRINGSTRINGDictionary encoding is invisible.
BINARY, fixed-length binaryBYTESIncludes UUIDs written as 16 bytes.
DATE32, DATE64 (pyarrow)DATE
TIME in milliseconds or microsecondsTIME
TIME in nanosecondsINTEGER36001000005000, nanoseconds since midnight.
TIMESTAMP in milliseconds or microseconds, UTC-adjusted or notTIMESTAMPA timestamp without a timezone is read as UTC.
TIMESTAMP in nanoseconds, UTC-adjusted or notINTEGER1772359200123456000. No error, no warning.
INT96TIMESTAMPMicroseconds kept: 10:00:00.123456.
DECIMAL, any precisionNUMERICDepends on the values, not on the declared precision. See below.
LISTRECORD with a list fieldUnless list inference is on. See below.
MAPRECORD with a key_value fieldAlways, even with list inference on.
Nested structRECORDNesting is kept.

A few of these differ from what you would guess.

Nanosecond timestamps are not timestamps. Any writer that keeps nanosecond precision produces this, such as a pyarrow timestamp('ns') column. The load succeeds and the column is an INTEGER. The fix is below. INT96, millisecond and microsecond timestamps are fine. In pyarrow, pq.write_table(table, path, coerce_timestamps="us", allow_truncated_timestamps=True) writes microseconds, and that file loaded as TIMESTAMP with 10:00:00.123456.

A timestamp without a timezone arrives as TIMESTAMP, read as UTC. A pyarrow timestamp('us') (no tz) loaded as TIMESTAMP with the wall-clock time unchanged: 10:00:00.123456 became 2026-03-01T10:00:00.123456Z. A UTC-adjusted column loaded to the same value. If the values are local wall-clock times, you probably want a DATETIME column. Pass a schema (next section).

Declared decimal precision doesn’t matter. We loaded decimal(10,2), decimal(38,10), decimal(40,10), decimal(50,2) and decimal(76,38) columns holding 1.5, and all became NUMERIC. NUMERIC holds 38 digits with 9 after the decimal point. A load fails only when a value doesn’t fit:

Value in the fileDefault result
1234567890123456789012345678.1234567891, decimal(38,10)fails: The value for column 'd' is out of valid NUMERIC range: Value will lose precision after scaling down to NUMERIC type; input length: 16; scale: 10
12345678901234567890123456789012345678.12, decimal(40,2)fails: Value is out of range after scaling to NUMERIC type
1.12345678901234567890123456789012345678, decimal(76,38)fails: same as the first, with scale: 38

The error message ends with “Consider specifying decimal target types”. With --decimal_target_types=BIGNUMERIC, all three loaded exactly (BIGNUMERIC), and --decimal_target_types=STRING loaded the 38-fraction-digit value as a string. The flag takes one type; NUMERIC,BIGNUMERIC is rejected by bq (value should be one of <NUMERIC|BIGNUMERIC|STRING>). A decimal with a scale of 0 and small values (decimal(9,0)) came in as NUMERIC with 15, not as INTEGER.

A LIST arrives wrapped in list.element

This is the one that surprises most people. We loaded a Parquet column of type list<string> holding ['a', 'b']. BigQuery created a RECORD named tags with one REPEATED RECORD field named list, and that has one field named element:

{ "tags": { "list": [{ "element": "a" }, { "element": "b" }] } }

That is the Parquet file’s physical layout (tags > list > element) copied as it is. Every query then has to write tags.list[OFFSET(0)].element or unnest tags.list. The same happens to a list of structs (items.list[].element.sku) and to a list of lists (element.list[].element). Files written by DuckDB and by pyarrow both have this layout.

Turn on list inference and BigQuery removes the wrapper:

bq load --source_format=PARQUET --parquet_enable_list_inference \
  mydataset.orders orders.parquet

With it, tags is a plain STRING column in REPEATED mode, and items is a REPEATED RECORD with sku and qty. Here is what changed and what failed, each in its own load:

FileWithout inferenceWith --parquet_enable_list_inference
['a','b'] and []tags.list[].element; [] becomes {"list": []}tags as ARRAY<STRING>: ["a","b"] and []
a NULL listtags is NULLan empty array []
['a', NULL, 'b']loads, with {"element": null}fails: Array cannot have a null element; error in writing field tags
[[1,2],[3]]loads as nested list.element structsfails: list of lists is not supported with LIST logical type inference enabled
[{sku, qty}, ...]items.list[].element.skuitems as ARRAY<STRUCT<sku STRING, qty INT64>>
a MAPm.key_value[] with key and valuethe same: inference doesn’t touch maps

So inference is not a safe default. A list that contains a NULL element, or a list of lists, loads without the flag and fails with it. A BigQuery array can’t hold a NULL or another array directly (see Arrays), which is where both failures come from. If you can’t turn inference on, unwrap the structure in SQL:

flatten tags.list[].element into an arrayOpen in Querylab.io
123456789101112
-- Without list inference a Parquet LIST arrives as tags.list[].element.
WITH loaded AS (
  SELECT 'A-100' AS order_id,
         STRUCT([STRUCT('gift' AS element), STRUCT(CAST(NULL AS STRING)), STRUCT('rush')] AS list) AS tags
  UNION ALL
  SELECT 'A-101', STRUCT([STRUCT('rush' AS element)] AS list)
)
SELECT
  order_id,
  ARRAY(SELECT t.element FROM UNNEST(tags.list) AS t WHERE t.element IS NOT NULL) AS tags
FROM loaded
ORDER BY order_id
Result in BigQuery
Roworder_idtags
1A-100[gift, rush]
2A-101[rush]
2 rows

The NULL element is dropped by the WHERE, which also makes this the way to clean a list before you use it.

A map has no inference flag at all. It always arrives as m.key_value, an array of key and value records:

read one entry from a loaded Parquet mapOpen in Querylab.io
123456789101112
-- A Parquet MAP arrives as m.key_value[] with key and value fields.
WITH loaded AS (
  SELECT 'sensor-1' AS device,
         STRUCT([STRUCT('temp' AS key, 21 AS value), STRUCT('humidity', 40)] AS key_value) AS readings
  UNION ALL
  SELECT 'sensor-2', STRUCT([STRUCT('temp' AS key, 19 AS value)] AS key_value)
)
SELECT
  device,
  (SELECT kv.value FROM UNNEST(readings.key_value) AS kv WHERE kv.key = 'humidity') AS humidity
FROM loaded
ORDER BY device
Result in BigQuery
Rowdevicehumidity
1sensor-140
2sensor-2null
2 rows

For other ways to reshape such data, see Nested and repeated data.

Fixing a column type at load time

If you pass --schema, BigQuery reads only the columns you list, and converts when it can. We found:

Schema you passFile columnResult
t:DATETIMEtimestamp, microseconds, any timezone2026-03-01T10:00:00.123456; the UTC wall clock, not your local time
i:TIMESTAMPINT64 1700000000123 (milliseconds)loads, but as microseconds: 1970-01-20 16:13:20.000123
t:TIMESTAMPtimestamp in nanosecondsfails: Invalid timestamp nanoseconds value 1772359200123456000 of logical type TIMESTAMP_NANOS
t:STRINGany timestampfails: Parquet column 't' has type INT64 which does not match the target cpp_type STRING_PIECE.
s:TIMESTAMPa string columnfails: Parquet column 's' has type BYTE_ARRAY which does not match the target cpp_type INT64.
i16:FLOATan INT32 columnfails: Parquet column 'i16' has type INT32 which does not match the target cpp_type DOUBLE.
i8:INTEGERa file with ten columnsloads one column and ignores the rest
t:INTEGER,x:STRINGa file with no x columnx is NULL for every row

Two practical points. A schema is a way to pick columns and to ask for DATETIME, not a general cast: a Parquet type that doesn’t match the target type fails the load. And a column that is already INTEGER can be turned into a timestamp in SQL after the load, which is how to handle nanoseconds:

nanosecond INTEGER to TIMESTAMPOpen in Querylab.io
123456789101112
-- A TIMESTAMP(NANOS) column loads as INTEGER, in nanoseconds since the epoch.
WITH loaded AS (
  SELECT 1 AS event_id, 1772359200123456000 AS created_at
  UNION ALL
  SELECT 2, 1772359260000000000
)
SELECT
  event_id,
  created_at,
  TIMESTAMP_MICROS(DIV(created_at, 1000)) AS created_ts
FROM loaded
ORDER BY event_id
Result in BigQuery
Rowevent_idcreated_atcreated_ts
1117723592001234560002026-03-01 10:00:00.123456 UTC
2217723592600000000002026-03-01 10:01:00 UTC
2 rows

DIV(created_at, 1000) turns nanoseconds into microseconds, which TIMESTAMP_MICROS expects. The nanosecond digits below the microsecond are lost.

The millisecond case in the table is the same trap in reverse. BigQuery reads an integer loaded into a TIMESTAMP column as microseconds, so a Unix epoch in milliseconds lands in January 1970:

milliseconds read as microsecondsOpen in Querylab.io
12345
-- An INT64 loaded into a TIMESTAMP column is read as microseconds.
SELECT
  1772359200123 AS ms_since_epoch,
  TIMESTAMP_MICROS(1772359200123) AS misread_as_micros,
  TIMESTAMP_MILLIS(1772359200123) AS read_as_millis
Result in BigQuery
Rowms_since_epochmisread_as_microsread_as_millis
117723592001231970-01-21 12:19:19.200123 UTC2026-03-01 10:00:00.123000 UTC
1 row

Load such a column as INTEGER and use TIMESTAMP_MILLIS().

Column names

Parquet doesn’t restrict column names, but BigQuery does. What we loaded:

Column nameResult
user-id, a b, 1x, größeloads, with the name as it is
Name and name in one filefails: Duplicated field name 'name'. Column names are case-insensitive.
@t, a.bfails: Field name '@t' is not supported by the current character map. Please change your field name or use character map V2 to let the system modify the field names.
_TABLE_xfails: Invalid field name "_TABLE_x". Field names are not allowed to start with the (case-insensitive) prefixes _PARTITION, _TABLE_, _FILE_, _ROW_TIMESTAMP, __ROOT__, _COLIDENTIFIER, and __DREMEL_PK_MERGED_STRUCT_
301 charactersfails: Invalid field name "xxx...". Fields must contain only...; the limit is 300
an empty namefails: Parquet file contains invalid empty field name in schema
a nested field named a-bloads

The error for @t mentions “character map V2”. That is the --column_name_character_map option; we did not test it. Names with spaces, dashes and a leading digit load without it, so the older rule that names must start with a letter and use only letters, digits and underscores does not describe what a load accepts today. Rename awkward columns in the writer, or alias them in a query after loading.

Appending a file whose schema differs

A load into an existing table checks the file’s schema against the table’s. We loaded id INT64, name STRING first, then appended other files:

Appended fileResult
an extra column extrafails: Provided Schema does not match Table ... Cannot add fields (field: extra)
the same, with --schema_update_option=ALLOW_FIELD_ADDITIONloads; old rows get NULL in extra
id as a stringfails: Field id has changed type from INTEGER to STRING
name missingloads; name is NULL for the new rows
ID and Name in capitalsloads into the existing id and name columns

A missing column is fine because Parquet columns are NULLABLE. A column written as required in Parquet arrives as REQUIRED (we loaded one), and appending a nullable file to that table fails: Field id has changed mode from REQUIRED to NULLABLE. That needs --schema_update_option=ALLOW_FIELD_RELAXATION.

The Parquet docs say a wildcard load can’t mix files with different schemas, and that if BigQuery has to pick a schema it uses the alphabetically last file. We could not test a wildcard load locally, so we only quote the docs here.

Empty and broken files

  • A Parquet file with a schema and zero rows loads: the table is created with the columns and has 0 rows.
  • A zero-byte file fails: Input file is not in Parquet format.

Compression and limits

Compression is inside the Parquet file, so file.parquet stays one file. Files written with Snappy, gzip, zstd, Brotli and LZ4 all loaded. Google’s list of codecs is GZip, LZO, LZ4_RAW, Snappy and ZSTD, which doesn’t include Brotli; ours loaded anyway, but don’t rely on that.

LimitValue
Size of one load job15 TB (not applied with a reservation)
Files in one load job10,000,000
Columns per table10,000
Load jobs per table per day1,500, failed ones included
Load job run time6 hours
Row size, to avoid resourcesExceeded50 MB

Two input guidelines from Google: use row groups of at least 16 MiB, because smaller ones slow the load, and with more than 100 columns consider a page size below the 1 MiB default.

ORC

ORC loads the same way: bq load --source_format=ORC. Our results:

In the ORC fileBecomes
int, string, float, boolean, date, binaryINTEGER, STRING, FLOAT, BOOLEAN, DATE, BYTES
decimal(10,2)NUMERIC; --decimal_target_types=BIGNUMERIC gave BIGNUMERIC
timestamp (no timezone), nanosecondsTIMESTAMP, microseconds kept; --schema t:DATETIME also works
timestamp with local timezone (TIMESTAMP_INSTANT)fails: Unaccepted ORC type kind: 18
list of stringSTRING in REPEATED mode, with no list.element wrapper
mapREPEATED RECORD with key and value
structRECORD

Unlike Parquet, ORC lists and maps need no flag. Zero rows loads. A column of ORC timestamps with a timezone doesn’t load at all, so write them without one.

What Querylab.io does

The Import tab accepts a .parquet file, reads its schema in the browser, and shows the detected columns before anything is written. When you confirm, it uploads the file unchanged in a load job, together with the schema from the preview, so a column you changed in the preview is the type you get. It does not set parquetOptions, so list inference is off. See Import.

What it costs

A batch load on the shared slot pool is free. You pay to store the table, and for the Parquet file in Cloud Storage while it sits there. See Storage pricing. If you only need to query the files now and then, an external table reads Parquet in place without a load. The rest of the loading workflow, including partition loads, is in Loading data into BigQuery.

Open the playground. It is the real app on public datasets and needs no sign-in.