BigQuery updates
Google's BigQuery release notes since 2011, cut to the changes a SQL user can act on. Entries from August 2026 on also say whether the Querylab.io editor supports the feature.
This page follows Google’s BigQuery release notes back to the first one in 2011 and keeps the entries a SQL user can act on: syntax, functions, INFORMATION_SCHEMA views, model names, editions and pricing. Drivers, SDKs, agents, conversational analytics, pipelines, admin consoles and data transfer connectors are left out. Entries run newest first, and each date links to Google’s note. A launch stage appears only when Google’s note states one.
From August 2026 on, each entry also has a Querylab.io line that says whether the editor knows the feature, such as completions, hover and signature help. Queries run on BigQuery, so this line is about editing help only. Older entries list what BigQuery added and nothing more.
Last updated 18 November 2026, covering Google’s notes through 17 November.
November 2026
- 17 November, GA.
AI.RANK. A table function that sorts the rows of a table or query by how well the text in one column answers a search query. It returns arankand ascorebeside the input columns. BigQuery bills it through the ranking API, not by bytes read. Querylab.io: supported. The editor offersAI.RANKafterAI., completesrank,scoreand the input columns, and shows hover and signature help with the cost note. When the arguments are literals and the editor can see the input’s columns, it reports acolumn_to_rankthat is missing or not aSTRING, an emptyquery, an input column calledrankorscore, and atop_nbelow 1, with BigQuery’s own messages. It does not check themodelname or the argument count and types, and the cost estimate still shows bytes only. The local engine used by Learn does not run it. - 17 November, GA. Reranking in
AI.SEARCH. Withrerank => TRUE,AI.SEARCHreorders the rows it found by how well their text answers the query, the same wayAI.RANKdoes. The result gets arerank_scorecolumn, andoptionstakes arerankerobject withnum_candidatesandmodel. Querylab.io: supported. The editor offersrerank =>withTRUEandFALSE, and shows thererankeroptions, the requirements ofrerank => TRUEand the cost note on hover. When the arguments are literals, it reports with BigQuery’s own messages: an ObjectRef column, aquery_valuethat is empty or is not a string literal or parameter,top_k => 0, arerankerwithoutrerank => TRUE, and anum_candidatesthat does not fittop_k. It judges only a call over a named table, not over a CTE or a subquery, because BigQuery checks that input first. It does not check themodelname, the cap of 10,000,000 candidates (BigQuery checks it only when the query runs), or thererank_scorecolumn, which is not offered in completions yet. - 17 November, GA. Optimized mode for
AI.IFandAI.CLASSIFY. Withoptimization_mode => 'MINIMIZE_COST'and anembeddingscolumn, BigQuery calls the model on a sample of rows and answers the rest with a smaller model it trains on that sample.AI.CLASSIFYcan do this with several labels per row. Querylab.io:AI.IF,AI.CLASSIFYandAI.SCOREknow the new named arguments (examples,embeddings,optimization_mode,max_error_ratio;AI.SCOREonly hasmax_error_ratio). The editor offers the values ofoptimization_modeandoutput_mode, and marks a value BigQuery refuses: an unknownoptimization_modeoroutput_mode, anembeddingsarray written out as a literal, and amax_error_ratiooutside 0 to 1.AI.CLASSIFYwithoutput_modeis typedARRAY<STRING>, andoutput_modetakessingleormulti. The local engine does not run these functions. - 13 November, Preview. MySQL, Oracle and PostgreSQL connections. Federated queries can read from these databases through a BigQuery connection. Querylab.io: no change in the editor.
- 12 November, GA. Model
gemini-4-hydrogen. A new Gemini model for the generative AI functions. Querylab.io: the editor offers the name in completions for theendpointof the generative AI functions and for theENDPOINToption of a remote model inCREATE MODEL. It still accepts any name. - 11 November, GA.
JSON_FLATTEN. Returns every value in aJSONvalue that is not an array, looking inside nested arrays. Querylab.io: supported, with completions, hover and signature help. - 10 November, GA. Graph measures. A graph property can be a measure, an aggregate tied to a key so it does not overcount after joins. You read measures with the
GRAPH_EXPANDtable function and theAGGfunction.GRAPH_EXPANDnow takes a start node, so one graph can hold several fact tables. Querylab.io: the editor parses measures,GRAPH_EXPANDandAGG, and shows hover on measure properties. It offersstart_node =>andflatten_results =>, and the node tables of the graph asstart_nodevalues. When the graph is loaded,SELECT *and column completion over astart_nodecall list only the tables reachable from that node, andflatten_results => FALSElists the start node’s columns and the nestedSTRUCTper joined table.CALL BQ.SHOW_GRAPH_EXPAND_SCHEMAand the five otherCALL BQ.*system procedures get completions, hover and signature help, and a wrong number of arguments, a literal of the wrong type or a value where the output variable goes is marked with BigQuery’s own message. A graph with several roots and nostart_node, and astart_nodethat is not a node table, are marked with BigQuery’s own messages. See Graph queries. - 10 November, GA.
extensionsin graphOPTIONS. Labels and properties of a graph take anextensionsoption, a list of string pairs for custom metadata, besidedescriptionandsynonyms. Querylab.io: supported. The editor parsesOPTIONSafterDEFAULT LABELand after a property, offers the three option names, and reports an unknown option, a value of the wrong type and a repeated option with BigQuery’s own wording. - 10 November, GA. Model
gemini-4-carbon. A new Gemini model for the generative AI functions. Querylab.io: the editor offers the name in completions for theendpointof the generative AI functions and for theENDPOINToption of a remote model inCREATE MODEL. It still accepts any name. - 9 November, Preview. Compute cost attribution. Editions customers can opt in to have billed compute charges attributed to the projects that run queries on a reservation. Querylab.io: no change in the editor.
- 9 November, Preview. Compression functions.
ZSTD_COMPRESScompresses aSTRINGorBYTESvalue with Zstandard.ZSTD_DECOMPRESS_TO_BYTESandZSTD_DECOMPRESS_TO_STRINGturn it back intoBYTESorSTRING. Querylab.io: supported, with completions, hover and signature help. - 6 November, Announcement. Graph queries on on-demand pricing. GQL queries run on on-demand pricing and on Enterprise and Enterprise Plus reservations. Standard edition reservations cannot run them. Querylab.io: supported. Auto routing can send a graph query to on-demand or an Enterprise reservation, never to a Standard one. See Graph queries.
- 3 November. SQL UDFs in continuous queries. Continuous queries can call persistent and temporary SQL user-defined functions. Other kinds of UDF are still not allowed. Querylab.io: no change in the editor.
- 2 November, GA.
AI.PARSE_DOCUMENT. Reads a document with the Document AI layout parser and returns its titles, headings, paragraphs, lists and tables asJSON. Querylab.io: supported, with completions, hover and signature help. See AI functions. - 2 November, Preview.
SECURE_CONTEXT. Returns a value passed in at query time, so a logical view can filter rows by user or tenant. Querylab.io: supported, with completions, hover and signature help. The editor also reports the call when it is used outside a logical view or row access policy definition, as BigQuery does. - 2 November, GA.
APPENDSand continuous queries on Iceberg managed tables. TheAPPENDStable function lists the rows added to an Iceberg managed table in a time range. Continuous queries can read from and write to these tables. Querylab.io: the editor parsesAPPENDSand does not check its arguments.
October 2026
- 27 October, GA.
AI.COUNT_TOKENS. Estimates how many tokens a text string uses. Querylab.io: supported, with completions, hover and signature help. See AI functions. - 26 October, Preview.
REGEXP_EXTRACT_GROUPS. Returns the text matched by each capturing group of a regular expression. Querylab.io: supported, with completions, hover and signature help. - 22 October, GA.
AI.TRANSLATE. Translates text between languages with a translation model or an LLM. Querylab.io: supported, with completions, hover and signature help. See AI functions. - 21 October, GA.
AI.CHUNK_DOCUMENT. A table function that splits documents into text chunks for embeddings. Querylab.io: supported, with completions, hover and signature help. - 15 October, GA. Reusable column aliases. A
SELECTalias can be used by later items in the same list and by theWHEREof the same query. It is off until you turn on the project option. Querylab.io: supported. The editor checks the project option and follows it. See Reusable column aliases. - 14 October, GA.
output_schemaforAI.AGG. Sets the fields of the function’s result as a list of names and types. Querylab.io: supported. - 13 October, GA.
AI.AGG. Combines many rows of unstructured input into one answer, following an instruction in plain language. Querylab.io: supported, with completions, hover and signature help. See AI functions. - 13 October, GA. Model
gemini-4-argon. A new Gemini model for the generative AI functions. Querylab.io: the editor offers the name in completions for theendpointof the generative AI functions and for theENDPOINToption of a remote model inCREATE MODEL. It still accepts any name. - 7 October, Preview.
TimesFM 3.0. A time series model. WithAI.FORECASTit can forecast several series together and use covariates.AI.EVALUATEandAI.DETECT_ANOMALIEStake it for a single series. Querylab.io:'TimesFM 3.0'is offered in completions formodelin all three functions. See Forecasting. - 5 October, GA. Model
gemini-embedding-2. An embedding model forAI.EMBED,AI.SIMILARITYandAI.GENERATE_EMBEDDING. Querylab.io: the editor offers the name in completions for theendpointofAI.EMBEDandAI.SIMILARITY.AI.GENERATE_EMBEDDINGtakes a model, not a name, so there is nothing to offer. The editor still accepts any name. - 5 October, GA. Flexible column names on Iceberg tables. Iceberg external and managed tables allow column names with spaces and non-ASCII characters by default. Querylab.io: supported. Such names are quoted with backticks in completions, in Expand SELECT * and in the SQL converters.
September 2026
- 30 September, GA.
OBJ.LIST. A table function that lists files in Cloud Storage with their metadata andObjectRefvalues, without an object table. Querylab.io: supported, with completions, hover and signature help. See Object tables. - 29 September, GA.
AI.KEY_DRIVERS. Shows which combinations of dimension values explain a change in a metric you can sum. Querylab.io: supported, with completions, hover and signature help. - 29 September, Preview.
INFORMATION_SCHEMA.FAILOVER_HISTORY. Lists failover events for reservations that use managed disaster recovery. It needs a region qualifier. Querylab.io: supported. The editor accepts the view with a region qualifier and flags it without one. See Replication and disaster recovery. - 28 September. Continuous queries can
INSERTinto Iceberg managed tables. A continuous query can write its output rows to an Iceberg managed table with anINSERTstatement. Querylab.io: no change in the editor. - 21 September. Model
gemini-3.8-flash. A new Gemini model for the generative AI functions. Querylab.io: the editor offers the name in completions for theendpointof the generative AI functions and for theENDPOINToption of a remote model inCREATE MODEL. It still accepts any name. - 14 September, Preview.
WHEREinside an aggregate call.SUM(x WHERE x > 0)filters the rows that feed one aggregate. Querylab.io: supported, including withDISTINCT. See Aggregate functions. - 10 September, Preview.
ML.METRICS. Computes classification or regression metrics from a table of actual and predicted values, with no stored model. Querylab.io: supported, with completions, hover and signature help. - 10 September, Preview.
AI.CAUSAL_EFFECT. Estimates how much an event, such as a campaign, changed a time series. Querylab.io: supported, with completions, hover and signature help. - 9 September. Models
gemini-3.5-flash-lite,gemini-3.6-flashandgemini-3.7-flash. Three new Gemini models for the generative AI functions. Querylab.io: the editor offers the names in completions for theendpointof the generative AI functions and for theENDPOINToption of a remote model inCREATE MODEL. It still accepts any name.
August 2026
- 31 August, GA. BigQuery Graph. Graph is generally available. GQL gains the
CALLstatement and the path functionsIS_ACYCLIC,IS_SIMPLEandIS_TRAIL. Querylab.io: supported. Completions offer the three path functions. See Graph queries. - 31 August, Preview. Identity columns.
GENERATED ALWAYS AS IDENTITY (...)andGENERATED BY DEFAULT AS IDENTITY (...)make BigQuery generate a unique integer for each inserted row. Querylab.io: supported, including theSTART WITHandINCREMENT BYoptions. See Defaults and identity columns. - 31 August, Preview. TabFM. A pretrained model for tabular data.
AI.PREDICTandAI.EVALUATEselect it withmodel => 'TabFM 1.0'. Querylab.io:'TabFM 1.0'is offered in completions formodel. - 27 August, GA.
XGBOOST_VERSION. ACREATE MODELoption that picks the XGBoost version (0.9,1.1or2.1) for boosted tree and random forest models. Querylab.io: supported. The option appears in completions. See BigQuery ML. - 27 August, Announcement. Graph processing and editions. Google said core graph processing needs an Enterprise or Enterprise Plus reservation, and that allowlisted users could keep Standard edition or on-demand pricing until 26 April 2027. Graph measures stay available on on-demand pricing. On-demand pricing is allowed for all graph queries since 6 November, see above. The same rule was first announced on 20 August. See On-demand vs capacity.
- 20 August, Preview.
ML.TREND,ML.SEASONALITYandML.DETECT_CHANGE_POINTS. Table functions that report the trend, the seasonal pattern and the points where a time series changes level or slope. Querylab.io: supported, with completions, hover and signature help. - 11 August, GA.
ANY TABLEparameters. A table function can take a table of any shape as a parameter. Querylab.io: supported. See Table functions. - 10 August, GA. Models
gemini-3.1-flash-liteandgemini-3.5-flash. Two Gemini models for the generative AI functions. Querylab.io: the editor offers the names in completions for theendpointof the generative AI functions and for theENDPOINToption of a remote model inCREATE MODEL. It still accepts any name. - 3 August, Announcement. Hybrid search restored.
VECTOR_SEARCHcan combine semantic and keyword search, andAI.SEARCHacceptsHYBRIDmode. Querylab.io: supported, including the lexical arguments ofVECTOR_SEARCH. See Vector search.
July 2026
- 27 July, GA.
APPENDSandCHANGES. The change history functions return the rows that were appended to or changed in a table during a time range. - 15 July, Preview. Data governance tags. A Resource Manager tag can be attached to sensitive columns. Data policies use the tag to enforce column-level security and data masking.
- 13 July, GA. Iceberg managed tables. Table partitioning, multi-statement transactions and advanced runtime work on Apache Iceberg managed tables.
- 13 July, Preview. Project caps. A reservation can limit the maximum slots and the concurrency of each project in it. Google also calls them scheduling policies.
- 13 July, Preview.
ALTER SEARCH INDEX. Changes the configuration of an existing search index. - 9 July. Hybrid search in
VECTOR_SEARCHdisabled. Google turned off the combined semantic and keyword search for now and says it is working to restore it. - 8 July, Preview. Multi-level aggregation. An aggregate function can be used as an argument of another aggregate function in GoogleSQL.
June 2026
- 30 June.
principalon reservation assignments. An optionalprincipalproperty routes queries to a reservation based on the user, service account or third-party identity that runs the job. - 25 June, Preview. Hybrid search in
VECTOR_SEARCH.VECTOR_SEARCHcan combine a semantic search with a keyword search.AI.SEARCHtakes aHYBRIDmode on tables with autonomous embedding generation, and a vector index can include keyword information to speed up the keyword part. - 25 June, Preview. DDL and DML on Lakehouse Iceberg tables. Apache Iceberg tables in the Lakehouse runtime catalog accept
CREATE,ALTERandDROPstatements andINSERT,UPDATE,DELETEandMERGEfrom BigQuery. - 17 June, GA. Autonomous embedding generation.
CREATE TABLEandALTER TABLEcan turn on an embedding column that BigQuery fills and updates from a source column. - 15 June. Daily token quotas turned off. Configuring daily token quotas for the generative AI functions is disabled for now. Google says it is working to restore it.
- 12 June, GA.
ObjectRefinput for AI functions. The AI functions takeObjectRefvalues directly, without a call toOBJ.GET_ACCESS_URL. - 11 June, Preview.
AI.KEY_DRIVERSrestored. The function finds the data segments that cause statistically significant changes in a summable metric. - 10 June, Preview.
ARRAY_AGGandSTRING_AGGin continuous queries. Continuous queries support both aggregate functions. - 8 June, GA. Daily token quotas. Daily token quotas limit the cost of the generative AI functions.
- 3 June, GA. Fluid scaling. Autoscaling reservations are billed per second with no minimum duration.
- 2 June, GA. Custom path in remote function endpoints. Several remote functions can share one Cloud Run service by using different path suffixes on the same endpoint URL.
May 2026
- 20 May, GA. Python UDFs. A scalar function written in Python can be called from a SQL query. It can install packages from PyPI and reach external services through a Cloud resource connection.
- 20 May, Preview.
AI.AGG. Aggregates unstructured input data by following an instruction in natural language. - 18 May, GA. Reservation groups. Reservations in a group share idle slots with each other before the slots go to other reservations in the project.
- 15 May, GA. Single input table for
AI.DETECT_ANOMALIES. The function takes one table that holds both the historical and the target data. - 14 May.
AI.KEY_DRIVERSturned off. The Preview of the function is disabled for now. Google says it is working to restore it. - 12 May, Preview.
AI.COUNT_TOKENS. Estimates the token count of a text input. For some generative AI functions, a query can also show the input, output, thought and cache tokens for each modality.
April 2026
- 29 April, GA.
PARTITION BYonCREATE VECTOR INDEX. Partitions a TreeAH vector index, which allows partition pruning and can lower I/O cost. - 29 April, GA.
VECTOR_INDEX.STATISTICSandALTER VECTOR INDEX REBUILD.VECTOR_INDEX.STATISTICSmeasures how far the table data has drifted since the index was created.ALTER VECTOR INDEX REBUILDrebuilds the index without downtime. - 28 April, GA. Materialized views over CDC tables. A materialized view can be created over a table with active change data capture.
- 22 April, Preview. Graph measures. Some graphs can define measures that lock an aggregation to a key, which avoids overcounting. The
GRAPH_EXPANDtable function flattens the graph into a table, and theAGGfunction queries the measures there. Labels and properties of a graph can also carry descriptions and synonyms. - 22 April, Preview. Model
gemini-embedding-2-preview. Works inAI.EMBED,AI.SIMILARITYandAI.GENERATE_EMBEDDINGand builds one embedding from a mix of text, image, audio, video and PDF input. - 20 April, Preview. Python UDF options and cost column.
CREATE FUNCTIONtakes acontainer_request_concurrencyoption that caps concurrent requests per Python UDF container instance. Python UDF cost appears in theexternal_service_costscolumn ofINFORMATION_SCHEMA.JOBS. Quotas apply: 10 GiB image storage and 30 mutations per minute, per project per region. - 16 April, Preview.
AI.KEY_DRIVERS. Identifies the segments of data that cause statistically significant changes in a summable metric. - 15 April, Preview.
EXPORT DATAto AlloyDB.EXPORT DATAstatements can reverse ETL BigQuery data into AlloyDB. - 15 April, Preview. Iceberg version 3 external tables. Apache Iceberg external tables support Iceberg version 3, including binary deletion vectors.
- 13 April.
AI.AGGturned off. The Preview of the function is disabled for now. Google says it is working to restore it. - 13 April, Preview. Optimized mode for
AI.IFandAI.CLASSIFY. Optimized mode cuts LLM token use and query latency on large datasets. - 13 April, GA.
AI.IF,AI.SCOREandAI.CLASSIFY.AI.IFfilters or joins text and unstructured data on a condition written in natural language.AI.SCORErates data to rank it.AI.CLASSIFYsorts it into categories you define. - 9 April, Preview. Stateful continuous queries. Continuous queries can keep information across rows or time intervals with
JOINs and windowing aggregations. - 9 April, Preview. BigQuery Graph. A graph is created from existing tables that hold entities and relationships, and Graph Query Language (GQL) queries it.
- 8 April, GA.
@@session_idin routines and views. The system variable works in SQL user-defined functions, table functions and logical views. - 7 April, Preview. Model
embeddinggemma-300m. A built-in text embedding model forAI.EMBEDandAI.SIMILARITY. It generates embeddings on your BigQuery slots. - 6 April, Preview.
AI.AGG. Aggregates unstructured input data by following an instruction in natural language. - 2 April, GA.
CREATE CONNECTION,ALTER CONNECTION SET OPTIONSandDROP CONNECTION. DDL statements manage Cloud resource connections.GRANTandREVOKEalso take theconnectionuser type and thePROJECTresource type. - 2 April, GA. Column granularity for search indexes.
CREATE SEARCH INDEXcan store extra column information in the index to speed up search queries.
March 2026
- 31 March, GA.
ObjectRefupdates. TheObjectReffunctions run with direct or delegated access.OBJ.MAKE_REFfetches the latest Cloud Storage metadata intoref.details.OBJ.GET_READ_URLreturns aSTRUCTwith a read URL and status columns. - 30 March, GA. Forecasting and anomaly detection updates.
AI.DETECT_ANOMALIESandAI.EVALUATEtake a custom context window of recent data points.AI.FORECASTtakes the latest timestamp to forecast from.AI.EVALUATEalso returns the mean absolute scaled error. - 30 March, GA. Materialized views over Spanner data. Non-incremental materialized views cache the results of queries on Spanner data.
- 26 March, GA.
EXPORT DATAto Spanner with a Cloud resource connection. The statement reverse ETLs BigQuery data into Spanner through the connection. - 23 March, GA.
AI.EMBEDandAI.SIMILARITY.AI.EMBEDcreates embeddings from text or images.AI.SIMILARITYcomputes the semantic similarity between pairs of text, pairs of images, or text and images. - 17 March, GA. Automatically deployed open models. BigQuery ML can deploy open models to Vertex AI endpoints, manage the Vertex AI resources, reserve capacity with Compute Engine reservations, and undeploy the model automatically or immediately.
- 16 March, GA. Global default location. The default location applies when a request sets no location and none can be inferred. It can be set at the organization or project level.
- 12 March. Advanced runtime is the default. Advanced runtime is turned on for all projects.
- 6 March, GA. Embedding remote models. A remote model can be based on Vertex AI
gemini-embedding-001, or on an open embedding model from Model Garden or Hugging Face deployed to Vertex AI.AI.GENERATE_EMBEDDINGuses these models, andAI.EMBEDcan call thegemini-embedding-001endpoint directly. - 5 March, Preview. Single-vector syntax for
VECTOR_SEARCH. An alternate call syntax speeds up a search for one vector. - 4 March, GA. Continuous queries to Spanner. A continuous query can stream BigQuery data to Spanner as it arrives.
February 2026
- 25 February. Legacy SQL limited from 1 June 2026. An organization or project that does not use legacy SQL between 1 November 2025 and 1 June 2026 cannot use it afterwards. Those that do use it keep their existing workloads, and new ones might not run.
- 23 February, GA. Undelete a dataset. A dataset inside its time travel window can be restored to the state it had when it was deleted.
- 17 February, Preview. Global queries. One query can reference data stored in more than one region.
- 12 February, GA. Field descriptions in
AI.GENERATEandAI.GENERATE_TABLE. A custom output schema can carry a description for each field. - 12 February, Preview. Multiple categories in
AI.CLASSIFY. The function can put an input into more than one category. - 4 February, GA. Data policies on columns. A data policy can be attached directly to a column to control access and apply masking or transformation rules.
January 2026
- 29 January.
RANDOM_HASHmasking rule. A predefined masking rule that returns a salted hash of the column value. Google describes it as stronger than theHash (SHA-256)rule. - 23 January.
endpointargument forAI.IF,AI.SCOREandAI.CLASSIFY. An optionalendpointargument picks the model. - 22 January. End-user credentials for
AI.IF,AI.SCOREandAI.CLASSIFY. The functions can run with your end-user credentials instead of a BigQuery connection. - 22 January. Table parameters in table-valued functions. Support was restored.
- 6 January, GA. Options for
CREATE EXTERNAL TABLEandLOAD DATA.time_zonesets the time zone used when loading.date_format,datetime_format,time_formatandtimestamp_formatdefine how date and time values look in the source files.null_markerssets the strings that meanNULLin CSV files.source_column_matchmatches loaded columns to the schema by position or by name.
2025
- 2 December 2025, Preview. Autonomous embedding generation. A table made with
CREATE TABLEcan keep a column of embeddings based on a source column. BigQuery generates or updates the embeddings when the source column changes.AI.SEARCHruns semantic search on such tables. - 19 November 2025, Preview.
JSON_FLATTEN. Extracts all non-array values that sit directly in aJSONvalue or in arrays nested inside it. - 18 November 2025. Gemini 3.0 in generative AI functions. Functions such as
AI.GENERATEacceptgemini-3-pro-preview. The call needs the full global endpoint argument,https://aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/global/publishers/google/models/gemini-3-pro-preview. - 18 November 2025, GA.
AI.GENERATE. Generates free text from text, images, audio, video or documents, and can extract entities or return structured output.AI.GENERATE_BOOL,AI.GENERATE_DOUBLEandAI.GENERATE_INTreturn scalar values for filtering, scoring and counting. These functions support authentication with end-user credentials. - 18 November 2025, Preview.
AI.EMBEDandAI.SIMILARITY.AI.EMBEDturns text, images, audio, video or documents into embeddings.AI.SIMILARITYcomputes semantic similarity between pairs of text, pairs of images, or text and images. - 18 November 2025, GA.
AI.GENERATE_TABLE,AI.GENERATE_TEXTandAI.GENERATE_EMBEDDING.AI.GENERATE_TABLEreturns a table of structured output from unstructured data.AI.GENERATE_TEXTandAI.GENERATE_EMBEDDINGare the preferred versions ofML.GENERATE_TEXTandML.GENERATE_EMBEDDING, with the same functionality and simpler output column names. - 10 November 2025.
TimesFM 2.5. The time series model works inAI.FORECAST,AI.EVALUATEandAI.DETECT_ANOMALIES, with better forecasting accuracy and lower latency. - 10 November 2025, Preview.
AI.DETECT_ANOMALIES. Finds anomalies in time series data with a TimesFM model, using historical data as the baseline. - 10 November 2025, Preview. Partitioning for BigLake tables for Apache Iceberg in BigQuery.
- 5 November 2025, GA.
MATCH_RECOGNIZE. A clause that filters and aggregates matches across rows in a table. - 29 October 2025, Preview. Reservation groups. Reservations in a group share idle slots with each other before the slots go to other reservations in the project.
- 28 October 2025, GA.
job_principal_subjectinINFORMATION_SCHEMA.SHARED_DATASET_USAGE. Holds the principal of the user who ran a job or query against a linked dataset, when subscriber email logging is on for the listing or data exchange. - 27 October 2025, Preview.
AI.IF,AI.SCOREandAI.CLASSIFY. Managed AI functions that use Gemini.AI.IFfilters and joins text or multimodal data by a condition written in natural language.AI.SCORErates input to rank data.AI.CLASSIFYsorts text into categories you define. - 22 October 2025, GA. Built-in TimesFM model.
AI.FORECASTandAI.EVALUATEcan use the built-in TimesFM univariate model.AI.FORECASTsupports a larger context window. - 22 October 2025, Issue. Table parameters in table-valued functions. Support was temporarily disabled.
- 20 October 2025, Preview. BigQuery-managed open models. Open models run as Vertex AI endpoints that BigQuery manages. SQL queries manage the Vertex AI resources, and the models can be undeployed automatically or immediately.
- 9 October 2025, GA. Maximum slot limit for a reservation. The maximum size is set when the reservation is created or updated.
- 9 October 2025, GA. Fair idle slot allocation. Idle slots are shared about equally across the reservations in one admin project.
- 8 October 2025, Breaking change. Default
QueryUsagePerDaylimit. The default limit for on-demand pricing is 200 TiB for all new projects. Existing projects got a default based on their usage over the last 30 days. Projects with custom cost controls or reservations are not affected. - 8 October 2025, GA. Reservation chosen at query time. A query can name the reservation it uses at runtime, and IAM policies can be set directly on reservations.
- 8 October 2025, GA. Labels on reservations. Labels organize reservations and help with billing analysis.
- 6 October 2025, GA. Columns added to
INFORMATION_SCHEMA.SHARED_DATASET_USAGE.shared_resource_idandshared_resource_typedescribe the queried resource.referenced_tablesholdsproject_id,dataset_id,table_idandprocessed_bytesof the base table. They report usage of external tables and routines. - 29 September 2025, Preview. Non-incremental materialized views over Spanner data. The view caches results periodically to speed up queries.
- 29 September 2025, Announcement. History-based query optimizations. They are on by default. A project or organization that turned them off can turn them back on.
- 25 September 2025, GA.
ARRAY_FIRST,ARRAY_LASTandARRAY_SLICE. - 22 September 2025, GA. Federated queries to Spanner PostgreSQL dialect databases. They run through BigQuery external datasets with GoogleSQL, including cross-region federated queries.
- 16 September 2025, GA.
FOR SYSTEM_TIME AS OFon Iceberg external tables. The clause reads snapshots that are kept in the Iceberg metadata. - 16 September 2025, GA.
JSON_KEYSand JSONPath modes.JSON_KEYSextracts the unique keys of a JSON expression. Some JSON functions that take a JSONPath accept a mode that sets how the path matches the JSON structure. - 15 September 2025, GA. BigQuery advanced runtime. It improves query execution time and slot usage. Between 15 September 2025 and early 2026 it becomes the default runtime for all projects.
- 9 September 2025. Supervised tuning of
gemini-2.5-proandgemini-2.5-flash-literemote models. It applies to BigQuery ML remote models based on those Vertex AI models. - 8 September 2025, Preview. Table parameters in table-valued functions. A TVF can take a table as a parameter.
- 3 September 2025, GA.
per_second_detailsinINFORMATION_SCHEMA.RESERVATIONS_TIMELINE. Reports reservation capacity and usage per second, including autoscale utilization. - 2 September 2025, Preview.
gemini-embedding-001remote model.ML.GENERATE_EMBEDDINGgenerates embeddings with a remote model based on it. - 2 September 2025, Preview. Remote models on open embedding models. The model comes from Vertex Model Garden or Hugging Face and is deployed to Vertex AI, such as E5 Embedding.
ML.GENERATE_EMBEDDINGgenerates embeddings with it. - 25 August 2025, GA.
ST_REGIONSTATS. Combines raster data from Earth Engine with vector data stored in BigQuery. - 22 August 2025, Preview. Multi-statement transactions on BigLake Iceberg tables in BigQuery.
- 14 August 2025, GA. Cross region federated queries to Spanner. They query Spanner tables from regions other than the BigQuery region and incur Spanner network egress charges.
- 11 August 2025, GA.
WITHexpressions. Create temporary variables inside a GoogleSQL expression. - 11 August 2025, GA. Chained function call syntax. Makes deeply nested function calls easier to read.
- 6 August 2025, Preview. Short query optimizations. Turning on the advanced runtime includes them.
- 31 July 2025. Higher throughput for generative AI functions. Throughput of
ML.GENERATE_TEXT,AI.GENERATE_TABLE,AI.GENERATE,AI.GENERATE_BOOL,AI.GENERATE_DOUBLEandAI.GENERATE_INTimproved by more than 100x. A typical 6-hour job can process millions of rows. - 31 July 2025, Preview. Continuous queries export to Spanner. A continuous query can export BigQuery data to Spanner in real time.
- 28 July 2025, Preview. Data policies on columns. A data policy can be attached directly to a column to control access and apply masking and transformation rules.
- 22 July 2025, Preview.
VECTOR_INDEX.STATISTICS. Calculates how far an indexed table’s data has drifted since the vector index was created.ALTER VECTOR INDEX REBUILDrebuilds the index when the drift is large enough. - 22 July 2025, Preview.
null_markersandsource_column_matchoptions.CREATE EXTERNAL TABLEandLOAD DATAaccept both.null_markersdefines the strings that stand forNULLin CSV files.source_column_matchmatches loaded columns to the schema by position or by name. - 22 July 2025, Preview.
MATCH_RECOGNIZE. A clause that filters and aggregates matches across rows in a table. - 21 July 2025, GA.
DISTINCTpipe operator. Selects distinct rows in pipe syntax queries. - 17 July 2025, GA.
WITHpipe operator and named windows. TheWITHpipe operator defines common table expressions in pipe syntax queries. Named windows also work in pipe syntax queries. - 8 July 2025, Announcement. GoogleSQL as the default dialect for the CLI and API. From 1 August 2025 queries run from the command line or the API use GoogleSQL unless the request asks for legacy SQL. Legacy SQL can also be chosen with the
default_sql_dialect_optionsetting set to'default_legacy_sql'at project or organization level. - 1 July 2025, GA.
@@location. The system variable sets the location in which a query runs. - 26 June 2025, Preview.
PARTITION BYinCREATE VECTOR INDEX. Partitions TreeAH vector indexes. Queries can then prune partitions, which can lower I/O costs. - 26 June 2025, GA.
INFORMATION_SCHEMA.SEARCH_INDEXES_BY_ORGANIZATION. Shows how much of the free search index management limit the organization has used, by project and table. - 16 June 2025, GA.
TIME_SERIES_ID_COLforARIMA_PLUS_XREG. The option forecasts several time series at once with a multivariate model. - 16 June 2025, GA. IAM tags in SQL. IAM tags on datasets and tables can be managed with SQL.
- 11 June 2025, Preview.
ARRAY_FIRST,ARRAY_LASTandARRAY_SLICE.ARRAY_FIRSTreturns the first element of an array andARRAY_LASTthe last.ARRAY_SLICEreturns consecutive elements. - 10 June 2025, GA. Provisioned Throughput for
ML.GENERATE_TEXTandAI.GENERATE. Vertex AI Provisioned Throughput gives these functions consistent high throughput on supported Gemini models. - 9 June 2025, GA. Iceberg external tables in materialized views. A materialized view can reference them, so the data does not have to move into BigQuery storage.
- 4 June 2025. Organization-level
default_sql_dialect_optionandquery_runtime. These configuration settings are no longer supported at organization level. - 3 June 2025, Preview. BigQuery advanced runtime. It improves query execution time and slot usage.
- 2 June 2025, Preview. Date and time options for
CREATE EXTERNAL TABLEandLOAD DATA.time_zonesets the time zone used when loading data.date_format,datetime_format,time_formatandtimestamp_formatdefine how values are formatted in the source files. - 27 May 2025, GA. Optional job creation mode. Speeds up small queries in dashboards and data exploration. Eligible queries are optimized automatically and use a cache to cut latency.
- 22 May 2025.
INFORMATION_SCHEMA.OBJECT_PRIVILEGESpermission. From 17 March 2026 viewing a dataset’s access controls and querying this view needsbigquery.datasets.getIamPolicy. - 21 May 2025. Supervised tuning of
gemini-2.0-flash-001andgemini-2.0-flash-lite-001remote models. It applies to BigQuery ML remote models based on those Vertex AI models. - 19 May 2025, GA. Continuous queries. Long-lived SQL statements that analyze and process incoming data, and run ML inference on it, in real time. They can use slot autoscaling and a custom job ID prefix.
- 19 May 2025, Preview. Cross regional federated queries to Spanner. Spanner tables in regions other than the BigQuery region can be queried. There are no Spanner network egress charges during the preview.
- 14 May 2025, GA. TreeAH vector index. Vector indexes support the TreeAH index type, which uses Google’s ScaNN algorithm. It suits batch processing of a few thousand to hundreds of thousands of embeddings at once.
- 14 May 2025, GA. Cross-region load and export. A single
LOAD DATAorEXPORT DATAstatement can move data between any two regions or multi-regions. - 13 May 2025, GA.
GROUP BY STRUCT,GROUP BY ARRAYandGROUP BY ALL.STRUCTandARRAYvalues can be grouped and used withSELECT DISTINCT.GROUP BY ALLgroups by all non-aggregated select items. - 12 May 2025, Preview. Multimodal data with
ObjectRef.ObjectRefvalues bring unstructured data into standard tables, andObjectRefRuntimevalues work with it in queries. BigQuery ML SQL functions with Gemini and multimodal embedding models analyze it. Python UDFs can transform images and chunk PDF files. - 23 April 2025, Preview. Reservation features. A query can name the reservation it uses at runtime, and IAM policies can be set on reservations. A reservation can have a maximum slot limit. Idle slots can be allocated fairly across reservations in one admin project.
- 21 April 2025, Announcement. Spend-based committed use discounts. A commitment to spend a minimum amount per hour on pay-as-you-go compute earns a discount. The commitment period is one or three years.
- 9 April 2025, Preview.
ST_REGIONSTATS. Combines raster and vector data for geospatial analysis. - 8 April 2025, Preview. Built-in TimesFM model.
AI.FORECASTcan forecast with the built-in TimesFM univariate model, with no model to create or train. - 7 April 2025, GA. Llama and Mistral AI remote models.
ML.GENERATE_TEXTworks with remote models based on Llama and Mistral AI models in Vertex AI. - 7 April 2025, GA. Smart-tuning for materialized views. It applies when the view is in the same project as one of its base tables, or in the project running the query.
- 7 April 2025. Token-based batching for embedding generation. BigQuery ML puts as many rows as possible into one embedding request, with an average 10x improvement against query-per-minute quotas.
- 4 April 2025, Preview.
AI.GENERATE,AI.GENERATE_BOOL,AI.GENERATE_INTandAI.GENERATE_DOUBLE. Analyze text with a Vertex AI Gemini model. The output matches the type in the function name. - 3 April 2025, Preview.
AI.GENERATE_TABLEandoutput_schema. Returns structured data from Gemini 1.5 Pro, Gemini 1.5 Flash and Gemini 2.0 Flash. Theoutput_schemaargument takes a SQL schema like the one inCREATE TABLE. - 2 April 2025, Preview. Python UDFs. User-defined functions written in Python can use additional libraries and external APIs.
- 1 April 2025, GA. Contribution analysis models.
CREATE MODELbuilds a contribution analysis model, with thetop_k_insights_by_apriori_supportandpruning_methodoptions.ML.GET_INSIGHTSreports changes in key metrics for summable, summable ratio and summable by category metrics. - 1 April 2025, GA. Pipe syntax. A linear query structure meant to make queries easier to read, write and maintain.
- 31 March 2025, GA. Merge-on-read for Iceberg external tables. Queries can read Iceberg tables with position deletes and equality deletes.
- 31 March 2025, GA. Subqueries in row-level access policies. They work with BigLake managed tables and the BigQuery Storage Read API too.
- 31 March 2025, GA.
_CHANGE_SEQUENCE_NUMBER. Sets the order of streamingUPSERTchanges in BigQuery change data capture. - 26 March 2025, Preview. Column granularity for search indexes. Set when the search index is created, it stores more column information in the index to speed up search queries.
- 24 March 2025, Preview. KLL quantile functions. Compute approximate quantiles efficiently.
- 24 March 2025, Preview. Labels on reservations. Labels organize reservations and help with billing analysis.
- 20 March 2025, GA. Anthropic Claude remote models.
ML.GENERATE_TEXTworks with remote models based on Claude in Vertex AI, andML.EVALUATEcan evaluate them. - 17 March 2025, GA.
EXPORT DATAto Spanner. Reverse ETL from BigQuery to Spanner. - 17 March 2025, GA. Spanner external datasets. An external dataset in BigQuery links to an existing Spanner database.
- 17 March 2025, GA.
TYPEOF. Returns the data type of an expression. - 3 March 2025, GA. SQL
CREATE AGGREGATE FUNCTION. Creates a SQL user-defined aggregate function. - 24 February 2025, Preview.
@@location. The system variable sets the location in which a query runs. - 6 February 2025, GA. JavaScript
CREATE AGGREGATE FUNCTION. Creates a JavaScript user-defined aggregate function. - 3 February 2025, GA.
BY NAMEandCORRESPONDING. Set operations match columns by name instead of by position. - 3 February 2025.
output_formatforML.BUCKETIZEandML.QUANTILE_BUCKETIZE. The argument formats the output as a string likebin_<bucket_index>, as a string in interval notation, or as a JSON-formatted string. - 27 January 2025, GA. Remote models on open models. The model comes from Vertex Model Garden or Hugging Face and is deployed to Vertex AI, such as Llama and Gemma.
ML.GENERATE_TEXTandML.EVALUATEwork with these remote models. - 21 January 2025, Preview.
ML.EVALUATEfor Claude models. Evaluates Anthropic Claude remote models. The Claude quotas in BigQuery ML match the Vertex AI quotas. - 13 January 2025, Preview.
TIME_SERIES_ID_COLforARIMA_PLUS_XREG. The option forecasts several time series at once with a multivariate model.
2024
- 11 December 2024, Preview. Model
gemini-2.0-flash-exp. A remote model can be based on this Vertex AI model.ML.GENERATE_TEXTruns it on text in tables and on image, video, audio, PDF or text content in object tables. - 11 December 2024, GA. Cross-region dataset replication. A dataset can be replicated from its source region to one or more other regions.
- 11 December 2024, GA. Managed disaster recovery. Gives managed failover and redundant compute capacity for a total region outage. It needs the Enterprise Plus edition.
- 19 November 2024, GA. Search indexes on
INT64andTIMESTAMPcolumns.CREATE SEARCH INDEXaccepts these columns, and BigQuery can optimize predicates that use them. - 14 November 2024, GA. Models
gemini-1.5-flashandgemini-1.5-pro. Remote models can be based on these Vertex AI models.ML.GENERATE_TEXTruns them on text in tables and on image, video, audio, PDF or text content in object tables. - 11 November 2024, GA. Tuning and evaluating LLMs. A remote model based on Gemini 1.5 flash or pro can be tuned with supervised tuning.
ML.EVALUATEevaluates pre-trained PaLM and Gemini models and tuned Gemini models. - 14 October 2024, Preview. Fine-grained DML. Optimizes the execution of
UPDATE,DELETEandMERGEstatements on tables. - 10 October 2024, Preview. Tables for Apache Iceberg. BigQuery storage optimization applies to Iceberg tables that stay in your own Cloud Storage buckets, so the data does not move.
- 8 October 2024, Preview. Pipe syntax. A linear query structure that can be used anywhere GoogleSQL is written.
- 3 October 2024, Preview. Spanner external datasets. An external dataset links BigQuery to an existing Spanner database.
- 30 September 2024, GA. History-based optimizations. They can be turned on and off, and their effect on queries can be analyzed.
- 30 September 2024, GA. Flexible column names. Tables and views accept more characters in column names for extracting, loading, streaming and querying.
- 24 September 2024, GA.
ML.GENERATE_EMBEDDINGfor more models. It works with a remote model based on Vertex AImultimodalembeddingto embed text, images and video in one space. It also works with PCA, autoencoder and matrix factorization models. - 24 September 2024, GA.
ML.PROCESS_DOCUMENTandML.TRANSCRIBE.ML.PROCESS_DOCUMENTprocesses documents in object tables with a remote model based on Document AI.ML.TRANSCRIBEtranscribes audio files in object tables with a remote model based on Speech-to-Text. - 19 September 2024, GA. Model monitoring functions.
ML.DESCRIBE_DATA,ML.VALIDATE_DATA_SKEW,ML.VALIDATE_DATA_DRIFT,ML.TFDV_DESCRIBEandML.TFDV_VALIDATEcompute statistics for training and serving data and compare them to find anomalous differences. - 16 September 2024, Preview. Contribution analysis models.
CREATE MODELcreates a contribution analysis model.ML.GET_INSIGHTSuses it to report insights about changes to key metrics in multi-dimensional data. - 16 September 2024, GA. Stored columns in vector indexes. Columns stored in a vector index can be used to pre-filter
VECTOR_SEARCH. - 4 September 2024, GA.
VECTOR_SEARCHand vector indexes.VECTOR_SEARCHsearches embeddings for semantically similar entities. A vector index makes it more efficient and returns more approximate results. - 28 August 2024, GA. Phrase search in
SEARCH.SEARCHsupports phrase matching. - 28 August 2024, Preview.
GROUP BYandSELECT DISTINCTonARRAYandSTRUCT. Both clauses accept columns of these types. - 28 August 2024, GA. AlloyDB federated queries. A federated query can read data in AlloyDB.
- 26 August 2024, Preview. Anthropic Claude models. A remote model can be based on a Claude model in Vertex AI.
ML.GENERATE_TEXTruns it on text in tables. - 26 August 2024, GA.
EXPORT DATAto Bigtable.EXPORT DATAwrites BigQuery data directly to Bigtable. - 20 August 2024, GA. Anomaly detection with
ARIMA_PLUS_XREG. Multivariate time series models can detect anomalies in historical data or in new data with several feature columns. - 19 August 2024, Preview.
INSIGHTS,RECOMMENDATIONSandRECOMMENDATIONS_BY_ORGANIZATION. TheseINFORMATION_SCHEMAviews list BigQuery insights and recommendations. - 14 August 2024, Preview. Short query optimized mode. Lowers latency for small queries. BigQuery decides which queries to accelerate, and the rest run as before.
- 12 August 2024, GA. Time series and range functions. They support time series analysis.
- 8 August 2024, Preview.
JSON_KEYS. Extracts the unique keys from aJSONexpression. - 8 August 2024, Preview. JSONPath modes. Some JSON functions that take a JSONPath accept a mode that sets how strictly the path matches the JSON structure.
- 7 August 2024, Preview. Materialized views over Iceberg tables. The view can be partition aligned with the base table. Only time-based partition transforms are supported:
YEAR,MONTH,DAYandHOUR. - 31 July 2024, GA. Autoscaler changes. The autoscaler scales up immediately and to the nearest multiple of 50 slots instead of 100. Capacity commitments, baseline slots and autoscale max slots are set in steps of 50. After one minute or more since the last increase, capacity can be reduced several times in a row without waiting a minute between decreases.
- 30 July 2024, Preview.
output_dimensionalityforML.GENERATE_EMBEDDING. Sets the number of dimensions of the embeddings when the function uses a remote model based on Vertex AImultimodalembedding. - 29 July 2024, GA.
RANGEas a JSON encoding. TheRANGEtype can be encoded toJSON. - 29 July 2024, Preview. TreeAH vector index. A vector index can use the TreeAH type, which uses Google’s ScaNN algorithm. It suits batch queries with hundreds or more query vectors.
- 22 July 2024, Preview. Continuous queries. Long-lived SQL statements that process incoming data in BigQuery in real time and can run ML inference on it. They can write results to a Pub/Sub topic, a Bigtable instance or another BigQuery table.
- 22 July 2024, GA. DML on recently streamed rows.
UPDATE,DELETEandMERGEcan modify rows that the Storage Write API wrote recently. - 22 July 2024, Preview.
CHANGES. A table function that returns the history of changes to a table over a window of time. It coversCREATE TABLE,INSERT,MERGE,UPDATE,DELETEandTRUNCATE TABLE, loads, streaming ingestion,WRITE_TRUNCATEjobs and partition deletions. - 17 July 2024, GA. Default storage billing model. The default storage billing model for new datasets can be configured.
- 11 July 2024, Preview.
EXPORT DATAto Spanner.EXPORT DATAwrites BigQuery data to Spanner. - 18 June 2024, GA. Collation in
NULLIF.NULLIFfollows collation and can be used in collation-aware comparisons withSTRUCT. - 17 June 2024, Preview. Tuning
gemini-1.0-pro-002. A remote model based on this model supports supervised tuning. - 5 June 2024, GA. Slot recommender. Analyzes historical usage and recommends capacity to buy for edition and on-demand workloads.
- 5 June 2024.
output_dimensionalityfortext-embeddingmodels.ML.GENERATE_EMBEDDINGaccepts it fortext-embeddingandtext-multilingual-embeddingmodels to set the number of dimensions. - 29 May 2024. Partition limit raised. A partitioned table can have 10,000 partitions, up from 4,000.
- 28 May 2024, Preview. Model
gemini-1.5-flash. A remote model can be based on this Vertex AI model.ML.GENERATE_TEXTruns it on text in tables and on image, video, audio, PDF or text content in object tables. - 23 May 2024.
FORECAST_LIMIT_LOWER_BOUNDandFORECAST_LIMIT_UPPER_BOUNDwithTIME_SERIES_ID_COL. These options of univariate time series models work together withTIME_SERIES_ID_COL. - 23 May 2024. Grounding and safety settings in
ML.GENERATE_TEXT. For Vertex AI Gemini models,ground_with_google_searchlets the model use information from the internet.safety_settingssets the attributes the model uses to filter responses. - 23 May 2024, Preview. Video embeddings.
ML.GENERATE_EMBEDDINGcreates multimodal embeddings that include video with a remote model based on Vertex AImultimodalembedding. - 22 May 2024, Preview. AlloyDB federated queries. A federated query can read data in AlloyDB.
- 21 May 2024, Preview. Model
gemini-1.5-pro. A remote model can be based on this Vertex AI model.ML.GENERATE_TEXTruns it on text in tables and on image, video, audio, PDF or text content in object tables. - 20 May 2024, GA. DLP functions. They encrypt and decrypt data between BigQuery and Sensitive Data Protection using AES-SIV.
- 20 May 2024, Preview. Search indexes on
INT64andTIMESTAMPcolumns. A search index can optimize lookups on these types. - 13 May 2024, Preview. Phrase search in
SEARCH.SEARCHsupports phrase matching. - 7 May 2024, Preview. JavaScript user-defined aggregate functions.
CREATE AGGREGATE FUNCTIONcreates a JavaScript UDAF. - 7 May 2024, Preview. Stored columns in vector indexes. Columns stored in a vector index can be used to pre-filter
VECTOR_SEARCH. - 6 May 2024, Preview. Managed disaster recovery. Gives managed failover and redundant compute capacity for a total region outage. It needs the Enterprise Plus edition.
- 30 April 2024, GA.
ML.TRANSFORMand transform-only models.ML.TRANSFORMapplies the transformations captured in theTRANSFORMclause of an existing model to input data. A transform-only model applies preprocessing functions and returns the processed data, so preprocessing is separate from training. - 30 April 2024, Preview. Iceberg tables in materialized views. A materialized view can reference Iceberg tables instead of data moved into BigQuery storage.
- 24 April 2024, Preview. SQL user-defined aggregate functions.
CREATE AGGREGATE FUNCTIONcreates a UDAF written with SQL expressions. - 18 April 2024, GA. Quantified
LIKE.LIKE ANYmatches when at least one pattern matches, andLIKE SOMEis a synonym.LIKE ALLmatches when every pattern matches. The patterns can be a list or an array. - 16 April 2024, Preview. Subqueries in row-level access policies. A row-level access policy can use a subquery.
- 9 April 2024, Preview. More
ML.GENERATE_EMBEDDINGmodels. It works with a remote model based on Vertex AImultimodalembeddingto embed text and images in one space. It also works with PCA, autoencoder and matrix factorization models. - 5 April 2024, GA. Differential privacy.
- 4 April 2024, GA.
allow_non_incremental_definitionandmax_stalenessfor materialized views. The first option allows a wider range of queries in a materialized view. The second keeps performance steady and cost controlled on large tables that change often. - 4 April 2024, Preview. Model monitoring functions.
ML.DESCRIBE_DATA,ML.VALIDATE_DATA_SKEW,ML.VALIDATE_DATA_DRIFT,ML.TFDV_DESCRIBEandML.TFDV_VALIDATEcompute statistics for training and serving data and compare them to find anomalous differences. - 3 April 2024. Materialized views on CDC tables. A materialized view can be built on a table that has change data capture enabled for streaming updates and deletes.
- 3 April 2024, GA. More collation support. Collation applies to the underscore in
LIKE.STRUCTvalues can be compared with=,!=,INandCASEunder collation. - 2 April 2024, Preview. Tuning and evaluating LLMs. A remote model based on a
text-bisonmodel supports supervised tuning.ML.EVALUATEevaluates pre-trainedtext-bison,text unicornandgemini-promodels and tunedtext-bisonmodels. - 1 April 2024, Preview. History-based optimizations. They can be turned on and off, and their effect on queries can be analyzed.
- 28 March 2024, Preview. SAP Datasphere federated queries. A federated query can read data in SAP Datasphere.
- 28 March 2024, GA. Search index optimization for string literals. Queries are optimized when they compare string literals with indexed data using
=,IN,LIKEorSTARTS_WITH. - 26 March 2024, Preview. Model
gemini-pro-vision. A remote model can be based on this Vertex AI model.ML.GENERATE_TEXTruns it on visual content in object tables, for tasks such as image or video captioning and visual Q&A. - 21 March 2024, Preview.
LEFT OUTER JOINandUNION ALLin incremental materialized views. Incremental materialized views accept both. - 19 March 2024, GA. Spark stored procedures. Stored procedures can be written in Python, Java and Scala and run on Apache Spark.
- 18 March 2024, Preview. Undelete a dataset. A dataset within the time travel window can be recovered to the state it had when it was deleted.
- 18 March 2024, GA. Text analysis and distance functions.
CREATE SEARCH INDEXtakes text analysis options. TheLOG_ANALYZERandPATTERN_ANALYZERanalyzers work in functions such asSEARCH.TEXT_ANALYZE,ML.BAG_OF_WORDS,ML.TF_IDF,BAG_OF_WORDS,TF_IDF,COSINE_DISTANCE,EUCLIDEAN_DISTANCEandEDIT_DISTANCEare also generally available. - 18 March 2024, GA. Hierarchical forecasts. Time series models can forecast hierarchically, aggregating and rolling up values for all series in the model.
- 6 March 2024, GA.
INFORMATION_SCHEMA.WRITE_API_TIMELINE*. These views hold per-minute aggregated statistics for Storage Write API ingestion. - 4 March 2024, GA.
BQ.REFRESH_EXTERNAL_METADATA_CACHE. A system procedure that refreshes the metadata cache of selected BigLake tables. - 29 February 2024, GA. Cross-cloud materialized views and joins. Materialized views can sit over Amazon S3 metadata cache-enabled BigLake tables, and replicas of those views avoid data egress costs. Information about the replicas is available through SQL. Cross-cloud joins run queries that span Google Cloud and BigQuery Omni regions.
- 28 February 2024, Preview. Materialized views over logical views. A materialized view can reference logical views.
- 28 February 2024, GA. Filtering Amazon S3 and Azure Blob Storage data before loading.
CREATE TABLE AS SELECT,CREATE TABLE IF NOT EXISTS AS SELECT,CREATE OR REPLACE TABLE AS SELECTandINSERT INTO SELECTcan filter files before the results go into BigQuery tables. Billing is enabled. - 28 February 2024, Preview. Anomaly detection with
ARIMA_PLUS_XREG. Multivariate time series models can detect anomalies in historical data or in new data with several feature columns. - 27 February 2024, GA.
DELETEof whole partitions. ADELETEthat targets all rows in a partition deletes the partition without scanning bytes or using slots. - 27 February 2024, Preview. Time series and range functions. They support time series analysis.
- 26 February 2024, Preview.
GROUP BY ALL. Groups rows by grouping keys inferred from theSELECTitems. - 26 February 2024, GA.
GROUP BY GROUPING SETS,GROUP BY CUBEandGROUPING.GROUPING SETSaggregates over chosen grouping sets,CUBEover all permutations of the grouping sets.GROUPINGreports whether a value inGROUP BYis aggregated. - 22 February 2024, GA. Text embeddings. A remote model can reference a Vertex AI
textembedding-gecko*model, andML.GENERATE_EMBEDDINGembeds text with it.ML.GENERATE_EMBEDDINGalso works with the NNLM, SWIVEL and BERT TensorFlow models. - 15 February 2024, GA. Model
gemini-pro. A remote model can be based on this Vertex AI model.ML.GENERATE_TEXTruns it on text in tables. - 8 February 2024, GA. Custom data masking. Custom masking routines, such as a salted hash, can be defined. It needs the Enterprise Plus edition.
- 6 February 2024, Announcement. Spark stored procedure billing. Billing begins on 12 March 2024. Until then Spark stored procedures cost nothing extra.
- 31 January 2024, Preview.
ORGANIZATION_OPTIONS_CHANGESandPROJECT_OPTIONS_CHANGES. TheseINFORMATION_SCHEMAviews show the history of configuration changes to the options of an organization and of a project. - 31 January 2024, Preview.
VECTOR_SEARCHand vector indexes.VECTOR_SEARCHsearches embeddings for semantically similar entities. A vector index makes it more efficient and returns more approximate results. - 24 January 2024.
APPROX_GLOBAL_FEATURE_CONTRIBdefault. For boosted tree and random forest models it defaults toTRUEwhenENABLE_GLOBAL_EXPLAINisTRUEandNUM_PARALLEL_TREEis above 10 (boosted tree) or 50 (random forest). This speeds up training. - 24 January 2024.
residualcolumn inML.EXPLAIN_FORECAST. ForARIMA_PLUSandARIMA_PLUS_XREGmodels, it holds the difference between the actual and the fitted series for the historical data. - 24 January 2024, GA.
ST_LINEINTERPOLATEPOINT. Returns the point at a given fraction along a linestring. - 16 January 2024, Preview. Cross-cloud joins. Queries can span Google Cloud and BigQuery Omni regions.
2023
- 12 December 2023, Preview. Cross-cloud materialized views. Materialized views can sit over Amazon S3 metadata cache-enabled BigLake tables. Replicas of those views avoid data egress costs.
- 7 December 2023, Preview.
ML.TRANSFORMand transform-only models.ML.TRANSFORMapplies the transformations captured in theTRANSFORMclause of an existing model to input data. A transform-only model applies preprocessing functions and returns the processed data. - 30 November 2023, Preview. Slot estimator for on-demand workloads. It gives project-level recommendations for cost-optimal commitments and autoscaling.
- 16 November 2023, GA. Remote model syntax for LLMs.
CREATE MODELfor remote models reaches all text generation and text embedding LLMs, such astext-bison-32kandtextembedding-gecko-multilingual, and supports different LLM versions. - 13 November 2023, GA.
ML.FEATURES_AT_TIMEandML.ENTITY_FEATURES_AT_TIME. They fetch features as of a point in time to avoid data leakage.ML.FEATURES_AT_TIMEuses one cutoff for all entities.ML.ENTITY_FEATURES_AT_TIMEreads features from several points in time for several entities. - 13 November 2023, Preview.
ML.PROCESS_DOCUMENTandML.TRANSCRIBE.ML.PROCESS_DOCUMENTprocesses documents in object tables with a remote model based on Document AI.ML.TRANSCRIBEtranscribes audio files in object tables with a remote model based on Speech-to-Text. - 6 November 2023, GA. Forecast limits and custom holidays.
FORECAST_LIMIT_LOWER_BOUNDandFORECAST_LIMIT_UPPER_BOUNDinCREATE MODELbound the forecasted values.CREATE MODELcan also specify custom holidays for time series models.ML.HOLIDAY_INFOlists the holidays anARIMA_PLUSorARIMA_PLUS_XREGmodel uses, andML.EXPLAIN_FORECASTexplains the effect of each holiday. - 2 November 2023, Preview. Text analysis and distance functions.
CREATE SEARCH INDEXtakes text analysis options. TheLOG_ANALYZERandPATTERN_ANALYZERanalyzers work in functions such asSEARCH.TEXT_ANALYZE,ML.BAG_OF_WORDS,ML.TF_IDF,BAG_OF_WORDS,TF_IDF,COSINE_DISTANCE,EUCLIDEAN_DISTANCEandEDIT_DISTANCEare added. - 1 November 2023, Preview.
TABLE_STORAGE_USAGE_TIMELINEandTABLE_STORAGE_USAGE_TIMELINE_BY_ORGANIZATION. TheseINFORMATION_SCHEMAviews give the total billable bytes per table per day, for a project and for an organization. - 1 November 2023, GA. Cached results shared between users. On the Enterprise and Enterprise Plus editions, a query can use the cached result of the same query run by another user in the same project.
- 31 October 2023, Preview. DML on recently streamed rows. DML can modify rows that the Storage Write API wrote recently.
- 23 October 2023, Preview. More functions in custom data masking. Custom masking routines can use more functions, including SHA hash functions with salt.
- 19 October 2023, Preview. Spark stored procedures without enrollment. Stored procedures for Apache Spark no longer need enrollment.
- 16 October 2023, Preview. DLP functions. They encrypt and decrypt data between BigQuery and DLP using AES-SIV.
- 12 October 2023, GA.
ST_LINESUBSTRINGandST_HAUSDORFFDISTANCE.ST_LINESUBSTRINGreturns a segment of a linestring between a start and an end fraction.ST_HAUSDORFFDISTANCEreturns the discrete Hausdorff distance between two geometries. - 9 October 2023, Preview. Grouping sets.
GROUP BY GROUPING SETSandGROUP BY CUBEare added, andGROUP BY ROLLUPaccepts sets of groupable items. TheGROUPINGfunction reports whether a value inGROUP BYis aggregated. - 9 October 2023, GA. Column descriptions on views.
CREATE VIEWandALTER COLUMNcan set descriptions on the columns of a view. - 28 September 2023, Preview.
ML.FEATURES_AT_TIMEandML.ENTITY_FEATURES_AT_TIME. They fetch features as of a point in time to avoid data leakage.ML.FEATURES_AT_TIMEuses one cutoff for all entities.ML.ENTITY_FEATURES_AT_TIMEreads features from several points in time for several entities. - 25 September 2023, GA. Authorized stored procedures. A stored procedure can be shared with users or groups who have no direct access to the underlying tables.
- 25 September 2023, GA. Materialized views over BigLake metadata cache-enabled tables. They can reference structured data in Cloud Storage and work like materialized views over BigQuery-managed tables, including automatic refresh and smart tuning.
- 12 September 2023, GA. Struct subscript operator. Accesses a
STRUCTfield by index, offset or ordinal. - 12 September 2023, GA. Array subscript by index. The array subscript operator returns a value directly by index. Before, only offset and ordinal were available.
- 30 August 2023, Preview.
EXPORT DATAto Bigtable.EXPORT DATAwrites BigQuery data directly to Bigtable. - 29 August 2023, GA. Model
text-bison. A remote model can be based on this Vertex AI LLM.ML.GENERATE_TEXTruns it on text in tables. - 29 August 2023, Preview. Data clean rooms. Several parties can share, join and analyze data without moving it or revealing the underlying data. The release also lists an aggregation threshold clause for queries and views.
- 24 August 2023, Preview. Text embeddings. A remote model can reference the Vertex AI
textembedding-geckomodel, andML.GENERATE_TEXT_EMBEDDINGembeds text with it.ARRAY<NUMERIC>works as an input feature type for other models, and the NNLM, SWIVEL and BERT TensorFlow models can generate text embeddings. - 24 August 2023, GA. BigQuery ML inference. ONNX, XGBoost and TensorFlow Lite models can be imported and run in BigQuery ML. Models can be hosted remotely on Vertex AI Prediction. BigQuery ML functions can run inference on pretrained Vertex AI models for natural language, translation and computer vision tasks.
- 22 August 2023, Preview. Custom masking routines. Masking rules written with
REGEX_REPLACEcan obfuscate sensitive data. - 17 August 2023, Preview. Cross-region dataset replication. A dataset can be replicated from its source region to one or more other regions.
- 14 August 2023. Network egress pricing. From 15 September 2023, prices apply to network egress from a BigQuery Google Cloud region to another Google Cloud region, on the same continent and between continents.
- 10 August 2023, GA. Protobuf export. User-defined functions can export BigQuery data as Protocol Buffer columns.
- 8 August 2023, GA.
ANY_VALUEwithHAVING MAXandHAVING MIN,MAX_BYandMIN_BY. They work in queries and in materialized views.MAX_BYis a synonym forANY_VALUE(x HAVING MAX y), andMIN_BYforANY_VALUE(x HAVING MIN y). - 7 August 2023, GA. JSON functions.
JSON_ARRAY,JSON_ARRAY_APPEND,JSON_ARRAY_INSERT,JSON_OBJECT,JSON_REMOVE,JSON_SET,JSON_STRIP_NULLS,LAX_BOOL,LAX_FLOAT64,LAX_INT64andLAX_STRING. - 7 August 2023, Preview. Quantified
LIKE.LIKE ANYmatches when at least one pattern matches, andLIKE SOMEis a synonym.LIKE ALLmatches when every pattern matches. - 3 August 2023, GA.
GRANTandREVOKEon materialized views. Access to materialized views can be granted and revoked with a SQL statement. - 28 July 2023, GA. Query queues. BigQuery sets query concurrency from available slots instead of a fixed limit and queues queries once the maximum is reached. Queues are on by default. The maximum concurrency target can be set for a reservation, and the queue timeout for interactive and batch queries comes from default configurations.
- 20 July 2023, GA.
ARIMA_PLUS_XREG. Multivariate time series forecasting with extra feature columns. - 20 July 2023. BigQuery ML explainability.
ML.EXPLAIN_FORECASTworks withARIMA_PLUS_XREGmodels and explains the holiday effect forARIMA_PLUSandARIMA_PLUS_XREG.ML.GLOBAL_EXPLAINworks with AutoML Tables models. Theapprox_global_feature_contribtraining option and theapprox_feature_contriboption ofML.EXPLAIN_PREDICTapproximate feature contribution for boosted tree and random forest models. - 19 July 2023, Preview. Search index optimization for string literals. Some queries are optimized when they compare string literals with indexed data using
=,IN,LIKEorSTARTS_WITH. - 17 July 2023, GA. Primary and foreign key constraints.
- 12 July 2023, GA.
TRANSFORMpreprocessing functions.ML.IMPUTER,ML.LABEL_ENCODER,ML.MAX_ABS_SCALER,ML.MULTI_HOT_ENCODER,ML.NORMALIZER,ML.ONE_HOT_ENCODERandML.ROBUST_SCALERpreprocess features. Models that use theTRANSFORMclause can be exported to the TensorFlow SavedModel format. - 12 July 2023, Preview. Custom holiday modeling.
CREATE MODELcan specify custom holidays for time series models.ML.HOLIDAY_INFOlists the holidays anARIMA_PLUSorARIMA_PLUS_XREGmodel uses, andML.EXPLAIN_FORECASTexplains the effect of each holiday. The public tablebigquery-public-data.ml_datasets.holidays_and_events_for_forecastinglists the built-in holidays. - 5 July 2023, GA. Fail-safe period. Seven more days of storage after the time travel window keep data available for emergency recovery. Fail-safe bytes are not billed until 17 July 2023.
- 5 July 2023, GA. Time travel window. It can be set from a minimum of two days to a maximum of seven.
- 5 July 2023, GA. Physical bytes storage billing. When a dataset uses the physical bytes billing model, active storage cost includes the bytes used for time travel and fail-safe.
- 5 July 2023, GA.
LOAD DATA. Loads Avro, CSV, newline-delimited JSON, JSON, ORC or Parquet files into a table. - 5 July 2023, Preview. Slot estimator. Gives cost-optimal commitment and autoscale recommendations based on editions pricing and historical performance.
- 5 July 2023, Announcement. Capacity commitments. Annual commitments are only available in the Enterprise or Enterprise Plus edition, and flat-rate annual commitments are gone. Monthly and flex commitments are no longer available.
- 26 June 2023, Preview. Spark stored procedures in Java and Scala. Stored procedures for Apache Spark can be written in Java or Scala.
- 21 June 2023, GA.
TRUNCATE TABLEin transactions. Multi-statement transactions supportTRUNCATE TABLE. - 20 June 2023, GA. Metadata caching. Cached metadata can speed up queries on BigLake tables and object tables that reference many objects, because the query skips listing objects in Cloud Storage. Metadata cache usage is billed from this date.
- 20 June 2023, GA. Apache Iceberg tables. BigQuery queries Iceberg tables created by open source engines.
- 15 June 2023, Preview. Model
text-bison. A remote model can be based on this Vertex AI LLM, andML.GENERATE_TEXTruns it on text in tables. Access needs an allowlist. - 14 June 2023, GA.
TABLE_STORAGEandTABLE_STORAGE_BY_ORGANIZATION. TheseINFORMATION_SCHEMAviews show a snapshot of current table storage usage for a project and for an organization. - 23 May 2023. DML and table modification limits. DML statements no longer count toward the daily limit on table and partitioned table modifications. The limit itself is unchanged.
- 19 May 2023, GA.
EXTERNAL_QUERYSQL pushdown. Column pruning fromSELECTand filter pushdown fromWHEREreduce the data fetched from sources such as Cloud SQL and Spanner. It applies toSELECT * FROM Tqueries, and not every data type supports filter pushdown. - 11 May 2023, GA. Object tables. Read-only tables of metadata for unstructured data in Cloud Storage. The release adds
ML.DECODE_IMAGE,ML.CONVERT_COLOR_SPACE,ML.CONVERT_IMAGE_TYPE,ML.RESIZE_IMAGE,ML.DISTANCEandML.LP_NORM. - 9 May 2023, GA.
EXTERNAL_QUERYSQL pushdown. Column pruning fromSELECTand filter pushdown fromWHEREreduce the data fetched from sources such as Cloud SQL and Spanner. It applies toSELECT * FROM Tqueries, and not every data type supports filter pushdown. - 8 May 2023, Preview. Differential privacy. The differential privacy clause and four differentially private aggregate functions anonymize data:
AVG,COUNT,SUMandPERCENTILE_CONT. - 8 May 2023, Preview.
INFORMATION_SCHEMA.MATERIALIZED_VIEW. This view and enhanced job statistics show materialized view usage and refresh jobs. - 5 May 2023, Preview.
INSERT INTO SELECTfrom Amazon S3 and Azure Blob Storage. The statement filters data from files in S3 and Blob Storage and appends it to BigQuery tables. - 3 May 2023, GA. Table clones.
- 1 May 2023, Preview. Query queue timeouts. With query queues, the interactive and batch queue timeouts can be set in the default configuration.
- 1 May 2023, Preview. Column descriptions on views.
CREATE VIEWandALTER COLUMNcan set descriptions on the columns of a view. - 27 April 2023, GA.
JSONmapping in Spanner federated queries. - 25 April 2023, GA. Dynamic data masking on repeated records. Masking works on
RECORDcolumns inREPEATEDmode. Before, querying such columns with masking applied returned internal errors. - 13 April 2023, GA. Rounding mode. A parameterized
NUMERICorBIGNUMERICcolumn can useROUND_HALF_EVENorROUND_HALF_AWAY_FROM_ZERO. A default can be set on a table or dataset and applies to columns added later.ROUND()takes the mode as an optional argument. - 10 April 2023. Cached results for table snapshots. Queries against table snapshots can return results from the cache.
- 10 April 2023, GA. Limits when querying Azure and Amazon S3 data. The maximum result size is 20 GiB of logical bytes and the maximum row size is 10 MiB. A query with
ORDER BYfails when its result is larger than 256 MB, where the limit was 2 MB. - 5 April 2023, Preview. Non-incremental materialized views. They support most SQL queries, including
OUTER JOIN,UNION,HAVINGand analytic functions. - 29 March 2023, GA. Editions and autoscaling slots. Compute is generally available in three editions: Standard, Enterprise and Enterprise Plus, which use the slot autoscaling model. Autoscaling reservations and commitments created in the preview became Enterprise edition.
- 28 March 2023, Preview. Model import and remote hosting. ONNX, XGBoost and TensorFlow Lite models can be imported for inference, and models can be hosted remotely on Vertex AI Prediction. Table-valued functions run inference with the Cloud Vision, Cloud Natural Language and Cloud Translation APIs. Enrollment is required.
- 28 March 2023.
tf_version,xgboost_versionandinstance_weight_col.tf_versionpicks the TensorFlow version for training, ‘1.15’ by default, and ‘2.8.0’ for TF2 with Keras.xgboost_versionpicks the XGBoost version, ‘0.9’ by default, or ‘1.1’.instance_weight_colnames the column of per-row weights, for boosted tree and random forest models with non-array features. - 22 March 2023, Preview. Unicode column names. Column names can use international character sets, alphanumeric and special characters. Existing columns get such names with
RENAME. - 20 March 2023, GA. AutoML Tables
OPTIMIZATION_OBJECTIVEoptions.MAXIMIZE_PRECISION_AT_RECALLandMAXIMIZE_RECALL_AT_PRECISIONare accepted. - 10 March 2023, Preview.
CREATE TABLE AS SELECTfrom Amazon S3 and Azure Blob Storage. The statement filters data from files in S3 and Blob Storage before the results go into BigQuery tables. - 7 March 2023, GA. Case-insensitive collation. The release adds
MIN,MAX,COUNTwithDISTINCTandPERCENTILE_DISCas window functions,ORDER BYandPARTITION BYin theWINDOWclause, theLIKEoperator with limits, views, materialized views, table functions and BI Engine. - 2 March 2023, GA.
WITH RECURSIVE. A query can include one or more recursive common table expressions. - 27 February 2023.
ARIMA_PLUS_XREGfor on-demand users. The multivariate time series model is available to on-demand users. - 27 February 2023, GA. Column default values. Columns in tables can have default values.
- 23 February 2023, Preview. Authorized stored procedures. A stored procedure can be shared with users or groups who have no direct access to the underlying tables.
- 22 February 2023, Preview. Primary and foreign key constraints.
CREATE TABLE,ALTER TABLE ADD PRIMARY KEYandALTER TABLE ADD CONSTRAINTdefine them. - 21 February 2023, GA.
ALTER TABLE RENAME COLUMNandALTER TABLE DROP COLUMN. - 15 February 2023, GA. Case-insensitive datasets.
CREATE SCHEMAandALTER SCHEMAcan make a dataset and its tables case-insensitive. - 13 February 2023, Preview. Materialized views over BigLake metadata cache-enabled tables. They reference structured data in Cloud Storage.
- 9 February 2023, Preview. Autoscaling slot reservations. Autoscaling reservations and their commitments can be created.
- 9 February 2023, GA. More dynamic data masking types. Date year, email, first four characters and last four characters masks.
- 6 February 2023, GA.
query_infocolumn.INFORMATION_SCHEMA.JOBS,JOBS_BY_FOLDERandJOBS_BY_ORGANIZATIONshow query processing information in this column. - 6 February 2023, Preview.
ANY_VALUEwithHAVING MAXandHAVING MIN. - 19 January 2023, Preview. Preprocessing functions and
TRANSFORMexport.ML.ROBUST_SCALER,ML.NORMALIZER,ML.ONE_HOT_ENCODER,ML.IMPUTER,ML.MAX_ABS_SCALERandML.LABEL_ENCODERare added. Models trained with theTRANSFORMclause can be exported in the TensorFlow SavedModel format. - 17 January 2023, GA. Sparse input. BigQuery ML model training supports sparse input, for data whose values are mostly zero or empty.
- 17 January 2023, Preview.
ARIMA_PLUS_XREG. Multivariate time series forecasting with extra feature columns. - 10 January 2023, GA.
ALTER CAPACITY SET OPTIONSandALTER RESERVATION SET OPTIONS.CREATE CAPACITY,CREATE RESERVATIONandCREATE ASSIGNMENTaccept anOPTIONSclause. - 9 January 2023, GA. Sessions. Temporary functions last until the session ends. Statements with
TEMPcan also useOR REPLACEandIF NOT EXISTS. - 4 January 2023, Preview. Vision models in BigQuery ML. Imported vision models run inference on images in object tables, for tasks such as object detection and OCR. Access needs a sign-up form.
2022
- 15 December 2022, GA. Cloud SQL federated queries work over a private connection.
- 1 December 2022, Preview. Apache Iceberg tables. Queries can read Iceberg tables created by open source engines.
- 17 November 2022, Preview. Metadata caching. Queries on BigLake tables and object tables can use cached metadata, which avoids listing objects in Cloud Storage. It helps tables that reference many objects.
- 17 November 2022, Preview. Object tables. Object tables are read-only tables holding metadata for unstructured data in Cloud Storage. BigQuery ML and remote functions can analyze the files and run inference on images, audio, documents and other file types.
- 9 November 2022, GA.
LOAD DATAfrom Amazon S3 and Azure Blob Storage. The statement loads hive-partitioned files, loads semi-structured JSON intoJSONcolumns without a schema, supports customer-managed encryption keys on the destination table, and transfers to theUSmulti-region andUS-EAST-4. - 3 November 2022, GA. Wrapped keyset functions.
KEYS.NEW_WRAPPED_KEYSET,KEYS.ROTATE_WRAPPED_KEYSETandKEYS.REWRAP_KEYSETcreate, rotate and rewrap wrapped keysets in SQL. Columns can be encrypted and decrypted with a wrapped keyset. - 31 October 2022, Preview.
max_stalenessmaterialized view option. The option sets how stale a materialized view may be, to keep performance steady and cost controlled on large tables that change often. - 31 October 2022, GA. Column-level data masking. Masking hides column data from selected groups of users while still giving them access to the column.
- 27 October 2022, GA. Search indexes and
SEARCH. Search indexes and theSEARCH()function find specific values in unstructured text and semi-structured data. - 26 October 2022, GA.
ST_ISCLOSED,ST_ISRING.ST_ISCLOSEDreturnsTRUEfor a non-empty geography where each element has an empty boundary.ST_ISRINGchecks whether a geography is a linestring that is closed and simple. - 18 October 2022, GA. Remote functions. Queries can call functions in Cloud Functions or Cloud Run.
- 11 October 2022, Preview. Stored procedures for Apache Spark.
- 10 October 2022, GA. Multi-statement transactions.
- 10 October 2022, Preview. Physical bytes for storage billing. A dataset can be billed on physical bytes.
- 3 October 2022, GA.
JSONcolumn type. - 29 September 2022, Preview.
ROUND_HALF_EVENrounding mode. ParameterizedNUMERICandBIGNUMERICcolumns accept it, andROUND()takes it as an optional argument. - 21 September 2022, GA. BigQuery Omni on-demand pricing. The on-demand model is offered for a limited time.
- 16 September 2022, BigQuery Omni limits. The quota for total query result size per project is 1 TB per day. The maximum result size of a query rises from 2 MB to 10 GB in Preview.
- 14 September 2022, Preview.
is_case_insensitiveschema option. It makes a dataset and its table names case-insensitive. - 9 September 2022, GA.
ARIMA_PLUSoptions.HOLIDAY_REGIONtakes more than one region string, and the holidays of all listed regions are used in the model.TREND_SMOOTHING_WINDOW_SIZEsmooths the trend component with a centered moving average. - 1 September 2022, GA. Random forest models.
- 29 August 2022, GA. Community contributed UDFs. They live in the
bigquery-utilsGitHub repository and in thebigquery-public-data.persistent_udfspublic dataset. - 19 August 2022, Preview.
ALTER TABLE RENAME COLUMN. - 17 August 2022, Preview. Column default values.
- 15 August 2022, GA. BigQuery Omni reservation DDL and access control DCL.
- 10 August 2022, GA. Default configurations. Defaults can be set at the project or organization level.
- 10 August 2022, GA. Query priority for Cloud Spanner federated queries.
- 9 August 2022, GA. Cloud Bigtable external data sources.
- 3 August 2022, Preview.
max_stalenessmaterialized view option. It sets a staleness limit for large, frequently changing tables. A note on 15 August says an issue was found and the feature is not available. - 1 August 2022, GA.
CBRT.CBRTcomputes the cube root of a value. - 1 August 2022, Preview.
LOAD DATA. The statement loads data from one or more files into a table. - 27 July 2022, GA.
COT,COTH,CSC,CSCH,SEC,SECH. They compute the cotangent, hyperbolic cotangent, cosecant, hyperbolic cosecant, secant and hyperbolic secant of an angle. - 25 July 2022, GA. BigLake. BigQuery ML models can be created from data in Cloud Storage through BigLake tables.
- 6 July 2022, Preview.
APPENDS. This change history table-valued function returns the appends to a table over a window of time. - 21 June 2022, Preview. Query queues. BigQuery sets query concurrency itself instead of using a fixed limit, for on-demand and flat-rate customers. Flat-rate customers can set their own concurrency target, and queries beyond it wait until resources free up.
- 15 June 2022, GA. Deterministic encryption functions.
DETERMINISTIC_ENCRYPT,DETERMINISTIC_DECRYPT_BYTESandDETERMINISTIC_DECRYPT_STRINGencrypt and decrypt columns in a way that still supports aggregation and joins. - 13 June 2022, GA.
@@dataset_project_id. The system variable sets the default project for datasets that do not name one in the query. It is also a connection property. - 10 June 2022, Multi-statement query time limit. The cumulative time limit rises from 6 hours to 24 hours.
- 3 June 2022, Preview. BigQuery Omni reservation DDL and access control DCL.
- 31 May 2022, Preview. Column-level data masking. Masking hides column data from selected groups of users while still giving them access to the column. Combined with column-level access control, access can range from full to none per group.
- 5 May 2022, GA.
%Jformat element. It gives the ISO 8601 day of the year, starting at 1, inDATE,TIME,DATETIMEandTIMESTAMPfunctions. - 5 May 2022.
PARSE_DATE,PARSE_TIME,PARSE_DATETIME,PARSE_TIMESTAMPformat elements. They accept%a,%A,%g,%G,%j,%u,%U,%V,%wand%W. - 3 May 2022, GA.
ARIMA_PLUSforecasting metrics and fast training.ML.EVALUATEreturns MAPE, SMAPE and MSE. The optionsTIME_SERIES_LENGTH_FRACTION,MIN_TIME_SERIES_LENGTHandMAX_TIME_SERIES_LENGTHspeed up training with little or no loss of accuracy. - 2 May 2022, Preview. Collation. Case-insensitive collation controls how strings compare and sort. It applies to comparison operators, joins,
ORDER BY,GROUP BY, set operations and some scalar and aggregate functions. TheCOLLATEfunction returns aSTRINGwith a collation specification. TheDEFAULT COLLATEclause ofCREATE SCHEMA,ALTER SCHEMA,CREATE TABLEandALTER TABLEsets a default for all columns that support collation. TheCOLLATEclause ofALTER TABLE ADD COLUMNandALTER COLUMN SET DATA TYPEsets it on one column. - 25 April 2022, Preview.
TABLE_STORAGEview. It shows current storage usage for tables and materialized views. - 25 April 2022, Preview. Configurable time travel window. The window can be set from two to seven days.
- 7 April 2022, Preview. BigLake. BigLake tables query data across cloud data lakes and warehouses, with access policies on columns or rows.
- 7 April 2022, Preview. Search indexes and
SEARCH. They find data elements in unstructured text and semi-structured data. - 1 April 2022, Preview. BigQuery ML and Vertex AI Model Registry. BigQuery ML models can be sent to the Vertex AI Model Registry, organized there and deployed to endpoints.
- 28 March 2022, GA. Wide-and-Deep models.
- 16 March 2022. Explicit schema for external tables. External tables over Parquet, ORC and Avro files can take a schema you give them. Before, the schema was always detected from the last file in lexicographic order.
- 4 March 2022, GA. Sessions. A session can be terminated automatically or by hand, and a label can be set for all queries in it.
INFORMATION_SCHEMA.SESSIONS_BY_PROJECTandINFORMATION_SCHEMA.SESSIONS_BY_USERreturn session metadata. - 16 February 2022, Preview. Remote functions. They let a function be written in languages other than SQL and JavaScript, or use libraries and services that user-defined functions do not allow.
- 15 February 2022, Preview. Table clones. A clone is a lightweight, writable copy of a table. Storage is charged only for data that differs from the base table.
- 14 February 2022.
ARIMA_PLUStraining speed. BigQuery ML trainsARIMA_PLUSmodels 5 times faster than before. - 14 February 2022, GA.
INFORMATION_SCHEMA.STREAMING_TIMELINE_*views. - 14 February 2022, GA.
QUALIFY. The clause filters the results of analytic functions. - 10 February 2022, Preview. BigQuery Omni
INFORMATION_SCHEMAviews. Omni supports theINFORMATION_SCHEMA.JOBS_*andINFORMATION_SCHEMA.RESERVATION*views. - 3 February 2022, GA. BigQuery ML hyperparameter tuning.
CREATE MODELsearches for the best hyperparameters while training. - 2 February 2022, Preview.
WITH RECURSIVE. A query in aWITHclause can refer to itself or to queries defined later in the clause. - 31 January 2022, GA. Materialized views without aggregation and with inner joins.
- 25 January 2022, GA. Explainable AI in BigQuery ML. It explains prediction and forecasting results.
- 6 January 2022, Preview.
JSONdata type.
2021
- 16 December 2021. Row-level security supports administrator access to historical data for tables with row-level access policies.
- 14 December 2021, GA. BI Engine SQL interface.
- 6 December 2021, GA.
ML.DETECT_ANOMALIES. The function detects anomalies in time-series data withARIMA_PLUSmodels. With K-means, autoencoder or PCA models it detects anomalies in independent and identically distributed data. - 3 December 2021, GA. PCA and autoencoder models. They serve dimensionality reduction, feature embedding and unsupervised anomaly detection.
- 9 November 2021, GA. Scripting statements.
CASEruns the first statement list whose boolean expression isTRUE.CASE search_expressionruns the first list whoseWHENexpression matches the search expression. Labels give an unconditional jump to the end of the labeled block or loop.REPEATruns a statement list until the condition at its end isTRUE.FOR...INloops over every row of a table expression. - 8 November 2021, GA.
DDLcolumn. TheROUTINES,SCHEMATAandTABLESviews inINFORMATION_SCHEMAhave aDDLcolumn holding the statement that creates the resource. - 2 November 2021, GA. Parameterized types.
STRING(L),BYTES(L),NUMERIC(P),NUMERIC(P, S),BIGNUMERIC(P)andBIGNUMERIC(P, S). - 28 October 2021, GA. Table snapshots. A snapshot can be created in a different project from its base table.
- 27 October 2021, GA. SQL column-level encryption with Cloud KMS. The AEAD encryption functions work with keysets that are encrypted by Cloud KMS.
- 25 October 2021, GA. BigQuery Omni. Omni is the multi-cloud analytics offering.
- 1 October 2021, Preview.
ST_BUFFER,ST_BUFFERWITHTOLERANCE. Both return aGEOGRAPHYfor the buffer around the input.ST_BUFFERtakes a number of segments andST_BUFFERWITHTOLERANCEtakes a tolerance, which set how far the result may deviate from the ideal radius. - 28 September 2021, GA. Table functions. Authorized table functions are supported.
- 27 September 2021, GA.
ST_BOUNDINGBOX,ST_EXTENT,S2_COVERINGCELLIDS,S2_CELLIDFROMPOINT.ST_BOUNDINGBOXreturns aSTRUCTfor the bounding box of a geography andST_EXTENTdoes so for a set of geographies.S2_COVERINGCELLIDSreturns an array of S2 cell IDs covering a geography.S2_CELLIDFROMPOINTreturns the S2 cell ID covering a point. - 17 September 2021, GA.
ST_EXTERIORRING,ST_INTERIORRINGS,ST_ANGLE,ST_AZIMUTH,ST_NUMGEOMETRIES,ST_GEOMETRYTYPE.ST_EXTERIORRINGreturns the outer ring of a polygon as a linestring andST_INTERIORRINGSreturns the inner rings.ST_ANGLEreturns the angle between two intersecting lines.ST_AZIMUTHreturns the azimuth of a segment between two points.ST_NUMGEOMETRIEScounts the geometries in a geography.ST_GEOMETRYTYPEreturns the OGC geometry type as a string. - 8 September 2021, Preview. Sessions. A session ties SQL activity across scripts and multi-statement transactions to one session identifier. Session variables such as the default time zone or dataset and temporary tables last for the whole session. The jobs
INFORMATION_SCHEMAviews have aSESSION_IDcolumn. - 19 August 2021, GA. Cloud Spanner federated queries.
- 6 August 2021, Preview. PCA models.
CREATE MODELsupports principal component analysis. - 3 August 2021, GA.
ALTER COLUMN SET DATA TYPE. - 3 August 2021, GA.
CREATE TABLE LIKEandCREATE TABLE COPY. - 28 July 2021, Preview. Wide-and-Deep models.
'DNN_LINEAR_COMBINED_CLASSIFIER'and'DNN_LINEAR_COMBINED_REGRESSOR'create Wide-and-Deep classifier and regressor models. - 27 July 2021, Preview. Explainable AI in BigQuery ML. It shows how each feature contributed to a prediction for classification and regression models.
- 27 July 2021, Preview.
INTERVALtype. The type represents a duration or an amount of time. - 26 July 2021, GA. Holiday effects for weekly time series. Time series models handled daily series before.
- 26 July 2021, GA. DML statistics. DML jobs return the number of rows inserted, deleted and updated, and the
INFORMATION_SCHEMA.JOBS_BY_*views carry the same statistics. - 19 July 2021, GA. Workload management statements.
CREATE CAPACITY,CREATE RESERVATION,CREATE ASSIGNMENT,DROP CAPACITY,DROP RESERVATIONandDROP ASSIGNMENT. - 19 July 2021, GA.
PIVOTandUNPIVOT. - 19 July 2021, GA.
CONTAINS_SUBSTR. - 7 July 2021, Preview. Materialized views without aggregation and with inner joins.
- 29 June 2021, Preview. Multi-statement transactions. They run mutating operations on one or more tables and commit or roll back the changes atomically.
- 28 June 2021, GA.
GRANTandREVOKE. TheINFORMATION_SCHEMA.OBJECT_PRIVILEGESview is in Preview. - 28 June 2021, Preview. Table functions. These user-defined table-valued functions return a table.
- 28 June 2021, GA. Casting features.
PARSE_BIGNUMERIC,PARSE_NUMERIC, theFORMATclause ofCASTforSTRING,DATE,DATETIME,TIME,TIMESTAMP, numeric andBYTEStypes, theINT64aliasesINT,SMALLINT,INTEGER,BIGINT,TINYINTandBYTEINT, andST_GEOGFROM. - 28 June 2021, GA.
ALTER COLUMN SET OPTIONS. - 25 June 2021, Preview. Table snapshots. A snapshot is a low-cost, read-only copy of a table as it was at a point in time.
- 22 June 2021, Preview. BigQuery ML anomaly detection, autoencoder and hyperparameter tuning.
ML.DETECT_ANOMALIESruns against time series withARIMA_PLUSmodels and against independent and identically distributed data withAUTOENCODERandKMEANSmodels. TheAUTOENCODERmodel type is available inCREATE MODEL, andML.PREDICTcan use it to reduce the dimensions of query results. Hyperparameter tuning searches for the best hyperparameters whileCREATE MODELtrains. - 21 June 2021, GA. Row-level security.
- 7 June 2021, Preview. Parameterized types.
STRING(L),BYTES(L),NUMERIC(P),NUMERIC(P, S),BIGNUMERIC(P)andBIGNUMERIC(P, S). - 20 May 2021, GA.
ST_STARTPOINT,ST_ENDPOINT,ST_POINTN. They return a point of a linestring geography as a point geography. - 19 May 2021, GA.
ALTER TABLE RENAME TO. - 18 May 2021, GA.
CREATE MODELfor AutoML Tables models. - 10 May 2021, Preview.
PIVOT,UNPIVOTandQUALIFY. - 30 April 2021, GA.
CREATE VIEWwith a column name list,ALTER COLUMN DROP NOT NULL. - 21 April 2021. Changing clustering. A non-clustered table can become clustered and the reverse, and the clustered columns of a clustered table can change. Google had documented this since October 2020 without a release note.
- 19 April 2021.
ARIMA_PLUSmodels. They replace theARIMAmodel type, which is deprecated.ARIMA_PLUSaddsML.EXPLAIN_FORECAST, a fullerML.ARIMA_EVALUATE, several ID columns throughtime_series_id_col, up to 500,000 time series forecast at once, the training optionsclean_spikes_and_dipsandadjust_step_changes, and theper_minutedata frequency. - 2 April 2021, Preview.
ALTER TABLE DROP COLUMN. - 2 April 2021. Name length limit. The maximum length of table column names, column alias names and user-defined function names rises from 128 to 300 characters.
- 31 March 2021, GA.
JSON_EXTRACT_STRING_ARRAY,JSON_QUERY_ARRAY,JSON_VALUE_ARRAY. - 31 March 2021, Preview. Partitions view.
INFORMATION_SCHEMAhas a view for table partitions. - 31 March 2021, Preview.
TABLESAMPLE. The operator queries random subsets of large tables. - 31 March 2021, GA.
CREATE SCHEMA,ALTER SCHEMA,DROP SCHEMA. - 31 March 2021, Preview.
DDLcolumn inINFORMATION_SCHEMA.TABLES. It holds the statement that recreates the table. - 31 March 2021, GA.
BIGNUMERICtype. - 25 February 2021, GA. Materialized views. They are precomputed views that cache the results of a query periodically.
- 12 February 2021. Cached results for Cloud Storage data. Standard SQL queries on Cloud Storage data can use cached query results.
- 29 January 2021, GA.
DATETIMEclustering columns.
2020
- 21 December 2020, Preview.
BIGNUMERICtype. It is meant for high-precision computations. - 23 November 2020, GA.
EXPORT MODEL. BigQuery ML models can be exported to Cloud Storage and used for online prediction. - 23 November 2020, GA. Boosted tree and DNN models.
CREATE MODELtrains boosted tree models with XGBoost and deep neural network models through AI Platform.ML.FEATURE_IMPORTANCEcovers the boosted tree models. - 26 October 2020, GA.
SUBSTRING. - 22 October 2020, Beta.
make_validandplanarparameters.ST_GEOGFROMGEOJSONandST_GEOGFROMTEXTtakemake_valid, which tries to correct polygon problems on import when set toTRUE.ST_GEOGFROMTEXTalso takesplanar, which treats imported WKT geometries as having planar edges when set toTRUE. - 14 October 2020, GA. Dynamic SQL.
EXECUTE IMMEDIATEgenerates and runs SQL statements at runtime. - 14 October 2020, GA. String and date functions.
ASCII,CHR,INITCAP,INSTR,LAST_DAY,LEFT,OCTET_LENGTH,REGEXP_INSTR,REGEXP_SUBSTR,RIGHT,SOUNDEX,TRANSLATEandUNICODE.REGEXP_EXTRACTtakes two more parameters, position and occurrence. - 14 October 2020, GA. Statements.
CREATE EXTERNAL TABLE,DROP EXTERNAL TABLE,ALTER TABLE ADD COLUMN,EXPORT DATAandTRUNCATE TABLE. - 14 October 2020.
DATEarithmetic operators. - 14 October 2020. Unicode table names.
- 14 October 2020, GA.
INFORMATION_SCHEMAviews.TABLES,TABLE_OPTIONS,COLUMNS,COLUMN_FIELD_PATHS,VIEWS,ROUTINES,ROUTINE_OPTIONS,PARAMETERS,SCHEMATAandSCHEMATA_OPTIONS. - 14 October 2020. Duplicate column names. A query result can have duplicate column names.
- 6 October 2020, GA. Authorized user-defined functions. They share query results without giving access to the underlying tables.
- 29 September 2020.
DATA_FREQUENCYfor multiple time series. Time series models accept aDATA_FREQUENCYother than the defaultAUTO_FREQUENCYwhenTIME_SERIES_ID_COLforecasts several series. - 21 September 2020, GA. Time-unit partitioning. Tables can be partitioned by hour, month or year, and by a
DATETIMEcolumn. - 11 September 2020.
BQ.JOBS.CANCEL. The system procedure cancels a job. - 27 August 2020, GA. Time series models. The
AUTO_ARIMA_MAX_ORDERtraining option is added. - 26 August 2020, GA. Reservations views.
INFORMATION_SCHEMAhas views for BigQuery reservations. - 25 August 2020. Jobs timeline views.
INFORMATION_SCHEMAhas views for jobs metadata by timeslice. - 19 August 2020. Consecutive
ONandUSINGclauses. Parentheses around the nested join are optional, soFROM A JOIN B JOIN C ON B.x = C.y USING (z)works likeFROM A JOIN (B JOIN C ON B.x = C.y) USING (z). - 18 August 2020. On-demand pricing with reservations. A project can be assigned to none, which keeps it on on-demand pricing.
- 17 August 2020, GA. Matrix factorization models.
CREATE MODELandML.RECOMMEND. - 11 August 2020. Flat-rate minimum. The minimum slot purchase is 100 slots, in 100-slot increments.
- 27 July 2020, Alpha. Streaming views.
INFORMATION_SCHEMAhas views with historical and real-time information about streaming into BigQuery. - 16 July 2020.
ST_CONVEXHULL,ST_DUMP.ST_CONVEXHULLreturns the smallest convexGEOGRAPHYthat covers the input.ST_DUMPreturns anARRAYof simpleGEOGRAPHYvalues, one per component of the input. - 15 July 2020. AutoML Tables data split options. Data split and validation options are available for AutoML Tables model training.
- 13 July 2020.
ASSERT. The statement checks that data matches an expectation. - 1 July 2020, Beta. Time series models.
CREATE MODELsupports time series models. - 29 June 2020, GA. Flex slots.
- 26 June 2020, Beta. Region-qualified
INFORMATION_SCHEMAviews. - 26 June 2020. Unqualified
SCHEMATAqueries. From mid-August, an unqualifiedINFORMATION_SCHEMAquery onSCHEMATAorSCHEMATA_OPTIONSreturns metadata from theUSmulti-region. - 16 June 2020, GA. Jobs views.
INFORMATION_SCHEMAhas views for jobs. - 16 June 2020, Preview. BigQuery ML and AI Platform.
CREATE MODELsupports AutoML Tables models, boosted tree models with XGBoost and deep neural network models. - 9 June 2020. Clustering for non-partitioned tables.
- 2 June 2020.
ST_SIMPLIFY. It returns a simplifiedGEOGRAPHYby replacing sections of the input with straight lines. - 2 June 2020. References in UDF and view bodies. The body of a standard SQL user-defined function can reference tables and views. The body of a standard SQL view can contain references without a project qualifier when the view is created through the
tables.insertAPI or lives in the project that runs theCREATE VIEWquery. - 20 May 2020, Beta. Hourly partitioned tables.
- 20 May 2020, GA. Cloud SQL federated queries.
- 20 May 2020, Beta. Dynamic SQL.
EXECUTE IMMEDIATEgenerates and runs SQL statements at runtime, in all regions. - 1 May 2020.
JSON_EXTRACT_ARRAY. It extracts the contents of a JSON document as a string array. - 1 May 2020.
NULLS FIRSTandNULLS LAST.ORDER BYaccepts both to place null and non-null values. - 22 April 2020, Beta. Exporting BigQuery ML models. Models can be exported to Cloud Storage and used for online prediction.
- 17 April 2020, Beta. Matrix factorization models.
CREATE MODELbuilds recommendation models. - 16 April 2020, GA. BigQuery Reservations. Reservations buy slots for flat-rate pricing and assign them to workloads.
- 16 April 2020. Dataset metadata views. Around the end of April, the
INFORMATION_SCHEMAdataset views return metadata for all datasets in a region instead of all datasets in the project across regions. A specific region can be queried, for exampleregion-us.INFORMATION_SCHEMA.SCHEMATA. Filtering on theLOCATIONcolumn ofSCHEMATAgives the same result before the change. - 8 April 2020, Beta. Materialized views.
- 31 March 2020, Alpha. Reservations views.
INFORMATION_SCHEMAhas views for BigQuery reservations. - 30 March 2020, GA. Scripting and stored procedures.
- 2 March 2020, GA. External data in ORC and Parquet, and hive partitioning. Queries can read ORC and Parquet files in Cloud Storage. Externally partitioned data in Cloud Storage with the default hive layout can be loaded and queried, in Avro, Parquet, ORC, CSV or JSON format.
- 26 February 2020. DML quotas. DML statements have no quota limits.
- 26 February 2020, Beta. Flex slots. They are available in all Reservations regions.
- 18 February 2020, GA. Integer range partitioning. Tables can be partitioned on an integer column.
- 14 February 2020, Beta. Jobs views.
INFORMATION_SCHEMAhas views for live and historical jobs metadata.
2019
- 20 December 2019, GA.
ALTER TABLEandALTER VIEW. They set options on tables and views. - 19 December 2019, GA. BigQuery ML preprocessing. The preprocessing functions and the
TRANSFORMclause handle feature engineering. - 13 December 2019, Beta. BigQuery Reservations. Reservations buy slots for flat-rate pricing and assign them to workloads, in the US and EU.
- 11 December 2019, Beta. Integer range partitioning. Tables can be partitioned on an integer column.
- 10 December 2019. On-demand query acceleration. On-demand queries can burst beyond the default slot capacity at no extra charge.
- 4 December 2019.
KMEANS++. It initializes the clusters of a k-means model and trains a better model than random initialization. - 26 November 2019.
INFORMATION_SCHEMApermission checks. A query against anINFORMATION_SCHEMAview returns anaccessDenied(403) error when the user lacks the IAM permission to read the metadata. Before, the results were filtered by those permissions.INFORMATION_SCHEMAis in beta. - 22 November 2019, GA. Persistent standard SQL UDFs.
- 22 November 2019, Beta. Routines metadata.
INFORMATION_SCHEMAviews return routine metadata. - 21 November 2019, Beta. BigQuery ML preprocessing.
- 21 November 2019. BigQuery ML encryption keys. ML models can be encrypted with your own Cloud KMS keys.
- 12 November 2019, GA. Omitting column names in
INSERTandMERGE. - 31 October 2019, Beta. External data in ORC and Parquet, and hive partitioning. Queries can read ORC and Parquet files in Cloud Storage. Externally partitioned data in Cloud Storage with the default hive layout can be loaded and queried, in Avro, Parquet, ORC, CSV or JSON format.
- 9 October 2019.
RANGE_BUCKET. The function scans a sorted array and returns the 0-based position of the upper bound of a given point. It helps group data into partitions, histograms and business-defined rules. - 3 October 2019, Beta. Scripting, stored procedures and temporary tables.
- 30 September 2019, GA. Importing TensorFlow models.
- 23 September 2019, GA. K-means clustering models.
- 27 August 2019. Google Sheets external tables. An external table linked to a Google Sheets file can name a single sheet or a cell range, not only the first sheet.
- 22 August 2019. Automatic re-clustering. BigQuery re-clusters all clustered tables itself, so manual re-clustering is no longer needed.
- 7 August 2019, Beta. Cloud SQL federated queries. A query can combine BigQuery and Cloud SQL data.
- 2 July 2019, Beta. Importing TensorFlow models.
- 1 July 2019, Beta. Persistent user-defined functions.
- 3 June 2019, Beta. Omitting column names in
INSERTandMERGE. - 29 May 2019, GA. BigQuery ML.
- 28 May 2019. Maximum query response size. It rises from 128 MB to 10 GB.
- 15 May 2019, GA. Clustering by
GEOGRAPHY. A table can be clustered by aGEOGRAPHYcolumn. - 14 May 2019.
DROP MODEL. The DDL statement deletes a BigQuery ML model. - 13 May 2019. Concurrent query limit. The rate limit for on-demand interactive queries rises from 50 to 100 concurrent queries.
- 10 April 2019. AEAD encryption functions.
- 10 April 2019, GA. Clustered tables.
- 10 April 2019. K-means models. BigQuery ML supports the k-means model type for clustering and customer segmentation.
- 4 April 2019, GA. BigQuery GIS.
- 18 March 2019.
CREATE MODELquery limit. The limit onCREATE MODELqueries rises from 100 to 1,000. - 18 March 2019.
ML.ROC_CURVE,ML.CONFUSION_MATRIX. Both work without input data. - 25 February 2019, GA.
MERGE. - 11 February 2019. DML quotas. The limit on combined
UPDATE,DELETEandMERGEstatements per table per day rises from 200 to 1,000. The limit of 10,000 such statements per project per day is removed. - 4 February 2019, Beta.
ALTER TABLEandALTER VIEWset options. - 4 February 2019, Beta.
INFORMATION_SCHEMA. The views return resource metadata. - 29 January 2019. Linear regression optimization strategies. BigQuery ML supports automatic, batch gradient descent and normal equation strategies.
2018
- 13 December 2018.
ML.PREDICTthresholds.ML.PREDICTsupports thresholds for binary logistic regression models. - 13 December 2018.
ML.WEIGHTSstandardization.ML.WEIGHTSsupports standardization. - 12 December 2018. Query length. The maximum length of a standard SQL query is 1 MB.
- 5 December 2018. Default partition expiration. A dataset can have a default partition expiration time, set or updated later.
- 19 October 2018. Label values in
CREATE MODEL. Multiclass logistic regression models support up to 50 unique label values, up from 10. - 11 October 2018. Model training and evaluation. The
randomdata split method is deterministic, so later training runs give the same split while the input data is unchanged. Input data forML.EVALUATEis optional. - 10 October 2018, GA. DML on partitioned tables.
- 10 October 2018, GA. Tables partitioned on a
DATEorTIMESTAMPcolumn. - 9 October 2018. Job location. A job that references resources outside the
USandEUregions no longer needs a location. - 25 September 2018.
GENERATE_UUID. - 13 September 2018. Multiclass logistic regression. BigQuery ML supports multiclass logistic regression models and
ML.CONFUSION_MATRIX. - 10 September 2018, Beta. BigQuery GIS.
- 20 August 2018, GA.
NUMERICtype. - 30 July 2018, Beta. Clustered tables.
- 25 July 2018, Beta. BigQuery ML.
- 25 July 2018, GA. Data definition language.
- 19 July 2018, Alpha. Clustered tables.
- 28 June 2018. Templated SQL UDFs.
- 28 June 2018.
GENERATE_TIMESTAMP_ARRAY. - 28 June 2018.
FROM_BASE32,TO_BASE32. - 15 May 2018, Beta.
NUMERICtype. - 4 May 2018. Quotas. A partitioned table can have up to 4,000 partitions, up from 2,500. The limit on combined
UPDATE,DELETEandMERGEstatements per table per day rises from 96 to 200. - 10 April 2018, Beta.
MERGE. - 30 March 2018.
ERRORand theSAFEprefix.ERRORreturns an error from an expression. TheSAFEprefix suppresses errors in a function call. - 5 March 2018. Create a table from a query. The data definition language can create a table from the result of a query.
- 5 March 2018, Beta. DML on partitioned tables.
- 8 February 2018, Beta. Tables partitioned on a
TIMESTAMPorDATEcolumn. - 17 January 2018, Beta. Data definition language.
- 5 January 2018.
ISOWEEKandISOYEAR.DATE_TRUNC,DATETIME_TRUNC,TIMESTAMP_TRUNC,DATE_DIFFandDATETIME_DIFFaccept them.
2017
- 30 November 2017.
FARM_FINGERPRINTin legacy SQL. - 30 November 2017.
_PARTITIONTIMEin wildcard tables. The pseudo column works when querying partitioned tables through a wildcard table. - 30 November 2017.
WEEK(<WEEKDAY>). It is a valid date part forEXTRACT. - 30 November 2017.
INTERSECT DISTINCTandEXCEPT DISTINCT. Standard SQL supports both set operators. - 14 November 2017. High compute queries. Opting in is no longer required, and a query under tier 100 is billed as tier 1 without a threshold flag. A query above tier 100 fails with a
RESOURCES_EXCEEDED_PER_BYTEerror unless it runs in a reserved instance. Users who ran one above tier 100 in the previous 30 days keep the higher billing tier until 31 October 2018. - 2 November 2017.
IGNORE NULLSandRESPECT NULLS.FIRST_VALUE,LAST_VALUEandNTH_VALUEaccept them. - 2 November 2017.
FOR SYSTEM_TIME AS OF. The expression is allowed in theFROMclause. - 29 September 2017.
GROUP BY ROLLUP. Standard SQL supports it. - 28 September 2017. Ingestion-time partitioned tables. Partitioning options given when query results or loaded data go into a new destination table create an ingestion-time partitioned table.
- 28 September 2017.
NORMALIZE,NORMALIZE_AND_CASEFOLD. - 1 September 2017.
PERCENTILE_CONT,PERCENTILE_DISC. Standard SQL supports both analytic functions. - 11 August 2017.
OVERfor statistical aggregates.COVAR_SAMP,COVAR_POPandCORRaccept anOVERclause. - 2 August 2017.
EXTRACTdate parts. Standard SQLEXTRACTsupportsWEEK,ISOWEEKandISOYEAR. - 26 July 2017.
TO_HEX,FROM_HEX. Standard SQL supports both. - 13 July 2017. DML on tables with required fields. DML statements can modify such tables. A
SELECTthat writes to a destination table can also update them. - 5 June 2017.
OVERfor statistical aggregates.STDDEV,STDDEV_POP,STDDEV_SAMP,VAR_POP,VAR_SAMPandVARIANCEaccept anOVERclause. - 2 June 2017.
TO_JSON_STRING. Standard SQL supports it. - 16 March 2017.
REVERSE,ARRAY_REVERSE. They reverseSTRING,BYTESandARRAYvalues. - 15 March 2017. HyperLogLog++ functions. They estimate the number of unique values in a large dataset.
- 15 March 2017.
ARRAY_AGGandSTRING_AGGas window functions. Both accept anOVERclause. - 8 March 2017, Beta. Cloud Bigtable external data sources.
- 8 March 2017, GA.
IGNORE NULLSandRESPECT NULLSinARRAY_AGGandAPPROX_QUANTILES. - 8 March 2017. DML limits. The daily limits for update and delete statements are higher.
- 8 March 2017, GA. DML for nonpartitioned tables.
- 1 February 2017. Standard SQL additions. Bitwise operators for
BYTES,BIT_COUNTforBYTES,LPAD,RPAD,REPEAT,FARM_FINGERPRINT, and theNETfunctionsNET.IP_FROM_STRING,NET.SAFE_IP_FROM_STRING,NET.IP_TO_STRING,NET.IP_NET_MASK,NET.IP_TRUNC,NET.IPV4_FROM_INT64andNET.IPV4_TO_INT64.
2016
- 22 December 2016. Hash functions. Standard SQL supports them.
- 22 December 2016.
TO_CODE_POINTS,CODE_POINTS_TO_BYTES,CODE_POINTS_TO_STRING. - 22 December 2016. URL functions. Standard SQL supports them.
- 22 December 2016. Parameterized queries. They help prevent SQL injection when a query is built from user input.
- 22 December 2016.
ORDER BYandLIMITinside aggregates.ARRAY_AGG,ARRAY_CONCAT_AGGandSTRING_AGGaccept both clauses. - 22 December 2016, Issue. Legacy SQL
OMIT...IF. A query returns an error whenOMIT...IFexcludes part of a record in a repeating field and the query also selects other independently repeating fields. Migrating the query to standard SQL avoids the error. - 30 November 2016. JSON functions. Standard SQL supports them.
- 1 November 2016. SQL variant prefix. A query prefix sets the SQL variant, legacy SQL or standard SQL.
- 1 November 2016. Safe casting functions. They return
NULLwhere a normal cast would raise an error. - 1 November 2016. Bracket notation in legacy SQL JSON functions.
- 29 September 2016, GA. Standard SQL. It follows the SQL 2011 standard, with extensions for nested and repeated data.
- 29 September 2016. DML for standard SQL. DML inserts, updates and deletes rows.
- 29 September 2016. Flat-rate pricing. It gives unlimited querying for a fixed monthly price.
- 2 June 2016. Time-partitioned tables.
- 2 June 2016, Beta. Standard SQL. It follows the SQL 2011 standard, with extensions for nested and repeated data.
- 6 May 2016. Google Drive data. A Google Drive document can be a federated data source, and a query can read one through a temporary table.
- 6 May 2016. Legacy SQL functions.
PARSE_PACKED_IPreturnsBYTES.FROM_BASE64,SHA1andTO_BASE64are added. - 23 March 2016.
BYTEStype.
2015
- 25 August 2015. User-defined functions.
- 25 August 2015. Slots and query pricing tiers. Slots guarantee resources regardless of demand on the shared pool.
- 25 August 2015. Quotas. The concurrent query rate limit rises to 50 queries and the daily query limit to 100,000.
- 25 August 2015. Cloud Storage data. Queries can read CSV and JSON data in Google Cloud Storage directly, as a federated data source.
- 16 April 2015. Undelete a table. A deleted table can be restored with table decorators.
- 16 April 2015. Query reference additions.
COALESCE,GROUP_CONCAT_UNQUOTED,RIGHT OUTER JOIN,FULL OUTER JOIN, and theROLLUPandGROUPINGmodifiers.
2014
- 25 March 2014. Table wildcard functions. They query a specific set of tables at lower cost.
- 25 March 2014. Joins.
CROSS JOINand multiple joins in one query are supported. - 25 March 2014. Views. A view is a virtual table defined by a SQL query.
- 25 March 2014. Batch queries. They no longer wait 30 minutes. A batch job starts as soon as idle resources are available, and becomes interactive if it has not started within 3 hours.
- 25 March 2014. JSON functions.
- 25 March 2014. Pricing model. Rates are lower and reserved capacity pricing is added.
2013
- 18 September 2013. Table decorators. They give time-based table snapshots.
- 18 September 2013. Aggregate functions. More aggregate functions are added.
- 11 June 2013. Large query results. A query can return large results.
- 11 June 2013. Query caching.
- 11 June 2013. Window functions.
- 14 March 2013.
GROUP EACH BY. It aggregates over more distributed values. - 14 March 2013.
JOIN EACH. It joins tables of any size. - 14 March 2013.
TIMESTAMPtype. Functionality for theTIMESTAMPdata type is added.
2012
- 28 August 2012. Batch queries. They run between 30 minutes and 3 hours later and cost less than interactive queries.
- 1 March 2012.
VARIANCE,STDDEV,AVG. The aggregate functions are supported. - 5 April 2012. Join behavior. Outer joins are not supported, and only the left-most table of a join can be large.
2011
- 1 October 2011. BigQuery V2.
JOINstatements are supported.