- BigQuery - Home
- BigQuery - Overview
- BigQuery - Initial Setup
- BigQuery vs Local SQL Engines
- BigQuery - Google Cloud Console
- BigQuery - Google Cloud Hierarchy
- What is Dremel?
- What is BigQuery Studio?
- BigQuery - Datasets
- BigQuery - Tables
- BigQuery - Views
- BigQuery - Create Table
- BigQuery - Basic Schema Design
- BigQuery - Alter Table
- BigQuery - Copy Table
- Delete and Recover Table
- BigQuery - Populate Table
- Standard SQL vs Legacy SQL
- BigQuery - Write First Query
- BigQuery - CRUD Operations
- Partitioning & Clustering
- BigQuery - Data Types
- BigQuery - Complex Data Types
- BigQuery - STRUCT Data Type
- BigQuery - ARRAY Data Type
- BigQuery - JSON Data Type
- BigQuery - Table Metadata
- BigQuery - User-defined Functions
- Connecting to External Sources
- Integrate Scheduled Queries
- Integrate BigQuery API
- BigQuery - Integrate Airflow
- Integrate Connected Sheets
- Integrate Data Transfers
- BigQuery - Materialized View
- BigQuery - Roles & Permissions
- BigQuery - Query Optimization
- BigQuery - BI Engine
- Monitoring Usage & Performance
- BigQuery - Data Warehouse
- Challenges & Best Practices
BigQuery - BI Engine
In addition to optimization settings within BigQuery, BigQuery also offers a concurrent service, BI Engine, whose purpose is to scan and optimize BigQuery query performance.
- BI Engine is an in-memory service that analyzes the scope of job being run versus the amount of slots and compute resources available at the time of execution.
- Not only does BI Engine analyze query resources, it actively accelerates their execution (hence the "engine") after allocating the available resources.
- BI Engine is configurable and customizable, meaning that developers can choose which tables and views to include within its scope.
- BI Engine is a product within BigQuery. To reach the BI landing page, simply search "BI Engine" in the Cloud Console search bar.
The BI Engine page will prompt you to Create a Reservation.
Once you click on "Create a Reservation", you'll be able to configure how many gigabytes you would like to set within the scope of BI Engine and add the tables you would like to include within the scope of BI Engine's performance optimization capabilities.
Query Acceleration of BI Engine
Additionally, BI Engine syncs with the BigQuery API to provide query acceleration benefits to tables loaded, updated or modified from automated processes.
BI Engine's crowning achievement is vectorized runtime, which allows it to leverage cloud CPUs and enables it to compress data for seamless runs.
The real power of BI Engine lies in its ability to integrate with BigQuery-adjacent platforms and applications. For instance, a Looker dashboard that creates data based on BigQuery queries would be eligible for BI Engine acceleration.
Use Cases of BI Engine
BI Engine would most benefit a user with high-volume data tables that are queried routinely.
BI Engine use cases include −
- Resource-intensive visualizations powered by BigQuery.
- You have specific large and frequently-queried tables.
- Multiple users who query resources at similar times and cause performance log jams.
Regardless, BI Engine remains a powerful optimization strategy for any user seeking to make processes more efficient and less compute-intensive.