Google BigQuery gets metadata service with Iceberg support [Video]

The BigQuery metastore is designed to help enterprises cut down the complexities of metadata management as it supports multiple query engines.

Credit: Michael Vi / Shutterstock

Google Cloud is adding a new metadata service that is compatible with Apache Iceberg to its managed data analytics service BigQuery in order to help enterprises cut down complexities around metadata management.

Named BigQuery metastore, the fully managed unified metadata service will provide processing engine interoperability while enabling governance, Google principal engineer Yuri Volobuev and senior product manager Vinod Ramachandran wrote in a blog post.

On the governance front, metastore will enable automated cataloging and universal search, business metadata, data profiling, data quality, fine-grained access controls, data masking, sharing, data lineage, and audit logging.

Unlike traditional metastores and other metadata management systems that are tightly coupled with data processing engines, metastore will work with multiple engines, such as BigQuery, Apache Spark, Apache Hive, Apache Flink, and the Icebergtable …