System Design - Data modeling for Datawarehouse

Designing a data model for a data warehouse is fundamentally different from designing the schema for your primary database. A data warehouse is meant for supporting aggregation queries that touch a huge number of records and are very expensive. The scale in terms of number of queries is at least an order of magnitude less than the number of queries seen by a consumer-facing primary database. This distinction leads to a fundamental different data layout and indexing model in the database used in a data warehouse. Such databases are called OLAP databases.

This course will take you into the internal architecture of an OLAP database with specific focus on areas that help you how a query runs on an OLAP database. We will walk though the levers that an OLAP database provides to optimize your queries. We will then cover various scenarios that will help you design the data model for an OLAP database used in a data warehouse.

Throughout this course, we will use AWS Redshift as the OLAP database. However, the principles covered in this course are general and can be applied in any data warehouse including Snowflake or  Hive. Some of these principles are also applicable in real-time column stores like HBase.