Metadata Management Fundamentals
Back to Basics
Created by Inf Sid, Last Updated 22-Oct-2019, Language:English
What Will I Get ?
- What is Metadata?
- Learn about the importance of Metadata to an Enterprise
- The Business Value of Metadata for an Enterprise
- Types of Metadata
- Real Time Use Case 1: How to fetch Metadata from different Data Layers of an Enterprise Data Warehouse?
- Real Time Use Case 2:How to fetch Metadata from different Data Layers of a Hybrid Enterprise Data Warehouse involving Big Data?
- How does the different data layers of an application connect via Metadata?
- What problems does the Metadata Management Solves?
- What business questions does the Metadata Management answer?
- What are Data Assets?
- What is Data Lineage?
- What is Data Stewardship?
- What are the different domains from which Metadata can be fetched and liked for trace-ability?
- How does Metadata Management relate to Data Management?
- Data Warehouse Concepts
- Data Management Concepts
- BigData Concepts
Let’s get started with the basics of Metadata Management.
Setting up the Metadata Management initiative at the enterprise level serves a variety of purposes. Data Asset discovery, effective cataloging, identifying resources, defining them by criteria, bringing similar resources together and the most important of them all is to have a Self Service approach to the enterprise assets.
It also is an effective means of organizing enterprise data assets, which is the need of the hour considering the huge variety and volumes of data and new ways of doing business online. Typically, all the metadata is stored in documents or excel sheets which are not usually up to date just for the namesake and these documents do not really have what one needs. A more efficient way is to use metadata to define, build lineage, relationships and create trace-ability so that the enterprise data citizens can easily find the data, the systems in which the data resides and the owners or stewards of the data.
Investing in metadata management/development have a lot of benefits:
Simplify data discovery and history of the data with a record of content-rich data across the enterprise.
Most of the enterprises have to manage increasingly complex systems in various locations and on varied platforms. By managing metadata effectively, organizations can create an inventory of its data and learn about its transformation across the life cycle, along with the variety of meaning and formats, and locations of each data object.
Drive consistency through data reuse, to increase productivity and reduce time needed for project implementation.
A managed metadata environment serves as the most effective way to identify the appropriate data elements / objects needed for any use.
Metadata Management allows companies to retire unused storage, reducing costs, and reducing time that was spent in deciding among “possibly correct” variations of an attribute.
Retain staff knowledge that is lost when business rules, definitions and other forms of metadata are not documented. Often, business metadata remains only in the minds of certain employees. When these individuals leave the company, this knowledge disappears with them. Implementing an enterprise approach to managing metadata preserves this knowledge and reduces the risk of losing valuable contextual knowledge.
Increase confidence in the data delivered to business users. Tracking data lineage provides important context to business users. Profiling data in source systems by business data stewards and IT staff can resolve data errors, which will result in accurate, reliable, high-quality data presented in reports, queries and analytics.
Improve IT performance in development, impact analysis, data integration, change management, etc… All of these enhancements will enable greater cooperation between business and IT, and ultimately lower total costs of any systems initiative. Metadata helps IT understand what data exists, where it is located, and what it means, minimizing information complexity. The ability to assess the impact of potential changes based on improved knowledge of the data can help managers estimate project duration and resource costs more accurately.
What is Metadata?Preview00:02:50
Common Metadata Examples from our day to day work/activitiesPreview00:09:47
Why do we need to manage Metadata?Preview00:06:32
What questions does Metadata Management helps us to answer?Preview00:05:06
Types of Metadata
Metadata Origins in an Enterprise
Real Time Use Case : Using Enterprise Data Warehouse Applications
Real Time Use Case : How to fetch Metadata from BigData Tools and Technologies?
Data Management Consultant
Business Intelligence Consultant and Trainer with 16+ years of extensive work experience on various client engagements. Delivered many large data warehousing projects and trained numerous professionals on business intelligence technologies. Extensively worked on all facets of data warehousing including requirement gathering, gap analysis, database design, data integration, data modeling, enterprise reporting, data analytics, data quality, data visualization, OLAP. Has worked on broad range of business verticals and hold exceptional expertise on various ETL tools like Informatica Powercenter, SSIS, ODI and IDQ, Data Virtualization, DVO, MDM.