Business Data Pipelines
How to Create a Business Data Pipeline
Interested in creating a BDP? Schedule an appointment to evaluate your business and domain needs by clicking the Get Started button.
Understanding Business Data Pipelines at BYU
Purpose of Business Data Pipelines
BDPs provide both transactional data (records of events) and analytical data (used for analysis and insights). The platform cam cater to various use cases depending on the domain's needs, whether transactional or analytical.
Understanding Business Data Pipelines at BYU
Business Data Pipelines (BDP), also known as “data mesh,” are secure channels designed for accessing and consuming BYU’s domain-specific data. Authorized users can request access to these data products through a Data Sharing Agreement (DSA).
What is Business Data Pipeline?
A BDP is a structured pathway that facilitates safe and efficient data sharing and integration across BYU departments. Only authorized users, who have obtained a DSA, can access the BDPs. The authorization process is managed through Alation, a catalog tool that helps users locate any available information within the pipeline.
Benefits of a Business Data Pipeline
1. Standardization and Normalization: BDPs ensure that data is consistently formatted and standardized. Users can trace the origin and calculation methods of the data, ensuring its accuracy and reliability.
2. Abstraction Layer: The pipeline acts as an abstraction layer, which means that data can be seamlessly connected and disconnected without disrupting the system. This abstraction allows for smooth integration and updating of information.
3. Enhanced Data Governance: Data owners have more control over their data, allowing them to govern, standardize, and manage it effectively.
4. Efficient Data Access: By meshing information from different sources, BDPs make fragmented data cohesively accessible across multiple organizations, saving valuable time and resources.
2. Abstraction Layer: The pipeline acts as an abstraction layer, which means that data can be seamlessly connected and disconnected without disrupting the system. This abstraction allows for smooth integration and updating of information.
3. Enhanced Data Governance: Data owners have more control over their data, allowing them to govern, standardize, and manage it effectively.
4. Efficient Data Access: By meshing information from different sources, BDPs make fragmented data cohesively accessible across multiple organizations, saving valuable time and resources.
Components of a BDP
• Domains: Categorized BYU data, each headed by a trustee and managed by stewards. These stewards appoint business experts who collaborate with BYU’s Office of Information Technology (OIT) to meet their data needs.
• Data Products: These include a product catalog for finding data, usage agreements to set sharing boundaries, usage trackers to monitor data usage, and performance metrics.
• Templates: Include datasets, interfaces, and pipelines for consistent data handling.
• Provisioning: Involves version control, updates, and configurations to maintain data integrity.
• Contracts: Detail usage agreements, terms, and billing policies, with OIT funding the platform for campus-wide use.
• Data Products: These include a product catalog for finding data, usage agreements to set sharing boundaries, usage trackers to monitor data usage, and performance metrics.
• Templates: Include datasets, interfaces, and pipelines for consistent data handling.
• Provisioning: Involves version control, updates, and configurations to maintain data integrity.
• Contracts: Detail usage agreements, terms, and billing policies, with OIT funding the platform for campus-wide use.