Anz learnings

   dbt uses ur keys from delta n definition n then identify what chged and adds processing col 23/24-scd2 config( materialized='delta', --table loading mechanism unique_key='id',  --PK of table stratergy='check', --mechanism how to handle new incoming data,always check  .data need to be checked in existing table with new incoming data and compared. properties={"partitioned_by":"ARRAY['dbt_valid_from']"},  --any specific properties that the table shd have  target_schema='pcc',  --schema where table will deploy check_cols="all",  --which col need to tested against for scd2 compare invalid_hard_deletes=True,  --mechanism to tell type 23 ,24, by default value is false and will type 24. if made true it is 23. updated_at=var('BUSINESS_DATE')  --this config set 'dbt_valid_from' date (=start date) when no value is set default to 'CURRENT_DATE' this can be passed via dt config or while falling dt run o...

1. WHAT IS GCFR > The GCFR (Global Control Framework) is an  ETL framework developed by Teradata. > The GCFR is a set of architectural standards, development methods, processing patterns and a code library that provides the basis for implementing a  true time variant (History)data warehouse. > GCFR tells how the data is loaded (tx type) and what data is loaded (tx) in the target table. (work as filter between source and target)  Purpose The Control Framework enforces audit trail and reduces programmer effort. It provides: -Full data and process lineage -Ability to prove data from source to end uses - Automated audit function 1.1 GCFR Processing Patterns IPO (input-process-output) model: > Each processing pattern in data warehouse adheres to the IPO model. > Data flow is from Input to Process to Output. >This simple model  applies to all data flows within the GCFR. INPUT -A single input to the process. The view that presents the input data set to the...