Notifications
Clear all

Efficient way to load Business Vault/Info mart incrementally?


Posts: 17
Topic starter
(@saranyakandasamy)
Active Member
Joined: 2 years ago

We are in the process of building a near-real time data warehouse in Snowflake. The changed data from source is loaded to Raw data vault in 5 mins increments with the changes that happened in the last 5 mins. Now we are in the process of creating Info marts which would be refreshed with the same frequency as the Raw data vault. We do not have any downtime to load the infomart so all the data need to be loaded incrementally through out the day. The views would not give performance with the amount of joins involved. Creating Bridges (to feed the facts) and PITS (to feed the dimensions) incrementally every 5 mins seems to be a challenge. The bridges are created from the Hubs and Links with if not exists. Later the Bridges are joined with effectivity sats to get the effective records. Any experience in efficiently loading the business vault/info mart objects incrementally 24/7, in the smallest time window?

Reply
1 Reply