Clear all

Rebuilding PIT tables

Posts: 3
Topic starter
New Member
Joined: 1 year ago

Hi All,

In the event of data loss from either a bad migration or malicious party, is there any way to rebuild a PIT table? Reading the course material and trying to remember the topic, PIT tables can be rebuilt, however, that only seems to be true if the underlying Raw Vault tables are unaffected so that you can re-use your AS_OF tables and the Load Timestamp that is applied to each row as a Hard Rule.

In a scenario where by changes make it through the test process and into production that resulted in a Raw Vault Satellite table losing data in a given column, how would you be able to rebuild a PIT table? PIT tables are built based on time windows driven by the Record Load timestamp, which is the point in time data lands in the raw vault (hard rules). Given the data loss, the Raw Vault would need to be re-loaded from source / original data meaning all timestamps are the same - no time windows to rebuild the PIT?

Alternatively you could leave the PIT table and rebuild just the satellite, but you would break the system of record /audit and you would never be able to rebuild the PIT table should you ever need too (your underlying satellite would have different load timestamps to when PIT table was originally built)

Is it possible to recover from something like this? While I understand the best approach is to prevent these changes from making it to the Data Vault, but no system is always perfect and mistakes do happen.

Topic Tags
5 Replies