Clear all

Data Vault on Databricks

Posts: 3
Topic starter
Joined: 6 months ago

I'm working with a client to implement Data Vault on Databricks. Databricks delta tables make the architecture much more integrated between the SQL and noSQL kind of work loads. So, in theory should have the best of both worlds - ability to interrogate and manage the data in a relational manner using data vault 2.0 methodology as well as do complex analytics through python/sql/scala/r code. Couple of questions i have are - 

1. Do we have any reference implementation of data vault on databricks and any key learnings from them? 

2. Are there any automation tools endorsed with data vault standards that support databricks? All of the tools I've tried to look into, do not have this combination. They either need to be enhanced to support DV2.0 standards or need enhancement to support databricks/spark. 

3 Replies