Notifications
Clear all

SHA1 and Informatica DEI


Posts: 4
Topic starter
(@ddecross)
New Member
Joined: 1 year ago

Hi,

We have fully embraced the Hash Key within our Data Vault architecture and personally I can not think of why I have not used this a long time ago.  But it has introduced an issue for us, and I am curious how others have dealt with it.  Our current Raw Data Vault/Business Data Vault is in HIVE.  We are using SHA1 as our hashing algorithm.  We have not had any issues in that we are using pyspark and it has no issues with SHA1.  We are now in the process of moving our Raw Data Vault/Business Data Vault to our Oracle platform.  We are also moving to Informatica DEI.  Sadly, SHA1 is not supported natively in DEI.  Has anyone worked around this issue and if so, how?  I know that we could use DEI to call a pyspark job but if we need to maintain the pyspark that seems to undermine the value proposition for DEI.  DEI will support MD5.  The concerns we have with this is first we will need to rebuild all our tables using MD5.  Second is if we do need to change our hash algorithm, is MD5 going to be around for a long time or is it on its way out and if so, what would be a good alternative.

 

Thanks.

Reply
8 Replies