While the craze of AI has been dominating several minds these days, there are two sides to reality that the majority of folks aren’t prepared for or don’t know the answer to. The first of course is that it’s not there yet. And, the second is that it’s getting there at an unprecedented pace.
Nevertheless, when the generational AI engines like GPT3 and GPT4 became all the rage via the Chat GPT interface, people are either thinking AI will completely replace them and/or they can simply rely on it for their job.
The truth lies somewhere in the middle (at least for now)
What’s my opinion?
These are tools. Just like an electric and ride lawn mower makes your job easier, these are additional tools in your toolbox which can help make you both more efficient and more lazy (they’re related aren’t they anyway).
Machine Learning Model
Let’s take a look at the stuff we do:
- Discovery – There are so many layers starting with the discovery of business concepts and their business keys. It starts with the conceptual and eventually you have to map the keys which is called entity and/or identity resolution. In DV 2.0 we have the extended taxonomy exercise. While it’s extremely difficult to replace the business user, there are algorithms within NLP and AutoML that can help us with this aspect.
- Profiling – This process has actually been more friendly to automation than any other process for the longest time and there are so many excellent tools that actually can do this for you.
- Metadata Management – AI can certainly help by analyzing metadata and using information generated from the metadata as well as the extended taxonomy exercise to forward engineer some models for us. Believe it or not, Dan had created a tool some 15-20 years ago that actually did this, and he had written all the code as a single programmer.
- Generating data loads – Almost all our friends in the automation space are already working on this and RPA (Robotic Process Automation) can help with this as well as generating documentation for the code. This is already a reality today.
- Next of course is the analytics. Once you have data in the data warehouse (enterprise memory archive of data), then you can run several algorithms that can learn from this data, besides standard reporting and analyst work.
Let’s explore number 5 a little more.
While I already know this, and even now trust humans more than machines for the most part, just for fun, I decided to ask the machine to answer the following question – “What specific AI algorithms can specifically used in business intelligence analytics based on data in a data warehouse?”
There are several AI algorithms that can be used specifically in business intelligence analytics based on data in a data warehouse. Here are some examples:
- Association Rule Learning: Association rule learning algorithms can be used to identify patterns and relationships between different variables in a data warehouse. By analyzing large datasets, these algorithms can identify frequent itemsets and association rules that can be used to inform business intelligence analytics.
- Clustering: Clustering algorithms can be used to group data points together based on their similarity or proximity. In a data warehouse context, clustering can be used to identify similar groups of customers or products, and to inform business intelligence analytics related to customer segmentation and product recommendations.
- Decision Trees: Decision tree algorithms can be used to classify data based on a series of binary decisions. In a business intelligence context, decision trees can be used to identify key factors that impact business outcomes, such as customer churn or product sales.
- Neural Networks: Neural network algorithms can be used to model complex relationships between different variables in a data warehouse. These algorithms can be used for predictive analytics, such as forecasting sales or predicting customer behavior.
- Time Series Analysis: Time series analysis algorithms can be used to analyze and model time-dependent data in a data warehouse. By identifying patterns and trends in time series data, these algorithms can inform business intelligence analytics related to forecasting and trend analysis.
Overall, these AI algorithms can be used to inform business intelligence analytics in a data warehouse context, and can help organizations gain insights and make data-driven decisions. It’s important to note that the specific algorithms used will depend on the needs of the organization and the data sources involved.
Not bad for a machine, huh? Still robotic though. Nevertheless, potentially useful.
Now, some of the best minds in the world are heading to Vermont for the WWDVC 2023. There’s only 10 more days till ticket sales close and there’s very limited seats left. Some of the leading folks in the DW automation space will be there, some of the leading minds in the industry will be there including a data mathematician. Those who aren’t there this year will definitely miss out.
You can still get tickets here -> World Wide Data Vault Consortium
On a lighter note, Wired did an experiment which gauged reactions of folks on the impact of AI on their jobs using a single type of AI (which was dumb to be honest), but interesting nonetheless. At the very least it’s entertaining.
