Published Paper Details:

The exponential growth of data and the development of artificial intelligence (AI) have generated huge data lakes, which provide organizations big, unstructured data sets. Although data lakes have the promise to provide useful insights to organizations, organizations are not able to use these assets effectively for strategic decision-making. The objective of this study is to explore the application of generative AI on big data lakes to generate meaningful insights and create embeddings to enhance business intelligence (BI) operations. A significant research gap exists in the practical implementation of generative AI models to unstructured data in data lakes, especially with regard to efficiency in processing and scalability. Most existing methodologies focus heavily on traditional data processing techniques and do not have the attention of overcoming the complexity and size of data involved in data lakes. Moreover, although AI has been implemented with smaller and structured data sets with significant success, applying AI on big, heterogeneous data sets requires new strategies to facilitate proper interpretation of data and actionable results. This research provides a conceptual model incorporating generative AI models with data lakes to recover contextual embeddings and generate insights in support of better-informed business decision-making. Identifying the importance of feature extraction through automated methods, anomalies, and advanced predictive models, this research is an attempt to address the prevalent vacuum of awareness surrounding the cost-efficient large-scale applications of AI to analyze real-time data. Last but not least, this research is about enhancing decision-making capabilities, cutting costs, and promoting organizational response effectiveness in emerging data ecosystems.