The volume of data generated annually is increasing exponentially, with estimates predicting it will exceed 150 zettabytes in 2024. While data is often termed the "oil" of the new millennium due to its potential to enhance business processes, lower production costs, and drive scientific discoveries, deriving insights from this data can be challenging and time-consuming, often demanding substantial expertise to optimize databases for such analysis.

Optimizing database performance is crucial for achieving prompt query responses, which is a key aspect of database management. In this talk, I will discuss a new generation of self-driving, AI-powered databases developed by my group. These databases utilize machine learning and artificial intelligence (AI) to automate the complex and labor-intensive tasks of database optimization. AI-powered databases streamline the transition from data availability to insight delivery by automatically adjusting to the type of analysis required, thereby optimizing performance. I will focus on three critical challenges in optimizing database performance: a) selecting appropriate physical design structures (e.g., indexes, materialized views) to accelerate data retrieval, b) determining the layout of these physical design structures, with an emphasis on learned indexes tailored for disk setups, and c) deciding which data to cache in advance through (semantics-driven) prefetching to further enhance data retrieval.