Operational hazards of managing PostgreSQL DBs over 100TB

Picture this: you start a new role, eager to learn and contribute with your ideas, and your next task is to get familiar with the database setup, and you start seeing these massive PostgreSQL databases - 100TB, 200TB, 300TB… How do you actually manage all this data? How do you keep storage under control? And more important: how do you keep queries fast when you're dealing with hundreds of terabytes? Here's the reality: what works at 100GB completely breaks at this scale. When reading blog articles, the best practices, the "how to" guides, things seem straightforward, but when you start bending PostgreSQL limits, you will end up needing to question the most fundamental assumptions on about how PostgreSQL works. Over the last years, my team has been exploring the boundaries of what PostgreSQL can do and today I will share our findings with you (at least the ones I can!).

Teresa Lopes

Teresa has worked with databases for almost 10 years. After several years as an Oracle DBA, she transitioned to PostgreSQL and never looked back! Extensibility and the Community are the two main aspects she finds fascinating about PostgreSQL. Teresa is also part of the PGDay Lowlands organization. Teresa started her career far away from databases, as a Civil Engineer. So, if you've always wanted to know how tunnels and bridges are built, just ask! In her free time, Teresa enjoys spending time outdoors (mainly admiring rocks), hiking, and cooking.