Ok, not going to lie, I rarely find anything of value in the dregs of r/dataengineering, mostly I fear, because it’s %90 freshers with little to no experience. These green behind the ear know-it-all engineers who’ve never written a line of Perl, SSH’d into a server, and have no idea what a LAMP stack is. Weak. Sad.
We used to program our way to glory, up hill both ways in the snow. All you do is script kiddy some Python code through Cursor.
A recent post on Data Modeling, specifically that data modeling is dead, caught my eye. A rare piece of gold mixed in the usual pile of crap. It some truth being spoken on the interwebs, hold onto your panties you bright eyed data zealot. I agree %100 with this sentiment.
DATA MODELING IS DEAD.
How is Data Modeling Dead?
Well, because this generation of milk toast Data Engineers were raised on a diet of Data Lakes and Lake Houses, from our uncaring and tyrant mothers called Databricks, Snowflake, and AWS. You think those purveyors of compute and ideals were interested academically in Data Modeling as a fundamental truth of life (unless you were talking about an RDS instance, and then only maybe).
Data Modeling died a slow and painful death, ignored by the community, all the while they pandered and fawned over their Modern Data Stack.
What say you? Oh, give me my beautiful Notebook attached to a never ending stream of Spark Compute. Oh, the joys, the Lake House architecture, that sweet and delicious Medallion Architecture in which we can dump a never ending stream of data without a second though about nit picky things like “normalization” or joining tables in a snowflake or star schema way.
Overwhelming noise from Saas Vendors and Missing Voices.
Why is Data Modeling dead? Because in the era of the Data Warehouse we (collectively) worshiped at the feed of Kimball and the Data Warehouse Toolkit. Sure, we still argued about Facts and Dimensions, but overall, we (collectively) agreed with a spit on the hand that Kimball data modeling was the north star towards which to march.
... continue reading