Data-Engineering

– Note: No LLM’s were harmed during the writing of this post, only my own brain. There’s such an overflow of interesting things out there related to data and software that it’s hard to keep track of them all. I’ve wanted to create a curated newsletter as a way to keep track of the projects I find interesting. Hopefully you can find some inspiration too! What I found interesting this week: Projects GammaVibe - a newsletter that generates start-up ideas. One idea per day.Explores GammaVibe, an automated startup idea generator, detailing its Python/PydanticAI tech stack and low-cost deployment architecture. https://gammavibe.com/newsletter The Ex-Googler Mirko Froehlich recently created this automated partly-free newsletter to generate start-up ideas. I subscribed to the free one, but there’s a more detailed paid version (5 dollars a month) available. ...

TLDR: In this blog, we’ll share key insights from our recent project migrating a sports sector client from SAS Viya to Snowflake and dbt, focusing on practical tips to make your migration smoother. • Snowflake and dbt offer a powerful combination of modern analytics due to their support for version control for SQL code, automated testing, and documentation. • We discuss practical tips such as using the dbt-audit-helper package for comparing migrated tables, the challenges of lift-and-shift vs remodeling, and the intricacies of incremental flows. A bit of background A client of ours in the sports sector had been relying on SAS Viya as their primary analytics platform for several years. The platform housed more than 50 analytical flows that had been built up over multiple years, representing significant business logic and institutional knowledge. These flows were important for the organization’s decision-making processes and downstream reporting. ...