In the context of big data, columnar storage & processing has emerged as a central theme for software engineers and data professionals. Columnar data is not just a trend; it’s a response to the increasing need for efficient data analytics.
Watch Jeremy Taylor as he delves into the world of columnar systems. In this session, Jeremy will:
- Explain the practical differences between row-oriented and columnar data storage, and specific scenarios where columnar data demonstrates clear advantages, particularly in the domain of analytics and for long-term retention of data
- Examine the role of hardware advancements, such as in-memory processing, and their interaction with columnar data to achieve better performance results
- Explore the emergence of hybrid OLAP and OLTP systems, which aim to bridge the capabilities of transactional & analytical processing — particularly applicable where columnar data can impact API design and application UX
- Discuss real-time analytics capabilities and how they are being enhanced through columnar engines
- Discover the advances behind cloud-native columnar systems like Snowflake, BigQuery, and Redshift, that leverage the cloud’s scalability and flexibility to optimize data analytics
Central to the discussion will be the concept of “mechanical sympathy”: understanding the harmony between hardware and software for maximum performance.
Columnar data aligns with this principle, taking advantage of cache locality and ensuring data is read in a way that is sympathetic to the architecture of modern hardware.
By understanding the mechanics at play, software engineers can better design and optimize systems.
Key Takeaways
-
Obtain a clear understanding of columnar data and the underlying mechanics that make it effective
-
Learn about the major players and technologies shaping the columnar data landscape
-
Get actionable insights on integrating or transitioning to columnar storage, emphasizing the principle of mechanical sympathy
Slides
You can access Jeremy’s slides here.
Webinar speakers
Jeremy Taylor — XTDB Head of Product at JUXT (XTDB is an open source database for solving complex problems with dynamic and temporal data)
Before joining JUXT in 2019, Jeremy held a sales engineering position within IBM working on eCommerce systems. For over a decade, Jeremy has been fascinated by relational and non-relational databases, and is motivated by the promise of data abstractions that can improve developer productivity across domains with frequently evolving schemas.
Connect with Jeremy on LinkedIn, Twitter, and Bluesky
James Henderson — XTDB Engineering Lead at JUXT
James has been working on XTDB since 2019. James is an experienced software engineer, and has held leadership positions in companies ranging from an international investment bank to small startups. He writes Clojure, Java and Kotlin, and has been responsible for wider software architecture, data engineering and system administration.
Connect with James on LinkedIn
