In theory, data lakes sound like a good idea: One big repository to store all data your organization needs to process, unifying myriads of data sources. In practice, most data lakes are a mess in one ...

Even though Spark and Delta are the perfect basis for your future data pipeline, and the Data Lakehouse architecture allows us to enable a tremendous amount of new use cases, the usability and ...

A monthly overview of things you need to know as an architect or aspiring architect. Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with ...

Low-Code Apache Spark and Delta Lake Historically, the need for cost-effective storage, performant processing, and low-latency querying required a two-tier architecture: a data lake for raw storage of ...