10:15 - 11:00
For many years now it has been said that data is the new oil. From everyday streaming services to financial decisions made by banks, everything is powered by data. Just like oil, handling large scale datasets is no simple job and poses many challenges. Equally, data flows through pipelines. This talk will guide you through building your first custom data pipeline. It will explain how data pipelines are designed and structured, introduce you to some of the technologies used such as: Apache Kafka, Amazon S3, and Google BigQuery and explore potential problem areas.