Join us on a literary world trip!
Add this book to bookshelf
Grey
Write a new comment Default profile 50px
Grey
Subscribe to read the full book or read the first pages for free!
All characters reduced
Data Engineering with Scala and Spark - Build streaming and batch pipelines that process massive amounts of data using Scala - cover

Data Engineering with Scala and Spark - Build streaming and batch pipelines that process massive amounts of data using Scala

Eric Tome, Rupam Bhattacharjee, David Radford

Publisher: Packt Publishing

  • 0
  • 0
  • 0

Summary

Most data engineers know that performance issues in a distributed computing environment can easily lead to issues impacting the overall efficiency and effectiveness of data engineering tasks. While Python remains a popular choice for data engineering due to its ease of use, Scala shines in scenarios where the performance of distributed data processing is paramount.This book will teach you how to leverage the Scala programming language on the Spark framework and use the latest cloud technologies to build continuous and triggered data pipelines. You’ll do this by setting up a data engineering environment for local development and scalable distributed cloud deployments using data engineering best practices, test-driven development, and CI/CD. You’ll also get to grips with DataFrame API, Dataset API, and Spark SQL API and its use. Data profiling and quality in Scala will also be covered, alongside techniques for orchestrating and performance tuning your end-to-end pipelines to deliver data to your end users. By the end of this book, you will be able to build streaming and batch data pipelines using Scala while following software engineering best practices.
Available since: 01/31/2024.
Print length: 300 pages.

Other books that might interest you

  • Escaping from Eden - Does Genesis teach that the human race was created by God or engineered by ETs? - cover

    Escaping from Eden - Does...

    Paul Wallis

    • 0
    • 0
    • 0
    The familiar stories of the book of Genesis affirm that God made the universe, planet Earth, and you and me. However, various anomalies in the text clue us that we are not reading the original version of these stories. So what were the original narratives and what did they say about who we are and where we all came from? What was the earlier story of human origins, almost obliterated from the Hebrew Scriptures in the 6th century BC, and suppressed from Christian writing in the 2nd and 3rd centuries AD? And what does any of this have to do with extraterrestrials? 
    Escaping from Eden will take you on a journey around the world and into the mythologies of ancient Sumeria, Mesoamerica, India, Africa, and Greece, to reveal a profound secret, hidden in plain sight in the text of the Bible. Far reaching and deepy controversial, this book points to truths about ourselves, the universe and everything which you may have long suspected but not dared to speak!
    Show book
  • Artificial Intelligence and Animal Ecology - A Review - cover

    Artificial Intelligence and...

    Amiyangshu De, Lidia Ghosh

    • 0
    • 0
    • 0
    Artificial Intelligence and Animal Ecology: A Review explores the transformative synergy between AI and animal ecology, unveiling how cutting-edge technology is revolutionizing ecological research and conservation. This pioneering book bridges these dynamic fields, demonstrating how AI techniques―such as evolutionary algorithms and optimization methods―both draw inspiration from and advance the study of animal behavior, species interactions, and environmental adaptation. With a strong focus on innovation, it examines groundbreaking AI applications, from bio-inspired algorithms and adaptive learning to breakthroughs in animal communication and behavioral analysis. Listeners will gain valuable insights into how AI deciphers complex ecological dynamics, including navigation, vocal communication, and interspecies relationships. The book also addresses ethical considerations, ensuring responsible AI integration in ecological research. 
    More than just a review, this book is a call to action. It empowers researchers, conservationists, and ecologists to embrace AI-driven solutions, fostering interdisciplinary collaboration and expanding the frontiers of ecological knowledge. As AI continues to evolve, Artificial Intelligence and Animal Ecology: A Review provides a vital roadmap for addressing environmental challenges with innovation and a deeper appreciation of the natural world.
    Show book
  • The Dopamine Trap - The Clinical Truth About the Modern Brain - cover

    The Dopamine Trap - The Clinical...

    Gregory Bicaj

    • 0
    • 0
    • 0
    You are not weak. You are not lazy. You are not broken. Your brain is simply running the wrong program for the world it lives in.
    In The Dopamine Trap, Albanian-Swiss neuroscientist and clinical psychologist Gregory Bicaj draws on twenty years of laboratory research and direct clinical work to explain why modern life leaves so many people anxious, distracted, exhausted, and strangely empty—despite having more comfort, convenience, and entertainment than any previous generation.
    The answer lies in dopamine. Not the "pleasure chemical" of popular mythology, but the brain's engine of wanting, anticipation, and craving. Smartphones, social media feeds, news cycles, and streaming platforms are not neutral tools. They are environments engineered to keep the brain's reward system perpetually activated—producing a state of chronic overstimulation that quietly degrades focus, mood, patience, creativity, and the capacity for genuine satisfaction.
    Bicaj introduces his "Three Brains Principles"—a working model grounded in evolutionary neuroscience—to show that most suffering and most foolish decisions arise not from personal failure, but from the wrong brain system being in charge at the wrong moment. With this framework, the chaos of compulsive phone-checking, emotional reactivity, and chronic restlessness becomes not a character flaw but a legible biological problem with practical solutions.
    Moving from neuroscience to clinic to daily life, The Dopamine Trap offers a complete, step-by-step path back to a mind that can think deeply, rest fully, and feel real pleasure again. Readers will learn:
    
    - Why dopamine fuels wanting far more than liking—and how that gap traps modern people in habits they do not enjoy
    - How variable-ratio reinforcement makes smartphones function like slot machines, and why willpower alone cannot break the cycle
    - How to measure their true "stimulation load" using practical audits from cognitive psychology research
    - How to run a 48-hour diagnostic reset that reveals the difference between preference and dependence
    - How to rebuild the brain's default mode network—the system responsible for creativity, self-reflection, and meaning-making—through structured rest
    - How to reintroduce pleasure safely without reigniting compulsive patterns
    - How to redesign their environment so that the easiest path leads toward focus instead of distraction
    - Why boredom is not the enemy of a meaningful life but one of its most important preconditions
    - A complete 30-day discipline playbook built on implementation intentions, identity-based change, and habit science
    Through vivid case studies, landmark research from Bruce Alexander, Anna Lembke, Gloria Mark, Anders Ericsson, Mihaly Csikszentmihalyi, and others, and concrete clinical experience, Bicaj shows that recovery is not about eliminating pleasure or achieving monastic discipline. It is about recalibrating a reward system that has been pushed far beyond its design limits—so that ordinary life, real conversation, sustained thought, and quiet moments can feel like enough again.
    For readers of Jonathan Haidt and Gabor Maté, The Dopamine Trap is both a rigorous explanation and a compassionate, practical guide. The goal is not a perfect brain. It is a freer one.
    Show book
  • Your First Steps in AI - cover

    Your First Steps in AI

    Tom Henricksen

    • 0
    • 0
    • 0
    Don't be left behind in the AI revolution! This concise ebook provides a clear and accessible introduction to the key concepts driving Artificial Intelligence. You'll learn about:Machine Learning: How algorithms learn from data to make predictions.Deep Learning: The power of neural networks and their applications.Generative AI: Exploring the exciting world of AI that creates new content.And much more! 
    Whether you're a curious individual or a professional seeking to understand the future of technology, this ebook is your starting point for the AI journey.
    Show book
  • DataOps - The Essential Guide to Data Operations for Modern Enterprises - cover

    DataOps - The Essential Guide to...

    Ben Rizer

    • 0
    • 0
    • 0
    DataOps: The Essential Guide to Data Operations for Modern Enterprises is an in-depth guide for businesses and data professionals seeking to transform their data pipelines and streamline operations in today’s data-centric world. This comprehensive resource demystifies the world of DataOps, detailing how it builds on DevOps principles to bring agility, automation, and quality to data processes. 
    From data engineers and analysts to data scientists and business leaders, this book offers readers a solid foundation in DataOps concepts, along with practical insights on implementing continuous integration and deployment (CI/CD), data quality management, governance, and automation in data pipelines. Each chapter is packed with step-by-step guidance, real-world examples, and actionable strategies to tackle common challenges in data operations and to ensure efficient, secure, and scalable data workflows. 
    Learn how to build and optimize robust DataOps workflows, foster a collaborative data-driven culture, and future-proof your data strategy with emerging technologies like AI, real-time processing, and IoT. Dive into case studies from industry leaders like Netflix and Uber to see how DataOps can drive innovation, and explore the book's roadmap for starting or scaling a DataOps initiative in your organization. 
    DataOps provides the essential roadmap for turning data into insights faster and more effectively. 
     
    Show book
  • Mind Hacking Secrets - Proven Strategies and Hacks to Double Your Reading Speed Make Calculations Faster Skyrocket Your Productivity and Unleash the Full Potential of Your Brain - cover

    Mind Hacking Secrets - Proven...

    Thomas Scofield

    • 0
    • 0
    • 0
    Would you like to double your reading speed without losing comprehension? Can you work out how much a $30-per-month gym membership will cost over 24 months without writing anything down or using a calculator? Do you want to improve your focus, memory, and productivity while training your brain? Improving your reading speed and developing mental-math skills can have a major impact on daily life. Imagine getting through emails in half the time, calculating discounts on the fly, or solving everyday quantity problems quickly. In this two-book collection, you’ll learn proven techniques to at least double your reading speed and improve comprehension, along with practical strategies to strengthen your math skills, make calculations faster, and solve everyday math problems more easily. This is not a boring collection of theory: it offers helpful, to-the-point techniques for reading faster and simplifying mental calculations. You’ll learn: 12 proven techniques to at least double your reading speed in just a few minutes a day, practical strategies for solving everyday math problems, an effective technique for reading 750–1,000 words per minute, how to quickly multiply any number by 3, 4, 5, 6, 9, and 10, how to calculate discounts on the fly, how to estimate mortgages and down payments mentally, 5 steps to improve comprehension and memory, common beginner mistakes and how to avoid them, and how classical music and chewing gum may support reading comprehension. Scroll to the top and click Buy!
    Show book