Join us on a literary world trip!
Add this book to bookshelf
Grey
Write a new comment Default profile 50px
Grey
Subscribe to read the full book or read the first pages for free!
All characters reduced
Data Engineering with Scala and Spark - Build streaming and batch pipelines that process massive amounts of data using Scala - cover

Data Engineering with Scala and Spark - Build streaming and batch pipelines that process massive amounts of data using Scala

Eric Tome, Rupam Bhattacharjee, David Radford

Publisher: Packt Publishing

  • 0
  • 0
  • 0

Summary

Most data engineers know that performance issues in a distributed computing environment can easily lead to issues impacting the overall efficiency and effectiveness of data engineering tasks. While Python remains a popular choice for data engineering due to its ease of use, Scala shines in scenarios where the performance of distributed data processing is paramount.
This book will teach you how to leverage the Scala programming language on the Spark framework and use the latest cloud technologies to build continuous and triggered data pipelines. You’ll do this by setting up a data engineering environment for local development and scalable distributed cloud deployments using data engineering best practices, test-driven development, and CI/CD. You’ll also get to grips with DataFrame API, Dataset API, and Spark SQL API and its use. Data profiling and quality in Scala will also be covered, alongside techniques for orchestrating and performance tuning your end-to-end pipelines to deliver data to your end users. 
By the end of this book, you will be able to build streaming and batch data pipelines using Scala while following software engineering best practices.
Available since: 01/31/2024.
Print length: 300 pages.

Other books that might interest you

  • The Quiltmaker's Butterfly Forest - Appliqué 12 Beautiful Butterflies & 6 Wreaths; 8 Fusible Projects - cover

    The Quiltmaker's Butterfly...

    Felicia T. Brenoe

    • 0
    • 0
    • 0
    Let your creativity take flight with eight whimsical projects inspired by the beautiful butterflies of the Amazon rainforest. Discover new quilting inspiration—from Ecuador’s Tiputini Biodiversity Research Station in the Amazon rainforest, where designer Felicia Brenoe was mesmerized by the sights that surrounded her, especially the butterflies. With this book, you can sew eight projects that capture nature’s beauty, beginning with the magnificent 12-block sampler of butterflies and wreaths—an ideal block-of-the-month quilt. Introduce contrast, embellishment, and negative space as you sew other large quilts, pillows, wallhangings, and placemats. Links to full-size appliqué patterns are included, plus an easy guide to raw-edge fusible appliqué.
    Show book
  • Behold the Black Caiman - A Chronicle of Ayoreo Life - cover

    Behold the Black Caiman - A...

    Lucas Bessire

    • 0
    • 0
    • 0
    In 2004, one of the world’s last bands of voluntarily isolated nomads left behind their ancestral life in the dwindling thorn forests of northern Paraguay, fleeing ranchers’ bulldozers.  Behold the Black Caiman is Lucas Bessire’s intimate chronicle of the journey of this small group of Ayoreo people, the terrifying new world they now face, and the precarious lives they are piecing together against the backdrop of soul-collecting missionaries, humanitarian NGOs, late liberal economic policies, and the highest deforestation rate in the world.  Drawing on ten years of fieldwork, Bessire highlights the stark disconnect between the desperate conditions of Ayoreo life for those out of the forest and the well-funded global efforts to preserve those Ayoreo still living in it. By showing how this disconnect reverberates within Ayoreo bodies and minds, his reflexive account takes aim at the devastating consequences of our society’s continued obsession with the primitive and raises important questions about anthropology’s potent capacity to further or impede indigenous struggles for sovereignty. The result is a timely update to the classic literary ethnographies of South America, a sustained critique of the so-called ontological turn—one of anthropology’s hottest trends—and, above all, an urgent call for scholars and activists alike to rethink their notions of difference. 
    Show book
  • Mastering OpenCV with Python - Use NumPy Scikit TensorFlow and Matplotlib to learn Advanced algorithms for Machine Learning through a set of Practical Projects - cover

    Mastering OpenCV with Python -...

    Ayush Vaishya

    • 0
    • 0
    • 0
    Unlocking Visual Insights: OpenCV Made Simple and Powerful.???Book Description"Mastering OpenCV with Python" immerses you in the captivating realm of computer vision, with a structured approach that equips you with the knowledge and skills essential for success in this rapidly evolving field.From grasping the fundamental concepts of image processing and OpenCV to mastering advanced techniques such as neural networks and object detection, you will gain a comprehensive understanding. Each chapter is enriched with hands-on exercises and real-world projects, ensuring the acquisition of practical skills that can be immediately applied in your professional journey.This book not only elevates your technical proficiency but also prepares you for a rewarding career. The technological job landscape is constantly evolving, and professionals who can harness the potential of computer vision are in high demand.By mastering the skills and insights contained within these pages, you will be well-prepared to explore exciting career opportunities, ranging from machine learning engineering to computer vision research. This book is your ticket to a future filled with innovation and professional advancement within the dynamic world of computer vision.
    Table of Contents
    1. Introduction to Computer Vision
    2. Getting Started with Images
    3. Image Processing Fundamentals
    4. Image Operations
    5. Image Histograms
    6. Image Segmentation
    7. Edges and Contours
    8. Machine Learning with Images
    9. Advanced Computer Vision Algorithms
    10. Neural Networks
    11. Object Detection Using OpenCV
    12. Projects Using OpenCV
    Index
    Show book
  • ColdFusion Presents - New Thinking: From Einstein to Artificial Intelligence the Science and Technology that Transformed Our World - cover

    ColdFusion Presents - New...

    Dagogo Altraide

    • 0
    • 0
    • 0
    What can history's greatest breakthroughs in science and technology teach us about the future? 
    New Thinking: The world has never been so unstable. Now, more than ever, we need to understand our history, learn from our mistakes, and embrace science and technology as we push into the future. In his debut book, New Thinking: The Technology and Science That Transformed Our World, Dagogo Altraide of YouTube's ColdFusion breaks down the history of gamechanger breakthroughs in the world of science and technology that have shaped our modern world and will impact the future. 
    Disruptive technology and innovation: The Industrial Revolution was the most significant event in human history since the domestication of animals and plants, leading to fifty years of growth and development making the western world almost unrecognizable. Today another revolution is taking place, and at its core is disruptive technology and innovation. It's clear that the pace of technology is increasing rapidly, and in the past 100 years there has been more change due to disruptive technology than we could have ever imagined. The pace of this disruptive technology revolution is swift, almost exponential, but what's the story behind it? What were those special moments in time that changed the future forever? 
    From the steam engine revolution to the electric world of Tesla, the first photograph to the invention of the internet, this book explores the hidden secrets of science and technology to help us understand each gamechanger that has shaped the future. 
    In New Thinking: The Technology and Science That Transformed Our World you will delight in learning about and appreciate: 
    How a technology can spawn a new technology, and how they influence each otherHow our modern world came to beOur incredible modern world and potential for the future
    Show book
  • Ultimate Salesforce LWC Developers’ Handbook - Build Dynamic Experiences Custom User Interfaces and Interact with Salesforce data using Lightning Web Components (LWC) for Salesforce Cloud - cover

    Ultimate Salesforce LWC...

    Cihan Hizar

    • 0
    • 0
    • 0
    Unleash Lightning Web Components (LWC) for Salesforce CloudBook DescriptionEmbark on a transformative journey into the world of Salesforce Lightning Web Components (LWC) with this comprehensive guide. From laying the foundation to honing advanced skills in LWC debugging and testing, each chapter seamlessly builds upon the last, empowering developers to navigate the intricate world of Salesforce LWC development. The book begins by creating your first LWC and delving into the intricacies of Lightning Web Components. It explores the seamless integration of Salesforce Data and master event management with Events in LWC.
    Next, it moves on to navigate the relationship between LWC and Flows, unlocking the potential of Lightning Out for standalone applications. Readers will dive deep into the aesthetics with Salesforce Lightning Design System, enhancing their components with style. It will also help you safeguard your applications by understanding LWC Security measures and learn debugging and testing in the LWC environment. In the final chapters, immerse yourself in working with Aura components and seamlessly migrate them to LWCs, providing an insightful transition to advanced development techniques.
    Table of Contents
    1. Introduction
    2. Creating Your First LWC
    3. LWC in Detail
    4. Using Salesforce Data
    5. Events In LWC
    6. LWC and FLOWS
    7. Lightning Out
    8. Salesforce Lightning Design System
    9. LWC Security
    10. LWC Debugging and Testing
    11. APPENDIX 1: AURA-LWC Co-Existence
    12. APPENDIX 2: Migrate Aura Components to LWCs
    Index
    Show book
  • Butterfly Safari - cover

    Butterfly Safari

    Andrew Fusek Peters

    • 0
    • 1
    • 0
    Andrew Fusek Peters has spent five years travelling around the UK to document and celebrate British butterflies. In Butterfly Safari, he shows butterflies in a new light, capturing close-ups of wing scales and the delicate structure of eggs, eyes and antennae. He has also pioneered never-before-seen flight shots and astonishing aerial sequences. Many of these photos have already appeared in the national papers and magazines. 
     
    'With so many of our butterflies in decline, this is a very timely book by one of the country's best photographers. It's a must-have for the bookshelves!' Iolo Williams 
     
    'This book captures the glorious beauty and heart-lifting spirit of butterflies better than any other.' Patrick Barkham, author of The Butterfly Isles 
     
    'This book acptures the beauty and movement of butterflies like no other. The flight sequences are sublime - they are pure poetry in motion.' Dr Martin Warren, former Chief Executive of Butterfly Conservation
    Show book