ApacheCon EU 2014 has ended
Register Now for ApacheCon Europe 2014 - November 17-21 in Budapest, Hungary. 

Tuesday, November 18 • 3:50pm - 4:40pm
Improving Spark Application Performance - William Benton, Red Hat

Sign up or log in to save this to your schedule and see who's attending!

Apache Spark presents an elegant and powerful set of high-level abstractions for developing distributed data-processing applications.  Analysts who use Spark can rapidly prototype applications and experiment with new techniques at scale.  However, to make the most of Spark, developers need to understand both the abstractions and how Spark will schedule and execute their code.  

This talk will show you how to improve Spark application performance by working with, not against, Spark's operational model.  We'll start with a real prototype Spark application and apply several simple, generally applicable transformations to make it more efficient and scalable.  For each transformation, we'll look both at *why* it works, considering the relevant details of Spark's internals, and *how well* it works, considering its impact on overall application performance.  You'll leave this talk with an improved understanding of how Spark runs your code and some additional tools to make your big data apps even more efficient.

avatar for William Benton

William Benton

Senior Principal Software Engineer, Red Hat, Inc.
William Benton leads a team of data scientists and engineers at Red Hat, where he has applied machine learning to problems ranging from forecasting public cloud costs to designing cycling workouts. His other professional experience includes static analysis, logic programming, and... Read More →

Tuesday November 18, 2014 3:50pm - 4:40pm

Attendees (0)