As a cluster-computing framework, Spark is yet another impressive contribution to the IT industry by the venerable computer science department at the University of California, Berkeley. It has rapidly become a familiar part of big data and analytics applications: a natural follow-on to MapReduce, with a more holistic view of big data work flow, and designed to go fast. Unlike the strictly disk-based Hadoop, Spark can use memory.
“But we need it to go way faster,” is what we keep hearing from enterprise customers. And so, that became an important test as we designed and implemented in the Cancun MemoryLake™ platform.
The MemoryLake software is transparent (or frictionless as some of our customers called it) so it slides in and does its thing. If we could make something as fast as Spark go way faster, that would be a big win for our customers.
No surprises here, of course! Yes, the MemoryLake software does make Spark go way faster. We’d love to tell you all about it so please contact us for a live demo!