The Global Data Management Community

Case Study: Deriving Spark Encoders and Schemas Using Implicits

Submitted by Anonymous (not verified) on Thu, 2020-02-13 10:30

Click to learn more about author Dávid Szakallas. In recent years, the size and complexity of our Identity Graph, a data lake containing identity information about people and businesses around the world, begged the addition of Big Data technologies in the ingestion process. We used Apache Pig initially, and then migrated to Apache Spark a […]
The post Case Study: Deriving Spark Encoders and Schemas Using Implicits appeared first on DATAVERSITY.