Aparch Spark is an open-source, parallel-processing framework that supports in-memory processing to boost the performance of big-data analysis applications. HDInsight is a managed Hadoop service. Use it deploy and manage Hadoop clusters in Azure. For batch processing, you can use Spark, Hive, Hive LLAP, MapReduce. Languages: R, Python, Java, Scala, SQL You can create an HDInsight Spark cluster using an Azure Resource Manager template. The template can be found in GitHub. References: https://docs.microsoft.com/en-us/azure/architecture/data-guide/technology-choices/batch-processing