You can use Databricks Pools to Speed up your Data Pipelines and Scale Clusters Quickly. Databricks Pools, a managed cache of virtual machine instances that enables clusters to start and scale 4 times faster. Reference: https://databricks.com/blog/2019/11/11/databricks-pools-speed-up-data-pipelines.html