Databricks Workload Optimization: Best Practices for Visibility, Performance and Savings

Overview

High Databricks costs, surprise bills, or underperforming clusters? In this session, you’ll learn practical strategies for optimizing Databricks workloads: how to spot expensive queries eating up your budget, improve cluster efficiency, and make smarter decisions about resource allocation to keep jobs running smoothly and costs in check. And, find out how nOps can help you achieve the best bang for every buck you spend on Databricks.

Key Learning Areas

1. Cluster Optimization Best Practices:

• Choosing the right VM size and type.
• Managing contract vehicles (On-Demand, Spot, RIs, and Savings Plans).

2. Performance Tuning Techniques:

• Leveraging tools like Adaptive Query Execution (AQE) and Delta Cache to boost query performance and reduce runtime costs.

3. Cost Governance Strategies:

• Using cluster tags to track and manage resource usage by team or project, improving cost visibility and accountability.

4. Supercharge costs & performance with nOps + Databricks 

• How to integrate Databricks with nOps to analyze your spending and save 30-60%