Databricks Tools

Databricks recently introduced Databricks Apps, a toolkit designed to simplify AI and data application development. By integrating native development platforms and offering automatic provisioning of serverless compute, the toolkit enables customers to more easily develop and deploy applications.

Thank you for reading this post, don't forget to subscribe!

Databricks Apps builds on the existing capabilities of Mosaic AI, which allows users to integrate large language models (LLMs) with their enterprise’s proprietary data. However, the ability to develop interactive AI applications, such as generative AI chatbots, was previously missing. Databricks Apps addresses this gap, allowing developers to build and deploy custom applications entirely within the secure Databricks environment.

According to Donald Farmer, founder and principal of TreeHive Strategy, Databricks Apps removes obstacles like the need to set up separate infrastructure for development and deployment, making the process easier and more efficient. The new features allow companies to go beyond implementing AI/ML models and create differentiated applications that leverage their unique data sets.

Kevin Petrie, an analyst at BARC U.S., highlighted the significance of Databricks Apps in helping companies develop custom AI applications, which are essential for maintaining a competitive edge.

Databricks, founded in 2013, was one of the pioneers of the data lakehouse storage format, and over the last two years, it has expanded its platform to focus on AI and machine learning (ML) capabilities. The company’s $1.3 billion acquisition of MosaicML in June 2023 was a key milestone in building its AI environment. Databricks has since launched DBRX, its own large language model, and introduced further functionalities through product development.

Databricks Apps, now available in public preview on AWS and Azure, advances these AI development capabilities, simplifying the process of building applications within a single platform. Developers can use frameworks like Dash, Flask, Gradio, Shiny, and Streamlit, or opt for integrated development environments (IDEs) like Visual Studio Code or PyCharm. The toolkit also provides prebuilt Python templates to accelerate development. Additionally, applications can be deployed and managed directly in Databricks, eliminating the need for external infrastructures. Databricks Apps includes security features such as access control and data lineage through the Unity Catalog.

Farmer noted that the support for popular developer frameworks and the automatic provisioning of serverless compute could significantly impact the AI development landscape by reducing the complexity of deploying data architectures.

While competitors like AWS, Google Cloud, Microsoft, and Snowflake have also made AI a key focus, Farmer pointed out that Databricks’ integration of AI tools into a unified platform sets it apart. Databricks Apps further enhances this competitive advantage.

Despite the added capabilities of Databricks Apps, Petrie cautioned that developing generative AI applications still requires a level of expertise in data, AI, and the business domain. While Databricks aims to make AI more accessible, users will still need substantial knowledge to effectively leverage these tools.

Databricks’ vice president of product management, Shanku Niyogi, explained that the new features in Databricks Apps were driven by customer feedback. As enterprise interest in AI grows, customers sought easier ways to develop and deploy internal data applications in a secure environment.

Looking ahead, Databricks plans to continue investing in simplifying AI application development, with a focus on enhancing Mosaic AI and expanding its collaborative AI partner ecosystem. Farmer suggested that the company should focus on supporting nontechnical users and emerging AI technologies like multimodal models, which will become increasingly important in the coming years.

The introduction of Databricks Apps marks a significant step forward in Databricks’ AI and machine learning strategy, offering users a more streamlined approach to building and deploying AI applications.