OverArchitected

What is it? Eight new connectors for LakeFlow Connect: Microsoft Dynamics 365, Jira, Confluence, Salesforce (now with incremental loads and formula field support), Meta Ads, Excel file reading, NetSuite, and PostgreSQL. These are fully managed, no-code ingestion pipelines that land data directly into Delta tables.

Is it for you? If you're currently running custom scripts or third-party ETL tools to get data from any of these sources into Databricks, this replaces all of that. Set up a connection, pick your tables, and data flows automatically with checkpointing and error handling.

Try it Navigate to your workspace, go to Data Ingestion > LakeFlow Connect, and create a new connection. OAuth setup takes about 5 minutes for most connectors.

What is it? Three updates. The Assistant now lives on docs.databricks.com as a chat interface. Agent Mode lets the Assistant interact directly with your workspace (run queries, create clusters, etc.) and is in public preview. And skills — an open standard from Anthropic — let you bundle instructions, scripts, and context into folders that the agent discovers progressively instead of loading everything at once.

Is it for you? Agent Mode is for anyone who wants to operate their workspace conversationally. Skills are for teams who want to encode their specific workflows, patterns, and tribal knowledge so the Assistant can learn them incrementally.

Try it Open any notebook and click the Assistant icon. For Agent Mode, toggle it on in the Assistant panel. For skills, create a .skills/ folder in your repo with a SKILLS.md file describing available capabilities.

What is it? Spark Declarative Pipelines (formerly DLT) now support forEachBatch() — giving you micro-batch control over how data is written. This unlocks custom sinks (JDBC, REST APIs), MERGE INTO operations, and any processing logic that needs to happen per-batch rather than per-record.

Is it for you? If you've been stuck on classic Structured Streaming because SDP couldn't handle your write pattern (merges, upserts, external sinks), this is your ticket in. If your pipelines are already working with simple append/overwrite, you don't need this.

Try it In your SDP pipeline definition, use the new sink parameter with a forEachBatch function. The syntax mirrors classic Structured Streaming's forEachBatch API.

What is it? Three new models on the Foundation Model API: Claude Opus 4.5, Claude Haiku 4.5 (both Anthropic), and GPT 5.2 (OpenAI). All available as Databricks-hosted endpoints — meaning you get inference tables, AI Gateway, PII detection, and usage tracking without any extra setup.

Is it for you? If you're building AI features on Databricks (agents, RAG, structured extraction), these are the models to use. Hosting them through Databricks means your data stays in your environment and every call is logged and governable.

Try it Available immediately in the AI Playground — no setup needed. For production use, enable AI Gateway (beta preview) on a foundation model endpoint to get inference tables, PII detection, rate limiting, and usage tracking.

What is it? Delta tables can now be shared to external consumers as Iceberg-formatted data via Delta Sharing. Any Iceberg-compatible client (Snowflake, Trino, Spark on another platform) can read your shared tables without needing Databricks.

Is it for you? If you need to share data with partners or teams that aren't on Databricks but use Iceberg-compatible tools, this eliminates the need for data export/copy workflows. Your data stays in place; they read it live.

Try it Enable Iceberg compatibility on your table (ALTER TABLE SET TBLPROPERTIES ('delta.universalFormat.enabledFormats' = 'iceberg')), then add it to a Delta Share. Note: deletion vectors must be disabled on the table.

What is it? The Knowledge Assistant in Agent Bricks is now generally available. It's a no-code RAG chatbot — point it at Unity Catalog files, volumes, or a vector search index, and it builds a question-answering agent with full citations (page numbers, source excerpts). Deployable as a serving endpoint.

Is it for you? If your team has documentation, manuals, policies, or knowledge bases that people constantly ask questions about, this turns them into a searchable, conversational interface in minutes. No ML expertise required.

Try it Go to Machine Learning > Agent Bricks, select Knowledge Assistant, upload your documents or point to a UC volume, and deploy. You'll have a working chatbot with citations in under 10 minutes.

What is it? Lakebase — Databricks-managed PostgreSQL — now supports autoscaling in public preview, including scale-to-zero. No more manually choosing capacity units. It also ships with ACL support for fine-grained access control and read-write access from the SQL editor.

Is it for you? If you need an operational database for serving applications, APIs, or low-latency lookups on top of your lakehouse data, Lakebase is the native answer. Autoscaling means you pay for what you use and don't manage infrastructure.

Try it In your workspace, go to SQL > Lakebase and create a new instance. It provisions in seconds. Connect from any Postgres-compatible client or use the built-in SQL editor.