Decision guide

This page turns the evidence in 06 into a choice. It uses plain words and says where the evidence stops.

Short answer

For code work in Databricks (SQL, notebooks, jobs, pipelines, debugging), yes: a coding agent in the Databricks shell is a credible replacement for Genie Code, provided you pick the model and the guard rails on purpose. For ambient, in-product help, it is not a replacement (what is not replaced).

Pick a setup

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#E8F0FE','primaryTextColor':'#1F2937','primaryBorderColor':'#A8C0E8','lineColor':'#94A3B8'}}}%%
flowchart TB
    Q1{"Can you use a strong model?<br/>for example Gemini 3.8 Flash<br/>through your own API key"}:::blue
    Q1 -- "Yes" --> A1["Setup A: stock Databricks CLI and skills<br/>plus our rules file and write fence<br/>(V6 style)"]:::green
    Q1 -- "No, only gpt-oss-120b<br/>from the AI Gateway" --> Q2{"Will people run it<br/>unattended or on shared data?"}:::blue
    Q2 -- "Yes" --> B1["Setup B: custom tool server with<br/>scoped writes, supervisor loop and<br/>report template (V4 style)"]:::yellow
    Q2 -- "No, a person watches" --> B2["Setup B with the CLI added<br/>(V6 style)"]:::yellow
    A1 --> G["In every case:<br/>service principal token, scratch schema only,<br/>read the report's 'odd or suspicious' section"]:::pink
    B1 --> G
    B2 --> G
    classDef blue fill:#E8F0FE,stroke:#A8C0E8,color:#1F2937
    classDef green fill:#E6F4EA,stroke:#A8D5B5,color:#1F2937
    classDef yellow fill:#FFF8E1,stroke:#F2D98D,color:#1F2937
    classDef pink fill:#FDE7EF,stroke:#F2B5CB,color:#1F2937

Do not choose by these numbers alone: each is a single run on small, clean data (06).

What a coding agent does not replace

Our exams show strong results for well-specified tasks on known tables. They do not show the same for open-ended "figure out what we should look at" work on a large, messy catalogue.

Cost in plain words

We did not measure money, and we did not have Genie Code usage data to compare against (spec §9.3). What we can say:

• Genie Code is metered in DBUs after a monthly allowance. An external agent is metered in model tokens, plus the sandbox compute.
• Strong models read and write about three times as many tokens: 5 to 9 million per eight tasks, against 2 to 3 million for gpt-oss (06). A better score is not free.
• Tool descriptions are sent on every turn, so a lean tool list saves money on every task, forever.
• A lower price per token does not help if the agent fails and you rerun it: the setup with the most points per run is often the cheaper one.

Run a two-week pilot and measure before telling anyone a saving.

A rollout in four steps

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#E8F0FE','primaryTextColor':'#1F2937','primaryBorderColor':'#A8C0E8','lineColor':'#94A3B8'}}}%%
flowchart LR
    S1["1. Try it<br/>follow doc 02 in a<br/>scratch workspace"]:::blue --> S2["2. Fence it<br/>service principal,<br/>scratch schema,<br/>doc 04"]:::pink
    S2 --> S3["3. Measure it<br/>run the exam suite<br/>on your model and<br/>your own tasks"]:::green
    S3 --> S4["4. Decide<br/>compare score, tokens<br/>and review time<br/>against Genie Code"]:::yellow
    classDef blue fill:#E8F0FE,stroke:#A8C0E8,color:#1F2937
    classDef pink fill:#FDE7EF,stroke:#F2B5CB,color:#1F2937
    classDef green fill:#E6F4EA,stroke:#A8D5B5,color:#1F2937
    classDef yellow fill:#FFF8E1,stroke:#F2D98D,color:#1F2937

1. Try it. 02 Setup takes about an hour. The launcher notebook means no local shell is needed.
2. Fence it. Create a service principal with rights on one scratch schema and folder; give its token to the sandbox (04).
3. Measure it. Copy a mission file from exams/missions/, write two or three tasks from your own work, and run them (05).
4. Decide. Compare score, tokens and the time a person spends reviewing the output.

Questions people ask

Can we use GitHub Copilot instead of OpenCode? Yes in principle, and it ran our exams. See 06 for scores and the proxy it needs in front of the Databricks gateway.

Do we need the custom tool server? With a strong model, no: Setup A uses the stock CLI. With the gateway's weaker model it was worth about 55 points in our exams (V4 94% against V5 39%).

Can the agent harm production? Yes, if it holds a powerful token. Words in a rules file do not stop a CLI command. Use a restricted token (04).

What if the model changes? Rerun the exams. That is what they are for.

Which parts are verified and which are advice? Every claim about what ran is backed by a file in exams/results/ or docs/e2e/. Recommendations such as the service principal are marked "not tested".

Back to the index.