How Posit AI’s Next Edit Suggestions work

I just loaded some data from the forested package into my R environment. It has a bunch of measurements of forest attributes across Washington State:

Let’s add a lon/lat plot of the data to this Quarto document:

In this video, I’m often just typing a couple characters and then waiting for Next Edit Suggestions (NES), part of our new Posit AI service. You might notice a couple things:

1. It doesn’t hallucinate dataset or column names.
2. It doesn’t f*** up the formatting of the Quarto document.
3. The suggestions arrive very quickly.
4. Broadly, the suggestions are quite reasonable.

At this point, 3) and 4) are table stakes for IDE next edit suggestions (aka supercomplete or AI autocomplete). 1) and 2), though, will be quite unfamiliar to those who have used NES in other IDEs before. I’m going to write a bit about how this system works and how it came to be.

Table stakes: reasonable suggestions at blistering speed

Loosely, IDEs’ Next Edit Suggestions features look like this:

• Rent out a GPU and put a small (8B or so) model on it.
• As your users type, take a snapshot of the file every few seconds and record the changes to generate an “edit history.”
• On request (either automatically after some typing delay or through an explicit request from a keyboard command), provide the state of the file as well as some log of those edits to the model. A few thousand input tokens.
• Have the model reply with the change that it wants to make. Output formats vary here, but usually a couple hundred output tokens.

Now, which small model do you use? How do you present the “edit history” to that model? How do you combine all of those pieces of the prompt together? How does the model specify the output format?

To help us answer this question, I put together an open-source evaluation called nesevals. Structured as an R package, the eval allowed us to try out a bunch of different models, input formats, and output formats. The raw data is available in the package, but we’ll walk through a plot that condenses a lot of the lessons learned onto two axes.

Each point is one scaffold—a combination of model and prompt format. The x-axis is median latency; the y-axis is a model-graded quality score. We want the top-left corner: fast and good.

The first thing you might reach for is the fastest model on a frontier lab’s API. Claude Haiku 4.5, Gemini 3.1 Flash-Lite, GPT-4.1 Nano, and GPT-5 Nano all score reasonably well—but their latency is far too high for real-time suggestions.

Even Gemini 3.1 Flash-Lite, the fastest of them, takes over a second per suggestion.

Moving left, a few inference-focused providers like Groq and Cerebras serve open models at better latencies.

GPT-OSS 20B on Groq scores well, but ~550ms is still too slow for a feature that fires every couple of seconds.

On the far left, many under 200ms, are our custom deployments on Baseten.

At the very bottom is Zeta, a fine-tune from the Zed team. The latency was great, but the model wasn’t resilient to our inclusion of variable metadata in the prompt.

The remaining points are all Qwen3-8B with different input formats. The two clusters reflect different prompt scaffolds.

That brings us to the winner—the top-left point. This is the scaffold deployed as part of the service.

A close read

We’ll walk through what one Next Edit Suggestion looks like using this forested example. With NES turned to “Automatic”, these are firing off every second or two as I type.

The system prompt is pretty minimal.

The “default” behavior of these models is to omit portions of the code that they’re not editing.

You are a code completion assistant. You will be given a
file excerpt for context and a region from that excerpt
to rewrite. Predict the user's next edit by rewriting
only the provided region.

Output the rewritten region wrapped in a code fence 
using exactly 5 backticks. Remove any cursor markers. 
Your output replaces the region entirely, so any line you 
omit will be deleted. Always reproduce the full region,
including lines you did not change.

The system prompt also includes a worked example; instead of repeating it here, we’ll just talk through the same sections that appear in the user prompt.

The user prompt is composed of four sections:

1. File context, showing a large portion of the document the user is editing.
2. The edit history, displaying the edits they’ve recently made to the document.
3. Variables from the user’s active computational session.
4. The region to apply an edit inside of. This is 5 lines before and after the user’s cursor maximum.

The file context provides a long excerpt from the file that the user is editing.

We always use 5 backticks as code fences so that we can include triple backticks from Quarto / RMarkdown / Markdown documents without issue.

We use 5 backticks rather than some other delineator as a way of “swimming downstream” with LLMs’ default behavior. Models usually want to start code cells with triple backticks; with other dilineators, if the model accidentially started with triple backticks, the response had to be thrown away.

The user’s cursor is marked with a <cursor> tag.

The next piece of the user prompt is the edit history.

Our initial reflex was that the most intuitive way to show edits would be diffs. e.g., inside of an excerpt, a code cell with + on the lines added and - on the lines deleted.

In evals, I found that this was not actually the case. I call this edit history format a “narrative” format, where we post-process the edits to situate them in natural language. Models seem to “see” the changes much better this way.

We only include one line of file context around the changes on either side.

I don’t know why it works, but one hypothesis is that it says what happened—the important part—twice: first in the narrative, second in the resulting code cell.

The “real” version of this edit history includes many more edits–up to 10.

The prompt also includes metadata on the variables present in the user’s R or Python environment.

Here, the model can see every column name and type in the forested data frame.

This is just enough information to write code to work the data and nothing more. Notably, we don’t include the values of columns or other variables in the session.

We use a few heuristics to determine which variables to show in this part of the prompt, but broadly, if it’s in the file excerpt, metadata on it will appear here.

This is how the model knows to suggest canopy_cover. Without this information, it would have to make a column name up; other NES systems do this regularly with data science code.

Finally, the prompt specifies exactly which lines the model should rewrite.

Note that this is a much smaller excerpt than 5 lines above and below the cursor. Since we’re in a Quarto document here, we use a different heuristic.

Even frontier models like Claude Opus 4.6 have trouble outputting code with mismatched backticks. It’s possible that, if we had just taken some fixed number of lines above and below the cursor, there’d be code fences without a “match” in the region. If the model mistakenly adds or removes them, that’d be processed as a proposed edit and, if accepted, botch the Quarto formatting.

Instead, we start at the cursor and “walk” in either direction. If we hit code fences, we stop.

This way, the model is never asked to output mismatched code fences. This works quite well.

## File context

Here is the code the user is currently editing.

`````file
```{r}
library(tidyverse)
library(forested)

forested
```

```{r}
ggplot(forested) +
  aes(x = lon, y = lat, <cursor>) +
  geom_point(size = .4)
```
`````

## Edit History

The following edits led to the current state of the file.

The user's code started like this:

`````
ggplot(forested)
`````

Then, the user added 2 lines:

`````
ggplot(forested) +
  aes(x = lon, y = lat) +
  geom_point(size = .4)
`````

Most recently, the user changed `aes(x = lon, y = lat) +`
to `aes(x = lon, y = lat, ) +`:

`````
ggplot(forested) +
  aes(x = lon, y = lat, ) +
  geom_point(size = .4)
`````

## Variables

The following variables are present in the user's computational environment:

forested: <data.frame>
 forested$forested: <numeric>
 forested$year: <numeric>
 forested$elevation: <numeric>
 forested$eastness: <numeric>
 forested$northness: <numeric>
 forested$roughness: <numeric>
 forested$tree_no_tree: <numeric>
 forested$dew_temp: <numeric>
 forested$precip_annual: <numeric>
 forested$vapor_min: <numeric>
 forested$vapor_max: <numeric>
 forested$canopy_cover: <numeric>
 forested$lon: <numeric>
 forested$lat: <numeric>
 forested$land_type: <numeric>
 forested$county: <numeric>

## Region

Rewrite the following region from the excerpt above to
predict the user's next edit:

`````file
ggplot(forested) +
  aes(x = lon, y = lat, <cursor>) +
  geom_point(size = .4)
`````

We send that all off to the model and, hopefully, get a rewritten version of the excerpt in reply.

The model suggests color = canopy_cover.

That will be post-processed by the NES extension and, since it’s a single-line edit starting at the user’s cursor, will be presented as “ghost text.”

`````file
ggplot(forested) +
  aes(x = lon, y = lat, color = canopy_cover) +
  geom_point(size = .4)
`````

If you’re interested in checking out Posit AI, see the release post and the product page. If you want to poke at this data yourself, check out the open-source nesevals package.