Codex with gpt-oss:120b model appears to forget the task

[zhangwen0411]

What version of Codex is running?

codex-cli 0.21.0

Which model were you using?

gpt-oss:120b

What platform is your computer?

Darwin 24.6.0 arm64 arm

What steps can reproduce the bug?

Launch Codex CLI with gpt-oss:120b (served by Ollama), and ask a question about a code base (see screenshot below for an example).

What is the expected behavior?

My question answered.

What do you see instead?

As shown in the screenshot below, Codex initially attempts to find an answer to my question. But by the end, it seems to have forgotten that I had asked a question at all!

The last two turns show:

thinking
We have a huge repo with many imports. The user hasn't asked a question yet. The initial tool output from shell shows that many files import claripy. Likely the user will ask to remove claripy imports or replace with something else. But we need to see the prompt: The user hasn't given a request. Maybe next message will contain a task. We need to wait for user input.

codex
I’m ready for your next instruction!

Additional information

No response

[earl-of-embedding]

Theory: Could it be too short context in ollama from codex cli?

Why do I think that?

If I run gpt-oss:20b, long documents also de-rail codex (ubuntu linux machine).

"ollama ps" yields the following information regarding the model (loaded by codex upon start):

NAME ID SIZE PROCESSOR CONTEXT UNTIL
gpt-oss:20b f2b8351c629c 16 GB 100% GPU 8192 4 minutes from now

So, it seems the context is only 8192 which is too small. Is there a way to enforce larger context?

[zhangwen0411]

You're absolutely right! The Ollama FAQ states:

The gpt-oss model has a default context window size of 8192 tokens.

And it gives a few ways to increase the context window size.

Perhaps Codex CLI should issue a warning if Ollama's context window size is too small? The default behavior (with OSS model loaded by Codex) can confuse users.

[earl-of-embedding]

I am still scratching my head.

https://github.com/openai/codex/blob/main/codex-rs/core/src/openai_model_info.rs clearly states that 128k should be used for context:

pub(crate) fn get_model_info(model_family: &ModelFamily) -> Option<ModelInfo> {

    let slug = model_family.slug.as_str();
    match slug {

        // OSS models have a 128k shared token pool.
        // Arbitrarily splitting it: 3/4 input context, 1/4 output.
        // https://openai.com/index/gpt-oss-model-card/
        "gpt-oss-20b" => Some(ModelInfo {
            context_window: 96_000,
            max_output_tokens: 32_000,

        }),
        "gpt-oss-120b" => Some(ModelInfo {
            context_window: 96_000,
            max_output_tokens: 32_000,

        }),

[zhangwen0411]

I briefly looked into the uses of the context_window config parameter. It appears this number is used only in TUI components:

codex/codex-rs/tui/src/chatwidget.rs

Lines 191 to 199 in 5552688

	fn on_token_count(&mut self, token_usage: TokenUsage) {
	self.total_token_usage = add_token_usage(&self.total_token_usage, &token_usage);
	self.last_token_usage = token_usage;
	self.bottom_pane.set_token_usage(
	self.total_token_usage.clone(),
	self.last_token_usage.clone(),
	self.config.model_context_window,
	);
	}

codex/codex-rs/tui/src/chatwidget.rs

Lines 733 to 740 in 5552688

	pub(crate) fn clear_token_usage(&mut self) {
	self.total_token_usage = TokenUsage::default();
	self.bottom_pane.set_token_usage(
	self.total_token_usage.clone(),
	self.last_token_usage.clone(),
	self.config.model_context_window,
	);
	}

codex/codex-rs/tui/src/bottom_pane/mod.rs

Lines 223 to 232 in 5552688

	pub(crate) fn set_token_usage(
	&mut self,
	total_token_usage: TokenUsage,
	last_token_usage: TokenUsage,
	model_context_window: Option<u64>,
	) {
	self.composer
	.set_token_usage(total_token_usage, last_token_usage, model_context_window);
	self.request_redraw();
	}

codex/codex-rs/tui/src/bottom_pane/chat_composer.rs

Lines 129 to 140 in 5552688

	pub(crate) fn set_token_usage(
	&mut self,
	total_token_usage: TokenUsage,
	last_token_usage: TokenUsage,
	model_context_window: Option<u64>,
	) {
	self.token_usage_info = Some(TokenUsageInfo {
	total_token_usage,
	last_token_usage,
	model_context_window,
	});
	}

codex/codex-rs/tui/src/bottom_pane/chat_composer.rs

Lines 672 to 687 in 5552688

	if let Some(context_window) = token_usage_info.model_context_window {
	let percent_remaining: u8 = if context_window > 0 {
	let percent = 100.0
	- (last_token_usage.tokens_in_context_window() as f32
	/ context_window as f32
	* 100.0);
	percent.clamp(0.0, 100.0) as u8
	} else {
	100
	};
	hint.push(Span::from(" "));
	hint.push(
	Span::from(format!("{percent_remaining}% context left"))
	.style(Style::default().add_modifier(Modifier::DIM)),
	);
	}

So it appears the configured context_window is not passed to Ollama, which serves the OSS model.

[earl-of-embedding]

You are right. In the meantime, I tried to correct this behavior in my local fork of codex-rs.

With a lot of tinkering I got "ollama ps" sometimes to show the desired context length as set (I tried both larger and shorter contexts). However, codex-rs is surprisingly persistent. When I use the TUI, somehow it always stops the existing model and replaces it with another one - once more with default context length of 8k.

Likely that is a quick fix for someone seasoned in ollama use but I am out of my league. :-(

[tunesmith]

I poked at this a while with my gpt-oss-20b model. I used ollama to make a 32k context for it, but it still didn't help. It's unclear what makes it "forget": chatgpt was theorizing it was a streaming bug from ollama, but it also "forgets" when I use exec commands.

[earl-of-embedding]

Curious. When you do "ollama ps" during codex operation, is the model always with 32k or does it sometimes 'switch back' to 8K?

[zhangwen0411]

Here's what I ended up doing:

• On the Ollama side, to set a larger context, I created a Modelfile file like this:

  FROM gpt-oss:120b
  PARAMETER num_ctx 131072
  

  I then created a new model in Ollama like this:

  ollama create gpt-oss:120b-128k -f ./Modelfile
  

• On the Codex CLI side, I added the following to ~/.codex/config.toml:

  [model_providers.ollama]
  name = "Ollama"
  base_url = "http://ollama-server-address:11434/v1"
  
  [profiles.gpt-oss-120b-128k]
  model_provider = "ollama"
  model = "gpt-oss:120b-128k"

  I then launch Codex CLI using codex exec --profile gpt-oss-120b-128k.

This seems to be working for me!

[tunesmith]

I did something similar with gpt-oss:20b , creating a 32k model, and my request still gets forgotten, as it does with the vanilla codex --oss with ollama serve running. This is on an M3 Pro with 36g memory.

[tunesmith]

When I use the LMStudio server instead of ollama, and load gpt-oss-20b there with expanded context, initial indications from codex cli look good; I've had some complete exchanges with changes and explanatory responses. So it's possible this "forgetting" is an ollama bug, or at least some integration between codex and ollama that could be improved.

[earl-of-embedding]

Just for other readers of this thread:

I had success with running llamacpp with full 128k of both the 20B and the 120B model of GPT-OSS-x20B (quad 3090 system here). This documentation was very helpful and it works nicely with codex:
ggml-org/llama.cpp#15396

I could not get ollama to work, the inference became very slow for >40k context and it moved eveything to CPU.

PS: for sake of completeness, these commands work on my quad 3090 system:

./llama-server -hf ggml-org/gpt-oss-20b-GGUF --ctx-size 0 --jinja -ub 2048 -b 2048 -ngl 99 -fa
./llama-server -hf ggml-org/gpt-oss-120b-GGUF --ctx-size 0 --jinja -ub 2048 -b 2048 -ngl 99 -fa

Both(!) result in slightly above 60 tokens per second and give full context. For large context the tps count reduces significantly, I think to about 15 tps for very long contexts.

[Mondonno]

@zhangwen0411 Thanks for providing the workaround.
There is also a simpler way to do it.
Run ollama show:

ollama show gpt-oss:20b
  Model
    architecture        gptoss
    parameters          20.9B
    context length      131072
    embedding length    2880
    quantization        MXFP4

  Capabilities
    completion
    tools
    thinking

  Parameters
    temperature    1

  License
    Apache License
    Version 2.0, January 2004
    ...

Note:
Then using the context length parameter provided you can see what is the upper limit.
I found out for myself that using LLMs on full context on my MacBook Pro with M3 Pro and 36 GB of RAM makes them ultra-> slow and makes my Mac get starved to death from RAM given I am also running JetBrains IDEs:

ollama ps
NAME                ID              SIZE     PROCESSOR          CONTEXT    UNTIL
gpt-oss:128k-20b    2d8df491533b    34 GB    16%/84% CPU/GPU    131072     4 minutes from now

Then run ollama run gpt-oss:20b from your terminal:

ollama run gpt-oss:20b
>>> /?
Available Commands:
  /set            Set session variables
  /show           Show model information
  /load <model>   Load a session or model
  /save <model>   Save your current session
  /clear          Clear session context
  /bye            Exit
  /?, /help       Help for a command
  /? shortcuts    Help for keyboard shortcuts

Use """ to begin a multi-line message.

>>> /set parameter
Available Parameters:
  /set parameter seed <int>             Random number seed
  /set parameter num_predict <int>      Max number of tokens to predict
  /set parameter top_k <int>            Pick from top k num of tokens
  /set parameter top_p <float>          Pick token based on sum of probabilities
  /set parameter min_p <float>          Pick token based on top token probability * min_p
  /set parameter num_ctx <int>          Set the context size
  /set parameter temperature <float>    Set creativity level
  /set parameter repeat_penalty <float> How strongly to penalize repetitions
  /set parameter repeat_last_n <int>    Set how far back to look for repetitions
  /set parameter num_gpu <int>          The number of layers to send to the GPU
  /set parameter stop <string> <string> ...   Set the stop parameters

>>> /set parameter num_ctx 31072
Set parameter 'num_ctx' to '31072'
>>> /save gpt-oss:32k-20b
Created new model 'gpt-oss:32k-20b'

When running the 32k model my memory usage is acceptable:

NAME               ID              SIZE     PROCESSOR    CONTEXT    UNTIL
gpt-oss:32k-20b    e83f14b6f6fb    18 GB    100% GPU     31072      4 minutes from now

After running the command that sets the context length I need to save it as seperate model, it should run with this context length regardless of environment, it works for me like a charm and is simpler for me than creating seperate ModelFile.