Ollama tool-calling loop does not complete for query/chat on v0.4.5 and v0.5.0-rc1

[mike-011]

Summary

openkb add works with local Ollama models, but openkb query does not reliably complete the tool-calling loop. This remains reproducible on OpenKB v0.5.0-rc1, even with the litellm.drop_params configuration introduced for Ollama compatibility.

The models can emit tool calls, but the CLI either:

• prints raw tool-call JSON instead of executing the tool and continuing;
• returns {};
• returns an incomplete/raw response;
• or times out without a final answer.

Environment

• Linux x86_64
• OpenKB v0.5.0-rc1
• openai-agents==0.17.3
• Ollama running as a separate local service
• Ollama accessed through its OpenAI-compatible/LiteLLM route
• Knowledge base containing indexed Markdown summaries
• Query executed through the normal CLI:

openkb --kb-dir <kb-dir> query "What topics are discussed in these messages?"

No cloud API, private endpoint, API key, token, or other credential is required to reproduce the local-backend behavior.

Configuration

The KB configuration was equivalent to:

model: ollama/llama3.1:8b-instruct-q8_0
api_base: http://ollama-host:11434
language: en
pageindex_threshold: 20
parallel_tool_calls: false
litellm:
  drop_params: true

The hostname above is intentionally a placeholder. The real endpoint is private and is not part of this report.

Reproduction

1. Create or use a KB with several indexed Markdown documents.
2. Configure the KB for a local Ollama model as shown above.
3. Run:

openkb --kb-dir <kb-dir> query "What topics are discussed in these messages?"

4. Observe that the query does not produce a normal final answer grounded in the KB.

Observed result with llama3.1:8b-instruct-q8_0

The CLI returned raw JSON similar to:

{
  "reqId": "<redacted>",
  "message": "<a requested wiki path was not found>",
  "toolCalls": [
    {
      "id": "<redacted>",
      "type": "function",
      "name": "read_file",
      "arguments": {"path": "summaries/index.md"}
    }
  ]
}

The tool call was not followed by a normal final answer.

The same model was also tested through the non-streaming Runner.run path. In that path it attempted to call a tool named search_strategy, which was not one of the tools registered by OpenKB:

ModelBehaviorError: Tool search_strategy not found in agent wiki-query

Additional local-model results

Using the same KB, question, and OpenKB v0.5.0-rc1 CLI:

Model	Result
llama3.1:8b-instruct-q8_0	raw tool-call JSON; no final answer
qwen3:14b	{}
qwen3.5:9b	empty output
gemma4:12b	timed out
deepseek-r1:14b	{}
qwen2.5-coder:14b	raw/incomplete response claiming that index.md was unavailable
llama3.2:1b	raw function-call JSON

openkb add had previously completed successfully on the same type of KB, including document summaries. The failure is specific to the query/chat agent path and tool-loop completion, not basic LiteLLM connectivity.

Expected behavior

For an Ollama-compatible model that returns a valid structured tool call, OpenKB should:

1. parse the tool call;
2. execute the registered OpenKB tool;
3. append the tool result to the agent conversation using the correct Chat Completions/LiteLLM format;
4. continue the agent loop;
5. return a final natural-language answer.

If a model returns an unsupported or malformed tool call, OpenKB should report a clear actionable error rather than emitting raw JSON or returning {}.

Investigation notes

• Issue [Feature] Plans to support litellm config in .openkb/config.yaml? (Encountered UnsupportedParamsError with Ollama) #137 and PR feat(config): pass through global LiteLLM settings via a litellm: config block #138 address the parallel_tool_calls parameter rejected by Ollama. Setting parallel_tool_calls: false and litellm.drop_params: true avoids that specific parameter error, but does not solve this issue.
• OpenKB query code still uses Runner.run_streamed(...) for the streaming path.
• The non-streaming path also fails with some local models, so this may involve the Agents SDK/LiteLLM message-adapter format, model-specific tool-call behavior, or both.
• Direct low-level LiteLLM tests can return structured tool calls for some Ollama models, but the full OpenKB query agent does not reliably consume and continue from them.

Request

Could the maintainers please confirm:

1. which Ollama/LiteLLM model and version combinations are officially supported for query/chat;
2. whether OpenKB expects OpenAI Responses API semantics or Chat Completions semantics for the LiteLLM provider;
3. whether the streaming path is expected to support Ollama tool calls;
4. whether a non-streaming fallback should be used for LiteLLM/Ollama providers;
5. whether malformed model-emitted tool names should be handled with a clearer validation error.

A small regression test covering one structured Ollama tool call, tool execution, and a final answer would help prevent this from recurring.

[mike-011]

Additional note: also reproducible on the latest stable v0.4.5

This is not limited to the v0.5.0-rc1 release candidate.

The same query/tool-loop failure was reproduced on the latest stable v0.4.5 with the same local Ollama setup and the same type of indexed KB. The observed symptoms were the same: raw tool-call JSON, {}, empty output, or a timeout instead of a final grounded answer.

The v0.5.0-rc1 test was performed to check whether the newer agent/configuration changes fixed the issue. It did not. Please therefore treat this as affecting both:

• v0.4.5 (latest stable release);
• v0.5.0-rc1 (release candidate).

The parallel_tool_calls / litellm.drop_params issue is related but not sufficient to resolve the complete tool-calling loop failure.

[mike-011]

Additional test: upgrading openai-agents to 0.18.3 does not fix the issue

I repeated the test after upgrading the runtime from openai-agents==0.17.3 to the latest PyPI release available at the time of testing, openai-agents==0.18.3 (which also installed openai==2.48.0).

For a clean run, the OpenKB container was recreated, stale query processes were stopped, and the local Ollama endpoint was verified from inside the container with HTTP 200. The test used OpenKB v0.5.0-rc1, the same indexed KB, the same question, and the same configuration:

api_base: http://ollama-host:11434
parallel_tool_calls: false
litellm:
  drop_params: true

The endpoint above is a placeholder; no private network address or credential is included here.

The query was run sequentially against all locally available models:

Model	Result with openai-agents==0.18.3
llama3.1:8b-instruct-q8_0	raw {"name": "read_index"}; no final answer
qwen3.5:9b	empty output
gemma4:12b	timeout after 150 seconds
deepseek-r1:14b	{"error": "File not found"}
qwen2.5-coder:14b	refusal/incomplete response instead of a grounded answer
qwen3:14b	{}
llama3.2:1b	raw function-call JSON instead of a final answer

Therefore, upgrading openai-agents from 0.17.3 to 0.18.3 had no material effect on the issue. The failure still appears to be in the end-to-end tool-calling loop between OpenKB, the Agents SDK, LiteLLM, and Ollama rather than being fixed by the Agents SDK version upgrade.