Skip to content

Python: feat(observability): add support for OpenAI cache write tokens in usage details - #7369

Merged
eavanvalkenburg merged 3 commits into
microsoft:mainfrom
cecheta:prompt-cache-write
Jul 30, 2026
Merged

Python: feat(observability): add support for OpenAI cache write tokens in usage details#7369
eavanvalkenburg merged 3 commits into
microsoft:mainfrom
cecheta:prompt-cache-write

Conversation

@cecheta

@cecheta cecheta commented Jul 28, 2026

Copy link
Copy Markdown
Member

Motivation & Context

Add cache write tokens to usage details in GPT-5.6 models.

Description & Review Guide

  • What are the major changes?
  • What is the impact of these changes?
  • What do you want reviewers to focus on?
  • cache_creation_input_token_count added to usage details for both Responses and Chat Completions API.
  • Fix sample (prompt_cache_key is required to use the improved caching).
  • Add openai.cache_write_tokens to OpenTelemetry

Related Issue

Fixes #7368

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Copilot AI review requested due to automatic review settings July 28, 2026 15:12
@cecheta
cecheta temporarily deployed to github-app-auth July 28, 2026 15:12 — with GitHub Actions Inactive
@cecheta
cecheta temporarily deployed to github-app-auth July 28, 2026 15:12 — with GitHub Actions Inactive
@cecheta
cecheta temporarily deployed to github-app-auth July 28, 2026 15:12 — with GitHub Actions Inactive
@agent-framework-automation agent-framework-automation Bot added the python Usage: [Issues, PRs], Target: Python label Jul 28, 2026
@cecheta
cecheta temporarily deployed to github-app-auth July 28, 2026 15:12 — with GitHub Actions Inactive
@github-actions github-actions Bot changed the title feat(observability): add support for OpenAI cache write tokens in usage details Python: feat(observability): add support for OpenAI cache write tokens in usage details Jul 28, 2026
@cecheta
cecheta temporarily deployed to github-app-auth July 28, 2026 15:13 — with GitHub Actions Inactive
@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/core/agent_framework
   observability.py10118092%425, 427–428, 431, 434, 437–438, 443–444, 450–451, 457–458, 465, 467–468, 471, 474, 477–478, 483–484, 490–491, 497–498, 505, 682–683, 893, 897–899, 901, 909–910, 914, 958, 960, 971–973, 975–977, 981, 989, 1113–1114, 1349, 1632–1633, 1744, 1894, 1937–1938, 2127, 2355–2356, 2360, 2400–2401, 2405, 2446–2449, 2490–2491, 2642, 2645, 2657, 2674, 2678–2679, 2682, 2688, 2808, 3025, 3027
packages/openai/agent_framework_openai
   _chat_client.py137411791%312, 325, 675–679, 687–690, 696–700, 750–757, 759–761, 768–770, 828, 836, 859, 1076, 1135, 1137, 1139, 1141, 1207, 1221, 1301, 1311, 1316, 1359, 1471–1472, 1487, 1760, 1867, 1872–1873, 1956, 1966, 1993, 1999, 2009, 2015, 2020, 2026, 2031–2032, 2112, 2156, 2159–2162, 2176, 2186–2187, 2199, 2241, 2306, 2323, 2326, 2353–2355, 2394, 2411, 2414, 2476, 2483, 2520–2521, 2556, 2594–2595, 2613–2614, 2786–2787, 2805, 2891–2899, 3077, 3092, 3142, 3152–3154, 3181–3183, 3193–3194, 3200, 3215, 3348–3349
   _chat_completion_client.py4061995%468, 564–565, 569, 764, 846, 848, 853, 856, 961, 963, 980, 1001, 1009, 1033, 1046, 1079, 1102, 1423
TOTAL46025447990% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
9516 34 💤 0 ❌ 0 🔥 2m 29s ⏱️

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes missing cache write token reporting for OpenAI GPT-5.6 usage by mapping the provider’s cache_write_tokens field into the framework’s standard usage_details key (cache_creation_input_token_count) and ensuring observability can map the provider-specific key to OTel attributes. It also updates the OpenAI prompt-caching sample to include the required prompt_cache_key and to display cache creation vs cache read usage.

Changes:

  • Parse OpenAI usage.input_tokens_details.cache_write_tokens and expose it via usage_details["cache_creation_input_token_count"] (and openai.cache_write_tokens).
  • Add/extend unit tests covering cache write token parsing and OTel usage-key mapping.
  • Update the prompt-caching sample to set prompt_cache_key and print cache creation/read token counts.
Show a summary per file
File Description
python/samples/02-agents/providers/openai/client_prompt_caching.py Adds prompt_cache_key and prints cache creation/read usage to demonstrate GPT-5.6 prompt caching behavior.
python/packages/openai/tests/openai/test_openai_chat_client.py Adds tests for parsing cache_write_tokens and omitting it when missing.
python/packages/openai/agent_framework_openai/_chat_client.py Maps OpenAI cache_write_tokens into both provider-specific and standard usage detail keys.
python/packages/core/tests/core/test_observability.py Verifies OTel attribute mapping for openai.cache_write_tokens.
python/packages/core/tests/core/test_function_invocation_logic.py Adds a blank line (formatting-only change).
python/packages/core/agent_framework/observability.py Adds usage-detail-to-OTel mapping for openai.cache_write_tokens.

Review details

  • Files reviewed: 6/6 changed files
  • Comments generated: 0
  • Review effort level: Low

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Comments suppressed due to low confidence (3)

python/packages/openai/tests/openai/test_openai_chat_completion_client.py:1196

  • This test checks for absence of openai.cache_write_tokens, but the Chat Completions client uses prompt/cache_write_tokens (and with the suggested getattr guard, it will simply omit that key when missing). Update the key being asserted.
    assert "openai.cache_write_tokens" not in details
    assert "cache_creation_input_token_count" not in details
    assert details["cache_read_input_token_count"] == 10

python/packages/openai/agent_framework_openai/_chat_completion_client.py:831

  • prompt_tokens_details.cache_write_tokens is accessed directly, which will raise AttributeError for providers / SDK versions where cache_write_tokens is not present (your own test uses a spec'd mock without it). Use getattr(..., "cache_write_tokens", None) (similar to _chat_client.py) before mapping to usage keys.
            if (tokens := usage.prompt_tokens_details.cache_write_tokens) is not None:
                details["prompt/cache_write_tokens"] = tokens
                details["cache_creation_input_token_count"] = tokens

python/samples/02-agents/providers/openai/client_prompt_caching.py:86

  • The two print(..."\n") calls already add a newline via print(), so embedding \n results in extra blank lines (now doubled vs before). Prefer emitting a single blank line explicitly after printing both cache lines.
        print(f"  Cached input tokens (read): {cached}\n")
        print(f"  Cached input tokens (created): {cached_write}\n")
  • Files reviewed: 8/8 changed files
  • Comments generated: 1
  • Review effort level: Low

@cecheta
cecheta temporarily deployed to github-app-auth July 28, 2026 15:32 — with GitHub Actions Inactive
@cecheta
cecheta force-pushed the prompt-cache-write branch from 9b661a0 to 1224927 Compare July 28, 2026 15:39
@cecheta
cecheta temporarily deployed to github-app-auth July 28, 2026 15:39 — with GitHub Actions Inactive
@eavanvalkenburg

Copy link
Copy Markdown
Member

please remove the edit to the test_function_invocation_logic, the rest looks good @cecheta

@cecheta
cecheta force-pushed the prompt-cache-write branch from 1224927 to 7e38668 Compare July 30, 2026 08:55
@cecheta
cecheta temporarily deployed to github-app-auth July 30, 2026 08:55 — with GitHub Actions Inactive
@eavanvalkenburg
eavanvalkenburg added this pull request to the merge queue Jul 30, 2026
Merged via the queue into microsoft:main with commit ae6923c Jul 30, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: usage_details.cache_creation_input_token_count is missing for gpt-5.6 models

4 participants