Python: feat(observability): add support for OpenAI cache write tokens in usage details - #7369
Conversation
There was a problem hiding this comment.
Pull request overview
This PR fixes missing cache write token reporting for OpenAI GPT-5.6 usage by mapping the provider’s cache_write_tokens field into the framework’s standard usage_details key (cache_creation_input_token_count) and ensuring observability can map the provider-specific key to OTel attributes. It also updates the OpenAI prompt-caching sample to include the required prompt_cache_key and to display cache creation vs cache read usage.
Changes:
- Parse OpenAI
usage.input_tokens_details.cache_write_tokensand expose it viausage_details["cache_creation_input_token_count"](andopenai.cache_write_tokens). - Add/extend unit tests covering cache write token parsing and OTel usage-key mapping.
- Update the prompt-caching sample to set
prompt_cache_keyand print cache creation/read token counts.
Show a summary per file
| File | Description |
|---|---|
| python/samples/02-agents/providers/openai/client_prompt_caching.py | Adds prompt_cache_key and prints cache creation/read usage to demonstrate GPT-5.6 prompt caching behavior. |
| python/packages/openai/tests/openai/test_openai_chat_client.py | Adds tests for parsing cache_write_tokens and omitting it when missing. |
| python/packages/openai/agent_framework_openai/_chat_client.py | Maps OpenAI cache_write_tokens into both provider-specific and standard usage detail keys. |
| python/packages/core/tests/core/test_observability.py | Verifies OTel attribute mapping for openai.cache_write_tokens. |
| python/packages/core/tests/core/test_function_invocation_logic.py | Adds a blank line (formatting-only change). |
| python/packages/core/agent_framework/observability.py | Adds usage-detail-to-OTel mapping for openai.cache_write_tokens. |
Review details
- Files reviewed: 6/6 changed files
- Comments generated: 0
- Review effort level: Low
5e82204 to
0260286
Compare
There was a problem hiding this comment.
Review details
Comments suppressed due to low confidence (3)
python/packages/openai/tests/openai/test_openai_chat_completion_client.py:1196
- This test checks for absence of
openai.cache_write_tokens, but the Chat Completions client usesprompt/cache_write_tokens(and with the suggestedgetattrguard, it will simply omit that key when missing). Update the key being asserted.
assert "openai.cache_write_tokens" not in details
assert "cache_creation_input_token_count" not in details
assert details["cache_read_input_token_count"] == 10
python/packages/openai/agent_framework_openai/_chat_completion_client.py:831
prompt_tokens_details.cache_write_tokensis accessed directly, which will raiseAttributeErrorfor providers / SDK versions wherecache_write_tokensis not present (your own test uses a spec'd mock without it). Usegetattr(..., "cache_write_tokens", None)(similar to_chat_client.py) before mapping to usage keys.
if (tokens := usage.prompt_tokens_details.cache_write_tokens) is not None:
details["prompt/cache_write_tokens"] = tokens
details["cache_creation_input_token_count"] = tokens
python/samples/02-agents/providers/openai/client_prompt_caching.py:86
- The two
print(..."\n")calls already add a newline viaprint(), so embedding\nresults in extra blank lines (now doubled vs before). Prefer emitting a single blank line explicitly after printing both cache lines.
print(f" Cached input tokens (read): {cached}\n")
print(f" Cached input tokens (created): {cached_write}\n")
- Files reviewed: 8/8 changed files
- Comments generated: 1
- Review effort level: Low
9b661a0 to
1224927
Compare
|
please remove the edit to the test_function_invocation_logic, the rest looks good @cecheta |
1224927 to
7e38668
Compare
Motivation & Context
Add cache write tokens to usage details in GPT-5.6 models.
Description & Review Guide
cache_creation_input_token_countadded to usage details for both Responses and Chat Completions API.prompt_cache_keyis required to use the improved caching).openai.cache_write_tokensto OpenTelemetryRelated Issue
Fixes #7368
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.