Skip to content

Fix #2317: Convert non-string dict keys to strings before JSON serialization - #2321

Closed
okiemute04 wants to merge 3 commits into
django-commons:mainfrom
okiemute04:fix-2317-nonstring-dict-keys
Closed

Fix #2317: Convert non-string dict keys to strings before JSON serialization#2321
okiemute04 wants to merge 3 commits into
django-commons:mainfrom
okiemute04:fix-2317-nonstring-dict-keys

Conversation

@okiemute04

@okiemute04 okiemute04 commented Mar 9, 2026

Copy link
Copy Markdown

Description

Fix #2317 by converting non-string keys at the panel level, as recommended by @matthiask.

Problem

When panel data contains dictionaries with non-string keys (tuples, UUIDs), json.dumps() fails when the store tries to serialize the data. The issue affects:

  • CachePanel - when cached values contain dicts with non-string keys
  • RequestPanel - when session data contains dicts with non-string keys

Solution

Instead of fixing this in the store, we now fix it at the panel level where the data is generated:

  1. Added utility function convert_keys_to_strings() that recursively converts non-string dict keys to strings
  2. Fixed CachePanel - converts args/kwargs in _store_call_info before storing
  3. Fixed RequestPanel - converts GET/POST/cookies/session data in generate_stats
  4. Added comprehensive tests for both panels covering tuple keys, UUID keys, and nested structures

Testing

  • ✅ CachePanel: 9/9 tests passing (including new non-string keys test)
  • ✅ RequestPanel: 17/17 tests passing (including 2 new tests for tuple keys)
  • ✅ Utils: 12/12 tests passing

Example

# Before - would crash when store tries to serialize
cache.cache.set((1, 2), "value")  # CachePanel records tuple key

# After - panel converts before storing
cache.cache.set((1, 2), "value")  # Stored as {"(1, 2)": "value"} in panel data

Checklist

  • Added tests
  • Updated code
  • All tests pass
  • No breaking changes
  • Follows maintainer's guidance to fix panels, not store

@matthiask

Copy link
Copy Markdown
Member

What we wanted to achieve by using JSON serialization is to force panels to prepare the data in such a way that it can be serialized as JSON; the cache panel and other panels aren't doing that. Instead of changing some of the data in the store we should instead fix the panels to not try to save such data in the first place I think.

And if we cannot do this we should evaluate going with msgpack or pickle or something like that again. We didn't want to use pickle because unserializing pickle actually runs code and therefore opens cans of worms we didn't want to open.

So, please reevaluate the approach and try fixing the issue in the panels themselves and not in the store.

@okiemute04

Copy link
Copy Markdown
Author

Thanks for the detailed feedback @matthiask! You're right - fixing it in the store masks the real problem. I'll update this PR to fix the panels directly (CachePanel and RequestPanel) so they produce JSON-serializable data from the start. I'll push the changes to this same PR shortly 🚀 .

- Add convert_keys_to_strings utility function
- Fix CachePanel to handle tuple and UUID keys
- Fix RequestPanel to safely handle POST data and convert session keys
- Add comprehensive tests for both panels
- All panel tests passing
@okiemute04
okiemute04 force-pushed the fix-2317-nonstring-dict-keys branch from deb652c to 04e4c68 Compare March 11, 2026 23:59
@okiemute04

Copy link
Copy Markdown
Author

Thanks so much @matthiask! All tests for CachePanel and RequestPanel are passing, including the new tests for non-string keys. The PR now contains only the panel-level fixes as requested.

@tim-schilling

Copy link
Copy Markdown
Member

Hi @okiemute04 I'm closing this PR as it seems to stray too far in the direction of using LLM output for every aspect of contributing. This is based on all the PRs (#2321, #2323, #2324) you've created. If you're willing to come back to interact with more of your self, we would be open to collaborating with you. I've written more about this challenge here.

@okiemute04

Copy link
Copy Markdown
Author

@tim-schilling Thank you so much for the honest feedback.... I understand the concern. I've been using AI tools to help structure my contributions, but I see now that it's made my work feel inauthentic. I genuinely want to contribute to this project and learn from the community and the process.

I'd like to rework these PRs with my own voice and understanding. Would it be acceptable to reopen them? I'm committed to doing the work myself and learning from the process. I am sorry, I take accountability, and I am willing to do the work and contribute authentically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

serialize() crashes with TypeError when panel data contains dicts with non-string keys

3 participants