Fix #223: URI encode links - #224
Merged
bezzad merged 2 commits intoApr 25, 2026
Merged
Conversation
… URI parsing Downloads with URLs containing RFC 3986-illegal path characters (e.g. square brackets in release-group tags like "[SubGroup] Show - 01 [1080p].mkv", curly braces, or unencoded spaces) were failing on Linux. The .NET URI parser on Linux is stricter than on Windows, so Request.Address either fell back to a bogus http://localhost base or produced a URI the HTTP stack transmitted incorrectly. Add UrlHelper.EnsurePathEncoded — a small, idempotent normalizer that percent-encodes non-pchar path characters as UTF-8 while preserving scheme, userinfo, host (including IPv6 literal brackets like [::1]), port, query, and fragment. Wire it in at every ingestion point where a server-supplied or user-supplied URL string reaches new Uri(): Request ctor, SocketClient redirect-Location handling, and the Referer header. Add UrlHelperTest covering brackets, braces, spaces, Unicode, IPv6 hosts, userinfo, query/fragment preservation, idempotency, and lone '%' handling.
Follow-up to the initial issue bezzad#223 fix, prompted by a self-review. No behavior changes to the encoder itself — only hardening around it: - Expand UrlHelper class docstring with the four security invariants it relies on (never decode, always encode control chars, authority untouched, output Uri-parseable) so a future maintainer understands why each property matters and doesn't relax one by accident. - Document the treatment of literal '?' and '#' in callers' filenames (must be pre-encoded). - Replace ReferenceEquals coupling between EnsurePathEncoded and EncodePath with straightforward string equality; the fast path still allocates nothing. - Add guard comments at both SocketClient call sites (Referer and redirect Location) explaining why the UrlHelper wrapper must not be removed in future refactors. Tests added to lock in the security posture: - Control characters (CR, LF, CRLF, TAB, NUL, SOH, FF, DEL) are all percent-encoded as UTF-8. - A CRLF injection payload ("\r\nHost: evil.com\r\nGET /admin") is defused — the attempted header/request-line smuggle becomes percent-escaped content inside the path. - Already-encoded input (%2e%2e%2f) is never unescaped, so server-side path-traversal defenses see the attacker's true input. - Authority is never altered by path encoding. - Request ctor handling of a CRLF-containing URL yields a clean AbsoluteUri with no raw control chars. - Empty path segments (double slashes) and scheme-only inputs pass through unchanged. Strengthen RequestConstructorAcceptsBracketedUrl to assert explicitly that we don't fall back to the http://localhost base URI and that AbsolutePath contains no raw '[' or ' '.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #223. Originally surfaced via the downstream consumer rdt-client#962.
Problem
Request.cspasses the raw address string straight toUri.TryCreate:Per RFC 3986, square brackets are reserved for IPv6 literals in the host component only;
[,],{,}, and unencoded spaces are illegal in a path segment and must be percent-encoded. .NET's URI parser is platform-sensitive here:TryCreateeither returnsfalse(falling through to thehttp://localhostbase-URI branch and producing a 404) or yields aUrithatHttpClienttransmits incorrectly.Real-world triggers are Debrid download links (Real-Debrid, TorBox, AllDebrid) that embed the torrent filename, e.g.:
Fix
Added
UrlHelper.EnsurePathEncoded(string)insrc/Downloader/Extensions/UrlHelper.cs. It:[::1]), port, query, and fragment are preserved byte-for-byte.pcharsafe set — alphanumerics plus-._~!$&'()*+,;=:@. Everything else in a segment is percent-encoded as UTF-8. This matches the behavior of Go'snet/url.PathEscape, Python'surllib.parse.quote, and Node's WHATWG URL, so downloads now send the same request linecurlor a browser would.%XXtriplets pass through unchanged (lowercase hex is normalized to uppercase). Safe on already-encoded or partially-encoded input.Uri, becauseUriis exactly what rejects these URLs on Linux.Wired in at every ingestion point where a server- or user-supplied URL string reaches
new Uri():Request.csAbstractDownloadService.InitialDownloaderSocketClient.csLocationheaderSocketClient.csRefererheaderNo public API changes.
UrlHelperisinternal.Why a helper, not an inline fix?
Three call sites, subtle correctness requirements (idempotency, IPv6-host preservation, pchar set). Centralizing gives one well-tested implementation rather than three near-duplicates.
Tests
Added
UrlHelperTest.cswith 26[Theory]cases plus 3[Fact]checks:[SubGroup] ... [1080p].mkv→ correctly encoded[ ][::1],[2001:db8::1]) left untouched — only path encodeduser:pass@host) preserved:@preserved in path (not over-encoded)file://scheme works the same way%(not followed by two hex digits) encoded to%25?or#but no path — unchangedUri.UnescapeDataStringRequestconstructor integration: bracketed Debrid URL now yields correctHostandAbsoluteUriRisk / compatibility
Uripreviously encoded many of these characters internally anyway; normalizing one step earlier produces identical wire output. No observable regression.DownloadPackage.Urls: these come fromreq.Address.OriginalString. After this change that's the normalized form. Resuming a package feeds the URL back throughRequest→ idempotent → identicalAddress. No migration needed.