Feature Proposal: Automatic Cookie Jar

[o-nnerb]

Overview

RequestDL has no cookie handling today — no Cookie type, no jar, nothing. This proposes a full, automatic cookie jar: cookies from Set-Cookie response headers get stored and automatically re-sent on subsequent matching requests, on by default, scoped per named Session.

What's already free, from async-http-client

• RFC 6265 Set-Cookie parsing is already implemented upstream — HTTPClient.Cookie(header:defaultDomain:) handles name/value/path/domain/expires/max-age/httpOnly/secure, including all three date-format variants the spec allows. RequestDL's own Cookie type (see below) delegates to this internally rather than reimplementing RFC 6265 parsing from scratch.
• Cross-origin redirect safety is already handled. transformRequestForRedirect (in async-http-client's RedirectState.swift) already strips the Cookie header when a redirect changes origin — the one security-sensitive edge case here is already correct upstream, nothing to duplicate.

Where the jar lives

Internals.Session is recreated fresh on every request (Resolve.build() constructs a new one each time), so it can't hold cross-request state. Internals.ClientManager (Internals.ClientManager.swift:54) already pools Internals.Client instances, but keys that pool by exact configuration equality, not identity alone (_reusableClient matches on item.sessionConfiguration == sessionConfiguration) — two requests sharing the same named Session can still end up with two different pooled Internals.Client instances if any session-level property between them differs (Timeout, Proxy, anything).

That matters because the cookie jar shouldn't fragment along with client pooling — cookies belong to "this named session" conceptually, the same way URLSession's HTTPCookieStorage doesn't care about per-request connection settings. So Internals.CookieStorage (a final class, reference type) does not live as a stored property on Internals.Client. It lives in Internals.ClientManager itself, in a separate table keyed by sessionProviderID alone — decoupled entirely from the config-equality-keyed client pool. Two differently-configured clients under the same session identity still share one jar.

The two integration points

• Incoming: Internals.ClientResponseReceiver.didReceiveHead(task:_:)
  (Internals/Sources/Session/Session/Models/Internals.ClientResponseReceiver.swift:86) already
  builds the response head from the raw headers and already has url: String (the request URL,
  used as RFC 6265's defaultDomain fallback when a Set-Cookie header omits its own Domain
  attribute) in scope. It gains two more constructor parameters — cookieStorage: Internals.CookieStorage?
  (looked up by session identity) and publicSuffixList: Internals.PublicSuffixList? (resolved per
  request, see below) — and reads head.headers["Set-Cookie"] there, calling
  cookieStorage?.store(cookies, forURL: url, publicSuffixList: publicSuffixList).
• Outgoing: Internals.Session's private execute(client:requestConfiguration:cache:logger:)
  (Internals/Sources/Session/Session/Internals.Session.swift:64) already has enough in scope
  right before requestConfiguration.build(eventLoop:) — that's where the session's cookieStorage
  gets looked up and .cookies(forURL: requestConfiguration.url) folded into a Cookie header on a
  mutated copy of requestConfiguration, before the request is built. No PSL involvement needed on
  this side at all — the public-suffix check only matters when storing an incoming cookie.

Matching semantics (RFC 6265)

Internals.CookieStorage.cookies(forURL:) filters stored cookies by: domain match (exact, or a
suffix match against a cookie whose Domain attribute was set), path match (cookie path is a
prefix of the request path), the secure flag (only included over HTTPS), and expiration (expired
or past-max-age cookies are dropped, not returned). store(_:forURL:publicSuffixList:) (called from the incoming
side) replaces any existing cookie sharing the same name/domain/path, matching RFC 6265's jar-update
semantics, rather than accumulating duplicates.

Merge policy

If a request already carries an explicit Cookie header (via Headers or any other property), the
jar contributes nothing — same "explicit wins over automatic" precedent every other pair of
explicit/automatic behavior in this codebase already follows (an explicit BaseURL/Proxy always
beats an automatically-resolved one).

Public Suffix List — consumer-supplied, not bundled

Without it, a Set-Cookie: session=x; Domain=co.uk response would be valid for every site under .co.uk under plain RFC 6265 domain-matching — a "supercookie." Real browsers guard against this with the Public Suffix List (a list of ~9,000 domain suffixes under which unrelated parties can register), rejecting a cookie whose Domain resolves to a bare public suffix.

RequestDL deliberately does not bundle or maintain a PSL snapshot itself — that's a genuine ongoing maintenance burden (the list changes as new gTLDs and dynamic-DNS providers appear) that doesn't belong to an HTTP networking library, the same reasoning that kept hashing out of #197/#203. Nor can it defer to the OS: on Apple platforms URLSession's protection is backed by a system-level list CFNetwork maintains privately, with no public API for third-party code to query, and swift-corelibs-foundation on Linux has no equivalent at all — deferring to "whatever the platform does" would mean silently weaker protection on Linux than Darwin.

So PSL support is consumer-pluggable, via a new property — a global configuration property, same category as Session(): if not specified, requests simply proceed without the public-suffix check, exactly like an unconfigured Session() just uses defaults.

PSLCookie(file: "/path/to/public_suffix_list.dat")
PSLCookie(data: pslData)

public struct PSLCookie: Property {
    private struct Node: PropertyNode {
        let source: Source

        func make(_ make: inout Make) async throws {
            make.requestConfiguration.publicSuffixList = try await Internals.PublicSuffixList(source)
        }
    }

    public var body: Never { bodyException() }
    let source: Source

    public init(file: String) { self.source = .file(file) }
    public init(data: Data) { self.source = .data(data) }
    // _makeProperty as usual: .leaf(Node(source: property.source))
}

Populating make.requestConfiguration.publicSuffixList (not sessionConfiguration) is deliberate: it keeps PSL data out of the Equatable comparison Internals.ClientManager uses for pool-reuse, so declaring/omitting/changing it never fragments client pooling or the cookie jar the way a session-level field would. PropertyNode.make being async throws lets file reading and parse-failure surface as a real, catchable error, rather than needing a workaround like the ones hit designing @Endpoint/Configured earlier.

Explicit, visible behavior when omitted: a request without PSLCookie still stores incoming cookies — just without the supercookie check. This is stated plainly rather than implied to always be on.

Out of scope (v1)

• Any built-in PSL data source (network fetch, bundled snapshot) — deliberately consumer-supplied only, per above.
• Any manual cookie-inspection/clearing API beyond the automatic jar (e.g. reading what's currently
  stored for a session) — can be added later without changing this design.

Milestone: 4.6.0

[o-nnerb]

Body updated with the Public Suffix List design, moved from "out of scope" into the actual proposal: a new PSLCookie(file:)/PSLCookie(data:) property, consumer-supplied rather than bundled (RequestDL doesn't want the ongoing maintenance burden of tracking publicsuffix.org, and can't defer to the OS either — Darwin's protection is private to CFNetwork with no public API, and Linux's swift-corelibs-foundation has no equivalent at all). Also corrected the cookie jar's own design: CookieStorage is keyed by session identity in Internals.ClientManager, independent of the client pool's exact-configuration-equality keying, so it doesn't fragment when other session-level properties vary between requests. PSLCookie populates RequestConfiguration (per-request), not Session.Configuration, keeping it out of that equality check entirely.