Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/ServiceComposer/ServiceComposer.AspNetCore/llms.txt

Use this file to discover all available pages before exploring further.

HttpGatherer is the built-in Gatherer<JsonNode> implementation that fetches data from a remote HTTP endpoint and returns the response items as a collection of JsonNode objects ready for aggregation. It uses a named IHttpClientFactory client (keyed on Key) to send a GET request, forwards incoming request headers to the downstream call by default, appends the incoming query string to the destination URL, and parses the expected JSON array response. HttpGatherer is designed to cover the most common case — a downstream REST endpoint that returns a JSON array — while still providing multiple extension points for subclassing when its defaults do not fit.

​
Constructor

public HttpGatherer(string key, string destinationUrl)
​
key
string
required
Unique identifier for this gatherer. Used as the named HttpClient key when resolving IHttpClientFactory, as the display name in telemetry spans, and in log messages. Throws ArgumentException if null or whitespace.
​
destinationUrl
string
required
The base URL of the downstream endpoint (e.g., "https://catalog/api/products"). The query string from the incoming request is appended by DefaultDestinationUrlMapper. Throws ArgumentException if null or whitespace.
var gatherer = new HttpGatherer(
    key: "CatalogItems",
    destinationUrl: "https://catalog-service/api/items");

​
Instance Properties

​
Key

public string Key { get; }
The unique key supplied in the constructor. Read-only after construction.

​
DestinationUrl

public string DestinationUrl { get; }
The base destination URL supplied in the constructor. Read-only after construction. The actual URL used at runtime is produced by DestinationUrlMapper (or MapDestinationUrl in a subclass).

​
DestinationUrlMapper

public Func<HttpRequest, string, string> DestinationUrlMapper { get; set; }
Delegate that transforms the base DestinationUrl into the final request URL. Defaults to DefaultDestinationUrlMapper, which appends the incoming request’s query string when present. Replace this property to apply custom URL rewriting, add path segments based on route values, or perform any other per-request URL manipulation. 
var gatherer = new HttpGatherer("Products", "https://catalog/api/products")
{
    DestinationUrlMapper = (request, destination) =>
    {
        var category = request.RouteValues["category"];
        return $"{destination}/{category}";
    }
};

​
ForwardHeaders

public bool ForwardHeaders { get; set; }
When true (default), all incoming request headers are forwarded to the downstream HttpRequestMessage via HeadersMapper. Set to false to suppress header forwarding entirely.

​
HeadersMapper

public Action<HttpRequest, HttpRequestMessage> HeadersMapper { get; set; }
Delegate applied to the outgoing HttpRequestMessage before the downstream call is sent. Defaults to DefaultHeadersMapper, which copies every incoming header using TryAddWithoutValidation. Replace this to filter sensitive headers, add authentication tokens, or otherwise customise the outgoing header set. 
var gatherer = new HttpGatherer("Secure", "https://secure-service/api/data")
{
    HeadersMapper = (incomingRequest, outgoingMessage) =>
    {
        // Only forward the Authorization header
        if (incomingRequest.Headers.TryGetValue("Authorization", out var auth))
            outgoingMessage.Headers.TryAddWithoutValidation("Authorization", (IEnumerable<string>)auth);
    }
};

​
IgnoreDownstreamRequestErrors

public bool IgnoreDownstreamRequestErrors { get; set; }
When true, any HttpRequestException thrown during the downstream call (including network failures and non-2xx status codes) is swallowed and an empty result is returned, allowing the other gatherers to still contribute to the composed response. Defaults to false, which causes the exception to propagate and fail the entire composed request. 
var gatherer = new HttpGatherer("OptionalEnrichment", "https://enrichment/api/extras")
{
    IgnoreDownstreamRequestErrors = true
};

​
Static Properties

​
DefaultDestinationUrlMapper

public static Func<HttpRequest, string, string> DefaultDestinationUrlMapper { get; }
The default DestinationUrlMapper implementation. When the incoming request has no query string the destination URL is returned unchanged; when a query string is present it is appended verbatim (e.g., https://catalog/api/items?page=2&size=10).

​
DefaultHeadersMapper

public static Action<HttpRequest, HttpRequestMessage> DefaultHeadersMapper { get; }
The default HeadersMapper implementation. Iterates over every header in the incoming HttpRequest and calls TryAddWithoutValidation on the outgoing HttpRequestMessage, forwarding the full header set.

​
Virtual Methods (Extension Points)

Override these in a subclass to customise HttpGatherer’s behaviour without replacing the entire Gather implementation.

​
MapDestinationUrl

protected virtual string MapDestinationUrl(HttpRequest request, string destination)
Called by Gather to produce the final request URL. The default implementation delegates to DestinationUrlMapper. Override to implement URL logic that has access to this instance state.
​
request
HttpRequest
required
The current incoming HTTP request.
​
destination
string
required
The value of DestinationUrl.
Returns string — the URL the HttpClient should target.

​
MapHeaders

protected virtual void MapHeaders(HttpRequest request, HttpRequestMessage requestMessage)
Called by Gather after the HttpRequestMessage is created and before it is sent. Respects ForwardHeaders; when false, the method returns immediately without calling HeadersMapper. Override to add headers that depend on instance state or async-resolved values.
​
request
HttpRequest
required
The current incoming HTTP request.
​
requestMessage
HttpRequestMessage
required
The outgoing request message to mutate.

​
TransformResponse

protected virtual async Task<IEnumerable<JsonNode>> TransformResponse(
    HttpResponseMessage responseMessage)
Called by Gather after a successful 2xx response. The default implementation reads the body as a string, parses it as a JSON array, detaches each element from the array (to avoid shared-parent errors when the aggregator stores them), and returns the node collection. Override this when the downstream service returns a different shape — for example, a paginated envelope or a single object instead of an array.
​
responseMessage
HttpResponseMessage
required
The successful downstream response.
Returns Task<IEnumerable<JsonNode>>.

​
Gather

public override async Task<IEnumerable<JsonNode>> Gather(HttpContext context)
The core implementation. Resolves a named IHttpClientFactory client using Key, calls MapDestinationUrl to build the final URL, calls MapHeaders to populate outgoing headers, sends the GET request, calls EnsureSuccessStatusCode(), and then passes the response to TransformResponse. If IgnoreDownstreamRequestErrors is true and an HttpRequestException is thrown, an empty collection is returned and the error is logged as a warning.

​
Subclassing example

public class AuthenticatedCatalogGatherer(string key, string url, string apiKey)
    : HttpGatherer(key, url)
{
    protected override void MapHeaders(HttpRequest request, HttpRequestMessage requestMessage)
    {
        // Start with the default forwarding behaviour
        base.MapHeaders(request, requestMessage);
        // Add a service-level API key
        requestMessage.Headers.TryAddWithoutValidation("X-Api-Key", apiKey);
    }

    protected override async Task<IEnumerable<JsonNode>> TransformResponse(
        HttpResponseMessage response)
    {
        // Downstream returns { "items": [...] } — unwrap the envelope
        var body = await response.Content.ReadAsStringAsync();
        var envelope = JsonNode.Parse(body);
        return envelope?["items"]?.AsArray() ?? [];
    }
}
Note: HttpGatherer uses IHttpClientFactory.CreateClient(Key), so you can configure a named client (base address, retry policy, timeouts) for each gatherer by calling builder.Services.AddHttpClient(key, ...) with the same key.

Build docs developers (and LLMs) love

Get started for freeTalk to us