---
title: Source content from a remote API
description: "Implement IContentService over a typed HttpClient to turn a remote JSON API — here the GitHub Releases API — into routed pages, navigation, search, and xref targets, with one cached fetch per build, rendered markdown bodies, and a fixture fallback when the API is down."
canonical_url: https://usepennington.net/how-to/content-services/source-from-a-remote-api/
sidecar_url: https://usepennington.net/how-to/content-services/source-from-a-remote-api.md
content_hash: sha256:373b574f19828d8b7785bfcc3d979df23e17486c3789bc4db4e8da6fe3f43f3e
tokens: 3051
uid: how-to.content-services.remote-api
reading_time_minutes: 7
---

Guides
# Source content from a remote API

Implement IContentService over a typed HttpClient to turn a remote JSON API — here the GitHub Releases API — into routed pages, navigation, search, and xref targets, with one cached fetch per build, rendered markdown bodies, and a fixture fallback when the API is down.

 
To build pages from a remote HTTP API instead of local files — a release feed, a CMS, a product catalog behind a JSON endpoint — implement `IContentService` over a typed `HttpClient`. This guide is the remote counterpart to [Source content from outside the markdown pipeline](https://usepennington.net/how-to/content-services/custom-content-service.md): that page covers the discovery, TOC, and cross-reference work every content service shares. This one adds the four things a *network* source needs:

 
 - awaiting HTTP in `DiscoverAsync`,
 - caching one fetch across every pipeline pass,
 - rendering markdown bodies that arrive over the wire,
 - and surviving a slow or unreachable API at build time.
 
 
The recipe references `examples/BeyondRemoteContentExample`, which turns the [GitHub Releases API](https://docs.github.com/en/rest/releases/releases) into `/releases/{version}/` pages.

 
## Before you begin

 
 - A working Pennington site on bare `AddPennington` (see [Create your first Pennington site](https://usepennington.net/tutorials/getting-started/first-site.md)).
 - The discovery shape from [Source content from outside the markdown pipeline](https://usepennington.net/how-to/content-services/custom-content-service.md) — this guide assumes you know how `DiscoverAsync`, `GetContentTocEntriesAsync`, and `GetCrossReferencesAsync` fit together, and focuses on what changes when the source is remote.
 - An API reachable over HTTP that returns JSON. The example uses an unauthenticated public endpoint; for an authenticated API, add the token to the typed client's default headers.
 
 
## Fetch the data with a typed `HttpClient`

 
Register a typed `HttpClient` with `AddHttpClient<T>`. Set a `User-Agent` — the GitHub API answers `403` without one — and a `Timeout`, so a stalled API cannot hang the build:

 
```csharp
builder.Services.AddHttpClient<GitHubReleasesClient>(client =>
{
    client.BaseAddress = new Uri("https://api.github.com/");
    client.DefaultRequestHeaders.UserAgent.ParseAdd("Pennington-Remote-Content-Example");
    client.Timeout = TimeSpan.FromSeconds(10);
});
```

 
The client itself is a thin wrapper that fetches and deserializes. `GetFromJsonAsync` with a snake-case naming policy maps GitHub's `tag_name`/`html_url`/`published_at` onto a PascalCase record:

 
```csharp:symbol
public async Task<ImmutableList<GitHubRelease>> GetReleasesAsync()
{
    try
    {
        // The User-Agent header (set in Program.cs) is required — GitHub answers
        // 403 without one. per_page caps the page; a busy repo paginates via the
        // response Link header.
        var releases = await _http.GetFromJsonAsync<List<GitHubRelease>>(
            $"repos/{Owner}/{Repo}/releases?per_page=20", JsonOptions);
  
        if (releases is { Count: > 0 })
        {
            return [.. releases.Where(r => !r.Draft)];
        }
  
        _logger.LogWarning("GitHub returned no releases; using the bundled fixture.");
    }
    catch (Exception ex) when (ex is HttpRequestException or TaskCanceledException or JsonException)
    {
        // Fail open: a slow, unreachable, or rate-limited API must not break the
        // build. To fail the build instead, rethrow here and let it propagate.
        _logger.LogWarning(ex, "GitHub releases fetch failed; using the bundled fixture.");
    }
  
    return await LoadFixtureAsync();
}
```

 
The `try`/`catch` is the build-time failure boundary — covered under Handle a slow or unreachable API below.

 
## Cache the fetch across every pass

 
`DiscoverAsync`, the TOC pass, the cross-reference pass, and the rendering endpoint each need the data. Fetching in each one would hit the API four-plus times per build. Cache the result in an `AsyncLazy<T>` (from `Pennington.Infrastructure`) created once in the constructor, and `await` it everywhere:

 
```csharp
private readonly AsyncLazy<ImmutableList<ReleaseEntry>> _entriesLazy;
  
public GitHubReleasesContentService(GitHubReleasesClient client)
    => _entriesLazy = new AsyncLazy<ImmutableList<ReleaseEntry>>(() => LoadAsync(client));
```

 
`AsyncLazy<T>` runs its factory once on first access and replays the same task to every later caller; a faulted fetch is evicted so the next access retries. `DiscoverAsync` awaits it like every other pass, pairing each route with `EndpointSource` so the build crawler fetches the URL through the endpoint below:

 
```csharp:symbol
public async IAsyncEnumerable<DiscoveredItem> DiscoverAsync()
{
    yield return new DiscoveredItem(
        ContentRouteFactory.FromUrl(new UrlPath("/releases/")),
        new EndpointSource());
  
    foreach (var entry in await _entriesLazy)
    {
        yield return new DiscoveredItem(
            ContentRouteFactory.FromUrl(new UrlPath($"/releases/{entry.Version}/")),
            new EndpointSource());
    }
}
```

 
Because this service reads no local files, there is nothing to file-watch — register it as a process-lifetime singleton (see Register the service). A service backed by files that change during a dev session needs the file-watched lifetimes described in [Publish a custom feed from a content service](https://usepennington.net/how-to/feeds/custom-feed.md), or it serves stale data.

 
## Render the API's markdown body yourself

 
GitHub returns each release's notes as markdown. `EndpointSource` routes are *not* run through Markdig automatically — the framework hands the route to your endpoint and renders nothing — so call `IContentRenderer` yourself. Build a `ParsedItem` from the markdown and render it; the result's `Content.Html` is the same output a markdown file would produce:

 
```csharp:symbol
public static async Task<string> RenderDetailAsync(IContentRenderer renderer, ReleaseEntry entry)
{
    var route = ContentRouteFactory.FromUrl(new UrlPath($"/releases/{entry.Version}/"));
    var parsed = new ParsedItem(route, new ReleaseFrontMatter(entry.Title), entry.BodyMarkdown ?? "");
    var rendered = await renderer.RenderAsync(parsed);
  
    var body = rendered.Value is RenderedItem item
        ? item.Content.Html
        : "<p>Release notes are unavailable.</p>";
  
    return Page(entry.Title, $"""
        <p class="meta">
          <time datetime="{entry.Date:yyyy-MM-dd}">{entry.Date:yyyy-MM-dd}</time>
          &middot; <a href="{entry.HtmlUrl}">View on GitHub</a>
        </p>
        {body}
        """);
}
```

 
Wrap that HTML in the element named by `SiteProjection.ContentSelector`. Set the selector to match that element in the `AddPennington` callback — here both are `<article>`:

 
```csharp
builder.Services.AddPennington(penn =>
{
    // The endpoint below wraps each release body in <article>; point the
    // projection selector at that element.
    penn.SiteProjection.ContentSelector = "article";
});
```

 
At build time the projection self-fetches every TOC-listed route through the live pipeline and splits the selected element into heading sections — so an `EndpointSource` page rendered this way **is indexed at heading level for search and llms.txt**, exactly like a markdown page. Without the selector match, the page chrome leaks into the index instead of the release body.

 
> [!NOTE]
> Because each release page serves real canonical HTML at a stable URL, it **is** included in `sitemap.xml` — `EndpointSource` routes are crawled like any other page. (Only `RedirectSource` and `LlmsOnlySource` routes, which have no canonical HTML, are left out.) See [Publish a sitemap](https://usepennington.net/how-to/feeds/sitemap.md).

 
## Register the service

 
Register the concrete type, then forward `IContentService` to the same instance so the endpoint and the pipeline share one cache:

 
```csharp
builder.Services.AddSingleton<GitHubReleasesContentService>();
builder.Services.AddSingleton<IContentService>(sp =>
    sp.GetRequiredService<GitHubReleasesContentService>());
```

 
A singleton holding a typed `HttpClient` is normally discouraged — the pooled handler stops rotating, so long-lived processes can pin stale DNS. Here it is fine: the client is used for a single fetch at startup and never again; the *cache*, not the client, is what lives for the process. For a long-running host that re-fetches periodically, inject `IHttpClientFactory` and create a client per fetch instead.

 
## Handle a slow or unreachable API at build time

 
A build that fetches from the network inherits the network's failure modes: the API can be down, rate-limited, or slow. The `Timeout` on the typed client bounds the slow case. For the rest, `GetReleasesAsync` **fails open** — on `HttpRequestException`, a timeout, or malformed JSON it logs a warning and falls back to a committed fixture (`fixtures/github-releases.json`), so an offline or rate-limited build still produces a complete site. The fixture also makes the build deterministic in CI.

 
To **fail closed** instead — stop the build when the data is unavailable rather than ship a snapshot that may be stale or empty — rethrow from the `catch` and let it propagate out of `DiscoverAsync`. Choose per site: a marketing build might prefer last-known-good data; a compliance build might prefer to fail loudly.

 
## Keep the published data fresh

 
> [!WARNING]
> A static build is a point-in-time snapshot. The published site shows the releases that existed *when you built it* and will not change until you build again — a new release on GitHub does not appear on its own.

 
For content that updates on its own schedule, rebuild on a schedule. The only thing a scheduled rebuild adds to an ordinary deploy workflow is a `schedule` trigger — a `cron` entry that fires the same build-and-deploy job on a timer instead of (or alongside) a push:

 
```yaml
on:
  schedule:
    - cron: "0 6 * * *"   # rebuild nightly at 06:00 UTC
  workflow_dispatch:       # plus a manual trigger
```

 
The job that this trigger runs — checkout, `setup-dotnet`, `dotnet run -- build`, and the deploy steps — is the same one a push build uses. Build it from [Deploy to GitHub Pages](https://usepennington.net/how-to/deployment/github-pages.md) and add the `schedule` trigger above.

 
## Verify

 
 - Run `dotnet run --project examples/BeyondRemoteContentExample` and open `/releases/`. The index lists every release and each `/releases/{version}/` renders its notes as formatted HTML (headings, lists, links), not raw markdown.
 - Run `dotnet run --project examples/BeyondRemoteContentExample -- build output`. Confirm `output/releases/` has one folder per release, the rendered markdown carries `<h2>` headings, `output/search/` indexes those heading texts (proving the `EndpointSource` bodies reach search via the self-fetch), and `output/sitemap.xml` lists every `/releases/{version}/` URL.
 - Disconnect from the network and rebuild. The build still succeeds, serving the fixture snapshot.
 
 
## Related

 
 - How-to: [Source content from outside the file system](https://usepennington.net/how-to/content-services/custom-content-service.md) — the base `IContentService` shape this guide builds on.
 - How-to: [Publish a custom feed from a content service](https://usepennington.net/how-to/feeds/custom-feed.md) — the DI lifetimes for a *file-backed* service, and the stale-cache trap.
 - How-to: [Publish a sitemap](https://usepennington.net/how-to/feeds/sitemap.md) — what the sitemap includes and excludes.
 - Background: [Why ContentSource is a union](https://usepennington.net/explanation/core/content-source.md) — what `EndpointSource` means and why its pages are still listed in the sitemap.
 
 
[Previous
                
                Source content from outside the markdown pipeline](https://usepennington.net/how-to/content-services/custom-content-service.md)[Next
                    
                Auto-generate an API reference tree for a class library](https://usepennington.net/how-to/content-services/auto-api-reference.md)