Possum - Querying public data from your PDS

/posts

I really like the idea of having a place on your personal website where you can just share your thoughts and opinions. When im visiting someone's blog I always look for a /posts somewhere in the page. Tell me more about yourself, and what interesting projects you have been working on!

I tried to make something similar during the first iteration of my website: During the build process of the my pages I also parsed some djot files into html. It worked alright for a while, but figuring out how to make syntax highlight work on my code blocks was not worth the effort at the time. Thats why I started using Leaflet for my blog posts.

standard.site

Leaflet feels really nice to use. The text editor UX is great and code blocks work no matter what language im decide to write.

The only problem is that now I dont have direct access to my posts: they are not stored on my website's codebase anymore, they live in my Personal Data Server (PDS), together with the rest of my records.

The good news is that its all public. Maybe, if I could find a way to access them, I could display my list of posts again! After some research I found that I can make HTTP requests directly to my PDS in order to query the data stored in its records, receiving the response as a JSON object. After I learned how to use the XRPC API, all I needed to do was to parse the response using the decode module from the gleam standard library.

const url = "https://bsky.social/xrpc/com.atproto.repo.listRecords"
  <> "?repo=kacaii.dev"
  <> "&collection=site.standard.document"

fn fetch_documents() -> effect.Effect(Message) {
  let decoder = {
    let value_decoder = decode.at(["value"], document_decoder())
    use documents <- decode.field("records", decode.list(value_decoder))
    decode.success(documents)
  }

  let handler = rsvp.expect_json(decoder, ApiReturnedDocuments)
  rsvp.get(url, handler)
}

Good enough, now I can continue writing on Leaflet peacefully, and my website will take care of displaying a preview of the latests posts.

Possum

It's just a HTTP request targetting an endpoint, despite the concept being simple it takes a bit of setup in order to do it correctly: You need to know what's your Decentralized Identifier (DID) and you also need to know where your PDS is located. If I can take care of that, I believe this could help other projects. There are quite a lot of gleam developers interested in the AT Protocol, but not many atproto related packages.

I decided to make a library for building this kind of requests. The code is actually really simple when you look closely: It just uses the gleam core libraries for HTTP and JSON. The user can take care of deciding how to send the requests and how to decode them.

const com_atproto_identity = "xrpc/com.atproto.identity"

/// Resolves an atproto handle (hostname) to a DID.
pub fn resolve_handle(
  request: request.Request(_),
  handle handle: String,
) -> request.Request(_) {
  request
  |> request.set_header("accept", "application/json")
  |> request.set_method(http.Get)
  |> request.set_path(com_atproto_identity <> ".resolveHandle")
  |> request.set_query([#("handle", handle)])
}

I included some examples on the documentation for the library that can serve as quick references. I want to avoid the usage of invalid identifiers, so I also included a submodule for DID parsing. The AT Protocol community was nice enough to include an basic regex pattern on the documentation.

import possum/did

// did parsing
let string = "did:plc:k5vecqzf4d5mdvkcu3mx6l5g"
let assert Ok(did) = did.parse(string)

Now that you have access to your DID, you'll need to know where your PDS is located. Luckly this can be done with the PLC Directory. Just send a GET request there with your DID on the path, and it will return plenty of useful information, including the address of your Personal Data Server!

/// Get current PLC Data for the indicated DID,
/// this returns information about a Decentralized Identifier (DID),
/// including their handle and service endpoint.
///
/// PLC is a persistent global identifier system, stands for
/// "Public Ledger of Credentials".
pub fn get_plc_data(
  request: request.Request(_),
  did: did.Did,
) -> request.Request(_) {
  request
  |> request.set_header("accept", "application/json")
  |> request.set_host("plc.directory")
  |> request.set_method(http.Get)
  |> request.set_path(did.to_string(did) <> "/data")
}

Building the requests

Now that you have access to both your Identifier and your PDS's endpoint, it will become really easy to query data from it. Just set the request host to your PDS, pass your DID as parameter to the functions and decode the response received into a custom type for your application. I also included a utility function for fetching your standard.site documents, its actually just a wrapper around a function that lists all records attached a collection.

fn fetch_documents(atproto: atproto.AtProto) -> effect.Effect(Message) {
  let decoder = {
    let value_decoder = decode.at(["value"], document_decoder())
    use documents <- decode.field("records", decode.list(value_decoder))
    decode.success(documents)
  }

  let request =
    request.new()
    |> request.set_host(atproto.pds)
    |> possum.list_standard_documents(atproto.did, option.None)

  let handler = rsvp.expect_json(decoder, ApiReturnedDocuments)
  rsvp.send(request, handler)
}

Possum v1.0.0 is available on hex package manager. I received a lot of support from both gleam and tangled communities, and im quite happy with how the project is going. Constructive feedback is welcome and I'm looking forward to learn more about the atproto ecossystem.