David Gasquez

Wrote about why the AT Protocol is the ultimate API for you and your agents!

davidgasquez.com/atproto-agents

We need more people building on the atmosphere!

New data oriented file format just dropped.

FastLanes, "like Parquet, but with 40% better compression and 40× faster decoding". 👀

Seems it can exploit correlations between columns and have fully SIMD friendly encodings to help with vectorization.

github.com/cwida/FastLa...

GitHub - cwida/FastLanes: Next-Gen Big Data File Format

Next-Gen Big Data File Format. Contribute to cwida/FastLanes development by creating an account on GitHub.

It doesn't only do Office files! You can use it to get YouTube transcript or transform articles to Markdown!

uvx --python 3.12 markitdown "https://www.youtube.com/watch?v=6KcV1C1Ui5s" | llm "Summarize the video"

kepano @stephango.com

Microsoft just released a tool that lets you convert Office files to Markdown. Never thought I'd see the day. Google also added Markdown export to Google Docs a few months ago. github.com/microsoft/markitdown

GitHub - microsoft/markitdown: Python tool for converting files and office documents to Markdown.

Python tool for converting files and office documents to Markdown. - microsoft/markitdown

In earlier eras of computing unreliable hardware forced system design discipline (checksums, retries, modularity).

There is a lot of that we are rediscovering in the "Agentic Engineering" era!

Wrote a small post about this idea.

davidgasquez.com/reliable-unr...

Wanted to write a Bluesky Python client this weekend. Realized there are a bunch of them much better written than I could! Sharing some of them.

- https://github.com/jbn/psychonaut from @generativist.xyz
- https://github.com/kiliczsh/gokyuzu from @kilic.bsky.social

Behold is the Bluesky World!

Got "The Bluesky Dataset" from @andrewconner.com and used Graphext (from @victoriano.bsky.social) to visualize all the connections. It looks beautiful! 😍

The best thing is that you can explore it too! 👇

https://public.graphext.com/fe28f76fe67518ab/index.html

Wonderful post by @summerscope.bsky.social! Hat tip to @mariozechner.at.

The Human-in-the-Loop is Tired.

pydantic.dev/articles/the...

The Human-in-the-Loop is Tired

On reward functions, dopamine, and what it actually feels like when the code starts writing itself

The future is now!

- Dial keys, not IPs, thanks to Iroh
- Fetch hashes, not URLs, thanks to DASL
- Build on shared records, not siloed databases, thanks to the AT Protocol

Iroh @iroh.computer

We just released the first stable version of iroh, v1.0! Read about it: www.iroh.computer/blog/v1 We're super proud of where we landed with iroh's 1.0. Since version 0.35 it feels like we've entirely replaced the "engine underneath". Are you a fellow key-dialer? Come tell us how you use iroh!

Iroh 1.0 - Dial Keys, not IPs

Iroh 1.0 is out. Now is the time to build.

Inspired by @simonwillison.net llm-docs repo, I did a similar one compiling projects docs into single TXT files that can be fed to LLMs.

Right now, it only has atproto docs but already been useful to me to answer random questions about the project.

github.com/davidgasquez...

GitHub - davidgasquez/docs-to-llmstxt: 🤖 Compile docs into text files for LLMs

🤖 Compile docs into text files for LLMs. Contribute to davidgasquez/docs-to-llmstxt development by creating an account on GitHub.

A Python browser sandbox, by the @pydantic.dev folks!

github.com/pydantic/pyd...

Great to share small snippets of code (e.g: pydantic.run/store/c544b1...).

Wrote a small post exploring different approaches to one of the most common tasks I perform as a data engineer.

Getting data in batches from random APIs.

davidgasquez.com/async-batch-...

David Gasquez

Posts, notes, and projects by David Gasquez

Loved this post from @gordon.bsky.social on the forces of (de)centralization and how networks evolve.

newsletter.squishy.computer/p/centraliza...

Centralization is inevitable

...but decentralization is right behind it.

Wrote a post about debugging @dagster.io in VSCode / Cursor!

If you're like me and use a lot of notebooks to write code, this might be a great alternative that relies on VSCode / Cursor debugger for a smooth development experience with Dagster or similar libraries.

davidgasquez.com/debug-dagste...

TIL about `uv tool install`! 🤦

Previously, I was aliasing packages to `uvx package` and dealing with small lag (installing deps from cache). This is the perfect way to manage Python CLI tools.

Installed llm, open-interpreter, and files-to-prompt.

Found this great tool by @theo.io to find people followed by lots of the people you follow (but not you)!

bsky-follow-finder.theo.io

Write a small and rough post on learnings and the process I took to export 5000 datasets (200GB) from the Spanish National Statistic Institute into 500MB worth of Parquet files on HuggingFace.

davidgasquez.com/exporting-in...

David Gasquez

Posts, notes, and projects by David Gasquez

David Gasquez @davidgasquez.com

Worked on a small side-project during the weekend to export ALL tables from The Spanish National Statistics Institute. - Code: github.com/davidgasquez... - HuggingFace Dataset: huggingface.co/datasets/dav...

Cool to see @hypercerts.org on the What's Hot list of community lexicons! 🤩

Really enjoyed this talk from @bcantrill.bsky.social on why trust _is_ infrastructure.

youtu.be/WF7J7qtZ8TA

Recommended watching!

Trust as Infrastructure | Bryan Cantrill | Monktoberfest 2025

YouTube video by RedMonk Tech Events

When people are pressured to meet a target value there are three ways they can proceed:

1) They can work to improve the system
2) They can distort the system
3) Or they can distort the data

— Donald Wheeler

commoncog.com/goodharts-la...

Goodhart's Law Isn't as Useful as You Might Think

Goodhart's Law is useless. It tells you about a phenomenon, but it doesn't tell you how to solve it. We look at how organisations actually prevent Goodhart's Law, and illustrate this with Amazon's Wee...

DuckDB for processing, Parquet for publishing. Simple and powerful!

www.robinlinacre.com/parquet_api/

Sharing a list of my favorite public data projects!

- @catalyst.coop: github.com/catalyst-coo...
- MIT Open Learning: github.com/mitodl/ol-da...
- @ourworldindata.org: github.com/owid/etl
- @gitlab.com : gitlab.com/gitlab-data/...
- OSO: github.com/opensource-o...

Published the script to generate the followers edge list (source, target) from profiles found in hashtag search.

github.com/davidgasquez...

GitHub - davidgasquez/atproto-data-tools: 🦋 Small scripts and tools to do data stuff with the AT Protocol.

🦋 Small scripts and tools to do data stuff with the AT Protocol. - davidgasquez/atproto-data-tools

David Gasquez @davidgasquez.com

New post up! ✨ Exploring AT Protocol with Python to visualize the #databs social graph! davidgasquez.com/exploring-at... Took less than 1 hour to get the data and plot it. Amazing what you can do with open APIs and great SDKs!

Excited about more people knowing about tools / ideas I've been working with in the last few years.

- Content addressed storage
- Fun data structures
- Non blockchain decentralized networks
- Local first approaches
- Open protocols towards common digital infrastructure

Simon Späti 🏔️ @ssp.sh

Amazing how we get back to the open web with bsky. Great talk. Love the concepts. Anybody knows when the SQLite dbs come into play, in the intermediate aggregator, or on our local client most probably?

Reply

This is also an amazing read if you're interested in the architecture / design of ATProto.

atproto.com/articles/atp...

ATProto for distributed systems engineers - AT Protocol

AT Protocol is the tech developed at Bluesky for open social networking. In this article we're going to explore AT Proto from the perspective of distributed backend engineering.