Simon Willison builds database-agnostic sqlite-utils with Codex

API applied to other engines

Simon Willison has started prototyping a new library called alchemy-utils, which aims to provide the same interface as his popular sqlite-utils, but backed by SQLAlchemy to support various database engines. The initial compatibility test targets PostgreSQL. The project began as a quick morning experiment to see how AI models handle porting a specific API.

Developers get a unified toolset

The sqlite-utils library is popular for quickly manipulating and introspecting data in SQLite databases without complex setup. If the same API, specifically the data insertion and update functions (insert, upsert) and table introspection, starts working on large production databases like PostgreSQL, developers will no longer need to switch context and syntax based on the underlying storage.

The limits of an abstraction layer

Relying on SQLAlchemy as a translation layer always carries the risk of losing database-specific optimizations and features. Models can generate a rough code translation very quickly, but ensuring that the abstraction layer works reliably across all edge cases in different SQL dialects requires much more than just a functional demo.

Model optimization cut the load to 35 seconds

The ability of AI to generate a prototype is one thing, but Willison immediately involved the model in debugging. Importing a dataset of all trees in San Francisco into DuckDB originally took nearly an hour. Willison then tasked Codex with optimization, bringing the execution time down to about 35 seconds, demonstrating the real value of the model for iterative refactoring.

Lilith's verdict

Describing a well-defined interface of an existing library to models and having them write an adapter is exactly the kind of work where AI shines. But adding immediate optimization that cuts an hour-long task to half a minute shows how copilots are becoming full-fledged developers.