Scraping on Max Halford

Autonomous web scraping with Claude Code

maxhalford25@gmail.com (Max Halford) — Mon, 11 May 2026 00:00:00 +0000

Andrej Karpathy’s autoresearch project made a lot of ripples. I guess it’s exciting because a program that modifies itself feels like actual AI. The concept of autonomous programs is not new – see genetic programming – but what we’re witnessing now is somewhat more convincing. We just got a bit closer to Neuromancer’s world, with a Turing Police that’s in charge of stopping AIs from rewriting themselves.

While we wait for AI to become self-conscious, there are mundane problems that can be solved with autoresearch. Web scraping is one of these. Indeed, turning unstructured data into knowledge is an excellent problem for LLMs to solve. But there are nuances to have in mind:

Scraping Google Calendar events

maxhalford25@gmail.com (Max Halford) — Sun, 12 Oct 2025 00:00:00 +0000

At my day job we deal with enterprise customers. They pay us a subscription fee, and in return we help them in various ways to reduce their carbon footprint. To keep the boat afloat, we need to make some money. We shouldn’t spend more money than we make. So we need to keep track of our revenue and costs. Our gross margin is (revenue - cost) / revenue, where the cost is mostly the salaries of our employees.

Do LLMs identify fonts?

maxhalford25@gmail.com (Max Halford) — Wed, 30 Jul 2025 00:00:00 +0000

Spoiler: not really

dafont.com is a wonderful website that contains a large collection of fonts. It’s more comprehensive and esoteric than Google Fonts. One of its features is a forum where users can ask for help identifying fonts – check out this poor fellow who’s been waiting for over two years and bumped his thread. I thought it would be interesting to see if an LLM could do this task, so I scraped the forum and set up a benchmark.

Web scraping, upside down

maxhalford25@gmail.com (Max Halford) — Thu, 11 Nov 2021 00:00:00 +0000

Motivation

Web scraping is the art of extracting information from web pages. A web page is essentially an amalgamation of HTML tags. Usually, we’re looking for a particular piece of information on a given web page. This may be done by fetching the HTML content of the page in question, and then running some HTML parsing logic. It’s quite straightforward.

There are many tools in the wild to perform web scraping. For instance, in Python, you may use requests in combination with Beautiful Soup. You can also automate some of the more mundane aspects of scraping by using Scrapy.

Manipulating ephemeral data with git

maxhalford25@gmail.com (Max Halford) — Thu, 07 Oct 2021 00:00:00 +0000