Links for August 2023
Fri Sep 01 2023E.W.Ayers
An attempt was made to go through my reading list. It's a Hydra and every link I read spawns 4 more links on the reading list.
1. Articles
How Figma's multiplayer technology works. Lots of high quality engineering articles.
Reflections on a decade of coding is fantastic, really helped me think about how to improve my own coding practices
How to Test. I want to read more of this blog.
Chris Warrick gives a very good article on why Python packaging is so broken and fragmented. This is an absolutely massive problem for Python.
2. LLMs taking over
If you want a non-brogrammer AI newsletter the best is Zvi Mowshowitz' Substack.
I've been told Ben's Bites is also good, will be checking it out next month.
The penny is starting to drop as to how LLMs are going to be used for software in the short term:
Davis Treybig: Software Systems in a World of LLMs
Geoffrey Litt Malleable software in the age of LLMs
100k Context: all the tricks in one place. There are loads of tricks and optimisations:
Better positional encoding (Alibi)
Sparse attention
Optimisations (FlashAttention, etc) What this and the various surrounding papers are telling me is we are very far from finding the best way of implementing attention for LLMs. Lots of software overhang.
ml-ane-transformers Apple's Python reference implementation for deploying transformers on Apple hardware.
3. Products
Replicache looks like a cool way of syncing state locally to get that sweet webapp snappyness.
I hadn't seen MobX before, looks like we've come back round to data-binding a la Knockout.
Factorio announce an expansion, so my productivity will nosedive in about a year's time.
I had a go at using Zig, but there is no version manager and the Vscode support isn't there yet. I want to return to Zig. I'm also interested in the rr debugger.
Really want to check out Salsa, it looks similar to what we were trying to do with HitSave.
asyncpg, I will be trying this next time I use Postgres from Python. Psycopg is a pain to use.
LabML looks like a good way of keeping track of new papers.
3.1. Whatever happened to LightTable?
While poring through Jamie Brandon's blog I discovered he worked on Eve which was a postcursor to LightTable.
I remember getting excited about LightTable during uni (it's a 'instarepl' IDE like replit), it looks like they abandoned it and switched to a Bret-Victor-ish product Eve, but it shut down. There is a post-mortem video, part 2. The main thing I got from this was that LightTable actually influenced IDE design a lot in the 2010s; eg Chris Lattner cites LightTable as an inspiration for XCode Playgrounds (see 'Swift' section in above link). Also appears to have inspired Replit. I guess they couldn't capture any of the market. I am so fascinated with why all these Bret Victor alternative programming systems never seem to catch on. I want to compile a list of them all and see what happened to them. Replit seems to be the first one that has got some momentum (maybe Glitch before that?).
It's such a common trap: programmers thinking that everyone is doing coding wrong and trying to make their own language/framework/database to fix everything. I've found myself teetering over that precipice multiple times. The trouble is: I keep coming back.
4. Media
Where's My Flying Car? I want to read more from Stripe Press.
Return of the Obra Dinn is a mystery game set on a boat.
Ayoade on Top relistening to the audiobook it's funny.
5. Random
There is a Linguistics Olympiad. I love Problem 1. See also Pasigraphy