Python is a bit broken
Wed Dec 14 2022E.W.Ayers
For the last few years I've been maintaining a list of things that I don't like about Python. For most of them, I have now seen the light and the main problem was that I was not being 'Pythonic'. There are still some remaining gripes so I am dumping them here.
I recently found a Twitter thread of a person I follow struggling with Python. Python is supposed to be the friendly, easy language and it's a shame that its this clunky.
1. Setting up a Python environment is fragmented and confusing
1.1. Installing python
To start with, just look at the Homebrew entry for Python. It is a three-page list of caveats and recommendations. Interpreter version management comes across as an unsolved problem.
Version 2.7 and 3.x are not compatible with each other. This would be fine except that they are both still used and there is no single way to make sure that you are using the right version (for example, for a new install / image, it is a coin flip whether typing
python in the shell will launch 2.7 or 3) Further, if you forget you can use
pip instead of
pip3 and then have a cycle of headscratching figuring out why your module doesn't work.
Ok, maybe instead we should use this thing called
conda I keep hearing about, maybe that will make it easy.
Another option - that I ultimately ended up sticking with - is PyEnv. Pyenv seemed to be the
How to fix it? The core Python team should adopt PyEnv or roll their own version manager, similar to
rustup for the Rust compiler.
1.2. Python packaging is obtuse and fragmented
Do you know the difference between
All of these help you manage packages, versioning and environments in some way. This SO answer differentiates some of them, see also this list. If you try using multiple of these tools, you risk putting your environment in an inconsistent broken state with lots of scary error messages. For a newcomer to programming, this makes Python borderline unusable. Having a fragmented set of third-party tooling for something as fundamental as package management is sad.
How to fix it? the Python core and PyPI team make and sanction a canonical CLI tool for package and environment management (like
rustup for Rust). I vote add it in to
pip. You can keep using your old random third-party tooling, but this is the standard way going forward.
Instead, the solution seems to be to make an abstraction over packaging programs (PEP 517 and PEP 621, PEP 631, PEP 660) so that they can all live together. This is a mistake because the space is still fragmented and confusing, and third-party tools will not obey the spec if it impedes functionality.
Also, all package writers need to be able to keep their version number in one place. So the official docs recommend 7 convoluted ways of doing that. The first idea in the list is text-scraping a python file.
1.3. How I solve this
I solve this problem by using PyEnv,
pip and nothing else. For packaging projects I still haven't found a good solution, I am having a go at using
hatch and report back.
pyenv. On mac it's easy but I remember it being a faff to get it set up on arch.
pyenv install 3.10.6. There are some caveats here, make sure you have some dlls:
zlib, possibly sqlite, openssl? If you don't have these, then at some point in the future you will get a strange error about a missing dll.
Activate this with
pyenv local 3.10.6.
python --versionin this directory should give
python -m venv .envcreates a new python 'virtual environment', this means you can install packages without breaking everything.
To activate the shell to use the environment do
If you open this directory in vscode with the python extension, it should ask you to use
.envas the environment and then it will automatically enter the environment in all the shells etc.
If you are in a python project ([todo] more on how to do this), you can 'install' the current project with
pip install -e .. If you get
ERROR: File "setup.py" or "setup.cfg" not found., you should upgrade pip with
pip install --upgrade pip.
to install extra deps do
pip install -r dev_requirements.txt. This is currently a grab-bag of everything you might possibly need and should be tidied up and incorporated in to pyproject.toml before release.
2. Python's module resolution is annoying
Meanwhile, importing the modules that you want to use breaks if you go slightly off what is deemed pythonic.
If your project has the form
You get told off for trying to import something from
There is an SO post titled Relative Imports for the Billionth Time that portrays how infuriating this is. You are only allowed to relative import things within 'packages'. Why? Most people coming to Python don't know what a package is!
Python is supposed to be this easy and dynamic programming language, but you can't import another file without doing a song-and-dance with
__init__.py and managing the python path? Maybe it's a security thing.
I need to get to the bottom of this.
3. Other gripes
ReStructuredText is truly awful. Lots of bizaare language choices (hyperlinks have to end in an underscore?). Check out the super-upvoted comment on this SO question.
datetimeseems to have a different interpretation of ISO dates to everyone else∶ SO question. Don't call it
datetime.fromisoformat()if it can't parse ISO standard datetimes!
A fun list of other gripes: WTF Python. My favourite is
a = 257; b = 257; a is b # Falsebut it is
3.1. Closures are weird
Can you predict what this code should do?
'b,b' because in python closures are messed up.
The same mutable variable
n is passed to both functions.
I realise that this is a big ask, but I really wish python had sandboxing of untrusted code. This would be a killer feature.
standard pipeline operator