Circa 2017 I was in charge of figuring out build problems at a startup that was using Python for machine learning. By the time I left I had a complete checklist of what the problems were but I was out of time.
I had notes for something that would have been equivalent to "uv" in many ways, in particular it would have a correct resolving algorithm. I had spike prototyped many interesting things, such as a system which could use http range request to download just the metadata for a wheel so it could rapidly scan a bunch of different versioned wheels to work out what would be compatible with what.
One problem I was worried about was, written in Python, it would be dependent on having a working Python environment to stand on -- my experience was that Python devs, particularly data scientists, could screw up their Pythons worse than I could imagine. uv's answer to be not written in Python was genius, not least because it contributes to speed (caching matters a lot, a pure-Python system could dot that) but because a static linked binary completely eliminates the dangers of corrupted environments. (poetry gets fubared if you use it long enough)
I didn't got forward with it because when I asked around I found Pythoners just didn't give a damn that the pip resolver didn't really work -- it works for them, if their project is simple. If your project is a little more complex it still works sometimes, works enough that you can whack it on the side the few times or start fresh occasionally. Only if you had the super-complex projects we were working on was it war all the time. I didn't think I could sell anyone else on using it so I gave up.
People absolutely can fork uv, and if Astral ever does anything shady then I would absolutely recommend that people do so.
I'm designing PAPER to be small and efficient (to the extent Python allows), with an explicit API — largely with the intent of making it easy for others to extend, wrap, fork, integrate, interoperate with etc. it.
> I had notes for something that would have been equivalent to "uv" in many ways, in particular it would have a correct resolving algorithm.
Pip gained a correct (relative to the standards and metadata availability of the time) resolving algorithm in 2020 or soon thereafter. uv adds performance largely via heuristics and shortcuts (i.e., given multiple options for trying to fix a problem, it tries to try them in an order that's more likely to find a solution earlier). This stuff seems like the hard part and I'm probably doing to just drop in pip's logic (which has already been isolated as `resolvelib`, though it requires some interfacing) at least to start.
> uv's answer to be not written in Python was genius, not least because it contributes to speed (caching matters a lot, a pure-Python system could dot that) but because a static linked binary completely eliminates the dangers of corrupted environments. (poetry gets fubared if you use it long enough)
All you needed to do was make your tool have its own isolated environment (in an out-of-the-way place), have it function cross-environment (pip now accomplishes this by a "re-run the code" hack; PAPER has logic to just inspect the target environment explicitly) and refuse to install anything in the tool's environment that isn't vetted as necessary for a tool plugin etc. Yes, I'm sure data scientists can screw up environments something fierce. That's why you let them have their own playground. My plan is to also create a default "sandbox" environment (like what pip originally tried to do with `--user`, except actually a virtual environment and not given special path-hack treatment) for "library" installs when no target is explicitly specified, and do all "application" installs in new environments by default.
I don't know what issues you encountered with Poetry. If they're still a thing in 2.x (apparently they're at 2.1.3 now), I'm sure they'd appreciate a reproducer.
> I didn't got forward with it because when I asked around I found Pythoners just didn't give a damn that the pip resolver didn't really work -- it works for them, if their project is simple.
Pip does work for a lot of people (although it's slow even when it does literally nothing), which is a big part of why it didn't face a lot of pressure to improve. Circa 2017, the complaints were more nebulous, and probably a bigger share of them could be attributed to poor ecosystem standards (in particular, metadata availability) and to Setuptools. Things in the pyproject.toml era are the same in many ways, but very different in others.
Circa 2017 I was in charge of figuring out build problems at a startup that was using Python for machine learning. By the time I left I had a complete checklist of what the problems were but I was out of time.
I had notes for something that would have been equivalent to "uv" in many ways, in particular it would have a correct resolving algorithm. I had spike prototyped many interesting things, such as a system which could use http range request to download just the metadata for a wheel so it could rapidly scan a bunch of different versioned wheels to work out what would be compatible with what.
One problem I was worried about was, written in Python, it would be dependent on having a working Python environment to stand on -- my experience was that Python devs, particularly data scientists, could screw up their Pythons worse than I could imagine. uv's answer to be not written in Python was genius, not least because it contributes to speed (caching matters a lot, a pure-Python system could dot that) but because a static linked binary completely eliminates the dangers of corrupted environments. (poetry gets fubared if you use it long enough)
I didn't got forward with it because when I asked around I found Pythoners just didn't give a damn that the pip resolver didn't really work -- it works for them, if their project is simple. If your project is a little more complex it still works sometimes, works enough that you can whack it on the side the few times or start fresh occasionally. Only if you had the super-complex projects we were working on was it war all the time. I didn't think I could sell anyone else on using it so I gave up.