So I’m no expert, but I have been a hobbyist C and Rust dev for a while now, and I’ve installed tons of programs from GitHub and whatnot that required manual compilation or other hoops to jump through, but I am constantly befuddled installing python apps. They seem to always need a very specific (often outdated) version of python, require a bunch of venv nonsense, googling gives tons of outdated info that no longer works, and generally seem incredibly not portable. As someone who doesn’t work in python, it seems more obtuse than any other language’s ecosystem. Why is it like this?
I’m not sure this can be really fixed with Python 3, maybe we just have to hope for Python 4
It’s fixed, and the python version had nothing to do with it. Just use hatch
Ah yes, the 15th standard we’ve been waiting for!
It’s not a standard, it’s built on standards.
You can also use Poetry (which recently grew standard metadata support) or plain
uv venvif you want to do things manually but fast.Just use this one… or any of this 4 others.
This is the issue for us, python outsiders. Each time we try we get a different answer with new tools. We are outside of the comtunity, we don’t know the trend, old and new, pro and cons.
Your first recommandation is hatch… first time I’ve heard of it. Uv seems trendy in this thread, but before that it was unknown to me too.
As I understands it, it should be pip’s job. When it detect I’m in a project it install packages in it and python use them. It can use any tool under the hood, but the default package manager shoud be able to do it on its own.
Uv and pip do the same thing, uv is just faster.
Hatch has the same role as Poetry or tox: managing environments for you.
Applications should be packaged properly, in a self contained installer for exactly this demographic. It’s not Python’s fault that this isn’t common practice.
Python is the only programming language that has forced me to question what the difference is between an egg and a wheel.
No, it’s not just you, Python’s tooling is a mess. It’s not necessarily anyone’s fault but there are a ton of options and a lot of very similarly naked things that accomplish different (but sometimes similar) tasks. As someone who considers themselves between beginner and intermediate proficiency in Python this is my biggest hurdle right now.
Python’s tooling is a mess.
Not only that. It’s a historic mess. Over the years, growing a better and better toolset left a lot of projects in a very messy state. So many answers on Stack Overflow that mention
easy_install- I still don’t know what it is, but I guess it was some kind of protouv.Every time I’m doing anything with Python I ask myself if Java’s tooling is this complicated or I’m just used to it by now. I think a big part of the weirdness is that a lot of Python tooling is tied to the Python installation whereas in Java things like Maven and Gradle are separate. In addition, I think dependencies you install get tied to that Python installation, while in Java they just are in a cache for Maven/Gradle. And in the horrible scenario where you need to use different versions of Maven/Gradle (one place I was at specifically needed Maven 3.0.3 for one project and a different for a different, don’t ask, it’s dumb and their own fault for setting it up that way) at least they still have one common cache for everything.
I guess it also helps that with Java you (often) don’t need platform specific jar files. But Python is often used as an easy and dynamic scripting interface over more performant, native code. So you don’t really run into things like “this artifact doesn’t have a 64 bit arm version for python 2” often with Java. But that’s not a fault of Python’s tooling, it’s just the reality of how it’s used.
No it’s not. E.g. nobody who starts a new project uses setup.py anymore
Are you sure? I’m not very active in that ecosystem, but if that was prevalent in the past, surely there’s still tutorials and stuff out there that people would follow and create such projects even today?
More than that, it seems to me that the official python docs for packaging [still] talks about setup.py. Why would people not use that?
Sure, there was some hyperbole. Some people need some specific setuptools plugin or something. Almost nobody.
when the official docs are telling you to use it, then it’s used. You can have no expectation of people to think the tooling isn’t shit when it’s literally the official recommendation.
OP seems to be trying to install older projects, rather than creating a new project.
Yep, they are not portable, every app should come bundled with its own interpreter. As to why, I think historically it didn’t target production grade application development.
I’ve started using poetry and the experience has improved.
The reason you do stuff in a venv is to isolate that environment from other python projects on your system, so one Python project doesn’t break another. I use Docker for similar reasons for a lot of non-Python projects.
A lot of Python projects involve specific versions of libraries, because things break. I’ve had similar issues with non-Python projects. I’m not sure I’d say Python is particularly worse about it.
There are tools in place that can make the sharing of Python projects incredibly easy and portable and consistent, but I only ever see the best maintained projects using them unfortunately.
Python is hacky, because it hacks. There’s a bunch of ways you can do anything. You can run it on numerous platforms, or even on web assembly. It’s not maintained centrally. Each “app” you find is just somebodies hack project they’re sharing with you for fun.
Python is the new Perl
After using python, I’m of the opinion that perl was much cleaner.
Nothing comes close to Perl’s abuse of global variables. Oh you called this function? Take a guess which global variables it will use.
On that note, I’m hesitant between writing my scripts in perl or python right now. Bash prevent sharing with Windows peoples… I just want to provide easy wrappers tools that are usually aroud 10 lines of shell, but testers ain’t on linux so they cannot use them.
I don’t know perl, but each time I interract with pyton’s projects I have a different venv/poetry/… to setup. Forget adout it the next time and nothing is kept easy to reuse.
Perl isn’t really any better. There aren’t easy tools that do the same thing as venv. They exist, but they are not easy. Plus there are a much larger amount of cpan modules that have c in them than python.
Docker might be solution here.
But from my experience most python scripts are absolute junk. The barrier for entry is low so there’s a massive disparity in quality.
Python is truly a mess when Docker is considered a solution.
Python never had much of a central design team. People mostly just scratched their own itch, so you get lots of different tools that do only a small part each, and aren’t necessarily compatible.
You re not stupid, python’s packaging & versionning is PITA. as long as you write it for yourself, you re good. As soon as you want to share it, you have a problem
as long as you write it for yourself, you re good. As soon as you want to share it, you have a problem
A perfect summary of the history of computer code!
It… depends. There is some great tooling for Python – this was less true only a few years ago, mind you – but the landscape is very much in flux, and usage of the modern stuff is not yet widespread. And a lot of the legacy stuff has a whole host of pitfalls.
Things are broadly progressing in the right direction, and I’d say I’m cautiously optimistic, although if you have to deal with anything related to conda then for the time being: good luck, and sorry.
Tried to install Automatic1111 for Stable Diffusion in an Arch distrobox, and despite editing the .sh file to point to the older tarballed Python version as advised on Github, it still tells me it uses the most up to date one that’s installed system wide and thus can’t install pytorch. And that’s pretty much where my personal knowledge ends, and apparently that of those (i.e. that one person) on Github. ¯\_(ツ)_/¯
Always funny when people urge you to ask for help but no one ends up actually helping.
Lol this is exactly why I made this post. I ended up using ComfyUI instead which has other, different python issues, but I got it working (kinda, no GPU but it’s fine it works)
venv nonsense
I mean, the fact that it isn’t more end-user invisible to me is annoying, and I wish that it could also include a version of Python, but I think that venv is pretty reasonable. It handles non-systemwide library versioning in what I’d call a reasonably straightforward way. Once you know how to do it, works the same way for each Python program.
Honestly, if there were just a frontend on venv that set up any missing environment and activated the venv, I’d be fine with it.
And I don’t do much Python development, so this isn’t from a “Python awesome” standpoint.
pyenv and uv let you install and switch between multiple Python versions.
As for uv, those come from the Python build standalone project, if I remember correctly, pyenv also installs from there, but don’t quote me on that.
I agree. Python is my language of choice 80% or so of the time.
But my god, it does packaging badly! Especially if it’s dependent on linking to compiled code!
Why it is like that, I couldn’t tell. The language is older than git, so that might be part of it.
However, you’re installing python libraries from github? I very very rarely have to do that. In what context do you have to do that regularly?
