I should have mentioned that I'm seeing this from the point of view of a distro maintainer involved in Fedora.
The thing is that in general you don't want to have many version of libraries available in the distro in parallel as each has to be separately maintained, security patches applied, build issues fixed, etc., eating valuable maintainer time. Also in some cases libraries of different version can't coexist on a system cleanly.
Then imagine every piece of software just pins versions of their dependencies to a specific random version that happened to work at the time. To satisfy those arbitrary dependencies the distro would have to maintain all these versions at the same time, which is simply impossible resource vise, not to mention incredibly wasteful (both in maintainer time & system resources).
As for stuff breaking if you don't pin dependency version - well, distros have mechanisms to handle that. For example for Fedora, there is a stable release very 6 months & stable releases are not expected to get major changes in libraries, just bug fixes and smaller enhancements.
And at the same time there is a rolling version of Fedora called Rawhide, where all the latest package versions land and where integration issues are addressed. So any breakage would happen on Rawhide and be addressed by maintainers (of the library/software affected or both) long before a new stable release is cut from Rawhide and users will actually use it.
For an example I'm maintaining the PyOtherSide Qt 5/Python bindings on Fedora. A while ago the build failed due to Python being updated to 3.9. I reported the issue upstream, which quickly fixed it and I've built an updated version in Rawhide. All this long before a stable Fedora version will get Python 3.9, but I can be sure that when this happens, all will work fine.
Yes - although from the developer perspective, they want to work cross-distro without having to test a big compatibility matrix, and generally use the language's package manager (which hopefully behaves the same across all platforms) rather than the distro's one.
Especially with Python, and especially with the 2/3 split, people got used to assuming that the distro version was something broken to work around (e.g. Redhat, OSX), and that all "real" work happened in one's local language-specific package cache or venv.
I'm increasingly of the opinion that it's a mistake for distros to ship Python or Ruby packages in their distro-specific package format, but I can also see that's going to be a holy war.
the compatibility matrix is exactly what I'm wondering about too. It's far easier to test against a single version of a dependency and deploy against it too.
I can't help but wonder if there's a better way that could scale to open source. I personally like monorepo-based development a lot - where you have one version of every library for the whole repo, and a total ordering on changesets (and a strong test suite to catch regressions). But organizing open source into a monorepo or even a virtual-monorepo seems tricky. Might be the sort of practice that can work well for companies but not so much when decision power is more distributed.
As developer I use language specific packager (bundler/gems) and I know it well.
As Linux user who sometimes build packages (tinker around) I need all kind of dependencies - python, ruby, perl, haskell. It is much safer and faster to use distro packages.
Breaking changes on major version is awful for any consumer. Python 2/3 story is a shame. These can't be arguments against distro packages.
So if I understand you correctly, you're asking developers to change how they work in order to make package maintainers' lives easier while providing no benefit to either developer or end user. Is that correct?
The benefit for developer is maintainers report compatibility issues before the developer hits them and in general take over a lot of the user support as well as the menial but necessary packaging tasks. For the user the benefit is they can actzally deploy the thing in a real production system (that is not the developrs laptop) with potentially clashing libraries and be reasonably sure it continues to be secure in the future by using up to date maintainer libraries, not some unstable release no longer getting any security fixes the developper pinned it to 5 years ago and never bothered to change.
On the other hand, inserting a volunteer middleman between the developer and the user cause extra unnecessary friction, delays, and has even been the cause of bugs [0], not to mention sometimes they explicitly ignore the wishes of the original developer.
[0] Remember when Debian generated predictable random numbers because a maintainer wanted valgrind to shut up?
On the other hand, stuff like the NPM leftpad issue can't easily happen as the package maintainer "middle man" usually inspects any upstream updates for general sanity. And even if it got in to Fedora for example, it would most likely break only the integration rolling release called Rawhide, where it would be quickly discovered and fixed, never reaching the overwhelming majority of users that are on Fedora stable releases.
Now in comparison people using NPM just blindly pulled random stuff directly from upstream without anyone doing any sanity checking at all - no wonder one package vanishing made the whole thing fall over, often directly in production.
Are we talking about source-code packaging for developers or binary packaging for users? Because the left-pad scenario really doesn't apply in the latter case.
The thing is that in general you don't want to have many version of libraries available in the distro in parallel as each has to be separately maintained, security patches applied, build issues fixed, etc., eating valuable maintainer time. Also in some cases libraries of different version can't coexist on a system cleanly.
Then imagine every piece of software just pins versions of their dependencies to a specific random version that happened to work at the time. To satisfy those arbitrary dependencies the distro would have to maintain all these versions at the same time, which is simply impossible resource vise, not to mention incredibly wasteful (both in maintainer time & system resources).
As for stuff breaking if you don't pin dependency version - well, distros have mechanisms to handle that. For example for Fedora, there is a stable release very 6 months & stable releases are not expected to get major changes in libraries, just bug fixes and smaller enhancements.
And at the same time there is a rolling version of Fedora called Rawhide, where all the latest package versions land and where integration issues are addressed. So any breakage would happen on Rawhide and be addressed by maintainers (of the library/software affected or both) long before a new stable release is cut from Rawhide and users will actually use it.
For an example I'm maintaining the PyOtherSide Qt 5/Python bindings on Fedora. A while ago the build failed due to Python being updated to 3.9. I reported the issue upstream, which quickly fixed it and I've built an updated version in Rawhide. All this long before a stable Fedora version will get Python 3.9, but I can be sure that when this happens, all will work fine.