Too many dependencies in Open Source Projects

While updating a smallish node project I'd been developing for a client, I noticed the massive number of packages, 19,000, packages audited. I only have 6 or so in my project. Where'd they all come from?

Turns out that's the number of packages you get when you trace back all of the dependency trees of all the packages that all the packages I installed run on. It doesn't seem like so many when you think, or right, these 6 packages are each their own little companies with their own stack and decisions. So you could have dozens of multiple versions of the same package or many functions that do the same work in separate packages. Lots of layers. Lots of extra code that over time, someone will have to maintain.

But wow. It does seem like I didn't need to get all these open source projects involved in my little ice cream store and blog. Right? I mean, how dependent am I on these? Well that depends on how often you'd like to be able to make changes to your app. Overtime, package maintainers quit, another library does it better and so they're often abandoned, at least on NPM. Big the real problem isn't that I'm using packages to make a user facing app, it's that the folks who write all of my packages are themselves relying too much on packages, other libraries and frameworks. So we have to ask ourselves. Do we need all of this code? And what are these projects that simply glue together other projects. These platforms everyone seems to need now to create a web page and make money on the internet.

So as businesses we kind of choose not to maintain the code. We just rewrite it as we build the same stuff over and over again. We move from some PHP shopping cart with plugins A to Cloud Checkout Service B and we spend a ton of money and fore everyone to learn something new. But many of the features these frameworks offer turn out to only be possible if you literally hack around the framework and just to the right thing in terms of the actual job of engineering. So we're stuck kind of maintaining another aging rock star, until a new, smaller, lighter framework comes along that maybe even just solves the problems of the last framework, until everyone's too confused about how web sites work and what level of abstraction to talk about.

It's an exhausting way to run a business and an exhausting way to work as a developer. Especially since under all of this these new frameworks still just need to interact with the actual web protocols, with the actual browser APIs.

On the other hand, you can start from the beginning of every project. Programmers will often reuse code from previous projects on new ones. Libraries sometimes do help put the task at hand in focus, leaving underlying details in the ether of open source, trust through sunshine. And companies rely on these libraries every day and turn a fine profit and grow and improve their software incrementally. All of that can be true at the same time as my point. It's not a religion, it's a value I'd like to argue must be added to the values our profession claims to revere.

Less dependencies. Library creators should just vendor whatever they need, cut out the kruft and have 0 formal dependencies. As far as a package ecosystem, NPM has failed. They make vulnerabilities and even computer viruses more potent and possible. Really endlessly possible as long as we have so much code we don't actually need running our web sites. Wordpress shouldn't need MySQL to run a blog. But it does and always will because Wordpress is a Platform. Other common libraries like Rail or jQuery or even using AWS are just the same homogeneity exploit in the major platform.

Less is less

The fewer lines of code, including third-party packages, the less we have to audit and confirm security around. There are a lot of different solutions to the same set of problems in software. We must choose the solution that takes security into it's design, not it's marketing pitch. This is not a prescription for regression to 1900's web pages. We must use additional code to achieve some modern designs, and yet much of what we're doing

Impossible to archive

We're just writing configuration at that point and the name of the game is how easy is it to solve the problem I'm having? Lots of stack overflow answers and a robust community of helpers does a lot to illuminate and smooth over the ill-conceived aspects of the framework's API.

https://www.btao.org/2020/10/02/npm-trust.html https://snyk.io/blog/a-post-mortem-of-the-malicious-event-stream-backdoor/ https://blog.npmjs.org/post/185397814280/plot-to-steal-cryptocurrency-foiled-by-the-npm https://blog.reversinglabs.com/blog/mining-for-malicious-ruby-gems