Lots of issues here Randy! You obviously know this stuff well :-)
On convention over configuration: Yes, J2EE went configuration-crazy. And they used XML, which was a terrible choice. So the result was a mess, and convention-over-configuration (COC) was a reaction to that, and it is just as bad. COC is tribal knowledge: you have to “know the framework” to even get started. I work across many frameworks, so I always forget the configuration rules. COC is a nightmare for people who do a lot of prototyping (like me), as well as junior devs learning, or anyone who maintains lots of different things that use different frameworks. COC is great for someone who spends years using the same set of tools, so the rules become second nature.
I think a better approach would have been to replace the J2EE XML hell with an OO configuration model, and a typesafe language for specifying the configuration — i.e., a deployment language (which is what K8 is).
K8 actually rests on an object model, but for some reason that escapes me, they make us use YAML or JSON — typeless hell. I friggin’ hate those. I don’t find out if I have an error until I do a deployment. A compiler should be able to tell me instantly — it could if K8 used a typesafe language instead of JSON/YAML.
AWS created their own hell with Cloud Formation Templates (CFTs). Now they are trying to dig their way out of that with CDK, which is an OO library for deployments. Ten lines of CDK do what 1000 lines of CFT do, and the CDK compiles and you can tell immediately if you are missing a parameter or have provided a String when instead you need an object ID. And CDK is composable — your CDK can instantiate other CDK objects. That is what K8 should have done. F*ck YAML.
Python is fast to code in, but it is a nightmare when you have many teams maintaining it. Having used third party Python machine learning libraries, I know from experience that it is a nightmare trying to figure out what kind of object a method returns — is it an array or an array of arrays? — you can’t tell, especially if the method calls anther method which calls another method. In a typesafe language I could tell at a glance, but in Python it as sometimes taken me hours to just figure out what something returns. What a waste of my time. So the programmer saved 2 seconds, but cost me 2 hours.
GraalVM is fantastic. I have not used it yet but an anxious to. And then there is https://quarkus.io/
Horizontal & elastic scaling are crucial; but for imagine that for Python you need 10,000 VMs, but for C you only need 100. The cost is very different.
Go is an alternative but it is a horrible language, from a maintainability perspective. It is a fun language, but try reverse engineering someone else’s code. I have: I had to reverse engineer the Docker Registry version 2 codebase. It was a horrible experience. It is mostly because of Go’s “duck typing” — you can’t figure out what “type” something is.
Rust is appealing. I plan to learn it.
I have defined my own language called flow, and am almost done with a compiler front end for it, but I just don’t get time to work on it. It is a data flow language, which I think is a far better paradigm for so many things.
People say that C is “old”, but a few years ago I wrote a utility program in Ruby. I had some trouble with it and finally I said “screw this” and I rewrote it in C. I had not written C in 20 years. The code was 300 lines. It took me 40 minutes to write it. I ran it, and it worked the first time. That was the second time in my career that code worked the first time. But my point it that C is actually very high productivity. To this day I have not matched my all-time highest productivity: in the late 80s on a Saturday I wrote 1000 lines of tested code in one eight hour day, on my Sun workstation with a large grayscale monitor.
Sometimes newer things are not better — they are just different. I like to scrutinize to see if something is actually better, and in which ways, and in which ways is it merely different. Horizontal scaling is new, and better. Commodity virtualization is better. Containers are better. Graal is better (I think). Python is not better — it is a toy language, IMO. (Read this) On-demand & coded provisioning/deployment (cloud) is better. DevOps is better. A lot of Agile is not better — it is worse, although some Agile ideas are better. (That’s why we created Agile 2) Nearly all of the 12-factors principles are better — that’s a great set of guidelines. Algorithms are where there has been staggering progress in the past few decades. REST is terrible — that’s why Google created gRPC. When REST was created, people like me were shaking our heads. REST was created because people were suffering from WSDL and SOAP, and we shook our heads at that too; but REST is an amateur’s response to WSDL. It is not the right approach: it mixed transport layer and application content. gRPC (protocol buffers) is a return to an object-oriented protocol. All the stuff about “representational state transfer” is bullshit. One just wants to push a message (type-safe message that is extensible) from one place to another, or make a remote request, depending what you are doing.
