Kinda a big announcement – Joel on Software

I took a few stupid years trying to be the CEO of a growing company during which I didn’t have time to code, and when I came back to web programming, after a break of about 10 years, I found Node, React, and other goodies, which are, don’t get me wrong, amazing? Really really great? But I also found that it took approximately the same amount of work to make a CRUD web app as it always has, and that there were some things (like handing a file upload, or centering) that were, shockingly, still just as randomly difficult as they were in VBScript twenty years ago.

Source: Kinda a big announcement – Joel on Software

It’s hard for me to express just how deeply wrong I find this to be, but I suppose that’s because I take it as an almost personal insult. Here’s a smart, driven guy who probably just became a (near) billionaire with the sale of a site devoted to programming Q&A, and yet, in my opinion, he’s completely out of touch with modern web development. I really resent this gaping hole in the collective knowledge of programmers on this planet.

I’ve been using Rails for 15 years now. I’ve used it to make dozens of applications. It is perfectly suited for making CRUD web apps. It was designed from the ground up to do so, and avoid the grunt work of other programming stacks, specifically Java. Unfortunately, Spolsky is not alone of his ignorance about it. I see lots of programmers singing the praises of Javascript, who dismiss Rails, usually because of its convention-over-configuration approach, but nothing can compare to the productivity of using Rails to write a CRUD web application. Nothing. It’s not even close. He’s absolutely right that Node and React offer no advantage over any other legacy option like Java or .NET. I went down the whole Java/Spring/Angular hole for one ill-fated project, and it’s a freakish, byzantine nightmare. The difference between the two stacks is so stark that I have to assume that people who make these kind of comments are completely oblivious to the fact that Rails exists at all.

Take file uploads for instance. Rails has had easily-configured and power capability from several hugely popular gems since (at least) the 3.x days. The stack has had its own implementation since 5.x. Either way, just configure a couple of lines in an initializer, pick a provider, enter your bucket name and API key, and then it’s literally just a few lines of code to add a file attachment to your model.

Spolsky continues to rant:

The biggest problem is that developers of programming tools love to add things and hate to take things away. So things get harder and harder and more and more complex because there are more and more ways to do the same thing, each has pros and cons, and you are likely to spend as much time just figuring out which “rich text editor” to use as you are to implement it.

This is the continuing, enduring beauty of Rails. They continue to add things to the stack, like file uploads, but they do so in a way that makes them optional. If you want them, it’s, like, 3 lines of configuration, and you’re rolling. A rich text editor, as it turns out, is another perfect example. There has been a popular gem to provide the WordPress editor for a long time now, but Rails started shipping a native rich text editor in 6.x, if you want it. I’m using it in a significant way in a production application right now. I added it well after the site was launched, but it was easy, and it’s terrific.

Today we’re pleased to announce that Stack Overflow is joining Prosus. Prosus is an investment and holding company, which means that the most important part of this announcement is that Stack Overflow will continue to operate independently, with the exact same team in place that has been operating it, according to the exact same plan and the exact same business practices. Don’t expect to see major changes or awkward “synergies”. The business of Stack Overflow will continue to focus on Reach and Relevance, and Stack Overflow for Teams. The entire company is staying in place: we just have different owners now.

This is where I get worried. An investment and holding company paying $1.8B to buy a site like Stack Overflow is going to want to recoup its investment, and make more money in the future. In the old days, they used to say that investments needed to start making money in 7 years. I’m not clear that this old rule of thumb still applies, and SO is a private company, so we can’t see a balance sheet, but does anyone think that “SO for Teams” is making $250M a year? Big M&A announcements like this always say the same things about keeping the product the same. Let’s revisit this in a year, and see where we really stand.

TECH | Stop using JPA/Hibernate · Blog de Laurent Stemmer

Here an example of a JPA entity (using Lombok for “simplicity”): <sarcasm quotes mine>

@Entity
@Table("offer")
@EqualsAndHashcode
@NoArgsConstructor // for Hibernate
@Setter // for Hibernate
@Getter
public class BankAccount {
    @Id
    @Column("id")
    private String id;
    @Column("opened")
    private boolean opened;
    @OneToMany(fetch = LAZY) // ...simplified
    private Set ownerIds;
}

Source: TECH | Stop using JPA/Hibernate · Blog de Laurent Stemmer

Through a very long series of unfortunate circumstances, I was backed into using Java/Spring/Hibernate/Angular in a doomed project. This page had me nodding my head in agreement, and this code reminded me of the Lombock portion, which was its own special nightmare. I just went looking for what I had written for that project, and it would appear that I’ve totally deleted it. I normally keep everything, so I can go back and refresh my mind when I recall some particular technique I’ve used in the past, so this should tell you something about the brain damage using this stack will incur.

I’m going to digress to setup a point. I used gvim for many years, with a complicated setup, using NerdTree and several other plugins, to give me a UI with my project’s directory on the left side, and tabs of open files on the right. At some point, I got tired of fiddling with the configuration, and finally started using someone’s massive-but-well-integrated ~/.vim configuration from a GitHub repo. Finally, I realized that I was spending all this time and effort on making gvim work just like ST did out of the box, and I could just start with that. So I did. While I’ve flirted with other editors (notably, Visual Studio Code, and the excellent IntelliJ, while working on Java), I’ve basically stuck with it for about 7 years now.

Here’s the parallel. The thing that fans of the Java ecosystem can’t admit to themselves is that this whole stack: Java, Spring, Hibernate, Lombock, Javascript, AngularJS, etc., et. al., ad naseum… is all just a terrible pile of Jenga blocks which putatively exist to give you a functional environment like… wait for it… Ruby on Rails! Lock, stock, and out of the box. It seems to me that the motivation of people who still like to use gvim when Sublime Text and Visual Studio Code exist is the same sort of motivation of people who like to use a Java stack over something like Rails. Maybe they’ve done it so long, they can bang out the boilerplate with their eyes closed. Maybe they like the way you have to do everything explicitly. Maybe it makes them feel like a hacker.

All of the code above reduces to this in Rails:

class BankAccount < ApplicationRecord
    self.table_name = 'offer'
    belongs_to :owner
end

In an absolutely brilliant display of one of the biggest problems with using this Java stack, I went to remind myself what the @Table("offer") directive does. I think it might specify the actual SQL database table name storing the instances of this object, but I literally can’t find any references to this pattern in the Lombok documentation. It is only through inferring it from a StackOverflow question that I am reasonably confident that this is, in fact, what it’s doing. And if it weren’t for Spring and Hibernate and Lombok, there’d be about a hundred more lines of boilerplate code in that single class file.

The top comment thread on the HN discussion about this blog post points out just how bad of an ORM Hibernate actually is. With 15 years of experience with Rails under my belt, I can assure you that almost none of those issues apply to ActiveRecord. Of course, I’ve seen people complain about AR, but I think their arguments are always exaggerated, and probably come from a place of general discontent with having to use Rails at all. People like to complain that Ruby is “slow” because it is interpreted, but it’s precisely that on-the-fly reflection that allows ActiveRecord to be so good at being an easily-programmed and powerful ORM. It’s trading machine time for ease of development and readability, and I have yet to see a situation where that was a bad tradeoff.

While I’m on the subject of ORM’s, I find EntityFramework just as bad as Hibernate. I suppose it’s just the nature of an ORM in the context of a compiled language. After giving it a real college try, I gave up on it. I wrote a serious application in Visual Basic and C# which accessed the database through a library of functions wrapping raw SQL, and called them from the WinForms side, and it worked out very well. I’m glad I didn’t try to force EF to work.

So, sure, rag on Rails. Call it slow. And, yes, compiled Java will always be technically faster than interpreted Ruby, but all the Java web sites I have to interact with are noticeably laggy and sluggish, compared to my apps, so there’s something to be said about implementation. And, while a whole team of Java devs are still writing class files, in both Java and Javascript, for their object model, I’m done with my app, and moving on to the next one.

So, yes, by all means, please stop using JPA/Hibernate, but, I would go one step further, and advise people to just stop using Java for web apps entirely. That horse got passed 15 years ago. Even if you don’t like Rails, there are at least a few other stacks that would be far more productive than Java for web apps these days. Heck, I’d try to do Javascript on the frontend and backend before I’d try doing Java again. <shiver>

And that’s my “2 minutes of hate” for today.

What I wish I had known about single page applications – Stack Overflow Blog

I settled on JHipster, a development platform for building web applications using modern technology: Angular, React or Vue for the client side, and Spring plus Gradle or Maven for the server side. It’s been around for years, is very well documented, and has great community support.

Source: What I wish I had known about single page applications – Stack Overflow Blog

I think the author hit at least one nail squarely on the head: Team size is an important consideration for the tech stack. I would argue that a Java/Angular stack is probably only appropriate for large teams, which wouldn’t need jHipster anyway. I tried it once, and it took FORTY-FIVE minutes to bootstrap a site on my top-of-the-line Dell laptop. There are an astonishing number of moving parts buried inside of it.

I’ve been using Rails for 15 years or so now, and one can argue about its strengths and weaknesses compared to other webdev toolkits, but it works really, really well for one-person “teams” writing highly-focused internal tools. I’ve spent the past month writing a single-page app in VueJS inside of my current tool, and it’s been an interesting experiment. I may have more to say about it later.

Google And Oracle’s Decade-Long Copyright Battle Reaches Supreme Court : NPR

Source: Google And Oracle’s Decade-Long Copyright Battle Reaches Supreme Court : NPR

I don’t want Oracle to win on the basis of software copyrights, but I do want Google to lose, and get hit with an astronomical penalty. I also would love to see a general chilling effect on the use of Java and Oracle, which I think are terrible technical choices today. But everyone involved here is part of the problem of our country being a captured corporatocracy now, so I’m very conflicted. If there’s a way that they both lose, and the public wins, I’d be for that.

How 4 Chinese Hackers Allegedly Took Down Equifax | WIRED

While the operation had a certain degree of complexity, Equifax itself made their job much easier than it should have. It should have patched that initial Apache Struts vulnerability, for starters. And an FTC complaint from last summer also found that the company stored administrative credentials in an unsecured file in plaintext. It kept 145 million Social Security numbers and other consumer data in plaintext as well, rather than encrypting them. It failed to segment the databases, which would have limited the fallout. It lacked appropriate file integrity monitoring and used long-expired security certificates. The list goes on. Equifax didn’t just let the alleged Chinese hackers into the vault; it left the skeleton key for every safe deposit box in plain sight.

Source: How 4 Chinese Hackers Allegedly Took Down Equifax | WIRED

This whole incident deeply offends me. I don’t like that our capitalistic society has given these credit-reporting companies so much control over our lives. I don’t like that they seem to be completely unaccountable for being so integral to so much of our economy. I don’t like that they hold all the information you would need to ruin someone’s life by impersonating them online. I don’t like that they are not being prosecuted for being so flippant with personally-identifying data.

I don’t like the fact that a sovereign foreign power committed industrial espionage on a critical part of our economy. I don’t like that they already did basically the same thing to a government personnel database the year before. I don’t like that China’s government exists to begin with, given their treatment of their own people, Hong Kong, and the Uighers. I also don’t like that China has been committing wholesale intellectual property theft for many decades. I don’t like the fact that we all know it, and nothing seems to be getting done to stop it.

The initial vulnerability the attackers leveraged was a problem in Apache Struts, which implies that Equifax’s web application uses Java. Using Java for a web application in 2017 is like driving a Model T in 1950. Sure, it was a reliable means of transportation, and revolutionary when it was introduced, but it’s 20 years out of date. It requires an inordinate amount of maintenance, spare parts are more difficult to find. Mechanics can be lazy, because they know they have the owner of the car over a barrel, and they can charge a premium for service, and take their time. There are many better options available, which start quicker, go faster, have safety features built in, and are far more comfortable.

Not all applications require encrypted this, and sharded that, and intrusion detection systems, and real-time monitoring, and everything else, but if any application needed these sorts of treatments, it would have been this one. Also, if any application needed its owners to stay on top of CVE disclosure reports, and fix affected layers of their stack, as appropriate, it would have been this one.

In sort, there’s literally no good news here. Nothing will happen to China, its government, or the actual individuals named in the indictment. The punishment to Equifax is a slap on the wrist. Everyone jumped on the settlement, and now no one will get anything. Everything about this is wrong, and nothing good will come of it.

IMHO: The Mythical Fullstack Engineer – Stack Overflow Blog

It’s my experience that the above MVFE is pretty uncommon. The profile describes a person with skills requiring thousands of hours to master, but who doesn’t take part in the holistic decision making process. By nature, the value of a fullstack engineer stems from their ability to make competent unilateral decisions (decisions without asking anyone for permission). I’m sure there are people who mostly fit the MVFE, but I wager that they are few and far between. You could probably summarize my view about the MVFE as:

It’s very impractical to become a fullstack engineer without understanding the big picture.

In my mind, a fullstack engineer’s value is mostly derived from their ability to single-handedly design, architect, execute, and operate an entire end-to-end system. Assuming this is possible, it almost completely eliminates integration overhead.

Source: IMHO: The Mythical Fullstack Engineer – Stack Overflow Blog

There’s a lot of subjectivity in this article, but I think it covers the topic pretty well. I consider myself a full-stack engineer, and that self-identification hinges on both emphasized points above. First, I’ve spent the time to learn all the pieces. Other people don’t see all the late nights, banging away on my home lab, or my church’s setup, forcing things to work when they didn’t want to, integrating pieces all over the stack, setting up solutions to help people get things done, or just to scratch an inquisitive itch.

For instance, I’ve run my own web server on a public address. This will cause you realize how hairy the raw, unfiltered internet is, and will force you to learn about the basics of hardening a server and firewalling a network connection in a New York minute. I’ve run my own email server. That will teach you about spam, attachments, white and blacklists, abuse addresses, and lots of stuff about making your server look legit to other servers. I’ve run a Windows domain for my whole family’s computers, with roaming profiles and everything. (Side note: do NOT use roaming profiles.) I’ve run my own personal cloud. My own mobile sync server. Media servers. TV recorders. The list goes on.

In my professional career, I’ve gotten to work on some of the biggest, baddest tech ever made. I’ve setup a Sun E10000 from scratch. When I took the official Sun training on the kit, I fixed the lab’s setup when it broke. I’ve configured a 384-tape robotic backup system. I’ve commissioned $15M of EMC disk cabinetry. I’ve trained extensively on Oracle, and setup a North-American-spanning network of 20 instances. These technological implementations are fading, now, but the concepts haven’t. We just answer the same questions with different hardware and software these days.

When I encounter a new technology, at this point, it usually doesn’t take long to slot it into the larger context of computing services. For instance, I recently tried to use Elasticsearch for a project at work. While I eventually found an easier way to do what I needed to do, through several weeks of experimentation, I now know what that technology is about, what problems it solves, how it works, and what it takes to implement it. Now I have this tool in my toolbox, and it’s very possible that I will yet use it for a different project. I’m incredibly grateful that I have a job where I can occasionally do a little “R&D” like this, to learn something new, but it takes substantially less time to divert my attention like this, than other people might spend, because of the experience I already have.

Second, I understand the field I write software to support, because I studied it. I think the modern incarnation of the programmer, toiling away in the bowels of a big company — which is most developers, by simple numbers — is the total inverse of this ideal. I have a degree in mechanical engineering. I’m a good engineer, for the same reason I consider myself a “full-stack” guy. I see the big picture, and how everything underneath it contributes to making it look the way it does.

Even more than studying the math and physics, I was drawn to engineering, because that’s how my mind works. When I look at an engine, I notice the systems that are interoperating: the mechanical masses, the fluid flows, the thermals, the electrical connections, the air flow. I feel these things in my gut and see them in my mind’s eye. I understand how all of these subsystems work to produce power and torque, the difference between those two things, and when it’s appropriate to focus on one over the other. To me, it’s the same thing with an IT solution. I can picture the large subsystems working together to make up the final system in my head, and see the servers, the services, the networks, the databases, the networking, and the automation that will be needed to implement it.

In a lot of ways, the training in how to think about a problem in engineering school is perfectly suited to creating full-stack solutions. Start with restating the problem. Get to the heart of the business problem you’re trying to solve. Where’s the friction? State the givens. What do we know already? What pieces of data do we have? How do we get that into the system? Finally, specify what you’re solving for. What are we missing? How are we going to transform what we have into what we want? How will the people who will use the system need the program to work, and the data to be shown?

Most people working in software in my industry have been trained in how to write some code, and that’s about the end of it. They might understand how to write a loop in Java, but they don’t understand how to setup a Java application server, or a load balancer, or a firewall, to say nothing about the database. They also don’t understand how our products work, how they’re designed, or what the engineers working on them need to help them in that endeavor. I find myself in the rather rare position of understanding both halves of this equation. In my 25-year career, I’ve met only a handful of people who can straddle the fence between the physical, engineering problem domain, and the IT implementation like this. In the manufacturing world, we are indeed few and far between.

There’s one other thing I want to talk about, and that this article’s presumption that Javascript is the piece for the front-end, in the jigsaw puzzle that is a full-stack web application these days. I still like Rails’ templates, and, of course, Microsoft is pushing Razor. Javascript enhances both of these things. However, the article hints at how a lot of people are doing the entire front-end in Javascript now, and I find that disappointing. In an aborted effort, I tried writing an application in Java with an Angular2 front end. The amount of duplication astounded me. When you combine this duplication with the fact that Java and typed Javascript are two of the most verbose languages to work with, well, you get a mess.

Using an API back-end and a pure Javascript front-end is, perhaps, the single greatest argument against full-stack development you can make. Given the sheer amount of work involved in separating the front-end, completely, from the back-end, it almost requires two different people or teams. If you sat down, and wrote out the most terrible theoretical idea you could come up with for software development, it would probably look like “write a single application, broken right down the middle, in 2 different languages.” Unfortunately, that’s the nature of web development right now. I lament that this is where we’re at in our technological evolution, but until network bandwidth takes another leap forward, this is what we’re stuck with.

Web Development Framework Trends

Back in April of 2014, I was vacillating between using Ruby on Rails, and Entity Framework on ASP.NET, for a new project. All other things being equal in programming or system administration, I like to sit on the intersection of functionality, for actual productivity, and popularity, for availability of reference material. To check on the relative amount of helpful documentation I could expect to find, I ran a comparison on Google Trends.

April, 2014

Disappointingly, Rails seemed to be losing ground to EF.NET, at least in terms of Google searches. I tried to console myself by saying that Rails was mature by that time, and EF was still struggling to find its niche, which both reflected in the results. Five years later, I stand by that interpretation.

For comparison, I wanted to see what the situation looked like today. Both technologies were trending down since the last snapshot. I took one guess as to why, and this is what I saw.

July, 2019

For the fun of it, I threw in another couple of terms…

July, 2019, with Frameworks

Yikes. The popularity of React and Angular has stomped the axis of the graph. Clearly, Javascript-based front-end technologies have taken over web development mindshare.

I find this state of affairs to be morose. Some time ago, through a series of inescapable constraints, I was backed into a corner to write a new web application in Java/Javascript. Through other, defaulting logic gates, I wound up trying to use Spring Boot and Angular 2, in particular. I found them both to be tedious, laborious, and almost utterly devoid of helpful documentation on the internet. The only consolation I can take from the graph, above, is that React seems to be winning against Angular. I haven’t tried it yet, but it gives me hope that it’s better.

In the end, after literally weeks of reading and searching, I found exactly one, non-trivial example of how to use this stack, and that was only because I sent an email to the guy who seemed to be the chief evangelist of Java/JS on the internet. While that was great, his example was so out of date, I couldn’t reconcile how to translate his approach into modern idioms. Coming from the Oracle/Java world, this stack is intended to be all things to all people, and it shows. There is no commonly-accepted way of doing things with it that people seem to agree on.

If you’re creating some sort of enterprise-y, company-wide system, containing highly-important data, I could see breaking the backend and the frontend apart along language/framework lines, to facilitate having different teams coding them. (Even though the strict typing of a JS frontend is going to drive both sides crazy.) But for a tiny, departmental web app? Containing no sensitive data? That just tracks dates? Which might be used by a handful of people? Using a Java/JS solution for this is like using a nuke to get rid of a gopher in your back yard.

Rails shines the brightest when making small, line-of-business apps like this. Fifteen years after the first release of Rails, there is still nothing in the web development world that can touch it for productivity. Ruby’s interpreted nature — while prone to being slower, compared to typed, compiled languages — is precisely what makes it so easy to use, and so flexible in the role of a database ORM.

It seems that Entity Framework never really got off the ground. Most people writing about it recommend using something else, like Dapper or nHibernate. Dapper does so little for you that you might as well just write text-substituted SQL yourself, and nHibernate is really out of date, so I’d rather just put up with EF’s limitations. And, again, I’m sad, because I’m pretty sure I’m going to get backed into a corner of using ASP.NET for another project. I’ll do my best to make sure it’s .NET Core, for future-proofing, but, for the same reason, EF Core isn’t any better.