Star Wars Stand Alone Films

Purely anecdotal, but the opening night crowd I saw TLJ with was very much with the movie until Luke's death. In fact, when it cuts from the fight between him and Kylo to show that he's still on Ahch-To and is force projecting an image, that got an audible cheer from the crowd. People were still very much into it. After he died? The entire mood of the theater changed. Some guy even punched a chair.

I think creators should work to make sure the things they're producing don't become complete fanservice. That gets bad really quickly (for example, the last season of Game of Thrones felt less like Martin's work and more like D&D checking off fanservice boxes. Some is fine, but too much is blah.), but at the same time I think Rian Johnson became so concerned with subverting expectations that he lost the forrest for the trees.

You just lost Han in The Force Awakens. People haven 't seen Luke doing Jedi things in over thirty years. We get teased with it and then its ripped away. Again, thematically I get what he's going for there. I have no problem with a disillusioned Luke wanting to throw in the towel, coming around on things, or even sacrificing himself for the greater good. In fact, on that last note I actually love the idea of him coming to realize that the "legend" of Luke Skywalker standing against the First Order is what will reignite the rebellion and see the Jedi reborn. Luke Skywalker as a symbol, as a legend can do more than real Luke Skywalker with a lightsaber could hope to. On paper, that's a pretty solid approach.

But it didn't feel earned. The entire story arc felt too rushed and compressed. And again, you've lost two major original trilogy characters in two movies. And while no one could foresee Carrie Fisher dying, that just compounds the issue. As much as I don't believe in all fanservice all the time, just asking yourself as the writer "Is killing off Han Solo and Luke Skywalker in back to back movies the best thing to do?" may be a smart move.

And yeah, force ghosts are a thing but it's not the same.
To be honest if there was one scene I would love to remove from TLJ, it's the one where Luke cavalierly tosses the light sabre over his shoulder down the hill and walks away. Like you said, fans like me have been waiting 30+ years to see Luke again and to have that be the first thing we really see after being teased two years ago? No way.

I will say the projection-Luke fighting Kylo, for the 20 seconds it happened - was awesome. 60 year old Luke with those moves was exactly what I would have paid $20 to see 2 hours of. Not 20 seconds of it.

I don't hate TLJ, but there were some things as you say that a modest amount of fan service would have fixed.