So, this is going to merge two related but separate things regarding the state of publishing today. The first is information that came out of the Random House - Simon and Schuster anti-trust trial. The second is a book looking at the history of the science fiction/fantasy publishing world, and how it got there.
First up, the anti-trust trial. This is a NYT "best selling" author who is learning the reality behind her industry by watching the trial. And that reality is that books aren't selling. The biggest takeaway is the first one: of 58,000 titles the two publishers put out last year, half sold less than one dozen books. She goes through the stats, but it breaks down to half sell less than 12 copies, 90% sell less than 2,000 copies, 98% sell less than 5,000 copies.
For anyone who's been paying attention to how some of the personages on the left can become "best selling" authors without you ever seeing someone actually buy the book, just realize that 10,000 copies ordered and then burned in a back alley makes that book a best seller.
Second is a new book by a "NewPub" fiction writer, JD Cowan titled The Last Fanatics: How the Genre Wars Killed Wonder. It looks at the state of the science fiction/fantasy/adventure genre from ~1930 to just before the release of Star Wars, and shows that the reason why the SF/F genre dwindled in that period was internal manipulations by people inside the industry who were fighting each other in repeated purity spirals.
The two sides boiled down to: people who wanted to inspire people into the tech world and those who just wanted to take over the industry and use it to preach at the stupid people who read these silly books. (Does this sound familiar to anyone?) Of the first group, one of the people in the "science fiction fan community" of the time said:
But [science fiction], far from being the stimulus to scientific study it should be, had become an end in itself... a sort of pseudo-scientific refuge for persons either incapable of pursuing a technical career, or else too lazy to do so.
And the stupid and lazy clique took over the industry, because while there were very few of them (definitely less than a thousand, probably less than 200), they were closer to the levers of power, louder, and willing to get dirty... up to and including hiring street thugs to beat up rivals in the clique to force them out under threat.
The solution to both this problem, the fanatics taking over the industry and running it into the ground, is to build a parallel industry. This is what is happening in the science fiction/fantasy/adventure world as several "rebel" publishers (Baen, Castalia House and Defiance Press to name a few) have now set up publishing houses that can do smaller production runs that will actually sell by authors who can't or won't get in the door at the "big" publishers.
Also, that phrase the SF writers of the 1940s and 50s always uttered? "Don't read anything written before 1940!" It's projection, they didn't want you to find out what used to exist before they wrecked it. Defy them, go find some of the old Burroughs adventure books, pulp comics and Gothic horror books and read them. And then find people who are making that now and support them.
I have some knowledge about publishing, though not this part of the industry specifically.
For every publisher--literally, every single one--you have a tiny number of hits, a bunch of books that do ok, and a long tail of books that are dogs.
Publishers are basically like venture capitalists. They have capital to invest in a certain number of projects. Publishers know some of these projects will fail and they hope that enough will succeed--and some will succeed fantastically--to grow wealth.
As an example, Bloomsbury Publishing was a small publisher in the UK. They published a bunch of books none of you have ever heard of. They were profitable and did just fine, but without any blockbusters. Then they published Harry Potter. That's a once in a century type of hit. They've since gone on a buying spree, buying up other publishing houses around the world (the fastest way to grow--acquire other companies), and last year they made a profit of more than $30 million.
Publishers also rely on their backlists. If you have a hit one year, it will generally sell worse for every year. But again, it's that long tail. There's a good chance it will keep selling for many, many years. Think about it like this. If you have one book that sells only 50 copies in a year, let's say you make $250. That's nothing, right? Now let's say you have 2,000 books that sell 50 copies a year. That's $500,000 a year. Not so shabby. Let's say you have 5,000 titles that sell 50 copies a year over 10 years. 12.5 million or so. Who would turn up their nose at that? These books are just sitting in a warehouse, no marketing costs, not development costs, just selling backlist titles.
(This is, of course, easier said than done.)
In some cases, yes. But the Thor Power Tool Co. v. Commissioner case ended the drive to keep a large backlist of all of a publisher's writers. Now, the backlist is for those who earned out their advance in a big way... the 2-5% of writers who sell 2k or 5k books.