Even the least interested science fiction fan over the last century could tell you that plots are becoming more complex and interesting. The updates between versions of Battlestar Galactica, Star Trek, and V are only the three most obvious examples. Watch the original Flash Gordon and compare that to Sliders. Buck Rogers against Stargate. You get the idea.
So why is it that aliens aren’t a mainstay? You can’t argue there aren’t any different sentient species out there. Even the most conservative estimates are that there should be thousands of different species in our galaxy alone. Nor can you say that they don’t make the plots more interesting. Watch an episode of even the original Star Trek and try to make the argument that the constant clashing of different cultures isn’t the best part of the show.
You might argue that even the grandmaster of science fiction, Isaac Asimov, didn’t feature them. That’s true, but it’s also true he had stories with aliens. The atmosphere of the time was a fear of the unknown, so any plots where aliens weren’t technologically inferior or comical were generally rejected.
We have no excuse now. Again, Stargate was immensely successful and it had humans confronting technologically superior aliens nearly every week. Edge of Tomorrow and Independence Day both featured superior aliens and both have sequels coming.
Why aren’t they a mainstay? Just a suggestion here, but generally either a story (t.v. show or movie) has been about either humans fighting each other or their creations, OR about humans united against a common foe. The plots couldn’t have handled both. But with the storytelling we’ve seen now with BSG, Gotham (from what I’ve been told), and a couple of the new crop of shows on Sci Fy, I think the storytelling has reached a point where it can accommodate both at one time – and I can’t wait to see what comes out of it. Imagine meshing the depth and variety that would be possible. Well, you imagine. I’ll wish my mind was capable of seeing all the possibilities.