Although I'd long noticed that the price of used 3.5 books was not coming down on Amazon (and was in fact increasing) indicating robust demand, I'd never really thought much about the notion of who plays what and trying to gauge the relative sizes of the various fractured "camps" within the D&D playing world. It wasn't really until recently that all of that crystalized in my head around the idea that there are basically four large camps which, while maybe not exactly equivalent in size, are certainly on the same order of magnitude. This includes 1) the OSR, 2) folks who never left their d20 D&D games, 3) folks who migrated to Pathfinder as the spiritual successor to the D&D that they used to play, and 4) of course, folks who did migrate to 4e after all. Because Pathfinder was positioned as the successor to 3.5, I figured that most folks who played in that same vein probably played Pathfinder rather than 3.5, and I presumed that my own hesitation with migrating to Pathfinder was merely my own inertia and conservatism in taste, as well as the fact that I'd spent a lot of time getting 3.5 to what I wanted it to be already and didn't need what were, in effect, Jason Buhlman's houserules on top of my own. That isn't to say that I didn't buy a lot of Pathfinder product (in pdf, where it was relatively cheap) or adopt a few of it's better rules as houserules of my own, but again--my assumption was that there was the OSR, Pathfinder and 4e, and lingering d20/3e/3.5 players like me and my group were the exception. I probably shouldn't have been surprised, given the sales of 3.5 books on Amazon, but I kinda was, and I hadn't really thought about the idea that maybe the group that's quietly been playing 3.5 (or even 3e) all along without migrating to either 4e or Pathfinder might have been a significant group; on the same order of magnitude as the other three.
So, in many ways, I've been a little reluctant to talk too much about the specific d20 nature of my games, assuming that that's not of interest to anyone. Granted, it's not like I have a big and loyal following of readers anyway, and I'm mostly doing this blog for my own benefit not for anyone else's... but subconsciously, I felt like talking too much about a system that I assumed was obsolete was wasted effort. Now that my admittedly somewhat speculative analysis leads me to the opinion that actually the d20-playing crowd is bigger than I thought (plus, it also encompasses Pathfinder players--although I suspect Pathfinder players are generally fairly committed to their system and aren't necessarily interested in "fixing" it) I feel more emboldened to talk about it more openly. For occasions when I do so, I've created a new label, d20 which will be for posts in which the d20 system explicitly plays some kind of role in what I'm talking about.
All that said, "fixing" a game system, of course, is heavily dependent on what you think the problems are, and they are unlikely to be universally regarded. I do tend to think that there are some problems I see with d20 that many players of d20 also see as a problem, but I believe that many other problems I have with d20--or at least specifically with the D&D iteration of it--are questions of taste and I may actually be in the minority with what I prefer. By and large, what I prefer is a system that runs fairly fast and loose at the table, without bogging down in minutiae and tedium. I do, however, really like a robust and detailed character generation method. I like being able to give characters plenty of definition through the rules... but when we're actually playing the game I want it to fade more into the background and be easy and intuitive to run.
Much of this can be accomplished by merely remembering the "Tools, not rules" maxim that was present at the launch of 3e back in 1999 or 2000 or so. If you believe that the GM adjudicating an attempted action by picking a skill that sounds relevent and making up a DC that sounds reasonable, for instance, then all that business in the skill description part of the rules becomes merely example and modeling of how it might be done rather than a straitjacket on how to play the game. I've always run this way, so much of the "choking on rules" complaints I've heard about d20 over the years struck me as a bit odd and I'm not sure why otherwise reasonable people who played that way in other systems get hung up on that in d20 specifically, but it's worth stating explicitly again for emphasis--tools, not rules. Use the tool that's appropriate for your game; when it's not appropriate, don't use it. Just as you wouldn't use a circular saw to put a nail in a 2x4, you don't need all the detail that the game gives you if it's not something that's benefitting your game. That should be intuitive, but... apparently it's not.
So, with that, I'll divide the remainder of my post into two blocks--generic system fixes, and fixes that cater to my specific taste. Each block will first identify a problem, and then provide a solution. For the most part, luckily for us, these solutions are already in print. Folks have been tweaking d20 for quite a while now, so it only seems fair that actual designers and others will have come up with great solutions that I should look at first before attempting to reinvent the wheel.
Generic system fixes
Problem #1) High level D&D is too complicated and cumbersome. The math doesn't work out very well or sensibly, it's too difficult to use, and it feels too different from D&D at lower level. It's as if the entire genre of the game changes.
Solution: Use E6. Avoid higher levels. Even without E6, you can have campaigns that end before you get into higher levels. E6 (or you can arbitrarily set it at E8, or any other fixed point you wish) allows for continuous play with the same characters, though, and avoids the problems with higher level D&D.
Problem #2) Base attack bonus increases with level, meaning that you're better at hitting things. Armor class does not. This means that unless you wear increasingly heavy armor, or have a high hit point total, you can't really be involved in melee very well. The idea of the swashbuckling rogue is a nice one, but it doesn't actually work in the system.
Solution: D&D itself patches this clumsily by assuming that you will have access to magical equipment that closes this gap. Every other d20 game--Wheel of Time, Star Wars, d20 Modern, etc. on the other hand, give you a level/class based AC progression. In other words, you get harder to hit as you advance, just as you also get better at hitting. There's also a rule in place for D&D for this in Unearthed Arcana, although it's not perfect. Here, since it replaced armor proficiencies, the same classes that have heavy armor are better at avoiding getting hit. Somehow this doesn't seem quite right; the swashbuckling rogue, for instance, isn't really much better in this system than he is in D&D normally. Ideally what would be done is that after looking at the d20 Modern or Wheel of Time or Star Wars game for inspiration, the progression would be manually assigned to each class as appropriate. This just means doing a little bit of shuffling on who has which bonus, though. And I freely admit that I might be biased in my opinion on who should have the better ones--because they're the classes that I most like the archetypes for.
Problem #3) Iterative attacks (i.e., once you have a BAB of +6, you get a second +1 attack as long as you make a full attack option) bog down combat. They also have the (probably unintended, although you never know) side effect of making combat very static. Since one of the major literary influences of the game is swashbuckling action stories, this somehow seems totally wrong.
Solution: In an E6 game, only characters with a full attack bonus will ever get to this point. Instead of advancing to +6/+1, however, they'll advance to +6... and then get the ability to move both before and after making an attack, similar to the Spring Attack feat. Unlike Spring Attack, however, you can make this move even in heavy armor--although, of course, your speed and therefore movement will be less.
Problem #4) Attacks of Opportunity create a major complication in combat, plus they force you to use the grid-based miniatures combat instead of narrative "just describe what happens" combat.
Solution: That problem isn't strictly speaking true, although I can see your point. You can play narrative combat with attacks of opportunity as long as you do a good job describing stuff, and make sensible rulings about when AoOs (and feats related to AoOs or the avoidance of them) come up. However, you'll just have to be careful to avoid playing with any overly rules-lawyerish or argumentative players. Frankly, as far as I'm concerned, that's a requirement anyway, and is only coincidentally related to solving this particular problem.
Problem #5) Combat takes way too long, and long combats get boring. Also,
Problem #6) Preparing for the session takes way too long. Statting up all these NPCs is a ton of work, and takes forever.
Solution: Schrödinger's stats. Do you really need to know all of those stats? I submit that you do not. And if you do not, your players will never know the difference anyway. Don't be a slave to the stats. I don't know how many combats I've run with NPCs or monsters who were never statted up at all. Even in the midst of combat, I didn't know what any of their stats were. I made up to hit and damage bonuses (and dice type) on the fly, and combat ended when I could tell that the players were ready to move on, or that the result was inevitable. I.e., they didn't necesssarily have set hit points either; I have the ability to decide as the GM that any given hit took the combatant into negative hit points by "retroactively" adjusting wha tthe hit point total is. This can be done even if there is no hit point total. It takes a bit of a paradigm shift, and sometimes we think that doing this is "cheating". It's not. It's the GM's responsibility as the adjudicator of the game to make sure that it runs at optimum at all times. And frankly, that doesn't mean always following all of the rules as written by the letter. Taking some kind of moral high ground because your game is more "correct" but less fun is a poor consolation prize.
Problem #7) There's too many skills. Some of them seem to overlap, or otherwise make it difficult to split out.
Solution: Eliminate Move Silently. Every Move Silently check can be done with the Hide skill. Eliminate Listen. Every Listen check can be done with the Spot skill. Maybe even eliminate Gather Information and Bluff and simply use Diplomacy. I'm a little less sold on that last one, though.
Problem #8) There's too many feats. Lots of people don't think you can do something if you don't have the proper feat to do it. Plus, feats are like little rules subsystems that don't operate the same way as the rest of the game (often) so it greatly increases complexity.
Solution: Yeah, there are a lot. Don't use them all. There's no reason you have to. Plus, although there's a lot, for the most part, players need to worry about the feats that their characters have, and none others. And you don't need to worry too much about it either. Feeling paralyzed because you think you need a feat in order to do something unusual isn't a problem with the rules so much as it is a problem with the approach to the game. You just have to... y'know... not play that way.
Problem #9) The 15 minute adventuring day.
Solution: Well, this is a problem partly caused by two problems. 1) The way magic works in D&D, which if you see below, I'll be changing, and 2) the way adventures are structured. That's all on you as the GM, buddy. If you have a problem with it, then do it differently.
Specific system fixes
Problem #1) I hate D&D magic. It feels very mechanical, bland, highly focused on combat-utility applications, and plus there's way too much of it. I haven't yet read a fantasy book with magic that felt like D&D magic... even D&D novels.
Solution: Luckily for you, there are a lot of alternate systems in print, much of which is Open Content. I prefer Incantations, which are in Unearthed Arcana and Urban Arcana (and the SRD.) Although it does require that the GM make a lot of spells, or create ritual costs associated with existing spells to adapt them as Incantations, which I admit is a minor drawback. But there are plenty of other options too. I've gotta have no less than half a dozen "magic systems" for d20 games, and only one or two of them was bought specifically because I was looking for other ways to do magic.
Problem #2) If you change the magic system, you've hardly got any classes left! Almost every class had a spellcasting progression except Rogue, Fighter and Barbarian. And monk too, I guess, but he had all kinds of weird supernatural abilities.
Solution: I have no idea how many alternate classes in OGC I have. Scores. Hundreds, probably. Lots anyway. It's certainly possible to give your players lots of options while still curtailing the overtly magical ones. Another way to increase the effective number of character classes without adding more actual classes is to adapt archetypes from Pathfinder. The archetypes work by, essentially, swapping out some class abilities with others that are more in keeping with a specific archetype. Some archetypes are only extremely modest changes to the base class, but others make for some interesting changes that significantly change the feel of the game. And the great thing is, they can mostly be adopted as is even into a 3.5 game that's not necessarily a Pathfinder game, although naturally keeping an eye open for some areas where they don't quite fit is important.
For example, the Rogue class has a number of archetypes introduced in the Advanced Player Guide section of the PRD (the Pathfinder version of the SRD.) These include the Acrobat, Burglar, Cutpurse, Investigator, Poisoner, Rake, Scout, Sniper, Spy, Swashbuckler, Thug, and Trapsmith. It also introduces a ton of new Rogue Talents, which are alternate class abilities useable by any archetype. Most of the archetypes really only change one or two class features, but then they also recommend Rogue Talents that complement the archetype. It's somewhat subtle, but many of them get rid of a lot of problems players might have with playing a Rogue, when the Rogue isn't exactly what they want. For many, the specifically Dungeoncrawling Trapfinding and Trapsense abilities are the ones that get sacrificed to give you something more flavorful. The Ultimate Combat section of the PRD gives us a bunch more archetypes, at least for non-magical classes.
Problem #3) How is it possible that there aren't any rules for running chase scenes in D&D? Isn't that pretty much one of the main, staple action sequences from any cool adventure story?
Solution: No kidding, right? What a major miss. Luckily, a few folks have stepped into the breach to offer us alternatives. I think the simplest and most native d20-feeling chase seqence ruleset I know is from Five Fingers and is written by Wolfgang Baur (I actually think it might have been lifted from some other source--such is the way open content works, after all.) I've actually posted them, in simplified form, here on the blog myself.
Problem #4) d20 doesn't have Madness, or whatever other rule that I really like from some other game.
Solution: So add it in. I prefer the Madness option from The D20 Freeport Companion, but there are other options out there in print, including one in the SRD which is the same as that in Call of Cthulhu--almost word for word.
Problem #5) D&D--and to a lesser extent, d20 games (and any RPGs for that matter) don't offer much support for structures other than dungeoneering. I don't like dungeoneering.
Solution: I don't either. But, as was done back in Ye Olde Days of the Hobbie, we just had to make it up ourselves. It's a shame that there aren't great products out there that support something other than dungeoncrawling, but there aren't really much. Luckily, the system is flexible enough that you can do other things easily enough with it and it still works, so... well, do your best. But you're kinda on your own.