There's a tricky balance between the mass of the powder charge, the burn rate of the particular powder, and the pressure generated.

A few things to keep in mind:
Regardless of the burn rate, most powders will make a pretty similar amount of gas for a given mass of powder.
Higher pressures increase the effective burn rate of all powders.
Bullet velocity is dependent not on peak pressure, but upon the area under the pressure curve. Pressure equals acceleration, but the duration of acceleration is just as important.

If you were to use a really fine powder, it burn rate would be increased of course. That increased burn rate would result in higher peak pressures, which would necessitate a smaller mass of powder to keep pressures low enough for safety. The smaller mass of powder would produce a smaller volume of gas. In essence the pressure curve would peak very quickly as all the powder burned, but it would taper off again just as quickly. You would get a very short pressure spike with less area under the curve than you would achieve with a slower burning powder.

If you did the opposite and used a coarser, slower powder; the peak pressure would drop. This would allow you to add more powder to bump up the peak pressure again. Pressure would build slightly slower than with the fast powder, but still fairly quickly. After peaking, the pressure curve would taper off more slowly due to the greater gas volume being produced by the larger mass of powder. The pressure curve would have more area under it and the bullet would end up going faster.

The highest velocities are usually seen with the powder that is just fast enough to reach the limit of safe peak pressure with a fully packed case.

Now, there are variations in the powder mass/gas volume ratios for different powders and the powder manufacturers are working on new powders with higher ratios (which is how some new rounds manage to achieve the velocity numbers that they do), but hopefully the above explanation gives some idea of why things work they way they do.

Right. We're limited by just how much we can get out of the chemicals we use and by the peak pressures our firearms can safely handle, so we have to limit the burn rate in order to be able to use as much powder as will fit in a case (if maximum velocity is our goal).

Right, and the easiest way I've found to explain burn rate-

Think of the bullet as a bowling ball.  You can achieve speed X by hitting it hard with a sledgehammer(fast powder) or the same speed X by launching it out of say, a three man slingshot(slow powder).  Either way you get the same speed, but the amount of and duration of the forces applied to get the same velocity vary greatly.  

The most efficient powders, as JesseL noted with give you nearly 100 % case fill with top velocity for the working pressure limits for the cartridge in question.

Also add the "object at rest tends to stay at rest" into the equation.  Basically the longer the powder can maintain peak pressure on the projectile, the longer the amount of time the projectile is accelerating, gaining more and more velocity.  This varies greatly with barrel length, slug length, and slug diameter. Barrel length-  amount of time the powder gasses are in a controlled expansion chamber.  Slug weight- resistance to motion.  Slug diameter-  Base area of the slug is the only area the powder can actually exert useful force.  The larger this area the more force is applied for a given pressure and cartridge case volume.  Look up the ballistics for a 30-06 and a 338-06 for an example.  Even with similar slug weights the 338-06 out runs the 30-06.  Same case, same slug weight, larger base area on the larger caliber slug to push against.