Why can't we build mid-priced housing?

Pretty much everyone agrees at this point that we have a crisis of housing affordability and choice... there just simply isn't a big enough supply of housing, particularly in the low-to-middle areas of the price scale, to meet the demand.  There are no easy solutions to this-- a multitude of factors have worked together to create this situation.  The links below highlight some of the areas that make compact urban development, particularly for non-luxury units, just not pencil out:

Establishing by-right development and streamlining local permitting processes will allow developers to respond nimbly to market demands and will relieve the “guilty until proven innocent” status of new building development, which depresses construction starts across the country by delaying and inhibiting housing projects.
— Washington Post