This is a bit late for the current job market for Economics PhDs, but say you have to choose among several job offers (lucky you). The departments are are all of equal prestige and working conditions, salary, and geographic environment are all comparable. The only difference is the size of the faculty. What offer should you take if you care about your future research output?
Clément Bosquet and Pierre-Philippe Combes say you should go for the uniformly good, more field diverse, and larger department. At least this applies to French academic economists. Even more interesting, they observe that department characteristics are as important as researcher characteristics. The initial placement of a researcher thus matters a lot for her future, and this may explain why there are so few success stories after an initially poor placement, at least in Economics. Of course, the academic market in France is very different from any other country, so I am not sure the results can generalize, but this is a start.