Brad, re your question about what might have been done vis-à-vis building more powerful steam locomotives, I think that it needs to be put into a context of time and circumstance. I suspect you're thinking in terms of today, and a situation where you're the chief motive power officer. But I think that today you'd also be wise to consider whether or not advanced traction control techniques could be applied, and also whether or not MU was practicable. That push-pull operation of steam locos was practiced suggests that the answer to the latter question must be yes. So the need for more power could be addressed by MU'ing say 4-8-4+4-8-4 Garratts, rather than getting into the complexities of Mallet-Garratts. (Front engine stability, getting enough of the weight on the front engine units, and length and complexity of steam pipes are three areas of concern.) But up-to-date steam is getting into the world of Porta and Wardale, so I'll quickly stop here before I get in over my head.
An alternative frame for the question is what might have been done in the 1945-50 period had say the diesel locomotive been a decade more retarded than it actually was? Here the constraint is what would the US roads have found acceptable. And it's a fairly safe bet that inside cylinders, Garratts and compounding (other than for low speeds) would have been a very difficult sell, as would FAs below 4. So, the question reduces to: could bigger and more powerful articulateds have been built within the existing loading gauge and generally prevailing axle loading restrictions? Let’s take 70 000 lb maximum and 67 500 lb mean as the axle loading number, and allow that installation of bigger turntables to accommodate longer locos might have been justified on a situational basis.
More power, at the same starting tractive force, was probably available from existing designs like BB, given appropriate detail attention to the whole air/gas/steam cycle, opening out the steam passages, and installing poppet valves. The last-mentioned had a patchy reputation in the USA, but I suspect that this was simply the result of incomplete mechanical development, rather than any fundamental shortcoming. But work elsewhere indicates that poppet valves are worth having. Who knows, with all of this maybe BB could have been taken to around 8000 dbhp.
Dusting off and updating the never-built N&W Y-7 2-8-8-2 simple articulated likely would have resulted in a more potent machine, although probably transgressing the 70 000 lb maximum axle loading a little bit, unless you went for a 4 wheel trailer truck. It's not clear to me at what speed the dynamics of a four-wheel pilot truck become necessary and not just desirable, but forced to a guess, I'll pick somewhere between 50 and 60 mile/h. Oh dear, if we're not careful here, we'll soon back into a N&W-ized BB; we've sort of already gotten to a N&W-ized DMIR 2-8-8-4.
More starting tractive effort at the same axle loading, whether or not with more power, than either of these machines could manage would require moving to an x-10-10-y. But this would be a restricted application machine because of both its rigid wheelbase and its hammer blow. (Lateral motion devices, lightweight rods and better balancing will help, but you can also put these on the x-8-8-y, so the latter will still go faster and bend easier. And with 10-coupled, you still have that huge vertical component of piston thrust to deal with.)
Quick calculations show that you could put 2-10-10-2 running gear, with 63" drivers spaced 67", under the BB without moving either set of cylinders, although ashpan conditions could be marginal. 25.5" x 32" cylinders would give 168 000 lbf TE; FA of 4 at 675 000 lb on drivers. Balance conditions wouldn't be ideal, and the front engine would have severe yawing and rolling forces that might crack even a cast steel bed, but continuous operation at 45 might be feasible. I'm not sure where you could use such a machine, though. Where are there combinations of relatively long grades with few curves, coupled with the absence of opportunity or need for fast (60 mile/h) running? Not Donner or Feather, I think - too many curves. Up the Wasatch maybe, but not along the Wyoming plateau, where speed is needed. Out west, you'd most likely be looking for locomotive runs of at least 500 miles, and maybe 1500 miles.
The VGN AE could be updated using N&W Y-6b proportions suitably extrapolated. I think that the cylinders would come out at 28x36X32" (for 300 lbf/in2 BP), but I'll leave the other calculations for Brad. But with 56 - 58" drivers, balance conditions would be bad, and I don't suppose that much more than 35 mile/h could be expected. So it would be a "grunt" machine for heavy grades, helper work, etc.
Whether the planned N&W Y-7 could have been extrapolated into a 2-10-10-2 without the boiler fouling the loading gauge I don’t know. If so, maybe it would have to be a 2-10-10-4 to keep the axle loading within bounds. And it still couldn't run that fast without pounding the track to pieces. But that, I'd say, is about as big and powerful as could have been achieved around 1950 within the generally accepted boundaries of the time.
So, no great revelations here. For general service, fine tune the existing designs as described above, and if you can, work out MU or some form of synchronized control system so that you can easily pair up BBs, Y-6Bs, etc., or admixtures thereof. Now that should have been doable circa 1950 with air motors for the throttles and cutoff, and maybe some sort of air-operated steam shut-off valve for slip arresting on the trailing unit. To support what I've just said, an air throttle control was used on the Pennsy 6-8-6 turbine, a slip control valve was used on the Pennsy Q-2 (OK, it needed a bit more work to become adequately reliable), and down under in NSW, Australia, rotary air motors were used to provide duplicate (right-hand side, reverse direction) cab control of cutoff on the 60 class Garratts.
Conclusion - weren't the US railroads lucky that the diesel was ready in 1945! - Sorry, Brad.

))