For years, there has been a lot of speculation about how fast an SR-71 could fly. The top speed was top secret for decades. However, the SR-71's pilot manual has been declassified and is available online here. The answer to the SR-71 top speed question is here. Cool.Maximum Mach
Mach 3.2 is the design Mach number. Mach 3.17 is the maximum scheduled speed recommended for normal operations. However, when authorized by the Commander, speeds up to Mach 3.3 may be flown if the limit CIT of 427 degrees C is not exceeded.
So, it appears the limiting factor was compressor inlet temperature. Slick.