To evaluate the performance of steel fiber-reinforced rubber concrete (SFRRC) in a sulfate environment, a rapid freeze-thaw testing procedure was employed to assess the influence of steel fiber content on parameters such as mass, relative dynamic modulus of elasticity, compressive strength, and damage layer thickness (Hf) of SFRRC. The testing revealed the deterioration pattern of SFRRC in a sulfate erosion and freeze-thaw environment. Additionally, the mercury intrusion porosimetry technique was utilized to further investigate the pore structure characteristics of SFRRC with the goal of revealing the damage mechanism from a microscopic perspective. The results indicate that SFRRC undergoes a lower degree of freeze-thaw damage in sulfate solution than rubber concrete without steel fibers. The degree of deterioration of SFRRC gradually decreases with an increasing steel fiber content, but its frost resistance is adversely affected at a content level of 2.0%. The Hf can be used to characterize the internal damage in the SFRRC. As the Hf increases, the loss of compressive strength in the damage layer becomes more pronounced. A correlation exists between the compressive strength of SFRRC and that of the damage layer under sulfate erosion and freeze-thaw conditions, enabling calculation of the latter based on the compressive strength of the SFRRC under the influence of environmental factors. An appropriate incorporation of steel fibers optimizes the pore structure of SFRRC.