Aggregate stability is important for soil architecture; however, the specific response to long-term fertilization in upland and paddy fields are unknown. Here, we studied this issue in two adjacent subtropical upland and paddy soils under > 3 decades of mineral and organic fertilizations. Soil aggregate distribution was measured by wet sieving method. Three aggregate stability indices, mean weight diameter (MWD), geometric mean diameter (GMD), and percentage of water stable aggregates (WSA), were calculated. The stabilizing agents including root biomass, soil organic matter (SOM), and polysaccharides were measured, in which the chemical composition of SOM was measured by pyrolysis–gas chromatography-mass spectrometry (Py-GC/MS). In the upland soil, only manure amendment improved soil aggregation: large macro-aggregates (> 2 mm) increased, whereas mineral fraction (< 53 μm) decreased. Aggregate stability indices (MWD, GMD, and WSA) increased by 52%, 84%, and 28%, respectively, compared with unfertilized soil. The aggregate stability was mainly attributed to root biomass, total nitrogen, and soil organic carbon, following by contents of amino acid derivatives and total N-containing compounds. In the paddy soil, however, the aggregate stability changed little under long-term fertilizations, and the aggregate stability was primarily regulated by polysaccharides. The upland and paddy soils had divergent responses of aggregate stability to long-term fertilization. Manure application increased soil aggregate stability in the upland rather than in the paddy, and therefore, manure amendment is a preferred strategy to improve soil architecture of the subtropical upland.