Characterizing coefficient of thermal expansion (CTE) for thin films is often challenging as the experimental signal is asymptotically reduced with decreasing thickness. Here, we present a method to measure CTE of thin films by locally confining an active thermal volume using harmonic Joule heating. Importantly, we simultaneously probe the harmonic expansion at atomic-scale thickness resolution using atomic force microscopy. We use a differential method on lithographically patterned thin films to isolate the topographical and harmonic thermal expansion contributions of the thin films. Based on the measured thermal expansion, we use numerical simulations to extract the CTE considering the stress induced from neighboring layers. We demonstrate our method using poly(methyl methacrylate), and the measured CTE of 55.0 × 10⁻⁶ ± 6.4 × 10⁻⁶ K⁻¹ shows agreement with previous works. This work paves an avenue for investigating thermo-mechanical characterization in numerous materials systems, including both organic and inorganic media.