We present monolithically integrated waveguide with thin isolated layers of dielectric material, which ensure efficient emission of guided light in multiple optical windows. The proposed waveguide is fabricated by using silicon oxynitride (SiON) which functions as the core guiding layer, and silicon nitride (SiN) which facilitates effective vertical emission of guided light via optical windows. The theoretical operating principle is analyzed by Finite-Difference-Time-Domain (FDTD) simulation, and the experimentally fabricated device is demonstrated to find the expected operational trend by measuring vertical emission of light through the optical windows. We further find that the physical origin of such efficient emission stems from the Fabry-Perot resonance induced laterally at the interface between the optical windows and cladding layers, which leads to the strong emission of light at the edge of optical windows. The output efficiencies of light emission from SiN optical windows, which are fabricated in cascaded fashion on top of the SiON waveguide, are 30, 45 and 55 percent at the wavelength of 457, 532, and 637 nm, respectively. The proposed design concept can be applied to various potential applications including optical integrated circuits with vertical interconnection, optical emission in wearable devices for virtual and augmented reality, and other emerging optical sensors.