Structural damage detection has been an interdisciplinary area of interest for various engineering fields. While the available damage detection methods have been in the process of adapting machine learning concepts, most machine learning based methods extract “hand-crafted” features which are fixed and manually selected in advance. Their performance varies significantly among various patterns of data depending on the particular structure under analysis. Convolutional neural networks (CNNs), on the other hand, can fuse and simultaneously optimize two major sets of an assessment task (feature extraction and classification) into a single learning block during the training phase. This ability not only provides an improved classification performance but also yields a superior computational efficiency. 1D CNNs have recently achieved state-of-the-art performance in vibration-based structural damage detection; however, it has been reported that the training of the CNNs requires significant amount of measurements especially in large structures. In order to overcome this limitation, this paper presents an enhanced CNN-based approach that requires only two measurement sets regardless of the size of the structure. This approach is verified using the experimental data of the Phase II benchmark problem of structural health monitoring which had been introduced by IASC-ASCE Structural Health Monitoring Task Group. As a result, it is shown that the enhanced CNN-based approach successfully estimated the actual amount of damage for the nine damage scenarios of the benchmark study.