Continuous and non-invasive respiratory rate (RR) monitoring would significantly improve patient outcomes. Currently, RR is under-recorded in clinical environments and is often measured by manually counting breaths. In this work, we investigate the use of respiratory signal quality quantification and several neural network (NN) structures for improved RR estimation. We extract respiratory modulation signals from the electrocardiogram (ECG) and photoplethysmogram (PPG) signals, and calculate a possible RR from each extracted signal. We develop a straightforward and efficient respiratory quality index (RQI) scheme that determines the quality of each moonddulation-extracted respiration signal. We then develop NNs for the estimation of RR, using estimated RRs and their corresponding quality index as input features. We determine that calculating RQIs for modulation-extracted RRs decreased the mean absolute error (MAE) of our NNs by up to 38.17%. When trained and tested using 60-sec waveform segments, the proposed scheme achieved an MAE of 0.638 breaths per minute. Based on these results, our scheme could be readily implemented into non-invasive wearable devices for continuous RR measurement in many healthcare applications.