Background In recent years, the rapid development of mobile medical technology has provided multiple ways for the long-term management of chronic diseases, especially diabetes. As a new type of management model, smartphone apps are global, convenient, cheap, and interactive. Although apps were proved to be more effective at glycemic control, compared with traditional computer- and Web-based telemedicine technologies, how to gain a further and sustained improvement is still being explored. Objective The objective of this study was to investigate the effectiveness of an app-based interactive management model by a professional health care team on glycemic control in Chinese patients with poorly controlled diabetes. Methods This study was a 6-month long, single-center, prospective randomized controlled trial. A total of 276 type 1 or type 2 diabetes patients were enrolled and randomized to the control group (group A), app self-management group (group B), and app interactive management group (group C) in a 1:1:1 ratio. The primary outcome was the change in glycated hemoglobin (HbA1c) level. Missing data were handled by multiple imputation. Results At months 3 and 6, all 3 groups showed significant decreases in HbA1c levels (all P<.05). Patients in the app interactive management group had a significantly lower HbA1clevel than those in the app self-management group at 6 months (P=.04). The average HbA1c reduction in the app interactive management group was larger than that in the app self-management and control groups at both months 3 and 6 (all P<.05). However, no differences in HbA1c reduction were observed between the app self-management and control groups at both months 3 and 6 (both P>.05). Multivariate line regression analyses also showed that the app interactive management group was associated with the larger reduction of HbA1c compared with groups A and B at both months 3 and 6 (all P>.05). In addition, the app interactive management group had better control of triglyceride and high-density lipoprotein cholesterol levels at both months 3 and 6 compared with baseline (both P<.05). Conclusions In Chinese patients with poorly controlled diabetes, it was difficult to achieve long-term effective glucose improvement by using app self-management alone, but combining it with interactive management can help achieve rapid and sustained glycemic control. Trial Registration ClinicalTrials.gov NCT02589730; https://clinicaltrials.gov/ct2/show/NCT02589730.