Starfruit (Carambola) is a tropical fruit, which is at high risk of various foliar and fruit diseases that affect yield and quality. In order to deal with this problem, we present an accurate and efficient hybrid deep learning model combining Graph Convolutional Networks (GCN) and Graph Attention Networks (GAT) for disease detection. Our GCN-GAT model combines both spatial characteristics as well as the relational dependencies between the image data, which leads to an enhanced classification performance. It showed high test accuracies of 96.44% on the original dataset and 97.42% on the augmented dataset. We further compared our method with popular CNN architectures such as InceptionV3, DenseNet121, MobileNet, Xception, ResNet50, VGG16 and VGG19 showing that our hybrid approach surpasses these baseline methods. We implemented a mobile application (CarambolaCam) for potential real-world usage, allowing the farmers to detect diseases based on images of leaves and fruits. These findings further identify the power of graph-based learning in agricultural disease diagnostics and its applicability towards building low-cost, field-ready smart farming solutions.