UI Component Detection in Mobile Interfaces — FMSS Research Project

This project was part of a TÜBİTAK-funded AI research collaboration with FMSS Information Technologies, focused on detecting UI components in mobile application interfaces using deep learning models.

The goal of the project was to explore how computer vision techniques could be applied to mobile interface understanding. Instead of analyzing traditional images, the task focused on application screens, where buttons, text areas, cards, inputs, icons, and other interface components need to be detected and classified in a structured way.

I worked as an AI Research Engineer within an 8-person research team. My main contribution was developing and experimenting with a Faster R-CNN-based object detection pipeline using a ResNet backbone. This involved preparing the dataset, setting up the training process, evaluating model performance, and analyzing how different architectures behaved on UI component detection tasks.

In addition to the Faster R-CNN pipeline, our team conducted comparative experiments with multiple deep learning architectures including DenseNet, EfficientNet, and VGG. We also explored YOLO-based fusion approaches to evaluate whether faster object detection architectures could improve the system’s practical usability.

One of the valuable parts of this project was learning how research work differs from normal product development. The focus was not only on making something work, but also on comparing methods, documenting experiments, understanding model behavior, and contributing to a research process with a larger technical team.

This project strengthened my experience in computer vision, dataset preprocessing, object detection, model evaluation, and academic-style experimentation. It also gave me a better understanding of how AI research can be connected to real software engineering problems in mobile applications.