College of Information Science capstone project makes AI powered video processing tools easier to use for U of A labs
As part of its increasing portfolio of partnerships and programming at the University of Arizona, the Data Science Institute with the U of A DataLab participates in mentoring high school, undergraduates, and graduate students with the development of their capstone projects that help to discover solutions to real-world challenges. The projects not only prepare students for careers, but they can also produce tools that have immediate impact on campus teaching and research. An example of this impact is the capstone project by Nick Ferrante and Adrian Girone.
In May, Nick and Adrian both earned their M.S. degrees in Data Science with the College of Information Science. For their INFO 698 course, they completed a joint capstone project, Deep Learning Pose Estimation, and developed “Fast Track” which is a cloud-based, containerized software tool that transforms animal pose estimation from a highly technical and often challenging area of computer vision into something accessible and usable for biologists and behavioral researchers. They developed Fast Track with the input of researchers and instructors who now use the software with their research and in their classrooms.
Pose estimation is a method of analyzing motion through video. It tracks the position and orientation of body parts over time with no physical sensors or intrusive attachments. In the context of animal research, this kind of analysis helps researchers observe behavior in a more accurate and ethical way. But setting up pose estimation software like DeepLabCut (DLC) or SLEAP typically requires a strong technical background. Most researchers in behavioral biology do not have that background and they should not need to have it. That was the challenge Nick and Adrian decided to address as their capstone project.
Their solution was twofold: simplify the software setup and deliver the computing power researchers need. For many, trying to install pose estimation tools would mean days of struggling with dependency errors, version mismatches, and GPU configuration. Nick and Adrian created pre-packaged environments that are ready-to-run apps hosted on powerful cloud platforms so that researchers can spend time conducting research. "Something that works, right out of the box. No self-assembly required,” as they put it.
They developed their solution by first containerizing the industry-standard pose estimation tools, DeepLabCut (DLC) and SLEAP, using Docker to ensure reproducibility and to eliminate dependency issues. They deployed the applications across the NSF-funded cloud platforms, CyVerse and Jetstream2, for browser-based access. Each platform provides something different: Jetstream2 allows flexible provisioning of resources for class or lab use and CyVerse provides a research-centered interface and data tools.
The tool suite provides users with access to cloud computing resources and allows them to analyze movement, position, and ultimately behavior of animal subjects either through CyVerse applications or Jetstream2 instances (which are created and accessed through CACAO). Using these cloud-based tools, users are able to upload labeled or unlabeled video data (of bees, flies, ants or whatever they are working with), and process them through DLC or SLEAP.
With the deployment tools, researchers no longer need their own high-end machines, local installation, or high-performance hardware which is important to researchers with limited computer capacity and budgets. The infrastructure is there. They just log in and go. Additionally, U of A faculty, staff, and students can get a free CyVerse account.
For some of their early users of Fast Track, this is a welcomed drastic improvement. Instructors, like Dr. Martha Bhattacharya at the U of A, have already begun using Fast Track. Her students helped pilot the software in a previous term by analyzing behavioral videos of flies. Through CyVerse, students were able to work directly with pose estimation workflows using tools like DLC and SLEAP, all from a browser-based interface.
While the capstone project focused on animal research, the tool application use-implications extend far beyond. Pose estimation is just as relevant in sports science, physical therapy, robotics, and biomechanics. The modular nature of the solution means future adaptations whether for human gait analysis or teaching robots to replicate human actions are realistically foreseeable.
In just one semester, Nick and Adrian delivered a project that was both technically impressive and immediately useful. Their work shows what is possible when advanced tools are shaped around real research needs and when computing power is treated as a shared resource, not a gatekeeping barrier. They set a high standard for what student-led work can achieve. Their capstone advanced the project’s goals and laid the groundwork for future progress. Importantly, they documented their project’s steps. Their detailed guide is available in their GitHub repository. It will equip the next group of students to build on their momentum without starting from scratch.
The capstone project was guided by mentors, Carlos Lizárraga and Nirav Merchant with the Data Science Institute, who helped prioritize features that bring real value to the research community. Their insight made sure Nick and Adrian focused on tools researchers were familiar with or would be interested in using, like SLEAP and DLC, with CyVerse and Jetstream. Michele Cosi (DSI), Edwin Skidmore (CyVerse), and Isaac Robinson (Wheeler Lab) provided vital technical assistance. Early feedback from animal researchers Shannon McWaters, Chinmay Joshi, and Martha Bhattacharya’s class helped ground the design in real-world use cases.
Reflecting on the experience, Nick and Adrian emphasized the importance of understanding the people behind the problem. “Technical people often get lost in the weeds and forget the problems they are solving are for human beings,” Adrian said. “If you want to get better at working with models, you need to get better at working with people.” That human-centered and outcome-focused mindset shaped their entire approach and is likely a key reason the project connects so strongly with researchers and their mentors.
And maybe, best of all, the result of their capstone project is already making a difference to researchers. The simplicity and accessibility of Fast Track makes it possible to bring advanced data science tools into classrooms and support project-based learning grounded in real datasets. It also ensures that students and researchers can focus on learning core scientific concepts without being overwhelmed by technical barriers.
For more information about the U of A DataLab or Nick and Adrian’s software, email Nirav Merchant at the Data Science Institute.
# # #