Franz Mayr's thesis begins with a key question: What exactly do neural networks learn?
His research focuses on models that process symbolic sequences—that is, information represented as an ordered sequence of symbols. This type of processing is found, for example, in language-related tasks (such as large language models), but also in other problems where data can be viewed as sequences.
“Today, these models are extremely complex and, in many cases, function like a black box,” he explains.
The problem is clear: even if we know how they are built, it is not always possible to understand why they make certain decisions.
In light of this, his work proposes a solution: to develop techniques that allow us to “open” that black box and generate a more understandable representation of the model. “It’s like building a blueprint of the machine, which allows us to understand how it works and verify that it meets certain properties,” he notes.
More reliable models
The central contribution of this thesis is to adapt classical computational tools, used to verify systems, to the field of artificial intelligence.
Simply put, this paves the way for more understandable models with greater analytical capabilities and reliable performance guarantees. In a context where artificial intelligence is increasingly being integrated into critical applications, having access to these types of tools is essential.
“What we did was develop techniques to demonstrate properties of these black boxes, thereby creating more understandable models,” he summarizes.
Applications and limitations
Although the work has potential applications, Franz points out that significant challenges remain.
The techniques developed can be applied in certain contexts and models, but they still face limitations when it comes to large-scale systems such as today’s language models. In these cases, it is not only the complexity of the problem that comes into play, but also factors such as available computing power. “It’s not just a matter of the technique, but also of the available hardware. Often, progress depends on that combination.”
A long-term journey
The thesis took between four and five years of sustained work on the same problem.
For Franz, the biggest challenge wasn't just technical. The real difficulty lay in staying focused over time, especially during moments of uncertainty when results don't always come.
“The biggest challenge is staying focused on the same problem for years. There are times when results don’t show up, but you have to keep going.”
That process also involved dealing with constant evaluations, academic reviews, and the need to stay motivated throughout the entire doctoral program.
Conducting research as a team
Far from viewing his dissertation as an individual achievement, Franz emphasizes the collective effort behind the research.
Theses are not the work of a single person; they are the result of a team’s efforts.
In this process, he emphasizes the importance of interacting with other researchers, both to enrich ideas and to keep the work moving forward. Research involves comparing approaches, discussing technical criteria, and learning to reach consensus in contexts where there isn’t always a single answer.
It also highlights the value of perseverance and resilience. In the academic world, even well-written papers may not be accepted at first, which requires revising, improving, and trying again. Along the way, dialogue with colleagues and support from the team are essential.
From student to teacher
His career at the School of Engineering combines training, research, and teaching, in a process that has gradually taken shape.
The transition from student to researcher involved not only delving deeper into a specific field, but also developing a more rigorous and nuanced approach to problem-solving. That learning is now reflected in his teaching role, where he strives to expose students to both theoretical concepts and the practical experience of conducting research.
In that sense, teaching becomes a natural extension of his academic work. It is not just about imparting knowledge, but also about sharing an approach to complex challenges— one based on analysis, curiosity, and the ability to sustain long-term processes.
A landmark along the way
Looking back on his career, there is one person he identifies as having been instrumental in his development. Franz singles out Sergio Yovine as one of the people who shaped his path in research.
Regardless of the thesis’s outcome, it underscores the value of having mentors during the educational process—not only for their technical expertise, but also for their support on a journey that demands perseverance, sound judgment, and the ability to persevere through long-term challenges.