The future of AI and finance: A Q&A with Renyuan Xu

The use of artificial intelligence in the financial world is not an entirely new phenomenon: since the 2000s, those working in trading and risk management have used simple statistical models and iterative algorithms. In more recent years, however, as computational power increased and massive datasets became available, researchers and practitioners have begun more seriously exploring the transformative power of AI for their field.

Five years ago, the Association for Computing Machinery (ACM) launched an annual International Conference on AI in Finance (ICAIF), aiming to bring together researchers from both academia and industry to share challenges, advances, and insights. ICAIF has since become the largest and most prestigious gathering of the burgeoning community of stakeholders, which includes members of government, regulatory agencies, financial institutions, and NGOs.

NYU Tandon had the honor of hosting the 2024 event, the first edition ever held in a university setting. (Previous conferences were mounted online because of pandemic restrictions, a hotel event space, and at J.P. Morgan, a major supporter.) Spearheading that effort was Assistant Professor of Finance and Risk Engineering Renyuan Xu, whose laurels include a J.P. Morgan AI Faculty Research Award, a SIAM Early Career Award, and an NSF CAREER Award, and who had previously served as the conference’s program director.

Xu joined Tandon just this year, and although organizing a conference of this magnitude while concurrently settling into a new faculty post is a tall order, it was an unmitigated success. More than 600 attendees from 30 countries participated, with a significant majority (well over 400) traveling to Brooklyn to take part in person. More than 250 scholarly papers were submitted for consideration – a 70 percent increase from only two years before – and on offer were a dizzying array of experiences, including four plenary talks, 11 workshops, four tutorials, two sponsor presentations, 12 oral presentation sessions, and two poster sessions.

The senior advisor of ICAIF’24, Professor Tucker Balch of Emory University's Goizueta Business School, also a founding member of ICAIF, says, "Everyone at this conference will remember NYU Tandon and the intellectual discussions and good memories they shared here."

Xu is insistent on sharing the credit for the event’s success. “It was a true team effort,” she asserts. “There are so many people who are owed thanks, including my department chair, Nizar Touzi; fellow faculty members Xin Zhang and Amine Aboussalah; and the entire FRE staff; as well as colleagues from around the world who agreed to co-chair and advise.*

Q&A: Exploring AI in Finance with Renyuan Xu

Could you start by sharing a bit about your background in finance and your current research focus? 

I work at the intersection of AI and finance, focusing on two main areas. First, I design data-driven decision-making methods for trading, particularly in the credit market, where data scarcity and lack of transparency pose challenges. Second, I work on generative AI, such as developing neural network architectures to simulate realistic financial scenarios. These simulation methods help improve risk management of strategies and test stability of financial systems.

What makes the credit market particularly challenging compared to other financial markets?

The corporate bond market operates over-the-counter, meaning there is no centralized order book like in equity markets. It is less liquid, less transparent, and harder to gather comprehensive data. Many bonds trade infrequently, and transactions are sometimes unreported. The lack of centralized interactions further complicates using data-driven approaches. However, the market's size and significance present substantial opportunities for improvement.

Does this challenging environment make the credit market ripe for AI applications?

Absolutely. The aim is to make the most of the available data  in this market while addressing the limitations of standard machine learning approaches. In such a challenging regime, domain knowledge and mathematical models should be employed to describe certain aspects of the data, complemented by machine learning tools to capture aspects for which theoretical insights are absent. This requires a deeper understanding of the market and careful alignment of AI models to ensure they are both effective and practical.

What are the key challenges in developing AI models for the credit market beyond data availability?

Combining technological advancements with deep financial domain knowledge is critical. Ensuring that algorithms are trustworthy and have a solid mathematical foundation is equally important. This foundation helps diagnose and address issues when they arise, ensuring the models remain robust and reliable in practice.

Can you share an example of your recent work in this area? 

We recently developed a multitask dynamic pricing algorithm for the credit market. Since corporate bonds often have limited individual data, we leveraged the structural similarities among bonds from the same company or sector. Our method pools data to create a meta model, which is then fine-tuned for individual bonds. This approach addresses the small data problem effectively and represents some of our most recent research, completed just last month.

What is the ultimate goal of developing these AI models for the finance industry?

The overarching aim is to uncover the underlying information and structures in financial data. Traditional models work well in simpler systems but struggle with the complexity of real-world financial markets. AI can help us better understand and model these complexities, ultimately supporting improved decision-making and risk management.

Could you tell us about the recent AI and Finance conference you helped organize?

The International Conference on AI in Finance focuses on the promising role of AI in addressing challenges in our field. This is the first time the conference has been hosted by an Academic institution, so we consider it a huge honor for the school and our department. The community has grown rapidly, with doubled registrations and submissions this year compared to previous years. Talks and papers covered a variety of topics including decision-making, generative AI, and large language models for financial services. We’ve also seen interest from regulators in understanding the systemic impacts of AI-driven trading and potential collusions. So it’s a great opportunity to interact with stakeholders across the industry.

What inspired you to join NYU Tandon, and how do you see the department’s role in this field?

NYU Tandon’s proximity to Wall Street provides invaluable opportunities to collaborate with industry professionals, such as quantitative researchers, and to identify real-world challenges. Additionally, the department is rapidly growing, attracting leading figures in the field. Working with such talented colleagues fosters innovative research and solutions in AI and finance.

*These include general co-hairs Dhagash Mehta (BlackRock) and Guiling “Grace” Wang (New Jersey Institute of Technology); program co-chairs Senthil Kumar (Capital One) and Hao Ni (UCL); workshop co-chairs Bo An (Nanyang Technical University), Yongjae Lee (Ulsan National Institute of Science and Technology), and Zhen Zeng (JP Morgan); and senior advisor Tucker Balch (Goizueta Business School, Emory University).