L
How JP Morgan Built An AI Agent for Investment Research with LangGraph | LangChain Interrupt
@ LangChain
05/30/25
Takeaways
DO
David Odomirok
05/30/25
@ LangChain
At JP Morgan Private Bank, we curate and manage thousands of investment products, each supported by extensive data, which leads to inevitable questions that our small research team must address.
DO
David Odomirok
05/30/25
@ LangChain
We challenged ourselves to automate the investment research process to deliver precise and accurate results, transforming how we answer investment questions with our AI solution, Ask David.
DO
David Odomirok
05/30/25
@ LangChain
With Ask David, we're providing curated answers, insights, and analytics quickly, enhancing our efficiency without replacing our roles.
DO
David Odomirok
05/30/25
@ LangChain
The stakes are high in investment research, with billions of dollars at risk, and we are committed to exceeding stakeholder expectations with our AI tool.
ZX
Zheng Xue
05/30/25
@ LangChain
We utilize proprietary models and APIs to generate insights, allowing us to scale our services and provide timely information to clients during meetings.
ZX
Zheng Xue
05/30/25
@ LangChain
Ask David stands for Data Analytics Visualization Insights and Decision-making Assistant, designed to streamline the investment research process.
ZX
Zheng Xue
05/30/25
@ LangChain
Our multi-agent system starts with a supervisor agent that orchestrates tasks, understands user intentions, and delegates to specialized sub-agents, ensuring high accuracy and reliability.
ZX
Zheng Xue
05/30/25
@ LangChain
The structured data agent translates natural language into SQL queries or API calls, while the unstructured data agent processes various documentations and media to derive insights.
ZX
Zheng Xue
05/30/25
@ LangChain
For complex queries, we employ text-to-code generation capabilities with human supervision to ensure accurate execution of tasks.
L
LangChain Cast
05/30/25
@ LangChain
The multi-agent architecture includes supervisor agents, specialized sub-agents, and human-in-the-loop workflows to automate investment research.
L
LangChain Cast
05/30/25
@ LangChain
The agent personalizes answers based on user roles, providing detailed responses for due diligence specialists and general answers for advisors.
L
LangChain Cast
05/30/25
@ LangChain
The reflection node uses a language model to ensure the generated answer makes sense, and if it doesn't, the process is repeated until a valid answer is produced.
L
LangChain Cast
05/30/25
@ LangChain
The summarization node updates memory and returns the final answer after summarizing the conversation.
L
LangChain Cast
05/30/25
@ LangChain
Starting simple and refactoring often is crucial; the initial version was a plain vanilla ReAct agent that evolved into a more complex system through iterations.
L
LangChain Cast
05/30/25
@ LangChain
An evaluation-driven development approach is essential, focusing on metrics to ensure continuous improvement, especially in accuracy for financial applications.
L
LangChain Cast
05/30/25
@ LangChain
Independently evaluating sub-agents helps identify weak links and improve overall accuracy in the system.
L
LangChain Cast
05/30/25
@ LangChain
Choosing the right metrics for evaluation is vital; for summarization, conciseness is a key metric to assess.
L
LangChain Cast
05/30/25
@ LangChain
Using large language models as judges, combined with human review, allows for scaling without overwhelming human subject matter experts.
L
LangChain Cast
05/30/25
@ LangChain
The last mile of achieving 100% accuracy is the hardest, often requiring human intervention to address inaccuracies in AI applications.
L
LangChain Cast
05/30/25
@ LangChain
Incorporating human oversight is critical, especially in high-stakes environments, to ensure accuracy and reliability in AI-generated responses.
L
LangChain Cast
05/30/25
@ LangChain
Key takeaways include iterating quickly, evaluating early, and maintaining human involvement in the process to ensure quality outcomes.