Case study

Industry
Banking and financial services
Platforms
Elastic, LangChain, Splunk
Duration
About 6 months
specialized agents shipped
delivery than before the refactor
maintainable base on Elastic and LangChain
The challenge
A large US bank was building a smart SRE assistant to take heavy production tasks off its engineers. It used LangChain as the agent framework and Elastic as the data store and retrieval (RAG) system.
Development was slow. The team lacked experience in agent programming and in Elastic. The agents ran as APIs wrapped in Python code that was hard to maintain and scale, and loosely defined Elasticsearch indices caused compatibility issues.
What we found
A review of the architecture, then of each feature’s code, showed the issue was structural: one agent did everything, and the code around it was hard to read and change.
What we did
Refactored the existing code onto an open-source agent framework, for more readable and maintainable code.
Designed the full agent layer, exposed through an API to the main application layer.
Designed the Elasticsearch indices for both vector and lexical search.
Replaced the single do-everything agent with specialized agents.
Set up continuous testing and an evaluation (evals) approach.
Results
9 agents shipped, on a faster timeline than before the refactor.
Specialized agents instead of one agent doing everything.
Less time spent debugging, thanks to cleaner, more maintainable code.
What it means for the client
The bank has a maintainable base to keep adding agents. The recommended next step is a guardrails framework and further testing frameworks.
Next case study


