Extracting structured client needs from free-text advisor notes to select and personalize an investment proposal — how would you approach it?

Research
1

Hi everyone,

I’m working on a project for a bank and I’d like to hear how the community would tackle it. I’ll describe the scenario and constraints; I’m deliberately not sharing our current approach, so I don’t bias the answers.

The scenario

  • A financial advisor meets a client and writes up meeting notes afterwards.
  • The notes are unstructured natural language, in Italian. Each advisor writes differently: some are short and telegraphic, others narrative.
  • The notes can contain facts, personal and financial information, needs, goals, time horizons, attitude towards risk, preferences and explicit requests from the client — but nothing guarantees that all of them are present, and some are only implied.
  • From these notes, the system must:
    1. understand and extract the concepts relevant to the decision;
    2. select an investment line from a predefined catalog of lines;
    3. personalize the selected line based on the requests that emerged in the meeting.
  • Today, selection, personalization and drafting of the proposal are all done manually.

A (fictional) example of the input

Original notes:

“Client, 52, executive, wants to put something aside for retirement but also keep liquidity for his son’s university in 3 years. He had a bad experience with equities in 2022 and doesn’t want to go through similar losses again, but he says he ‘can take some risk’. He doesn’t want to invest in companies in the weapons sector.”

Notice the tensions: two goals with different horizons, contradictory signals about risk, an exclusion request, and nothing about income or existing assets.

What I’d like to ask the community

  1. Overall architecture: how would you split the problem between ML/LLM components and other logic? Where would you draw the line on what a model is allowed to decide?
  2. Information extraction: which techniques or models would you use to get reliable structured output from noisy text? How would you handle implicit information, contradictions, and missing facts?
  3. Evaluation: with no labeled data to start with, how would you build a test set, which metrics would you track, and how would you monitor quality once in production?
  4. Reliability and audit: how would you detect when the system is wrong or unsure, and how would you make every output traceable to the source text?
  5. Models: any open models (or fine-tuning strategies) you’d recommend in this kind of task, especially with an on-premise deployment in mind?

Pointers to papers, datasets, libraries or Spaces are very welcome too.

Thanks in advance!

2

I’d advise against an embedding-based approach here because it’s too opaque and lacks precision. A better path is to establish a taxonomy first—such as purchasing power and risk tier—derive these tags from the text, and then drive recommendations based on them.

3

The LLM handles understanding, the rule engine enforces eligibility and constraints, and the LLM ultimately provides the explanation—not the decision.

4

I would like to manage the note as the input for the LLM. Afterwarda I want to ger the JSON from the note and it has to describe the customer need. Do you have any suggestion about any techinc to make this task more efficient?

5

agree with Dao on keeping the decision in rules. for the extraction step itself, what worked best for me:

define the JSON schema first and make every field nullable, with an explicit unknown. then fill it with structured outputs (json schema mode) so you never parse free text. the important bit is that each field also returns the exact quote from the note it came from. if the model can’t quote it, the field stays null instead of guessed, and the advisor can see why the system thinks the client is risk averse

for implied things like risk attitude, keep them as a separate inferred field with the quote and a low, medium or high confidence, so the rule engine can ask the advisor instead of acting on a guess

and before tuning prompts, hand label 50 to 100 real notes and measure precision and recall per field. telegraphic and narrative advisors fail differently and you only see that with per field numbers. italian is fine for current models, but check dates and amounts, that’s where i see most extraction errors