Looking for advice on how to approach a program of generating automatic feedback reports

Community Calls
1

I am looking for advice on how to handle a problem. I have been tasked with generating custom reports to essentially employee 360s.

We have 5 questions - qualitative in nature. We give the 5 questions out to 10 people. We end up with 50 comments.

I need to then automatically develop a report for the 50 comments. I considered pattern-recognition and having a pre-built library of text, but the problem is that it won’t be flexible and it will end up missing good content and feedback.

I considered topic modeling followed by topic labeling and then using bart to summarize, but I think the report iwll end up lacking structure.

Now, I am just lost. I have no one to ask. Please give me advice.

2

I would suggest going with your second approach - try Bertopic perhaps. You will always miss what you may believe is “good content” without sufficient training data - specially if the feedback contains text and language very specific to your domain. But that may be good enough for 360 feedbacks.

Take it one step at a time.

  1. See if topic labeling gives you relevant clusters. Employee feedback reports tend to be smaller datasets, and depending on demographics the topics can vary significantly from report to report.

  2. if your dataset is small, which typically 360s are, stop there. Show the stats and then link to the reports themselves for further review.

  3. You can optionally consider Summarization, but be careful at this step unless the feedback is using very generic language.

  4. Optionally if your company is interested to know if the feedback is aligned to certain company goals and or leadership principles, you can look at doing a zero shot classification or similar approach to see how closely each 360 lines up to a principle or a goal. This is similar to above - but forces structure.

  5. if your 360s are based on specific questions - you can do sentiment analysis on each question topic to see if the employees are positive or negative on each topic.