Most suitable IA model for lexical fields recognition

Beginners
1

Hello, I’m a beginner in the AI world, the furthest I’ve got in this field is implementing a linear regression model (which is not really AI) and fine-tuning a BEiT model for cat and dog recognition :smiley_cat::dog:
I started 5 days ago an intership of six months in a software company and I have the task to implement a software AI base that could help the commercial director to find the best software engineers and Business analysts in the company to fullfill the requirements of a new mission.
I’ve started to draw the big lines of the process:

Process:

  1. Mission Qualification: Identify the company, location, sector (banking, public service, craftsmanship, etc.), function (HR, accounting, sales, etc.), period, required technical skills, and language.
  2. Comparison of Requirements and Consultant Profiles: Compare the mission’s requirements and conditions with the consultants’ qualities. These qualities include place of residence, experience (seniority, sectors, functions, and companies they have previously worked with), availability, technical skills, spoken languages, and desired career progression.
  3. Weighting of Qualities Based on Client Preferences: Prioritize consultant qualities according to client preferences (e.g., if the client prefers a consultant they have previously worked with and who speaks Spanish, then “spoken languages” and “experience” will be prioritized in the selection process).

The technology should be able to fill both mission and software engineer qualifications fields evoked above (location, required skills etc.) into a sort o structured data way after reading the mission file and the competence file of the software engineers. Thus, it has to be able to have a wide understanding of the words it comes accross and simplify them into simpler word, e.g., “Bank”, “Financial institution”, “bank agency” should be in saved in the same keyword “Banking”. In other words, be aware of lexical fields.
Then the matching between the mission keyword and the software engineer keyword can operate.
I was wondering what kind of technology could do this ? Of course I could implement it with Python, but the goal here is to be efficient (I’m not in school anymore :smiling_face_with_tear:) . Thus, do you have other AI solution for this kind of task ? It ressembles much to a ATS process. Could I upload one on thos on hugging face ?
Thank you for your time.

1 Like
2

There is a way to leave everything to a single large LLM, but I think that many people will take an approach that combines multiple models when they are prioritizing cost-effectiveness and speed.

I tried having AI search for models in fields I don’t know much about. I think there are people on HF Discord who have experience tuning these models.

Summary of Proposed Models

Here’s a list of recommended models from the Hugging Face collection:

- FlauBERT for NER and multilingual processing.
- RoBERTa or Sentence-BERT for semantic similarity and keyword matching.
- XLM-RoBERTa for domain-specific fine-tuning.
- DistilBERT for text classification.
- TF-IDF for keyword extraction and prioritization.

By combining these tools, you can build an efficient and scalable system for mission qualification, consultant matching, and weighting based on client preferences.
3

Thank you so much @John6666. This is a great starting point.

1 Like