Seeking Advice: Training a Novel Review Assistant Model for Web Fiction

Beginners
1

Hello Hugging-face community,

I work as an editor for web novels and I’m looking to developing an AI assistant to help with my daily review tasks. I hope to get some advice on model selection and training approaches.

My Use Case

As an editor, I need to review new chapter updates to ensure they:

  • Have no grammatical errors

  • Comply with regulations and laws

  • Don’t promote inappropriate values or content.

I have a substantial dataset of novel passages with my review annotations and suggested modifications from my previous work.

My Plan

I’m considering fine-tuning a base model through Supervised Fine-Tuning (SFT) to create a specialized model for web novel content review that can help improve my workflow efficiency.

Questions

  1. Is this approach feasible given my use case?

  2. Which base models would you recommend for Chinese text review tasks?

  3. Any suggestions on training methodology or alternative approaches?

  4. How much data would I likely need for effective fine-tuning?

  5. Any particular challenges I should be aware of when training for content review tasks?

I’d greatly appreciate any insights or recommendations from the community!

Thank you!

2 Likes
2

I don’t know anything about the training itself, so I’ll leave it to someone else.

Is this approach feasible given my use case?

I think so. In cases where the performance of the execution environment is extremely limited, it may be necessary to save resources by inserting lightweight normal programs or text classification models, etc. However, if it is possible to do with just one adjusted LLM and a simple agent, I think that is more reliable and easier.

Which base models would you recommend for Chinese text review tasks?

You can get a relatively accurate idea of performance for various tasks by looking at benchmarks such as the leaderboard, but to put it crudely, I recommend the Qwen 2.5 series. The basic education is really good, and it listens to commands relatively well.
DeepSeek is also good, of course, but the Qwen series has been trained by users on Hugging Face for a long time, so there are various variations on the hub, and there must be educational know-how on the internet. Perhaps there is a lot of information, especially if you search in Chinese. I can’t read it without translation…
I think it would be helpful to look for the Qwen that has been trained by everyone on the hub.