I would like to use NLP techniques to implement a sockpuppet detector that would be used to identify people who post to forums using multiple accounts and trying to obscure the fact that they are not all the same person. I’m not sure how to pull this off though as I don’t see an obvious path for creating a system based on models on HuggingFace. I’ve done some looking and all I’ve found is sockpuppet detectors based on other factors not specifically analyzing the prose.
Ideally what I would be able to do is take an arbitrary number of posts inside a single topic and group them together based on writing style/word choice/other information present inside the content itself. Then I would be able to look at how the groups are organized and compare against the known account IDs tied to each post to get an estimate of who may be posting from more than one account.
I would like to avoid having to train the system about individual accounts so the system could be used out of the box on any forum.
This seems to be conceptually close to a semantic classifier however I don’t see a way to map semantic classification models to this specific domain with out resorting to something like mapping an individual account to a specific semantic output. I don’t see how such a solution could work anyway considering training the system would include the obfuscated identities of the posters as the training data would be blind to the sockpuppets existing.
My thoughts are that the individual posts could be sent through the encoding layer of something like Stable Diffusion and the latent space output could be analyzed for similarity to other posts though I’m not sure how such a similarity comparison could be performed. I know there are some models that can provide a distance between words such that the distance from dog to cat is less than the distance from cat to carrot; this seems to be highly related to what I want but also does not obviously map into what I’m trying to achieve.
The best concept I’ve managed to come up with so far would be to take two different posts that do not have labels associated with them (such as the account that created the post) and run each one through a RNN then use a binary classifier to output a “same author” decision. This is however massively non-optimal for the required task of trying to group together an arbitrary and possibly large number of different posts though at least there is an algorithm that could exist to do it using a lot of different comparisons.
I’d appreciate any thoughts or questions people can offer on how I could create a sockpuppet detector. Thank you!