Hey, so I’m a Software Engineering student in my last year who’s currently working on the final project.
My final project is, in short, video games cheat detection system. Simply, the system will get a gameplay video/image as an input, and then will rule if it detected any cheats as an output.
The system is supposed to use computer vision in order to identify some of the cheats. For example, ESP cheats:
(you can see the frame layout and HP bar around enemy player characters)
I’m trying to use YOLOv5 in order to identify those, but I’m having hard time. I used Roboflow in order to classify a few different classes, and around 300 images, most of them containing more than one class and more than one appearance of that class.
I can’t show you screenshots from Roboflow (due to website restriction) to show you how I classified those elements inside images, so if you’d like to know, tell me how to do it please. Maybe I could add a comment to this post with said screenshots if that’s required.
Anyway, I divided the dataset created by Roboflow to train/val/test and trained YOLOv5 with the dataset, but unfortunately it doesn’t work as expected. It just doesn’t (or barely) identifies..
Can I have any tips/walkthrough on how I should proceed?
It looks like it might slip through the cracks due to changes in the GUI of the cheat tool, but let’s not think about that for now…
There seem to be a few previous examples, but it doesn’t seem to be the same use, and it doesn’t seem to be the same as the training of the newer open source vision model.
When training a vision model effectively, you often need a huge data set, but relying on sheer numbers is a last resort. The way you create the data can also make a big difference to the learning efficiency. If the target has easily recognizable features, it’s good if you can make them easy for the model to understand. In this case, the player’s shadow and the HP bar on the side would be easy to recognize.
If you take screenshots while turning the cheat tool on and off, and then use Python to extract the differences, you might be able to generate them automatically to a certain extent.
Anyway, the characteristics of each cheat method will vary, so it’s a difficult project…
Creating a cheat detection system using YOLOv5 for ESP cheats involves overcoming several challenges. Here’s a structured approach to enhance the detection accuracy based on the identified issues:
Data Collection and Augmentation:
Increase Dataset Size: Expand the dataset beyond 300 images, including diverse scenarios, lighting conditions, and cheat indicators.
Data Augmentation: Apply techniques like rotation, scaling, and flipping to increase variety and help the model generalize better.
Labeling and Class Simplification:
Simplify Classes: Focus initially on a single class, such as the ESP indicator, to improve detection accuracy before introducing more complex classes.
Accurate Annotation: Ensure that labels are precise, especially for small or subtle indicators, using tools like Roboflow or LabelImg.
Model Choices and Architecture:
Experiment with Models: Consider using YOLOv8 or other models optimized for similar tasks to see if they perform better than YOLOv5.
Transfer Learning: Utilize pre-trained models on related tasks to leverage learned features, potentially improving performance.
Dataset Splitting and Fine-Tuning:
Balanced Splits: Ensure the training, validation, and test sets are balanced to avoid biases, especially in validation where the model tests generalization.
Hyperparameter Tuning: Adjust learning rates, batch sizes, and other parameters to optimize training.
Preprocessing and Feature Isolation:
Image Preprocessing: Preprocess images to highlight cheat indicators, such as enhancing contrast around potential ESP markers.
Feature Extraction: Use techniques like edge detection or filtering to isolate features that distinguish cheat indicators from other objects.
Testing and Validation:
Iterative Testing: Test the model on single classes first, then gradually introduce complexity.
Error Analysis: Visualize model predictions to identify common failures, whether in detection, classification, or localization.
Resource Utilization:
Compute Resources: Ensure sufficient hardware, considering that larger datasets or complex models may require more powerful GPUs for training.
By addressing these areas, you can enhance the model’s ability to detect ESP cheats effectively, ensuring a robust and accurate detection system.
