Separating people on rickshaw, bike with pedestrian

My current classification model accurately classifies 25 native and general classes on the street. But the problem comes when classifying people and people on rickshaws, bikes, etc. All classified as person. I trained an SVM for further classification, but the testing result wasn’t good enough. I can train a neural network, but first, I was wondering if there is another way to solve this problem—a mathematical solution or a lightweight context-based solution. (rickshaw is a tri-cycle like vehicle)
2024-09-25_10:10:58:450_Location15_Cam1_101_18213_1_UPP

2024-09-25_17:04:45:800_Location15_Cam1_173_7323_1_UPP