Going to a gym with people that don’t understand you is kinda hard

I don’t like complaining about this to people, because I am very grateful that I can participate in martial arts classes, and I love it a lot — but I’m feeling very guilty and discouraged. I’m a small girl with chronic health issues, so coming to class is hard sometimes, but I try nonetheless and do work out and stay healthy so I can give my all (+ the fact that doing martial arts does help manage some of my symptoms). My coaches are both great, and do understand my health issues and give me good tips and enough breaks to keep going without getting too fatigued, but I’m having a much different experience with some of the people I partner with. The majority of my classmates are really respectful, but I still get crap from some who don’t get that I have issues and am going to be weaker than them in general just because of my conditions… Just today I had to work with a guy who visibly loathed working with me (could see it on his face, he made a disgusted face when we were paired up) and was barely trying with the drills we were doing, even when I tried to make small talk in-between rounds and attempted to be more loose and enthusiastic. Last week when my class was doing light sparring (something we all agree to have fun with to practice technique and reflexes, not going 100%) a guy repeatedly punched me very hard in the head and complained that I wasn’t trying hard enough, even when I explained that the heat that day was making me more fatigued than usual. I understand I can be a pain to work with, I’m very short compared to 99% of the people in my class and I don’t give a good fight some of the time, but at least I’m trying? I’ve improved since I started a year ago, and try my best to hold pads and stuff correctly so my partners can still train as best they can with the height difference, but I feel like maybe I should quit. Maybe I’m completely wrong for signing up in the first place and inconveniencing my classmates. IDK. any advice??