No one should be restricted from their friends or family. To me this says that he does not value you as a person, but sees you as a trophy that he wants to keep to himself. If you had a daughter who's boyfriend treats her the way your boyfriend treats you, you can't say that warning signs wouldn't be flashing.
You don't have to be in a relationship. I'm sure you have the ability to make friends on your own, it's just uncomfortable to step out and introduce yourself to people. I have faith you can do it! If you remain with him, I see you either becoming a shell of your former self and being depressed or worse. Please do the right thing.
