Part of it was the video mode. EGA 640x350x16 had 16 simultanous colors, from a palette of 64 possibilities. And non square pixels as a bonus.
They might have made better choices from the palette, but the limitations were severe.
If you really want to stab your eyes out, CGA had a mode with white, bright pink, light blue and black. I remember playing Keen on it. I've never seen that mode used for anything nice.
The CGA colour palette was horrid on RGB monitors but on Composite it could look quite good for its time. Not Amiga-level good but good enough. Of course composite monitors were not really a thing in the IBM PC world so it was to little avail. Here's an example of a screen in the mentioned horrible hot-pink palette:
They might have made better choices from the palette, but the limitations were severe.
If you really want to stab your eyes out, CGA had a mode with white, bright pink, light blue and black. I remember playing Keen on it. I've never seen that mode used for anything nice.