I have a Throne of Glass theory.
Dorian's eyes and hair are different from both the King and Queen's. The King of Adarlan always refers to him as "boy" or "Prince" and never as his son. According to the genealogy charts Dorian looked through, no magic users married into the Havilliard bloodline for thousands of years.
It's obvious.
Dorian is not the King's biological son.