I actually took almost all the classes necessary to do that. I could even go further, I took classes that would allow me to design the processor on individual transistor level creating a design which could theoretically be built on silicon. But of course the classes were separate so I did not build upon everything.

The problem with that is that encompasses what have become two disciplines -- computer science, and electrical or computer engineering. I took classes encompassing both sides because I liked the classes, but most people are really interested in one or the other.