My vote for the cool SOC of the year (2003) is the Xilinx Virtex II Pro. It's a PPC 405 with a giant fpga. It's also a lot of IP which does all of the normal things like ethernet, serial, etc...
Maybe everyone knows this but looking over the interface docs I was having some major deja-vu from some work I did for IBM a few years back. It sure looks like the guts of the IP all came from IBM. Personally I like this because my impression is that IBM folks up in VT did a really nice job most of the time. (except for those odd PPC DCR registers - oh well, can't ask for everything)
Anyway, I've always been frustrated by this controller or that and always wanted to tweek things in the cpu/peripheral interface. Now it appears I can. We'll see... I plan to try.