I bought the XBone Day 1 because of the Killer Instinct reboot. To me, there was just as much hype behind it as there was for a Banjo-Kazooie 3. And the hype has definitely lived up to my expectations. KI 3 is absolutely amazing in so many ways and the game just keeps improving.
For me, that alone was worth the XBone as my main console. The Rare Replay was just icing on the cake. But I thoroughly enjoyed replaying BK, BT, JFG, and PD. I also never got a chance to play Blast Corps (amazing game), PD0, Viva Pinata (very enjoyable), Kameo, Ghoulies, Nuts and Bolts, and most of those older arcade games. Also the original KI is bundled with KI 3. I just don't see how the Xbox one isn't the definitive go-to console for any Rare fan. I personally moved to Xbox long ago whenever C: L&R first launched with the original Xbox. All my friends played this system anyways, and the interfaces of it feel great to me most of the time. (They occasionally experiment with the interface and things feel awkward).
As I look through my game library, nothing besides KI 3 (not technically a Rare game but it feels like KI Gold but better) and Rare Replay are Xbox exclusive. But Rare Replay has a lot to offer, and I've gotten even more hours out of KI 3. I guess it comes down to just how much of a Killer Instinct fan you are. And my games library doesn't look particularly "dudebro" at all. I don't know where that stereotype comes from because it seems like most of the games like CoD and Battlefield are multi-platform.
One other thing, I'm not sure if PS4 or Wii does this, but Xbox has a "games with gold" thing. It lets Gold subscription members download certain games for free. Lately I've been playing a lot of Peggle, Super Meat Boy, and Goat Simulator which were the recently featured games.
Edit: Just saw that you never played KI before. I guess it comes down to whether you're willing to take the chance that Rare doesn't produce anything great in the next few years for the xbone. You seem to have an interest in some of the PS4 exclusives, so it seems like that may be the system for you outside Replay, SoT, and anything in the future Rare makes. I personally have absolutely no interest in anything that's on the PS4 so I don't really have that weighing me down. I'm not really a Nintendo fan; solely a Rare fan. I think the last good game Nintendo made for a console was SSBM. The only thing I touch from Nintendo now is Fire Emblem and occasionally Pokemon, which are both on DS.