Banjo-Kazooie. The levels in Tooie were too big and a tad empty (to me), and there actually was one less of them. Also I was really disappointed that Gruntilda didn't rhyme anymore (nitpick, I know). And while it was great that Tooie had bosses for each level (the major advantage it has over the first game imo) the final battle was a real let-down. Have I really anticipated a fight with those three witches just to battle a tank?
My personal preferences also tend towards gloomy or sinister surroundings, which there were more of in the first game: Clanker's Cavern, Mad Monster Mansion and Rusty Bucket Bay opposed to Witchy World and Grunty Industries. Gruntilda's Lair also fit the bill better than Isle o' Hags. But yeah, here we're getting really subjective. It's interesting that bjaxx87 seems to have the opposite impression on this particular point (or not?).