The opening line: "Proper story’s supposed to start at the beginning. Ain’t so simple with this one." Due to the "Groundhog Day" Loop, you're at the beginning of the story. And at the end of the story. And in between loops.
The part where Rucks says the Kid hears a song, asks something to the effect of "How'd that go, again?" and then Build That Wall starts playing actually isn't breaking the fourth wall and makes perfect sense, since it turns out Zia was sitting right in front of him listening to him talk, and the music starting is actually Zia playing the music to Rucks, not her starting to sing it at the campsite.
The final level is Tazal Terminals. Despite long being established as an underground structure, when you get there, the Terminals are overflowing with snow. Why? Well, because the Calamity threw the Terminals up higher than it did anything else! Altitudes!
Zulf's theme (Mother, I'm Here). The song has quite an impact. However, play through the Kid's Who Knows Where, and you'll find that that backstory basically tells the same story as Zulf's theme. Coming home to an empty house, holding his mother's hand only to find out she is no longer with him. This makes the ending where the Kid decides to rescue Zulf all the more poignant. Note the ending song's refrain is "Mother,I'm Here." "We all are born from the Lorn Mother, and in the end, we all return to her." It's a song about dying and returning to Micia, the "Mother".
The ending:
It is heavily implied in a New Game Plus that restoring the world to before the Calamity sets the world into a "Groundhog Day" Loop, with possibly thousands of iterations. This makes the events and odd narration of the Mushroom Samba episode go from simple survivor's guilt to the Kid's subconscious rebelling against the loop. By resetting the world, the Kid is literally responsible for that particular Calamity.
The autosave and inability to go back can lead to some moments of Guide Dang It! but actually helps the theme. Just like Rucks, you do want to replay the game and do better each time, and the Kid, just like you, is implied to remember a little more each time. Followed to its conclusion, it implies that even if the Calamity keeps on repeating itself, the Kid keeps improving and eventually Rucks' solution might actually become permanent and save everyone. On the other hand, the player might end up just as shackled by the idea of a hundred percent completion and perfection that they might become little better than Rucks in the process.
If the Kid remembers a bit more each iteration, that would handily explain why he's such a badass at a young age, and a master of so many weapons and skills. He's not resetting things to the time of the Calamity, he's going back years to before his mother died...and failing to save her and Caelondia each and every time.
Rucks' line about Squirts getting territorial over cores makes much more sense when you find out they're made of what are essentially millions of Squirt eggs.
The Ura are pale because they live in holes, while the Caels are ruddy because they live outdoors.
In the Stranger's Dream, K stands for "Kid" and "a kid like you can do anything". It shows how much faith Rucks has in the Kid despite never asking for his real name.
In the first track of Bastion's OST album, Rucks is about to get to point two of three things he missed in the world before the Calamity happened when he stops himself. Given that the entire game Rucks is telling stories to Zia while they're both waiting for the Kid to finish his errands, one can suspect point two is either about the Caelondia-Ura conflict or, to be more amusing, adult talk.
The final level. The Kid slaughters Ura soldier after Ura soldier on his way to reclaiming the last Shard. It gets worse when he picks up a Battering Ram - the same weapon the Ura used in attempts to breach Caelondia's walls - and uses it to slaughter even more Ura. If Zulf isn't saved by the Kid at the end, its quite possible the Kid has killed every last Ura remaining on the planet by the time he gets back to the Bastion. The Kid is a One-Man Army and has succeeded where the Calamity failed. The wish of Caelondia's Mancers has been fulfilled.
It's possible that the Ura only turned on Zulf because they believed he had led the Kid right to them. He'd spent years in Caelondia, so what's to say he hadn't grown loyal to them and allowed the siege as a ruse? Whether you save him or not, the fact that his people, who he'd betrayed his friends for, turned on him and basically beat him to death in the street is an awful mix of fridge horror and tearjerker.
That also makes it partially your fault, albeit unintentionally. They wouldn't have turned on him if the Kid hadn't shown up and slaughtered hundreds of them.
The Mercy, one of the memorials, is completed by killing things.
