The premise that Taki and Mitsuha did not notice the three year jump every day relies on the fact that neither of them know the date of any given day. While most stories with this premise can handwave it by having the characters never look at a calendar, Taki and Mitsuha explicitly look at their phones every day, which should give them the date.
Most phones only show the date as the day and the month, not the year.
But they also show the day of the week which would cue on the year being off.
One possible explanation, that is quickly glossed over in the movie, is that they simply don’t remember the dates. Mitsuha mentions that their memories of their experiences as when they’re in each other’s bodies are hazy and get hazier, which is one of the reasons they keep the journal entries. But this still leaves the question of how they don’t seem to be concerned or even notice the different years in their environment, such as at school, calendars, news stations, etc.
I interpreted this as both having Weirdness Censor where they don't realize anything is amiss. My only evidence to this is that despite being in different bodies, they were able to navigate through life without questioning their own actions and the cell phone data getting scrambled while Taki is forgetting.
Mitsuha has an iPhone 5, and Taki has an iPhone 6. At the time of the film's release, the iPhone 6 was the latest iteration of the the iPhone, and three years prior in Mitsuha's timeline, the iPhone 5 was the latest iteration. This is an early hint at the two being separated by three years.
Taki while in Mitsuha's body never bothered with the hairstyle. The few times we see him in Mitsuha's body, the hair style is just a simple ponytail using a band. He simply could not do anything more complex, let alone the multi-layered insanity that is Mitsuha's normal hairstyle. And thus, he never found Mitsuha's hair ribbon, which she will give to him and become his bracelet.
Meanwhile his female counterpart has a habit of twirling the hair often.
Taki takes down his bracelet every night when he goes to sleep, and puts them back on when he wakes up. When Mitsuha is in his body, she didn't know about the bracelet and goes around with a bare wrist. Mitsuha was thus not aware that it's her hair ribbon and her relationship with Taki.
One moment during body swapping montages, Mitsuha and Taki insult each other by writing Baka and Aho on their counterpart's faces. Both mean 'idiot', but Baka is used mostly by those in central region such as Tokyo, while Aho is from outlying Kansai area. In one scene, there was a throwaway line discussing different dialects.
On the day Taki!Mitsuha delivers the kuchikamizake to the mountain crater, s/he wore a school uniform when s/he woke up, even though it wasn't a school day, to which Yotsuha commented on. While it looks like it's Taki having no idea of countryside things again, it is actually caused by the three years of time displacement - the weekdays are not in sync, so Taki thought that it was a school day (in 2016 Tokyo) when it wasn't (in 2013 Itomori).
The valley itself is another bit of foreshadowing of the impending disaster: the mountain crater itself, with the shrine god at the center. It's an impact crater, and the god itself was a previous fallen meteorite.
After reading Another Side: Earthbound, it becomes apparent that Mitsuha's issues aren't merely due to being belittled by her classmates and father, but also due to the massive shoes Futaba left for her to fill that she probably doesn't think she can live up to.
We see early on that Mitsuha doesn't wear a bra when she sleeps, and then we can see that she starts wearing one to bed. This is likely to solve the problems of not wanting Taki to touch her breasts and Taki not knowing how to or remembering to put on a bra.
The film opens with a flash forward of adult Mitsuha and Taki talking about how they feel like they've lost something. The film opens this way because just like us, the audience, they don't know what it is they lost.
As this Reddit comment notes, the series-style opening can be easily glossed over by a viewer who won't initially recognise it's a Spoiler Opening until a Rewatch. This fits perfectly with the film's theme of forgetting important things.
The Insert Song "Nandemonaiya" repeats the lyric "Just a little more" when it starts playing at the end of the film. Fitting, considering the audience is wondering just how or when this movie will end...
Why do Taki and Mitsuha lose their memories of each other? That's the price they need to pay to the god for entering his domain and failing to leave a proper offering behind.
The body of the god is located in a perfectly round caldera. Likely enough, Japan is full of volcanic activity. With the reveal that Itomori has been struck by a meteor once before, and is hit again during the movie's events, it becomes much more likely that the caldera was formed in an earlier meteor strike.
Fridge Horror
Just imagine what would happen if Taki and Mitsuha's Body Swap still goes on during the day when the meteorite falls and destroys Itomori. Taki would have been killed while he's in Mitsuha's body, while Mitsuha would've most likely been stuck in Taki's body three years into the future forever, with everything she knew and loved gone...
Possibly not. We see at least two occasions where Taki and Mitsuha switch while awake. Both of them relating to twilight, and we know that both have been awake into the night. It's likely that the connection would simply have broke with Taki waking up in his own body.
That brings up some different horror; Considering how Taki's memories of Mitsuha began to fade as soon as he found out that Itomori was destroyed and it took quite some time for him to find Itomori to begin with, had he been in Mutsuha's body when she died, would he have remembered her long enough to be able to go and save the town's residents?
In the ending the comet ends up destroying Itomori, meaning that the former lives of townsfolk there, including Mitsuha's family traditions, pretty much disappeared.
Then again, it's implied that it's what happened 200 years ago in Great Fire of Mayugoro. The tradition maybe will just "continue the form but lost its meaning".
the series-style opening can be easily glossed over by a viewer who won't initially recognise it's a Spoiler Opening until a Rewatch. This fits perfectly with the film's theme of forgetting important things.