Question

why does the poet say "the music in my heart i bore"?

Answer 1

because he was mesmerized by the girl's song and he could not forget that melodious song

Answer 2

The reaper's song touched the poet's heart. It was so melodious.Though it's dialects were not understood to poet, but it tune mesmerised him. The poet said these lines as when he left that place and moved forward, the tune of that song was still playing in his mind. It has a soothing, long lasting effect on him. He was not able to forget the song for a longer period of time.