Do babies go to heaven or hell when they die?

Although the scriptures do not specifically answer this question, there are implicit references to help with the answer. God told Jeremiah that he knew him from the womb, but the most convincing argument is in 2 Samuel.

King David has just completed his fasting after God told him that his son from Bathsheba would die. David fasted in prayer that God would not take his son. When the child died, David ended his fast. He was criticized and responded with the verse in 2 Samuel 12:23:

"But now he is dead, wherefore should I fast? Can I bring him back again? I SHALL GO TO HIM, but he shall not return to me."

Clearly, from the Psalms, David did not expect to go to hell. He spoke often about his salvation being in the Lord. Consequently, since David trusted in his salvation and said that he shall go to his son, he expected his son to receive the same salvation. God did not contradict or counter this belief, either directly or through the prophets.