In my Father’s house are many rooms (or mansions, KJV). If it were not so, would I have told you that I go to prepare a place for you? And if I go and prepare a place for you, I will come again, and will take you to myself, that where I am you may be also. (John 14:2-3)
These verses don’t say that God’s house is in heaven — in fact, every reference to God’s house in Scripture is to his house on earth. We are told, in 1 Peter 2:5, that it is the true Christian church that is the house of God. It is not a building, but the group of people who truly worship him in their heart. At the present time, this house of worship is being built on the foundation of Jesus Christ and the Apostles (Ephesians 2:19-22). Jesus says that he will return to the earth so that his followers may be with him.
The Greek word that is translated rooms or mansions in verse 2 occurs in only one other place in Scripture, in verse 23, where it is translated as home or abode. Jesus says, “If anyone loves me, he will keep my word, and my Father will love him, and we will come to him and make our home with him” (John 14:23). It is clear that Jesus is not teaching that people will go to heaven to be with God, but that God will come to them and be with them.