Peter uses two verbs to describe their commitment: (1) “believe” and (2) “know.” These two words are synonymous at times, but it is unnecessary to simply view their union as synonymous parallelism for the sake of emphasis (i.e., “we have really, really, really believed that you are the Holy One of God”). They share the personal pronoun ἡμεῖς (“we”). But the use of the first verb (“we have believed”) reaches back to what is expressed in v. 68 and vv. 47–51. In 6:47 the substantival particple ὁ πιστεύων (“he who believes”) lacks an object. By the way, some scribes saw a little ambiguity in that expression in that context and added the phrase εἰς τὸν θεόν (“in God”) to clear things up. They missed the point when they added those words though. The emphasis is on Jesus wanting them to believe the part of his message that follows in vv. 48–51. It’s connected to his deity, sure, but the participle is connected to the message. Its implied object is τοῖς ῥήμασιν (“in the words”). That draws a more obvious connection to Peter’s response in 6:68, which also contains a reference to “eternal life” (the object of the verb in 6:47).
The second verb more directly corresponds to the ὅτι clause that follows. Peter says that they’ve not only already believed the lesson that Jesus taught in 6:48–51, but they’ve actually begun to understand more fully that Jesus is the one referred to in the Scriptures, albeit on rare occasion, as the “Holy One.” It appears here in Peter’s confession as “the Holy One of God” (ὁ ἅγιος τοῦ θεοῦ, 6:69). The closest Old Testament referent, the one to which Peter most likely referred, is found in Psalm 16:
"I have set the LORD continually before me. Because he is at my right hand, I will not be shaken. Therefore, my heart is glad and my glory rejoices. My flesh also will dwell securely. For you will not abandon my soul to Sheol. Nor will you allow your Holy One to undergo decay. You will make known to me the path of life. In your presence is fullness of joy. In your right hand there are pleasures forever.” (Ps. 16:8–10).This seems like the specific referent since this section of Psalm 16 is specifically quoted by Peter at Pentecost. Introducing the quote, Peter says, “For David says of him . . .” (Acts 2:25), that is, specifically Jesus. And there, in place of “Holy One of God,” we find “your Holy One.” The use of Psalm 16 in Acts is clearly because of its connection to the resurrection, and Peter understood “will not undergo decay” as evidence for the resurrection. By the way, Paul did too (Acts 13:35). This is too early in Jesus’ ministry, however, for the disciples to really grasp the whole issue of Jesus’ resurrection. Up to this point in his ministry, Jesus has not been teaching his disciples openly about his approaching suffering, death, and resurrection. That comes later. Jesus speaks openly about these matters for the first time in Caesarea Philippi, following Peter’s next confession (Matt. 16:21).
Well, you notice that I've left off this attribution by the demon-possessed man from the beginning of Jesus' ministry. True, Peter could have heard it used in that context and picked it up and started using it. But it seems more likely that Peter has a sort of draw to Psalm 16, perhaps from hearing it taught in school or read in the synagogue. Little did he know, though, this verse wasn't teaching that the Messiah would never die . . . just that his body wouldn't lie in the grave very long at all. Peter's confession is accurate in John 6. He is the Holy One. He'll just come to understand more and more about that passage following the crucifixion and resurrection of Jesus.