It is important to study and consider the writings of Christians who came before us. Especially those of the Early Church Fathers. Jesus taught that hell itself would not overcome the Church (Matt 16:18), therefore its fair to reason that true Christian teachings should be found throughout Church history.
In order to find out where they land on this I have assembled a large collection of Patristic quotations that I originally found HERE. Though I am borrowing the source from that book all analysis is my own.
“After the foregoing instructions, baptize in the name of the Father, and of the Son, and of the Holy Spirit, in living [running] water. If you have no living water, then baptize in other water, and if you are not able in cold, then in warm. If you have neither, pour water three times on the head, in the name of the Father, and of the Son, and of the Holy Spirit.” —Didache 7:1 Date: 70 A.D.
“Regarding [baptism], we have the evidence of Scripture that Israel would refuse to accept the washing which confers the remission of sins and would set up a substitution of their own instead [Ps. 1:3–6]. Observe there how he describes both the water and the cross in the same figure. His meaning is, ‘Blessed are those who go down into the water with their hopes set on the cross.’ Here he is saying that after we have stepped down into the water, burdened with sin and defilement, we come up out of it bearing fruit, with reverence in our hearts and the hope of Jesus in our souls” —Barnabas, Letter of Barnabas 11:1–10 Date: 74 A.D.
“I have heard, sir,’ said I [to the Shepherd], ‘from some teacher, that there is no other repentance except that which took place when we went down into the water and obtained the remission of our former sins.’ He said to me, ‘You have heard rightly, for so it is” —Hermas, The Shepherd 4:3:1–2 Date: 80 A.D.
“As many as are persuaded and believe that what we [Christians] teach and say is true, and undertake to be able to live accordingly, and instructed to pray and to entreat God with fasting, for the remission of their sins that are past, we pray and fast with them. Then they are brought by us where there is water and are regenerated in the same manner in which we were ourselves regenerated. For, in the name of God, the Father . . . and of our Savior Jesus Christ, and of the Holy Spirit, they then receive the washing with water. For Christ also said, ‘Unless you are born again, you shall not enter into the kingdom of heaven”—Justin Martr, First Apology 61 Date: 151 A.D.
“For since you have read, O Trypho, as you yourself admitted, the doctrines taught by our Saviour, I do not think that I have done foolishly in adding some short utterances of His to the prophetic statements. Wash therefore, and be now clean, and put away iniquity from your souls, as God bids you be washed in this laver, and be circumcised with the true circumcision… The command of circumcision, again, bidding [them] always circumcise the children on the eighth day, was a type of the true circumcision, by which we are circumcised from deceit and iniquity through Him who rose from the dead on the first day after the Sabbath, [namely through] our Lord Jesus Christ”—Justin Martyr, Dialogue of Justin with Trypho, a Jew 18 & 41 Date: 160 A.D.
“our Christ, by being crucified on the tree, and by purifying [us] with water, has redeemed us, though plunged in the direst offenses which we have committed, and has made [us] a house of prayer and adoration… “the mystery of saved men appeared in the deluge. For righteous Noah, along with the other mortals at the deluge, i.e., with his own wife, his three sons and their wives, being eight in number, were a symbol of the eighth day, wherein Christ appeared when He rose from the dead, for ever the first in power. For Christ, being the first-born of every creature, became again the chief of another race regenerated by Himself through water, and faith, and wood, containing the mystery of the cross; even as Noah was saved by wood when he rode over the waters …I mean, that by water, faith, and wood, those who are afore-prepared, and who repent of the sins which they have committed, shall escape from the impending judgment of God.” —Justin Martyr, Dialogue of Justin with Trypho, a Jew 18 & 41 Date: 160 A.D.
“Then said Jesus unto them, ‘I have been given all authority in heaven and earth; and as my Father has sent me, so I also send you. Go now into all the world, and preach my gospel in all the creation; and teach all the peoples, and baptize them in the name of the Father and the Son and the Holy Spirit; and teach them to keep all whatsoever I commanded you: and lo, I am with you all the days, unto the end of the world”—Tatian the Syrian, The Diatesseron 55 Date: 170 A.D.
“Moreover, those things which were created from the waters were blessed by God, so that this might also be a sign that men would at a future time receive repentance and remission of sins through water and the bath of regeneration—all who proceed to the truth and are born again and receive a blessing from God”—Tehophilus of Antioch, To Autolycus 12:16 Date: 181 A.D.
“And when we come to refute them [the gnostic], we shall show in its fitting- place, that this class of men have been instigated by Satan to a denial of that baptism which is regeneration to God, and thus to a renunciation of the whole [Christian] faith…For the baptism instituted by the visible Jesus was for the remission of sins” —Irenaeus, Against Heresies 21:1-2 Date: 189 A.D.
“And [Naaman] dipped himself . . . seven times in the Jordan’ [2 Kgs. 5:14]. It was not for nothing that Naaman of old, when suffering from leprosy, was purified upon his being baptized, but [this served] as an indication to us. For as we are lepers in sin, we are made clean, by means of the sacred water and the invocation of the Lord, from our old transgressions, being spiritually regenerated as newborn babes, even as the Lord has declared: ‘Except a man be born again through water and the Spirit, he shall not enter into the kingdom of heaven” —Irenaeus, Fragment 34 Date: 190 A.D.
“When we are baptized, we are enlightened. Being enlightened, we are adopted as sons. Adopted as sons, we are made perfect. Made perfect, we become immortal . . . ‘and sons of the Most High’ [Ps. 82:6]. This work is variously called grace, illumination, perfection, and washing. It is a washing by which we are cleansed of sins, a gift of grace by which the punishments due our sins are remitted, an illumination by which we behold that holy light of salvation”—Clement of Alexandria, The Instructor of Children 1:6:26:1 Date: 191 A.D.
“Now faith occasions this for us even as the Elders, the disciples of the Apostles, have handed it down to us. First of all, it bids us to keep in mind that we have received baptism for the remission of sins in the name of God the Father and in the name of Jesus Christ, the Son of God, who was incarnate, died and rose again, and in the Holy Spirit of God. This baptism is the seal of eternal life and the new birth unto God that we should no longer be the sons of mortal men but of the eternal and perpetual God.” — Irenaeus, The Demonstration of the Apostolic Preaching, Date: 192 A.D.
“Happy is our sacrament of water, in that, by washing away the sins of our early blindness, we are set free and admitted into eternal life. . . . But we, little fishes after the example of our [Great] Fish, Jesus Christ, are born in water, nor have we safety in any other way than by permanently abiding in water. So that most monstrous creature, who had no right to teach even sound doctrine, knew full well how to kill the little fishes—by taking them away from the water!” —Tertullian, Baptism 1 Date: 203 A.D.
“Baptism itself is a corporal act by which we are plunged into the water, while its effect is spiritual, in that we are freed from our sins.” —Tertullian, Baptism 7:2 Date: 203 A.D.
“The apostle indeed teaches, in his Epistle to the Colossians, that we were once dead, alienated, and enemies to the Lord in our minds, whilst we were living in wicked works; that we were then buried with Christ in baptism, and also raised again with Him through the faith of the operation of God, who hath raised Him from the dead. “And you, (adds he), when ye were dead in sins and the uncircumcision of your flesh, hath He quickened together with Him, having forgiven you all trespasses.” And again: “If ye are dead with Christ from the elements of the world, why, as though living in the world, are ye subject to ordinances?” —Tertullian, On the Resurrection of the Flesh 23 Date: 211 A.D.
“And the bishop shall lay his hand upon them [the newly baptized], invoking and saying: ‘O Lord God, who did count these worthy of deserving the forgiveness of sins by the laver of regeneration, make them worthy to be filled with your Holy Spirit and send upon them thy grace [in confirmation], that they may serve you according to your will” —Hippolytus, The Apostolic Tradition 22:1 Date: 215 A.D.
“When the one being baptized goes down into the water, the one baptizing him shall put his hand on him and speak thus: ‘Do you believe in God, the Father Almighty?’ And he that is being baptized shall say: ‘I believe.’ Then, having his hand imposed upon the head of the one to be baptized, he shall baptize him once. Then he shall say: ‘Do you believe in Christ Jesus . . . ?’ And when he says: ‘I believe,’ he is baptized again. Again shall he say: ‘Do you believe in the Holy Spirit and the holy Church and the resurrection of the flesh?’ The one being baptized then says: ‘I believe.’ And so he is baptized a third time” —Hippolytus, The Apostolic Tradition 21 Date: 215 A.D.
“After his resurrection he promises in a pledge to his disciples that he will send them the promise of his Father; and lastly, he commands them to baptize into the Father and the Son and the Holy Ghost, not into a unipersonal God. And indeed it is not once only, but three times, that we are immersed into the three persons, at each several mention of their names” —Tertullian, Against Praxeas 26 Date: 216 A.D.
“[P]erhaps someone will ask, ‘What does it conduce unto piety to be baptized?’ In the first place, that you may do what has seemed good to God; in the next place, being born again by water unto God so that you change your first birth, which was from concupiscence, and are able to attain salvation, which would otherwise be impossible. For thus the [prophet] has sworn to us: ‘Amen, I say to you, unless you are born again with living water, into the name of the Father, Son, and Holy Spirit, you shall not enter into the kingdom of heaven.’ Therefore, fly to the water, for this alone can extinguish the fire. He who will not come to the water still carries around with him the spirit of insanity for the sake of which he will not come to the living water for his own salvation”— Hippolytus, Homilies 11:26 Date: 217 A.D.
“Baptism washes away all, absolutely all, our sins, whether of deed, word, or thought, whether sins original or added, whether knowingly or unknowingly contracted” —Augustine, Against Two Letters of the Pelagians 3:3:5 Date: 420 A.D.
“But you will perhaps say, ‘What does the baptism of water contribute toward the worship of God?’ In the first place, because that which has pleased God is fulfilled. In the second place, because when you are regenerated and born again of water and of God, the frailty of your former birth, which you have through men, is cut off, and so . . . you shall be able to attain salvation; but otherwise it is impossible. For thus has the true prophet [Jesus] testified to us with an oath: ‘Verily, I say to you, that unless a man is born again of water . . . he shall not enter into the kingdom of heaven” —The Recognitions of Clement 6:9 Date: 221 A.D.
“That unless a man have been baptized and born again, he cannot attain unto the kingdom of God. In the Gospel according to John: ‘Except a man be born again of water and the Spirit, he cannot enter into the kingdom of God” —Testimonies Concerning the Jews 3:2:25–26 Date: 240 A.D.
“While I was lying in darkness . . . I thought it indeed difficult and hard to believe . . . that divine mercy was promised for my salvation, so that anyone might be born again and quickened unto a new life by the laver of the saving water, he might put off what he had been before, and, although the structure of the body remained, he might change himself in soul and mind. . . . But afterwards, when the stain of my past life had been washed away by means of the water of rebirth, a light from above poured itself upon my chastened and now pure heart; afterwards, through the Spirit which is breathed from heaven, a second birth made of me a new man” — Cyprian of Carthage, To Donatus 3–4 Date: 246 A.D.
“Then Probus . . . leapt into the water, saying, ‘Jesus Christ, Son of God, and everlasting God, let all my sins be taken away by this water.’ And Paul said, ‘We baptize thee in the name of the Father and Son and Holy Ghost.’ After this he made him to receive the Eucharist of Christ” —The Acts of Xantippe and Polyxena, Acts of Xantippe and Polyxena 21 Date: 250 A.D.
“As to what pertains to the case of infants: You [Fidus] said that they ought not to be baptized within the second or third day after their birth, that the old law of circumcision must be taken into consideration, and that you did not think that one should be baptized and sanctified within the eighth day after his birth. In our council it seemed to us far otherwise. No one agreed to the course which you thought should be taken. Rather, we all judge that the mercy and grace of God ought to be denied to no man born…. If, in the case of the worst sinners and those who formerly sinned much against God, when afterwards they believe, the remission of their sins is granted and no one is held back from baptism and grace, how much more, then, should an infant not be held back, who, having but recently been born, has done no sin, except that, born of the flesh according to Adam, he has contracted the contagion of that old death from his first being born. For this very reason does he [an infant] approach more easily to receive the remission of sins: because the sins forgiven him are not his own but those of another”— Cyprian of Carthage, Letters 64:2 & 64:5 Date: 253 A.D.
“He [Jesus] commanded them to baptize the Gentiles in the name of the Father and of the Son and of the Holy Spirit. How then do some say that though a Gentile be baptized . . . never mind how or of whom, so long as it be done in the name of Jesus Christ, the remission of sins can follow—when Christ himself commands the nations to be baptized in the full and united Trinity?”—Cyprian of Carthage, Letters 73:18 Date: 253 A.D.
Peter showed and vindicated the unity of the church by commanding and warning that we can be saved only through the baptism of the one church. Just as in that baptism of the world by which the ancient iniquity was purged, the one who was not in the ark could not be saved through water, so now anyone who has not been baptized in the church cannot be saved, for the church has been founded in the unity of the Lord, as the sacrament of the one ark. —Cyprian of Carthage, Letters 74.11. Date: 253 A.D.
“[When] they receive also the baptism of the Church . . . then finally can they be fully sanctified and be the sons of God . . . since it is written, ‘Except a man be born again of water and of the Spirit, he cannot enter into the kingdom of God” —Cyprian of Carthage, Letters 71:1 Date: 253 A.D.
“And in the gospel our Lord Jesus Christ spoke with his divine voice, saying, ‘Except a man be born again of water and the Spirit, he cannot enter the kingdom of God.’ . . . Unless therefore they receive saving baptism in the Catholic Church, which is one, they cannot be saved, but will be condemned with the carnal in the judgment of the Lord Christ” —Seventh Carthage, Date: 256 A.D.
“If Zipporah has circumcised her son with the temporal circumcision and has averted death, will not death with much more reason be banished by the true baptism? The one baptism into Christ puts on the Living One who vivifies the whole world.”- Nisibene Hymn, Date: 4th Century
“We believe . . . each of these to be and to exist: the Father, truly Father, and the Son, truly Son, and the Holy Ghost, truly Holy Ghost, as also our Lord, sending forth his disciples for the preaching, said, ‘Go teach all nations, baptizing them in the name of the Father and of the Son, and of the Holy Ghost.’ Concerning whom we confidently affirm that so we hold, and so we think, and so we have held aforetime, and we maintain this faith unto the death, anathematizing every godless heresy” —Eusebius of Caesarea, Letter to the People of His Diocese 3 Date: 323 A.D.
“And I believe in one holy catholic and apostolic Church. I acknowledge one baptism for the remission of sins; and I look for the resurrection of the dead, and the life of the world to come.”—Nicene Creed, Council of Nicaea, Date: 325 A.D.
“From baptism we receive the Spirit of Christ. At that same moment in which the priests invoke the Spirit, heaven opens, and he descends and rests upon the waters, and those who are baptized are clothed in him. The Spirit is absent from all those who are born of the flesh, until they come to the water of rebirth, and then they receive the Holy Spirit. . . . [I]n the second birth, that through baptism, they receive the Holy Spirit” —Aphraahat the Persian, Treatises 6:14:4 Date: 340 A.D.
“Since man is of a twofold nature, composed of body and soul, the purification also is twofold: the corporeal for the corporeal and the incorporeal for the incorporeal. The water cleanses the body, and the Spirit seals the soul. . . . When you go down into the water, then, regard not simply the water, but look for salvation through the power of the Spirit. For without both you cannot attain to perfection. It is not I who says this, but the Lord Jesus Christ, who has the power in this matter. And he says, ‘Unless a man be born again,’ and he adds the words ‘of water and of the Spirit,’ ‘he cannot enter the kingdom of God.” —Cyril of Jerusalem, Catechetical Lectures 3:4 Date: 350 A.D.
“[A]s we are all from earth and die in Adam, so being regenerated from above of water and Spirit, in the Christ we are all quickened” —Athanasius, Four Discourses Against the Arians 3:26 Date: 360 A.D.
“And the whole faith is summed up, and secured in this, that a Trinity should ever be preserved, as we read in the Gospel, ‘Go ye and baptize all the nations in the name of the Father and of the Son and of the Holy Ghost’ (Matt. 28:19). And entire and perfect is the number of the Trinity —Athanasius, On the Councils of Arminum and Seleucia 2:28 Date: 361 A.D.
“For prisoners, baptism is ransom, forgiveness of debts, the death of sin, regeneration of the soul, a resplendent garment, an unbreakable seal, a chariot to heaven, a royal protector, a gift of adoption” —Basil the Great, Sermons on Moral and Practical Subjects 13:5 Date: 379 A.D.
“But not yet perhaps is there formed upon your soul any writing good or bad; and you want to be written upon today. . . . I will baptize you and make you a disciple in the name of the Father and of the Son and of the Holy Ghost; and these three have one common name, the Godhead. And you shall know, both by appearances and by words that you reject all ungodliness, and are united to all the Godhead” —Gregory of Nazianz, Orations 40:45 Date: 380 A.D. 380
“Who is the one who is born of the Spirit and is made Spirit? It is one who is renewed in the Spirit of his mind. It is one who is regenerated by water and the Holy Spirit. We receive the hope of eternal life through the laver of regeneration and renewing of the Holy Spirit. And elsewhere the apostle Peter says: “You shall be baptized with the Holy Spirit.” For who is he that is baptized with the Holy Spirit but he who is born again through water and the Holy Spirit? Therefore the Lord said of the Holy Spirit, “Truly, truly, I say to you, except a man be born again by water and the Spirit, he cannot enter into the kingdom of God.” And therefore he declared that we are born of him into the kingdom of God by being born again by water and the Spirit.”—Ambrose of Milan, Of the Holy Spirit 3.10.64 Date: 381 A.D.
“[S]eeing that a man, baptized in the name of the Father and the Son and the Holy Ghost, becomes a temple of the Lord, and that while the old abode is destroyed a new shrine is built for the Trinity, how can you say that sins can be remitted among the Arians without the coming of the Holy Ghost? How is a soul purged from its former stains which has not the Holy Ghost?” —Jerome, Dialogue Against the Luciferians 6 Date: 382 A.D.
“There came out from his side water and blood.” Beloved, do not pass this mystery by without a thought. For I have still another mystical explanation to give. I said that there was a symbol of baptism and the mysteries in that blood and water. It is from both of these that the church is sprung “through the bath of regeneration and renewal by the Holy Spirit,” through baptism and the mysteries. But the symbols of baptism and the mysteries, water and blood, come from the side of Christ. It is from his side, therefore, that Christ formed his church, just as he formed Eve from the side of Adam. —John Chrysostom, Baptismal Instructions 3.17 Date: 387 A.D.
“The Church was redeemed at the price of Christ’s blood. Jew or Greek, it makes no difference; but if he has believed, he must circumcise himself from his sins [in baptism (Col. 2:11–12)] so that he can be saved . . . for no one ascends into the kingdom of heaven except through the sacrament of baptism. . .‘Unless a man be born again of water and the Holy Spirit, he cannot enter the kingdom of God” —Ambrose of Milan, Abraham 2:11:79–84 Date: 387 A.D.
“This, too, is plain, that in him who is baptized the Son of God is crucified. Indeed, our flesh could not eliminate sin unless it were crucified in Jesus Christ.… And to the Colossians he says, “Buried with him by baptism, wherein you also rose again with him.” This was written with the intent that we should believe that he is crucified in us, that our sins may be purged through him, that he, who alone can forgive sins, may nail to his cross the handwriting which was against us.”—Ambrose of Milan, Concerning Repentance 2.2.9 Date: 388 A.D.
“Do you have an infant child? Allow sin no opportunity; rather, let the infant be sanctified from childhood. From his most tender age let him be consecrated by the Spirit. Do you fear the seal [of baptism] because of the weakness of nature? Oh, what a pusillanimous mother and of how little faith!”…“Well enough,’ some will say, ‘for those who ask for baptism, but what do you have to say about those who are still children, and aware neither of loss nor of grace? Shall we baptize them too?’ Certainly [I respond], if there is any pressing danger. Better that they be sanctified unaware, than that they depart unsealed and uninitiated” —Gregory of Nazianz, Oration on Holy Baptism 40:7 & 40:28 Date: 388 A.D.
“Such is the grace and power of baptism; not an overwhelming of the world as of old, but a purification of the sins of each individual, and a complete cleansing from all the bruises and stains of sin. And since we are double-made, I mean of body and soul, and the one part is visible, the other invisible, so the cleansing also is twofold, by water and the Spirit; the one received visibly in the body, the other concurring with it invisibly and apart from the body; the one typical, the other real and cleansing the depths”—Gregory of Nazianz, Oration on Holy Baptism 7–8 Date: 388 A.D.
“You see how many are the benefits of baptism, and some think its heavenly grace consists only in the remission of sins, but we have enumerated ten honors [it bestows]! For this reason we baptize even infants, though they are not defiled by [personal] sins, so that there may be given to them holiness, righteousness, adoption, inheritance, brotherhood with Christ, and that they may be his [Christ’s] members”—John Chrysostom, Baptismal Catechesis in Augustine, Against Julian 1:6:21 Date: 388 A.D.
“You have read, therefore, that the three witnesses in baptism are one: water, blood, and the Spirit (1 John 5:8): And if you withdraw any one of these, the sacrament of baptism is not valid. For what is the water without the cross of Christ? A common element with no sacramental effect. Nor on the other hand is there any mystery of regeneration without water, for ‘unless a man be born again of water and the Spirit, he cannot enter the kingdom of God” —Ambrose of Milan, The Mysteries 4:20 Date: A.D. 390
“Circumcision is no longer performed with a knife, Paul says, but in Christ himself; for no human hand circumcises … but the Spirit. The Spirit circumcises the whole man, not simply a part.… When and where? In baptism. And what Paul calls circumcision, he again calls burial.… But it is not burial only: for notice what he says, “Wherein you were also raised with him, through faith in the working of God, who raised him from the dead.” —John Chrysostom, Homilies on Colossians 6 Date: 395 A.D.
“Circumcision” refers to the life of immortality embraced through baptism, just as “uncircumcision” is the old life of mortality. Commentary on Colossians.” —Theodore of Mopsuestia, TEM 1:287, Date: Early 5th Century
“Through baptism comes the stripping away and circumcision of sins.… Those being baptized in the blood of Christ confess that they share in his death through baptism and that following this they enjoy the resurrection. Resurrection is used here in a twofold sense, the one spiritual and the other physical. All persons will rise through the resurrection of Christ from the dead. Those, however, who have not been baptized in Christ but have died without faith will share in the general resurrection. However, they will not enjoy the promise of redemption.… As many as were baptized into Christ, these have freely benefitted before the general resurrection from the spiritual resurrection, for they have already risen from the death of sins. Thus, Paul also says: “in whom you were raised,” not “in whom you will be raised.” —Severian of Gabala, Pauline Commentary, Date: Early 5th Century
“Baptism in the name of the Father and of the Son and of the Holy Ghost has Christ for its authority, not any man, whoever he may be; and Christ is the truth, not any man” —Augustine, On Baptism, Against the Donatists 4:24  Date: 400 A.D.
“What the universal Church holds, not as instituted [invented] by councils but as something always held, is most correctly believed to have been handed down by apostolic authority. Since others respond for children, so that the celebration of the sacrament may be complete for them, it is certainly availing to them for their consecration, because they themselves are not able to respond” — Augustine, On Baptism, Against the Donatists 4:24:31 Date: 400 A.D.
“Be ye likewise contented with one baptism alone, that which is into the death of the Lord [Rom. 6:3; Col. 2:12–13]. . . . [H]e that out of contempt will not be baptized shall be condemned as an unbeliever and shall be reproached as ungrateful and foolish. For the Lord says, ‘Except a man be baptized of water and of the Spirit, he shall by no means enter into the kingdom of heaven.” —Apostolic Constitutions 6:3:15 Date: 400 A.D.
“The custom of Mother Church in baptizing infants is certainly not to be scorned, nor is it to be regarded in any way as superfluous, nor is it to be believed that its tradition is anything except apostolic” —Augustine, The Literal Interpretation of Genesis 10:23:39 Date 408 A.D.
“O Lord our God, we believe in you, the Father and the Son and the Holy Spirit. For the truth would not say, ‘Go, baptize all nations in the name of the Father and of the Son and of the Holy Spirit,’ unless you were a Trinity” —Augustine, The Trinity 15:28 Date: 408 A.D.
“Cyprian was not issuing a new decree but was keeping to the most solid belief of the Church in order to correct some who thought that infants ought not be baptized before the eighth day after their birth. . . . He agreed with certain of his fellow bishops that a child is able to be duly baptized as soon as he is born” —Augustine, Letters 166:8:23 Date: 412 A.D.
“If anyone wonders why children born of the baptized should themselves be baptized, let him attend briefly to this. . . . The sacrament of baptism is most assuredly the sacrament of regeneration”—Augustine, Forgiveness and the Just Deserts of Sin, and the Baptism of Infants 2:27:43 Date: 412 A.D.
“It is this one Spirit who makes it possible for an infant to be regenerated . . . when that infant is brought to baptism; and it is through this one Spirit that the infant so presented is reborn. For it is not written, ‘Unless a man be born again by the will of his parents’ or ‘by the faith of those presenting him or ministering to him,’ but, ‘Unless a man be born again of water and the Holy Spirit.’ The water, therefore, manifesting exteriorly the sacrament of grace, and the Spirit effecting interiorly the benefit of grace, both regenerate in one Christ that man who was generated in Adam” —Augustine, Letters 98:2 Date: 412 A.D.
“[W]hoever says that infants fresh from their mothers’ wombs ought not to be baptized, or say that they are indeed baptized unto the remission of sins, but that they draw nothing of the original sin of Adam, which is expiated in the bath of regeneration . . . let him be anathema [excommunicated]. Since what the apostle [Paul] says, ‘Through one man sin entered into the world, and death through sin, and so passed to all men, in whom all have sinned’ [Rom. 5:12], must not be understood otherwise than the Catholic Church spread everywhere has always understood it. For on account of this rule of faith even infants, who in themselves thus far have not been able to commit any sin, are therefore truly baptized unto the remission of sins, so that that which they have contracted from generation may be cleansed in them by regeneration” —Council of Mieum II, Canon 3 Date: 416 A.D.
“If any man says that new-born children need not be baptized, or that they should indeed be baptized for the remission of sins, but that they have in them no original sin inherited from Adam which must be washed away in the bath of regeneration, so that in their ease the formula of baptism ‘for the remission of sins’ must not be taken literally, but figuratively, let him be anathema; because, according to Romans 5:12, the sin of Adam has passed upon all.”—Council of Carthage to Investigate Pelagianism, Canon 2 Date: 416 A.D.
“If the forgiveness of sins were not to be had in the church, there would be no hope of a future life and eternal liberation. We thank God, who gave his church such a gift. Here you are; you are going to come to the holy font, you will be washed in saving baptism, you will be renewed in “the bath of rebirth.” You will be without any sin at all as you come up from that bath. All the things that were plaguing you in the past will be blotted out.” —Augustine, Sermons 213.8 Date: 417 A.D.
“If any man says that new-born children need not be baptized, or that they should indeed be baptized for the remission of sins, but that they have in them no original sin inherited from Adam which must be washed away in the bath of regeneration, so that in their ease the formula of baptism ‘for the remission of sins’ must not be taken literally, but figuratively, let him be anathema; because, according to Romans 5:12, the sin of Adam has passed upon all.”—Council of Carthage, Canon 2 Date: 418 A.D.
“Those who, though they have not received the washing of regeneration, die for the confession of Christ—it avails them just as much for the forgiveness of their sins as if they had been washed in the sacred font of baptism. For he that said, ‘If anyone is not reborn of water and the Spirit, he will not enter the kingdom of heaven,’ made an exception for them in that other statement in which he says no less generally, ‘Whoever confesses me before men, I too will confess him before my Father, who is in heaven —Augustine, The City of God 13:7 Date: 419 A.D.
“This is the meaning of the great sacrament of baptism, which is celebrated among us: all who attain to this grace die thereby to sin—as he himself [Jesus] is said to have died to sin because he died in the flesh (that is, ‘in the likeness of sin’)—and they are thereby alive by being reborn in the baptismal font, just as he rose again from the sepulcher. This is the case no matter what the age of the body. For whether it be a newborn infant or a decrepit old man—since no one should be barred from baptism—just so, there is no one who does not die to sin in baptism. Infants die to original sin only; adults, to all those sins which they have added, through their evil living, to the burden they brought with them at birth” —Augustine, Handbook on Faith, Hope, and Love 13 Date: 421 A.D.
I did try to find Patristic evidence for the belief that baptism doesn’t do anything but has to be done by immersion only as a sign of making a decision for Jesus. I wasn’t able to, from what I can see this is a teaching that didn’t appear in Christianity until about 500 years ago.
This of course doesn’t do the work all on it’s own, one has to appeal to scripture as the ultimate authority. I would say however, that if one believes that all of Christianity was wrong on this all the way back to the first century then they must also not believe that the Bible is sufficient enough to speak clearly. It shouldn’t take 1500 years for anyone to understand a text that speaks clearly.