The answer to your question is simple on the surface, but this is actually a very complex question.
What they teach you in the textbooks and class is that the infinitive does NOT equate to a purpose clause in Latin. This is because of a false analogy with English. We say "I want to do this and we are using an infinitive to state our purpose. However, the Latin infinitive does not work like this. The analogy is false, because you must remember that we say “amare” means “to love,” but it’s not the same construction. The English word “to” has so many of its own English uses that we cannot map it onto the Latin infinitive.
Therefore, when you’re learning how to form purpose clauses, if you ever use the infinitive you’re sure to get marked off for it. As you pointed out, the big three forms they make you learn are the gerund/ive, ut/ne+subjunctive, and the supine. The supine itself is fairly minor and uncommon, so you can really get through introductory Latin without ever learning it (this is what happened to me!).
Now for the complicated stuff: in actual usage, there are a variety of times when the infinitive can be used to express purpose. This isn’t an error; even the best Roman authors used it. With that said, you can’t use it whenever you want. Some verbs allow it as an option and some don’t. I just read up on the different usages in my Allen and Greenough grammar, and I’m not sure I understand them all yet, but here are some basics:
The infinitive is used in isolated passages instead of a subjuctive clause after habeo, do, ministro. “tantum habeo polliceri,” “so much I have to promise.” [The more formal construction would be “quod pollicear.”]
Paratus, suetus, and their ocmpounds, and some other participles take the infinitive like the verbs from which they come: “id quod parati sunt facere,” “that which they are ready to do.” This is apparently more common in poetry than in prose.
Poets and early writers often use the infinitive when there is no analogy with any prose construction [I don’t understand what A&G means by this]: “filius intro iit videre quid agat,” “your son has gone in to see what he is doing.” In prose, you’d get the supine “visum” instead of “videre” to accompany the verb of movement “iit.” These usages become rarer in classic period prose writers.
Many verbs take eitehr a subjunctive clause or a complementary infinitive, without difference of meaning. “decernere optabat” vs “optavit ut tolleretur;” “oppugnare contendit” vs “contendit ut caperet”
In conclusion, I wouldn’t worry about learning all these usages. I sure haven’t. Figure them out when you encounter them while reading. If you’re writing in Latin, stick with the “big three.”
It isn’t an option for any of those types of sentences, unless you’re a poet and use it sparingly.
Also, Venit ad legendum libros, although grammatically possible, is not the preferred construction for classical prose (Allen & Greenough calls it “inadmissible”). The gerund of a transitive verb after ad is not supposed to have a direct object. If you need to express an object of the gerund you use the gerundive instead, as in Venit ad libros legendos. The gerund is only used after ad when the verb is intransitive, or you don’t wish to express a direct object. You’d use the gerund, for example, with verbs that take the dative or genitive or ablative instead of the accusative, as in Conspiraverunt ad regi nocendum “They conspired to harm the king”.
The genitive of a gerund with a direct object may be used instead of the genitive gerundive, however. This is often done for stylistic purposes in order to avoid the cumbersome repetition of -orum and -arum endings, e.g. armorum capiendorum cupidus est “he was eager to take up arms” may be shortened to arma capiendi cupidus est. Likewise to distinguish neuter pronouns or substantive adjectives from their masculine counterparts, i.e. artem vera ac falsa diiudicandi “the art of distinguishing true and false [things]” vs. the ambiguous artem verorum ac falsorum diiudicandorum “art of distinguishing true and false men/things”.
