Banana tree, whoever you are, I'm 99% certain we are actually in the same class.
I would guess that φύσει should be read together with ἀσθενέστατοι, so "weak by nature", "naturally weak".
As for τῷ σώματι, I wonder... The commentaries in our book wants to group it with ἀσθενέστατοι, but couldn't it just as well belong to μελετήσαντες, so that the translation would be "...after having trained (with regards to) the body..."? At least, unless I'm mistaken, that is how it is used in the following sentence: "...τῷ σώματι ἀεὶ τὰ συντυγχάνοντα μελετῶντα καρτερεῖν...", which I read as "...always training 'with regards to the body' to endure that which happen...". Or maybe τῷ σώματι here should be taken together with καρτερεῖν? Or even συντυγχάνοντα?!