A few years ago I started Seneca by just reading a bunch of his Epistulae Morales, more or less at random. As Qimmik points out, they are short and snappy (a difference of Silver/Golden age Latin) and low commitment.
De Brevitate Vitae which is also pretty accessible. I don't recall having any particular problems with Seneca, other than a handful of words I hadn't seen before that aren't common in Caesar/Cicero etc. Seneca tends to be down to earth in his language, so you may get references to everyday things in Roman life that you hadn't seen elsewhere.
Horae quidem cedunt et dies et menses et anni, nec praeteritum tempus umquam revertitur nec quid sequatur sciri potest. Quod cuique temporis ad vivendum datur, eo debet esse contentus. --Cicero, De Senectute