I think I'm kind of in the same boat as you. I too would say I am at an intermediate level and find that my biggest barrier is vocabulary. I think some people acquire vocabulary easily, I find that I had an easy time with the grammar, but have a harder time growing my vocabulary (probably mostly because there is just way more vocabulary to be learned than there are paradigms).
The problem of a limited vocabulary is a bit of a catch-22. It is said that the best way to increase your vocabulary is to read a lot and read widely. But if your vocabulary is limited, then you are not able to read widely. If it takes me an hour to read a page of text, then I can't easily read a very large quantity of text. And if I have to look up several words in a sentence, then I don't really get an intuitive, contextual feel for the use of the word.
So I feel that, at my stage, I have to augment my reading with a lot of flash-card drills. One thing that has been useful for me is the dissertation by Paul Diederich (at http://users.erols.com/whitaker/freq.htm
). It has some interesting information about Latin vocabulary and it has a "Recommended Basic Vocabulary" and list of the 4000-some most common words, listed along with their frequency of occurrence.
One thing I have been doing is to maintain a spreadsheet of "words I don't know." Every time, in my reading, that I come across a word I have to look up, I add it to the list in my spreadsheet, if it isn't in there already. Then I enter the frequency of occurrence from Diederich's list (if it's on that list). If its a word that I encounter multiple times, then I increase the frequency number in my spreadsheet. Then every day I spend time doing flashcard drills for (and reviewing for a week or two) the most frequent words in my spreadsheet. I try to add about 6 new words to my vocabulary from the spreadsheet every day (and review previous flashcards a lot). That ends up being about 30-45 minutes per day of doing vocabulary flashcards. It would be less if I did fewer than 6 new words per day.