I would definitely only recommend the first two books, Red Dragon and Silence of the Lambs. I read Hannibal and didn't hate it, and actually kind of appreciated how crazy the author was to just go for it, but yeah. You can for sure skip that and everything afterwards.