Japanese is my daily-use second language because I live in Japan and need to engage in most normal adult situations in Japanese. I can easily go my entire day without using English, and that includes discussions of the humanities (usually politics, but also art, history, philosophy, etc). Hard science is pretty much beyond my ability, and my reading and writing is way behind my verbal. I can only read and write at about a 4th grade level. I'm naturally good at languages, but kanji is a very real barrier to my reading and writing, and I eventually will need to have formal lessons. I only know about 400 without the use of a computer/smart phone. I can recognise many more, so with a computer/smart phone, I can write things which, ironically, I might not even be able to read (except, of course, I happen to know what I'm saying). I have dreamt in Japanese. I have lived in Japan for nearly five years.
I still struggle with most modern anime series. I've gotten to the point where I can watch almost any children's programming, and several anime which have younger demographics. Just this year I have reached the point where I can watch the original Sailor Moon quite easily. The older the anime, the more likely I am to be able to follow along. However, most recent series throw around so many technical terms (even slice of life!) that I often get lost in the jargon, and once I get lost, it's hard to "reset" and catch up. And yonkoma often rely on very complex understandings of idiom and pun. A show like Nichijou is extremely difficult for me to follow without subtitles. The yonkoma manga itself is incomprehensible to me. Even if I think I know what I'm reading in a dictionary sense, I don't yet have the years of experience needed to understand the multiple meanings beyond the dictionary meanings, and that is where the humor is found.
You cannot, cannot, CANNOT learn Japanese from watching anime. Hell, as I tell my students (after all, I teach a foreign language- English to my junior high schoolers here in Japan) that if they are really serious, they have to up and move to an English speaking country, at least for a while. Japanese is the same way. Want to learn Japanese? Move here. Otherwise it's going to take a long, long, long time and you will miss out on all the common, everyday slangy expressions which you can never find in a textbook.